Spaces:
Sleeping
Sleeping
File size: 3,048 Bytes
c67a455 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | # Stage Four — ViT-Tiny (ImageNet Subset) Validation
**Rendered Frame Theory (RFT)**
Author: Liam S. Grinstead
Date: Oct‑2025
---
## Abstract
Stage Four evaluates RFT on a transformer vision model (ViT‑Tiny), extending beyond CNN baselines. Using ImageNet‑1K subsets (or synthetic stand‑ins), we compare RFT (DCLR + Ψ–Ω coupling) against Adam. Results show lower energy per step and tighter drift/flux stability at matched accuracy, demonstrating that coherence‑aware optimisation benefits attention layers as well as convolutions.
---
## Objective
Verify that RFT’s efficiency and coherence gains persist in transformer architectures by benchmarking ViT‑Tiny on an ImageNet‑like workload with identical hyperparameters across modes and unified telemetry.
---
## Methodology
- **Model:** ViT‑Tiny (patch 16, dim 192, depth 12, heads 3) with CLS token
- **Data:** ImageNet‑subset via ImageFolder (train/val), or synthetic images/labels if no `--data_dir` supplied
- **Setup:** PyTorch 2.x, Python 3.10, A100/H100 (BF16 autocast if available), seed 1234
- **Metrics:** Loss, accuracy, J/step (NVML if present; proxy otherwise), drift, flux, energy‑retention (E_ret), coherence (coh), ΔT
- **Parity:** Same batch, learning rate, and steps across RFT and BASE
- **Orbital Coupler:** Ψ–Ω drift/flux synchronisation each iteration
- **Optimisers:** DCLR (RFT) vs Adam (BASE)
---
## Results
- **RFT (DCLR + Ψ–Ω):** Lower energy‑per‑step at comparable loss/accuracy; drift remains tightly bounded, flux stable; coherence near unity throughout the run.
- **Baseline (Adam):** Higher J/step and looser drift/flux behaviour at matched settings.
- **Synthetic data fallback:** Reproduces the same qualitative efficiency pattern, indicating optimiser–telemetry interaction is the source of gains, not dataset artefacts.
---
## Discussion
ViT‑Tiny confirms that RFT’s coherence mechanisms generalise to attention‑based architectures. The reduction in J/step with stable drift/flux mirrors Stage One (CNN behaviour) and Stage Two (multi‑agent dynamical stability), indicating architecture‑agnostic benefits driven by coherence‑aware optimisation.
---
## Conclusion
RFT scales cleanly to transformers: energy efficiency improves without compromising learning signals, and coherence remains robust across iterations. This stage prepares for deeper vision models (ViT‑Small, ViT‑Base) and multi‑modal settings.
---
## Reproducibility
- **Script:** `stage4.py`
- **Log Output:** `stage4_vit_tiny.jsonl`
- **Seed:** 1234
- **Hardware:** A100/H100 (CPU fallback supported)
- **Sealing:** All runs are sealed with SHA‑512 hashes
---
## Usage
- **RFT mode:**
`python stage4.py --mode RFT --steps 1000 --batch 256 --lr 5e-4 --data_dir /path/to/imagenet_subset`
- **BASE mode (Adam):**
`python stage4.py --mode BASE --steps 1000 --batch 256 --lr 5e-4 --data_dir /path/to/imagenet_subset`
- **Synthetic fallback:** Omit `--data_dir` to auto‑generate data.
|