Spaces:
Running
Running
Create README_stage4.md
Browse files- README_stage4.md +61 -0
README_stage4.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Stage Four — ViT-Tiny (ImageNet Subset) Validation
|
| 2 |
+
|
| 3 |
+
**Rendered Frame Theory (RFT)**
|
| 4 |
+
Author: Liam S. Grinstead
|
| 5 |
+
Date: Oct‑2025
|
| 6 |
+
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
## Abstract
|
| 10 |
+
Stage Four evaluates RFT on a transformer vision model (ViT‑Tiny), extending beyond CNN baselines. Using ImageNet‑1K subsets (or synthetic stand‑ins), we compare RFT (DCLR + Ψ–Ω coupling) against Adam. Results show lower energy per step and tighter drift/flux stability at matched accuracy, demonstrating that coherence‑aware optimisation benefits attention layers as well as convolutions.
|
| 11 |
+
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
## Objective
|
| 15 |
+
Verify that RFT’s efficiency and coherence gains persist in transformer architectures by benchmarking ViT‑Tiny on an ImageNet‑like workload with identical hyperparameters across modes and unified telemetry.
|
| 16 |
+
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
## Methodology
|
| 20 |
+
- **Model:** ViT‑Tiny (patch 16, dim 192, depth 12, heads 3) with CLS token
|
| 21 |
+
- **Data:** ImageNet‑subset via ImageFolder (train/val), or synthetic images/labels if no `--data_dir` supplied
|
| 22 |
+
- **Setup:** PyTorch 2.x, Python 3.10, A100/H100 (BF16 autocast if available), seed 1234
|
| 23 |
+
- **Metrics:** Loss, accuracy, J/step (NVML if present; proxy otherwise), drift, flux, energy‑retention (E_ret), coherence (coh), ΔT
|
| 24 |
+
- **Parity:** Same batch, learning rate, and steps across RFT and BASE
|
| 25 |
+
- **Orbital Coupler:** Ψ–Ω drift/flux synchronisation each iteration
|
| 26 |
+
- **Optimisers:** DCLR (RFT) vs Adam (BASE)
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
|
| 30 |
+
## Results
|
| 31 |
+
- **RFT (DCLR + Ψ–Ω):** Lower energy‑per‑step at comparable loss/accuracy; drift remains tightly bounded, flux stable; coherence near unity throughout the run.
|
| 32 |
+
- **Baseline (Adam):** Higher J/step and looser drift/flux behaviour at matched settings.
|
| 33 |
+
- **Synthetic data fallback:** Reproduces the same qualitative efficiency pattern, indicating optimiser–telemetry interaction is the source of gains, not dataset artefacts.
|
| 34 |
+
|
| 35 |
+
---
|
| 36 |
+
|
| 37 |
+
## Discussion
|
| 38 |
+
ViT‑Tiny confirms that RFT’s coherence mechanisms generalise to attention‑based architectures. The reduction in J/step with stable drift/flux mirrors Stage One (CNN behaviour) and Stage Two (multi‑agent dynamical stability), indicating architecture‑agnostic benefits driven by coherence‑aware optimisation.
|
| 39 |
+
|
| 40 |
+
---
|
| 41 |
+
|
| 42 |
+
## Conclusion
|
| 43 |
+
RFT scales cleanly to transformers: energy efficiency improves without compromising learning signals, and coherence remains robust across iterations. This stage prepares for deeper vision models (ViT‑Small, ViT‑Base) and multi‑modal settings.
|
| 44 |
+
|
| 45 |
+
---
|
| 46 |
+
|
| 47 |
+
## Reproducibility
|
| 48 |
+
- **Script:** `stage4.py`
|
| 49 |
+
- **Log Output:** `stage4_vit_tiny.jsonl`
|
| 50 |
+
- **Seed:** 1234
|
| 51 |
+
- **Hardware:** A100/H100 (CPU fallback supported)
|
| 52 |
+
- **Sealing:** All runs are sealed with SHA‑512 hashes
|
| 53 |
+
|
| 54 |
+
---
|
| 55 |
+
|
| 56 |
+
## Usage
|
| 57 |
+
- **RFT mode:**
|
| 58 |
+
`python stage4.py --mode RFT --steps 1000 --batch 256 --lr 5e-4 --data_dir /path/to/imagenet_subset`
|
| 59 |
+
- **BASE mode (Adam):**
|
| 60 |
+
`python stage4.py --mode BASE --steps 1000 --batch 256 --lr 5e-4 --data_dir /path/to/imagenet_subset`
|
| 61 |
+
- **Synthetic fallback:** Omit `--data_dir` to auto‑generate data.
|