docs: add EXPERIMENTS.md to log SEU validation failure

Author: TasfinMahmud

EXPERIMENTS.md

@@ -0,0 +1,90 @@
+# Experiments
+A running log of every validation run, benchmark, and cross-domain test for CNSD.
+
+**Discipline for this file** (so it stays a record, not a trophy case):
+- Every entry is tied to a **commit** and a **fixed seed**. The commit's git timestamp is the authoritative date — no manually entered dates.
+- The **run record** (command, commit, environment, config, sample counts) is **auto-generated by the run script** and pasted in verbatim — not transcribed by hand.
+- The experiment and its purpose are stated *before* the result.
+- Null, weak, and unflattering results (abstention rates, accuracy drops, limitations) are recorded alongside the headline numbers.
+- Each entry carries a status: `planned` · `running` · `preliminary` · `validated` · `superseded`.
+
+**How to reproduce any entry**: check out the commit in its run record, prepare the dataset as described in `data/README` (layout + expected sample count / checksum), install the pinned environment (`requirements.txt`), and run the exact command shown in the run record. Numbers should match within run-to-run noise (seeds are fixed; minor GPU nondeterminism is expected).
+
+---
+
+## Index
+
+| # | Experiment | Domain | Status |
+|---|------------|--------|--------|
+| 1 | CWRU baseline (Protocol B) | Bearing (CWRU) | preliminary |
+| 2 | Threshold sweep | Bearing (CWRU) | preliminary |
+| 3 | Cross-condition robustness (AWGN) | Bearing (CWRU) | preliminary |
+| 4 | Multi-seed headline | Bearing (CWRU) | planned |
+| 5 | Cross-domain: SEU gearbox | Gear (SEU) | preliminary (failed) |
+
+---
+
+## 1. CWRU baseline — Protocol B (cross-load)
+
+* **Status:** preliminary
+* **Purpose:** confirm the full five-layer pipeline runs end-to-end on real CWRU and establish the baseline diagnosis result.
+* **Setup:** train loads 0–2, test load 3. All 10 classes. 12 kHz, window 1024.
+
+**Run record**
+```text
+commit:   cd7771ab3668caf9b33109c3a0a9d89f24fd111c
+command:  python validate_run.py --seed 42
+data:     5806 train / 2019 test samples
+```
+
+**Layer-2 physics verification rate**
+| Verdict | Rate |
+|---------|------|
+| CONFIRMED | TBD |
+| CONFLICT | TBD |
+| INCONCLUSIVE | TBD |
+
+**Headline — CNN accuracy by physics verdict**
+| Verdict | n | CNN accuracy |
+|---------|---|--------------|
+| CONFIRMED | TBD | TBD |
+| CONFLICT | TBD | TBD |
+| INCONCLUSIVE | TBD | TBD |
+| **Gap (CONFIRMED - CONFLICT)** | | **TBD** |
+
+* **Causal (Layer 3)** — `do(Z)`: rung *TBD*, max_contrast *TBD*, p *TBD*
+* **Counterfactual (Layer 3B)**: DoWhy available *TBD*; method *TBD*
+* **Notes / limitations**: record the INCONCLUSIVE rate and any seed drift.
+
+---
+
+## 5. Cross-domain — SEU gearbox (GearProvider)
+
+* **Status:** preliminary (failed validation)
+* **Purpose:** demonstrate the framework is genuinely domain-agnostic — same engine, different machine class, only the provider changes.
+* **Setup:** full pipeline on SEU gearset using `GearProvider` (gear-mesh physics). `N_TEETH_INPUT` confirmed against rig spec; fixed channel chosen up front; threshold tuned on a held-out split.
+
+**Run record**
+```text
+commit:   <PENDING PR 12 MERGE>
+command:  python validate_seu.py
+data:     5115 train / 5115 test samples
+n_teeth_input: 20 | channel: 2 (planetary x-axis)
+```
+
+**Layer-2 physics verification rate**
+| Verdict | Rate |
+|---------|------|
+| CONFIRMED | 0.2% |
+| CONFLICT | 0.3% |
+| INCONCLUSIVE | 99.5% |
+
+**Headline — CNN accuracy by physics verdict**
+| Verdict | n | CNN accuracy |
+|---------|---|--------------|
+| CONFIRMED | 9 | 0.333 |
+| CONFLICT | 17 | 0.824 |
+| INCONCLUSIVE | 5089 | 0.514 |
+| **Gap (CONFIRMED - CONFLICT)** | | **-0.491 (FAILED)** |
+
+* **Known caveats to report honestly:** The accuracy gap is currently backwards and practically noise due to a 99.5% inconclusive rate. This is pending a strict `tau` threshold calibration sweep for gear physics, as well as confirming that GMF strength aligns with the same numerical scale as bearing physics.