The Dirty Man

MoE routes on input pixels (content routing). The Dirty Man routes on eye-detected features (meta routing). Three architectural differences:

Routing stability test (Theorem 7 made concrete): We measure the total-variation distance between clean routing and noised routing at 9 noise levels. MoE content routing shifts TV = 0.081 on average — routing degrades under noise. Dirty Man meta routing achieves TV = 0.000 — perfectly stable. MoE is 81× less stable than the Switchyard under distribution shift.

For robotics (SARCOS inverse dynamics), MoE switches between feed-forward networks of the same type — a change in parameters. The Switchyard switches between computation families: linear lens for slow configurations (gravity dominates), nonlinear lens for fast ones (Coriolis terms matter). This is why the routed program reaches 0.0185 NMSE vs 0.0198 for the best fixed path.

For sim2real transfer, structure adaptation (4,659 router parameters) matches weight adaptation (16,330 parameters) at 0.475 vs 0.395 real accuracy with 3.5x fewer parameters. The key insight: the toolbox bank already knows real corruption — only the routing needs rewiring.

Every result above asks routing to be a bit better than the best fixed network. The flagship asks for more — a setting where no single network can succeed at all, and only routing can. A pendulum obeys three different physical laws in three regimes: conservative (energy is conserved), damped (energy decays), and driven (energy is pumped). Each law demands a mutually-exclusive inductive bias: a map that conserves energy cannot simultaneously dissipate it.

This is inverted design: instead of one model trying to learn every law by brute force, embed each known law as an exact, closed-form physics harness, and learn only what is not known — here, which law is governing. The router's eye watches a 16-step trajectory, detects the energy trend (flat vs. decaying vs. pumped), and routes to the right expert. It identifies the governing law at 92–99% accuracy (full scale, T4 GPU). Every fixed expert is exact on its own regime (0.000) and wrong elsewhere (3.5–8.4); a single brute-force MLP fails on every regime (6.6–12.4); the routed system is 5× better even on its worst regime, with up to 336× lower energy error.

A fair objection to the flagship: you hardcoded the winning laws, of course routing wins. The discovered-law variant closes that gap. Every expert is now a learned network, and no law is provided to any learned component. The inverted-design principle moves inside each expert: the known physics (the pendulum's (g/L) sin θ torque and the integrator) is the unchangeable skeleton, and what is learned is the residual force law of the regime — a conservative specialist learns ~0, a damped one learns ~−bω, a driven one learns ~A sin(Ωt). Learning the residual on an exact skeleton instead of raw next-state dynamics is what keeps rollouts stable.

Each learned specialist is near-exact on its own regime; the static MLP still fails on every regime; and the routed system is 5–720× better, tracking the oracle specialist. The router still detects the governing law at 93–99% — nothing was hardcoded. And as the number of laws grows, a single map is stuck at its error floor while routing stays an order of magnitude lower (4.5–9× better at every law count): each law gets its own specialist instead of one compromise.

The flagship routes between whole networks — a useful escape from a single map, but a desk editor files it under mixture-of-experts. The sharper claim is routing inside the network: at every depth, a router picks the primitive layer that processes this sample's current representation. The architecture is no longer a fixed stack of layers; it is a per-sample program of computation — each input walks its own path, and different inputs take different paths. That is the difference between "choose the right network" (MoE) and "the network itself is non-static."

Where does that help, and where does it not? On SARCOS — real telemetry from a 7-DOF robot arm (44,484 training records → 7 joint torques) — the answer is clean. Near-static configurations are nearly linear in pose (gravity compensation dominates); at speed, Coriolis and centrifugal terms make the mapping strongly nonlinear. The routed program learns exactly that split: it routes slow configurations to the linear lens (‖q̇‖ = 1.62) and fast ones to the nonlinear lens (‖q̇‖ = 2.56), and by doing so beats the best of nine fixed paths through the same op bank (0.0185 vs 0.0198 normalized MSE, −6.4%) with no physics hardcoded. This is the "identify the feature, pick the lens" thesis on real physical data.

On SVHN (real street-view digit photos) the same per-depth routing also runs and discovers per-class programs (every class picks a 5×5 convolution at depth 1; depths 2–3 differ per class) — but the task is structurally homogeneous, so routing does not beat a well-chosen static path (router 0.767 vs best fixed conv5 0.805), and a coarse whole-network MoE switch collapses (0.654). The contrast is the point: routing pays exactly when the input is genuinely heterogeneous — where a single fixed network must compromise — and it is a pure cost where it is not. We report both sides of that boundary.

Every deployed network today is a compromise: one topology, chosen before training, asked to serve every input it will ever meet. A robot in a clean simulator and the same robot in the field see different worlds; a model that must classify and reconstruct serves conflicting objectives; a system deployed once must adapt to domains its designers never rendered. The Switch Operator is built the other way around: the architecture itself is a decision, made per sample, from what it sees and what it wants. The measured capabilities below are the directions this opens — each card pairs a measured result with the world it points at.

These are the directions the architecture opens — not claims it has met. Every number on this page is measured; every vision above it is a measured capability pointed at a problem. The honest boundary is stated in the papers: the mixed-domain margin over the best static is small, routing loses on homogeneous SVHN, and the SARCOS robot result is a single pilot that still needs multi-seed matched-budget replication before a high-impact submission.

Four parts, one decision. Let \(x\) be an input and \(g\) a goal embedding. The eye \(e = \mathrm{Eye}(x)\in\mathbb{R}^{32}\) extracts visual cues; the goal pathway embeds the task \(g\in\mathbb{R}^{16}\); the router produces logits over the \(K=9\) primitives:

Each primitive emits into a shared bottleneck latent space \(\ell_k = \mathrm{Prim}_k(x)\in\mathbb{R}^{64}\), and the mixture \(\ell = \sum_k p_k \ell_k\) feeds the task head. Because the Gumbel-Softmax is a continuous relaxation, gradients flow to all primitives early in training; the temperature anneals from soft to hard so the operator commits at deployment:

Staged training is the secret. Training from scratch collapses — an untrained router concentrates on the most expressive primitive and switching never happens (the classic MoE failure). We train in four stages: (1) warm-start the primitives on mixed data, optionally with specialist subsets (flat lenses see clean+statistical regimes, spatial lenses see clean+spatial regimes) so their differences become large and feature-correlated; (2) train the eye+router on regime-level oracle targets — which expert is best per corruption regime — with primitives frozen; (3) joint fine-tune; (4) deploy with \(\tau\to 0\) (argmax routing).

Every number below is read from the committed JSON result files (results/protocol_*.json), three seeds each, on the procedural glyph benchmark (6,000 train / 2,000 test, batch 128, 12 epochs, disjoint train/test random streams).

The toolbox bank is pretrained on mixed data (like a foundation model); the router is trained on clean sim only (deployment experience). Zero-shot, the operator reaches 0.474 real accuracy vs 0.388 for a sim-trained static CNN — the transfer gap drops from 0.611 to 0.526. With 200 real labels, adapting only the router wins:

The next direction combines the Dirty Man with a JEPA-like latent objective. Two augmented views pass through an online/EMA encoder pair; three candidate predictors compete to predict the target latent, and the router is trained from balanced counterfactual competence rather than regime or expert labels.

On a 2,000-train/500-test real-MNIST probe, counterfactual regret was 6.1×10⁻⁵ and agreement with the unconstrained cheapest predictor was 30.2%. The balanced pseudo-policy covered all predictors, but hard deployment collapsed to linear (1.0 utilization). That is the useful scientific result at this stage: low prediction regret does not prove that a non-static policy has learned meaningful specialization. The collapse diagnostic is therefore part of the release, and this prototype is not presented as a competitive JEPA or foundation model.

Run it with python predictive_program.py --dataset mnist --n-train 2000 --n-test 500 --epochs 3. The implementation, JSON output, and anti-collapse tests are included in the repository.

Mixture of experts (Shazeer et al. 2017; Bengio et al. 2013) routes tokens to parallel subnetworks; we route on visual cues plus goal, and train with an oracle-supervised staged protocol that makes the policy learnable and readable. Dynamic networks (Han et al. 2021) change depth, width, or skip connections; we change the family of computation — convolutional vs recurrent vs flat. Neural architecture search finds one structure per dataset at heavy cost; we switch structure per sample, at inference, trained in the same loop. Structural re-parameterization (RepVGG) folds branches at inference; we keep the branches live and route between them. Sim-to-real transfer adapts weights or renders domains; we show adapting which network computes is more parameter-efficient than adapting how it computes.

Python 3.9+ with torch and numpy (matplotlib for figures). CUDA is auto-detected and used automatically (--device cuda to force); every batch and model is moved to the device end-to-end. Every paper number is produced by these scripts from the committed result files — nothing is hard-coded. Runs checkpoint per item and resume across interruptions.

See the repository for the full README, methods, baselines, and the honest failure modes found along the way. LaTeX sources for both papers (nmi_paper.tex, ieee_paper.tex) are included in docs/papers/ and regenerate with xelatex / pdflatex. The headline protocols also run on GPU: a Kaggle kernel and a Hugging Face repo.