Identity Through Interruption

A motion-signature-first decision architecture for multi-object re-identification — staged, hypothesis-holding identity decisions with gait, trajectory-style, and geometry as primary signals, and appearance demoted to a supporting, auditable gate.

Sehaj Randhir Singh · 2026 · simulation + MOT17 public-detection benchmark

[Code] [NMI manuscript] [WACV 2027] [IEEE] [BibTeX]

The problem

Watch: identity kept through the interruptions

Simulated streams with persistent-ID overlays. Solid boxes are observed detections; dashed boxes are the belief stream — the ghost-predicted trajectory emitted while the target is hidden, so the path never teleports.

The architecture

Two-loop online architecture. A high-frequency loop (SORT-lineage temp tracks + ghost prediction with growing uncertainty) and a low-frequency loop (rolling pose buffers, cadenced ST-GCN gait signatures) feed a decision layer that binds, spawns, or holds hypotheses. Outputs: an observation stream (MOT-comparable) and a belief stream (a continuous predicted trajectory for every non-parked ID, every frame).

Synthetic hard segments: zero switches where appearance breaks

Hard-segment IDF1 / IDSW, 7 scenarios × 20 seeds (identical detection streams for all trackers). Bold = best IDSW; the paper reports both wins and losses honestly.

motion_reid and DeepSORT are the only trackers that stay at (statistically) zero IDSW through every interruption. The wins are reported with their costs: hypothesis-holding trades dense-scene IDF1 (0.467/0.430 vs DeepSORT 0.547/0.605) for dramatically fewer switches — a deliberate, quantifiable trade.

Synthetic IDSW by scenario. The “zero through interruption” panel — SORT and ByteTrack switch on occlusion in every seed; our hard-segment switches are at most one, on one seed.

Ablations. The hypothesis-holding cliff (dense 14.9 → 84.4 without it) and the group-level cluster contrast (full vs −cluster).

Tracking, not teleportation. On re-entry, the observation stream fragments into 34 interruptions (shaded); the belief stream stays continuous — 0 interruptions, 0 teleports, easing into re-emergence.

MOT17 hybrid. The cluster mechanism cuts IDSW 1,265 → 995 (−21%, no pose); removing the uncalibrated CNN appearance gate cuts 995 → 532 (−47%, IDF1 rising); real-skeleton gait leaves the tracker nearly unchanged.

Real-data transfer (MOT17, all seven train sequences)

Hard-segment totals on the identical FRCNN public-detection stream, pose inherited from YOLOv8-pose, appearance from ImageNet-pretrained MobileNet-v3 (DeepSORT receives the same embeddings).

Two measured asymmetries carry the real-data story. The architecture transfers; the uncalibrated channels do not. The pose-free cluster mechanism cuts MOT17 identity switches 1,265 → 995 (−21.3%) and raises hard-IDF1 0.300 → 0.324 (MOT17-04 −47.6%, MOT17-11 −30.0%), and the belief stream reports zero interruptions and zero teleports on all seven real sequences. Meanwhile an ImageNet-generic MobileNet-v3 embedding at ≈34 px crop scale is not an identity embedding: used as an appearance gate it produced 463 of 995 switches (47%), and removing it raised IDF1 — the same finding holds for DeepSORT (0.522 → 0.458). The paper formalizes the channel-calibration audit: a channel is flagged untrustworthy if removing it does not lower IDF1. Both appearance channels fail it; appearance-free trackers are invariant by construction.

Detector-recall ceiling. A real YOLOv8 on 1080p MOT17 recalls only ~34% of visible pedestrians (median person ≈34 px tall). No tracker escapes the detection floor.

Pose-coverage headroom. The gait channel's benefit collapses inside the real MOT17 pose-coverage band (0–41%): the tracker was never gait-limited — it is pose-coverage-limited.

The honest numbers

Every number on this page and in the papers is regenerable from the committed JSONs in results/. python audit_numbers.py re-verifies all 131 headline checks against the ground-truth files.

• Zero IDSW through every constructed interruption (synthetic, 20 seeds): 0 switches in 20/20 seeds on re-entry, the ball, and the twin crossing; at most one switch on occlusion. The ball — no usable appearance — is won by motion (IDF1 0.817 vs 0.668 best appearance baseline).
• Hypothesis-holding is the load-bearing mechanism: disabling it multiplies dense-scene IDSW ≈5.7× (14.9 → 84.4, p_holm = 4×10⁻⁴).
• Belief trajectories are continuous: 0 interruptions / 0 teleports, mean single-frame belief jump 21.7 px (max 53.5 px) — synthetic and real.
• Group-level pile-up tracking works without pose: +4.5 pp pile-up IDF1 synthetically and −21.3% IDSW on real MOT17 (1,265 → 995).
• Transfer is asymmetric: the architecture transfers (0 interruptions / 0 teleports on all seven real cameras); the uncalibrated appearance channel harms every tracker that trusts it — 47% of our switches (995 → 532, IDF1 rising) and a 0.522 → 0.458 hit to DeepSORT.
• The constants survive cross-validation and self-calibration: leave-one-out re-derivation gives mean hard-IDF1 0.287 ± 0.063 (IDSW 534 vs 0.333/533 fixed), and the stream-derived constants (9.1/13.7 vs GT-derived 8.0/12.0) reproduce the box-only result with no ground truth anywhere in the loop.
• The roadmap is closed and measured: removing the uncalibrated appearance gate removes 47% of switches; real-skeleton gait training more than doubles pooled embedding recall@1 (0.223 → 0.515) yet moves tracking by nothing measurable (995 → 1,000, p = 0.75) — pose coverage, not embedding quality, is the binding constraint.
• The velocity gate closes most of the detector-stress gap: MOT17-04 IDSW 33 → 15, IDF1 +3 pp, gap to SORT 0.7 pp (0.177 vs 0.184), exactly scoped — enabled, it leaves every published synthetic number unchanged.

Papers

BibTeX

@article{singh2026identity,
  title   = {Identity Through Interruption: A Motion-Signature-First Decision
             Architecture for Multi-Object Re-Identification},
  author  = {Singh, Sehaj Randhir},
  year    = {2026},
  note    = {Pre-submission draft; all measurements reproducible from
             committed result JSONs},
  url     = {https://github.com/sehajr-singhs/motion-reid}
}

Reproduce

python train_gait.py            # ST-GCN triplet model (results/gait_model.pt)
python benchmark.py             # synthetic benchmark (results/benchmark.json)
python ablate.py                # ablations (results/ablation.json)
python make_demo.py             # belief-stream videos (results/demos/*.mp4)
python real_benchmark.py        # MOT17 box-only, 7 seqs
python real_hybrid_benchmark.py # MOT17 hybrid + --ablate variants
python stress_lowrecall.py      # detector-stress + recall sweep
python real_selfcal.py          # self-calibrated transfer constants
python audit_channels.py        # channel-calibration audit
python audit_numbers.py         # re-verify all 131 headline numbers
python -m unittest discover -s tests   # 27 unit tests (core + MOT17 loader + continuity)