🚦 AI Model Release Control Center
Train → Evaluate → Compare → Investigate → Gate → Ship → Monitor → Learn
A production-oriented AI evaluation and release-engineering system focused on the engineering interface between post-training and production. Phase 4 adds hardened live-provider evaluation, optional LLM judging, production-trace ingestion, model-registry semantics and evidence-linked promotion.
Edit the candidate metrics to run a lightweight counterfactual post-training experiment. The release decision and hypothesis ranking recompute from the observed regression pattern.
Baseline vs candidate after training intervention
What would I investigate?
A regression number is an observation, not an explanation. Rank plausible causes, then design experiments that can falsify them.
Potential causes
Requires HF_TOKEN configured as a Hugging Face Space secret. GitHub Actions secrets do not automatically become Space secrets.
Normalized evaluation
Promotion is evidence-linked: candidate → approved → production; HOLD routes a model to blocked. The public demo emits the promotion record but does not mutate a production registry service.
Research-to-production trust model
- Post-training interventions are represented as experiments with explicit baseline, candidate, dataset and evaluation set.
- Improvement is multi-objective: quality gains do not erase safety, latency or cost regressions.
- Hypotheses are kept distinct from findings; follow-up experiments are designed to falsify plausible causes.
- Provider credentials exist only server-side.
- Provider outputs are normalized before release policy consumes them.
- Security and correctness checks are deterministic and independently testable.
- The LLM judge is optional and cannot overrule deterministic critical failures.
- Production traces are schema-validated before aggregation.
- Model promotion references an evidence identity from the release-control plane.
- Generated code is not executed in the Space process; production code execution should use an isolated sandbox/worker boundary.
Evidence chain: GitHub · Dataset · Methodology · Portfolio