Skip to content
AlphaBase
HomeDocs

Launch Gates

Release gate checklist for launch readiness, and how to certify a BYOK model.

Use these gates before promoting release channels.

Validate The Harness Without Provider Calls

Before spending any model budget, confirm the gate pipeline itself is sound:

Terminal window
bun run test:agentic:dryrun

This runs the full agentic task matrix against a canned no-model stub instead of a real provider — it validates case iteration, output parsing, file/regex checks, and pass-rate computation end to end with zero API calls or spend. A clean bun run test:agentic:dryrun means any failure in a real run below is genuine model/runtime signal, not a harness problem.

Certifying A BYOK Model

AlphaBase is BYOK. A live certification consumes the selected provider’s quota and is meaningful only for the exact provider/model and environment under test. There is no AlphaBase-wide free testing tier or default free certification model. Use this to measure whether a connected model is reliable enough for your own workload:

Terminal window
MODEL=<provider/model> \
MODEL_LABEL=<short-name> \
REQUIRED_PASS_RATE=85 \
bun run certify:model
  • MODEL — required for a live run; use a provider/model that is configured and authorized in the test environment.
  • REQUIRED_PASS_RATE — percent of graded cases required by the matrix (default 100). Choose a different threshold only when your release policy explicitly allows it.
  • Provider-quota, authentication, and network blocks are reported as environment conditions; they do not prove model capability and should not be silently counted as a pass.

A passing certification at your chosen threshold is the evidence that a given model is ready to be recommended as a default, rather than assuming it from the model’s reputation alone.

Required Commands

Terminal window
bun run typecheck
bun run test:smoke:launch
bun run test:agentic:gate # set MODEL=provider/model for a live run
bun run test:tenant:isolation
MODEL=<provider/model> bun run test:launch:gate

P0 Runtime Gates

  • no silent task drop
  • no post-success continuation loop
  • no internal lifecycle artifacts in user-visible response
  • explicit terminal completion/failure handover

These are enforced by the runtime terminal and deliverable contracts. A task that cannot complete should end with a visible terminal status, not disappear or continue indefinitely.

Clean Install Gate (VS Code)

  • build/install the VSIX from the source checkout or use a specifically published release artifact
  • verify diagnostics/sophia/chat runtime from clean state
  • archive prompt + screenshot evidence

Rollback Gate

  • previous known-good version remains installable
  • rollback path is tested at least once per channel promotion

Multi-Tenant Safety Gate

  • tenant isolation checks pass
  • access controls and audit logs are active