🧪 Testing Plan: Tenki GitHub Runners Evaluation
📅 Thursday 6/26 — Phase 1 & Phase 2: Staging & Controlled Evaluation
Phase 1: Staging Load Test
Objective: Validate stability and responsiveness of the new VM-based runners under parallel job load.
Setup:
Trigger ~50 GitHub Actions jobs in parallel using the gh-runner-test repository.
Definition of Done (DoD):
- Jobs are picked up within 30 seconds.
- Job duration is within +5% of baseline execution time from existing Docker-based runners.
Phase 2: Tenki Test Suite Evaluation
Condition: Executed only if Phase 1 is successful.
Objective: Assess runner performance using real-world workflows from the test suite.
Setup:
- Switch the GitHub Actions workspace used by
LuxorLabs/tenki-teststo the new VM-based runners. - Monitor CI jobs for performance and reliability.
Definition of Done (DoD):
- End-to-end performance delta is < 5% compared to current production metrics.
📅 Friday — Phase 3: “Pre-Production” Migration
Phase 3: Luxor Workflow Migration
Precondition: All DoDs from Phase 1 and Phase 2 must be fully met.
Objective: Transition production workloads to the new runners, based on successful Thursday validation.
Setup:
- Migrate all GitHub workflows under the Luxor Tenki Workspace to the new VM-based runners.
Definition of Done (DoD):
- All jobs are successful.
- Performance delta is < 5% compared to current production metrics.