Manual and automation testing, in one place
Most QA teams keep two parallel records of what they tested — a spreadsheet (or test management tool) for the manual side, and CI logs for the automated side. Then they spend the rest of the week reconciling them. Here’s the seam we’re building.
- Most teams split manual and automated testing across two records. The reconciliation tax is real and nobody invoices for it.
- The fix isn’t to manualise automation or automate the manual. It’s to give them a single run object that both sides post into.
- Today, TestOrchestrator is the home for the manual half — cases, exploratory sessions, sign-off cycles. A public ingest API so CI can post into the same cycle is on the roadmap.
- The structure is already there. When ingest ships, your existing cycles become the merge point — no migration.
01The two-tool tax
Walk into a typical QA function and you’ll find roughly this picture: a test management tool (or spreadsheet) holds the manual cases. A CI system runs the automated suite. The two systems don’t know about each other. The QA lead, at the end of every release, sits down with a coffee and stitches them into one story for the release meeting.
That stitch is the tax. It looks like reporting work, but it’s actually integration work being done with a human in the middle — every release.
Two parallel records of “what we tested”
One record, two streams writing to it
02Why the lanes diverged
The two lanes diverged for good historical reasons. Manual testing is a knowledge-work activity — a tester reads a case, executes it, records what they observed. Automated testing is a CI activity — a runner executes a suite, emits an XML report. They have different cadences (manual: per release; automated: per commit), different consumers (manual: QA, PM; automated: dev), and different failure modes (manual: subjective; automated: flaky).
Slow, observant, irreplaceable
A human reads the case, follows the steps, watches what actually happens. Catches things no assertion would ever catch — visual weirdness, copy mistakes, “this feels wrong.”
- Per-release cadence
- Knowledge-work tooling (steps, notes, screenshots)
- Consumed by QA + PM
Fast, repeatable, narrow
A test runner asserts. It runs on every commit, finds regressions, catches the obvious-but-easy-to-miss. It does not notice that a button is now slightly off-brand or that the empty state reads weirdly.
- Per-commit cadence
- CI tooling (frameworks, reporters, JUnit XML)
- Consumed by dev + on-call
The mistake is taking that legitimate divergence in execution and using it as an excuse for divergence in recording. The two lanes are doing the same job — testing the same product against the same release — and the record of what was tested should be one thing.
03What “in one place” means
“One place” sounds like a feature claim. It’s actually a data-model claim. The thing that has to be unified isn’t the UI — it’s the object that says “here’s what we tested, against what build, with what outcome.”
Both lanes write into the same cycle. A cycle is “the test record for release 24.05.” A run inside it is “this case, executed against build 24.05.3, by Maria, on Tuesday.” The cycle shouldn’t care whether the runner is a human or a CI job — both produce results with the same shape: case ID, build, status, evidence.
04Where this is heading
The mechanism we’re building is small on purpose. CI will post results to a cycle by ID, optionally with a build identifier and an evidence URL. Once ingest is live, manual runs will land in the same cycle via the app. From the cycle’s perspective, a result is a result — status, build, tester (human or service account), timestamp.
A public ingest endpoint for posting CI results into TestOrchestrator runs is planned. Today, manual cycles and exploratory sessions live in TestOrchestrator alongside any automated coverage you choose to track manually; deeper CI integration is on the way, not currently shipped.
The provenance will be preserved — you’ll be able to filter by “automated only” or “manual only” — but the record will be one. Until then, the manual half lives here. The structure is already designed around the merge.
05What we don’t try to be
This is the part where most “unified” tools quietly overreach. To be clear about what TestOrchestrator is not:
- We don’t run your automated tests. Your CI runs them. We’ll receive results when ingest ships. The runner is whatever you already use.
- We’re not a script editor. Your Playwright tests live in your repo. They should. We don’t try to be Postman or Cypress Studio.
- We don’t replace your bug tracker. External references let cases link out to Jira, GitHub, Linear — your tracker stays the source of truth for engineering work.
What we’re building is the seam — the single record that both lanes write into, and that downstream consumers (release sign-off, audit, coverage reporting) read from. More on the CI/CD shape →
06Frequently asked questions
Does TestOrchestrator have a public ingest API for CI results today?
Not yet. A public ingest endpoint for posting automated test results into TestOrchestrator runs is on the roadmap. Today, the manual half — cases, exploratory sessions, sign-off cycles — lives here; CI continues to run and report your automated suite separately.
What can I do today to consolidate?
Run your manual cases, exploratory sessions, and release sign-off cycles in TestOrchestrator. Keep CI dashboards as the source of truth for automated runs. The release-readiness review pulls from both — a known split, with a single place for everything CI doesn’t cover.
What about flaky tests?
Once CI ingest is live, each result will record its build, which means flake-rate per case across the last N runs becomes a column rather than a separate dashboard. Today: flake tracking stays in your CI.
What if our team is mostly manual today and automation is aspirational?
Then this is good news. The unified model means you can adopt now (manual only) and just start sending results from your first CI suite when ingest ships. Nothing migrates — the cycle was always ready for both.
One cycle. The manual half today. The merge tomorrow.
Free plan, no card. Stand up a cycle this week; CI ingest joins it when the API ships.