AI Test Case Generation

Draft test cases with AI from a requirement — pasted text, or an issue in Linear, Jira, GitHub, Azure DevOps or Notion. Pick the folder, template and how many cases you want, then review each draft before any case is created. You can also add the drafts straight to the list. Generation is available on every plan, including Free, which has a limited monthly allowance.

Before you can use it

If you cannot find Generate with AI, the usual reasons are:

• The workspace has not turned AI on. A Tenant Admin does this once in AI Settings, which includes accepting the AI Data Processing Addendum. Until then no AI control appears for anyone in the workspace.
• AI is switched off for this project. A Tenant Admin can turn AI off for individual projects — see AI Settings.
• You do not have the Create test cases permission. Generating cases writes cases, so it needs the same permission as creating one by hand.
• You are composing a test run. The AI entries are hidden until you leave the run composer, so a generated batch can never be added to a run by accident.

If the AI terms were updated since your workspace accepted them, the generator still opens and says so. An admin can accept the new terms in AI Settings. Anyone else sees who to ask.

Opening the generator

Open it from anywhere in Test Cases. That includes All cases, because the generator has its own folder picker:

• A folder’s context menu — Generate cases with AI (the folder is pre-selected).
• The sparkle button in the Test Cases toolbar.
• The command palette (⌘/Ctrl+K) — Generate cases with AI.
• The Generate with AI button on an empty folder or an empty project.

The folder you are in is pre-selected as the destination. At All cases the generator picks the first top-level folder. You can change it before generating.

The expand button in the generator’s header (Expand composer; Collapse composer to go back) makes the generator larger on every step, the same way the Explorations composer expands. Expanded, each draft in the review list also shows its steps and expected results under the title. The generator always opens at its normal size.

Choose a source

The source is the requirement the AI reads. It sits in a row of tabs along the top: Pasted text, Linear, Jira, GitHub, Azure DevOps and Notion. Each tracker tab has a dot. A filled dot means the project has that integration connected and it can read issues. An empty ring means it is not set up yet — or, for a tracker added without credentials (link only), that it cannot read issues.

The search works the same way as linking an issue from a test case: results show each issue’s status, key and title, a tracker that cannot be reached says so on its own line, and the arrow keys and Enter pick a result (Esc closes the results without closing the generator). A tracker that can only be read by key, rather than searched, shows a key field instead: type or paste the key and press Enter to look it up. A tracker connected for links only (no read access) cannot be used as a source; a workspace admin can add a token with read access.

For a tracker, the line under the issue names the integration in use, and the issue’s description is what the model reads. If the project has more than one integration for a tracker, the search covers all of them and picking an issue picks its integration; you can also choose one from that line to search only it. If it has only one, it is selected for you. A tracker that is not connected shows a link to Admin › Integrations, where a workspace admin can connect it. See Integrations.

Folder, template and count

Below the source, three pills set how the cases are drafted:

• Folder — where the cases are created. It opens a searchable list of your folder tree.
• Template — the test case template the drafts fill in, including steps and priority. The project’s default template is pre-selected.
• Count — 3, 5, 8, 10, 15 or 20 cases.

Optionally, steer the drafts in the Guide the AI field between the source and the pills (up to 200 characters; a counter appears as you near the limit). While it is empty it offers three suggestions — Edge cases only, Focus on regression and Negative paths — that fill it in one click, and you can edit the text afterwards. The field is unavailable while the selected tracker is not connected. If Generate is disabled, the footer says what it is waiting for, and so does the button’s tooltip. Press ⌘/Ctrl+Enter to generate from the keyboard.

Review first, or add instantly

The arrow next to Generate chooses what happens with the drafts. The generator remembers your choice.

• Review drafts first (default) — you read, edit, keep or drop each draft before any case is created.
• Add instantly — the generator closes right away and the drafts are added to the folder as soon as they are ready. You can keep working in the meantime, and edit or delete the new cases from the list.

While it generates

A progress line shows three steps (reading the source, drafting the cases, matching the template’s fields) and how long it has been running. If it takes longer than usual, a note says so. Select Stop to cancel at any time. Nothing is added, and your text, issue and settings are kept so you can adjust them and try again. Stopping does not refund the attempt: it still counts toward your AI usage.

Review the drafts

The review list shows how many drafts came back and which model wrote them. For each draft you can:

• Keep or skip it with its checkbox, or use Select all / Clear. Only selected drafts are added.
• Edit the title in place (up to 255 characters).
• Expand it to read the numbered steps. Each step shows its action and the expected result.

Each row also shows the draft’s priority and its number of steps. If fewer drafts came back than you asked for, a note explains why. For example, “Returned 6 of 15 — your text supports about 6 distinct cases”. The model’s answer is also checked against your template: drafts that repeat each other and values that do not fit a field are removed before you see them, and a short line under the list says how many. If you close the generator with drafts still selected, it asks once before discarding them, because generating them used AI allowance.

Adding the cases, and Undo

Select Add (or press ⌘/Ctrl+Enter). The generator closes and the new cases appear in the list at once, marked Saving… until each one gets its case number. A single notification follows the whole batch, for example “5 cases added to Checkout”, with two actions:

• View — go to the destination folder.
• Undo — delete the cases this batch just created. It is shown only if you have permission to delete test cases.

If a case cannot be saved, the error is shown on that row, with Retry and Discard. Retrying cannot create a duplicate. Cases generated from a tracker issue are linked back to that issue, and the link shows the issue’s title and status as soon as you open the case. If the integration the drafts came from was turned off or removed while you were reviewing, the notification says so instead of offering a retry that cannot work.

Links made by older versions of the generator may show only the issue key. The first time you open such a case, its linked issues are read from the tracker once and their titles fill in, without reloading the page.

Plans, limits & errors

• Every plan includes AI generation. The Free plan comes with a limited monthly allowance, and the generator’s footer says so. Upgrading to Starter or above gives you more.
• Monthly limit reached. Every generation counts toward the workspace’s monthly AI allowance, which also covers AI insights on linked issues and the QA agent’s analysis. Once it runs out, the generator says so, names the date the allowance resets, and offers View plans. An attempt that is refused this way is not counted.
• Too many requests. Generation is rate-limited per person and per workspace. If you hit the limit, the generator shows how many seconds to wait, and Generate turns back on by itself.
• The tracker could not be read. The generator says what went wrong, for example that the issue was not found or the tracker did not answer, and offers Try again. If the integration needs reconnecting, it tells you where to do that. For GitHub it also says when the number is not an issue reference, when the issue is not in the connected repository, and when the GitHub tracker has no token and can only store links — a workspace admin adds one in Admin › Integrations.
• Generation failed. The error is shown in the generator with Try again, and your inputs are kept.
• The project’s AI model is being retired. A note above the requirement names the model and the date. Generation keeps working until then. Ask a workspace admin to pick another model in AI Settings.

Usage is shown as a percentage in AI Settings, on Settings → Subscription and on the workspace home, never as a token count. You can compare plans on the pricing page.

AI Settings

Settings → AI Settings is where a workspace turns AI on and chooses the model its projects use. Only people who can manage user roles can open it.

• Not turned on. The Workspace AI panel reads Off and explains that AI lets project members generate test cases and add AI insights to linked issues, both counted toward one monthly allowance. Accept and turn on AI accepts the data-processing terms and turns it on.
• Unavailable. If AI has not been turned on for the whole TestOrchestrator deployment, the page shows only a note saying so.
• Not included. If test case generation is not part of your workspace, the status reads Not included with Contact support to turn on test case generation, and the model cannot be changed.

Once AI is on, the page has two tabs: General and Projects.

General

The Workspace AI panel opens with one sentence — for example Projects use Claude Sonnet 4.6 unless they override it. 2 projects use another model, 1 has AI turned off. Each count opens the Projects tab filtered to those projects. Below it:

• Provider — shown only when more than one AI provider is configured.
• Model — used by every project that does not override it. A pick takes effect at once, and its confirmation has an Undo. If the model is being retired, a warning above these rows says so.
• Data processing — when the addendum was accepted, and Turn off AI…. It asks first, Turn off AI for this workspace?: Keep AI on leaves it alone, and Turn off AI stops new AI work in every project. Usage history and audit logs are kept.

The Usage panel shows how much of this month’s AI allowance is used, as a percentage, and how many days are left until it resets. Generating test cases, linked-issue insights and QA agent analysis are counted together. From 80% the meter turns amber; at 100% a line says the allowance is used up, with Compare plans for people who manage billing.

Projects

The Projects tab lists every project with its model. Narrow it with All, Overrides (projects with their own model, marked Override) and AI off, or search by name. Click a row, or choose Edit AI settings… from its ··· menu, to open AI for <project>:

• Use AI in this project — turn it off for a project that must not send requirements to an AI provider.
• Model, and Provider when more than one is configured — Workspace default (<model>) follows the workspace setting when it changes; anything else is an override.

The row menu also has Turn off AI or Turn on AI, and Reset to workspace default for a project that has its own model. Every change shows at once, and its confirmation has an Undo.

Related