Trackers

A tracker is a configured connection to one external system — for example, your company's Jira Cloud site or a GitHub repository. (Earlier versions of this page called trackers “integration sources”.) Once a tracker is set up, project members can link test cases, runs and explorations to issues in it.

Connecting Linear? Linear uses a one-click OAuth connection instead of a manual API token, and unlocks live issue search, AI insight on linked issues, and test-case generation from the Linear sidebar. See the dedicated Linear Integration guide.

GitLab Issues is not available. TestOrchestrator has no GitLab connector yet, so GitLab is not offered in the provider list or under Not set up. A GitLab tracker created before this change still appears in the list, but linking an issue to it does not work. Every other provider — Jira Cloud, GitHub Issues, Azure DevOps, Notion, Linear and Custom Tracker — has a working implementation.

Supported providers

TestOrchestrator supports the following external systems out of the box, plus a Custom Tracker option for any other system.

The Trackers tab

Open Admin → Integrations. The first tab, Trackers, is one list: Linear first (it is set up by connecting, see the Linear Integration guide), then every tracker you have configured, then a Not set up section listing each provider you have not configured yet, each with a Set up button.

Each row shows the tracker's name and host, its provider, which projects it is available in, and its status: Enabled, Disabled, or Can’t connect when an enabled tracker's last connection test failed (hover the status for the tracker's own error message). A disabled tracker always reads Disabled. Every row action is in the row's … menu: Test connection, Edit tracker…, Enable/Disable and Delete tracker….

Adding a tracker

Click Add tracker, or Set up on a provider in the Not set up section. The tracker editor opens as its own page, with one save bar at the bottom. It has these sections:

• Tracker — the Provider (Jira, Azure DevOps, GitHub Issues, Notion or Custom tracker; Linear is not offered here), the Name people see in the link picker, and whether it is Enabled. Changing the provider re-shapes the rest of the form and clears the credentials.
• Connection — the Base URL, any fields the provider needs (for example the Azure DevOps organisation and project, or the GitHub owner and repository), Sign in with (see Authentication types) and the credentials themselves.
• Issue links — the Issue URL, the Issue key format, a Try a key preview, the New issue URL and, for providers that file issues, the defaults new issues are created with (for example Jira's default project key and issue type).
• What TestOrchestrator can do — the capabilities described below.
• Projects — where the tracker is available.
• Custom API spec — Custom trackers only: the JSON spec that tells TestOrchestrator how to read and create issues. It must be a JSON object.

If something is missing or wrong, the save bar names it (“Give the tracker a name”, “Enter the API token”) and the field says what to fix.

Issue URL, key format and the live preview

• Issue URL — optional. {key} is replaced by the issue key, for example https://yourcompany.atlassian.net/browse/{key}. {key} is the only placeholder: a template without it cannot be saved, and it must be on the same host as the base URL. Leave it empty to build links from the base URL.
• Issue key format — optional. A regular expression that keys must match when someone links an issue (for example ^[A-Z]+-\d+$ for Jira). An expression that is not valid blocks the save.
• Try a key — type a sample key and the editor shows the link it produces (“PAY-142 → https://yourcompany.atlassian.net/browse/PAY-142”), or says that the key does not match the key format.
• New issue URL — where Create issue sends people. Leave it empty to hide that action. Notion has no new issue URL.

Saved credentials

Credentials are encrypted and never shown again after saving. When you edit a tracker that has credentials, the Connection section says so instead of showing empty fields — for example API token saved, with when it was set. To change it, click Replace… and enter the new values; Keep the saved credentials takes you back without changing anything. Saving while you keep the saved credentials leaves the stored secret exactly as it was. Changing the provider or Sign in with always asks for new credentials — a secret saved for one sign-in method is never reused for another. Switching Sign in with to No credentials removes the saved secret when you save; the form says so before you do.

What TestOrchestrator can do

Capabilities control what TestOrchestrator may do with this tracker. Each can be switched on or off. With Sign in with: No credentials, reading issues, commenting and webhooks are switched off and locked, because they need credentials; Notion cannot create issues.

Add comments and Process webhooks are coming in a future release. You can enable these flags when configuring a source, but they are not yet active.

Projects

• All projects — the tracker is available in every project, including projects created later.
• Selected projects — only the projects you pick can use it. You must pick at least one; a tracker available nowhere cannot be saved.

Authentication types

The authentication type determines how TestOrchestrator identifies itself when calling the external system's API.

The credential fields change with the provider and the sign-in method you choose. They are stored encrypted and never displayed after saving — see Saved credentials.

Testing a connection

There are two ways to test a tracker:

• Before saving — in the tracker editor, click Test connection in the Connection section. TestOrchestrator tries the details on the page without saving anything. When you are editing a tracker and keeping its saved credentials, the test uses the saved ones. The result appears under the credentials.
• From the list — choose Test connection in a tracker's … menu. The result is saved: a failure shows Can’t connect on the row (hover it for the reason) until a later test succeeds or the tracker's connection details change, even after a reload.

Either way, a failure also appears as a message at the bottom of the page. Testing does not require the tracker to be enabled.

Connection tests are limited to 10 a minute for each person (and 30 a minute for the workspace). When you reach the limit, the button reads Try again in N s until another test is allowed.

Enabling and disabling trackers

Choose Disable or Enable in the row's … menu. The change is immediate, and the message that confirms it has an Undo.

• Enabled — the tracker is offered when project members link an issue.
• Disabled — the tracker is hidden from the link picker. Existing links to its issues stay on cases and runs.

Editing a tracker

Choose Edit tracker… in the row's … menu. Every field can be changed; saving takes effect immediately.

Deleting a tracker

Choose Delete tracker… in the row's … menu. The row briefly shows Checking use while TestOrchestrator counts the links to the tracker's issues, then the dialog states what will be deleted — for example “This removes the tracker and its 41 issue links across 2 projects.” — and you confirm. If the tracker is enabled, the dialog also offers Disable instead. The count is exact and covers every project in the workspace, because deleting a tracker deletes its links everywhere — including projects it is no longer available in. If exploration findings are linked to the tracker's issues, the dialog also says how many lose their link (the findings themselves are kept). If the count cannot be loaded, the dialog opens without a number and says the links could not be counted.

Deleting a tracker deletes its links. The tracker's settings, its saved credentials and every link to its issues are removed — cases and runs lose those links, and there is no way to restore them. If you only want to stop people adding new links, disable the tracker instead: disabling keeps every link and can be reversed.

Who can do what

Seeing this page needs the View external integrations permission; changing trackers needs Manage external integrations. Without it the page is read-only and says so.