Skip to content

Commit 42dccf6

Browse files
jsfezjsfez
authored
Merge pull request #202 from argos-ci/review-with-ai-workflow
feat: review with AI workflow
2 parents 7d98d2d + 16c1c49 commit 42dccf6

3 files changed

Lines changed: 208 additions & 3 deletions

File tree

‎docs/learn/review-workflow/review-builds-with-ai-agents.mdx‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
---
2+
slug: /review-builds-with-ai-agents
3+
sidebar_label: AI agents
4+
description: Use Argos to allow AI agents to review visual changes in a PR.
5+
---
6+
7+
# Review builds with AI agents
8+
9+
Argos lets AI agents review visual changes in a pull request. The agent can use Argos build data to understand what changed, check whether the screenshots match the pull request's intent, and detect regressions.
10+
11+
By default, ask the agent to summarize the visual changes first. If you want it to take action, it can also create an Argos build review to approve the changes or request changes.
12+
13+
## What you need
14+
15+
- A pull request with an Argos build.
16+
- The [Argos CLI](/argos-cli).
17+
- The Argos agent skills, when your assistant supports skills.
18+
- An Argos project token to read build data with `build get` and `build snapshots`.
19+
- A user token to create a build review, only if you want the agent to approve or request changes.
20+
21+
## Quick start
22+
23+
Install the skills:
24+
25+
```bash
26+
npx skills add argos-ci/argos-javascript@argos-cli
27+
npx skills add argos-ci/argos-javascript@argos-pr-review
28+
```
29+
30+
Set a project token so the agent can read Argos build data:
31+
32+
```bash
33+
export ARGOS_TOKEN=<project-token>
34+
```
35+
36+
If you want the agent to create the Argos build review from your local CLI session, authenticate as your Argos user:
37+
38+
```bash
39+
npx @argos-ci/cli login
40+
```
41+
42+
Then start a new AI chat and give the agent the pull request URL:
43+
44+
```text
45+
Use $argos-pr-review to review this pull request with its Argos build.
46+
https://github.com/acme/app/pull/123
47+
```
48+
49+
The `$argos-pr-review` skill contains the detailed review workflow. The `$argos-cli` skill gives the agent the CLI commands, flags, authentication rules, and output formats.
50+
51+
## What the agent does
52+
53+
The agent will inspect the pull request, find the Argos build from the pull request status check or Argos comment, and use Argos build data to understand the visual changes.
54+
55+
It should first summarize what changed visually. Then it compares the screenshots with the pull request intent.
56+
57+
When the screenshots match the pull request intent, the agent can suggest approving the build. When they reveal a regression, it can suggest requesting changes and explain which snapshots are affected.
58+
59+
If the build is still pending, the agent should wait before reviewing. If there are no visual changes or the build is already accepted, no visual review is needed.
60+
61+
---
62+
63+
## Other cases
64+
65+
### If the agent cannot access the pull request
66+
67+
You can provide the Argos build URL and a short description of the expected change:
68+
69+
```text
70+
Use $argos-pr-review to review this Argos build:
71+
https://app.argos-ci.com/acme/app/builds/456
72+
73+
Expected change:
74+
The login page was redesigned with social login options.
75+
```
76+
77+
The build data still lets the agent explain what changed visually, but the pull request context makes the review decision stronger.
78+
79+
### If your assistant does not support skills
80+
81+
Use a more explicit prompt:
82+
83+
```text
84+
Review the visual changes in this pull request with Argos:
85+
https://github.com/acme/app/pull/123
86+
87+
Use the pull request title, description, linked issue, and code diff to understand the intended change.
88+
Find the Argos build, fetch the build metadata, and fetch the snapshots that need review.
89+
Use the diff mask, baseline file, current file, and metadata as visual evidence.
90+
91+
First summarize what changed visually in the build.
92+
Then compare the screenshots with the pull request intent.
93+
94+
If the snapshots match the intent, suggest approving the Argos build.
95+
If they reveal a regression, suggest requesting changes and list the affected snapshot names.
96+
If they look flaky, explain the flake signal and do not suggest approval until it is resolved.
97+
```
98+
99+
To let the assistant create the Argos build review, add an explicit instruction. The CLI must be authenticated as an Argos user with `argos login`, or the assistant must pass a personal access token to `build review`:
100+
101+
```text
102+
After summarizing the visual changes, create an Argos build review with the appropriate conclusion.
103+
```
104+
105+
### Authentication details
106+
107+
Reading build data requires a project token. Creating a build review requires a user token because the review is attributed to an Argos user and checked against that user's project permissions.
108+
109+
For details about token precedence, `--token`, and personal access tokens, see [Argos CLI authentication](/argos-cli#authentication).
110+
111+
## Limits
112+
113+
Argos still performs deterministic visual comparison. AI agents use Argos build data as review evidence; they do not replace your team's ownership of review decisions.

‎docs/sdks/cli.mdx‎

Lines changed: 95 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,27 +5,42 @@ sidebar_position: 1
55
---
66

77
import { InstallDevDep, RunPkgCommand } from "@site/src/partials";
8+
import personalSettingsTokensPage from "@site/src/img/personal-settings-tokens.png";
89

910
# Argos Command Line Interface (CLI)
1011

11-
Use the Argos CLI for seamless screenshot uploads to our visual testing platform, ideally integrated into your CI workflow.
12+
Use the Argos CLI to upload screenshots, inspect builds, and create build reviews from scripts, local workflows, or AI agents.
1213

1314
You can access `@argos-ci/cli` through our [npm package](https://www.npmjs.com/package/@argos-ci/cli). The source code is available on our [GitHub repository](https://github.com/argos-ci/argos-javascript/tree/main/packages/cli).
1415

1516
## Installation
1617

1718
<InstallDevDep dependency="@argos-ci/cli" showCliLink={false} />
1819

19-
## Configuration
20+
## Authentication
2021

21-
Your configuration requirements may vary depending on the CI you use. Typically, you'll need to set ARGOS_TOKEN as a environment variable (ie. secret).
22+
Your configuration requirements vary depending on the command you run.
23+
24+
For CI uploads, set `ARGOS_TOKEN` as an environment variable, usually as a CI secret.
2225

2326
:::note
2427

2528
You can also specify the token with the `--token=<your-repository-token>` argument. However, for security reasons, we recommend using an environment variable instead.
2629

2730
:::
2831

32+
For local review commands, sign in once with the browser-based login flow:
33+
34+
<RunPkgCommand command="argos login" />
35+
36+
`argos login` authorizes the CLI and stores a user token on your machine. This token is used by commands that need to act as a user, such as creating a build review.
37+
38+
:::warning
39+
40+
Do not use `argos login` in CI. CI uploads should use `ARGOS_TOKEN`, `--token`, or tokenless authentication when supported by your CI integration.
41+
42+
:::
43+
2944
## Upload Command
3045

3146
Use the `upload` command to upload screenshots stored in your `./screenshots` directory.
@@ -41,6 +56,83 @@ You can enable debug mode by setting the `DEBUG` environment variable.
4156
envVariable="DEBUG=@argos-ci/core"
4257
/>
4358

59+
## Build Commands
60+
61+
Use `build` commands to fetch build metadata, fetch snapshot diffs, and create a review on a build.
62+
63+
### Get build metadata
64+
65+
Use `build get` to fetch status, branch, commit, stats, and the build URL.
66+
67+
<RunPkgCommand command="argos build get <buildNumber>" />
68+
69+
You can also pass an Argos build URL directly:
70+
71+
<RunPkgCommand command="argos build get https://app.argos-ci.com/team/project/builds/72652" />
72+
73+
Use `--json` when another tool needs to parse the response:
74+
75+
<RunPkgCommand command="argos build get https://app.argos-ci.com/team/project/builds/72652 --json" />
76+
77+
### Fetch snapshot diffs
78+
79+
Use `build snapshots` to fetch snapshot diffs for a build.
80+
81+
<RunPkgCommand command="argos build snapshots https://app.argos-ci.com/team/project/builds/72652 --json" />
82+
83+
Add `--needs-review` to only return snapshots that need a review decision:
84+
85+
<RunPkgCommand command="argos build snapshots https://app.argos-ci.com/team/project/builds/72652 --needs-review --json" />
86+
87+
Each snapshot diff includes the status, score, diff mask URL, baseline file, current file, and metadata provided by the SDK.
88+
89+
### Submit a build review
90+
91+
Use `build review` to approve a build or request changes.
92+
93+
<RunPkgCommand command="argos build review https://app.argos-ci.com/team/project/builds/72652 --conclusion approve" />
94+
95+
To reject the visual changes:
96+
97+
<RunPkgCommand command="argos build review https://app.argos-ci.com/team/project/builds/72652 --conclusion request-changes" />
98+
99+
When you pass a build number instead of a full URL, also pass the project path:
100+
101+
<RunPkgCommand command="argos build review 72652 --project team/project --conclusion approve" />
102+
103+
### Project tokens and personal access tokens
104+
105+
Uploads and build reads can use a project token.
106+
107+
Creating a build review requires a personal access token because the review is attributed to an Argos user and checked against that user's project permissions.
108+
109+
To create a personal access token manually, go to your personal account settings, open **Tokens**, then click **Generate new token**.
110+
111+
<img
112+
src={personalSettingsTokensPage}
113+
alt="Personal settings tokens page"
114+
className="rounded border"
115+
style={{ marginBottom: 20 }}
116+
/>
117+
_A personal settings tokens page_
118+
119+
For `build review`, authentication is resolved in this order:
120+
121+
1. `--token <token>`
122+
2. `ARGOS_TOKEN`
123+
3. token stored by `argos login`
124+
125+
Project tokens can inspect build data, but they cannot create reviews. If `ARGOS_TOKEN` contains a project token, use `argos login` locally or pass a personal access token with `--token`.
126+
127+
## AI Agent Skills
128+
129+
The [`argos-javascript`](https://github.com/argos-ci/argos-javascript) repository includes skills that help AI agents use Argos CLI commands and review pull requests with Argos build data:
130+
131+
- [`argos-cli`](https://github.com/argos-ci/argos-javascript/tree/main/skills/argos-cli): use Argos CLI commands, flags, authentication, and output formats.
132+
- [`argos-pr-review`](https://github.com/argos-ci/argos-javascript/tree/main/skills/argos-pr-review): review a pull request with an Argos build as visual evidence.
133+
134+
See [Review builds with AI agents](/review-builds-with-ai-agents) to install and use the skills in a pull request review workflow.
135+
44136
## Help Command
45137

46138
To view a list of available options, use the argos help command.

‎src/img/personal-settings-tokens.png‎

172 KB
Loading

0 commit comments

Comments
 (0)