|
1 | 1 | --- |
2 | 2 | slug: /about-flaky |
3 | 3 | sidebar_position: 10 |
4 | | -description: Tackle flaky tests with Argos strategies that improve reliability, reduce noise, and support accessibility for a stable visual testing workflow. |
| 4 | +description: Understand what makes visual tests flaky and how Argos helps you detect, ignore, and fix the root causes for a stable testing workflow. |
5 | 5 | --- |
6 | 6 |
|
7 | 7 | # Flaky Tests |
8 | 8 |
|
9 | | -Navigate the challenge of flaky tests with Argos: Discover strategies to enhance test reliability and promote accessibility for a stable visual testing environment. |
| 9 | +Flaky tests fail or change without a meaningful code change, eroding trust in your test suite. Argos tackles flakiness from two angles: it **detects** unstable tests so you can decide with confidence, and it gives you **tools and strategies** to address the root causes. |
10 | 10 |
|
11 | | -## Emphasis on Accessibility |
| 11 | +## How Argos helps |
12 | 12 |
|
13 | | -At Argos, we advocate for accessibility-focused end-to-end testing. Enhancing accessibility not only supports users with disabilities but also strengthens test consistency. |
| 13 | +- **Flaky test detection**: Argos flags unstable tests with a flaky badge, a stability score, and a detailed history view. See [Flaky Test Detection](/flaky-test-detection) for details. |
| 14 | +- **Ignore noisy changes**: Dismiss a specific change directly from the build or test page so the same change is no longer reported. |
| 15 | +- **Auto-ignore recurring flaky changes**: Configure Argos to automatically ignore changes that recur over the last 7 days, filtering out noise while keeping real regressions visible. |
14 | 16 |
|
15 | | -## Common Causes |
| 17 | +These features let you separate signal from noise without losing the ability to catch real regressions. |
16 | 18 |
|
17 | | -Flakiness often stems from: |
| 19 | +## Common causes |
18 | 20 |
|
19 | | -- **Dynamic Content**: Flakiness due to content changes, like ads or user-generated content. |
20 | | -- **Asynchronous Loading**: Inconsistent element or style loading affecting visuals. |
21 | | -- **Rendering Differences**: Browser or device variations altering visual output. |
22 | | -- **External Dependencies**: Reliance on external systems causing variability. |
23 | | -- **Animations and Transitions**: Unhandled animations leading to different visual states. |
24 | | -- **Resolution and Scaling**: Screen resolution or scaling differences impacting visuals. |
| 21 | +Visual flakiness usually stems from one of the following: |
25 | 22 |
|
26 | | -Solutions involve identifying and addressing these root causes. |
| 23 | +- **Dynamic content**: Ads, user-generated content, or other data that changes between runs. |
| 24 | +- **Asynchronous loading**: Elements or styles that render at inconsistent times. |
| 25 | +- **Rendering differences**: Browser, OS, or device variations that alter visual output. |
| 26 | +- **External dependencies**: Reliance on third-party systems that introduce variability. |
| 27 | +- **Animations and transitions**: Unhandled motion captured mid-frame. |
| 28 | +- **Resolution and scaling**: Screen size or pixel density differences between environments. |
27 | 29 |
|
28 | | -## Policy |
| 30 | +## Fix the root cause |
29 | 31 |
|
30 | | -We don't offer a "flaky test ignore" option, aligning with our ethos that addressing issues is preferable to bypassing them. |
| 32 | +Ignoring noise is useful, but the most reliable suite is one where flakiness is addressed at the source. The Playbook covers practical techniques: |
| 33 | + |
| 34 | +- [Wait for Loading](/wait-for-loading) — capture screenshots only after the page is fully loaded. |
| 35 | +- [Stabilize Date & Time](/date-stabilization) — hide or freeze dynamic dates and times. |
| 36 | +- [Browser Glitches](/browser-glitches) — standardize environments and handle rendering quirks. |
| 37 | +- [Argos Helpers](/argos-helpers) — use `data-visual-test` attributes to control how dynamic elements are captured. |
| 38 | + |
| 39 | +## Emphasis on accessibility |
| 40 | + |
| 41 | +At Argos, we advocate for accessibility-focused end-to-end testing. Improving accessibility (semantic markup, `aria-busy`, predictable focus states) not only supports users with disabilities but also makes tests more deterministic and less prone to flakiness. |
0 commit comments