📖 The Story: The 50 GB Artifact Folder

A team enabled video recording for all E2E tests with video: 'on'. They had 200 tests running on 3 browsers in CI, with 2 retries. Every CI run produced 1200 video files — about 50 GB. The artifact storage bill tripled in a month.

Worse, when a test failed, the engineer had to download the artifact, unzip it, and hunt through 12 video files (6 per retry attempt) to find the one from the failing run. Half the time, they gave up and just re-ran the test locally.

The team tried video: 'retain-on-failure', which only keeps videos for failing tests. But with retries, it kept videos for every failing attempt — if a test failed twice and passed on retry, they still had 2 videos, not 1.

The new video modes in 1.61 fix this. ‘retain-on-failure-and-retries’ keeps only the final failing run’s video — one file, the right one.

🎯 Why It Matters

🌍 Real World Example

A team runs 200 E2E tests with 2 retries. Before 1.61, video: 'on' produced 1200 videos per CI run (200 tests × 3 browsers × 2 retries). With video: 'retain-on-failure-and-retries', they get 0 videos on a green run, and only 1 video per actually-failing test (the final failing attempt). Artifact storage dropped from 50 GB to 200 MB per run.

🧒 Explain Like I'm 10

Imagine you’re filming a soccer game. Before, you filmed every game, every player, every minute — even the boring parts. You had so much footage you couldn’t find the goals.

Now, you only film when something goes wrong (a foul, a missed goal), and you only keep the last clip of each incident. One clip per problem, easy to find.

🎓 Professional Explanation

Playwright’s video recording has always supported modes like 'on' (record every test), 'retain-on-failure' (keep only failing tests’ videos), and 'off'. But these modes didn’t account for retries — if a test failed on attempt 1 and passed on attempt 2 (retry), 'retain-on-failure' kept the video from attempt 1 even though the test ultimately passed.

Playwright 1.61 adds three new modes that integrate with the retry system: 'on-all-retries' (record every retry attempt), 'retain-on-first-failure' (keep only the first failing attempt), and 'retain-on-failure-and-retries' (keep only the final failing attempt if all retries fail). These match how teams actually triage failures.

📊 The Flow

⚙️ Video Modes & expect.soft.poll

Configure video retention in playwright.config.ts via use.video. The new modes mirror the existing trace modes, so you can use the same value for both.

class=class="tk-str">"tk-com">// playwright.config.ts
import { defineConfig } from class="tk-str">'@playwright/test';

export default defineConfig({
  use: {
    class=class="tk-str">"tk-com">// Keep video only for the final failing run (after all retries)
    video: class="tk-str">'retain-on-failure-and-retries',

    class=class="tk-str">"tk-com">// Match trace to the same mode for consistent debugging
    trace: class="tk-str">'retain-on-failure-and-retries',
  },
  retries: 2,
});

class=class="tk-str">"tk-com">// All video modes (1.61+)
video: class="tk-str">'off'                           class=class="tk-str">"tk-com">// No video
video: class="tk-str">'on'                            class=class="tk-str">"tk-com">// Every test, every attempt
video: class="tk-str">'retain-on-failure'             class=class="tk-str">"tk-com">// Failing attempts only (any retry that fails)
video: class="tk-str">'on-first-failure'              class=class="tk-str">"tk-com">// First failing attempt only
video: class="tk-str">'on-all-retries'                class=class="tk-str">"tk-com">// Every retry attempt (new in 1.61)
video: class="tk-str">'retain-on-first-failure'       class=class="tk-str">"tk-com">// First failing attempt, skip retries (new in 1.61)
video: class="tk-str">'retain-on-failure-and-retries' class=class="tk-str">"tk-com">// Final failing attempt only (new in 1.61)

class=class="tk-str">"tk-com">// expect.soft.poll — non-fatal polling assertion (1.61+)
import { test, expect } from class="tk-str">'@playwright/test';

test(class="tk-str">'collect multiple soft failures', async ({ page }) => {
  await page.goto(class="tk-str">'https:class="tk-com">//app.example.com/dashboard');

  class=class="tk-str">"tk-com">// Soft assertions never throw — they collect failures
  class=class="tk-str">"tk-com">// The test continues, and all failures are reported at the end
  await expect.soft.poll(() =>
    page.getByTestId(class="tk-str">'queue-length').textContent()
  ).toContain(class="tk-str">'0');

  await expect.soft.poll(() =>
    page.getByTestId(class="tk-str">'user-count').textContent()
  ).toBeGreaterThan(class="tk-str">'100');

  class=class="tk-str">"tk-com">// If both fail, you see BOTH failures in the report
  class=class="tk-str">"tk-com">// (regular expect would stop at the first failure)
});

🔄 Before vs After

Before: retain-on-failure with retries (1.60 and earlier)

class=class="tk-str">"tk-com">// Keeps video from EVERY failing attempt — even if retry passes
export default defineConfig({
  use: {
    video: class="tk-str">'retain-on-failure',
  },
  retries: 2,
});

class=class="tk-str">"tk-com">// Test fails attempt 1, passes attempt 2
class=class="tk-str">"tk-com">// Result: 1 video kept (from attempt 1) even though test ultimately passed

After: retain-on-failure-and-retries (1.61+)

import { defineConfig } from class="tk-str">'@playwright/test';

export default defineConfig({
  use: {
    video: class="tk-str">'retain-on-failure-and-retries',
    trace: class="tk-str">'retain-on-failure-and-retries',
  },
  retries: 2,
});

class=class="tk-str">"tk-com">// Test fails attempt 1, passes attempt 2
class=class="tk-str">"tk-com">// Result: 0 videos (test ultimately passed)

class=class="tk-str">"tk-com">// Test fails all 3 attempts
class=class="tk-str">"tk-com">// Result: 1 video (final attempt only)

⚠️ Common Mistakes

Mistake 1: Using 'on' in CI

video: 'on' records every test, every attempt. For 200 tests × 3 browsers × 2 retries, that’s 1200 videos — 50+ GB. Always use a retain-* mode in CI.

Mistake 2: Mismatching video and trace modes

If video: 'retain-on-failure-and-retries' but trace: 'on', you get traces for every test but videos only for failures. Mismatched artifacts make debugging harder. Use the same mode for both.

Mistake 3: Forgetting retention-days on artifacts

Even with smart video modes, failing tests accumulate. Without retention-days on your upload-artifact step, videos stay forever. Set 7–14 days.

✅ Best Practices

1. Use 'retain-on-failure-and-retries' as the default — one video per actually-failing test.
2. Match video and trace modes for consistent debugging artifacts.
3. Set retention-days: 7–14 on artifact uploads to avoid storage bloat.
4. Use expect.soft.poll for exploratory tests where you want to see every failure in one run.
5. Use regular expect for CI gates where you want fast-fail behavior.

🏗️ Senior Engineer Deep Dive

When to use expect.soft.poll vs expect.poll

expect.poll(fn) throws on failure — the test stops. expect.soft.poll(fn) collects failures — the test continues. Use soft.poll in exploratory test runs (you want to see every problem in one pass), use regular poll in CI gates (you want fast feedback). Soft assertions are also useful in teardown: assert cleanup happened, but don’t fail the test if it didn’t.

Video vs screencast

Video recording (use.video) produces a .webm file of the full test. The screencast API (1.59+) gives you programmatic control with start(), stop(), showActions(), and showChapter() for annotations. Use video for CI forensics (set-and-forget), use screencast for narrated demos and bug reports (you control exactly what gets recorded).

💼 Interview Questions

🏋️ Practical Exercise

🚀 Mini Project

📝 Quiz

❓ FAQ

Q: Can I use different video modes per project?

A: Yes. Set video in each project’s use block. For example, your PR project can use 'retain-on-failure-and-retries' while your nightly flaky-hunt project uses 'on-all-retries' for full forensics.

Q: Does expect.soft.poll respect the test timeout?

A: Yes. The polling continues until the assertion passes or the test timeout is reached. If the test times out, the soft failure is recorded as ‘timed out’ in the report.

Q: Can I mix soft and regular expects in the same test?

A: Yes. Regular expect throws immediately on failure (stops the test). expect.soft collects failures and continues. If a regular expect fails, the test stops and any soft failures collected so far are also reported.

📦 Summary