Soft & Custom Assertions — Beyond the Basics
Hard assertions stop at the first failure. Soft assertions collect every failure. Custom assertions speak your team's language. Master these three tools and your tests become diagnostic instruments, not just pass/fail machines.
📖 The Story: The Test That Stopped Too Early
Priya was a senior SDET at a healthcare startup. She had built a comprehensive patient dashboard test that verified twelve different widgets: blood pressure chart, medication list, allergies panel, lab results, appointment history, insurance card, primary care physician, emergency contacts, vaccination record, recent visits, billing summary, and a notes section.
One Monday morning, a deployment broke three of those widgets: the allergies panel, the lab results, and the vaccination record. But Priya's test reported only one failure — the allergies panel. The other two broken widgets went undetected for nine days until a nurse filed a bug report.
The problem? Every assertion in Priya's test was a hard assertion. When the first one failed, Playwright threw an error and stopped executing the test. The remaining nine assertions never ran. The test was technically "correct" — it had caught a bug — but it had missed two other bugs sitting in the same screen.
Priya rewrote the test using expect.soft() for the non-critical visual checks. The next time the same deployment ran, the test reported all three failures in a single run. The developers fixed all three bugs before lunch.
Sometimes one failure is not the whole story. Soft assertions let your tests tell the complete story.
🎯 Why Learn Soft & Custom Assertions?
See All Failures
Soft assertions collect every failure in a single test run — no more fix-one-rerun-fix-one loops.
Retry Anything
expect.poll() adds auto-retry to API calls, localStorage, and computed values.
Wait for Complex State
expect.toPass() waits for multi-step conditions no single assertion can express.
Speak Your Domain
Custom assertion helpers let your tests read like product specifications.
🌍 Real World Example
Imagine an e-commerce checkout page where you need to verify:
- Cart total matches the sum of item prices
- Discount code applied correctly
- Tax calculated based on shipping address
- Shipping method label is correct
- Order summary section is visible
- Payment method dropdown is enabled
- Coupon success message displayed
- "Place Order" button is enabled
If you use hard assertions and the tax calculation is wrong, the test stops and never checks the shipping label, payment dropdown, or place-order button. You fix the tax bug, rerun, and discover the button is also broken. Three deployment cycles later, you've finally verified everything.
With soft assertions, you see all eight failures in one report. You fix them all before the next deploy.
🧒 Explain Like I'm 10
Imagine a teacher grading your homework. The teacher has two ways to grade:
Hard grading: The teacher finds the first wrong answer, marks it with a red X, hands the paper back, and says "fix this and resubmit." You fix it, hand it back, the teacher finds the NEXT wrong answer, marks it, hands it back. This goes on for days. Each round, you discover only one mistake.
Soft grading: The teacher reads the entire paper, marks EVERY wrong answer with a red X, and hands it back with a list of all mistakes. You fix everything in one sitting.
Soft assertions are the second teacher. They check everything, collect all the mistakes, and hand you a complete list at the end. The hard teacher (regular expect) stops at the first mistake.
Both teachers care about quality. But the soft teacher respects your time.
🎓 Professional Explanation
Playwright ships four assertion modes:
- Web-first assertions (
expect(locator).toBeVisible()) — auto-retry, throw on failure, stop test. - Soft assertions (
expect.soft(locator).toBeVisible()) — auto-retry, collect failure, continue test, throw aggregate at end. - Polling assertions (
expect.poll(() => value).toBe(x)) — auto-retry on arbitrary functions, throw on failure. - Pass conditions (
await expect.toPass(async () => {...})) — retry an async block until it stops throwing.
Soft assertions maintain an internal error list attached to the current test context. When the test function returns, Playwright inspects that list and throws a single aggregated Error containing every collected failure with its stack trace. This is why soft failures still mark the test as failed in reports — they just delay the throw.
⚖️ Hard vs Soft: When to Use Each
| Aspect | Hard Assertion | Soft Assertion |
|---|---|---|
| On failure | Stops test immediately | Continues, collects error |
| Auto-retry | ✅ Yes (locator assertions) | ✅ Yes (locator assertions) |
| Final test result | Failed if assertion fails | Failed if any soft assertion fails |
| Use case | Critical path checks | Non-critical, multi-element verification |
| Example | Login button enabled | Logo, footer, header visible |
| Downside | Hides failures after the first one | Slower (runs full test even if early failure) |
Hard Assertion Flow vs Soft Assertion Flow
🔴 Hard Assertion
🟢 Soft Assertion
🛡️ expect.soft() — The Multi-Failure Collector
Syntax is identical to regular expect — just add .soft:
import { test, expect } from '@playwright/test'; test('dashboard renders all critical widgets', async ({ page }) => { await page.goto('/dashboard'); // Critical: hard assertion — if this fails, nothing else matters await expect(page.getByRole('heading', { name: 'Patient Dashboard' })).toBeVisible(); // Non-critical: soft assertions — collect failures, keep going await expect.soft(page.getByTestId('blood-pressure-chart')).toBeVisible(); await expect.soft(page.getByTestId('medication-list')).toBeVisible(); await expect.soft(page.getByTestId('allergies-panel')).toContainText('Penicillin'); await expect.soft(page.getByTestId('lab-results')).toBeVisible(); await expect.soft(page.getByTestId('vaccination-record')).toBeVisible); // If any soft assertion failed, the test fails HERE at the end // with a combined error message listing every failure. });
Pattern: Always start with one hard assertion that verifies the page actually loaded (heading, URL, or key element). Then use soft assertions for the secondary checks. This way, if the entire page fails to load, you don't get 12 noisy soft failures — you get one clear hard failure.
Reading Soft Assertion Failures
When soft assertions fail, the report shows all failures together:
Error: 3 soft assertions failed:
1) [expect.soft] Locator.getByTestId('allergies-panel') toContainText "Penicillin"
Expected: "Penicillin"
Received: ""
62 | await expect.soft(page.getByTestId('allergies-panel')).toContainText('Penicillin');
2) [expect.soft] Locator.getByTestId('lab-results') toBeVisible
Call log:
- waiting for getByTestId('lab-results')
- timeout 5000ms exceeded
63 | await expect.soft(page.getByTestId('lab-results')).toBeVisible();
3) [expect.soft] Locator.getByTestId('vaccination-record') toBeVisible
Call log:
- waiting for getByTestId('vaccination-record')
64 | await expect.soft(page.getByTestId('vaccination-record')).toBeVisible();
Each failure includes the locator, expected value, received value, line number, and (for visibility assertions) the retry call log. You see all three bugs in one test run instead of three separate runs.
⏳ expect.poll() — Auto-Retry for Non-Locator Values
Web-first assertions auto-retry because locators are live queries against the DOM. But what about values that are not locators — API responses, localStorage entries, or values you compute from multiple DOM reads? Regular expect checks them once and fails immediately if they're not ready.
expect.poll() adds the same retry behavior to any function:
// ❌ No retry — fails if API hasn't updated yet const count = await request.get('/api/cart/count').then(r => r.json()); expect(count).toBe(3); // ✅ Auto-retry — polls until count is 3 or timeout expires await expect.poll(async () => { const resp = await request.get('/api/cart/count'); const { count } = await resp.json(); return count; }).toBe(3);
Polling localStorage
// Wait until token appears in localStorage await expect.poll(() => page.evaluate(() => { return localStorage.getItem('auth_token'); })).toBeTruthy(); // Poll a computed value from multiple elements await expect.poll(async () => { const prices = await page.getByTestId('price').allInnerTexts(); return prices.reduce((sum, p) => sum + parseFloat(p.replace('$', '')), 0); }).toBe(125.97);
Why this matters: Without expect.poll, you'd have to write a manual while loop with setTimeout, which is fragile and unergonomic. expect.poll gives you the same retry semantics as locator assertions for any value.
Configuring Polling Intervals
// Override default polling config per assertion await expect.poll(() => getApiCount(), { timeout: 10000, // total time to wait (default: 5000ms) intervals: [100, 500, 1000, 2000] // delay between polls }).toBe(3);
🔁 expect.toPass() — Retry Arbitrary Async Blocks
expect.toPass() wraps an async function and retries it until it stops throwing or times out. Use it for conditions that involve multiple steps or checks that must all succeed together.
// Wait until cart has exactly 3 items AND total is $50 await expect.toPass(async () => { const items = await page.getByTestId('cart-item').count(); const total = await page.getByTestId('cart-total').textContent(); expect(items).toBe(3); expect(total).toContain('$50'); }); // Configure timeout and intervals await expect.toPass(async () => { // multi-step verification logic }, { timeout: 15000, intervals: [0, 100, 250, 500] });
expect.poll vs expect.toPass: Use poll when you have a single value that should match a condition (returns a value, assert with matcher). Use toPass when you have a block containing multiple assertions that must all succeed together. poll is value-based; toPass is block-based.
💬 Custom Assertion Messages
Every Playwright assertion accepts a second argument — a custom message that appears in failure reports. This is the single most underused feature in Playwright.
// Without custom message — cryptic failure await expect(page.getByTestId('cart-total')).toHaveText('$50.00'); // Failure: "Locator expected to have text '$50.00' but got '$45.00'" // With custom message — instantly diagnosable await expect( page.getByTestId('cart-total'), 'Cart total should match sum of item prices after 10% discount' ).toHaveText('$50.00'); // Failure: "Cart total should match sum of item prices after 10% discount" // "Locator expected to have text '$50.00' but got '$45.00'"
Why this is powerful: In CI logs, you see the business reason for the assertion before the technical details. A junior developer reading the failure immediately knows what feature is broken, not just that "some text didn't match."
🧩 Building Custom Assertion Helpers
For domain-specific checks you perform repeatedly, wrap them in helper functions. This turns your tests into readable product specifications.
import { Page, expect } from '@playwright/test'; export async function expectCartToContain(page: Page, items: string[]) { for (const item of items) { await expect.soft( page.getByTestId('cart-item'), `Cart should contain: ${item}` ).toContainText(item); } } export async function expectCartTotal(page: Page, expectedTotal: number) { const totalText = await page.getByTestId('cart-total').textContent(); const actualTotal = parseFloat(totalText!.replace(/[^0-9.]/g, '')); expect( actualTotal, `Cart total should be $${expectedTotal.toFixed(2)} (calculated from item prices + tax)` ).toBe(expectedTotal); } export async function expectUserToBeLoggedIn(page: Page, userName: string) { await expect(page.getByTestId('user-avatar')).toBeVisible(); await expect(page.getByTestId('user-name')).toHaveText(userName); await expect(page.getByTestId('login-button')).toBeHidden(); }
Using the Helpers
import { test } from '@playwright/test'; import { expectCartToContain, expectCartTotal, expectUserToBeLoggedIn } from '../helpers/cart-assertions'; test('checkout flow with multiple items', async ({ page }) => { await page.goto('/cart'); await expectUserToBeLoggedIn(page, 'Alex Doe'); await expectCartToContain(page, ['Laptop', 'Mouse', 'Keyboard']); await expectCartTotal(page, 125.97); });
Notice how the test reads like a product requirement: "user is logged in as Alex Doe, cart contains Laptop/Mouse/Keyboard, total is $125.97." That's the power of custom helpers — they hide implementation details and reveal intent.
📊 Visualizing the Polling Mechanism
How expect.poll() Retries Over Time
Default: 5 second timeout, polling every ~100ms initially, backing off exponentially
❌ 0
❌ 0
❌ 1
❌ 2
❌ 2
✅ 3
If the value never reaches the target, the final tick becomes a timeout failure at 5000ms.
⚠️ Common Mistakes
Mistake 1: Using Soft Assertions for Critical Checks
// ❌ Login button visibility is critical — using soft is dangerous await expect.soft(page.getByRole('button', { name: 'Login' })).toBeVisible); // ✅ Use hard assertion for critical path elements await expect(page.getByRole('button', { name: 'Login' })).toBeVisible();
Rule of thumb: if the test result is meaningless without this assertion passing, use a hard assertion. Use soft only for "nice to verify" checks.
Mistake 2: Forgetting to Await Soft Assertions
// ❌ Missing await — soft assertion never executes expect.soft(page.getByText('Welcome')).toBeVisible(); // ✅ Always await, even for soft assertions await expect.soft(page.getByText('Welcome')).toBeVisible();
Mistake 3: Using expect.poll with Side Effects
// ❌ Polling function mutates state — results are unpredictable let retries = 0; await expect.poll(() => ++retries).toBe(5); // ✅ Polling function should be pure — read state, don't change it await expect.poll(() => page.getByTestId('count').textContent()).toBe('5');
Mistake 4: Mixing Soft and Hard for the Same Element
// ❌ First soft, then hard on same element — confusing semantics await expect.soft(page.getByTestId('total')).toHaveText('$50'); await expect(page.getByTestId('total')).toHaveText('$50'); // ✅ Pick one strategy per element — soft OR hard, not both
✅ Best Practices
- Always start with one hard assertion that verifies the page loaded before any soft assertions. This catches "page didn't load at all" cleanly.
- Use soft assertions for non-critical visual checks: logos, headers, footers, secondary widgets.
- Use hard assertions for the critical path: login button, submit button, primary content, success messages.
- Always add custom messages to assertions in business-critical tests. The CI log will thank you.
- Build domain-specific helpers for repeated multi-step checks. They make tests read like specifications.
- Use
expect.pollfor any non-locator value that might change over time (API responses, localStorage, computed values). - Use
expect.toPassfor multi-assertion conditions that must all succeed together. - Keep polling functions pure — no side effects, no mutation of outer state.
- Configure longer timeouts for slow operations like file uploads or third-party API calls.
- Group related soft assertions with comments indicating what group they verify.
⚡ Performance Tips
- Soft assertions extend test duration because the test keeps running after a failure. Don't use them in performance-sensitive smoke tests.
- Polling intervals matter. For fast-changing values, use short intervals (
[100, 250, 500]). For slow APIs, use longer ones ([1000, 2000, 5000]). - Avoid polling in parallel tests if the polled resource has rate limits (e.g., public APIs).
- Use
expect.toPasssparingly — it can mask real bugs if the timeout is too long. - Cache DOM reads inside
expect.pollcallbacks — don't re-query the same locator five times.
Run soft-assertion-heavy tests in a separate projects block in playwright.config.ts with longer timeouts. This prevents them from slowing down your fast smoke tests in CI.
🐞 Debugging Tips
- Soft failures appear at the END of the test. If a test fails with three soft errors but no hard errors, the failure point is the last line of the test — that's where Playwright aggregates the throw.
- Use Playwright Trace Viewer to see exactly when each soft assertion was evaluated. Each assertion creates a snapshot in the trace.
- For
expect.pollfailures, the error message includes the last polled value. If it showsundefined, your polling function is likely throwing before returning. - For
expect.toPassfailures, the error shows which assertion inside the block failed last. Wrap each assertion in try/catch to get individual failure details. - Run with
--last-failedafter fixing soft failures to verify your fixes without rerunning the whole suite.
🏗️ Senior Engineer Deep Dive
How Soft Assertions Work Internally
When you call expect.soft(), Playwright does NOT throw immediately on failure. Instead:
- The matcher evaluates the condition (with auto-retry for locators).
- If it fails, the error object (with stack trace, locator snapshot, expected/received values) is pushed to an array stored on the current
TestInfoinstance. - Execution continues to the next statement.
- When the test function returns (or a hard assertion fails), Playwright checks the soft-failures array.
- If the array is non-empty, a single aggregated
Erroris thrown containing all collected failures.
This is why soft assertions still mark the test as failed in HTML reports — they delay the throw, they don't suppress it.
Why expect.poll Exists Separately from Web-First Assertions
Web-first assertions are deeply integrated with Playwright's locator engine — they can wait for actionability checks, retry on DOM mutations, and capture element snapshots. expect.poll is a generic retry wrapper around any function. The two systems share the same polling infrastructure but serve different data sources: locators (live DOM queries) vs arbitrary functions (API calls, computations, etc.).
Trade-offs: Soft Assertions vs Multiple Tests
An alternative to soft assertions is splitting one test into multiple smaller tests, each with one hard assertion. This approach gives you isolated failures and parallel execution, but loses the "single page state" context. Soft assertions win when the cost of setting up the test (login, navigation, data prep) is high relative to the assertions themselves.
Enterprise Pattern: Assertion Libraries as Product Specs
At scale, mature QA teams build assertion helper libraries that mirror their product's domain language. Examples: expectOrderToBeConfirmed(), expectUserToHaveRole(), expectInventoryToBeSynced(). These helpers compose locators, API calls, and soft assertions internally, exposing a clean interface to test authors. The result: tests read like product requirement documents, and product managers can review them.
Memory Implications
Soft assertions store error objects with full stack traces and DOM snapshots. In tests with hundreds of soft assertions, this can consume meaningful memory. Playwright caps the soft-failure collection at a reasonable size, but for pathological cases, consider splitting into multiple tests.
💼 Interview Questions
Show Answer
expect.soft()) checks a condition but does not stop the test on failure. All failures are collected and reported together at the end of the test. Use soft assertions for non-critical checks like logos, footers, or secondary widgets where you want to see all failures in one run, not just the first one.Show Answer
expect.poll and a regular expect on a non-locator value?Show Answer
expect checks the value once and fails immediately if it doesn't match. expect.poll wraps a function and retries it on a polling interval until the value matches or the timeout expires. Use expect.poll for values that may change over time (API responses, localStorage, computed DOM values) and regular expect for static values.expect.toPass over expect.poll?Show Answer
expect.poll when you have a single value to assert against a matcher (it returns a value, then you call a matcher on it). Use expect.toPass when you have a block of code containing multiple assertions that must all succeed together, or when your condition involves side effects like clicks or navigation that can't be expressed as a pure value return.Show Answer
expect.soft() call evaluates the matcher; on failure, the error object (with stack trace, snapshots, expected/received values) is pushed to an array stored on the current TestInfo. When the test function returns, Playwright inspects this array. If non-empty, it throws a single aggregated Error listing all collected failures with their individual details and line numbers. The aggregation happens at the end of test execution, not at assertion time.Show Answer
Show Answer
expect.soft() while keeping a hard assertion on the dashboard's main heading (to verify the page loaded). This way, if any of the 15 elements fail, all failures are reported in one test run. Add custom messages to each soft assertion explaining what each element represents, so developers immediately know which feature is broken without reading the test code.Show Answer
expect.toPass with a 10-second timeout:
await expect.toPass(async () => {
const resp = await request.get('/api/status');
expect(resp.status()).toBe(200);
const body = await resp.json();
expect(body.message).toContain('success');
}, { timeout: 10000 });
This retries the entire block (request + both assertions) until both pass or the timeout expires.
🏋️ Practical Exercise
Refactor a Hard-Assertion Test to Use Soft Assertions
Take any existing Playwright test you've written that has 5+ assertions. Refactor it using these rules:
- Keep the FIRST assertion as a hard assertion (the "page loaded" check).
- Convert all subsequent assertions to
expect.soft(). - Add a custom message to each soft assertion explaining what feature it verifies.
- Group related assertions with comments (e.g.,
// Header section,// Cart section). - Run the test against a working site — it should pass.
- Temporarily break 3 different elements (via DevTools or by modifying selectors) and rerun. Confirm all 3 failures appear in one report.
Success criteria: When you break multiple elements, you see all failures in a single test run, each with a clear custom message explaining what's broken.
🚀 Mini Project
🏗️ Build a Domain-Specific Assertion Library
Create a file helpers/ecomm-assertions.ts with these custom helpers:
expectProductInCart(page, productName)— soft assertion that verifies the product appears in the cart.expectCartSubtotal(page, expectedAmount)— hard assertion that verifies subtotal math.expectDiscountApplied(page, discountCode)— soft assertion that verifies discount badge is visible with correct code.expectCheckoutButtonEnabled(page)— hard assertion that the checkout button is enabled.expectOrderSummaryComplete(page, { items, subtotal, tax, total })— usesexpect.toPassto wait until all order summary fields are populated correctly.
Then write a single test checkout.spec.ts that uses all five helpers in sequence. The test should read like a checkout specification document, not like implementation code.
Bonus: Add JSDoc comments to each helper explaining when to use it. Generate a TypeDoc report and review whether a non-technical product manager could understand the test.
📝 Quiz
Test your understanding. Click an option to check your answer.
expect.soft() assertion fails?expect(api).toBe(200)expect.soft(api).toBe(200)expect.poll(async () => await getStatus()).toBe(200)expect(api).toPass(200)expect.poll wraps a function and retries it on a polling interval, perfect for retrying API calls until they return the expected value.expect.poll and expect.toPass?poll is for numbers, toPass is for stringspoll returns a value to assert on; toPass retries a block of multiple assertionstoPass is faster than pollexpect.poll(fn).toBe(x) polls a function and matches its return value. expect.toPass(async () => {...}) retries an async block containing one or more assertions until they all stop throwing.expect.soft() for the critical login button visibility check?await a soft assertion?await, the assertion promise is created but never resolved. The test passes regardless of the actual condition — a silent failure that's extremely dangerous.❓ FAQ
Q: Can I use soft assertions inside a loop?
A: Yes. Each soft assertion inside a loop is collected independently. This is useful for verifying a list of elements where you want to see ALL missing items, not just the first one. Just be careful with large loops — hundreds of soft failures can make the report unreadable.
Q: Do soft assertions work with expect.poll?
A: No direct expect.soft.poll() syntax. If you need polling with soft behavior, wrap the poll inside a try/catch and use expect.soft on a value you extract manually, or use expect.toPass with internal try/catch blocks.
Q: Can I customize the polling intervals for expect.poll?
A: Yes. Pass a second argument with intervals array and timeout:
{ intervals: [100, 500, 1000], timeout: 10000 }. The default intervals are [100, 250, 500, 1000].
Q: Are soft assertions supported in Playwright Test Runner only, or also in standalone Playwright?
A: Soft assertions are part of @playwright/test's expect. They require the test runner context (TestInfo) to store collected failures. If you're using Playwright as a library without the test runner, soft assertions won't aggregate properly.
Q: What's the maximum number of soft assertions I should use in one test?
A: There's no hard limit, but as a guideline, if you're exceeding 20 soft assertions in one test, consider splitting the test. Large numbers of soft failures become hard to triage and the test becomes a "kitchen sink" that's hard to maintain.
Q: Can I convert a soft assertion failure into a warning instead of a test failure?
A: Not directly. Soft assertions always mark the test as failed if any of them fail. If you want pure warnings, log them manually with console.warn or use test.info().annotations to attach annotations to the test report without failing.
📦 Summary
🎯 Key Takeaways
- Soft assertions (
expect.soft) collect failures and report them all at the end — perfect for non-critical multi-element checks. - Hard assertions stop the test on first failure — use for critical-path elements like login buttons and submit buttons.
expect.polladds auto-retry to non-locator values — essential for API responses, localStorage, and computed values.expect.toPassretries a block of multiple assertions — use for complex multi-step conditions.- Custom messages (second argument to any assertion) make CI logs instantly diagnosable.
- Custom helper functions turn tests into readable product specifications.
- Always start tests with one hard assertion verifying page load, then use soft assertions for secondary checks.
- Always
awaitsoft assertions — fire-and-forget soft assertions silently pass.
