Trace Viewer, Debugging & Visual Testing

🕒 Teaching Flow

1. Story Intro (5 min)

👉 "Even the best tests fail sometimes. Instead of guessing why, Playwright gives us Trace Viewer, screenshots, and videos to replay failures. Once tests are stable, we can go further into Visual Testing — catching UI layout or style bugs that assertions can't."

Debugging Analogy:
Think of Trace Viewer like a security camera that records everything happening in your test. When something goes wrong, you can rewind and see exactly what happened, step by step. Visual testing is like having a photo comparison tool that catches even the smallest visual changes that might break your UI.

Trace Viewer & Debugging (40 min)

Enable Trace in Config

📂 playwright.config.js

import { defineConfig } from "@playwright/test";

export default defineConfig({
  use: {
    // Trace recording options
    trace: "on-first-retry", // record trace only on retries
    screenshot: "only-on-failure", // capture screenshots on failures
    video: "retain-on-failure", // record videos on failures
    
    // Additional debugging options
    headless: false, // run in headed mode for debugging
    slowMo: 1000, // slow down operations by 1 second
  },
  
  // Global test configuration
  retries: 2, // retry failed tests twice
  timeout: 30000, // 30 second timeout per test
  
  // Reporter configuration
  reporter: [
    ['html'], // HTML report
    ['json', { outputFile: 'test-results.json' }], // JSON report
  ],
});

Trace Recording Options:

"on": Record trace for every test run
"on-first-retry": Record only when test retries (recommended)
"retain-on-failure": Keep trace files only for failed tests
"off": Disable trace recording

Fail a Test Intentionally

📂 tests/trace-fail.spec.js

import { test, expect } from "@playwright/test";

test("Intentional Failure for Trace", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  
  // Wait for page to load completely
  await page.waitForLoadState('networkidle');
  
  // Wrong locator to force failure
  const bool = await page.locator("h1:has-text('NonExistent')").isVisible();
  expect(bool).toBeTruthy();
});

test("Login Failure with Trace", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  
  // Navigate to login page
  await page.getByRole('link', { name: 'Login' }).click();
  await page.waitForLoadState('networkidle');
  
  // Fill incorrect credentials
  await page.fill('#email', 'wrong@email.com');
  await page.fill('#password', 'wrongpassword');
  
  // Click login button
  await page.getByRole('button', { name: 'Login' }).click();
  
  // This will fail and generate trace
  await expect(page.locator('.success-message')).toBeVisible();
});

test("Network Failure Simulation", async ({ page }) => {
  // Simulate network failure
  await page.route('**/*', route => route.abort());
  
  await page.goto("https://learn-playwright.great-site.net/");
  
  // This will fail due to network issues
  await expect(page.locator('h1')).toBeVisible();
});

Run with Trace Recording

# Run tests with trace recording
npx playwright test --trace on

# Run specific test with trace
npx playwright test tests/trace-fail.spec.js --trace on

# Run with video recording
npx playwright test --video on

# Run with all debugging options
npx playwright test --trace on --video on --headed

Open and Analyze Trace

# Open trace viewer
npx playwright show-trace test-results/trace.zip

# Open specific trace file
npx playwright show-trace test-results/chromium/trace-fail.spec.js-trace.zip

# Open HTML report
npx playwright show-report

                    What to Show in Trace Viewer:
                    Steps Timeline: See every click, input, and navigation
DOM Snapshot: View page state at failure point
Network Requests: Monitor API calls and responses
Console Logs: Check for JavaScript errors
Performance Metrics: Analyze page load times

                

Advanced Debugging Techniques

📂 tests/advanced-debugging.spec.js

import { test, expect } from "@playwright/test";

test("Debug with Console Logs", async ({ page }) => {
  // Listen to console messages
  page.on('console', msg => console.log('PAGE LOG:', msg.text()));
  
  // Listen to network requests
  page.on('request', request => console.log('REQUEST:', request.url()));
  page.on('response', response => console.log('RESPONSE:', response.url(), response.status()));
  
  await page.goto("https://learn-playwright.great-site.net/");
  
  // Add custom logging
  console.log('Page title:', await page.title());
  console.log('Current URL:', page.url());
  
  // Take screenshot at specific point
  await page.screenshot({ path: 'debug-screenshot.png' });
  
  // Pause execution for manual inspection
  await page.pause();
});

test("Debug with Network Interception", async ({ page }) => {
  // Intercept and modify network requests
  await page.route('**/api/**', async route => {
    console.log('Intercepted API call:', route.request().url());
    
    // Mock response
    await route.fulfill({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify({ success: true, data: 'mocked' })
    });
  });
  
  await page.goto("https://learn-playwright.great-site.net/");
  
  // Your test logic here
  await expect(page.locator('h1')).toBeVisible();
});

test("Debug with Element Inspection", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  
  // Get element information
  const element = page.locator('h1');
  const text = await element.textContent();
  const isVisible = await element.isVisible();
  const boundingBox = await element.boundingBox();
  
  console.log('Element text:', text);
  console.log('Is visible:', isVisible);
  console.log('Bounding box:', boundingBox);
  
  // Highlight element
  await element.highlight();
  
  // Wait for manual inspection
  await page.pause();
});

Visual Testing (40 min)

Add Visual Snapshot Assertion

📂 tests/visual-home.spec.js

import { test, expect } from "@playwright/test";

test("Visual Regression - Homepage", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  
  // Wait for page to fully load
  await page.waitForLoadState('networkidle');
  
  // Take full page screenshot
  await expect(page).toHaveScreenshot("homepage.png");
});

test("Visual Regression - Homepage Header Only", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  await page.waitForLoadState('networkidle');
  
  // Take screenshot of specific element
  const header = page.locator('header');
  await expect(header).toHaveScreenshot("homepage-header.png");
});

test("Visual Regression - Mobile View", async ({ page }) => {
  // Set mobile viewport
  await page.setViewportSize({ width: 375, height: 667 });
  
  await page.goto("https://learn-playwright.great-site.net/");
  await page.waitForLoadState('networkidle');
  
  await expect(page).toHaveScreenshot("homepage-mobile.png");
});

Visual Regression for Product Page

📂 tests/visual-product.spec.js

import { test, expect } from "@playwright/test";

test("Visual Regression - Product Page", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  
  // Navigate to product page
  await page.getByRole("link", { name: "Clothes" }).click();
  await page.waitForLoadState('networkidle');
  
  // Take screenshot of product page
  await expect(page).toHaveScreenshot("product-page.png");
});

test("Visual Regression - Product Grid", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  await page.getByRole("link", { name: "Clothes" }).click();
  await page.waitForLoadState('networkidle');
  
  // Take screenshot of product grid only
  const productGrid = page.locator('.product-grid');
  await expect(productGrid).toHaveScreenshot("product-grid.png");
});

test("Visual Regression - Product Details", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  await page.getByRole("link", { name: "Clothes" }).click();
  await page.waitForLoadState('networkidle');
  
  // Click on first product
  await page.locator('.product-item').first().click();
  await page.waitForLoadState('networkidle');
  
  // Take screenshot of product details
  await expect(page).toHaveScreenshot("product-details.png");
});

Advanced Visual Testing

📂 tests/visual-advanced.spec.js

import { test, expect } from "@playwright/test";

test("Visual Regression - Multiple Viewports", async ({ page }) => {
  const viewports = [
    { width: 1920, height: 1080, name: 'desktop' },
    { width: 1024, height: 768, name: 'tablet' },
    { width: 375, height: 667, name: 'mobile' }
  ];
  
  for (const viewport of viewports) {
    await page.setViewportSize({ width: viewport.width, height: viewport.height });
    await page.goto("https://learn-playwright.great-site.net/");
    await page.waitForLoadState('networkidle');
    
    await expect(page).toHaveScreenshot(`homepage-${viewport.name}.png`);
  }
});

test("Visual Regression - Different Browsers", async ({ page, browserName }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  await page.waitForLoadState('networkidle');
  
  // Take browser-specific screenshots
  await expect(page).toHaveScreenshot(`homepage-${browserName}.png`);
});

test("Visual Regression - With Animations Disabled", async ({ page }) => {
  // Disable animations for consistent screenshots
  await page.addStyleTag({
    content: `
      *, *::before, *::after {
        animation-duration: 0s !important;
        animation-delay: 0s !important;
        transition-duration: 0s !important;
        transition-delay: 0s !important;
      }
    `
  });
  
  await page.goto("https://learn-playwright.great-site.net/");
  await page.waitForLoadState('networkidle');
  
  await expect(page).toHaveScreenshot("homepage-no-animations.png");
});

test("Visual Regression - Custom Threshold", async ({ page }) => {
  await page.goto("https://learn-playwright.great-site.net/");
  await page.waitForLoadState('networkidle');
  
  // Use custom threshold for comparison
  await expect(page).toHaveScreenshot("homepage-custom-threshold.png", {
    threshold: 0.2, // Allow 20% difference
    maxDiffPixels: 1000 // Allow up to 1000 different pixels
  });
});

Manage Snapshots

# First run - creates baseline images
npx playwright test --grep "Visual Regression"

# Update snapshots after intentional UI changes
npx playwright test --update-snapshots

# Update specific test snapshots
npx playwright test tests/visual-home.spec.js --update-snapshots

# Run visual tests only
npx playwright test --grep "Visual"

# Compare snapshots manually
npx playwright show-report

Snapshot Management Best Practices:

Baseline Creation: First run creates baseline images in __screenshots__/
Comparison: Next runs compare against baseline
Mismatch Handling: Playwright shows diff when screenshots don't match
Approval Process: Use --update-snapshots only after reviewing changes
Version Control: Commit baseline images to repository

Visual Testing Utilities

📂 utils/visual-testing.js

import { expect } from "@playwright/test";

export class VisualTestingUtils {
  static async takeFullPageScreenshot(page, name) {
    await page.waitForLoadState('networkidle');
    await expect(page).toHaveScreenshot(`${name}.png`);
  }
  
  static async takeElementScreenshot(page, selector, name) {
    const element = page.locator(selector);
    await element.waitFor();
    await expect(element).toHaveScreenshot(`${name}.png`);
  }
  
  static async takeScreenshotWithOptions(page, name, options = {}) {
    await page.waitForLoadState('networkidle');
    await expect(page).toHaveScreenshot(`${name}.png`, {
      threshold: 0.2,
      maxDiffPixels: 1000,
      ...options
    });
  }
  
  static async disableAnimations(page) {
    await page.addStyleTag({
      content: `
        *, *::before, *::after {
          animation-duration: 0s !important;
          animation-delay: 0s !important;
          transition-duration: 0s !important;
          transition-delay: 0s !important;
        }
      `
    });
  }
  
  static async hideDynamicContent(page, selectors = []) {
    const defaultSelectors = [
      '.timestamp',
      '.date',
      '.time',
      '[data-testid*="timestamp"]',
      '[data-testid*="date"]'
    ];
    
    const allSelectors = [...defaultSelectors, ...selectors];
    
    for (const selector of allSelectors) {
      await page.addStyleTag({
        content: `${selector} { visibility: hidden !important; }`
      });
    }
  }
}

🧑‍💻 Interactive Questions & Answers

1. What's the difference between `trace: "on"` vs `"on-first-retry"`?

Answer:

trace: "on": Records trace for every test run, regardless of success or failure
trace: "on-first-retry": Records trace only when a test fails and retries
Performance Impact: "on" creates larger trace files and slower execution
Storage: "on-first-retry" saves disk space by only keeping traces for failed tests
Recommended: Use "on-first-retry" for CI/CD, "on" for local debugging

Example:

// For CI/CD (recommended)
trace: "on-first-retry"

// For local debugging
trace: "on"

// For production (minimal)
trace: "retain-on-failure"

2. Why keep video/screenshot only on failure?

Answer:

Storage Efficiency: Saves disk space by not recording successful tests
Performance: Reduces test execution time for passing tests
Focus: Only captures evidence when something goes wrong
CI/CD Benefits: Reduces artifact storage costs in continuous integration
Debugging: Provides visual evidence of failure points

Example:

// Efficient configuration
screenshot: "only-on-failure",
video: "retain-on-failure"

// Alternative for debugging
screenshot: "on",
video: "on"

3. What bugs can visual testing catch that normal assertions cannot?

Answer:

Layout Issues: Broken CSS, misaligned elements, responsive design problems
Visual Regressions: Unintended style changes, color mismatches, font issues
Cross-browser Differences: Rendering inconsistencies between browsers
Responsive Design: Layout breaks at different screen sizes
Dynamic Content: Images, charts, or visual elements that change
Accessibility: Visual contrast issues, focus indicators

Example:

// Normal assertion - only checks if element exists
await expect(page.locator('.button')).toBeVisible();

// Visual testing - checks entire visual appearance
await expect(page.locator('.button')).toHaveScreenshot('button.png');

// Catches: color changes, size changes, border issues, etc.

4. What's the risk of overusing `toHaveScreenshot()`?

Answer:

False Positives: Tests fail due to minor, acceptable visual changes
Maintenance Overhead: Constant snapshot updates for every UI change
Flaky Tests: Screenshots sensitive to timing, animations, or dynamic content
Storage Costs: Large number of baseline images to maintain
Slow Execution: Screenshot comparison takes time

Best Practices:

// Good: Test critical UI components
await expect(page.locator('.checkout-form')).toHaveScreenshot();

// Bad: Test every small element
await expect(page.locator('.icon')).toHaveScreenshot();
await expect(page.locator('.tooltip')).toHaveScreenshot();

// Better: Use for key user flows
await expect(page).toHaveScreenshot('checkout-flow.png');

5. How would you set different visual baselines for different browsers/devices?

Answer:

Browser-specific Baselines: Use browserName in screenshot names
Device-specific Baselines: Use viewport size in screenshot names
Configuration: Set up different test configurations for each environment

Example:

// Browser-specific screenshots
test("Visual Regression - Different Browsers", async ({ page, browserName }) => {
  await page.goto("https://example.com");
  await expect(page).toHaveScreenshot(`homepage-${browserName}.png`);
});

// Device-specific screenshots
test("Visual Regression - Different Devices", async ({ page }) => {
  const devices = [
    { width: 1920, height: 1080, name: 'desktop' },
    { width: 768, height: 1024, name: 'tablet' },
    { width: 375, height: 667, name: 'mobile' }
  ];
  
  for (const device of devices) {
    await page.setViewportSize({ width: device.width, height: device.height });
    await page.goto("https://example.com");
    await expect(page).toHaveScreenshot(`homepage-${device.name}.png`);
  }
});

6. How do you handle dynamic content in visual testing?

Answer:

Hide Dynamic Elements: Use CSS to hide timestamps, dates, random content
Mock Data: Replace dynamic content with static test data
Wait for Stability: Wait for animations and loading to complete
Custom Thresholds: Allow for minor differences in dynamic content

Example:

// Hide dynamic content
await page.addStyleTag({
  content: `
    .timestamp, .date, .time { visibility: hidden !important; }
    .loading-spinner { display: none !important; }
  `
});

// Wait for stability
await page.waitForLoadState('networkidle');
await page.waitForTimeout(1000); // Wait for animations

// Take screenshot
await expect(page).toHaveScreenshot('stable-page.png');

7. How do you debug flaky visual tests?

Answer:

Identify Root Cause: Use trace viewer to see what's different
Check Timing: Ensure page is fully loaded before screenshot
Disable Animations: Remove CSS animations for consistent screenshots
Use Retries: Configure retries for flaky visual tests

Example:

// Debug flaky test
test("Visual Regression - Debug Flaky", async ({ page }) => {
  // Disable animations
  await page.addStyleTag({
    content: `* { animation: none !important; transition: none !important; }`
  });
  
  await page.goto("https://example.com");
  
  // Wait for complete stability
  await page.waitForLoadState('networkidle');
  await page.waitForFunction(() => document.readyState === 'complete');
  
  // Take screenshot with retry
  await expect(page).toHaveScreenshot('stable-page.png', {
    threshold: 0.1,
    maxDiffPixels: 100
  });
});

8. How do you integrate visual testing into CI/CD pipelines?

Answer:

Baseline Management: Store baseline images in version control
Artifact Storage: Save test results and diff images as build artifacts
Approval Workflow: Require manual approval for snapshot updates
Parallel Execution: Run visual tests in parallel for faster feedback

Example:

# CI/CD Pipeline Configuration
- name: Run Visual Tests
  run: |
    npx playwright test --grep "Visual" --reporter=html
    
- name: Upload Test Results
  uses: actions/upload-artifact@v3
  with:
    name: visual-test-results
    path: playwright-report/
    
- name: Update Snapshots (Manual Approval)
  if: github.event_name == 'workflow_dispatch'
  run: npx playwright test --update-snapshots

🎯 Day 9 Homework Tasks

🟢 Beginner

Task 1

Create a failing login test → open trace viewer and analyze the failure step by step.

Task 2

Add screenshot recording for failed tests only in your playwright.config.js.

🟡 Intermediate

Task 3

Add visual test for Cart page with proper wait conditions and error handling.

Task 4

Fail visual test intentionally by zooming browser → see diff image and understand the comparison.

🔴 Advanced

Task 5

Run visual test in Chromium + WebKit → observe differences and create browser-specific baselines.

Task 6

Create a custom visual test utility that captures only header section of homepage and validates snapshot with custom thresholds.

Best Practices & Tips

Debugging Best Practices:

Use trace viewer for complex failures and network issues
Enable video recording for flaky tests to see what's happening
Add console logging to understand test execution flow
Use page.pause() for manual inspection during development
Implement proper wait conditions before taking screenshots

Visual Testing Pitfalls:

Don't test every element - focus on critical user flows
Avoid testing dynamic content without proper handling
Don't ignore cross-browser differences in visual tests
Be careful with timing - ensure page stability before screenshots
Don't commit snapshot updates without reviewing changes

Performance & Reliability Tips:

Use appropriate trace recording levels for your environment
Implement retry logic for flaky visual tests
Use custom thresholds for tests with expected minor differences
Disable animations for consistent screenshot comparisons
Store baseline images in version control for team collaboration

✅ Outcomes

Understand Trace Viewer for debugging failed test runs
Learn to record video + screenshot for failures
Implement Visual Testing with baseline snapshots
Manage snapshot updates and avoid false positives
Handle dynamic content and cross-browser differences
Integrate visual testing into CI/CD pipelines
Create custom visual testing utilities and configurations
Debug flaky tests using advanced tracing techniques