Screenshotly
Tutorial

Automated Visual Regression Testing with Screenshots

A complete guide to visual regression testing with automated screenshots. Detect layout shifts, style changes, and UI bugs automatically.

Asad AliJanuary 28, 20269 min read

Visual regression testing catches UI bugs that traditional unit and integration tests miss. A button that moved 5 pixels, a font that changed weight, a card that lost its shadow—these issues slip through code-level tests but are immediately obvious in screenshots.

In this guide, you'll learn how to implement automated visual regression testing using screenshot APIs. We'll cover the fundamentals, build a complete testing pipeline, and share best practices from teams running visual tests in production.

What is Visual Regression Testing?

Visual regression testing compares screenshots of your UI across different versions to detect unintended changes. The workflow is simple:

  1. Capture baseline: Screenshot your UI in a known-good state
  2. Make changes: Develop new features or fix bugs
  3. Capture current: Screenshot the same UI after changes
  4. Compare: Diff the images to find differences
  5. Review: Evaluate whether differences are expected or bugs

Why Screenshots Beat DOM Comparison

Some visual testing tools compare DOM structure instead of actual rendered output. Screenshot-based testing has advantages:

  • Catches CSS issues: Style changes are invisible in DOM comparison
  • Detects rendering bugs: Font rendering, image loading, animation states
  • Matches user experience: Tests what users actually see
  • Works across technologies: Same approach for React, Vue, or static HTML

Setting Up Visual Regression Testing

Prerequisites

  • Node.js 18+
  • A screenshot API key (we'll use Screenshotly)
  • A CI/CD pipeline (GitHub Actions, GitLab CI, etc.)
  • Image comparison library (pixelmatch, resemblejs)

Project Structure

visual-tests/
├── baselines/           # Baseline screenshots
├── current/             # Current screenshots
├── diffs/               # Difference images
├── config.json          # Test configuration
├── capture.js           # Screenshot capture script
├── compare.js           # Image comparison script
└── report.js            # Report generation

Configuration

Define pages and viewports to test:

{
  "baseUrl": "https://staging.yourapp.com",
  "pages": [
    { "name": "home", "path": "/" },
    { "name": "pricing", "path": "/pricing" },
    { "name": "dashboard", "path": "/dashboard", "authenticated": true },
    { "name": "settings", "path": "/settings", "authenticated": true }
  ],
  "viewports": [
    { "name": "desktop", "width": 1920, "height": 1080 },
    { "name": "tablet", "width": 768, "height": 1024 },
    { "name": "mobile", "width": 375, "height": 812 }
  ],
  "threshold": 0.01
}

Capture Script

// capture.js
const fs = require('fs');
const path = require('path');
const config = require('./config.json');

const API_KEY = process.env.SCREENSHOTLY_API_KEY;
const OUTPUT_DIR = process.argv[2] || 'current';

async function captureScreenshot(page, viewport) {
  const response = await fetch('https://api.screenshotly.app/screenshot', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      url: `${config.baseUrl}${page.path}`,
      viewport: {
        width: viewport.width,
        height: viewport.height,
      },
      format: 'png',
      // Remove dynamic elements that cause false positives
      aiRemoval: {
        enabled: true,
        types: ['cookie-banner', 'chat-widget'],
      },
      // Wait for content to stabilize
      delay: 1000,
    }),
  });

  if (!response.ok) {
    throw new Error(`Failed to capture ${page.name}: ${response.status}`);
  }

  return Buffer.from(await response.arrayBuffer());
}

async function captureAll() {
  const outputDir = path.join(__dirname, OUTPUT_DIR);
  fs.mkdirSync(outputDir, { recursive: true });

  const results = [];

  for (const page of config.pages) {
    for (const viewport of config.viewports) {
      const filename = `${page.name}-${viewport.name}.png`;
      const filepath = path.join(outputDir, filename);

      console.log(`Capturing: ${filename}`);

      try {
        const screenshot = await captureScreenshot(page, viewport);
        fs.writeFileSync(filepath, screenshot);
        results.push({ filename, status: 'success' });
      } catch (error) {
        console.error(`Failed: ${filename} - ${error.message}`);
        results.push({ filename, status: 'failed', error: error.message });
      }
    }
  }

  // Write results
  fs.writeFileSync(
    path.join(outputDir, 'capture-results.json'),
    JSON.stringify(results, null, 2)
  );

  console.log(`Captured ${results.filter(r => r.status === 'success').length} screenshots`);
}

captureAll();

Comparison Script

// compare.js
const fs = require('fs');
const path = require('path');
const { PNG } = require('pngjs');
const pixelmatch = require('pixelmatch');
const config = require('./config.json');

const BASELINE_DIR = path.join(__dirname, 'baselines');
const CURRENT_DIR = path.join(__dirname, 'current');
const DIFF_DIR = path.join(__dirname, 'diffs');

async function compareScreenshots() {
  fs.mkdirSync(DIFF_DIR, { recursive: true });

  const results = [];
  const baselines = fs.readdirSync(BASELINE_DIR).filter(f => f.endsWith('.png'));

  for (const filename of baselines) {
    const baselinePath = path.join(BASELINE_DIR, filename);
    const currentPath = path.join(CURRENT_DIR, filename);
    const diffPath = path.join(DIFF_DIR, filename);

    // Check if current screenshot exists
    if (!fs.existsSync(currentPath)) {
      results.push({
        filename,
        status: 'missing',
        message: 'Current screenshot not found',
      });
      continue;
    }

    // Load images
    const baseline = PNG.sync.read(fs.readFileSync(baselinePath));
    const current = PNG.sync.read(fs.readFileSync(currentPath));

    // Check dimensions match
    if (baseline.width !== current.width || baseline.height !== current.height) {
      results.push({
        filename,
        status: 'dimension_mismatch',
        baseline: { width: baseline.width, height: baseline.height },
        current: { width: current.width, height: current.height },
      });
      continue;
    }

    // Create diff image
    const diff = new PNG({ width: baseline.width, height: baseline.height });

    // Compare pixels
    const numDiffPixels = pixelmatch(
      baseline.data,
      current.data,
      diff.data,
      baseline.width,
      baseline.height,
      { threshold: 0.1 }
    );

    // Calculate difference percentage
    const totalPixels = baseline.width * baseline.height;
    const diffPercentage = (numDiffPixels / totalPixels) * 100;

    // Save diff image if there are differences
    if (numDiffPixels > 0) {
      fs.writeFileSync(diffPath, PNG.sync.write(diff));
    }

    const passed = diffPercentage <= (config.threshold * 100);

    results.push({
      filename,
      status: passed ? 'passed' : 'failed',
      diffPixels: numDiffPixels,
      diffPercentage: diffPercentage.toFixed(4),
      threshold: config.threshold * 100,
    });
  }

  // Check for new screenshots not in baseline
  const currentFiles = fs.readdirSync(CURRENT_DIR).filter(f => f.endsWith('.png'));
  for (const filename of currentFiles) {
    if (!baselines.includes(filename)) {
      results.push({
        filename,
        status: 'new',
        message: 'New screenshot, no baseline exists',
      });
    }
  }

  return results;
}

async function main() {
  console.log('Comparing screenshots...\n');

  const results = await compareScreenshots();

  // Summary
  const passed = results.filter(r => r.status === 'passed').length;
  const failed = results.filter(r => r.status === 'failed').length;
  const newScreenshots = results.filter(r => r.status === 'new').length;
  const missing = results.filter(r => r.status === 'missing').length;

  console.log('Results:');
  console.log(`  ✅ Passed: ${passed}`);
  console.log(`  ❌ Failed: ${failed}`);
  console.log(`  🆕 New: ${newScreenshots}`);
  console.log(`  ⚠️  Missing: ${missing}`);
  console.log('');

  // Details for failures
  if (failed > 0) {
    console.log('Failed tests:');
    results
      .filter(r => r.status === 'failed')
      .forEach(r => {
        console.log(`  - ${r.filename}: ${r.diffPercentage}% different`);
      });
  }

  // Write results
  fs.writeFileSync(
    path.join(__dirname, 'comparison-results.json'),
    JSON.stringify(results, null, 2)
  );

  // Exit with error if tests failed
  process.exit(failed > 0 ? 1 : 0);
}

main();

CI/CD Integration

# .github/workflows/visual-tests.yml
name: Visual Regression Tests

on:
  pull_request:
    branches: [main]

jobs:
  visual-tests:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
      
      - name: Install dependencies
        run: npm ci
        working-directory: visual-tests
      
      - name: Capture current screenshots
        env:
          SCREENSHOTLY_API_KEY: ${{ secrets.SCREENSHOTLY_API_KEY }}
        run: node capture.js current
        working-directory: visual-tests
      
      - name: Compare screenshots
        run: node compare.js
        working-directory: visual-tests
      
      - name: Upload diff artifacts
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: visual-diffs
          path: |
            visual-tests/current/
            visual-tests/diffs/
            visual-tests/comparison-results.json

Best Practices

1. Handle Dynamic Content

Dynamic content causes false positives. Handle it by:

Remove timestamps and dates:

// Inject CSS to hide dynamic elements
body: JSON.stringify({
  url: pageUrl,
  css: `
    .timestamp, .date, .time { visibility: hidden !important; }
    [data-testid="current-time"] { visibility: hidden !important; }
  `,
})

Use AI removal for common dynamic elements:

aiRemoval: {
  enabled: true,
  types: ['cookie-banner', 'chat-widget', 'notification'],
}

2. Set Appropriate Thresholds

Not all pixel differences are bugs:

  • 0.1%: Very strict, catches subtle changes
  • 1%: Balanced, ignores anti-aliasing differences
  • 5%: Lenient, focuses on major layout changes

Start strict and loosen if false positives become a problem.

3. Test Critical User Flows

Prioritize screenshots that matter:

  1. Landing page (first impression)
  2. Pricing page (conversion critical)
  3. Signup/login flows (user acquisition)
  4. Core feature pages (product value)
  5. Error states (user experience)

4. Capture Multiple Viewports

Don't just test desktop:

const viewports = [
  { name: 'desktop-1080p', width: 1920, height: 1080 },
  { name: 'desktop-720p', width: 1280, height: 720 },
  { name: 'tablet', width: 768, height: 1024 },
  { name: 'mobile-large', width: 414, height: 896 },
  { name: 'mobile-small', width: 375, height: 667 },
];

5. Maintain Baselines in Version Control

Store baseline screenshots in git:

.gitattributes:
baselines/*.png binary

.gitignore:
visual-tests/current/
visual-tests/diffs/

Update baselines intentionally:

# Update all baselines
npm run capture -- baselines

# Review changes
git diff --stat
git add visual-tests/baselines/
git commit -m "Update visual baselines for feature X"

Handling Authenticated Pages

For logged-in states, pass session cookies:

async function captureAuthenticatedPage(page, sessionToken) {
  return fetch('https://api.screenshotly.app/screenshot', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      url: `${config.baseUrl}${page.path}`,
      cookies: [
        {
          name: 'session_token',
          value: sessionToken,
          domain: new URL(config.baseUrl).hostname,
        },
      ],
      viewport: page.viewport,
      format: 'png',
    }),
  });
}

Conclusion

Visual regression testing with screenshots provides confidence that UI changes are intentional. By automating captures and comparisons in CI/CD, you catch visual bugs before users do.

Key takeaways:

  1. Screenshot APIs simplify capture - No browser infrastructure needed
  2. AI removal reduces false positives - No more cookie banner differences
  3. Multiple viewports catch responsive bugs - Test all breakpoints
  4. Store baselines in version control - Track visual changes with code
  5. Set appropriate thresholds - Balance sensitivity with practicality

Start with critical pages and expand coverage over time. Even basic visual testing catches bugs that slip through other test types.

Ready to add visual regression testing?

Get your free API key → - 100 free screenshots to get started.

Learn more about automated testing use cases →

testing
visual regression
automation
ci/cd
quality assurance

About the Author

Asad Ali

Asad Ali

Full-Stack Developer and Founder of ZTabs with 8+ years of experience building scalable web applications and APIs. Specializes in performance optimization, SaaS development, and modern web technologies.

Credentials: Founder & CEO at ZTabs, Full-Stack Developer, Expert in Next.js, React, Node.js, and API optimization

TwitterLinkedInGitHub

Ready to capture your first screenshot?

Get started with 100 free screenshots. No credit card required.

Start Free TrialTry Playground

Related Articles

Case Study

How TechFlow Reduced Documentation Maintenance by 85% with Screenshotly

8 min read

Tutorial

Python Screenshot API: Complete Tutorial with Examples

10 min read

Tutorial

Node.js Screenshot API: Complete Integration Guide

11 min read

Back to Blog