Guide

Batch Screenshot Processing: Bulk Capture with Jobs and Batches

Capture hundreds of pages without timeouts or duplicate work: pace synchronous calls to your plan's per-minute limit, queue async jobs with idempotency keys, and submit up to 20 captures per batch request.

Asad AliDecember 10, 202411 min read

When a loop of single requests stops being enough

Capturing one page is a single POST /api/screenshot. Capturing every page of a site audit, a product catalog or a nightly monitoring list adds four problems that a simple loop does not solve:

  1. Throughput is capped by your plan. Each plan has a per-minute request limit, and a loop that ignores it collects 429 responses.
  2. Slow pages hit request timeouts. A synchronous request holds an HTTP connection open while the page loads and renders. Heavy pages and full-page captures can run out of time.
  3. Client retries create duplicates. If your process crashes halfway through a run, re-running it should not capture everything twice.
  4. You need to know what finished. At a few hundred URLs, "which ones failed?" needs a real answer.

Screenshotly gives you three ways to run bulk captures. Which one fits depends on your plan and on how long you can wait for each result.

Approach Endpoint Plan Best for
Paced synchronous calls POST /api/screenshot All plans Small runs, fast pages, results needed inline
Async jobs POST /api/jobs Basic and above Slow pages, crash-safe runs, webhook delivery
Batch jobs POST /api/jobs/batch Growth and Scale Large runs submitted 20 captures at a time

All three accept the same capture body: the fields documented in the screenshot API reference. Plan limits come from the pricing page. For a product-level overview of the async model, see async screenshots.

Know your limits before you start

Two separate limits apply to every capture request:

Plan Requests per minute Captures
Free 5 100 total
Basic 40 2,500 per month
Growth 80 12,000 per month
Scale 200 50,000 per month
  • The per-minute limit is counted per account and plan, not per API key. Ten keys on one account share one budget.
  • The capture allowance counts successful captures only. Failed captures are free, and an optional hard usage limit in your account settings can lower the ceiling further.
  • When either limit is exceeded, the API returns 429. The synchronous and single-job endpoints include a Retry-After header (seconds) and a retryAfter field in the JSON body.

The rate limits guide covers the headers in detail. For batch work, the practical rule is: pace your client to stay under the per-minute limit, and treat a 429 as a signal to wait, not to retry immediately.

Approach 1: paced synchronous calls (all plans)

On the Free plan, or for runs of a few dozen fast pages, call the synchronous endpoint and pace the requests yourself. This client keeps a rolling record of request start times and waits whenever the last 60 seconds already contain your plan's limit:

const API = 'https://screenshotly.app';
const KEY = process.env.SCREENSHOTLY_API_KEY;
const PER_MINUTE = 40; // Free 5, Basic 40, Growth 80, Scale 200

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const started = [];

async function waitForSlot() {
  for (;;) {
    const now = Date.now();
    while (started.length && now - started[0] >= 60_000) started.shift();
    if (started.length < PER_MINUTE) {
      started.push(now);
      return;
    }
    await sleep(60_000 - (now - started[0]) + 50);
  }
}

async function capture(url) {
  for (let attempt = 1; attempt <= 3; attempt += 1) {
    await waitForSlot();
    const response = await fetch(`${API}/api/screenshot`, {
      method: 'POST',
      headers: { 'x-api-key': KEY, 'Content-Type': 'application/json' },
      body: JSON.stringify({ url, width: 1280, height: 800, format: 'png' }),
    });
    if (response.ok) return Buffer.from(await response.arrayBuffer());
    if (response.status === 429 || response.status >= 500) {
      const retryAfter = Number(response.headers.get('retry-after')) || 2 ** attempt;
      await sleep(retryAfter * 1000);
      continue;
    }
    // 400, 401, 403 and 413 will not succeed on retry.
    throw new Error(`${response.status}: ${await response.text()}`);
  }
  throw new Error(`Gave up on ${url}`);
}

async function run(urls, concurrency = 4) {
  const queue = [...urls];
  const results = [];
  await Promise.all(Array.from({ length: concurrency }, async () => {
    while (queue.length) {
      const url = queue.shift();
      try {
        results.push({ url, ok: true, image: await capture(url) });
      } catch (error) {
        results.push({ url, ok: false, error: error.message });
      }
    }
  }));
  return results;
}

The request body in this example works on every plan:

{ "url": "https://example.com/pricing", "width": 1280, "height": 800, "format": "png" }

Two details matter here. Concurrency and the per-minute limit are separate controls: concurrency decides how many connections are open at once, and the slot tracker decides how many requests start per minute. And the synchronous endpoint returns the image bytes directly, so write them to disk or object storage as they arrive instead of holding hundreds of buffers in memory.

The retry rules follow the error handling guide: retry 429 and 5xx with a delay, and fail fast on 4xx validation, authentication and plan errors.

Approach 2: async jobs (Basic and above)

Async jobs separate submitting a capture from waiting for it. POST /api/jobs validates the body, stores it encrypted and returns 202 Accepted with a job ID. A worker then claims the job, captures it, stores the result and optionally calls your webhook. The async jobs reference is the contract; this section shows how to use it for a bulk run.

Async jobs need an account API key on the Basic plan or above. This body is valid on Basic:

{
  "url": "https://example.com/pricing",
  "width": 1280,
  "height": 800,
  "fullPage": true,
  "format": "webp",
  "quality": 80,
  "blockAds": true
}

Submit with an idempotency key

Pass an Idempotency-Key header so a retried submission returns the original job instead of a new one. The key must be 1 to 200 printable ASCII characters, with no spaces, and it is scoped to your workspace. Derive it from something stable, such as the run ID plus a hash of the URL, so a crashed run can be restarted safely:

import { createHash } from 'node:crypto';

async function submitJob(runId, url) {
  const key = `${runId}:${createHash('sha256').update(url).digest('hex').slice(0, 32)}`;
  const response = await fetch(`${API}/api/jobs`, {
    method: 'POST',
    headers: {
      'x-api-key': KEY,
      'Content-Type': 'application/json',
      'Idempotency-Key': key,
    },
    body: JSON.stringify({ url, width: 1280, height: 800, fullPage: true, format: 'webp', quality: 80, blockAds: true }),
  });
  if (response.status === 429) {
    const retryAfter = Number(response.headers.get('retry-after')) || 60;
    throw Object.assign(new Error('rate limited'), { retryAfter });
  }
  if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
  return response.json(); // { id, status, replayed, statusUrl }
}

A new job returns 202 with "replayed": false. A repeated key returns 200 with "replayed": true and the existing job. Note that the server matches on the key alone. It does not compare request bodies, so reusing a key with different capture options still returns the original job. Change the key whenever the options change.

Submitting a job is still a request, so POST /api/jobs counts toward your per-minute limit. Reuse the waitForSlot() pacer from the first approach.

Poll for the result

GET /api/jobs/{id} returns the job's state. statusUrl and the Location header are relative paths, so resolve them against https://screenshotly.app. The status moves through these values:

Status Meaning
QUEUED Waiting for a worker, or waiting before a retry
RUNNING A worker is capturing the page
SUCCEEDED The result is stored; resultUrl is included
FAILED All attempts failed; see errorCode and errorMessage
CANCELED Canceled through DELETE /api/jobs/{id}
async function waitForJob(statusUrl, { timeoutMs = 10 * 60_000 } = {}) {
  const deadline = Date.now() + timeoutMs;
  let delay = 2_000;
  while (Date.now() < deadline) {
    const response = await fetch(new URL(statusUrl, API), { headers: { 'x-api-key': KEY } });
    if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
    const job = await response.json();
    if (['SUCCEEDED', 'FAILED', 'CANCELED'].includes(job.status)) return job;
    await sleep(delay);
    delay = Math.min(delay * 2, 15_000);
  }
  throw new Error(`Timed out waiting for ${statusUrl}`);
}

A succeeded job includes asset (content type, size, SHA-256 checksum, expiresAt) and a signed resultUrl. That URL is short-lived (about 15 minutes), so download it as soon as you read it. If it expires, read the job again to get a fresh one. The stored file itself is kept for your account's retention setting: 1 to 30 days, 7 by default. You can compare the downloaded file's SHA-256 against asset.checksum to confirm it arrived intact.

Reading job status does not consume captures. Polling with backoff, as above, keeps the load reasonable. If you would rather not poll, add a webhook object to each job and let Screenshotly call you when it finishes. The webhooks guide covers signatures and delivery retries.

Retries happen on the server

If a capture fails inside the worker, the job goes back to QUEUED with a delay and is tried again, up to three attempts in total. The second attempt starts no sooner than about 5 seconds later and the third no sooner than about 15 seconds after that; worker scheduling can add more time. Only after the final attempt does the job become FAILED. Your client therefore does not need to resubmit a job that is still QUEUED with errorCode: "CAPTURE_RETRY". Treat a FAILED job as final, record it and decide later whether to resubmit it with a new idempotency key.

Cancel what you no longer need

DELETE /api/jobs/{id} cancels a queued or running job and returns { "id": "...", "status": "CANCELED" }. A queued job is canceled immediately. A running job may finish its browser work first, but its output is discarded and the job ends as CANCELED.

Approach 3: batch jobs (Growth and Scale)

On Growth and Scale, POST /api/jobs/batch queues up to 20 captures in one request. The body is strict: a single captures array and nothing else.

{
  "captures": [
    { "url": "https://example.com/", "width": 1280, "height": 800, "format": "png" },
    { "url": "https://example.com/pricing", "width": 1280, "height": 800, "fullPage": true, "format": "webp", "quality": 80 },
    { "url": "https://example.com/docs", "device": "mobile", "format": "jpeg", "quality": 80 }
  ]
}

The response lists one entry per capture, in the same order:

{
  "jobs": [
    { "id": "…", "status": "QUEUED", "replayed": false, "statusUrl": "/api/jobs/…" }
  ]
}

Behaviors to design around:

  • All or nothing validation. Every item is checked against the schema, URL safety rules and your plan's feature gates before anything is queued. One bad URL or one Growth-only option on the wrong plan rejects the whole batch with 400 or 403.
  • Quota is checked up front. If the batch has more captures than your remaining allowance, the request fails with 429 and nothing is queued. This batch 429 does not include a Retry-After header, so check your usage before retrying.
  • One request, many captures. The batch counts as one request against the per-minute limit, and its size is checked against your remaining monthly captures.
  • Idempotency per item. A batch Idempotency-Key can be up to 180 characters. Item N is stored under <key>:<N>, so resubmitting the same chunk in the same order replays the same jobs. The response is 200 when every item was a replay and 202 otherwise. Do not reorder a chunk before retrying it.
function chunk(items, size = 20) {
  const chunks = [];
  for (let i = 0; i < items.length; i += size) chunks.push(items.slice(i, i + size));
  return chunks;
}

async function submitBatches(runId, urls) {
  const jobs = [];
  for (const [index, urlsInChunk] of chunk(urls).entries()) {
    await waitForSlot();
    const response = await fetch(`${API}/api/jobs/batch`, {
      method: 'POST',
      headers: {
        'x-api-key': KEY,
        'Content-Type': 'application/json',
        'Idempotency-Key': `${runId}:chunk-${index}`,
      },
      body: JSON.stringify({
        captures: urlsInChunk.map((url) => ({ url, width: 1280, height: 800, format: 'png' })),
      }),
    });
    if (!response.ok) throw new Error(`Chunk ${index} failed with ${response.status}: ${await response.text()}`);
    const { jobs: queued } = await response.json();
    queued.forEach((job, i) => jobs.push({ url: urlsInChunk[i], ...job }));
  }
  return jobs;
}

Keep runId stable for a given input list. If the script dies after chunk 7, re-running it replays chunks 0 to 7 (no new captures are queued) and continues with chunk 8.

Batch submission is fast, but completion is not instant. Queued jobs are processed by Screenshotly's capture workers, and the API does not publish a completion-rate guarantee. Plan your run around polling or webhooks, not around how quickly you can submit.

Tracking a large run

There is no bulk status endpoint. GET /api/jobs lists the 50 most recent jobs in your workspace, which is useful for a dashboard but not for tracking a run of hundreds. Keep your own record of { url, jobId, status } and update it as you poll:

async function trackRun(jobs, concurrency = 4) {
  const pending = [...jobs];
  const finished = [];
  await Promise.all(Array.from({ length: concurrency }, async () => {
    while (pending.length) {
      const job = pending.shift();
      const result = await waitForJob(job.statusUrl);
      finished.push({ url: job.url, id: job.id, status: result.status, resultUrl: result.resultUrl, error: result.errorMessage });
    }
  }));
  const failed = finished.filter((job) => job.status !== 'SUCCEEDED');
  console.log(`${finished.length - failed.length} succeeded, ${failed.length} did not`);
  return { finished, failed };
}

Write finished to a database or a JSON file as you go. When the run ends, the failed list is your retry candidates. Each is a URL whose job failed three times on the server, so check the error before resubmitting.

Choosing an approach

  • Free plan, or a few dozen fast pages: paced synchronous calls. Simple, and results come back in the response.
  • Basic plan, slow pages, or runs that must survive restarts: async jobs with idempotency keys and polling or webhooks.
  • Growth or Scale, hundreds of pages: batch jobs in chunks of 20, then the same polling or webhook flow.

Whichever you choose, make each capture lighter where you can: a smaller viewport, no full-page capture unless you need it, and JPEG or WebP output. The performance guide covers these settings. If the same URLs come up in repeated runs, the caching guide explains when cache and cacheTtl help.

Next steps

Ready to try it? Create a free account to test synchronous captures, then compare plans on the pricing page when you need jobs and batches.

batch
automation
async
jobs
scale

About the Author

Asad Ali

Asad Ali

CTO at ZTABS and the developer behind Screenshotly. Writes about screenshot automation, API design, and building reliable SaaS backends with Next.js, Node.js, and TypeScript.

Credentials: CTO at ZTABS, full-stack developer, builder and maintainer of Screenshotly and its official JavaScript and Python clients

Frequently Asked Questions

How many screenshots can I submit in one batch request?

Up to 20. POST /api/jobs/batch accepts a body of the form {"captures": [...]} with 1 to 20 capture objects, each using the same fields as the synchronous endpoint. Batch requests require the Growth or Scale plan. For larger runs, split the list into chunks of 20 and submit each chunk as its own batch.

How fast can I send capture requests?

Each plan has a per-minute request limit: Free 5, Basic 40, Growth 80 and Scale 200 requests per minute. The limit is counted per account, so creating more API keys does not raise it. Pace your client below the limit and honor the Retry-After header when you get a 429.

What happens if a batch contains one invalid capture?

The whole batch is rejected and nothing is queued. Every item is validated against the capture schema, URL safety checks and your plan's feature gates before any job is created, so fix the item named in the error and resubmit the batch.

How do I avoid creating duplicate jobs when my client retries?

Send an Idempotency-Key header. On POST /api/jobs, a repeated key returns the original job with replayed: true and HTTP 200 instead of 202. On POST /api/jobs/batch, the key is a base: item N is stored as <key>:<N>, so resubmitting the same chunk in the same order replays the same jobs.

Do failed captures count against my monthly quota?

No. Only successful captures count toward your monthly allowance. An async job gets up to three attempts in the worker before it is marked FAILED, and none of those failed attempts consume a capture.

Ready to capture your first screenshot?

Get started with 100 free screenshots. No credit card required.

Related Articles