What are monthly crawl tokens?
Your monthly crawl quota contains a fixed number of tokens. Every URL fetched during a crawl consumes tokens from this quota.
The number of tokens consumed per URL depends on the crawl configuration:
Tokens consumed = URLs fetched x token multiplier
Your crawl tokens are renewed every month while your subscription remains active. You can find your next renewal date on the Quotas page.
How many tokens does each URL consume?
Crawl configuration | Tokens consumed per URL |
Standard crawling | 1 token |
JavaScript crawling | 3 tokens total |
One or more JavaScript audits | 5 tokens total |
Standard crawling: 1 token
Fetching a URL without JavaScript crawling consumes 1 crawl token.
JavaScript crawling: 3 tokens total
Fetching and rendering a URL with JavaScript consumes 3 crawl tokens in total.
Enabling Shadow DOM does not change this cost. JavaScript crawling consumes 3 tokens per URL whether Shadow DOM is enabled or not.
JavaScript audits: 5 tokens total
Running a JavaScript audit consumes 5 crawl tokens per URL in total, including JavaScript crawling.
JavaScript audits include:
Core Web Vitals analysis
Accessibility checks
The cost remains 5 tokens whether you enable one audit or both. The multipliers are not added together: JavaScript audits cost 5 tokens total, not 3 + 5 tokens.
JavaScript crawling is required to run JavaScript audits.
Example token calculations
For a crawl of 50,000 URLs:
Crawl configuration | Calculation | Tokens consumed |
Standard crawling | 50,000 x 1 | 50,000 |
JavaScript crawling | 50,000 x 3 | 150,000 |
One or more JavaScript audits | 50,000 x 5 | 250,000 |
Enabling both Core Web Vitals and Accessibility checks would still consume 250,000 tokens.
What happens to tokens when a crawl is deleted or cancelled?
URLs fetched during a crawl consume tokens even if the crawl is later deleted.
Cancelled crawls do not consume tokens.
What happens when you run out of tokens?
If your monthly quota is exhausted, crawls currently running will stop. You will not be able to launch another crawl until your quota is renewed or increased.
How does a crawl URL limit affect token consumption?
A URL limit controls the maximum number of URLs fetched during a crawl. It does not define the maximum number of tokens the crawl can consume.
To estimate the URL limit for a specific token budget, divide the available tokens by the applicable multiplier:
Maximum URLs = available tokens / token multiplier
For a budget of 100,000 tokens:
Crawl configuration | Maximum URL limit |
Standard crawling | 100,000 URLs |
JavaScript crawling | 33,333 URLs |
One or more JavaScript audits | 20,000 URLs |
Always round the result down to avoid exceeding your available quota.