All Collections
General information
General information about how to use Oncrawl
45 articles
ProjectsWhat is a project in Oncrawl and how to manage projects
Oncrawl's new interfaceFind your way around the new look and feel of the Oncrawl platform.
Auditing Core Web VitalsCore Web Vitals are a confirmed, official ranking signal. You can measure your site's performance on these metrics with Oncrawl
Understanding your monthly URL quota
[Tutorial] How to use OnCrawl's InrankInrank, or internal PageRank measures on-site page popularity and link equity. Here's how to interpret it.
[Tutorial] How to share a project in OnCrawlYou might want to share a project with coworkers or clients. Here's how.
[Tutorial] How to use the OnCrawl Chrome extensionOnCrawl's free chrome extension highlights on-page metrics directly in your browser, helping you to spot issues easily.
[Tutorial] How to monitor page speed with OnCrawlDiscover how measure page speed on your site with OnCrawl. But don't stop there: learn how OnCrawl helps you monitor page speed.
Tutorial: How to spot changing status codes with our Data Explorer
Tutorial: How to use the OnCrawl near duplicate detector
Tutorial: How to monitor your main tags' performances with OnCrawlDiscover how to monitor your meta tags and Hn performances using OnCrawl. A complete guide about what OnCrawl is analyzing.
What files are considered to be "resources" in Oncrawl?When the crawler or log analyzer encounters a reference to certain types of files, it groups them as "resources". Here's the complete list.
GA4: metricsIntroduction to the metrics available using a GA4 data source
How to check if OnCrawl services are up or downCheck whether everything's working as expected and receive maintenance notifications
Archived crawls and how to un-archive themOnCrawl archives old crawls. The data's still there. Here's how to unarchive all or part of an archived crawl.
Rename a projectNeed to change the name of a project? It's easy.
Invite a new member to a workspaceHow to invite someone to view or collaborate in Oncrawl
How to open a export CSV file in Excel or SheetsYou've downloaded your Data Explorer results as a CSV file, but you usually work in Excel or Sheets. Now what?
How to crawl a site with a robots.txt crawl delay greater than 1 secondOur bot supports the Crawl Delay directive of robots.txt file. If you have validated the project you can overcome this, learn how now!
How to modify crawl limits while crawlingHow to change crawl speed and max crawl depth after launching your crawl
How does OnCrawl calculate load time?What is load time, and how does OnCrawl measure it?
How to quickly scan and analyze a list of URLs?Sometimes you may need to analyze a more or less important list of URLs without browsing the entire site. Here is how to proceed.
What format can I use for my sitemaps?You can use sitemaps to compare and add data to a crawl. Here are the formats OnCrawl supports.
How to explore all of the URLs in my sitemapSometimes you just want data on the URLs in your sitemap. It's possible to do this using a list of URLs.
How to check urls in sitemap?Here is how to make sure that your sitemaps will be taken into account during the analysis
How can I find the target of a 3xx (redirection) status code?3xx status codes indicate a redirected page. Here’s how to find the page that was redirected, as well as the page it was redirected to.
How to use crawl results to find redirection loopsYour crawl results can help you to quickly produce a list of all redirected pages and the pages to which they are redirected.
How to create a file listing all links pointing to a 301 URL, the old URL, and the new URLYou have permanently redirected (301) URLs. Here's how to create a list of all the links, the URL, and the URL they are redirected to.
How to crawl a staging/pre-prod websiteIt's good practice to protect staging websites and pre-production websites from bots. Here's how to crawl one before it goes live.
Why is my data different between Oncrawl and Google Analytics (GA4)Are you seeing differences between numbers in Google Analytics (GA4) and numbers in Oncrawl, when looking at your GA4 data?
How to bypass geographic redirects by crawling with HTTP headersCrawl problems because your site redirects the Oncrawl bot because it doesn't have the right location-based cookie? Here's how to fix it.
OnCrawl and the GDPRData protection and privacy for personal data. Where is data hosted? What data is collected? Can you refuse to give OnCrawl personal data?
Migrating from GA Universal Analytics to GA4 in OncrawlAre you replacing your GA-UA account with GA4? Here's everything you need to do to update your data sources in Oncrawl.
Manage workspace members with access to a projectHow to check or modify the workspace members with access to a specific project
Link DatasetUnderstanding the information collected by Oncrawl about a website's links
REGEX in OnCrawlUse pattern detection in fields to get to the essentials faster. Use regular expressions to create filters (Data Explorer & Segmentations)
Managing crawl alerts with OncrawlSet up alerts and use Oncrawl to monitor technical SEO on your site
Lucene REGEX Cheat SheetThis article is based on the Elastic Search Article
Custom dashboardsWhat are custom dashboards and how can you create and use them?
Crawl over CrawlA Crawl over Crawl compares the results of two crawls to show you what the impact of your changes is or how two sites differ.
Data scraping and custom fieldsData scraping is an option that allows you to analyse a portion of the source code extracted during a crawl using custom fields.
How to crawl a site that uses JavaScriptIf all or part of your website is built using JavaScript (JS), you may need to render pages in order to for a bot to be able to crawl them.
User-agent and bot nameChange the bot name to test rules aimed at different bots
Oncrawl APIOncrawl is based on a platform built around an API. You can create your own application to request this API very easily, here's how to do it
Oncrawl connector for Looker StudioAnswers to your questions about the Oncrawl Looker Studio (ex Data Studio) connector