Best practices & guides
Follow our guides to make the most of Oncrawl’s features.
29 articles
How to segment groups of pages based on your Data Layer - Google Tag Manager exampleWant to make your Google Analytics data consistent with OnCrawl data & reports? Here is how to create categories based on your Data Layer
How to create a segmentationCreate page groups (segments) and categorize pages according to any metrics you want.
Segmentation and page groupsSegmentation is a way of creating page groups in order to view your technical SEO data in a meaningful way.
How to use regex to define page groupsYou can use regular expressions (regex) to create segmentations with page groups based on information in URLs and other metrics.
How to create a segmentation based on scraped data for publication datesYou need to analyze your pages based on publication date, relative to today's date. We'll take it step by step.
Introducing Segmentation Export
Crawl frequencyWhat is crawl frequency? Best practices to crawl frequency?
Crawl ratio & unique pagesHow to optimize crawl ratio? What are unique pages?
Crawl part of a websiteUse a virtual robots.txt in order to limit the crawl to a subset of pages on a website
How (and why) to use a virtual robot.txt fileUse a virtual robot.txt file to tell the Oncrawl bot how to crawl your site
How to quickly scan and analyze a list of URLs?Sometimes you may need to analyze a more or less important list of URLs without browsing the entire site. Here is how to proceed.
How to check URLs in a sitemapHere is how to make sure that your sitemaps will be taken into account during the analysis
Customize your user-agent in OncrawlLearn how to change the bot identity used to crawl your website.
How to explore all of the URLs in my sitemapSometimes you just want data on the URLs in your sitemap. It's possible to do this using a list of URLs with Oncrawl.
Dealing with Duplicate Content: SEO Best PracticesManage duplicate content effectively with Oncrawl’s best practices to enhance SEO and improve rankings.
Using the Management of pages with content duplication issues chartExplore how Oncrawl's Duplicate Content Cluster Map works to identify and manage duplicate content on your site. Learn about content blocks,
Text to Code ratioLearn about the text to code ratio and its significance in SEO. Discover how it can indicate technical issues and impact page performance.
Content Lens scoring and metrics definitionsUnderstand how the content quality feature rates your webpages content
Content Lens Guide - How to enable the content quality analysisHow to use the content lens and best practices
Understanding Status Codes in SEOUnderstand the SEO impact of HTTP status codes like 200, 301, 404 with Oncrawl and the best practices involving them.
Understanding SEO Visits in OncrawlLearn what SEO visits are in Oncrawl and the best practices for increasing them.
How can I find the target of a 3xx (redirection) status code?3xx status codes indicate a redirected page. Here’s how to find the page that was redirected, as well as the page it was redirected to.
How does Oncrawl calculate load time?This article explores what is load time and how it is calculated in Oncraw.
How to use REGEX in OncrawlUse pattern detection in fields to get to the essentials faster. Use regular expressions to create filters (Data Explorer & Segmentations)
Lucene REGEX Cheat SheetIn this article you'll learn how to use Regex with Oncrawl. This article is based on the Elastic Search Article
How to create a file listing all links pointing to a 301 URL, the old URL, and the new URLYou have permanently redirected (301) URLs. Here's how to create a list of all the links, the URL, and the URL they are redirected to.
Filtering URLs on dashboards with an OQL filterFilter out pages you don't want to see in your charts
Build your own chart and leverage all Oncrawl data endpointsTake complete control of your chart design and easily create a variety of visualizations