Best practices & guides
Follow our guides to make the most of Oncrawl’s features.
28 articles
- How to segment groups of pages based on your Data Layer - Google Tag Manager exampleWant to make your Google Analytics data consistent with OnCrawl data & reports? Here is how to create categories based on your Data Layer
- How to create a segmentationCreate page groups (segments) and categorize pages according to any metrics you want.
- Segmentation and page groupsSegmentation is a way of creating page groups in order to view your technical SEO data in a meaningful way.
- How to use regex to define page groupsYou can use regular expressions (regex) to create segmentations with page groups based on information in URLs and other metrics.
- How to create a segmentation based on scraped data for publication datesYou need to analyze your pages based on publication date, relative to today's date. We'll take it step by step.
- Crawl frequencyWhat is crawl frequency? Best practices to crawl frequency?
- Crawl ratio & unique pagesHow to optimize crawl ratio? What are unique pages?
- Crawl part of a websiteUse a virtual robots.txt in order to limit the crawl to a subset of pages on a website
- How (and why) to use a virtual robot.txt fileUse a virtual robot.txt file to tell the Oncrawl bot how to crawl your site
- How to quickly scan and analyze a list of URLs?Sometimes you may need to analyze a more or less important list of URLs without browsing the entire site. Here is how to proceed.
- How to check URLs in a sitemapHere is how to make sure that your sitemaps will be taken into account during the analysis
- Customize your user-agent in OncrawlLearn how to change the bot identity used to crawl your website.
- How to explore all of the URLs in my sitemapSometimes you just want data on the URLs in your sitemap. It's possible to do this using a list of URLs with Oncrawl.
- Dealing with Duplicate Content: SEO Best PracticesManage duplicate content effectively with Oncrawl’s best practices to enhance SEO and improve rankings.
- Using the Management of pages with content duplication issues chartExplore how Oncrawl's Duplicate Content Cluster Map works to identify and manage duplicate content on your site. Learn about content blocks,
- Text to Code ratioLearn about the text to code ratio and its significance in SEO. Discover how it can indicate technical issues and impact page performance.
- Content Lens scoring and metrics definitionsUnderstand how the content quality feature rates your webpages content
- Content Lens Guide - How to enable the content quality analysisHow to use the content lens and best practices
- Understanding Status Codes in SEOUnderstand the SEO impact of HTTP status codes like 200, 301, 404 with Oncrawl and the best practices involving them.
- Understanding SEO Visits in OncrawlLearn what SEO visits are in Oncrawl and the best practices for increasing them.
- How can I find the target of a 3xx (redirection) status code?3xx status codes indicate a redirected page. Here’s how to find the page that was redirected, as well as the page it was redirected to.
- How does Oncrawl calculate load time?This article explores what is load time and how it is calculated in Oncraw.
- How to use REGEX in OncrawlUse pattern detection in fields to get to the essentials faster. Use regular expressions to create filters (Data Explorer & Segmentations)
- Lucene REGEX Cheat SheetIn this article you'll learn how to use Regex with Oncrawl. This article is based on the Elastic Search Article
- How to create a file listing all links pointing to a 301 URL, the old URL, and the new URLYou have permanently redirected (301) URLs. Here's how to create a list of all the links, the URL, and the URL they are redirected to.
- Filtering URLs on dashboards with an OQL filterFilter out pages you don't want to see in your charts
- Build your own chart and leverage all Oncrawl data endpointsTake complete control of your chart design and easily create a variety of visualizations