Go to Oncrawl
English
Français
All Collections
General information
General information
General information about how to use Oncrawl
+1
41 articles in this collection
Written by
Francois Goube,
Tanguy,
Emma Labrador
and 1 other
Projects
What is a project in Oncrawl and how to manage projects
Written by
Rebecca Berbel
Updated over a week ago
Oncrawl's new interface
Find your way around the new look and feel of the Oncrawl platform.
Written by
Rebecca Berbel
Updated over a week ago
Auditing Core Web Vitals
Core Web Vitals are a confirmed, official ranking signal. You can measure your site's performance on these metrics with Oncrawl
Written by
Rebecca Berbel
Updated over a week ago
Understanding your monthly URL quota
Written by
Rebecca Berbel
Updated over a week ago
Video tutorials
[Tutorial] How to use OnCrawl's Inrank
Inrank, or internal PageRank measures on-site page popularity and link equity. Here's how to interpret it.
Written by
Rebecca Berbel
Updated over a week ago
[Tutorial] How to share a project in OnCrawl
You might want to share a project with coworkers or clients. Here's how.
Written by
Rebecca Berbel
Updated over a week ago
[Tutorial] How to use the OnCrawl Chrome extension
OnCrawl's free chrome extension highlights on-page metrics directly in your browser, helping you to spot issues easily.
Written by
Rebecca Berbel
Updated over a week ago
[Tutorial] How to monitor page speed with OnCrawl
Discover how measure page speed on your site with OnCrawl. But don't stop there: learn how OnCrawl helps you monitor page speed.
Written by
Rebecca Berbel
Updated over a week ago
Tutorial: How to spot changing status codes with our Data Explorer
Written by
Emma Labrador
Updated over a week ago
Tutorial: How to use the OnCrawl near duplicate detector
Written by
Francois Goube
Updated over a week ago
Tutorial: How to monitor your main tags' performances with OnCrawl
Discover how to monitor your meta tags and Hn performances using OnCrawl. A complete guide about what OnCrawl is analyzing.
Written by
Emma Labrador
Updated over a week ago
FAQ
What files are considered to be "resources" in Oncrawl?
When the crawler or log analyzer encounters a reference to certain types of files, it groups them as "resources". Here's the complete list.
Written by
Rebecca Berbel
Updated over a week ago
How to check if OnCrawl services are up or down
Check whether everything's working as expected and receive maintenance notifications
Written by
Rebecca Berbel
Updated over a week ago
Archived crawls and how to un-archive them
OnCrawl archives old crawls. The data's still there. Here's how to unarchive all or part of an archived crawl.
Written by
Rebecca Berbel
Updated over a week ago
Rename a project
Need to change the name of a project? It's easy.
Written by
Rebecca Berbel
Updated over a week ago
Share a project
A step by step article to understand how to share projects in different modes.
Written by
Emma Labrador
Updated over a week ago
How to open a export CSV file in Excel or Sheets
You've downloaded your Data Explorer results as a CSV file, but you usually work in Excel or Sheets. Now what?
Written by
Rebecca Berbel
Updated over a week ago
How to crawl a site with a robots.txt crawl delay greater than 1 second
Our bot supports the Crawl Delay directive of robots.txt file. If you have validated the project you can overcome this, learn how now!
Written by
Tanguy
Updated over a week ago
How to modify crawl limits while crawling
How to change crawl speed and max crawl depth after launching your crawl
Written by
Rebecca Berbel
Updated over a week ago
How does OnCrawl calculate load time?
What is load time, and how does OnCrawl measure it?
Written by
Rebecca Berbel
Updated over a week ago
How to quickly scan and analyze a list of URLs?
Sometimes you may need to analyze a more or less important list of URLs without browsing the entire site. Here is how to proceed.
Written by
Rebecca Berbel
Updated over a week ago
What format can I use for my sitemaps?
You can use sitemaps to compare and add data to a crawl. Here are the formats OnCrawl supports.
Written by
Rebecca Berbel
Updated over a week ago
How to explore all of the URLs in my sitemap
Sometimes you just want data on the URLs in your sitemap. It's possible to do this using a list of URLs.
Written by
Rebecca Berbel
Updated over a week ago
How to check urls in sitemap?
Here is how to make sure that your sitemaps will be taken into account during the analysis
Written by
Rebecca Berbel
Updated over a week ago
How can I find the target of a 3xx (redirection) status code?
3xx status codes indicate a redirected page. Here’s how to find the page that was redirected, as well as the page it was redirected to.
Written by
Rebecca Berbel
Updated over a week ago
How to use crawl results to find redirection loops
Your crawl results can help you to quickly produce a list of all redirected pages and the pages to which they are redirected.
Written by
Rebecca Berbel
Updated over a week ago
How to create a file listing all links pointing to a 301 URL, the old URL, and the new URL
You have permanently redirected (301) URLs. Here's how to create a list of all the links, the URL, and the URL they are redirected to.
Written by
Rebecca Berbel
Updated over a week ago
How to crawl a staging/pre-prod website
It's good practice to protect staging websites and pre-production websites from bots. Here's how to crawl one before it goes live.
Written by
Rebecca Berbel
Updated over a week ago
Why are there session volume differences between Google Analytics and my OnCrawl report?
If you follow your data closely you may have noticed volumes differences, here's why: OnCrawl is more reliable than GA over longer periods.
Written by
Rebecca Berbel
Updated over a week ago
How to bypass geographic redirects by crawling with HTTP headers
Crawl problems because your site redirects the Oncrawl bot because it doesn't have the right location-based cookie? Here's how to fix it.
Written by
Rebecca Berbel
Updated over a week ago
OnCrawl and the GDPR
Data protection and privacy for personal data. Where is data hosted? What data is collected? Can you refuse to give OnCrawl personal data?
Written by
Rebecca Berbel
Updated over a week ago
Advanced uses and settings
Link Dataset
Understanding the information collected by Oncrawl about a website's links
Written by
Rebecca Berbel
Updated over a week ago
REGEX in OnCrawl
Use pattern detection in fields to get to the essentials faster. Use regular expressions to create filters (Data Explorer & Segmentations)
Written by
Rebecca Berbel
Updated over a week ago
Lucene REGEX Cheat Sheet
This article is based on the Elastic Search Article
Written by
Rebecca Berbel
Updated over a week ago
Custom dashboards
What are custom dashboards and how can you create and use them?
Written by
Rebecca Berbel
Updated over a week ago
Crawl over Crawl
A Crawl over Crawl compares the results of two crawls to show you what the impact of your changes is or how two sites differ.
Written by
Rebecca Berbel
Updated over a week ago
Data scraping and custom fields
Data scraping is an option that allows you to analyse a portion of the source code extracted during a crawl using custom fields.
Written by
Rebecca Berbel
Updated over a week ago
How to crawl a site that uses JavaScript
If all or part of your website is built using JavaScript (JS), you may need to render pages in order to for a bot to be able to crawl them.
Written by
Rebecca Berbel
Updated over a week ago
User-agent and bot name
Change the bot name to test rules aimed at different bots
Written by
Rebecca Berbel
Updated over a week ago
Oncrawl API
Oncrawl is based on a platform built around an API. You can create your own application to request this API very easily, here's how to do it
Written by
Rebecca Berbel
Updated over a week ago
OnCrawl connector for Google Data Studio
Answers to your questions about the OnCrawl Data Studio connector
Written by
Rebecca Berbel
Updated over a week ago