Technical SEO
Crawl budget
Crawl budget is the number of pages a search engine is willing and able to crawl on your site in a given period. It mainly matters for large sites with thousands of URLs.
When to worry about it
Most SaaS marketing sites are small enough that crawl budget isn't a problem. It becomes one with large documentation, many filter or parameter URLs, or programmatic pages, where important pages can go unvisited for weeks.
How to use it well
- Block low-value URLs, such as internal search and endless filters.
- Fix redirect chains and broken links.
- Keep the XML sitemap limited to pages you want indexed.
- Keep the server fast, so crawlers can fetch more per visit.
Frequently asked questions
Use the Crawl stats report in Google Search Console, under Settings.
Generally yes. Google crawls more when the server responds quickly and reliably.