Analyse What Googlebot Sees
Within Its 2MB Crawl Limit
Simulate Googlebot's view of your page within the 2MB crawl budget. Identify content and schema hidden from crawlers by bloated code — and fix crawlability issues before they cost you rankings and AI citations.
The Crawl Simulation Tool That Finds What Googlebot Cannot See
Googlebot applies a 2MB crawl limit to every page it processes. On pages built with visual page builders, heavy CSS frameworks, or large JavaScript bundles, schema and content beyond that byte limit is simply not processed — invisible to both Google and AI engines regardless of how well it is implemented.
- ✓Byte-accurate simulation of Googlebot's crawl view of your page
- ✓Schema crawlability audit showing which blocks are within and beyond the 2MB limit
- ✓Prioritised bloat reduction recommendations to recover hidden signals
Three steps to crawlsight results
Real example output from CrawlSight
Everything CrawlSight does for you
Byte-Accurate Crawl Simulation
Maps exactly which content and schema elements fall within and beyond Googlebot's 2MB crawl limit — showing precisely what is and is not being processed.
Schema Crawlability Audit
Tests the byte position of every schema block on your page and reports which are within the crawl limit (processed) and which are beyond it (invisible to AI engines and Google).
Bloat Source Identification
Identifies the specific sources of page bloat — inline CSS from page builders, large JavaScript bundles, base64-encoded images — with size contribution and fix recommendation per source.
Quick Fix Prioritisation
Ranks fixes by impact-per-effort: moving schema to the head section is typically a 15-minute fix that recovers all schema crawlability immediately, before addressing the larger inline CSS bloat.
Before and After Verification
Re-analyse after implementing fixes to confirm all schema and content is now within the 2MB crawl window — giving you certainty before investing further in content and schema optimisation.
CrawlSight Score
Rates your page's crawlability from 0 to 100 — providing a single metric that tracks improvement as bloat is reduced and schema is relocated to crawl-safe positions.
Who uses CrawlSight
- ✓Diagnose why Elementor or Divi pages have poor AI citation rates despite good schema
- ✓Identify the exact bloat sources costing you schema crawlability
- ✓Fix crawlability before investing further in schema implementation
- ✓Add crawlability simulation to standard technical audit workflow
- ✓Deliver crawl limit analysis as a premium service for content-heavy sites
- ✓Quantify the business impact of crawlability issues with citation rate data
- ✓Ensure every piece of schema you implement is actually being processed
- ✓Stop wasting time implementing schema that is invisible beyond the crawl limit
- ✓Verify crawlability as part of every new page publishing workflow
Without vs With CrawlSight
Frequently asked questions
about CrawlSight
Googlebot applies a 2MB crawl limit to control the computational cost of crawling billions of web pages. When Googlebot fetches a page, it downloads and processes the HTML up to the 2MB threshold and then stops — any content, schema, or signals appearing beyond that byte count are simply not processed. For pages built with visual page builders, heavy CSS frameworks, or large JavaScript bundles injected inline, the 2MB limit can be reached before all content and schema are processed.
The four most common causes are: page builder inline styles (Elementor, Divi, and Visual Composer inject 200 to 500KB of CSS per page), inline JavaScript from analytics, chat widgets, and advertising scripts, verbose HTML output from CMSs with excessive attributes and nested elements, and base64-encoded images embedded directly in the HTML rather than loaded as external files.
Schema appearing beyond the 2MB limit is not processed during Googlebot's initial crawl. The consistently safe approach is to ensure all schema blocks appear within the first 500KB of the HTML — typically in the document head or immediately after the opening body tag. CrawlSight shows you exactly where in the byte count each schema block appears, making it straightforward to identify which ones need to be moved.
Moving schema from the bottom of the page to the document head is the fastest fix. In WordPress, this is typically done using the wp_head action or a header injection plugin. The JSON-LD schema blocks can be relocated to the head section without changing anything else about the page. For pages where inline CSS is the primary bloat source, extracting styles to an external stylesheet can reduce page size by 30 to 50% on page builder sites.
Page speed and crawlability are related but distinct. Crawlability measures whether your full page content can be processed within Googlebot's byte budget — a different constraint from Core Web Vitals and load time. However, many of the same actions that improve crawlability (reducing inline CSS, deferring JavaScript, removing base64-encoded images) also improve page speed, so fixing crawlability issues typically has a positive secondary effect on Core Web Vitals.
Yes. Fully available on the free plan with 15 runs per month. Each run provides the complete byte-accurate crawl simulation, content visibility map, schema crawlability audit, bloat source identification, priority fix list, and CrawlSight score.
The CrawlSight report shows the exact byte position of every schema block — for example, FAQPage schema at byte 340KB is well within the 2MB limit, while FAQPage schema at byte 2.3MB is beyond it. As a safe target, aim to have all schema within the first 500KB of your HTML. This is achievable on any CMS by placing JSON-LD blocks in the document head section. The CrawlSight score includes a schema safety percentage showing what proportion of your schema is currently in the safe zone.
Yes. Googlebot uses separate mobile and desktop crawlers, and mobile pages are often larger on total load size due to additional mobile-specific elements. The CrawlSight tool analyses both desktop and mobile versions of your page and reports the crawl limit exposure separately for each. On page builder sites like Elementor, mobile pages can be 15 to 30% larger than desktop due to mobile-specific style overrides — meaning schema that is safely within the desktop crawl limit may be beyond the mobile crawl limit.
For WordPress sites with page builder bloat, the fastest non-developer fix is using a plugin like WP Rocket or Perfmatters to: extract inline CSS to external stylesheets (removes 200 to 500KB of inline CSS on Elementor sites), defer JavaScript not needed for initial page render (removes 100 to 300KB from initial parse), and enable CDN for static assets (no size change but improves load time). These plugin settings typically take 15 to 30 minutes to configure and can reduce page size by 30 to 50% without touching any code.
No. Google's 2MB limit applies to HTML content. Bing has a similar but slightly different crawl budget constraint. AI crawlers like GPTBot and ClaudeBot do not have a fixed byte limit but use efficiency heuristics that similarly deprioritise content deep in oversized pages. CrawlSight's analysis is calibrated primarily for Googlebot's 2MB constraint, but pages fixed to meet this threshold are typically also crawled more completely by AI crawlers. The CrawlSight score reflects Googlebot compliance specifically.