Our initial analysis was conducted using data from Google Search Console in conjunction with data from log file analysis. The data showed us that Cromwell’s site was not being indexed fully because of their use of the React JavaScript framework. Google could not access the content that was only visible with JavaScript enabled so, while the framework allowed our client to have the functionality they desired in their UI, it also made it nearly impossible for Google to crawl.
Using data from Screaming Frog, we were able to collate a list of all available pages on the site. These were then categorised according to the company’s aims, identifying immediately those which could be no-indexed. In this way, we improved crawl efficiency from the outset by removing pages that didn’t need to be indexed, allowing more time for Google to access important content.
We also built a script that allowed us to pull Cromwell’s current site using Google’s headless browser “Headless Chrome”/”Puppeteer”, so we could view the site as Google’s crawlers did. Data from this script-based analysis enabled us to uncover specific rendering issues within each page and area of the site and educate the client on where problems lay.