Screaming Frog Review

by Abbs

---

Posted on 2023-11-15 12:39:57

---

---

Screaming Frog SEO Spider is one of the most popular SEO tools used by digital marketers and agencies today. This versatile crawler allows you to audit the technical SEO of your website or your competitors' to uncover issues and opportunities.

In this comprehensive Screaming Frog review, we'll cover everything you need to know, including:

• Overview of Screaming Frog SEO Spider
• Key Features and Benefits
• Crawling Your Website
• Using the Data
• Pricing
• Pros and Cons
• Who Should Use Screaming Frog?
• Screaming Frog Alternatives
• FAQs

Let's dive in!

Overview of Screaming Frog SEO Spider

Screaming Frog SEO Spider is a desktop crawler developed by UK-based Screaming Frog Ltd. First released in 2010, it has become one of the most widely used SEO tools on the market.

It works by crawling a website to extract key onsite elements including page titles, meta descriptions, headings, images, canonical tags, and more. This allows you to analyze the crawled pages for SEO issues that could be hurting your site's performance.

Some key things the Screaming Frog SEO crawler can help uncover:

• Broken links and redirects
• Duplicate content
• Missing meta data like page titles and descriptions
• Incorrect structured data implementation
• Crawl errors like 404s and 500s
• Slow page speed issues

The detailed crawl data enables you to identify SEO problems and optimize pages to improve rankings, traffic, anbd conversion rates.

Unlike some SEO tools, Screaming Frog is desktop-based software rather than a cloud-based platform. This gives it some advantages:

• Works offline once downloaded
• Provides faster crawl speeds
• Lower security risks

However, it does mean there's no web interface - you need to install it directly on your computer to use it.

Now let's look at some of the key features and benefits of Screaming Frog.

Key Features and Benefits

Here are some of the most notable features and benefits of using Screaming Frog SEO Spider:

Comprehensive On-Page Analysis with Screaming Frog

The main purpose of Screaming Frog is to analyze the on-page SEO elements of a website.

It crawls each page and grabs important optimization data like:

• Page titles - identify titles that are duplicated, missing, or too long/short
• Meta descriptions - check for missing or duplicate descriptions
• Headings (H1, H2, etc) - find missing headings or optimize hierarchy
• Images - check for missing ALT text descriptions
• Links (internal, external, broken) - analyze link equity and find broken links
• Redirects - identify redirect chains, loops, and status codes

This data enables you to optimize content and technical SEO issues on each URL.

Detailed Reports

Screaming Frog provides a wide range of reports and filters to segment your crawled data.

Some key reports include:

• Summary - overview of key crawl metrics
• Internal - internal linking and equity
• External - external references, links, and anchors
• Images - image optimization analysis
• Redirects - redirect issues and chains
• Content - duplicate content, word count
• Response codes - crawl errors and status codes

You can also export reports as CSVs or Excel files for additional analysis and sharing with clients.

Site Visualization

Screaming Frog offers two types of visualizations for showing your site's structure:

• Crawl Diagrams - color-coded nodes and links
• Tree Graphs - hierarchical site architecture

These help you analyze internal linking patterns and equity flows to optimize page authority.

Custom Extraction

The custom extraction tool allows you to customize the data Screaming Frog collects from each page.

You can use CSS selectors, XPath, or regex to extract specific elements like:

• Schema or structured data
• Additional headings (H3, H4, etc)
• Social meta tags
• Prices
• Anything in the page HTML

THis gives you flexibility to tailor crawls to your specific needs.

API Integrations

Screaming Frog can connect directly to the Google Analytics, Search Console, and PageSpeed Insights APIs.

This allows you to import additional performance and analytics data to enhance your SEO analysis.

Flexible Configuration

Unlike some crawlers, Screaming Frog gives yuo granular control over settings like:

• Crawl scope - subfolder, domain, sitemap
• Crawl speed - pages per second
• Bandwidth - throttle crawl speed
• User agent - custom user agent strings
• Rendering - optionally execute JavaScript

This allows advanced users to customize crawls.

Alternative Versions

Screaming Frog offers a free version with limited functionality alongside paid versions unlocking all features.

This allows you to try before you buy.

CRawling Your Website with screaming frog

Now let's walk through how to use Screaming Frog SEO Spider to crawl your site:

Step 1: Download and Install

Since Screaming Frog is desktop software, you need to download andd install it first.

Visit their website and click the blue "Download SEO Spider" button to download the installer file.

Run the installer, select your desired language, accept the license agreement, and install to your program files.

Screaming Frog runs on WIndows, Mac, Linux, and Ubuntu operating systems.

Step 2: Enter a URL to Crawl

Once installed, open the Screaming Frog application. You'll see a box prompting you to enter a URL:

Enter the homepage URL of the website you watn to crawl.

Step 3: Configure Your Crawl

Before crawling, you can click the Advanced button to configure settings:

Here you can set:

• Crawl scope - entire domain, subdomain, or limit to certain paths
• Crawl depth - levels of pages to crawl
• Threads - number of concurrent requests
• Crawl delay - time between page requests
• Custom extraction - add custom selectors

Use these to limit large sites or customize data collection.

Step 4: Start the Crawl

When ready, click the green "Start" button to initiate the crawl.

Screaming Frog will now begin crawling all pages within your defined scope, extracting optimization elements.

Depending on site size, this may take from a few minutes to a few hours to fully complete. You can pause/resume thge crawl as needed.

Step 5: Analyze Crawls Reports

Once crawling finishes, you can click on the various reports and filters to analyze your site's SEO:
Check for duplicate content under the Content report

• Review metadata under Page Titles and Meta Descriptions
• Find broken links under Status Codes > > Client Errors
• Assess internal equity with Internal links
• And more!

This data enables you to identify optimization opportunities.

Step 6: Export and Share Data

Lastly, export your desired reports as CSVs or Excel files. You can then:

• Analyze further in other tools
• Share with coworkers or clients
• Compare versions after making changes

Now you have an SEO audit to inform your optimization work!

Using the Data

ONce you've completed a site crawl, what should you actually do with all that data?

Here are some ways to leverage Screaming Frog SEO Spider's reports:

Find and Fix Broken Links

Broken links are a common technical SEO issue taht can hurt user experience and search rankings.

To find them in Screaming Frog:

• Go to Status Codes > > Client Errors
• Filter for 404s and 500s
• Export the list
• Redirect or remove each broken link

This improves site navigation and prevents crawl errors.

Optimize Duplicate COntent

Unintended duplicate content on a site can confuse search engines.

Use Screaming Frog to:

• Go to Content > > Duplicate Content
• Review exact and near-duplicate pages
• Consolidate or refresh the duplicated pages with unique content

This helps search engines properly index and value your pages.

Enhance Page Speed

Slow page speed hurts SEO and UX. Analyze using Screaming Frog:

• Go to Response Times report
• Sort pages by longest load time
• Diagnose and optimize the slowest pages

Improving page speed improves conversions and SEO.

Refresh Metadata

Stale or missing metadata negatively impacts clickthrough rates.

Leverage Screaming Frog to:

• Check Page Titles and Meta Descriptions
• Find pages with missing or duplicate data
• Update titles and descriptions to be compelling and unique

Optimized metadata boosts CTRs and traffic.

Review Indexability

Certain pages may be blocked from search indexing.

Confirm proper status with Screaming Frog:

• Go to Indexability report
• Check for noindex or nofollow pages
• Ensure important pages are indexable

This helps maximize search visibility and discovery.

Pricing

Screaming Frog offers both free and paid versions of its SEO Spider tool:

Free Version

• Crawl up to 500 URLs
• Limited features and settings
• Can not save crawl data

Paid Version

• £& pound; 149 (|$180 USD) per year
• Unlimited crawls
• Full features and customization
• Bulk discounts available
• Free updates during license period

The paid version unlocks the full power of Screaming Frog, while the free version is great for smaller sites.

They also offer discounted acedemic and charity licenses.

Compared to paid alternatives like Botify and DeepCrawl, Screaming Frog provides excellent value given its robust feature set and competitive pricing.

Pros and Cons

Here are the main pros and cons of using Screaming Frog SEO Spider:

Pros

• Very affordable pricing
• Tons of useful features
• Easy to install and use
• FAst and efficient site crawls
• Granular customization and filtering
• Desktop software with no cloud limits
• Crawls dynamic JavaScript sites
• Integrates APIs for data enrichment
• Excellent for technical SEO audits

Cons

• No web interface or cloud version
• Steep learning curve for beginners
• Very limited capabilities in free version
• No rank tracking or backlink analysis
• Interface looks a bit dated visually

Overall, Screaming Frog provides immense value - just be ready to invest some time upfront learning the tool.

Who Should Use Screaming Frog?

Here are the types of users who can benefit most from Screaming Frog SEO Spider:

• SEO Professionals - analyze client websites for optimization opportunities
• Web Developers - test sites for technical SEO best practices
• Digital Marketers - research competitors' on-page SEO
• Agencies - scale audits across large accounts and properties
• Site Owners - improve their own site's technical SEO and content

It's built for technical SEO, so some marketing generalists may prefer an all-in-one SEO platform. But for most serious SEO practitioners, Screaming Frog is an essential tool.

Screaming Frog Alternatives

Screaming Frog SEO Spider is one of the top website crawlers in the industry. But here are some alternatives worth considering:

• Botify - cloud-based crawler for large-scale SEO
• DeepCrawl - robust crawler with campaign management
• OnCrawl - smart crawler with page-level insights
• Sitebulb - broad SEO platform including crawling
• SEMrush - huge SEO toolkit including a site auditor
• Ahrefs - backlink analysis and site crawler
• Ryte - AI-powered crawler and website auditor

While Screaming Frog focuses specifically on technical SEO crawling, some tools above offer more all-in-one SEO capabilities.

Evaluate your specific needs to select the right fit.

FAQs

Is Screaming Frog free?

Screaming Frog offers a free version with limited functionality. It allows up to 500 URL crawls. The paid version unlocks additional features and unlimited crawls.

Does Screaming Frog Check backlinks?

No, Screaming Frog focuses specifically on crawling the on-page content of a website. It does not provide backlink analysis. Alternative tools like Ahrefs and SEMrush are better for backlink tracking.

Is Screaming Frog safe to use?

Yes, Screaming Frog is safe to use on your own website. Be careful about excessive use on third-party sites, as very frequent crawling may be perceived as an attack.

Does Screaming Frog work on mobile sites?

Yes, Screaming Frog can crawl and audit mobile websites. You can even configure the user agent to mimic mobile bots.

Can Screaming Frog render JavaScript?

Yes, the paid versions of Screaming Frog SEO Spider can execute JavaScript using headless Chromium browser rendering. This allows crawling of JavaScript-heavy sites.

How long do Screaming Frog crawls take?

Crawl times vary dramatically based on site size adn configuration. Small sites may take minutes, while large ecommerce sites may take hours. Use features like crawl delay and scope to manage large crawls.

Screaming Frog: Powerful SEO Crawler

In summary, Screaming Frog SEO Spider is an invaluable tool for comprehensively auditing the technical SEO of websites. With robust configuration options and detailed reporting, it provides immense value at a very affordable price point.

If you regularly need to analyze on-page optimization for clients or your company, Screaming Frog can make the process far more efficient. While beginners may face a learning curve, taking time to master it pays dividends through actionable SEO insights.

To learn more and download Screaming Frog SEO Spider, visit their official website.

---