Screaming Frog released version 25.0 of its SEO Spider desktop crawler on November 3, 2026. This is more than a typical point release. The Screaming Frog SEO Spider 25.0 release notes describe a JSON-based config file that opens the tool up to Model Context Protocol (MCP) automation. The update also adds an auto-throttle feature for rate-limited crawls, a new Dashboard tab, configurable issue types, and a Social tab for Open Graph and X Card checks. None of this is flashy. Most of it solves problems technical SEOs and marketing teams hit every week, including crawls that get rate-limited mid-audit and reporting that takes too long to assemble by hand. This piece walks through what actually shipped in Screaming Frog SEO Spider 25.0, what each change means day to day, and which features deserve your time first.
What Changed in Screaming Frog SEO Spider 25.0
Dan Sharp, founder and director of Screaming Frog, published the update on the company’s blog under the internal codename “eaten your lunch.” A smaller bug-fix release, version 25.1, followed on November 4. It fixed a crash tied to Windows display scaling and bumped the tool’s Node.js component for security reasons, according to the comments on that same post.
The core Screaming Frog SEO Spider workflow itself has not changed much. You still point it at a domain, set crawl rules, and let it work through URLs. What 25.0 changes is how the tool is configured, how it behaves under pressure from slow or rate-limiting servers, and how results turn into something a client can actually read.
Eight features make up the bulk of the release:
- JSON config files, built for CLI and MCP automation
- MCP debugging tools that can flag a crawl problem and suggest a fix
- A Custom Dashboard tab with chart, counter, and table widgets
- Configurable issue types, priorities, and visibility per crawl
- Link exclusion by link path, not just by URL pattern
- Automatic throttling when a host returns 429 or 503 responses
- Filename tokens for exports, covering date, domain, project, and report name
- A Social tab checking Open Graph and X Card tags
Several smaller additions round out the release. There are four new issue types, covering missing image titles and hreflang mismatches. Scheduled crawls can now be paused and resumed. The CLI gained new arguments for managing crawls and projects. Sitemaps accept relative URLs. Bulk Markdown export is now supported. Turkish and Dutch language support was added too. Network Idle is now the default JavaScript rendering wait condition. Screaming Frog says this tracks closer to how Google decides a page has finished rendering.
The Screaming Frog SEO Spider MCP Integration, Explained
The feature getting the most attention is also the hardest to picture without seeing it run. MCP, or Model Context Protocol, is the standard that lets an AI assistant like Claude or ChatGPT call external tools directly. It skips the usual back-and-forth of just reading and writing text.
Screaming Frog had basic MCP support before 25.0. What changes now is the config format underneath it. Crawl settings live in a JSON file instead of the old proprietary format. That means an AI assistant can read and edit the file in a way that actually works, then hand instructions back to the Screaming Frog SEO Spider through MCP. Screaming Frog’s own post describes asking an assistant to enable JavaScript rendering, build an audience segment, write custom JavaScript for extraction, or generate an XPath expression. All of it happens from a plain-English request.
The second half of this is the MCP debugging tools. Say a crawl stalls, returns strange data, or misses pages it should have caught. The connected AI assistant can inspect the problem and propose a fix. You no longer have to dig through configuration menus alone to find what went wrong.
My take: this is a genuinely useful automation feature for teams running the same crawl setup across many client sites. Someone usually has to rebuild similar configs by hand each time, and that is exactly the kind of repetitive work AI assistants handle well. It is less useful for a solo practitioner running one or two sites a week. Opening the settings menu yourself takes about the same time as typing a prompt in that case. I would also flag the obvious point here. Handing crawl configuration to an AI assistant is still handing it a decision-making role. Anyone doing this for client work should check the output before it runs against a live site. DevByteDaily has covered the broader question of how much decision-making belongs with an AI agent in marketing work, along with the governance questions that come with it. Both are worth a read before wiring MCP tools into a production workflow.
Auto Throttle and Other Fixes Built for Real Crawls
Of everything in this release, auto throttle is the one most SEOs will actually feel. A crawler running at full speed against a server with aggressive rate limiting ends up blocked, throttled, or serving broken pages mid-crawl. That is a familiar problem for anyone who runs crawls regularly. Screaming Frog SEO Spider 25.0 now detects 429 and 503 responses, or a host that drops connections outright. It slows its own request rate in response, automatically.
Speed recovers gradually once the host stops pushing back. A new “Connection Failure Retries” setting under Config, Spider, Advanced controls how many rate-limited attempts the crawler makes before giving up. If the host stops responding entirely, the crawl pauses rather than failing outright. Throttling status now shows at the bottom of the interface, next to crawl speed.
This matters most for anyone auditing sites on shared infrastructure, smaller hosting plans, or servers behind a web application firewall tuned to flag automated traffic. I have had crawls get blocked partway through on exactly this kind of setup. Restarting a long crawl from scratch wastes real time on a client engagement. An automatic, self-correcting throttle removes a chunk of that manual babysitting.
Link exclusion by link path is a smaller but genuinely practical addition. Excluding a navigation menu or faceted filter block used to require a URL pattern. That does not work when the same URL shows up in multiple contexts on a page. Now you can exclude links based on where they sit structurally instead. It is a more accurate way to clean up crawl data from large ecommerce or directory sites.
Custom Dashboards and Configurable Issues
The new Dashboard tab lets you assemble charts, counters, and tables into a view built around whatever you are reporting on that week. You can build and save more than one dashboard, set a default, and export the result for a client summary. This directly answers a long-standing complaint about Screaming Frog SEO Spider. The raw export tables are thorough, but they take real spreadsheet work to turn into something presentable.
Configurable issues solve a related problem. Every crawl flags a long list of issues, warnings, and opportunities. Not every client or site needs to see all of them. The update lets you disable specific issues entirely, or reclassify them, from the right-hand Issues tab. An agency running an enterprise site audit can now downgrade or hide checks that do not apply to a client’s CMS setup. That beats explaining in every single report why a particular warning does not matter. It pairs naturally with the structured review covered in DevByteDaily’s enterprise SEO audit checklist, where cutting noise from a crawl report is half the job.
The new Social tab is a smaller but welcome addition. It surfaces Open Graph and X Card tags directly. It flags missing tags, broken image URLs, and incorrectly sized social images. Social preview problems now show up during a technical crawl, not after a post goes live with a broken link card.
Screaming Frog SEO Spider Pricing: Free vs Paid in 2026
Pricing has not changed with this release, but it matters for anyone deciding whether to upgrade or buy in for the first time. Screaming Frog’s own pricing page lists the free version with a 500-URL crawl limit. That is enough for a small business site, but not for most agency work. A single paid license for Screaming Frog SEO Spider runs 245 euros a year, with volume discounts starting at five licenses. The license lifts the crawl limit to what the company calls “unlimited.” The real ceiling still depends on your machine’s memory and storage, not a fixed number the software enforces.
One detail stands out here: Screaming Frog’s own feature comparison lists identical functionality for free and paid tiers, aside from the crawl limit. Every feature in 25.0, including the new MCP tools and the Dashboard tab, is available on the free tier too, as long as your crawl stays under 500 URLs. For a small site or a single-page audit, that is more than enough room to test the update before paying for anything.
How the New Screaming Frog SEO Spider Features Stack Up
Not every feature in this release deserves the same attention. Here is how I would rank the main Screaming Frog SEO Spider 25.0 additions for a typical in-house marketing team or small agency deciding where to spend time first.
| Feature | What It Solves | Best For | Priority |
|---|---|---|---|
| Auto throttle | Crawls blocked or slowed by rate limits | Agencies, multi-client audits | High |
| Configurable issues | Noisy reports with irrelevant warnings | Agencies sending client reports | High |
| Custom Dashboard | Manual spreadsheet work after every crawl | Teams reporting to stakeholders | Medium |
| MCP config and debugging | Repetitive crawl setup across many sites | Teams already using AI assistants | Medium |
| Link exclusion by path | Noisy crawl data from nav or filter links | Large ecommerce and directory sites | Medium |
| Social tab | Broken Open Graph and X Card previews | Content and social teams | Low to Medium |
| Filename tokens | Manually renaming export files | Anyone running scheduled exports | Low |
Who Should Install This Update
If you run technical SEO audits for multiple clients, install the Screaming Frog SEO Spider 25.0 update this week. The combination of auto throttle, configurable issues, and custom dashboards cuts real manual work out of a recurring process. None of the three require learning a new workflow from scratch.
If you run occasional one-off audits on your own site or a handful of small clients, the update is still worth having, but nothing in it is urgent. Auto throttle will quietly save you a restarted crawl at some point. The free tier gets everything, so there is no cost reason to wait.
The MCP and AI assistant integration inside Screaming Frog SEO Spider is the part I would treat as experimental rather than production-ready. It is a genuinely interesting direction for a desktop crawler to take. It also fits where a lot of SEO tooling is heading, as AI assistants get wired into more of the stack. But test it on a non-critical crawl first. Read what it changes before letting it run unattended. Treat its suggestions the way you would treat a junior analyst’s first draft: useful, but checked before it goes anywhere near a client site.
Screaming Frog SEO Spider has built its reputation on being a reliable, dependable tool for close to fifteen years. Version 25.0 adds genuinely useful reliability fixes alongside a bet on where AI-assisted tooling is going. Both halves of that are worth knowing about, even if you are not ready to use all of it yet.
Frequently Asked Questions
Does the free version of Screaming Frog SEO Spider get the new 25.0 features?
Yes. Screaming Frog’s pricing page lists identical feature sets for free and paid tiers. The Dashboard tab, MCP tools, auto throttle, and Social tab all work on the free version. The only limit is the 500-URL crawl cap.
What is MCP, and why does an SEO crawler need it?
MCP, or Model Context Protocol, is a standard that lets an AI assistant call external software tools directly, rather than only exchanging text. The new JSON config format in Screaming Frog SEO Spider makes it easier for an assistant to read and adjust crawl settings through that standard. That is why the company highlighted it as the headline change in 25.0.
Will upgrading to Screaming Frog SEO Spider 25.0 break my existing crawl setups?
Screaming Frog says the old config format still works, so existing setups should not break. The company does flag that PageSpeed and Lighthouse-related metrics changed in an earlier release, version 23.0. That is worth checking separately if automated reports rely on those specific scores.
Is auto throttle turned on by default?
The release notes describe auto throttle as part of standard crawl behavior in 25.0. A configurable “Connection Failure Retries” setting under Config, Spider, Advanced controls how many rate-limited attempts the crawler makes before it pauses.
Leave a response