Crawling
Fast HTTP/2 crawling, robots.txt and sitemaps, pause and resume, optional JavaScript rendering and sites behind a login.
Free desktop app · Windows
Seobify Spider: a free SEO crawler
Crawl your site on your own computer and see technical SEO issues in order of priority. There is no URL limit and no account to create.
Windows 10 and 11, 64-bit. The installer is not code-signed yet: if Windows shows "Windows protected your PC", choose "More info" and then "Run anyway".
- Price
- Free, commercial use included
- System
- Windows 10 and 11 (64-bit)
- URL limit
- None
- Account
- Not needed
- Your data
- Stays on your computer
- Install
- No administrator rights needed
01 · Audit
What it finds
Audit
200+ SEO rules, plus duplicate content, canonical and hreflang audits, broken links, redirect chains, internal PageRank, a site health score, accessibility (WCAG 2.1 AA) and Turkish readability.
AI search visibility (GEO)
llms.txt checks, an AI crawler access audit and a citability score.
Reports
HTML, CSV and XLSX reports, Search Console and GA4 joins and scheduled crawls.
02 · Priorities
It tells you what to fix first
Spider does more than list every finding. It groups issues that share a cause and ranks the first jobs under "What should I do now?". If the same logo without alt text appears on 200 pages, you see one job instead of 200 rows.
Each job comes with a fix task written for your platform (WordPress, Shopify, Astro and others). Once you publish the fix, the "I fixed it, verify" button checks a sample of that job's pages on the live site again.
03 · AI agents
It works with your AI agent
The MCP server in the installer connects Spider to Claude Code, Codex, Cursor and Claude Desktop. The agent crawls the site, takes the top jobs, makes the fix in your site's code and verifies the result on the live site. It works while Spider is open. For Claude Code, in PowerShell:
claude mcp add seobify-spider -- "$env:LOCALAPPDATA\Seobify Spider\seobify-engine.exe" --mcpThe Seobify Hub tools Log and Migrate also read Spider crawls.
04 · Content trace
It finds what you cannot see in the text
- invisible characters mixed into the text (zero-width spaces, direction marks) and Cyrillic or Greek letters that look like Latin ones,
- attributes left in the HTML by text pasted from a chat tool,
- uniform writing where every sentence has the same length, and stock transition phrases.
If you want, you can have a page rewritten with your own AI key. Spider cleans the characters first, then has the text rewritten in your site's tone of address and terms. It keeps numbers, proper names and links, and shows the changes before and after, side by side.
05 · Privacy
Your data stays on your computer
Crawls and settings are stored on your computer. The app has no analytics or tracking code. Spider only connects to:
- the sites you ask it to crawl,
- the downloads repository, to check for updates,
- the Chromium download, the first time you turn on JavaScript rendering,
- Google Search Console, GA4, PageSpeed Insights and your chosen AI provider, only if you connect them (with your own key).
06 · Install
Install in three steps
- Download
Seobify.Spider_<version>_x64-setup.exefrom the releases page. - Run the file. You do not need Python or any other program.
- Open the app, type your site's address and start the crawl. When a new version is out, the app asks you first; it never installs silently.
07 · FAQ
Frequently asked questions
Is it really free?
Yes. There is no paid edition: no URL limit, and JavaScript rendering, Search Console and GA4 connections and saved crawls are open to everyone. Personal and commercial use, including client work, is allowed (terms of use).
How is it different from Screaming Frog?
The free edition of Screaming Frog stops at 500 URLs per crawl, and JavaScript rendering, the Search Console, Analytics and PageSpeed connections and saving crawls need a paid licence (Screaming Frog pricing, November 2026). All of these are free in Spider. Screaming Frog also runs on macOS and Linux; Spider runs on Windows only.
Does it run on Mac or Linux?
Not yet. Spider runs on Windows 10 and 11 (64-bit).
Will crawling slow down my server?
By default Spider sends at most 5 requests at a time and follows robots.txt. If the site answers 429 or 5xx, it waits. On a small server you can add a delay between requests. Details for site owners: Seobify crawler identities.
Do the AI features cost extra?
Spider charges nothing. The AI features connect to the provider you choose (OpenAI, Anthropic, Gemini) with your own API key, and that provider bills you. You can also use a model running on your computer through Ollama.
Why does Windows show a warning?
The installer is not code-signed yet, and Windows warns about new unsigned files with "Windows protected your PC". Choose "More info" and then "Run anyway". Download the file only from the official releases page.
We can apply the findings for you
Spider shows what needs to change. If you want it done, we make the fixes on your site and measure the result after release.