Spicrawl now works with OpenAI Agents SDK, LangChain, Codex, Claude Code and more.See integrations
Skip to content
Spicrawl homeBeta
Signup

Live world data for agents

Unlock world data for your agents/systems, build better products.

Signup(free while beta)
Try:

Try it free: any public page as clean Markdown, no sign-up.

Start from a working example

Open-source tools, plugins and SDKs built on Spicrawl. Clone one, add your API key, ship.

Browse all repositories

Everything the Spicrawl web scraping API does

Markdown and JSON output, JavaScript rendering, Cloudflare and anti-bot handling, browser actions, sessions, batch jobs up to 10,000 URLs, structured extraction, network capture, screenshots and PDFs, and caching. Each one is a parameter on the same API, and each is also available from the CLI and the MCP server.

3/3Served, billed for that tier

Plain, JavaScript or full browser

  • 1, 3 or 8 credits by tier. mode: auto starts cheap and climbs only when needed.
  • You pay for the tier that served. Failures cost 0.
  • js_render
  • mode
  • wait_for
render.sh
curl https://api.spicrawl.com/v1/scrape \  -H "Authorization: Bearer spicrawl_live_…" \  -H "Content-Type: application/json" \  -d '{    "url": "https://example.com/products",    "js_render": true,    "wait_for": ".product-card",    "response_format": "json"  }'

spicrawl scrape <url> --render

Read the guide

Explore the API docs

Try it on a page you scrape today

One API key, 1,000 free credits a month, no credit card.

Signup (free while beta)Read the quickstart

Up to 97% fewer tokens than raw HTML

Menus, scripts and ads are stripped before the page reaches your model, so the same facts take a fraction of the context. Raw HTML on the left, Spicrawl Markdown on the right.

Spicrawl docs page
−97%tokens
GitHub README
−94%tokens
Wikipedia article
−76%tokens
See how we measured

Measure the savings on your own URLs

Run the same pages both ways and compare token counts.

Signup (free while beta)See pricing

Two ways in: ask your agent, or call the API

Describe the page in plain words, or send one POST request. Both return the same clean output.

  1. Just ask your AI

    Connect once, then ask in plain words. No code to write.

    Set up in Claude Code

  2. Call the API

    One POST returns Markdown, JSON or HTML. Batch up to 10,000 URLs.

    Read the API docs

Connect your AI agent with Spicrawl MCP

A hosted MCP server: scrape, batch, sessions and browser actions, with no glue code.

MCP endpoint: https://mcp.spicrawl.com/mcp

  • Scrape
  • Batch jobs
  • Sessions
  • Actions
  • Request history
  • Usage
  • Docs
Set up the MCP server Read the integration docs
Claude Code scrapes a page

Are you an AI agent?

Fetch the skill and start scraping.

$curl -s https://docs.spicrawl.com/skill.md

View the skill llms.txt

Works with the tools you already use

Give your agent the web in one prompt

Works with Claude Code, Cursor, Codex and any MCP client.

Signup (free while beta)Agent quickstart

Get Markdown, JSON or structured data from one request

Pick the format your next step needs: Markdown for a prompt, JSON for your app, a screenshot or PDF for a record.

example.com/product/1
Meet Model X — Precision in every click
Wireless Mouse — Model X
Wireless Mouse — Model X
$49.00In stock★ 4.6 (1,284)
Low-latency 2.4 GHz receiver, 70-day battery life and silent switches.
SpecValue
Connection2.4 GHz, BT5Battery70 days
<script type="application/ld+json">
result.md
# Wireless Mouse — Model X**$49.00** · In stock · 4.6 (1,284 reviews)Low-latency 2.4 GHz receiver, 70-day battery lifeand silent switches.| Spec      | Value        || --------- | ------------ || Connection| 2.4 GHz, BT5 || Battery   | 70 days      |
Reading the page
Raw HTML12,480 tokens
Markdown0 tokens

Menus, footers and ads removed: 3% to 24% of the raw HTML tokens. Example page.

Convert any website to Markdown for LLMs

Scrape JavaScript sites, logins and dynamic pages

Start with a plain fetch. Spicrawl moves up to a real browser only when the page needs one. Keep a session to stay logged in, or bring your own proxy.

Spicrawl control panel
Auto escalate
Browser
Fingerprint
Proxy
Managed proxies: coming soon
Session
Actions
Block assets
Network capture
Cache
POST /v1/scrape
{  "url": "https://shop.example/product/1",  "mode": "auto",  "cache": true,  "response_format": "markdown"}
Example response headers
X-Target-Status
200
X-Engine
fetch
X-Proxy-Source
direct
Cache-State
hit
X-Credits-Charged
1
Auto escalation

Starts with a plain request and moves up to a browser only if the site blocks it. Blocked requests cost nothing.

Example only, not a live request. The beta is free.

Test in the dashboard, then ship with the API or CLI

Try a URL, copy the request, and run it from your code or your terminal.

CLI

Scrape one URL, batch thousands and manage sessions from your terminal.

Read CLI docs

Dashboard

Run test requests, read logs and usage, and manage API keys.

Open dashboard

Docs

Guides, the API reference and an OpenAPI spec for your tooling.

Read the docs

Try what people build with Spicrawl

Small working tools, each built on the Spicrawl API.

  1. change-monitor.spicrawl.com

    Change monitor

    Re-check a page and see what changed.

    Try it live
  2. price-tracker.spicrawl.com

    Price tracker

    Paste a product link, get its price.

    Try it live
  3. docs2skill.spicrawl.com

    Docs → Skill

    Turn a docs site into an agent skill.

    Try it live

What you can build with a web scraping API

Each job links to a guide in the docs, with working code.

  1. 01

    Feed LLMs and RAG pipelines

    Turn articles, docs and PDFs into clean Markdown, ready to embed.

    Markdown for LLMs
  2. 02

    Give AI agents web access

    Connect Claude Code, Cursor or Codex to read live pages.

    Build with agents
  3. 03

    Collect product and price data

    Pull names, prices and stock into JSON.

    Extract structured data
  4. 04

    Scrape thousands of URLs

    Send up to 10,000 URLs in one batch job.

    Run a batch job
  5. 05

    Read pages behind a login

    Log in once, reuse the session on every page.

    Use sessions
  6. 06

    Capture the data behind a page

    Read the JSON a page loads instead of parsing HTML.

    Capture network calls

Pay per page, only when it works

You get 1,000 free credits every month, with no credit card. A failed request, bot challenge or timeout costs nothing. Free while in beta.

1credit a page
Plain fetch
Static pages. The default.
3credits a page
JavaScript rendering
Pages that build themselves in the browser.
8credits a page
Full browser
Screenshots, PDFs, clicks and forms.

1,000 free credits a month is 1,000 plain, 333 JavaScript or 125 full-browser pages.

Signup (free while beta)See full pricing

Frequently asked questions

What is Spicrawl?

Spicrawl is a web scraping API that turns any URL into clean Markdown, HTML, a PDF or structured JSON for your code or AI agent, through an API, CLI or MCP server.

Is Spicrawl free?

Yes. It is free during the beta, with 1,000 credits a month and no credit card.

Can I scrape JavaScript websites?

Yes. Set js_render to true to load the page in a real browser. With mode=auto, Spicrawl tries a plain request first and uses a browser only if needed.

Can Spicrawl get past bot protection like Cloudflare?

Often, but not always. Use a real browser fingerprint, browser rendering or your own proxy. A bot challenge returns a clear error and costs nothing. Stealth mode and managed proxies are not in the beta.

Can I use Spicrawl with AI agents?

Yes. The hosted MCP server lets Claude Code, Cursor and Codex scrape pages, run batches, reuse sessions and run browser actions. Run spicrawl init to set it up.

Do failed requests cost credits?

No. Only successful requests cost credits: 1 for plain, 3 for JavaScript rendering and 8 for full browser. Errors, bot challenges and timeouts are free.

All questions