API Docs — Scrappilot

Overview

Scrappilot provides a single API endpoint to scrape any public web page. We handle the rendering, HTML cleaning, and Markdown conversion. Just send us a URL, get back clean content.

Simple

One POST endpoint. Send URL, get content back.

Credit-based

Pay as you go. 1 credit = 1 page. No monthly fees.

Three formats

Get Markdown, HTML, or plain text output.

Authentication

Every request requires an API key. Pass it in the Authorization header. Get your key from the dashboard after signing up.

Authorization: sk_live_abc123def456

Endpoints

POST /api/scrape

Scrape a URL and return the content in your chosen format.

Parameter	Type	Default	Description
url	string	—	The URL to scrape (required)
format	string	`"md"`	Output format: `md`, `html`, or `text`
js	boolean	`false`	Enable JavaScript rendering (uses more credits — see pricing)

Parameters in detail

url required

The fully qualified URL to scrape. Must include the protocol (https://).

format optional, default: "md"

Choose the output format:

"md" — Clean Markdown (HTML tags stripped, headings/links converted)
"html" — Raw HTML as returned by Crawlbase
"text" — Plain text (all HTML tags stripped)

js optional, default: false

Enable JavaScript rendering for SPAs and dynamically loaded content. Note: JS requests consume 25 credits instead of 1 due to higher Crawlbase costs.

Response format

All responses are JSON. A successful scrape returns:

{

"url": "https://example.com",

"format": "md",

"result": "# Example Domain\n\nThis domain is for use...",

"markdown": "# Example Domain\n\nThis domain is for use...",

"credits_remaining": 499

}

Code examples

curl

curl https://www.scrappilot.com/api/scrape \
-H "Authorization: sk_live_xxx" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "format": "md"}'

Python

import httpx

resp = httpx.post(

"https://www.scrappilot.com/api/scrape",

headers={"Authorization": "sk_live_xxx"},

json={"url": "https://example.com", "format": "md"},

)

data = resp.json()

print(data["result"])

JavaScript (fetch)

const resp = await fetch("https://www.scrappilot.com/api/scrape", {

method: "POST",

headers: {

"Authorization": "sk_live_xxx",

"Content-Type": "application/json",

},

body: JSON.stringify({

url: "https://example.com",

format: "md",

}),

});

const data = await resp.json();

console.log(data.result);

PHP

$client = new \GuzzleHttp\Client();

$resp = $client->post("https://www.scrappilot.com/api/scrape", [

"headers" => ["Authorization" => "sk_live_xxx"],

"json" => ["url" => "https://example.com"],

]);

$data = json_decode($resp->getBody(), true);

echo $data["result"];

Ruby

require "httparty"

resp = HTTParty.post(

"https://www.scrappilot.com/api/scrape",

headers: { "Authorization" => "sk_live_xxx" },

body: { url: "https://example.com", format: "md" }.to_json,

)

puts resp["result"]

Error codes

Code	Meaning	Action
400	Bad request — missing or invalid `url` parameter	Check your request body
401	Missing or invalid API key	Get a key from your dashboard
402	Insufficient credits	Buy more credits
422	Invalid JSON body	Check your JSON syntax
500	Upstream scraping error	Retry with exponential backoff

Errors return a JSON body with a detail field describing the issue.

Pricing

Feature	Cost
Standard scrape (no JS)	1 credit
JavaScript-rendered scrape	25 credits
Free credits on signup	100 credits
5,000 credits pack	$4 (buy)
50,000 credits pack	$29 (buy)

Grok Build plugin

Scrappilot has an official plugin for Grok Build. Install it and scrape URLs directly from your agent conversations.

Install

/plugin install web-scraper

Usage

/scrape https://example.com --key sk_live_xxx --format md --js

Available flags

Flag	Description
--key	Your API key (required)
--format	Output format: md, html, or text (default: md)
--js	Enable JavaScript rendering

Rate limits

Rate limits depend on the Crawlbase plan tied to your token. Current limits:

Limit	Value
Max response size	~2 MB
Request timeout	30 seconds
Concurrent requests (approx)	25

API Documentation