K

Kairostamp · Robots Check

by Kairostamp

GETBase

$0.001

per call · USD Coin on Base

robots.txt check: fetches the origin's /robots.txt server-side and reports whether a given URL path is allowed or blocked for a user agent, with the matched rule, the robots URL, the HTTP status, a timestamp and a tamper-evident receipt. Query parameters: url (required), user_agent (optional, defaults to *). Only robots.txt is fetched; no crawling. Pair with GET /api/v1/link-map or /api/v1/sitemap to plan an allowed fetch.

Endpoint

GET https://kairostamp.com/api/v1/robots-check

1

Calls / 30d

1

Unique payers / 30d

Aug 23

Last called

exact

Payment scheme

Call this service

TypeScript · @x402/fetch
import { wrapFetchWithPayment } from "@x402/fetch";
import { privateKeyToAccount } from "viem/accounts";

const account = privateKeyToAccount(process.env.PRIVATE_KEY);
const fetchWithPay = wrapFetchWithPayment(fetch, account);

const res = await fetchWithPay("https://kairostamp.com/api/v1/robots-check", {
  method: "GET",
});
const data = await res.json();
cURL
curl -X GET \
  "https://kairostamp.com/api/v1/robots-check?url=https%3A%2F%2Fexample.com%2Fpricing&user_agent=GPTBot" \
  -H "X-PAYMENT: <signed x402 payment>"

Example response

{
  "chain_hash": "d084fff3e5d852089814f7d4d3ffe9d04d1a39ce9d2e0573a95f24cf78fb584d",
  "decision": "allowed",
  "fetched_at": "2026-08-23T05:32:12.771Z",
  "matched_rule": "Allow: /pricing",
  "matched_user_agent": "*",
  "note": "Decision derived from the origin's /robots.txt as fetched at this timestamp. Only that one file is fetched; the target URL itself is not requested and nothing is crawled. robots.txt is a crawler convention, not a legal permission.",
  "receipt_id": "85cd424c-b68b-440e-8f4f-e02147799f30",
  "robots_status": 200,
  "robots_url": "https://example.com/robots.txt",
  "target_url": "https://example.com/pricing",
  "user_agent": "GPTBot"
}

Payment details

Pay to0xdb6f694ae695c632c82677a50593b46137808453
AssetUSD Coin · 0x833589fcd6edb6e08f4c7c32d4f71b54bda02913
NetworksBase
Schemesexact

Is this your API?

Pin it to the top of Developer Tools and the homepage with a featured placement.

Get featured →

More from Kairostamp & similar services