Crawl pre-flight for a domain: will you be blocked, is content behind a paywall, and how much is actually there. Returns per-bot robots.txt verdicts for 16 AI crawlers, crawlable page count from sitemaps, detected CDN and paid-access signals (x402, TollBit), plus a fetch/skip recommendation. Call before spending requests on an unknown domain.
| Network | Scheme | Amount | Pay To |
|---|---|---|---|
| Base | exact | $0.02 USDC | 0x02d1...2bC2 |
| Polygon | exact | $0.02 USDC | 0x02d1...2bC2 |
| Arbitrum One | exact | $0.02 USDC | 0x02d1...2bC2 |