Discover a website's URLs in one call: reads robots.txt, its declared sitemap(s) (sitemap indexes and gzipped sitemaps included, /sitemap.xml as the fallback) and the start page's internal links, then returns a same-host, normalized, deduplicated list (up to 500) with an optional substring filter. Hard budgets: at most 6 fetches, 15 seconds, 5 MB. Use it to pick which pages to crawl or extract next.
| Network | Scheme | Amount | Pay To |
|---|---|---|---|
| Base | exact | $0.005000 USDC | 0xaBF4...a9D0 |
| Polygon | exact | $0.005000 USDC | 0xaBF4...a9D0 |
| Arbitrum One | exact | $0.005000 USDC | 0xaBF4...a9D0 |
| eip155:143 | exact | 0.01 tokens | 0xaBF4...a9D0 |
| eip155:43114 | exact | 0.01 tokens | 0xaBF4...a9D0 |
| eip155:1329 | exact | 0.01 tokens | 0xaBF4...a9D0 |
| Optimism | exact | $0.006000 USDC | 0xaBF4...a9D0 |
| eip155:4663 | exact | 0.01 tokens | 0xaBF4...a9D0 |
| Celo | exact | 0.01 tokens | 0xaBF4...a9D0 |
| Base | upto | $0.005000 USDC | 0xaBF4...a9D0 |
| solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp | exact | 0.01 tokens | J7aN3P...3xwg |
| stellar:pubnet | exact | 0.05 tokens | GDNJXC...WWRL |
| algorand:wGHE2Pwdvd7S12BL5FaOP20EGYesN73ktiC1qzkkit8= | exact | 0.01 tokens | C7IIHG...2XIE |