Extract every link from a public page for crawling agents: de-duplicated absolute URLs with anchor text, internal/external counts, declared feeds and a sitemap hint.