Read a public web page as clean text for an AI agent: title, description, main text (scripts, styles and navigation removed) and links. Respects robots.txt.