web-search-and-content

Document Parsing, OCR, and Layout Quality. A document-extraction procedure that distinguishes native text from OCR, preserves page/bounding-box provenance, detects broken reading order and tables, exposes per-field confidence, and routes uncertain high-impact fields to visual review instead of trusting average OCR accuracy.

Building Your Own Web Crawler & Indexer. How to build a web crawler and search index: a from-scratch Python approach (requests + BeautifulSoup, deque/set queue management, robots.txt and scope control) or frameworks (Scrapy, Crawlee, Colly), then indexing crawled fields into Elasticsearch, Meilisearch, or SQLite FTS5 for full-text search.

URL Metadata Extract: URL title, description, Open Graph image and canonical metadata Executed deterministically by the AmanChain node against live consensus state (real blocks, pools, balances) — paid per call via x402, settled in USDC on Base. Input: JSON body {"serviceId":"aman-url-meta","request":" "}. No account, no API key.

Live Web Search: live web search with titles, URLs and snippets Executed deterministically by the AmanChain node against live consensus state (real blocks, pools, balances) — paid per call via x402, settled in USDC on Base. Input: JSON body {"serviceId":"aman-web-search","request":" "}. No account, no API key.

Keyword & Entity Extraction: extract keywords, entities and topics from text Executed deterministically by the AmanChain node against live consensus state (real blocks, pools, balances) — paid per call via x402, settled in USDC on Base. Input: JSON body {"serviceId":"aman-keywords","request":" "}. No account, no API key.

Use when an agent needs to read web pages: returns each page's main content as clean markdown (Readability + Turndown) with title, byline and word count. $0.002 per URL; pass up to 10 comma-separated URLs in 'urls' for one batched call. Not charged if every URL fails.

Academic Paper Search (250M+ works, OpenAlex). Academic paper search across 250M+ scholarly works (OpenAlex): title/abstract match, sort by relevance, citations or newest, filter by year and open access. Each paper: authors, venue, year, DOI, arXiv id, PMID, citation count, topic, open-access PDF link and abstract.

Read a Web Page as Clean Markdown. Web page to markdown: read any public URL as clean markdown for an agent — fetches the page (redirects followed), extracts the main article content, drops navigation, footers, cookie banners and link farms, and keeps headings, lists, links, code, tables and images, with title, author, publish date, language and canonical URL. ?max_chars= up to 100000, ?links=true for page links. Private addresses, error pages and non-HTML files are refused before payment.

Bulk readability score: up to 20 urls in one call, processed concurrently, results returned in input order with a per-item error field and a count of failures. Same answer per item as the single /site/readability endpoint (Readability of a page's text). Batch enrichment for agents that hold a list. $0.01 per batch.

Bulk entity extract heuristic: up to 20 urls in one call, processed concurrently, results returned in input order with a per-item error field and a count of failures. Same answer per item as the single /page/entities endpoint (Named-entity candidates from a page without an LLM). Batch enrichment for agents that hold a list. $0.01 per batch.

Bulk html fetch: up to 20 urls in one call, processed concurrently, results returned in input order with a per-item error field and a count of failures. Same answer per item as the single /page/html endpoint (Raw HTML fetch). Batch enrichment for agents that hold a list. $0.01 per batch.

Bulk analytics id extract: up to 20 urls in one call, processed concurrently, results returned in input order with a per-item error field and a count of failures. Same answer per item as the single /site/analytics-ids endpoint (Analytics and pixel ID extractor). Batch enrichment for agents that hold a list. $0.01 per batch.

Doc Tools: Parse an RSS, Atom or JSON Feed into normalized JSON items.

Doc Tools: List a site's URLs from its sitemaps (follows robots.txt and sitemap indexes).

Web Extract: Fetch a URL and return markdown, metadata and links.

Catalog of 19 synthetic datasets for AI agent training and testing — financial markets, x402 agent payment traces, MCP tool-call transcripts, healthcare claims, IoT sensor logs, and more. Each category is generated on demand and purchasable individually via /datasets/detail. Catalog access is priced at $0.035 per delivered catalog row per successful call.

Run fast HTML accessibility heuristics for language, title, headings, images, links, buttons, and form controls.

Doc Tools: Extract per-page text from a PDF URL (up to 20MB).

Capture a URL and return its structured content (title, headings, text, links, images) as JSON

Bulk extract domains from page: up to 20 urls in one call, processed concurrently, results in input order with a per-item error field and a failure count. Same answer per item as the single endpoint. Batch enrichment for agents that hold a list. $0.10 per batch.

Headless browser sessions, web search, and page fetching for AI agents.

Chat completions, embeddings, image generation, and audio with model-tier pricing.

Fetch the HTML of a page that blocks your requests, through a residential route. Fetch a page that blocks you, and get the HTML back. The request is routed through a French residential IP with a real Chromium browser, resolving JavaScript and passing the defences that reject datacenter ranges and plain HTTP clients. Use it as a fallback whenever your own fetch returns a challenge page, an empty shell or…

Website technology detection: CMS and frameworks (WordPress, Shopify, Next.js, React, Webflow...), analytics and tag managers, hosting/server and CDN hints, generator meta tag, from the live homepage. Tech stack lookup for sales prospecting, competitor research and integration planning. $0.05 per site.