- Sitemap URL Extractor & Change Monitor
Sitemap URL extractor that reads robots.txt, sitemap indexes, .xml.gz and plain-text sitemaps and returns one row per URL with lastmod, changefreq, priority — plus a new/removed diff between runs.
- Website to Markdown — Content Crawler for LLM & RAG
Content Crawler turns any site into clean Markdown per page for LLMs, RAG pipelines and vector DBs — no headless browser, $1 per 1,000 pages.
- RAG Text Chunker (Markdown to Chunks, Token Counts)
Split Markdown or text into RAG-ready chunks with exact OpenAI token counts (o200k/cl100k): heading-aware, code blocks kept whole, sentence-boundary overlap, heading path on every chunk. Chunks any dataset from Website to Markdown, PDF to Markdown or your own texts.
- llms.txt Generator (Website to llms.txt & llms-full.txt)
Turn any website or docs site into an llms.txt (sectioned links with one-line descriptions) and llms-full.txt (every page as clean Markdown) for AI assistants, RAG and AI search. Plain HTTP, sitemap-aware, robots.txt-friendly.
These are suggested research steps. Review each input schema and map the output fields between steps; each tool is billed separately at its listed price.