Crawlio
Download any website. Browse it offline.
Three steps to offline
Crawl
Point it at any URL. The engine maps every page, stylesheet, script, image, and font — following links with BFS or depth-first traversal.
Download
Concurrent connections pull everything in parallel. Per-host fairness, retry logic, and respect for robots.txt keep things clean.
Browse
Links are rewritten to work from your filesystem. Open index.html in any browser — the whole site works offline.
Built for real-world crawling
Swift actors handle all shared state. TaskGroups manage parallelism. No data races, no locks.
actor CrawlEngine {
func startCrawl(url: URL, destination: String) async throws {
await withTaskGroup(of: DownloadResult?.self) { group in
while let entry = frontier.dequeue() {
group.addTask { await self.downloadAndParse(entry) }
}
}
}
}Five parsers. Every link found.
HTML
Custom scanner. 18 tags, srcset, style blocks, data-* attributes. Encoding detection across 9 charsets.
CSS
@import chains, url() references, comment stripping. Quoted and unquoted paths.
PDFKit integration. Link annotations and in-text URL pattern matching.
Sitemap
XML urlset and sitemapindex. Transparent gzip decompression.
Robots.txt
User-agent matching, wildcards, end anchors, crawl-delay, sitemap directives.
Every link rewritten
Attribute-targeted replacement. Text content stays untouched.
<!-- Before -->
<a href="https://example.com/about">About</a>
<img src="https://example.com/images/logo.png"><!-- After -->
<a href="about.html">About</a>
<img src="images/logo.png">Four ways out
Folder
Complete directory tree. Open index.html in any browser.
ZIP
Compressed archive. Share, backup, transfer.
Single HTML
One self-contained file. All CSS, JS, and images inlined as data URIs.
WARC
ISO 28500 web archive. Compatible with Wayback Machine and warctools.
Secure by default
Authentication
HTTP Basic, Digest, SSL trust. Credentials stored in macOS Keychain.
Robots.txt
Respected by default. Wildcards, allow/disallow, crawl-delay.
Path traversal protection
Three-layer validation: PathMapper, DownloadController, ExportManager.
Tracking strip
UTM and analytics parameters removed before download.
Download Crawlio
macOS 14+. Free and open source.
Download v1.0.0Direct download — no App Store required