← Meta questions
System DesignReported in a candidate post, Jul 2025

Design a web crawler

Filed under Meta · Also reported at Grab, Reddit

Design a web crawler. Describe how it discovers, fetches and stores pages at scale, and explain how you would avoid crawling the same web page again within the same iteration.

You've read your 3 free write-ups

Sign up free to keep reading. Your account opens the first 10 questions of every company.

Related Meta questions

Collected from public candidate posts and coding forums. Updated Nov 2026.