Introduction to the Democratic Crawler

Democratic Crawler is webtool that:

• is free!
• anyone can use to crawl their own, or other people's, websites
• accepts schedulled crawls
• respects robots.txt
• is easy to use in every way

For now you can only use it to retrieve out and inbound links and email addresses printed on a single webpage. To crawl a whole website would be costly for Bubaweb due to the fact that bandwidth is limited. But you can use this data for all kinds of things:

See how it looks like

Next version will incorporate:

- Quick registration form
- Cookies storage
- Inserted data validation (although main issues are covered)