At NewsHack Day in San Francisco, the team behind Haystax started with a wish list, outlining several common types of public databases users might want to scrape. Here's an outline of our next steps. In each case, the desired outcome would be to download a complete copy of the database, preferably in CSV format.
Accepts a null search, returning a paginated table of all results. Although it appears as a simple table, it does require users to navigate through to detail pages. Up to 100 results are returned at a time, depending on a parameter set by the user from a drop down box.
Search results limited to 200 results.State Bar of CA
Search results limited to 500 results. Contains detail pages.Obama-Biden transition team memos
Uses div tags instead of HTML tables. Requires file downloads.