We are researchers from Princeton and Harvard University who are studying local government meetings. This job posting seeks someone to write a web scraper (for example, in Python) to download meeting record files from local government websites. We would provide you a list of local government names (e.g. New York City Council), and would ask you to write a scraper to collect meeting records (PDF and Word files) from government websites. These files are publicly available and posted on government websites, but too time-consuming to collect by hand. The final deliverable would be a collected set of files from these localities.
The most challenging part of this project is that websites are not standard between cities, so the web scraper will need to accommodate various paths to the files we need. For example, in one city the files may be posted on a 'City Council Meetings' page, while on another they may be under 'Meeting Minutes.'
We anticipate this being done most easily with Python, but we are open to any technology that will get the job done.
I will share a detailed version and more information with shortlisted candidates.
About the recuiterMember since May 20, 2018 Devin
from California, United States