Scraping employee data from law firm web sites and fixing existing scrapers. Currently, I have around 150 active scrapers that need to be maintained (fixed as they break). Scraped data flows into 'Datatables'. Ongoing maintenance/monitoring of scrapers and Datatables.
Skills needed:
Python (Main language)
Django (Server framework)
AWS Services (Hosting provider)
Requests (Python module for HTTP requests)
BeautifulSoup (Python module for parsing HTML)
dryscrape (Python module for rendering JS heavy sites)
Bootstrap (CSS framework)
Datatables
Docker-Compose
Items to be scraped include: 1) Attorney Name (Clickable) 2) Location 3) Position 4) E-mail.
You can refer to existing scrapers as a prototype for what I'm looking for.
Occasionally, I will have other types of tasks pertaining to the tool that go beyond scrapers.
Communication will take place via Slack and Trello.
I need someone who can implement website scrapers using requests and bs4. NOT selenium. Not a junior developer, at least intermediate. Also someone capable of working with hundreds of scrapers, where we pass parameters to a generic scraper class to make it do something rather than writing everything in python code.
About the recuiterMember since Mar 14, 2020 Pooja Shivaraj
from Pohjois-Pohjanmaa, Finland