Backend developer needed to build a custom data scraper & parser service & API

The goal of this project is to scrape, sanitize, and organize the California State Bar Association data set. Specifically, to crawl a website for all attorneys. Data is classified by a variety of status parameters, and we are only interested in a specific subset.

From the individual attorney results, each unique datapoint will need to be stored against a key (potentially the state bar number). We do not have a specified schema, so we expect this schema to be developed based on the data. However, we will provide a list of specific fields that wed want to reconcile.

For example:
Matching and filling out a company field based on physical address and/or e-mail address when the company name is not present in the results.
Certain keywords will also be provided that will also need to be used to map specific fixed values.
Specific rules will be provided about how the presence of certain provided keywords from data we possess should add values to the specific record. There are also rules around what data wed want to exclude.

As an output, we will need all of the data in a single file in a machine-readable format (CSV, XML, etc.). Additionally, providing the data via a documented API that delivers the output in XML/JSON could be part of the initial scope or another phase.

While this custom scraper and parser project is initially focused on the California State Bar Associations members, there may be opportunities to work on subsequent projects around other state bar data depending on the success of this initial engagement.
About the recuiter
Member since May 20, 2018
Ali
from Osaka, Japan

---

---

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions