Help me find & specify the best solution to manage my data processing

I am looking for an individual who would help me find / come up with / specify the best automated solution for data processing. The goal is to come up with such a solution and its specification that could be then used as another job posting on Toogit.

The current situation is:
- at the moment we are processing large amounts of data, mostly points of interest (POI)
- some are gathered via automated scrapers and verified manually, some are collected entirely manually
- we have no problem gathering the data but we struggle with updating them - we can do it, it just is very time consuming.

I am seeking an automated solution, a database solution of sort, which would keep our data up to date. That means:
- get data, either by API or by some manual import (CSV, JSON,...);
- compare those data with what it has in storage;
- update stored data accordingly;
- allow distribution of stored data, either as whole or changes only.

I would prefer the possible solution to use the G Suite infrastructure, but its not a must-have.

For your cover letter please consider the following scenario and suggest a solution:
- we are processing an ATM database
- this database is delivered to our clients on monthly basis
- clients dont care whether the database is delivered as a whole (however in this case they require each item to have a unique identifier) or in form of changes only
- data get scraped and a human sorts them to a particular structure so all information are comparable
- we need a solution that wouldnt neccessarily eliminate the human-sorting labour but that would ensure that data processed in this way are processed with already existing data resulting in an up-to-date data set.
About the recuiter
Member since Sep 12, 2017
Marina Kuznetso
from Moscow, Russia

---

---

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions