Create a database with data from individual property entries - each apartment, detached, land, building, farm, garage, office - up to a max of 34 fields - scrapping 8 real estate websites.
The tool will identify the new property listing, date it, provide a unique internal id and add them to the database.
In case of changes to a previous listing, it is considered as a new property listing. If the property becomes unavailable, then a unique field called 'availability' is updated with a 'No'. Ideally in the future, changes to a listing are tracked in a specific field created for that purpose but at this point we can leave this out of the development.
The scrapper needs to be fast, run in 5h at most for all 7 websites and all properties.
Extraction per demand or daily and easily read in Excel/Access or exportable to csv.
IP unblockable.
The work needs to include the setup of an AWS free account or equivalent.
Have a simple backoffice to be able identify a few keywords (22 at most) within the text of each individual property entry and those keywords need to be defined by the user, some have already been identified and the list is provided at the end. Please beware that keywords might be expressions of up to 3 words. Each keyword is, per design, a field on the database. Hence some sort of backoffice will be important to easily setup.
As an additional challenge, some of the target websites only exist in the Portuguese language which will be harder for an English native speaker to test but you need to be able to cope.
The tool needs to backup the previous day data.
About the recuiterMember since Mar 14, 2020 Shyam S.
from Uttar Pradesh, India