Need a Website Crawler that can crawl certain websites and website searches everyday for 'updated' and 'newly' added listings to from site. Will added more crawlers as we go along.
I need crawlers that can crawl selected websites and website searches for 'updated' listings those sites or and/or searches within those sites.
For example, I may give a site that holds a listing of contractors in Nebraska. The Crawler will do an initial crawl to collect all of the contractors listings already on the site, and then crawl everyday to spit out 'newly' added listings on that site. This set crawler will do this continually, every day, spitting out newly added listings on this site if the leads are good.
After that is successful, then we will select another site to do the same thing in conjunction with the previous.. For example, We may choose to do a specific search. For example, we may choose to crawl a search in yellowpages.com for 'Handyman' in Los Angelas Again, the Crawler will do an initial crawl to index or see what's already there (because I don't care about old listing), and then it will crawl daily to pick up and spit out newly added listings to this search within the website.
Then we select another site or search within that site to create another crawler to continuously crawl etc
About the recuiterMember since May 20, 2018 Mark Wiles
from Lombardia, Italy