i have already build scraper
but the site are scraping data from is keep blocking our ips
so i need to avoid bot detection
and right new we are scraping 100 links for this site every mint
i need to scrap 1000 links every mint
here is the flow
right now scraper ... scrapes 100 cars and store them in data base
then next time when scraper , scrapes data it compare the results and if the there is any new link in the list .. it update the list
i am want the same thing with 1000 links
those are main two requirements
there are one also that make the server crash .. that need to be resolved as well
I will prove all the code as well for my existing scraper","employmentType":["FULL_TIME","PART_TIME","CONTRACTOR","TEMPORARY","PER_DIEM"],"jobLocationType":"TELECOMMUTE","hiringOrganization":{"@type":"Organization","name":"Toogit","sameAs":"https://www.toogit.com/","logo":"https://www.toogit.com/images/toogit_logo_initial.png"},"identifier":{"@type":"PropertyValue","name":"Toogit","value":304218},"skills":["software development","Website Development"],"applicantLocationRequirements":[{"@type":"Country","name":"IN"},{"@type":"Country","name":"Canada"},{"@type":"Country","name":"USA"},{"@type":"Country","name":"Germany"},{"@type":"Country","name":"Pakistan"},{"@type":"Country","name":"Philippines"},{"@type":"Country","name":"Indonesia"},{"@type":"Country","name":"Sri Lanka"},{"@type":"Country","name":"Nigeria"},{"@type":"Country","name":"China"},{"@type":"Country","name":"Russia"},{"@type":"Country","name":"Bangladesh"}],"validThrough":"2024-08-25T08:12:20+05:30","url":"https://www.toogit.com/freelance-jobs/MzA0MjE4"}
Remote Web Development Job In IT And Programming
Find more Web Development remote jobs posted recently Worldwide
Work from Anywhere
40 hrs / weekHourly Type
Remote Job$28.74
Cost Looking for help? Checkout our video tutorial
How to search and apply for jobs
How to apply? Do you have more questions about the Job?
See frequently asked questions
We are looking for a specialist who can tweak and maintain Linux servers for us when needed. We usually work with these servers
We are working on a new project that will hold more than 10 million records and will search through them. Need yo...read more
We need a Bolty Blocks product expert to help analyze their developer solution with the following deliverables. I have a PPT for the format for most of the content
1) App / Solution Development walkthrough - provide 20, 30 or more screensho...read more
I need someone who can integrate a contract and communications with clients in zapier or some automated system. I have aquity scheduling, active campaign, zoom, google docs, jotform.
I am building a new website for my consulting work and am looking for someone familiar with wordpress to help with design, layout, and the small details Im not thinking about.
The ideal person will be responsive and detail oriented and willing...read more
We are looking for a Druid consultant with the following criteria and scope of work:
1. Has good knowledge of Apache Druid and has worked with this technology in the past.
2. Will need to conduct a POC using Druid consisting of the foll...read more