We will need a programmer well versed in CasperJs, nodeJS and the HTTPs module for airline scraping.
The script takes in 5 parameters:
Departure Date,
Return Date,
Departure City
Destination City
Number of passengers
This information may need to populated manually based on fields in the HTML. The output returned from the airline will be a JSON of flights information. Some of the information on the flights' results page may be needed to be pulled from the HTTP response of the airline server. The spec of the JSON can be shared during negotiation.
The main goal is to be able to reuse the cookies so that multiple (searches / HTTP) requests can be done with the same cookies essentially turning the website back end into an API. The website in question send multiple HTTP requests and one of them is for the flight results we need.
When the cookies have timed out, new cookies should be created so that we can use them to make more queries.
This should be done with performance and scale in mind. Part of the deliverable is performance analysis showing how many concurrent scrapes can be done before issues come up if any.
The simple scraping itself is already done, so the only goal of this project is to get the scraping working with same cookies so that we can run multiple queries at the same time.
About the recuiterMember since Jul 18, 2017 Kai C.
from Veracruz, Mexico