Hello,
I am a German software developer and I have a German sportswear shop as a customer.
They run a successful online shop and sell sportswear from over 60 vendors.
My part in this business is to deal with the B2B part.
Together with another freelancer (who has gone for medical reasons), we developed a Python 3 based framework to retrieve B2B data from vendor sites through screen scraping.
We retrieve three kinds of data:
- product data: for a given list of article IDs, we search and extract article data like title, colours, images and product descriptions (57 vendors)
- stock data: for a given list of article IDs, we retrieve how many articles are on stock in the vendors B2B portals (30 vendors)
- orders: for seven vendors we implemented an automatic ordering of items (to reorder new products)
- images: in some case we need to retrieve article images from assets sites of some vendors (3 vendors)
So we have an active code base of about 100 grabbers!
The underlying framework is implemented on Python 3 and Selenium (Chrome/Chromium under the hood) and partly with Scrapy. Ever screen scraper type (e.g. for stocks) has a base grabber with a unified database structure for all vendors.
So specific grabber implement specific methods like 'get_product_title', 'get_desciption' etc. specific a vendor product or B2B site.
I am looking for am EXPERIENCED AND RELIABLE Python developer that is fluent in Python 3 and heavily experienced with screen scraping.
It is your task to develop new screen scrapers for new vendors, maintain the current code base (work on fixes and updates, if needed).
We basically communicate through Bitbucket issues (the code is on Bitbucket) and email if needed. I am the person between my customer and you. So new customer requirements and task first go through my hands before I assign them to you. Finished work is assigned back to me, I perform some QA and provide the work for testing on the customer's servers.
With my former Python developer, we agreed on fixed prices for new screen scrapers for new vendors and on hourly rate updates and bug fixes. The annual volume of work was always in the range of 3000-4000 USD per year. The fixed prices are different for stock, content, order and images grabbers depending on their complexity.
Experience and reliability is a must. I do not expect work to be delivered by the next day. Good communication and being able to work on your own is a major requirement.
The communication language with me is English. I am teaching my customer to provide all instructions in a more or less formalized way also in English.
All codes and issues are managed on Bitbucket in private repositories.
This is a long-running project. We need and expect long-time support. We don't expect cheap IT works, we want to work with smart and reliable developers.
Please state your hourly rate for updates and fixes.
The fixed prices for new screen scrapers will be agreed on a mutual basis.
About the recuiterMember since Jul 24, 2017 Bessie O.
from Washington, United States