Real-time Web Scraping for a list of Large News We...
Real-time Web Scraping for a list of Large News Website (need scalability)
Posted by - Winnersportswea at Sep 22, 2022 Yamagata, Japan
Toogit Instant Connect Enabled
Oct 13, 2022
The task is straight forward: design an script (python preferred) to scrape all article information from a News website in real-time.
There are a few requirements though: 1. Real-time: able to keep the program running and detect any new posted articles. 2. Web application compatible: the script should be capable of being integrated into a web application so that the article information can be pushed onto the web app realtime. 3. Production ready: the script should be able to handle most the exception and detect any anomaly and be production-ready 4. For those who are also web app developer, it would be a plus if you can also handle the integration with the web app.