Ensuring 2x Faster and 30% Cheaper Deployment on Azure
We collaborated with a startup that provides insights on airline discounts, good deals, and other travel-related information. The startup’s web scraping system involved a Flask API for triggering the scraping process, Celery workers for actual scraping, Redis for caching scraping results, and Kafka for communication. However, the startup faced challenges in deploying their system in a scalable and reliable manner.
Overview
Uitware collaborated with a startup that provides insights on airline discounts, good deals, and other travel-related information. The startup’s web scraping system involved a Flask API for triggering the scraping process, Celery workers for actual scraping, Redis for caching scraping results, and Kafka for communication. However, the startup faced challenges in deploying their system in a scalable and reliable manner.
Business Challenge
The startup required a cheap yet powerful containerized deployment solution on Azure, utilizing managed services where applicable to reduce operational overhead. The challenge was to set up a CI/CD solution, container orchestration solution, storage and database solution with simple yet effective ETL (Extract, Transform, Load). Additionally, the startup needed to minimize maintenance costs and administrative overhead.