Revolutionizing Real-Time Speech Recognition: Tarteel's move to a Multi-cloud AI Stack
increase in deployments
clusters managed through Zeet
8 Million Users
cloud providers
infrastructure engineer
Abstract
Tarteel, an AI-powered Quran learning platform, has successfully leveraged Zeet's infrastructure to enhance their service delivery and save money on engineers and GPUs across 4 clouds. With a small team and a complex AI infrastructure stack, Tarteel, like many start ups, was trying to keep the engineers free from deploy and infrastructure tasks. As Tarteel has used Zeet, they’ve been able to scale their infrastructure, serve 8 million users worldwide, manage 6 clusters across 4 clouds, dozens of GPUs, and get more productive - all with just 5 engineers and 1 infrastructure engineer.
"The demand for AI-powered learning assistants is rapidly increasing as students seek personalized and adaptive learning experiences. To deliver these experiences at scale, we need the right infrastructure, including powerful hardware, software, and cloud platforms that can handle the processing requirements of AI and machine learning workloads. With the right technology in place, we can empower students to achieve their full potential and help them succeed in the 21st-century..."
Backstory
Tarteel is harnessing the power of AI to enable millions of Muslims to get closer to their faith. With their AI-powered Quran learning assistant, they are serving users all around the world, and they’re doing this across 6 clusters on 4 different cloud providers (AWS, Coreweave, Linode, GCP).
Tarteel's team was already using Zeet to deploy their infrastructure and application. Tarteel’s CEO Anas Abou-Allaban is a second-time founder, so he knew that the main focus of any early stage company is building a product that people want, not managing the infrastructure to support it.
We were using AWS CDK before and it was such a pain to maintain prior to moving to Zeet.
I've worked at AWS and still working with AWS can be needlessly complex at times. - Anas Abou-Allaban, Co-Founder, Tarteel
Now the next challenge they were facing is a growing user base. Tarteel’s global users were growing quickly. So the team needed to increase the number of GPUs simply to handle the demand for the product and ensure a realtime experience with their mistake detection features. Those are good problems to have as a small company, but those are time consuming problems that require effort, not just money to solve.
Like most small-to-medium sized companies, Tarteel's pain points revolved around shipping product and scaling while on a small-company budget. They have engineering horsepower on the team, but where to allocate that is as much a decision around productivity as it is finances.
That begged the question: how can they meet growing user needs and support the growth and product roadmap? Will they need more engineers to build out the supporting infrastructure?
"I think it's a mistake, especially for early-stage companies, to try to manage infrastructure without the help of internal tools. It just does not make sense from a financial or time perspective. There's no reason to grow a dedicated Devops team when you can have a development platform instead. You should be focusing on product-market fit, your product, getting money in the bank, and improving your customer satisfaction."
Solution
Zeet became an important part of the solution for Tarteel. They keyed in on a few Zeet features to help them:
- Zeet’s multi-cloud capability gave them access to clouds with PoPs worldwide and GPUs at a moment's notice.
- Zeet’s cluster management and auto-scaling would allow them to build out the infrastructure needed to support their global user needs
- Zeet’s dashboard and API interfaces made it so the time to do all these complex operations is in the minutes and hours, not days or weeks.
Even with an AI stack and complex requirements, Tarteel's transition was seamless and quick.
Tarteel has 4 clouds for various needs, from hosting their applications and the infrastructure and APIs that support the product, as well as NVIDIA V100 and A100 Tensor Core GPU in the cloud for inference. They were able to connect all of these to Zeet in a few minutes, and from there, manage a half dozen clusters from the Zeet dashboard.
Creating new services proved just as easy. When they needed a new a Database, they used a Blueprint by Zeet to quickly configure a production-ready database that was configured to their specification. In fact, for every one of the services they needed, they relied on Zeet's Project Creation experience to create and deploy infrastructure and services, including containerized and serverless apps.
Using NVIDIA Tools for Automatic Speech Recognition (ASR)
Tarteel used NVIDIA speech AI tools, including NVIDIA Riva, the GPU-accelerated SDK for building customized speech applications, and NVIDIA NeMo, the open-source framework that enables developers to build and train state-of-the-art conversational AI models. These tools help Tarteel provide real-time feedback on Quran recitation.
Speeding up the Pace of Innovation
Simplicity of operating infrastructure wasn’t the only benefit. Tarteel’s deployment pace actually got faster with Zeet. Over time Tarteel’s team has increased deployments per team member by 50%.
“I push code and it's deployed. That's all I need to worry about. If I push bad code, it does not get deployed. We don't need to worry about production being down and some guy being on call getting paged. If it’s 2 am or whatever, I can push code.” - Anas Abou-Allaban, Co-Founder, Tarteel
With the existing team of engineers using Zeet, Tarteel was able to transition the infrastructure to a multi-cloud, global approach while increasing product reliability and reducing cloud costs.
“For us, it's about being able to operate across multiple cloud providers. It's important because our application requires a lot of bandwidth, and relying solely on one provider's rates would be financially challenging. Additionally, network quality varies among clouds, and it's difficult to assess without actually running our workflows. Using Zeet allows us to quickly deploy and run on different clouds simultaneously, and we currently run on three different clouds in production. It's seamless and helps us avoid downtime, like when one of our cloud providers experienced issues recently, traffic was rerouted to alternative data centers without causing any user-visible downtime. It's been a huge benefit."
Conclusion
With Zeet in place providing infrastructure automation and Kubernetes deployments, Tarteel was able to boost deployments, cut downtime, and scale effortlessly. They are confidently taking on their company's next stage of growth and continuing their mission to create a platform for Muslims to read, understand, memorize, and recite their holy book.
As Tarteel's product continued to grow, Anas ultimately found the timing was appropriate for his startup to hire the company's first DevOps Engineer.
Now Zeet has became the central internal tool where the DevOps Engineer and the Application Engineers work together, with everyone able to focus on high-value work. Tarteel's DevOps Engineer uses Zeet to execute on his infra roadmap and the company's investments in this area, while the application devs can write code and work on the product while being free from deploys tasks. They can all manage multi-cloud infrastructure and services from one Zeet dashboard.
"The magic of Zeet is that it's scaled with us from 0 to where we are now, providing us with the simplicity but also the flexibility of deploying almost any kind of workload we'd like. Zeet has simplified the whole process of setting up our multi-region, multi-cloud infra that's capable of performing real-time STT inference for our users all around the world."