Designing ML Infra for ML & LLM Use Cases
Introduction
- Everyone knew about DevOps but not many people knew that it's a rendition of DevOps into machine learning that's what we call Notes/MLOps/MLOps people did know that there is something existed as DevOps in machine learning but not many people or other organizations knew how to get the best out of MLOps so now more and more organization including the clients that I work with day in and day out they have understood the value that MLOps brings to the table.
- As far as continuous integration and continuous delivery is concerned and they have started reaping the benefits they have started understanding that the whole life cycle process of a machine learning system is long and to make it purposeful for day-to-day business activities they need to shorten it and streamline it and the best way to streamline is is to use MLOps.
Goals of MLOps
Note
One of the main goals of the MLOps team is to shorten the time it takes from Model Experimentation to Model Deployment in Production.
To break it down further:
- Reducing the time in data exploration stage how am I
- Providing standardized libraries or SDKs in the training phase
- Data quality Insights -> Which dataset performed the best
Reasons to bring in MLOps
- Data Drift
- Concept Drift
- Biased Data
Reason to not do full automation
- GPU costs in case of LLMs
Implementation
- As an MLOps engineer, find who all are your primary, secondary and tertiary customer. It can be data scientists, ML Engineers and BI Engineers.
- Find the pain points from them like: Issue in retraining etc.
- Resolve these pain points.
Software Engineer -> AI Engineer
- There are mainly three problems SE needs to take care of:
- Scalability
- Model Bias
- Model serving
- Compliance and Privacy (Even with 3rd party APIs)
