KubeCon + CloudNativeCon India 2024

In the newer world of generative AI models, enterprises bet on integrating large language models into their various business use cases.  Due to the complex infrastructure requirements of large language models, building scalable optimized end-to-end GenAI pipelines connecting data and compute is not easy compared to traditional machine learning models. Cluster admins need better visibility into infrastructure to ensure the best utilization of cluster resources, including expensive accelerators. In contrast, data scientists need a clean Pythonic interface without exposure to any underlying stack details. In this talk, we will cover how the Kubeflow Platform helps in LLMOps journey from training an LLM on the custom dataset to fine-tuning the pipeline for the best results and, finally, deployment of the trained models at scale.  We will discuss an optimized Kubernetes native ML reference stack for your LLM needs that provides maximum infra utilization.