KubeCon + CloudNativeCon India 2024

Applying modern deployment strategies, such as incremental releases, to complex ML architectures presents unique challenges, including managing resource constraints and ensuring cost efficiency. In the GPU environment, new replicas can take up to an hour to become operational due to extended image pull times. Despite these challenges, implementing advanced deployment strategies is crucial to minimise the impact of faulty versions, much like in traditional CPU workloads. In this session we will present on how to overcome these hurdles in GPU-based workloads. We will: - Start with traditional deployment approaches. - Progressively address advanced scenarios, such as implementing cost-efficient progressive delivery for high CPU-consuming workloads on GPU nodes and orchestrating sequential multi-resource deployments. - Reference our recent experience designing CI/CD for the GenAI applications. - Conclude by sharing cost comparison and best practices for GenAI model deployments.