KubeCon + CloudNativeCon India 2024

Upgrading Kubernetes clusters often presents numerous challenges, including potential downtime, compatibility issues, and the complexity of managing multiple versions. The Mixed Version Proxy feature introduced in Kubernetes 1.28 aims to mitigate these challenges. This talk will delve into the technical intricacies of the Mixed Version Proxy, exploring its design and implementation. We will then highlight the substantial benefits it offers for cluster upgrades, such as minimizing downtime and enhancing overall reliability. Attendees will gain practical knowledge through (possibly a demonstration) on enabling and utilizing the Mixed Version Proxy. Finally, we will provide insights into the future roadmap for this feature, including upcoming beta releases and enhancements.

Simplify authentication and authorization across applications, focusing solely on business logic while ensuring secure services. Our solution features a multitenant proxy to streamline authentication on the SAAS platform. Istio service mesh intercepts every request, with a multitenant-aware proxy (enhanced OAuth2Proxy and caching) connecting to a centralized Identity Provider (IDP) for authentication. Post-authentication, the proxy collaborates with an authorization agent like OPA, passing context for dynamic policy evaluation. Authentication and Authorization are decoupled from the code, allowing developers to use a centrally managed auth service. This shift lets development teams concentrate on business functions, leaving security rules to security analysts, saving time and resources. Externalized authorization management offers runtime controls, including policy management, enforcement, and decision modeling for fine-grained access to applications, services, transactions and data.

Doesn't it get messy when resources are left behind after a test?How do we manage the life cycle of the infrastructure resources? Have you wondered if there's a way to handle it? You are at the right place then, welcome to the world of Boskos where resource management is easier than before. In this talk, we will explore some challenges faced in Continuous Integration(CI) with testing on various infrastructures. It will give an overview on how to use the resource manager service, Boskos to help you better manage your infrastructure and make sure that no stales are left unattended.

Kubernetes relies on Kubelet and Containerd to manage the lifecycle of pods. These components provide a wealth of log data, but the sheer volume and complexity can be overwhelming, particularly for those new to the Kubernetes ecosystem. This talk aims to dissect a pod's journey from birth to termination, pinpointing key events like volume mounts, health checks, and the dreaded teardown. Listeners will learn to filter, search, and interpret logs like a pro, turning raw data into actionable insights. Whether you're troubleshooting a misbehaving pod or simply curious about its inner workings, this talk aims to equip the listeners with the skills to navigate the Kubernetes log labyrinth with confidence.

In this session, Shashank Mittal, a Google Summer of Code (GSoC) 2024 contributor for Kubeflow, will discuss his work on enhancing the Katib component's Experiment APIs to support a wide range of parameter distributions for hyperparameter optimization. Katib is an integral part of the Kubeflow ecosystem that enables scalable and efficient hyperparameter tuning in machine learning workflows. The session will focus on the technical challenges and solutions involved in adding support for distributions such as uniform, log-uniform, normal, and lognormal, among others. Attendees will gain insights into the process of integrating these distributions into Katib's APIs and suggestion services like Optuna and Hyperopt. This talk is designed for developers and data scientists interested in MLOps and cloud-native machine learning infrastructure, highlighting the importance of advanced hyperparameter optimization techniques in improving model performance.

Cell-based architecture offers concentrated, self-contained functionalities but can be challenging to implement with Kubernetes clusters. However, integrating this architecture with Kubernetes brings significant benefits like enhanced scalability, resiliency, and resource management. This session will demystify the process, focusing on secure inter-cell communication, API gateway integration, and building resilience using Kubernetes-native features. I’ll demonstrate how to use Istio for secure inter-cell communication, ensuring efficient interaction between services while maintaining strict security. This includes configuring traffic management rules and leveraging Istio’s mutual TLS for secure communications with minimal complexity. We’ll also explore enforcing separation of concerns and governance for multi-tenancy and privacy. Attendees will gain actionable insights into leveraging cell-based architecture on Kubernetes, transforming complex setups into robust, scalable solutions.

Showcasing Azimuth - AstraZeneca’s cutting-edge Enterprise Cloud Native Machine Learning Platform. It is built on Kubernetes and integrates a diverse array of cloud-native tools, enabling seamless development, deployment, and management of machine learning workflows. My presentation will delve into the architecture, key components, real-world applications, and the integration with Cloudability for cost management, highlighting its role in empowering data science teams and accelerating innovation within AstraZeneca. The tech stack involves Kubeflow, Weights & Biases, Ray, Volcano Scheduler, Grafana, Prometheus, ELK, Harbor, NetApp Ontap FSx, Kyverno, GitHub Actions, ArgoCD, Argo Rollouts, CloudNativePG, etc

As organisations increasingly integrate AI solutions, the demand for environmentally sustainable practices within this space has never been more critical. This presentation delves into the collaborative effort between the Cloud Native Computing Foundation (CNCF) AI WG and the TAG Environmental Sustainability to define a repeatable design approach aimed at fostering sustainable AI in cloud-native environments. Our discussion will outline the crucial considerations in such approach, including efficient management of compute resources, storage optimisation, and advanced networking solutions. Attendees will gain insights into the lifecycle of AI/ML deployments, from inception through operation, emphasising resilience, scalability, and resource efficiency. By highlighting innovative "green" strategies, this session will provide actionable best practices and recommendations, alongside a forward-looking perspective on future trends and research directions in sustainable AI.

Machine learning platforms aim to streamline the workflow for ML practitioners, allowing them to focus on developing their models while the platform handles repetitive tasks like packaging code, dependencies and configurations. Traditional methods using Dockerfiles require ML engineers to navigate complex Linux processes and maintain multiple Dockerfiles for different projects, which can be time-consuming and prone to errors. Additionally, security mandates for regular patching and updates adding further to the complexity. Join this talk to explore how Cloud Native Buildpacks can simplify and secure MLOps deployments. By automating the packaging of ML projects, including custom libraries and hardware specifications, Buildpacks enhance flexibility, maintainability and security. This approach reduces the operational burden on both developers and security teams, ensuring a more efficient and scalable MLOps deployment process

Deploying large language models (LLMs) is inherently complex, challenging, and expensive. This case study demonstrates how Kubernetes, specifically Kserve with Modelcar OCI storage backend, simplifies the deployment and management of private LLM services. First, we explore how Kserve enables efficient and scalable model serving within a Kubernetes environment, allowing seamless integration and optimized GPU utilization. Second, we delve into how Modelcar OCI artifacts streamline artifact delivery beyond container images, reducing duplicate storage usage, increasing download speeds, and minimizing governance overhead. The session will cover implementation details, benefits, best practices, and lessons learned. Walk away learning how to leverage Kubernetes, Kserve, and OCI artifacts to enhance your MLOps journey, achieving significant efficiency gains and overcoming common challenges in deploying and scaling private LLM services.