KubeCon + CloudNativeCon India 2024

Cloud Load Balancers (CLBs) have evolved significantly, now featuring advanced capabilities such as proxying, SSL/TLS termination, and auto-scaling. The underlying data-path technologies have advanced from traditional Linux networking to include eBPF, DPDK, VPP, and P4. Each of these technologies offers unique advantages and trade-offs: eBPF excels in real-time processing but lacks some comprehensive features, DPDK provides high performance with reduced transparency, VPP offers modular packet handling with strong Linux compatibility, and P4-based solutions, while still evolving, promise enhanced flexibility and performance. This session will delve into these data-path technologies, examining their maturity, limitations, and suitability for modern data centers. As specialized hardware units become increasingly integral to large data centers, we will highlight how these units effectively leverage data-path technologies to perform numerous tasks, thereby enhancing overall efficiency.

As EarnIn, one of the leading fintech companies, rapidly scaled in the number of microservices and user base, we recognized the need to ensure our network layer could keep pace with our scale. Initially relying on an enterprise service mesh that eventually became a bottleneck for our applications as well as our operators, we explored all the available service mesh out in the market, ultimately choosing Linkerd. This talk will delve into our decision-making process, focusing on why we chose LinkerD and how it has transformed our network and application architectures. We'll cover the crucial aspects of Linkerd that enhanced our system's performance, robustness, and security, all while simplifying operations and ensuring reliabilityᅳensuring our operators can rest easy. Join us to learn how LinkerD can elevate your service mesh strategy to meet the demands of substantial growth while keeping your architecture simple.

This talk will explain new options for proxyless service mesh to enable secure authentication with serverless-based backends. Vanity URLs and reverse proxies have historically posed an issue for proxyless service mesh, making it difficult to use them, and the speaker will talk about upcoming new features in gRPC that make this possible, while also ensuring robust security and preventing misrouted traffic. The audience will discover how proxyless service meshes secure authentication with serverless backends, opening up new possibilities for modern architectures. The audience will also come to know how the client can intelligently determine the control plane's support for these new features, enhancing adaptability and streamlining deployment.

While anyone working with Kubernetes knows about Controllers and Custom Resources, most don't know that there are many ways of designing a Controller, but just one good practice. That practice is "single resource": your controller should only manage a single resource, instead of having a single controller for multiple resources or multiple controllers for a single resource. This talk guides you on implementation of single-resource Controllers, showcases examples of a well-designed controller and good practice of designing Single-Resource controllers by comparing it with discouraged alternative designs. This talk focuses on practical insights, on aforementioned design’s pros and cons, and how its implementation affects the efficiency of Operators. After the talk, attendees will be able to design and implement single-resource controller, and have a deeper understanding of good practices of designing a kubernetes controller

The rapidly evolving landscape of Machine Learning and Large Language Models demands efficient scalable ways to run distributed workloads to train, fine-tune and serve models. Ray is an Open Source framework that simplifies distributed machine learning, and Kubernetes streamlines deployment. In this introductory talk, we'll uncover how to combine Ray and Kubernetes for your ML projects. You will learn about: - Basic Ray concepts (actors, tasks) and their relevance to ML - Setting up a simple Ray cluster within Kubernetes - Running your first distributed ML training job

Traditionally, search engines and vector DBs (the boundaries got blurry lately) are deployed as monoliths in a cluster, managed by a StatefulSet and a lot of technology-specific logic. Newer use-cases (e.g. RAG, semantic search in general) implicitly push for more stateless work: e.g. generate embeddings on the fly, re-rank, local LLMs. Decoupling this work and managing it separately makes more sense now, but is it worth the complexity? This session takes a deep dive into the architectures of search engines (mainly Vespa, but Elasticsearch, Solr and others will be in the picture, too) to highlight both pros and cons of separating concerns in information retrieval. And how much separation is good for which scenario. In the end, you should have a better idea about both the technology and the deployment type that fits your use-case best.

It all started when CoreDNS pods entered a crash loop backoff state soon after the cluster was deployed. The only change was an unprivileged execution desired in the latest coreDNS image. The issue occurred only in worker nodes, not in control plane nodes, and refreshing the image by deleting and pulling it again resolved the issue. It happened in one OS distribution flavor but not in another. We investigated Docker build, setcap, and libc, and found that the required capability was missing in nodes during the issue. The stack was complex, involving FIPS/CGO, build system, runtime configuration, and binary packaging in OVA. Debugging included CoreDNS binary, containerd, runc, SELinux, AppArmor, Photon OS, and the kernel... lots of learning which could be useful for many developers/admins or cluster operators to debug unprivileged pod execution. Finally, we discovered something interesting in the bootstrapping scripts. Let's debug.

As organizations scale, robust disaster recovery becomes critical. As clusters multiply and complexities grow, maintaining consistent RPO and RTO becomes challenging, requiring new techniques for efficient large-scale data backup and recovery. While GitOps excels at maintaining the desired state of infrastructure using Git as the single source of truth, it falls short for stateful workloads like databases and S3 buckets. Tools like Crossplane help deploy these resources in a GitOps model, but disaster recovery remains complex. You can't just spin up a new K8s cluster, point it to a Git repo, and expect a seamless recovery of critical stateful data Disaster recovery isn't 'one size fits all', each scenario requires a different approach depending on the scale of the disaster. In this talk, we’ll explore different strategies for handling different levels of disasters, from losing a single K8s cluster to facing a full region outage, helping folks to be better prepared for the unexpected.

As Large Language Models gain prominence, the discussion around AI fairness and ethics has never been more pressing. Cloud Native is emerging as the standard infrastructure for training, deploying, and serving AI workloads, but its role doesn't stop there—it can also play a crucial part in minimizing bias in AI models. This session will delve into: - Mapping Cloud Native principles like attestation and supply chain fundamentals to AI workloads, enhancing explainability and providing actionable recourse. - Leveraging Kubernetes' reconciliation loop to offer feedback that corrects false positives in AI output. - Utilizing the distributed nature of cloud computing to decentralize AI power, promoting more responsible automation. This session is designed for AI practitioners seeking insights from the Cloud Native tech stack, infrastructure beginners looking to understand its extensibility, and experts and maintainers aiming to extend the boundaries of current Cloud Native infrastructure.

Imagine adding all your favorite products to your shopping cart, only to have the page reload and—poof—everything’s gone. Frustrating, right?. For mission-critical apps, seamless upgrades in Kubernetes demand a solid grasp of how to tackle these challenges effectively. This session equips you with practical strategies for performing zero-downtime upgrades in Kubernetes. We will cover the role of readiness and liveness probes, explore rolling update strategies and Blue/Green deployments, and compare their benefits. We will also discuss handling SIGTERM signals, managing sidecar containers, and the importance of PodDisruptionBudgets and TopologySpreadConstraints. Finally, we will examine how Priority Classes and Pod Eviction impact stability during Kubernetes upgrades. By the end, you'll be equipped to confidently handle upgrades without downtime, ensuring your applications stay online and reliable, even during the most challenging updates.

As AI becomes more popular, it's bringing some new headaches when it comes to managing models, from building them to getting them into production. The usual DevOps tools don't always cut it, especially when you need everything to be consistent and handle large datasets without a hitch. As ML projects grow, trying to manage everything manually just gets messy and prone to mistakes. In this talk, I'll break down how GitOps can make life easier by automating and simplifying ML operations. With Git as the go-to source for everything, GitOps keeps your ML models consistent and reproducible across different environments. I'll share some real-world examples, talk about the challenges, and give you practical tips on how to bring GitOps into your ML workflow. The goal is to help you deploy models faster and with fewer headaches.

gRPC is often chosen for its high-performance communication, making it crucial to minimize latency. Distributed tracing helps identify and diagnose latency bottlenecks in gRPC calls, ensuring that the performance benefits of gRPC are fully realized. Setting up tracing from scratch for gRPC applications is complex and warrants significant maintenance overhead. Moreover, there is not much literature on how to do it properly, resulting in bespoke and suboptimal solutions. In this talk, Purnesh Dixit from the gRPC team will talk about the best practices for utilizing distributed tracing effectively in gRPC environments by configuring the new OpenTelemetry plug-in, created by the gRPC team in google.