Cloud Native Platform Engineering Community – Community Contribution

Blog: Join the Infrastructure Lifecycle Working Group 💙

Wed, 16 Oct 2024 12:00:00 +0000

Are you looking to influence the future of infrastructure management in cloud-native environments? The CNCF Technical Oversight Committee has recently approved the Infrastructure Lifecycle working group within the TAG App Delivery, and we need your expertise!

As cloud-native practices evolve, infrastructure needs are becoming more complex. IT professionals now face the challenge of managing workloads across different environments. These include cloud platforms, IoT edge devices, on-premises data centers, and hybrid setups. There were already considerable investments in both open-source and commercial solutions. However, IT professionals still look for standards that address the whole infrastructure lifecycle.

This working group aims to define technology-agnostic best practices in infrastructure lifecycle management. We will deliver a framework for managing the lifecycle of cloud-native infrastructure, with a focus on components such as:

Infrastructure as Code
Control Planes
State Management
Disaster Recovery
Automation
Testing
Observability

To accomplish our deliverable, we need your help in collaborating with relevant Technical Advisory Groups (TAGs), Working Groups (WGs), vendors, and end-users to integrate domain-specific expertise. Your participation will help us develop practical, real-world solutions that address the diverse needs of the community. Every voice matters, and we warmly welcome and encourage contributions from everyone interested in this field! 🎤

Co-chaired by Ryan Nowak (Microsoft), Bruno Schaatsbergen (HashiCorp), and Jochen Zehnder (Inventx) and various active members, the group brings together expertise from leading industry professionals.

Excited?

Are you a developer interested in shaping the future of cloud-native infrastructure lifecycle management? We’d love for you to be part of it! 💙 The charter provides more information about the working group’s goals and direction.

Bi-weekly meetings: Participate in our discussions on each month’s first and third Fridays.
CNCF Slack Channel: Connect with us on the #wg-infra-lifecycle channel on the CNCF Slack workspace.

Are you attending KubeCon + CloudNativeCon North America 2024? Don’t miss the chance to connect with the co-chairs and members of the Infrastructure Lifecycle Working Group at the TAG App Delivery booth to learn more about our initiatives!

- Ryan Nowak, Bruno Schaatsbergen, and Jochen Zehnder.

Blog: Platform Engineering in 2024, Industry Trends and Emerging Focus (An holistic proposal for Internal Developer Platforms named Platform Engineering ++)

Tue, 25 Jun 2024 15:21:00 +0000

In this blog post, I’ll explore Platform Engineering, covering its diverse interpretations and implementations across organizations.

I’m trying to answer to the following question: “Should Internal Developer Platforms be limited to self-service infrastructure provisioning and application deployment?”

Organizations approach Platform Engineering in different ways:

Some build self-service tools for infrastructure provisioning, giving developers autonomy over infrastructure management.
Others focus on enhancing the developer experience, simplifying coding and deployment.
Some adopt a marketplace-centric approach, creating a repository of reusable components like containers, data, and APIs.

While current Platform Engineering practices effectively address many aspects of simplifying people’s lives, there are other areas that require attention. Data, ML, API, Security, Privacy, and others are crucial for a better Software Product Lifecycle. It is essential to consider whether these teams can benefit from the existing Platform Engineering practices.

By expanding the focus beyond a singular aspect, we can ensure a comprehensive and enhanced experience for all users.

I believe that Platform Engineering should encompass the entire end-to-end value chain to deliver products and applications to end users effectively. Often, Platform Engineering is “solely” associated with Infrastructure and DevOps simplification, with the primary goal of providing a self-service platform for developers.

By envisioning this, we run the risk of creating new barriers between Platform Engineers and Developers, similar to the barriers that existed in the past between IT Operations and Developers. Furthermore, we now have Data, ML, API, and other engineers to consider.

In my opinion, expanding Platform Engineering and its tools to encompass the entire spectrum of Digital Applications is a worthwhile endeavor also for not typical aspects of what is usually called platform.

For example we can consider parts of a Platform also:

a Design System reusable by several teams;
a repository of libraries;
a metadata catalog;
the teams working agreements;
a set of guardrails for ensuring legal and compliance requirements;
a set of standards that application should follow.

I suggest calling this expansion Platform Engineering ++. The core elements, Infrastructure and DevOps plus Data/ML Engineering and plus Software Composability (API, Events, Micro Frontend, Libraries and all aspects of a Software Application that can be reusable and evolved with an Inner Source approach).

I believe this will lead to a more comprehensive and effective approach to application development and management. I will provide further justification for this position in the following paragraphs.

Introduction to Platforms, a metaphor

Like a library where people bring books to share and people borrow books to read, Platforms are a place where people share resources. In the digital realm, platforms serve as virtual libraries, enabling individuals to share and access a diverse range of resources.

You can represent a Library as a Platform where Assets are provided in the form of Books, Video, Papers and more. The Library offers Capabilities like Catalog Collections, Reading Rooms, Cafes and people can use those capabilities thanks to a Research Assistant, a Website, a Subscription Service. Librarians keep everything working and well organized providing a self service experience to users that can be private people, schools, and other groups.

Library users are not only readers, but also potential writers. They may draw inspiration, gather facts, and formulate thesis from other books they find in the library, which they can use to create new works.

Internal Developer Platforms

In this analogy, we can liken Librarians to Platform Engineers. Their primary objective is to make sure that users of the platform receive top-notch services and have a delightful experience while interacting with it. In the realm of metaphors, books can be likened to various resources and tools (just some examples):

A kubernetes cluster with all necessary software and configurations.
A crossplane CRD to create an infrastructure resource.
A kafka topic used for data streaming.
A data product available for consumption.
A software library designed for logging in the correct format, including the traceId.
A web component library to compose in a micro frontend.
An API for initiating payments through a digital payment provider.
A complete application accessible to end users.

The facilities of a library can be (just some examples):

A service catalog where all consumable and composable resources are listed.
A self-service UI to create a new software element (infrastructure, application or other).
A command line interface to view live logs in production.
A monitoring system to see microservice metrics.
A set of scorecards to gather information about team metrics.

The library, which I like to call the Internal Platform (or Internal Developer Platform, see next paragraph), is where all the technological magic happens. Platform Engineers manage this platform, ensuring that all Internal Platform Users have an exceptional experience. Their approach is product-oriented, focused on delivering a seamless and intuitive user experience.

Depending on the usage patterns of the platform users and the organization’s specific requirements, Platform Engineers fine-tune the platform to deliver the necessary services.

A platform is a virtual place where resources (I will call them Items as a synonym) are systematically arranged and disseminated. By optimizing time utilization, more resources become accessible.

By effectively leveraging these items, that were books in the analogy, organizations can create innovative solutions, optimize operations, and gain a competitive advantage.

Within the Internal Platform’s Platform Capabilities, capability providers offer a range of fundamental resources (Items), including Infrastructure, DevOps Toolchains, SaaS Services, and Tools, which are organized and consistently curated by Platform Engineers.

Each capability constitutes an Item of the Platform and is characterized by a:

definition: describes inputs, outputs, configurable behaviors, metrics and documentation;
life cycle: an item goes through a life cycle that includes development, testing, deployment, operation and decommissioning.
consumption and composition: items can be consumed and composed to create more complex applications and become other items to be consumed.

Product Teams can engage with and use these Items through user interfaces, command lines, and APIs. AI Agents can assist the team in simplifying usage, while the Items are organized in Catalogs with the corresponding documentation.

As a result of the collaboration, new applications are created. These applications can then be transformed into Raw Assets, which can be reused repeatedly by other teams, enhancing the organization’s DevX.

This cycle can be seen as a circular economy in software development, where resources are continuously reused and repurposed, leading to a more efficient and sustainable development process.

Platform Items Lifecycle

Platform items come in several varieties, but possessing common characteristics:

are described by a plain text file (there are several format that are emerging and I hope in the future we will have a standard);
can be put in relationships each to other;
have a life cycle for create, ship, run, operate and catalog that item in order to be consumed by Product and Application teams.

It’s important to note that users can interact with the system through a UI or CLI, or through another software that automates some tasks, or via an AI agent based on LLM.

If you had access to a comprehensive internal platform containing all the descriptions of behaviors and relationships necessary to run your application, how beneficial would it be to have an AI companion?

Imagine having a companion that simplifies tasks such as finding items, configuring them, and releasing and operating those items in production, all through simple conversations with the AI. This companion would significantly streamline your workflow and enhance your overall productivity.

I propose referring to this as “Conversational DevX”, which will be the focus of my next blog post.

Internal Developer Platform or Platform of Platforms or Internal Platform?

I don’t know if it should be better to say that an Internal Developer Platform is an ensemble of different platforms or that it’s just one of the platforms. This is an open discussion where we welcome conversation and other ideas.

Names are important but in this blog post I don’t want to focus on the name but on the content and I hope that together with the tag-app-delivery ( https://github.com/cncf/tag-app-delivery) people we will figure out a standard nomenclature.

As mentioned in the introduction of this blog post, Platform Engineering is more than Infrastructure and it isn’t an evolution of DevOps but is a way to reduce the cognitive load for people that are creating, delivering and operating software items like Applications, Data, API and ML Models.

The trend I see is that each type of software item has its platform. However, the user experiences across these platforms are becoming increasingly similar. This suggests a potential convergence of all software platforms into a single platform or a “platform of platforms.”

An Item in the Internal Platform can be of different types:

Infrastructure Resources: this encompasses the underlying hardware and software components that support your applications and services. It includes physical servers, virtual machines, containers, cloud platforms, data stores, databases, ML runtimes and more;
DevOps or Developer Platform Resources: all tools for defining and running a toolchain, plus tools to operate and observe workloads, data, APIs, event, AI models at runtime;
Data, Events, APIs Resources: data is a critical asset for many organizations. It can be structured, unstructured, or semi-structured and can come from various sources. It can be at rest or in movement. It can be updated by policies and emit events;
ML and AI Resources: machine learning models are algorithms that can learn from data and make predictions. You can define, training, deploy and improve your models;
Composable Resources: items can be orchestrated or choreographed. Depending on the type of the item you may use different patterns like Sagas or Micro Frontend composition; Composable Items are tools that allow you to compose;
DevX Resources: DevX as a Product Resources (internal marketplace and software catalog): each item of the Internal Platform can be a valuable asset for other people. Provide a marketplace for that items and a way to manage the marketplace lifecycle: create, publish, curate, consume, review is relevant for creating a software circular economy;
Team Collaboration Resources: teams work together within internal platforms, utilizing various items to facilitate communication and organize workflows. These items include issues, backlogs, documentation, and more. Understanding the Platform itself is essential for effective collaboration.

We can explode the diagram with different Platforms of the Internal Platform.

The Internal Platform obtains “raw” assets such as virtual machines (VMs), Kubernetes as a Service clusters, and Function Platform as a Service tools as runtime resources.

DevOps tools are essential for managing resources and code lifecycle, forming a fundamental pillar alongside observability, security, and identity management services. Data stores such as NoSQL and SQL databases, data streams and object storage are commonly utilized in application development.

Additionally, LLMs models are frequently employed as a service via APIs or embedded within the runtime to enhance applications created by teams. SaaS applications, such as Salesforce, Dynamics 365, and SAP, serve as central hubs for various customer and product information, playing a significant role in the development of cloud-native applications.

All capabilities are managed with an Infrastructure as Code approach and the IaC manifests are managed with the lifecycle: code, ship, run, operate and organize team collaboration.

The same approach resource lifecycle is shared among different kinds of platform items: developers, infrastructure, application orchestration (API, Events, Flows), machine learning and data. All of them are Platforms inside the Internal Platform that are interconnected and may share resources like a database cluster or a security policy.

The descriptor of that items are manifest that can be gathered in the concept of Application as Code (AaC): with the same approach of describing the desired state of the infrastructure you describe the desired state of the items of all platforms that compose the cloud native application.

Inside the Internal Marketplace are defined: the reusable blueprint for all platform items, the software catalog that you can reuse and all other items that come from each team that can be a building block for other teams. Thanks to that Marketplace you can enable a Platform Composability strategy.

The Internal Platform offers various interfaces, including a user interface (UI) and a command-line interface (CLI). Additionally, users can interact with the platform programmatically through an API. The AI serves as a companion. IaC and AaC manifests are provided as a context to the LLM and the AI simplify tasks for platform users. These tasks include resource discovery, troubleshooting microservices and resources, and creating new resources.

Platform Engineering++ is a software engineering discipline that focuses on the Internal Platforms with a holistic approach.

Conclusions

By adopting that Platform Engineering++ paradigma, I believe that the Platform Engineering initiative’s return on investment (ROI) will grow. This is because the platform becomes the foundation for business applications, eliminating obstacles between teams. As a result, all teams have a unified perspective of the end-to-end applications built on top of it.

For now I stop here and I ask for feedbacks :-). What do you think about it? Please leave a comment in the issue https://github.com/cncf/tag-app-delivery/issues/586 and join CNCF wg-platforms working group if you are interested.

The journey is just at the beginning!

References

In this blog post, when I mention “Platform,” I specifically refer to “Internal Developer Platform” or, as I prefer, “Internal Platform,” as these platforms are not exclusive to developers but also encompass Data, ML, Cloud, and other engineers. It’s important to distinguish these Internal Platforms from Business Platforms such as Uber, Airbnb, or Netflix. Internal Platforms assist organizations in building and operating their Business Platforms. For example, Spotify created Backstage as an Internal Developer Portal and has incorporated additional tools to develop its Internal Platform to operate Spotify consumer platform.

Please note that there are already papers that are defining some aspects that I’m going to discuss in this blog post:

the CNCF Platform White Paper ( https://github.com/Cloud-Native-Platform-Engineering/cnpe-community/blob/main/platforms-whitepaper/v1/index.md)
the Platform Glossary ( https://github.com/Cloud-Native-Platform-Engineering/cnpe-community/blob/main/platforms-wg/glossary/_index.md)
the Platform Engineering Maturity Model ( https://github.com/Cloud-Native-Platform-Engineering/cnpe-community/blob/main/platforms-maturity-model/v1/index.md)
the Platform of Platform initiative ( https://github.com/cncf/tag-app-delivery/issues/542).

I hope that this blog will contribute to brainstorming.

Thanks

This is an original blog post written for the CNCF TAG App Delivery community. I hope you enjoy it!

I express my sincere gratitude to the CNCF TAG App Delivery community for establishing a virtual place that fosters open-minded discussions and inclusivity.

Special thanks to who has reviewed this post! Atulpriya Sharma, Lou Bichard, Tyler Pate, Abby Bangser, Stefan Daugaard Poulsen, Chris Plank, Marsh Gardiner, Rob White and David Stenglein for their insightful reviews.

Blog: Enterprise IDPs must mature fast. Here’s how infrastructure optimization can help

Tue, 28 May 2024 12:00:00 +0000

Enterprises are expected to benefit from platform engineering sooner and bigger than anyone else. This has 2 main reasons:

With engineering bodies 100’s or 1,000’s large, standardization across the board is both a pressing need and a major event.
In large organizations, concerns like security, regulatory compliance, and cost efficiency can impede development.

With these two combined, the enterprise route to faster time-to-market must also go through scalable, effective guardrails along the product lifecycle. Therefore, Internal Developer Platform (IDP) capabilities that might be dismissed as day-2, effectively become day-1 imperatives. In other words, enterprise IDPs must mature fast – at least in some respects like compliance, security, and efficiency. When the MVP emerges, those must be already thoroughly worked.

3 cloud-native frameworks might prove useful in designing your platform MVP for faster time-to-(business) value. Let’s see what they mean to enterprises, identifying one takeaway from each towards that value.

CNCF IDP maturity model

The platform engineering maturity model presents five IDP aspects (investment, adoption, interfaces, operations, measurement), each described in four maturity states (provisional, operational, scalable, optimizing).

This matrix format means that, as the IDP evolves, some aspects might mature before the others. For instance, a certain IDP might be merely “operational” on its investment and adoption aspects, but demonstrate “optimized” interfaces, operations, and measurement.

This scenario becomes likely once your IDP includes cost optimization capabilities; unlike other managed SaaS, those typically repay their subscription price and beyond. So, no need to wait until a fully “scalable” stage before they’re introduced. Let’s illustrate it with a hypothetical example:

Example: soonicorn.io IDP status


Aspect	Descriptive example	Maturity
Investment	Dedicated platform team of 4: Senior platform engineer, full-stack developer, product manager, part-time UX designer.	Scalable
Adoption	Push: Teams are instructed to stop using their own deployment scripts; Cloud expenditure for Dev & QA is only authorized if resources are spun up via the IDP.	Operational
Interfaces	Integrated services: Every new project receives a space in a task runner (pipelines) and a runtime environment (Kubernetes namespace) and can opt into serverless runtime.Automation schedules pods to run on spot instances in a Dev cluster configured for bin packing.	Optimized
Operations	Managed services: VM users are only asked to declare their app as stateful/stateless and fault tolerant/intolerant. VM provisioning then spins up a spot instance configured for state reattachment and fallback or utilizes existing commitments.	Optimized
Measurement	Qualitative and quantitative: Within 1 year of MVP release, Dev vCPU hours increase by 30% while total Dev cloud costs only increase by 15%. Integrated FinOps tool analyzes each node to provide detailed cost attribution per project and team.	Optimized

This scenario is highly relevant for enterprises, which are financially conservative by nature. For those, measurement is, above all, financial. The board is all in for engineering productivity, but it must be shown expected ROI to approve any IDP investment. In other words, an enterprise platform is expected to repay itself by saving software engineering hours or otherwise money in a direct, measurable way. Therefore, the IDP mandate may depend on the ability to demonstrate how it saves the organization money:

Does it improve DORA time-centric metrics? (i.e. lead time for change, time to restore service)
Does it decrease downtime whereas the cost of downtime is known?
Does it lead to reduced cloud expenditure?

Achieving this may require the IDP to integrate both monitoring (e.g. GitLab) and guardrails for responsible cloud usage, backed by integrated tooling that applies savings techniques and visually tracks their results.

One takeaway: an IDP can earn initial credit by generating financial value as soon as its provisional investment or adoption phase, i.e. without changing the way the engineers work. For example, it can enforce cost-effective cloud resource provisioning policies in the backend. The cost reduction generated by those is easily measured and understood. Thus, this so-called day-2 capability makes more sense as part of the platform’s MVP. The value they unlock might actually win the mandate to keep maturing the platform in other aspects.

CNoE from a platform engineering perspective

The Cloud Native Operational Excellence (CNoE) initiative aims to help enterprises ease into cloud-native tooling, especially open-source. Platform engineering is its #1 use case. CNoE advocates several tenets to increase cloud-native maturity in enterprises:

Tools over practices – as hard as it is, tool adoption is still easier than culture change. Pick tools that automate the practices you want to implement – especially those your developers are not used to perform hands-on, like cloud resource provisioning.
Convergence – prefer tools that fulfill multiple desired capabilities, and can close a circle on data collection, reporting, recommendations and execution.
Managed solutions – the support gap can be closed by commercial vendors offering manages services for popular cloud-native solutions like Kubernetes, Linux and distros, or popular IDP projects Backstage and Crossplane.
Customizable standard for infrastructure – balance between the enforcement of requirements to developer usability
Powered by Kubernetes but not confined to it – prefer tooling that isn’t confined to a specific container engine, so besides different types of managed Kubernetes they may use ECS, Openshift, VMWare Tanzu and others.

These tenets are no longer mere theory: CNoE’s centerpiece is the idpBuilder tool, accompanied by a non-branded reference architecture. For enterprises looking for even more out-of-the-box open-source IDPs, BACK Stack emerges as another practical application of CNoE principles, with opinionated tooling selection which can be modified later.

Is this nirvana? I’m not sure. In my opinion, the challenges CNoE sets to solve (rapid evolution of standards, DevOps not always a good fit) aren’t the most painful ones in enterprises.

To complete the picture, here are some grave open-source concerns I heard directly from enterprise engineers:

Fragmentation – open-source tools often don’t integrate natively, so they don’t always work off the same datasets and policies – which might lead to frequent failures. When one tool only recommends (e.g. Opencost) and another only executes (e.g. Terraform), you need at least one critical pair of human eyeballs at the intersection. And at scale, there are 1,000’s of those intersections daily.
No support - Open-source tools lack professional support, which places extra load on in-house engineers.
Privacy concerns – GDPR-compliant organizations usually rule out open-source software and DBs, and also SaaS hosted on hyperscalers (AWS, Azure or GCP)The potential fine for working on data acquired unethically or exposing their own users’ data is just too big. Even if it’s free, some procurement guidelines still apply!
Inability to measure added value – in the case of cloud-native SaaS which serves as part of a stack, or does something specific along the software supply chain, its contribution to metrics improvement cannot always be isolated.

One takeaway: in the previous section we established the importance of cost optimization as an IDP’s day-1 capability. To realize this according to CNoE tenets, we want to integrate into our IDP a cost optimization solution that is managed, converged, highly automated, and suits all containerized and non-containerized settings.

Well-architected AWS, Azure, and GCP

Cost optimization is a Well-Architected pillar in AWS, Azure, and GCP alike. Each hyperscaler outlines pillar slightly different, but all touch aspects of designing & planning, monitoring & managing, and cost-reducing activities. Out of all 3, I find Azure’s to be the most straightforward:

Develop cost-management discipline (by FinOps practices)
Design with a cost-efficiency mindset
Design for usage optimization
Design for rate optimization
Monitor and optimize over time

Intuitively, what’s right for your entire public cloud estate is right for an IDP that runs in it. This leads to a two-fold conclusion: the IDP itself must run cost-efficiently on cloud resources; and it should help cultivate the same cost efficiency practices in the entire Dev organization.

From a FinOps standpoint, you want your IDP to provide visualized cost observability and accurate usage attribution of each node’s components by label, namespace etc. This means it will facilitate collaboration between the engineering body and other cost stakeholders, FinOps first.

From an activity standpoint, you want it to leverage discount compute (spots, RIs, SPs) and to continuously “squeeze the lemon” of each machine’s CPU, memory, storage, and network capacity. This is achievable by automating Kubernetes optimization techniques like bin packing, rightsizing, shutdown scheduling, and dynamic storage.

One takeaway: Your IDP’s ideal integrated cost optimization solution has dual DevOps/FinOps qualities: On one hand, it has workable IaC integrations to automate the actions needed to lower cloud spend and minimize it going forward; On the other hand, it provides cost visibility & analysis that support cloud cost attribution and accountability.

How do I make this my own?

If you work with external platform or portal tooling, e.g. Backstage, Port, Crossplane, Cortex etc. – follow the next steps:

Check integration catalog for cloud resource management solutions. These might also be classified as provisioning, autoscaling & optimization.
Consider starting with open-source reporting tools like Kubecost to learn about your optimization potential.
Research externally for tools capable of automating the heavy lifting of continuous optimization. Today, these are nearly non-existent in the integration catalogs of platform vendors – because they are considered day-2 capabilities, and most platforms are simply not there. No reason to worry though – autoscaling & co. is a saturated, competitive segment. Once you start a POC, it’s in the vendor’s best interest to develop a plugin or provider for your chosen platform tool. Such a plugin might help put them in front of many other users.

Platforming from scratch? Start off from step 2 above. When you hit step 3, look for the vendor’s GitHub repo, or Terraform module, to create the integration. If you don’t have Dev resources to dedicate, you may request it from the vendor once your POC becomes a paid subscription.

Special thanks to Artem Lajko and Abby Bangser for their insightful reviews.

Blog: Platform Engineering At KubeCon EU 2024 - Recap

Mon, 06 May 2024 12:00:00 +0000

Exactly a month ago, Kubernetes users and experts gathered in the City of Lights, Paris, for KubeCon’s Europe edition. With over 12,000 in-person attendees, this KubeCon was amongst the largest in recent times.

While there was A LOT of attention and chatter around Artificial Intelligence and Generative AI, Platform Engineering was well represented. This was the first “Platform Engineering Day,” where enthusiasts had dedicated space to discuss anything and everything related to platform engineering.

In this KubeCon 2024 recap post, I’ll shed light on the platform engineering talks and discussions that took place at KubeCon.

TAG App Delivery & WG-Platforms at KubeCon EU

The TAG App Delivery is dedicated to supporting projects and initiatives related to delivering cloud-native applications, including building, packaging, deploying, managing, and operating them. One of the initiatives at TAG is the Platforms working group, which is dedicated to supporting, improving, and advancing the platform engineering initiatives.

At KubeCon EU, talks and sessions were organized at the booth to spread awareness about the group’s work. You can read more about it in our previous blog post. If you’re new to the TAG App Delivery or platforms working groups, you can watch the following talks to know more:

Platform Engineering Day

First up, I’ll share the amazing talks from the first edition of Platform Engineering Day. I remember entering the hall a little late and seeing the place full. People were standing and listening to the talks.

Talks

TAG App Delivery Platforms Working Group Update - Colin Griffin from Krumware provides an overview of what the Platforms Working Group is and how to get involved.
Sometimes, Lipstick Is Exactly What a Pig Needs! - Abby Bangser from Syntasso & Whitney Lee from VMware highlight the importance of investing in user interfaces and adoption for internal developer platforms, even more than perfecting the underlying technology.
Beyond Platform Thinking at Ritchie Brothers - Build Things No One Expects, in a Place No One Expect - Bryan Oliver from Thoughtworks & Ranbir Chawla from Ritchie Bros discuss how Ritchie Bros has expanded the capabilities of Kubernetes beyond just a delivery platform to drive their core enterprise.
Blueprints of Innovation: Engineering Paved Paths for a User-Friendly Developer Platform- Ahmed Bebars from The New York Times - provides valuable insights into the nuances of platform engineering in the cloud, giving a blueprint for implementing strategies in their organizations.
Building a Platform Engineering API Layer with kcp - Marvin Beckers from Kubermatic GmbH discusses how kcp supercharges platform engineering with a global control plane for all internal services.
Breaking the Mold: Unveiling Anti-Architectural Patterns in Platform as a Product - Vamshi Krishna Samudrala from American Airlines - discusses the intricate landscape of designing and implementing effective platforms.
Empowering Giants: Guide Your Enterprise with CNOE in Operational Tech Choices - Engin Diri from Pulumi introduces the CNOE Framework, and explores how participation can benefit organizations in overcoming challenges.
Designing for Success: UX Principles for Internal Developer Platforms - Kirsten Schwarzer from Octopus Deploy shows practical UX principles and tools you can use to design an Internal Developer Platform (IDP) that your developers love using.
Boosting Developer Platform Teams with Product Thinking - Samantha Coffman from Spotify talks about the effectiveness of taking the product approach for building platforms.
Building an AI-Powered, Paved Road Platform with Cloud-Native OSS - Todd Ekenstam & Avni Sharma from Intuit share insights on how open-source projects, such as Open Application Model, Istio, Karpenter, Argo Rollouts, can be integrated and extended to build your AI-native application platform.
Unlocking Innovation: How NatWest Bank Uses Cloud Native Tools to Deliver Platform as a Product - Chris Plank from Natwest Group & Derik Evangelista from Syntasso discuss how they focused on a GitOps approach and incorporated a range of tools to enable platform users to have a seamless developer experience.
To K8S and Beyond – Maturing Your Platform Engineering Initiative - Nicki Watt, from OpenCredo shares insights on how the recently released CNCF platform maturity model can be used as part of a toolbox to help guide organisations think through.

Panel Discussions

Panel: Navigating the Path to Platform Engineering Excellence: A Comprehensive Guide- Cortney Nickerson from Kubeshop; William Rizzo from SUSE; Abby Bangser from Syntasso; Areti Panou from SAP SE and Aparna Subramanian from Shopify discussed actionable steps to ensure effective platform engineering and dissect critical considerations for anyone looking to invest in their own platform.
Panel: The Platform Rock-Paper-Scissors: Build, Adopt, Buy- Jorge Lainfiesta, Independent Contributor; Leena Mooneeram from Chainalysis; Victor Araujo from Wolt; Jinhong Brejnholt from Saxo Bank and Edgaras Petovradžius from LEGO share their thoughts on how to wrangle budgets, lock-ins, and licenses to make decisions as you put together the foundations of your platform

Talks and Panel Discussions at KubeCon

While Platform Engineering was one of the hottest co-located events at KubeCon 2024, talks and panel discussions took place in other co-located events, such as BackstageCon, AppDevCon, ArgoCon, MultitenancyCon, and OpenShift Commons to name a few.

Here’s a round-up of all of them:

Day 1 - March 20

Bloomberg’s Journey to a Multi-Cluster Workflow Orchestration Platform - Yao Lin & Reinhard Tartler from Bloomberg talk about how they investigated related projects and what inspiration we took from Karmada, OCM, and others to build their own orchestration platform.
Building a Large Scale Multi-Cloud Multi-Region SaaS Platform with Kubernetes Controllers- Sébastien Guilloux from Elastic describes an architecture made of hundreds of Kubernetes clusters, and talks about the challenges we have faced along the way to build a multi-cloud, multi-region platform.
Simplified Inner and Outer Cloud Native Developer Loops - Oleg Šelajev from AtomicJar & Alice Gibbons from Diagrid explore tools to simplify and improve developer productivity through a platform engineering and polyglot approach.
Building AI-Ready Platforms - Symphony for Developer and Platform Engineer - Thomas Vitale from Systematic & Lize Raes from LangChain4j share details to bridge the gap between platform engineers and developers, focusing on adapting your platform for AI while providing a smooth developer experience.
State of Platform Maturity in the Norwegian Public Sector - Hans Kristian Flaatten from the Norwegian Labor and Welfare Administration uses the newly published CNCF Platform Engineering Maturity Model to measure how mature their platforms are, and what technologies they have chosen.
Cultural Shifts: Fostering a Chaos First Mindset in Platform Engineering - Sayan Mondal from Harness & Raj Vadheraju from FIS talk about how organizations can enhance their platform engineering practices by leveraging chaos-first principles.

Day 2 - March 21

Unlocking New Platform Experiences with Open Interfaces - Thomas Vitale from Systematic & Mauricio “Salaboy” Salatino from Diagrid explore existing CNCF projects to implement an end-to-end experience to build platforms.
Keeping the Bricks Flowing: The LEGO Group’s Approach to Platform Engineering for Manufacturing - Mads Høgstedt Danquah & Jeppe Lund Andersen from The LEGO Group share their story of how the LEGO Group builds platforms and products that cope with constraints like 24/7 production, limited internet connectivity, high resilience and low latency.
Why Kubernetes Is Inappropriate for Platforms, and How to Make It Better. - Stefan Schimanski from Upbound; Mangirdas Judeikis from Cast AI; Sebastian Scheele from Kubermatic - extending Kube, adapting its architecture to be a better fit for a platform.

Day 3 - March 22

Rapid IDP Capability Development and Automated Testing at Autodesk - Jesse Sanford & Greg Haynes from Autodesk show how IDPBuilder can stand up a CNOE reference architecture in minutes, with nothing other than Docker as a pre-dependency.
Search at Shopify: Highly Available Platform for Data Resilience and Compliance - Leila Vayghan from Shopify show how Kafka is used for in-order and real-time indexing of millions of documents per minute to achieve high availability

Panel Discussions and On-Booth Talks

You see, there was so much chatter about platform engineering at KubeCon!

Apart from the talks in the main conference, there were panel discussions, and talks at other co-located events and the TAG App delivery booth as well. While we don’t have videos for all these talks, there were some interesting ones around self-service infrastructure with Backstage, and Kubernetes developer experience patterns, to name a few.

Lian, Thomas, Mauricio and Atul participated in a panel discussion - From Platform Engineering To Developer Success at the RedHat OpenCommons colocated event.

Based on my experience of attending KubeCons, this has to be one of the most prolific ones when it comes to platform engineering. The talks were engaging, the platform working group was active and the booth was abuzz.

With the CFPs for the KubeCon NA 2024 already open, share your interesting proposals on platform engineering and let the world know about the amazing things you’re doing.

Platform Coffee Meetups - Paris Edition

One of the things that I absolutely love about the platforms working group is the in-person coffee meetups that we organize during KubeCons. The first one I attended was in KubeCon Chicago and I loved the concept.

Anyone and everyone is invited to discuss anything and everything about platforms and platform engineering over a cup of coffee. It’s an amazing experience to learn about what is happening in the world of platform engineering and to meet the amazing people behind it.

The Platform Lean Coffee meetup in progress. We've some REALLY amazing topics that we're discussing today.

Kudos to the WG-Platforms group at @CNCFTAGApp for this one.

Also thanks to our sponsors @syntasso @krumware @gitpod 🙏🏻😇 pic.twitter.com/NPMHZBlgQo
— Atulmaharaj 🥑 (@TheTechMaharaj) March 21, 2024

This time, we had it for four days, and we had people turning up in great numbers each day. With Croissants, Pain au Chocolat, and Cafe Au Lait, we all discussed everything from what platforms actually mean to discussing the success criteria for platforms - these were some insightful discussions.

Key Takeaways

This was my second in-person KubeCon and fourth overall. It was also the second time I met the members of the platform working group in person. While we connect over calls monthly and work on exciting things like the Platform Whitepaper, Platform Maturity Model, and ongoing Platform as a Product paper, meeting in person is way better.

Further, a huge shoutout to the members of the TAG App delivery and WG Platforms who were helpful right from the day KubeCon was announced to being hyperactive during KubeCon. Further, the first Platform Engineering Day was a huge success, and rooms filled with people are a testament to the importance and interest in platform engineering.

Apart from being a part of a platform engineering panel, I got to listen to some interesting talks on how people navigate the platform engineering maze. I was particularly intrigued by discussions around customized measuring the success of platforms and AI-ready platforms. The one thing I absolutely enjoyed about the discussions was the focus on developer experience and its larger productivity aspects.

Join Us!

Being a part of this fun and intellectual group, I can vouch that this is one of the most interactive and helpful groups. Whether you’ve just learned about Platform Engineering or are professional building platforms for years, you’re welcome to join the platform working group and help us shape the future of platforms.

Join our WG-Platforms Slack to get started. Just drop a note to introduce yourself and let us know what you want. Someone from the team will help you get started :)