What is platform engineering?

Platform engineering is the discipline of building and operating internal developer platforms (IDPs) that give product teams self-service access to infrastructure, tooling, and paved-road workflows so they can ship software without mastering every underlying system. Gartner projected that by 2026, 80% of large software engineering organizations would run platform engineering teams, up from 45% in 2022.

The core move is treating the platform as a product: a dedicated team builds golden paths, supported self-service routes for common tasks like deploying a service or provisioning a database, and product engineers consume them the way they would any external service. The goal, as the CNCF Platforms whitepaper puts it, is "an integrated collection of capabilities defined and presented according to the needs of the platform's users."

How does platform engineering work? #

Platform engineering works by centralizing repeated infrastructure decisions into one team that ships them as a product. The platform team builds self-service abstractions over compute, deployment, data, and telemetry; product teams consume those abstractions through templates, APIs, and portals instead of re-solving the same problems service by service.

The organizing idea comes from Team Topologies (Matthew Skelton and Manuel Pais, IT Revolution Press, 2019), which names the platform team as one of four fundamental team types. Its purpose is reducing the cognitive load on stream-aligned product teams: every Kubernetes manifest, TLS certificate, CI pipeline, or telemetry backend a product engineer must operate directly is load that isn't going into the product. Skelton and Pais argue for a "thinnest viable platform": the smallest set of capabilities that keeps product teams fast, not an empire of internal tooling.

Adoption is the truest measure of a platform's success. If engineers route around it, that is treated as product feedback, the same way churn is for a commercial product.

What is an internal developer platform? #

An internal developer platform (IDP) is the product a platform engineering team builds: an integrated layer of self-service tools, templates, and services through which developers provision infrastructure, deploy code, and operate services. It presents curated defaults for the 90% case while allowing escape hatches for teams with unusual needs.

Typical IDP components:

• Developer portal. The front door: a service catalog, documentation, and scaffolding UI. Backstage, created at Spotify and a CNCF incubating project since March 2022, is the best-known open-source example.
• Golden paths. Templated, supported routes for common tasks: "create a new service" scaffolds a repo with CI, deployment config, and instrumentation already wired in.
• Self-service infrastructure. APIs or portals for provisioning databases, queues, environments, and secrets without filing a ticket.
• CI/CD pipelines. Standardized build, test, and deploy workflows that teams inherit rather than assemble.
• Observability defaults. Telemetry collection, dashboards, and alerting that every scaffolded service gets on day one.

Platform engineering vs DevOps vs SRE #

Platform engineering, DevOps, and SRE answer different questions and coexist in most large organizations. DevOps is a cultural philosophy of shared ownership of software from development through production, captured in Werner Vogels' 2006 line "you build it, you run it." SRE is a reliability discipline with prescribed mechanics: SLOs, error budgets, and blameless postmortems. Platform engineering builds the internal product both rely on.

The common misreading is that platform engineering replaces DevOps. In practice it makes "you build it, you run it" sustainable at scale by making the running part self-service, so a product team can own its service in production without each member becoming an infrastructure expert.

What are the challenges of platform engineering? #

The recurring failure mode is building a platform nobody asked for. A platform team that mandates its tooling by policy rather than earning adoption by quality recreates the centralized-ops bottleneck DevOps was a reaction against.

Specific pitfalls:

• Platform as a side project. A platform maintained by whoever has spare time decays into unowned glue code. It needs dedicated ownership, a roadmap, and users treated as customers.
• Golden paths without escape hatches. If the paved road is the only road, teams with legitimate edge cases are blocked; if escape hatches carry no ownership expectations, the platform's guarantees erode.
• Abstraction leaks. Hiding Kubernetes behind a template works until the abstraction breaks and a product engineer must debug what the template generated.
• Measuring the wrong things. Counting services onboarded says little; time-to-first-deploy, ticket volume for infrastructure requests, and voluntary adoption say more.

How does observability fit into platform engineering? #

Observability fits into platform engineering as a golden path: the platform team makes instrumentation, telemetry collection, and dashboards the default output of scaffolding a service, so product teams get production visibility without building it themselves. Done well, a new service emits useful telemetry on its first deploy with zero bespoke effort.

The standard building block is OpenTelemetry, the CNCF's vendor-neutral instrumentation project. Because OpenTelemetry decouples how telemetry is produced from where it is stored, a platform team can bake SDK setup and collector routing into service templates once, and retain the freedom to change backends later without touching application code. A scaffolded service's telemetry defaults reduce to a small config fragment:

1# Illustrative golden-path defaults injected by a service template
2telemetry:
3  sdk: opentelemetry        # auto-instrumentation enabled at build time
4  exporter: otlp            # ship to the platform collector, not a vendor SDK
5  endpoint: otel-gateway.platform.svc:4317
6  resource:
7    service.name: "{{ service_name }}"   # semantic-convention attributes
8    deployment.environment: "{{ env }}"

What the platform team owns downstream of that fragment (collectors, storage, query, dashboards, retention) is an architecture decision in its own right, covered in how observability architectures are structured. The platform lens sharpens the requirements: one shared backend must absorb telemetry from every team (high cardinality by construction, since service.name and team label everything) and serve queries back to all of them, which is what makes observability a data problem before it is a tooling problem.

Observability as a golden path with ClickStack #

A platform team offering observability as a paved road needs a backend that is OpenTelemetry-native and cheap enough to serve every team from one place. ClickStack, the ClickHouse observability stack, fits that shape: it ingests logs, metrics, and traces through a bundled OpenTelemetry collector, stores them as wide events in ClickHouse, and exposes them through HyperDX, its UI. The golden-path config fragment above needs only the gateway endpoint pointed at it. Platform teams evaluating a shared telemetry backend can explore ClickStack as the storage and query layer behind their paved road.