Thanks to the trend towards microservices, backend applications are becoming more complex and distributed. Many teams struggle to accommodate these architectures with the traditional approach to deploying systems - configuration management with tools like Puppet, Chef, Ansible, or Salt. The playbooks/recipes required by these new application stacks grow in complexity until they’re no longer maintainable or comprehensible.

Enter Kubernetes. It brings with it a new hope: a standardized declarative API for doling out hardware resources and managing your containers’ lifecycle.

However, Kubernetes is not an end in itself. What most organizations need is a complete orchestration platform. Kubernetes provides a few (albeit very important) pieces of such a platform - the API server and scheduler. But, generally speaking, this is not enough.

What Kubernetes does bring to the table is an API - a set of abstractions for describing the desired state of your system. For example, it defines objects for specifying various types of application deployments: which containers should run, where they should be placed, how much resource they require, and so on. It also defines abstractions around storage, service discovery, network policy, and many other aspects. But, in many cases, it leaves the choice of implementation up to you, the system administrator.

Complete orchestration platform

As a result, the industry has seen a Cambrian explosion of sorts - a huge range of proprietary and open source implementations of the various verticals within the Kube API. In this article, we attempt to map out those verticals for engineers considering or planning the buildout of a Kubernetes-based orchestration platform. In future articles, we will delve deeper into each of these topics.

Components of a complete container orchestration platform

So, if you’re an operations engineer or manager thinking of building a complete platform for your organization, what should you include? Here are some ideas you might want to consider for your platform.

Provisioning and updates

The first question that usually comes up is: how do you install Kubernetes and all of the associated components, including the OS - consistently across your entire cluster? What configuration system should be put in place to tune the various knobs and dials associated with the system?

Once the cluster is installed, how do you ensure that the system always runs the latest security patches? How do you manage upgrades from one Kubernetes version to another, especially when it may involve migrating configuration data? Most importantly, how do you ensure that all of the components work well with each other?

Software-defined networking

A software-defined networking paradigm is all but required for the effective operation of a container orchestration platform. Application instances are dynamically allocated, so static network configuration and firewall rulesets are no longer up to the task.

Kubernetes requires a separate IP address for each Pod. There are a variety of schemes for how this can work - but generally speaking, the Pod IP space is different from the host IP space. Packets from a pod on one machine to a pod on another machine must traverse some form of IP routing or encapsulation. A number of solutions are available, with various tradeoffs.


When thinking about container orchestration, the most obvious resources to be managed are CPU and RAM. However, an equally important - but more challenging - resource is persistent storage. Virtually all applications require some form of storage, whether it be a RDBMS, NoSQL database, or just a filesystem. Some teams choose to manage storage entirely outside of the container orchestration stack. However, this has some drawbacks as it does not take advantage of the automation afforded by Kubernetes. This automation can significantly decrease operational complexity, lower the burden for on-call personnel, and shorten the time to recover from failure.

Kubernetes has support for a range of providers out of the box, although setting up the infrastructure itself is out of its scope.


Kubernetes provides a robust authorization system, structured as role-based access control (RBAC). This allows cluster administrators to define, with fine granularity, the actions users or groups are allowed to perform. However, it does not provide a production-ready turn-key mechanism for identifying a user. Instead, it defines a set of interfaces that allows the administrator to plug in their own authentication system:

  • TLS certificates. Users must present a valid TLS certificate signed by an accepted authority. Administrators must set up a public key infrastructure for managing the certificates.

  • OpenID Connect. This could be an obvious choice if you use a compliant identity provider, such as Google.

  • Webhook provider. When making requests to Kubernetes, users present a token. Kubernetes calls out to a standalone service to determine the user’s identity from the token. The choice of authentication service itself (or custom development of such a service) is left to the administrator.

  • Authenticating proxy. Instead of talking to the Kubernetes API server directly, users connect through a proxy that manages the authentication.


Visibility into the operation of the stack is extremely important. It allows you to debug problems, understand your application’s performance characteristics, manage cost, and scale the cluster efficiently. At the time of this writing (January 2018), the pipeline for metrics collection in Kubernetes is an area of active development and is changing rapidly. Kubernetes collects basic metrics from across the cluster and makes the latest values available via a standard API. It is left up to the system administrator to select and integrate a system for collecting these metrics over time, and presenting them to users in the form of graphs or tables. It is also left up to the system administrator to collect application-level metrics.


By default, Kubernetes makes use of Docker’s underlying logging drivers. Each container’s stdout and stderr streams are forwarded into a file that sits on each node until the pod is deleted. If you wish to have cluster-level log aggregation and processing, or if you want logs to persist after the pod has been cleaned up from its host, you must set up a system of your own.


If you’re hosting an application that will be accessed from outside of the cluster (most applications), you’ll need a way for external traffic to reach the appropriate service running inside the cluster. To accomplish this, some of the machines in the cluster will be exposed externally through a load balancer. These machines will need to run an “Ingress controller” - a piece of software that listens for incoming requests, possibly transforms them according to a set of policies, and forwards them to the correct Service. Several ingress controllers are available.

Service mesh

In distributed systems, most of the traffic does not come from external sources; it is generated by services talking to each other. In Kubernetes, this kind of traffic generally goes through a “service mesh”: a layer that performs a variety of functions, including:

  • Load balancing
  • Traffic partitioning (e.g. staging vs production instances)
  • Failure handling (e.g. circuit breakers and timeouts)
  • Request tracing

Kubernetes comes with a primitive IPTables-based service mesh (kube-proxy), which does basic load balancing only. For a production system, you may want to integrate something more sophisticated.

But wait… there’s more!

The considerations listed above are, in our experience, the most important for the basic operation of a production cluster. However, there are plenty of other systems that you might want to put in place:

  • Hardware inventory system to manage your bare-metal hosts;

  • Cost accounting system to answer the question of how much each component of your application costs or how much specific teams are spending;

  • Distributed tracing to provide additional visibility into the performance of your distributed application;

  • Alerting to notify your developers and operators when something has gone wrong;

  • PKI and encryption for data in flight across system components;

  • And many others.

We hope this article has provided you with some food for thought when planning your organization’s container orchestration platform. In follow-up posts we will explore the requirements for each of the components in greater depth.

Do you wish you could get the benefits of a container orchestration platform, without having to select, evaluate, and implement each of these components for you on-premise clusters?

Do you want a stack that “just works”?

Try Operos: the first complete infrastructure and application management stack. It integrates Kubernetes orchestration, Ceph distributed storage, provisioning, and many other features in an easy-to-use, turn-key package.