“The Cloud Management Plane” Pattern to Manage 1000s of Edges – Architecture

 Platform 9 architecture is based on a managed control plane. The core idea of the managed control plane is a layered architecture that can support the lifecycle management of a diverse collection of open source cloud service implementations across many geographical regions. These resources primarily include VM workloads (VMWare and KVM/OpenStack), container-based microservices orchestrated by Kubernetes, and resources under them (such as SDN and Storage subsystems).

We believe that in order to successfully transform an old school cloud delivery model, a different approach needs to be taken whereby these 5 core themes are taken into account:

1. Freedom to leverage any infrastructure type;
2. Freedom to deploy applications in any runtime (bare metal, virtual machines, containerized and even serverless);
3. Freedom to deploy containers/VMs  to any distributed location;
4. Freedom from operational complexity by introducing a 0-touch automation engine  throughout the life cycle;
5. Freedom from vendor and cloud lock-in by leveraging open source cloud orchestration platforms.

Key Challenges and Lessons Learnt

 The Platform9 model uses a dedicated management plane for managing the life cycles and configuration of VMs, Bare metal servers, and Kubernetes clusters (across multiple cloud services). To enable scalability in the management and deployment of these diverse workloads, the architecture includes support for multiple regions with each region modeled as a collection of hosts or by workloads.

The following are the key benefits enabled by this architecture.

1. Scalability – It is also important to note that we provide other mechanisms that can be used to increase deployment scale, such as the number of regions and deployments. For example, it is recommended to distribute the management plane across multiple availability zones or regions to manage and monitor their own VMs/K8s worker nodes in this case.
2. Self Service – The central management plane provides self service for both workload provisioning, cluster ops and a catalog of certified workloads (e.g. Nginx, Kafka, Tomcat, Git etc) across all of these regions.
3. Multitenancy – Each region can implement their own vanilla or custom multi tenancy requirement based on RBAC, namespaces, network policies and resource quotas
4. Hybrid Cloud – Each region can be based on a different IaaS/private cloud deployment than the others thus allowing lines of business to define their own strategy for deployment. For example, a customer can mix Bare metal, VMware, AWS and Azure clouds as different regions within one global deployment
5. Security via centralized RBAC

Concept of Edge Inheritance –

 When an edge is onboarded,  a primary DU/Management Plane is first created and hosted on its own set of hardware with backup capabilities. The customer can then add child regions to the main region. The child management planes inherit configuration, users, and other properties (e.g. types of deployment supported) from the parent. This is the idea of Inheritance within the control plane. The child DU can then evolve over time based on special regional requirements etc.

However, the central management plane is a single UI across all regions with different roles (administrator, self-service user) backed by federated identity, across lines of business having different views of the same underlying infrastructure. Child DUs can then create their own UI representations for their specific regions.

The benefits of this approach are that:

• Easy and seamless onboarding – entirely automatic
• Each Edge site/DU can be upgraded separately
• Optionally upgrade hosts (and hence AZs) separately, This may not be the best alternative, but can be done in a controlled fashion
• Monitoring is centralized
• Pod/operator ratio can be kept extremely high