Musings on SDN/Virtualization Integration (Part 1)

Introduction

Various posts on this site have touched on the topic of SDN. One of the key features provided by SDN is network programmability – the control plane directed data packets using the data plane or the forwarding functionality in the device. The forwarding logic depends on various rules, tables, and policies programmed by the vendor into the said device. Other key requirements missing from the older model are visibility into the topology and the real-time performance of the network/devices themselves. In short, the network of old was a black box – too siloed to program, provision and monitor using a single programmatic interface.

Limitations of Virtualization based Provisioning and networking

In most legacy hypervisor-based organizations, the process of provisioning the correct set of IT resources and configuration  required for the various applications looks something like this (http://www.vamsitalkstech.com/?p=7636):

1. For instance, VMware-based organizations commonly have vCloud platform deployed across the board to manage large fleets of virtual instances – often hundreds of thousands
2. Most of the infrastructure being managed is hosted across several on-premises data centers spanning different GEOs or regions
3. There is growing interest from lines of business in using public cloud infrastructure in addition to their on-prem data centers. This is often either due to their dissatisfaction with the service provided by their own central IT team and long lead times (typically between 1 and 2 months) or due to the need of some applications to burst into the public cloud for spikes in demand.
4. These organizations often have to support hundreds of applications, with the demand that keeps growing. To minimize Shadow IT and environment sprawl, they have to tighten their internal processes around permissions, compliance, and quota management when enabling business units to create or consume IT resources for their apps.
5. IT teams then follow a structured process to identify the appropriate infrastructure provider, technology stack, tools, and configuration, in order to support the most suitable environment for different types of applications. This involves provisioning a mix of compute, storage and network infrastructure using VMware components.
6. In an effort to improve agility and time to market, these organizations in parallel also start re-architecting their traditional monolithic, 3-tier applications, to be more microservices-based, and start adopting DevOps practices and CI/CD to improve service quality, cycle times, and release cadence.
7. With DevOps busting silos between developers and operations, there’s an expectation from the business to accelerate software delivery, enable the digital transformation, and influence the technology choices and buying decisions traditionally made by IT.
8. As the business pressure to deliver software faster increases, and as developers rely more on CI/CD and DevOps practices that require faster feedback loops, Shadow IT expands even more. This is a common scenario, as developers and some business units attempt to bypass traditional IT bottlenecks & delays and be able to take advantage of more modern technologies and architectures, by deploying their applications in the public cloud.
9. In parallel, these organizations see that the costs associated with proprietary virtualization solutions are constraining budgets and try to accelerate the move to open-source, Linux-based virtualization that is running on commodity hardware. However, system administration skills and costs associated with migration remain a significant bottleneck.
10. Feedback from end users and on-going update requests from the business owners get slowly implemented and reflected in the product, causing dissatisfaction with customers and possible loss of revenue or user retention.

The above-described provisioning cycles can take weeks to months. The term “It takes 45 days to provision a group of VMs” is heard as a developer joke in large enterprises.

Issues with Networking Configuration pre-SDN

As covered above, if an enterprise does not have an SDN capable controller, their DevOps/Cloud Native initiatives will suffer from the below delays from a networking standpoint.:

• Manual configuration of tenant isolation using VLAN or VXLans and DHCP servers per tenant. This hampers quota management, security and compliance
• Manual firewall, security groups, load balancer integration
• Manual integration with external routers and firewalls
• Lots of siloed scripting to work with different vendors networking equipment

These manual steps cause friction and delays in time to market as discussed above.

Virtualization/SDN Integration Patterns

To that end, one of the key patterns I have done work on is integrating traditional virtualization (e.g KVM/VMware) into an SDN control plane, with a view to automating all of the above tasks. While the SDN product is Cisco ACI, this pattern should work with all major SDN vendor products.

1. Internal groups within the customer enterprise will each be defined as a tenant in the SDN platform – e.g. ACI fabric. Each tenant will be isolated and setup within ACI with each tenant having its own Layer 3 VRF (Tenant namespaces) with firewalls to provide isolation. It is assumed that the ACI controller will be connected to a bunch of TOR switches in a Spine Leaf topology. ACI will use VXLAN to create logical overlays. vCenter will directly control the hypervisors while Platform9 provides the overall orchestration, service catalog, cloud management and integration with platforms such as Service Now.
2. ACI will pre-provision VLANs trunked down to a bunch of ports and leaf switches. Thus, the subnets in the VLAN will be available on all the ports connected to all the hypervisors. Once a VLAN is trunked and provided to the vSphere host, it is split into different networks for Management, Operations, Compute etc. This is to support specialized operations such as vMotion and SRM etc. in the ESXi host. The compute host is located in its own VXLANs along with NFS/Storage support etc.
3. ACI and vCenter interact with an API handshake for networking access. This handshake allows APIC to create a VDS (Virtual Distributed Switch) which can be consumed by Platform9 via vCenter.
4. ACI will learn the location of every ESXi host which is attached to the spine leaf fabric via LLDP as depicted. This integration already exists between VMware and ACI.
5. Next step is the creation of Endpoint Groups (EPGs) e.g. Web Server, Application Server, Database, Custom Applications. These are individual application network profiles with Web, DB, App endpoints. APIC creates the EPGs which are made available to vCenter.  APIC has a pool of VLAN IDs and assigns each VLAN ID to a port group (e.g. Web) and sends it down to the VDS for consumption.
6. OpenStack OVS continues to consume DVS port groups whether they’re provisioned by the network team using the APIC-vCenter integration, or, whether they’re provisioned manually using vCenter.  The APIC- vCenter integration depicted in the diagram is just one way of achieving this and there may be other ways to achieve this.

The next blogpost will cover a deeper approach to this integration.