AWS Networking Overview - Part 2
In part 2, learn how AWS implements Kubernetes networking and why you need to set up and configure Virtual Private Networks.
The other part to understanding networking on Kubernetes running on Amazon is the underlying Amazon network technology. AWS started out with a simple flat network, but due to customer demand for segmented networks, and to provide a more full-featured network implementation, it now includes VPC (virtual private cloud).
VPC allows for more traditional network configurations.
VM’s come with the option to setup a set number of elastic network interfaces (ENIs) on your VM (the exact number depends on the instance type), which attach to the VPC network using public or private subnets. VPC also provides fine-grained controls for routing and also packet filtering of traffic between these subnets. Configuration of your VPC can be done through the Amazon console or programmatically using a variety of languages and tools.
With VPC you can create ENI’s that:
- Directly attach to a public subnet
- Directly attach to a private subnet for non-public facing services
- Use Network Address Translation to map public IP addresses to specific ENI’s on private subnets
Furthermore you can:
- Use use VPC native tools to control traffic between subnets using configurable packet filters
- Setup dedicated VPN connections between your internal datacenter and the VPC
This combination of features provides the ability to create a full featured network topology, controlled by Amazon’s VPC interface. Philosophically, Amazon has chosen to allow maximum flexibility in VPC network design, allowing customers to mirror traditional network topologies. The advantages of the this system is that it allows network administrators to implement topologies that mimic traditional zone based network segmentation.
Amazon VPC is a mature product, and Amazon has also implemented a number of helpful technologies to improve operations and security inside of a VPC.
- VPC Flow Logs to track accepted traffic, rejected traffic, or all traffic to a subnet, interface, or an entire VPC. These flow logs can be used to track network activity into, out of, and within your AWS network.
- Security Groups, which act as virtual firewalls on your instances. Security groups allow stateful traffic filtering.
- Network ACLs that operate as virtual, horizontally scalable, stateless packet filtering devices at the subnet level.
- VPC Routing Tables that work at the VPC and subnet layer to control traffic routability within your network.
These tools allow for a flexible security in depth approach to service networking. Routing tables, network ACL’s and Security Groups can all be applied to prevent unwanted network traffic.
However, there are a couple of key points of philosophical divergence between this AWS model and the internal Kubernetes network model are:
- Not all VM’s are routable from one another
- NAT is allowed and even encouraged for public facing services
Of course it is entirely possible to simplify the AWS VPC model down to map directly to a Kubernetes cluster in which all nodes are routable to one another and NAT is not used for inter-service communication. Amazon has worked with Heptio to create a CNI provider which does exactly that. Creating a CNI plugin that operates by attaching secondary ENI’s in one or more mutually routable subnets to worker nodes in your Kubernetes cluster, and assigning them directly to Pod IP’s
However, there are also cases where the constraints of the VPC model (the number of EIC’s available per node in particular) become an issue that may push you towards using another CNI provider on top of the Amazon VPC.
In Part 3, I explore those options and tradeoffs in more detail.
Links to other posts in the series:
