A brief introduction to the CNI (Container Network Interface), the implementation of docker bridge network and the CNI usage, including why we develop the CNI, how to use the CNI and what is CNI. We also introduction the pause container the kubernetes PoD and how to use the CNI in the kubernetes. In the end, we use the flannel as an example to show how to install the CNI into your kubernetes cluster

1. 從入門到放棄，原來
CNI 這麼複雜

2. WHO AM I
✖ Hung-Wei Chiu (邱宏瑋)
✖ Hwchiu.com
✖ Experience
○ Software Engineer at Linker Networks
(now)
○ Software Engineer Synology
○ Co-organizer of SDNDS-TW
○ Co-organizer of CUTNG
✖ Fields
○ Linux Kernel, Network Stack
○ SDN/SDS/Kubernetes

3. OUTLINE
✖ What is CNI
✖ How Kubernetes use CNI
✖ What is Flannel

4. What is CNI
Container Network Interface

5. Before that, we need to quick review
how Docker setup the network for its
container.

7. How Docker Works
✖ Mount namespaces
✖ IPC namespaces
✖ PID namespaces
✖ Network namespaces
✖ User namespaces
✖ UTS namespaces

8. We use the default docker network,
Bridge Mode.

9. ✖ Docker run –p 12345:80 nginx
✖ You can access the nginx via
localhost:12345
✖ The nginx container has the network
connectivity

10. Linux Bridge Network
✖ Create a linux bridge
✖ Create a linux network namespace
✖ Create a veth pair
✖ Attach the veth pair into the
namespace and linux bridge
✖ Set the ip address
✖ Set the route rules
✖ Set the iptables

11. br0 br0
br0br0br0
ns1
ns1ns1ns1
vth1
vth0
vth1
vth0
vth1
vth0
Linux Host Linux Host Linux Host
Linux HostLinux HostLinux Host

12. Now, How about the Docker OVS
network ?

13. OVS Network
✖ Create a OVS bridge
✖ Create a linux network namespace
✖ Create a veth pair
✖ Attach the veth pair into the
namespace and linux bridge
✖ Set the ip address
✖ Set the route rules
✖ Set the OVS options

14. As a network plugin developer
✖ How many container system
✖ Docker
✖ Rkt
✖ LXC/LXD
✖ OpenVZ
✖ RunC
✖ ….

15. Should I Rewrite My Network Plugin for
Every Containers?

18. Container Network Interface

19. Container Network Interface
✖ Cloud Native Computing Foundation
Project
✖ Consists of a specification and
libraries.
✖ Configure network interfaces in Linux
containers
✖ Concerns itself only with network
connectivity of containers
○ Create/Remove

20. Container Network Interface
✖ Removing allocated resources when
the container is deleted
✖ Based on golang

21. Why develop CNI?

22. To avoid duplication,

23. Who is using CNI?

24. From the GITHUB
 rkt - container engine
 Kubernetes - a system to simplify container operations
 OpenShift - Kubernetes with additional enterprise
features
 Cloud Foundry - a platform for cloud applications
 Apache Mesos - a distributed systems kernel
 Amazon ECS - a highly scalable, high performance
container management service

25. So, How to use the CNI?

26.  CNI_COMMAND=ADD
 CNI_CONTAINERID=ns1
 CNI_NETNS=/var/run/netns/ns1
 CNI_IFNAME=eth10
 CNI_STDIN=….

27.  CNI_COMMAND=ADD
 CNI_CONTAINERID=ns1
 CNI_NETNS=/var/run/netns/ns1
 CNI_IFNAME=eth10
 CNI_STDIN=….
Take the linux bridge as an example
✖ Create a linux network namespace
✖ Create a linux bridge
✖ Create a veth pair
✖ Attach the veth pair into the
namespace and linux bridge
✖ Set the ip address
✖ Set the route rules
✖ Set the iptables
IPAM

28. Cni_stdnin example

29. How k8s use cni

30. How Docker Works
✖ Create a kubernetes cluster
✖ Setup your CNI plugin
✖ Deploy your first POD

31. Just follow the installation to install the
kubernetes

32. How do we install the CNI?

34. Hand by hand
✖ In the kubelet, we have the following
parameters for CNI.
✖ --cni-bin-dir
○ /opt/cni/bin
✖ --cni-conf-dir
○ /etc/cni/net.d/
✖ We should config the CNI for every
k8s nodes.

35. Let Deploy a POD

36. Before we start
✖ Pod
○ A collection of containers

37. Steps
✖ Load the Pod config
○ Multiple containers
✖ Find a node to deploy the pod
✖ Create a Pause container
✖ Load the CNI config
✖ Execute the CNI
✖ Create target containers and attach to
Pause container

38. Linux Host

39. Linux Host
Pause
Container

40. Linux Host
Pause
Container
Call /opt/cni/bin/CNI
/etc/cni/net.d/
xxx.conf

41. Linux Host
Pause
Container
Network
Connectivity

42. Linux Host
Pause
Container
Network
Connectivity
contaienr1
contaienr2
contaienr3

43. Linux Host
Pause
Container
Network
Connectivity
contaienr1
contaienr2
contaienr3

44. Linux Host
Pause
Container
Network
Connectivity
contaienr1
contaienr2
contaienr3
POD

45. What is flannel

46. flannel
✖ Famous CNI
✖ Created by CoreOS
✖ Layer3 Network (VXLAN, UDP)
○ VXLAN >>>> UDP
○ Kernel Space >>>> User Space
✖ Easy to setup (one yaml)
✖ Centrally manage by etcd/k8s API.
○ For K8S API, we need to set –pod-cidr for
kubelet

47. What is VXLAN
✖ Virtual eXtensible LAN
✖ Overlay network
✖ Based on UDP

51. How it works
192.168.78.2 -> 192.168.87.2Layer 2UDP/VXLAN138.197.204.124 -> 138.68.49.202
Original Packet HeaderAdditional Header

52. ✖ VTEP-1 should know that
that 192.1.87/24 should
forward to 138.68.49.202
✖ VTEP-2 should know that
that 192.1.78/24 should
forward to 138.197.204.124
How it works

53. How vetps knows that
✖ Multicast
✖ L3 Routing
○ BGP
✖ Unicast (Flannel)
○ Event-Driven
■ Listen netlink event
○ Static Setting
■ Just setting rules with timeout

54. Steps by steps about falnnel
✖ Install the kubernetes cluster
✖ Apply the Flannel YAML
○ Deploy a CNI config to all k8s nodes.

55. Before we start
✖ Config Map
○ A global config in whole k8s cluster
✖ DaemonSet
○ A container running on all nodes.

56. ✖ Create a k8s-config-map
✖ cni-conf -> for CNI
✖ net-conf -> for Flanneld
step1

57. step2
✖ Create a daemon set

58. The Problem
✖ Flannel is a CNI
✖ DaemonSet is a POD, needs the IP
address
✖ Need the IP address from the
Flannel CNI

59. step2
✖ Create a daemon set
✖ Two containers

60. step3
✖ Daemon load the net-
conf.json
✖ Get the IP subnet via
etcd/API
✖ Output the file into
/run/flannel/subnet.env
Subnet.env
FLANNEL_NETWORK=10.1.0.0/16
FLANNEL_SUBNET=10.1.17.1/24
FLANNEL_MTU=1472
FLANNEL_IPMASQ=true

61. step4
✖ Fetch the vxlan infor from etcd/API
✖ Create an VXLAN interface
✖ Set the VXLAN routing rules

62. step5
✖ When k8s decides to deploy a POD on this node.
✖ Call the flannel CNI
○ Load the config from the /etc/cni/net.d
○ We copy that files in the daemon set.
✖ The flannel CNI
○ Load the /fun/flannel/subnet.env
○ Get one available IP address and assign to the
PoD

63. Summary
Node 1 Node 2 Node 3
Network
172.16.2.2 172.16.2.3 172.16.2.4
Kubernetes Cluster

64. Summary
Node 1 Node 2 Node 3
Network
172.16.2.2 172.16.2.3 172.16.2.4
Kubernetes Cluster
ConfigMap:
Cni.conf
Net.con

65. Summary
Node 1 Node 2 Node 3
Network
172.16.2.2 172.16.2.3 172.16.2.4
Kubernetes Cluster
ConfigMap:
Cni.conf
Net.con
FlannelD
172.16.2.2
FlannelD
172.16.2.3
FlannelD
172.16.2.4
Copy the cni.conf to
/etc/cni/net.d
Generate the
/run/flannel/sub.env
Copy the cni.conf to
/etc/cni/net.d
Generate the
/run/flannel/sub.env
Copy the cni.conf to
/etc/cni/net.d
Generate the
/run/flannel/sub.env

66. Summary
Node 1 Node 2 Node 3
Network
172.16.2.2 172.16.2.3 172.16.2.4
Kubernetes Cluster
ConfigMap:
Cni.conf
Net.con
FlannelD
172.16.2.2
FlannelD
172.16.2.3
FlannelD
172.16.2.4
FLANNEL_NETWORK=10.1.0.0/16
FLANNEL_SUBNET=10.1.18.1/24
FLANNEL_MTU=1472
FLANNEL_IPMASQ=true
FLANNEL_NETWORK=10.1.0.0/16
FLANNEL_SUBNET=10.1.19.1/24
FLANNEL_MTU=1472
FLANNEL_IPMASQ=true
FLANNEL_NETWORK=10.1.0.0/16
FLANNEL_SUBNET=10.1.17.1/24
FLANNEL_MTU=1472
FLANNEL_IPMASQ=true
Assume the network_cidr is
10.1.0.0/16

67. Summary
Node 1 Node 2 Node 3
Network
172.16.2.2 172.16.2.3 172.16.2.4
Kubernetes Cluster
ConfigMap:
Cni.conf
Net.con
FlannelD
172.16.2.2
FlannelD
172.16.2.3
FlannelD
172.16.2.4
Deploy three busybox
Busybox Busybox Busybox
10.1.17.5 10.1.18.6
10.1.19.7

68. Summary
✖ CNI
○ Avoid Duplicatin
✖ K8S
○ Pause Container
○ Other use –net=` pause
containerID` to attach
✖ Flannel
○ VXLAN
○ DaemonSet + ConfigMap

69. Thanks!
Any questions?