EKS Network 정책 실습 (w/Calico)

ㅇ Egress Access Control 구현 (EKS workload가 다른 Amazon 서비스 (RDS  또는 EC2 인스턴스) 또는 다른 API End Point와 통신)

ㅇ 서로 통신할 수 없는 Microservice Troubleshooting

ㅇ EKS에서 Enterprise 보안 control 구현 및 리포트 

 

ㅁ Install Calico

 

aws/amazon-vpc-cni-k8s GitHub project에서 Calico manifest를 받아서 적용하면  kube-system namespace에 daemon set으로 생성된다.

kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/master/config/v1.6/calico.yaml

 

[calico가 잘 설치되었는지 확인]

 

calico-node daemon이 Ready에서 Desired 수의 포드와 같을때까지 기다림

kubectl get daemonset calico-node --namespace=kube-system

NAME          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                 AGE
calico-node   3         3         3       3            3           beta.kubernetes.io/os=linux   2m35s

 

[Starts Namespace]

mkdir ~/environment/calico_resources
cd ~/environment/calico_resources

cd ~/environment/calico_resources
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/namespace.yaml

cat namespace.yaml 로 데이터 확인

kind: Namespace
apiVersion: v1
metadata:
  name: stars

 

[namespace stars 생성]

kubectl apply -f namespace.yaml

 

[frontend/backend deployment and service yaml download]

cd ~/environment/calico_resources
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/management-ui.yaml
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/backend.yaml
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/frontend.yaml
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/client.yaml

 

[management-ui.yaml 확인]

apiVersion: v1
kind: Namespace
metadata:
  name: management-ui
  labels:
    role: management-ui
---
apiVersion: v1
kind: Service
metadata:
  name: management-ui
  namespace: management-ui
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 9001
  selector:
    role: management-ui
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: management-ui
  namespace: management-ui
spec:
  replicas: 1
  selector:
    matchLabels:
      role: management-ui
  template:
    metadata:
      labels:
        role: management-ui
    spec:
      containers:
      - name: management-ui
        image: calico/star-collect:v0.1.0
        imagePullPolicy: Always
        ports:
        - containerPort: 9001

 

[management ui 생성]

kubectl apply -f management-ui.yaml

 

[backend.yaml 확인]

apiVersion: v1
kind: Service
metadata:
  name: backend
  namespace: stars
spec:
  ports:
  - port: 6379
    targetPort: 6379
  selector:
    role: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
  namespace: stars
spec:
  replicas: 1
  selector:
    matchLabels:
      role: backend
  template:
    metadata:
      labels:
        role: backend
    spec:
      containers:
      - name: backend
        image: calico/star-probe:v0.1.0
        imagePullPolicy: Always
        command:
        - probe
        - --http-port=6379
        - --urls=http://frontend.stars:80/status,http://backend.stars:6379/status,http://client.client:9000/status
        ports:
        - containerPort: 6379

 

[frontend.yaml 확인]

apiVersion: v1
kind: Service
metadata:
  name: frontend
  namespace: stars
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    role: frontend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: stars
spec:
  replicas: 1
  selector:
    matchLabels:
      role: frontend
  template:
    metadata:
      labels:
        role: frontend
    spec:
      containers:
      - name: frontend
        image: calico/star-probe:v0.1.0
        imagePullPolicy: Always
        command:
        - probe
        - --http-port=80
        - --urls=http://frontend.stars:80/status,http://backend.stars:6379/status,http://client.client:9000/status
        ports:
        - containerPort: 80

 

[stars namespace에 frontend, backend service 및 deployment 생성]

kubectl apply -f backend.yaml
kubectl apply -f frontend.yaml

 

[client.yaml 확인]

kind: Namespace
apiVersion: v1
metadata:
  name: client
  labels:
    role: client
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: client
  namespace: client
spec:
  replicas: 1
  selector:
    matchLabels:
      role: client
  template:
    metadata:
      labels:
        role: client
    spec:
      containers:
      - name: client
        image: calico/star-probe:v0.1.0
        imagePullPolicy: Always
        command:
        - probe
        - --urls=http://frontend.stars:80/status,http://backend.stars:6379/status
        ports:
        - containerPort: 9000
---
apiVersion: v1
kind: Service
metadata:
  name: client
  namespace: client
spec:
  ports:
  - port: 9000
    targetPort: 9000
  selector:
    role: client

 

[client.yaml 적용]

kubectl apply -f client.yaml

 

[전체 namespace에 운영되는 Pods 확인]

kubectl get pods --all-namespaces

 

ㅁ Default Pod to Pod Communication

아래 Command로 management UI의 Endpoint를 확인하여 접속해 본다.

$ kubectl get svc -o wide -n management-ui
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP                                                                  PORT(S)        AGE   SELECTOR
management-ui   LoadBalancer   10.100.82.215   a2eb3d26797d9460eb0205acaf79d7ed-42635177.ap-northeast-2.elb.amazonaws.com   80:30695/TCP   17m   role=management-ui

상용환경에서 모든 Pod간 통신이 오픈 되면 보안에 문제가 발생한다.

각 서비스를 서로간에 단절 시키는 것이 필요하다.

 

다음 yaml 파일을 받는다.

cd ~/environment/calico_resources
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/apply_network_policies.files/default-deny.yaml

 

[default-deny.yaml 확인]

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: default-deny
spec:
  podSelector:
    matchLabels: {}

이 yaml을 적용하면 podSelector에 matchLabel에 아무것도 없으므로 모든 pod가 액세스 하지 못하게 차단한다.

 

deny yaml 파일을 stars namespace(frontend and backend service)와 client namespace(client service)에 모두 적용한다.

kubectl apply -n stars -f default-deny.yaml
kubectl apply -n client -f default-deny.yaml

 위의 management UI endpoint를 다시 호출해 보면 아무것도 나타나지 않는다.

 

kubernetes의 네트워크 정책은 Label을 사용하여 Pod를 선택하고 해당 Pod에 도달할 수 있는 트래픽에 대한 규칙을 정의한다. 수신/송신/모두 지정할 수 있다. 각 Rule은 from 및 Port 섹션 모두와 일치하는 트래픽을 허용한다.

 

두개의 새 네트워크 정책을 만든다.

 

cd ~/environment/calico_resources
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/apply_network_policies.files/allow-ui.yaml
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/apply_network_policies.files/allow-ui-client.yaml

 

[allow-ui.yaml 내용 확인]

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: stars
  name: allow-ui
spec:
  podSelector:
    matchLabels: {}
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              role: management-ui

 

[allow-ui-client.yaml 내용 확인]

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: client
  name: allow-ui
spec:
  podSelector:
    matchLabels: {}
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              role: management-ui

 

[management-ui 통신 재적용]

kubectl apply -f allow-ui.yaml
kubectl apply -f allow-ui-client.yaml

management-ui Endpoint를 통해 모든 서비스가 보이긴 하지만 각 서비스간의 통신은 되고 있지 않은 상태이다.

 

ㅁ Allow Directional Traffic

 

ㅇ Client로부터 frontend까지, frontend로부터 backend까지의 트래픽 허용

 

[yaml 파일 다운로드]

cd ~/environment/calico_resources
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/directional_traffic.files/backend-policy.yaml
wget https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/directional_traffic.files/frontend-policy.yaml

 

[backend-policy.yaml 확인]

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: stars
  name: backend-policy
spec:
  podSelector:
    matchLabels:
      role: backend
  ingress:
    - from:
        - podSelector:
            matchLabels:
              role: frontend
      ports:
        - protocol: TCP
          port: 6379

 

[frontend-policy.yaml 확인]

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: stars
  name: frontend-policy
spec:
  podSelector:
    matchLabels:
      role: frontend
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              role: client
      ports:
        - protocol: TCP
          port: 80

Client --> FrontEnd --> BackEnd 로 통신

Client --> FrontEnd로는 namesapceSelector로 오픈, backend, frontend는 같은 namespace이므로 podSelector로 오픈한다.

반대로는 오픈되지 않았으므로 단방향 통신이 됨을 알 수 있다.

 

ㅁ Cleanup 

 

namespace를 삭제하는 것으로 적용된 정책을 모두 삭제할 수 있다.

kubectl delete namespace client stars management-ui