Resource Quotas

ResoureQuotas는 CPU, Mem, Storage 및 서비스와 같은 리소스를 제한하는 데 사용된다. 이 섹션에서는 ResourceQuotas를 Blue, Red 두 팀을 만들어 설정한다.

 

# Create different namespaces
kubectl create namespace blue
kubectl create namespace red

 

Resource Quota 생성

 

두 팀이 같은 리소스를 공유한다. Red팀은 Load Balancer 수에 제한을 두고 Blue팀은 메모리/CPU 사용량 제한을 한다.

kubectl create quota blue-team --hard=limits.cpu=1,limits.memory=1G --namespace blue
kubectl create quota red-team --hard=services.loadbalancers=1 --namespace red

 

참고 - 할당량 설정 가능 객체 목록

kubernetes.io/docs/concepts/policy/resource-quotas/

Resource Quotas

 

Pods 생성

 

리소스 생성 시도를 하면서 실패 및 성공을 확인한다.

 

실패 시도

 

ResourceQuota 사양을 벗어난 포드를 생성하면 오류 발생

# Error when creating a resource without defined limit
kubectl run --namespace blue --image hande007/stress-ng blue-cpu-pod --restart=Never --  --vm-keep --vm-bytes 512m --timeout 600s --vm 2 --oomable --verbose

# Error when creating a deployment without specifying limits (Replicaset has errors)
kubectl create --namespace blue deployment blue-cpu-deploy --image hande007/stress-ng
kubectl describe --namespace blue replicaset -l app=blue-cpu-deploy  

# Error when creating more than one AWS Load Balancer
kubectl run --namespace red --image nginx:latest red-nginx-pod --restart=Never --limits=cpu=0.1,memory=100M
kubectl expose --namespace red pod red-nginx-pod --port 80 --type=LoadBalancer --name red-nginx-service-1
kubectl expose --namespace red pod red-nginx-pod --port 80 --type=LoadBalancer --name red-nginx-service-2

 

수행 결과

eksuser:~/environment $ kubectl create quota blue-team --hard=limits.cpu=1, limits.memory=1G --namespace blue
error: exactly one NAME is required, got 2
See 'kubectl create quota -h' for help and examples
eksuser:~/environment $ kubectl create quota blue-team --hard=limits.cpu=1,limits.memory=1G --namespace blue
resourcequota/blue-team created
eksuser:~/environment $ kubectl create quota red-team --hard=services.loadbalancers=1 --namespace red
resourcequota/red-team created
eksuser:~/environment $ kubectl run --namespace blue --image hande007/stress-ng blue-cpu-pod --restart=Never -- --vm-keep --vm-bytes 512m --timeout 600s --vm 2 --oomable --verbose
Error from server (Forbidden): pods "blue-cpu-pod" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
eksuser:~/environment $ kubectl create --namespace blue deployment blue-cpu-deploy --image hande007/stress-ng
deployment.apps/blue-cpu-deploy created
eksuser:~/environment $ kubectl describe --namespace blue replicaset -l app=blue-cpu-deploy
Name:           blue-cpu-deploy-67d5df5ff4
Namespace:      blue
Selector:       app=blue-cpu-deploy,pod-template-hash=67d5df5ff4
Labels:         app=blue-cpu-deploy
                pod-template-hash=67d5df5ff4
Annotations:    deployment.kubernetes.io/desired-replicas: 1
                deployment.kubernetes.io/max-replicas: 2
                deployment.kubernetes.io/revision: 1
Controlled By:  Deployment/blue-cpu-deploy
Replicas:       0 current / 1 desired
Pods Status:    0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=blue-cpu-deploy
           pod-template-hash=67d5df5ff4
  Containers:
   stress-ng:
    Image:        hande007/stress-ng
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type             Status  Reason
  ----             ------  ------
  ReplicaFailure   True    FailedCreate
Events:
  Type     Reason        Age               From                   Message
  ----     ------        ----              ----                   -------
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-fhftw" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-d7jwx" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-czrb9" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-nk784" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-hs7n9" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-hll2g" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-pdc58" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  23s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-rvjr6" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  22s               replicaset-controller  Error creating: pods "blue-cpu-deploy-67d5df5ff4-dq4l7" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory
  Warning  FailedCreate  3s (x4 over 21s)  replicaset-controller  (combined from similar events): Error creating: pods "blue-cpu-deploy-67d5df5ff4-r7blp" is forbidden: failed quota: blue-team: must specify limits.cpu,limits.memory

eksuser:~/environment $ kubectl run --namespace red --image nginx:latest red-nginx-pod --restart=Never --limits=cpu=0.1,memory=100M
pod/red-nginx-pod created
eksuser:~/environment $ kubectl expose --namespace red pod red-nginx-pod --port 80 --type=LoadBalancer --name red-nginx-service-1
service/red-nginx-service-1 exposed
eksuser:~/environment $ kubectl expose --namespace red pod red-nginx-pod --port 80 --type=LoadBalancer --name red-nginx-service-2
Error from server (Forbidden): services "red-nginx-service-2" is forbidden: exceeded quota: red-team, requested: services.loadbalancers=1, used: services.loadbalancers=1, limited: services.loadbalancers=1
eksuser:~/environment $ 

 

성공적인 시도

 

할당된 리소스의 75%까지 생성 시도

# Create Pod
kubectl run --namespace blue --limits=cpu=0.25,memory=250M --image nginx blue-nginx-pod-1 --restart=Never --restart=Never
kubectl run --namespace blue --limits=cpu=0.25,memory=250M --image nginx blue-nginx-pod-2 --restart=Never --restart=Never
kubectl run --namespace blue --limits=cpu=0.25,memory=250M --image nginx blue-nginx-pod-3 --restart=Never --restart=Never

 

현재 리소스 할당량 사용량 확인

eksuser:~/environment $ kubectl describe quota blue-team --namespace blue
Name:          blue-team
Namespace:     blue
Resource       Used  Hard
--------       ----  ----
limits.cpu     750m  1
limits.memory  750M  1G
eksuser:~/environment $ kubectl describe quota red-team --namespace red
Name:                   red-team
Namespace:              red
Resource                Used  Hard
--------                ----  ----
services.loadbalancers  1     1

전체 포드 정리

kubectl delete namespace red
kubectl delete namespace blue