prometheus와 grafana를 kuberenetes 환경에 설치하기 위해서는 먼저
helm repo를 먼저 추가한다.
# add prometheus Helm repo
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
# add grafana Helm repo
helm repo add grafana https://grafana.github.io/helm-charts
Deploy prometheus
먼저 prometheus의 namespace를 만들고 gp2 EBS Volume을 사용하여 prometheus 저장 공간을 생성한다.
kubectl create namespace prometheus
helm install prometheus prometheus-community/prometheus \
--namespace prometheus \
--set alertmanager.persistentVolume.storageClass="gp2" \
--set server.persistentVolume.storageClass="gp2"
output
eksuser:~/environment $ helm install prometheus prometheus-community/prometheus \
> --namespace prometheus \
> --set alertmanager.persistentVolume.storageClass="gp2" \
> --set server.persistentVolume.storageClass="gp2"
NAME: prometheus
LAST DEPLOYED: Fri Mar 19 23:51:10 2021
NAMESPACE: prometheus
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-server.prometheus.svc.cluster.local
Get the Prometheus server URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace prometheus port-forward $POD_NAME 9090
The Prometheus alertmanager can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-alertmanager.prometheus.svc.cluster.local
Get the Alertmanager URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=alertmanager" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace prometheus port-forward $POD_NAME 9093
#################################################################################
###### WARNING: Pod Security Policy has been moved to a global property. #####
###### use .Values.podSecurityPolicy.enabled with pod-based #####
###### annotations #####
###### (e.g. .Values.nodeExporter.podSecurityPolicy.annotations) #####
#################################################################################
The Prometheus PushGateway can be accessed via port 9091 on the following DNS name from within your cluster:
prometheus-pushgateway.prometheus.svc.cluster.local
Get the PushGateway URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=pushgateway" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace prometheus port-forward $POD_NAME 9091
For more information on running Prometheus, visit:
https://prometheus.io/prometheus가 정상적으로 설치 되었는지 확인한다.
eksuser:~/environment $ kubectl get all -n prometheus
NAME READY STATUS RESTARTS AGE
pod/prometheus-alertmanager-7959c96f57-xnpds 2/2 Running 0 72s
pod/prometheus-kube-state-metrics-6bfcd6f648-8qdxk 1/1 Running 0 72s
pod/prometheus-node-exporter-f2ddh 1/1 Running 0 72s
pod/prometheus-node-exporter-fdq76 1/1 Running 0 72s
pod/prometheus-node-exporter-q42w9 1/1 Running 0 72s
pod/prometheus-pushgateway-5987dd58b7-f4wjq 1/1 Running 0 72s
pod/prometheus-server-bf4f4cd78-4x7t8 2/2 Running 0 72s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/prometheus-alertmanager ClusterIP 10.100.68.224 <none> 80/TCP 72s
service/prometheus-kube-state-metrics ClusterIP 10.100.136.225 <none> 8080/TCP 72s
service/prometheus-node-exporter ClusterIP None <none> 9100/TCP 72s
service/prometheus-pushgateway ClusterIP 10.100.190.236 <none> 9091/TCP 72s
service/prometheus-server ClusterIP 10.100.197.135 <none> 80/TCP 72s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/prometheus-node-exporter 3 3 3 3 3 <none> 72s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/prometheus-alertmanager 1/1 1 1 72s
deployment.apps/prometheus-kube-state-metrics 1/1 1 1 72s
deployment.apps/prometheus-pushgateway 1/1 1 1 72s
deployment.apps/prometheus-server 1/1 1 1 72s
NAME DESIRED CURRENT READY AGE
replicaset.apps/prometheus-alertmanager-7959c96f57 1 1 1 72s
replicaset.apps/prometheus-kube-state-metrics-6bfcd6f648 1 1 1 72s
replicaset.apps/prometheus-pushgateway-5987dd58b7 1 1 1 72s
replicaset.apps/prometheus-server-bf4f4cd78 1 1 1 72sprometheus 서버 URL에 엑세스 하기 위해 kubectl port-forward 명령을 사용하여 애플리케이션에 액세스 할 것이다.
Cloud9 다음을 실행 해보자
kubectl port-forward -n prometheus deploy/prometheus-server 8080:9090
Cloud9 환경엥서 Tools > Preview > Preview Running application을 클릭 후 url의 amazonaws.com 뒤에 /targets을 붙여서 접속해 본다. (Prometheus Menu에서 Status -> Targets 을 클릭해서 접속 가능하다.
ㅁ Grafana 배포
이 예에서는 Grafana 기본값을 사용하지만 여러 매개 변수를 재정의한다. Prometheus와 마찬가지로 Storage Class를 gp2로 설정하겨 admin 비밀번호 설정, 데이터 소스를 Prometheus를 가리키도록 구성하고 서비스를 위한 외부 load blaancer를 생성한다.
다음과 같이 grafana.yaml을 생성한다.
mkdir ${HOME}/environment/grafana
cat << EoF > ${HOME}/environment/grafana/grafana.yaml
datasources:
datasources.yaml:
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
url: http://prometheus-server.prometheus.svc.cluster.local
access: proxy
isDefault: true
EoF
grafana.yaml 설정값으로 grafana를 생성한다.
kubectl create namespace grafana
helm install grafana grafana/grafana \
--namespace grafana \
--set persistence.storageClassName="gp2" \
--set persistence.enabled=true \
--set adminPassword='EKS!sAWSome' \
--values ${HOME}/environment/grafana/grafana.yaml \
--set service.type=LoadBalancer
output
eksuser:~/environment $ helm install grafana grafana/grafana \
> --namespace grafana \
> --set persistence.storageClassName="gp2" \
> --set persistence.enabled=true \
> --set adminPassword='EKS!sAWSome' \
> --values ${HOME}/environment/grafana/grafana.yaml \
> --set service.type=LoadBalancer
NAME: grafana
LAST DEPLOYED: Sat Mar 20 00:43:18 2021
NAMESPACE: grafana
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:
kubectl get secret --namespace grafana grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
2. The Grafana server can be accessed via port 80 on the following DNS name from within your cluster:
grafana.grafana.svc.cluster.local
Get the Grafana URL to visit by running these commands in the same shell:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get svc --namespace grafana -w grafana'
export SERVICE_IP=$(kubectl get svc --namespace grafana grafana -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
http://$SERVICE_IP:80
3. Login with the password from step 1 and the username: admin정상적으로 grafana가 설치되었는지 확인한다.
kubectl get all -n grafanaoutput
eksuser:~/environment $ kubectl get all -n grafana
NAME READY STATUS RESTARTS AGE
pod/grafana-5464fcd5bc-dkhlc 1/1 Running 0 51s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/grafana LoadBalancer 10.100.190.140 ae2e52d2093024583896f73cf7b80619-1580620614.ap-northeast-2.elb.amazonaws.com 80:31425/TCP 51s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/grafana 1/1 1 1 51s
NAME DESIRED CURRENT READY AGE
replicaset.apps/grafana-5464fcd5bc 1 1 1 51s
EXTERNAL-IP로 grafana에 접근 가능하지만 아래의 명령어로 정확한 url을 확인한다.
export ELB=$(kubectl get svc -n grafana grafana -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "http://$ELB"
username admin과 그 패스워드를 확인한다.
kubectl get secret --namespace grafana grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
이후 위의 URL로 접속하여 Grafana로 접속한다.
ㅁ Cluster Monitoring Dashboard 구성
3119를 넣어서 load 한다.
ㅁ Pods Monitoring Dashboard 구성
6417을 선택하고 load 한다.
ㅁ Cleanup
helm uninstall prometheus --namespace prometheus
kubectl delete ns prometheus
helm uninstall grafana --namespace grafana
kubectl delete ns grafana
rm -rf ${HOME}/environment/grafana
'AWS EKS 실습 > EKS Intermediate' 카테고리의 다른 글
| CI/CD with CodePipeline (0) | 2021.03.20 |
|---|---|
| Implement Logging with EFK (0) | 2021.03.19 |
| CI/CD with CodePipeline (다시 볼것) (0) | 2021.03.19 |
| Deploying Jenkins for Kubernetes (0) | 2021.03.19 |
| Advanced POD CPU and Memory management (0) | 2021.03.18 |
