Deploying Jenkins for Kubernetes

먼저 CodeCommit repository를 생성한다. 이 repository는 application code와 Jenkinsfile을 저장하기 위한 장소이다.

aws codecommit create-repository --repository-name eksworkshop-app

{
    "repositoryMetadata": {
        "repositoryName": "eksworkshop-app", 
        "cloneUrlSsh": "ssh://git-codecommit.ap-northeast-2.amazonaws.com/v1/repos/eksworkshop-app", 
        "lastModifiedDate": 1616109672.528, 
        "repositoryId": "826bfbfd-2a75-4242-ac8d-87ad614193fd", 
        "cloneUrlHttp": "https://git-codecommit.ap-northeast-2.amazonaws.com/v1/repos/eksworkshop-app", 
        "creationDate": 1616109672.528, 
        "Arn": "arn:aws:codecommit:ap-northeast-2:221745184950:eksworkshop-app", 
        "accountId": "221745184950"
    }
}

 

AWS CodeCommit에 대한 HTTPS Git 자격증명을 갖은 IAM 사용자를 생성하여 Repository로 복제하고 추가 Commit을 Push 한다. 이 사용자는 CodeCommit에 엑세스를 위한 IAM Policy를 필요로 한다.

 

aws iam create-user \
  --user-name git-user

aws iam attach-user-policy \
  --user-name git-user \
  --policy-arn arn:aws:iam::aws:policy/AWSCodeCommitPowerUser

aws iam create-service-specific-credential \
  --user-name git-user --service-name codecommit.amazonaws.com \
  | tee /tmp/gituser_output.json

GIT_USERNAME=$(cat /tmp/gituser_output.json | jq -r '.ServiceSpecificCredential.ServiceUserName')
GIT_PASSWORD=$(cat /tmp/gituser_output.json | jq -r '.ServiceSpecificCredential.ServicePassword')
CREDENTIAL_ID=$(cat /tmp/gituser_output.json | jq -r '.ServiceSpecificCredential.ServiceSpecificCredentialId')

 

적용 결과는 다음과 같다.

{
    "User": {
        "UserName": "git-user", 
        "Path": "/", 
        "CreateDate": "2021-03-18T23:24:41Z", 
        "UserId": "AIDATHIILAC3KMDOJ7XH5", 
        "Arn": "arn:aws:iam::221745184950:user/git-user"
    }
}

eksuser:~/environment $ aws iam attach-user-policy \
>   --user-name git-user \
>   --policy-arn arn:aws:iam::aws:policy/AWSCodeCommitPowerUser
eksuser:~/environment $ 
eksuser:~/environment $ aws iam create-service-specific-credential \
>   --user-name git-user --service-name codecommit.amazonaws.com \
>   | tee /tmp/gituser_output.json
{
    "ServiceSpecificCredential": {
        "UserName": "git-user", 
        "Status": "Active", 
        "CreateDate": "2021-03-18T23:36:01Z", 
        "ServiceName": "codecommit.amazonaws.com", 
        "ServicePassword": "ViyHMvMSbBEUmCq7f7QbFuQ3f8ngDmcmnjzyLexrtQU=", 
        "ServiceSpecificCredentialId": "ACCATHIILAC3BSHPEOUCB", 
        "ServiceUserName": "git-user-at-221745184950"
    }
}

 

Repository에는 몇가지 초기 Code가 필요하므로 repository를 clone하고 간단한 Go Application을 추가한다.

sudo pip install git-remote-codecommit

WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Collecting git-remote-codecommit
  Using cached https://files.pythonhosted.org/packages/1f/82/7c22f218a7fd3177def489febc9b8c262a3b2bcb6785d05e15d435ddcab8/git-remote-codecommit-1.15.1.tar.gz
Requirement already satisfied: botocore>=1.17.0 in /usr/lib/python2.7/site-packages (from git-remote-codecommit)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python2.7/site-packages (from botocore>=1.17.0->git-remote-codecommit)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/lib/python2.7/site-packages (from botocore>=1.17.0->git-remote-codecommit)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python2.7/site-packages (from botocore>=1.17.0->git-remote-codecommit)
Requirement already satisfied: six>=1.5 in /usr/lib/python2.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore>=1.17.0->git-remote-codecommit)
Installing collected packages: git-remote-codecommit
  Running setup.py install for git-remote-codecommit ... done
Successfully installed git-remote-codecommit-1.15.1

git clone codecommit::${AWS_REGION}://eksworkshop-app

Cloning into 'eksworkshop-app'...
warning: You appear to have cloned an empty repository.

 

Sample server.go 프로그램을 작성한다.

cat << EOF > server.go

package main

import (
    "fmt"
    "net/http"
)

func helloWorld(w http.ResponseWriter, r *http.Request){
    fmt.Fprintf(w, "Hello World")
}

func main() {
    http.HandleFunc("/", helloWorld)
    http.ListenAndServe(":8080", nil)
}
EOF

Sampe server_test.go 프로그램을 작성한다.

cat << EOF > server_test.go

package main

import (
	"net/http"
	"net/http/httptest"
	"testing"
)

func Test_helloWorld(t *testing.T) {
	req, err := http.NewRequest("GET", "http://domain.com/", nil)
	if err != nil {
		t.Fatal(err)
	}

	res := httptest.NewRecorder()
	helloWorld(res, req)

	exp := "Hello World"
	act := res.Body.String()
	if exp != act {
		t.Fatalf("Expected %s got %s", exp, act)
	}
}

EOF

 

The Jenkinsfile will contain our pipeline declaration, the additional containers in our build agent pods, and which container will be used for each step of the pipeline.

 

cat << EOF > Jenkinsfile
pipeline {
  agent {
    kubernetes {
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: golang
    image: golang:1.13
    command:
    - cat
    tty: true
"""
    }
  }
  stages {
    stage('Run tests') {
      steps {
        container('golang') {
          sh 'go test'
        }
      }
    }
    stage('Build') {
        steps {
            container('golang') {
              sh 'go build -o eksworkshop-app'
              archiveArtifacts "eksworkshop-app"
            }
            
        }
    }
    
  }
}

EOF

 

위에서 생성한 code를 repository에 commit 하고 push 한다.

git add --all && git commit -m "Initial commit." && git push
cd ~/environment

 

Kuberenetes가 CodeCommit API 작업을 수행해야 하는 경우 Pod에 부여할 서비스 계정을 생성해야 한다. 이를 통해 Jenkins는 새로운 repository, branch, commit을 수행할 수 있다.

 

먼저 eks cluster를 위한 OIDC identity provider를 생성한다.

eksctl utils associate-iam-oidc-provider --cluster eksworkshop-eksctl --approve

 

이제 iamserviceaccount를 생성한다. 

eksctl create iamserviceaccount \
    --name jenkins \
    --namespace default \
    --cluster eksworkshop-eksctl \
    --attach-policy-arn arn:aws:iam::aws:policy/AWSCodeCommitPowerUser \
    --approve \
    --override-existing-serviceaccounts

 

ㅁ Deploy Jenkins

 

Jenkins installation의 configuration을 선언하기 위하여 values.yaml을 생성한다.

cat << EOF > values.yaml
---
master:
  additionalPlugins:
    - aws-codecommit-jobs:0.3.0
  resources:
    requests:
      cpu: "1024m"
      memory: "4Gi"
    limits:
      cpu: "4096m"
      memory: "8Gi"
  javaOpts: "-Xms4000m -Xmx4000m"
  servicePort: 80
  serviceType: LoadBalancer
agent:
  Enabled: false
rbac:
  create: true
serviceAccount:
  create: false
  name: "jenkins"
EOF

 

value.yaml을 사용하여 helm으로 jenkins를 설치한다.

helm install cicd stable/jenkins -f values.yaml

 설치를 진행하면 admin password 와 ELB의 호스트 이름을 가져오는 방법과 같은 몇가지 추가적인 정보를 제공한다.

 

kubectl get pods -w

NAME                           READY   STATUS    RESTARTS   AGE
cicd-jenkins-7854d9569-xxd7c   2/2     Running   0          5m14s

를 해보면 pod의 상태를 볼수 있다. 이 후 아래 명령어를 통해 jenkins 접근 url을 확인한다.

export SERVICE_IP=$(kubectl get svc --namespace default cicd-jenkins --template "{{ range (index .status.loadBalancer.ingress 0) }}{{ . }}{{ end }}")

echo http://$SERVICE_IP/login

정상적으로 Jenkin 로그인 화면이 뜨면 된다.

 

로그인을 위한 user name은 admin이며 password는 아래 명령어를 통해 확인한다.

 

printf $(kubectl get secret --namespace default cicd-jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo