목차
개요
- ECR 로 demo 이미지 push
- push 된 demo 이미지를 가리키는 kustomization.yaml 생성 후 cluster 변화 확인
- ECR 의 demo 이미지 태그 버전 번경 후 kustomization.yaml 수정 및 cluster 반영 확인
1. ECR 생성 및 demo image PUSH
Dockerfile
FROM nginx
RUN apt-get update
COPY ./ /usr/share/nginx/html/
EXPOSE 80
index.html
<!doctype html>
<html>
<head>
<title>ecr with argocd</title>
</head>
<body>
<H2>ecr with argocd</H2>
<HR>
ecr with argocd
</body>
</html>
aws configure
$ aws configure
AWS Access Key ID [None]: my-access-key
AWS Secret Access Key [None]: my-secret-access-key
Default region name [None]: ap-northeast-2
Default output format [None]: json
1.1 ECR 클라이언트 인증
aws ecr get-login-password --region ap-northeast-2 | docker login --username AWS --password-stdin [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com
WARNING! Your password will be stored unencrypted in /home/cyaninn/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
1.2 build/push
# build
$ docker build --tag test-gitlab-cicd:v.1 .
[+] Building 55.6s (9/9) FINISHED
# tagging
$ docker tag test-gitlab-cicd:v.1 [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd:v.1
cyaninn@cyaninn-15U780-GR36K:~/test-ecr-kusto$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
[aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd v.1 a51c0d5534a4 2 minutes ago 206MB
# push
$ docker push [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd:v.1
1.3 확인
2. k8s secret 생성
private ECR 에서 pull 해오려면 aws 자격증명을 secret 에 등록해서 사용해야 함
2.1 cluster node 에 awscli 설치
$ sudo apt install awscli
$ aws configure
$ aws --version
aws-cli/1.22.34 Python/3.10.6 Linux/5.19.0-1025-aws botocore/1.23.34
$ aws configure
AWS Access Key ID [****************IRZV]:
AWS Secret Access Key [****************F/PL]:
Default region name [ap-northeast-2]:
Default output format [json]:
2.2 secret 생성
k create secret docker-registry regcred \
--docker-server=[aws_account_id].dkr.ecr.[region].amazonaws.com \
--docker-username=AWS \
--docker-password=$(aws ecr get-login-password) \
--namespace=testnamespace, secret 이름 주의
생성 확인
ubuntu@k3s-master:~$ k create secret docker-registry regcred \
--docker-server=${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com \
--docker-username=AWS \
--docker-password=$(aws ecr get-login-password) \
--namespace=test
secret/regcred created
ubuntu@k3s-master:~$ k get secret -n test
NAME TYPE DATA AGE
regcred kubernetes.io/dockerconfigjson 1 10s
3. kustomization.yaml
3.1 github repository 생성
3.2 push manifest
kustomization.yaml
resources:
- pod-nginx.yaml
images:
- name: [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd
newTag: v.1
pod-nginx.yaml
apiVersion: v1
kind: Pod
metadata:
name: my-app
namespace: test
spec:
containers:
- name: nginx
image: [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd
ports:
- containerPort: 80
imagePullSecrets:
- name: regcredImage Pull Policy: Always enforce image force pull to avoid unexpected issues when k8s doesn't pull an image from a remote repository.
push
$ git add .
$ git commit -m "create k3s manifest"
$ git push
Username for 'https://github.com': cyaninn-entj
Password for 'https://cyaninn-entj@github.com':
Enumerating objects: 6, done.
Counting objects: 100% (6/6), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 561 bytes | 561.00 KiB/s, done.
Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/cyaninn-entj/ecr-argocd-manifest-test.git
9d7ef9a..9b9615c main -> mainpersonal access token 발급받아 push 함
확인
4. argoCD app 생성
4.1 namespace 생성
k create namespace test
4.2 argocd app 생성
5. 확인
$ k get all -n test
NAME READY STATUS RESTARTS AGE
pod/my-app 1/1 Running 0 7m53s
6. 버전 변경 및 확인
6.1 버전 변경 및 push
# build
$ docker build --tag test-gitlab-cicd:v.2 .
[+] Building 55.6s (9/9) FINISHED
# tagging
$ docker tag test-gitlab-cicd:v.2 [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd:v.2
cyaninn@cyaninn-15U780-GR36K:~/test-ecr-kusto$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
[aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd v.2 a51c0d5534a4 2 minutes ago 206MB
# push
$ docker push [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd:v.2
6.2 kustomization.yaml 수정 후 push
# kustomization.yaml
resources:
- pod-nginx.yaml
images:
- name: [aws_account_id].dkr.ecr.[my-region].amazonaws.com/test-gitlab-cicd
newTag: v.2git add .
git commit -m "change image version"
git push
6.3 확인
$ k describe pod my-app -n test
Name: my-app
Namespace: test
Priority: 0
Service Account: default
Node: k3s-master/172.31.14.132
Start Time: Tue, 01 Aug 2023 04:19:12 +0900
Labels: app.kubernetes.io/instance=test-privaate-ecr-with-kustomize
Annotations: <none>
Status: Running
IP: 10.42.0.46
IPs:
IP: 10.42.0.46
Containers:
nginx:
Container ID: containerd://d3fd41de0167f57e01fcda166a24e8dc90f940749dec355f5aa10ea0df8797ab
Image: [aws_account_id].dkr.ecr.ap-northeast-2.amazonaws.com/test-gitlab-cicd:v.2describe 확인
ERRORS
Unable to create application: application spec for test-ecr-argocd is invalid: InvalidSpecError: Unable to generate manifests in .: rpc error: code = Unknown desc = `kustomize build /tmp/_argocd-repo/a9e5bec9-bf1e-4011-9eb3-cfc6ec733aa9` failed exit status 1: Error: invalid Kustomization: json: unknown field "repository"private AWS ECR 에서 이미지를 가져오려면 인증이 필요함
참고
https://docs.aws.amazon.com/ko_kr/AmazonECR/latest/userguide/Registries.html#registry_auth
Amazon ECR 프라이빗 레지스트리 - Amazon ECR
이 페이지에 작업이 필요하다는 점을 알려 주셔서 감사합니다. 실망시켜 드려 죄송합니다. 잠깐 시간을 내어 설명서를 향상시킬 수 있는 방법에 대해 말씀해 주십시오.
docs.aws.amazon.com
https://docs.aws.amazon.com/ko_kr/AmazonECR/latest/userguide/docker-push-ecr-image.html
Docker 이미지 푸시 - Amazon ECR
이 페이지에 작업이 필요하다는 점을 알려 주셔서 감사합니다. 실망시켜 드려 죄송합니다. 잠깐 시간을 내어 설명서를 향상시킬 수 있는 방법에 대해 말씀해 주십시오.
docs.aws.amazon.com
https://docs.aws.amazon.com/ko_kr/AmazonECR/latest/userguide/docker-pull-ecr-image.html
이미지 가져오기 - Amazon ECR
이 페이지에 작업이 필요하다는 점을 알려 주셔서 감사합니다. 실망시켜 드려 죄송합니다. 잠깐 시간을 내어 설명서를 향상시킬 수 있는 방법에 대해 말씀해 주십시오.
docs.aws.amazon.com
https://skryvets.com/blog/2021/03/15/kubernetes-pull-image-from-private-ecr-registry/#prerequisites
Kubernetes - pull an image from private ECR registry. Auto refresh ECR token. | Sergey Kryvets Blog
Although there are a lot of instructions available, I haven't found a straightforward way of deploying a container to Kubernetes cluster that is hosted in a private ECR registry. In this short article, I would like to share a sequence of steps that can be
skryvets.com