kube-bench

목차

kube-bench 는 kubernetes cluster 가 보안적으로 안전하게 구성되어있는지 검사해주는 툴이다.

취약점 스캔 결과를 PASS, FAIL, WARN 3가지로 출력해주고,

Remediations 항목에서 어떻게하면 개선되는지 가이드도 해준다고 한다.

 

이 방법은 kube-bench container 를 cluster 에 배포해서 cluster 전체를 스캔하는 방법이다.

master, worker, etcd, policies 등을 한번에 검사할 수 있다고 한다.

 

클러스터는 이전에 생성한 kind 클러스터를 검사할 예정

 

kubebench.yaml 파일 생성

https://github.com/aquasecurity/kube-bench/blob/main/job.yaml

GitHub - aquasecurity/kube-bench: Checks whether Kubernetes is deployed according to security best practices as defined in the C

---
apiVersion: batch/v1
kind: Job
metadata:
  name: kube-bench
spec:
  template:
    metadata:
      labels:
        app: kube-bench
    spec:
      hostPID: true
      containers:
        - name: kube-bench
          image: aquasec/kube-bench:v0.6.8
          command: ["kube-bench"]
          volumeMounts:
            - name: var-lib-etcd
              mountPath: /var/lib/etcd
              readOnly: true
            - name: var-lib-kubelet
              mountPath: /var/lib/kubelet
              readOnly: true
            - name: var-lib-kube-scheduler
              mountPath: /var/lib/kube-scheduler
              readOnly: true
            - name: var-lib-kube-controller-manager
              mountPath: /var/lib/kube-controller-manager
              readOnly: true
            - name: etc-systemd
              mountPath: /etc/systemd
              readOnly: true
            - name: lib-systemd
              mountPath: /lib/systemd/
              readOnly: true
            - name: srv-kubernetes
              mountPath: /srv/kubernetes/
              readOnly: true
            - name: etc-kubernetes
              mountPath: /etc/kubernetes
              readOnly: true
              # /usr/local/mount-from-host/bin is mounted to access kubectl / kubelet, for auto-detecting the Kubernetes version.
              # You can omit this mount if you specify --version as part of the command.
            - name: usr-bin
              mountPath: /usr/local/mount-from-host/bin
              readOnly: true
            - name: etc-cni-netd
              mountPath: /etc/cni/net.d/
              readOnly: true
            - name: opt-cni-bin
              mountPath: /opt/cni/bin/
              readOnly: true
      restartPolicy: Never
      volumes:
        - name: var-lib-etcd
          hostPath:
            path: "/var/lib/etcd"
        - name: var-lib-kubelet
          hostPath:
            path: "/var/lib/kubelet"
        - name: var-lib-kube-scheduler
          hostPath:
            path: "/var/lib/kube-scheduler"
        - name: var-lib-kube-controller-manager
          hostPath:
            path: "/var/lib/kube-controller-manager"
        - name: etc-systemd
          hostPath:
            path: "/etc/systemd"
        - name: lib-systemd
          hostPath:
            path: "/lib/systemd"
        - name: srv-kubernetes
          hostPath:
            path: "/srv/kubernetes"
        - name: etc-kubernetes
          hostPath:
            path: "/etc/kubernetes"
        - name: usr-bin
          hostPath:
            path: "/usr/bin"
        - name: etc-cni-netd
          hostPath:
            path: "/etc/cni/net.d/"
        - name: opt-cni-bin
          hostPath:
            path: "/opt/cni/bin/"

 

kubebench.yaml 배포

kubectl apply -f kubebench.yml
kubectl get pods

배포 후 스캔이 이루어지고 pods 를 확인했을때 Completed 상태여야함

 

 

결과 확인

kubectl logs kube-bench-q8djv > kubebench-output.log

Summary master 에서 각 상태의 합계를 확인할 수 있다

 

 

상태 개선 해보기

간단한 테스트를 위해 만든 클러스터이기 때문에 하나만 고쳐보자

 

트러블슈팅

1.2.19 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
on the control plane node and set the --audit-log-path parameter to a suitable path and
file where you would like audit logs to be written, for example,
--audit-log-path=/var/log/apiserver/audit.log

1.2.20 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
on the control plane node and set the --audit-log-maxage parameter to 30
or as an appropriate number of days, for example,
--audit-log-maxage=30

1.2.21 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
on the control plane node and set the --audit-log-maxbackup parameter to 10 or to an appropriate
value. For example,
--audit-log-maxbackup=10

위 3개 에러는 비슷한 오류로 보여서 고쳐보려고 함

 

해당 디렉토리로 이동해서 수정해보려고 하니까 컨테이너 안에 vi가 안깔리고 apt 업데이트도 안되네

 

백업파일을 만들고 내용을 모두 긁은 뒤 밖에서 작업하고 docker cp 할 예정

 

코드 수정

docker cp .\kube-apiserver.yaml kind-control-plane:/etc/kubernetes/manifests/

 

확인

 

새로 copy 되서 신났나 색깔이 혼자 왜저럼

 

Scan again

kube-bench 삭제 후 재배포

kubectl delete -f kubebench.yaml
kubectl apply -f kubebench.yaml
kubectl logs kube-bench-hrkld > kubebench-output-2.log

kube-bench pod 이름 확인해야 한다

빈파일 나와서 당황했었다

 

fail 수가 줄어듬 확인

 

참고

https://ikcoo.tistory.com/388

98. kube-bench

https://madhuakula.com/content/attacking-and-auditing-docker-containers-and-kubernetes-clusters/kube-bench/index.html

Running kube-bench - Attacking and Auditing Docker Containers and Kubernetes Clusters

https://kubernetes.io/ko/docs/tasks/debug/debug-application/get-shell-running-container/

동작중인 컨테이너의 셸에 접근하기