현상
2022.07.11 - [k8s] Error: rendered manifests contain a resource that already exists. Unable to continue with install: existing resource conflict 에러 글에서, 설치했던 helm chart로 설치한 release를 완전히 삭제하고 재설치하려고 했더니 아래와 같은 에러 메시지가 발생하면서 설치에 실패했다.
helm install prometheus-operator prometheus-community/kube-prometheus-stack --dry-run
Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "Alertmanager" in
version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "Prometheus" in version
"monitoring.coreos.com/v1", unable to recognize "": no matches for kind "PrometheusRule" in version
"monitoring.coreos.com/v1", unable to recognize "": no matches for kind "ServiceMonitor" in version
"monitoring.coreos.com/v1"]
원인
리소스 생성에 필요한 CRD가 설치되지 않아 문제가 발생했다.
내 경우에는 chart로 설치한 모든 리소스를 삭제하면서 CRD를 삭제했기 때문에 발생한 현상이다. 참고로 Helm은 차트 업그레이드 시 새 CRD를 자동으로 업그레이드하거나 설치하지 않으므로 업데이트하기 전에 CRD를 수동으로 설치해야 한다.
해결
설치에 필요한 CRD를 수동으로 설치하거나 업그레이드하여 문제를 해결할 수 있다.
내 경우, kube-prometheus-stack chart를 설치하고자 했으므로 Github에서 CRD 설치 관련 부분을 확인했다.
현재 기준 최신 버전은 36.x로 설치 명령은 아래와 같다.
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagerconfigs.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_probes.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_prometheusrules.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.57.0/example/prometheus-operator-crd/monitoring.coreos.com_thanosrulers.yaml
그리고 설치하고자 했던 helm chart는 kube-prometheus-stack-11.1.3인데, 18.x 버전을 설치해야 정상적으로 chart를 설치할 수 있었다.
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagerconfigs.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_probes.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_prometheusrules.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.49.0/example/prometheus-operator-crd/monitoring.coreos.com_thanosrulers.yaml
참고 문서
https://github.com/prometheus-community/helm-charts/issues/590