minikube addons enable ingress 启动错误

开启 minkube ingress 时错误

minikube addons enable ingress --alsologtostderr

Verifying ingress addon...
  Verifying ingress addon...
I1002 16:59:15.398329  827754 kapi.go:75] Waiting for pod with label "app.kubernetes.io/name=ingress-nginx" in ns "kube-system" ...
I1002 16:59:15.416235  827754 kapi.go:86] Found 2 Pods for label selector app.kubernetes.io/name=ingress-nginx
I1002 16:59:15.416270  827754 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I1002 16:59:15.920492  827754 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I1002 16:59:16.419519  827754 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I1002 16:59:16.919852  827754 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I1002 16:59:17.419814  827754 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

查看对应POD启动日志
k get pods -A

kube-system            ingress-nginx-controller-56b4dfdb44-xfp2c   0/1     ImagePullBackOff   0          96s

k logs ingress-nginx-controller-56b4dfdb44-xfp2c -n kube-system

Error from server (BadRequest): container "controller" in pod "ingress-nginx-controller-56b4dfdb44-xfp2c" is waiting to start: trying and failing to pull image

查看POD详细信息
k describe pod/ingress-nginx-controller-56b4dfdb44-xfp2c -n kube-system

Events:
  Type     Reason            Age                   From               Message
  ----     ------            ----                  ----               -------
  Warning  FailedScheduling  <unknown>                                0/1 nodes are available: 1 node(s) didn't have free ports for the requested pod ports.
  Warning  FailedScheduling  <unknown>                                0/1 nodes are available: 1 node(s) didn't have free ports for the requested pod ports.
  Normal   Scheduled         <unknown>                                Successfully assigned kube-system/ingress-nginx-controller-56b4dfdb44-xfp2c to minikube
  Normal   Pulling           2m1s (x4 over 3m30s)  kubelet, minikube  Pulling image "registry.cn-hangzhou.aliyuncs.com/google_containers/controller:v0.40.2"
  Warning  Failed            2m1s (x4 over 3m29s)  kubelet, minikube  Failed to pull image "registry.cn-hangzhou.aliyuncs.com/google_containers/controller:v0.40.2": rpc error: code = Unknown desc = Error response from daemon: pull access denied for registry.cn-hangzhou.aliyuncs.com/google_containers/controller, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
  Warning  Failed            2m1s (x4 over 3m29s)  kubelet, minikube  Error: ErrImagePull
  Warning  Failed            105s (x6 over 3m29s)  kubelet, minikube  Error: ImagePullBackOff
  Normal   BackOff           91s (x7 over 3m29s)   kubelet, minikube  Back-off pulling image "registry.cn-hangzhou.aliyuncs.com/google_containers/controller:v0.40.2"

重点错误信息：

Back-off pulling image "registry.cn-hangzhou.aliyuncs.com/google_containers/controller:v0.40.2"

一般我们都是用的阿里云的容器镜像源，但是controller:v0.40.2版本的镜像不存在。

通过搜索有其他镜像可以用

registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.25.1
registry.aliyuncs.com/google_containers/nginx-ingress-controller:0.26.1

要想修改POD镜像，需要从 deployment 对象 ingress-nginx-controller 开始。

k get  deployment --all-namespaces

NAMESPACE              NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
kube-system            ingress-nginx-controller    0/1     1            0           20h

图1
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-GYsYdgbu-1633313713300)(https://raw.githubusercontent.com/Plen-wang/blogsImage/master/githubimages/minikubeingress/1.png)]

图2
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-3w3Zbhgg-1633313713303)(https://raw.githubusercontent.com/Plen-wang/blogsImage/master/githubimages/minikubeingress/2.png)]

对上图两处image参数进行修改。使用上面的两个镜像地址选一个替换。

删掉历史ReplicaSet 对象。

k get rs --all-namespaces

POD启动权限相关错误

k get pods -A

NAMESPACE              NAME                                        READY   STATUS             RESTARTS   AGE
kube-system            ingress-nginx-controller-79b88546bb-vpxlr   0/1     CrashLoopBackOff   26         112m
          0          8h

k logs ingress-nginx-controller-79b88546bb-vpxlr -n kube-system
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       0.26.1
  Build:         git-2de5a893a
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: openresty/1.15.8.2

-------------------------------------------------------------------------------

W1002 11:08:49.444500       7 flags.go:243] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W1002 11:08:49.444562       7 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I1002 11:08:49.444686       7 main.go:182] Creating API client for https://10.96.0.1:443
I1002 11:08:49.459177       7 main.go:226] Running in Kubernetes cluster version v1.20 (v1.20.0) - git (clean) commit af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38 - platform linux/amd64
F1002 11:08:49.608094       7 ssl.go:389] unexpected error storing fake SSL Cert: could not create PEM certificate file /etc/ingress-controller/ssl/default-fake-certificate.pem: open /etc/ingress-controller/ssl/default-fake-certificate.pem: permission denied

编辑 deployment 修改runAsUser=33

图3
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-KaTC2SyI-1633313713305)(https://raw.githubusercontent.com/Plen-wang/blogsImage/master/githubimages/minikubeingress/3.png)]

图4

创建 ingress 错误

k create -f kubia-ingress.yaml

error when creating "kubia-ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission.kube-system.svc:443/networking/v1beta1/ingresses?timeout=10s": x509: certificate signed by unknown authority

这个比较好处理，网上能找到办法。直接删除准入验证。
详细参考：https://kubernetes.io/zh/docs/reference/access-authn-authz/extensible-admission-controllers/

kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission