Ingress-Nginx 服务暴露基础学习与实践

WeiyiGeek.WWW-Authenticate

示例6.Ingress Redirect 域名重定向功能

描述: 我们可以利用Ingress中的域名重定向(Redirect)注解来实现访问重定向。

# 简单格式:
annotations:
  nginx.ingress.kubernetes.io/permanent-redirect: 'https://blog.weiyigeek.top'

Ingress 规则配置清单示例:

tee Ingress-Redirect.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress 
metadata:
  name: ingress-redirect
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: 'https://blog.weiyigeek.top'
spec:
  rules: 
    - host: redirect.weiyigeek.top
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: front-svc
              port:
                number: 80
  defaultBackend:
    service:
      name: front-svc
      port:
        number: 80   
EOF

部署资源清单以及查看

# 创建 Ingress 资源对象
kubectl create -f Ingress-Redirect.yaml

# 查看Ingress控制器下面创建的应用信息以及详细信息
kubectl get ing -n demo ingress-redirect && kubectl describe ing -n demo ingress-redirect
  # NAME               CLASS    HOSTS                    ADDRESS   PORTS   AGE
  # ingress-redirect   <none>   redirect.weiyigeek.top             80      23s
  ....

# 验证重定向观察设置的虚拟主机的域名与未设置虚拟主机域名之间的不同
curl -v http://10.10.107.221 -H 'host: redirect.weiyigeek.top' -k  # 重定向到我们指定的域名
curl -v http://10.10.107.221 -H 'host: r.weiyigeek.top' -k         # 不存在的虚拟主机名称则访问默认 defaultBackend 服务。

WeiyiGeek.redirect

示例7.Ingress 黑白名单访问限制

描述: 某些内部系统可能只允许某些IP地址访问时，可利用 Ingress-nginx 的黑白名单，其实现的方式有如下两个。

Tips: 建议黑名单可以使用ConfigMap去配置，白名单建议使用Annotations去配置。

# 方式1.annotation方式
tee ingress-whilte-annotation.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-whilte
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/service-weight: '1'
    nginx.ingress.kubernetes.io/whitelist-source-range: 10.20.172.103,10.10.107.220
spec:
  rules: 
    - host: whilte.weiyigeek.top
      http:
        paths:
        - path: /
          pathType: ImplementationSpecific
          backend:
            service:
              name: front-svc
              port:
                number: 80
EOF

# 方式2.ConfigMap方式
tee ingress-whilte-ConfigMap.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: ingress-whilte
  namespace: demo
data:
  whitelist-source-range: 10.10.107.0/24
EOF


# 创建 Ingress 资源对象
$ kubectl create -f ingress-whilte-annotation.yaml

$ kubectl get ingress -n demo ingress-whilte
  # NAME             CLASS    HOSTS                  ADDRESS   PORTS   AGE
  # ingress-whilte   <none>   whilte.weiyigeek.top             80      75s

$ kubectl describe ingress -n demo ingress-whilte
  # Rules:
  #   Host                  Path  Backends
  #   ----                  ----  --------
  #   whilte.weiyigeek.top
  #                         /   front-svc:80 (192.168.109.91:80,192.168.109.92:80)
  # Annotations:            nginx.ingress.kubernetes.io/service-weight: 1
  #                         nginx.ingress.kubernetes.io/whitelist-source-range: 10.20.172.103,10.10.107.220

# 主机: 10.10.107.220 (拥有访问权限时正常返回数据)
10.10.107.220 $ curl http://10.10.107.221/ -H 'host: whilte.weiyigeek.top' -k -i
  # HTTP/1.1 200 OK
  # Date: Fri, 16 Jul 2021 07:47:00 GMT
  # Content-Type: text/html
  # Content-Length: 37
  # Connection: keep-alive
  # Last-Modified: Thu, 15 Jul 2021 12:24:55 GMT
  # ETag: "60f02917-25"
  # Accept-Ranges: bytes
# hostname-web-front-0-WeiyiGeek-Nginx

# 主机: 10.10.107.221 (无访问权限则显示403)
10.10.107.221 $ curl http://10.10.107.221/ -H 'host: whilte.weiyigeek.top' -k -i
  # HTTP/1.1 403 Forbidden
  # Date: Fri, 16 Jul 2021 07:47:54 GMT
  # Content-Type: text/html
  # Content-Length: 146
  # Connection: keep-alive
# <center><h1>403 Forbidden</h1></center>

# - Annotation
tee ingress-black-annotation.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-black
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
      deny 10.10.107.220;
      deny 10.20.172.103;
      allow all;
spec:
  rules: 
    - host: black.weiyigeek.top
      http:
        paths:
        - path: /
          pathType: ImplementationSpecific
          backend:
            service:
              name: backend-svc
              port:
                number: 8080
status:
  loadBalancer: {}
EOF

# - ConfigMap
tee ingress-black-ConfigMap.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: ingress-black
  namespace: demo
data:
  whitelist-source-range: 10.20.172.0/24
  block-cidrs: 10.10.107.220
EOF

# - 创建 Ingress 对象
kubectl apply -f ingress-black-annotation.yaml

# - 查看与详细 Ingress 对象
kubectl get ing -n demo ingress-black && kubectl describe ing -n demo ingress-black
  # NAME            CLASS    HOSTS                 ADDRESS   PORTS   AGE
  # ingress-black   <none>   black.weiyigeek.top             80      20s

  # Rules:
  #   Host                 Path  Backends
  #   ----                 ----  --------
  #   black.weiyigeek.top  /     backend-svc:8080 (192.168.109.90:8080,192.168.109.93:8080)
  # Annotations:           nginx.ingress.kubernetes.io/server-snippet:
  #                          deny 10.10.107.220;
  #                          deny 10.20.172.103;
  #                          allow all;

# 验证 Ingress 代理的后端应用
# (1) 10.10.107.220 (黑名单无法访问)
curl http://10.10.107.221/ -H 'host: black.weiyigeek.top' -k -i
  # HTTP/1.1 403 Forbidden
  # ......................
  # <center><h1>403 Forbidden</h1></center>

# (2) 10.10.107.221 (未在黑名单之内则可以访问)  
curl http://10.10.107.221/ -H 'host: black.weiyigeek.top' -k -i
  # HTTP/1.1 200
  # Date: Fri, 16 Jul 2021 13:23:36 GMT
  # Content-Type: text/html
  # Content-Length: 40
  # Connection: keep-alive
  # Accept-Ranges: bytes
  # ETag: W/"40-1626351895000"
  # Last-Modified: Thu, 15 Jul 2021 12:24:55 GMT

  # hostname-web-front-0-WeiyiGeek-Nginx

示例8.Ingress Header 请求头匹配

描述: Ingress-nginx 的 匹配请求头，可以采用nginx.ingress.kubernetes.io/server-snippet注解。

# - 资源清单
tee ingress-header-request.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-header
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
      set $agentflag 0;
      if ($http_user_agent ~* "(firefox)" ){
        set $agentflag 1;
      }
      if ($agentflag = 1) {
        return 301 https://www.weiyigeek.top;
      }
spec:
  rules: 
    - host: header.weiyigeek.top
      http:
        paths:
        - path: /
          pathType: ImplementationSpecific
          backend:
            service:
              name: backend-svc
              port:
                number: 8080
status:
  loadBalancer: {}
EOF

# - 创建查看 Ingress 资源对象
kubectl create -f ingress-header-request.yaml
kubectl get ing -n demo ingress-header && kubectl describe ing -n demo ingress-header
  # Rules:
  #   Host                  Path  Backends
  #   ----                  ----  --------
  #   header.weiyigeek.top  /     backend-svc:8080 (192.168.109.90:8080,192.168.109.93:8080)

# - 验证 Ingress 资源对象
# (1) 指定 User-Agent 请求头 包含 Firefox
$ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' -k -i
  # HTTP/1.1 301 Moved Permanently
  # Date: Fri, 16 Jul 2021 14:00:04 GMT
  # Content-Type: text/html
  # Content-Length: 162
  # Connection: keep-alive
  # Location: https://www.weiyigeek.top

  # <html>
  # <head><title>301 Moved Permanently</title></head>
  # <body>
  # <center><h1>301 Moved Permanently</h1></center>
  # <hr><center>nginx</center>
  # </body>
  # </html>

# (2) 指定 User-Agent 请求头 不包含 Firefox 
$ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0' -k -i
  # HTTP/1.1 200
  # Date: Fri, 16 Jul 2021 14:03:29 GMT
  # Content-Type: text/html
  # Content-Length: 40
  # Connection: keep-alive
  # Accept-Ranges: bytes
  # ETag: W/"40-1626351892000"
  # Last-Modified: Thu, 15 Jul 2021 12:24:52 GMT

  # hostname-web-backend-0-WeiyiGeek-Tomcat

示例9.Ingress 灰度金丝雀发布

描述: 在某些情况下，您可能希望通过向与生产服务不同的服务发送少量请求来“金丝雀”一组新的更改。金丝雀注解使 Ingress 规范能够根据应用的规则充当路由请求的替代服务,在nginx.ingress.kubernetes.io/canary: "true"设置后可以启用以下用于配置金丝雀的注释：。

Nginx Annotations 支持以下 4 种 Canary 规则:

* nginx.ingress.kubernetes.io/canary-weight：基于服务权重的流量切分，适用于蓝绿部署，权重范围 0 - 100 按百分比将请求路由到 Canary Ingress 中指定的服务。权重为 0 意味着该金丝雀规则不会向 Canary 入口的服务发送任何请求。权重为 100 意味着所有请求都将被发送到 Canary 入口。
* nginx.ingress.kubernetes.io/canary-by-header：基于 Request Header 的流量切分，适用于灰度发布以及 A/B 测试。当 Request Header 设置为 always时，请求将会被一直发送到 Canary 版本；当 Request Header 设置为 never时，请求不会被发送到 Canary 入口；对于任何其他 Header 值，将忽略 Header，并通过优先级将请求与其他金丝雀规则进行优先级的比较。
* nginx.ingress.kubernetes.io/canary-by-header-value：要匹配的 Request Header 的值，用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务。当 Request Header 设置为此值时，它将被路由到 Canary 入口。该规则允许用户自定义 Request Header 的值，必须与上一个 annotation (即：canary-by-header）一起使用。
* nginx.ingress.kubernetes.io/canary-by-header-pattern：这与canary-by-header-valuePCRE 正则表达式匹配的工作方式相同。请注意，canary-by-header-value设置此注释时将被忽略。当给定的 Regex 在请求处理过程中导致错误时，该请求将被视为不匹配。
* nginx.ingress.kubernetes.io/canary-by-cookie：基于 Cookie 的流量切分，适用于灰度发布与 A/B 测试。用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务的cookie。当 cookie 值设置为 always时，它将被路由到 Canary 入口；当 cookie 值设置为 never时，请求不会被发送到 Canary 入口；对于任何其他值，将忽略 cookie 并将请求与其他金丝雀规则进行优先级的比较。 定义两个版本的代码。

Tips : 金丝雀规则是按优先顺序计算的。优先级如下：canary by header->canary by cookie->canary weight

Tips : 注意当您将入口标记为 Canary 时，除nginx.ingress.kubernetes.io/load-balanceand之外的所有其他非 Canary 注释都将被忽略（从相应的主入口继承）nginx.ingress.kubernetes.io/upstream-hash-by。`

资源配置清单:

# - 默认访问前端
tee ingress-canary-v1.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-canary-v1
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: canary
    nginx.ingress.kubernetes.io/canary-by-header-value: v1
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  rules: 
    - host: canary.weiyigeek.top
      http:
        paths:
        - path: /
          pathType: ImplementationSpecific
          backend:
            service:
              name: front-svc
              port:
                number: 80
EOF

# - 利用金丝雀访问后端
tee ingress-canary-v2.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-canary-v2
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: canary
    nginx.ingress.kubernetes.io/canary-by-header-value: v2
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  rules: 
    - host: canary.weiyigeek.top
      http:
        paths:
        - path: /
          pathType: ImplementationSpecific
          backend:
            service:
              name: backend-svc
              port:
                number: 8080
EOF

~/k8s/ingress/demo1# kubectl create -f ingress-canary-v1.yaml
  # ingress.networking.k8s.io/ingress-canary-v1 created
~/k8s/ingress/demo1# kubectl create -f ingress-canary-v2.yaml
  # ingress.networking.k8s.io/ingress-canary-v2 created

~/k8s/ingress/demo1# kubectl get ing -n demo ingress-canary-v1 ingress-canary-v2
  # NAME                CLASS    HOSTS                  ADDRESS   PORTS   AGE
  # ingress-canary-v1   <none>   canary.weiyigeek.top             80      3m12s
  # ingress-canary-v2   <none>   canary.weiyigeek.top             80      3m4s

# 验证金丝雀 Ingress 对象应用
for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -i;done
for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -H 'weiyigeek: v2' -i;done

示例10.Ingress 相关配置补充

描述: 可以修改 deploy.yaml 中指定 nginx-ingress-controller 运行的参数，在此处添加- --default-backend-service=名称空间/SVC名称参数

$ grep -A 3 -n "/nginx-ingress" deploy.yaml
332:            - /nginx-ingress-controller
333-            - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
334-            - --election-id=ingress-controller-leader
335-            - --ingress-class=nginx
                - --default-backend-service=demo/error-svc

* nginx.ingress.kubernetes.io/limit-connections：单个IP地址允许的并发连接数。超出此限制时，将返回503错误。
* nginx.ingress.kubernetes.io/limit-rps：每秒从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数，默认乘数为5。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503。
* nginx.ingress.kubernetes.io/limit-rpm：每分钟从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数，默认乘数为5。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503。
* nginx.ingress.kubernetes.io/limit-burst-multiplier：突发大小限制速率的倍数。默认的脉冲串乘数为5，此注释将覆盖默认的乘数。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503。
* nginx.ingress.kubernetes.io/limit-rate-after：最初的千字节数，在此之后，对给定连接的响应的进一步传输将受到速率的限制。必须在启用代理缓冲的情况下使用此功能。
* nginx.ingress.kubernetes.io/limit-rate：每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。
* nginx.ingress.kubernetes.io/limit-whitelist：客户端IP源范围要从速率限制中排除。该值是逗号分隔的CIDR列表。

示例说明:

# - 资源清单配置:
tee ingress-limit.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-limit
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/limit-rate: "100K"               # 限制客户端每秒传输的字节数
    nginx.ingress.kubernetes.io/limit-whitelist: "10.10.107.220" # 白名单中的IP不限速
    nginx.ingress.kubernetes.io/limit-rps: 1                     # 单个IP每秒的连接数
    nginx.ingress.kubernetes.io/limit-rpm: 30                    # 单个IP每分钟的连接数
spec:
  rules: 
    - host: limit.weiyigeek.top
      http:
        paths:
        - path: /
          pathType: ImplementationSpecific
          backend:
            service:
              name: backend-svc
              port:
                number: 8080
status:
  loadBalancer: {}
EOF

Tips : 通过 Ingress ConfigMaps 查看详细的配置参数。

Ingress 规则存储配置清单：

# --configmap=
tee nginx-config.yaml <<'EOF'
kind: ConfigMap
apiVersion: v1
metadata:
  name: ingress-nginx-controller  # 注意此ConfigMap名称与/nginx-ingress-controller 中 `--configmap=$(POD_NAMESPACE)/ingress-nginx-controller` 对应
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
data:
  proxy-connect-timeout: "15"   #这里有个坑，到家一定要注意是'-'不是'_'
EOF

# --tcp-services-configmap=
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

# --udp-services-configmap
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

nginx-ingress-controller 使用 Configmap 作为配置存在控制器:

# 可以在 nginx-ingress-controller 部署资源清单中看见`--configmap=`参数
vim +328 deploy.yaml
 containers:
        - name: controller
          image: pollyduan/ingress-nginx-controller:v0.47.0
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller   # 使用 confimap 配置文件
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key

查看ingress-nginx名称空间中的ConfigMap配置

kubectl apply -f nginx-config.yaml
  # configmap/ingress-nginx-controller configured

kubectl get cm -n ingress-nginx
  # NAME                              DATA   AGE
  # ingress-controller-leader-nginx   0      2d8h
  # ingress-nginx-controller          0      2d8h
  # kube-root-ca.crt                  1      2d8h

kubectl describe cm -n ingress-nginx ingress-nginx-controller
  # Name:         ingress-nginx-controller
  # Namespace:    ingress-nginx
  # Labels:       app.kubernetes.io/component=controller
  #               app.kubernetes.io/instance=ingress-nginx
  #               app.kubernetes.io/managed-by=Helm
  #               app.kubernetes.io/name=ingress-nginx
  #               app.kubernetes.io/version=0.47.0
  #               helm.sh/chart=ingress-nginx-3.33.0
  # Annotations:  <none>

  # Data
  # ====
  # proxy-connect-timeout:
  # ----
  # 15
  # Events:
  #   Type    Reason  Age   From                      Message
  #   ----    ------  ----  ----                      -------
  #   Normal  UPDATE  25s   nginx-ingress-controller  ConfigMap ingress-nginx/ingress-nginx-controller

验证是否在ingress-nginx-controller中配置生效:

kubectl get pod -n ingress-nginx -l app.kubernetes.io/component=controller
  # NAME                                        READY   STATUS    RESTARTS   AGE
  # ingress-nginx-controller-5df4f79b95-6p8rr   1/1     Running   0          2d8h

# 在 ingress-nginx-controller-5df4f79b95-6p8r 的 Pod 中 查看 nginx.conf 文件中 proxy_connect_timeout 参数是否为 15s
root@k8s-master-1:~/k8s/ingress/deployment# kubectl -n ingress-nginx exec -it ingress-nginx-controller-5df4f79b95-6p8rr bash
bash-5.1$ grep "proxy_connect_timeout" nginx.conf
          proxy_connect_timeout                   15s;   #

企业实战

1.利用Ingress访问后端的Nginx Web和Tomcat后端综合实践。

实验目标:

详细流程:

WeiyiGeek.SSL证书申请

tee sts-svc-nginx-tomcat.yaml<<'EOF'
apiVersion: v1
kind: Namespace
metadata:
  name: demo
---
# Service 服务创建(服务是软件服务的命名抽象)。
# kubectl api-resources | grep "service"
apiVersion: v1
kind: Service
metadata:
  name: front-svc
  namespace: demo
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    app: nginx-web
  ports:
    - port: 80 
      targetPort: 80 
      protocol: TCP 
---
apiVersion: v1
kind: Service
metadata:
  name: backend-svc
  namespace: demo
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    app: tomcat-web
  ports:
    - port: 8080
      targetPort: 8080 
      protocol: TCP
  sessionAffinity: None
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web-front
  namespace: demo
spec:
  replicas: 2
  serviceName: front-svc
  selector:
    matchLabels:
      app: nginx-web
  template:
    metadata:
      labels:
        app: nginx-web
    spec:
      initContainers:
      - name: init-html
        image: busybox:latest
        imagePullPolicy: IfNotPresent
        command: ['sh', '-c', "echo hostname-${HOSTNAME}-${MSG} > /usr/share/nginx/html/index.html"]
        env:
        - name: MSG
          value: "WeiyiGeek-Nginx"
        volumeMounts:
        - name: web
          mountPath: "/usr/share/nginx/html"
        securityContext:
          privileged: true
      containers:
      - name: nginx   
        image: nginx:latest
        imagePullPolicy: IfNotPresent
        env:
        - name: TZ
          value: "Asia/Shanghai"
        ports:
        - name: nginx
          containerPort: 80
        volumeMounts:
        - name: web
          mountPath: "/usr/share/nginx/html"
      volumes:
      - name: web 
        emptyDir: {}
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web-backend
  namespace: demo
spec:
  replicas: 2
  serviceName: backend-svc
  selector:
    matchLabels:
      app: tomcat-web
  template:
    metadata:
      labels:
        app: tomcat-web
    spec:
      initContainers:
      - name: init-html
        image: busybox:latest
        imagePullPolicy: IfNotPresent
        command: ['sh', '-c', "echo hostname-${HOSTNAME}-${MSG} > /usr/local/tomcat/webapps/ROOT/index.html"]
        env:
        - name: MSG
          value: WeiyiGeek-Tomcat
        volumeMounts:
        - name: web
          mountPath: "/usr/local/tomcat/webapps/ROOT/"
        securityContext:
          privileged: true
      containers:
      - name: nginx   
        image: "tomcat:8.5.69-jdk8-openjdk-slim-buster"
        imagePullPolicy: IfNotPresent
        env:
        - name: TZ
          value: "Asia/Shanghai"
        ports:
        - name: nginx
          containerPort: 8080
        volumeMounts:
        - name: web
          mountPath: "/usr/local/tomcat/webapps/ROOT/"
      volumes:
      - name: web 
        emptyDir: {}
EOF

# 利用资源清单进行创建nginx、Tomcat应用
kubectl create -f sts-svc-nginx-tomcat.yaml
  # namespace/demo created
  # service/front-svc created
  # service/backend-svc created
  # statefulset.apps/web-front created
  # statefulset.apps/web-backend created

# 查看 Service 控制器对象
$ kubectl get svc -n demo -o wide
  # NAME          TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE   SELECTOR
  # backend-svc   ClusterIP   None         <none>        8080/TCP   12m   name=tomcat-web
  # front-svc     ClusterIP   None         <none>        80/TCP     12m   name=nginx-web

# 查看 StatefulSet 控制器对象
$ kubectl get sts -n demo -o wide
  # NAME          READY   AGE   CONTAINERS   IMAGES
  # web-backend   2/2     13m   nginx        tomcat:8.5.69-jdk8-openjdk-slim-buster
  # web-front     2/2     13m   nginx        nginx:latest

# 查看 nginx 与 tomcat 相关 Pod 信息
$ kubectl get pod -n demo -o wide --show-labels
  # NAME            READY   STATUS    RESTARTS   AGE   IP               NODE        LABELS
  # web-backend-0   1/1     Running   0          13m   192.168.109.90   k8s-node-1  app=tomcat-web,controller-reVision-hash=web-backend-776459fc9d,statefulset.kubernetes.io/pod-name=web-backend-0
  # web-backend-1   1/1     Running   0          13m   192.168.109.93   k8s-node-1  app=tomcat-web,controller-revision-hash=web-backend-776459fc9d,statefulset.kubernetes.io/pod-name=web-backend-1
  # web-front-0     1/1     Running   0          13m   192.168.109.91   k8s-node-1  app=nginx-web,controller-revision-hash=web-front-89f7fcc75,statefulset.kubernetes.io/pod-name=web-front-0
  # web-front-1     1/1     Running   0          13m   192.168.109.92   k8s-node-1  app=nginx-web,controller-revision-hash=web-front-89f7fcc75,statefulset.kubernetes.io/pod-name=web-front-1

# 查看 SVC 服务发现利用CoreDNS进行域名解析
$ nslookup front-svc.demo.svc.cluster.local. 10.96.0.10
  # Server:         10.96.0.10
  # Address:        10.96.0.10#53
  # Name:   front-svc.demo.svc.cluster.local
  # Address: 192.168.109.91
  # Name:   front-svc.demo.svc.cluster.local
  # Address: 192.168.109.92
$ nslookup backend-svc.demo.svc.cluster.local. 10.96.0.10
  # ...
  # Name:   backend-svc.demo.svc.cluster.local
  # Address: 192.168.109.90
  # Name:   backend-svc.demo.svc.cluster.local

# 请求查看 Nginx 与 Tomcat 应用回显。
~/k8s/ingress/demo# curl front-svc.demo.svc.cluster.local
  # hostname-web-front-0-WeiyiGeek-Nginx
~/k8s/ingress/demo# curl front-svc.demo.svc.cluster.local
  # hostname-web-front-1-WeiyiGeek-Nginx
~/k8s/ingress/demo# curl backend-svc.demo.svc.cluster.local:8080
  # hostname-web-backend-1-WeiyiGeek-Tomcat
~/k8s/ingress/demo# curl backend-svc.demo.svc.cluster.local:8080
  # hostname-web-backend-0-WeiyiGeek-Tomcat

# Tips:  kubectl api-resources  | grep "secret" 可查看apiversions以及namespace是否支持
# 假设你已经将证书上传到k8s集群目录中并解压
$ ls
demo.weiyigeek.top.zip  test.weiyigeek.top.zip

$ unzip demo.weiyigeek.top.zip -d demo.weiyigeek.top
$ unzip test.weiyigeek.top.zip -d test.weiyigeek.top

# 创建 tls 类型的 secret
$ cd /tmp/demo.weiyigeek.top/Nginx
kubectl -n demo create secret tls demoweiyigeektop --key 2_demo.weiyigeek.top.key --cert 1_demo.weiyigeek.top_bundle.crt
  # secret/demoweiyigeektop created

$ cd /tmp/test.weiyigeek.top/Nginx
kubectl -n demo create secret tls testweiyigeektop --key 2_test.weiyigeek.top.key --cert 1_test.weiyigeek.top_bundle.crt
  # secret/testweiyigeektop created

tee ingress-demo.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-demo
  namespace: demo
  annotations:
    # 指定 Ingress Controller 的类型
    kubernetes.io/ingress.class: "nginx"
    # 指定我们的 rules 的 path 可以使用正则表达式
    nginx.ingress.kubernetes.io/use-regex: "true"
    # 连接超时时间，默认为 5s
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    # 后端服务器回转数据超时时间，默认为 60s
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    # 后端服务器响应超时时间，默认为 60s
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    # 客户端上传文件，最大大小，默认为 20m
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
    # URL 重写
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  # 默认后端服务指定
  defaultBackend:
    service:
      name: front-svc
      port:
        number: 80
  tls:
  - hosts:
    - test.weiyigeek.top
    secretName: testweiyigeektop
  - hosts:
    - demo.weiyigeek.top
    secretName: demoweiyigeektop
  rules:
  - host: test.weiyigeek.top
    http:
      paths:
      - pathType: Prefix
        path: /tomcat
        backend:
          service:
            name: backend-svc
            port:
              number: 8080
      - pathType: Prefix
        path: /nginx
        backend:
          service:
            name: front-svc
            port:
              number: 80
  - host: demo.weiyigeek.top
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: front-svc
            port:
              number: 80
EOF

$ kubectl create -f ingress-demo.yaml
  # ingress.networking.k8s.io/ingress-demo created  

# 查看 ingress-nginx 的 SVC 
kubectl get svc -n ingress-nginx
  # NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
  # ingress-nginx-controller             LoadBalancer   10.104.171.212   <pending>     80:31122/TCP,443:32155/TCP   24h
  # ingress-nginx-controller-admission   ClusterIP      10.99.168.221    <none>        443/TCP                      24h

# 查看 ingress-demo 对象相关信息
$ kubectl get ing -n demo ingress-demo
  # NAME           CLASS    HOSTS                                   ADDRESS   PORTS     AGE
  # ingress-demo   <none>   test.weiyigeek.top,demo.weiyigeek.top             80, 443   24h

$ kubectl describe ing -n demo ingress-demo
  # Name:             ingress-demo
  # Namespace:        demo
  # Address:
  # Default backend:  front-svc:80 (192.168.109.91:80,192.168.109.92:80)
  # TLS:
  #   testweiyigeektop terminates test.weiyigeek.top
  #   demoweiyigeektop terminates demo.weiyigeek.top
  # Rules:
  #   Host                Path  Backends
  #   ----                ----  --------
  #   test.weiyigeek.top /tomcat   backend-svc:8080 (192.168.109.90:8080,192.168.109.93:8080)
  #   demo.weiyigeek.top /         front-svc:80 (192.168.109.91:80,192.168.109.92:80)
  # Annotations:          kubernetes.io/ingress.class: nginx
  #                       nginx.ingress.kubernetes.io/proxy-body-size: 10m
  #                       nginx.ingress.kubernetes.io/proxy-connect-timeout: 600
  #                       nginx.ingress.kubernetes.io/proxy-read-timeout: 600
  #                       nginx.ingress.kubernetes.io/proxy-send-timeout: 600
  #                       nginx.ingress.kubernetes.io/rewrite-target: /
  #                       nginx.ingress.kubernetes.io/use-regex: true
  # Events:               <none>

$ curl 'https://10.10.107.221' -H 'host: demo.weiyigeek.top' -i -k
  # HTTP/2 200
  # date: Sat, 17 Jul 2021 04:39:37 GMT
  # content-type: text/html
  # content-length: 37
  # last-modified: Thu, 15 Jul 2021 12:24:55 GMT
  # etag: "60f02917-25"
  # accept-ranges: bytes
  # strict-transport-security: max-age=15724800; includeSubDomains

  # hostname-web-front-1-WeiyiGeek-Nginx

$ curl 'https://10.10.107.221/tomcat' -H 'host: test.weiyigeek.top' -i -k
  # HTTP/2 200
  # date: Sat, 17 Jul 2021 04:39:50 GMT
  # content-type: text/html
  # content-length: 40
  # accept-ranges: bytes
  # etag: W/"40-1626351892000"
  # last-modified: Thu, 15 Jul 2021 12:24:52 GMT
  # strict-transport-security: max-age=15724800; includeSubDomains

  # hostname-web-backend-0-WeiyiGeek-Tomcat

WeiyiGeek.Ingress 应用