k8s集群模式部署nacos v3.0.1

写在前面：

本指南是用于集群模式部署Nacos v3.0.1，按照官网的部署指南，大概率会出现容器无法启动的问题（截止20250624），故参考官方指南踩坑总结以下指南。

本环境是在测试环境中部署，其中出入取舍请自斟酌。

本指南为了部署步骤简洁清晰，会将很多集中的配置拆开成单独的步骤，保证有眼睛就能部署成功（所以会看上去很啰嗦，明明可以几个合并一起的非要拆开写，多此一举云云，有经验的童鞋可以自行合并）。

目标：

• 在k8s中以集群模式部署Nacos v3.0.1（3副本集群）

详情：

• mysql：v5.7 单机部署

• nacos：v3.0.1 集群部署

步骤导航

1. 准备工作目录

2. 创建ConfigMap

3. 数据库

   • 创建数据库
   • 初始化数据库
   • 创建数据库Service

4. nacos数据持久化

   • 创建StorageClass
   • 创建PV

5. nacos集群

   • 创建nacos集群
   • 创建nacos headless Service
   • 创建nacos Service
   • 创建nacos ingress

6. 访问工作台

1. 准备工作目录

作用：准备工作目录是用来存放部署时所需的yaml文件。

# 创建yaml文件路径
mkdir -p /data/kubernetes/nacos-fat && cd /data/kubernetes/nacos-fat

‍

2. 创建ConfigMap

主要是包含数据库的一些配置，通过ConfigMap传递给POD。

 vim nacos-cm.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
  namespace: nacos-fat
data:
  mode: cluster
  mysql.db.name: nacos-fat
  mysql.db.host: mysql
  mysql.port: "3306"
  mysql.user: root
  mysql.password: yourSqlPasswd
  mysql.param: "characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true&useSSL=false"
  nacos.servers: >-
    nacos3-0.nacos-hs.nacos-fat.svc.cluster.local:8848
    nacos3-1.nacos-hs.nacos-fat.svc.cluster.local:8848
    nacos3-2.nacos-hs.nacos-fat.svc.cluster.local:8848

TIP:

namespace: 本实例中将部署在nacos-fat的命名空间中，可根据实际情况修改

nacos.servers: 包含nacos集群三个副本的dns名称，由headless service管理，名称规则：pod-name.headless-name.namespace.svc.cluster.local

# 创建ConfigMap
kubectl apply -f nacos-cm.yaml

# 检查ConfigMap
kubectl get cm -n nacos-fat

‍

3. 数据库

3.1 创建数据库

nacos支持内置数据库和Mysql，但是使用集群模式部署的话必须使用MySql作为数据库。

vim nacos-mysql.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: mysql
  namespace: nacos-fat
  labels:
    name: mysql
spec:
  replicas: 1
  selector:
    name: mysql
  template:
    metadata:
      labels:
        name: mysql
    spec:
      containers:
      - name: mysql
        image: nacos/nacos-mysql:5.7
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: mysql-data
          mountPath: /var/lib/mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "yourSqlPasswd"
        - name: MYSQL_DATABASE
          value: "nacos-fat"
        - name: MYSQL_USER
          value: "nacos"
        - name: MYSQL_PASSWORD
          value: "yourSqlPasswd"
      volumes:
      - name: mysql-data
        hostPath:
          path: /data/nacos-fat/mysql
          type: DirectoryOrCreate

TIP

如果使用官方镜像，默认的数据库MYSQL_DATABASE：nacos-devtest 则不需要进行数据库初始化，此处我的是 nacos-fat，所以需要进行数据库初始化；
密码参数请自行设置；
数据库有高可用的需求请自行解决，此处就一个实例。

# 创建mysql pod
kubectl apply -f nacos-mysql.yaml

# 检查pod
kubectl get pod -n nacos-fat -o wide

3.2 初始化数据库

执行数据库初始化

# 下载初始化语句
wget https://github.com/alibaba/nacos/blob/master/distribution/conf/mysql-schema.sql

# 执行初始化
kubectl exec -i mysql-ctzmj -n nacos-fat -- mysql -uroot -pyourSqlPasswd nacos-fat < mysql-schema.sql

TIP：
请替换pod名，-u -p无空格

3.3 创建数据库Service

用于暴露数据的端口给nacos连接

vim mysql-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql
  namespace: nacos-fat
  labels:
    name: mysql
spec:
  ports:
  - port: 3306
    targetPort: 3306
  selector:
    name: mysql

# 创建mysql service
kubectl apply -f mysql-service.yaml

# 检查pod
kubectl get svc -n nacos-fat -o wide

‍

4. nacos数据持久化

4.1 创建StorageClass

作用：创建StorageClass作用是为后续部署Nacos集群时使用 volumeClaimTemplates 自动创建PVC

vim nacos-sc.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nacos-local
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

TIP：
这里使用hostPath的PV模式，所以 provisioner​的值是 kubernetes.io/no-provisioner ，如使用NFS或其他的可根据实际情况修改。

# 创建StorageClass
kubectl apply -f nacos-sc.yaml

# 检查StorageClass
kubectl get sc

4.2 创建PV

注：如你在创建StorageClass时使用了NFS或者其他provisioner，本步骤可以省略。

vim nacos-pv.yaml

kind: PersistentVolume
apiVersion: v1
metadata:
  name: nacos-pv0
spec:
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nacos-local
  hostPath:
    path: /data/nacos-fat/nacos-0
    type: DirectoryOrCreate
  capacity:
    storage: 10Gi
---
kind: PersistentVolume
apiVersion: v1
metadata:
  name: nacos-pv1
spec:
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nacos-local
  hostPath:
    path: /data/nacos-fat/nacos-1
    type: DirectoryOrCreate
  capacity:
    storage: 10Gi
---
kind: PersistentVolume
apiVersion: v1
metadata:
  name: nacos-pv2
spec:
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nacos-local
  hostPath:
    path: /data/nacos-fat/nacos-2
    type: DirectoryOrCreate
  capacity:
    storage: 10Gi

本实例中创建3个PV，对应nacos集群的3个副本，后面的nacos通过StatefulSet部署，理论上会有序地进行bond pv，所以没必要添加selector了，如有必要可以自行添加。

storageClassName 的值需对应StorageClass 的meta.name值。

# 创建pv
kubectl apply -f nacos-pv.yaml

# 检查StorageClass
kubectl get pv

‍

5. nacos集群

5.1 创建nacos集群

vim nacos-cluster.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  annotations:
    applicationVersion: nacos3.0.1
  name: nacos3
  namespace: nacos-fat
spec:
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Retain
    whenScaled: Retain
  podManagementPolicy: Parallel
  replicas: 3
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: nacos-fat
  serviceName: nacos-hs
  template:
    metadata:
      labels:
        app: nacos-fat
    spec:
      containers:
      - env:
        - name: NACOS_SERVER_PORT
          value: "8848"
        - name: NACOS_APPLICATION_PORT
          value: "8848"       
        - name: NACOS_SERVERS
          valueFrom:
           configMapKeyRef:
             name: nacos-cm
             key: nacos-servers  
        - name: PREFER_HOST_MODE
          value: "hostname"
        - name: NACOS_AUTH_TOKEN
          value: "ZDJhMzJlMTc5ZDAyZGU3YTRhMzYyYWQxYjFjNGQxMjM="
        - name: NACOS_AUTH_IDENTITY_KEY
          value: "nacos"
        - name: NACOS_AUTH_IDENTITY_VALUE
          value: "nacos"
        - name: SERVICE_NAME
          value: "nacos-hs"
        - name: DOMAIN_NAME
          value: "cluster.local"
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: SPRING_DATASOURCE_PLATFORM
          value: "mysql"
        - name: MYSQL_SERVICE_HOST
          valueFrom:
            configMapKeyRef:
              name: nacos-cm
              key: mysql.db.host
        - name: MYSQL_SERVICE_DB_NAME
          valueFrom:
            configMapKeyRef:
              name: nacos-cm
              key: mysql.db.name
        - name: MYSQL_SERVICE_PORT
          valueFrom:
            configMapKeyRef:
              name: nacos-cm
              key: mysql.port
        - name: MYSQL_SERVICE_USER
          valueFrom:
            configMapKeyRef:
              name: nacos-cm
              key: mysql.user
        - name: MYSQL_SERVICE_PASSWORD
          valueFrom:
            configMapKeyRef:
              name: nacos-cm
              key: mysql.password
        - name: MYSQL_SERVICE_DB_PARAM
          valueFrom:
            configMapKeyRef:
              name: nacos-cm
              key: mysql.param
        image: nacos/nacos-server:v3.0.1
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /v3/console/health/liveness
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 5
          successThreshold: 1
          timeoutSeconds: 10
        name: nacos
        ports:
        - containerPort: 8848
          name: client-http
          protocol: TCP
        - containerPort: 9848
          name: client-rpc
          protocol: TCP
        - containerPort: 9849
          name: raft-rpc
          protocol: TCP
        - containerPort: 7848
          name: old-raft-rpc
          protocol: TCP
        - containerPort: 8080
          name: console
          protocol: TCP
        - containerPort: 9080
          name: mcp
          protocol: TCP
        resources:
          requests:
            cpu: 500m
            memory: 2Gi
        startupProbe:
          failureThreshold: 3
          httpGet:
            path: /v3/console/health/readiness
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 180
          periodSeconds: 5
          successThreshold: 1
          timeoutSeconds: 10
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - name: nacos-pvc
          mountPath: /home/nacos/data
          subPath: data
        - name: nacos-pvc
          mountPath: /home/nacos/logs
          subPath: logs
      dnsPolicy: ClusterFirst
      initContainers:
      - image: nacos/nacos-peer-finder-plugin:1.1
        imagePullPolicy: IfNotPresent
        name: peer-finder-plugin-install
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /home/nacos/plugins/peer-finder
          name: nacos-pvc
          subPath: peer-finder
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
  volumeClaimTemplates:
  - metadata:
      name: nacos-pvc
      namespace: nacos-fat
    spec:
      storageClassName: nacos-local
      accessModes:
      - ReadWriteOnce
      volumeMode: Filesystem
      resources:
        requests:
          storage: 10Gi
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate

TIP

NACOS_AUTH_TOKEN字段的值是一个base64编码的字符串

此处使用 volumeClaimTemplates 动态创建PVC

# 创建nacos statefulset
kubectl apply -f nacos-cluster.yaml

# 检查pod
kubectl get pod -n nacos-fat -o wide

# 检查pvc 
kubectl get pvc -n nacos-fat

5.2 创建nacos headless Service

vim nacos-hs-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: nacos-hs
  namespace: nacos-fat
spec:
  clusterIP: None
  ports:
  - name: http
    port: 8848
    protocol: TCP
    targetPort: 8848
  - name: client-rpc
    port: 9848
    protocol: TCP
    targetPort: 9848
  - name: raft-rpc
    port: 9849
    protocol: TCP
    targetPort: 9849
  - name: console
    port: 8080
    protocol: TCP
    targetPort: 8080
  - name: mcp
    port: 9080
    protocol: TCP
    targetPort: 9080
  - name: old-raft-rpc
    port: 7848
    protocol: TCP
    targetPort: 7848
  publishNotReadyAddresses: true
  selector:
    app: nacos-fat
  type: ClusterIP

# 创建nacos headless service
kubectl apply -f nacos-hs-service.yaml

# 检查
kubectl get svc -n nacos-fat -o wide

‍

5.3 创建nacos Service

vim nacos-cs-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: nacos-cs
  namespace: nacos-fat
spec:
  ports:
  - name: http
    nodePort: 30848
    port: 8848
    protocol: TCP
    targetPort: 8848
  - name: client-rpc
    nodePort: 30948
    port: 9848
    protocol: TCP
    targetPort: 9848
  - name: raft-rpc
    nodePort: 30949
    port: 9849
    protocol: TCP
    targetPort: 9849
  - name: console
    nodePort: 30880
    port: 8080
    protocol: TCP
    targetPort: 8080
  - name: mcp
    nodePort: 30980
    port: 9080
    protocol: TCP
    targetPort: 9080
  - name: old-raft-rpc
    nodePort: 30748
    port: 7848
    protocol: TCP
    targetPort: 7848
  selector:
    app: nacos-fat
  type: NodePort

# 创建nacos service
kubectl apply -f nacos-cs-service.yaml

# 检查
kubectl get svc -n nacos-fat -o wide

‍

5.4 创建nacos ingress

vim nacos-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nacos-fat-ingress
  namespace: nacos-fat
spec:
  ingressClassName: apisix
  rules:
  - host: nacos-fat.eam.com
    http:
      paths:
      - backend:
          service:
            name: nacos-cs
            port:
              number: 8080
        path: /
        pathType: Prefix

TIP

我的K8S ingress controller使用的apisix，所以 ingressClassName 为 apisix，如果使用ingress-nginx则改为nginx

此处只需要发布8080端口的web服务，域名自定义（注意在DNS服务器自行添加域名记录）

# 创建nacos headless service
kubectl apply -f nacos-ingress.yaml

# 检查
kubectl get ingress -n nacos-fat -o wide

‍

6. 访问控制台

6.1 浏览器打开

集群ingress注册的域名:ingress_port/index.html

用户名和密码都是：nacos

6.2 检查集群状态

​​

参考：https://nacos.io/docs/latest/overview