# TKE上动态部署jenkins-slaves

本章我们来讲一下如何在TKE上搭建jenkins这个CI/CD工具，这边我们在进行构建的时候可以自动生成一个slave pod来进行走流水线，流水线跑完之后会自动销毁。

## 部署jenkins的rbac权限

```
[root@VM_0_13_centos jenkins]# cat rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins2
  namespace: jenkins

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins2
rules:
  - apiGroups: ["extensions", "apps"]
    resources: ["deployments"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: jenkins2
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkins2
subjects:
  - kind: ServiceAccount
    name: jenkins2
    namespace: jenkins
```

## 创建pvc进行jenkins的数据持久化

这边通过cbs卷挂载jenkins的数据

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F822d3aeeb0a49eaa3d83b8b5ec89b823f244bae2.png?generation=1606373409702821\&alt=media)

## 部署jenkins的应用

```
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins2
  namespace: kube-ops
spec:
  template:
    metadata:
      labels:
        app: jenkins2
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins2
      containers:
      - name: jenkins
        image: jenkins/jenkins:lts
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        - containerPort: 50000
          name: agent
          protocol: TCP
        resources:
          limits:
            cpu: 1000m
            memory: 1Gi
          requests:
            cpu: 500m
            memory: 512Mi
        livenessProbe:
          httpGet:
            path: /login
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12
        readinessProbe:
          httpGet:
            path: /login
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12
        volumeMounts:
        - name: jenkinshome
          subPath: jenkins2
          mountPath: /var/jenkins_home
      securityContext:
        fsGroup: 1000
      volumes:
      - name: jenkinshome
        persistentVolumeClaim:
          claimName: opspvc

---
apiVersion: v1
kind: Service
metadata:
  name: jenkins2
  namespace: kube-ops
  labels:
    app: jenkins2
spec:
  selector:
    app: jenkins2
  type: NodePort
  ports:
  - name: web
    port: 8080
    targetPort: web
    nodePort: 30002
  - name: agent
    port: 50000
    targetPort: agent
```

## 采用treafik暴露域名给jenkins

```
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: jenkins
  namespace: jenkins
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`jenkins.tx.niewx.club`)
    kind: Rule
    services:
    - name: jenkins2
      port: 8080
```

## 访问初始化jenkins

查看登录的初始密码，然后安装好推荐的插件，安装好之后配置账号，直接登录即可。

```
cat /var/lib/jenkins/secrets/initialAdminPassword
```

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F799c79ea4e61ad2c80ee89326951388e33dc7765.png?generation=1606373408786634\&alt=media)

## 安装Kubernetes-plugin 插件

我这边已经安装好了，可以选择可选插件输入Kubernetes搜索找到插件进行安装

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F65a3d2717f5845fbe0f38cd38af54d6a0c16b2bf.png?generation=1606373408820951\&alt=media)

## 连接TKE集群

在“系统配置”面板最下方，选择“云”模块下的【新增一个云】>【Kubernetes】。如下图所示

对应的证书文件和token以及api地址可以从TKE集群中节点上的下面文件获取

集群证书获取

```
[root@VM_1_4_centos ~]# cat /etc/kubernetes/cluster-ca.crt
-----BEGIN CERTIFICATE-----
MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
cm5ldGVzMB4XDTIwMDYwMjAzMTYzNVoXDTMwMDUzMTAzMTYzNVowFTETMBEGA1UE
AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANmh
Z/tm6Ej66vNgkcRk4zirSGpoLpHaofq/q9eOE8zLr+QfzwNlDImeNfpX/OG/atz2
bl6DY4kAt7CsHUVs60leCIs1PZHY212vevlL/nWceLHqVeskynwT4xwJKI48DnwM
UgABoXTqm3S2k02CRJcm0Ucx1pw+E0l0Cy/hJcEwa8kFaqsskHqL1jHhRGB4ENGO
6ocjEzZstZdjv1Ab1WoOmHhvvI3mT4Gqb1BSd3BFqlu0id1wJvIXuood0bYDTXSD
KHHKZFOnrxwFFAg3ZxmrFDkfqfmILV0dcA4KKW/1q0g7+XepM2gZrrbwMAjVaOpH
vPUEaLn/kJ1LSUNz09UCAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKUMA8GA1UdEwEB
/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAC6jddY1Y5ofHZkthZu8Hj2Op36q
kY9pv1fpexcPXtcRuxgw6+mQjEzKraEDoDYdX/DkHkuYVulWl6Sqv+P9WRY+hLzH
/K+GHh4uuuIlL+C6Uk1nV0clrznv+CqXKY9AZ8sY+x6l9d3hIxaDxwKWDfzJe2sU
zMJZsbxyy4wIkwlG5+1lSD3YYRo1TooYGeFELD0OdGMrH9XRP0E/6Q5XdRBecH54
JA6Tj4vXI2SgOX4MWR47FFI7agBgATiOLLYGVcRqzf6rK/QFbOnT1R1JAR4=
-----END CERTIFICATE-----
```

apiserver地址和token

```
[root@VM_1_4_centos ~]# cat .kube/config
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJNDQWJD....
    server: https://169.xx.xx.xx:60002
  name: local
contexts:
- context:
    cluster: local
    user: admin
  name: master
current-context: master
kind: Config
preferences: {}
users:
- name: admin
  user:
    token: MUrM6Aj4qwgKKjwF....
```

将上述获取的填入集群配置

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2Fcedf279886635a640015c2495b52d72812de52f1.png?generation=1606373408037564\&alt=media)

其他配置如下图所示，我们jenkins的master采用的hosts模式，所以这边直接用节点的8080端口

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F63d89a6d528451d062ed51cce076557579b202ff.png?generation=1606373410072547\&alt=media)

slave 的 pod的配置如下

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2Fa441783b54bc9bd83dd11d2dfdb4a5277ff0ef33.png?generation=1606373408928883\&alt=media)

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2Fd9dddd283bac6c5372bc017df03da358e77d3049.png?generation=1606373408921904\&alt=media)

## 创建测试项目进行构建

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2Febf86bae27df70872fa22478fadc7ebc2992795b.png?generation=1606373410099743\&alt=media)

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F262c6c1e3d527afdc986c9e20e158f5c9a692e43.png?generation=1606373408489216\&alt=media)

```
echo "测试 Kubernetes 动态生成 jenkins slave"
echo "==============docker in docker==========="
docker info

echo "=============kubectl============="
kubectl get pods
```

## 执行任务构建

执行构建会自动生成一个slave pod，执行任务结束后会自动删除掉

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F620fd9fbc1b4ed1b273937d899a5614e1dc5f1c6.png?generation=1606373409545454\&alt=media)

```
[root@VM_1_4_centos ~]# kubectl get pod -n jenkins
NAME                        READY   STATUS    RESTARTS   AGE
jenkins2-574678b486-8r4qz   1/1     Running   0          2d5h
jnlp-7zv3k                  1/1     Running   0          14s
```

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F21a7ad64e9f9e17c142e26b6e372762f9f41e045.png?generation=1606373409058707\&alt=media)

![upload-image](https://1743139827-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMy-ccvHJYp-MpGyagk%2Fsync%2F0422b90c213ebce4eac5cc23926f9da31e3f0472.png?generation=1606373409324023\&alt=media)

## 参考文档

<https://www.qikqiak.com/post/kubernetes-jenkins1/>

<https://cloud.tencent.com/document/product/457/41396>
