mirror of
https://github.com/LCTT/TranslateProject.git
synced 2024-12-23 21:20:42 +08:00
submit tech/20180416 Running Jenkins builds in containers.md
This commit is contained in:
songshunqiang
parent
4f109fa91e
commit
1d8a0cca50
@ -1,94 +1,76 @@
|
||||
pinewall translating
|
||||
|
||||
Running Jenkins builds in containers
|
||||
在容器中运行 Jenkins 构建
|
||||
======
|
||||
|
||||
|
||||
Running applications in containers has become a well-accepted practice in the enterprise sector, as [Docker][1] with [Kubernetes][2] (K8s) now provides a scalable, manageable application platform. The container-based approach also suits the [microservices architecture][3] that's gained significant momentum in the past few years.
|
||||
由于 [Docker][1] 和 [Kubernetes][2] (K8s) 目前提供了可扩展、可管理的应用平台，将应用运行在容器中的实践已经被企业广泛接受。近些年势头很猛的[微服务架构][3]也很适合用容器实现。
|
||||
|
||||
One of the most important advantages of a container application platform is the ability to dynamically bring up isolated containers with resource limits. Let's check out how this can change the way we run our continuous integration/continuous development (CI/CD) tasks.
|
||||
容器应用平台可以动态启动指定资源配额、互相隔离的容器，这是其最主要的优势之一。让我们看看这会对我们运行持续集成/持续部署 (continuous integration/continuous development, CI/CD) 任务的方式产生怎样的改变。
|
||||
|
||||
Building and packaging an application requires an environment that can download the source code, access dependencies, and have the build tools installed. Running unit and component tests as part of the build may use local ports or require third-party applications (e.g., databases, message brokers, etc.) to be running. In the end, we usually have multiple, pre-configured build servers with each running a certain type of job. For tests, we maintain dedicated instances of third-party apps (or try to run them embedded) and avoid running jobs in parallel that could mess up each other's outcome. The pre-configuration for such a CI/CD environment can be a hassle, and the required number of servers for different jobs can significantly change over time as teams shift between versions and development platforms.
|
||||
构建并打包应用需要一定的环境，要求能够下载源代码、使用相关依赖及已经安装构建工具。作为构建的一部分，运行单元及组件测试可能会用到本地端口或需要运行第三方应用 (例如数据库及消息中间件等)。另外，我们一般定制化多台构建服务器，每台执行一种指定类型的构建任务。为方便测试，我们维护一些实例专门用于运行第三方应用 (或者试图在构建服务器上启动这些第三方应用)，避免并行运行构建任务防止结果互相干扰。为 CI/CD 环境定制化构建服务器是一项繁琐的工作，而且随着开发团队使用的开发平台或其版本变更，所需构建服务器的数量也会变更。
|
||||
|
||||
Once we have access to a container platform (onsite or in the cloud), it makes sense to move the resource-intensive CI/CD task executions into dynamically created containers. In this scenario, build environments can be independently started and configured for each job execution. Tests during the build have free reign to use available resources in this isolated box, while we can also bring up a third-party application in a side container that exists only for this job's lifecycle.
|
||||
一旦我们有了容器管理平台 (自建或在云端)，将资源密集型的 CI/CD 任务在动态生成的容器中执行是比较合理的。在这种方案中，每个构建任务运行在独立启动并配置的构建环境中。构建过程中，构建任务的测试环节可以任意使用隔离环境中的可用资源；此外，我们也可以在辅助容器中启动一个第三方应用，只在构建任务生命周期中为测试提供服务。
|
||||
|
||||
It sounds nice… Let's see how it works in real life.
|
||||
听上去不错，让我们在现实环境中实践一下。
|
||||
|
||||
Note: This article is based on a real-world solution for a project running on a [Red Hat OpenShift][4] v3.7 cluster. OpenShift is the enterprise-ready version of Kubernetes, so these practices work on a K8s cluster as well. To try, download the [Red Hat CDK][5] and run the `jenkins-ephemeral` or `jenkins-persistent` [templates][6] that create preconfigured Jenkins masters on OpenShift.
|
||||
注：本文基于现实中已有的解决方案，即在 [Red Hat OpenShift][4] v3.7 集群上运行项目。OpenShift 是企业就绪版本的 Kubernetes，故这些实践也适用于 K8s 集群。如果愿意尝试，可以下载 [Red Hat CDK][5]，运行 `jenkins-ephemeral` 或 `jenkins-persistent` [模板][6]在 OpenShift 上创建定制化好的 Jenkins 管理节点。
|
||||
|
||||
### Solution overview
|
||||
### 解决方案概述
|
||||
|
||||
The solution to executing CI/CD tasks (builds, tests, etc.) in containers on OpenShift is based on [Jenkins distributed builds][7], which means:
|
||||
在 OpenShift 容器中执行 CI/CD 任务 (构建和测试等) 的方案基于[分布式 Jenkins 构建][7]，具体如下：
|
||||
* 我们需要一个 Jenkins 主节点；可以运行在集群中，也可以是外部提供
|
||||
* 支持 Jenkins 特性和插件，故已有项目仍可使用
|
||||
* 可以用 Jenkins GUI 配置、运行任务或查看任务输出
|
||||
* 如果你愿意编码，也可以使用 [Jenkins Pipeline][8]
|
||||
|
||||
* We need a Jenkins master; it may run inside the cluster but also works with an external master
|
||||
* Jenkins features/plugins are available as usual, so existing projects can be used
|
||||
* The Jenkins GUI is available to configure, run, and browse job output
|
||||
* if you prefer code, [Jenkins Pipeline][8] is also available
|
||||
|
||||
|
||||
|
||||
From a technical point of view, the dynamic containers to run jobs are Jenkins agent nodes. When a build kicks off, first a new node starts and "reports for duty" to the Jenkins master via JNLP (port 5000). The build is queued until the agent node comes up and picks up the build. The build output is sent back to the master—just like with regular Jenkins agent servers—but the agent container is shut down once the build is done.
|
||||
从技术角度来看，运行任务的动态容器是 Jenkins 代理节点。当构建启动时，首先是一个新节点启动，通过 Jenkins 主节点的 JNLP (5000 端口) 告知就绪状态。在代理节点启动并提取构建任务之前，构建任务处于排队状态。就像通常 Jenkins 代理服务器那样，构建输出会送达主节点；不同的是，构建完成后代理节点容器会自动关闭。
|
||||
|
||||
|
||||
|
||||
Different kinds of builds (e.g., Java, NodeJS, Python, etc.) need different agent nodes. This is nothing new—labels could previously be used to restrict which agent nodes should run a build. To define the config for these Jenkins agent containers started for each job, we will need to set the following:
|
||||
不同类型的构建任务 (例如 Java, NodeJS, Python等) 对应不同的代理节点。这并不新奇，之前也是使用标签来限制哪些代理节点可以运行指定的构建任务。启动用于构建任务的 Jenkins 代理节点容器需要配置参数，具体如下:
|
||||
* 用于启动容器的 Docker 镜像
|
||||
* 资源限制
|
||||
* 环境变量
|
||||
* 挂载卷
|
||||
|
||||
* The Docker image to boot up
|
||||
* Resource limits
|
||||
* Environment variables
|
||||
* Volumes mounted
|
||||
这里用到的关键组件是 [Jenkins Kubernetes 插件][9]。该插件 (通过使用一个服务账号) 与 K8s 集群交互，可以启动和关闭代理节点。在插件的配置管理中，多种代理节点类型表现为多种Kubernetes pod 模板，它们通过项目标签对应。
|
||||
|
||||
这些[代理节点镜像][10]以开箱即用的方式提供 (也有 [CentOS7][11] 系统的版本):
|
||||
* [jenkins-slave-base-rhel7][12]：基础镜像，启动与 Jenkins 主节点连接的代理节点；其中 Java 堆大小根据容器内容设置
|
||||
* [jenkins-slave-maven-rhel7][13]：用于 Maven 和 Gradle 构建的镜像 (从基础镜像扩展)
|
||||
* [jenkins-slave-nodejs-rhel7][14]：包含 NodeJS4 工具的镜像 (从基础镜像扩展)
|
||||
|
||||
注意：本解决方案与 OpenShift 中的 [Source-to-Image (S2I)][15] 构建不同，虽然后者也可以用于某些特定的 CI/CD 任务。
|
||||
|
||||
The core component here is the [Jenkins Kubernetes plugin][9]. This plugin interacts with the K8s cluster (by using a ServiceAccount) and starts/stops the agent nodes. Multiple agent types can be defined as Kubernetes pod templates under the plugin's configuration (refer to them by label in projects).
|
||||
### 入门学习资料
|
||||
|
||||
These [agent images][10] are provided out of the box (also on [CentOS7][11]):
|
||||
有很多不错的博客和文档介绍了如何在 OpenShift 上执行 Jenkins 构建。不妨从下面这些开始：
|
||||
* [OpenShift Jenkins][29] 镜像文档及 [源代码][30]
|
||||
* 网络直播[基于 OpenShift 的 CI/CD][31]
|
||||
* [外部 Jenkins 集成][32] playbook
|
||||
|
||||
* [jenkins-slave-base-rhel7][12]: Base image starting the agent that connects to Jenkins master; the Java heap is set according to container memory
|
||||
* [jenkins-slave-maven-rhel7][13]: Image for Maven and Gradle builds (extends base)
|
||||
* [jenkins-slave-nodejs-rhel7][14]: Image with NodeJS4 tools (extends base)
|
||||
阅读这些博客和文档有助于完整的理解本解决方案。在本文中，我们主要关注具体实践中遇到的各类问题。
|
||||
|
||||
### 构建我的应用
|
||||
|
||||
作为[示例项目][16]，我们选取了包含如下构建步骤的 Java 项目：
|
||||
|
||||
Note: This solution is not related to OpenShift's [Source-to-Image (S2I)][15] build, which can also be used for certain CI/CD tasks.
|
||||
* **代码源：** 从一个Git代码库中获取项目代码
|
||||
* **使用 Maven 编译：** 依赖可从内部仓库获取，(不妨使用 Apache Nexus) 从外部 Maven 仓库镜像
|
||||
* **发布 artifact：** 将编译好的 JAR 上传至内部仓库
|
||||
|
||||
### Background learning material
|
||||
在 CI/CD 过程中，我们需要与 Git 和 Nexus 交互，故 Jenkins 任务需要能够访问这些系统。这涉及参数配置和已存储凭证，可以在下列位置进行存放及管理：
|
||||
* **在 Jenkins 中：** 我们可以在 Jenkins 中添加凭证，通过 Git 插件获取项目代码 (使用容器不会改变操作)
|
||||
* **在 OpenShift 中：** 使用 ConfigMap 和 Secret 对象，以文件或环境变量的形式附加到 Jenkins 代理容器中
|
||||
* **在高度定制化的 Docker 容器中：** 镜像是定制化的，已包含完成特定类型构建的全部特性；从一个代理镜像进行扩展即可得到。
|
||||
|
||||
There are several good blogs and documentation about Jenkins builds on OpenShift. The following are good to start with:
|
||||
你可以按自己的喜好选择一种实现方式，甚至你最终可能混用多种实现方式。下面我们采用第二种实现方式，即首选在 OpenShift 中管理参数配置。使用 Kubernetes 插件配置来定制化 Maven 代理容器，包括设置环境变量和映射文件等。
|
||||
|
||||
* [OpenShift Jenkins][29] image documentation and [source][30]
|
||||
* [CI/CD with OpenShift][31] webcast
|
||||
* [External Jenkins Integration][32] playbook
|
||||
注意：对于 Kubernetes 插件 v1.0 版，由于 [bug][17]，在 UI 界面增加环境变量并不生效。可以升级插件，或 (作为变通方案) 直接修改 `config.xml` 文件并重启 Jenkins。
|
||||
|
||||
Take a look at them to understand the overall solution. In this article, we'll look at the different issues that come up while applying those practices.
|
||||
### 从 Git 获取源代码
|
||||
|
||||
### Build my application
|
||||
|
||||
For our [example][16], let's assume a Java project with the following build steps:
|
||||
|
||||
* **Source:** Pull project source from a Git repository
|
||||
* **Build with Maven:** Dependencies come from an internal repository (let's use Apache Nexus) mirroring external Maven repos
|
||||
* **Deploy artifact:** The built JAR is uploaded to the repository
|
||||
|
||||
|
||||
|
||||
During the CI/CD process, we need to interact with Git and Nexus, so the Jenkins jobs have be able to access those systems. This requires configuration and stored credentials that can be managed at different places:
|
||||
|
||||
* **In Jenkins:** We can add credentials to Jenkins that the Git plugin can use and add files to the project (using containers doesn't change anything).
|
||||
* **In OpenShift:** Use ConfigMap and secret objects that are added to the Jenkins agent containers as files or environment variables.
|
||||
* **In a fully customized Docker image:** These are pre-configured with everything to run a type of job; just extend one of the agent images.
|
||||
|
||||
|
||||
|
||||
Which approach you use is a question of taste, and your final solution may be a mix. Below we'll look at the second option, where the configuration is managed primarily in OpenShift. Customize the Maven agent container via the Kubernetes plugin configuration by setting environment variables and mounting files.
|
||||
|
||||
Note: Adding environment variables through the UI doesn't work with Kubernetes plugin v1.0 due to a [bug][17]. Either update the plugin or (as a workaround) edit `config.xml` directly and restart Jenkins.
|
||||
|
||||
### Pull source from Git
|
||||
|
||||
Pulling a public Git is trivial. For a private Git repo, authentication is required and the client also needs to trust the server for a secure connection. A Git pull can typically be done via two protocols:
|
||||
|
||||
* HTTPS: Authentication is with username/password. The server's SSL certificate must be trusted by the job, which is only tricky if it's signed by a custom CA.
|
||||
从公共 Git 仓库获取源代码很容易。但对于私有 Git 仓库，不仅需要认证操作，客户端还需要信任服务器以便建立安全连接。一般而言，通过两种协议获取源代码：
|
||||
* HTTPS：验证通过用户名/密码完成。Git 服务器的 SSL 证书必须被代理节点信任，这仅在证书被自建 CA 签名时才需要特别关注。
|
||||
|
||||
|
||||
```
|
||||
@ -96,7 +78,7 @@ git clone https://git.mycompany.com:443/myapplication.git
|
||||
|
||||
```
|
||||
|
||||
* SSH: Authentication is with a private key. The server is trusted when its public key's fingerprint is found in the `known_hosts` file.
|
||||
* SSH：验证通过私钥完成。如果服务器的公钥指纹出现在 `known_hosts` 文件中，那么该服务器是被信任的。
|
||||
|
||||
|
||||
```
|
||||
@ -104,24 +86,23 @@ git clone ssh://git@git.mycompany.com:22/myapplication.git
|
||||
|
||||
```
|
||||
|
||||
Downloading the source through HTTP with username/password is OK when it's done manually; for automated builds, SSH is better.
|
||||
对于手动操作，使用用户名/密码通过 HTTP 方式下载源代码是可行的；但对于自动构建而言，SSH 是更佳的选择。
|
||||
|
||||
#### Git with SSH
|
||||
#### 通过 SSH 方式使用 Git
|
||||
|
||||
For a SSH download, we need to ensure that the SSH connection works between the agent container and the Git's SSH port. First, we need a private-public key pair. To generate one, run:
|
||||
要通过 SSH 方式下载源代码，我们需要保证代理容器与 Git 的 SSH 端口之间可以建立 SSH 连接。首先，我们需要创建一个私钥-公钥对。使用如下命令生成：
|
||||
```
|
||||
ssh keygen -t rsa -b 2048 -f my-git-ssh -N ''
|
||||
|
||||
```
|
||||
|
||||
It generates a private key in `my-git-ssh` (empty passphrase) and the matching public key in `my-git-ssh.pub`. Add the public key to the user on the Git server (preferably a ServiceAccount); web UIs usually support upload. To make the SSH connection work, we need two files on the agent container:
|
||||
命令生成的私钥位于 `my-git-ssh` 文件中 (无密码口令)，对应的公钥位于 `my-git-ssh.pub` 文件中。将公钥添加至 Git 服务器的对应用户下 (推荐使用服务账号)；网页界面一般支持公钥上传。为建立 SSH 连接，我们还需要在代理容器上配置两个文件：
|
||||
|
||||
* The private key at `~/.ssh/id_rsa`
|
||||
* The server's public key in `~/.ssh/known_hosts`. To get this, try `ssh git.mycompany.com` and accept the fingerprint; this will create a new line in the `known_hosts` file. Use that.
|
||||
* 私钥文件位于 `~/.ssh/id_rsa`
|
||||
* 服务器的公钥位于 `~/.ssh/known_hosts`。要实现这一点，运行 `ssh git.mycompany.com` 并接受服务器指纹，系统会在 `~/.ssh/known_hosts` 文件中增加一行。这样需求得到了满足。
|
||||
|
||||
|
||||
|
||||
Store the private key as `id_rsa` and server's public key as `known_hosts` in an OpenShift secret (or config map).
|
||||
将 `id_rsa` 对应的私钥和 `known_hosts` 对应的公钥保存到一个 OpenShift secret (或 config map) 对象中。
|
||||
```
|
||||
apiVersion: v1
|
||||
|
||||
@ -147,7 +128,7 @@ stringData:
|
||||
|
||||
```
|
||||
|
||||
Then configure this as a volume in the Kubernetes plugin for the Maven pod at mount point `/home/jenkins/.ssh/`. Each item in the secret will be a file matching the key name under the mount directory. We can use the UI (`Manage Jenkins / Configure / Cloud / Kubernetes`), or edit Jenkins config `/var/lib/jenkins/config.xml`:
|
||||
在 Kubernetes 插件中将 secret 对象配置为卷，挂载到 `/home/jenkins/.ssh/`，供 Maven pod 使用。secret 中的每个对象对应挂载目录的一个文件，文件名与 key 名称相符。我们可以使用 UI (管理 Jenkins / 配置 / 云 / Kubernetes)，也可以直接编辑 Jenkins 配置文件 `/var/lib/jenkins/config.xml`:
|
||||
```
|
||||
<org.csanchez.jenkins.plugins.kubernetes.PodTemplate>
|
||||
|
||||
@ -169,9 +150,9 @@ Then configure this as a volume in the Kubernetes plugin for the Maven pod at mo
|
||||
|
||||
```
|
||||
|
||||
Pulling a Git source through SSH should work in the jobs running on this agent now.
|
||||
此时，在代理节点上运行的任务应该可以通过 SSH 方式从 Git 代码库获取源代码。
|
||||
|
||||
Note: It's also possible to customize the SSH connection in `~/.ssh/config`, for example, if we don't want to bother with `known_hosts` or the private key is mounted to a different location:
|
||||
注：我们也可以在 `~/.ssh/config` 文件中自定义 SSH 连接。例如，如果你不想处理 `known_hosts` 或 私钥位于其它挂载目录中：
|
||||
```
|
||||
Host git.mycompany.com
|
||||
|
||||
@ -181,11 +162,11 @@ Host git.mycompany.com
|
||||
|
||||
```
|
||||
|
||||
#### Git with HTTP
|
||||
#### 通过 HTTP 方式使用 Git
|
||||
|
||||
If you prefer an HTTP download, add the username/password to a [Git-credential-store][18] file somewhere:
|
||||
如果你选择使用 HTTP 方式下载，在指定的 [Git-credential-store][18] 文件中添加用户名/密码：
|
||||
|
||||
* E.g. `/home/jenkins/.config/git-secret/credentials` from an OpenShift secret, one site per line:
|
||||
* 例如，在一个 OpenShift secret 对象中增加 `/home/jenkins/.config/git-secret/credentials` 文件对应，其中每个站点对应文件中的一行:
|
||||
|
||||
|
||||
```
|
||||
@ -195,7 +176,7 @@ https://user:pass@github.com
|
||||
|
||||
```
|
||||
|
||||
* Enable it in [git-config][19] expected at `/home/jenkins/.config/git/config`:
|
||||
* 在 [git-config][19] 配置中启用该文件，其中配置文件默认路径为 `/home/jenkins/.config/git/config`：
|
||||
|
||||
|
||||
```
|
||||
@ -204,11 +185,10 @@ https://user:pass@github.com
|
||||
helper = store --file=/home/jenkins/.config/git-secret/credentials
|
||||
|
||||
```
|
||||
如果 Git 服务使用了自有 CA 签名的证书，为代理容器设置环境变量 `GIT_SSL_NO_VERIFY=true` 是最便捷的方式。更恰当的解决方案包括如下两步：
|
||||
|
||||
If the Git service has a certificate signed by a custom certificate authority (CA), the quickest hack is to set the `GIT_SSL_NO_VERIFY=true` environment variable (EnvVar) for the agent. The proper solution needs two things:
|
||||
|
||||
* Add the custom CA's public certificate to the agent container from a config map to a path (e.g. `/usr/ca/myTrustedCA.pem`).
|
||||
* Tell Git the path to this cert in an EnvVar `GIT_SSL_CAINFO=/usr/ca/myTrustedCA.pem` or in the `git-config` file mentioned above:
|
||||
* 利用 config map 将自有 CA 的公钥映射到一个路径下的文件中，例如 `/usr/ca/myTrustedCA.pem`)。
|
||||
* 通过环境变量 `GIT_SSL_CAINFO=/usr/ca/myTrustedCA.pem` 或上面提到的 `git-config` 文件的方式，将证书路径告知 Git。
|
||||
|
||||
|
||||
```
|
||||
@ -218,26 +198,25 @@ If the Git service has a certificate signed by a custom certificate authority (C
|
||||
|
||||
```
|
||||
|
||||
Note: In OpenShift v3.7 (and earlier), the config map and secret mount points [must not overlap][20], so we can't map to `/home/jenkins` and `/home/jenkins/dir` at the same time. This is why we didn't use the well-known file locations above. A fix is expected in OpenShift v3.9.
|
||||
注：在 OpenShift v3.7 及早期版本中，config map 及 secret 的挂载点之间[不能相互覆盖][20]，故我们不能同时映射 `/home/jenkins` 和 `/home/jenkins/dir`。因此，上面的代码中并没有使用常见的文件路径。预计 OpenShift v3.9 版本会修复这个问题。
|
||||
|
||||
### Maven
|
||||
|
||||
To make a Maven build work, there are usually two things to do:
|
||||
要完成 Maven 构建，一般需要完成如下两步：
|
||||
|
||||
* A corporate Maven repository (e.g., Apache Nexus) should be set up to act as a proxy for external repos. Use this as a mirror.
|
||||
* This internal repository may have an HTTPS endpoint with a certificate signed by a custom CA.
|
||||
* 建立一个社区 Maven 库 (例如 Apache Nexus)，充当外部库的代理。将其当作镜像使用。
|
||||
* 这个内部库可能提供 HTTPS 服务，其中使用自建 CA 签名的证书。
|
||||
|
||||
|
||||
对于容器中运行构建的实践而言，使用内部 Maven 库是非常关键的，因为容器启动后并没有本地库或缓存，这导致每次构建时 Maven 都下载全部的 Jar 文件。在本地网络使用内部代理库下载明显快于从因特网下载。
|
||||
|
||||
Having an internal Maven repository is practically essential if builds run in containers because they start with an empty local repository (cache), so Maven downloads all the JARs every time. Downloading from an internal proxy repo on the local network is obviously quicker than downloading from the Internet.
|
||||
|
||||
The [Maven Jenkins agent][13] image supports an environment variable that can be used to set the URL for this proxy. Set the following in the Kubernetes plugin container template:
|
||||
[Maven Jenkins 代理][13]镜像允许配置环境变量，指定代理的 URL。在 Kubernetes 插件的容器模板中设置如下：
|
||||
```
|
||||
MAVEN_MIRROR_URL=https://nexus.mycompany.com/repository/maven-public
|
||||
|
||||
```
|
||||
|
||||
The build artifacts (JARs) should also be archived in a repository, which may or may not be the same as the one acting as a mirror for dependencies above. Maven `deploy` requires the repo URL in the `pom.xml` under [Distribution management][21] (this has nothing to do with the agent image):
|
||||
构建好的 artifacts (JARs) 也应该保存到库中，可以是上面提到的用于提供依赖的镜像库，也可以是其它库。Maven 完成 `deploy` 操作需要在 `pom.xml` 的[分发管理][21] 下配置库 URL，这与代理镜像无关。
|
||||
```
|
||||
<project ...>
|
||||
|
||||
@ -263,9 +242,9 @@ The build artifacts (JARs) should also be archived in a repository, which may or
|
||||
|
||||
```
|
||||
|
||||
Uploading the artifact may require authentication. In this case, username/password must be set in the `settings.xml` under the server ID matching the one in `pom.xml`. We need to mount a whole `settings.xml` with the URL, username, and password on the Maven Jenkins agent container from an OpenShift secret. We can also use environment variables as below:
|
||||
上传 artifact 可能涉及认证。在这种情况下，需要在 `settings.xml` 中配置用户名/密码，其中 server ID 要与 `pom.xml` 文件中的 server ID 对应。我们可以使用 OpenShift secret 将包含 URL、用户名和密码的完整 `settings.xml` 映射到 Maven Jenkins 代理容器中。另外，也可以使用环境变量。具体如下：
|
||||
|
||||
* Add environment variables from a secret to the container:
|
||||
* 利用 secret 为容器添加环境变量：
|
||||
|
||||
|
||||
```
|
||||
@ -275,7 +254,7 @@ MAVEN_SERVER_PASSWORD=admin123
|
||||
|
||||
```
|
||||
|
||||
* Mount `settings.xml` from a config map to `/home/jenkins/.m2/settings.xml`:
|
||||
* 利用 config map 将 `settings.xml` 挂载至 `/home/jenkins/.m2/settings.xml`：
|
||||
|
||||
|
||||
```
|
||||
@ -313,9 +292,9 @@ MAVEN_SERVER_PASSWORD=admin123
|
||||
|
||||
```
|
||||
|
||||
Disable interactive mode (use batch mode) to skip the download log by using `-B` for Maven commands or by adding `<interactiveMode>false</interactiveMode>` to `settings.xml`.
|
||||
禁用交互模式 (即使用批处理模式) 可以忽略下载日志，一种方式是在 Maven 命令中增加 `-B` 参数，另一种方式是在 `settings.xml` 配置文件中增加 `<interactiveMode>false</interactiveMode>` 配置。
|
||||
|
||||
If the Maven repository's HTTPS endpoint uses a certificate signed by a custom CA, we need to create a Java KeyStore using the [keytool][22] containing the CA certificate as trusted. This KeyStore should be uploaded as a config map in OpenShift. Use the `oc` command to create a config map from files:
|
||||
如果 Maven 库的 HTTPS 服务使用自建 CA 签名的证书，我们需要使用 [keytool][22] 工具创建一个将 CA 公钥添加至信任列表的 Java KeyStore。在 OpenShift 中使用 config map 将这个 Keystore 上传。使用 `oc` 命令基于文件创建一个 config map：
|
||||
```
|
||||
oc create configmap maven-settings --from-file=settings.xml=settings.xml --from-
|
||||
|
||||
@ -323,9 +302,9 @@ file=myTruststore.jks=myTruststore.jks
|
||||
|
||||
```
|
||||
|
||||
Mount the config map somewhere on the Jenkins agent. In this example we use `/home/jenkins/.m2`, but only because we have `settings.xml` in the same config map. The KeyStore can go under any path.
|
||||
将这个 config map 挂载至 Jenkins 代理容器。在本例中我们使用 `/home/jenkins/.m2` 目录，但这仅仅是因为配置文件 `settings.xml` 也对应这个 config map，KeyStore 可以放置在任意路径下。
|
||||
|
||||
Then make the Maven Java process use this file as a trust store by setting Java parameters in the `MAVEN_OPTS` environment variable for the container:
|
||||
接着在容器环境变量 `MAVEN_OPTS` 中设置 Java 参数，以便让 Maven 对应的 Java 进程使用该文件：
|
||||
```
|
||||
MAVEN_OPTS=
|
||||
|
||||
@ -335,30 +314,28 @@ MAVEN_OPTS=
|
||||
|
||||
```
|
||||
|
||||
### Memory usage
|
||||
### 内存使用量
|
||||
|
||||
This is probably the most important part—if we don't set max memory correctly, we'll run into intermittent build failures after everything seems to work.
|
||||
这可能是最重要的一部分设置，如果没有正确的设置最大内存，我们会遇到间歇性构建失败，虽然每个组件都似乎工作正常。
|
||||
|
||||
Running Java in a container can cause high memory usage errors if we don't set the heap in the Java command line. The JVM [sees the total memory of the host machine][23] instead of the container's memory limit and sets the [default max heap][24] accordingly. This is typically much more than the container's memory limit, and OpenShift simply kills the container when a Java process allocates more memory for the heap.
|
||||
如果没有在 Java 命令行中设置堆大小，在容器中运行 Java 可能导致高内存使用量的报错。JVM [可以利用全部的主机内存][23]，因而使用[默认的堆大小限制][24]。这通常会超过容器的内存资源总额，故当 Java 进程为堆分配过多内存时，OpenShift 会直接杀掉容器。
|
||||
|
||||
Although the `jenkins``-slave-base` image has a built-in [script to set max heap ][25]to half the container memory (this can be modified via EnvVar `CONTAINER_HEAP_PERCENT=0.50`), it only applies to the Jenkins agent Java process. In a Maven build, we have important additional Java processes running:
|
||||
虽然 `jenkins` `-slave-base` 镜像包含一个内建[脚本设置堆最大为][25]容器内存的一半 (可以通过环境变量 `CONTAINER_HEAP_PERCENT=0.50` 修改)，但这只适用于 Jenkins 代理节点中的 Java 进程。在 Maven 构建中，还有其它重要的 Java 进程运行：
|
||||
|
||||
* The `mvn` command itself is a Java tool.
|
||||
* The [Maven Surefire-plugin][26] executes the unit tests in a forked JVM by default.
|
||||
* `mvn` 命令本身就是一个 Java 工具。
|
||||
* [Maven Surefire 插件][26] 按默认参数派生的 JVM 用于运行单元测试。
|
||||
|
||||
|
||||
总结一下，容器中同时运行着三个重要的 Java 进程，预估内存使用量以避免 pod 被误杀是很重要的。每个进程都有不同的方式设置 JVM 参数：
|
||||
|
||||
At the end of the day, we'll have three Java processes running at the same time in the container, and it's important to estimate their memory usage to avoid unexpectedly killed pods. Each process has a different way to set JVM options:
|
||||
|
||||
* Jenkins agent heap is calculated as mentioned above, but we definitely shouldn't let the agent have such a big heap. Memory is needed for the other two JVMs. Setting `JAVA_OPTS` works for the Jenkins agent.
|
||||
* The `mvn` tool is called by the Jenkins job. Set `MAVEN_OPTS` to customize this Java process.
|
||||
* The JVM spawned by the Maven `surefire` plugin for the unit tests can be customized by the [argLine][27] Maven property. It can be set in the `pom.xml`, in a profile in `settings.xml` or simply by adding `-DargLine=… to mvn` command in `MAVEN_OPTS`.
|
||||
* 我们在上面提到了 Jenkins 代理容器堆最大值的计算方法，但我们显然不应该让代理容器使用如此大的堆，毕竟还有两个 JVM 需要使用内存。对于 Jenkins 代理容器，可以设置 `JAVA_OPTS`。
|
||||
* `mvn` 工具被 Jenkins 任务调用。设置 `JAVA_OPTS` 可以用于自定义这类 Java 进程。
|
||||
* Maven `surefire` 插件派生的用于单元测试的 JVM 可以通过 Maven [argLine][27] 属性自定义。可以在 `pom.xml` 或 `settings.xml` 的某个配置文件中设置，也可以直接在 `maven` 命令参数 `MAVEN_OPS` 中增加 `-DargLine=…`。
|
||||
|
||||
|
||||
|
||||
Here is an example of how to set these environment variables for the Maven agent container:
|
||||
下面例子给出 Maven 代理容器环境变量设置方法：
|
||||
```
|
||||
JAVA_OPTS=-Xms64m -Xmx64m
|
||||
JAVA_OPTS=-Xms64m -Xmx64m
|
||||
|
||||
MAVEN_OPTS=-Xms128m -Xmx128m -DargLine=${env.SUREFIRE_OPTS}
|
||||
|
||||
@ -366,17 +343,17 @@ SUREFIRE_OPTS=-Xms256m -Xmx256m
|
||||
|
||||
```
|
||||
|
||||
These numbers worked in our tests with 1024Mi agent container memory limit building and running unit tests for a SpringBoot app. These are relatively low numbers and a bigger heap size; a higher limit may be needed for complex Maven projects and unit tests.
|
||||
我们的测试环境是具有 1024Mi 内存限额的代理容器，使用上述参数可以正常构建一个 SpringBoot 应用并进行单元测试。测试环境使用的资源相对较小，对于复杂的 Maven 项目和对应的单元测试，我们需要更大的堆大小及更大的容器内存限额。
|
||||
|
||||
Note: The actual memory usage of a Java8 process is something like `HeapSize + MetaSpace + OffHeapMemory`, and this can be significantly more than the max heap size set. With the settings above, the three Java processes took more than 900Mi memory in our case. See RSS memory for processes within the container: `ps -e -o ``pid``,user``,``rss``,comm``,args`
|
||||
注：Java8 进程的实际内存使用量包括 `堆大小 + 元数据 + 堆外内存`，因此内存使用量会明显高于设置的最大堆大小。在我们上面的测试环境中，三个 Java 进程使用了超过 900Mi 的内存。可以在容器内查看进程的 RSS 内存使用情况，命令如下：`ps -e -o ``pid``,user``,``rss``,comm``,args`。
|
||||
|
||||
The Jenkins agent images have both JDK 64 bit and 32 bit installed. For `mvn` and `surefire`, the 64-bit JVM is used by default. To lower memory usage, it makes sense to force 32-bit JVM as long as `-Xmx` is less than 1.5 GB:
|
||||
Jenkins 代理镜像同时安装了 JDK 64 位和 32 位版本。对于 `mvn` 和 `surefire`，默认使用 64 位版本 JVM。为减低内存使用量，只要 `-Xmx` 不超过 1.5 GB，强制使用 32 位 JVM 都是有意义的。
|
||||
```
|
||||
JAVA_HOME=/usr/lib/jvm/Java-1.8.0-openjdk-1.8.0.161–0.b14.el7_4.i386
|
||||
|
||||
```
|
||||
|
||||
Note that it's also possible to set Java arguments in the `JAVA_TOOL_OPTIONS` EnvVar, which is picked up by any JVM started. The parameters in `JAVA_OPTS` and `MAVEN_OPTS` overwrite the ones in `JAVA_TOOL_OPTIONS`, so we can achieve the same heap configuration for our Java processes as above without using `argLine`:
|
||||
注意到我们可以在 `JAVA_TOOL_OPTIONS` 环境变量中设置 Java 参数，每个 JVM 启动时都会读取该参数。`JAVA_OPTS` 和 `MAVEN_OPTS` 中的参数会覆盖 `JAVA_TOOL_OPTIONS` 中的对应值，故我们可以不使用 `argLine`，实现对 Java 进程同样的堆配置：
|
||||
```
|
||||
JAVA_OPTS=-Xms64m -Xmx64m
|
||||
|
||||
@ -386,11 +363,11 @@ JAVA_TOOL_OPTIONS=-Xms256m -Xmx256m
|
||||
|
||||
```
|
||||
|
||||
It's still a bit confusing, as all JVMs log `Picked up JAVA_TOOL_OPTIONS:`
|
||||
但缺点是每个 JVM 的日志中都会显示 `Picked up JAVA_TOOL_OPTIONS:`，这可能让人感到迷惑。
|
||||
|
||||
### Jenkins Pipeline
|
||||
### Jenkins 流水线
|
||||
|
||||
Following the settings above, we should have everything prepared to run a successful build. We can pull the code, download the dependencies, run the unit tests, and upload the artifact to our repository. Let's create a Jenkins Pipeline project that does this:
|
||||
完成上述配置，我们应该已经可以完成一次成功的构建。我们可以获取源代码，下载依赖，运行单元测试并将 artifact 上传到我们的库中。我们可以通过创建一个 Jenkins 流水线项目来完成上述操作。
|
||||
```
|
||||
pipeline {
|
||||
|
||||
@ -442,15 +419,15 @@ pipeline {
|
||||
|
||||
```
|
||||
|
||||
For a real project, of course, the CI/CD pipeline should do more than just the Maven build; it could deploy to a development environment, run integration tests, promote to higher environments, etc. The learning articles linked above show examples of how to do those things.
|
||||
当然，对应真实项目，CI/CD 流水线不仅仅完成 Maven 构建，还可以部署到开发环境，运行集成测试，提升至更接近于生产的环境等。上面给出的学习资料中有执行这些操作的案例。
|
||||
|
||||
### Multiple containers
|
||||
### 多容器
|
||||
|
||||
One pod can be running multiple containers with each having their own resource limits. They share the same network interface, so we can reach started services on `localhost`, but we need to think about port collisions. Environment variables are set separately, but the volumes mounted are the same for all containers configured in one Kubernetes pod template.
|
||||
一个 pod 可以运行多个容器，每个容器有单独的资源限制。这些容器共享网络接口，故我们可以从 `localhost` 访问已启动的服务，但我们需要考虑端口冲突的问题。在一个 Kubernetes pod 模板中，每个容器的环境变量是单独设置的，但挂载的卷是统一的。
|
||||
|
||||
Bringing up multiple containers is useful when an external service is required for unit tests and an embedded solution doesn't work (e.g., database, message broker, etc.). In this case, this second container also starts and stops with the Jenkins agent.
|
||||
当一个外部服务需要单元测试且嵌入式方案无法工作 (例如，数据库、消息中间件等) 时，可以启动多个容器。在这种情况下，第二个容器会随着 Jenkins 代理容器启停。
|
||||
|
||||
See the Jenkins `config.xml` snippet where we start an `httpbin` service on the side for our Maven build:
|
||||
查看 Jenkins `config.xml` 片段，其中我们启动了一个辅助的 `httpbin` 服务用于 Maven 构建：
|
||||
```
|
||||
<org.csanchez.jenkins.plugins.kubernetes.PodTemplate>
|
||||
|
||||
@ -508,9 +485,9 @@ See the Jenkins `config.xml` snippet where we start an `httpbin` service on the
|
||||
|
||||
```
|
||||
|
||||
### Summary
|
||||
### 总结
|
||||
|
||||
For a summary, see the created OpenShift resources and the Kubernetes plugin configuration from Jenkins `config.xml` with the configuration described above.
|
||||
作为总结，我们查看上面已描述配置的 Jenkins `config.xml` 对应创建的 OpenShift 资源以及 Kubernetes 插件的配置。
|
||||
```
|
||||
apiVersion: v1
|
||||
|
||||
@ -608,7 +585,7 @@ items:
|
||||
|
||||
```
|
||||
|
||||
One additional config map was created from files:
|
||||
基于文件创建另一个 config map：
|
||||
```
|
||||
oc create configmap maven-settings --from-file=settings.xml=settings.xml
|
||||
|
||||
@ -616,7 +593,7 @@ One additional config map was created from files:
|
||||
|
||||
```
|
||||
|
||||
Kubernetes plugin configuration:
|
||||
Kubernetes 插件配置如下：
|
||||
```
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
|
||||
@ -914,16 +891,16 @@ MIIC6jCC...
|
||||
|
||||
```
|
||||
|
||||
Happy builds!
|
||||
尝试愉快的构建吧！
|
||||
|
||||
This was originally published on [ITNext][28] and is reprinted with permission.
|
||||
原文发表于 [ITNext][28]，已获得翻版授权。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/18/4/running-jenkins-builds-containers
|
||||
|
||||
作者：[Balazs Szeti][a]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
译者：[pinewall](https://github.com/pinewall)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
选题：[lujun9972](https://github.com/lujun9972)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user