mirror of
https://github.com/LCTT/TranslateProject.git
synced 2024-12-26 21:30:55 +08:00
translated
20210510 Make Jenkins logs pretty is translated by DCOLIVERSUN
This commit is contained in:
Qian.Sun
parent
12185957d1
commit
1cef37047e
@ -1,183 +0,0 @@
|
|||||||
[#]: subject: (Make Jenkins logs pretty)
|
|
||||||
[#]: via: (https://opensource.com/article/21/5/jenkins-logs)
|
|
||||||
[#]: author: (Evan "Hippy" Slatis https://opensource.com/users/hippyod)
|
|
||||||
[#]: collector: (lujun9972)
|
|
||||||
[#]: translator: (DCOLIVERSUN)
|
|
||||||
[#]: reviewer: ( )
|
|
||||||
[#]: publisher: ( )
|
|
||||||
[#]: url: ( )
|
|
||||||
|
|
||||||
Make Jenkins logs pretty
|
|
||||||
======
|
|
||||||
Jenkins' default logs can be hard to read, but they don't have to be.
|
|
||||||
![Person using a laptop][1]
|
|
||||||
|
|
||||||
Jenkins is a free and open source automation server for building, testing, and deploying code. It's the backbone of continuous integration and continuous delivery (CI/CD) and can save developers hours each day and protect them from having failed code go live. When code does fail, or when a developer needs to see the output of tests, [Jenkins][2] provides log files for review.
|
|
||||||
|
|
||||||
The default Jenkins pipeline logs can be difficult to read. This quick summary of Jenkins logging basics offers some tips (and code) on how to make them more readable.
|
|
||||||
|
|
||||||
### What you get
|
|
||||||
|
|
||||||
Jenkins pipelines are split into [stages][3]. Jenkins automatically logs the beginning of each stage, like this:
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
[Pipeline] // stage
|
|
||||||
[Pipeline] stage (hide)
|
|
||||||
[Pipeline] { (Apply all openshift resources)
|
|
||||||
[Pipeline] dir
|
|
||||||
```
|
|
||||||
|
|
||||||
The text is displayed without much contrast, and important things (like the beginning of a stage) aren't highlighted. In a pipeline log several hundred lines long, finding where one stage starts and another ends, especially if you're casually browsing the logs looking for a particular stage, can be daunting.
|
|
||||||
|
|
||||||
Jenkins pipelines are written as a mix of [Groovy][4] and shell scripting. In the Groovy code, logging is sparse; many times, it consists of grayed-out text in the command without details. In the shell scripts, debugging mode (`set -x`) is turned on, so every shell command is fully realized (variables are dereferenced and values printed) and logged in detail, as is the output.
|
|
||||||
|
|
||||||
It can be tedious to read through the logs to get relevant information, given that there can be so much. Since the Groovy logs that proceed and follow a shell script in a pipeline aren't very expressive, many times they lack context:
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
[Pipeline] dir
|
|
||||||
Running in /home/jenkins/agent/workspace/devop-master/devops-server-pipeline/my-repo-dir/src
|
|
||||||
[Pipeline] { (hide)
|
|
||||||
[Pipeline] findFiles
|
|
||||||
[Pipeline] findFiles
|
|
||||||
[Pipeline] readYaml
|
|
||||||
[Pipeline] }
|
|
||||||
```
|
|
||||||
|
|
||||||
I can see what directory I am working in, and I know I was searching for file(s) and reading a YAML file using Jenkins' steps. But what was I looking for, and what did I find and read?
|
|
||||||
|
|
||||||
### What can be done?
|
|
||||||
|
|
||||||
I'm glad you asked because there are a few simple practices and some small snippets of code that can help. First, the code:
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
def echoBanner(def ... msgs) {
|
|
||||||
echo createBanner(msgs)
|
|
||||||
}
|
|
||||||
|
|
||||||
def errorBanner(def ... msgs) {
|
|
||||||
error(createBanner(msgs))
|
|
||||||
}
|
|
||||||
|
|
||||||
def createBanner(def ... msgs) {
|
|
||||||
return """
|
|
||||||
===========================================
|
|
||||||
|
|
||||||
${msgFlatten(null, msgs).join("\n ")}
|
|
||||||
|
|
||||||
===========================================
|
|
||||||
"""
|
|
||||||
}
|
|
||||||
|
|
||||||
// flatten function hack included in case Jenkins security
|
|
||||||
// is set to preclude calling Groovy flatten() static method
|
|
||||||
// NOTE: works well on all nested collections except a Map
|
|
||||||
def msgFlatten(def list, def msgs) {
|
|
||||||
list = list ?: []
|
|
||||||
if (!(msgs instanceof String) && !(msgs instanceof GString)) {
|
|
||||||
msgs.each { msg ->
|
|
||||||
list = msgFlatten(list, msg)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
list += msgs
|
|
||||||
}
|
|
||||||
|
|
||||||
return list
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Add this code to the end of each pipeline or, to be more efficient, [load a Groovy file][5] or make it part of a [Jenkins shared library][6].
|
|
||||||
|
|
||||||
At the start of each stage (or at particular points within a stage), simply call `echoBanner`:
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
`echoBanner("MY STAGE", ["DOING SOMETHING 1", "DOING SOMETHING 2"])`
|
|
||||||
```
|
|
||||||
|
|
||||||
Your logs in Jenkins will display the following:
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
===========================================
|
|
||||||
|
|
||||||
MY STAGE
|
|
||||||
DOING SOMETHING 1
|
|
||||||
DOING SOMETHING 2
|
|
||||||
|
|
||||||
===========================================
|
|
||||||
```
|
|
||||||
|
|
||||||
The banners are very easy to pick out in the logs. They also help define the pipeline flow when used properly, and they break the logs up nicely for reading.
|
|
||||||
|
|
||||||
I have used this for a while now professionally in a few places. The feedback has been very positive regarding helping make pipeline logs more readable and the flow more understandable.
|
|
||||||
|
|
||||||
The `errorBanner` method above works the same way, but it fails the script immediately. This helps highlight where and what caused the failure.
|
|
||||||
|
|
||||||
### Best practices
|
|
||||||
|
|
||||||
1. Use `echo` Jenkins steps liberally throughout your Groovy code to inform the user what you're doing. These can also help with documenting your code.
|
|
||||||
2. Use empty log statements (an empty echo step in Groovy, `echo ''`, or just `echo` in shell) to break up the output for easier readability. You probably use empty lines in your code for the same purpose.
|
|
||||||
3. Avoid the trap of using `set +x` in your scripts, which hides logging executed shell statements. It doesn't so much clean up your logs as it makes your pipelines a black box that hides what your pipeline is doing and any errors that appear. Make sure your pipelines' functionality is as transparent as possible.
|
|
||||||
4. If your pipeline creates intermediate artifacts that developers and/or DevOps personnel could use to help debug issues, then log their contents, too. Yes, it makes the logs longer, but it's only text. It will be useful information at some point, and what else is a log (if utilized properly) than a wealth of information about what happened and why?
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### Kubernetes Secrets: Where full transparency won't work
|
|
||||||
|
|
||||||
There are some things that you _don't_ want to end up in your logs and be exposed. If you're using Kubernetes and referencing data held in a Kubernetes Secret, then you definitely don't want that data exposed in a log because the data is only obfuscated and not encrypted.
|
|
||||||
|
|
||||||
Imagine you want to take some data held in a Secret and inject it into a templated JSON file. (The full contents of the Secret and the JSON template are irrelevant for this example.) You want to be transparent and log what you're doing since that's best practice, but you don't want to expose your Secret data.
|
|
||||||
|
|
||||||
Change your script's mode from debugging (`set -x`) to command logging (`set -v`). At the end of the sensitive portion of the script, reset the shell to debugging mode:
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
sh """
|
|
||||||
# change script mode from debugging to command logging
|
|
||||||
set +x -v
|
|
||||||
|
|
||||||
# capture data from secret in shell variable
|
|
||||||
MY_SECRET=\$(kubectl get secret my-secret --no-headers -o 'custom-column=:.data.my-secret-data')
|
|
||||||
|
|
||||||
# replace template placeholder inline
|
|
||||||
sed s/%TEMPLATE_PARAM%/${MY_SECRET_DATA}/ my-template-file.json
|
|
||||||
|
|
||||||
# do something with modified template-file.json...
|
|
||||||
|
|
||||||
# reset the shell to debugging mode
|
|
||||||
set -x +v
|
|
||||||
"""
|
|
||||||
```
|
|
||||||
|
|
||||||
This will output this line to the logs:
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
`sed s/%TEMPLATE_PARAM%/${MY_SECRET_DATA}/ my-template-file.json`
|
|
||||||
```
|
|
||||||
|
|
||||||
This doesn't realize the shell variable `MY_SECRET_DATA`, unlike in shell debug mode. Obviously, this isn't as helpful as debug mode if a problem occurs at this point in the pipeline and you're trying to figure out what went wrong. But it's the best balance between keeping your pipeline execution transparent for both developers and DevOps while also keeping your Secrets hidden.
|
|
||||||
|
|
||||||
--------------------------------------------------------------------------------
|
|
||||||
|
|
||||||
via: https://opensource.com/article/21/5/jenkins-logs
|
|
||||||
|
|
||||||
作者：[Evan "Hippy" Slatis][a]
|
|
||||||
选题：[lujun9972][b]
|
|
||||||
译者：[译者ID](https://github.com/译者ID)
|
|
||||||
校对：[校对者ID](https://github.com/校对者ID)
|
|
||||||
|
|
||||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
|
||||||
|
|
||||||
[a]: https://opensource.com/users/hippyod
|
|
||||||
[b]: https://github.com/lujun9972
|
|
||||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/laptop_screen_desk_work_chat_text.png?itok=UXqIDRDD (Person using a laptop)
|
|
||||||
[2]: https://www.jenkins.io/
|
|
||||||
[3]: https://www.jenkins.io/doc/book/pipeline/syntax/#stage
|
|
||||||
[4]: https://opensource.com/article/20/12/groovy
|
|
||||||
[5]: https://www.jenkins.io/doc/pipeline/steps/workflow-cps/#load-evaluate-a-groovy-source-file-into-the-pipeline-script
|
|
||||||
[6]: https://www.jenkins.io/doc/book/pipeline/shared-libraries/
|
|
||||||
183
translated/tech/20210510 Make Jenkins logs pretty.md
Normal file
183
translated/tech/20210510 Make Jenkins logs pretty.md
Normal file
@ -0,0 +1,183 @@
|
|||||||
|
[#]: subject: (Make Jenkins logs pretty)
|
||||||
|
[#]: via: (https://opensource.com/article/21/5/jenkins-logs)
|
||||||
|
[#]: author: (Evan "Hippy" Slatis https://opensource.com/users/hippyod)
|
||||||
|
[#]: collector: (lujun9972)
|
||||||
|
[#]: translator: (DCOLIVERSUN)
|
||||||
|
[#]: reviewer: ( )
|
||||||
|
[#]: publisher: ( )
|
||||||
|
[#]: url: ( )
|
||||||
|
|
||||||
|
美化 Jenkins 日志
|
||||||
|
======
|
||||||
|
Jenkins 默认日志难以阅读，但日志本身不必如此。
|
||||||
|
![用笔记本电脑的人][1]
|
||||||
|
|
||||||
|
Jenkins 是一个免费的开源自动化服务器，用于构建、测试和部署代码。它是<ruby>持续集成<rt>Continuous Integration, CI</rt></ruby>>、<ruby>持续交付<rt>Continuous Delivery, CD</rt></ruby>的基础，每天可以为开发人员节省时间，防止他们上线错误的代码。一旦代码失效或开发人员需要查看测试输出时，[Jenkins][2] 将提供日志文件以供检查。
|
||||||
|
|
||||||
|
默认的 Jenkins <ruby>管道<rt>Pipeline</rt></ruby>日志难以阅读。Jenkins 日志的摘要提供了一些技巧（和代码），可以提升它们的可读性。
|
||||||
|
|
||||||
|
### 你获得什么
|
||||||
|
|
||||||
|
Jenkins 管道分为[多个阶段][3]。Jenkins 自动在每个阶段初自动记录，记录内容如下：
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
[Pipeline] // stage
|
||||||
|
[Pipeline] stage (hide)
|
||||||
|
[Pipeline] { (Apply all openshift resources)
|
||||||
|
[Pipeline] dir
|
||||||
|
```
|
||||||
|
|
||||||
|
上文显示的内容没有太大区分度，重要的内容（如阶段的开始）未突出显示。在多达数百行的管道日志中，找到阶段的起始、终止位置可能会很艰巨。当随意浏览日志寻找特定阶段的时候，这种艰巨尤其明显。
|
||||||
|
|
||||||
|
Jenkins 管道是 [Groovy][4] 和 Shell 脚本混合编写的。在 Groovy 代码中，日志记录很少。很多时候，日志是由命令中的灰色文本组成，没有详细信息。在 Shell 脚本中，打开调试模式（`set -x`），打印每条 Shell 命令与结果，以输出形式详细记录在日志中。
|
||||||
|
|
||||||
|
鉴于可能有很多内容，通读日志获取相关信息可能是很繁琐的。在管道中执行、遵循 Shell 脚本的 Groovy 日志可读性差，它们很多时候缺少上下文：
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
[Pipeline] dir
|
||||||
|
Running in /home/jenkins/agent/workspace/devop-master/devops-server-pipeline/my-repo-dir/src
|
||||||
|
[Pipeline] { (hide)
|
||||||
|
[Pipeline] findFiles
|
||||||
|
[Pipeline] findFiles
|
||||||
|
[Pipeline] readYaml
|
||||||
|
[Pipeline] }
|
||||||
|
```
|
||||||
|
|
||||||
|
我可以知道我正在使用的目录，并且知道我正在搜索文件、使用 Jenkins 的步骤读取 YAML 文件。但是我在寻找什么？我读取的内容是什么？
|
||||||
|
|
||||||
|
### 能做什么？
|
||||||
|
|
||||||
|
我很高兴你被问问倒，因为这里有一些简单的实现和一些小的代码片段可以提供帮助。首先，代码如下：
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
def echoBanner(def ... msgs) {
|
||||||
|
echo createBanner(msgs)
|
||||||
|
}
|
||||||
|
|
||||||
|
def errorBanner(def ... msgs) {
|
||||||
|
error(createBanner(msgs))
|
||||||
|
}
|
||||||
|
|
||||||
|
def createBanner(def ... msgs) {
|
||||||
|
return """
|
||||||
|
===========================================
|
||||||
|
|
||||||
|
${msgFlatten(null, msgs).join("\n ")}
|
||||||
|
|
||||||
|
===========================================
|
||||||
|
"""
|
||||||
|
}
|
||||||
|
|
||||||
|
// flatten function hack included in case Jenkins security
|
||||||
|
// is set to preclude calling Groovy flatten() static method
|
||||||
|
// NOTE: works well on all nested collections except a Map
|
||||||
|
def msgFlatten(def list, def msgs) {
|
||||||
|
list = list ?: []
|
||||||
|
if (!(msgs instanceof String) && !(msgs instanceof GString)) {
|
||||||
|
msgs.each { msg ->
|
||||||
|
list = msgFlatten(list, msg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
list += msgs
|
||||||
|
}
|
||||||
|
|
||||||
|
return list
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
将这段代码添加到每个管道的末尾，也可以[加载 Groovy 文件][5]或者使其成为 [Jenkins 共享库][6] 的一部分，这样更有效。
|
||||||
|
|
||||||
|
在每个阶段起始处（或者在阶段中指定位置），只需调用 `echoBanner`：
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
`echoBanner("MY STAGE", ["DOING SOMETHING 1", "DOING SOMETHING 2"])`
|
||||||
|
```
|
||||||
|
|
||||||
|
你的 Jenkins 日志会如下展示：
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
===========================================
|
||||||
|
|
||||||
|
MY STAGE
|
||||||
|
DOING SOMETHING 1
|
||||||
|
DOING SOMETHING 2
|
||||||
|
|
||||||
|
===========================================
|
||||||
|
```
|
||||||
|
|
||||||
|
标志很容易从日志中分辨出来。当正确使用它们时，它们还有助于定义管道流，并且可以很好得分解日志进行读取。
|
||||||
|
|
||||||
|
我已经在某些地方运行这段代码一些时间了。反馈非常积极，可以提升管道日志可读性，更容易理解管道流。
|
||||||
|
|
||||||
|
上述的 `errorBanner` 方法以相同的方式工作，但是它会立即使脚本失效。这有助于突显故障的位置与原因。
|
||||||
|
|
||||||
|
### 最佳实践
|
||||||
|
|
||||||
|
1. 在 Groovy 代码中自由地使用 `echo` Jenkins 步骤来通知用户你的行为。这些也可以帮助记录你的代码。
|
||||||
|
2. 使用空的日志语句（Groovy 中空 echo 步骤、`echo ''`或着 Shell 中的 `echo`）打断输出，提高可读性。你可以出于相同的目的在代码中使用空行。
|
||||||
|
3. 避免在脚本中使用 `set +x` 的陷阱，因为这会影响日志记录已执行的 Shell 语句。它不会过多清理你的日志，因为它使你的管道成为一个黑盒子，该黑盒子会隐藏管道正在做的行为以及出现的任何错误。确保管道功能尽可能透明。
|
||||||
|
4. 如果你的管道创建了<ruby>中间组件<rt>Intermediate Artifacts</rt></ruby>，开发人员和 DevOps 人员可以使用这些组件来帮助调试问题，那么也要记录内容。是的，它会加长日志，但这只是文本。在某些时候，这会是有用的信息，如果使用得当的话，除了大量有关发生的事情以及原因的信息之外，日志还有什么？
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Kubernetes Secret：无法完全透明的地方
|
||||||
|
|
||||||
|
有些事情你不希望出现在日志里暴露出来。如果你在使用 Kubernetes 并引用保存在 Kubernetes Secret 中的数据，那么你绝对不希望在日志中公开该数据，因为这些数据只会被混淆，而不会被加密。
|
||||||
|
|
||||||
|
加入你想获取一些保存在 Secret 中的数据，然后将其注入模板化 JSON 文件中。（此例与 Secret 和 JSON 模板的完整内容无关。）由于这是最佳做法，你希望保持透明并记录你的操作，但你不想公开 Secret 数据。
|
||||||
|
|
||||||
|
将脚本模式从调试（`set -x`）更改为命令日志记录(`set -v`)。在脚本易错部分结尾，将 Shell 重置为调试模式：
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
sh """
|
||||||
|
# change script mode from debugging to command logging
|
||||||
|
set +x -v
|
||||||
|
|
||||||
|
# capture data from secret in shell variable
|
||||||
|
MY_SECRET=\$(kubectl get secret my-secret --no-headers -o 'custom-column=:.data.my-secret-data')
|
||||||
|
|
||||||
|
# replace template placeholder inline
|
||||||
|
sed s/%TEMPLATE_PARAM%/${MY_SECRET_DATA}/ my-template-file.json
|
||||||
|
|
||||||
|
# do something with modified template-file.json...
|
||||||
|
|
||||||
|
# reset the shell to debugging mode
|
||||||
|
set -x +v
|
||||||
|
"""
|
||||||
|
```
|
||||||
|
|
||||||
|
这将输出此行到日志：
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
`sed s/%TEMPLATE_PARAM%/${MY_SECRET_DATA}/ my-template-file.json`
|
||||||
|
```
|
||||||
|
|
||||||
|
与 Shell 调试模式中不同，这不会实现 Shell 变量 `MY_SECRET_DATA`。显然，如果管道中在这一点出现问题，你试图找出问题出在哪里，那么这不如调试模式有用。但这是开发人员和 DevOps 人员保持管道透明性和 Secret 数据隐蔽性之间平衡的最佳做法。
|
||||||
|
|
||||||
|
--------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
via: https://opensource.com/article/21/5/jenkins-logs
|
||||||
|
|
||||||
|
作者：[Evan "Hippy" Slatis][a]
|
||||||
|
选题：[lujun9972][b]
|
||||||
|
译者：[DCOLIVERSUN](https://github.com/DCOLIVERSUN)
|
||||||
|
校对：[校对者ID](https://github.com/校对者ID)
|
||||||
|
|
||||||
|
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||||
|
|
||||||
|
[a]: https://opensource.com/users/hippyod
|
||||||
|
[b]: https://github.com/lujun9972
|
||||||
|
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/laptop_screen_desk_work_chat_text.png?itok=UXqIDRDD (Person using a laptop)
|
||||||
|
[2]: https://www.jenkins.io/
|
||||||
|
[3]: https://www.jenkins.io/doc/book/pipeline/syntax/#stage
|
||||||
|
[4]: https://opensource.com/article/20/12/groovy
|
||||||
|
[5]: https://www.jenkins.io/doc/pipeline/steps/workflow-cps/#load-evaluate-a-groovy-source-file-into-the-pipeline-script
|
||||||
|
[6]: https://www.jenkins.io/doc/book/pipeline/shared-libraries/
|
||||||
Loading…
Reference in New Issue
Block a user