document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2025-01-31 23:30:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'LCTT/master'

Browse Source
This commit is contained in:
Xingyu Wang 2019-07-31 21:19:21 +08:00
commit e6479fb0c1
4 changed files with 205 additions and 225 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

109
published/20190705 Bash Script to Monitor Messages Log (Warning, Error and Critical) on Linux.md Normal file
View File

@ -0,0 +1,109 @@
[#]: collector: (lujun9972)
[#]: translator: (wxy)
[#]: reviewer: (wxy)
[#]: publisher: (wxy)
[#]: url: (https://linux.cn/article-11171-1.html)
[#]: subject: (Bash Script to Monitor Messages Log (Warning, Error and Critical) on Linux)
[#]: via: (https://www.2daygeek.com/linux-bash-script-to-monitor-messages-log-warning-error-critical-send-email/)
[#]: author: (Magesh Maruthamuthu https://www.2daygeek.com/author/magesh/)
在 Linux 上用 Bash 脚本监控 messages 日志
======
目前市场上有许多开源监控工具可用于监控 Linux 系统的性能。当系统达到指定的阈值限制时，它将发送电子邮件警报。它可以监视 CPU 利用率、内存利用率、交换利用率、磁盘空间利用率等所有内容。
如果你只有很少的系统并且想要监视它们，那么编写一个小的 shell 脚本可以使你的任务变得非常简单。
在本教程中，我们添加了一个 shell 脚本来监视 Linux 系统上的 messages 日志。
我们过去添加了许多有用的 shell 脚本。如果要查看这些内容，请导航至以下链接。
- [如何使用 shell 脚本监控系统的日常活动？][1]
此脚本将检查 `/var/log/messages` 文件中的 “warning“、“error” 和 “critical”，如果发现任何有关的东西，就给指定电子邮件地址发邮件。
如果服务器有许多匹配的字符串，我们就不能经常运行这个可能填满收件箱的脚本，我们可以在一天内运行一次。
为了解决这个问题，我让脚本以不同的方式触发电子邮件。
如果 `/var/log/messages` 文件中昨天的日志中找到任何给定字符串，则脚本将向给定的电子邮件地址发送电子邮件警报。
**注意：**你需要更改电子邮件地址，而不是我们的电子邮件地址。
```
# vi /opt/scripts/os-log-alert.sh
```
```
#!/bin/bash
#Set the variable which equal to zero
prev_count=0
count=$(grep -i "`date --date='yesterday' '+%b %e'`" /var/log/messages | egrep -wi 'warning|error|critical' | wc -l)
if [ "$prev_count" -lt "$count" ] ; then
# Send a mail to given email id when errors found in log
SUBJECT="WARNING: Errors found in log on "`date --date='yesterday' '+%b %e'`""
# This is a temp file, which is created to store the email message.
MESSAGE="/tmp/logs.txt"
TO="2daygeek@gmail.com"
echo "ATTENTION: Errors are found in /var/log/messages. Please Check with Linux admin." >> $MESSAGE
echo "Hostname: `hostname`" >> $MESSAGE
echo -e "\n" >> $MESSAGE
echo "+------------------------------------------------------------------------------------+" >> $MESSAGE
echo "Error messages in the log file as below" >> $MESSAGE
echo "+------------------------------------------------------------------------------------+" >> $MESSAGE
grep -i "`date --date='yesterday' '+%b %e'`" /var/log/messages | awk '{ $3=""; print}' | egrep -wi 'warning|error|critical' >> $MESSAGE
mail -s "$SUBJECT" "$TO" < $MESSAGE
#rm $MESSAGE
fi
```
为 `os-log-alert.sh` 文件设置可执行权限。
```
$ chmod +x /opt/scripts/os-log-alert.sh
```
最后添加一个 cron 任务来自动执行此操作。它将每天 7 点钟运行。
```
# crontab -e
```
```
0 7 * * * /bin/bash /opt/scripts/os-log-alert.sh
```
**注意：**你将在每天 7 点收到昨天日志的电子邮件提醒。
**输出：**你将收到类似下面的电子邮件提醒。
```
ATTENTION: Errors are found in /var/log/messages. Please Check with Linux admin.
+-----------------------------------------------------+
Error messages in the log file as below
+-----------------------------------------------------+
Jul 3 02:40:11 ns1 kernel: php-fpm[3175]: segfault at 299 ip 000055dfe7cc7e25 sp 00007ffd799d7d38 error 4 in php-fpm[55dfe7a89000+3a7000]
Jul 3 02:50:14 ns1 kernel: lmtp[8249]: segfault at 20 ip 00007f9cc05295e4 sp 00007ffc57bca1a0 error 4 in libdovecot-storage.so.0.0.0[7f9cc04df000+148000]
Jul 3 15:36:09 ns1 kernel: php-fpm[17846]: segfault at 299 ip 000055dfe7cc7e25 sp 00007ffd799d7d38 error 4 in php-fpm[55dfe7a89000+3a7000]
Jul 3 15:45:54 ns1 pure-ftpd: (?@5.188.62.5) [WARNING] Authentication failed for user [daygeek]
Jul 3 16:25:36 ns1 pure-ftpd: (?@104.140.148.58) [WARNING] Sorry, cleartext sessions and weak ciphers are not accepted on this server.#012Please reconnect using TLS security mechanisms.
Jul 3 16:44:20 ns1 kernel: php-fpm[8979]: segfault at 299 ip 000055dfe7cc7e25 sp 00007ffd799d7d38 error 4 in php-fpm[55dfe7a89000+3a7000]
```
--------------------------------------------------------------------------------
via: https://www.2daygeek.com/linux-bash-script-to-monitor-messages-log-warning-error-critical-send-email/
作者：[Magesh Maruthamuthu][a]
选题：[lujun9972][b]
译者：[wxy](https://github.com/wxy)
校对：[wxy](https://github.com/wxy)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.2daygeek.com/author/magesh/
[b]: https://github.com/lujun9972
[1]: https://www.2daygeek.com/category/shell-script/

98
sources/tech/20190613 Continuous integration testing for the Linux kernel.md
View File

@ -1,98 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: (LazyWolfLin)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Continuous integration testing for the Linux kernel)
[#]: via: (https://opensource.com/article/19/6/continuous-kernel-integration-linux)
[#]: author: (Major Hayden https://opensource.com/users/mhayden)
Continuous integration testing for the Linux kernel
======
How this team works to prevent bugs from being merged into the Linux
kernel.
![Linux kernel source code \(C\) in Visual Studio Code][1]
With 14,000 changesets per release from over 1,700 different developers, it's clear that the Linux kernel moves quickly, and brings plenty of complexity. Kernel bugs range from small annoyances to larger problems, such as system crashes and data loss.
As the call for continuous integration (CI) grows for more and more projects, the [Continuous Kernel Integration (CKI)][2] team forges ahead with a single mission: prevent bugs from being merged into the kernel.
### Linux testing problems
Many Linux distributions test the Linux kernel when needed. This testing often occurs around release time, or when users find a bug.
Unrelated issues sometimes appear, and maintainers scramble to find which patch in a changeset full of tens of thousands of patches caused the new, unrelated bug. Diagnosing the bug may require specialized hardware, a series of triggers, and specialized knowledge of that portion of the kernel.
#### CI and Linux
Most modern software repositories have some sort of automated CI testing that tests commits before they find their way into the repository. This automated testing allows the maintainers to find software quality issues, along with most bugs, by reviewing the CI report. Simpler projects, such as a Python library, come with tons of tools to make this process easier.
Linux must be configured and compiled prior to any testing. Doing so takes time and compute resources. In addition, that kernel must boot in a virtual machine or on a bare metal machine for testing. Getting access to certain system architectures requires additional expense or very slow emulation. From there, someone must identify a set of tests which trigger the bug or verify the fix.
#### How the CKI team works
The CKI team at Red Hat currently follows changes from several internal kernels, as well as upstream kernels such as the [stable kernel tree][3]. We watch for two critical events in each repository:
1. When maintainers merge pull requests or patches, and the resulting commits in the repository change.
2. When developers propose changes for merging via patchwork or the stable patch queue.
As these events occur, automation springs into action and [GitLab CI pipelines][4] begin the testing process. Once the pipeline runs [linting][5] scripts, merges any patches, and compiles the kernel for multiple architectures, the real testing begins. We compile kernels in under six minutes for four architectures and submit feedback to the stable mailing list usually in two hours or less. Over 100,000 kernel tests run each month and over 11,000 GitLab pipelines have completed (since January 2019).
Each kernel is booted on its native architecture, which includes:
● [aarch64][6]: 64-bit [ARM][7], such as the [Cavium (now Marvell) ThunderX][8].
● [ppc64/ppc64le][9]: Big and little endian [IBM POWER][10] systems.
● [s390x][11]: [IBM Zseries][12] mainframes.
● [x86_64][13]: [Intel][14] and [AMD][15] workstations, laptops, and servers.
Multiple tests run on these kernels, including the [Linux Test Project (LTP)][16], which contains a myriad of tests using a common test harness. My CKI team open-sourced over 44 tests with more on the way.
### Get involved
The upstream kernel testing effort grows day-by-day. Many companies provide test output for various kernels, including [Google][17], Intel, [Linaro][18], and [Sony][19]. Each effort is focused on bringing value to the upstream kernel as well as each company’s customer base.
If you or your company want to join the effort, please come to the [Linux Plumbers Conference 2019][20] in Lisbon, Portugal. Join us at the Kernel CI hackfest during the two days after the conference, and drive the future of rapid kernel testing.
For more details, [review the slides][21] from my Texas Linux Fest 2019 talk.
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/6/continuous-kernel-integration-linux
作者：[Major Hayden][a]
选题：[lujun9972][b]
译者：[LazyWolfLin](https://github.com/LazyWolfLin)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/mhayden
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/linux_kernel_clang_vscode.jpg?itok=fozZ4zrr (Linux kernel source code (C) in Visual Studio Code)
[2]: https://cki-project.org/
[3]: https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
[4]: https://docs.gitlab.com/ee/ci/pipelines.html
[5]: https://en.wikipedia.org/wiki/Lint_(software)
[6]: https://en.wikipedia.org/wiki/ARM_architecture
[7]: https://www.arm.com/
[8]: https://www.marvell.com/server-processors/thunderx-arm-processors/
[9]: https://en.wikipedia.org/wiki/Ppc64
[10]: https://www.ibm.com/it-infrastructure/power
[11]: https://en.wikipedia.org/wiki/Linux_on_z_Systems
[12]: https://www.ibm.com/it-infrastructure/z
[13]: https://en.wikipedia.org/wiki/X86-64
[14]: https://www.intel.com/
[15]: https://www.amd.com/
[16]: https://github.com/linux-test-project/ltp
[17]: https://www.google.com/
[18]: https://www.linaro.org/
[19]: https://www.sony.com/
[20]: https://www.linuxplumbersconf.org/
[21]: https://docs.google.com/presentation/d/1T0JaRA0wtDU0aTWTyASwwy_ugtzjUcw_ZDmC5KFzw-A/edit?usp=sharing

127
sources/tech/20190705 Bash Script to Monitor Messages Log (Warning, Error and Critical) on Linux.md
View File

@ -1,127 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Bash Script to Monitor Messages Log (Warning, Error and Critical) on Linux)
[#]: via: (https://www.2daygeek.com/linux-bash-script-to-monitor-messages-log-warning-error-critical-send-email/)
[#]: author: (Magesh Maruthamuthu https://www.2daygeek.com/author/magesh/)
Bash Script to Monitor Messages Log (Warning, Error and Critical) on Linux
======
There are many open source monitoring tools are currently available in market to monitor Linux systems performance.
It will send an email alert when the system reaches the specified threshold limit.
It monitors everything such as CPU utilization, Memory utilization, swap utilization, disk space utilization and much more.
If you only have few systems and want to monitor them then writing a small shell script can make your task very easy.
In this tutorial we have added a shell script to monitor Messages Log on Linux system.
We had added many useful shell scripts in the past. If you want to check those, navigate to the below link.
* **[How to automate day to day activities using shell scripts?][1]**
This script will check **“warning, error and critical”** in the `/var/log/messages` file and trigger a mail to given email id, if it’s found anything related it.
We can’t run this script frequently that may fill up your inbox if the server has many matching strings, instead we can run once in a day.
To overcome this issue, i made the script to trigger an email in a different manner.
If any given strings are found in the **“/var/log/messages”** file for yesterday’s date then the script will send an email alert to given email id.
**Note:** You need to change the email id instead of ours. Also, you can change the Memory utilization threshold value as per your requirement.
```
# vi /opt/scripts/os-log-alert.sh
#!/bin/bash
#Set the variable which equal to zero
prev_count=0
count=$(grep -i "`date --date='yesterday' '+%b %e'`" /var/log/messages | egrep -wi 'warning|error|critical' | wc -l)
if [ "$prev_count" -lt "$count" ] ; then
# Send a mail to given email id when errors found in log
SUBJECT="WARNING: Errors found in log on "`date --date='yesterday' '+%b %e'`""
# This is a temp file, which is created to store the email message.
MESSAGE="/tmp/logs.txt"
TO="[email protected]"
echo "ATTENTION: Errors are found in /var/log/messages. Please Check with Linux admin." >> $MESSAGE
echo "Hostname: `hostname`" >> $MESSAGE
echo -e "\n" >> $MESSAGE
echo "+------------------------------------------------------------------------------------+" >> $MESSAGE
echo "Error messages in the log file as below" >> $MESSAGE
echo "+------------------------------------------------------------------------------------+" >> $MESSAGE
grep -i "`date --date='yesterday' '+%b %e'`" /var/log/messages | awk '{ $3=""; print}' | egrep -wi 'warning|error|critical' >> $MESSAGE
mail -s "$SUBJECT" "$TO" < $MESSAGE
#rm $MESSAGE
fi
```
Set an executable permission to `os-log-alert.sh` file.
```
$ chmod +x /opt/scripts/os-log-alert.sh
```
Finally add a cronjob to automate this. It will run everyday at 7'o clock.
```
# crontab -e
0 7 * * * /bin/bash /opt/scripts/os-log-alert.sh
```
**Note:** You will be getting an email alert everyday at 7 o'clock, which is for yesterday's log.
**Output:** You will be getting an email alert similar to below.
```
ATTENTION: Errors are found in /var/log/messages. Please Check with Linux admin.
+-----------------------------------------------------+
Error messages in the log file as below
+-----------------------------------------------------+
Jul 3 02:40:11 ns1 kernel: php-fpm[3175]: segfault at 299 ip 000055dfe7cc7e25 sp 00007ffd799d7d38 error 4 in php-fpm[55dfe7a89000+3a7000]
Jul 3 02:50:14 ns1 kernel: lmtp[8249]: segfault at 20 ip 00007f9cc05295e4 sp 00007ffc57bca1a0 error 4 in libdovecot-storage.so.0.0.0[7f9cc04df000+148000]
Jul 3 15:36:09 ns1 kernel: php-fpm[17846]: segfault at 299 ip 000055dfe7cc7e25 sp 00007ffd799d7d38 error 4 in php-fpm[55dfe7a89000+3a7000]
Jul 3 15:45:54 ns1 pure-ftpd: ([email protected]) [WARNING] Authentication failed for user [daygeek]
Jul 3 16:25:36 ns1 pure-ftpd: ([email protected]) [WARNING] Sorry, cleartext sessions and weak ciphers are not accepted on this server.#012Please reconnect using TLS security mechanisms.
Jul 3 16:44:20 ns1 kernel: php-fpm[8979]: segfault at 299 ip 000055dfe7cc7e25 sp 00007ffd799d7d38 error 4 in php-fpm[55dfe7a89000+3a7000]
```
--------------------------------------------------------------------------------
via: https://www.2daygeek.com/linux-bash-script-to-monitor-messages-log-warning-error-critical-send-email/
作者：[Magesh Maruthamuthu][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.2daygeek.com/author/magesh/
[b]: https://github.com/lujun9972
[1]: https://www.2daygeek.com/category/shell-script/

96
translated/tech/20190613 Continuous integration testing for the Linux kernel.md Normal file
View File

@ -0,0 +1,96 @@
[#]: collector: (lujun9972)
[#]: translator: (LazyWolfLin)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Continuous integration testing for the Linux kernel)
[#]: via: (https://opensource.com/article/19/6/continuous-kernel-integration-linux)
[#]: author: (Major Hayden https://opensource.com/users/mhayden)
Linux 内核的持续集成测试
======
> 这个团队是如何防止 bug 被合并到 Linux 内核中。
![Linux kernel source code \(C\) in Visual Studio Code][1]
Linux 内核的每个发布版本包含了来自 1,700 个开发者产生的 14,000 个更改，很显然，这使得 Linux 内核快速迭代的同时也产生了巨大的复杂性问题。内核上 Bug 有小麻烦也有大问题，例如系统奔溃和数据丢失。
随着越来越多的项目对于持续集成（CI）的呼声，[内核持续集成（CKI）][2]小组秉承着一个任务目标：防止 Bug 被合并到内核当中。
### Linux 测试问题
许多 Linux 发行版只在需要的时候对 Linux 内核进行测试。而这种测试往往只在版本发布时或者用户发现错误时进行。
有时候，出现玄学问题时，维护人员需要在包含了数万个补丁的变更中匆忙地寻找哪个补丁导致这个新的玄学 Bug。诊断 Bug 需要专业的硬件设备、一系列的触发器以及内核相关的专业知识。
#### CI 和 Linux
许多现代软件代码库都采用某种自动化 CI 测试机制，能够在提交进入代码存储库之前对其进行测试。这种自动化测试使得维护人员可以通过查看 CI 测试报告来发现软件质量问题以及大多数的错误。一些更方便的项目，比如某个 python 库，附带的大量工具使得整个检查过程更简单。
在任何测试之前都需要配置和编译 Linux。而这么做将耗费大量的时间和计算资源。此外，Linux 内核必需在虚拟机或者裸机上启动才能进行测试。而访问某些硬件架构需要额外的开销或者非常慢的仿真。因此，必需有人确定一组能够触发错误或者验证修复的测试集。
#### CKI 团队如何运作？
Red Hat 公司的 CKI 团队当前正追踪来自数个内部内核分支和上游的[稳定内核分支树][3]等内核分支的更改。我们关注每个代码库的两类关键事件：
1. 当维护人员合并 PR 或者补丁时，代码库变化后的最终结果。
2. 当开发人员通过拼凑的或者稳定的补丁队列发起合并时。
当这些事件发生时，自动化工具开始执行，[GitLab CI 管道][4]开始进行测试。一旦管道开始执行 [linting][5] 脚本，合并每一个补丁，并为多种硬件架构编译内核，真正的测试便开始了。我们会在六分钟内完成四种硬件架构的内核编译工作，并且通常会在两个小时或更短的时间内将反馈提交到稳定邮件列表中。（自 2019 年 1 月起）每月执行超过 100,000 次内核测试，并有超过 11,000 个 GitLab 管道已经完成。
每个内核都会在本地硬件架构上启动，其中包含：
* [aarch64][6]：64-bit [ARM][7], 例如 [Cavium（当前是 Marvell）ThunderX][8]。
* [ppc64/ppc64le][9]：大端和小端的 [IBM POWER][10] 系统。
* [s390x][11]：[IBM Zseries][12] 大型机
* [x86_64][13]：[Intel][14] 和 [AMD][15] 工作站，笔记本和服务器。
这些内核上运行了包括 [Linux 测试项目（LTP）][16]在内的多个测试，其中包括使用常用测试工具的大量测试。我们 CKI 团队开源了超过 44 个测试并将继续开源更多测试。
### 参与其中
上游的内核测试工作日渐增多。包括[Google][17]、Intel、[Linaro][18] 和 [Sony][19] 在内的许多公司为各种内核提供测试输出。每一项工作都专注于为上游内核以及每个公司的客户群带来价值。
如果你或者你的公司想要参与这一工作，请参加在葡萄牙里斯本举办的 [Linux Plumbers Conference 2019][20]。在会议结束后的两天加入我们的 Kernel CI hackfest 活动，并推动快速内核测试的发展。
更多详细信息，[请见][21]我在 Texas Linux Fest 2019 上的演讲。
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/6/continuous-kernel-integration-linux
作者：[Major Hayden][a]
选题：[lujun9972][b]
译者：[LazyWolfLin](https://github.com/LazyWolfLin)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/mhayden
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/linux_kernel_clang_vscode.jpg?itok=fozZ4zrr "Linux kernel source code (C) in Visual Studio Code"
[2]: https://cki-project.org/
[3]: https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
[4]: https://docs.gitlab.com/ee/ci/pipelines.html
[5]: https://en.wikipedia.org/wiki/Lint_(software)
[6]: https://en.wikipedia.org/wiki/ARM_architecture
[7]: https://www.arm.com/
[8]: https://www.marvell.com/server-processors/thunderx-arm-processors/
[9]: https://en.wikipedia.org/wiki/Ppc64
[10]: https://www.ibm.com/it-infrastructure/power
[11]: https://en.wikipedia.org/wiki/Linux_on_z_Systems
[12]: https://www.ibm.com/it-infrastructure/z
[13]: https://en.wikipedia.org/wiki/X86-64
[14]: https://www.intel.com/
[15]: https://www.amd.com/
[16]: https://github.com/linux-test-project/ltp
[17]: https://www.google.com/
[18]: https://www.linaro.org/
[19]: https://www.sony.com/
[20]: https://www.linuxplumbersconf.org/
[21]: https://docs.google.com/presentation/d/1T0JaRA0wtDU0aTWTyASwwy_ugtzjUcw_ZDmC5KFzw-A/edit?usp=sharing