mirror of
https://github.com/LCTT/TranslateProject.git
synced 2024-12-26 21:30:55 +08:00
TSL
This commit is contained in:
Xingyu Wang
parent
eca97baef5
commit
efe6095f3c
@ -1,78 +0,0 @@
|
|||||||
[#]: collector: (lujun9972)
|
|
||||||
[#]: translator: (wxy)
|
|
||||||
[#]: reviewer: ( )
|
|
||||||
[#]: publisher: ( )
|
|
||||||
[#]: url: ( )
|
|
||||||
[#]: subject: (The lifecycle of Linux kernel testing)
|
|
||||||
[#]: via: (https://opensource.com/article/19/8/linux-kernel-testing)
|
|
||||||
[#]: author: (Major Hayden https://opensource.com/users/mhaydenhttps://opensource.com/users/mhaydenhttps://opensource.com/users/marcobravohttps://opensource.com/users/mhayden)
|
|
||||||
|
|
||||||
The lifecycle of Linux kernel testing
|
|
||||||
======
|
|
||||||
The Continuous Kernel Integration (CKI) project aims to prevent bugs
|
|
||||||
from entering the Linux kernel.
|
|
||||||
![arrows cycle symbol for failing faster][1]
|
|
||||||
|
|
||||||
In _[Continuous integration testing for the Linux kernel][2]_, I wrote about the [Continuous Kernel Integration][3] (CKI) project and its mission to change how kernel developers and maintainers work. This article is a deep dive into some of the more technical aspects of the project and how all the pieces fit together.
|
|
||||||
|
|
||||||
### It all starts with a change
|
|
||||||
|
|
||||||
Every exciting feature, improvement, and bug in the kernel starts with a change proposed by a developer. These changes appear on myriad mailing lists for different kernel repositories. Some repositories focus on certain subsystems in the kernel, such as storage or networking, while others focus on broad aspects of the kernel. The CKI project springs into action when developers propose a change, or patchset, to the kernel or when a maintainer makes changes in the repository itself.
|
|
||||||
|
|
||||||
The CKI project maintains triggers that monitor these patchsets and take action. Software projects such as [Patchwork][4] make this process much easier by collating multi-patch contributions into a single patch series. This series travels as a unit through the CKI system and allows for publishing a single report on the series.
|
|
||||||
|
|
||||||
Other triggers watch the repository for changes. This occurs when kernel maintainers merge patchsets, revert patches, or create new tags. Testing these critical changes ensures that developers always have a solid baseline to use as a foundation for writing new patches.
|
|
||||||
|
|
||||||
All of these changes make their way into a GitLab pipeline and pass through multiple stages and multiple systems.
|
|
||||||
|
|
||||||
### Prepare the build
|
|
||||||
|
|
||||||
Everything starts with getting the source ready for compile time. This requires cloning the repository, applying the patchset proposed by the developer, and generating a kernel config file. These config files have thousands of options that turn features on or off, and config files differ incredibly between different system architectures. For example, a fairly standard x86_64 system may have a ton of options available in its config file, but an s390x system (IBM zSeries mainframes) likely has much fewer options. Some options might make sense on that mainframe but they have no purpose on a consumer laptop.
|
|
||||||
|
|
||||||
The kernel moves forward and transforms into a source artifact. The artifact contains the entire repository, with patches applied, and all kernel configuration files required for compiling. Upstream kernels move on as a tarball, while Red Hat kernels become a source RPM for the next step.
|
|
||||||
|
|
||||||
### Piles of compiles
|
|
||||||
|
|
||||||
Compiling the kernel turns the source code into something that a computer can boot up and use. The config file describes what to build, scripts in the kernel describe how to build it, and tools on the system (like GCC and glibc) do the building. This process takes a while to complete, but the CKI project needs it done quickly for four architectures: aarch64 (64-bit ARM), ppc64le (POWER), s390x (IBM zSeries), and x86_64. It's important that we compile kernels quickly so that we keep our backlog manageable and developers receive prompt feedback.
|
|
||||||
|
|
||||||
Adding more CPUs provides plenty of speed improvements, but every system has its limits. The CKI project compiles kernels within containers in an OpenShift deployment; although OpenShift allows for tons of scalability, the deployment still has a finite number of CPUs available. The CKI team allocates 20 virtual CPUs for compiling each kernel. With four architectures involved, this balloons to 80 CPUs!
|
|
||||||
|
|
||||||
Another speed increase comes from a tool called [ccache][5]. Kernel development moves quickly, but a large amount of the kernel remains unchanged even between multiple releases. The ccache tool caches the built objects (small pieces of the overall kernel) during the compile on a disk. When another kernel compile comes along later, ccache looks for unchanged pieces of the kernel that it saw before. Ccache pulls the cached object from the disk and reuses it. This allows for faster compiles and lower overall CPU usage. Kernels that took 20 minutes to compile now race to the finish line in less than a few minutes.
|
|
||||||
|
|
||||||
### Testing time
|
|
||||||
|
|
||||||
The kernel moves onto its last step: testing on real hardware. Each kernel boots up on its native architecture using Beaker, and myriad tests begin poking it to find problems. Some tests look for simple problems, such as issues with containers or error messages on boot-up. Other tests dive deep into various kernel subsystems to find regressions in system calls, memory allocation, and threading.
|
|
||||||
|
|
||||||
Large testing frameworks, such as the [Linux Test Project][6] (LTP), contain tons of tests that look for troublesome regressions in the kernel. Some of these regressions could roll back critical security fixes, and there are tests to ensure those improvements remain in the kernel.
|
|
||||||
|
|
||||||
One critical step remains when tests finish: reporting. Kernel developers and maintainers need a concise report that tells them exactly what worked, what did not work, and how to get more information. Each CKI report contains details about the source code used, the compile parameters, and the testing output. That information helps developers know where to begin looking to fix an issue. Also, it helps maintainers know when a patchset needs to be held for another look before a bug makes its way into their kernel repository.
|
|
||||||
|
|
||||||
### Summary
|
|
||||||
|
|
||||||
The CKI project team strives to prevent bugs from entering the Linux kernel by providing timely, automated feedback to kernel developers and maintainers. This work makes their job easier by finding the low-hanging fruit that leads to kernel bugs, security issues, and performance problems.
|
|
||||||
|
|
||||||
* * *
|
|
||||||
|
|
||||||
_To learn more, you can attend the [CKI Hackfest][7] on September 12-13 following the [Linux Plumbers Conference][8] September 9-11 in Lisbon, Portugal._
|
|
||||||
|
|
||||||
--------------------------------------------------------------------------------
|
|
||||||
|
|
||||||
via: https://opensource.com/article/19/8/linux-kernel-testing
|
|
||||||
|
|
||||||
作者：[Major Hayden][a]
|
|
||||||
选题：[lujun9972][b]
|
|
||||||
译者：[译者ID](https://github.com/译者ID)
|
|
||||||
校对：[校对者ID](https://github.com/校对者ID)
|
|
||||||
|
|
||||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
|
||||||
|
|
||||||
[a]: https://opensource.com/users/mhaydenhttps://opensource.com/users/mhaydenhttps://opensource.com/users/marcobravohttps://opensource.com/users/mhayden
|
|
||||||
[b]: https://github.com/lujun9972
|
|
||||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/fail_progress_cycle_momentum_arrow.png?itok=q-ZFa_Eh (arrows cycle symbol for failing faster)
|
|
||||||
[2]: https://opensource.com/article/19/6/continuous-kernel-integration-linux
|
|
||||||
[3]: https://cki-project.org/
|
|
||||||
[4]: https://github.com/getpatchwork/patchwork
|
|
||||||
[5]: https://ccache.dev/
|
|
||||||
[6]: https://linux-test-project.github.io
|
|
||||||
[7]: https://cki-project.org/posts/hackfest-agenda/
|
|
||||||
[8]: https://www.linuxplumbersconf.org/
|
|
||||||
@ -0,0 +1,75 @@
|
|||||||
|
[#]: collector: (lujun9972)
|
||||||
|
[#]: translator: (wxy)
|
||||||
|
[#]: reviewer: ( )
|
||||||
|
[#]: publisher: ( )
|
||||||
|
[#]: url: ( )
|
||||||
|
[#]: subject: (The lifecycle of Linux kernel testing)
|
||||||
|
[#]: via: (https://opensource.com/article/19/8/linux-kernel-testing)
|
||||||
|
[#]: author: (Major Hayden https://opensource.com/users/mhaydenhttps://opensource.com/users/mhaydenhttps://opensource.com/users/marcobravohttps://opensource.com/users/mhayden)
|
||||||
|
|
||||||
|
Linux 内核测试的生命周期
|
||||||
|
======
|
||||||
|
|
||||||
|
> 内核持续集成（CKI）项目旨在防止错误进入 Linux 内核。
|
||||||
|
|
||||||
|
![arrows cycle symbol for failing faster][1]
|
||||||
|
|
||||||
|
在 [Linux 内核的持续集成测试][2] 中，我介绍了 [内核持续集成][3]（CKI）项目及其改变内核开发人员和维护人员工作的使命。本文深入探讨了该项目的某些技术方面以及所有部分如何组合在一起。
|
||||||
|
|
||||||
|
### 一切始于更改
|
||||||
|
|
||||||
|
内核中每一项令人兴奋的功能、改进和错误都始于开发人员提出的更改。这些更改出现在各个内核存储库的大量邮件列表中。一些存储库关注内核中的某些子系统，例如存储或网络，而其它存储库关注内核的更多方面。 当开发人员向内核提出更改或补丁集时，或者维护者在存储库本身中进行更改时，CKI 项目就会付诸行动。
|
||||||
|
|
||||||
|
CKI 项目维护用于监视这些补丁集并采取措施的触发器。诸如 [Patchwork][4] 之类的软件项目通过将多个补丁贡献整合为单个补丁系列，使此过程变得更加容易。补丁系列作为一个整体通过 CKI 系统传播，并可以针对该系列发布单个报告。
|
||||||
|
|
||||||
|
其他触发器监视存储库中的更改。当内核维护人员合并补丁集、还原补丁或创建新标签时，就会发生这种情况。测试这些关键的更改可确保开发人员始终具有坚实的基线，可以用作编写新补丁的基础。
|
||||||
|
|
||||||
|
所有这些更改都进入了 GitLab CI 管道，并历经多个阶段和多个系统。
|
||||||
|
|
||||||
|
### 准备构建
|
||||||
|
|
||||||
|
首先要准备好要编译的源代码。这需要克隆存储库、打上开发人员建议的补丁集，并生成内核配置文件。这些配置文件具有成千上万个用于打开或关闭功能的选项，并且配置文件在不同的系统体系结构之间差异非常大。 例如，一个相当标准的 x86\_64 系统在其配置文件中可能有很多可用选项，但是 s390x 系统（IBM zSeries 大型机）的选项可能要少得多。在该大型机上，某些选项可能有意义，但在消费类笔记本电脑上没有任何作用。
|
||||||
|
|
||||||
|
内核进一步转换为源代码工件。该工件包含整个存储库（已打上补丁）以及编译所需的所有内核配置文件。 上游内核会打包成压缩包，而 Red Hat 的内核生成下一步所用的源代码 RPM 包。
|
||||||
|
|
||||||
|
### 成堆的编译
|
||||||
|
|
||||||
|
编译内核会将源代码转换为计算机可以启动和使用的代码。配置文件描述了要构建的内容，内核中的脚本描述了如何构建它，系统上的工具（例如 GCC 和 glibc）完成构建。此过程需要一段时间才能完成，但是 CKI 项目需要针对四种体系结构快速完成：aarch64（64 位 ARM）、ppc64le（POWER）、s390x（IBM zSeries）和 x86\_64。重要的是，我们必须快速编译内核，以便使工作任务不会积压，而开发人员可以及时收到反馈。
|
||||||
|
|
||||||
|
添加更多的 CPU 可以大大提高速度，但是每个系统都有其局限性。CKI 项目在 OpenShift 的部署环境中的容器内编译内核；尽管 OpenShift 可以实现大量的可伸缩性，但部署环境中的可用 CPU 仍然是数量有限的。CKI 团队分配 20 个虚拟 CPU 来编译每个内核。涉及到四个体系结构，这就涨到看 80 个 CPU！
|
||||||
|
|
||||||
|
另一个速度的提高来自称为 [ccache][5] 的工具。内核开发进展迅速，但是即使在多个发布版本之间，内核的大部分仍保持不变。ccache 工具进行编译期间会在磁盘上缓存已构建的对象（整个内核的一小部分）。稍后再进行另一个内核编译时，ccache 会查找以前看到的内核的未更改部分。ccache 会从磁盘中提取缓存的对象并重新使用它。这样可以加快编译速度并降低总体 CPU 使用率。现在，耗时 20 分钟编译的内核在不到几分钟的时间内就完成了。
|
||||||
|
|
||||||
|
### 测试时间
|
||||||
|
|
||||||
|
内核进入最后一步：在真实硬件上进行测试。每个内核都使用 Beaker 在其原生体系结构上启动，并且开始无数次的测试以发现问题。一些测试会寻找简单的问题，例如容器问题或启动时的错误消息。其他测试则深入到各种内核子系统中，以查找系统调用、内存分配和线程中的回归问题。
|
||||||
|
|
||||||
|
大型测试框架，例如 [Linux Test Project][6]（LTP），包含了大量测试，这些测试在内核中寻找麻烦的回归问题。其中一些回归问题可能会回滚关键的安全修复程序，并且进行测试以确保这些改进仍保留在内核中。
|
||||||
|
|
||||||
|
测试完成后，关键的一步仍然是：报告。内核开发人员和维护人员需要一份简明的报告，准确地告诉他们哪些有效、哪些无效以及如何获取更多信息。每个 CKI 报告都包含所用源代码、编译参数和测试输出的详细信息。该信息可帮助开发人员知道从哪里开始寻找解决问题的方法。此外，它还可以帮助维护人员在漏洞进入内核存储库之前知道何时需要保留补丁集以进行其他的查看。
|
||||||
|
|
||||||
|
### 总结
|
||||||
|
|
||||||
|
CKI 项目团队通过向内核开发人员和维护人员提供及时、自动的反馈，努力防止错误进入 Linux 内核。这项工作通过发现导致内核错误、安全性问题和性能问题等易于找到的问题，使他们的工作更加轻松。
|
||||||
|
|
||||||
|
--------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
via: https://opensource.com/article/19/8/linux-kernel-testing
|
||||||
|
|
||||||
|
作者：[Major Hayden][a]
|
||||||
|
选题：[lujun9972][b]
|
||||||
|
译者：[wxy](https://github.com/wxy)
|
||||||
|
校对：[校对者ID](https://github.com/校对者ID)
|
||||||
|
|
||||||
|
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||||
|
|
||||||
|
[a]: https://opensource.com/users/mhaydenhttps://opensource.com/users/mhaydenhttps://opensource.com/users/marcobravohttps://opensource.com/users/mhayden
|
||||||
|
[b]: https://github.com/lujun9972
|
||||||
|
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/fail_progress_cycle_momentum_arrow.png?itok=q-ZFa_Eh (arrows cycle symbol for failing faster)
|
||||||
|
[2]: https://opensource.com/article/19/6/continuous-kernel-integration-linux
|
||||||
|
[3]: https://cki-project.org/
|
||||||
|
[4]: https://github.com/getpatchwork/patchwork
|
||||||
|
[5]: https://ccache.dev/
|
||||||
|
[6]: https://linux-test-project.github.io
|
||||||
|
[7]: https://cki-project.org/posts/hackfest-agenda/
|
||||||
|
[8]: https://www.linuxplumbersconf.org/
|
||||||
Loading…
Reference in New Issue
Block a user