document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2024-12-26 21:30:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

PRF&PUB: 20190702 One CI-CD pipeline per product to rule them all

Browse Source
This commit is contained in:
chunibyo-wly 2020-07-03 10:27:51 +08:00
parent ab9ecd2e67
commit b90b00f31f
2 changed files with 135 additions and 136 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

136
sources/tech/20190702 One CI-CD pipeline per product to rule them all.md
View File

@ -1,136 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: (chunibyo-wly)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (One CI/CD pipeline per product to rule them all)
[#]: via: (https://opensource.com/article/19/7/cicd-pipeline-rule-them-all)
[#]: author: (Willy-Peter Schaub https://opensource.com/users/wpschaub/users/bclaster/users/matt-micene/users/barkerd427)
One CI/CD pipeline per product to rule them all
======
Is the idea of a unified continuous integration and delivery pipeline a
pipe dream?
![An intersection of pipes.][1]
When I joined the cloud ops team, responsible for cloud operations and engineering process streamlining, at WorkSafeBC, I shared my dream for one instrumented pipeline, with one continuous integration build and continuous deliveries for every product.
According to Lukas Klose, [flow][2] (within the context of software engineering) is "the state of when a system produces value at a steady and predictable rate." I think it is one of the greatest challenges and opportunities, especially in the complex domain of emergent solutions. Strive towards a continuous and incremental delivery model with consistent, efficient, and quality solutions, building the right things and delighting our users. Find ways to break down our systems into smaller pieces that are valuable on their own, enabling teams to deliver value incrementally. This requires a change of mindset for both business and engineering.
### Continuous integration and delivery (CI/CD) pipeline
The CI/CD pipeline is a DevOps practice for delivering code changes more often, consistently, and reliably. It enables agile teams to increase _deployment frequency_ and decrease _lead time for change_, _change-failure rate_, and _mean time to recovery_ key performance indicators (KPIs), thereby improving _quality_ and delivering _value_ faster. The only prerequisites are a solid development process, a mindset for quality and accountability for features from ideation to deprecation, and a comprehensive pipeline (as illustrated below).
![Prerequisites for a solid development process][3]
It streamlines the engineering process and products to stabilize infrastructure environments; optimize flow; and create consistent, repeatable, and automated tasks. This enables us to turn complex tasks into complicated tasks, as outlined by Dave Snowden's [Cynefin Sensemaking][4] model, reducing maintenance costs and increasing quality and reliability.
Part of streamlining our flow is to minimize waste for the [wasteful practice types][5] Muri (overloaded), Mura (variation), and Muda (waste).
* **Muri:** avoid over-engineering, features that do not link to business value, and excessive documentation
* **Mura:** improve approval and validation processes (e.g., security signoffs); drive the [shift-left][6] initiative to push unit testing, security vulnerability scanning, and code quality inspection; and improve risk assessment
* **Muda:** avoid waste such as technical debt, bugs, and upfront, detailed documentation
It appears that 80% of the focus and intention is on products that provide an integrated and collaborative engineering system that can take an idea and plan, develop, test, and monitor your solutions. However, a successful transformation and engineering system is only 5% about products, 15% about process, and 80% about people.
There are many products at our disposal. For example, Azure DevOps offers rich support for continuous integration (CI), continuous delivery (CD), extensibility, and integration with open source and commercial off-the-shelve (COTS) software as a service (SaaS) solutions such as Stryker, SonarQube, WhiteSource, Jenkins, and Octopus. For engineers, it is always a temptation to focus on products, but remember that they are only 5% of our journey.
![5% about products, 15% about process, 80% about people][7]
The biggest challenge is breaking down a process based on decades of rules, regulations, and frustrating areas of comfort: "_It is how we have always done it; why change?_" 
The friction between people in development and operation results in a variety of fragmented, duplicated, and incessant integration and delivery pipelines. Development wants access to everything, to iterate continuously, to enable users, and to release continuously and fast. Operations wants to lock down everything to protect the business and users and drive quality. This inadvertently and often entails processes and governance that are hard to automate, which results in slower-than-expected release cycles.
Let us explore the pipeline with snippets from a recent whiteboard discussion.
The variation of pipelines is difficult and costly to support; the inconsistency of versioning and traceability complicates live site incidents, and continuous streamlining of the development process and pipelines is a challenge.
![Improving quality and visibility of pipelines][8]
I advocate a few principles that enable one universal pipeline per product:
* Automate everything automatable
* Build once
* Maintain continuous integration and delivery
* Maintain continuous streamlining and improvement
* Maintain one build definition
* Maintain one release pipeline definition
* Scan for vulnerabilities early and often, and _fail fast_
* Test early and often, and _fail fast_
* Maintain traceability and observability of releases
If I poke the hornet's nest, however, the most important principle is to _keep it simple_. If you cannot explain the reason (_what_, _why_) and the process (_how_) of your pipelines, you do not understand your engineering process. Most of us are not looking for the best, ultramodern, and revolutionary pipeline—we need one that is functional, valuable, and an enabler for engineering. Tackle the 80%—the culture, people, and their mindset—first. Ask your CI/CD knights in shining armor, with their TLA (two/three-lettered acronym) symbols on their shield, to join the might of practical and empirical engineering.
### Unified pipeline
Let us walk through one of our design practice whiteboard sessions.
![CI build/CD release pipeline][9]
Define one CI/CD pipeline with one build definition per application that is used to trigger _pull-request pre-merge validation_ and _continuous integration_ builds. Generate a _release_ build with debug information and upload to the [Symbol Server][10]. ****This enables developers to debug locally and remotely in production without having to worry which build and symbols they need to load—the symbol server performs that magic for us.
![Breaking down the CI build pipeline][11]
Perform as many validations as possible in the build—_shift left_—allowing feature teams to fail fast, continuously raise the overall product quality, and include invaluable evidence for the reviewers with every pull request. Do you prefer a pull request with a gazillion commits? Or a pull request with a couple of commits and supporting evidence such as security vulnerabilities, test coverage, code quality, and [Stryker][12] mutant remnants? Personally, I vote for the latter.
![Breaking down the CD release pipeline][13]
Do not use build transformation to generate multiple, environment-specific builds. Create one build and perform release-time _transformation_, _tokenization_, and/or XML/JSON _value replacement_. In other words, _shift-right_ the environment-specific configuration.
![Shift-right the environment-specific configuration][14]
Securely store release configuration data and make it available to both Dev and Ops teams based on the level of _trust_ and _sensitivity_ of the data. Use the open source Key Manager, Azure Key Vault, AWS Key Management Service, or one of many other products—remember, there are many hammers in your toolkit!
![Dev-QA-production pipeline][15]
Use _groups_ instead of _users_ to move approver management from multiple stages across multiple pipelines to simple group membership.
![Move approver management to simple group membership][16]
Instead of duplicating pipelines to give teams access to their _areas of interest_, create one pipeline and grant access to _specific stages_ of the delivery environments.
![Pipeline with access to specific delivery stages][17]
Last, but not least, embrace pull requests to help raise insight and transparency into your codebase, improve the overall quality, collaborate, and release pre-validation builds into selected environments; e.g., the Dev environment.
Here is a more formal view of the whole whiteboard sketch.
![The full pipeline][18]
So, what are your thoughts and learnings with CI/CD pipelines? Is my dream of _one pipeline to rule them all_ a pipe dream?
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/7/cicd-pipeline-rule-them-all
作者：[Willy-Peter Schaub][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/chunibyo-wly)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/wpschaub/users/bclaster/users/matt-micene/users/barkerd427
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/LAW-Internet_construction_9401467_520x292_0512_dc.png?itok=RPkPPtDe (An intersection of pipes.)
[2]: https://continuingstudies.sauder.ubc.ca/courses/agile-delivery-methods/ii861
[3]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-2.png (Prerequisites for a solid development process)
[4]: https://en.wikipedia.org/wiki/Cynefin_framework
[5]: https://www.lean.org/lexicon/muda-mura-muri
[6]: https://en.wikipedia.org/wiki/Shift_left_testing
[7]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-3.png (5% about products, 15% about process, 80% about people)
[8]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-4_0.png (Improving quality and visibility of pipelines)
[9]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-5_0.png (CI build/CD release pipeline)
[10]: https://en.wikipedia.org/wiki/Microsoft_Symbol_Server
[11]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-6.png (Breaking down the CI build pipeline)
[12]: https://stryker-mutator.io/
[13]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-7.png (Breaking down the CD release pipeline)
[14]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-8.png (Shift-right the environment-specific configuration)
[15]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-9.png (Dev-QA-production pipeline)
[16]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-10.png (Move approver management to simple group membership)
[17]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-11.png (Pipeline with access to specific delivery stages)
[18]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-12.png (The full pipeline)

135
translated/tech/20190702 One CI-CD pipeline per product to rule them all.md Normal file
View File

@ -0,0 +1,135 @@
[#]: collector: (lujun9972)
[#]: translator: (chunibyo-wly)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (One CI/CD pipeline per product to rule them all)
[#]: via: (https://opensource.com/article/19/7/cicd-pipeline-rule-them-all)
[#]: author: (Willy-Peter Schaub https://opensource.com/users/wpschaub/users/bclaster/users/matt-micene/users/barkerd427)
使用一条CI/CD流水线管理所有的产品
======
统一的持续集成与持续交付的流水线的构想是一种梦想吗？
![An intersection of pipes.][1]
当我加入 WorkSafeBC 的云端运维团队，我负责的是云端运维和工作流水线优化，我分享了一个可以对任意产品进行持续集成和持续交付的流水线。
根据 Lukas Klose 所说，[flow][2]（在软件工程范围内）是“软件系统可以在一种稳定和可预测的状态下创造价值”一种状态。我认为这是我遇到的最大的挑战和机遇之一，特别是在解决紧急问题的复杂领域。我力求通过一种持续，高效和优质的解决方案，提供一种持续交付模式，并且能够构建正确的事物让我们的用户感到满意。
### 持续集成和持续交付的 (CI/CD) 流水线
CI/CD 流水线是一种代码变更的频率更高，一致并且可靠的持续交付 DevOps 实践。它可以帮组敏捷开发团队增加**部署的频率**并且减少**变更所需要的时间**，**变更失败的频率**，和**故障恢复的时间**等关键绩效指标 (KPIS) ，从而提高质量并且实现更快的交付。唯一的先决条件就是坚实的开发基础，对需求从构想到放弃的责任心，和一个全面的流水线（如下图所示）。
![Prerequisites for a solid development process][3]
它简化了工程流程和产品用以稳定基础架构环境；优化工作流程；创建一致的，可重复的，自动化的任务。正如 Dave Snowden [Cynefin Sensemaking][4] 所说的那样，这样就允许我们将复杂不可解决的任务变成了复杂可解决的任务，降低了维护成本，提高了质量和可靠性。
精简流程的一部分是需要最大程度地减少浪费 [wasteful practice types][5] Muri (过载), Mura (变异), 和 Muda (浪费)
* **Muri:** 避免过度工程化，和与业务价值不相关的功能以及过多的文档
* **Mura:** 改善确认和审批流（比如，安全审核）；驱动测试 [shift-left][6]，安全漏洞扫描与代码质量检查；并且可以改进风险评定。
* **Muda:** 避免浪费比如技术债务，bugs 或者前期详细的文档等。
看起来 80％ 的重点都集中在提供一种可以持续集成和协作的工程产品，这些系统可以采用构想一个创意，计划，开发，测试和监视您的解决方案。然而，一个成功的转型和工程系统是由 5％ 的产品，15％ 的过程和 80％ 的开发人员组成的。
有很多产品供我们使用。比如，Azure DevOps 提供了持续集成 (CI) 和持续交付 (CD) 的丰富支持。可扩展，开源集成和商用现成品或技术 (COTS) 像软件即服务 (SaaS) 比如 Stryker, SonarQube, WhiteSource, Jenkins，和 Octopus 等。
![5% about products, 15% about process, 80% about people][7]
最大的挑战是打破数十年的规则，规定和已经步入舒适区的流程：“*我们一直都是这样做的；为什么需要改变呢？*”
开发和运维人员之间的摩擦导致了各种支离破碎，重复的，不间断的集成和交付管道。开发人员希望能访问所有东西，以便快速的持续迭代，用户使用和持续发布。运维人员希望将所有东西说起来来保护业务，用户和品质。这些矛盾在不经意间导致了很难做到一种自动化的流程，进而导致发布周期晚于预期。
让我们使用最近的白板讨论中的片段来探索流水线。
想要支持流水线的变化是一项困难并且花费巨大的工作；版本控制和可追溯更使这个问题变得更加复杂，因此不断精简开发流程和流水线是一项挑战。
![Improving quality and visibility of pipelines][8]
我主张一些原则使得每个产品都能使用通用流水线：
* 使一切自动化
* 一次构建
* 保持持续集成和持续交付
* 保持持续精简和改进
* 保持一个定义构建
* 保持一个发布的流水线定义
* 频繁和尽早的扫描漏洞，并且*尽快的失败*
* 频繁和尽早的进行测试，并且*尽快的失败*
* 保持已发布版本的可追踪和监控
但是，如果我要打破这些，最重要的原则就是*保持简单*。如果你不能说明流水线化的原因，你或许是不了解自己的软件过程的。我们大多数想要的不是最好的，超现代的和具有革命意义的流水线——我们仅仅是需要一个功能强大，有价值的和能适配不同工程的流水线。首先需要解决的是那 80% —— 文化，人员和他们的心态。
### 统一流水线
让我们逐步完成我们的白板会议实践。
![CI build/CD release pipeline][9]
每个应用使用一套构建定义来定义一个 CI/CD 流水线，用来触发*合并请求前的验证*与*持续集成*的构建。生成一个带有 debug 信息的*发布*的构建并且将其上传到 [Symbol Server][10]。这允许了开发者们可以在不需要考虑什么被构建或者什么符号需要被加载的情况下在本地和远程生产环境进行 debug，符号服务器对我们来说就是这样的魔法。
![Breaking down the CI build pipeline][11]
在构建过程中进行尽可能多的构建——*左移测试*——允许制作新特性的团队可以经常的失败，不断的提高整体的产品质量，并且可以为代码审核员提供每个 pull request 的无价证据。你喜欢有大量提交的 pull request 吗？或者一个带有少数提交和提供了漏洞检查，测试覆盖率，代码质量检查和 [Stryker][12] 突变残余的 pull request？
![Breaking down the CD release pipeline][13]
不要通过转变构建去生成多个环境特定的构建。通过一个构建实现*发布时转型*，*标记化*和 XML/JSON 的*值替换*。换句话说，*左移测试*具体环境的配置。
![Shift-right the environment-specific configuration][14]
安全存储发布的配置数据，并且使他基于*可信任*和*敏感*的数据对开发和运维都能有效。使用开源的密钥管理工具，Azure 密钥金库，AWS 密钥管理服务，或者其他产品，记住你的工具箱中有很多方便的工具。
![Dev-QA-production pipeline][15]
使用*用户组*而不是*用户*移动权限系统，使用多流水线的多阶段转移到简单的用户组成员资格。
![Move approver management to simple group membership][16]
创建一条流水线并且对赋予特定的交付阶段的权限，而不是重复流水线让团队进入他们感兴趣的地方。
![Pipeline with access to specific delivery stages][17]
最后但并非不重要，包括 pull request 帮助提高代码仓库洞察力和透明度，增进总体质量，协作和发布预验证构建到已筛选的环境；比如，开发环境。
这是整个白板更正式的视图。
![The full pipeline][18]
所以，您对CI/CD管道有什么想法和经验？是我通过*一条管道来管理它们*的这个梦想吗？
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/7/cicd-pipeline-rule-them-all
作者：[Willy-Peter Schaub][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/chunibyo-wly)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/wpschaub/users/bclaster/users/matt-micene/users/barkerd427
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/LAW-Internet_construction_9401467_520x292_0512_dc.png?itok=RPkPPtDe (An intersection of pipes.)
[2]: https://continuingstudies.sauder.ubc.ca/courses/agile-delivery-methods/ii861
[3]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-2.png (Prerequisites for a solid development process)
[4]: https://en.wikipedia.org/wiki/Cynefin_framework
[5]: https://www.lean.org/lexicon/muda-mura-muri
[6]: https://en.wikipedia.org/wiki/Shift_left_testing
[7]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-3.png (5% about products, 15% about process, 80% about people)
[8]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-4_0.png (Improving quality and visibility of pipelines)
[9]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-5_0.png (CI build/CD release pipeline)
[10]: https://en.wikipedia.org/wiki/Microsoft_Symbol_Server
[11]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-6.png (Breaking down the CI build pipeline)
[12]: https://stryker-mutator.io/
[13]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-7.png (Breaking down the CD release pipeline)
[14]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-8.png (Shift-right the environment-specific configuration)
[15]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-9.png (Dev-QA-production pipeline)
[16]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-10.png (Move approver management to simple group membership)
[17]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-11.png (Pipeline with access to specific delivery stages)
[18]: https://opensource.com/sites/default/files/uploads/devops_pipeline_pipe-12.png (The full pipeline)