From 6f6e1840f1d25b2ff748f832c470547f95150de7 Mon Sep 17 00:00:00 2001 From: pityonline Date: Wed, 15 Aug 2018 12:06:59 +0800 Subject: [PATCH] =?UTF-8?q?=E5=AE=8C=E6=88=90=E7=BF=BB=E8=AF=91=20#9805=20?= =?UTF-8?q?20180806=20What=20is=20CI-CD.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- sources/tech/20180806 What is CI-CD.md | 225 ---------------------- translated/tech/20180806 What is CI-CD.md | 207 ++++++++++++++++++++ 2 files changed, 207 insertions(+), 225 deletions(-) delete mode 100644 sources/tech/20180806 What is CI-CD.md create mode 100644 translated/tech/20180806 What is CI-CD.md diff --git a/sources/tech/20180806 What is CI-CD.md b/sources/tech/20180806 What is CI-CD.md deleted file mode 100644 index 03e28519c4..0000000000 --- a/sources/tech/20180806 What is CI-CD.md +++ /dev/null @@ -1,225 +0,0 @@ -translating by pityonline - -What is CI/CD? -====== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/fail_progress_cycle_momentum_arrow.png?itok=q-ZFa_Eh) - -Continuous integration (CI) and continuous delivery (CD) are extremely common terms used when talking about producing software. But what do they really mean? In this article, I'll explain the meaning and significance behind these and related terms, such as continuous testing and continuous deployment. - -### Quick summary - -An assembly line in a factory produces consumer goods from raw materials in a fast, automated, reproducible manner. Similarly, a software delivery pipeline produces releases from source code in a fast, automated, and reproducible manner. The overall design for how this is done is called "continuous delivery." The process that kicks off the assembly line is referred to as "continuous integration." The process that ensures quality is called "continuous testing" and the process that makes the end product available to users is called "continuous deployment." And the overall efficiency experts that make everything run smoothly and simply for everyone are known as "DevOps" practitioners. - -### What does "continuous" mean? - -Continuous is used to describe many different processes that follow the practices I describe here. It doesn't mean "always running." It does mean "always ready to run." In the context of creating software, it also includes several core concepts/best practices. These are: - - * **Frequent releases:** The goal behind continuous practices is to enable delivery of quality software at frequent intervals. Frequency here is variable and can be defined by the team or company. For some products, once a quarter, month, week, or day may be frequent enough. For others, multiple times a day may be desired and doable. Continuous can also take on an "occasional, as-needed" aspect. The end goal is the same: Deliver software updates of high quality to end users in a repeatable, reliable process. Often this may be done with little to no interaction or even knowledge of the users (think device updates). - - * **Automated processes:** A key part of enabling this frequency is having automated processes to handle nearly all aspects of software production. This includes building, testing, analysis, versioning, and, in some cases, deployment. - - * **Repeatable:** If we are using automated processes that always have the same behavior given the same inputs, then processing should be repeatable. That is, if we go back and enter the same version of code as an input, we should get the same set of deliverables. This also assumes we have the same versions of external dependencies (i.e., other deliverables we don't create that our code uses). Ideally, this also means that the processes in our pipelines can be versioned and re-created (see the DevOps discussion later on). - - * **Fast processing:** "Fast" is a relative term here, but regardless of the frequency of software updates/releases, continuous processes are expected to process changes from source code to deliverables in an efficient manner. Automation takes care of much of this, but automated processes may still be slow. For example, integrated testing across all aspects of a product that takes most of the day may be too slow for product updates that have a new candidate release multiple times per day. - - - - -### What is a "continuous delivery pipeline"? - -The different tasks and jobs that handle transforming source code into a releasable product are usually strung together into a software "pipeline" where successful completion of one automatic process kicks off the next process in the sequence. Such pipelines go by many different names, such as continuous delivery pipeline, deployment pipeline, and software development pipeline. An overall supervisor application manages the definition, running, monitoring, and reporting around the different pieces of the pipeline as they are executed. - -### How does a continuous delivery pipeline work? - -The actual implementation of a software delivery pipeline can vary widely. There are a large number and variety of applications that may be used in a pipeline for the various aspects of source tracking, building, testing, gathering metrics, managing versions, etc. But the overall workflow is generally the same. A single orchestration/workflow application manages the overall pipeline, and each of the processes runs as a separate job or is stage-managed by that application. Typically, the individual "jobs" are defined in a syntax and structure that the orchestration application understands and can manage as a workflow. - -Jobs are created to do one or more functions (building, testing, deploying, etc.). Each job may use a different technology or multiple technologies. The key is that the jobs are automated, efficient, and repeatable. If a job is successful, the workflow manager application triggers the next job in the pipeline. If a job fails, the workflow manager alerts developers, testers, and others so they can correct the problem as quickly as possible. Because of the automation, errors can be found much more quickly than by running a set of manual processes. This quick identification of errors is called "fail fast" and can be just as valuable in getting to the pipeline's endpoint. - -### What is meant by "fail fast"? - -One of a pipeline's jobs is to quickly process changes. Another is to monitor the different tasks/jobs that create the release. Since code that doesn't compile or fails a test can hold up the pipeline, it's important for the users to be notified quickly of such situations. Fail fast refers to the idea that the pipeline processing finds problems as soon as possible and quickly notifies users so the problems can be corrected and code resubmitted for another run through the pipeline. Often, the pipeline process can look at the history to determine who made that change and notify the person and their team. - -### Do all parts of a continuous delivery pipeline have to be automated? - -Nearly all parts of the pipeline should be automated. For some parts, it may make sense to have a spot for human intervention/interaction. An example might be for user-acceptance testing (having end users try out the software and make sure it does what they want/expect). Another case might be deployment to production environments where groups want to have more human control. And, of course, human intervention is required if the code isn't correct and breaks. - -With that background on the meaning of continuous, let's look at the different types of continuous processing and what each means in the context of a software pipeline. - -### What is continuous integration? - -Continuous integration (CI) is the process of automatically detecting, pulling, building, and (in most cases) doing unit testing as source code is changed for a product. CI is the activity that starts the pipeline (although certain pre-validations—often called "pre-flight checks"—are sometimes incorporated ahead of CI). - -The goal of CI is to quickly make sure a new change from a developer is "good" and suitable for further use in the code base. - -### How does continuous integration work? - -The basic idea is having an automated process "watching" one or more source code repositories for changes. When a change is pushed to the repositories, the watching process detects the change, pulls down a copy, builds it, and runs any associated unit tests. - -### How does continuous integration detect changes? - -These days, the watching process is usually an application like [Jenkins][1] that also orchestrates all (or most) of the processes running in the pipeline and monitors for changes as one of its functions. The watching application can monitor for changes in several different ways. These include: - - * **Polling:** The monitoring program repeatedly asks the source management system, "Do you have anything new in the repositories I'm interested in?" When the source management system has new changes, the monitoring program "wakes up" and does its work to pull the new code and build/test it. - - * **Periodic:** The monitoring program is configured to periodically kick off a build regardless of whether there are changes or not. Ideally, if there are no changes, then nothing new is built, so this doesn't add much additional cost. - - * **Push:** This is the inverse of the monitoring application checking with the source management system. In this case, the source management system is configured to "push out" a notification to the monitoring application when a change is committed into a repository. Most commonly, this can be done in the form of a "webhook"—a program that is "hooked" to run when new code is pushed and sends a notification over the internet to the monitoring program. For this to work, the monitoring program must have an open port that can receive the webhook information over the internet. - - - - -### What are "pre-checks" (aka pre-flight checks)? - -Additional validations may be done before code is introduced into the source repository and triggers continuous integration. These follow best practices such as test builds and code reviews. They are usually built into the development process before the code is introduced in the pipeline. But some pipelines may also include them as part of their monitored processes or workflows. - -As an example, a tool called [Gerrit][2] allows for formal code reviews, validations, and test builds after a developer has pushed code but before it is allowed into the ([Git][3] remote) repository. Gerrit sits between the developer's workspace and the Git remote repository. It "catches" pushes from the developer and can do pass/fail validations to ensure they pass before being allowed to make it into the repository. This can include detecting the proposed change and kicking off a test build (a form of CI). It also allows for groups to do formal code reviews at that point. In this way, there is an extra measure of confidence that the change will not break anything when it is merged into the codebase. - -### What are "unit tests"? - -Unit tests (also known as "commit tests") are small, focused tests written by developers to ensure new code works in isolation. "In isolation" here means not depending on or making calls to other code that isn't directly accessible nor depending on external data sources or other modules. If such a dependency is required for the code to run, those resources can be represented by mocks. Mocks refer to using a code stub that looks like the resource and can return values but doesn't implement any functionality. - -In most organizations, developers are responsible for creating unit tests to prove their code works. In fact, one model (known as test-driven development [TDD]) requires unit tests to be designed first as a basis for clearly identifying what the code should do. Because such code changes can be fast and numerous, they must also be fast to execute. - -As they relate to the continuous integration workflow, a developer creates or updates the source in their local working environment and uses the unit tests to ensure the newly developed function or method works. Typically, these tests take the form of asserting that a given set of inputs to a function or method produces a given set of outputs. They generally test to ensure that error conditions are properly flagged and handled. Various unit-testing frameworks, such as [JUnit][4] for Java development, are available to assist. - -### What is continuous testing? - -Continuous testing refers to the practice of running automated tests of broadening scope as code goes through the CD pipeline. Unit testing is typically integrated with the build processes as part of the CI stage and focused on testing code in isolation from other code interacting with it. - -Beyond that, there are various forms of testing that can/should occur. These can include: - - * **Integration testing** validates that groups of components and services all work together. - - * **Functional testing** validates the result of executing functions in the product are as expected. - - * **Acceptance testing** measures some characteristic of the system against acceptable criteria. Examples include performance, scalability, stress, and capacity. - - - - -All of these may not be present in the automated pipeline, and the lines between some of the different types can be blurred. But the goal of continuous testing in a delivery pipeline is always the same: to prove by successive levels of testing that the code is of a quality that it can be used in the release that's in progress. Building on the continuous principle of being fast, a secondary goal is to find problems quickly and alert the development team. This is usually referred to as fail fast. - -### Besides testing, what other kinds of validations can be done against code in the pipeline? - -In addition to the pass/fail aspects of tests, applications exist that can also tell us the number of source code lines that are exercised (covered) by our test cases. This is an example of a metric that can be computed across the source code. This metric is called code-coverage and can be measured by tools (such as [JaCoCo][5] for Java source). - -Many other types of metrics exist, such as counting lines of code, measuring complexity, and comparing coding structures against known patterns. Tools such as [SonarQube][6] can examine source code and compute these metrics. Beyond that, users can set thresholds for what kind of ranges they are willing to accept as "passing" for these metrics. Then, processing in the pipeline can be set to check the computed values against the thresholds, and if the values aren't in the acceptable range, processing can be stopped. Applications such as SonarQube are highly configurable and can be tuned to check only for the things that a team is interested in. - -### What is continuous delivery? - -Continuous delivery (CD) generally refers to the overall chain of processes (pipeline) that automatically gets source code changes and runs them through build, test, packaging, and related operations to produce a deployable release, largely without any human intervention. - -The goals of CD in producing software releases are automation, efficiency, reliability, reproducibility, and verification of quality (through continuous testing). - -CD incorporates CI (automatically detecting source code changes, executing build processes for the changes, and running unit tests to validate), continuous testing (running various kinds of tests on the code to gain successive levels of confidence in the quality of the code), and (optionally) continuous deployment (making releases from the pipeline automatically available to users). - -### How are multiple versions identified/tracked in pipelines? - -Versioning is a key concept in working with CD and pipelines. Continuous implies the ability to frequently integrate new code and make updated releases available. But that doesn't imply that everyone always wants the "latest and greatest." This may be especially true for internal teams that want to develop or test against a known, stable release. So, it is important that the pipeline versions objects that it creates and can easily store and access those versioned objects. - -The objects created in the pipeline processing from the source code can generally be called artifacts. Artifacts should have versions applied to them when they are built. The recommended strategy for assigning version numbers to artifacts is called semantic versioning. (This also applies to versions of dependent artifacts that are brought in from external sources.) - -Semantic version numbers have three parts: major, minor, and patch. (For example, 1.4.3 reflects major version 1, minor version 4, and patch version 3.) The idea is that a change in one of these parts represents a level of update in the artifact. The major version is incremented only for incompatible API changes. The minor version is incremented when functionality is added in a backward-compatible manner. And the patch version is incremented when backward-compatible bug fixes are made. These are recommended guidelines, but teams are free to vary from this approach, as long as they do so in a consistent and well-understood manner across the organization. For example, a number that increases each time a build is done for a release may be put in the patch field. - -### How are artifacts "promoted"? - -Teams can assign a promotion "level" to artifacts to indicate suitability for testing, production, etc. There are various approaches. Applications such as Jenkins or [Artifactory][7] can be enabled to do promotion. Or a simple scheme can be to add a label to the end of the version string. For example, -snapshot can indicate the latest version (snapshot) of the code was used to build the artifact. Various promotion strategies or tools can be used to "promote" the artifact to other levels such as -milestone or -production as an indication of the artifact's stability and readiness for release. - -### How are multiple versions of artifacts stored and accessed? - -Versioned artifacts built from source can be stored via applications that manage "artifact repositories." Artifact repositories are like source management for built artifacts. The application (such as Artifactory or [Nexus][8]) can accept versioned artifacts, store and track them, and provide ways for them to be retrieved. - -Pipeline users can specify the versions they want to use and have the pipeline pull in those versions. - -### What is continuous deployment? - -Continuous deployment (CD) refers to the idea of being able to automatically take a release of code that has come out of the CD pipeline and make it available for end users. Depending on the way the code is "installed" by users, that may mean automatically deploying something in a cloud, making an update available (such as for an app on a phone), updating a website, or simply updating the list of available releases. - -An important point here is that just because continuous deployment can be done doesn't mean that every set of deliverables coming out of a pipeline is always deployed. It does mean that, via the pipeline, every set of deliverables is proven to be "deployable." This is accomplished in large part by the successive levels of continuous testing (see the section on Continuous Testing in this article). - -Whether or not a release from a pipeline run is deployed may be gated by human decisions and various methods employed to "try out" a release before fully deploying it. - -### What are some ways to test out deployments before fully deploying to all users? - -Since having to rollback/undo a deployment to all users can be a costly situation (both technically and in the users' perception), numerous techniques have been developed to allow "trying out" deployments of new functionality and easily "undoing" them if issues are found. These include: - -#### Blue/green testing/deployments - -In this approach to deploying software, two identical hosting environments are maintained — a _blue_ one and a _green_ one. (The colors are not significant and only serves as identifers.) At any given point, one of these is the _production_ deployment and the other is the _candidate_ deployment. - -In front of these instances is a router or other system that serves as the customer “gateway” to the product or application. By pointing the router to the desired blue or green instance, customer traffic can be directed to the desired deployment. In this way, swapping out which deployment instance is pointed to (blue or green) is quick, easy, and transparent to the user. - -When a new release is ready for testing, it can be deployed to the non-production environment. After it’s been tested and approved, the router can be changed to point the incoming production traffic to it (so it becomes the new production site). Now the hosting environment that was production is available for the next candidate. - -Likewise, if a problem is found with the latest deployment and the previous production instance is still deployed in the other environment, a simple change can point the customer traffic back to the previous production instance — effectively taking the instance with the problem “offline” and rolling back to the previous version. The new deployment with the problem can then be fixed in the other area. - -#### Canary testing/deployment - -In some cases, swapping out the entire deployment via a blue/green environment may not be workable or desired. Another approach is known as _canary_ testing/deployment. In this model, a portion of customer traffic is rerouted to new pieces of the product. For example, a new version of a search service in a product may be deployed alongside the current production version of the service. Then, 10% of search queries may be routed to the new version to test it out in a production environment. - -If the new service handles the limited traffic with no problems, then more traffic may be routed to it over time. If no problems arise, then over time, the amount of traffic routed to the new service can be increased until 100% of the traffic is going to it. This effectively “retires” the previous version of the service and puts the new version into effect for all customers. - -#### Feature toggles - -For new functionality that may need to be easily backed out (in case a problem is found), developers can add a feature toggle. This is a software if-then switch in the code that only activates the code if a data value is set. This data value can be a globally accessible place that the deployed application checks to see whether it should execute the new code. If the data value is set, it executes the code; if not, it doesn't. - -This gives developers a remote "kill switch" to turn off the new functionality if a problem is found after deployment to production. - -#### Dark launch - -In this practice, code is incrementally tested/deployed into production, but changes are not made visible to users (thus the "dark" name). For example, in the production release, some portion of web queries might be redirected to a service that queries a new data source. This information can be collected by development for analysis—without exposing any information about the interface, transaction, or results back to users. - -The idea here is to get real information on how a candidate change would perform under a production load without impacting users or changing their experience. Over time, more load can be redirected until either a problem is found or the new functionality is deemed ready for all to use. Feature flags can be used actually to handle the mechanics of dark launches. - -### What is DevOps? - -[DevOps][9] is a set of ideas and recommended practices around how to make it easier for development and operational teams to work together on developing and releasing software. Historically, development teams created products but did not install/deploy them in a regular, repeatable way, as customers would do. That set of install/deploy tasks (as well as other support tasks) were left to the operations teams to sort out late in the cycle. This often resulted in a lot of confusion and problems, since the operations team was brought into the loop late in the cycle and had to make what they were given work in a short timeframe. As well, development teams were often left in a bad position—because they had not sufficiently tested the product's install/deploy functionality, they could be surprised by problems that emerged during that process. - -This often led to a serious disconnect and lack of cooperation between development and operations teams. The DevOps ideals advocate ways of doing things that involve both development and operations staff from the start of the cycle through the end, such as CD. - -### How does CD intersect with DevOps? - -The CD pipeline is an implementation of several DevOps ideals. The later stages of a product, such as packaging and deployment, can always be done on each run of the pipeline rather than waiting for a specific point in the product development cycle. As well, both development and operations staff can clearly see when things work and when they don't, from development to deployment. For a cycle of a CD pipeline to be successful, it must pass through not only the processes associated with development but also the ones associated with operations. - -Carried to the next level, DevOps suggests that even the infrastructure that implements the pipeline be treated like code. That is, it should be automatically provisioned, trackable, easy to change, and spawn a new run of the pipeline if it changes. This can be done by implementing the pipeline as code. - -### What is "pipeline-as-code"? - -Pipeline-as-code is a general term for creating pipeline jobs/tasks via programming code, just as developers work with source code for products. The goal is to have the pipeline implementation expressed as code so it can be stored with the code, reviewed, tracked over time, and easily spun up again if there is a problem and the pipeline must be stopped. Several tools allow this, including [Jenkins 2][1]. - -### How does DevOps impact infrastructure for producing software? - -Traditionally, individual hardware systems used in pipelines were configured with software (operating systems, applications, development tools, etc.) one at a time. At the extreme, each system was a custom, hand-crafted setup. This meant that when a system had problems or needed to be updated, that was frequently a custom task as well. This kind of approach goes against the fundamental CD ideal of having an easily reproducible and trackable environment. - -Over the years, applications have been developed to standardize provisioning (installing and configuring) systems. As well, virtual machines were developed as programs that emulate computers running on top of other computers. These VMs require a supervisory program to run them on the underlying host system. And they require their own operating system copy to run. - -Next came containers. Containers, while similar in concept to VMs, work differently. Instead of requiring a separate program and a copy of an OS to run, they simply use some existing OS constructs to carve out isolated space in the operating system. Thus, they behave similarly to a VM to provide the isolation but don't require the overhead. - -Because VMs and containers are created from stored definitions, they can be destroyed and re-created easily with no impact to the host systems where they are running. This allows a re-creatable system to run pipelines on. Also, for containers, we can track changes to the definition file they are built from—just as we would for source code. - -Thus, if we run into a problem in a VM or container, it may be easier and quicker to just destroy and re-create it instead of trying to debug and make a fix to the existing one. - -This also implies that any change to the code for the pipeline can trigger a new run of the pipeline (via CI) just as a change to code would. This is one of the core ideals of DevOps regarding infrastructure. - --------------------------------------------------------------------------------- - -via: https://opensource.com/article/18/8/what-cicd - -作者：[Brent Laster][a] -选题：[lujun9972](https://github.com/lujun9972) -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://opensource.com/users/bclaster -[1]:https://jenkins.io -[2]:https://www.gerritcodereview.com -[3]:https://opensource.com/resources/what-is-git -[4]:https://junit.org/junit5/ -[5]:https://www.eclemma.org/jacoco/ -[6]:https://www.sonarqube.org/ -[7]:https://jfrog.com/artifactory/ -[8]:https://www.sonatype.com/nexus-repository-sonatype -[9]:https://opensource.com/resources/devops diff --git a/translated/tech/20180806 What is CI-CD.md b/translated/tech/20180806 What is CI-CD.md new file mode 100644 index 0000000000..b178accf76 --- /dev/null +++ b/translated/tech/20180806 What is CI-CD.md @@ -0,0 +1,207 @@ +什么是 CI/CD？ +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/fail_progress_cycle_momentum_arrow.png?itok=q-ZFa_Eh) + +在谈论软件开发时，经常会提到持续集成Continuous Integration（CI）和持续交付Continuous Delivery（CD）这几个术语。但它们真正的意思是什么呢？在本文中，我将解释这些和相关术语背后的含义和意义，例如持续测试Continuous Testing和持续部署Continuous Deployment。 + +### 概览 + +工厂里的装配线以快速、自动化、可重复的方式从原材料生产出消费品。同样，软件交付管道以快速，自动化和可重复的方式从源代码生成发布版本。如何完成这项工作的总体设计称为“持续交付”。启动装配线的过程称为“持续集成”。确保质量的过程称为“持续测试”，将最终产品提供给用户的过程称为“持续部署”。一些专家让这一切简单、顺畅、高效地运行，这些人被称为运维开发DevOps。 + +### 持续（Continuous）是什么意思？ + +Continuous 用于描述遵循我在此提到的许多不同流程实践。这并不意味着“一直在运行”，而是“随时可运行”。在软件开发领域，它还包括几个核心概念/最佳实践。这些是： + +* **频繁发布**：持续实践背后目标是能够频繁地交付高质量的软件。此处的交付频率是可变的，可由开发团队或公司定义。对于某些产品，一季度，一个月，一周或一天交付一次可能已经足够频繁了。对于另一些来说，一天可能需要多次交付也是可行的。所谓持续也有“偶尔，按需”的方面。最终目标是相同的：在可重复，可靠的过程中为最终用户提供高质量的软件更新。通常，这可以通过很少甚至没有用户的交互或知识来完成（想想设备更新）。 +* **自动化流程**：实现此频率的关键是用自动化流程来处理软件生产中的方方面面。这包括构建、测试、分析、版本控制，以及在某些情况下的部署。 +* **可重复**：如果我们使用的自动化流程在给定相同输入的情况下始终具有相同的行为，则这个过程应该是可重复的。也就是说，如果我们把某个历史版本的代码作为输入，我们应该得到对应相同的可交付产出。这也假设我们有相同版本的外部依赖项（即我们不创建该版本代码使用的其它交付物）。理想情况下，这也意味着可以对管道中的流程进行版本控制和重建（请参阅稍后的 DevOps 讨论）。 +* **快速迭代**：“快速”在这里是个相对术语，但无论软件更新/发布的频率如何，预期的持续过程都会以高效的方式将源代码转换为交付物。自动化负责大部分工作，但自动化处理的过程可能仍然很慢。例如，对于每天需要多次发布候选版更新的产品来说，一轮集成测试integrated testing下来耗时就要大半天可能就太慢了。 + +### 什么是持续交付管道（continuous delivery pipeline）？ + +将源代码转换为可发布品的多个不同的任务task和作业job通常串联成一个软件“管道”，一个自动流程成功完成后会启动管道中的下一个流程。这些管道有许多不同的叫法，例如持续交付管道，部署管道和软件开发管道。大体上讲，程序管理者在管道执行时管理管道各部分的定义、运行、监控和报告。 + +### 持续交付管道是如何工作的？ + +软件交付管道的实际实现可以有很大不同。有许多程序可用在管道中，用于源跟踪、构建、测试、指标采集，版本管理等各个方面。但整体工作流程通常是相同的。单个业务流程/工作流应用程序管理整个管道，每个流程作为独立的作业运行或由该应用程序进行阶段管理。通常，在业务流程中，这些独立作业是应用程序可理解并可作为工作流程管理的语法和结构中定义的。 + +这些作业被用于一个或多个功能（构建、测试、部署等）。每个作业可能使用不同的技术或多种技术。关键是作业是自动化的，高效的，并且可重复的。如果作业成功，则工作流管理器将触发管道中的下一个作业。如果作业失败，工作流管理器会向开发人员，测试人员和其他人发出警报，以便他们尽快纠正问题。这个过程是自动化的，所以比手动运行一组过程可更快地找到错误。这种快速排错称为快速失败fail fast，并且在获取管道端点方面同样有价值。 + +### 快速失败（fail fast）是什么意思？ + +管道的工作之一就是快速处理变更。另一个是监视创建发布的不同任务/作业。由于编译失败或测试未通过的代码可以阻止管道继续运行，因此快速通知用户此类情况非常重要。快速失败指的是在管道流程中尽快发现问题并快速通知用户的想法，这样可以及时修正问题并重新提交代码以便使管道再次运行。通常在管道流程中可通过查看历史记录来确定是谁做了那次修改并通知此人及其团队。 + +### 所有持续交付管道都应该被自动化吗？ + +管道的几乎所有部分都是应该自动化的。对于某些部分，有一些人为干预/互动的地方可能是有意义的。一个例子可能是用户验收测试user-acceptance testing（让最终用户试用软件并确保它能达到他们想要/期望的水平）。另一种情况可能是部署到生产环境时用户希望拥有更多的人为控制。当然，如果代码不正确或不能运行，则需要人工干预。 + +有了对 continuous 理解的背景，让我们看看不同类型的持续流程以及它们在软件管道上下文中的含义。 + +### 什么是持续集成（continuous integration）？ + +持续集成（CI）是在源代码变更后自动检测、拉取、构建和（在大多数情况下）进行单元测试的过程。持续集成是启动管道的环节（尽管某些预验证 —— 通常称为上线前检查pre-flight checks —— 有时会被归在持续集成之前）。 + +持续信札的目标是快速确保开发人员新提交的变更是好的，并且适合在代码库中进一步使用。 + +### 持续集成是如何工作的？ + +持续集成的基本思想是让一个自动化过程监测一个或多个源代码仓库是否有变更。当变更被推送到仓库时，它会监测到更改，下载副本，构建并运行任何相关的单元测试。 + +### 持续集成如何监测变更？ + +目前，监测程序通常是像 [Jenkins][1] 这样的应用程序，它还协调管道中运行的所有（或大多数）进程，监视变更是其功能之一。监测程序可以以几种不同方式监测变更。这些包括： + +* **轮询**：监测程序反复询问代码管理系统，“代码仓库里有什么我感兴趣的新东西吗？”当代码管理系统有新的变更时，监测程序会“唤醒”并完成其工作以获取新代码并构建/测试它。 +* **定期**：监测程序配置为定期启动构建，无论源码是否有变更。理想情况下，如果没有变更，则不会构建任何新内容，因此这不会增加额外的成本。 +* **推送**：这与用于代码管理系统检查的监测程序相反。在这种情况下，代码管理系统被配置为提交变更到仓库时将通知“推送”到监测程序。最常见的是，这可以以 webhook 的形式完成 —— 在新代码被推送时一个挂勾hook的程序通过互联网向监测程序发送通知。为此，监测程序必须具有可以通过网络接收 webhook 信息的开放端口。 + +### 什么是预检查（pre-checks 又称上线前检查 pre-flight checks）？ + +在将代码引入仓库并触发持续集成之前，可以进行其它验证。这遵循了最佳实践，例如测试构建test builds和代码审查code review。它们通常在代码引入管道之前构建到开发过程中。但是一些管道也可能将它们作为其监控流程或工作流的一部分。 + +例如，一个名为 [Gerrit][2] 的工具允许在开发人员推送代码之后但在允许进入（[Git][3] 远程）仓库之前进行正式的代码审查，验证和测试构建。Gerrit 位于开发人员的工作区和 Git 远程仓库之间。它会“接收”来自开发人员的推送，并且可以执行通过/失败验证以确保它们在被允许进入仓库之前的检查是通过的。这可以包括检测新变更并启动构建测试（CI 的一种形式）。它还允许开发者在那时进行正式的代码审查。这种方式有一种额外的可信度评估机制，即当变更的代码被合并到代码库中时不会破坏任何内容。 + +### 什么是单元测试（unit test）？ + +单元测试（也称为“提交测试”），是由开发人员编写的小型的专项测试，以确保新代码独立工作。“独立”这里意味着不依赖或调用其它不可直接访问的代码，也不依赖外部数据源或其它模块。如果运行代码需要这样的依赖关系，那么这些资源可以用模拟mock来表示。模拟是指使用看起来像资源的代码存根code stub，可以返回值但不实现任何功能。 + +在大多数组织中，开发人员负责创建单元测试以证明其代码正确。事实上，一种称为测试驱动开发test-driven develop（TDD）的模型要求将首先设计单元测试作为清楚地验证代码功能的基础。因为这样的代码更改速度快且改动量大，所以它们也必须执行很快。 + +由于这与持续集成工作流有关，因此开发人员在本地工作环境中编写或更新代码，并通单元测试来确保新开发的功能或方法正确。通常，这些测试采用断言形式，即函数或方法的给定输入集产生给定的输出集。它们通常进行测试以确保正确标记和处理出错条件。有很多单元测试框架都很有用，例如用于 Java 开发的 [JUnit][4]。 + +### 什么是持续测试（continuous testing）？ + +持续测试是指在代码通过持续交付管道时运行扩展范围的自动化测试的实践。单元测试通常与构建过程集成，作为持续集成阶段的一部分，并专注于和其它与之交互的代码隔离的测试。 + +除此之外，还有各种形式的测试可以或应该出现。这些可包括： + +* **集成测试** 验证组件和服务组合在一起是否正常。 +* **功能测试** 验证产品中执行功能的结果是否符合预期。 +* **验收测试** 根据可接受的标准验证产品的某些特征。如性能、可伸缩性、抗压能力和容量。 + +所有这些可能不存在于自动化的管道中，并且一些不同类型的测试分类界限也不是很清晰。但是，在交付管道中持续测试的目标始终是相同的：通过持续的测试级别证明代码的质量可以在正在进行的发布中使用。在持续集成快速的原则基础上，第二个目标是快速发现问题并提醒开发团队。这通常被称为快速失败。 + +### 除了测试之外，还可以对管道中的代码进行哪些其它类型的验证？ + +除了测试是否通过之外，还有一些应用程序可以告诉我们测试用例执行（覆盖）的源代码行数。这是一个可以衡量代码量指标的例子。这个指标称为代码覆盖率code-coverage，可以通过工具（例如用于 Java 的 [JaCoCo][5]）进行统计。 + +还有很多其它类型的指标统计，例如代码行数，复杂度以及代码结构对比分析等。诸如 [SonarQube][6] 之类的工具可以检查源代码并计算这些指标。此外，用户还可以为他们可接受的“合格”范围的指标设置阈值。然后可以在管道中针对这些阈值设置一个检查，如果结果不在可接受范围内，则流程终端上。SonarQube 等应用程序具有很高的可配置性，可以设置仅检查团队感兴趣的内容。 + +### 什么是持续交付（continuous delivery）？ + +持续交付（CD）通常是指整个流程链（管道），它自动监测源代码变更并通过构建，测试，打包和相关操作运行它们以生成可部署的版本，基本上没有任何人为干预。 + +持续交付在软件开发过程中的目标是自动化、效率、可靠性、可重复性和质量保障（通过持续测试）。 + +持续交付包含持续集成（自动检测源代码变更，执行构建过程，运行单元测试以验证变更），持续测试（对代码运行各种测试以保障代码质量），和（可选）持续部署（通过管道发布版本自动提供给用户）。 + +### 如何在管道中识别/跟踪多个版本？ + +版本控制是持续交付和管道的关键概念。持续意味着能够经常集成新代码并提供更新版本。但这并不意味着每个人都想要“最新，最好的”。对于想要开发或测试已知的稳定版本的内部团队来说尤其如此。因此，管道创建并轻松存储和访问的这些版本化对象非常重要。 + +在管道中从源代码创建的对象通常可以称为工件artifacts。工件在构建时应该有应用于它们的版本。将版本号分配给工件的推荐策略称为语义化版本控制semantic versioning。（这也适用于从外部源引入的依赖工件的版本。） + +语义版本号有三个部分：major，minor 和 patch。（例如，1.4.3 反映了主要版本 1，次要版本 4 和补丁版本 3。）这个想法是，其中一个部分的更改表示工件中的更新级别。主要版本仅针对不兼容的 API 更改而递增。当以向后兼容backward-compatible的方式添加功能时，次要版本会增加。当进行向后兼容的版本 bug 修复时，补丁版本会增加。这些是建议的指导方针，但只要团队在整个组织内以一致且易于理解的方式这样做，团队就可以自由地改变这种方法。例如，每次为发布完成构建时增加的数字可以放在补丁字段中。 + +### 如何“分销”（promote）工件？ + +团队可以为工件分配分销级别以指示适用于测试，生产等环境或用途。有很多方法。可以用 Jenkins 或 [Artifactory][7] 等应用程序进行分销。或者一个简单的方案可以在版本号字符串的末尾添加标签。例如，`-snapshot` 可以指示用于构建工件的代码的最新版本（快照）。可以使用各种分销策略或工具将工件“提升”到其它级别，例如 `-milestone` 或 `-production`，作为工件稳定性和完备性版本的标记。 + +### 如何存储和访问多个工件版本？ + +从源代码构建的版本化工件可以通过管理工件仓库artifact repository的应用程序进行存储。工件仓库就像构建工件的版本控制工具一样。像 Artifactory 或 [Nexus][8] 这类应用可以接受版本化工件，存储和跟踪它们，并提供检索的方法。 + +管道用户可以指定他们想要使用的版本，并在这些版本中使用管道。 + +### 什么是持续部署（continuous deployment）？ + +持续部署（CD）是指能够自动提供持续交付管道中发布版本给最终用户使用的想法。根据用户的安装方式，可能是在云环境中自动部署，app 升级（如手机上的应用程序），更新网站或只更新可用版本列表。 + +这里的一个重点是，仅仅因为可以进行持续部署并不意味着始终部署来自管道的每组可交付成果。它实际上指，通过管道每套可交付成果都被证明是“可部署的”。这在很大程度上是由持续测试的连续级别完成的（参见本文中的持续测试部分）。 + +管道构建的发布成果是否被部署可以通过人工决策，或利用在完全部署之前“试用”发布的各种方法来进行控制。 + +### 在完全部署到所有用户之前，有哪些方法可以测试部署？ + +由于必须回滚/撤消对所有用户的部署可能是一种代价高昂的情况（无论是技术上还是用户的感知），已经有许多技术允许“尝试”部署新功能并在发现问题时轻松“撤消”它们。这些包括： + +#### 蓝/绿测试/部署（blue/green testing/deployments） + +在这种部署软件的方法中，维护了两个相同的主机环境 —— 一个 _蓝色_ 和一个 _绿色_。（颜色并不重要，仅作为标识。）对应来说，其中一个是 _生产环境_，另一个是 _预发布环境_。 + +在这些实例的前面是调度系统，它们充当产品或应用程序的客户“网关”。通过将调度系统指向蓝色或绿色实例，可以将客户流量引流到期望的部署环境。通过这种方式，切换指向哪个部署实例（蓝色或绿色）对用户来说是快速，简单和透明的。 + +当新版本准备好进行测试时，可以将其部署到非生产环境中。在经过测试和批准后，可以更改调度系统设置以将传入的线上流量指向它（因此它将成为新的生产站点）。现在，曾作为生产环境实例可供下一次候选发布使用。 + +同理，如果在最新部署中发现问题并且之前的生产实例仍然可用，则简单的更改可以将客户流量引流回到之前的生产实例 —— 有效地将问题实例“下线”并且回滚到以前的版本。然后有问题的新实例可以在其它区域中修复。 + +#### 金丝雀测试/部署（canary testing/deployment） + +在某些情况下，通过蓝/绿发布切换整个部署可能不可行或不是期望的那样。另一种方法是为 _金丝雀_ 测试/部署。在这种模型中，一部分客户流量被重新引流到新的版本部署中。例如，新版本的搜索服务可以与当前服务的生产版本一起部署。然后，可以将 10％ 的搜索查询引流到新版本，以在生产环境中对其进行测试。 + +如果服务那些流量的新版本没问题，那么可能会有更多的流量会被逐渐引流过去。如果仍然没有问题出现，那么随着时间的推移，可以对新版本增量部署，直到 100％ 的流量都调度到新版本。这有效地“更替”了以前版本的服务，并让新版本对所有客户生效。 + +#### 功能开关（feature toggles） + +对于可能需要轻松关掉的新功能（如果发现问题），开发人员可以添加功能开关。这是代码中的 `if-then` 软件功能开关，仅在设置数据值时才激活新代码。此数据值可以是全局可访问的位置，部署的应用程序将检查该位置是否应执行新代码。如果设置了数据值，则执行代码；如果没有，则不执行。 + +这为开发人员提供了一个远程“终止开关”，以便在部署到生产环境后发现问题时关闭新功能。 + +#### 暗箱发布（dark launch） + +在这种实践中，代码被逐步测试/部署到生产环境中，但是用户不会看到更改（因此名称中有 dark 一词）。例如，在生产版本中，网页查询的某些部分可能会重定向到查询新数据源的服务。开发人员可收集此信息进行分析，而不会将有关接口，事务或结果的任何信息暴露给用户。 + +这个想法是想获取候选版本在生产环境负载下如何执行的真实信息，而不会影响用户或改变他们的经验。随着时间的推移，可以调度更多负载，直到遇到问题或认为新功能已准备好供所有人使用。实际上功能开关标志可用于这种暗箱发布机制。 + +### 什么是运维开发（DevOps）？ + +[DevOps][9] 是关于如何使开发和运维团队更容易合作开发和发布软件的一系列想法和推荐的实践。从历史上看，开发团队研发了产品，但没有像客户那样以常规，可重复的方式安装/部署它们。在整个周期中，这组安装/部署任务（以及其它支持任务）留给运维团队负责。这经常导致很多混乱和问题，因为运维团队在后期才开始介入，并且必须在短时间内完成他们的工作。同样，开发团队经常处于不利地位 —— 因为他们没有充分测试产品的安装/部署功能，他们可能会对该过程中出现的问题感到惊讶。 + +这往往导致开发和运维团队之间严重脱节和缺乏合作。DevOps 理念主张是贯穿整个开发周期的开发和运维综合协作的工作方式，就像持续交付那样。 + +### 持续交付如何与运维开发相交？ + +持续交付管道是几个 DevOps 理念的实现。产品开发的后期阶段（如打包和部署）始终可以在管道的每次运行中完成，而不是等待产品开发周期中的特定时间。同样，从开发到部署过程中，开发和运维都可以清楚地看到事情何时起作用，何时不起作用。要使持续交付管道循环成功，不仅要通过与开发相关的流程，还要通过与运维相关的流程。 + +说得更远一些，DevOps 建议实现管道的基础架构也会被视为代码。也就是说，它应该自动配置、可跟踪、易于修改，并在管道发生变化时触发新一轮运行。这可以通过将管道实现为代码来完成。 + +### 什么是管道即代码（pipeline-as-code）？ + +管道即代码pipeline-as-code是通过编写代码创建管道作业/任务的通用术语，就像开发人员编写代码一样。它的目标是将管道实现表示为代码，以便它可以与代码一起存储、评审、跟踪，如果出现问题并且必须终止管道，则可以轻松地重建。有几个工具允许这样做，如 [Jenkins 2][1]。 + +### DevOps 如何影响生产软件的基础设施？ + +传统意义上，管道中使用的各个硬件系统都有配套的软件（操作系统，应用程序，开发工具等）。在极端情况下，每个系统都是手工设置来定制的。这意味着当系统出现问题或需要更新时，这通常也是一项自定义任务。这种方法违背了持续交付的基本理念，即具有易于重现和可跟踪的环境。 + +多年来，很多应用被开发用于标准化交付（安装和配置）系统。同样，虚拟机virtual machine被开发为模拟在其它计算机之上运行的计算机程序。这些 VM 要有管理程序才能在底层主机系统上运行，并且它们需要自己的操作系统副本才能运行。 + +后来有了容器container。容器虽然在概念上与 VM 类似，但工作方式不同。它们只需使用一些现有的操作系统结构来划分隔离空间，而不需要运行单独的程序和操作系统的副本。因此，它们的行为类似于 VM 以提供隔离但不需要过多的开销。 + +VM 和容器是根据配置定义创建的，因此可以轻易地销毁和重建，而不会影响运行它们的主机系统。这允许运行管道的系统也可重建。此外，对于容器，我们可以跟踪其构建定义文件的更改 —— 就像对源代码一样。 + +因此，如果遇到 VM 或容器中的问题，我们可以更容易、更快速地销毁和重建它们，而不是在当前环境尝试调试和修复。 + +这也意味着对管道代码的任何更改都可以触发管道新一轮运行（通过 CI），就像对代码的更改一样。这是 DevOps 关于基础架构的核心理念之一。 + +--- + +via: https://opensource.com/article/18/8/what-cicd + +作者：[Brent Laster][a] +选题：[lujun9972](https://github.com/lujun9972) +译者：[pityonline](https://github.com/pityonline) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/bclaster +[1]: https://jenkins.io +[2]: https://www.gerritcodereview.com +[3]: https://opensource.com/resources/what-is-git +[4]: https://junit.org/junit5/ +[5]: https://www.eclemma.org/jacoco/ +[6]: https://www.sonarqube.org/ +[7]: https://jfrog.com/artifactory/ +[8]: https://www.sonatype.com/nexus-repository-sonatype +[9]: https://opensource.com/resources/devops