submit tech/20171220 Containers without Docker at Red Hat.md

This commit is contained in:
songshunqiang 2018-04-20 09:49:01 +08:00
parent b113d9c1e5
commit 7ef3a20bcb
2 changed files with 116 additions and 117 deletions

View File

@ -1,117 +0,0 @@
pinewall translating
Containers without Docker at Red Hat
======
The Docker (now [Moby][1]) project has done a lot to popularize containers in recent years. Along the way, though, it has generated concerns about its concentration of functionality into a single, monolithic system under the control of a single daemon running with root privileges: `dockerd`. Those concerns were reflected in a [talk][2] by Dan Walsh, head of the container team at Red Hat, at [KubeCon \+ CloudNativeCon][3]. Walsh spoke about the work the container team is doing to replace Docker with a set of smaller, interoperable components. His rallying cry is "no big fat daemons" as he finds them to be contrary to the venerated Unix philosophy.
### The quest to modularize Docker
As we saw in an [earlier article][4], the basic set of container operations is not that complicated: you need to pull a container image, create a container from the image, and start it. On top of that, you need to be able to build images and push them to a registry. Most people still use Docker for all of those steps but, as it turns out, Docker isn't the only name in town anymore: an early alternative was `rkt`, which led to the creation of various standards like CRI (runtime), OCI (image), and CNI (networking) that allow backends like [CRI-O][5] or Docker to interoperate with, for example, [Kubernetes][6].
These standards led Red Hat to create a set of "core utils" like the CRI-O runtime that implements the parts of the standards that Kubernetes needs. But Red Hat's [OpenShift][7] project needs more than what Kubernetes provides. Developers will want to be able to build containers and push them to the registry. Those operations need a whole different bag of tricks.
It turns out that there are multiple tools to build containers right now. Apart from Docker itself, a [session][8] from Michael Ducy of Sysdig reviewed eight image builders, and that's probably not all of them. Ducy identified the ideal build tool as one that would create a minimal image in a reproducible way. A minimal image is one where there is no operating system, only the application and its essential dependencies. Ducy identified [Distroless][9], [Smith][10], and [Source-to-Image][11] as good tools to build minimal images, which he called "micro-containers".
A reproducible container is one that you can build multiple times and always get the same result. For that, Ducy said you have to use a "declarative" approach (as opposed to "imperative"), which is understandable given that he comes from the Chef configuration-management world. He gave the examples of [Ansible Container][12], [Habitat][13], [nixos-container][14], and Smith (yes, again) as being good approaches, provided you were familiar with their domain-specific languages. He added that Habitat ships its own supervisor in its containers, which may be superfluous if you already have an external one, like systemd, Docker, or Kubernetes. To complete the list, we should mention the new [BuildKit][15] from Docker and [Buildah][16], which is part of Red Hat's [Project Atomic][17].
### Building containers with Buildah
![\[Buildah logo\]][18] Buildah's name apparently comes from Walsh's colorful [Boston accent][19]; the Boston theme permeates the branding of the tool: the logo, for example, is a Boston terrier dog (seen at right). This project takes a different approach from Ducy's decree: instead of enforcing a declarative configuration-management approach to containers, why not build simple tools that can be used by your favorite configuration-management tool? If you want to use regular command-line commands like `cp` (instead of Docker's custom `COPY` directive, for example), you can. But you can also use Ansible or Puppet, OS-specific or language-specific installers like APT or pip, or whatever other system to provision the content of your containers. This is what building a container looks like with regular shell commands and simply using `make` to install a binary inside the container:
```
# pull a base image, equivalent to a Dockerfile's FROM command
buildah from redhat
# mount the base image to work on it
crt=$(buildah mount)
cp foo $crt
make install DESTDIR=$crt
# then make a snapshot
buildah commit
```
An interesting thing with this approach is that, since you reuse normal build tools from the host environment, you can build really minimal images because you don't need to install all the dependencies in the image. Usually, when building a container image, the target application build dependencies need to be installed within the container. For example, building from source usually requires a compiler toolchain in the container, because it is not meant to access the host environment. A lot of containers will also ship basic Unix tools like `ps` or `bash` which are not actually necessary in a micro-container. Developers often forget to (or simply can't) remove some dependencies from the built containers; that common practice creates unnecessary overhead and attack surface.
The modular approach of Buildah means you can run at least parts of the build as non-root: the `mount` command still needs the `CAP_SYS_ADMIN` capability, but there is an [issue][20] open to resolve this. However, Buildah [shares][21] the same [limitation][22] as Docker in that it can't build containers inside containers. For Docker, you need to run the container in "privileged" mode, which is not possible in certain environments (like [GitLab Continuous Integration][23], for example) and, even when it is possible, the configuration is [messy][24] at best.
The manual commit step allows fine-grained control over when to create container snapshots. While in a Dockerfile every line creates a new snapshot, with Buildah commit checkpoints are explicitly chosen, which reduces unnecessary snapshots and saves disk space. This is useful to isolate sensitive material like private keys or passwords which sometimes mistakenly end up in public images as well.
While Docker builds non-standard, Docker-specific images, Buildah produces standard OCI images among [other output formats][25]. For backward compatibility, it has a command called `build-using-dockerfile` or [`buildah bud`][26] that parses normal Dockerfiles. Buildah has a `enter` command to inspect images from the inside directly and a `run` command to start containers on the fly. It does all the work without any "fat daemon" running in the background and uses standard tools like `runc`.
Ducy's criticism of Buildah was that it was not declarative, which made it less reproducible. When allowing shell commands anything can happen: for example, a shell script might download arbitrary binaries, without any way of subsequently retracing where those come from. Shell command effects may vary according to the environment. In contrast to shell-based tools, configuration-management systems like Puppet or Chef are designed to "converge" over a final configuration that is more reliable, at least in theory: in practice you can call shell commands from configuration-management systems. Walsh, however, argued that existing configuration management can be used on top of Buildah, but it doesn't force users down that path. This fits well with the classic "separation" principle of the Unix philosophy ("mechanism not policy").
At this point, Buildah is in beta and Red Hat is working on integrating it into OpenShift. I have tested Buildah while writing this article and, short of some documentation issues, it generally works reliably. It could use some polishing in error handling, but it is definitely a great asset to add to your container toolbox.
### Replacing the rest of the Docker command-line
Walsh continued his presentation by giving an overview of another project that Red Hat is working on, tentatively called [libpod][27]. The name derives from a "pod" in Kubernetes, which is a way to group containers inside a host, to share namespaces, for example.
Libpod includes the `kpod` command to inspect and manipulate container storage directly. Walsh explained this can be useful if, for example, `dockerd` hangs or if a Kubernetes cluster crashes. `kpod` is basically an independent re-implementation of the `docker` command-line tool. There is a command to list running containers (`kpod ps`) or images (`kpod images`). In fact, there is a [translation cheat sheet][28] documenting all Docker commands with a `kpod` equivalent.
One of the nice things with the modular approach is that when you run a container with `kpod run`, the container is directly started as a subprocess of the current shell, instead of a subprocess of `dockerd`. In theory, this allows running containers directly from systemd, removing the duplicate work `dockerd` is doing. It enables things like [socket-activated containers][29], which is something that is [not straightforward][30] to do with Docker, or [even with Kubernetes][31] right now. In my experiments, however, I have found that containers started with `kpod` lack some fundamental functionality, namely networking (!), although there is an [issue in progress][32] to complete that implementation.
A final command we haven't covered is `push`. While the above commands provide a good process for working with local containers, they don't cover remote registries, which allow developers to actively collaborate on application packaging. Registries are also an essential part of a continuous-deployment framework. This is where the [skopeo][33] project comes in. Skopeo is another Atomic project that "performs various operations on container images and image repositories", according to the `README` file. It was originally designed to inspect the contents of container registries without actually downloading the sometimes voluminous images as `docker pull` does. Docker [refused patches][34] to support inspection, suggesting the creation of a separate tool, which led to Skopeo. After `pull`, `push` was the logical next step and Skopeo can now do a bunch of other things like copying and converting images between registries without having to store a copy locally. Because this functionality was useful to other projects as well, a lot of the Skopeo code now lives in a reusable library called [containers/image][35]. That library is in turn used by [Pivotal][36], Google's [container-diff][37], `kpod push`, and `buildah push`.
`kpod` is not directly tied to Kubernetes, so the name might change in the future -- especially since Red Hat legal has not cleared the name yet. (In fact, just as this article was going to "press", the name was changed to [`podman`][38].) The team wants to implement more "pod-level" commands which would allow operations on multiple containers, a bit like what [`docker compose`][39] might do. But at that level, a better tool might be [Kompose][40] which can execute [Compose YAML files][41] into a Kubernetes cluster. Some Docker commands (like [`swarm`][42]) will never be implemented, on purpose, as they are best left for Kubernetes itself to handle.
It seems that the effort to modularize Docker that started a few years ago is finally bearing fruit. While, at this point, `kpod` is under heavy development and probably should not be used in production, the design of those different tools is certainly interesting; a lot of it is ready for development environments. Right now, the only way to install libpod is to compile it from source, but we should expect packages coming out for your favorite distribution eventually.
> This article [first appeared][43] in the [Linux Weekly News][44].
--------------------------------------------------------------------------------
via: https://anarc.at/blog/2017-12-20-docker-without-docker/
作者:[À propos de moi][a]
译者:[译者ID](https://github.com/译者ID)
校对:[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://anarc.at
[1]:https://mobyproject.org/
[2]:https://kccncna17.sched.com/event/CU8j/cri-o-hosted-by-daniel-walsh-red-hat
[3]:http://events.linuxfoundation.org/events/kubecon-and-cloudnativecon-north-america
[4]:https://lwn.net/Articles/741897/
[5]:http://cri-o.io/
[6]:https://kubernetes.io/
[7]:https://www.openshift.com/
[8]:https://kccncna17.sched.com/event/CU6B/building-better-containers-a-survey-of-container-build-tools-i-michael-ducy-chef
[9]:https://github.com/GoogleCloudPlatform/distroless
[10]:https://github.com/oracle/smith
[11]:https://github.com/openshift/source-to-image
[12]:https://www.ansible.com/ansible-container
[13]:https://www.habitat.sh/
[14]:https://nixos.org/nixos/manual/#ch-containers
[15]:https://github.com/moby/buildkit
[16]:https://github.com/projectatomic/buildah
[17]:https://www.projectatomic.io/
[18]:https://raw.githubusercontent.com/projectatomic/buildah/master/logos/buildah-logomark_large.png (Buildah logo)
[19]:https://en.wikipedia.org/wiki/Boston_accent
[20]:https://github.com/projectatomic/buildah/issues/171
[21]:https://github.com/projectatomic/buildah/issues/158
[22]:https://github.com/moby/moby/issues/27886#issuecomment-281278525
[23]:https://about.gitlab.com/features/gitlab-ci-cd/
[24]:https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/
[25]:https://github.com/projectatomic/buildah/blob/master/docs/buildah-push.md
[26]:https://github.com/projectatomic/buildah/blob/master/docs/buildah-bud.md
[27]:https://github.com/projectatomic/libpod
[28]:https://github.com/projectatomic/libpod/blob/master/transfer.md#development-transfer
[29]:http://0pointer.de/blog/projects/socket-activated-containers.html
[30]:https://legacy-developer.atlassian.com/blog/2015/03/docker-systemd-socket-activation/
[31]:https://github.com/kubernetes/kubernetes/issues/484
[32]:https://github.com/projectatomic/libpod/issues/129
[33]:https://github.com/projectatomic/skopeo
[34]:https://github.com/moby/moby/pull/14258
[35]:https://github.com/containers/image
[36]:https://pivotal.io/
[37]:https://github.com/GoogleCloudPlatform/container-diff
[38]:https://github.com/projectatomic/libpod/blob/master/docs/podman.1.md
[39]:https://docs.docker.com/compose/overview/#compose-documentation
[40]:http://kompose.io/
[41]:https://docs.docker.com/compose/compose-file/
[42]:https://docs.docker.com/engine/swarm/
[43]:https://lwn.net/Articles/741841/
[44]:http://lwn.net/

View File

@ -0,0 +1,116 @@
Red Hat 的去 Docker 化容器实践
======
最近几年开源项目Docker (已更名为[Moby][1]) 在容器普及化方面建树颇多。然而,它的功能特性不断集中到一个单一、庞大的系统,该系统由具有 root 权限运行的守护进程 `dockerd` 管控,这引发了人们的焦虑。对这些焦虑的阐述,具有代表性的是 Red Hat 公司的容器团队负责人 Dan Walsh 在 [KubeCon \+ CloudNativecon][3] 会议中的[演讲][2]。Walsh讲述了他的容器团队目前的工作方向即使用一系列更小、可协同工作的组件替代 Docker。他的战斗口号是”拒绝臃肿的守护进程“理由是与公认的 Unix 哲学相违背。
### Docker 模块化实践
就像我们在[早期文献][4]中看到的那样,容器的基础操作不复杂:你首先拉取一个容器镜像,利用该镜像创建一个容器,最后启动这个容器。除此之外,你要懂得如何构建镜像并推送至镜像仓库。大多数人在上述这些步骤中使用 Docker但其实 Docker 并不是唯一的选择,目前的可替换选择是 `rkt`。rkt引发了一系列标准的创建包括运行时标准 CRI镜像标准 OCI 及网络标准 CNI 等。遵守这些标准的后端,如 [CRI-O][5] 和 Docker可以与 [Kubernetes][6] 为代表的管理软件协同工作。
这些标准促使 Red Hat 公司开发了一系列部分实现标准的”核心应用“供 Kubernetes 使用,例如 CRI-O 运行时。但 Kubernetes 提供的功能不足以满足 Red Hat公司的 [OpenShift][7] 项目所需。开发者可能需要构建容器并推送至镜像仓库,实现这些操作需要额外的一整套方案。
事实上,目前市面上已有多种构建容器的工具。来自 Sysdig 公司的 Michael Ducy 在[分会场][8]中回顾了 Docker 本身之外的8种镜像构建工具而这也很可能不是全部的工具。Ducy 将理想的构建工具定义如下可以用可重现的方式创建最小化镜像。最小化镜像并不包含操作系统只包含应用本身及其依赖。Ducy 认为 [Distroless][9], [Smith][10] 及 [Source-to-Image][11] 都是很好的工具可用于构建最小化镜像。Ducy 将最小化镜像称为”微容器“。
可重现镜像是指构建多次结果保持不变的镜像。为达到这个目标Ducy 表示应该使用“宣告式”而不是“命令式”的方式。考虑到 Ducy 来自 Chef 配置管理工具领域你应该能理解他的意思。Ducy 给出了符合标准的几个不错的实现,包括 [Ansible 容器][12], [Habitat][13], [nixos-容器][14]和 [Simth][10] 等但你需要了解这些项目对应的编程语言。Ducy 额外指出 Habitat 构建的容器自带管理功能如果你已经使用了systemd, Docker 或 Kubernetes 等外部管理工具Habitat 的管理功能可能是冗余的。除此之外,我们还要提从 Docker 和 [Buildah][16] 项目诞生的新项目 [BuildKit][15], 它是 Red Hat 公司 [Atomic 工程][17]的一个组件。
### 使用Buildah构建容器
![\[Buildah logo\]][18] Buildah 名称显然来自于 Walsh 风趣的 [Boston 口音][19]; 该工具的品牌宣传中充满了 Boston 风格,例如 logo 使用了 Boston 梗犬(如图所示)。该项目的实现思路与 Ducy 不同:为了构建容器,与其被迫使用宣告式配置管理的方案,不如构建一些简单工具,结合你最喜欢的配置管理工具使用。这样你可以如愿的使用命令行,例如使用 `cp` 命令代替 Docker 的自定义指令 `COPY` 。除此之外你可以使用如下工具为容器提供内容1) 配置管理工具例如Ansible 或 Puppet2) 操作系统相关或编程语言相关的安装工具,例如 APT 和 pip; 3) 其它系统。下面展示了基于通用 shell 命令的容器构建场景,其中只需要使用 `make` 命令即可为容器安装可执行文件。
```
# 拉取基础镜像, 类似 Dockerfile 中的 FROM 命令
buildah from redhat
# 挂载基础镜像, 在其基础上工作
crt=$(buildah mount)
ap foo $crt
make install DESTDIR=$crt
# 下一步,生成快照
buildah commit
```
有趣的是,基于这个思路,你可以复用主机环境中的构建工具,无需在镜像中安装这些依赖,故可以构建非常微小的镜像。通常情况下,构建容器镜像时需要在容器中安装目标应用的构建依赖。例如,从源码构建需要容器中有编译器工具链,这是因为构建并不在主机环境进行。大量的容器也包含了 `ps``bash` 这样的 Unix 命令,对微容器而言其实是多余的。开发者经常忘记或无法从构建好的容器中移除一些依赖,增加了不必要的开销和攻击面。
Buildah的模块化方案能够以非 root 方式进行部分构建;但`mount` 命令仍然需要 `CAP_SYS_ADMIN``等同 root 访问权限` 的能力,有一个 [issue][20] 试图解决该问题。但 Buildah 与 Docker [都有][21]同样的限制[22],即无法在容器内构建容器。对于 Docker你需要使用“特权”模式运行容器一些特殊的环境很难满足这个条件例如 [GitLab 持续集成][23];即使满足该条件,配置也特别[繁琐][24]。
手动提交的步骤可以对创建容器快照的时间节点进行细粒度控制。Dockerfile 每一行都会创建一个新的快照相比而言Buildah 的提交检查点都是事先选择好的,这可以减少不必要的快照并节省磁盘空间。这也有利于隔离私钥或密码等敏感信息,避免其出现在公共镜像中。
Docker 构建的镜像是非标准的、仅供其自身使用相比而言Buildah 提供[多种输出格式][25],其中包括符合 OCI 标准的镜像。为向后兼容Buildah 提供 一个 `使用Dockerfile构建` 的命令,即 [`buildah bud`][26], 它可以解析标准的 Dockerfile。Buildah 提供 `enter` 命令直接查看镜像内部信息,`run` 命令启动一个容器。实现这些功能仅使用了 `runc` 在内的标准工具,无需在后台运行一个“臃肿的守护进程”。
Ducy 对 Buildah 表示质疑,认为采用非宣告性不利于可重现性。如果允许使用 shell 命令,可能产生很多预想不到的情况;例如,一个 shell 脚本下载了任意的可执行程序但后续无法追溯文件的来源。shell 命令的执行受环境变量影响,执行结果可能大相径庭。与基于 shell 的工具相比Puppet 或 Chef 这样的配置管理系统在理论上更加可靠,因为他们的设计初衷就是收敛于最终配置;事实上,可以通过配置管理系统调用 shell 命令。但 Walsh 对此提出反驳,认为已有的配置管理工具可以在 Buildah 的基础上工作,用户可以选择是否使用配置管理;这样更加符合“机制与策略分离”的经典 Unix 哲学。
目前 Buildah 处于测试阶段Red Hat 公司正努力将其集成到 OpenShift。我写这篇文章时已经测试过 Buildah它缺少一些主题的文档但基本可以稳定运行。尽管在错误处理方面仍有待提高但它确实是一款值得你关注的容器工具。
### 替换其它 Docker 命令行
Walsh 在其演讲中还简单介绍了 Redhat 公司 正在开发的另一个暂时叫做 [libpod][24] 的项目。项目名称来源于 Kubernetes 中的 “pod” 在 Kubernetes 中 “pod” 用于分组主机内的容器,分享名字空间等。
Libpod 提供 `kpod` 命令用于直接检查和操作容器存储。Walsh 分析了该命令发挥作用的场景,例如 `dockerd` 停止响应或 Kubernetes 集群崩溃。基本上,`kpod` 独立地再次实现了 `docker` 命令行工具。`kpod ps` 返回运行中的容器列表,`kpod images` 返回镜像列表。事实上,[命令转换速查手册][28] 中给出了每一条 Docker 命令对应的 `kpod` 命令。
这种模块化实现的一个好处是,当你使用 `kpod run` 运行容器时,容器直接作为当前 shell 而不是 `dockerd` 的子进程启动。理论上,可以直接使用 systemd 启动容器,这样可以消除 `dockerd` 引入的冗余。这让[由套接字激活的容器][29]成为可能,但暂时基于 Docker 实现该特性[并不容易][30][即使借助 Kubernetes][31] 也是如此。但我在测试过程中发现,使用 `kpod` 启动的容器有一些基础功能性缺失,具体而言是网络功能(!),相关实现在[活跃开发][32]过程中。
我们最后提到的命令是 `push`。虽然上述命令已经足以满足本地使用容器的需求,但没有提到远程仓库,借助远程仓库开发者可以活跃地进行应用打包协作。仓库也是持续部署框架的核心组件。[skopeo][33] 项目用于填补这个空白,它是另一个 Atomic 成员项目,按其 `README` 文件描述,“包含容器镜像及镜像库的多种操作”。该项目的设计初衷是,在不用类似 `docker pull` 那样实际下载可能体积庞大的镜像的前提下检查容器镜像的内容。Docker [拒绝加入][34] 检查功能,建议通过一个额外的工具实现该功能,这促成了 Skopeo 项目。除了`pull`,`push`Skopeo现在还可以完成很多其它操作例如在不产生本地副本的情况下将镜像在不同的仓库中复制和转换。由于部分功能比较基础可供其它项目使用目前很大一部分 Skopeo 代码位于一个叫做 [containers/image][35] 的基础库。[Pivotal][36], Google的 [container-diff][37] ,`kpod push` 及 `buildah push` 都使用了该库。
`kpod` 与 Kubernetes 并没有紧密的联系,故未来可能会更换名称(事实上,在本文刊发过程中,已经更名为 [`podman`][38]),毕竟 Red Hat 法务部门还没有明确其名称。该团队希望实现更多 pod 级别的命令,这样可以对多个容器进行操作,有点类似于 [`docker compose`][39] 实现的功能。但在这方面,[Kompose][40] 是更好的工具,可以通过 [复合 YAML 文件][41] 在 Kubernetes 集群中运行容器。按计划,我们不会实现类似于 [`swarm`] 的 Docker 命令,这部分功能最好由 Kubernetes 本身完成。
目前看来,已经持续数年的 Docker 模块化努力终将硕果累累。但目前 `kpod` 处于快速迭代过程中,不太适合用于生产环境,但那些工具的众不同的设计理念让人很感兴趣,而且其中大部分的工具已经可以用于开发环境。目前只能通过编译源码的方式安装 libpod但最终会提供各个发行版的二进制包。
> 本文[最初发表][43]于 [Linux Weekly News][44]。
--------------------------------------------------------------------------------
链接: https://anarc.at/blog/2017-12-20-docker-without-docker/
作者:[À propos de moi][a]
译者:[pinewall](https://github.com/pinewall)
校对:[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://anarc.at
[1]:https://mobyproject.org/
[2]:https://kccncna17.sched.com/event/CU8j/cri-o-hosted-by-daniel-walsh-red-hat
[3]:http://events.linuxfoundation.org/events/kubecon-and-cloudnativecon-north-america
[4]:https://lwn.net/Articles/741897/
[5]:http://cri-o.io/
[6]:https://kubernetes.io/
[7]:https://www.openshift.com/
[8]:https://kccncna17.sched.com/event/CU6B/building-better-containers-a-survey-of-container-build-tools-i-michael-ducy-chef
[9]:https://github.com/GoogleCloudPlatform/distroless
[10]:https://github.com/oracle/smith
[11]:https://github.com/openshift/source-to-image
[12]:https://www.ansible.com/ansible-container
[13]:https://www.habitat.sh/
[14]:https://nixos.org/nixos/manual/#ch-containers
[15]:https://github.com/moby/buildkit
[16]:https://github.com/projectatomic/buildah
[17]:https://www.projectatomic.io/
[18]:https://raw.githubusercontent.com/projectatomic/buildah/master/logos/buildah-logomark_large.png (Buildah logo)
[19]:https://en.wikipedia.org/wiki/Boston_accent
[20]:https://github.com/projectatomic/buildah/issues/171
[21]:https://github.com/projectatomic/buildah/issues/158
[22]:https://github.com/moby/moby/issues/27886#issuecomment-281278525
[23]:https://about.gitlab.com/features/gitlab-ci-cd/
[24]:https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/
[25]:https://github.com/projectatomic/buildah/blob/master/docs/buildah-push.md
[26]:https://github.com/projectatomic/buildah/blob/master/docs/buildah-bud.md
[27]:https://github.com/projectatomic/libpod
[28]:https://github.com/projectatomic/libpod/blob/master/transfer.md#development-transfer
[29]:http://0pointer.de/blog/projects/socket-activated-containers.html
[30]:https://legacy-developer.atlassian.com/blog/2015/03/docker-systemd-socket-activation/
[31]:https://github.com/kubernetes/kubernetes/issues/484
[32]:https://github.com/projectatomic/libpod/issues/129
[33]:https://github.com/projectatomic/skopeo
[34]:https://github.com/moby/moby/pull/14258
[35]:https://github.com/containers/image
[36]:https://pivotal.io/
[37]:https://github.com/GoogleCloudPlatform/container-diff
[38]:https://github.com/projectatomic/libpod/blob/master/docs/podman.1.md
[39]:https://docs.docker.com/compose/overview/#compose-documentation
[40]:http://kompose.io/
[41]:https://docs.docker.com/compose/compose-file/
[42]:https://docs.docker.com/engine/swarm/
[43]:https://lwn.net/Articles/741841/
[44]:http://lwn.net/