document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2024-12-26 21:30:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #26125 from lkxed/20210104-Docker-Compose--a-nice-way-to-set-up-a-dev-environment

Browse Source 
[提交译文][tech]: 20210104 Docker Compose- a nice way to set up a dev environment.md
This commit is contained in:
六开箱 2022-06-18 21:26:24 +08:00 committed by GitHub
commit 560d7711f7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 243 additions and 249 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

249
sources/tech/20210104 Docker Compose- a nice way to set up a dev environment.md
View File

@ -1,249 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: (lkxed)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Docker Compose: a nice way to set up a dev environment)
[#]: via: (https://jvns.ca/blog/2021/01/04/docker-compose-is-nice/)
[#]: author: (Julia Evans https://jvns.ca/)
Docker Compose: a nice way to set up a dev environment
======
Hello! Here is another post about [computer tools that I’ve appreciated][1]. This one is about Docker Compose!
This post is mostly just about how delighted I was that it does what it’s supposed to do and it seems to work and to be pretty straightforward to use. I’m also only talking about using Docker Compose for a dev environment here, not using it in production.
I’ve been thinking about this kind of personal dev environment setup more recently because I now do all my computing with a personal cloud budget of like $20/month instead of spending my time at work thinking about how to manage thousands of AWS servers.
I’m very happy about this because previous to trying Docker Compose I spent two days getting frustrated with trying to set up a dev environment with other tools and Docker Compose was a lot easier and simpler. And then I told my sister about my docker-compose experiences and she was like “I KNOW, DOCKER COMPOSE IS GREAT RIGHT?!?!” So I thought I’d write a blog post about it, and here we are.
### the problem: setting up a dev environment
Right now I’m working on a Ruby on Rails service (the backend for a sort of computer debugging game). On my production server, I have:
* a nginx proxy
* a Rails server
* a Go server (which proxies some SSH connections with [gotty][2])
* a Postgres database
Setting up the Rails server locally was pretty straightforward without resorting to containers (I just had to install Postgres and Ruby, fine, no big deal), but then I wanted send `/proxy/*` to the Go server and everything else to the Rails server, so I needed nginx too. And installing nginx on my laptop felt too messy to me.
So enter `docker-compose`!
### docker-compose lets you run a bunch of Docker containers
Docker Compose basically lets you run a bunch of Docker containers that can communicate with each other.
You configure all your containers in one file called `docker-compose.yml`. I’ve pasted my entire `docker-compose.yml` file here for my server because I found it to be really short and straightforward.
```
version: "3.3"
services:
db:
image: postgres
volumes:
- ./tmp/db:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: password # yes I set the password to 'password'
go_server:
# todo: use a smaller image at some point, we don't need all of ubuntu to run a static go binary
image: ubuntu
command: /app/go_proxy/server
volumes:
- .:/app
rails_server:
build: docker/rails
command: bash -c "rm -f tmp/pids/server.pid && source secrets.sh && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/app
web:
build: docker/nginx
ports:
- "8777:80" # this exposes port 8777 on my laptop
```
There are two kinds of containers here: for some of them I’m just using an existing image (`image: postgres` and `image: ubuntu`) without modifying it at all. And for some I needed to build a custom container image – `build: docker/rails` says to use `docker/rails/Dockerfile` to build a custom container.
I needed to give my Rails server access to some API keys and things, so `source secrets.sh` puts a bunch of secrets in environment variables. Maybe there’s a better way to manage secrets but it’s just me so this seemed fine.
### how to start everything: `docker-compose build` then `docker-compose up`
I’ve been starting my containers just by running `docker-compose build` to build the containers, then `docker-compose up` to run everything.
You can set `depends_on` in the yaml file to get a little more control over when things start in, but for my set of services the start order doesn’t matter, so I haven’t.
### the networking is easy to use
It’s important here that the containers be able to connect to each other. Docker Compose makes that super simple! If I have a Rails server running in my `rails_server` container on port 3000, then I can access that with `http://rails_server:3000`. So simple!
Here’s a snippet from my nginx configuration file with how I’m using that in practice (I removed a bunch of `proxy_set_header` lines to make it more clear)
```
location ~ /proxy.* {
proxy_pass http://go_server:8080;
}
location @app {
proxy_pass http://rails_server:3000;
}
```
Or here’s a snippet from my Rails project’s database configuration, where I use the name of the database container (`db`):
```
development:
<<: *default
database: myproject_development
host: db # <-------- this "magically" resolves to the database container's IP address
username: postgres
password: password
```
I got a bit curious about how `rails_server` was actually getting resolved to an IP address. It seems like Docker is running a DNS server somewhere on my computer to resolve these names. Here are some DNS queries where we can see that each container has its own IP address:
```
$ dig +short @127.0.0.11 rails_server
172.18.0.2
$ dig +short @127.0.0.11 db
172.18.0.3
$ dig +short @127.0.0.11 web
172.18.0.4
$ dig +short @127.0.0.11 go_server
172.18.0.5
```
### who’s running this DNS server?
I dug into how this DNS server is set up a very tiny bit.
I ran all these commands outside the container, because I didn’t have a lot of networking tools installed in the container.
**step 1**: find the PID of my Rails server with `ps aux | grep puma`
It’s 1837916. Cool.
**step 2**: find a UDP server running in the same network namespace as PID `1837916`
I did this by using `nsenter` to run `netstat` in the same network namespace as the `puma` process. (technically I guess you could run `netstat -tupn` to just show UDP servers, but my fingers only know how to type `netstat -tulpn` at this point)
```
$ sudo nsenter -n -t 1837916 netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.11:32847 0.0.0.0:* LISTEN 1333/dockerd
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 1837916/puma 4.3.7
udp 0 0 127.0.0.11:59426 0.0.0.0:* 1333/dockerd
```
So there’s a UDP server running on port `59426`, run by `dockerd`! Maybe that’s the DNS server?
**step 3**: check that it’s a DNS server
We can use `dig` to make a DNS query to it:
```
$ sudo nsenter -n -t 1837916 dig +short @127.0.0.11 59426 rails_server
172.18.0.2
```
But – when we ran `dig` earlier, we weren’t making a DNS query to port 59426, we were querying port 53! What’s going on?
**step 4**: iptables
My first guess for “this server seems to be running on port X but I’m accessing it on port Y, what’s going on?” was “iptables”.
So I ran iptables-save in the container’s network namespace, and there we go:
```
$ sudo nsenter -n -t 1837916 iptables-save
.... redacted a bunch of output ....
-A DOCKER_POSTROUTING -s 127.0.0.11/32 -p udp -m udp --sport 59426 -j SNAT --to-source :53
COMMIT
```
There’s an iptables rule that sends traffic on port 53 to 59426. Fun!
### it stores the database files in a temp directory
One nice thing about this is: instead of managing a Postgres installation on my laptop, I can just mount the Postgres container’s data directory at `./tmp/db`.
I like this because I really do not want to administer a Postgres installation on my laptop (I don’t really know how to configure Postgres), and conceptually I like having my dev database literally be in the same directory as the rest of my code.
### I can access the Rails console with `docker-compose exec rails_server rails console`
Managing Ruby versions is always a little tricky and even when I have it working, I always kind of worry I’m going to screw up my Ruby installation and have to spend like ten years fixing it.
With this setup, if I need access to the Rails console (a REPL with all my Rails code loaded), I can just run:
```
$ docker-compose exec rails_server rails console
Running via Spring preloader in process 597
Loading development environment (Rails 6.0.3.4)
irb(main):001:0>
```
Nice!
### small problem: no history in my Rails console
I ran into a problem though: I didn’t have any history in my Rails console anymore, because I was restarting the container all the time.
I figured out a pretty simple solution to this though: I added a `/root/.irbrc` to my container that changed the IRB history file’s location to be something that would persist between container restarts. It’s just one line:
```
IRB.conf[:HISTORY_FILE] = "/app/tmp/irb_history"
```
### I still don’t know how well it works in production
Right now my production setup for this project is still “I made a digitalocean droplet and edited a lot of files by hand”.
I think I’ll try to use docker-compose to run this thing in production. My guess is that it should work fine because this service is probably going to have at most like 2 users at a time and I can easily afford to have 60 seconds of downtime during a deploy if I want, but usually something goes wrong that I haven’t thought of.
A few notes from folks on Twitter about docker-compose in production:
* `docker-compose up` will only restart the containers that need restarting, which makes restarts faster
* there’s a small bash script [wait-for-it][3] that you can use to make a container wait for another service to be available
* You can have 2 docker-compose.yaml files: `docker-compose.yaml` for DEV, and `docker-compose-prod.yaml` for prod. I think I’ll use this to expose different nginx ports: 8999 in dev and 80 in prod.
* folks seemed to agree that docker-compose is fine in production if you have a small website running on 1 computer
* one person suggested that Docker Swarm might be better for a slightly more complicated production setup, but I haven’t tried that (or of course Kubernetes, but the whole point of Docker Compose is that it’s super simple and Kubernetes is certainly not simple :) )
Docker also seems to have a feature to [automatically deploy your docker-compose setup to ECS][4], which sounds cool in theory but I haven’t tried it.
### when doesn’t docker-compose work well?
I’ve heard that docker-compose doesn’t work well:
* when you have a very large number of microservices (a simple setup is best)
* when you’re trying to include data from a very large database (like putting hundreds of gigabytes of data on everyone’s laptop)
* on Mac computers, I’ve heard that Docker can be a lot slower than on Linux (presumably because of the extra VM). I don’t have a Mac so I haven’t run into this.
### that’s all!
I spent an entire day before this trying to configure a dev environment by using Puppet to provision a Vagrant virtual machine only to realize that VMs are kind of slow to start and that I don’t really like writing Puppet configuration (I know, huge surprise :)).
So it was nice to try Docker Compose and find that it was straightforward to get to work!
--------------------------------------------------------------------------------
via: https://jvns.ca/blog/2021/01/04/docker-compose-is-nice/
作者：[Julia Evans][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://jvns.ca/
[b]: https://github.com/lujun9972
[1]: https://jvns.ca/#cool-computer-tools---features---ideas
[2]: https://github.com/yudai/gotty/
[3]: https://github.com/vishnubob/wait-for-it
[4]: https://docs.docker.com/cloud/ecs-integration/

243
translated/tech/20210104 Docker Compose- a nice way to set up a dev environment.md Normal file
View File

@ -0,0 +1,243 @@
[#]: collector: (lujun9972)
[#]: translator: (lkxed)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Docker Compose: a nice way to set up a dev environment)
[#]: via: (https://jvns.ca/blog/2021/01/04/docker-compose-is-nice/)
[#]: author: (Julia Evans https://jvns.ca/)
Docker Compose：搭建开发环境的好办法
======
大家好！我又写了一篇关于 [我最喜欢的电脑工具][1] 的文章。这一篇讲的是 Docker Compose！
这篇文章主要就是讲一讲我对 Docker Compose 有多么满意啦（不讨论它的缺点）！咳咳，因为它总能够完成它该做的，并且似乎总能奏效，更棒的是，它使用起来还非常简单。另外，在本文中，我只讨论我是怎么用 Docker Compose 来搭建开发环境的，而不涉及它在生产中的使用。
最近，我考虑了很多关于这种搭建个人开发环境的方式，原因是，我现在把所有的计算工作都搬到了一个私有云上，大概 20 美元/月的样子。这样一来，我就不用在工作的时候花时间去思考应该如何管理几千台 AWS 服务器了。
在此之前，我曾花了两天的时间，尝试使用其他的工具来尝试搭建一个开发环境，搭到后面，我实在是心累了。相比起来，Docker Compose 就简单易用多了，我非常满意。于是，我和妹妹分享了我的 `docker-compose` 使用经历，她略显惊讶：“是吧！你也觉得 Docker Compose 真棒对吧！” 嗯，我觉得我应该写一篇博文把过程记录下来，于是就有了你们看到的这篇文章。
### 我们的目标是：搭建一个开发环境
目前，我正在编写一个 Ruby on Rails 服务（它是一个计算机“调试”游戏的后端）。在我的生产服务器上，我安装了：
* 一个 Nginx 服务器
* 一个 Rails 服务
* 一个 Go 服务 (使用了 [gotty][2] 来代理一些 SSH 连接)
* 一个 Postgres 数据库
在本地搭建 Rails 服务非常简单，用不着容器（我只需要安装 Postgres 和 Ruby 就行了，小菜一碟）。但是，我还想要把匹配 `/proxy/*` 的请求的发送到 Go 服务，其他所有请求都发送到 Rails 服务，所以我还要用到 Nginx。问题来了，在笔记本电脑上安装 Nginx 对我来说太麻烦了。
是时候使用 `docker-compose` 了！
### docker-compose 允许你运行一组 Docker 容器
基本上，Docker Compose 的作用就是允许你运行一组可以互相通信 Docker 容器。
你可以在一个叫做 `docker-compose.yml` 的文件中，配置你所有的容器。下面，我将贴上我为这个服务编写的 `docker-compose.yml` 文件（的全部内容），因为我觉得它真的很简洁、直接！
```
version: "3.3"
services:
db:
image: postgres
volumes:
- ./tmp/db:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: password # yes I set the password to 'password'
go_server:
# todo: use a smaller image at some point, we don't need all of ubuntu to run a static go binary
image: ubuntu
command: /app/go_proxy/server
volumes:
- .:/app
rails_server:
build: docker/rails
command: bash -c "rm -f tmp/pids/server.pid && source secrets.sh && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/app
web:
build: docker/nginx
ports:
- "8777:80" # this exposes port 8777 on my laptop
```
这个配置包含了两种容器。对于前面两个容器，我不加修改地使用了既有的镜像（`image: postgres` 和 `image: ubuntu`）。对于后面两个，我不得不构建一个自定义容器镜像，其中， `build: docker/rails` 的作用就是告诉 Docker Compose，它应该使用 `docker/rails/Dockerfile` 来构建一个自定义容器。
我需要允许我的 Rails 服务访问一些 API 密钥和其他东西，因此，我使用了 `source secrets.sh`，它的作用就是在环境变量中预设一组密钥。
### 如何启动所有服务：先 “build” 后 “up”
我一直都是先运行 `docker-compose build` 来构建容器，然后再运行 `docker-compose up` 把所有服务启动起来。
你可以在 yaml 文件中设置 `depends_on`，这样你可以获得更多容器启动时的控制。不过，对于我的这些服务而言，启动顺序并不重要，所以我没有设置它。
### 使用网络通信也非常简单
有件很重要的事：容器之间得能够互相连接才行。Docker Compose 让这件事变得超级简单！假设我有一个 Rails 服务正在名为 `rails_server` 的容器中运行，端口是 3000，那么我就可以通过 `http://rails_server:3000` 来访问该服务。就是这么简单！
以下代码片段截取自我的 Nginx 配置文件，它是根据我的使用需求配置的（我删除了许多 `proxy_set_headers` 行，让它看起来更清楚）：
```
location ~ /proxy.* {
proxy_pass http://go_server:8080;
}
location @app {
proxy_pass http://rails_server:3000;
}
```
或者，你也看下面这个代码片段，它截取自我的 Rails 项目的数据库配置，我在其中使用了数据库容器的名称（`db`）：
```
development:
<<: *default
database: myproject_development
host: db # <-------- 它会被“神奇地”解析为数据库容器的 IP 地址
username: postgres
password: password
```
至于 `rails_server` 究竟是如何被解析成一个 IP 地址的，我还真有点儿好奇。貌似是 Docker 在我的计算机上运行了一个 DNS 服务来解析这些名字。下面是一些 DNS 查询记录，我们可以看到，每个容器都有它自己的 IP 地址：
```
$ dig +short @127.0.0.11 rails_server
172.18.0.2
$ dig +short @127.0.0.11 db
172.18.0.3
$ dig +short @127.0.0.11 web
172.18.0.4
$ dig +short @127.0.0.11 go_server
172.18.0.5
```
### 是谁在运行这个 DNS 服务？
我（稍微）研究了一下这个 DNS 服务是怎么搭建起来的。
以下所有命令都是在容器外执行的，因为我没有在容器里安装很多网络工具。
**第一步：**：使用 `ps aux | grep puma`，找到我的 Rails 服务的进程 ID。
找到了，它是 `1837916`！感觉不错哦～
**第二步：**：找到和 `1837916` 运行在同一个网络命名空间的 UDP 服务。
我使用了 `nsenter` 来在 `puma` 进程的网络命令空间内运行 `netstat`（理论上，我猜想你也可以使用 `netstat -tupn` 来只显示 UDP 服务，但此时，我的手指头只会打出 `netstat -tulpn`）。
```
$ sudo nsenter -n -t 1837916 netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.11:32847 0.0.0.0:* LISTEN 1333/dockerd
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 1837916/puma 4.3.7
udp 0 0 127.0.0.11:59426 0.0.0.0:* 1333/dockerd
```
我们可以看到，此时有一个运行在 `59426` 端口的 UDP 服务，它是由 `dockerd` 运行的！或许它就是我们要找的 DNS 服务？
**第三步**：确定它是不是我们要找的 DNS 服务
我们可以使用 `dig` 工具来向它发送一个 DNS 查询：
```
$ sudo nsenter -n -t 1837916 dig +short @127.0.0.11 59426 rails_server
172.18.0.2
```
奇怪，我们之前运行 `dig` 的时候，DNS 查询怎么没有发送到 `59426` 端口，而是发送到了 `53` 端口呢？这到底是怎么回事呀？
**第四步**：iptables
对于类似“这个服务似乎正运行在 X 端口上，但我却在 Y 端口上访问到了它，这是什么回事呢？”的问题，我的第一念头都是“一定是 iptables 在作怪”。
于是，我运行了容器的网络命令空间内运行了 `iptables-save`，果不其然，真相大白：
```
$ sudo nsenter -n -t 1837916 iptables-save
.... redacted a bunch of output ....
-A DOCKER_POSTROUTING -s 127.0.0.11/32 -p udp -m udp --sport 59426 -j SNAT --to-source :53
COMMIT
```
在输出中有一条 iptables 规则，它将 `53` 端口的流量发送到了 `59426` 上。哈哈，真有意思！
### 数据库文件储存在一个临时目录中
这样做有一个好处：我可以直接挂载 Postgres 容器的数据目录 `./tmp/db`，而不需要在我的笔记本电脑上管理 Postgres 环境。
我很喜欢这种方式，因为我真的不想在笔记本电脑上，亲自管理一个 Postgres 环境（我也真的不知道该如何配置 Postgres）。还有就是，出于习惯，我更喜欢让开发环境的数据库和代码放在同一个目录下。
### 仅需一行命令，我就可以访问 Rails 控制台
管理 Ruby 的版本总是有点棘手，并且，即使我暂时搞定了它，我也总是有点担心自己会把 Ruby 环境搞坏，然后就要修它修个十年（夸张）。
（使用 Docker Compose）搭建好这个开发环境后，如果我需要访问 Rails <ruby>控制台<rt>console</rt></ruby>（一个交互式环境，加载了所有我的 Rails 代码），我只需要运行一行代码即可：
```
$ docker-compose exec rails_server rails console
Running via Spring preloader in process 597
Loading development environment (Rails 6.0.3.4)
irb(main):001:0>
```
好耶！
### 小问题：Rails 控制台的历史记录丢失了
我碰到了一个问题：Rails 控制台的历史记录丢失了，因为我一直在不断地重启它。
不过，我也找到了一个相当简单的解决方案（嘿嘿）：我往容器中添加了一个 `/root/.irbrc` 文件，它能够把 IRB 历史记录文件的保存位置，修改到一个不受容器重启影响的地方。只需要一行代码就够啦：
```
IRB.conf[:HISTORY_FILE] = "/app/tmp/irb_history"
```
### 我还是不知道它在生产环境的表现会怎么样
到目前为止，这个项目的生产环境搭建进度，还停留在“我制作了一个 digitalocean droplet（LCCT 译注：一种 Linux 虚拟机服务），并手工编辑了很多文件”的阶段。
嗯……我相信我以后会在生产环境中使用 docker-compose 来运行一下它的。我猜它能够会正常工作，因为这个服务很可能最多只有两个用户在使用，并且，如果我愿意，我可以容忍它在部署过程中有 60 秒的不可用时间。不过话又说回来，出错的往往是我想不到的地方。
推特网友提供了一些在生产中使用 docker-compose 的注意事项：
* `docker-compose up` 只会重启那些需要重启的容器，这会让重启速度更快。
* 有一个 Bash 小脚本 [wait-for-it][3]，你可以用它来让一个容器保持等待，直到另一个容器的服务可用。
* 你可以准备两份 `docker-compose.yaml` 文件：用于开发环境的 `docker-compose.yaml` 和 用于生产环境的 `docker-compose-prod.yaml`。我想我会在分别为 Nginx 指定不同的端口：开发时使用 `8999`，生产中使用 `80`。
* 人们似乎一致认为，如果你的项目是一台计算机上运行的小网站，那么 docker-compose 在生产中不会有问题。
* 有个人建议说，如果愿意在生产环境搭建复杂那么一丢丢，Docker Swarm 就或许会是更好的选择，不过我还没试过（当然，如果要这么说的话，干嘛不用 Kubernetes 呢？Docker Compose 的意义就是它超级简单，而 Kubernetes 肯定不简单 :)）。
Docker 似乎还有一个特性，它能够 [把你用 docker-compose 搭建的环境，自动推送到弹性容器服务（ESC）上][4]，听上去好酷的样子，但是我还没有试过。
### docker-compose 会有不适用的场景吗
我听说 docker-compose 在以下场景的表现差强人意：
* 当你有很多微服务的时候（还是自己搭建比较好）
* 当你尝试从一个很大的数据库中导入数据时（就像把几百 G 的数据存到每个人的笔记本电脑里一样）
* 当你在 Mac 电脑上运行 Docker 时。我听说 Docker 在 macOS 上比在 Linux 上要慢很多（我猜想是因为它需要做额外的虚拟化）。我没有 Mac 电脑，所以我还没有碰到这个问题。
### 以上就是全部内容啦！
在此之前，我曾花了一整天时间，尝试使用 Puppet 来配置 Vagrant 虚拟机，然后在这个虚拟机里配置开发环境。结果，我发现虚拟机启动起来实在是有点慢啊，还有就是，我也不喜欢编写 Puppet 配置（哈哈，没想到吧）。
幸好，我尝试了 Docker Compose，它真好简单，马上就可以开始工作啦！
--------------------------------------------------------------------------------
via: https://jvns.ca/blog/2021/01/04/docker-compose-is-nice/
作者：[Julia Evans][a]
选题：[lujun9972][b]
译者：[lkxed](https://github.com/lkxed)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://jvns.ca/
[b]: https://github.com/lujun9972
[1]: https://jvns.ca/#cool-computer-tools---features---ideas
[2]: https://github.com/yudai/gotty/
[3]: https://github.com/vishnubob/wait-for-it
[4]: https://docs.docker.com/cloud/ecs-integration/