document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2024-12-26 21:30:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

20150128-4 选题

Browse Source
This commit is contained in:
DeadFire 2015-01-28 14:47:57 +08:00
parent 210f5b5c5f
commit 35bdd1c742
3 changed files with 575 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

81
sources/tech/20150128 Docker-1 Moving to Docker.md Normal file
View File

@ -0,0 +1,81 @@
Moving to Docker
================================================================================
![](http://cocoahunter.com/content/images/2015/01/docker1.jpeg)
[TL;DR] This is the first post in a series of 3 on how my company moved its infrastructure from PaaS to Docker based deployment. If you want, you can skip the intro (this post) and head directly to the technical topics (links at the bottom of the page).
----------
In the last month I've been strggling with devops. This is my very personal story and experience in trying to streamline a deployment process of a Raila app with Docker.
When I started my company – [Touchware][1] – in 2012 I was a lone developer. Things were small, uncomplicated, they didn't require a lot of maintenance, nor they needed to scale all that much. During the course of last year though, we grew quite a lot (we are now a team of 10 people) and our server-side applications and API grew both in terms of scope and scale.
### Step 1 - Heroku ###
We still are a very small team and we need to make things going and run as smoothly as possible. When we looked for possible solutions, we decided to stick with something that would have removed from our shoulders the burden of managing hardware. Since we develop mainly Rails based applications and Heroku has a great support for RoR and various kind of DBs and cached (Postgres / Mongo / Redis etc.), the smartest choice seemed to be going with [Heroku][2]. And that's what we did.
Heroku has a great support and great documentation and deploying apps is just so snappy! Only problem is, when you start growing, you need to have piles of cash around to pay the bills. Not the best deal, really.
### Step 2 - Dokku ###
In a rush to try and cut the costs, we decided to try with Dokku. [Dokku][3], quoting the Github repo is a
> Docker powered mini-Heroku in around 100 lines of Bash
We launched some instances on [DigitalOcean][4] with Dokku pre-installed and we gave it spin. Dokku is very much like Heroku, but when you have complex applications for whom you need to twear params, or where you need certain dependencies, it's just not gonna work out. We had an app where we needed to apply multiple transformations on images and we couldn't find a way to install the correct version of imagemagick into the dokku-based Docker container that was hosting our Rails app. We still have a couple of very simple apps that are running on Dokku, but we had to move some of them back to Heroku.
### Step 3 - Docker ###
A couple of months ago, since the problem of devops and managing production apps was resurfacing, I decided to try out [Docker][5]. Docker, in simple terms, allows developers to containerize applications and to ease the deployment. Since a Docker container basically has all the dependencies it needs to run your app, if everything runs fine on your laptop, you can be sure it'll also run like a champ in production on a remote server, be it an AWS E2C instance or a VPS on DigitalOcean.
Docker IMHO is particularly interesting for the following reasons:
- it promotes modularization and separation of concerns: you need to start thinking about your apps in terms of logical components (load balancer: 1 container, DB: 1 container, webapp: 1 container etc.);
- it's very flexible in terms of deployment options: containers can be deployed to a wide variety of HW and can be easily redeployed to different servers / providers;
- it allows for a very fine grained tuning of your app environment: you build the images your containers runs from, so you have plenty of options for configuring your environment exactly as you would like to.
There are howerver some downsides:
- the learning curve is quite steep (this is probably a very personal problem, but I'm talking as a software dev and not as a skilled operations professional);
- setup is not simple, especially if you want to have a private registry / repository (more about this later).
Following are some tips I put together during the course of the last week with the findings of someone that is new to the game.
----------
In the following articles we'll see how to setup a semi-automated Docker based deployment system.
- [Setting up a private Docker registry][6]
- [Configuring a Rails app for semi-automated deployment][7]
--------------------------------------------------------------------------------
via: http://cocoahunter.com/2015/01/23/docker-1/
作者：[Michelangelo Chasseur][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译，[Linux中国](http://linux.cn/) 荣誉推出
[a]:http://cocoahunter.com/author/michelangelo/
[1]:http://www.touchwa.re/
[2]:http://cocoahunter.com/2015/01/23/docker-1/www.heroku.com
[3]:https://github.com/progrium/dokku
[4]:http://cocoahunter.com/2015/01/23/docker-1/www.digitalocean.com
[5]:http://www.docker.com/
[6]:http://cocoahunter.com/2015/01/23/docker-2/
[7]:http://cocoahunter.com/2015/01/23/docker-3/
[8]:
[9]:
[10]:
[11]:
[12]:
[13]:
[14]:
[15]:
[16]:
[17]:
[18]:
[19]:
[20]:

241
sources/tech/20150128 Docker-2 Setting up a private Docker registry.md Normal file
View File

@ -0,0 +1,241 @@
Setting up a private Docker registry
================================================================================
![](http://cocoahunter.com/content/images/2015/01/docker2.jpg)
[TL;DR] This is the second post in a series of 3 on how my company moved its infrastructure from PaaS to Docker based deployment.
- [First part][1]: where I talk about the process we went thru before approaching Docker;
- [Third pard][2]: where I show how to automate the entire process of building images and deploying a Rails app with Docker.
----------
Why would ouy want ot set up a provate registry? Well, for starters, Docker Hub only allows you to have one free private repo. Other companies are beginning to offer similar services, but they are all not very cheap. In addition, if you need to deploy production ready applications built with Docker, you might not want to publish those images on the public Docker Hub.
This is a very pragmatic approach to dealing with the intricacies of setting up a private Docker registry. For the tutorial we will be using a small 512MB instance on DigitalOcean (from now on DO). I also assume you already know the basics of Docker since I will be concentrating on some more complicated stuff.
### Local set up ###
First of all you need to install **boot2docker** and docker CLI. If you already have your basic Docker environment up and running, you can just skip to the next section.
From the terminal run the following command[1][3]:
brew install boot2docker docker
If everything is ok[2][4], you will now be able to start the VM inside which Docker will run with the following command:
boot2docker up
Follow the instructions, copy and paste the export commands that boot2docker will print in the terminal. If you now run `docker ps` you should be greeted by the following line
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Ok, Docker is ready to go. This will be enough for the moment. Let's go back to setting up the registry.
### Creating the server ###
Log into you DO account and create a new Droplet by selecting an image with Docker pre-installed[^n].
![](http://cocoahunter.com/content/images/2015/01/Screenshot-2015-01-20-18-26-14.png)
You should receive your root credentials via email. Log into your instance and run `docker ps` to see if eveything is ok.
### Setting up AWS S3 ###
We are going to use Amazon Simple Storage Service (S3) as the storage layer for our registry / repository. We will need to create a bucket and user credentials to allow our docker container accessoing it.
Login into your AWS account (if you don't have one you can set one up at [http://aws.amazon.com/][5]) and from the console select S3 (Simple Storage Service).
![](http://cocoahunter.com/content/images/2015/01/Screenshot-2015-01-20-19-29-21.png)
Click on **Create Bucket**, enter a unique name for your bucket (and write it down, we're gonna need it later), then click on **Create**.
![](http://cocoahunter.com/content/images/2015/01/Screenshot-2015-01-20-19-22-50.png)
That's it! We're done setting up the storage part.
### Setup AWS access credentials ###
We are now going to create a new user. Go back to your AWS console and select IAM (Identity & Access Management).
![](http://cocoahunter.com/content/images/2015/01/Screenshot-2015-01-20-19-29-08.png)
In the dashboard, on the left side of the webpage, you should click on Users. Then select **Create New Users**.
You should be presented with the following screen:
![](http://cocoahunter.com/content/images/2015/01/Screenshot-2015-01-20-19-31-42.png)
Enter a name for your user (e.g. docker-registry) and click on Create. Write down (or download the csv file with) your Access Key and Secret Access Key that we'll need when running the Docker container. Go back to your users list and select the one you just created.
Under the Permission section, click on Attach User Policy. In the next screen, you will be presented with multiple choices: select Custom Policy.
![](http://cocoahunter.com/content/images/2015/01/Screenshot-2015-01-20-19-41-21.png)
Here's the content of the custom policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SomeStatement",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::docker-registry-bucket-name/*",
"arn:aws:s3:::docker-registry-bucket-name"
]
}
]
}
This will allow the user (i.e. the registry) to manage (read/write) content on the bucket (make sure to use the bucket name you previously defined when setting up AWS S3). To sum it up: when you'll be pushing Docker images from your local machine to your repository, the server will be able to upload them to S3.
### Installing the registry ###
Now let's head back to our DO server and SSH into it. We are going to use[^n] one of the [official Docker registry images][6].
Let's start our registry with the following command:
docker run \
-e SETTINGS_FLAVOR=s3 \
-e AWS_BUCKET=bucket-name \
-e STORAGE_PATH=/registry \
-e AWS_KEY=your_aws_key \
-e AWS_SECRET=your_aws_secret \
-e SEARCH_BACKEND=sqlalchemy \
-p 5000:5000 \
--name registry \
-d \
registry
Docker should pull the required fs layers from the Docker Hub and eventually start the daemonised container.
### Testing the registry ###
If everything worked out, you should now be able to test the registry by pinging it and by searching its content (though for the time being it's still empty).
Our registry is very basic and it does not provide any means of authentication. Since there are no easy ways of adding authentication (at least none that I'm aware of that are easy enough to implment in order to justify the effort), I've decided that the easiest way of querying / pulling / pushing the registry is an unsecure (over HTTP) connection tunneled thru SSH.
Opening an SSH tunnel from your local machine is straightforward:
ssh -N -L 5000:localhost:5000 root@your_registry.com
The command is tunnelling connections over SSH from port 5000 of the registry server (which is the one we exposed with the `docker run` command in the previous paragraph) to port 5000 on the localhost.
If you now browse to the following address [http://localhost:5000/v1/_ping][7] you should get the following very simple response
{}
This just means that the registry is working correctly. You can also list the whole content of the registry by browsing to [http://localhost:5000/v1/search][8] that will get you a similar response:
{
"num_results": 2,
"query": "",
"results": [
{
"description": "",
"name": "username/first-repo"
},
{
"description": "",
"name": "username/second-repo"
}
]
}
### Building an image ###
Let's now try and build a very simple Docker image to test our newly installed registry. On your local machine, create a Dockerfile with the following content[^n]:
# Base image with ruby 2.2.0
FROM ruby:2.2.0
MAINTAINER Michelangelo Chasseur <michelangelo.chasseur@touchwa.re>
...and build it:
docker build -t localhost:5000/username/repo-name .
The `localhost:5000` part is especially important: the first part of the name of a Docker image will tell the `docker push` command the endpoint towards which we are trying to push our image. In our case, since we are connecting to our remote private registry via an SSH tunnel, `localhost:5000` represents exactly the reference to our registry.
If everything works as expected, when the command returns, you should be able to list your newly created image with the `docker images` command. Run it and see it for yourself.
### Pushing to the registry ###
Now comes the trickier part. It took a me a while to realize what I'm about to describe, so just be patient if you don't get it the first time you read and try to follow along. I know that all this stuff will seem pretty complicated (and it would be if you didn't automate the process), but I promise in the end it will all make sense. In the next post I will show a couple of shell scripts and Rake tasks that will automate the whole process and will let you deploy a Rails to your registry app with a single easy command.
The docker command you are running from your terminal is actually using the boot2docker VM to run the containers and do all the magic stuff. So when we run a command like `docker push some_repo` what is actually happening is that it's the boot2docker VM that is reacing out for the registry, not our localhost.
This is an extremely important point to understand: in order to push the Docker image to the remote private registry, the SSH tunnel needs to be established from the boot2docker VM and not from your local machine.
There are a couple of ways to go with it. I will show you the shortest one (which is not probably the easiest to understand, but it's the one that will let us automate the process with shell scripts).
First of all though we need to sort one last thing with SSH.
### Setting up SSH ###
Let's add our boot2docker SSH key to our remote server (registry) known hosts. We can do so using the ssh-copy-id utility that you can install with the following command shouldn't you already have it:
brew install ssh-copy-id
Then run:
ssh-copy-id -i /Users/username/.ssh/id_boot2docker root@your-registry.com
Make sure to substitute `/Users/username/.ssh/id_boot2docker` with the correct path of your ssh key.
This will allow us to connect via SSH to our remote registry without being prompted for the password.
Finally let's test it out:
boot2docker ssh "ssh -o 'StrictHostKeyChecking no' -i /Users/michelangelo/.ssh/id_boot2docker -N -L 5000:localhost:5000 root@registry.touchwa.re &" &
To break things out a little bit:
- `boot2docker ssh` lets you pass a command as a parameter that will be executed by the boot2docker VM;
- the final `&` indicates that we want our command to be executed in the background;
- `ssh -o 'StrictHostKeyChecking no' -i /Users/michelangelo/.ssh/id_boot2docker -N -L 5000:localhost:5000 root@registry.touchwa.re &` is the actual command our boot2docker VM will run;
- the `-o 'StrictHostKeyChecking no'` will make sure that we are not prompted with security questions;
- the `-i /Users/michelangelo/.ssh/id_boot2docker` indicates which SSH key we want our VM to use for authentication purposes (note that this should be the key you added to your remote registry in the previous step);
- finally we are opening a tunnel on mapping port 5000 to localhost:5000.
### Pulling from another server ###
You should now be able to push your image to the remote registry by simply issuing the following command:
docker push localhost:5000/username/repo_name
In the [next post][9] we'll se how to automate some of this stuff and we'll containerize a real Rails application. Stay tuned!
P.S. Please use the comments to let me know of any inconsistencies or fallacies in my tutorial. Hope you enjoyed it!
1. I'm also assuming you are running on OS X.
1. For a complete list of instructions to set up your docker environment and requirements, please visit [http://boot2docker.io/][10]
1. Select Image > Applications > Docker 1.4.1 on 14.04 at the time of this writing.
1. [https://github.com/docker/docker-registry/][11]
1. This is just a stub, in the next post I will show you how to bundle a Rails application into a Docker container.
--------------------------------------------------------------------------------
via: http://cocoahunter.com/2015/01/23/docker-2/
作者：[Michelangelo Chasseur][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译，[Linux中国](http://linux.cn/) 荣誉推出
[a]:http://cocoahunter.com/author/michelangelo/
[1]:http://cocoahunter.com/2015/01/23/docker-1/
[2]:http://cocoahunter.com/2015/01/23/docker-3/
[3]:http://cocoahunter.com/2015/01/23/docker-2/#fn:1
[4]:http://cocoahunter.com/2015/01/23/docker-2/#fn:2
[5]:http://aws.amazon.com/
[6]:https://registry.hub.docker.com/_/registry/
[7]:http://localhost:5000/v1/_ping
[8]:http://localhost:5000/v1/search
[9]:http://cocoahunter.com/2015/01/23/docker-3/
[10]:http://boot2docker.io/
[11]:https://github.com/docker/docker-registry/

253
sources/tech/20150128 Docker-3 Automated Docker-based Rails deployments.md Normal file
View File

@ -0,0 +1,253 @@
Automated Docker-based Rails deployments
================================================================================
![](http://cocoahunter.com/content/images/2015/01/docker3.jpeg)
[TL;DR] This is the third post in a series of 3 on how my company moved its infrastructure from PaaS to Docker based deployment.
- [First part][1]: where I talk about the process we went thru before approaching Docker;
- [Second part][2]: where I explain how setting up a private registry for in house secure deployments.
----------
In this final part we will see how to automate the whole deployment process with a real world (though very basic) example.
### Basic Rails app ###
Let's dive into the topic right away and bootstrap a basic Rails app. For the purpose of this demonstration I'm going to use Ruby 2.2.0 and Rails 4.1.1
From the terminal run:
$ rvm use 2.2.0
$ rails new && cd docker-test
Let's create a basic controller:
$ rails g controller welcome index
...and edit `routes.rb` so that the root of the project will point to our newly created welcome#index method:
root 'welcome#index'
Running `rails s` from the terminal and browsing to [http://localhost:3000][3] should bring you to the index page. We're not going to make anything fancier to the app, it's just a basic example to prove that when we'll build and deploy the container everything is working.
### Setup the webserver ###
We are going to use Unicorn as our webserver. Add `gem 'unicorn'` and `gem 'foreman'` to the Gemfile and bundle it up (run `bundle install` from the command line).
Unicorn needs to be configured when the Rails app launches, so let's put a **unicorn.rb** file inside the **config** directory. [Here is an example][4] of a Unicorn configuration file. You can just copy & paste the content of the Gist.
Let's also add a Procfile with the following content inside the root of the project so that we will be able to start the app with foreman:
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
If you now try to run the app with **foreman start** everything should work as expected and you should have a running app on [http://localhost:5000][5]
### Building a Docker image ###
Now let's build the image inside which our app is going to live. In the root of our Rails project, create a file named **Dockerfile** and paste in it the following:
# Base image with ruby 2.2.0
FROM ruby:2.2.0
# Install required libraries and dependencies
RUN apt-get update && apt-get install -qy nodejs postgresql-client sqlite3 --no-install-recommends && rm -rf /var/lib/apt/lists/*
# Set Rails version
ENV RAILS_VERSION 4.1.1
# Install Rails
RUN gem install rails --version "$RAILS_VERSION"
# Create directory from where the code will run
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Make webserver reachable to the outside world
EXPOSE 3000
# Set ENV variables
ENV PORT=3000
# Start the web app
CMD ["foreman","start"]
# Install the necessary gems
ADD Gemfile /usr/src/app/Gemfile
ADD Gemfile.lock /usr/src/app/Gemfile.lock
RUN bundle install --without development test
# Add rails project (from same dir as Dockerfile) to project directory
ADD ./ /usr/src/app
# Run rake tasks
RUN RAILS_ENV=production rake db:create db:migrate
Using the provided Dockerfile, let's try and build an image with the following command[1][7]:
$ docker build -t localhost:5000/your_username/docker-test .
And again, if everything worked out correctly, the last line of the long log output should read something like:
Successfully built 82e48769506c
$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
localhost:5000/your_username/docker-test latest 82e48769506c About a minute ago 884.2 MB
Let's try and run the container!
$ docker run -d -p 3000:3000 --name docker-test localhost:5000/your_username/docker-test
You should be able to reach your Rails app running inside the Docker container at port 3000 of your boot2docker VM[2][8] (in my case [http://192.168.59.103:3000][6]).
### Automating with shell scripts ###
Since you should already know from the previous post3 how to push your newly created image to a private regisitry and deploy it on a server, let's skip this part and go straight to automating the process.
We are going to define 3 shell scripts and finally tie it all together with rake.
### Clean ###
Every time we build our image and deploy we are better off always clean everything. That means the following:
- stop (if running) and restart boot2docker;
- remove orphaned Docker images (images that are without tags and that are no longer used by your containers).
Put the following into a **clean.sh** file in the root of your project.
echo Restarting boot2docker...
boot2docker down
boot2docker up
echo Exporting Docker variables...
sleep 1
export DOCKER_HOST=tcp://192.168.59.103:2376
export DOCKER_CERT_PATH=/Users/user/.boot2docker/certs/boot2docker-vm
export DOCKER_TLS_VERIFY=1
sleep 1
echo Removing orphaned images without tags...
docker images | grep "<none>" | awk '{print $3}' | xargs docker rmi
Also make sure to make the script executable:
$ chmod +x clean.sh
### Build ###
The build process basically consists in reproducing what we just did before (docker build). Create a **build.sh** script at the root of your project with the following content:
docker build -t localhost:5000/your_username/docker-test .
Make the script executable.
### Deploy ###
Finally, create a **deploy.sh** script with this content:
# Open SSH connection from boot2docker to private registry
boot2docker ssh "ssh -o 'StrictHostKeyChecking no' -i /Users/username/.ssh/id_boot2docker -N -L 5000:localhost:5000 root@your-registry.com &" &
# Wait to make sure the SSH tunnel is open before pushing...
echo Waiting 5 seconds before pushing image.
echo 5...
sleep 1
echo 4...
sleep 1
echo 3...
sleep 1
echo 2...
sleep 1
echo 1...
sleep 1
# Push image onto remote registry / repo
echo Starting push!
docker push localhost:5000/username/docker-test
If you don't understand what's going on here, please make sure you've read thoroughfully [part 2][9] of this series of posts.
Make the script executable.
### Tying it all together with rake ###
Having 3 scripts would now require you to run them individually each time you decide to deploy your app:
1. clean
1. build
1. deploy / push
That wouldn't be much of an effort, if it weren't for the fact that developers are lazy! And lazy be it, then!
The final step to wrap things up, is tying the 3 parts together with rake.
To make things even simpler you can just append a bunch of lines of code to the end of the already present Rakefile in the root of your project. Open the Rakefile file - pun intended :) - and paste the following:
namespace :docker do
desc "Remove docker container"
task :clean do
sh './clean.sh'
end
desc "Build Docker image"
task :build => [:clean] do
sh './build.sh'
end
desc "Deploy Docker image"
task :deploy => [:build] do
sh './deploy.sh'
end
end
Even if you don't know rake syntax (which you should, because it's pretty awesome!), it's pretty obvious what we are doing. We have declared 3 tasks inside a namespace (docker).
This will create the following 3 tasks:
- rake docker:clean
- rake docker:build
- rake docker:deploy
Deploy is dependent on build, build is dependent on clean. So every time we run from the command line
$ rake docker:deploy
All the script will be executed in the required order.
### Test it ###
To see if everything is working, you just need to make a small change in the code of your app and run
$ rake docker:deploy
and see the magic happening. Once the image has been uploaded (and the first time it could take quite a while), you can ssh into your production server and pull (thru an SSH tunnel) the docker image onto the server and run. It's that easy!
Well, maybe it takes a while to get accustomed to how everything works, but once it does, it's almost (almost) as easy as deploying with Heroku.
P.S. As always, please let me have your ideas. I'm not sure this is the best, or the fastest, or the safest way of doing devops with Docker, but it certainly worked out for us.
- make sure to have **boot2docker** up and running.
- If you don't know your boot2docker VM address, just run `$ boot2docker ip`
- if you don't, you can read it [here][10]
--------------------------------------------------------------------------------
via: http://cocoahunter.com/2015/01/23/docker-3/
作者：[Michelangelo Chasseur][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译，[Linux中国](http://linux.cn/) 荣誉推出
[a]:http://cocoahunter.com/author/michelangelo/
[1]:http://cocoahunter.com/docker-1
[2]:http://cocoahunter.com/2015/01/23/docker-2/
[3]:http://localhost:3000/
[4]:https://gist.github.com/chasseurmic/0dad4d692ff499761b20
[5]:http://localhost:5000/
[6]:http://192.168.59.103:3000/
[7]:http://cocoahunter.com/2015/01/23/docker-3/#fn:1
[8]:http://cocoahunter.com/2015/01/23/docker-3/#fn:2
[9]:http://cocoahunter.com/2015/01/23/docker-2/
[10]:http://cocoahunter.com/2015/01/23/docker-2/