From 9b8af9305ab8da75fe8cdafa6c72a9539787c3fc Mon Sep 17 00:00:00 2001 From: wenchunyang Date: Mon, 4 Jul 2016 11:05:23 +0800 Subject: [PATCH 0001/2437] =?UTF-8?q?=C3=A2Finish=20tranlating=20awk=20ser?= =?UTF-8?q?ies=20part4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Comparison Operators with Awk in Linux.md | 97 ------------------- ... Comparison Operators with Awk in Linux.md | 95 ++++++++++++++++++ 2 files changed, 95 insertions(+), 97 deletions(-) delete mode 100644 sources/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md create mode 100644 translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md diff --git a/sources/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md b/sources/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md deleted file mode 100644 index 1d0ef007af..0000000000 --- a/sources/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md +++ /dev/null @@ -1,97 +0,0 @@ -chunyang-wen translating - -How to Use Comparison Operators with Awk in Linux -=================================================== - -![](http://www.tecmint.com/wp-content/uploads/2016/05/Use-Comparison-Operators-with-AWK.png) - -When dealing with numerical or string values in a line of text, filtering text or strings using comparison operators comes in handy for Awk command users. - -In this part of the Awk series, we shall take a look at how you can filter text or strings using comparison operators. If you are a programmer then you must already be familiar with comparison operators but those who are not, let me explain in the section below. - -### What are Comparison operators in Awk? - -Comparison operators in Awk are used to compare the value of numbers or strings and they include the following: - -- `>` – greater than -- `<` – less than -- `>=` – greater than or equal to -- `<=` – less than or equal to -- `==` – equal to -- `!=` – not equal to -- `some_value ~ / pattern/` – true if some_value matches pattern -- `some_value !~ / pattern/` – true if some_value does not match pattern - -Now that we have looked at the various comparison operators in Awk, let us understand them better using an example. - -In this example, we have a file named food_list.txt which is a shopping list for different food items and I would like to flag food items whose quantity is less than or equal 20 by adding `(**)` at the end of each line. - -``` -File – food_list.txt -No Item_Name Quantity Price -1 Mangoes 45 $3.45 -2 Apples 25 $2.45 -3 Pineapples 5 $4.45 -4 Tomatoes 25 $3.45 -5 Onions 15 $1.45 -6 Bananas 30 $3.45 -``` - -The general syntax for using comparison operators in Awk is: - -``` -# expression { actions; } -``` - -To achieve the above goal, I will have to run the command below: - -``` -# awk '$3 <= 30 { printf "%s\t%s\n", $0,"**" ; } $3 > 30 { print $0 ;}' food_list.txt - -No Item_Name` Quantity Price -1 Mangoes 45 $3.45 -2 Apples 25 $2.45 ** -3 Pineapples 5 $4.45 ** -4 Tomatoes 25 $3.45 ** -5 Onions 15 $1.45 ** -6 Bananas 30 $3.45 ** -``` - -In the above example, there are two important things that happen: - -- The first expression `{ action ; }` combination, `$3 <= 30 { printf “%s\t%s\n”, $0,”**” ; }` prints out lines with quantity less than or equal to 30 and adds a `(**)` at the end of each line. The value of quantity is accessed using `$3` field variable. -- The second expression `{ action ; }` combination, `$3 > 30 { print $0 ;}` prints out lines unchanged since their quantity is greater then `30`. - -One more example: - -``` -# awk '$3 <= 20 { printf "%s\t%s\n", $0,"TRUE" ; } $3 > 20 { print $0 ;} ' food_list.txt - -No Item_Name Quantity Price -1 Mangoes 45 $3.45 -2 Apples 25 $2.45 -3 Pineapples 5 $4.45 TRUE -4 Tomatoes 25 $3.45 -5 Onions 15 $1.45 TRUE -6 Bananas 30 $3.45 -``` - -In this example, we want to indicate lines with quantity less or equal to 20 with the word (TRUE) at the end. - -### Summary - -This is an introductory tutorial to comparison operators in Awk, therefore you need to try out many other options and discover more. - -In case of any problems you face or any additions that you have in mind, then drop a comment in the comment section below. Remember to read the next part of the Awk series where I will take you through compound expressions. - --------------------------------------------------------------------------------- - -via: http://www.tecmint.com/comparison-operators-in-awk/ - -作者:[Aaron Kili][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.tecmint.com/author/aaronkili/ diff --git a/translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md b/translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md new file mode 100644 index 0000000000..ef4f17f1d5 --- /dev/null +++ b/translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md @@ -0,0 +1,95 @@ +在 Linux 下如何使用 Awk 比较操作符 +=================================================== + +![](http://www.tecmint.com/wp-content/uploads/2016/05/Use-Comparison-Operators-with-AWK.png) + +对于 Awk 命令的用户来说,处理一行文本中的数字或者字符串时,使用比较运算符来过滤文本和字符串是十分方便的。 + +在 Awk 系列的此部分中,我们将探讨一下如何使用比较运算符来过滤文本或者字符串。如果你是程序员,那么你应该已经熟悉比较运算符;对于其它人,下面的部分将介绍比较运算符。 + +### Awk 中的比较运算符是什么? + +Awk 中的比较运算符用于比较字符串和或者数值,包括以下类型: + +- `>` – 大于 +- `<` – 小于 +- `>=` – 大于等于 +- `<=` – 小于等于 +- `==` – 等于 +- `!=` – 不等于 +- `some_value ~ / pattern/` – 如果some_value匹配模式pattern,则返回true +- `some_value !~ / pattern/` –如果some_value不匹配模式pattern,则返回true + +现在我们通过例子来熟悉 Awk 中各种不同的比较运算符。 + +在这个例子中,我们有一个文件名为 food_list.txt 的文件,里面包括不同食物的购买列表。我想给食物数量小于或等于30的物品所在行的后面加上`(**)` + +``` +File – food_list.txt +No Item_Name Quantity Price +1 Mangoes 45 $3.45 +2 Apples 25 $2.45 +3 Pineapples 5 $4.45 +4 Tomatoes 25 $3.45 +5 Onions 15 $1.45 +6 Bananas 30 $3.45 +``` + +Awk 中使用比较运算符的通用语法如下: + +``` +# expression { actions; } +``` + +为了实现刚才的目的,执行下面的命令: + +``` +# awk '$3 <= 30 { printf "%s\t%s\n", $0,"**" ; } $3 > 30 { print $0 ;}' food_list.txt + +No Item_Name` Quantity Price +1 Mangoes 45 $3.45 +2 Apples 25 $2.45 ** +3 Pineapples 5 $4.45 ** +4 Tomatoes 25 $3.45 ** +5 Onions 15 $1.45 ** +6 Bananas 30 $3.45 ** +``` + +在刚才的例子中,发生如下两件重要的事情: + +- 第一表达式 `{ action ; }` 组合, `$3 <= 30 { printf “%s\t%s\n”, $0,”**” ; }` 打印出数量小于等于30的行,并且在后面增加`(**)`。物品的数量是通过 `$3`这个域变量获得的。 +- 第二个表达式 `{ action ; }` 组合, `$3 > 30 { print $0 ;}` 原样输出数量小于等于 `30` 的行。 + +再举一个例子: + +``` +# awk '$3 <= 20 { printf "%s\t%s\n", $0,"TRUE" ; } $3 > 20 { print $0 ;} ' food_list.txt + +No Item_Name Quantity Price +1 Mangoes 45 $3.45 +2 Apples 25 $2.45 +3 Pineapples 5 $4.45 TRUE +4 Tomatoes 25 $3.45 +5 Onions 15 $1.45 TRUE +6 Bananas 30 $3.45 +``` + +在这个例子中,我们想通过在行的末尾增加 (TRUE) 来标记数量小于等于20的行。 + +### 总结 + +这是一篇对 Awk 中的比较运算符介绍性的指引,因此你需要尝试其他选项,发现更多使用方法。 + +如果你遇到或者想到任何问题,请在下面评论区留下评论。请记得阅读 Awk 系列下一部分的文章,那里我将介绍组合表达式。 + +-------------------------------------------------------------------------------- + +via: http://www.tecmint.com/comparison-operators-in-awk/ + +作者:[Aaron Kili][a] +译者:[chunyang-wen](https://github.com/chunyang-wen) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/author/aaronkili/ From 1a2ad60c48090971b924cd3e96d86095df3c23d3 Mon Sep 17 00:00:00 2001 From: Chunyang Wen Date: Mon, 4 Jul 2016 11:15:31 +0800 Subject: [PATCH 0002/2437] Update Part 4 - How to Use Comparison Operators with Awk in Linux.md --- ...t 4 - How to Use Comparison Operators with Awk in Linux.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md b/translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md index ef4f17f1d5..86649a52d5 100644 --- a/translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md +++ b/translated/tech/awk/Part 4 - How to Use Comparison Operators with Awk in Linux.md @@ -18,7 +18,7 @@ Awk 中的比较运算符用于比较字符串和或者数值,包括以下类 - `==` – 等于 - `!=` – 不等于 - `some_value ~ / pattern/` – 如果some_value匹配模式pattern,则返回true -- `some_value !~ / pattern/` –如果some_value不匹配模式pattern,则返回true +- `some_value !~ / pattern/` – 如果some_value不匹配模式pattern,则返回true 现在我们通过例子来熟悉 Awk 中各种不同的比较运算符。 @@ -57,7 +57,7 @@ No Item_Name` Quantity Price 在刚才的例子中,发生如下两件重要的事情: -- 第一表达式 `{ action ; }` 组合, `$3 <= 30 { printf “%s\t%s\n”, $0,”**” ; }` 打印出数量小于等于30的行,并且在后面增加`(**)`。物品的数量是通过 `$3`这个域变量获得的。 +- 第一个表达式 `{ action ; }` 组合, `$3 <= 30 { printf “%s\t%s\n”, $0,”**” ; }` 打印出数量小于等于30的行,并且在后面增加`(**)`。物品的数量是通过 `$3`这个域变量获得的。 - 第二个表达式 `{ action ; }` 组合, `$3 > 30 { print $0 ;}` 原样输出数量小于等于 `30` 的行。 再举一个例子: From 80cc470eef1d07cf5dae5d0c513f72d9649c26de Mon Sep 17 00:00:00 2001 From: wenchunyang Date: Tue, 5 Jul 2016 10:03:18 +0800 Subject: [PATCH 0003/2437] translating: How to Hide Linux Command Line History by Going Incognito --- ... How to Hide Linux Command Line History by Going Incognito.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md b/sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md index 274ff3b175..0d5ce6e6e1 100644 --- a/sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md +++ b/sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md @@ -1,3 +1,4 @@ +chunyang-wen translating How to Hide Linux Command Line History by Going Incognito ================================================================ From 4e3cf59a1987f19f1f2240882da4a73309b9c92a Mon Sep 17 00:00:00 2001 From: wenchunyang Date: Tue, 12 Jul 2016 13:49:34 +0800 Subject: [PATCH 0004/2437] finish translating How to Hide Linux Command Line History by Going Incognito --- ...Command Line History by Going Incognito.md | 125 ------------------ ...Command Line History by Going Incognito.md | 124 +++++++++++++++++ 2 files changed, 124 insertions(+), 125 deletions(-) delete mode 100644 sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md create mode 100644 translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md diff --git a/sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md b/sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md deleted file mode 100644 index 0d5ce6e6e1..0000000000 --- a/sources/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md +++ /dev/null @@ -1,125 +0,0 @@ -chunyang-wen translating -How to Hide Linux Command Line History by Going Incognito -================================================================ - -![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/06/commandline-history-featured.jpg) - -If you’re a Linux command line user, you’ll agree that there are times when you do not want certain commands you run to be recorded in the command line history. There could be many reasons for this. For example, you’re at a certain position in your company, and you have some privileges that you don’t want others to abuse. Or, there are some critical commands that you don’t want to run accidentally while you’re browsing the history list. - -But is there a way to control what goes into the history list and what doesn’t? Or, in other words, can we turn on a web browser-like incognito mode in the Linux command line? The answer is yes, and there are many ways to achieve this, depending on what exactly you want. In this article we will discuss some of the popular solutions available. - -Note: all the commands presented in this article have been tested on Ubuntu. - -### Different ways available - -The first two ways we’ll describe here have already been covered in [one of our previous articles][1]. If you are already aware of them, you can skip over these. However, if you aren’t aware, you’re advised to go through them carefully. - -#### 1. Insert space before command - -Yes, you read it correctly. Insert a space in the beginning of a command, and it will be ignored by the shell, meaning the command won’t be recorded in history. However, there’s a dependency – the said solution will only work if the HISTCONTROL environment variable is set to “ignorespace” or “ignoreboth,” which is by default in most cases. - -So, a command like the following: - -``` -[space]echo "this is a top secret" -``` - -Won’t appear in the history if you’ve already done this command: - -``` -export HISTCONTROL = ignorespace -``` - -The below screenshot is an example of this behavior. - -![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/06/commandline-history-bash-command-space.png) - -The fourth “echo” command was not recorded in the history as it was run with a space in the beginning. - -#### 2. Disable the entire history for the current session - -If you want to disable the entire history for a session, you can easily do that by unsetting the HISTSIZE environment variable before you start with your command line work. To unset the variable run the following command: - -``` -export HISTFILE=0 -``` - -HISTFILE is the number of lines (or commands) that can be stored in the history list for an ongoing bash session. By default, this variable has a set value – for example, 1000 in my case. - -So, the command mentioned above will set the environment variable’s value to zero, and consequently nothing will be stored in the history list until you close the terminal. Keep in mind that you’ll also not be able to see the previously run commands by pressing the up arrow key or running the history command. - -#### 3. Erase the entire history after you’re done - -This can be seen as an alternative to the solution mentioned in the previous section. The only difference is that in this case you run a command AFTER you’re done with all your work. Thh following is the command in question: - -``` -history -cw -``` - -As already mentioned, this will have the same effect as the HISTFILE solution mentioned above. - -#### 4. Turn off history only for the work you do - -While the solutions (2 and 3) described above do the trick, they erase the entire history, something which might be undesired in many situations. There might be cases in which you want to retain the history list up until the point you start your command line work. For situations like these you need to run the following command before starting with your work: - -``` -[space]set +o history -``` - -Note: [space] represents a blank space. - -The above command will disable the history temporarily, meaning whatever you do after running this command will not be recorded in history, although all the stuff executed prior to the above command will be there as it is in the history list. - -To re-enable the history, run the following command: - -``` -[Space]set -o history -``` - -This brings things back to normal again, meaning any command line work done after the above command will show up in the history. - -#### 5. Delete specific commands from history - -Now suppose the history list already contains some commands that you didn’t want to be recorded. What can be done in this case? It’s simple. You can go ahead and remove them. The following is how to accomplish this: - -``` -[space]history | grep "part of command you want to remove" -``` - -The above command will output a list of matching commands (that are there in the history list) with a number [num] preceding each of them. - -Once you’ve identified the command you want to remove, just run the following command to remove that particular entry from the history list: - -``` -history -d [num] -``` - -The following screenshot is an example of this. - -![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/06/commandline-history-delete-specific-commands.png) - -The second ‘echo’ command was removed successfully. - -Alternatively, you can just press the up arrow key to take a walk back through the history list, and once the command of your interest appears on the terminal, just press “Ctrl + U” to totally blank the line, effectively removing it from the list. - -### Conclusion - -There are multiple ways in which you can manipulate the Linux command line history to suit your needs. Keep in mind, however, that it’s usually not a good practice to hide or remove a command from history, although it’s also not wrong, per se, but you should be aware of what you’re doing and what effects it might have. - --------------------------------------------------------------------------------- - -via: https://www.maketecheasier.com/linux-command-line-history-incognito/?utm_medium=feed&utm_source=feedpress.me&utm_campaign=Feed%3A+maketecheasier - -作者:[Himanshu Arora][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://www.maketecheasier.com/author/himanshu/ -[1]: https://www.maketecheasier.com/command-line-history-linux/ - - - - - diff --git a/translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md b/translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md new file mode 100644 index 0000000000..83e7a33e8b --- /dev/null +++ b/translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md @@ -0,0 +1,124 @@ +如何进入无痕模式进而隐藏 Linux 的命令行历史 +================================================================ + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/06/commandline-history-featured.jpg) + +如果你是 Linux 命令行的用户,你会同意有的时候你不希望某些命令记录在你的命令行历史中。其中原因可能很多。例如,你在公司处于某个职位,你有一些不希望被其它人滥用的特权。亦或者有些特别重要的命令,你不希望在你浏览历史列表时误执行。 + +然而,有方法可以控制哪些命令进入历史列表,哪些不进入吗?或者换句话说,我们在 Linux 终端中可以开启像浏览器一样的无痕模式吗?答案是肯定的,而且根据你想要的具体目标,有很多实现方法。在这篇文章中,我们将讨论一些行之有效的方法。 + +注意:文中出现的所有命令都在 Ubuntu 下测试过。 + +### 不同的可行方法 + +前面两种方法已经在之前[一篇文章][1]中描述了。如果你已经了解,这部分可以略过。然而,如果你不了解,建议仔细阅读。 + +#### 1. 在命令前插入空格 + +是的,没看错。在命令前面插入空格,这条命令会被终端忽略,也就意味着它不会出现在历史记录中。但是这种方法有个前提,只有在你的环境变量 HISTCONTROL 设置为 "ignorespace" 或者 "ignoreboth" 才会起作用。在大多数情况下,这个是默认值。 + +所以,像下面的命令: + +``` +[space]echo "this is a top secret" +``` + +你执行后,它不会出现在历史记录中。 + +``` +export HISTCONTROL = ignorespace +``` + +下面的截图是这种方式的一个例子。 + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/06/commandline-history-bash-command-space.png) + +第四个 "echo" 命令因为前面有空格,它没有被记录到历史中。 + +#### 2. 禁用当前会话的所有历史记录 + +如果你想禁用某个会话所有历史,你可以简单地在开始命令行工作前清除环境变量 HISTSIZE 的值。执行下面的命令来清除其值: + +``` +export HISTFILE=0 +``` + +HISTFILE 表示对于 bash 会话其历史中可以保存命令的个数。默认情况,它设置了一个非零值,例如 在我的电脑上,它的值为 1000。 + +所以上面所提到的命令将其值设置为 0,结果就是直到你关闭终端,没有东西会存储在历史记录中。记住同样你也不能通过按向上的箭头按键来执行之前的命令,也不能运行 history 命令。 + +#### 3. 工作结束后清除整个历史 + +这可以看作是前一部分所提方案的另外一种实现。唯一的区别是在你完成所有工作之后执行这个命令。下面是刚讨论的命令: + +``` +history -cw +``` + +刚才已经提到,这个和 HISTFILE 方法有相同效果。 + +#### 4. 只针对你的工作关闭历史记录 + +虽然前面描述的方法(2 和 3)可以实现目的,它们清除整个历史,在很多情况下,有些可能不是我们所期望的。有时候你可能想保存直到你开始命令行工作之间的历史记录。类似的需求需要在你开始工作前执行下述命令: + +``` +[space]set +o history +``` + +备注:[space] 表示空格。 + +上面的命令会临时禁用历史功能,这意味着在这命令之后你执行的所有操作都不会记录到历史中,然而这个命令之前的所有东西都会原样记录在历史列表中。 + +要重新开启历史功能,执行下面的命令: + +``` +[Space]set -o history +``` + +它将环境恢复原状,也就是你完成你的工作,执行上述命令之后的命令都会出现在历史中。 + +#### 5. 从历史记录中删除指定的命令 + +现在假设历史记录中有一些命令你不希望被记录。这种情况下我们怎么办?很简单。直接动手删除它们。通过下面的命令来删除: + +``` +[space]history | grep "part of command you want to remove" +``` + +上面的命令会输出历史记录中匹配的命令,每一条前面会有个数字。 + +一旦你找到你想删除的命令,执行下面的命令,从历史记录中删除那个指定的项: + +``` +history -d [num] +``` + +下面是这个例子的截图。 + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/06/commandline-history-delete-specific-commands.png) + +第二个 ‘echo’命令被成功的删除了。 + +同样的,你可以使用向上的箭头一直往回翻看历史记录。当你发现你感兴趣的命令出现在终端上时,按下 “Ctrl + U”清除整行,也会从历史记录中删除它。 + +### 总结 + +有多种不同的方法可以操作 Linux 命令行历史来满足你的需求。然而请记住,从历史中隐藏或者删除命令通常不是一个好习惯,尽管本质上这并没有错。但是你必须知道你在做什么,以及可能产生的后果。 + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/linux-command-line-history-incognito/?utm_medium=feed&utm_source=feedpress.me&utm_campaign=Feed%3A+maketecheasier + +作者:[Himanshu Arora][a] +译者:[译者ID](https://github.com/chunyang-wen) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.maketecheasier.com/author/himanshu/ +[1]: https://www.maketecheasier.com/command-line-history-linux/ + + + + + From 140cf4068722acd5c2bffb12f86ff536dbfb110e Mon Sep 17 00:00:00 2001 From: wenchunyang Date: Tue, 12 Jul 2016 13:52:18 +0800 Subject: [PATCH 0005/2437] update translator --- ...How to Hide Linux Command Line History by Going Incognito.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md b/translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md index 83e7a33e8b..9ef701559b 100644 --- a/translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md +++ b/translated/tech/20160625 How to Hide Linux Command Line History by Going Incognito.md @@ -110,7 +110,7 @@ history -d [num] via: https://www.maketecheasier.com/linux-command-line-history-incognito/?utm_medium=feed&utm_source=feedpress.me&utm_campaign=Feed%3A+maketecheasier 作者:[Himanshu Arora][a] -译者:[译者ID](https://github.com/chunyang-wen) +译者:[chunyang-wen](https://github.com/chunyang-wen) 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From 24f6895b50bab921ac74cdd4fad72ec715333829 Mon Sep 17 00:00:00 2001 From: wenchunyang Date: Wed, 13 Jul 2016 09:59:55 +0800 Subject: [PATCH 0006/2437] translating game part2 and 3 --- ...g online multiplayer game with python and asyncio - part 2.md | 1 + ...g online multiplayer game with python and asyncio - part 3.md | 1 + 2 files changed, 2 insertions(+) diff --git a/sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md b/sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md index a33b0b0fdc..75df13d9b5 100644 --- a/sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md +++ b/sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md @@ -1,3 +1,4 @@ +chunyang-wen translating Writing online multiplayer game with python and asyncio - Part 2 ================================================================== diff --git a/sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md b/sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md index 90cf71c466..3c38af72ac 100644 --- a/sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md +++ b/sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md @@ -1,3 +1,4 @@ +chunyang-wen translating Writing online multiplayer game with python and asyncio - Part 3 ================================================================= From 65dd7a91fef6e9b951e83ced1782883d25888280 Mon Sep 17 00:00:00 2001 From: cposture Date: Fri, 5 Aug 2016 02:08:01 +0800 Subject: [PATCH 0007/2437] 75 --- ...18 An Introduction to Mocking in Python.md | 33 ++++++++++--------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/sources/tech/20160618 An Introduction to Mocking in Python.md b/sources/tech/20160618 An Introduction to Mocking in Python.md index d28fae272d..cdf9c24a69 100644 --- a/sources/tech/20160618 An Introduction to Mocking in Python.md +++ b/sources/tech/20160618 An Introduction to Mocking in Python.md @@ -223,8 +223,8 @@ class RemovalServiceTestCase(unittest.TestCase): mock_os.remove.assert_called_with("any path") ``` -很好,我们知道 RemovalService 会如期工作。接下来让我们创建另一个服务,将其声明为一个依赖 -Great, so we now know that the RemovalService works as planned. Let’s create another service which declares it as a dependency: +很好,我们知道 RemovalService 会如期工作。接下来让我们创建另一个服务,将 RemovalService 声明为一个依赖 +: ``` #!/usr/bin/env python @@ -250,18 +250,18 @@ class UploadService(object): self.removal_service.rm(filename) ``` -Since we already have test coverage on the RemovalService, we’re not going to validate internal functionality of the rm method in our tests of UploadService. Rather, we’ll simply test (without side-effects, of course) that UploadService calls the RemovalService.rm method, which we know “just works™” from our previous test case. +因为我们的测试覆盖了 RemovalService,因此我们不会对我们测试用例中 UploadService 的内部函数 rm 进行验证。相反,我们将调用 UploadService 的 RemovalService.rm 方法来进行简单测试(当然是没有其他副作用),我们通过之前的测试用例便能知道它可以正确地工作。 -There are two ways to go about this: +这里有两种方法来实现它: -1. Mock out the RemovalService.rm method itself. -2. Supply a mocked instance in the constructor of UploadService. +1. 模拟 RemovalService.rm 方法本身。 +2. 在 UploadService 的构造函数中提供一个模拟实例。 -As both methods are often important in unit-testing, we’ll review both. +因为这两种方法都是单元测试中非常重要的方法,所以我们将同时对这两种方法进行回顾。 -### Option 1: Mocking Instance Methods +### 方法 1:模拟实例的方法 -The mock library has a special method decorator for mocking object instance methods and properties, the @mock.patch.object decorator: +该模拟库有一个特殊的方法装饰器,可以模拟对象势力的方法和属性,即 @mock.patch.object decorator: ``` #!/usr/bin/env python @@ -314,11 +314,12 @@ class UploadServiceTestCase(unittest.TestCase): removal_service.rm.assert_called_with("my uploaded file") ``` -Great! We’ve validated that the UploadService successfully calls our instance’s rm method. Notice anything interesting in there? The patching mechanism actually replaced the rm method of all RemovalService instances in our test method. That means that we can actually inspect the instances themselves. If you want to see more, try dropping in a breakpoint in your mocking code to get a good feel for how the patching mechanism works. +非常棒!我们验证了 UploadService 成功调用了我们实例的 rm 方法。你是否注意到一些有趣的地方?这种修补机制实际上替换了我们测试用例中的所有 RemovalService 实例的 rm 方法。这意味着我们可以检查实例本身。如果你想要了解更多,可以试着在你模拟的代码下断点,以对这种修补机制的原理获得更好的认识。 -### Pitfall: Decorator Order +### 陷阱:装饰顺序 + +当我们在测试方法中使用多个装饰器,其顺序是很重要的,并且很容易混乱。基本桑,当装饰器被映射到方法参数时,[装饰器的工作顺序是反向的][3]。思考这个例子: -When using multiple decorators on your test methods, order is important, and it’s kind of confusing. Basically, when mapping decorators to method parameters, [work backwards][3]. Consider this example: ``` @mock.patch('mymodule.sys') @@ -328,16 +329,17 @@ When using multiple decorators on your test methods, order is important, and it pass ``` -Notice how our parameters are matched to the reverse order of the decorators? That’s partly because of [the way that Python works][4]. With multiple method decorators, here’s the order of execution in pseudocode: +注意到我们的参数和装饰器的顺序是反向匹配了吗?这多多少少是由 [Python 的工作方式][4] 导致的。这里是使用多个装饰器的情况下它们的执行顺序的伪代码: ``` patch_sys(patch_os(patch_os_path(test_something))) ``` -Since the patch to sys is the outermost patch, it will be executed last, making it the last parameter in the actual test method arguments. Take note of this well and use a debugger when running your tests to make sure that the right parameters are being injected in the right order. +因为 sys 补丁位于最外层,所以它会最晚执行,使得它成为实际测试方法参数的最后一个参数。请特别注意这一点,并且在运行你的测试用例时,使用调试器来保证正确的参数以正确的顺序注入。 -### Option 2: Creating Mock Instances +### 方法 2:创建 Mock 实例 +我们可以使用构造函数为 UploadService 提供一个 Mock 实例,而不是模拟特定的实例方法。我推荐上面的方法 1,因为它更加精确,但在多数情况,方法 2 或许更加有效和必要。让我们再次重构测试用例: Instead of mocking the specific instance method, we could instead just supply a mocked instance to UploadService with its constructor. I prefer option 1 above, as it’s a lot more precise, but there are many cases where option 2 might be efficient or necessary. Let’s refactor our test again: ``` @@ -387,6 +389,7 @@ class UploadServiceTestCase(unittest.TestCase): mock_removal_service.rm.assert_called_with("my uploaded file") ``` +在这个例子中,我们甚至不需要补充任何功能,只需创建一个 In this example, we haven’t even had to patch any functionality, we simply create an auto-spec for the RemovalService class, and then inject this instance into our UploadService to validate the functionality. The [mock.create_autospec][5] method creates a functionally equivalent instance to the provided class. What this means, practically speaking, is that when the returned instance is interacted with, it will raise exceptions if used in illegal ways. More specifically, if a method is called with the wrong number of arguments, an exception will be raised. This is extremely important as refactors happen. As a library changes, tests break and that is expected. Without using an auto-spec, our tests will still pass even though the underlying implementation is broken. From e254672ba1b66dee06de9e82b5cc312ec76c0029 Mon Sep 17 00:00:00 2001 From: cposture Date: Fri, 5 Aug 2016 14:09:10 +0800 Subject: [PATCH 0008/2437] Translated by cposture --- ...18 An Introduction to Mocking in Python.md | 101 +++++++++--------- 1 file changed, 48 insertions(+), 53 deletions(-) rename {sources => translated}/tech/20160618 An Introduction to Mocking in Python.md (56%) diff --git a/sources/tech/20160618 An Introduction to Mocking in Python.md b/translated/tech/20160618 An Introduction to Mocking in Python.md similarity index 56% rename from sources/tech/20160618 An Introduction to Mocking in Python.md rename to translated/tech/20160618 An Introduction to Mocking in Python.md index cdf9c24a69..b8daaed937 100644 --- a/sources/tech/20160618 An Introduction to Mocking in Python.md +++ b/translated/tech/20160618 An Introduction to Mocking in Python.md @@ -1,32 +1,34 @@ Mock 在 Python 中的使用介绍 ===================================== -http://www.oschina.net/translate/an-introduction-to-mocking-in-python?cmp + 本文讲述的是 Python 中 Mock 的使用 -**如何在避免测试你的耐心的情景下执行单元测试** +**如何在避免测试你的耐心的情况下执行单元测试** -通常,我们编写的软件会直接与我们称之为肮脏无比的服务交互。用外行人的话说:交互已设计好的服务对我们的应用程序很重要,但是这会带来我们不希望的副作用,也就是那些在我们自己测试的时候不希望的功能。例如:我们正在写一个社交 app,并且想要测试一下我们 "发布到 Facebook" 的新功能,但是不想每次运行测试集的时候真的发布到 Facebook。 +很多时候,我们编写的软件会直接与那些被标记为肮脏无比的服务交互。用外行人的话说:交互已设计好的服务对我们的应用程序很重要,但是这会给我们带来不希望的副作用,也就是那些在一个自动化测试运行的上下文中不希望的功能。 + +例如:我们正在写一个社交 app,并且想要测试一下 "发布到 Facebook" 的新功能,但是不想每次运行测试集的时候真的发布到 Facebook。 -Python 的单元测试库包含了一个名为 unittest.mock 或者可以称之为依赖的子包,简言之为 mock——其提供了极其强大和有用的方法,通过它们可以模拟和打桩我们不希望的副作用。 +Python 的 `unittest` 库包含了一个名为 `unittest.mock` 或者可以称之为依赖的子包,简称为 +`mock` —— 其提供了极其强大和有用的方法,通过它们可以模拟和打桩来去除我们不希望的副作用。 >Source | -注意:mock [最近收录][1]到了 Python 3.3 的标准库中;先前发布的版本必须通过 [PyPI][2] 下载 Mock 库。 +> 注意:`mock` [最近收录][1]到了 Python 3.3 的标准库中;先前发布的版本必须通过 [PyPI][2] 下载 Mock 库。 -### -### Fear System Calls +### 恐惧系统调用 -再举另一个例子,思考一个我们会在余文讨论的系统调用。不难发现,这些系统调用都是主要的模拟对象:无论你是正在写一个可以弹出 CD 驱动的脚本,还是一个用来删除 /tmp 下过期的缓存文件的 Web 服务,这些调用都是在你的单元测试上下文中不希望的副作用。 +再举另一个例子,思考一个我们会在余文讨论的系统调用。不难发现,这些系统调用都是主要的模拟对象:无论你是正在写一个可以弹出 CD 驱动的脚本,还是一个用来删除 /tmp 下过期的缓存文件的 Web 服务,或者一个绑定到 TCP 端口的 socket 服务器,这些调用都是在你的单元测试上下文中不希望的副作用。 > 作为一个开发者,你需要更关心你的库是否成功地调用了一个可以弹出 CD 的系统函数,而不是切身经历 CD 托盘每次在测试执行的时候都打开了。 作为一个开发者,你需要更关心你的库是否成功地调用了一个可以弹出 CD 的系统函数(使用了正确的参数等等),而不是切身经历 CD 托盘每次在测试执行的时候都打开了。(或者更糟糕的是,很多次,在一个单元测试运行期间多个测试都引用了弹出代码!) -同样,保持你的单元测试的效率和性能意味着需要让如此多的 "缓慢代码" 远离自动测试,比如文件系统和网络访问。 +同样,保持单元测试的效率和性能意味着需要让如此多的 "缓慢代码" 远离自动测试,比如文件系统和网络访问。 -对于我们首个例子,我们要从原始形式到使用 mock 地重构一个标准 Python 测试用例。我们会演示如何使用 mock 写一个测试用例使我们的测试更加智能、快速,并且能展示更多关于我们软件的工作原理。 +对于首个例子,我们要从原始形式到使用 `mock` 重构一个标准 Python 测试用例。我们会演示如何使用 mock 写一个测试用例,使我们的测试更加智能、快速,并展示更多关于我们软件的工作原理。 ### 一个简单的删除函数 @@ -42,9 +44,9 @@ def rm(filename): os.remove(filename) ``` -很明显,我们的 rm 方法此时无法提供比相关 os.remove 方法更多的功能,但我们的基础代码会逐步改善,允许我们在这里添加更多的功能。 +很明显,我们的 `rm` 方法此时无法提供比 `os.remove` 方法更多的相关功能,但我们可以在这里添加更多的功能,使我们的基础代码逐步改善。 -让我们写一个传统的测试用例,即,没有使用 mock: +让我们写一个传统的测试用例,即,没有使用 `mock`: ``` #!/usr/bin/env python @@ -71,7 +73,7 @@ class RmTestCase(unittest.TestCase): self.assertFalse(os.path.isfile(self.tmpfilepath), "Failed to remove the file.") ``` -我们的测试用例相当简单,但是当它每次运行的时候,它都会创建一个临时文件并且随后删除。此外,我们没有办法测试我们的 rm 方法是否正确地将我们的参数向下传递给 os.remove 调用。我们可以基于以上的测试认为它做到了,但还有很多需要改进的地方。 +我们的测试用例相当简单,但是在它每次运行的时候,它都会创建一个临时文件并且随后删除。此外,我们没有办法测试我们的 `rm` 方法是否正确地将我们的参数向下传递给 `os.remove` 调用。我们可以基于以上的测试认为它做到了,但还有很多需要改进的地方。 ### 使用 Mock 重构 @@ -95,29 +97,25 @@ class RmTestCase(unittest.TestCase): mock_os.remove.assert_called_with("any path") ``` -使用这些重构,我们从根本上改变了该测试用例的运行方式。现在,我们有一个可以用于验证其他功能的内部对象。 +使用这些重构,我们从根本上改变了测试用例的操作方式。现在,我们有一个可以用于验证其他功能的内部对象。 ### 潜在陷阱 -第一件需要注意的事情就是,我们使用了位于 mymodule.os 且用于模拟对象的 mock.patch 方法装饰器,并且将该 mock 注入到我们的测试用例方法。相比在 mymodule.os 引用它,那么只是模拟 os 本身,会不会更有意义呢? -One of the first things that should stick out is that we’re using the mock.patch method decorator to mock an object located at mymodule.os, and injecting that mock into our test case method. Wouldn’t it make more sense to just mock os itself, rather than the reference to it at mymodule.os? +第一件需要注意的事情就是,我们使用了 `mock.patch` 方法装饰器,用于模拟位于 `mymodule.os` 的对象,并且将 mock 注入到我们的测试用例方法。那么只是模拟 `os` 本身,而不是 `mymodule.os` 下 `os` 的引用(注意 `@mock.patch('mymodule.os')` 便是模拟 `mymodule.os` 下的 `os`,译者注),会不会更有意义呢? -当然,当涉及到导入和管理模块,Python 的用法非常灵活。在运行时,mymodule 模块拥有被导入到本模块局部作用域的 os。因此,如果我们模拟 os,我们是看不到模拟在 mymodule 模块中的作用的。 +当然,当涉及到导入和管理模块,Python 的用法非常灵活。在运行时,`mymodule` 模块拥有被导入到本模块局部作用域的 `os`。因此,如果我们模拟 `os`,我们是看不到 mock 在 `mymodule` 模块中的作用的。 这句话需要深刻地记住: > 模拟测试一个项目,只需要了解它用在哪里,而不是它从哪里来。 -> Mock an item where it is used, not where it came from. -如果你需要为 myproject.app.MyElaborateClass 模拟 tempfile 模块,你可能需要 -If you need to mock the tempfile module for myproject.app.MyElaborateClass, you probably need to apply the mock to myproject.app.tempfile, as each module keeps its own imports. +如果你需要为 `myproject.app.MyElaborateClass` 模拟 `tempfile` 模块,你可能需要将 mock 用于 `myproject.app.tempfile`,而其他模块保持自己的导入。 先将那个陷阱置身事外,让我们继续模拟。 -With that pitfall out of the way, let’s keep mocking. ### 向 ‘rm’ 中加入验证 -之前定义的 rm 方法相当的简单。在盲目地删除之前,我们倾向于拿它来验证一个路径是否存在,并验证其是否是一个文件。让我们重构 rm 使其变得更加智能: +之前定义的 rm 方法相当的简单。在盲目地删除之前,我们倾向于验证一个路径是否存在,并验证其是否是一个文件。让我们重构 rm 使其变得更加智能: ``` @@ -132,7 +130,7 @@ def rm(filename): os.remove(filename) ``` -很好。现在,让我们调整测试用例来保持测试的覆盖程度。 +很好。现在,让我们调整测试用例来保持测试的覆盖率。 ``` #!/usr/bin/env python @@ -168,9 +166,9 @@ class RmTestCase(unittest.TestCase): ### 将文件删除作为服务 -到目前为止,我们只是对函数功能提供模拟测试,并没对需要传递参数的对象和实例的方法进行模拟测试。接下来我们将介绍如何对对象的方法进行模拟测试。 +到目前为止,我们只是将 mock 应用在函数上,并没应用在需要传递参数的对象和实例的方法。我们现在开始涵盖对象的方法。 -首先,我们将rm方法重构成一个服务类。实际上将这样一个简单的函数转换成一个对象,在本质上,这不是一个合理的需求,但它能够帮助我们了解mock的关键概念。让我们开始重构: +首先,我们将 `rm` 方法重构成一个服务类。实际上将这样一个简单的函数转换成一个对象,在本质上这不是一个合理的需求,但它能够帮助我们了解 `mock` 的关键概念。让我们开始重构: ``` #!/usr/bin/env python @@ -187,8 +185,7 @@ class RemovalService(object): os.remove(filename) ``` -### 你会注意到我们的测试用例没有太大的变化 -### You’ll notice that not much has changed in our test case: +你会注意到我们的测试用例没有太大变化: ``` #!/usr/bin/env python @@ -223,7 +220,7 @@ class RemovalServiceTestCase(unittest.TestCase): mock_os.remove.assert_called_with("any path") ``` -很好,我们知道 RemovalService 会如期工作。接下来让我们创建另一个服务,将 RemovalService 声明为一个依赖 +很好,我们知道 `RemovalService` 会如期工作。接下来让我们创建另一个服务,将 `RemovalService` 声明为它的一个依赖: : ``` @@ -250,9 +247,9 @@ class UploadService(object): self.removal_service.rm(filename) ``` -因为我们的测试覆盖了 RemovalService,因此我们不会对我们测试用例中 UploadService 的内部函数 rm 进行验证。相反,我们将调用 UploadService 的 RemovalService.rm 方法来进行简单测试(当然是没有其他副作用),我们通过之前的测试用例便能知道它可以正确地工作。 +因为我们的测试覆盖了 `RemovalService`,因此我们不会对我们测试用例中 `UploadService` 的内部函数 `rm` 进行验证。相反,我们将调用 `UploadService` 的 `RemovalService.rm` 方法来进行简单测试(当然没有其他副作用),我们通过之前的测试用例便能知道它可以正确地工作。 -这里有两种方法来实现它: +这里有两种方法来实现测试: 1. 模拟 RemovalService.rm 方法本身。 2. 在 UploadService 的构造函数中提供一个模拟实例。 @@ -261,7 +258,7 @@ class UploadService(object): ### 方法 1:模拟实例的方法 -该模拟库有一个特殊的方法装饰器,可以模拟对象势力的方法和属性,即 @mock.patch.object decorator: +`mock` 库有一个特殊的方法装饰器,可以模拟对象实例的方法和属性,即 `@mock.patch.object decorator` 装饰器: ``` #!/usr/bin/env python @@ -314,33 +311,32 @@ class UploadServiceTestCase(unittest.TestCase): removal_service.rm.assert_called_with("my uploaded file") ``` -非常棒!我们验证了 UploadService 成功调用了我们实例的 rm 方法。你是否注意到一些有趣的地方?这种修补机制实际上替换了我们测试用例中的所有 RemovalService 实例的 rm 方法。这意味着我们可以检查实例本身。如果你想要了解更多,可以试着在你模拟的代码下断点,以对这种修补机制的原理获得更好的认识。 +非常棒!我们验证了 UploadService 成功调用了我们实例的 rm 方法。你是否注意到一些有趣的地方?这种修补机制(patching mechanism)实际上替换了我们测试用例中的所有 `RemovalService` 实例的 `rm` 方法。这意味着我们可以检查实例本身。如果你想要了解更多,可以试着在你模拟的代码下断点,以对这种修补机制的原理获得更好的认识。 ### 陷阱:装饰顺序 -当我们在测试方法中使用多个装饰器,其顺序是很重要的,并且很容易混乱。基本桑,当装饰器被映射到方法参数时,[装饰器的工作顺序是反向的][3]。思考这个例子: +当我们在测试方法中使用多个装饰器,其顺序是很重要的,并且很容易混乱。基本上,当装饰器被映射到方法参数时,[装饰器的工作顺序是反向的][3]。思考这个例子: ``` -@mock.patch('mymodule.sys') + @mock.patch('mymodule.sys') @mock.patch('mymodule.os') @mock.patch('mymodule.os.path') def test_something(self, mock_os_path, mock_os, mock_sys): pass ``` -注意到我们的参数和装饰器的顺序是反向匹配了吗?这多多少少是由 [Python 的工作方式][4] 导致的。这里是使用多个装饰器的情况下它们的执行顺序的伪代码: +注意到我们的参数和装饰器的顺序是反向匹配了吗?这多多少少是由 [Python 的工作方式][4] 导致的。这里是使用多个装饰器的情况下它们执行顺序的伪代码: ``` patch_sys(patch_os(patch_os_path(test_something))) ``` -因为 sys 补丁位于最外层,所以它会最晚执行,使得它成为实际测试方法参数的最后一个参数。请特别注意这一点,并且在运行你的测试用例时,使用调试器来保证正确的参数以正确的顺序注入。 +因为 sys 补丁位于最外层,所以它最晚执行,使得它成为实际测试方法参数的最后一个参数。请特别注意这一点,并且在运行你的测试用例时,使用调试器来保证正确的参数以正确的顺序注入。 ### 方法 2:创建 Mock 实例 -我们可以使用构造函数为 UploadService 提供一个 Mock 实例,而不是模拟特定的实例方法。我推荐上面的方法 1,因为它更加精确,但在多数情况,方法 2 或许更加有效和必要。让我们再次重构测试用例: -Instead of mocking the specific instance method, we could instead just supply a mocked instance to UploadService with its constructor. I prefer option 1 above, as it’s a lot more precise, but there are many cases where option 2 might be efficient or necessary. Let’s refactor our test again: +我们可以使用构造函数为 UploadService 提供一个 Mock 实例,而不是模拟特定的实例方法。我更推荐方法 1,因为它更加精确,但在多数情况,方法 2 或许更加有效和必要。让我们再次重构测试用例: ``` #!/usr/bin/env python @@ -389,14 +385,13 @@ class UploadServiceTestCase(unittest.TestCase): mock_removal_service.rm.assert_called_with("my uploaded file") ``` -在这个例子中,我们甚至不需要补充任何功能,只需创建一个 -In this example, we haven’t even had to patch any functionality, we simply create an auto-spec for the RemovalService class, and then inject this instance into our UploadService to validate the functionality. +在这个例子中,我们甚至不需要补充任何功能,只需为 `RemovalService` 类创建一个 auto-spec,然后将实例注入到我们的 `UploadService` 以验证功能。 -The [mock.create_autospec][5] method creates a functionally equivalent instance to the provided class. What this means, practically speaking, is that when the returned instance is interacted with, it will raise exceptions if used in illegal ways. More specifically, if a method is called with the wrong number of arguments, an exception will be raised. This is extremely important as refactors happen. As a library changes, tests break and that is expected. Without using an auto-spec, our tests will still pass even though the underlying implementation is broken. +`mock.create_autospec` 方法为类提供了一个同等功能实例。实际上来说,这意味着在使用返回的实例进行交互的时候,如果使用了非法的方式将会引发异常。更具体地说,如果一个方法被调用时的参数数目不正确,将引发一个异常。这对于重构来说是非常重要。当一个库发生变化的时候,中断测试正是所期望的。如果不使用 auto-spec,尽管底层的实现已经被破坏,我们的测试仍然会通过。 -### Pitfall: The mock.Mock and mock.MagicMock Classes +### 陷阱:mock.Mock 和 mock.MagicMock 类 -The mock library also includes two important classes upon which most of the internal functionality is built upon: [mock.Mock][6] and mock.MagicMock. When given a choice to use a mock.Mock instance, a mock.MagicMock instance, or an auto-spec, always favor using an auto-spec, as it helps keep your tests sane for future changes. This is because mock.Mock and mock.MagicMock accept all method calls and property assignments regardless of the underlying API. Consider the following use case: +`mock` 库包含了两个重要的类 [mock.Mock](http://www.voidspace.org.uk/python/mock/mock.html) 和 [mock.MagicMock](http://www.voidspace.org.uk/python/mock/magicmock.html#magic-mock),大多数内部函数都是建立在这两个类之上的。当在选择使用 `mock.Mock` 实例,`mock.MagicMock` 实例或 auto-spec 的时候,通常倾向于选择使用 auto-spec,因为对于未来的变化,它更能保持测试的健全。这是因为 `mock.Mock` 和 `mock.MagicMock` 会无视底层的 API,接受所有的方法调用和属性赋值。比如下面这个用例: ``` class Target(object): @@ -407,7 +402,7 @@ def method(target, value): return target.apply(value) ``` -We can test this with a mock.Mock instance like this: +我们可以像下面这样使用 mock.Mock 实例进行测试: ``` class MethodTestCase(unittest.TestCase): @@ -420,7 +415,7 @@ class MethodTestCase(unittest.TestCase): target.apply.assert_called_with("value") ``` -This logic seems sane, but let’s modify the Target.apply method to take more parameters: +这个逻辑看似合理,但如果我们修改 `Target.apply` 方法接受更多参数: ``` class Target(object): @@ -431,11 +426,11 @@ class Target(object): return None ``` -Re-run your test, and you’ll find that it still passes. That’s because it isn’t built against your actual API. This is why you should always use the create_autospec method and the autospec parameter with the @patch and @patch.object decorators. +重新运行你的测试,你会发现它仍能通过。这是因为它不是针对你的 API 创建的。这就是为什么你总是应该使用 `create_autospec` 方法,并且在使用 `@patch`和 `@patch.object` 装饰方法时使用 `autospec` 参数。 -### Real-World Example: Mocking a Facebook API Call +### 现实例子:模拟 Facebook API 调用 -To finish up, let’s write a more applicable real-world example, one which we mentioned in the introduction: posting a message to Facebook. We’ll write a nice wrapper class and a corresponding test case. +为了完成,我们写一个更加适用的现实例子,一个在介绍中提及的功能:发布消息到 Facebook。我将写一个不错的包装类及其对应的测试用例。 ``` import facebook @@ -450,7 +445,7 @@ class SimpleFacebook(object): self.graph.put_object("me", "feed", message=message) ``` -Here’s our test case, which checks that we post the message without actually posting the message: +这是我们的测试用例,它可以检查我们发布的消息,而不是真正地发布消息: ``` import facebook @@ -469,18 +464,18 @@ class SimpleFacebookTestCase(unittest.TestCase): mock_put_object.assert_called_with(message="Hello World!") ``` -As we’ve seen so far, it’s really simple to start writing smarter tests with mock in Python. +正如我们所看到的,在 Python 中,通过 mock,我们可以非常容易地动手写一个更加智能的测试用例。 -### Mocking in python Conclusion +### Python Mock 总结 -Python’s mock library, if a little confusing to work with, is a game-changer for [unit-testing][7]. We’ve demonstrated common use-cases for getting started using mock in unit-testing, and hopefully this article will help [Python developers][8] overcome the initial hurdles and write excellent, tested code. +对 [单元测试][7] 来说,Python 的 `mock` 库可以说是一个游戏变革者,即使对于它的使用还有点困惑。我们已经演示了单元测试中常见的用例以开始使用 `mock`,并希望这篇文章能够帮助 [Python 开发者][8] 克服初期的障碍,写出优秀、经受过考验的代码。 -------------------------------------------------------------------------------- via: http://slviki.com/index.php/2016/06/18/introduction-to-mocking-in-python/ 作者:[Dasun Sucharith][a] -译者:[译者ID](https://github.com/译者ID) +译者:[cposture](https://github.com/cposture) 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 From 239166046b8c918e284ad446f9d8b22d1412ac23 Mon Sep 17 00:00:00 2001 From: cposture Date: Sat, 6 Aug 2016 14:45:13 +0800 Subject: [PATCH 0009/2437] Translated by cposture --- ...ce portfolio - Machine learning project.md | 130 +++++++++--------- 1 file changed, 62 insertions(+), 68 deletions(-) diff --git a/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md b/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md index 3bec1d0a98..71e7cc0dbe 100644 --- a/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md +++ b/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md @@ -1,17 +1,15 @@ -Translating by cposture 2016-08-02 -### Making predictions +### 做出预测 -Now that we have the preliminaries out of the way, we’re ready to make predictions. We’ll create a new file called predict.py that will use the train.csv file we created in the last step. The below code will: +既然完成了前期准备,我们可以开始准备做出预测了。我将创建一个名为 predict.py 的新文件,它会使用我们在最后一步创建的 train.csv 文件。下面的代码: -- Import needed libraries. -- Create a function called cross_validate that: - - Creates a logistic regression classifier with the right keyword arguments. - - Creates a list of columns that we want to use to train the model, removing id and foreclosure_status. - - Run cross validation across the train DataFrame. - - Return the predictions. +- 导入所需的库 +- 创建一个名为 `cross_validate` 的函数: + - 使用正确的关键词参数创建逻辑回归分类器(logistic regression classifier) + - 创建移除了 `id` 和 `foreclosure_status` 属性的用于训练模型的列 + - 跨 `train` 数据帧使用交叉验证 + - 返回预测结果 - -``` +```python import os import settings import pandas as pd @@ -29,22 +27,19 @@ def cross_validate(train): return predictions ``` -### Predicting error +### 预测误差 -Now, we just need to write a few functions to compute error. The below code will: +现在,我们仅仅需要写一些函数来计算误差。下面的代码: -- Create a function called compute_error that: - - Uses scikit-learn to compute a simple accuracy score (the percentage of predictions that matched the actual foreclosure_status values). -- Create a function called compute_false_negatives that: - - Combines the target and the predictions into a DataFrame for convenience. - - Finds the false negative rate. -- Create a function called compute_false_positives that: - - Combines the target and the predictions into a DataFrame for convenience. - - Finds the false positive rate. - - Finds the number of loans that weren’t foreclosed on that the model predicted would be foreclosed on. - - Divide by the total number of loans that weren’t foreclosed on. +- 创建函数 `compute_error`: + - 使用 scikit-learn 计算一个简单的精确分数(与实际 `foreclosure_status` 值匹配的预测百分比) +- 创建函数 `compute_false_negatives`: + - 为了方便,将目标和预测结果合并到一个数据帧 + - 查找漏报率 + - 找到原本应被预测模型取消但没有取消的贷款数目 + - 除以没被取消的贷款总数目 -``` +```python def compute_error(target, predictions): return metrics.accuracy_score(target, predictions) @@ -57,21 +52,20 @@ def compute_false_positives(target, predictions): return df[(df["target"] == 0) & (df["predictions"] == 1)].shape[0] / (df[(df["target"] == 0)].shape[0] + 1) ``` +### 聚合到一起 -### Putting it all together +现在,我们可以把函数都放在 `predict.py`。下面的代码: -Now, we just have to put the functions together in predict.py. The below code will: +- 读取数据集 +- 计算交叉验证预测 +- 计算上面的 3 个误差 +- 打印误差 -- Read in the dataset. -- Compute cross validated predictions. -- Compute the 3 error metrics above. -- Print the error metrics. - -``` +```python def read(): train = pd.read_csv(os.path.join(settings.PROCESSED_DIR, "train.csv")) return train - + if __name__ == "__main__": train = read() predictions = cross_validate(train) @@ -83,11 +77,11 @@ if __name__ == "__main__": print("False Positives: {}".format(fp)) ``` -Once you’ve added the code, you can run python predict.py to generate predictions. Running everything shows that our false negative rate is .26, which means that of the foreclosed loans, we missed predicting 26% of them. This is a good start, but can use a lot of improvement! +一旦你添加完代码,你可以运行 `python predict.py` 来产生预测结果。运行结果向我们展示漏报率为 `.26`,这意味着我们没能预测 `26%` 的取消贷款。这是一个好的开始,但仍有很多改善的地方! -You can find the complete predict.py file [here][41]. +你可以在[这里][41]找到完整的 `predict.py` 文件 -Your file tree should now look like this: +你的文件树现在看起来像下面这样: ``` loan-prediction @@ -110,47 +104,47 @@ loan-prediction ├── settings.py ``` -### Writing up a README +### 撰写 README -Now that we’ve finished our end to end project, we just have to write up a README.md file so that other people know what we did, and how to replicate it. A typical README.md for a project should include these sections: +既然我们完成了端到端的项目,那么我们可以撰写 README.md 文件了,这样其他人便可以知道我们做的事,以及如何复制它。一个项目典型的 README.md 应该包括这些部分: -- A high level overview of the project, and what the goals are. -- Where to download any needed data or materials. -- Installation instructions. - - How to install the requirements. -- Usage instructions. - - How to run the project. - - What you should see after each step. -- How to contribute to the project. - - Good next steps for extending the project. +- 一个高水准的项目概览,并介绍项目目的 +- 任何必需的数据和材料的下载地址 +- 安装命令 + - 如何安装要求依赖 +- 使用命令 + - 如何运行项目 + - 每一步之后会看到的结果 +- 如何为这个项目作贡献 + - 扩展项目的下一步计划 -[Here’s][42] a sample README.md for this project. +[这里][42] 是这个项目的一个 README.md 样例。 -### Next steps +### 下一步 -Congratulations, you’re done making an end to end machine learning project! You can find a complete example project [here][43]. It’s a good idea to upload your project to [Github][44] once you’ve finished it, so others can see it as part of your portfolio. +恭喜你完成了端到端的机器学习项目!你可以在[这里][43]找到一个完整的示例项目。一旦你完成了项目,把它上传到 [Github][44] 是一个不错的主意,这样其他人也可以看到你的文件夹的部分项目。 -There are still quite a few angles left to explore with this data. Broadly, we can split them up into 3 categories – extending this project and making it more accurate, finding other columns to predict, and exploring the data. Here are some ideas: +这里仍有一些留待探索数据的角度。总的来说,我们可以把它们分割为 3 类 - 扩展这个项目并使它更加精确,发现预测其他列并探索数据。这是其中一些想法: -- Generate more features in annotate.py. -- Switch algorithms in predict.py. -- Try using more data from Fannie Mae than we used in this post. -- Add in a way to make predictions on future data. The code we wrote will still work if we add more data, so we can add more past or future data. -- Try seeing if you can predict if a bank should have issued the loan originally (vs if Fannie Mae should have acquired the loan). - - Remove any columns from train that the bank wouldn’t have known at the time of issuing the loan. - - Some columns are known when Fannie Mae bought the loan, but not before. - - Make predictions. -- Explore seeing if you can predict columns other than foreclosure_status. - - Can you predict how much the property will be worth at sale time? -- Explore the nuances between performance updates. - - Can you predict how many times the borrower will be late on payments? - - Can you map out the typical loan lifecycle? -- Map out data on a state by state or zip code by zip code level. - - Do you see any interesting patterns? +- 在 `annotate.py` 中生成更多的特性 +- 切换 `predict.py` 中的算法 +- 尝试使用比我们发表在这里的更多的来自 `Fannie Mae` 的数据 +- 添加对未来数据进行预测的方法。如果我们添加更多数据,我们所写的代码仍然可以起作用,这样我们可以添加更多过去和未来的数据。 +- 尝试看看是否你能预测一个银行是否应该发放贷款(相对地,`Fannie Mae` 是否应该获得贷款) + - 移除 train 中银行不知道发放贷款的时间的任何列 + - 当 Fannie Mae 购买贷款时,一些列是已知的,但不是之前 + - 做出预测 +- 探索是否你可以预测除了 foreclosure_status 的其他列 + - 你可以预测在销售时财产值多少? +- 探索探索性能更新之间的细微差别 + - 你能否预测借款人会逾期还款多少次? + - 你能否标出的典型贷款周期? +- 标出一个州到州或邮政编码到邮政级水平的数据 + - 你看到一些有趣的模式了吗? -If you build anything interesting, please let us know in the comments! +如果你建立了任何有趣的东西,请在评论中让我们知道! -If you liked this, you might like to read the other posts in our ‘Build a Data Science Porfolio’ series: +如果你喜欢这个,你可能会喜欢阅读 ‘Build a Data Science Porfolio’ 系列其他文章: - [Storytelling with data][45]. - [How to setup up a data science blog][46]. From fb71a5ca0d1df18fa0b8b6b2c4a29e50b7ff9ec3 Mon Sep 17 00:00:00 2001 From: cposture Date: Sat, 6 Aug 2016 14:54:18 +0800 Subject: [PATCH 0010/2437] Translated by cposture --- ...lding a data science portfolio - Machine learning project.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md b/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md index 71e7cc0dbe..2ea994c119 100644 --- a/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md +++ b/sources/team_test/part 6 - Building a data science portfolio - Machine learning project.md @@ -144,7 +144,7 @@ loan-prediction 如果你建立了任何有趣的东西,请在评论中让我们知道! -如果你喜欢这个,你可能会喜欢阅读 ‘Build a Data Science Porfolio’ 系列其他文章: +如果你喜欢这个,你可能会喜欢阅读 ‘Build a Data Science Porfolio’ 系列的其他文章: - [Storytelling with data][45]. - [How to setup up a data science blog][46]. From 4126a52d34d48317c51ce4a01fe4742377346f39 Mon Sep 17 00:00:00 2001 From: cposture Date: Sun, 7 Aug 2016 13:02:32 +0800 Subject: [PATCH 0011/2437] =?UTF-8?q?=E5=BA=94=20erlinux=20=E7=9A=84?= =?UTF-8?q?=E8=A6=81=E6=B1=82=EF=BC=8C=E6=8A=8A=E6=9C=AA=E7=BF=BB=E8=AF=91?= =?UTF-8?q?=E5=AE=8C=E7=9A=84=E6=96=87=E7=AB=A0=E7=A7=BB=E5=88=B0=20source?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Rapid prototyping with docker-compose.md | 142 ++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 sources/tech/20160512 Rapid prototyping with docker-compose.md diff --git a/sources/tech/20160512 Rapid prototyping with docker-compose.md b/sources/tech/20160512 Rapid prototyping with docker-compose.md new file mode 100644 index 0000000000..0c67223697 --- /dev/null +++ b/sources/tech/20160512 Rapid prototyping with docker-compose.md @@ -0,0 +1,142 @@ + +Rapid prototyping with docker-compose +======================================== + +In this write-up we'll look at a Node.js prototype for **finding stock of the Raspberry PI Zero** from three major outlets in the UK. + +I wrote the code and deployed it to an Ubuntu VM in Azure within a single evening of hacking. Docker and the docker-compose tool made the deployment and update process extremely quick. + +### Remember linking? + +If you've already been through the [Hands-On Docker tutorial][1] then you will have experience linking Docker containers on the command line. Linking a Node hit counter to a Redis server on the command line may look like this: + +``` +$ docker run -d -P --name redis1 +$ docker run -d hit_counter -p 3000:3000 --link redis1:redis +``` + +Now imagine your application has three tiers + +- Web front-end +- Batch tier for processing long running tasks +- Redis or mongo database + +Explicit linking through `--link` is just about manageable with a couple of containers, but can get out of hand as we add more tiers or containers to the application. + +### Enter docker-compose + +![](http://blog.alexellis.io/content/images/2016/05/docker-compose-logo-01.png) +>Docker Compose logo + +The docker-compose tool is part of the standard Docker Toolbox and can also be downloaded separately. It provides a rich set of features to configure all of an application's parts through a plain-text YAML file. + +The above example would look like this: + +``` +version: "2.0" +services: + redis1: + image: redis + hit_counter: + build: ./hit_counter + ports: + - 3000:3000 +``` + +From Docker 1.10 onwards we can take advantage of network overlays to help us scale out across multiple hosts. Prior to this linking only worked across a single host. The `docker-compose scale` command can be used to bring on more computing power as the need arises. + +>View the [docker-compose][2] reference on docker.com + +### Real-world example: Raspberry PI Stock Alert + +![](http://blog.alexellis.io/content/images/2016/05/Raspberry_Pi_Zero_ver_1-3_1_of_3_large.JPG) +>The new Raspberry PI Zero v1.3 image courtesy of Pimoroni + +There is a huge buzz around the Raspberry PI Zero - a tiny microcomputer with a 1GHz CPU and 512MB RAM capable of running full Linux, Docker, Node.js, Ruby and many other popular open-source tools. One of the best things about the PI Zero is that costs only 5 USD. That also means that stock gets snapped up really quickly. + +*If you want to try Docker or Swarm on the PI check out the tutorial below.* + +>[Docker Swarm on the PI Zero][3] + +### Original site: whereismypizero.com + +I found a webpage which used screen scraping to find whether 4-5 of the most popular outlets had stock. + +- The site contained a static HTML page +- Issued one XMLHttpRequest per outlet accessing /public/api/ +- The server issued the HTTP request to each shop and performed the scraping + +Every call to /public/api/ took 3 seconds to execute and using Apache Bench (ab) I was only able to get through 0.25 requests per second. + +### Reinventing the wheel + +The retailers didn't seem to mind whereismypizero.com scraping their sites for stock, so I set about writing a similar tool from the ground up. I had the intention of handing a much higher amount of requests per second through caching and de-coupling the scrape from the web tier. Redis was the perfect tool for the job. It allowed me to set an automatically expiring key/value pair (i.e. a simple cache) and also to transmit messages between Node processes through pub/sub. + +>Fork or star the code on Github: [alexellis/pi_zero_stock][4] + +If you've worked with Node.js before then you will know it is single-threaded and that any CPU intensive tasks such as parsing HTML or JSON could lead to a slow-down. One way to mitigate that is to use a second worker process and a Redis messaging channel as connective tissue between this and the web tier. + +- Web tier + -Gives 200 for cache hit (Redis key exists for store) + -Gives 202 for cache miss (Redis key doesn't exist, so issues message) + -Since we are only ever reading a Redis key the response time is very quick. +- Stock Fetcher + -Performs HTTP request + -Scrapes for different types of web stores + -Updates a Redis key with a cache expire of 60 seconds + -Also locks a Redis key to prevent too many in-flight HTTP requests to the web stores. +``` +version: "2.0" +services: + web: + build: ./web/ + ports: + - "3000:3000" + stock_fetch: + build: ./stock_fetch/ + redis: + image: redis +``` + +*The docker-compose.yml file from the example.* + +Once I had this working locally deploying to an Ubuntu 16.04 image in the cloud (Azure) took less than 5 minutes. I logged in, cloned the repository and typed in `docker compose up -d`. That was all it took - rapid prototyping a whole system doesn't get much better. Anyone (including the owner of whereismypizero.com) can deploy the new solution with just two lines: + +``` +$ git clone https://github.com/alexellis/pi_zero_stock +$ docker-compose up -d +``` + +Updating the site is easy and just involves a `git pull` followed by a `docker-compose up -d` with the `--build` argument passed along. + +If you are still linking your Docker containers manually, try Docker Compose for yourself or my code below: + +>Fork or star the code on Github: [alexellis/pi_zero_stock][5] + +### Check out the test site + +The test site is currently deployed now using docker-compose. + +>[stockalert.alexellis.io][6] + +![](http://blog.alexellis.io/content/images/2016/05/Screen-Shot-2016-05-16-at-22-34-26-1.png) + +Preview as of 16th of May 2016 + +---------- +via: http://blog.alexellis.io/rapid-prototype-docker-compose/ + +作者:[Alex Ellis][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://blog.alexellis.io/author/alex/ +[1]: http://blog.alexellis.io/handsondocker +[2]: https://docs.docker.com/compose/compose-file/ +[3]: http://blog.alexellis.io/dockerswarm-pizero/ +[4]: https://github.com/alexellis/pi_zero_stock +[5]: https://github.com/alexellis/pi_zero_stock +[6]: http://stockalert.alexellis.io/ + From 182f9f1a244b8d0fb0027d8171378c789fd2deda Mon Sep 17 00:00:00 2001 From: cposture Date: Sun, 7 Aug 2016 13:04:09 +0800 Subject: [PATCH 0012/2437] =?UTF-8?q?=E5=BA=94=20erlinux=20=E7=9A=84?= =?UTF-8?q?=E8=A6=81=E6=B1=82=EF=BC=8C=E6=8A=8A=E6=9C=AA=E7=BF=BB=E8=AF=91?= =?UTF-8?q?=E5=AE=8C=E7=9A=84=E6=96=87=E7=AB=A0=E7=A7=BB=E5=88=B0=20source?= =?UTF-8?q?,=E5=90=8C=E6=97=B6=E5=88=A0=E9=99=A4=20translated=E6=96=87?= =?UTF-8?q?=E7=AB=A0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Rapid prototyping with docker-compose.md | 141 ------------------ 1 file changed, 141 deletions(-) delete mode 100644 translated/tech/20160512 Rapid prototyping with docker-compose.md diff --git a/translated/tech/20160512 Rapid prototyping with docker-compose.md b/translated/tech/20160512 Rapid prototyping with docker-compose.md deleted file mode 100644 index a32f894d4b..0000000000 --- a/translated/tech/20160512 Rapid prototyping with docker-compose.md +++ /dev/null @@ -1,141 +0,0 @@ -使用docker快速组成样品机 -======================================== - -在写前,我们将看看 Node.js 样机 ** 找寻树莓派 PI Zero ** 的供应在英国三个主要销售. - -我写的代码,黑客部署到 Azure Ubuntu 虚拟机一个晚上就可以到位。Docker 和 docker-compose 工具做出调配和更新过程非常快。 - -### 建立链接? - - -如果您已经通过 [动手 Docker 教程指南] [1] 那么你已有在命令行建立 Docker 容器的经验。链接一个Redis 服务器计数器节点在命令行上可能是这样: - -``` -$ docker run -d -P --name redis1 -$ docker run -d hit_counter -p 3000:3000 --link redis1:redis -``` - -现在,假设应用程序中有三个等级 - -- Web 前端 -- 批次层处理长时间运行的任务 -- Redis 或 MongoDB 数据库 - -通过 `--link` 管理几个容器,但可能失效,可以添加多层级或容器到应用程序。 - -### 键入 docker 撰写 - -![](http://blog.alexellis.io/content/images/2016/05/docker-compose-logo-01.png) ->Docker 撰写图标 - -docker-compose 工具是标准的 docker工具箱的一部分,也可以单独下载。它提供了丰富功能,通过一个纯文本YAML文件配置所有应用程序组件。 - -上述提供了一个例子: - -``` -version: "2.0" -services: - redis1: - image: redis - hit_counter: - build: ./hit_counter - ports: - - 3000:3000 -``` - -从Docker 1.10起,我们可以充分利用网络来帮助我们在多个主机进行扩展覆盖。在此之前,仅通过单个主机工作。“docker-compose scale” 命令可用于更多计算能力有需要时。 - ->参考docker.com上关于"docker-compose" - -### 真实例子:树莓派 PI 到货通知 - -![](http://blog.alexellis.io/content/images/2016/05/Raspberry_Pi_Zero_ver_1-3_1_of_3_large.JPG) ->新版树莓派 PI Zero V1.3 图片提供来自Pimoroni - -树莓派 PI Zero - 巨大的轰动一个微型计算机具有一个1GHz 处理器 和 512MB 内存能够运行完整 Linux,Docker,Node.js,Ruby 和许多流行的开源工具。一个关于 PI Zero 的好消息是,成本只有5美元。这也意味着,存量迅速抢购一空。 - -*如果您想尝试Docker 或集群在PI看看下面的教程。* - ->[Docker Swarm on the PI Zero][3] - -### Original site: whereismypizero.com - -I found a webpage which used screen scraping to find whether 4-5 of the most popular outlets had stock. - -- The site contained a static HTML page -- Issued one XMLHttpRequest per outlet accessing /public/api/ -- The server issued the HTTP request to each shop and performed the scraping - -Every call to /public/api/ took 3 seconds to execute and using Apache Bench (ab) I was only able to get through 0.25 requests per second. - -### Reinventing the wheel - -The retailers didn't seem to mind whereismypizero.com scraping their sites for stock, so I set about writing a similar tool from the ground up. I had the intention of handing a much higher amount of requests per second through caching and de-coupling the scrape from the web tier. Redis was the perfect tool for the job. It allowed me to set an automatically expiring key/value pair (i.e. a simple cache) and also to transmit messages between Node processes through pub/sub. - ->Fork or star the code on Github: [alexellis/pi_zero_stock][4] - -If you've worked with Node.js before then you will know it is single-threaded and that any CPU intensive tasks such as parsing HTML or JSON could lead to a slow-down. One way to mitigate that is to use a second worker process and a Redis messaging channel as connective tissue between this and the web tier. - -- Web tier - -Gives 200 for cache hit (Redis key exists for store) - -Gives 202 for cache miss (Redis key doesn't exist, so issues message) - -Since we are only ever reading a Redis key the response time is very quick. -- Stock Fetcher - -Performs HTTP request - -Scrapes for different types of web stores - -Updates a Redis key with a cache expire of 60 seconds - -Also locks a Redis key to prevent too many in-flight HTTP requests to the web stores. -``` -version: "2.0" -services: - web: - build: ./web/ - ports: - - "3000:3000" - stock_fetch: - build: ./stock_fetch/ - redis: - image: redis -``` - -*The docker-compose.yml file from the example.* - -Once I had this working locally deploying to an Ubuntu 16.04 image in the cloud (Azure) took less than 5 minutes. I logged in, cloned the repository and typed in `docker compose up -d`. That was all it took - rapid prototyping a whole system doesn't get much better. Anyone (including the owner of whereismypizero.com) can deploy the new solution with just two lines: - -``` -$ git clone https://github.com/alexellis/pi_zero_stock -$ docker-compose up -d -``` - -Updating the site is easy and just involves a `git pull` followed by a `docker-compose up -d` with the `--build` argument passed along. - -If you are still linking your Docker containers manually, try Docker Compose for yourself or my code below: - ->Fork or star the code on Github: [alexellis/pi_zero_stock][5] - -### Check out the test site - -The test site is currently deployed now using docker-compose. - ->[stockalert.alexellis.io][6] - -![](http://blog.alexellis.io/content/images/2016/05/Screen-Shot-2016-05-16-at-22-34-26-1.png) - -Preview as of 16th of May 2016 - ----------- -via: http://blog.alexellis.io/rapid-prototype-docker-compose/ - -作者:[Alex Ellis][a] -译者:[erlinux](https://github.com/erlinux) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://blog.alexellis.io/author/alex/ -[1]: http://blog.alexellis.io/handsondocker -[2]: https://docs.docker.com/compose/compose-file/ -[3]: http://blog.alexellis.io/dockerswarm-pizero/ -[4]: https://github.com/alexellis/pi_zero_stock -[5]: https://github.com/alexellis/pi_zero_stock -[6]: http://stockalert.alexellis.io/ From 083c512d89d1a5d746a915809a37c69c7ee959d4 Mon Sep 17 00:00:00 2001 From: cposture Date: Sat, 20 Aug 2016 17:33:20 +0800 Subject: [PATCH 0013/2437] Translating --- ...hon unittest - assertTrue is truthy - assertFalse is falsy.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160512 Python unittest - assertTrue is truthy - assertFalse is falsy.md b/sources/tech/20160512 Python unittest - assertTrue is truthy - assertFalse is falsy.md index bfed46fb0e..30589a9d63 100644 --- a/sources/tech/20160512 Python unittest - assertTrue is truthy - assertFalse is falsy.md +++ b/sources/tech/20160512 Python unittest - assertTrue is truthy - assertFalse is falsy.md @@ -1,3 +1,4 @@ +Translating by cposture Python unittest: assertTrue is truthy, assertFalse is falsy =========================== From 6701bfaa97d2b53faca4888326bf673a9ecd769f Mon Sep 17 00:00:00 2001 From: cposture Date: Sat, 20 Aug 2016 23:07:20 +0800 Subject: [PATCH 0014/2437] Translated by cposture --- .../20160820 Protocol Buffer Basics C++.md | 396 ++++++++++++++++++ 1 file changed, 396 insertions(+) create mode 100644 translated/tech/20160820 Protocol Buffer Basics C++.md diff --git a/translated/tech/20160820 Protocol Buffer Basics C++.md b/translated/tech/20160820 Protocol Buffer Basics C++.md new file mode 100644 index 0000000000..11bda510d4 --- /dev/null +++ b/translated/tech/20160820 Protocol Buffer Basics C++.md @@ -0,0 +1,396 @@ +这篇教程提供了一个面向 C++ 程序员、关于 `protocol buffers` 的基础介绍。通过创建一个简单的示例应用程序,它将向我们展示: + +* 在 `.proto` 文件中定义消息格式 +* 使用 `protocol buffer` 编译器 +* 使用 `C++ protocol buffer API` 读写消息 + +这不是一个关于使用 C++ protocol buffers 的全面指南。要获取更详细的信息,请参考 [Protocol Buffer Language Guide](https://developers.google.com/protocol-buffers/docs/proto) 和 [Encoding Reference](https://developers.google.com/protocol-buffers/docs/encoding)。 + +# 为什么使用 Protocol Buffers + +我们接下来要使用的例子是一个非常简单的"地址簿"应用程序,它能从文件中读取联系人详细信息。地址簿中的每一个人都有一个名字,ID,邮件地址和联系电话。 + +如何序列化和获取结构化的数据?这里有几种解决方案: + +* 以二进制形式发送/接收原生的内存数据结构。通常,这是一种脆弱的方法,因为接收/读取代码的编译必须基于完全相同的内存布局、大小端等等。同时,当文件增加时,原始格式数据会随着与该格式相连的软件拷贝而迅速扩散,这将很难扩展文件格式。 + +* 你可以创造一种 `ad-hoc` 方法,将数据项编码为一个字符串——比如将 4 个整数编码为 "12:3:-23:67"。虽然它需要编写一次性的编码和解码代码且解码需要耗费小的运行时成本,但这是一种简单灵活的方法。这最适合编码非常简单的数据。 + +* 序列化数据为 `XML`。这种方法是非常吸引人的,因为 `XML` 是一种适合人阅读的格式,并且有为许多语言开发的库。如果你想与其他程序和项目共享数据,这可能是一种不错的选择。然而,众所周知,`XML` 是空间密集型的,且在编码和解码时,它对程序会造成巨大的性能损失。同时,使用 XML DOM 树被认为比操作一个类的简单字段更加复杂。 + +`Protocol buffers` 是针对这个问题的一种灵活、高效、自动化的解决方案。使用 `Protocol buffers`,你需要写一个 `.proto` 说明,用于描述你所希望存储的数据结构。利用 `.proto` 文件,protocol buffer 编译器可以创建一个类,用于实现自动化编码和解码高效的二进制格式的 protocol buffer 数据。产生的类提供了构造 `protocol buffer` 的字段的 getters 和 setters,并且作为一个单元,关注读写 `protocol buffer` 的细节。重要的是,`protocol buffer` 格式支持扩展格式,代码仍然可以读取以旧格式编码的数据。 + +# 在哪可以找到示例代码 + +示例代码被包含于源代码包,位于 "examples" 文件夹。在[这](https://developers.google.com/protocol-buffers/docs/downloads)下载代码。 + +# 定义你的协议格式 + +为了创建自己的地址簿应用程序,你需要从 `.proto` 开始。`.proto` 文件中的定义很简单:为你所需要序列化的数据结构添加一个消息(message),然后为消息中的每一个字段指定一个名字和类型。这里是定义你消息的 `.proto` 文件,`addressbook.proto`。 + +``` +package tutorial; + +message Person { + required string name = 1; + required int32 id = 2; + optional string email = 3; + + enum PhoneType { + MOBILE = 0; + HOME = 1; + WORK = 2; + } + + message PhoneNumber { + required string number = 1; + optional PhoneType type = 2 [default = HOME]; + } + + repeated PhoneNumber phone = 4; +} + +message AddressBook { + repeated Person person = 1; +} +``` + +如你所见,其语法类似于 C++ 或 Java。我们开始看看文件的每一部分内容做了什么。 + +`.proto` 文件以一个 package 声明开始,这可以避免不同项目的命名冲突。在 C++,你生成的类会被置于与 package 名字一样的命名空间。 + +下一步,你需要定义消息(message)。消息只是一个包含一系列类型字段的集合。大多标准简单数据类型是可以作为字段类型的,包括 `bool`、`int32`、`float`、`double` 和 `string`。你也可以通过使用其他消息类型作为字段类型,将更多的数据结构添加到你的消息中——在以上的示例,`Person` 消息包含了 `PhoneNumber` 消息,同时 `AddressBook` 消息包含 `Person` 消息。你甚至可以定义嵌套在其他消息内的消息类型——如你所见,`PhoneNumber` 类型定义于 `Person` 内部。如果你想要其中某一个字段拥有预定义值列表中的某个值,你也可以定义 `enum` 类型——这儿你想指定一个电话号码可以是 `MOBILE`、`HOME` 或 `WORK` 中的某一个。 + +每一个元素上的 “=1”、"=2" 标记确定了用于二进制编码的唯一"标签"(tag)。标签数字 1-15 的编码比更大的数字少需要一个字节,因此作为一种优化,你可以将这些标签用于经常使用或 repeated 元素,剩下 16 以及更高的标签用于非经常使用或 optional 元素。每一个 repeated 字段的元素需要重新编码标签数字,因此 repeated 字段对于这优化是一个特别好的候选者。 + +每一个字段必须使用下面的修饰符加以标注: + +* required:必须提供字段的值,否则消息会被认为是 "未初始化的"(uninitialized)。如果 `libprotobuf` 以 debug 模式编译,序列化未初始化的消息将引起一个断言失败。以优化形式构建,将会跳过检查,并且无论如何都会写入消息。然而,解析未初始化的消息总是会失败(通过 parse 方法返回 `false`)。除此之外,一个 required 字段的表现与 optional 字段完全一样。 + +* optional:字段可能会被设置,也可能不会。如果一个 optional 字段没被设置,它将使用默认值。对于简单类型,你可以指定你自己的默认值,正如例子中我们对电话号码的 `type` 一样,否则使用系统默认值:数字类型为 0、字符串为空字符串、布尔值为 false。对于嵌套消息,默认值总为消息的"默认实例"或"原型",它的所有字段都没被设置。调用 accessor 来获取一个没有显式设置的 optional(或 required) 字段的值总是返回字段的默认值。 + +* repeated:字段可以重复任意次数(包括 0)。repeated 值的顺序会被保存于 protocol buffer。可以将 repeated 字段想象为动态大小的数组。 + +你可以查找关于编写 `.proto` 文件的完整指导——包括所有可能的字段类型——在 [Protocol Buffer Language Guide](https://developers.google.com/protocol-buffers/docs/proto)。不要在这里面查找与类继承相似的特性,因为 protocol buffers 不会做这些。 + +Required Is Forever You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field – old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some engineers at Google have come to the conclusion that using required does more harm than good; they prefer to use only optional and repeated. However, this view is not universal. + +> required 是永久性的,在把一个字段标识为 required 的时候,你应该特别小心。如果在某些情况下你不想写入或者发送一个 required 的字段,那么将该字段更改为 optional 可能会遇到问题——旧版本的读者(译者注:即读取、解析旧版本 Protocol Buffer 消息的一方)会认为不含该字段的消息是不完整的,从而有可能会拒绝解析。在这种情况下,你应该考虑编写特别针对于应用程序的、自定义的消息校验函数。Google 的一些工程师得出了一个结论:使用 required 弊多于利;他们更愿意使用 optional 和 repeated 而不是 required。当然,这个观点并不具有普遍性。 + +# 编译你的 Protocol Buffers + +既然你有了一个 `.proto`,那你需要做的下一件事就是生成一个将用于读写 `AddressBook` 消息的类(从而包括 `Person` 和 `PhoneNumber`)。为了做到这样,你需要在你的 `.proto` 上运行 protocol buffer 编译器 `protoc`: + +1. 如果你没有安装编译器,请[下载这个包](https://developers.google.com/protocol-buffers/docs/downloads.html),并按照 README 中的指令进行安装。 +2. 现在运行编译器,知道源目录(你的应用程序源代码位于哪里——如果你没有提供任何值,将使用当前目录),目标目录(你想要生成的代码放在哪里;常与 `$SRC_DIR` 相同),并且你的 `.proto` 路径。在此示例,你...: + +``` +protoc -I=$SRC_DIR --cpp_out=$DST_DIR $SRC_DIR/addressbook.proto +``` + +因为你想要 C++ 的类,所以你使用了 `--cpp_out` 选项——也为其他支持的语言提供了类似选项。 + +在你指定的目标文件夹,将生成以下的文件: + +* `addressbook.pb.h`,声明你生成类的头文件。 +* `addressbook.pb.cc`,包含你的类的实现。 + +# Protocol Buffer API + +让我们看看生成的一些代码,了解一下编译器为你创建了什么类和函数。如果你查看 `tutorial.pb.h`,你可以看到有一个在 `tutorial.proto` 中指定所有消息的类。关注 `Person` 类,可以看到编译器为每个字段生成了读写函数(accessors)。例如,对于 `name`、`id`、`email` 和 `phone` 字段,有下面这些方法: + +```c++ +// name +inline bool has_name() const; +inline void clear_name(); +inline const ::std::string& name() const; +inline void set_name(const ::std::string& value); +inline void set_name(const char* value); +inline ::std::string* mutable_name(); + +// id +inline bool has_id() const; +inline void clear_id(); +inline int32_t id() const; +inline void set_id(int32_t value); + +// email +inline bool has_email() const; +inline void clear_email(); +inline const ::std::string& email() const; +inline void set_email(const ::std::string& value); +inline void set_email(const char* value); +inline ::std::string* mutable_email(); + +// phone +inline int phone_size() const; +inline void clear_phone(); +inline const ::google::protobuf::RepeatedPtrField< ::tutorial::Person_PhoneNumber >& phone() const; +inline ::google::protobuf::RepeatedPtrField< ::tutorial::Person_PhoneNumber >* mutable_phone(); +inline const ::tutorial::Person_PhoneNumber& phone(int index) const; +inline ::tutorial::Person_PhoneNumber* mutable_phone(int index); +inline ::tutorial::Person_PhoneNumber* add_phone(); +``` + +正如你所见到,getters 的名字与字段的小写名字完全一样,并且 setter 方法以 set_ 开头。同时每个单一(singular)(required 或 optional)字段都有 `has_` 方法,该方法在字段被设置了值的情况下返回 true。最后,所有字段都有一个 `clear_` 方法,用以清除字段到空(empty)状态。 + +数字 `id` 字段仅有上述的基本读写函数集合(accessors),而 `name` 和 `email` 字段有两个额外的方法,因为它们是字符串——一个是可以获得字符串直接指针的`mutable_` getter ,另一个为额外的 setter。注意,尽管 `email` 还没被设置(set),你也可以调用 `mutable_email`;因为 `email` 会被自动地初始化为空字符串。在本例中,如果你有一个单一的(required 或 optional)消息字段,它会有一个 `mutable_` 方法,而没有 `set_` 方法。 + +repeated 字段也有一些特殊的方法——如果你看看 repeated `phone` 字段的方法,你可以看到: + +* 检查 repeated 字段的 `_size`(也就是说,与 `Person` 相关的电话号码的个数) +* 使用下标取得特定的电话号码 +* 更新特定下标的电话号码 +* 添加新的电话号码到消息中,之后你便可以编辑。(repeated 标量类型有一个 `add_` 方法,用于传入新的值) + +为了获取 protocol 编译器为所有字段定义生成的方法的信息,可以查看 [C++ generated code reference](https://developers.google.com/protocol-buffers/docs/reference/cpp-generated)。 + +### 枚举和嵌套类(Enums and Nested Classes) + +与 `.proto` 的枚举相对应,生成的代码包含了一个 `PhoneType` 枚举。你可以通过 `Person::PhoneType` 引用这个类型,通过 `Person::MOBILE`、`Person::HOME` 和 `Person::WORK` 引用它的值。(实现细节有点复杂,但是你无须了解它们而可以直接使用) + +编译器也生成了一个 `Person::PhoneNumber` 的嵌套类。如果你查看代码,你可以发现真正的类型为 `Person_PhoneNumber`,但它通过在 `Person` 内部使用 typedef 定义,使你可以把 `Person_PhoneNumber` 当成嵌套类。唯一产生影响的一个例子是,如果你想要在其他文件前置声明该类——在 C++ 中你不能前置声明嵌套类,但是你可以前置声明 `Person_PhoneNumber`。 + +### 标准的消息方法 + +所有的消息方法都包含了许多别的方法,用于检查和操作整个消息,包括: + +* `bool IsInitialized() const;` :检查是否所有 `required` 字段已经被设置。 +* `string DebugString() const;`:返回人类可读的消息表示,对 debug 特别有用。 +* `void CopyFrom(const Person& from);`:使用给定的值重写消息。 +* `void Clear();`:清除所有元素为空(empty)的状态。 + +上面这些方法以及下一节要讲的 I/O 方法实现了被所有 C++ protocol buffer 类共享的消息(Message)接口。为了获取更多信息,请查看 [complete API documentation for Message](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message)。 + +### 解析和序列化(Parsing and Serialization) + +最后,所有 protocol buffer 类都有读写你选定类型消息的方法,这些方法使用了特定的 protocol buffer [二进制格式](https://developers.google.com/protocol-buffers/docs/encoding) +。这些方法包括: + +* `bool SerializeToString(string* output) const;`:序列化消息以及将消息字节数据存储在给定的字符串。注意,字节数据是二进制格式的,而不是文本格式;我们只使用 `string` 类作为合适的容器。 +* `bool ParseFromString(const string& data);`:从给定的字符创解析消息。 +* `bool SerializeToOstream(ostream* output) const;`:将消息写到给定的 C++ `ostream`。 +* `bool ParseFromIstream(istream* input);`:从给定的 C++ `istream` 解析消息。 + +这些只是两个用于解析和序列化的选择。再次说明,可以查看 `Message API reference` 完整的列表。 + +> Protocol Buffers 和 面向对象设计的 Protocol buffer 类通常只是纯粹的数据存储器(像 C++ 中的结构体);它们在对象模型中并不是一等公民。如果你想向生成的 protocol buffer 类中添加更丰富的行为,最好的方法就是在应用程序中对它进行封装。如果你无权控制 .proto 文件的设计的话,封装 protocol buffers 也是一个好主意(例如,你从另一个项目中重用一个 .proto 文件)。在那种情况下,你可以用封装类来设计接口,以更好地适应你的应用程序的特定环境:隐藏一些数据和方法,暴露一些便于使用的函数,等等。但是你绝对不要通过继承生成的类来添加行为。这样做的话,会破坏其内部机制,并且不是一个好的面向对象的实践。 + +# 写消息(Writing A Message) + +现在我们尝试使用 protocol buffer 类。你的地址簿程序想要做的第一件事是将个人详细信息写入到地址簿文件。为了做到这一点,你需要创建、填充 protocol buffer 类实例,并且将它们写入到一个输出流(output stream)。 + +这里的程序可以从文件读取 `AddressBook`,根据用户输入,将新 `Person` 添加到 `AddressBook`,并且再次将新的 `AddressBook` 写回文件。这部分直接调用或引用 protocol buffer 类的代码会高亮显示。 + +```c++ +#include +#include +#include +#include "addressbook.pb.h" +using namespace std; + +// This function fills in a Person message based on user input. +void PromptForAddress(tutorial::Person* person) { + cout << "Enter person ID number: "; + int id; + cin >> id; + person->set_id(id); + cin.ignore(256, '\n'); + + cout << "Enter name: "; + getline(cin, *person->mutable_name()); + + cout << "Enter email address (blank for none): "; + string email; + getline(cin, email); + if (!email.empty()) { + person->set_email(email); + } + + while (true) { + cout << "Enter a phone number (or leave blank to finish): "; + string number; + getline(cin, number); + if (number.empty()) { + break; + } + + tutorial::Person::PhoneNumber* phone_number = person->add_phone(); + phone_number->set_number(number); + + cout << "Is this a mobile, home, or work phone? "; + string type; + getline(cin, type); + if (type == "mobile") { + phone_number->set_type(tutorial::Person::MOBILE); + } else if (type == "home") { + phone_number->set_type(tutorial::Person::HOME); + } else if (type == "work") { + phone_number->set_type(tutorial::Person::WORK); + } else { + cout << "Unknown phone type. Using default." << endl; + } + } +} + +// Main function: Reads the entire address book from a file, +// adds one person based on user input, then writes it back out to the same +// file. +int main(int argc, char* argv[]) { + // Verify that the version of the library that we linked against is + // compatible with the version of the headers we compiled against. + GOOGLE_PROTOBUF_VERIFY_VERSION; + + if (argc != 2) { + cerr << "Usage: " << argv[0] << " ADDRESS_BOOK_FILE" << endl; + return -1; + } + + tutorial::AddressBook address_book; + + { + // Read the existing address book. + fstream input(argv[1], ios::in | ios::binary); + if (!input) { + cout << argv[1] << ": File not found. Creating a new file." << endl; + } else if (!address_book.ParseFromIstream(&input)) { + cerr << "Failed to parse address book." << endl; + return -1; + } + } + + // Add an address. + PromptForAddress(address_book.add_person()); + + { + // Write the new address book back to disk. + fstream output(argv[1], ios::out | ios::trunc | ios::binary); + if (!address_book.SerializeToOstream(&output)) { + cerr << "Failed to write address book." << endl; + return -1; + } + } + + // Optional: Delete all global objects allocated by libprotobuf. + google::protobuf::ShutdownProtobufLibrary(); + + return 0; +} +``` + +注意 `GOOGLE_PROTOBUF_VERIFY_VERSION` 宏。它是一种好的实践——虽然不是严格必须的——在使用 C++ Protocol Buffer 库之前执行该宏。它可以保证避免不小心链接到一个与编译的头文件版本不兼容的库版本。如果被检查出来版本不匹配,程序将会终止。注意,每个 `.pb.cc` 文件在初始化时会自动调用这个宏。 + +同时注意在程序最后调用 `ShutdownProtobufLibrary()`。它用于释放 Protocol Buffer 库申请的所有全局对象。对大部分程序,这不是必须的,因为虽然程序只是简单退出,但是 OS 会处理释放程序的所有内存。然而,如果你使用了内存泄漏检测工具,工具要求全部对象都要释放,或者你正在写一个库,该库可能会被一个进程多次加载和卸载,那么你可能需要强制 Protocol Buffer 清除所有东西。 + +# 读取消息 + +当然,如果你无法从它获取任何信息,那么这个地址簿没多大用处!这个示例读取上面例子创建的文件,并打印文件里的所有内容。 + +```c++ +#include +#include +#include +#include "addressbook.pb.h" +using namespace std; + +// Iterates though all people in the AddressBook and prints info about them. +void ListPeople(const tutorial::AddressBook& address_book) { + for (int i = 0; i < address_book.person_size(); i++) { + const tutorial::Person& person = address_book.person(i); + + cout << "Person ID: " << person.id() << endl; + cout << " Name: " << person.name() << endl; + if (person.has_email()) { + cout << " E-mail address: " << person.email() << endl; + } + + for (int j = 0; j < person.phone_size(); j++) { + const tutorial::Person::PhoneNumber& phone_number = person.phone(j); + + switch (phone_number.type()) { + case tutorial::Person::MOBILE: + cout << " Mobile phone #: "; + break; + case tutorial::Person::HOME: + cout << " Home phone #: "; + break; + case tutorial::Person::WORK: + cout << " Work phone #: "; + break; + } + cout << phone_number.number() << endl; + } + } +} + +// Main function: Reads the entire address book from a file and prints all +// the information inside. +int main(int argc, char* argv[]) { + // Verify that the version of the library that we linked against is + // compatible with the version of the headers we compiled against. + GOOGLE_PROTOBUF_VERIFY_VERSION; + + if (argc != 2) { + cerr << "Usage: " << argv[0] << " ADDRESS_BOOK_FILE" << endl; + return -1; + } + + tutorial::AddressBook address_book; + + { + // Read the existing address book. + fstream input(argv[1], ios::in | ios::binary); + if (!address_book.ParseFromIstream(&input)) { + cerr << "Failed to parse address book." << endl; + return -1; + } + } + + ListPeople(address_book); + + // Optional: Delete all global objects allocated by libprotobuf. + google::protobuf::ShutdownProtobufLibrary(); + + return 0; +} +``` + +# 扩展 Protocol Buffer + +早晚在你发布了使用 protocol buffer 的代码之后,毫无疑问,你会想要 "改善" + protocol buffer 的定义。如果你想要新的 buffers 向后兼容,并且老的 buffers 向前兼容——几乎可以肯定你很渴望这个——这里有一些规则,你需要遵守。在新的 protocol buffer 版本: + + * 你绝不可以修改任何已存在字段的标签数字 + * 你绝不可以添加或删除任何 required 字段 + * 你可以删除 optional 或 repeated 字段 + * 你可以添加新的 optional 或 repeated 字段,但是你必须使用新的标签数字(也就是说,标签数字在 protocol buffer 中从未使用过,甚至不能是已删除字段的标签数字)。 + + (这是对于上面规则的一些[异常情况](https://developers.google.com/protocol-buffers/docs/proto#updating),但它们很少用到。) + + 如果你能遵守这些规则,旧代码则可以欢快地读取新的消息,并且简单地忽略所有新的字段。对于旧代码来说,被删除的 optional 字段将会简单地赋予默认值,被删除的 `repeated` 字段会为空。新代码显然可以读取旧消息。然而,请记住新的 optional 字段不会呈现在旧消息中,因此你需要显式地使用 `has_` 检查它们是否被设置或者在 `.proto` 文件在标签数字后使用 `[default = value]` 提供一个合理的默认值。如果一个 optional 元素没有指定默认值,它将会使用类型特定的默认值:对于字符串,默认值为空字符串;对于布尔值,默认值为 false;对于数字类型,默认类型为 0。注意,如果你添加一个新的 repeated 字段,新代码将无法辨别它被留空(left empty)(被新代码)或者从没被设置(被旧代码),因为 repeated 字段没有 `has_` 标志。 + +# 优化技巧 + +C++ Protocol Buffer 库已极度优化过了。但是,恰当的用法能够更多地提高性能。这里是一些技巧,可以帮你从库中挤压出最后一点速度: + +* 尽可能复用消息对象。即使它们被清除掉,消息也会尽量保存所有被分配来重用的内存。因此,如果我们正在处理许多相同类型或一系列相似结构的消息,一个好的办法是重用相同的消息对象,从而减少内存分配的负担。但是,随着时间的流逝,对象可能会膨胀变大,尤其是当你的消息尺寸(译者注:各消息内容不同,有些消息内容多一些,有些消息内容少一些)不同的时候,或者你偶尔创建了一个比平常大很多的消息的时候。你应该自己通过调用 [SpaceUsed](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message.SpaceUsed.details) 方法监测消息对象的大小,并在它太大的时候删除它。 + +* 对于在多线程中分配大量小对象的情况,你的操作系统内存分配器可能优化得不够好。你可以尝试使用 google 的 [tcmalloc](http://code.google.com/p/google-perftools/)。 + +# 高级用法 + +Protocol Buffers 绝不仅用于简单的数据存取以及序列化。请阅读 [C++ API reference](https://developers.google.com/protocol-buffers/docs/reference/cpp/index.html) 来看看你还能用它来做什么。 + +protocol 消息类所提供的一个关键特性就是反射。你不需要编写针对一个特殊的消息类型的代码,就可以遍历一个消息的字段并操作它们的值。一个使用反射的有用方法是 protocol 消息与其他编码互相转换,比如 XML 或 JSON。反射的一个更高级的用法可能就是可以找出两个相同类型的消息之间的区别,或者开发某种 "协议消息的正则表达式",利用正则表达式,你可以对某种消息内容进行匹配。只要你发挥你的想像力,就有可能将 Protocol Buffers 应用到一个更广泛的、你可能一开始就期望解决的问题范围上。 + +反射是由 [Message::Reflection interface](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message.Reflection) 提供的。 + +-------------------------------------------------------------------------------- + +via: https://developers.google.com/protocol-buffers/docs/cpptutorial + +作者:[Google][a] +译者:[cposture](https://github.com/cposture) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From 6e297e7ae07dd686f6f237030e8f296ddf1869fb Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 20 Aug 2016 23:25:40 +0800 Subject: [PATCH 0015/2437] =?UTF-8?q?=E6=9B=B4=E6=96=B0=E6=A0=BC=E5=BC=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../20160820 Protocol Buffer Basics C++.md | 67 ++++++++++++------- 1 file changed, 41 insertions(+), 26 deletions(-) diff --git a/translated/tech/20160820 Protocol Buffer Basics C++.md b/translated/tech/20160820 Protocol Buffer Basics C++.md index 11bda510d4..6554564ca2 100644 --- a/translated/tech/20160820 Protocol Buffer Basics C++.md +++ b/translated/tech/20160820 Protocol Buffer Basics C++.md @@ -1,12 +1,15 @@ +Protocol Buffer Basics: C++ +============================ + 这篇教程提供了一个面向 C++ 程序员、关于 `protocol buffers` 的基础介绍。通过创建一个简单的示例应用程序,它将向我们展示: * 在 `.proto` 文件中定义消息格式 * 使用 `protocol buffer` 编译器 * 使用 `C++ protocol buffer API` 读写消息 -这不是一个关于使用 C++ protocol buffers 的全面指南。要获取更详细的信息,请参考 [Protocol Buffer Language Guide](https://developers.google.com/protocol-buffers/docs/proto) 和 [Encoding Reference](https://developers.google.com/protocol-buffers/docs/encoding)。 +这不是一个关于使用 C++ protocol buffers 的全面指南。要获取更详细的信息,请参考 [Protocol Buffer Language Guide][1] 和 [Encoding Reference][2]。 -# 为什么使用 Protocol Buffers +### 为什么使用 Protocol Buffers 我们接下来要使用的例子是一个非常简单的"地址簿"应用程序,它能从文件中读取联系人详细信息。地址簿中的每一个人都有一个名字,ID,邮件地址和联系电话。 @@ -22,7 +25,7 @@ # 在哪可以找到示例代码 -示例代码被包含于源代码包,位于 "examples" 文件夹。在[这](https://developers.google.com/protocol-buffers/docs/downloads)下载代码。 +示例代码被包含于源代码包,位于 "examples" 文件夹。在[这][4]下载代码。 # 定义你的协议格式 @@ -71,17 +74,15 @@ message AddressBook { * repeated:字段可以重复任意次数(包括 0)。repeated 值的顺序会被保存于 protocol buffer。可以将 repeated 字段想象为动态大小的数组。 -你可以查找关于编写 `.proto` 文件的完整指导——包括所有可能的字段类型——在 [Protocol Buffer Language Guide](https://developers.google.com/protocol-buffers/docs/proto)。不要在这里面查找与类继承相似的特性,因为 protocol buffers 不会做这些。 - -Required Is Forever You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field – old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some engineers at Google have come to the conclusion that using required does more harm than good; they prefer to use only optional and repeated. However, this view is not universal. +你可以查找关于编写 `.proto` 文件的完整指导——包括所有可能的字段类型——在 [Protocol Buffer Language Guide][6]。不要在这里面查找与类继承相似的特性,因为 protocol buffers 不会做这些。 > required 是永久性的,在把一个字段标识为 required 的时候,你应该特别小心。如果在某些情况下你不想写入或者发送一个 required 的字段,那么将该字段更改为 optional 可能会遇到问题——旧版本的读者(译者注:即读取、解析旧版本 Protocol Buffer 消息的一方)会认为不含该字段的消息是不完整的,从而有可能会拒绝解析。在这种情况下,你应该考虑编写特别针对于应用程序的、自定义的消息校验函数。Google 的一些工程师得出了一个结论:使用 required 弊多于利;他们更愿意使用 optional 和 repeated 而不是 required。当然,这个观点并不具有普遍性。 -# 编译你的 Protocol Buffers +### 编译你的 Protocol Buffers 既然你有了一个 `.proto`,那你需要做的下一件事就是生成一个将用于读写 `AddressBook` 消息的类(从而包括 `Person` 和 `PhoneNumber`)。为了做到这样,你需要在你的 `.proto` 上运行 protocol buffer 编译器 `protoc`: -1. 如果你没有安装编译器,请[下载这个包](https://developers.google.com/protocol-buffers/docs/downloads.html),并按照 README 中的指令进行安装。 +1. 如果你没有安装编译器,请[下载这个包][4],并按照 README 中的指令进行安装。 2. 现在运行编译器,知道源目录(你的应用程序源代码位于哪里——如果你没有提供任何值,将使用当前目录),目标目录(你想要生成的代码放在哪里;常与 `$SRC_DIR` 相同),并且你的 `.proto` 路径。在此示例,你...: ``` @@ -95,7 +96,7 @@ protoc -I=$SRC_DIR --cpp_out=$DST_DIR $SRC_DIR/addressbook.proto * `addressbook.pb.h`,声明你生成类的头文件。 * `addressbook.pb.cc`,包含你的类的实现。 -# Protocol Buffer API +### Protocol Buffer API 让我们看看生成的一些代码,了解一下编译器为你创建了什么类和函数。如果你查看 `tutorial.pb.h`,你可以看到有一个在 `tutorial.proto` 中指定所有消息的类。关注 `Person` 类,可以看到编译器为每个字段生成了读写函数(accessors)。例如,对于 `name`、`id`、`email` 和 `phone` 字段,有下面这些方法: @@ -143,15 +144,15 @@ repeated 字段也有一些特殊的方法——如果你看看 repeated `phone` * 更新特定下标的电话号码 * 添加新的电话号码到消息中,之后你便可以编辑。(repeated 标量类型有一个 `add_` 方法,用于传入新的值) -为了获取 protocol 编译器为所有字段定义生成的方法的信息,可以查看 [C++ generated code reference](https://developers.google.com/protocol-buffers/docs/reference/cpp-generated)。 +为了获取 protocol 编译器为所有字段定义生成的方法的信息,可以查看 [C++ generated code reference][5]。 -### 枚举和嵌套类(Enums and Nested Classes) +#### 枚举和嵌套类(Enums and Nested Classes) 与 `.proto` 的枚举相对应,生成的代码包含了一个 `PhoneType` 枚举。你可以通过 `Person::PhoneType` 引用这个类型,通过 `Person::MOBILE`、`Person::HOME` 和 `Person::WORK` 引用它的值。(实现细节有点复杂,但是你无须了解它们而可以直接使用) 编译器也生成了一个 `Person::PhoneNumber` 的嵌套类。如果你查看代码,你可以发现真正的类型为 `Person_PhoneNumber`,但它通过在 `Person` 内部使用 typedef 定义,使你可以把 `Person_PhoneNumber` 当成嵌套类。唯一产生影响的一个例子是,如果你想要在其他文件前置声明该类——在 C++ 中你不能前置声明嵌套类,但是你可以前置声明 `Person_PhoneNumber`。 -### 标准的消息方法 +#### 标准的消息方法 所有的消息方法都包含了许多别的方法,用于检查和操作整个消息,包括: @@ -160,12 +161,11 @@ repeated 字段也有一些特殊的方法——如果你看看 repeated `phone` * `void CopyFrom(const Person& from);`:使用给定的值重写消息。 * `void Clear();`:清除所有元素为空(empty)的状态。 -上面这些方法以及下一节要讲的 I/O 方法实现了被所有 C++ protocol buffer 类共享的消息(Message)接口。为了获取更多信息,请查看 [complete API documentation for Message](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message)。 +上面这些方法以及下一节要讲的 I/O 方法实现了被所有 C++ protocol buffer 类共享的消息(Message)接口。为了获取更多信息,请查看 [complete API documentation for Message][7]。 -### 解析和序列化(Parsing and Serialization) +#### 解析和序列化(Parsing and Serialization) -最后,所有 protocol buffer 类都有读写你选定类型消息的方法,这些方法使用了特定的 protocol buffer [二进制格式](https://developers.google.com/protocol-buffers/docs/encoding) -。这些方法包括: +最后,所有 protocol buffer 类都有读写你选定类型消息的方法,这些方法使用了特定的 protocol buffer [二进制格式][8]。这些方法包括: * `bool SerializeToString(string* output) const;`:序列化消息以及将消息字节数据存储在给定的字符串。注意,字节数据是二进制格式的,而不是文本格式;我们只使用 `string` 类作为合适的容器。 * `bool ParseFromString(const string& data);`:从给定的字符创解析消息。 @@ -176,7 +176,7 @@ repeated 字段也有一些特殊的方法——如果你看看 repeated `phone` > Protocol Buffers 和 面向对象设计的 Protocol buffer 类通常只是纯粹的数据存储器(像 C++ 中的结构体);它们在对象模型中并不是一等公民。如果你想向生成的 protocol buffer 类中添加更丰富的行为,最好的方法就是在应用程序中对它进行封装。如果你无权控制 .proto 文件的设计的话,封装 protocol buffers 也是一个好主意(例如,你从另一个项目中重用一个 .proto 文件)。在那种情况下,你可以用封装类来设计接口,以更好地适应你的应用程序的特定环境:隐藏一些数据和方法,暴露一些便于使用的函数,等等。但是你绝对不要通过继承生成的类来添加行为。这样做的话,会破坏其内部机制,并且不是一个好的面向对象的实践。 -# 写消息(Writing A Message) +### 写消息(Writing A Message) 现在我们尝试使用 protocol buffer 类。你的地址簿程序想要做的第一件事是将个人详细信息写入到地址簿文件。为了做到这一点,你需要创建、填充 protocol buffer 类实例,并且将它们写入到一个输出流(output stream)。 @@ -282,7 +282,7 @@ int main(int argc, char* argv[]) { 同时注意在程序最后调用 `ShutdownProtobufLibrary()`。它用于释放 Protocol Buffer 库申请的所有全局对象。对大部分程序,这不是必须的,因为虽然程序只是简单退出,但是 OS 会处理释放程序的所有内存。然而,如果你使用了内存泄漏检测工具,工具要求全部对象都要释放,或者你正在写一个库,该库可能会被一个进程多次加载和卸载,那么你可能需要强制 Protocol Buffer 清除所有东西。 -# 读取消息 +### 读取消息 当然,如果你无法从它获取任何信息,那么这个地址簿没多大用处!这个示例读取上面例子创建的文件,并打印文件里的所有内容。 @@ -355,7 +355,7 @@ int main(int argc, char* argv[]) { } ``` -# 扩展 Protocol Buffer +### 扩展 Protocol Buffer 早晚在你发布了使用 protocol buffer 的代码之后,毫无疑问,你会想要 "改善" protocol buffer 的定义。如果你想要新的 buffers 向后兼容,并且老的 buffers 向前兼容——几乎可以肯定你很渴望这个——这里有一些规则,你需要遵守。在新的 protocol buffer 版本: @@ -365,25 +365,25 @@ int main(int argc, char* argv[]) { * 你可以删除 optional 或 repeated 字段 * 你可以添加新的 optional 或 repeated 字段,但是你必须使用新的标签数字(也就是说,标签数字在 protocol buffer 中从未使用过,甚至不能是已删除字段的标签数字)。 - (这是对于上面规则的一些[异常情况](https://developers.google.com/protocol-buffers/docs/proto#updating),但它们很少用到。) + (这是对于上面规则的一些[异常情况][9],但它们很少用到。) 如果你能遵守这些规则,旧代码则可以欢快地读取新的消息,并且简单地忽略所有新的字段。对于旧代码来说,被删除的 optional 字段将会简单地赋予默认值,被删除的 `repeated` 字段会为空。新代码显然可以读取旧消息。然而,请记住新的 optional 字段不会呈现在旧消息中,因此你需要显式地使用 `has_` 检查它们是否被设置或者在 `.proto` 文件在标签数字后使用 `[default = value]` 提供一个合理的默认值。如果一个 optional 元素没有指定默认值,它将会使用类型特定的默认值:对于字符串,默认值为空字符串;对于布尔值,默认值为 false;对于数字类型,默认类型为 0。注意,如果你添加一个新的 repeated 字段,新代码将无法辨别它被留空(left empty)(被新代码)或者从没被设置(被旧代码),因为 repeated 字段没有 `has_` 标志。 -# 优化技巧 +### 优化技巧 C++ Protocol Buffer 库已极度优化过了。但是,恰当的用法能够更多地提高性能。这里是一些技巧,可以帮你从库中挤压出最后一点速度: -* 尽可能复用消息对象。即使它们被清除掉,消息也会尽量保存所有被分配来重用的内存。因此,如果我们正在处理许多相同类型或一系列相似结构的消息,一个好的办法是重用相同的消息对象,从而减少内存分配的负担。但是,随着时间的流逝,对象可能会膨胀变大,尤其是当你的消息尺寸(译者注:各消息内容不同,有些消息内容多一些,有些消息内容少一些)不同的时候,或者你偶尔创建了一个比平常大很多的消息的时候。你应该自己通过调用 [SpaceUsed](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message.SpaceUsed.details) 方法监测消息对象的大小,并在它太大的时候删除它。 +* 尽可能复用消息对象。即使它们被清除掉,消息也会尽量保存所有被分配来重用的内存。因此,如果我们正在处理许多相同类型或一系列相似结构的消息,一个好的办法是重用相同的消息对象,从而减少内存分配的负担。但是,随着时间的流逝,对象可能会膨胀变大,尤其是当你的消息尺寸(译者注:各消息内容不同,有些消息内容多一些,有些消息内容少一些)不同的时候,或者你偶尔创建了一个比平常大很多的消息的时候。你应该自己通过调用 [SpaceUsed][10] 方法监测消息对象的大小,并在它太大的时候删除它。 -* 对于在多线程中分配大量小对象的情况,你的操作系统内存分配器可能优化得不够好。你可以尝试使用 google 的 [tcmalloc](http://code.google.com/p/google-perftools/)。 +* 对于在多线程中分配大量小对象的情况,你的操作系统内存分配器可能优化得不够好。你可以尝试使用 google 的 [tcmalloc][11]。 -# 高级用法 +### 高级用法 -Protocol Buffers 绝不仅用于简单的数据存取以及序列化。请阅读 [C++ API reference](https://developers.google.com/protocol-buffers/docs/reference/cpp/index.html) 来看看你还能用它来做什么。 +Protocol Buffers 绝不仅用于简单的数据存取以及序列化。请阅读 [C++ API reference][12] 来看看你还能用它来做什么。 protocol 消息类所提供的一个关键特性就是反射。你不需要编写针对一个特殊的消息类型的代码,就可以遍历一个消息的字段并操作它们的值。一个使用反射的有用方法是 protocol 消息与其他编码互相转换,比如 XML 或 JSON。反射的一个更高级的用法可能就是可以找出两个相同类型的消息之间的区别,或者开发某种 "协议消息的正则表达式",利用正则表达式,你可以对某种消息内容进行匹配。只要你发挥你的想像力,就有可能将 Protocol Buffers 应用到一个更广泛的、你可能一开始就期望解决的问题范围上。 -反射是由 [Message::Reflection interface](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message.Reflection) 提供的。 +反射是由 [Message::Reflection interface][13] 提供的。 -------------------------------------------------------------------------------- @@ -394,3 +394,18 @@ via: https://developers.google.com/protocol-buffers/docs/cpptutorial 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://developers.google.com/protocol-buffers/docs/cpptutorial +[1]: https://developers.google.com/protocol-buffers/docs/proto +[2]: https://developers.google.com/protocol-buffers/docs/encoding +[3]: https://developers.google.com/protocol-buffers/docs/downloads +[4]: https://developers.google.com/protocol-buffers/docs/downloads.html +[5]: https://developers.google.com/protocol-buffers/docs/reference/cpp-generated +[6]: https://developers.google.com/protocol-buffers/docs/proto +[7]: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message +[8]: https://developers.google.com/protocol-buffers/docs/encoding +[9]: https://developers.google.com/protocol-buffers/docs/proto#updating +[10]: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message.SpaceUsed.details +[11]: http://code.google.com/p/google-perftools/ +[12]: https://developers.google.com/protocol-buffers/docs/reference/cpp/index.html +[13]: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message.html#Message.Reflection From 235cfd46d6eca3821c8889008474ef9441b6ebb4 Mon Sep 17 00:00:00 2001 From: cposture Date: Sun, 21 Aug 2016 14:21:48 +0800 Subject: [PATCH 0016/2437] Translated by cposture --- translated/tech/20160820 Protocol Buffer Basics C++.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/translated/tech/20160820 Protocol Buffer Basics C++.md b/translated/tech/20160820 Protocol Buffer Basics C++.md index 6554564ca2..ed86014010 100644 --- a/translated/tech/20160820 Protocol Buffer Basics C++.md +++ b/translated/tech/20160820 Protocol Buffer Basics C++.md @@ -23,11 +23,11 @@ Protocol Buffer Basics: C++ `Protocol buffers` 是针对这个问题的一种灵活、高效、自动化的解决方案。使用 `Protocol buffers`,你需要写一个 `.proto` 说明,用于描述你所希望存储的数据结构。利用 `.proto` 文件,protocol buffer 编译器可以创建一个类,用于实现自动化编码和解码高效的二进制格式的 protocol buffer 数据。产生的类提供了构造 `protocol buffer` 的字段的 getters 和 setters,并且作为一个单元,关注读写 `protocol buffer` 的细节。重要的是,`protocol buffer` 格式支持扩展格式,代码仍然可以读取以旧格式编码的数据。 -# 在哪可以找到示例代码 +### 在哪可以找到示例代码 示例代码被包含于源代码包,位于 "examples" 文件夹。在[这][4]下载代码。 -# 定义你的协议格式 +### 定义你的协议格式 为了创建自己的地址簿应用程序,你需要从 `.proto` 开始。`.proto` 文件中的定义很简单:为你所需要序列化的数据结构添加一个消息(message),然后为消息中的每一个字段指定一个名字和类型。这里是定义你消息的 `.proto` 文件,`addressbook.proto`。 From dbff81552c0af8291aec9a66d32262705589eeb8 Mon Sep 17 00:00:00 2001 From: hkurj <663831938@qq.com> Date: Mon, 29 Aug 2016 22:40:16 +0800 Subject: [PATCH 0017/2437] translating --- sources/tech/20160811 5 best linux init system.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160811 5 best linux init system.md b/sources/tech/20160811 5 best linux init system.md index 460acdef3f..74b1d44d6d 100644 --- a/sources/tech/20160811 5 best linux init system.md +++ b/sources/tech/20160811 5 best linux init system.md @@ -1,3 +1,4 @@ +hkurj translating 5 Best Modern Linux ‘init’ Systems (1992-2015) ============================================ From c5e9ef7a5c2ece98a2b693280441b879d03013c8 Mon Sep 17 00:00:00 2001 From: hkurj <663831938@qq.com> Date: Mon, 29 Aug 2016 22:47:35 +0800 Subject: [PATCH 0018/2437] translating --- sources/tech/20160811 5 best linux init system.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sources/tech/20160811 5 best linux init system.md b/sources/tech/20160811 5 best linux init system.md index 74b1d44d6d..2296b6d751 100644 --- a/sources/tech/20160811 5 best linux init system.md +++ b/sources/tech/20160811 5 best linux init system.md @@ -1,5 +1,5 @@ -hkurj translating -5 Best Modern Linux ‘init’ Systems (1992-2015) +hkurj translating +I Best Modern Linux ‘init’ Systems (1992-2015) ============================================ In Linux and other Unix-like operating systems, the init (initialization) process is the first process executed by the kernel at boot time. It has a process ID (PID) of 1, it is executed in the background until the system is shut down. From c8a5ffe07bbe2d985ff05b66e914252ec42c3114 Mon Sep 17 00:00:00 2001 From: wxy Date: Wed, 31 Aug 2016 11:29:22 +0800 Subject: [PATCH 0019/2437] PUB:20160810 A brief introduction to Linux containers and image signing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @Tanete 翻译的很棒! --- ...n to Linux containers and image signing.md | 59 +++++++++++++++++++ ...n to Linux containers and image signing.md | 59 ------------------- 2 files changed, 59 insertions(+), 59 deletions(-) create mode 100644 published/20160810 A brief introduction to Linux containers and image signing.md delete mode 100644 translated/tech/20160810 A brief introduction to Linux containers and image signing.md diff --git a/published/20160810 A brief introduction to Linux containers and image signing.md b/published/20160810 A brief introduction to Linux containers and image signing.md new file mode 100644 index 0000000000..7e6f6ef2e7 --- /dev/null +++ b/published/20160810 A brief introduction to Linux containers and image signing.md @@ -0,0 +1,59 @@ +浅谈 Linux 容器和镜像签名 +==================== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/containers_2015-1-osdc-lead.png?itok=E1imOYe4) + +从根本上说,几乎所有的主要软件,即使是开源软件,都是在基于镜像的容器技术出现之前设计的。这意味着把软件放到容器中相当于是一次平台移植。这也意味着一些程序可以很容易就迁移,[而一些就更困难][1]。 + +我大约在三年半前开展基于镜像的容器相关工作。到目前为止,我已经容器化了大量应用。我了解到什么是现实情况,什么是迷信。今天,我想简要介绍一下 Linux 容器是如何设计的,以及谈谈镜像签名。 + +### Linux 容器是如何设计的 + +对于基于镜像的 Linux 容器,让大多数人感到困惑的是,它把操作系统分割成两个部分:[内核空间与用户空间][2]。在传统操作系统中,内核运行在硬件上,你无法直接与其交互。用户空间才是你真正能交互的,这包括所有你可以通过文件浏览器或者运行`ls`命令能看到的文件、类库、程序。当你使用`ifconfig`命令调整 IP 地址时,你实际上正在借助用户空间的程序来使内核根据 TCP 协议栈改变。这点经常让没有研究过 [Linux/Unix 基础][3]的人大吃一惊。 + +过去,用户空间中的类库支持了与内核交互的程序(比如 ifconfig、sysctl、tuned-adm)以及如网络服务器和数据库之类的面向用户的程序。这些所有的东西都堆积在一个单一的文件系统结构中。用户可以在 /sbin 或者 /lib 文件夹中找到所有操作系统本身支持的程序和类库,或者可以在 /usr/sbin 或 /usr/lib 文件夹中找到所有面向用户的程序或类库(参阅[文件系统层次结构标准][4])。这个模型的问题在于操作系统程序和业务支持程序没有绝对的隔离。/usr/bin 中的程序可能依赖 /lib 中的类库。如果一个应用所有者需要改变一些东西,就很有可能破坏操作系统。相反地,如果负责安全更新的团队需要改变一个类库,就(常常)有可能破坏面向业务的应用。这真是一团糟。 + +借助基于镜像的容器,比如 Docker、LXD、RKT,应用程序所有者可以打包和调整所有放在 /sbin、/lib、/usr/bin 和 /usr/lib 中的依赖部分,而不用担心破坏底层操作系统。本质上讲,容器技术再次干净地将操作系统隔离为两部分:内核空间与用户空间。现在开发人员和运维人员可以分别独立地更新各自的东西。 + +然而还是有些令人困扰的地方。通常,每个应用所有者(或开发者)并不想负责更新这些应用依赖:像 openssl、glibc,或很底层的基础组件,比如,XML 解析器、JVM,再或者处理与性能相关的设置。过去,这些问题都委托给运维团队来处理。由于我们在容器中打包了很多依赖,对于很多组织来讲,对容器内的所有东西负责仍是个严峻的问题。 + +### 迁移现有应用到 Linux 容器 + +把应用放到容器中算得上是平台移植,我准备突出介绍究竟是什么让移植某些应用到容器当中这么困难。 + +(通过容器,)开发者现在对 /sbin 、/lib、 /usr/bin、 /usr/lib 中的内容有完全的控制权。但是,他们面临的挑战是,他们仍需要将数据和配置放到 /etc 或者 /var/lib 文件夹中。对于基于镜像的容器来说,这是一个糟糕的想法。我们真正需要的是代码、配置以及数据的隔离。我们希望开发者把代码放在容器当中,而数据和配置通过不同的环境(比如,开发、测试或生产环境)来获得。 + +这意味着我们(或者说平台)在实例化容器时,需要挂载 /etc 或 /var/lib 中的一些文件或文件夹。这会允许我们到处移动容器并仍能从环境中获得数据和配置。听起来很酷吧?这里有个问题,我们需要能够干净地隔离配置和数据。很多现代开源软件比如 Apache、MySQL、MongoDB、Nginx 默认就这么做了。[但很多自产的、历史遗留的、或专有程序并未默认这么设计][5]。对于很多组织来讲,这是主要的痛点。对于开发者来讲的最佳实践是,开始架构新的应用,移植遗留代码,以完成配置和数据的完全隔离。 + +### 镜像签名简介 + +信任机制是容器的重要议题。容器镜像签名允许用户添加数字指纹到镜像中。这个指纹随后可被加密算法测试验证。这使得容器镜像的用户可以验证其来源并信任。 + +容器社区经常使用“容器镜像”这个词组,但这个命名方法会让人相当困惑。Docker、LXD 和 RKT 推行获取远程文件来当作容器运行这样的概念。这些技术各自通过不同的方式处理容器镜像。LXD 用单独的一层来获取单独一个容器,而 Docker 和 RKT 使用基于开放容器镜像(OCI)格式,可由多层组成。糟糕的是,会出现不同团队和组织对容器镜像中的不同层负责的情况。容器镜像概念下隐含的是容器镜像格式的概念。拥有标准的镜像格式比如 OCI 会让容器生态系统围绕着镜像扫描、签名,和在不同云服务提供商间转移而繁荣发展。 + +现在谈到签名了。 + +容器存在一个问题,我们把一堆代码、二进制文件和类库放入其中。一旦我们打包了代码,我们就要把它和必要的文件服务器(注册服务器)共享。代码只要被共享,它基本上就是不具名的,缺少某种密文签名。更糟糕的是,容器镜像经常由不同人或团队控制的各个镜像层组成。每个团队都需要能够检查上一个团队的工作,增加他们自己的工作内容,并在上面添加他们自己的批准印记。然后他们需要继续把工作交给下个团队。 + +(由很多镜像组成的)容器镜像的最终用户需要检查监管链。他们需要验证每个往其中添加文件的团队的可信度。对于最终用户而言,对容器镜像中的每一层都有信心是极其重要的。 + +*作者 Scott McCarty 于 8 月 24 日在 ContainerCon 会议上作了题为 [Containers for Grownups: Migrating Traditional & Existing Applications][6] 的报告,更多内容请参阅报告[幻灯片][7]。* + +-------------------------------------------------------------------------------- + +via: https://opensource.com/bus/16/8/introduction-linux-containers-and-image-signing + +作者:[Scott McCarty][a] +译者:[Tanete](https://github.com/Tanete) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/fatherlinux +[1]: http://rhelblog.redhat.com/2016/04/21/architecting-containers-part-4-workload-characteristics-and-candidates-for-containerization/ +[2]: http://rhelblog.redhat.com/2015/07/29/architecting-containers-part-1-user-space-vs-kernel-space/ +[3]: http://rhelblog.redhat.com/tag/architecting-containers/ +[4]: https://linux.cn/article-6132-1.html +[5]: http://rhelblog.redhat.com/2016/04/21/architecting-containers-part-4-workload-characteristics-and-candidates-for-containerization/ +[6]: https://lcccna2016.sched.org/event/7JUc/containers-for-grownups-migrating-traditional-existing-applications-scott-mccarty-red-hat +[7]: http://schd.ws/hosted_files/lcccna2016/91/Containers%20for%20Grownups_%20Migrating%20Traditional%20%26%20Existing%20Applications.pdf diff --git a/translated/tech/20160810 A brief introduction to Linux containers and image signing.md b/translated/tech/20160810 A brief introduction to Linux containers and image signing.md deleted file mode 100644 index 019d3d03ba..0000000000 --- a/translated/tech/20160810 A brief introduction to Linux containers and image signing.md +++ /dev/null @@ -1,59 +0,0 @@ -Linux 容器和镜像签名简介 -==================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/containers_2015-1-osdc-lead.png?itok=E1imOYe4) - -从根本上说,几乎所有的主要软件,即使开源软件,都是在基于镜像的容器技术出现之前设计的。这意味着把软件放到容器中相当于是一次平台移植。这也意味着一些程序可以很容易就迁移,[而一些就更困难][1]。 - -我大约在 3 年半前开展基于镜像的容器相关工作。到目前为止,我已经容器化了大量应用。我了解到什么是真实的,什么是迷信。今天,我想简要介绍一下 Linux 容器是如何设计的,以及谈谈镜像签名。 - -### Linux 容器是如何设计的 - -对于基于镜像的 Linux 容器,让大多数人感到困惑的是,它把操作系统分割成两个部分:[内核空间与用户空间][2]。在传统操作系统中,内核运行在硬件上,你无法直接与其交互。用户空间才是你真正能交互的,这包括所有你可以通过文件浏览器或者运行`ls`命令能看到的文件、类库、程序。当你使用`ifconfig`命令调整 IP 地址时,你实际上正在借助用户空间的程序来使内核根据 TCP 协议栈改变。这点经常让没有研究过 [Linux/Unix 基础][3]的人大吃一惊。 - -过去,用户空间中的类库曾经支持过与内核交互的程序比如 (ifconfig, sysctl, tuned-adm) 以及如网络服务器和数据库之类的面向用户的程序。但这些都在单文件系统层次体系中取消了。用户可以通过检查 /sbin 或者 /lib 文件夹,查看所有面向用户的程序和类库(查阅[文件系统层次结构标准][4])。这个模型的问题在于操作系统程序和业务支持程序没有绝对的隔离。/usr/bin 中的程序可能依赖 /lib 中的类库。如果一个应用所有者需要改变一些东西,就很有可能破坏操作系统。相反地,如果负责安全更新的团队需要改变一个类库,就可能(常有的事)破坏面向业务的应用。这真是一团糟。 - -借助基于镜像的容器,比如 Docker, LXD, RKT, 应用程序所有者可以打包和调整所有 /sbin, /lib, /usr/bin, 和 /usr/lib 中的依赖,而不用担心破坏底层操作系统。本质上讲,容器技术再次干净地将操作系统隔离为两部分,内核空间与用户空间。现在开发人员和运维人员可以分别独立地更新各自的东西。 - -然而还是有些令人困扰的地方。通常,每个应用所有者(或开发者)并不想负责更新这些应用依赖:像 openssl、glibc,或很底层的基础组件,比如,XML 解析器、JVM,再或者处理与性能相关的设置。过去,这些问题都委托给运维团队来处理。由于我们在容器中打包了很多依赖,对于很多组织来讲,对容器内的所有东西负责仍是个严峻的问题。 - -### 迁移现有应用到 Linux 容器 - -把应用放到容器中算得上是平台移植,我准备突出介绍究竟是什么让移植某些应用到容器当中这么困难。 - -开发者现在对 /sbin /lib /usr/bin /usr/lib 中的内容有完全的控制权。但是,他们面临的挑战是,他们仍需要将数据和配置放到 /etc 或者 /var/lib 文件夹中。对于基于镜像的容器来说,这是一个糟糕的想法。我们真正需要的是代码、配置以及数据的隔离。我们希望开发者把代码放在容器当中,而数据和配置通过环境来获得,比如,开发、测试或生产环境。 - -这意味着我们在实例化容器时,需要挂载 /etc 或 /var/lib 中的一些文件或文件夹。这会允许我们到处移动容器并仍能从环境中获得数据和配置。听起来很酷吧?这里有个问题,我们需要能够干净地隔离配置和数据。很多现代开源软件比如 Apache,MySQL,MongoDB, Nginx 默认就这么做了。[但很多自产的,遗留的,或专有程序并未默认这么设计][5]。对于很多组织来讲,这是主要的痛点。对于开发者来讲的最佳实践是,开始架构新的应用,移植遗留代码,以完成配置和数据的完全隔离。 - -### 镜像签名简介 - -信任机制是容器的重要议题。容器镜像签名允许用户添加数字指纹到镜像中。这个指纹随后可被加密测试验证。这使得容器镜像的用户可以验证其来源并信任。 - -容器社区经常使用“容器镜像”这个词组,但这个命名方法会让人相当困惑。Docker,LXD,和 RKT 推行这样的概念,获取远程文件来当作容器运行。这些技术各自通过不同的方式处理容器镜像。LXD 用单独的一层来获取单独一个容器,而 Docker 和 RKT 使用基于开放容器镜像(OCI)格式,可由多层组成。糟糕的是,会存在不同团队和组织对容器镜像中的不同层负责。容器镜像概念下隐含的是容器镜像格式的概念。拥有统一的镜像格式比如 OCI 会让容器生态系统围绕着镜像扫描,签名,和在不同云服务提供商间转移而繁荣发展。 - -现在谈到签名了。 - -容器存在一个问题,我们把一堆代码,二进制文件,和类库放放入其中。一旦我们打包了代码,我们就要把它和必要的文件服务器(注册服务器)共享。代码只要被共享,它基本上就是不具名的,缺少某种密文签名。更糟糕的是,容器镜像经常由不同人或团队控制的各个镜像层组成。每个团队都需要能够检查上一个团队的工作,增加他们自己的工作,并在上面添加批准的印记。他们需要继续把工作交给下个团队。 - -(由很多镜像组成的)容器镜像的最终用户需要检查监管链。他们需要验证每个往其中添加文件的团队的可信度。对于最终用户而言,对容器镜像中的每一层都有信心是极其重要的。 - -作者 Scott McCarty 于 8 月 24 日在 ContainerCon 会议上作了题为 [Containers for Grownups: Migrating Traditional & Existing Applications][6] 的报告,更多内容请参阅报告[幻灯片][7]。 - --------------------------------------------------------------------------------- - -via: https://opensource.com/bus/16/8/introduction-linux-containers-and-image-signing - -作者:[Scott McCarty][a] -译者:[Tanete](https://github.com/Tanete) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/fatherlinux -[1]: http://rhelblog.redhat.com/2016/04/21/architecting-containers-part-4-workload-characteristics-and-candidates-for-containerization/ -[2]: http://rhelblog.redhat.com/2015/07/29/architecting-containers-part-1-user-space-vs-kernel-space/ -[3]: http://rhelblog.redhat.com/tag/architecting-containers/ -[4]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard -[5]: http://rhelblog.redhat.com/2016/04/21/architecting-containers-part-4-workload-characteristics-and-candidates-for-containerization/ -[6]: https://lcccna2016.sched.org/event/7JUc/containers-for-grownups-migrating-traditional-existing-applications-scott-mccarty-red-hat -[7]: http://schd.ws/hosted_files/lcccna2016/91/Containers%20for%20Grownups_%20Migrating%20Traditional%20%26%20Existing%20Applications.pdf From b133687364d7038c136529f283030d5cad497739 Mon Sep 17 00:00:00 2001 From: Bian Jiaping Date: Thu, 1 Sep 2016 00:51:25 +0800 Subject: [PATCH 0020/2437] [Translated] What Datatype Should You Use to Represent Time in MySQL --- ...ould You Use to Represent Time in MySQL.md | 192 ------ ...ould You Use to Represent Time in MySQL.md | 563 ++++++++++++++++++ 2 files changed, 563 insertions(+), 192 deletions(-) delete mode 100644 sources/tech/20160823 What Datatype Should You Use to Represent Time in MySQL.md create mode 100644 translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md diff --git a/sources/tech/20160823 What Datatype Should You Use to Represent Time in MySQL.md b/sources/tech/20160823 What Datatype Should You Use to Represent Time in MySQL.md deleted file mode 100644 index fed4229110..0000000000 --- a/sources/tech/20160823 What Datatype Should You Use to Represent Time in MySQL.md +++ /dev/null @@ -1,192 +0,0 @@ -Translating by bianjp - -What Datatype Should You Use to Represent Time in MySQL? We Compare Datetime, Timestamp and INT -========================================================== - -![](http://www.vertabelo.com/_file/blog/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int/clock.jpg) - -Whenever you need to save datetime data, a question arises about what MySQL type to use. Do you go with a native MySQL DATE type or use an INT field to store date and time info as a plain number? - -In this article, I’ll explain MySQL’s native options and give you a comparison table of the most common datatypes. We’ll also benchmark some typical queries and reach some conclusions about which datatype to use in a given situation. - -### Native MySQL Datetime Datatypes - -Datetime data represents a point in time. This could be a log entry, an Internet of Things timestamp, calendar event data, etc. MySQL has two native types that allow us to save this information in a single field: Datetime and Timestamp. Here’s what the MySQL documentation says about these datatypes: - ->The DATETIME type is used for values that contain both date and time parts. MySQL retrieves and displays DATETIME values in 'YYYY-MM-DD HH:MM:SS' format. - ->The TIMESTAMP data type is used for values that contain both date and time parts. - ->A DATETIME or TIMESTAMP value can include a trailing fractional seconds part in up to microseconds (6 digits) precision. - ->The TIMESTAMP and DATETIME data types offer automatic initialization and updating to the current date and time using DEFAULT CURRENT_TIMESTAMP and ON UPDATE CURRENT_TIMESTAMP clauses in column definitions. - - -So, as an example: - -``` -CREATE TABLE `datetime_example` ( - `id` int(11) NOT NULL AUTO_INCREMENT, - `measured_on` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, - PRIMARY KEY (`id`), - KEY `measured_on` (`measured_on`) -) ENGINE=InnoDB; -``` - -``` -CREATE TABLE `timestamp_example` ( - `id` int(11) NOT NULL AUTO_INCREMENT, - `measured_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, - PRIMARY KEY (`id`), - KEY `measured_on` (`measured_on`) -) ENGINE=InnoDB; -``` -Besides the native datetime representations, there is another common approach to storing date and time information. This is to use an INT field to store the Unix time (the number of seconds that have elapsed since the foundation of Coordinated Universal Time (UTC) on January 1, 1970). - -MySQL also provides a way to save part of a time representation by using the Date, Year or even Time types. Since this article is about the best way to store an exact point in time, we are not including these less-precise partial types in our discussion. - -### Using the INT Type with Unix Time - -Using a plain INT column type to store Unix time is the most trivial approach. With INT, you can feel certain that the number you’re storing can quickly and reliably be inserted into a table, like this: - -``` -INSERT INTO `vertabelo`.`sampletable` -( - `id`, - `measured_on` #INT TYPE COLUMN -) -VALUES -( - 1, - 946684801 - #unix stamp equivalent to 01/01/2000 @ 12:00am (UTC) http://unixtimestamp.com -); -``` - -That’s all there is to it. It’s just a plain ol’ INT column and MySQL will treat it as such, using 4 bytes to store that data internally. So if you perform a SELECT on this you will get a number, period. If you would like to make any comparison using this column as a date, the following query wouldn’t work properly: - -``` -SELECT - id, measured_on, FROM_UNIXTIME(measured_on) -FROM - vertabelo.inttimestampmeasures -WHERE - measured_on > '2016-01-01' #measured_on is compared as a string to resolve the query -LIMIT 5; -``` - -This is because MySQL sees INT as a number, not a date. In order to make a date comparison, you must either obtain the seconds elapsed up to 2016-01-01 or you will need to use the FROM_UNIXTIME() MySQL function to convert the INT column to a Datetype. The following query demonstrates the use of the FROM_UNIXTIME() function: - -``` -SELECT - id, measured_on, FROM_UNIXTIME(measured_on) -FROM - vertabelo.inttimestampmeasures -WHERE - FROM_UNIXTIME(measured_on) > '2016-01-01' -LIMIT 5; -``` - -This will properly retrieve records of dates after 2016-01-01. You can also compare your number directly against the Unix representation of 2016-01-01, which is 1451606400. Doing this means there’s no need to use any special function, since you are comparing straight numbers. Here’s the query: - -``` -SELECT - id, measured_on, FROM_UNIXTIME(measured_on) -FROM - vertabelo.inttimestampmeasures -WHERE - measured_on > 1451606400 -LIMIT 5; -``` - -What if it’s simply not efficient or even feasible to make this conversion in advance? For instance, say that you want all the records from every Wednesday in 2016. In order to do this without any MySQL date functions, you’d have to get the Unix timestamp for the start and end date and time for each of the Wednesdays in 2016. Then you’d have to write a huge query that will have at least 104 comparisons in the WHERE. (There are 52 Wednesdays in 2016, and you have to consider the start (0:00 am) and end of the day (11:59:59 pm)...) - -The bottom line is that it’s quite probable that you will end up using the FROM_UNIXTIME() conversion function after all. So, why not give the actual date types a try? - -### Using Datetime and Timestamp - -Datetime and Timestamp work pretty much the same way. Both store date and time information with up to six digits precision on fractional seconds. Also, using a human-readable date like ‘2016-01-01’ (to facilitate comparisons) will work. And both formats support “relaxed formatting” when performing queries. Relaxed syntax will permit any punctuation character as the delimiter. For instance, take a string in either YYYY-MM-DD HH:MM:SS or YY-MM-DD HH:MM:SS format. Any of these versions will work in a relaxed formatting situation: - -``` -2012-12-31 11:30:45 -2012^12^31 11+30+45 -2012/12/31 11*30*45 -2012@12@31 11^30^45 -``` - -Other relaxed formats are allowed; you can find them all in the [MySQL Reference Manual][1]. - -By default, both Datetime and Timestamp formats are retrieved in the standard output format – year-month-day hour:minute:second (e.g. 2016-01-01 23:59:59). If fractional seconds are used, they will come after the seconds as a decimal value (e.g. 2016-01-01 23:59:59.5). - -The core differences between Timestamp and Datetime relate primarily to how MySQL represents this information internally: both are stored as binary rather than strings, but Timestamp will use one less byte (four bytes) than Datetime (five bytes) to represent the date/time part. Both will use additional space (1-3 bytes) when saving fractional seconds. If you store 1.5 million records, this 1-byte difference is negligible: - -``` -1.5 million records * 1 extra byte per record / (1048576 bytes/MB) = 1.43 MB -``` - -Timestamp’s one-byte savings comes at a price: you can only store a value from '1970-01-01 00:00:01.000000' to '2038-01-19 03:14:07.999999'. Datetime, however, allows you to save any value from '1000-01-01 00:00:00.000000' to '9999-12-31 23:59:59.999999. - -An important difference – one that many MySQL developers are not aware of – is that MySQL uses the server time zone to convert a Timestamp value to its UTC equivalent and saves that. It will apply the time zone conversion again when retrieving the value, so you get your “original” date/time value back again. Maybe. Here’s what can happen. - -Ideally, if you stay in the same time zone, MySQL will retrieve the same value you stored. In my experience, you can run into issues if your database deals with a change in time zone. For instance, this can happen as a change in the server (e.g. you change your db from a server in Dublin to one in California, or you simply change the time zone in the server). Either way, if your time zone is different when you retrieve your data, the data may be impacted. - -Datetime columns are not changed by the db. They will store and retrieve the same value every time, independent from the configured time zone. Personally, I consider this a more consistent option. - -#### From MySQL documentation: - ->MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME, which is stored “as is”.) By default, the current time zone for each connection is the server’s time. The time zone can be set on a per-connection basis. As long as the time zone setting remains constant, you get back the same value you store. If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored. This occurs because the same time zone was not used for conversion in both directions. The current time zone is available as the value of the time_zone system variable. For more information, “MySQL Server Time Zone Support”. - -### Comparison Summary - -Before digging into the actual performance difference when using each of these datatypes, let’s consider a summary table that will give you some more insights. The weaknesses of each type are in red. - -Feature |Datetime | Timestamp | Int (store Unix time) -:--|:--|:--|:-- -Native time representation | Yes | Yes | No, so most operations need a conversion function first, like FROM_UNIXTIME() -Can store fractional seconds |Yes, up to 6 digits precision |Yes, up to 6 digits precision | No -Valid range | '1000-01-01 00:00:00.000000' to '9999-12-31 23:59:59.999999 | '1970-01-01 00:00:01.000000' to '2038-01-19 03:14:07.999999' | If unsigned, '1970-01-01 00:00:01.000000; theoretically up to '2106-2-07 06:28:15' -Auto-initialization (MySQL 5.6.5+) | Yes |Yes | No -Relaxed interpretation (MySQL docs) | Yes | Yes |No, you must use a proper format -Value is changed to UTC when stored |No | Yes | No -Can be changed to another type | Yes, if the value results in a valid Timestamp range | Yes, always |Yes, if the value results in a valid range and using a conversion function -Storage requirements (MySQL 5.6.4+) | 5 bytes (plus up to 3 bytes for fractional seconds, if used) | 4 bytes (plus up to 3 bytes for fractional seconds, if used) | 4 bytes (no fractional seconds allowed) -Readable as an actual date without further functions | Yes | Yes |No, you have to format the output -Partitioning | Yes | Yes, using the UNIX_TIMESTAMP(); any other expressions involving TIMESTAMP values are not permitted as per MySQL 5.7. Also, note these considerations on partition pruning. |Yes, using any valid operation on INTs - -Ideally, if you stay in the same time zone, MySQL will retrieve the same value you stored. In my experience, you can run into issues if your database deals with a change in time zone. For instance, this can happen as a change in the server (e.g. you change your db from a server in Dublin to one in California, or you simply change the time zone in the server). Either way, if your time zone is different when you retrieve your data, the data may be impacted. - -Datetime columns are not changed by the db. They will store and retrieve the same value every time, independent from the configured time zone. Personally, I consider this a more consistent option. - -#### From MySQL documentation: - ->MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME, which is stored “as is”.) By default, the current time zone for each connection is the server’s time. The time zone can be set on a per-connection basis. As long as the time zone setting remains constant, you get back the same value you store. If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored. This occurs because the same time zone was not used for conversion in both directions. The current time zone is available as the value of the time_zone system variable. For more information, “MySQL Server Time Zone Support”. - -### Comparison Summary - -Before digging into the actual performance difference when using each of these datatypes, let’s consider a summary table that will give you some more insights. The weaknesses of each type are in red. - -### Benchmarking INT, Timestamp, and Datetime Performance - -To compare the performance of each of these types, I’m using 1.5 million records (1,497,421 to be more precise) from a weather station network I built. This network collects data every minute. To make these tests replicable, I have removed some private columns so you can run your own tests on this data. - -From my original table, I created three versions: - -- The `datetimemeasures` table uses Datetime in the `measured_on` column to represent the moment that the weather station record was captured. -- The `timestampmeasures` table uses Timestamp for the `measured_on` column. -- The `inttimestampmeasures` uses INT (unsigned) for the `measured_on` column. - -These three tables have exactly the same data; the only difference is the `measured_on` field type. All tables have an index set on the `measured_on` column. - --------------------------------------------------------------------------------- - -via: http://www.vertabelo.com/blog/technical-articles/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int?utm_source=dbweekly&utm_medium=email - -作者:[Francisco Claria][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.axones.com.ar/ -[1]: http://dev.mysql.com/doc/refman/5.7/en/date-and-time-literals.html - diff --git a/translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md b/translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md new file mode 100644 index 0000000000..7e6ab10c0e --- /dev/null +++ b/translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md @@ -0,0 +1,563 @@ +MySQL 中你应该使用什么数据类型表示时间?(比较 Datetime, Timestamp 和 Int) +========================================================== + +![](http://www.vertabelo.com/_file/blog/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int/clock.jpg) + +_当你需要保存日期时间数据时,一个问题来了:使用 MySQL 的什么类型。使用 MySQL 原生的 DATE 类型还是使用 INT 字段把日期和时间保存为一个纯数字呢?_ + +在这篇文章中,我将解释 MySQL 原生的方案,并给出一个最常用数据类型的对比表。我们也将对一些典型的查询做基准测试然后得出在给定场景下应该使用什么数据类型的结论。 + +### 原生的 MySQL Datetime 数据类型 + +Datetime 数据表示一个时间点。这可以是条日志记录、物联网时间戳、日历事件数据,等等。MySQL 有两种原生的类型可以将这种信息保存在单个字段中:Datetime 和 Timestamp。MySQL 文档中是这么介绍这些数据类型的: + +>DATETIME 类型用于保存同时包含日期和时间两部分的值。MySQL 以 'YYYY-MM-DD HH:MM:SS' 形式接收和显示 DATETIME 类型的值。 + +>TIMESTAMP 类型用于保存同时包含日期和时间两部分的值。 + +>DATETIME 或 TIMESTAMP 类型的值可以在尾部包含一个毫秒部分,精确度最高到微秒(6 位数)。 + +>TIMESTAMP 和 DATETIME 数据类型提供自动初始化和更新到当前的日期和时间的功能,只需在列的定义中设置 DEFAULT CURRENT_TIMESTAMP 和 ON UPDATE CURRENT_TIMESTAMP。 + +So, as an example: +作为一个例子: + +```SQL +CREATE TABLE `datetime_example` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `measured_on` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + PRIMARY KEY (`id`), + KEY `measured_on` (`measured_on`) +) ENGINE=InnoDB; +``` + +```SQL +CREATE TABLE `timestamp_example` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `measured_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + PRIMARY KEY (`id`), + KEY `measured_on` (`measured_on`) +) ENGINE=InnoDB; +``` + +除了原生的日期时间表示方法,还有另一种常用的方法用以存储日期和时间信息。即使用 INT 字段保存 Unix 时间(从1970年1月1日协调世界时(UTC)建立经过的秒数)。 + +MySQL 也提供了只保存时间信息中的一部分的方式,通过使用 Date, Year, 或 Time 类型。由于这篇文章是关于保存准确时间点的最佳方式的,我们没有讨论这些不那么精确的局部类型。 + +### 使用 INT 类型保存 Unix 时间 + +使用一个简单的 INT 列保存 Unix 时间是最普通的方法。使用 INT,你可以确保你要保存的数字可以快速、可靠地插入到表中,就像这样: + +```SQL +INSERT INTO `vertabelo`.`sampletable` +( + `id`, + `measured_on` # INT 类型的列 +) +VALUES +( + 1, + 946684801 + #unix stamp equivalent to 01/01/2000 @ 12:00am (UTC) http://unixtimestamp.com +); +``` + +这就是关于它的所有了。它仅仅是个简单的 INT 列,MySQL 会这么处理它,在内部使用 4 bytes 保存那些数据。所以如果你在这个列上使用 SELECT 你将会得到一个数字。如果你想把这个列用作日期进行比较,下面的查询并不能正确工作: + +```SQL +SELECT + id, measured_on, FROM_UNIXTIME(measured_on) +FROM + vertabelo.inttimestampmeasures +WHERE + measured_on > '2016-01-01' # measured_on 作为字符串比较以解析查询 +LIMIT 5; +``` + +这是因为 MySQL 把 INT 视为数字,而非日期。为了进行日期比较,你必须要么获取(译注:从 1970-01-01)到 2016-01-01 经过的秒数,要么使用 MySQL 的 FROM_UNIXTIME() 函数把 INT 列转为 Date 类型。下面的查询展示了 FROM_UNIXTIME() 函数的用法: + +```SQL +SELECT + id, measured_on, FROM_UNIXTIME(measured_on) +FROM + vertabelo.inttimestampmeasures +WHERE + FROM_UNIXTIME(measured_on) > '2016-01-01' +LIMIT 5; +``` + +这会正确地获取到日期在 2016-01-01 之后的记录。你也可以直接比较数字和 2016-01-01 的 Unix 表示形式,即 1451606400。这样做意味着不用使用任何特殊的函数,因为你是在直接比较数字。查询如下: + +```SQL +SELECT + id, measured_on, FROM_UNIXTIME(measured_on) +FROM + vertabelo.inttimestampmeasures +WHERE + measured_on > 1451606400 +LIMIT 5; +``` + +如果这种方式仅仅是不够高效甚至提前做这种转换是不可行的,那该怎么办?例如,你想获取 2016 年所有星期三的记录。要做到这样而不使用任何 MySQL 日期函数,你不得不查出 2016 年每个星期三的开始和结束时间的 Unix 时间戳。然后你不得不写很大的查询,至少要在 WHERE 中包含 104 个比较。(2016 年有 52 个星期三,你不得不考虑一天的开始(0:00 am)和结束(11:59:59 pm)...) + +结果是你很可能最终会使用 FROM_UNIXTIME() 转换函数。既然如此,为什么不试下真正的日期类型呢? + +### 使用 Datetime 和 Timestamp + +Datetime 和 Timestamp 几乎以同样的方式工作。两种都保存日期和时间信息,毫秒部分最高精确度都是 6 位数。同时,使用人类可读的日期形式如 "2016-01-01" (为了便于比较)都能工作。查询时两种类型都支持“宽松格式”。宽松的语法允许任何标点符号作为分隔符。例如,"YYYY-MM-DD HH:MM:SS" 和 "YY-MM-DD HH:MM:SS" 两种形式都可以。在宽松格式情况下以下任何一种形式都能工作: + +``` +2012-12-31 11:30:45 +2012^12^31 11+30+45 +2012/12/31 11*30*45 +2012@12@31 11^30^45 +``` + +其它宽松格式也是允许的;你可以在 [MySQL 参考手册](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-literals.html) 找到所有的。 + +默认情况下,Datetime 和 Timestamp 两种类型都以标准输出格式接收 —— 年-月-日 时:分:秒 (如 2016-01-01 23:59:59)。如果使用了毫秒部分,它们应该以小数值出现在秒后面 (如 2016-01-01 23:59:59.5)。 + +Timestamp 和 Datetime 的核心不同点主要在于 MySQL 在内部如何表示这些信息:两种都以二进制而非字符串形式存储,但在表示日期/时间部分时 Timestamp (4 bytes) 比 Datetime (5 bytes) 少使用 1 byte。当保存毫秒部分时两种都使用额外的空间 (1-3 bytes)。如果你存储 150 万条记录,这种 1 byte 的差异是微不足道的: + +>150 万条记录 * 1 byte 每条记录 / (1048576 bytes/MB) = __1.43 MB__ + +Timestamp 节省的 1 byte 是有代价的:你只能存储从 '1970-01-01 00:00:01.000000' 到 '2038-01-19 03:14:07.999999' 之间的时间。而 Datetime 允许你存储从 '1000-01-01 00:00:00.000000' 到 '9999-12-31 23:59:59.999999' 之间的任何时间。 + +一个重要的差别 —— 很多 MySQL 开发者没意识到的 —— 是 MySQL 使用__服务器的时区__转换 Timestamp 值到它的 UTC 等价值再保存。当获取值是它会再次进行时区转换,所以你得回了你“原始的”日期/时间值。也许是。下面这些情况可能会发生。 + +理想情况下,如果你保持在同一个时区,MySQL 会获取到和你存储的同样的值。以我的经验,如果你的数据库涉及时区变换,你可能会遇到问题。例如,服务器变化(比如,你把数据库从都柏林的一台服务器迁移到加利福尼亚的一台服务器上,或者你只是修改了服务器的时区)时可能会发生这种情况。不管哪种方式,如果你获取数据时的时区是不同的,数据就会受影响。 + +Datetime 列不会被数据库改变。无论时区怎样配置,每次都会保存和获取到同样的值。就我而言,我认为这是一个更一致的选择。 + +>__MySQL 文档:__ + +>MySQL 把 TIMESTAMP 值从当前的时区转换到 UTC 再存储,获取时再从 UTC 转回当前的时区。(其它类型如 DATETIME 不会这样,它们会原样保存。) 默认情况下,每个连接的当前时区都是服务器的时区。时区可以基于连接设置。只要时区设置保持一致,你就能得到和保存的相同的值。如果你保存了一个 TIMESTAMP 值,然后改变了时区再获取这个值,获取到的值和你存储的是不同的。这是因为在不同方向的会话上没有使用同一个时区。当前时区可以通过系统变量 [time_zone](https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_time_zone) 的值得到。更多信息,请查看 [MySQL Server Time Zone Support](https://dev.mysql.com/doc/refman/5.7/en/time-zone-support.html)。 + +### 对比总结 + +在深入探讨使用各数据类型的性能差异之前,让我们先看一个总结表格以给你更多了解。每种类型的弱点以红色显示。 + +特性 | Datetime | Timestamp | Int (保存 Unix 时间) +:--|:--|:--|:-- +原生时间表示 | 是 | 是 | 否,所以大多数操作需要先使用转换函数,如 FROM_UNIXTIME() +能保存毫秒 | 是,最高 6 位精度 | 是,最高 6 位精度 | 否 +合法范围 | '1000-01-01 00:00:00.000000' to '9999-12-31 23:59:59.999999 | '1970-01-01 00:00:01.000000' to '2038-01-19 03:14:07.999999' | 若使用 unsigned, '1970-01-01 00:00:01.000000; 理论上最大到 '2106-2-07 06:28:15' +自动初始化(MySQL 5.6.5+) | 是 | 是 | 否 +宽松解释 ([MySQL docs](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-literals.html)) | 是 | 是 | 否,必须使用正确的格式 +值被转换到 UTC 存储 | 否 | 是 | 否 +可转换到其它类型 | 是,如果值在合法的 Timestamp 范围中 | 是,总是 | 是,如果值在合法的范围中并使用转换函数 +存储需求([MySQL 5.6.4+](https://dev.mysql.com/doc/refman/5.7/en/storage-requirements.html)) | 5 bytes (如果使用了毫秒部分,再加最多 3 bytes) | 4 bytes (如果使用了毫秒部分,再加最多 3 bytes) | 4 bytes (不允许毫秒部分) +无需使用函数即可作为真实日期可读 | 是 | 是 | 否,你不得不格式化输出 +分区 | 是 | 是,使用 [UNIX_TIMESTAMP()](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html#function_unix-timestamp);在 MySQL 5.7 中包含 [TIMESTAMP](https://dev.mysql.com/doc/refman/5.7/en/datetime.html) 值的其它表达式是不允许的。同时,注意[分区裁剪时的这些考虑](https://dev.mysql.com/doc/refman/5.7/en/partitioning-pruning.html) | 是,使用 INTs 上的任何合法操作 + + +### 基准测试 INT, Timestamp 和 Datetime 的性能 + +为了比较这些类型的性能,我会使用来自我创建的一个天气预报网络的 150 万记录(准确说是 1,497,421)。这个网络每分钟都收集数据。为了让这些测试可复现,我已经删除了一些私有列,所以你可以使用这些数据运行你自己的测试。 + +基于我原始的表格,我创建了三个版本: + +- `datetimemeasures` 表在 `measured_on` 列使用 Datetime 类型,表示天气预报记录的测量时间 +- `timestampmeasures` 表在 `measured_on` 列使用 Timestamp 类型 +- `inttimestampmeasures` 表在 `measured_on` 列使用 INT (unsigned) 类型 + +这三个表拥有完全相同的数据;唯一的差别就是 `measured_on` 字段的类型。所有表都在 `measured_on` 列上设置了一个索引。 + +![](./images/table-datetime.png) +![](./images/table-timestamp.png) +![](./images/table-int.png) + +#### 基准测试工具 + +为了评估这些数据类型的性能,我使用了两种方法。一种基于 [Sysbench](https://github.com/akopytov/sysbench),它的官网是这么描述的: + +_“... 一个模块化、跨平台和多线程的基准测试工具,用以评估那些对运行高负载数据库的系统非常重要的系统参数。”_ + +这个工具是 [MySQL 文档](https://dev.mysql.com/downloads/benchmarks.html)中建议的. + +如果你使用 Windows (就像我),你可以下载一个包含可执行文件和我使用的测试查询的 zip 文件。他们基于 [一种推荐的基准测试方法](https://dba.stackexchange.com/questions/39221/stress-test-mysql-with-queries-captured-with-general-log-in-mysql)。 + +为了执行一个给定的测试,你可以使用下面的命令(插入你自己的连接参数): + +```bash +sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=sysbench_test_file.lua --num-threads=8 --max-requests=100 run +``` + +这会正常工作,如果 `sysbench_test_file.lua` 是测试文件,并包含各个测试中指向各个表的 SQL 查询。 + +为了进一步验证结果,我也运行了 [mysqlslap](https://dev.mysql.com/doc/refman/5.7/en/mysqlslap.html)。它的官网是这么描述的: + +_“[mysqlslap](https://dev.mysql.com/doc/refman/5.7/en/mysqlslap.html) 是一个诊断程序,为模拟 MySQL 服务器的客户端负载并报告各个阶段的用时而设计。它工作起来就像是很多客户端在同时访问服务器。_ + +记得这些测试中最重要的不是所需的_绝对_时间。而是在不同数据类型上执行相同查询时的_相对_时间。这两个基准测试工具的测试时间不一定相同,因为不同工具的工作方式不同。重要的是数据类型的比较,随着我们深入到测试中,这将会变得清楚。 + +#### 基准测试 + +我将使用三种可以评估几个性能方面的查询: + +* 时间范围选择 + + * 在 Datetime 和 Timestamp 数据类型上这允许我们直接比较而不需要使用任何特殊的日期函数 + * 同时,我们可以评估在 INT 类型的列上使用日期函数相对于使用简单的数值比较的影响。为了做到这些我们需要把范围转换为 Unix 时间戳数值。 + +* 日期函数选择: + + * 与前个测试中比较操作针对一个简单的 DATE 值相反,这个测试使得我们可以评估使用日期函数作为 “WHERE” 子句的一部分的性能。 + * 我们还可以测试一个场景即我们必须使用一个函数将 INT 列转换未一个合法的 DATE 类型然后执行查询。 + +* count() 查询 + + * 作为对前面测试的补充,这将评估在三种不同的表示类型上进行典型的摘要查询的性能。 + +我们将在这些测试中覆盖一些常见的场景,并看到三种类型上的性能表现。 + +#### 关于 SQL_NO_CACHE + +当在查询中使用 SQL_NO_CACHE 时,服务器不使用查询缓存。它既不检查查询缓存以确认结果是不是已经在那儿了,也不会保存查询结果。因此,每个查询将反映真实的性能影响,就像每次查询都是第一次被调用。 + +#### 测试 1:选择在一个日期范围中的值 + +这个查询返回总计 1,497,421 行记录中的 75,706 行。 + +##### 查询 1 和 Datetime: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.datetimemeasures m +WHERE + m.measured_on > '2016-01-01 00:00:00.0' + AND m.measured_on < '2016-02-01 00:00:00.0'; +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 152 | 296 | +| 最大 | 1261 | 3203 | +| 平均 | 362 | 809 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.datetimemeasures m WHERE m.measured_on > '2016-01-01 00:00:00.0' AND m.measured_on < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +##### 查询 1 和 Timestamp: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.timestampmeasures m +WHERE + m.measured_on > '2016-01-01 00:00:00.0' + AND m.measured_on < '2016-02-01 00:00:00.0'; +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 214 | 359 | +| 最大 | 1389 | 3313 | +| 平均 | 431 | 1004 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.timestampmeasures m WHERE m.measured_on > '2016-01-01 00:00:00.0' AND m.measured_on < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +##### 查询 1 和 INT: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.inttimestampmeasures m +WHERE + FROM_UNIXTIME(m.measured_on) > '2016-01-01 00:00:00.0' + AND FROM_UNIXTIME(m.measured_on) < '2016-02-01 00:00:00.0'; +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2472 | 7968 | +| 最大 | 6554 | 10312 | +| 平均 | 4107 | 8527 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE FROM_UNIXTIME(m.measured_on) > '2016-01-01 00:00:00.0' AND FROM_UNIXTIME(m.measured_on) < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +##### 另一种 INT 上的查询 1: + +由于这是个相当直接的范围搜索而且查询中的日期可以轻易地转为简单的数值比较,我将它包含在了这个测试中。结果证明这是最快的方法 (你大概已经预料到了),因为它仅仅是比较数字而没有使用任何日期转换函数: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.inttimestampmeasures m +WHERE + m.measured_on > 1451617200 + AND m.measured_on < 1454295600; +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 88 | 171 | +| 最大 | 275 | 2157 | +| 平均 | 165 | 514 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=basic_int.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE m.measured_on > 1451617200 AND m.measured_on < 1454295600" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +##### 测试 1 总结 + +| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | +|-----------------|----------|----------------------|-----------|----------------------| +| Datetime | 362 | - | 809 | - | +| Timestamp | 431 | 慢 19% | 1004 | 慢 24% | +| INT | 4107 | 慢 1134% | 8527 | 慢 1054% | +| 另一种 INT 查询 | 165 | 快 55% | 514 | 快 36% | + +两种基准测试工具都显示 Datetime 比 Timestamp 和 INT 更快。但 Datetime 没有我们在另一种 INT 查询中使用的简单数值比较快。 + +#### 测试 2:选择星期一产生的记录 + +这个查询返回总计 1,497,421 行记录中的 221,850 行。 + +##### 查询 2 和 Datetime: + +```SQL +SELECT SQL_NO_CACHE measured_on +FROM + vertabelo.datetimemeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 1874 | 4343 | +| 最大 | 6168 | 7797 | +| 平均 | 3127 | 6103 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime_1.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.datetimemeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +##### 查询 2 和 Timestamp: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.timestampmeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2688 | 5953 | +| 最大 | 6666 | 13531 | +| 平均 | 3653 | 8412 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp_1.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.timestampmeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +##### 查询 2 和 INT: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.inttimestampmeasures m +WHERE + WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0; # MONDAY +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2051 | 5844 | +| 最大 | 7007 | 10469 | +| 平均 | 3486 | 8088 | + + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int_1.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +##### 测试 2 总结 + +| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | +|-----------------|----------|----------------------|-----------|----------------------| +| Datetime | 3127 | - | 6103 | - | +| Timestamp | 3653 | 慢 17% | 8412 | 慢 38% | +| INT | 3486 | 慢 11% | 8088 | 慢 32% | + +再次,在两个基准测试工具中 Datetime 比 Timestamp 和 INT 快。但在这个测试中,INT 查询 —— 即使它使用了一个函数以转换日期 —— 比 Timestamp 查询更快得到结果。 + +#### 测试 3:选择星期一产生的记录总数 + +这个查询返回一行,包含产生于所有星期一的记录的总数(从总共 1,497,421 行可用记录中)。 + +##### 查询 3 和 Datetime: + +```SQL +SELECT SQL_NO_CACHE + COUNT(measured_on) +FROM + vertabelo.datetimemeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 1720 | 4063 | +| 最大 | 4594 | 7812 | +| 平均 | 2797 | 5540 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime_1_count.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.datetimemeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +##### 查询 3 和 Timestamp: + +```SQL +SELECT SQL_NO_CACHE + COUNT(measured_on) +FROM + vertabelo.timestampmeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 1907 | 4578 | +| 最大 | 5437 | 10235 | +| 平均 | 3408 | 7102 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp_1_count.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.timestampmeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +##### 查询 3 和 INT: + +```SQL +SELECT SQL_NO_CACHE + COUNT(measured_on) +FROM + vertabelo.inttimestampmeasures m +WHERE + WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0; # MONDAY +``` + +##### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2108 | 5609 | +| 最大 | 4764 | 9735 | +| 平均 | 3307 | 7416 | + +```bash +**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int_1_count.lua --num-threads=8 --max-requests=100 run +``` + +```bash +**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.inttimestampmeasures m WHERE WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +##### 测试 3 总结 + +| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | +|-----------------|----------|----------------------|-----------|----------------------| +| Datetime | 2797 | - | 5540 | - | +| Timestamp | 3408 | 慢 22% | 7102 | 慢 28% | +| INT | 3307 | 慢 18% | 7416 | 慢 33% | + +再一次,两个基准测试工具都显示 Datetime 比 Timestamp 和 INT 快。不能判断 INT 是否比 Timestamp 快,因为 mysqlslap 显示 INT 比 Timestamp 略快而 Sysbench 却相反。 + +_注意:_ 所有测试都是在一台 Windows 10 机器上本地运行的,这台机器拥有一个双核 i7 CPU,16GB 内存,运行 MariaDB v10.1.9,使用 innoDB 引擎。 + +### 结论 + +基于这些数据,我确信 Datetime 是大多数场景下的最佳选择。原因是: + +* 更快(根据我们的三个基准测试)。 +* 无需任何转换即是人类可读的。 +* 不会因为时区变换产生问题。 +* 只比它的对手们多用 1 byte +* 支持更大的日期范围(从 1000 年到 9999 年) + +如果你只是存储 Unix 时间戳(并且在它的合法日期范围内),而且你真的不打算在它上面使用任何基于日期的查询,我觉得使用 INT 是可以的。我们已经看到,它执行简单数值比较查询时非常快,因为只是在处理简单的数字。 + +Timestamp 怎么样呢?如果 Datetime 相对于 Timestamp 的优势不适用于你特殊的场景,你最好使用时间戳。阅读这篇文章后,你对三种类型间的区别应该有了更好的理解,可以根据你的需要做出最佳的选择。 + +-------------------------------------------------------------------------------- + +via: http://www.vertabelo.com/blog/technical-articles/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int?utm_source=dbweekly&utm_medium=email + +作者:[Francisco Claria][a] +译者:[bianjp](https://github.com/bianjp) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.axones.com.ar/ From ea4961fd506077fd81a9179ded5d60517de7b717 Mon Sep 17 00:00:00 2001 From: Tom Date: Thu, 1 Sep 2016 12:36:24 +0800 Subject: [PATCH 0021/2437] [translating] 20160814 How I Got to be an Android Dev And What I Learned From It --- ...How I Got to be an Android Dev And What I Learned From It.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md b/sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md index f8fe109506..7f8b4a109c 100644 --- a/sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md +++ b/sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md @@ -1,3 +1,5 @@ +JianhuanZhuo + How I Got to be an Android Dev And What I Learned From It ============================= From 5c45627d53263c43419354664b35344c5cfdeb42 Mon Sep 17 00:00:00 2001 From: WangYue <815420852@qq.com> Date: Thu, 1 Sep 2016 16:48:05 +0800 Subject: [PATCH 0022/2437] =?UTF-8?q?=E3=80=90=E7=BF=BB=E8=AF=91=E4=B8=AD?= =?UTF-8?q?=E3=80=9120160808=20Why=20measuring=20IT=20productivity=20is=20?= =?UTF-8?q?so=20challenging.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 【翻译中】20160808 Why measuring IT productivity is so challenging.md --- ...0160808 Why measuring IT productivity is so challenging.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sources/talk/20160808 Why measuring IT productivity is so challenging.md b/sources/talk/20160808 Why measuring IT productivity is so challenging.md index a0a29c57ae..62f446673d 100644 --- a/sources/talk/20160808 Why measuring IT productivity is so challenging.md +++ b/sources/talk/20160808 Why measuring IT productivity is so challenging.md @@ -1,3 +1,7 @@ +[WangYueScream 翻译中] + +=========================== + Why measuring IT productivity is so challenging =========================== From 25dd7df69e38c3d03ba43c20d27a2705e113aaf7 Mon Sep 17 00:00:00 2001 From: Markgolzh Date: Fri, 2 Sep 2016 09:23:18 +0800 Subject: [PATCH 0023/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E4=B8=AD=EF=BC=8D?= =?UTF-8?q?=EF=BC=8Dbyzky001?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 翻译中 --- sources/tech/20160816 Deploying React with Zero Configuration.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160816 Deploying React with Zero Configuration.md b/sources/tech/20160816 Deploying React with Zero Configuration.md index e2cb16a223..16b53e461c 100644 --- a/sources/tech/20160816 Deploying React with Zero Configuration.md +++ b/sources/tech/20160816 Deploying React with Zero Configuration.md @@ -1,3 +1,4 @@ +翻译中--byzky001 Deploying React with Zero Configuration ======================== From 45241dbbfd26e296f0d5aa391616e18e276a08b9 Mon Sep 17 00:00:00 2001 From: Markgolzh Date: Fri, 2 Sep 2016 09:43:18 +0800 Subject: [PATCH 0024/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=EF=BC=8Dby=20zky001?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...ilter and Edit - Demonstrated in Pandas.md | 328 ------------------ 1 file changed, 328 deletions(-) delete mode 100644 sources/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md diff --git a/sources/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md b/sources/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md deleted file mode 100644 index 4e9c973066..0000000000 --- a/sources/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md +++ /dev/null @@ -1,328 +0,0 @@ -翻译中 by-zky001 -Excel “Filter and Edit” - Demonstrated in Pandas -================================================== - -![](http://pbpython.com/images/Boolean-Indexing-Example.png) - -### Introduction - -I have heard from various people that my [previous][1] [articles][2] on common Excel tasks in pandas were useful in helping new pandas users translate Excel processes into equivalent pandas code. This article will continue that tradition by illustrating various pandas indexing examples using Excel’s Filter function as a model for understanding the process. - -One of the first things most new pandas users learn is basic data filtering. Despite working with pandas over the past few months, I recently realized that there was another benefit to the pandas filtering approach that I was not using in my day to day work. Namely that you can filter on a given set of columns but update another set of columns using a simplified pandas syntax. This is similar to what I’ll call the “Filter and Edit” process in Excel. - -This article will walk through some examples of filtering a pandas DataFrame and updating the data based on various criteria. Along the way, I will explain some more about panda’s indexing and how to use indexing methods such as .loc , .ix and .iloc to quickly and easily update a subset of data based on simple or complex criteria. - -### Excel: “Filter and Edit” - -Outside of the Pivot Table, one of the top go-to tools in Excel is the Filter. This simple tool allows a user to quickly filter and sort the data by various numeric, text and formatting criteria. Here is a basic screenshot of some sample data with data filtered by several different criteria: - -![](http://pbpython.com/images/filter-example.png) - -The Filter process is intuitive and is easy to grasp for even the most novice Excel user. I have also noticed that people will use this feature to select rows of data, then update additional columns based on the row criteria. The example below shows what I’m describing: - -![](http://pbpython.com/images/commission-example.png) - -In the example, I have filtered the data on Account Number, SKU and Unit Price. Then I manually added a Commission_Rate column and typed in 0.01 in each cell. The benefit to this approach is that it is easy to understand and can help someone manage relatively complex data without writing long Excel formulas or getting into VBA. The downside of this approach is that it is not repeatable and can be difficult for someone from the outside to understand which criteria were used for any filter. - -For instance, if you look at the screenshot about, there is no obvious way to tell what is filtered without looking at each column. Fortunately, we can do something very similar in pandas. Not surprisingly, it is easy in pandas to execute this “Filter and Edit” model with simple and clean code. - -### Boolean Indexing - -Now that you have a feel for the problem, I want to step through some details of boolean indexing in pandas. This is an important concept to understand if you want to understand pandas’ [Indexing and Selecting of Data][3] in the most broad sense. This idea may seem a little complex to the new pandas user (and maybe too basic for experienced users) but I think it is important to take some time and understand it. If you grasp this concept, the basic process of working with data in pandas will be more straightforward. - -Pandas supports indexing (or selecting data) by using labels, position based integers or a list of boolean values (True/False). Using a list of boolean values to select a row is called boolean indexing and will be the focus of the rest of this article. - -I find that my pandas workflow tends to focus mostly on using lists of boolean values for selecting my data. In other words, when I create pandas DataFrames, I tend to keep the default index in the DataFrame. Therefore the index is not really meaningful on its own and not straightforward for selecting data. - ->Key Point -> Boolean indexing is one (of several) powerful and useful ways of selecting rows of data in pandas. - -Let’s look at some example DataFrames to help clarify the what a boolean index in pandas does. - -First, we will create a very small DataFrame purely from a python list and use it to show how boolean indexing works. - -``` -import pandas as pd -sales = [('account', ['Jones LLC', 'Alpha Co', 'Blue Inc', 'Mega Corp']), - ('Total Sales', [150, 200, 75, 300]), - ('Country', ['US', 'UK', 'US', 'US'])] -df = pd.DataFrame.from_items(sales) -``` - - |account |Total Sales |Country -:--|:-- |:-- |: -0 |Jones LLC |150 |US -1 |Alpha Co |200 |UK -2 |Blue Inc |75 |US -3 |Mega Corp |300 |US - -Notice how the values 0-3, get automatically assigned to the rows? Those are the indices and they are not particularly meaningful in this data set but are useful to pandas and are important to understand for other use cases not described below. - -When we refer to boolean indexing, we simply mean that we can pass in a list of True or False values representing each row we want to view. - -In this case, if we want to view the data for Jones LLC, Blue Inc and Mega Corp, we can see that the True False list would look like this: - -``` -indices = [True, False, True, True] -``` - -It should be no surprise, that you can pass this list to your DataFrame and it will only display the rows where our value is True : - -``` -df[indices] -``` - - |account |Total Sales |Country -:--|:--|:--|:-- -0 |Jones LLC |150 |US -2 |Blue Inc |75 |US -3 |Mega Corp |300 |US - -Here is a visual of what just happened: - -![](http://pbpython.com/images/Boolean-Indexing-Example.png) - -This manual list creation of the index works but obviously is not scaleable or very useful for anything more than a trivial data set. Fortunately pandas makes it very easy to create these boolean indexes using a simple query language that should be familiar to someone that has used python (or any language for that matter). - -For an example, let’s look at all sales lines from the US. If we execute a python expression based on the Country column: - -``` -df.Country == 'US' -``` - -``` -0 True -1 False -2 True -3 True -Name: Country, dtype: bool -``` - -The example shows how pandas will take your traditional python logic, apply it to a DataFrame and return a list of boolean values. This list of boolean values can then be passed to the DataFrame to get the corresponding rows of data. - -In real code, you would not do this two step process. The shorthand method for doing this would typically look like this: - -``` -df[df["Country"] == 'US'] -``` - - |account |Total Sales |Country -:--|:--|:--|:-- -0 |Jones LLC |150| US -2 |Blue Inc |75 |US -3 |Mega Corp| 300| US - -While this concept is simple, you can write fairly complex logic to filter your data using the power of python. - ->Key Point ->In this example, `df[df.Country == 'US']` is equivalent to `df[df["Country"] == 'US']` The ‘.’ notation is cleaner but will not work when there are spaces in your column names. - -### Selecting the Columns - -Now that we have figured out how to select rows of data, how can we control which columns to display? In the example above, there’s no obvious way to do that. Pandas can support this use case using three types of location based indexing: .loc , iloc , and .ix . These functions also allow us to select columns in addition to the row selection we have seen so far. - -There is a lot of confusion about when to use .loc , iloc , or .ix . The quick summary of the difference is that: - -- .loc is used for label indexing -- .iloc is used for position based integers -- .ix is a shortcut that will try to use labels (like .loc ) but will fall back to position based integers (like .iloc ) - -So, the question is, which one should I use? I will profess that I get tripped up some times on this one too. I have found that I use .loc most frequently. Mainly because my data does not lend itself to meaningful position based indexing (in other words, I rarely find myself needing .iloc ) so I stick with .loc . - -To be fair each of these methods do have their place and are useful in many situations. One area in particular is when dealing with MultiIndex DataFrames. I will not cover that topic in this article - maybe in a future post. - -Now that we have covered this topic, let’s show how to filter a DataFrame on values in a row and select specific columns to display. - -Continuing with our example, what if we just want to show the account names that correspond to our index? Using .loc it is simple: - -``` -df.loc[[True, True, False, True], "account"] -``` - -``` -1 Alpha Co -2 Blue Inc -3 Mega Corp -Name: account, dtype: object -``` - -If you would like to see multiple columns, just pass a list: - -``` -df.loc[[True, True, False, True], ["account", "Country"]] -``` - - | account |Country -:--|:--|:-- -0 |Jones LLC| US -1 |Alpha Co |UK -3 |Mega Corp |US - -The real power is when you create more complex queries on your data. In this case, let’s show all account names and Countries where sales > 200: - -``` -df.loc[df["Total Sales"] > 200, ["account", "Country"]] -``` - - |account| Country - :--|:--|:-- -3 |Mega Corp| US - -This process can be thought of somewhat equivalent to Excel’s Filter we discussed above. You have the added benefit that you can also limit the number of columns you retrieve, not just the rows. - -### Editing Columns - -All of this is good background but where this process really shines is when you use a similar approach for updating one or more columns based on a row selection. - -For one simple example, let’s add a commission rate column to our data: - -``` -df["rate"] = 0.02 -``` - - | account |Total Sales| Country |rate -:--|:--|:--|:--|:-- -0 |Jones LLC |150 |US |0.02 -1 |Alpha Co |200 |UK |0.02 -2 |Blue Inc |75 |US |0.02 -3 |Mega Corp |300 |US |0.02 - -Let’s say that if you sold more than 100, your rate is 5%. The basic process is to setup a boolean index to select the columns, then assign the value to the rate column: - -``` -df.loc[df["Total Sales"] > 100, ["rate"]] = .05 -``` - - | account |Total Sales| Country| rate -:--|:--|:--|:--|:-- -0 |Jones LLC |150| US| 0.05 -1 |Alpha Co |200 |UK |0.05 -2| Blue Inc |75| US |0.02 -3 |Mega Corp |300| US| 0.05 - -Hopefully if you stepped through this article, this will make sense and that it will help you understand how this syntax works. Now you have the fundamentals of the “Filter and Edit” approach. The final section will show this process in a little more detail in Excel and pandas. - -### Bringing It All Together - -For the final example, we will create a simple commissions calculator using the following rules: - -- All commissions calculated at the transaction level -- Base commission on all sales is 2% -- All shirts will get a commission of 2.5% -- A special program is going on where selling > 10 belts in one transaction gets 4% commission -- There is a special bonus of $250 plus a 4.5% commission for all shoe sales > $1000 in a single transaction - -In order to do this in Excel, using the Filter and edit approach: - -- Add a commission column with 2% -- Add a bonus column of $0 -- Filter on shirts and change the vale to 2.5% -- Clear the filter -- Filter for belts and quantity > 10 and change the value to 4% -- Clear the filter -- Filter for shoes > $1000 and add commission and bonus values of 4.5% and $250 respectively - -I am not going to show a screen shot of each step but here is the last filter: - -![](http://pbpython.com/images/filter-2.png) - -This approach is simple enough to manipulate in Excel but it is not very repeatable nor audit-able. There are certainly other approaches to accomplish this in Excel - such as formulas or VBA. However, this Filter and Edit approach is common and is illustrative of the pandas logic. - -Now, let’s walk through the whole example in pandas. - -First, read in the [Excel file][4] and add a column with the 2% default rate: - -``` -import pandas as pd -df = pd.read_excel("https://github.com/chris1610/pbpython/blob/master/data/sample-sales-reps.xlsx?raw=true") -df["commission"] = .02 -df.head() -``` - - | account number| customer name| sales rep| sku |category |quantity |unit price| ext price| date| commission -:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:-- -0 |680916 |Mueller and Sons |Loring Predovic |GP-14407 | Belt |19 |88.49 |1681.31 |2015-11-17 05:58:34| 0.02 -1 |680916 |Mueller and Sons |Loring Predovic |FI-01804| Shirt |3| 78.07| 234.21 |2016-02-13 04:04:11 |0.02 -2 |530925 |Purdy and Sons| Teagan O’Keefe |EO-54210 |Shirt |19 |30.21 |573.99 |2015-08-11 12:44:38 |0.02 -3 |14406| Harber, Lubowitz and Fahey| Esequiel Schinner| NZ-99565| Shirt| 12| 90.29 |1083.48 |2016-01-23 02:15:50 |0.02 -4 |398620| Brekke Ltd |Esequiel Schinner |NZ-99565 |Shirt |5| 72.64 |363.20 |2015-08-10 07:16:03 |0.02 - -The next commission rule is for all shirts to get 2.5% and Belt sales > 10 get a 4% rate: - -``` -df.loc[df["category"] == "Shirt", ["commission"]] = .025 -df.loc[(df["category"] == "Belt") & (df["quantity"] >= 10), ["commission"]] = .04 -df.head() -``` - -| account number |customer name |sales rep| sku |category |quantity| unit price| ext price |date |commission - :--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:-- -0 |680916| Mueller and Sons| Loring Predovic| GP-14407| Belt| 19 |88.49| 1681.31 |2015-11-17 05:58:34| 0.040 -1 |680916| Mueller and Sons| Loring Predovic| FI-01804| Shirt |3 |78.07 |234.21| 2016-02-13 04:04:11 |0.025 -2 |530925 |Purdy and Sons |Teagan O’Keefe| EO-54210 |Shirt| 19 |30.21 |573.99 |2015-08-11 12:44:38| 0.025 -3 |14406| Harber, Lubowitz and Fahey| Esequiel Schinner| NZ-99565| Shirt| 12| 90.29| 1083.48| 2016-01-23 02:15:50 |0.025 -4 |398620| Brekke Ltd| Esequiel Schinner| NZ-99565| Shirt| 5 |72.64 |363.20| 2015-08-10 07:16:03 |0.025 - -The final commission rule is to add the special bonus: - -``` -df["bonus"] = 0 -df.loc[(df["category"] == "Shoes") & (df["ext price"] >= 1000 ), ["bonus", "commission"]] = 250, 0.045 - -# Display a sample of rows that show this bonus -df.ix[3:7] -``` - -| account number| customer name |sales rep| sku |category| quantity| unit price| ext price| date| commission| bonus - :--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:-- -3| 14406| Harber, Lubowitz and Fahey| Esequiel Schinner| NZ-99565| Shirt |12 |90.29| 1083.48 |2016-01-23 02:15:50 |0.025 |0 -4| 398620| Brekke Ltd| Esequiel Schinner |NZ-99565| Shirt| 5| 72.64| 363.20| 2015-08-10 07:16:03| 0.025| 0 -5| 282122| Connelly, Abshire and Von Beth| Skiles| GJ-90272| Shoes| 20| 96.62| 1932.40 |2016-03-17 10:19:05 |0.045 |250 -6 |398620 |Brekke Ltd| Esequiel Schinner |DU-87462 |Shirt| 10| 67.64 |676.40| 2015-11-25 22:05:36| 0.025| 0 -7| 218667| Jaskolski-O’Hara| Trish Deckow| DU-87462| Shirt |11| 91.86| 1010.46 |2016-04-24 15:05:58| 0.025| 0 - -In order to do the commissions calculation: - -``` -# Calculate the compensation for each row -df["comp"] = df["commission"] * df["ext price"] + df["bonus"] - -# Summarize and round the results by sales rep -df.groupby(["sales rep"])["comp"].sum().round(2) -``` - -``` -sales rep -Ansley Cummings 2169.76 -Beth Skiles 3028.60 -Esequiel Schinner 10451.21 -Loring Predovic 10108.60 -Shannen Hudson 5275.66 -Teagan O'Keefe 7989.52 -Trish Deckow 5807.74 -Name: comp, dtype: float64 -``` - -If you are interested, an example notebook is hosted on [github][5]. - -### Conclusion - -Thanks for reading through the article. I find that one of the biggest challenges for new users in learning how to use pandas is figuring out how to use their Excel-based knowledge to build an equivalent pandas-based solution. In many cases the pandas solution is going to be more robust, faster, easier to audit and more powerful. However, the learning curve can take some time. I hope that this example showing how to solve a problem using Excel’s Filter tool will be a useful guide for those just starting on this pandas journey. Good luck! - --------------------------------------------------------------------------------- - -via: http://pbpython.com/excel-filter-edit.html - -作者:[Chris Moffitt ][a] -译者:[zky001](https://github.com/zky001) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://pbpython.com/author/chris-moffitt.html -[1]: http://pbpython.com/excel-pandas-comp.html -[2]: http://pbpython.com/excel-pandas-comp-2.html -[3]: http://pandas.pydata.org/pandas-docs/stable/indexing.html -[4]: https://github.com/chris1610/pbpython/blob/master/data/sample-sales-reps.xlsx?raw=true -[5]: https://github.com/chris1610/pbpython/blob/master/notebooks/Commissions-Example.ipynb - From 5b9e93fdae3ca14950138c84f9664bb68fbfbda9 Mon Sep 17 00:00:00 2001 From: Markgolzh Date: Fri, 2 Sep 2016 09:50:30 +0800 Subject: [PATCH 0025/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=EF=BC=8Dbyzky001?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 翻译完成 --- ...ilter and Edit - Demonstrated in Pandas.md | 342 ++++++++++++++++++ 1 file changed, 342 insertions(+) create mode 100644 translated/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md diff --git a/translated/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md b/translated/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md new file mode 100644 index 0000000000..1742ab1f98 --- /dev/null +++ b/translated/tech/20160615 Excel Filter and Edit - Demonstrated in Pandas.md @@ -0,0 +1,342 @@ +Excel “过滤和编辑” - Pandas中的示范操作 +================================================== + +![](http://pbpython.com/images/Boolean-Indexing-Example.png) + +### 介绍 + +我听许多人说我[之前的关于pandas处理一般的Excel任务][1]的文章对于帮助新的pandas用户将Excel任务转化成对应的pandas代码是有用的。这边文章将会保持那个传统通过介绍不同的pandas使用Excel的过滤功能作为一个模型进行索引的例子来理解这样的处理。 + +大多数pandas的新手首先学到的东西就是基本的数据过滤。即使在过去的时间中使用pandas工作持续数月,我最近意识到我没有在我每天的工作中使用的pandas过滤的方法还有另外一个好处。也就是说,你可以在一个给定的列上过滤,但更新另一组列通过使用一个简化的pandas的语法。这和我所说的Excel中的“过滤和编辑”很像。 + +这篇文章将会介绍一些过滤一个pandas数据帧同时更新各种标准的例子。同时,我将解释关于panda的索引和如何使用索引功能例如 .loc , .ix和 .iloc来简单快速的更新一个基于简单或者是复杂标准的数据子集。 + +### Excel: “过滤与编辑” + +在数据透视表以外,在EXCEL工具的顶部是一过滤器。这个简单的工具允许用户通过不同的数据,文本与格式标准去快速过滤与排序数据。这里是由几个不同的条件过滤数据产生的样本数据的基本截图: + +![](http://pbpython.com/images/filter-example.png) +该过滤过程是直观的,即使是最新手的用户,也很容易掌握。我也注意到,人们将使用此功能来选择行的数据,然后根据行的标准更新额外的列。下面的例子显示了我所描述的情形: + +![](http://pbpython.com/images/commission-example.png) + +在这个例子中,我过滤了Account Number, SKU和Unit Price的数据。接着我手工增加了Commission_Rate这列并且在每个单元格中输入0.01。这种方法的好处是可以很容易理解和管理有联系的复杂数据而不用写长Excel公式或者进入VBA。这种方法的缺点是,它是不可重复的,同时,对于从外面入门的人了解哪些标准被用于任何过滤器可能是困难的。 + +例如,假设你看相关的截图,如果不看每一列数据,没有明显的方法去知道什么被过滤了。幸运的是,我们可以在pandas中完成相似操作。不奇怪的是,在pandas中使用简单干净的代码来执行“过滤和编辑”模式是简单的。 + + +### 布尔检索 + +现在,我想通过pandas中一些布尔索引的细节来使你对这个问题有点感觉。 +如果你想要去理解pandas的在大部分广泛的[索引和数据选择][3]这是一个重要的概念去理解。这种想法可能看起来有点复杂,对新的pandas用户(可能对经验丰富的用户来说太基础),但我认为重要的是要花一些时间,了解它。如果你掌握了这一概念,用pandas进行数据工作的基本过程将更简单。 + +pandas支持索引(或者选择数据)通过使用标签,基于数字的位置或者一个布尔值的队列(True/False)。使用一个布尔值的列表来选择一个行被称为布尔索引,将是本文其余部分的重点。 + +我发现,我的pandas的工作流程往往侧重于使用布尔值的列表选择我的数据。换句话说,当我创建了pandas数据帧,我倾向于保持默认的索引在数据框架。因此,索引本身并不具有真正意义,同时也不是简单的选择数据。 + + + +>关键点 +> pandas中布尔索引是一个(几个)选择行的数据的强大的和有用的方法。 + +让我们看一些例子,数据框架帮助澄清布尔指标在pandas中做的是什么。 + +首先,我们将创建一个非常小的数据帧仅仅从一个Python的列表获得并使用它来展示布尔检索是如何工作的。 + + +``` +import pandas as pd +sales = [('account', ['Jones LLC', 'Alpha Co', 'Blue Inc', 'Mega Corp']), + ('Total Sales', [150, 200, 75, 300]), + ('Country', ['US', 'UK', 'US', 'US'])] +df = pd.DataFrame.from_items(sales) +``` + + |account |Total Sales |Country +:--|:-- |:-- |: +0 |Jones LLC |150 |US +1 |Alpha Co |200 |UK +2 |Blue Inc |75 |US +3 |Mega Corp |300 |US + +注意值0-3,是怎么样会自动分配给行?这些都是指数,在这个数据集它们不是特别有意义的,但对pandas是有用的,重要的是要了解如下其他没有描述的使用情况下。 + +当我们引用布尔索引时,我们只是说,我们可以通过一个真实或错误的值的列表表示我们要查看的每一行。 + + +在这种情况下,如果我们想查看Jones有限责任公司,Blue公司和Mega公司的数据,我们可以看到,True False名单会看起来像这样: + +``` +indices = [True, False, True, True] +``` +它应该是毫不奇怪,你可以通过把这个列表,传入你的数据,你就看到它只会显示帧中我们的值是True的行: + +``` +df[indices] +``` + + |account |Total Sales |Country +:--|:--|:--|:-- +0 |Jones LLC |150 |US +2 |Blue Inc |75 |US +3 |Mega Corp |300 |US + +这里是一个刚刚发生的操作图像: + +![](http://pbpython.com/images/Boolean-Indexing-Example.png) + +手工的列表创建索引是工作的。但是显然对任何一个超过微不足道的数据集不是可扩展或是很有用的。幸运的是,pandas就可以用一个简单的查询语言,有熟悉用Python创建这些布尔指标应该很容易(或任何语言)。 + +例如,让我们看看来自美国的所有销售线。如果我们执行的Python表达式基于Country列: + + +``` +df.Country == 'US' +``` + +``` +0 True +1 False +2 True +3 True +Name: Country, dtype: bool +``` + +这个例子显示了pandas如何将你的Python的传统逻辑,把它应用到一个数据帧并返回一个布尔值列表。那么这个布尔值的列表可以通过帧的数据获取相应的行。 + +在真正的代码中,你不需要做这两个步骤。简洁的做这事的方法典型的看上去如下: + +``` +df[df["Country"] == 'US'] +``` + + |account |Total Sales |Country +:--|:--|:--|:-- +0 |Jones LLC |150| US +2 |Blue Inc |75 |US +3 |Mega Corp| 300| US + +虽然这个概念很简单,你可以编写非常复杂的逻辑,使用Python的威力过滤数据。 + + +>关键点 +>在这个例子中, `df[df.Country == 'US']` 等价于 `df[df["Country"] == 'US']` ‘.’ 标记法是简洁的但是在你列名有空格时不会工作。 + + +### 选择需要的列 + +现在,我们已经找到了如何选择行的数据,我们如何控制哪些列显示?在上面的例子中,没有明显的方法去做。Pandas能使用三种基于位置的索引支持这个用法:.loc , iloc , 和 .ix . 。这些功能也允许我们选择除了我们所看到的行挑选列。 + + + +在这里有很大的困惑关于什么时候选择使用 .loc , iloc , 或者是 .ix。 快速总结的区别是: + +- .loc 用于标签索引 +- .iloc 用于基于整数的位置 +- .ix 是一个将尝试使用标签的快捷方式 (like .loc ) 但失败后将回落到基于位置的整数 (like .iloc ) + +因此问题是我该使用哪个?我必须坦诚我有的时候也会被这些搞混.我发现我最常使用 .loc 。主要是因为我的数据不适合于有意义的基于位置的索引(换句话说,我很少发现自己需要使用 .iloc ) 所以我坚持使用 .loc。 + +公平讲,每一个方法都有自己的位置,在许多情况下都是有用的。特别的一个场景是当处理多指标数据帧时。我不会在这篇文章中讨论那个话题-也许在未来的博文会说。 + +现在我们已经涵盖了这样一个话题,让我们来展示如何筛选一行的值的数据帧,具体选择要显示的列。 + +继续我们的例子,如果我们只是想显示对应于我们的指数的帐户名称,该怎么做?使用.loc很简单: + + +``` +df.loc[[True, True, False, True], "account"] +``` + +``` +1 Alpha Co +2 Blue Inc +3 Mega Corp +Name: account, dtype: object +``` + +如果你想看到多个列,只需通过一个列表: + +``` +df.loc[[True, True, False, True], ["account", "Country"]] +``` + + | account |Country +:--|:--|:-- +0 |Jones LLC| US +1 |Alpha Co |UK +3 |Mega Corp |US + +真正的威力是当你在你的数据上创建更复杂的查询。在这种情况下,让我们展示所有的帐户名称和销售> 200的国家: + +``` +df.loc[df["Total Sales"] > 200, ["account", "Country"]] +``` + +|account| Country + :--|:--|:-- +3 |Mega Corp| US + +这个过程可以被认为有点相当于我们上面讨论过的Excel的过滤器。你已经加入了额外的好处,你也可以限制你检索的列数,而不仅仅是行。 + + +### 编辑列 + +所有这些都是好的背景,但当你使用一个类似的方法来更新一个或多个列的基础上的行选择这个过程真的是闪光的。 + + +一个简单的例子,让我们增加一个佣金率列到我们的数据: + +``` +df["rate"] = 0.02 +``` + + | account |Total Sales| Country |rate +:--|:--|:--|:--|:-- +0 |Jones LLC |150 |US |0.02 +1 |Alpha Co |200 |UK |0.02 +2 |Blue Inc |75 |US |0.02 +3 |Mega Corp |300 |US |0.02 + +让我们说,如果你卖了超过100,你的利率是5%。基本的过程是设置一个布尔索引来选择列,然后将该值赋给利率列: + + +``` +df.loc[df["Total Sales"] > 100, ["rate"]] = .05 +``` + + | account |Total Sales| Country| rate +:--|:--|:--|:--|:-- +0 |Jones LLC |150| US| 0.05 +1 |Alpha Co |200 |UK |0.05 +2| Blue Inc |75| US |0.02 +3 |Mega Corp |300| US| 0.05 + +希望如果你看过这篇文章,这将是有意义的,它会帮助你理解这个语法是如何工作的。现在你有了“过滤和编辑”方法的基本原理。最后一节将更详细的在Excel和pandas显示这个过程。 + + +### 将这些合并在一起 + +对于最后的例子,我们将创建一个简单的佣金计算器,使用以下规则: + +- 所有的佣金计算在交易水平 +- 所有销售的基础佣金为2% +- 所有衬衫都将获得2.5%的佣金 +- 一个特殊的程序正在进行中,在一个交易中销售的数额>10带得到4%的佣金 +- 所有在一个单一的交易鞋类销售> 1000美元有一个特别的奖金为250美元加上一个4.5%的的佣金 + +为了在Excel做到这一点,使用的过滤器和编辑方法: + +- 添加一个2%的佣金列 +- 添加一个0美元的奖金列 +- 衬衫上的过滤器,并将vale改为2.5% +- 清楚过滤器 +- 带和数量的过滤器>10和将值更改为4% +- 清除过滤器 +- 鞋类过滤器> 1000美元,并增加佣金和奖金价值分别为4.5%和250美元 + + +我不会显示每一步的屏幕快照,但这里是最后一个过滤器的屏幕快照: + +![](http://pbpython.com/images/filter-2.png) + +这种方法很简单,可以在很好的操作,但它不具备很好的可重复性,也不能审核。当然还有其他的方法,如在Excel公式或VBA实现这些。然而,这种过滤器和编辑的方法是常见的,是说明性的pandas的逻辑。 + +现在, 让我们在pandas中运行这个例子. + +首先,读入[Excel 文件][4]同时加入默认值2%的利息一列: +``` +import pandas as pd +df = pd.read_excel("https://github.com/chris1610/pbpython/blob/master/data/sample-sales-reps.xlsx?raw=true") +df["commission"] = .02 +df.head() +``` + + | account number| customer name| sales rep| sku |category |quantity |unit price| ext price| date| commission +:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:-- +0 |680916 |Mueller and Sons |Loring Predovic |GP-14407 | Belt |19 |88.49 |1681.31 |2015-11-17 05:58:34| 0.02 +1 |680916 |Mueller and Sons |Loring Predovic |FI-01804| Shirt |3| 78.07| 234.21 |2016-02-13 04:04:11 |0.02 +2 |530925 |Purdy and Sons| Teagan O’Keefe |EO-54210 |Shirt |19 |30.21 |573.99 |2015-08-11 12:44:38 |0.02 +3 |14406| Harber, Lubowitz and Fahey| Esequiel Schinner| NZ-99565| Shirt| 12| 90.29 |1083.48 |2016-01-23 02:15:50 |0.02 +4 |398620| Brekke Ltd |Esequiel Schinner |NZ-99565 |Shirt |5| 72.64 |363.20 |2015-08-10 07:16:03 |0.02 + +下一个规则是所有衬衫获得2.5%和带销售>10得到一个4%的利息: + +``` +df.loc[df["category"] == "Shirt", ["commission"]] = .025 +df.loc[(df["category"] == "Belt") & (df["quantity"] >= 10), ["commission"]] = .04 +df.head() +``` + +| account number |customer name |sales rep| sku |category |quantity| unit price| ext price |date |commission + :--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:-- +0 |680916| Mueller and Sons| Loring Predovic| GP-14407| Belt| 19 |88.49| 1681.31 |2015-11-17 05:58:34| 0.040 +1 |680916| Mueller and Sons| Loring Predovic| FI-01804| Shirt |3 |78.07 |234.21| 2016-02-13 04:04:11 |0.025 +2 |530925 |Purdy and Sons |Teagan O’Keefe| EO-54210 |Shirt| 19 |30.21 |573.99 |2015-08-11 12:44:38| 0.025 +3 |14406| Harber, Lubowitz and Fahey| Esequiel Schinner| NZ-99565| Shirt| 12| 90.29| 1083.48| 2016-01-23 02:15:50 |0.025 +4 |398620| Brekke Ltd| Esequiel Schinner| NZ-99565| Shirt| 5 |72.64 |363.20| 2015-08-10 07:16:03 |0.025 + +最后的佣金规则是加上特别的奖金: + +``` +df["bonus"] = 0 +df.loc[(df["category"] == "Shoes") & (df["ext price"] >= 1000 ), ["bonus", "commission"]] = 250, 0.045 + +# Display a sample of rows that show this bonus +df.ix[3:7] +``` + +| account number| customer name |sales rep| sku |category| quantity| unit price| ext price| date| commission| bonus + :--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:-- +3| 14406| Harber, Lubowitz and Fahey| Esequiel Schinner| NZ-99565| Shirt |12 |90.29| 1083.48 |2016-01-23 02:15:50 |0.025 |0 +4| 398620| Brekke Ltd| Esequiel Schinner |NZ-99565| Shirt| 5| 72.64| 363.20| 2015-08-10 07:16:03| 0.025| 0 +5| 282122| Connelly, Abshire and Von Beth| Skiles| GJ-90272| Shoes| 20| 96.62| 1932.40 |2016-03-17 10:19:05 |0.045 |250 +6 |398620 |Brekke Ltd| Esequiel Schinner |DU-87462 |Shirt| 10| 67.64 |676.40| 2015-11-25 22:05:36| 0.025| 0 +7| 218667| Jaskolski-O’Hara| Trish Deckow| DU-87462| Shirt |11| 91.86| 1010.46 |2016-04-24 15:05:58| 0.025| 0 + +为了做好佣金的计算: + +``` +# Calculate the compensation for each row +df["comp"] = df["commission"] * df["ext price"] + df["bonus"] + +# Summarize and round the results by sales rep +df.groupby(["sales rep"])["comp"].sum().round(2) +``` + +``` +sales rep +Ansley Cummings 2169.76 +Beth Skiles 3028.60 +Esequiel Schinner 10451.21 +Loring Predovic 10108.60 +Shannen Hudson 5275.66 +Teagan O'Keefe 7989.52 +Trish Deckow 5807.74 +Name: comp, dtype: float64 +``` + + +如果你有兴趣,一个例子手册被托管在[github][5]. + +### 总结 + + +感谢阅读文章。 我发现,在学习如何使用pandas的新用户的最大的挑战之一是如何找出如何使用他们的基于Excel的知识,以建立一个等效的pandas为基础的解决方案。在许多情况下,andas的解决方案将是更健壮,更快,更容易审计和更强大的。然而,学习曲线可以花一些时间。我希望这个例子显示了如何解决一个问题,使用的是电子表格的过滤工具将是一个有用的指南,为那些刚刚在这个pandas开始的旅程的。祝你好运! + +-------------------------------------------------------------------------------- + +via: http://pbpython.com/excel-filter-edit.html + +作者:[Chris Moffitt ][a] +译者:[zky001](https://github.com/zky001) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://pbpython.com/author/chris-moffitt.html +[1]: http://pbpython.com/excel-pandas-comp.html +[2]: http://pbpython.com/excel-pandas-comp-2.html +[3]: http://pandas.pydata.org/pandas-docs/stable/indexing.html +[4]: https://github.com/chris1610/pbpython/blob/master/data/sample-sales-reps.xlsx?raw=true +[5]: https://github.com/chris1610/pbpython/blob/master/notebooks/Commissions-Example.ipynb From 8b8fda4520d0ad5a844531d800b1f1db0de8569d Mon Sep 17 00:00:00 2001 From: wxy Date: Sat, 3 Sep 2016 00:11:15 +0800 Subject: [PATCH 0026/2437] =?UTF-8?q?=E5=BD=92=E6=A1=A3=20201608?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...nux Administration Tasks and Application Deployments Over SSH.md | 0 .../{ => 201608}/20151208 6 creative ways to use ownCloud.md | 0 published/{ => 201608}/20151220 GCC-Inline-Assembly-HOWTO.md | 0 .../20160306 5 Favorite Open Source Django Packages.md | 0 .../{ => 201608}/20160309 Let’s Build A Web Server. Part 1.md | 0 .../{ => 201608}/20160316 Growing a career alongside Linux.md | 0 .../{ => 201608}/20160406 Let’s Build A Web Server. Part 2.md | 0 ...6 A newcomer's guide to navigating OpenStack Infrastructure.md | 0 ...20160502 The intersection of Drupal, IoT, and open hardware.md | 0 ...Torvalds Talks IoT Smart Devices Security Concerns and More.md | 0 published/{ => 201608}/20160512 Bitmap in Linux Kernel.md | 0 .../{ => 201608}/20160516 Scaling Collaboration in DevOps.md | 0 .../{ => 201608}/20160518 Python 3 - An Intro to Encryption.md | 0 ...ted with Python Programming and Scripting in Linux – Part 1.md | 0 published/{ => 201608}/20160531 The Anatomy of a Linux User.md | 0 ...rol Flow and Loops to Write and Tune Shell Scripts – Part 2.md | 0 ...d and deploy a Facebook Messenger bot with Python and Flask.md | 0 ...ts Memory Consumption Per-Process and Per-User Basis in Linux.md | 0 ... Mandatory Access Control with SELinux or AppArmor in Linux.md | 0 ...ap takes charge of Linux desktop and IoT software distribution.md | 0 ...ation of “Everything is a File” and Types of Files in Linux.md | 0 .../{ => 201608}/20160618 An Introduction to Mocking in Python.md | 0 published/{ => 201608}/20160620 5 SSH Hardening Tips.md | 0 .../20160621 Flatpak brings standalone apps to Linux.md | 0 .../20160623 72% Of The People I Follow On Twitter Are Men.md | 0 .../20160623 Advanced Image Processing with Python.md | 0 published/{ => 201608}/20160628 Python 101 An Intro to urllib.md | 0 ... IDEs for C+C++ Programming or Source Code Editors on Linux.md | 0 .../20160705 How to Encrypt a Flash Drive Using VeraCrypt.md | 0 .../20160706 Doing for User Space What We Did for Kernel Space.md | 0 ...ocker Swarm and deploying a replicated Python 3 Application.md | 0 published/{ => 201608}/20160706 What is Git.md | 0 published/{ => 201608}/20160711 Getting started with Git.md | 0 published/{ => 201608}/20160715 bc - Command line calculator.md | 0 .../20160717 BEST TEXT EDITORS FOR LINUX COMMAND LINE.md | 0 .../{ => 201608}/20160718 Creating your first Git repository.md | 0 .../20160721 5 tricks for getting started with Vim.md | 0 .../{ => 201608}/20160722 Keeweb A Linux Password Manager.md | 0 .../20160726 How to restore older file versions in Git.md | 0 ...ctions-to-speed-up-apt-get on Ubuntu Linux 16.04 LTS server.md | 0 .../20160801 Linux how long a process has been running.md | 0 published/{ => 201608}/20160802 3 graphical tools for Git.md | 0 .../{ => 201608}/20160808 How to cache static files on nginx.md | 0 published/{ => 201608}/20160809 7 reasons to love Vim.md | 0 ... A brief introduction to Linux containers and image signing.md | 0 45 files changed, 0 insertions(+), 0 deletions(-) rename published/{ => 201608}/20151118 Fabric – Automate Your Linux Administration Tasks and Application Deployments Over SSH.md (100%) rename published/{ => 201608}/20151208 6 creative ways to use ownCloud.md (100%) rename published/{ => 201608}/20151220 GCC-Inline-Assembly-HOWTO.md (100%) rename published/{ => 201608}/20160306 5 Favorite Open Source Django Packages.md (100%) rename published/{ => 201608}/20160309 Let’s Build A Web Server. Part 1.md (100%) rename published/{ => 201608}/20160316 Growing a career alongside Linux.md (100%) rename published/{ => 201608}/20160406 Let’s Build A Web Server. Part 2.md (100%) rename published/{ => 201608}/20160416 A newcomer's guide to navigating OpenStack Infrastructure.md (100%) rename published/{ => 201608}/20160502 The intersection of Drupal, IoT, and open hardware.md (100%) rename published/{ => 201608}/20160506 Linus Torvalds Talks IoT Smart Devices Security Concerns and More.md (100%) rename published/{ => 201608}/20160512 Bitmap in Linux Kernel.md (100%) rename published/{ => 201608}/20160516 Scaling Collaboration in DevOps.md (100%) rename published/{ => 201608}/20160518 Python 3 - An Intro to Encryption.md (100%) rename published/{ => 201608}/20160525 Getting Started with Python Programming and Scripting in Linux – Part 1.md (100%) rename published/{ => 201608}/20160531 The Anatomy of a Linux User.md (100%) rename published/{ => 201608}/20160601 Learn Python Control Flow and Loops to Write and Tune Shell Scripts – Part 2.md (100%) rename published/{ => 201608}/20160602 How to build and deploy a Facebook Messenger bot with Python and Flask.md (100%) rename published/{ => 201608}/20160604 Smem – Reports Memory Consumption Per-Process and Per-User Basis in Linux.md (100%) rename published/{ => 201608}/20160608 Implementing Mandatory Access Control with SELinux or AppArmor in Linux.md (100%) rename published/{ => 201608}/20160614 ​Ubuntu Snap takes charge of Linux desktop and IoT software distribution.md (100%) rename published/{ => 201608}/20160615 Explanation of “Everything is a File” and Types of Files in Linux.md (100%) rename published/{ => 201608}/20160618 An Introduction to Mocking in Python.md (100%) rename published/{ => 201608}/20160620 5 SSH Hardening Tips.md (100%) rename published/{ => 201608}/20160621 Flatpak brings standalone apps to Linux.md (100%) rename published/{ => 201608}/20160623 72% Of The People I Follow On Twitter Are Men.md (100%) rename published/{ => 201608}/20160623 Advanced Image Processing with Python.md (100%) rename published/{ => 201608}/20160628 Python 101 An Intro to urllib.md (100%) rename published/{ => 201608}/20160630 18 Best IDEs for C+C++ Programming or Source Code Editors on Linux.md (100%) rename published/{ => 201608}/20160705 How to Encrypt a Flash Drive Using VeraCrypt.md (100%) rename published/{ => 201608}/20160706 Doing for User Space What We Did for Kernel Space.md (100%) rename published/{ => 201608}/20160706 Getting started with Docker Swarm and deploying a replicated Python 3 Application.md (100%) rename published/{ => 201608}/20160706 What is Git.md (100%) rename published/{ => 201608}/20160711 Getting started with Git.md (100%) rename published/{ => 201608}/20160715 bc - Command line calculator.md (100%) rename published/{ => 201608}/20160717 BEST TEXT EDITORS FOR LINUX COMMAND LINE.md (100%) rename published/{ => 201608}/20160718 Creating your first Git repository.md (100%) rename published/{ => 201608}/20160721 5 tricks for getting started with Vim.md (100%) rename published/{ => 201608}/20160722 Keeweb A Linux Password Manager.md (100%) rename published/{ => 201608}/20160726 How to restore older file versions in Git.md (100%) rename published/{ => 201608}/20160727-How-to-use-multiple-connections-to-speed-up-apt-get on Ubuntu Linux 16.04 LTS server.md (100%) rename published/{ => 201608}/20160801 Linux how long a process has been running.md (100%) rename published/{ => 201608}/20160802 3 graphical tools for Git.md (100%) rename published/{ => 201608}/20160808 How to cache static files on nginx.md (100%) rename published/{ => 201608}/20160809 7 reasons to love Vim.md (100%) rename published/{ => 201608}/20160810 A brief introduction to Linux containers and image signing.md (100%) diff --git a/published/20151118 Fabric – Automate Your Linux Administration Tasks and Application Deployments Over SSH.md b/published/201608/20151118 Fabric – Automate Your Linux Administration Tasks and Application Deployments Over SSH.md similarity index 100% rename from published/20151118 Fabric – Automate Your Linux Administration Tasks and Application Deployments Over SSH.md rename to published/201608/20151118 Fabric – Automate Your Linux Administration Tasks and Application Deployments Over SSH.md diff --git a/published/20151208 6 creative ways to use ownCloud.md b/published/201608/20151208 6 creative ways to use ownCloud.md similarity index 100% rename from published/20151208 6 creative ways to use ownCloud.md rename to published/201608/20151208 6 creative ways to use ownCloud.md diff --git a/published/20151220 GCC-Inline-Assembly-HOWTO.md b/published/201608/20151220 GCC-Inline-Assembly-HOWTO.md similarity index 100% rename from published/20151220 GCC-Inline-Assembly-HOWTO.md rename to published/201608/20151220 GCC-Inline-Assembly-HOWTO.md diff --git a/published/20160306 5 Favorite Open Source Django Packages.md b/published/201608/20160306 5 Favorite Open Source Django Packages.md similarity index 100% rename from published/20160306 5 Favorite Open Source Django Packages.md rename to published/201608/20160306 5 Favorite Open Source Django Packages.md diff --git a/published/20160309 Let’s Build A Web Server. Part 1.md b/published/201608/20160309 Let’s Build A Web Server. Part 1.md similarity index 100% rename from published/20160309 Let’s Build A Web Server. Part 1.md rename to published/201608/20160309 Let’s Build A Web Server. Part 1.md diff --git a/published/20160316 Growing a career alongside Linux.md b/published/201608/20160316 Growing a career alongside Linux.md similarity index 100% rename from published/20160316 Growing a career alongside Linux.md rename to published/201608/20160316 Growing a career alongside Linux.md diff --git a/published/20160406 Let’s Build A Web Server. Part 2.md b/published/201608/20160406 Let’s Build A Web Server. Part 2.md similarity index 100% rename from published/20160406 Let’s Build A Web Server. Part 2.md rename to published/201608/20160406 Let’s Build A Web Server. Part 2.md diff --git a/published/20160416 A newcomer's guide to navigating OpenStack Infrastructure.md b/published/201608/20160416 A newcomer's guide to navigating OpenStack Infrastructure.md similarity index 100% rename from published/20160416 A newcomer's guide to navigating OpenStack Infrastructure.md rename to published/201608/20160416 A newcomer's guide to navigating OpenStack Infrastructure.md diff --git a/published/20160502 The intersection of Drupal, IoT, and open hardware.md b/published/201608/20160502 The intersection of Drupal, IoT, and open hardware.md similarity index 100% rename from published/20160502 The intersection of Drupal, IoT, and open hardware.md rename to published/201608/20160502 The intersection of Drupal, IoT, and open hardware.md diff --git a/published/20160506 Linus Torvalds Talks IoT Smart Devices Security Concerns and More.md b/published/201608/20160506 Linus Torvalds Talks IoT Smart Devices Security Concerns and More.md similarity index 100% rename from published/20160506 Linus Torvalds Talks IoT Smart Devices Security Concerns and More.md rename to published/201608/20160506 Linus Torvalds Talks IoT Smart Devices Security Concerns and More.md diff --git a/published/20160512 Bitmap in Linux Kernel.md b/published/201608/20160512 Bitmap in Linux Kernel.md similarity index 100% rename from published/20160512 Bitmap in Linux Kernel.md rename to published/201608/20160512 Bitmap in Linux Kernel.md diff --git a/published/20160516 Scaling Collaboration in DevOps.md b/published/201608/20160516 Scaling Collaboration in DevOps.md similarity index 100% rename from published/20160516 Scaling Collaboration in DevOps.md rename to published/201608/20160516 Scaling Collaboration in DevOps.md diff --git a/published/20160518 Python 3 - An Intro to Encryption.md b/published/201608/20160518 Python 3 - An Intro to Encryption.md similarity index 100% rename from published/20160518 Python 3 - An Intro to Encryption.md rename to published/201608/20160518 Python 3 - An Intro to Encryption.md diff --git a/published/20160525 Getting Started with Python Programming and Scripting in Linux – Part 1.md b/published/201608/20160525 Getting Started with Python Programming and Scripting in Linux – Part 1.md similarity index 100% rename from published/20160525 Getting Started with Python Programming and Scripting in Linux – Part 1.md rename to published/201608/20160525 Getting Started with Python Programming and Scripting in Linux – Part 1.md diff --git a/published/20160531 The Anatomy of a Linux User.md b/published/201608/20160531 The Anatomy of a Linux User.md similarity index 100% rename from published/20160531 The Anatomy of a Linux User.md rename to published/201608/20160531 The Anatomy of a Linux User.md diff --git a/published/20160601 Learn Python Control Flow and Loops to Write and Tune Shell Scripts – Part 2.md b/published/201608/20160601 Learn Python Control Flow and Loops to Write and Tune Shell Scripts – Part 2.md similarity index 100% rename from published/20160601 Learn Python Control Flow and Loops to Write and Tune Shell Scripts – Part 2.md rename to published/201608/20160601 Learn Python Control Flow and Loops to Write and Tune Shell Scripts – Part 2.md diff --git a/published/20160602 How to build and deploy a Facebook Messenger bot with Python and Flask.md b/published/201608/20160602 How to build and deploy a Facebook Messenger bot with Python and Flask.md similarity index 100% rename from published/20160602 How to build and deploy a Facebook Messenger bot with Python and Flask.md rename to published/201608/20160602 How to build and deploy a Facebook Messenger bot with Python and Flask.md diff --git a/published/20160604 Smem – Reports Memory Consumption Per-Process and Per-User Basis in Linux.md b/published/201608/20160604 Smem – Reports Memory Consumption Per-Process and Per-User Basis in Linux.md similarity index 100% rename from published/20160604 Smem – Reports Memory Consumption Per-Process and Per-User Basis in Linux.md rename to published/201608/20160604 Smem – Reports Memory Consumption Per-Process and Per-User Basis in Linux.md diff --git a/published/20160608 Implementing Mandatory Access Control with SELinux or AppArmor in Linux.md b/published/201608/20160608 Implementing Mandatory Access Control with SELinux or AppArmor in Linux.md similarity index 100% rename from published/20160608 Implementing Mandatory Access Control with SELinux or AppArmor in Linux.md rename to published/201608/20160608 Implementing Mandatory Access Control with SELinux or AppArmor in Linux.md diff --git a/published/20160614 ​Ubuntu Snap takes charge of Linux desktop and IoT software distribution.md b/published/201608/20160614 ​Ubuntu Snap takes charge of Linux desktop and IoT software distribution.md similarity index 100% rename from published/20160614 ​Ubuntu Snap takes charge of Linux desktop and IoT software distribution.md rename to published/201608/20160614 ​Ubuntu Snap takes charge of Linux desktop and IoT software distribution.md diff --git a/published/20160615 Explanation of “Everything is a File” and Types of Files in Linux.md b/published/201608/20160615 Explanation of “Everything is a File” and Types of Files in Linux.md similarity index 100% rename from published/20160615 Explanation of “Everything is a File” and Types of Files in Linux.md rename to published/201608/20160615 Explanation of “Everything is a File” and Types of Files in Linux.md diff --git a/published/20160618 An Introduction to Mocking in Python.md b/published/201608/20160618 An Introduction to Mocking in Python.md similarity index 100% rename from published/20160618 An Introduction to Mocking in Python.md rename to published/201608/20160618 An Introduction to Mocking in Python.md diff --git a/published/20160620 5 SSH Hardening Tips.md b/published/201608/20160620 5 SSH Hardening Tips.md similarity index 100% rename from published/20160620 5 SSH Hardening Tips.md rename to published/201608/20160620 5 SSH Hardening Tips.md diff --git a/published/20160621 Flatpak brings standalone apps to Linux.md b/published/201608/20160621 Flatpak brings standalone apps to Linux.md similarity index 100% rename from published/20160621 Flatpak brings standalone apps to Linux.md rename to published/201608/20160621 Flatpak brings standalone apps to Linux.md diff --git a/published/20160623 72% Of The People I Follow On Twitter Are Men.md b/published/201608/20160623 72% Of The People I Follow On Twitter Are Men.md similarity index 100% rename from published/20160623 72% Of The People I Follow On Twitter Are Men.md rename to published/201608/20160623 72% Of The People I Follow On Twitter Are Men.md diff --git a/published/20160623 Advanced Image Processing with Python.md b/published/201608/20160623 Advanced Image Processing with Python.md similarity index 100% rename from published/20160623 Advanced Image Processing with Python.md rename to published/201608/20160623 Advanced Image Processing with Python.md diff --git a/published/20160628 Python 101 An Intro to urllib.md b/published/201608/20160628 Python 101 An Intro to urllib.md similarity index 100% rename from published/20160628 Python 101 An Intro to urllib.md rename to published/201608/20160628 Python 101 An Intro to urllib.md diff --git a/published/20160630 18 Best IDEs for C+C++ Programming or Source Code Editors on Linux.md b/published/201608/20160630 18 Best IDEs for C+C++ Programming or Source Code Editors on Linux.md similarity index 100% rename from published/20160630 18 Best IDEs for C+C++ Programming or Source Code Editors on Linux.md rename to published/201608/20160630 18 Best IDEs for C+C++ Programming or Source Code Editors on Linux.md diff --git a/published/20160705 How to Encrypt a Flash Drive Using VeraCrypt.md b/published/201608/20160705 How to Encrypt a Flash Drive Using VeraCrypt.md similarity index 100% rename from published/20160705 How to Encrypt a Flash Drive Using VeraCrypt.md rename to published/201608/20160705 How to Encrypt a Flash Drive Using VeraCrypt.md diff --git a/published/20160706 Doing for User Space What We Did for Kernel Space.md b/published/201608/20160706 Doing for User Space What We Did for Kernel Space.md similarity index 100% rename from published/20160706 Doing for User Space What We Did for Kernel Space.md rename to published/201608/20160706 Doing for User Space What We Did for Kernel Space.md diff --git a/published/20160706 Getting started with Docker Swarm and deploying a replicated Python 3 Application.md b/published/201608/20160706 Getting started with Docker Swarm and deploying a replicated Python 3 Application.md similarity index 100% rename from published/20160706 Getting started with Docker Swarm and deploying a replicated Python 3 Application.md rename to published/201608/20160706 Getting started with Docker Swarm and deploying a replicated Python 3 Application.md diff --git a/published/20160706 What is Git.md b/published/201608/20160706 What is Git.md similarity index 100% rename from published/20160706 What is Git.md rename to published/201608/20160706 What is Git.md diff --git a/published/20160711 Getting started with Git.md b/published/201608/20160711 Getting started with Git.md similarity index 100% rename from published/20160711 Getting started with Git.md rename to published/201608/20160711 Getting started with Git.md diff --git a/published/20160715 bc - Command line calculator.md b/published/201608/20160715 bc - Command line calculator.md similarity index 100% rename from published/20160715 bc - Command line calculator.md rename to published/201608/20160715 bc - Command line calculator.md diff --git a/published/20160717 BEST TEXT EDITORS FOR LINUX COMMAND LINE.md b/published/201608/20160717 BEST TEXT EDITORS FOR LINUX COMMAND LINE.md similarity index 100% rename from published/20160717 BEST TEXT EDITORS FOR LINUX COMMAND LINE.md rename to published/201608/20160717 BEST TEXT EDITORS FOR LINUX COMMAND LINE.md diff --git a/published/20160718 Creating your first Git repository.md b/published/201608/20160718 Creating your first Git repository.md similarity index 100% rename from published/20160718 Creating your first Git repository.md rename to published/201608/20160718 Creating your first Git repository.md diff --git a/published/20160721 5 tricks for getting started with Vim.md b/published/201608/20160721 5 tricks for getting started with Vim.md similarity index 100% rename from published/20160721 5 tricks for getting started with Vim.md rename to published/201608/20160721 5 tricks for getting started with Vim.md diff --git a/published/20160722 Keeweb A Linux Password Manager.md b/published/201608/20160722 Keeweb A Linux Password Manager.md similarity index 100% rename from published/20160722 Keeweb A Linux Password Manager.md rename to published/201608/20160722 Keeweb A Linux Password Manager.md diff --git a/published/20160726 How to restore older file versions in Git.md b/published/201608/20160726 How to restore older file versions in Git.md similarity index 100% rename from published/20160726 How to restore older file versions in Git.md rename to published/201608/20160726 How to restore older file versions in Git.md diff --git a/published/20160727-How-to-use-multiple-connections-to-speed-up-apt-get on Ubuntu Linux 16.04 LTS server.md b/published/201608/20160727-How-to-use-multiple-connections-to-speed-up-apt-get on Ubuntu Linux 16.04 LTS server.md similarity index 100% rename from published/20160727-How-to-use-multiple-connections-to-speed-up-apt-get on Ubuntu Linux 16.04 LTS server.md rename to published/201608/20160727-How-to-use-multiple-connections-to-speed-up-apt-get on Ubuntu Linux 16.04 LTS server.md diff --git a/published/20160801 Linux how long a process has been running.md b/published/201608/20160801 Linux how long a process has been running.md similarity index 100% rename from published/20160801 Linux how long a process has been running.md rename to published/201608/20160801 Linux how long a process has been running.md diff --git a/published/20160802 3 graphical tools for Git.md b/published/201608/20160802 3 graphical tools for Git.md similarity index 100% rename from published/20160802 3 graphical tools for Git.md rename to published/201608/20160802 3 graphical tools for Git.md diff --git a/published/20160808 How to cache static files on nginx.md b/published/201608/20160808 How to cache static files on nginx.md similarity index 100% rename from published/20160808 How to cache static files on nginx.md rename to published/201608/20160808 How to cache static files on nginx.md diff --git a/published/20160809 7 reasons to love Vim.md b/published/201608/20160809 7 reasons to love Vim.md similarity index 100% rename from published/20160809 7 reasons to love Vim.md rename to published/201608/20160809 7 reasons to love Vim.md diff --git a/published/20160810 A brief introduction to Linux containers and image signing.md b/published/201608/20160810 A brief introduction to Linux containers and image signing.md similarity index 100% rename from published/20160810 A brief introduction to Linux containers and image signing.md rename to published/201608/20160810 A brief introduction to Linux containers and image signing.md From 4fe9bde98f5e92831939f17b135ad39ea00284a1 Mon Sep 17 00:00:00 2001 From: wxy Date: Sat, 3 Sep 2016 00:12:09 +0800 Subject: [PATCH 0027/2437] PUB:20160823 What Datatype Should You Use to Represent Time in MySQL MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @bianjp 辛苦了! --- ...ould You Use to Represent Time in MySQL.md | 554 +++++++++++++++++ ...ould You Use to Represent Time in MySQL.md | 563 ------------------ 2 files changed, 554 insertions(+), 563 deletions(-) create mode 100644 published/20160823 What Datatype Should You Use to Represent Time in MySQL.md delete mode 100644 translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md diff --git a/published/20160823 What Datatype Should You Use to Represent Time in MySQL.md b/published/20160823 What Datatype Should You Use to Represent Time in MySQL.md new file mode 100644 index 0000000000..3baf1eab27 --- /dev/null +++ b/published/20160823 What Datatype Should You Use to Represent Time in MySQL.md @@ -0,0 +1,554 @@ +MySQL 中你应该使用什么数据类型表示时间? +========================================================== + +![](http://www.vertabelo.com/_file/blog/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int/clock.jpg) + +_当你需要保存日期时间数据时,一个问题来了:你应该使用 MySQL 中的什么类型?使用 MySQL 原生的 DATE 类型还是使用 INT 字段把日期和时间保存为一个纯数字呢?_ + +在这篇文章中,我将解释 MySQL 原生的方案,并给出一个最常用数据类型的对比表。我们也将对一些典型的查询做基准测试,然后得出在给定场景下应该使用什么数据类型的结论。 + +## 原生的 MySQL Datetime 数据类型 + +Datetime 数据表示一个时间点。这可以用作日志记录、物联网时间戳、日历事件数据,等等。MySQL 有两种原生的类型可以将这种信息保存在单个字段中:Datetime 和 Timestamp。MySQL 文档中是这么介绍这些数据类型的: + +> DATETIME 类型用于保存同时包含日期和时间两部分的值。MySQL 以 'YYYY-MM-DD HH:MM:SS' 形式接收和显示 DATETIME 类型的值。 + +> TIMESTAMP 类型用于保存同时包含日期和时间两部分的值。 + +> DATETIME 或 TIMESTAMP 类型的值可以在尾部包含一个毫秒部分,精确度最高到微秒(6 位数)。 + +> TIMESTAMP 和 DATETIME 数据类型提供自动初始化和更新到当前的日期和时间的功能,只需在列的定义中设置 DEFAULT CURRENT_TIMESTAMP 和 ON UPDATE CURRENT_TIMESTAMP。 + +作为一个例子: + +```SQL +CREATE TABLE `datetime_example` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `measured_on` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + PRIMARY KEY (`id`), + KEY `measured_on` (`measured_on`) +) ENGINE=InnoDB; +``` + +```SQL +CREATE TABLE `timestamp_example` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `measured_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + PRIMARY KEY (`id`), + KEY `measured_on` (`measured_on`) +) ENGINE=InnoDB; +``` + +除了原生的日期时间表示方法,还有另一种常用的存储日期和时间信息的方法。即使用 INT 字段保存 Unix 时间(从1970 年 1 月 1 日协调世界时(UTC)建立所经过的秒数)。 + +MySQL 也提供了只保存时间信息中的一部分的方式,通过使用 Date、Year 或 Time 类型。由于这篇文章是关于保存准确时间点的最佳方式的,我们没有讨论这些不那么精确的局部类型。 + +## 使用 INT 类型保存 Unix 时间 + +使用一个简单的 INT 列保存 Unix 时间是最普通的方法。使用 INT,你可以确保你要保存的数字可以快速、可靠地插入到表中,就像这样: + +```SQL +INSERT INTO `vertabelo`.`sampletable` +( + `id`, + `measured_on` ### INT 类型的列 +) +VALUES +( + 1, + 946684801 + ### 至 01/01/2000 @ 12:00am (UTC) 的 UNIX 时间戳 http://unixtimestamp.com +); +``` + +这就是关于它的所有内容了。它仅仅是个简单的 INT 列,MySQL 的处理方式是这样的:在内部使用 4 个字节保存那些数据。所以如果你在这个列上使用 SELECT 你将会得到一个数字。如果你想把这个列用作日期进行比较,下面的查询并不能正确工作: + +```SQL +SELECT + id, measured_on, FROM_UNIXTIME(measured_on) +FROM + vertabelo.inttimestampmeasures +WHERE + measured_on > '2016-01-01' ### measured_on 会被作为字符串比较以进行查询 +LIMIT 5; +``` + +这是因为 MySQL 把 INT 视为数字,而非日期。为了进行日期比较,你必须要么获取( LCTT 译注:从 1970-01-01 00:00:00)到 2016-01-01 经过的秒数,要么使用 MySQL 的 FROM\_UNIXTIME() 函数把 INT 列转为 Date 类型。下面的查询展示了 FROM\_UNIXTIME() 函数的用法: + +```SQL +SELECT + id, measured_on, FROM_UNIXTIME(measured_on) +FROM + vertabelo.inttimestampmeasures +WHERE + FROM_UNIXTIME(measured_on) > '2016-01-01' +LIMIT 5; +``` + +这会正确地获取到日期在 2016-01-01 之后的记录。你也可以直接比较数字和 2016-01-01 的 Unix 时间戳表示形式,即 1451606400。这样做意味着不用使用任何特殊的函数,因为你是在直接比较数字。查询如下: + +```SQL +SELECT + id, measured_on, FROM_UNIXTIME(measured_on) +FROM + vertabelo.inttimestampmeasures +WHERE + measured_on > 1451606400 +LIMIT 5; +``` + +假如这种方式不够高效甚至提前做这种转换是不可行的话,那该怎么办?例如,你想获取 2016 年所有星期三的记录。要做到这样而不使用任何 MySQL 日期函数,你就不得不查出 2016 年每个星期三的开始和结束时间的 Unix 时间戳。然后你不得不写很大的查询,至少要在 WHERE 中包含 104 个比较。(2016 年有 52 个星期三,你不得不考虑一天的开始(0:00 am)和结束(11:59:59 pm)...) + +结果是你很可能最终会使用 FROM\_UNIXTIME() 转换函数。既然如此,为什么不试下真正的日期类型呢? + +## 使用 Datetime 和 Timestamp + +Datetime 和 Timestamp 几乎以同样的方式工作。两种都保存日期和时间信息,毫秒部分最高精确度都是 6 位数。同时,使用人类可读的日期形式如 "2016-01-01" (为了便于比较)都能工作。查询时两种类型都支持“宽松格式”。宽松的语法允许任何标点符号作为分隔符。例如,"YYYY-MM-DD HH:MM:SS" 和 "YY-MM-DD HH:MM:SS" 两种形式都可以。在宽松格式情况下以下任何一种形式都能工作: + +``` +2012-12-31 11:30:45 +2012^12^31 11+30+45 +2012/12/31 11*30*45 +2012@12@31 11^30^45 +``` + +其它宽松格式也是允许的;你可以在 [MySQL 参考手册](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-literals.html) 找到所有的格式。 + +默认情况下,Datetime 和 Timestamp 两种类型查询结果都以标准输出格式显示 —— 年-月-日 时:分:秒 (如 2016-01-01 23:59:59)。如果使用了毫秒部分,它们应该以小数值出现在秒后面 (如 2016-01-01 23:59:59.5)。 + +Timestamp 和 Datetime 的核心不同点主要在于 MySQL 在内部如何表示这些信息:两种都以二进制而非字符串形式存储,但在表示日期/时间部分时 Timestamp (4 字节) 比 Datetime (5 字节) 少使用 1 字节。当保存毫秒部分时两种都使用额外的空间 (1-3 字节)。如果你存储 150 万条记录,这种 1 字节的差异是微不足道的: + +> 150 万条记录 * 每条记录 1 字节 / (1048576 字节/MB) = __1.43 MB__ + +Timestamp 节省的 1 字节是有代价的:你只能存储从 '1970-01-01 00:00:01.000000' 到 '2038-01-19 03:14:07.999999' 之间的时间。而 Datetime 允许你存储从 '1000-01-01 00:00:00.000000' 到 '9999-12-31 23:59:59.999999' 之间的任何时间。 + +另一个重要的差别 —— 很多 MySQL 开发者没意识到的 —— 是 MySQL 使用__服务器的时区__转换 Timestamp 值到它的 UTC 等价值再保存。当获取值是它会再次进行时区转换,所以你得回了你“原始的”日期/时间值。有可能,下面这些情况会发生。 + +理想情况下,如果你一直使用同一个时区,MySQL 会获取到和你存储的同样的值。以我的经验,如果你的数据库涉及时区变换,你可能会遇到问题。例如,服务器变化(比如,你把数据库从都柏林的一台服务器迁移到加利福尼亚的一台服务器上,或者你只是修改了一下服务器的时区)时可能会发生这种情况。不管哪种方式,如果你获取数据时的时区是不同的,数据就会受影响。 + +Datetime 列不会被数据库改变。无论时区怎样配置,每次都会保存和获取到同样的值。就我而言,我认为这是一个更可靠的选择。 + +> __MySQL 文档:__ + +> MySQL 把 TIMESTAMP 值从当前的时区转换到 UTC 再存储,获取时再从 UTC 转回当前的时区。(其它类型如 DATETIME 不会这样,它们会“原样”保存。) 默认情况下,每个连接的当前时区都是服务器的时区。时区可以基于连接设置。只要时区设置保持一致,你就能得到和保存的相同的值。如果你保存了一个 TIMESTAMP 值,然后改变了时区再获取这个值,获取到的值和你存储的是不同的。这是因为在写入和查询的会话上没有使用同一个时区。当前时区可以通过系统变量 [time_zone](https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_time_zone) 的值得到。更多信息,请查看 [MySQL Server Time Zone Support](https://dev.mysql.com/doc/refman/5.7/en/time-zone-support.html)。 + +## 对比总结 + +在深入探讨使用各数据类型的性能差异之前,让我们先看一个总结表格以给你更多了解。每种类型的弱点以红色显示。 + +特性 | Datetime | Timestamp | Int (保存 Unix 时间) +:--|:--|:--|:-- +原生时间表示 | 是 | 是 | 否,所以大多数操作需要先使用转换函数,如 FROM_UNIXTIME() +能保存毫秒 | 是,最高 6 位精度 | 是,最高 6 位精度 | **否** +合法范围 | '1000-01-01 00:00:00.000000' 到 '9999-12-31 23:59:59.999999 | **'1970-01-01 00:00:01.000000' 到 '2038-01-19 03:14:07.999999'** | 若使用 unsigned, '1970-01-01 00:00:01.000000; 理论上最大到 '2106-2-07 06:28:15' +自动初始化(MySQL 5.6.5+) | 是 | 是 | **否** +宽松解释 ([MySQL docs](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-literals.html)) | 是 | 是 | **否,必须使用正确的格式** +值被转换到 UTC 存储 | 否 | **是** | 否 +可转换到其它类型 | 是,如果值在合法的 Timestamp 范围中 | 是,总是 | 是,如果值在合法的范围中并使用转换函数 +存储需求([MySQL 5.6.4+](https://dev.mysql.com/doc/refman/5.7/en/storage-requirements.html)) | **5 字节**(如果使用了毫秒部分,再加最多 3 字节) | 4 字节 (如果使用了毫秒部分,再加最多 3 字节) | 4 字节 (不允许毫秒部分) +无需使用函数即可作为真实日期可读 | 是 | 是 | **否,你必须格式化输出** +数据分区 | 是 | 是,使用 [UNIX_TIMESTAMP()](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html#function_unix-timestamp);在 MySQL 5.7 中其它表达式是不允许包含 [TIMESTAMP](https://dev.mysql.com/doc/refman/5.7/en/datetime.html) 值的。同时,注意[分区裁剪时的这些考虑](https://dev.mysql.com/doc/refman/5.7/en/partitioning-pruning.html) | 是,使用 INT 上的任何合法操作 + +## 基准测试 INT、Timestamp 和 Datetime 的性能 + +为了比较这些类型的性能,我会使用我创建的一个天气预报网络的 150 万记录(准确说是 1,497,421)。这个网络每分钟都收集数据。为了让这些测试可复现,我已经删除了一些私有列,所以你可以使用这些数据运行你自己的测试。 + +基于我原始的表格,我创建了三个版本: + +- `datetimemeasures` 表在 `measured_on` 列使用 Datetime 类型,表示天气预报记录的测量时间 +- `timestampmeasures` 表在 `measured_on` 列使用 Timestamp 类型 +- `inttimestampmeasures` 表在 `measured_on` 列使用 INT (unsigned) 类型 + +这三个表拥有完全相同的数据;唯一的差别就是 `measured_on` 字段的类型。所有表都在 `measured_on` 列上设置了一个索引。 + + + +### 基准测试工具 + +为了评估这些数据类型的性能,我使用了两种方法。一种基于 [Sysbench](https://github.com/akopytov/sysbench),它的官网是这么描述的: + +_“... 一个模块化、跨平台和多线程的基准测试工具,用以评估那些对运行高负载数据库的系统非常重要的系统参数。”_ + +这个工具是 [MySQL 文档](https://dev.mysql.com/downloads/benchmarks.html)中推荐的。 + +如果你使用 Windows (就像我),你可以下载一个包含可执行文件和我使用的测试查询的 zip 文件。它们基于 [一种推荐的基准测试方法](https://dba.stackexchange.com/questions/39221/stress-test-mysql-with-queries-captured-with-general-log-in-mysql)。 + +为了执行一个给定的测试,你可以使用下面的命令(插入你自己的连接参数): + +```bash +sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=sysbench_test_file.lua --num-threads=8 --max-requests=100 run +``` + +这会正常工作,这里 `sysbench_test_file.lua` 是测试文件,并包含了各个测试中指向各个表的 SQL 查询。 + +为了进一步验证结果,我也运行了 [mysqlslap](https://dev.mysql.com/doc/refman/5.7/en/mysqlslap.html)。它的官网是这么描述的: + +_“[mysqlslap](https://dev.mysql.com/doc/refman/5.7/en/mysqlslap.html) 是一个诊断程序,为模拟 MySQL 服务器的客户端负载并报告各个阶段的用时而设计。它工作起来就像是很多客户端在同时访问服务器。_ + +记得这些测试中最重要的不是所需的_绝对_时间。而是在不同数据类型上执行相同查询时的_相对_时间。这两个基准测试工具的测试时间不一定相同,因为不同工具的工作方式不同。重要的是数据类型的比较,随着我们深入到测试中,这将会变得清楚。 + +### 基准测试 + +我将使用三种可以评估几个性能方面的查询: + +* 时间范围选择 + * 在 Datetime 和 Timestamp 数据类型上这允许我们直接比较而不需要使用任何特殊的日期函数。 + * 同时,我们可以评估在 INT 类型的列上使用日期函数相对于使用简单的数值比较的影响。为了做到这些我们需要把范围转换为 Unix 时间戳数值。 +* 日期函数选择 + * 与前个测试中比较操作针对一个简单的 DATE 值相反,这个测试使得我们可以评估使用日期函数作为 “WHERE” 子句的一部分的性能。 + * 我们还可以测试一个场景,即我们必须使用一个函数将 INT 列转换为一个合法的 DATE 类型然后执行查询。 +* count() 查询 + * 作为对前面测试的补充,这将评估在三种不同的表示类型上进行典型的统计查询的性能。 + +我们将在这些测试中覆盖一些常见的场景,并看到三种类型上的性能表现。 + +### 关于 SQL\_NO\_CACHE + +当在查询中使用 SQL\_NO\_CACHE 时,服务器不使用查询缓存。它既不检查查询缓存以确认结果是不是已经在那儿了,也不会保存查询结果。因此,每个查询将反映真实的性能影响,就像每次查询都是第一次被调用。 + +### 测试 1:选择一个日期范围中的值 + +这个查询返回总计 1,497,421 行记录中的 75,706 行。 + +#### 查询 1 和 Datetime: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.datetimemeasures m +WHERE + m.measured_on > '2016-01-01 00:00:00.0' + AND m.measured_on < '2016-02-01 00:00:00.0'; +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 152 | 296 | +| 最大 | 1261 | 3203 | +| 平均 | 362 | 809 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.datetimemeasures m WHERE m.measured_on > '2016-01-01 00:00:00.0' AND m.measured_on < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +#### 查询 1 和 Timestamp: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.timestampmeasures m +WHERE + m.measured_on > '2016-01-01 00:00:00.0' + AND m.measured_on < '2016-02-01 00:00:00.0'; +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 214 | 359 | +| 最大 | 1389 | 3313 | +| 平均 | 431 | 1004 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.timestampmeasures m WHERE m.measured_on > '2016-01-01 00:00:00.0' AND m.measured_on < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +#### 查询 1 和 INT: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.inttimestampmeasures m +WHERE + FROM_UNIXTIME(m.measured_on) > '2016-01-01 00:00:00.0' + AND FROM_UNIXTIME(m.measured_on) < '2016-02-01 00:00:00.0'; +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2472 | 7968 | +| 最大 | 6554 | 10312 | +| 平均 | 4107 | 8527 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE FROM_UNIXTIME(m.measured_on) > '2016-01-01 00:00:00.0' AND FROM_UNIXTIME(m.measured_on) < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +#### 另一种 INT 上的查询 1: + +由于这是个相当直接的范围搜索,而且查询中的日期可以轻易地转为简单的数值比较,我将它包含在了这个测试中。结果证明这是最快的方法 (你大概已经预料到了),因为它仅仅是比较数字而没有使用任何日期转换函数: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.inttimestampmeasures m +WHERE + m.measured_on > 1451617200 + AND m.measured_on < 1454295600; +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 88 | 171 | +| 最大 | 275 | 2157 | +| 平均 | 165 | 514 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=basic_int.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE m.measured_on > 1451617200 AND m.measured_on < 1454295600" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo +``` + +#### 测试 1 总结 + +| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | +|-----------------|----------|----------------------|-----------|----------------------| +| Datetime | 362 | - | 809 | - | +| Timestamp | 431 | 慢 19% | 1004 | 慢 24% | +| INT | 4107 | 慢 1134% | 8527 | 慢 1054% | +| 另一种 INT 查询 | 165 | 快 55% | 514 | 快 36% | + +两种基准测试工具都显示 Datetime 比 Timestamp 和 INT 更快。但 Datetime 没有我们在另一种 INT 查询中使用的简单数值比较快。 + +### 测试 2:选择星期一产生的记录 + +这个查询返回总计 1,497,421 行记录中的 221,850 行。 + +#### 查询 2 和 Datetime: + +```SQL +SELECT SQL_NO_CACHE measured_on +FROM + vertabelo.datetimemeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 1874 | 4343 | +| 最大 | 6168 | 7797 | +| 平均 | 3127 | 6103 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime_1.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.datetimemeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +#### 查询 2 和 Timestamp: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.timestampmeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2688 | 5953 | +| 最大 | 6666 | 13531 | +| 平均 | 3653 | 8412 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp_1.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.timestampmeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +#### 查询 2 和 INT: + +```SQL +SELECT SQL_NO_CACHE + measured_on +FROM + vertabelo.inttimestampmeasures m +WHERE + WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0; # MONDAY +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2051 | 5844 | +| 最大 | 7007 | 10469 | +| 平均 | 3486 | 8088 | + + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int_1.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +#### 测试 2 总结 + +| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | +|-----------------|----------|----------------------|-----------|----------------------| +| Datetime | 3127 | - | 6103 | - | +| Timestamp | 3653 | 慢 17% | 8412 | 慢 38% | +| INT | 3486 | 慢 11% | 8088 | 慢 32% | + +再次,在两个基准测试工具中 Datetime 比 Timestamp 和 INT 快。但在这个测试中,INT 查询 —— 即使它使用了一个函数以转换日期 —— 比 Timestamp 查询更快得到结果。 + +### 测试 3:选择星期一产生的记录总数 + +这个查询返回一行,包含产生于星期一的所有记录的总数(从总共 1,497,421 行可用记录中)。 + +#### 查询 3 和 Datetime: + +```SQL +SELECT SQL_NO_CACHE + COUNT(measured_on) +FROM + vertabelo.datetimemeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 1720 | 4063 | +| 最大 | 4594 | 7812 | +| 平均 | 2797 | 5540 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime_1_count.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.datetimemeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +#### 查询 3 和 Timestamp: + +```SQL +SELECT SQL_NO_CACHE + COUNT(measured_on) +FROM + vertabelo.timestampmeasures m +WHERE + WEEKDAY(m.measured_on) = 0; # MONDAY +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 1907 | 4578 | +| 最大 | 5437 | 10235 | +| 平均 | 3408 | 7102 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp_1_count.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.timestampmeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +#### 查询 3 和 INT: + +```SQL +SELECT SQL_NO_CACHE + COUNT(measured_on) +FROM + vertabelo.inttimestampmeasures m +WHERE + WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0; # MONDAY +``` + +#### 性能 + +| 响应时间 (ms) | Sysbench | mysqlslap | +|-------------|----------|-----------| +| 最小 | 2108 | 5609 | +| 最大 | 4764 | 9735 | +| 平均 | 3307 | 7416 | + +```bash +Sysbench cmd> sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int_1_count.lua --num-threads=8 --max-requests=100 run +``` + +```bash +mysqlslap cmd> mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.inttimestampmeasures m WHERE WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo +``` + +#### 测试 3 总结 + +| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | +|-----------------|----------|----------------------|-----------|----------------------| +| Datetime | 2797 | - | 5540 | - | +| Timestamp | 3408 | 慢 22% | 7102 | 慢 28% | +| INT | 3307 | 慢 18% | 7416 | 慢 33% | + +再一次,两个基准测试工具都显示 Datetime 比 Timestamp 和 INT 快。不能判断 INT 是否比 Timestamp 快,因为 mysqlslap 显示 INT 比 Timestamp 略快而 Sysbench 却相反。 + +_注意:_ 所有测试都是在一台 Windows 10 机器上本地运行的,这台机器拥有一个双核 i7 CPU,16GB 内存,运行 MariaDB v10.1.9,使用 innoDB 引擎。 + +## 结论 + +基于这些数据,我确信 Datetime 是大多数场景下的最佳选择。原因是: + +* 更快(根据我们的三个基准测试)。 +* 无需任何转换即是人类可读的。 +* 不会因为时区变换产生问题。 +* 只比它的对手们多用 1 字节 +* 支持更大的日期范围(从 1000 年到 9999 年) + +如果你只是存储 Unix 时间戳(并且在它的合法日期范围内),而且你真的不打算在它上面使用任何基于日期的查询,我觉得使用 INT 是可以的。我们已经看到,它执行简单数值比较查询时非常快,因为只是在处理简单的数字。 + +Timestamp 怎么样呢?如果 Datetime 相对于 Timestamp 的优势不适用于你特殊的场景,你最好使用时间戳。阅读这篇文章后,你对三种类型间的区别应该有了更好的理解,可以根据你的需要做出最佳的选择。 + +-------------------------------------------------------------------------------- + +via: http://www.vertabelo.com/blog/technical-articles/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int + +作者:[Francisco Claria][a] +译者:[bianjp](https://github.com/bianjp) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.axones.com.ar/ diff --git a/translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md b/translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md deleted file mode 100644 index 7e6ab10c0e..0000000000 --- a/translated/20160823 What Datatype Should You Use to Represent Time in MySQL.md +++ /dev/null @@ -1,563 +0,0 @@ -MySQL 中你应该使用什么数据类型表示时间?(比较 Datetime, Timestamp 和 Int) -========================================================== - -![](http://www.vertabelo.com/_file/blog/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int/clock.jpg) - -_当你需要保存日期时间数据时,一个问题来了:使用 MySQL 的什么类型。使用 MySQL 原生的 DATE 类型还是使用 INT 字段把日期和时间保存为一个纯数字呢?_ - -在这篇文章中,我将解释 MySQL 原生的方案,并给出一个最常用数据类型的对比表。我们也将对一些典型的查询做基准测试然后得出在给定场景下应该使用什么数据类型的结论。 - -### 原生的 MySQL Datetime 数据类型 - -Datetime 数据表示一个时间点。这可以是条日志记录、物联网时间戳、日历事件数据,等等。MySQL 有两种原生的类型可以将这种信息保存在单个字段中:Datetime 和 Timestamp。MySQL 文档中是这么介绍这些数据类型的: - ->DATETIME 类型用于保存同时包含日期和时间两部分的值。MySQL 以 'YYYY-MM-DD HH:MM:SS' 形式接收和显示 DATETIME 类型的值。 - ->TIMESTAMP 类型用于保存同时包含日期和时间两部分的值。 - ->DATETIME 或 TIMESTAMP 类型的值可以在尾部包含一个毫秒部分,精确度最高到微秒(6 位数)。 - ->TIMESTAMP 和 DATETIME 数据类型提供自动初始化和更新到当前的日期和时间的功能,只需在列的定义中设置 DEFAULT CURRENT_TIMESTAMP 和 ON UPDATE CURRENT_TIMESTAMP。 - -So, as an example: -作为一个例子: - -```SQL -CREATE TABLE `datetime_example` ( - `id` int(11) NOT NULL AUTO_INCREMENT, - `measured_on` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, - PRIMARY KEY (`id`), - KEY `measured_on` (`measured_on`) -) ENGINE=InnoDB; -``` - -```SQL -CREATE TABLE `timestamp_example` ( - `id` int(11) NOT NULL AUTO_INCREMENT, - `measured_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, - PRIMARY KEY (`id`), - KEY `measured_on` (`measured_on`) -) ENGINE=InnoDB; -``` - -除了原生的日期时间表示方法,还有另一种常用的方法用以存储日期和时间信息。即使用 INT 字段保存 Unix 时间(从1970年1月1日协调世界时(UTC)建立经过的秒数)。 - -MySQL 也提供了只保存时间信息中的一部分的方式,通过使用 Date, Year, 或 Time 类型。由于这篇文章是关于保存准确时间点的最佳方式的,我们没有讨论这些不那么精确的局部类型。 - -### 使用 INT 类型保存 Unix 时间 - -使用一个简单的 INT 列保存 Unix 时间是最普通的方法。使用 INT,你可以确保你要保存的数字可以快速、可靠地插入到表中,就像这样: - -```SQL -INSERT INTO `vertabelo`.`sampletable` -( - `id`, - `measured_on` # INT 类型的列 -) -VALUES -( - 1, - 946684801 - #unix stamp equivalent to 01/01/2000 @ 12:00am (UTC) http://unixtimestamp.com -); -``` - -这就是关于它的所有了。它仅仅是个简单的 INT 列,MySQL 会这么处理它,在内部使用 4 bytes 保存那些数据。所以如果你在这个列上使用 SELECT 你将会得到一个数字。如果你想把这个列用作日期进行比较,下面的查询并不能正确工作: - -```SQL -SELECT - id, measured_on, FROM_UNIXTIME(measured_on) -FROM - vertabelo.inttimestampmeasures -WHERE - measured_on > '2016-01-01' # measured_on 作为字符串比较以解析查询 -LIMIT 5; -``` - -这是因为 MySQL 把 INT 视为数字,而非日期。为了进行日期比较,你必须要么获取(译注:从 1970-01-01)到 2016-01-01 经过的秒数,要么使用 MySQL 的 FROM_UNIXTIME() 函数把 INT 列转为 Date 类型。下面的查询展示了 FROM_UNIXTIME() 函数的用法: - -```SQL -SELECT - id, measured_on, FROM_UNIXTIME(measured_on) -FROM - vertabelo.inttimestampmeasures -WHERE - FROM_UNIXTIME(measured_on) > '2016-01-01' -LIMIT 5; -``` - -这会正确地获取到日期在 2016-01-01 之后的记录。你也可以直接比较数字和 2016-01-01 的 Unix 表示形式,即 1451606400。这样做意味着不用使用任何特殊的函数,因为你是在直接比较数字。查询如下: - -```SQL -SELECT - id, measured_on, FROM_UNIXTIME(measured_on) -FROM - vertabelo.inttimestampmeasures -WHERE - measured_on > 1451606400 -LIMIT 5; -``` - -如果这种方式仅仅是不够高效甚至提前做这种转换是不可行的,那该怎么办?例如,你想获取 2016 年所有星期三的记录。要做到这样而不使用任何 MySQL 日期函数,你不得不查出 2016 年每个星期三的开始和结束时间的 Unix 时间戳。然后你不得不写很大的查询,至少要在 WHERE 中包含 104 个比较。(2016 年有 52 个星期三,你不得不考虑一天的开始(0:00 am)和结束(11:59:59 pm)...) - -结果是你很可能最终会使用 FROM_UNIXTIME() 转换函数。既然如此,为什么不试下真正的日期类型呢? - -### 使用 Datetime 和 Timestamp - -Datetime 和 Timestamp 几乎以同样的方式工作。两种都保存日期和时间信息,毫秒部分最高精确度都是 6 位数。同时,使用人类可读的日期形式如 "2016-01-01" (为了便于比较)都能工作。查询时两种类型都支持“宽松格式”。宽松的语法允许任何标点符号作为分隔符。例如,"YYYY-MM-DD HH:MM:SS" 和 "YY-MM-DD HH:MM:SS" 两种形式都可以。在宽松格式情况下以下任何一种形式都能工作: - -``` -2012-12-31 11:30:45 -2012^12^31 11+30+45 -2012/12/31 11*30*45 -2012@12@31 11^30^45 -``` - -其它宽松格式也是允许的;你可以在 [MySQL 参考手册](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-literals.html) 找到所有的。 - -默认情况下,Datetime 和 Timestamp 两种类型都以标准输出格式接收 —— 年-月-日 时:分:秒 (如 2016-01-01 23:59:59)。如果使用了毫秒部分,它们应该以小数值出现在秒后面 (如 2016-01-01 23:59:59.5)。 - -Timestamp 和 Datetime 的核心不同点主要在于 MySQL 在内部如何表示这些信息:两种都以二进制而非字符串形式存储,但在表示日期/时间部分时 Timestamp (4 bytes) 比 Datetime (5 bytes) 少使用 1 byte。当保存毫秒部分时两种都使用额外的空间 (1-3 bytes)。如果你存储 150 万条记录,这种 1 byte 的差异是微不足道的: - ->150 万条记录 * 1 byte 每条记录 / (1048576 bytes/MB) = __1.43 MB__ - -Timestamp 节省的 1 byte 是有代价的:你只能存储从 '1970-01-01 00:00:01.000000' 到 '2038-01-19 03:14:07.999999' 之间的时间。而 Datetime 允许你存储从 '1000-01-01 00:00:00.000000' 到 '9999-12-31 23:59:59.999999' 之间的任何时间。 - -一个重要的差别 —— 很多 MySQL 开发者没意识到的 —— 是 MySQL 使用__服务器的时区__转换 Timestamp 值到它的 UTC 等价值再保存。当获取值是它会再次进行时区转换,所以你得回了你“原始的”日期/时间值。也许是。下面这些情况可能会发生。 - -理想情况下,如果你保持在同一个时区,MySQL 会获取到和你存储的同样的值。以我的经验,如果你的数据库涉及时区变换,你可能会遇到问题。例如,服务器变化(比如,你把数据库从都柏林的一台服务器迁移到加利福尼亚的一台服务器上,或者你只是修改了服务器的时区)时可能会发生这种情况。不管哪种方式,如果你获取数据时的时区是不同的,数据就会受影响。 - -Datetime 列不会被数据库改变。无论时区怎样配置,每次都会保存和获取到同样的值。就我而言,我认为这是一个更一致的选择。 - ->__MySQL 文档:__ - ->MySQL 把 TIMESTAMP 值从当前的时区转换到 UTC 再存储,获取时再从 UTC 转回当前的时区。(其它类型如 DATETIME 不会这样,它们会原样保存。) 默认情况下,每个连接的当前时区都是服务器的时区。时区可以基于连接设置。只要时区设置保持一致,你就能得到和保存的相同的值。如果你保存了一个 TIMESTAMP 值,然后改变了时区再获取这个值,获取到的值和你存储的是不同的。这是因为在不同方向的会话上没有使用同一个时区。当前时区可以通过系统变量 [time_zone](https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_time_zone) 的值得到。更多信息,请查看 [MySQL Server Time Zone Support](https://dev.mysql.com/doc/refman/5.7/en/time-zone-support.html)。 - -### 对比总结 - -在深入探讨使用各数据类型的性能差异之前,让我们先看一个总结表格以给你更多了解。每种类型的弱点以红色显示。 - -特性 | Datetime | Timestamp | Int (保存 Unix 时间) -:--|:--|:--|:-- -原生时间表示 | 是 | 是 | 否,所以大多数操作需要先使用转换函数,如 FROM_UNIXTIME() -能保存毫秒 | 是,最高 6 位精度 | 是,最高 6 位精度 | 否 -合法范围 | '1000-01-01 00:00:00.000000' to '9999-12-31 23:59:59.999999 | '1970-01-01 00:00:01.000000' to '2038-01-19 03:14:07.999999' | 若使用 unsigned, '1970-01-01 00:00:01.000000; 理论上最大到 '2106-2-07 06:28:15' -自动初始化(MySQL 5.6.5+) | 是 | 是 | 否 -宽松解释 ([MySQL docs](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-literals.html)) | 是 | 是 | 否,必须使用正确的格式 -值被转换到 UTC 存储 | 否 | 是 | 否 -可转换到其它类型 | 是,如果值在合法的 Timestamp 范围中 | 是,总是 | 是,如果值在合法的范围中并使用转换函数 -存储需求([MySQL 5.6.4+](https://dev.mysql.com/doc/refman/5.7/en/storage-requirements.html)) | 5 bytes (如果使用了毫秒部分,再加最多 3 bytes) | 4 bytes (如果使用了毫秒部分,再加最多 3 bytes) | 4 bytes (不允许毫秒部分) -无需使用函数即可作为真实日期可读 | 是 | 是 | 否,你不得不格式化输出 -分区 | 是 | 是,使用 [UNIX_TIMESTAMP()](https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html#function_unix-timestamp);在 MySQL 5.7 中包含 [TIMESTAMP](https://dev.mysql.com/doc/refman/5.7/en/datetime.html) 值的其它表达式是不允许的。同时,注意[分区裁剪时的这些考虑](https://dev.mysql.com/doc/refman/5.7/en/partitioning-pruning.html) | 是,使用 INTs 上的任何合法操作 - - -### 基准测试 INT, Timestamp 和 Datetime 的性能 - -为了比较这些类型的性能,我会使用来自我创建的一个天气预报网络的 150 万记录(准确说是 1,497,421)。这个网络每分钟都收集数据。为了让这些测试可复现,我已经删除了一些私有列,所以你可以使用这些数据运行你自己的测试。 - -基于我原始的表格,我创建了三个版本: - -- `datetimemeasures` 表在 `measured_on` 列使用 Datetime 类型,表示天气预报记录的测量时间 -- `timestampmeasures` 表在 `measured_on` 列使用 Timestamp 类型 -- `inttimestampmeasures` 表在 `measured_on` 列使用 INT (unsigned) 类型 - -这三个表拥有完全相同的数据;唯一的差别就是 `measured_on` 字段的类型。所有表都在 `measured_on` 列上设置了一个索引。 - -![](./images/table-datetime.png) -![](./images/table-timestamp.png) -![](./images/table-int.png) - -#### 基准测试工具 - -为了评估这些数据类型的性能,我使用了两种方法。一种基于 [Sysbench](https://github.com/akopytov/sysbench),它的官网是这么描述的: - -_“... 一个模块化、跨平台和多线程的基准测试工具,用以评估那些对运行高负载数据库的系统非常重要的系统参数。”_ - -这个工具是 [MySQL 文档](https://dev.mysql.com/downloads/benchmarks.html)中建议的. - -如果你使用 Windows (就像我),你可以下载一个包含可执行文件和我使用的测试查询的 zip 文件。他们基于 [一种推荐的基准测试方法](https://dba.stackexchange.com/questions/39221/stress-test-mysql-with-queries-captured-with-general-log-in-mysql)。 - -为了执行一个给定的测试,你可以使用下面的命令(插入你自己的连接参数): - -```bash -sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=sysbench_test_file.lua --num-threads=8 --max-requests=100 run -``` - -这会正常工作,如果 `sysbench_test_file.lua` 是测试文件,并包含各个测试中指向各个表的 SQL 查询。 - -为了进一步验证结果,我也运行了 [mysqlslap](https://dev.mysql.com/doc/refman/5.7/en/mysqlslap.html)。它的官网是这么描述的: - -_“[mysqlslap](https://dev.mysql.com/doc/refman/5.7/en/mysqlslap.html) 是一个诊断程序,为模拟 MySQL 服务器的客户端负载并报告各个阶段的用时而设计。它工作起来就像是很多客户端在同时访问服务器。_ - -记得这些测试中最重要的不是所需的_绝对_时间。而是在不同数据类型上执行相同查询时的_相对_时间。这两个基准测试工具的测试时间不一定相同,因为不同工具的工作方式不同。重要的是数据类型的比较,随着我们深入到测试中,这将会变得清楚。 - -#### 基准测试 - -我将使用三种可以评估几个性能方面的查询: - -* 时间范围选择 - - * 在 Datetime 和 Timestamp 数据类型上这允许我们直接比较而不需要使用任何特殊的日期函数 - * 同时,我们可以评估在 INT 类型的列上使用日期函数相对于使用简单的数值比较的影响。为了做到这些我们需要把范围转换为 Unix 时间戳数值。 - -* 日期函数选择: - - * 与前个测试中比较操作针对一个简单的 DATE 值相反,这个测试使得我们可以评估使用日期函数作为 “WHERE” 子句的一部分的性能。 - * 我们还可以测试一个场景即我们必须使用一个函数将 INT 列转换未一个合法的 DATE 类型然后执行查询。 - -* count() 查询 - - * 作为对前面测试的补充,这将评估在三种不同的表示类型上进行典型的摘要查询的性能。 - -我们将在这些测试中覆盖一些常见的场景,并看到三种类型上的性能表现。 - -#### 关于 SQL_NO_CACHE - -当在查询中使用 SQL_NO_CACHE 时,服务器不使用查询缓存。它既不检查查询缓存以确认结果是不是已经在那儿了,也不会保存查询结果。因此,每个查询将反映真实的性能影响,就像每次查询都是第一次被调用。 - -#### 测试 1:选择在一个日期范围中的值 - -这个查询返回总计 1,497,421 行记录中的 75,706 行。 - -##### 查询 1 和 Datetime: - -```SQL -SELECT SQL_NO_CACHE - measured_on -FROM - vertabelo.datetimemeasures m -WHERE - m.measured_on > '2016-01-01 00:00:00.0' - AND m.measured_on < '2016-02-01 00:00:00.0'; -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 152 | 296 | -| 最大 | 1261 | 3203 | -| 平均 | 362 | 809 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.datetimemeasures m WHERE m.measured_on > '2016-01-01 00:00:00.0' AND m.measured_on < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo -``` - -##### 查询 1 和 Timestamp: - -```SQL -SELECT SQL_NO_CACHE - measured_on -FROM - vertabelo.timestampmeasures m -WHERE - m.measured_on > '2016-01-01 00:00:00.0' - AND m.measured_on < '2016-02-01 00:00:00.0'; -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 214 | 359 | -| 最大 | 1389 | 3313 | -| 平均 | 431 | 1004 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.timestampmeasures m WHERE m.measured_on > '2016-01-01 00:00:00.0' AND m.measured_on < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo -``` - -##### 查询 1 和 INT: - -```SQL -SELECT SQL_NO_CACHE - measured_on -FROM - vertabelo.inttimestampmeasures m -WHERE - FROM_UNIXTIME(m.measured_on) > '2016-01-01 00:00:00.0' - AND FROM_UNIXTIME(m.measured_on) < '2016-02-01 00:00:00.0'; -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 2472 | 7968 | -| 最大 | 6554 | 10312 | -| 平均 | 4107 | 8527 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE FROM_UNIXTIME(m.measured_on) > '2016-01-01 00:00:00.0' AND FROM_UNIXTIME(m.measured_on) < '2016-02-01 00:00:00.0'" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo -``` - -##### 另一种 INT 上的查询 1: - -由于这是个相当直接的范围搜索而且查询中的日期可以轻易地转为简单的数值比较,我将它包含在了这个测试中。结果证明这是最快的方法 (你大概已经预料到了),因为它仅仅是比较数字而没有使用任何日期转换函数: - -```SQL -SELECT SQL_NO_CACHE - measured_on -FROM - vertabelo.inttimestampmeasures m -WHERE - m.measured_on > 1451617200 - AND m.measured_on < 1454295600; -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 88 | 171 | -| 最大 | 275 | 2157 | -| 平均 | 165 | 514 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=basic_int.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE m.measured_on > 1451617200 AND m.measured_on < 1454295600" --host=localhost --user=root --concurrency=8 --iterations=100 --no-drop --create-schema=vertabelo -``` - -##### 测试 1 总结 - -| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | -|-----------------|----------|----------------------|-----------|----------------------| -| Datetime | 362 | - | 809 | - | -| Timestamp | 431 | 慢 19% | 1004 | 慢 24% | -| INT | 4107 | 慢 1134% | 8527 | 慢 1054% | -| 另一种 INT 查询 | 165 | 快 55% | 514 | 快 36% | - -两种基准测试工具都显示 Datetime 比 Timestamp 和 INT 更快。但 Datetime 没有我们在另一种 INT 查询中使用的简单数值比较快。 - -#### 测试 2:选择星期一产生的记录 - -这个查询返回总计 1,497,421 行记录中的 221,850 行。 - -##### 查询 2 和 Datetime: - -```SQL -SELECT SQL_NO_CACHE measured_on -FROM - vertabelo.datetimemeasures m -WHERE - WEEKDAY(m.measured_on) = 0; # MONDAY -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 1874 | 4343 | -| 最大 | 6168 | 7797 | -| 平均 | 3127 | 6103 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime_1.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.datetimemeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo -``` - -##### 查询 2 和 Timestamp: - -```SQL -SELECT SQL_NO_CACHE - measured_on -FROM - vertabelo.timestampmeasures m -WHERE - WEEKDAY(m.measured_on) = 0; # MONDAY -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 2688 | 5953 | -| 最大 | 6666 | 13531 | -| 平均 | 3653 | 8412 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp_1.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.timestampmeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo -``` - -##### 查询 2 和 INT: - -```SQL -SELECT SQL_NO_CACHE - measured_on -FROM - vertabelo.inttimestampmeasures m -WHERE - WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0; # MONDAY -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 2051 | 5844 | -| 最大 | 7007 | 10469 | -| 平均 | 3486 | 8088 | - - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int_1.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE measured_on FROM vertabelo.inttimestampmeasures m WHERE WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo -``` - -##### 测试 2 总结 - -| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | -|-----------------|----------|----------------------|-----------|----------------------| -| Datetime | 3127 | - | 6103 | - | -| Timestamp | 3653 | 慢 17% | 8412 | 慢 38% | -| INT | 3486 | 慢 11% | 8088 | 慢 32% | - -再次,在两个基准测试工具中 Datetime 比 Timestamp 和 INT 快。但在这个测试中,INT 查询 —— 即使它使用了一个函数以转换日期 —— 比 Timestamp 查询更快得到结果。 - -#### 测试 3:选择星期一产生的记录总数 - -这个查询返回一行,包含产生于所有星期一的记录的总数(从总共 1,497,421 行可用记录中)。 - -##### 查询 3 和 Datetime: - -```SQL -SELECT SQL_NO_CACHE - COUNT(measured_on) -FROM - vertabelo.datetimemeasures m -WHERE - WEEKDAY(m.measured_on) = 0; # MONDAY -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 1720 | 4063 | -| 最大 | 4594 | 7812 | -| 平均 | 2797 | 5540 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=datetime_1_count.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.datetimemeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo -``` - -##### 查询 3 和 Timestamp: - -```SQL -SELECT SQL_NO_CACHE - COUNT(measured_on) -FROM - vertabelo.timestampmeasures m -WHERE - WEEKDAY(m.measured_on) = 0; # MONDAY -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 1907 | 4578 | -| 最大 | 5437 | 10235 | -| 平均 | 3408 | 7102 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=timestamp_1_count.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.timestampmeasures m WHERE WEEKDAY(m.measured_on) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo -``` - -##### 查询 3 和 INT: - -```SQL -SELECT SQL_NO_CACHE - COUNT(measured_on) -FROM - vertabelo.inttimestampmeasures m -WHERE - WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0; # MONDAY -``` - -##### 性能 - -| 响应时间 (ms) | Sysbench | mysqlslap | -|-------------|----------|-----------| -| 最小 | 2108 | 5609 | -| 最大 | 4764 | 9735 | -| 平均 | 3307 | 7416 | - -```bash -**Sysbench cmd>** sysbench --MySQL-table-engine=innodb --MySQL-db=vertabelo --MySQL-user=root --MySQL-host=localhost --MySQL-password= --test=int_1_count.lua --num-threads=8 --max-requests=100 run -``` - -```bash -**mysqlslap cmd>** mysqlslap --query="SELECT SQL_NO_CACHE COUNT(measured_on) FROM vertabelo.inttimestampmeasures m WHERE WEEKDAY(FROM_UNIXTIME(m.measured_on)) = 0" --host=localhost --user=root --concurrency=8 --iterations=25 --no-drop --create-schema=vertabelo -``` - -##### 测试 3 总结 - -| 平均响应时间 (ms) | Sysbench | 相对于 Datetime 的速度 | mysqlslap | 相对于 Datetime 的速度 | -|-----------------|----------|----------------------|-----------|----------------------| -| Datetime | 2797 | - | 5540 | - | -| Timestamp | 3408 | 慢 22% | 7102 | 慢 28% | -| INT | 3307 | 慢 18% | 7416 | 慢 33% | - -再一次,两个基准测试工具都显示 Datetime 比 Timestamp 和 INT 快。不能判断 INT 是否比 Timestamp 快,因为 mysqlslap 显示 INT 比 Timestamp 略快而 Sysbench 却相反。 - -_注意:_ 所有测试都是在一台 Windows 10 机器上本地运行的,这台机器拥有一个双核 i7 CPU,16GB 内存,运行 MariaDB v10.1.9,使用 innoDB 引擎。 - -### 结论 - -基于这些数据,我确信 Datetime 是大多数场景下的最佳选择。原因是: - -* 更快(根据我们的三个基准测试)。 -* 无需任何转换即是人类可读的。 -* 不会因为时区变换产生问题。 -* 只比它的对手们多用 1 byte -* 支持更大的日期范围(从 1000 年到 9999 年) - -如果你只是存储 Unix 时间戳(并且在它的合法日期范围内),而且你真的不打算在它上面使用任何基于日期的查询,我觉得使用 INT 是可以的。我们已经看到,它执行简单数值比较查询时非常快,因为只是在处理简单的数字。 - -Timestamp 怎么样呢?如果 Datetime 相对于 Timestamp 的优势不适用于你特殊的场景,你最好使用时间戳。阅读这篇文章后,你对三种类型间的区别应该有了更好的理解,可以根据你的需要做出最佳的选择。 - --------------------------------------------------------------------------------- - -via: http://www.vertabelo.com/blog/technical-articles/what-datatype-should-you-use-to-represent-time-in-mysql-we-compare-datetime-timestamp-and-int?utm_source=dbweekly&utm_medium=email - -作者:[Francisco Claria][a] -译者:[bianjp](https://github.com/bianjp) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.axones.com.ar/ From 0d0e2069cf4bea1869c8c8255de4a645809a3267 Mon Sep 17 00:00:00 2001 From: rusking Date: Sun, 4 Sep 2016 12:31:34 +0800 Subject: [PATCH 0028/2437] Update 20160728 I've been Linuxing since before you were born.md --- .../20160728 I've been Linuxing since before you were born.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/talk/20160728 I've been Linuxing since before you were born.md b/sources/talk/20160728 I've been Linuxing since before you were born.md index d9fbeb9970..2466813a7d 100644 --- a/sources/talk/20160728 I've been Linuxing since before you were born.md +++ b/sources/talk/20160728 I've been Linuxing since before you were born.md @@ -1,3 +1,5 @@ +rusking 翻译中 + I've been Linuxing since before you were born ===================== From c9dd1f8adf6c881771fe0bedac6def2b4c863509 Mon Sep 17 00:00:00 2001 From: rusking Date: Sun, 4 Sep 2016 12:40:35 +0800 Subject: [PATCH 0029/2437] Update 20160808 What is open source.md --- sources/talk/20160808 What is open source.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/talk/20160808 What is open source.md b/sources/talk/20160808 What is open source.md index 1ff197dee4..b82747b426 100644 --- a/sources/talk/20160808 What is open source.md +++ b/sources/talk/20160808 What is open source.md @@ -1,3 +1,5 @@ +rusking 翻译中 + What is open source =========================== From e25b318029711ea2bc8e91da7221079288f9c07f Mon Sep 17 00:00:00 2001 From: wxy Date: Sun, 4 Sep 2016 16:37:06 +0800 Subject: [PATCH 0030/2437] PUB:20160505 Confessions of a cross-platform developer @vim-kakali --- ...nfessions of a cross-platform developer.md | 73 +++++++++++++ ...nfessions of a cross-platform developer.md | 103 ------------------ 2 files changed, 73 insertions(+), 103 deletions(-) create mode 100644 published/20160505 Confessions of a cross-platform developer.md delete mode 100644 translated/talk/20160505 Confessions of a cross-platform developer.md diff --git a/published/20160505 Confessions of a cross-platform developer.md b/published/20160505 Confessions of a cross-platform developer.md new file mode 100644 index 0000000000..72716e9eb8 --- /dev/null +++ b/published/20160505 Confessions of a cross-platform developer.md @@ -0,0 +1,73 @@ +一位跨平台开发者的自白 +============================================= + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/business_clouds.png?itok=cucHuJnU) + +[Andreia Gaita][1] 在 OSCON 开源大会上发表了一个题为[跨平台开发者的自白][2]的演讲。她长期从事于开源工作,并且为 [Mono][3] 工程(LCTT 译注:一个致力于开创 .NET 在 Linux 上使用的开源工程)做着贡献,主要以 C#/C++ 开发。Andreia 任职于 GitHub,她的工作是专注构建 Visual Studio 的 GitHub 扩展管理器。 + +我在她发表演讲前就迫不及待的想要问她一些关于跨平台开发的事,问问她作为一名跨平台开发者在这 16 年之中学习到了什么。 + +![](https://opensource.com/sites/default/files/images/life/Interview%20banner%20Q%26A.png) + +**在你开发跨平台代码中,你使用过的最简单的和最难的代码语言是什么?** + +我很少讨论某种语言的好坏,更多是讨论是那些语言有哪些库和工具。语言的编译器、解释器以及构建系统决定了用它们做跨平台开发的难易程度(或者它们是否可能做跨平台开发),可用的 UI 库和对本地系统的访问能力决定了与该操作系统集成的紧密程度。按照我的观点,我认为 C# 最适合完成跨平台开发工作。这种语言自身包括了允许快速的本地调用和精确的内存映射的功能,如果你希望你的代码能够与系统和本地函数库进行交互就需要这些功能。而当我需要非常特殊的系统功能时,我就会切换到 C 或者 C++。 + +**你使用的跨平台开发工具或者抽象层有哪些?** + +我的大部分跨平台工作都是为其它需要开发跨平台应用的人开发工具、库和绑定(binding),一般是用 MONO/C# 和 C/C++。在抽象的层面我用的不多,更多是在 glib 库和友元(friends)方面。大多数时候,我用 Mono 去完成各种跨平台应用的,包括 UI,或者偶然在游戏开发中用到 Unity3D 的部分。我经常使用 Electron(LCTT 译注:Atom 编辑器的兄弟项目,可以用 Electron 开发桌面应用)。 + +**你接触过哪些构建系统?它们之间的区别是由于语言还是平台的不同?** + +我试着选择适合我使用的语言的构建系统。那样,就会很少遇到让我头疼的问题(希望如此)。它需要支持平台和体系结构间的选择、构建输出结果位置可以智能些(多个并行构建),以及易配置性等。大多数时候,我的项目会结合使用 C/C++ 和 C#,我要从同一源代码同时构建不同的配置环境(调试、发布、Windows、OSX、Linux、Android、iOS 等等),这通常需要为每个构建的输出结果选择带有不同参数的不同编译器。构建系统可以帮我做到这一切而不用让我(太)费心。我时常尝试着用不同的构建系统,看看有些什么新的变化,但是最终,我还是回到了使用 makefile 的情况,并结合使用 shell 和批处理脚本或 Perl 脚本来完成工作(因为如果我希望用户来构建我的软件,我还是最好选择一种到处都可以用的命令行脚本语言)。 + +**你怎样平衡在这种使用统一的用户界面下提供原生的外观和体验的需求呢?** + +跨平台的用户界面的实现很困难。在过去几年中我已经使用了一些跨平台 GUI,并且我认为这些事情上并没有最优解。基本上有两种选择。你可以选择一个跨平台工具去做一个并不是完全适合你所有支持的平台的 UI,但是代码库比较小,维护成本比较低。或者你可以选择去开发针对平台的 UI,那样看起来更原生,集成的也更好,但是需要更大的代码库和更高的维护成本。这种决定完全取决于 APP 的类型、它有多少功能、你有多少资源,以及你要把它运行在多少平台上? + +最后,我认为用户比较接受这种“一个 UI 打通关”了,就比如 Electron 框架。我有个 Chromium+C+C# 的框架侧项目,有一天我希望可以用 C# 构建 Electron 型的 app,这样的话我就可以做到两全其美了。 + +**构建/打包系统的依赖性对你有影响吗 ?** + +我依赖的使用方面很保守,我被崩溃的 ABI(LCTT 译注:应用程序二进制接口)、冲突的符号、以及丢失的包等问题困扰了太多次。我决定我要针对的操作系统版本,并选择最低的公有部分来使问题最小化。通常这就意味着有五种不同的 Xcode 和 OSX 框架库,要在同样的机器上相应安装五种不同的 Visual Studio 版本,多种 clang(LCTT 译注:C语言、C++、Object-C、C++ 语言的轻量级编译器)和 gcc 版本,一系列的运行着各种发行版的虚拟机。如果我不能确定我要使用的操作系统的包的状态,我有时就会静态连接库,有时会子模块化依赖以确保它们一直可用。大多时候,我会避免这些很棘手的问题,除非我非常需要使用他们。 + +**你使用持续集成(CI)、代码审查以及相关的工具吗?** + +基本每天都用。这是保持高效的唯一方式。我在一个项目中做的第一件事情是配置跨平台构建脚本,保证每件事尽可能自动化完成。当你面向多平台开发的时候,持续集成是至关重要的。没有人能在一个机器上构建各种平台的不同组合,并且一旦你的构建过程没有包含所有的平台,你就不会注意到你搞砸的事情。在一个共享式的多平台代码库中,不同的人拥有不同的平台和功能,所以保证质量的唯一的方法是跨团队代码审查结合持续集成和其他分析工具。这不同于其他的软件项目,如果不使用相关的工具就会面临失败。 + +**你依赖于自动构建测试,或者倾向于在每个平台上构建并且进行本地测试吗?** + +对于不包括 UI 的工具和库,我通常使用自动构建测试。如果有 UI,两种方法我都会用到——针对已有的 GUI 工具的可靠的、可脚本化的 UI 自动化少到几乎没有,所以我要么我去针对我要跨我所支持的平台创建 UI 自动化工具,要么手动进行测试。如果一个项目使用了定制的 UI 库(比如说一个类似 Unity3D 的 OpenGL UI),开发可编程的自动化工具并自动化大多数工作就相当容易。不过,没有什么东西会像人一样双击就测试出问题。 + +**如果你要做跨平台开发,你喜欢用跨编辑器的构建系统,比如在 Windows 上使用 Visual Studio,在 Linux 上使用 Qt Creator,在 Mac 上使用 XCode 吗?还是你更趋向于使用 Eclipse 这样的可以在所有平台上使用的单一平台?** + +我喜欢使用跨编辑器的构建系统。我更喜欢在不同的IDE上保存项目文件(这样可以使增加 IDE 变得更容易),通过使用构建脚本让 IDE 在它们支持的平台上去构建。对于一个开发者来说编辑器是最重要的工具,学习它们是需要花费时间和精力的,而它们是不可相互替代的。我有我自己喜欢的编辑器和工具,每个人也可以使用他们最喜爱的工具。 + +**在跨平台开发的时候,你更喜欢使用什么样的编辑器、开发环境和 IDE 呢?** + +跨平台开发者被限制在只能选择可以在多数平台上工作的所共有的不多选择之一。我爱用 Visual Studio,但是我不能依赖它完成除 Windows 平台之外的工作(你可能不想让 Windows 成为你的主要的交叉编译平台),所以我不会使用它作为我的主要 IDE。即使可以,跨平台开发者的核心技能也是尽可能的了解和使用大量的平台。这就意味着必须很熟悉它们——使用该平台上的编辑器和库,了解这种操作系统及其适用场景、运行方式以及它的局限性等。做这些事情就需要头脑清醒(我的捷径是加强记忆),我必须依赖跨平台的编辑器。所以我使用 Emacs 和 Sublime。 + +**你之前和现在最喜欢的跨平台项目是什么?** + +我一直很喜欢 Mono,并且得心应手,其它的项目大部分都是以某种方式围绕着它进行的。Gluezilla 是我在多年前开发的一个 Mozilla 绑定(binding),可以把 C# 开发的应用嵌入到 Web 浏览器页面中,并且看起来很有特色。我开发过一个 Winform 应用,它是在 linux 上开发的,它可以在 Windows 上运行在一个 Mozilla 浏览器页面里嵌入的 GTK 视图中。CppSharp 项目(以前叫做 Cxxi,更早时叫做 CppInterop)是一个我开始为 C++ 库生成 C# 绑定(binding)的项目,这样就可以在 C# 中调用、创建实例、子类化 C++ 类。这样,它在运行的时候就能够检测到所使用的平台,以及用来创建本地运行库的是什么编译器,并为它生成正确的 C# 绑定(binding)。这多么有趣啊。 + +**你怎样看跨平台开发的未来趋势呢?** + +我们构建本地应用程序的方式已经改变了,我感觉在各种桌面操作系统的明显差异在慢慢变得模糊;所以构建跨平台的应用程序将会更加容易,而且对系统的集成也不需要完全本地化。不好的是,这可能意味着应用程序易用性更糟,并且在发挥操作系统特性方面所能做的更少。库、工具以及运行环境的跨平台开发是一种我们知道怎样做的更好,但是跨平台应用程序的开发仍然需要我们的努力。 + + +-------------------------------------------------------------------------------- + +via: https://opensource.com/business/16/5/oscon-interview-andreia-gaita + +作者:[Marcus D. Hanwell][a] +译者:[vim-kakali](https://github.com/vim-kakali) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/mhanwell +[1]: https://twitter.com/sh4na +[2]: http://conferences.oreilly.com/oscon/open-source-us/public/schedule/detail/48702 +[3]: http://www.mono-project.com/ + diff --git a/translated/talk/20160505 Confessions of a cross-platform developer.md b/translated/talk/20160505 Confessions of a cross-platform developer.md deleted file mode 100644 index 611ccf506c..0000000000 --- a/translated/talk/20160505 Confessions of a cross-platform developer.md +++ /dev/null @@ -1,103 +0,0 @@ - -一位跨平台开发者的自白 -============================================= - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/business_clouds.png?itok=cucHuJnU) - -Andreia Gaita[1]将会在OSCON开源大会上发表一个题为[跨平台开发者的自白][2]的演讲。她长期从事于开源工作,并且为Mono[3]【译者注:一个致力于开创.NET在Linux上使用的开源工程】工程做着贡献。Andreia任职于GitHub,她的工作是专注于为Visual Studio构建github上的可扩展管理器。 - - -我在她发表演讲前就迫不及待的想要问她一些关于跨平台开发的事,作为一名跨平台开发者,她已经有了16年的学习经验了。 - -![](https://opensource.com/sites/default/files/images/life/Interview%20banner%20Q%26A.png) - - -**在你跨平台工作的过程中,你使用过的最简单的和最难的代码语言是什么?** - - -很少讨论某种语言的好坏,大多数是关于对于这种语言的库和工具的易用性。编译器、解释器以及构建器语言决定了用它们做跨平台开发的难易程度(不论它是否可能),能够实现UI和本地系统访问的函数库的可用性都决定了开发时兼容操作系统的程度。如果这样想,我认为C#最适合完成跨平台开发工作。这种语言有它自己的特色,它允许本地快速唤醒和精确的内存地址;如果你希望你的代码能够与系统和本地函数库进行交互,那么你就会需要C#。当我需要特殊的系统集成的时候,我就会切换到C或者C++。 - - -**你使用的跨平台开发工具或者抽象有哪些?** - -我的大部分跨平台工作都是为其他人开发工具和库,然后把它们组合起来开发跨平台的应用程序,其中大多用到MONO/C#。说真的,我不太懂抽象。大多数时候,我依赖Mono去完成一个跨平台的app开发以及它的UI,或者一个偶然的游戏开发中的Unity3D部分。我经常使用Electron【译者注:Atom编辑器的兄弟项目,可以用Electron开发桌面应用】。 - - - -**你接触过哪些构建系统?它们之间的区别是由于语言还是平台的不同?** - - -我试着选择适合我使用语言的构建系统。那样 ,就会很少遇到让我头疼的问题。它需要支持平台和体系结构间的选择、构建智能工件位置(多个并行构建)以及易配置性等。大多数时候,我的项目会结合C/C++和C#,同时我也想要为一些开源分支构建相应的配置环境(Debug, Release, Windows, OSX, Linux, Android, iOS, etc, etc.)),通常也需要为每个构建的智能工件选择带有不同标志的编译器。所以我不得不做很多工作,而这种以我的方式进行的工作并没有很多收获。我时常尝试着用不同的构建系统,仅仅是想了解到最新的情况,但是最终我还是回到使用makefile的情况,采用shell和批处理脚本之一与Perl脚本相结合的方式,以达到使用他们的目的(因为如果我想用户去做很多我做的事情,我还是最好选择一种命令行脚本语言,这样它在哪里都可以用)。 - - - -**你怎样平衡在这种统一用户接口视觉体验需求上的强烈渴望呢?** - - - -用户接口的跨平台实现很困难。在过去几年中我已经使用了一些跨平台GUI,并且我认为一些问题没有最优解。那些问题都基于两种操作,你也可以选择一个跨平台工具去做一个UI,去用你所喜欢的所有的平台,这样感觉不是太对,用小的代码库和比较低的维护费用。或者你可以选择去开发一个特有平台的UI,那样看起来就会是很本地化并且能很好的使其与一个大型的代码库结合,也会有很高的维护费用。这种决定完全取决于APP的类型。它的特色在哪里呢?你有什么资源?你可以把它运行在多少平台上? - - -最后,我认为用户对于这种框架性的“一个UI统治所有”的UI的容忍度更大了,就比如Electron。我曾经有个Chromium+C+C#的框架侧项目,并且希望我在一天内用C#构建Electron型的app,这样的话我就可以做到两全其美了。 - - - **你对构建或者打包有依赖性吗 ?** - - - 我很少谈及依赖性问题。很多次我都被ABI【译者注:应用程序二进制接口】的崩溃、存在冲突的征兆以及包的丢失问题所困扰。我决定我要使用的操作系统的版本,都是选择最低的依赖去使问题最小化。通常这就意味着有五种不同的Xcode的副本和OSX框架库 ,在同样的机器上有五种不同的Visual Studio版本需要相应的被安装,多种clang【译者注:C语言、C++、Object-C、C++语言的轻量级编译器】和gcc版本,一系列的可以运行的其他的VM版本。如果我不能确定我要使用的操作系统的包的规定,我时常会连接静态库与子模块之间的依赖确保它们一直可用。大多时候,我会避免这些很棘手的问题,除非我非常需要使用他们。 - - - **你使用能够持续集成的、代码重读的相关工具吗?** - - 基本每天都用。这是保持高效的唯一方式。我在一个项目中做的第一件事情是配置跨平台构建脚本,保证每件事尽可能自动化完成。当你想要使用多平台的时候,CI【译者注:持续集成】是至关重要的。在一个机器上,没有人能结合所有的不同的平台。并且一旦你的构建过程没有包含所有的平台,你就不会注意到你搞砸的事情。在一个共享的多平台代码库中 ,不同的人拥有不同的平台和特征,所以仅有的方法是保证跨团队浏览代码时结合CI和其他分析工具的公平性。这不同于其他的软件项目,如果不使用相关的工具就只有失败了。 - - - **你依赖于自动构建测试或者趋向于在每个平台上构建并且进行局部测试吗?** - - - 对于不包括UI的工具和库,我通常能够侥幸完成自动构建测试。如果那是一个UI,两种方法我都会用到——做一个可靠的,可自动编写脚本的UI,因为基本没有GUI工具,所以我不得不在去创建UI自动化工具,这种工具可以工作在所有的我用到的平台上,或者我也可以手动完成。如果一个项目使用一个定制的UI工具(一个像Unity3D那样做的OpenGL UI ),开发自动化的可编写脚本工具和更多的自动化工具就相当容易。不过,没有什么东西会像人类一样通过双击而搞砸事情。 - - - - **如果你要做跨平台开发,你想要在不同的平台上使用不同的编辑器,比如在Windows上使用Visual Studio,在Linux上使用Qt Creator,在Mac上使用XCode吗?还是你更趋向于使用Eclipse这样的可以在所有平台上使用的编辑器?** - - - - 我喜欢使用不同的编辑器构建系统。我更喜欢在不同的带有构建脚本的IDE上保存项目文件(可以使增加IDE变得更容易),这些脚本可以为他们支持的平台去驱动IDE开始工作。对于一个开发者来说编辑器是最重要的工具,开发者花费时间和精力去学习使用各种编辑器,但是它们都不能相互替代。我使用我最喜欢的编辑器和工具,每个人也应该能使用他们最喜爱的工具。 - - - - **在跨平台开发的时候,你更喜欢使用什么开发环境和IDE呢?** - - - 跨平台开发者好像被诅咒一样,他们不得不选择小众化的编辑器去完成大多数跨平台的工作。我爱用Visual Studio,但是我不能依赖它完成除windows平台外的工作(你可能不想让windows成为你的初级交叉编译平台,所以我不会使用它作为我的初级集成开发环境。即使我这么做了,跨平台开发者的潜意识也知道有可能会用到很多平台。这就意味着必须很熟悉他们——使用一种平台上的编辑器就必须知道这种操作系统的设置,运行方式以及它的局限性等。做这些事情就需要头脑清醒(我的捷径是加强记忆),我不得不依赖于跨平台编辑器。所以我使用Emacs 和Sublime。 - - - - **你最喜欢的过去跨平台项目是什么**? - - - 我一直很喜欢Mono,并且得心应手,在一些开发中大多数的项目都是用一些方法围绕着它进行的。Gluezilla曾经是我在多年前开发的一个Mozilla【译者注:Mozilla基金会,为支持和领导开源Mozilla项目而设立的非营利组织】结合器,可以C#开发的app嵌入到web浏览器试图中,并且看起来很明显。在这一点上,我开发过一个窗体app,它是在linux上开发的,它运行在带有一个嵌入GTK试图的windows系统上,并且这个系统将会运行一个Mozilla浏览器试图。CppSharp项目(以前叫做Cxxi,更早时叫做CppInterop)是一个我开始结合绑定有C#的C++库的项目,这样就可以唤醒和创建实例来把C#结合到C++中。这样做的话,它在运行的时候就能够发现所使用的平台以及用来创建本地运行库的编译器;而且还为它生成正确的C#绑定。这多么有趣啊。 - - - **你怎样看跨平台开发的未来趋势呢?** - - - 我们构建本地应用程序的方式已经改变了,我觉得在各种桌面操作系统之间存在差异,而且这种差异将会变得更加微妙;所以构建跨平台的应用程序将会更加容易,而且这种应用程序即使没有在本平台也可以完全兼容。不好的是,这可能意味着应用程序很难获得,并且当在操作系统上使用的时候功能得不到最好的发挥。我们知道怎样把库和工具以及运行环境的跨平台开发做的更好,但是跨平台应用程序的开发仍然需要我们的努力。 - - - -------------------------------------------------------------------------------- - - via: https://opensource.com/business/16/5/oscon-interview-andreia-gaita - - 作者:[Marcus D. Hanwell ][a] - 译者:[vim-kakali](https://github.com/vim-kakali) - 校对:[校对者ID](https://github.com/校对者ID) - - 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - - [a]: https://opensource.com/users/mhanwell - [1]: https://twitter.com/sh4na - [2]: http://conferences.oreilly.com/oscon/open-source-us/public/schedule/detail/48702 - [3]: http://www.mono-project.com/ - From c0675917ef1ed2037b5bd70ce251bdbbd3823ced Mon Sep 17 00:00:00 2001 From: wxy Date: Sun, 4 Sep 2016 18:06:02 +0800 Subject: [PATCH 0031/2437] PUB:Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB) @ChrisLeeGit --- ...leshoot Grand Unified Bootloader (GRUB).md | 61 +++++++++++-------- 1 file changed, 34 insertions(+), 27 deletions(-) rename {translated/tech => published}/LFCS/Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB).md (78%) diff --git a/translated/tech/LFCS/Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB).md b/published/LFCS/Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB).md similarity index 78% rename from translated/tech/LFCS/Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB).md rename to published/LFCS/Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB).md index d8b7b294e2..f8e7fb1834 100644 --- a/translated/tech/LFCS/Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB).md +++ b/published/LFCS/Part 13 - How to Configure and Troubleshoot Grand Unified Bootloader (GRUB).md @@ -1,43 +1,45 @@ LFCS 系列第十三讲:如何配置并排除 GNU 引导加载程序(GRUB)故障 ===================================================================================== -由于 LFCS 考试需求的变动已于 2016 年 2 月 2 日生效,因此我们向 [LFCS 系列][1] 添加了一些必要的话题。为了准备认证考试,我们也强烈推荐你去看 [LFCE 系列][2]。 +由于 LFCS 考试需求的变动已于 2016 年 2 月 2 日生效,因此我们向 [LFCS 系列][1] 添加了一些必要的话题。为了准备认证考试,我们也强烈推荐你去看看 [LFCE 系列][2]。 ![](http://www.tecmint.com/wp-content/uploads/2016/03/Configure-Troubleshoot-Grub-Boot-Loader.png) ->LFCS 系列第十三讲:配置并排除 Grub 引导加载程序故障。 -本文将会向你介绍 GRUB 的知识,并会说明你为什么需要一个引导加载程序,以及它是如何增强系统通用性的。 +*LFCS 系列第十三讲:配置并排除 Grub 引导加载程序故障。* -[Linux 引导过程][3] 是从你按下你的电脑电源键开始,直到你拥有一个全功能的系统为止,整个过程遵循着这样的高层次顺序: +本文将会向你介绍 GRUB 的知识,并会说明你为什么需要一个引导加载程序,以及它是如何给系统增加功能的。 + +[Linux 引导过程][3] 是从你按下你的电脑电源键开始,直到你拥有一个全功能的系统为止,整个过程遵循着这样的主要步骤: * 1. 一个叫做 **POST**(**上电自检**)的过程会对你的电脑硬件组件做全面的检查。 * 2. 当 **POST** 完成后,它会把控制权转交给引导加载程序,接下来引导加载程序会将 Linux 内核(以及 **initramfs**)加载到内存中并执行。 -* 3. 内核首先检查并访问硬件,然后运行初始进程(主要以它的通用名 **init** 而为人熟知),接下来初始进程会启动一些服务,最后完成系统启动过程。 +* 3. 内核首先检查并访问硬件,然后运行初始化进程(主要以它的通用名 **init** 而为人熟知),接下来初始化进程会启动一些服务,最后完成系统启动过程。 -在该系列的第七讲(“[SysVinit, Upstart, 和 Systemd][4]”)中,我们介绍了现代 Linux 发行版使用的一些服务管理系统和工具。在继续学习之前,你可能想要回顾一下那一讲的知识。 +在该系列的第七讲(“[SysVinit、Upstart 和 Systemd][4]”)中,我们介绍了现代 Linux 发行版使用的一些服务管理系统和工具。在继续学习之前,你可能想要回顾一下那一讲的知识。 ### GRUB 引导装载程序介绍 -在现代系统中,你会发现有两种主要的 **GRUB** 版本(一种是偶尔被成为 **GRUB Legacy** 的 **v1** 版本,另一种则是 **v2** 版本),虽说多数最新版本的发行版系统都默认使用了 **v2** 版本。如今,只有 **红帽企业版 Linux 6** 及其衍生系统仍在使用 **v1** 版本。 +在现代系统中,你会发现有两种主要的 **GRUB** 版本(一种是有时被称为 **GRUB Legacy** 的 **v1** 版本,另一种则是 **v2** 版本),虽说多数最新版本的发行版系统都默认使用了 **v2** 版本。如今,只有 **红帽企业版 Linux 6** 及其衍生系统仍在使用 **v1** 版本。 因此,在本指南中,我们将着重关注 **v2** 版本的功能。 不管 **GRUB** 的版本是什么,一个引导加载程序都允许用户: -* 1). 通过指定使用不同的内核来修改系统的表现方式; -* 2). 从多个操作系统中选择一个启动; -* 3). 添加或编辑配置节点来改变启动选项等。 +1. 通过指定使用不同的内核来修改系统的行为; +2. 从多个操作系统中选择一个启动; +3. 添加或编辑配置区块来改变启动选项等。 如今,**GNU** 项目负责维护 **GRUB**,并在它们的网站上提供了丰富的文档。当你在阅读这篇指南时,我们强烈建议你看下 [GNU 官方文档][6]。 当系统引导时,你会在主控制台看到如下的 **GRUB** 画面。最开始,你可以根据提示在多个内核版本中选择一个内核(默认情况下,系统将会使用最新的内核启动),并且可以进入 **GRUB** 命令行模式(使用 `c` 键),或者编辑启动项(按下 `e` 键)。 ![](http://www.tecmint.com/wp-content/uploads/2016/03/GRUB-Boot-Screen.png) -> GRUB 启动画面 -你会考虑使用一个旧版内核启动的原因之一是之前工作正常的某个硬件设备在一次升级后出现了“怪毛病(acting up)”(例如,你可以参考 AskUbuntu 论坛中的 [这条链接][7])。 +*GRUB 启动画面* -**GRUB v2** 的配置文件会在启动时从 `/boot/grub/grub.cfg` 或 `/boot/grub2/grub.cfg` 文件中读取,而 **GRUB v1** 使用的配置文件则来自 `/boot/grub/grub.conf` 或 `/boot/grub/menu.lst`。这些文件不能直接手动编辑,而是根据 `/etc/default/grub` 的内容和 `/etc/grub.d` 目录中的文件来修改的。 +你会考虑使用一个旧版内核启动的原因之一是之前工作正常的某个硬件设备在一次升级后出现了“怪毛病(acting up)”(例如,你可以参考 AskUbuntu 论坛中的[这条链接][7])。 + +在启动时会从 `/boot/grub/grub.cfg` 或 `/boot/grub2/grub.cfg` 文件中读取**GRUB v2** 的配置文件,而 **GRUB v1** 使用的配置文件则来自 `/boot/grub/grub.conf` 或 `/boot/grub/menu.lst`。这些文件**不应该**直接手动编辑,而应通过 `/etc/default/grub` 的内容和 `/etc/grub.d` 目录中的文件来更新。 在 **CentOS 7** 上,当系统最初完成安装后,会生成如下的配置文件: @@ -65,7 +67,7 @@ GRUB_DISABLE_RECOVERY="true" 使用上述命令,你会发现 `GRUB_TIMEOUT` 用于设置启动画面出现和系统自动开始启动(除非被用户中断)之间的时间。当该变量值为 `-1` 时,除非用户主动做出选择,否则不会开始启动。 -当同一台机器上安装了多个操作系统或内核后,`GRUB_DEFAULT` 就需要用一个整数来指定 GRUB 启动画面默认选择启动的操作系统或内核条目。我们既可以通过上述启动画查看启动条目列表,也可以使用下面的命令: +当同一台机器上安装了多个操作系统或内核后,`GRUB_DEFAULT` 就需要用一个整数来指定 GRUB 启动画面默认选择启动的操作系统或内核条目。我们既可以通过上述启动画面查看启动条目列表,也可以使用下面的命令: ### 在 CentOS 和 openSUSE 系统上 @@ -86,18 +88,20 @@ GRUB_DEFAULT=3 ``` ![](http://www.tecmint.com/wp-content/uploads/2016/03/Boot-System-with-Old-Kernel-Version.png) -> 使用旧版内核启动系统 -最后一个需要特别关注的 GRUB 配置变量是 `GRUB_CMDLINE_LINUX`,它是用来给内核传递选项的。我们可以在 [内核变量文件][8] 和 [man 7 bootparam][9] 中找到能够通过 GRUB 传递给内核的选项的详细文档。 +*使用旧版内核启动系统* + +最后一个需要特别关注的 GRUB 配置变量是 `GRUB_CMDLINE_LINUX`,它是用来给内核传递选项的。我们可以在 [内核变量文件][8] 和 [`man 7 bootparam`][9] 中找到能够通过 GRUB 传递给内核的选项的详细文档。 我的 **CentOS 7** 服务器上当前的选项是: ``` GRUB_CMDLINE_LINUX="vconsole.keymap=la-latin1 rd.lvm.lv=centos_centos7-2/swap crashkernel=auto vconsole.font=latarcyrheb-sun16 rd.lvm.lv=centos_centos7-2/root rhgb quiet" ``` -为什么你希望修改默认的内核参数或者传递额外的选项呢?简单来说,在很多情况下,你需要告诉内核某些由内核自身无法判断的硬件参数,或者是覆盖一些内核会检测的值。 -不久之前,就在我身上发生过这样的事情,当时我在自己已用了 10 年的老笔记本上尝试衍生自 **Slackware** 的 **Vector Linux**。完成安装后,内核并没有检测出我的显卡的正确配置,所以我不得不通过 GRUB 传递修改过的内核选项来让它工作。 +为什么你希望修改默认的内核参数或者传递额外的选项呢?简单来说,在很多情况下,你需要告诉内核某些由内核自身无法判断的硬件参数,或者是覆盖一些内核检测的值。 + +不久之前,就在我身上发生过这样的事情,当时我在自己已用了 10 年的老笔记本上尝试了衍生自 **Slackware** 的 **Vector Linux**。完成安装后,内核并没有检测出我的显卡的正确配置,所以我不得不通过 GRUB 传递修改过的内核选项来让它工作。 另外一个例子是当你需要将系统切换到单用户模式以执行维护工作时。为此,你可以直接在 `GRUB_CMDLINE_LINUX` 变量中直接追加 `single` 并重启即可: @@ -107,7 +111,7 @@ GRUB_CMDLINE_LINUX="vconsole.keymap=la-latin1 rd.lvm.lv=centos_centos7-2/swap cr 编辑完 `/etc/default/grub` 之后,你需要运行 `update-grub` (在 Ubuntu 上)或者 `grub2-mkconfig -o /boot/grub2/grub.cfg` (在 **CentOS** 和 **openSUSE** 上)命令来更新 `grub.cfg` 文件(否则,改动会在系统启动时丢失)。 -这条命令会处理早先提到的一些启动配置文件来更新 `grub.cfg` 文件。这种方法可以确保改动持久化,而在启动时刻通过 GRUB 传递的选项仅在当前会话期间有效。 +这条命令会处理早先提到的那些启动配置文件来更新 `grub.cfg` 文件。这种方法可以确保改动持久化,而在启动时刻通过 GRUB 传递的选项仅在当前会话期间有效。 ### 修复 Linux GRUB 问题 @@ -116,13 +120,14 @@ GRUB_CMDLINE_LINUX="vconsole.keymap=la-latin1 rd.lvm.lv=centos_centos7-2/swap cr 在启动画面中按下 `c` 键进入 GRUB 命令行模式(记住,你也可以按下 `e` 键编辑默认启动选项),并可以在 GRUB 提示中输入 `help` 命令获得可用命令: ![](http://www.tecmint.com/wp-content/uploads/2016/03/Fix-Grub-Issues-in-Linux.png) -> 修复 Linux 的 Grub 配置问题 -我们将会着重关注 **ls** 命令,它会列出已安装的设备和文件系统,并且我们将会看看它可以查找什么。在下面的图片中,我们可以看到有 4 块硬盘(`hd0` 到 `hd3`)。 +*修复 Linux 的 Grub 配置问题* + +我们将会着重关注 **ls** 命令,它会列出已安装的设备和文件系统,并且我们将会看看它查找到的东西。在下面的图片中,我们可以看到有 4 块硬盘(`hd0` 到 `hd3`)。 貌似只有 `hd0` 已经分区了(msdos1 和 msdos2 可以证明,这里的 1 和 2 是分区号,msdos 则是分区方案)。 -现在我们来看看能否在第一个分区 `hd0`(**msdos1**)上找到 GRUB。这种方法允许我们启动 Linux,并且使用高级工具修复配置文件或者如果有必要的话,干脆重新安装 GRUB: +现在我们来看看能否在第一个分区 `hd0`(**msdos1**)上找到 GRUB。这种方法允许我们启动 Linux,并且使用高级工具修复配置文件,或者如果有必要的话,干脆重新安装 GRUB: ``` # ls (hd0,msdos1)/ @@ -131,7 +136,8 @@ GRUB_CMDLINE_LINUX="vconsole.keymap=la-latin1 rd.lvm.lv=centos_centos7-2/swap cr 从高亮区域可以发现,`grub2` 目录就在这个分区: ![](http://www.tecmint.com/wp-content/uploads/2016/03/Find-Grub-Configuration.png) -> 查找 Grub 配置 + +*查找 Grub 配置* 一旦我们确信了 GRUB 位于 (**hd0, msdos1**),那就让我们告诉 GRUB 该去哪儿查找它的配置文件并指示它去尝试启动它的菜单: @@ -143,9 +149,10 @@ normal ``` ![](http://www.tecmint.com/wp-content/uploads/2016/03/Find-and-Launch-Grub-Menu.png) -> 查找并启动 Grub 菜单 -然后,在 GRUB 菜单中,选择一个条目并按下 **Enter** 键以使用它启动。一旦系统成功启动后,你就可以运行 `grub2-install /dev/sdX` 命令修复问题了(将 `sdX` 改成你想要安装 GRUB 的设备)。然后启动信息将会更新,并且所有相关文件都会得到恢复。 +*查找并启动 Grub 菜单* + +然后,在 GRUB 菜单中,选择一个条目并按下回车键以使用它启动。一旦系统成功启动后,你就可以运行 `grub2-install /dev/sdX` 命令修复问题了(将 `sdX` 改成你想要安装 GRUB 的设备)。然后启动信息将会更新,并且所有相关文件都会得到恢复。 ``` # grub2-install /dev/sdX @@ -167,7 +174,7 @@ via: http://www.tecmint.com/configure-and-troubleshoot-grub-boot-loader-linux/ 作者:[Gabriel Cánepa][a] 译者:[ChrisLeeGit](https://github.com/chrisleegit) -校对:[校对者ID](https://github.com/校对者ID) +校对:[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 @@ -175,7 +182,7 @@ via: http://www.tecmint.com/configure-and-troubleshoot-grub-boot-loader-linux/ [1]: http://www.tecmint.com/sed-command-to-create-edit-and-manipulate-files-in-linux/ [2]: http://www.tecmint.com/installing-network-services-and-configuring-services-at-system-boot/ [3]: http://www.tecmint.com/linux-boot-process/ -[4]: http://www.tecmint.com/linux-boot-process-and-manage-services/ +[4]: https://linux.cn/article-7365-1.html [5]: http://www.tecmint.com/best-linux-log-monitoring-and-management-tools/ [6]: http://www.gnu.org/software/grub/manual/ [7]: http://askubuntu.com/questions/82140/how-can-i-boot-with-an-older-kernel-version From 29bcf0a91e20177453054c2043e99af1555f777a Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 21:04:38 +0800 Subject: [PATCH 0032/2437] =?UTF-8?q?20160904-1=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...ith Apache Spark on YARN - Big Data 101.md | 245 ++++++++++++++++++ 1 file changed, 245 insertions(+) create mode 100644 sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md diff --git a/sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md b/sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md new file mode 100644 index 0000000000..0f42fc6fc9 --- /dev/null +++ b/sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md @@ -0,0 +1,245 @@ +A Raspberry Pi Hadoop Cluster with Apache Spark on YARN: Big Data 101 +====== + +Sometimes on DQYDJ we like to reveal a little bit of the process behind the data. In that past, that has meant articles on how we ingest fixed width data with R and how we model the cost of a Negative Income Tax in America, amongst many other fine examples. + +Today I’ll be sharing some explorations with Big Data. For a change of pace I’ll be doing it with the world’s most famous Little Computer – the Raspberry Pi. If that’s not your thing, we’ll see you in the next piece (probably with some already finished data). For the rest of you, read on… today we’re going to build a Raspberry Pi Hadoop Cluster! + +### I. Why Even Build a Raspberry Pi Hadoop Cluster? + +![](https://dqydj.com/wp-content/uploads/2016/08/IMG_9132-245x300.png) +>3/4 Angle of my Three Node Raspberry Pi Hadoop Cluster + +We do a lot of work with data here on DQYDJ, but nothing we have done qualifies as Big Data. + +Like so many controversial topics, the difference between “Big” Data and lower-case data might be best explained with a joke: + +>“If it fits in RAM, it’s not Big Data.” – Unknown (If you know, pass it along!) + +It seems like there are two solutions to the problem here: + +1. We can find a dataset big enough to overwhelm any actual physical or virtual RAM on our home computers. +2. Or, we can buy some computers where the data we have now would overwhelm the machine without special treatment. + +Enter the Raspberry Pi 2 Model B! + +These impressive little works of art and engineering have 1 GB of RAM and your choice of MicroSD card for a hard drive. Furthermore, you can get them for under $50 each, so you can build out a Hadoop cluster for under $250. + +There might be no cheaper ticket into the Big Data world! + + +### II. Building The Raspberry Pi Hadoop Cluster + +![](https://dqydj.com/wp-content/uploads/2016/08/IMG_9124-218x300.png) +>My favorite part of the build – the raw materials! + +Now I’ll link you to what I used to build the cluster itself. Note that if you do buy on Amazon using these links you’ll be supporting the site. (Thank you for the support!) + +- Raspberry Pi 2 Model B (x3) +- Pictures of ingredients for a Raspberry Pi Hadoop Cluster +- 4 Layer Acrylic Case/Stand +- 6 Post USB Charger (I picked the White RAVPower 50W 10A 6-Port USB Charger) +- 5 Port Ethernet Switch (Fast is fine, the ethernet ports on the Pi are only 100 Mbit) +- MicroSD Cards (This 5 pack of 32 GB cards was perfectly fine) +- Short MicroUSB Cables (to power the Pis) +- Short Ethernet Patch Cables +- Two sided foam tape (I had some 3M brand, it worked wonderfully) + +#### Now Bake the Pis! + +1. First, mount the three Raspberry Pis, one each to an acrylic panel (see the below image). +2. Next, mount the ethernet switch to a fourth acrylic panel with 2 sided foam tape. +3. Mount the USB charger on the acrylic panel which will become the ‘top’ (again with 2 sided foam tape) +4. Follow the instructions for the acrylic case and build the levels – I chose to put the Pis below the switch and charger (see the two completed build shots). + +Figure out a way to get the wires where you need them – if you bought the same USB cables and Ethernet cables as me, I was able to wrap them around the mounting posts between levels. + +Leave the Pis unplugged for now – you will burn the Raspbian OS on the MicroSDs before continuing. + +![](https://dqydj.com/wp-content/uploads/2016/08/IMG_9126.png) +>Raspberry Pis Mounted on Acrylic + +#### Burn Raspbian to the MicroSDs! + +Put Raspbian on 3 MicroSDs – follow the instructions on the [Raspberry Pi Raspbian page][1]. I use my Windows 7 PC for this, and [Win32DiskImager][2]. + +Pop one of the cards into whatever Pi you want to be the master node, then start it up by plugging in USB! + +#### Setup the Master + +There is no need for me to repeat an excellent guide here – you should [follow the instructions in Because We Can Geek’s guide][3] and install Hadoop 2.7.1. + +There were a few caveats with my setup – I’ll walk you through what I ended up using to get a working setup. Note that I am using the IP addresses 192.168.1.50 – 192.168.1.52, with the two slaves on .51 and .52 and the master on .50. Your network will vary, you will have to look around or discuss this step in the comments if you need help setting up static addresses on your network. + +Once you are done with the guide, you should enable a swapfile. Spark on YARN will be cutting it very close on RAM, so faking it with a swapfile is what you’ll need as you approach memory limits. + +(If you haven’t done this before, [this tutorial will get you there][4]. Keep swappiness at a low level – MicroSD cards don’t handle it well). + +Now I’m ready to share some subtle differences between my setup and Because We Can Geek’s. + +For starters, make sure you pick uniform names for all of your Pi nodes – in /etc/hostname, I used ‘RaspberryPiHadoopMaster’ for the master, and ‘RaspberryPiHadoopSlave#’ for the slaves. + +Here is my /etc/hosts on the Master: + +You will also need to edit a few configuration files in order to get Hadoop, YARN, and Spark playing nicely. (You may as well make these edits now.) + +Here is hdfs-site.xml: + +``` +yarn-site.xml (Note the changes in memory!): + +slaves: + +core-site.xml: +``` + +#### Setup the 2 Slaves: + +Again, [just follow the instructions on Because We Can Geek][5]. You will need to make minor edits to the files above – note that the master doesn’t change in yarn-site.xml, and you do not need to have a slaves file on the slaves themselves. + + + + +### III. Test YARN on Our Raspberry Pi Hadoop Cluster! + +![](https://dqydj.com/wp-content/uploads/2016/08/IMG_9130-235x300.png) +>The Raspberry Pi Hadoop Cluster Straight On! + +If everything is working, on the master you should be able to do a: + +>\> start-dfs.sh + +>\> start-yarn.sh + +And see everything come up! Do this from the Hadoop user, which will be ‘hduser’ if you obey the tutorials. + +Next, run ‘hdfs dfsadmin -report’ to see if all three datanodes have come up. Verify you have the bolded line in your report, ‘Live datanodes (3)’: + +``` +Configured Capacity: 93855559680 (87.41 GB) +Raspberry Pi Hadoop Cluster picture Straight On +Present Capacity: 65321992192 (60.84 GB) +DFS Remaining: 62206627840 (57.93 GB) +DFS Used: 3115364352 (2.90 GB) +DFS Used%: 4.77% +Under replicated blocks: 0 +Blocks with corrupt replicas: 0 +Missing blocks: 0 +Missing blocks (with replication factor 1): 0 +————————————————- +Live datanodes (3): +Name: 192.168.1.51:50010 (RaspberryPiHadoopSlave1) +Hostname: RaspberryPiHadoopSlave1 +Decommission Status : Normal + +… + +… + +… +``` + +You may now run through some ‘Hello, World!’ type exercises, or feel free to move to the next step! + + + +### IV. Installing Spark on YARN on Pi! + +YARN stands for Yet Another Resource Negotiator, and is included in the base Hadoop install as an easy to use resource manager. + +[Apache Spark][6] is another package in the Hadoop ecosystem – it’s an execution engine, much like the (in)famous and [bundled MapReduce][7]. At a general level, Spark benefits from in-memory storage over the disk-based storage of MapReduce. Some workloads will run at 10-100x the speed of a comparable MapReduce workload – after you’ve got your cluster built, read a bit on the difference between Spark and MapReduce. + +On a personal level, I was particularly impressed with the Spark offering because of the easy integration of two languages used quite often by Data Engineers and Scientists – Python and R. + +Installation of Apache Spark is very easy – in your home directory, ‘wget ’ ([from this page][8]). Then, ‘tar -xzf ’. Finally, copy the resulting directory to /opt and clean up any of the files you downloaded – that’s all there is to it! + +I also have a two line ‘spark-env.sh’ file, which is inside the ‘conf’ directory of Spark: + +>SPARK_MASTER_IP=192.168.1.50 + +>SPARK_WORKER_MEMORY=512m + +(I’m not even sure this is necessary, since we will be running on YARN) + + +### V. Hello, World! Finding an Interesting Data-set for Apache Spark! + + +The canonical ‘Hello, World!’ in the Hadoop world is to do some sort of word counting. + +I decided that I wanted my introduction to do something introspective… why not count my most commonly used words on this site? Perhaps some big data about myself would be useful…maybe? + +It’s a simple two step process to dump and sanitize your blog data, if you’re running on WordPress like we are. + +1. I used ‘[Export to Text][9]‘ to dump all of my content into a text file. +2. I wrote some quick Python to strip all the HTML using [the library bleach][10]: + +Now you’ll have a smaller file suitable for copying to HDFS on your Pi cluster! + +If you didn’t do this work on the master Pi, find a way to transfer it over (scp, rsync, etc.), and copy it to HDFS with a command like this: + +>hdfs dfs -copyFromLocal dqydj_stripped.txt /dqydj_stripped.txt + +You’re now ready for the final step – writing some code for Apache Spark! + + + +### VI: Lighting Apache Spark + +Cloudera had an excellent program to use as a base for our super-wordcount program, which you can find here. We’re going to modify it for our introspective Word Counting program. + +Install the package ‘stop-words’ for Python on your master Pi. Although interesting (I’ve used the word ‘the’ on DQYDJ 23,295 times!), you probably don’t want to see your stop words dominating the top of your data. Additionally, replace any references to my dqydj files in the following code with the path to your own special dataset: + +Save wordCount.py and make sure all of your paths are correct. + +Now you’re ready for the incantation that got Spark running on YARN for me, and (drumroll, please) revealed which words I use the most on DQYDJ: + +>/opt/spark-2.0.0-bin-hadoop2.7/bin/spark-submit –master yarn –executor-memory 512m –name wordcount –executor-cores 8 wordCount.py /dqydj_stripped.txt + + + + + +### VII. The Conclusion: Which Words Do I Use the Most on DQYDJ? + + +he top ten “non-stop words” on DQYDJ? “can, will, it’s, one, even, like, people, money, don’t, also“. + +Hey, not bad – money snuck into the top ten. That seems like a good thing to talk about on a website dedicated to finance, investing and economics – right? + +Here’s the rest of the top 50 most frequently used words by yours truly. Please use them to draw wild conclusions about the quality of the rest of my posts! + +![](https://dqydj.com/wp-content/uploads/2016/08/dqydj_pk_most_used_words.png) + +I hope you enjoyed this introduction to Hadoop, YARN, and Apache Spark – you’re now set to go off and write other applications on top of Spark. + +Your next step is to start to read through the documentation of Pyspark (and the libraries for other languages too, if you’re so inclined) to learn some of the functionality available to you. Depending on your interests and how your data is actually stored, you’ll have a number of paths to explore further – there are packages for streaming data, SQL, and even machine learning! + +What do you think? Are you going to build a Raspberry Pi Hadoop Cluster? Want to rent some time on mine? What’s the most surprising word you see up there, and why is it ‘S&P’? + + +-------------------------------------------------------------------------------- + +via: https://dqydj.com/raspberry-pi-hadoop-cluster-apache-spark-yarn/?utm_source=dbweekly&utm_medium=email + +作者:[PK][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://dqydj.com/about/#contact_us +[1]: https://www.raspberrypi.org/downloads/raspbian/ +[2]: https://sourceforge.net/projects/win32diskimager/ +[3]: http://www.becausewecangeek.com/building-a-raspberry-pi-hadoop-cluster-part-1/ +[4]: https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04 +[5]: http://www.becausewecangeek.com/building-a-raspberry-pi-hadoop-cluster-part-2/ +[6]: https://spark.apache.org/ +[7]: https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html +[8]: https://spark.apache.org/downloads.html +[9]: https://wordpress.org/support/plugin/export-to-text +[10]: https://pypi.python.org/pypi/bleach + + + + From ba35fad75d3d1dfede17c1f0b33bf4273882a3ad Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 21:43:39 +0800 Subject: [PATCH 0033/2437] =?UTF-8?q?20160904-2=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...nerally available in Azure SQL Database.md | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 sources/tech/20160824 JSON support is generally available in Azure SQL Database.md diff --git a/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md b/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md new file mode 100644 index 0000000000..13dd4cf0d3 --- /dev/null +++ b/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md @@ -0,0 +1,55 @@ +JSON support is generally available in Azure SQL Database +=========== + +We are happy to announce that you can now query and store both relational and textual data formatted in JavaScript Object Notation (JSON) using Azure SQL Database. Azure SQL Database provides simple built-in functions that read data from JSON text, transform JSON text into table, and format data from SQL tables as JSON. + +![](https://azurecomcdn.azureedge.net/mediahandler/acomblog/media/Default/blog/1cc536a5-a822-467b-a4a2-4557746f70cc.png) + + +You can use JSON functions that enable you to extract value from JSON text (JSON_VALUE), extract object from JSON (JSON_QUERY), update some value in JSON text (JSON_MODIFY), and verify that JSON text is properly formatted (ISJSON). OPENJSON function enables you to convert JSON text into a table structure. Finally, JSON functionalities enable you to easily format results of any SQL query as JSON text using the FOR JSON clause. + +### What can you do with JSON? + +JSON in Azure SQL Database enables you to build and exchange data with modern web, mobile, and HTM5/JavaScript single-page applications, NoSql stores such as Azure DocumentDB that contain data formatted as JSON, and to analyze logs and messages collected from different systems and services. Now you can easily integrate your Azure SQL Database with any service that uses JSON. + +#### Easily expose your data to modern frameworks and services + +Do you use services that exchange data in JSON format, such as REST services or Azure App Services? Do you have components or frameworks that use JSON, such as Angular JS, ReactJS, D3, or JQuery? With new JSON functionalities, you can easily format data stored in Azure SQL Database as JSON and expose it to any modern service or application. + +#### Easy ingestion of JSON data + +Are you working with mobile devices or sensors, services that produce JSON such as Azure Stream Analytics or Application Insight, or systems that store data in JSON format such as Azure DocumentDB or MongoDB? Do you need to query and analyze JSON data using well-known SQL language or tools that work with Azure SQL Database? Now, you can easily ingest JSON data and store it into Azure SQL Database, and use any language or tool that works with Azure SQL Database to query and analyze loaded information. + +#### Simplify your data models + +Do you need to store and query both relational and semi-structured data in your database? Do you need to simplify your data models like in NoSQL data platforms? Now you can combine structured relational data with schema-less data stored as JSON text in the same table. In Azure SQL Database you can use the best approaches both from relational and NoSQL worlds to tune your data model. Azure SQL Database enables you to query both relational and JSON data with the standard Transact-SQL language. Applications and tools would not see any difference between values taken from table columns and the values extracted from JSON text. + +### Next steps + +To learn how to integrate JSON in your application, check out our [Getting Started][1] page or [Channel 9 video][2]. To learn about various scenarios that show how to integrate JSON in your application, see demos in this Channel 9 video or find some scenario that might be interesting for your use case in these [JSON Blog posts][3]. + +Stay tuned because we will constantly add new JSON features and make JSON support even better. + + + +-------------------------------------------------------------------------------- + +via: https://azure.microsoft.com/en-us/blog/json-support-is-generally-available-in-azure-sql-database/?utm_source=dbweekly&utm_medium=email + +作者:[Jovan Popovic][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://azure.microsoft.com/en-us/blog/author/jovanpop/ +[1]: https://azure.microsoft.com/en-us/documentation/articles/sql-database-json-features/ +[2]: https://channel9.msdn.com/Shows/Data-Exposed/SQL-Server-2016-and-JSON-Support +[3]: http://blogs.msdn.com/b/sqlserverstorageengine/archive/tags/json/ + + + + + + + From 6213dd7015a9a2f7f502e94483706841f2d4ac76 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 21:55:39 +0800 Subject: [PATCH 0034/2437] =?UTF-8?q?20160904-3=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...rk Scale - A 60 TB production use case.md | 131 ++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 sources/tech/20160831 Apache Spark Scale - A 60 TB production use case.md diff --git a/sources/tech/20160831 Apache Spark Scale - A 60 TB production use case.md b/sources/tech/20160831 Apache Spark Scale - A 60 TB production use case.md new file mode 100644 index 0000000000..28fbd202ec --- /dev/null +++ b/sources/tech/20160831 Apache Spark Scale - A 60 TB production use case.md @@ -0,0 +1,131 @@ +Apache Spark @Scale: A 60 TB+ production use case +=========== + +Facebook often uses analytics for data-driven decision making. Over the past few years, user and product growth has pushed our analytics engines to operate on data sets in the tens of terabytes for a single query. Some of our batch analytics is executed through the venerable [Hive][1] platform (contributed to Apache Hive by Facebook in 2009) and [Corona][2], our custom MapReduce implementation. Facebook has also continued to grow its Presto footprint for ANSI-SQL queries against several internal data stores, including Hive. We support other types of analytics such as graph processing and machine learning ([Apache Giraph][3]) and streaming (e.g., [Puma][4], [Swift][5], and [Stylus][6]). + +While the sum of Facebook's offerings covers a broad spectrum of the analytics space, we continually interact with the open source community in order to share our experiences and also learn from others. [Apache Spark][7] was started by Matei Zaharia at UC-Berkeley's AMPLab in 2009 and was later contributed to Apache in 2013. It is currently one of the fastest-growing data processing platforms, due to its ability to support streaming, batch, imperative (RDD), declarative (SQL), graph, and machine learning use cases all within the same API and underlying compute engine. Spark can efficiently leverage larger amounts of memory, optimize code across entire pipelines, and reuse JVMs across tasks for better performance. Recently, we felt Spark had matured to the point where we could compare it with Hive for a number of batch-processing use cases. In the remainder of this article, we describe our experiences and lessons learned while scaling Spark to replace one of our Hive workload + +### Use case: Feature preparation for entity ranking + +Real-time entity ranking is used in a variety of ways at Facebook. For some of these online serving platforms raw feature values are generated offline with Hive and data loaded into its real-time affinity query system. The old Hive-based infrastructure built years ago was computationally resource intensive and challenging to maintain because the pipeline was sharded into hundreds of smaller Hive jobs. In order to enable fresher feature data and improve manageability, we took one of the existing pipelines and tried to migrate it to Spark. + +### Previous Hive implementation + +The Hive-based pipeline was composed of three logical stages where each stage corresponded to hundreds of smaller Hive jobs sharded by entity_id, since running large Hive jobs for each stage was less reliable and limited by the maximum number of tasks per job. + + +![](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xaf1/t39.2365-6/14050196_257613611304247_245043082_n.jpg) + +The three logical steps can be summarized as follows: + +1. Filter out non-production features and noise. +2. Aggregate on each (entity_id, target_id) pair. +3. Shard the table into N number of shards and pipe each shard through a custom binary to generate a custom index file for online querying. + +The Hive-based pipeline building the index took roughly three days to complete. It was also challenging to manage, because the pipeline contained hundreds of sharded jobs that made monitoring difficult. There was no easy way to gauge the overall progress of the pipeline or calculate an ETA. When considering the aforementioned limitations of the existing Hive pipeline, we decided to attempt to build a faster and more manageable pipeline with Spark. + +### Spark implementation + +Debugging at full scale can be slow, challenging, and resource intensive. We started off by converting the most resource intensive part of the Hive-based pipeline: stage two. We started with a sample of 50 GB of compressed input, then gradually scaled up to 300 GB, 1 TB, and then 20 TB. At each size increment, we resolved performance and stability issues, but experimenting with 20 TB is where we found our largest opportunity for improvement. + +While running on 20 TB of input, we discovered that we were generating too many output files (each sized around 100 MB) due to the large number of tasks. Three out of 10 hours of job runtime were spent moving files from the staging directory to the final directory in HDFS. Initially, we considered two options: Either improve batch renaming in HDFS to support our use case, or configure Spark to generate fewer output files (difficult due to the large number of tasks — 70,000 — in this stage). We stepped back from the problem and considered a third alternative. Since the tmp_table2 table we generate in step two of the pipeline is temporary and used only to store the pipeline's intermediate output, we were essentially compressing, serializing, and replicating three copies for a single read workload with terabytes of data. Instead, we went a step further: Remove the two temporary tables and combine all three Hive stages into a single Spark job that reads 60 TB of compressed data and performs a 90 TB shuffle and sort. The final Spark job is as follows: + + +![](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xfp1/t39.2365-6/14146896_1073876729364007_1912864323_n.jpg) + +### How did we scale Spark for this job? + +Of course, running a single Spark job for such a large pipeline didn't work on the first try, or even on the 10th try. As far as we know, this is the largest real-world Spark job attempted in terms of shuffle data size (Databrick's Petabyte sort was on synthetic data). It took numerous improvements and optimizations to the core Spark infrastructure and our application to get this job to run. The upside of this effort is that many of these improvements are applicable to other large-scale workloads for Spark, and we were able to contribute all our work back into the open source Apache Spark project — see the JIRAs for additional details. Below, we highlight the major improvements that enabled one of the entity ranking pipelines to be deployed into production. + +### Reliability fixes + +#### Dealing with frequent node reboots + +In order to reliably execute long-running jobs, we want the system to be fault-tolerant and recover from failures (mainly due to machine reboots that can occur due to normal maintenance or software errors). Although Spark is designed to tolerate machine reboots, we found various bugs/issues that needed to be addressed before it was robust enough to handle common failures. + +- Make PipedRDD robust to fetch failure (SPARK-13793): The previous implementation of PipedRDD was not robust enough to fetch failures that occur due to node reboots, and the job would fail whenever there was a fetch failure. We made change in the PipedRDD to handle fetch failure gracefully so the job can recover from these types of fetch failure. +- Configurable max number of fetch failures (SPARK-13369): With long-running jobs such as this one, probability of fetch failure due to a machine reboot increases significantly. The maximum allowed fetch failures per stage was hard-coded in Spark, and, as a result, the job used to fail when the max number was reached. We made a change to make it configurable and increased it from four to 20 for this use case, which made the job more robust against fetch failure. +- Less disruptive cluster restart: Long-running jobs should be able to survive a cluster restart so we don't waste all the processing completed so far. Spark's restartable shuffle service feature lets us preserve the shuffle files after node restart. On top of that, we implemented a feature in Spark driver to be able to pause scheduling of tasks so the jobs don't fail due to excessive task failure due to cluster restart. + +#### Other reliability fixes + +- Unresponsive driver (SPARK-13279): Spark driver was stuck due to O(N^2) operations while adding tasks, resulting in the job being stuck and killed eventually. We fixed the issue by removing the unnecessary O(N^2) operations. +- Excessive driver speculation: We discovered that the Spark driver was spending a lot of time in speculation when managing a large number of tasks. In the short term, we disabled speculation for this job. We are currently working on a change in the Spark driver to reduce speculation time in the long term. +- TimSort issue due to integer overflow for large buffer (SPARK-13850): We found that Spark's unsafe memory operation had a bug that leads to memory corruption in TimSort. Thanks to Databricks folks for fixing this issue, which enabled us to operate on large in-memory buffer. +- Tune the shuffle service to handle large number of connections: During the shuffle phase, we saw many executors timing out while trying to connect to the shuffle service. Increasing the number of Netty server threads (spark.shuffle.io.serverThreads) and backlog (spark.shuffle.io.backLog) resolved the issue. +- Fix Spark executor OOM (SPARK-13958) (deal maker): It was challenging to pack more than four reduce tasks per host at first. Spark executors were running out of memory because there was bug in the sorter that caused a pointer array to grow indefinitely. We fixed the issue by forcing the data to be spilled to disk when there is no more memory available for the pointer array to grow. As a result, now we can run 24 tasks/host without running out of memory. + +### Performance improvements + +After implementing the reliability improvements above, we were able to reliably run the Spark job. At this point, we shifted our efforts on performance-related projects to get the most out of Spark. We used Spark's metrics and several profilers to find some of the performance bottlenecks. + +#### Tools we used to find performance bottleneck + +- Spark UI Metrics: Spark UI provides great insight into where time is being spent in a particular phase. Each task's execution time is split into sub-phases that make it easier to find the bottleneck in the job. +- Jstack: Spark UI also provides an on-demand jstack function on an executor process that can be used to find hotspots in the code. +- Spark Linux Perf/Flame Graph support: Although the two tools above are very handy, they do not provide an aggregated view of CPU profiling for the job running across hundreds of machines at the same time. On a per-job basis, we added support for enabling Perf profiling (via libperfagent for Java symbols) and can customize the duration/frequency of sampling. The profiling samples are aggregated and displayed as a Flame Graph across the executors using our internal metrics collection framework. + +### Performance optimizations + +- Fix memory leak in the sorter (SPARK-14363) (30 percent speed-up): We found an issue when tasks were releasing all memory pages but the pointer array was not being released. As a result, large chunks of memory were unused and caused frequent spilling and executor OOMs. Our change now releases memory properly and enabled large sorts to run efficiently. We noticed a 30 percent CPU improvement after this change. +- Snappy optimization (SPARK-14277) (10 percent speed-up): A JNI method — (Snappy.ArrayCopy) — was being called for each row being read/written. We raised this issue, and the Snappy behavior was changed to use the non-JNI based System.ArrayCopy instead. This change alone provided around 10 percent CPU improvement. +- Reduce shuffle write latency (SPARK-5581) (up to 50 percent speed-up): On the map side, when writing shuffle data to disk, the map task was opening and closing the same file for each partition. We made a fix to avoid unnecessary open/close and observed a CPU improvement of up to 50 percent for jobs writing a very high number of shuffle partitions. +- Fix duplicate task run issue due to fetch failure (SPARK-14649): The Spark driver was resubmitting already running tasks when a fetch failure occurred, which led to poor performance. We fixed the issue by avoiding rerunning the running tasks, and we saw the job was more stable when fetch failures occurred. +- Configurable buffer size for PipedRDD (SPARK-14542) (10 percent speed-up): While using a PipedRDD, we found out that the default buffer size for transferring the data from the sorter to the piped process was too small and our job was spending more than 10 percent of time in copying the data. We made the buffer size configurable to avoid this bottleneck. +- Cache index files for shuffle fetch speed-up (SPARK-15074): We observed that the shuffle service often becomes the bottleneck, and the reducers spend 10 percent to 15 percent of time waiting to fetch map data. Digging deeper into the issue, we found out that the shuffle service is opening/closing the shuffle index file for each shuffle fetch. We made a change to cache the index information so that we can avoid file open/close and reuse the index information for subsequent fetches. This change reduced the total shuffle fetch time by 50 percent. +- Reduce update frequency of shuffle bytes written metrics (SPARK-15569) (up to 20 percent speed-up): Using the Spark Linux Perf integration, we found that around 20 percent of the CPU time was being spent probing and updating the shuffle bytes written metrics. +- Configurable initial buffer size for Sorter (SPARK-15958) (up to 5 percent speed-up): The default initial buffer size for the Sorter is too small (4 KB), and we found that it is very small for large workloads — and as a result we waste a significant amount of time expending the buffer and copying the contents. We made a change to make the buffer size configurable, and with large buffer size of 64 MB we could avoid significant data copying, making the job around 5 percent faster. +- Configuring number of tasks: Since our input size is 60 T and each HDFS block size is 256 M, we were spawning more than 250,000 tasks for the job. Although we were able to run the Spark job with such a high number of tasks, we found that there is significant performance degradation when the number of tasks is too high. We introduced a configuration parameter to make the map input size configurable, so we can reduce that number by 8x by setting the input split size to 2 GB. + +After all these reliability and performance improvements, we are pleased to report that we built and deployed a faster and more manageable pipeline for one of our entity ranking systems, and we provided the ability for other similar jobs to run in Spark. + +### Spark pipeline vs. Hive pipeline performance comparison + +We used the following performance metrics to compare the Spark pipeline against the Hive pipeline. Please note that these numbers aren't a direct comparison of Spark to Hive at the query or job level, but rather a comparison of building an optimized pipeline with a flexible compute engine (e.g., Spark) instead of a compute engine that operates only at the query/job level (e.g., Hive). + +CPU time: This is the CPU usage from the perspective of the OS. For example, if you have a job that is running only one process on a 32-core machine using 50 percent of all CPU for 10 seconds, then your CPU time would be 32 * 0.5 * 10 = 160 CPU seconds. + +![](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xpt1/t39.2365-6/14146892_595234533986285_2004398348_n.jpg) + +CPU reservation time: This is the CPU reservation from the perspective of the resource management framework. For example, if we reserve a 32-core machine for 10 seconds to run the job, the CPU reservation time is 32 * 10 = 320 CPU seconds. The ratio of CPU time to CPU reservation time reflects how well are we utilizing the reserved CPU resources on the cluster. When accurate, the reservation time provides a better comparison between execution engines when running the same workloads when compared with CPU time. For example, if a process requires 1 CPU second to run but must reserve 100 CPU seconds, it is less efficient by this metric than a process that requires 10 CPU seconds but reserves only 10 CPU seconds to do the same amount of work. We also compute the memory reservation time but do not include it here, since the numbers were similar to the CPU reservation time due to running experiments on the same hardware, and that in both the Spark and Hive cases we do not cache data in memory. Spark has the ability to cache data in memory, but due to our cluster memory limitations we decided to work out-of-core, similar to Hive. + +![](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xfa1/t39.2365-6/14129680_325754934432503_513809233_n.jpg) + +Latency: End-to-end elapsed time of the job. + +![](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xap1/t39.2365-6/14129681_178723715883876_1030939470_n.jpg) + +### Conclusion and future work + +Facebook uses performant and scalable analytics to assist in product development. Apache Spark offers the unique ability to unify various analytics use cases into a single API and efficient compute engine. We challenged Spark to replace a pipeline that decomposed to hundreds of Hive jobs into a single Spark job. Through a series of performance and reliability improvements, we were able to scale Spark to handle one of our entity ranking data processing use cases in production. In this particular use case, we showed that Spark could reliably shuffle and sort 90 TB+ intermediate data and run 250,000 tasks in a single job. The Spark-based pipeline produced significant performance improvements (4.5-6x CPU, 3-4x resource reservation, and ~5x latency) compared with the old Hive-based pipeline, and it has been running in production for several months. + +While this post details our most challenging use case for Spark, a growing number of customer teams have deployed Spark workloads into production. Performance, maintainability, and flexibility are the strengths that continue to drive more use cases to Spark. Facebook is excited to be a part of the Spark open source community and will work together to develop Spark toward its full potential. + + + + + + + + +-------------------------------------------------------------------------------- + +via: https://code.facebook.com/posts/1671373793181703/apache-spark-scale-a-60-tb-production-use-case/?utm_source=dbweekly&utm_medium=email + +作者:[Sital Kedia][a] [王硕杰][b] [Avery Ching][c] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.facebook.com/sitalkedia +[b]: https://www.facebook.com/shuojiew +[c]: https://code.facebook.com/posts/1671373793181703/apache-spark-scale-a-60-tb-production-use-case/?utm_source=dbweekly&utm_medium=email# +[1]: https://code.facebook.com/posts/370832626374903/even-faster-data-at-the-speed-of-presto-orc/ +[2]: https://www.facebook.com/notes/facebook-engineering/under-the-hood-scheduling-mapreduce-jobs-more-efficiently-with-corona/10151142560538920/ +[3]: https://code.facebook.com/posts/509727595776839/scaling-apache-giraph-to-a-trillion-edges/ +[4]: https://research.facebook.com/publications/realtime-data-processing-at-facebook/ +[5]: https://research.facebook.com/publications/realtime-data-processing-at-facebook/ +[6]: https://research.facebook.com/publications/realtime-data-processing-at-facebook/ +[7]: http://spark.apache.org/ + From 18c774337f35c3e69d22e60a0c35679df54ed88e Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 22:04:34 +0800 Subject: [PATCH 0035/2437] =?UTF-8?q?20160904-4=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...20160830 Spark comparison - AWS vs. GCP.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 sources/tech/20160830 Spark comparison - AWS vs. GCP.md diff --git a/sources/tech/20160830 Spark comparison - AWS vs. GCP.md b/sources/tech/20160830 Spark comparison - AWS vs. GCP.md new file mode 100644 index 0000000000..acb3c23367 --- /dev/null +++ b/sources/tech/20160830 Spark comparison - AWS vs. GCP.md @@ -0,0 +1,82 @@ +Spark comparison: AWS vs. GCP +=========== + + +>Tianhui Michael Li and Ariel M’ndange-Pfupfu will lead a hands-on online course Oct 10, 12, and 14, 2016: Distributed Computing with Spark for Beginners. Instruction includes building functioning applications from end-to-end and mastering critical tooling around Spark. + +There’s little doubt that cloud computing will play an important role in data science for the foreseeable future. The flexible, scalable, on-demand computing power available is an important resource, and as a result, there’s a lot of competition between the providers of this service. Two of the biggest players in the space are [Amazon Web Services (AWS)][1] and [Google Cloud Platform (GCP)][2]. + +This article includes a short comparison of distributed Spark workloads in AWS and GCP—both in terms of setup time and operating cost. We ran this experiment with our students at The Data Incubator, [a big data training organization][3] that helps companies hire top-notch data scientists and train their employees on the latest data science skills. Even with the efficiencies built into Spark, the cost and time of distributed workloads can be substantial, and we are always looking for the most efficient technologies so our students are learning the best and fastest tools. + +### Submitting Spark jobs to the cloud + +Spark is a popular distributed computation engine that incorporates MapReduce-like aggregations into a more flexible, abstract framework. There are APIs for Python and Java, but writing applications in Spark’s native Scala is preferable. That makes job submission simple, as you can package your application and all its dependencies into one JAR file. + + +It’s common to use Spark in conjunction with HDFS for distributed data storage, and YARN for cluster management; this makes Spark a perfect fit for AWS’s Elastic MapReduce (EMR) clusters and GCP’s Dataproc clusters. Both EMR and Dataproc clusters have HDFS and YARN preconfigured, with no extra work required. + +### Configuring cloud services + +Managing data, clusters, and jobs from the command line is more scalable than using the web interface. For AWS, this means installing and using the command-line interface (cli). You’ll have to set up your credentials beforehand as well as make a separate keypair for the EC2 instances that are used under the hood. You’ll also need to set up roles—basically permissions—for both users (making sure they have sufficient rights) and EMR itself (usually running aws emr create-default-roles in the cli is good enough to get started). + +For GCP the process is more straightforward. If you install the Google Cloud SDK and sign in with your Google account, you should be able to do most things right off the bat. The thing to remember here is to enable the relevant APIs in the API Manager: Compute Engine, Dataproc, and Cloud Storage JSON. + +Once you have things set up to your liking, the fun part begins! Using commands like aws s3 cp or gsutil cp you can copy your data into the cloud. Once you have buckets set up for your inputs, outputs, and anything else you might need, running your app is as easy as starting up a cluster and submitting the JAR file. Make sure you know where the logs are kept—it can be tricky to track down problems or bugs in a cloud environment. + +### You get what you pay for + +When it comes to cost, Google’s service is more affordable in several ways. First, the raw cost of purchasing computing power is cheaper. Running a Google Compute Engine machine with 4 vCPUs and 15 GB of RAM will run you $0.20 every hour, or $0.24 with Dataproc. An identically-specced AWS instance will cost you $0.336 per hour running EMR. + +The second factor to consider is the granularity of the billing. AWS charges by the hour, so you pay the full rate even if your job takes 15 minutes. GCP charges by the minute, with a 10-minute minimum charge. This ends up being a huge difference in cost in a lot of use cases. + +Both services have various other discounts. You can effectively bid on spare cloud capacity with AWS’s spot instances or GCP’s preemptible instances. These will be cheaper than dedicated, on-demand instances, but they’re not guaranteed to be available. Discounted rates are available on GCP if your instances live for long periods of time (25% to 100% of the month). On AWS, paying some of the costs upfront or buying in bulk can save you some money. The bottom line is, if you’re a power user and you use cloud computing on a regular or even constant basis, you’ll need to delve deeper and perform your own calculations. + +Lastly, the costs for new users wanting to try out these services are lower for GCP. They offer a 60-day free trial with $300 in credit to use however you want. AWS only offers a free tier where certain services are free to a certain point or discounted, so you will end up paying to run Spark jobs.This means that if you want to test out Spark for the first time, you’ll have more freedom to do what you want on GCP without worrying about price. + +### Performance comparison + +We set up a trial to compare the performance and cost of a typical Spark workload. The trial used clusters with one master and five core instances of AWS’s m3.xlarge and GCP’s n1-standard-4. They differ slightly in specification, but the number of virtual cores and amount of memory is the same. In fact, they behaved almost identically when it came to job execution time. + +The job itself involved parsing, filtering, joining, and aggregating data from the publicly available Stack Exchange Data Dump. We ran the same JAR on a ~50M subset of the data (Cross Validated) and then on the full ~9.5G data set. + +![](https://d3ansictanv2wj.cloudfront.net/1400_img_1_AWS_GCP-25ed6069029112a8439d89999796be18.jpg) +>Figure 1. Credit: Michael Li and Ariel M'ndange-Pfupfu. + +![](https://d3ansictanv2wj.cloudfront.net/1400_img_2_AWS_GCP-448714718896b21e32f8b47d4657fc8c.jpg) +>Figure 2. Credit: Michael Li and Ariel M'ndange-Pfupfu. + +The short job clearly benefited from GCP’s by-the-minute billing, being charged only for 10 minutes of cluster time, whereas AWS charged for a full hour. But even the longer job was cheaper on GPS both because of fractional-hour billing and a lower per-unit time cost for comparable performance. It’s also worth noting that storage costs weren’t included in this comparison. + +### Conclusion + +AWS was the first mover in the space, and this shows in the API. Its ecosystem is vast, but its permissions model is a little dated, and its configuration is a little arcane. By contrast, Google is the shiny new entrant in this space and has polished off some of the rough edges. It is missing some features on our wishlist, like an easy way to auto-terminate clusters and detailed billing information broken down by job. Also, for managing tasks programmatically in Python, the API client library isn’t as full-featured as AWS’s Boto. + +If you’re new to cloud computing, GCP is easier to get up and running, and the credits make it a tempting platform. Even if you are already used to AWS, you may still find the cost savings make switching worth it, although the switching costs may not make moving to GCP worth it. + +Ultimately, it’s difficult to make sweeping statements about these services because they’re not just one entity; they’re entire ecosystems of integrated parts, and both have pros and cons. The real winners are the users. As an example, at The Data Incubator, our Ph.D. data science fellows really appreciate the cost reduction as they learn about distributed workloads. And while our big data corporate training clients may be less price sensitive, they appreciate being able to crunch enterprise data faster, while holding price constant. Data scientists can now enjoy the multitude of options available, and the benefits of having a competitive cloud computing market. + + + + + + + + + + + +-------------------------------------------------------------------------------- + +via: https://www.oreilly.com/ideas/spark-comparison-aws-vs-gcp?utm_source=dbweekly&utm_medium=email + +作者:[Michael Li][a] [Ariel M'Ndange-Pfupfu][b] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.oreilly.com/people/76a5b-michael-li +[b]: https://www.oreilly.com/people/Ariel-Mndange-Pfupfu +[1]: https://aws.amazon.com/ +[2]: https://cloud.google.com/ +[3]: https://www.thedataincubator.com/training.html?utm_source=OReilly&utm_medium=blog&utm_campaign=AWSvsGCP From ecd467d1d48590d617b7a6b7d40cd312c9e67cb3 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 22:15:04 +0800 Subject: [PATCH 0036/2437] =?UTF-8?q?20160904-5=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @wxy 文中3个动态图怎么处理? --- ...tate Of JavaScript - JavaScript Flavors.md | 104 ++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md diff --git a/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md b/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md new file mode 100644 index 0000000000..4354869d2a --- /dev/null +++ b/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md @@ -0,0 +1,104 @@ +The State Of JavaScript: JavaScript Flavors +=========== + + +One thing that distinguishes JavaScript from other programming languages is that JavaScript isn’t just one language: it’s actually more like a family of closely related cousins. + +What started with CoffeeScript back in 2009 has become an explosion of choice over the past couple years: ES6, TypeScript, Elm… they all have their strengths, and they all compile down to good old JavaScript. + +So after last week’s look at front-end frameworks, let’s look at what the State Of JavaScript survey can tell us about JavaScript Flavors. + +Disclaimer: these are preliminary results extracted from a partial dataset. They’re just a way for me to share some insights while I take my time to come up with the best way to present the complete results. + +Note: if you haven’t taken the survey yet, now would be the perfect time to do it! It’ll only take 10 minutes and you can come back here after :) + +### Awareness + +First, I wanted to find out the percentage of respondents that were aware of each of the six options’ existence: + +- Good Old Plain JavaScript: 97% +- ES6: 98% +- CoffeeScript: 99% +- TypeScript: 98% +- Elm: 66% +- ClojureScript: 77% + +You would expect that “Good Old Plain JavaScript” would score 100% awareness, but I imagine some people couldn’t resist the temptation of picking “I’ve never heard of JavaScript” in a JavaScript survey… + +ES6, CoffeeScript, and TypeScript all have near-perfect awareness, which surprised me since TypeScript isn’t quite as widespread as the other two. + +Elm and ClojureScript on the other hand have much lower scores, which makes sense since they’re more tied to their own ecosystems, and harder to use in existing apps. + +### Interest + +Next, let’s look at which flavors have been generating the most interest among developers who haven’t used them yet: + +![](https://embed.chartblocks.com/1.0/?c=57c4dc599973d2525fee820a&referrer=https%3A%2F%2Fmedium.com%2Fmedia%2F63092db0523a37d9d33ce6c014d727f6%3FmaxWidth%3D700&t=3efc9491eba2ce2#) + +Remember we’re looking at non-users here, so it makes sense that there would be very few people who haven’t used Plain JavaScript. + +It’s interesting to look at ES6: a large proportion of developers have already jumped on the bandwagon, and almost all (89%) of those who haven’t yet want to learn it as well. + +TypeScript and Elm are in the same boat: not many people have used them, but they have 53% and 58% interest scores respectively, which isn’t bad by any means. + +If I had to guess, I’d say that both TypeScript and Elm might be having a hard time articulating their advantages to the average JavaScript developer. After all it’s hard to understand the advantages of something like static typing if all you know is JavaScript. + +Also, few developers have used CoffeeScript, and apparently almost nobody wants to learn it. There goes my plan to write a 12-volume CoffeeScript Encyclopedia… + +### Satisfaction + +We now come to the key question: how many developers have used each specific flavor, and how many would use it again? + +![](https://embed.chartblocks.com/1.0/?c=57c4e5f79973d29461ee820a&referrer=https%3A%2F%2Fmedium.com%2Fmedia%2F1fe4bbdf807f87883fa108e31c6927d5%3FmaxWidth%3D700&t=1061d2ab8fc9838#) + + +While plain JavaScript has larger usage numbers as expected, in terms of satisfaction the big winner here is ES6, and I think it’s safe to say it’s now the default way to write JavaScript apps. +TypeScript and Elm both also have similarly high satisfaction percentages, around 85%. And once more, poor CoffeeScript trails the poll with only 17% of developers willing to consider it again. + +### Happiness + +Finally, I asked people how happy they were with their current way of writing JavaScript: + +![](https://embed.chartblocks.com/1.0/?c=57c4cd8c9973d2d95bee820a&referrer=https%3A%2F%2Fmedium.com%2Fmedia%2F538138b2d91d1fa99f1696bef4dd4d3f%3FmaxWidth%3D700&t=f53efb029ea4456#) + + +The high scores we saw in the previous question are confirmed here: with an average score of 3.96 overall, people are really happy with JavaScript as a language. + +It’s hard to say if this is because of JavaScript’s recent improvements, or because maybe (just maybe) JavaScript isn’t as horrible a language as people make it to be. But it’s certainly comforting. + +### Conclusions + +If React and Vue were the clear winners last time, I would say that here it’s without a doubt ES6. This is not groundbreaking news by any means, but it’s nice to know the community is embracing the direction the language is taking. + +It will be really interesting to ask these questions again a year or so from now, and see if TypeScript, Elm, and ClojureScript have made any progress. + +Personally, I suspect this explosion of flavors is just the beginning, and that the way we write JavaScript a couple years from now might be quite different! + +### Share the Word & Stay Tuned + +When it comes to surveys like this one, more data equals better data! The more people take the survey, the more representative of the overall JavaScript community it will be. + +So if you can, I encourage you to share the survey: + +[On Twitter][1] + +[On Facebook][2] + +And if you’d like to know next time I publish results, [head to the survey homepage][3] and leave your email there to be notified! + + +-------------------------------------------------------------------------------- + +via: https://www.oreilly.com/ideas/spark-comparison-aws-vs-gcp?utm_source=dbweekly&utm_medium=email + +作者:[Michael Li][a] [Ariel M'Ndange-Pfupfu][b] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.oreilly.com/people/76a5b-michael-li +[b]: https://www.oreilly.com/people/Ariel-Mndange-Pfupfu +[1]: https://aws.amazon.com/ +[2]: https://cloud.google.com/ +[3]: https://www.thedataincubator.com/training.html?utm_source=OReilly&utm_medium=blog&utm_campaign=AWSvsGCP From 5cc7515c876927b1deaaadb032bb202e6d192a6d Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 22:17:47 +0800 Subject: [PATCH 0037/2437] 20160904-5 --- ...30 The State Of JavaScript - JavaScript Flavors.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md b/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md index 4354869d2a..a455843983 100644 --- a/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md +++ b/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md @@ -91,14 +91,13 @@ And if you’d like to know next time I publish results, [head to the survey hom via: https://www.oreilly.com/ideas/spark-comparison-aws-vs-gcp?utm_source=dbweekly&utm_medium=email -作者:[Michael Li][a] [Ariel M'Ndange-Pfupfu][b] +作者:[Sacha Greif][a] 译者:[译者ID](https://github.com/译者ID) 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 -[a]: https://www.oreilly.com/people/76a5b-michael-li -[b]: https://www.oreilly.com/people/Ariel-Mndange-Pfupfu -[1]: https://aws.amazon.com/ -[2]: https://cloud.google.com/ -[3]: https://www.thedataincubator.com/training.html?utm_source=OReilly&utm_medium=blog&utm_campaign=AWSvsGCP +[a]: https://medium.com/@sachagreif +[1]: https://twitter.com/intent/tweet/?text=The%20State%20Of%20JavaScript%3A%20take%20a%20short%20survey%20about%20popular%20JavaScript%20technologies%20http%3A%2F%2Fstateofjs.com%20%23stateofjs +[2]: https://facebook.com/sharer/sharer.php?u=http%3A%2F%2Fstateofjs.com +[3]: http://stateofjs.com/ From 4dcd1734a2df6ad9f06fe23cb0316a47ab9d3bc2 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 22:27:38 +0800 Subject: [PATCH 0038/2437] =?UTF-8?q?20160904-6=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...Creates a Language in 200 Lines of Code.md | 263 ++++++++++++++++++ 1 file changed, 263 insertions(+) create mode 100644 sources/tech/20160830 Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code.md diff --git a/sources/tech/20160830 Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code.md b/sources/tech/20160830 Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code.md new file mode 100644 index 0000000000..67c117ee0b --- /dev/null +++ b/sources/tech/20160830 Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code.md @@ -0,0 +1,263 @@ +Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code +=========== + +Parsers are an incredibly useful software libraries. While conceptually simple, they can be challenging to implement and are often considered a dark art in computer science. In this blog series, I’ll show you why you don’t need to be Harry Potter to master parsers. But bring your wand just in case. + +We’ll explore a new open source Javascript library called Ohm that makes building parsers easy and easier to reuse. In this series, we use Ohm to recognize numbers, build a calculator, and more. By the end of this series you will have created a complete programming language in under 200 lines of code. This powerful tool will let you do things that you might have thought impossible otherwise. + +### Why Parsers are Hard + +Parsers are useful. There are lots of times you might need a parser. A new file format might come along that you need to process and no one else has written a library for it yet. Or maybe you find files in an old file format and the existing parsers aren’t built for the platform you need. I’ve seen this happen over and over. Code will come and go but data is forever. + +Fundamentally parsers are simple: just transform one data structure into another. So why does it feel like you need to be Dumbledore to figure them out? + +The challenge that parsers have historically been surprisingly difficult to write, and most of the existing tools are old and assume a fair amount of arcane computer science knowledge. If you took a compilers class in college the textbook may well have techniques from the 1970s. Fortunately, parser technology has improved a great deal since then. + +Typically a parser is created by defining what you want to parse using a special syntax called a formal grammar. Then you feed this into several tools like Bison and Yacc which generate a bunch of C code that you the need to modify or link into whatever programming language you are actually writing in. The other option is to manually write a parser in your preferred language, which is slow and error prone. That’s a lot of extra work before you get to actually use the parser. + +Imagine if your description of the thing you wanted to parse, the grammar, was also the parser? What if you could just run the grammar directly, then add hooks only where you want? That’s what Ohm does. + +### Introducing Ohm + +[Ohm][1] is a new kind of parsing system. While it resembles the grammars you may have seen in text books it’s a lot more powerful and easier to use. With Ohm you write your format definition in a very flexible syntax in a .ohm file, then attach semantic meaning to it using your host language. For this blog we will use JavaScript as the host language. + +Ohm is based on years of research into making parsers easier and more flexible. VPRI’s [STEPS program][2] (pdf) created many custom languages for specific tasks (like a fully parallelizable graphics renderer in 400 lines of code!) using Ohm’s precursor [OMeta][3]. + +Ohm has many interesting features and notations, but rather than explain them all I think we should just dive in and build something. + +### Parsing Integers + +Let’s parse some numbers. It seems like this would be easy. Just look for adjacent digits in a string of text; but let’s try to handle all forms of numbers: Integers and floating point. Hex and octal. Scientific notation. A leading negative digit. Parsing numbers is easy. Doing it right is hard + +To build this code by hand would be difficult and buggy, with lots of special cases which sometimes conflict with each other. A regular expression could probably do it, but would be ugly and hard to maintain. Let’s do it with Ohm instead. + +Every parser in Ohm involves three parts: the grammar, the semantics, and the tests. I usually pick part of the problem and write tests for it, then build enough of the grammar and semantics to make the tests pass. Then I pick another part of the problem, add more tests, update the grammar and semantics, while making sure all of the tests continue to pass. Even with our new powerful tool, writing parsers is still conceptually complicated. Tests are the only way to build parsers in a reasonable manner. Now let’s dig in. + +We’ll start with an integer number. An integer is composed of a sequences of digits next to each other. Let’s put this into a file called grammar.ohm: + +``` +CoolNums { + // just a basic integer + Number = digit+ +} +``` + +This creates a single rule called Number which matches one or more digits. The + means one or more, just like in a regular expression. This rule will match if there is one digit or more than one digit. It won’t match if there are zero digits or something something other than a digit. A digit is defined as the characters for the numbers 0 to 9. digit is also a rule like Number is, but it’s one of Ohm’s built in rules so we don’t have to define it ourselves. We could override if it we wanted to but that wouldn’t make sense in this case. After all we don’t plan to invent a new form of number (yet!) + +Now we can read in this grammar and process it with the Ohm library. + +Put this into test1.js + +``` +var ohm = require('ohm-js'); +var fs = require('fs'); +var assert = require('assert'); +var grammar = ohm.grammar(fs.readFileSync('src/blog_numbers/syntax1.ohm').toString()); +``` + +The ohm.grammar call will read in the file and parse it into a grammar object. Now we can add semantics. Add this to your Javascript file: + +``` +var sem = grammar.createSemantics().addOperation('toJS', { + Number: function(a) { + return parseInt(this.sourceString,10); + } +}); +``` + +This creates a set of semantics called sem with the operation toJS. The semantics are essentially a bunch of functions matching up to each rule in the grammar. Each function will be called when the corresponding rule in the grammar is parsed. The Number function above will be called when the Number rule in the grammar is parsed. The grammar defines what chunks are in the language. The semantics define what to do with chunks once they’ve been parsed. + +Our semantics functions can do anything we want, such as print debugging information, create objects, or recursively call toJS on any sub-nodes. In this case we just want to convert the matched text into a real Javascript integer. + +All semantic functions have an implicit this object with some useful properties. The source property represents the part of the input text that matches this node. this.sourceString is the matched input as a string. Calling the built in JavaScript function parseInt turns this string to a number. The 10 argument to parseInt tells JavaScript that we are giving it a number in base ten. If we leave it out then JS will assume it’s base 10 anyway, but I’ve included it because later on we will support base 16 (hex) numbers, so it’s good to be explicit. + +Now that we have some semantics, let’s actually parse something to see if our parser works. How do we know our parser works? By testing it. Lots and lots of testing. Every possible edge case needs a test. + +With the standard assert API, here is a test function which matches some input then applies our semantics to it to turn it into a number, then compares the number with the expected input. +``` + function test(input, answer) { + var match = grammar.match(input); + if(match.failed()) return console.log("input failed to match " + input + match.message); + var result = sem(match).toJS(); + assert.deepEqual(result,answer); + console.log('success = ', result, answer); + } +``` + +That’s it. Now we can write a bunch of tests for different numbers. If the match fails then our script will throw an exception. If not it will print success. Let’s try it out. Add this to the script + +``` + test("123",123); + test("999",999); + test("abc",999); +``` + +Then run the script with node test1.js + +Your output should look like this: + +``` +success = 123 123 +success = 999 999 +input failed to match abcLine 1, col 1: +> 1 | abc + ^ +Expected a digit +``` + +Cool. The first two succeed and the third one fails, as it should. Even better, Ohm automatically gave us a nice error message pointing to the match failure. + +### Floating Point + +Our parser works, but it doesn’t do anything very interesting. Let’s extend it to parse both integers and floating point numbers. Change the grammar.ohm file to look like this: + +``` +CoolNums { + // just a basic integer + Number = float | int + int = digit+ + float = digit+ "." digit+ +} +``` + +This changes the Number rule to point to either a float or an int. The | means or. We read this as “a Number is composed of a float or an int.” Then int is defined as digit+ and float is defined as digit+ followed by a period followed by another digit+. This means there must be at least one digit before the period and at least one after. If there is not a period then it won’t be a float at all, so int will match instead. + +Now let’s go look at our semantic actions again. Since we now have new rules we need new action functions: one for int and one for float. + +``` +var sem = grammar.createSemantics().addOperation('toJS', { + Number: function(a) { + return a.toJS(); + }, + int: function(a) { + console.log("doing int", this.sourceString); + return parseInt(this.sourceString,10); + }, + float: function(a,b,c) { + console.log("doing float", this.sourceString); + return parseFloat(this.sourceString); + } +}); +``` + +There’s two things to note here. First, int and float and Number all have matching grammar rules and functions. However, the action for Number no longer does anything interesting. It receives the child node ‘a’ and returns the result of toJS on the child. In other words the Number rule simply returns whatever its child rule matched. Since this is the default behavior of any rule in Ohm we can actually just leave the Number action out. Ohm will do it for us. + +Second, int has one argument a while float has three: a, b, and c. This is because of the rule’s arity. Arity means how many arguments a rule has. If we look back at the grammar, the rule for float is + +``` + float = digit+ "." digit+ +``` + +The float rule is defined by three parts: the first digit+, the "." and the second digit+. All three of those parts will be passed as parameters to the action function for float. Thus float must have three arguments or else the Ohm library will give us an error. In this case we don’t care about the arguments because we will just grab the input string directly, but we still need the arguments listed to avoid compiler errors. Later on we will actually use some of these parameters. + +Now we can add a few more tests for our new floating point number support. + +``` +test("123",123); +test("999",999); +//test("abc",999); +test('123.456',123.456); +test('0.123',0.123); +test('.123',0.123); +``` + +Note that the last test will fail. A floating point number must begin with a digit, even if it’s just zero. .123 is not valid, and in fact the real JavaScript language has the same rule. + +### Hexadecimal + +So now we have integers and floats but there’s a few other number syntaxes that might be good to support: hexadecimal and scientific notation. Hex numbers are integers in base sixteen. The digits can be from 0 to 9 and A to F. Hex is often used in computer science when working with binary data because you can represent 0 through 255 exactly with only two digits. + +In most C derived programming languages (including JavaScript) hex numbers are preceded by `0x` to indicate to the compiler that what follows is a hexadecimal number. To support hex numbers in our parser we just need to add another rule. + +``` + Number = hex | float | int + int = digit+ + float = digit+ "." digit+ + hex = "0x" hexDigit+ + hexDigit := "0".."9" | "a".."f" | "A".."F" +``` + +I’ve actually added two rules. `hex` says that a hex number is the string `0x` followed by one or more `hexDigits`. A `hexDigit` is any letter from 0 to 9, or a to f, or A to F (covering both upper and lower case). I also modified Number to recognize hex as another possible option. Now we just need another action rule for hex. + +``` + hex: function(a,b) { + return parseInt(this.sourceString,16); + } +``` + +Notice that in this case we are passing `16` as the radix to `parseInt` because we want JavaScript to know that this is a hexadecimal number. + +I skipped over something important to notice. The rule for `hexDigit` looks like this. + +``` + hexDigit := "0".."9" | "a".."f" | "A".."F" +``` + +Notice that I used `:=` instead of `=`. In Ohm, the `:=` is used when you are overriding a rule. It turns out Ohm already has a default rule for `hexDigit`, just as it does for `digit`, `space` and a bunch of others. If I had used = then Ohm would have reported an error. This is a check so I can’t override a rule unintentionally. Since our new hexDigit rule is actually the same as Ohm’s built in rule we can just comment it out and let Ohm do it. I left the rule in just so we can see what’s really going on. + +Now we can add some more tests and see that our hex digits really work: + +``` +test('0x456',0x456); +test('0xFF',255); +``` + +### Scientific Notation + +Finally let’s support scientific notation. This is for very large or small numbers like 1.8 x 10^3 In most programming languages numbers in scientific notation would be written as 1.8e3 for 18000 or 1.8e-3 for .0018. Let’s add another couple of rules to support this exponent notation. + +``` + float = digit+ "." digit+ exp? + exp = "e" "-"? digit+ +``` + +This adds a the exp rule to the end of the float rule with a question mark. The ? means zero or one, so exp is optional but there can’t be more than one. Adding the exp rule also changes the arity of the float rule, so we need to add another argument to the float action, even if we don’t use it. + +``` + float: function(a,b,c,d) { + console.log("doing float", this.sourceString); + return parseFloat(this.sourceString); + }, +``` + +And now our new tests can pass: + +``` +test('4.8e10',4.8e10); +test('4.8e-10',4.8e-10); +``` + +### Conclusion + +Ohm is a great tool for building parsers because it’s easy to get started and you can incrementally add to it. It also has other great features that I didn’t cover today, like a debugging visualizer and sub-classing. + +So far we have used Ohm to translate character strings into JavaScript numbers, and often Ohm is used for this very purpose: converting one representation to another. However, Ohm can be used for a lot more. By putting in a different set of semantic actions you can use Ohm to actually process and calculate things. This is one of Ohm’s magic features. A single grammar can be used with many different semantics. + +In the next article of this series I’ll show you how to not just parse numbers but actually evaluate math expressions like `(4.8+5 * (238-68)/2)`, just like a real calculator. + +Bonus challenge: Can you extend the grammar with support for octal numbers? These are numbers in base 8 and can be represented with only the digits 0 to 7, preceded by a zero and the letter o. See if you are right with these test cases. Next time I’ll show you the answer. + +``` +test('0o77',7*8+7); +test('0o23',0o23); +``` + + + + +-------------------------------------------------------------------------------- + +via: https://www.pubnub.com/blog/2016-08-30-javascript-parser-ohm-makes-creating-a-programming-language-easy/?utm_source=javascriptweekly&utm_medium=email + +作者:[Josh Marinacci][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.pubnub.com/blog/author/josh/ +[1]: https://github.com/cdglabs/ohm +[2]: http://www.vpri.org/pdf/tr2012001_steps.pdf +[3]: http://tinlizzie.org/ometa/ + + From ebe4fd31eac524638040d3782d7f987eb5f33f93 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sun, 4 Sep 2016 22:36:03 +0800 Subject: [PATCH 0039/2437] =?UTF-8?q?20160904-7=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Twitter - efficiency and optimization.md | 128 ++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md diff --git a/sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md b/sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md new file mode 100644 index 0000000000..53ad32416f --- /dev/null +++ b/sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md @@ -0,0 +1,128 @@ +The infrastructure behind Twitter: efficiency and optimization +=========== + + +In the past, we've published details about Finagle, Manhattan, and the summary of how we re-architected the site to be able to handle events like Castle in the Sky, the Super Bowl, 2014 World Cup, the global New Year's Eve celebration, among others. In this infrastructure series, we're focusing on the core infrastructure and components that run Twitter. We're also going to focus each blog on efforts surrounding scalability, reliability, and efficiency in a way that highlights the history of our infrastructure, challenges we've faced, lessons learned, upgrades made, and where we're heading. + +### Data center efficiency + +#### History + +Twitter hardware and data centers are at the scale few technology companies ever reach. However, this was not accomplished without a few missteps along the way. Our uptime has matured through a combination of physical improvements and software-based changes. + +During the period when the fail whale was prevalent, outages occurred due to software limitations, as well as physical failures at the hardware or infrastructure level. Failure domains existed in various definitions which had to be aggregated to determine the risk and required redundancy for services. As the business scaled in customers, services, media content, and global presence, the strategy evolved to efficiently and resiliently support the service. + +#### Challenges + +Software dependencies on bare metal were further dependant on our data centers' ability to operate and maintain uptime of power, fiber connectivity, and environment. These discrete physical failure domains had to be reviewed against the services distributed on the hardware to provide for fault tolerance. + +The initial decision of which data center service provider to scale with was done when specialization in site selection, operation, and design was in its infancy. We began in a hosted provider then migrated to a colocation facility as we scaled. Early service interruptions occurred as result of equipment failures, data center design issues, maintenance issues, and human error. As a result, we continually iterated on the physical layer designs to increase the resiliency of the hardware and the data center operations. + +The physical reasons for service interruptions were inclusive of hardware failures at the server component level, top of rack switch, and core switches. For example, during the initial evaluation of our customized servers, the hardware team determined the cost of the second power supply was not warranted given the low rate of failure of server power supplies — so they were removed from the design. The data center power topology provides redundancy through separate physical whips to the racks and requires the second power supply. Removal of the second power supply eliminated the redundant power path, leaving the hardware vulnerable to impact during distribution faults in the power system. To mitigate the impact of the single power supply, ATS units were required to be added at the rack level to allow a secondary path for power. + +The layering of systems with diverse fiber paths, power sources, and physical domains continued to separate services from impacts at relatively small scale interruptions, thus improving resiliency. + +#### Lessons learned and major technology upgrades, migrations, and adoptions + +We learned to model dependencies between the physical failure domains, (i.e. building power and cooling, hardware, fiber) and the services distributed across them to better predict fault tolerance and drive improvements. + +We added additional data centers providing regional diversity to mitigate risk from natural disaster and the ability to fail between regions when it was needed during major upgrades, deploys or incidents. The active-active operation of data centers provided for staged code deployment reducing overall impacts of code rollouts. + +The efficiency of power use by the data centers has improved with expanding the operating ranges of the environmental envelope and designing the hardware for resiliency at the higher operating temperatures. + +#### Future work + +Our data centers continue to evolve in strategy and operation, providing for live changes to the operating network and hardware without interruption to the users. Our strategy will continue to focus on scale within the existing power and physical footprints through optimization and maintaining flexibility while driving efficiency in the coming years. + +### Hardware efficiency + +#### History and challenges + +Our hardware engineering team was started to qualify and validate performance of off-the-shelf purchased hardware, and evolved into customization of hardware for cost and performance optimizations. + +Procuring and consuming hardware at Twitter's scale comes with a unique set of challenges. In order to meet the demands of our internal customers, we initially started a program to qualify and ensure the quality of purchased hardware. The team was primarily focused on performance and reliability testing ensuring that systems could meet the demands. Running systematic tests to validate the behavior was predictable, and there were very few bugs introduced. + +As we scaled our major workloads (Mesos, Hadoop, Manhattan, and MySQL) it became apparent the available market offerings didn't quite meet the needs. Off-the-shelf servers come with enterprise features, like raid controllers and hot swap power supplies. These components improve reliability at small scale, but often decrease performance and increase cost; for example some raid controllers interfered with the performance of SSDs and could be a third of the cost of the system. + +At the time, we were a large user of mysql databases. Issues arose from both supply and performance of SAS media. The majority of deployments were 1u servers, and the total number of drives used plus a writeback cache could predict the performance of a system often time limited to a sustained 2000 sequential IOPS. In order to continue scaling this workload, we were stranding CPU cores and disk capacity to meet IOPS requirement. We were unable to find cost-effective solutions at this time. + +As our volume of hardware reached a critical mass, it made sense to invest in a hardware engineering team for customized white box solutions with focus on reducing the capital expenses and increased performance metrics. + +#### Major technology changes and adoption + +We've made many transitions in our hardware technology stack. Below is a timeline for adoptions of new technology and internally developed platforms. + +- 2012 - SSDs become the primary storage media for our MySQL and key/value databases. +- 2013 - Our first custom solution for Hadoop workloads is developed, and becomes our primary bulk storage solution. +- 2013 - Our custom solution is developed for Mesos, TFE, and cache workloads. +- 2014 - Our custom SSD key/value server completes development. +- 2015 - Our custom database solution is developed. +- 2016 - We developed GPU systems for inference and training of machine learning models. + +#### Lessons learned + +The objective of our Hardware Engineering team is to significantly reduce the capital expenditure and operating expenditure by making small tradeoffs that improve our TCO. Two generalizations can apply to reduce the cost of a server: + +1. Removing the unused components +2. Improving utilization + +Twitter's workload is divided into four main verticals: storage, compute, database, and gpu. Twitter defines requirements on a per vertical basis, allowing Hardware Engineering to produce a focused feature set for each. This approach allows us to optimize component selection where the equipment may go unused or underutilized. For example, our storage configuration has been designed specifically for Hadoop workloads and was delivered at a TCO reduction of 20% over the original OEM solution. At the same time, the design improved both the performance and reliability of the hardware. Similarly, for our compute vertical, the Hardware Engineering Team has improved the efficiency of these systems by removing unnecessary features. + +There is a minimum overhead required to operate a server, and we quickly reached a point where it could no longer remove components to reduce cost. In the compute vertical specifically, we decided the best approach was to look at solutions that replaced multiple nodes with a single node, and rely on Aurora/Mesos to manage the capacity. We settled on a design that replaced two of our previous generation compute nodes with a single node. + +Our design verification began with a series of rough benchmarks, and then progressed to a series of production load tests confirming a scaling factor of 2. Most of this improvement came from simply increasing the thread count of the CPU, but our testing confirmed a 20-50% improvement in our per thread performance. Additionally we saw a 25% increase in our per thread power efficiency, due to sharing the overhead of the server across more threads. + +For the initial deployment, our monitoring showed a 1.5 replacement factor, which was well below the design goal. An examination of the performance data revealed there was a flawed assumption in the workload characteristics, and that it needed to be identified. + +Our Hardware Engineering Team's initial action was to develop a model to predict the packing efficiency of the current Aurora job set into various hardware configurations. This model correctly predicted the scaling factor we were observing in the fleet, and suggested we were stranding cores due to unforeseen storage requirements. Additionally, the model predicted we would see a still improved scaling factor by changing the memory configuration as well. + +Hardware configuration changes take time to implement, so Hardware Engineering identified a few large jobs and worked with our SRE teams to adjust the scheduling requirements to reduce the storage needs. These changes were quick to deploy, and resulted in an immediate improvement to a 1.85 scaling factor. + +In order to address the situation permanently, we needed to adjust to configuration of the server. Simply expanding the installed memory and disk capacity resulted in a 20% improvement in the CPU core utilization, at a minimal cost increase. Hardware Engineering worked with our manufacturing partners to adjust the bill of materials for the initial shipments of these servers. Follow up observations confirmed a 2.4 scaling factor exceeding the target design. + +### Migration from bare metal to mesos + +Until 2012, running a service inside Twitter required hardware requisitions. Service owners had to find out and request the particular model or class of server, worry about your rack diversity, maintain scripts to deploy code, and manage dead hardware. There was essentially no "service discovery." When a web service needed to talk to the user service, it typically loaded up a YAML file containing all of the host IPs and ports of the user service and the service used that list (port reservations were tracked in a wiki page). As hardware died or was added, managing required editing and committing changes to the YAML file that would go out with the next deploy. Making changes in the caching tier meant many deploys over hours and days, adding a few hosts at a time and deploying in stages. Dealing with cache inconsistencies during the deploy was a common occurrence, since some hosts would be using the new list and some the old. It was possible to have a host running old code (because the box was temporarily down during the deploy) resulting in a flaky behavior with the site. + +In 2012/2013, two things started to get adopted at Twitter: service discovery (via a zookeeper cluster and a library in the core module of Finagle) and Mesos (including our own scheduler framework on top of Mesos called Aurora, now an Apache project). + +Service discovery no longer required static YAML host lists. A service either self-registered on startup or was automatically registered under mesos into a "serverset" (which is just a path to a list of znodes in zookeeper based on the role, environment, and service name). Any service that needed to talk to that service would just watch that path and get a live view of what servers were out there. + +With Mesos/Aurora, instead of having a script (we were heavy users of Capistrano) that took a list of hosts, pushed binaries around and orchestrated a rolling restart, a service owner pushed the package into a service called "packer" (which is a service backed by HDFS), uploaded an aurora configuration that described the service (how many CPUs it needed, how much memory, how many instances needed, the command lines of all the tasks each instance should run) and Aurora would complete the deploy. It schedules instances on an available hosts, downloads the artifact from packer, registers it in service discovery, and launches it. If there are any failures (hardware dies, network fails, etc), Mesos/Aurora automatically reschedules the instance on another host. + +#### Twitter's Private PaaS + +Mesos/Aurora and Service Discovery in combination were revolutionary. There were many bugs and growing pains over the next few years and many hard lessons learned about distributed systems, but the fundamental design was sound. In the old world, the teams were constantly dealing with and thinking about hardware and its management. In the new world, the engineers only have to think about how best to configure their services and how much capacity to deploy. We were also able to radically improve the CPU utilization of Twitter's fleet over time, since generally each service that got their own bare metal hardware didn't fully utilize its resources and did a poor job of managing capacity. Mesos allows us to pack multiple services into a box without having to think about it, and adding capacity to a service is only requesting quota, changing one line of a config, and doing a deploy. + +Within two years, most "stateless" services moved into Mesos. Some of the most important and largest services (including our user service and our ads serving system) were among the first to move. Being the largest, they saw the biggest benefit to their operational burden. This allowed them to reduce their operational burden. + +We are continuously looking for ways to improve the efficiency and optimization of the infrastructure. As part of this, we regularly benchmark against public cloud providers and offerings to validate our TCO and performance expectations of the infrastructure. We also have a good presence in public cloud, and will continue to utilize the public cloud when it's the best available option. The next series of this post will mainly focus on the scale of our infrastructure. + +Special thanks to Jennifer Fraser, David Barr, Geoff Papilion, Matt Singer, and Lam Dong for all their contributions to this blog post. + + + + + +-------------------------------------------------------------------------------- + +via: https://blog.twitter.com/2016/the-infrastructure-behind-twitter-efficiency-and-optimization?utm_source=webopsweekly&utm_medium=email + +作者:[mazdakh][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://twitter.com/intent/user?screen_name=mazdakh +[1]: https://twitter.com/jenniferfraser +[2]: https://twitter.com/davebarr +[3]: https://twitter.com/gpapilion +[4]: https://twitter.com/lamdong + + + + + + + From 4fd07c7616446efc4a6aec22483810149786e83a Mon Sep 17 00:00:00 2001 From: geekpi Date: Mon, 5 Sep 2016 15:11:57 +0800 Subject: [PATCH 0040/2437] translating --- ...JSON support is generally available in Azure SQL Database.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md b/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md index 13dd4cf0d3..3a3b8416fe 100644 --- a/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md +++ b/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md @@ -1,3 +1,5 @@ +translating---geekpi + JSON support is generally available in Azure SQL Database =========== From 1f45f93d0fa5f354b4f4b4ffe764254331cf6061 Mon Sep 17 00:00:00 2001 From: Tom Date: Mon, 5 Sep 2016 15:44:39 +0800 Subject: [PATCH 0041/2437] Translated By JianhuanZhuo --- ... Android Dev And What I Learned From It.md | 61 ------------------- ... Android Dev And What I Learned From It.md | 60 ++++++++++++++++++ 2 files changed, 60 insertions(+), 61 deletions(-) delete mode 100644 sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md create mode 100644 translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md diff --git a/sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md b/sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md deleted file mode 100644 index 7f8b4a109c..0000000000 --- a/sources/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md +++ /dev/null @@ -1,61 +0,0 @@ -JianhuanZhuo - -How I Got to be an Android Dev And What I Learned From It -============================= - -They say all relationships go through a rough patch at two, seven, then ten years. I don't remember who said it, but someone told me that many years ago. - -Next week will be my moving-to-Sydney second anniversary, so I figured this is a good time to write this post. - -During I/O last May, I met one of the coolest ladies ever, Yasmine. She asked me how I got into Android development, and when I was done telling her she said I should blog about it. So here it is, Yasmine. Better late than never. ;) - -### In the beginning... - -If there's one thing you should know about me, it's that I find it very hard to make decisions. Who's your best friend? What's your favourite food? What should you name your stuffed panda? I don't know the answer to these things. So imagine 16-year-old me, about to graduate high school, and I had Zero Idea what I wanted to major in. The first university I applied to? I wrote down what major I was applying for right in front of the registrar, literally before handing her my application form (Business Economics). - -I ended up going to another school, majoring in Electronics and Communications Engineering. I had one computer programming subject in freshman year. And I hated it. I hated it with a passion. I couldn't figure how anything works, and I swore I would never do that again. - -My first job after uni was with Intel as a product engineer. I worked there for two years. Lived in the middle of nowhere, worked long hours. But I thought that's par for the course; part of being an adult is working hard, right? And then the semiconductor industry in the Philippines started flipping out. A lot of other factories closed down, some of the products we used to look after were being transferred to other sites. I decided I'd rather look for another job now, than being retrenched and not knowing how long I will be jobless for. - -### What Now? - -I wanted a job back in the city, and I kinda don't want to stay in a sinking industry. But then again, there is nothing else I know how to do. Yeah, I am a licensed engineer, so technically I could work for a telco, or a TV station even! But at that time, if you want to for a telco, you'd have a better chance of getting hired if you interned with them right out of uni. And I didn't, so that's out. There were a lot of job postings for software developers though. But I hated programming! I don't know how to do it! - -And this is when my first lucky break came. I am so fortunate that I met a manager who trusted in me. I was upfront with her, I don't know shit. I would have to learn on the job, so it would be a slow start. Needless to say, I learned a lot in that job. I worked on some pretty cool stuff (we made apps installed in SIM cards), and met a lot of really nice people. But more importantly, it kickstarted my foray into software development. - -I eventually worked on more enterprise-y stuff (boring). Until the time we ran out of projects. I mean I'm all for coming in puttering around the office doing nothing and getting paid for it. But after two days it turns out it kinda sucks. It was 2009 and I keep on hearing about this new OS from Google called Android and that the SDK is Out Now! and that You Should Try It Out. So I installed all the things and started Android-ing. - -### Things Get Interesting - -So now that I have built a shiny, new Hello World app that runs on an emulator, I took that as a sign that I have the creds to apply for an Android development job. I joined a start up, and again, I was upfront about it -- I don't know how to do this, I have just been playing around, but if you want to pay me to play around, then we can be friends. And so I was met with another lucky break. - -It was an encouraging time to be a dev. The Android Dev StackOverflow community was much smaller, we are all learning at the same time, and honestly, I think everyone was kinder and more forgiving+. - -I eventually worked for a company whose mobile team is distributed across offices in Manila, Sydney, and New York. I was the first Android developer in the Manila office, but by then I was so used to it that I didn't mind. - -It was there that I met the guy who would eventually refer me to Domain, and for that I am forever grateful to him. Domain has done so much for me, both personally and professionally. I work with a really talented team, and I have never seen a company love a product so much. Domain made my dream of attending IO a reality, and through my work with them I got to work on a lot of pretty sweet features that I never dreamed of++. Another lucky break, and I mean to make the most out of it. - -### And? - -I guess what I mean to say is that I have just been winging it all these years. But at least I'm honest about it, right? It there's anything I have learned, it is that there is nothing wrong with saying "I don't know". There are times when we need to pretend to know things, but there are a lot more times when we need to accept that we do not know things. - -Do not be afraid to try something new, no matter how scared it makes you feel. Easier said than done, I know. But sometimes it really helps to take a deep breath, close your eyes, and just jump+++. Lundagin mo, baby! - - - -`+` I was looking at my old StackOverflow questions, and I seriously think that if I asked them today, I would have a lot of "What are you, stupid?" comments. Or maybe I'm just old and cynical. I don't know. The point is, we have all been there, so be kind to another, okay? -`++` This merits a post all on its own -`+++` I distinctly remember that's how I applied to my first Android job. I wrote my cover letter, proof read it, hovered my mouse over the Send button, took a deep breath, then clicked it before I could change my mind. - - --------------------------------------------------------------------------------- - -via: http://www.zdominguez.com/2016/08/winging-it-how-i-got-to-be-android-dev.html?utm_source=Android+Weekly&utm_campaign=9314c56ae3-Android_Weekly_219&utm_medium=email&utm_term=0_4eb677ad19-9314c56ae3-338075705 - -作者:[Zarah Dominguez ][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://plus.google.com/102371834744366149197 diff --git a/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md b/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md new file mode 100644 index 0000000000..b5f9b6f2a0 --- /dev/null +++ b/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md @@ -0,0 +1,60 @@ +JianhuanZhuo + +我的安卓开发者之路以及我在其中学到了什么 +============================= + +别人都说所有的关系都是需要经历2年、七年甚至十年的磨砺的。我忘了是谁说的这句话,但肯定有人在几年前这么跟我说过。 + +下一周是我来悉尼两周年的日子,所以我想当前正是我写这篇文章的好时候。 + +去年五月份参加 I/O年会的时候,我遇到了一位十分漂亮的女士,亚斯曼。她向我请教我是如何成长为一名安卓开发者的,当我跟她说起我的经历时,她认为我应该写个博客记下来。所以亚斯曼,如你所愿,虽然迟了点,但好过没做。;) + +### 故事的开始 + +如果有件事你需要了解我的,那就是我发现自己有选择困难症。你最好的朋友是谁?你最喜欢的食物是什么?你应该给你的玩具熊猫命名吗?我连这些问题都不知道该怎么回答才好。所以你可以想象得到,16岁高中毕业的我对于专业选择根本就没有任何想法。那我最初申请的大学是?在交表给注册员前,我在她面前逐字掂量着写下这个打算申请的专业(商业经济学)。 + +可我最后去了另外一间学校,就读电子与通信工程。大一时我有一门计算机编程课程。但我很讨厌编程,十分的讨厌。关于编程的一切我都一无所知。我曾发誓再也不会干这种事了。 + +我大学毕业后的第一份工作是在因特尔做产品工程师并在那呆了两年。我生活上餐不继下餐,整天长时间工作,但这在我意料之中,身为成年人是该努力工作的,难道不是吗?可之后菲律宾的半导体行业开始呈现颓势,大批工厂纷纷倒闭,我们以前常常光顾的一些产品被收购转到了另一个网站。我便决定去找另一份工作而不是等着被裁员,因为被裁员后我都不知道自己会失业多久。 + +### 现在呢? + +我想留在城市里找到一份工作,但我不想呆在正在下沉的半导体行业里了。但话说回来,我又不知道该做什么好。对了,我可是拿了毕业证书的工程师,所以技术上来说我可以在电信运营商或电视台找到工作。可这种时候,如果你想入职电信运营商,应该在大学毕业之际去他们那实习,你就能有一个好的录用渠道,可惜我没有,所以我放弃了这个想法。虽然有是很多软件开发人员的招聘信息,但我讨厌编程,所以我真的不知道怎么做才好。 + +接下来是我第一个幸运的机遇,我很幸运地遇到了信任我的上司,我也和她坦诚了我什么都不会。之后我不得不学习以胜任这份工作,一开始这个过程很漫长。无需多言,我在这份工作上学到了很多,也结识了很多很好的人,与我一起是一群很厉害的同事(我们曾开发出安装在 SIM 卡上的 APP)。但更重要的是我开始踏上了软件开发的征途。 + +最后我做得更多是一些公司的琐事(十分无聊)直到项目完结。换句话说,我总在是在办公室里闲逛并坐等发薪。之后我发现这确实是在浪费时间,2009 年的时候我对谷歌的安卓新系统有耳闻了,现在它的 SDK 上线了!于是我安装了所有相关软件并着手安卓开发。 + +### 事情变得有趣了 + +所以现在我能够构建了一个在运行在仿真器的 Hello World 应用,在我看来意味着我有胜任安卓开发工作的能力。我又一次加入了一个创业公司,并在里面做前端开发 -- 我不知道该怎么做,对此我手足无措,但如果你为我的手足无措买单的话,我们就可以做朋友了,所以我很幸运遇到另一个机遇。 + +那时成为开发者是一件令人欣喜的事。安卓开发的 StackOverflow 社区非常小,我们都在相互交流学习,说真的,我认为里面的所有人都很友好、很豪迈 +。 + +我最后去了一家企业,这家企业的移动开发团队在马尼拉、悉尼、纽约都设有办公地点。而我是马尼拉办公地点的第一个安卓开发人员,但那时我很习惯,并没有在意。 + +在那里我认识了最后令我永远感激的引荐我参与 Domain 项目的人。Domain 项目不管在个人或职业上对我来说都意味深重。我和一支很有才华的团队一起共事,也从没见过一个公司能如此执着于一款产品。Domain 让我实现了参加 IO 年会的梦。与他们共事后我懂得了很多之前没想到的可爱特性 ++。另一个幸运的机遇,我是说最大限度地利用它。 + +### 然后呢? + +我想说的是虽然这些年都在飘荡,但至少我很诚实,对吧?如上就是我所学到的全部东西。说一句「我不懂」没什么可怕的。有时候我们是该装懂,但大部分时间我们需要坦诚地接受这样一个事实:我们还不懂。 + +别害怕尝试新事物,不管它让你感觉多害怕。我知道说比做简单。但总有一些东西能让你鼓起勇气动手去尝试的 +++。Lundagin mo, baby!(LCTT 译者注:一首歌名) + + +`+` 我查找过我以前提问的 StackOverflow 问题,认真地想想如果现在我问他们这些,估计会收到很多「你是谁啊,傻瓜」的评论。可能是我老了,我也不知道。但关键是,我们曾经在那个社区里彼此很友好,是吗? +`++` 这一点写在另一篇文章里了。 +`+++` 我还清晰地记得申请第一份安卓开发职位的情形:我写完求职信后又通读了一遍,提交前鼠标在发送按钮上不断徘徊,深呼吸之后我趁改变主意之前把它发出去了。 + + +-------------------------------------------------------------------------------- + +via: http://www.zdominguez.com/2016/08/winging-it-how-i-got-to-be-android-dev.html?utm_source=Android+Weekly&utm_campaign=9314c56ae3-Android_Weekly_219&utm_medium=email&utm_term=0_4eb677ad19-9314c56ae3-338075705 + +作者:[Zarah Dominguez ][a] +译者:[JianhuanZhuo](https://github.com/JianhuanZhuo) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://plus.google.com/102371834744366149197 From da25b432084a746c0fe2ac6f8d97b6a6cc92551b Mon Sep 17 00:00:00 2001 From: Tom Date: Mon, 5 Sep 2016 16:11:59 +0800 Subject: [PATCH 0042/2437] Translated by JianhuanZhuo --- ...lding a Real-Time Recommendation Engine with Data Science.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160817 Building a Real-Time Recommendation Engine with Data Science.md b/sources/tech/20160817 Building a Real-Time Recommendation Engine with Data Science.md index 822c102a76..235ef44e25 100644 --- a/sources/tech/20160817 Building a Real-Time Recommendation Engine with Data Science.md +++ b/sources/tech/20160817 Building a Real-Time Recommendation Engine with Data Science.md @@ -1,3 +1,5 @@ +JianhuanZhuo + Building a Real-Time Recommendation Engine with Data Science ====================== From 3ef879cc0f075fa5c55eeb42ba095c2373518512 Mon Sep 17 00:00:00 2001 From: cposture Date: Mon, 5 Sep 2016 16:32:52 +0800 Subject: [PATCH 0043/2437] Translating by cposture 2016-09-05 --- ... Remote Linux Filesystem or Directory Using SSHFS Over SSH.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160811 How to Mount Remote Linux Filesystem or Directory Using SSHFS Over SSH.md b/sources/tech/20160811 How to Mount Remote Linux Filesystem or Directory Using SSHFS Over SSH.md index 6b51e41a9c..3ba803ca1d 100644 --- a/sources/tech/20160811 How to Mount Remote Linux Filesystem or Directory Using SSHFS Over SSH.md +++ b/sources/tech/20160811 How to Mount Remote Linux Filesystem or Directory Using SSHFS Over SSH.md @@ -1,3 +1,4 @@ +Translating by cposture How to Mount Remote Linux Filesystem or Directory Using SSHFS Over SSH ============================ From afca1c7c8f6223e37ebc002b841e5fb941cdc228 Mon Sep 17 00:00:00 2001 From: geekpi Date: Mon, 5 Sep 2016 16:41:13 +0800 Subject: [PATCH 0044/2437] translated --- ...nerally available in Azure SQL Database.md | 57 ------------------- ...nerally available in Azure SQL Database.md | 57 +++++++++++++++++++ 2 files changed, 57 insertions(+), 57 deletions(-) delete mode 100644 sources/tech/20160824 JSON support is generally available in Azure SQL Database.md create mode 100644 translated/tech/20160824 JSON support is generally available in Azure SQL Database.md diff --git a/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md b/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md deleted file mode 100644 index 3a3b8416fe..0000000000 --- a/sources/tech/20160824 JSON support is generally available in Azure SQL Database.md +++ /dev/null @@ -1,57 +0,0 @@ -translating---geekpi - -JSON support is generally available in Azure SQL Database -=========== - -We are happy to announce that you can now query and store both relational and textual data formatted in JavaScript Object Notation (JSON) using Azure SQL Database. Azure SQL Database provides simple built-in functions that read data from JSON text, transform JSON text into table, and format data from SQL tables as JSON. - -![](https://azurecomcdn.azureedge.net/mediahandler/acomblog/media/Default/blog/1cc536a5-a822-467b-a4a2-4557746f70cc.png) - - -You can use JSON functions that enable you to extract value from JSON text (JSON_VALUE), extract object from JSON (JSON_QUERY), update some value in JSON text (JSON_MODIFY), and verify that JSON text is properly formatted (ISJSON). OPENJSON function enables you to convert JSON text into a table structure. Finally, JSON functionalities enable you to easily format results of any SQL query as JSON text using the FOR JSON clause. - -### What can you do with JSON? - -JSON in Azure SQL Database enables you to build and exchange data with modern web, mobile, and HTM5/JavaScript single-page applications, NoSql stores such as Azure DocumentDB that contain data formatted as JSON, and to analyze logs and messages collected from different systems and services. Now you can easily integrate your Azure SQL Database with any service that uses JSON. - -#### Easily expose your data to modern frameworks and services - -Do you use services that exchange data in JSON format, such as REST services or Azure App Services? Do you have components or frameworks that use JSON, such as Angular JS, ReactJS, D3, or JQuery? With new JSON functionalities, you can easily format data stored in Azure SQL Database as JSON and expose it to any modern service or application. - -#### Easy ingestion of JSON data - -Are you working with mobile devices or sensors, services that produce JSON such as Azure Stream Analytics or Application Insight, or systems that store data in JSON format such as Azure DocumentDB or MongoDB? Do you need to query and analyze JSON data using well-known SQL language or tools that work with Azure SQL Database? Now, you can easily ingest JSON data and store it into Azure SQL Database, and use any language or tool that works with Azure SQL Database to query and analyze loaded information. - -#### Simplify your data models - -Do you need to store and query both relational and semi-structured data in your database? Do you need to simplify your data models like in NoSQL data platforms? Now you can combine structured relational data with schema-less data stored as JSON text in the same table. In Azure SQL Database you can use the best approaches both from relational and NoSQL worlds to tune your data model. Azure SQL Database enables you to query both relational and JSON data with the standard Transact-SQL language. Applications and tools would not see any difference between values taken from table columns and the values extracted from JSON text. - -### Next steps - -To learn how to integrate JSON in your application, check out our [Getting Started][1] page or [Channel 9 video][2]. To learn about various scenarios that show how to integrate JSON in your application, see demos in this Channel 9 video or find some scenario that might be interesting for your use case in these [JSON Blog posts][3]. - -Stay tuned because we will constantly add new JSON features and make JSON support even better. - - - --------------------------------------------------------------------------------- - -via: https://azure.microsoft.com/en-us/blog/json-support-is-generally-available-in-azure-sql-database/?utm_source=dbweekly&utm_medium=email - -作者:[Jovan Popovic][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://azure.microsoft.com/en-us/blog/author/jovanpop/ -[1]: https://azure.microsoft.com/en-us/documentation/articles/sql-database-json-features/ -[2]: https://channel9.msdn.com/Shows/Data-Exposed/SQL-Server-2016-and-JSON-Support -[3]: http://blogs.msdn.com/b/sqlserverstorageengine/archive/tags/json/ - - - - - - - diff --git a/translated/tech/20160824 JSON support is generally available in Azure SQL Database.md b/translated/tech/20160824 JSON support is generally available in Azure SQL Database.md new file mode 100644 index 0000000000..1368340312 --- /dev/null +++ b/translated/tech/20160824 JSON support is generally available in Azure SQL Database.md @@ -0,0 +1,57 @@ +Azure SQL数据库已经支持JSON +=========== + +我们很高兴地宣布你现在可以在Azure SQL中查询及存储关系型数据或者JSON了、Azure SQL数据库提供了简单的内置函数来读取JSON文本数据,将JSON文本转化成表,将表的数据转化成JSON。 + +![](https://azurecomcdn.azureedge.net/mediahandler/acomblog/media/Default/blog/1cc536a5-a822-467b-a4a2-4557746f70cc.png) + + +你可以使用JSON函数来从JSON文本中提取值(JSON_VALUE), 提取对象(JSON_QUERY), 更新JSON中的值(JSON_MODIFY),并且验证JSON文本的正确性(ISJSON)。OPENJSON函数让你可以将JSON文本转化成表结构。最后,JSON功能函数可以让你很简单地从SQL查询中使用FOR JSON从句来获得JSON文本结果。 + +### 你可以用JSON做什么? + + +Azure SQL数据库中的JSON可以让您构建并与现代web、移动设备和HTML5/单页应用、NoSQL中存储的诸如Azure DocumentDB中的格式化成JSON的数据交换数据,分析来自不同系统和服务的日志和消息。现在你可以轻易地将Azure SQL数据库与任何使用使用JSON的服务集成。 + +#### 轻易地开放数据给现代框架和服务 + +你有没有在使用诸如REST或者Azure App使用JSON来交换数据的服务?你有使用诸如AngularJS、ReactJS、D3或者Jquery使用JSON的组件或框架么?使用新的JSON功能函数,你可以轻易地格式化存储在Azure SQL数据库中的数据,并将它开放在任何现代服务或者应用。 + +#### 轻松采集JSON数据 + +你有在使用移动设备、传感器、如Azure Stream Analytics或者Insight这样产生JSON的服务、如Azure DocumentDB 或者MongoDB这样存储JSON的系统么?你需要在Azure SQL数据中使用熟悉的SQL语句来查询并分析JSON数据么?现在你可以轻松采集JSON数据并存储到Azure SQL数据库中,并且可以使用任何Azure SQL数据库支持的语言或者工具来查询和分析加载的数据 + +#### 简化你的数据模型 + +你需要同时存储及查询数据库中关系型及半结构化的数据么?你需简化像NoSQL平台下的数据模型么?现在你可以在一张表中同时存储结构化数据及非结构化数据了。在Azure SQL数据库中,你可以同时从关系型及NoSQL的世界中使用最好的方法来调整你的数据模型。Azure SQL数据库让你可以使用Transact-SQL语言来查询关系及JSON数据。程序和工具将不会在从表中取出的值及JSON文本中提取的值看出差别。 + +### 下一步 + +要学习如何在你的应用中集成JSON,查看我们的[开始学习][1]页面或者[Channel 9的视频][2]。要了解不同的情景下如何集成JSON,观看Channel 9的视频或者在这些[JSON分类文章][3]中查找你感兴趣的使用情景。 + +我们将持续增加新的JSON特性并让JSON支持的更好。请敬请关注。 + + + + +-------------------------------------------------------------------------------- + +via: https://azure.microsoft.com/en-us/blog/json-support-is-generally-available-in-azure-sql-database/?utm_source=dbweekly&utm_medium=email + +作者:[Jovan Popovic][a] +译者:[geekpi](https://github.com/geekpi) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://azure.microsoft.com/en-us/blog/author/jovanpop/ +[1]: https://azure.microsoft.com/en-us/documentation/articles/sql-database-json-features/ +[2]: https://channel9.msdn.com/Shows/Data-Exposed/SQL-Server-2016-and-JSON-Support +[3]: http://blogs.msdn.com/b/sqlserverstorageengine/archive/tags/json/ + + + + + + + From ff886503ff22616c98a0de46321b76dd195daa7d Mon Sep 17 00:00:00 2001 From: Purling Nayuki Date: Mon, 5 Sep 2016 17:04:04 +0800 Subject: [PATCH 0045/2437] Proofread 20160814 How I Got to be an Android Dev And What I Learned From It --- ... Android Dev And What I Learned From It.md | 38 +++++++++---------- 1 file changed, 18 insertions(+), 20 deletions(-) diff --git a/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md b/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md index b5f9b6f2a0..8a513bb13b 100644 --- a/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md +++ b/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md @@ -1,59 +1,57 @@ -JianhuanZhuo - -我的安卓开发者之路以及我在其中学到了什么 +我的 Android 开发者之路以及我在其中学到了什么 ============================= -别人都说所有的关系都是需要经历2年、七年甚至十年的磨砺的。我忘了是谁说的这句话,但肯定有人在几年前这么跟我说过。 +大都家说所有的关系都需要经历两年、七年甚至十年的磨砺。我忘了是谁说的这句话,但肯定有人在几年前这么跟我说过。 -下一周是我来悉尼两周年的日子,所以我想当前正是我写这篇文章的好时候。 +下周是我来悉尼两周年,所以我想现在正是我写这篇文章的好时候。 -去年五月份参加 I/O年会的时候,我遇到了一位十分漂亮的女士,亚斯曼。她向我请教我是如何成长为一名安卓开发者的,当我跟她说起我的经历时,她认为我应该写个博客记下来。所以亚斯曼,如你所愿,虽然迟了点,但好过没做。;) +去年五月份参加 I/O 年会的时候,我遇到了亚斯曼女士,她十分漂亮。她向我询问我是如何成长为一名安卓开发者的,当我说完我的经历时,她认为我应该写个博客记下来。所以亚斯曼,如你所愿,虽然迟了点,但好过没做。;) ### 故事的开始 -如果有件事你需要了解我的,那就是我发现自己有选择困难症。你最好的朋友是谁?你最喜欢的食物是什么?你应该给你的玩具熊猫命名吗?我连这些问题都不知道该怎么回答才好。所以你可以想象得到,16岁高中毕业的我对于专业选择根本就没有任何想法。那我最初申请的大学是?在交表给注册员前,我在她面前逐字掂量着写下这个打算申请的专业(商业经济学)。 +如果有件事你需要了解我的,那就是我发现自己有选择困难症。你最好的朋友是谁?你最喜欢的食物是什么?你应该给你的玩具熊猫命名吗?我连这些问题都不知道该怎么回答才好。所以你可以想象到,16 岁的、即将高中毕业的我对于专业选择根本就没有任何想法。那我最初申请的大学是?在交表给注册员前,我在她面前逐字掂量着写下这个打算申请的专业(商业经济学)。 -可我最后去了另外一间学校,就读电子与通信工程。大一时我有一门计算机编程课程。但我很讨厌编程,十分的讨厌。关于编程的一切我都一无所知。我曾发誓再也不会干这种事了。 +可我最后去了另外一间学校,就读电子与通信工程。大一时我有一门计算机编程课程。但我很讨厌编程,十分地讨厌。关于编程的一切我都一无所知。我曾发誓再也不要写代码了。 -我大学毕业后的第一份工作是在因特尔做产品工程师并在那呆了两年。我生活上餐不继下餐,整天长时间工作,但这在我意料之中,身为成年人是该努力工作的,难道不是吗?可之后菲律宾的半导体行业开始呈现颓势,大批工厂纷纷倒闭,我们以前常常光顾的一些产品被收购转到了另一个网站。我便决定去找另一份工作而不是等着被裁员,因为被裁员后我都不知道自己会失业多久。 +我大学毕业后的第一份工作是在英特尔做产品工程师并在那呆了两年。我很迷茫,无所适从,整天长时间工作。这在我意料之中,身为成年人难道不该努力工作吗?可之后菲律宾的半导体行业开始呈现颓势,大批工厂纷纷倒闭,以前由我们维护一些产品被转移到其他分公司。我便决定去找另一份工作而不是等着被裁员,因为被裁员后我都不知道自己多久才能找到另一份工作。 ### 现在呢? -我想留在城市里找到一份工作,但我不想呆在正在下沉的半导体行业里了。但话说回来,我又不知道该做什么好。对了,我可是拿了毕业证书的工程师,所以技术上来说我可以在电信运营商或电视台找到工作。可这种时候,如果你想入职电信运营商,应该在大学毕业之际去他们那实习,你就能有一个好的录用渠道,可惜我没有,所以我放弃了这个想法。虽然有是很多软件开发人员的招聘信息,但我讨厌编程,所以我真的不知道怎么做才好。 +我想留在城市里找到一份工作,但我不想呆在正在没落的半导体行业里了。但话说回来,我又不知道该做什么好。对了,我可是拿了毕业证书的工程师,所以技术上来说我可以在电信运营商或电视台找到工作。可这种时候,如果想入职电信运营商,我应该在大学毕业之际去他们那实习,这样更容易被录用。可惜我没有,所以我放弃了这个想法。虽然有是很多软件开发人员的招聘信息,但我讨厌编程,所以我真的不知道怎么做才好。 -接下来是我第一个幸运的机遇,我很幸运地遇到了信任我的上司,我也和她坦诚了我什么都不会。之后我不得不学习以胜任这份工作,一开始这个过程很漫长。无需多言,我在这份工作上学到了很多,也结识了很多很好的人,与我一起是一群很厉害的同事(我们曾开发出安装在 SIM 卡上的 APP)。但更重要的是我开始踏上了软件开发的征途。 +接下来是我第一个幸运的机遇,我很幸运地遇到了信任我的上司,我也和她坦诚了我什么都不会。之后我不得不边工作边学习,一开始这个过程很漫长。无需多言,我在这份工作上学到了很多,也结识了很多很好的人,与我一起是一群很厉害的同事(我们曾开发出安装在 SIM 卡上的 APP)。但更重要的是我开始踏上了软件开发的征途。 -最后我做得更多是一些公司的琐事(十分无聊)直到项目完结。换句话说,我总在是在办公室里闲逛并坐等发薪。之后我发现这确实是在浪费时间,2009 年的时候我对谷歌的安卓新系统有耳闻了,现在它的 SDK 上线了!于是我安装了所有相关软件并着手安卓开发。 +最后我做得更多是一些公司的琐事(十分无聊)直到项目完结。换句话说,我总在是在办公室里闲逛并坐等发薪。之后我发现这确实是在浪费时间,2009 年的时候,我不停地接触到关于谷歌的新系统 Android,并得知它的 SDK 已经公布!是时候尝试一波了。于是我安装了所有相关软件并着手 Android 开发。 ### 事情变得有趣了 -所以现在我能够构建了一个在运行在仿真器的 Hello World 应用,在我看来意味着我有胜任安卓开发工作的能力。我又一次加入了一个创业公司,并在里面做前端开发 -- 我不知道该怎么做,对此我手足无措,但如果你为我的手足无措买单的话,我们就可以做朋友了,所以我很幸运遇到另一个机遇。 +所以现在我能够构建了一个在运行在仿真器的 Hello World 应用,在我看来意味着我有胜任安卓开发工作的能力。我加入了一个创业公司,并且再次坦诚我不知道该怎么做,我只是接触过一些;但如果你们愿意付薪水给我继续尝试,我们就可以成为朋友。然后我很幸运遇到另一个机遇。 -那时成为开发者是一件令人欣喜的事。安卓开发的 StackOverflow 社区非常小,我们都在相互交流学习,说真的,我认为里面的所有人都很友好、很豪迈 +。 +那时成为开发者是一件令人欣喜的事。StackOverflow 上的 Android 开发社区非常小,我们都在相互交流学习,说真的,我认为里面的所有人都很友好、很豪迈 +。 我最后去了一家企业,这家企业的移动开发团队在马尼拉、悉尼、纽约都设有办公地点。而我是马尼拉办公地点的第一个安卓开发人员,但那时我很习惯,并没有在意。 -在那里我认识了最后令我永远感激的引荐我参与 Domain 项目的人。Domain 项目不管在个人或职业上对我来说都意味深重。我和一支很有才华的团队一起共事,也从没见过一个公司能如此执着于一款产品。Domain 让我实现了参加 IO 年会的梦。与他们共事后我懂得了很多之前没想到的可爱特性 ++。另一个幸运的机遇,我是说最大限度地利用它。 +在那里我认识了最后令我永远感激的引荐我参与 Domain 项目的人。Domain 项目不管在个人或职业上对我来说都意味深重。我和一支很有才华的团队一起共事,也从没见过一个公司能如此执着于一款产品。Domain 让我实现了参加 I/O 年会的梦。与他们共事后我懂得了很多之前没想到的可爱特性 ++。这是又一个幸运的机遇,我是说最大限度地利用它。 ### 然后呢? -我想说的是虽然这些年都在飘荡,但至少我很诚实,对吧?如上就是我所学到的全部东西。说一句「我不懂」没什么可怕的。有时候我们是该装懂,但大部分时间我们需要坦诚地接受这样一个事实:我们还不懂。 +我想说的是,虽然这些年都在飘荡,但至少我很诚实,对吧?如上就是我所学到的全部东西。说一句「我不懂」没什么可怕的。有时候我们是该装懂,但更多时候我们需要坦诚地接受这样一个事实:我们还不懂。 别害怕尝试新事物,不管它让你感觉多害怕。我知道说比做简单。但总有一些东西能让你鼓起勇气动手去尝试的 +++。Lundagin mo, baby!(LCTT 译者注:一首歌名) -`+` 我查找过我以前提问的 StackOverflow 问题,认真地想想如果现在我问他们这些,估计会收到很多「你是谁啊,傻瓜」的评论。可能是我老了,我也不知道。但关键是,我们曾经在那个社区里彼此很友好,是吗? +`+` 我翻阅着以前在 StackOverflow 提的问题,认真想想,如果现在我问他们这些,估计会收到很多「你是谁啊,傻瓜」的评论。我不知道是不是因为我老了,而且有些愤世妒俗。但关键是,我们有缘在同一个社区中,所以大家相互之间友善些,好吗? `++` 这一点写在另一篇文章里了。 -`+++` 我还清晰地记得申请第一份安卓开发职位的情形:我写完求职信后又通读了一遍,提交前鼠标在发送按钮上不断徘徊,深呼吸之后我趁改变主意之前把它发出去了。 +`+++` 我还清晰地记得第一次申请 Android 开发职位的情形:我写完求职信后又通读了一遍,提交前鼠标在发送按钮上不断徘徊,深呼吸之后我趁改变主意之前把它发出去了。 -------------------------------------------------------------------------------- via: http://www.zdominguez.com/2016/08/winging-it-how-i-got-to-be-android-dev.html?utm_source=Android+Weekly&utm_campaign=9314c56ae3-Android_Weekly_219&utm_medium=email&utm_term=0_4eb677ad19-9314c56ae3-338075705 -作者:[Zarah Dominguez ][a] +作者:[Zarah Dominguez][a] 译者:[JianhuanZhuo](https://github.com/JianhuanZhuo) -校对:[校对者ID](https://github.com/校对者ID) +校对:[PurlingNayuki](https://github.com/PurlingNayuki) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From 7ea9ceeea9ad56542e0640f8f58cd0b93508473f Mon Sep 17 00:00:00 2001 From: wxy Date: Mon, 5 Sep 2016 20:15:10 +0800 Subject: [PATCH 0046/2437] PUB:20160814 How I Got to be an Android Dev And What I Learned From It @JianhuanZhuo @PurlingNayuki --- ... Android Dev And What I Learned From It.md | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) rename {translated/talk => published}/20160814 How I Got to be an Android Dev And What I Learned From It.md (57%) diff --git a/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md b/published/20160814 How I Got to be an Android Dev And What I Learned From It.md similarity index 57% rename from translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md rename to published/20160814 How I Got to be an Android Dev And What I Learned From It.md index 8a513bb13b..0caca5f4df 100644 --- a/translated/talk/20160814 How I Got to be an Android Dev And What I Learned From It.md +++ b/published/20160814 How I Got to be an Android Dev And What I Learned From It.md @@ -1,7 +1,7 @@ 我的 Android 开发者之路以及我在其中学到了什么 ============================= -大都家说所有的关系都需要经历两年、七年甚至十年的磨砺。我忘了是谁说的这句话,但肯定有人在几年前这么跟我说过。 +大家都说所有的关系都需要经历两年、七年甚至十年的磨砺。我忘了是谁说的这句话,但肯定有人在几年前这么跟我说过。 下周是我来悉尼两周年,所以我想现在正是我写这篇文章的好时候。 @@ -9,7 +9,7 @@ ### 故事的开始 -如果有件事你需要了解我的,那就是我发现自己有选择困难症。你最好的朋友是谁?你最喜欢的食物是什么?你应该给你的玩具熊猫命名吗?我连这些问题都不知道该怎么回答才好。所以你可以想象到,16 岁的、即将高中毕业的我对于专业选择根本就没有任何想法。那我最初申请的大学是?在交表给注册员前,我在她面前逐字掂量着写下这个打算申请的专业(商业经济学)。 +如果有件事我可能希望你知道,那就是我发现自己有选择困难症。你最好的朋友是谁?你最喜欢的食物是什么?你应该给你的玩具熊猫命名吗?我连这些问题都不知道该怎么回答才好。所以你可以想象到,16 岁的、即将高中毕业的我对于专业选择根本就没有任何想法。那我最初申请的大学是?在交表给注册员前,我在她面前逐字掂量着写下这个打算申请的专业(商业经济学)。 可我最后去了另外一间学校,就读电子与通信工程。大一时我有一门计算机编程课程。但我很讨厌编程,十分地讨厌。关于编程的一切我都一无所知。我曾发誓再也不要写代码了。 @@ -17,37 +17,37 @@ ### 现在呢? -我想留在城市里找到一份工作,但我不想呆在正在没落的半导体行业里了。但话说回来,我又不知道该做什么好。对了,我可是拿了毕业证书的工程师,所以技术上来说我可以在电信运营商或电视台找到工作。可这种时候,如果想入职电信运营商,我应该在大学毕业之际去他们那实习,这样更容易被录用。可惜我没有,所以我放弃了这个想法。虽然有是很多软件开发人员的招聘信息,但我讨厌编程,所以我真的不知道怎么做才好。 +我想留在城市里找到一份工作,但我不想呆在正在没落的半导体行业里了。但话说回来,我又不知道该做什么好。对了,我可是拿了毕业证书的工程师,所以从技术上来说我可以在电信运营商或电视台找到工作。可这种时候,如果想入职电信运营商,我应该在大学毕业之际就去他们那实习,这样更容易被录用。可惜我没有,所以我放弃了这个想法。虽然有很多软件开发人员的招聘信息,但我讨厌编程,所以我真的不知道怎么做才好。 -接下来是我第一个幸运的机遇,我很幸运地遇到了信任我的上司,我也和她坦诚了我什么都不会。之后我不得不边工作边学习,一开始这个过程很漫长。无需多言,我在这份工作上学到了很多,也结识了很多很好的人,与我一起是一群很厉害的同事(我们曾开发出安装在 SIM 卡上的 APP)。但更重要的是我开始踏上了软件开发的征途。 +接下来是我第一个幸运的机遇,我很幸运地遇到了信任我的上司,我也和她坦诚了我什么都不会。之后我不得不边工作边学习,一开始这个过程很漫长。无需多言,我在这份工作上学到了很多,也结识了很多很好的人,与我一起的是一群很厉害的同事(我们曾开发出安装在 SIM 卡上的 APP)。但更重要的是我开始踏上了软件开发的征途。 -最后我做得更多是一些公司的琐事(十分无聊)直到项目完结。换句话说,我总在是在办公室里闲逛并坐等发薪。之后我发现这确实是在浪费时间,2009 年的时候,我不停地接触到关于谷歌的新系统 Android,并得知它的 SDK 已经公布!是时候尝试一波了。于是我安装了所有相关软件并着手 Android 开发。 +最后我做得更多是一些公司的琐事(十分无聊)直到项目完结。换句话说,我总在是在办公室里闲逛并坐等发薪。之后我发现这确实是在浪费时间,2009 年的时候,我不停地接触到关于谷歌的新系统 Android 的消息,并得知它的 SDK 已经公布!是时候尝试一波了。于是我安装了所有相关软件并着手 Android 开发。 ### 事情变得有趣了 -所以现在我能够构建了一个在运行在仿真器的 Hello World 应用,在我看来意味着我有胜任安卓开发工作的能力。我加入了一个创业公司,并且再次坦诚我不知道该怎么做,我只是接触过一些;但如果你们愿意付薪水给我继续尝试,我们就可以成为朋友。然后我很幸运遇到另一个机遇。 +所以现在我能够构建一个在运行在仿真器的 Hello World 应用,在我看来意味着我有胜任安卓开发工作的能力。我加入了一个创业公司,并且再次坦诚我不知道该怎么做,我只是接触过一些;但如果你们愿意付薪水给我继续尝试,我们就可以成为朋友。然后我很幸运地遇到另一个机遇。 -那时成为开发者是一件令人欣喜的事。StackOverflow 上的 Android 开发社区非常小,我们都在相互交流学习,说真的,我认为里面的所有人都很友好、很豪迈 +。 +那时成为开发者是一件令人欣喜的事。StackOverflow 上的 Android 开发社区非常小,我们都在相互交流学习,说真的,我认为里面的所有人都很友好、很豪迈(注 1)! 我最后去了一家企业,这家企业的移动开发团队在马尼拉、悉尼、纽约都设有办公地点。而我是马尼拉办公地点的第一个安卓开发人员,但那时我很习惯,并没有在意。 -在那里我认识了最后令我永远感激的引荐我参与 Domain 项目的人。Domain 项目不管在个人或职业上对我来说都意味深重。我和一支很有才华的团队一起共事,也从没见过一个公司能如此执着于一款产品。Domain 让我实现了参加 I/O 年会的梦。与他们共事后我懂得了很多之前没想到的可爱特性 ++。这是又一个幸运的机遇,我是说最大限度地利用它。 +在那里我认识了最后令我永远感激的引荐我参与 Domain 项目的人。Domain 项目不管在个人或职业上对我来说都意味深重。我和一支很有才华的团队一起共事,也从没见过一个公司能如此执着于一款产品。Domain 让我实现了参加 I/O 年会的梦。与他们共事后我懂得了很多之前没想到的可爱特性(注 2)。这是又一个幸运的机遇,我是说最大限度地利用它。 ### 然后呢? -我想说的是,虽然这些年都在飘荡,但至少我很诚实,对吧?如上就是我所学到的全部东西。说一句「我不懂」没什么可怕的。有时候我们是该装懂,但更多时候我们需要坦诚地接受这样一个事实:我们还不懂。 +我想说的是,虽然这些年都在晃荡,但至少我很诚实,对吧?如上就是我所学到的全部东西。说一句「我不懂」没什么可怕的。有时候我们是该装懂,但更多时候我们需要坦诚地接受这样一个事实:我们还不懂。 -别害怕尝试新事物,不管它让你感觉多害怕。我知道说比做简单。但总有一些东西能让你鼓起勇气动手去尝试的 +++。Lundagin mo, baby!(LCTT 译者注:一首歌名) +别害怕尝试新事物,不管它让你感觉多害怕。我知道说比做简单。但总有一些东西能让你鼓起勇气动手去尝试的(注3)。Lundagin mo, baby!(LCTT 译注:一首歌名) -`+` 我翻阅着以前在 StackOverflow 提的问题,认真想想,如果现在我问他们这些,估计会收到很多「你是谁啊,傻瓜」的评论。我不知道是不是因为我老了,而且有些愤世妒俗。但关键是,我们有缘在同一个社区中,所以大家相互之间友善些,好吗? -`++` 这一点写在另一篇文章里了。 -`+++` 我还清晰地记得第一次申请 Android 开发职位的情形:我写完求职信后又通读了一遍,提交前鼠标在发送按钮上不断徘徊,深呼吸之后我趁改变主意之前把它发出去了。 +注 1: 我翻阅着以前在 StackOverflow 提的问题,认真想想,如果现在我问他们这些,估计会收到很多「你是谁啊,傻瓜」的评论。我不知道是不是因为我老了,而且有些愤世妒俗。但关键是,我们有缘在同一个社区中,所以大家相互之间友善些,好吗? +注 2: 这一点写在另一篇文章里了。 +注 3: 我还清晰地记得第一次申请 Android 开发职位的情形:我写完求职信后又通读了一遍,提交前鼠标在发送按钮上不断徘徊,深呼吸之后我趁改变主意之前把它发出去了。 -------------------------------------------------------------------------------- -via: http://www.zdominguez.com/2016/08/winging-it-how-i-got-to-be-android-dev.html?utm_source=Android+Weekly&utm_campaign=9314c56ae3-Android_Weekly_219&utm_medium=email&utm_term=0_4eb677ad19-9314c56ae3-338075705 +via: http://www.zdominguez.com/2016/08/winging-it-how-i-got-to-be-android-dev.html 作者:[Zarah Dominguez][a] 译者:[JianhuanZhuo](https://github.com/JianhuanZhuo) From 25b9b6a18e5bb8c657ff39b358ebd43eaa93dd29 Mon Sep 17 00:00:00 2001 From: wxy Date: Mon, 5 Sep 2016 20:15:36 +0800 Subject: [PATCH 0047/2437] PUB:Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis @ictlyh --- ... Set Process Limits on a Per-User Basis.md | 71 +++++++++---------- 1 file changed, 35 insertions(+), 36 deletions(-) rename {translated/tech => published}/LFCS/Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis.md (78%) diff --git a/translated/tech/LFCS/Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis.md b/published/LFCS/Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis.md similarity index 78% rename from translated/tech/LFCS/Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis.md rename to published/LFCS/Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis.md index c6db6be7cc..0ba5df1b34 100644 --- a/translated/tech/LFCS/Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis.md +++ b/published/LFCS/Part 14 - Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis.md @@ -1,10 +1,11 @@ -LFCS 系列第十四讲: Linux 进程资源使用监控和基于每个用户的进程限制设置 -============================================================================================= +LFCS 系列第十四讲: Linux 进程资源用量监控和按用户设置进程限制 +============================================================ 由于 2016 年 2 月 2 号开始启用了新的 LFCS 考试要求,我们在已经发表的 [LFCS 系列][1] 基础上增加了一些必要的主题。为了准备考试,同时也建议你看看 [LFCE 系列][2] 文章。 ![](http://www.tecmint.com/wp-content/uploads/2016/03/Linux-Process-Monitoring-Set-Process-Limits-Per-User.png) ->第十四讲: 监控 Linux 进程并为每个用户设置进程限制 + +*第十四讲: 监控 Linux 进程并为每个用户设置进程限制* 每个 Linux 系统管理员都应该知道如何验证硬件、资源和主要进程的完整性和可用性。另外,基于每个用户设置资源限制也是其中一项必备技能。 @@ -26,13 +27,13 @@ LFCS 系列第十四讲: Linux 进程资源使用监控和基于每个用户 安装完 **mpstat** 之后,就可以使用它生成处理器统计信息的报告。 -你可以使用下面的命令每隔 2 秒显示所有 CPU(用 `-P` ALL 表示) 的 CPU 利用率(`-u`),共显示 **3** 次。 +你可以使用下面的命令每隔 2 秒显示所有 CPU(用 `-P` ALL 表示)的 CPU 利用率(`-u`),共显示 **3** 次。 ``` # mpstat -P ALL -u 2 3 ``` -### 事例输出 +**示例输出:** ``` Linux 3.19.0-32-generic (tecmint.com) Wednesday 30 March 2016 _x86_64_ (4 CPU) @@ -72,7 +73,7 @@ Average: 3 12.25 0.00 1.16 0.00 0.00 0.00 0.00 0.00 # mpstat -P 0 -u 2 3 ``` -### 事例输出 +**示例输出:** ``` Linux 3.19.0-32-generic (tecmint.com) Wednesday 30 March 2016 _x86_64_ (4 CPU) @@ -86,16 +87,16 @@ Average: 0 5.58 0.00 0.34 0.85 0.00 0.00 0.00 0.00 上面命令的输出包括这些列: -* `CPU`: 整数表示的处理器号或者 all 表示所有处理器的平局值。 +* `CPU`: 整数表示的处理器号或者 all 表示所有处理器的平均值。 * `%usr`: 运行在用户级别的应用的 CPU 利用率百分数。 * `%nice`: 和 `%usr` 相同,但有 nice 优先级。 * `%sys`: 执行内核应用的 CPU 利用率百分比。这不包括用于处理中断或者硬件请求的时间。 -* `%iowait`: 指定(或所有) CPU 的空闲时间百分比,这表示当前 CPU 处于 I/O 操作密集的状态。更详细的解释(附带示例)可以查看[这里][4]。 +* `%iowait`: 指定(或所有)CPU 的空闲时间百分比,这表示当前 CPU 处于 I/O 操作密集的状态。更详细的解释(附带示例)可以查看[这里][4]。 * `%irq`: 用于处理硬件中断的时间所占百分比。 -* `%soft`: 和 `%irq` 相同,但是软中断。 -* `%steal`: 当一个客户虚拟机在竞争 CPU 时,非自主等待(时间片窃取)所占时间的百分比。应该保持这个值尽可能小。如果这个值很大,意味着虚拟机正在或者将要停止运转。 +* `%soft`: 和 `%irq` 相同,但是是软中断。 +* `%steal`: 虚拟机非自主等待(时间片窃取)所占时间的百分比,即当虚拟机在竞争 CPU 时所从虚拟机管理程序那里“赢得”的时间。应该保持这个值尽可能小。如果这个值很大,意味着虚拟机正在或者将要停止运转。 * `%guest`: 运行虚拟处理器所用的时间百分比。 -* `%idle`: CPU(s) 没有运行任何任务所占时间的百分比。如果你观察到这个值很小,意味着系统负载很重。在这种情况下,你需要查看详细的进程列表、以及下面将要讨论的内容来确定这是什么原因导致的。 +* `%idle`: CPU 没有运行任何任务所占时间的百分比。如果你观察到这个值很小,意味着系统负载很重。在这种情况下,你需要查看详细的进程列表、以及下面将要讨论的内容来确定这是什么原因导致的。 运行下面的命令使处理器处于极高负载,然后在另一个终端执行 mpstat 命令: @@ -109,7 +110,8 @@ Average: 0 5.58 0.00 0.34 0.85 0.00 0.00 0.00 0.00 最后,和 “正常” 情况下 **mpstat** 的输出作比较: ![](http://www.tecmint.com/wp-content/uploads/2016/03/Report-Processors-Related-Statistics.png) -> Linux 处理器相关统计信息报告 + +*Linux 处理器相关统计信息报告* 正如你在上面图示中看到的,在前面两个例子中,根据 `%idle` 的值可以判断 **CPU 0** 负载很高。 @@ -126,7 +128,8 @@ Average: 0 5.58 0.00 0.34 0.85 0.00 0.00 0.00 0.00 上面的命令只会显示 `PID`、`PPID`、和进程相关的命令、 CPU 使用率以及 RAM 使用率,并按照 CPU 使用率降序排序。创建 .iso 文件的时候运行上面的命令,下面是输出的前面几行: ![](http://www.tecmint.com/wp-content/uploads/2016/03/Find-Linux-Processes-By-CPU-Usage.png) ->根据 CPU 使用率查找进程 + +*根据 CPU 使用率查找进程* 一旦我们找到了感兴趣的进程(例如 `PID=2822` 的进程),我们就可以进入 `/proc/PID`(本例中是 `/proc/2822`) 列出目录内容。 @@ -134,10 +137,9 @@ Average: 0 5.58 0.00 0.34 0.85 0.00 0.00 0.00 0.00 #### 例如: -* `/proc/2822/io` 包括该进程的 IO 统计信息( IO 操作时的读写字符数)。 +* `/proc/2822/io` 包括该进程的 IO 统计信息(IO 操作时的读写字符数)。 * `/proc/2822/attr/current` 显示了进程当前的 SELinux 安全属性。 -* `/proc/2822/attr/current` shows the current SELinux security attributes of the process. -* `/proc/2822/cgroup` 如果启用了 CONFIG_CGROUPS 内核设置选项,这会显示该进程所属的控制组(简称 cgroups),你可以使用下面命令验证是否启用了 CONFIG_CGROUPS: +* `/proc/2822/cgroup` 如果启用了 CONFIG_CGROUPS 内核设置选项,这会显示该进程所属的控制组(简称 cgroups),你可以使用下面命令验证是否启用了 CONFIG_CGROUPS: ``` # cat /boot/config-$(uname -r) | grep -i cgroups @@ -154,7 +156,8 @@ CONFIG_CGROUPS=y `/proc/2822/fd` 这个目录包含每个打开的描述进程的文件的符号链接。下面的截图显示了 tty1(第一个终端) 中创建 **.iso** 镜像进程的相关信息: ![](http://www.tecmint.com/wp-content/uploads/2016/03/Find-Linux-Process-Information.png) ->查找 Linux 进程信息 + +*查找 Linux 进程信息* 上面的截图显示 **stdin**(文件描述符 **0**)、**stdout**(文件描述符 **1**)、**stderr**(文件描述符 **2**) 相应地被映射到 **/dev/zero**、 **/root/test.iso** 和 **/dev/tty1**。 @@ -170,14 +173,15 @@ CONFIG_CGROUPS=y * hard nproc 10 ``` -第一个字段可以用来表示一个用户、组或者所有`(*)`, 第二个字段强制限制可以使用的进程数目(nproc) 为 **10**。退出并重新登录就可以使设置生效。 +第一个字段可以用来表示一个用户、组或者所有人`(*)`, 第二个字段强制限制可以使用的进程数目(nproc) 为 **10**。退出并重新登录就可以使设置生效。 -然后,让我们来看看非 root 用户(合法用户或非法用户) 试图引起 shell fork bomb[WiKi][12] 时会发生什么。如果我们没有设置限制, shell fork bomb 会无限制地启动函数的两个实例,然后无限循环地复制任意一个实例。最终导致你的系统卡死。 +然后,让我们来看看非 root 用户(合法用户或非法用户) 试图引起 shell fork 炸弹 (参见 [WiKi][12]) 时会发生什么。如果我们没有设置限制, shell fork 炸弹会无限制地启动函数的两个实例,然后无限循环地复制任意一个实例。最终导致你的系统卡死。 -但是,如果使用了上面的限制,fort bomb 就不会成功,但用户仍然会被锁在外面直到系统管理员杀死相关的进程。 +但是,如果使用了上面的限制,fort 炸弹就不会成功,但用户仍然会被锁在外面直到系统管理员杀死相关的进程。 ![](http://www.tecmint.com/wp-content/uploads/2016/03/Shell-Fork-Bomb.png) ->运行 Shell Fork Bomb + +*运行 Shell Fork 炸弹* **提示**: `limits.conf` 文件中可以查看其它 **ulimit** 可以更改的限制。 @@ -185,7 +189,7 @@ CONFIG_CGROUPS=y 除了上面讨论的工具, 一个系统管理员还可能需要: -**a)** 通过使用 **renice** 调整执行优先级(系统资源使用)。这意味着内核会根据分配的优先级(众所周知的 “**niceness**”,它是一个范围从 `-20` 到 `19` 的整数)给进程分配更多或更少的系统资源。 +**a)** 通过使用 **renice** 调整执行优先级(系统资源的使用)。这意味着内核会根据分配的优先级(众所周知的 “**niceness**”,它是一个范围从 `-20` 到 `19` 的整数)给进程分配更多或更少的系统资源。 这个值越小,执行优先级越高。普通用户(而非 root)只能调高他们所有的进程的 niceness 值(意味着更低的优先级),而 root 用户可以调高或调低任何进程的 niceness 值。 @@ -195,9 +199,9 @@ renice 命令的基本语法如下: # renice [-n] identifier ``` -如果没有 new priority 后面的参数(为空),默认就是 PID。在这种情况下, **PID=identifier** 的进程的 niceness 值会被设置为 ``。 +如果 new priority 后面的参数没有(为空),默认就是 PID。在这种情况下,**PID=identifier** 的进程的 niceness 值会被设置为 ``。 -**b)** 需要的时候中断一个进程的正常执行。这也就是通常所说的 [“杀死”进程][9]。实质上,这意味着给进程发送一个信号使它恰当地结束运行并以有序的方式释放任何占用的资源。 +**b)** 需要的时候中断一个进程的正常执行。这也就是通常所说的[“杀死”进程][9]。实质上,这意味着给进程发送一个信号使它恰当地结束运行并以有序的方式释放任何占用的资源。 按照下面的方式使用 **kill** 命令[杀死进程][10]: @@ -205,7 +209,7 @@ renice 命令的基本语法如下: # kill PID ``` -另外,你也可以使用 [pkill][11] 结束指定用户`(-u)`、指定组`(-G)` 甚至有共同 PPID`(-P)` 的所有进程。这些选项后面可以使用数字或者名称表示的标识符。 +另外,你也可以使用 [pkill][11] 结束指定用户`(-u)`、指定组`(-G)` 甚至有共同的父进程 ID `(-P)` 的所有进程。这些选项后面可以使用数字或者名称表示的标识符。 ``` # pkill [options] identifier @@ -217,9 +221,7 @@ renice 命令的基本语法如下: # pkill -G 1000 ``` -会杀死组 `GID=1000` 的所有进程。 - -而 +会杀死组 `GID=1000` 的所有进程。而 ``` # pkill -P 4993 @@ -236,7 +238,8 @@ renice 命令的基本语法如下: 用下面的图片说明: ![](http://www.tecmint.com/wp-content/uploads/2016/03/List-User-Running-Processes.png) ->在 Linux 中查找用户运行的进程 + +*在 Linux 中查找用户运行的进程* ### 总结 @@ -246,17 +249,13 @@ renice 命令的基本语法如下: 我们希望本篇中介绍的概念能对你有所帮助。如果你有任何疑问或者评论,可以使用下面的联系方式联系我们。 - - - - -------------------------------------------------------------------------------- -via: http://www.tecmint.com/linux-basic-shell-scripting-and-linux-filesystem-troubleshooting/ +via: http://www.tecmint.com/monitor-linux-processes-and-set-process-limits-per-user/ 作者:[Gabriel Cánepa][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) +译者:[ictlyh](https://github.com/ictlyh) +校对:[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 From 31297d9bdb1513b9ed61862bc736be281a8227da Mon Sep 17 00:00:00 2001 From: Ezio Date: Tue, 6 Sep 2016 09:24:01 +0800 Subject: [PATCH 0048/2437] =?UTF-8?q?20160906-1=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...ecure, Open Source Evernote Alternative.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md diff --git a/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md b/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md new file mode 100644 index 0000000000..f86d28c966 --- /dev/null +++ b/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md @@ -0,0 +1,82 @@ +Turtl: Secure, Open Source Evernote Alternative +============= + + +[Turtl][9] is a secure, open source Evernote alternative, available for Linux, Windows, Mac, and Android. An iOS version is "coming soon". Firefox and Chrome bookmarking extensions are also available. + +![](https://3.bp.blogspot.com/-cNoUUjaU4A0/V7MFKCasZJI/AAAAAAAAYTk/r7oWe-z_HB87hDvlKLViiiHUMfagnC6LQCLcB/s400/turtl-desktop-linux.png) + +The application, which is currently in beta, lets you keep your notes (with Markdown support for the note editor), website bookmarks, passwords, documents, photos, and so on, in a single private place. + +Notes can be organized in boards, which support nesting, and can be shared with other Turtl users: + +![](https://2.bp.blogspot.com/-G-Ln3T1c2QA/V7MFmrqkukI/AAAAAAAAYTs/dXMPEB9MPREicixlEJlQVqg9SFjBX1pwgCLcB/s400/turtl-boards.png) + +You can also add tags to your notes. The Turtle search allows sorting by creation date, last edited date, or by tags. + +Here's the note editor (for a file note): + +![](https://1.bp.blogspot.com/-8cNHV69iCWM/V7MFX7sBlMI/AAAAAAAAYTo/ZUVTYwiCSy8uzrVKdf6NcsQZlHtylIyvgCEw/s400/turtl-edit-note.png) + +So what about security? Turtl encrypts the data before storing it, using a cryptographic key, and the password is not stored on the server. Only you and those you choose to share with can read your data. You can read more about the Turtl security and encryption [HERE][1]. + +Update (thanks to Dimitry!): according to a [bug report][2], Turtl has a pretty serious security issue. Turtl allows creating multiple accounts with the same username, distinguishing between them by passwords. Hopefully this will be fixed soon. + +The Turtl developers provide a hosted service for synchronizing your notes, which is completely free "until your profile grows past a certain size or you require certain features". At the time I'm writing this article, the premium service is not available. + +However, you don't have to use the self hosted server - you can run your own [Turtl server][3] since it's free, open source software, just like the desktop and mobile applications. + +Turtl is not as feature rich as Evernote, however, quite a few new features are listed in its [roadmap][4], like import/export to plaintext and Evernote data format, native PDF reader support, interface locking, and more. + +I should also mention that the desktop application requires entering the password every time it's started, which might be good for security reasons, but can be considered annoying by some. + + +### Download Turtl + + +[Download Turtl application][5] (binaries available for Linux - 32bit and 64bit, Windows 64bit, Mac 64bit, Android, as well as Chrome and Firefox bookmarking add-ons) + +**Update**: Turtl uses a new server. To get the app to work, logout and in Advanced settings, under the login box, set the Turtl server to "https://api.turtlapp.com/v2" (without the quotes). + +To download the source code (desktop, mobile and server), report bugs, etc., see the Turtl @ [GitHub][6]. + +Arch Linux users can install Turtl via [AUR][7]. + +To install Turtl in Linux, extract the downloaded archive and run the "install.sh" script. Before installing it, make sure the ~/.local/share/applications folder exists: + +``` +mkdir -p ~/.local/share/applications +``` + +Important: installing Turtl with sudo makes the application runnable as root only, so either install it without sudo (somewhere in your home folder), or manually fix the permissions (you can take a look at the AUR [package][8] for what permissions to set). + +For instance, to install Turtl in the ~/turtl folder, use the following command (assumes you've extracted Turtl in your home folder): + +```` +~/turtl-*/install.sh ~/turtl +``` + +You can use "~/.turtl" instead of "~/turtl" to install Turtl to a hidden folder in your home directory. Or you can hide the ~/turtl folder using a simple trick. + +If Turtl doesn't show up in the menu / Unity Dash, restart the session (logout / login). + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/08/turtl-secure-open-source-evernote.html + +作者:[Andrew ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: https://turtl.it/docs/security/ +[2]: https://github.com/turtl/api/issues/20 +[3]: https://turtl.it/docs/server/ +[4]: https://trello.com/b/yIQGkHia/turtl-product-dev +[5]: https://turtl.it/download/ +[6]: https://github.com/turtl +[7]: https://aur.archlinux.org/packages/turtl/ +[8]: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=turtl +[9]: https://turtlapp.com/ From ed0e14bb627face32d794437e531fb3758419a9a Mon Sep 17 00:00:00 2001 From: Ezio Date: Tue, 6 Sep 2016 09:29:56 +0800 Subject: [PATCH 0049/2437] =?UTF-8?q?20160906-2=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... LIST APP THAT INTEGRATES WITH OWNCLOUD.md | 80 +++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md diff --git a/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md b/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md new file mode 100644 index 0000000000..0e73bc9daa --- /dev/null +++ b/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md @@ -0,0 +1,80 @@ +QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD +=============== + +[QOwnNotes][1] is a free, open source note taking and todo list application available for Linux, Windows, and Mac. + +The application saves your notes as plain-text files, and it features Markdown support and tight ownCloud integration. + +![](https://2.bp.blogspot.com/-a2vsrOG0zFk/V81gyHWlaaI/AAAAAAAAYZs/uzY16JtNcT8bnje1rTKJx1135WueY6V9gCLcB/s400/qownnotes.png) + +What makes QOwnNotes stand out is its ownCloud integration (which is optional). Using the ownCloud Notes app, you are able to edit and search notes from the web, or from mobile devices (by using an app like [CloudNotes][2]). + +Furthermore, connecting QOwnNotes with your ownCloud account allows you to share notes and access / restore previous versions (or trashed files) of your notes from the ownCloud server. + +In the same way, QOwnNotes can also integrate with the ownCloud tasks or Tasks Plus apps. + +In case you're not familiar with [ownCloud][3], this is a free software alternative to proprietary web services such as Dropbox, Google Drive, and others, which can be installed on your own server. It comes with a web interface that provides access to file management, calendar, image gallery, music player, document viewer, and much more. The developers also provide desktop sync clients, as well as mobile apps. + +Since the notes are saved as plain text, they can be synchronized across devices using other cloud storage services, like Dropbox, Google Drive, and so on, but this is not done directly from within the application. + +As a result, the features I mentioned above, like restoring previous note versions, are only available with ownCloud (although Dropbox, and others, do provide access to previous file revisions, but you won't be able to access this directly from QOwnNotes). + +As for the QOwnNotes note taking features, the app supports Markdown (with a built-in Markdown preview mode), tagging notes, searching in tags and notes, adding links to notes, and inserting images: + +![](https://4.bp.blogspot.com/-SuBhC43gzkY/V81oV7-zLBI/AAAAAAAAYZ8/l6nLQQSUv34Y7op_Xrma8XYm6EdWrhbIACLcB/s400/qownnotes_2.png) + +Hierarchical note tagging and note subfolders are also supported. + +The todo manager feature is pretty basic and could use some improvements, as it currently opens in a separate window, and it doesn't use the same editor as the notes, not allowing you to insert images, or use Markdown. + +![](https://3.bp.blogspot.com/-AUeyZS3s_ck/V81opialKtI/AAAAAAAAYaA/xukIiZZUdNYBVZ92xgKEsEFew7q961CDwCLcB/s400/qownnotes-tasks.png) + +It does allow you to search your todo items, set item priority, add reminders, and show completed items. Also, todo items can be inserted into notes. + +The application user interface is customizable, allowing you to increase or decrease the font size, toggle panes (Markdown preview, note edit and tag panes), and more. A distraction-free mode is also available: + +![](https://4.bp.blogspot.com/-Pnzw1wZde50/V81rrE6mTWI/AAAAAAAAYaM/0UZnH9ktbAgClkuAk1g6fgXK87kB_Bh0wCLcB/s400/qownnotes-distraction-free.png) + +From the application settings, you can enable the dark mode (this was buggy in my test under Ubuntu 16.04 - some toolbar icons were missing), change the toolbar icon size, fonts, and color scheme (light or dark): + +![](https://1.bp.blogspot.com/-K1MGlXA8sxs/V81rv3fwL6I/AAAAAAAAYaQ/YDhhhnbJ9gY38B6Vz1Na_pHLCjLHhPWiwCLcB/s400/qownnotes-settings.png) + +Other QOwnNotes features include encryption support (notes can only be decrypted in QOwnNotes), customizable keyboard shortcuts, export notes to PDF or Markdown, customizable note saving interval, and more. + +Check out the QOwnNotes [homepage][11] for a complete list of features. + + +### Download QOwnNotes + + +For how to install QownNotes, see its [installation][4] page (packages / repositories available for Debian, Ubuntu, Linux Mint, openSUSE, Fedora, Arch Linux, KaOS, Gentoo, Slakware, CentOS, as well as Mac OSX and Windows). + +A QOwnNotes [snap][5] package is also available (in Ubuntu 16.04 and newer, you should be able to install it directly from Ubuntu Software). + +To integrate QOwnNotes with ownCloud you'll need [ownCloud server][6], as well as [Notes][7], [QOwnNotesAPI][8], and [Tasks][9] or [Tasks Plus][10] ownCloud apps. These can be installed from the ownCloud web interface, without having to download anything manually. + +Note that the QOenNotesAPI and Notes ownCloud apps are listed as experimental, so you'll need to enable experimental apps to be able to find and install them. This can be done from the ownCloud web interface, under Apps, by clicking on the settings icon in the lower left-hand side corner. + + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/09/qownnotes-is-note-taking-and-todo-list.html + +作者:[Andrew][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: http://www.qownnotes.org/ +[2]: http://peterandlinda.com/cloudnotes/ +[3]: https://owncloud.org/ +[11]: http://www.qownnotes.org/ +[4]: http://www.qownnotes.org/installation +[5]: https://uappexplorer.com/app/qownnotes.pbek +[6]: https://download.owncloud.org/download/repositories/stable/owncloud/ +[7]: https://github.com/owncloud/notes +[8]: https://github.com/pbek/qownnotesapi +[9]: https://apps.owncloud.com/content/show.php/Tasks?content=164356 +[10]: https://apps.owncloud.com/content/show.php/Tasks+Plus?content=170561 From c8819b56d0f7119db7bcfb3d4e4724b8ecef4dc4 Mon Sep 17 00:00:00 2001 From: Ezio Date: Tue, 6 Sep 2016 09:34:46 +0800 Subject: [PATCH 0050/2437] =?UTF-8?q?20160906-3=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...For LXDE Xfce And MATE - Multiload-ng.md | 102 ++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md diff --git a/sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md b/sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md new file mode 100644 index 0000000000..256c075b21 --- /dev/null +++ b/sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md @@ -0,0 +1,102 @@ +Alternative System Monitor Applet For LXDE, Xfce, And MATE: Multiload-ng +====== + +[Multiload-ng][1] is a GTK2 graphical system monitor for the Xfce, LXDE, and MATE panels, forked from the old GNOME Multiload applet. It can also run in a standalone window. + +![](https://2.bp.blogspot.com/-U8CFzhSPJho/V7GigDbcLWI/AAAAAAAAYS0/pJMM6Rt5-HkbKljmxzP4-v0oGGxjvH8AgCLcB/s1600/multiload-ng-lxde.png) + +Multiload-ng features: + +- supported graphs: CPU, memory, network, swap, load average, disk, and temperature; +- highly customizable; +- color schemes support; +- automatically adapts to container changes (panel or wiondow); +- little CPU / memory footprint; +- basic or detailed tooltip information; +- custom actions on double click. + +Compared to the old Multiload applet, Multiload-ng comes with an additional graph (temperature), more individual graphical customizations, like individual border color, color schemes support, it responds to mouse events with customizable actions, the orientation can be set regardless of panel orientation. + +It can also run in a standalone window, without a panel: + +![](https://1.bp.blogspot.com/-hHoipwFlHrg/V7Gw2s107zI/AAAAAAAAYTQ/fS5OtiL7VvwDEzr6qO_gdEA_qB9YvJa5gCLcB/s400/multiload-ng-standalone.png) + +Furthermore, its GitHub page says that more graphs are coming soon. + +Here's Multiload-ng in Xubuntu 16.04, with a vertical panel, with horizontal and vertical applet orientation: + +![](https://3.bp.blogspot.com/-xa0OML8T-lg/V7Gixksbt8I/AAAAAAAAYS4/Jxo-MukDh3sYlOOk9A1YGtARmte490g8ACLcB/s400/multiload-ng-xfce-horizontal.png) + +![](https://1.bp.blogspot.com/-WAD5MdDObD8/V7GixgVU0DI/AAAAAAAAYS8/uMhHJri1GJccRWvmf_tZkYeenVdxiENQwCLcB/s400/multiload-ng-xfce-vertical.png) + +The applet preferences window isn't exactly pretty, but there are plans to improve it: + +![](https://2.bp.blogspot.com/-P-ophDpc-gI/V7Gi_54b7JI/AAAAAAAAYTA/AHQck_JF_RcwZ1KbgHbaO2JRt24ZZdO3gCLcB/s320/multiload-ng-preferences.png) + +Multiload-ng currently uses GTK2, so it won't work with Xfce or MATE (panels) if they are built with GTK3. + +As far as Ubuntu is concerned, only Ubuntu MATE 16.10 uses GTK3. However, the MATE System Monitor applet is also a fork of Multiload GNOME applet, so they share most features (minus the extra customization provided by Multiload-ng, and the temperature graph). + +The applet wishlist page mentions plans for a GTK3 port, and various other improvements, like more sources for the temperature graph, the ability to show both decimal and binary units, and more. + +### Install Multiload-ng + + +Note that Multiload-ng can't be built on Lubuntu 14.04 due to its dependencies. + +Multiload-ng is available in the main WebUpd8 PPA (for Ubuntu 14.04 - 16.04 / Linux Mint 17.x and 18). To add the PPA and update the software sources, use the following commands: + +``` +sudo add-apt-repository ppa:nilarimogard/webupd8 +sudo apt update +``` + +Then, install the applet using the following command: + +- for LXDE (Lubuntu): + +``` +sudo apt install lxpanel-multiload-ng-plugin +``` + +- for Xfce (Xubuntu, Linux Mint Xfce): + +``` +sudo apt install xfce4-multiload-ng-plugin +``` + +- for MATE (Ubuntu MATE, Linux Mint MATE): + +``` +sudo apt install mate-multiload-ng-applet +``` + +- standalone (doesn't require a panel): + +``` +sudo apt install multiload-ng-standalone +``` + +Once installed, add it to the panel like any other applet. Note that in LXDE, Multiload-ng won't show up in the applet list until the panel is restarted. You can do this by restarting the session (logout/login) or by restarting the panel using the following command: + +``` +lxpanelctl restart +``` + +Multiload-ng Standalone can be launched from the menu, like a regular application. + +To download the source, report bugs, etc., see the Multiload-ng [GitHub page][2]. + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/08/alternative-system-monitor-applet-for.html + +作者:[Andrew][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: https://github.com/udda/multiload-ng +[2]: https://github.com/udda/multiload-ng From f6c8ded145f7b6b8820064588c5a6d73cfe523c0 Mon Sep 17 00:00:00 2001 From: Ezio Date: Tue, 6 Sep 2016 10:01:10 +0800 Subject: [PATCH 0051/2437] =?UTF-8?q?20160906-4=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...0160820 Building your first Atom plugin.md | 757 ++++++++++++++++++ 1 file changed, 757 insertions(+) create mode 100644 sources/tech/20160820 Building your first Atom plugin.md diff --git a/sources/tech/20160820 Building your first Atom plugin.md b/sources/tech/20160820 Building your first Atom plugin.md new file mode 100644 index 0000000000..91a3fa35a6 --- /dev/null +++ b/sources/tech/20160820 Building your first Atom plugin.md @@ -0,0 +1,757 @@ +Building your first Atom plugin +===== + +>Authored by [GitHub Campus Expert][1] @NickTikhonov. + +This tutorial will teach you how to write your first package for the Atom text editor. We'll be building a clone of [Sourcerer][2], a plugin for finding and using code snippets from StackOverflow. By the end of this tutorial you will have written a plugin that converts programming problems written in English into code snippets pulled from StackOverflow: + +![](https://cloud.githubusercontent.com/assets/6755555/17759382/836dd780-64ab-11e6-8f6a-329f66f01fd7.gif) + +### What you need to know + +Atom is written using web technologies. Our package will be built entirely using the EcmaScript 6 standard for JavaScript. You will need to be familiar with: + +- Using the command line +- JavaScript programming +- Promises +- HTTP + +### Tutorial repository + +You can follow this tutorial step-by-step or check out the [supplementary repository on GitHub][3], which contains the plugin source code. The repository history contains one commit for each step outlined here. + +### Getting Started + +#### Installing Atom + +Download Atom by following the instructions on the [Atom website][4]. We will also need to install apm, the Atom Package Manager command line tool. You can do this by opening Atom and navigating to Atom > Install Shell Commands in the application menu. Check that apm was installed correctly by opening your command line terminal and running apm -v, which should print the version of the tool and related environments: + +``` +apm -v +> apm 1.9.2 +> npm 2.13.3 +> node 0.10.40 +> python 2.7.10 +> git 2.7.4 +``` + +#### Generating starter code + +Let's begin by creating a new package using a utility provided by Atom. + +- Launch the editor and press Cmd+Shift+P (on MacOS) or Ctrl+Shift+P (on Windows/Linux) to open the Command Palette. + +- Search for "Package Generator: Generate Package" and click the corresponding item on the list. You will see a prompt where you can enter the name of the package - "sourcefetch". + +- Press enter to generate the starter package, which should automatically be opened in Atom. + +If you don't see package files appear in the sidebar, press Cmd+K Cmd+B (on MacOS) or Ctrl+K Ctrl+B (on Windows/Linux). + +![](https://cloud.githubusercontent.com/assets/6755555/17759387/8387a354-64ab-11e6-97db-ea469f008bef.gif) + +The Command Palette lets you find and run package commands using fuzzy search. This is a convenient way to run commands without navigating menus or remembering shortcuts. We will be using it throughout this tutorial. + +#### Running the starter code + +Let's try out the starter package before diving into the code itself. We will first need to reload Atom to make it aware of the new package that was added. Open the Command Palette again and run the "Window: Reload" command. + +Reloading the current window ensures that Atom runs the latest version of our source code. We will be running this command every time we want to test the changes we make to our package. + +Run the package toggle command by navigating to Packages > sourcefetch > Toggle using the editor menu, or run sourcefetch: Toggle using the Command Palette. You should see a black box appear at the top of the screen. Hide it by running the command again. + +![](https://cloud.githubusercontent.com/assets/6755555/17759386/83799fc0-64ab-11e6-9f0c-0df9b1dbff8b.gif) + +#### The "toggle" command + +Let's open lib/sourcefetch.js, which contains the package logic and defines the toggle command. + +``` +toggle() { + console.log('Sourcefetch was toggled!'); + return ( + this.modalPanel.isVisible() ? + this.modalPanel.hide() : + this.modalPanel.show() + ); +} +``` + +toggle is a function exported by the module. It uses a ternary operator to call show and hide on the modal panel based on its visibility. modalPanel is an instance of Panel, a UI element provided by the Atom API. We declare modalPanel inside export default, which lets us access it as an instance variable with this. + +``` +this.subscriptions.add(atom.commands.add('atom-workspace', { + 'sourcefetch:toggle': () => this.toggle() +})); +``` + +The above statement tells Atom to execute toggle every time the user runs sourcefetch:toggle. We subscribe an anonymous function, () => this.toggle(), to be called every time the command is run. This is an example of event-driven programming, a common paradigm in JavaScript. + +#### Atom Commands + +Commands are nothing more than string identifiers for events triggered by the user, defined within a package namespace. We've already used: + +- package-generator:generate-package +- window:reload +- sourcefetch:toggle + +Packages subscribe to commands in order to execute code in response to these events. + +### Making your first code change + +Let's make our first code change—we're going to change toggle to reverse text selected by the user. + +#### Change "toggle" + +- Change the toggle function to match the snippet below. + +``` +toggle() { + let editor + if (editor = atom.workspace.getActiveTextEditor()) { + let selection = editor.getSelectedText() + let reversed = selection.split('').reverse().join('') + editor.insertText(reversed) + } +} +``` + +#### Test your changes + +- Reload Atom by running Window: Reload in the Command Palette + +- Navigate to File > New to create a new file, type anything you like and select it with the cursor. + +- Run the sourcefetch:toggle command using the Command Palette, Atom menu, or by right clicking and selecting "Toggle sourcefetch" + +The updated command will toggle the order of the selected text: + +![](https://cloud.githubusercontent.com/assets/6755555/17759381/836acd60-64ab-11e6-84dc-4ef4471a361f.gif) + +See all code changes for this step in the [sourcefetch tutorial repository][4]. + +### The Atom Editor API + +The code we added uses the TextEditor API to access and manipulate the text inside the editor. Let's take a closer look. + +``` +let editor +if (editor = atom.workspace.getActiveTextEditor()) { /* ... */ } +``` + +The first two lines obtain a reference to a TextEditor instance. The variable assignment and following code is wrapped in a conditional to handle the case where there is no text editor instance available, for example, if the command was run while the user was in the settings menu. + +``` +let selection = editor.getSelectedText() +``` + +Calling getSelectedText gives us access to text selected by the user. If no text is currently selected, the function returns an empty string. + +``` +let reversed = selection.split('').reverse().join('') +editor.insertText(reversed) +``` + +Our selected text is reversed using [JavaScript String methods][6] . Finally, we call insertText to replace the selected text with the reversed counterpart. You can learn more about the different TextEditor methods available by reading the [Atom API documentation][5]. + +### Exploring the starter package + +Now that we've made our first code change, let's take a closer look at how an Atom package is organized by exploring the starter code. + +#### The main file + +The main file is the entry-point to an Atom package. Atom knows where to find the main file from an entry in package.json: + +``` +"main": "./lib/sourcefetch", +``` + +The file exports an object with lifecycle functions which Atom calls on certain events. + +- activate is called when the package is initially loaded by Atom. This function is used to initialize objects such as user interface elements needed by the package, and to subscribe handler functions to package commands. + +- deactivate is called when the package is deactivated, for example, when the editor is closed or refreshed by the user. + +- serialize is called by Atom to allow you to save the state of the package between uses. The returned value is passed as an argument to activate when the package is next loaded by Atom. + +We are going to rename our package command to fetch, and remove user interface elements we won't be using. Update the file to match the version below: + +``` +'use babel'; + +import { CompositeDisposable } from 'atom' + +export default { + + subscriptions: null, + + activate() { + this.subscriptions = new CompositeDisposable() + + this.subscriptions.add(atom.commands.add('atom-workspace', { + 'sourcefetch:fetch': () => this.fetch() + })) + }, + + deactivate() { + this.subscriptions.dispose() + }, + + fetch() { + let editor + if (editor = atom.workspace.getActiveTextEditor()) { + let selection = editor.getSelectedText() + selection = selection.split('').reverse().join('') + editor.insertText(selection) + } + } +}; +``` + +### Activation commands + +To improve performance, Atom packages can be lazy loading. We can tell Atom to load our package only when certain commands are run by the user. These commands are called activation commands and are defined in package.json: + +``` +"activationCommands": { + "atom-workspace": "sourcefetch:toggle" +}, +``` + +Update this entry to make fetch an activation command. + +``` +"activationCommands": { + "atom-workspace": "sourcefetch:fetch" +}, +``` + +Some packages, such as those which modify Atom's appearance need to be loaded on startup. In those cases, activationCommands can be omitted entirely. + +### Triggering commands + +#### Menu items + +JSON files inside the menus folder specify which menu items are created for our package. Let's take a look at `menus/sourcefetch.json`: + +``` +"context-menu": { + "atom-text-editor": [ + { + "label": "Toggle sourcefetch", + "command": "sourcefetch:toggle" + } + ] +}, +``` + +The context-menu object lets us define new items in the right-click menu. Each item is defined by a label to be displayed in the menu and a command to run when the item is clicked. + +``` +"context-menu": { + "atom-text-editor": [ + { + "label": "Fetch code", + "command": "sourcefetch:fetch" + } + ] +}, +``` + +The menu object in the same file defines custom application menu items created for the package. We're going to rename this entry as well: + +``` +"menu": [ + { + "label": "Packages", + "submenu": [ + { + "label": "sourcefetch", + "submenu": [ + { + "label": "Fetch code", + "command": "sourcefetch:fetch" + } + ] + } + ] + } +] +``` + +#### Keyboard shortcuts + +Commands can also be triggered with keyboard shortcuts, defined with JSON files in the keymaps directory: + +``` +{ + "atom-workspace": { + "ctrl-alt-o": "sourcefetch:toggle" + } +} +``` + +The above lets package users call toggle with Ctrl+Alt+O on Windows/Linux or Cmd+Alt+O on MacOS. + +Rename the referenced command to fetch: + +``` +"ctrl-alt-o": "sourcefetch:fetch" +``` + +Reload Atom by running the Window: Reload command. You should see that the application and right-click menus are updated, and the reverse functionality should work as before. + +See all code changes for this step in the [sourcefetch tutorial repository][7]. + +### Using NodeJS modules + +Now that we've made our first code change and learned about Atom package structure, let's introduce our first dependency—a module from Node Package Manager (npm). We will use the request module to make HTTP requests and download the HTML of a website. This functionality will be needed later, to scrape StackOverflow pages. + +#### Installing dependencies + +Open your command line application, navigate to your package root directory and run: + +``` +npm install --save request@2.73.0 +apm install +``` + +These commands add the request Node module to our dependencies list and install the module into the node_modules directory. You should see a new entry in package.json. The @ symbol tells npm to install the specific version we will be using for this tutorial. Running apm install lets Atom know to use our newly installed module. + +``` +"dependencies": { + "request": "^2.73.0" +} +``` + +#### Downloading and logging HTML to the Developer Console + +Import request into our main file by adding an import statement to the top of lib/sourcefetch.js: + +``` +import { CompositeDisposable } from 'atom' +import request from 'request' +``` + +Now, add a new function, download to the module's exports, below fetch: + +``` +export default { + + /* subscriptions, activate(), deactivate() */ + + fetch() { + ... + }, + + download(url) { + request(url, (error, response, body) => { + if (!error && response.statusCode == 200) { + console.log(body) + } + }) + } +} +``` + +This function uses request to download the contents of a web page and logs the output to the Developer Console. When the HTTP request completes, our callback function will be called with the response as an argument. + +The final step is to update fetch so that it calls download: + +``` +fetch() { + let editor + if (editor = atom.workspace.getActiveTextEditor()) { + let selection = editor.getSelectedText() + this.download(selection) + } +}, +``` + +Instead of reversing the selected text, fetch now treats the selection as a URL, passing it to download. Let's see our changes in action: + +- Reload Atom by running the Window: Reload command. + +- Open the Developer Tools. To do this, navigate to View > Developer > Toggle Developer Tools in the menu. + +- Create a new file, navigate to File > New. + +- Enter and select a URL, for example, http://www.atom.io. + +- Run our package command in any of the three ways previously described: + +![](https://cloud.githubusercontent.com/assets/6755555/17759384/836ea91c-64ab-11e6-8fbe-7d15fb482c6d.gif) + +>Developer Tools make it easy to debug Atom packages. Any console.log statement will print to the interactive console, and you can use the Elements tab to explore the visual structure of the whole applicatio—which is just an HTML [Document Object Model (DOM)][8]. + +See all code changes for this step in the [sourcefetch tutorial repository][9]. + +### Using Promises to insert downloaded HTML into the editor + +Ideally, we would like our download function to return the HTML as a string instead of just printing page contents into the console. Returning body won't work, however, since we get access to body inside of the callback rather than download itself. + +We will solve this problem by returning a Promise rather than the value itself. Let's change download to return a Promise: + +``` +download(url) { + return new Promise((resolve, reject) => { + request(url, (error, response, body) => { + if (!error && response.statusCode == 200) { + resolve(body) + } else { + reject({ + reason: 'Unable to download page' + }) + } + }) + }) +} +``` + +Promises allow us to return values obtained asynchronously by wrapping asynchronous logic in a function that provides two callbacks— resolve for returning a value successfully, and reject for notifying the caller of an error. We call reject if an error is returned by request, and resolve the HTML otherwise. + +Let's change fetch to work with the Promise returned by download: + +``` +fetch() { + let editor + if (editor = atom.workspace.getActiveTextEditor()) { + let selection = editor.getSelectedText() + this.download(selection).then((html) => { + editor.insertText(html) + }).catch((error) => { + atom.notifications.addWarning(error.reason) + }) + } +}, +``` + +In our new version of fetch, we get access to the HTML by calling then on the Promise returned by download. This lets us insert the HTML into the editor. We also accept and handle any errors returned by calling catch. We handle errors by displaying a warning notification using the Atom Notification API. + +Let's see what changed. Reload Atom and run the package command on a selected URL: + +![](https://cloud.githubusercontent.com/assets/6755555/17759379/8357bb08-64ab-11e6-9bd2-6f63b8f50dcc.gif) + +If the command is run on an invalid URL, a warning notification will be displayed: + +![](https://cloud.githubusercontent.com/assets/6755555/17759378/833ab09e-64ab-11e6-9896-2f874b0fdc8a.gif) + +See all code changes for this step in the [sourcefetch tutorial repository][10]. + +#### Building a scraper to extract code snippets from StackOverflow HTML + +The next step involves extracting code snippets from the HTML of a StackOverflow page we obtained in the previous step. In particular, we're interested in code from the accepted answer—an answer chosen to be correct by the question author. We can greatly simplify our package implementation by assuming any such answer to be relevant and correct. + +#### Constructing a query using jQuery and Chrome Developer Tools + +This section assumes you are using the Chrome web browser. You may be able to follow along using another browser, but instructions may change. + +Let's take a look at a typical StackOverflow page that contains an accepted answer with a code snippet. We are going to explore the HTML using Chrome Developer Tools: + +- Open Chrome and navigate to any StackOverflow page containing an accepted answer with code, such as this hello world example in Python or this question about reading text from a file in C. + +- Scroll down to the accepted answer and highlight a section of the code snippet. + +- Right click and select Inspect + +- Inspect the location of the code snippet within the HTML code using the Elements browser. + +Note that the document has the following structure: + +``` +
+ ... + ... +
+        
+          ...snippet elements...
+        
+      
+ ... + ... +
+``` + +- The accepted answer is denoted by a div with class accepted-answer + +- Block code snippets are located inside a pre element + +- Elements that render the code snippet itself sit inside a code tag + +![](https://cloud.githubusercontent.com/assets/6755555/17759380/83689a90-64ab-11e6-89b2-7172c03baae7.gif) + +Now let's construct a jQuery statement for extracting code snippets: + +- Click the Console tab within Developer Tools to access the JavaScript console. + +- Type $('div.accepted-answer pre code').text() into the console and press Enter. + +You should see the accepted answer code snippets printed out in the console. The code we just ran uses a special $ function provided by jQuery. $ accepts a query string to select and return certain HTML elements from the website. Let's take a look at how this code works by considering a couple of intermediate example queries: + +``` +$('div.accepted-answer') +> [
] +``` + +The above query will match all
elements that contain the class accepted-answer, in our case - just one div. + +``` +$('div.accepted-answer pre code') +> [...] +``` + +Building upon the previous, this query will match any `` element that is inside a `
` element contained within the previously matched `
`. + +``` +$('div.accepted-answer pre code').text() +> "print("Hello World!")" +``` + +The text function extracts and concatenates all text from the list of elements that would otherwise be returned by the previous query. This also strips out elements used for syntax highlighting purposes from the code. + +### Introducing Cheerio + +Our next step involves using the query we created to implement a scraping function using Cheerio, a jQuery implementation for server-side applications. + +#### Install Cheerio + +Open your command line application, navigate to your package root directory and run: + +``` +npm install --save cheerio@0.20.0 +apm install +``` + +#### Implement the scraping function + +- Add an import statement for cheerio in lib/sourcefetch.js: + +``` +import { CompositeDisposable } from 'atom' +import request from 'request' +import cheerio from 'cheerio' +``` + +- Now create a new function that extracts code snippets given StackOverflow HTML, called scrape: + +``` +fetch() { + ... +}, + +scrape(html) { + $ = cheerio.load(html) + return $('div.accepted-answer pre code').text() +}, + +download(url) { + ... +} +``` + +- Finally, let's change fetch to pass downloaded HTML to scrape instead of inserting it into the editor: + +``` +fetch() { + let editor + let self = this + + if (editor = atom.workspace.getActiveTextEditor()) { + let selection = editor.getSelectedText() + this.download(selection).then((html) => { + let answer = self.scrape(html) + if (answer === '') { + atom.notifications.addWarning('No answer found :(') + } else { + editor.insertText(answer) + } + }).catch((error) => { + console.log(error) + atom.notifications.addWarning(error.reason) + }) + } +}, +``` + +Our scraping function is implemented in just two lines because cheerio does all of the work for us! We create a $ function by calling load with our HTML string, and use this function to run our jQuery statement and return the results. You can explore the entire Cheerio API in their developer documentation. + +### Testing the updated package + +- Reload Atom and run soucefetch:fetch on a selected StackOverflow URL to see the progress so far. + +If we run the command on a page with an accepted answer, it will be inserted into the editor: + +![](https://cloud.githubusercontent.com/assets/6755555/17759383/836e26b8-64ab-11e6-9f16-321903470ce2.gif) + +If we run the command on a page with no accepted answer, a warning notification will be displayed instead: + + +![](https://cloud.githubusercontent.com/assets/6755555/17759388/838d3864-64ab-11e6-8091-b4d15bd56025.gif) + +Our new iteration of fetch gives us the code snippet within a StackOverflow page instead of the entire HTML contents. Note that our updated fetch function checks for the absence of an answer and displays a notification to alert the user. + +See all code changes for this step in the [sourcefetch tutorial repository][11]. + +### Implementing Google search to find relevant StackOverflow URLs + +Now that we can turn StackOverflow URLs into code snippets, let's implement our final function, search, which will return a relevant URL given the description of a snippet, such as "hello world" or "quicksort". We will be using Google search via the unofficial google npm module, which allows us to search programmatically. + +#### Installing the Google npm module + +- Install google by opening your command line application at the package root directory, and run: + +``` +npm install --save google@2.0.0 +apm install +``` + +#### Importing and configuring the module + +Add an import statement for google at the top of lib/sourcefetch.js: + +``` +import google from "google" +``` + +We will configure the library to limit the number of results returned during search. Add the following line below the import statement to limit returned results to just the top one. + +``` +google.resultsPerPage = 1 +``` + +#### Implementing the search function + +Next, let's implement our search function itself: + +``` +fetch() { + ... +}, + +search(query, language) { + return new Promise((resolve, reject) => { + let searchString = `${query} in ${language} site:stackoverflow.com` + + google(searchString, (err, res) => { + if (err) { + reject({ + reason: 'A search error has occured :(' + }) + } else if (res.links.length === 0) { + reject({ + reason: 'No results found :(' + }) + } else { + resolve(res.links[0].href) + } + }) + }) +}, + +scrape() { + ... +} +``` + +The code above searches Google for a StackOverflow page relevant to the given query and programming language, returning the URL of the top result. Let's take a look at how it works: + +``` +let searchString = `${query} in ${language} site:stackoverflow.com` +``` + +We construct the search string using the query entered by the user and the current language selected. For example, if the user types "hello world" while editing Python, the query will be hello world in python site:stackoverflow.com. The final part of the string is a filter provided by Google Search that lets us limit results to those linked to StackOverflow. + +``` +google(searchString, (err, res) => { + if (err) { + reject({ + reason: 'A search error has occured :(' + }) + } else if (res.links.length === 0) { + reject({ + reason: 'No results found :(' + }) + } else { + resolve(res.links[0].href) + } +}) +``` + +We wrap the call to google inside a Promise so that we can return our URL asynchronously. We propagate any errors returned by the library, also returning an error when there are no results available. We resolve the URL of the top result otherwise. + +### Updating fetch to use search + +Our final step is to update fetch to use search: + +``` +fetch() { + let editor + let self = this + + if (editor = atom.workspace.getActiveTextEditor()) { + let query = editor.getSelectedText() + let language = editor.getGrammar().name + + self.search(query, language).then((url) => { + atom.notifications.addSuccess('Found google results!') + return self.download(url) + }).then((html) => { + let answer = self.scrape(html) + if (answer === '') { + atom.notifications.addWarning('No answer found :(') + } else { + atom.notifications.addSuccess('Found snippet!') + editor.insertText(answer) + } + }).catch((error) => { + atom.notifications.addWarning(error.reason) + }) + } +} +``` + +Let's take a look at what changed: + +- Our selected text is now treated as the query entered by the user. + +- We obtain the language of the current editor tab using the TextEditor API + +- We call search to obtain a URL, which we access by calling then on the resulting Promise + +Instead of calling then on the Promise returned by download, we instead return the Promise itself and chain another then call onto the original call. This helps us avoid callback hell + +See all code changes for this step in the [sourcefetch tutorial repository][12]. + +### Testing the final plugin + +And we're done! See the final plugin in action by reloading Atom and running our package command on a problem description, and don't forget to select a language in the bottom-right corner. + +![](https://cloud.githubusercontent.com/assets/6755555/17759382/836dd780-64ab-11e6-8f6a-329f66f01fd7.gif) + +### Next steps + +Now that you know the basics of hacking Atom, feel free to practice what you've learned [by forking the sourcefetch repository and adding your own features][13]. + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/09/qownnotes-is-note-taking-and-todo-list.html + +作者:[NickTikhonov][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://github.com/NickTikhonov +[1]: https://education.github.com/experts +[2]: https://github.com/NickTikhonov/sourcerer +[3]: https://github.com/NickTikhonov/sourcefetch-guide +[4]: https://github.com/NickTikhonov/sourcefetch-tutorial/commit/89e174ab6ec6e270938338b34905f75bb74dbede +[5]: https://atom.io/docs/api/latest/TextEditor +[6]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String +[7]: https://github.com/NickTikhonov/sourcefetch-tutorial/commit/aa3ec5585b0aa049393351a30be14590df09c29a +[8]: https://www.wikipedia.com/en/Document_Object_Model +[9]: https://github.com/NickTikhonov/sourcefetch-tutorial/commit/85992043e57c802ca71ff6e8a4f9c477fbfd13db +[10]: https://github.com/NickTikhonov/sourcefetch-tutorial/commit/896d160dca711f4a53ff5b182018b39cf78d2774 +[11]: https://github.com/NickTikhonov/sourcefetch-tutorial/commit/039a1e1e976d029f7d6b061b4c0dac3eb4a3b5d2 +[12]: https://github.com/NickTikhonov/sourcefetch-tutorial/commit/aa9d0b5fc4811a70292869730e0f60ddf0bcf2aa +[13]: https://github.com/NickTikhonov/sourcefetch-tutorial From 5689a15b2093897f6ebba85884248e676486defe Mon Sep 17 00:00:00 2001 From: Ezio Date: Tue, 6 Sep 2016 10:01:58 +0800 Subject: [PATCH 0052/2437] =?UTF-8?q?20160906-4=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- sources/tech/20160820 Building your first Atom plugin.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/tech/20160820 Building your first Atom plugin.md b/sources/tech/20160820 Building your first Atom plugin.md index 91a3fa35a6..76915cc37d 100644 --- a/sources/tech/20160820 Building your first Atom plugin.md +++ b/sources/tech/20160820 Building your first Atom plugin.md @@ -733,7 +733,7 @@ Now that you know the basics of hacking Atom, feel free to practice what you've -------------------------------------------------------------------------------- -via: http://www.webupd8.org/2016/09/qownnotes-is-note-taking-and-todo-list.html +via: https://github.com/blog/2231-building-your-first-atom-plugin 作者:[NickTikhonov][a] 译者:[译者ID](https://github.com/译者ID) From 362918d8780928b799e576dc0cc4fefadaa493b8 Mon Sep 17 00:00:00 2001 From: wxy Date: Tue, 6 Sep 2016 13:44:24 +0800 Subject: [PATCH 0053/2437] PUB:20160824 JSON support is generally available in Azure SQL Database @geekpi --- ...nerally available in Azure SQL Database.md | 53 +++++++++++++++++ ...nerally available in Azure SQL Database.md | 57 ------------------- 2 files changed, 53 insertions(+), 57 deletions(-) create mode 100644 published/20160824 JSON support is generally available in Azure SQL Database.md delete mode 100644 translated/tech/20160824 JSON support is generally available in Azure SQL Database.md diff --git a/published/20160824 JSON support is generally available in Azure SQL Database.md b/published/20160824 JSON support is generally available in Azure SQL Database.md new file mode 100644 index 0000000000..42c38958fc --- /dev/null +++ b/published/20160824 JSON support is generally available in Azure SQL Database.md @@ -0,0 +1,53 @@ +Azure SQL 数据库已经支持 JSON +=========== + +我们很高兴地宣布你现在可以在 Azure SQL 中查询及存储关系型数据或者 JSON 了、Azure SQL 数据库提供了读取 JSON 文本数据的简单的内置函数,将 JSON 文本转化成表,以及将表的数据转化成 JSON。 + +![](https://azurecomcdn.azureedge.net/mediahandler/acomblog/media/Default/blog/1cc536a5-a822-467b-a4a2-4557746f70cc.png) + +你可以使用 JSON 函数来从 JSON 文本中提取值([JSON_VALUE][4])、提取对象([JSON_QUERY][5]), 更新JSON 中的值([JSON_MODIFY][6]),并且验证 JSON 文本的正确性([ISJSON][7])。[OPENJSON][8] 函数让你可以将 JSON 文本转化成表结构。最后,JSON 功能函数可以让你很简单地从 SQL 查询中使用 [FOR JSON][9] 从句来获得 JSON 文本结果。 + +### 你可以用 JSON 做什么? + +Azure SQL 数据库中的 JSON 可以让您构建并与现代 web、移动设备和 HTML5/单页应用、诸如 Azure DocumentDB 等包含 JSON 格式化数据的 NoSQL 存储等交换数据,分析来自不同系统和服务的日志和消息。现在你可以轻易地将 Azure SQL 数据库与任何使用使用 JSON 的服务集成。 + +#### 轻易地开放数据给现代框架和服务 + +你有没有在使用诸如 REST 或者 Azure App 使用 JSON 来交换数据的服务?你有使用诸如 AngularJS、ReactJS、D3 或者 JQuery 等使用 JSON 的组件或框架么?使用新的 JSON 功能函数,你可以轻易地格式化存储在 Azure SQL 数据库中的数据,并将它用在任何现代服务或者应用中。 + +#### 轻松采集 JSON 数据 + +你有在使用移动设备、传感器、如 Azure Stream Analytics 或者 Insight 这样产生 JSON 的服务、如 Azure DocumentDB 或者 MongoDB 这样存储 JSON 的系统么?你需要在 Azure SQL 数据中使用熟悉的 SQL 语句来查询并分析 JSON 数据么?现在你可以轻松采集 JSON 数据并存储到 Azure SQL 数据库中,并且可以使用任何 Azure SQL 数据库支持的语言或者工具来查询和分析加载的数据。 + +#### 简化你的数据模型 + +你需要同时存储及查询数据库中关系型及半结构化的数据么?你需简化像 NoSQL 平台下的数据模型么?现在你可以在一张表中同时存储结构化数据及非结构化数据了。在 Azure SQL 数据库中,你可以同时从关系型及 NoSQL 的世界中使用最好的方法来调整你的数据模型。Azure SQL 数据库让你可以使用 Transact-SQL 语言来查询关系及 JSON 数据。程序和工具将不会在从表中取出的值及 JSON 文本中提取的值看出差别。 + +### 下一步 + +要学习如何在你的应用中集成 JSON,查看我们的[开始学习][1]页面或者 [Channel 9的视频][2]。要了解不同的情景下如何集成 JSON,观看 Channel 9 的视频或者在这些 [JSON 分类文章][3]中查找你感兴趣的使用情景。 + +我们将持续增加新的 JSON 特性并让 JSON 支持的更好。请敬请关注。 + + +-------------------------------------------------------------------------------- + +via: https://azure.microsoft.com/en-us/blog/json-support-is-generally-available-in-azure-sql-database/ + +作者:[Jovan Popovic][a] +译者:[geekpi](https://github.com/geekpi) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://azure.microsoft.com/en-us/blog/author/jovanpop/ +[1]: https://azure.microsoft.com/en-us/documentation/articles/sql-database-json-features/ +[2]: https://channel9.msdn.com/Shows/Data-Exposed/SQL-Server-2016-and-JSON-Support +[3]: http://blogs.msdn.com/b/sqlserverstorageengine/archive/tags/json/ +[4]: https://msdn.microsoft.com/en-us/library/dn921898.aspx +[5]: https://msdn.microsoft.com/en-us/library/dn921884.aspx +[6]: https://msdn.microsoft.com/en-us/library/dn921892.aspx +[7]: https://msdn.microsoft.com/en-us/library/dn921896.aspx +[8]: https://msdn.microsoft.com/en-us/library/dn921885.aspx +[9]: https://msdn.microsoft.com/en-us/library/dn921882.aspx + diff --git a/translated/tech/20160824 JSON support is generally available in Azure SQL Database.md b/translated/tech/20160824 JSON support is generally available in Azure SQL Database.md deleted file mode 100644 index 1368340312..0000000000 --- a/translated/tech/20160824 JSON support is generally available in Azure SQL Database.md +++ /dev/null @@ -1,57 +0,0 @@ -Azure SQL数据库已经支持JSON -=========== - -我们很高兴地宣布你现在可以在Azure SQL中查询及存储关系型数据或者JSON了、Azure SQL数据库提供了简单的内置函数来读取JSON文本数据,将JSON文本转化成表,将表的数据转化成JSON。 - -![](https://azurecomcdn.azureedge.net/mediahandler/acomblog/media/Default/blog/1cc536a5-a822-467b-a4a2-4557746f70cc.png) - - -你可以使用JSON函数来从JSON文本中提取值(JSON_VALUE), 提取对象(JSON_QUERY), 更新JSON中的值(JSON_MODIFY),并且验证JSON文本的正确性(ISJSON)。OPENJSON函数让你可以将JSON文本转化成表结构。最后,JSON功能函数可以让你很简单地从SQL查询中使用FOR JSON从句来获得JSON文本结果。 - -### 你可以用JSON做什么? - - -Azure SQL数据库中的JSON可以让您构建并与现代web、移动设备和HTML5/单页应用、NoSQL中存储的诸如Azure DocumentDB中的格式化成JSON的数据交换数据,分析来自不同系统和服务的日志和消息。现在你可以轻易地将Azure SQL数据库与任何使用使用JSON的服务集成。 - -#### 轻易地开放数据给现代框架和服务 - -你有没有在使用诸如REST或者Azure App使用JSON来交换数据的服务?你有使用诸如AngularJS、ReactJS、D3或者Jquery使用JSON的组件或框架么?使用新的JSON功能函数,你可以轻易地格式化存储在Azure SQL数据库中的数据,并将它开放在任何现代服务或者应用。 - -#### 轻松采集JSON数据 - -你有在使用移动设备、传感器、如Azure Stream Analytics或者Insight这样产生JSON的服务、如Azure DocumentDB 或者MongoDB这样存储JSON的系统么?你需要在Azure SQL数据中使用熟悉的SQL语句来查询并分析JSON数据么?现在你可以轻松采集JSON数据并存储到Azure SQL数据库中,并且可以使用任何Azure SQL数据库支持的语言或者工具来查询和分析加载的数据 - -#### 简化你的数据模型 - -你需要同时存储及查询数据库中关系型及半结构化的数据么?你需简化像NoSQL平台下的数据模型么?现在你可以在一张表中同时存储结构化数据及非结构化数据了。在Azure SQL数据库中,你可以同时从关系型及NoSQL的世界中使用最好的方法来调整你的数据模型。Azure SQL数据库让你可以使用Transact-SQL语言来查询关系及JSON数据。程序和工具将不会在从表中取出的值及JSON文本中提取的值看出差别。 - -### 下一步 - -要学习如何在你的应用中集成JSON,查看我们的[开始学习][1]页面或者[Channel 9的视频][2]。要了解不同的情景下如何集成JSON,观看Channel 9的视频或者在这些[JSON分类文章][3]中查找你感兴趣的使用情景。 - -我们将持续增加新的JSON特性并让JSON支持的更好。请敬请关注。 - - - - --------------------------------------------------------------------------------- - -via: https://azure.microsoft.com/en-us/blog/json-support-is-generally-available-in-azure-sql-database/?utm_source=dbweekly&utm_medium=email - -作者:[Jovan Popovic][a] -译者:[geekpi](https://github.com/geekpi) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://azure.microsoft.com/en-us/blog/author/jovanpop/ -[1]: https://azure.microsoft.com/en-us/documentation/articles/sql-database-json-features/ -[2]: https://channel9.msdn.com/Shows/Data-Exposed/SQL-Server-2016-and-JSON-Support -[3]: http://blogs.msdn.com/b/sqlserverstorageengine/archive/tags/json/ - - - - - - - From ab68845005dfc0329c0ed00d49caef1b43c1af76 Mon Sep 17 00:00:00 2001 From: rusking Date: Tue, 6 Sep 2016 15:26:31 +0800 Subject: [PATCH 0054/2437] Delete 20160728 I've been Linuxing since before you were born.md --- ...een Linuxing since before you were born.md | 68 ------------------- 1 file changed, 68 deletions(-) delete mode 100644 sources/talk/20160728 I've been Linuxing since before you were born.md diff --git a/sources/talk/20160728 I've been Linuxing since before you were born.md b/sources/talk/20160728 I've been Linuxing since before you were born.md deleted file mode 100644 index 2466813a7d..0000000000 --- a/sources/talk/20160728 I've been Linuxing since before you were born.md +++ /dev/null @@ -1,68 +0,0 @@ -rusking 翻译中 - -I've been Linuxing since before you were born -===================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_Penguin_Image_520x292_12324207_0714_mm_v1a.png?itok=WfAkwbFy) - -Once upon a time, there was no Linux. No, really! It did not exist. It was not like today, with Linux everywhere. There were multiple flavors of Unix, there was Apple, and there was Microsoft Windows. - -When it comes to Windows, the more things change, the more they stay the same. Despite adding 20+ gigabytes of gosh-knows-what, Windows is mostly the same. (Except you can't drop to a DOS prompt to get actual work done.) Hey, who remembers Gorilla.bas, the exploding banana game that came in DOS? Fun times! The Internet never forgets, and you can play a Flash version on Kongregate.com. - -Apple changed, evolving from a friendly system that encouraged hacking to a sleek, sealed box that you are not supposed to open, and that dictates what hardware interfaces you are allowed to use. 1998: no more floppy disk. 2012: no more optical drive. The 12-inch MacBook has only a single USB Type-C port that supplies power, Bluetooth, Wi-Fi, external storage, video output, and accessories. If you want to plug in more than one thing at a time and don't want to tote a herd of dongles and adapters around with you, too bad. Next up: The headphone jack. Yes, the one remaining non-proprietary standard hardware port in Apple-land is doomed. - -There was a sizable gaggle of other operating systems such as Amiga, BeOS, OS/2, and dozens more that you can look up should you feel so inclined, which I encourage because looking things up is so easy now there is no excuse not to. Amiga, BeOS, and OS/2 were noteworthy for having advanced functionality such as 32-bit multitasking and advanced graphics handling. But marketing clout defeats higher quality, and so the less-capable Apple and Windows dominated the market while the others faded away. - -Then came Linux, and the world changed. - -### First PC - -The first PC I ever used was an Apple IIc, somewheres around 1994, when Linux was three years old. A friend loaned it to me and it was all right, but too inflexible. Then I bought a used Tandy PC for something like $500, which was sad for the person who sold it because it cost a couple thousand dollars. Back then, computers depreciated very quickly. It was a monster: an Intel 386SX CPU, 4 megabytes RAM, a 107-megabyte hard drive, 14-inch color CRT monitor, running MS-DOS 5 and Windows 3.1. - -I tore that poor thing apart multiple times and reinstalled Windows and DOS many times. Windows was marginally usable, so I did most of my work in DOS. I loved gory video games and played Doom, Duke Nukem, Quake, and Heretic. Ah, those glorious, jiggedy 8-bit graphics. - -In those days the hardware was always behind the software, so I upgraded frequently. Now we have all the horsepower we need and then some. I haven't had to upgrade the hardware in any of my computers for several years. - -### Computer bits - -Back in those golden years, there were independent computer stores on every corner and you could hardly walk down the block without tripping over a local independent Internet service provider (ISP). ISPs were very different then. They were not horrid customer-hostile megacorporations like our good friends the American telcos and cable companies. They were nice, and offered all kinds of extra services like Bulletin Board Services (BBS), file downloads, and Multi-User Domains (MUDs), which were multi-player online games. - -I spent a lot of time buying parts in the computer stores, and half the fun was shocking the store staff by being a woman. It is puzzling how this is so upsetting to some people. Now I'm an old fart of 58 and they're still not used to it. I hope that being a woman nerd will become acceptable by the time I die. - -Those stores had racks of Computer Bits magazine. Check out this old issue of Computer Bits on the Internet Archive. Computer Bits was a local free paper with good articles and tons of ads. Unfortunately, the print ads did not appear in the online edition, so you can't see how they included a wealth of detailed information. You know how advertisers are so whiny, and complain about ad blockers, and have turned tech journalism into barely-disguised advertising? They need to learn some lessons from the past. Those ads were useful. People wanted to read them. I learned everything about computer hardware from reading the ads in Computer Bits and other computer magazines. Computer Shopper was an especially fabulous resource; it had several hundred pages of ads and high-quality articles. - -![](https://opensource.com/sites/default/files/resize/march2002-300x387.jpg) - -The publisher of Computer Bits, Paul Harwood, launched my writing career. My first ever professional writing was for Computer Bits. Paul, if you're still around, thank you! - -You can see something in the Internet Archives of Computer Bits that barely exists anymore, and that is the classified ads section. Classified ads generated significant income for print publications. Craigslist killed the classifieds, which killed newspapers and publications like Computer Bits. - -One of my cherished memories is when the 12-year-old twit who ran my favorite computer store, who could never get over my blatant woman-ness and could never accept that I knew what I was doing, handed me a copy of Computer Bits as a good resource for beginners. I opened it to show him one of my Linux articles and said "Oh yes, I know." He turned colors I didn't think were physiologically possible and scuttled away. (No, my fine literalists, he was not really 12, but in his early 20s. Perhaps by now his emotional maturity has caught up a little.) - -### Discovering Linux - -I first learned about Linux in Computer Bits magazine, maybe around 1997 or so. My first Linuxes were Red Hat 5 and Mandrake Linux. Mandrake was glorious. It was the first easy-to-install Linux, and it bundled graphics and sound drivers so I could immediately play Tux Racer. Unlike most Linux nerds at the time I did not come from a Unix background, so I had a steep learning curve. But that was alright, because everything I learned was useful. In contrast to my Windows adventures, where most of what I learned was working around its awfulness, then giving up and dropping to a DOS shell. - -Playing with computers was so much fun I drifted into freelance consulting, forcing Windows computers to more or less function, and helping IT staff in small shops migrate to Linux servers. Usually we did this on the sneak, because those were the days of Microsoft calling Linux a cancer, and implying that it was a communist conspiracy to sap and impurify all of our precious bodily fluids. - -### Linux won - -I continued consulting for a number of years, doing a little bit of everything: Repairing and upgrading hardware, pulling cable, system and network administration, and running mixed Apple/Windows/Linux networks. Apple and Windows were absolutely hellish to integrate into mixed networks as they deliberately tried to make it impossible. One of the coolest things about Linux and FOSS is there is always someone ready and able to defeat the barriers erected by proprietary vendors. - -It is very different now. There are still proprietary barriers to interoperability, and there is still no Tier 1 desktop Linux OEM vendor. In my opinion this is because Microsoft and Apple have a hard lock on retail distribution. Maybe they're doing us a favor, because you get way better service and quality from wonderful independent Linux vendors like ZaReason and System76. They're Linux experts, and they don't treat us like unwelcome afterthoughts. - -Aside from the retail desktop, Linux dominates all aspects of computing from embedded to supercomputing and distributed computing. Open source dominates software development. All of the significant new frontiers in software, such as containers, clusters, and artificial intelligence are all powered by open source software. When you measure the distance from my first little old 386SX PC til now, that is phenomenal progress. - -It would not have happened without Linux and open source. - --------------------------------------------------------------------------------- - -via: https://opensource.com/life/16/7/my-linux-story-carla-schroder - -作者:[Carla Schroder ][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/carlaschroder From 7bbb5a5e2489b4c701e493b1b988b51a9f9b412c Mon Sep 17 00:00:00 2001 From: rusking Date: Tue, 6 Sep 2016 15:29:33 +0800 Subject: [PATCH 0055/2437] =?UTF-8?q?I've=20been=20Linuxing=20since=20befo?= =?UTF-8?q?re=20you=20were=20born=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=AF=95?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 翻译完毕 --- ...e been Linuxing since before you were born | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 translated/talk/I've been Linuxing since before you were born diff --git a/translated/talk/I've been Linuxing since before you were born b/translated/talk/I've been Linuxing since before you were born new file mode 100644 index 0000000000..2ff2df8c83 --- /dev/null +++ b/translated/talk/I've been Linuxing since before you were born @@ -0,0 +1,68 @@ + +我玩 Linux 那会儿你还不知道在哪呢 +===================== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_Penguin_Image_520x292_12324207_0714_mm_v1a.png?itok=WfAkwbFy) + +在很久以前,世界上没有 Linux 系统。真的没有!之前也从未存在过。不像现在, Linux 系统随处可见。有各种风格的 Unix 系统,有苹果的操作系统,有微软的 Windows 操作系统。 + +随时 Windows 桌面的到来,改的东西很多,但是保留下来的东西的更多。尽管增加至20GB以上的存储,Windows 还是大体保持不变。(除了不能在DOS的提示下完成实际的工作)嘿,谁还记得 Gorilla.bas 那个出现在 DOS 系统里的炸香蕉的游戏吗?多么美好的时光啊!互联网永不会忘记,而且你可以在 Kongregate.com 这个网站玩Flash版本的这个游戏。 + +苹果系统也改变了,变得更加友好、安全、美观。你别想着打开这个密封的盒子,它强制规定了用户能使用的硬件接口。 1998年:没有软盘。 2012年:没有光驱。 12英寸的 MacBook 只有一个提供电源的USB Type-C 端口、蓝牙、无线网卡、外部存储、视频输出接口和其它的一些配件。如果你想每次想接入一个新的设备而不想使用其它的配件和软件,这真是太抓狂了。其它:头戴式耳机。这是个非独有的硬件接口在 Apple 的领域里注定保留下来。 + +还有其它的操作系统,比如: Amiga, BeOS, OS/2 以及无数的你能查到的操作系统。我支持的这些系统是因为现在都很容易查寻到,也没有理由不推荐它们。 Amiga、 BeOS 和 OS/2 这些系统都值得关注,因为它们有强大的功能,比如32位的多任务和高级图形处理能力。但是市场的影响击败了强大的系统性能,因此普通的 Apple 和 Windows 操作系统统治了市场的主导地位,而那些曾经的系统也逐渐销声匿迹了。 + +然后 Linux 系统出现了,世界也因此改变了。 + +### 第一款电脑 + +我曾经使用过的第一款电脑是 Apple IIc ,大概在 1994年左右,那个时候 Linux 系统刚出来 3年。这是我从一个朋友那里借来的,用起来各方面都还不错,但是很不方便。所以我自己花了将近 500美元买了一台二手的 Tandy牌子的电脑。这对于卖电脑的人来说是一件很伤心的事,因为新电脑要花费双倍的价钱。那个时候,电脑贬值的速度非常快。这个电脑看起来像是个怪物:一个英特尔 386SX 的 CPU, 4MB的内存,一个 107MB的硬盘, 14英寸的彩色 CRT 显示器,运行 MS-DOS 5和 Windows 3.1系统。 + +我曾无数次的拆开那个可怜的怪物,并且多次重新安装 Windows 和 DOS 系统。因为 Windows 桌面用的比较少,所以我的大部分工作都是在 DOS 下完成的。我喜欢玩血腥暴力的视频游戏,包括 Doom, Duke Nukem,Quake 和 Heretic。啊!那些美好的,让人心动的 8位图像。 + +那个时候硬件的发展一直落后于软件,因此我经常升级硬件。现在我们能买到满足我们需求的任何配置的电脑。我已经好多年都没有再更新我的任何电脑硬件了。 + +### 比特杂志 + +回到那些曾经辉煌的年代,电脑商店布满大街小巷,找到本地的一家网络服务提供商(ISP)也不用走遍整条街。ISP 那个时候真的非比寻常。他们都是讨人喜欢的一些友好的大公司,像美国电信公司和有线电视公司等。他们都非常友好,并且提供各种各样的像 BBS、文件下载、MUDs (多玩家在线游戏)等的额外服务。 + +我花了很多的时间在电脑商店购买配件,但是很多时候我一个女人家去那里会让店里的员工感到吃惊,我真的很无语了,这怎么就会让一些人看不惯了。我现成已经是一位58岁的老奶了,但是他们还是一样的看不惯我。我希望我这个让人讨厌的女人在我死之前能被他们所接受。 + +那些商店的书架上摆满了比特杂志。有关比特杂志的历史信息可以在互联网档案库中查到。比特杂志是当地一家免费的杂志,有很多关于计算机方面的优秀的文章和大量的广告。可惜当时的广告没有网络版,因此大家不能再看到那些很有价值的关于计算机方面的详细信息了。你知道现在的广告商们有多么的抓狂吗?他们埋怨那些排斥广告的人,致使他们不得不在科技新闻中植入一些隐藏的广告。他们应该学习一下过去的广告模式,那时候的广告里有很多有价值的信息,大家都喜欢阅读。我从比特杂志和其它的电脑杂志的广告中学到了所有关于计算机硬件的知识。电脑购买者杂志更是非常好的学习资料,其中有上百页的广告和很多高质量的文章。 + + +![](https://opensource.com/sites/default/files/resize/march2002-300x387.jpg) + +比特杂志的出版者 Paul Harwood 开启了我的写作生涯。我的第一篇计算机专业性质的文章就是在比特杂志发表的。 Paul,仿佛你一直都在我的身旁,谢谢你。 + +在互联网档案库中,关于比特杂志的分类广告信息已经几乎查询不到了。分类广告模式曾经给出版商带来巨大的收入。免费的分类广告网站 Craigslist 在这个领域独占鳌头,同时也扼杀了像比特杂志这种以传统的报纸和出版为主的杂志行业。 + +其中有一些让我最难以忘怀的记忆就是一个12岁左右的吊儿郎当的小屁孩,匆匆忙忙地跑到我钟爱的电脑店里递给我一本初学者好书比特杂志,他脸上挂满了对我这种光鲜亮丽的女人所做工作的不屑和不理解的表情。我拆开杂志,指着其中一篇我写的关于Linux系统的文章给他看,他说“哦,确实是这样的,我明白了”。他尴尬的脸变成了那种正常生理上都不可能呈现的颜色,然后很仓促地夹着尾巴溜走了。(不是的,我亲爱的、诚实的读者们,他不是真的只有12岁,而应该是20来岁。他现在应该比以前更成熟懂事一些了吧!) + +###发现 Linux + +我第一次了解到 Linux 系统是在比特杂志上,大概在 1977年左右。我一开始用的一些 Linux操作系统版本是 Red Hat 5 和 Mandrake Linux。 Mandrake真是太棒了,它是第一款易安装型的Linux系统,并且还附带图形界面和声卡驱动,因此我可以快速玩转 Tux Racer 游戏。不像那时候大多数的有教育背景的 Linux 用户那样,因为我之前没接触过 Unix系统,所以我学习起来比较难。但是一切都还顺利吧,因为我学到的东西都很有用。相对于学习 Windows 系统来说,我在 Linux 系统里学到的东西都很实用,也没必要像在Windows环境中那样非得把任务拖放到DOS下来执行。 + +玩转电脑真是充满了太多的乐趣,后来我转行成为计算机自由顾问,去帮助一些小型公司的IT部门把数据迁移到 Linux 服务器上,这迫使 Windows 系统或多或少的失去一些用武之地。通常情况下我们都是在背地里偷偷地做这些工作的,因为那段时期微软把 Linux 称为毒瘤,诬蔑 Linux 系统是一种共产主义用来削弱和吞噬我们身体里的珍贵血液的阴谋。 + +### Linux 赢了 + +我持续做了很多年的顾问工作,也做其它一些相关的工作,比如:电脑硬件修理和升级、布线、系统和网络管理,还有其它一些混合的包括 Apple、Windows、Linux 系统在内的网络工作。Apple 和 Windows 系统故意不兼容对方,因此这两个系统真的是最头疼,也最难整合到同一网络中。但是在Linux系统和其它开源软件中最有趣的一件事是总有一些人能够时时处理这些厂商之间的兼容性问题。 + +现在已经大不同了。在系统间的互操作方面一直存在一些兼容性问题,而且也没有1级桌面的 Linux 系统 OEM 厂商。我觉得这是因为微软和苹果公司在零售行业在大力垄断造成的。也许他们这样做反而帮了我们帮大忙,因为我们可以从 ZaReason 和 System76 这样独立的 Linux 系统开发商得到更好的服务和更高性能的系统。他们对 Linux 系统都很专业,也特别欢迎我们这样的用户。 + +Linux 除了在零售行业的桌面版系统占有一席之地以外,从嵌入式系统到超级计算机以及分布式计算系统等各个方面都占据着主要地位。开源技术掌握着软件行业的发展方向,所有的软件行业的前沿技术,比如容器、集群以及人工智能的发展都离不开开源技术的支持。从我的第一台老式的 386SX 电脑到现在,Linux 和开源技术都取得了巨大的进步。 + +如果没有 Linux 系统和开源,这一切都不可能发生。 + +-------------------------------------------------------------------------------- + +via: https://opensource.com/life/16/7/my-linux-story-carla-schroder + +作者:[Carla Schroder ][a] +译者:[译者ID](https://github.com/rusking) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/carlaschroder From 59e73eb151e7a67f3bd8b98ed519d3f37ace4efd Mon Sep 17 00:00:00 2001 From: rusking Date: Tue, 6 Sep 2016 12:24:34 +0800 Subject: [PATCH 0056/2437] Delete 20160728 I've been Linuxing since before you were born.md --- ...een Linuxing since before you were born.md | 68 ------------------- 1 file changed, 68 deletions(-) delete mode 100644 sources/talk/20160728 I've been Linuxing since before you were born.md diff --git a/sources/talk/20160728 I've been Linuxing since before you were born.md b/sources/talk/20160728 I've been Linuxing since before you were born.md deleted file mode 100644 index 2466813a7d..0000000000 --- a/sources/talk/20160728 I've been Linuxing since before you were born.md +++ /dev/null @@ -1,68 +0,0 @@ -rusking 翻译中 - -I've been Linuxing since before you were born -===================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_Penguin_Image_520x292_12324207_0714_mm_v1a.png?itok=WfAkwbFy) - -Once upon a time, there was no Linux. No, really! It did not exist. It was not like today, with Linux everywhere. There were multiple flavors of Unix, there was Apple, and there was Microsoft Windows. - -When it comes to Windows, the more things change, the more they stay the same. Despite adding 20+ gigabytes of gosh-knows-what, Windows is mostly the same. (Except you can't drop to a DOS prompt to get actual work done.) Hey, who remembers Gorilla.bas, the exploding banana game that came in DOS? Fun times! The Internet never forgets, and you can play a Flash version on Kongregate.com. - -Apple changed, evolving from a friendly system that encouraged hacking to a sleek, sealed box that you are not supposed to open, and that dictates what hardware interfaces you are allowed to use. 1998: no more floppy disk. 2012: no more optical drive. The 12-inch MacBook has only a single USB Type-C port that supplies power, Bluetooth, Wi-Fi, external storage, video output, and accessories. If you want to plug in more than one thing at a time and don't want to tote a herd of dongles and adapters around with you, too bad. Next up: The headphone jack. Yes, the one remaining non-proprietary standard hardware port in Apple-land is doomed. - -There was a sizable gaggle of other operating systems such as Amiga, BeOS, OS/2, and dozens more that you can look up should you feel so inclined, which I encourage because looking things up is so easy now there is no excuse not to. Amiga, BeOS, and OS/2 were noteworthy for having advanced functionality such as 32-bit multitasking and advanced graphics handling. But marketing clout defeats higher quality, and so the less-capable Apple and Windows dominated the market while the others faded away. - -Then came Linux, and the world changed. - -### First PC - -The first PC I ever used was an Apple IIc, somewheres around 1994, when Linux was three years old. A friend loaned it to me and it was all right, but too inflexible. Then I bought a used Tandy PC for something like $500, which was sad for the person who sold it because it cost a couple thousand dollars. Back then, computers depreciated very quickly. It was a monster: an Intel 386SX CPU, 4 megabytes RAM, a 107-megabyte hard drive, 14-inch color CRT monitor, running MS-DOS 5 and Windows 3.1. - -I tore that poor thing apart multiple times and reinstalled Windows and DOS many times. Windows was marginally usable, so I did most of my work in DOS. I loved gory video games and played Doom, Duke Nukem, Quake, and Heretic. Ah, those glorious, jiggedy 8-bit graphics. - -In those days the hardware was always behind the software, so I upgraded frequently. Now we have all the horsepower we need and then some. I haven't had to upgrade the hardware in any of my computers for several years. - -### Computer bits - -Back in those golden years, there were independent computer stores on every corner and you could hardly walk down the block without tripping over a local independent Internet service provider (ISP). ISPs were very different then. They were not horrid customer-hostile megacorporations like our good friends the American telcos and cable companies. They were nice, and offered all kinds of extra services like Bulletin Board Services (BBS), file downloads, and Multi-User Domains (MUDs), which were multi-player online games. - -I spent a lot of time buying parts in the computer stores, and half the fun was shocking the store staff by being a woman. It is puzzling how this is so upsetting to some people. Now I'm an old fart of 58 and they're still not used to it. I hope that being a woman nerd will become acceptable by the time I die. - -Those stores had racks of Computer Bits magazine. Check out this old issue of Computer Bits on the Internet Archive. Computer Bits was a local free paper with good articles and tons of ads. Unfortunately, the print ads did not appear in the online edition, so you can't see how they included a wealth of detailed information. You know how advertisers are so whiny, and complain about ad blockers, and have turned tech journalism into barely-disguised advertising? They need to learn some lessons from the past. Those ads were useful. People wanted to read them. I learned everything about computer hardware from reading the ads in Computer Bits and other computer magazines. Computer Shopper was an especially fabulous resource; it had several hundred pages of ads and high-quality articles. - -![](https://opensource.com/sites/default/files/resize/march2002-300x387.jpg) - -The publisher of Computer Bits, Paul Harwood, launched my writing career. My first ever professional writing was for Computer Bits. Paul, if you're still around, thank you! - -You can see something in the Internet Archives of Computer Bits that barely exists anymore, and that is the classified ads section. Classified ads generated significant income for print publications. Craigslist killed the classifieds, which killed newspapers and publications like Computer Bits. - -One of my cherished memories is when the 12-year-old twit who ran my favorite computer store, who could never get over my blatant woman-ness and could never accept that I knew what I was doing, handed me a copy of Computer Bits as a good resource for beginners. I opened it to show him one of my Linux articles and said "Oh yes, I know." He turned colors I didn't think were physiologically possible and scuttled away. (No, my fine literalists, he was not really 12, but in his early 20s. Perhaps by now his emotional maturity has caught up a little.) - -### Discovering Linux - -I first learned about Linux in Computer Bits magazine, maybe around 1997 or so. My first Linuxes were Red Hat 5 and Mandrake Linux. Mandrake was glorious. It was the first easy-to-install Linux, and it bundled graphics and sound drivers so I could immediately play Tux Racer. Unlike most Linux nerds at the time I did not come from a Unix background, so I had a steep learning curve. But that was alright, because everything I learned was useful. In contrast to my Windows adventures, where most of what I learned was working around its awfulness, then giving up and dropping to a DOS shell. - -Playing with computers was so much fun I drifted into freelance consulting, forcing Windows computers to more or less function, and helping IT staff in small shops migrate to Linux servers. Usually we did this on the sneak, because those were the days of Microsoft calling Linux a cancer, and implying that it was a communist conspiracy to sap and impurify all of our precious bodily fluids. - -### Linux won - -I continued consulting for a number of years, doing a little bit of everything: Repairing and upgrading hardware, pulling cable, system and network administration, and running mixed Apple/Windows/Linux networks. Apple and Windows were absolutely hellish to integrate into mixed networks as they deliberately tried to make it impossible. One of the coolest things about Linux and FOSS is there is always someone ready and able to defeat the barriers erected by proprietary vendors. - -It is very different now. There are still proprietary barriers to interoperability, and there is still no Tier 1 desktop Linux OEM vendor. In my opinion this is because Microsoft and Apple have a hard lock on retail distribution. Maybe they're doing us a favor, because you get way better service and quality from wonderful independent Linux vendors like ZaReason and System76. They're Linux experts, and they don't treat us like unwelcome afterthoughts. - -Aside from the retail desktop, Linux dominates all aspects of computing from embedded to supercomputing and distributed computing. Open source dominates software development. All of the significant new frontiers in software, such as containers, clusters, and artificial intelligence are all powered by open source software. When you measure the distance from my first little old 386SX PC til now, that is phenomenal progress. - -It would not have happened without Linux and open source. - --------------------------------------------------------------------------------- - -via: https://opensource.com/life/16/7/my-linux-story-carla-schroder - -作者:[Carla Schroder ][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/carlaschroder From 9d3c5e6e0a07a058094d741c50a23aecf7aa9b32 Mon Sep 17 00:00:00 2001 From: rusking Date: Tue, 6 Sep 2016 12:35:36 +0800 Subject: [PATCH 0057/2437] I've been Linuxing since before you were born MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 翻译完成 --- ...e been Linuxing since before you were born | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 translated/I've been Linuxing since before you were born diff --git a/translated/I've been Linuxing since before you were born b/translated/I've been Linuxing since before you were born new file mode 100644 index 0000000000..2ff2df8c83 --- /dev/null +++ b/translated/I've been Linuxing since before you were born @@ -0,0 +1,68 @@ + +我玩 Linux 那会儿你还不知道在哪呢 +===================== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_Penguin_Image_520x292_12324207_0714_mm_v1a.png?itok=WfAkwbFy) + +在很久以前,世界上没有 Linux 系统。真的没有!之前也从未存在过。不像现在, Linux 系统随处可见。有各种风格的 Unix 系统,有苹果的操作系统,有微软的 Windows 操作系统。 + +随时 Windows 桌面的到来,改的东西很多,但是保留下来的东西的更多。尽管增加至20GB以上的存储,Windows 还是大体保持不变。(除了不能在DOS的提示下完成实际的工作)嘿,谁还记得 Gorilla.bas 那个出现在 DOS 系统里的炸香蕉的游戏吗?多么美好的时光啊!互联网永不会忘记,而且你可以在 Kongregate.com 这个网站玩Flash版本的这个游戏。 + +苹果系统也改变了,变得更加友好、安全、美观。你别想着打开这个密封的盒子,它强制规定了用户能使用的硬件接口。 1998年:没有软盘。 2012年:没有光驱。 12英寸的 MacBook 只有一个提供电源的USB Type-C 端口、蓝牙、无线网卡、外部存储、视频输出接口和其它的一些配件。如果你想每次想接入一个新的设备而不想使用其它的配件和软件,这真是太抓狂了。其它:头戴式耳机。这是个非独有的硬件接口在 Apple 的领域里注定保留下来。 + +还有其它的操作系统,比如: Amiga, BeOS, OS/2 以及无数的你能查到的操作系统。我支持的这些系统是因为现在都很容易查寻到,也没有理由不推荐它们。 Amiga、 BeOS 和 OS/2 这些系统都值得关注,因为它们有强大的功能,比如32位的多任务和高级图形处理能力。但是市场的影响击败了强大的系统性能,因此普通的 Apple 和 Windows 操作系统统治了市场的主导地位,而那些曾经的系统也逐渐销声匿迹了。 + +然后 Linux 系统出现了,世界也因此改变了。 + +### 第一款电脑 + +我曾经使用过的第一款电脑是 Apple IIc ,大概在 1994年左右,那个时候 Linux 系统刚出来 3年。这是我从一个朋友那里借来的,用起来各方面都还不错,但是很不方便。所以我自己花了将近 500美元买了一台二手的 Tandy牌子的电脑。这对于卖电脑的人来说是一件很伤心的事,因为新电脑要花费双倍的价钱。那个时候,电脑贬值的速度非常快。这个电脑看起来像是个怪物:一个英特尔 386SX 的 CPU, 4MB的内存,一个 107MB的硬盘, 14英寸的彩色 CRT 显示器,运行 MS-DOS 5和 Windows 3.1系统。 + +我曾无数次的拆开那个可怜的怪物,并且多次重新安装 Windows 和 DOS 系统。因为 Windows 桌面用的比较少,所以我的大部分工作都是在 DOS 下完成的。我喜欢玩血腥暴力的视频游戏,包括 Doom, Duke Nukem,Quake 和 Heretic。啊!那些美好的,让人心动的 8位图像。 + +那个时候硬件的发展一直落后于软件,因此我经常升级硬件。现在我们能买到满足我们需求的任何配置的电脑。我已经好多年都没有再更新我的任何电脑硬件了。 + +### 比特杂志 + +回到那些曾经辉煌的年代,电脑商店布满大街小巷,找到本地的一家网络服务提供商(ISP)也不用走遍整条街。ISP 那个时候真的非比寻常。他们都是讨人喜欢的一些友好的大公司,像美国电信公司和有线电视公司等。他们都非常友好,并且提供各种各样的像 BBS、文件下载、MUDs (多玩家在线游戏)等的额外服务。 + +我花了很多的时间在电脑商店购买配件,但是很多时候我一个女人家去那里会让店里的员工感到吃惊,我真的很无语了,这怎么就会让一些人看不惯了。我现成已经是一位58岁的老奶了,但是他们还是一样的看不惯我。我希望我这个让人讨厌的女人在我死之前能被他们所接受。 + +那些商店的书架上摆满了比特杂志。有关比特杂志的历史信息可以在互联网档案库中查到。比特杂志是当地一家免费的杂志,有很多关于计算机方面的优秀的文章和大量的广告。可惜当时的广告没有网络版,因此大家不能再看到那些很有价值的关于计算机方面的详细信息了。你知道现在的广告商们有多么的抓狂吗?他们埋怨那些排斥广告的人,致使他们不得不在科技新闻中植入一些隐藏的广告。他们应该学习一下过去的广告模式,那时候的广告里有很多有价值的信息,大家都喜欢阅读。我从比特杂志和其它的电脑杂志的广告中学到了所有关于计算机硬件的知识。电脑购买者杂志更是非常好的学习资料,其中有上百页的广告和很多高质量的文章。 + + +![](https://opensource.com/sites/default/files/resize/march2002-300x387.jpg) + +比特杂志的出版者 Paul Harwood 开启了我的写作生涯。我的第一篇计算机专业性质的文章就是在比特杂志发表的。 Paul,仿佛你一直都在我的身旁,谢谢你。 + +在互联网档案库中,关于比特杂志的分类广告信息已经几乎查询不到了。分类广告模式曾经给出版商带来巨大的收入。免费的分类广告网站 Craigslist 在这个领域独占鳌头,同时也扼杀了像比特杂志这种以传统的报纸和出版为主的杂志行业。 + +其中有一些让我最难以忘怀的记忆就是一个12岁左右的吊儿郎当的小屁孩,匆匆忙忙地跑到我钟爱的电脑店里递给我一本初学者好书比特杂志,他脸上挂满了对我这种光鲜亮丽的女人所做工作的不屑和不理解的表情。我拆开杂志,指着其中一篇我写的关于Linux系统的文章给他看,他说“哦,确实是这样的,我明白了”。他尴尬的脸变成了那种正常生理上都不可能呈现的颜色,然后很仓促地夹着尾巴溜走了。(不是的,我亲爱的、诚实的读者们,他不是真的只有12岁,而应该是20来岁。他现在应该比以前更成熟懂事一些了吧!) + +###发现 Linux + +我第一次了解到 Linux 系统是在比特杂志上,大概在 1977年左右。我一开始用的一些 Linux操作系统版本是 Red Hat 5 和 Mandrake Linux。 Mandrake真是太棒了,它是第一款易安装型的Linux系统,并且还附带图形界面和声卡驱动,因此我可以快速玩转 Tux Racer 游戏。不像那时候大多数的有教育背景的 Linux 用户那样,因为我之前没接触过 Unix系统,所以我学习起来比较难。但是一切都还顺利吧,因为我学到的东西都很有用。相对于学习 Windows 系统来说,我在 Linux 系统里学到的东西都很实用,也没必要像在Windows环境中那样非得把任务拖放到DOS下来执行。 + +玩转电脑真是充满了太多的乐趣,后来我转行成为计算机自由顾问,去帮助一些小型公司的IT部门把数据迁移到 Linux 服务器上,这迫使 Windows 系统或多或少的失去一些用武之地。通常情况下我们都是在背地里偷偷地做这些工作的,因为那段时期微软把 Linux 称为毒瘤,诬蔑 Linux 系统是一种共产主义用来削弱和吞噬我们身体里的珍贵血液的阴谋。 + +### Linux 赢了 + +我持续做了很多年的顾问工作,也做其它一些相关的工作,比如:电脑硬件修理和升级、布线、系统和网络管理,还有其它一些混合的包括 Apple、Windows、Linux 系统在内的网络工作。Apple 和 Windows 系统故意不兼容对方,因此这两个系统真的是最头疼,也最难整合到同一网络中。但是在Linux系统和其它开源软件中最有趣的一件事是总有一些人能够时时处理这些厂商之间的兼容性问题。 + +现在已经大不同了。在系统间的互操作方面一直存在一些兼容性问题,而且也没有1级桌面的 Linux 系统 OEM 厂商。我觉得这是因为微软和苹果公司在零售行业在大力垄断造成的。也许他们这样做反而帮了我们帮大忙,因为我们可以从 ZaReason 和 System76 这样独立的 Linux 系统开发商得到更好的服务和更高性能的系统。他们对 Linux 系统都很专业,也特别欢迎我们这样的用户。 + +Linux 除了在零售行业的桌面版系统占有一席之地以外,从嵌入式系统到超级计算机以及分布式计算系统等各个方面都占据着主要地位。开源技术掌握着软件行业的发展方向,所有的软件行业的前沿技术,比如容器、集群以及人工智能的发展都离不开开源技术的支持。从我的第一台老式的 386SX 电脑到现在,Linux 和开源技术都取得了巨大的进步。 + +如果没有 Linux 系统和开源,这一切都不可能发生。 + +-------------------------------------------------------------------------------- + +via: https://opensource.com/life/16/7/my-linux-story-carla-schroder + +作者:[Carla Schroder ][a] +译者:[译者ID](https://github.com/rusking) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/carlaschroder From d0df2cf168a0d370a84f1c5c2c63ee3a0946d16e Mon Sep 17 00:00:00 2001 From: rusking Date: Tue, 6 Sep 2016 12:37:03 +0800 Subject: [PATCH 0058/2437] Delete I've been Linuxing since before you were born MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 目录放错了,重新删除 --- ...e been Linuxing since before you were born | 68 ------------------- 1 file changed, 68 deletions(-) delete mode 100644 translated/I've been Linuxing since before you were born diff --git a/translated/I've been Linuxing since before you were born b/translated/I've been Linuxing since before you were born deleted file mode 100644 index 2ff2df8c83..0000000000 --- a/translated/I've been Linuxing since before you were born +++ /dev/null @@ -1,68 +0,0 @@ - -我玩 Linux 那会儿你还不知道在哪呢 -===================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_Penguin_Image_520x292_12324207_0714_mm_v1a.png?itok=WfAkwbFy) - -在很久以前,世界上没有 Linux 系统。真的没有!之前也从未存在过。不像现在, Linux 系统随处可见。有各种风格的 Unix 系统,有苹果的操作系统,有微软的 Windows 操作系统。 - -随时 Windows 桌面的到来,改的东西很多,但是保留下来的东西的更多。尽管增加至20GB以上的存储,Windows 还是大体保持不变。(除了不能在DOS的提示下完成实际的工作)嘿,谁还记得 Gorilla.bas 那个出现在 DOS 系统里的炸香蕉的游戏吗?多么美好的时光啊!互联网永不会忘记,而且你可以在 Kongregate.com 这个网站玩Flash版本的这个游戏。 - -苹果系统也改变了,变得更加友好、安全、美观。你别想着打开这个密封的盒子,它强制规定了用户能使用的硬件接口。 1998年:没有软盘。 2012年:没有光驱。 12英寸的 MacBook 只有一个提供电源的USB Type-C 端口、蓝牙、无线网卡、外部存储、视频输出接口和其它的一些配件。如果你想每次想接入一个新的设备而不想使用其它的配件和软件,这真是太抓狂了。其它:头戴式耳机。这是个非独有的硬件接口在 Apple 的领域里注定保留下来。 - -还有其它的操作系统,比如: Amiga, BeOS, OS/2 以及无数的你能查到的操作系统。我支持的这些系统是因为现在都很容易查寻到,也没有理由不推荐它们。 Amiga、 BeOS 和 OS/2 这些系统都值得关注,因为它们有强大的功能,比如32位的多任务和高级图形处理能力。但是市场的影响击败了强大的系统性能,因此普通的 Apple 和 Windows 操作系统统治了市场的主导地位,而那些曾经的系统也逐渐销声匿迹了。 - -然后 Linux 系统出现了,世界也因此改变了。 - -### 第一款电脑 - -我曾经使用过的第一款电脑是 Apple IIc ,大概在 1994年左右,那个时候 Linux 系统刚出来 3年。这是我从一个朋友那里借来的,用起来各方面都还不错,但是很不方便。所以我自己花了将近 500美元买了一台二手的 Tandy牌子的电脑。这对于卖电脑的人来说是一件很伤心的事,因为新电脑要花费双倍的价钱。那个时候,电脑贬值的速度非常快。这个电脑看起来像是个怪物:一个英特尔 386SX 的 CPU, 4MB的内存,一个 107MB的硬盘, 14英寸的彩色 CRT 显示器,运行 MS-DOS 5和 Windows 3.1系统。 - -我曾无数次的拆开那个可怜的怪物,并且多次重新安装 Windows 和 DOS 系统。因为 Windows 桌面用的比较少,所以我的大部分工作都是在 DOS 下完成的。我喜欢玩血腥暴力的视频游戏,包括 Doom, Duke Nukem,Quake 和 Heretic。啊!那些美好的,让人心动的 8位图像。 - -那个时候硬件的发展一直落后于软件,因此我经常升级硬件。现在我们能买到满足我们需求的任何配置的电脑。我已经好多年都没有再更新我的任何电脑硬件了。 - -### 比特杂志 - -回到那些曾经辉煌的年代,电脑商店布满大街小巷,找到本地的一家网络服务提供商(ISP)也不用走遍整条街。ISP 那个时候真的非比寻常。他们都是讨人喜欢的一些友好的大公司,像美国电信公司和有线电视公司等。他们都非常友好,并且提供各种各样的像 BBS、文件下载、MUDs (多玩家在线游戏)等的额外服务。 - -我花了很多的时间在电脑商店购买配件,但是很多时候我一个女人家去那里会让店里的员工感到吃惊,我真的很无语了,这怎么就会让一些人看不惯了。我现成已经是一位58岁的老奶了,但是他们还是一样的看不惯我。我希望我这个让人讨厌的女人在我死之前能被他们所接受。 - -那些商店的书架上摆满了比特杂志。有关比特杂志的历史信息可以在互联网档案库中查到。比特杂志是当地一家免费的杂志,有很多关于计算机方面的优秀的文章和大量的广告。可惜当时的广告没有网络版,因此大家不能再看到那些很有价值的关于计算机方面的详细信息了。你知道现在的广告商们有多么的抓狂吗?他们埋怨那些排斥广告的人,致使他们不得不在科技新闻中植入一些隐藏的广告。他们应该学习一下过去的广告模式,那时候的广告里有很多有价值的信息,大家都喜欢阅读。我从比特杂志和其它的电脑杂志的广告中学到了所有关于计算机硬件的知识。电脑购买者杂志更是非常好的学习资料,其中有上百页的广告和很多高质量的文章。 - - -![](https://opensource.com/sites/default/files/resize/march2002-300x387.jpg) - -比特杂志的出版者 Paul Harwood 开启了我的写作生涯。我的第一篇计算机专业性质的文章就是在比特杂志发表的。 Paul,仿佛你一直都在我的身旁,谢谢你。 - -在互联网档案库中,关于比特杂志的分类广告信息已经几乎查询不到了。分类广告模式曾经给出版商带来巨大的收入。免费的分类广告网站 Craigslist 在这个领域独占鳌头,同时也扼杀了像比特杂志这种以传统的报纸和出版为主的杂志行业。 - -其中有一些让我最难以忘怀的记忆就是一个12岁左右的吊儿郎当的小屁孩,匆匆忙忙地跑到我钟爱的电脑店里递给我一本初学者好书比特杂志,他脸上挂满了对我这种光鲜亮丽的女人所做工作的不屑和不理解的表情。我拆开杂志,指着其中一篇我写的关于Linux系统的文章给他看,他说“哦,确实是这样的,我明白了”。他尴尬的脸变成了那种正常生理上都不可能呈现的颜色,然后很仓促地夹着尾巴溜走了。(不是的,我亲爱的、诚实的读者们,他不是真的只有12岁,而应该是20来岁。他现在应该比以前更成熟懂事一些了吧!) - -###发现 Linux - -我第一次了解到 Linux 系统是在比特杂志上,大概在 1977年左右。我一开始用的一些 Linux操作系统版本是 Red Hat 5 和 Mandrake Linux。 Mandrake真是太棒了,它是第一款易安装型的Linux系统,并且还附带图形界面和声卡驱动,因此我可以快速玩转 Tux Racer 游戏。不像那时候大多数的有教育背景的 Linux 用户那样,因为我之前没接触过 Unix系统,所以我学习起来比较难。但是一切都还顺利吧,因为我学到的东西都很有用。相对于学习 Windows 系统来说,我在 Linux 系统里学到的东西都很实用,也没必要像在Windows环境中那样非得把任务拖放到DOS下来执行。 - -玩转电脑真是充满了太多的乐趣,后来我转行成为计算机自由顾问,去帮助一些小型公司的IT部门把数据迁移到 Linux 服务器上,这迫使 Windows 系统或多或少的失去一些用武之地。通常情况下我们都是在背地里偷偷地做这些工作的,因为那段时期微软把 Linux 称为毒瘤,诬蔑 Linux 系统是一种共产主义用来削弱和吞噬我们身体里的珍贵血液的阴谋。 - -### Linux 赢了 - -我持续做了很多年的顾问工作,也做其它一些相关的工作,比如:电脑硬件修理和升级、布线、系统和网络管理,还有其它一些混合的包括 Apple、Windows、Linux 系统在内的网络工作。Apple 和 Windows 系统故意不兼容对方,因此这两个系统真的是最头疼,也最难整合到同一网络中。但是在Linux系统和其它开源软件中最有趣的一件事是总有一些人能够时时处理这些厂商之间的兼容性问题。 - -现在已经大不同了。在系统间的互操作方面一直存在一些兼容性问题,而且也没有1级桌面的 Linux 系统 OEM 厂商。我觉得这是因为微软和苹果公司在零售行业在大力垄断造成的。也许他们这样做反而帮了我们帮大忙,因为我们可以从 ZaReason 和 System76 这样独立的 Linux 系统开发商得到更好的服务和更高性能的系统。他们对 Linux 系统都很专业,也特别欢迎我们这样的用户。 - -Linux 除了在零售行业的桌面版系统占有一席之地以外,从嵌入式系统到超级计算机以及分布式计算系统等各个方面都占据着主要地位。开源技术掌握着软件行业的发展方向,所有的软件行业的前沿技术,比如容器、集群以及人工智能的发展都离不开开源技术的支持。从我的第一台老式的 386SX 电脑到现在,Linux 和开源技术都取得了巨大的进步。 - -如果没有 Linux 系统和开源,这一切都不可能发生。 - --------------------------------------------------------------------------------- - -via: https://opensource.com/life/16/7/my-linux-story-carla-schroder - -作者:[Carla Schroder ][a] -译者:[译者ID](https://github.com/rusking) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/carlaschroder From 76225f5a92bf2f69641061ad3e3327bc6ee53f09 Mon Sep 17 00:00:00 2001 From: rusking Date: Tue, 6 Sep 2016 12:37:47 +0800 Subject: [PATCH 0059/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90I?= =?UTF-8?q?'ve=20been=20Linuxing=20since=20before=20you=20were=20born?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...e been Linuxing since before you were born | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 translated/talk/I've been Linuxing since before you were born diff --git a/translated/talk/I've been Linuxing since before you were born b/translated/talk/I've been Linuxing since before you were born new file mode 100644 index 0000000000..2ff2df8c83 --- /dev/null +++ b/translated/talk/I've been Linuxing since before you were born @@ -0,0 +1,68 @@ + +我玩 Linux 那会儿你还不知道在哪呢 +===================== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_Penguin_Image_520x292_12324207_0714_mm_v1a.png?itok=WfAkwbFy) + +在很久以前,世界上没有 Linux 系统。真的没有!之前也从未存在过。不像现在, Linux 系统随处可见。有各种风格的 Unix 系统,有苹果的操作系统,有微软的 Windows 操作系统。 + +随时 Windows 桌面的到来,改的东西很多,但是保留下来的东西的更多。尽管增加至20GB以上的存储,Windows 还是大体保持不变。(除了不能在DOS的提示下完成实际的工作)嘿,谁还记得 Gorilla.bas 那个出现在 DOS 系统里的炸香蕉的游戏吗?多么美好的时光啊!互联网永不会忘记,而且你可以在 Kongregate.com 这个网站玩Flash版本的这个游戏。 + +苹果系统也改变了,变得更加友好、安全、美观。你别想着打开这个密封的盒子,它强制规定了用户能使用的硬件接口。 1998年:没有软盘。 2012年:没有光驱。 12英寸的 MacBook 只有一个提供电源的USB Type-C 端口、蓝牙、无线网卡、外部存储、视频输出接口和其它的一些配件。如果你想每次想接入一个新的设备而不想使用其它的配件和软件,这真是太抓狂了。其它:头戴式耳机。这是个非独有的硬件接口在 Apple 的领域里注定保留下来。 + +还有其它的操作系统,比如: Amiga, BeOS, OS/2 以及无数的你能查到的操作系统。我支持的这些系统是因为现在都很容易查寻到,也没有理由不推荐它们。 Amiga、 BeOS 和 OS/2 这些系统都值得关注,因为它们有强大的功能,比如32位的多任务和高级图形处理能力。但是市场的影响击败了强大的系统性能,因此普通的 Apple 和 Windows 操作系统统治了市场的主导地位,而那些曾经的系统也逐渐销声匿迹了。 + +然后 Linux 系统出现了,世界也因此改变了。 + +### 第一款电脑 + +我曾经使用过的第一款电脑是 Apple IIc ,大概在 1994年左右,那个时候 Linux 系统刚出来 3年。这是我从一个朋友那里借来的,用起来各方面都还不错,但是很不方便。所以我自己花了将近 500美元买了一台二手的 Tandy牌子的电脑。这对于卖电脑的人来说是一件很伤心的事,因为新电脑要花费双倍的价钱。那个时候,电脑贬值的速度非常快。这个电脑看起来像是个怪物:一个英特尔 386SX 的 CPU, 4MB的内存,一个 107MB的硬盘, 14英寸的彩色 CRT 显示器,运行 MS-DOS 5和 Windows 3.1系统。 + +我曾无数次的拆开那个可怜的怪物,并且多次重新安装 Windows 和 DOS 系统。因为 Windows 桌面用的比较少,所以我的大部分工作都是在 DOS 下完成的。我喜欢玩血腥暴力的视频游戏,包括 Doom, Duke Nukem,Quake 和 Heretic。啊!那些美好的,让人心动的 8位图像。 + +那个时候硬件的发展一直落后于软件,因此我经常升级硬件。现在我们能买到满足我们需求的任何配置的电脑。我已经好多年都没有再更新我的任何电脑硬件了。 + +### 比特杂志 + +回到那些曾经辉煌的年代,电脑商店布满大街小巷,找到本地的一家网络服务提供商(ISP)也不用走遍整条街。ISP 那个时候真的非比寻常。他们都是讨人喜欢的一些友好的大公司,像美国电信公司和有线电视公司等。他们都非常友好,并且提供各种各样的像 BBS、文件下载、MUDs (多玩家在线游戏)等的额外服务。 + +我花了很多的时间在电脑商店购买配件,但是很多时候我一个女人家去那里会让店里的员工感到吃惊,我真的很无语了,这怎么就会让一些人看不惯了。我现成已经是一位58岁的老奶了,但是他们还是一样的看不惯我。我希望我这个让人讨厌的女人在我死之前能被他们所接受。 + +那些商店的书架上摆满了比特杂志。有关比特杂志的历史信息可以在互联网档案库中查到。比特杂志是当地一家免费的杂志,有很多关于计算机方面的优秀的文章和大量的广告。可惜当时的广告没有网络版,因此大家不能再看到那些很有价值的关于计算机方面的详细信息了。你知道现在的广告商们有多么的抓狂吗?他们埋怨那些排斥广告的人,致使他们不得不在科技新闻中植入一些隐藏的广告。他们应该学习一下过去的广告模式,那时候的广告里有很多有价值的信息,大家都喜欢阅读。我从比特杂志和其它的电脑杂志的广告中学到了所有关于计算机硬件的知识。电脑购买者杂志更是非常好的学习资料,其中有上百页的广告和很多高质量的文章。 + + +![](https://opensource.com/sites/default/files/resize/march2002-300x387.jpg) + +比特杂志的出版者 Paul Harwood 开启了我的写作生涯。我的第一篇计算机专业性质的文章就是在比特杂志发表的。 Paul,仿佛你一直都在我的身旁,谢谢你。 + +在互联网档案库中,关于比特杂志的分类广告信息已经几乎查询不到了。分类广告模式曾经给出版商带来巨大的收入。免费的分类广告网站 Craigslist 在这个领域独占鳌头,同时也扼杀了像比特杂志这种以传统的报纸和出版为主的杂志行业。 + +其中有一些让我最难以忘怀的记忆就是一个12岁左右的吊儿郎当的小屁孩,匆匆忙忙地跑到我钟爱的电脑店里递给我一本初学者好书比特杂志,他脸上挂满了对我这种光鲜亮丽的女人所做工作的不屑和不理解的表情。我拆开杂志,指着其中一篇我写的关于Linux系统的文章给他看,他说“哦,确实是这样的,我明白了”。他尴尬的脸变成了那种正常生理上都不可能呈现的颜色,然后很仓促地夹着尾巴溜走了。(不是的,我亲爱的、诚实的读者们,他不是真的只有12岁,而应该是20来岁。他现在应该比以前更成熟懂事一些了吧!) + +###发现 Linux + +我第一次了解到 Linux 系统是在比特杂志上,大概在 1977年左右。我一开始用的一些 Linux操作系统版本是 Red Hat 5 和 Mandrake Linux。 Mandrake真是太棒了,它是第一款易安装型的Linux系统,并且还附带图形界面和声卡驱动,因此我可以快速玩转 Tux Racer 游戏。不像那时候大多数的有教育背景的 Linux 用户那样,因为我之前没接触过 Unix系统,所以我学习起来比较难。但是一切都还顺利吧,因为我学到的东西都很有用。相对于学习 Windows 系统来说,我在 Linux 系统里学到的东西都很实用,也没必要像在Windows环境中那样非得把任务拖放到DOS下来执行。 + +玩转电脑真是充满了太多的乐趣,后来我转行成为计算机自由顾问,去帮助一些小型公司的IT部门把数据迁移到 Linux 服务器上,这迫使 Windows 系统或多或少的失去一些用武之地。通常情况下我们都是在背地里偷偷地做这些工作的,因为那段时期微软把 Linux 称为毒瘤,诬蔑 Linux 系统是一种共产主义用来削弱和吞噬我们身体里的珍贵血液的阴谋。 + +### Linux 赢了 + +我持续做了很多年的顾问工作,也做其它一些相关的工作,比如:电脑硬件修理和升级、布线、系统和网络管理,还有其它一些混合的包括 Apple、Windows、Linux 系统在内的网络工作。Apple 和 Windows 系统故意不兼容对方,因此这两个系统真的是最头疼,也最难整合到同一网络中。但是在Linux系统和其它开源软件中最有趣的一件事是总有一些人能够时时处理这些厂商之间的兼容性问题。 + +现在已经大不同了。在系统间的互操作方面一直存在一些兼容性问题,而且也没有1级桌面的 Linux 系统 OEM 厂商。我觉得这是因为微软和苹果公司在零售行业在大力垄断造成的。也许他们这样做反而帮了我们帮大忙,因为我们可以从 ZaReason 和 System76 这样独立的 Linux 系统开发商得到更好的服务和更高性能的系统。他们对 Linux 系统都很专业,也特别欢迎我们这样的用户。 + +Linux 除了在零售行业的桌面版系统占有一席之地以外,从嵌入式系统到超级计算机以及分布式计算系统等各个方面都占据着主要地位。开源技术掌握着软件行业的发展方向,所有的软件行业的前沿技术,比如容器、集群以及人工智能的发展都离不开开源技术的支持。从我的第一台老式的 386SX 电脑到现在,Linux 和开源技术都取得了巨大的进步。 + +如果没有 Linux 系统和开源,这一切都不可能发生。 + +-------------------------------------------------------------------------------- + +via: https://opensource.com/life/16/7/my-linux-story-carla-schroder + +作者:[Carla Schroder ][a] +译者:[译者ID](https://github.com/rusking) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/carlaschroder From d28385802d512616cf08ecaf184974424e12c177 Mon Sep 17 00:00:00 2001 From: rusking Date: Tue, 6 Sep 2016 12:38:30 +0800 Subject: [PATCH 0060/2437] Rename I've been Linuxing since before you were born to 20160906I've been Linuxing since before you were born --- ...born => 20160906I've been Linuxing since before you were born} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename translated/talk/{I've been Linuxing since before you were born => 20160906I've been Linuxing since before you were born} (100%) diff --git a/translated/talk/I've been Linuxing since before you were born b/translated/talk/20160906I've been Linuxing since before you were born similarity index 100% rename from translated/talk/I've been Linuxing since before you were born rename to translated/talk/20160906I've been Linuxing since before you were born From 2ca32adde0ec1610a108653f20a615ee118d8bc5 Mon Sep 17 00:00:00 2001 From: VicYu Date: Tue, 6 Sep 2016 15:32:43 +0800 Subject: [PATCH 0061/2437] Modify name to raw name --- ...orn => 20160728 I've been Linuxing since before you were born} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename translated/talk/{20160906I've been Linuxing since before you were born => 20160728 I've been Linuxing since before you were born} (100%) diff --git a/translated/talk/20160906I've been Linuxing since before you were born b/translated/talk/20160728 I've been Linuxing since before you were born similarity index 100% rename from translated/talk/20160906I've been Linuxing since before you were born rename to translated/talk/20160728 I've been Linuxing since before you were born From 684f48ce656a9f14be99e20c85eb860b85542d0d Mon Sep 17 00:00:00 2001 From: Ezio Date: Tue, 6 Sep 2016 15:35:59 +0800 Subject: [PATCH 0062/2437] =?UTF-8?q?=E5=BE=85=E6=A0=A1=E5=AF=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... => 20160728 I've been Linuxing since before you were born.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename translated/talk/{I've been Linuxing since before you were born => 20160728 I've been Linuxing since before you were born.md} (100%) diff --git a/translated/talk/I've been Linuxing since before you were born b/translated/talk/20160728 I've been Linuxing since before you were born.md similarity index 100% rename from translated/talk/I've been Linuxing since before you were born rename to translated/talk/20160728 I've been Linuxing since before you were born.md From ef908ece96a9c3ec4a5e4f50764ade2c0d282082 Mon Sep 17 00:00:00 2001 From: WangYue <815420852@qq.com> Date: Tue, 6 Sep 2016 16:20:32 +0800 Subject: [PATCH 0063/2437] =?UTF-8?q?[=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?]20160808=20Why=20measuring=20IT=20productivity=20is=20so=20cha?= =?UTF-8?q?llenging.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [翻译完成]20160808 Why measuring IT productivity is so challenging.md --- ...uring IT productivity is so challenging.md | 43 ------------------- 1 file changed, 43 deletions(-) delete mode 100644 sources/talk/20160808 Why measuring IT productivity is so challenging.md diff --git a/sources/talk/20160808 Why measuring IT productivity is so challenging.md b/sources/talk/20160808 Why measuring IT productivity is so challenging.md deleted file mode 100644 index 62f446673d..0000000000 --- a/sources/talk/20160808 Why measuring IT productivity is so challenging.md +++ /dev/null @@ -1,43 +0,0 @@ -[WangYueScream 翻译中] - -=========================== - -Why measuring IT productivity is so challenging -=========================== - -![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/cio_talent_6.png?itok=JV-zSor3) - -In some professions, there are metrics that one can use to determine an individual’s productivity. If you’re a widget maker, for instance, you might be measured on how many widgets you can make in a month. If you work in a call center – how many calls did you answer, and what’s your average handle time? These are over-simplified examples, but even if you're a doctor, you may be measured on how many operations you perform, or how many patients you can see in a month. Whether or not these are the right metrics, they offer a general picture of how an individual performed within a set timeframe. - -In the case of technology, however, it becomes almost impossible to measure a person’s productivity because there is so much variability. For example, it may be tempting to measure a developer's time by lines of code built. But, depending on the coding language, one line of code in one language versus another might be significantly more or less time consuming or difficult. - -Has it always been this nuanced? Many years ago, you might have heard about or experienced IT measurement in terms of function points. These measurements were about the critical features that developers were able to create. But that, too, is becoming harder to do in today’s environment, where developers are often encapsulating logic that may already exist, such as integration of function points through a vendor. This makes it harder to measure productivity based simply on the number of function points built. - -These two examples shed light on why CIOs sometimes struggle when we talk to our peers about IT productivity. Consider this hypothetical conversation: - ->IT leader: “Wow, I think this developer is great.” ->HR: “Really? What do they do?” ->IT leader: “They built this excellent application.” ->HR: “Well, are they better than the other developer who built ten applications?” ->IT leader: “That depends on what you mean by 'better.'” - -Typically, when in the midst of a conversation like the above, there is so much subjectivity involved that it's difficult to answer the question. This is just the tip of the iceberg when it comes to measuring IT performance in a meaningful way. And it doesn't just make conversations harder – it makes it harder for CIOs to showcase the value of their organization to the business. - -This certainly isn't a new problem. I’ve been trying to figure this out for the last 30 years, and I’ve mostly come to the conclusion that we really shouldn’t bother with productivity – we'll never get there. - -I believe we need to change the conversation and stop trying to speak in terms of throughput and cost and productivity but instead, focus on measuring the overall business value of IT. Again, it won't be easy. Business value realization is a hard thing to do. But if CIOs can partner with the business to figure that out, then attributing real value can become more of a science than an art form. - - - --------------------------------------------------------------------------------- - -via: https://enterprisersproject.com/article/2016/8/why-measuring-it-productivity-so-challenging - -作者:[Anil Cheriyan][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://enterprisersproject.com/user/anil-cheriyan - From bf1b062cf38706c59f424c5b2885bc1c5134ec8f Mon Sep 17 00:00:00 2001 From: WangYue <815420852@qq.com> Date: Tue, 6 Sep 2016 16:34:21 +0800 Subject: [PATCH 0064/2437] =?UTF-8?q?=E3=80=90=E7=BF=BB=E8=AF=91=E5=AE=8C?= =?UTF-8?q?=E6=88=90=E3=80=9120160808=20Why=20measuring=20IT=20productivit?= =?UTF-8?q?y=20is=20so=20challenging.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 【翻译完成添加译文】20160808 Why measuring IT productivity is so challenging.md --- ...uring IT productivity is so challenging.md | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 translated/talk/20160808 Why measuring IT productivity is so challenging.md diff --git a/translated/talk/20160808 Why measuring IT productivity is so challenging.md b/translated/talk/20160808 Why measuring IT productivity is so challenging.md new file mode 100644 index 0000000000..cc035981ab --- /dev/null +++ b/translated/talk/20160808 Why measuring IT productivity is so challenging.md @@ -0,0 +1,38 @@ +为什么测试 IT 的生产效率如此具有挑战性 +=========================== + +![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/cio_talent_6.png?itok=JV-zSor3) + +在某些行业里,人们可以根据一些测量标准判定一个人的生产力。比如,如果你是一个小部件制造商,可以通过一个月你能够制造的小部件数量来确定你的生产效率。如果你在客户服务中心工作,你解答了多少个客户来电,你的平均解答时间都会成为评判你的生产效率的依据。即便你是一位医生,也可以通过你主刀的临床手术次数或者一个月你确诊的患者数量来确定你的生产效率,但这些都是相当简单的案例。虽然这些评判标准不是完全正确的,但他们提供了一个通用的方法来在给定时间内评断一个人的执行能力。 + +然而在 IT 这方面,通过上述方法来衡量一个人的生产力是不可能的,因为 IT 有太多的变化性。比如,通过一个开发者编写的代码行数和耗费的时间来衡量开发者的生产效率看起来很诱人。但是,编程的语言很大程度上能影响到根据这种方法得到的结论。因为一种编程语言编辑的一行代码比用其他编程语言编写所花费的时间和难度可能会明显的多或少。 + +它总是这样微妙吗?多年以前,你可能听说过或者经历过根据功能点衡量 IT 工作人员的生产效率。这些措施是针对开发者们能够创建的关键特征来衡量开发者的生产效率的。但这种方法在今天也变得逐渐难以实施,开发者经常将可能已经存在逻辑封装进内部。比如,通过供应商来整合功能点。这使得仅仅是基于功能点的数目来估量生产效率难度加大。 + +这两个例子能够阐述为什么当我们与同事谈论 IT 生产效率的时候有时会引起 CIO 之间的争论。考虑以下这个假想中的谈话: + +>IT leader:“天啊,我想这些开发者一定很厉害。” +>HR:“真的假的,他们做了什么?” +>IT leader:“他们做了个相当好的应用。” +>HR:“好吧,那他们比那些做了 10 个应用的开发者更好吗” +>IT leader:“这要看你怎么理解 ‘更好’。” + +这个对话比较有代表性。当我们处于上诉的这种交谈时,这里面有太多的主观因素导致我们很难回答这个问题。当我们用一种有意义的方法来测试 IT 的效率时,类似上诉谈话的这种问题仅仅是冰山一角。这不仅仅使谈话更加困难-它还会使 CIO 很难展示他们的团队在商业上的价值。 + +可以确实这不是一个新出现的问题。我已经花费差不多 30 年的时间来思考这个问题。我得出的结论是我们真的不应该在谈论 IT 的生产效率这件事上面相互烦扰-因为我们的 IT 生产效率永远不可能能到达到值得我们为此争论的程度。 + +我认为我们需要改变这种对话同时停止根据生产能力和成本来谈论 IT 的生产效率,将目光集中于衡量 IT 的整体商业价值。这个过程不会很容易。商业价值的实现是一件困难的事情。但如果 CIO 能够和商业人员合作来解决这个问题,就可以将实际价值变的更加科学而非一种艺术形式。 + + + +-------------------------------------------------------------------------------- + +via: https://enterprisersproject.com/article/2016/8/why-measuring-it-productivity-so-challenging + +作者:[Anil Cheriyan][a] +译者:[LemonDemo](https://github.com/LemonDemo) [WangYueScream](https://github.com/WangYueScream) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://enterprisersproject.com/user/anil-cheriyan From 097a918dbab764cb6e6df48824a42d24d657b162 Mon Sep 17 00:00:00 2001 From: ch_cn Date: Tue, 6 Sep 2016 18:13:12 +0800 Subject: [PATCH 0065/2437] ch_cn translating --- sources/tech/20160805 Introducing React Native Ubuntu.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160805 Introducing React Native Ubuntu.md b/sources/tech/20160805 Introducing React Native Ubuntu.md index 48e7be997c..eba89ad8b0 100644 --- a/sources/tech/20160805 Introducing React Native Ubuntu.md +++ b/sources/tech/20160805 Introducing React Native Ubuntu.md @@ -1,3 +1,4 @@ +ch_cn translating Introducing React Native Ubuntu ===================== From 066384f176fc5a0145b13c034054ef6d27604a06 Mon Sep 17 00:00:00 2001 From: Bestony Date: Tue, 6 Sep 2016 18:48:48 +0800 Subject: [PATCH 0066/2437] Update 20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md --- ...60218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md b/sources/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md index 7e0c87cd80..babdf6d4c5 100644 --- a/sources/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md +++ b/sources/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md @@ -1,6 +1,6 @@ -【flankershen翻译中】 +【Bestony翻译中】 From 7ae1a30705020029ddfe68a8f1026a7237863c79 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=91=A8=E5=AE=B6=E6=9C=AA?= Date: Tue, 6 Sep 2016 20:52:46 +0800 Subject: [PATCH 0067/2437] Translating by GitFuture 16.09.06 Translating 20160817 Dependency Injection for the Android platform 101 - Part 1.md --- ...ependency Injection for the Android platform 101 - Part 1.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md b/sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md index 93ddf0f20a..47c186ed0d 100644 --- a/sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md +++ b/sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md @@ -1,3 +1,5 @@ +Translating by GitFuture + Dependency Injection for the Android platform 101 - Part 1 =========================== From c6595f20b62f14f28b538a66dd4a65037f4ef32c Mon Sep 17 00:00:00 2001 From: Bestony Date: Tue, 6 Sep 2016 21:10:26 +0800 Subject: [PATCH 0068/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=20(#4369)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 翻译完成 * 移动文件 --- ...ginx as Reverse Proxy on Centos7 CPanel.md | 55 +++++++++---------- 1 file changed, 25 insertions(+), 30 deletions(-) rename {sources => translated}/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md (72%) diff --git a/sources/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md b/translated/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md similarity index 72% rename from sources/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md rename to translated/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md index babdf6d4c5..f8281c1d7e 100644 --- a/sources/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md +++ b/translated/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md @@ -1,21 +1,16 @@ - -【Bestony翻译中】 - - - -How to Set Nginx as Reverse Proxy on Centos7 CPanel +如何在 CentOS 7 cPanel 上 配置 Nginx 作为反向代理 ================================================================================ -Nginx is one of the fastest and most powerful web-server. It is known for its high performance and low resource utilization. It can be installed as both a standalone and a Reverse Proxy Web-server. In this article, I'm discussing about the installation of Nginx as a reverse proxy along with Apache on a CPanel server with latest CentOS 7 installed. +Nginx 是最快和最强大的 Web-Server 之一。以其高性能和低资源利用率而闻名。它可以被安装为一个独立的Web服务器和反向代理Web服务器。在这篇文章,我将讨论在安装了 cPanel 服务器的 Centos 7 服务器上安装Nginx作为Apache的反向代理服务器. -Nginx as a reverse proxy will work as a frontend webserver serving static contents along with Apache serving the dynamic files in backend. This setup will boost up the overall server performance. +Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动态文件提供服务。这个设置将整体提高服务器的性能。 -Let's walk through the installation steps for Nginx as reverse proxy in CentOS7 x86_64 bit server with cPanel 11.52 installed. +让我们过一遍在已经安装好 cPanel 11.52 的 CentOS 7 x86_64 服务器上配置 Nginx 作为反向代理的安装过程。 -First of all, we need to install the EPEL repo to start-up with the process. +首先,我们需要安装 EPEL 库来启动这个进程 -### Step 1: Install the EPEL repo. ### +### 第一步: 安装EPEL库. ### root@server1 [/usr]# yum -y install epel-release Loaded plugins: fastestmirror, tsflags, universal-hooks @@ -37,7 +32,7 @@ First of all, we need to install the EPEL repo to start-up with the process. Installing: epel-release noarch 7-5 extras 14 k -### Step 2: After installing the repo, we can start with the installation of the nDeploy RPM repo for CentOS to install our required nDeploy Webstack and Nginx plugin. ### +### 第二步: 安装完成这个 Repo ,我们可以安装 nDeploy的 CentOS RPM 库来安装我们需要的 nDeploy Webstack 和 Nginx 插件. ### root@server1 [/usr]# yum -y install http://rpm.piserve.com/nDeploy-release-centos-1.0-1.noarch.rpm Loaded plugins: fastestmirror, tsflags, universal-hooks @@ -57,7 +52,7 @@ First of all, we need to install the EPEL repo to start-up with the process. Installing: nDeploy-release-centos noarch 1.0-1 /nDeploy-release-centos-1.0-1.noarch 110 -### Step 3: Install the nDeploy and Nginx nDeploy plugins. ### +### 第三步:安装 nDeploy 和 Nginx nDeploy 插件. ### root@server1 [/usr]# yum --enablerepo=ndeploy install nginx-nDeploy nDeploy Loaded plugins: fastestmirror, tsflags, universal-hooks @@ -87,9 +82,9 @@ First of all, we need to install the EPEL repo to start-up with the process. =============================================================================================================================================== Install 2 Packages (+5 Dependent packages) -With these steps, we've completed with the installation of Nginx plugin in our server. Now we need to configure Nginx as reverse proxy and create the virtualhost for the existing cPanel user accounts. For that we can run the following script. +有以上这些步骤,我们完成了在我们的服务器上 Nginx 插件的安装。现在我们可以配置 Nginx 作为反向代理和为已有的 cPanel 用户账户创建虚拟主机,为此我们可以运行如下脚本。 -### Step 4: To enable Nginx as a front end Web Server and create the default configuration files. ### +### 第四步:启动Nginx作为默认的前端 Web 服务器 并创建默认的配置文件。 ### root@server1 [/usr]# /opt/nDeploy/scripts/cpanel-nDeploy-setup.sh enable Modifying apache http and https port in cpanel @@ -101,9 +96,9 @@ With these steps, we've completed with the installation of Nginx plugin in our s ConfGen:: saheetha ConfGen:: satest -As you can see these script will modify the Apache port from 80 to another port to make Nginx run as a front end web server and create the virtual host configuration files for the existing cPanel accounts. Once it is done, confirm the status of both Apache and Nginx. +你可以看到这个脚本将修改 Apache 的端口从 80 到另一个端口来让 Nginx 作为前端 Web 服务器并为现有的 cPanel 用户创建虚拟主机配置文件。一旦完成,确认 Apache 和 Nginx 的状态。 -### Apache Status: ### +### Apache 状态: ### root@server1 [/var/run/httpd]# systemctl status httpd ● httpd.service - Apache Web Server @@ -118,7 +113,7 @@ As you can see these script will modify the Apache port from 80 to another port Jan 18 06:34:23 server1.centos7-test.com apachectl[25606]: httpd (pid 24760) already running Jan 18 06:34:23 server1.centos7-test.com systemd[1]: Started Apache Web Server. -### Nginx Status: ### +### Nginx 状态: ### root@server1 [~]# systemctl status nginx ● nginx.service - nginx-nDeploy - high performance web server @@ -137,7 +132,7 @@ As you can see these script will modify the Apache port from 80 to another port Jan 17 17:18:29 server1.centos7-test.com nginx[3804]: nginx: configuration file /etc/nginx/nginx.conf test is successful Jan 17 17:18:29 server1.centos7-test.com systemd[1]: Started nginx-nDeploy - high performance web server. -Nginx act as a frontend webserver running on port 80 and Apache configuration is modified to listen on http port 9999 and https port 4430. Please see their status below: +Nginx 作为前端服务器运行在 80 端口,Apache 配置被更改为监听 http 端口 9999 和 https 端口 4430。请看他们的情况 root@server1 [/usr/local/src]# netstat -plan | grep httpd tcp 0 0 0.0.0.0:4430 0.0.0.0:* LISTEN 17270/httpd @@ -151,13 +146,13 @@ Nginx act as a frontend webserver running on port 80 and Apache configuration is tcp 0 0 127.0.0.1:80 0.0.0.0:* LISTEN 17802/nginx: master tcp 0 0 45.79.183.73:80 0.0.0.0:* LISTEN 17802/nginx: master -The virtualhost entries created for the existing users as located in the folder "**/etc/nginx/sites-enabled**". This file path is included in the Nginx main configuration file. +为已有用户创建的虚拟主机的配置文件在 "**/etc/nginx/sites-enabled**". 这个文件路径包含了 Nginx 主要配置文件. root@server1 [/etc/nginx/sites-enabled]# ll | grep .conf -rw-r--r-- 1 root root 311 Jan 17 09:02 saheetha.com.conf -rw-r--r-- 1 root root 336 Jan 17 09:02 saheethastest.com.conf -### Sample Vhost for a domain: ### +### 一个域名的示例虚拟主机: ### server { @@ -173,7 +168,7 @@ The virtualhost entries created for the existing users as located in the folder } -We can confirm the working of the web server status by calling a website in the browser. Please see the web server information on my server after the installation. +我们可以启动浏览器查看完整来确定 Web 服务器的工作状态。安装后,请阅读服务器上的 web 服务信息。 root@server1 [/home]# ip a | grep -i eth0 3: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000 @@ -183,23 +178,23 @@ We can confirm the working of the web server status by calling a website in the ![webserver-status](http://blog.linoxide.com/wp-content/uploads/2016/01/webserver.png) -Nginx will create the virtual host automatically for any newly created accounts in cPanel. With these simple steps we can configure Nginx as reverse proxy on a CentOS 7/CPanel server. -### Advantages of Nginx as Reverse Proxy: ### +Nginx 将会为任何最新在 cPanel 中创建的账户创建虚拟主机。通过这些简单的的步骤,我们能够在一台 CentOS 7 / cPanel 的服务器上配置 Nginx 作为反向代理 +### Nginx 作为反向代理的优势 : ### - 1. Easy to install and configure - 2. Performance and efficiency - 3. Prevent DDOS attacks - 4. Allows .htaccess PHP rewrite rules + 1. 便于安装和配置 + 2. 效率高性能好 + 3. 防止 Ddos 攻击 + 4. 支持使用.htaccess作为PHP的重写规则 -I hope this article is useful for you guys. Thank you for referring to this. I would appreciate your valuable comments and suggestions on this for further improvements. +我希望这篇文章对你们有用。感谢你提到它。我非常高兴收到你的宝贵意见和建议,并进一步改善。 -------------------------------------------------------------------------------- via: http://linoxide.com/linux-how-to/set-nginx-reverse-proxy-centos-7-cpanel/ 作者:[Saheetha Shameer][a] -译者:[译者ID](https://github.com/译者ID) +译者:[bestony](https://github.com/bestony) 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From 68c204c2e58f49174cc31e1ccd61a9a747fb6e10 Mon Sep 17 00:00:00 2001 From: wxy Date: Wed, 7 Sep 2016 06:34:54 +0800 Subject: [PATCH 0069/2437] PUB:20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel @bestony --- ...ginx as Reverse Proxy on Centos7 CPanel.md | 63 +++++++++---------- 1 file changed, 31 insertions(+), 32 deletions(-) rename {translated/tech => published}/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md (73%) diff --git a/translated/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md b/published/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md similarity index 73% rename from translated/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md rename to published/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md index f8281c1d7e..ac17a5eca3 100644 --- a/translated/tech/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md +++ b/published/20160218 How to Set Nginx as Reverse Proxy on Centos7 CPanel.md @@ -1,16 +1,15 @@ - -如何在 CentOS 7 cPanel 上 配置 Nginx 作为反向代理 +如何在 CentOS 7 用 cPanel 配置 Nginx 反向代理 ================================================================================ -Nginx 是最快和最强大的 Web-Server 之一。以其高性能和低资源利用率而闻名。它可以被安装为一个独立的Web服务器和反向代理Web服务器。在这篇文章,我将讨论在安装了 cPanel 服务器的 Centos 7 服务器上安装Nginx作为Apache的反向代理服务器. +Nginx 是最快和最强大的 Web 服务器之一,以其高性能和低资源占用率而闻名。它既可以被安装为一个独立的 Web 服务器,也可以安装成反向代理 Web 服务器。在这篇文章,我将讨论在安装了 cPanel 管理系统的 Centos 7 服务器上安装 Nginx 作为 Apache 的反向代理服务器。 -Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动态文件提供服务。这个设置将整体提高服务器的性能。 +Nginx 作为前端服务器用反向代理为静态文件提供服务,Apache 作为后端为动态文件提供服务。这个设置将整体提高服务器的性能。 让我们过一遍在已经安装好 cPanel 11.52 的 CentOS 7 x86_64 服务器上配置 Nginx 作为反向代理的安装过程。 首先,我们需要安装 EPEL 库来启动这个进程 -### 第一步: 安装EPEL库. ### +### 第一步: 安装 EPEL 库### root@server1 [/usr]# yum -y install epel-release Loaded plugins: fastestmirror, tsflags, universal-hooks @@ -26,13 +25,13 @@ Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动 Dependencies Resolved - =============================================================================================================================================== - Package Arch Version Repository Size - =============================================================================================================================================== + ======================================================================================== + Package Arch Version Repository Size + ======================================================================================== Installing: epel-release noarch 7-5 extras 14 k -### 第二步: 安装完成这个 Repo ,我们可以安装 nDeploy的 CentOS RPM 库来安装我们需要的 nDeploy Webstack 和 Nginx 插件. ### +### 第二步: 可以安装 nDeploy 的 CentOS RPM 库来安装我们所需的 nDeploy Web 类软件和 Nginx 插件 ### root@server1 [/usr]# yum -y install http://rpm.piserve.com/nDeploy-release-centos-1.0-1.noarch.rpm Loaded plugins: fastestmirror, tsflags, universal-hooks @@ -46,13 +45,13 @@ Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动 Dependencies Resolved - =============================================================================================================================================== + ======================================================================================== Package Arch Version Repository Size - =============================================================================================================================================== + ======================================================================================== Installing: nDeploy-release-centos noarch 1.0-1 /nDeploy-release-centos-1.0-1.noarch 110 -### 第三步:安装 nDeploy 和 Nginx nDeploy 插件. ### +### 第三步:安装 nDeploy 和 Nginx nDeploy 插件 ### root@server1 [/usr]# yum --enablerepo=ndeploy install nginx-nDeploy nDeploy Loaded plugins: fastestmirror, tsflags, universal-hooks @@ -65,9 +64,9 @@ Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动 Dependencies Resolved - =============================================================================================================================================== + ======================================================================================== Package Arch Version Repository Size - =============================================================================================================================================== + ======================================================================================== Installing: nDeploy noarch 2.0-11.el7 ndeploy 80 k nginx-nDeploy x86_64 1.8.0-34.el7 ndeploy 36 M @@ -79,12 +78,12 @@ Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动 python-lxml x86_64 3.2.1-4.el7 base 758 k Transaction Summary - =============================================================================================================================================== + ======================================================================================== Install 2 Packages (+5 Dependent packages) -有以上这些步骤,我们完成了在我们的服务器上 Nginx 插件的安装。现在我们可以配置 Nginx 作为反向代理和为已有的 cPanel 用户账户创建虚拟主机,为此我们可以运行如下脚本。 +通过以上这些步骤,我们完成了在我们的服务器上 Nginx 插件的安装。现在我们可以配置 Nginx 作为反向代理和为已有的 cPanel 用户账户创建虚拟主机,为此我们可以运行如下脚本。 -### 第四步:启动Nginx作为默认的前端 Web 服务器 并创建默认的配置文件。 ### +### 第四步:启动 Nginx 作为默认的前端 Web 服务器,并创建默认的配置文件### root@server1 [/usr]# /opt/nDeploy/scripts/cpanel-nDeploy-setup.sh enable Modifying apache http and https port in cpanel @@ -96,9 +95,9 @@ Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动 ConfGen:: saheetha ConfGen:: satest -你可以看到这个脚本将修改 Apache 的端口从 80 到另一个端口来让 Nginx 作为前端 Web 服务器并为现有的 cPanel 用户创建虚拟主机配置文件。一旦完成,确认 Apache 和 Nginx 的状态。 +你可以看到这个脚本将修改 Apache 的端口从 80 到另一个端口来让 Nginx 作为前端 Web 服务器,并为现有的 cPanel 用户创建虚拟主机配置文件。一旦完成,确认 Apache 和 Nginx 的状态。 -### Apache 状态: ### +#### Apache 状态:#### root@server1 [/var/run/httpd]# systemctl status httpd ● httpd.service - Apache Web Server @@ -113,7 +112,7 @@ Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动 Jan 18 06:34:23 server1.centos7-test.com apachectl[25606]: httpd (pid 24760) already running Jan 18 06:34:23 server1.centos7-test.com systemd[1]: Started Apache Web Server. -### Nginx 状态: ### +#### Nginx 状态:#### root@server1 [~]# systemctl status nginx ● nginx.service - nginx-nDeploy - high performance web server @@ -132,7 +131,7 @@ Nginx 作为反向代理为静态文件提供服务,Apache 作为后端为动 Jan 17 17:18:29 server1.centos7-test.com nginx[3804]: nginx: configuration file /etc/nginx/nginx.conf test is successful Jan 17 17:18:29 server1.centos7-test.com systemd[1]: Started nginx-nDeploy - high performance web server. -Nginx 作为前端服务器运行在 80 端口,Apache 配置被更改为监听 http 端口 9999 和 https 端口 4430。请看他们的情况 +Nginx 作为前端服务器运行在 80 端口,Apache 配置被更改为监听 http 端口 9999 和 https 端口 4430。请看他们的情况: root@server1 [/usr/local/src]# netstat -plan | grep httpd tcp 0 0 0.0.0.0:4430 0.0.0.0:* LISTEN 17270/httpd @@ -146,13 +145,13 @@ Nginx 作为前端服务器运行在 80 端口,Apache 配置被更改为监听 tcp 0 0 127.0.0.1:80 0.0.0.0:* LISTEN 17802/nginx: master tcp 0 0 45.79.183.73:80 0.0.0.0:* LISTEN 17802/nginx: master -为已有用户创建的虚拟主机的配置文件在 "**/etc/nginx/sites-enabled**". 这个文件路径包含了 Nginx 主要配置文件. +为已有用户创建的虚拟主机的配置文件在 "**/etc/nginx/sites-enabled**"。 这个文件路径包含了 Nginx 主要配置文件。 root@server1 [/etc/nginx/sites-enabled]# ll | grep .conf -rw-r--r-- 1 root root 311 Jan 17 09:02 saheetha.com.conf -rw-r--r-- 1 root root 336 Jan 17 09:02 saheethastest.com.conf -### 一个域名的示例虚拟主机: ### +#### 一个域名的示例虚拟主机:### server { @@ -168,7 +167,7 @@ Nginx 作为前端服务器运行在 80 端口,Apache 配置被更改为监听 } -我们可以启动浏览器查看完整来确定 Web 服务器的工作状态。安装后,请阅读服务器上的 web 服务信息。 +我们可以启动浏览器查看网站来确定 Web 服务器的工作状态。安装后,请阅读服务器上的 web 服务信息。 root@server1 [/home]# ip a | grep -i eth0 3: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000 @@ -178,16 +177,16 @@ Nginx 作为前端服务器运行在 80 端口,Apache 配置被更改为监听 ![webserver-status](http://blog.linoxide.com/wp-content/uploads/2016/01/webserver.png) +Nginx 将会为任何最新在 cPanel 中创建的账户创建虚拟主机。通过这些简单的的步骤,我们能够在一台 CentOS 7 / cPanel 的服务器上配置 Nginx 作为反向代理。 -Nginx 将会为任何最新在 cPanel 中创建的账户创建虚拟主机。通过这些简单的的步骤,我们能够在一台 CentOS 7 / cPanel 的服务器上配置 Nginx 作为反向代理 -### Nginx 作为反向代理的优势 : ### +### Nginx 作为反向代理的优势### - 1. 便于安装和配置 - 2. 效率高性能好 - 3. 防止 Ddos 攻击 - 4. 支持使用.htaccess作为PHP的重写规则 +1. 便于安装和配置。 +2. 效率高、性能好。 +3. 防止 Ddos 攻击。 +4. 支持使用 .htaccess 作为 PHP 的重写规则。 -我希望这篇文章对你们有用。感谢你提到它。我非常高兴收到你的宝贵意见和建议,并进一步改善。 +我希望这篇文章对你们有用。感谢你看它。我非常高兴收到你的宝贵意见和建议,并进一步改善。 -------------------------------------------------------------------------------- @@ -195,7 +194,7 @@ via: http://linoxide.com/linux-how-to/set-nginx-reverse-proxy-centos-7-cpanel/ 作者:[Saheetha Shameer][a] 译者:[bestony](https://github.com/bestony) -校对:[校对者ID](https://github.com/校对者ID) +校对:[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From b6f74e7811282e8545fa75d1757429585ad52dde Mon Sep 17 00:00:00 2001 From: "eric.xiaozhen" Date: Wed, 7 Sep 2016 10:01:08 +0800 Subject: [PATCH 0070/2437] =?UTF-8?q?eriwoon=20=E7=BF=BB=E8=AF=91=E8=AE=A4?= =?UTF-8?q?=E9=A2=86?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../20160830 The State Of JavaScript - JavaScript Flavors.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md b/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md index a455843983..df649cd722 100644 --- a/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md +++ b/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md @@ -1,3 +1,4 @@ +Eriwoon 认领翻译 The State Of JavaScript: JavaScript Flavors =========== From 8a951f6c7eeb642f6de9dfe2ac9e5bc121b049fb Mon Sep 17 00:00:00 2001 From: Bestony Date: Wed, 7 Sep 2016 11:02:59 +0800 Subject: [PATCH 0071/2437] =?UTF-8?q?=E5=BC=80=E5=A7=8B=E7=BF=BB=E8=AF=91?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 已和原译者ynmlml沟通 --- .../20160620 5 Best Linux Package Managers for Linux Newbies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md b/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md index a92c5265e5..caac94b3bc 100644 --- a/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md +++ b/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md @@ -1,4 +1,4 @@ -transalting by ynmlml +transalting by bestony 5 Best Linux Package Managers for Linux Newbies ===================================================== From a8c40380f4eaff1a30ada0ae19c1ac4a7bec0d12 Mon Sep 17 00:00:00 2001 From: Ezio Date: Wed, 7 Sep 2016 17:28:45 +0800 Subject: [PATCH 0072/2437] =?UTF-8?q?20160907-1=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...Project Documentation with GitHub Pages.md | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md diff --git a/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md b/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md new file mode 100644 index 0000000000..ff8cd2385e --- /dev/null +++ b/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md @@ -0,0 +1,49 @@ +Publish Your Project Documentation with GitHub Pages +===== + +You might be familiar with [how GitHub Pages helps you share your work with the world][3] or maybe you have [attended a class][4] that helped you build your first GitHub Pages site. Recent improvements to GitHub Pages have made it easier to [publish your site from a variety of sources][5]. One of these sources is your repository's /docs folder. + +Quality documentation is a hallmark of any healthy software project. For open-source projects, however, maintaining a robust compendium of knowledge detailing all the ins and outs is paramount. Well-curated documentation increases your project's approachability, provides asynchronous guidance, and fosters the type of uncoordinated collaboration that propels open-source software development. + +Hosting your documentation on the web can present time-consuming challenges that make publishing and maintaining it an unrewarding experience — one that it's often easy to avoid. Grappling with multiple disparate publishing tools like FTP servers and databases means files often exist in various states and multiple locations, all of which require manual synchronization. To be clear, conventional web publishing provides unparalleled flexibility and power; but it comes at the expense of simplicity and, in many cases, utility. + +When it comes to documentation, a path with less resistance is often the better approach. + +[GitHub Pages][2] gives you a direct path to create websites for your projects, which makes it a natural choice for publishing and maintaining documentation. Because GitHub Pages supports Jekyll, you can pen your documentation in plain text or Markdown to help maintain a lower barrier to contribution. Jekyll also includes support for many helpful tools like variables, templates, and automatic code highlighting, which gives you much of the flexibility you'd find in a bulkier platform without the added complexity. + +Most importantly, using GitHub Pages means your documentation lives alongside your code on GitHub, where you can use things like Issues and Pull Requests to ensure it receives the high level of care it deserves; and because GitHub Pages lets you publish from the /docs directory on the master branch, you can maintain your codebase and its published documentation on the same branch. + +### Get started today + +Publishing your first documentation page only takes a few minutes. + +1. Create a /docs/index.md file on your repository's master branch. + +2. Add your content and any necessary Jekyll front matter, then commit your changes. + +![](https://cloud.githubusercontent.com/assets/3477155/17778793/47c5a586-6533-11e6-982c-ebd41ec6968c.gif) + +1. Visit your repository's settings tab and select master branch /docs folder as the GitHub Pages source. Click save, and you're done. + +![](https://cloud.githubusercontent.com/assets/3477155/17778792/47c2ecc4-6533-11e6-828a-91980daa7297.gif) + +GitHub Pages will read the contents of your /docs directory, convert the index.md into HTML, and publish the results at your GitHub Pages URL. + +This will generate the most basic HTML output that you can further customize with templates, CSS, and other features available in Jekyll. To see examples of what all is possible, take a look at the [GitHub Pages Showcase][1]. + +-------------------------------------------------------------------------------- + +via: https://github.com/blog/2233-publish-your-project-documentation-with-github-pages + +作者:[ loranallensmith ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://github.com/loranallensmith +[1]: https://github.com/showcases/github-pages-examples +[2]: https://pages.github.com/ +[3]: https://www.youtube.com/watch?v=2MsN8gpT6jY +[4]: https://www.youtube.com/watch?v=RaKX4A5EiQo +[5]: https://help.github.com/articles/configuring-a-publishing-source-for-github-pages/ From 131f1dc054ba5efa6e8cd82cd8fc28bfede8613e Mon Sep 17 00:00:00 2001 From: Ezio Date: Wed, 7 Sep 2016 17:36:13 +0800 Subject: [PATCH 0073/2437] =?UTF-8?q?20160907-2=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...0160817 Build an integration for GitHub.md | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 sources/tech/20160817 Build an integration for GitHub.md diff --git a/sources/tech/20160817 Build an integration for GitHub.md b/sources/tech/20160817 Build an integration for GitHub.md new file mode 100644 index 0000000000..2a9c57b3bb --- /dev/null +++ b/sources/tech/20160817 Build an integration for GitHub.md @@ -0,0 +1,54 @@ +Build an integration for GitHub +============= + +![](https://cloud.githubusercontent.com/assets/4432363/17537572/934e2f9c-5e52-11e6-9c24-50c7be5b5ae3.png) + +You can now access more tools from our [Integrations Directory][1]. The Directory now has more than 15 categories — from [API management][2] to [application monitoring][3], there's a GitHub integration to support every stage of your development cycle. + +We're inviting developers of all levels of expertise to create an integration that will help developers work better. If you've built a great integration for GitHub, we want to highlight it! [Gitter][4], [AppVeyor][5], and [ZenHub][6] did it, and so can you. + +### What we're looking for + +Good software development relies on quality tools and developers today have a wide range of choices—language, framework, workflow, and environment among other factors. We're looking for tools that create a better overall development experience. + +#### A couple of guidelines for Integrations Directory listings: + +- Demonstrates solid growth (you should currently have over 500 users via GitHub OAuth) +- Meets our [Technical Requirements][7] +- Follows our [Terms of Service][8] and [Privacy Policy][9] +- Focuses on the software development life cycle + +### Helpful resources + +To get listed, please follow the steps outlined in the [listing requirements page][10]. + +You also should read our[ marketing guidelines][11] and the [existing Directory listings][12] to get a better idea of how to put it all together. Please draft the content for your listing in a [secret gist][13] (markdown format) and email us at . If you have any questions, please don't hesitate to reach out to us at . + +-------------------------------------------------------------------------------- + +via: https://github.com/blog/2226-build-an-integration-for-github + +作者:[chobberoni ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://github.com/chobberoni + +[1]: https://github.com/integrations +[2]: https://github.com/integrations/feature/api-management +[3]: https://github.com/integrations/feature/monitoring +[4]: https://github.com/integrations/feature/monitoring +[5]: https://github.com/integrations/appveyor +[6]: https://github.com/integrations/zenhub +[7]: https://developer.github.com/integrations-directory/getting-listed/#technical-requirements +[8]: https://help.github.com/articles/github-terms-of-service/ +[9]: https://help.github.com/articles/github-privacy-policy/ +[10]: https://developer.github.com/integrations-directory/getting-listed/ +[11]: https://developer.github.com/integrations-directory/marketing-guidelines/ +[12]: https://github.com/integrations +[13]: https://gist.github.com/ + + + From f5ed2e2f57b3c6fa1b2ebb90facf11c1e0be5fff Mon Sep 17 00:00:00 2001 From: GitFuture Date: Wed, 7 Sep 2016 19:07:53 +0800 Subject: [PATCH 0074/2437] Finish Translation --- ...n for the Android platform 101 - Part 1.md | 109 ---------- ...n for the Android platform 101 - Part 1.md | 188 ++++++++++++++++++ 2 files changed, 188 insertions(+), 109 deletions(-) delete mode 100644 sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md create mode 100644 translated/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md diff --git a/sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md b/sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md deleted file mode 100644 index 47c186ed0d..0000000000 --- a/sources/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md +++ /dev/null @@ -1,109 +0,0 @@ -Translating by GitFuture - -Dependency Injection for the Android platform 101 - Part 1 -=========================== - -![](https://d262ilb51hltx0.cloudfront.net/max/2000/1*YWlAzAY20KLLGIyyD_mzZw.png) - -When we first start studying software engineering, we usually bump into something like: - ->Software shall be SOLID. - -but what does that mean, in fact? Well, let’s just say that each character of the acronym means something really important for the architecture, such as: - -- [Single Responsibility Principle][1] -- [Open/closed principle][2] -- [Liskov substitution principle][3] -- [Interface segregation principle][4] -- [Dependency inversion principle][5] which is the core concept on which the dependency injection is based. - -Simply, we need to provide a class with all the objects it needs in order to perform its duties. - -### Overview - -Dependency Injection sounds like a very complex term for something that is, in fact, really easy and could be explained with this example: - -As we can see, in the first case, we create the dependency in the constructor, while in the second case it gets passed as a parameter. The second case is what we call dependency injection. We do this so that our class does not depend on the specific implementation of its dependency but it just uses it. -Moreover, based on whether the parameter is passed over to the constructor or to a method, we talk either about constructor dependency injection or method dependency injection: - -If you want to know more about Dependency Injection in general, be sure to check out this amazing talk from Dan Lew that actually inspired this overview. - -On Android, we have different choices when it comes to frameworks for solving this particular problem but the most famous is Dagger 2, first made by the awesome guys at Square and then evolved by Google itself. Specifically, Dagger 1 was made by the first and then Big G took over the project and created the second version, with major changes such being based on annotations and doing its job at compile time. - -### Importing the framework - -Setting up Dagger is no big deal, but it requires us to import the android-apt plugin by adding its dependency in the build.gradle file in the root directory of the project: - -``` -buildscript{ - ... - dependencies{ - ... - classpath ‘com.neenbedankt.gradle.plugins:android-apt:1.8’ - } -} -``` - -Then, we need to apply the android-apt plugin in the top part of the app’s build.gradle file, right below the Android application one: - -``` -apply plugin: ‘com.neenbedankt.android-apt’ -``` - -At this point, we just need to add the dependencies so that we are now able to use the libraries and its annotations: - -``` -dependencies{ - ... - compile ‘com.google.dagger:dagger:2.6’ - apt ‘com.google.dagger:dagger-compiler:2.6’ - provided ‘javax.annotation:jsr250-api:1.0’ -} -``` - ->The last dependency is needed because the @Generated annotation is not yet available on Android, but it’s pure Java. - -### Dagger Modules - -For injecting our dependencies, we first need to tell the framework what we can provide (i.e. the Context) and how that specific object is built. In order to do this, we annotate a specific class with the @Module annotation (so that Dagger is able to pick it up) scan for the @Provide annotated methods and generate the graph that will give us the object we request. - -Let’s see an example where we create a module that will give us the ConnectivityManager. So we need the Context that we will pass in the constructor of the module: - ->A very interesting feature of Dagger is providing a Singleton by simply annotating a method, dealing with all the issues inherited from Java by itself. - -### The Components - -Once we have a module, we need to tell Dagger where we want our dependencies to be injected: we do this in a Component, a specifically annotated interface in which we create different methods, where the parameters are the classes into which we want our dependencies to be injected. - -Let’s give an example and say that we want our MainActivity class to be able to receive the ConnectivityManager (or any other dependency in the graph). We would simply do something like this: - ->As we can see, the @Component annotation takes some parameters, one being an array of the modules supported, meaning the dependencies it can provide. In our case, that would be both the Context and the ConnectivityManager, as they are declared in the ApplicationModule class. - -### Wiring up - -At this point, what we need to do is create the Component as soon as possible (such as in the onCreate phase of the Application) and return it, so that the classes can use it for injecting the dependencies: - ->In order to have the DaggerApplicationComponent automatically generated by the framework, we need to build our projects so that Dagger can scan our codebase and generate the parts we need. - -In our MainActivity, though, the two things we need to do are annotate a property we want to inject with the @Inject annotation and invoke the method we declared in the ApplicationComponent interface (note that this last part varies based on what kind of injection we are performing, but for the moment we can leave it as is for the simplicity), so that our dependencies get injected and we can use them freely: - -### Conclusion - -Of course, we could do Dependency Injection manually, managing all the different objects, but Dagger takes away a lot of the “noise” involved with such boilerplate, giving us some nice additions (such as Singleton) that would otherwise be pretty awful to deal with in Java. - --------------------------------------------------------------------------------- - -via: https://medium.com/di-101/di-101-part-1-81896c2858a0#.3hg0jj14o - -作者:[Roberto Orgiu][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://medium.com/@_tiwiz -[1]: https://en.wikipedia.org/wiki/Single_responsibility_principle -[2]: https://en.wikipedia.org/wiki/Open/closed_principle -[3]: http://liskov_substitution_principle/ -[4]: https://en.wikipedia.org/wiki/Interface_segregation_principle -[5]: https://en.wikipedia.org/wiki/Dependency_inversion_principle diff --git a/translated/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md b/translated/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md new file mode 100644 index 0000000000..6ae6ddccfb --- /dev/null +++ b/translated/tech/20160817 Dependency Injection for the Android platform 101 - Part 1.md @@ -0,0 +1,188 @@ +安卓平台上的依赖注入 - 第一部分 +=========================== + +![](https://d262ilb51hltx0.cloudfront.net/max/2000/1*YWlAzAY20KLLGIyyD_mzZw.png) + +刚开始学习软件工程的时候,我们经常会碰到像这样的事情: + +>软件应该符合 SOLID 原则。 + +但这句话实际是什么意思?让我们看看 SOLID 中每个字母在架构里所代表的重要含义,例如: + +- [S 单职责原则][1] +- [O 开闭原则][2] +- [L Liskov 替换原则][3] +- [I 接口分离原则][4] +- [D 依赖反转原则][5] 这也是依赖注入的核心概念。 + +简单来说,我们需要提供一个类,这个类有它所需要的所有对象,以便实现其功能。 + +### 概述 + +依赖注入听起来像是描述非常复杂的东西的一个术语,但实际上它很简单,看下面这个例子你就明白了: + +``` +class NoDependencyInjection { + private Dependency d; + + public NoDependencyInjection() { + d = new Dependency(); + } +} + +class DependencyInjection { + private Dependency d; + + public DependencyInjection(Dependency d) { + this.d = d; + } +} +``` + +正如我们所见,第一种情况是我们在构造器里创建了依赖对象,但在第二种情况下,它作为参数被传递给构造器,这就是我们所说的依赖注入。这样做是为了让我们所写的类不依靠特定依赖关系的实现,却能直接使用它。 + +参数传递的目标是构造器,我们就称之为构造器依赖注入;或者是某个方法,就称之为方法依赖注入: + +``` +class Example { + private ConstructorDependency cd; + private MethodDependency md; + Example(ConstructorDependency cd) { + this.cd = cd; //Constructor Dependency Injection + } + + public setMethodDependency(MethodDependency md) { + this.md = md; //Method Dependency Injection + } +} +``` + + +要是你想总体深入地了解依赖注入,可以看看由 [Dan Lew][t2] 发表的[精彩的演讲][t1],事实上是这个演讲启迪了这个概述。 + +在 Android 平台,当需要框架来处理依赖注入这个特殊的问题时,我们有不同的选择,其中最有名的框架就是 [Dagger 2][t3]。它最开始是由 Square 公司(译者注:Square 是美国一家移动支付公司)里一些很棒的开发者开发出来的,然后慢慢发展成由 Google 自己开发。特别地,Dagger 1 先被开发出来,然后 Big G 接手这个项目,做了很多改动,比如以注释为基础,在编译的时候就完成 Dagger 的任务,也就是第二个版本。 + +### 导入框架 + +安装 Dagger 并不难,但需要导入 `android-apt` 插件,通过向项目的根目录下的 build.gradle 文件中添加它的依赖关系: + +``` +buildscript{ + ... + dependencies{ + ... + classpath ‘com.neenbedankt.gradle.plugins:android-apt:1.8’ + } +} +``` + +然后,我们需要将 `android-apt` 插件应用到项目 build.gradle 文件,放在文件顶部 Android 应用那一句的下一行: + +``` +apply plugin: ‘com.neenbedankt.android-apt’ +``` + +这个时候,我们只用添加依赖关系,然后就能使用库和注释了: + +``` +dependencies{ + ... + compile ‘com.google.dagger:dagger:2.6’ + apt ‘com.google.dagger:dagger-compiler:2.6’ + provided ‘javax.annotation:jsr250-api:1.0’ +} +``` + +>需要加上最后一个依赖关系是因为 @Generated 注解在 Android 里还不可用,但它是[原生的 Java 注解][t4]。 + +### Dagger 模块 + +要注入依赖,首先需要告诉框架我们能提供什么(比如说上下文)以及特定的对象应该怎样创建。为了完成注入,我们用 `@Module` 注释对一个特殊的类进行了注解(这样 Dagger 就能识别它了),寻找 `@Provide` 标记的方法,生成图表,能够返回我们所请求的对象。 + +看下面的例子,这里我们创建了一个模块,它会返回给我们 `ConnectivityManager`,所以我们要把 `Context` 对象传给这个模块的构造器。 + +``` +@Module +public class ApplicationModule { + private final Context context; + + public ApplicationModule(Context context) { + this.context = context; + } + + @Provides @Singleton + public Context providesContext() { + return context; + } + + @Provides @Singleton + public ConnectivityManager providesConnectivityManager(Context context) { + return (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE); + } +} +``` + +>Dagger 中十分有意思的一点是只用在一个方法前面添加一个 Singleton 注解,就能处理所有从 Java 中继承过来的问题。 + +### 容器 + +当我们有一个模块的时候,我们需要告诉 Dagger 想把依赖注入到哪里:我们在一个容器里,一个特殊的注解过的接口里完成依赖注入。我们在这个接口里创造不同的方法,而接口的参数是我们想注入依赖关系的类。 + +下面给出一个例子并告诉 Dagger 我们想要 `MainActivity` 类能够接受 `ConnectivityManager`(或者在图表里的其它依赖对象)。我们只要做类似以下的事: + +``` +@Singleton +@Component(modules = {ApplicationModule.class}) +public interface ApplicationComponent { + + void inject(MainActivity activity); +} +``` + +>正如我们所见,@Component 注解有几个参数,一个是所支持的模块的数组,意味着它能提供的依赖。这里既可以是 Context 也可以是 ConnectivityManager,因为他们在 ApplicationModule 类中有声明。 + +### 使用 + +这时,我们要做的是尽快创建容器(比如在应用的 onCreate 方法里面)并且返回这个容器,那么类就能用它来注入依赖了: + +>为了让框架自动生成 DaggerApplicationComponent,我们需要构建项目以便 Dagger 能够扫描我们的代码库,并且生成我们需要的部分。 + +在 `MainActivity` 里,我们要做的两件事是用 `@Inject` 注解符对想要注入的属性进行注释,调用我们在 `ApplicationComponent` 接口中声明的方法(请注意后面一部分会因我们使用的注入类型的不同而变化,但这里简单起见我们不去管它),然后依赖就被注入了,我们就能自由使用他们: + +``` +public class MainActivity extends AppCompatActivity { + @Inject + ConnectivityManager manager; + + @Override + protected void onCreate(Bundle savedInstanceState) { + ... + ((App) getApplication()).getComponent().inject(this); + } +} +``` + +### 总结 + +当然了,我们可以手动注入依赖,管理所有不同的对象,但 Dagger 打消了很多有关模板的“噪声”,Dagger 给我们有用的附加品(比如 `Singleton`),而仅用 Java 处理将会很糟糕。 + +-------------------------------------------------------------------------------- + +via: https://medium.com/di-101/di-101-part-1-81896c2858a0#.3hg0jj14o + +作者:[Roberto Orgiu][a] +译者:[GitFuture](https://github.com/GitFuture) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://medium.com/@_tiwiz +[1]: https://en.wikipedia.org/wiki/Single_responsibility_principle +[2]: https://en.wikipedia.org/wiki/Open/closed_principle +[3]: http://liskov_substitution_principle/ +[4]: https://en.wikipedia.org/wiki/Interface_segregation_principle +[5]: https://en.wikipedia.org/wiki/Dependency_inversion_principle +[t1]: https://realm.io/news/daniel-lew-dependency-injection-dagger/ +[t2]: https://twitter.com/danlew42 +[t3]: http://google.github.io/dagger/ +[t4]: https://docs.oracle.com/javase/7/docs/api/javax/annotation/Generated.html From f1fe0d05c0c7811378e656717db35f3dfc93ae2e Mon Sep 17 00:00:00 2001 From: John Joe Date: Wed, 7 Sep 2016 20:50:26 +0800 Subject: [PATCH 0075/2437] Update 20160820 Building your first Atom plugin.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 翻译选题 --- sources/tech/20160820 Building your first Atom plugin.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160820 Building your first Atom plugin.md b/sources/tech/20160820 Building your first Atom plugin.md index 76915cc37d..8cecc9c09f 100644 --- a/sources/tech/20160820 Building your first Atom plugin.md +++ b/sources/tech/20160820 Building your first Atom plugin.md @@ -1,3 +1,4 @@ +Tranlating by lengmi. Building your first Atom plugin ===== From 26b001a81403255fc38f119fe735bd230f60a304 Mon Sep 17 00:00:00 2001 From: vim-kakali <1799225723@qq.com> Date: Wed, 7 Sep 2016 20:58:15 +0800 Subject: [PATCH 0076/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E4=B8=AD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...0627 TOP 5 BEST VIDEO EDITING SOFTWARE FOR LINUX IN 2016.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sources/tech/20160627 TOP 5 BEST VIDEO EDITING SOFTWARE FOR LINUX IN 2016.md b/sources/tech/20160627 TOP 5 BEST VIDEO EDITING SOFTWARE FOR LINUX IN 2016.md index 44c66359f9..e0f69aca84 100644 --- a/sources/tech/20160627 TOP 5 BEST VIDEO EDITING SOFTWARE FOR LINUX IN 2016.md +++ b/sources/tech/20160627 TOP 5 BEST VIDEO EDITING SOFTWARE FOR LINUX IN 2016.md @@ -1,3 +1,6 @@ +vim-kakali translating + + TOP 5 BEST VIDEO EDITING SOFTWARE FOR LINUX IN 2016 ===================================================== From 293f8705fa5cb34de1b852a57334077afb7312e2 Mon Sep 17 00:00:00 2001 From: wxy Date: Thu, 8 Sep 2016 06:55:10 +0800 Subject: [PATCH 0077/2437] PUB:20160509 Android vs. iPhone Pros and Cons @joVoV --- ...160509 Android vs. iPhone Pros and Cons.md | 89 +++++++++++++++++++ ...160509 Android vs. iPhone Pros and Cons.md | 87 ------------------ 2 files changed, 89 insertions(+), 87 deletions(-) create mode 100644 published/20160509 Android vs. iPhone Pros and Cons.md delete mode 100644 translated/talk/20160509 Android vs. iPhone Pros and Cons.md diff --git a/published/20160509 Android vs. iPhone Pros and Cons.md b/published/20160509 Android vs. iPhone Pros and Cons.md new file mode 100644 index 0000000000..76993e842c --- /dev/null +++ b/published/20160509 Android vs. iPhone Pros and Cons.md @@ -0,0 +1,89 @@ +对比 Android 和 iPhone 的优缺点 +=================================== + +>当我们比较 Android 与 iPhone 的时候,很显然 Android 具有一定的优势,而 iPhone 则在一些关键方面更好。但是,究竟哪个比较好呢? + +对 Android 与 iPhone 比较是个个人的问题。 + +就好比我来说,我两个都用。我深知这两个平台的优缺点。所以,我决定分享我关于这两个移动平台的观点。另外,然后谈谈我对新的 Ubuntu 移动平台的印象和它的优势。 + +### iPhone 的优点 + +虽然这些天我是个十足的 Android 用户,但我必须承认 iPhone 在某些方面做的是不错。首先,苹果公司在他们的设备更新方面有更好的成绩。这对于运行着 iOS 的旧设备来说尤其是这样。反观 Android ,如果不是谷歌亲生的 Nexus,它最好也不过是一个更高端的运营商支持的手机,你将发现它们的更新少的可怜或者根本没有。 + +其中 iPhone 做得很好的另一个领域是应用程序的可用性。展开来说,iPhone 应用程序几乎总是有一个简洁的外观。这并不是说 Android 应用程序就是丑陋的,而是,它们可能只是没有像 iOS 上一样的保持不变的操控习惯和一以贯之的用户体验。两个典型的例子, [Dark Sky][1] (天气)和 [Facebook Paper][2] 很好表现了 iOS 独有的布局。 + +再有就是备份过程。 Android 可以备份,默认情况下是备份到谷歌。但是对应用数据起不了太大作用。对比 iPhone ,iCloud 基本上可以对你的 iOS 设备进行了完整备份。 + +### iPhone 令我失望的地方 + +对 iPhone 来说,最无可争辩的问题是它的硬件限制要比软件限制更大,换句话来说,就是存储容量问题。 + +你看,对于大多数 Android 手机,我可以买一个容量较小的手机,然后以后可以添加 SD 卡。这意味着两件事:第一,我可以使用 SD 卡来存储大量的媒体文件。其次,我甚至可以用 SD 卡来存储“一些”我的应用程序。而苹果完全不能这么做。 + +另一个 iPhone 让我失望的地方是它提供的选择很少。备份您的设备?希望你喜欢 iTunes 或 iCloud 吧。但对一些像我一样用 Linux 的人,那就意味着,我唯一的选择便是使用 iCloud。 + +要公平的说,如果你愿意越狱,你的 iPhone 还有一些其他解决方案的,但这并不是这篇文章所讲的。 Android 的 解锁 root 也一样。本文章针对的是两个平台的原生设置。 + +最后,让我们不要忘记这件看起来很小的事—— [iTunes 会删掉用户的音乐][3],因为它认为和苹果音乐的内容重复了……或者因为一些其它的类似规定。这不是 iPhone 特有的情况?我不同意,因为那些音乐最终就是在 iPhone 上没有了。我能十分肯定地这么说,是因为不管在哪里我都不会说这种谎话。 + +![](http://www.datamation.com/imagesvr_ce/5552/mobile-abstract-icon-200x150.jpg) + +*Android 和 iPhone 的对决取决于什么功能对你来说最重要。* + +### Android 的优点 + +Android 给我最大的好处就是 iPhone 所提供不了的:选择。这包括对应用程序、设备以及手机是整体如何工作的选择。 + +我爱桌面小工具!对于 iPhone 用户来说,它们也许看上去很蠢。但我可以告诉你,它们可以让我不用打开应用程序就可以看到所需的数据,而无需额外的麻烦。另一个类似的功能,我喜欢安装定制的桌面界面,而不是我的手机默认的那个! + +最后,我可以通过像 [Airdroid][4] 和 [Tasker][5] 这样的工具给我的智能手机添加计算机级的完整功能。AirDroid 可以让我把我的 Android 手机当成带有一个文件管理和通信功能的计算机——这可以让我可以轻而易举的使用鼠标和键盘。Tasker 更厉害,我可以用它让我手机根据环境变得可联系或不可联系,当我设置好了之后,当我到会议室之后我的手机就会自己进入会议模式,甚至变成省电模式。我还可以设置它当我到达特定的目的地时自动启动某个应用程序。 + +### Android 让我失望的地方 + +Android 备份选项仅限于特定的用户数据,而不是手机的完整克隆。如果不解锁 root,要么你只能听之任之,要么你必须用 Android SDK 来解决。期望普通用户会解锁 root 或运行 SDK 来完成备份(我的意思是一切都备份)显然是一个笑话。 + +是的,谷歌的备份服务会备份谷歌应用程序的数据、以及其他相关的自定义设置。但它是远不及我们所看到的苹果服务一样完整。为了完成类似于苹果那样的功能,我发现你就要么必须解锁 root ,要么将其连接到一个在 PC 机上使用一些不知道是什么的软件来干这个。 + +不过,公平的说,我知道使用 Nexus 的人能从该设备特有的[完整备份服务][6]中得到帮助。对不起,但是谷歌的默认备份方案是不行的。对于通过在 PC 上使用 adb (Android Debug Bridge) 来备份也是一样的——不会总是如预期般的恢复。 + +等吧,它会变好的。经过了很多失败的失望和挫折之后,我发现有一个应用程序,看起来它“可能”提供了一点点微小的希望,它叫 Helium 。它不像我发现的其他应用程序那样拥有误导性的和令人沮丧的局限性,[Helium][7] 最初看起来像是谷歌应该一直提供的备份应用程序——注意,只是“看起来像”。可悲的是,它绊了我一跤。第一次运行时,我不仅需要将它连接到我的计算机上,甚至使用他们提供的 Linux 脚本都不工作。在删除他们的脚本后,我弄了一个很好的老式 adb 来备份到我的 Linux PC 上。你可能要知道的是:你需要在开发工具里打开一箩筐东西,而且如果你运行 Twilight 应用的话还要关闭它。当 adb 的备份选项在我的手机上不起作用时,我花了一点时间把这个搞好了。 + +最后,Android 为非 root 用户也提供了可以轻松备份一些如联系人、短信等简单东西的选择。但是,要深度手机备份的话,以我经验还是通过有线连接和 adb。 + +### Ubuntu 能拯救我们吗? + +在手机领域,通过对这两大玩家之间的优劣比较,我们很期望从 Ubuntu 看到更好的表现。但是,迄今为止,它的表现相当低迷。 + +我喜欢开发人员基于这个操作系统正在做的那些努力,我当然想在 iPhone 和 Android 手机之外有第三种选择。但是不幸的是,它在手机和平板上并不受欢迎,而且由于硬件的低端以及在 YouTube 上的糟糕的演示,有很多不好的传闻。 + +公平来说,我在以前也用过 iPhone 和 Android 的低端机,所以这不是对 Ubuntu 的挖苦。但是它要表现出准备与 iPhone 和 Android 相竞争的功能生态时,那就另说了,这还不是我现在特别感兴趣的东西。在以后的日子里,也许吧,我会觉得 Ubuntu 手机可以满足我的需要了。 + +###Android 与 iPhone 之争:为什么 Android 将终究赢得胜利 + +忽视 Android 那些痛苦的缺点,它起码给我了选择。它并没有把我限制在只有两种备份数据的方式上。是的,一些 Android 的限制事实上是由于它的关注点在于让我选择如何处理我的数据。但是,我可以选择我自己的设备,想加内存就加内存。Android 可以我让做很多很酷的东西,而 iPhone 根本就没有能力这些事情。 + +从根本上来说, Android 给非 root 用户提供了访问手机功能的更大自由。无论是好是坏,这是人们想要的一种自由。现在你们其中有很多 iPhone 的粉丝应该感谢像 [libimobiledevice][8] 这样项目带来的影响。看看苹果阻止 Linux 用户所做的事情……然后问问你自己:作为一个 Linux 用户这真的值得吗? + +发表下评论吧,分享你对 iPhone 、Android 或 Ubuntu 的看法。 + +------------------------------------------------------------------------------ + +via: http://www.datamation.com/mobile-wireless/android-vs.-iphone-pros-and-cons.html + +作者:[Matt Hartley][a] +译者:[jovov](https://github.com/jovov) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.datamation.com/author/Matt-Hartley-3080.html +[1]: http://darkskyapp.com/ +[2]: https://www.facebook.com/paper/ +[3]: https://blog.vellumatlanta.com/2016/05/04/apple-stole-my-music-no-seriously/ +[4]: https://www.airdroid.com/ +[5]: http://tasker.dinglisch.net/ +[6]: https://support.google.com/nexus/answer/2819582?hl=en +[7]: https://play.google.com/store/apps/details?id=com.koushikdutta.backup&hl=en +[8]: http://www.libimobiledevice.org/ + diff --git a/translated/talk/20160509 Android vs. iPhone Pros and Cons.md b/translated/talk/20160509 Android vs. iPhone Pros and Cons.md deleted file mode 100644 index efbef243b9..0000000000 --- a/translated/talk/20160509 Android vs. iPhone Pros and Cons.md +++ /dev/null @@ -1,87 +0,0 @@ -Android 和 iPhone 的优缺点 -=================================== - - ->当我们比较 Android 与 iPhone 的时候,很显然 iPhone 和 Android 在一些关键方面都具有一定的优势,但是,究竟哪个比较好呢? - - Android 与 iPhone 两者比较是个私人的问题。 - -就好比我来说,我两个都用。我深知这两个平台的优劣势。所以,我决定分享关于这两个移动平台的观点。另外,然后谈谈我对新的 Ubuntu 移动平台的印象和它的优势。 - -### iPhone 的优点 - -虽然这些天我是个十足的 Android 用户,但我必须承认到 iPhone 在某些方面做的是不错。首先,苹果公司在更新他们的设备有更好的战绩。尤其是在旧设备也能运行 iOS 。反观 Android ,如果它不是谷歌福利的关系,它最好是一个更高端的运营商支持的手机。否则,你将发现找那些更新少的可怜或者不存在。 - -其中 iPhone 做得很好的另一个领域是应用程序的可用性。扩展上: iPhone 应用程序几乎总是有一个简洁的外观。这并不是说 Android 应用程序是难看的,相反,他们可能没有和预期的流动性和一致性当建立在 iOS 上。有两个例子 [Dark Sky][1] (天气)和 [Facebook Paper][2] 很好表现了 iOS 的布局。 - -再有就是备份过程。 Android 可以备份,默认情况下是备份到谷歌。但是对应用数据起不了太多作用。对比 iPhone ,iCloud 基本上可以让你的 iOS 设备进行完整备份。 - -### iPhone 令我失望的地方 - -我使用 iPhone 的最大的不容置疑的问题是它的硬件限制大于软件,换句话来说,就是存储问题。 - -你看,对于大多数 Android 手机,我可以买一个容量较小的手机,然后在以后添加 SD 卡。这做了两件事:第一,我可以使用 SD 卡来存储大量的媒体文件。其次,我甚至可以用 SD 卡来存储我的应用程序的一些文件。苹果完全不能这么做。 - -另一个 iPhone 让我失望的地方是它提供的选择很少。备份您的设备?希望大家喜欢 iTunes 或 iCloud 。但对一些像我一样用 Linux 的人,那就意味着,我唯一的选择便是使用 iCloud 。 - -为了最终公平的,如果你愿意越狱,你的 iPhone 还有一些其他解决方案的。但这并不是文章所讲的。 Android 的 root 也一样。本文章针对的是两个用户的原生设置。 - -最后,让我们不要忘记这小小的玩意儿—— [iTunes 决定删除用户的音乐][3] 因为它被视为苹果音乐内容的重复...或者类似的规定。 iPhone 并不明确?我不同意,就算音乐在有些时候很好地在结束在 iPhone 上。我也十分肯定地说在任何地方我会不会忍受这种废话! - -![](http://www.datamation.com/imagesvr_ce/5552/mobile-abstract-icon-200x150.jpg) -> Android 和 iPhone 的对决取决于什么功能对你来说最重要。 - -### Android 的优点 - - Android 给我最大的事情就是 iPhone 提供不了的选择。包括应用程序,设备和我的手机是如何工作的整体布局。 - -我爱桌面小工具!对于 iPhone 用户,它们也许看上去很蠢。但我可以告诉你,他们可以让我不用打开应用程序就可以看到所需的数据,而无需额外的麻烦。另一个类似的功能,我喜欢安装自定义应用,而不是我的手机的默认! - -最后,我可以利用像 [Airdroid][4] 和 [Tasker][5] 工具添加全电脑式的功能到我的智能手机。AirDroid 可以让我对待我的 Android 手机就像一个文件管理和通信的计算机–这使得使用我的鼠标和键盘变得轻而易举的。Tasker 很厉害,我可以用它让我手机可联系或不可联系,当我设置参数时,我可以把我的手机处在会议模式,甚至把它自己变成省电模式。我甚至可以设置它来启动应用程序时当我到达特定的目的地时。 - -### Android 让我失望的地方 - -备份选项仅限于特定的用户数据,而不是手机的完整克隆。没有 root ,你将要么乘风而去或者你必须看看 Android SDK 开发解决方案。期望普通用户 root 他们的手机或运行 SDK来进行所有的Android的备份(我的意思是一切)将是一个笑话。 - -是的,谷歌的备份服务将备份谷歌应用程序的数据,以及其他相关的自定义设置。但它是远不及我们所看到的苹果一样完整。为了完成类似于在苹果的功能,我发现你就必须要 root 你的安卓手机或利用一些随机程序将其连接到一个在 PC 机上来。 - -为了公平的,但是,我相信 Nexus 所有者受益于一个 [完整备份服务][6] ,这是设备特定。对不起,但谷歌的默认备份是不削减它。同样的应用来备份您的电脑——他们不总是恢复预期的东西。 - -等待,它会变得更好。现在经过了很多失败的失望和挫折,我发现有一个应用程序,看起来它“可能”提供了一个微小的希望,它被称为 Helium 。它不像我发现的其他应用程序那样拥有误导性的和令人沮丧的局限性,[Helium][7] 最初看起来像是谷歌应该一直提供的备份应用程序——强调“看起来像”。可悲的是,这是一个巨大的失望。我不仅需要将它连接到我的计算机上进行第一次运行,而且它甚至不使用他们提供的 Linux 脚本。删除他们的脚本后,我弄了一个很好的老式 adb (Android Debug Bridge) 备份到我的Linux PC 。有趣的事实:你需要在开发工具里打开一箩筐东西,再加上如果你运行 Twilight app,那是需要被关闭的。当 adb (Android Debug Bridge) 的备份选项在手机上不起作用时,它花了我一点时间把这个弄在一起。 - -最终,Android 为非 root 用户也提供了可以轻松备份一些如联系人,短信等简单东西的选择。但是,要深度手机备份的话,以我经验还是通过有线连接和 adb (Android Debug Bridge) 。 - -### Ubuntu 会救我们吗? - -在手机领域,通过两大玩家之间的好坏考核,我们将有很多的希望从 Ubuntu 看到好的一方面。但是,迄今为止,它已经相当低迷。 - -我喜欢开发人员正在基于 OS 所做的,我当然喜欢除了 iPhone 和 Android 手机的第三个选项的想法。但是不幸的是,它在手机和平板上并不受欢迎且受到很多坏新闻,就是由于不符合标准的硬件和一个 YouTube 上的糟糕的示范。 - -公平来说,我在以前用 iPhone 和 Android 也不是很规范。所以这不是对 Ubuntu 的挖苦。但是直到它开始表现出准备提供生态系统功能来与 iPhone 和 Android 竞争,那就另说了,这还不是我现在特别感兴趣的东西。在以后的日子里,也许,我会觉得 Ubuntu 手机可以满足我的需要了。 - -###Android pk iPhone:为什么Android 长期胜利 - -忽视 Android 那些痛苦的缺点,它起码对待我像一个成年人。它并没有把我困在只有两种方法来备份我的数据。是的,一些 Android 的限制是由于它的关注点在让我选择如何处理我的数据。但是,我也可以选择我自己的设备,一时兴起扩充内存。 Android 让我做了很多很酷的东西,那些手机根本就没有能力做的事情。 - -在其核心, Android 给非 root 用户提供更大的访问手机的功能。无论是好是坏,这是人们倾向的一种自由。现在你们其中有很多用 iPhone 谩骂的人多亏了像 [libimobiledevice][8] 类似影响的项目。但要看看苹果阻止 Linux 用户所做的事情……然后问自己:作为一个 Linux 用户这是真的值得吗?评论,分享你对 iPhone 、 Android 或 Ubuntu 的看法。 - ------------------------------------------------------------------------------- - -via: http://www.datamation.com/mobile-wireless/android-vs.-iphone-pros-and-cons.html - -作者:[Matt Hartley][a] -译者:[jovov](https://github.com/jovov) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.datamation.com/author/Matt-Hartley-3080.html -[1]: http://darkskyapp.com/ -[2]: https://www.facebook.com/paper/ -[3]: https://blog.vellumatlanta.com/2016/05/04/apple-stole-my-music-no-seriously/ -[4]: https://www.airdroid.com/ -[5]: http://tasker.dinglisch.net/ -[6]: https://support.google.com/nexus/answer/2819582?hl=en -[7]: https://play.google.com/store/apps/details?id=com.koushikdutta.backup&hl=en -[8]: http://www.libimobiledevice.org/ - From cfd8d78fc87e1135bfa4360ec69720764a6c4163 Mon Sep 17 00:00:00 2001 From: wxy Date: Thu, 8 Sep 2016 14:18:07 +0800 Subject: [PATCH 0078/2437] PUB:20160527 A Python Interpreter Written in Python MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @qingyunha 好长的文章,翻译的不错! --- ... A Python Interpreter Written in Python.md | 205 +++++++++--------- 1 file changed, 100 insertions(+), 105 deletions(-) rename {translated/tech => published}/20160527 A Python Interpreter Written in Python.md (53%) diff --git a/translated/tech/20160527 A Python Interpreter Written in Python.md b/published/20160527 A Python Interpreter Written in Python.md similarity index 53% rename from translated/tech/20160527 A Python Interpreter Written in Python.md rename to published/20160527 A Python Interpreter Written in Python.md index dbf8dc7cf4..18c2feedd7 100644 --- a/translated/tech/20160527 A Python Interpreter Written in Python.md +++ b/published/20160527 A Python Interpreter Written in Python.md @@ -1,37 +1,37 @@ -用Python实现Python解释器 +用 Python 实现 Python 解释器 === -_Allison是Dropbox的工程师,在那里她维护着世界上最大的由Python客户组成的网络。在Dropbox之前,她是Recurse Center的导师, 曾在纽约写作。在北美的PyCon做过关于Python内部机制的演讲,并且她喜欢奇怪的bugs。她的博客地址是[akaptur.com](http://akaptur.com)._ +_Allison 是 Dropbox 的工程师,在那里她维护着这个世界上最大的 Python 客户端网络之一。在去 Dropbox 之前,她是 Recurse Center 的协调人, 是这个位于纽约的程序员深造机构的作者。她在北美的 PyCon 做过关于 Python 内部机制的演讲,并且她喜欢研究奇怪的 bug。她的博客地址是 [akaptur.com](http://akaptur.com)。_ -## Introduction +### 介绍 -Byterun是一个用Python实现的Python解释器。随着我在Byterun上的工作,我惊讶并很高兴地的发现,这个Python解释器的基础结构可以满足500行的限制。在这一章我们会搞清楚这个解释器的结构,给你足够的知识探索下去。我们的目标不是向你展示解释器的每个细节---像编程和计算机科学其他有趣的领域一样,你可能会投入几年的时间去搞清楚这个主题。 +Byterun 是一个用 Python 实现的 Python 解释器。随着我对 Byterun 的开发,我惊喜地的发现,这个 Python 解释器的基础结构用 500 行代码就能实现。在这一章我们会搞清楚这个解释器的结构,给你足够探索下去的背景知识。我们的目标不是向你展示解释器的每个细节---像编程和计算机科学其他有趣的领域一样,你可能会投入几年的时间去深入了解这个主题。 -Byterun是Ned Batchelder和我完成的,建立在Paul Swartz的工作之上。它的结构和主要的Python实现(CPython)差不多,所以理解Byterun会帮助你理解大多数解释器特别是CPython解释器。(如果你不知道你用的是什么Python,那么很可能它就是CPython)。尽管Byterun很小,但它能执行大多数简单的Python程序。 +Byterun 是 Ned Batchelder 和我完成的,建立在 Paul Swartz 的工作之上。它的结构和主要的 Python 实现(CPython)差不多,所以理解 Byterun 会帮助你理解大多数解释器,特别是 CPython 解释器。(如果你不知道你用的是什么 Python,那么很可能它就是 CPython)。尽管 Byterun 很小,但它能执行大多数简单的 Python 程序(这一章是基于 Python 3.5 及其之前版本生成的字节码的,在 Python 3.6 中生成的字节码有一些改变)。 -### A Python Interpreter +#### Python 解释器 -在开始之前,让我们缩小一下“Pyhton解释器”的意思。在讨论Python的时候,“解释器”这个词可以用在很多不同的地方。有的时候解释器指的是REPL,当你在命令行下敲下`python`时所得到的交互式环境。有时候人们会相互替代的使用Python解释器和Python来说明执行Python代码的这一过程。在本章,“解释器”有一个更精确的意思:执行Python程序过程中的最后一步。 +在开始之前,让我们限定一下“Pyhton 解释器”的意思。在讨论 Python 的时候,“解释器”这个词可以用在很多不同的地方。有的时候解释器指的是 Python REPL,即当你在命令行下敲下 `python` 时所得到的交互式环境。有时候人们会或多或少的互换使用 “Python 解释器”和“Python”来说明从头到尾执行 Python 代码的这一过程。在本章中,“解释器”有一个更精确的意思:Python 程序的执行过程中的最后一步。 -在解释器接手之前,Python会执行其他3个步骤:词法分析,语法解析和编译。这三步合起来把源代码转换成_code object_,它包含着解释器可以理解的指令。而解释器的工作就是解释code object中的指令。 +在解释器接手之前,Python 会执行其他 3 个步骤:词法分析,语法解析和编译。这三步合起来把源代码转换成代码对象(code object),它包含着解释器可以理解的指令。而解释器的工作就是解释代码对象中的指令。 -你可能很奇怪执行Python代码会有编译这一步。Python通常被称为解释型语言,就像Ruby,Perl一样,它们和编译型语言相对,比如C,Rust。然而,这里的术语并不是它看起来的那样精确。大多数解释型语言包括Python,确实会有编译这一步。而Python被称为解释型的原因是相对于编译型语言,它在编译这一步的工作相对较少(解释器做相对多的工作)。在这章后面你会看到,Python的编译器比C语言编译器需要更少的关于程序行为的信息。 +你可能很奇怪执行 Python 代码会有编译这一步。Python 通常被称为解释型语言,就像 Ruby,Perl 一样,它们和像 C,Rust 这样的编译型语言相对。然而,这个术语并不是它看起来的那样精确。大多数解释型语言包括 Python 在内,确实会有编译这一步。而 Python 被称为解释型的原因是相对于编译型语言,它在编译这一步的工作相对较少(解释器做相对多的工作)。在这章后面你会看到,Python 的编译器比 C 语言编译器需要更少的关于程序行为的信息。 -### A Python Python Interpreter +#### Python 的 Python 解释器 -Byterun是一个用Python写的Python解释器,这点可能让你感到奇怪,但没有比用C语言写C语言编译器更奇怪。(事实上,广泛使用的gcc编译器就是用C语言本身写的)你可以用几乎的任何语言写一个Python解释器。 +Byterun 是一个用 Python 写的 Python 解释器,这点可能让你感到奇怪,但没有比用 C 语言写 C 语言编译器更奇怪的了。(事实上,广泛使用的 gcc 编译器就是用 C 语言本身写的)你可以用几乎任何语言写一个 Python 解释器。 -用Python写Python既有优点又有缺点。最大的缺点就是速度:用Byterun执行代码要比用CPython执行慢的多,CPython解释器是用C语言实现的并做了优化。然而Byterun是为了学习而设计的,所以速度对我们不重要。使用Python最大优点是我们可以*仅仅*实现解释器,而不用担心Python运行时的部分,特别是对象系统。比如当Byterun需要创建一个类时,它就会回退到“真正”的Python。另外一个优势是Byterun很容易理解,部分原因是它是用高级语言写的(Python!)(另外我们不会对解释器做优化 --- 再一次,清晰和简单比速度更重要) +用 Python 写 Python 既有优点又有缺点。最大的缺点就是速度:用 Byterun 执行代码要比用 CPython 执行慢的多,CPython 解释器是用 C 语言实现的,并做了认真优化。然而 Byterun 是为了学习而设计的,所以速度对我们不重要。使用 Python 最大优势是我们可以*仅仅*实现解释器,而不用担心 Python 运行时部分,特别是对象系统。比如当 Byterun 需要创建一个类时,它就会回退到“真正”的 Python。另外一个优势是 Byterun 很容易理解,部分原因是它是用人们很容易理解的高级语言写的(Python !)(另外我们不会对解释器做优化 --- 再一次,清晰和简单比速度更重要) -## Building an Interpreter +### 构建一个解释器 -在我们考察Byterun代码之前,我们需要一些对解释器结构的高层次视角。Python解释器是如何工作的? +在我们考察 Byterun 代码之前,我们需要从高层次对解释器结构有一些了解。Python 解释器是如何工作的? -Python解释器是一个_虚拟机_,模拟真实计算机的软件。我们这个虚拟机是栈机器,它用几个栈来完成操作(与之相对的是寄存器机器,它从特定的内存地址读写数据)。 +Python 解释器是一个虚拟机(virtual machine),是一个模拟真实计算机的软件。我们这个虚拟机是栈机器(stack machine),它用几个栈来完成操作(与之相对的是寄存器机器(register machine),它从特定的内存地址读写数据)。 -Python解释器是一个_字节码解释器_:它的输入是一些命令集合称作_字节码_。当你写Python代码时,词法分析器,语法解析器和编译器生成code object让解释器去操作。每个code object都包含一个要被执行的指令集合 --- 它就是字节码 --- 另外还有一些解释器需要的信息。字节码是Python代码的一个_中间层表示_:它以一种解释器可以理解的方式来表示源代码。这和汇编语言作为C语言和机器语言的中间表示很类似。 +Python 解释器是一个字节码解释器(bytecode interpreter):它的输入是一些称作字节码(bytecode)的指令集。当你写 Python 代码时,词法分析器、语法解析器和编译器会生成代码对象(code object)让解释器去操作。每个代码对象都包含一个要被执行的指令集 —— 它就是字节码 —— 以及还有一些解释器需要的信息。字节码是 Python 代码的一个中间层表示( intermediate representation):它以一种解释器可以理解的方式来表示源代码。这和汇编语言作为 C 语言和机器语言的中间表示很类似。 -### A Tiny Interpreter +#### 微型解释器 为了让说明更具体,让我们从一个非常小的解释器开始。它只能计算两个数的和,只能理解三个指令。它执行的所有代码只是这三个指令的不同组合。下面就是这三个指令: @@ -39,7 +39,7 @@ Python解释器是一个_字节码解释器_:它的输入是一些命令集合 - `ADD_TWO_VALUES` - `PRINT_ANSWER` -我们不关心词法,语法和编译,所以我们也不在乎这些指令是如何产生的。你可以想象,你写下`7 + 5`,然后一个编译器为你生成那三个指令的组合。如果你有一个合适的编译器,你甚至可以用Lisp的语法来写,只要它能生成相同的指令。 +我们不关心词法、语法和编译,所以我们也不在乎这些指令集是如何产生的。你可以想象,当你写下 `7 + 5`,然后一个编译器为你生成那三个指令的组合。如果你有一个合适的编译器,你甚至可以用 Lisp 的语法来写,只要它能生成相同的指令。 假设 @@ -58,13 +58,13 @@ what_to_execute = { "numbers": [7, 5] } ``` -Python解释器是一个_栈机器_,所以它必须通过操作栈来完成这个加法。(\aosafigref{500l.interpreter.stackmachine}.)解释器先执行第一条指令,`LOAD_VALUE`,把第一个数压到栈中。接着它把第二个数也压到栈中。然后,第三条指令,`ADD_TWO_VALUES`,先把两个数从栈中弹出,加起来,再把结果压入栈中。最后一步,把结果弹出并输出。 +Python 解释器是一个栈机器(stack machine),所以它必须通过操作栈来完成这个加法(见下图)。解释器先执行第一条指令,`LOAD_VALUE`,把第一个数压到栈中。接着它把第二个数也压到栈中。然后,第三条指令,`ADD_TWO_VALUES`,先把两个数从栈中弹出,加起来,再把结果压入栈中。最后一步,把结果弹出并输出。 -\aosafigure[240pt]{interpreter-images/interpreter-stack.png}{A stack machine}{500l.interpreter.stackmachine} +![栈机器](http://aosabook.org/en/500L/interpreter-images/interpreter-stack.png) -`LOAD_VALUE`这条指令告诉解释器把一个数压入栈中,但指令本身并没有指明这个数是多少。指令需要一个额外的信息告诉解释器去哪里找到这个数。所以我们的指令集有两个部分:指令本身和一个常量列表。(在Python中,字节码就是我们称为的“指令”,而解释器执行的是_code object_。) +`LOAD_VALUE`这条指令告诉解释器把一个数压入栈中,但指令本身并没有指明这个数是多少。指令需要一个额外的信息告诉解释器去哪里找到这个数。所以我们的指令集有两个部分:指令本身和一个常量列表。(在 Python 中,字节码就是我们所称的“指令”,而解释器“执行”的是代码对象。) -为什么不把数字直接嵌入指令之中?想象一下,如果我们加的不是数字,而是字符串。我们可不想把字符串这样的东西加到指令中,因为它可以有任意的长度。另外,我们这种设计也意味着我们只需要对象的一份拷贝,比如这个加法 `7 + 7`, 现在常量表 `"numbers"`只需包含一个`7`。 +为什么不把数字直接嵌入指令之中?想象一下,如果我们加的不是数字,而是字符串。我们可不想把字符串这样的东西加到指令中,因为它可以有任意的长度。另外,我们这种设计也意味着我们只需要对象的一份拷贝,比如这个加法 `7 + 7`, 现在常量表 `"numbers"`只需包含一个`[7]`。 你可能会想为什么会需要除了`ADD_TWO_VALUES`之外的指令。的确,对于我们两个数加法,这个例子是有点人为制作的意思。然而,这个指令却是建造更复杂程序的轮子。比如,就我们目前定义的三个指令,只要给出正确的指令组合,我们可以做三个数的加法,或者任意个数的加法。同时,栈提供了一个清晰的方法去跟踪解释器的状态,这为我们增长的复杂性提供了支持。 @@ -89,7 +89,7 @@ class Interpreter: self.stack.append(total) ``` -这三个方法完成了解释器所理解的三条指令。但解释器还需要一样东西:一个能把所有东西结合在一起并执行的方法。这个方法就叫做`run_code`, 它把我们前面定义的字典结构`what-to-execute`作为参数,循环执行里面的每条指令,如何指令有参数,处理参数,然后调用解释器对象中相应的方法。 +这三个方法完成了解释器所理解的三条指令。但解释器还需要一样东西:一个能把所有东西结合在一起并执行的方法。这个方法就叫做 `run_code`,它把我们前面定义的字典结构 `what-to-execute` 作为参数,循环执行里面的每条指令,如果指令有参数就处理参数,然后调用解释器对象中相应的方法。 ```python def run_code(self, what_to_execute): @@ -106,20 +106,20 @@ class Interpreter: self.PRINT_ANSWER() ``` -为了测试,我们创建一个解释器对象,然后用前面定义的 7 + 5 的指令集来调用`run_code`。 +为了测试,我们创建一个解释器对象,然后用前面定义的 7 + 5 的指令集来调用 `run_code`。 ```python interpreter = Interpreter() interpreter.run_code(what_to_execute) ``` -显然,它会输出12 +显然,它会输出12。 -尽管我们的解释器功能受限,但这个加法过程几乎和真正的Python解释器是一样的。这里,我们还有几点要注意。 +尽管我们的解释器功能十分受限,但这个过程几乎和真正的 Python 解释器处理加法是一样的。这里,我们还有几点要注意。 -首先,一些指令需要参数。在真正的Python bytecode中,大概有一半的指令有参数。像我们的例子一样,参数和指令打包在一起。注意_指令_的参数和传递给对应方法的参数是不同的。 +首先,一些指令需要参数。在真正的 Python 字节码当中,大概有一半的指令有参数。像我们的例子一样,参数和指令打包在一起。注意指令的参数和传递给对应方法的参数是不同的。 -第二,指令`ADD_TWO_VALUES`不需要任何参数,它从解释器栈中弹出所需的值。这正是以栈为基础的解释器的特点。 +第二,指令`ADD_TWO_VALUES`不需要任何参数,它从解释器栈中弹出所需的值。这正是以基于栈的解释器的特点。 记得我们说过只要给出合适的指令集,不需要对解释器做任何改变,我们就能做多个数的加法。考虑下面的指令集,你觉得会发生什么?如果你有一个合适的编译器,什么代码才能编译出下面的指令集? @@ -134,11 +134,11 @@ class Interpreter: "numbers": [7, 5, 8] } ``` -从这点出发,我们开始看到这种结构的可扩展性:我们可以通过向解释器对象增加方法来描述更多的操作(只要有一个编译器能为我们生成组织良好的指令集)。 +从这点出发,我们开始看到这种结构的可扩展性:我们可以通过向解释器对象增加方法来描述更多的操作(只要有一个编译器能为我们生成组织良好的指令集就行)。 -#### Variables +##### 变量 -接下来给我们的解释器增加变量的支持。我们需要一个保存变量值的指令,`STORE_NAME`;一个取变量值的指令`LOAD_NAME`;和一个变量到值的映射关系。目前,我们会忽略命名空间和作用域,所以我们可以把变量和值的映射直接存储在解释器对象中。最后,我们要保证`what_to_execute`除了一个常量列表,还要有个变量名字的列表。 +接下来给我们的解释器增加变量的支持。我们需要一个保存变量值的指令 `STORE_NAME`;一个取变量值的指令`LOAD_NAME`;和一个变量到值的映射关系。目前,我们会忽略命名空间和作用域,所以我们可以把变量和值的映射直接存储在解释器对象中。最后,我们要保证`what_to_execute`除了一个常量列表,还要有个变量名字的列表。 ```python >>> def s(): @@ -159,9 +159,9 @@ class Interpreter: "names": ["a", "b"] } ``` -我们的新的的实现在下面。为了跟踪哪名字绑定到那个值,我们在`__init__`方法中增加一个`environment`字典。我们也增加了`STORE_NAME`和`LOAD_NAME`方法,它们获得变量名,然后从`environment`字典中设置或取出这个变量值。 +我们的新的实现在下面。为了跟踪哪个名字绑定到哪个值,我们在`__init__`方法中增加一个`environment`字典。我们也增加了`STORE_NAME`和`LOAD_NAME`方法,它们获得变量名,然后从`environment`字典中设置或取出这个变量值。 -现在指令参数就有两个不同的意思,它可能是`numbers`列表的索引,也可能是`names`列表的索引。解释器通过检查所执行的指令就能知道是那种参数。而我们打破这种逻辑 ,把指令和它所用何种参数的映射关系放在另一个单独的方法中。 +现在指令的参数就有两个不同的意思,它可能是`numbers`列表的索引,也可能是`names`列表的索引。解释器通过检查所执行的指令就能知道是那种参数。而我们打破这种逻辑 ,把指令和它所用何种参数的映射关系放在另一个单独的方法中。 ```python class Interpreter: @@ -207,7 +207,7 @@ class Interpreter: self.LOAD_NAME(argument) ``` -仅仅五个指令,`run_code`这个方法已经开始变得冗长了。如果保持这种结构,那么每条指令都需要一个`if`分支。这里,我们要利用Python的动态方法查找。我们总会给一个称为`FOO`的指令定义一个名为`FOO`的方法,这样我们就可用Python的`getattr`函数在运行时动态查找方法,而不用这个大大的分支结构。`run_code`方法现在是这样: +仅仅五个指令,`run_code`这个方法已经开始变得冗长了。如果保持这种结构,那么每条指令都需要一个`if`分支。这里,我们要利用 Python 的动态方法查找。我们总会给一个称为`FOO`的指令定义一个名为`FOO`的方法,这样我们就可用 Python 的`getattr`函数在运行时动态查找方法,而不用这个大大的分支结构。`run_code`方法现在是这样: ```python def execute(self, what_to_execute): @@ -222,9 +222,9 @@ class Interpreter: bytecode_method(argument) ``` -## Real Python Bytecode +### 真实的 Python 字节码 -现在,放弃我们的小指令集,去看看真正的Python字节码。字节码的结构和我们的小解释器的指令集差不多,除了字节码用一个字节而不是一个名字来指示这条指令。为了理解它的结构,我们将考察一个函数的字节码。考虑下面这个例子: +现在,放弃我们的小指令集,去看看真正的 Python 字节码。字节码的结构和我们的小解释器的指令集差不多,除了字节码用一个字节而不是一个名字来代表这条指令。为了理解它的结构,我们将考察一个函数的字节码。考虑下面这个例子: ```python >>> def cond(): @@ -236,7 +236,7 @@ class Interpreter: ... ``` -Python在运行时会暴露一大批内部信息,并且我们可以通过REPL直接访问这些信息。对于函数对象`cond`,`cond.__code__`是与其关联的code object,而`cond.__code__.co_code`就是它的字节码。当你写Python代码时,你永远也不会想直接使用这些属性,但是这可以让我们做出各种恶作剧,同时也可以看看内部机制。 +Python 在运行时会暴露一大批内部信息,并且我们可以通过 REPL 直接访问这些信息。对于函数对象`cond`,`cond.__code__`是与其关联的代码对象,而`cond.__code__.co_code`就是它的字节码。当你写 Python 代码时,你永远也不会想直接使用这些属性,但是这可以让我们做出各种恶作剧,同时也可以看看内部机制。 ```python >>> cond.__code__.co_code # the bytecode as raw bytes @@ -247,9 +247,9 @@ b'd\x01\x00}\x00\x00|\x00\x00d\x02\x00k\x00\x00r\x16\x00d\x03\x00Sd\x04\x00Sd\x0 100, 4, 0, 83, 100, 0, 0, 83] ``` -当我们直接输出这个字节码,它看起来完全无法理解 --- 唯一我们了解的是它是一串字节。很幸运,我们有一个很强大的工具可以用:Python标准库中的`dis`模块。 +当我们直接输出这个字节码,它看起来完全无法理解 —— 唯一我们了解的是它是一串字节。很幸运,我们有一个很强大的工具可以用:Python 标准库中的`dis`模块。 -`dis`是一个字节码反汇编器。反汇编器以为机器而写的底层代码作为输入,比如汇编代码和字节码,然后以人类可读的方式输出。当我们运行`dis.dis`, 它输出每个字节码的解释。 +`dis`是一个字节码反汇编器。反汇编器以为机器而写的底层代码作为输入,比如汇编代码和字节码,然后以人类可读的方式输出。当我们运行`dis.dis`,它输出每个字节码的解释。 ```python >>> dis.dis(cond) @@ -270,9 +270,9 @@ b'd\x01\x00}\x00\x00|\x00\x00d\x02\x00k\x00\x00r\x16\x00d\x03\x00Sd\x04\x00Sd\x0 29 RETURN_VALUE ``` -这些都是什么意思?让我们以第一条指令`LOAD_CONST`为例子。第一列的数字(`2`)表示对应源代码的行数。第二列的数字是字节码的索引,告诉我们指令`LOAD_CONST`在0位置。第三列是指令本身对应的人类可读的名字。如果第四列存在,它表示指令的参数。如果第5列存在,它是一个关于参数是什么的提示。 +这些都是什么意思?让我们以第一条指令`LOAD_CONST`为例子。第一列的数字(`2`)表示对应源代码的行数。第二列的数字是字节码的索引,告诉我们指令`LOAD_CONST`在位置 0 。第三列是指令本身对应的人类可读的名字。如果第四列存在,它表示指令的参数。如果第五列存在,它是一个关于参数是什么的提示。 -考虑这个字节码的前几个字节:[100, 1, 0, 125, 0, 0]。这6个字节表示两条带参数的指令。我们可以使用`dis.opname`,一个字节到可读字符串的映射,来找到指令100和指令125代表是什么: +考虑这个字节码的前几个字节:[100, 1, 0, 125, 0, 0]。这 6 个字节表示两条带参数的指令。我们可以使用`dis.opname`,一个字节到可读字符串的映射,来找到指令 100 和指令 125 代表的是什么: ```python >>> dis.opname[100] @@ -281,11 +281,11 @@ b'd\x01\x00}\x00\x00|\x00\x00d\x02\x00k\x00\x00r\x16\x00d\x03\x00Sd\x04\x00Sd\x0 'STORE_FAST' ``` -第二和第三个字节 --- 1 ,0 ---是`LOAD_CONST`的参数,第五和第六个字节 --- 0,0 --- 是`STORE_FAST`的参数。就像我们前面的小例子,`LOAD_CONST`需要知道的到哪去找常量,`STORE_FAST`需要找到名字。(Python的`LOAD_CONST`和我们小例子中的`LOAD_VALUE`一样,`LOAD_FAST`和`LOAD_NAME`一样)。所以这六个字节代表第一行源代码`x = 3`.(为什么用两个字节表示指令的参数?如果Python使用一个字节,每个code object你只能有256个常量/名字,而用两个字节,就增加到了256的平方,65536个)。 +第二和第三个字节 —— 1 、0 ——是`LOAD_CONST`的参数,第五和第六个字节 —— 0、0 —— 是`STORE_FAST`的参数。就像我们前面的小例子,`LOAD_CONST`需要知道的到哪去找常量,`STORE_FAST`需要知道要存储的名字。(Python 的`LOAD_CONST`和我们小例子中的`LOAD_VALUE`一样,`LOAD_FAST`和`LOAD_NAME`一样)。所以这六个字节代表第一行源代码`x = 3` (为什么用两个字节表示指令的参数?如果 Python 使用一个字节,每个代码对象你只能有 256 个常量/名字,而用两个字节,就增加到了 256 的平方,65536个)。 -### Conditionals and Loops +#### 条件语句与循环语句 -到目前为止,我们的解释器只能一条接着一条的执行指令。这有个问题,我们经常会想多次执行某个指令,或者在特定的条件下跳过它们。为了可以写循环和分支结构,解释器必须能够在指令中跳转。在某种程度上,Python在字节码中使用`GOTO`语句来处理循环和分支!让我们再看一个`cond`函数的反汇编结果: +到目前为止,我们的解释器只能一条接着一条的执行指令。这有个问题,我们经常会想多次执行某个指令,或者在特定的条件下跳过它们。为了可以写循环和分支结构,解释器必须能够在指令中跳转。在某种程度上,Python 在字节码中使用`GOTO`语句来处理循环和分支!让我们再看一个`cond`函数的反汇编结果: ```python >>> dis.dis(cond) @@ -306,11 +306,11 @@ b'd\x01\x00}\x00\x00|\x00\x00d\x02\x00k\x00\x00r\x16\x00d\x03\x00Sd\x04\x00Sd\x0 29 RETURN_VALUE ``` -第三行的条件表达式`if x < 5`被编译成四条指令:`LOAD_FAST`, `LOAD_CONST`, `COMPARE_OP`和 `POP_JUMP_IF_FALSE`。`x < 5`对应加载`x`,加载5,比较这两个值。指令`POP_JUMP_IF_FALSE`完成`if`语句。这条指令把栈顶的值弹出,如果值为真,什么都不发生。如果值为假,解释器会跳转到另一条指令。 +第三行的条件表达式`if x < 5`被编译成四条指令:`LOAD_FAST`、 `LOAD_CONST`、 `COMPARE_OP`和 `POP_JUMP_IF_FALSE`。`x < 5`对应加载`x`、加载 5、比较这两个值。指令`POP_JUMP_IF_FALSE`完成这个`if`语句。这条指令把栈顶的值弹出,如果值为真,什么都不发生。如果值为假,解释器会跳转到另一条指令。 -这条将被加载的指令称为跳转目标,它作为指令`POP_JUMP`的参数。这里,跳转目标是22,索引为22的指令是`LOAD_CONST`,对应源码的第6行。(`dis`用`>>`标记跳转目标。)如果`X < 5`为假,解释器会忽略第四行(`return yes`),直接跳转到第6行(`return "no"`)。因此解释器通过跳转指令选择性的执行指令。 +这条将被加载的指令称为跳转目标,它作为指令`POP_JUMP`的参数。这里,跳转目标是 22,索引为 22 的指令是`LOAD_CONST`,对应源码的第 6 行。(`dis`用`>>`标记跳转目标。)如果`X < 5`为假,解释器会忽略第四行(`return yes`),直接跳转到第6行(`return "no"`)。因此解释器通过跳转指令选择性的执行指令。 -Python的循环也依赖于跳转。在下面的字节码中,`while x < 5`这一行产生了和`if x < 10`几乎一样的字节码。在这两种情况下,解释器都是先执行比较,然后执行`POP_JUMP_IF_FALSE`来控制下一条执行哪个指令。第四行的最后一条字节码`JUMP_ABSOLUT`(循环体结束的地方),让解释器返回到循环开始的第9条指令处。当 `x < 10`变为假,`POP_JUMP_IF_FALSE`会让解释器跳到循环的终止处,第34条指令。 +Python 的循环也依赖于跳转。在下面的字节码中,`while x < 5`这一行产生了和`if x < 10`几乎一样的字节码。在这两种情况下,解释器都是先执行比较,然后执行`POP_JUMP_IF_FALSE`来控制下一条执行哪个指令。第四行的最后一条字节码`JUMP_ABSOLUT`(循环体结束的地方),让解释器返回到循环开始的第 9 条指令处。当 `x < 10`变为假,`POP_JUMP_IF_FALSE`会让解释器跳到循环的终止处,第 34 条指令。 ```python >>> def loop(): @@ -340,23 +340,23 @@ Python的循环也依赖于跳转。在下面的字节码中,`while x < 5`这 38 RETURN_VALUE ``` -### Explore Bytecode +#### 探索字节码 我希望你用`dis.dis`来试试你自己写的函数。一些有趣的问题值得探索: -- 对解释器而言for循环和while循环有什么不同? +- 对解释器而言 for 循环和 while 循环有什么不同? - 能不能写出两个不同函数,却能产生相同的字节码? - `elif`是怎么工作的?列表推导呢? -## Frames +### 帧 -到目前为止,我们已经知道了Python虚拟机是一个栈机器。它能顺序执行指令,在指令间跳转,压入或弹出栈值。但是这和我们期望的解释器还有一定距离。在前面的那个例子中,最后一条指令是`RETURN_VALUE`,它和`return`语句相对应。但是它返回到哪里去呢? +到目前为止,我们已经知道了 Python 虚拟机是一个栈机器。它能顺序执行指令,在指令间跳转,压入或弹出栈值。但是这和我们期望的解释器还有一定距离。在前面的那个例子中,最后一条指令是`RETURN_VALUE`,它和`return`语句相对应。但是它返回到哪里去呢? -为了回答这个问题,我们必须再增加一层复杂性:frame。一个frame是一些信息的集合和代码的执行上下文。frames在Python代码执行时动态的创建和销毁。每个frame对应函数的一次调用。--- 所以每个frame只有一个code object与之关联,而一个code object可以有多个frame。比如你有一个函数递归的调用自己10次,这会产生11个frame,每次调用对应一个,再加上启动模块对应的一个frame。总的来说,Python程序的每个作用域有一个frame,比如,模块,函数,类。 +为了回答这个问题,我们必须再增加一层复杂性:帧(frame)。一个帧是一些信息的集合和代码的执行上下文。帧在 Python 代码执行时动态地创建和销毁。每个帧对应函数的一次调用 —— 所以每个帧只有一个代码对象与之关联,而一个代码对象可以有多个帧。比如你有一个函数递归的调用自己 10 次,这会产生 11 个帧,每次调用对应一个,再加上启动模块对应的一个帧。总的来说,Python 程序的每个作用域都有一个帧,比如,模块、函数、类定义。 -Frame存在于_调用栈_中,一个和我们之前讨论的完全不同的栈。(你最熟悉的栈就是调用栈,就是你经常看到的异常回溯,每个以"File 'program.py'"开始的回溯对应一个frame。)解释器在执行字节码时操作的栈,我们叫它_数据栈_。其实还有第三个栈,叫做_块栈_,用于特定的控制流块,比如循环和异常处理。调用栈中的每个frame都有它自己的数据栈和块栈。 +帧存在于调用栈(call stack)中,一个和我们之前讨论的完全不同的栈。(你最熟悉的栈就是调用栈,就是你经常看到的异常回溯,每个以"File 'program.py'"开始的回溯对应一个帧。)解释器在执行字节码时操作的栈,我们叫它数据栈(data stack)。其实还有第三个栈,叫做块栈(block stack),用于特定的控制流块,比如循环和异常处理。调用栈中的每个帧都有它自己的数据栈和块栈。 -让我们用一个具体的例子来说明。假设Python解释器执行到标记为3的地方。解释器正在`foo`函数的调用中,它接着调用`bar`。下面是frame调用栈,块栈和数据栈的示意图。我们感兴趣的是解释器先从最底下的`foo()`开始,接着执行`foo`的函数体,然后到达`bar`。 +让我们用一个具体的例子来说明一下。假设 Python 解释器执行到下面标记为 3 的地方。解释器正处于`foo`函数的调用中,它接着调用`bar`。下面是帧调用栈、块栈和数据栈的示意图。我们感兴趣的是解释器先从最底下的`foo()`开始,接着执行`foo`的函数体,然后到达`bar`。 ```python >>> def bar(y): @@ -372,32 +372,30 @@ Frame存在于_调用栈_中,一个和我们之前讨论的完全不同的栈 3 ``` -\aosafigure[240pt]{interpreter-images/interpreter-callstack.png}{The call stack}{500l.interpreter.callstack} +![调用栈](http://aosabook.org/en/500L/interpreter-images/interpreter-callstack.png) -现在,解释器在`bar`函数的调用中。调用栈中有3个frame:一个对应于模块层,一个对应函数`foo`,别一个对应函数`bar`。(\aosafigref{500l.interpreter.callstack}.)一旦`bar`返回,与它对应的frame就会从调用栈中弹出并丢弃。 +现在,解释器处于`bar`函数的调用中。调用栈中有 3 个帧:一个对应于模块层,一个对应函数`foo`,另一个对应函数`bar`。(见上图)一旦`bar`返回,与它对应的帧就会从调用栈中弹出并丢弃。 -字节码指令`RETURN_VALUE`告诉解释器在frame间传递一个值。首先,它把位于调用栈栈顶的frame中的数据栈的栈顶值弹出。然后把整个frame弹出丢弃。最后把这个值压到下一个frame的数据栈中。 +字节码指令`RETURN_VALUE`告诉解释器在帧之间传递一个值。首先,它把位于调用栈栈顶的帧中的数据栈的栈顶值弹出。然后把整个帧弹出丢弃。最后把这个值压到下一个帧的数据栈中。 -当Ned Batchelder和我在写Byterun时,很长一段时间我们的实现中一直有个重大的错误。我们整个虚拟机中只有一个数据栈,而不是每个frame都有一个。我们写了很多测试代码,同时在Byterun和真正的Python上运行,希望得到一致结果。我们几乎通过了所有测试,只有一样东西不能通过,那就是生成器。最后,通过仔细的阅读CPython的源码,我们发现了错误所在[^thanks]。把数据栈移到每个frame就解决了这个问题。 +当 Ned Batchelder 和我在写 Byterun 时,很长一段时间我们的实现中一直有个重大的错误。我们整个虚拟机中只有一个数据栈,而不是每个帧都有一个。我们写了很多测试代码,同时在 Byterun 和真正的 Python 上运行,希望得到一致结果。我们几乎通过了所有测试,只有一样东西不能通过,那就是生成器(generators)。最后,通过仔细的阅读 CPython 的源码,我们发现了错误所在(感谢 Michael Arntzenius 对这个 bug 的洞悉)。把数据栈移到每个帧就解决了这个问题。 -[^thanks]: 感谢 Michael Arntzenius 对这个bug的洞悉。 +回头在看看这个 bug,我惊讶的发现 Python 真的很少依赖于每个帧有一个数据栈这个特性。在 Python 中几乎所有的操作都会清空数据栈,所以所有的帧公用一个数据栈是没问题的。在上面的例子中,当`bar`执行完后,它的数据栈为空。即使`foo`公用这一个栈,它的值也不会受影响。然而,对应生成器,它的一个关键的特点是它能暂停一个帧的执行,返回到其他的帧,一段时间后它能返回到原来的帧,并以它离开时的相同状态继续执行。 -回头在看看这个bug,我惊讶的发现Python真的很少依赖于每个frame有一个数据栈这个特性。在Python中几乎所有的操作都会清空数据栈,所以所有的frame公用一个数据栈是没问题的。在上面的例子中,当`bar`执行完后,它的数据栈为空。即使`foo`公用这一个栈,它的值也不会受影响。然而,对应生成器,一个关键的特点是它能暂停一个frame的执行,返回到其他的frame,一段时间后它能返回到原来的frame,并以它离开时的相同状态继续执行。 +### Byterun -## Byterun +现在我们有足够的 Python 解释器的知识背景去考察 Byterun。 -现在我们有足够的Python解释器的知识背景去考察Byterun。 +Byterun 中有四种对象。 -Byterun中有四种对象。 +- `VirtualMachine`类,它管理高层结构,尤其是帧调用栈,并包含了指令到操作的映射。这是一个比前面`Inteprter`对象更复杂的版本。 +- `Frame`类,每个`Frame`类都有一个代码对象,并且管理着其他一些必要的状态位,尤其是全局和局部命名空间、指向调用它的整的指针和最后执行的字节码指令。 +- `Function`类,它被用来代替真正的 Python 函数。回想一下,调用函数时会创建一个新的帧。我们自己实现了`Function`,以便我们控制新的`Frame`的创建。 +- `Block`类,它只是包装了块的 3 个属性。(块的细节不是解释器的核心,我们不会花时间在它身上,把它列在这里,是因为 Byterun 需要它。) -- `VirtualMachine`类,它管理高层结构,frame调用栈,指令到操作的映射。这是一个比前面`Inteprter`对象更复杂的版本。 -- `Frame`类,每个`Frame`类都有一个code object,并且管理者其他一些必要的状态信息,全局和局部命名空间,指向调用它的frame的指针和最后执行的字节码指令。 -- `Function`类,它被用来代替真正的Python函数。回想一下,调用函数时会创建一个新的frame。我们自己实现`Function`,所以我们控制新frame的创建。 -- `Block`类,它只是包装了block的3个属性。(block的细节不是解释器的核心,我们不会花时间在它身上,把它列在这里,是因为Byterun需要它。) +#### `VirtualMachine` 类 -### The `VirtualMachine` Class - -程序运行时只有一个`VirtualMachine`被创建,因为我们只有一个解释器。`VirtualMachine`保存调用栈,异常状态,在frame中传递的返回值。它的入口点是`run_code`方法,它以编译后的code object为参数,以创建一个frame为开始,然后运行这个frame。这个frame可能再创建出新的frame;调用栈随着程序的运行增长缩短。当第一个frame返回时,执行结束。 +每次程序运行时只会创建一个`VirtualMachine`实例,因为我们只有一个 Python 解释器。`VirtualMachine` 保存调用栈、异常状态、在帧之间传递的返回值。它的入口点是`run_code`方法,它以编译后的代码对象为参数,以创建一个帧为开始,然后运行这个帧。这个帧可能再创建出新的帧;调用栈随着程序的运行而增长和缩短。当第一个帧返回时,执行结束。 ```python class VirtualMachineError(Exception): @@ -418,9 +416,9 @@ class VirtualMachine(object): ``` -### The `Frame` Class +#### `Frame` 类 -接下来,我们来写`Frame`对象。frame是一个属性的集合,它没有任何方法。前面提到过,这些属性包括由编译器生成的code object;局部,全局和内置命名空间;前一个frame的引用;一个数据栈;一个块栈;最后执行的指令指针。(对于内置命名空间我们需要多做一点工作,Python在不同模块中对这个命名空间有不同的处理;但这个细节对我们的虚拟机不重要。) +接下来,我们来写`Frame`对象。帧是一个属性的集合,它没有任何方法。前面提到过,这些属性包括由编译器生成的代码对象;局部、全局和内置命名空间;前一个帧的引用;一个数据栈;一个块栈;最后执行的指令指针。(对于内置命名空间我们需要多做一点工作,Python 在不同模块中对这个命名空间有不同的处理;但这个细节对我们的虚拟机不重要。) ```python class Frame(object): @@ -441,7 +439,7 @@ class Frame(object): self.block_stack = [] ``` -接着,我们在虚拟机中增加对frame的操作。这有3个帮助函数:一个创建新的frame的方法,和压栈和出栈的方法。第四个函数,`run_frame`,完成执行frame的主要工作,待会我们再讨论这个方法。 +接着,我们在虚拟机中增加对帧的操作。这有 3 个帮助函数:一个创建新的帧的方法(它负责为新的帧找到名字空间),和压栈和出栈的方法。第四个函数,`run_frame`,完成执行帧的主要工作,待会我们再讨论这个方法。 ```python class VirtualMachine(object): @@ -481,9 +479,9 @@ class VirtualMachine(object): # we'll come back to this shortly ``` -### The `Function` Class +#### `Function` 类 -`Function`的实现有点扭曲,但是大部分的细节对理解解释器不重要。重要的是当调用函数时 --- `__call__`方法被调用 --- 它创建一个新的`Frame`并运行它。 +`Function`的实现有点曲折,但是大部分的细节对理解解释器不重要。重要的是当调用函数时 —— 即调用 `__call__`方法 —— 它创建一个新的`Frame`并运行它。 ```python class Function(object): @@ -534,7 +532,7 @@ def make_cell(value): return fn.__closure__[0] ``` -接着,回到`VirtualMachine`对象,我们对数据栈的操作也增加一些帮助方法。字节码操作的栈总是在当前frame的数据栈。这些帮助函数让我们能实现`POP_TOP`,`LOAD_FAST`字节码,并且让其他操作栈的指令可读性更高。 +接着,回到`VirtualMachine`对象,我们对数据栈的操作也增加一些帮助方法。字节码操作的栈总是在当前帧的数据栈。这些帮助函数让我们的`POP_TOP`、`LOAD_FAST`以及其他操作栈的指令的实现可读性更高。 ```python class VirtualMachine(object): @@ -562,11 +560,11 @@ class VirtualMachine(object): return [] ``` -在我们运行frame之前,我们还需两个方法。 +在我们运行帧之前,我们还需两个方法。 -第一个方法,`parse_byte_and_args`,以一个字节码为输入,先检查它是否有参数,如果有,就解析它的参数。这个方法同时也更新frame的`last_instruction`属性,它指向最后执行的指令。一条没有参数的指令只有一个字节长度,而有参数的字节有3个字节长。参数的意义依赖于指令是什么。比如,前面说过,指令`POP_JUMP_IF_FALSE`,它的参数指的是跳转目标。`BUILD_LIST`, 它的参数是列表的个数。`LOAD_CONST`,它的参数是常量的索引。 +第一个方法,`parse_byte_and_args` 以一个字节码为输入,先检查它是否有参数,如果有,就解析它的参数。这个方法同时也更新帧的`last_instruction`属性,它指向最后执行的指令。一条没有参数的指令只有一个字节长度,而有参数的字节有3个字节长。参数的意义依赖于指令是什么。比如,前面说过,指令`POP_JUMP_IF_FALSE`,它的参数指的是跳转目标。`BUILD_LIST`,它的参数是列表的个数。`LOAD_CONST`,它的参数是常量的索引。 -一些指令用简单的数字作为参数。对于另一些,虚拟机需要一点努力去发现它含意。标准库中的`dis`模块中有一个备忘单,它解释什么参数有什么意思,这让我们的代码更加简洁。比如,列表`dis.hasname`告诉我们`LOAD_NAME`, `IMPORT_NAME`,`LOAD_GLOBAL`,以及另外的9个指令都有同样的意思:名字列表的索引。 +一些指令用简单的数字作为参数。对于另一些,虚拟机需要一点努力去发现它含意。标准库中的`dis`模块中有一个备忘单,它解释什么参数有什么意思,这让我们的代码更加简洁。比如,列表`dis.hasname`告诉我们`LOAD_NAME`、 `IMPORT_NAME`、`LOAD_GLOBAL`,以及另外的 9 个指令的参数都有同样的意义:对于这些指令,它们的参数代表了代码对象中的名字列表的索引。 ```python class VirtualMachine(object): @@ -600,8 +598,7 @@ class VirtualMachine(object): return byte_name, argument ``` -下一个方法是`dispatch`,它查看给定的指令并执行相应的操作。在CPython中,这个分派函数用一个巨大的switch语句实现,有超过1500行的代码。幸运的是,我们用的是Python,我们的代码会简洁的多。我们会为每一个字节码名字定义一个方法,然后用`getattr`来查找。就像我们前面的小解释器一样,如果一条指令叫做`FOO_BAR`,那么它对应的方法就是`byte_FOO_BAR`。现在,我们先把这些方法当做一个黑盒子。每个指令方法都会返回`None`或者一个字符串`why`,有些情况下虚拟机需要这个额外`why`信息。这些指令方法的返回值,仅作为解释器状态的内部指示,千万不要和执行frame的返回值相混淆。 - +下一个方法是`dispatch`,它查找给定的指令并执行相应的操作。在 CPython 中,这个分派函数用一个巨大的 switch 语句实现,有超过 1500 行的代码。幸运的是,我们用的是 Python,我们的代码会简洁的多。我们会为每一个字节码名字定义一个方法,然后用`getattr`来查找。就像我们前面的小解释器一样,如果一条指令叫做`FOO_BAR`,那么它对应的方法就是`byte_FOO_BAR`。现在,我们先把这些方法当做一个黑盒子。每个指令方法都会返回`None`或者一个字符串`why`,有些情况下虚拟机需要这个额外`why`信息。这些指令方法的返回值,仅作为解释器状态的内部指示,千万不要和执行帧的返回值相混淆。 ```python class VirtualMachine(object): @@ -662,13 +659,13 @@ class VirtualMachine(object): return self.return_value ``` -### The `Block` Class +#### `Block` 类 在我们完成每个字节码方法前,我们简单的讨论一下块。一个块被用于某种控制流,特别是异常处理和循环。它负责保证当操作完成后数据栈处于正确的状态。比如,在一个循环中,一个特殊的迭代器会存在栈中,当循环完成时它从栈中弹出。解释器需要检查循环仍在继续还是已经停止。 -为了跟踪这些额外的信息,解释器设置了一个标志来指示它的状态。我们用一个变量`why`实现这个标志,它可以是`None`或者是下面几个字符串这一,`"continue"`, `"break"`,`"excption"`,`return`。他们指示对块栈和数据栈进行什么操作。回到我们迭代器的例子,如果块栈的栈顶是一个`loop`块,`why`是`continue`,迭代器就因该保存在数据栈上,不是如果`why`是`break`,迭代器就会被弹出。 +为了跟踪这些额外的信息,解释器设置了一个标志来指示它的状态。我们用一个变量`why`实现这个标志,它可以是`None`或者是下面几个字符串之一:`"continue"`、`"break"`、`"excption"`、`return`。它们指示对块栈和数据栈进行什么操作。回到我们迭代器的例子,如果块栈的栈顶是一个`loop`块,`why`的代码是`continue`,迭代器就应该保存在数据栈上,而如果`why`是`break`,迭代器就会被弹出。 -块操作的细节比较精细,我们不会花时间在这上面,但是有兴趣的读者值得仔细的看看。 +块操作的细节比这个还要繁琐,我们不会花时间在这上面,但是有兴趣的读者值得仔细的看看。 ```python Block = collections.namedtuple("Block", "type, handler, stack_height") @@ -737,9 +734,9 @@ class VirtualMachine(object): return why ``` -## The Instructions +### 指令 -剩下了的就是完成那些指令方法了:`byte_LOAD_FAST`,`byte_BINARY_MODULO`等等。而这些指令的实现并不是很有趣,这里我们只展示了一小部分,完整的实现在这儿https://github.com/nedbat/byterun。(足够执行我们前面所述的所有代码了。) +剩下了的就是完成那些指令方法了:`byte_LOAD_FAST`、`byte_BINARY_MODULO`等等。而这些指令的实现并不是很有趣,这里我们只展示了一小部分,完整的实现[在 GitHub 上](https://github.com/nedbat/byterun)。(这里包括的指令足够执行我们前面所述的所有代码了。) ```python class VirtualMachine(object): @@ -926,11 +923,11 @@ class VirtualMachine(object): return "return" ``` -## Dynamic Typing: What the Compiler Doesn't Know +### 动态类型:编译器不知道它是什么 -你可能听过Python是一种动态语言 --- 是它是动态类型的。在我们建造解释器的过程中,已经透露出这样的信息。 +你可能听过 Python 是一种动态语言 —— 它是动态类型的。在我们建造解释器的过程中,已经透露出这样的信息。 -动态的一个意思是很多工作在运行时完成。前面我们看到Python的编译器没有很多关于代码真正做什么的信息。举个例子,考虑下面这个简单的函数`mod`。它取两个参数,返回它们的模运算值。从它的字节码中,我们看到变量`a`和`b`首先被加载,然后字节码`BINAY_MODULO`完成这个模运算。 +动态的一个意思是很多工作是在运行时完成的。前面我们看到 Python 的编译器没有很多关于代码真正做什么的信息。举个例子,考虑下面这个简单的函数`mod`。它取两个参数,返回它们的模运算值。从它的字节码中,我们看到变量`a`和`b`首先被加载,然后字节码`BINAY_MODULO`完成这个模运算。 ```python >>> def mod(a, b): @@ -944,25 +941,25 @@ class VirtualMachine(object): 4 ``` -计算19 % 5得4,--- 一点也不奇怪。如果我们用不同类的参数呢? +计算 19 % 5 得4,—— 一点也不奇怪。如果我们用不同类的参数呢? ```python >>> mod("by%sde", "teco") 'bytecode' ``` -刚才发生了什么?你可能见过这样的语法,格式化字符串。 +刚才发生了什么?你可能在其它地方见过这样的语法,格式化字符串。 ``` >>> print("by%sde" % "teco") bytecode ``` -用符号`%`去格式化字符串会调用字节码`BUNARY_MODULO`.它取栈顶的两个值求模,不管这两个值是字符串,数字或是你自己定义的类的实例。字节码在函数编译时生成(或者说,函数定义时)相同的字节码会用于不同类的参数。 +用符号`%`去格式化字符串会调用字节码`BUNARY_MODULO`。它取栈顶的两个值求模,不管这两个值是字符串、数字或是你自己定义的类的实例。字节码在函数编译时生成(或者说,函数定义时)相同的字节码会用于不同类的参数。 -Python的编译器关于字节码的功能知道的很少。而取决于解释器来决定`BINAYR_MODULO`应用于什么类型的对象并完成正确的操作。这就是为什么Python被描述为_动态类型_:直到运行前你不必知道这个函数参数的类型。相反,在一个静态类型语言中,程序员需要告诉编译器参数的类型是什么(或者编译器自己推断出参数的类型。) +Python 的编译器关于字节码的功能知道的很少,而取决于解释器来决定`BINAYR_MODULO`应用于什么类型的对象并完成正确的操作。这就是为什么 Python 被描述为动态类型(dynamically typed):直到运行前你不必知道这个函数参数的类型。相反,在一个静态类型语言中,程序员需要告诉编译器参数的类型是什么(或者编译器自己推断出参数的类型。) -编译器的无知是优化Python的一个挑战 --- 只看字节码,而不真正运行它,你就不知道每条字节码在干什么!你可以定义一个类,实现`__mod__`方法,当你对这个类的实例使用`%`时,Python就会自动调用这个方法。所以,`BINARY_MODULO`其实可以运行任何代码。 +编译器的无知是优化 Python 的一个挑战 —— 只看字节码,而不真正运行它,你就不知道每条字节码在干什么!你可以定义一个类,实现`__mod__`方法,当你对这个类的实例使用`%`时,Python 就会自动调用这个方法。所以,`BINARY_MODULO`其实可以运行任何代码。 看看下面的代码,第一个`a % b`看起来没有用。 @@ -972,25 +969,23 @@ def mod(a,b): return a %b ``` -不幸的是,对这段代码进行静态分析 --- 不运行它 --- 不能确定第一个`a % b`没有做任何事。用 `%`调用`__mod__`可能会写一个文件,或是和程序的其他部分交互,或者其他任何可以在Python中完成的事。很难优化一个你不知道它会做什么的函数。在Russell Power和Alex Rubinsteyn的优秀论文中写道,“我们可以用多快的速度解释Python?”,他们说,“在普遍缺乏类型信息下,每条指令必须被看作一个`INVOKE_ARBITRARY_METHOD`。” +不幸的是,对这段代码进行静态分析 —— 不运行它 —— 不能确定第一个`a % b`没有做任何事。用 `%`调用`__mod__`可能会写一个文件,或是和程序的其他部分交互,或者其他任何可以在 Python 中完成的事。很难优化一个你不知道它会做什么的函数。在 Russell Power 和 Alex Rubinsteyn 的优秀论文中写道,“我们可以用多快的速度解释 Python?”,他们说,“在普遍缺乏类型信息下,每条指令必须被看作一个`INVOKE_ARBITRARY_METHOD`。” -## Conclusion +### 总结 -Byterun是一个比CPython容易理解的简洁的Python解释器。Byterun复制了CPython的主要结构:一个基于栈的指令集称为字节码,它们顺序执行或在指令间跳转,向栈中压入和从中弹出数据。解释器随着函数和生成器的调用和返回,动态的创建,销毁frame,并在frame间跳转。Byterun也有着和真正解释器一样的限制:因为Python使用动态类型,解释器必须在运行时决定指令的正确行为。 +Byterun 是一个比 CPython 容易理解的简洁的 Python 解释器。Byterun 复制了 CPython 的主要结构:一个基于栈的解释器对称之为字节码的指令集进行操作,它们顺序执行或在指令间跳转,向栈中压入和从中弹出数据。解释器随着函数和生成器的调用和返回,动态的创建、销毁帧,并在帧之间跳转。Byterun 也有着和真正解释器一样的限制:因为 Python 使用动态类型,解释器必须在运行时决定指令的正确行为。 -我鼓励你去反汇编你的程序,然后用Byterun来运行。你很快会发现这个缩短版的Byterun所没有实现的指令。完整的实现在https://github.com/nedbat/byterun,或者仔细阅读真正的CPython解释器`ceval.c`,你也可以实现自己的解释器! +我鼓励你去反汇编你的程序,然后用 Byterun 来运行。你很快会发现这个缩短版的 Byterun 所没有实现的指令。完整的实现在 https://github.com/nedbat/byterun ,或者你可以仔细阅读真正的 CPython 解释器`ceval.c`,你也可以实现自己的解释器! -## Acknowledgements +### 致谢 -Thanks to Ned Batchelder for originating this project and guiding my contributions, Michael Arntzenius for his help debugging the code and editing the prose, Leta Montopoli for her edits, and the entire Recurse Center community for their support and interest. Any errors are my own. +感谢 Ned Batchelder 发起这个项目并引导我的贡献,感谢 Michael Arntzenius 帮助调试代码和这篇文章的修订,感谢 Leta Montopoli 的修订,以及感谢整个 Recurse Center 社区的支持和鼓励。所有的不足全是我自己没搞好。 -------------------------------------- -via:http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html - -作者: Allison Kaptur Allison +via: http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html +作者: Allison Kaptur 译者:[qingyunha](https://github.com/qingyunha) - -校对:[校对者ID](https://github.com/校对者ID) +校对:[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](http://linux.cn/) 荣誉推出 From 795035082e71ef6a4c97f603fe3d210a1d8e277b Mon Sep 17 00:00:00 2001 From: "eric.xiaozhen" Date: Thu, 8 Sep 2016 15:15:19 +0800 Subject: [PATCH 0079/2437] Translate finished --- ...tate Of JavaScript - JavaScript Flavors.md | 104 ------------------ ...tate Of JavaScript - JavaScript Flavors.md | 99 +++++++++++++++++ 2 files changed, 99 insertions(+), 104 deletions(-) delete mode 100644 sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md create mode 100644 translated/tech/20160830 The State Of JavaScript - JavaScript Flavors.md diff --git a/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md b/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md deleted file mode 100644 index df649cd722..0000000000 --- a/sources/tech/20160830 The State Of JavaScript - JavaScript Flavors.md +++ /dev/null @@ -1,104 +0,0 @@ -Eriwoon 认领翻译 -The State Of JavaScript: JavaScript Flavors -=========== - - -One thing that distinguishes JavaScript from other programming languages is that JavaScript isn’t just one language: it’s actually more like a family of closely related cousins. - -What started with CoffeeScript back in 2009 has become an explosion of choice over the past couple years: ES6, TypeScript, Elm… they all have their strengths, and they all compile down to good old JavaScript. - -So after last week’s look at front-end frameworks, let’s look at what the State Of JavaScript survey can tell us about JavaScript Flavors. - -Disclaimer: these are preliminary results extracted from a partial dataset. They’re just a way for me to share some insights while I take my time to come up with the best way to present the complete results. - -Note: if you haven’t taken the survey yet, now would be the perfect time to do it! It’ll only take 10 minutes and you can come back here after :) - -### Awareness - -First, I wanted to find out the percentage of respondents that were aware of each of the six options’ existence: - -- Good Old Plain JavaScript: 97% -- ES6: 98% -- CoffeeScript: 99% -- TypeScript: 98% -- Elm: 66% -- ClojureScript: 77% - -You would expect that “Good Old Plain JavaScript” would score 100% awareness, but I imagine some people couldn’t resist the temptation of picking “I’ve never heard of JavaScript” in a JavaScript survey… - -ES6, CoffeeScript, and TypeScript all have near-perfect awareness, which surprised me since TypeScript isn’t quite as widespread as the other two. - -Elm and ClojureScript on the other hand have much lower scores, which makes sense since they’re more tied to their own ecosystems, and harder to use in existing apps. - -### Interest - -Next, let’s look at which flavors have been generating the most interest among developers who haven’t used them yet: - -![](https://embed.chartblocks.com/1.0/?c=57c4dc599973d2525fee820a&referrer=https%3A%2F%2Fmedium.com%2Fmedia%2F63092db0523a37d9d33ce6c014d727f6%3FmaxWidth%3D700&t=3efc9491eba2ce2#) - -Remember we’re looking at non-users here, so it makes sense that there would be very few people who haven’t used Plain JavaScript. - -It’s interesting to look at ES6: a large proportion of developers have already jumped on the bandwagon, and almost all (89%) of those who haven’t yet want to learn it as well. - -TypeScript and Elm are in the same boat: not many people have used them, but they have 53% and 58% interest scores respectively, which isn’t bad by any means. - -If I had to guess, I’d say that both TypeScript and Elm might be having a hard time articulating their advantages to the average JavaScript developer. After all it’s hard to understand the advantages of something like static typing if all you know is JavaScript. - -Also, few developers have used CoffeeScript, and apparently almost nobody wants to learn it. There goes my plan to write a 12-volume CoffeeScript Encyclopedia… - -### Satisfaction - -We now come to the key question: how many developers have used each specific flavor, and how many would use it again? - -![](https://embed.chartblocks.com/1.0/?c=57c4e5f79973d29461ee820a&referrer=https%3A%2F%2Fmedium.com%2Fmedia%2F1fe4bbdf807f87883fa108e31c6927d5%3FmaxWidth%3D700&t=1061d2ab8fc9838#) - - -While plain JavaScript has larger usage numbers as expected, in terms of satisfaction the big winner here is ES6, and I think it’s safe to say it’s now the default way to write JavaScript apps. -TypeScript and Elm both also have similarly high satisfaction percentages, around 85%. And once more, poor CoffeeScript trails the poll with only 17% of developers willing to consider it again. - -### Happiness - -Finally, I asked people how happy they were with their current way of writing JavaScript: - -![](https://embed.chartblocks.com/1.0/?c=57c4cd8c9973d2d95bee820a&referrer=https%3A%2F%2Fmedium.com%2Fmedia%2F538138b2d91d1fa99f1696bef4dd4d3f%3FmaxWidth%3D700&t=f53efb029ea4456#) - - -The high scores we saw in the previous question are confirmed here: with an average score of 3.96 overall, people are really happy with JavaScript as a language. - -It’s hard to say if this is because of JavaScript’s recent improvements, or because maybe (just maybe) JavaScript isn’t as horrible a language as people make it to be. But it’s certainly comforting. - -### Conclusions - -If React and Vue were the clear winners last time, I would say that here it’s without a doubt ES6. This is not groundbreaking news by any means, but it’s nice to know the community is embracing the direction the language is taking. - -It will be really interesting to ask these questions again a year or so from now, and see if TypeScript, Elm, and ClojureScript have made any progress. - -Personally, I suspect this explosion of flavors is just the beginning, and that the way we write JavaScript a couple years from now might be quite different! - -### Share the Word & Stay Tuned - -When it comes to surveys like this one, more data equals better data! The more people take the survey, the more representative of the overall JavaScript community it will be. - -So if you can, I encourage you to share the survey: - -[On Twitter][1] - -[On Facebook][2] - -And if you’d like to know next time I publish results, [head to the survey homepage][3] and leave your email there to be notified! - - --------------------------------------------------------------------------------- - -via: https://www.oreilly.com/ideas/spark-comparison-aws-vs-gcp?utm_source=dbweekly&utm_medium=email - -作者:[Sacha Greif][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://medium.com/@sachagreif -[1]: https://twitter.com/intent/tweet/?text=The%20State%20Of%20JavaScript%3A%20take%20a%20short%20survey%20about%20popular%20JavaScript%20technologies%20http%3A%2F%2Fstateofjs.com%20%23stateofjs -[2]: https://facebook.com/sharer/sharer.php?u=http%3A%2F%2Fstateofjs.com -[3]: http://stateofjs.com/ diff --git a/translated/tech/20160830 The State Of JavaScript - JavaScript Flavors.md b/translated/tech/20160830 The State Of JavaScript - JavaScript Flavors.md new file mode 100644 index 0000000000..ce43bf2300 --- /dev/null +++ b/translated/tech/20160830 The State Of JavaScript - JavaScript Flavors.md @@ -0,0 +1,99 @@ +JavaScript 现状:方言篇 +=========== + +JavaScript 和其他编程语言有一个很大的不同,它不像单纯的一个语言,而像一个由众多方言组成大家族。 + +从2009年 CoffeeScript 的出现开始,近几年出现了大量基于 JavaScript 语言,或者叫方言,例如 ES6,TypeScript,Elm 等等。它们都有自己的优势,且都可以被完美编译成标准 JavaScript。 + +所以,继上周的前端框架篇,今天带来 JavaScript 现状之方言篇,看一下大家对于 JavaScript 的方言是怎么选择的。 + +> 声明:下面的结论是根据部分数据得出的初步结果,希望能将我的一些想法带给大家,对于完整数据后面我会用更好方式呈现出来。 + +> 注意:如果你还没有参与这个调查,现在就来参加吧,可以花十分钟完成调查然后再回来看这篇文章。 + +### 认知度 + +首先,我想看一下参与问卷调查的人是否**知道**下面六种语言: + +- 经典的 JavaScript: 97% +- ES6: 98% +- CoffeeScript: 99% +- TypeScript: 98% +- Elm: 66% +- ClojureScript: 77% + +你可能觉得100%的人都应该知道『经典的 JavaScript 』,我想是有人无法抵抗『我从来没有听说过 JavaScript 』这个选项的强大诱惑吧…… + +几乎所有人都知道 ES6、CoffeeScript 和 TypeScript 这三种语言,比较令我惊讶的是 TypeScript 竟然会稍微落后于 ES6 和 CoffeeScript。 + +另一方面,Elm 和 ClojureScript 得分就要低得多,当然这也有道理,因为它们跟自己的生态环境绑定的比较紧密,也很难在已有的 App 中进行使用。 + +### 兴趣度 + +接下来,让我们一起看一下,哪一种方言吸引新开发者的能力更强一些: + +![](https://d3ugvbs94d921r.cloudfront.net/57c4dc599973d2525fee820a.png?t=3efc9491eba2ce2) + +要注意,表是统计该语言对从未使用过它们的用户的吸引度,因为只有很少人没有用过经典 JavaScript,所以『经典 JavaScript 』这一列的数值很低。 + +ES6的数值很有趣:已有很大比例的用户在使用 ES6 了,没有用过的人中的绝大部分(89%)也很想学习它。 + +TypeScript 和 Elm 的状态差不多:用过的人不多,但感兴趣的比例表现不错,分别是 53% 和 58%。 + +如果让我预测一下,那我觉得 TypeScript 和 Elm 都很难向普通的 JavaScript 开发者讲明自己的优势。毕竟如果开发者只懂 JavaScript 的话,你很难解释清楚静态类型的好处。 + +另外,只有很少开发者用过 CoffeeScript,而且很明显几乎没人想去学。我觉得我该写一本 12 卷厚的 CoffeeScript 百科全书了…… + +### 满意度 + +现在是最重要的问题的时间了:有多少开发者用过这些语言,有多少人还想继续使用这些方言呢? + +![](https://d3ugvbs94d921r.cloudfront.net/57c4e5f79973d29461ee820a.png?t=1061d2ab8fc9838) + +虽然经典 JavaScript 拥有最多的用户量,但就满意度来说 ES6 才是大赢家,而且我想现在已经能安全的说,ES6 可以作为开发 JavaScript App 默认的语言。 +TypeScript 和 Elm 有相似的高满意度,都在 85% 上下。然后,只有可怜的 17% 的开发者会考虑继续使用 CoffeeScript。 + +### 快乐度 + +最后一个问题,我问大家在用现在的方式写 JavaScript 时是否感到快乐: + +![](https://d3ugvbs94d921r.cloudfront.net/57c4cd8c9973d2d95bee820a.png?t=f53efb029ea4456) + +这个问题的答案和上一个问题的满意度想匹配:平均分达到 3.96 分(1 - 5 分),大家在使用 JavaScript 时候确实是快乐的。 + +不过很难说高分是因为 JavaScript 最近的一些改进造成的呢,还是发现 JavaScript 可能(仅仅是可能)没有大家认为的那么讨厌。总之,JavaScript 令人满意。 + +### 总结 + +如果说上次的赢家是 React 和 Vue,那此次调查的冠军毫无争议是 ES6 了。 ES6 并带来没有开天辟地的变化,但整个社区都还是很认可当前 JavaScript 演进方向的。 + +我觉得一年之后我们再来一次这样的调查,结果会很有趣。同时也可以关注一下 TypeScript、Elm 还有ClojureScript 有没有继续进步。 + +个人认为,当前 JavaScript 大家庭百花齐放的现象还只是一个开始,或许几年之后 JavaScript 就会变得非常不同了。 + +### 结语&敬请期待 + +对于我这样的调查来说数据越多就意味着数据越准确!越多人参加这个调查,那就越能代表整个 JavaScript 社区。 + +所以,我十分希望你能帮忙分享这个调查问卷: + +[On Twitter][1] + +[On Facebook][2] + +另外,如果你想收到我下一个调查结果分析,前往 [调查问卷主页][3] 并留下自己的邮箱吧。 + +-------------------------------------------------------------------------------- + +via: https://www.oreilly.com/ideas/spark-comparison-aws-vs-gcp?utm_source=dbweekly&utm_medium=email + +作者:[Sacha Greif][a] +译者:[eriwoon](https://github.com/eriwoon) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://medium.com/@sachagreif +[1]: https://twitter.com/intent/tweet/?text=The%20State%20Of%20JavaScript%3A%20take%20a%20short%20survey%20about%20popular%20JavaScript%20technologies%20http%3A%2F%2Fstateofjs.com%20%23stateofjs +[2]: https://facebook.com/sharer/sharer.php?u=http%3A%2F%2Fstateofjs.com +[3]: http://stateofjs.com/ From 2a890fea5524d18a12fc885f2bff99e5638a1773 Mon Sep 17 00:00:00 2001 From: "eric.xiaozhen" Date: Thu, 8 Sep 2016 18:54:45 +0800 Subject: [PATCH 0080/2437] Under translating by Eriwoon --- ...frastructure behind Twitter - efficiency and optimization.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md b/sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md index 53ad32416f..d451393499 100644 --- a/sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md +++ b/sources/tech/20160823 The infrastructure behind Twitter - efficiency and optimization.md @@ -1,3 +1,4 @@ +Eriwoon Start to translate this article The infrastructure behind Twitter: efficiency and optimization =========== From f92f6e068425f04eb60d5c9ee1cc91ae07eb9843 Mon Sep 17 00:00:00 2001 From: wxy Date: Fri, 9 Sep 2016 14:07:32 +0800 Subject: [PATCH 0081/2437] PUB:20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu @vim-kakali --- ...Audio files with Octave 4.0.0 on Ubuntu.md | 31 ++++----- ...e been Linuxing since before you were born | 68 ------------------- .../talk/20160803 The revenge of Linux.md | 2 - 3 files changed, 13 insertions(+), 88 deletions(-) rename {translated/tech => published}/20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu.md (69%) delete mode 100644 translated/talk/20160728 I've been Linuxing since before you were born diff --git a/translated/tech/20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu.md b/published/20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu.md similarity index 69% rename from translated/tech/20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu.md rename to published/20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu.md index 4bfc4b7c21..56bddceaa1 100644 --- a/translated/tech/20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu.md +++ b/published/20160616 Part 1 - Scientific Audio Processing How to read and write Audio files with Octave 4.0.0 on Ubuntu.md @@ -1,14 +1,13 @@ - -科学音频处理--怎样在 Ubuntu 上使用 Octave 对音频文件进行读写操作 +科学音频处理(一):怎样使用 Octave 对音频文件进行读写操作 ================ -Octave 是一个类似于 Linux 上的 Matlab 的软件,它有数量众多的函数和命令进行声音采集,记录,回放以及娱乐应用程序,研究,医学方面的音频信号的处理,这也包括任何其他的科学领域。在本教程中,我们会在 Ubuntu 上使用 Octave 的 4.0.0 版本读取音频文件,然后通过生成信号并且播放来模仿在各种情况下对音频信号的使用。 +Octave 是一个类似于 Linux 上的 Matlab 的软件,它拥有数量众多的函数和命令,支持声音采集、记录、回放以及音频信号的数字化处理,用于娱乐应用、研究、医学以及其它科学领域。在本教程中,我们会在 Ubuntu 上使用 Octave 的 4.0.0 版本读取音频文件,然后通过生成信号并且播放来模仿在各种情况下对音频信号的使用。 -本教程中关注的不是安装和学习使用已有的音频处理软件,而是从设计和音频工程的角度理解它是如何工作的。 +本教程中关注的不是安装和学习使用安装好的音频处理软件,而是从设计和音频工程的角度理解它是如何工作的。 ### 环境准备 -首先是安装 octave,在 Ubuntu 终端运行下面的命令添加 Octave PPA 然后安装 Octave 。 +首先是安装 octave,在 Ubuntu 终端运行下面的命令添加 Octave PPA,然后安装 Octave 。 ``` sudo apt-add-repository ppa:octave/stable @@ -16,7 +15,6 @@ sudo apt-get update sudo apt-get install octave ``` - ### 步骤1:打开 Octave 在这一步中我们单击软件图标打开 Octave,可以通过单击下拉式按钮选择工作路径。 @@ -25,8 +23,8 @@ sudo apt-get install octave ### 步骤2:音频信息 +使用`audioinfo`命令查看要处理的音频文件的相关信息。 -使用“audioinfo”命令查看要处理的音频文件的相关信息。 ``` >> info = audioinfo ('testing.ogg') ``` @@ -36,8 +34,8 @@ sudo apt-get install octave ### 步骤3:读取音频文件 +在本教程中我会使用 ogg 文件来读取这种文件的属性,比如采样、音频类型(stereo 和 mono)、信道数量等。必须声明的一点是教程中使用的所有的命令都是在 Octave 终端窗口中执行的。首先,我们必须要把这个 ogg 文件赋给一个变量。注意:**文件必须在 Octave 的工作路径中。** -在本教程中我会使用 ogg 文件,因为读取这种文件的属性比较容易,比如采样,音频类型(stereo 和 mono),信道数量等。必须声明的一点是教程中使用的所有的命令都是在 Octave 终端窗口中执行的。首先,我们必须要把这个 ogg 文件赋给一个变量。注意:文件必须在 Octave 的工作路径中。 ``` >> file='yourfile.ogg' ``` @@ -46,8 +44,8 @@ sudo apt-get install octave >> [M, fs] = audioread(file) ``` - 这里的 M 是一个一列或两列的矩阵,取决于信道的数量,fs 是采样率。 + ![](https://www.howtoforge.com/images/how-to-read-and-write-audio-files-with-octave-4-in-ubuntu/reading.png) ![](https://www.howtoforge.com/images/how-to-read-and-write-audio-files-with-octave-4-in-ubuntu/matrix.png) @@ -73,21 +71,19 @@ samples 指定开始帧和结束帧,datatype 指定返回的数据类型。可 >> [y, fs] = audioread (filename, samples) ``` - 数据类型: ``` >> [y,Fs] = audioread(filename,'native') ``` -如果值是"native",那么它的数据类型就依数据在音频文件中的存储情况而定。 +如果值是“native”,那么它的数据类型就依数据在音频文件中的存储情况而定。 ### 步骤4:音频文件的写操作 新建一个 ogg 文件: - -我们会创建一个带有余弦值的 ogg 文件。采样率是每秒 44100 次,这个文件最少进行 10 秒的采样。余弦信号的频率是 440 Hz。 +我们会从一个余弦值创建一个 ogg 文件。采样率是每秒 44100 次,这个文件最少进行 10 秒的采样。余弦信号的频率是 440 Hz。 ``` >> filename='cosine.ogg'; @@ -99,18 +95,17 @@ samples 指定开始帧和结束帧,datatype 指定返回的数据类型。可 ``` 这就在工作路径中创建了一个 'cosine.ogg' 文件,这个文件中包含余弦信号。 + ![](https://www.howtoforge.com/images/how-to-read-and-write-audio-files-with-octave-4-in-ubuntu/cosinefile.png) - 播放这个 'cosine.ogg' 文件就会产生一个 440Hz 的 音调,这个音调正好是乐理中的 'A' 调。如果需要查看保存在文件中的值就必须使用 'audioread' 函数读取文件。在后续的教程中,我们会看到怎样在两个信道中读取一个音频文件。 - ### 步骤5:播放音频文件 Octave 有一个默认的音频播放器,可以用这个音频播放器进行测试。使用下面的函数: ``` - >> [y,fs]=audioread('yourfile.ogg'); +>> [y,fs]=audioread('yourfile.ogg'); >> player=audioplayer(y, fs, 8) scalar structure containing the fields: @@ -129,7 +124,7 @@ Octave 有一个默认的音频播放器,可以用这个音频播放器进行 ``` -在教程的后续,我们会进入音频处理的高级特性部分,可能会接触到一些科学和商业应用中的实例。 +在这个教程的续篇,我们会进入音频处理的高级特性部分,可能会接触到一些科学和商业应用中的实例。 -------------------------------------------------------------------------------- @@ -137,7 +132,7 @@ via: https://www.howtoforge.com/tutorial/how-to-read-and-write-audio-files-with- 作者:[David Duarte][a] 译者:[vim-kakali](https://github.com/vim-kakali) -校对:[校对ID](https://github.com/校对ID) +校对:[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/translated/talk/20160728 I've been Linuxing since before you were born b/translated/talk/20160728 I've been Linuxing since before you were born deleted file mode 100644 index 2ff2df8c83..0000000000 --- a/translated/talk/20160728 I've been Linuxing since before you were born +++ /dev/null @@ -1,68 +0,0 @@ - -我玩 Linux 那会儿你还不知道在哪呢 -===================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_Penguin_Image_520x292_12324207_0714_mm_v1a.png?itok=WfAkwbFy) - -在很久以前,世界上没有 Linux 系统。真的没有!之前也从未存在过。不像现在, Linux 系统随处可见。有各种风格的 Unix 系统,有苹果的操作系统,有微软的 Windows 操作系统。 - -随时 Windows 桌面的到来,改的东西很多,但是保留下来的东西的更多。尽管增加至20GB以上的存储,Windows 还是大体保持不变。(除了不能在DOS的提示下完成实际的工作)嘿,谁还记得 Gorilla.bas 那个出现在 DOS 系统里的炸香蕉的游戏吗?多么美好的时光啊!互联网永不会忘记,而且你可以在 Kongregate.com 这个网站玩Flash版本的这个游戏。 - -苹果系统也改变了,变得更加友好、安全、美观。你别想着打开这个密封的盒子,它强制规定了用户能使用的硬件接口。 1998年:没有软盘。 2012年:没有光驱。 12英寸的 MacBook 只有一个提供电源的USB Type-C 端口、蓝牙、无线网卡、外部存储、视频输出接口和其它的一些配件。如果你想每次想接入一个新的设备而不想使用其它的配件和软件,这真是太抓狂了。其它:头戴式耳机。这是个非独有的硬件接口在 Apple 的领域里注定保留下来。 - -还有其它的操作系统,比如: Amiga, BeOS, OS/2 以及无数的你能查到的操作系统。我支持的这些系统是因为现在都很容易查寻到,也没有理由不推荐它们。 Amiga、 BeOS 和 OS/2 这些系统都值得关注,因为它们有强大的功能,比如32位的多任务和高级图形处理能力。但是市场的影响击败了强大的系统性能,因此普通的 Apple 和 Windows 操作系统统治了市场的主导地位,而那些曾经的系统也逐渐销声匿迹了。 - -然后 Linux 系统出现了,世界也因此改变了。 - -### 第一款电脑 - -我曾经使用过的第一款电脑是 Apple IIc ,大概在 1994年左右,那个时候 Linux 系统刚出来 3年。这是我从一个朋友那里借来的,用起来各方面都还不错,但是很不方便。所以我自己花了将近 500美元买了一台二手的 Tandy牌子的电脑。这对于卖电脑的人来说是一件很伤心的事,因为新电脑要花费双倍的价钱。那个时候,电脑贬值的速度非常快。这个电脑看起来像是个怪物:一个英特尔 386SX 的 CPU, 4MB的内存,一个 107MB的硬盘, 14英寸的彩色 CRT 显示器,运行 MS-DOS 5和 Windows 3.1系统。 - -我曾无数次的拆开那个可怜的怪物,并且多次重新安装 Windows 和 DOS 系统。因为 Windows 桌面用的比较少,所以我的大部分工作都是在 DOS 下完成的。我喜欢玩血腥暴力的视频游戏,包括 Doom, Duke Nukem,Quake 和 Heretic。啊!那些美好的,让人心动的 8位图像。 - -那个时候硬件的发展一直落后于软件,因此我经常升级硬件。现在我们能买到满足我们需求的任何配置的电脑。我已经好多年都没有再更新我的任何电脑硬件了。 - -### 比特杂志 - -回到那些曾经辉煌的年代,电脑商店布满大街小巷,找到本地的一家网络服务提供商(ISP)也不用走遍整条街。ISP 那个时候真的非比寻常。他们都是讨人喜欢的一些友好的大公司,像美国电信公司和有线电视公司等。他们都非常友好,并且提供各种各样的像 BBS、文件下载、MUDs (多玩家在线游戏)等的额外服务。 - -我花了很多的时间在电脑商店购买配件,但是很多时候我一个女人家去那里会让店里的员工感到吃惊,我真的很无语了,这怎么就会让一些人看不惯了。我现成已经是一位58岁的老奶了,但是他们还是一样的看不惯我。我希望我这个让人讨厌的女人在我死之前能被他们所接受。 - -那些商店的书架上摆满了比特杂志。有关比特杂志的历史信息可以在互联网档案库中查到。比特杂志是当地一家免费的杂志,有很多关于计算机方面的优秀的文章和大量的广告。可惜当时的广告没有网络版,因此大家不能再看到那些很有价值的关于计算机方面的详细信息了。你知道现在的广告商们有多么的抓狂吗?他们埋怨那些排斥广告的人,致使他们不得不在科技新闻中植入一些隐藏的广告。他们应该学习一下过去的广告模式,那时候的广告里有很多有价值的信息,大家都喜欢阅读。我从比特杂志和其它的电脑杂志的广告中学到了所有关于计算机硬件的知识。电脑购买者杂志更是非常好的学习资料,其中有上百页的广告和很多高质量的文章。 - - -![](https://opensource.com/sites/default/files/resize/march2002-300x387.jpg) - -比特杂志的出版者 Paul Harwood 开启了我的写作生涯。我的第一篇计算机专业性质的文章就是在比特杂志发表的。 Paul,仿佛你一直都在我的身旁,谢谢你。 - -在互联网档案库中,关于比特杂志的分类广告信息已经几乎查询不到了。分类广告模式曾经给出版商带来巨大的收入。免费的分类广告网站 Craigslist 在这个领域独占鳌头,同时也扼杀了像比特杂志这种以传统的报纸和出版为主的杂志行业。 - -其中有一些让我最难以忘怀的记忆就是一个12岁左右的吊儿郎当的小屁孩,匆匆忙忙地跑到我钟爱的电脑店里递给我一本初学者好书比特杂志,他脸上挂满了对我这种光鲜亮丽的女人所做工作的不屑和不理解的表情。我拆开杂志,指着其中一篇我写的关于Linux系统的文章给他看,他说“哦,确实是这样的,我明白了”。他尴尬的脸变成了那种正常生理上都不可能呈现的颜色,然后很仓促地夹着尾巴溜走了。(不是的,我亲爱的、诚实的读者们,他不是真的只有12岁,而应该是20来岁。他现在应该比以前更成熟懂事一些了吧!) - -###发现 Linux - -我第一次了解到 Linux 系统是在比特杂志上,大概在 1977年左右。我一开始用的一些 Linux操作系统版本是 Red Hat 5 和 Mandrake Linux。 Mandrake真是太棒了,它是第一款易安装型的Linux系统,并且还附带图形界面和声卡驱动,因此我可以快速玩转 Tux Racer 游戏。不像那时候大多数的有教育背景的 Linux 用户那样,因为我之前没接触过 Unix系统,所以我学习起来比较难。但是一切都还顺利吧,因为我学到的东西都很有用。相对于学习 Windows 系统来说,我在 Linux 系统里学到的东西都很实用,也没必要像在Windows环境中那样非得把任务拖放到DOS下来执行。 - -玩转电脑真是充满了太多的乐趣,后来我转行成为计算机自由顾问,去帮助一些小型公司的IT部门把数据迁移到 Linux 服务器上,这迫使 Windows 系统或多或少的失去一些用武之地。通常情况下我们都是在背地里偷偷地做这些工作的,因为那段时期微软把 Linux 称为毒瘤,诬蔑 Linux 系统是一种共产主义用来削弱和吞噬我们身体里的珍贵血液的阴谋。 - -### Linux 赢了 - -我持续做了很多年的顾问工作,也做其它一些相关的工作,比如:电脑硬件修理和升级、布线、系统和网络管理,还有其它一些混合的包括 Apple、Windows、Linux 系统在内的网络工作。Apple 和 Windows 系统故意不兼容对方,因此这两个系统真的是最头疼,也最难整合到同一网络中。但是在Linux系统和其它开源软件中最有趣的一件事是总有一些人能够时时处理这些厂商之间的兼容性问题。 - -现在已经大不同了。在系统间的互操作方面一直存在一些兼容性问题,而且也没有1级桌面的 Linux 系统 OEM 厂商。我觉得这是因为微软和苹果公司在零售行业在大力垄断造成的。也许他们这样做反而帮了我们帮大忙,因为我们可以从 ZaReason 和 System76 这样独立的 Linux 系统开发商得到更好的服务和更高性能的系统。他们对 Linux 系统都很专业,也特别欢迎我们这样的用户。 - -Linux 除了在零售行业的桌面版系统占有一席之地以外,从嵌入式系统到超级计算机以及分布式计算系统等各个方面都占据着主要地位。开源技术掌握着软件行业的发展方向,所有的软件行业的前沿技术,比如容器、集群以及人工智能的发展都离不开开源技术的支持。从我的第一台老式的 386SX 电脑到现在,Linux 和开源技术都取得了巨大的进步。 - -如果没有 Linux 系统和开源,这一切都不可能发生。 - --------------------------------------------------------------------------------- - -via: https://opensource.com/life/16/7/my-linux-story-carla-schroder - -作者:[Carla Schroder ][a] -译者:[译者ID](https://github.com/rusking) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/carlaschroder diff --git a/translated/talk/20160803 The revenge of Linux.md b/translated/talk/20160803 The revenge of Linux.md index 96eb9f362f..6308cadf64 100644 --- a/translated/talk/20160803 The revenge of Linux.md +++ b/translated/talk/20160803 The revenge of Linux.md @@ -1,5 +1,3 @@ -Translated by H-mudcup - Linux 的逆袭 ======================== From e1d75e8a28a72291ea40cfe166312bdd80c8ab9a Mon Sep 17 00:00:00 2001 From: Bestony Date: Fri, 9 Sep 2016 14:58:58 +0800 Subject: [PATCH 0082/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...inux Package Managers for Linux Newbies.md | 96 +++++++++---------- 1 file changed, 47 insertions(+), 49 deletions(-) diff --git a/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md b/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md index caac94b3bc..624cb14cd5 100644 --- a/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md +++ b/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md @@ -1,106 +1,104 @@ transalting by bestony -5 Best Linux Package Managers for Linux Newbies + +5个对 Linux 新人来说最好的 Linux 包管理器 ===================================================== +一个 Linux 新用户应该知道 他/她 进步是源于他们使用已经存在的多个 Linux 发行版,使用不同的方式来管理包。 -One thing a new Linux user will get to know as he/she progresses in using it is the existence of several Linux distributions and the different ways they manage packages. - -Package management is very important in Linux, and knowing how to use multiple package managers can proof life saving for a power user, since downloading or installing software from repositories, plus updating, handling dependencies and uninstalling software is very vital and a critical section in Linux system Administration. +在 Linux 中,包管理器非常重要,知道如何使用多种包管理器可以为一个高级用户优化生活,从到仓库下载软件、安装软件、更新软件、处理依赖是非常重要的,这也是Linux 系统管理的一个重要部分。 ![](http://www.tecmint.com/wp-content/uploads/2016/06/Best-Linux-Package-Managers.png) ->Best Linux Package Managers +>最好的Linux包管理器 -Therefore to become a Linux power user, it is significant to understand how the major Linux distributions actually handle packages and in this article, we shall take a look at some of the best package managers you can find in Linux. +为了成为一个 Linux 高级用户,一个标志是了解主要的 Linux 发行版如何处理包,在这篇文章中,我们应该看一些你能找到的Linux的最好的包管理器, -Here, our main focus is on relevant information about some of the best package managers, but not how to use them, that is left to you to discover more. But I will provide meaningful links that point out usage guides and many more. +在这里,我们的主要重点是关于一些最好包管理器的相关信息,但不是如何使用它们,这些留给你亲自发现。但我会提供一些有意义的链接,使用指南或更多。 -### 1. DPKG – Debian Package Management System +### 1. DPKG – Debian 包管理系统 -Dpkg is a base package management system for the Debian Linux family, it is used to install, remove, store and provide information about `.deb` packages. +Dpkg 是 Debian Linux 家族的基础包管理系统,他用于安装、删除、存储和提供`.deb`包的信息 -It is a low-level tool and there are front-end tools that help users to obtain packages from remote repositories and/or handle complex package relations and these include: +这是一个低级的工具,并且他的前面有多个前端工具来帮助用户从远程的仓库获取包,或处理复杂的包关系的工具,包括: -Don’t Miss: [15 Practical Examples of “dpkg commands” for Debian Based Distros][1] +不要错过: [15 Practical Examples of “dpkg commands” for Debian Based Distros][1] -#### APT (Advanced Packaging Tool) +#### APT (高级打包工具) -It is a very popular, free, powerful and more so, useful command line package management system that is a front end for dpkg package management system. +这个是一个基于 DPKG 包管理系统非常受欢迎的、免费的、强大的,有用的命令行包管理器系统 -Users of Debian or its derivatives such as Ubuntu and Linux Mint should be familiar with this package management tool. +Debian 的用户 或他的衍生版 例如 Ubuntu 和 Linux Mint 在包管理工具上非常的相似 -To understand how it actually works, you can go over these how to guides: +想要了解他是如何工作的,你可以去看面这些 HOW TO 指南 -Don’t Miss: [15 Examples of How to Use New Advanced Package Tool (APT) in Ubuntu/Debian][2] +不要错过: [15 Examples of How to Use New Advanced Package Tool (APT) in Ubuntu/Debian][2] -Don’t Miss: [25 Useful Basic Commands of APT-GET and APT-CACHE for Package Management][3] +不要错过: [25 Useful Basic Commands of APT-GET and APT-CACHE for Package Management][3] -#### Aptitude Package Manager +#### Aptitude 包管理器 -This is also a popular command line front-end package management tool for Debian Linux family, it works similar to APT and there have been a lot of comparisons between the two, but above all, testing out both can make you understand which one actually works better. +这个也是 Debian Linux 家族一个非常出名的命令行前端包管理工具,它工作很像 APT ,它们之间有很多比较,但是测试可以让你明白那个工作的更好。 -It was initially built for Debian and its derivatives but now its functionality stretches to RHEL family as well. You can refer to this guide for more understanding of APT and Aptitude: +它最初为 Debian 和它的衍生版设计的,但是现在它的功能延伸到 RHEL 家族.你可以参考这个指南了解更多关于 APT 和 Aptitude. -Don’t Miss: [What is APT and Aptitude? and What’s real Difference Between Them?][4] +不要错过: [What is APT and Aptitude? and What’s real Difference Between Them?][4] -#### Synaptic Package Manager +#### Synaptic 包管理器 -Synaptic is a GUI package management tool for APT based on GTK+ and it works fine for users who may not want to get their hands dirty on a command line. It implements the same features as apt-get command line tool. +Synaptic是一个基于GTK+的APT的可视化包管理器,对于一些不想使用命令行的用户,它非常好用。 -### 2. RPM (Red Hat Package Manager) +### 2. RPM (红帽包管理器) -This is the Linux Standard Base packing format and a base package management system created by RedHat. Being the underlying system, there several front-end package management tools that you can use with it and but we shall only look at the best and that is: +这个是红帽创建的Linux 基础打包格式和包管理系统。基于底层的系统,你可以使用的多个前端包管理工具,但我们应该只看那些最好的,比如: #### YUM (Yellowdog Updater, Modified) -It is an open source and popular command line package manager that works as a interface for users to RPM. You can compare it to APT under Debian Linux systems, it incorporates the common functionalities that APT has. You can get a clear understanding of YUM with examples from this how to guide: +这个是一个作为对用户的 RPM 管理接口的开源、流行的命令行包管理器。你可以把它和 Debian Linux 系统中的 APT 进行对比,它和 APT 拥有相同的功能。你可以从这个 How to 指南中的例子更加清晰的理解YUM +不要错过: [20 Linux YUM Commands for Package Management][5] -Don’t Miss: [20 Linux YUM Commands for Package Management][5] +#### DNF – 优美的 Yum -#### DNF – Dandified Yum +这个也是一个基于RPM的发行版的包管理器,Fedora 18 引入了它,它是下一代YUM。 -It is also a package manager for the RPM-based distributions, introduced in Fedora 18 and it is the next generation of version of YUM. +如果你用 Fedora 22 或更新,你需要意识到,它是默认的包管理器.这里有一些链接,将为你提供更多关于 DNF 的信息和如何使用它。 -If you have been using Fedora 22 onwards, you must have realized that it is the default package manager. Here are some links that will provide you more information about DNF and how to use it: +不要错过: [DNF – The Next Generation Package Management for RPM Based Distributions][6] -Don’t Miss: [DNF – The Next Generation Package Management for RPM Based Distributions][6] +不要错过: [27 ‘DNF’ Commands Examples to Manage Fedora Package Management][7] -Don’t Miss: [27 ‘DNF’ Commands Examples to Manage Fedora Package Management][7] +### 3. Pacman 包管理器 – Arch Linux -### 3. Pacman Package Manager – Arch Linux +这个是一个流行而且强大的包管理器,它服务于 Arch Linux 和其他的一些小众发行版。它提供了一些其他包管理器提供的基本功能,包括安装、升级、卸载和降级软件。 -It is a popular and powerful yet simple package manager for Arch Linux and some little known Linux distributions, it provides some of the fundamental functionalities that other common package managers provide including installing, automatic dependency resolution, upgrading, uninstalling and also downgrading software. +但是很有效果,它为 Arch 用户 创建了一个简单易用的包管理方式。你可以阅读 [Pacman overview][8],它会解释上面提到的一些功能。 -But most effectively, it is built to be simple for easy package management by Arch users. You can read this [Pacman overview][8] which explains into details some of its functions mentioned above. +### 4. Zypper 包管理器 – openSUSE -### 4. Zypper Package Manager – openSUSE +这个是一个使用 libzypp 库制作的用于 OpenSUSE 系统上的命令行包管理器,它的功能包括访问仓库、安装包、解决依赖问题和其他功能。 -It is a command line package manager on OpenSUSE Linux and makes use of the libzypp library, its common functionalities include repository access, package installation, resolution of dependencies issues and many more. +更重要的是,它也可以处理存储库扩展,如模式、补丁和产品。新的 OpenSUSE 用户可以参考下面的链接来掌控它。 -Importantly, it can also handle repository extensions such as patterns, patches, and products. New OpenSUSE user can refer to this following guide to master it. +不要错过: [45 Zypper Commands to Master OpenSUSE Package Management][9] -Don’t Miss: [45 Zypper Commands to Master OpenSUSE Package Management][9] +### 5. Portage 包管理器 – Gentoo -### 5. Portage Package Manager – Gentoo +这个是 Gentoo 的包管理器,当下不怎么流行的一个发行版,但是这并不阻止他是 Linux 下最好的软件包管理器之一。 -It is a package manager for Gentoo, a less popular Linux distribution as of now, but this won’t limit it as one of the best package managers in Linux. +Portage项目的主要目标是创建一个简单、无故障的包含向后兼容、自动化等功能的包管理系统。 -The main aim of the Portage project is to make a simple and trouble free package management system to include functionalities such as backwards compatibility, automation plus many more. +如果希望理解的更清晰,可以看下: [Portage project page][10]. -For better understanding, try reading [Portage project page][10]. +### 结束语 -### Concluding Remarks - -As I already hinted at the beginning, the main purpose of this guide was to provide Linux users a list of the best package managers but knowing how to use them can be done by following the necessary links provided and trying to test them out. - -Users of the different Linux distributions will have to learn more on their own to better understand the different package managers mentioned above. +正如我在开始时提到的,这个指南的主要意图是给 Linux 用户提供一个最好的包管理器的列表,但知道如何使用它们可以通过下面提供的重要的链接和尝试测试它们。 +不同发行版的用户将学习超出他们本身的来理解上述提到的不同的包管理器。 -------------------------------------------------------------------------------- via: http://www.tecmint.com/linux-package-managers/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+tecmint+%28Tecmint%3A+Linux+Howto%27s+Guide%29 作者:[Ravi Saive][a] -译者:[译者ID](https://github.com/译者ID) +译者:[Bestony](https://github.com/bestony) 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 From 6753a4be6b7b190bbb8beaf55a934606292a1550 Mon Sep 17 00:00:00 2001 From: Bestony Date: Fri, 9 Sep 2016 14:59:38 +0800 Subject: [PATCH 0083/2437] =?UTF-8?q?=E7=A7=BB=E5=8A=A8=E6=96=87=E4=BB=B6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../20160620 5 Best Linux Package Managers for Linux Newbies.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename {sources/talk => translated/tech}/20160620 5 Best Linux Package Managers for Linux Newbies.md (100%) diff --git a/sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md b/translated/tech/20160620 5 Best Linux Package Managers for Linux Newbies.md similarity index 100% rename from sources/talk/20160620 5 Best Linux Package Managers for Linux Newbies.md rename to translated/tech/20160620 5 Best Linux Package Managers for Linux Newbies.md From d449f88f4f94715a53bb3513e6c7bea3e8c3e5f9 Mon Sep 17 00:00:00 2001 From: Bestony Date: Fri, 9 Sep 2016 15:10:13 +0800 Subject: [PATCH 0084/2437] =?UTF-8?q?=E8=AE=A4=E9=A2=86=E6=96=87=E7=AB=A0?= =?UTF-8?q?=E3=80=82?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...0823 Publish Your Project Documentation with GitHub Pages.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md b/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md index ff8cd2385e..76bcc0fa7a 100644 --- a/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md +++ b/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md @@ -1,3 +1,5 @@ +translating by Bestony + Publish Your Project Documentation with GitHub Pages ===== From c84714bc058f3428299d6e4ee7ac2de086dcf881 Mon Sep 17 00:00:00 2001 From: wxy Date: Fri, 9 Sep 2016 15:14:43 +0800 Subject: [PATCH 0085/2437] PUB:20160803 The revenge of Linux @H-mudcup --- published/20160803 The revenge of Linux.md | 36 +++++++++++++++++++ .../talk/20160803 The revenge of Linux.md | 36 ------------------- 2 files changed, 36 insertions(+), 36 deletions(-) create mode 100644 published/20160803 The revenge of Linux.md delete mode 100644 translated/talk/20160803 The revenge of Linux.md diff --git a/published/20160803 The revenge of Linux.md b/published/20160803 The revenge of Linux.md new file mode 100644 index 0000000000..5417b62a05 --- /dev/null +++ b/published/20160803 The revenge of Linux.md @@ -0,0 +1,36 @@ +Linux 的逆袭 +======================== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/penguin%20swimming.jpg?itok=mfhEdRdM) + +Linux 系统在早期的时候被人们嘲笑,它什么也干不了。而现在,Linux 无处不在! + +我当时还是个在巴西学习计算机工程的大三学生,并同时在一个全球审计和顾问公司兼职系统管理员。公司决定用 Oracle 数据库开发一些企业资源计划(ERP)软件。因此,我得以在 Digital UNIX OS (DEC Alpha) 进行训练,这个训练颠覆了我的三观。 + +UNIX 系统非常的强大,而且给予了我们对机器上包括存储系统、网络、应用和其他一切的绝对控制权。 + +我开始在 ksh 和 Bash 里编写大量的脚本让系统进行自动备份、文件传输、提取转换加载(ETL)操作、自动化 DBA 日常工作,还为各种不同的项目创建了许多服务。此外,调整数据库和操作系统的工作让我更好的理解了如何让服务器以最佳方式运行。在那时,我在自己的个人电脑上使用的是 Windows 95 系统,而我非常想要在我的个人电脑里放进一个 Digital UNIX,或者哪怕是 Solaris 或 HP-UX 也行,但是那些 UNIX 系统都得在特定的硬件才能上运行。我阅读了所有的系统文档,还找过其它的书籍以求获得更多的信息,也在我们的开发环境里对这些疯狂的想法进行了实验。 + +后来在大学里,我从我的同事那听说了 Linux。我那时非常激动的从还在用拨号方式连接的因特网上下载了它。在我的正宗的个人电脑里装上 UNIX 这类系统的这个想法真是太酷了! + +Linux 不同于 UNIX 系统,它设计用来在各种常见个人电脑硬件上运行,在起初,让它开始工作确实有点困难,Linux 针对的用户群只有系统管理员和极客们。我为了让它能运行,甚至用 C 语言修改了驱动软件。我之前使用 UNIX 的经历让我在编译 Linux 内核,排错这些过程中非常的顺手。由于它不同于那些只适合特定硬件配置的封闭系统,所以让 Linux 跟各种意料之外的硬件配置一起工作真的是件非常具有挑战性的事。 + +我曾见过 Linux 在数据中心获得一席之地。一些具有冒险精神的系统管理员使用它来帮他们完成每天监视和管理基础设施的工作,随后,Linux 作为 DNS 和 DHCP 服务器、打印管理和文件服务器等赢得了更多的使用。企业曾对 Linux 有着很多顾虑(恐惧,不确定性,怀疑(FUD:fear, uncertainty and doubt))和诟病:谁是它的拥有者?由谁来支持它?有适用于它的应用吗? + +但现在看来,Linux 在各个地方进行着逆袭!从开发者的个人电脑到大企业的服务器;我们能在智能手机、智能手表以及像树莓派这样的物联网(IoT)设备里找到它。甚至 Mac OS X 有些命令跟我们所熟悉的命令一样。微软在制造它自己的发行版,在 Azure 上运行,然后…… Windows 10 要装备 Bash。 + +有趣的是 IT 市场会不断地创造并迅速的用新技术替代,但是我们所掌握的 Digital UNIX、HP-UX 和 Solaris 这些旧系统的知识还依然有效并跟 Linux 息息相关,不论是为了工作还是玩。现在我们能完全的掌控我们的系统,并使它发挥最大的效用。此外,Linux 有个充满热情的社区。 + +我真的建议想在计算机方面发展的年轻人学习 Linux,不论你处于 IT 界里的哪个分支。如果你深入了解了一个普通的家用个人电脑是如何工作的,你就可以以基本相同的方式来面对任何机器。你可以通过 Linux 学习最基本的计算机知识,并通过它建立能在 IT 界任何地方都有用的能力! + +-------------------------------------------------------------------------------- + +via: https://opensource.com/life/16/8/revenge-linux + +作者:[Daniel Carvalho][a] +译者:[H-mudcup](https://github.com/H-mudcup) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/danielscarvalho diff --git a/translated/talk/20160803 The revenge of Linux.md b/translated/talk/20160803 The revenge of Linux.md deleted file mode 100644 index 6308cadf64..0000000000 --- a/translated/talk/20160803 The revenge of Linux.md +++ /dev/null @@ -1,36 +0,0 @@ -Linux 的逆袭 -======================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/penguin%20swimming.jpg?itok=mfhEdRdM) - -Linux 系统在早期的时候被人们嘲笑,它什么也干不了。现在,Linux 无处不在! - -我当时还是个在巴西学习计算机工程的大三学生,并同时在一个全球审计和顾问公司兼职系统管理员。公司决定用 Oracle 数据库实现企业资源计划(ERP)软件。我得以在 Digital UNIX OS (DEC Alpha) 进行训练,这个训练颠覆了我的三观。 - -那个 UNIX 系统非常的强大并且给予了我们堆积起的绝对控制权:存储系统、网络、应用和其他一切。 - -我开始在 ksh 和 Bash 里编写大量的脚本让系统自动进行备份、文件转移、提取转换加载(ETL)操作、自动化 DBA 日常工作,还创造了很多从各种不同的项目提出的其他的服务。此外,调整数据库和操作系统的工作让我更好的理解了如何让服务器以最佳方式运行。在那时,我在自己的个人电脑上使用的是 Windows 95 系统,而我非常想要在我的个人电脑里放进一个 Digital UNIX,或者即使是 Solaris 或 HP-UX 也行,但是那些 UNIX 系统都得在特定的硬件才能上运行。我阅读了所有系统的文档,还找过额外的书籍以求获得更多的信息,也在我们的开发环境里对一些疯狂的想法进行了实验。 - -后来在大学里,我从我的同事那听说了 Linux。我那时非常激动的从还在用拨号方式连接的因特网上下载了它。在我标准的个人电脑里装上 UNIX 这类系统的这个想法真是太酷了! - -由于 Linux 不同于 UNIX 系统,它能在几乎所有的个人电脑硬件上运行,在起初,让它开始工作真的是非常困难的一件事。Linux 针对的用户群只有系统管理员和极客们。我为了让它能运行,甚至用 C 语言修改了驱动软件。我之前使用 UNIX 的经历让我在编译 Linux 内核,排错这些过程中非常的顺手。由于它不同于那些只适合特定硬件配置的封闭系统,所以让 Linux 跟各种意料之外的硬件配置一起工作真的是件非常具有挑战性的事。 - -我曾见过 Linux 在数据中心获得一席之地。一些具有冒险精神的系统管理员使用它来帮他们完成每天监视和管理基础设施的工作,随后,Linux 同 DNS 和 DHCP 服务器、打印管理和文件服务器一起攻城略地。企业曾对 Linux 有着很多顾虑(恐惧,不确定性,怀疑)和诟病:谁是它的拥有者?由谁来支持它?有适用于它的应用吗? - -但现在看来,Linux 在各个地方进行着逆袭!从开发者的个人电脑到大企业的服务器;我们能在智能手机、智能手表以及像树莓派这样的物联网(IoT)设备里找到它。甚至 Mac OS X 有些 DOS 命令跟我们所熟悉的命令一样。微软在制造它自己的发行版,在 Azure 上运行,然后…… Windows 10 要装备 Bash。 - -有趣的是 IT 市场会创造并迅速的代替新技术,但是我们所掌握的 Digital UNIX、HP-UX 和 Solaris 这些旧系统的知识还依然有效并跟 Linux 息息相关,不论是为了工作还是玩。现在我们能完全的掌控我们的系统,并使它发挥最大的效用。此外,Linux 有个充满热情的社区。 - -我真的建议想在计算机方面发展的年轻人学习 Linux。不论你处于 IT 界里的哪个分支。如过你深入了解了一个标准的家用个人电脑是如何工作的,你就可以在任何机器面前使用基本相同的语言。你可以通过 Linux 学习最基本的计算机知识,并通过它建立能在 IT 界任何地方都有用的能力。 - --------------------------------------------------------------------------------- - -via: https://opensource.com/life/16/8/revenge-linux - -作者:[Daniel Carvalho][a] -译者:[H-mudcup](https://github.com/H-mudcup) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/danielscarvalho From 55daa437c682cddd96593035d399e524b15e6b23 Mon Sep 17 00:00:00 2001 From: Bestony Date: Sat, 10 Sep 2016 11:11:34 +0800 Subject: [PATCH 0086/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...Project Documentation with GitHub Pages.md | 33 +++++++++---------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md b/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md index 76bcc0fa7a..2e1b2746f9 100644 --- a/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md +++ b/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md @@ -1,44 +1,43 @@ -translating by Bestony - -Publish Your Project Documentation with GitHub Pages +使用Github Pages 发布你的项目文档 ===== -You might be familiar with [how GitHub Pages helps you share your work with the world][3] or maybe you have [attended a class][4] that helped you build your first GitHub Pages site. Recent improvements to GitHub Pages have made it easier to [publish your site from a variety of sources][5]. One of these sources is your repository's /docs folder. +你可能比较熟悉[Github Pages 如何帮你分享你的工作][3] 又或许你加入了[一堂课][4] 来帮你建立你的第一个 Github Pages 的网站。近期 Github Pages 的改进使[从不同的源来发布您的网站][5]更加的方便.其中的来源之一就是你的仓库的 /docs 目录. -Quality documentation is a hallmark of any healthy software project. For open-source projects, however, maintaining a robust compendium of knowledge detailing all the ins and outs is paramount. Well-curated documentation increases your project's approachability, provides asynchronous guidance, and fosters the type of uncoordinated collaboration that propels open-source software development. +文档的质量是任何一个健康的软件项目的标志。对于开源项目来说,维护一个强大的知识大纲详细说明所有的细节是至关重要的。维护很好的文档可以增强项目的可用性,提供一步一步的指导和培养不同步合作可以推动开源软件开发的协作进程。 -Hosting your documentation on the web can present time-consuming challenges that make publishing and maintaining it an unrewarding experience — one that it's often easy to avoid. Grappling with multiple disparate publishing tools like FTP servers and databases means files often exist in various states and multiple locations, all of which require manual synchronization. To be clear, conventional web publishing provides unparalleled flexibility and power; but it comes at the expense of simplicity and, in many cases, utility. +在Web上托管你的文档是一个消耗时间的挑战而且发布和维护不会有什么有用的经验,而这是可以避免的。面对不同的发布工具,比如 FTP 服务器 和 数据库 意味着文件存在于多个不同的状态下和不同的地点,而这些都需要手动来同步。需要澄清的是,传统的 Web 发布提供了无与伦比的灵活性和性能,但是在许多情况下,这是以牺牲简单为代价的。 -When it comes to documentation, a path with less resistance is often the better approach. +当作为文档时,一个路径显然更容易去接触到。 -[GitHub Pages][2] gives you a direct path to create websites for your projects, which makes it a natural choice for publishing and maintaining documentation. Because GitHub Pages supports Jekyll, you can pen your documentation in plain text or Markdown to help maintain a lower barrier to contribution. Jekyll also includes support for many helpful tools like variables, templates, and automatic code highlighting, which gives you much of the flexibility you'd find in a bulkier platform without the added complexity. -Most importantly, using GitHub Pages means your documentation lives alongside your code on GitHub, where you can use things like Issues and Pull Requests to ensure it receives the high level of care it deserves; and because GitHub Pages lets you publish from the /docs directory on the master branch, you can maintain your codebase and its published documentation on the same branch. +[GitHub Pages][2] 给你一个指定的路径来为你的项目创建网站,这使得他天然适合发布和维护文档。因为 Github Pages 支持 Jekyll ,所以你可以使用纯文本或 Markdown 来书写你的文档,降低你维护的成本,减少维护时的障碍。Jekyll还支持许多有用的工具比如变量、模板、以及自动的代码高亮,他会给你更多的灵活性而不会增加复杂性,这些你在一些笨重的平台是见不到的。 -### Get started today +最重要的是,在 Github 上使用 GitHub Pages 意味着你的文档和代码可以使用诸如 Issues 和 Pull Requests 来确保它受到到应有的高水平的维护,而且因为 GitHub Pages 允许您发布主分支的 /docs 目录,您可以在同一分支维护你的代码库及其文档。 -Publishing your first documentation page only takes a few minutes. +### 现在开始! -1. Create a /docs/index.md file on your repository's master branch. +发布你的第一个文档页面只需要短短几分钟。 -2. Add your content and any necessary Jekyll front matter, then commit your changes. +1. 在你的仓库的主分支里创建一个 /docs/index.md 文件。 + +2. 把你的内容以 Jekyll 格式添加进去,并提交你的修改。 ![](https://cloud.githubusercontent.com/assets/3477155/17778793/47c5a586-6533-11e6-982c-ebd41ec6968c.gif) -1. Visit your repository's settings tab and select master branch /docs folder as the GitHub Pages source. Click save, and you're done. +3. 查看你的仓库的设置分支然后选择主分支 /docs 目录,将其设置为 GitHub Pages 的源 ,点击保存,你就搞定了。 ![](https://cloud.githubusercontent.com/assets/3477155/17778792/47c2ecc4-6533-11e6-828a-91980daa7297.gif) -GitHub Pages will read the contents of your /docs directory, convert the index.md into HTML, and publish the results at your GitHub Pages URL. +GitHub Pages 将会从你的 /docs 目录中读取内容,转换 index.md 为 HTML。然后把它发布到你的 GitHub Pages URL 上。 -This will generate the most basic HTML output that you can further customize with templates, CSS, and other features available in Jekyll. To see examples of what all is possible, take a look at the [GitHub Pages Showcase][1]. +这样将会创建一个最基础的HTML并输出,而且你可以在Jekyll自定义模板、CSS 和其他特性.如果想要看所有的可能,你可以看看 [GitHub Pages Showcase][1]。 -------------------------------------------------------------------------------- via: https://github.com/blog/2233-publish-your-project-documentation-with-github-pages 作者:[ loranallensmith ][a] -译者:[译者ID](https://github.com/译者ID) +译者:[Bestony](https://github.com/bestony) 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From 3e07645e210ad63d75e219d250857a841f2ba39b Mon Sep 17 00:00:00 2001 From: Bestony Date: Sat, 10 Sep 2016 11:11:57 +0800 Subject: [PATCH 0087/2437] =?UTF-8?q?=E7=A7=BB=E5=8A=A8=E6=96=87=E4=BB=B6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...160823 Publish Your Project Documentation with GitHub Pages.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename {sources => translated}/tech/20160823 Publish Your Project Documentation with GitHub Pages.md (100%) diff --git a/sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md b/translated/tech/20160823 Publish Your Project Documentation with GitHub Pages.md similarity index 100% rename from sources/tech/20160823 Publish Your Project Documentation with GitHub Pages.md rename to translated/tech/20160823 Publish Your Project Documentation with GitHub Pages.md From b6be861816e89671fa06228c9ecee3786415b721 Mon Sep 17 00:00:00 2001 From: Zxp Date: Sat, 10 Sep 2016 11:55:40 +0800 Subject: [PATCH 0088/2437] Update 20160817 Turtl - Secure, Open Source Evernote Alternative.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 认领 --- ...20160817 Turtl - Secure, Open Source Evernote Alternative.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md b/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md index f86d28c966..5c67768cde 100644 --- a/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md +++ b/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md @@ -1,3 +1,5 @@ +chisper is translating + Turtl: Secure, Open Source Evernote Alternative ============= From 5854c82c59f73fbdf5ed3d17fe9fdfcfc33b9233 Mon Sep 17 00:00:00 2001 From: wxy Date: Sat, 10 Sep 2016 14:12:31 +0800 Subject: [PATCH 0089/2437] =?UTF-8?q?LCTT=202016=20=E6=80=BB=E7=BB=93?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LCTT 三岁啦! --- README.md | 71 +++++++++++++++++++++++------------------------------ lctt2016.md | 42 +++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+), 40 deletions(-) create mode 100644 lctt2016.md diff --git a/README.md b/README.md index bbc3753a39..0e2ee3dc87 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ LCTT 的组成 * 2013/09/16 公开发布了翻译组成立消息后,又有新的成员申请加入了。并从此建立见习成员制度。 * 2013/09/24 鉴于大家使用 GitHub 的水平不一,容易导致主仓库的一些错误,因此换成了常规的 fork+PR 的模式来进行翻译流程。 * 2013/10/11 根据对 LCTT 的贡献,划分了 Core Translators 组,最先的加入成员是 vito-L 和 tinyeyeser。 -* 2013/10/12 取消对 LINUX.CN 注册用户的依赖,在 QQ 群内、文章内都采用 GitHub 的注册 ID。 +* 2013/10/12 取消对 LINUX.CN 注册用户的关联,在 QQ 群内、文章内都采用 GitHub 的注册 ID。 * 2013/10/18 正式启动 man 翻译计划。 * 2013/11/10 举行第一次北京线下聚会。 * 2014/01/02 增加了 Core Translators 成员: geekpi。 @@ -52,7 +52,10 @@ LCTT 的组成 * 2015/04/19 发起 LFS-BOOK-7.7-systemd 项目。 * 2015/06/09 提升 ictlyh 和 dongfengweixiao 为 Core Translators 成员。 * 2015/11/10 提升 strugglingyouth、FSSlc、Vic020、alim0x 为 Core Translators 成员。 +* 2016/02/18 由于选题 DeadFire 重病,任命 oska874 接手选题工作。 +* 2016/02/29 选题 DeadFire 病逝。 * 2016/05/09 提升 PurlingNayuki 为校对。 +* 2016/09/10 LCTT 三周年。 活跃成员 ------------------------------- @@ -61,25 +64,26 @@ LCTT 的组成 - Leader @wxy, - Source @oska874, - Proofreader @PurlingNayuki, +- Proofreader @carolinewuyan, - CORE @geekpi, - CORE @GOLinux, - CORE @ictlyh, -- CORE @carolinewuyan, - CORE @strugglingyouth, - CORE @FSSlc - CORE @zpl1025, - CORE @runningwater, - CORE @bazz2, - CORE @Vic020, -- CORE @dongfengweixiao, - CORE @alim0x, +- CORE @tinyeyeser, +- CORE @Locez, - Senior @DeadFire, - Senior @reinoir222, -- Senior @tinyeyeser, - Senior @vito-L, - Senior @jasminepeng, - Senior @willqian, - Senior @vizv, +- Senior @dongfengweixiao, - ZTinoZ, - martin2011qi, - theo-l, @@ -87,94 +91,81 @@ LCTT 的组成 - wi-cuckoo, - disylee, - haimingfg, -- KayGuoWhu, - wwy-hust, - felixonmars, +- KayGuoWhu, +- mr-ping, - su-kaiyao, -- GHLandy, -- ivo-wang, -- cvsher, +- StdioA, - wyangsun, +- ivo-wang, +- GHLandy, +- cvsher, - DongShuaike, - flsf, - SPccman, - Stevearzh -- mr-ping, - Linchenguang, - Linux-pdz, - 2q1w2007, - H-mudcup, -- cposture, +- MikeCoder, +- NearTan, +- goreliu, - xiqingongzi, - goreliu, -- NearTan, - TxmszLou, - ZhouJ-sh, - wangjiezhe, - icybreaker, +- vim-kakali, - shipsw, +- Moelf, +- name1e5s, - johnhoow, - soooogreen, +- kokialoves, - linuhap, +- GitFuture, +- ChrisLeeGit, - blueabysm, - boredivan, -- name1e5s, -- StdioA, - yechunxiao19, -- l3b2w1, - XLCYun, - KevinSJ, +- zky001, +- l3b2w1, - tenght, - coloka, - luoyutiantang, - sonofelice, - jiajia9linuxer, - scusjs, -- tnuoccalanosrep, - woodboow, - 1w2b3l, - JonathanKang, +- bestony, - crowner, - dingdongnigetou, - mtunique, -- CNprober, +- Rekii008, - hyaocuk, - szrlee, -- KnightJoker, - Xuanwo, - nd0104, -- Moelf, - xiaoyu33, -- guodongxiaren, - ynmlml, -- vim-kakali, +- Flowsnow, - ggaaooppeenngg, -- Ricky-Gong, -- zky001, -- lfzark, -- 213edu, -- bestony, - mudongliang, +- Tanete, +- lfzark, - liuaiping, -- Timeszoro, - rogetfan, - JeffDing, - Yuking-net, - -(按增加行数排名前百) - -LFS 项目活跃成员有: - -- @ictlyh -- @dongfengweixiao -- @wxy -- @H-mudcup -- @zpl1025 -- @KevinSJ -- @Yuking-net - -(更新于2016/06/20) +(按增加行数排名前百,更新于2016/09/10) 谢谢大家的支持! diff --git a/lctt2016.md b/lctt2016.md new file mode 100644 index 0000000000..3481a9d40b --- /dev/null +++ b/lctt2016.md @@ -0,0 +1,42 @@ +LCTT 2016:LCTT 成立三周年了! +=========================== + +不知不觉,LCTT 已经成立三年了,对于我这样已经迈过四张的人来说,愈发的感觉时间过得真快。 + +这三年来,我们 LCTT 经历了很多事情,有些事情想起来仍恍如昨日。 + +三年前的这一天,我的一个偶发的想法促使我在 Linux 中国的 QQ 群里面发了一则消息,具体的消息内容已经不可考了,大意是鉴于英文 man 手册的中文翻译太差,想组织一些人来重新翻译。不料发出去之后很快赢得一些有热情、有理想的小伙伴们的响应。于是我匆匆建了一个群,拉了一些人进来,甚至当时连翻译组的名称都没有确定。LCTT (Linux.Cn Translation Team)这个名称都是后来逐步定下来的。 + +关于 LCTT 的早期发展情况,可以参考 LCTT 2014 年周年[总结](http://linux.cn/article-3784-1.html)。 + +虽然说“翻译 man 手册”这个最初建群的目标因为种种原因搁浅而一直未能重启,但是,这三年来,我们组织了 [213 位志愿者](https://github.com/LCTT/TranslateProject/graphs/contributors)翻译了 2155 篇文章,接受了 [4263 个 PR](https://github.com/LCTT/TranslateProject/pulls?q=is%3Apr+is%3Aclosed),得到了 476 个星。 + +这三年来,我们经历了 man 项目的流产、 LFS 手册的翻译发布、选题 DeadFire 的离去。得益于 Linux 中国的网站和微博,乃至微信的兴起后的传播,志愿者们的译文传播很广,切实的为国内的开源社区做出了贡献(当然,与此同时,Linux 中国社区也随之更加壮大)。 + +这些年间,LCTT 来了很多人,也有人慢慢淡出,这里面涌现了不少做出了卓越贡献的人,比如: + +- geekpi,作为整个 LCTT 项目中翻译量最大贡献者,却鲜少在群内说话,偶尔露面,被戏称为“鸡排兄”。 +- GOLinux,紧追“鸡排兄”的第二位强人,嗯,群内大部分人的昵称都是他起的,包括楼上。 +- tinyeyeser,“小眼儿”以翻译风趣幽默著称,是 LCTT 早期初创成员之一。 +- Vito-L,早期成员,LCTT 的多数 Wiki 出自于其手。 +- DeadFire,创始成员,从最开始到其离世,一直负责 LCTT 的所有选题工作。 +- oska874,在接过选题工作的重任后,全面主持 LCTT 的工作。 +- carolinewuyan,承担了相当多的校对工作。 +- alim0x,独立完成了 Android 编年史系列的翻译(多达 26 篇,现在还没发布完)等等。 + +其它还有 ictlyh、strugglingyouth、FSSlc、zpl1025、runningwater、bazz2、Vic020、dongfengweixiao、jasminepeng、willqian、vizv、ZTinoZ、martin2011qi、felixonmars、su-kaiyao、GHLandy、flsf、H-mudcup、StdioA、crowner、vim-kakali 等等,以及还有很多这里没有提到名字的人,都对 LCTT 做出不可磨灭的贡献。 + +具体的贡献排行榜,可以看[这里](https://github.com/LCTT/TranslateProject/graphs/contributors)。 + +每年写总结时,我都需要和 gource 以及 ffmpeg 搏斗半天,今年,我又用 gource 重新制作了一份 LCTT 的 GitHub 版本仓库的变迁视频,以飨众人。 + +本来想写很多,或许 LCTT 和 Linux 中国已经成了我的生活的一部分,竟然不知道该写点什么了,那就此搁笔罢。 + +另外,为 LCTT 的诸位兄弟姐妹们献上我及管理团队的祝福,也欢迎更多的志愿者加入 LCTT ,传送门在此: + +- 项目网站:https://lctt.github.io/ ,请先访问此处了解情况。 +- “Linux中国”开源社区:https://linux.cn/ ,所有翻译的文章都在这里以及它的同名微博、微信发布。 + +LCTT 组长 wxy + +2016/9/10 \ No newline at end of file From de3b44173ef31009220b6df282d8556e92253c82 Mon Sep 17 00:00:00 2001 From: wxy Date: Sat, 10 Sep 2016 16:04:15 +0800 Subject: [PATCH 0090/2437] PUB:20160620 5 Best Linux Package Managers for Linux Newbies @bestony --- ...inux Package Managers for Linux Newbies.md | 114 +++++++++++++++++ ...inux Package Managers for Linux Newbies.md | 116 ------------------ 2 files changed, 114 insertions(+), 116 deletions(-) create mode 100644 published/20160620 5 Best Linux Package Managers for Linux Newbies.md delete mode 100644 translated/tech/20160620 5 Best Linux Package Managers for Linux Newbies.md diff --git a/published/20160620 5 Best Linux Package Managers for Linux Newbies.md b/published/20160620 5 Best Linux Package Managers for Linux Newbies.md new file mode 100644 index 0000000000..e264bf8819 --- /dev/null +++ b/published/20160620 5 Best Linux Package Managers for Linux Newbies.md @@ -0,0 +1,114 @@ +5 个给 Linux 新手的最佳包管理器 +===================================================== + +一个 Linux 新用户应该知道他或她的进步源自于对 Linux 发行版的使用,而 Linux 发行版有好几种,并以不同的方式管理软件包。 + +在 Linux 中,包管理器非常重要,知道如何使用多种包管理器可以让你像一个高手一样活得很舒适,从在仓库下载软件、安装软件,到更新软件、处理依赖和删除软件是非常重要的,这也是Linux 系统管理的一个重要部分。 + +![](http://www.tecmint.com/wp-content/uploads/2016/06/Best-Linux-Package-Managers.png) + +*最好的Linux包管理器* + +成为一个 Linux 高手的一个标志是了解主要的 Linux 发行版如何处理包,在这篇文章中,我们应该看一些你在 Linux 上能找到的最佳的包管理器, + +在这里,我们的主要重点是关于一些最佳包管理器的相关信息,但不是如何使用它们,这些留给你亲自发现。但我会提供一些有意义的链接,使用指南或更多。 + +### 1. DPKG - Debian 包管理系统(Debian Package Management System) + +Dpkg 是 Debian Linux 家族的基础包管理系统,它用于安装、删除、存储和提供`.deb`包的信息。 + +这是一个低层面的工具,并且有多个前端工具可以帮助用户从远程的仓库获取包,或处理复杂的包关系的工具,包括如下: + +- 参考:[15 个用于基于 Debian 的发行版的 “dpkg” 命令实例][1] + +#### APT (高级打包工具(Advanced Packaging Tool)) + +这个是一个 dpkg 包管理系统的前端工具,它是一个非常受欢迎的、自由而强大的,有用的命令行包管理器系统。 + +Debian 及其衍生版,例如 Ubuntu 和 Linux Mint 的用户应该非常熟悉这个包管理工具。 + +想要了解它是如何工作的,你可以去看看下面这些 HOW TO 指南: + +- 参考:[15 个怎样在 Ubuntu/Debian 上使用新的 APT 工具的例子][2] +- 参考:[25 个用于包管理的有用的 APT-GET 和 APT-CACHE 的基础命令][3] + +#### Aptitude 包管理器 + +这个也是 Debian Linux 家族一个非常出名的命令行前端包管理工具,它工作方式类似 APT ,它们之间有很多可以比较的地方,不过,你应该两个都试试才知道哪个工作的更好。 + +它最初为 Debian 及其衍生版设计的,但是现在它的功能延伸到 RHEL 家族。你可以参考这个指南了解更多关于 APT 和 Aptitude。 + +- 参考:[APT 和 Aptitude 是什么?它们知道到底有什么不同?][4] + +#### Synaptic 包管理器 + +Synaptic是一个基于GTK+的APT的可视化包管理器,对于一些不想使用命令行的用户,它非常好用。 + +### 2. RPM 红帽包管理器(Red Hat Package Manager) + +这个是红帽创建的 Linux 基本标准(LSB)打包格式和基础包管理系统。基于这个底层系统,有多个前端包管理工具可供你使用,但我们应该只看那些最好的,那就是: + +#### YUM (黄狗更新器,修改版(Yellowdog Updater, Modified)) + +这个是一个开源、流行的命令行包管理器,它是用户使用 RPM 的界面(之一)。你可以把它和 Debian Linux 系统中的 APT 进行对比,它和 APT 拥有相同的功能。你可以从这个 HOW TO 指南中的例子更加清晰的理解YUM: + +- 参考:[20 个用于包管理的 YUM 命令][5] + +#### DNF(优美的 Yum(Dandified Yum)) + +这个也是一个用于基于 RPM 的发行版的包管理器,Fedora 18 引入了它,它是下一代 YUM。 + +如果你用 Fedora 22 及更新版本,你肯定知道它是默认的包管理器。这里有一些链接,将为你提供更多关于 DNF 的信息和如何使用它。 + +- 参考:[DNF - 基于 RPM 的发行版的下一代通用包管理软件][6] +- 参考: [27 个管理 Fedora 软件包的 ‘DNF’ 命令例子][7] + +### 3. Pacman 包管理器 – Arch Linux + +这个是一个流行的、强大而易用的包管理器,它用于 Arch Linux 和其他的一些小众发行版。它提供了一些其他包管理器提供的基本功能,包括安装、自动解决依赖关系、升级、卸载和降级软件。 + +但是最大的用处是,它为 Arch 用户创建了一个简单易用的包管理方式。你可以阅读 [Pacman 概览][8],它会解释上面提到的一些功能。 + +### 4. Zypper 包管理器 – openSUSE + +这个是一个使用 libzypp 库制作的用于 OpenSUSE 系统上的命令行包管理器,它的常用功能包括访问仓库、安装包、解决依赖问题和其他功能。 + +更重要的是,它也可以支持存储库扩展功能,如模式、补丁和产品。新的 OpenSUSE 用户可以参考下面的链接来掌控它。 + +- 参考:[45 个让你精通 openSUSE 包管理的 Zypper 命令][9] + +### 5. Portage 包管理器 – Gentoo + +这个是 Gentoo 的包管理器,当下不怎么流行的一个发行版,但是这并不阻止它成为 Linux 下最好的软件包管理器之一。 + +Portage 项目的主要目标是创建一个简单、无故障的包管理系统,包含向后兼容、自动化等功能。 + +如果希望理解的更清晰,可以看下: [Portage 项目页][10]。 + +### 结束语 + +正如我在开始时提到的,这个指南的主要意图是给 Linux 用户提供一个最佳软件包管理器的列表,但知道如何使用它们可以通过其后提供的重要的链接,并实际去试试它们。 + +各个发行版的用户需要学习超出他们的发行版之外的一些东西,才能更好理解上述提到的这些不同的包管理器。 + +-------------------------------------------------------------------------------- + +via: http://www.tecmint.com/linux-package-managers/ + +作者:[Ravi Saive][a] +译者:[Bestony](https://github.com/bestony) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/author/admin/ +[1]: http://www.tecmint.com/dpkg-command-examples/ +[2]: http://www.tecmint.com/apt-advanced-package-command-examples-in-ubuntu/ +[3]: http://www.tecmint.com/useful-basic-commands-of-apt-get-and-apt-cache-for-package-management/ +[4]: http://www.tecmint.com/difference-between-apt-and-aptitude/ +[5]: http://www.tecmint.com/20-linux-yum-yellowdog-updater-modified-commands-for-package-mangement/ +[6]: http://www.tecmint.com/dnf-next-generation-package-management-utility-for-linux/ +[7]: http://www.tecmint.com/dnf-commands-for-fedora-rpm-package-management/ +[8]: https://wiki.archlinux.org/index.php/Pacman +[9]: http://www.tecmint.com/zypper-commands-to-manage-suse-linux-package-management/ +[10]: https://wiki.gentoo.org/wiki/Project:Portage diff --git a/translated/tech/20160620 5 Best Linux Package Managers for Linux Newbies.md b/translated/tech/20160620 5 Best Linux Package Managers for Linux Newbies.md deleted file mode 100644 index 624cb14cd5..0000000000 --- a/translated/tech/20160620 5 Best Linux Package Managers for Linux Newbies.md +++ /dev/null @@ -1,116 +0,0 @@ -transalting by bestony - -5个对 Linux 新人来说最好的 Linux 包管理器 -===================================================== - -一个 Linux 新用户应该知道 他/她 进步是源于他们使用已经存在的多个 Linux 发行版,使用不同的方式来管理包。 - -在 Linux 中,包管理器非常重要,知道如何使用多种包管理器可以为一个高级用户优化生活,从到仓库下载软件、安装软件、更新软件、处理依赖是非常重要的,这也是Linux 系统管理的一个重要部分。 - -![](http://www.tecmint.com/wp-content/uploads/2016/06/Best-Linux-Package-Managers.png) ->最好的Linux包管理器 - -为了成为一个 Linux 高级用户,一个标志是了解主要的 Linux 发行版如何处理包,在这篇文章中,我们应该看一些你能找到的Linux的最好的包管理器, - -在这里,我们的主要重点是关于一些最好包管理器的相关信息,但不是如何使用它们,这些留给你亲自发现。但我会提供一些有意义的链接,使用指南或更多。 - -### 1. DPKG – Debian 包管理系统 - -Dpkg 是 Debian Linux 家族的基础包管理系统,他用于安装、删除、存储和提供`.deb`包的信息 - -这是一个低级的工具,并且他的前面有多个前端工具来帮助用户从远程的仓库获取包,或处理复杂的包关系的工具,包括: - -不要错过: [15 Practical Examples of “dpkg commands” for Debian Based Distros][1] - -#### APT (高级打包工具) - -这个是一个基于 DPKG 包管理系统非常受欢迎的、免费的、强大的,有用的命令行包管理器系统 - -Debian 的用户 或他的衍生版 例如 Ubuntu 和 Linux Mint 在包管理工具上非常的相似 - -想要了解他是如何工作的,你可以去看面这些 HOW TO 指南 - -不要错过: [15 Examples of How to Use New Advanced Package Tool (APT) in Ubuntu/Debian][2] - -不要错过: [25 Useful Basic Commands of APT-GET and APT-CACHE for Package Management][3] - -#### Aptitude 包管理器 - -这个也是 Debian Linux 家族一个非常出名的命令行前端包管理工具,它工作很像 APT ,它们之间有很多比较,但是测试可以让你明白那个工作的更好。 - -它最初为 Debian 和它的衍生版设计的,但是现在它的功能延伸到 RHEL 家族.你可以参考这个指南了解更多关于 APT 和 Aptitude. - -不要错过: [What is APT and Aptitude? and What’s real Difference Between Them?][4] - -#### Synaptic 包管理器 - -Synaptic是一个基于GTK+的APT的可视化包管理器,对于一些不想使用命令行的用户,它非常好用。 - -### 2. RPM (红帽包管理器) - -这个是红帽创建的Linux 基础打包格式和包管理系统。基于底层的系统,你可以使用的多个前端包管理工具,但我们应该只看那些最好的,比如: - -#### YUM (Yellowdog Updater, Modified) - -这个是一个作为对用户的 RPM 管理接口的开源、流行的命令行包管理器。你可以把它和 Debian Linux 系统中的 APT 进行对比,它和 APT 拥有相同的功能。你可以从这个 How to 指南中的例子更加清晰的理解YUM -不要错过: [20 Linux YUM Commands for Package Management][5] - -#### DNF – 优美的 Yum - -这个也是一个基于RPM的发行版的包管理器,Fedora 18 引入了它,它是下一代YUM。 - -如果你用 Fedora 22 或更新,你需要意识到,它是默认的包管理器.这里有一些链接,将为你提供更多关于 DNF 的信息和如何使用它。 - -不要错过: [DNF – The Next Generation Package Management for RPM Based Distributions][6] - -不要错过: [27 ‘DNF’ Commands Examples to Manage Fedora Package Management][7] - -### 3. Pacman 包管理器 – Arch Linux - -这个是一个流行而且强大的包管理器,它服务于 Arch Linux 和其他的一些小众发行版。它提供了一些其他包管理器提供的基本功能,包括安装、升级、卸载和降级软件。 - -但是很有效果,它为 Arch 用户 创建了一个简单易用的包管理方式。你可以阅读 [Pacman overview][8],它会解释上面提到的一些功能。 - -### 4. Zypper 包管理器 – openSUSE - -这个是一个使用 libzypp 库制作的用于 OpenSUSE 系统上的命令行包管理器,它的功能包括访问仓库、安装包、解决依赖问题和其他功能。 - -更重要的是,它也可以处理存储库扩展,如模式、补丁和产品。新的 OpenSUSE 用户可以参考下面的链接来掌控它。 - -不要错过: [45 Zypper Commands to Master OpenSUSE Package Management][9] - -### 5. Portage 包管理器 – Gentoo - -这个是 Gentoo 的包管理器,当下不怎么流行的一个发行版,但是这并不阻止他是 Linux 下最好的软件包管理器之一。 - -Portage项目的主要目标是创建一个简单、无故障的包含向后兼容、自动化等功能的包管理系统。 - -如果希望理解的更清晰,可以看下: [Portage project page][10]. - -### 结束语 - -正如我在开始时提到的,这个指南的主要意图是给 Linux 用户提供一个最好的包管理器的列表,但知道如何使用它们可以通过下面提供的重要的链接和尝试测试它们。 - -不同发行版的用户将学习超出他们本身的来理解上述提到的不同的包管理器。 - --------------------------------------------------------------------------------- - -via: http://www.tecmint.com/linux-package-managers/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+tecmint+%28Tecmint%3A+Linux+Howto%27s+Guide%29 - -作者:[Ravi Saive][a] -译者:[Bestony](https://github.com/bestony) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.tecmint.com/author/admin/ -[1]: http://www.tecmint.com/dpkg-command-examples/ -[2]: http://www.tecmint.com/apt-advanced-package-command-examples-in-ubuntu/ -[3]: http://www.tecmint.com/useful-basic-commands-of-apt-get-and-apt-cache-for-package-management/ -[4]: http://www.tecmint.com/difference-between-apt-and-aptitude/ -[5]: http://www.tecmint.com/20-linux-yum-yellowdog-updater-modified-commands-for-package-mangement/ -[6]: http://www.tecmint.com/dnf-next-generation-package-management-utility-for-linux/ -[7]: http://www.tecmint.com/dnf-commands-for-fedora-rpm-package-management/ -[8]: https://wiki.archlinux.org/index.php/Pacman -[9]: http://www.tecmint.com/zypper-commands-to-manage-suse-linux-package-management/ -[10]: https://wiki.gentoo.org/wiki/Project:Portage From 9b9fcf055c5f9a3335f9cd542bf1c5726b4e644f Mon Sep 17 00:00:00 2001 From: Stdio A Date: Sat, 10 Sep 2016 16:34:36 +0800 Subject: [PATCH 0091/2437] Finish translating 20160809 Part 3 - Let's Build A Web Server.md --- ...60809 Part 3 - Let’s Build A Web Server.md | 1014 ---------------- ...60809 Part 3 - Let’s Build A Web Server.md | 1024 +++++++++++++++++ 2 files changed, 1024 insertions(+), 1014 deletions(-) delete mode 100644 sources/tech/20160809 Part 3 - Let’s Build A Web Server.md create mode 100644 translated/tech/20160809 Part 3 - Let’s Build A Web Server.md diff --git a/sources/tech/20160809 Part 3 - Let’s Build A Web Server.md b/sources/tech/20160809 Part 3 - Let’s Build A Web Server.md deleted file mode 100644 index 4301ab8afb..0000000000 --- a/sources/tech/20160809 Part 3 - Let’s Build A Web Server.md +++ /dev/null @@ -1,1014 +0,0 @@ -translating by StdioA - -Part 3 - Let’s Build A Web Server -===================================== - ->“We learn most when we have to invent” —Piaget - -In Part 2 you created a minimalistic WSGI server that could handle basic HTTP GET requests. And I asked you a question, “How can you make your server handle more than one request at a time?” In this article you will find the answer. So, buckle up and shift into high gear. You’re about to have a really fast ride. Have your Linux, Mac OS X (or any *nix system) and Python ready. All source code from the article is available on [GitHub][1]. - -First let’s remember what a very basic Web server looks like and what the server needs to do to service client requests. The server you created in Part 1 and Part 2 is an iterative server that handles one client request at a time. It cannot accept a new connection until after it has finished processing a current client request. Some clients might be unhappy with it because they will have to wait in line, and for busy servers the line might be too long. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it1.png) - -Here is the code of the iterative server [webserver3a.py][2]: - -``` -##################################################################### -# Iterative server - webserver3a.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -##################################################################### -import socket - -SERVER_ADDRESS = (HOST, PORT) = '', 8888 -REQUEST_QUEUE_SIZE = 5 - - -def handle_request(client_connection): - request = client_connection.recv(1024) - print(request.decode()) - http_response = b"""\ -HTTP/1.1 200 OK - -Hello, World! -""" - client_connection.sendall(http_response) - - -def serve_forever(): - listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - listen_socket.bind(SERVER_ADDRESS) - listen_socket.listen(REQUEST_QUEUE_SIZE) - print('Serving HTTP on port {port} ...'.format(port=PORT)) - - while True: - client_connection, client_address = listen_socket.accept() - handle_request(client_connection) - client_connection.close() - -if __name__ == '__main__': - serve_forever() -``` - -To observe your server handling only one client request at a time, modify the server a little bit and add a 60 second delay after sending a response to a client. The change is only one line to tell the server process to sleep for 60 seconds. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it2.png) - -And here is the code of the sleeping server [webserver3b.py][3]: - -``` -######################################################################### -# Iterative server - webserver3b.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -# # -# - Server sleeps for 60 seconds after sending a response to a client # -######################################################################### -import socket -import time - -SERVER_ADDRESS = (HOST, PORT) = '', 8888 -REQUEST_QUEUE_SIZE = 5 - - -def handle_request(client_connection): - request = client_connection.recv(1024) - print(request.decode()) - http_response = b"""\ -HTTP/1.1 200 OK - -Hello, World! -""" - client_connection.sendall(http_response) - time.sleep(60) # sleep and block the process for 60 seconds - - -def serve_forever(): - listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - listen_socket.bind(SERVER_ADDRESS) - listen_socket.listen(REQUEST_QUEUE_SIZE) - print('Serving HTTP on port {port} ...'.format(port=PORT)) - - while True: - client_connection, client_address = listen_socket.accept() - handle_request(client_connection) - client_connection.close() - -if __name__ == '__main__': - serve_forever() -``` - -Start the server with: - -``` -$ python webserver3b.py -``` - -Now open up a new terminal window and run the curl command. You should instantly see the “Hello, World!” string printed on the screen: - -``` -$ curl http://localhost:8888/hello -Hello, World! -``` - -And without delay open up a second terminal window and run the same curl command: - - -``` -$ curl http://localhost:8888/hello -``` - -If you’ve done that within 60 seconds then the second curl should not produce any output right away and should just hang there. The server shouldn’t print a new request body on its standard output either. Here is how it looks like on my Mac (the window at the bottom right corner highlighted in yellow shows the second curl command hanging, waiting for the connection to be accepted by the server): - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it3.png) - -After you’ve waited long enough (more than 60 seconds) you should see the first curl terminate and the second curl print “Hello, World!” on the screen, then hang for 60 seconds, and then terminate: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it4.png) - -The way it works is that the server finishes servicing the first curl client request and then it starts handling the second request only after it sleeps for 60 seconds. It all happens sequentially, or iteratively, one step, or in our case one client request, at a time. - -Let’s talk about the communication between clients and servers for a bit. In order for two programs to communicate with each other over a network, they have to use sockets. And you saw sockets both in Part 1 and Part 2. But what is a socket? - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_socket.png) - -A socket is an abstraction of a communication endpoint and it allows your program to communicate with another program using file descriptors. In this article I’ll be talking specifically about TCP/IP sockets on Linux/Mac OS X. An important notion to understand is the TCP socket pair. - ->The socket pair for a TCP connection is a 4-tuple that identifies two endpoints of the TCP connection: the local IP address, local port, foreign IP address, and foreign port. A socket pair uniquely identifies every TCP connection on a network. The two values that identify each endpoint, an IP address and a port number, are often called a socket.[1 -][4] - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_socketpair.png) - -So, the tuple {10.10.10.2:49152, 12.12.12.3:8888} is a socket pair that uniquely identifies two endpoints of the TCP connection on the client and the tuple {12.12.12.3:8888, 10.10.10.2:49152} is a socket pair that uniquely identifies the same two endpoints of the TCP connection on the server. The two values that identify the server endpoint of the TCP connection, the IP address 12.12.12.3 and the port 8888, are referred to as a socket in this case (the same applies to the client endpoint). - -The standard sequence a server usually goes through to create a socket and start accepting client connections is the following: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_server_socket_sequence.png) - -1. The server creates a TCP/IP socket. This is done with the following statement in Python: - -``` -listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) -``` - -2. The server might set some socket options (this is optional, but you can see that the server code above does just that to be able to re-use the same address over and over again if you decide to kill and re-start the server right away). - -``` -listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) -``` - -3. Then, the server binds the address. The bind function assigns a local protocol address to the socket. With TCP, calling bind lets you specify a port number, an IP address, both, or neither.[1][4] - -``` -listen_socket.bind(SERVER_ADDRESS) -``` - -4. Then, the server makes the socket a listening socket - -``` -listen_socket.listen(REQUEST_QUEUE_SIZE) -``` - -The listen method is only called by servers. It tells the kernel that it should accept incoming connection requests for this socket. - -After that’s done, the server starts accepting client connections one connection at a time in a loop. When there is a connection available the accept call returns the connected client socket. Then, the server reads the request data from the connected client socket, prints the data on its standard output and sends a message back to the client. Then, the server closes the client connection and it is ready again to accept a new client connection. - -Here is what a client needs to do to communicate with the server over TCP/IP: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_client_socket_sequence.png) - -Here is the sample code for a client to connect to your server, send a request and print the response: - -``` -import socket - - # create a socket and connect to a server - sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - sock.connect(('localhost', 8888)) - - # send and receive some data - sock.sendall(b'test') - data = sock.recv(1024) - print(data.decode()) -``` - -After creating the socket, the client needs to connect to the server. This is done with the connect call: - -``` -sock.connect(('localhost', 8888)) -``` - -The client only needs to provide the remote IP address or host name and the remote port number of a server to connect to. - -You’ve probably noticed that the client doesn’t call bind and accept. The client doesn’t need to call bind because the client doesn’t care about the local IP address and the local port number. The TCP/IP stack within the kernel automatically assigns the local IP address and the local port when the client calls connect. The local port is called an ephemeral port, i.e. a short-lived port. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_ephemeral_port.png) - -A port on a server that identifies a well-known service that a client connects to is called a well-known port (for example, 80 for HTTP and 22 for SSH). Fire up your Python shell and make a client connection to the server you run on localhost and see what ephemeral port the kernel assigns to the socket you’ve created (start the server webserver3a.py or webserver3b.py before trying the following example): - -``` ->>> import socket ->>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ->>> sock.connect(('localhost', 8888)) ->>> host, port = sock.getsockname()[:2] ->>> host, port -('127.0.0.1', 60589) -``` - -In the case above the kernel assigned the ephemeral port 60589 to the socket. - -There are some other important concepts that I need to cover quickly before I get to answer the question from Part 2. You will see shortly why this is important. The two concepts are that of a process and a file descriptor. - -What is a process? A process is just an instance of an executing program. When the server code is executed, for example, it’s loaded into memory and an instance of that executing program is called a process. The kernel records a bunch of information about the process - its process ID would be one example - to keep track of it. When you run your iterative server webserver3a.py or webserver3b.py you run just one process. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_server_process.png) - -Start the server webserver3b.py in a terminal window: - -``` -$ python webserver3b.py -``` - -And in a different terminal window use the ps command to get the information about that process: - -``` -$ ps | grep webserver3b | grep -v grep -7182 ttys003 0:00.04 python webserver3b.py -``` - -The ps command shows you that you have indeed run just one Python process webserver3b. When a process gets created the kernel assigns a process ID to it, PID. In UNIX, every user process also has a parent that, in turn, has its own process ID called parent process ID, or PPID for short. I assume that you run a BASH shell by default and when you start the server, a new process gets created with a PID and its parent PID is set to the PID of the BASH shell. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_ppid_pid.png) - -Try it out and see for yourself how it all works. Fire up your Python shell again, which will create a new process, and then get the PID of the Python shell process and the parent PID (the PID of your BASH shell) using os.getpid() and os.getppid() system calls. Then, in another terminal window run ps command and grep for the PPID (parent process ID, which in my case is 3148). In the screenshot below you can see an example of a parent-child relationship between my child Python shell process and the parent BASH shell process on my Mac OS X: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_pid_ppid_screenshot.png) - -Another important concept to know is that of a file descriptor. So what is a file descriptor? A file descriptor is a non-negative integer that the kernel returns to a process when it opens an existing file, creates a new file or when it creates a new socket. You’ve probably heard that in UNIX everything is a file. The kernel refers to the open files of a process by a file descriptor. When you need to read or write a file you identify it with the file descriptor. Python gives you high-level objects to deal with files (and sockets) and you don’t have to use file descriptors directly to identify a file but, under the hood, that’s how files and sockets are identified in UNIX: by their integer file descriptors. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_process_descriptors.png) - -By default, UNIX shells assign file descriptor 0 to the standard input of a process, file descriptor 1 to the standard output of the process and file descriptor 2 to the standard error. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_default_descriptors.png) - -As I mentioned before, even though Python gives you a high-level file or file-like object to work with, you can always use the fileno() method on the object to get the file descriptor associated with the file. Back to your Python shell to see how you can do that: - - -``` ->>> import sys ->>> sys.stdin -', mode 'r' at 0x102beb0c0> ->>> sys.stdin.fileno() -0 ->>> sys.stdout.fileno() -1 ->>> sys.stderr.fileno() -2 -``` - -And while working with files and sockets in Python, you’ll usually be using a high-level file/socket object, but there may be times where you need to use a file descriptor directly. Here is an example of how you can write a string to the standard output using a write system call that takes a file descriptor integer as a parameter: - -``` ->>> import sys ->>> import os ->>> res = os.write(sys.stdout.fileno(), 'hello\n') -hello -``` - -And here is an interesting part - which should not be surprising to you anymore because you already know that everything is a file in Unix - your socket also has a file descriptor associated with it. Again, when you create a socket in Python you get back an object and not a non-negative integer, but you can always get direct access to the integer file descriptor of the socket with the fileno() method that I mentioned earlier. - -``` ->>> import socket ->>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ->>> sock.fileno() -3 -``` -One more thing I wanted to mention: have you noticed that in the second example of the iterative server webserver3b.py, when the server process was sleeping for 60 seconds you could still connect to the server with the second curl command? Sure, the curl didn’t output anything right away and it was just hanging out there but how come the server was not accept ing a connection at the time and the client was not rejected right away, but instead was able to connect to the server? The answer to that is the listen method of a socket object and its BACKLOG argument, which I called REQUEST_QUEUE_SIZE in the code. The BACKLOG argument determines the size of a queue within the kernel for incoming connection requests. When the server webserver3b.py was sleeping, the second curl command that you ran was able to connect to the server because the kernel had enough space available in the incoming connection request queue for the server socket. - -While increasing the BACKLOG argument does not magically turn your server into a server that can handle multiple client requests at a time, it is important to have a fairly large backlog parameter for busy servers so that the accept call would not have to wait for a new connection to be established but could grab the new connection off the queue right away and start processing a client request without delay. - -Whoo-hoo! You’ve covered a lot of ground. Let’s quickly recap what you’ve learned (or refreshed if it’s all basics to you) so far. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) - -- Iterative server -- Server socket creation sequence (socket, bind, listen, accept) -- Client connection creation sequence (socket, connect) -- Socket pair -- Socket -- Ephemeral port and well-known port -- Process -- Process ID (PID), parent process ID (PPID), and the parent-child relationship. -- File descriptors -- The meaning of the BACKLOG argument of the listen socket method - -Now I am ready to answer the question from Part 2: “How can you make your server handle more than one request at a time?” Or put another way, “How do you write a concurrent server?” - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_service_clients.png) - -The simplest way to write a concurrent server under Unix is to use a fork() system call. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_fork.png) - -Here is the code of your new shiny concurrent server webserver3c.py that can handle multiple client requests at the same time (as in our iterative server example webserver3b.py, every child process sleeps for 60 secs): - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it2.png) - -``` -########################################################################### -# Concurrent server - webserver3c.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -# # -# - Child process sleeps for 60 seconds after handling a client's request # -# - Parent and child processes close duplicate descriptors # -# # -########################################################################### -import os -import socket -import time - -SERVER_ADDRESS = (HOST, PORT) = '', 8888 -REQUEST_QUEUE_SIZE = 5 - - -def handle_request(client_connection): - request = client_connection.recv(1024) - print( - 'Child PID: {pid}. Parent PID {ppid}'.format( - pid=os.getpid(), - ppid=os.getppid(), - ) - ) - print(request.decode()) - http_response = b"""\ -HTTP/1.1 200 OK - -Hello, World! -""" - client_connection.sendall(http_response) - time.sleep(60) - - -def serve_forever(): - listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - listen_socket.bind(SERVER_ADDRESS) - listen_socket.listen(REQUEST_QUEUE_SIZE) - print('Serving HTTP on port {port} ...'.format(port=PORT)) - print('Parent PID (PPID): {pid}\n'.format(pid=os.getpid())) - - while True: - client_connection, client_address = listen_socket.accept() - pid = os.fork() - if pid == 0: # child - listen_socket.close() # close child copy - handle_request(client_connection) - client_connection.close() - os._exit(0) # child exits here - else: # parent - client_connection.close() # close parent copy and loop over - -if __name__ == '__main__': - serve_forever() -``` - -Before diving in and discussing how fork works, try it, and see for yourself that the server can indeed handle multiple client requests at the same time, unlike its iterative counterparts webserver3a.py and webserver3b.py. Start the server on the command line with: - -``` -$ python webserver3c.py -``` - -And try the same two curl commands you’ve tried before with the iterative server and see for yourself that, now, even though the server child process sleeps for 60 seconds after serving a client request, it doesn’t affect other clients because they are served by different and completely independent processes. You should see your curl commands output “Hello, World!” instantly and then hang for 60 secs. You can keep on running as many curl commands as you want (well, almost as many as you want :) and all of them will output the server’s response “Hello, World” immediately and without any noticeable delay. Try it. - -The most important point to understand about fork() is that you call fork once but it returns twice: once in the parent process and once in the child process. When you fork a new process the process ID returned to the child process is 0. When the fork returns in the parent process it returns the child’s PID. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_how_fork_works.png) - -I still remember how fascinated I was by fork when I first read about it and tried it. It looked like magic to me. Here I was reading a sequential code and then “boom!”: the code cloned itself and now there were two instances of the same code running concurrently. I thought it was nothing short of magic, seriously. - -When a parent forks a new child, the child process gets a copy of the parent’s file descriptors: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_shared_descriptors.png) - -You’ve probably noticed that the parent process in the code above closed the client connection: - -``` -else: # parent - client_connection.close() # close parent copy and loop over -``` - -So how come a child process is still able to read the data from a client socket if its parent closed the very same socket? The answer is in the picture above. The kernel uses descriptor reference counts to decide whether to close a socket or not. It closes the socket only when its descriptor reference count becomes 0. When your server creates a child process, the child gets the copy of the parent’s file descriptors and the kernel increments the reference counts for those descriptors. In the case of one parent and one child, the descriptor reference count would be 2 for the client socket and when the parent process in the code above closes the client connection socket, it merely decrements its reference count which becomes 1, not small enough to cause the kernel to close the socket. The child process also closes the duplicate copy of the parent’s listen_socket because the child doesn’t care about accepting new client connections, it cares only about processing requests from the established client connection: - -``` -listen_socket.close() # close child copy -``` - -I’ll talk about what happens if you do not close duplicate descriptors later in the article. - -As you can see from the source code of your concurrent server, the sole role of the server parent process now is to accept a new client connection, fork a new child process to handle that client request, and loop over to accept another client connection, and nothing more. The server parent process does not process client requests - its children do. - -A little aside. What does it mean when we say that two events are concurrent? - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_concurrent_events.png) - -When we say that two events are concurrent we usually mean that they happen at the same time. As a shorthand that definition is fine, but you should remember the strict definition: - ->Two events are concurrent if you cannot tell by looking at the program which will happen first.[2][5] - -Again, it’s time to recap the main ideas and concepts you’ve covered so far. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) - -- The simplest way to write a concurrent server in Unix is to - use the fork() system call -- When a process forks a new process it becomes a parent process to that newly forked child process. -- Parent and child share the same file descriptors after the call to fork. -- The kernel uses descriptor reference counts to decide whether to close the file/socket or not -- The role of a server parent process: all it does now is accept a new connection from a client, fork a child to handle the client request, and loop over to accept a new client connection. - -Let’s see what is going to happen if you don’t close duplicate socket descriptors in the parent and child processes. Here is a modified version of the concurrent server where the server does not close duplicate descriptors, webserver3d.py: - -``` -########################################################################### -# Concurrent server - webserver3d.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -########################################################################### -import os -import socket - -SERVER_ADDRESS = (HOST, PORT) = '', 8888 -REQUEST_QUEUE_SIZE = 5 - - -def handle_request(client_connection): - request = client_connection.recv(1024) - http_response = b"""\ -HTTP/1.1 200 OK - -Hello, World! -""" - client_connection.sendall(http_response) - - -def serve_forever(): - listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - listen_socket.bind(SERVER_ADDRESS) - listen_socket.listen(REQUEST_QUEUE_SIZE) - print('Serving HTTP on port {port} ...'.format(port=PORT)) - - clients = [] - while True: - client_connection, client_address = listen_socket.accept() - # store the reference otherwise it's garbage collected - # on the next loop run - clients.append(client_connection) - pid = os.fork() - if pid == 0: # child - listen_socket.close() # close child copy - handle_request(client_connection) - client_connection.close() - os._exit(0) # child exits here - else: # parent - # client_connection.close() - print(len(clients)) - -if __name__ == '__main__': - serve_forever() -``` - -Start the server with: - -``` -$ python webserver3d.py -``` - -Use curl to connect to the server: - -``` -$ curl http://localhost:8888/hello -Hello, World! -``` - -Okay, the curl printed the response from the concurrent server but it did not terminate and kept hanging. What is happening here? The server no longer sleeps for 60 seconds: its child process actively handles a client request, closes the client connection and exits, but the client curl still does not terminate. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_child_is_active.png) - -So why does the curl not terminate? The reason is the duplicate file descriptors. When the child process closed the client connection, the kernel decremented the reference count of that client socket and the count became 1. The server child process exited, but the client socket was not closed by the kernel because the reference count for that socket descriptor was not 0, and, as a result, the termination packet (called FIN in TCP/IP parlance) was not sent to the client and the client stayed on the line, so to speak. There is also another problem. If your long-running server doesn’t close duplicate file descriptors, it will eventually run out of available file descriptors: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_out_of_descriptors.png) - -Stop your server webserver3d.py with Control-C and check out the default resources available to your server process set up by your shell with the shell built-in command ulimit: - -``` -$ ulimit -a -core file size (blocks, -c) 0 -data seg size (kbytes, -d) unlimited -scheduling priority (-e) 0 -file size (blocks, -f) unlimited -pending signals (-i) 3842 -max locked memory (kbytes, -l) 64 -max memory size (kbytes, -m) unlimited -open files (-n) 1024 -pipe size (512 bytes, -p) 8 -POSIX message queues (bytes, -q) 819200 -real-time priority (-r) 0 -stack size (kbytes, -s) 8192 -cpu time (seconds, -t) unlimited -max user processes (-u) 3842 -virtual memory (kbytes, -v) unlimited -file locks (-x) unlimited -``` - -As you can see above, the maximum number of open file descriptors (open files) available to the server process on my Ubuntu box is 1024. - -Now let’s see how your server can run out of available file descriptors if it doesn’t close duplicate descriptors. In an existing or new terminal window, set the maximum number of open file descriptors for your server to be 256: - -``` -$ ulimit -n 256 -``` - -Start the server webserver3d.py in the same terminal where you’ve just run the $ ulimit -n 256 command: - -``` -$ python webserver3d.py -``` - -and use the following client client3.py to test the server. - -``` -##################################################################### -# Test client - client3.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -##################################################################### -import argparse -import errno -import os -import socket - - -SERVER_ADDRESS = 'localhost', 8888 -REQUEST = b"""\ -GET /hello HTTP/1.1 -Host: localhost:8888 - -""" - - -def main(max_clients, max_conns): - socks = [] - for client_num in range(max_clients): - pid = os.fork() - if pid == 0: - for connection_num in range(max_conns): - sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - sock.connect(SERVER_ADDRESS) - sock.sendall(REQUEST) - socks.append(sock) - print(connection_num) - os._exit(0) - - -if __name__ == '__main__': - parser = argparse.ArgumentParser( - description='Test client for LSBAWS.', - formatter_class=argparse.ArgumentDefaultsHelpFormatter, - ) - parser.add_argument( - '--max-conns', - type=int, - default=1024, - help='Maximum number of connections per client.' - ) - parser.add_argument( - '--max-clients', - type=int, - default=1, - help='Maximum number of clients.' - ) - args = parser.parse_args() - main(args.max_clients, args.max_conns) -``` - -In a new terminal window, start the client3.py and tell it to create 300 simultaneous connections to the server: - -``` -$ python client3.py --max-clients=300 -``` - -Soon enough your server will explode. Here is a screenshot of the exception on my box: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_too_many_fds_exc.png) - -The lesson is clear - your server should close duplicate descriptors. But even if you close duplicate descriptors, you are not out of the woods yet because there is another problem with your server, and that problem is zombies! - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_zombies.png) - -Yes, your server code actually creates zombies. Let’s see how. Start up your server again: - -``` -$ python webserver3d.py -``` - -Run the following curl command in another terminal window: - -``` -$ curl http://localhost:8888/hello -``` - -And now run the ps command to show running Python processes. This the example of ps output on my Ubuntu box: - -``` -$ ps auxw | grep -i python | grep -v grep -vagrant 9099 0.0 1.2 31804 6256 pts/0 S+ 16:33 0:00 python webserver3d.py -vagrant 9102 0.0 0.0 0 0 pts/0 Z+ 16:33 0:00 [python] -``` - -Do you see the second line above where it says the status of the process with PID 9102 is Z+ and the name of the process is ? That’s our zombie there. The problem with zombies is that you can’t kill them. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_kill_zombie.png) - -Even if you try to kill zombies with $ kill -9 , they will survive. Try it and see for yourself. - -What is a zombie anyway and why does our server create them? A zombie is a process that has terminated, but its parent has not waited for it and has not received its termination status yet. When a child process exits before its parent, the kernel turns the child process into a zombie and stores some information about the process for its parent process to retrieve later. The information stored is usually the process ID, the process termination status, and the resource usage by the process. Okay, so zombies serve a purpose, but if your server doesn’t take care of these zombies your system will get clogged up. Let’s see how that happens. First stop your running server and, in a new terminal window, use the ulimit command to set the max user processess to 400(make sure to set open files to a high number, let’s say 500 too): - -``` -$ ulimit -u 400 -$ ulimit -n 500 -``` - -Start the server webserver3d.py in the same terminal where you’ve just run the $ ulimit -u 400 command: - -``` -$ python webserver3d.py -``` - -In a new terminal window, start the client3.py and tell it to create 500 simultaneous connections to the server: - -``` -$ python client3.py --max-clients=500 -``` - -And, again, soon enough your server will blow up with an OSError: Resource temporarily unavailable exception when it tries to create a new child process, but it can’t because it has reached the limit for the maximum number of child processes it’s allowed to create. Here is a screenshot of the exception on my box: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_resource_unavailable.png) - -As you can see, zombies create problems for your long-running server if it doesn’t take care of them. I will discuss shortly how the server should deal with that zombie problem. - -Let’s recap the main points you’ve covered so far: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) - -- If you don’t close duplicate descriptors, the clients won’t terminate because the client connections won’t get closed. -- If you don’t close duplicate descriptors, your long-running server will eventually run out of available file descriptors (max open files). -- When you fork a child process and it exits and the parent process doesn’t wait for it and doesn’t collect its termination status, it becomes a zombie. -- Zombies need to eat something and, in our case, it’s memory. Your server will eventually run out of available processes (max user processes) if it doesn’t take care of zombies. -- You can’t kill a zombie, you need to wait for it. - -So what do you need to do to take care of zombies? You need to modify your server code to wait for zombies to get their termination status. You can do that by modifying your server to call a wait system call. Unfortunately, that’s far from ideal because if you call wait and there is no terminated child process the call to wait will block your server, effectively preventing your server from handling new client connection requests. Are there any other options? Yes, there are, and one of them is the combination of a signal handler with the wait system call. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_signaling.png) - -Here is how it works. When a child process exits, the kernel sends a SIGCHLD signal. The parent process can set up a signal handler to be asynchronously notified of that SIGCHLD event and then it can wait for the child to collect its termination status, thus preventing the zombie process from being left around. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part_conc4_sigchld_async.png) - -By the way, an asynchronous event means that the parent process doesn’t know ahead of time that the event is going to happen. - -Modify your server code to set up a SIGCHLD event handler and wait for a terminated child in the event handler. The code is available in webserver3e.py file: - -``` -########################################################################### -# Concurrent server - webserver3e.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -########################################################################### -import os -import signal -import socket -import time - -SERVER_ADDRESS = (HOST, PORT) = '', 8888 -REQUEST_QUEUE_SIZE = 5 - - -def grim_reaper(signum, frame): - pid, status = os.wait() - print( - 'Child {pid} terminated with status {status}' - '\n'.format(pid=pid, status=status) - ) - - -def handle_request(client_connection): - request = client_connection.recv(1024) - print(request.decode()) - http_response = b"""\ -HTTP/1.1 200 OK - -Hello, World! -""" - client_connection.sendall(http_response) - # sleep to allow the parent to loop over to 'accept' and block there - time.sleep(3) - - -def serve_forever(): - listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - listen_socket.bind(SERVER_ADDRESS) - listen_socket.listen(REQUEST_QUEUE_SIZE) - print('Serving HTTP on port {port} ...'.format(port=PORT)) - - signal.signal(signal.SIGCHLD, grim_reaper) - - while True: - client_connection, client_address = listen_socket.accept() - pid = os.fork() - if pid == 0: # child - listen_socket.close() # close child copy - handle_request(client_connection) - client_connection.close() - os._exit(0) - else: # parent - client_connection.close() - -if __name__ == '__main__': - serve_forever() -``` - -Start the server: - -``` -$ python webserver3e.py -``` - -Use your old friend curl to send a request to the modified concurrent server: - -``` -$ curl http://localhost:8888/hello -``` - -Look at the server: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_eintr.png) - -What just happened? The call to accept failed with the error EINTR. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_eintr_error.png) - -The parent process was blocked in accept call when the child process exited which caused SIGCHLD event, which in turn activated the signal handler and when the signal handler finished the accept system call got interrupted: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_eintr_accept.png) - -Don’t worry, it’s a pretty simple problem to solve, though. All you need to do is to re-start the accept system call. Here is the modified version of the server webserver3f.py that handles that problem: - - -``` -########################################################################### -# Concurrent server - webserver3f.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -########################################################################### -import errno -import os -import signal -import socket - -SERVER_ADDRESS = (HOST, PORT) = '', 8888 -REQUEST_QUEUE_SIZE = 1024 - - -def grim_reaper(signum, frame): - pid, status = os.wait() - - -def handle_request(client_connection): - request = client_connection.recv(1024) - print(request.decode()) - http_response = b"""\ -HTTP/1.1 200 OK - -Hello, World! -""" - client_connection.sendall(http_response) - - -def serve_forever(): - listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - listen_socket.bind(SERVER_ADDRESS) - listen_socket.listen(REQUEST_QUEUE_SIZE) - print('Serving HTTP on port {port} ...'.format(port=PORT)) - - signal.signal(signal.SIGCHLD, grim_reaper) - - while True: - try: - client_connection, client_address = listen_socket.accept() - except IOError as e: - code, msg = e.args - # restart 'accept' if it was interrupted - if code == errno.EINTR: - continue - else: - raise - - pid = os.fork() - if pid == 0: # child - listen_socket.close() # close child copy - handle_request(client_connection) - client_connection.close() - os._exit(0) - else: # parent - client_connection.close() # close parent copy and loop over - - -if __name__ == '__main__': - serve_forever() -``` - -Start the updated server webserver3f.py: - -``` -$ python webserver3f.py -``` - -Use curl to send a request to the modified concurrent server: - -``` -$ curl http://localhost:8888/hello -``` - -See? No EINTR exceptions any more. Now, verify that there are no more zombies either and that your SIGCHLD event handler with wait call took care of terminated children. To do that, just run the ps command and see for yourself that there are no more Python processes with Z+ status (no more processes). Great! It feels safe without zombies running around. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) - -- If you fork a child and don’t wait for it, it becomes a zombie. -- Use the SIGCHLD event handler to asynchronously wait for a terminated child to get its termination status -- When using an event handler you need to keep in mind that system calls might get interrupted and you need to be prepared for that scenario - -Okay, so far so good. No problems, right? Well, almost. Try your webserver3f.py again, but instead of making one request with curl use client3.py to create 128 simultaneous connections: - -``` -$ python client3.py --max-clients 128 -``` - -Now run the ps command again - -``` -$ ps auxw | grep -i python | grep -v grep -``` - -and see that, oh boy, zombies are back again! - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc5_zombies_again.png) - -What went wrong this time? When you ran 128 simultaneous clients and established 128 connections, the child processes on the server handled the requests and exited almost at the same time causing a flood of SIGCHLD signals being sent to the parent process. The problem is that the signals are not queued and your server process missed several signals, which left several zombies running around unattended: - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc5_signals_not_queued.png) - -The solution to the problem is to set up a SIGCHLD event handler but instead of wait use a waitpid system call with a WNOHANG option in a loop to make sure that all terminated child processes are taken care of. Here is the modified server code, webserver3g.py: - -``` -########################################################################### -# Concurrent server - webserver3g.py # -# # -# Tested with Python 2.7.9 & Python 3.4 on Ubuntu 14.04 & Mac OS X # -########################################################################### -import errno -import os -import signal -import socket - -SERVER_ADDRESS = (HOST, PORT) = '', 8888 -REQUEST_QUEUE_SIZE = 1024 - - -def grim_reaper(signum, frame): - while True: - try: - pid, status = os.waitpid( - -1, # Wait for any child process - os.WNOHANG # Do not block and return EWOULDBLOCK error - ) - except OSError: - return - - if pid == 0: # no more zombies - return - - -def handle_request(client_connection): - request = client_connection.recv(1024) - print(request.decode()) - http_response = b"""\ -HTTP/1.1 200 OK - -Hello, World! -""" - client_connection.sendall(http_response) - - -def serve_forever(): - listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - listen_socket.bind(SERVER_ADDRESS) - listen_socket.listen(REQUEST_QUEUE_SIZE) - print('Serving HTTP on port {port} ...'.format(port=PORT)) - - signal.signal(signal.SIGCHLD, grim_reaper) - - while True: - try: - client_connection, client_address = listen_socket.accept() - except IOError as e: - code, msg = e.args - # restart 'accept' if it was interrupted - if code == errno.EINTR: - continue - else: - raise - - pid = os.fork() - if pid == 0: # child - listen_socket.close() # close child copy - handle_request(client_connection) - client_connection.close() - os._exit(0) - else: # parent - client_connection.close() # close parent copy and loop over - -if __name__ == '__main__': - serve_forever() -``` - -Start the server: - -``` -$ python webserver3g.py -``` - -Use the test client client3.py: - -``` -$ python client3.py --max-clients 128 -``` - -And now verify that there are no more zombies. Yay! Life is good without zombies :) - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc5_no_zombies.png) - -Congratulations! It’s been a pretty long journey but I hope you liked it. Now you have your own simple concurrent server and the code can serve as a foundation for your further work towards a production grade Web server. - -I’ll leave it as an exercise for you to update the WSGI server from Part 2 and make it concurrent. You can find the modified version here. But look at my code only after you’ve implemented your own version. You have all the necessary information to do that. So go and just do it :) - -What’s next? As Josh Billings said, - ->“Be like a postage stamp — stick to one thing until you get there.” - -Start mastering the basics. Question what you already know. And always dig deeper. - -![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_dig_deeper.png) - ->“If you learn only methods, you’ll be tied to your methods. But if you learn principles, you can devise your own methods.” —Ralph Waldo Emerson - -Below is a list of books that I’ve drawn on for most of the material in this article. They will help you broaden and deepen your knowledge about the topics I’ve covered. I highly recommend you to get those books somehow: borrow them from your friends, check them out from your local library, or just buy them on Amazon. They are the keepers: - -1. [Unix Network Programming, Volume 1: The Sockets Networking API (3rd Edition)][6] -2. [Advanced Programming in the UNIX Environment, 3rd Edition][7] -3. [The Linux Programming Interface: A Linux and UNIX System Programming Handbook][8] -4. [TCP/IP Illustrated, Volume 1: The Protocols (2nd Edition) (Addison-Wesley Professional Computing Series)][9] -5. [The Little Book of SEMAPHORES (2nd Edition): The Ins and Outs of Concurrency Control and Common Mistakes][10]. Also available for free on the author’s site [here][11]. - -BTW, I’m writing a book “Let’s Build A Web Server: First Steps” that explains how to write a basic web server from scratch and goes into more detail on the topics I just covered. Subscribe to the mailing list to get the latest updates about the book and the release date. - --------------------------------------------------------------------------------- - -via: https://ruslanspivak.com/lsbaws-part3/ - -作者:[Ruslan][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://github.com/rspivak/ - -[1]: https://github.com/rspivak/lsbaws/blob/master/part3/ -[2]: https://github.com/rspivak/lsbaws/blob/master/part3/webserver3a.py -[3]: https://github.com/rspivak/lsbaws/blob/master/part3/webserver3b.py -[4]: https://ruslanspivak.com/lsbaws-part3/#fn:1 -[5]: https://ruslanspivak.com/lsbaws-part3/#fn:2 -[6]: http://www.amazon.com/gp/product/0131411551/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0131411551&linkCode=as2&tag=russblo0b-20&linkId=2F4NYRBND566JJQL -[7]: http://www.amazon.com/gp/product/0321637739/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0321637739&linkCode=as2&tag=russblo0b-20&linkId=3ZYAKB537G6TM22J -[8]: http://www.amazon.com/gp/product/1593272200/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1593272200&linkCode=as2&tag=russblo0b-20&linkId=CHFOMNYXN35I2MON -[9]: http://www.amazon.com/gp/product/0321336313/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0321336313&linkCode=as2&tag=russblo0b-20&linkId=K467DRFYMXJ5RWAY -[10]: http://www.amazon.com/gp/product/1441418687/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1441418687&linkCode=as2&tag=russblo0b-20&linkId=QFOAWARN62OWTWUG -[11]: http://greenteapress.com/semaphores/ diff --git a/translated/tech/20160809 Part 3 - Let’s Build A Web Server.md b/translated/tech/20160809 Part 3 - Let’s Build A Web Server.md new file mode 100644 index 0000000000..41a128a8a7 --- /dev/null +++ b/translated/tech/20160809 Part 3 - Let’s Build A Web Server.md @@ -0,0 +1,1024 @@ +translating by StdioA + +搭个 Web 服务器(三) +===================================== + +>“当我们必须创造时,才能够学到更多。” ——伯爵 + +在本系列的第二部分中,你创造了一个可以处理基本 HTTP GET 请求的、朴素的 WSGI 服务器。当时我问了一个问题:“你该如何让你的服务器在同一时间处理多个请求呢?”在这篇文章中,你会找到答案。系好安全带,我们要认真起来,全速前进了!你将会体验到一段非常快速的旅程。准备好你的 Linux,Mac OS X(或者其他 *nix 系统),还有你的 Python. 本文中所有源代码均可在 [GitHub][1] 上找到。 + +首先,我们来回顾一下 Web 服务器的基本结构,以及服务器处理来自客户端的请求时,所需的必要步骤。你在第一及第二部分中创建的轮询服务器只能够在同一时间内处理一个请求。在处理完当前请求之前,它不能够打开一个新的客户端连接。所有请求为了等待服务都需要排队,在服务繁忙时,这个队伍可能会排的很长,一些客户端可能会感到不开心。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it1.png) + +这是轮询服务器 [webserver3a.py][2] 的代码: + +``` +##################################################################### +# 轮询服务器 - webserver3a.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +##################################################################### +import socket + +SERVER_ADDRESS = (HOST, PORT) = '', 8888 +REQUEST_QUEUE_SIZE = 5 + + +def handle_request(client_connection): + request = client_connection.recv(1024) + print(request.decode()) + http_response = b"""\ +HTTP/1.1 200 OK + +Hello, World! +""" + client_connection.sendall(http_response) + + +def serve_forever(): + listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listen_socket.bind(SERVER_ADDRESS) + listen_socket.listen(REQUEST_QUEUE_SIZE) + print('Serving HTTP on port {port} ...'.format(port=PORT)) + + while True: + client_connection, client_address = listen_socket.accept() + handle_request(client_connection) + client_connection.close() + +if __name__ == '__main__': + serve_forever() +``` + +为了观察到你的服务器在同一时间只能处理一个请求,我们对服务器的代码做一点点修改:在将响应发送至客户端之后,将程序阻塞 60 秒。这个修改只需要一行代码,来告诉服务器进程暂停 60 秒钟。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it2.png) + +这是我们更改后的代码,包含暂停语句的服务器 [webserver3b.py][3]: + +``` +######################################################################### +# 轮询服务器 - webserver3b.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +# # +# - 服务器向客户端发送响应之后,会阻塞 60 秒 # +######################################################################### +import socket +import time + +SERVER_ADDRESS = (HOST, PORT) = '', 8888 +REQUEST_QUEUE_SIZE = 5 + + +def handle_request(client_connection): + request = client_connection.recv(1024) + print(request.decode()) + http_response = b"""\ +HTTP/1.1 200 OK + +Hello, World! +""" + client_connection.sendall(http_response) + time.sleep(60) # 睡眠语句,阻塞该进程 60 秒 + + +def serve_forever(): + listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listen_socket.bind(SERVER_ADDRESS) + listen_socket.listen(REQUEST_QUEUE_SIZE) + print('Serving HTTP on port {port} ...'.format(port=PORT)) + + while True: + client_connection, client_address = listen_socket.accept() + handle_request(client_connection) + client_connection.close() + +if __name__ == '__main__': + serve_forever() +``` + +用以下命令启动服务器: + +``` +$ python webserver3b.py +``` + +现在,打开一个新的命令行窗口,然后运行 `curl` 语句。你应该可以立刻看到屏幕上显示的字符串“Hello, World!”: + +``` +$ curl http://localhost:8888/hello +Hello, World! +``` + +然后,立刻打开第二个命令行窗口,运行相同的 `curl` 命令: + +``` +$ curl http://localhost:8888/hello +``` + +如果你在 60 秒之内完成了以上步骤,你会看到第二条 `curl` 指令不会立刻产生任何输出,而只是挂在了哪里。同样,服务器也不会在标准输出流中输出新的请求内容。这是这个过程在我的 Mac 电脑上的运行结果(在右下角用黄色框标注出来的窗口中,我们能看到第二个 `curl` 指令被挂起,正在等待连接被服务器接受): + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it3.png) + +当你等待足够长的时间(60 秒以上)后,你会看到第一个 `curl` 程序完成,而第二个 `curl` 在屏幕上输出了“Hello, World!”,然后休眠 60 秒,进而停止运行。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it4.png) + +这两个程序这样运行,是因为在服务器在处理完第一个来自 `curl` 的请求之后,只有等待 60 秒才能开始处理第二个请求。这个处理请求的过程按顺序进行(也可以说,迭代进行),一步一步进行,在我们刚刚给出的例子中,在同一时间内只能处理一个请求。 + +现在,我们来简单讨论一下客户端与服务器的交流过程。为了让两个程序在网络中互相交流,它们必须使用套接字。你应当在本系列的前两部分中见过它几次了。但是,套接字是什么? + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_socket.png) + +套接字是一个交互通道的端点的抽象形式,它可以让你的程序通过文件描述符来与其它程序进行交流。在这篇文章中,我只会单独讨论 Linux 或 Mac OS X 中的 TCP/IP 套接字。这里有一个重点概念需要你去理解:TCP 套接字对。 + +> TCP 连接使用的套接字对是一个由 4 个元素组成的元组,它确定了 TCP 连接的两端:本地 IP 地址、本地端口、远端 IP 地址及远端端口。一个套接字对独一无二地确定了网络中的每一个 TCP 连接。在连接一端的两个值:一个 IP 地址和一个端口,通常被称作一个套接字。[1][4] + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_socketpair.png) + +所以,元组 {10.10.10.2:49152, 12.12.12.3:8888} 就是一个能够在客户端确定 TCP 连接两端的套接字对,而元组 {12.12.12.3:8888, 10.10.10.2:49152} 则是在服务端确定 TCP 连接两端的套接字对。在这个例子中,确定 TCP 服务端的两个值(IP 地址 `12.12.12.3` 及端口 `8888`),代表一个套接字;另外两个值则代表客户端的套接字。 + +一个服务器创建一个套接字并开始建立连接的基本工作流程如下: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_server_socket_sequence.png) + +1. 服务器创建一个 TCP/IP 套接字。我们可以用下面那条 Python 语句来创建: + +``` +listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +``` + +2. 服务器可能会设定一些套接字选项(这个步骤是可选的,但是你可以看到上面的服务器代码做了设定,这样才能够在重启服务器时多次复用同一地址)。 + +``` +listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) +``` + +3. 然后,服务器绑定一个地址。绑定函数可以将一个本地协议地址赋给套接字。若使用 TCP 协议,调用绑定函数时,需要指定一个端口号,一个 IP 地址,或两者兼有,或两者兼无。[1][4] + +``` +listen_socket.bind(SERVER_ADDRESS) +``` + +4. 然后,服务器开启套接字的监听模式。 + +``` +listen_socket.listen(REQUEST_QUEUE_SIZE) +``` + +监听函数只应在服务端调用。它会通知操作系统内核,标明它会接受所有向该套接字发送请求的链接。 + +以上四步完成后,服务器将循环接收来自客户端的连接,一次循环处理一条。当有连接可用时,`accept` 函数将会返回一个已连接的客户端套接字。然后,服务器从客户端套接字中读取请求数据,将它在标准输出流中打印出来,并向客户端回送一条消息。然后,服务器会关闭这个客户端连接,并准备接收一个新的客户端连接。 + +这是客户端使用 TCP/IP 协议与服务器通信的必要步骤: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_client_socket_sequence.png) + +下面是一段示例代码,使用这段代码,客户端可以连接你的服务器,发送一个请求,并打印响应内容: + +``` +import socket + +# 创建一个套接字,并连接值服务器 +sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +sock.connect(('localhost', 8888)) + +# 发送一段数据,并接收响应数据 +sock.sendall(b'test') +data = sock.recv(1024) +print(data.decode()) +``` + +在创建套接字后,客户端需要连接至服务器。我们可以调用 `connect` 函数来完成这个操作: + +``` +sock.connect(('localhost', 8888)) +``` + +客户端只需提供待连接服务器的 IP 地址(或主机名),及端口号,即可连接至远端服务器。 + +你可能已经注意到了,客户端不需要调用 `bind` 及 `accept` 函数,就可以与服务器建立连接。客户端不需要调用 `bind` 函数是因为客户端不需要关注本地 IP 地址及端口号。操作系统内核中的 TCP/IP 协议栈会在客户端调用 `connect` 函数时,自动为套接字分配本地 IP 地址及本地端口号。这个本地端口被称为临时端口,也就是一个短暂开放的端口。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_ephemeral_port.png) + +服务器中有一些端口被用于承载一些众所周知的服务,它们被称作通用端口:如 80 端口用于 HTTP 服务,22 端口用于 SSH 服务。打开你的 Python shell,与你在本地运行的服务器建立一个连接,来看看内核给你的客户端套接字分配了哪个临时端口(在尝试这个例子之前,你需要运行服务器程序 `webserver3a.py` 或 `webserver3b.py`): + +``` +>>> import socket +>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +>>> sock.connect(('localhost', 8888)) +>>> host, port = sock.getsockname()[:2] +>>> host, port +('127.0.0.1', 60589) +``` + +在上面的例子中,内核将临时端口 60589 分配给了你的套接字。 + +在我开始回答我在第二部分中提出的问题之前,我还需要快速讲解一些概念。你很快就会明白这些概念为什么非常重要。这两个概念,一个是进程,另外一个是文件描述符。 + +什么是进程?进程就是一个程序执行的实体。举个例子:当你的服务器代码被执行时,它会被载入内存,而内存中表现此次程序运行的实体就叫做进程。内核记录了进程的一系列有关信息——比如进程 ID——来追踪它的运行情况。当你在执行轮询服务器 `webserver3a.py` 或 `webserver3b.py` 时,你只启动了一个进程。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_server_process.png) + +我们在终端窗口中运行 `webserver3b.py`: +Start the server webserver3b.py in a terminal window: + +``` +$ python webserver3b.py +``` + +在另一个终端窗口中,我们可以使用 `ps` 命令获取该进程的相关信息: + +``` +$ ps | grep webserver3b | grep -v grep +7182 ttys003 0:00.04 python webserver3b.py +``` + +`ps` 命令显示,我们刚刚只运行了一个 Python 进程 `webserver3b`。当一个进程被创建时,内核会为其分配一个进程 ID,也就是 PID。在 UNIX 中,所有用户进程都有一个父进程;当然,这个父进程也有进程 ID,叫做父进程 ID,缩写为 PPID。假设你默认使用 BASH shell,那当你启动服务器时,一个新的进程会被启动,同时被赋予一个 PID,而它的父进程 PID 会被设为 BASH shell 的 PID。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_ppid_pid.png) + +自己尝试一下,看看这一切都是如何工作的。重新开启你的 Python shell,它会创建一个新进程,然后在其中使用系统调用 `os.getpid()` 及 `os.getppid()` 来获取 Python shell 进程的 PID 及其父进程 PID(也就是你的 BASH shell 的 PID)。然后,在另一个终端窗口中运行 `ps` 命令,然后用 `grep` 来查找 PPID(父进程 ID,在我的例子中是 3148)。在下面的屏幕截图中,你可以看到一个我的 Mac OS X 系统中关于进程父子关系的例子,在这个例子中,子进程是我的 Python shell 进程,而父进程是 BASH shell 进程: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_pid_ppid_screenshot.png) + +另外一个需要了解的概念,就是文件描述符。什么是文件描述符?文件描述符是一个非负整数,当进程打开一个现有文件、创建新文件或创建一个新的套接字时,内核会将这个数返回给进程。你以前可能听说过,在 UNIX 中,一切皆是文件。内核会根据一个文件描述符来为一个进程打开一个文件。当你需要读取文件或向文件写入时,我们同样通过文件描述符来定位这个文件。Python 提供了高层次的文件(或套接字)对象,所以你不需要直接通过文件描述符来定位文件。但是,在高层对象之下,我们就是用它来在 UNIX 中定位文件及套接字:整形的文件描述符。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_process_descriptors.png) + +一般情况下,UNIX shell 会将一个进程的标准输入流的文件描述符设为 0,标准输出流设为 1,而标准错误打印的文件描述符会被设为 2。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it_default_descriptors.png) + +我之前提到过,即使 Python 提供了高层次的文件对象或类文件对象来供你操作,你仍然可以在对象上使用 `fileno()` 方法,来获取与该文件相关联的文件描述符。回到 Python shell 中,我们来看看你该怎么做到这一点: + + +``` +>>> import sys +>>> sys.stdin +', mode 'r' at 0x102beb0c0> +>>> sys.stdin.fileno() +0 +>>> sys.stdout.fileno() +1 +>>> sys.stderr.fileno() +2 +``` + +当你在 Python 中操作文件及套接字时,你可能会使用高层次的文件/套接字对象,但是你仍然有可能会直接使用文件描述符。下面有一个例子,来演示如何用文件描述符做参数来进行一次写入的系统调用: + +``` +>>> import sys +>>> import os +>>> res = os.write(sys.stdout.fileno(), 'hello\n') +hello +``` + +下面是比较有趣的部分——不过你可能不会为此感到惊讶,因为你已经知道在 Unix 中,一切皆为文件——你的套接字对象同样有一个相关联的文件描述符。和刚才操纵文件时一样,当你在 Python 中创建一个套接字时,你会得到一个对象而不是一个非负整数,但你永远可以用我之前提到过的 `fileno()` 方法获取套接字对象的文件描述符,并可以通过这个文件描述符来直接操纵套接字。 + +``` +>>> import socket +>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +>>> sock.fileno() +3 +``` + +我还想再提一件事:不知道你有没有注意到,在我们的第二个轮询服务器 `webserver3b.py` 中,当你的服务器休眠 60 秒的过程中,你仍然可以通过第二个 `curl` 命令连接至服务器。当然 `curl` 命令并没有立刻输出任何内容而是挂在哪里,但是既然服务器没有接受连接,那它为什么不立即拒绝掉连接,而让它还能够继续与服务器建立连接呢?这个问题的答案是:当我在调用套接字对象的 `listen` 方法时,我为该方法提供了一个 `BACKLOG` 参数,在代码中用 `REQUEST_QUEUE_SIZE` 变量来表示。`BACKLOG` 参数决定了在内核中为存放即将到来的连接请求所创建的队列的大小。当服务器 `webserver3b.py` 被挂起的时候,你运行的第二个 `curl` 命令依然能够连接至服务器,因为内核中用来存放即将接收的连接请求的队列依然拥有足够大的可用空间。 + +尽管增大 `BACKLOG` 参数并不能神奇地使你的服务器同时处理多个请求,但当你的服务器很繁忙时,将它设置为一个较大的值还是相当重要的。这样,在你的服务器调用 `accept` 方法时,不需要再等待一个新的连接建立,而可以立刻直接抓取队列中的第一个客户端连接,并不加停顿地立刻处理它。 + +欧耶!现在你已经了解了一大块内容。我们来快速回顾一下我们刚刚讲解的知识(当然,如果这些对你来说都是基础知识的话,那我们就当复习好啦)。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) + +- 迭代服务器 +- 服务端套接字创建流程(创建套接字,绑定,监听及接受) +- 客户端连接创建流程(创建套接字,连接) +- 套接字对 +- 套接字 +- 临时端口及通用端口 +- 进程 +- 进程 ID(PID),父进程 ID(PPID),以及进程父子关系 +- 文件描述符 +- 套接字的 `listen` 方法中,`BACKLOG` 参数的含义 + +现在,我可以开始回答第二部分中的那个问题了:“你该如何让你的服务器在同一时间处理多个请求呢?”或者换一种说法:“如何编写一个并发服务器?” + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_service_clients.png) + +在 UNIX 系统中编写一个并发服务器最简单的方法,就是使用系统调用 `fork()`。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_fork.png) + +下面是全新出炉的并发服务器 `webserver3c.py` 的代码,它可以同时处理多个请求(和我们之前的例子 `webserver3b.py` 一样,每个子进程都会休眠 60 秒): + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_it2.png) + +``` +####################################################### +# 并发服务器 - webserver3c.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +# # +# - 完成客户端请求处理之后,子进程会休眠 60 秒 # +# - 父子进程会关闭重复的描述符 # +# # +####################################################### +import os +import socket +import time + +SERVER_ADDRESS = (HOST, PORT) = '', 8888 +REQUEST_QUEUE_SIZE = 5 + + +def handle_request(client_connection): + request = client_connection.recv(1024) + print( + 'Child PID: {pid}. Parent PID {ppid}'.format( + pid=os.getpid(), + ppid=os.getppid(), + ) + ) + print(request.decode()) + http_response = b"""\ +HTTP/1.1 200 OK + +Hello, World! +""" + client_connection.sendall(http_response) + time.sleep(60) + + +def serve_forever(): + listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listen_socket.bind(SERVER_ADDRESS) + listen_socket.listen(REQUEST_QUEUE_SIZE) + print('Serving HTTP on port {port} ...'.format(port=PORT)) + print('Parent PID (PPID): {pid}\n'.format(pid=os.getpid())) + + while True: + client_connection, client_address = listen_socket.accept() + pid = os.fork() + if pid == 0: # 子进程 + listen_socket.close() # 关闭子进程中复制的套接字对象 + handle_request(client_connection) + client_connection.close() + os._exit(0) # 子进程在这里退出 + else: # 父进程 + client_connection.close() # 关闭父进程中的客户端连接对象,并循环执行 + +if __name__ == '__main__': + serve_forever() +``` + +在深入研究代码、讨论 `fork` 如何工作之前,先尝试运行它,自己看一看这个服务器是否真的可以同时处理多个客户端请求,而不是像轮询服务器 `webserver3a.py` 和 `webserver3b.py` 一样。在命令行中使用如下命令启动服务器: + +``` +$ python webserver3c.py +``` + +然后,像我们之前测试轮询服务器那样,运行两个 `curl` 命令,来看看这次的效果。现在你可以看到,即使子进程在处理客户端请求后会休眠 60 秒,但它并不会影响其它客户端连接,因为他们都是由完全独立的进程来处理的。你应该看到你的 `curl` 命令立即输出了“Hello, World!”然后挂起 60 秒。你可以按照你的想法运行尽可能多的 `curl` 命令(好吧,并不能运行特别特别多 ^_^),所有的命令都会立刻输出来自服务器的响应“Hello, World!”,并不会出现任何可被察觉到的延迟行为。试试看吧。 + +如果你要理解 `fork()`,那最重要的一点是:你调用了它一次,但是它会返回两次:一次在父进程中,另一次是在子进程中。当你创建了一个新进程,那么 `fork()` 在子进程中的返回值是 0。如果是在父进程中,那 `fork()` 函数会返回子进程的 PID。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_how_fork_works.png) + +我依然记得在第一次看到它并尝试使用 `fork()` 的时候,我是多么的入迷。它在我眼里就像是魔法一样。这就好像我在读一段顺序执行的代码,然后“砰”地一声,代码变成了两份,然后出现了两个实体,同时并行地运行相同的代码。讲真,那个时候我觉得它真的跟魔法一样神奇。 + +当父进程创建出一个新的子进程时,子进程会复制从父进程中复制一份文件描述符: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_shared_descriptors.png) + +你可能注意到,在上面的代码中,父进程关闭了客户端连接: + +``` +else: # parent + client_connection.close() # close parent copy and loop over +``` + +不过,既然父进程关闭了这个套接字,那为什么子进程仍然能够从来自客户端的套接字中读取数据呢?答案就在上面的图片中。内核会使用描述符引用计数器来决定是否要关闭一个套接字。当你的服务器创建一个子进程时,子进程会复制父进程的所有文件描述符,内核中改描述符的引用计数也会增加。如果只有一个父进程及一个子进程,那客户端套接字的文件描述符引用数应为 2;当父进程关闭客户端连接的套接字时,内核只会减少它的引用计数,将其变为 1,但这仍然不会使内核关闭该套接字。子进程也关闭了父进程中 `listen_socket` 的复制实体,因为子进程不需要关注新的客户端连接,而只需要处理已建立的客户端连接中的请求。 + +``` +listen_socket.close() # 关闭子进程中的复制实体 +``` + +我们将会在后文中讨论,如果你不关闭那些重复的描述符,会发生什么。 + +你可以从你的并发服务器源码看到,父进程的主要职责为:接受一个新的客户端连接,复制出一个子进程来处理这个连接,然后继续循环来接受另外的客户端连接,仅此而已。服务器父进程并不会处理客户端连接——子进程才会做这件事。 + +打个岔:当我们说两个事件并发执行时,我们在说什么? +A little aside. What does it mean when we say that two events are concurrent? + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc2_concurrent_events.png) + +当我们说“两个事件并发执行”时,它通常意味着这两个事件同时发生。简单来讲,这个定义没问题,但你应该记住它的严格定义: + +> 如果你阅读代码时,无法判断两个事件的发生顺序,那这两个事件就是并发执行的。[2][5] + +好的,现在你又该回顾一下你刚刚学过的知识点了。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) + +- 在 Unix 中,编写一个并发服务器的最简单的方式——使用 `fork()` 系统调用; +- 当一个进程复制出另一个进程时,它会变成刚刚复制出的进程的父进程; +- 在进行 `fork` 调用后,父进程和子进程共享相同的文件描述符; +- 系统内核通过描述符引用计数来决定是否要关闭该描述符对应的文件或套接字; +- 服务器父进程的主要职责:现在它做的只是从客户端接受一个新的连接,复制出子进程来处理这个客户端连接,然后开始下一轮循环,去接收新的客户端连接。 + +我们来看看,如果我们不在父进程与子进程中关闭重复的套接字描述符会发生什么。下面是刚才的并发服务器代码的修改版本,这段代码(`webserver3d.py` 中,服务器不会关闭重复的描述符): + +``` +####################################################### +# 并发服务器 - webserver3d.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +####################################################### +import os +import socket + +SERVER_ADDRESS = (HOST, PORT) = '', 8888 +REQUEST_QUEUE_SIZE = 5 + + +def handle_request(client_connection): + request = client_connection.recv(1024) + http_response = b"""\ +HTTP/1.1 200 OK + +Hello, World! +""" + client_connection.sendall(http_response) + + +def serve_forever(): + listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listen_socket.bind(SERVER_ADDRESS) + listen_socket.listen(REQUEST_QUEUE_SIZE) + print('Serving HTTP on port {port} ...'.format(port=PORT)) + + clients = [] + while True: + client_connection, client_address = listen_socket.accept() + # 将引用存储起来,否则在下一轮循环时,他们会被垃圾回收机制销毁 + clients.append(client_connection) + pid = os.fork() + if pid == 0: # 子进程 + listen_socket.close() # 关闭子进程中多余的套接字 + handle_request(client_connection) + client_connection.close() + os._exit(0) # 子进程在这里结束 + else: # 父进程 + # client_connection.close() + print(len(clients)) + +if __name__ == '__main__': + serve_forever() +``` + +用以下命令来启动服务器: + +``` +$ python webserver3d.py +``` + +用 `curl` 命令连接服务器: + +``` +$ curl http://localhost:8888/hello +Hello, World! +``` + +好,`curl` 命令输出了来自并发服务器的响应内容,但程序并没有退出,而是仍然挂起。到底发生了什么?这个服务器并不会挂起 60 秒:子进程只处理客户端连接,关闭连接然后退出,但客户端的 `curl` 命令并没有终止。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_child_is_active.png) + +所以,为什么 `curl` 不终止呢?原因就在于多余的文件描述符。当子进程关闭客户端连接时,系统内核会减少客户端套接字的引用计数,将其变为 1。服务器子进程退出了,但客户端套接字并没有被内核关闭,因为该套接字的描述符引用计数并没有变为 0,所以,这就导致了连接终止包(在 TCP/IP 协议中称作 `FIN`)不会被发送到客户端,所以客户端会一直保持连接。这里就会出现另一个问题:如果你的服务器在长时间运行,并且不关闭重复的文件描述符,那么可用的文件描述符会被消耗殆尽: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_out_of_descriptors.png) + +使用 `Control-C` 关闭服务器 `webserver3d.py`,然后在 shell 中使用内置命令 `ulimit` 来查看系统默认为你的服务器进程分配的可用资源数: + +``` +$ ulimit -a +core file size (blocks, -c) 0 +data seg size (kbytes, -d) unlimited +scheduling priority (-e) 0 +file size (blocks, -f) unlimited +pending signals (-i) 3842 +max locked memory (kbytes, -l) 64 +max memory size (kbytes, -m) unlimited +open files (-n) 1024 +pipe size (512 bytes, -p) 8 +POSIX message queues (bytes, -q) 819200 +real-time priority (-r) 0 +stack size (kbytes, -s) 8192 +cpu time (seconds, -t) unlimited +max user processes (-u) 3842 +virtual memory (kbytes, -v) unlimited +file locks (-x) unlimited +``` + +你可以从上面的结果看到,在我的 Ubuntu box 中,系统为我的服务器进程分配的最大可用文件描述符(文件打开)数为 1024。 + +现在我们来看一看,如果你的服务器不关闭重复的描述符,它会如何消耗可用的文件描述符。在一个已有的或新建的终端窗口中,将你的服务器进程的最大可用文件描述符设为 256: + +``` +$ ulimit -n 256 +``` + +在你刚刚运行 `ulimit -n 256` 的终端窗口中运行服务器 `webserver3d.py`: + +``` +$ python webserver3d.py +``` + +然后使用下面的客户端 `client3.py` 来测试你的服务器。 + +``` +####################################################### +# 测试客户端 - client3.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +####################################################### +import argparse +import errno +import os +import socket + + +SERVER_ADDRESS = 'localhost', 8888 +REQUEST = b"""\ +GET /hello HTTP/1.1 +Host: localhost:8888 + +""" + + +def main(max_clients, max_conns): + socks = [] + for client_num in range(max_clients): + pid = os.fork() + if pid == 0: + for connection_num in range(max_conns): + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.connect(SERVER_ADDRESS) + sock.sendall(REQUEST) + socks.append(sock) + print(connection_num) + os._exit(0) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser( + description='Test client for LSBAWS.', + formatter_class=argparse.ArgumentDefaultsHelpFormatter, + ) + parser.add_argument( + '--max-conns', + type=int, + default=1024, + help='Maximum number of connections per client.' + ) + parser.add_argument( + '--max-clients', + type=int, + default=1, + help='Maximum number of clients.' + ) + args = parser.parse_args() + main(args.max_clients, args.max_conns) +``` + +在一个新建的终端窗口中,运行 `client3.py` 然后让它与服务器同步创建 300 个连接: + +``` +$ python client3.py --max-clients=300 +``` + +过一会,你的服务器就该爆炸了。这是我的环境中出现的异常截图: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_too_many_fds_exc.png) + +这个例子很明显——你的服务器应该关闭重复的描述符。但是,即使你关闭了多余的描述符,你依然没有摆脱险境,因为你的服务器还有一个问题,这个问题在于“僵尸”! + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_zombies.png) + + +没错,这个服务器代码确实在制造僵尸进程。我们来看看怎么回事。重新运行你的服务器: + +``` +$ python webserver3d.py +``` + +在另一个终端窗口中运行以下 `curl` 命令: + +``` +$ curl http://localhost:8888/hello +``` + +现在,运行 `ps` 环境,来查看正在运行的 Python 进程。下面是我的环境中 `ps` 的运行结果: + +``` +$ ps auxw | grep -i python | grep -v grep +vagrant 9099 0.0 1.2 31804 6256 pts/0 S+ 16:33 0:00 python webserver3d.py +vagrant 9102 0.0 0.0 0 0 pts/0 Z+ 16:33 0:00 [python] +``` + +你看到第二行中,pid 为 9102,状态为 Z+,名字里面有个 `` 的进程了吗?那就是我们的僵尸进程。这个僵尸进程的问题在于:你无法将它杀掉。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_kill_zombie.png) + +就算你尝试使用 `kill -9` 来杀死僵尸进程,它们仍旧会存活。自己试试看,看看结果。 + +这个僵尸到底是什么,为什么我们的服务器会造出它们呢?一个僵尸进程是一个已经结束的进程,但它的父进程并没有等待它结束,并且也没有收到它的终结状态。如果一个进程在父进程退出之前退出,系统内核会把它变为一个僵尸进程,存储它的部分信息,以便父进程读取。内核保存的进程信息通常包括进程 ID,进程终止状态,以及进程的资源占用情况。OK,所以僵尸进程确实有存在的意义,但如果服务器不管这些僵尸进程,你的系统调用将会被阻塞。我们来看看这个要如何发生。首先,关闭你的服务器;然后,在一个新的终端窗口中,使用 `ulimit` 命令将最大用户进程数设为 400(同时,要确保你的最大可用描述符数大于这个数字,我们在这里设为 500): + +``` +$ ulimit -u 400 +$ ulimit -n 500 +``` + +在你刚刚运行 `ulimit -u 400` 命令的终端中,运行服务器 `webserver3d.py`: + +``` +$ python webserver3d.py +``` + +在一个新的终端窗口中,运行 `client3.py`,并且让它与服务器同时创建 500 个连接: + +``` +$ python client3.py --max-clients=500 +``` + +然后,过一会,你的服务器应该会再次爆炸,它会在创建新进程时抛出一个 `OSError: 资源暂时不可用` 异常。但它并没有达到系统允许的最大进程数。这是我的环境中输出的异常信息截图: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc3_resource_unavailable.png) + +你可以看到,如果服务器不管僵尸进程,它们会引发问题。我会简单探讨一下僵尸进程问题的解决方案。 + +我们来回顾一下你刚刚掌握的知识点: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) + +- 如果你不关闭重复的描述符,客户端就不会在请求处理完成后终止,因为客户端连接没有被关闭; +- 如果你不关闭重复的描述符,长久运行的服务器最终会把可用的文件描述符(最大文件打开数)消耗殆尽; +- 当你创建一个新进程,而父进程不等待子进程,也不在子进程结束后收集它的终止状态,它会变为一个僵尸进程; +- 僵尸通常都会吃东西,在我们的例子中,僵尸进程会占用资源。如果你的服务器不管僵尸进程,它最终会消耗掉所有的可用进程(最大用户进程数); +- 你不能杀死僵尸进程,你需要等待它。 + +所以,你需要做什么来处理僵尸进程呢?你需要修改你的服务器代码,来等待僵尸进程,并收集它们的终止信息。你可以在代码中使用系统调用 `wait` 来完成这个任务。不幸的是,这个方法里理想目标还很远,因为在没有终止的子进程存在的情况下调用 `wait` 会导致程序阻塞,这会阻碍你的服务器处理新的客户端连接请求。那么,我们有其他选择吗?嗯,有的,其中一个解决方案需要结合信号处理以及 `wait` 系统调用。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_signaling.png) + +这是它的工作流程。当一个子进程退出时,内核会发送 `SIGCHLD` 信号。父进程可以设置一个信号处理器,它可以异步响应 `SIGCHLD` 信号,并在信号响应函数中等待子进程收集终止信息,从而阻止了僵尸进程的存在。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part_conc4_sigchld_async.png) + +顺便,异步事件意味着父进程无法提前知道事件的发生时间。 + +修改你的服务器代码,设置一个 `SIGCHLD` 信号处理器,在信号处理器中等待终止的子进程。修改后的代码如下(webserver3e.py): + +``` +####################################################### +# 并发服务器 - webserver3e.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +####################################################### +import os +import signal +import socket +import time + +SERVER_ADDRESS = (HOST, PORT) = '', 8888 +REQUEST_QUEUE_SIZE = 5 + + +def grim_reaper(signum, frame): + pid, status = os.wait() + print( + 'Child {pid} terminated with status {status}' + '\n'.format(pid=pid, status=status) + ) + + +def handle_request(client_connection): + request = client_connection.recv(1024) + print(request.decode()) + http_response = b"""\ +HTTP/1.1 200 OK + +Hello, World! +""" + client_connection.sendall(http_response) + # 挂起进程,来允许父进程完成循环,并在 "accept" 处阻塞 + time.sleep(3) + + +def serve_forever(): + listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listen_socket.bind(SERVER_ADDRESS) + listen_socket.listen(REQUEST_QUEUE_SIZE) + print('Serving HTTP on port {port} ...'.format(port=PORT)) + + signal.signal(signal.SIGCHLD, grim_reaper) + + while True: + client_connection, client_address = listen_socket.accept() + pid = os.fork() + if pid == 0: # 子进程 + listen_socket.close() # 关闭子进程中多余的套接字 + handle_request(client_connection) + client_connection.close() + os._exit(0) + else: # 父进程 + client_connection.close() + +if __name__ == '__main__': + serve_forever() +``` + +运行服务器: + +``` +$ python webserver3e.py +``` + +使用你的老朋友——`curl` 命令来向修改后的并发服务器发送一个请求: + +``` +$ curl http://localhost:8888/hello +``` + +再来看看服务器: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_eintr.png) + +刚刚发生了什么?`accept` 调用失败了,错误信息为 `EINTR` + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_eintr_error.png) + +当子进程退出并触发 `SIGCHLD` 事件时,父进程的 `accept` 调用被阻塞了,系统转去运行信号处理器,当信号处理函数完成时,`accept` 系统调用被打断: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc4_eintr_accept.png) + +别担心,这个问题很好解决。你只需要重新运行 `accept` 系统调用即可。这是修改后的服务器代码 `webserver3f.py`,它可以解决这个问题: + + +``` +####################################################### +# 并发服务器 - webserver3f.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +####################################################### +import errno +import os +import signal +import socket + +SERVER_ADDRESS = (HOST, PORT) = '', 8888 +REQUEST_QUEUE_SIZE = 1024 + + +def grim_reaper(signum, frame): + pid, status = os.wait() + + +def handle_request(client_connection): + request = client_connection.recv(1024) + print(request.decode()) + http_response = b"""\ +HTTP/1.1 200 OK + +Hello, World! +""" + client_connection.sendall(http_response) + + +def serve_forever(): + listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listen_socket.bind(SERVER_ADDRESS) + listen_socket.listen(REQUEST_QUEUE_SIZE) + print('Serving HTTP on port {port} ...'.format(port=PORT)) + + signal.signal(signal.SIGCHLD, grim_reaper) + + while True: + try: + client_connection, client_address = listen_socket.accept() + except IOError as e: + code, msg = e.args + # 若 'accept' 被打断,那么重启它 + if code == errno.EINTR: + continue + else: + raise + + pid = os.fork() + if pid == 0: # 子进程 + listen_socket.close() # 关闭子进程中多余的描述符 + handle_request(client_connection) + client_connection.close() + os._exit(0) + else: # 父进程 + client_connection.close() # 关闭父进程中多余的描述符,继续下一轮循环 + + +if __name__ == '__main__': + serve_forever() +``` + +运行更新后的服务器 `webserver3f.py`: + +``` +$ python webserver3f.py +``` + +用 `curl` 来向更新后的并发服务器发送一个请求: + +``` +$ curl http://localhost:8888/hello +``` + +看到了吗?没有 EINTR 异常出现了。现在检查一下,确保没有僵尸进程存活,调用 `wait` 函数的 `SIGCHLD` 信号处理器能够正常处理被终止的子进程。我们只需使用 `ps` 命令,然后看看现在没有处于 Z+ 状态(或名字包含 `` )的 Python 进程就好了。很棒!僵尸进程没有了,我们很安心。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_checkpoint.png) + +- 如果你创建了一个子进程,但是不等待它,它就会变成一个僵尸进程; +- 使用 `SIGCHLD` 信号处理器可以异步地等待子进程终止,并收集其终止状态; +- 当使用事件处理器时,你需要牢记,系统调用可能会被打断,所以你需要处理这种情况发生时带来的异常。 + +好的,一切顺利。是不是没问题了?额,几乎是。重新尝试运行 `webserver3f.py` 但我们这次不会只发送一个请求,而是同步创建 128 个连接: + +``` +$ python client3.py --max-clients 128 +``` + +现在再次运行 `ps` 命令: + +``` +$ ps auxw | grep -i python | grep -v grep +``` + +看到了吗?天啊,僵尸进程又出来了! + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc5_zombies_again.png) + +这回怎么回事?当你同时运行 128 个客户端,建立 128 个连接时,服务器的子进程几乎会在同一时间处理好你的请求,然后退出。这会导致非常多的 `SIGCHLD` 信号被发送到父进程。问题在于,这些信号不会存储在队列中,所以你的服务器进程会错过很多信号,这也就导致了几个僵尸进程处于无主状态: + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc5_signals_not_queued.png) + +这个问题的解决方案依然是设置 `SIGCHLD` 事件处理器。但我们这次将会用 `WNOHANG` 参数循环调用 `waitpid`,来保证所有处于终止状态的子进程都会被处理。下面是修改后的代码,`webserver3g.py`: + +``` +####################################################### +# 并发服务器 - webserver3g.py # +# # +# 使用 Python 2.7.9 或 3.4 # +# 在 Ubuntu 14.04 及 Mac OS X 环境下测试通过 # +####################################################### +import errno +import os +import signal +import socket + +SERVER_ADDRESS = (HOST, PORT) = '', 8888 +REQUEST_QUEUE_SIZE = 1024 + + +def grim_reaper(signum, frame): + while True: + try: + pid, status = os.waitpid( + -1, # 等待所有子进程 + os.WNOHANG # 无终止进程时,不阻塞进程,并抛出 EWOULDBLOCK 错误 + ) + except OSError: + return + + if pid == 0: # 没有僵尸进程存在了 + return + + +def handle_request(client_connection): + request = client_connection.recv(1024) + print(request.decode()) + http_response = b"""\ +HTTP/1.1 200 OK + +Hello, World! +""" + client_connection.sendall(http_response) + + +def serve_forever(): + listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listen_socket.bind(SERVER_ADDRESS) + listen_socket.listen(REQUEST_QUEUE_SIZE) + print('Serving HTTP on port {port} ...'.format(port=PORT)) + + signal.signal(signal.SIGCHLD, grim_reaper) + + while True: + try: + client_connection, client_address = listen_socket.accept() + except IOError as e: + code, msg = e.args + # 若 'accept' 被打断,那么重启它 + if code == errno.EINTR: + continue + else: + raise + + pid = os.fork() + if pid == 0: # 子进程 + listen_socket.close() # 关闭子进程中多余的描述符 + handle_request(client_connection) + client_connection.close() + os._exit(0) + else: # 父进程 + client_connection.close() # 关闭父进程中多余的描述符,继续下一轮循环 + +if __name__ == '__main__': + serve_forever() +``` + +运行服务器: + +``` +$ python webserver3g.py +``` + +使用测试客户端 `client3.py`: + +``` +$ python client3.py --max-clients 128 +``` + +现在来查看一下,确保没有僵尸进程存在。耶!没有僵尸的生活真美好 ^_^ + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_conc5_no_zombies.png) + +恭喜!你刚刚经历了一段很长的旅程,我希望你能够喜欢它。现在你拥有了自己的建议并发服务器,并且这段代码能够为你在继续研究生产级 Web 服务器的路上奠定基础。 + +我将会留一个作业:你需要将第二部分中的 WSGI 服务器升级,将它改造为一个并发服务器。你可以在[这里][12]找到更改后的代码。但是,当你实现了自己的版本之后,你才应该来看我的代码。你已经拥有了实现这个服务器所需的所有信息。所以,快去实现它吧 ^_^ + +然后要做什么呢?乔希·比林斯说过: + +> “我们应该做一枚邮票——专注于一件事,不达目的不罢休。” + +开始学习基本知识。回顾你已经学过的知识。然后一步一步深入。 + +![](https://ruslanspivak.com/lsbaws-part3/lsbaws_part3_dig_deeper.png) + +> “如果你只学会了方法,你将会被这些方法所困。但如果你学会了原理,那你就能发明出新的方法。”——拉尔夫·沃尔多·爱默生 + +下面是一份书单,我从这些书中提炼出了这篇文章所需的素材。他们能助你在我刚刚所述的几个方面中发掘出兼具深度和广度的知识。我极力推荐你们去搞到这几本书看看:从你的朋友那里借,在当地的图书馆中阅读,或者直接在亚马逊上把它买回来。下面是我的典藏秘籍: + +1. [UNIX网络编程 (卷1):套接字联网API (第3版)][6] +2. [UNIX环境高级编程 (第3版)][7] +3. [Linux/UNIX系统编程手册][8] +4. [TCP/IP详解 (卷1):协议 (第2版) (爱迪生-韦斯莱专业编程系列)][9] +5. [信号系统简明手册 (第二版): 并发控制深入浅出及常见错误][10]. 这本书也可以从[作者的个人网站][11]中买到。 + +顺便,我在撰写一本名为《搭个 Web 服务器:从头开始》的书。这本书讲解了如何从头开始编写一个基本的 Web 服务器,里面包含本文中没有的更多细节。订阅邮件列表,你就可以获取到这本书的最新进展,以及发布日期。 + +-------------------------------------------------------------------------------- + +via: https://ruslanspivak.com/lsbaws-part3/ + +作者:[Ruslan][a] +译者:[StdioA](https://github.com/StdioA) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://github.com/rspivak/ + +[1]: https://github.com/rspivak/lsbaws/blob/master/part3/ +[2]: https://github.com/rspivak/lsbaws/blob/master/part3/webserver3a.py +[3]: https://github.com/rspivak/lsbaws/blob/master/part3/webserver3b.py +[4]: https://ruslanspivak.com/lsbaws-part3/#fn:1 +[5]: https://ruslanspivak.com/lsbaws-part3/#fn:2 +[6]: http://www.amazon.com/gp/product/0131411551/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0131411551&linkCode=as2&tag=russblo0b-20&linkId=2F4NYRBND566JJQL +[7]: http://www.amazon.com/gp/product/0321637739/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0321637739&linkCode=as2&tag=russblo0b-20&linkId=3ZYAKB537G6TM22J +[8]: http://www.amazon.com/gp/product/1593272200/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1593272200&linkCode=as2&tag=russblo0b-20&linkId=CHFOMNYXN35I2MON +[9]: http://www.amazon.com/gp/product/0321336313/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0321336313&linkCode=as2&tag=russblo0b-20&linkId=K467DRFYMXJ5RWAY +[10]: http://www.amazon.com/gp/product/1441418687/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1441418687&linkCode=as2&tag=russblo0b-20&linkId=QFOAWARN62OWTWUG +[11]: http://greenteapress.com/semaphores/ +[12]: https://github.com/rspivak/lsbaws/blob/master/part3/webserver3h.py From 19b77361a6bcb556959d82ce1fa7b4c4d946a0e0 Mon Sep 17 00:00:00 2001 From: Zxp Date: Sat, 10 Sep 2016 20:01:40 +0800 Subject: [PATCH 0092/2437] Delete 20160817 Turtl - Secure, Open Source Evernote Alternative.md --- ...ecure, Open Source Evernote Alternative.md | 84 ------------------- 1 file changed, 84 deletions(-) delete mode 100644 sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md diff --git a/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md b/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md deleted file mode 100644 index 5c67768cde..0000000000 --- a/sources/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md +++ /dev/null @@ -1,84 +0,0 @@ -chisper is translating - -Turtl: Secure, Open Source Evernote Alternative -============= - - -[Turtl][9] is a secure, open source Evernote alternative, available for Linux, Windows, Mac, and Android. An iOS version is "coming soon". Firefox and Chrome bookmarking extensions are also available. - -![](https://3.bp.blogspot.com/-cNoUUjaU4A0/V7MFKCasZJI/AAAAAAAAYTk/r7oWe-z_HB87hDvlKLViiiHUMfagnC6LQCLcB/s400/turtl-desktop-linux.png) - -The application, which is currently in beta, lets you keep your notes (with Markdown support for the note editor), website bookmarks, passwords, documents, photos, and so on, in a single private place. - -Notes can be organized in boards, which support nesting, and can be shared with other Turtl users: - -![](https://2.bp.blogspot.com/-G-Ln3T1c2QA/V7MFmrqkukI/AAAAAAAAYTs/dXMPEB9MPREicixlEJlQVqg9SFjBX1pwgCLcB/s400/turtl-boards.png) - -You can also add tags to your notes. The Turtle search allows sorting by creation date, last edited date, or by tags. - -Here's the note editor (for a file note): - -![](https://1.bp.blogspot.com/-8cNHV69iCWM/V7MFX7sBlMI/AAAAAAAAYTo/ZUVTYwiCSy8uzrVKdf6NcsQZlHtylIyvgCEw/s400/turtl-edit-note.png) - -So what about security? Turtl encrypts the data before storing it, using a cryptographic key, and the password is not stored on the server. Only you and those you choose to share with can read your data. You can read more about the Turtl security and encryption [HERE][1]. - -Update (thanks to Dimitry!): according to a [bug report][2], Turtl has a pretty serious security issue. Turtl allows creating multiple accounts with the same username, distinguishing between them by passwords. Hopefully this will be fixed soon. - -The Turtl developers provide a hosted service for synchronizing your notes, which is completely free "until your profile grows past a certain size or you require certain features". At the time I'm writing this article, the premium service is not available. - -However, you don't have to use the self hosted server - you can run your own [Turtl server][3] since it's free, open source software, just like the desktop and mobile applications. - -Turtl is not as feature rich as Evernote, however, quite a few new features are listed in its [roadmap][4], like import/export to plaintext and Evernote data format, native PDF reader support, interface locking, and more. - -I should also mention that the desktop application requires entering the password every time it's started, which might be good for security reasons, but can be considered annoying by some. - - -### Download Turtl - - -[Download Turtl application][5] (binaries available for Linux - 32bit and 64bit, Windows 64bit, Mac 64bit, Android, as well as Chrome and Firefox bookmarking add-ons) - -**Update**: Turtl uses a new server. To get the app to work, logout and in Advanced settings, under the login box, set the Turtl server to "https://api.turtlapp.com/v2" (without the quotes). - -To download the source code (desktop, mobile and server), report bugs, etc., see the Turtl @ [GitHub][6]. - -Arch Linux users can install Turtl via [AUR][7]. - -To install Turtl in Linux, extract the downloaded archive and run the "install.sh" script. Before installing it, make sure the ~/.local/share/applications folder exists: - -``` -mkdir -p ~/.local/share/applications -``` - -Important: installing Turtl with sudo makes the application runnable as root only, so either install it without sudo (somewhere in your home folder), or manually fix the permissions (you can take a look at the AUR [package][8] for what permissions to set). - -For instance, to install Turtl in the ~/turtl folder, use the following command (assumes you've extracted Turtl in your home folder): - -```` -~/turtl-*/install.sh ~/turtl -``` - -You can use "~/.turtl" instead of "~/turtl" to install Turtl to a hidden folder in your home directory. Or you can hide the ~/turtl folder using a simple trick. - -If Turtl doesn't show up in the menu / Unity Dash, restart the session (logout / login). - --------------------------------------------------------------------------------- - -via: http://www.webupd8.org/2016/08/turtl-secure-open-source-evernote.html - -作者:[Andrew ][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.webupd8.org/p/about.html -[1]: https://turtl.it/docs/security/ -[2]: https://github.com/turtl/api/issues/20 -[3]: https://turtl.it/docs/server/ -[4]: https://trello.com/b/yIQGkHia/turtl-product-dev -[5]: https://turtl.it/download/ -[6]: https://github.com/turtl -[7]: https://aur.archlinux.org/packages/turtl/ -[8]: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=turtl -[9]: https://turtlapp.com/ From 646720a758d8e6b61e780909feb4d7d4f4b539c4 Mon Sep 17 00:00:00 2001 From: Zxp Date: Sat, 10 Sep 2016 20:03:29 +0800 Subject: [PATCH 0093/2437] Create 20160817 Turtl - Secure, Open Source Evernote Alternative.md --- ...ecure, Open Source Evernote Alternative.md | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md diff --git a/translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md b/translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md new file mode 100644 index 0000000000..46fd3b59f2 --- /dev/null +++ b/translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md @@ -0,0 +1,76 @@ +Turtl:安全、开源的Evernote替代品 + +Turtl是一个安全、开源的Evernote替代品,在Linux, Windows, Mac, and Android等系统上都能使用。iOS版本仍在开发当中,Firefox和Chrome也有扩展程序可以使用。 + +![](https://3.bp.blogspot.com/-cNoUUjaU4A0/V7MFKCasZJI/AAAAAAAAYTk/r7oWe-z_HB87hDvlKLViiiHUMfagnC6LQCLcB/s400/turtl-desktop-linux.png) + +这个产品仍在测试阶段,能够让你把你的笔记,网站书签,密码,文档,图片单独放在一个隐秘地方。 + +笔记可以按模块组织起来,支持嵌套,也可以和其他Turtl用户分享。 + +![](https://2.bp.blogspot.com/-G-Ln3T1c2QA/V7MFmrqkukI/AAAAAAAAYTs/dXMPEB9MPREicixlEJlQVqg9SFjBX1pwgCLcB/s400/turtl-boards.png) + +给你的笔记打上标签,这样Turtl就会通过创建时间,最后修改时间或者标签来找你的笔记。 + +这个是文本编辑器: + +![](https://1.bp.blogspot.com/-8cNHV69iCWM/V7MFX7sBlMI/AAAAAAAAYTo/ZUVTYwiCSy8uzrVKdf6NcsQZlHtylIyvgCEw/s400/turtl-edit-note.png) + +那么安全性如何呢?Turtl会在保存数据之前加密,。只有你和你想要分享的人能获取数据。你可以从这获得跟多关于Turtl安全和加密的信息[HERE][1]。 + +更新(感谢Dimitry!):根据错误反馈[bug report][2],Turtl有个严重的安全性问题。Turtl允许同一个用户名创建多个账号,却只使用密码来区分它们。希望能马上修复这个问题。 + +Turtl团队提供了一个主机服务来同步你的记录,它是完全免费的,”直到你的文件足够大,或者你需要更好的服务”,在我写这篇文章的时候这个服务还不能用。 + +并且你也不一定要用这个服务,因为它是一个免费,开源的软件,所以你可以自己搭建一个[Turtl server][3]。 + +Turtl没有像Evernote那么多的功能,但它也有一些新的功能[roadmap][4],比如:支持导入/导出文本,Evernote格式的数据,支持PDF,界面锁定等。 + +不得不提醒的是,每次启动都要输入密码,虽然安全,但有时候实在是麻烦。 + +###下载Turtl + +[Download Turtl application][5] (Linux二进制文件-32bit or 64bit,Windows 64bit,Mac 64bit,Android,Chrome和Firefox浏览器插件) + +**更新**:Turtl用了一个新的服务器,注销然后在登陆框的下面选择高级设置,把Turtl服务器设置为"https://api.turtlapp.com/v2"(没有引号)。 + +下载源代码,反馈问题等,参见Turtl的GitHub[GitHub][6]。 + +在Linux上安装,把安装包解压后运行install.sh,安装之前请确保~/.local/share/applications目录存在,若不存在请自行创建: + +``` +mkdir -p ~/.local/share/applications +``` + +注意:如果使用sudo命令安装那么只有root用户才能使用。所以,要么不用sudo命令安装,要么在安装完成后修改权限。你可以在这里[package][8]参考如何修改权限。 + +使用如下命令把Turtl安装到~/turtl文件夹下(假定你已经把安装包解压在你家目录下了): + +```` +~/turtl-*/install.sh ~/turtl +``` + +可以使用~/.turtl代替~/turtl把Turtl安装到你家目录的隐藏文件夹下。你也可以用些小技巧把它隐藏起来。 + +如果Turtl没有在你的Unity Dash上显示出来,请注销/登陆以重启会话。 + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/08/turtl-secure-open-source-evernote.html + +作者:[Andrew ][a] +译者:[chisper](https://github.com/chisper) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: https://turtl.it/docs/security/ +[2]: https://github.com/turtl/api/issues/20 +[3]: https://turtl.it/docs/server/ +[4]: https://trello.com/b/yIQGkHia/turtl-product-dev +[5]: https://turtl.it/download/ +[6]: https://github.com/turtl +[7]: https://aur.archlinux.org/packages/turtl/ +[8]: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=turtl +[9]: https://turtlapp.com/ From edbe8670e191b811639d94812bfb0fbab5abaa42 Mon Sep 17 00:00:00 2001 From: Bestony Date: Sat, 10 Sep 2016 22:09:38 +0800 Subject: [PATCH 0094/2437] =?UTF-8?q?=E9=80=89=E9=A2=98=E7=BF=BB=E8=AF=91?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- sources/tech/20160817 Build an integration for GitHub.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160817 Build an integration for GitHub.md b/sources/tech/20160817 Build an integration for GitHub.md index 2a9c57b3bb..c080d4e2bd 100644 --- a/sources/tech/20160817 Build an integration for GitHub.md +++ b/sources/tech/20160817 Build an integration for GitHub.md @@ -1,3 +1,4 @@ +Translating by Bestony Build an integration for GitHub ============= From 7dad016b1c0be3c5a41fc567f3bc3443bb3cf99f Mon Sep 17 00:00:00 2001 From: wxy Date: Sat, 10 Sep 2016 23:05:15 +0800 Subject: [PATCH 0095/2437] PUB:20160516 Linux will be the major operating system of 21st century cars @XLCyun --- ...r operating system of 21st century cars.md | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-) rename {translated/talk => published}/20160516 Linux will be the major operating system of 21st century cars.md (51%) diff --git a/translated/talk/20160516 Linux will be the major operating system of 21st century cars.md b/published/20160516 Linux will be the major operating system of 21st century cars.md similarity index 51% rename from translated/talk/20160516 Linux will be the major operating system of 21st century cars.md rename to published/20160516 Linux will be the major operating system of 21st century cars.md index 541ea23853..08a96d56c9 100644 --- a/translated/talk/20160516 Linux will be the major operating system of 21st century cars.md +++ b/published/20160516 Linux will be the major operating system of 21st century cars.md @@ -1,24 +1,25 @@ -Linux 将成为21世纪汽车的主要操作系统 +Linux 将成为 21 世纪汽车的主要操作系统 =============================================================== ->汽车可不单单是由引擎和华丽外壳组成的,汽车里还有许许多多的计算部件,而 Linux 就在它们里面跑着。 +> 汽车可不单单是由引擎和华丽外壳组成的,汽车里还有许许多多的计算部件,而 Linux 就在它们里面跑着。 -Linux 不只运行在你的服务器和手机(安卓)上。它还运行在你的车里。当然了,没人会因为某个车载系统而去买辆车。但是 Linux 已经为像丰田,日产,捷豹路虎,马自达,三菱,斯巴鲁,这些大型汽车制造商改进了信息娱乐系统、平视显示以及其联网汽车(connected car)的 4G 与 Wi-Fi 系统,而且 [Linux 即将登陆福特汽车][1]。 +Linux 不只运行在你的服务器和手机(安卓)上。它还运行在你的车里。当然了,没人会因为某个车载系统而去买辆车。但是 Linux 已经为像丰田、日产、捷豹路虎这些大型汽车制造商提供了信息娱乐系统、平视显示以及其联网汽车(connected car)的 4G 与 Wi-Fi 系统,而且 [Linux 即将登陆福特汽车][1]、马自达、三菱、斯巴鲁。 ![](http://zdnet4.cbsistatic.com/hub/i/2016/05/10/743f0c14-6458-4d1e-8723-d2d94d0d0e69/c297b7d52e27e97d8721d4cb46bb371b/agl-logo.jpg) ->如今,所有的 Linux 和开源汽车软件的成果都已经在 Automotive Grade Linux 项目下统一标准化了。 -传统软件公司也进入了移动物联网领域。 Movimento, Oracle, Qualcomm, Texas Instruments, UIEvolution 和 VeriSilicon 都已经 [加入 Automotive Grade Linux(AGL)项目][2]。 [AGL][3] 是一个相互协作的开源项目,志在于为联网汽车打造一个基于 Linux 的通用软件栈。 +*如今,所有的 Linux 和开源汽车软件的成果都已经在 Linux 基金会的 Automotive Grade Linux (AGL)项目下统一标准化了。* + +传统软件公司也进入了移动物联网领域。 Movimento、甲骨文、高通、Texas Instruments、UIEvolution 和 VeriSilicon 都已经[加入 Automotive Grade Linux(AGL)项目][2]。 [AGL][3] 是一个相互协作的开源项目,志在于为联网汽车打造一个基于 Linux 的通用软件栈。 “随着联网汽车技术和信息娱乐系统需求的快速增长,AGL 过去几年中得到了极大的发展,” Linux 基金会汽车总经理 Dan Cauchy 如是说。 -Cauchy 又补充道,“我们的会员基础不单单只是迅速壮大,而且通过横跨不同的业界实现多元化,从半导体和车载软件到 IoT 和连接云服务。这是一个明显的迹象,即联网汽车的革命已经间接影响到许多行业纵向市场。” +Cauchy 又补充道,“我们的会员基础不单单只是迅速壮大,而且通过横跨不同的业界实现了多元化,从半导体和车载软件到 IoT 和连接云服务。这是一个明显的迹象,即联网汽车的革命已经间接影响到许多行业纵向市场。” -这些公司在 AGL 发布了新的 AGL Unified Code Base (UCB) 之后加入了 AGL 项目。这个新的 Linux 发行版基于 AGL 和另外两个汽车开源项目: [Tizen][4] 和 [GENIVI Alliance][5] 。 UCB 是第二代 Linux 汽车系统。它从底层开始开发,一直到特定的汽车应用软件。它能处理导航,通信,安全,安保和信息娱乐系统, +这些公司在 AGL 发布了新的 AGL Unified Code Base (UCB) 之后加入了 AGL 项目。这个新的 Linux 发行版基于 AGL 和另外两个汽车开源项目: [Tizen][4] 和 [GENIVI Alliance][5] 。 UCB 是第二代 Linux 汽车系统。它从底层开始开发,一直到特定的汽车应用软件。它能处理导航、通信、安全、安保和信息娱乐系统。 -“汽车行业需要一个标准的开源系统和框架来让汽车制造商和供应商能够快速地将类似智能手机的功能带入到汽车中来。” Cauchy 说。“这个新的发行版将 AGL, Tizen, GENIVI 项目和相关开源代码中的精华部分整合进 AGL Unified Code Base ( UCB ) 中,使得汽车制造商能够利用一个通用平台进行快速创新。 在汽车中采用基于 Linux 的系统来实现所有功能时, AGL 的 UCB 发行版将扮演一个重大的角色。” +“汽车行业需要一个标准的开源系统和框架来让汽车制造商和供应商能够快速地将类似智能手机的功能带入到汽车中来。” Cauchy 说。“这个新的发行版将 AGL、Tizen、GENIVI 项目和相关开源代码中的精华部分整合进 AGL Unified Code Base (UCB)中,使得汽车制造商能够利用一个通用平台进行快速创新。 在汽车中采用基于 Linux 的系统来实现所有功能时, AGL 的 UCB 发行版将扮演一个重大的角色。” -他说得对。自从 2016 年 1 月发布以来,已有四个汽车公司和十个新的软件厂商加入了 AGL。Esso, 如今的 Exxon, 曾让 “把老虎装入油箱” 这条广告语出了名。我怀疑 “把企鹅装到引擎盖下” 这样的广告语是否也会变得家喻户晓,但是它却道出了事实。 Linux 正在慢慢成为 21 世纪汽车的主要操作系统。 +他说得对。自从 2016 年 1 月发布以来,已有四个汽车公司和十个新的软件厂商加入了 AGL。Esso,如今的 Exxon, 曾让 “把老虎装入油箱” 这条广告语出了名。我怀疑 “把企鹅装到引擎盖下” 这样的广告语是否也会变得家喻户晓,但是它却道出了事实。 Linux 正在慢慢成为 21 世纪汽车的主要操作系统。 ------------------------------------------------------------------------------ @@ -26,7 +27,7 @@ via: http://www.zdnet.com/article/the-linux-in-your-car-movement-gains-momentum/ 作者:[Steven J. Vaughan-Nichols][a] 译者:[XLCYun](https://github.com/XLCYun) -校对:[校对者ID](https://github.com/校对者ID) +校对:[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 From d6cf19cd159bdd755b39137710d764d975450880 Mon Sep 17 00:00:00 2001 From: wxy Date: Sat, 10 Sep 2016 23:17:57 +0800 Subject: [PATCH 0096/2437] =?UTF-8?q?=E4=BF=AE=E6=AD=A3=E6=96=87=E4=BB=B6?= =?UTF-8?q?=E5=90=8D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @oska874 --- ...Script Parser that Creates a Language in 200 Lines of Code.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename sources/tech/{20160830 Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code.md => 20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md} (100%) diff --git a/sources/tech/20160830 Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code.md b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md similarity index 100% rename from sources/tech/20160830 Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code.md rename to sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md From 69327af8906ceefa9fc81a2f378bd77154fb66a3 Mon Sep 17 00:00:00 2001 From: wxy Date: Sat, 10 Sep 2016 23:46:38 +0800 Subject: [PATCH 0097/2437] PUB:20160830 The State Of JavaScript - JavaScript Flavors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @eriwoon 翻译的不错,加油! --- ...tate Of JavaScript - JavaScript Flavors.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) rename {translated/tech => published}/20160830 The State Of JavaScript - JavaScript Flavors.md (79%) diff --git a/translated/tech/20160830 The State Of JavaScript - JavaScript Flavors.md b/published/20160830 The State Of JavaScript - JavaScript Flavors.md similarity index 79% rename from translated/tech/20160830 The State Of JavaScript - JavaScript Flavors.md rename to published/20160830 The State Of JavaScript - JavaScript Flavors.md index ce43bf2300..2261426046 100644 --- a/translated/tech/20160830 The State Of JavaScript - JavaScript Flavors.md +++ b/published/20160830 The State Of JavaScript - JavaScript Flavors.md @@ -3,13 +3,13 @@ JavaScript 现状:方言篇 JavaScript 和其他编程语言有一个很大的不同,它不像单纯的一个语言,而像一个由众多方言组成大家族。 -从2009年 CoffeeScript 的出现开始,近几年出现了大量基于 JavaScript 语言,或者叫方言,例如 ES6,TypeScript,Elm 等等。它们都有自己的优势,且都可以被完美编译成标准 JavaScript。 +从 2009 年 CoffeeScript 出现开始,近几年出现了大量基于 JavaScript 语言,或者叫方言,例如 ES6、TypeScript、Elm 等等。它们都有自己的优势,且都可以被完美编译成标准 JavaScript。 所以,继上周的前端框架篇,今天带来 JavaScript 现状之方言篇,看一下大家对于 JavaScript 的方言是怎么选择的。 -> 声明:下面的结论是根据部分数据得出的初步结果,希望能将我的一些想法带给大家,对于完整数据后面我会用更好方式呈现出来。 +> 声明:下面的部分结论来自部分数据,这是在我想要展示完整数据时找到的最好的办法,这便于我分享我的一些想法。 -> 注意:如果你还没有参与这个调查,现在就来参加吧,可以花十分钟完成调查然后再回来看这篇文章。 +> 注意:如果你还没有参与[这个调查][3],现在就来参加吧,可以花十分钟完成调查然后再回来看这篇文章。 ### 认知度 @@ -22,7 +22,7 @@ JavaScript 和其他编程语言有一个很大的不同,它不像单纯的一 - Elm: 66% - ClojureScript: 77% -你可能觉得100%的人都应该知道『经典的 JavaScript 』,我想是有人无法抵抗『我从来没有听说过 JavaScript 』这个选项的强大诱惑吧…… +你可能觉得 100% 的人都应该知道『经典的 JavaScript 』,我想是有人无法抵抗在一个 JavaScript 调查中投『我从来没有听说过 JavaScript 』这个选项的强大诱惑吧…… 几乎所有人都知道 ES6、CoffeeScript 和 TypeScript 这三种语言,比较令我惊讶的是 TypeScript 竟然会稍微落后于 ES6 和 CoffeeScript。 @@ -34,7 +34,7 @@ JavaScript 和其他编程语言有一个很大的不同,它不像单纯的一 ![](https://d3ugvbs94d921r.cloudfront.net/57c4dc599973d2525fee820a.png?t=3efc9491eba2ce2) -要注意,表是统计该语言对从未使用过它们的用户的吸引度,因为只有很少人没有用过经典 JavaScript,所以『经典 JavaScript 』这一列的数值很低。 +要注意,该表是统计该语言对从未使用过它们的用户的吸引度,因为只有很少人没有用过经典 JavaScript,所以『经典 JavaScript 』这一列的数值很低。 ES6的数值很有趣:已有很大比例的用户在使用 ES6 了,没有用过的人中的绝大部分(89%)也很想学习它。 @@ -51,6 +51,7 @@ TypeScript 和 Elm 的状态差不多:用过的人不多,但感兴趣的比 ![](https://d3ugvbs94d921r.cloudfront.net/57c4e5f79973d29461ee820a.png?t=1061d2ab8fc9838) 虽然经典 JavaScript 拥有最多的用户量,但就满意度来说 ES6 才是大赢家,而且我想现在已经能安全的说,ES6 可以作为开发 JavaScript App 默认的语言。 + TypeScript 和 Elm 有相似的高满意度,都在 85% 上下。然后,只有可怜的 17% 的开发者会考虑继续使用 CoffeeScript。 ### 快乐度 @@ -71,25 +72,24 @@ TypeScript 和 Elm 有相似的高满意度,都在 85% 上下。然后,只 个人认为,当前 JavaScript 大家庭百花齐放的现象还只是一个开始,或许几年之后 JavaScript 就会变得非常不同了。 -### 结语&敬请期待 +### 结语 & 敬请期待 对于我这样的调查来说数据越多就意味着数据越准确!越多人参加这个调查,那就越能代表整个 JavaScript 社区。 所以,我十分希望你能帮忙分享这个调查问卷: -[On Twitter][1] - -[On Facebook][2] +- [在 Twitter 上][1] +- [在 Facebook 上][2] 另外,如果你想收到我下一个调查结果分析,前往 [调查问卷主页][3] 并留下自己的邮箱吧。 -------------------------------------------------------------------------------- -via: https://www.oreilly.com/ideas/spark-comparison-aws-vs-gcp?utm_source=dbweekly&utm_medium=email +via: https://medium.com/@sachagreif/the-state-of-javascript-javascript-flavors-1e02b0bfefb6 作者:[Sacha Greif][a] 译者:[eriwoon](https://github.com/eriwoon) -校对:[校对者ID](https://github.com/校对者ID) +校对:[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From 9c3d60bab21c42d53115af90fe3e666f2f2a028e Mon Sep 17 00:00:00 2001 From: Zxp Date: Sun, 11 Sep 2016 09:08:53 +0800 Subject: [PATCH 0098/2437] Update 20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md --- ...i Hadoop Cluster with Apache Spark on YARN - Big Data 101.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md b/sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md index 0f42fc6fc9..f001632f4f 100644 --- a/sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md +++ b/sources/tech/20160901 A Raspberry Pi Hadoop Cluster with Apache Spark on YARN - Big Data 101.md @@ -1,3 +1,5 @@ +chisper is translating. + A Raspberry Pi Hadoop Cluster with Apache Spark on YARN: Big Data 101 ====== From 0369b3e77b1a7ffb942b59e2c3b18962fc1c22a4 Mon Sep 17 00:00:00 2001 From: Bestony Date: Sun, 11 Sep 2016 11:02:01 +0800 Subject: [PATCH 0099/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=20(#4390)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 翻译完成。 * 移动文件 --- ...0160817 Build an integration for GitHub.md | 55 ------------------- ...0160817 Build an integration for GitHub.md | 54 ++++++++++++++++++ 2 files changed, 54 insertions(+), 55 deletions(-) delete mode 100644 sources/tech/20160817 Build an integration for GitHub.md create mode 100644 translated/tech/20160817 Build an integration for GitHub.md diff --git a/sources/tech/20160817 Build an integration for GitHub.md b/sources/tech/20160817 Build an integration for GitHub.md deleted file mode 100644 index c080d4e2bd..0000000000 --- a/sources/tech/20160817 Build an integration for GitHub.md +++ /dev/null @@ -1,55 +0,0 @@ -Translating by Bestony -Build an integration for GitHub -============= - -![](https://cloud.githubusercontent.com/assets/4432363/17537572/934e2f9c-5e52-11e6-9c24-50c7be5b5ae3.png) - -You can now access more tools from our [Integrations Directory][1]. The Directory now has more than 15 categories — from [API management][2] to [application monitoring][3], there's a GitHub integration to support every stage of your development cycle. - -We're inviting developers of all levels of expertise to create an integration that will help developers work better. If you've built a great integration for GitHub, we want to highlight it! [Gitter][4], [AppVeyor][5], and [ZenHub][6] did it, and so can you. - -### What we're looking for - -Good software development relies on quality tools and developers today have a wide range of choices—language, framework, workflow, and environment among other factors. We're looking for tools that create a better overall development experience. - -#### A couple of guidelines for Integrations Directory listings: - -- Demonstrates solid growth (you should currently have over 500 users via GitHub OAuth) -- Meets our [Technical Requirements][7] -- Follows our [Terms of Service][8] and [Privacy Policy][9] -- Focuses on the software development life cycle - -### Helpful resources - -To get listed, please follow the steps outlined in the [listing requirements page][10]. - -You also should read our[ marketing guidelines][11] and the [existing Directory listings][12] to get a better idea of how to put it all together. Please draft the content for your listing in a [secret gist][13] (markdown format) and email us at . If you have any questions, please don't hesitate to reach out to us at . - --------------------------------------------------------------------------------- - -via: https://github.com/blog/2226-build-an-integration-for-github - -作者:[chobberoni ][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://github.com/chobberoni - -[1]: https://github.com/integrations -[2]: https://github.com/integrations/feature/api-management -[3]: https://github.com/integrations/feature/monitoring -[4]: https://github.com/integrations/feature/monitoring -[5]: https://github.com/integrations/appveyor -[6]: https://github.com/integrations/zenhub -[7]: https://developer.github.com/integrations-directory/getting-listed/#technical-requirements -[8]: https://help.github.com/articles/github-terms-of-service/ -[9]: https://help.github.com/articles/github-privacy-policy/ -[10]: https://developer.github.com/integrations-directory/getting-listed/ -[11]: https://developer.github.com/integrations-directory/marketing-guidelines/ -[12]: https://github.com/integrations -[13]: https://gist.github.com/ - - - diff --git a/translated/tech/20160817 Build an integration for GitHub.md b/translated/tech/20160817 Build an integration for GitHub.md new file mode 100644 index 0000000000..cd04dc451c --- /dev/null +++ b/translated/tech/20160817 Build an integration for GitHub.md @@ -0,0 +1,54 @@ +为 Github 创造一种集成方式 +============= + +![](https://cloud.githubusercontent.com/assets/4432363/17537572/934e2f9c-5e52-11e6-9c24-50c7be5b5ae3.png) + +现在你可以从我们的 [集成方式目录][1]接触到更多工具。这个目录目前有超过15个分类 — 从 [API 管理][2] 到 [应用监控][3], 这里有 Github 的集成方式,可以支持您的开发周期的每一个阶段。 + +我们邀请了各级的开发人员,来创造一个新的集成方式,将有助于帮助开发者更好的工作。如果你曾经为Github构建过一个不错的集成方式,我们希望让他更出名! [Gitter][4], [AppVeyor][5], 和 [ZenHub][6]都做到了,你也可以! + +### 我们在寻找什么? + +良好的软件开发依赖于质量工具和开发人员如今有一个广泛范围来选择语言,框架,工作流程,和其他因素之间的环境。我们正在寻找能够创造更好的整体开发经验的开发工具。 + +#### 一份集成目录清单指南: + +- 扎实的增长( 你的Github OAuth 接口应该有超过500个用户 ) +- 查看我们的 [技术要求][7] +- 遵从 [服务条款][8] 和 [隐私政策][9] +- 专注于软件开发生命周期 + +### 有帮助的资源 + +如果想要被列出,请按照[列出需求页][10]中概述的步骤 + +你也应该阅读我们的[ 营销指南 ][11] 和 [已有目录清单][12] 来更好的了解如何把他们所有的放在一起。 请把你的列表的内容记录在一个[私密 gist][13]中(markdown 格式) 并且通过邮件联系我们 . 如果你有任何问题,不要害羞,请联系 . + +-------------------------------------------------------------------------------- + +via: https://github.com/blog/2226-build-an-integration-for-github + +作者:[chobberoni ][a] +译者:[Bestony](https://github.com/Bestony) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://github.com/chobberoni + +[1]: https://github.com/integrations +[2]: https://github.com/integrations/feature/api-management +[3]: https://github.com/integrations/feature/monitoring +[4]: https://github.com/integrations/feature/monitoring +[5]: https://github.com/integrations/appveyor +[6]: https://github.com/integrations/zenhub +[7]: https://developer.github.com/integrations-directory/getting-listed/#technical-requirements +[8]: https://help.github.com/articles/github-terms-of-service/ +[9]: https://help.github.com/articles/github-privacy-policy/ +[10]: https://developer.github.com/integrations-directory/getting-listed/ +[11]: https://developer.github.com/integrations-directory/marketing-guidelines/ +[12]: https://github.com/integrations +[13]: https://gist.github.com/ + + + From 5aadcc70adf0ef4395175c3466518a87d68e2a17 Mon Sep 17 00:00:00 2001 From: LinuxBars Date: Sun, 11 Sep 2016 11:51:19 +0800 Subject: [PATCH 0100/2437] LinuxBars translating --- ...4 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md b/sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md index 3856f4579c..7b79854bb7 100644 --- a/sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md +++ b/sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md @@ -1,3 +1,6 @@ +LinuxBars Translating +LinuxBars 翻译中 + Baidu Takes FPGA Approach to Accelerating SQL at Scale =================== From 7bbc09141c9ce7c51530041d7c918ac695d62713 Mon Sep 17 00:00:00 2001 From: Chao-zhi Liu Date: Sun, 11 Sep 2016 16:05:37 +0800 Subject: [PATCH 0101/2437] =?UTF-8?q?Update=2020160627=20Linux=20Applicati?= =?UTF-8?q?ons=20That=20Works=20On=20All=20Distributions=20=E2=80=93=20Are?= =?UTF-8?q?=20They=20Any=20Good.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Chao-zhi开始翻译 --- ...tions That Works On All Distributions – Are They Any Good.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md b/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md index 98967d866b..8ca553896b 100644 --- a/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md +++ b/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md @@ -1,3 +1,5 @@ +Translating by Chao-zhi + Linux Applications That Works On All Distributions – Are They Any Good? ============================================================================ From 16b76cc2982d7c0b8aa4f2cedbe478b57d51ca2c Mon Sep 17 00:00:00 2001 From: theArcticOcean <2722488604@qq.com> Date: Sun, 11 Sep 2016 16:53:12 +0800 Subject: [PATCH 0102/2437] Update 20160624 DAISY A Linux-compatible text format for the visually impaired.md Translating by theArcticOcean. --- ...Y A Linux-compatible text format for the visually impaired.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160624 DAISY A Linux-compatible text format for the visually impaired.md b/sources/tech/20160624 DAISY A Linux-compatible text format for the visually impaired.md index 4bf5ccaf42..87c84eb644 100644 --- a/sources/tech/20160624 DAISY A Linux-compatible text format for the visually impaired.md +++ b/sources/tech/20160624 DAISY A Linux-compatible text format for the visually impaired.md @@ -1,3 +1,4 @@ +Translating by 19761332 DAISY : A Linux-compatible text format for the visually impaired ================================================================= From d045c9fbcc9d74d6786903d38d7853a5f91c0c75 Mon Sep 17 00:00:00 2001 From: Chang Liu Date: Sun, 11 Sep 2016 20:31:58 +0800 Subject: [PATCH 0103/2437] [Translated]20160815 Alternative System Monitor Applet For LXDE Xfce and MATE - Multiload-ng.md (#4396) --- ...For LXDE Xfce And MATE - Multiload-ng.md | 102 ------------------ ...For LXDE Xfce And MATE - Multiload-ng.md | 101 +++++++++++++++++ 2 files changed, 101 insertions(+), 102 deletions(-) delete mode 100644 sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md create mode 100644 translated/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md diff --git a/sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md b/sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md deleted file mode 100644 index 256c075b21..0000000000 --- a/sources/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md +++ /dev/null @@ -1,102 +0,0 @@ -Alternative System Monitor Applet For LXDE, Xfce, And MATE: Multiload-ng -====== - -[Multiload-ng][1] is a GTK2 graphical system monitor for the Xfce, LXDE, and MATE panels, forked from the old GNOME Multiload applet. It can also run in a standalone window. - -![](https://2.bp.blogspot.com/-U8CFzhSPJho/V7GigDbcLWI/AAAAAAAAYS0/pJMM6Rt5-HkbKljmxzP4-v0oGGxjvH8AgCLcB/s1600/multiload-ng-lxde.png) - -Multiload-ng features: - -- supported graphs: CPU, memory, network, swap, load average, disk, and temperature; -- highly customizable; -- color schemes support; -- automatically adapts to container changes (panel or wiondow); -- little CPU / memory footprint; -- basic or detailed tooltip information; -- custom actions on double click. - -Compared to the old Multiload applet, Multiload-ng comes with an additional graph (temperature), more individual graphical customizations, like individual border color, color schemes support, it responds to mouse events with customizable actions, the orientation can be set regardless of panel orientation. - -It can also run in a standalone window, without a panel: - -![](https://1.bp.blogspot.com/-hHoipwFlHrg/V7Gw2s107zI/AAAAAAAAYTQ/fS5OtiL7VvwDEzr6qO_gdEA_qB9YvJa5gCLcB/s400/multiload-ng-standalone.png) - -Furthermore, its GitHub page says that more graphs are coming soon. - -Here's Multiload-ng in Xubuntu 16.04, with a vertical panel, with horizontal and vertical applet orientation: - -![](https://3.bp.blogspot.com/-xa0OML8T-lg/V7Gixksbt8I/AAAAAAAAYS4/Jxo-MukDh3sYlOOk9A1YGtARmte490g8ACLcB/s400/multiload-ng-xfce-horizontal.png) - -![](https://1.bp.blogspot.com/-WAD5MdDObD8/V7GixgVU0DI/AAAAAAAAYS8/uMhHJri1GJccRWvmf_tZkYeenVdxiENQwCLcB/s400/multiload-ng-xfce-vertical.png) - -The applet preferences window isn't exactly pretty, but there are plans to improve it: - -![](https://2.bp.blogspot.com/-P-ophDpc-gI/V7Gi_54b7JI/AAAAAAAAYTA/AHQck_JF_RcwZ1KbgHbaO2JRt24ZZdO3gCLcB/s320/multiload-ng-preferences.png) - -Multiload-ng currently uses GTK2, so it won't work with Xfce or MATE (panels) if they are built with GTK3. - -As far as Ubuntu is concerned, only Ubuntu MATE 16.10 uses GTK3. However, the MATE System Monitor applet is also a fork of Multiload GNOME applet, so they share most features (minus the extra customization provided by Multiload-ng, and the temperature graph). - -The applet wishlist page mentions plans for a GTK3 port, and various other improvements, like more sources for the temperature graph, the ability to show both decimal and binary units, and more. - -### Install Multiload-ng - - -Note that Multiload-ng can't be built on Lubuntu 14.04 due to its dependencies. - -Multiload-ng is available in the main WebUpd8 PPA (for Ubuntu 14.04 - 16.04 / Linux Mint 17.x and 18). To add the PPA and update the software sources, use the following commands: - -``` -sudo add-apt-repository ppa:nilarimogard/webupd8 -sudo apt update -``` - -Then, install the applet using the following command: - -- for LXDE (Lubuntu): - -``` -sudo apt install lxpanel-multiload-ng-plugin -``` - -- for Xfce (Xubuntu, Linux Mint Xfce): - -``` -sudo apt install xfce4-multiload-ng-plugin -``` - -- for MATE (Ubuntu MATE, Linux Mint MATE): - -``` -sudo apt install mate-multiload-ng-applet -``` - -- standalone (doesn't require a panel): - -``` -sudo apt install multiload-ng-standalone -``` - -Once installed, add it to the panel like any other applet. Note that in LXDE, Multiload-ng won't show up in the applet list until the panel is restarted. You can do this by restarting the session (logout/login) or by restarting the panel using the following command: - -``` -lxpanelctl restart -``` - -Multiload-ng Standalone can be launched from the menu, like a regular application. - -To download the source, report bugs, etc., see the Multiload-ng [GitHub page][2]. - --------------------------------------------------------------------------------- - -via: http://www.webupd8.org/2016/08/alternative-system-monitor-applet-for.html - -作者:[Andrew][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.webupd8.org/p/about.html -[1]: https://github.com/udda/multiload-ng -[2]: https://github.com/udda/multiload-ng diff --git a/translated/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md b/translated/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md new file mode 100644 index 0000000000..1c68316953 --- /dev/null +++ b/translated/tech/20160815 Alternative System Monitor Applet For LXDE Xfce And MATE - Multiload-ng.md @@ -0,0 +1,101 @@ +LXDE、Xfce 及 MATE 桌面环境下的又一系统监视器应用:Multiload-ng +====== + +[Multiload-ng][1] 是一个 GTK2 图形化系统监视器应用,可集成到 Xfce、LXDE 及 MATE 的桌面面板中, 它 fork 自原来的 GNOME Multiload 应用。它也可以运行在一个独立的窗口中。 + +![](https://2.bp.blogspot.com/-U8CFzhSPJho/V7GigDbcLWI/AAAAAAAAYS0/pJMM6Rt5-HkbKljmxzP4-v0oGGxjvH8AgCLcB/s1600/multiload-ng-lxde.png) + +Multiload-ng 的特点有: + +- 支持图形化: CPU,内存,网络,交换空间,平均负载,磁盘以及温度; +- 高度可定制; +- 支持配色方案; +- 自动适应容器(面板或窗口)的改变; +- 极低的 CPU 和内存占用; +- 提供基本或详细的提示信息; +- 可自定义双击触发的动作。 + +相比于原来的 Multiload 应用,Multiload-ng 含有一个额外的图形块(温度),更多独立的图形自定义选项,例如独立的边框颜色,支持配色方案,可根据自定义的动作对鼠标的点击做出反应,图形块的方向可以被设定为与面板的方向无关。 + +它也可以运行在一个独立的窗口中,而不需要面板: + +![](https://1.bp.blogspot.com/-hHoipwFlHrg/V7Gw2s107zI/AAAAAAAAYTQ/fS5OtiL7VvwDEzr6qO_gdEA_qB9YvJa5gCLcB/s400/multiload-ng-standalone.png) + +另外,它的 GitHub page 上说还会带来更多的图形块支持。 + +下图展示的是在带有一个垂直面板的 Xubuntu 16.04 中,该应用分别处于水平和垂直方向的效果: + +![](https://3.bp.blogspot.com/-xa0OML8T-lg/V7Gixksbt8I/AAAAAAAAYS4/Jxo-MukDh3sYlOOk9A1YGtARmte490g8ACLcB/s400/multiload-ng-xfce-horizontal.png) + +![](https://1.bp.blogspot.com/-WAD5MdDObD8/V7GixgVU0DI/AAAAAAAAYS8/uMhHJri1GJccRWvmf_tZkYeenVdxiENQwCLcB/s400/multiload-ng-xfce-vertical.png) + +这个应用的偏好设置窗口虽然不是非常好看,但有很多方式去改进它: + +![](https://2.bp.blogspot.com/-P-ophDpc-gI/V7Gi_54b7JI/AAAAAAAAYTA/AHQck_JF_RcwZ1KbgHbaO2JRt24ZZdO3gCLcB/s320/multiload-ng-preferences.png) + +Multiload-ng 当前使用的是 GTK2,所以它不能在构建自 GTK3 下的 Xfce 或 MATE 桌面环境(面板)下工作。 + +对于 Ubuntu 系统而言,只有 Ubuntu MATE 16.10 使用 GTK3。但是鉴于 MATE 的系统监视器应用也是 Multiload GNOME 的一个分支,所以它们共享大多数的特点(除了 Multiload-ng 提供的额外自定义选项和温度图形块)。 + +该应用的 [愿望清单][2] 中提及到了计划支持 GTK3 的集成以及各种各样的改进,例如温度块资料的更多来源,能够显示十进制(KB, MB, GB...)或二进制(KiB, MiB, GiB...)单位等等。 + +### 安装 Multiload-ng + +请注意因为依赖的关系, Multiload-ng 不能在 Lubuntu 14.04 上构建。 + +Multiload-ng 可在 WebUpd8 的主 PPA (针对 Ubuntu 14.04 - 16.04 / Linux Mint 17.x 和 18)中获取到。可以使用下面的命令来添加 PPA 并更新软件源: + +``` +sudo add-apt-repository ppa:nilarimogard/webupd8 +sudo apt update +``` + +然后可以使用下面的命令来安装这个应用: + +- 对于 LXDE (Lubuntu): + +``` +sudo apt install lxpanel-multiload-ng-plugin +``` + +- 对于 Xfce (Xubuntu,Linux Mint Xfce): + +``` +sudo apt install xfce4-multiload-ng-plugin +``` + +- 对于 MATE (Ubuntu MATE,Linux Mint MATE): + +``` +sudo apt install mate-multiload-ng-applet +``` + +- 独立安装 (不需要集成到面板): + +``` +sudo apt install multiload-ng-standalone +``` + +一旦安装完毕,便可以像其他应用那样添加到桌面面板中了。需要注意的是在 LXDE 中,Multiload-ng 不能马上出现在面板清单中,除非面板被重新启动。你可以通过重启会话(登出后再登录)或者使用下面的命令来重启面板: + +``` +lxpanelctl restart +``` + +独立的 Multiload-ng 应用可以像其他正常应用那样从菜单中启动。 + +如果要下载源码或报告 bug 等,请看 Multiload-ng 的 [GitHub page][3]。 +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/08/alternative-system-monitor-applet-for.html + +作者:[Andrew][a] +译者:[FSSlc](https://github.com/FSSlc) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: https://github.com/udda/multiload-ng +[2]: https://github.com/udda/multiload-ng/wiki/Wishlist +[3]: https://github.com/udda/multiload-ng From a8215f6f27a146469e9e25fa36c91f1adf0add23 Mon Sep 17 00:00:00 2001 From: wxy Date: Mon, 12 Sep 2016 07:27:49 +0800 Subject: [PATCH 0104/2437] PUB:20160823 Publish Your Project Documentation with GitHub Pages @bestony --- ...Project Documentation with GitHub Pages.md | 49 ++++++++++++++++++ ...Project Documentation with GitHub Pages.md | 50 ------------------- 2 files changed, 49 insertions(+), 50 deletions(-) create mode 100644 published/20160823 Publish Your Project Documentation with GitHub Pages.md delete mode 100644 translated/tech/20160823 Publish Your Project Documentation with GitHub Pages.md diff --git a/published/20160823 Publish Your Project Documentation with GitHub Pages.md b/published/20160823 Publish Your Project Documentation with GitHub Pages.md new file mode 100644 index 0000000000..762597195e --- /dev/null +++ b/published/20160823 Publish Your Project Documentation with GitHub Pages.md @@ -0,0 +1,49 @@ +使用 Github Pages 发布你的项目文档 +===== + +你可能比较熟悉[如何用 Github Pages 来分享你的工作][3],又或许你看过[一堂][4]教你建立你的第一个 Github Pages 网站的教程。近期 Github Pages 的改进使得[从不同的数据源来发布您的网站][5]更加的方便,其中的来源之一就是你的仓库的 /docs 目录。 + +文档的质量是一个软件项目健康发展的标志。对于开源项目来说,维护一个可靠而不出错的知识库、详细说明所有的细节是至关重要的。精心策划的文档可以让增加项目的亲切感,提供一步步的指导并促进各种方式的合作可以推动开源软件开发的协作进程。 + +在 Web 上托管你的文档是一个消耗时间的挑战,而且对于它的发布和维护也没有省事的办法,然而这是并非不可避免的。面对多种不同的发布工具,又是 FTP 服务器,又是数据库,文件以各种不同的方式存放在不同的位置下,而这些都需要你手动来调整。需要说明的是,传统的 Web 发布方式提供了无与伦比的灵活性和性能,但是在许多情况下,这是以牺牲简单易用为代价的。 + +当作为文档使用时,麻烦更少的方式显然更容易去维护。 + +[GitHub Pages][2] 可以以指定的方式为你的项目创建网站,这使得它天然地适合发布和维护文档。因为 Github Pages 支持 Jekyll,所以你可以使用纯文本或 Markdown 来书写你的文档,从而降低你维护的成本、减少维护时的障碍。Jekyll 还支持许多有用的工具比如变量、模板、以及自动代码高亮等等,它会给你更多的灵活性而不会增加复杂性,这些你在一些笨重的平台是见不到的。 + +最重要的是,在 Github 上使用 GitHub Pages 意味着你的文档和代码可以使用诸如 Issues 和 Pull Requests 来确保它得到应有的高水平维护,而且因为 GitHub Pages 允许您发布代码库主分支上的 /docs 目录,这样您就可以在同一分支同时维护你的代码库及其文档。 + +### 现在开始! + +发布你的第一个文档页面只需要短短几分钟。 + +1. 在你的仓库的主分支里创建一个 /docs/index.md 文件。 + +2. 把你的内容以 Jekyll 格式添加进去,并提交你的修改。 + + ![](https://cloud.githubusercontent.com/assets/3477155/17778793/47c5a586-6533-11e6-982c-ebd41ec6968c.gif) + +3. 查看你的仓库的设置分支然后选择主分支 /docs 目录,将其设置为 GitHub Pages 的源 ,点击保存,你就搞定了。 + +![](https://cloud.githubusercontent.com/assets/3477155/17778792/47c2ecc4-6533-11e6-828a-91980daa7297.gif) + +GitHub Pages 将会从你的 /docs 目录中读取内容,转换 index.md 为 HTML。然后把它发布到你的 GitHub Pages 的 URL 上。 + +这样将会创建并输出一个最基础的 HTML ,而且你可以使用 Jekyll 的自定义模板、CSS 和其他特性。如果想要看所有的可能,你可以看看 [GitHub Pages Showcase][1]。 + +-------------------------------------------------------------------------------- + +via: https://github.com/blog/2233-publish-your-project-documentation-with-github-pages + +作者:[loranallensmith][a] +译者:[Bestony](https://github.com/bestony) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://github.com/loranallensmith +[1]: https://github.com/showcases/github-pages-examples +[2]: https://pages.github.com/ +[3]: https://www.youtube.com/watch?v=2MsN8gpT6jY +[4]: https://www.youtube.com/watch?v=RaKX4A5EiQo +[5]: https://help.github.com/articles/configuring-a-publishing-source-for-github-pages/ diff --git a/translated/tech/20160823 Publish Your Project Documentation with GitHub Pages.md b/translated/tech/20160823 Publish Your Project Documentation with GitHub Pages.md deleted file mode 100644 index 2e1b2746f9..0000000000 --- a/translated/tech/20160823 Publish Your Project Documentation with GitHub Pages.md +++ /dev/null @@ -1,50 +0,0 @@ -使用Github Pages 发布你的项目文档 -===== - -你可能比较熟悉[Github Pages 如何帮你分享你的工作][3] 又或许你加入了[一堂课][4] 来帮你建立你的第一个 Github Pages 的网站。近期 Github Pages 的改进使[从不同的源来发布您的网站][5]更加的方便.其中的来源之一就是你的仓库的 /docs 目录. - -文档的质量是任何一个健康的软件项目的标志。对于开源项目来说,维护一个强大的知识大纲详细说明所有的细节是至关重要的。维护很好的文档可以增强项目的可用性,提供一步一步的指导和培养不同步合作可以推动开源软件开发的协作进程。 - -在Web上托管你的文档是一个消耗时间的挑战而且发布和维护不会有什么有用的经验,而这是可以避免的。面对不同的发布工具,比如 FTP 服务器 和 数据库 意味着文件存在于多个不同的状态下和不同的地点,而这些都需要手动来同步。需要澄清的是,传统的 Web 发布提供了无与伦比的灵活性和性能,但是在许多情况下,这是以牺牲简单为代价的。 - -当作为文档时,一个路径显然更容易去接触到。 - - -[GitHub Pages][2] 给你一个指定的路径来为你的项目创建网站,这使得他天然适合发布和维护文档。因为 Github Pages 支持 Jekyll ,所以你可以使用纯文本或 Markdown 来书写你的文档,降低你维护的成本,减少维护时的障碍。Jekyll还支持许多有用的工具比如变量、模板、以及自动的代码高亮,他会给你更多的灵活性而不会增加复杂性,这些你在一些笨重的平台是见不到的。 - -最重要的是,在 Github 上使用 GitHub Pages 意味着你的文档和代码可以使用诸如 Issues 和 Pull Requests 来确保它受到到应有的高水平的维护,而且因为 GitHub Pages 允许您发布主分支的 /docs 目录,您可以在同一分支维护你的代码库及其文档。 - -### 现在开始! - -发布你的第一个文档页面只需要短短几分钟。 - -1. 在你的仓库的主分支里创建一个 /docs/index.md 文件。 - -2. 把你的内容以 Jekyll 格式添加进去,并提交你的修改。 - -![](https://cloud.githubusercontent.com/assets/3477155/17778793/47c5a586-6533-11e6-982c-ebd41ec6968c.gif) - -3. 查看你的仓库的设置分支然后选择主分支 /docs 目录,将其设置为 GitHub Pages 的源 ,点击保存,你就搞定了。 - -![](https://cloud.githubusercontent.com/assets/3477155/17778792/47c2ecc4-6533-11e6-828a-91980daa7297.gif) - -GitHub Pages 将会从你的 /docs 目录中读取内容,转换 index.md 为 HTML。然后把它发布到你的 GitHub Pages URL 上。 - -这样将会创建一个最基础的HTML并输出,而且你可以在Jekyll自定义模板、CSS 和其他特性.如果想要看所有的可能,你可以看看 [GitHub Pages Showcase][1]。 - --------------------------------------------------------------------------------- - -via: https://github.com/blog/2233-publish-your-project-documentation-with-github-pages - -作者:[ loranallensmith ][a] -译者:[Bestony](https://github.com/bestony) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://github.com/loranallensmith -[1]: https://github.com/showcases/github-pages-examples -[2]: https://pages.github.com/ -[3]: https://www.youtube.com/watch?v=2MsN8gpT6jY -[4]: https://www.youtube.com/watch?v=RaKX4A5EiQo -[5]: https://help.github.com/articles/configuring-a-publishing-source-for-github-pages/ From 803650a93151f12a14e94ebf9d576f5bb19296e5 Mon Sep 17 00:00:00 2001 From: "Cathon.ZHD" Date: Mon, 12 Sep 2016 10:54:44 +0800 Subject: [PATCH 0105/2437] translated --- ...08 Simple Python Framework from Scratch.md | 462 ----------------- ...08 Simple Python Framework from Scratch.md | 466 ++++++++++++++++++ 2 files changed, 466 insertions(+), 462 deletions(-) delete mode 100644 sources/tech/20160608 Simple Python Framework from Scratch.md create mode 100644 translated/tech/20160608 Simple Python Framework from Scratch.md diff --git a/sources/tech/20160608 Simple Python Framework from Scratch.md b/sources/tech/20160608 Simple Python Framework from Scratch.md deleted file mode 100644 index b4e9d20c1b..0000000000 --- a/sources/tech/20160608 Simple Python Framework from Scratch.md +++ /dev/null @@ -1,462 +0,0 @@ -[Cathon is translating...] -Simple Python Framework from Scratch -=================================== - -Why would you want to build a web framework? I can think of a few reasons: - -- Novel idea that will displace other frameworks. -- Get some mad street cred. -- Your problem domain is so unique that other frameworks don't fit. -- You're curious how web frameworks work because you want to become a better web developer. - -I'll focus on the last point. This post aims to describe what I learned by writing a small server and framework by explaining the design and implementation process step by step, function by function. The complete code for this project can be found in this [repository][1]. - -I hope this encourages other people to try because the it was fun, taught me a lot about how web applications work, and it was a lot easier than I thought! - -### Scope - -Frameworks handle things like the request-response cycle, authentication, database access, generating templates, and lots more. Web developers use frameworks because most web applications share a lot of functionality and it doesn't make sense to re-implement all of that for every project. - -Bigger frameworks like Rails or Django operate on a high level of abstraction and are said to be "batteries-included". It would take thousands of man-hours to implement all of those features so it's important to focus on tiny subset for this project. Before setting down a single line of code I created a list of features and constraints. - -Features: - -- Must handle GET and POST HTTP requests. You can get a brief overview of HTTP in [this wiki article][2]). -- Must be asynchronous (I'm loving the Python 3 asyncio module). -- Must include simple routing logic along with capturing parameters. -- Must provide a simple user-facing API like other cool microframeworks. -- Must handle authentication, because it's cool to learn that too (saved for Part 2). - -Constraints: - -- Will only handle a small subset of HTTP/1.1: no transfer-encoding, no http-auth, no content-encoding (gzip), no [persistant connections][3]. -- No MIME-guessing for responses - users will have to set this manually. -- No WSGI - just simple TCP connection handling. -- No database support. - -I decided a small use case would make the above more concrete. It would also demonstrate the framework's API: - -``` -from diy_framework import App, Router -from diy_framework.http_utils import Response - - -# GET simple route -async def home(r): - rsp = Response() - rsp.set_header('Content-Type', 'text/html') - rsp.body = 'test' - return rsp - - -# GET route + params -async def welcome(r, name): - return "Welcome {}".format(name) - -# POST route + body param -async def parse_form(r): - if r.method == 'GET': - return 'form' - else: - name = r.body.get('name', '')[0] - password = r.body.get('password', '')[0] - - return "{0}:{1}".format(name, password) - -# application = router + http server -router = Router() -router.add_routes({ - r'/welcome/{name}': welcome, - r'/': home, - r'/login': parse_form,}) - -app = App(router) -app.start_server() -``` - -The user is supposed to be able to define a few asynchronous functions that either return strings or Response objects, then pair those functions up with strings that represent routes, and finally start handling requests with a single function call (start_server). - -Having created the designs, I condensed it into abstract things I needed to code up: - -- Something that should accept TCP connections and schedule an asynchronous function to handle them. -- Something to parse raw text into some kind of abstracted container. -- Something to decide which function should be called for each request. -- Something that bundles all of the above together and presents a simple interface to a developer. - -I started by writing tests that were essentially contracts describing each piece of functionality. After a few refactorings, the layout settled into a handful of composable pieces. Each piece is relatively decoupled from the other components, which is especially beneficial in this case because each piece can be studied on its own. The pieces are concrete reflections of the abstractions that I listed above: - -- A HTTPServer object that holds onto a Router object and a http_parser module and uses them to initialize... -- HTTPConnection objects, where each one represents a single client HTTP connection and takes care of the request-response cycle: parses incoming bytes into a Request object using the http_parser module; uses an instance of Router to find the correct function to call to generate a response; finally send the response back to the client. -- A pair of Request and Response objects that give a user a comfortable way to work with what are in essence specially formatted strings of bytes. The user shouldn't be aware of the correct message format or delimiters. -- A Router object that contains route:function pairs. It exposes a way to add these pairs and a way, given a URL path, to find a function. -- Finally, an App object that contains configuration and uses it to instantiate a HTTPServer instance. - -Let's go over each of these pieces, starting from HTTPConnection. - -### Modelling Asynchronous Connections - -To satisfy the contraints, each HTTP request is a separate TCP connection. This makes request handling slower because of the relatively high cost of establishing multiple TCP connections (cost of DNS lookup, hand-shake, [Slow Start][4], etc.) but it's much easier to model. For this task, I chose the fairly high-level [asyncio-stream][5] module that rests on top of [asyncio's transports and protocols][6]. I recommend checking out the code for it in the stdlib because it's a joy to read! - -An instance of HTTPConnection handles multiple tasks. First, it reads data incrementally from a TCP connection using an asyncio.StreamReader object and stores it in a buffer. After each read operation, it tries to parse whatever is in the buffer and build out a Request object. Once it receives the whole request, it generates a reply and sends it back to the client through an asyncio.StreamWriter object. It also handles two more tasks: timing out a connection and handling errors. - -You can view the complete code for the class [here][7]. I'll introduce each part of the code separately, with the docstrings removed for brevity: - -``` -class HTTPConnection(object): - def init(self, http_server, reader, writer): - self.router = http_server.router - self.http_parser = http_server.http_parser - self.loop = http_server.loop - - self._reader = reader - self._writer = writer - self._buffer = bytearray() - self._conn_timeout = None - self.request = Request() -``` - -The init method is boring: it just collects objects to work with later. It stores a router, an http_parser, and loop objects, which are used to generate responses, parse requests, and schedule things in the event loop. - -Next, it stores the reader-writer pair, which together represent a TCP connection, and an empty [bytearray][8] that serves as buffer for raw bytes. _conn_timeout stores an instance of [asyncio.Handle][9] that is used to manage the timeout logic. Finally, it also stores a single instance of Request. - -The following code handles the core functionality of receiving and sending data: - -``` -async def handle_request(self): - try: - while not self.request.finished and not self._reader.at_eof(): - data = await self._reader.read(1024) - if data: - self._reset_conn_timeout() - await self.process_data(data) - if self.request.finished: - await self.reply() - elif self._reader.at_eof(): - raise BadRequestException() - except (NotFoundException, - BadRequestException) as e: - self.error_reply(e.code, body=Response.reason_phrases[e.code]) - except Exception as e: - self.error_reply(500, body=Response.reason_phrases[500]) - - self.close_connection() -``` - -Everything here is closed in a try-except block so that any exceptions thrown during parsing a request or replying to one are caught and an error response is sent back to the client. - -The request is read inside of a while loop until the parser sets self.request.finished = True or until the client closed the connection, signalled by self._reader.at_eof() method returning True. The code tries to read data from a StreamReader on every iteration of the loop and incrementally build out self.request through the call to self.process_data(data). The connection timeout timer is reset everytime the loop reads any data. - -There's an error in there - can you spot it? I'll get back to it shortly. I also have to note that this loop can potentially eat up all of the CPU because self._reader.read() returns b'' objects if there is nothing to read - meaning the loop will cycle millions of times, doing nothing. A possible solution here would be to wait a bit of time in a non-blocking fashion: await asyncio.sleep(0.1). I'll hold off on optimizing this until it's needed. - -Remember the error that I mentioned at the start of the previous paragraph? The self._reset_conn_timeout() method is called only when data is read from the StreamReader. The way it's set up now means that the timeout is not initiated until the first byte arrives. If a client opens a connection to the server and doesn't send any data - it never times out. This could be used to exhaust system resources and cause a denial of service. The fix is to simply call self._reset_conn_timeout() in the init method. - -When the request is received or when the connection drops, the code hits an if-else block. This block determines if the parser, having received all the data, finished parsing the request? Yes? Great - generate a reply and send it back! No? Uh-oh - something's wrong with the request - raise an exception! Finally, self.close_connection is called to perform cleanup. - -Parsing the parts of a request is done inside the self.process_data method. It's a very short and simple method that's easy to test: - -``` -async def process_data(self, data): - self._buffer.extend(data) - - self._buffer = self.http_parser.parse_into( - self.request, self._buffer) -``` - -Each call accumulates data into self._buffer and then tries to parse whatever has gathered inside of the buffer using self.http_parser. It's worth pointing out here that this code exhibits a pattern called [Dependency Injection][10]. If you remember the init function, you know that I pass in a http_server object, which contains a http_parser object. In this case, the http_parser object is a module that is part of the diy_framework package, but it could potentially be anything else that has a parse_into function that accepts a Request object and a bytearray. This is useful for two reasons. First, it means that this code is easier to extend. Someone could come along and want to use HTTPConnection with a different parser - no problem - just pass it in as an argument! Second, it makes testing much easier because the http_parser is not hard coded anywhere so replacing it with a dummy or [mock][11] object is super easy. - -The next interesting piece is the reply method: - -``` -async def reply(self): - request = self.request - handler = self.router.get_handler(request.path) - - response = await handler.handle(request) - - if not isinstance(response, Response): - response = Response(code=200, body=response) - - self._writer.write(response.to_bytes()) - await self._writer.drain() -``` - -Here, an instance of HTTPConnection uses a router object it got from the HTTPServer (another example of Dependency Injection) to obtain an object that will generate a response. A router can be anything that has a get_handler method that accepts a string and returns a callable or raises a NotFoundException. The callable object is then used to process the request and generate a response. Handlers are written by the users of the framework, as outlined in use case above, and should return either strings or Response objects. Response objects give us a nice interface so the simple if block ensures that whatever a handler returns, the code further along ends up with a uniform Response object. - -Next, the StreamWriter instance assigned to self._writer is called to send back a string of bytes to the client. Before the function returns, it awaits at await self._writer.drain(), which ensures that all the data has been sent to the client. This ensures that a call to self._writer.close won't happen when there is still unsent data in the buffer. - -There are two more interesting parts of the HTTPConnection class: a method that closes the connection and a group of methods that handle the timeout mechanism. First, closing a connection is accomplished by this little function: - -``` -def close_connection(self): - self._cancel_conn_timeout() - self._writer.close() -``` - -Any time a connection is to be closed, the code has to first cancel the timeout to clean it out of the event loop. - -The timeout mechanism is a set of three related functions: a function that acts on the timeout by sending an error message to the client and closing the connection; a function that cancels the current timeout; and a function that schedules the timeout. The first two are simple and I add them for completeness, but I'll explain the third one — _reset_conn_timeout — in more detail. - -``` -def _conn_timeout_close(self): - self.error_reply(500, 'timeout') - self.close_connection() - -def _cancel_conn_timeout(self): - if self._conn_timeout: - self._conn_timeout.cancel() - -def _reset_conn_timeout(self, timeout=TIMEOUT): - self._cancel_conn_timeout() - self._conn_timeout = self.loop.call_later( - timeout, self._conn_timeout_close) -``` - -Everytime _reset_conn_timeout is called, it first cancels any previously set asyncio.Handle object assigned to self._conn_timeout. Then, using the BaseEventLoop.call_later function, it schedules the _conn_timeout_close function to run after timeout seconds. If you remember the contents of the handle_request function, you'll know that this function gets called every time any data is received. This cancels any existing timeout and re-schedules the _conn_timeout_close function timeout seconds in the future. As long as there is data coming, this cycle will keep resetting the timeout callback. If no data is received inside of timeout seconds, the _conn_timeout_close finally gets called. - -### Creating the Connections - -Something has to create HTTPConnection objects and it has to do it correctly. This task is delegated to the HTTPServer class, which is a very simple container that helps to store some configuration (the parser, router, and event loop instances) and then use that configuration to create instances of HTTPConnection: - -``` -class HTTPServer(object): - def init(self, router, http_parser, loop): - self.router = router - self.http_parser = http_parser - self.loop = loop - - async def handle_connection(self, reader, writer): - connection = HTTPConnection(self, reader, writer) - asyncio.ensure_future(connection.handle_request(), loop=self.loop) -``` - -Each instance of HTTPServer can listen on one port. It has an asynchronous handle_connection method that creates instances of HTTPConnection and schedules them for execution in the event loop. This method is passed to [asyncio.start_server][12] and serves as a callbacks: it's called every time a TCP connection is initiated with a StreamReader and StreamWriter as the arguments. - -``` - self._server = HTTPServer(self.router, self.http_parser, self.loop) - self._connection_handler = asyncio.start_server( - self._server.handle_connection, - host=self.host, - port=self.port, - reuse_address=True, - reuse_port=True, - loop=self.loop) -``` - -This forms the core of how the application works: asyncio.start_server accepts TCP connections and calls a method on a preconfigured HTTPServer object. This method handles all the logic for a single connection: reading, parsing, generating and sending a reply back to the client, and closing the connection. It focuses on IO logic and coordinates parsing and generating a reply. - -With the core IO stuff out of the way lets move on over to... - -### Parsing Requests - -The users of this tiny framework are spoiled and don't want to work with bytes. They want a higher level of abstraction - a more convenientt way of working with requests. The tiny framework includes a simple HTTP parser that transforms bytes into Request objects. - -These Request objects are containers that looks like: - -``` -class Request(object): - def init(self): - self.method = None - self.path = None - self.query_params = {} - self.path_params = {} - self.headers = {} - self.body = None - self.body_raw = None - self.finished = False -``` - -It has everything that a developer needs in order to accept data coming from a client in an easily understandable package. Well, everything except cookies, which are crucial in order to do things like authentication. I'll leave that for part 2. - -Each HTTP request contains certain required pieces - like the path or the method. It also contains certain optional pieces like the body, headers, or URL parameters. Furthermore, owing to the popularity of REST, the URL, minus the URL parameters, may also contain pieces of information eg. "/users/1/edit" contains a user's id. - -Each part of a request has to be identified, parsed, and assigned to the correct part of a Request object. The fact that HTTP/1.1 is text protocol simplifies things (HTTP/2 is binary protocol - whole 'nother level of fun). - -The http_parser module is a group of functions inside because the parser does not need to keep track of state. Instead, the calling code has to manage a Request object and pass it into the parse_into function along with a bytearray containing the raw bytes of a request. To this end, the parser modifies both the request object as well as the bytearray buffer passed to it. The request object gets fuller and fuller while the bytearray buffer gets emptier and emptier. - -The core functionality of the http_parser module is inside the parse_into function: - -``` -def parse_into(request, buffer): - _buffer = buffer[:] - if not request.method and can_parse_request_line(_buffer): - (request.method, request.path, - request.query_params) = parse_request_line(_buffer) - remove_request_line(_buffer) - - if not request.headers and can_parse_headers(_buffer): - request.headers = parse_headers(_buffer) - if not has_body(request.headers): - request.finished = True - - remove_intro(_buffer) - - if not request.finished and can_parse_body(request.headers, _buffer): - request.body_raw, request.body = parse_body(request.headers, _buffer) - clear_buffer(_buffer) - request.finished = True - return _buffer -``` - -As you can see in the code above, I divided the parsing process into three parts: parsing the request line (the line that goes GET /resource HTTP/1.1), parsing the headers, and parsing the body. - -The request line contains the HTTP method and the URL. The URL in turn contains yet more information: the path, url parameters, and developer defined url parameters. Parsing out the method and URL is easy - it's a matter of splitting the string appropriately. The urlparse.parse function is used to parse out the URL parameters, if any, from the URL. The developer defined url parameters are extracted using regular expressions. - -Next up are the HTTP headers. These are simply lines of text that are key-value pairs. The catch is that a there may be multiple headers of the same name but with different values. An important header to watch out for is the Content-Length header that specifies the length of the body (not the whole request, just the body!), which is important in determining whether to parse the body at all. - -Finally, the parser looks at the HTTP method and headers and decides whether to parse the request's body. - -### Routing! - -The router is a bridge between the framework and the user in the sense that the user creates a Router object and fills it with path/function pairs using the appropriate methods and then gives the Router object to the App. The App object in turn uses the get_handler function to get a hold of a callable that generates a response. In short, the router is responsible for two things - storing pairs of paths/functions and handing back a pair to whatever asks for one. - -There are two methods in the Router class that allow an end-developer to add routes: add_routes and add_route. Since add_routes is a convenient wrapper around add_route, I'll skip describing it and focus on add_route: - -``` -def add_route(self, path, handler): - compiled_route = self.class.build_route_regexp(path) - if compiled_route not in self.routes: - self.routes[compiled_route] = handler - else: - raise DuplicateRoute -``` - -This method first "compiles" a route — a string like '/cars/{id}' — into a compiled regexp object using the Router.build_route_regexp class method. These compiled regexp objects serve both to match a request's path and to extract developer defined URL parameters specified by that route. Next there's a check that raises an exception if the same route already exists, and finally the route/handler pair is added to a simple dictionary — self.routes. - -Here's how the Router "compiles" routes: - -``` -@classmethod -def build_route_regexp(cls, regexp_str): - """ - Turns a string into a compiled regular expression. Parses '{}' into - named groups ie. '/path/{variable}' is turned into - '/path/(?P[a-zA-Z0-9_-]+)'. - - :param regexp_str: a string representing a URL path. - :return: a compiled regular expression. - """ - def named_groups(matchobj): - return '(?P<{0}>[a-zA-Z0-9_-]+)'.format(matchobj.group(1)) - - re_str = re.sub(r'{([a-zA-Z0-9_-]+)}', named_groups, regexp_str) - re_str = ''.join(('^', re_str, '$',)) - return re.compile(re_str) -``` - -The method uses regular expressions to substitute all occurrences of "{variable}" with named regexp groups: "(?P...)". Then it adds the ^ and $ regexp signifiers at the beginning and end of the resulting string, and finally compiles regexp object out of it. - -Storing a route is just half the battle, here's how to get one back: - -``` -def get_handler(self, path): - logger.debug('Getting handler for: {0}'.format(path)) - for route, handler in self.routes.items(): - path_params = self.class.match_path(route, path) - if path_params is not None: - logger.debug('Got handler for: {0}'.format(path)) - wrapped_handler = HandlerWrapper(handler, path_params) - return wrapped_handler - - raise NotFoundException() -``` - -Once the App has a Request object, it also has the path part of the URL (ie. /users/15/edit). Having that, it needs a matching function to generate the response or a 404 error. get_handler takes a path as an argument, loops over routes and calls the Router.match_path class method on each one to check if any compiled regexp matches the request's path. If it does, it wraps the route's function in a HandleWrapper. The path_params dictionary contains path variables (ie. the '15' from /users/15/edit) or is left empty if the route doesn't specify any variables. Finally it returns the wrapped route's function to the App. - -If the code iterates through all the routes and none of them matches the path, the function raises a NotFoundException. - -The Route.match class method is simple: - -``` -def match_path(cls, route, path): - match = route.match(path) - try: - return match.groupdict() - except AttributeError: - return None -``` - -It uses the regexp object's match method to check if the route and path matches. It returns None on no matches. - -Finally, we have the HandleWrapper class. Its only job is to wrap an asynchronous function, store the path_params dictionary, and expose a uniform interface through the handle method: - -``` -class HandlerWrapper(object): - def init(self, handler, path_params): - self.handler = handler - self.path_params = path_params - self.request = None - - async def handle(self, request): - return await self.handler(request, **self.path_params) -``` - -### Bringing it All Together - -The last piece of the framework is the one that ties everything together — the App class. - -The App class serves to gather all the configuration details. An App object uses its single method — start_server — to create an instance of HTTPServer using some of the configuration data, and then feed it to the function asyncio.start_server more info here. The asyncio.start_server function will call the HTTPServer object's handle_connection method for every incoming TCP connection: - -``` -def start_server(self): - if not self._server: - self.loop = asyncio.get_event_loop() - self._server = HTTPServer(self.router, self.http_parser, self.loop) - self._connection_handler = asyncio.start_server( - self._server.handle_connection, - host=self.host, - port=self.port, - reuse_address=True, - reuse_port=True, - loop=self.loop) - - logger.info('Starting server on {0}:{1}'.format( - self.host, self.port)) - self.loop.run_until_complete(self._connection_handler) - - try: - self.loop.run_forever() - except KeyboardInterrupt: - logger.info('Got signal, killing server') - except DiyFrameworkException as e: - logger.error('Critical framework failure:') - logger.error(e.traceback) - finally: - self.loop.close() - else: - logger.info('Server already started - {0}'.format(self)) -``` - -### Lessons learned - -If you look at the repo, you'll notice that the whole thing is roughly 320 lines of code if you don't count the tests (it's ~540 if you do). It really surprised me that it's possible to fit so much functionality in so little code. Granted, this framework does not offer useful pieces like templates, authentication, or database access, but hey, it's something fun to work on :). This also gave me an idea how other framework like Django or Tornado work at a general level and it's already paying off in how quickly I'm able to debug things. - -This is also the first project that I did in true TDD fashion and it's just amazing how pleasant and productive the process was. Writing tests first forced me to think about design and architecture and not just on gluing bits of code together to "make it work". Don't get me wrong, there are many scenarios when the latter approach is preferred, but if you're placing a premium on low-maintenance code that you and others will be able to work in weeks or months in the future then TDD is exactly what you need. - -I explored things like [the Clean Architecture][13] and dependency injection, which is most evident in how the Router class is a higher-level abstraction (Entity?) that's close to the "core" whereas pieces like the http_parser or App are somewhere on the outer edges because they do either itty-bitty string or bytes work or interface with mid-level IO stuff. Whereas TDD forced me to think about each small part separately, this made me ask myself questions like: Does this combination of method calls compose into an understandable action? Do the class names accurately reflect the problem that I'm solving? Is it easy to distinguish different levels of abstraction in my code? - -Go ahead, write a small framework, it's a ton of fun! - --------------------------------------------------------------------------------- - -via: http://mattscodecave.com/posts/simple-python-framework-from-scratch.html - -作者:[Matt][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://mattscodecave.com/hire-me.html -[1]: https://github.com/sirMackk/diy_framework -[2]:https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol -[3]: https://en.wikipedia.org/wiki/HTTP_persistent_connection -[4]: https://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm#Slow_start -[5]: https://docs.python.org/3/library/asyncio-stream.html -[6]: https://docs.python.org/3/library/asyncio-protocol.html -[7]: https://github.com/sirMackk/diy_framework/blob/88968e6b30e59504251c0c7cd80abe88f51adb79/diy_framework/http_server.py#L46 -[8]: https://docs.python.org/3/library/functions.html#bytearray -[9]: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Handle -[10]: https://en.wikipedia.org/wiki/Dependency_injection -[11]: https://docs.python.org/3/library/unittest.mock.html -[12]: https://docs.python.org/3/library/asyncio-stream.html#asyncio.start_server -[13]: https://blog.8thlight.com/uncle-bob/2012/08/13/the-clean-architecture.html diff --git a/translated/tech/20160608 Simple Python Framework from Scratch.md b/translated/tech/20160608 Simple Python Framework from Scratch.md new file mode 100644 index 0000000000..0ebdd1c0b2 --- /dev/null +++ b/translated/tech/20160608 Simple Python Framework from Scratch.md @@ -0,0 +1,466 @@ +从零构建一个简单的 Python 框架 +=================================== + +为什么你想要自己构建一个 web 框架呢?我想,原因有以下几点: + +- 你有一个新奇的想法,觉得将会取代其他的框架 +- 你想要获得一些名气 +- 你的问题领域很独特,以至于现有的框架不太合适 +- 你对 web 框架是如何工作的很感兴趣,因为你想要成为一位更好的 web 开发者。 + +接下来的笔墨将着重于最后一点。这篇文章旨在通过对设计和实现过程一步一步的阐述告诉读者,我在完成一个小型的服务器和框架之后学到了什么。你可以在这个[代码仓库][1]中找到这个项目的完整代码。 + +我希望这篇文章可以鼓励更多的人来尝试,因为这确实很有趣。它让我知道了 web 应用是如何工作的,而且这比我想的要容易的多! + +### 范围 + +框架可以处理请求-响应周期,身份认证,数据库访问,模板生成等。Web 开发者使用框架是因为,大多数的 web 应用拥有大量相同的功能,而对每个项目都重新实现同样的功能意义不大。 + +比较大的的框架如 Rails 和 Django 实现了高层次的抽象,开箱即用功能齐备。而实现所有的这些功能可能要花费数千小时,因此在这个项目上,我们重点完成其中的一小部分。在开始写代码前,我先列举一下所需的功能以及限制。 + +功能: + +- 处理 HTTP 的 GET 和 POST 请求。你可以在[这篇 wiki][2] 中对 HTTP 有个大致的了解。 +- 实现异步操作(我喜欢 Python 3 的 asyncio 模块)。 +- 简单的路由逻辑以及参数捕获。 +- 像其他微型框架一样,提供一个简单的用户级 API 。 +- 支持身份认证,因为学会这个很酷啊(微笑)。 + +限制: + +- 将只支持 HTTP 1.1 的一个小子集,不支持传输编码,http-auth,内容编码(gzip)以及[持久化连接][3]。 +- 不支持对响应的 MIME 判断 - 用户需要手动设置。 +- 不支持 WSGI - 仅能处理简单的 TCP 连接。 +- 不支持数据库。 + +我觉得一个小的用例可以让上述内容更加具体,也可以用来演示这个框架的 API: + +``` +from diy_framework import App, Router +from diy_framework.http_utils import Response + + +# GET simple route +async def home(r): + rsp = Response() + rsp.set_header('Content-Type', 'text/html') + rsp.body = 'test' + return rsp + + +# GET route + params +async def welcome(r, name): + return "Welcome {}".format(name) + +# POST route + body param +async def parse_form(r): + if r.method == 'GET': + return 'form' + else: + name = r.body.get('name', '')[0] + password = r.body.get('password', '')[0] + + return "{0}:{1}".format(name, password) + +# application = router + http server +router = Router() +router.add_routes({ + r'/welcome/{name}': welcome, + r'/': home, + r'/login': parse_form,}) + +app = App(router) +app.start_server() +``` +' +用户需要定义一些能够返回字符串或响应对象的异步函数,然后将这些函数与表示路由的字符串配对,最后通过一个函数(start_server)调用开始处理请求。 + + +完成设计之后,我将它抽象为几个我需要编码的部分: + +- 接受 TCP 连接以及调度一个异步函数来处理这些连接的部分 +- 将原始文本解析成某种抽象容器的部分 +- 对于每个请求,用来决定调用哪个函数的部分 +- 将上述部分集中到一起,并为开发者提供一个简单接口的部分 + +我先编写一些测试,这些测试被用来描述每个部分的功能。几次重构后,整个设计被分成若干部分,每个部分之间是相对解耦的。这样就非常好,因为每个部分可以被独立地研究学习。以下是我上文列出的抽象的具体体现: + +- 一个 HTTPServer 对象,需要一个 Router 对象和一个 http_parser 模块,并使用它们来初始化。 +- HTTPConnection 对象,每一个对象表示一个单独的客户端 HTTP 连接,并且处理其请求-响应周期:使用 http_parser 模块将收到的字节流解析为一个 Request 对象;使用一个 Router 实例寻找并调用正确的函数来生成一个响应; 最后将这个响应发送回客户端。 +- 一对 Request 和 Response 对象为用户提供了一种友好的方式,来处理实质上是字节流的字符串。用户不需要知道正确的消息格式和分隔符。 +- 一个 Router 对象包含路由:函数对。它提供一个添加配对的方法,还有一个根据 URL 路径查找相应函数的方法。 +- 最后,一个 App 对象。它包含配置信息,并使用它们实例化一个 HTTPServer 实例。 + +让我们从 HTTPConnection 开始来讲解各个部分。 + +### 模拟异步连接 + +为了满足约束条件,每一个 HTTP 请求都是一个单独的 TCP 连接。这使得处理请求的速度变慢了,因为建立多个 TCP 连接需要相对高的花销(DNS 查询,TCP 三次握手,[慢启动][4]的花销等等),不过这样更加容易模拟。对于这一任务,我选择相对高级的 [asyncio-stream][5] 模块,它建立在 asyncio 的传输和协议的基础之上。我强烈推荐阅读标准库中的相应代码! + +一个 HTTPConnection 的实例能够处理多个任务。首先,它使用 asyncio.StreamReader 对象以增量的方式从 TCP 连接中读取数据,并存储在缓存中。每一个读取操作完成后,它会尝试解析缓存中的数据,并生成一个 Request 对象。一旦收到了完整的请求,它就生成一个回复,并通过 asyncio.StreamWriter 对象发送回客户端。当然,它还有两个任务:超时连接以及错误处理。 + +你可以在[这里][7]浏览这个类的完整代码。我将分别介绍代码的每一部分。为了简单起见,我移除了代码文档。 + +``` +class HTTPConnection(object): + def init(self, http_server, reader, writer): + self.router = http_server.router + self.http_parser = http_server.http_parser + self.loop = http_server.loop + + self._reader = reader + self._writer = writer + self._buffer = bytearray() + self._conn_timeout = None + self.request = Request() +``` + +这个 init 方法仅仅收集了一些对象以供后面使用。它存储了一个 router 对象,一个 http_parser 对象以及 loop 对象,分别用来生成响应,解析请求以及在事件循环中调度任务。 + +然后,它存储了代表一个 TCP 连接的读写对,和一个充当缓冲区的空[字节数组][8]。_conn_timeout 存储了一个 [asyncio.Handle][9] 的实例,用来管理超时逻辑。最后,它还存储了一个 Request 对象的实例。 + +下面的代码是用来接受和发送数据的核心功能: + +``` +async def handle_request(self): + try: + while not self.request.finished and not self._reader.at_eof(): + data = await self._reader.read(1024) + if data: + self._reset_conn_timeout() + await self.process_data(data) + if self.request.finished: + await self.reply() + elif self._reader.at_eof(): + raise BadRequestException() + except (NotFoundException, + BadRequestException) as e: + self.error_reply(e.code, body=Response.reason_phrases[e.code]) + except Exception as e: + self.error_reply(500, body=Response.reason_phrases[500]) + + self.close_connection() +``` + +所有内容被包含在 try-except 代码块中,在解析请求或响应期间抛出的异常可以被捕获,然后一个错误响应会发送回客户端。 + +在 while 循环中不断读取请求,直到解析器将 self.request.finished 设置为 True 或者客户端关闭连接,使得 self._reader_at_eof() 函数返回值为 True 为止。这段代码尝试在每次循环迭代中从 StreamReader 中读取数据,并通过调用 self.process_data(data) 函数以增量方式生成 self.request 。每次循环读取数据时,连接超时计数器被重值。 + +这儿有个错误,你发现了吗?稍后我们会再讨论这个。需要注意的是,这个循环可能会耗尽 CPU 资源,因为如果没有消息读取 self._reader.read() 函数将会返回一个空的字节对象。这就意味着循环将会不断运行,却什么也不做。一个可能的解决方法是,用非阻塞的方式等待一小段时间:await asyncio.sleep(0.1)。我们暂且不对它做优化。 + +还记得上一段我提到的那个错误吗?只有 StreamReader 读取数据时,self._reset_conn_timeout() 函数才会被调用。这就意味着,直到第一个字节到达时,timeout 才被初始化。如果有一个客户端建立了与服务器的连接却不发送任何数据,那就永远不会超时。这可能被用来消耗系统资源,或导致拒绝式服务攻击。修复方法就是在 init 函数中调用 self._reset_conn_timeout() 函数。 + +当请求接受完成或连接中断时,程序将运行到 if-else 代码块。这部分代码会判断解析器收到完整的数据后是否完成了解析。如果是,好,生成一个回复并发送回客户端。如果不是,那么请求信息可能有错误,抛出一个异常!最后,我们调用 self.close_connection 执行清理工作。 + +解析请求的部分在 self.process_data 方法中。这个方法非常简短,也易于测试: + +``` +async def process_data(self, data): + self._buffer.extend(data) + + self._buffer = self.http_parser.parse_into( + self.request, self._buffer) +``` + +每一次调用都将数据累积到 self._buffer 中,然后试着用 self.http_parser 来解析已经收集的数据。这里需要指出的是,这段代码展示了一种称为[依赖注入][10]的模式。如果你还记得 init 函数的话,应该知道我们传入了一个包含 http_parser 的 http_server 对象。在这个例子里,http_parser 对象是 diy_framework 包中的一个模块。不过它也可以是任何含有 parse_into 函数的类。这个 parse_into 函数接受一个 Request 对象以及字节数组作为参数。这很有用,原因有二。一是,这意味着这段代码更易扩展。如果有人想通过一个不同的解析器来使用 HTTPConnection,没问题,只需将它作为参数传入即可。二是,这使得测试更加容易,因为 http_parser 不是硬编码的,所以使用虚假数据或者 [mock][11] 对象来替代是很容易的。 + +下一段有趣的部分就是 reply 方法了: + +``` +async def reply(self): + request = self.request + handler = self.router.get_handler(request.path) + + response = await handler.handle(request) + + if not isinstance(response, Response): + response = Response(code=200, body=response) + + self._writer.write(response.to_bytes()) + await self._writer.drain() +``` + +这里,一个 HTTPConnection 的实例使用了 HTTPServer 中的路由对象来得到一个生成响应的对象。一个路由可以是任何一个拥有 get_handler 方法的对象,这个方法接收一个字符串作为参数,返回一个可调用的对象或者抛出 NotFoundException 异常。而这个可调用的对象被用来处理请求以及生成响应。处理程序由框架的使用者编写,如上文所说的那样,应该返回字符串或者 Response 对象。Response 对象提供了一个友好的接口,因此这个简单的 if 语句保证了无论处理程序返回什么,代码最终都得到一个统一的 Response 对象。 + +接下来,赋值给 self._writer 的 StreamWriter 实例被调用,将字节字符串发送回客户端。函数返回前,程序在 self._writer.drain() 处等待,以确保所有的数据被发送给客户端。只要缓存中还有未发送的数据,self._writer.close() 方法就不会执行。 + +HTTPConnection 类还有两个更加有趣的部分:一个用于关闭连接的方法,以及一组用来处理超时机制的方法。首先,关闭一条连接由下面这个小函数完成: + +``` +def close_connection(self): + self._cancel_conn_timeout() + self._writer.close() +``` + +每当一条连接将被关闭时,这段代码首先取消超时,然后把连接从事件循环中清除。 + +超时框架由三个有关的函数组成:第一个函数在超时后给客户端发送错误消息并关闭连接; 第二个函数取消当前的超时; 第三个函数调度超时功能。前两个函数比较简单,我将详细解释第三个函数 _reset_cpmm_timeout() 。 + +``` +def _conn_timeout_close(self): + self.error_reply(500, 'timeout') + self.close_connection() + +def _cancel_conn_timeout(self): + if self._conn_timeout: + self._conn_timeout.cancel() + +def _reset_conn_timeout(self, timeout=TIMEOUT): + self._cancel_conn_timeout() + self._conn_timeout = self.loop.call_later( + timeout, self._conn_timeout_close) +``` + +每当 _reset_conn_timeout 函数被调用时,它会先取消之前所有赋值给 self._conn_timeout 的 asyncio.Handle 对象。然后,使用 BaseEventLoop.call_later 函数让 _conn_timeout_close 函数在超时数秒后执行。如果你还记得 handle_request 函数的内容,就知道每当接收到数据时,这个函数就会被调用。这就取消了当前的超时并且重新安排 _conn_timeout_close 函数在超时数秒后执行。只要接收到数据,这个循环就会不断地重置超时回调。如果在超时时间内没有接收到数据,最后函数 _conn_timeout_close 就会被调用。 + +### 创建连接 + +我们需要创建 HTTPConnection 对象,并且正确地使用它们。这一任务由 HTTPServer 类完成。HTTPServer 类是一个简单的容器,可以存储着一些配置信息(解析器,路由和事件循环实例),并使用这些配置来创建 HTTPConnection 实例: + +``` +class HTTPServer(object): + def init(self, router, http_parser, loop): + self.router = router + self.http_parser = http_parser + self.loop = loop + + async def handle_connection(self, reader, writer): + connection = HTTPConnection(self, reader, writer) + asyncio.ensure_future(connection.handle_request(), loop=self.loop) +``` + +HTTPServer 的每一个实例能够监听一个端口。它有一个 handle_connection 的异步方法来创建 HTTPConnection 的实例,并安排它们在事件循环中运行。这个方法被传递给 [asyncio.start_server][12] 作为一个回调函数。也就是说,每当一个 TCP 连接初始化时,它就会被调用。 + +``` + self._server = HTTPServer(self.router, self.http_parser, self.loop) + self._connection_handler = asyncio.start_server( + self._server.handle_connection, + host=self.host, + port=self.port, + reuse_address=True, + reuse_port=True, + loop=self.loop) +``` + +这就是整个应用程序工作原理的核心:asyncio.start_server 接受 TCP 连接,然后在预配置的 HTTPServer 对象上调用一个方法。这个方法将处理一条 TCP 连接的所有逻辑:读取,解析,生成响应并发送回客户端,以及关闭连接。它的重点是 IO 逻辑,解析和生成响应。 + +讲解了核心的 IO 部分,让我们继续。 + +### 解析请求 + +这个微型框架的使用者被宠坏了,不愿意和字节打交道。它们想要一个更高层次的抽象 - 一种更加简单的方法来处理请求。这个微型框架就包含了一个简单的 HTTP 解析器,能够将字节流转化为 Request 对象。 + +这些 Request 对象是像这样的容器: + +``` +class Request(object): + def init(self): + self.method = None + self.path = None + self.query_params = {} + self.path_params = {} + self.headers = {} + self.body = None + self.body_raw = None + self.finished = False +``` + +它包含了所有需要的数据,可以用一种容易理解的方法从客户端接受数据。哦,不包括 cookie ,它对身份认证是非常重要的,我会将它留在第二部分。 + +每一个 HTTP 请求都包含了一些必需的内容,如请求路径和请求方法。它们也包含了一些可选的内容,如请求体,请求头,或是 URL 参数。随着 REST 的流行,除了 URL 参数,URL 本身会包含一些信息。比如,"/user/1/edit" 包含了用户的 id 。 + +一个请求的每个部分都必须被识别,解析,并正确地赋值给 Request 对象的对应属性。HTTP/1.1 是一个文本协议,这简化了很多。(HTTP/2 是一个二进制协议,这又是另一种乐趣了) + +The http_parser module is a group of functions inside because the parser does not need to keep track of state. Instead, the calling code has to manage a Request object and pass it into the parse_into function along with a bytearray containing the raw bytes of a request. To this end, the parser modifies both the request object as well as the bytearray buffer passed to it. The request object gets fuller and fuller while the bytearray buffer gets emptier and emptier. + +解析器不需要跟踪状态,因此 http_parser 模块其实就是一组函数。调用函数需要用到 Request 对象,并将它连同一个包含原始请求信息的字节数组传递给 parse_into 函数。然后解析器会修改 request 对象以及充当缓存的字节数组。字节数组的信息被逐渐地解析到 request 对象中。 + +http_parser 模块的核心功能就是下面这个 parse_into 函数: + +``` +def parse_into(request, buffer): + _buffer = buffer[:] + if not request.method and can_parse_request_line(_buffer): + (request.method, request.path, + request.query_params) = parse_request_line(_buffer) + remove_request_line(_buffer) + + if not request.headers and can_parse_headers(_buffer): + request.headers = parse_headers(_buffer) + if not has_body(request.headers): + request.finished = True + + remove_intro(_buffer) + + if not request.finished and can_parse_body(request.headers, _buffer): + request.body_raw, request.body = parse_body(request.headers, _buffer) + clear_buffer(_buffer) + request.finished = True + return _buffer +``` + +从上面的代码中可以看到,我把解析的过程分为三个部分:解析请求行(请求行有这样的格式:GET /resource HTTP/1.1),解析请求头以及解析请求体。 + +请求行包含了 HTTP 请求方法以及 URL 地址。而 URL 地址则包含了更多的信息:路径,url 参数和开发者自定义的 url 参数。解析请求方法和 URL 还是很容易的 - 合适地分割字符串就好了。函数 urlparse.parse 可以用来解析 URL 参数。开发者自定义的 URL 参数可以通过正则表达式来解析。 + +接下来是 HTTP 头部。它们是一行行由键值对组成的简单文本。问题在于,可能有多个 HTTP 头有相同的名字,却有不同的值。一个值得关注的 HTTP 头部是 Content-Length,它描述了请求体的字节长度(仅仅是请求体)。这对于决定是否解析请求体有很重要的作用。 + +最后,解析器根据 HTTP 方法和头部来决定是否解析请求体。 + +### 路由! + +在某种意义上,路由就像是连接框架和用户的桥梁,用户用合适的方法创建 Router 对象并为其设置路径/函数对,然后将它赋值给 App 对象。而 App 对象依次调用 get_handler 函数生成相应的回调函数。简单来说,路由就负责两件事,一是存储路径/函数对,二是返回需要的路径/函数对 + +Router 类中有两个允许开发者添加路由的方法,分别是 add_routes 和 add_route。因为 add_routes 就是 add_route 函数的一层封装,我们将主要讲解 add_route 函数: + +``` +def add_route(self, path, handler): + compiled_route = self.class.build_route_regexp(path) + if compiled_route not in self.routes: + self.routes[compiled_route] = handler + else: + raise DuplicateRoute +``` + +首先,这个函数使用 Router.build_router_regexp 的类方法,将一条路由规则(如 '/cars/{id}' 这样的字符串),“编译”到一个已编译的正则表达式对象。这些已编译的正则表达式用来匹配请求路径,以及解析开发者自定义的 URL 参数。如果存在一个相同的路由,程序就会抛出一个异常。最后,这个路由/处理程序对被添加到一个简单的字典(self.routes)中。 + +下面展示 Router 是如何“编译”路由的: + +``` +@classmethod +def build_route_regexp(cls, regexp_str): + """ + Turns a string into a compiled regular expression. Parses '{}' into + named groups ie. '/path/{variable}' is turned into + '/path/(?P[a-zA-Z0-9_-]+)'. + + :param regexp_str: a string representing a URL path. + :return: a compiled regular expression. + """ + def named_groups(matchobj): + return '(?P<{0}>[a-zA-Z0-9_-]+)'.format(matchobj.group(1)) + + re_str = re.sub(r'{([a-zA-Z0-9_-]+)}', named_groups, regexp_str) + re_str = ''.join(('^', re_str, '$',)) + return re.compile(re_str) +``` + +The method uses regular expressions to substitute all occurrences of "{variable}" with named regexp groups: "(?P...)". Then it adds the ^ and $ regexp signifiers at the beginning and end of the resulting string, and finally compiles regexp object out of it. + +这个方法使用正则表达式将所有出现的 "{variable}" 替换为 "(?P)"。然后在字符串头尾分别添加 ^ 和 $ 标记,最后编译正则表达式对象。 + +完成了路由存储仅成功了一半,下面是如何得到路由对应的函数: + +``` +def get_handler(self, path): + logger.debug('Getting handler for: {0}'.format(path)) + for route, handler in self.routes.items(): + path_params = self.class.match_path(route, path) + if path_params is not None: + logger.debug('Got handler for: {0}'.format(path)) + wrapped_handler = HandlerWrapper(handler, path_params) + return wrapped_handler + + raise NotFoundException() +``` + +一旦 App 对象获得一个 Request 对象,也就获得了 URL 的路径部分(如 /users/15/edit)。然后,我们需要匹配函数来生成一个响应或者 404 错误。get_handler 函数将路径作为参数,循环遍历路由,对每条路由调用 Router.match_path 类方法检查是否有已编译的正则对象与这个请求路径匹配。如果存在,我们就调用 HandleWrapper 来包装路由对应的函数。path_params 字典包含了路径变量(如 '/users/15/edit' 中的 '15'),若路由没有指定变量,字典就为空。最后,我们将包装好的函数返回给 App 对象。 + +如果遍历了所有的路由都找不到与路径匹配的,函数就会抛出 NotFoundException 异常。 + +这个 Route.match 类方法挺简单: + +``` +def match_path(cls, route, path): + match = route.match(path) + try: + return match.groupdict() + except AttributeError: + return None +``` + +它使用正则对象的 match 方法来检查路由是否与路径匹配。若果不匹配,则返回 None 。 + +最后,我们有 HandleWraapper 类。它的唯一任务就是封装一个异步函数,存储 path_params 字典,并通过 handle 方法对外提供一个统一的接口。 + +``` +class HandlerWrapper(object): + def init(self, handler, path_params): + self.handler = handler + self.path_params = path_params + self.request = None + + async def handle(self, request): + return await self.handler(request, **self.path_params) +``` + +### 合并到一起 + +框架的最后部分就是用 App 类把所有的部分联系起来。 + +App 类用于集中所有的配置细节。一个 App 对象通过其 start_server 方法,使用一些配置数据创建一个 HTTPServer 的实例,然后将它传递给 asyncio.start_server 函数。asyncio.start_server 函数会对每一个 TCP 连接调用 HTTPServer 对象的 handle_connection 方法。 + +``` +def start_server(self): + if not self._server: + self.loop = asyncio.get_event_loop() + self._server = HTTPServer(self.router, self.http_parser, self.loop) + self._connection_handler = asyncio.start_server( + self._server.handle_connection, + host=self.host, + port=self.port, + reuse_address=True, + reuse_port=True, + loop=self.loop) + + logger.info('Starting server on {0}:{1}'.format( + self.host, self.port)) + self.loop.run_until_complete(self._connection_handler) + + try: + self.loop.run_forever() + except KeyboardInterrupt: + logger.info('Got signal, killing server') + except DiyFrameworkException as e: + logger.error('Critical framework failure:') + logger.error(e.traceback) + finally: + self.loop.close() + else: + logger.info('Server already started - {0}'.format(self)) +``` + +### 总结 + +如果你查看源码,就会发现所有的代码仅 320 余行(包括测试代码共 540 余行)。这么少的代码实现了这么多的功能,让我有点惊讶。这个框架没有提供模板,身份认证以及数据库访问等功能(这些内容也很有趣哦)。这也让我知道,像 Django 和 Tornado 这样的框架是如何工作的,而且我能够快速地调试它们了。 + +这也是我按照测试驱动开发完成的第一个项目,整个过程有趣而有意义。先编写测试用例迫使我思考设计和架构,而不仅仅是把代码放到一起,让它们可以运行。不要误解我的意思,有很多时候,后者的方式更好。不过如果你想给确保这些不怎么维护的代码在之后的几周甚至几个月依然工作,那么测试驱动开发正是你需要的。 + +我研究了下[整洁架构][13]以及依赖注入模式,这些充分体现在 Router 类是如何作为一个更高层次的抽象的。Router 类是比较接近核心的,像 http_parser 和 App 的内容比较边缘化,因为它们只是完成了极小的字符串和字节流,或是中层 IO 的工作。测试驱动开发迫使我独立思考每个小部分,这使我问自己这样的问题:方法调用的组合是否易于理解?类名是否准确地反映了我正在解决的问题?我的代码中是否很容易区分出不同的抽象层? + +来吧,写个小框架,真的很有趣:) + +-------------------------------------------------------------------------------- + +via: http://mattscodecave.com/posts/simple-python-framework-from-scratch.html + +作者:[Matt][a] +译者:[Cathon](https://github.com/Cathon) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://mattscodecave.com/hire-me.html +[1]: https://github.com/sirMackk/diy_framework +[2]:https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol +[3]: https://en.wikipedia.org/wiki/HTTP_persistent_connection +[4]: https://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm#Slow_start +[5]: https://docs.python.org/3/library/asyncio-stream.html +[6]: https://docs.python.org/3/library/asyncio-protocol.html +[7]: https://github.com/sirMackk/diy_framework/blob/88968e6b30e59504251c0c7cd80abe88f51adb79/diy_framework/http_server.py#L46 +[8]: https://docs.python.org/3/library/functions.html#bytearray +[9]: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Handle +[10]: https://en.wikipedia.org/wiki/Dependency_injection +[11]: https://docs.python.org/3/library/unittest.mock.html +[12]: https://docs.python.org/3/library/asyncio-stream.html#asyncio.start_server +[13]: https://blog.8thlight.com/uncle-bob/2012/08/13/the-clean-architecture.html From 1792e61f23093693d4e435c06adb1e1fec21f000 Mon Sep 17 00:00:00 2001 From: wxy Date: Mon, 12 Sep 2016 11:59:11 +0800 Subject: [PATCH 0106/2437] PUB:20160817 Turtl - Secure, Open Source Evernote Alternative MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @chrisper ,请忠实原文,不要丢句。 --- ...ecure, Open Source Evernote Alternative.md | 79 +++++++++++++++++++ ...ecure, Open Source Evernote Alternative.md | 76 ------------------ 2 files changed, 79 insertions(+), 76 deletions(-) create mode 100644 published/20160817 Turtl - Secure, Open Source Evernote Alternative.md delete mode 100644 translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md diff --git a/published/20160817 Turtl - Secure, Open Source Evernote Alternative.md b/published/20160817 Turtl - Secure, Open Source Evernote Alternative.md new file mode 100644 index 0000000000..fec3a36be5 --- /dev/null +++ b/published/20160817 Turtl - Secure, Open Source Evernote Alternative.md @@ -0,0 +1,79 @@ +Turtl:安全、开源的 Evernote 替代品 +============================= + +Turtl 是一个安全、开源的 Evernote 替代品,在Linux、Windows、Mac 和 Android 等系统上都能使用。iOS版本仍在开发当中,Firefox 和 Chrome 也有扩展程序可以使用。 + +![](https://3.bp.blogspot.com/-cNoUUjaU4A0/V7MFKCasZJI/AAAAAAAAYTk/r7oWe-z_HB87hDvlKLViiiHUMfagnC6LQCLcB/s400/turtl-desktop-linux.png) + +这个产品仍在测试阶段,它能够让你把你的笔记(便签编辑器支持 Markdown)、网站书签、密码、文档、图片等单独放在一个隐秘地方。 + +笔记可以按模块组织起来,支持嵌套,也可以和其他 Turtl 用户分享。 + +![](https://2.bp.blogspot.com/-G-Ln3T1c2QA/V7MFmrqkukI/AAAAAAAAYTs/dXMPEB9MPREicixlEJlQVqg9SFjBX1pwgCLcB/s400/turtl-boards.png) + +你可以给你的笔记打上标签。Turtl 通过创建时间、最后修改时间或者标签来找你的笔记。 + +这个是便签编辑器(文件便签): + +![](https://1.bp.blogspot.com/-8cNHV69iCWM/V7MFX7sBlMI/AAAAAAAAYTo/ZUVTYwiCSy8uzrVKdf6NcsQZlHtylIyvgCEw/s400/turtl-edit-note.png) + +那么安全性如何呢?Turtl 会在保存数据之前加密,使用的是一个加密密钥,而密码并不保存在服务器上。只有你和你想要分享的人能获取数据。你可以从[这里][1]获得更多关于 Turtl 安全和加密的信息。 + +更新(感谢 Dimitry!):根据[错误反馈][2],Turtl 有个严重的安全性问题。Turtl 允许创建多个相同用户名的账号,却只使用密码来区分它们。希望能马上修复这个问题。 + +Turtl 团队提供了一个托管服务来同步你的记录,它是完全免费的,”除非你的文件足够大,或者你需要更好的服务”,在我写这篇文章的时候这个高级服务还不能用。 + +并且你也不一定要用这个托管服务,因为就像其桌面应用和手机应用一样,这个自托管服务器也是一个自由、开源的软件,所以你可以自己搭建一个 [Turtl 服务器][3]。 + +Turtl 没有像 Evernote 那么多的功能,但它在它的[计划][4]中也有一些新的功能,比如:支持导入/导出文本和Evernote 格式的数据、原生支持 PDF 阅读器、界面锁定等。 + +不得不提醒的是,每次启动都要输入密码,虽然安全,但有时候实在是麻烦。 + +###下载 Turtl + +[下载 Turtl 应用][5](二进制文件支持 Linux (32位/64位)、Windows 64 位、Mac 64位、Android,以及 Chrome 和Firefox 浏览器插件) + +**更新**:Turtl 用了一个新的服务器,注销然后在登录框的下面选择高级设置,把 Turtl 服务器设置为 "https://api.turtlapp.com/v2"(没有引号)。 + +下载源代码(桌面应用、移动应用和服务器)、反馈问题等,参见 Turtl 的 [GitHub][6] 项目站点。 + +Arch Linux 用户可以通过 [AUR][7] 来安装 Turtl。 + +要在 Linux 上安装,把安装包解压后运行 install.sh,安装之前请确保 ~/.local/share/applications 目录存在,若不存在请自行创建: + +``` +mkdir -p ~/.local/share/applications +``` + +注意:如果使用 sudo 命令安装,那么只有 root 用户才能使用。所以,要么不用 sudo 命令安装,要么在安装完成后修改权限。你可以参考[AUR 软件包的设置][8]来了解如何修改权限。 + +使用如下命令把 Turtl 安装到 ~/turtl 文件夹下(假定你已经把安装包解压在你的家目录下了): + +```` +~/turtl-*/install.sh ~/turtl +``` + +可以使用 ~/.turtl 代替 ~/turtl 把 Turtl 安装到你的家目录的隐藏文件夹下。你也可以用些小技巧把它隐藏起来。 + +如果 Turtl 没有在你的 Unity Dash 上显示出来,请注销/登录以重启会话。 + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/08/turtl-secure-open-source-evernote.html + +作者:[Andrew][a] +译者:[chisper](https://github.com/chisper) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: https://turtl.it/docs/security/ +[2]: https://github.com/turtl/api/issues/20 +[3]: https://turtl.it/docs/server/ +[4]: https://trello.com/b/yIQGkHia/turtl-product-dev +[5]: https://turtl.it/download/ +[6]: https://github.com/turtl +[7]: https://aur.archlinux.org/packages/turtl/ +[8]: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=turtl +[9]: https://turtlapp.com/ diff --git a/translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md b/translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md deleted file mode 100644 index 46fd3b59f2..0000000000 --- a/translated/tech/20160817 Turtl - Secure, Open Source Evernote Alternative.md +++ /dev/null @@ -1,76 +0,0 @@ -Turtl:安全、开源的Evernote替代品 - -Turtl是一个安全、开源的Evernote替代品,在Linux, Windows, Mac, and Android等系统上都能使用。iOS版本仍在开发当中,Firefox和Chrome也有扩展程序可以使用。 - -![](https://3.bp.blogspot.com/-cNoUUjaU4A0/V7MFKCasZJI/AAAAAAAAYTk/r7oWe-z_HB87hDvlKLViiiHUMfagnC6LQCLcB/s400/turtl-desktop-linux.png) - -这个产品仍在测试阶段,能够让你把你的笔记,网站书签,密码,文档,图片单独放在一个隐秘地方。 - -笔记可以按模块组织起来,支持嵌套,也可以和其他Turtl用户分享。 - -![](https://2.bp.blogspot.com/-G-Ln3T1c2QA/V7MFmrqkukI/AAAAAAAAYTs/dXMPEB9MPREicixlEJlQVqg9SFjBX1pwgCLcB/s400/turtl-boards.png) - -给你的笔记打上标签,这样Turtl就会通过创建时间,最后修改时间或者标签来找你的笔记。 - -这个是文本编辑器: - -![](https://1.bp.blogspot.com/-8cNHV69iCWM/V7MFX7sBlMI/AAAAAAAAYTo/ZUVTYwiCSy8uzrVKdf6NcsQZlHtylIyvgCEw/s400/turtl-edit-note.png) - -那么安全性如何呢?Turtl会在保存数据之前加密,。只有你和你想要分享的人能获取数据。你可以从这获得跟多关于Turtl安全和加密的信息[HERE][1]。 - -更新(感谢Dimitry!):根据错误反馈[bug report][2],Turtl有个严重的安全性问题。Turtl允许同一个用户名创建多个账号,却只使用密码来区分它们。希望能马上修复这个问题。 - -Turtl团队提供了一个主机服务来同步你的记录,它是完全免费的,”直到你的文件足够大,或者你需要更好的服务”,在我写这篇文章的时候这个服务还不能用。 - -并且你也不一定要用这个服务,因为它是一个免费,开源的软件,所以你可以自己搭建一个[Turtl server][3]。 - -Turtl没有像Evernote那么多的功能,但它也有一些新的功能[roadmap][4],比如:支持导入/导出文本,Evernote格式的数据,支持PDF,界面锁定等。 - -不得不提醒的是,每次启动都要输入密码,虽然安全,但有时候实在是麻烦。 - -###下载Turtl - -[Download Turtl application][5] (Linux二进制文件-32bit or 64bit,Windows 64bit,Mac 64bit,Android,Chrome和Firefox浏览器插件) - -**更新**:Turtl用了一个新的服务器,注销然后在登陆框的下面选择高级设置,把Turtl服务器设置为"https://api.turtlapp.com/v2"(没有引号)。 - -下载源代码,反馈问题等,参见Turtl的GitHub[GitHub][6]。 - -在Linux上安装,把安装包解压后运行install.sh,安装之前请确保~/.local/share/applications目录存在,若不存在请自行创建: - -``` -mkdir -p ~/.local/share/applications -``` - -注意:如果使用sudo命令安装那么只有root用户才能使用。所以,要么不用sudo命令安装,要么在安装完成后修改权限。你可以在这里[package][8]参考如何修改权限。 - -使用如下命令把Turtl安装到~/turtl文件夹下(假定你已经把安装包解压在你家目录下了): - -```` -~/turtl-*/install.sh ~/turtl -``` - -可以使用~/.turtl代替~/turtl把Turtl安装到你家目录的隐藏文件夹下。你也可以用些小技巧把它隐藏起来。 - -如果Turtl没有在你的Unity Dash上显示出来,请注销/登陆以重启会话。 - --------------------------------------------------------------------------------- - -via: http://www.webupd8.org/2016/08/turtl-secure-open-source-evernote.html - -作者:[Andrew ][a] -译者:[chisper](https://github.com/chisper) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.webupd8.org/p/about.html -[1]: https://turtl.it/docs/security/ -[2]: https://github.com/turtl/api/issues/20 -[3]: https://turtl.it/docs/server/ -[4]: https://trello.com/b/yIQGkHia/turtl-product-dev -[5]: https://turtl.it/download/ -[6]: https://github.com/turtl -[7]: https://aur.archlinux.org/packages/turtl/ -[8]: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=turtl -[9]: https://turtlapp.com/ From 4b25981294659a9978e0f005354f5a4288541d82 Mon Sep 17 00:00:00 2001 From: wxy Date: Mon, 12 Sep 2016 12:14:29 +0800 Subject: [PATCH 0107/2437] PUB:20160817 Build an integration for GitHub @bestony --- ...0160817 Build an integration for GitHub.md | 54 +++++++++++++++++++ ...0160817 Build an integration for GitHub.md | 54 ------------------- 2 files changed, 54 insertions(+), 54 deletions(-) create mode 100644 published/20160817 Build an integration for GitHub.md delete mode 100644 translated/tech/20160817 Build an integration for GitHub.md diff --git a/published/20160817 Build an integration for GitHub.md b/published/20160817 Build an integration for GitHub.md new file mode 100644 index 0000000000..8d2212e714 --- /dev/null +++ b/published/20160817 Build an integration for GitHub.md @@ -0,0 +1,54 @@ +为 Github 创造集成件(Integration) +============= + +![](https://cloud.githubusercontent.com/assets/4432363/17537572/934e2f9c-5e52-11e6-9c24-50c7be5b5ae3.png) + +现在你可以从我们的 [集成件目录][1]里面找到更多工具。这个目录目前有超过 15 个分类 — 从 [API 管理][2] 到 [应用监控][3], Github 的集成件可以支持您的开发周期的每一个阶段。 + +我们邀请了具有不同层面的专长的开发人员,来创造有助于开发者更好的工作的集成件。如果你曾经为 Github 构建过一个很棒的集成件,我们希望来让更多人知道它! [Gitter][4]、[AppVeyor][5] 和 [ZenHub][6] 都做到了,你也可以! + +### 我们在寻找什么? + +良好的软件开发依赖于上乘的工具,开发人员如今有了更多的选择,无论是语言、框架、工作流程,还是包含了其他因素的环境。我们正在寻找能够创造更好的整体开发体验的开发工具。 + +#### 进入集成件目录清单的指南: + +- 稳步增多( 你的 Github OAuth 接口当前可以支持超过 500 个用户 ) +- 查看我们的 [技术要求][7] +- 遵从[服务条款][8] 和[隐私政策][9] +- 专注于软件开发生命周期 + +### 有帮助的资源 + +如果想要被列在目录里,请按照[列出需求页][10]中概述的步骤。 + +你也应该阅读我们的[营销指南][11]和[已有目录清单][12]来更好的了解如何把它们全都放在一起。请把你的列表的内容记录在一个[私密 gist][13] 中(markdown 格式),并且通过邮件联系我们 。 如果你有任何问题,不要犹疑,请联系 。 + +-------------------------------------------------------------------------------- + +via: https://github.com/blog/2226-build-an-integration-for-github + +作者:[chobberoni][a] +译者:[Bestony](https://github.com/Bestony) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://github.com/chobberoni + +[1]: https://github.com/integrations +[2]: https://github.com/integrations/feature/api-management +[3]: https://github.com/integrations/feature/monitoring +[4]: https://github.com/integrations/feature/monitoring +[5]: https://github.com/integrations/appveyor +[6]: https://github.com/integrations/zenhub +[7]: https://developer.github.com/integrations-directory/getting-listed/#technical-requirements +[8]: https://help.github.com/articles/github-terms-of-service/ +[9]: https://help.github.com/articles/github-privacy-policy/ +[10]: https://developer.github.com/integrations-directory/getting-listed/ +[11]: https://developer.github.com/integrations-directory/marketing-guidelines/ +[12]: https://github.com/integrations +[13]: https://gist.github.com/ + + + diff --git a/translated/tech/20160817 Build an integration for GitHub.md b/translated/tech/20160817 Build an integration for GitHub.md deleted file mode 100644 index cd04dc451c..0000000000 --- a/translated/tech/20160817 Build an integration for GitHub.md +++ /dev/null @@ -1,54 +0,0 @@ -为 Github 创造一种集成方式 -============= - -![](https://cloud.githubusercontent.com/assets/4432363/17537572/934e2f9c-5e52-11e6-9c24-50c7be5b5ae3.png) - -现在你可以从我们的 [集成方式目录][1]接触到更多工具。这个目录目前有超过15个分类 — 从 [API 管理][2] 到 [应用监控][3], 这里有 Github 的集成方式,可以支持您的开发周期的每一个阶段。 - -我们邀请了各级的开发人员,来创造一个新的集成方式,将有助于帮助开发者更好的工作。如果你曾经为Github构建过一个不错的集成方式,我们希望让他更出名! [Gitter][4], [AppVeyor][5], 和 [ZenHub][6]都做到了,你也可以! - -### 我们在寻找什么? - -良好的软件开发依赖于质量工具和开发人员如今有一个广泛范围来选择语言,框架,工作流程,和其他因素之间的环境。我们正在寻找能够创造更好的整体开发经验的开发工具。 - -#### 一份集成目录清单指南: - -- 扎实的增长( 你的Github OAuth 接口应该有超过500个用户 ) -- 查看我们的 [技术要求][7] -- 遵从 [服务条款][8] 和 [隐私政策][9] -- 专注于软件开发生命周期 - -### 有帮助的资源 - -如果想要被列出,请按照[列出需求页][10]中概述的步骤 - -你也应该阅读我们的[ 营销指南 ][11] 和 [已有目录清单][12] 来更好的了解如何把他们所有的放在一起。 请把你的列表的内容记录在一个[私密 gist][13]中(markdown 格式) 并且通过邮件联系我们 . 如果你有任何问题,不要害羞,请联系 . - --------------------------------------------------------------------------------- - -via: https://github.com/blog/2226-build-an-integration-for-github - -作者:[chobberoni ][a] -译者:[Bestony](https://github.com/Bestony) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://github.com/chobberoni - -[1]: https://github.com/integrations -[2]: https://github.com/integrations/feature/api-management -[3]: https://github.com/integrations/feature/monitoring -[4]: https://github.com/integrations/feature/monitoring -[5]: https://github.com/integrations/appveyor -[6]: https://github.com/integrations/zenhub -[7]: https://developer.github.com/integrations-directory/getting-listed/#technical-requirements -[8]: https://help.github.com/articles/github-terms-of-service/ -[9]: https://help.github.com/articles/github-privacy-policy/ -[10]: https://developer.github.com/integrations-directory/getting-listed/ -[11]: https://developer.github.com/integrations-directory/marketing-guidelines/ -[12]: https://github.com/integrations -[13]: https://gist.github.com/ - - - From 8107d892bea3485c83ddaaef07f3ecd3bd4b79f5 Mon Sep 17 00:00:00 2001 From: wxy Date: Mon, 12 Sep 2016 12:38:41 +0800 Subject: [PATCH 0108/2437] PUB:20160808 Why measuring IT productivity is so challenging @LemonDemo @WangYueScream --- ...uring IT productivity is so challenging.md | 36 ++++++++++++++++++ ...uring IT productivity is so challenging.md | 38 ------------------- 2 files changed, 36 insertions(+), 38 deletions(-) create mode 100644 published/20160808 Why measuring IT productivity is so challenging.md delete mode 100644 translated/talk/20160808 Why measuring IT productivity is so challenging.md diff --git a/published/20160808 Why measuring IT productivity is so challenging.md b/published/20160808 Why measuring IT productivity is so challenging.md new file mode 100644 index 0000000000..920b1d689d --- /dev/null +++ b/published/20160808 Why measuring IT productivity is so challenging.md @@ -0,0 +1,36 @@ +为什么计量 IT 的生产力如此具有挑战性? +=========================== + +![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/cio_talent_6.png?itok=JV-zSor3) + +在某些行业里,人们可以根据一些测量标准判定一个人的生产力。比如,如果你是一个零件制造商,可以通过一个月你能够制造的零件数量来确定你的生产效率。如果你在客户服务中心工作,你解答了多少个客户来电,你的平均解答时间都会成为评判你的生产效率的依据。这些都是相当简单的案例,但即便你是一位医生,也可以通过你主刀的临床手术次数或者一个月你确诊的患者数量来确定你的生产效率。无论这些评判标准正确与否,但它们提供了一个通用的方法来评断一个人在给定时间内的执行能力。 + +然而在 IT 这方面,通过上述方法来衡量一个人的生产力是不可能的,因为 IT 有太多的变化性。比如,通过一个开发者编写的代码行数来衡量开发者所用的时间看起来很诱人。但是,编程的语言很大程度上能影响到根据这种方法得到的结论。因为一种编程语言的一行代码比用其他编程语言编写所花费的时间和难度可能会明显的多或少。 + +它总是这样不可捉摸吗?多年以前,你可能听说过或者经历过根据功能点来衡量 IT 工作人员的生产效率。这些措施是针对开发者们能够创建的关键特征来衡量开发者的生产效率的。但这种方法在今天也变得逐渐难以实施,开发者经常将可能已有的逻辑封装进内部,比如,按供应商来整合功能点。这使得仅仅是基于功能点的数目来估量生产效率难度加大。 + +这两个例子能够阐述为什么当我们 CIO 之间谈论 IT 生产效率的时候有时会引起争论。考虑以下这个假想中的谈话: + +> IT leader:“天啊,我想这些开发者一定很厉害。” +> HR:“真的假的,他们做了什么?” +> IT leader:“他们做了个相当好的应用。” +> HR:“好吧,那他们比那些做了 10 个应用的开发者更好吗” +> IT leader:“这要看你怎么理解 ‘更好’。” + +这个对话比较有代表性。当我们处于上述的这种交谈时,这里面有太多的主观因素导致我们很难回答这个问题。当我们用一种有意义的方法来测试 IT 的效率时,类似上述谈话的这种问题仅仅是冰山一角。这不仅仅使谈话更加困难-它还会使 CIO 们很难展示他们的团队在商业上的价值。 + +确实这不是一个新出现的问题。我已经花费差不多 30 年的时间来思考这个问题。我得出的结论是我们真的不应该在谈论 IT 的生产效率这件事上面相互烦扰-因为我们永远不可能有结论。 + +我认为我们需要在改变这种对话同时停止根据生产能力和成本来谈论 IT 的生产效率,将目光集中于衡量 IT 的整体商业价值上。重申一下,这个过程不会很容易。商业价值的实现是一件困难的事情。但如果 CIO 们能够和商业人员合作来解决这个问题,就可以将实际价值变的更加科学而非一种艺术形式。 + +-------------------------------------------------------------------------------- + +via: https://enterprisersproject.com/article/2016/8/why-measuring-it-productivity-so-challenging + +作者:[Anil Cheriyan][a] +译者:[LemonDemo](https://github.com/LemonDemo) [WangYueScream](https://github.com/WangYueScream) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://enterprisersproject.com/user/anil-cheriyan diff --git a/translated/talk/20160808 Why measuring IT productivity is so challenging.md b/translated/talk/20160808 Why measuring IT productivity is so challenging.md deleted file mode 100644 index cc035981ab..0000000000 --- a/translated/talk/20160808 Why measuring IT productivity is so challenging.md +++ /dev/null @@ -1,38 +0,0 @@ -为什么测试 IT 的生产效率如此具有挑战性 -=========================== - -![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/cio_talent_6.png?itok=JV-zSor3) - -在某些行业里,人们可以根据一些测量标准判定一个人的生产力。比如,如果你是一个小部件制造商,可以通过一个月你能够制造的小部件数量来确定你的生产效率。如果你在客户服务中心工作,你解答了多少个客户来电,你的平均解答时间都会成为评判你的生产效率的依据。即便你是一位医生,也可以通过你主刀的临床手术次数或者一个月你确诊的患者数量来确定你的生产效率,但这些都是相当简单的案例。虽然这些评判标准不是完全正确的,但他们提供了一个通用的方法来在给定时间内评断一个人的执行能力。 - -然而在 IT 这方面,通过上述方法来衡量一个人的生产力是不可能的,因为 IT 有太多的变化性。比如,通过一个开发者编写的代码行数和耗费的时间来衡量开发者的生产效率看起来很诱人。但是,编程的语言很大程度上能影响到根据这种方法得到的结论。因为一种编程语言编辑的一行代码比用其他编程语言编写所花费的时间和难度可能会明显的多或少。 - -它总是这样微妙吗?多年以前,你可能听说过或者经历过根据功能点衡量 IT 工作人员的生产效率。这些措施是针对开发者们能够创建的关键特征来衡量开发者的生产效率的。但这种方法在今天也变得逐渐难以实施,开发者经常将可能已经存在逻辑封装进内部。比如,通过供应商来整合功能点。这使得仅仅是基于功能点的数目来估量生产效率难度加大。 - -这两个例子能够阐述为什么当我们与同事谈论 IT 生产效率的时候有时会引起 CIO 之间的争论。考虑以下这个假想中的谈话: - ->IT leader:“天啊,我想这些开发者一定很厉害。” ->HR:“真的假的,他们做了什么?” ->IT leader:“他们做了个相当好的应用。” ->HR:“好吧,那他们比那些做了 10 个应用的开发者更好吗” ->IT leader:“这要看你怎么理解 ‘更好’。” - -这个对话比较有代表性。当我们处于上诉的这种交谈时,这里面有太多的主观因素导致我们很难回答这个问题。当我们用一种有意义的方法来测试 IT 的效率时,类似上诉谈话的这种问题仅仅是冰山一角。这不仅仅使谈话更加困难-它还会使 CIO 很难展示他们的团队在商业上的价值。 - -可以确实这不是一个新出现的问题。我已经花费差不多 30 年的时间来思考这个问题。我得出的结论是我们真的不应该在谈论 IT 的生产效率这件事上面相互烦扰-因为我们的 IT 生产效率永远不可能能到达到值得我们为此争论的程度。 - -我认为我们需要改变这种对话同时停止根据生产能力和成本来谈论 IT 的生产效率,将目光集中于衡量 IT 的整体商业价值。这个过程不会很容易。商业价值的实现是一件困难的事情。但如果 CIO 能够和商业人员合作来解决这个问题,就可以将实际价值变的更加科学而非一种艺术形式。 - - - --------------------------------------------------------------------------------- - -via: https://enterprisersproject.com/article/2016/8/why-measuring-it-productivity-so-challenging - -作者:[Anil Cheriyan][a] -译者:[LemonDemo](https://github.com/LemonDemo) [WangYueScream](https://github.com/WangYueScream) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://enterprisersproject.com/user/anil-cheriyan From 512e5bb68ea413ecf486d256edca0c371692c092 Mon Sep 17 00:00:00 2001 From: echoma Date: Mon, 12 Sep 2016 18:48:30 +0800 Subject: [PATCH 0109/2437] =?UTF-8?q?echoma=E7=BF=BB=E8=AF=91=E7=94=B3?= =?UTF-8?q?=E9=A2=86?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Accelerating Node.js applications with HTTP2 Server Push.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md b/sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md index 9ea8c9079a..291fe06dde 100644 --- a/sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md +++ b/sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md @@ -1,3 +1,5 @@ +echoma 翻译中 + Accelerating Node.js applications with HTTP/2 Server Push ========================================================= From 7377a39046078005603a14d65d6a09d504aa5f2b Mon Sep 17 00:00:00 2001 From: Chao-zhi Liu Date: Mon, 12 Sep 2016 22:51:57 +0800 Subject: [PATCH 0110/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 删除 --- ...n All Distributions – Are They Any Good.md | 92 ------------------- 1 file changed, 92 deletions(-) delete mode 100644 sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md diff --git a/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md b/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md deleted file mode 100644 index 8ca553896b..0000000000 --- a/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md +++ /dev/null @@ -1,92 +0,0 @@ -Translating by Chao-zhi - -Linux Applications That Works On All Distributions – Are They Any Good? -============================================================================ - -![](http://www.iwillfolo.com/wordpress/wp-content/uploads/2016/06/Bundled-applications.jpg) - - -A revisit of Linux community’s latest ambitions – promoting decentralized applications in order to tackle distribution fragmentation. - -Following last week’s article: [Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful][1]?, a couple of new opinions rose to the surface which may contain crucial information about the usefulness of such apps. - -### The Con Side - -Commenting on the subject [here][2], a [Gentoo][3] user who goes by the name Till, gave rise to a few points which hasn’t been addressed to the fullest the last time we covered the issue. - -While previously we settled on merely calling it bloat, Till here on the other hand, is dissecting that bloat further so to help us better understand both its components and its consequences. - -Referring to such apps as “bundled applications” – for the way they work on all distributions is by containing dependencies together with the apps themselves, Till says: - ->“bundles ship a lot of software that now needs to be maintained by the application developer. If library X has a security problem and needs an update, you rely on every single applications to ship correct updates in order to make your system save.” - -Essentially, Till raises an important security point. However, it isn’t necessarily has to be tied to security alone, but can also be linked to other aspects such as system maintenance, atomic updates, etc… - -Furthermore, if we take that notion one step further and assume that dependencies developers may cooperate, therefore releasing their software in correlation with the apps who use them (an utopic situation), we shall then get an overall slowdown of the entire platform development. - -Another problem that arises from the same point made above is dependencies-transparency becomes obscure, that is, if you’d want to know which libraries are bundled with a certain app, you’ll have to rely on the developer to publish such data. - -Or, as Till puts it: “Questions like, did package XY already include the updated library Z, will be your daily bread”. - -For comparison, with the standard methods available on Linux nowadays (both binary and source distributions), you can easily notice which libraries are being updated upon a system update. - -And you can also rest assure that all other apps on the system will use it, freeing you from the need to check each app individually. - -Other cons that may be deduced from the term bloat include: bigger package size (each app is bundled with dependencies), higher memory usage (no more library sharing) and also – - -One less filter mechanism to prevent malicious software – distributions package maintainers also serve as a filter between developers and users, helping to assure users get quality software. - -With bundled apps this may no longer be the case. - -As a finalizing general point, Till asserts that although useful in some cases, for the most part, bundled apps weakens the free software position in distributions (as proprietary vendors will now be able to deliver software without sharing it on public repositories). - -And apart from that, it introduces many other issues. Many problems are simply moved towards the developers. - -### The Pro Side - -In contrast, another comment by a person named Sven tries to contradict common claims that basically go against the use of bundled applications, hence justifying and promoting the use of it. - -“waste of space” – Sven claims that in today’s world we have many other things that wastes disk space, such as movies stored on hard drive, installed locals, etc… - -Ultimately, these things are infinitely more wasteful than a mere “100 MB to run a program you use all day … Don’t be ridiculous.” - -“waste of RAM” – the major points in favor are: - -- Shared libraries waste significantly less RAM compared to application runtime data. -- RAM is cheap today. - -“security nightmare” – not every application you run is actually security-critical. - -Also, many applications never even see any security updates, unless on a ‘rolling distro’. - -In addition to Sven’s opinions, who try to stick to the pragmatic side, a few advantages were also pointed by Till who admits that bundled apps has their merits in certain cases: - -- Proprietary vendors who want to keep their code out of the public repositories will be able to do so more easily. -- Niche applications, which are not packaged by your distribution, will now be more readily available. -- Testing on binary distributions which do not have beta packages will become easier. -- Freeing users from solving dependencies problems. - -### Final Thoughts - -Although shedding new light onto the matter, it seems that one conclusion still stands and accepted by all parties – bundled apps has their niche to fill in the Linux ecosystem. - -Nevertheless, the role that niche should take, whether main or marginal one, appears to be a lot clearer now, at least from a theoretical point of view. - -Users who are looking to make their system optimized as possible, should, in the majority of cases, avoid using bundled apps. - -Whereas, users that are after ease-of-use, meaning – doing the least of work in order to maintain their systems, should and probably would feel very comfortable adopting the new method. - --------------------------------------------------------------------------------- - -via: http://www.iwillfolo.com/linux-applications-that-works-on-all-distributions-are-they-any-good/ - -作者:[Editorials][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.iwillfolo.com/category/editorials/ -[1]: http://www.iwillfolo.com/ubuntus-snap-red-hats-flatpack-and-is-one-fits-all-linux-packages-useful/ -[2]: http://www.proli.net/2016/06/25/gnulinux-bundled-application-ramblings/ -[3]: http://www.iwillfolo.com/5-reasons-use-gentoo-linux/ From 46111a9f9cee55b3c32c71b6d5545af63cea152a Mon Sep 17 00:00:00 2001 From: Chao-zhi Liu Date: Mon, 12 Sep 2016 22:53:11 +0800 Subject: [PATCH 0111/2437] Add files via upload --- ... 应用适用于所有发行版——这有什么好处呢?.md | 87 +++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 translated/talk/Linux 应用适用于所有发行版——这有什么好处呢?.md diff --git a/translated/talk/Linux 应用适用于所有发行版——这有什么好处呢?.md b/translated/talk/Linux 应用适用于所有发行版——这有什么好处呢?.md new file mode 100644 index 0000000000..45232346bf --- /dev/null +++ b/translated/talk/Linux 应用适用于所有发行版——这有什么好处呢?.md @@ -0,0 +1,87 @@ +适用于所有发行版的 Linux 应用程序——是否真的有好处呢? +============================================================================ + +![](http://www.iwillfolo.com/wordpress/wp-content/uploads/2016/06/Bundled-applications.jpg) + + +让我们回顾一下 Linux 社区最初的愿景——促进应用程序的分发来解决发行版的碎片化。 + +继上周的文章:[Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful?][1] 之后,一系列新观点浮出水面,他们可能包含关于这些应用程序有效性的重要信息。 + +### 缺点 + +就这个话题在[这里][2]的评论,一个叫 Till 的 [Gentoo][3] 使用者,对于上一次我们未能完全解释的问题给出了一些新的观点。 + +对于上一次我们选择仅仅称之为膨胀的的东西,Till 在这里进行了深入的剖析,这帮助我们更好的理解它的组成和其影响。 + +这些被称之为“捆绑应用”的应用程序能够工作在所有发行版的机制是——将它依赖的库都包含在它们的应用软件之中,Till 说: + + +>“捆绑应用装载了大量的并不被应用开发者所维护的软件。如果其中的函数库X被发现了一个安全问题,你得为每一个独立的应用程序安装更新来确保你的系统安全。” + +本质上,Till 提出了一个重要的安全问题。但是它并不仅仅与安全有关系,它还关系到许多方面,比如说系统维护,原子更新等等。 + +此外,如果我们进一步假设:依赖开发者们也许会合作,将他们的软件与使用它的应用程序一起发布(一个乌托邦的情况),这将导致整个平台的开发整体放缓。 + +另一个将会导致的问题是依赖关系变得模糊,就是说,如果你想知道一个应用程序捆绑了哪些依赖关系,你必须依靠开发者发布这些数据。 + +或者就像 Till 说的:“比如说像包XY是否包含了函数库Z的更新这样的问题将会是你每天需要面对的” + +与之相反,对于 Linux 现行的标准的包管理方法(包括二进制包和源码包),你能够很容易的注意到哪些函数库已经在系统中更新了。 +并且,你也可以很轻松的知道哪些应用使用了这个函数库,这就将你从繁琐的单独检查每一个应用程序的工作中解救了出来。 + +其他可能由膨胀导致的缺点包括:更大的包体积(每一个应用程序捆绑了依赖),更高的内存占用(没有分享函数库),并且,少了一个包过滤机制来防止恶意软件:发行版的包维护者也充当了一个在开发者和用户之间的过滤者,他保障了用户获得高质量的软件。而捆绑应用中就不再是这种情况了。 + +最后一点,Till 声称,尽管在某些情况下很有用,但是在大多数情况下,捆绑应用程序将消弱自由软件在发行版中的地位(专有的供应商将被允许发布他们的软件而不在公共储存库中分享它)。 + +除此之外,它引出了许多其他问题。很多问题仅仅只与开发人员有关。 + +### 优点 + +相比之下,另一个名叫 Sven 的人的评论试图反驳目前普遍反对使用捆绑应用程序的观点,从而证明和支持使用它。 + +“浪费空间?”——斯文声称在当今世界我们有很多其他事情在浪费磁盘空间,比如电影存储在硬盘上、本地安装等等…… + +最终,这些事情浪费的空间要远远多于仅仅 100 MB 而你每天都要使用的程序。因此浪费空间的说法实在很荒谬。 + +“浪费运行内存?”——主要的观点有: + +- 共享库浪费的内存要远远少于程序运行时产生的数据 + +- 而今运行内存已经很便宜了 + +“安全梦魇”——不是每个应用程序的运行真正上要注重安全。 + +而且,许多应用程序甚至从来没有看到任何安全更新,除非在“滚动更新的发行版”。 + +除了 Sven 这种从实用出发的观点以外,Till 其实也指出了捆绑应用在一些情况下也有着其优点: + +- 专有的供应商想要保持他们的代码游离于公共存储库之外将更加容易。 +- 不被你的发行版打包的小生态的应用程序将变得更加可行。 +- 在没有Beta包的二进制发行版中测试应用将变得简单。 +- 将用户从复杂的依赖关系中解放出来。 + +### 最后的思考 + +虽然关于此问题有着不同的想法,但是有一个被大家共同接受的观点是:捆绑应用对于填补 Linux 生态系统有着其独到的作用。 + +虽然如此,它的定位,不论是主流的还是边缘的,都变得愈发清晰,至少理论上是这样。 + +想要尽可能优化其系统的用户,在大多数情况下应该要避免使用捆绑应用。 + +而讲究易用性、尽可能在维护系统上少费劲的用户,可能应该会感觉这种新应用十分舒爽。 + +-------------------------------------------------------------------------------- + +via: http://www.iwillfolo.com/linux-applications-that-works-on-all-distributions-are-they-any-good/ + +作者:[Editorials][a] +译者:[Chao-zhi](https://github.com/Chao-zhi) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.iwillfolo.com/category/editorials/ +[1]: http://www.iwillfolo.com/ubuntus-snap-red-hats-flatpack-and-is-one-fits-all-linux-packages-useful/ +[2]: http://www.proli.net/2016/06/25/gnulinux-bundled-application-ramblings/ +[3]: http://www.iwillfolo.com/5-reasons-use-gentoo-linux/ From cfeb64600b5d97fb6c86d66bc5c274cb5a2e82d9 Mon Sep 17 00:00:00 2001 From: Chao-zhi Liu Date: Mon, 12 Sep 2016 22:54:23 +0800 Subject: [PATCH 0112/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=EF=BC=8C=E5=BE=85=E6=A0=A1=E5=AF=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Translated by Chao-zhi --- ... 20160627 Linux Applications That Works On All Distributions – Are They Any Good.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename translated/talk/{Linux 应用适用于所有发行版——这有什么好处呢?.md => 20160627 Linux Applications That Works On All Distributions – Are They Any Good.md} (100%) diff --git a/translated/talk/Linux 应用适用于所有发行版——这有什么好处呢?.md b/translated/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md similarity index 100% rename from translated/talk/Linux 应用适用于所有发行版——这有什么好处呢?.md rename to translated/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md From f3736ed6b10d93e691d6ac8af4ff0627f0d9d9b4 Mon Sep 17 00:00:00 2001 From: Chao-zhi Liu Date: Mon, 12 Sep 2016 23:01:06 +0800 Subject: [PATCH 0113/2437] =?UTF-8?q?[=E5=BC=80=E5=A7=8B=E7=BF=BB=E8=AF=91?= =?UTF-8?q?]?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 开始翻译 2016-9-12 --- ...d Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md b/sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md index 35088e4be1..b1c955bf5b 100644 --- a/sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md +++ b/sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md @@ -1,3 +1,5 @@ +Translating by Chao-zhi + Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful? ================================================================================= From f356c5d60ac5b540b938fe90b4410dea85a4ad06 Mon Sep 17 00:00:00 2001 From: Chao-zhi Liu Date: Mon, 12 Sep 2016 23:22:57 +0800 Subject: [PATCH 0114/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=2020160627=20Linux=20Applications=20That=20Works=20On=20All=20?= =?UTF-8?q?Distributions=20=E2=80=93=20Are=20They=20Any=20Good.md=20(#4401?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 翻译完成 删除 * Add files via upload * 翻译完成,待校对 Translated by Chao-zhi --- ...n All Distributions – Are They Any Good.md | 92 ------------------- ...n All Distributions – Are They Any Good.md | 87 ++++++++++++++++++ 2 files changed, 87 insertions(+), 92 deletions(-) delete mode 100644 sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md create mode 100644 translated/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md diff --git a/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md b/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md deleted file mode 100644 index 8ca553896b..0000000000 --- a/sources/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md +++ /dev/null @@ -1,92 +0,0 @@ -Translating by Chao-zhi - -Linux Applications That Works On All Distributions – Are They Any Good? -============================================================================ - -![](http://www.iwillfolo.com/wordpress/wp-content/uploads/2016/06/Bundled-applications.jpg) - - -A revisit of Linux community’s latest ambitions – promoting decentralized applications in order to tackle distribution fragmentation. - -Following last week’s article: [Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful][1]?, a couple of new opinions rose to the surface which may contain crucial information about the usefulness of such apps. - -### The Con Side - -Commenting on the subject [here][2], a [Gentoo][3] user who goes by the name Till, gave rise to a few points which hasn’t been addressed to the fullest the last time we covered the issue. - -While previously we settled on merely calling it bloat, Till here on the other hand, is dissecting that bloat further so to help us better understand both its components and its consequences. - -Referring to such apps as “bundled applications” – for the way they work on all distributions is by containing dependencies together with the apps themselves, Till says: - ->“bundles ship a lot of software that now needs to be maintained by the application developer. If library X has a security problem and needs an update, you rely on every single applications to ship correct updates in order to make your system save.” - -Essentially, Till raises an important security point. However, it isn’t necessarily has to be tied to security alone, but can also be linked to other aspects such as system maintenance, atomic updates, etc… - -Furthermore, if we take that notion one step further and assume that dependencies developers may cooperate, therefore releasing their software in correlation with the apps who use them (an utopic situation), we shall then get an overall slowdown of the entire platform development. - -Another problem that arises from the same point made above is dependencies-transparency becomes obscure, that is, if you’d want to know which libraries are bundled with a certain app, you’ll have to rely on the developer to publish such data. - -Or, as Till puts it: “Questions like, did package XY already include the updated library Z, will be your daily bread”. - -For comparison, with the standard methods available on Linux nowadays (both binary and source distributions), you can easily notice which libraries are being updated upon a system update. - -And you can also rest assure that all other apps on the system will use it, freeing you from the need to check each app individually. - -Other cons that may be deduced from the term bloat include: bigger package size (each app is bundled with dependencies), higher memory usage (no more library sharing) and also – - -One less filter mechanism to prevent malicious software – distributions package maintainers also serve as a filter between developers and users, helping to assure users get quality software. - -With bundled apps this may no longer be the case. - -As a finalizing general point, Till asserts that although useful in some cases, for the most part, bundled apps weakens the free software position in distributions (as proprietary vendors will now be able to deliver software without sharing it on public repositories). - -And apart from that, it introduces many other issues. Many problems are simply moved towards the developers. - -### The Pro Side - -In contrast, another comment by a person named Sven tries to contradict common claims that basically go against the use of bundled applications, hence justifying and promoting the use of it. - -“waste of space” – Sven claims that in today’s world we have many other things that wastes disk space, such as movies stored on hard drive, installed locals, etc… - -Ultimately, these things are infinitely more wasteful than a mere “100 MB to run a program you use all day … Don’t be ridiculous.” - -“waste of RAM” – the major points in favor are: - -- Shared libraries waste significantly less RAM compared to application runtime data. -- RAM is cheap today. - -“security nightmare” – not every application you run is actually security-critical. - -Also, many applications never even see any security updates, unless on a ‘rolling distro’. - -In addition to Sven’s opinions, who try to stick to the pragmatic side, a few advantages were also pointed by Till who admits that bundled apps has their merits in certain cases: - -- Proprietary vendors who want to keep their code out of the public repositories will be able to do so more easily. -- Niche applications, which are not packaged by your distribution, will now be more readily available. -- Testing on binary distributions which do not have beta packages will become easier. -- Freeing users from solving dependencies problems. - -### Final Thoughts - -Although shedding new light onto the matter, it seems that one conclusion still stands and accepted by all parties – bundled apps has their niche to fill in the Linux ecosystem. - -Nevertheless, the role that niche should take, whether main or marginal one, appears to be a lot clearer now, at least from a theoretical point of view. - -Users who are looking to make their system optimized as possible, should, in the majority of cases, avoid using bundled apps. - -Whereas, users that are after ease-of-use, meaning – doing the least of work in order to maintain their systems, should and probably would feel very comfortable adopting the new method. - --------------------------------------------------------------------------------- - -via: http://www.iwillfolo.com/linux-applications-that-works-on-all-distributions-are-they-any-good/ - -作者:[Editorials][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.iwillfolo.com/category/editorials/ -[1]: http://www.iwillfolo.com/ubuntus-snap-red-hats-flatpack-and-is-one-fits-all-linux-packages-useful/ -[2]: http://www.proli.net/2016/06/25/gnulinux-bundled-application-ramblings/ -[3]: http://www.iwillfolo.com/5-reasons-use-gentoo-linux/ diff --git a/translated/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md b/translated/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md new file mode 100644 index 0000000000..45232346bf --- /dev/null +++ b/translated/talk/20160627 Linux Applications That Works On All Distributions – Are They Any Good.md @@ -0,0 +1,87 @@ +适用于所有发行版的 Linux 应用程序——是否真的有好处呢? +============================================================================ + +![](http://www.iwillfolo.com/wordpress/wp-content/uploads/2016/06/Bundled-applications.jpg) + + +让我们回顾一下 Linux 社区最初的愿景——促进应用程序的分发来解决发行版的碎片化。 + +继上周的文章:[Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful?][1] 之后,一系列新观点浮出水面,他们可能包含关于这些应用程序有效性的重要信息。 + +### 缺点 + +就这个话题在[这里][2]的评论,一个叫 Till 的 [Gentoo][3] 使用者,对于上一次我们未能完全解释的问题给出了一些新的观点。 + +对于上一次我们选择仅仅称之为膨胀的的东西,Till 在这里进行了深入的剖析,这帮助我们更好的理解它的组成和其影响。 + +这些被称之为“捆绑应用”的应用程序能够工作在所有发行版的机制是——将它依赖的库都包含在它们的应用软件之中,Till 说: + + +>“捆绑应用装载了大量的并不被应用开发者所维护的软件。如果其中的函数库X被发现了一个安全问题,你得为每一个独立的应用程序安装更新来确保你的系统安全。” + +本质上,Till 提出了一个重要的安全问题。但是它并不仅仅与安全有关系,它还关系到许多方面,比如说系统维护,原子更新等等。 + +此外,如果我们进一步假设:依赖开发者们也许会合作,将他们的软件与使用它的应用程序一起发布(一个乌托邦的情况),这将导致整个平台的开发整体放缓。 + +另一个将会导致的问题是依赖关系变得模糊,就是说,如果你想知道一个应用程序捆绑了哪些依赖关系,你必须依靠开发者发布这些数据。 + +或者就像 Till 说的:“比如说像包XY是否包含了函数库Z的更新这样的问题将会是你每天需要面对的” + +与之相反,对于 Linux 现行的标准的包管理方法(包括二进制包和源码包),你能够很容易的注意到哪些函数库已经在系统中更新了。 +并且,你也可以很轻松的知道哪些应用使用了这个函数库,这就将你从繁琐的单独检查每一个应用程序的工作中解救了出来。 + +其他可能由膨胀导致的缺点包括:更大的包体积(每一个应用程序捆绑了依赖),更高的内存占用(没有分享函数库),并且,少了一个包过滤机制来防止恶意软件:发行版的包维护者也充当了一个在开发者和用户之间的过滤者,他保障了用户获得高质量的软件。而捆绑应用中就不再是这种情况了。 + +最后一点,Till 声称,尽管在某些情况下很有用,但是在大多数情况下,捆绑应用程序将消弱自由软件在发行版中的地位(专有的供应商将被允许发布他们的软件而不在公共储存库中分享它)。 + +除此之外,它引出了许多其他问题。很多问题仅仅只与开发人员有关。 + +### 优点 + +相比之下,另一个名叫 Sven 的人的评论试图反驳目前普遍反对使用捆绑应用程序的观点,从而证明和支持使用它。 + +“浪费空间?”——斯文声称在当今世界我们有很多其他事情在浪费磁盘空间,比如电影存储在硬盘上、本地安装等等…… + +最终,这些事情浪费的空间要远远多于仅仅 100 MB 而你每天都要使用的程序。因此浪费空间的说法实在很荒谬。 + +“浪费运行内存?”——主要的观点有: + +- 共享库浪费的内存要远远少于程序运行时产生的数据 + +- 而今运行内存已经很便宜了 + +“安全梦魇”——不是每个应用程序的运行真正上要注重安全。 + +而且,许多应用程序甚至从来没有看到任何安全更新,除非在“滚动更新的发行版”。 + +除了 Sven 这种从实用出发的观点以外,Till 其实也指出了捆绑应用在一些情况下也有着其优点: + +- 专有的供应商想要保持他们的代码游离于公共存储库之外将更加容易。 +- 不被你的发行版打包的小生态的应用程序将变得更加可行。 +- 在没有Beta包的二进制发行版中测试应用将变得简单。 +- 将用户从复杂的依赖关系中解放出来。 + +### 最后的思考 + +虽然关于此问题有着不同的想法,但是有一个被大家共同接受的观点是:捆绑应用对于填补 Linux 生态系统有着其独到的作用。 + +虽然如此,它的定位,不论是主流的还是边缘的,都变得愈发清晰,至少理论上是这样。 + +想要尽可能优化其系统的用户,在大多数情况下应该要避免使用捆绑应用。 + +而讲究易用性、尽可能在维护系统上少费劲的用户,可能应该会感觉这种新应用十分舒爽。 + +-------------------------------------------------------------------------------- + +via: http://www.iwillfolo.com/linux-applications-that-works-on-all-distributions-are-they-any-good/ + +作者:[Editorials][a] +译者:[Chao-zhi](https://github.com/Chao-zhi) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.iwillfolo.com/category/editorials/ +[1]: http://www.iwillfolo.com/ubuntus-snap-red-hats-flatpack-and-is-one-fits-all-linux-packages-useful/ +[2]: http://www.proli.net/2016/06/25/gnulinux-bundled-application-ramblings/ +[3]: http://www.iwillfolo.com/5-reasons-use-gentoo-linux/ From 1f92fc905b3d8310bdc78a64cd0293dfdbfc6018 Mon Sep 17 00:00:00 2001 From: Echo Ma Date: Mon, 12 Sep 2016 10:23:24 -0500 Subject: [PATCH 0115/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=20(#4400)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 翻译基本完成 * 翻已完成 * 翻译完成,删掉源文件 --- ....js applications with HTTP2 Server Push.md | 116 ------------------ ....js applications with HTTP2 Server Push.md | 113 +++++++++++++++++ 2 files changed, 113 insertions(+), 116 deletions(-) delete mode 100644 sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md create mode 100644 translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md diff --git a/sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md b/sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md deleted file mode 100644 index 291fe06dde..0000000000 --- a/sources/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md +++ /dev/null @@ -1,116 +0,0 @@ -echoma 翻译中 - -Accelerating Node.js applications with HTTP/2 Server Push -========================================================= - -In April, we [announced support for HTTP/2 Server][3] Push via the HTTP Link header. My coworker John has demonstrated how easy it is to [add Server Push to an example PHP application][4]. - -![](https://blog.cloudflare.com/content/images/2016/08/489477622_594bf9e3d9_z.jpg) - -We wanted to make it easy to improve the performance of contemporary websites built with Node.js. we developed the netjet middleware to parse the generated HTML and automatically add the Link headers. When used with an example Express application you can see the headers being added: - -![](https://blog.cloudflare.com/content/images/2016/08/2016-08-11_13-32-45.png) - -We use Ghost to power this blog, so if your browser supports HTTP/2 you have already benefited from Server Push without realizing it! More on that below. - -In netjet, we use the PostHTML project to parse the HTML with a custom plugin. Right now it is looking for images, scripts and external stylesheets. You can implement this same technique in other environments too. - -Putting an HTML parser in the response stack has a downside: it will increase the page load latency (or "time to first byte"). In most cases, the added latency will be overshadowed by other parts of your application, such as database access. However, netjet includes an adjustable LRU cache keyed by ETag headers, allowing netjet to insert Link headers quickly on pages already parsed. - -If you are designing a brand new application, however, you should consider storing metadata on embedded resources alongside your content, eliminating the HTML parse, and possible latency increase, entirely. - -Netjet is compatible with any Node.js HTML framework that supports Express-like middleware. Getting started is as simple as adding netjet to the beginning of your middleware chain. - -``` -var express = require('express'); -var netjet = require('netjet'); -var root = '/path/to/static/folder'; - -express() - .use(netjet({ - cache: { - max: 100 - } - })) - .use(express.static(root)) - .listen(1337); -``` - -With a little more work, you can even use netjet without frameworks. - -``` -var http = require('http'); -var netjet = require('netjet'); - -var port = 1337; -var hostname = 'localhost'; -var preload = netjet({ - cache: { - max: 100 - } -}); - -var server = http.createServer(function (req, res) { - preload(req, res, function () { - res.statusCode = 200; - res.setHeader('Content-Type', 'text/html'); - res.end('

Hello World

'); - }); -}); - -server.listen(port, hostname, function () { - console.log('Server running at http://' + hostname + ':' + port+ '/'); -}); -``` - -See the [netjet documentation][1] for more information on the supported options. - -### Seeing what’s pushed - -![](https://blog.cloudflare.com/content/images/2016/08/2016-08-02_10-49-33.png) - -Chrome's Developer Tools makes it easy to verify that your site is using Server Push. The Network tab shows pushed assets with "Push" included as part of the initiator. - -Unfortunately, Firefox's Developers Tools don't yet directly expose if the resource pushed. You can, however, check for the cf-h2-pushed header in the page's response headers, which contains a list of resources that CloudFlare offered browsers over Server Push. - -Contributions to improve netjet or the documentation are greatly appreciated. I'm excited to hear where people are using netjet. - -### Ghost and Server Push - -Ghost is one such exciting integration. With the aid of the Ghost team, I've integrated netjet, and it has been available as an opt-in beta since version 0.8.0. - -If you are running a Ghost instance, you can enable Server Push by modifying the server's config.js file and add the preloadHeaders option to the production configuration block. - - -``` -production: { - url: 'https://my-ghost-blog.com', - preloadHeaders: 100, - // ... -} -``` - -Ghost has put together [a support article][2] for Ghost(Pro) customers. - -### Conclusion - -With netjet, your Node.js applications can start to use browser preloading and, when used with CloudFlare, HTTP/2 Server Push today. - -At CloudFlare, we're excited to make tools to help increase the performance of websites. If you find this interesting, we are hiring in Austin, Texas; Champaign, Illinois; London; San Francisco; and Singapore. - - --------------------------------------------------------------------------------- - -via: https://blog.cloudflare.com/accelerating-node-js-applications-with-http-2-server-push/?utm_source=nodeweekly&utm_medium=email - -作者:[Terin Stock][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://blog.cloudflare.com/author/terin-stock/ -[1]: https://www.npmjs.com/package/netjet -[2]: http://support.ghost.org/preload-headers/ -[3]: https://www.cloudflare.com/http2/server-push/ -[4]: https://blog.cloudflare.com/using-http-2-server-push-with-php/ diff --git a/translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md b/translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md new file mode 100644 index 0000000000..adba7d092f --- /dev/null +++ b/translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md @@ -0,0 +1,113 @@ +echoma 翻译中 + +使用HTTP/2服务端推送技术加速Node.js应用 +========================================================= + +四月份,我们宣布了对[HTTP/2服务端推送技术][3]的支持,我们是通过HTTP的[Link头](https://www.w3.org/wiki/LinkHeader)来实现这项支持的。我的同事John曾经通过一个例子演示了[在PHP里支持服务端推送功能][4]是多么的简单。 + +![](https://blog.cloudflare.com/content/images/2016/08/489477622_594bf9e3d9_z.jpg) + +我们想让使用Node.js构建的网站能够更加轻松的获得性能提升。为此,我们开发了netjet中间件,它可以解析应用生成的HTML并自动添加Link头。当结合Express框架使用这个中间件时,我们可以看到应用程序的输出多了如下HTTP头: + +![](https://blog.cloudflare.com/content/images/2016/08/2016-08-11_13-32-45.png) + +[本博客][5]是使用 [Ghost](https://ghost.org/)(译者注:一个博客发布平台)进行发布的, 因此如果你的浏览器支持HTTP/2,你已经在不知不觉中享受了服务端推送技术带来的好处了。接下来,我们将进行更详细的说明。 + +netjet使用了带有自制插件的[PostHTML](https://github.com/posthtml/posthtml)来解析HTML。目前,netjet用它来查找图片、脚本和外部样式。 + +在响应过程中增加HTML解析器有个明显的缺点:这将增加页面加载的时延(加载到第一个字节的所花的时间)。大多数情况下,新增的延时被应用里的其他耗时掩盖掉了,比如数据库访问。为了解决这个问题,netjet包含了一个可调节的LRU缓存,该缓存以HTTP的ETag头作为索引,这使得netjet可以非常快的为已经解析过的页面插入Link头。 + +在这种情况下,如果我们现在从头设计一款全新的应用,我们就需要全面的考量如何减少HTML解析和页面加载延时了。把页面内容和页面中的元数据分开存放是一种值得考虑的方法。 + +任意的Node.js HTML框架,只要它支持类似Express这样的中间件,netjet都是能够兼容的。只要把netjet像下面这样加到中间件加载链里就可以了。 + +```javascript +var express = require('express'); +var netjet = require('netjet'); +var root = '/path/to/static/folder'; + +express() + .use(netjet({ + cache: { + max: 100 + } + })) + .use(express.static(root)) + .listen(1337); +``` + +稍微加点代码,netjet也可以摆脱HTML框架,独立工作: + +```javascript +var http = require('http'); +var netjet = require('netjet'); + +var port = 1337; +var hostname = 'localhost'; +var preload = netjet({ + cache: { + max: 100 + } +}); + +var server = http.createServer(function (req, res) { + preload(req, res, function () { + res.statusCode = 200; + res.setHeader('Content-Type', 'text/html'); + res.end('

Hello World

'); + }); +}); + +server.listen(port, hostname, function () { + console.log('Server running at http://' + hostname + ':' + port+ '/'); +}); +``` + +[netjet文档里][1]有更多选项的信息。 + +### 查看推送了什么数据 + +![](https://blog.cloudflare.com/content/images/2016/08/2016-08-02_10-49-33.png) + +访问[本文][5]时,通过Chrome的开发者工具,我们可以轻松的验证网站是否正在使用服务器推送技术(译者注:Chrome版本至少为53)。在"Network"选项卡中,我们可以看到有些图片的"Initiator"这一列中包含了`Push`字样,这些图片就是服务器端推送的。 + +目前Firefox的开发者工具还不能直观的展示被推送的自选。不过我们可以通过页面响应头里的`cf-h2-pushed`头看到一个列表,这个列表包含了本页面主动推送给浏览器的资源。 + +希望大家能够踊跃为netjet添砖加瓦,我也乐于看到有人正在使用netjet。 + +### Ghost和服务端推送技术 + +Ghost真是包罗万象。在Ghost团队的帮助下,我把netjet也集成到里面了,而且作为测试版内容可以在Ghost的0.8.0版本中用上它。 + +如果你正在使用Ghost,你可以通过修改config.js、并在`production`配置块中增加preloadHeaders选项来启用服务端推送。 + +```javascript +production: { + url: 'https://my-ghost-blog.com', + preloadHeaders: 100, + // ... +} +``` + +Ghost已经为其用户整理了[一篇支持文档][2]. + +### 结论 + +使用netjet,你的Node.js应用也可以使用浏览器预加载技术。并且[本站][5]已经使用它在提供了HTTP/2服务端推送了。 + +-------------------------------------------------------------------------------- + +via: https://blog.cloudflare.com/accelerating-node-js-applications-with-http-2-server-push/ + +作者:[Terin Stock][a] +译者:[译者ID](https://github.com/echoma) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://blog.cloudflare.com/author/terin-stock/ +[1]: https://www.npmjs.com/package/netjet +[2]: http://support.ghost.org/preload-headers/ +[3]: https://www.cloudflare.com/http2/server-push/ +[4]: https://blog.cloudflare.com/using-http-2-server-push-with-php/ +[5]: https://blog.cloudflare.com/accelerating-node-js-applications-with-http-2-server-push/ \ No newline at end of file From 1ece52b0a62bdb98ecf789408dec982ac8960989 Mon Sep 17 00:00:00 2001 From: may Date: Mon, 12 Sep 2016 23:24:00 +0800 Subject: [PATCH 0116/2437] finish translation (#4399) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * translating by maywanting * finish translating * translating start * delete the file * 终于补完了! --- ...160809 How to build your own Git server.md | 225 ------------------ ...160809 How to build your own Git server.md | 222 +++++++++++++++++ 2 files changed, 222 insertions(+), 225 deletions(-) delete mode 100644 sources/tech/20160809 How to build your own Git server.md create mode 100644 translated/tech/20160809 How to build your own Git server.md diff --git a/sources/tech/20160809 How to build your own Git server.md b/sources/tech/20160809 How to build your own Git server.md deleted file mode 100644 index 7f75c33195..0000000000 --- a/sources/tech/20160809 How to build your own Git server.md +++ /dev/null @@ -1,225 +0,0 @@ -translating by maywanting - -How to build your own Git server -==================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/bus-big-data.png?itok=sOQHDuID) - -Now we will learn how to build a Git server, and how to write custom Git hooks to trigger specific actions on certain events (such as notifications), and publishing your code to a website. - -Up until now, the focus has been interacting with Git as a user. In this article I'll discuss the administration of Git, and the design of a flexible Git infrastructure. You might think it sounds like a euphemism for "advanced Git techniques" or "only read this if you're a super-nerd", but actually none of these tasks require advanced knowledge or any special training beyond an intermediate understanding of how Git works, and in some cases a little bit of knowledge about Linux. - -### Shared Git server - -Creating your own shared Git server is surprisingly simple, and in many cases well worth the trouble. Not only does it ensure that you always have access to your code, it also opens doors to stretching the reach of Git with extensions such as personal Git hooks, unlimited data storage, and continuous integration and deployment. - -If you know how to use Git and SSH, then you already know how to create a Git server. The way Git is designed, the moment you create or clone a repository, you have already set up half the server. Then enable SSH access to the repository, and anyone with access can use your repo as the basis for a new clone. - -However, that's a little ad hoc. With some planning a you can construct a well-designed Git server with about the same amount of effort, but with better scalability. - -First things first: identify your users, both current and in the future. If you're the only user then no changes are necessary, but if you intend to invite contributors aboard, then you should allow for a dedicated shared system user for your developers. - -Assuming that you have a server available (if not, that's not exactly a problem Git can help with, but CentOS on a Raspberry Pi 3 is a good start), then the first step is to enable SSH logins using only SSH key authorization. This is much stronger than password logins because it is immune to brute-force attacks, and disabling a user is as simple as deleting their key. - -Once you have SSH key authorization enabled, create the gituser. This is a shared user for all of your authorized users: - -``` -$ su -c 'adduser gituser' -``` - -Then switch over to that user, and create an ~/.ssh framework with the appropriate permissions. This is important, because for your own protection SSH will default to failure if you set the permissions too liberally. - -``` -$ su - gituser -$ mkdir .ssh && chmod 700 .ssh -$ touch .ssh/authorized_keys -$ chmod 600 .ssh/authorized_keys -``` - -The authorized_keys file holds the SSH public keys of all developers you give permission to work on your Git project. Your developers must create their own SSH key pairs and send you their public keys. Copy the public keys into the gituser's authorized_keys file. For instance, for a developer called Bob, run these commands: - -``` -$ cat ~/path/to/id_rsa.bob.pub >> \ -/home/gituser/.ssh/authorized_keys -``` - -As long as developer Bob has the private key that matches the public key he sent you, Bob can access the server as gituser. - -However, you don't really want to give your developers access to your server, even if only as gituser. You only want to give them access to the Git repository. For this very reason, Git provides a limited shell called, appropriately, git-shell. Run these commands as root to add git-shell to your system, and then make it the default shell for your gituser: - -``` -# grep git-shell /etc/shells || su -c \ -"echo `which git-shell` >> /etc/shells" -# su -c 'usermod -s git-shell gituser' -``` - -Now the gituser can only use SSH to push and pull Git repositories, and cannot access a login shell. You should add yourself to the corresponding group for the gituser, which in our example server is also gituser. - -For example: - -``` -# usermod -a -G gituser seth -``` - -The only step remaining is to make a Git repository. Since no one is going to interact with it directly on the server (that is, you're not going to SSH to the server and work directly in this repository), make it a bare repository. If you want to use the repo on the server to get work done, you'll clone it from where it lives and work on it in your home directory. - -Strictly speaking, you don't have to make this a bare repository; it would work as a normal repo. However, a bare repository has no *working tree* (that is, no branch is ever in a `checkout` state). This is important because remote users are not permitted to push to an active branch (how would you like it if you were working in a `dev` branch and suddenly someone pushed changes into your workspace?). Since a bare repo can have no active branch, that won't ever be an issue. - -You can place this repository anywhere you please, just as long as the users and groups you want to grant permission to access it can do so. You do NOT want to store the directory in a user's home directory, for instance, because the permissions there are pretty strict, but in a common shared location, such as /opt or /usr/local/share. - -Create a bare repository as root: - -``` -# git init --bare /opt/jupiter.git -# chown -R gituser:gituser /opt/jupiter.git -# chmod -R 770 /opt/jupiter.git -``` - -Now any user who is either authenticated as gituser or is in the gituser group can read from and write to the jupiter.git repository. Try it out on a local machine: - -``` -$ git clone gituser@example.com:/opt/jupiter.git jupiter.clone -Cloning into 'jupiter.clone'... -Warning: you appear to have cloned an empty repository. -``` - -Remember: developers MUST have their public SSH key entered into the authorized_keys file of gituser, or if they have accounts on the server (as you would), then they must be members of the gituser group. - -### Git hooks - -One of the nice things about running your own Git server is that it makes Git hooks available. Git hosting services sometimes provide a hook-like interface, but they don't give you true Git hooks with access to the file system. A Git hook is a script that gets executed at some point during a Git process; a hook can be executed when a repository is about to receive a commit, or after it has accepted a commit, or before it receives a push, or after a push, and so on. - -It is a simple system: any executable script placed in the .git/hooks directory, using a standard naming scheme, is executed at the designated time. When a script should be executed is determined by the name; a pre-push script is executed before a push, a post-receive script is executed after a commit has been received, and so on. It's more or less self-documenting. - -Scripts can be written in any language; if you can execute a language's hello world script on your system, then you can use that language to script a Git hook. By default, Git ships with some samples but does not have any enabled. - -Want to see one in action? It's easy to get started. First, create a Git repository if you don't already have one: - -``` -$ mkdir jupiter -$ cd jupiter -$ git init . -``` - -Then write a "hello world" Git hook. Since I use tcsh at work for legacy support, I'll stick with that as my scripting language, but feel free to use your preferred language (Bash, Python, Ruby, Perl, Rust, Swift, Go) instead: - -``` -$ echo "#\!/bin/tcsh" > .git/hooks/post-commit -$ echo "echo 'POST-COMMIT SCRIPT TRIGGERED'" >> \ -~/jupiter/.git/hooks/post-commit -$ chmod +x ~/jupiter/.git/hooks/post-commit -``` - -Now test it out: - -``` -$ echo "hello world" > foo.txt -$ git add foo.txt -$ git commit -m 'first commit' -! POST-COMMIT SCRIPT TRIGGERED -[master (root-commit) c8678e0] first commit -1 file changed, 1 insertion(+) -create mode 100644 foo.txt -``` - -And there you have it: your first functioning Git hook. - -### The famous push-to-web hook - -A popular use of Git hooks is to automatically push changes to a live, in-production web server directory. It is a great way to ditch FTP, retain full version control of what is in production, and integrate and automate publication of content. - -If done correctly, it works brilliantly and is, in a way, exactly how web publishing should have been done all along. It is that good. I don't know who came up with the idea initially, but the first I heard of it was from my Emacs- and Git- mentor, Bill von Hagen at IBM. His article remains the definitive introduction to the process: [Git changes the game of distributed Web development][1]. - -### Git variables - -Each Git hook gets a different set of variables relevant to the Git action that triggered it. You may or may not need to use those variables; it depends on what you're writing. If all you want is a generic email alerting you that someone pushed something, then you don't need specifics, and probably don't even need to write the script as the existing samples may work for you. If you want to see the commit message and author of a commit in that email, then your script becomes more demanding. - -Git hooks aren't run by the user directly, so figuring out how to gather important information can be confusing. In fact, a Git hook script is just like any other script, accepting arguments from stdin in the same way that BASH, Python, C++, and anything else does. The difference is, we aren't providing that input ourselves, so to use it you need to know what to expect. - -Before writing a Git hook, look at the samples that Git provides in your project's .git/hooks directory. The pre-push.sample file, for instance, states in the comments section: - -``` -# $1 -- Name of the remote to which the push is being done -# $2 -- URL to which the push is being done -# If pushing without using a named remote those arguments will be equal. -# -# Information about commit is supplied as lines -# to the standard input in this form: -# -``` - -Not all samples are that clear, and documentation on what hook gets what variable is still a little sparse (unless you want to read the source code of Git), but if in doubt, you can learn a lot from the [trials of other users][2] online, or just write a basic script and echo $1, $2, $3, and so on. - -### Branch detection example - -I have found that a common requirement in production instances is a hook that triggers specific events based on what branch is being affected. Here is an example of how to tackle such a task. - -First of all, Git hooks are not, themselves, version controlled. That is, Git doesn't track its own hooks because a Git hook is part of Git, not a part of your repository. For that reason, a Git hook that oversees commits and pushes probably make most sense living in a bare repository on your Git server, rather than as a part of your local repositories. - -Let's write a hook that runs upon post-receive (that is, after a commit has been received). The first step is to identify the branch name: - -``` -#!/bin/tcsh - -foreach arg ( $< ) - set argv = ( $arg ) - set refname = $1 -end -``` - -This for-loop reads in the first arg ($1) and then loops again to overwrite that with the value of the second ($2), and then again with the third ($3). There is a better way to do that in Bash: use the read command and put the values into an array. However, this being tcsh and the variable order being predictable, it's safe to hack through it. - -When we have the refname of what is being commited, we can use Git to discover the human-readable name of the branch: - -``` -set branch = `git rev-parse --symbolic --abbrev-ref $refname` -echo $branch #DEBUG -``` - -And then compare the branch name to the keywords we want to base the action on: - -``` -if ( "$branch" == "master" ) then - echo "Branch detected: master" - git \ - --work-tree=/path/to/where/you/want/to/copy/stuff/to \ - checkout -f $branch || echo "master fail" -else if ( "$branch" == "dev" ) then - echo "Branch detected: dev" - Git \ - --work-tree=/path/to/where/you/want/to/copy/stuff/to \ - checkout -f $branch || echo "dev fail" - else - echo "Your push was successful." - echo "Private branch detected. No action triggered." -endif -``` - -Make the script executable: - -``` -$ chmod +x ~/jupiter/.git/hooks/post-receive -``` - -Now when a user commits to the server's master branch, the code is copied to an in-production directory, a commit to the dev branch get copied someplace else, and any other branch triggers no action. - -It's just as simple to create a pre-commit script that, for instance, checks to see if someone is trying to push to a branch that they should not be pushing to, or to parse commit messages for approval strings, and so on. - -Git hooks can get complex, and they can be confusing due to the level of abstraction that working through Git imposes, but they're a powerful system that allows you to design all manner of actions in your Git infrastructure. They're worth dabbling in, if only to become familiar with the process, and worth mastering if you're a serious Git user or full-time Git admin. - -In our next and final article in this series, we will learn how to use Git to manage non-text binary blobs, such as audio and graphics files. - - --------------------------------------------------------------------------------- - -via: https://opensource.com/life/16/8/how-construct-your-own-git-server-part-6 - -作者:[Seth Kenlon][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/seth -[1]: http://www.ibm.com/developerworks/library/wa-git/ -[2]: https://www.analysisandsolutions.com/code/git-hooks-summary-cheat-sheet.htm diff --git a/translated/tech/20160809 How to build your own Git server.md b/translated/tech/20160809 How to build your own Git server.md new file mode 100644 index 0000000000..03d77a9607 --- /dev/null +++ b/translated/tech/20160809 How to build your own Git server.md @@ -0,0 +1,222 @@ +如何搭建你自己的 Git 服务器 +==================== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/bus-big-data.png?itok=sOQHDuID) + +现在我们将要学习如何搭建 git 服务器,如何编写自定义的 Git 钩子来在特定的事件触发相应的动作(例如通知),或者是发布你的代码到一个站点。 + +直到现在,作为一个使用者注意力还是被 Git 影响。这篇文章中我将讨论 Git 的管理,并且设计一个灵活的 Git 框架。你可能会觉得这听起来是 “高阶 Git 技术” 或者 “只有狂热粉才能阅读”的一句委婉的说法,但是事实是这里面的每个任务都不需要很深的知识或者其他特殊的训练,就能立刻理解 Git 的工作原理,有可能需要一丁点关于 Linux 的知识 + +### 共享 Git 服务器 + +创建你自己的共享 Git 服务器意外地简单,而且遇到各方面的问题时都很值得。不仅仅是因为它保证你有权限查看自己的代码,它还对于利用扩展来维持 Git 的通信敞开了一扇大门,例如个人 Git 钩子,无限制的数据存储,和持续的整合与发布 + +如果你知道如何使用 Git 和 SSH,那么你已经知道怎么创建一个 Git 服务器了。Git 的设计方式,就是让你在创建或者 clone 一个仓库的时候,就完成了一半服务器的搭建。然后允许用 SSH 访问仓库,而且任何有权限访问的人都可以使用你的仓库,作为 clone 的新仓库的基础。 + +但是,这是一个小的自组织。按照一些计划你可以以同样的效果创建一些设计优良的 Git 服务器,同时有更好的拓展性。 + +首要之事:确认你的用户们,现在的用户以及之后的用户都要考虑。如果你是唯一的用户那么没有任何改动的必要。但是如果你试图邀请其他的代码贡献者,那么你应该允许一个专门的分享系统用户给你的开发者们。 + +假定你有一个可用的服务器(如果没有,这不成问题,Git 会帮忙解决,CentOS 的 Raspberry Pi 3 是个不错的开始),然后第一步就是只允许使用 SSH 密钥认证的 SSH 登录。 + +一旦你启用了 SSH 密钥认证,创建 gituser 用户。这是给你的所有确认的开发者们的公共用户。 + +``` +$ su -c 'adduser gituser' +``` + +然后切换到刚创建的 gituser 用户,创建一个 `~/.ssh` 的框架,并设置好合适的权限。这很重要,如果权限设置得太开放会使自己受保护的 SSH 默认失败。 + +``` +$ su - gituser +$ mkdir .ssh && chmod 700 .ssh +$ touch .ssh/authorized_keys +$ chmod 600 .ssh/authorized_keys +``` + +`authorized_keys` 文件里包含所有你的开发者们的 SSH 公钥,你开放权限允许他们可以在你的 Git 项目上工作。他们必须创建他们自己的 SSH 密钥对然后把他们的公钥给你。复制公钥到 gituser 用户下的 `authorized_keys` 文件中。例如,为一个叫 Bob 的开发者,执行以下命令: + +``` +$ cat ~/path/to/id_rsa.bob.pub >> \ +/home/gituser/.ssh/authorized_keys +``` + +只要开发者 Bob 有私钥并且把相对应的公钥给你,Bob 就可以用 gituser 用户访问服务器。 + +但是,你并不是想让你的开发者们能使用服务器,即使只是以 gituser 的身份访问。你只是想给他们访问 Git 仓库的权限。因为这个特殊的原因,Git 提供了一个限制的 shell,准确的说是 git-shell。以 root 身份执行以下命令,把 git-shell 添加到你的系统中,然后设置成 gituser 用户默认的shell。 + +``` +# grep git-shell /etc/shells || su -c \ +"echo `which git-shell` >> /etc/shells" +# su -c 'usermod -s git-shell gituser' +``` + +现在 gituser 用户只能使用 SSH 来 push 或者 pull Git 仓库,并且无法使用任何一个可以登录的 shell。你应该把你自己添加到和 gituser 一样的组中,在我们的样例服务器中仍是这个组的名字仍是 gituser。 + +举个例子: + +``` +# usermod -a -G gituser seth +``` + +仅剩下的一步就是创建一个 Git 仓库。因为没有人能在服务器上与 Git 交互(也就是说,你之后不能 SSH 到服务器然后直接操作这个仓库),创一个最原始的仓库 。如果你想使用位于服务器上的仓库来完成工作,你可以从它的所在处 clone 下来然后直接在你的 home 目录下工作。 + +说白了,你不需要让它变成一个空的仓库,你需要像工作在一个正常的仓库一样。但是,一个空的仓库没有 *working tree* (也就是说,使用 `checkout` 并没有任何分支显示)。这很重要,因为远程使用者们并不被允许 push 一个有效的分支(如果你正在 `dev` 分支工作然后突然有人把一些变更 push 到你的工作分支,你会有怎么样的感受?)。因为一个空的仓库可以没有有效的分支,这不会成为一个问题。 + +你可以把这个仓库放到任何你想放的地方,只要你想要放开权限给用户和用户组,让他们可以在仓库下工作。千万不要保存目录到例如一个用户的 home 目录下,因为有严格的权限控制。保存到一个常规的共享地址,例如 `/opt` 或者 `/usr/local/share`。 + +以 root 身份创建一个空的仓库: + +``` +# git init --bare /opt/jupiter.git +# chown -R gituser:gituser /opt/jupiter.git +# chmod -R 770 /opt/jupiter.git +``` + +现在任何一个用户,只要他被认证为 gituser 或者在 gituser 组中,就可以从 jupiter.git 库中读取或者写入。在本地机器尝试以下操作。 + +``` +$ git clone gituser@example.com:/opt/jupiter.git jupiter.clone +Cloning into 'jupiter.clone'... +Warning: you appear to have cloned an empty repository. +``` + +谨记:开发者们一定要把他们的 SSH 公钥加入到 gituser 用户下的 `authorized_keys` 文件里,或者说,如果他们有服务器上的用户(如果你给了他们用户),那么他们的用户必须属于 gituser 用户组。 + +### Git 钩子 + +将自己的 Git 服务器跑起来,很赞的一件事就是可以使用 Git 钩子。支持 Git 的设备有时提供一个类钩子接口,但是并没有给你真正的 Git 钩子来访问文件系统。Git 钩子是一个脚本,它将在一个 Git 过程的某些点运行;当一个仓库即将接收一个 commit,或者接受一个 commit 之后,或者即将接收一次 push,或者一次 push 之后等等 + +这是一个简单的系统:任何放在 `.git/hooks` 目录下的脚本,使用标准的命名体系,就可按设计好的时间运行。一个脚本是否应该被运行取决于它的名字; pre-push 脚本在 push 之前运行,post-received 脚本在 push 之后运行,post-receive 脚本在接受 commit 之后运行等等。 + +脚本可以用任何语言写;如果在你的系统上有可以执行的脚本语言,例如输出 ‘hello world’ ,那么你就用这个语言来写 Git 钩子脚本。Git 默认会给出一些例子,但是并不能用。 + +想要动手试一个?这很简单。如果你没有现成的 Git 仓库,首先创建一个 Git 仓库: + +``` +$ mkdir jupiter +$ cd jupiter +$ git init . +``` + +然后写一个功能为输出 “hello world” 的 Git 钩子。因为我使用 tsch 作为传统系统的长期支持 shell,所以我仍然用它作为我的脚本语言,你可以自由的使用自己喜欢的语言(Bash,Python,Ruby,Perl,Rust,Swift,Go): + +``` +$ echo "#\!/bin/tcsh" > .git/hooks/post-commit +$ echo "echo 'POST-COMMIT SCRIPT TRIGGERED'" >> \ +~/jupiter/.git/hooks/post-commit +$ chmod +x ~/jupiter/.git/hooks/post-commit +``` + +现在测试它的输出: + +``` +$ echo "hello world" > foo.txt +$ git add foo.txt +$ git commit -m 'first commit' +! POST-COMMIT SCRIPT TRIGGERED +[master (root-commit) c8678e0] first commit +1 file changed, 1 insertion(+) +create mode 100644 foo.txt +``` + +然后你已经实现了:你的第一个有功能的 Git 钩子 + +### 有名的 push-to-web 钩子 + +Git 钩子最流行的用法就是自动 push 更改的代码到一个正在使用中的网络服务器目录下。这是摆脱 FTP 的很好的方式,对于正在使用的产品保留完整的版本控制,自动整合发布的内容 + +如果操作正确,在一直以来的网站发布维护中 Git 会完成的很好,而且在某种程度上,很精准。Git 真的好棒。我不知道谁最初想到这个主意,但是我第一次听到它是从 Emacs 和 Git 方面的专家,IBM 的 Bill von Hagen。他的文章包含一系列明确的介绍:[Git 改变了分布式网页开发的游戏规则][1]。 + +### Git 变量 + +每一个 Git 钩子都有一系列不同的变量对应触发钩子的不同 Git 行为。你需不需要这些变量,主要取决于你写的程序。如果你需要一个当某人 push 代码时候的通用邮件通知,那么你就不需要特殊化,并且甚至也不需要编写额外的脚本,因为已经有现成的适合你的脚本。如果你想在邮件里查看 commit 信息和 commit 的作者,那么你的脚本就会变得相对棘手些。 + +Git 钩子并不是被用户直接执行,所以要弄清楚如何收集可能会误解的重要信息。事实上,Git 钩子脚本就像其他的脚本一样,像 BASH, Python, C++ 等等,从标准输入读取参数。不同的是,我们不会给它提供这个输入,所以,你在使用的时候,需要知道可能的输入参数。 + +在写 Git 钩子之前,看一下 Git 在你的项目目录下 `.git/hooks` 目录中提供的一些例子。举个例子,在这个 `pre-push.sample` 文件里,注释部分说明了如下内容: + +``` +# $1 -- 即将 push 的远方仓库的名字 +# $2 -- 即将 push 的远方仓库的 URL +# 如果 push 的时候,并没有一个命名的远方仓库,那么这两个参数将会一样。 +# +# 对于这个表单的标准输入, +# 参数通过行的形式输入 +# +``` + +并不是所有的例子都是这么清晰,而且关于钩子获取变量的文档依旧缺乏(除非你去读 Git 的源码)。但是,如果你有疑问,你可以从[其他用户的尝试中][2]学习,或者你只是写一些基本的脚本,比如 `echo $1, $2, $3` 等等。 + +### 分支检测示例 + +我发现,对于生产环境来说有一个共同的需求,就是需要一个只有在特定分支被修改之后,才会触发事件的钩子。以下就是如何跟踪分支的示例。 + +首先,Git 钩子本身是不受版本控制的。 Git 并不会跟踪他自己的钩子,因为对于钩子来说,他是 Git 的一部分,而不是你仓库的一部分。所以,Git 钩子可以监控你的 Git 服务器上的 Git 仓库的 commit 记录和 push 记录,而不是你本地仓库的一部分。 + +我们来写一个 post-receive(也就是说,在 commit 被接受之后触发)钩子。第一步就是需要确定分支名: + +``` +#!/bin/tcsh + +foreach arg ( $< ) + set argv = ( $arg ) + set refname = $1 +end +``` + +这个 for 循环用来读入第一个参数 `arg($1)` 然后循环用第二个参数 `($2)` 去重写,然后用第三个参数 `($3)` 。在 Bash 中有一个更好的方法,使用 read 命令,并且把值放入数组里。但是,这里是 tcsh,并且变量的顺序可以预测的,所以,这个方法也是可行的。 + +当我们有了 commit 记录的 refname,我们就能使用 Git 去找到这个分支的人类能读的名字: + +``` +set branch = `git rev-parse --symbolic --abbrev-ref $refname` +echo $branch #DEBUG +``` + +然后把这个分支名和我们想要触发的事件的分支名关键字进行比较。 + +``` +if ( "$branch" == "master" ) then + echo "Branch detected: master" + git \ + --work-tree=/path/to/where/you/want/to/copy/stuff/to \ + checkout -f $branch || echo "master fail" +else if ( "$branch" == "dev" ) then + echo "Branch detected: dev" + Git \ + --work-tree=/path/to/where/you/want/to/copy/stuff/to \ + checkout -f $branch || echo "dev fail" + else + echo "Your push was successful." + echo "Private branch detected. No action triggered." +endif +``` + +给这个脚本分配可执行权限: + +``` +$ chmod +x ~/jupiter/.git/hooks/post-receive +``` + +现在,当一个用户在服务器的 master 分支 commit 代码,这个代码就会被复制到一个生产环境的目录,dev 分支的一个 commit 记录也会被复制到其他地方,其他分支将不会触发这些操作。 + +同时,创造一个 pre-commit 脚本也很简单。比如,判断一个用户是否在他们不该 push 的分支上 push 代码,或者对 commit 信息进行解析等等。 + +Git 钩子也可以变得复杂,而且他们因为 Git 的工作流的抽象层次不同而变得难以理解,但是他们确实是一个强大的系统,让你能够在你的 Git 基础设施上针对所有的行为进行对应的操作。如果你是一个 Git 重度用户,或者一个全职 Git 管理员,那么 Git 钩子是值得学习的。 + +在我们这个系列之后和最后的文章,我们将会学习如何使用 Git 来管理非文本的二进制数据,比如音频和图片。 + +-------------------------------------------------------------------------------- + +via: https://opensource.com/life/16/8/how-construct-your-own-git-server-part-6 + +作者:[Seth Kenlon][a] +译者:[maywanting](https://github.com/maywanting) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/seth +[1]: http://www.ibm.com/developerworks/library/wa-git/ +[2]: https://www.analysisandsolutions.com/code/git-hooks-summary-cheat-sheet.htm From 06b2f0309bdea85566e437ee3d5fcb1b8354b0fa Mon Sep 17 00:00:00 2001 From: wxy Date: Tue, 13 Sep 2016 00:45:18 +0800 Subject: [PATCH 0117/2437] PUB:20160608 Simple Python Framework from Scratch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @Cathon 辛苦了,翻译的很不错,很认真! --- ...08 Simple Python Framework from Scratch.md | 464 +++++++++++++++++ ...08 Simple Python Framework from Scratch.md | 466 ------------------ 2 files changed, 464 insertions(+), 466 deletions(-) create mode 100644 published/20160608 Simple Python Framework from Scratch.md delete mode 100644 translated/tech/20160608 Simple Python Framework from Scratch.md diff --git a/published/20160608 Simple Python Framework from Scratch.md b/published/20160608 Simple Python Framework from Scratch.md new file mode 100644 index 0000000000..c7f9174dd6 --- /dev/null +++ b/published/20160608 Simple Python Framework from Scratch.md @@ -0,0 +1,464 @@ +从零构建一个简单的 Python 框架 +=================================== + +为什么你想要自己构建一个 web 框架呢?我想,原因有以下几点: + +- 你有一个新奇的想法,觉得将会取代其他的框架 +- 你想要获得一些名气 +- 你遇到的问题很独特,以至于现有的框架不太合适 +- 你对 web 框架是如何工作的很感兴趣,因为你想要成为一位更好的 web 开发者。 + +接下来的笔墨将着重于最后一点。这篇文章旨在通过对设计和实现过程一步一步的阐述告诉读者,我在完成一个小型的服务器和框架之后学到了什么。你可以在这个[代码仓库][1]中找到这个项目的完整代码。 + +我希望这篇文章可以鼓励更多的人来尝试,因为这确实很有趣。它让我知道了 web 应用是如何工作的,而且这比我想的要容易的多! + +### 范围 + +框架可以处理请求-响应周期、身份认证、数据库访问、模板生成等部分工作。Web 开发者使用框架是因为,大多数的 web 应用拥有大量相同的功能,而对每个项目都重新实现同样的功能意义不大。 + +比较大的的框架如 Rails 和 Django 实现了高层次的抽象,或者说“自备电池”(“batteries-included”,这是 Python 的口号之一,意即所有功能都自足。)。而实现所有的这些功能可能要花费数千小时,因此在这个项目上,我们重点完成其中的一小部分。在开始写代码前,我先列举一下所需的功能以及限制。 + +功能: + +- 处理 HTTP 的 GET 和 POST 请求。你可以在[这篇 wiki][2] 中对 HTTP 有个大致的了解。 +- 实现异步操作(我*喜欢* Python 3 的 asyncio 模块)。 +- 简单的路由逻辑以及参数撷取。 +- 像其他微型框架一样,提供一个简单的用户级 API 。 +- 支持身份认证,因为学会这个很酷啊(微笑)。 + +限制: + +- 将只支持 HTTP 1.1 的一个小子集,不支持传输编码(transfer-encoding)、HTTP 认证(http-auth)、内容编码(content-encoding,如 gzip)以及[持久化连接][3]等功能。 +- 不支持对响应内容的 MIME 判断 - 用户需要手动指定。 +- 不支持 WSGI - 仅能处理简单的 TCP 连接。 +- 不支持数据库。 + +我觉得一个小的用例可以让上述内容更加具体,也可以用来演示这个框架的 API: + +``` +from diy_framework import App, Router +from diy_framework.http_utils import Response + + +# GET simple route +async def home(r): + rsp = Response() + rsp.set_header('Content-Type', 'text/html') + rsp.body = 'test' + return rsp + + +# GET route + params +async def welcome(r, name): + return "Welcome {}".format(name) + +# POST route + body param +async def parse_form(r): + if r.method == 'GET': + return 'form' + else: + name = r.body.get('name', '')[0] + password = r.body.get('password', '')[0] + + return "{0}:{1}".format(name, password) + +# application = router + http server +router = Router() +router.add_routes({ + r'/welcome/{name}': welcome, + r'/': home, + r'/login': parse_form,}) + +app = App(router) +app.start_server() +``` +' +用户需要定义一些能够返回字符串或 `Response` 对象的异步函数,然后将这些函数与表示路由的字符串配对,最后通过一个函数调用(`start_server`)开始处理请求。 + +完成设计之后,我将它抽象为几个我需要编码的部分: + +- 接受 TCP 连接以及调度一个异步函数来处理这些连接的部分 +- 将原始文本解析成某种抽象容器的部分 +- 对于每个请求,用来决定调用哪个函数的部分 +- 将上述部分集中到一起,并为开发者提供一个简单接口的部分 + +我先编写一些测试,这些测试被用来描述每个部分的功能。几次重构后,整个设计被分成若干部分,每个部分之间是相对解耦的。这样就非常好,因为每个部分可以被独立地研究学习。以下是我上文列出的抽象的具体体现: + +- 一个 HTTPServer 对象,需要一个 Router 对象和一个 http_parser 模块,并使用它们来初始化。 +- HTTPConnection 对象,每一个对象表示一个单独的客户端 HTTP 连接,并且处理其请求-响应周期:使用 http_parser 模块将收到的字节流解析为一个 Request 对象;使用一个 Router 实例寻找并调用正确的函数来生成一个响应;最后将这个响应发送回客户端。 +- 一对 Request 和 Response 对象为用户提供了一种友好的方式,来处理实质上是字节流的字符串。用户不需要知道正确的消息格式和分隔符是怎样的。 +- 一个包含“路由:函数”对应关系的 Router 对象。它提供一个添加配对的方法,可以根据 URL 路径查找到相应的函数。 +- 最后,一个 App 对象。它包含配置信息,并使用它们实例化一个 HTTPServer 实例。 + +让我们从 `HTTPConnection` 开始来讲解各个部分。 + +### 模拟异步连接 + +为了满足上述约束条件,每一个 HTTP 请求都是一个单独的 TCP 连接。这使得处理请求的速度变慢了,因为建立多个 TCP 连接需要相对高的花销(DNS 查询,TCP 三次握手,[慢启动][4]等等的花销),不过这样更加容易模拟。对于这一任务,我选择相对高级的 [asyncio-stream][5] 模块,它建立在 [asyncio 的传输和协议][6]的基础之上。我强烈推荐你读一读标准库中的相应代码,很有意思! + +一个 `HTTPConnection` 的实例能够处理多个任务。首先,它使用 `asyncio.StreamReader` 对象以增量的方式从 TCP 连接中读取数据,并存储在缓存中。每一个读取操作完成后,它会尝试解析缓存中的数据,并生成一个 `Request` 对象。一旦收到了这个完整的请求,它就生成一个回复,并通过 `asyncio.StreamWriter` 对象发送回客户端。当然,它还有两个任务:超时连接以及错误处理。 + +你可以在[这里][7]浏览这个类的完整代码。我将分别介绍代码的每一部分。为了简单起见,我移除了代码文档。 + +``` +class HTTPConnection(object): + def init(self, http_server, reader, writer): + self.router = http_server.router + self.http_parser = http_server.http_parser + self.loop = http_server.loop + + self._reader = reader + self._writer = writer + self._buffer = bytearray() + self._conn_timeout = None + self.request = Request() +``` + +这个 `init` 方法没啥意思,它仅仅是收集了一些对象以供后面使用。它存储了一个 `router` 对象、一个 `http_parser` 对象以及 `loop` 对象,分别用来生成响应、解析请求以及在事件循环中调度任务。 + +然后,它存储了代表一个 TCP 连接的读写对,和一个充当原始字节缓冲区的空[字节数组][8]。`_conn_timeout` 存储了一个 [asyncio.Handle][9] 的实例,用来管理超时逻辑。最后,它还存储了 `Request` 对象的一个单一实例。 + +下面的代码是用来接受和发送数据的核心功能: + +``` +async def handle_request(self): + try: + while not self.request.finished and not self._reader.at_eof(): + data = await self._reader.read(1024) + if data: + self._reset_conn_timeout() + await self.process_data(data) + if self.request.finished: + await self.reply() + elif self._reader.at_eof(): + raise BadRequestException() + except (NotFoundException, + BadRequestException) as e: + self.error_reply(e.code, body=Response.reason_phrases[e.code]) + except Exception as e: + self.error_reply(500, body=Response.reason_phrases[500]) + + self.close_connection() +``` + +所有内容被包含在 `try-except` 代码块中,这样在解析请求或响应期间抛出的异常可以被捕获到,然后一个错误响应会发送回客户端。 + +在 `while` 循环中不断读取请求,直到解析器将 `self.request.finished` 设置为 True ,或者客户端关闭连接所触发的信号使得 `self._reader_at_eof()` 函数返回值为 True 为止。这段代码尝试在每次循环迭代中从 `StreamReader` 中读取数据,并通过调用 `self.process_data(data)` 函数以增量方式生成 `self.request`。每次循环读取数据时,连接超时计数器被重置。 + +这儿有个错误,你发现了吗?稍后我们会再讨论这个。需要注意的是,这个循环可能会耗尽 CPU 资源,因为如果没有读取到东西 `self._reader.read()` 函数将会返回一个空的字节对象 `b''`。这就意味着循环将会不断运行,却什么也不做。一个可能的解决方法是,用非阻塞的方式等待一小段时间:`await asyncio.sleep(0.1)`。我们暂且不对它做优化。 + +还记得上一段我提到的那个错误吗?只有从 `StreamReader` 读取数据时,`self._reset_conn_timeout()` 函数才会被调用。这就意味着,**直到第一个字节到达时**,`timeout` 才被初始化。如果有一个客户端建立了与服务器的连接却不发送任何数据,那就永远不会超时。这可能被用来消耗系统资源,从而导致拒绝服务式攻击(DoS)。修复方法就是在 `init` 函数中调用 `self._reset_conn_timeout()` 函数。 + +当请求接受完成或连接中断时,程序将运行到 `if-else` 代码块。这部分代码会判断解析器收到完整的数据后是否完成了解析。如果是,好,生成一个回复并发送回客户端。如果不是,那么请求信息可能有错误,抛出一个异常!最后,我们调用 `self.close_connection` 执行清理工作。 + +解析请求的部分在 `self.process_data` 方法中。这个方法非常简短,也易于测试: + +``` +async def process_data(self, data): + self._buffer.extend(data) + + self._buffer = self.http_parser.parse_into( + self.request, self._buffer) +``` + +每一次调用都将数据累积到 `self._buffer` 中,然后试着用 `self.http_parser` 来解析已经收集的数据。这里需要指出的是,这段代码展示了一种称为[依赖注入(Dependency Injection)][10]的模式。如果你还记得 `init` 函数的话,应该知道我们传入了一个包含 `http_parser` 对象的 `http_server` 对象。在这个例子里,`http_parser` 对象是 `diy_framework` 包中的一个模块。不过它也可以是任何含有 `parse_into` 函数的类,这个 `parse_into` 函数接受一个 `Request` 对象以及字节数组作为参数。这很有用,原因有二:一是,这意味着这段代码更易扩展。如果有人想通过一个不同的解析器来使用 `HTTPConnection`,没问题,只需将它作为参数传入即可。二是,这使得测试更加容易,因为 `http_parser` 不是硬编码的,所以使用虚假数据或者 [mock][11] 对象来替代是很容易的。 + +下一段有趣的部分就是 `reply` 方法了: + +``` +async def reply(self): + request = self.request + handler = self.router.get_handler(request.path) + + response = await handler.handle(request) + + if not isinstance(response, Response): + response = Response(code=200, body=response) + + self._writer.write(response.to_bytes()) + await self._writer.drain() +``` + +这里,一个 `HTTPConnection` 的实例使用了 `HTTPServer` 中的 `router` 对象来得到一个生成响应的对象。一个路由可以是任何一个拥有 `get_handler` 方法的对象,这个方法接收一个字符串作为参数,返回一个可调用的对象或者抛出 `NotFoundException` 异常。而这个可调用的对象被用来处理请求以及生成响应。处理程序由框架的使用者编写,如上文所说的那样,应该返回字符串或者 `Response` 对象。`Response` 对象提供了一个友好的接口,因此这个简单的 if 语句保证了无论处理程序返回什么,代码最终都得到一个统一的 `Response` 对象。 + +接下来,被赋值给 `self._writer` 的 `StreamWriter` 实例被调用,将字节字符串发送回客户端。函数返回前,程序在 `await self._writer.drain()` 处等待,以确保所有的数据被发送给客户端。只要缓存中还有未发送的数据,`self._writer.close()` 方法就不会执行。 + +`HTTPConnection` 类还有两个更加有趣的部分:一个用于关闭连接的方法,以及一组用来处理超时机制的方法。首先,关闭一条连接由下面这个小函数完成: + +``` +def close_connection(self): + self._cancel_conn_timeout() + self._writer.close() +``` + +每当一条连接将被关闭时,这段代码首先取消超时,然后把连接从事件循环中清除。 + +超时机制由三个相关的函数组成:第一个函数在超时后给客户端发送错误消息并关闭连接;第二个函数用于取消当前的超时;第三个函数调度超时功能。前两个函数比较简单,我将详细解释第三个函数 `_reset_cpmm_timeout()` 。 + +``` +def _conn_timeout_close(self): + self.error_reply(500, 'timeout') + self.close_connection() + +def _cancel_conn_timeout(self): + if self._conn_timeout: + self._conn_timeout.cancel() + +def _reset_conn_timeout(self, timeout=TIMEOUT): + self._cancel_conn_timeout() + self._conn_timeout = self.loop.call_later( + timeout, self._conn_timeout_close) +``` + +每当 `_reset_conn_timeout` 函数被调用时,它会先取消之前所有赋值给 `self._conn_timeout` 的 `asyncio.Handle` 对象。然后,使用 [BaseEventLoop.call_later][12] 函数让 `_conn_timeout_close` 函数在超时数秒(`timeout`)后执行。如果你还记得 `handle_request` 函数的内容,就知道每当接收到数据时,这个函数就会被调用。这就取消了当前的超时并且重新安排 `_conn_timeout_close` 函数在超时数秒(`timeout`)后执行。只要接收到数据,这个循环就会不断地重置超时回调。如果在超时时间内没有接收到数据,最后函数 `_conn_timeout_close` 就会被调用。 + +### 创建连接 + +我们需要创建 `HTTPConnection` 对象,并且正确地使用它们。这一任务由 `HTTPServer` 类完成。`HTTPServer` 类是一个简单的容器,可以存储着一些配置信息(解析器,路由和事件循环实例),并使用这些配置来创建 `HTTPConnection` 实例: + +``` +class HTTPServer(object): + def init(self, router, http_parser, loop): + self.router = router + self.http_parser = http_parser + self.loop = loop + + async def handle_connection(self, reader, writer): + connection = HTTPConnection(self, reader, writer) + asyncio.ensure_future(connection.handle_request(), loop=self.loop) +``` + +`HTTPServer` 的每一个实例能够监听一个端口。它有一个 `handle_connection` 的异步方法来创建 `HTTPConnection` 的实例,并安排它们在事件循环中运行。这个方法被传递给 [asyncio.start_server][13] 作为一个回调函数。也就是说,每当一个 TCP 连接初始化时(以 `StreamReader` 和 `StreamWriter` 为参数),它就会被调用。 + +``` + self._server = HTTPServer(self.router, self.http_parser, self.loop) + self._connection_handler = asyncio.start_server( + self._server.handle_connection, + host=self.host, + port=self.port, + reuse_address=True, + reuse_port=True, + loop=self.loop) +``` + +这就是构成整个应用程序工作原理的核心:`asyncio.start_server` 接受 TCP 连接,然后在一个预配置的 `HTTPServer` 对象上调用一个方法。这个方法将处理一条 TCP 连接的所有逻辑:读取、解析、生成响应并发送回客户端、以及关闭连接。它的重点是 IO 逻辑、解析和生成响应。 + +讲解了核心的 IO 部分,让我们继续。 + +### 解析请求 + +这个微型框架的使用者被宠坏了,不愿意和字节打交道。它们想要一个更高层次的抽象 —— 一种更加简单的方法来处理请求。这个微型框架就包含了一个简单的 HTTP 解析器,能够将字节流转化为 Request 对象。 + +这些 Request 对象是像这样的容器: + +``` +class Request(object): + def init(self): + self.method = None + self.path = None + self.query_params = {} + self.path_params = {} + self.headers = {} + self.body = None + self.body_raw = None + self.finished = False +``` + +它包含了所有需要的数据,可以用一种容易理解的方法从客户端接受数据。哦,不包括 cookie ,它对身份认证是非常重要的,我会将它留在第二部分。 + +每一个 HTTP 请求都包含了一些必需的内容,如请求路径和请求方法。它们也包含了一些可选的内容,如请求体、请求头,或是 URL 参数。随着 REST 的流行,除了 URL 参数,URL 本身会包含一些信息。比如,"/user/1/edit" 包含了用户的 id 。 + +一个请求的每个部分都必须被识别、解析,并正确地赋值给 Request 对象的对应属性。HTTP/1.1 是一个文本协议,事实上这简化了很多东西。(HTTP/2 是一个二进制协议,这又是另一种乐趣了) + +解析器不需要跟踪状态,因此 `http_parser` 模块其实就是一组函数。调用函数需要用到 `Request` 对象,并将它连同一个包含原始请求信息的字节数组传递给 `parse_into` 函数。然后解析器会修改 `Request` 对象以及充当缓存的字节数组。字节数组的信息被逐渐地解析到 request 对象中。 + +`http_parser` 模块的核心功能就是下面这个 `parse_into` 函数: + +``` +def parse_into(request, buffer): + _buffer = buffer[:] + if not request.method and can_parse_request_line(_buffer): + (request.method, request.path, + request.query_params) = parse_request_line(_buffer) + remove_request_line(_buffer) + + if not request.headers and can_parse_headers(_buffer): + request.headers = parse_headers(_buffer) + if not has_body(request.headers): + request.finished = True + + remove_intro(_buffer) + + if not request.finished and can_parse_body(request.headers, _buffer): + request.body_raw, request.body = parse_body(request.headers, _buffer) + clear_buffer(_buffer) + request.finished = True + return _buffer +``` + +从上面的代码中可以看到,我把解析的过程分为三个部分:解析请求行(这行像这样:GET /resource HTTP/1.1),解析请求头以及解析请求体。 + +请求行包含了 HTTP 请求方法以及 URL 地址。而 URL 地址则包含了更多的信息:路径、url 参数和开发者自定义的 url 参数。解析请求方法和 URL 还是很容易的 - 合适地分割字符串就好了。函数 `urlparse.parse` 可以用来解析 URL 参数。开发者自定义的 URL 参数可以通过正则表达式来解析。 + +接下来是 HTTP 头部。它们是一行行由键值对组成的简单文本。问题在于,可能有多个 HTTP 头有相同的名字,却有不同的值。一个值得关注的 HTTP 头部是 `Content-Length`,它描述了请求体的字节长度(不是整个请求,仅仅是请求体)。这对于决定是否解析请求体有很重要的作用。 + +最后,解析器根据 HTTP 方法和头部来决定是否解析请求体。 + +### 路由! + +在某种意义上,路由就像是连接框架和用户的桥梁,用户用合适的方法创建 `Router` 对象并为其设置路径/函数对,然后将它赋值给 App 对象。而 App 对象依次调用 `get_handler` 函数生成相应的回调函数。简单来说,路由就负责两件事,一是存储路径/函数对,二是返回需要的路径/函数对 + +`Router` 类中有两个允许最终开发者添加路由的方法,分别是 `add_routes` 和 `add_route`。因为 `add_routes` 就是 `add_route` 函数的一层封装,我们将主要讲解 `add_route` 函数: + +``` +def add_route(self, path, handler): + compiled_route = self.class.build_route_regexp(path) + if compiled_route not in self.routes: + self.routes[compiled_route] = handler + else: + raise DuplicateRoute +``` + +首先,这个函数使用 `Router.build_router_regexp` 的类方法,将一条路由规则(如 '/cars/{id}' 这样的字符串),“编译”到一个已编译的正则表达式对象。这些已编译的正则表达式用来匹配请求路径,以及解析开发者自定义的 URL 参数。如果已经存在一个相同的路由,程序就会抛出一个异常。最后,这个路由/处理程序对被添加到一个简单的字典`self.routes`中。 + +下面展示 Router 是如何“编译”路由的: + +``` +@classmethod +def build_route_regexp(cls, regexp_str): + """ + Turns a string into a compiled regular expression. Parses '{}' into + named groups ie. '/path/{variable}' is turned into + '/path/(?P[a-zA-Z0-9_-]+)'. + + :param regexp_str: a string representing a URL path. + :return: a compiled regular expression. + """ + def named_groups(matchobj): + return '(?P<{0}>[a-zA-Z0-9_-]+)'.format(matchobj.group(1)) + + re_str = re.sub(r'{([a-zA-Z0-9_-]+)}', named_groups, regexp_str) + re_str = ''.join(('^', re_str, '$',)) + return re.compile(re_str) +``` + +这个方法使用正则表达式将所有出现的 `{variable}` 替换为 `(?P)`。然后在字符串头尾分别添加 `^` 和 `$` 标记,最后编译正则表达式对象。 + +完成了路由存储仅成功了一半,下面是如何得到路由对应的函数: + +``` +def get_handler(self, path): + logger.debug('Getting handler for: {0}'.format(path)) + for route, handler in self.routes.items(): + path_params = self.class.match_path(route, path) + if path_params is not None: + logger.debug('Got handler for: {0}'.format(path)) + wrapped_handler = HandlerWrapper(handler, path_params) + return wrapped_handler + + raise NotFoundException() +``` + +一旦 `App` 对象获得一个 `Request` 对象,也就获得了 URL 的路径部分(如 /users/15/edit)。然后,我们需要匹配函数来生成一个响应或者 404 错误。`get_handler` 函数将路径作为参数,循环遍历路由,对每条路由调用 `Router.match_path` 类方法检查是否有已编译的正则对象与这个请求路径匹配。如果存在,我们就调用 `HandleWrapper` 来包装路由对应的函数。`path_params` 字典包含了路径变量(如 '/users/15/edit' 中的 '15'),若路由没有指定变量,字典就为空。最后,我们将包装好的函数返回给 `App` 对象。 + +如果遍历了所有的路由都找不到与路径匹配的,函数就会抛出 `NotFoundException` 异常。 + +这个 `Route.match` 类方法挺简单: + +``` +def match_path(cls, route, path): + match = route.match(path) + try: + return match.groupdict() + except AttributeError: + return None +``` + +它使用正则对象的 [match 方法][14]来检查路由是否与路径匹配。若果不匹配,则返回 None 。 + +最后,我们有 `HandleWraapper` 类。它的唯一任务就是封装一个异步函数,存储 `path_params` 字典,并通过 `handle` 方法对外提供一个统一的接口。 + +``` +class HandlerWrapper(object): + def init(self, handler, path_params): + self.handler = handler + self.path_params = path_params + self.request = None + + async def handle(self, request): + return await self.handler(request, **self.path_params) +``` + +### 组合到一起 + +框架的最后部分就是用 `App` 类把所有的部分联系起来。 + +`App` 类用于集中所有的配置细节。一个 `App` 对象通过其 `start_server` 方法,使用一些配置数据创建一个 `HTTPServer` 的实例,然后将它传递给 [asyncio.start_server 函数][15]。`asyncio.start_server` 函数会对每一个 TCP 连接调用 `HTTPServer` 对象的 `handle_connection` 方法。 + +``` +def start_server(self): + if not self._server: + self.loop = asyncio.get_event_loop() + self._server = HTTPServer(self.router, self.http_parser, self.loop) + self._connection_handler = asyncio.start_server( + self._server.handle_connection, + host=self.host, + port=self.port, + reuse_address=True, + reuse_port=True, + loop=self.loop) + + logger.info('Starting server on {0}:{1}'.format( + self.host, self.port)) + self.loop.run_until_complete(self._connection_handler) + + try: + self.loop.run_forever() + except KeyboardInterrupt: + logger.info('Got signal, killing server') + except DiyFrameworkException as e: + logger.error('Critical framework failure:') + logger.error(e.traceback) + finally: + self.loop.close() + else: + logger.info('Server already started - {0}'.format(self)) +``` + +### 总结 + +如果你查看源码,就会发现所有的代码仅 320 余行(包括测试代码的话共 540 余行)。这么少的代码实现了这么多的功能,让我有点惊讶。这个框架没有提供模板、身份认证以及数据库访问等功能(这些内容也很有趣哦)。这也让我知道,像 Django 和 Tornado 这样的框架是如何工作的,而且我能够快速地调试它们了。 + +这也是我按照测试驱动开发完成的第一个项目,整个过程有趣而有意义。先编写测试用例迫使我思考设计和架构,而不仅仅是把代码放到一起,让它们可以运行。不要误解我的意思,有很多时候,后者的方式更好。不过如果你想给确保这些不怎么维护的代码在之后的几周甚至几个月依然工作,那么测试驱动开发正是你需要的。 + +我研究了下[整洁架构][16]以及依赖注入模式,这些充分体现在 `Router` 类是如何作为一个更高层次的抽象的(实体?)。`Router` 类是比较接近核心的,像 `http_parser` 和 `App` 的内容比较边缘化,因为它们只是完成了极小的字符串和字节流、或是中层 IO 的工作。测试驱动开发(TDD)迫使我独立思考每个小部分,这使我问自己这样的问题:方法调用的组合是否易于理解?类名是否准确地反映了我正在解决的问题?我的代码中是否很容易区分出不同的抽象层? + +来吧,写个小框架,真的很有趣:) + +-------------------------------------------------------------------------------- + +via: http://mattscodecave.com/posts/simple-python-framework-from-scratch.html + +作者:[Matt][a] +译者:[Cathon](https://github.com/Cathon) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://mattscodecave.com/hire-me.html +[1]: https://github.com/sirMackk/diy_framework +[2]:https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol +[3]: https://en.wikipedia.org/wiki/HTTP_persistent_connection +[4]: https://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm#Slow_start +[5]: https://docs.python.org/3/library/asyncio-stream.html +[6]: https://docs.python.org/3/library/asyncio-protocol.html +[7]: https://github.com/sirMackk/diy_framework/blob/88968e6b30e59504251c0c7cd80abe88f51adb79/diy_framework/http_server.py#L46 +[8]: https://docs.python.org/3/library/functions.html#bytearray +[9]: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Handle +[10]: https://en.wikipedia.org/wiki/Dependency_injection +[11]: https://docs.python.org/3/library/unittest.mock.html +[12]: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.BaseEventLoop.call_later +[13]: https://docs.python.org/3/library/asyncio-stream.html#asyncio.start_server +[14]: https://docs.python.org/3/library/re.html#re.match +[15]: https://docs.python.org/3/library/asyncio-stream.html?highlight=start_server#asyncio.start_server +[16]: https://blog.8thlight.com/uncle-bob/2012/08/13/the-clean-architecture.html diff --git a/translated/tech/20160608 Simple Python Framework from Scratch.md b/translated/tech/20160608 Simple Python Framework from Scratch.md deleted file mode 100644 index 0ebdd1c0b2..0000000000 --- a/translated/tech/20160608 Simple Python Framework from Scratch.md +++ /dev/null @@ -1,466 +0,0 @@ -从零构建一个简单的 Python 框架 -=================================== - -为什么你想要自己构建一个 web 框架呢?我想,原因有以下几点: - -- 你有一个新奇的想法,觉得将会取代其他的框架 -- 你想要获得一些名气 -- 你的问题领域很独特,以至于现有的框架不太合适 -- 你对 web 框架是如何工作的很感兴趣,因为你想要成为一位更好的 web 开发者。 - -接下来的笔墨将着重于最后一点。这篇文章旨在通过对设计和实现过程一步一步的阐述告诉读者,我在完成一个小型的服务器和框架之后学到了什么。你可以在这个[代码仓库][1]中找到这个项目的完整代码。 - -我希望这篇文章可以鼓励更多的人来尝试,因为这确实很有趣。它让我知道了 web 应用是如何工作的,而且这比我想的要容易的多! - -### 范围 - -框架可以处理请求-响应周期,身份认证,数据库访问,模板生成等。Web 开发者使用框架是因为,大多数的 web 应用拥有大量相同的功能,而对每个项目都重新实现同样的功能意义不大。 - -比较大的的框架如 Rails 和 Django 实现了高层次的抽象,开箱即用功能齐备。而实现所有的这些功能可能要花费数千小时,因此在这个项目上,我们重点完成其中的一小部分。在开始写代码前,我先列举一下所需的功能以及限制。 - -功能: - -- 处理 HTTP 的 GET 和 POST 请求。你可以在[这篇 wiki][2] 中对 HTTP 有个大致的了解。 -- 实现异步操作(我喜欢 Python 3 的 asyncio 模块)。 -- 简单的路由逻辑以及参数捕获。 -- 像其他微型框架一样,提供一个简单的用户级 API 。 -- 支持身份认证,因为学会这个很酷啊(微笑)。 - -限制: - -- 将只支持 HTTP 1.1 的一个小子集,不支持传输编码,http-auth,内容编码(gzip)以及[持久化连接][3]。 -- 不支持对响应的 MIME 判断 - 用户需要手动设置。 -- 不支持 WSGI - 仅能处理简单的 TCP 连接。 -- 不支持数据库。 - -我觉得一个小的用例可以让上述内容更加具体,也可以用来演示这个框架的 API: - -``` -from diy_framework import App, Router -from diy_framework.http_utils import Response - - -# GET simple route -async def home(r): - rsp = Response() - rsp.set_header('Content-Type', 'text/html') - rsp.body = 'test' - return rsp - - -# GET route + params -async def welcome(r, name): - return "Welcome {}".format(name) - -# POST route + body param -async def parse_form(r): - if r.method == 'GET': - return 'form' - else: - name = r.body.get('name', '')[0] - password = r.body.get('password', '')[0] - - return "{0}:{1}".format(name, password) - -# application = router + http server -router = Router() -router.add_routes({ - r'/welcome/{name}': welcome, - r'/': home, - r'/login': parse_form,}) - -app = App(router) -app.start_server() -``` -' -用户需要定义一些能够返回字符串或响应对象的异步函数,然后将这些函数与表示路由的字符串配对,最后通过一个函数(start_server)调用开始处理请求。 - - -完成设计之后,我将它抽象为几个我需要编码的部分: - -- 接受 TCP 连接以及调度一个异步函数来处理这些连接的部分 -- 将原始文本解析成某种抽象容器的部分 -- 对于每个请求,用来决定调用哪个函数的部分 -- 将上述部分集中到一起,并为开发者提供一个简单接口的部分 - -我先编写一些测试,这些测试被用来描述每个部分的功能。几次重构后,整个设计被分成若干部分,每个部分之间是相对解耦的。这样就非常好,因为每个部分可以被独立地研究学习。以下是我上文列出的抽象的具体体现: - -- 一个 HTTPServer 对象,需要一个 Router 对象和一个 http_parser 模块,并使用它们来初始化。 -- HTTPConnection 对象,每一个对象表示一个单独的客户端 HTTP 连接,并且处理其请求-响应周期:使用 http_parser 模块将收到的字节流解析为一个 Request 对象;使用一个 Router 实例寻找并调用正确的函数来生成一个响应; 最后将这个响应发送回客户端。 -- 一对 Request 和 Response 对象为用户提供了一种友好的方式,来处理实质上是字节流的字符串。用户不需要知道正确的消息格式和分隔符。 -- 一个 Router 对象包含路由:函数对。它提供一个添加配对的方法,还有一个根据 URL 路径查找相应函数的方法。 -- 最后,一个 App 对象。它包含配置信息,并使用它们实例化一个 HTTPServer 实例。 - -让我们从 HTTPConnection 开始来讲解各个部分。 - -### 模拟异步连接 - -为了满足约束条件,每一个 HTTP 请求都是一个单独的 TCP 连接。这使得处理请求的速度变慢了,因为建立多个 TCP 连接需要相对高的花销(DNS 查询,TCP 三次握手,[慢启动][4]的花销等等),不过这样更加容易模拟。对于这一任务,我选择相对高级的 [asyncio-stream][5] 模块,它建立在 asyncio 的传输和协议的基础之上。我强烈推荐阅读标准库中的相应代码! - -一个 HTTPConnection 的实例能够处理多个任务。首先,它使用 asyncio.StreamReader 对象以增量的方式从 TCP 连接中读取数据,并存储在缓存中。每一个读取操作完成后,它会尝试解析缓存中的数据,并生成一个 Request 对象。一旦收到了完整的请求,它就生成一个回复,并通过 asyncio.StreamWriter 对象发送回客户端。当然,它还有两个任务:超时连接以及错误处理。 - -你可以在[这里][7]浏览这个类的完整代码。我将分别介绍代码的每一部分。为了简单起见,我移除了代码文档。 - -``` -class HTTPConnection(object): - def init(self, http_server, reader, writer): - self.router = http_server.router - self.http_parser = http_server.http_parser - self.loop = http_server.loop - - self._reader = reader - self._writer = writer - self._buffer = bytearray() - self._conn_timeout = None - self.request = Request() -``` - -这个 init 方法仅仅收集了一些对象以供后面使用。它存储了一个 router 对象,一个 http_parser 对象以及 loop 对象,分别用来生成响应,解析请求以及在事件循环中调度任务。 - -然后,它存储了代表一个 TCP 连接的读写对,和一个充当缓冲区的空[字节数组][8]。_conn_timeout 存储了一个 [asyncio.Handle][9] 的实例,用来管理超时逻辑。最后,它还存储了一个 Request 对象的实例。 - -下面的代码是用来接受和发送数据的核心功能: - -``` -async def handle_request(self): - try: - while not self.request.finished and not self._reader.at_eof(): - data = await self._reader.read(1024) - if data: - self._reset_conn_timeout() - await self.process_data(data) - if self.request.finished: - await self.reply() - elif self._reader.at_eof(): - raise BadRequestException() - except (NotFoundException, - BadRequestException) as e: - self.error_reply(e.code, body=Response.reason_phrases[e.code]) - except Exception as e: - self.error_reply(500, body=Response.reason_phrases[500]) - - self.close_connection() -``` - -所有内容被包含在 try-except 代码块中,在解析请求或响应期间抛出的异常可以被捕获,然后一个错误响应会发送回客户端。 - -在 while 循环中不断读取请求,直到解析器将 self.request.finished 设置为 True 或者客户端关闭连接,使得 self._reader_at_eof() 函数返回值为 True 为止。这段代码尝试在每次循环迭代中从 StreamReader 中读取数据,并通过调用 self.process_data(data) 函数以增量方式生成 self.request 。每次循环读取数据时,连接超时计数器被重值。 - -这儿有个错误,你发现了吗?稍后我们会再讨论这个。需要注意的是,这个循环可能会耗尽 CPU 资源,因为如果没有消息读取 self._reader.read() 函数将会返回一个空的字节对象。这就意味着循环将会不断运行,却什么也不做。一个可能的解决方法是,用非阻塞的方式等待一小段时间:await asyncio.sleep(0.1)。我们暂且不对它做优化。 - -还记得上一段我提到的那个错误吗?只有 StreamReader 读取数据时,self._reset_conn_timeout() 函数才会被调用。这就意味着,直到第一个字节到达时,timeout 才被初始化。如果有一个客户端建立了与服务器的连接却不发送任何数据,那就永远不会超时。这可能被用来消耗系统资源,或导致拒绝式服务攻击。修复方法就是在 init 函数中调用 self._reset_conn_timeout() 函数。 - -当请求接受完成或连接中断时,程序将运行到 if-else 代码块。这部分代码会判断解析器收到完整的数据后是否完成了解析。如果是,好,生成一个回复并发送回客户端。如果不是,那么请求信息可能有错误,抛出一个异常!最后,我们调用 self.close_connection 执行清理工作。 - -解析请求的部分在 self.process_data 方法中。这个方法非常简短,也易于测试: - -``` -async def process_data(self, data): - self._buffer.extend(data) - - self._buffer = self.http_parser.parse_into( - self.request, self._buffer) -``` - -每一次调用都将数据累积到 self._buffer 中,然后试着用 self.http_parser 来解析已经收集的数据。这里需要指出的是,这段代码展示了一种称为[依赖注入][10]的模式。如果你还记得 init 函数的话,应该知道我们传入了一个包含 http_parser 的 http_server 对象。在这个例子里,http_parser 对象是 diy_framework 包中的一个模块。不过它也可以是任何含有 parse_into 函数的类。这个 parse_into 函数接受一个 Request 对象以及字节数组作为参数。这很有用,原因有二。一是,这意味着这段代码更易扩展。如果有人想通过一个不同的解析器来使用 HTTPConnection,没问题,只需将它作为参数传入即可。二是,这使得测试更加容易,因为 http_parser 不是硬编码的,所以使用虚假数据或者 [mock][11] 对象来替代是很容易的。 - -下一段有趣的部分就是 reply 方法了: - -``` -async def reply(self): - request = self.request - handler = self.router.get_handler(request.path) - - response = await handler.handle(request) - - if not isinstance(response, Response): - response = Response(code=200, body=response) - - self._writer.write(response.to_bytes()) - await self._writer.drain() -``` - -这里,一个 HTTPConnection 的实例使用了 HTTPServer 中的路由对象来得到一个生成响应的对象。一个路由可以是任何一个拥有 get_handler 方法的对象,这个方法接收一个字符串作为参数,返回一个可调用的对象或者抛出 NotFoundException 异常。而这个可调用的对象被用来处理请求以及生成响应。处理程序由框架的使用者编写,如上文所说的那样,应该返回字符串或者 Response 对象。Response 对象提供了一个友好的接口,因此这个简单的 if 语句保证了无论处理程序返回什么,代码最终都得到一个统一的 Response 对象。 - -接下来,赋值给 self._writer 的 StreamWriter 实例被调用,将字节字符串发送回客户端。函数返回前,程序在 self._writer.drain() 处等待,以确保所有的数据被发送给客户端。只要缓存中还有未发送的数据,self._writer.close() 方法就不会执行。 - -HTTPConnection 类还有两个更加有趣的部分:一个用于关闭连接的方法,以及一组用来处理超时机制的方法。首先,关闭一条连接由下面这个小函数完成: - -``` -def close_connection(self): - self._cancel_conn_timeout() - self._writer.close() -``` - -每当一条连接将被关闭时,这段代码首先取消超时,然后把连接从事件循环中清除。 - -超时框架由三个有关的函数组成:第一个函数在超时后给客户端发送错误消息并关闭连接; 第二个函数取消当前的超时; 第三个函数调度超时功能。前两个函数比较简单,我将详细解释第三个函数 _reset_cpmm_timeout() 。 - -``` -def _conn_timeout_close(self): - self.error_reply(500, 'timeout') - self.close_connection() - -def _cancel_conn_timeout(self): - if self._conn_timeout: - self._conn_timeout.cancel() - -def _reset_conn_timeout(self, timeout=TIMEOUT): - self._cancel_conn_timeout() - self._conn_timeout = self.loop.call_later( - timeout, self._conn_timeout_close) -``` - -每当 _reset_conn_timeout 函数被调用时,它会先取消之前所有赋值给 self._conn_timeout 的 asyncio.Handle 对象。然后,使用 BaseEventLoop.call_later 函数让 _conn_timeout_close 函数在超时数秒后执行。如果你还记得 handle_request 函数的内容,就知道每当接收到数据时,这个函数就会被调用。这就取消了当前的超时并且重新安排 _conn_timeout_close 函数在超时数秒后执行。只要接收到数据,这个循环就会不断地重置超时回调。如果在超时时间内没有接收到数据,最后函数 _conn_timeout_close 就会被调用。 - -### 创建连接 - -我们需要创建 HTTPConnection 对象,并且正确地使用它们。这一任务由 HTTPServer 类完成。HTTPServer 类是一个简单的容器,可以存储着一些配置信息(解析器,路由和事件循环实例),并使用这些配置来创建 HTTPConnection 实例: - -``` -class HTTPServer(object): - def init(self, router, http_parser, loop): - self.router = router - self.http_parser = http_parser - self.loop = loop - - async def handle_connection(self, reader, writer): - connection = HTTPConnection(self, reader, writer) - asyncio.ensure_future(connection.handle_request(), loop=self.loop) -``` - -HTTPServer 的每一个实例能够监听一个端口。它有一个 handle_connection 的异步方法来创建 HTTPConnection 的实例,并安排它们在事件循环中运行。这个方法被传递给 [asyncio.start_server][12] 作为一个回调函数。也就是说,每当一个 TCP 连接初始化时,它就会被调用。 - -``` - self._server = HTTPServer(self.router, self.http_parser, self.loop) - self._connection_handler = asyncio.start_server( - self._server.handle_connection, - host=self.host, - port=self.port, - reuse_address=True, - reuse_port=True, - loop=self.loop) -``` - -这就是整个应用程序工作原理的核心:asyncio.start_server 接受 TCP 连接,然后在预配置的 HTTPServer 对象上调用一个方法。这个方法将处理一条 TCP 连接的所有逻辑:读取,解析,生成响应并发送回客户端,以及关闭连接。它的重点是 IO 逻辑,解析和生成响应。 - -讲解了核心的 IO 部分,让我们继续。 - -### 解析请求 - -这个微型框架的使用者被宠坏了,不愿意和字节打交道。它们想要一个更高层次的抽象 - 一种更加简单的方法来处理请求。这个微型框架就包含了一个简单的 HTTP 解析器,能够将字节流转化为 Request 对象。 - -这些 Request 对象是像这样的容器: - -``` -class Request(object): - def init(self): - self.method = None - self.path = None - self.query_params = {} - self.path_params = {} - self.headers = {} - self.body = None - self.body_raw = None - self.finished = False -``` - -它包含了所有需要的数据,可以用一种容易理解的方法从客户端接受数据。哦,不包括 cookie ,它对身份认证是非常重要的,我会将它留在第二部分。 - -每一个 HTTP 请求都包含了一些必需的内容,如请求路径和请求方法。它们也包含了一些可选的内容,如请求体,请求头,或是 URL 参数。随着 REST 的流行,除了 URL 参数,URL 本身会包含一些信息。比如,"/user/1/edit" 包含了用户的 id 。 - -一个请求的每个部分都必须被识别,解析,并正确地赋值给 Request 对象的对应属性。HTTP/1.1 是一个文本协议,这简化了很多。(HTTP/2 是一个二进制协议,这又是另一种乐趣了) - -The http_parser module is a group of functions inside because the parser does not need to keep track of state. Instead, the calling code has to manage a Request object and pass it into the parse_into function along with a bytearray containing the raw bytes of a request. To this end, the parser modifies both the request object as well as the bytearray buffer passed to it. The request object gets fuller and fuller while the bytearray buffer gets emptier and emptier. - -解析器不需要跟踪状态,因此 http_parser 模块其实就是一组函数。调用函数需要用到 Request 对象,并将它连同一个包含原始请求信息的字节数组传递给 parse_into 函数。然后解析器会修改 request 对象以及充当缓存的字节数组。字节数组的信息被逐渐地解析到 request 对象中。 - -http_parser 模块的核心功能就是下面这个 parse_into 函数: - -``` -def parse_into(request, buffer): - _buffer = buffer[:] - if not request.method and can_parse_request_line(_buffer): - (request.method, request.path, - request.query_params) = parse_request_line(_buffer) - remove_request_line(_buffer) - - if not request.headers and can_parse_headers(_buffer): - request.headers = parse_headers(_buffer) - if not has_body(request.headers): - request.finished = True - - remove_intro(_buffer) - - if not request.finished and can_parse_body(request.headers, _buffer): - request.body_raw, request.body = parse_body(request.headers, _buffer) - clear_buffer(_buffer) - request.finished = True - return _buffer -``` - -从上面的代码中可以看到,我把解析的过程分为三个部分:解析请求行(请求行有这样的格式:GET /resource HTTP/1.1),解析请求头以及解析请求体。 - -请求行包含了 HTTP 请求方法以及 URL 地址。而 URL 地址则包含了更多的信息:路径,url 参数和开发者自定义的 url 参数。解析请求方法和 URL 还是很容易的 - 合适地分割字符串就好了。函数 urlparse.parse 可以用来解析 URL 参数。开发者自定义的 URL 参数可以通过正则表达式来解析。 - -接下来是 HTTP 头部。它们是一行行由键值对组成的简单文本。问题在于,可能有多个 HTTP 头有相同的名字,却有不同的值。一个值得关注的 HTTP 头部是 Content-Length,它描述了请求体的字节长度(仅仅是请求体)。这对于决定是否解析请求体有很重要的作用。 - -最后,解析器根据 HTTP 方法和头部来决定是否解析请求体。 - -### 路由! - -在某种意义上,路由就像是连接框架和用户的桥梁,用户用合适的方法创建 Router 对象并为其设置路径/函数对,然后将它赋值给 App 对象。而 App 对象依次调用 get_handler 函数生成相应的回调函数。简单来说,路由就负责两件事,一是存储路径/函数对,二是返回需要的路径/函数对 - -Router 类中有两个允许开发者添加路由的方法,分别是 add_routes 和 add_route。因为 add_routes 就是 add_route 函数的一层封装,我们将主要讲解 add_route 函数: - -``` -def add_route(self, path, handler): - compiled_route = self.class.build_route_regexp(path) - if compiled_route not in self.routes: - self.routes[compiled_route] = handler - else: - raise DuplicateRoute -``` - -首先,这个函数使用 Router.build_router_regexp 的类方法,将一条路由规则(如 '/cars/{id}' 这样的字符串),“编译”到一个已编译的正则表达式对象。这些已编译的正则表达式用来匹配请求路径,以及解析开发者自定义的 URL 参数。如果存在一个相同的路由,程序就会抛出一个异常。最后,这个路由/处理程序对被添加到一个简单的字典(self.routes)中。 - -下面展示 Router 是如何“编译”路由的: - -``` -@classmethod -def build_route_regexp(cls, regexp_str): - """ - Turns a string into a compiled regular expression. Parses '{}' into - named groups ie. '/path/{variable}' is turned into - '/path/(?P[a-zA-Z0-9_-]+)'. - - :param regexp_str: a string representing a URL path. - :return: a compiled regular expression. - """ - def named_groups(matchobj): - return '(?P<{0}>[a-zA-Z0-9_-]+)'.format(matchobj.group(1)) - - re_str = re.sub(r'{([a-zA-Z0-9_-]+)}', named_groups, regexp_str) - re_str = ''.join(('^', re_str, '$',)) - return re.compile(re_str) -``` - -The method uses regular expressions to substitute all occurrences of "{variable}" with named regexp groups: "(?P...)". Then it adds the ^ and $ regexp signifiers at the beginning and end of the resulting string, and finally compiles regexp object out of it. - -这个方法使用正则表达式将所有出现的 "{variable}" 替换为 "(?P)"。然后在字符串头尾分别添加 ^ 和 $ 标记,最后编译正则表达式对象。 - -完成了路由存储仅成功了一半,下面是如何得到路由对应的函数: - -``` -def get_handler(self, path): - logger.debug('Getting handler for: {0}'.format(path)) - for route, handler in self.routes.items(): - path_params = self.class.match_path(route, path) - if path_params is not None: - logger.debug('Got handler for: {0}'.format(path)) - wrapped_handler = HandlerWrapper(handler, path_params) - return wrapped_handler - - raise NotFoundException() -``` - -一旦 App 对象获得一个 Request 对象,也就获得了 URL 的路径部分(如 /users/15/edit)。然后,我们需要匹配函数来生成一个响应或者 404 错误。get_handler 函数将路径作为参数,循环遍历路由,对每条路由调用 Router.match_path 类方法检查是否有已编译的正则对象与这个请求路径匹配。如果存在,我们就调用 HandleWrapper 来包装路由对应的函数。path_params 字典包含了路径变量(如 '/users/15/edit' 中的 '15'),若路由没有指定变量,字典就为空。最后,我们将包装好的函数返回给 App 对象。 - -如果遍历了所有的路由都找不到与路径匹配的,函数就会抛出 NotFoundException 异常。 - -这个 Route.match 类方法挺简单: - -``` -def match_path(cls, route, path): - match = route.match(path) - try: - return match.groupdict() - except AttributeError: - return None -``` - -它使用正则对象的 match 方法来检查路由是否与路径匹配。若果不匹配,则返回 None 。 - -最后,我们有 HandleWraapper 类。它的唯一任务就是封装一个异步函数,存储 path_params 字典,并通过 handle 方法对外提供一个统一的接口。 - -``` -class HandlerWrapper(object): - def init(self, handler, path_params): - self.handler = handler - self.path_params = path_params - self.request = None - - async def handle(self, request): - return await self.handler(request, **self.path_params) -``` - -### 合并到一起 - -框架的最后部分就是用 App 类把所有的部分联系起来。 - -App 类用于集中所有的配置细节。一个 App 对象通过其 start_server 方法,使用一些配置数据创建一个 HTTPServer 的实例,然后将它传递给 asyncio.start_server 函数。asyncio.start_server 函数会对每一个 TCP 连接调用 HTTPServer 对象的 handle_connection 方法。 - -``` -def start_server(self): - if not self._server: - self.loop = asyncio.get_event_loop() - self._server = HTTPServer(self.router, self.http_parser, self.loop) - self._connection_handler = asyncio.start_server( - self._server.handle_connection, - host=self.host, - port=self.port, - reuse_address=True, - reuse_port=True, - loop=self.loop) - - logger.info('Starting server on {0}:{1}'.format( - self.host, self.port)) - self.loop.run_until_complete(self._connection_handler) - - try: - self.loop.run_forever() - except KeyboardInterrupt: - logger.info('Got signal, killing server') - except DiyFrameworkException as e: - logger.error('Critical framework failure:') - logger.error(e.traceback) - finally: - self.loop.close() - else: - logger.info('Server already started - {0}'.format(self)) -``` - -### 总结 - -如果你查看源码,就会发现所有的代码仅 320 余行(包括测试代码共 540 余行)。这么少的代码实现了这么多的功能,让我有点惊讶。这个框架没有提供模板,身份认证以及数据库访问等功能(这些内容也很有趣哦)。这也让我知道,像 Django 和 Tornado 这样的框架是如何工作的,而且我能够快速地调试它们了。 - -这也是我按照测试驱动开发完成的第一个项目,整个过程有趣而有意义。先编写测试用例迫使我思考设计和架构,而不仅仅是把代码放到一起,让它们可以运行。不要误解我的意思,有很多时候,后者的方式更好。不过如果你想给确保这些不怎么维护的代码在之后的几周甚至几个月依然工作,那么测试驱动开发正是你需要的。 - -我研究了下[整洁架构][13]以及依赖注入模式,这些充分体现在 Router 类是如何作为一个更高层次的抽象的。Router 类是比较接近核心的,像 http_parser 和 App 的内容比较边缘化,因为它们只是完成了极小的字符串和字节流,或是中层 IO 的工作。测试驱动开发迫使我独立思考每个小部分,这使我问自己这样的问题:方法调用的组合是否易于理解?类名是否准确地反映了我正在解决的问题?我的代码中是否很容易区分出不同的抽象层? - -来吧,写个小框架,真的很有趣:) - --------------------------------------------------------------------------------- - -via: http://mattscodecave.com/posts/simple-python-framework-from-scratch.html - -作者:[Matt][a] -译者:[Cathon](https://github.com/Cathon) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://mattscodecave.com/hire-me.html -[1]: https://github.com/sirMackk/diy_framework -[2]:https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol -[3]: https://en.wikipedia.org/wiki/HTTP_persistent_connection -[4]: https://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm#Slow_start -[5]: https://docs.python.org/3/library/asyncio-stream.html -[6]: https://docs.python.org/3/library/asyncio-protocol.html -[7]: https://github.com/sirMackk/diy_framework/blob/88968e6b30e59504251c0c7cd80abe88f51adb79/diy_framework/http_server.py#L46 -[8]: https://docs.python.org/3/library/functions.html#bytearray -[9]: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Handle -[10]: https://en.wikipedia.org/wiki/Dependency_injection -[11]: https://docs.python.org/3/library/unittest.mock.html -[12]: https://docs.python.org/3/library/asyncio-stream.html#asyncio.start_server -[13]: https://blog.8thlight.com/uncle-bob/2012/08/13/the-clean-architecture.html From 3f826576a28098be41ee0a037ed60743df45057d Mon Sep 17 00:00:00 2001 From: bazz2 Date: Tue, 13 Sep 2016 10:32:19 +0800 Subject: [PATCH 0118/2437] jiajia9linuxer translating --- ...Photos While Traveling With an Ipad Pro and a Raspberry Pi.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md b/sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md index b7b3e8d5f1..4a5c2f1267 100644 --- a/sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md +++ b/sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md @@ -1,3 +1,4 @@ +[jiajia9linuxer translating] Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi =================================================================== From ff2d4808b04f8638121119a14e64ecac87a70279 Mon Sep 17 00:00:00 2001 From: wenchunyang Date: Tue, 13 Sep 2016 13:20:38 +0800 Subject: [PATCH 0119/2437] finish part2 & part3 --- ...r game with python and asyncio - part 2.md | 234 ++++++++++++++++++ ...r game with python and asyncio - part 3.md | 138 +++++++++++ 2 files changed, 372 insertions(+) create mode 100644 translated/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md create mode 100644 translated/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md diff --git a/translated/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md b/translated/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md new file mode 100644 index 0000000000..73dd55d516 --- /dev/null +++ b/translated/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md @@ -0,0 +1,234 @@ +使用 Python 和 asyncio 编写在线多用人游戏 - 第2部分 +================================================================== + +![](https://7webpages.com/media/cache/fd/d1/fdd1f8f8bbbf4166de5f715e6ed0ac00.gif) + +你曾经写过异步的 Python 程序吗?这里我将告诉你如果如何做,而且在接下来的部分用一个[实例][1] - 专为多玩家设计的、受欢迎的贪吃蛇游戏来演示。 + +介绍和理论部分参见第一部分[异步化[第1部分]][2]。 + +试玩游戏[3]。 + +### 3. 编写游戏循环主体 + +游戏循环是每一个游戏的核心。它持续地读取玩家的输入,更新游戏的状态,并且在屏幕上渲染游戏结果。在在线游戏中,游戏循环分为客户端和服务端两部分,所以一般有两个循环通过网络通信。通常客户端的角色是获取玩家输入,比如按键或者鼠标移动,将数据传输给服务端,然后接收需要渲染的数据。服务端处理来自玩家的所有数据,更新游戏的状态,执行渲染下一帧的必要计算,然后将结果传回客户端,例如游戏中对象的位置。如果没有可靠的理由,不混淆客户端和服务端的角色很重要。如果你在客户端执行游戏逻辑的计算,很容易就会和其它客户端失去同步,其实你的游戏也可以通过简单地传递客户端的数据来创建。 + +游戏循环的一次迭代称为一个嘀嗒。嘀嗒表示当前游戏循环的迭代已经结束,下一帧(或者多帧)的数据已经就绪。在后面的例子中,我们使用相同的客户端,使用 WebSocket 连接服务端。它执行一个简单的循环,将按键码发送给服务端,显示来自服务端的所有信息。[客户端代码戳这里][4]。 + +#### 例子3.1:基本游戏循环 + +[例子3.1源码][5]。 + +我们使用 [aiohttp][6] 库来创建游戏服务器。它可以通过 asyncio 创建网页服务器和客户端。这个库的一个优势是它同时支持普通 http 请求和 websocket。所以我们不用其他网页服务器来渲染游戏的 html 页面。 + +下面是启动服务器的方法: + +``` +app = web.Application() +app["sockets"] = [] + +asyncio.ensure_future(game_loop(app)) + +app.router.add_route('GET', '/connect', wshandler) +app.router.add_route('GET', '/', handle) + +web.run_app(app) +``` + +`web.run_app` 是创建服务主任务的快捷方法,通过他的 `run_forever()` 方法来执行 asyncio 事件循环。建议你查看这个方法的源码,弄清楚服务器到底是如何创建和结束的。 + +`app` 变量就是一个类似于字典的对象,它可以在所连接的客户端之间共享数据。我们使用它来存储连接套接字的列表。随后会用这个列表来给所有连接的客户端发送消息。`asyncio.ensure_future()` 调用会启动主游戏循环的任务,每隔2s向客户端发送嘀嗒消息。这个任务会在同样的 asyncio 事件循环中和网页服务器并行执行。 + +有两个网页请求处理器:提供 html 页面的处理器 (`handle`);`wshandler` 是主要的 websocket 服务器任务,处理和客户端之间的交互。在事件循环中,每一个连接的客户端都会创建一个新的 `wshandler`。 + +在启动的任务中,我们在 asyncio 的主事件循环中启动 worker 循环。任务之间的切换发生在他们任何一个使用 `await`语句来等待某个协程结束。例如 `asyncio.sleep` 仅仅是将程序执行权交给调度器指定的时间;`ws.receive` 等待 websocket 的消息,此时调度器可能切换到其它任务。 + +在浏览器中打开主页,连接上服务器后,试试随便按下键。他们的键值会从服务端返回,每隔2秒这个数字会被游戏循环发给所有客户端的嘀嗒消息覆盖。 + +我们刚刚创建了一个处理客户端按键的服务器,主游戏循环在后台做一些处理,周期性地同时更新所有的客户端。 + +#### 例子 3.2: 根据请求启动游戏 + +[例子 3.2的源码][7] + +在前一个例子中,在服务器的生命周期内,游戏循环一直运行着。但是现实中,如果没有一个人连接服务器,空运行游戏循环通常是不合理的。而且,同一个服务器上可能有不同的’游戏房间‘。在这种假设下,每一个玩家创建一个游戏会话(多人游戏中的一个比赛或者大型多人游戏中的副本),这样其他用户可以加入其中。当游戏会话开始时,游戏循环才开始执行。 + +在这个例子中,我们使用一个全局标记来检测游戏循环是否在执行。当第一个用户发起连接时,启动它。最开始,游戏循环不在执行,标记设置为 `False`。游戏循环是通过客户端的处理方法启动的。 + +``` + if app["game_is_running"] == False: + asyncio.ensure_future(game_loop(app)) +``` + +当游戏的循环(`loop()`)运行时,这个标记设置为 `True`;当所有客户端都断开连接时,其又被设置为 `False`。 + +#### 例子 3.3:管理任务 + +[例子3.3源码][8] + +这个例子用来解释如何和任务对象协同工作。我们把游戏循环的任务直接存储在游戏循环的全局字典中,代替标记的使用。在这个简单例子中并不一定是最优的,但是有时候你可能需要控制所有已经启动的任务。 + +``` + if app["game_loop"] is None or \ + app["game_loop"].cancelled(): + app["game_loop"] = asyncio.ensure_future(game_loop(app)) +``` + +这里 `ensure_future()` 返回我们存放在全局字典中的任务对象,当所有用户都断开连接时,我们使用下面方式取消任务: + +``` + app["game_loop"].cancel() +``` + +这个 `cancel()` 调用将通知所有的调度器不要向这个协程提交任何执行任务,而且将它的状态设置为已取消,之后可以通过 `cancelled()` 方法来检查是否已取消。这里有一个值得一提的小注意点:当你持有一个任务对象的外部引用时,而这个任务执行中抛出了异常,这个异常不会抛出。取而代之的是为这个任务设置一个异常状态,可以通过 `exception()` 方法来检查是否出现了异常。这种悄无声息地失败在调试时不是很有用。所以,你可能想用抛出所有异常来取代这种做法。你可以对所有未完成的任务显示地调用 `result()` 来实现。可以通过如下的回调来实现: + +``` + app["game_loop"].add_done_callback(lambda t: t.result()) +``` + +如果我们打算在我们代码中取消任务,但是又不想产生 `CancelError` 异常,有一个检查 `cancelled` 状态的点: + +``` + app["game_loop"].add_done_callback(lambda t: t.result() + if not t.cancelled() else None) +``` + +注意仅当你持有任务对象的引用时必须要这么做。在前一个例子,所有的异常都是没有额外的回调,直接抛出所有异常。 + +#### 例子 3.4:等待多个事件 + +[例子 3.4 源码][9] + +在许多场景下,在客户端的处理方法中你需要等待多个事件的发生。除了客户端的消息,你可能需要等待不同类型事件的发生。比如,如果你的游戏时间有限制,那么你可能需要等一个来自定时器的信号。或者你需要使用管道来等待来自其它进程的消息。亦或者是使用分布式消息系统网络中其它服务器的信息。 + +为了简单起见,这个例子是基于例子 3.1。但是这个例子中我们使用 `Condition` 对象来保证已连接客户端游戏循环的同步。我们不保存套接字的全局列表,因为只在方法中使用套接字。当游戏循环停止迭代时,我们使用 `Condition.notify_all()` 方法来通知所有的客户端。这个方法允许在 `asyncio` 的事件循环中使用发布/订阅的模式。 + +为了等待两个事件,首先我们使用 `ensure_future()` 来封装任务中可以等待的对象。 + +``` + if not recv_task: + recv_task = asyncio.ensure_future(ws.receive()) + if not tick_task: + await tick.acquire() + tick_task = asyncio.ensure_future(tick.wait()) +``` + +在我们调用 `Condition.wait()` 之前,我们需要在背后获取一把锁。这就是我们为什么先调用 `tick.acquire()` 的原因。在调用 `tick.wait()` 之后,锁会被释放,这样其他的协程也可以使用它。但是当我们收到通知时,会重新获取锁,所以在收到通知后需要调用 `tick.release()` 来释放它。 + +我们使用 `asyncio.wait()` 协程来等待两个任务。 + +``` + done, pending = await asyncio.wait( + [recv_task, + tick_task], + return_when=asyncio.FIRST_COMPLETED) +``` + +程序会阻塞,直到列表中的任意一个任务完成。然后它返回两个列表:执行完成的任务列表和仍然在执行的任务列表。如果任务执行完成了,其对应变量赋值为 `None`,所以在下一个迭代时,它可能会被再次创建。 + +#### 例子 3.5: 结合多个线程 + +[例子 3.5 源码][10] + +在这个例子中,我们结合 asyncio 循环和线程,在一个单独的线程中执行主游戏循环。我之前提到过,由于 `GIL` 的存在,Python 代码的真正并行执行是不可能的。所以使用其它线程来执行复杂计算并不是一个好主意。然而,在使用 `asyncio` 时结合线程有原因的:当我们使用的其它库不支持 `asyncio` 时。在主线程中调用这些库会阻塞循环的执行,所以异步使用他们的唯一方法是在不同的线程中使用他们。 + +在 asyncio 的循环和 `ThreadPoolExecutor` 中,我们通过 `run_in_executor()` 方法来执行游戏循环。注意 `game_loop()` 已经不再是一个协程了。它是一个由其它线程执行的函数。然而我们需要和主线程交互,在游戏事件到来时通知客户端。asyncio 本身不是线程安全的,它提供了可以在其它线程中执行你的代码的方法。普通函数有 `call_soon_threadsafe()`, 协程有 `run_coroutine_threadsafe()`。我们在 `notify()` 协程中增加代码通知客户端游戏的嘀嗒,然后通过另外一个线程执行主事件循环。 + +``` +def game_loop(asyncio_loop): + print("Game loop thread id {}".format(threading.get_ident())) + async def notify(): + print("Notify thread id {}".format(threading.get_ident())) + await tick.acquire() + tick.notify_all() + tick.release() + + while 1: + task = asyncio.run_coroutine_threadsafe(notify(), asyncio_loop) + # blocking the thread + sleep(1) + # make sure the task has finished + task.result() +``` + +当你执行这个例子时,你会看到 "Notify thread id" 和 "Main thread id" 相等,因为 `notify()` 协程在主线程中执行。与此同时 `sleep(1)` 在另外一个线程中执行,因此它不会阻塞主事件循环。 + +#### 例子 3.6:多进程和扩展 + +[例子 3.6 源码][11] + +单线程的服务器可能运行得很好,但是它只能使用一个CPU核。为了将服务扩展到多核,我们需要执行多个进程,每个进程执行各自的事件循环。这样我们需要在进程间交互信息或者共享游戏的数据。而且在一个游戏中经常需要进行复杂的计算,例如路径查找。这些任务有时候在一个游戏嘀嗒中没法快速完成。在协程中不推荐进行费时的计算,因为它会阻塞事件的处理。在这种情况下,将这个复杂任务交给并行执行地其它进程可能更合理。 + +最简单的使用多个核的方法是启动多个使用单核的服务器,就像之前的例子中一样,每个服务器占用不同的端口。你可以使用 `supervisord` 或者其它进程控制的系统。这个时候你需要一个负载均衡器,像 `HAProxy`,使得连接的客户端在多个进程间均匀分布。有一些适配 asyncio 消息系统和存储系统。例如: + +- [aiomcache][12] for memcached client +- [aiozmq][13] for zeroMQ +- [aioredis][14] for Redis storage and pub/sub + +你可以在 github 或者 pypi 上找到其它的安装包,大部分以 `aio` 开头。 + +使用网络服务在存储持久状态和交互信息时可能比较有效。但是如果你需要进行进程通信的实时处理,它的性能可能不足。此时,使用标准的 unix 管道可能更合适。asyncio 支持管道,这个仓库有个 [使用pipe且比较底层的例子][15] + +在当前的例子中,我们使用 Python 的高层库 [multiprocessing][16] 来在不同的核上启动复杂的计算,使用 `multiprocessing.Queue` 来进行进程间的消息交互。不幸的是,当前的 multiprocessing 实现与 asyncio 不兼容。所以每一个阻塞方法的调用都会阻塞事件循环。但是此时线程正好可以起到帮助作用,因为如果在不同线程里面执行 multiprocessing 的代码,它就不会阻塞主线程。所有我们需要做的就是把所有进程间的通信放到另外一个线程中去。这个例子会解释如何使用这个方法。和上面的多线程例子非常类似,但是我们从线程中创建的是一个新的进程。 + +``` +def game_loop(asyncio_loop): + # coroutine to run in main thread + async def notify(): + await tick.acquire() + tick.notify_all() + tick.release() + + queue = Queue() + + # function to run in a different process + def worker(): + while 1: + print("doing heavy calculation in process {}".format(os.getpid())) + sleep(1) + queue.put("calculation result") + + Process(target=worker).start() + + while 1: + # blocks this thread but not main thread with event loop + result = queue.get() + print("getting {} in process {}".format(result, os.getpid())) + task = asyncio.run_coroutine_threadsafe(notify(), asyncio_loop) + task.result() +``` + +这里我们在另外一个进程中运行 `worker()` 函数。它包括一个执行复杂计算的循环,然后把计算结果放到 `queue` 中,这个 `queue` 是 `multiprocessing.Queue` 的实例。然后我们就可以在另外一个线程的主事件循环中获取结果并通知客户端,就是例子 3.5 一样。这个例子已经非常简化了,它没有合理的结束进程。而且在真实的游戏中,我们可能需要另外一个队列来将数据传递给 `worker`。 + +有一个项目叫 [aioprocessing][17],它封装了 multiprocessing,使得它可以和 asyncio 兼容。但是实际上它只是和上面例子使用了完全一样的方法:从线程中创建进程。它并没有给你带来任何方便,除了它使用了简单的接口隐藏了后面的这些技巧。希望在 Python 的下一个版本中,我们能有一个基于协程且支持 asyncio 的 multiprocessing 库。 + +> 注意!如果你从主线程或者主进程中创建了一个不同的线程或者子进程来运行另外一个 asyncio 事件循环,你需要显示地使用 `asyncio.new_event_loop()` 来创建循环,不然的话可能程序不会正常工作。 + +-------------------------------------------------------------------------------- + +via: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-writing-game-loop/ + +作者:[Kyrylo Subbotin][a] +译者:[chunyang-wen](https://github.com/chunyang-wen) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-writing-game-loop/ +[1]: http://snakepit-game.com/ +[2]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ +[3]: http://snakepit-game.com/ +[4]: https://github.com/7WebPages/snakepit-game/blob/master/simple/index.html +[5]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_basic.py +[6]: http://aiohttp.readthedocs.org/ +[7]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_handler.py +[8]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_global.py +[9]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_wait.py +[10]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_thread.py +[11]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_process.py +[12]: https://github.com/aio-libs/aiomcache +[13]: https://github.com/aio-libs/aiozmq +[14]: https://github.com/aio-libs/aioredis +[15]: https://github.com/KeepSafe/aiohttp/blob/master/examples/mpsrv.py +[16]: https://docs.python.org/3.5/library/multiprocessing.html +[17]: https://github.com/dano/aioprocessing diff --git a/translated/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md b/translated/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md new file mode 100644 index 0000000000..2d4a2316bd --- /dev/null +++ b/translated/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md @@ -0,0 +1,138 @@ +使用 Python 和 asyncio 来编写在线多人游戏 - 第3部分 +================================================================= + +![](https://7webpages.com/media/cache/17/81/178135a6db5074c72a1394d31774c658.gif) + +在这个系列中,我们基于多人游戏 [贪吃蛇][1] 来制作一个异步的 Python 程序。前一篇文章聚焦于[编写游戏循环][2]上,而本系列第1部分涵盖了 [异步化][3]。 + +代码戳[这里][4] + +### 4. 制作一个完成的游戏 + +![](https://7webpages.com/static/img/14chs7.gif) + +#### 4.1 工程概览 +#### 4.1 Project's overview + +在此部分,我们将回顾一个完整在线游戏的设计。这是一个经典的贪吃蛇游戏,增加了多玩家支持。你可以自己在 () 亲自试玩。源码在 Github的这个[仓库][5]。游戏包括下列文件: + +- [server.py][6] - 处理主游戏循环和连接服务器。 +- [game.py][7] - 主要 `Game` 类。实现游戏的逻辑和游戏的大部分通信协议。 +- [player.py][8] - `Player` 类,包括每一个独立玩家的数据和蛇的表示。这个类负责获取玩家的输入以及根据输入相应地移动蛇。 +- [datatypes.py][9] - 基本数据结构。 +- [settings.py][10] - 游戏设置,在注释中有相关的说明。 +- [index.html][11] - 一个文件中包括客户端所有的 html 和 javascript代码。 + +#### 4.2 游戏循环内窥 + +多人的贪吃蛇游戏是个十分好的例子,因为它简单。所有的蛇在每个帧中移动到一个位置,而且帧之间的变化频率较低,这样你就可以一探一个游戏引擎到底是如何工作的。因为速度慢,对于玩家的按键不会立马响应。按键先是记录下来,然后在一个游戏迭代的最后计算下一帧时使用。 + +> 现代的动作游戏帧频率更高,而且服务端和客户端的帧频率不相等。客户端的帧频率通常依赖于客户端的硬件性能,而服务端的帧频率是固定的。一个客户端可能根据一个游戏嘀嗒的数据渲染多个帧。这样就可以创建平滑的动画,这个受限于客户端的性能。在这个例子中,服务器不仅传输物体的当前位置,也要传输他们的移动方向,速度和加速度。客户端的帧频率称之为 FPS(frames per second),服务端的帧频率称之为 TPS(ticks per second)。在这个贪吃蛇游戏的例子中,二者的值是相等的,客户端帧的展现和服务端的嘀嗒是同步的。 + +我们使用文本模式一样的游戏区域,事实上是 html 表格中的一个字符宽的小格。游戏中的所有对象都是通过表格中的不同颜色字符来表示。大部分时候,客户端将按键码发送至服务器,然后每个 tick 更新游戏区域。服务端一次更新包括需要更新字符的坐标和颜色。所以我们将所有游戏逻辑放置在服务端,只将需要渲染的数据发送给客户端。此外,我们通过替换网络上发送的数据来最小化游戏被破解的概率。 + +#### 4.3 它是如何运行的? + +这个游戏中的服务端出于简化的目的,它和例子 3.2 类似。但是我们用一个所有服务器都可访问的 Game 对象来代替之前保存所有已连接 websocket 的全局列表。一个 Game 实例包括玩家的列表 (self._players),表示连接到此游戏的玩家,他们的个人数据和 websocket 对象。将所有游戏相关的数据存储在一个 Game 对象中,会方便我们增加多个游戏房间这个功能。这样的话,我们只要维护多个 Game 对象,每个游戏开始时创建相应的 Game 对象。 + +客户端和服务端的所有交互都是通过编码成 json 的消息来完成。来自客户端的消息仅包含玩家所按下键对应的编码。其它来自客户端消息使用如下格式: + +``` +[command, arg1, arg2, ... argN ] +``` + +来自服务端的消息以列表的形式发送,因为通常一次要发送多个消息 (大多数情况下是渲染的数据): + +``` +[[command, arg1, arg2, ... argN ], ... ] +``` + +在每次游戏循环迭代的最后会计算下一帧,并且将数据发送给所有的客户端。当然,每次不是发送完整的帧,而是发送两帧之间的变化列表。 + +注意玩家连接上服务器后不是立马加入游戏。连接开始时是观望者 (spectator) 模式,玩家可以观察其它玩家如何玩游戏。如果游戏已经开始或者上一个游戏会话已经在屏幕上显示 "game over" (游戏结束),用户此时可以按下 "Join" (参与),加入一个已经存在的游戏或者如果游戏不在运行(没有其它玩家)则创建一个新的游戏。后一种情况,游戏区域在开始前会被先清空。 + +游戏区域存储在 `Game._field` 这个属性中,它是二维的嵌套列表,用于内部存储游戏区域的状态。数组中的每一个元素表示区域中的一个小格,最终小格会被渲染成 html 表格的格子。如果它的类型是 Char,它是一个 `namedtuple` ,包括一个字符和颜色。在所有连接的客户端之间保证游戏区域的同步很重要,所以所有游戏区域的更新都必须依据发送到客户端的相应的信息。这是通过 `Game.apply_render()` 来实现的。它接受一个 `Draw` 对象的列表,其用于内部更新游戏区域和发送渲染消息给客户端。 + +我们使用 `namedtuple` 不仅因为它表示简单数据结构很方便,也因为用它生成 json 格式的消息时相对于字典更省空间。如果你在一个真实的游戏循环中需要发送完整的数据结构,建议先将它们序列化成一个简单的,更短的格式,甚至打包成二进制格式(例如 bson,而不是 json),以减少网络传输。 + +`ThePlayer` 对象包括用双端队列表示的蛇。这种数据类型和列表相似,但是在两端增加和删除元素时效率更高,用它来表示蛇很理想。它的主要方法是 `Player.render_move()`,它返回移动玩家蛇至下一个位置的渲染数据。一般来说它在新的位置渲染蛇的头部,移除上一帧中表示蛇的尾巴元素。如果蛇吃了一个数字,需要增长,在相应的多个帧中尾巴是不需要移动的。蛇的渲染数据在主要类的 `Game.next_frame()` 中使用,该方法中实现所有的游戏逻辑。这个方法渲染所有蛇的移动,检查每一个蛇前面的障碍物,而且生成数字和石头。每一个嘀嗒,`game_loop()` 都会直接调用它来生成下一帧。 + +如果蛇头前面有障碍物,在 `Game.next_frame()` 中会调用 `Game.game_over()`。所有的客户端都会收到那个蛇死掉的通知 (会调用 `player.render_game_over()` 方法将其变成石头),然后更新表中的分数排行榜。`Player` 的存活标记被置为 `False`,当渲染下一帧时,这个玩家会被跳过,除非他重新加入游戏。当没有蛇存活时,游戏区域会显示 "game over" (游戏结束) 。而且,主游戏循环会停止,设置 `game.running` 标记为 `False`。当某个玩家下次按下 "Join" (加入) 时,游戏区域会被清空。 + +在渲染游戏的每个下一帧时都会产生数字和石头,他们是由随机值决定的。产生数字或者石头的概率可以在 settings.py 中修改。注意数字是针对游戏区域每一个活的蛇产生的,所以蛇越多,产生的数字就越多,这样他们都有足够的食物来消费。 + +#### 4.4 网络协议 +#### 4.4 Network protocol + +从客户端发送消息的列表: + +Command | Parameters |Description +:-- |:-- |:-- +new_player | [name] |Setting player's nickname +join | |Player is joining the game + + +从服务端发送消息的列表 + +Command | Parameters |Description +:-- |:-- |:-- +handshake |[id] |Assign id to a player +world |[[(char, color), ...], ...] |Initial play field (world) map +reset_world | |Clean up world map, replacing all characters with spaces +render |[x, y, char, color] |Display character at position +p_joined |[id, name, color, score] |New player joined the game +p_gameover |[id] |Game ended for a player +p_score |[id, score] |Setting score for a player +top_scores |[[name, score, color], ...] |Update top scores table + +典型的消息交换顺序 + +Client -> Server |Server -> Client |Server -> All clients |Commentaries +:-- |:-- |:-- |:-- +new_player | | |Name passed to server + |handshake | |ID assigned + |world | |Initial world map passed + |top_scores | |Recent top scores table passed +join | | |Player pressed "Join", game loop started + | |reset_world |Command clients to clean up play field + | |render, render, ... |First game tick, first frame rendered +(key code) | | |Player pressed a key + | |render, render, ... |Second frame rendered + | |p_score |Snake has eaten a digit + | |render, render, ... |Third frame rendered + | | |... Repeat for a number of frames ... + | |p_gameover |Snake died when trying to eat an obstacle + | |top_scores |Updated top scores table (if updated) + +### 5. 总结 + +说实话,我十分享受 Python 最新的异步特性。新的语法很友善,所以异步代码很容易阅读。可以明显看出哪些调用是非阻塞的,什么时候发生 greenthread 的切换。所以现在我可以宣称 Python 是异步编程的好工具。 + +SnakePit 在 7WebPages 团队中非常受欢迎。如果你在公司想休息一下,不要忘记给我们在 [Twitter][12] 或者 [Facebook][13] 留下反馈。 + +更多详见: + +-------------------------------------------------------------------------------- + +via: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-part-3/ + +作者:[Saheetha Shameer][a] +译者:[chunyang-wen](https://github.com/chunyang-wen) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-part-3/ +[1]: http://snakepit-game.com/ +[2]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-writing-game-loop/ +[3]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ +[4]: https://github.com/7WebPages/snakepit-game +[5]: https://github.com/7WebPages/snakepit-game +[6]: https://github.com/7WebPages/snakepit-game/blob/master/server.py +[7]: https://github.com/7WebPages/snakepit-game/blob/master/game.py +[8]: https://github.com/7WebPages/snakepit-game/blob/master/player.py +[9]: https://github.com/7WebPages/snakepit-game/blob/master/datatypes.py +[10]: https://github.com/7WebPages/snakepit-game/blob/master/settings.py +[11]: https://github.com/7WebPages/snakepit-game/blob/master/index.html +[12]: https://twitter.com/7WebPages +[13]: https://www.facebook.com/7WebPages/ From b42ec0a8d9ec90bd6dfaf148b64dcc179bc6ae9d Mon Sep 17 00:00:00 2001 From: wenchunyang Date: Tue, 13 Sep 2016 13:24:20 +0800 Subject: [PATCH 0120/2437] delete sources of part2 & part3 --- ...r game with python and asyncio - part 2.md | 234 ------------------ ...r game with python and asyncio - part 3.md | 138 ----------- 2 files changed, 372 deletions(-) delete mode 100644 sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md delete mode 100644 sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md diff --git a/sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md b/sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md deleted file mode 100644 index 75df13d9b5..0000000000 --- a/sources/tech/20160531 Writing online multiplayer game with python and asyncio - part 2.md +++ /dev/null @@ -1,234 +0,0 @@ -chunyang-wen translating -Writing online multiplayer game with python and asyncio - Part 2 -================================================================== - -![](https://7webpages.com/media/cache/fd/d1/fdd1f8f8bbbf4166de5f715e6ed0ac00.gif) - -Have you ever made an asynchronous Python app? Here I’ll tell you how to do it and in the next part, show it on a [working example][1] - a popular Snake game, designed for multiple players. - -see the intro and theory about how to [Get Asynchronous [part 1]][2] - -[Play the game][3] - -### 3. Writing game loop - -The game loop is a heart of every game. It runs continuously to get player's input, update state of the game and render the result on the screen. In online games the loop is divided into client and server parts, so basically there are two loops which communicate over the network. Usually, client role is to get player's input, such as keypress or mouse movement, pass this data to a server and get back the data to render. The server side is processing all the data coming from players, updating game's state, doing necessary calculations to render next frame and passes back the result, such as new placement of game objects. It is very important not to mix client and server roles without a solid reason. If you start doing game logic calculations on the client side, you can easily go out of sync with other clients, and your game can also be created by simply passing any data from the client side. - -A game loop iteration is often called a tick. Tick is an event meaning that current game loop iteration is over and the data for the next frame(s) is ready. -In the next examples we will use the same client, which connects to a server from a web page using WebSocket. It runs a simple loop which passes pressed keys' codes to the server and displays all messages that come from the server. [Client source code is located here][4]. - -#### Example 3.1: Basic game loop - -[Example 3.1 source code][5] - -We will use [aiohttp][6] library to create a game server. It allows creating web servers and clients based on asyncio. A good thing about this library is that it supports normal http requests and websockets at the same time. So we don't need other web servers to render game's html page. - -Here is how we run the server: - -``` -app = web.Application() -app["sockets"] = [] - -asyncio.ensure_future(game_loop(app)) - -app.router.add_route('GET', '/connect', wshandler) -app.router.add_route('GET', '/', handle) - -web.run_app(app) -``` - -web.run_app is a handy shortcut to create server's main task and to run asyncio event loop with it's run_forever() method. I suggest you check the source code of this method to see how the server is actually created and terminated. - -An app is a dict-like object which can be used to share data between connected clients. We will use it to store a list of connected sockets. This list is then used to send notification messages to all connected clients. A call to asyncio.ensure_future() will schedule our main game_loop task which sends 'tick' message to clients every 2 seconds. This task will run concurrently in the same asyncio event loop along with our web server. - -There are 2 web request handlers: handle just serves a html page and wshandler is our main websocket server's task which handles interaction with game clients. With every connected client a new wshandler task is launched in the event loop. This task adds client's socket to the list, so that game_loop task may send messages to all the clients. Then it echoes every keypress back to the client with a message. - -In the launched tasks we are running worker loops over the main event loop of asyncio. A switch between tasks happens when one of them uses await statement to wait for a coroutine to finish. For instance, asyncio.sleep just passes execution back to a scheduler for a given amount of time, and ws.receive() is waiting for a message from websocket, while the scheduler may switch to some other task. - -After you open the main page in a browser and connect to the server, just try to press some keys. Their codes will be echoed back from the server and every 2 seconds this message will be overwritten by game loop's 'tick' message which is sent to all clients. - -So we have just created a server which is processing client's keypresses, while the main game loop is doing some work in the background and updates all clients periodically. - -#### Example 3.2: Starting game loop by request - -[Example 3.2 source code][7] - -In the previous example a game loop was running continuously all the time during the life of the server. But in practice, there is usually no sense to run game loop when no one is connected. Also, there may be different game "rooms" running on one server. In this concept one player "creates" a game session (a match in a multiplayer game or a raid in MMO for example) so other players may join it. Then a game loop runs while the game session continues. - -In this example we use a global flag to check if a game loop is running, and we start it when the first player connects. In the beginning, a game loop is not running, so the flag is set to False. A game loop is launched from the client's handler: - -``` - if app["game_is_running"] == False: - asyncio.ensure_future(game_loop(app)) -``` - -This flag is then set to True at the start of game loop() and then back to False in the end, when all clients are disconnected. - -#### Example 3.3: Managing tasks - -[Example 3.3 source code][8] - -This example illustrates working with task objects. Instead of storing a flag, we store game loop's task directly in our application's global dict. This may be not an optimal thing to do in a simple case like this, but sometimes you may need to control already launched tasks. -``` - if app["game_loop"] is None or \ - app["game_loop"].cancelled(): - app["game_loop"] = asyncio.ensure_future(game_loop(app)) -``` - -Here ensure_future() returns a task object that we store in a global dict; and when all users disconnect, we cancel it with - -``` - app["game_loop"].cancel() -``` - -This cancel() call tells scheduler not to pass execution to this coroutine anymore and sets its state to cancelled which then can be checked by cancelled() method. And here is one caveat worth to mention: when you have external references to a task object and exception happens in this task, this exception will not be raised. Instead, an exception is set to this task and may be checked by exception() method. Such silent fails are not useful when debugging a code. Thus, you may want to raise all exceptions instead. To do so you need to call result() method of unfinished task explicitly. This can be done in a callback: - -``` - app["game_loop"].add_done_callback(lambda t: t.result()) -``` - -Also if we are going to cancel this task in our code and we don't want to have CancelledError exception, it has a point checking its "cancelled" state: -``` - app["game_loop"].add_done_callback(lambda t: t.result() - if not t.cancelled() else None) -``` - -Note that this is required only if you store a reference to your task objects. In the previous examples all exceptions are raised directly without additional callbacks. - -#### Example 3.4: Waiting for multiple events - -[Example 3.4 source code][9] - -In many cases, you need to wait for multiple events inside client's handler. Beside a message from a client, you may need to wait for different types of things to happen. For instance, if your game's time is limited, you may wait for a signal from timer. Or, you may wait for a message from other process using pipes. Or, for a message from a different server in the network, using a distributed messaging system. - -This example is based on example 3.1 for simplicity. But in this case we use Condition object to synchronize game loop with connected clients. We do not keep a global list of sockets here as we are using sockets only within the handler. When game loop iteration ends, we notify all clients using Condition.notify_all() method. This method allows implementing publish/subscribe pattern within asyncio event loop. - -To wait for two events in the handler, first, we wrap awaitable objects in a task using ensure_future() - -``` - if not recv_task: - recv_task = asyncio.ensure_future(ws.receive()) - if not tick_task: - await tick.acquire() - tick_task = asyncio.ensure_future(tick.wait()) -``` - -Before we can call Condition.wait(), we need to acquire a lock behind it. That is why, we call tick.acquire() first. This lock is then released after calling tick.wait(), so other coroutines may use it too. But when we get a notification, a lock will be acquired again, so we need to release it calling tick.release() after received notification. - -We are using asyncio.wait() coroutine to wait for two tasks. - -``` - done, pending = await asyncio.wait( - [recv_task, - tick_task], - return_when=asyncio.FIRST_COMPLETED) -``` - -It blocks until either of tasks from the list is completed. Then it returns 2 lists: tasks which are done and tasks which are still running. If the task is done, we set it to None so it may be created again on the next iteration. - -#### Example 3.5: Combining with threads - -[Example 3.5 source code][10] - -In this example we combine asyncio loop with threads by running the main game loop in a separate thread. As I mentioned before, it's not possible to perform real parallel execution of python code with threads because of GIL. So it is not a good idea to use other thread to do heavy calculations. However, there is one reason to use threads with asyncio: this is the case when you need to use other libraries which do not support asyncio. Using these libraries in the main thread will simply block execution of the loop, so the only way to use them asynchronously is to run in a different thread. - -We run game loop using run_in_executor() method of asyncio loop and ThreadPoolExecutor. Note that game_loop() is not a coroutine anymore. It is a function that is executed in another thread. However, we need to interact with the main thread to notify clients on the game events. And while asyncio itself is not threadsafe, it has methods which allow running your code from another thread. These are call_soon_threadsafe() for normal functions and run_coroutine_threadsafe() for coroutines. We will put a code which notifies clients about game's tick to notify() coroutine and runs it in the main event loop from another thread. - -``` -def game_loop(asyncio_loop): - print("Game loop thread id {}".format(threading.get_ident())) - async def notify(): - print("Notify thread id {}".format(threading.get_ident())) - await tick.acquire() - tick.notify_all() - tick.release() - - while 1: - task = asyncio.run_coroutine_threadsafe(notify(), asyncio_loop) - # blocking the thread - sleep(1) - # make sure the task has finished - task.result() -``` - -When you launch this example, you will see that "Notify thread id" is equal to "Main thread id", this is because notify() coroutine is executed in the main thread. While sleep(1) call is executed in another thread, and, as a result, it will not block the main event loop. - -#### Example 3.6: Multiple processes and scaling up - -[Example 3.6 source code][11] - -One threaded server may work well, but it is limited to one CPU core. To scale the server beyond one core, we need to run multiple processes containing their own event loops. So we need a way for processes to interact with each other by exchanging messages or sharing game's data. Also in games, it is often required to perform heavy calculations, such as path finding and alike. These tasks are sometimes not possible to complete quickly within one game tick. It is not recommended to perform time-consuming calculations in coroutines, as it will block event processing, so in this case, it may be reasonable to pass the heavy task to other process running in parallel. - -The easiest way to utilize multiple cores is to launch multiple single core servers, like in the previous examples, each on a different port. You can do this with supervisord or similar process-controller system. Then, you may use a load balancer, such as HAProxy, to distribute connecting clients between the processes. There are different ways for processes to interact wich each other. One is to use network-based systems, which allows you to scale to multiple servers as well. There are already existing adapters to use popular messaging and storage systems with asyncio. Here are some examples: - -- [aiomcache][12] for memcached client -- [aiozmq][13] for zeroMQ -- [aioredis][14] for Redis storage and pub/sub - -You can find many other packages like this on github and pypi, most of them have "aio" prefix. - -Using network services may be effective to store persistent data and exchange some kind of messages. But its performance may be not enough if you need to perform real-time data processing that involves inter-process communications. In this case, a more appropriate way may be using standard unix pipes. asyncio has support for pipes and there is a [very low-level example of the server which uses pipes][15] in aiohttp repository. - -In the current example, we will use python's high-level [multiprocessing][16] library to instantiate new process to perform heavy calculations on a different core and to exchange messages with this process using multiprocessing.Queue. Unfortunately, the current implementation of multiprocessing is not compatible with asyncio. So every blocking call will block the event loop. But this is exactly the case where threads will be helpful because if we run multiprocessing code in a different thread, it will not block our main thread. All we need is to put all inter-process communications to another thread. This example illustrates this technique. It is very similar to multi-threading example above, but we create a new process from a thread. - -``` -def game_loop(asyncio_loop): - # coroutine to run in main thread - async def notify(): - await tick.acquire() - tick.notify_all() - tick.release() - - queue = Queue() - - # function to run in a different process - def worker(): - while 1: - print("doing heavy calculation in process {}".format(os.getpid())) - sleep(1) - queue.put("calculation result") - - Process(target=worker).start() - - while 1: - # blocks this thread but not main thread with event loop - result = queue.get() - print("getting {} in process {}".format(result, os.getpid())) - task = asyncio.run_coroutine_threadsafe(notify(), asyncio_loop) - task.result() -``` - -Here we run worker() function in another process. It contains a loop doing heavy calculations and putting results to the queue, which is an instance of multiprocessing.Queue. Then we get the results and notify clients in the main event loop from a different thread, exactly as in the example 3.5. This example is very simplified, it doesn't have a proper termination of the process. Also, in a real game, we would probably use the second queue to pass data to the worker. - -There is a project called [aioprocessing][17], which is a wrapper around multiprocessing that makes it compatible with asyncio. However, it uses exactly the same approach as described in this example - creating processes from threads. It will not give you any advantage, other than hiding these tricks behind a simple interface. Hopefully, in the next versions of Python, we will get a multiprocessing library based on coroutines and supports asyncio. - ->Important! If you are going to run another asyncio event loop in a different thread or sub-process created from main thread/process, you need to create a loop explicitly, using asyncio.new_event_loop(), otherwise, it will not work. - --------------------------------------------------------------------------------- - -via: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-writing-game-loop/ - -作者:[Kyrylo Subbotin][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-writing-game-loop/ -[1]: http://snakepit-game.com/ -[2]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ -[3]: http://snakepit-game.com/ -[4]: https://github.com/7WebPages/snakepit-game/blob/master/simple/index.html -[5]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_basic.py -[6]: http://aiohttp.readthedocs.org/ -[7]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_handler.py -[8]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_global.py -[9]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_wait.py -[10]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_thread.py -[11]: https://github.com/7WebPages/snakepit-game/blob/master/simple/game_loop_process.py -[12]: https://github.com/aio-libs/aiomcache -[13]: https://github.com/aio-libs/aiozmq -[14]: https://github.com/aio-libs/aioredis -[15]: https://github.com/KeepSafe/aiohttp/blob/master/examples/mpsrv.py -[16]: https://docs.python.org/3.5/library/multiprocessing.html -[17]: https://github.com/dano/aioprocessing diff --git a/sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md b/sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md deleted file mode 100644 index 3c38af72ac..0000000000 --- a/sources/tech/20160606 Writing online multiplayer game with python and asyncio - part 3.md +++ /dev/null @@ -1,138 +0,0 @@ -chunyang-wen translating -Writing online multiplayer game with python and asyncio - Part 3 -================================================================= - -![](https://7webpages.com/media/cache/17/81/178135a6db5074c72a1394d31774c658.gif) - -In this series, we are making an asynchronous Python app on the example of a multiplayer [Snake game][1]. The previous article focused on [Writing Game Loop][2] and Part 1 was covering how to [Get Asynchronous][3]. - -You can find the code [here][4]. - -### 4. Making a complete game - -![](https://7webpages.com/static/img/14chs7.gif) - -#### 4.1 Project's overview - -In this part, we will review a design of a complete online game. It is a classic snake game with added multiplayer. You can try it yourself at (). A source code is located [in github repository][5]. The game consists of the following files: - -- [server.py][6] - a server handling main game loop and connections. -- [game.py][7] - a main Game class, which implements game's logic and most of the game's network protocol. -- [player.py][8] - Player class, containing individual player's data and snake's representation. This one is responsible for getting player's input and moving the snake accordingly. -- [datatypes.py][9] - basic data structures. -- [settings.py][10] - game settings, and it has descriptions in commentaries. -- [index.html][11] - all html and javascript client part in one file. - -#### 4.2 Inside a game loop - -Multiplayer snake game is a good example to learn because of its simplicity. All snakes move to one position every single frame, and frames are changing at a very slow rate, allowing you to watch how game engine is working actually. There is no instant reaction to player's keypresses because of the slow speed. A pressed key is remembered and then taken into account while calculating the next frame at the end of game loop's iteration. - -> Modern action games are running at much higher frame rates and often frame rates of server and client are not equal. Client frame rate usually depends on the client hardware performance, while server frame rate is fixed. A client may render several frames after getting the data corresponding to one "game tick". This allows to create smooth animations, which are only limited by client's performance. In this case, a server should pass not only current positions of the objects but also their moving directions, speeds and velocities. And while client frame rate is called FPS (frames per second), sever frame rate is called TPS (ticks per second). In this snake game example both values are equal, and one frame displayed by a client is calculated within one server's tick. - -We will use textmode-like play field, which is, in fact, a html table with one-char cells. All objects of the game are displayed with characters of different colors placed in table's cells. Most of the time client passes pressed keys' codes to the server and gets back play field updates with every "tick". An update from server consists of messages representing characters to render along with their coordinates and colors. So we are keeping all game logic on the server and we are sending to client only rendering data. In addition, we minimize the possibilities to hack the game by substituting its information sent over the network. - -#### 4.3 How does it work? - -The server in this game is close to Example 3.2 for simplicity. But instead of having a global list of connected websockets, we have one server-wide Game object. A Game instance contains a list of Player objects (inside self._players attribute) which represents players connected to this game, their personal data and websocket objects. Having all game-related data in a Game object also allows us to have multiple game rooms if we want to add such feature. In this case, we need to maintain multiple Game objects, one per game started. - -All interactions between server and clients are done with messages encoded in json. Message from the client containing only a number is interpreted as a code of the key pressed by the player. Other messages from client are sent in the following format: - -``` -[command, arg1, arg2, ... argN ] -``` - -Messages from server are sent as a list because there is often a bunch of messages to send at once (rendering data mostly): - -``` -[[command, arg1, arg2, ... argN ], ... ] -``` - -At the end of every game loop iteration, the next frame is calculated and sent to all the clients. Of course, we are not sending complete frame every time, but only a list of changes for the next frame. - -Note that players are not joining the game immediately after connecting to the server. The connection starts in "spectator" mode, so one can watch how others are playing. if the game is already started, or a "game over" screen from the previous game session. Then a player may press "Join" button to join the existing game or to create a new game if the game is not currently running (no other active players). In the later case, the play field is cleared before the start. - -The play field is stored in Game._world attribute, which is a 2d array made of nested lists. It is used to keep game field's state internally. Each element of an array represents a field's cell which is then rendered to a html table cell. It has a type of Char, which is a namedtuple consisting of a character and color. It is important to keep play field in sync with all the connected clients, so all updates to the play field should be made only along with sending corresponding messages to the clients. This is performed by Game.apply_render() method. It receives a list of Draw objects, which is then used to update play field internally and also to send render message to clients. - -We are using namedtuple not only because it is a good way to represent simple data structures, but also because it takes less space comparing to dict when sending in a json message. If you are sending complex data structures in a real game app, it is recommended to serialize them into a plain and shorter format or even pack in a binary format (such as bson instead of json) to minimize network traffic. - -ThePlayer object contains snake's representation in a deque object. This data type is similar to a list but is more effective for adding and removing elements on its sides, so it is ideal to represent a moving snake. The main method of the class is Player.render_move(), it returns rendering data to move player's snake to the next position. Basically, it renders snake's head in the new position and removes the last element where the tail was in the previous frame. In case the snake has eaten a digit and has to grow, a tail is not moving for a corresponding number of frames. The snake rendering data is used in Game.next_frame() method of the main class, which implements all game logic. This method renders all snake moves and checks for obstacles in front of every snake and also spawns digits and "stones". It is called directly from game_loop() to generate the next frame at every "tick". - -In case there is an obstacle in front of snake's head, a Game.game_over() method is called from Game.next_frame(). It notifies all connected clients about the dead snake (which is turned into stones by player.render_game_over() method) and updates top scores table. Player object's alive flag is set to False, so this player will be skipped when rendering the next frames, until joining the game once again. In case there are no more snakes alive, a "game over" message is rendered at the game field. Also, the main game loop will stop and set game.running flag to False, which will cause a game field to be cleared when some player will press "Join" button next time. - -Spawning of digits and stones is also happening while rendering every next frame, and it is determined by random values. A chance to spawn a digit or a stone can be changed in settings.py along with some other values. Note that digit spawning is happening for every live snake in the play field, so the more snakes are there, the more digits will appear, and they all will have enough food to consume. - -#### 4.4 Network protocol -List of messages sent from client - -Command | Parameters |Description -:-- |:-- |:-- -new_player | [name] |Setting player's nickname -join | |Player is joining the game - - -List of messages sent from server - -Command | Parameters |Description -:-- |:-- |:-- -handshake |[id] |Assign id to a player -world |[[(char, color), ...], ...] |Initial play field (world) map -reset_world | |Clean up world map, replacing all characters with spaces -render |[x, y, char, color] |Display character at position -p_joined |[id, name, color, score] |New player joined the game -p_gameover |[id] |Game ended for a player -p_score |[id, score] |Setting score for a player -top_scores |[[name, score, color], ...] |Update top scores table - -Typical messages exchange order - -Client -> Server |Server -> Client |Server -> All clients |Commentaries -:-- |:-- |:-- |:-- -new_player | | |Name passed to server - |handshake | |ID assigned - |world | |Initial world map passed - |top_scores | |Recent top scores table passed -join | | |Player pressed "Join", game loop started - | |reset_world |Command clients to clean up play field - | |render, render, ... |First game tick, first frame rendered -(key code) | | |Player pressed a key - | |render, render, ... |Second frame rendered - | |p_score |Snake has eaten a digit - | |render, render, ... |Third frame rendered - | | |... Repeat for a number of frames ... - | |p_gameover |Snake died when trying to eat an obstacle - | |top_scores |Updated top scores table (if updated) - -### 5. Conclusion - -To tell the truth, I really enjoy using the latest asynchronous capabilities of Python. The new syntax really makes a difference, so async code is now easily readable. It is obvious which calls are non-blocking and when the green thread switching is happening. So now I can claim with confidence that Python is a good tool for asynchronous programming. - -SnakePit has become very popular at 7WebPages team, and if you decide to take a break at your company, please, don’t forget to leave a feedback for us, say, on [Twitter][12] or [Facebook][13] . - -Get to know more from: - - - --------------------------------------------------------------------------------- - -via: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-part-3/ - -作者:[Saheetha Shameer][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-part-3/ -[1]: http://snakepit-game.com/ -[2]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-and-asyncio-writing-game-loop/ -[3]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ -[4]: https://github.com/7WebPages/snakepit-game -[5]: https://github.com/7WebPages/snakepit-game -[6]: https://github.com/7WebPages/snakepit-game/blob/master/server.py -[7]: https://github.com/7WebPages/snakepit-game/blob/master/game.py -[8]: https://github.com/7WebPages/snakepit-game/blob/master/player.py -[9]: https://github.com/7WebPages/snakepit-game/blob/master/datatypes.py -[10]: https://github.com/7WebPages/snakepit-game/blob/master/settings.py -[11]: https://github.com/7WebPages/snakepit-game/blob/master/index.html -[12]: https://twitter.com/7WebPages -[13]: https://www.facebook.com/7WebPages/ From d57f94c3c762a07383185d21a6ce6e9df6c8cc57 Mon Sep 17 00:00:00 2001 From: bazz2 Date: Tue, 13 Sep 2016 13:34:35 +0800 Subject: [PATCH 0121/2437] jiajia9linuxer translating --- ...OTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md b/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md index 0e73bc9daa..f6f6c838a2 100644 --- a/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md +++ b/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md @@ -1,3 +1,4 @@ +jiajia9linuxer QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD =============== From 248e9b378f1990846573b382653d9f4bb3b028a1 Mon Sep 17 00:00:00 2001 From: wxy Date: Wed, 14 Sep 2016 06:37:16 +0800 Subject: [PATCH 0122/2437] PUB:20160524 Writing online multiplayer game with python and asyncio - part 1 @xinglianfly --- ...r game with python and asyncio - part 1.md | 74 +++++++++++++++++++ ...r game with python and asyncio - part 1.md | 71 ------------------ 2 files changed, 74 insertions(+), 71 deletions(-) create mode 100644 published/20160524 Writing online multiplayer game with python and asyncio - part 1.md delete mode 100644 translated/tech/20160524 Writing online multiplayer game with python and asyncio - part 1.md diff --git a/published/20160524 Writing online multiplayer game with python and asyncio - part 1.md b/published/20160524 Writing online multiplayer game with python and asyncio - part 1.md new file mode 100644 index 0000000000..4bc91af85f --- /dev/null +++ b/published/20160524 Writing online multiplayer game with python and asyncio - part 1.md @@ -0,0 +1,74 @@ +使用 Python 和 Asyncio 编写在线多人游戏(一) +=================================================================== + +你在 Python 中用过异步编程吗?本文中我会告诉你怎样做,而且用一个[能工作的例子][1]来展示它:这是一个流行的贪吃蛇游戏,而且是为多人游戏而设计的。 + +- [游戏入口在此,点此体验][2]。 + +###1、简介 + +在技术和文化领域,大规模多人在线游戏(MMO)毋庸置疑是我们当今世界的潮流之一。很长时间以来,为一个 MMO 游戏写一个服务器这件事总是会涉及到大量的预算与复杂的底层编程技术,不过在最近这几年,事情迅速发生了变化。基于动态语言的现代框架允许在中档的硬件上面处理大量并发的用户连接。同时,HTML5 和 WebSockets 标准使得创建基于实时图形的游戏的直接运行至浏览器上的客户端成为可能,而不需要任何的扩展。 + +对于创建可扩展的非堵塞性的服务器来说,Python 可能不是最受欢迎的工具,尤其是和在这个领域里最受欢迎的 Node.js 相比而言。但是最近版本的 Python 正在改变这种现状。[asyncio][3] 的引入和一个特别的 [async/await][4] 语法使得异步代码看起来像常规的阻塞代码一样,这使得 Python 成为了一个值得信赖的异步编程语言,所以我将尝试利用这些新特点来创建一个多人在线游戏。 + +###2、异步 + +一个游戏服务器应该可以接受尽可能多的用户并发连接,并实时处理这些连接。一个典型的解决方案是创建线程,然而在这种情况下并不能解决这个问题。运行上千的线程需要 CPU 在它们之间不停的切换(这叫做上下文切换),这将导致开销非常大,效率很低下。更糟糕的是使用进程来实现,因为,不但如此,它们还会占用大量的内存。在 Python 中,甚至还有一个问题,Python 的解释器(CPython)并不是针对多线程设计的,相反它主要针对于单线程应用实现最大的性能。这就是为什么它使用 GIL(global interpreter lock),这是一个不允许同时运行多线程 Python 代码的架构,以防止同一个共享对象出现使用不可控。正常情况下,在当前线程正在等待的时候,解释器会转换到另一个线程,通常是等待一个 I/O 的响应(举例说,比如等待 Web 服务器的响应)。这就允许在你的应用中实现非阻塞 I/O 操作,因为每一个操作仅仅阻塞一个线程而不是阻塞整个服务器。然而,这也使得通常的多线程方案变得几近无用,因为它不允许你并发执行 Python 代码,即使是在多核心的 CPU 上也是这样。而与此同时,在一个单一线程中拥有非阻塞 I/O 是完全有可能的,因而消除了经常切换上下文的需要。 + +实际上,你可以用纯 Python 代码来实现一个单线程的非阻塞 I/O。你所需要的只是标准的 [select][5] 模块,这个模块可以让你写一个事件循环来等待未阻塞的 socket 的 I/O。然而,这个方法需要你在一个地方定义所有 app 的逻辑,用不了多久,你的 app 就会变成非常复杂的状态机。有一些框架可以简化这个任务,比较流行的是 [tornade][6] 和 [twisted][7]。它们被用来使用回调方法实现复杂的协议(这和 Node.js 比较相似)。这种框架运行在它自己的事件循环中,按照定义的事件调用你的回调函数。并且,这或许是一些情况的解决方案,但是它仍然需要使用回调的方式编程,这使你的代码变得碎片化。与写同步代码并且并发地执行多个副本相比,这就像我们在普通的线程上做的一样。在单个线程上这为什么是不可能的呢? + +这就是为什么出现微线程(microthread)概念的原因。这个想法是为了在一个线程上并发执行任务。当你在一个任务中调用阻塞的方法时,有一个叫做“manager” (或者“scheduler”)的东西在执行事件循环。当有一些事件准备处理的时候,一个 manager 会转移执行权给一个任务,并等着它执行完毕。任务将一直执行,直到它遇到一个阻塞调用,然后它就会将执行权返还给 manager。 + +> 微线程也称为轻量级线程(lightweight threads)或绿色线程(green threads)(来自于 Java 中的一个术语)。在伪线程中并发执行的任务叫做 tasklets、greenlets 或者协程(coroutines)。 + + Python 中的微线程最早的实现之一是 [Stackless Python][8]。它之所以这么知名是因为它被用在了一个叫 [EVE online][9] 的非常有名的在线游戏中。这个 MMO 游戏自称说在一个持久的“宇宙”中,有上千个玩家在做不同的活动,这些都是实时发生的。Stackless 是一个独立的 Python 解释器,它代替了标准的函数栈调用,并且直接控制程序运行流程来减少上下文切换的开销。尽管这非常有效,这个解决方案不如在标准解释器中使用“软”库更流行,像 [eventlet][10] 和 [gevent][11] 的软件包配备了修补过的标准 I/O 库,I/O 函数会将执行权传递到内部事件循环。这使得将正常的阻塞代码转变成非阻塞的代码变得简单。这种方法的一个缺点是从代码上看这并不分明,它的调用是非阻塞的。新版本的 Python 引入了本地协程作为生成器的高级形式。在 Python 的 3.4 版本之后,引入了 asyncio 库,这个库依赖于本地协程来提供单线程并发。但是仅仅到了 Python 3.5 ,协程就变成了 Python 语言的一部分,使用新的关键字 async 和 await 来描述。这是一个简单的例子,演示了使用 asyncio 来运行并发任务。 + +``` +import asyncio + +async def my_task(seconds): + print("start sleeping for {} seconds".format(seconds)) + await asyncio.sleep(seconds) + print("end sleeping for {} seconds".format(seconds)) + +all_tasks = asyncio.gather(my_task(1), my_task(2)) +loop = asyncio.get_event_loop() +loop.run_until_complete(all_tasks) +loop.close() +``` + +我们启动了两个任务,一个睡眠 1 秒钟,另一个睡眠 2 秒钟,输出如下: + +``` +start sleeping for 1 seconds +start sleeping for 2 seconds +end sleeping for 1 seconds +end sleeping for 2 seconds +``` + +正如你所看到的,协程不会阻塞彼此——第二个任务在第一个结束之前启动。这发生的原因是 asyncio.sleep 是协程,它会返回执行权给调度器,直到时间到了。 + +在下一节中,我们将会使用基于协程的任务来创建一个游戏循环。 + +-------------------------------------------------------------------------------- + +via: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ + +作者:[Kyrylo Subbotin][a] +译者:[xinglianfly](https://github.com/xinglianfly) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ +[1]: http://snakepit-game.com/ +[2]: http://snakepit-game.com/ +[3]: https://docs.python.org/3/library/asyncio.html +[4]: https://docs.python.org/3/whatsnew/3.5.html#whatsnew-pep-492 +[5]: https://docs.python.org/2/library/select.html +[6]: http://www.tornadoweb.org/ +[7]: http://twistedmatrix.com/ +[8]: http://www.stackless.com/ +[9]: http://www.eveonline.com/ +[10]: http://eventlet.net/ +[11]: http://www.gevent.org/ diff --git a/translated/tech/20160524 Writing online multiplayer game with python and asyncio - part 1.md b/translated/tech/20160524 Writing online multiplayer game with python and asyncio - part 1.md deleted file mode 100644 index 26ddea6b1e..0000000000 --- a/translated/tech/20160524 Writing online multiplayer game with python and asyncio - part 1.md +++ /dev/null @@ -1,71 +0,0 @@ -使用python 和asyncio编写在线多人游戏 - 第1部分 -=================================================================== - -你曾经把async和python关联起来过吗?在这里我将告诉你怎样做,而且在[working example][1]这个例子里面展示-一个流行的贪吃蛇游戏,这是为多人游戏而设计的。 -[Play game][2] - -###1.简介 - -在技术和文化领域,大量的多人在线游戏毋庸置疑是我们这个世界的主流之一。同时,为一个MMO游戏写一个服务器一般和大量的预算与低水平的编程技术相关,在最近这几年,事情发生了很大的变化。基于动态语言的现代框架允许在稳健的硬件上面处理大量并发的用户连接。同时,HTML5 和 WebSockets 标准允许基于实时的图形游戏直接在web浏览器上创建客户端,而不需要任何的扩展。 - -对于创建可扩展非堵塞的服务器,Python可能不是最受欢迎的工具,尤其是和在这个领域最受欢迎的node.js相比。但是最近版本的python打算改变这种现状。[asyncio][3]的介绍和一个特别的[async/await][4] 语法使得异步代码看起来像常规的阻塞代码,这使得python成为一个值得信赖的异步编程语言。所以我将尝试利用这些新特点来创建一个多人在线游戏。 - -###2.异步 -一个游戏服务器应该处理最大数量的用户的并发连接和实时处理这些连接。一个典型的解决方案----创建线程,然而在这种情况下并不能解决这个问题。运行上千的线程需要CPU在它们之间不停的切换(这叫做上下文切换),这将开销非常大,效率很低下。更糟糕的是,因为,此外,它们会占用大量的内存。在python中,还有一个问题,python的解释器(CPython)并不是针对多线程设计的,它主要针对于单线程实现最大数量的行为。这就是为什么它使用GIL(global interpreter lock),一个不允许同时运行多线程python代码的架构,来防止共享物体的不可控用法。正常情况下当当前线程正在等待的时候,解释器转换到另一个线程,通常是一个I/O的响应(像一个服务器的响应一样)。这允许在你的应用中有非阻塞I/O,因为每一个操作仅仅堵塞一个线程而不是堵塞整个服务器。然而,这也使得通常的多线程变得无用,因为它不允许你并发执行python代码,即使是在多核心的cpu上。同时在单线程中拥有非阻塞IO是完全有可能的,因而消除了经常切换上下文的需要。 - -实际上,你可以用纯python代码来实现一个单线程的非阻塞IO。你所需要的只是标准的[select][5]模块,这个模块可以让你写一个事件循环来等待未阻塞的socket的io。然而,这个方法需要你在一个地方定义所有app的逻辑,不久之后,你的app就会变成非常复杂的状态机。有一些框架可以简化这个任务,比较流行的是[tornade][6] 和 [twisted][7]。他们被用来使用回调方法实现复杂的协议(这和node.js比较相似)。这个框架运行在他自己的事件循环中,这个事件在定义的事件上调用你的回调。并且,这或许是一些情况的解决方案,但是它仍然需要使用回调的方式编程,这使你的代码碎片化。和写同步代码并且并发执行多个副本相比,就像我们会在普通的线程上做一样。这为什么在单个线程上是不可能的呢? - -这就是为什么microthread出现的原因。这个想法是为了在一个线程上并发执行任务。当你在一个任务中调用阻塞的方法时,有一个叫做"manager" (或者“scheduler”)的东西在执行事件循环。当有一些事件准备处理的时候,一个manager会让等这个事件的“任务”单元去执行,直到自己停了下来。然后执行完之后就返回那个管理器(manager)。 - ->Microthreads are also called lightweight threads or green threads (a term which came from Java world). Tasks which are running concurrently in pseudo-threads are called tasklets, greenlets or coroutines.(Microthreads 也会被称为lightweight threads 或者 green threads(java中的一个术语)。在伪线程中并发执行的任务叫做tasklets,greenlets或者coroutines). - -microthreads的其中一种实现在python中叫做[Stackless Python][8]。这个被用在了一个叫[EVE online][9]的非常有名的在线游戏中,所以它变得非常有名。这个MMO游戏自称说在一个持久的宇宙中,有上千个玩家在做不同的活动,这些都是实时发生的。Stackless 是一个单独的python解释器,它代替了标准的栈调用并且直接控制流来减少上下文切换的开销。尽管这非常有效,这个解决方案不如使用标准解释器的“soft”库有名。像[eventlet][10]和[gevent][11] 的方式配备了标准的I / O库的补丁的I / O功能在内部事件循环执行。这使得将正常的阻塞代码转变成非阻塞的代码变得简单。这种方法的一个缺点是从代码看这并不明显,这被称为非阻塞。Python的新的版本介绍了本地协同程序作为生成器的高级形式。在Python 的3.4版本中,引入了asyncio库,这个库依赖于本地协同程序来提供单线程并发。但是在Python 3.5 协同程序变成了Python语言的一部分,使用新的关键字 async 和 await 来描述。这是一个简单的例子,这表明了使用asyncio来运行 并发任务。 - -``` -import asyncio - -async def my_task(seconds): - print("start sleeping for {} seconds".format(seconds)) - await asyncio.sleep(seconds) - print("end sleeping for {} seconds".format(seconds)) - -all_tasks = asyncio.gather(my_task(1), my_task(2)) -loop = asyncio.get_event_loop() -loop.run_until_complete(all_tasks) -loop.close() -``` - -我们启动了两个任务,一个睡眠1秒钟,另一个睡眠2秒钟,输出如下: - -``` -start sleeping for 1 seconds -start sleeping for 2 seconds -end sleeping for 1 seconds -end sleeping for 2 seconds -``` - -正如你所看到的,协同程序不会阻塞彼此-----第二个任务在第一个结束之前启动。这发生的原因是asyncio.sleep是协同程序,它会返回一个调度器的执行直到时间过去。在下一节中, -我们将会使用coroutine-based的任务来创建一个游戏循环。 - --------------------------------------------------------------------------------- - -via: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ - -作者:[Kyrylo Subbotin][a] -译者:[xinglianfly](https://github.com/xinglianfly) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://7webpages.com/blog/writing-online-multiplayer-game-with-python-asyncio-getting-asynchronous/ -[1]: http://snakepit-game.com/ -[2]: http://snakepit-game.com/ -[3]: https://docs.python.org/3/library/asyncio.html -[4]: https://docs.python.org/3/whatsnew/3.5.html#whatsnew-pep-492 -[5]: https://docs.python.org/2/library/select.html -[6]: http://www.tornadoweb.org/ -[7]: http://twistedmatrix.com/ -[8]: http://www.stackless.com/ -[9]: http://www.eveonline.com/ -[10]: http://eventlet.net/ -[11]: http://www.gevent.org/ From 222f29581d516c74d949e78ab6a8139b0ffaf8d4 Mon Sep 17 00:00:00 2001 From: jiajia <1534574360@qq.com> Date: Tue, 13 Sep 2016 20:12:54 -0500 Subject: [PATCH 0123/2437] [jiajia9linuxer] translate complete (#4406) --- ...ing With an Ipad Pro and a Raspberry Pi.md | 408 ------------------ ... LIST APP THAT INTEGRATES WITH OWNCLOUD.md | 81 ---- ...0626 旅行时通过树莓派和iPad Pro备份图片.md | 405 +++++++++++++++++ ...录笔记和待办事项的APP并且集成OwnCloud云服务.md | 80 ++++ 4 files changed, 485 insertions(+), 489 deletions(-) delete mode 100644 sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md delete mode 100644 sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md create mode 100755 translated/tech/20160626 旅行时通过树莓派和iPad Pro备份图片.md create mode 100755 translated/tech/20160905 QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务.md diff --git a/sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md b/sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md deleted file mode 100644 index 4a5c2f1267..0000000000 --- a/sources/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md +++ /dev/null @@ -1,408 +0,0 @@ -[jiajia9linuxer translating] -Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi -=================================================================== - -![](http://www.movingelectrons.net/images/bkup_photos_main.jpg) ->Backup Photos While Traveling - Gear. - -### Introduction - -I’ve been on a quest to finding the ideal travel photo backup solution for a long time. Relying on just tossing your SD cards in your camera bag after they are full during a trip is a risky move that leaves you too exposed: SD cards can be lost or stolen, data can get corrupted or cards can get damaged in transit. Backing up to another medium - even if it’s just another SD card - and leaving that in a safe(r) place while traveling is the best practice. Ideally, backing up to a remote location would be the way to go, but that may not be practical depending on where you are traveling to and Internet availability in the region. - -My requirements for the ideal backup procedure are: - -1. Use an iPad to manage the process instead of a laptop. I like to travel light and since most of my trips are business related (i.e. non-photography related), I’d hate to bring my personal laptop along with my business laptop. My iPad, however; is always with me, so using it as a tool just makes sense. -2. Use as few hardware devices as practically possible. -3. Connection between devices should be secure. I’ll be using this setup in hotels and airports, so closed and encrypted connection between devices is ideal. -4. The whole process should be sturdy and reliable. I’ve tried other options using router/combo devices and [it didn’t end up well][1]. - -### The Setup - -I came up with a setup that meets the above criteria and is also flexible enough to expand on it in the future. It involves the use of the following gear: - -1. [iPad Pro 9.7][2] inches. It’s the most powerful, small and lightweight iOS device at the time of writing. Apple pencil is not really needed, but it’s part of my gear as I so some editing on the iPad Pro while on the road. All the heavy lifting will be done by the Raspberry Pi, so any other device capable of connecting through SSH would fit the bill. -2. [Raspberry Pi 3][3] with Raspbian installed. -3. [Micro SD card][4] for Raspberry Pi and a Raspberry Pi [box/case][5]. -5. [128 GB Pen Drive][6]. You can go bigger, but 128 GB is enough for my user case. You can also get a portable external hard drive like [this one][7], but the Raspberry Pi may not provide enough power through its USB port, which means you would have to get a [powered USB hub][8], along with the needed cables, defeating the purpose of having a lightweight and minimalistic setup. -6. [SD card reader][9] -7. [SD Cards][10]. I use several as I don’t wait for one to fill up before using a different one. That allows me to spread photos I take on a single trip amongst several cards. - -The following diagram shows how these devices will be interacting with each other. - -![](http://www.movingelectrons.net/images/bkup_photos_diag.jpg) ->Backup Photos While Traveling - Process Diagram. - -The Raspberry Pi will be configured to act as a secured Hot Spot. It will create its own WPA2-encrypted WiFi network to which the iPad Pro will connect. Although there are many online tutorials to create an Ad Hoc (i.e. computer-to-computer) connection with the Raspberry Pi, which is easier to setup; that connection is not encrypted and it’s relatively easy for other devices near you to connect to it. Therefore, I decided to go with the WiFi option. - -The camera’s SD card will be connected to one of the Raspberry Pi’s USB ports through an SD card reader. Additionally, a high capacity Pen Drive (128 GB in my case) will be permanently inserted in one of the USB ports on the Raspberry Pi. I picked the [Sandisk Ultra Fit][11] because of its tiny size. The main idea is to have the Raspberry Pi backup the photos from the SD Card to the Pen Drive with the help of a Python script. The backup process will be incremental, meaning that only changes (i.e. new photos taken) will be added to the backup folder each time the script runs, making the process really fast. This is a huge advantage if you take a lot of photos or if you shoot in RAW format. The iPad will be used to trigger the Python script and to browse the SD Card and Pen Drive as needed. - -As an added benefit, if the Raspberry Pi is connected to Internet through a wired connection (i.e. through the Ethernet port), it will be able to share the Internet connection with the devices connected to its WiFi network. - -### 1. Raspberry Pi Configuration - -This is the part where we roll up our sleeves and get busy as we’ll be using Raspbian’s command-line interface (CLI) . I’ll try to be as descriptive as possible so it’s easy to go through the process. - -#### Install and Configure Raspbian - -Connect a keyboard, mouse and an LCD monitor to the Raspberry Pi. Insert the Micro SD in the Raspberry Pi’s slot and proceed to install Raspbian per the instructions in the [official site][12]. - -After the installation is done, go to the CLI (Terminal in Raspbian) and type: - -``` -sudo apt-get update -sudo apt-get upgrade -``` - -This will upgrade all software on the machine. I configured the Raspberry Pi to connect to the local network and changed the default password as a safety measure. - -By default SSH is enabled on Raspbian, so all sections below can be done from a remote machine. I also configured RSA authentication, but that’s optional. More info about it [here][13]. - -This is a screenshot of the SSH connection to the Raspberry Pi from [iTerm][14] on Mac: - -##### Creating Encrypted (WPA2) Access Point - -The installation was made based on [this][15] article, it was optimized for my user case. - -##### 1. Install Packages - -We need to type the following to install the required packages: - -``` -sudo apt-get install hostapd -sudo apt-get install dnsmasq -``` - -hostapd allows to use the built-in WiFi as an access point. dnsmasq is a combined DHCP and DNS server that’s easy to configure. - -##### 2. Edit dhcpcd.conf - -Connect to the Raspberry Pi through Ethernet. Interface configuration on the Raspbery Pi is handled by dhcpcd, so first we tell it to ignore wlan0 as it will be configured with a static IP address. - -Open up the dhcpcd configuration file with sudo nano `/etc/dhcpcd.conf` and add the following line to the bottom of the file: - -``` -denyinterfaces wlan0 -``` - -Note: This must be above any interface lines that may have been added. - -##### 3. Edit interfaces - -Now we need to configure our static IP. To do this, open up the interface configuration file with sudo nano `/etc/network/interfaces` and edit the wlan0 section so that it looks like this: - -``` -allow-hotplug wlan0 -iface wlan0 inet static - address 192.168.1.1 - netmask 255.255.255.0 - network 192.168.1.0 - broadcast 192.168.1.255 -# wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf -``` - -Also, the wlan1 section was edited to be: - -``` -#allow-hotplug wlan1 -#iface wlan1 inet manual -# wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf -``` - -Important: Restart dhcpcd with sudo service dhcpcd restart and then reload the configuration for wlan0 with `sudo ifdown eth0; sudo ifup wlan0`. - -##### 4. Configure Hostapd - -Next, we need to configure hostapd. Create a new configuration file with `sudo nano /etc/hostapd/hostapd.conf with the following contents: - -``` -interface=wlan0 - -# Use the nl80211 driver with the brcmfmac driver -driver=nl80211 - -# This is the name of the network -ssid=YOUR_NETWORK_NAME_HERE - -# Use the 2.4GHz band -hw_mode=g - -# Use channel 6 -channel=6 - -# Enable 802.11n -ieee80211n=1 - -# Enable QoS Support -wmm_enabled=1 - -# Enable 40MHz channels with 20ns guard interval -ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40] - -# Accept all MAC addresses -macaddr_acl=0 - -# Use WPA authentication -auth_algs=1 - -# Require clients to know the network name -ignore_broadcast_ssid=0 - -# Use WPA2 -wpa=2 - -# Use a pre-shared key -wpa_key_mgmt=WPA-PSK - -# The network passphrase -wpa_passphrase=YOUR_NEW_WIFI_PASSWORD_HERE - -# Use AES, instead of TKIP -rsn_pairwise=CCMP -``` - -Now, we also need to tell hostapd where to look for the config file when it starts up on boot. Open up the default configuration file with `sudo nano /etc/default/hostapd` and find the line `#DAEMON_CONF=""` and replace it with `DAEMON_CONF="/etc/hostapd/hostapd.conf"`. - -##### 5. Configure Dnsmasq - -The shipped dnsmasq config file contains tons of information on how to use it, but we won’t be using all the options. I’d recommend moving it (rather than deleting it), and creating a new one with - -``` -sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig -sudo nano /etc/dnsmasq.conf -``` - -Paste the following into the new file: - -``` -interface=wlan0 # Use interface wlan0 -listen-address=192.168.1.1 # Explicitly specify the address to listen on -bind-interfaces # Bind to the interface to make sure we aren't sending things elsewhere -server=8.8.8.8 # Forward DNS requests to Google DNS -domain-needed # Don't forward short names -bogus-priv # Never forward addresses in the non-routed address spaces. -dhcp-range=192.168.1.50,192.168.1.100,12h # Assign IP addresses in that range with a 12 hour lease time -``` - -##### 6. Set up IPV4 forwarding - -One of the last things that we need to do is to enable packet forwarding. To do this, open up the sysctl.conf file with `sudo nano /etc/sysctl.conf`, and remove the # from the beginning of the line containing `net.ipv4.ip_forward=1`. This will enable it on the next reboot. - -We also need to share our Raspberry Pi’s internet connection to our devices connected over WiFi by the configuring a NAT between our wlan0 interface and our eth0 interface. We can do this by writing a script with the following lines. - -``` -sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT -sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT -``` - -I named the script hotspot-boot.sh and made it executable with: - -``` -sudo chmod 755 hotspot-boot.sh -``` - -The script should be executed when the Raspberry Pi boots. There are many ways to accomplish this, and this is the way I went with: - -1. Put the file in `/home/pi/scripts`. -2. Edit the rc.local file by typing `sudo nano /etc/rc.local` and place the call to the script before the line that reads exit 0 (more information [here][16]). - -This is how the rc.local file looks like after editing it. - -``` -#!/bin/sh -e -# -# rc.local -# -# This script is executed at the end of each multiuser runlevel. -# Make sure that the script will "exit 0" on success or any other -# value on error. -# -# In order to enable or disable this script just change the execution -# bits. -# -# By default this script does nothing. - -# Print the IP address -_IP=$(hostname -I) || true -if [ "$_IP" ]; then - printf "My IP address is %s\n" "$_IP" -fi - -sudo /home/pi/scripts/hotspot-boot.sh & - -exit 0 - -``` - -#### Installing Samba and NTFS Compatibility. - -We also need to install the following packages to enable the Samba protocol and allow the File Browser App to see the connected devices to the Raspberry Pi as shared folders. Also, ntfs-3g provides NTFS compatibility in case we decide to connect a portable hard drive to the Raspberry Pi. - -``` -sudo apt-get install ntfs-3g -sudo apt-get install samba samba-common-bin -``` - -You can follow [this][17] article for details on how to configure Samba. - -Important Note: The referenced article also goes through the process of mounting external hard drives on the Raspberry Pi. We won’t be doing that because, at the time of writing, the current version of Raspbian (Jessie) auto-mounts both the SD Card and the Pendrive to `/media/pi/` when the device is turned on. The article also goes over some redundancy features that we won’t be using. - -### 2. Python Script - -Now that the Raspberry Pi has been configured, we need to work on the script that will actually backup/copy our photos. Note that this script just provides certain degree of automation to the backup process. If you have a basic knowledge of using the Linux/Raspbian CLI, you can just SSH into the Raspberry Pi and copy yourself all photos from one device to the other by creating the needed folders and using either the cp or the rsync command. We’ll be using the rsync method on the script as it’s very reliable and allows for incremental backups. - -This process relies on two files: the script itself and the configuration file `backup_photos.conf`. The latter just have a couple of lines indicating where the destination drive (Pendrive) is mounted and what folder has been mounted to. This is what it looks like: - -``` -mount folder=/media/pi/ -destination folder=PDRIVE128GB -``` - -Important: Do not add any additional spaces between the `=` symbol and the words to both sides of it as the script will break (definitely an opportunity for improvement). - -Below is the Python script, which I named `backup_photos.py` and placed in `/home/pi/scripts/`. I included comments in between the lines of code to make it easier to follow. - -``` -#!/usr/bin/python3 - -import os -import sys -from sh import rsync - -''' -The script copies an SD Card mounted on /media/pi/ to a folder with the same name -created in the destination drive. The destination drive's name is defined in -the .conf file. - - -Argument: label/name of the mounted SD Card. -''' - -CONFIG_FILE = '/home/pi/scripts/backup_photos.conf' -ORIGIN_DEV = sys.argv[1] - -def create_folder(path): - - print ('attempting to create destination folder: ',path) - if not os.path.exists(path): - try: - os.mkdir(path) - print ('Folder created.') - except: - print ('Folder could not be created. Stopping.') - return - else: - print ('Folder already in path. Using that instead.') - - - -confFile = open(CONFIG_FILE,'rU') -#IMPORTANT: rU Opens the file with Universal Newline Support, -#so \n and/or \r is recognized as a new line. - -confList = confFile.readlines() -confFile.close() - - -for line in confList: - line = line.strip('\n') - - try: - name , value = line.split('=') - - if name == 'mount folder': - mountFolder = value - elif name == 'destination folder': - destDevice = value - - - except ValueError: - print ('Incorrect line format. Passing.') - pass - - -destFolder = mountFolder+destDevice+'/'+ORIGIN_DEV -create_folder(destFolder) - -print ('Copying files...') - -# Comment out to delete files that are not in the origin: -# rsync("-av", "--delete", mountFolder+ORIGIN_DEV, destFolder) -rsync("-av", mountFolder+ORIGIN_DEV+'/', destFolder) - -print ('Done.') -``` - -### 3. iPad Pro Configuration - -Since all the heavy-lifting will be done on the Raspberry Pi and no files will be transferred through the iPad Pro, which was a huge disadvantage in [one of the workflows I tried before][18]; we just need to install [Prompt 2][19] on the iPad to access the Raspeberry Pi through SSH. Once connected, you can either run the Python script or copy the files manually. - -![](http://www.movingelectrons.net/images/bkup_photos_ipad&rpi_prompt.jpg) ->SSH Connection to Raspberry Pi From iPad Using Prompt. - -Since we installed Samba, we can access USB devices connected to the Raspberry Pi in a more graphical way. You can stream videos, copy and move files between devices. [File Browser][20] is perfect for that. - -### 4. Putting it All Together - -Let’s suppose that `SD32GB-03` is the label of an SD card connected to one of the USB ports on the Raspberry Pi. Also, let’s suppose that `PDRIVE128GB` is the label of the Pendrive, also connected to the device and defined on the `.conf` file as indicated above. If we wanted to backup the photos on the SD Card, we would need to go through the following steps: - -1. Turn on Raspberry Pi so that drives are mounted automatically. -2. Connect to the WiFi network generated by the Raspberry Pi. -3. Connect to the Raspberry Pi through SSH using the [Prompt][21] App. -4. Type the following once you are connected: - -``` -python3 backup_photos.py SD32GB-03 -``` - -The first backup my take some minutes depending on how much of the card is used. That means you need to keep the connection alive to the Raspberry Pi from the iPad. You can get around this by using the [nohup][22] command before running the script. - -``` -nohup python3 backup_photos.py SD32GB-03 & -``` - -![](http://www.movingelectrons.net/images/bkup_photos_ipad&rpi_finished.png) ->iTerm Screenshot After Running Python Script. - -### Further Customization - -I installed a VNC server to access Raspbian’s graphical interface from another computer or the iPad through [Remoter App][23]. I’m looking into installing [BitTorrent Sync][24] for backing up photos to a remote location while on the road, which would be the ideal setup. I’ll expand this post once I have a workable solution. - -Feel free to either include your comments/questions below or reach out to me. My contact info is at the footer of this page. - - --------------------------------------------------------------------------------- - -via: http://www.movingelectrons.net/blog/2016/06/26/backup-photos-while-traveling-with-a-raspberry-pi.html - -作者:[Editor][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.movingelectrons.net/blog/2016/06/26/backup-photos-while-traveling-with-a-raspberry-pi.html -[1]: http://bit.ly/1MVVtZi -[2]: http://www.amazon.com/dp/B01D3NZIMA/?tag=movinelect0e-20 -[3]: http://www.amazon.com/dp/B01CD5VC92/?tag=movinelect0e-20 -[4]: http://www.amazon.com/dp/B010Q57T02/?tag=movinelect0e-20 -[5]: http://www.amazon.com/dp/B01F1PSFY6/?tag=movinelect0e-20 -[6]: http://amzn.to/293kPqX -[7]: http://amzn.to/290syFY -[8]: http://amzn.to/290syFY -[9]: http://amzn.to/290syFY -[10]: http://amzn.to/290syFY -[11]: http://amzn.to/293kPqX -[12]: https://www.raspberrypi.org/downloads/noobs/ -[13]: https://www.raspberrypi.org/documentation/remote-access/ssh/passwordless.md -[14]: https://www.iterm2.com/ -[15]: https://frillip.com/using-your-raspberry-pi-3-as-a-wifi-access-point-with-hostapd/ -[16]: https://www.raspberrypi.org/documentation/linux/usage/rc-local.md -[17]: http://www.howtogeek.com/139433/how-to-turn-a-raspberry-pi-into-a-low-power-network-storage-device/ -[18]: http://bit.ly/1MVVtZi -[19]: https://itunes.apple.com/us/app/prompt-2/id917437289?mt=8&uo=4&at=11lqkH -[20]: https://itunes.apple.com/us/app/filebrowser-access-files-on/id364738545?mt=8&uo=4&at=11lqkH -[21]: https://itunes.apple.com/us/app/prompt-2/id917437289?mt=8&uo=4&at=11lqkH -[22]: https://en.m.wikipedia.org/wiki/Nohup -[23]: https://itunes.apple.com/us/app/remoter-pro-vnc-ssh-rdp/id519768191?mt=8&uo=4&at=11lqkH -[24]: https://getsync.com/ diff --git a/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md b/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md deleted file mode 100644 index f6f6c838a2..0000000000 --- a/sources/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md +++ /dev/null @@ -1,81 +0,0 @@ -jiajia9linuxer -QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD -=============== - -[QOwnNotes][1] is a free, open source note taking and todo list application available for Linux, Windows, and Mac. - -The application saves your notes as plain-text files, and it features Markdown support and tight ownCloud integration. - -![](https://2.bp.blogspot.com/-a2vsrOG0zFk/V81gyHWlaaI/AAAAAAAAYZs/uzY16JtNcT8bnje1rTKJx1135WueY6V9gCLcB/s400/qownnotes.png) - -What makes QOwnNotes stand out is its ownCloud integration (which is optional). Using the ownCloud Notes app, you are able to edit and search notes from the web, or from mobile devices (by using an app like [CloudNotes][2]). - -Furthermore, connecting QOwnNotes with your ownCloud account allows you to share notes and access / restore previous versions (or trashed files) of your notes from the ownCloud server. - -In the same way, QOwnNotes can also integrate with the ownCloud tasks or Tasks Plus apps. - -In case you're not familiar with [ownCloud][3], this is a free software alternative to proprietary web services such as Dropbox, Google Drive, and others, which can be installed on your own server. It comes with a web interface that provides access to file management, calendar, image gallery, music player, document viewer, and much more. The developers also provide desktop sync clients, as well as mobile apps. - -Since the notes are saved as plain text, they can be synchronized across devices using other cloud storage services, like Dropbox, Google Drive, and so on, but this is not done directly from within the application. - -As a result, the features I mentioned above, like restoring previous note versions, are only available with ownCloud (although Dropbox, and others, do provide access to previous file revisions, but you won't be able to access this directly from QOwnNotes). - -As for the QOwnNotes note taking features, the app supports Markdown (with a built-in Markdown preview mode), tagging notes, searching in tags and notes, adding links to notes, and inserting images: - -![](https://4.bp.blogspot.com/-SuBhC43gzkY/V81oV7-zLBI/AAAAAAAAYZ8/l6nLQQSUv34Y7op_Xrma8XYm6EdWrhbIACLcB/s400/qownnotes_2.png) - -Hierarchical note tagging and note subfolders are also supported. - -The todo manager feature is pretty basic and could use some improvements, as it currently opens in a separate window, and it doesn't use the same editor as the notes, not allowing you to insert images, or use Markdown. - -![](https://3.bp.blogspot.com/-AUeyZS3s_ck/V81opialKtI/AAAAAAAAYaA/xukIiZZUdNYBVZ92xgKEsEFew7q961CDwCLcB/s400/qownnotes-tasks.png) - -It does allow you to search your todo items, set item priority, add reminders, and show completed items. Also, todo items can be inserted into notes. - -The application user interface is customizable, allowing you to increase or decrease the font size, toggle panes (Markdown preview, note edit and tag panes), and more. A distraction-free mode is also available: - -![](https://4.bp.blogspot.com/-Pnzw1wZde50/V81rrE6mTWI/AAAAAAAAYaM/0UZnH9ktbAgClkuAk1g6fgXK87kB_Bh0wCLcB/s400/qownnotes-distraction-free.png) - -From the application settings, you can enable the dark mode (this was buggy in my test under Ubuntu 16.04 - some toolbar icons were missing), change the toolbar icon size, fonts, and color scheme (light or dark): - -![](https://1.bp.blogspot.com/-K1MGlXA8sxs/V81rv3fwL6I/AAAAAAAAYaQ/YDhhhnbJ9gY38B6Vz1Na_pHLCjLHhPWiwCLcB/s400/qownnotes-settings.png) - -Other QOwnNotes features include encryption support (notes can only be decrypted in QOwnNotes), customizable keyboard shortcuts, export notes to PDF or Markdown, customizable note saving interval, and more. - -Check out the QOwnNotes [homepage][11] for a complete list of features. - - -### Download QOwnNotes - - -For how to install QownNotes, see its [installation][4] page (packages / repositories available for Debian, Ubuntu, Linux Mint, openSUSE, Fedora, Arch Linux, KaOS, Gentoo, Slakware, CentOS, as well as Mac OSX and Windows). - -A QOwnNotes [snap][5] package is also available (in Ubuntu 16.04 and newer, you should be able to install it directly from Ubuntu Software). - -To integrate QOwnNotes with ownCloud you'll need [ownCloud server][6], as well as [Notes][7], [QOwnNotesAPI][8], and [Tasks][9] or [Tasks Plus][10] ownCloud apps. These can be installed from the ownCloud web interface, without having to download anything manually. - -Note that the QOenNotesAPI and Notes ownCloud apps are listed as experimental, so you'll need to enable experimental apps to be able to find and install them. This can be done from the ownCloud web interface, under Apps, by clicking on the settings icon in the lower left-hand side corner. - - --------------------------------------------------------------------------------- - -via: http://www.webupd8.org/2016/09/qownnotes-is-note-taking-and-todo-list.html - -作者:[Andrew][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.webupd8.org/p/about.html -[1]: http://www.qownnotes.org/ -[2]: http://peterandlinda.com/cloudnotes/ -[3]: https://owncloud.org/ -[11]: http://www.qownnotes.org/ -[4]: http://www.qownnotes.org/installation -[5]: https://uappexplorer.com/app/qownnotes.pbek -[6]: https://download.owncloud.org/download/repositories/stable/owncloud/ -[7]: https://github.com/owncloud/notes -[8]: https://github.com/pbek/qownnotesapi -[9]: https://apps.owncloud.com/content/show.php/Tasks?content=164356 -[10]: https://apps.owncloud.com/content/show.php/Tasks+Plus?content=170561 diff --git a/translated/tech/20160626 旅行时通过树莓派和iPad Pro备份图片.md b/translated/tech/20160626 旅行时通过树莓派和iPad Pro备份图片.md new file mode 100755 index 0000000000..50e67f5e88 --- /dev/null +++ b/translated/tech/20160626 旅行时通过树莓派和iPad Pro备份图片.md @@ -0,0 +1,405 @@ +旅行时通过树莓派和iPad Pro备份图片 +=================================================================== + +![](http://www.movingelectrons.net/images/bkup_photos_main.jpg) +>旅行中备份图片 - Gear. + +### 介绍 + +我在很长的时间内一直在寻找一个旅行中备份图片的理想方法,把SD卡放进你的相机包是比较危险和暴露的,SD卡可能丢失或者被盗,数据可能损坏或者在传输过程中失败。比较好的一个选择是复制到另外一个设备即使它也是个SD卡,并且将它放到一个比较安全的地方去,备份到远端也是一个可行的办法,但是如果去了一个没有网络的地方就不太可行了。 + +我理想的备份步骤需要下面的工具: + +1. 用一台iPad pro而不是一台笔记本。我喜欢简便的旅行,我的旅行大部分是商务的而不是拍摄休闲的,这很显然我为什么选择了iPad Pro +2. 用尽可能少的设备 +3. 设备之间的连接需要很安全。我需要在旅馆和机场使用,所以设备之间的连接需要时封闭的加密的。 +4. 整个过程应该是稳定的安全的,我还用过其他的移动设备,但是效果不太理想[1]. + +### 设置 + +我制定了一个满足上面条件并且在未来可以扩充的设定,它包含下面这些部件的使用: + +1. [2]9.7寸写作时最棒的又小又轻便的IOS系统的iPad Pro,苹果笔不是不许的,但是当我在路上进行一些编辑的时候依然需要,所有的重活由树莓派做 ,其他设备通过ssh连接设备 +2. [3] 树莓派3包含Raspbian系统 +3. [4]Mini SD卡 [box/case][5]. +5. [6]128G的优盘,对于我是够用了,你可以买个更大的,你也可以买个移动硬盘,但是树莓派没办法给移动硬盘供电,你需要额外准备一个供电的hub,当然优质的线缆能提供可靠便捷的安装和连接。 +6. [9]SD读卡器 +7. [10]另外的sd卡,SD卡我在用满之前就会立即换一个,这样就会让我的照片分布在不同的sd卡上 + +下图展示了这些设备之间如何相互连接. + +![](http://www.movingelectrons.net/images/bkup_photos_diag.jpg) +>旅行时照片的备份-过程表格. + +树莓派会作为一个热点. 它会创建一个WIFI网络,当然也可以建立一个Ad Hoc网络,更简单一些,但是它不会加密设备之间的连接,因此我选择创建WIFI网络。 + +SD卡放进SD读卡器插到树莓派USB端口上,128G的大容量优盘一直插在树莓派的USB端口上,我选择了一款闪迪的体积比较小。主要的思路就是通过脚本把SD卡的图片备份到优盘上,脚本是增量备份,而且脚本会自动运行,使备份特别快,如果你有很多的照片或者拍摄了很多没压缩的照片,这个任务量就比较大,用ipad来运行Python脚本,而且用来浏览SD卡和优盘的文件。 + +如果给树莓派连上一根能上网的网线,那样连接树莓派wifi的设备就可以上网啦! + +### 1. 树莓派的设置 + +这部分要用到命令行模式,我会尽可能详细的介绍,方便大家进行下去。 + +#### 安装和配置Raspbian + +给树莓派连接鼠标键盘和显示器,将SD卡插到树莓派上,在官网按步骤安装Raspbian [12]. + +安装完后执行下面的命令: + +``` +sudo apt-get update +sudo apt-get upgrade +``` + +升级机器上所有的软件到最新,我将树莓派连接到本地网络,而且为了安全更改了默认的密码。 + +Raspbian默认开启了SSH,这样所有的设置可以在一个远程的设备上完成。我也设置了RSA验证,那是个可选的功能,查看能多信息 [这里][13]. + +这是一个在MAC上建立SSH连接到树莓派上的截图[14]: + +##### 建立WPA2验证的WIFI + +这个安装过程是基于这篇文章,只适用于我自己做的例子[15]. + +##### 1. 安装软件包 + +我们需要安装下面的软件包: + +``` +sudo apt-get install hostapd +sudo apt-get install dnsmasq +``` + +hostapd用来创建wifi,dnsmasp用来做dhcp和dns服务,很容易设置. + +##### 2. 编辑dhcpcd.conf + +通过网络连接树莓派,网络设置树莓派需要dhcpd,首先我们将wlan0设置为一个静态的IP。 + +用sudo nano `/etc/dhcpcd.conf`命令打开配置文件,在最后一行添加上如下信息: + +``` +denyinterfaces wlan0 +``` + +注意: 必须先配置这个接口才能配置其他接口. + +##### 3. 编辑端口 + +现在设置静态IP,sudo nano `/etc/network/interfaces`打开端口配置文件按照如下信息编辑wlan0选项: + +``` +allow-hotplug wlan0 +iface wlan0 inet static + address 192.168.1.1 + netmask 255.255.255.0 + network 192.168.1.0 + broadcast 192.168.1.255 +# wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf +``` + +同样, 然后添加wlan1信息: + +``` +#allow-hotplug wlan1 +#iface wlan1 inet manual +# wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf +``` + +重要: sudo service dhcpcd restart命令重启dhcpd服务`sudo ifdown eth0; sudo ifup wlan0`命令用来关闭eth0端口再开启用来生效配置文件. + +##### 4. 配置Hostapd + +接下来我们配置hostapd,`sudo nano /etc/hostapd/hostapd.conf` 用这个命令创建并填写配置信息到文件中: + +``` +interface=wlan0 + +# Use the nl80211 driver with the brcmfmac driver +driver=nl80211 + +# This is the name of the network +ssid=YOUR_NETWORK_NAME_HERE + +# Use the 2.4GHz band +hw_mode=g + +# Use channel 6 +channel=6 + +# Enable 802.11n +ieee80211n=1 + +# Enable QoS Support +wmm_enabled=1 + +# Enable 40MHz channels with 20ns guard interval +ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40] + +# Accept all MAC addresses +macaddr_acl=0 + +# Use WPA authentication +auth_algs=1 + +# Require clients to know the network name +ignore_broadcast_ssid=0 + +# Use WPA2 +wpa=2 + +# Use a pre-shared key +wpa_key_mgmt=WPA-PSK + +# The network passphrase +wpa_passphrase=YOUR_NEW_WIFI_PASSWORD_HERE + +# Use AES, instead of TKIP +rsn_pairwise=CCMP +``` + +配置完成后,我们需要运行 `sudo nano /etc/default/hostapd` 命令打开这个配置文件然后找到`#DAEMON_CONF=""` 替换成`DAEMON_CONF="/etc/hostapd/hostapd.conf"`以便hostapd服务能够找到对应的配置文件. + +##### 5. 配置Dnsmasq + +dnsmasp配置文件包含很多信息方便你使用它,但是我们不需要那么多选项,我建议用下面两条命令把它放到别的地方,不要删除它,然后自己创建一个文件 + +``` +sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig +sudo nano /etc/dnsmasq.conf +``` + +粘贴下面的信息到新文件中: + +``` +interface=wlan0 # Use interface wlan0 +listen-address=192.168.1.1 # Explicitly specify the address to listen on +bind-interfaces # Bind to the interface to make sure we aren't sending things elsewhere +server=8.8.8.8 # Forward DNS requests to Google DNS +domain-needed # Don't forward short names +bogus-priv # Never forward addresses in the non-routed address spaces. +dhcp-range=192.168.1.50,192.168.1.100,12h # Assign IP addresses in that range with a 12 hour lease time +``` + +##### 6. 设置IPv4转发 + +最后我们需要做的事就是配置包转发,用`sudo nano /etc/sysctl.conf`命令打开sysctl.conf文件,将containing `net.ipv4.ip_forward=1`之前的#号删除,然后重启生效 + +我们还需要给连接树莓派的设备通过WIFI分享一个网络连接,做一个wlan0和eth0的NAT,我们可以参照下面的脚本来实现。 + +``` +sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE +sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT +sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT +``` + +我命名了一个hotspot-boot.sh的脚本然后让它可以运行: + +``` +sudo chmod 755 hotspot-boot.sh +``` + +脚本会在树莓派启动的时候运行,有很多方法实现,下面是我实现的方式: + +1. 把文件放到`/home/pi/scripts`目录下. +2. 编辑rc.local文件,输入`sudo nano /etc/rc.local`命令将运行脚本命令放到exit0之前[16]). + +下面是实例. + +``` +#!/bin/sh -e +# +# rc.local +# +# This script is executed at the end of each multiuser runlevel. +# Make sure that the script will "exit 0" on success or any other +# value on error. +# +# In order to enable or disable this script just change the execution +# bits. +# +# By default this script does nothing. + +# Print the IP address +_IP=$(hostname -I) || true +if [ "$_IP" ]; then + printf "My IP address is %s\n" "$_IP" +fi + +sudo /home/pi/scripts/hotspot-boot.sh & + +exit 0 + +``` + +#### 安装Samba服务和NTFS兼容驱动. + +我们要安装下面几个软件使我们能够访问树莓派分享的文件夹,ntfs-3g可以使我们能够方位ntfs文件系统的文件. + +``` +sudo apt-get install ntfs-3g +sudo apt-get install samba samba-common-bin +``` + +你可以参照这些文档来配置Samba[17] . + +重要提示,推荐的文档要先挂在外置硬盘,我们不这样做,因为在这篇文章写作的时候树莓派在启动时的auto-mounts功能同时将sd卡和优盘挂载到`/media/pi/`上,这篇文章有一些多余的功能我们也不会采用。 + +### 2. Python脚本 + +树莓派配置好后,我们需要让脚本拷贝和备份照片的时候真正的起作用,脚本只提供了特定的自动化备份进程,如果你有基本的cli操作的技能,你可以ssh进树莓派,然后拷贝你自己的照片从一个设备到另外一个设备用cp或者rsync命令。在脚本里我们用rsync命令,这个命令比较可靠而且支持增量备份。 + +这个过程依赖两个文件,脚本文件自身和`backup_photos.conf`这个配置文件,后者只有几行包含已挂载的目的驱动器和应该挂载到哪个目录,它看起来是这样的: + +``` +mount folder=/media/pi/ +destination folder=PDRIVE128GB +``` + +重要提示: 在这个符号`=`前后不要添加多余的空格,否则脚本会失效. + +下面是这个Python脚本,我把它命名为`backup_photos.py`,把它放到了`/home/pi/scripts/`目录下,我在每行都做了注释可以方便的查看各行的功能. + +``` +#!/usr/bin/python3 + +import os +import sys +from sh import rsync + +''' +脚本将挂载到/media/pi的sd卡上的内容复制到一个目的磁盘的同名目录下,目的驱动器的名字在.conf文件里定义好了. + + +Argument: label/name of the mounted SD Card. +''' + +CONFIG_FILE = '/home/pi/scripts/backup_photos.conf' +ORIGIN_DEV = sys.argv[1] + +def create_folder(path): + + print ('attempting to create destination folder: ',path) + if not os.path.exists(path): + try: + os.mkdir(path) + print ('Folder created.') + except: + print ('Folder could not be created. Stopping.') + return + else: + print ('Folder already in path. Using that instead.') + + + +confFile = open(CONFIG_FILE,'rU') +#IMPORTANT: rU Opens the file with Universal Newline Support, +#so \n and/or \r is recognized as a new line. + +confList = confFile.readlines() +confFile.close() + + +for line in confList: + line = line.strip('\n') + + try: + name , value = line.split('=') + + if name == 'mount folder': + mountFolder = value + elif name == 'destination folder': + destDevice = value + + + except ValueError: + print ('Incorrect line format. Passing.') + pass + + +destFolder = mountFolder+destDevice+'/'+ORIGIN_DEV +create_folder(destFolder) + +print ('Copying files...') + +# Comment out to delete files that are not in the origin: +# rsync("-av", "--delete", mountFolder+ORIGIN_DEV, destFolder) +rsync("-av", mountFolder+ORIGIN_DEV+'/', destFolder) + +print ('Done.') +``` + +### 3.iPad Pro的配置 + +树莓派做了最重的活,而且iPad Pro根本没参与传输文件,我们在iPad上只需要安装上Prompt2来通过ssh连接树莓派就行了,这样你既可以运行Python脚本也可以复制文件了。[18]; [19]. + +![](http://www.movingelectrons.net/images/bkup_photos_ipad&rpi_prompt.jpg) +>iPad用prompt通过SSH连接树莓派. + +我们安装了Samba,我们可以通过图形方式通过树莓派连接到USB设备,你可以看视频,在不同的设备之间复制和移动文件,文件浏览器是必须的[20] . + +### 4. 将它们都放到一起 + +我们假设`SD32GB-03`是连接到树莓派的SD卡名字,`PDRIVE128GB`是那个优盘通过事先的配置文件挂载好,如果我们想要备份SD卡上的图片,我们需要这么做: + +1. 让树莓派先正常运行,将设备挂载好. +2. 连接树莓派配置好的WIFI网络. +3. 用prompt这个app通过ssh连接树莓派[21]. +4. 连接好后输入下面的命令: + +``` +python3 backup_photos.py SD32GB-03 +``` + +首次备份需要一些时间基于SD卡的容量,你需要保持好设备之间的连接,在脚本运行之前你可以通过下面这个命令绕过. + +``` +nohup python3 backup_photos.py SD32GB-03 & +``` + +![](http://www.movingelectrons.net/images/bkup_photos_ipad&rpi_finished.png) +>运行完成的脚本如图所示. + +### 未来的定制 + +我在树莓派上安装了vnc服务,这样我可以通过ipad连接树莓派的图形界面,我安装了bittorrent用来远端备份我的图片,当然需要先设置好,我会放出这些当我完成这些工作后[23[24]。 + +你可以在下面发表你的评论和问题,我会在此页下面回复。. + + +-------------------------------------------------------------------------------- + +via: http://www.movingelectrons.net/blog/2016/06/26/backup-photos-while-traveling-with-a-raspberry-pi.html + +作者:[Editor][a] +译者:[jiajia9linuxer](https://github.com/jiajia9linuxer) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.movingelectrons.net/blog/2016/06/26/backup-photos-while-traveling-with-a-raspberry-pi.html +[1]: http://bit.ly/1MVVtZi +[2]: http://www.amazon.com/dp/B01D3NZIMA/?tag=movinelect0e-20 +[3]: http://www.amazon.com/dp/B01CD5VC92/?tag=movinelect0e-20 +[4]: http://www.amazon.com/dp/B010Q57T02/?tag=movinelect0e-20 +[5]: http://www.amazon.com/dp/B01F1PSFY6/?tag=movinelect0e-20 +[6]: http://amzn.to/293kPqX +[7]: http://amzn.to/290syFY +[8]: http://amzn.to/290syFY +[9]: http://amzn.to/290syFY +[10]: http://amzn.to/290syFY +[11]: http://amzn.to/293kPqX +[12]: https://www.raspberrypi.org/downloads/noobs/ +[13]: https://www.raspberrypi.org/documentation/remote-access/ssh/passwordless.md +[14]: https://www.iterm2.com/ +[15]: https://frillip.com/using-your-raspberry-pi-3-as-a-wifi-access-point-with-hostapd/ +[16]: https://www.raspberrypi.org/documentation/linux/usage/rc-local.md +[17]: http://www.howtogeek.com/139433/how-to-turn-a-raspberry-pi-into-a-low-power-network-storage-device/ +[18]: http://bit.ly/1MVVtZi +[19]: https://itunes.apple.com/us/app/prompt-2/id917437289?mt=8&uo=4&at=11lqkH +[20]: https://itunes.apple.com/us/app/filebrowser-access-files-on/id364738545?mt=8&uo=4&at=11lqkH +[21]: https://itunes.apple.com/us/app/prompt-2/id917437289?mt=8&uo=4&at=11lqkH +[22]: https://en.m.wikipedia.org/wiki/Nohup +[23]: https://itunes.apple.com/us/app/remoter-pro-vnc-ssh-rdp/id519768191?mt=8&uo=4&at=11lqkH +[24]: https://getsync.com/ diff --git a/translated/tech/20160905 QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务.md b/translated/tech/20160905 QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务.md new file mode 100755 index 0000000000..532ad8ab72 --- /dev/null +++ b/translated/tech/20160905 QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务.md @@ -0,0 +1,80 @@ +QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务 +=============== + +[QOwnNotes][1]是一款免费的开源的笔记记录和待办事项的应用,可以运行在linux,windows和mac上. + +这款程序将你的笔记保存为纯文本文件,而且它有Markdown支持并且集成云服务. + +![](https://2.bp.blogspot.com/-a2vsrOG0zFk/V81gyHWlaaI/AAAAAAAAYZs/uzY16JtNcT8bnje1rTKJx1135WueY6V9gCLcB/s400/qownnotes.png) + +WwnNotes的亮点就是它集成可用的云服务,用这款APP,你可以在网路上记录和搜索你的笔记,或者从移动设备上,比如一款像CloudNotes的软件[2]). + +不久以后,用你的云账户连接QOwnNotes你可以分享查看或者恢复之前版本记录的笔记. + +同样,QOwnNotes也集成云任务或者应用程序附加的任务. + +如果你不熟悉ownCloud,这是一款替代注入Dropbox,Google Drive和其他类似网络服务的软件,它可以安装在你自己的服务器上。它有一个网络端口,提供了文件管理,日历,照片,音乐,文件浏览等等其他功能。开发者同样提供桌面同步客户端,和移动APP类似。[3]. + +因为笔记被保存为纯文本,它们可以在不同的设备之间通过云存储服务进行同步,比如Dropbox,Google Drive等等,但是如果没有程序的话是无法做到的。. + +我提到的上述特点,比如恢复之前的笔记,只能在ownCloud下可用(尽管Dropbox和其他类似的也提供这样的服务,但是你用QOwnnotes做不到这点. + +鉴于QOwnNotes有这么多优点,它支持Markdown语言,可以标记笔记,进行搜索,在笔记中加入超链接,也可以插入图片: + +![](https://4.bp.blogspot.com/-SuBhC43gzkY/V81oV7-zLBI/AAAAAAAAYZ8/l6nLQQSUv34Y7op_Xrma8XYm6EdWrhbIACLcB/s400/qownnotes_2.png) + +分层标记和子文件夹同样支持. + +代办事项管理功能比较基本还可以做一些改进,它现在打开被放在一个单独的窗口里,它也不用和笔记一样的编辑器,不允许添加图片或者使用Markdown语言. + +![](https://3.bp.blogspot.com/-AUeyZS3s_ck/V81opialKtI/AAAAAAAAYaA/xukIiZZUdNYBVZ92xgKEsEFew7q961CDwCLcB/s400/qownnotes-tasks.png) + +它可以让你搜索,设置事项优先级,添加提醒和现实完整条目。同样,待办事项可以加入笔记中。. + +这款软件的界面是可定制的,允许你放大或缩小字体,切换窗格等等,也支持无干扰模式。: + +![](https://4.bp.blogspot.com/-Pnzw1wZde50/V81rrE6mTWI/AAAAAAAAYaM/0UZnH9ktbAgClkuAk1g6fgXK87kB_Bh0wCLcB/s400/qownnotes-distraction-free.png) + +从程序的设置里,你可以开启黑夜模式(这里有个bug,在Ubuntu 16.04里有些状态条和图表消失了)),改变状态条大小,字体和颜色方案: + +![](https://1.bp.blogspot.com/-K1MGlXA8sxs/V81rv3fwL6I/AAAAAAAAYaQ/YDhhhnbJ9gY38B6Vz1Na_pHLCjLHhPWiwCLcB/s400/qownnotes-settings.png) + +其他的特点有支持加密,自定义键盘快捷键,输出笔记为pdf或者Markdown,自定义笔记保存间隔等等。. + +访问QOwnNotes主页查看完整的特性[11]. + + +### 下载QOwnNotes + + +如何安装,请查看安装页(支持Debian, Ubuntu, Linux Mint, openSUSE, Fedora, Arch Linux, KaOS, Gentoo, Slakware, CentOS, as well as Mac OSX and Windows). + +QOwnNotes的snap包也是可用的,你可以通过Ubuntu的软件管理器直接安装它。. + +为了集成QOwnNotes到ownCloud你需要ownCloud server,同样也需要Notes,QOwnNotesAPI,Tasks,Tasks Plus ownColud apps。这些可以从ownCloud的网页上安装,不需要手动下载. + +请注意QOenNotesAPi和Notes ownCloud程序是实验性的,你需要启用实验程序来发现并安装他们,这些可以从ownCloud的网页上安装,在Apps下,在左下角点击设置按钮. + + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/09/qownnotes-is-note-taking-and-todo-list.html + +作者:[Andrew][a] +译者:[jiajia9linuxer](https://github.com/jiajia9linuxer) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: http://www.qownnotes.org/ +[2]: http://peterandlinda.com/cloudnotes/ +[3]: https://owncloud.org/ +[11]: http://www.qownnotes.org/ +[4]: http://www.qownnotes.org/installation +[5]: https://uappexplorer.com/app/qownnotes.pbek +[6]: https://download.owncloud.org/download/repositories/stable/owncloud/ +[7]: https://github.com/owncloud/notes +[8]: https://github.com/pbek/qownnotesapi +[9]: https://apps.owncloud.com/content/show.php/Tasks?content=164356 +[10]: https://apps.owncloud.com/content/show.php/Tasks+Plus?content=170561 From 1498a6599fd7ddb552596dce7df96a97c5ea6385 Mon Sep 17 00:00:00 2001 From: wxy Date: Wed, 14 Sep 2016 13:13:22 +0800 Subject: [PATCH 0124/2437] =?UTF-8?q?=E4=B8=8D=E8=A6=81=E6=94=B9=E6=96=87?= =?UTF-8?q?=E4=BB=B6=E5=90=8D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @jiajia9linuxer @oska874 --- ...60626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md} | 0 ...905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md} | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename translated/tech/{20160626 旅行时通过树莓派和iPad Pro备份图片.md => 20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md} (100%) rename translated/tech/{20160905 QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务.md => 20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md} (100%) diff --git a/translated/tech/20160626 旅行时通过树莓派和iPad Pro备份图片.md b/translated/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md similarity index 100% rename from translated/tech/20160626 旅行时通过树莓派和iPad Pro备份图片.md rename to translated/tech/20160626 Backup Photos While Traveling With an Ipad Pro and a Raspberry Pi.md diff --git a/translated/tech/20160905 QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务.md b/translated/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md similarity index 100% rename from translated/tech/20160905 QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务.md rename to translated/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md From 50397d0ee7c052b3656da24f7994d53456be65db Mon Sep 17 00:00:00 2001 From: wxy Date: Wed, 14 Sep 2016 13:52:53 +0800 Subject: [PATCH 0125/2437] PUB:20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD @jiajia9linuxer --- ... LIST APP THAT INTEGRATES WITH OWNCLOUD.md | 77 ++++++++++++++++++ ... LIST APP THAT INTEGRATES WITH OWNCLOUD.md | 80 ------------------- 2 files changed, 77 insertions(+), 80 deletions(-) create mode 100755 published/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md delete mode 100755 translated/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md diff --git a/published/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md b/published/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md new file mode 100755 index 0000000000..ccef2057ef --- /dev/null +++ b/published/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md @@ -0,0 +1,77 @@ +QOwnNotes:一款记录笔记和待办事项的应用,集成 ownCloud 云服务 +=============== + +[QOwnNotes][1] 是一款自由而开源的笔记记录和待办事项的应用,可以运行在 Linux、Windows 和 mac 上。 + +这款程序将你的笔记保存为纯文本文件,它支持 Markdown 支持,并与 ownCloud 云服务紧密集成。 + +![](https://2.bp.blogspot.com/-a2vsrOG0zFk/V81gyHWlaaI/AAAAAAAAYZs/uzY16JtNcT8bnje1rTKJx1135WueY6V9gCLcB/s400/qownnotes.png) + +QOwnNotes 的亮点就是它集成了 ownCloud 云服务(当然是可选的)。在 ownCloud 上用这款 APP,你就可以在网路上记录和搜索你的笔记,也可以在移动设备上使用(比如一款像 CloudNotes 的软件[2])。 + +不久以后,用你的 ownCloud 账户连接上 QOwnNotes,你就可以从你 ownCloud 服务器上分享笔记和查看或恢复之前版本记录的笔记(或者丢到垃圾箱的笔记)。 + +同样,QOwnNotes 也可以与 ownCloud 任务或者 Tasks Plus 应用程序相集成。 + + 如果你不熟悉 [ownCloud][3] 的话,这是一款替代 Dropbox、Google Drive 和其他类似商业性的网络服务的自由软件,它可以安装在你自己的服务器上。它有一个网络界面,提供了文件管理、日历、照片、音乐、文档浏览等等功能。开发者同样提供桌面同步客户端以及移动 APP。 + +因为笔记被保存为纯文本,它们可以在不同的设备之间通过云存储服务进行同步,比如 Dropbox,Google Drive 等等,但是在这些应用中不能完全替代 ownCloud 的作用。 + +我提到的上述特点,比如恢复之前的笔记,只能在 ownCloud 下可用(尽管 Dropbox 和其他类似的也提供恢复以前的文件的服务,但是你不能在 QOwnnotes 中直接访问到)。 + +鉴于 QOwnNotes 有这么多优点,它支持 Markdown 语言(内置了 Markdown 预览模式),可以标记笔记,对标记和笔记进行搜索,在笔记中加入超链接,也可以插入图片: + +![](https://4.bp.blogspot.com/-SuBhC43gzkY/V81oV7-zLBI/AAAAAAAAYZ8/l6nLQQSUv34Y7op_Xrma8XYm6EdWrhbIACLcB/s400/qownnotes_2.png) + +标记嵌套和笔记文件夹同样支持。 + +代办事项管理功能比较基本还可以做一些改进,它现在打开在一个单独的窗口里,它也不用和笔记一样的编辑器,也不允许添加图片或者使用 Markdown 语言。 + +![](https://3.bp.blogspot.com/-AUeyZS3s_ck/V81opialKtI/AAAAAAAAYaA/xukIiZZUdNYBVZ92xgKEsEFew7q961CDwCLcB/s400/qownnotes-tasks.png) + +它可以让你搜索你代办事项,设置事项优先级,添加提醒和显示完成的事项。此外,待办事项可以加入笔记中。 + +这款软件的界面是可定制的,允许你放大或缩小字体,切换窗格等等,也支持无干扰模式。 + +![](https://4.bp.blogspot.com/-Pnzw1wZde50/V81rrE6mTWI/AAAAAAAAYaM/0UZnH9ktbAgClkuAk1g6fgXK87kB_Bh0wCLcB/s400/qownnotes-distraction-free.png) + +从程序的设置里,你可以开启黑夜模式(这里有个 bug,在 Ubuntu 16.04 里有些工具条图标消失了),改变状态条大小,字体和颜色方案(白天和黑夜): + +![](https://1.bp.blogspot.com/-K1MGlXA8sxs/V81rv3fwL6I/AAAAAAAAYaQ/YDhhhnbJ9gY38B6Vz1Na_pHLCjLHhPWiwCLcB/s400/qownnotes-settings.png) + +其他的特点有支持加密(笔记只能在 QOwnNotes 中加密),自定义键盘快捷键,输出笔记为 pdf 或者 Markdown,自定义笔记自动保存间隔等等。 + +访问 [QOwnNotes][11] 主页查看完整的特性。 + +### 下载 QOwnNotes + +如何安装,请查看安装页(支持 Debian、Ubuntu、Linux Mint、openSUSE、Fedora、Arch Linux、KaOS、Gentoo、Slackware、CentOS 以及 Mac OSX 和 Windows)。 + +QOwnNotes 的 [snap][5] 包也是可用的,在 Ubuntu 16.04 或更新版本中,你可以通过 Ubuntu 的软件管理器直接安装它。 + +为了集成 QOwnNotes 到 ownCloud,你需要有 [ownCloud 服务器][6],同样也需要 [Notes][7]、[QOwnNotesAPI][8]、[Tasks][9]、[Tasks Plus][10] 等 ownColud 应用。这些可以从 ownCloud 的 Web 界面上安装,不需要手动下载。 + +请注意 QOenNotesAPI 和 Notes ownCloud 应用是实验性的,你需要“启用实验程序”来发现并安装他们,可以从 ownCloud 的 Web 界面上进行设置,在 Apps 菜单下,在左下角点击设置按钮。 + +-------------------------------------------------------------------------------- + +via: http://www.webupd8.org/2016/09/qownnotes-is-note-taking-and-todo-list.html + +作者:[Andrew][a] +译者:[jiajia9linuxer](https://github.com/jiajia9linuxer) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.webupd8.org/p/about.html +[1]: http://www.qownnotes.org/ +[2]: http://peterandlinda.com/cloudnotes/ +[3]: https://owncloud.org/ +[4]: http://www.qownnotes.org/installation +[5]: https://uappexplorer.com/app/qownnotes.pbek +[6]: https://download.owncloud.org/download/repositories/stable/owncloud/ +[7]: https://github.com/owncloud/notes +[8]: https://github.com/pbek/qownnotesapi +[9]: https://apps.owncloud.com/content/show.php/Tasks?content=164356 +[10]: https://apps.owncloud.com/content/show.php/Tasks+Plus?content=170561 +[11]: http://www.qownnotes.org/ diff --git a/translated/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md b/translated/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md deleted file mode 100755 index 532ad8ab72..0000000000 --- a/translated/tech/20160905 QOWNNOTES IS A NOTE TAKING AND TODO LIST APP THAT INTEGRATES WITH OWNCLOUD.md +++ /dev/null @@ -1,80 +0,0 @@ -QOwnNotes是一款记录笔记和待办事项的APP并且集成OwnCloud云服务 -=============== - -[QOwnNotes][1]是一款免费的开源的笔记记录和待办事项的应用,可以运行在linux,windows和mac上. - -这款程序将你的笔记保存为纯文本文件,而且它有Markdown支持并且集成云服务. - -![](https://2.bp.blogspot.com/-a2vsrOG0zFk/V81gyHWlaaI/AAAAAAAAYZs/uzY16JtNcT8bnje1rTKJx1135WueY6V9gCLcB/s400/qownnotes.png) - -WwnNotes的亮点就是它集成可用的云服务,用这款APP,你可以在网路上记录和搜索你的笔记,或者从移动设备上,比如一款像CloudNotes的软件[2]). - -不久以后,用你的云账户连接QOwnNotes你可以分享查看或者恢复之前版本记录的笔记. - -同样,QOwnNotes也集成云任务或者应用程序附加的任务. - -如果你不熟悉ownCloud,这是一款替代注入Dropbox,Google Drive和其他类似网络服务的软件,它可以安装在你自己的服务器上。它有一个网络端口,提供了文件管理,日历,照片,音乐,文件浏览等等其他功能。开发者同样提供桌面同步客户端,和移动APP类似。[3]. - -因为笔记被保存为纯文本,它们可以在不同的设备之间通过云存储服务进行同步,比如Dropbox,Google Drive等等,但是如果没有程序的话是无法做到的。. - -我提到的上述特点,比如恢复之前的笔记,只能在ownCloud下可用(尽管Dropbox和其他类似的也提供这样的服务,但是你用QOwnnotes做不到这点. - -鉴于QOwnNotes有这么多优点,它支持Markdown语言,可以标记笔记,进行搜索,在笔记中加入超链接,也可以插入图片: - -![](https://4.bp.blogspot.com/-SuBhC43gzkY/V81oV7-zLBI/AAAAAAAAYZ8/l6nLQQSUv34Y7op_Xrma8XYm6EdWrhbIACLcB/s400/qownnotes_2.png) - -分层标记和子文件夹同样支持. - -代办事项管理功能比较基本还可以做一些改进,它现在打开被放在一个单独的窗口里,它也不用和笔记一样的编辑器,不允许添加图片或者使用Markdown语言. - -![](https://3.bp.blogspot.com/-AUeyZS3s_ck/V81opialKtI/AAAAAAAAYaA/xukIiZZUdNYBVZ92xgKEsEFew7q961CDwCLcB/s400/qownnotes-tasks.png) - -它可以让你搜索,设置事项优先级,添加提醒和现实完整条目。同样,待办事项可以加入笔记中。. - -这款软件的界面是可定制的,允许你放大或缩小字体,切换窗格等等,也支持无干扰模式。: - -![](https://4.bp.blogspot.com/-Pnzw1wZde50/V81rrE6mTWI/AAAAAAAAYaM/0UZnH9ktbAgClkuAk1g6fgXK87kB_Bh0wCLcB/s400/qownnotes-distraction-free.png) - -从程序的设置里,你可以开启黑夜模式(这里有个bug,在Ubuntu 16.04里有些状态条和图表消失了)),改变状态条大小,字体和颜色方案: - -![](https://1.bp.blogspot.com/-K1MGlXA8sxs/V81rv3fwL6I/AAAAAAAAYaQ/YDhhhnbJ9gY38B6Vz1Na_pHLCjLHhPWiwCLcB/s400/qownnotes-settings.png) - -其他的特点有支持加密,自定义键盘快捷键,输出笔记为pdf或者Markdown,自定义笔记保存间隔等等。. - -访问QOwnNotes主页查看完整的特性[11]. - - -### 下载QOwnNotes - - -如何安装,请查看安装页(支持Debian, Ubuntu, Linux Mint, openSUSE, Fedora, Arch Linux, KaOS, Gentoo, Slakware, CentOS, as well as Mac OSX and Windows). - -QOwnNotes的snap包也是可用的,你可以通过Ubuntu的软件管理器直接安装它。. - -为了集成QOwnNotes到ownCloud你需要ownCloud server,同样也需要Notes,QOwnNotesAPI,Tasks,Tasks Plus ownColud apps。这些可以从ownCloud的网页上安装,不需要手动下载. - -请注意QOenNotesAPi和Notes ownCloud程序是实验性的,你需要启用实验程序来发现并安装他们,这些可以从ownCloud的网页上安装,在Apps下,在左下角点击设置按钮. - - --------------------------------------------------------------------------------- - -via: http://www.webupd8.org/2016/09/qownnotes-is-note-taking-and-todo-list.html - -作者:[Andrew][a] -译者:[jiajia9linuxer](https://github.com/jiajia9linuxer) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.webupd8.org/p/about.html -[1]: http://www.qownnotes.org/ -[2]: http://peterandlinda.com/cloudnotes/ -[3]: https://owncloud.org/ -[11]: http://www.qownnotes.org/ -[4]: http://www.qownnotes.org/installation -[5]: https://uappexplorer.com/app/qownnotes.pbek -[6]: https://download.owncloud.org/download/repositories/stable/owncloud/ -[7]: https://github.com/owncloud/notes -[8]: https://github.com/pbek/qownnotesapi -[9]: https://apps.owncloud.com/content/show.php/Tasks?content=164356 -[10]: https://apps.owncloud.com/content/show.php/Tasks+Plus?content=170561 From 8a1fa8c114b08683765e025c8e1826f2a16789a6 Mon Sep 17 00:00:00 2001 From: wxy Date: Wed, 14 Sep 2016 14:31:10 +0800 Subject: [PATCH 0126/2437] PUB:20160816 Accelerating Node.js applications with HTTP2 Server Push MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @echoma 译文行文很轻松,不错。不过尽量应该保留原文全部信息。 --- ....js applications with HTTP2 Server Push.md | 111 +++++++++++++++++ ....js applications with HTTP2 Server Push.md | 113 ------------------ 2 files changed, 111 insertions(+), 113 deletions(-) create mode 100644 published/20160816 Accelerating Node.js applications with HTTP2 Server Push.md delete mode 100644 translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md diff --git a/published/20160816 Accelerating Node.js applications with HTTP2 Server Push.md b/published/20160816 Accelerating Node.js applications with HTTP2 Server Push.md new file mode 100644 index 0000000000..d2f33e24f0 --- /dev/null +++ b/published/20160816 Accelerating Node.js applications with HTTP2 Server Push.md @@ -0,0 +1,111 @@ +使用 HTTP/2 服务端推送技术加速 Node.js 应用 +========================================================= + +四月份,我们宣布了对 [HTTP/2 服务端推送技术][3]的支持,我们是通过 HTTP 的 [Link 头部](https://www.w3.org/wiki/LinkHeader)来实现这项支持的。我的同事 John 曾经通过一个例子演示了[在 PHP 里支持服务端推送功能][4]是多么的简单。 + +![](https://blog.cloudflare.com/content/images/2016/08/489477622_594bf9e3d9_z.jpg) + +我们想让现今使用 Node.js 构建的网站能够更加轻松的获得性能提升。为此,我们开发了 [netjet][1] 中间件,它可以解析应用生成的 HTML 并自动添加 Link 头部。当在一个示例的 Express 应用中使用这个中间件时,我们可以看到应用程序的输出多了如下 HTTP 头: + +![](https://blog.cloudflare.com/content/images/2016/08/2016-08-11_13-32-45.png) + +[本博客][5]是使用 [Ghost](https://ghost.org/)(LCTT 译注:一个博客发布平台)进行发布的,因此如果你的浏览器支持 HTTP/2,你已经在不知不觉中享受了服务端推送技术带来的好处了。接下来,我们将进行更详细的说明。 + +netjet 使用了带有定制插件的 [PostHTML](https://github.com/posthtml/posthtml) 来解析 HTML。目前,netjet 用它来查找图片、脚本和外部 CSS 样式表。你也可以用其它的技术来实现这个。 + +在响应过程中增加 HTML 解析器有个明显的缺点:这将增加页面加载的延时(到加载第一个字节所花的时间)。大多数情况下,所新增的延时被应用里的其他耗时掩盖掉了,比如数据库访问。为了解决这个问题,netjet 包含了一个可调节的 LRU 缓存,该缓存以 HTTP 的 ETag 头部作为索引,这使得 netjet 可以非常快的为已经解析过的页面插入 Link 头部。 + +不过,如果我们现在从头设计一款全新的应用,我们就应该考虑把页面内容和页面中的元数据分开存放,从而整体地减少 HTML 解析和其它可能增加的延时了。 + +任意的 Node.js HTML 框架,只要它支持类似 Express 这样的中间件,netjet 都是能够兼容的。只要把 netjet 像下面这样加到中间件加载链里就可以了。 + +```javascript +var express = require('express'); +var netjet = require('netjet'); +var root = '/path/to/static/folder'; + +express() + .use(netjet({ + cache: { + max: 100 + } + })) + .use(express.static(root)) + .listen(1337); +``` + +稍微加点代码,netjet 也可以摆脱 HTML 框架,独立工作: + +```javascript +var http = require('http'); +var netjet = require('netjet'); + +var port = 1337; +var hostname = 'localhost'; +var preload = netjet({ + cache: { + max: 100 + } +}); + +var server = http.createServer(function (req, res) { + preload(req, res, function () { + res.statusCode = 200; + res.setHeader('Content-Type', 'text/html'); + res.end('

Hello World

'); + }); +}); + +server.listen(port, hostname, function () { + console.log('Server running at http://' + hostname + ':' + port+ '/'); +}); +``` + +[netjet 文档里][1]有更多选项的信息。 + +### 查看推送了什么数据 + +![](https://blog.cloudflare.com/content/images/2016/08/2016-08-02_10-49-33.png) + +访问[本文][5]时,通过 Chrome 的开发者工具,我们可以轻松的验证网站是否正在使用服务器推送技术(LCTT 译注: Chrome 版本至少为 53)。在“Network”选项卡中,我们可以看到有些资源的“Initiator”这一列中包含了`Push`字样,这些资源就是服务器端推送的。 + +不过,目前 Firefox 的开发者工具还不能直观的展示被推送的资源。不过我们可以通过页面响应头部里的`cf-h2-pushed`头部看到一个列表,这个列表包含了本页面主动推送给浏览器的资源。 + +希望大家能够踊跃为 netjet 添砖加瓦,我也乐于看到有人正在使用 netjet。 + +### Ghost 和服务端推送技术 + +Ghost 真是包罗万象。在 Ghost 团队的帮助下,我把 netjet 也集成到里面了,而且作为测试版内容可以在 Ghost 的 0.8.0 版本中用上它。 + +如果你正在使用 Ghost,你可以通过修改 config.js、并在`production`配置块中增加 `preloadHeaders` 选项来启用服务端推送。 + +```javascript +production: { + url: 'https://my-ghost-blog.com', + preloadHeaders: 100, + // ... +} +``` + +Ghost 已经为其用户整理了[一篇支持文档][2]。 + +### 总结 + +使用 netjet,你的 Node.js 应用也可以使用浏览器预加载技术。并且 [CloudFlare][5] 已经使用它在提供了 HTTP/2 服务端推送了。 + +-------------------------------------------------------------------------------- + +via: https://blog.cloudflare.com/accelerating-node-js-applications-with-http-2-server-push/ + +作者:[Terin Stock][a] +译者:[echoma](https://github.com/echoma) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://blog.cloudflare.com/author/terin-stock/ +[1]: https://www.npmjs.com/package/netjet +[2]: http://support.ghost.org/preload-headers/ +[3]: https://www.cloudflare.com/http2/server-push/ +[4]: https://blog.cloudflare.com/using-http-2-server-push-with-php/ +[5]: https://blog.cloudflare.com/accelerating-node-js-applications-with-http-2-server-push/ \ No newline at end of file diff --git a/translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md b/translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md deleted file mode 100644 index adba7d092f..0000000000 --- a/translated/tech/20160816 Accelerating Node.js applications with HTTP2 Server Push.md +++ /dev/null @@ -1,113 +0,0 @@ -echoma 翻译中 - -使用HTTP/2服务端推送技术加速Node.js应用 -========================================================= - -四月份,我们宣布了对[HTTP/2服务端推送技术][3]的支持,我们是通过HTTP的[Link头](https://www.w3.org/wiki/LinkHeader)来实现这项支持的。我的同事John曾经通过一个例子演示了[在PHP里支持服务端推送功能][4]是多么的简单。 - -![](https://blog.cloudflare.com/content/images/2016/08/489477622_594bf9e3d9_z.jpg) - -我们想让使用Node.js构建的网站能够更加轻松的获得性能提升。为此,我们开发了netjet中间件,它可以解析应用生成的HTML并自动添加Link头。当结合Express框架使用这个中间件时,我们可以看到应用程序的输出多了如下HTTP头: - -![](https://blog.cloudflare.com/content/images/2016/08/2016-08-11_13-32-45.png) - -[本博客][5]是使用 [Ghost](https://ghost.org/)(译者注:一个博客发布平台)进行发布的, 因此如果你的浏览器支持HTTP/2,你已经在不知不觉中享受了服务端推送技术带来的好处了。接下来,我们将进行更详细的说明。 - -netjet使用了带有自制插件的[PostHTML](https://github.com/posthtml/posthtml)来解析HTML。目前,netjet用它来查找图片、脚本和外部样式。 - -在响应过程中增加HTML解析器有个明显的缺点:这将增加页面加载的时延(加载到第一个字节的所花的时间)。大多数情况下,新增的延时被应用里的其他耗时掩盖掉了,比如数据库访问。为了解决这个问题,netjet包含了一个可调节的LRU缓存,该缓存以HTTP的ETag头作为索引,这使得netjet可以非常快的为已经解析过的页面插入Link头。 - -在这种情况下,如果我们现在从头设计一款全新的应用,我们就需要全面的考量如何减少HTML解析和页面加载延时了。把页面内容和页面中的元数据分开存放是一种值得考虑的方法。 - -任意的Node.js HTML框架,只要它支持类似Express这样的中间件,netjet都是能够兼容的。只要把netjet像下面这样加到中间件加载链里就可以了。 - -```javascript -var express = require('express'); -var netjet = require('netjet'); -var root = '/path/to/static/folder'; - -express() - .use(netjet({ - cache: { - max: 100 - } - })) - .use(express.static(root)) - .listen(1337); -``` - -稍微加点代码,netjet也可以摆脱HTML框架,独立工作: - -```javascript -var http = require('http'); -var netjet = require('netjet'); - -var port = 1337; -var hostname = 'localhost'; -var preload = netjet({ - cache: { - max: 100 - } -}); - -var server = http.createServer(function (req, res) { - preload(req, res, function () { - res.statusCode = 200; - res.setHeader('Content-Type', 'text/html'); - res.end('

Hello World

'); - }); -}); - -server.listen(port, hostname, function () { - console.log('Server running at http://' + hostname + ':' + port+ '/'); -}); -``` - -[netjet文档里][1]有更多选项的信息。 - -### 查看推送了什么数据 - -![](https://blog.cloudflare.com/content/images/2016/08/2016-08-02_10-49-33.png) - -访问[本文][5]时,通过Chrome的开发者工具,我们可以轻松的验证网站是否正在使用服务器推送技术(译者注:Chrome版本至少为53)。在"Network"选项卡中,我们可以看到有些图片的"Initiator"这一列中包含了`Push`字样,这些图片就是服务器端推送的。 - -目前Firefox的开发者工具还不能直观的展示被推送的自选。不过我们可以通过页面响应头里的`cf-h2-pushed`头看到一个列表,这个列表包含了本页面主动推送给浏览器的资源。 - -希望大家能够踊跃为netjet添砖加瓦,我也乐于看到有人正在使用netjet。 - -### Ghost和服务端推送技术 - -Ghost真是包罗万象。在Ghost团队的帮助下,我把netjet也集成到里面了,而且作为测试版内容可以在Ghost的0.8.0版本中用上它。 - -如果你正在使用Ghost,你可以通过修改config.js、并在`production`配置块中增加preloadHeaders选项来启用服务端推送。 - -```javascript -production: { - url: 'https://my-ghost-blog.com', - preloadHeaders: 100, - // ... -} -``` - -Ghost已经为其用户整理了[一篇支持文档][2]. - -### 结论 - -使用netjet,你的Node.js应用也可以使用浏览器预加载技术。并且[本站][5]已经使用它在提供了HTTP/2服务端推送了。 - --------------------------------------------------------------------------------- - -via: https://blog.cloudflare.com/accelerating-node-js-applications-with-http-2-server-push/ - -作者:[Terin Stock][a] -译者:[译者ID](https://github.com/echoma) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://blog.cloudflare.com/author/terin-stock/ -[1]: https://www.npmjs.com/package/netjet -[2]: http://support.ghost.org/preload-headers/ -[3]: https://www.cloudflare.com/http2/server-push/ -[4]: https://blog.cloudflare.com/using-http-2-server-push-with-php/ -[5]: https://blog.cloudflare.com/accelerating-node-js-applications-with-http-2-server-push/ \ No newline at end of file From 1ae711da22a37a48a6d968addb2f2fdf571768ab Mon Sep 17 00:00:00 2001 From: LinuxBars Date: Wed, 14 Sep 2016 17:39:14 +0800 Subject: [PATCH 0127/2437] translated --- ...A Approach to Accelerating SQL at Scale.md | 62 ------------------- ...A Approach to Accelerating SQL at Scale.md | 39 ++++++++++++ 2 files changed, 39 insertions(+), 62 deletions(-) delete mode 100644 sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md create mode 100644 translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md diff --git a/sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md b/sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md deleted file mode 100644 index 7b79854bb7..0000000000 --- a/sources/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md +++ /dev/null @@ -1,62 +0,0 @@ -LinuxBars Translating -LinuxBars 翻译中 - -Baidu Takes FPGA Approach to Accelerating SQL at Scale -=================== - -![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGAFeatured-200x114.png) - -While much of the work at Baidu we have focused on this year has centered on the Chinese search giant’s [deep learning initiatives][1], many other critical, albeit less bleeding edge applications present true big data challenges. - -As Baidu’s Jian Ouyang detailed this week at the Hot Chips conference, Baidu sits on over an exabyte of data, processes around 100 petabytes per day, updates 10 billion webpages daily, and handles over a petabyte of log updates every 24 hours. These numbers are on par with Google and as one might imagine, it takes a Google-like approach to problem solving at scale to get around potential bottlenecks. - -Just as we have described Google looking for any way possible to beat Moore’s Law, Baidu is on the same quest. While the exciting, sexy machine learning work is fascinating, acceleration of the core mission-critical elements of the business is as well—because it has to be. As Ouyang notes, there is a widening gap between the company’s need to deliver top-end services based on their data and what CPUs are capable of delivering. - -![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA1.png) - -As for Baidu’s exascale problems, on the receiving end of all of this data are a range of frameworks and platforms for data analysis; from the company’s massive knowledge graph, multimedia tools, natural language processing frameworks, recommendation engines, and click stream analytics. In short, the big problem of big data is neatly represented here—a diverse array of applications matched with overwhelming data volumes. - -When it comes to acceleration for large-scale data analytics at Baidu, there are several challenges. Ouyang says it is difficult to abstract the computing kernels to find a comprehensive approach. “The diversity of big data applications and variable computing types makes this a challenge. It is also difficult to integrate all of this into a distributed system because there are also variable platforms and program models (MapReduce, Spark, streaming, user defined, and so on). Further there is more variance in data types and storage formats.” - -Despite these barriers, Ouyang says teams looked for the common thread. And as it turns out, that string that ties together many of their data-intensive jobs is good old SQL. “Around 40% of our data analysis jobs are already written in SQL and rewriting others to match it can be done.” Further, he says they have the benefit of using existing SQL system that mesh with existing frameworks like Hive, Spark SQL, and Impala. The natural thing to do was to look for SQL acceleration—and Baidu found no better hardware than an FPGA. - -![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA2.png) - -These boards, called processing elements (PE on coming slides), automatically handle key SQL functions as they come in. With that said, a disclaimer note here about what we were able to glean from the presentation. Exactly what the FPGA is talking to is a bit of a mystery and so by design. If Baidu is getting the kinds of speedups shown below in their benchmarks, this is competitive information. Still, we will share what was described. At its simplest, the FPGAs are running in the database and when it sees SQL queries coming it, the software the team designed ([and presented at Hot Chips two years ago][2] related to DNNs) kicks into gear. - -![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA3.png) - -One thing Ouyang did note about the performance of their accelerator is that their performance could have been higher but they were bandwidth limited with the FPGA. In the evaluation below, Baidu setup with a 12-core 2.0 Ghz Intel E26230 X2 sporting 128 GB of memory. The SDA had five processing elements (the 300 MHzFPGA boards seen above) each of which handles core functions (filter, sort, aggregate, join and group by.). - -To make the SQL accelerator, Baidu picked apart the TPC-DS benchmark and created special engines, called processing elements, that accelerate the five key functions in that benchmark test. These include filter, sort, aggregate, join, and group by SQL functions. (And no, we are not going to put these in all caps to shout as SQL really does.) The SDA setup employs an offload model, with the accelerator card having multiple processing elements of varying kinds shaped into the FPGA logic, with the type of SQL function and the number per card shaped by the specific workload. As these queries are being performed on Baidu’s systems, the data for the queries is pushed to the accelerator card in columnar format (which is blazingly fast for queries) and through a unified SDA API and driver, the SQL work is pushed to the right processing elements and the SQL operations are accelerated. - -The SDA architecture uses a data flow model, and functions not supported by the processing elements are pushed back to the database systems and run natively there. More than any other factor, the performance of the SQL accelerator card developed by Baidu is limited by the memory bandwidth of the FPGA card. The accelerator works across clusters of machines, by the way, but the precise mechanism of how data and SQL operations are parsed out to multiple machines was not disclosed by Baidu. - -![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA4.png) - -![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA5.png) - -We’re limited in some of the details Baidu was willing to share but these benchmark results are quite impressive, particularly for Terasort. We will follow up with Baidu after Hot Chips to see if we can get more detail about how this is hooked together and how to get around the memory bandwidth bottlenecks. - - - --------------------------------------------------------------------------------- - -via: http://www.nextplatform.com/2016/08/24/baidu-takes-fpga-approach-accelerating-big-sql/?utm_source=dbweekly&utm_medium=email - -作者:[Nicole Hemsoth][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.nextplatform.com/author/nicole/ -[1]: http://www.nextplatform.com/?s=baidu+deep+learning -[2]: http://www.hotchips.org/wp-content/uploads/hc_archives/hc26/HC26-12-day2-epub/HC26.12-5-FPGAs-epub/HC26.12.545-Soft-Def-Acc-Ouyang-baidu-v3--baidu-v4.pdf - - - - - - - diff --git a/translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md b/translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md new file mode 100644 index 0000000000..cccb85cdc1 --- /dev/null +++ b/translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md @@ -0,0 +1,39 @@ +百度运用 FPGA 方法从规模上加速 SQL 查询 +=================================================================== + + 尽管今年我们对于百度工作的焦点集中在中国搜索巨头的深度学习上,许多其他的关键的,尽管不那么前沿的应用表现出了大数据带来的挑战。 + + 正如百度主任架构师在本周 Hot Chips 大会上谈论的,百度坐拥超过 1 EB 的数据,每天处理大约 100 PB 的数据,每天更新 100 亿的网页,每 24 小时更新处理超过 1 PB 的日志更新,这些数字和 Google 不分上下,正如人们所想象的。百度采用了类似 Google 的方法去处理规模上潜在的瓶颈。 + + 正如刚刚我们谈到的,Google 寻找一切可能的方法去打败摩尔定律,百度也在进行相同的探索,而令人激动的,使人着迷的机器学习工作是迷人的。业务的核心关键任务的加速同样也是,因为必须如此,欧阳提到,公司基于自身的数据提供高端服务的需求和 CPU 可以承载的能力之间的差距将会逐渐增大。 + + 对于百度的百亿亿级问题,在所有数据的接受端是一系列用于数据分析的框架和平台,从公司的大量的知识图,多媒体工具,自然语言处理框架,推荐引擎,和点击流分析,简而言之,大数据的首要问题就是恰当地在这里表现出一系列的和压倒一切的数据价值的应用。 + +每当谈到关于百度的大数据分析加速,这里有几个挑战,欧阳谈到提取运算核心去寻找一个广泛的方法是困难的。大数据应用和变量计算类型的多样性使得这成为一个挑战,把所有这些整合成为一个分布式系统是困难的,因为有多变的平台和编程模型(MapReduce,Spark,streaming,user defined,等等)。更进一步这里有更多的数据类型和存储格式的变化。 + + + 尽管存在这些障碍,欧阳讲到团队寻找主线,而且随着不断发展,传统的 SQL 把许多数据密集型的任务连系在一起。“大约有 40% 的数据分析任务已经用 SQL 重写,重写其他的去匹配其能做的” 更进一步,他讲道他们可以享受现有的可以和已经存在的框架匹配的系统比如 Hive,Spark,SQL,和 Impala 。下一步要做的事情就是 SQL 查询加速,百度没有找到 FPGA 更好的硬件。 + + 这些主板,被称为处理单元( 下图中的 PE ),自动地处理关键的 SQL 功能当他们进来,那意味着,一个放弃者在这指出我们现在所能收集到的东西,这正是 FPGA 设计神秘的地方,如果百度在基准测试中得到了下图中的加速,这是有竞争力的信息,我们还要分享所描述的东西,简单来说,FPGA 在数据库中运行,当其收到 SQL 查询的时候,团队设计的软件与之啮合。 + + 欧阳提到了一件事,他们的加速器受限与 FPGA 的带宽不然性能表现本可以更高,在下面的评价中,百度安装了 2 块intl E26230 具有 12 核心,主频 2.0 GHZ,支持 128G 内存的 CPU,SDA 具有 5 个处理单元,(上图中的300MHzFPGA 主板)每个处理核心功能(过滤,排序,聚合,加入,和分组) + + 为了实现 SQL 查询加速,百度将 TPC-DS 基准拆散并且创建了特殊的引擎。叫做处理单元,在基准测试中加速 5 个关键功能,这包括过滤,排序,融合,加入和分组,(我们并没有把这些单词都大写的像 SQL 真的那样),SDA 设置使用卸载模型。和具有多个不同种类的处理单元的的加速卡在 FPGA 逻辑中形成。SQL 功能的类型和每张卡的数量由特定的工作量决定。由于这些查询在百度的系统中执行,用来查询的数据被以列 格式存储到加速卡中(这会使得查询非常快速)而且通过一个统一的 SDA API 和驱动,SQL 查询工作被分发到正确的处理单元而且 SQL 操作实现了加速。 + + SDA 架构采用一种数据流模型,加速单元不支持的操作被退回到数据库系统然后在那里本地运行,比其他任何因素,百度开发的 SQL 加速卡的性能被 FPGA 卡的内存带宽所限制。加速卡跨真个集群机器工作,顺便提一下,但是数据和 SQL 操作如何分发到多个机器的准确原理没有被百度披露。 + + 我们受限与百度分享细节的意愿但是这些基准测试结果是非常感人的,尤其是传输测试,我们将跟随百度 Hot Chips 大会之后的脚步去看看我们是否能得到关于这是如何连接到一起的和如何解决内存带宽瓶颈的细节。 + +-------------------------------------------------------------------------------- + +via: http://www.nextplatform.com/2016/08/24/baidu-takes-fpga-approach-accelerating-big-sql/?utm_source=dbweekly&utm_medium=email + +作者:[Nicole Hemsoth][a] +译者:[LinuxBars](https://github.com/LinuxBars) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.nextplatform.com/author/nicole/ +[1]: http://www.nextplatform.com/?s=baidu+deep+learning +[2]: http://www.hotchips.org/wp-content/uploads/hc_archives/hc26/HC26-12-day2-epub/HC26.12-5-FPGAs-epub/HC26.12.545-Soft-Def-Acc-Ouyang-baidu-v3--baidu-v4.pdf From 8a52d6445c1a8d6ea6fc8ed7384432391ae5e52c Mon Sep 17 00:00:00 2001 From: LinuxBars Date: Wed, 14 Sep 2016 17:59:52 +0800 Subject: [PATCH 0128/2437] LinuxBars translating --- ...mputing college feminism and increasing diversity in tech.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md b/sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md index 57faa7998f..78c6cbaecf 100644 --- a/sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md +++ b/sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md @@ -1,3 +1,5 @@ +LinuxBars translating + Torvalds 2.0: Patricia Torvalds on computing, college, feminism, and increasing diversity in tech ================================================================================ From 3b70c35e8c36649639fa00ae35dc1ff29d52f43b Mon Sep 17 00:00:00 2001 From: echoma Date: Wed, 14 Sep 2016 20:28:25 +0800 Subject: [PATCH 0129/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E7=94=B3=E9=A2=86?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../20160718 Tips for managing your project's issue tracker.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/talk/20160718 Tips for managing your project's issue tracker.md b/sources/talk/20160718 Tips for managing your project's issue tracker.md index 1b89fe5851..b2233d5b97 100644 --- a/sources/talk/20160718 Tips for managing your project's issue tracker.md +++ b/sources/talk/20160718 Tips for managing your project's issue tracker.md @@ -1,3 +1,5 @@ +echoma 翻译中 + Tips for managing your project's issue tracker ============================================== From aeac17a4ca5b1e77da4f02a074e777bf50bd2640 Mon Sep 17 00:00:00 2001 From: echoma Date: Wed, 14 Sep 2016 20:53:14 +0800 Subject: [PATCH 0130/2437] =?UTF-8?q?=E7=BF=BB=E5=B7=B2=E5=AE=8C=E6=88=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...r managing your project's issue tracker.md | 103 ------------------ ...r managing your project's issue tracker.md | 101 +++++++++++++++++ 2 files changed, 101 insertions(+), 103 deletions(-) delete mode 100644 sources/talk/20160718 Tips for managing your project's issue tracker.md create mode 100644 translated/talk/20160718 Tips for managing your project's issue tracker.md diff --git a/sources/talk/20160718 Tips for managing your project's issue tracker.md b/sources/talk/20160718 Tips for managing your project's issue tracker.md deleted file mode 100644 index b2233d5b97..0000000000 --- a/sources/talk/20160718 Tips for managing your project's issue tracker.md +++ /dev/null @@ -1,103 +0,0 @@ -echoma 翻译中 - -Tips for managing your project's issue tracker -============================================== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/BUSINESS_opennature_3.png?itok=30fRGfpv) - -Issue-tracking systems are important for many open source projects, and there are many open source tools that provide this functionality but many projects opt to use GitHub's built-in issue tracker. - -Its simple structure makes it easy for others to weigh in, but issues are really only as good as you make them. - -Without a process, your repository can become unwieldy, overflowing with duplicate issues, vague feature requests, or confusing bug reports. Project maintainers can become burdened by the organizational load, and it can become difficult for new contributors to understand where priorities lie. - -In this article, I'll discuss how to take your GitHub issues from good to great. - -### The issue as user story - -My team spoke with open source expert [Jono Bacon][1]—author of [The Art of Community][2], a strategy consultant, and former Director of Community at GitHub—who said that high-quality issues are at the core of helping a projects succeed. He says that while some see issues as merely a big list of problems you have to tend to, well-managed, triaged, and labeled issues can provide incredible insight into your code, your community, and where the problem spots are. - -"At the point of submission of an issue, the user likely has little patience or interest in providing expansive detail. As such, you should make it as easy as possible to get the most useful information from them in the shortest time possible," Jono Bacon said. - -A consistent structure can take a lot of burden off project maintainers, particularly for open source projects. We've found that encouraging a user story approach helps make clarity a constant. The common structure for a user story addresses the "who, what, and why" of a feature: As a [user type], I want to [task] so that [goal]. - -Here's what that looks like in practice: - ->As a customer, I want to create an account so that I can make purchases. - -We suggest sticking that user story in the issue's title. You can also set up [issue templates][3] to keep things consistent. - -![](https://opensource.com/sites/default/files/resize/issuetemplate-new-520x293.png) -> Issue templates bring consistency to feature requests. - -The point is to make the issue well-defined for everyone involved: it identifies the audience (or user), the action (or task), and the outcome (or goal) as simply as possible. There's no need to obsess over this structure, though; as long as the what and why of a story are easy to spot, you're good. - -### Qualities of a good issue - -Not all issues are created equal—as any OSS contributor or maintainer can attest. A well-formed issue meets these qualities outlined in [The Agile Samurai][4]. - -Ask yourself if it is... - -- something of value to customers -- avoids jargon or mumbo jumbo; a non-expert should be able to understand it -- "slices the cake," which means it goes end-to-end to deliver something of value -- independent from other issues if possible; dependent issues reduce flexibility of scope -- negotiable, meaning there are usually several ways to get to the stated goal -- small and easily estimable in terms of time and resources required -- measurable; you can test for results - -### What about everything else? Working with constraints - -If an issue is difficult to measure or doesn't seem feasible to complete within a short time period, you can still work with it. Some people call these "constraints." - -For example, "the product needs to be fast" doesn't fit the story template, but it is non-negotiable. But how fast is fast? Vague requirements don't meet the criteria of a "good issue", but if you further define these concepts—for example, "the product needs to be fast" can be "each page needs to load within 0.5 seconds"—you can work with it more easily. Constraints can be seen as internal metrics of success, or a landmark to shoot for. Your team should test for them periodically. - -### What's inside your issue? - -In agile, user stories typically include acceptance criteria or requirements. In GitHub, I suggest using markdown checklists to outline any tasks that make up an issue. Issues should get more detail as they move up in priority. - -Say you're creating an issue around a new homepage for a website. The sub-tasks for that task might look something like this. - -![](https://opensource.com/sites/default/files/resize/markdownchecklist-520x255.png) ->Use markdown checklists to split a complicated issue into several parts. - -If necessary, link to other issues to further define a task. (GitHub makes this really easy.) - -Defining features as granularly as possible makes it easier to track progress, test for success, and ultimately ship valuable code more frequently. - -Once you've gathered some data points in the form of issues, you can use APIs to glean deeper insight into the health of your project. - -"The GitHub API can be hugely helpful here in identifying patterns and trends in your issues," Bacon said. "With some creative data science, you can identify problem spots in your code, active members of your community, and other useful insights." - -Some issue management tools provide APIs that add additional context, like time estimates or historical progress. - -### Getting others on board - -Once your team decides on an issue structure, how do you get others to buy in? Think of your repo's ReadMe.md file as your project's "how-to." It should clearly define what your project does (ideally using searchable language) and explain how others can contribute (by submitting requests, bug reports, suggestions, or by contributing code itself.) - -![](https://opensource.com/sites/default/files/resize/readme-520x184.png) ->Edit your ReadMe file with clear instructions for new collaborators. - -This is the perfect spot to share your GitHub issue guidelines. If you want feature requests to follow the user story format, share that here. If you use a tracking tool to organize your product backlog, share the badge so others can gain visibility. - -"Issue templates, sensible labels, documentation for how to file issues, and ensuring your issues get triaged and responded to quickly are all important" for your open source project, Bacon said. - -Remember: It's not about adding process for the process' sake. It's about setting up a structure that makes it easy for others to discover, understand, and feel confident contributing to your community. - -"Focus your community growth efforts not just on growing the number of programmers, but also [on] people interested in helping issues be accurate, up to date, and a source of active conversation and productive problem solving," Bacon said. - --------------------------------------------------------------------------------- - -via: https://opensource.com/life/16/7/how-take-your-projects-github-issues-good-great - -作者:[Matt Butler][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://opensource.com/users/mattzenhub -[1]: http://www.jonobacon.org/ -[2]: http://www.artofcommunityonline.org/ -[3]: https://help.github.com/articles/creating-an-issue-template-for-your-repository/ -[4]: https://www.amazon.ca/Agile-Samurai-Masters-Deliver-Software/dp/1934356581 diff --git a/translated/talk/20160718 Tips for managing your project's issue tracker.md b/translated/talk/20160718 Tips for managing your project's issue tracker.md new file mode 100644 index 0000000000..a56bdca1d8 --- /dev/null +++ b/translated/talk/20160718 Tips for managing your project's issue tracker.md @@ -0,0 +1,101 @@ +几个小窍门帮你管理项目的问题追踪器 +============================================== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/BUSINESS_opennature_3.png?itok=30fRGfpv) + +对于大多数开源项目来讲,问题追踪系统是至关重要的。虽然市面上有非常多的开源工具提供了这样的功能,但是大量项目还是选择了GitHub自带的问题追踪器(Issue Tracker)。 + +它结构简单,因此其他人可以非常轻松地参与进来,但这才刚刚开始。 + +如果没有适当的处理,你的代码仓库会挤满重复的问题单、模糊不明的特性需求单、混淆不清的bug报告单。项目维护者会被大量的组织内协调工作压得喘不过气来,新的贡献者也搞不清楚项目当前的重点工作是什么。 + +接下来,我们一起研究下,如何玩转GitHub的问题单。 + +### 问题单就是用户的故事 + +我的团队曾经和开源专家[Jono Bacon][1]做过一次对话,他是[The Art of Community][2]的作者、GitHub的战略顾问和前社区总监。他告诉我们,高质量的问题单是项目成功的关键。尽管有些人把问题单仅仅看作是一堆难题的列表,但是他认为这个难题列表是我们必须要时刻关注、完善管理并进行分类的。他还认为,给问题单打上标签的做法,会令人意想不到的提升我们对代码和社区的了解程度,也让我们更清楚问题的关键点在哪里。 + +“在提交问题单时,用户不太会有耐心或者有兴趣把问题的细节描述清楚。在这种情况下,你应当努力花最短的时间,尽量多的获取有用的信息。”,Jono Bacon说。 + +统一的问题单模板可以大大减轻项目维护者的负担,尤其是开源项目的维护者。我们发现,让用户讲故事的方法总是可以把问题描述的非常清楚。用户讲故事时需要说明“是谁,做了什么,为什么而做”,也就是:我是【何种用户】,为了【达到何种目的】,我要【做何种操作】。 + +实际操作起来,大概是这样的: + +>我是一名顾客,我想付钱,所以我想创建个账户。 + +我们建议,问题单的标题始终使用这样的用户故事形式。你可以设置[问题单模板][3]来保证这点。 + +![](https://opensource.com/sites/default/files/resize/issuetemplate-new-520x293.png) +> 问题单模板让特性需求单保持统一的形式 + +这个做法的核心点在于,问题单要被清晰的呈现给它涉及的每一个人:它要尽量简单的指明受众(或者说用户),操作(或者说任务),和收益(或者说目标)。你不需要拘泥于这个具体的模板,只要能把故事里的是什么事情或者是什么原因搞清楚,就达到目的了。 + +### 高质量的问题单 + +问题单的质量是参差不齐的,这一点任何一个开源软件的贡献者或维护者都能证实。具有良好格式的问题单所应具备的素质在[The Agile Samurai][4]有过概述。 + +问题单需要满足如下条件: + +- 客户价值所在 +- 避免使用术语或晦涩的文字,就算不是专家也能看懂 +- 可以切分,也就是说我们可以一小步一小步的对最终价值进行交付 +- 尽量跟其他问题单没有瓜葛,这会降低我们在问题范围上的灵活性 +- 可以协商,也就说我们有好几种办法达到目标 +- 问题足够小,可以非常容易的评估出所需时间和资源 +- 可衡量,我们可以对结果进行测试 + +### 那其他的呢? 要有约束 + +如果一个问题单很难衡量,或者很难在短时间内完成,你也一样有办法搞定它。有些人把这种办法叫做”约束“。 + +例如,”这个软件要快“,这种问题单是不符合我们的故事模板的,而且是没办法协商的。多快才是快呢?这种模糊的需求没有达到”好问题单“的标准,但是如果你把一些概念进一步定义一下,例如”每个页面都需要在0.5秒内加载完“,那我们就能更轻松的解决它了。我们可以把”约束“看作是成功的标尺,或者是里程碑。每个团队都应该定期的对”约束“进行测试。 + +### 问题单里面有什么? + +敏捷方法中,用户的故事里通常要包含验收指标或者标准。如果是在GitHub里,建议大家使用markdown的清单来概述完成这个问题单需要完成的任务。优先级越高的问题单应当包含更多的细节。 + +比如说,你打算提交一个问题单,关于网站的新版主页的。那这个问题单的子任务列表可能就是这样的: + +![](https://opensource.com/sites/default/files/resize/markdownchecklist-520x255.png) +>使用markdown的清单把复杂问题拆分成多个部分 + +在必要的情况下,你还可以链接到其他问题单,那些问题单每个都是一个要完成的任务。(GitHub里做这个挺方便的) + +将特性进行细粒度的拆分,这样更轻松的跟踪整体的进度和测试,要能更高频的发布有价值的代码。 + +一旦以问题单的形式收到数据,我们还可以用API更深入的了解软件的健康度。 + +”在统计问题单的类型和趋势时,GitHub的API可以发挥巨大作用“,Bacon告诉我们,”如果再做些数据挖掘工作,你就能发现代码里的问题点,谁是社区的活跃成员,或者其他有用的信息。“ + +有些问题单管理工具提供了API,通过API可以增加额外的信息,比如预估时间或者历史进度。 + +### 让大伙都上车 + +一旦你的团队决定使用某种问题单模板,你要想办法让所有人都按照模板来。代码仓库里的ReadMe.md其实也可以是项目的”How-to“文档。这个文档会描述清除这个项目是做什么的(最好是用可以搜索的语言),并且解释其他贡献者应当如何参与进来(比如提交需求单、bug报告、建议,或者直接贡献代码) + +![](https://opensource.com/sites/default/files/resize/readme-520x184.png) +>为新来的合作者在ReadMe文件里增加清晰的说明 + +ReadMe文件是提供”问题单指引“的完美场所。如果你希望特性需求单遵循”用户讲故事“的格式,那就把格式写在ReadMe里。如果你使用某种跟踪工具来管理待办事项,那就标记在ReadMe里,这样别人也能看到。 + +”问题单模板,合理的标签,如何提交问题单的文档,确保问题单被分类,所有的问题单都及时做出回复,这些对于开源项目都至关重要“,Bacon说。 + +记住一点:这不是为了完成工作而做的工作。这时让其他人更轻松的发现、了解、融入你的社区而设立的规则。 + +"关注社区的成长,不仅要关注参与开发者的的数量增长,也要关注那些在问题单上帮助我们的人,他们让问题单更加明确、保持更新,这是活跃沟通和高效解决问题的力量源泉",Bacon说。 + +-------------------------------------------------------------------------------- + +via: https://opensource.com/life/16/7/how-take-your-projects-github-issues-good-great + +作者:[Matt Butler][a] +译者:[echoma](https://github.com/echoma) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/mattzenhub +[1]: http://www.jonobacon.org/ +[2]: http://www.artofcommunityonline.org/ +[3]: https://help.github.com/articles/creating-an-issue-template-for-your-repository/ +[4]: https://www.amazon.ca/Agile-Samurai-Masters-Deliver-Software/dp/1934356581 From a50d1a339d7c2e2259956cf33d3c8f54064403f5 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Wed, 14 Sep 2016 23:41:53 +0800 Subject: [PATCH 0131/2437] Update 20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md --- ...cript Parser that Creates a Language in 200 Lines of Code.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md index 67c117ee0b..df2178ce8e 100644 --- a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md +++ b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md @@ -1,3 +1,5 @@ +Translating by ucasFL + Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code =========== From 891e570730484ed4ff217ba6cb3a99bb4cc01941 Mon Sep 17 00:00:00 2001 From: wxy Date: Fri, 16 Sep 2016 13:17:35 +0800 Subject: [PATCH 0132/2437] PUB:20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @LinuxBars 这篇比较晦涩,句式也比较复杂,还涉及到很多特有的技术细节,翻译不容易,辛苦了。 --- ...A Approach to Accelerating SQL at Scale.md | 48 +++++++++++++++++++ ...A Approach to Accelerating SQL at Scale.md | 39 --------------- 2 files changed, 48 insertions(+), 39 deletions(-) create mode 100644 published/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md delete mode 100644 translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md diff --git a/published/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md b/published/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md new file mode 100644 index 0000000000..c53cf1f982 --- /dev/null +++ b/published/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md @@ -0,0 +1,48 @@ +百度运用 FPGA 方法大规模加速 SQL 查询 +=================================================================== + +尽管我们对百度今年工作焦点的关注集中在这个中国搜索巨头在深度学习方面的举措上,许多其他的关键的,尽管不那么前沿的应用表现出了大数据带来的挑战。 + +正如百度的欧阳剑在本周 Hot Chips 大会上谈论的,百度坐拥超过 1 EB 的数据,每天处理大约 100 PB 的数据,每天更新 100 亿的网页,每 24 小时更新处理超过 1 PB 的日志更新,这些数字和 Google 不分上下,正如人们所想象的。百度采用了类似 Google 的方法去大规模地解决潜在的瓶颈。 + +正如刚刚我们谈到的,Google 寻找一切可能的方法去打败摩尔定律,百度也在进行相同的探索,而令人激动的、使人着迷的机器学习工作是迷人的,业务的核心关键任务的加速同样也是,因为必须如此。欧阳提到,公司基于自身的数据提供高端服务的需求和 CPU 可以承载的能力之间的差距将会逐渐增大。 + +![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA1.png) + +对于百度的百亿亿级问题,在所有数据的接受端是一系列用于数据分析的框架和平台,从该公司的海量知识图谱,多媒体工具,自然语言处理框架,推荐引擎,和点击流分析都是这样。简而言之,大数据的首要问题就是这样的:一系列各种应用和与之匹配的具有压倒性规模的数据。 + +当谈到加速百度的大数据分析,所面临的几个挑战,欧阳谈到抽象化运算核心去寻找一个普适的方法是困难的。“大数据应用的多样性和变化的计算类型使得这成为一个挑战,把所有这些整合成为一个分布式系统是困难的,因为有多变的平台和编程模型(MapReduce,Spark,streaming,user defined,等等)。将来还会有更多的数据类型和存储格式。” + +尽管存在这些障碍,欧阳讲到他们团队找到了(它们之间的)共同线索。如他所指出的那样,那些把他们的许多数据密集型的任务相连系在一起的就是传统的 SQL。“我们的数据分析任务大约有 40% 是用 SQL 写的,而其他的用 SQL 重写也是可用做到的。” 更进一步,他讲道他们可以享受到现有的 SQL 系统的好处,并可以和已有的框架相匹配,比如 Hive,Spark SQL,和 Impala 。下一步要做的事情就是 SQL 查询加速,百度发现 FPGA 是最好的硬件。 + +![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA2.png) + +这些主板,被称为处理单元( 下图中的 PE ),当执行 SQL 时会自动地处理关键的 SQL 功能。这里所说的都是来自演讲,我们不承担责任。确切的说,这里提到的 FPGA 有点神秘,或许是故意如此。如果百度在基准测试中得到了如下图中的提升,那这可是一个有竞争力的信息。后面我们还会继续介绍这里所描述的东西。简单来说,FPGA 运行在数据库中,当其收到 SQL 查询的时候,该团队设计的软件就会与之紧密结合起来。 + +![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA3.png) + +欧阳提到了一件事,他们的加速器受限于 FPGA 的带宽,不然性能表现本可以更高,在下面的评价中,百度安装了 2 块12 核心,主频 2.0 GHz 的 intl E26230 CPU,运行在 128G 内存。SDA 具有 5 个处理单元,(上图中的 300MHz FPGA 主板)每个分别处理不同的核心功能(筛选(filter),排序(sort),聚合(aggregate),联合(join)和分组(group by)) + +为了实现 SQL 查询加速,百度针对 TPC-DS 的基准测试进行了研究,并且创建了称做处理单元(PE)的特殊引擎,用于在基准测试中加速 5 个关键功能,这包括筛选(filter),排序(sort),聚合(aggregate),联合(join)和分组(group by),(我们并没有把这些单词都像 SQL 那样大写)。SDA 设备使用卸载模型,具有多个不同种类的处理单元的加速卡在 FPGA 中组成逻辑,SQL 功能的类型和每张卡的数量由特定的工作量决定。由于这些查询在百度的系统中执行,用来查询的数据被以列格式推送到加速卡中(这会使得查询非常快速),而且通过一个统一的 SDA API 和驱动程序,SQL 查询工作被分发到正确的处理单元而且 SQL 操作实现了加速。 + +SDA 架构采用一种数据流模型,加速单元不支持的操作被退回到数据库系统然后在那里本地运行,比其他任何因素,百度开发的 SQL 加速卡的性能被 FPGA 卡的内存带宽所限制。加速卡跨整个集群机器工作,顺便提一下,但是数据和 SQL 操作如何分发到多个机器的准确原理没有被百度披露。 + +![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA4.png) + +![](http://www.nextplatform.com/wp-content/uploads/2016/08/BaiduFPGA5.png) + +我们受限与百度所愿意披露的细节,但是这些基准测试结果是十分令人鼓舞的,尤其是 Terasort 方面,我们将在 Hot Chips 大会之后跟随百度的脚步去看看我们是否能得到关于这是如何连接到一起的和如何解决内存带宽瓶颈的细节。 + +-------------------------------------------------------------------------------- + +via: http://www.nextplatform.com/2016/08/24/baidu-takes-fpga-approach-accelerating-big-sql/ + +作者:[Nicole Hemsoth][a] +译者:[LinuxBars](https://github.com/LinuxBars) +校对:[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.nextplatform.com/author/nicole/ +[1]: http://www.nextplatform.com/?s=baidu+deep+learning +[2]: http://www.hotchips.org/wp-content/uploads/hc_archives/hc26/HC26-12-day2-epub/HC26.12-5-FPGAs-epub/HC26.12.545-Soft-Def-Acc-Ouyang-baidu-v3--baidu-v4.pdf diff --git a/translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md b/translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md deleted file mode 100644 index cccb85cdc1..0000000000 --- a/translated/tech/20160824 Baidu Takes FPGA Approach to Accelerating SQL at Scale.md +++ /dev/null @@ -1,39 +0,0 @@ -百度运用 FPGA 方法从规模上加速 SQL 查询 -=================================================================== - - 尽管今年我们对于百度工作的焦点集中在中国搜索巨头的深度学习上,许多其他的关键的,尽管不那么前沿的应用表现出了大数据带来的挑战。 - - 正如百度主任架构师在本周 Hot Chips 大会上谈论的,百度坐拥超过 1 EB 的数据,每天处理大约 100 PB 的数据,每天更新 100 亿的网页,每 24 小时更新处理超过 1 PB 的日志更新,这些数字和 Google 不分上下,正如人们所想象的。百度采用了类似 Google 的方法去处理规模上潜在的瓶颈。 - - 正如刚刚我们谈到的,Google 寻找一切可能的方法去打败摩尔定律,百度也在进行相同的探索,而令人激动的,使人着迷的机器学习工作是迷人的。业务的核心关键任务的加速同样也是,因为必须如此,欧阳提到,公司基于自身的数据提供高端服务的需求和 CPU 可以承载的能力之间的差距将会逐渐增大。 - - 对于百度的百亿亿级问题,在所有数据的接受端是一系列用于数据分析的框架和平台,从公司的大量的知识图,多媒体工具,自然语言处理框架,推荐引擎,和点击流分析,简而言之,大数据的首要问题就是恰当地在这里表现出一系列的和压倒一切的数据价值的应用。 - -每当谈到关于百度的大数据分析加速,这里有几个挑战,欧阳谈到提取运算核心去寻找一个广泛的方法是困难的。大数据应用和变量计算类型的多样性使得这成为一个挑战,把所有这些整合成为一个分布式系统是困难的,因为有多变的平台和编程模型(MapReduce,Spark,streaming,user defined,等等)。更进一步这里有更多的数据类型和存储格式的变化。 - - - 尽管存在这些障碍,欧阳讲到团队寻找主线,而且随着不断发展,传统的 SQL 把许多数据密集型的任务连系在一起。“大约有 40% 的数据分析任务已经用 SQL 重写,重写其他的去匹配其能做的” 更进一步,他讲道他们可以享受现有的可以和已经存在的框架匹配的系统比如 Hive,Spark,SQL,和 Impala 。下一步要做的事情就是 SQL 查询加速,百度没有找到 FPGA 更好的硬件。 - - 这些主板,被称为处理单元( 下图中的 PE ),自动地处理关键的 SQL 功能当他们进来,那意味着,一个放弃者在这指出我们现在所能收集到的东西,这正是 FPGA 设计神秘的地方,如果百度在基准测试中得到了下图中的加速,这是有竞争力的信息,我们还要分享所描述的东西,简单来说,FPGA 在数据库中运行,当其收到 SQL 查询的时候,团队设计的软件与之啮合。 - - 欧阳提到了一件事,他们的加速器受限与 FPGA 的带宽不然性能表现本可以更高,在下面的评价中,百度安装了 2 块intl E26230 具有 12 核心,主频 2.0 GHZ,支持 128G 内存的 CPU,SDA 具有 5 个处理单元,(上图中的300MHzFPGA 主板)每个处理核心功能(过滤,排序,聚合,加入,和分组) - - 为了实现 SQL 查询加速,百度将 TPC-DS 基准拆散并且创建了特殊的引擎。叫做处理单元,在基准测试中加速 5 个关键功能,这包括过滤,排序,融合,加入和分组,(我们并没有把这些单词都大写的像 SQL 真的那样),SDA 设置使用卸载模型。和具有多个不同种类的处理单元的的加速卡在 FPGA 逻辑中形成。SQL 功能的类型和每张卡的数量由特定的工作量决定。由于这些查询在百度的系统中执行,用来查询的数据被以列 格式存储到加速卡中(这会使得查询非常快速)而且通过一个统一的 SDA API 和驱动,SQL 查询工作被分发到正确的处理单元而且 SQL 操作实现了加速。 - - SDA 架构采用一种数据流模型,加速单元不支持的操作被退回到数据库系统然后在那里本地运行,比其他任何因素,百度开发的 SQL 加速卡的性能被 FPGA 卡的内存带宽所限制。加速卡跨真个集群机器工作,顺便提一下,但是数据和 SQL 操作如何分发到多个机器的准确原理没有被百度披露。 - - 我们受限与百度分享细节的意愿但是这些基准测试结果是非常感人的,尤其是传输测试,我们将跟随百度 Hot Chips 大会之后的脚步去看看我们是否能得到关于这是如何连接到一起的和如何解决内存带宽瓶颈的细节。 - --------------------------------------------------------------------------------- - -via: http://www.nextplatform.com/2016/08/24/baidu-takes-fpga-approach-accelerating-big-sql/?utm_source=dbweekly&utm_medium=email - -作者:[Nicole Hemsoth][a] -译者:[LinuxBars](https://github.com/LinuxBars) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.nextplatform.com/author/nicole/ -[1]: http://www.nextplatform.com/?s=baidu+deep+learning -[2]: http://www.hotchips.org/wp-content/uploads/hc_archives/hc26/HC26-12-day2-epub/HC26.12-5-FPGAs-epub/HC26.12.545-Soft-Def-Acc-Ouyang-baidu-v3--baidu-v4.pdf From ef26f79b29eaab1dd19d70d41c3b34fc40ca876a Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Fri, 16 Sep 2016 14:16:30 +0800 Subject: [PATCH 0133/2437] Update 20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md --- ...cript Parser that Creates a Language in 200 Lines of Code.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md index df2178ce8e..087a8d3fda 100644 --- a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md +++ b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md @@ -2,7 +2,7 @@ Translating by ucasFL Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code =========== - +Ohm: 一种能用可以用两百行代码创造一种语言的 JavaScript 解释器 Parsers are an incredibly useful software libraries. While conceptually simple, they can be challenging to implement and are often considered a dark art in computer science. In this blog series, I’ll show you why you don’t need to be Harry Potter to master parsers. But bring your wand just in case. We’ll explore a new open source Javascript library called Ohm that makes building parsers easy and easier to reuse. In this series, we use Ohm to recognize numbers, build a calculator, and more. By the end of this series you will have created a complete programming language in under 200 lines of code. This powerful tool will let you do things that you might have thought impossible otherwise. From ced4235a5bfc61106c211d2de5ab42858150dad2 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Fri, 16 Sep 2016 14:23:31 +0800 Subject: [PATCH 0134/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...个致力于“计算”,大学,女权,科技多样性的人 | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 translated/talk/Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 diff --git a/translated/talk/Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 b/translated/talk/Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 new file mode 100644 index 0000000000..daf386bb36 --- /dev/null +++ b/translated/talk/Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 @@ -0,0 +1,49 @@ +![Image by : Photo by Becky Svartström. Modified by Opensource.com. CC BY-SA 4.0](http://opensource.com/sites/default/files/styles/image-full-size/public/images/life/osdc-lead-patriciatorvalds.png) +Image by : Photo by Becky S. Modified by Opensource.com. [CC BY-SA 4.0][1] +图片来源:照片来自Becky Svartström, 修改自Opensource.com. +Patricia Torvalds 并不是那个在 Linux 和开源领域非常有名同样叫做 Torvalds 的人。 +![](http://opensource.com/sites/default/files/images/life-uploads/ptorvalds.png) +18岁的时候,Patricia 已经是一个有多项科技成就并且拥有开源产业经验的女权主义者。紧接着,她把目光投入到在杜克大学地球科学学院工程学第一学年的学习中,以实习生的身份在位于美国奥勒冈州伯特兰市的 Puppet 实验室工作。但不久后,她又直接去了北卡罗纳州的达拉莫,开始秋季学期的大学学习。 +在这次独家采访中,Patricia 表示,使她对计算机科学和工程学感兴趣的(剧透警告:不是她的父亲)原因,包括高中时候对科技的偏爱,女权主义在她的生活中扮演了重要角色以及对科技多样性缺乏的思考。 +![](http://opensource.com/sites/default/files/images/life/Interview%20banner%20Q%26A.png) +###什么东西使你钟情于学习计算机科学和工程学?### +我在科技方面的兴趣的确产生于高中时代。我曾一度想投身于生物学,直到大约大学二年级的时候。大二结束以后,我在波特兰 VA 当网页设计实习生。与此同时,我参加了一个叫做“风险勘探”的工程学课程,在我大二学年的后期,我们把一个水下机器人送到了太平洋。但是,转折点大概是在青年时代的中期当我因被授予“NCWIT Aspiration in Computing"奖而被称作地区和民族英雄的时候出现的。 +这个奖项的获得我感觉确立了自己的兴趣。当然,我认为最重要的部分是我加入到一个所有获奖者在里面的 Facebook 群,里面那些已经获奖的女孩们简直难以置信的相互支持。由于在 XV 和 VA +的工作,我在获奖前就已经确定了致力于计算机科学,但是和这些女孩的交谈更加坚定了这份兴趣,使之更加强壮。再后来,初高年级后期的时候教授 XV 也使我体会到工程学和计算机科学对我来说的确很有趣。 +###你打算学习什么?毕业以后你已经知道自己想干什么了吗?### +我希望主修力学或电气科学以及计算机工程学和计算机科学还有女性学。毕业以后,我希望在一个支持或者创造科技为社会造福的公司工作,或者自己开公司。 +###我的女儿在高中有一门 Visual Basic的编程课。她是整个班上唯一的一个女生,并且以疲倦和痛苦的经历结束这门课程。你的经历是什么样的呢?### +我的高中在高年级的时候开设计算机科学的课程,我也学习了 Visual Basic!这门课不是很糟糕,但我的确是20多个人的班级里唯一的三四个女生之一。其他的计算机课程似乎也有相似的性别比例差异。然而,我所在的高中极其小并且老师对科技非常支持和包容,所以我并没有感到厌倦。希望在未来的一些年里计算机方面的课程会变得更加多样化。 +###你的学校做了哪些促进科技的智举?它们如何能够变得更好?### +我的高中学校给了我们长时间的机会接触到计算机,老师们会突然在不相关的课程上安排科技基础任务,比如有好多次任务,我们必须建一个供社会学习课程使用的网站,我认为这很棒因为它使我们每一个人都能接触到科技。机器人俱乐部也很活跃并且资金充足,但是非常小,我不是其中的成员。学校的科技/工程学项目中一个非常强大的组成部分是一门叫做”风险勘测“的学生教学工程学课程,这是一门需要亲自动手的课程,并且每年处理一个工程学或者计算机科学难题。我和我的一个同学在这儿教授了两年,在课程结束以后,有学生上来告诉我他们对从事工程学或者计算机科学感兴趣。 +然而,我的高中没有特别的关注于让年轻女性加入到这些项目中来,并且在人种上也没有呈现多样化。计算机课程和俱乐部大量的主要成员都是男性白人。这的确应该能够有所改善。 +###在成长过程中,你如何在家使用科技?### +老实说,小的时候,我使用电脑(我的父亲设了一个跟踪装置,当我们上网一个小时就会断线)玩尼奥宠物和或者相似的游戏。我想我本可以毁坏跟踪装置或者在不连接网络的情况下玩游戏,但我没有这样做。我有时候也会和我的父亲做一些小的科学项目,我还记得我和他在电脑终端上打印出”Hello world"无数次。但是大多数时候,我都是和我的妹妹一起玩网络游戏,直到高中的时候才开始学习“计算”。 +###你在高中学校的女权俱乐部很积极,从这份经历中你学到了什么?现在对你来说什么女权问题是最重要的?### +在高中二年级的后期,我和我的朋友一起建立了女权俱乐部。刚开始,我们受到了很多反对和抵抗,并且这从来就没有完全消失过。到我们毕业的时候,女权主义理想已经彻底成为了学校文化的一个部分。我们在学校做的女权主义工作通常是以一些比较直接的方式并集中于像着装要求这样一些问题。 +就我个人来说,我更集中于交叉地带的女权主义,把女权主义运用到缓解其他方面的压迫比如种族歧视和阶级歧视。Facebook 网页《Gurrilla Feminism》是交叉地带女权主义一个非常好的例子并且我从中学到了很多。我目前管理波特兰分支。 +在科技多样性方面女权主义对我也非常重要,尽管作为一名和科技世界有很强联系的上流社会女性,女权主义问题对我产生的影响相比其他人来说非常少,我所涉及的交叉地带女权主义也是同样的。出版集团比如《Model View Culture》非常鼓舞我,并且我很感激 Shanley Kane 所做的一切。 +###你会给想教他们的孩子学习编程的父母什么样的建议?### +老实说,从没有人把我推向计算机科学或者工程学。正如我前面说的,在很长一段时间里,我想成为一名遗传学家。大二结束的那个夏天,我在 VA 当了一个夏季的网页设计实习生,这彻底改变了我之前的想法。所以我不知道我是否能够完整的回答这个问题。 +我的确认为真实的兴趣很重要。如果在我12岁的时候,我的父亲让我坐在一台电脑前,教我安装网站服务器,我认为我不会对计算机科学感兴趣。相反,我的父母给了我很多可以支配的自由让我去做自己想做的事情,绝大多数时候是为我的尼奥宠物游戏编糟糕的HTML网站。比我小的妹妹们没有一个对工程学或计算机科学感兴趣,我的父母也不在乎。我感到很幸运的是我的父母给了我和我的妹妹们鼓励和资源去探索自己的兴趣。 +仍然,在我成长过程中我也常说未来要和我的父亲做同样的职业,尽管我还不知道我父亲是干什么的,只知道他有一个很酷的工作。另外,中学的时候有一次,我告诉我的父亲这件事,然后他没有发表什么看法只是告诉我高中的时候不要想这事。所以我猜想这从一定程度上刺激了我。 +###对于开源社区的领导者们,你有什么建议给他们来吸引和维持更加多样化的贡献者?### +我实际上在开源社区不是特别积极和活跃。和女性讨论“计算”我感觉更舒服。我是“NCWIT Aspirarion in Computing"网站的一名成员,这是我对科技有持久兴趣的一个重要方面,同样也包括Facebook群”Ladies Storm Hackathons". +我认为对于吸引和维持多种多样有天赋的贡献者,安全空间很重要。我过去看到在一些开源社区有人发表关于女性歧视和种族主义的评论,人们指出这一问题随后该人被解职。我认为要维持一个专业的社区必须就骚扰事件和不正当行为有一个很高的标准。当然,人们已经有或者将有很多的选择关于在开源社区或其他任何社区能够表达什么。然而,如果社区领导人真的想吸引和维持多种多样有天赋的人员,他们必须创造一个安全的空间并且把社区成员维持在很高的标准上。我也认为一些一些社区领导者不明白多样性的价值。很容易说明科技象征着精英社会,并且很多人被科技忽视的原因是他们不感兴趣,这一问题在很早的准备过程中就提出了。他们争论如果一个人在自己的工作上做得很好,那么他的性别或者民族还有性取向这些情况都变得不重要了。这很容易反驳,但我不想为错误找理由。我认为多样性缺失是一个错误,我们应该为之负责并尽力去改善这件事。 +-------------------------------------------------------------------------------- + +via: http://opensource.com/life/15/8/patricia-torvalds-interview + +作者:[Rikki Endsley][a] +译者:[ucasFL](https://github.com/ucasFL) +校对:[校对者ID](https://github.com/校对者ID) +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://opensource.com/users/rikki-endsley +[1]:https://creativecommons.org/licenses/by-sa/4.0/ +[2]:https://puppetlabs.com/ +[3]:https://www.aspirations.org/ +[4]:https://www.facebook.com/guerrillafeminism +[5]:https://modelviewculture.com/ +[6]:https://www.aspirations.org/ +[7]:https://www.facebook.com/groups/LadiesStormHackathons/ From 9b3be3454f83143c6b02f65bc2dcbcdf7cfc07c4 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Fri, 16 Sep 2016 15:45:39 +0800 Subject: [PATCH 0135/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=EF=BC=88=E6=A0=87=E9=A2=98=E4=BF=AE=E6=94=B9=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...人 => Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename translated/talk/{Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 => Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化} (91%) diff --git a/translated/talk/Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 b/translated/talk/Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化 similarity index 91% rename from translated/talk/Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 rename to translated/talk/Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化 index daf386bb36..5290015524 100644 --- a/translated/talk/Patricia Torvalds: 一个致力于“计算”,大学,女权,科技多样性的人 +++ b/translated/talk/Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化 @@ -4,7 +4,7 @@ Image by : Photo by Becky S. Modified by Opensource.com. [CC BY-SA 4.0][1] Patricia Torvalds 并不是那个在 Linux 和开源领域非常有名同样叫做 Torvalds 的人。 ![](http://opensource.com/sites/default/files/images/life-uploads/ptorvalds.png) 18岁的时候,Patricia 已经是一个有多项科技成就并且拥有开源产业经验的女权主义者。紧接着,她把目光投入到在杜克大学地球科学学院工程学第一学年的学习中,以实习生的身份在位于美国奥勒冈州伯特兰市的 Puppet 实验室工作。但不久后,她又直接去了北卡罗纳州的达拉莫,开始秋季学期的大学学习。 -在这次独家采访中,Patricia 表示,使她对计算机科学和工程学感兴趣的(剧透警告:不是她的父亲)原因,包括高中时候对科技的偏爱,女权主义在她的生活中扮演了重要角色以及对科技多样性缺乏的思考。 +在这次独家采访中,Patricia 表示,使她对计算机科学和工程学感兴趣的(剧透警告:不是她的父亲)原因,包括高中时候对科技的偏爱,女权主义在她的生活中扮演了重要角色以及对科技多元化缺乏的思考。 ![](http://opensource.com/sites/default/files/images/life/Interview%20banner%20Q%26A.png) ###什么东西使你钟情于学习计算机科学和工程学?### 我在科技方面的兴趣的确产生于高中时代。我曾一度想投身于生物学,直到大约大学二年级的时候。大二结束以后,我在波特兰 VA 当网页设计实习生。与此同时,我参加了一个叫做“风险勘探”的工程学课程,在我大二学年的后期,我们把一个水下机器人送到了太平洋。但是,转折点大概是在青年时代的中期当我因被授予“NCWIT Aspiration in Computing"奖而被称作地区和民族英雄的时候出现的。 @@ -27,9 +27,9 @@ Patricia Torvalds 并不是那个在 Linux 和开源领域非常有名同样叫 老实说,从没有人把我推向计算机科学或者工程学。正如我前面说的,在很长一段时间里,我想成为一名遗传学家。大二结束的那个夏天,我在 VA 当了一个夏季的网页设计实习生,这彻底改变了我之前的想法。所以我不知道我是否能够完整的回答这个问题。 我的确认为真实的兴趣很重要。如果在我12岁的时候,我的父亲让我坐在一台电脑前,教我安装网站服务器,我认为我不会对计算机科学感兴趣。相反,我的父母给了我很多可以支配的自由让我去做自己想做的事情,绝大多数时候是为我的尼奥宠物游戏编糟糕的HTML网站。比我小的妹妹们没有一个对工程学或计算机科学感兴趣,我的父母也不在乎。我感到很幸运的是我的父母给了我和我的妹妹们鼓励和资源去探索自己的兴趣。 仍然,在我成长过程中我也常说未来要和我的父亲做同样的职业,尽管我还不知道我父亲是干什么的,只知道他有一个很酷的工作。另外,中学的时候有一次,我告诉我的父亲这件事,然后他没有发表什么看法只是告诉我高中的时候不要想这事。所以我猜想这从一定程度上刺激了我。 -###对于开源社区的领导者们,你有什么建议给他们来吸引和维持更加多样化的贡献者?### +###对于开源社区的领导者们,你有什么建议给他们来吸引和维持更加多元化的贡献者?### 我实际上在开源社区不是特别积极和活跃。和女性讨论“计算”我感觉更舒服。我是“NCWIT Aspirarion in Computing"网站的一名成员,这是我对科技有持久兴趣的一个重要方面,同样也包括Facebook群”Ladies Storm Hackathons". -我认为对于吸引和维持多种多样有天赋的贡献者,安全空间很重要。我过去看到在一些开源社区有人发表关于女性歧视和种族主义的评论,人们指出这一问题随后该人被解职。我认为要维持一个专业的社区必须就骚扰事件和不正当行为有一个很高的标准。当然,人们已经有或者将有很多的选择关于在开源社区或其他任何社区能够表达什么。然而,如果社区领导人真的想吸引和维持多种多样有天赋的人员,他们必须创造一个安全的空间并且把社区成员维持在很高的标准上。我也认为一些一些社区领导者不明白多样性的价值。很容易说明科技象征着精英社会,并且很多人被科技忽视的原因是他们不感兴趣,这一问题在很早的准备过程中就提出了。他们争论如果一个人在自己的工作上做得很好,那么他的性别或者民族还有性取向这些情况都变得不重要了。这很容易反驳,但我不想为错误找理由。我认为多样性缺失是一个错误,我们应该为之负责并尽力去改善这件事。 +我认为对于吸引和维持多种多样有天赋的贡献者,安全空间很重要。我过去看到在一些开源社区有人发表关于女性歧视和种族主义的评论,人们指出这一问题随后该人被解职。我认为要维持一个专业的社区必须就骚扰事件和不正当行为有一个很高的标准。当然,人们已经有或者将有很多的选择关于在开源社区或其他任何社区能够表达什么。然而,如果社区领导人真的想吸引和维持多元化有天赋的人员,他们必须创造一个安全的空间并且把社区成员维持在很高的标准上。我也认为一些一些社区领导者不明白多元化的价值。很容易说明科技象征着精英社会,并且很多人被科技忽视的原因是他们不感兴趣,这一问题在很早的准备过程中就提出了。他们争论如果一个人在自己的工作上做得很好,那么他的性别或者民族还有性取向这些情况都变得不重要了。这很容易反驳,但我不想为错误找理由。我认为多元化缺失是一个错误,我们应该为之负责并尽力去改善这件事。 -------------------------------------------------------------------------------- via: http://opensource.com/life/15/8/patricia-torvalds-interview From b142c98b7ea6bf84bb526d0e218da226855c33c6 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Fri, 16 Sep 2016 16:11:52 +0800 Subject: [PATCH 0136/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=EF=BC=88=E6=96=87=E4=BB=B6=E5=90=8D=E4=BF=AE=E6=94=B9=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Patricia Torvalds on computing, college, feminism and increasing diversity in tech} | 1 + 1 file changed, 1 insertion(+) rename translated/talk/{Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化 => Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech} (99%) diff --git a/translated/talk/Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化 b/translated/talk/Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech similarity index 99% rename from translated/talk/Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化 rename to translated/talk/Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech index 5290015524..e650dfde93 100644 --- a/translated/talk/Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义与科技界的多元化 +++ b/translated/talk/Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech @@ -1,3 +1,4 @@ +Torvalds2.0: Patricia Torvalds 谈“计算”,大学,女权主义和科技界的多元化 ![Image by : Photo by Becky Svartström. Modified by Opensource.com. CC BY-SA 4.0](http://opensource.com/sites/default/files/styles/image-full-size/public/images/life/osdc-lead-patriciatorvalds.png) Image by : Photo by Becky S. Modified by Opensource.com. [CC BY-SA 4.0][1] 图片来源:照片来自Becky Svartström, 修改自Opensource.com. From 668df8878b3a9b13709e44c7d14b39337e6590f1 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Fri, 16 Sep 2016 22:49:25 +0800 Subject: [PATCH 0137/2437] Update 20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md --- ...Creates a Language in 200 Lines of Code.md | 189 ++++++------------ 1 file changed, 60 insertions(+), 129 deletions(-) diff --git a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md index 087a8d3fda..810ba28c2a 100644 --- a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md +++ b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md @@ -1,64 +1,41 @@ Translating by ucasFL -Ohm: JavaScript Parser that Creates a Language in 200 Lines of Code + =========== -Ohm: 一种能用可以用两百行代码创造一种语言的 JavaScript 解释器 -Parsers are an incredibly useful software libraries. While conceptually simple, they can be challenging to implement and are often considered a dark art in computer science. In this blog series, I’ll show you why you don’t need to be Harry Potter to master parsers. But bring your wand just in case. - -We’ll explore a new open source Javascript library called Ohm that makes building parsers easy and easier to reuse. In this series, we use Ohm to recognize numbers, build a calculator, and more. By the end of this series you will have created a complete programming language in under 200 lines of code. This powerful tool will let you do things that you might have thought impossible otherwise. - -### Why Parsers are Hard - -Parsers are useful. There are lots of times you might need a parser. A new file format might come along that you need to process and no one else has written a library for it yet. Or maybe you find files in an old file format and the existing parsers aren’t built for the platform you need. I’ve seen this happen over and over. Code will come and go but data is forever. - -Fundamentally parsers are simple: just transform one data structure into another. So why does it feel like you need to be Dumbledore to figure them out? - -The challenge that parsers have historically been surprisingly difficult to write, and most of the existing tools are old and assume a fair amount of arcane computer science knowledge. If you took a compilers class in college the textbook may well have techniques from the 1970s. Fortunately, parser technology has improved a great deal since then. - -Typically a parser is created by defining what you want to parse using a special syntax called a formal grammar. Then you feed this into several tools like Bison and Yacc which generate a bunch of C code that you the need to modify or link into whatever programming language you are actually writing in. The other option is to manually write a parser in your preferred language, which is slow and error prone. That’s a lot of extra work before you get to actually use the parser. - -Imagine if your description of the thing you wanted to parse, the grammar, was also the parser? What if you could just run the grammar directly, then add hooks only where you want? That’s what Ohm does. - -### Introducing Ohm - -[Ohm][1] is a new kind of parsing system. While it resembles the grammars you may have seen in text books it’s a lot more powerful and easier to use. With Ohm you write your format definition in a very flexible syntax in a .ohm file, then attach semantic meaning to it using your host language. For this blog we will use JavaScript as the host language. - -Ohm is based on years of research into making parsers easier and more flexible. VPRI’s [STEPS program][2] (pdf) created many custom languages for specific tasks (like a fully parallelizable graphics renderer in 400 lines of code!) using Ohm’s precursor [OMeta][3]. - -Ohm has many interesting features and notations, but rather than explain them all I think we should just dive in and build something. - -### Parsing Integers - -Let’s parse some numbers. It seems like this would be easy. Just look for adjacent digits in a string of text; but let’s try to handle all forms of numbers: Integers and floating point. Hex and octal. Scientific notation. A leading negative digit. Parsing numbers is easy. Doing it right is hard - -To build this code by hand would be difficult and buggy, with lots of special cases which sometimes conflict with each other. A regular expression could probably do it, but would be ugly and hard to maintain. Let’s do it with Ohm instead. - -Every parser in Ohm involves three parts: the grammar, the semantics, and the tests. I usually pick part of the problem and write tests for it, then build enough of the grammar and semantics to make the tests pass. Then I pick another part of the problem, add more tests, update the grammar and semantics, while making sure all of the tests continue to pass. Even with our new powerful tool, writing parsers is still conceptually complicated. Tests are the only way to build parsers in a reasonable manner. Now let’s dig in. - -We’ll start with an integer number. An integer is composed of a sequences of digits next to each other. Let’s put this into a file called grammar.ohm: - +Ohm: 一种可以用两百行代码创造一种语言的 JavaScript 解释器 +解释器是一种非常有用的软件库。从概念上简单的说,它们的实现很有挑战并且在计算机科学中经常被认为是暗黑艺术。在这个系列的博文中,我会向你们展示为什么你不需要成为哈利波特就能够很好的控制解释器。但是为了以防万一带上你的魔杖。 +我们将探索一种叫做 Ohm 的新的开源库,它使得搭建解释器很简单并且更加容易再利用。在这个系列里,我们使用 Ohm 去识别数字,构建计算器等等。在这个系列的最后你将已经用少于 200 行的代码发明了一种完整的编程语言。这个强大的工具将让你能够做一些你可能过去认为不可能的事情。 +###为什么解释器很困难? +解释器非常有用。在很多时候你可能需要一个解释器。一种新的文件格式可能出现,你需要去处理但还没有人为它写了一个库;或者你发现了一种老格式的文件但是现存的解释器不能构建你需要的平台。我已经看到这样的事发生无数次。代码会来来去去但数据却是永恒的。 +基础的解释器很简单:只是把一个数据结构转化成另一个。所以为什么感觉你需要成为 邓布利多【魔法师】才能够把它们做出来。 +解释器的一些历史性的挑战是很难写,绝大多数工具很老并且假设了大量晦涩难懂的计算机科学知识。如果你在大学里上过编译器课程那么课本里可能也有从 1970 年以来的技术。幸运的是,解释器技术从那时候起已经提高了很多。 +代表性地,解释器是通过使用一种叫作形式语法的特殊语法来定义你想要解析的东西这样发明的,然后你需要把它放入像 Bison 和 Yacc 的工具中,这些工具能够产生一堆你需要修改的 C 代码或者链接到你实际写入额的编程语言。另外的选择是用你更喜欢的语言亲自动手写一个解释器,这很慢且很容易出错,在你能够真正使用它之前还有许多额外的工作。 +想像一下,是否你关于你想要解析的东西的语法描述也是解释器?如果你能够仅仅直接运行语法,然后在你需要的地方增加挂钩,那是什么?那就是 Ohm 所做的事。 +###解释器简介 +[Ohm][1]是一种新的解析系统。它类似你可能已经在课本里面看到的语法并且它更强大,使用起来更简单。通过 Ohm, 你能够使用一种灵活的语法以 .ohm 文件格式来写格式定义,然后使用你的宿主语言把语义加入到里面。在这篇博文里,我们将用 JavaScript 作为宿主语言。 +Ohm 建立在一个为制造更简单、更灵活的解释器的一个多年调查基础之上。VPRI 的 [STEPS program](pdf) 使用 Ohm 的前驱为许多特殊的工作创造了专门的语言(比如一个有 400 行代码的平行制图描绘器)[Ometa][3]. +Ohm 有许多有趣的特点和符号,但是不是要全部解释它们,我认为我们应该只需投入其中并构建一些东西。 +###解析整数 +让我们来解析一些数字。这看起来会很简单,只需在一个文本串中寻找毗邻的数字,但是让我们尝试去处理所有形式的数字:整数和浮点数,十六进制数和八进制数,科学计数,负数。解析数字很简单,正确解析却很难。 +亲自构建这个代码将会很困难,会有很多故障,会伴随有许多特殊的情况,比如有时会相互矛盾。 +用 Ohm 构建的解释器涉及三个部分:语法、语义和测试。我通常挑选一个问题的一部分为它写测试,然后构建足够的语法和语义来使测试通过。然后我再挑选问题的另一部分,增加更多的测试,更新语法和语义,从而确保所有的测试能够持续通过。即使我们有了新的强大的工具,写解释器从概念上来说依旧很困难。测试是用一种合理的方式来构建解释器的唯一方法。现在,让我们开始工作。 +我们将从整数开始。一个整数由一系列相互毗邻的数字组成。让我们把下面的内容放入一个叫做 grammar.ohm 的文件中: ``` CoolNums { // just a basic integer Number = digit+ } ``` - -This creates a single rule called Number which matches one or more digits. The + means one or more, just like in a regular expression. This rule will match if there is one digit or more than one digit. It won’t match if there are zero digits or something something other than a digit. A digit is defined as the characters for the numbers 0 to 9. digit is also a rule like Number is, but it’s one of Ohm’s built in rules so we don’t have to define it ourselves. We could override if it we wanted to but that wouldn’t make sense in this case. After all we don’t plan to invent a new form of number (yet!) - -Now we can read in this grammar and process it with the Ohm library. - -Put this into test1.js - +这创造了一条撮合一个或多个数字叫作 Number 的单一规则。+ 意味着一个或更多,就像一个常规的表达。当有一个或更多的数字时,这个规则将会撮合它们,如果没有数字或者有一些不是数字的东西将不会撮合。一个数字定义成从 0 到 9 其中的一个字符。数字也是像 Number 一样的规则,但是它是 Ohm 的其中一条构建规则因此我们不需要去定义它。我们可以推翻它如果我们想的话但在这时候这没有任何意义,毕竟我们不打算去发明一种新的数。 +现在,我们可以读入这个语法并用 Ohm 库来运行它。 +把它放入 test1.js ``` var ohm = require('ohm-js'); var fs = require('fs'); var assert = require('assert'); var grammar = ohm.grammar(fs.readFileSync('src/blog_numbers/syntax1.ohm').toString()); ``` - -The ohm.grammar call will read in the file and parse it into a grammar object. Now we can add semantics. Add this to your Javascript file: - +Ohm 的语法调用将把文件读入并解释成一个语法对象。现在我们可以增加一些语义。把下面内容增加到你的 JavaScript 文件中: ``` var sem = grammar.createSemantics().addOperation('toJS', { Number: function(a) { @@ -66,16 +43,11 @@ var sem = grammar.createSemantics().addOperation('toJS', { } }); ``` - -This creates a set of semantics called sem with the operation toJS. The semantics are essentially a bunch of functions matching up to each rule in the grammar. Each function will be called when the corresponding rule in the grammar is parsed. The Number function above will be called when the Number rule in the grammar is parsed. The grammar defines what chunks are in the language. The semantics define what to do with chunks once they’ve been parsed. - -Our semantics functions can do anything we want, such as print debugging information, create objects, or recursively call toJS on any sub-nodes. In this case we just want to convert the matched text into a real Javascript integer. - -All semantic functions have an implicit this object with some useful properties. The source property represents the part of the input text that matches this node. this.sourceString is the matched input as a string. Calling the built in JavaScript function parseInt turns this string to a number. The 10 argument to parseInt tells JavaScript that we are giving it a number in base ten. If we leave it out then JS will assume it’s base 10 anyway, but I’ve included it because later on we will support base 16 (hex) numbers, so it’s good to be explicit. - -Now that we have some semantics, let’s actually parse something to see if our parser works. How do we know our parser works? By testing it. Lots and lots of testing. Every possible edge case needs a test. - -With the standard assert API, here is a test function which matches some input then applies our semantics to it to turn it into a number, then compares the number with the expected input. +这创造了一系列叫作 sem with the operation to JS[伴有 JavaScript 操作的语义] 的语义。这些语义至关重要,一群函数和语法中的每一条规则相匹配。一个函数将会被调用当与它相匹配的语法规则被解析时。上面的 Number 函数将会被调用当语法中的 Number 规则被解析时。语法定义在语言中 chunks[大块] 是什么,语义定义当 chunks[大块] 被解析时应该做什么。 +语义函数能够做我们想做的任何事,比如打印初故障信息,创造对象,或者递归调用 toJS 作用于任何子节点。此时我们仅仅想把文本转换成真正的 JavaScript 整数。 +所有的语义函数有一个包含一些有用性质的暗含对象。源属性代表输入文本和这个节点相匹配。这个 sourceString[源串] 是一个匹配输入串,调用构建在 JavaScript 中的parseInt 函数会把这个串转换成一个数。parseInt 中 10 这个参数告诉 JavaScript 我们输入的是一个以 10 为基底的数。如果少了这个参数, JavaScript 也会假定以 10 为基底,但是我们把它包含在里面因为后面我们将支持以 16 为基底的数,所以使之明确比较好。 +既然我们有一些语法,让我们来实际解析一些东西看一看我们的解释器是否能够工作。如果知道我们的解释器工作?通过测试它,许多许多的测试,每一个边缘情况都需要一个测试。 +伴随标准断言 API,有一个测试函数能够匹配一些输入并运用我们的语义把它转换成一个数,然后比较转换生成的数和我们期望的输入。 ``` function test(input, answer) { var match = grammar.match(input); @@ -85,19 +57,14 @@ With the standard assert API, here is a test function which matches some input t console.log('success = ', result, answer); } ``` - -That’s it. Now we can write a bunch of tests for different numbers. If the match fails then our script will throw an exception. If not it will print success. Let’s try it out. Add this to the script - +这个函数就是上面这个。现在我们能够为不同的数写一堆测试。如果匹配失败我们的脚本将会丢弃一个例外。如果不能打印成功,让我们尝试一下,把下面这些内容加入到脚本中: ``` test("123",123); test("999",999); test("abc",999); ``` - -Then run the script with node test1.js - -Your output should look like this: - +然后用节点 test.js 运行脚本 +你的输出应该是这样: ``` success = 123 123 success = 999 999 @@ -106,13 +73,9 @@ input failed to match abcLine 1, col 1: ^ Expected a digit ``` - -Cool. The first two succeed and the third one fails, as it should. Even better, Ohm automatically gave us a nice error message pointing to the match failure. - -### Floating Point - -Our parser works, but it doesn’t do anything very interesting. Let’s extend it to parse both integers and floating point numbers. Change the grammar.ohm file to look like this: - +真酷。正如理所当然的那样,前两个成功了,第三个失败了。更好的是,Ohm 自动给了我们一个很棒的错误信息指出匹配失败。 +###浮点数 +我们的解释器工作了,但是它不能做任何非常有趣的事。让我们把它扩展成既能解析整数又能解析浮点数。改变 grammar.ohm 文件使它看起来像下面这样: ``` CoolNums { // just a basic integer @@ -121,11 +84,8 @@ CoolNums { float = digit+ "." digit+ } ``` - -This changes the Number rule to point to either a float or an int. The | means or. We read this as “a Number is composed of a float or an int.” Then int is defined as digit+ and float is defined as digit+ followed by a period followed by another digit+. This means there must be at least one digit before the period and at least one after. If there is not a period then it won’t be a float at all, so int will match instead. - -Now let’s go look at our semantic actions again. Since we now have new rules we need new action functions: one for int and one for float. - +这把 Number 规则改变成指向一个浮点数或者一个整数。我的意思是,我们把这读成"一个 Number 由一个浮点数或者一个整数构成。”然后整数定义成 digit+, 浮点数定义成 digit+ 后面跟着一个句号然后再跟着另一个 digit+. 这意味着在句号前和句号后都至少要有一个数字。如果一个数中没有一个句号那么它就不是一个浮点数,因此就是一个整数。 +现在,让我们再次看一下我们的语义作用。由于我们现在有了新的规则所以我们需要新的作用函数:一个作为整数的,一个作为浮点数的。 ``` var sem = grammar.createSemantics().addOperation('toJS', { Number: function(a) { @@ -141,19 +101,13 @@ var sem = grammar.createSemantics().addOperation('toJS', { } }); ``` - -There’s two things to note here. First, int and float and Number all have matching grammar rules and functions. However, the action for Number no longer does anything interesting. It receives the child node ‘a’ and returns the result of toJS on the child. In other words the Number rule simply returns whatever its child rule matched. Since this is the default behavior of any rule in Ohm we can actually just leave the Number action out. Ohm will do it for us. - -Second, int has one argument a while float has three: a, b, and c. This is because of the rule’s arity. Arity means how many arguments a rule has. If we look back at the grammar, the rule for float is - +这里有两件事情需要注意。首先,整数,浮点数和数都有相匹配的语法规则和函数。然而,针对数的作用不再有任何意义。它接收子节点 'a' 然后通过子节点返回 toJS 的结果。换句话说,Number 规则简单的返回相匹配的子规则。由于这是在 Ohm 中任何规则的默认行为,因此实际上我们不用去考虑 Number 的作用,Ohm 会替我们来做这件事。 +第二,整数有一个参数然而浮点数有三个:a, b, 和 c. 这是由于规则的参数数量。参数数量意味着一个规则里面有多少参数。如果我们回过头去看语法,浮点数的规则是: ``` float = digit+ "." digit+ ``` - -The float rule is defined by three parts: the first digit+, the "." and the second digit+. All three of those parts will be passed as parameters to the action function for float. Thus float must have three arguments or else the Ohm library will give us an error. In this case we don’t care about the arguments because we will just grab the input string directly, but we still need the arguments listed to avoid compiler errors. Later on we will actually use some of these parameters. - -Now we can add a few more tests for our new floating point number support. - +浮点数规则通过三个部分来定义:第一个 digit+, '.', 还有第二个 digit+. 这三个部分都会作为参数传递给浮点数的作用函数。因此浮点数必须有三个参数否则 Ohm 库给出一个错误。在这种情况下我们不用在意参数因为我们仅仅直接攫取了输入串,但是我们仍然需要列表的参数来回避编译器错误。后面我们将实际使用其中一些参数。 +现在我们可以为新的浮点数支持添加更多的测试。 ``` test("123",123); test("999",999); @@ -162,15 +116,10 @@ test('123.456',123.456); test('0.123',0.123); test('.123',0.123); ``` - -Note that the last test will fail. A floating point number must begin with a digit, even if it’s just zero. .123 is not valid, and in fact the real JavaScript language has the same rule. - -### Hexadecimal - -So now we have integers and floats but there’s a few other number syntaxes that might be good to support: hexadecimal and scientific notation. Hex numbers are integers in base sixteen. The digits can be from 0 to 9 and A to F. Hex is often used in computer science when working with binary data because you can represent 0 through 255 exactly with only two digits. - -In most C derived programming languages (including JavaScript) hex numbers are preceded by `0x` to indicate to the compiler that what follows is a hexadecimal number. To support hex numbers in our parser we just need to add another rule. - +注意最后一个测试将会失败。一个浮点数必须以一个数开始,即使它仅仅是 0, .123 不是有效的,实际上真正的 JavaScript 语言有相同的规则。 +###十六进制数 +现在我们已经有了整数和浮点数,但是有一些其他的数的语法可能能够很好的支持:十六进制数和科学计数。十六进制数是是以 16 为基底2的数。十六进制数的数字能从 0 到 9 和从 A 到 F. 十六进制数经常用在计算机科学中当用二进制数据工作时,因为你可以仅仅使用两个数字表示 0 到 255 的数。 +在绝大多数源自 C 的编程语言(包括 JavaScript), 十六进制数通过在前面加上 ‘0x' 来向编译器表明后面跟的是一个十六进制数。为了让我们的解释器支持十六进制数,我们只需要添加另一条规则。 ``` Number = hex | float | int int = digit+ @@ -178,67 +127,49 @@ In most C derived programming languages (including JavaScript) hex numbers are p hex = "0x" hexDigit+ hexDigit := "0".."9" | "a".."f" | "A".."F" ``` - -I’ve actually added two rules. `hex` says that a hex number is the string `0x` followed by one or more `hexDigits`. A `hexDigit` is any letter from 0 to 9, or a to f, or A to F (covering both upper and lower case). I also modified Number to recognize hex as another possible option. Now we just need another action rule for hex. - +我实际上已经增加了两条规则。'hex' 表明十六进制数是一个 'ox' 后面一个或多个 ’hexDigits'[十六进制数子] 的串。一个 'hexDigit' 是从 0 到 9, 或从 a 到 f, 或 A 到 F(包扩大写和小写的情况)的一个字符。我也修改了 Number 规则来识别十六进制数作为其他可能的选择。现在我们只需要另一条针对十六进制数的作用规则。 ``` hex: function(a,b) { return parseInt(this.sourceString,16); } ``` +注意到,在这种情况下,我们把 '16' 作为基数传递给 'parseInt', 因为我们希望 JavaScript 知道这是一个十六进制数。 -Notice that in this case we are passing `16` as the radix to `parseInt` because we want JavaScript to know that this is a hexadecimal number. - -I skipped over something important to notice. The rule for `hexDigit` looks like this. - +我略过了一些很重要需要注意的事。针对 'hexDigit' 的规则像下面这样: ``` hexDigit := "0".."9" | "a".."f" | "A".."F" ``` - -Notice that I used `:=` instead of `=`. In Ohm, the `:=` is used when you are overriding a rule. It turns out Ohm already has a default rule for `hexDigit`, just as it does for `digit`, `space` and a bunch of others. If I had used = then Ohm would have reported an error. This is a check so I can’t override a rule unintentionally. Since our new hexDigit rule is actually the same as Ohm’s built in rule we can just comment it out and let Ohm do it. I left the rule in just so we can see what’s really going on. - +注意我使用的是 ':=' 而不是 '='. 在 Ohm 中,'=' 是当你需要推翻一条规则的时候使用。证明是 Ohm 已经有了针对 'hexDigit' 的默认规则,就像针对 'digit', 'space' 等一堆其他的东西。如果我使用了 '=', Ohm 将会报告一个错误。这是一个检查从而我不能无意识的推翻一个规则。由于新的 hexDigit 规则和 Ohm 的构建规则一样,所以我们可以仅仅对它添加注释然后让 Ohm 来实现它。我留下这个规则仅仅是因为这样我们可以看到它实际上是如何进行的。 Now we can add some more tests and see that our hex digits really work: - +现在,我们可以添加更多的测试然后看到十六进制数真的工作。 ``` test('0x456',0x456); test('0xFF',255); ``` - -### Scientific Notation - -Finally let’s support scientific notation. This is for very large or small numbers like 1.8 x 10^3 In most programming languages numbers in scientific notation would be written as 1.8e3 for 18000 or 1.8e-3 for .0018. Let’s add another couple of rules to support this exponent notation. - +###科学计数 +最后,让我们来支持科学计数。科学计数是针对非常大或非常小的数比如 1.8×10^3, 在大多数编程语言中,科学计数法表示的数会写成这样:1.8e3 表示 18000, 或者 1.8e-3 表示 .018. 让我们增加另外一对规则来支持这个指数表示: ``` float = digit+ "." digit+ exp? exp = "e" "-"? digit+ ``` - -This adds a the exp rule to the end of the float rule with a question mark. The ? means zero or one, so exp is optional but there can’t be more than one. Adding the exp rule also changes the arity of the float rule, so we need to add another argument to the float action, even if we don’t use it. - +上面增加了一个指数规则通过在浮点数规则末尾加上一个 '?'. '?' 表示 0 或 1,所以指数是可选择的但是不能超过一个。增加指数规则也改变了浮点数规则的参数数量,所以我们需要为浮点数作用增加又一个参数,即使我们不使用它。 ``` float: function(a,b,c,d) { console.log("doing float", this.sourceString); return parseFloat(this.sourceString); }, ``` - -And now our new tests can pass: - +现在我们的测试可以通过了: ``` test('4.8e10',4.8e10); test('4.8e-10',4.8e-10); ``` - -### Conclusion - -Ohm is a great tool for building parsers because it’s easy to get started and you can incrementally add to it. It also has other great features that I didn’t cover today, like a debugging visualizer and sub-classing. - -So far we have used Ohm to translate character strings into JavaScript numbers, and often Ohm is used for this very purpose: converting one representation to another. However, Ohm can be used for a lot more. By putting in a different set of semantic actions you can use Ohm to actually process and calculate things. This is one of Ohm’s magic features. A single grammar can be used with many different semantics. - -In the next article of this series I’ll show you how to not just parse numbers but actually evaluate math expressions like `(4.8+5 * (238-68)/2)`, just like a real calculator. - -Bonus challenge: Can you extend the grammar with support for octal numbers? These are numbers in base 8 and can be represented with only the digits 0 to 7, preceded by a zero and the letter o. See if you are right with these test cases. Next time I’ll show you the answer. - +###结论 +Ohm 是构建解释器的一个很棒的工具,因为它很容易开始并且你可以递增的增加规则。Ohm 也还有其他我今天没有写到的很棒的特点,比如故障观察仪和子类化。 +s. +到目前为止,我们已经使用 Ohm 来把字符串翻译成 JavaScript 数,并且 Ohm 经常由于需要把一个表示转化成另一个这一目的而使用。然而,Ohm 还有更多的用途。通过放入以系列不同的语义作用你可以使用 Ohm 来真正处理和计算东西。一个单独的语法可以被许多不同的语义使用,这是 Ohm 其中一个不可思议的特点。 +在这个系列的下一篇文章中,我将向你们展示如何计算像(4.85 + 5 * (238 - 68)/2) 这样的数学表达式,不仅仅是解析数。 +额外的挑战:你能够扩展语法来支持八进制数吗?这些以 8 为基底的数能够只用 0 到 7 这几个数字来表示,前面加上一个数字 0 或者字母 o. 看看针对下面这些测试情况是够正确。下次我将给出答案。 ``` test('0o77',7*8+7); test('0o23',0o23); @@ -252,7 +183,7 @@ test('0o23',0o23); via: https://www.pubnub.com/blog/2016-08-30-javascript-parser-ohm-makes-creating-a-programming-language-easy/?utm_source=javascriptweekly&utm_medium=email 作者:[Josh Marinacci][a] -译者:[译者ID](https://github.com/译者ID) +译者:[ucasFL](https://github.com/ucasFL) 校对:[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 From 241c446721a121765f828f1e257af3c70d4d2d41 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Fri, 16 Sep 2016 22:53:42 +0800 Subject: [PATCH 0138/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...hat Create a Language in 200 Lines of Code | 192 ++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code diff --git a/translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code b/translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code new file mode 100644 index 0000000000..70dd8091a4 --- /dev/null +++ b/translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code @@ -0,0 +1,192 @@ +Ohm: 一种可以用两百行代码创造一种语言的 JavaScript 解释器 +解释器是一种非常有用的软件库。从概念上简单的说,它们的实现很有挑战并且在计算机科学中经常被认为是暗黑艺术。在这个系列的博文中,我会向你们展示为什么你不需要成为哈利波特就能够很好的控制解释器。但是为了以防万一带上你的魔杖。 +我们将探索一种叫做 Ohm 的新的开源库,它使得搭建解释器很简单并且更加容易再利用。在这个系列里,我们使用 Ohm 去识别数字,构建计算器等等。在这个系列的最后你将已经用少于 200 行的代码发明了一种完整的编程语言。这个强大的工具将让你能够做一些你可能过去认为不可能的事情。 +###为什么解释器很困难? +解释器非常有用。在很多时候你可能需要一个解释器。一种新的文件格式可能出现,你需要去处理但还没有人为它写了一个库;或者你发现了一种老格式的文件但是现存的解释器不能构建你需要的平台。我已经看到这样的事发生无数次。代码会来来去去但数据却是永恒的。 +基础的解释器很简单:只是把一个数据结构转化成另一个。所以为什么感觉你需要成为 邓布利多【魔法师】才能够把它们做出来。 +解释器的一些历史性的挑战是很难写,绝大多数工具很老并且假设了大量晦涩难懂的计算机科学知识。如果你在大学里上过编译器课程那么课本里可能也有从 1970 年以来的技术。幸运的是,解释器技术从那时候起已经提高了很多。 +代表性地,解释器是通过使用一种叫作形式语法的特殊语法来定义你想要解析的东西这样发明的,然后你需要把它放入像 Bison 和 Yacc 的工具中,这些工具能够产生一堆你需要修改的 C 代码或者链接到你实际写入额的编程语言。另外的选择是用你更喜欢的语言亲自动手写一个解释器,这很慢且很容易出错,在你能够真正使用它之前还有许多额外的工作。 +想像一下,是否你关于你想要解析的东西的语法描述也是解释器?如果你能够仅仅直接运行语法,然后在你需要的地方增加挂钩,那是什么?那就是 Ohm 所做的事。 +###解释器简介 +[Ohm][1]是一种新的解析系统。它类似你可能已经在课本里面看到的语法并且它更强大,使用起来更简单。通过 Ohm, 你能够使用一种灵活的语法以 .ohm 文件格式来写格式定义,然后使用你的宿主语言把语义加入到里面。在这篇博文里,我们将用 JavaScript 作为宿主语言。 +Ohm 建立在一个为制造更简单、更灵活的解释器的一个多年调查基础之上。VPRI 的 [STEPS program](pdf) 使用 Ohm 的前驱为许多特殊的工作创造了专门的语言(比如一个有 400 行代码的平行制图描绘器)[Ometa][3]. +Ohm 有许多有趣的特点和符号,但是不是要全部解释它们,我认为我们应该只需投入其中并构建一些东西。 +###解析整数 +让我们来解析一些数字。这看起来会很简单,只需在一个文本串中寻找毗邻的数字,但是让我们尝试去处理所有形式的数字:整数和浮点数,十六进制数和八进制数,科学计数,负数。解析数字很简单,正确解析却很难。 +亲自构建这个代码将会很困难,会有很多故障,会伴随有许多特殊的情况,比如有时会相互矛盾。 +用 Ohm 构建的解释器涉及三个部分:语法、语义和测试。我通常挑选一个问题的一部分为它写测试,然后构建足够的语法和语义来使测试通过。然后我再挑选问题的另一部分,增加更多的测试,更新语法和语义,从而确保所有的测试能够持续通过。即使我们有了新的强大的工具,写解释器从概念上来说依旧很困难。测试是用一种合理的方式来构建解释器的唯一方法。现在,让我们开始工作。 +我们将从整数开始。一个整数由一系列相互毗邻的数字组成。让我们把下面的内容放入一个叫做 grammar.ohm 的文件中: +``` +CoolNums { + // just a basic integer + Number = digit+ +} +``` +这创造了一条撮合一个或多个数字叫作 Number 的单一规则。+ 意味着一个或更多,就像一个常规的表达。当有一个或更多的数字时,这个规则将会撮合它们,如果没有数字或者有一些不是数字的东西将不会撮合。一个数字定义成从 0 到 9 其中的一个字符。数字也是像 Number 一样的规则,但是它是 Ohm 的其中一条构建规则因此我们不需要去定义它。我们可以推翻它如果我们想的话但在这时候这没有任何意义,毕竟我们不打算去发明一种新的数。 +现在,我们可以读入这个语法并用 Ohm 库来运行它。 +把它放入 test1.js +``` +var ohm = require('ohm-js'); +var fs = require('fs'); +var assert = require('assert'); +var grammar = ohm.grammar(fs.readFileSync('src/blog_numbers/syntax1.ohm').toString()); +``` +Ohm 的语法调用将把文件读入并解释成一个语法对象。现在我们可以增加一些语义。把下面内容增加到你的 JavaScript 文件中: +``` +var sem = grammar.createSemantics().addOperation('toJS', { + Number: function(a) { + return parseInt(this.sourceString,10); + } +}); +``` +这创造了一系列叫作 sem with the operation to JS[伴有 JavaScript 操作的语义] 的语义。这些语义至关重要,一群函数和语法中的每一条规则相匹配。一个函数将会被调用当与它相匹配的语法规则被解析时。上面的 Number 函数将会被调用当语法中的 Number 规则被解析时。语法定义在语言中 chunks[大块] 是什么,语义定义当 chunks[大块] 被解析时应该做什么。 +语义函数能够做我们想做的任何事,比如打印初故障信息,创造对象,或者递归调用 toJS 作用于任何子节点。此时我们仅仅想把文本转换成真正的 JavaScript 整数。 +所有的语义函数有一个包含一些有用性质的暗含对象。源属性代表输入文本和这个节点相匹配。这个 sourceString[源串] 是一个匹配输入串,调用构建在 JavaScript 中的parseInt 函数会把这个串转换成一个数。parseInt 中 10 这个参数告诉 JavaScript 我们输入的是一个以 10 为基底的数。如果少了这个参数, JavaScript 也会假定以 10 为基底,但是我们把它包含在里面因为后面我们将支持以 16 为基底的数,所以使之明确比较好。 +既然我们有一些语法,让我们来实际解析一些东西看一看我们的解释器是否能够工作。如果知道我们的解释器工作?通过测试它,许多许多的测试,每一个边缘情况都需要一个测试。 +伴随标准断言 API,有一个测试函数能够匹配一些输入并运用我们的语义把它转换成一个数,然后比较转换生成的数和我们期望的输入。 +``` + function test(input, answer) { + var match = grammar.match(input); + if(match.failed()) return console.log("input failed to match " + input + match.message); + var result = sem(match).toJS(); + assert.deepEqual(result,answer); + console.log('success = ', result, answer); + } +``` +这个函数就是上面这个。现在我们能够为不同的数写一堆测试。如果匹配失败我们的脚本将会丢弃一个例外。如果不能打印成功,让我们尝试一下,把下面这些内容加入到脚本中: +``` + test("123",123); + test("999",999); + test("abc",999); +``` +然后用节点 test.js 运行脚本 +你的输出应该是这样: +``` +success = 123 123 +success = 999 999 +input failed to match abcLine 1, col 1: +> 1 | abc + ^ +Expected a digit +``` +真酷。正如理所当然的那样,前两个成功了,第三个失败了。更好的是,Ohm 自动给了我们一个很棒的错误信息指出匹配失败。 +###浮点数 +我们的解释器工作了,但是它不能做任何非常有趣的事。让我们把它扩展成既能解析整数又能解析浮点数。改变 grammar.ohm 文件使它看起来像下面这样: +``` +CoolNums { + // just a basic integer + Number = float | int + int = digit+ + float = digit+ "." digit+ +} +``` +这把 Number 规则改变成指向一个浮点数或者一个整数。我的意思是,我们把这读成"一个 Number 由一个浮点数或者一个整数构成。”然后整数定义成 digit+, 浮点数定义成 digit+ 后面跟着一个句号然后再跟着另一个 digit+. 这意味着在句号前和句号后都至少要有一个数字。如果一个数中没有一个句号那么它就不是一个浮点数,因此就是一个整数。 +现在,让我们再次看一下我们的语义作用。由于我们现在有了新的规则所以我们需要新的作用函数:一个作为整数的,一个作为浮点数的。 +``` +var sem = grammar.createSemantics().addOperation('toJS', { + Number: function(a) { + return a.toJS(); + }, + int: function(a) { + console.log("doing int", this.sourceString); + return parseInt(this.sourceString,10); + }, + float: function(a,b,c) { + console.log("doing float", this.sourceString); + return parseFloat(this.sourceString); + } +}); +``` +这里有两件事情需要注意。首先,整数,浮点数和数都有相匹配的语法规则和函数。然而,针对数的作用不再有任何意义。它接收子节点 'a' 然后通过子节点返回 toJS 的结果。换句话说,Number 规则简单的返回相匹配的子规则。由于这是在 Ohm 中任何规则的默认行为,因此实际上我们不用去考虑 Number 的作用,Ohm 会替我们来做这件事。 +第二,整数有一个参数然而浮点数有三个:a, b, 和 c. 这是由于规则的参数数量。参数数量意味着一个规则里面有多少参数。如果我们回过头去看语法,浮点数的规则是: +``` + float = digit+ "." digit+ +``` +浮点数规则通过三个部分来定义:第一个 digit+, '.', 还有第二个 digit+. 这三个部分都会作为参数传递给浮点数的作用函数。因此浮点数必须有三个参数否则 Ohm 库给出一个错误。在这种情况下我们不用在意参数因为我们仅仅直接攫取了输入串,但是我们仍然需要列表的参数来回避编译器错误。后面我们将实际使用其中一些参数。 +现在我们可以为新的浮点数支持添加更多的测试。 +``` +test("123",123); +test("999",999); +//test("abc",999); +test('123.456',123.456); +test('0.123',0.123); +test('.123',0.123); +``` +注意最后一个测试将会失败。一个浮点数必须以一个数开始,即使它仅仅是 0, .123 不是有效的,实际上真正的 JavaScript 语言有相同的规则。 +###十六进制数 +现在我们已经有了整数和浮点数,但是有一些其他的数的语法可能能够很好的支持:十六进制数和科学计数。十六进制数是是以 16 为基底2的数。十六进制数的数字能从 0 到 9 和从 A 到 F. 十六进制数经常用在计算机科学中当用二进制数据工作时,因为你可以仅仅使用两个数字表示 0 到 255 的数。 +在绝大多数源自 C 的编程语言(包括 JavaScript), 十六进制数通过在前面加上 ‘0x' 来向编译器表明后面跟的是一个十六进制数。为了让我们的解释器支持十六进制数,我们只需要添加另一条规则。 +``` + Number = hex | float | int + int = digit+ + float = digit+ "." digit+ + hex = "0x" hexDigit+ + hexDigit := "0".."9" | "a".."f" | "A".."F" +``` +我实际上已经增加了两条规则。'hex' 表明十六进制数是一个 'ox' 后面一个或多个 ’hexDigits'[十六进制数子] 的串。一个 'hexDigit' 是从 0 到 9, 或从 a 到 f, 或 A 到 F(包扩大写和小写的情况)的一个字符。我也修改了 Number 规则来识别十六进制数作为其他可能的选择。现在我们只需要另一条针对十六进制数的作用规则。 +``` + hex: function(a,b) { + return parseInt(this.sourceString,16); + } +``` +注意到,在这种情况下,我们把 '16' 作为基数传递给 'parseInt', 因为我们希望 JavaScript 知道这是一个十六进制数。 + +我略过了一些很重要需要注意的事。针对 'hexDigit' 的规则像下面这样: +``` + hexDigit := "0".."9" | "a".."f" | "A".."F" +``` +注意我使用的是 ':=' 而不是 '='. 在 Ohm 中,'=' 是当你需要推翻一条规则的时候使用。证明是 Ohm 已经有了针对 'hexDigit' 的默认规则,就像针对 'digit', 'space' 等一堆其他的东西。如果我使用了 '=', Ohm 将会报告一个错误。这是一个检查从而我不能无意识的推翻一个规则。由于新的 hexDigit 规则和 Ohm 的构建规则一样,所以我们可以仅仅对它添加注释然后让 Ohm 来实现它。我留下这个规则仅仅是因为这样我们可以看到它实际上是如何进行的。 +Now we can add some more tests and see that our hex digits really work: +现在,我们可以添加更多的测试然后看到十六进制数真的工作。 +``` +test('0x456',0x456); +test('0xFF',255); +``` +###科学计数 +最后,让我们来支持科学计数。科学计数是针对非常大或非常小的数比如 1.8×10^3, 在大多数编程语言中,科学计数法表示的数会写成这样:1.8e3 表示 18000, 或者 1.8e-3 表示 .018. 让我们增加另外一对规则来支持这个指数表示: +``` + float = digit+ "." digit+ exp? + exp = "e" "-"? digit+ +``` +上面增加了一个指数规则通过在浮点数规则末尾加上一个 '?'. '?' 表示 0 或 1,所以指数是可选择的但是不能超过一个。增加指数规则也改变了浮点数规则的参数数量,所以我们需要为浮点数作用增加又一个参数,即使我们不使用它。 +``` + float: function(a,b,c,d) { + console.log("doing float", this.sourceString); + return parseFloat(this.sourceString); + }, +``` +现在我们的测试可以通过了: +``` +test('4.8e10',4.8e10); +test('4.8e-10',4.8e-10); +``` +###结论 +Ohm 是构建解释器的一个很棒的工具,因为它很容易开始并且你可以递增的增加规则。Ohm 也还有其他我今天没有写到的很棒的特点,比如故障观察仪和子类化。 +s. +到目前为止,我们已经使用 Ohm 来把字符串翻译成 JavaScript 数,并且 Ohm 经常由于需要把一个表示转化成另一个这一目的而使用。然而,Ohm 还有更多的用途。通过放入以系列不同的语义作用你可以使用 Ohm 来真正处理和计算东西。一个单独的语法可以被许多不同的语义使用,这是 Ohm 其中一个不可思议的特点。 +在这个系列的下一篇文章中,我将向你们展示如何计算像(4.85 + 5 * (238 - 68)/2) 这样的数学表达式,不仅仅是解析数。 +额外的挑战:你能够扩展语法来支持八进制数吗?这些以 8 为基底的数能够只用 0 到 7 这几个数字来表示,前面加上一个数字 0 或者字母 o. 看看针对下面这些测试情况是够正确。下次我将给出答案。 +``` +test('0o77',7*8+7); +test('0o23',0o23); +``` + + + + +-------------------------------------------------------------------------------- + +via: https://www.pubnub.com/blog/2016-08-30-javascript-parser-ohm-makes-creating-a-programming-language-easy/?utm_source=javascriptweekly&utm_medium=email + +作者:[Josh Marinacci][a] +译者:[ucasFL](https://github.com/ucasFL) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.pubnub.com/blog/author/josh/ +[1]: https://github.com/cdglabs/ohm +[2]: http://www.vpri.org/pdf/tr2012001_steps.pdf +[3]: http://tinlizzie.org/ometa/ + + From 1ec4e0e34002c65f02b93e69226aa8b9fb3f8ee7 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Fri, 16 Sep 2016 23:09:24 +0800 Subject: [PATCH 0139/2437] Delete 20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md --- ...Creates a Language in 200 Lines of Code.md | 196 ------------------ 1 file changed, 196 deletions(-) delete mode 100644 sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md diff --git a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md b/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md deleted file mode 100644 index 810ba28c2a..0000000000 --- a/sources/tech/20160830 Ohm JavaScript Parser that Creates a Language in 200 Lines of Code.md +++ /dev/null @@ -1,196 +0,0 @@ -Translating by ucasFL - - -=========== -Ohm: 一种可以用两百行代码创造一种语言的 JavaScript 解释器 -解释器是一种非常有用的软件库。从概念上简单的说,它们的实现很有挑战并且在计算机科学中经常被认为是暗黑艺术。在这个系列的博文中,我会向你们展示为什么你不需要成为哈利波特就能够很好的控制解释器。但是为了以防万一带上你的魔杖。 -我们将探索一种叫做 Ohm 的新的开源库,它使得搭建解释器很简单并且更加容易再利用。在这个系列里,我们使用 Ohm 去识别数字,构建计算器等等。在这个系列的最后你将已经用少于 200 行的代码发明了一种完整的编程语言。这个强大的工具将让你能够做一些你可能过去认为不可能的事情。 -###为什么解释器很困难? -解释器非常有用。在很多时候你可能需要一个解释器。一种新的文件格式可能出现,你需要去处理但还没有人为它写了一个库;或者你发现了一种老格式的文件但是现存的解释器不能构建你需要的平台。我已经看到这样的事发生无数次。代码会来来去去但数据却是永恒的。 -基础的解释器很简单:只是把一个数据结构转化成另一个。所以为什么感觉你需要成为 邓布利多【魔法师】才能够把它们做出来。 -解释器的一些历史性的挑战是很难写,绝大多数工具很老并且假设了大量晦涩难懂的计算机科学知识。如果你在大学里上过编译器课程那么课本里可能也有从 1970 年以来的技术。幸运的是,解释器技术从那时候起已经提高了很多。 -代表性地,解释器是通过使用一种叫作形式语法的特殊语法来定义你想要解析的东西这样发明的,然后你需要把它放入像 Bison 和 Yacc 的工具中,这些工具能够产生一堆你需要修改的 C 代码或者链接到你实际写入额的编程语言。另外的选择是用你更喜欢的语言亲自动手写一个解释器,这很慢且很容易出错,在你能够真正使用它之前还有许多额外的工作。 -想像一下,是否你关于你想要解析的东西的语法描述也是解释器?如果你能够仅仅直接运行语法,然后在你需要的地方增加挂钩,那是什么?那就是 Ohm 所做的事。 -###解释器简介 -[Ohm][1]是一种新的解析系统。它类似你可能已经在课本里面看到的语法并且它更强大,使用起来更简单。通过 Ohm, 你能够使用一种灵活的语法以 .ohm 文件格式来写格式定义,然后使用你的宿主语言把语义加入到里面。在这篇博文里,我们将用 JavaScript 作为宿主语言。 -Ohm 建立在一个为制造更简单、更灵活的解释器的一个多年调查基础之上。VPRI 的 [STEPS program](pdf) 使用 Ohm 的前驱为许多特殊的工作创造了专门的语言(比如一个有 400 行代码的平行制图描绘器)[Ometa][3]. -Ohm 有许多有趣的特点和符号,但是不是要全部解释它们,我认为我们应该只需投入其中并构建一些东西。 -###解析整数 -让我们来解析一些数字。这看起来会很简单,只需在一个文本串中寻找毗邻的数字,但是让我们尝试去处理所有形式的数字:整数和浮点数,十六进制数和八进制数,科学计数,负数。解析数字很简单,正确解析却很难。 -亲自构建这个代码将会很困难,会有很多故障,会伴随有许多特殊的情况,比如有时会相互矛盾。 -用 Ohm 构建的解释器涉及三个部分:语法、语义和测试。我通常挑选一个问题的一部分为它写测试,然后构建足够的语法和语义来使测试通过。然后我再挑选问题的另一部分,增加更多的测试,更新语法和语义,从而确保所有的测试能够持续通过。即使我们有了新的强大的工具,写解释器从概念上来说依旧很困难。测试是用一种合理的方式来构建解释器的唯一方法。现在,让我们开始工作。 -我们将从整数开始。一个整数由一系列相互毗邻的数字组成。让我们把下面的内容放入一个叫做 grammar.ohm 的文件中: -``` -CoolNums { - // just a basic integer - Number = digit+ -} -``` -这创造了一条撮合一个或多个数字叫作 Number 的单一规则。+ 意味着一个或更多,就像一个常规的表达。当有一个或更多的数字时,这个规则将会撮合它们,如果没有数字或者有一些不是数字的东西将不会撮合。一个数字定义成从 0 到 9 其中的一个字符。数字也是像 Number 一样的规则,但是它是 Ohm 的其中一条构建规则因此我们不需要去定义它。我们可以推翻它如果我们想的话但在这时候这没有任何意义,毕竟我们不打算去发明一种新的数。 -现在,我们可以读入这个语法并用 Ohm 库来运行它。 -把它放入 test1.js -``` -var ohm = require('ohm-js'); -var fs = require('fs'); -var assert = require('assert'); -var grammar = ohm.grammar(fs.readFileSync('src/blog_numbers/syntax1.ohm').toString()); -``` -Ohm 的语法调用将把文件读入并解释成一个语法对象。现在我们可以增加一些语义。把下面内容增加到你的 JavaScript 文件中: -``` -var sem = grammar.createSemantics().addOperation('toJS', { - Number: function(a) { - return parseInt(this.sourceString,10); - } -}); -``` -这创造了一系列叫作 sem with the operation to JS[伴有 JavaScript 操作的语义] 的语义。这些语义至关重要,一群函数和语法中的每一条规则相匹配。一个函数将会被调用当与它相匹配的语法规则被解析时。上面的 Number 函数将会被调用当语法中的 Number 规则被解析时。语法定义在语言中 chunks[大块] 是什么,语义定义当 chunks[大块] 被解析时应该做什么。 -语义函数能够做我们想做的任何事,比如打印初故障信息,创造对象,或者递归调用 toJS 作用于任何子节点。此时我们仅仅想把文本转换成真正的 JavaScript 整数。 -所有的语义函数有一个包含一些有用性质的暗含对象。源属性代表输入文本和这个节点相匹配。这个 sourceString[源串] 是一个匹配输入串,调用构建在 JavaScript 中的parseInt 函数会把这个串转换成一个数。parseInt 中 10 这个参数告诉 JavaScript 我们输入的是一个以 10 为基底的数。如果少了这个参数, JavaScript 也会假定以 10 为基底,但是我们把它包含在里面因为后面我们将支持以 16 为基底的数,所以使之明确比较好。 -既然我们有一些语法,让我们来实际解析一些东西看一看我们的解释器是否能够工作。如果知道我们的解释器工作?通过测试它,许多许多的测试,每一个边缘情况都需要一个测试。 -伴随标准断言 API,有一个测试函数能够匹配一些输入并运用我们的语义把它转换成一个数,然后比较转换生成的数和我们期望的输入。 -``` - function test(input, answer) { - var match = grammar.match(input); - if(match.failed()) return console.log("input failed to match " + input + match.message); - var result = sem(match).toJS(); - assert.deepEqual(result,answer); - console.log('success = ', result, answer); - } -``` -这个函数就是上面这个。现在我们能够为不同的数写一堆测试。如果匹配失败我们的脚本将会丢弃一个例外。如果不能打印成功,让我们尝试一下,把下面这些内容加入到脚本中: -``` - test("123",123); - test("999",999); - test("abc",999); -``` -然后用节点 test.js 运行脚本 -你的输出应该是这样: -``` -success = 123 123 -success = 999 999 -input failed to match abcLine 1, col 1: -> 1 | abc - ^ -Expected a digit -``` -真酷。正如理所当然的那样,前两个成功了,第三个失败了。更好的是,Ohm 自动给了我们一个很棒的错误信息指出匹配失败。 -###浮点数 -我们的解释器工作了,但是它不能做任何非常有趣的事。让我们把它扩展成既能解析整数又能解析浮点数。改变 grammar.ohm 文件使它看起来像下面这样: -``` -CoolNums { - // just a basic integer - Number = float | int - int = digit+ - float = digit+ "." digit+ -} -``` -这把 Number 规则改变成指向一个浮点数或者一个整数。我的意思是,我们把这读成"一个 Number 由一个浮点数或者一个整数构成。”然后整数定义成 digit+, 浮点数定义成 digit+ 后面跟着一个句号然后再跟着另一个 digit+. 这意味着在句号前和句号后都至少要有一个数字。如果一个数中没有一个句号那么它就不是一个浮点数,因此就是一个整数。 -现在,让我们再次看一下我们的语义作用。由于我们现在有了新的规则所以我们需要新的作用函数:一个作为整数的,一个作为浮点数的。 -``` -var sem = grammar.createSemantics().addOperation('toJS', { - Number: function(a) { - return a.toJS(); - }, - int: function(a) { - console.log("doing int", this.sourceString); - return parseInt(this.sourceString,10); - }, - float: function(a,b,c) { - console.log("doing float", this.sourceString); - return parseFloat(this.sourceString); - } -}); -``` -这里有两件事情需要注意。首先,整数,浮点数和数都有相匹配的语法规则和函数。然而,针对数的作用不再有任何意义。它接收子节点 'a' 然后通过子节点返回 toJS 的结果。换句话说,Number 规则简单的返回相匹配的子规则。由于这是在 Ohm 中任何规则的默认行为,因此实际上我们不用去考虑 Number 的作用,Ohm 会替我们来做这件事。 -第二,整数有一个参数然而浮点数有三个:a, b, 和 c. 这是由于规则的参数数量。参数数量意味着一个规则里面有多少参数。如果我们回过头去看语法,浮点数的规则是: -``` - float = digit+ "." digit+ -``` -浮点数规则通过三个部分来定义:第一个 digit+, '.', 还有第二个 digit+. 这三个部分都会作为参数传递给浮点数的作用函数。因此浮点数必须有三个参数否则 Ohm 库给出一个错误。在这种情况下我们不用在意参数因为我们仅仅直接攫取了输入串,但是我们仍然需要列表的参数来回避编译器错误。后面我们将实际使用其中一些参数。 -现在我们可以为新的浮点数支持添加更多的测试。 -``` -test("123",123); -test("999",999); -//test("abc",999); -test('123.456',123.456); -test('0.123',0.123); -test('.123',0.123); -``` -注意最后一个测试将会失败。一个浮点数必须以一个数开始,即使它仅仅是 0, .123 不是有效的,实际上真正的 JavaScript 语言有相同的规则。 -###十六进制数 -现在我们已经有了整数和浮点数,但是有一些其他的数的语法可能能够很好的支持:十六进制数和科学计数。十六进制数是是以 16 为基底2的数。十六进制数的数字能从 0 到 9 和从 A 到 F. 十六进制数经常用在计算机科学中当用二进制数据工作时,因为你可以仅仅使用两个数字表示 0 到 255 的数。 -在绝大多数源自 C 的编程语言(包括 JavaScript), 十六进制数通过在前面加上 ‘0x' 来向编译器表明后面跟的是一个十六进制数。为了让我们的解释器支持十六进制数,我们只需要添加另一条规则。 -``` - Number = hex | float | int - int = digit+ - float = digit+ "." digit+ - hex = "0x" hexDigit+ - hexDigit := "0".."9" | "a".."f" | "A".."F" -``` -我实际上已经增加了两条规则。'hex' 表明十六进制数是一个 'ox' 后面一个或多个 ’hexDigits'[十六进制数子] 的串。一个 'hexDigit' 是从 0 到 9, 或从 a 到 f, 或 A 到 F(包扩大写和小写的情况)的一个字符。我也修改了 Number 规则来识别十六进制数作为其他可能的选择。现在我们只需要另一条针对十六进制数的作用规则。 -``` - hex: function(a,b) { - return parseInt(this.sourceString,16); - } -``` -注意到,在这种情况下,我们把 '16' 作为基数传递给 'parseInt', 因为我们希望 JavaScript 知道这是一个十六进制数。 - -我略过了一些很重要需要注意的事。针对 'hexDigit' 的规则像下面这样: -``` - hexDigit := "0".."9" | "a".."f" | "A".."F" -``` -注意我使用的是 ':=' 而不是 '='. 在 Ohm 中,'=' 是当你需要推翻一条规则的时候使用。证明是 Ohm 已经有了针对 'hexDigit' 的默认规则,就像针对 'digit', 'space' 等一堆其他的东西。如果我使用了 '=', Ohm 将会报告一个错误。这是一个检查从而我不能无意识的推翻一个规则。由于新的 hexDigit 规则和 Ohm 的构建规则一样,所以我们可以仅仅对它添加注释然后让 Ohm 来实现它。我留下这个规则仅仅是因为这样我们可以看到它实际上是如何进行的。 -Now we can add some more tests and see that our hex digits really work: -现在,我们可以添加更多的测试然后看到十六进制数真的工作。 -``` -test('0x456',0x456); -test('0xFF',255); -``` -###科学计数 -最后,让我们来支持科学计数。科学计数是针对非常大或非常小的数比如 1.8×10^3, 在大多数编程语言中,科学计数法表示的数会写成这样:1.8e3 表示 18000, 或者 1.8e-3 表示 .018. 让我们增加另外一对规则来支持这个指数表示: -``` - float = digit+ "." digit+ exp? - exp = "e" "-"? digit+ -``` -上面增加了一个指数规则通过在浮点数规则末尾加上一个 '?'. '?' 表示 0 或 1,所以指数是可选择的但是不能超过一个。增加指数规则也改变了浮点数规则的参数数量,所以我们需要为浮点数作用增加又一个参数,即使我们不使用它。 -``` - float: function(a,b,c,d) { - console.log("doing float", this.sourceString); - return parseFloat(this.sourceString); - }, -``` -现在我们的测试可以通过了: -``` -test('4.8e10',4.8e10); -test('4.8e-10',4.8e-10); -``` -###结论 -Ohm 是构建解释器的一个很棒的工具,因为它很容易开始并且你可以递增的增加规则。Ohm 也还有其他我今天没有写到的很棒的特点,比如故障观察仪和子类化。 -s. -到目前为止,我们已经使用 Ohm 来把字符串翻译成 JavaScript 数,并且 Ohm 经常由于需要把一个表示转化成另一个这一目的而使用。然而,Ohm 还有更多的用途。通过放入以系列不同的语义作用你可以使用 Ohm 来真正处理和计算东西。一个单独的语法可以被许多不同的语义使用,这是 Ohm 其中一个不可思议的特点。 -在这个系列的下一篇文章中,我将向你们展示如何计算像(4.85 + 5 * (238 - 68)/2) 这样的数学表达式,不仅仅是解析数。 -额外的挑战:你能够扩展语法来支持八进制数吗?这些以 8 为基底的数能够只用 0 到 7 这几个数字来表示,前面加上一个数字 0 或者字母 o. 看看针对下面这些测试情况是够正确。下次我将给出答案。 -``` -test('0o77',7*8+7); -test('0o23',0o23); -``` - - - - --------------------------------------------------------------------------------- - -via: https://www.pubnub.com/blog/2016-08-30-javascript-parser-ohm-makes-creating-a-programming-language-easy/?utm_source=javascriptweekly&utm_medium=email - -作者:[Josh Marinacci][a] -译者:[ucasFL](https://github.com/ucasFL) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: https://www.pubnub.com/blog/author/josh/ -[1]: https://github.com/cdglabs/ohm -[2]: http://www.vpri.org/pdf/tr2012001_steps.pdf -[3]: http://tinlizzie.org/ometa/ - - From 17610cc21fa11167cc5a3327335d4b7d853a0794 Mon Sep 17 00:00:00 2001 From: wxy Date: Fri, 16 Sep 2016 23:18:58 +0800 Subject: [PATCH 0140/2437] =?UTF-8?q?=E8=A1=A5=E5=AE=8C=20PR?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @ucasFL --- ...minism and increasing diversity in tech.md | 84 ------------------- ...inism and increasing diversity in tech.md} | 0 ...Create a Language in 200 Lines of Code.md} | 0 3 files changed, 84 deletions(-) delete mode 100644 sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md rename translated/talk/{Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech => Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech.md} (100%) rename translated/tech/{20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code => 20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code.md} (100%) diff --git a/sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md b/sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md deleted file mode 100644 index 78c6cbaecf..0000000000 --- a/sources/talk/20150806 Torvalds 2.0--Patricia Torvalds on computing college feminism and increasing diversity in tech.md +++ /dev/null @@ -1,84 +0,0 @@ -LinuxBars translating - -Torvalds 2.0: Patricia Torvalds on computing, college, feminism, and increasing diversity in tech -================================================================================ - -![Image by : Photo by Becky Svartström. Modified by Opensource.com. CC BY-SA 4.0](http://opensource.com/sites/default/files/styles/image-full-size/public/images/life/osdc-lead-patriciatorvalds.png) -Image by : Photo by Becky Svartström. Modified by Opensource.com. [CC BY-SA 4.0][1] - -Patricia Torvalds isn't the Torvalds name that pops up in Linux and open source circles. Yet. - -![](http://opensource.com/sites/default/files/images/life-uploads/ptorvalds.png) - -At 18, Patricia is a feminist with a growing list of tech achievements, open source industry experience, and her sights set on diving into her freshman year of college at Duke University's Pratt School of Engineering. She works for [Puppet Labs][2] in Portland, Oregon, as an intern, but soon she'll head to Durham, North Carolina, to start the fall semester of college. - -In this exclusive interview, Patricia explains what got her interested in computer science and engineering (spoiler alert: it wasn't her father), what her high school did "right" with teaching tech, the important role feminism plays in her life, and her thoughts on the lack of diversity in technology. - -![](http://opensource.com/sites/default/files/images/life/Interview%20banner%20Q%26A.png) - -### What made you interested in studying computer science and engineering? ### - -My interest in tech really grew throughout high school. I wanted to go into biology for a while, until around my sophomore year. I had a web design internship at the Portland VA after my sophomore year. And I took an engineering class called Exploratory Ventures, which sent an ROV into the Pacific ocean late in my sophomore year, but the turning point was probably when I was named a regional winner and national runner up for the [NCWIT Aspirations in Computing][3] award halfway through my junior year. - -The award made me feel validated in my interest, of course, but I think the most important part of it was getting to join a Facebook group for all the award winners. The girls who have won the award are absolutely incredible and so supportive of each other. I was definitely interested in computer science before I won the award, because of my work in XV and at the VA, but having these girls to talk to solidified my interest and has kept it really strong. Teaching XV—more on that later—my junior and senior year, also, made engineering and computer science really fun for me. - -### What do you plan to study? And do you already know what you want to do after college? ### - -I hope to major in either Mechanical or Electrical and Computer Engineering as well as Computer Science, and minor in Women's Studies. After college, I hope to work for a company that supports or creates technology for social good, or start my own company. - -### My daughter had one high school programming class—Visual Basic. She was the only girl in her class, and she ended up getting harassed and having a miserable experience. What was your experience like? ### - -My high school began offering computer science classes my senior year, and I took Visual Basic as well! The class wasn't bad, but I was definitely one of three or four girls in the class of 20 or so students. Other computing classes seemed to have similar gender breakdowns. However, my high school was extremely small and the teacher was supportive of inclusivity in tech, so there was no harassment that I noticed. Hopefully the classes become more diverse in future years. - -### What did your schools do right technology-wise? And how could they have been better? ### - -My high school gave us consistent access to computers, and teachers occasionally assigned technology-based assignments in unrelated classes—we had to create a website for a social studies class a few times—which I think is great because it exposes everyone to tech. The robotics club was also pretty active and well-funded, but fairly small; I was not a member. One very strong component of the school's technology/engineering program is actually a student-taught engineering class called Exploratory Ventures, which is a hands-on class that tackles a new engineering or computer science problem every year. I taught it for two years with a classmate of mine, and have had students come up to me and tell me they're interested in pursuing engineering or computer science as a result of the class. - -However, my high school was not particularly focused on deliberately including young women in these programs, and it isn't very racially diverse. The computing-based classes and clubs were, by a vast majority, filled with white male students. This could definitely be improved on. - -### Growing up, how did you use technology at home? ### - -Honestly, when I was younger I used my computer time (my dad created a tracker, which logged us off after an hour of Internet use) to play Neopets or similar games. I guess I could have tried to mess with the tracker or played on the computer without Internet use, but I just didn't. I sometimes did little science projects with my dad, and I remember once printing "Hello world" in the terminal with him a thousand times, but mostly I just played online games with my sisters and didn't get my start in computing until high school. - -### You were active in the Feminism Club at your high school. What did you learn from that experience? What feminist issues are most important to you now? ### - -My friend and I co-founded Feminism Club at our high school late in our sophomore year. We did receive lots of resistance to the club at first, and while that never entirely went away, by the time we graduated feminist ideals were absolutely a part of the school's culture. The feminist work we did at my high school was generally on a more immediate scale and focused on issues like the dress code. - -Personally, I'm very focused on intersectional feminism, which is feminism as it applies to other aspects of oppression like racism and classism. The Facebook page [Guerrilla Feminism][4] is a great example of an intersectional feminism and has done so much to educate me. I currently run the Portland branch. - -Feminism is also important to me in terms of diversity in tech, although as an upper-class white woman with strong connections in the tech world, the problems here affect me much less than they do other people. The same goes for my involvement in intersectional feminism. Publications like [Model View Culture][5] are very inspiring to me, and I admire Shanley Kane so much for what she does. - -### What advice would you give parents who want to teach their children how to program? ### - -Honestly, nobody ever pushed me into computer science or engineering. Like I said, for a long time I wanted to be a geneticist. I got a summer internship doing web design for the VA the summer after my sophomore year and totally changed my mind. So I don't know if I can fully answer that question. - -I do think genuine interest is important, though. If my dad had sat me down in front of the computer and told me to configure a webserver when I was 12, I don't think I'd be interested in computer science. Instead, my parents gave me a lot of free reign to do what I wanted, which was mostly coding terrible little HTML sites for my Neopets. Neither of my younger sisters are interested in engineering or computer science, and my parents don't care. I'm really lucky my parents have given me and my sisters the encouragement and resources to explore our interests. - -Still, I grew up saying my future career would be "like my dad's"—even when I didn't know what he did. He has a pretty cool job. Also, one time when I was in middle school, I told him that and he got a little choked up and said I wouldn't think that in high school. So I guess that motivated me a bit. - -### What suggestions do you have for leaders in open source communities to help them attract and maintain a more diverse mix of contributors? ### - -I'm actually not active in particular open source communities. I feel much more comfortable discussing computing with other women; I'm a member of the [NCWIT Aspirations in Computing][6] network and it's been one of the most important aspects of my continued interest in technology, as well as the Facebook group [Ladies Storm Hackathons][7]. - -I think this applies well to attracting and maintaining a talented and diverse mix of contributors: Safe spaces are important. I have seen the misogynistic and racist comments made in some open source communities, and subsequent dismissals when people point out the issues. I think that in maintaining a professional community there have to be strong standards on what constitutes harassment or inappropriate conduct. Of course, people can—and will—have a variety of opinions on what they should be able to express in open source communities, or any community. However, if community leaders actually want to attract and maintain diverse talent, they need to create a safe space and hold community members to high standards. - -I also think that some community leaders just don't value diversity. It's really easy to argue that tech is a meritocracy, and the reason there are so few marginalized people in tech is just that they aren't interested, and that the problem comes from earlier on in the pipeline. They argue that if someone is good enough at their job, their gender or race or sexual orientation doesn't matter. That's the easy argument. But I was raised not to make excuses for mistakes. And I think the lack of diversity is a mistake, and that we should be taking responsibility for it and actively trying to make it better. - --------------------------------------------------------------------------------- - -via: http://opensource.com/life/15/8/patricia-torvalds-interview - -作者:[Rikki Endsley][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://opensource.com/users/rikki-endsley -[1]:https://creativecommons.org/licenses/by-sa/4.0/ -[2]:https://puppetlabs.com/ -[3]:https://www.aspirations.org/ -[4]:https://www.facebook.com/guerrillafeminism -[5]:https://modelviewculture.com/ -[6]:https://www.aspirations.org/ -[7]:https://www.facebook.com/groups/LadiesStormHackathons/ diff --git a/translated/talk/Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech b/translated/talk/Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech.md similarity index 100% rename from translated/talk/Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech rename to translated/talk/Torvalds2.0: Patricia Torvalds on computing, college, feminism and increasing diversity in tech.md diff --git a/translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code b/translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code.md similarity index 100% rename from translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code rename to translated/tech/20160830 Ohm JavaScript Parser that Create a Language in 200 Lines of Code.md From bec76212d6935164ff7d6bc133aa0d7e5f11e283 Mon Sep 17 00:00:00 2001 From: qingyun <845767657@qq.com> Date: Sat, 17 Sep 2016 12:53:29 +0800 Subject: [PATCH 0141/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=2020160917=20A=20Web=20Crawler=20With=20asyncio=20Coroutines.m?= =?UTF-8?q?d?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...7 A Web Crawler With asyncio Coroutines.md | 1088 +++++++++++++++++ 1 file changed, 1088 insertions(+) create mode 100644 translated/tech/20160917 A Web Crawler With asyncio Coroutines.md diff --git a/translated/tech/20160917 A Web Crawler With asyncio Coroutines.md b/translated/tech/20160917 A Web Crawler With asyncio Coroutines.md new file mode 100644 index 0000000000..1bce0fd22a --- /dev/null +++ b/translated/tech/20160917 A Web Crawler With asyncio Coroutines.md @@ -0,0 +1,1088 @@ +一个使用asyncio协程的网络爬虫 +=== + + +## 介绍 + +经典的计算机科学强调高效的算法,尽可能快地完成计算。但是很多网络程序的时间并不是消耗在计算上,而是在等待许多慢速的连接或者低频事件的发生。这些程序暴露出一个新的挑战:如何高效的等待大量网络事件。一个现代的解决方案是异步I/O。 + +这一章我们将实现一个简单的网络爬虫。这个爬虫只是一个原型式的异步应用,因为它等待许多响应而只做少量的计算。一次爬的网页越多,它就能越快的完成任务。如果它为每个动态的请求启动一个线程的话,随着并发请求数量的增加,它会在耗尽套接字之前,耗尽内存或者线程相关的资源。使用异步I/O可以避免这个的问题。 + +我们将分三个阶段展示这个例子。首先,我们会实现一个事件循环并用这个事件循环和回调来勾画出一个网络爬虫。它很有效,但是当把它扩展成更复杂的问题时,就会导致无法管理的混乱代码。然后,由于Python的协程不仅有效而且可扩展,我们将用Python的生成器函数实现一个简单的协程。在最后一个阶段,我们将使用Python标准库"asyncio"中功能完整的协程和异步队列完成这个网络爬虫。 + +## 任务 + +网络爬虫寻找并下载一个网站上的所有网页,也许还会把它们存档,为它们建立索引。从根URL开始,它获取每个网页,解析出没有遇到过的链接加到队列中。当网页没有未见到过的链接并且队列为空时,它便停止运行。 + +我们可以通过同时下载大量的网页来加快这一过程。当爬虫发现新的链接,它使用一个新的套接字并行的处理这个新链接,解析响应,添加新链接到队列。当并发很大时,可能会导致性能下降,所以我们会限制并发的数量,在队列保留那些未处理的链接,直到一些正在执行的任务完成。 + +## 传统方式 + +怎么使一个爬虫并发?传统的做法是创建一个线程池,每个线程使用一个套接字在一段时间内负责一个网页的下载。比如,下载xkcd.com网站的一个网页: + +```python +def fetch(url): + sock = socket.socket() + sock.connect(('xkcd.com', 80)) + request = 'GET {} HTTP/1.0\r\nHost: xkcd.com\r\n\r\n'.format(url) + sock.send(request.encode('ascii')) + response = b'' + chunk = sock.recv(4096) + while chunk: + response += chunk + chunk = sock.recv(4096) + + # Page is now downloaded. + links = parse_links(response) + q.add(links) +``` + +套接字操作默认是阻塞的:当一个线程调用一个类似`connect`和`recv`方法时,它会阻塞,直到操作完成.[^15]因此,为了同一时间内下载多个网页,我们需要很多线程。一个复杂的应用会通过线程池保持空闲的线程来分摊创建线程的开销。同样的做法也适用于套接字,使用连接池。 + +到目前为止,线程是昂贵的,操作系统对一个进程,一个用户,一台机器能使用线程做了不同的硬性限制。在Jesse系统中,一个Python线程需要50K的内存,开启上万个线程会失败。每个线程的开销和系统的限制就是这种方式的瓶颈所在。 + +在Dan Kegel那一篇很有影响力的文章"The C10K problem"[^8]中,它提出多线程方式在I/O并发上的局限性。他在开始写道, + +>是时候网络服务器要同时处理成千上万的客户啦,你不这样认为么?毕竟,现在网络是个很大的地方。 + +Kegel在1999年创造出"C10K"术语。一万个连接在今天看来还是可接受的,但是问题依然存在,只不过大小不同。回到那时候,对于C10K问题,每个连接启一个线程是不切实际的。现在这个限制已经成指数级增长。确实,我们的玩具网络爬虫使用线程也可以工作的很好。但是,对于有着千万级连接的大规模应用来说,限制依然存在:会消耗掉所有线程,即使套接字还够用。那么我们该如何解决这个问题? + +## 异步 + +异步I/O框架在一个线程中完成并发操作。让我们看看这是怎么做到的。 + +异步框架使用*非阻塞*套接字。异步爬虫中,我们在发起到服务器的连接前把套接字设为非阻塞: + +```python +sock = socket.socket() +sock.setblocking(False) +try: + sock.connect(('xkcd.com', 80)) +except BlockingIOError: + pass +``` + +对一个非阻塞套接字调用`connect`方法会立即抛出异常,即使它正常工作。这个异常模拟了底层C语言函数的行为,它把`errno`设置为`EINPROGRESS`,告诉你操作已经开始。 + +现在我们的爬虫需要一种知道连接何时建立的方法,这样它才能发送HTTP请求。我们可以简单地使用循环来重试: + +```python +request = 'GET {} HTTP/1.0\r\nHost: xkcd.com\r\n\r\n'.format(url) +encoded = request.encode('ascii') + +while True: + try: + sock.send(encoded) + break # Done. + except OSError as e: + pass + +print('sent') +``` + +这种方法不仅消耗CPU,也不能有效的等待*多个*套接字。在远古时代,BSD Unix的解决方法是`select`,一个C函数,它在一个或一组非阻塞套接字上等待事件发生。现在,互联网应用大量连接的需求,导致`select`被`poll`代替,以及BSD的`kqueue`和Linux的`epoll`。它们的API和`select`相似,但在大数量的连接中也能有较好的性能。 + +Python 3.4的`DefaultSelector`使用你系统上最好的类`select`函数。去注册一个网络I/O事件,我们创建一个非阻塞套接字,并使用默认的selector注册。 + +```python +from selectors import DefaultSelector, EVENT_WRITE + +selector = DefaultSelector() + +sock = socket.socket() +sock.setblocking(False) +try: + sock.connect(('xkcd.com', 80)) +except BlockingIOError: + pass + +def connected(): + selector.unregister(sock.fileno()) + print('connected!') + +selector.register(sock.fileno(), EVENT_WRITE, connected) +``` + +我们不理会这个伪造的错误,调用`selector.register`,传递套接字文件描述符,一个表示我们想要监听什么事件的常量。为了当连接建立时收到提醒,我们使用`EVENT_WRITE`:它表示什么时候这个套接字可写。我们还传递了一个Python函数,`connected`,当对应事件发生时被调用。这样的函数被称为*回调*。 + +我们在一个循环中处理I/O提醒,随着selector接收到它们。 + +```python +def loop(): + while True: + events = selector.select() + for event_key, event_mask in events: + callback = event_key.data + callback() +``` + +`connected`回调函数被保存在`event_key.data`中,一旦这个非阻塞套接字建立连接,它就会被取出来执行。 + +不像我们前面那个快速重试的循环,这里的`select`调用会阻塞,等待下一个I/O事件,接着执行等待这个事件的回调函数。 + +到目前为止我们展现了什么?我们展示了如何开始一个I/O操作和当操作准备好时调用回调函数。异步*框架*,它在单线程中执行并发操作,建立在两个功能之上,非阻塞套接字和事件循环。 + +## 回调 + +用我们刚刚建立的异步框架,怎么才能完成一个网络爬虫?即使是一个简单的网页下载程序也是很难写的。 + +首先,我们有一个未获取的URL集合,和一个已经解析过的URL集合。 + +```python +urls_todo = set(['/']) +seen_urls = set(['/']) +``` + +这两个集合加在一起就是所有的URL。用"/"初始化它们。 + +获取一个网页需要一系列的回调。在套接字连接建立时`connected`回调触发,它向服务器发送一个GET请求。但是它要等待响应,所以我们需要注册另一个回调函数,当回调被调用,它也不能一次读取完整的请求,所以,需要再一次注册,如此反复。 + +让我们把这些回调放在一个`Fetcher`对象中,它需要一个URL,一个套接字,还需要一个地方保存返回的字节: + +```python +class Fetcher: + def __init__(self, url): + self.response = b'' # Empty array of bytes. + self.url = url + self.sock = None +``` + +我们的入口点在`Fetcher.fetch`: + +```python + # Method on Fetcher class. + def fetch(self): + self.sock = socket.socket() + self.sock.setblocking(False) + try: + self.sock.connect(('xkcd.com', 80)) + except BlockingIOError: + pass + + # Register next callback. + selector.register(self.sock.fileno(), + EVENT_WRITE, + self.connected) +``` + +`fetch`方法从连接一个套接字开始。但是要注意这个方法在连接建立前就返回了。它必须返回到事件循环中等待连接建立。为了理解为什么要要这样,假设我们程序的整体结构如下: + +```python +# Begin fetching http://xkcd.com/353/ +fetcher = Fetcher('/353/') +fetcher.fetch() + +while True: + events = selector.select() + for event_key, event_mask in events: + callback = event_key.data + callback(event_key, event_mask) +``` + +所有的事件提醒都在事件循环中的`select`函数后处理。所以`fetch`必须把控制权交给事件循环。这样我们的程序才能知道什么时候连接已建立,接着循环调用`connected`回调,它已经在`fetch`方法中注册过。 + +这里是我们`connected`方法的实现: + +```python + # Method on Fetcher class. + def connected(self, key, mask): + print('connected!') + selector.unregister(key.fd) + request = 'GET {} HTTP/1.0\r\nHost: xkcd.com\r\n\r\n'.format(self.url) + self.sock.send(request.encode('ascii')) + + # Register the next callback. + selector.register(key.fd, + EVENT_READ, + self.read_response) +``` + +这个方法发送一个GET请求。一个真正的应用会检查`send`的返回值,以防所有的信息没能一次发送出去。但是我们的请求很小,应用也不复杂。它只是简单的调用`send`,然后等待响应。当然,它必须注册另一个回调并把控制权交给事件循环。接下来也是最后一个回调函数`read_response`,它处理服务器的响应: + +```python + # Method on Fetcher class. + def read_response(self, key, mask): + global stopped + + chunk = self.sock.recv(4096) # 4k chunk size. + if chunk: + self.response += chunk + else: + selector.unregister(key.fd) # Done reading. + links = self.parse_links() + + # Python set-logic: + for link in links.difference(seen_urls): + urls_todo.add(link) + Fetcher(link).fetch() # <- New Fetcher. + + seen_urls.update(links) + urls_todo.remove(self.url) + if not urls_todo: + stopped = True +``` + +这个回调在每次selector发现套接字*可读*时被调用,可读有两种情况:套接字接受到数据或它被关闭。 + +这个回调函数从套接字读取4K数据。如果没有4k,那么有多少读多少。如果比4K多,`chunk`只包4K数据并且这个套接字保持可读,这样在事件循环的下一个周期,会在次回到这个回调函数。当响应完成时,服务器关闭这个套接字,`chunk`为空。 + +没有展示的`parse_links`方法,它返回一个URL集合。我们为每个新的URL启动一个fetcher。注意一个使用异步回调方式编程的好处:我们不需要为共享数据加锁,比如我们往`seen_urls`增加新链接时。这是一种非抢占式的多任务,它不会在我们代码中的任意一个地方中断。 + +我们增加了一个全局变量`stopped`,用它来控制这个循环: + +```python +stopped = False + +def loop(): + while not stopped: + events = selector.select() + for event_key, event_mask in events: + callback = event_key.data + callback() +``` + +一旦所有的网页被下载下来,fetcher停止这个事件循环,程序退出。 + +这个例子让异步编程的一个问题明显的暴露出来:意大利面代码。 + +我们需要某种方式来表达一串计算和I/O操作,并且能够调度多个这样的操作让他们并发的执行。但是,没有线程你不能把这一串操作写在一个函数中:当函数开始一个I/O操作,它明确的把未来所需的状态保存下来,然后返回。你需要考虑如何写这个状态保存的代码。 + +让我们来解释下这到底是什么意思。考虑在线程中使用通常的阻塞套接字来获取一个网页时是多么简单。 + +```python +# Blocking version. +def fetch(url): + sock = socket.socket() + sock.connect(('xkcd.com', 80)) + request = 'GET {} HTTP/1.0\r\nHost: xkcd.com\r\n\r\n'.format(url) + sock.send(request.encode('ascii')) + response = b'' + chunk = sock.recv(4096) + while chunk: + response += chunk + chunk = sock.recv(4096) + + # Page is now downloaded. + links = parse_links(response) + q.add(links) +``` + +在一个套接字操作和下一个操作之间这个函数到底记住了什么?它有一个套接字,一个URL和一个可增长的`response`。运行在线程中的函数使用编程语言的基本功能,栈中的局部变量来保存临时的状态。这样的函数有一个"continuation"----在I/O结束后它要执行的代码。运行时通过线程的指令指针来记住这个continuation。你不必考虑怎么在I/O操作后恢复局部变量和这个continuation。语言本身的特性帮你解决。 + +但是用一个基于回调的异步框架,这些语言特性不能提供一点帮助。当等待I/O操作时,一个函数必须明确的保存它的状态,因为它会在I/O操作完成之前返回并清除栈帧。为了在我们基于回调的例子中代替局部变量,我们把`sock`和`response`作为Fetcher实例`self`属性。为了代替指令指针,它通过注册`connnected`和`read_response`回调来保存continuation。随着应用功能的增长,我们手动保存回调的复杂性也会增加。如此繁复的记账式工作会让编码者感到头痛。 + +更糟糕的是,当我们的回调函数抛出异常会发生什么?假设我们没有写好`parse_links`方法,它在解析HTML时抛出异常: + +``` +Traceback (most recent call last): + File "loop-with-callbacks.py", line 111, in + loop() + File "loop-with-callbacks.py", line 106, in loop + callback(event_key, event_mask) + File "loop-with-callbacks.py", line 51, in read_response + links = self.parse_links() + File "loop-with-callbacks.py", line 67, in parse_links + raise Exception('parse error') +Exception: parse error +``` + +这个堆栈回溯只能显示出事件循环调用了一个回调。我们不知道是什么导致了这个错误。这条链的两边都被破坏:不知道从哪来也不知到哪去。这种丢失上下文的现象被称为"stack ripping",它还会阻止我们为回调链设置异常处理。 + +所以,除了关于多线程和异步那个更高效的争议,还有一个关于这两者之间的争论,谁更容易出错。如果在同步上出现失误,线程更容易出现数据竞争的问题,而回调因为"stack ripping"问题而非常难于调试。 + +## 协程 + +还记得我们对你许下的承诺么?我们可以写出这样的异步代码,它既有回调方式的高效,也有多线程代码的简洁。这个结合是同过一种称为协程的模式来实现的。使用Python3.4标准库asyncio和一个叫"aiohttp"的包,在协程中获取一个网页是非常直接的[^10]: + +```python + @asyncio.coroutine + def fetch(self, url): + response = yield from self.session.get(url) + body = yield from response.read() +``` + +它也是可扩展的。在Jesse系统上,与每个线程50k内存相比,一个Python协程只需要3k内存。Python很容易就可以启动上千个协程。 + +协程的概念可以追溯到计算机科学的远古时代,它很简单,一个可以暂停和恢复的子过程。线程是被操作系统控制的抢占式多任务,而协程是可合作的,它们自己选择什么时候暂停去执行下一个协程。 + +有很多协程的实现。甚至在Python中也有几种。Python3.4标准库asyncio中的协程,它是建立在生成器,一个Future类和"yield from"语句之上。从Python3.5开始,协程变成了语言本身的特性。然而,理解Python3.4中这个通过语言原有功能实现的协程,是我们处理Python3.5中原生协程的基础。 + +## 生成器如何工作 + +在你理解生成器之前,你需要知道普通的Python函数是怎么工作的。当一个函数调用一个子过程,这个被调用函数获得控制权。直到它返回或者有异常发生,才把控制权交给调用者: + +```python +>>> def foo(): +... bar() +... +>>> def bar(): +... pass +``` + +标准的Python解释器是C语言写的。一个Python函数被调用对应的C函数是`PyEval_EvalFrameEx`。它获得一个Python栈帧结构并在这个栈帧的上下文中执行Python字节码。这里是`foo`的字节码: + +```python +>>> import dis +>>> dis.dis(foo) + 2 0 LOAD_GLOBAL 0 (bar) + 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) + 6 POP_TOP + 7 LOAD_CONST 0 (None) + 10 RETURN_VALUE +``` + +`foo`函数在它栈中加载`bar`并调用它,然后把`bar`的返回值从栈中弹出,加载`None`值并返回。 + +当`PyEval_EvalFrameEx`遇到`CALL_FUNCTION`字节码时,它会创建一个新的栈帧,并用这个栈帧递归的调用`PyEval_EvalFrameEx`来执行`bar`函数。 + +图 + +非常重要的一点是,Python的栈帧在堆中分配!Python解释器是一个标准的C程序,所以他的栈帧是正常的栈帧。但是Python的栈帧是在堆中处理。这意味着Python栈帧在函数调用结束后依然可以存在。我们在`bar`函数中保存当前的栈帧,交互式的看看这种现象: + +```python +>>> import inspect +>>> frame = None +>>> def foo(): +... bar() +... +>>> def bar(): +... global frame +... frame = inspect.currentframe() +... +>>> foo() +>>> # The frame was executing the code for 'bar'. +>>> frame.f_code.co_name +'bar' +>>> # Its back pointer refers to the frame for 'foo'. +>>> caller_frame = frame.f_back +>>> caller_frame.f_code.co_name +'foo' +``` + +现在该说Python生成器了,它使用同样构件--code object和栈帧--去完成一个不可思议的任务。 + +这是一个生成器函数: + +```python +>>> def gen_fn(): +... result = yield 1 +... print('result of yield: {}'.format(result)) +... result2 = yield 2 +... print('result of 2nd yield: {}'.format(result2)) +... return 'done' +... +``` + +在Python把`gen_fn`编译成字节码的过程中,一旦它看到`yield`语句就知道这是一个生成器函数而不是普通的函数。它就会设置一个标志来记住这个事实: + +```python +>>> # The generator flag is bit position 5. +>>> generator_bit = 1 << 5 +>>> bool(gen_fn.__code__.co_flags & generator_bit) +True +``` + +当你调用一个生成器函数,Python看到这个标志,就不会运行它而是创建一个生成器: + +```python +>>> gen = gen_fn() +>>> type(gen) + +``` + +Python生成器封装了一个栈帧和函数体代码: + +```python +>>> gen.gi_code.co_name +'gen_fn' +``` + +所有通过调用`gen_fn`的生成器指向同一段代码,但都有各自的栈帧。这些栈帧不再任何一个C函数栈中,而是在堆空间中等待被使用: + +图 + +栈帧中有一个指向最后执行指令的指针。初始化为-1,意味着它没开始运行: + +```python +>>> gen.gi_frame.f_lasti +-1 +``` + +当我们调用`send`时,生成器一直运行到第一个`yield`语句处停止。并且`send`返回1,yield语句后的表达式的值。 + +```python +>>> gen.send(None) +1 +``` + +现在生成器的指令指针是3,字节码一共有56个字节: + +```python +>>> gen.gi_frame.f_lasti +3 +>>> len(gen.gi_code.co_code) +56 +``` + +这个生成器可以在任何时候,任何函数中恢复运行,因为它的栈帧并不在真正的栈中,而是堆中。在调用链中它的位置也是不确定的,它不必遵循普通函数先进后出的顺序。它像云一样自由。 + +我们可以传递一个`hello`给生成器,它会成为yield语句的结果,并且生成器运行到第二个yield语句处。 + +```python +>>> gen.send('hello') +result of yield: hello +2 +``` + +现在栈帧中包含局部变量`result`: + +```python +>>> gen.gi_frame.f_locals +{'result': 'hello'} +``` + +其它从`gen_fn`创建的生成器有着它自己的栈帧和局部变量。 + +当我们在一次调用`send`,生成器从第二个yield开始运行,以抛出一个特殊的`StopIteration`异常为结束。 + +```python +>>> gen.send('goodbye') +result of 2nd yield: goodbye +Traceback (most recent call last): + File "", line 1, in +StopIteration: done +``` + +这个异常有一个值"done",它就是生成器的返回值。 + +## 使用生成器构建协程 + +所以生成器可以暂停,可以给它一个值让它恢复,并且它还有一个返回值。这些特性看起来很适合去建立一个不使用回调的异步编程模型。我们想创造一个协程:一个在程序中可以和其他过程合作调度的过程。我们的协程将会是标准库`asyncio`中协程的一个简化版本,我们将使用生成器,futures和`yield from`语句。 + +首先,我们需要一种方法去代表协程需要等待的未来事件。一个简化的版本是: + +```python +class Future: + def __init__(self): + self.result = None + self._callbacks = [] + + def add_done_callback(self, fn): + self._callbacks.append(fn) + + def set_result(self, result): + self.result = result + for fn in self._callbacks: + fn(self) +``` + +一个future初始化为未解决的,它同过调用`set_result`来解决。[^12] + +让我们用futures和协程来改写我们的fetcher。我们之前用回调写的fetcher如下: + +```python +class Fetcher: + def fetch(self): + self.sock = socket.socket() + self.sock.setblocking(False) + try: + self.sock.connect(('xkcd.com', 80)) + except BlockingIOError: + pass + selector.register(self.sock.fileno(), + EVENT_WRITE, + self.connected) + + def connected(self, key, mask): + print('connected!') + # And so on.... +``` + +`fetch`方法开始连接一个套接字,然后注册`connected`回调函数,它会在套接字建立连接后调用。现在我们使用协程把这两步合并: + +```python + def fetch(self): + sock = socket.socket() + sock.setblocking(False) + try: + sock.connect(('xkcd.com', 80)) + except BlockingIOError: + pass + + f = Future() + + def on_connected(): + f.set_result(None) + + selector.register(sock.fileno(), + EVENT_WRITE, + on_connected) + yield f + selector.unregister(sock.fileno()) + print('connected!') +``` + +现在,`fetch`是一个生成器,因为他有一个`yield`语句。我们创建一个未决的future,然后yield它,暂停执行直到套接字连接建立。内函数`on_connected`解决这个future。 + +但是当future被解决,谁来恢复这个生成器?我们需要一个协程驱动器。让我们叫它`task`: + +```python +class Task: + def __init__(self, coro): + self.coro = coro + f = Future() + f.set_result(None) + self.step(f) + + def step(self, future): + try: + next_future = self.coro.send(future.result) + except StopIteration: + return + + next_future.add_done_callback(self.step) + +# Begin fetching http://xkcd.com/353/ +fetcher = Fetcher('/353/') +Task(fetcher.fetch()) + +loop() +``` + +task通过传递一个None值给`fetch`来启动它。`fetch`运行到它yeild一个future,这个future被task捕获作为`next_future`。当套接字连接建立,事件循环运行回调函数`on_connected`,这里future被解决,`step`被调用,生成器恢复运行。 + +## 用`yield from`重构协程 + +一旦套接字连接建立,我们就可以发送HTTP请求,然后读取服务器响应。不再需要哪些分散在各处的回调函数,我们把它们放在同一个生成器函数中: + +```python + def fetch(self): + # ... connection logic from above, then: + sock.send(request.encode('ascii')) + + while True: + f = Future() + + def on_readable(): + f.set_result(sock.recv(4096)) + + selector.register(sock.fileno(), + EVENT_READ, + on_readable) + chunk = yield f + selector.unregister(sock.fileno()) + if chunk: + self.response += chunk + else: + # Done reading. + break +``` + +从套接字中读取所有信息的代码看起来很通用。我们能不把它提取成一个子过程?现在该Python3的`yield from`登场了。它能让一个生成器委托另一个生成器。 + +让我们先回到原来那个简单的生成器: + +```python +>>> def gen_fn(): +... result = yield 1 +... print('result of yield: {}'.format(result)) +... result2 = yield 2 +... print('result of 2nd yield: {}'.format(result2)) +... return 'done' +... +``` + +为了从其他生成器调用这个生成器,我们使用`yield from`: + + +```python +>>> # Generator function: +>>> def caller_fn(): +... gen = gen_fn() +... rv = yield from gen +... print('return value of yield-from: {}' +... .format(rv)) +... +>>> # Make a generator from the +>>> # generator function. +>>> caller = caller_fn() +``` + +这个`caller`的行为的和它委派的生成器表现的完全一致: + +```python +>>> caller.send(None) +1 +>>> caller.gi_frame.f_lasti +15 +>>> caller.send('hello') +result of yield: hello +2 +>>> caller.gi_frame.f_lasti # Hasn't advanced. +15 +>>> caller.send('goodbye') +result of 2nd yield: goodbye +return value of yield-from: done +Traceback (most recent call last): + File "", line 1, in +StopIteration +``` + +注意到`caller`的指令指针保持15不变,就是`yield from`的地方,即使内部的生成器从一个yield语句运行到下一个yield,它始终不变。[^13]从`caller`外部来看,我们无法分辨yield出的值是来自`caller`还是它委派的生成器。而从`gen`内部来看,我们也不能分辨传给它的值是来自`caller`还是`caller`的外面。`yield from`是一个光滑的管道,值通过它进出`gen`,一直到`gen`结束。 + +协程可以用`yield from`把工作委派给子协程,还可以接受子协程的返回值。注意到上面的`caller`打印出"return value of yield-from: done"。当`gen`完成后,它的返回值成为`caller`中`yield from`语句的值。 + +```python + rv = yield from gen +``` + +我们批评过基于回调的异步编程模式,其中最大的不满是关于`stack ripping`:当一个回调抛出异常,它的堆栈回溯通常是毫无用处的。它只显示出事件循环运行了它,而没有说为什么。那么协程怎么样? + +```python +>>> def gen_fn(): +... raise Exception('my error') +>>> caller = caller_fn() +>>> caller.send(None) +Traceback (most recent call last): + File "", line 1, in + File "", line 3, in caller_fn + File "", line 2, in gen_fn +Exception: my error +``` + +这还是比较有用的,当异常抛出时,堆栈回溯显示出`caller_fn`委派了`gen_fn`。令人更欣慰的是,你可以像正常函数一样使用异常处理: + +```python +>>> def gen_fn(): +... yield 1 +... raise Exception('uh oh') +... +>>> def caller_fn(): +... try: +... yield from gen_fn() +... except Exception as exc: +... print('caught {}'.format(exc)) +... +>>> caller = caller_fn() +>>> caller.send(None) +1 +>>> caller.send('hello') +caught uh oh +``` + +所以我们可以像提取子过程一样提取子协程。让我们从fetcher中提取一些有用的子协程。我们先写一个可以读一块数据的协程`read`: + +```python +def read(sock): + f = Future() + + def on_readable(): + f.set_result(sock.recv(4096)) + + selector.register(sock.fileno(), EVENT_READ, on_readable) + chunk = yield f # Read one chunk. + selector.unregister(sock.fileno()) + return chunk +``` + +在`read`的基础上,`read_all`协程读取整个信息: + +```python +def read_all(sock): + response = [] + # Read whole response. + chunk = yield from read(sock) + while chunk: + response.append(chunk) + chunk = yield from read(sock) + + return b''.join(response) +``` + +如果你换个角度看,它们看起来就像在做阻塞I/O的普通函数一样。但是事实上,`read`和`read_all`都是协程。yield from`read`暂停`read_all`直到I/O操作完成。当`read_all`暂停时,事件循环正在做其它的工作等待其他的I/O操作。`read`在下次循环中完成I/O操作时,`read_all`恢复运行。 + +现在,`fetch`可以直接调用`read_all`: + +```python +class Fetcher: + def fetch(self): + # ... connection logic from above, then: + sock.send(request.encode('ascii')) + self.response = yield from read_all(sock) +``` + +神奇的是,Task类不需要做任何改变,它像以前一样驱动`fetch`协程: + +```python +Task(fetcher.fetch()) +loop() +``` + +当`read`yield一个future时,task从`yield from`管道中接受它,就像直接从`fetch`接受一样。当循环解决一个future时,task把它的结果送给`fetch`,通过管道,`read`接受到这个值,这完全就像task直接驱动`read`一样: + +图 + +亲爱的读者,我们已经完成了对asyncio协程探索。我们深入观察了生成器的机制,实现了简单的future和task。我们指出协程是如何利用两个世界的优点:比线程高效,比回调清晰。当然真正的asyncio比我们这个简化版本要复杂的多。真正的框架需要处理zero-copyI/0,公平调度,异常处理和其他大量特性。 + +使用asyncio编写协程代码比你现在看到的要简单的多。在前面的代码中,我们从基本原理去实现协程,所以你看到了回调,task和future,甚至非阻塞套接字和``select``调用。但是当用asyncio编写应用,这些都不会出现在你的代码中。我们承诺过,你可以像这样下载一个网页: + +```python + @asyncio.coroutine + def fetch(self, url): + response = yield from self.session.get(url) + body = yield from response.read() +``` + +对我们的探索还满意么?回到我们原始的任务:使用asyncio写一个网络爬虫。 + +## 使用协程 + +我们从描述爬虫如何工作开始。现在是时候用asynio去实现它了。 + +我们爬虫从获取第一个网页开始,解析出链接并把它们加到队列中。此后它开始傲游整个网站,并发的获取网页。倒是由于客户端和服务端的限制,我们希望有一个最大数目的worker。任何时候一个worker完成一个网页的获取,它应该立即从队列中取出下一个链接。我们会遇到没有事干的时候,所以worker必须能够暂停。一旦又有worker获取一个有很多链接的网页,队列会突增,暂停的worker立马被唤醒。最后,当任务完成后我们的程序必须能退出。 + +假如你的worker是线程,怎样去描述你的算法?我们可以使用Python标准库中的同步队列。每次有新的一项加入,队列增加它的tasks计数器。线程worker完成一个任务后调用`task_done`。主线程阻塞在`Queue.join`,直到tasks计数器与`task_done`调用次数相匹配,然后退出。 + +通过个一asynio队列,协程使用和线程一样的模式来实现。首先我们导入它[^6]: + +```python +try: + from asyncio import JoinableQueue as Queue +except ImportError: + # In Python 3.5, asyncio.JoinableQueue is + # merged into Queue. + from asyncio import Queue +``` + +我们把worker的共享状态收集在一个crawler类中,主要的逻辑写在`crawl`方法中。我们在一个协程中启动`crawl`,运行asyncio的事件循环直到`crawl`完成: + +```python +loop = asyncio.get_event_loop() + +crawler = crawling.Crawler('http://xkcd.com', + max_redirect=10) + +loop.run_until_complete(crawler.crawl()) +``` + +crawler用一个跟URL和最大重定向数来初始化,它把`(URL, max_redirect`)序对放入队列中。(为什么要这样做,敬请期待) + +```python +class Crawler: + def __init__(self, root_url, max_redirect): + self.max_tasks = 10 + self.max_redirect = max_redirect + self.q = Queue() + self.seen_urls = set() + + # aiohttp's ClientSession does connection pooling and + # HTTP keep-alives for us. + self.session = aiohttp.ClientSession(loop=loop) + + # Put (URL, max_redirect) in the queue. + self.q.put((root_url, self.max_redirect)) +``` + +现在队列中未完成的任务数是1。回到我们的主程序,启动事件循环和`crawl`方法: + +```python +loop.run_until_complete(crawler.crawl()) +``` +`crawl`协程唤起workers。它像一个主线程:阻塞在`join`上直到所有任务完成,同时workers在后台运行。 + +```python + @asyncio.coroutine + def crawl(self): + """Run the crawler until all work is done.""" + workers = [asyncio.Task(self.work()) + for _ in range(self.max_tasks)] + + # When all work is done, exit. + yield from self.q.join() + for w in workers: + w.cancel() +``` + +如果worker是线程,可能我们不会一次把它们全部创建出来。为了避免创建线程的昂贵代价,通常一个线程池会按需增长。但是协程很便宜,我们简单的把他们全部创建出来。 + +怎么关闭这个`crawler`很有趣。当`join`完成,worker存活但是被暂停:他们等待更多的URL。所以主协程在退出之前清除它们。否则Python解释器关闭调用所有对象的析构函数,活着的worker叫喊到: + +``` +ERROR:asyncio:Task was destroyed but it is pending! +``` + +`cancel`又是如何工作的呢?生成器还有一个我们还没介绍的特点。你可以从外部抛一个异常给它: + + +```python +>>> gen = gen_fn() +>>> gen.send(None) # Start the generator as usual. +1 +>>> gen.throw(Exception('error')) +Traceback (most recent call last): + File "", line 3, in + File "", line 2, in gen_fn +Exception: error +``` + +生成器被`throw`恢复,但是他现在抛出一个异常。如何生成器没有捕获异常的代码,这个异常被传递到顶层。所以注销一个协程: + +```python + # Method of Task class. + def cancel(self): + self.coro.throw(CancelledError) +``` + +任何时候生成器在`yield from`语句暂停,被恢复并且抛出一个异常。我们在task的`step`方法中处理撤销。 + +现在worker直到他被注销了,所以当它被销毁时,它不再抱怨。 + +一旦`crawl`注销了worker,它就退出。同时事件循环看见这个协程结束了,也就退出l。 + +```python +loop.run_until_complete(crawler.crawl()) +``` + +`crawl`方法包含了所有主协程需要做的事。而worker则完成了从队列中获取URL,获取网页,解析它们得到新的链接。每个worker独立的运行`worker`协程: + +```python + @asyncio.coroutine + def work(self): + while True: + url, max_redirect = yield from self.q.get() + + # Download page and add new links to self.q. + yield from self.fetch(url, max_redirect) + self.q.task_done() +``` + +Python看见这段代码包含`yield from`语句,就把它编译成生成器函数。所以在`crawl`方法中,我们调用了10次`self.work`,但并没有真正执行,它仅仅创建了10个生成器对象并把它们包装成Task对象。task接收生成器yield的future,通过调用send方法,future的结果做为send的参数,来驱动它。由于生成器有自己的栈帧,它们可以独立运行,独立的局部变量和指令指针。 + +worker使用队列来协调, 等待新的URL: + +```python + url, max_redirect = yield from self.q.get() +``` + +队列的`get `方法也是一个协程,它一直暂停到有新的URL进入队列。 + +碰巧,这也是最后crawl停止时,协程暂停的地方。当主协程注销worker时,从协程的角度,`yield from`抛出`CancelledError`结束了它在循环中的最后旅程。 + +worker获取一个网页,解析链接,把新的链接放入队列中,接着调用`task_done`减小计数器。最终一个worker遇到一个没有新链接的网页,并且队列里也没有任务,这次`task_done`的调用使计数器减为0,而`crawl`正阻塞在`join`方法上,现在它就可以结束了。 + +我们承诺过要解释为什么队列中要使用序对,像这样: + +```python +# URL to fetch, and the number of redirects left. +('http://xkcd.com/353', 10) +``` + +新的URL的重定向次数是10。获取一个特别的URL会重定向一个新的位置。我们减小重定向次数,并把新的URL放入队列中。 + +```python +# URL with a trailing slash. Nine redirects left. +('http://xkcd.com/353/', 9) +``` + +我们使用的`aiohttp`默认的会重定向返回最终的结果。但是,我们告诉它不要这样做,爬虫自己处理重定向。所以它可以合并那些目的相同的重定向路径:如果我们已经看到一个URL,说明它已经从其他的地方走过这条路了。 + + +```python + @asyncio.coroutine + def fetch(self, url, max_redirect): + # Handle redirects ourselves. + response = yield from self.session.get( + url, allow_redirects=False) + + try: + if is_redirect(response): + if max_redirect > 0: + next_url = response.headers['location'] + if next_url in self.seen_urls: + # We have been down this path before. + return + + # Remember we have seen this URL. + self.seen_urls.add(next_url) + + # Follow the redirect. One less redirect remains. + self.q.put_nowait((next_url, max_redirect - 1)) + else: + links = yield from self.parse_links(response) + # Python set-logic: + for link in links.difference(self.seen_urls): + self.q.put_nowait((link, self.max_redirect)) + self.seen_urls.update(links) + finally: + # Return connection to pool. + yield from response.release() +``` + +如果这是多进程代码,就有可能遇到讨厌的竞争条件。比如,一个work检查一个链接是否在`seen_urls`中,如果没有它就把这个链接加到队列中并把它放到`seen_urls`中。如果它在这两步操作之间被中断,而另一个work解析到相同的链接,发现它并没有出现在`seen_urls`中就把它加入队列中。这导致同样的链接在队列中出现两次,做了重复的工作和错误的统计。 + +然而,一个协程只在`yield from`是才会被中断。这是协程比多线程少遇到竞争条件的关键。多线程必须获得锁来明确的进入一个临界区,否则它就是可中断的。而Python的协程默认是不会被中断的,只有它yield主动放弃控制权。 + +我们不再需要在用回调方式时的fetcher类了。这个类只是不高效回调的一个变通方法:在等待I/O时,它需要一个存储状态的地方,因为局部变量并不能在函数调用间保留。倒是`fetch`协程可以像普通函数一样用局部变量保存它的状态,所以我们不再需要一个类。 + +当`fetch`完成对服务器回应的处理,它返回到调用它的work。work调用`task_done`,接着从队列中取出一个URL。 + +当`fetch`把新的链接放入队列中,它增加未完成的任务计数器。主协程在等待`q.join`。而当没有新的链接并且这是队列中最后一个URL,work调用`task_done`,任务计数器变为0,主协程从`join`中退出。 + +与work和主协程一起工作的队列代码像这样: + +```python +class Queue: + def __init__(self): + self._join_future = Future() + self._unfinished_tasks = 0 + # ... other initialization ... + + def put_nowait(self, item): + self._unfinished_tasks += 1 + # ... store the item ... + + def task_done(self): + self._unfinished_tasks -= 1 + if self._unfinished_tasks == 0: + self._join_future.set_result(None) + + @asyncio.coroutine + def join(self): + if self._unfinished_tasks > 0: + yield from self._join_future +``` + +主协程`crawl`yield from`join`。所以当最后一个workd把计数器减为0,它告诉`crawl`恢复运行。 + +旅程快要结束了。我们的程序从`crawl`调用开始: + +```python +loop.run_until_complete(self.crawler.crawl()) +``` + +程序如何结束?因为`crawl`是一个生成器函数。调用它返回一个生成器。为了驱动它,asyncio把它包装成一个task: + + +class EventLoop: + def run_until_complete(self, coro): + """Run until the coroutine is done.""" + task = Task(coro) + task.add_done_callback(stop_callback) + try: + self.run_forever() + except StopError: + pass + +class StopError(BaseException): + """Raised to stop the event loop.""" + +def stop_callback(future): + raise StopError +``` + +当这个任务完成,它抛出`StopError`, 事件循环把这个异常当作正常退出的信号。 + +但是,task的`add_done_callbock`和`result`方法又是什么呢?你可能认为task就像一个future,不错,你的直觉是对的。我们必须承认一个向你隐藏的细节,task是future。 + +```python +class Task(Future): + """A coroutine wrapped in a Future.""" +``` + +通常,一个future被别人调用`set_result`。但是task,当协程结束时,它自己解决自己。记得我们解释过当Python生成器返回时,它抛出一个特殊的`StopIteration`异常: + +```python + # Method of class Task. + def step(self, future): + try: + next_future = self.coro.send(future.result) + except CancelledError: + self.cancelled = True + return + except StopIteration as exc: + + # Task resolves itself with coro's return + # value. + self.set_result(exc.value) + return + + next_future.add_done_callback(self.step) +``` + +所以当事件循环调用`task.add_done_callback(stop_callback)`,它就准备被这个task结束。在看一次`run_until_complete`; + +```python + # Method of event loop. + def run_until_complete(self, coro): + task = Task(coro) + task.add_done_callback(stop_callback) + try: + self.run_forever() + except StopError: + pass +``` + +当task捕获`StopIteration`并解决自己,这个回调重循环中抛出`StopError`。循环结束调用栈回到`run_until_complete`。我们的程序结束。 + +## 总结 + +现代的程序越来越多是I/O密集型而不是CPU密集型。对于这样的程序,Python的线程和不合适:全局锁阻止真正的并行计算,并且抢占切换也导致他们更容易出现竞争。异步通常是正确的选择。但是随着基于回调的异步代码增加,它会变得非常混乱。协程是一个更整洁的替代者。它们自然的重构成子过程,有健全的异常处理和栈追溯。 + +如果我们换个角度看`yield from`语句,一个协程看起来像一个传统的线程。甚至我们采用金典的多线程模式编程,不需要重新发明。因此,与回调相比,协程更适合有经验的多线程的编码者。 + +但是当我们打开眼睛关注`yield from`语句,我们能看到协程放弃控制权的标志点。不像多线程,协程展示出我们的代码哪里可以被中断哪里不能。在Glyph Lefkowitz富有启发性的文章"Unyielding"[^4]:"线程让局部推理变得困难,然而局部推理可能是软件开发中最重要的事"。然而,明确的yield,让"通过子过程本身而不是整个系统理解它的行为(应此,正确性)"成为可能。 + +这章写于Python和异步的复兴时期。你刚学到的基于生成器的的协程,在2014年发布在Python 3.4的"asyncio"模块。2015年9月,Python 3.5发布,协程成为语言的一部分。这个原生的协程通过"async def"来声明, 使用"await"而不是"yield from"委托一个协程或者等待Future。 + + +除了这些优点,核心的思想不变。Python新的原生协程与生成器只是在语法上不同,工作原理非常相似。事实上,在Python解释器中它们公用同一个实现方法。Task,Future和事件循环扮演这在asynico中同样的角色。 + +你已经知道asyncio协程是如何工作的了,现在你可以忘记大部分的细节。这些机制隐藏在一个整洁的接口下。但是你对这基本原理的理解能让你在现代异步环境下正确而高效的编写代码。 + +[^4]: [https://glyph.twistedmatrix.com/2014/02/unyielding.html](https://glyph.twistedmatrix.com/2014/02/unyielding.html) + +[^5]: [https://docs.python.org/3/library/queue.html](https://docs.python.org/3/library/queue.html) + +[^6]: [https://docs.python.org/3/library/asyncio-sync.html](https://docs.python.org/3/library/asyncio-sync.html) + +[^7]: For a complex solution to this problem, see [http://www.tornadoweb.org/en/stable/stack_context.html](http://www.tornadoweb.org/en/stable/stack_context.html) + +[^8]: [http://www.kegel.com/c10k.html](http://www.kegel.com/c10k.html) + +[^9]: The actual `asyncio.Queue` implementation uses an `asyncio.Event` in place of the Future shown here. The difference is an Event can be reset, whereas a Future cannot transition from resolved back to pending. + +[^10]: The `@asyncio.coroutine` decorator is not magical. In fact, if it decorates a generator function and the `PYTHONASYNCIODEBUG` environment variable is not set, the decorator does practically nothing. It just sets an attribute, `_is_coroutine`, for the convenience of other parts of the framework. It is possible to use asyncio with bare generators not decorated with `@asyncio.coroutine` at all. + + +[^11]: Jesse listed indications and contraindications for using async in "What Is Async, How Does It Work, And When Should I Use It?", available at pyvideo.org. +[^bayer]: Mike Bayer compared the throughput of asyncio and multithreading for different workloads in his "Asynchronous Python and Databases": http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ + + +[^11]: Jesse listed indications and contraindications for using async in ["What Is Async, How Does It Work, And When Should I Use It?":](http://pyvideo.org/video/2565/what-is-async-how-does-it-work-and-when-should). Mike Bayer compared the throughput of asyncio and multithreading for different workloads in ["Asynchronous Python and Databases":](http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/) + + +[^12]: This future has many deficiencies. For example, once this future is resolved, a coroutine that yields it should resume immediately instead of pausing, but with our code it does not. See asyncio's Future class for a complete implementation. + +[^13]: In fact, this is exactly how "yield from" works in CPython. A function increments its instruction pointer before executing each statement. But after the outer generator executes "yield from", it subtracts 1 from its instruction pointer to keep itself pinned at the "yield from" statement. Then it yields to *its* caller. The cycle repeats until the inner generator throws `StopIteration`, at which point the outer generator finally allows itself to advance to the next instruction. + +[^14]: Python's global interpreter lock prohibits running Python code in parallel in one process anyway. Parallelizing CPU-bound algorithms in Python requires multiple processes, or writing the parallel portions of the code in C. But that is a topic for another day. + +[^15]: Even calls to `send` can block, if the recipient is slow to acknowledge outstanding messages and the system's buffer of outgoing data is full. + + +[^16]: Guido introduced the standard asyncio library, called "Tulip" then, at [PyCon 2013](http://pyvideo.org/video/1667/keynote). + + +[^16]: Guido introduced the standard asyncio library, called "Tulip" then, at PyCon 2013. + + +[^17]: Python 3.5's built-in coroutines are described in [PEP 492](https://www.python.org/dev/peps/pep-0492/), "Coroutines with async and await syntax." + +-------------------------------------- +via: http://aosabook.org/en/500L/pages/a-web-crawler-with-asyncio-coroutines.html + +作者: A. Jesse Jiryu Davis and Guido van Rossum +译者:[qingyunha](https://github.com/qingyunha) +校对: + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](http://linux.cn/) 荣誉推出 From e3e8a3b3418d36eb494b92d00e125b24e847a701 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 20:23:41 +0800 Subject: [PATCH 0142/2437] =?UTF-8?q?20160917-1=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...ke LibreOffice and OpenOffice one again.md | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md diff --git a/sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md b/sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md new file mode 100644 index 0000000000..3aba30f29b --- /dev/null +++ b/sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md @@ -0,0 +1,64 @@ +It's time to make LibreOffice and OpenOffice one again +========== + +![](http://tr2.cbsistatic.com/hub/i/2016/09/14/2e91089b-7ebd-4579-bf8f-74c34d1a94ce/e7e9c8dd481d8e068f2934c644788928/openofficedeathhero.jpg) + +Let's talk about OpenOffice. More than likely you've already read, countless times, that Apache OpenOffice is near the end. The last stable iteration was 4.1.2 (released October, 2015) and a recent major security flaw took a month to patch. A lack of coders has brought development to a creeping crawl. And then, the worst possible news hit the ether; the project suggested users switch to MS Office (or LibreOffice). + +For whom the bells tolls? The bell tolls for thee, OpenOffice. + +I'm going to say something that might ruffle a few feathers. Are you ready for it? + +The end of OpenOffice will be a good thing for open source and for users. + +Let me explain. + +### One fork to rule them all + +When LibreOffice was forked from OpenOffice we saw yet another instance of the fork not only improving on the original, but vastly surpassing it. LibreOffice was an instant success. Every Linux distribution that once shipped with OpenOffice migrated to the new kid on the block. LibreOffice burst out of the starting gate and immediately hit its stride. Updates came at an almost breakneck speed and the improvements were plenty and important. + +After a while, OpenOffice became an afterthought for the open source community. This, of course, was exacerbated when Oracle decided to discontinue the project in 2011 and donated the code to the Apache Project. By this point OpenOffice was struggling to move forward and that brings us to now. A burgeoning LibreOffice and a suffering, stuttering OpenOffice. + +But I say there is a light at the end of this rather dim tunnel. + +### Unfork them + +This may sound crazy, but I think it's time LibreOffice and OpenOffice became one again. Yes, I know there are probably political issues and egos at stake, but I believe the two would be better served as one. The benefits of this merger would be many. Off the top of my head: + +- Bring the MS Office filters together: OpenOffice has a strong track record of better importing certain files from MS Office (whereas LibreOffice has been known to be improving, but spotty). +- More developers for LibreOffice: Although OpenOffice wouldn't bring with it a battalion of developers, it would certainly add to the mix. +- End the confusion: Many users assume OpenOffice and LibreOffice are the same thing. Some don't even know that LibreOffice exists. This would end that confusion. +- Combine their numbers: Separate, OpenOffice and LibreOffice have impressive usage numbers. Together, they would be a force. +### A golden opportunity + +The possible loss of OpenOffice could actually wind up being a golden opportunity for open source office suites in general. Why? I would like to suggest something that I believe has been necessary for a while now. If OpenOffice and LibreOffice were to gather their forces, diff their code, and merge, they could then do some much-needed retooling of not just the internal works of the whole, but also of the interface. + +Let's face it, the LibreOffice and (by extension) OpenOffice UIs are both way out of date. When I install LibreOffice 5.2.1.2 the tool bar is an absolute disaster (Figure A). + +### Figure A + +![](http://tr2.cbsistatic.com/hub/i/2016/09/14/cc5250df-48cd-40e3-a083-34250511ffab/c5ac8eb1e2cb12224690a6a3525999f0/openofficea.jpg) + +#### The LibreOffice default toolbar setup. + +As much as I support and respect (and use daily) LibreOffice, it has become all too clear the interface needs a complete overhaul. What we're dealing with now is a throwback to the late 90s/early 2000s and it has to go. When a new user opens up LibreOffice for the first time, they are inundated with buttons, icons, and toolbars. Ubuntu Unity helped this out with the Head up Display (HUD), but that did nothing for other desktops and distributions. Sure, the enlightened user has no problem knowing what to look for and where it is (or to even customize the toolbars to reflect their specific needs), but for a new or average user, that interface is a nightmare. Now would be the perfect time for this change. Bring in the last vestiges of the OpenOffice developers and have them join the fight for an improved interface. With the combination of the additional import filters from OpenOffice and a modern interface, LibreOffice could finally make some serious noise on both the home and business desktops. + +### Will this actually happen? + +This needs to happen. Will it? I have no idea. But even if the powers that be decide the UI isn't in need of retooling (which would be a mistake), bringing OpenOffice into the fold would still be a big step forward. The merging of the two efforts would bring about a stronger focus on development, easier marketing, and far less confusion by the public at large. + +I realize this might seem a bit antithetical to the very heart and spirit of open source, but merging LibreOffice and OpenOffice would combine the strengths of the two constituent pieces and possibly jettison the weaknesses. + +From my perspective, that's a win-win. + +-------------------------------------------------------------------------------- + +via: http://www.techrepublic.com/article/its-time-to-make-libreoffice-and-openoffice-one-again/ + +作者:[Jack Wallen ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.techrepublic.com/search/?a=jack%2Bwallen From e0c3f4b8befaaefe2a2cc7523260099c7214e5d0 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 20:29:59 +0800 Subject: [PATCH 0143/2437] =?UTF-8?q?20160917-2=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...for building containerized applications.md | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 sources/tech/20160912 8 best practices for building containerized applications.md diff --git a/sources/tech/20160912 8 best practices for building containerized applications.md b/sources/tech/20160912 8 best practices for building containerized applications.md new file mode 100644 index 0000000000..d67ee5c473 --- /dev/null +++ b/sources/tech/20160912 8 best practices for building containerized applications.md @@ -0,0 +1,74 @@ +8 best practices for building containerized applications +==== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/containers_2015-2-osdc-lead.png?itok=0yid3gFY) + +Containers are a major trend in deploying applications in both public and private clouds. But what exactly are containers, why have they become a popular deployment mechanism, and how will you need to modify your application to optimize it for a containerized environment? + +### What are containers? + +The technology behind containers has a long history beginning with SELinux in 2000 and Solaris zones in 2005. Today, containers are a combination of several kernel features including SELinux, Linux namespaces, and control groups, providing isolation of end user processes, networking, and filesystem space. + +### Why are they so popular? + +The recent widespread adoption of containers is largely due to the development of standards aimed at making them easier to use, such as the Docker image format and distribution model. This standard calls for immutable images, which are the launching point for a container runtime. Immutable images guarantee that the same image the development team releases is what gets tested and deployed into the production environment. + +The lightweight isolation that containers provide creates a better abstraction for an application component. Components running in containers won't interfere with each other the way they might running directly on a virtual machine. They can be prevented from starving each other of system resources, and unless they are sharing a persistent volume won't block attempting to write to the same files. Containers have helped to standardize practices like logging and metric collection, and they allow for increased multi-tenant density on physical and virtual machines, all of which leads to lower deployment costs. + +### How do you build a container-ready application? + +Changing your application to run inside of a container isn't necessarily a requirement. The major Linux distributions have base images that can run anything that runs on a virtual machine. But the general trend in containerized applications is following a few best practices: + +- 1. Instances are disposable + +Any given instance of your application shouldn't need to be carefully kept running. If one system running a bunch of containers goes down, you want to be able to spin up new containers spread out across other available systems. + +- 2. Retry instead of crashing + +When one service in your application depends on another service, it should not crash when the other service is unreachable. For example, your API service is starting up and detects the database is unreachable. Instead of failing and refusing to start, you design it to retry the connection. While the database connection is down the API can respond with a 503 status code, telling the clients that the service is currently unavailable. This practice should already be followed by applications, but if you are working in a containerized environment where instances are disposable, then the need for it becomes more obvious. + +- 3. Persistent data is special + +Containers are launched based on shared images using a copy-on-write (COW) filesystem. If the processes the container is running choose to write out to files, then those writes will only exist as long as the container exists. When the container is deleted, that layer in the COW filesystem is deleted. Giving a container a mounted filesystem path that will persist beyond the life of the container requires extra configuration, and extra cost for the physical storage. Clearly defining the abstraction for what storage is persisted promotes the idea that instances are disposable. Having the abstraction layer also allows a container orchestration engine to handle the intricacies of mounting and unmounting persistent volumes to the containers that need them. + +- 4. Use stdout not log files + +You may now be thinking, if persistent data is special, then what do I do with log files? The approach the container runtime and orchestration projects have taken is that processes should instead [write to stdout/stderr][1], and have infrastructure for archiving and maintaining [container logs][2]. + +- 5. Secrets (and other configurations) are special too + +You should never hard-code secret data like passwords, keys, and certificates into your images. Secrets are typically not the same when your application is talking to a development service, a test service, or a production service. Most developers do not have access to production secrets, so if secrets are baked into the image then a new image layer will have to be created to override the development secrets. At this point, you are no longer using the same image that was created by your development team and tested by quality engineering (QE), and have lost the benefit of immutable images. Instead, these values should be abstracted away into environment variables or files that are injected at container startup. + +- 6. Don't assume co-location of services + +In an orchestrated container environment you want to allow the orchestrator to send your containers to whatever node is currently the best fit. Best fit could mean a number of things: it could be based on whichever node has the most space right now, the quality of service the container is requesting, whether the container requires persistent volumes, etc. This could easily mean your frontend, API, and database containers all end up on different nodes. While it is possible to force an API container to each node (see [DaemonSets][3] in Kubernetes), this should be reserved for containers that perform tasks like monitoring the nodes themselves. + +- 7. Plan for redundancy / high availability + +Even if you don't have enough load to require an HA setup, you shouldn't write your service in a way that prevents you from running multiple copies of it. This will allow you to use rolling deployments, which make it easy to move load off one node and onto another, or to upgrade from one version of a service to the next without taking any downtime. + +- 8. Implement readiness and liveness checks + +It is common for applications to have startup time before they are able to respond to requests, for example, an API server that needs to populate in-memory data caches. Container orchestration engines need a way to check that your container is ready to serve requests. Providing a readiness check for new containers allows a rolling deployment to keep an old container running until it is no longer needed, preventing downtime. Similarly, a liveness check is a way for the orchestration engine to continue to check that the container is in a healthy state. It is up to the application creator to decide what it means for their container to be healthy, or "live". A container that is no longer live will be killed, and a new container created in its place. + +### Want to find out more? + +I'll be at the Grace Hopper Celebration of Women in Computing in October, come check out my talk: [Containerization of Applications: What, Why, and How][4]. Not headed to GHC this year? Then read on about containers, orchestration, and applications on the [OpenShift][5] and [Kubernetes][6] project sites. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/life/16/9/8-best-practices-building-containerized-applications + +作者:[Jessica Forrester ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/jwforres +[1]: https://docs.docker.com/engine/reference/commandline/logs/ +[2]: http://kubernetes.io/docs/getting-started-guides/logging/ +[3]: http://kubernetes.io/docs/admin/daemons/ +[4]: https://www.eiseverywhere.com/ehome/index.php?eventid=153076&tabid=351462&cid=1350690&sessionid=11443135&sessionchoice=1& +[5]: https://www.openshift.org/ +[6]: http://kubernetes.io/ From 0e23269b58adfd4b558569002db7f396e4173b32 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 20:40:24 +0800 Subject: [PATCH 0144/2437] =?UTF-8?q?20160917-3=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...Monitoring with Shinken on Ubuntu 16.04.md | 364 ++++++++++++++++++ 1 file changed, 364 insertions(+) create mode 100644 sources/tech/20160914 Server Monitoring with Shinken on Ubuntu 16.04.md diff --git a/sources/tech/20160914 Server Monitoring with Shinken on Ubuntu 16.04.md b/sources/tech/20160914 Server Monitoring with Shinken on Ubuntu 16.04.md new file mode 100644 index 0000000000..9580189e1e --- /dev/null +++ b/sources/tech/20160914 Server Monitoring with Shinken on Ubuntu 16.04.md @@ -0,0 +1,364 @@ +Server Monitoring with Shinken on Ubuntu 16.04 +===== + + +Shinken is an open source computer and network monitoring framework written in python and compatible with Nagios. Shinken can be used on all operating systems that can run python applications like Linux, Unix, and Windows. Shinken was written by Jean Gabes as proof of concept for a new Nagios architecture, but it was turned down by the Nagios author and became an independent network and system monitoring tool that stays compatible with Nagios. + +In this tutorial, I will show you how to install Shinken from source and add a Linux host to the monitoring system. I will use Ubuntu 16.04 Xenial Xerus as the operating system for the Shinken server and monitored host. + +### Step 1 - Install Shinken Server + +Shinken is a python framework, we can install it with pip or install it from source. In this step, we will install Shinken from source. + +There are some tasks that have to be completed before we start installing Shinken. + +Install some new python packages and create Linux user with the name "shinken": + +``` +sudo apt-get install python-setuptools python-pip python-pycurl +useradd -m -s /bin/bash shinken +``` + +Download the Shinken source from GitHub repository: + +``` +git clone https://github.com/naparuba/shinken.git +cd shinken/ +``` + +Then install Shinken with the command below: + +``` +git checkout 2.4.3 +python setup.py install +``` + +Next, for better results, we need to install 'python-cherrypy3' from the ubuntu repository: + +``` +sudo apt-get install python-cherrypy3 +``` + +Now Shinken is installed, next we add Shinken to start at boot time and start it: + +``` +update-rc.d shinken defaults +systemctl start shinken +``` + +### Step 2 - Install Shinken Webui2 + +Webui2 is the Shinken web interface available from shinken.io. The easiest way to install Sshinken webui2 is by using the shinken CLI command (which has to be executed as shinken user). + +Login to the shinken user: + +``` +su - shinken +``` + +Initialize the shinken configuration file - The command will create a new configuration .shinken.ini: + +``` +shinken --init +``` + +And install webui2 with this shinken CLI command: + +``` +shinken install webui2 +``` + +![](https://www.howtoforge.com/images/server-monitoring-with-shinken-on-ubuntu-16-04/6.png) + +Webui2 is installed, but we need to install MongoDB and another python package with pip. Run command below as root: + +``` +sudo apt-get install mongodb +pip install pymongo>=3.0.3 requests arrow bottle==0.12.8 +``` + +Next, go to the shinken directory and add the new webui2 module by editing the 'broker-master.cfg' file: + +``` +cd /etc/shinken/brokers/ +vim broker-master.cfg +``` + +Add a new option inside module on line 40: + +``` +modules webui2 +``` + +Save the file and exit the editor. + +Now go to the contacts directory and edit the file 'admin.cfg' for the admin configuration. + +``` +cd /etc/shinken/contacts/ +vim admin.cfg +``` + +Change the values shown below: + +``` +contact_name admin # Username 'admin' +password yourpass # Pass 'mypass' +``` + +Save and exit. + +### Step 3 - Install Nagios-plugins and Shinken Packages + +In this step, we will install Nagios-plugins and some Perl module. Then install additional shinken packages from shinken.io to perform the monitoring. + +Install Nagios-plugins and cpanminus which is required for building and installing the Perl modules: + +``` +sudo apt-get install nagios-plugins* cpanminus +``` + +Install these Perl modules with the cpanm command: + +``` +cpanm Net::SNMP +cpanm Time::HiRes +cpanm DBI +``` + +Now create new link for utils.pm file to shinken the directory and create a new directory for Log_File_Health: + +``` +chmod u+s /usr/lib/nagios/plugins/check_icmp +ln -s /usr/lib/nagios/plugins/utils.pm /var/lib/shinken/libexec/ +mkdir -p /var/log/rhosts/ +touch /var/log/rhosts/remote-hosts.log +``` + +Next, install the shinken packages ssh and linux-snmp for monitoring SSH and SNMP sources from shinken.io: + +``` +su - shinken +shinken install ssh +shinken install linux-snmp +``` + +### Step 4 - Add a New Linux Host/host-one + +We will add a new Linux host that shall be monitored by using an Ubuntu 16.04 server with IP address 192.168.1.121 and hostname 'host-one'. + +Connect to the Linux host-one: + +``` +ssh host1@192.168.1.121 +``` + +Install the snmp and snmpd packages from the Ubuntu repository: + +``` +sudo apt-get install snmp snmpd +``` + +Next, edit the configuration file 'snmpd.conf' with vim: + +``` +vim /etc/snmp/snmpd.conf +``` + +Comment line 15 and uncomment line 17: + +``` +#agentAddress udp:127.0.0.1:161 +agentAddress udp:161,udp6:[::1]:161 +``` + +Comment line 51 and 53, then add new line configuration below: + +``` +#rocommunity mypass default -V systemonly +#rocommunity6 mypass default -V systemonly + +rocommunity mypass +``` + +Save and exit. + +Now start the snmpd service with the systemctl command: + +``` +systemctl start snmpd +``` + +Go to the shinken server and define the new host by creating a new file in the 'hosts' directory. + +``` +cd /etc/shinken/hosts/ +vim host-one.cfg +``` + +Paste configuration below: + +``` +define host{ + use generic-host,linux-snmp,ssh + contact_groups admins + host_name host-one + address 192.168.1.121 + _SNMPCOMMUNITY mypass # SNMP Pass Config on snmpd.conf + } +``` + +Save and exit. + +Edit the SNMP configuration on the Shinken server: + +``` +vim /etc/shinken/resource.d/snmp.cfg +``` + +Change 'public' to 'mypass' - must be the same password that you used in the snmpd configuration file on the client host-one. + +``` +$SNMPCOMMUNITYREAD$=mypass +``` + +Save and exit. + +Now reboot both servers - Shinken server and the monitored Linux host: + +``` +reboot +``` + +The new Linux host has been added successfully to the Shinken server. + +### Step 5 - Access Shinken Webui2 + +Visit the Shinken webui2 on port 7677 (replace the IP in the URL with your IP): + +``` +http://192.168.1.120:7767 +``` + +Log in with user admin and your password (the one that you have set in the admin.cfg configuration file). + +![](https://www.howtoforge.com/images/server-monitoring-with-shinken-on-ubuntu-16-04/1.png) + +Shinken Dashboard in Webui2. + +![](https://www.howtoforge.com/images/server-monitoring-with-shinken-on-ubuntu-16-04/2.png) + +Our 2 servers are monitored with Shinken. + +![](https://www.howtoforge.com/images/server-monitoring-with-shinken-on-ubuntu-16-04/3.png) + +List all services that are monitored with linux-snmp. + +![](https://www.howtoforge.com/images/server-monitoring-with-shinken-on-ubuntu-16-04/4.png) + +Status of all hosts and services. + + +![](https://www.howtoforge.com/images/server-monitoring-with-shinken-on-ubuntu-16-04/5.png) + +### Step 6 - Common Problems with Shinken + +- Problems with the NTP server + +When you get this error with NTP. + +``` +TimeSync - CRITICAL ( NTP CRITICAL: No response from the NTP server) +TimeSync - CRITICAL ( NTP CRITICAL: Offset unknown ) +``` + +To solve this problem, install ntp on all Linux hosts. + +``` +sudo apt-get install ntp ntpdate +``` + +Edit the ntp configuration: + +``` +vim /etc/ntp.conf +``` + +Comment all the pools and replace it with: + +``` +#pool 0.ubuntu.pool.ntp.org iburst +#pool 1.ubuntu.pool.ntp.org iburst +#pool 2.ubuntu.pool.ntp.org iburst +#pool 3.ubuntu.pool.ntp.org iburst + +pool 0.id.pool.ntp.org +pool 1.asia.pool.ntp.org +pool 0.asia.pool.ntp.org +``` + +Next, add a new line inside restrict: + +``` +# Local users may interrogate the ntp server more closely. +restrict 127.0.0.1 +restrict 192.168.1.120 #shinken server IP address +restrict ::1 +NOTE: 192.168.1.120 is the Shinken server IP address. +``` + +Save and exit. + +Start ntp and check the Shinken dashboard: + +``` +ntpd +``` + +- Problem check_netint.pl Not Found + +Download the source from the github repository to the shinken lib directory: + +``` +cd /var/lib/shinken/libexec/ +wget https://raw.githubusercontent.com/Sysnove/shinken-plugins/master/check_netint.pl +chmod +x check_netint.pl +chown shinken:shinken check_netint.pl +``` + +- Problem with NetworkUsage + +There is error message: + +``` +ERROR : Unknown interface eth\d+ +``` + +Check your network interface and edit the linux-snmp template. + +On my Ubuntu server, the network interface is 'enp0s8', not eth0, so I got this error. + +Edit the linux-snmp template packs with vim: + +``` +vim /etc/shinken/packs/linux-snmp/templates.cfg +``` + +Add the network interface to line 24: + +``` +_NET_IFACES eth\d+|em\d+|enp0s8 +``` + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/tutorial/server-monitoring-with-shinken-on-ubuntu-16-04/ + +作者:[Muhammad Arul][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.howtoforge.com/tutorial/server-monitoring-with-shinken-on-ubuntu-16-04/ +Save and exit. From 6eabafb53c63934808ebfc66872ba08a9bf60c0a Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 20:46:03 +0800 Subject: [PATCH 0145/2437] =?UTF-8?q?20160917-4=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...he Headphone Jack Here Are Our Thoughts.md | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 sources/talk/20160915 Should Smartphones Do Away with the Headphone Jack Here Are Our Thoughts.md diff --git a/sources/talk/20160915 Should Smartphones Do Away with the Headphone Jack Here Are Our Thoughts.md b/sources/talk/20160915 Should Smartphones Do Away with the Headphone Jack Here Are Our Thoughts.md new file mode 100644 index 0000000000..0bf4b144f9 --- /dev/null +++ b/sources/talk/20160915 Should Smartphones Do Away with the Headphone Jack Here Are Our Thoughts.md @@ -0,0 +1,54 @@ +Should Smartphones Do Away with the Headphone Jack? Here Are Our Thoughts +==== + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/09/Writers-Opinion-Headphone-Featured.jpg) + +Even though Apple removing the headphone jack from the iPhone 7 has been long-rumored, after the official announcement last week confirming the news, it has still become a hot topic. + +For those not in the know on this latest news, Apple has removed the headphone jack from the phone, and the headphones will now plug into the lightning port. Those that want to still use their existing headphones may, as there is an adapter that ships with the phone along with the lightning headphones. They are also selling a new product: AirPods. These are wireless and are inserted into your ear. The biggest advantage is that by eliminating the jack they were able to make the phone dust and water-resistant. + +Being it’s such a big story right now, we asked our writers, “What are your thoughts on Smartphones doing away with the headphone jack?” + +### Our Opinion + +Derrik believes that “Apple’s way of doing it is a play to push more expensive peripherals that do not comply to an open standard.” He also doesn’t want to have to charge something every five hours, meaning the AirPods. While he understands that the 3.5mm jack is aging, as an “audiophile” he would love a new, open standard, but “proprietary pushes” worry him about device freedom. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/09/headphone-jacks.jpg) + +Damien doesn’t really even use the headphone jack these days as he has Bluetooth headphones. He hates that wire anyway, so feels “this is a good move.” Yet he also understands Derrik’s point about the wireless headphones running out of battery, leaving him with “nothing to fall back on.” + +Trevor is very upfront in saying he thought it was “dumb” until he heard you couldn’t charge the phone and use the headphones at the same time and realized it was “dumb X 2.” He uses the headphones/headphone jack all the time in a work van without Bluetooth and listens to audio or podcasts. He uses the plug-in style as Bluetooth drains his battery. + +Simon is not a big fan. He hasn’t seen much reasoning past it leaving more room within the device. He figures “it will then come down to whether or not consumers favor wireless headphones, an adapter, and water-resistance over not being locked into AirPods, lightning, or an adapter”. He fears it might be “too early to jump into removing ports” and likes a “one pair fits all” standard. + +James believes that wireless technology is progressive, so he sees it as a good move “especially for Apple in terms of hardware sales.” He happens to use expensive headphones, so personally he’s “yet to be convinced,” noting his Xperia is waterproof and has a jack. + +Jeffry points out that “almost every transition attempt in the tech world always starts with strong opposition from those who won’t benefit from the changes.” He remembers the flak Apple received when they removed the floppy disk drive and decided not to support Flash, and now both are industry standards. He believes everything is evolving for the better, removing the audio jack is “just the first step toward the future,” and Apple is just the one who is “brave enough to lead the way (and make a few bucks in doing so).” + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/09/Writers-Opinion-Headphone-Headset.jpg) + +Vamsi doesn’t mind the removal of the headphone jack as long as there is a “better solution applicable to all the users that use different headphones and other gadgets.” He doesn’t feel using headphones via a lightning port is a good solution as it renders nearly all other headphones obsolete. Regarding Bluetooth headphones, he just doesn’t want to deal with another gadget. Additionally, he doesn’t get the argument of it being good for water resistance since there are existing devices with headphone jacks that are water resistant. + +Mahesh prefers a phone with a jack because many times he is charging his phone and listening to music simultaneously. He believes we’ll get to see how it affects the public in the next few months. + +Derrik chimed back in to say that by “removing open standard ports and using a proprietary connection too.,” you can be technical and say there are adapters, but Thunderbolt is also closed, and Apple can stop selling those adapters at any time. He also notes that the AirPods won’t be Bluetooth. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/09/Writers-Opinion-Headphone-AirPods.jpg) + +As for me, I’m always up for two things: New technology and anything Apple. I’ve been using iPhones since a few weeks past the very first model being introduced, yet I haven’t updated since 2012 and the iPhone 5, so I was overdue. I’ll be among the first to get my hands on the iPhone 7. I hate that stupid white wire being in my face, so I just might be opting for AirPods at some point. I am very appreciative of the phone becoming water-resistant. As for charging vs. listening, the charge on new iPhones lasts so long that I don’t expect it to be much of a problem. Even my old iPhone 5 usually lasts about twenty hours on a good day and twelve hours on a bad day. So I don’t expect that to be a problem. + +### Your Opinion + +Our writers have given you a lot to think about. What are your thoughts on Smartphones doing away with the headphone jack? Will you miss it? Is it a deal breaker for you? Or do you relish the upgrade in technology? Will you be trying the iPhone 5 or the AirPods? Let us know in the comments below. + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/should-smartphones-do-away-with-the-headphone-jack/?utm_medium=feed&utm_source=feedpress.me&utm_campaign=Feed%3A+maketecheasier + +作者:[Laura Tucker][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.maketecheasier.com/author/lauratucker/ From c037ad1f28a2a0b23168821a902facfe02e90212 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 20:50:49 +0800 Subject: [PATCH 0146/2437] =?UTF-8?q?20160917-5=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...f permissive open source licenses means.md | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 sources/talk/20160915 What the rise of permissive open source licenses means.md diff --git a/sources/talk/20160915 What the rise of permissive open source licenses means.md b/sources/talk/20160915 What the rise of permissive open source licenses means.md new file mode 100644 index 0000000000..9a17fec1e0 --- /dev/null +++ b/sources/talk/20160915 What the rise of permissive open source licenses means.md @@ -0,0 +1,57 @@ +What the rise of permissive open source licenses means +==== + +Why restrictive licenses such as the GNU GPL are steadily falling out of favor. + +"If you use any open source software, you have to make the rest of your software open source." That's what former Microsoft CEO Steve Ballmer said back in 2001, and while his statement was never true, it must have spread some FUD (fear, uncertainty and doubt) about free software. Probably that was the intention. + +This FUD about open source software is mainly about open source licensing. There are many different licenses, some more restrictive (some people use the term "protective") than others. Restrictive licenses such as the GNU General Public License (GPL) use the concept of copyleft, which grants people the right to freely distribute copies and modified versions of a piece of software as long as the same rights are preserved in derivative works. The GPL (v3) is used by open source projects such as bash and GIMP. There's also the Affero GPL, which provides copyleft to software that is offered over a network (for example as a web service.) + +What this means is that if you take code that is licensed in this way and you modify it by adding some of your own proprietary code, then in some circumstances the whole new body of code, including your code, becomes subject to the restrictive open source license. It was this type of license that Ballmer was probably referring to when he made his statement. + +But permissive licenses are a different animal. The MIT License, for example, lets anyone take open source code and do what they want with it — including modifying and selling it — as long as they provide attribution and don't hold the developer liable. Another popular permissive open source license, the Apache License 2.0, also provides an express grant of patent rights from contributors to users. JQuery, the .NET Core and Rails are licensed using the MIT license, while the Apache 2.0 license is used by software including Android, Apache and Swift. + +Ultimately both license types are intended to make software more useful. Restrictive licenses aim to foster the open source ideals of participation and sharing so everyone gets the maximum benefit from software. And permissive licenses aim to ensure that people can get the maximum benefit from software by allowing them to do what they want with it — even if that means they take the code, modify it and keep it for themselves or even sell the resulting work as proprietary software without contributing anything back. + +Figures compiled by open source license management company Black Duck Software show that the restrictive GPL 2.0 was the most commonly used open source license last year with about 25 percent of the market. The permissive MIT and Apache 2.0 licenses were next with about 18 percent and 16 percent respectively, followed by the GPL 3.0 with about 10 percent. That's almost evenly split at 35 percent restrictive and 34 percent permissive. + +But this snapshot misses the trend. Black Duck's data shows that in the six years from 2009 to 2015 the MIT license's share of the market has gone up 15.7 percent and Apache's share has gone up 12.4 percent. GPL v2 and v3's share during the same period has dropped by a staggering 21.4 percent. In other words there was a significant move away from restrictive licenses and towards permissive ones during that period. + +And the trend is continuing. Black Duck's [latest figures][1] show that MIT is now at 26 percent, GPL v2 21 percent, Apache 2 16 percent, and GPL v3 9 percent. That's 30 percent restrictive, 42 percent permissive — a huge swing from last year’s 35 percent restrictive and 34 percent permissive. Separate [research][2] of the licenses used on GitHub appears to confirm this shift. It shows that MIT is overwhelmingly the most popular license with a 45 percent share, compared to GLP v2 with just 13 percent and Apache with 11 percent. + +![](http://images.techhive.com/images/article/2016/09/open-source-licenses.jpg-100682571-large.idge.jpeg) + +### Driving the trend + +What’s behind this mass move from restrictive to permissive licenses? Do companies fear that if they let restrictive software into the house they will lose control of their proprietary software, as Ballmer warned? In fact, that may well be the case. Google, for example, has [banned Affero GPL software][3] from its operations. + +Jim Farmer, chairman of [Instructional Media + Magic][4], a developer of open source technology for education, believes that many companies avoid restrictive licenses to avoid legal difficulties. "The problem is really about complexity. The more complexity in a license, the more chance there is that someone has a cause of action to bring you to court. Complexity makes litigation more likely," he says. + +He adds that fear of restrictive licenses is being driven by lawyers, many of whom recommend that clients use software that is licensed with the MIT or Apache 2.0 licenses, and who specifically warn against the Affero license. + +This has a knock-on effect with software developers, he says, because if companies avoid software with restrictive licenses then developers have more incentive to license their new software with permissive ones if they want it to get used. + +But Greg Soper, CEO of SalesAgility, the company behind the open source SuiteCRM, believes that the move towards permissive licenses is also being driven by some developers. "Look at an application like Rocket.Chat. The developers could have licensed that with GPL 2.0 or Affero but they chose a permissive license," he says. "That gives the app the widest possible opportunity, because a proprietary vendor can take it and not harm their product or expose it to an open source license. So if a developer wants an application to be used inside a third-party application it makes sense to use a permissive license." + +Soper points out that restrictive licenses are designed to help an open source project succeed by stopping developers from taking other people's code, working on it, and then not sharing the results back with the community. "The Affero license is critical to the health of our product because if people could make a fork that was better than ours and not give the code back that would kill our product," he says. "For Rocket.Chat it's different because if it used Affero then it would pollute companies' IP and so it wouldn't get used. Different licenses have different use cases." + +Michael Meeks, an open source developer who has worked on Gnome, OpenOffice and now LibreOffice, agrees with Jim Farmer that many companies do choose to use software with permissive licenses for fear of legal action. "There are risks with copyleft licenses, but there are also huge benefits. Unfortunately people listen to lawyers, and lawyers talk about risk but they never tell you that something is safe." + +Fifteen years after Ballmer made his inaccurate statement it seems that the FUD it generated it is still having an effect — even if the move from restrictive licenses to permissive ones is not quite the effect he intended. + +-------------------------------------------------------------------------------- + +via: http://www.cio.com/article/3120235/open-source-tools/what-the-rise-of-permissive-open-source-licenses-means.html + +作者:[Paul Rubens ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.cio.com/author/Paul-Rubens/ +[1]: https://www.blackducksoftware.com/top-open-source-licenses +[2]: https://github.com/blog/1964-open-source-license-usage-on-github-com +[3]: http://www.theregister.co.uk/2011/03/31/google_on_open_source_licenses/ +[4]: http://immagic.com/ + From 05c606771846beb89d33acfd36244210fd0d3976 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 20:55:40 +0800 Subject: [PATCH 0147/2437] =?UTF-8?q?20160917-6=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...from having open source program offices.md | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 sources/tech/20160913 4 big ways companies benefit from having open source program offices.md diff --git a/sources/tech/20160913 4 big ways companies benefit from having open source program offices.md b/sources/tech/20160913 4 big ways companies benefit from having open source program offices.md new file mode 100644 index 0000000000..674ddfdb85 --- /dev/null +++ b/sources/tech/20160913 4 big ways companies benefit from having open source program offices.md @@ -0,0 +1,50 @@ +4 big ways companies benefit from having open source program offices +==== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/BUSINESS_creativity.png?itok=x2HTRKVW) + +In the first article in my series on open source program offices, I took a deep dive into [what an open source program office is and why your company might need one][1]. Next I looked at [how Google created a new kind of open source program office][2]. In this article, I'll explain a few benefits of having an open source program office. + +At first glance, one big reason why a company not in the business of software development might more enthusiastically embrace an open source program office is because they have less to lose. After all, they're not gambling with software products that are directly tied to revenue. Facebook, for example, can easily unleash a distributed key-value datastore as an open source project because they don't sell a product called "enterprise key-value datastore." That answers the question of risk, but it still doesn't answer the question of what they gain from contributing to the open source ecosystem. Let's look at a few potential reasons and then tackle each. You'll notice a lot of overlap with vendor open source program offices, but some of the motivations are slightly different. + +### Recruiting + +Recruiting is perhaps the easiest way to sell an open source program office to upper management. Show them the costs associated with recruiting, as well as the return on investment, and then explain how developing relationships with talented engineers results in a pipeline of talented developers who are actually familiar with your technology and excited to help work on it. We don't really need to go in more depth here—it's self-explanatory, right? + +### Technology influence + +Once upon a time, companies that didn't specialize in selling software were powerless to influence development cycles of their software vendors directly, especially if they were not a large customer. Open source completely changed that dynamic and brought users onto a more level playing field with vendors. With the rise of open source development, anyone could push technology into a chosen direction, assuming they were willing to invest the time and resources. But these companies learned that simply investing in developer time, although fruitful, could be even more useful if tied to an overarching strategic effort&mdash. Think bug fixes vs. software architects—lots of companies push bug fixes to upstream open source projects, but some of these same companies began to learn that coordinating a sustained effort with a deeper commitment paid off with faster feature development, which could be good for business. With the open source program office model, companies have staffers who can sniff out strategically important open source communities in which they then invest developer resources. + +With rapidly growing companies such as Google and Facebook, providing leadership in existing open source projects still proved insufficient for their expanding businesses. Facing the challenges of intense growth and building out hyperscale systems, many of the largest companies had built highly customized stacks of software for internal use only. What if they could convince others to collaborate on some of these infrastructure projects? Thus, while they maintained investments in areas such as the Linux kernel, Apache, and other existing projects, they also began to release their own large projects. Facebook released Cassandra, Twitter created Mesos, and eventually Google created the Kubernetes project. These projects have become major platforms for industry innovation, proving to be spectacular successes for the companies involved. (Note that Facebook stopped using Cassandra internally after it needed to create a new software project to solve the problem at a larger scale. However, by that time Cassandra had already become popular and DataStax had formed to take on development). Each of these projects have spawned entire ecosystems of developers, related projects, and end users that serve to accelerate growth and development. + +This would not have been possible without coordination between an open source program office and strategic company initiatives. Without that effort, each of the companies mentioned would still be trying to solve these problems individually—and more slowly. Not only have these projects helped solve business problems internally, they also helped establish the companies that created them as industry heavyweights. Sure, Google has been an industry titan for a few years now, but the growth of Kubernetes ensures both better software, and a direct say in the future direction of container technologies, even more than it already had. These companies are still known for their hyperscale infrastructure and for simply being large Silicon Valley stalwarts. Lesser known, but possibly even more important, is their new relevance as technology producers. Open source program offices guide these efforts and maximize their impact, through technology recommendations and relationships with influential developers, not to mention deep expertise in community governance and people management. + +### Marketing power + +Going hand-in-hand with technology influence is how each company talks about its open source efforts. By honing the messages around these projects and communities, an open source program office is able to deliver maximum impact through targeted marketing campaigns. Marketing has long been a dirty word in open source circles, because everyone has had a bad experience with corporate marketing. In the case of open source communities, marketing takes on a vastly different form from a traditional approach and involves amplifying what is already happening in the communities of strategic importance. Thus, an open source program office probably won't create whiz-bang slides about a project that hasn't even released any code yet, but they'll talk about the software they've created and other initiatives they've participated in. Basically, no vaporware here. + +Think of the first efforts made by Google's open source program office. They didn't simply contribute code to the Linux kernel and other projects—they talked about it a lot, often in keynotes at open source conferences. They didn't just give money to students who write open source code—they created a global program, the Google Summer of Code, that became a cultural touchstone of open source development. This marketing effort cemented Google's status as a major open source developer long before Kubernetes was even developed. As a result, Google wielded major influence during the creation of the GPLv3 license, and company speakers and open source program office representatives became staples at tech events. The open source program office is the entity best situated to coordinate these efforts and deliver real value for the parent company. + +### Improve internal processes + +Improving internal processes may not sound like a big benefit, but overcoming chaotic internal processes is a challenge for every open source program office, whether software vendor or company-driven. Whereas a software vendor must make sure that their processes don't step on products they release (for example, unintentionally open sourcing proprietary software), a user is more concerned with infringement of intellectual property (IP) law: patents, copyrights, and trademarks. No one wants to get sued simply for releasing software. Without an active open source program office to manage and coordinate licensing and other legal questions, large companies face great difficulty in arriving at a consensus around open source processes and governance. Why is this important? If different groups release software under incompatible licenses, not only will this prove to be an embarrassment, it also will provide a significant obstacle to achieving one of the most basic goals—improved collaboration. + +Combined with the fact that many of these companies are still growing incredibly quickly, an inability to establish basic rules around process will prove unwieldy sooner than expected. I've seen large spreadsheets with a matrix of approved and unapproved licenses as well as guidelines for how to (and how not to) create open source communities while complying with legal restrictions. The key is to have something that developers can refer to when they need to make decisions, without incurring the legal overhead of a massive, work-slowing IP review every time a developer wants to contribute to an open source community. + +Having an active open source program office that maintains rules over license compliance and source contribution, as well as establishing training programs for engineers, helps to avoid potential legal pitfalls and costly lawsuits. After all, what good is better collaboration on open source projects if the company loses real money because someone didn't read the license? The good news is that companies have less to worry about with respect to proprietary IP when compared to software vendors. The bad news is that their legal questions are no less complex, especially when they run directly into the legal headwinds of a software vendor. + +How has your organization benefited from having an open source program office? Let me know about it in the comments. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/business/16/9/4-big-ways-companies-benefit-having-open-source-program-offices + +作者:[John Mark Walker][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://opensource.com/users/johnmark +[1]: https://opensource.com/business/16/5/whats-open-source-program-office +[2]: https://opensource.com/business/16/8/google-open-source-program-office From 0a9a592e894f13a14a2ec850100577a1d7ac8494 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:01:18 +0800 Subject: [PATCH 0148/2437] =?UTF-8?q?20160917-7=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...s a Slick Desktop REST Client for Linux.md | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 sources/tech/20160915 Insomnia 3.0 Is a Slick Desktop REST Client for Linux.md diff --git a/sources/tech/20160915 Insomnia 3.0 Is a Slick Desktop REST Client for Linux.md b/sources/tech/20160915 Insomnia 3.0 Is a Slick Desktop REST Client for Linux.md new file mode 100644 index 0000000000..f3a0fc717e --- /dev/null +++ b/sources/tech/20160915 Insomnia 3.0 Is a Slick Desktop REST Client for Linux.md @@ -0,0 +1,51 @@ +Insomnia 3.0 Is a Slick Desktop REST Client for Linux +===== + +![](http://www.omgubuntu.co.uk/wp-content/uploads/2016/09/insomnia-app-screenshot.png) + +Looking for a free REST client for the Linux desktop? Don’t lose sleep: get [Insomnia][1]. + +The app is cross-platform and works on Linux, macOS and Windows. Its developer, Gregory Schier, told us that he created the app “to help developers communicate with [REST APIs][2].” + +He also told that Insomnia already has around 10,000 active users — 9% of which are on Linux. + +“So far, the feedback from Linux users has been very positive because similar applications (not nice ones anyway) aren’t usually available for Linux.” + +Insomnia aims to ‘speed up your API testing workflow’, by letting you organise, run and debug HTTP requests through a cleanly designed interface. + +The app also includes advanced features like cookie management, global environments, SSL validation, and code snippet generation. + +As I am not a developer I can’t evaluate this app first-hand, nor tell you why it rocks or highlight any major feature deficiencies. + +But I thought I’d bring the app to your attention and let you decide for yourself. If you’ve been hunting for a slickly designed GUI alternative to command-line tools like HTTPie, it might be well worth giving it a whirl. + +### Download Insomnia 3.0 for Linux + +Insomnia 3.0 (not to be confused with Insomnia v2.0 which is only available on Chrome) is available to download for Windows, macOS and Linux. + +[Download Insomnia 3.0][4] + +An installer is available for Ubuntu 14.04 LTS and up, as is a cross-distro AppImage: + +[Download Insomnia 3.0 (.AppImage)][5] + +If you want to keep pace with development of the app you can follow [Insomnia on Twitter][6]. + + +-------------------------------------------------------------------------------- + +via: http://www.omgubuntu.co.uk/2016/09/insomnia-3-is-free-rest-client-for-linux?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+d0od+%28OMG%21+Ubuntu%21%29 + +作者:[JOEY-ELIJAH SNEDDON ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://plus.google.com/117485690627814051450/?rel=author +[1]: http://insomnia.rest/ +[2]: https://en.wikipedia.org/wiki/Representational_state_transfer +[3]: https://github.com/jkbrzt/httpie +[4]: https://insomnia.rest/download/ +[5]: https://builds.insomnia.rest/downloads/linux/latest +[6]: https://twitter.com/GetInsomnia From fe9c6eec5c36341051d3078f5532966955383c10 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:05:14 +0800 Subject: [PATCH 0149/2437] =?UTF-8?q?20160917-7=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...peed Up LibreOffice with 4 Simple Steps.md | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 sources/tech/20160914 How to Speed Up LibreOffice with 4 Simple Steps.md diff --git a/sources/tech/20160914 How to Speed Up LibreOffice with 4 Simple Steps.md b/sources/tech/20160914 How to Speed Up LibreOffice with 4 Simple Steps.md new file mode 100644 index 0000000000..a3dc030f3d --- /dev/null +++ b/sources/tech/20160914 How to Speed Up LibreOffice with 4 Simple Steps.md @@ -0,0 +1,90 @@ +How to Speed Up LibreOffice with 4 Simple Steps +==== + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/08/speed-up-libreoffice-featured-2.jpg) + +For many fans and supporters of Open Source software, LibreOffice is the best alternative to Microsoft Office, and it has definitely seen huge improvements over the last few releases. However, the initial startup experience still leaves a lot to be desired. There are ways to improve launch time and overall performance of LibreOffice. + +I will go over some practical steps that you can take to improve the load time and responsiveness of LibreOffice in the paragraphs below. + +### 1. Increase Memory Per Object and Image Cache + +This will help the program load faster by allocating more memory resources to the image cache and objects. + +1. Launch LibreOffice Writer (or Calc) + +2. Navigate to “Tools -> Options” in the menubar or use the keyboard shortcut “Alt + F12.” + +3. Click “Memory” under LibreOffice and increase “Use for LibreOffice” to 128MB. + +4. Also increase “Memory per object” to 20Mb. + +5. Click “Ok” to save your changes. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/08/speed-up-libreoffice-step-1.png) + +Note: You can set the numbers higher or lower than the suggested values depending on how powerful your machine is. It is best to experiment and see which value gives you the optimum performance. + +### 2. Enable LibreOffice QuickStarter + +If you have a generous amount of RAM on your machine, say 4GB and above, you can enable the “Systray Quickstarter” option to keep part of LibreOffice in memory for quicker response with opening new documents. + +You will definitely see improved performance in opening new documents after enabling this option. + +1. Open the options dialog by navigating to “Tools -> Options.” + +2. In the sidebar under “LibreOffice”, select “Memory.” + +3. Tick the “Enable Systray Quickstarter” checkbox. + +4. Click “OK” to save the changes. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/08/speed-up-libreoffice-2.png) + +Once this option is enabled, you will see the LibreOffice icon in your system tray with options to open any type of document. + +### 3. Disable Java Runtime + +Another easy way to speed up the launch time and responsiveness of LibreOffice is to disable Java. + +1. Open the Options dialog using “Alt + F12.” + +2. In the sidebar, select “LibreOffice,” then “Advanced.” + +3. Uncheck the “Use Java runtime environment” option. + +4. Click “OK” to close the dialog. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/08/speed-up-libreoffice-3.png) + +If all you use is Writer and Calc, disabling Java will not stop you from working with your files as normal. But to use LibreOffice Base and some other special features, you may need to re-enable it again. In that case, you will get a popup asking if you wish to turn it back on. + +### 4. Reduce Number of Undo Steps + +By default, LibreOffice allows you to undo up to 100 changes to a document. Most users do not need anywhere near that, so holding that many steps in memory is largely a waste of resources. + +I recommend that you reduce this number to 20 to free up memory for other things, but feel free to customise this part to suit your needs. + +1. Open the options dialog by navigating to “Tools -> Options.” + +2. In the sidebar under “LibreOffice,” select “Memory.” + +3. Under “Undo” and change the number of steps to your preferred value. + +4. Click “OK” to save the changes. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/08/speed-up-libreoffice-5.png) + +If the tips provided helped you speed up the launch time of your LibreOffice Suite, let us know in the comments. Also, please share any other tips you may know for others to benefit as well. + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/speed-up-libreoffice/?utm_medium=feed&utm_source=feedpress.me&utm_campaign=Feed%3A+maketecheasier + +作者:[Ayo Isaiah][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.maketecheasier.com/author/ayoisaiah/ From f30802a6ad382c1bb81fd6b086903246a18b3626 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:08:00 +0800 Subject: [PATCH 0150/2437] =?UTF-8?q?20160917-8=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...Linux Server Distros Worth Checking Out.md | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 sources/tech/20160912 Five Linux Server Distros Worth Checking Out.md diff --git a/sources/tech/20160912 Five Linux Server Distros Worth Checking Out.md b/sources/tech/20160912 Five Linux Server Distros Worth Checking Out.md new file mode 100644 index 0000000000..68816e2b8e --- /dev/null +++ b/sources/tech/20160912 Five Linux Server Distros Worth Checking Out.md @@ -0,0 +1,44 @@ +Five Linux Server Distros Worth Checking Out +==== + +>Pretty much any of the nearly 300 Linux distributions you'll find listed on Distrowatch can be made to work as servers. Here are those that stand out above the rest. + +![](http://windowsitpro.com/site-files/windowsitpro.com/files/imagecache/large_img/uploads/2016/09/cloudservers.jpg) + +Pretty much any of the nearly 300 Linux distributions you'll find listed on Distrowatch can be made to work as servers. Since Linux's earliest days, users have been provisioning "all purpose" distributions such as Slackware, Debian and Gentoo to do heavy lifting as servers for home and business. That may be fine for the hobbyist, but its a lot of unnecessary work for the professional. + +From the beginning, however, there have been distributions with no other purpose but to serve files and applications, help workstations share common peripherals, serve-up web pages and all the other things we ask servers to do, whether in the cloud, in a data center or on a shelf in a utility closet. + +Here's a look at four of the most used Linux server distros, as well as one distro that might fit the bill for smaller businesses. + +**Red Hat Enterprise Linux**: Perhaps the best known of Linux server distros, RHEL has a reputation for being a solid distribution ready for the most demanding mission critical tasks -- like running the New York Stock Exchange for instance. It's also backed by Red Hat's best-of-breed support. + +The downside? While Red Hat is known for offering customer service and support that's second to none, its support subscriptions aren't cheap. Some might point out, however, that you get what you pay for. Cheaper third party support for RHEL is available, but you might want to do some research before going that route. + +**CentOS**: Anyone who likes RHEL but would like to avoid shoveling money to Red Hat for support should take a look at CentOS, which is basically an RHEL fork. Although it's been around since 2004, in 2014 it became officially sponsored by Red Hat, which now employs most of the project's developers. This means that security patches and bug fixes are made available to CentOS soon after they're pushed to Red Hat. + +If you're going to deploy CentOS, you'll need people with Linux skills on staff, because as far as technical support goes, you're mainly on your own. The good news is that the CentOS community offers excellent resources, such as mailing lists, web forums, and chat rooms, so help is available to those who search. + +**Ubuntu Server**: When Canonical announced many years back that it was coming out with a server edition of Ubuntu, you could hear the snickers. Laughter turned into amazement rather quickly, however, as Ubuntu Server rapidly took hold. This was partly due to the DNA it shares as a derivative of Debian, which has long been a favorite base for Linux servers. Ubuntu filled a gap by adding affordable technical support, superior hardware support, developer tools and lots of polish. + +How popular is Ubuntu Server? Recent figures show it being the most deployed operating system both on OpenStack and on the Amazon Elastic Compute Cloud, where it outpaces second place Amazon Linux Amazon Machine Image by a mile and leaves third place Windows in the virtual dust. Another study shows it as the most used Linux web server. + +**SUSE Linux Enterprise Server**: This German distro has a large base of users in Europe, and was a top server distro on this side of the Atlantic until PR issues arose after it was bought by Novell in the early part of the century. With those days long behind it, SUSE has been gaining ground in the US, and its use will probably accelerate now that HPE is naming it as its preferred Linux partner. + +SUSE Linux Enterprise Server, or SLES, is stable and easy to maintain, which you'd expect for a distro that's been around for nearly as long as Linux itself. Affordable 24/7 "rapid-response" technical support is available, making it suitable for mission critical deployments. + +**ClearOS**: Based on RHEL, ClearOS is included here because it's simple enough for anyone, even most non-techies, to configure. Targeted at small to medium sized businesses, it can also be used as an entertainment server by home users. Using a web-based administration interface for ease-of-use, it's built with the premise in mind that "building your IT infrastructure should be as simple as downloading apps on your smart phone." + +The latest release, version 7.2, includes capabilities that might not be expected from a "lightweight" offering, such as VM support which includes Microsoft Hyper-V, support for the XFS and BTRFS file systems, as well as support for LVM caching and IPv6. Available in a free version or in an inexpensive "professional" version that comes with a variety of support options. + +-------------------------------------------------------------------------------- + +via: http://windowsitpro.com/industry/five-linux-server-distros-worth-checking-out + +作者:[Christine Hall][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://windowsitpro.com/industry/five-linux-server-distros-worth-checking-out From f769e2ccf0a50e7610d435ebbe00868db25edbe3 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:17:28 +0800 Subject: [PATCH 0151/2437] =?UTF-8?q?20160917-9=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... NOW YOU CAN GOOGLE FROM LINUX TERMINAL.md | 129 ++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 sources/tech/20160915 GOOGLER: NOW YOU CAN GOOGLE FROM LINUX TERMINAL.md diff --git a/sources/tech/20160915 GOOGLER: NOW YOU CAN GOOGLE FROM LINUX TERMINAL.md b/sources/tech/20160915 GOOGLER: NOW YOU CAN GOOGLE FROM LINUX TERMINAL.md new file mode 100644 index 0000000000..29220291bf --- /dev/null +++ b/sources/tech/20160915 GOOGLER: NOW YOU CAN GOOGLE FROM LINUX TERMINAL.md @@ -0,0 +1,129 @@ +GOOGLER: NOW YOU CAN GOOGLE FROM LINUX TERMINAL! +==== + +![](https://itsfoss.com/wp-content/uploads/2016/09/google-from-linux-terminal.jpg) + +A quick question: What do you do every day? Of course, a lot of things. But I can tell one thing, you search on Google almost every day (if not every day). Am I right? + +Now, if you are a Linux user (which I’m guessing you are) here’s another question: wouldn’t it be nice if you can Google without even leaving the terminal? Without even firing up a Browser window? + +If you are a *nix enthusiast and also one of those people who just love the view of the terminal, I know your answer is – Yes. And I think, the rest of you will also like the nifty little tool I’m going to introduce today. It’s called Googler! + +### GOOGLER: GOOGLE IN YOUR LINUX TERMINAL + +Googler is a straightforward command-line utility for Google-ing right from your terminal window. Googler mainly supports three types of Google Searches: + +- Google Search: Simple Google searching, equivalent to searching on Google homepage. +- Google News Search: Google searching for News, equivalent to searching on Google News. +- Google Site Search: Google searching for results from a specific site. + +Googler shows the search results with the title, URL and page excerpt. The search results can be opened directly in the browser with only a couple of keystrokes. + +![](https://itsfoss.com/wp-content/uploads/2016/09/googler-1.png) + +### INSTALLATION ON UBUNTU + +Let’s go through the installation process first. + +At first make sure you have python version 3.3 or later using this command: + +``` +python3 --version +``` + +If not, upgrade it. Googler requires python 3.3+ for running. + +Though Googler is yet not available through package repository on Ubuntu, we can easily install it from the GitHub repository. All we have to do is run the following commands: + +``` +cd /tmp +git clone https://github.com/jarun/googler.git +cd googler +sudo make install +cd auto-completion/bash/ +sudo cp googler-completion.bash /etc/bash_completion.d/ +``` + +And that’s it. Googler is installed along with command autocompletion feature. + +### FEATURES & BASIC USAGE + +If we go through all its features, Googler is actually quite powerful a tool. Some of the main features are: + +Interactive Interface: Run the following command in terminal: + +``` +googler +``` + +The interactive interface will be opened. The developer of Googler, Arun [Prakash Jana][1] calls it the omniprompt. You can enter ? for available commands on omniprompt. + +![](https://itsfoss.com/wp-content/uploads/2016/09/googler-2.png) + +From the omniprompt, enter any search phrases to initiate the search. You can then enter n or p to navigate next or previous page of search results. + +To open any search result in a browser window, just enter the index number of that result. Or you can open the search page itself by entering o . + +- News Search: If you want to search News, start googler with the N optional argument: + +``` +googler -N +``` + +The subsequent omniprompt will fetch results from Google News. + +- Site Search: If you want to search pages from a specific site, run googler with w {domain} argument: + +``` +googler -w itsfoss.com +``` + +The subsequent omniprompt with fetch results only from It’s FOSS blog! + +- Manual Page: Run the following command for Googler manual page equipped with various examples: + +``` +man googler +``` + +- Google country/domain specific search: + +``` +googler -c in "hello world" +``` + +The above example command will open search results from Google’s Indian domain (in for India). + +- Filter search results by duration and language preference. +- Google search keywords support, such as: site:example.com or filetype:pdf etc. +- HTTPS proxy support. +- Shell commands autocomplete. +- Disable automatic spelling correction. + +There are much more. You can twist Googler to suit your needs. + +Googler can also be integrated with a text-based browser ( like – [elinks][2], [links][3], [lynx][4], w3m etc.), so that you wouldn’t even need to leave the terminal for browsing web pages. The instructions can be found on the [GitHub project page of Googler][5]. + +If you want a graphical demonstration of Googler’s various features, feel free to check the terminal recording attached to the GitHub project page : [jarun/googler v2.7 quick demo][6]. + +### THOUGHTS ON GOOGLER? + +Though Googler might not feel necessary or desired to everybody, for someone who doesn’t want to open the browser just for searching on google or simply want to spend as much as time possible on the terminal window, it is a great tool indeed. What do you think? + +-------------------------------------------------------------------------------- + +via: http://linoxide.com/firewall/pfsense-setup-basic-configuration/ + +作者:[Munif Tanjim][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://itsfoss.com/author/munif/ +[1]: https://github.com/jarun +[2]: http://elinks.or.cz/ +[3]: http://links.twibright.com/ +[4]: http://lynx.browser.org/ +[5]: https://github.com/jarun/googler#faq +[6]: https://asciinema.org/a/85019 From 62c22d62a2625f8b1e83d8d250be92929879c90f Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:24:50 +0800 Subject: [PATCH 0152/2437] =?UTF-8?q?20160917-10=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../20160506 Setup honeypot in Kali Linux.md | 84 +++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 sources/tech/20160506 Setup honeypot in Kali Linux.md diff --git a/sources/tech/20160506 Setup honeypot in Kali Linux.md b/sources/tech/20160506 Setup honeypot in Kali Linux.md new file mode 100644 index 0000000000..cb85992389 --- /dev/null +++ b/sources/tech/20160506 Setup honeypot in Kali Linux.md @@ -0,0 +1,84 @@ +Setup honeypot in Kali Linux +==== + +The Pentbox is a safety kit containing various tools for streamlining PenTest conducting a job easily. It is programmed in Ruby and oriented to GNU / Linux, with support for Windows, MacOS and every systems where Ruby is installed. In this small article we will explain how to set up a honeypot in Kali Linux. If you don’t know what is a honeypot, “a honeypot is a computer security mechanism set to detect, deflect, or, in some manner, counteract attempts at unauthorized use of information systems.” + +### Download Pentbox: + +Simply type in the following command in your terminal to download pentbox-1.8. + +``` +root@kali:~# wget http://downloads.sourceforge.net/project/pentbox18realised/pentbox-1.8.tar.gz +``` + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-1.jpg) + +### Uncompress pentbox files + +Decompressing the file with the following command: + +``` +root@kali:~# tar -zxvf pentbox-1.8.tar.gz +``` + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-2.jpg) + +### Run pentbox ruby script + +Change directory into pentbox folder + +``` +root@kali:~# cd pentbox-1.8/ +``` + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-3.jpg) + +Run pentbox using the following command + +``` +root@kali:~# ./pentbox.rb +``` + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-4.jpg) + +### Setup a honeypot + +Use option 2 (Network Tools) and then option 3 (Honeypot). + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-5.jpg) + +Finally for first test, choose option 1 (Fast Auto Configuration) + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-6.jpg) + +This opens up a honeypot in port 80. Simply open browser and browse to http://192.168.160.128 (where 192.168.160.128 is your IP Address. You should see an Access denied error. + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-7.jpg) + +and in the terminal you should see “HONEYPOT ACTIVATED ON PORT 80” followed by “INTRUSION ATTEMPT DETECTED”. + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-8.jpg) + +Now, if you do the same steps but this time select Option 2 (Manual Configuration), you should see more extra options + +![](https://www.blackmoreops.com/wp-content/uploads/2016/05/Set-up-a-honeypot-in-Kali-Linux-blackMORE-Ops-9.jpg) + +Do the same steps but select port 22 this time (SSH Port). Then do a port forwarding in your home router to forward port external port 22 to this machines’ port 22. Alternatively, set it up in a VPS in your cloud server. + +You’d be amazed how many bots out there scanning port SSH continuously. You know what you do then? You try to hack them back for the lulz! + +Here’s a video of setting up honeypot if video is your thing: + + + +-------------------------------------------------------------------------------- + +via: https://www.blackmoreops.com/2016/05/06/setup-honeypot-in-kali-linux/ + +作者:[blackmoreops.com][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: blackmoreops.com From 64ced29ce53b49c80454936526a9c96c17ced53a Mon Sep 17 00:00:00 2001 From: Chao-zhi Liu Date: Sat, 17 Sep 2016 21:29:19 +0800 Subject: [PATCH 0153/2437] =?UTF-8?q?=E7=BF=BB=E8=AF=91=E5=AE=8C=E6=88=90?= =?UTF-8?q?=202016-9-17?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...And Is ‘One Fits All’ Linux Packages Useful.md | 112 ------------------ ...And Is ‘One Fits All’ Linux Packages Useful.md | 111 +++++++++++++++++ 2 files changed, 111 insertions(+), 112 deletions(-) delete mode 100644 sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md create mode 100644 translated/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md diff --git a/sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md b/sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md deleted file mode 100644 index b1c955bf5b..0000000000 --- a/sources/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md +++ /dev/null @@ -1,112 +0,0 @@ -Translating by Chao-zhi - -Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful? -================================================================================= - -![](http://www.iwillfolo.com/wordpress/wp-content/uploads/2016/06/Flatpak-and-Snap-Packages.jpg) - -An in-depth look into the new generation of packages starting to permeate the Linux ecosystem. - - -Lately we’ve been hearing more and more about Ubuntu’s Snap packages and Flatpak (formerly referred to as xdg-app) created by Red Hat’s employee Alexander Larsson. - -These 2 types of next generation packages are in essence having the same goal and characteristics which are: being standalone packages that doesn’t rely on 3rd-party system libraries in order to function. - -This new technology direction which Linux seems to be headed is automatically giving rise to questions such as, what are the advantages / disadvantages of standalone packages? does this lead us to a better Linux overall? what are the motives behind it? - -To answer these questions and more, let us explore the things we know about Snap and Flatpak so far. - -### The Motive - -According to both [Flatpak][1] and [Snap][2] statements, the main motive behind them is to be able to bring one and the same version of application to run across multiple Linux distributions. - ->“From the very start its primary goal has been to allow the same application to run across a myriad of Linux distributions and operating systems.” Flatpak - ->“… ‘snap’ universal Linux package format, enabling a single binary package to work perfectly and securely on any Linux desktop, server, cloud or device.” Snap - -To be more specific, the guys behind Snap and Flatpak (S&F) believe that there’s a barrier of fragmentation on the Linux platform. - -A barrier which holds back the platform advancement by burdening developers with more, perhaps unnecessary, work to get their software run on the many distributions out there. - -Therefore, as leading Linux distributions (Ubuntu & Red Hat), they wish to eliminate the barrier and strengthen the platform in general. - -But what are the more personal gains which motivate the development of S&F? - -#### Personal Gains? - -Although not officially stated anywhere, it may be assumed that by leading the efforts of creating a unified package that could potentially be adopted by the vast majority of Linux distros (if not all of them), the captains of these projects could assume a key position in determining where the Linux ship sails. - -### The Advantages - -The benefits of standalone packages are diverse and can depend on different factors. - -Basically however, these factors can be categorized under 2 distinct criteria: - -#### User Perspective - -+ From a Linux user point of view, Snap and Flatpak both bring the possibility of installing any package (software / app) on any distribution the user is using. - -That is, for instance, if you’re using a not so popular distribution which has only a scarce supply of packages available in their repo, due to workforce limitations probably, you’ll now be able to easily and significantly increase the amount of packages available to you – which is a great thing. - -+ Also, users of popular distributions that do have many packages available in their repos, will enjoy the ability of installing packages that might not have behaved with their current set of installed libraries. - -For example, a Debian user who wants to install a package from ‘testing branch’ will not have to convert his entire system into ‘testing’ (in order for the package to run against newer libraries), rather, that user will simply be able to install only the package he wants from whichever branch he likes and on whatever branch he’s on. - -The latter point, was already basically possible for users who were compiling their packages straight from source, however, unless using a source based distribution such as Gentoo, most users will see this as just an unworthily hassle. - -+ The advanced user, or perhaps better put, the security aware user might feel more comfortable with this type of packages as long as they come from a reliable source as they tend to provide another layer of isolation since they are generally isolated from system packages. - -* Both S&F are being developed with enhanced security in mind, which generally makes use of “sandboxing” i.e isolation in order to prevent cases where they carry a virus which can infect the entire system, similar to the way .exe files on MS Windows may. (More on MS and S&F later) - -#### Developer Perspective - -For developers, the advantages of developing S&F packages will probably be a lot clearer than they are to the average user, some of these were already hinted in a previous section of this post. - -Nonetheless, here they are: - -+ S&F will make it easier on devs who want to develop for more than one Linux distribution by unifying the process of development, therefore minimizing the amount of work a developer needs to do in order to get his app running on multiple distributions. - -++ Developers could therefore gain easier access to a wider range of distributions. - -+ S&F allow devs to privately distribute their packages without being dependent on distribution maintainers to stabilize their package for each and every distro. - -++ Through the above, devs may gain access to direct statistics of user adoption / engagement for their software. - -++ Also through the above, devs could get more directly involved with users, rather than having to do so through a middleman, in this case, the distribution. - -### The Downsides - -– Bloat. Simple as that. Flatpak and Snap aren’t just magic making dependencies evaporate into thin air. Rather, instead of relying on the target system to provide the required dependencies, S&F comes with the dependencies prebuilt into them. - -As the saying goes “if the mountain won’t come to Muhammad, Muhammad must go to the mountain…” - -– Just as the security-aware user might enjoy S&F packages extra layer of isolation, as long they come from a trusted source. The less knowledgeable user on the hand, might be prone to the other side of the coin hazard which is using a package from an unknown source which may contain malicious software. - -The above point can be said to be valid even with today’s popular methods, as PPAs, overlays, etc might also be maintained by untrusted sources. - -However, with S&F packages the risk increases since malicious software developers need to create only one version of their program in order to infect a large number of distributions, whereas, without it they’d needed to create multiple versions in order to adjust their malware to other distributions. - -Was Microsoft Right All Along? - -With all that’s mentioned above in mind, it’s pretty clear that for the most part, the advantages of using S&F packages outweighs the drawbacks. - -At the least for users of binary-based distributions, or, non lightweight focused distros. - -Which eventually lead me to asking the above question – could it be that Microsoft was right all along? if so and S&F becomes the Linux standard, would you still consider Linux a Unix-like variant? - -Well apparently, the best one to answer those questions is probably time. - -Nevertheless, I’d argue that even if not entirely right, MS certainly has a good point to their credit, and having all these methods available here on Linux out of the box is certainly a plus in my book. - - --------------------------------------------------------------------------------- - -via: http://www.iwillfolo.com/ubuntus-snap-red-hats-flatpack-and-is-one-fits-all-linux-packages-useful/ - -作者:[Editorials][a] -译者:[译者ID](https://github.com/译者ID) -校对:[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: http://www.iwillfolo.com/category/editorials/ diff --git a/translated/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md b/translated/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md new file mode 100644 index 0000000000..7d4bbb6f25 --- /dev/null +++ b/translated/talk/20160625 Ubuntu’s Snap, Red Hat’s Flatpak And Is ‘One Fits All’ Linux Packages Useful.md @@ -0,0 +1,111 @@ +Ubuntu 的 Snap, Red Hat 的 Flatpak 这种通吃所有发行版的打包方式真的好用吗? +================================================================================= + +![](http://www.iwillfolo.com/wordpress/wp-content/uploads/2016/06/Flatpak-and-Snap-Packages.jpg) + +深入研究新一代的包开始进入到到Linux系统。 + +最近我们听到越来越多的消息是有关于 Ubuntu 的 Snap 包和 Red Hat 员工 Alexander Larsson 创造的 Flatpak (曾经叫做 xdg-app) + +这两种下一代打包方法在本质上拥有相同的目标和特点:即独立包装和不依赖于第三方系统函数库。 + +这种 Linux 新技术方向似乎自然会导致这样的问题:独立包的优点/缺点是什么?这是否让我们拥有更好的 Linux 系统?其背后的动机是什么? + +为了回答这些问题,让我们先深入了解一下 Snap 和 Flatpak。 + +### 动机 + +根据 [Flatpak][1] 和 [Snap][2] 的声明,背后的主要动机是使同一版本的应用程序能够运行在多个 Linux 发行版。 + +>“从一开始它的主要目标是允许相同的应用程序运行在各种 Linux 发行版和操作系统上。” Flatpak + +>“……‘snap’ 通用 Linux 包格式,使简单的二进制包能够完美的安全的运行在任何 Linux 桌面、服务器、云和设备上。” Snap + +说得更具体一点,站在 Snap 和 Flatpak (以下称之为S&F)背后的人认为,Linux 平台存在碎片化的障碍。 + +这个障碍导致了开发者们需要做许多不必要的工作来使他的软件能够运行在各种不同的发行版上,这影响了整个平台的前进。 + +所以,作为 Linux 发行版(Ubuntu 和 Red Hat)的领导者,他们希望消除这个障碍,推动平台发展。 + +但是,是否是更多的个人收益刺激了 S&F 的开发? + +#### 个人收益? + +虽然没有任何官方声明,但是试想一下,如果能够创造这种可能会被大多数发行版(可能不是全部)所采用的打包方式,那么这个项目的领导者将可能成为一个能够决定 Linux 大船航向的重要人物。 + +### 优势 + +这种独立包的好处多多,并且取决于不同的因素。 + +这些因素基本上可以归为两类: + +#### 用户角度 + ++ Liunx 用户的观点认为:Snap 和 Flatpak 带来了将任何应用安装在用户使用的任何发行版上的可能性。 + +例如你在使用一个不是很流行的发行版,由于开发工作的缺乏,它的 repo 只有很稀少的包。现在,通过 S&F 你就可以显著的增加包的数量,这是一个多么美好的事情。 + ++ 同样,对于使用流行的发行版的用户,即使该发行版的 repo 上有很多的包,他也可以在不改变它现有的函数库的同时安装一个新的包。 + +比方说, 一个 Debian 的用户想要安装一个 “测试分支” 的包,但是他又不想将他的整个系统变成测试版(以防该包需要运行在更新的函数库上)。现在,他可以简单的想安装哪个版本就安装哪个版本,而不需要考虑库的问题。 + +对于后者的观点,已经考虑到用户有可能使用源文件编译他们的包,然而,除非你使用基于源的发行版,否则大多数用户将从头编译视为是一个恶心到吐的事情。 + ++ 高级用户,或者称之为 “拥有安全意识的用户” 可能会觉得更容易接受这种类型的包,只要它们来自可靠来源,这种包倾向于提供另一层隔离,因为它们通常是与系统包想隔离的。 + +* 不论是 Snap 还是 Flatpak 都在不断努力增强它们的安全性,通常他们都使用 “沙盒” 来隔离,以防止它们可能携带病毒感染整个系统,就像微软 windows 系统中的 .exe 程序一样。(大多数在微软,而 S&F 紧随其后) + +#### 开发者角度 + +与普通用户相比,对于开发者来说,开发 S&F 包的优点可能更加清楚。这一点已经在上一节有所提示。 + +尽管如此,这些优点有: + ++ S&F 通过统一开发的过程,将多发行版的开发变得简单了起来。对于需要将他的应用运行在多个发行版的开发者来说,这大大的减少了他们的工作量。 + +++ 因此,开发者能够更容易的使他的应用运行在更多的设备上。 + ++ S&F 允许开发者私自发布他的包,不需要依靠发行版维护者在每一个发行版稳定发布他的包 + +++ 通过上述方法,开发者可以不依赖发行版而直接获取并统计用户的对其软件的采用和支持数据 +。 + +++ 同样是通过上述方法,开发者可以更好的直接与用户互动,而不需要通过中间媒介,比如发行版这种中间媒介。 + +### 缺点 + +– 膨胀. 就是这么简单. Flatpak 和 Snap 并不是凭空变出来他的依赖关系。相反,他是通过将依赖关系预构建在其中来代替使用系统中的依赖关系。 + +就像谚语说的:“如果山不过来,我们就去山上吧” + +– 之前提到安全意识强的用户会喜欢 S&F 提供的额外的一层隔离,只要该应用来自一个受信任的来源。但是从另外一个角度看,知识较少的用户,可能会从一个不靠谱的地方弄来一个包含恶意软件的包从而导致危害。 + +上面提到的观点可以说是有很有意义的,虽说今天的流行方法,像ppa、overlays等也可能是来自不受信任的来源。 + +但是,S&F 包更加增加这个风险,因为恶意软件开发者只需要开发一个版本就可以感染各种发行版。相反,如果没有 S&F,恶意软件的开发者就需要创建不同的版本以适应不同的发行版。 + +微软的方法都是正确的吗? + +考虑到上面提到的,很显然,在大多数情况下,使用 S&F 包的优点超过缺点。 + +至少对于二进制发行版的用户,或者非轻量级集中发行版的用户是这样。 + +这使我问出这个问题,可能微软一直是正确的吗?如果是的,那么当 S&F 变成 Linux 的标准后,你还会一如既往的使用 Linux 或者 类 Unix 系统吗? + +很显然,时间会是这个问题的最好答案。 + +不过,我得承认也许并不是全部正确的,但是从微软的信誉来看,它有着很好的观点,并且所有这些可以用在开箱即用的 Linux 上的可行的方法都将加在我的书中。 + +-------------------------------------------------------------------------------- + +via: http://www.iwillfolo.com/ubuntus-snap-red-hats-flatpack-and-is-one-fits-all-linux-packages-useful/ + +作者:[Editorials][a] +译者:[Chao-zhi](https://github.com/Chao-zhi) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.iwillfolo.com/category/editorials/ +[1]: http://flatpak.org/press/2016-06-21-flatpak-released.html +[2]: https://insights.ubuntu.com/2016/06/14/universal-snap-packages-launch-on-multiple-linux-distros From bdd23462cf5d88f914cc5e5d7019a93c2e6e66e8 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:30:45 +0800 Subject: [PATCH 0154/2437] =?UTF-8?q?20160917-11=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...ltiple Operating Systems on Local Network.md | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 sources/talk/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md diff --git a/sources/talk/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md b/sources/talk/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md new file mode 100644 index 0000000000..0210a50789 --- /dev/null +++ b/sources/talk/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md @@ -0,0 +1,98 @@ +NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network +==== + +One of the most important uses of a network is for file sharing purposes. There are multiple ways Linux and Windows, Mac OS X users on a network can now share files with each other and in this post, we shall cover Nitroshare, a cross-platform, open-source and easy-to-use application for sharing files across a local network. + +Nitroshare tremendously simplifies file sharing on a local network, once installed, it integrates with the operating system seamlessly. On Ubuntu, simply open it from the applications indicator, and on Windows, check it in the system tray. + +Additionally, it automatically detects every other device on a network that has Nitroshare installed thereby enabling a user to easily transfer files from one machine to another by selecting which device to transfer to. + +The following are the illustrious features of Nitroshare: + +- Cross-platform, runs on Linux, Windows and Mac OS X +- Easy to setup, no configurations required +- It’s simple to use +- Supports automatic discovery of devices running Nitroshare on local network +- Supports optional TSL encryption for security +- Works at high speeds on fast networks +- Supports transfer of files and directories (folders on Windows) +- Supports desktop notifications about sent files, connected devices and more + +The latest version of Nitroshare was developed using Qt 5, it comes with some great improvements such as: + +- Polished user interfaces +- Simplified device discovery process +- Removal of file size limitation from other versions +- Configuration wizard has also been removed to make it easy to use + +### How To Install Nitroshare on Linux Systems + + +NitroShare is developed to run on a wide variety of modern Linux distributions and desktop environments. + +#### On Debian Sid and Ubuntu 16.04+ + +NitroShare is included in the Debian and Ubuntu software repositories and can be easily installed with the following command. + +``` +$ sudo apt-get install nitroshare +``` + +But the available version might be out of date, however, to install the latest version of Nitroshare, issue the command below to add the PPA for the latest packages: + +``` +$ sudo apt-add-repository ppa:george-edison55/nitroshare +$ sudo apt-get update +$ sudo apt-get install nitroshare +``` + +#### On Fedora 24-23 + +Recently, NitroShare has been included to Fedora repositories and can be installed with the following command: + +``` +$ sudo dnf install nitroshare +``` + +#### On Arch Linux + +For Arch Linux, NitroShare packages are available from the AUR and can be built/installed with the following commands: + +``` +# wget https://aur.archlinux.org/cgit/aur.git/snapshot/nitroshare.tar.gz +# tar xf nitroshare.tar.gz +# cd nitroshare +# makepkg -sri +``` + +### How to Use NitroShare on Linux + +Note: As I had already mentioned earlier on, all other machines that you wish to share files with on the local network must have Nitroshare installed and running. + +After successfully installing it, search for Nitroshare in the system dash or system menu and launch it. + +![](http://www.tecmint.com/wp-content/uploads/2016/09/NitroShare-Send-Files.png) + +After selecting the files, click on “Open” to proceed to choosing the destination device as in the image below. Select the device and click “Ok” that is if you have any devices running Nitroshare on the local network. + +![](http://www.tecmint.com/wp-content/uploads/2016/09/NitroShare-Local-Devices.png) + +![](http://www.tecmint.com/wp-content/uploads/2016/09/NitroShare-File-Transfer-Progress.png) + +From the NitroShare settings – General tab, you can add the device name, set default downloads location and in Advance settings you can set port, buffer, timeout, etc. only if you needed. + +Homepage: + +That’s it for now, if you have any issues regarding Nitroshare, you can share with us using our comment section below. You can as well make suggestions and let us know of any wonderful, cross-platform file sharing applications out there that we probably have no idea about and always remember to stay connected to Tecmint. + +-------------------------------------------------------------------------------- + +via: http://linoxide.com/firewall/pfsense-setup-basic-configuration/ + +作者:[Aaron Kili][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/author/aaronkili/ From 2a880b9426d4644dadb3558c9b734029f3bce85a Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:31:35 +0800 Subject: [PATCH 0155/2437] =?UTF-8?q?20160917-11=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...are Files Between Multiple Operating Systems on Local Network.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename sources/{talk => tech}/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md (100%) diff --git a/sources/talk/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md b/sources/tech/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md similarity index 100% rename from sources/talk/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md rename to sources/tech/20160914 NitroShare – Easily Share Files Between Multiple Operating Systems on Local Network.md From 518ee1e20ad341b4ae12e27410b8b39d70e98360 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:35:24 +0800 Subject: [PATCH 0156/2437] =?UTF-8?q?20160917-12=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...esktop in Ubuntu 16.04 and Fedora 22-24.md | 80 +++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md diff --git a/sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md b/sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md new file mode 100644 index 0000000000..070d464077 --- /dev/null +++ b/sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md @@ -0,0 +1,80 @@ +How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24 +==== + +Xfce is a modern, open source and lightweight desktop environment for Linux systems. It also works well on many other Unix-like systems such as Mac OS X, Solaris, *BSD plus several others. It is fast and also user friendly with a simple and elegant user interface. + +Installing a desktop environment on servers can sometimes prove helpful, as certain applications may require a desktop interface for efficient and reliable administration and one of the remarkable properties of Xfce is its low system resources utilization such as low RAM consumption, thereby making it a recommended desktop environment for servers if need be. + +### XFCE Desktop Features + +Additionally, some of its noteworthy components and features are listed below: + +- Xfwm windows manager +- Thunar file manager +- User session manger to deal with logins, power management and beyond +- Desktop manager for setting background image, desktop icons and many more +- An application manager +- It’s highly pluggable as well plus several other minor features + +The latest stable release of this desktop is Xfce 4.12, all its features and changes from previous versions are listed here. + +#### Install Xfce Desktop on Ubuntu 16.04 + +Linux distributions such as Xubuntu, Manjaro, OpenSUSE, Fedora Xfce Spin, Zenwalk and many others provide their own Xfce desktop packages, however, you can install the latest version as follows. + +``` +$ sudo apt update +$ sudo apt install xfce4 +``` + +Wait for the installation process to complete, then logout out of your current session or you can possibly restart your system as well. At the login interface, choose Xfce desktop and login as in the screen shot below: + +![](http://www.tecmint.com/wp-content/uploads/2016/09/Select-Xfce-Desktop-at-Login.png) + +![](http://www.tecmint.com/wp-content/uploads/2016/09/XFCE-Desktop.png) + +#### Install Xfce Desktop in Fedora 22-24 + +If you have an existing Fedora distribution and wanted to install xfce desktop, you can use yum or dnf to install it as shown. + +``` +-------------------- On Fedora 22 -------------------- +# yum install @xfce +-------------------- On Fedora 23-24 -------------------- +# dnf install @xfce-desktop-environment +``` + +After installing Xfce, you can choose the xfce login from the Session menu or reboot the system. + +![](http://www.tecmint.com/wp-content/uploads/2016/09/Select-Xfce-Desktop-at-Fedora-Login.png) + +![](http://www.tecmint.com/wp-content/uploads/2016/09/Install-Xfce-Desktop-in-Fedora.png) + +If you don’t want Xfce desktop on your system anymore, use the command below to uninstall it: + +``` +-------------------- On Ubuntu 16.04 -------------------- +$ sudo apt purge xfce4 +$ sudo apt autoremove +-------------------- On Fedora 22 -------------------- +# yum remove @xfce +-------------------- On Fedora 23-24 -------------------- +# dnf remove @xfce-desktop-environment +``` + +In this simple how-to guide, we walked through the steps for installation of latest version of Xfce desktop, which I believe were easy to follow. If all went well, you can enjoy using xfce, as one of the [best desktop environments for Linux systems][1]. + +However, to get back to us, you can use the feedback section below and remember to always stay connected to Tecmint. + +-------------------------------------------------------------------------------- + +via: http://linoxide.com/firewall/pfsense-setup-basic-configuration/ + +作者:[Aaron Kili ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/author/aaronkili/ +[1]: http://www.tecmint.com/best-linux-desktop-environments/ From 6cf2569553c666ba6fa170795e46cd7fd1c5d34a Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:38:34 +0800 Subject: [PATCH 0157/2437] =?UTF-8?q?20160917-13=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../20160827 4 Best Linux Boot Loaders.md | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 sources/tech/20160827 4 Best Linux Boot Loaders.md diff --git a/sources/tech/20160827 4 Best Linux Boot Loaders.md b/sources/tech/20160827 4 Best Linux Boot Loaders.md new file mode 100644 index 0000000000..91deb14d58 --- /dev/null +++ b/sources/tech/20160827 4 Best Linux Boot Loaders.md @@ -0,0 +1,76 @@ +4 Best Linux Boot Loaders +==== + +When you turn on your machine, immediately after POST (Power On Self Test) is completed successfully, the BIOS locates the configured bootable media, and reads some instructions from the master boot record (MBR) or GUID partition table which is the first 512 bytes of the bootable media. The MBR contains two important sets of information, one is the boot loader and two, the partition table. + +### What is a Boot Loader? + +A boot loader is a small program stored in the MBR or GUID partition table that helps to load an operating system into memory. Without a boot loader, your operating system can not be loaded into memory. + +There are several boot loaders we can install together with Linux on our systems and in this article, we shall briefly talk about a handful of the best Linux boot loaders to work with. + +### 1. GNU GRUB + +GNU GRUB is a popular and probably the most used multiboot Linux boot loader available, based on the original GRUB (GRand Unified Bootlader) which was created by Eirch Stefan Broleyn. It comes with several improvements, new features and bug fixes as enhancements of the original GRUB program. + +Importantly, GRUB 2 has now replaced the GRUB. And notably, the name GRUB was renamed to GRUB Legacy and is not actively developed, however, it can be used for booting older systems since bug fixes are still on going. + +GRUB has the following prominent features: + +- Supports multiboot +- Supports multiple hardware architectures and operating systems such as Linux and Windows +- Offers a Bash-like interactive command line interface for users to run GRUB commands as well interact with configuration files +- Enables access to GRUB editor +- Supports setting of passwords with encryption for security +- Supports booting from a network combined with several other minor features + +Visit Homepage: + +### 2. LILO (Linux Loader) + +LILO is a simple yet powerful and stable Linux boot loader. With the growing popularity and use of GRUB, which has come with numerous improvements and powerful features, LILO has become less popular among Linux users. + +While it loads, the word “LILO” is displayed on the screen and each letter appears before or after a particular event has occurred. However, the development of LILO was stopped in December 2015, it has a number of features as listed below: + +- Does not offer an interactive command line interface +- Supports several error codes +- Offers no support for booting from a network +- All its files are stored in the first 1024 cylinders of a drive +- Faces limitation with BTFS, GPT and RAID plus many more. + +Visit Homepage: + +### 3. BURG – New Boot Loader + +Based on GRUB, BURG is a relatively new Linux boot loader. Because it is derived from GRUB, it ships in with some of the primary GRUB features, nonetheless, it also offers remarkable features such as a new object format to support multiple platforms including Linux, Windows, Mac OS, FreeBSD and beyond. + +Additionally, it supports a highly configurable text and graphical mode boot menu, stream plus planned future improvements for it to work with various input/output devices. + +Visit Homepage: + +### 4. Syslinux + +Syslinux is an assortment of light weight boot loaders that enable booting from CD-ROMs, from a network and so on. It supports filesystems such as FAT for MS-DOS, and ext2, ext3, ext4 for Linux. It as well supports uncompressed single-device Btrfs. + +Note that Syslinux only accesses files in its own partition, therefore, it does not offer multi-filesystem boot capabilities. + +Visit Homepage: + +### Conclusion + +A boot loader allows you to manage multiple operating systems on your machine and select which one to use at a particular time, without it, your machine can not load the kernel and the rest of the operating system files. + +Have we missed any tip-top Linux boot loader here? If so, then let us know by using the comment form below by making suggestions of any commendable boot loaders that can support Linux operating system. + + +-------------------------------------------------------------------------------- + +via: http://linoxide.com/firewall/pfsense-setup-basic-configuration/ + +作者:[Aaron Kili][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/best-linux-boot-loaders/ From e74771dbf9f9f54ea1c4f101ea4ea77b14cfcfc2 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:45:23 +0800 Subject: [PATCH 0158/2437] =?UTF-8?q?20160917-14=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...Shell Commands and Their Usage in Linux.md | 219 ++++++++++++++++++ 1 file changed, 219 insertions(+) create mode 100644 sources/tech/20160826 Understanding Different Classifications of Shell Commands and Their Usage in Linux.md diff --git a/sources/tech/20160826 Understanding Different Classifications of Shell Commands and Their Usage in Linux.md b/sources/tech/20160826 Understanding Different Classifications of Shell Commands and Their Usage in Linux.md new file mode 100644 index 0000000000..29b3bf04da --- /dev/null +++ b/sources/tech/20160826 Understanding Different Classifications of Shell Commands and Their Usage in Linux.md @@ -0,0 +1,219 @@ +Understanding Different Classifications of Shell Commands and Their Usage in Linux +==== + +When it comes to gaining absolute control over your Linux system, then nothing comes close to the command line interface (CLI). In order to become a Linux power user, one must understand the [different types of shell commands][1] and the appropriate ways of using them from the terminal. + +In Linux, there are several types of commands, and for a new Linux user, knowing the meaning of different commands enables for efficient and precise usage. Therefore, in this article, we shall walk through the various classifications of shell commands in Linux. + +One important thing to note is that the command line interface is different from the shell, it only provides a means for you to access the shell. The shell, which is also programmable then makes it possible to communicate with the kernel using commands. + +Different classifications of Linux commands fall under the following classifications: + +### 1. Program Executables (File System Commands) + +When you run a command, Linux searches through the directories stored in the $PATH environmental variable from left to right for the executable of that specific command. + +You can view the directories in the $PATH as follows: + +``` +$ echo $PATH +/home/aaronkilik/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games +``` + +In the above order, the directory /home/aaronkilik/bin will be searched first followed by /usr/local/sbin and so on, the order is significant in the search process. + +Examples of file system commands in /usr/bin directory: + +``` +$ ll /bin/ +``` + +Sample Output + +``` +total 16284 +drwxr-xr-x 2 root root 4096 Jul 31 16:30 ./ +drwxr-xr-x 23 root root 4096 Jul 31 16:29 ../ +-rwxr-xr-x 1 root root 6456 Apr 14 18:53 archdetect* +-rwxr-xr-x 1 root root 1037440 May 17 16:15 bash* +-rwxr-xr-x 1 root root 520992 Jan 20 2016 btrfs* +-rwxr-xr-x 1 root root 249464 Jan 20 2016 btrfs-calc-size* +lrwxrwxrwx 1 root root 5 Jul 31 16:19 btrfsck -> btrfs* +-rwxr-xr-x 1 root root 278376 Jan 20 2016 btrfs-convert* +-rwxr-xr-x 1 root root 249464 Jan 20 2016 btrfs-debug-tree* +-rwxr-xr-x 1 root root 245368 Jan 20 2016 btrfs-find-root* +-rwxr-xr-x 1 root root 270136 Jan 20 2016 btrfs-image* +-rwxr-xr-x 1 root root 249464 Jan 20 2016 btrfs-map-logical* +-rwxr-xr-x 1 root root 245368 Jan 20 2016 btrfs-select-super* +-rwxr-xr-x 1 root root 253816 Jan 20 2016 btrfs-show-super* +-rwxr-xr-x 1 root root 249464 Jan 20 2016 btrfstune* +-rwxr-xr-x 1 root root 245368 Jan 20 2016 btrfs-zero-log* +-rwxr-xr-x 1 root root 31288 May 20 2015 bunzip2* +-rwxr-xr-x 1 root root 1964536 Aug 19 2015 busybox* +-rwxr-xr-x 1 root root 31288 May 20 2015 bzcat* +lrwxrwxrwx 1 root root 6 Jul 31 16:19 bzcmp -> bzdiff* +-rwxr-xr-x 1 root root 2140 May 20 2015 bzdiff* +lrwxrwxrwx 1 root root 6 Jul 31 16:19 bzegrep -> bzgrep* +-rwxr-xr-x 1 root root 4877 May 20 2015 bzexe* +lrwxrwxrwx 1 root root 6 Jul 31 16:19 bzfgrep -> bzgrep* +-rwxr-xr-x 1 root root 3642 May 20 2015 bzgrep* +``` + +### 2. Linux Aliases + +These are user defined commands, they are created using the alias shell built-in command, and contain other shell commands with some options and arguments. The ideas is to basically use new and short names for lengthy commands. + +The syntax for creating an alias is as follows: + +``` +$ alias newcommand='command -options' +``` + +To list all aliases on your system, issue the command below: + +``` +$ alias -p +alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"' +alias egrep='egrep --color=auto' +alias fgrep='fgrep --color=auto' +alias grep='grep --color=auto' +alias l='ls -CF' +alias la='ls -A' +alias ll='ls -alF' +alias ls='ls --color=auto' +``` + +To create a new alias in Linux, go through some below examples. + +``` +$ alias update='sudo apt update' +$ alias upgrade='sudo apt dist-upgrade' +$ alias -p | grep 'up' +``` + +![](http://www.tecmint.com/wp-content/uploads/2016/08/Create-Aliase-in-Linux.png) + +However, the aliases we have created above only work temporarily, when the system is restarted, they will not work after the next boot. You can set permanent aliases in your `.bashrc` file as shown below. + +![](http://www.tecmint.com/wp-content/uploads/2016/08/Set-Linux-Aliases-Permanent.png) + +After adding them, run the command below to active. + +``` +$ source ~/.bashrc +``` + +### 3. Linux Shell Reserved Words + +In shell programming, words such as if, then, fi, for, while, case, esac, else, until and many others are shell reserved words. As the description implies, they have specialized meaning to the shell. + +You can list out all Linux shell keywords using type command as shown: + +``` +$ type if then fi for while case esac else until +if is a shell keyword +then is a shell keyword +fi is a shell keyword +for is a shell keyword +while is a shell keyword +case is a shell keyword +esac is a shell keyword +else is a shell keyword +until is a shell keyword +``` + +Suggested Read: 10 Useful Linux Chaining Operators with Practical Examples + +### 4. Linux Shell Functions + +A shell function is a group of commands that are executed collectively within the current shell. Functions help to carry out a specific task in a shell script. The conventional form of writing shell functions in a script is: + +``` +function_name() { +command1 +command2 +……. +} +``` + +Alternatively, + +``` +function function_name { +command1 +command2 +……. +} +``` + +Let’s take a look at how to write shell functions in a script named shell_functions.sh. + +``` +#!/bin/bash +#write a shell function to update and upgrade installed packages +upgrade_system(){ +sudo apt update; +sudo apt dist-upgrade; +} +#execute function +upgrade_system +``` + +Instead of executing the two commands: sudo apt update and sudo apt dist-upgrade from the command line, we have written a simple shell function to execute the two commands as a single command, upgrade_system within a script. + +Save the file and thereafter, make the script executable. Finally run it as below: + +``` +$ chmod +x shell_functions.sh +$ ./shell_functions.sh +``` + +![](http://www.tecmint.com/wp-content/uploads/2016/08/Linux-Shell-Functions-Script.png) + +### 5. Linux Shell Built-in Commands + +These are Linux commands that built into the shell, thus you cannot find them within the file system. They include pwd, cd, bg, alias, history, type, source, read, exit and many others. + +You can list or check Linux built-in commands using type command as shown: + +``` +$ type pwd +pwd is a shell builtin +$ type cd +cd is a shell builtin +$ type bg +bg is a shell builtin +$ type alias +alias is a shell builtin +$ type history +history is a shell builtin +``` + +Learn about some Linux built-in Commands usage: + +- [15 ‘pwd’ Command Examples in Linux][2] +- [15 ‘cd’ Command Examples in Linux][3] +- [Learn The Power of Linux ‘history’ Command][4] + +### Conclusion + +As a Linux user, it is always important to know the type of command you are running. I believe, with the precise and simple-to-understand explanation above including a few relevant illustrations, you probably have a good understanding of the [various categories of Linux commands][5]. + +You can as well get in tough through the comment section below for any questions or supplementary ideas that you would like to offer us. + +-------------------------------------------------------------------------------- + +via: http://linoxide.com/firewall/pfsense-setup-basic-configuration/ + +作者:[Aaron Kili ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/author/aaronkili/ +[1]: http://www.tecmint.com/different-types-of-linux-shells/ +[2]: http://www.tecmint.com/pwd-command-examples/ +[3]: http://www.tecmint.com/cd-command-in-linux/ +[4]: http://www.tecmint.com/history-command-examples/ +[5]: http://www.tecmint.com/category/linux-commands/ From de1d6ba687e5bd81d7e39918b3162497ad305567 Mon Sep 17 00:00:00 2001 From: Bian Jiaping Date: Sat, 17 Sep 2016 21:48:21 +0800 Subject: [PATCH 0159/2437] [Translating] It's time to make LibreOffice and OpenOffice one again --- ...14 It's time to make LibreOffice and OpenOffice one again.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md b/sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md index 3aba30f29b..d91d08d03a 100644 --- a/sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md +++ b/sources/tech/20160914 It's time to make LibreOffice and OpenOffice one again.md @@ -1,3 +1,5 @@ +Translating by bianjp + It's time to make LibreOffice and OpenOffice one again ========== From 05fe2f738345dd06976e11fba2a504e596880c8f Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:52:24 +0800 Subject: [PATCH 0160/2437] =?UTF-8?q?20160917-15=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...w to Use Flow Control Statements in Awk.md | 246 ++++++++++++++++++ 1 file changed, 246 insertions(+) create mode 100644 sources/tech/awk/part 12 - How to Use Flow Control Statements in Awk.md diff --git a/sources/tech/awk/part 12 - How to Use Flow Control Statements in Awk.md b/sources/tech/awk/part 12 - How to Use Flow Control Statements in Awk.md new file mode 100644 index 0000000000..37bd91461f --- /dev/null +++ b/sources/tech/awk/part 12 - How to Use Flow Control Statements in Awk.md @@ -0,0 +1,246 @@ +How to Use Flow Control Statements in Awk - part12 +==== + +When you review all the Awk examples we have covered so far, right from the start of the Awk series, you will notice that all the commands in the various examples are executed sequentially, that is one after the other. But in certain situations, we may want to run some text filtering operations based on some conditions, that is where the approach of flow control statements sets in. + +![](http://www.tecmint.com/wp-content/uploads/2016/08/Use-Flow-Control-Statements-in-Awk.png) + +There are various flow control statements in Awk programming and these include: + +- if-else statement +- for statement +- while statement +- do-while statement +- break statement +- continue statement +- next statement +- nextfile statement +- exit statement + +However, for the scope of this series, we shall expound on: if-else, for, while and do while statements. Remember that we already walked through how to use next statement in Part 6 of this Awk series. + +### 1. The if-else Statement + +The expected syntax of the if statement is similar to that of the shell if statement: + +``` +if (condition1) { +actions1 +} +else { +actions2 +} +``` + +In the above syntax, condition1 and condition2 are Awk expressions, and actions1 and actions2 are Awk commands executed when the respective conditions are satisfied. + +When condition1 is satisfied, meaning it’s true, then actions1 is executed and the if statement exits, otherwise actions2 is executed. + +The if statement can also be expanded to a if-else_if-else statement as below: + +``` +if (condition1){ +actions1 +} +else if (conditions2){ +actions2 +} +else{ +actions3 +} +``` + +For the form above, if condition1 is true, then actions1 is executed and the if statement exits, otherwise condition2 is evaluated and if it is true, then actions2 is executed and the if statement exits. However, when condition2 is false then, actions3 is executed and the if statement exits. + +Here is a case in point of using if statements, we have a list of users and their ages stored in the file, users.txt. + +We want to print a statement indicating a user’s name and whether the user’s age is less or more than 25 years old. + +``` +aaronkilik@tecMint ~ $ cat users.txt +Sarah L 35 F +Aaron Kili 40 M +John Doo 20 M +Kili Seth 49 M +``` + +We can write a short shell script to carry out our job above, here is the content of the script: + +``` +#!/bin/bash +awk ' { +if ( $3 <= 25 ){ +print "User",$1,$2,"is less than 25 years old." ; +} +else { +print "User",$1,$2,"is more than 25 years old" ; +} +}' ~/users.txt +``` + +Then save the file and exit, make the script executable and run it as follows: + +``` +$ chmod +x test.sh +$ ./test.sh +``` + +Sample Output + +``` +User Sarah L is more than 25 years old +User Aaron Kili is more than 25 years old +User John Doo is less than 25 years old. +User Kili Seth is more than 25 years old +``` + +### 2. The for Statement + +In case you want to execute some Awk commands in a loop, then the for statement offers you a suitable way to do that, with the syntax below: + +Here, the approach is simply defined by the use of a counter to control the loop execution, first you need to initialize the counter, then run it against a test condition, if it is true, execute the actions and finally increment the counter. The loop terminates when the counter does not satisfy the condition. + +``` +for ( counter-initialization; test-condition; counter-increment ){ +actions +} +``` + +The following Awk command shows how the for statement works, where we want to print the numbers 0-10: + +``` +$ awk 'BEGIN{ for(counter=0;counter<=10;counter++){ print counter} }' +``` + +Sample Output + +``` +0 +1 +2 +3 +4 +5 +6 +7 +8 +9 +10 +``` + +### 3. The while Statement + +The conventional syntax of the while statement is as follows: + +``` +while ( condition ) { +actions +} +``` + +The condition is an Awk expression and actions are lines of Awk commands executed when the condition is true. + +Below is a script to illustrate the use of while statement to print the numbers 0-10: + +``` +#!/bin/bash +awk ' BEGIN{ counter=0 ; +while(counter<=10){ +print counter; +counter+=1 ; +} +} +``` + +Save the file and make the script executable, then run it: + +``` +$ chmod +x test.sh +$ ./test.sh +``` + +Sample Output + +``` +0 +1 +2 +3 +4 +5 +6 +7 +8 +9 +10 +``` + +### 4. The do while Statement + +It is a modification of the while statement above, with the following underlying syntax: + +``` +do { +actions +} +while (condition) +``` + +The slight difference is that, under do while, the Awk commands are executed before the condition is evaluated. Using the very example under while statement above, we can illustrate the use of do while by altering the Awk command in the test.sh script as follows: + +``` +#!/bin/bash +awk ' BEGIN{ counter=0 ; +do{ +print counter; +counter+=1 ; +} +while (counter<=10) +} +' +``` + +After modifying the script, save the file and exit. Then make the script executable and execute it as follows: + +``` +$ chmod +x test.sh +$ ./test.sh +``` + +Sample Output + +``` +0 +1 +2 +3 +4 +5 +6 +7 +8 +9 +10 +``` + +### Conclusion + +This is not a comprehensive guide regarding Awk flow control statements, as I had mentioned earlier on, there are several other flow control statements in Awk. + +Nonetheless, this part of the Awk series should give you a clear fundamental idea of how execution of Awk commands can be controlled based on certain conditions. + +You can as well expound more on the rest of the flow control statements to gain more understanding on the subject matter. Finally, in the next section of the Awk series, we shall move into writing Awk scripts. + +-------------------------------------------------------------------------------- + +via: http://www.tecmint.com/use-flow-control-statements-with-awk-command/ + +作者:[Aaron Kili][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/author/aaronkili/ + + From 6e2e5976065f885d2e6df303a43fe48126a605dd Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 21:56:44 +0800 Subject: [PATCH 0161/2437] =?UTF-8?q?20160917-16=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Scripts Using Awk Programming Language.md | 160 ++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 sources/tech/awk/Part 13 - How to Write Scripts Using Awk Programming Language.md diff --git a/sources/tech/awk/Part 13 - How to Write Scripts Using Awk Programming Language.md b/sources/tech/awk/Part 13 - How to Write Scripts Using Awk Programming Language.md new file mode 100644 index 0000000000..05ef0902fe --- /dev/null +++ b/sources/tech/awk/Part 13 - How to Write Scripts Using Awk Programming Language.md @@ -0,0 +1,160 @@ +Part 13 - How to Write Scripts Using Awk Programming Language +==== + +All along from the beginning of the Awk series up to Part 12, we have been writing small Awk commands and programs on the command line and in shell scripts respectively. + +However, Awk, just as Shell, is also an interpreted language, therefore, with all that we have walked through from the start of this series, you can now write Awk executable scripts. + +Similar to how we write a shell script, Awk scripts start with the line: + +``` +#! /path/to/awk/utility -f +``` + +For example on my system, the Awk utility is located in /usr/bin/awk, therefore, I would start an Awk script as follows: + +``` +#! /usr/bin/awk -f +``` + +Explaining the line above: + +``` +#! – referred to as Shebang, which specifies an interpreter for the instructions in a script +/usr/bin/awk – is the interpreter +-f – interpreter option, used to read a program file +``` + +That said, let us now dive into looking at some examples of Awk executable scripts, we can start with the simple script below. Use your favorite editor to open a new file as follows: + +``` +$ vi script.awk +``` + +And paste the code below in the file: + +``` +#!/usr/bin/awk -f +BEGIN { printf "%s\n","Writing my first Awk executable script!" } +``` + +Save the file and exit, then make the script executable by issuing the command below: + +``` +$ chmod +x script.awk +``` + +Thereafter, run it: + +``` +$ ./script.awk +``` + +Sample Output + +``` +Writing my first Awk executable script! +``` + +A critical programmer out there must be asking, “where are the comments?”, yes, you can also include comments in your Awk script. Writing comments in your code is always a good programming practice. + +It helps other programmers looking through your code to understand what you are trying to achieve in each section of a script or program file. + +Therefore, you can include comments in the script above as follows. + +``` +#!/usr/bin/awk -f +#This is how to write a comment in Awk +#using the BEGIN special pattern to print a sentence +BEGIN { printf "%s\n","Writing my first Awk executable script!" } +``` + +Next, we shall look at an example where we read input from a file. We want to search for a system user named aaronkilik in the account file, /etc/passwd, then print the username, user ID and user GID as follows: + +Below is the content of our script called second.awk. + +``` +#! /usr/bin/awk -f +#use BEGIN sepecial character to set FS built-in variable +BEGIN { FS=":" } +#search for username: aaronkilik and print account details +/aaronkilik/ { print "Username :",$1,"User ID :",$3,"User GID :",$4 } +``` + +Save the file and exit, make the script executable and execute it as below: + +``` +$ chmod +x second.awk +$ ./second.awk /etc/passwd +``` + +Sample Output + +``` +Username : aaronkilik User ID : 1000 User GID : 1000 +``` + +In the last example below, we shall use do while statement to print out numbers from 0-10: + +Below is the content of our script called do.awk. + +``` +#! /usr/bin/awk -f +#printing from 0-10 using a do while statement +#do while statement +BEGIN { +#initialize a counter +x=0 +do { +print x; +x+=1; +} +while(x<=10) +} +``` + +After saving the file, make the script executable as we have done before. Afterwards, run it: + +``` +$ chmod +x do.awk +$ ./do.awk +``` + +Sample Output + +``` +0 +1 +2 +3 +4 +5 +6 +7 +8 +9 +10 +``` + +### Summary + +We have come to the end of this interesting Awk series, I hope you have learned a lot from all the 13 parts, as an introduction to Awk programming language. + +As I mentioned from the beginning, Awk is a complete text processing language, for that reason, you can learn more other aspects of Awk programming language such as environmental variables, arrays, functions (built-in & user defined) and beyond. + +There is yet additional parts of Awk programming to learn and master, so, below, I have provided some links to important online resources that you can use to expand your Awk programming skills, these are not necessarily all that you need, you can also look out for useful Awk programming books. + + +For any thoughts you wish to share or questions, use the comment form below. Remember to always stay connected to Tecmint for more exciting series. + +-------------------------------------------------------------------------------- + +via: http://www.tecmint.com/write-shell-scripts-in-awk-programming/ + +作者:[Aaron Kili |][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.tecmint.com/author/aaronkili/ From fd26bb6a7bb274b48bf70174ffad4bdbf2c64a0c Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:05:24 +0800 Subject: [PATCH 0162/2437] =?UTF-8?q?20160917-17=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...sing Ansible to Provision Vagrant Boxes.md | 159 ++++++++++++++++++ 1 file changed, 159 insertions(+) create mode 100644 sources/tech/20160915 Using Ansible to Provision Vagrant Boxes.md diff --git a/sources/tech/20160915 Using Ansible to Provision Vagrant Boxes.md b/sources/tech/20160915 Using Ansible to Provision Vagrant Boxes.md new file mode 100644 index 0000000000..57003390d8 --- /dev/null +++ b/sources/tech/20160915 Using Ansible to Provision Vagrant Boxes.md @@ -0,0 +1,159 @@ +Using Ansible to Provision Vagrant Boxes +==== + +![](https://i1.wp.com/cdn.fedoramagazine.org/wp-content/uploads/2016/08/vagrant-plus-ansible.jpg?w=1352&ssl=1) + +Ansible is a great tool for system administrators who want to automate system administration tasks. From configuration management to provisioning and managing containers for application deployments, Ansible [makes it easy][1]. The lightweight module based architecture is ideal for system administration. One advantage is that when the node is not being managed by Ansible, no resources are used. + +This article covers how to use Ansible to provision Vagrant boxes. A [Vagrant box][2] in simple terms is a virtual machine prepackaged with tools required to run the development environment. You can use these boxes to distribute the development environment used by other team members for project work. Using Ansible, you can automate provisioning the Vagrant boxes with your development packages. + +This tutorial uses Fedora 24 as the host system and [CentOS][3] 7 as the Vagrant box. + +### Setting up prerequisites + +To configure Vagrant boxes using Ansible, you’ll need a few things setup. This tutorial requires you to install Ansible and Vagrant on the host machine. On your host machine, execute the following command to install the tools: + +``` +sudo dnf install ansible vagrant vagrant-libvirt +``` + +The above command installs both Ansible and Vagrant on your host system, along with Vagrant’s libvirt provider. Vagrant doesn’t provide functionality to host your virtual machine guests (VMs). Rather, it depends on third party providers such as libvirt, VirtualBox, VMWare, etc. to host the VMs. This provider works directly with libvirt and KVM on your Fedora system. + +Next, make sure your user is in the wheel group. This special group allows you to run system administration commands. If you created your user as an administrator, such as during installation, you’ll have this group membership. Run the following command: + +``` +id | grep wheel +``` + +If you see output, your user is in the group, and you can move on to the next section. If not, run the following command. You’ll need to provide the password for the root account. Substitute your user name for the text : + +``` +su -c 'usermod -a -G wheel ' +``` + +Then you will need to logout, and log back in, to inherit the group membership properly. + +Now it’s time to create your first Vagrant box, which you’ll then configure using Ansible. + +### Setting up the Vagrant box + +Before you use Ansible to provision a box, you must create the box. To start, create a new directory which will store files related to the Vagrant box. To create this directory and make it the current working directory, issue the following command: + +``` +mkdir -p ~/lampbox && cd ~/lampbox +``` + +Before you create the box, you should understand the goal. This box is a simple example that runs CentOS 7 as its base system, along with the Apache web server, MariaDB (the popular open source database server from the original developers of MySQL) and PHP. + +To initialize the Vagrant box, use the vagrant init command: + +``` +vagrant init centos/7 +``` + +This command initializes the Vagrant box and creates a file named Vagrantfile, with some pre-configured variables. Open this file so you can modify it. The following line lists the base box used by this configuration. + +``` +config.vm.box = "centos/7" +``` + +Now setup port forwarding, so after you finish setup and the Vagrant box is running, you can test the server. To setup port forwarding, add the following line just before the end statement in Vagrantfile: + +``` +config.vm.network "forwarded_port", guest: 80, host: 8080 +``` + +This option maps port 80 of the Vagrant Box to port 8080 of the host machine. + +The next step is to set Ansible as our provisioning provider for the Vagrant Box. Add the following lines before the end statement in your Vagrantfile to set Ansible as the provisioning provider: + +``` +config.vm.provision :ansible do |ansible| + ansible.playbook = "lamp.yml" +end +``` + +(You must add all three lines before the final end statement.) Notice the statement ansible.playbook = “lamp.yml”. This statement defines the name of the playbook used to provision the box. + +### Creating the Ansible playbook + +In Ansible, playbooks describe a policy to be enforced on your remote nodes. Put another way, playbooks manage configurations and deployments on remote nodes. Technically speaking, a playbook is a YAML file in which you write tasks to perform on remote nodes. In this tutorial, you’ll create a playbook named lamp.yml to provision the box. + +To make the playbook, create a file named lamp.yml in the same directory where your Vagrantfile is located and add the following lines to it: + +``` +--- +- hosts: all + become: yes + become_user: root + tasks: + - name: Install Apache + yum: name=httpd state=latest + - name: Install MariaDB + yum: name=mariadb-server state=latest + - name: Install PHP5 + yum: name=php state=latest + - name: Start the Apache server + service: name=httpd state=started + - name: Install firewalld + yum: name=firewalld state=latest + - name: Start firewalld + service: name=firewalld state=started + - name: Open firewall + command: firewall-cmd --add-service=http --permanent +``` + +An explanation of each line of lamp.yml follows. + +- hosts: all specifies the playbook should run over every host defined in the Ansible configuration. Since no hosts are configured hosts yet, the playbook will run on localhost. +- sudo: true states the tasks should be performed with root privileges. +- tasks: specifies the tasks to perform when the playbook runs. Under the tasks section: +- - name: … provides a descriptive name to the task +- - yum: … specifies the task should be executed by the yum module. The options name and state are key=value pairs for use by the yum module. + +When this playbook executes, it installs the latest versions of the Apache (httpd) web server, MariaDB, and PHP. Then it installs and starts firewalld, and opens a port for the Apache server. You’re now done writing the playbook for the box. Now it’s time to provision it. + +### Provisioning the box + +A few final steps remain before using the Vagrant Box provisioned using Ansible. To run this provisioning, execute the following command: + +``` +vagrant up --provider libvirt +``` + +The above command starts the Vagrant box, downloads the base box image to the host system if not already present, and then runs the playbook lamp.yml to provision. + +If everything works fine, the output looks somewhat similar to this example: + +![](https://i1.wp.com/cdn.fedoramagazine.org/wp-content/uploads/2016/08/vagrant-ansible-playbook-run.png?w=574&ssl=1) + +This output shows that the box has been provisioned. Now check whether the server is accessible. To confirm, open your web browser on the host machine and point it to the address http://localhost:8080. Remember, port 8080 of the local host is forwarded to port 80 of the Vagrant box. You should be greeted with the Apache welcome page like the one shown below: + +![](https://i0.wp.com/cdn.fedoramagazine.org/wp-content/uploads/2016/08/vagrant-ansible-apache-up.png?w=1004&ssl=1) + +To make changes to your Vagrant box, first edit the Ansible playbook lamp.yml. You can find plentiful documentation on Ansible at [its official website][4]. Then run the following command to re-provision the box: + +``` +vagrant provision +``` + +### Conclusion + +You’ve now seen how to use Ansible to provision Vagrant boxes. This was a basic example, but you can use these tools for many other use cases. For example, you can deploy complete applications along with up-to-date version of required tools. Be creative as you use Ansible to provision your remote nodes or containers. + + +-------------------------------------------------------------------------------- + +via: https://fedoramagazine.org/using-ansible-provision-vagrant-boxes/ + +作者:[Saurabh Badhwar][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://h4xr.id.fedoraproject.org/ +[1]: https://ansible.com/ +[2]: https://www.vagrantup.com/ +[3]: https://centos.org/ +[4]: http://docs.ansible.com/ansible/index.html From ea4ae2d514b30bc7d2ae07e1c99bd1e51c513248 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:08:38 +0800 Subject: [PATCH 0163/2437] =?UTF-8?q?20160917-18=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...rkdown in WordPress to Improve Workflow.md | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 sources/tech/20160913 How to Use Markdown in WordPress to Improve Workflow.md diff --git a/sources/tech/20160913 How to Use Markdown in WordPress to Improve Workflow.md b/sources/tech/20160913 How to Use Markdown in WordPress to Improve Workflow.md new file mode 100644 index 0000000000..a199eddabe --- /dev/null +++ b/sources/tech/20160913 How to Use Markdown in WordPress to Improve Workflow.md @@ -0,0 +1,58 @@ +How to Use Markdown in WordPress to Improve Workflow +==== + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/09/markdown-wordpress-featured-2.jpg) + +Markdown is a simple markup language that helps you format your plain text documents with minimal effort. You may be used to formatting your articles using HTML or the Visual Editor in WordPress, but using markdown makes formatting a lot easier, and you can always export it to several formats including (but not limited to) HTML. + +WordPress does not come with native markdown support, but there are plugins that can add this functionality to your website if you so desire. + +In this tutorial I will demonstrate how to use the popular WP-Markdown plugin to add markdown support to a WordPress website. + +### Installation + +You can install this plugin directly by navigating to “Plugins -> Add New” and entering “[wp-markdown][1]” in the search box provided. The plugin should appear as the first option on the list. Click “Install Now” to install it. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/08/markdown-wordpress-install-plugin-1.png) + +### Configuration + +Once you have installed the plugin and activated it, navigate to “Settings -> Writing” in the menu and scroll down until you get to the markdown section. + +You can enable markdown support in posts, pages and comments. You can also enable a help bar for your post editor or comments which could be handy if you’re just learning the markdown syntax. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/09/markdown-wordpress-configuration.png) + +If you include code snippets in your blog posts, enabling the “Prettify syntax highlighter” option will automatically provide syntax highlighting for your code snippets. + +Once you are satisfied with your selections, click “Save Changes” to save your settings. + +### Write your posts with Markdown + +Once you have enabled markdown support on your website, you can start using it right away. + +Create a new post by going to “Posts -> Add New.” You will notice that the default Visual and Plain Text editors have been replaced by the markdown editor. + +If you did not enable the markdown help bar in the configuration options, you will not see a live preview of your formatted markdown. Nonetheless, as long as your syntax is correct, your markdown will be converted to valid HTML when you save or publish the post. + +However, if you’re a beginner to markdown and the live preview feature is important to you, simply go back to the settings to enable the help bar option, and you will get a nice live preview area at the bottom of your posts. In addition, you also get some buttons on top that will help you quickly insert markdown syntax into your posts. This could be a potentially amazing setting if people use it. You can adjust the priority of the notification on individual apps. This will let you choose what you see in the notification bar. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/08/markdown-wordpress-create-post.png) + +### Wrap up + +As you can see adding markdown support to a WordPress website is really easy, and it will only take a few minutes of your time. If you are completely new to markdown, you might also check out our [markdown cheatsheet][2] which provides a comprehensive reference to the markdown syntax. + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/use-markdown-in-wordpress/?utm_medium=feed&utm_source=feedpress.me&utm_campaign=Feed%3A+maketecheasier + +作者:[Ayo Isaiah][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.maketecheasier.com/author/ayoisaiah/ +[1]: https://wordpress.org/plugins/wp-markdown/ +[2]: https://www.maketecheasier.com/productive-with-markdown-cheatsheet/ From b989a9598bbb44ada270021f01759ca819a3ce5b Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:12:51 +0800 Subject: [PATCH 0164/2437] =?UTF-8?q?20160917-19=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...reveals his favorite programming laptop.md | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 sources/talk/20160913 Linus Torvalds reveals his favorite programming laptop.md diff --git a/sources/talk/20160913 Linus Torvalds reveals his favorite programming laptop.md b/sources/talk/20160913 Linus Torvalds reveals his favorite programming laptop.md new file mode 100644 index 0000000000..9741a58fcc --- /dev/null +++ b/sources/talk/20160913 Linus Torvalds reveals his favorite programming laptop.md @@ -0,0 +1,62 @@ +Linus Torvalds reveals his favorite programming laptop +==== + +>It's the Dell XPS 13 Developer Edition. Here's why. + +I recently talked with some Linux developers about what the best laptop is for serious programmers. As a result I checked out several laptops from a programmer's viewpoint. The winner in my book? The 2016 Dell XPS 13 Developer Edition. I'm in good company. Linus Torvalds, Linux's creator, agrees. The Dell XPS 13 Developer Edition, for him, is the best laptop around. + +![](http://zdnet3.cbsistatic.com/hub/i/r/2016/07/18/702609c3-db38-4603-9f5f-4dcc3d71b140/resize/770xauto/50a8ba1c2acb1f0994aec2115d2e55ce/2016-dell-xps-13.jpg) + +Torvald's requirements may not be yours though. + +On Google+, Torvalds explained, "First off: [I don't use my laptop as a desktop replacement][1], and I only travel for a small handful of events each year. So for me, the laptop is a fairly specialized thing that doesn't get daily (or even weekly) use, so the main criteria are not some kind of "average daily use", but very much "travel use". + +Therefore, for Torvalds, "I end up caring a lot about it being fairly small and light, because I may end up carrying it around all day at a conference. I also want it to have a good screen, because by now I'm just used to it at my main desktop, and I want my text to be legible but small." + +The Dell's display is powered by Intel's Iris 540 GPU. In my experience it works really well. + +The Iris powers a 13.3 inch display with a 3,200×1,800 touchscreen. That's 280 pixels per inch, 40 more than my beloved [2015 Chromebook Pixel][2] and 60 more than a [MacBook Pro with Retina][3]. + +However, getting that hardware to work and play well with the [Gnome][4] desktop isn't easy. As Torvalds explained in another post, it "has the [same resolution as my desktop][5], but apparently because the laptop screen is smaller, Gnome seems to decide on its own that I need an automatic scaling factor of 2, which blows up all the stupid things (window decorations, icons etc) to a ridiculous degree". + +The solution? You can forget about looking to the user interface. You need to go to the shell and run: gsettings set org.gnome.desktop.interface scaling-factor 1. + +Torvalds may use Gnome, but he's [never liked the Gnome 3.x family much][6]. I can't argue with him. That's why I use [Cinnamon][7] instead. + +He also wants "a reasonably powerful CPU, because when I'm traveling I still build the kernel a lot. I don't do my normal full 'make allmodconfig' build between each pull request like I do at home, but I'd like to do it more often than I did with my previous laptop, which is actually (along with the screen) the main reason I wanted to upgrade." + +Linus doesn't describe the features of his XPS 13, but my review unit was a high-end model. It came with dual-core, 2.2GHz 6th Generation Intel Core i7-6560U Skylake processor and 16GBs of DDR3 RAM with a half a terabyte, PCIe solid state drive (SSD). I'm sure Torvalds' system is at least that well-equipped. + +Some features you may care about aren't on Torvalds' list. + +>"What I don't tend to care about is touch-screens, because my fingers are big and clumsy compared to the text I'm looking at (I also can't handle the smudges: maybe I just have particularly oily fingers, but I really don't want to touch that screen). + +I also don't care deeply about some 'all day battery life', because quite frankly, I can't recall the last time I didn't have access to power. I might not want to bother to plug it in for some quick check, but it's just not a big overwhelming issue. By the time battery life is in 'more than a couple of hours', I just don't care very much any more." +Dell claims the XPS 13, with its 56wHR, 4-Cell Battery, has about a 12-hour battery life. It has well over 10 in my experience. I haven't tried to run it down to the dregs. + +Torvalds also didn't have any trouble with the Intel Wi-Fi set. The non Developer Edition uses a Broadcom chip set and that has proven troublesome for both Windows and Linux users. Dell technical support was extremely helpful to me in getting this problem under control. + +Some people have trouble with the XPS 13 touchpad. Neither I nor Torvalds have any worries. Torvalds wrote, the "XPS13 touchpad works very well for me. That may be a personal preference thing, but it seems to be both smooth and responsive." + +Still, while Torvalds likes the XPS 13, he's also fond of the latest Lenovo X1 Carbon, HP Spectre 13 x360, and last year's Lenovo Yoga 900. Me? I like the XPS 13 Developer Editor. The price tag, which for the model I reviewed was $1949.99, may keep you from reaching for your credit card. + +Still, if you want to develop like one of the world's top programmers, the Dell XPS 13 Developer Edition is worth the money. + +-------------------------------------------------------------------------------- + +via: http://www.zdnet.com/article/linus-torvalds-reveals-his-favorite-programming-laptop/ + +作者:[Steven J. Vaughan-Nichols ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.zdnet.com/meet-the-team/us/steven-j-vaughan-nichols/ +[1]: https://plus.google.com/+LinusTorvalds/posts/VZj8vxXdtfe +[2]: http://www.zdnet.com/article/the-best-chromebook-ever-the-chromebook-pixel-2015/ +[3]: http://www.zdnet.com/product/apple-15-inch-macbook-pro-with-retina-display-mid-2015/ +[4]: https://www.gnome.org/ +[5]: https://plus.google.com/+LinusTorvalds/posts/d7nfnWSXjfD +[6]: http://www.zdnet.com/article/linus-torvalds-finds-gnome-3-4-to-be-a-total-user-experience-design-failure/ +[7]: http://www.zdnet.com/article/how-to-customise-your-linux-desktop-cinnamon/ From f6ff62bf57f73d351b8b3e173c109f2353eab383 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:15:30 +0800 Subject: [PATCH 0165/2437] =?UTF-8?q?20160917-20=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...You Should Be Using It instead of Slack.md | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 sources/tech/20160913 Ryver: Why You Should Be Using It instead of Slack.md diff --git a/sources/tech/20160913 Ryver: Why You Should Be Using It instead of Slack.md b/sources/tech/20160913 Ryver: Why You Should Be Using It instead of Slack.md new file mode 100644 index 0000000000..f4f4c54fcb --- /dev/null +++ b/sources/tech/20160913 Ryver: Why You Should Be Using It instead of Slack.md @@ -0,0 +1,62 @@ +Ryver: Why You Should Be Using It instead of Slack +===== + +It seems like everyone has heard of Slack, a team communication tool that can be used across multiple platforms to stay in the loop. It has revolutionised the way users discuss and plan projects, and it’s a clear upgrade to emails. + +I work in small writing teams, and I’ve never had a problem with communicating with others on my phone or computer while using it. If you want to keep up to date with team of any size, it’s a great way to stay in the loop. + +So, why are we here? Ryver is supposed to be the next big thing, offering an upgraded service in comparison to Slack. It’s completely free, and they’re pushing for a larger share of the market. + +Is it good enough to be a Slack-Killer? What are the differences between two similar sounding services? + +Read on to find out more. + +### Why Ryver? + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/04/Ryver.jpg) + +Why mess with something that works? The developers at Ryver are well aware of Slack, and they’re hoping their improved service will be enough to make you switch over. They promise a completely free team-communication service with no hidden charges along the way. + +Thankfully, they deliver on their main aim with a high quality product. + +Extra content is the name of the game, and they promise to remove some of the limits you’ll find on a free account with Slack. Unlimited data storage is a major plus point, and it’s also more open in a number of ways. If storage limits are an issue for you, you have to check out Ryver. + +It’s a simple system to use, as it was built so that all functions are always one click away. It’s a mantra used to great success by Apple, and there aren’t many growing pains when you first get started. + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/09/ryver-web-interface.png) + +Conversations are split between personal chats and public posts, and it means there’s a clear line between team platforms and personal use. It should help to avoid broadcasting any embarrassing announcements to your colleagues, and I’ve seen a few during my time as a Slack user. + +Integration with a number of existing apps is supported, and there are native applications for most platforms. + +You can add guests when needed at no additional cost, and it’s useful if you deal with external clients regularly. Guests can add more guests, so there’s an element of fluidity that isn’t seen with the more popular option. + +Think of Ryver as a completely different service that will cater to different needs. If you need to deal with numerous clients on the same account, it’s worth trying out. + +The question is how is it free? The quick answer is premium users will be paying your way. Like Spotify and other services, there’s a minority paying for the rest of us. Here’s a direct link to their download page if you’re interested in giving it a go. + +### Should You Switch to Ryver? + +![](https://maketecheasier-2d0f.kxcdn.com/assets/uploads/2016/04/Slack-homepage.jpg) + +Slack is great as long as you stick to smaller teams like I do, but Ryver has a lot to offer. The idea of a completely free team messaging program is noble, and it works perfectly. + +There’s nothing wrong with using both, so make sure to try out the competition if you’re not willing to pay for a premium Slack account. You might find that both are better in different situations, depending on what you need. + +Above all, Ryver is a great free alternative, and it’s more than just a Slack clone. They have a clear idea of what they’re trying to achieve, and they have a decent product that offers something different in a crowded marketplace. + +However, there’s a chance that it will disappear if there’s a sustained lack of funding in the future. It could leave your teams and discussions in disarray. Everything is fine for now, but be careful if you plan to export a larger business over to the new upstart. + +If you’re tired of Slack’s limitations on a free account, you’ll be impressed by what Ryver has to offer. To learn more, check out their website for information about the service. + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/why-use-ryver-instead-of-slack/?utm_medium=feed&utm_source=feedpress.me&utm_campaign=Feed%3A+maketecheasier + +作者:[James Milin-Ashmore][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://www.maketecheasier.com/author/james-ashmore/ From 33ff30721bd778cf04110b1da621de5dfc437aa6 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:22:04 +0800 Subject: [PATCH 0166/2437] =?UTF-8?q?20160917-21=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...tar command practical examples in Linux.md | 308 ++++++++++++++++++ 1 file changed, 308 insertions(+) create mode 100644 sources/tech/20160911 17 tar command practical examples in Linux.md diff --git a/sources/tech/20160911 17 tar command practical examples in Linux.md b/sources/tech/20160911 17 tar command practical examples in Linux.md new file mode 100644 index 0000000000..d4634ef23c --- /dev/null +++ b/sources/tech/20160911 17 tar command practical examples in Linux.md @@ -0,0 +1,308 @@ +17 tar command practical examples in Linux +===== + +Tar (tape archive ) is the most widely used command in Unix like operating system for creating archive of multiple files and folders into a single archive file and that archive file can be further compressed using gzip and bzip2 techniques. In other words we can say that tar command is used to take backup by archiving multiple files and directory into a single tar or archive file and later on files & directories can be extracted from the tar compressed file. + +In this article we will discuss 17 practical examples of tar command in Linux. + +Syntax of tar command: + +``` +# tar +``` + +Some of the commonly used options in tar command are listed below : + +![](http://www.linuxtechi.com/wp-content/uploads/2016/09/tar-command-options.jpg) + +Note : hyphen ( – ) in the tar command options are optional. + +### Example: 1 Create a tar archive file + +Let’s create a tar file of /etc directory and ‘/root/ anaconda-ks.cfg’ file. + +``` +[root@linuxtechi ~]# tar -cvf myarchive.tar /etc /root/anaconda-ks.cfg +``` + +Above command will create a tar file with the name “myarchive” in the current folder. Tar file contains all the files and directories of /etc folder and anaconda-ks.cfg file. + +In the tar command ‘-c‘ option specify to create a tar file, ‘-v’ is used for verbose output and ‘-f’ option is used to specify the archive file name. + +``` +[root@linuxtechi ~]# ls -l myarchive.tar +-rw-r--r--. 1 root root 22947840 Sep 7 00:24 myarchive.tar +[root@linuxtechi ~]# +``` + +### Example:2 List the contents of tar archive file. + +Using ‘–t‘ option in tar command we can view the contents of tar files without extracting it. + +``` +[root@linuxtechi ~]# tar -tvf myarchive.tar +``` + +Listing a specific file or directory from tar file. In the below example i am trying to view whether ‘anaconda-ks.cfg’ file is there in the tar file or not. + +``` +[root@linuxtechi ~]# tar -tvf myarchive.tar root/anaconda-ks.cfg +-rw------- root/root 953 2016-08-24 01:33 root/anaconda-ks.cfg +[root@linuxtechi ~]# +``` + +### Example:3 Append or add files to end of archive or tar file. + +‘-r‘ option in the tar command is used to append or add file to existing tar file. Let’s add /etc/fstab file in ‘data.tar‘ + +``` +[root@linuxtechi ~]# tar -rvf data.tar /etc/fstab +``` + +Note: In the Compressed tar file we can’t append file or directory. + +### Example:4 Extracting files and directories from tar file. + +‘-x‘ option is used to extract the files and directories from the tar file. Let’s extract the content of above created tar file. + +``` +[root@linuxtechi ~]# tar -xvf myarchive.tar +``` + +This command will extract all the files and directories of myarchive tar file in the current working directory. + +### Example:5 Extracting tar file to a particular folder. + +In case you want to extract tar file to a particular folder or directory then use ‘-C‘ option and after that specify the path of a folder. + +``` +[root@linuxtechi ~]# tar -xvf myarchive.tar -C /tmp/ +``` + +### Example:6 Extracting particular file or directory from tar file. + +Let’s assume you want to extract only anaconda-ks.cfg file from the tar file under /tmp folder. + +Syntax : + +``` +# tar –xvf {tar-file } {file-to-be-extracted } -C {path-where-to-extract} + +[root@linuxtechi tmp]# tar -xvf /root/myarchive.tar root/anaconda-ks.cfg -C /tmp/ +root/anaconda-ks.cfg +[root@linuxtechi tmp]# ls -l /tmp/root/anaconda-ks.cfg +-rw-------. 1 root root 953 Aug 24 01:33 /tmp/root/anaconda-ks.cfg +[root@linuxtechi tmp]# +``` + +### Example:7 Creating and compressing tar file (tar.gz or .tgz ) + +Let’s assume that we want to create a tar file of /etc and /opt folder and also want to compress it using gzip tool. This can be achieved using ‘-z‘ option in tar command. Extensions of such tar files will be either tar.gz or .tgz + +``` +[root@linuxtechi ~]# tar -zcpvf myarchive.tar.gz /etc/ /opt/ +``` + +Or + +``` +[root@linuxtechi ~]# tar -zcpvf myarchive.tgz /etc/ /opt/ +``` + +### Example:8 Creating and compressing tar file ( tar.bz2 or .tbz2 ) + +Let’s assume that we want to create compressed (bzip2) tar file of /etc and /opt folder. This can be achieved by using the option ( -j) in the tar command.Extensions of such tar files will be either tar.bz2 or .tbz + +``` +[root@linuxtechi ~]# tar -jcpvf myarchive.tar.bz2 /etc/ /opt/ +``` + +Or + +``` +[root@linuxtechi ~]# tar -jcpvf myarchive.tbz2 /etc/ /opt/ +``` + +### Example:9 Excluding particular files or type while creating tar file. + +Using “–exclude” option in tar command we can exclude the particular file or file type while creating tar file. Let’s assume we want to exclude the file type of html while creating the compressed tar file. + +``` +[root@linuxtechi ~]# tar -zcpvf myarchive.tgz /etc/ /opt/ --exclude=*.html +``` + +### Example:10 Listing the contents of tar.gz or .tgz file + +Contents of tar file with the extensions tar.gz or .tgz is viewed by using the option ‘-t’. Example is shown below : + +``` +[root@linuxtechi ~]# tar -tvf myarchive.tgz | more +............................................. +drwxr-xr-x root/root 0 2016-09-07 08:41 etc/ +-rw-r--r-- root/root 541 2016-08-24 01:23 etc/fstab +-rw------- root/root 0 2016-08-24 01:23 etc/crypttab +lrwxrwxrwx root/root 0 2016-08-24 01:23 etc/mtab -> /proc/self/mounts +-rw-r--r-- root/root 149 2016-09-07 08:41 etc/resolv.conf +drwxr-xr-x root/root 0 2016-09-06 03:55 etc/pki/ +drwxr-xr-x root/root 0 2016-09-06 03:15 etc/pki/rpm-gpg/ +-rw-r--r-- root/root 1690 2015-12-09 04:59 etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 +-rw-r--r-- root/root 1004 2015-12-09 04:59 etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Debug-7 +-rw-r--r-- root/root 1690 2015-12-09 04:59 etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Testing-7 +-rw-r--r-- root/root 3140 2015-09-15 06:53 etc/pki/rpm-gpg/RPM-GPG-KEY-foreman +.......................................................... +``` + +### Example:11 Listing the contents of tar.bz2 or .tbz2 file. + +Contents of tar file with the extensions tar.bz2 or .tbz2 is viewed by using the option ‘-t’. Example is shown below : + +``` +[root@linuxtechi ~]# tar -tvf myarchive.tbz2 | more +........................................................ +rwxr-xr-x root/root 0 2016-08-24 01:25 etc/pki/java/ +lrwxrwxrwx root/root 0 2016-08-24 01:25 etc/pki/java/cacerts -> /etc/pki/ca-trust/extracted/java/cacerts +drwxr-xr-x root/root 0 2016-09-06 02:54 etc/pki/nssdb/ +-rw-r--r-- root/root 65536 2010-01-12 15:09 etc/pki/nssdb/cert8.db +-rw-r--r-- root/root 9216 2016-09-06 02:54 etc/pki/nssdb/cert9.db +-rw-r--r-- root/root 16384 2010-01-12 16:21 etc/pki/nssdb/key3.db +-rw-r--r-- root/root 11264 2016-09-06 02:54 etc/pki/nssdb/key4.db +-rw-r--r-- root/root 451 2015-10-21 09:42 etc/pki/nssdb/pkcs11.txt +-rw-r--r-- root/root 16384 2010-01-12 15:45 etc/pki/nssdb/secmod.db +drwxr-xr-x root/root 0 2016-08-24 01:26 etc/pki/CA/ +drwxr-xr-x root/root 0 2015-06-29 08:48 etc/pki/CA/certs/ +drwxr-xr-x root/root 0 2015-06-29 08:48 etc/pki/CA/crl/ +drwxr-xr-x root/root 0 2015-06-29 08:48 etc/pki/CA/newcerts/ +drwx------ root/root 0 2015-06-29 08:48 etc/pki/CA/private/ +drwx------ root/root 0 2015-11-20 06:34 etc/pki/rsyslog/ +drwxr-xr-x root/root 0 2016-09-06 03:44 etc/pki/pulp/ +.............................................................. +``` + +### Example:12 Extracting or unzip tar.gz or .tgz files. + +tar files with extension tar.gz or .tgz is extracted or unzipped with option ‘-x’ and ‘-z’. Example is shown below : + +``` +[root@linuxtechi ~]# tar -zxpvf myarchive.tgz -C /tmp/ +``` + +Above command will extract tar file under /tmp folder. + +Note : Now a days tar command will take care compression file types automatically while extracting, it means it is optional for us to specify compression type in tar command. Example is shown below : + +``` +[root@linuxtechi ~]# tar -xpvf myarchive.tgz -C /tmp/ +``` + +### Example:13 Extracting or unzip tar.bz2 or .tbz2 files + +tar files with the extension tar.bz2 or .tbz2 is extract with option ‘-j’ and ‘-x’. Example is shown below: + +``` +[root@linuxtechi ~]# tar -jxpvf myarchive.tbz2 -C /tmp/ +``` + +Or + +``` +[root@linuxtechi ~]# tar xpvf myarchive.tbz2 -C /tmp/ +``` + +### Example:14 Scheduling backup with tar command + +There are some real time scenarios where we have to create the tar files of particular files and directories for backup purpose on daily basis. Let’s suppose we have to take backup of whole /opt folder on daily basis, this can be achieved by creating a cron job of tar command. Example is shown below : + +``` +[root@linuxtechi ~]# tar -zcvf optbackup-$(date +%Y-%m-%d).tgz /opt/ +``` + +Create a cron job for above command. + +### Example:15 Creating compressed archive or tar file with -T and -X option. + +There are some real time scenarios where we want tar command to take input from a file and that file will consists of path of files & directory that are to be archived and compressed, there might some files that we would like to exclude in the archive which are mentioned in input file. + +In the tar command input file is specified after ‘-T’ option and file which consists of exclude list is specified after ‘-X’ option. + +Let’s suppose we want to archive and compress the directories like /etc , /opt and /home and want to exclude the file ‘/etc/sysconfig/kdump’ and ‘/etc/sysconfig/foreman‘, Create a text file ‘/root/tar-include’ and ‘/root/tar-exclude’ and put the following contents in respective file. + +``` +[root@linuxtechi ~]# cat /root/tar-include +/etc +/opt +/home +[root@linuxtechi ~]# +[root@linuxtechi ~]# cat /root/tar-exclude +/etc/sysconfig/kdump +/etc/sysconfig/foreman +[root@linuxtechi ~]# +``` + +Now run the below command to create and compress archive file. + +``` +[root@linuxtechi ~]# tar zcpvf mybackup-$(date +%Y-%m-%d).tgz -T /root/tar-include -X /root/tar-exclude +``` + +### Example:16 View the size of .tar , .tgz and .tbz2 file. + +Use the below commands to view the size of tar and compressed tar files. + +``` +[root@linuxtechi ~]# tar -czf - data.tar | wc -c +427 +[root@linuxtechi ~]# tar -czf - mybackup-2016-09-09.tgz | wc -c +37956009 +[root@linuxtechi ~]# tar -czf - myarchive.tbz2 | wc -c +30835317 +[root@linuxtechi ~]# +``` + +### Example:17 Splitting big tar file into smaller files. + +In Unix like operating big file is divided or split into smaller files using split command. Big tar file can also be divided into the smaller parts using split command. + +Let’s assume we want to split ‘mybackup-2016-09-09.tgz‘ file into smaller parts of each 6 MB. + +``` +Syntax : split -b . “prefix-name” +``` + +``` +[root@linuxtechi ~]# split -b 6M mybackup-2016-09-09.tgz mybackup-parts +``` + +Above command will split the mybackup compressed tar file into the smaller files each of size 6 MB in current working directory and split file names will starts from mybackup-partsaa … mybackup-partsag. In case if you want to append numbers in place of alphabets then use ‘-d’ option in above split command. + +``` +[root@linuxtechi ~]# ls -l mybackup-parts* +-rw-r--r--. 1 root root 6291456 Sep 10 03:05 mybackup-partsaa +-rw-r--r--. 1 root root 6291456 Sep 10 03:05 mybackup-partsab +-rw-r--r--. 1 root root 6291456 Sep 10 03:05 mybackup-partsac +-rw-r--r--. 1 root root 6291456 Sep 10 03:05 mybackup-partsad +-rw-r--r--. 1 root root 6291456 Sep 10 03:05 mybackup-partsae +-rw-r--r--. 1 root root 6291456 Sep 10 03:05 mybackup-partsaf +-rw-r--r--. 1 root root 637219 Sep 10 03:05 mybackup-partsag +[root@linuxtechi ~]# +``` + +Now we can move these files into another server over the network and then we can merge all the files into a single tar compressed file using below mentioned command. + +``` +[root@linuxtechi ~]# cat mybackup-partsa* > mybackup-2016-09-09.tgz +[root@linuxtechi ~]# +``` + +That’s all, hope you like different examples of tar command. Please share your feedback and comments. + +-------------------------------------------------------------------------------- + +via: http://www.linuxtechi.com/17-tar-command-examples-in-linux/ + +作者:[Pradeep Kumar ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.linuxtechi.com/author/pradeep/ From 4b9c568f8ebcc4dcfae70b88d68d38b22b10ad09 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:24:28 +0800 Subject: [PATCH 0167/2437] =?UTF-8?q?20160917-22=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...adership advice for starting a new role.md | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 sources/talk/20160912 Adobe's new CIO shares leadership advice for starting a new role.md diff --git a/sources/talk/20160912 Adobe's new CIO shares leadership advice for starting a new role.md b/sources/talk/20160912 Adobe's new CIO shares leadership advice for starting a new role.md new file mode 100644 index 0000000000..a4a962e8d2 --- /dev/null +++ b/sources/talk/20160912 Adobe's new CIO shares leadership advice for starting a new role.md @@ -0,0 +1,50 @@ +Adobe's new CIO shares leadership advice for starting a new role +==== + +![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/CIO_Leadership_3.png?itok=QWUGMw-V) + +I’m currently a few months into a new CIO role at a highly-admired, cloud-based technology company. One of my first tasks was to get to know the organization’s people, culture, and priorities. + +As part of that goal, I am visiting all the major IT sites. While In India, less than two months into the job, I was asked directly: “What are you going to do? What is your plan?” My response, which will not surprise seasoned CIOs, was that I was still in discovery mode, and I was there to listen and learn. + +I’ve never gone into an organization with a set blueprint for what I’ll do. I know some CIOs have a playbook for how they will operate. They’ll come in and blow the whole organization up and put their set plan in motion. + +Yes, there may be situations where things are massively broken and not working, so that course of action makes sense. Once I’m inside a company, however, my strategy is to go through a discovery process. I don’t want to have any preconceived notions about the way things should be or what’s working versus what’s not. + +Here are my guiding principles as a newly-appointed leader: + +### Get to know your people + +This means building relationships, and it includes your IT staff as well as your business users and your top salespeople. What are the top things on their lists? What do they want you to focus on? What’s working well? What’s not? How is the customer experience? Knowing how you can help everyone be more successful will help you shape the way you deliver services to them. + +If your department is spread across several floors, as mine is, consider meet-and-greet lunches or mini-tech fairs so people can introduce themselves, discuss what they’re working on, and share stories about their family, if they feel comfortable doing that. If you have an open-door office policy, make sure they know that as well. If your staff spreads across countries or continents, get out there and visit as soon as you reasonably can. + +### Get to know your products and company culture + +One of the things that surprised me coming into to Adobe was how broad our product portfolio is. We have a platform of solutions and services across three clouds – Adobe Creative Cloud, Document Cloud and Marketing Cloud – and a vast portfolio of products within each. You’ll never know how much opportunity your new company presents until you get to know your products and learn how to support all of them. At Adobe we use many of our digital media and digital marketing solutions as Customer Zero, so we have first-hand experiences to share with our customers + +### Get to know customers + +Very early on, I started getting requests to meet with customers. Meeting with customers is a great way to jump-start your thinking into the future of the IT organization, which includes the different types of technologies, customers, and consumers we could have going forward. + +### Plan for the future + +As a new leader, I have a fresh perspective and can think about the future of the organization without getting distracted by challenges or obstacles. + +What CIOs need to do is jump-start IT into its next generation. When I meet my staff, I’m asking them what we want to be three to five years out so we can start positioning ourselves for that future. That means discussing the initiatives and priorities. + +After that, it makes sense to bring the leadership team together so you can work to co-create the next generation of the organization – its mission, vision, modes of alignment, and operating norms. If you start changing IT from the inside out, it will percolate into business and everything else you do. + +Through this whole process, I’ve been very open with people that this is not going to be a top-down directive. I have ideas on priorities and what we need to focus on, but we have to be in lockstep, working as a team and figuring out what we want to do jointly. + +-------------------------------------------------------------------------------- + +via: https://enterprisersproject.com/article/2016/9/adobes-new-cio-shares-leadership-advice-starting-new-role + +作者:[Cynthia Stoddard][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://enterprisersproject.com/user/cynthia-stoddard From ef05f3622569e25b1abd3260c29d3ea4890b264c Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:31:49 +0800 Subject: [PATCH 0168/2437] =?UTF-8?q?20160917-23=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...en Source Artificial Intelligence Tools.md | 125 ++++++++++++++++++ 1 file changed, 125 insertions(+) create mode 100644 sources/tech/20160912 15 Top Open Source Artificial Intelligence Tools.md diff --git a/sources/tech/20160912 15 Top Open Source Artificial Intelligence Tools.md b/sources/tech/20160912 15 Top Open Source Artificial Intelligence Tools.md new file mode 100644 index 0000000000..56f78f84b9 --- /dev/null +++ b/sources/tech/20160912 15 Top Open Source Artificial Intelligence Tools.md @@ -0,0 +1,125 @@ +15 Top Open Source Artificial Intelligence Tools +==== + +Artificial Intelligence (AI) is one of the hottest areas of technology research. Companies like IBM, Google, Microsoft, Facebook and Amazon are investing heavily in their own R&D, as well as buying up startups that have made progress in areas like machine learning, neural networks, natural language and image processing. Given the level of interest, it should come as no surprise that a recent [artificial intelligence report][1] from experts at Stanford University concluded that "increasingly useful applications of AI, with potentially profound positive impacts on our society and economy are likely to emerge between now and 2030." + +In a recent [article][2], we provided an overview of 45 AI projects that seem particularly promising or interesting. In this slideshow, we're focusing in on open source artificial intelligence tools, with a closer look at fifteen of the best-known open source AI projects. + +![](http://www.datamation.com/imagesvr_ce/5668/00AI.jpg) + +Open Source Artificial Intelligence + +These open source AI applications are on the cutting edge of artificial intelligence research. + +![](http://www.datamation.com/imagesvr_ce/8922/01Caffe.JPG) + +### 1. Caffe + +The brainchild of a UC Berkeley PhD candidate, Caffe is a deep learning framework based on expressive architecture and extensible code. It's claim to fame is its speed, which makes it popular with both researchers and enterprise users. According to its website, it can process more than 60 million images in a single day using just one NVIDIA K40 GPU. It is managed by the Berkeley Vision and Learning Center (BVLC), and companies like NVIDIA and Amazon have made grants to support its development. + +![](http://www.datamation.com/imagesvr_ce/1232/02CNTK.JPG) + +### 2. CNTK + +Short for Computational Network Toolkit, CNTK is one of Microsoft's open source artificial intelligence tools. It boasts outstanding performance whether it is running on a system with only CPUs, a single GPU, multiple GPUs or multiple machines with multiple GPUs. Microsoft has primarily utilized it for research into speech recognition, but it is also useful for applications like machine translation, image recognition, image captioning, text processing, language understanding and language modeling. + +![](http://www.datamation.com/imagesvr_ce/2901/03Deeplearning4j.JPG) + +### 3. Deeplearning4j + +Deeplearning4j is an open source deep learning library for the Java Virtual Machine (JVM). It runs in distributed environments and integrates with both Hadoop and Apache Spark. It makes it possible to configure deep neural networks, and it's compatible with Java, Scala and other JVM languages. + +The project is managed by a commercial company called Skymind, which offers paid support, training and an enterprise distribution of Deeplearning4j. + +![](http://www.datamation.com/imagesvr_ce/7269/04DMLT.JPG) + +### 4. Distributed Machine Learning Toolkit + +Like CNTK, the Distributed Machine Learning Toolkit (DMTK) is one of Microsoft's open source artificial intelligence tools. Designed for use in big data applications, it aims to make it faster to train AI systems. It consists of three key components: the DMTK framework, the LightLDA topic model algorithm, and the Distributed (Multisense) Word Embedding algorithm. As proof of DMTK's speed, Microsoft says that on an eight-cluster machine, it can "train a topic model with 1 million topics and a 10-million-word vocabulary (for a total of 10 trillion parameters), on a document collection with over 100-billion tokens," a feat that is unparalleled by other tools. + +![](http://www.datamation.com/imagesvr_ce/2890/05H2O.JPG) + +### 5. H20 + +Focused more on enterprise uses for AI than on research, H2O has large companies like Capital One, Cisco, Nielsen Catalina, PayPal and Transamerica among its users. It claims to make is possible for anyone to use the power of machine learning and predictive analytics to solve business problems. It can be used for predictive modeling, risk and fraud analysis, insurance analytics, advertising technology, healthcare and customer intelligence. + +It comes in two open source versions: standard H2O and Sparkling Water, which is integrated with Apache Spark. Paid enterprise support is also available. + +![](http://www.datamation.com/imagesvr_ce/1127/06Mahout.JPG) + +### 6. Mahout + +An Apache Foundation project, Mahout is an open source machine learning framework. According to its website, it offers three major features: a programming environment for building scalable algorithms, premade algorithms for tools like Spark and H2O, and a vector-math experimentation environment called Samsara. Companies using Mahout include Adobe, Accenture, Foursquare, Intel, LinkedIn, Twitter, Yahoo and many others. Professional support is available through third parties listed on the website. + +![](http://www.datamation.com/imagesvr_ce/4038/07MLlib.JPG) + +### 7. MLlib + +Known for its speed, Apache Spark has become one of the most popular tools for big data processing. MLlib is Spark's scalable machine learning library. It integrates with Hadoop and interoperates with both NumPy and R. It includes a host of machine learning algorithms for classification, regression, decision trees, recommendation, clustering, topic modeling, feature transformations, model evaluation, ML pipeline construction, ML persistence, survival analysis, frequent itemset and sequential pattern mining, distributed linear algebra and statistics. + +![](http://www.datamation.com/imagesvr_ce/839/08NuPIC.JPG) + +### 8. NuPIC + +Managed by a company called Numenta, NuPIC is an open source artificial intelligence project based on a theory called Hierarchical Temporal Memory, or HTM. Essentially, HTM is an attempt to create a computer system modeled after the human neocortex. The goal is to create machines that "approach or exceed human level performance for many cognitive tasks." + +In addition to the open source license, Numenta also offers NuPic under a commercial license, and it also offers licenses on the patents that underlie the technology. + +![](http://www.datamation.com/imagesvr_ce/99/09OpenNN.JPG) + +### 9. OpenNN + +Designed for researchers and developers with advanced understanding of artificial intelligence, OpenNN is a C++ programming library for implementing neural networks. Its key features include deep architectures and fast performance. Extensive documentation is available on the website, including an introductory tutorial that explains the basics of neural networks. Paid support for OpenNNis available through Artelnics, a Spain-based firm that specializes in predictive analytics. + +![](http://www.datamation.com/imagesvr_ce/4168/10OpenCyc.JPG) + +### 10. OpenCyc + +Developed by a company called Cycorp, OpenCyc provides access to the Cyc knowledge base and commonsense reasoning engine. It includes more than 239,000 terms, about 2,093,000 triples, and about 69,000 owl:sameAs links to external semantic data namespaces. It is useful for rich domain modeling, semantic data integration, text understanding, domain-specific expert systems and game AIs. The company also offers two other versions of Cyc: one for researchers that is free but not open source and one for enterprise use that requires a fee. + +![](http://www.datamation.com/imagesvr_ce/9761/11Oryx2.JPG) + +### 11. Oryx 2 + +Built on top of Apache Spark and Kafka, Oryx 2 is a specialized application development framework for large-scale machine learning. It utilizes a unique lambda architecture with three tiers. Developers can use Oryx 2 to create new applications, and it also includes some pre-built applications for common big data tasks like collaborative filtering, classification, regression and clustering. The big data tool vendor Cloudera created the original Oryx 1 project and has been heavily involved in continuing development. + +![](http://www.datamation.com/imagesvr_ce/7423/12.%20PredictionIO.JPG) + +### 12. PredictionIO + +In February this year, Salesforce bought PredictionIO, and then in July, it contributed the platform and its trademark to the Apache Foundation, which accepted it as an incubator project. So while Salesforce is using PredictionIO technology to advance its own machine learning capabilities, work will also continue on the open source version. It helps users create predictive engines with machine learning capabilities that can be used to deploy Web services that respond to dynamic queries in real time. + +![](http://www.datamation.com/imagesvr_ce/6886/13SystemML.JPG) + +### 13. SystemML + +First developed by IBM, SystemML is now an Apache big data project. It offers a highly-scalable platform that can implement high-level math and algorithms written in R or a Python-like syntax. Enterprises are already using it to track customer service on auto repairs, to direct airport traffic and to link social media data with banking customers. It can run on top of Spark or Hadoop. + +![](http://www.datamation.com/imagesvr_ce/5742/14TensorFlow.JPG) + +### 14. TensorFlow + +TensorFlow is one of Google's open source artificial intelligence tools. It offers a library for numerical computation using data flow graphs. It can run on a wide variety of different systems with single- or multi-CPUs and GPUs and even runs on mobile devices. It boasts deep flexibility, true portability, automatic differential capabilities and support for Python and C++. The website includes a very extensive list of tutorials and how-tos for developers or researchers interested in using or extending its capabilities. + +![](http://www.datamation.com/imagesvr_ce/9018/15Torch.JPG) + +### 15. Torch + +Torch describes itself as "a scientific computing framework with wide support for machine learning algorithms that puts GPUs first." The emphasis here is on flexibility and speed. In addition, it's fairly easy to use with packages for machine learning, computer vision, signal processing, parallel processing, image, video, audio and networking. It relies on a scripting language called LuaJIT that is based on Lua. + + + + +-------------------------------------------------------------------------------- + +via: http://www.datamation.com/open-source/slideshows/15-top-open-source-artificial-intelligence-tools.html + +作者:[Cynthia Harvey][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.datamation.com/author/Cynthia-Harvey-6460.html +[1]: https://ai100.stanford.edu/sites/default/files/ai_100_report_0906fnlc_single.pdf +[2]: http://www.datamation.com/applications/artificial-intelligence-software-45-ai-projects-to-watch-1.html From 949d1414273a67d50becd14789851dbf80c4c83e Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:34:47 +0800 Subject: [PATCH 0169/2437] =?UTF-8?q?20160917-24=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...illiant Command-Line TODO App For Linux.md | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 sources/tech/20160913 Taskwarrior: A Brilliant Command-Line TODO App For Linux.md diff --git a/sources/tech/20160913 Taskwarrior: A Brilliant Command-Line TODO App For Linux.md b/sources/tech/20160913 Taskwarrior: A Brilliant Command-Line TODO App For Linux.md new file mode 100644 index 0000000000..6b44101e53 --- /dev/null +++ b/sources/tech/20160913 Taskwarrior: A Brilliant Command-Line TODO App For Linux.md @@ -0,0 +1,52 @@ +Taskwarrior: A Brilliant Command-Line TODO App For Linux +==== + +Taskwarrior is a simple, straight-forward command-line based TODO app for Ubuntu/Linux. This open-source app has to be one of the easiest of all [CLI based apps][4] I've ever used. Taskwarrior helps you better organize yourself, and without installing bulky new apps which sometimes defeats the whole purpose of TODO apps. + +![](https://2.bp.blogspot.com/-pQnRlOUNIxk/V9cuc3ytsBI/AAAAAAAAKHs/yYxyiAk4PwMIE0HTxlrm6arWOAPcBRRywCLcB/s1600/taskwarrior-todo-app.png) + +### Taskwarrior: A Simple CLI Based TODO App That Gets The Job Done! + +Taskwarrior is an open-source and cross-platform, command-line based TODO app, which lets you manage your to-do lists right from the Terminal. The app lets you add tasks, shows you the list, and removes tasks from that list with much ease. And what's more, it's available within your default repositories, no need to fiddle with PPAs. In Ubuntu 16.04 LTS and similar, do the following in Terminal to install Taskwarrior. + +``` +sudo apt-get install task +``` + +A simple use case can be as follows: + +``` +$ task add Read a book +Created task 1. +$ task add priority:H Pay the bills +Created task 2. +``` + +This is the same example I used in the screenshot above. Yes, you can set priority levels (H, L or M) as shown. And then you can use 'task' or 'task next' commands to see your newly-created todo list. For example: + +``` +$ task next + +ID Age P Description Urg +-- --- - -------------------------------- ---- + 2 10s H Pay the bills 6 + 1 20s Read a book 0 +``` + +And once its completed, you can use 'task 1 done' or 'task 2 done' commands to clear the lists. A more comprehensive list of commands, use-cases [can be found here][1]. Also, Taskwarrior is cross-platform, which means you'll find a version that [fits your needs][2] no matter what. There's even an [Android version][3] if you want one. Enjoy! + +-------------------------------------------------------------------------------- + +via: http://www.techdrivein.com/2016/09/taskwarrior-command-line-todo-app-linux.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+techdrivein+%28Tech+Drive-in%29 + +作者:[Manuel Jose ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.techdrivein.com/2016/09/taskwarrior-command-line-todo-app-linux.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+techdrivein+%28Tech+Drive-in%29 +[1]: https://taskwarrior.org/docs/ +[2]: https://taskwarrior.org/download/ +[3]: https://taskwarrior.org/news/news.20160225.html +[4]: http://www.techdrivein.com/search/label/Terminal From 2bf34b32bd49cc4dc48800b99b030577fd337c1c Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:42:45 +0800 Subject: [PATCH 0170/2437] =?UTF-8?q?20160917-25=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...and dirty with Windows Nano Server 2016.md | 85 +++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 sources/tech/20160914 Down and dirty with Windows Nano Server 2016.md diff --git a/sources/tech/20160914 Down and dirty with Windows Nano Server 2016.md b/sources/tech/20160914 Down and dirty with Windows Nano Server 2016.md new file mode 100644 index 0000000000..613d3de86e --- /dev/null +++ b/sources/tech/20160914 Down and dirty with Windows Nano Server 2016.md @@ -0,0 +1,85 @@ +Down and dirty with Windows Nano Server 2016 +==== + +![](http://images.techhive.com/images/article/2016/04/pokes-fun-at-1164459_1280-100654917-primary.idge.jpg) + +>Nano Server is a very fast, powerful tool for remotely administering Windows servers, but you need to know what you're doing + +There's been a good deal of talk around the [upcoming Nano version of Windows Server 2016][1], the remote-administered, command-line version designed with private clouds and datacenters in mind. But there's also a big difference between talking about it and getting your hands into it. Let's get into the guts. + +Nano has no local login, is 64-bit all the way (applications, tools, and agents), and is fast to set up, update, and restart (for the rare times it needs to restart). It's perfect for compute hosts in or out of a cluster, a storage host, a DNS server, an IIS web server, and any server-hosting applications running in a container or virtual-machine guest operating system. + +A Nano Server isn't all that fun to play with: You have to know what you want to accomplish. Otherwise, you'll be looking at a remote PowerShell connection and wondering what you're supposed to do next. But if you know what you want, it's very fast and powerful. + +Microsoft has provided a [quick-start guide][2] to setting up Nano Server. Here, I take the boots-on-the-ground approach to show you what it's like in the real world. + +First, you have to create a .vhd virtual hard drive file. As you can see in Figure 1, I had a few issues with files not being in the right place. PowerShell errors often indicate a mistyped line, but in this case, I had to keep double-checking where I put the files so that it could use the ISO information (which has to be copied and pasted to the server you want to create the .vhd file on). Once you have everything in place, you should see it go through the process of creating the .vhd file. + +![](http://images.techhive.com/images/article/2016/09/nano-server-1-100682371-large.idge.jpg) +>Figure 1: One of the many file path errors I got when trying to run the New-NanoServerImage script. Once I worked out the file-location issues, it went through and created the .vhd file (as shown here). + +Next, when you create the VM in Hyper-V using the VM wizard, you need to point to an existing virtual hard disk and point to the new .vhd file you created (Figure 2). + +![](http://images.techhive.com/images/article/2016/09/nano-server-2-100682368-large.idge.jpg) +>Figure 2: Connecting to a virtual hard disk (the one you created at the start). + +When you start up the Nano server, you may get a memory error depending on how much memory you allocated and how much memory the Hyper-V server has left if you have other VMs running. I had to shut off a few VMs and increase the RAM until it finally started up. That was unexpected -- [Microsoft's Nano system][3] requirements say you can run it with 512MB, although it recommends you give it at least 800MB. (I ended up allocating 8GB after 1GB didn't work; I was impatient, so I didn't try increments in between.) + +I finally came to the login screen, then signed in to get the Nano Server Recovery Console (Figure 3), which is essentially Nano server's terminal screen. + +![](http://images.techhive.com/images/article/2016/09/nano-server-3-100682369-large.idge.jpg) +>Figure 3: The Nano Server Recovery Console. + +Once I was in, I thought I was golden. But in trying to figure out a few details (how to join a domain, how to inject drivers I might not have, how to add roles), I realized that some configuration pieces would have been easier to add when I ran the New-NanoServerImage cmdlet by popping in a few more parameters. + +However, once you have the server up and running, there are ways to configure it live. It all starts with a Remote PowerShell connection, as Figure 4 shows. + +![](http://images.techhive.com/images/article/2016/09/nano-server-4-100682370-large.idge.jpg) +>Figure 4: Getting information from the Nano Server Recovery Console that you can use to perform a PowerShell Remote connection. + +Microsoft provides direction on how to make the connection happen, but after trying four different sites, I found MSDN has the clearest (working) direction on the subject. Figure 5 shows the result. + +![](http://images.techhive.com/images/article/2016/09/nano-server-5-100682372-large.idge.jpg) +>Figure 5: Making the remote PowerShell connection to your Nano Server. + +Note: Once you've done the remote connection the long way, you can connect more quickly using a single line: + +``` +Enter-PSSession –ComputerName "192.168.0.100"-Credential ~\Administrator. +``` + +If you knew ahead of time that this server was going to be a DNS server or be part of a compute cluster and so on, you would have added those roles or feature packages when you were creating the .vhd image in the first place. If you're looking to do so after the fact, you'll need to make the remote PowerShell connection, then install the NanoServerPackage and import it. Then you can see which packages you want to deploy using Find-NanoServerPackage (shown in Figure 6). + +![](http://images.techhive.com/images/article/2016/09/nano-server-6-100682373-large.idge.jpg) +>Figure 6: Once you have installed and imported the NanoServerPackage, you can find the one you need to get your Nano Server up and running with the roles and features you require. + +I tested this out by running the DNS package with the following command: `Install-NanoServerPackage –Name Microsoft-NanoServer-DNS-Package`. Once it was installed, I had to enable it with the following command: `Enable-WindowsOptionalFeature –Online –FeatureName DNS-Server-Full-Role`. + +Obviously I didn't know these commands ahead of time. I have never run them before in my life, nor had I ever enabled a DNS role this way, but with a little research I had a DNS (Nano) Server up and running. + +The next part of the process involves using PowerShell to configure the DNS server. That's a completely different topic and one best researched online. But it doesn't appear to be mind-blowingly difficult once you've learned the cmdlets to use: Add a zone? Use the Add-DNSServerPrimaryZone cmdlet. Add a record in that zone? Use the Add-DNSServerResourceRecordA. And so on. + +After doing all this command-line work, you'll likely want proof that any of this is working. You should be able to do a quick review of PowerShell commands and not the many DNS ones that now present themselves (using Get-Command). + +But if you need a GUI-based confirmation, you can open Server Manager on a GUI-based server and add the IP address of the Nano Server. Then right-click that server and choose Manage As to provide your credentials (~\Administrator and password). Once you have connected, right-click the server in Server Manager and choose Add Roles and Features; it should show that you have DNS installed as a role, as Figure 7 shows. + +![](http://images.techhive.com/images/article/2016/09/nano-server-7-100682374-large.idge.jpg) +>Figure 7: Proving through the GUI that DNS was installed. + +Don't bother trying to remote-desktop into the server. There is only so much you can do through the Server Manager tool, and that isn't one of them. And just because you can confirm the DNS role doesn't mean you have the ability to add new roles and features through the GUI. It's all locked down. Nano Server is how you'll make any needed adjustments. + +-------------------------------------------------------------------------------- + +via: http://www.infoworld.com/article/3119770/windows-server/down-and-dirty-with-windows-nano-server-2016.html?utm_source=webopsweekly&utm_medium=email + +作者:[J. Peter Bruzzese ][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://www.infoworld.com/author/J.-Peter-Bruzzese/ +[1]: http://www.infoworld.com/article/3049191/windows-server/nano-server-a-slimmer-slicker-windows-server-core.html +[2]: https://technet.microsoft.com/en-us/windows-server-docs/compute/nano-server/getting-started-with-nano-server +[3]: https://technet.microsoft.com/en-us/windows-server-docs/get-started/system-requirements--and-installation + From 5254daae23ca68ac6ad982192b163f3f330cbe21 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:47:31 +0800 Subject: [PATCH 0171/2437] =?UTF-8?q?20160917-26=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ... Principles of Monitoring Microservices.md | 95 +++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 sources/tech/20160913 The Five Principles of Monitoring Microservices.md diff --git a/sources/tech/20160913 The Five Principles of Monitoring Microservices.md b/sources/tech/20160913 The Five Principles of Monitoring Microservices.md new file mode 100644 index 0000000000..9d283f8bdf --- /dev/null +++ b/sources/tech/20160913 The Five Principles of Monitoring Microservices.md @@ -0,0 +1,95 @@ +The Five Principles of Monitoring Microservices +==== + +![](http://thenewstack.io/wp-content/uploads/2016/09/toppicsysdig.jpg) + +The need for microservices can be summed up in just one word: speed. The need to deliver more functionality and reliability faster has revolutionized the way developers create software. Not surprisingly, this change has caused ripple effects within software management, including monitoring systems. In this post, we’ll focus on the radical changes required to monitor your microservices in production efficiently. We’ll lay out five guiding principles for adapting your monitoring approach for this new software architecture. + +Monitoring is a critical piece of the control systems of microservices, as the more complex your software gets, the harder it is to understand its performance and troubleshoot problems. Given the dramatic changes to software delivery, however, monitoring needs an overhaul to perform well in a microservice environment. The rest of this article presents the five principles of monitoring microservices, as follows: + +1. Monitor containers and what’s inside them. +2. Alert on service performance, not container performance. +3. Monitor services that are elastic and multi-location. +4. Monitor APIs. +5. Map your monitoring to your organizational structure. + +Leveraging these five principles will allow you to establish more effective monitoring as you make your way towards microservices. These principles will allow you to address both the technological changes associated with microservices, in addition to the organizational changes related to them. + +### The Principles of Microservice Monitoring + +#### 1. Monitor Containers and What’s Running Inside Them + +Containers gained prominence as the building blocks of microservices. The speed, portability, and isolation of containers made it easy for developers to embrace a microservice model. There’s been a lot written on the benefits of containers so we won’t recount it all here. + +Containers are black boxes to most systems that live around them. That’s incredibly useful for development, enabling a high level of portability from development through production, from developer laptop to cloud. But when it comes to operating, monitoring and troubleshooting a service, black boxes make common activities harder, leading us to wonder: what’s running in the container? How is the application/code performing? Is it spitting out important custom metrics? From the DevOps perspective, you need deep visibility inside containers rather than just knowing that some containers exist. + +![](http://thenewstack.io/wp-content/uploads/2016/09/greatfordev.jpg) + +The typical process for instrumentation in a non-containerized environment — an agent that lives in the user space of a host or VM — doesn’t work particularly well for containers. That’s because containers benefit from being small, isolated processes with as few dependencies as possible. + +And, at scale, running thousands of monitoring agents for even a modestly-sized deployment is an expensive use of resources and an orchestration nightmare. Two potential solutions arise for containers: 1) ask your developers to instrument their code directly, or 2) leverage a universal kernel-level instrumentation approach to see all application and container activity on your hosts. We won’t go into depth here, but each method has pros and cons. + +#### 2. Leverage Orchestration Systems to Alert on Service Performance + +Making sense of operational data in a containerized environment is a new challenge. The metrics of a single container have a much lower marginal value than the aggregate information from all the containers that make up a function or a service. + +This particularly applies to application-level information, like which queries have the slowest response times or which URLs are seeing the most errors, but also applies to infrastructure-level monitoring, like which services’ containers are using the most resources beyond their allocated CPU shares. + +Increasingly, software deployment requires an orchestration system to “translate” a logical application blueprint into physical containers. Common orchestration systems include Kubernetes, Mesosphere DC/OS and Docker Swarm. Teams use an orchestration system to (1) define your microservices and (2) understand the current state of each service in deployment. You could argue that the orchestration system is even more important than the containers. The actual containers are ephemeral — they matter only for the short time that they exist — while your services matter for the life of their usefulness. + +DevOps teams should redefine alerts to focus on characteristics that get as close to monitoring the experience of the service as possible. These alerts are the first line of defense in assessing if something is impacting the application. But getting to these alerts is challenging, if not impossible unless your monitoring system is container-native. + +Container-native solutions leverage orchestration metadata to dynamically aggregate container and application data and calculate monitoring metrics on a per-service basis. Depending on your orchestration tool, you might have different layers of a hierarchy that you’d like to drill into. For example, in Kubernetes, you typically have a Namespace, ReplicaSets, Pods and some containers. Aggregating at these various layers is essential for logical troubleshooting, regardless of the physical deployment of the containers that make up the service. + +![](http://thenewstack.io/wp-content/uploads/2016/09/servicemonitoring.jpg) + +#### 3. Be Prepared for Services that are Elastic and Multi-Location + +Elastic services are certainly not a new concept, but the velocity of change is much faster in container-native environments than virtualized environments. Rapidly changing environments can wreak havoc on brittle monitoring systems. + +Frequently monitoring legacy systems required manual tuning of metrics and checks based on individual deployments of software. This tuning can be as specific as defining the individual metrics to be captured, or configuring collection based on what application is operating in a particular container. While that may be acceptable on a small scale (think tens of containers), it would be unbearable in anything larger. Microservice focused monitoring must be able to comfortably grow and shrink in step with elastic services, without human intervention. + +For example, if the DevOps team must manually define what service a container is included in for monitoring purposes, they no doubt drop the ball as Kubernetes or Mesos spins up new containers regularly throughout the day. Similarly, if Ops were required to install a custom stats endpoint when new code is built and pushed into production, challenges may arise as developers pull base images from a Docker registry. + +In production, build monitoring toward a sophisticated deployment that spans multiple data centers or multiple clouds. Leveraging, for example, AWS CloudWatch will only get you so far if your services span your private data center as well as AWS. That leads back to implementing a monitoring system that can span these different locations as well as operate in dynamic, container-native environments. + +#### 4. Monitor APIs + +In microservice environments, APIs are the lingua franca. They are essentially the only elements of a service that are exposed to other teams. In fact, response and consistency of the API may be the “internal SLA” even if there isn’t a formal SLA defined. + +As a result, API monitoring is essential. API monitoring can take many forms but clearly, must go beyond binary up/down checks. For instance, it’s valuable to understand the most frequently used endpoints as a function of time. This allows teams to see if anything noticeable has changed in the usage of services, whether it be due to a design change or a user change. + +You can also consider the slowest endpoints of your service, as these can reveal significant problems, or, at the very least, point to areas that need the most optimization in your system. + +Finally, the ability to trace service calls through your system represents another critical capability. While typically used by developers, this type of profiling will help you understand the overall user experience while breaking information down into infrastructure and application-based views of your environment. + +#### 5. Map Monitoring to Your Organizational Structure + +While most of this post has been focused on the technological shift in microservices and monitoring, like any technology story, this is as much about people as it is about software bits. + +For those of you familiar with Conway’s law, he reminds us that the design of systems is defined by the organizational structure of the teams building them. The allure of creating faster, more agile software has pushed teams to think about restructuring their development organization and the rules that govern it. + +![](http://thenewstack.io/wp-content/uploads/2016/09/mapmonitoring.jpg) + +So if an organization wants to benefit from this new software architecture approach, their teams must, therefore, mirror microservices themselves. That means smaller teams, loosely coupled; that can choose their direction as long as it still meets the needs of the whole. Within each team, there is more control than ever over languages used, how bugs are handled, or even operational responsibilities. + +DevOps teams can enable a monitoring platform that does exactly this: allows each microservice team to isolate their alerts, metrics, and dashboards, while still giving operations a view into the global system. + +### Conclusion + +There’s one, clear trigger event that precipitated the move to microservices: speed. Organizations wanted to deliver more capabilities to their customers in less time. Once this happened, technology stepped in, the architectural move to micro-services and the underlying shift to containers make speed happen. Anything that gets in the way of this progress train is going to get run over on the tracks. + +As a result, the fundamental principles of monitoring need to adapt to the underlying technology and organizational changes that accompany microservices. Operations teams that recognize this shift can adapt to microservices earlier and easier. + +-------------------------------------------------------------------------------- + +via: http://linoxide.com/firewall/pfsense-setup-basic-configuration/ + +作者:[Apurva Dave][a] [Loris Degioanni][b] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: http://thenewstack.io/author/apurvadave/ +[b]: http://thenewstack.io/author/lorisdegioanni/ From e13ed6e31cd21d3a9c563c2622c150835ef72bc2 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 22:55:11 +0800 Subject: [PATCH 0172/2437] =?UTF-8?q?20160917-27=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...tainers with Elasticsearch and cAdvisor.md | 207 ++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 sources/tech/20160913 Monitoring Docker Containers with Elasticsearch and cAdvisor.md diff --git a/sources/tech/20160913 Monitoring Docker Containers with Elasticsearch and cAdvisor.md b/sources/tech/20160913 Monitoring Docker Containers with Elasticsearch and cAdvisor.md new file mode 100644 index 0000000000..fe5d8025a7 --- /dev/null +++ b/sources/tech/20160913 Monitoring Docker Containers with Elasticsearch and cAdvisor.md @@ -0,0 +1,207 @@ +Monitoring Docker Containers with Elasticsearch and cAdvisor +======= + +If you’re running a Swarm Mode cluster or even a single Docker engine, you’ll end up asking this question: + +>How do I keep track of all that’s happening? + +The answer is “not easily.” + +You need a few things to have a complete overview of stuff like: + + +1. Number and status of containers +2. If, where, and when a container has been moved to another node +3. Number of containers on a given node +4. Traffic peaks at a given time +5. Orphan volumes and networks +6. Free disk space, free inodes +7. Number of containers against number of veths attached to the docker0 and docker_gwbridge bridges +8. Up and down Swarm nodes +9. Centralize logs + +The goal of this post is to demonstrate the use of [Elasticsearch][1] + [Kibana][2] + [cAdvisor][3] as tools to analyze and gather metrics and visualize dashboards for Docker containers. + +Later on in this post, you can find a dashboard trying to address a few points from the previous list. There are also points that can’t be addressed by simply using cAdvisor, like the status of Swarm Mode nodes. + +Also, if you have specific needs that aren’t covered by cAdvisor or another tool, I encourage you to write your own data collector and data shipper (e.g., [Beats][4]). Note that I won’t be showing you how to centralize Docker containers log on Elasticsearch. + +>[“How do you keep track of all that’s happening in a Swarm Mode cluster? Not easily.” via @fntlnz][5] + +### Why Do We Need to Monitor Containers? + +Imagine yourself in the classic situation of managing a virtual machine, either just one or several. You are a tmux hero, so you have your sessions preconfigured to do basically everything, monitoring included. There’s a problem in production? You just do a top, htop, iotop, jnettop, whatevertop on all your machines, and you’re ready for troubleshooting! + +Now imagine that you have the same three nodes but split into 50 containers. You need some history displayed nicely in a single place where you can perform queries to know what happened instead of just risking your life in front of those ncurses tools. + +### What Is the Elastic Stack? + +The Elastic Stack is a set of tools composed of: + +- Elasticsearch +- Kibana +- Logstash +- Beats + +We’re going to use a few open-source tools from the Elastic Stack, such as Elasticsearch for the JSON-based analytics engine and Kibana to visualize data and create dashboards. + +Another important piece of the Elastic Stack is Beats, but in this post, we’re focused on containers. There’s no official Beat for Docker, so we’ll just use cAdvisor that can natively talk with Elasticsearch. + +cAdvisor is a tool that collects, aggregates, and exports metrics about running containers. In our case, those metrics are being exported to an Elasticsearch storage. + +Two cool facts about cAdvisor are: + +- It’s not limited to Docker containers. +- It has its own webserver with a simple dashboard to visualize gathered metrics for the current node. + +### Set Up a Test Cluster or BYOI + +As I did in my previous posts, my habit is to provide a small script to allow the reader to set up a test environment on which to try out my project’s steps in no time. So you can use the following not-for-production-use script to set up a little Swarm Mode cluster with Elasticsearch running as a container. + +>If you have enough time/experience, you can BYOI (Bring Your Own Infrastructure). + + +To follow this post, you’ll just need: + +- One or more nodes running the Docker daemon >= 1.12 +- At least a stand-alone Elasticsearch node 2.4.X + +Again, note that this post is not about setting up a production-ready Elasticsearch cluster. A single node cluster is not recommended for production. So if you’re planning a production installation, please refer to [Elastic guidelines][6]. + +### A friendly note for early adopters + +I’m usually an early adopter (and I’m already using the latest alpha version in production, of course). But for this post, I chose not to use the latest Elasticsearch 5.0.0 alpha. Their roadmap is not perfectly clear to me, and I don’t want be the root cause of your problems! + +So the Elasticsearch reference version for this post is the latest stable version, 2.4.0 at the moment of writing. + +### Test cluster setup script + +As said, I wanted to provide this script for everyone who would like to follow the blog without having to figure out how to create a Swarm cluster and install an Elasticsearch. Of course, you can skip this if you choose to use your own Swarm Mode engines and your own Elasticsearch nodes. + +To execute the setup script, you’ll need: + +- [Docker Machine][7] – latest version: to provision Docker engines on DigitalOcean +- [DigitalOcean API Token][8]: to allow docker-machine to start nodes on your behalf + +![](https://resources.codeship.com/hubfs/CTAs/EVAL/Codeship_Request_Trial_Access.png?t=1473869513342) + +### Create Cluster Script + +Now that you have everything we need, you can copy the following script in a file named create-cluster.sh: + +``` +#!/usr/bin/env bash +# +# Create a Swarm Mode cluster with a single master and a configurable number of workers + +workers=${WORKERS:-"worker1 worker2"} + +####################################### +# Creates a machine on Digital Ocean +# Globals: +# DO_ACCESS_TOKEN The token needed to access DigitalOcean's API +# Arguments: +# $1 the actual name to give to the machine +####################################### +create_machine() { + docker-machine create \ + -d digitalocean \ + --digitalocean-access-token=$DO_ACCESS_TOKEN \ + --digitalocean-size 2gb \ + $1 +} + +####################################### +# Executes a command on the specified machine +# Arguments: +# $1 The machine on which to run the command +# $2..$n The command to execute on that machine +####################################### +machine_do() { + docker-machine ssh $@ +} + +main() { + + if [ -z "$DO_ACCESS_TOKEN" ]; then + echo "Please export a DigitalOcean Access token: https://cloud.digitalocean.com/settings/api/tokens/new" + echo "export DO_ACCESS_TOKEN=" + exit 1 + fi + + if [ -z "$WORKERS" ]; then + echo "You haven't provided your workers by setting the \$WORKERS environment variable, using the default ones: $workers" + fi + + # Create the first and only master + echo "Creating the master" + + create_machine master1 + + master_ip=$(docker-machine ip master1) + + # Initialize the swarm mode on it + echo "Initializing the swarm mode" + machine_do master1 docker swarm init --advertise-addr $master_ip + + # Obtain the token to allow workers to join + worker_tkn=$(machine_do master1 docker swarm join-token -q worker) + echo "Worker token: ${worker_tkn}" + + # Create and join the workers + for worker in $workers; do + echo "Creating worker ${worker}" + create_machine $worker + machine_do $worker docker swarm join --token $worker_tkn $master_ip:2377 + done +} + +main $@ +``` + +And make it executable: + +``` +chmod +x create-cluster.sh +``` + +### Create the cluster + +As the name suggests, we’ll use the script to create the cluster. By default, the script will create a cluster with a single master and two workers. If you want to configure the number of workers, you can do that by setting the WORKERS environment variable. + +Now, let’s create that cluster! + +``` +./create-cluster.sh +``` + +Ok, now you can go out for a coffee. This will take a while. + +Finally the cluster is ready! + +-------------------------------------------------------------------------------- + +via: https://blog.codeship.com/monitoring-docker-containers-with-elasticsearch-and-cadvisor/ + +作者:[Lorenzo Fontana][a] +译者:[译者ID](https://github.com/译者ID) +校对:[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://blog.codeship.com/author/lorenzofontana/ + + + + + + + +[1]: https://github.com/elastic/elasticsearch +[2]: https://github.com/elastic/kibana +[3]: https://github.com/google/cadvisor +[4]: https://github.com/elastic/beats +[5]: https://twitter.com/share?text=%22How+do+you+keep+track+of+all+that%27s+happening+in+a+Swarm+Mode+cluster%3F+Not+easily.%22+via+%40fntlnz&url=https://blog.codeship.com/monitoring-docker-containers-with-elasticsearch-and-cadvisor/ +[6]: https://www.elastic.co/guide/en/elasticsearch/guide/2.x/deploy.html +[7]: https://docs.docker.com/machine/install-machine/ +[8]: https://cloud.digitalocean.com/settings/api/tokens/new From de56c577e577482b8c272152e422b6584e62e99f Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Sat, 17 Sep 2016 23:00:45 +0800 Subject: [PATCH 0173/2437] =?UTF-8?q?ucasFL=E7=BF=BB=E8=AF=91=E4=B8=AD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...tall Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md b/sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md index 070d464077..516873c9ea 100644 --- a/sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md +++ b/sources/tech/20160917 How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24.md @@ -1,3 +1,5 @@ +translating by ucasFL + How to Install Latest XFCE Desktop in Ubuntu 16.04 and Fedora 22-24 ==== From 6c9a5b1942185fe4327943872272cd8824bd62c5 Mon Sep 17 00:00:00 2001 From: Lv Feng Date: Sat, 17 Sep 2016 23:06:06 +0800 Subject: [PATCH 0174/2437] =?UTF-8?q?ucasFL=E7=BF=BB=E8=AF=91=E4=B8=AD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- sources/tech/20160827 4 Best Linux Boot Loaders.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sources/tech/20160827 4 Best Linux Boot Loaders.md b/sources/tech/20160827 4 Best Linux Boot Loaders.md index 91deb14d58..7ea63cdb94 100644 --- a/sources/tech/20160827 4 Best Linux Boot Loaders.md +++ b/sources/tech/20160827 4 Best Linux Boot Loaders.md @@ -1,3 +1,5 @@ +translating by ucasFL + 4 Best Linux Boot Loaders ==== From 09c7446a1f3e8a7e94e35011cb9e7c2d34b5b1c8 Mon Sep 17 00:00:00 2001 From: Ezio Date: Sat, 17 Sep 2016 23:09:23 +0800 Subject: [PATCH 0175/2437] =?UTF-8?q?20160917-28=20=E9=80=89=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...ecurity Policy, Your Future Best Friend.md | 257 ++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100644 sources/tech/20160912 Content Security Policy, Your Future Best Friend.md diff --git a/sources/tech/20160912 Content Security Policy, Your Future Best Friend.md b/sources/tech/20160912 Content Security Policy, Your Future Best Friend.md new file mode 100644 index 0000000000..80f435bd4e --- /dev/null +++ b/sources/tech/20160912 Content Security Policy, Your Future Best Friend.md @@ -0,0 +1,257 @@ +Content Security Policy, Your Future Best Friend +===== + +A long time ago, my personal website was attacked. I do not know how it happened, but it happened. Fortunately, the damage from the attack was quite minor: A piece of JavaScript was inserted at the bottom of some pages. I updated the FTP and other credentials, cleaned up some files, and that was that. + +One point made me mad: At the time, there was no simple solution that could have informed me there was a problem and — more importantly — that could have protected the website’s visitors from this annoying piece of code. + +A solution exists now, and it is a technology that succeeds in both roles. Its name is content security policy (CSP). + +### What Is A CSP? Link + +The idea is quite simple: By sending a CSP header from a website, you are telling the browser what it is authorized to execute and what it is authorized to block. + +Here is an example with PHP: + +``` +"); +?> +``` + +#### SOME DIRECTIVES LINK + +You may define global rules or define rules related to a type of asset: + +``` +default-src 'self' ; + # self = same port, same domain name, same protocol => OK +``` + +The base argument is default-src: If no directive is defined for a type of asset, then the browser will use this value. + +``` +script-src 'self' www.google-analytics.com ; + # JS files on these domains => OK +``` + +In this example, we’ve authorized the domain name www.google-analytics.com as a source of JavaScript files to use on our website. We’ve added the keyword 'self'; if we redefined the directive script-src with another rule, it would override default-src rules. + +If no scheme or port is specified, then it enforces the same scheme or port from the current page. This prevents mixed content. If the page is https://example.com, then you wouldn’t be able to load http://www.google-analytics.com/file.js because it would be blocked (the scheme wouldn’t match). However, there is an exception to allow a scheme upgrade. If http://example.com tries to load https://www.google-analytics.com/file.js, then the scheme or port would be allowed to change to facilitate the scheme upgrade. + +``` +style-src 'self' data: ; + # Data-Uri in a CSS => OK +``` + +In this example, the keyword data: authorizes embedded content in CSS files. + +Under the CSP level 1 specification, you may also define rules for the following: + +- `img-src` + +valid sources of images + +- `connect-src` + +applies to XMLHttpRequest (AJAX), WebSocket or EventSource + +- `font-src` + +valid sources of fonts + +- `object-src` + +valid sources of plugins (for example, `, , `) + +- `media-src` + +valid sources of `