document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2025-03-03 01:10:13 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'LCTT/master'

Browse Source
This commit is contained in:
wxy 2018-03-02 19:54:21 +08:00
commit 7da2a3ba2c
11 changed files with 1013 additions and 90 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

90
translated/tech/20171109 Concurrent Servers- Part 4 - libuv.md → published/20171109 Concurrent Servers- Part 4 - libuv.md
View File

@ -12,17 +12,17 @@
### 使用 libuv 抽象出事件驱动循环
在 [第三节][11] 中，我们看到了基于 `select` 和 `epoll` 的服务器的相似之处，并且，我说过，在它们之间抽象出细微的差别是件很有吸引力的事。许多库已经做到了这些，所以在这一部分中我将去选一个并使用它。我选的这个库是 [libuv][12]，它最初设计用于 Node.js 底层的可移植平台层，并且，后来发现在其它的项目中已有使用。libuv 是用 C 写的，因此，它具有很高的可移植性，非常适用嵌入到像 JavaScript 和 Python 这样的高级语言中。
在 [第三节][11] 中，我们看到了基于 `select` 和 `epoll` 的服务器的相似之处，并且，我说过，在它们之间抽象出细微的差别是件很有吸引力的事。许多库已经做到了这些，所以在这一部分中我将去选一个并使用它。我选的这个库是 [libuv][12]，它最初设计用于 Node.js 底层的可移植平台层，并且，后来发现在其它的项目中也有使用。libuv 是用 C 写的，因此，它具有很高的可移植性，非常适用嵌入到像 JavaScript 和 Python 这样的高级语言中。
虽然 libuv 为抽象出底层平台细节已经变成了一个相当大的框架，但它仍然是以 _事件循环_ 思想为中心的。在我们第三部分的事件驱动服务器中，事件循环在 `main` 函数中是很明确的；当使用 libuv 时，该循环通常隐藏在库自身中，而用户代码仅需要注册事件句柄（作为一个回调函数）和运行这个循环。此外，libuv 会在给定的平台上使用更快的事件循环实现，对于 Linux 它是 epoll，等等。
虽然 libuv 为了抽象出底层平台细节已经变成了一个相当大的框架，但它仍然是以 _事件循环_ 思想为中心的。在我们第三部分的事件驱动服务器中，事件循环是显式定义在 `main` 函数中的；当使用 libuv 时，该循环通常隐藏在库自身中，而用户代码仅需要注册事件句柄（作为一个回调函数）和运行这个循环。此外，libuv 会在给定的平台上使用更快的事件循环实现，对于 Linux 它是 `epoll`，等等。
![libuv loop](https://eli.thegreenplace.net/images/2017/libuvloop.png)
libuv 支持多路事件循环，并且，因此事件循环在库中是非常重要的；它有一个句柄 —— `uv_loop_t`，和创建/杀死/启动/停止循环的函数。也就是说，在这篇文章中，我将仅需要使用 “默认的” 循环，libuv 可通过 `uv_default_loop()` 提供它；多路循环大多用于多线程事件驱动的服务器，这是一个更高级别的话题，我将留在这一系列文章的以后部分。
libuv 支持多路事件循环，因此事件循环在库中是非常重要的；它有一个句柄 —— `uv_loop_t`，以及创建/杀死/启动/停止循环的函数。也就是说，在这篇文章中，我将仅需要使用 “默认的” 循环，libuv 可通过 `uv_default_loop()` 提供它；多路循环大多用于多线程事件驱动的服务器，这是一个更高级别的话题，我将留在这一系列文章的以后部分。
### 使用 libuv 的并发服务器
为了对 libuv 有一个更深的印象，让我们跳转到我们的可靠协议的服务器，它通过我们的这个系列已经有了一个强大的重新实现。这个服务器的结构与第三部分中的基于 select 和 epoll 的服务器有一些相似之处，因为，它也依赖回调。完整的 [示例代码在这里][13]；我们开始设置这个服务器的套接字绑定到一个本地端口：
为了对 libuv 有一个更深的印象，让我们跳转到我们的可靠协议的服务器，它通过我们的这个系列已经有了一个强大的重新实现。这个服务器的结构与第三部分中的基于 `select` 和 `epoll` 的服务器有一些相似之处，因为，它也依赖回调。完整的 [示例代码在这里][13]；我们开始设置这个服务器的套接字绑定到一个本地端口：
```
int portnum = 9090;
@ -47,9 +47,9 @@ if ((rc = uv_tcp_bind(&server_stream, (const struct sockaddr*)&server_address, 0
}
```
除了它被封装进 libuv API 中之外，你看到的是一个相当标准的套接字。在它的返回中，我们取得一个可工作于任何 libuv 支持的平台上的可移植接口。
除了它被封装进 libuv API 中之外，你看到的是一个相当标准的套接字。在它的返回中，我们取得了一个可工作于任何 libuv 支持的平台上的可移植接口。
这些代码也展示了很认真负责的错误处理；多数的 libuv 函数返回一个整数状态，返回一个负数意味着出现了一个错误。在我们的服务器中，我们把这些错误看做致命问题进行处理，但也可以设想为一个更优雅的错误恢复。
这些代码也展示了很认真负责的错误处理；多数的 libuv 函数返回一个整数状态，返回一个负数意味着出现了一个错误。在我们的服务器中，我们把这些错误看做致命问题进行处理，但也可以设想一个更优雅的错误恢复。
现在，那个套接字已经绑定，是时候去监听它了。这里我们运行首个回调注册：
@ -73,7 +73,7 @@ uv_run(uv_default_loop(), UV_RUN_DEFAULT);
return uv_loop_close(uv_default_loop());
```
注意，在运行事件循环之前，只有一个回调是通过 main 注册的；我们稍后将看到怎么去添加更多的回调。在事件循环的整个运行过程中，添加和删除回调并不是一个问题 —— 事实上，大多数服务器就是这么写的。
注意，在运行事件循环之前，只有一个回调是通过 `main` 注册的；我们稍后将看到怎么去添加更多的回调。在事件循环的整个运行过程中，添加和删除回调并不是一个问题 —— 事实上，大多数服务器就是这么写的。
这是一个 `on_peer_connected`，它处理到服务器的新的客户端连接：
@ -132,8 +132,8 @@ void on_peer_connected(uv_stream_t* server_stream, int status) {
这些代码都有很好的注释，但是，这里有一些重要的 libuv 语法我想去强调一下：
* 传入自定义数据到回调中：因为 C 还没有闭包，这可能是个挑战，libuv 在它的所有的处理类型中有一个 `void* data` 字段；这些字段可以被用于传递用户数据。例如，注意 `client->data` 是如何指向到一个 `peer_state_t` 结构上，以便于 `uv_write` 和 `uv_read_start` 注册的回调可以知道它们正在处理的是哪个客户端的数据。
* 内存管理：在带有垃圾回收的语言中进行事件驱动编程是非常容易的，因为，回调通常运行在一个它们注册的完全不同的栈帧中，使得基于栈的内存管理很困难。它总是需要传递堆分配的数据到 libuv 回调中（当所有回调运行时，除了 main，其它的都运行在栈上），并且，为了避免泄漏，许多情况下都要求这些数据去安全释放。这些都是些需要实践的内容 [[1]][6]。
* 传入自定义数据到回调中：因为 C 语言还没有闭包，这可能是个挑战，libuv 在它的所有的处理类型中有一个 `void* data` 字段；这些字段可以被用于传递用户数据。例如，注意 `client->data` 是如何指向到一个 `peer_state_t` 结构上，以便于 `uv_write` 和 `uv_read_start` 注册的回调可以知道它们正在处理的是哪个客户端的数据。
* 内存管理：在带有垃圾回收的语言中进行事件驱动编程是非常容易的，因为，回调通常运行在一个与它们注册的地方完全不同的栈帧中，使得基于栈的内存管理很困难。它总是需要传递堆分配的数据到 libuv 回调中（当所有回调运行时，除了 `main`，其它的都运行在栈上），并且，为了避免泄漏，许多情况下都要求这些数据去安全释放（`free()`）。这些都是些需要实践的内容 ^注1 。
这个服务器上对端的状态如下：
@ -146,7 +146,7 @@ typedef struct {
} peer_state_t;
```
它与第三部分中的状态非常类似；我们不再需要 sendptr，因为，在调用 "done writing" 回调之前，`uv_write` 将确保去发送它提供的整个缓冲。我们也为其它的回调使用保持了一个到客户端的指针。这里是 `on_wrote_init_ack`：
它与第三部分中的状态非常类似；我们不再需要 `sendptr`，因为，在调用 “done writing” 回调之前，`uv_write` 将确保发送它提供的整个缓冲。我们也为其它的回调使用保持了一个到客户端的指针。这里是 `on_wrote_init_ack`：
```
void on_wrote_init_ack(uv_write_t* req, int status) {
@ -171,7 +171,7 @@ void on_wrote_init_ack(uv_write_t* req, int status) {
}
```
然后，我们确信知道了这个初始的 '*' 已经被发送到对端，我们通过调用 `uv_read_start` 去监听从这个对端来的入站数据，它注册一个回调（`on_peer_read`）去被调用，不论什么时候，事件循环都在套接字上接收来自客户端的调用：
然后，我们确信知道了这个初始的 `'*'` 已经被发送到对端，我们通过调用 `uv_read_start` 去监听从这个对端来的入站数据，它注册一个将被事件循环调用的回调（`on_peer_read`），不论什么时候，事件循环都在套接字上接收来自客户端的调用：
```
void on_peer_read(uv_stream_t* client, ssize_t nread, const uv_buf_t* buf) {
@ -236,11 +236,11 @@ void on_peer_read(uv_stream_t* client, ssize_t nread, const uv_buf_t* buf) {
}
```
这个服务器的运行时行为非常类似于第三部分的事件驱动服务器：所有的客户端都在一个单个的线程中并发处理。并且一些行为被维护在服务器代码中：服务器的逻辑实现为一个集成的回调，并且长周期运行是禁止的，因为它会阻塞事件循环。这一点也很类似。让我们进一步探索这个问题。
这个服务器的运行时行为非常类似于第三部分的事件驱动服务器：所有的客户端都在一个单个的线程中并发处理。并且类似的，一些特定的行为必须在服务器代码中维护：服务器的逻辑实现为一个集成的回调，并且长周期运行是禁止的，因为它会阻塞事件循环。这一点也很类似。让我们进一步探索这个问题。
### 在事件驱动循环中的长周期运行的操作
单线程的事件驱动代码使它先天地对一些常见问题非常敏感：整个循环中的长周期运行的代码块。参见如下的程序：
单线程的事件驱动代码使它先天就容易受到一些常见问题的影响：长周期运行的代码会阻塞整个循环。参见如下的程序：
```
void on_timer(uv_timer_t* timer) {
@ -280,23 +280,21 @@ on_timer [18850 ms]
...
```
`on_timer` 忠实地每秒执行一次，直到随机出现的睡眠为止。在那个时间点，`on_timer` 不再被调用，直到睡眠时间结束；事实上，_没有其它的回调_  在这个时间帧中被调用。这个睡眠调用阻塞当前线程，它正是被调用的线程，并且也是事件循环使用的线程。当这个线程被阻塞后，事件循环也被阻塞。
`on_timer` 忠实地每秒执行一次，直到随机出现的睡眠为止。在那个时间点，`on_timer` 不再被调用，直到睡眠时间结束；事实上，_没有其它的回调_  会在这个时间帧中被调用。这个睡眠调用阻塞了当前线程，它正是被调用的线程，并且也是事件循环使用的线程。当这个线程被阻塞后，事件循环也被阻塞。
这个示例演示了在事件驱动的调用中为什么回调不能被阻塞是多少的重要。并且，同样适用于 Node.js 服务器、客户端侧的 Javascript、大多数的 GUI 编程框架、以及许多其它的异步编程模型。
但是，有时候运行耗时的任务是不可避免的。并不是所有任务都有一个异步 APIs；例如，我们可能使用一些仅有同步 API 的库去处理，或者，正在执行一个可能的长周期计算。我们如何用事件驱动编程去结合这些代码？线程可以帮到你！
但是，有时候运行耗时的任务是不可避免的。并不是所有任务都有一个异步 API；例如，我们可能使用一些仅有同步 API 的库去处理，或者，正在执行一个可能的长周期计算。我们如何用事件驱动编程去结合这些代码？线程可以帮到你！
### “转换” 阻塞调用到异步调用的线程
### “转换” 阻塞调用为异步调用的线程
一个线程池可以被用于去转换阻塞调用到异步调用，通过与事件循环并行运行，并且当任务完成时去由它去公布事件。一个给定的阻塞函数 `do_work()`，这里介绍了它是怎么运行的：
一个线程池可以用于转换阻塞调用为异步调用，通过与事件循环并行运行，并且当任务完成时去由它去公布事件。以阻塞函数 `do_work()` 为例，这里介绍了它是怎么运行的：
1. 在一个回调中，用 `do_work()` 代表直接调用，我们将它打包进一个 “任务”，并且请求线程池去运行这个任务。当任务完成时，我们也为循环去调用它注册一个回调；我们称它为 `on_work_done()`。
1. 不在一个回调中直接调用 `do_work()` ，而是将它打包进一个 “任务”，让线程池去运行这个任务。当任务完成时，我们也为循环去调用它注册一个回调；我们称它为 `on_work_done()`。
2. 在这个时间点，我们的回调就可以返回了，而事件循环保持运行；在同一时间点，线程池中的有一个线程运行这个任务。
3. 一旦任务运行完成，通知主线程（指正在运行事件循环的线程），并且事件循环调用 `on_work_done()`。
2. 在这个时间点，我们的回调可以返回并且事件循环保持运行；在同一时间点，线程池中的一个线程运行这个任务。
3. 一旦任务运行完成，通知主线程（指正在运行事件循环的线程），并且，通过事件循环调用 `on_work_done()`。
让我们看一下，使用 libuv 的工作调度 API，是怎么去解决我们前面的 timer/sleep 示例中展示的问题的：
让我们看一下，使用 libuv 的工作调度 API，是怎么去解决我们前面的计时器/睡眠示例中展示的问题的：
```
void on_after_work(uv_work_t* req, int status) {
@ -327,7 +325,7 @@ int main(int argc, const char** argv) {
}
```
通过一个 work_req [[2]][14] 类型的句柄，我们进入一个任务队列，代替在 `on_timer` 上直接调用 sleep，这个函数在任务中（`on_work`）运行，并且，一旦任务完成（`on_after_work`），这个函数被调用一次。`on_work` 在这里是指发生的 “work”（阻塞中的/耗时的操作）。在这两个回调传递到 `uv_queue_work` 时，注意一个关键的区别：`on_work` 运行在线程池中，而 `on_after_work` 运行在事件循环中的主线程上 - 就好像是其它的回调一样。
通过一个 `work_req` ^注2 类型的句柄，我们进入一个任务队列，代替在 `on_timer` 上直接调用 sleep，这个函数在任务中（`on_work`）运行，并且，一旦任务完成（`on_after_work`），这个函数被调用一次。`on_work` 是指 “work”（阻塞中的/耗时的操作）进行的地方。注意在这两个回调传递到 `uv_queue_work` 时的一个关键区别：`on_work` 运行在线程池中，而 `on_after_work` 运行在事件循环中的主线程上 —— 就好像是其它的回调一样。
让我们看一下这种方式的运行：
@ -347,25 +345,25 @@ on_timer [97578 ms]
...
```
即便在 sleep 函数被调用时，定时器也每秒钟滴答一下，睡眠（sleeping）现在运行在一个单独的线程中，并且不会阻塞事件循环。
即便在 sleep 函数被调用时，定时器也每秒钟滴答一下，睡眠现在运行在一个单独的线程中，并且不会阻塞事件循环。
### 一个用于练习的素数测试服务器
因为通过睡眼去模拟工作并不是件让人兴奋的事，我有一个事先准备好的更综合的一个示例 - 一个基于套接字接受来自客户端的数字的服务器，检查这个数字是否是素数，然后去返回一个 “prime" 或者 “composite”。完整的 [服务器代码在这里][15] - 我不在这里粘贴了，因为它太长了，更希望读者在一些自己的练习中去体会它。
因为通过睡眠去模拟工作并不是件让人兴奋的事，我有一个事先准备好的更综合的一个示例 —— 一个基于套接字接受来自客户端的数字的服务器，检查这个数字是否是素数，然后去返回一个 “prime" 或者 “composite”。完整的 [服务器代码在这里][15] —— 我不在这里粘贴了，因为它太长了，更希望读者在一些自己的练习中去体会它。
这个服务器使用了一个原生的素数测试算法，因此，对于大的素数可能花很长时间才返回一个回答。在我的机器中，对于 2305843009213693951，它花了 ~5 秒钟去计算，但是，你的方法可能不同。
练习 1：服务器有一个设置（通过一个名为 MODE 的环境变量）要么去在套接字回调（意味着在主线程上）中运行素数测试，要么在 libuv 工作队列中。当多个客户端同时连接时，使用这个设置来观察服务器的行为。当它计算一个大的任务时，在阻塞模式中，服务器将不回复其它客户端，而在非阻塞模式中，它会回复。
练习 1：服务器有一个设置（通过一个名为 `MODE` 的环境变量）要么在套接字回调（意味着在主线程上）中运行素数测试，要么在 libuv 工作队列中。当多个客户端同时连接时，使用这个设置来观察服务器的行为。当它计算一个大的任务时，在阻塞模式中，服务器将不回复其它客户端，而在非阻塞模式中，它会回复。
练习 2；libuv 有一个缺省大小的线程池，并且线程池的大小可以通过环境变量配置。你可以通过使用多个客户端去实验找出它的缺省值是多少？找到线程池缺省值后，使用不同的设置去看一下，在重负载下怎么去影响服务器的响应能力。
练习 2：libuv 有一个缺省大小的线程池，并且线程池的大小可以通过环境变量配置。你可以通过使用多个客户端去实验找出它的缺省值是多少？找到线程池缺省值后，使用不同的设置去看一下，在重负载下怎么去影响服务器的响应能力。
### 在非阻塞文件系统中使用工作队列
对于仅傻傻的演示和 CPU 密集型的计算来说，将可能的阻塞操作委托给一个线程池并不是明智的；libuv 在它的文件系统 APIs 中本身就大量使用了这种性能。通过这种方式，libuv 使用一个异步 API，在一个轻便的方式中，显示出它强大的文件系统的处理能力。
对于只是呆板的演示和 CPU 密集型的计算来说，将可能的阻塞操作委托给一个线程池并不是明智的；libuv 在它的文件系统 API 中本身就大量使用了这种能力。通过这种方式，libuv 使用一个异步 API，以一个轻便的方式显示出它强大的文件系统的处理能力。
让我们使用 `uv_fs_read()`，例如，这个函数从一个文件中（以一个 `uv_fs_t` 句柄为代表)读取一个文件到一个缓冲中 [[3]][16]，并且当读取完成后调用一个回调。换句话说，`uv_fs_read()` 总是立即返回，甚至如果文件在一个类似 NFS 的系统上，并且，数据到达缓冲区可能需要一些时间。换句话说，这个 API 与这种方式中其它的 libuv APIs 是异步的。这是怎么工作的呢？
让我们使用 `uv_fs_read()`，例如，这个函数从一个文件中（表示为一个 `uv_fs_t` 句柄）读取一个文件到一个缓冲中 ^注3，并且当读取完成后调用一个回调。换句话说，`uv_fs_read()` 总是立即返回，即使是文件在一个类似 NFS 的系统上，而数据到达缓冲区可能需要一些时间。换句话说，这个 API 与这种方式中其它的 libuv API 是异步的。这是怎么工作的呢？
在这一点上，我们看一下 libuv 的底层；内部实际上非常简单，并且它是一个很好的练习。作为一个便携的库，libuv 对于 Windows 和 Unix 系统在它的许多函数上有不同的实现。我们去看一下在 libuv 源树中的 src/unix/fs.c。
在这一点上，我们看一下 libuv 的底层；内部实际上非常简单，并且它是一个很好的练习。作为一个可移植的库，libuv 对于 Windows 和 Unix 系统在它的许多函数上有不同的实现。我们去看一下在 libuv 源树中的 `src/unix/fs.c`。
这是 `uv_fs_read` 的代码：
@ -400,9 +398,9 @@ int uv_fs_read(uv_loop_t* loop, uv_fs_t* req,
}
```
第一次看可能觉得很困难，因为它延缓真实的工作到 INIT 和 POST 宏中，在 POST 中与一些本地变量一起设置。这样做可以避免了文件中的许多重复代码。
第一次看可能觉得很困难，因为它延缓真实的工作到 `INIT` 和 `POST` 宏中，以及为 `POST` 设置了一些本地变量。这样做可以避免了文件中的许多重复代码。
这是 INIT 宏：
这是 `INIT` 宏：
```
#define INIT(subtype) \
@ -421,9 +419,9 @@ int uv_fs_read(uv_loop_t* loop, uv_fs_t* req,
while (0)
```
它设置了请求，并且更重要的是，设置 `req->fs_type` 域为真实的 FS 请求类型。因为 `uv_fs_read` 调用 invokes INIT(READ)，它意味着 `req->fs_type` 被分配一个常数 `UV_FS_READ`。
它设置了请求，并且更重要的是，设置 `req->fs_type` 域为真实的 FS 请求类型。因为 `uv_fs_read` 调用 `INIT(READ)`，它意味着 `req->fs_type` 被分配一个常数 `UV_FS_READ`。
这是 POST 宏：
这是 `POST` 宏：
```
#define POST \
@ -440,31 +438,25 @@ int uv_fs_read(uv_loop_t* loop, uv_fs_t* req,
while (0)
```
它做什么取决于回调是否为 NULL。在 libuv 文件系统 APIs 中，一个 NULL 回调意味着我们真实地希望去执行一个 _同步_ 操作。在这种情况下，POST 直接调用 `uv__fs_work`（我们需要了解一下这个函数的功能），而对于一个 non-NULL 回调，它提交 `uv__fs_work` 作为一个工作事项到工作队列（指的是线程池），然后，注册 `uv__fs_done` 作为回调；该函数执行一些登记并调用用户提供的回调。
它做什么取决于回调是否为 `NULL`。在 libuv 文件系统 API 中，一个 `NULL` 回调意味着我们真实地希望去执行一个 _同步_ 操作。在这种情况下，`POST` 直接调用 `uv__fs_work`（我们需要了解一下这个函数的功能），而对于一个非 `NULL` 回调，它把 `uv__fs_work` 作为一个工作项提交到工作队列（指的是线程池），然后，注册 `uv__fs_done` 作为回调；该函数执行一些登记并调用用户提供的回调。
如果我们去看 `uv__fs_work` 的代码，我们将看到它使用很多宏去按需路由工作到真实的文件系统调用。在我们的案例中，对于 `UV_FS_READ` 这个调用将被 `uv__fs_read` 生成，它（最终）使用普通的 POSIX APIs 去读取。这个函数可以在一个 _阻塞_ 方式中很安全地实现。因为，它通过异步 API 调用时被置于一个线程池中。
如果我们去看 `uv__fs_work` 的代码，我们将看到它使用很多宏按照需求将工作分发到实际的文件系统调用。在我们的案例中，对于 `UV_FS_READ` 这个调用将被 `uv__fs_read` 生成，它（最终）使用普通的 POSIX API 去读取。这个函数可以在一个 _阻塞_ 方式中很安全地实现。因为，它通过异步 API 调用时被置于一个线程池中。
在 Node.js 中，fs.readFile 函数是映射到 `uv_fs_read` 上。因此，可以在一个非阻塞模式中读取文件，甚至是当底层文件系统 API 是阻塞方式时。
在 Node.js 中，`fs.readFile` 函数是映射到 `uv_fs_read` 上。因此，可以在一个非阻塞模式中读取文件，甚至是当底层文件系统 API 是阻塞方式时。
* * *
[[1]][1] 为确保服务器不泄露内存，我在一个启用泄露检查的 Valgrind 中运行它。因为服务器经常是被设计为永久运行，这是一个挑战；为克服这个问题，我在服务器上添加了一个 “kill 开关” - 一个从客户端接收的特定序列，以使它可以停止事件循环并退出。这个代码在 `theon_wrote_buf` 句柄中。
[[2]][2] 在这里我们不过多地使用 `work_req`；讨论的素数测试服务器接下来将展示怎么被用于去传递上下文信息到回调中。
[[3]][3] `uv_fs_read()` 提供了一个类似于 preadv Linux 系统调用的通用 API：它使用多缓冲区用于排序，并且支持一个到文件中的偏移。基于我们讨论的目的可以忽略这些特性。
- 注1： 为确保服务器不泄露内存，我在一个启用泄露检查的 Valgrind 中运行它。因为服务器经常是被设计为永久运行，这是一个挑战；为克服这个问题，我在服务器上添加了一个 “kill 开关” —— 一个从客户端接收的特定序列，以使它可以停止事件循环并退出。这个代码在 `theon_wrote_buf` 句柄中。
- 注2： 在这里我们不过多地使用 `work_req`；讨论的素数测试服务器接下来将展示怎么被用于去传递上下文信息到回调中。
- 注3： `uv_fs_read()` 提供了一个类似于 `preadv` Linux 系统调用的通用 API：它使用多缓冲区用于排序，并且支持一个到文件中的偏移。基于我们讨论的目的可以忽略这些特性。
--------------------------------------------------------------------------------
via: https://eli.thegreenplace.net/2017/concurrent-servers-part-4-libuv/
作者：[Eli Bendersky ][a]
作者：[Eli Bendersky][a]
译者：[qhwdw](https://github.com/qhwdw)
校对：[校对者ID](https://github.com/校对者ID)
校对：[wxy](https://github.com/wxy)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出

192
sources/talk/20180227 Top 10 open source legal stories that shook 2017.md Normal file
View File

@ -0,0 +1,192 @@
LightonXue翻译中
Top 10 open source legal stories that shook 2017
======
![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/law_legal_gavel_court.jpg?itok=tc27pzjI)
Like every year, legal issues were a hot topic in the open source world in 2017. While we're deep into the first quarter of the year, it's still worthwhile to look back at the top legal news in open source last year.
### 1. GitHub revises ToS
In February 2017, GitHub [announced][1] it was revising its terms of service and invited comments on the changes, several of which concerned rights in the user-uploaded content. The [earlier GitHub terms][2] included an agreement by the user to allow others to "view and fork" public repositories, as well as an indemnification provision protecting GitHub against third-party claims. The new terms added a license from the user to GitHub to allow it to store and serve content, a default ["inbound=outbound"][3] contributor license, and an agreement by the user to comply with third-party licenses covering uploaded content. While keeping the "view and fork" language, the new terms state that further rights can be granted by adopting an open source license. The terms also add a waiver of [moral rights][4] with a two-level fallback license, the second license granting GitHub permission to use content without attribution and to make reasonable adaptations "as necessary to render the website and provide the service."
In March, after the new terms became effective, concerns were raised by several developers, notably [Thorsten Glaser][5] and [Joey][6] [Hess][7], who said they would be removing their repositories from GitHub. As Glaser and Hess read the new terms, they seemed to require users to grant rights to GitHub and other users that were broader than third-party licenses would permit, particularly copyleft licenses like the GPL and licenses requiring attribution. Moreover, the license to GitHub could be read as giving it a more favorable license in users' own content than ordinary users would receive under the nominal license. Donald Robertson of the Free Software Foundation (FSF) [wrote][8] that, while GitHub's terms were confusing, they were not in conflict with copyleft: "Because it's highly unlikely that GitHub intended to destroy their business model and user base, we don't read the ambiguity in the terms as granting or requiring overly broad permissions outside those already granted by the GPL."
GitHub eventually added a sentence addressing the issue; it can be seen in the [current version][9] of the terms: "If you upload Content that already comes with a license granting GitHub the permissions we need to run our Service, no additional license is required."
### 2. Kernel enforcement statement
[Section 4][10] of GPLv2 speaks of automatic termination of rights for those who violate the terms of the license. By [2006][11] the FSF had come to see this provision as unduly harsh in the case of inadvertent violations. GPLv3 modifies the GPLv2 approach to termination by providing a 30-day cure opportunity for first-time violators as well as a 60-day period of repose. Many GPL projects like the Linux kernel continue to be licensed under GPLv2.
As I [wrote][12] last year, 2016 saw public condemnation of the GPL enforcement tactics of former Netfilter contributor Patrick McHardy. In a further reaction to McHardy's conduct, the Linux Foundation [Technical Advisory Board][13] (TAB), elected by the individual kernel developers, drafted a Linux Kernel Enforcement Statement, which was [announced by Greg Kroah-Hartman][14] on the Linux kernel mailing list (LKML) on October 16, 2017. The [statement][15], now part of the kernel's Documentation directory, incorporates the GPLv3 cure and repose language verbatim as a "commitment to users of the Linux kernel on behalf of ourselves and any successors to our copyright interests." The commitment, described as a grant of additional permission under GPLv2, applies to non-defensive assertions of GPLv2 rights. The kernel statement in effect adopts a recommendation in the [Principles of Community-Oriented GPL Enforcement][16]. To date, the statement has been signed by over 100 kernel developers. Kroah-Hartman published an [FAQ][17] on the statement and a detailed [explanation][18] authored by several TAB members.
### 3. Red Hat, Facebook, Google, and IBM announce GPLv2/LGPLv2.x cure commitment
A month after the announcement of the kernel enforcement statement on LKML, a coalition of companies led by Red Hat and including Facebook, Google, and IBM [announced][19] their own commitment to [extend the GPLv3 cure][20] and repose opportunities to all code covered by their copyrights and licensed under GPLv2, LGPLv2, and LGPLv2.1. (The termination provision in LGPLv2.x is essentially identical to that in GPLv2.) As with the kernel statement, the commitment does not apply to defensive proceedings or claims brought in response to some prior proceeding or claim (for example, a GPL violation counterclaim in a patent infringement lawsuit, as occurred in [Twin Peaks Software v. Red Hat][21]).
### 4. EPL 2.0 released
The [Eclipse Public License version 1.0][22], a weak copyleft license that descends from the [Common Public License][23] and indirectly the [IBM Public License][24], has been the primary license of Eclipse Foundation projects. It sees significant use outside of Eclipse as well; for example, EPL is the license of the [Clojure][25] language implementation and the preferred open source license of the Clojure community, and it is the main license of [OpenDaylight][26].
Following a quiet two-year community review process, in August 2017 the Eclipse Foundation [announced][27] that a new version 2 of the EPL had been approved by the Eclipse Foundation board and by the OSI. The Eclipse Foundation intends EPL 2.0 to be the default license for Eclipse community projects.
EPL 2.0 is a fairly conservative license revision. Perhaps the most notable change concerns GPL compatibility. EPL 1.0 is regarded as GPL-incompatible by both the [FSF][28] and the [Eclipse Foundation][29]. The FSF has suggested that this is at least because of the conflicting copyleft requirements in the two licenses, and (rather more dubiously) the choice of law clause in EPL 1.0, which has been removed in EPL 2.0. As a weak copyleft license, EPL normally requires at least some subset of derivative works to be licensed under EPL if distributed in source code form. [FSF][30] and [Eclipse][31] published opinions about the use of GPL for Eclipse IDE plugins several years ago. Apart from the issue of license compatibility, the Eclipse Foundation generally prohibits projects from distributing third-party code under GPL and LGPL.
While EPL 2.0 remains GPL-incompatible by default, it enables the initial "Contributor" to authorize the licensing of EPL-covered source code under a "Secondary License"—GPLv2, GPLv3, or a later version of the GPL, which may include identified GPL exceptions or additional permissions like the [Classpath Exception][32]—if the EPL-covered code is combined with GPL-licensed code contained in a separate file. Some Eclipse projects have already relicensed to EPL 2.0 and are making use of this "Secondary License" feature, including [OMR][33] and [OpenJ9][34]. As the FSF [observes][35], invocation of the Secondary License feature is roughly equivalent to dual-licensing the code under EPL / GPL.
### 5. Java EE migration to Eclipse
The [Java Community Process][36] (JCP) facilitates development of Java technology specifications (Java Specification Requests, aka JSRs), including those defining the [Java Enterprise Edition][37] platform (Java EE). The JCP rests on a complex legal architecture centered around the Java Specification Participation Agreement (JSPA). While JCP governance is shared among multiple organizational and individual participants, the JCP is in no way vendor-neutral. Oracle owns the Java trademark and has special controls over the JCP. Some JCP [reforms][38] were adopted several years ago, including measures to mandate open source licensing and open source project development practices for JSR reference implementations (RIs), but efforts to modernize the JSPA stalled during the pendency of the Oracle v. Google litigation.
In August 2017, Oracle announced it would explore [moving Java EE][39] to an open source foundation. Following consultation with IBM and Red Hat, the two other major contributors to Java EE, Oracle announced in September that it had [selected the Eclipse Foundation][40] to house the successor to Java EE.
The migration to Eclipse has been underway since then. The Eclipse board approved a new top-level Eclipse project, [EE4J][41] (Eclipse Enterprise for Java), to serve as the umbrella project for development of RIs and technology compatibility kits (TCKs) for the successor platform. The [GlassFish][42] project, consisting of source code of the RIs for the majority of Java EE JSRs for which Oracle has served as specification lead, has mostly been under a dual license of CDDL and GPLv2 plus the Classpath Exception. Oracle is in the process of [relicensing this code][43] to EPL 2.0 with GPLv2 plus the Classpath Exception as the Secondary License (see EPL 2.0 topic). In addition, Oracle is expected to relicense proprietary Java EE TCKs so they can be developed as Eclipse open source projects. Still to be determined are the name of an Eclipse-owned certification mark to succeed Java EE and the development of a new specification process in place of the one defined in the JSPA.
### 6. React licensing controversy
Open source licenses that specifically address patent licensing often couple the patent license grant with a "patent defense" clause, terminating the patent license upon certain acts of litigation brought by the licensee, an approach borrowed from standards agreements. The early period of corporate experimentation with open source licensing was characterized by enthusiasm for patent defense clauses that were broad (in the sense that a relatively wide range of conduct would trigger termination). The arrival of the Apache License 2.0 and Eclipse Public License 1.0 in 2004 marked an end to that era; their patent termination criteria are basically limited to patent lawsuits in which the user accuses the licensed software itself of infringement.
In May 2013 Facebook released the [React][44] JavaScript library under the Apache License 2.0, but the 0.12.0 release (October 2014) switched to the 3-clause BSD license along with a patent license grant in a separate `PATENTS` file. The idea of using a simple, standard permissive open source license with a bespoke patent license in a separate file has some precedent in projects maintained by [Google][45] and [Microsoft][46]. However, the patent defense clauses in those cases take the narrow Apache/EPL approach. The React `PATENTS` language terminated the patent license in cases where the licensee brought a patent infringement action against Facebook, or against any party "arising from" any Facebook product, even where the claim was unrelated to React, as well as where the licensee alleged that a Facebook patent was invalid or unenforceable. In response to criticism from the community, Facebook [revised][47] the patent license language in April 2015, but the revised version continued to include as termination criteria patent litigation against Facebook and patent litigation "arising from" Facebook products.
Facebook came to apply the React license to many of its community projects. In April 2017 an [issue][48] was opened in the Apache Software Foundation (ASF) "Legal Discuss" issue tracker concerning whether Apache Cassandra could use [RocksDB][49], another Facebook project using the React license, as a dependency. In addition to the several other ASF projects that were apparently already using RocksDB, a large number of ASF projects used React itself. In June, Chris Mattmann, VP of legal affairs for the ASF, [ruled][50] that the React license was relegated to the forbidden Category X (see my discussion of the [JSON license][12] last year)—despite the fact that the ASF has long placed open source licenses with similarly broad patent defense clauses (MPL 1.1, IBM-PL, CPL) in its semi-favored Category B. In response, Facebook relicensed RocksDB under [GPLv2][51] and the [Apache License][52] 2.0, and a few months later announced it was [relicensing React][53] and three other identically licensed projects under the MIT license. More recent Facebook project license changes from the React approach to conventional open source licenses include [osquery][54] (GPLv2 / Apache License 2.0) and [React Native][55] (MIT).
Much of the community criticism of the React license was rather misinformed and often seemed to be little more than ad hominem attack against Facebook. One of the few examples of sober, well-reasoned analysis of the topic is [Heather Meeker's article][56] on Opensource.com. Whatever actual merits the React license may have, Facebook's decision to use it without making it licensor-neutral and without seeking OSI approval were tactical mistakes, as [Simon Phipps points out][57].
### 7. OpenSSL relicensing effort
The [license][58] covering most of OpenSSL is a conjunction of two 1990s-vintage BSD-derivative licenses. The first closely resembles an early license of the Apache web server. The second is the bespoke license of OpenSSL's predecessor project SSLeay. Both licenses contain an advertising clause like that in the 4-clause BSD license. The closing sentence of the SSLeay license, a gratuitous snipe at the GPL, supports an interpretation, endorsed by the FSF but no doubt unintended, that the license is copyleft. If only because of the advertising clauses, the OpenSSL license has long been understood to be GPL-incompatible, as Mark McLoughlin explained in a now-classic [essay][59].
In 2015, a year after the disclosure of the Heartbleed vulnerability and the Linux Foundation's subsequent formation of the [Core Infrastructure Initiative][60], Rich Salz said in a [blog post][61] that OpenSSL planned to relicense to the Apache License 2.0. The OpenSSL team followed up in March 2017 with a [press release][62] announcing the relicensing initiative and set up a website to collect agreements to the license change from the project's several hundred past contributors.
A form email sent to identified individual contributors, asking for permission to relicense, soon drew criticism, mainly because of its closing sentence: "If we do not hear from you, we will assume that you have no objection." Some raised policy and legal concerns over what Theo de Raadt called a "[manufacturing consent in volume][63]" approach. De Raadt mocked the effort by [posting][64] a facetious attempt to relicense GCC to the [ISC license][65].
Salz posted an [update][66] on the relicensing effort in June. At that point, 40% of contacted contributors had responded, with the vast majority in favor of the license change and fewer than a dozen objections, amounting to 86 commits, with half of them surviving in the master branch. Salz described in detail the reasonable steps the project had taken to review those objections, resulting in a determination that at most 10 commits required removal and rewriting.
### 8. Open Source Security v. Perens
Open Source Security, Inc. (OSS) is the commercial vehicle through which Brad Spengler maintains the out-of-tree [grsecurity][67] patchset to the Linux kernel. In 2015, citing concerns about GPL noncompliance by users and misuse of the grsecurity trademark, OSS began [limiting access][68] to the stable patchset to paying customers. In 2017 OSS [ceased][69] releasing any public branches of grsecurity. The [Grsecurity Stable Patch Access Agreement][70] affirms that grsecurity is licensed under GPLv2 and that the user has all GPLv2 "rights and obligations," but states a policy of terminating access to future updates if a user redistributes patchsets or changelogs "outside of the explicit obligations under the GPL to User's customers."
In June 2017, Bruce Perens published a [blog post][71] contending that the grsecurity agreement violated the GPL. OSS sued Perens in the Northern District of California, with claims for defamation, false light, and tortious interference with prospective advantage. In December the court [granted][72] Perens' motion to dismiss, denied without prejudice Perens' motion to strike under the California [anti-SLAPP][73] statute, and denied OSS's motion for partial summary judgment. In essence, the court said that as statements of opinion by a non-lawyer, Perens' blog posts were not defamatory. OSS has said it intends to appeal.
### 9. Artifex Software v. Hancom
Artifex Software licenses [Ghostscript][74] gratis under the [GPL][75] (more recently AGPL) and for revenue under proprietary licenses. In December 2016 Artifex sued Hancom, a South Korean vendor of office suite software, in the Northern District of California. Artifex alleged that Hancom had incorporated Ghostscript into its Hangul word processing program and Hancom Office product without obtaining a proprietary license or complying with the GPL. The [complaint][76] includes claims for breach of contract as well as copyright infringement. In addition to monetary damages, Artifex requested injunctive relief, including an order compelling Hancom to distribute the source code of Hangul and Hancom Office to Hancom's customers.
In April 2017 the court [denied][77] Hancom's motion to dismiss. One of Hancom's arguments was that Artifex did not plead the existence of a contract because there was no demonstration of mutual assent. The court disagreed, stating that the allegations of Hancom's use of Ghostscript, failure to obtain a proprietary license, and public representation that its use of Ghostscript was licensed under the GPL were sufficient to plead the existence of a contract. In addition, Artifex's allegations regarding its dual-licensing scheme were deemed sufficient to plead damages for breach of contract. The denial of the motion to dismiss was widely misreported and sensationalized as a ruling that the GPL itself was "an enforceable contract."
In September the court [denied][78] Hancom's motion for summary judgment on the breach of contract claim. Hancom first argued that as a matter of law Artifex was not entitled to money damages, essentially because GPL compliance required no payment to Artifex. The court rejected this argument, as the value of a royalty-bearing license and an unjust enrichment theory could serve as the measure of Artifex's damages. Second, Hancom argued in essence that any damages for contract breach could not be based on continuing GPL-noncompliant activity after Hancom first began shipping Ghostscript in violation of the GPL, because at that moment Hancom's GPL license was automatically terminated. In rejecting this argument, the court noted that GPLv3's language suggested Hancom's GPL obligations persisted beyond the termination of its GPL rights. The parties reached a settlement in December.
Special thanks to Chris Gillespie for his research and analysis of the Artifex case.
### 10. SFLC/Conservancy trademark dispute
In 2006 the Software Freedom Law Center formed a [separate nonprofit organization][79], which it named the Software Freedom Conservancy. By July 2011, the two organizations no longer had any board members, officers, or employees in common, and SFLC ceased providing legal services to Conservancy. SFLC obtained a registration from the USPTO for the service mark SOFTWARE FREEDOM LAW CENTER in early 2011. In November 2011 Conservancy applied to register the mark SOFTWARE FREEDOM CONSERVANCY; the registration issued in September 2012. SFLC continues to be run by its founder Eben Moglen, while Conservancy is managed by former SFLC employees Karen Sandler and Bradley Kuhn. The two organizations are known to have opposing positions on a number of significant legal and policy matters (see, for example, my discussion of the [ZFS-on-Linux][12] issue last year).
In September 2017, SFLC filed a [petition][80] with the [Trademark Trial and Appeal Board][81] to cancel Conservancy's trademark registration under Section 14 of the Lanham Trademark Act of 1946, [15 U.S.C.][82][§][82][1064][82], claiming that Conservancy's mark is confusingly similar to SFLC's. In November, Conservancy submitted its [answer][83] listing its affirmative defenses, and in December Conservancy filed a [summary judgment motion][84] on those defenses. The TTAB in effect [denied the summary judgment motion][85] on the basis that the affirmative defenses in Conservancy's answer were insufficiently pleaded.
Moglen publicly [proposed a mutual release][86] of all claims "in return for an iron-clad agreement for mutual non-disparagement," including "a perpetual, royalty-free trademark license for Conservancy to keep and use its current name." [Conservancy responded][87] in a blog post that it could not "accept any settlement offer that includes a trademark license we don't need. Furthermore, any trademark license necessarily gives SFLC perpetual control over how we pursue our charitable mission."
SFLC [moved][88] for leave to amend its petition to add a second ground for cancellation, that Conservancy's trademark registration was obtained by fraud. Conservancy's [response][89] argues that the proposed amendment does not state a claim for fraud. Meanwhile, Conservancy has submitted [applications for trademarks][90] for "THE SOFTWARE CONSERVANCY."
--------------------------------------------------------------------------------
via: https://opensource.com/article/18/2/top-10-open-source-legal-stories-shook-2017
作者：[Richard Fontana][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://opensource.com/users/fontana
[1]:https://github.com/blog/2314-new-github-terms-of-service
[2]:https://web.archive.org/web/20170131092801/https:/help.github.com/articles/github-terms-of-service/
[3]:https://opensource.com/law/11/7/trouble-harmony-part-1
[4]:https://en.wikipedia.org/wiki/Moral_rights
[5]:https://www.mirbsd.org/permalinks/wlog-10_e20170301-tg.htm#e20170301-tg_wlog-10
[6]:https://joeyh.name/blog/entry/removing_everything_from_github/
[7]:https://joeyh.name/blog/entry/what_I_would_ask_my_lawyers_about_the_new_Github_TOS/
[8]:https://www.fsf.org/blogs/licensing/do-githubs-updated-terms-of-service-conflict-with-copyleft
[9]:https://help.github.com/articles/github-terms-of-service/
[10]:https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html#section4
[11]:http://gplv3.fsf.org/gpl-rationale-2006-01-16.html#SECTION00390000000000000000
[12]:https://opensource.com/article/17/1/yearbook-7-notable-legal-developments-2016
[13]:https://www.linuxfoundation.org/about/technical-advisory-board/
[14]:https://lkml.org/lkml/2017/10/16/122
[15]:https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/kernel-enforcement-statement.rst?h=v4.15
[16]:https://sfconservancy.org/copyleft-compliance/principles.html
[17]:http://kroah.com/log/blog/2017/10/16/linux-kernel-community-enforcement-statement-faq/
[18]:http://kroah.com/log/blog/2017/10/16/linux-kernel-community-enforcement-statement/
[19]:https://www.redhat.com/en/about/press-releases/technology-industry-leaders-join-forces-increase-predictability-open-source-licensing
[20]:https://www.redhat.com/en/about/gplv3-enforcement-statement
[21]:https://lwn.net/Articles/516735/
[22]:https://www.eclipse.org/legal/epl-v10.html
[23]:https://opensource.org/licenses/cpl1.0.php
[24]:https://opensource.org/licenses/IPL-1.0
[25]:https://clojure.org/
[26]:https://www.opendaylight.org/
[27]:https://www.eclipse.org/org/press-release/20170829eplv2.php
[28]:https://www.gnu.org/licenses/license-list.en.html#EPL
[29]:http://www.eclipse.org/legal/eplfaq.php#GPLCOMPATIBLE
[30]:https://www.fsf.org/blogs/licensing/using-the-gpl-for-eclipse-plug-ins
[31]:https://mmilinkov.wordpress.com/2010/04/06/epl-gpl-commentary/
[32]:https://www.gnu.org/software/classpath/license.html
[33]:https://github.com/eclipse/omr/blob/master/LICENSE
[34]:https://github.com/eclipse/openj9/blob/master/LICENSE
[35]:https://www.gnu.org/licenses/license-list.en.html#EPL2
[36]:https://jcp.org/en/home/index
[37]:http://www.oracle.com/technetwork/java/javaee/overview/index.html
[38]:https://jcp.org/en/jsr/detail?id=348
[39]:https://blogs.oracle.com/theaquarium/opening-up-java-ee
[40]:https://blogs.oracle.com/theaquarium/opening-up-ee-update
[41]:https://projects.eclipse.org/projects/ee4j/charter
[42]:https://javaee.github.io/glassfish/
[43]:https://mmilinkov.wordpress.com/2018/01/23/ee4j-current-status-and-whats-next/
[44]:https://reactjs.org/
[45]:https://www.webmproject.org/license/additional/
[46]:https://github.com/dotnet/coreclr/blob/master/PATENTS.TXT
[47]:https://github.com/facebook/react/blob/v0.13.3/PATENTS
[48]:https://issues.apache.org/jira/browse/LEGAL-303
[49]:http://rocksdb.org/
[50]:https://issues.apache.org/jira/browse/LEGAL-303?focusedCommentId=16052957&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16052957
[51]:https://github.com/facebook/rocksdb/pull/2226
[52]:https://github.com/facebook/rocksdb/pull/2589
[53]:https://code.facebook.com/posts/300798627056246/relicensing-react-jest-flow-and-immutable-js/
[54]:https://github.com/facebook/osquery/pull/4007
[55]:https://github.com/facebook/react-native/commit/26684cf3adf4094eb6c405d345a75bf8c7c0bf88
[56]:https://opensource.com/article/17/9/facebook-patents-license
[57]:https://opensource.com/article/17/9/5-reasons-facebooks-react-license-was-mistake
[58]:https://www.openssl.org/source/license.html
[59]:https://people.gnome.org/~markmc/openssl-and-the-gpl.html
[60]:https://www.coreinfrastructure.org/
[61]:https://www.openssl.org/blog/blog/2015/08/01/cla/
[62]:https://www.coreinfrastructure.org/news/announcements/2017/03/openssl-re-licensing-apache-license-v-20-encourage-broader-use-other-foss
[63]:https://marc.info/?l=openbsd-tech&m=149028829020600&w=2
[64]:https://marc.info/?l=openbsd-tech&m=149032069130072&w=2
[65]:https://opensource.org/licenses/ISC
[66]:https://www.openssl.org/blog/blog/2017/06/17/code-removal/
[67]:https://grsecurity.net/
[68]:https://grsecurity.net/announce.php
[69]:https://grsecurity.net/passing_the_baton.php
[70]:https://web.archive.org/web/20170805231029/https:/grsecurity.net/agree/agreement.php
[71]:https://perens.com/2017/06/28/warning-grsecurity-potential-contributory-infringement-risk-for-customers/
[72]:https://www.courtlistener.com/docket/6132658/53/open-source-security-inc-v-perens/
[73]:https://en.wikipedia.org/wiki/Strategic_lawsuit_against_public_participation
[74]:https://www.ghostscript.com/
[75]:https://www.gnu.org/licenses/licenses.en.html
[76]:https://www.courtlistener.com/recap/gov.uscourts.cand.305835.1.0.pdf
[77]:https://ia801909.us.archive.org/13/items/gov.uscourts.cand.305835/gov.uscourts.cand.305835.32.0.pdf
[78]:https://ia801909.us.archive.org/13/items/gov.uscourts.cand.305835/gov.uscourts.cand.305835.54.0.pdf
[79]:https://www.softwarefreedom.org/news/2006/apr/03/conservancy-launch/
[80]:http://ttabvue.uspto.gov/ttabvue/v?pno=92066968&pty=CAN&eno=1
[81]:https://www.uspto.gov/trademarks-application-process/trademark-trial-and-appeal-board
[82]:https://www.law.cornell.edu/uscode/text/15/1064
[83]:http://ttabvue.uspto.gov/ttabvue/v?pno=92066968&pty=CAN&eno=5
[84]:http://ttabvue.uspto.gov/ttabvue/v?pno=92066968&pty=CAN&eno=6
[85]:http://ttabvue.uspto.gov/ttabvue/v?pno=92066968&pty=CAN&eno=8
[86]:https://www.softwarefreedom.org/blog/2017/dec/22/conservancy/
[87]:https://sfconservancy.org/blog/2017/dec/22/sflc-escalation/
[88]:http://ttabvue.uspto.gov/ttabvue/v?pno=92066968&pty=CAN&eno=7
[89]:http://ttabvue.uspto.gov/ttabvue/v?pno=92066968&pty=CAN&eno=9
[90]:http://tsdr.uspto.gov/documentviewer?caseId=sn87670034&docId=FTK20171106083425#docIndex=0&page=1

2
sources/tech/20090518 How to use yum-cron to automatically update RHEL-CentOS Linux.md
View File

@ -1,3 +1,5 @@
translating by shipsw
How to use yum-cron to automatically update RHEL/CentOS Linux
======
The yum command line tool is used to install and update software packages under RHEL / CentOS Linux server. I know how to apply updates using [yum update command line][1], but I would like to use cron to update packages where appropriate manually. How do I configure yum to install software patches/updates [automatically with cron][2]?

610
sources/tech/20140510 Managing Digital Files (e.g., Photographs) in Files and Folders.md Normal file
View File

@ -0,0 +1,610 @@
Managing Digital Files (e.g., Photographs) in Files and Folders
======
Update 2014-05-14: added real world example
Update 2015-03-16: filtering photographs according to their GPS coordinates
Update 2016-08-29: replaced outdated `show-sel.sh` method with new `filetags --filter` method
Update 2017-08-28: Email comment on geeqie video thumbnails
I am a passionate photographer when being on vacation or whenever I see something beautiful. This way, I collected many [JPEG][1] files over the past years. Here, I describe how I manage my digital photographs while avoiding any [vendor lock-in][2] which binds me to a temporary solution and leads to loss of data. Instead, I prefer solutions where I am able to **invest my time and effort for a long-term relationship**.
This (very long) entry is **not about image files only** : I am going to explain further things like my folder hierarchy, file name convention, and so forth. Therefore, this information applies to all kind of files I process.
Before I start explaining my method, we should come to an agreement whether or not we do have the same set of requirements I am trying to match with my method. If you are into [raw image formats][3], storing your photographs somewhere in the cloud or anything else very special to you (and not to me), you might not get satisfied with the things described here. Decide yourself.
### My requirements
For **getting the photographs (and movies) from my digital camera to my computer** , I just want to put the SD card into my computer and invoke the fetch-workflow. This thing also has to **pre-process the files** to meet my file name convention (described further down) and to rotate images that are in portrait orientation (and not in landscape).
Those files are written to my photography inbox folder `$HOME/tmp/digicam/`. In this folder, I want to **look through my image files and play movies** to **sort out/delete, rename, add/remove tags, and put sets of related files into separate destination folders**.
After that, I want to **navigate through my set of folders** containing the sets of image/movie files. In rare occasions, I want to **open an image file in an independent image processing tool** like [the GIMP][4]. Just for **rotating JPEG files** , I want to have a quick method which does not require an image processing tool and which is rotating JPEG images [in a loss-less way][5].
My digital camera has now support for tagging images with [GPS][6] coordinates. Therefore, I need a method to **visualize GPS coordinates for single files as well as for a set of files** showing the path I was walking.
There is another nice feature I want to use: imagine a beautiful vacation in Venice where you took hundreds of photographs. Each of them is so beautiful so that you do not want to delete some of them. On the other side, you might want to get a smaller set of photographs for presenting to your friends at home. And they are only expecting maybe two dozens of files before being too jealous. Therefore, I want to be able to **define and show a certain sub-set of photos**.
In terms of being independent and **avoid lock-in effects** , I do not want to use a tool I am not able to use when a company discontinues a product or service. For the very same reason and because I am a privacy-aware person, **I do not want to use any cloud-based service**. In order to keep myself open for new possibilities, I do not want to invest any effort in something which is only available on one specific operating system platform. **Basic stuff has to be available on any platform** (viewing, navigation, ...). But the **full set of requirements have to work on GNU/Linux** , in my case Debian GNU/Linux.
Before I describe my current solutions to this fairly large set of requirements mentioned above, I have to explain my general folder structure and file naming convention I also use for digital photographs. But first, there is an important fact you have to consider:
#### iPhoto, Picasa, and such considered harmful
Software tools which manage collections of photographs do provide pretty cool features. They offer a nice user interface and try to give you cozy work-flows for all kinds of requirements.
The big issue I do have got with them is numerous. They mostly use proprietary storage formats for almost everything: image files, meta-data, and so forth. This is a huge issue, when you are going to change to a different software in a couple of years. Trust me: **you are going to switch** , some day, in any case, for multiple reasons.
If you are in the position where you need to switch your tool, you are going to realize that iPhoto or Picasa do store original image files and everything you did to them separately. Rotation of images, adding description to image files, tags, cropping, and so forth: **everything will be lost forever** if you are not able to export it and re-import it to the new tool. Chances are very high that you are not going to do this without loss of information or data.
I do not want to invest any effort in a tool which locks away my work. **I refuse to lock-in myself to any proprietary tool.** Been there, done that. Learned my lessons.
This is the reason why I keep time-stamps, image descriptions, or tags in the file name itself. File names are permanent unless I manually change them. They do not get lost when I backup my photographs or when I copy them to USB memory sticks or other operating systems. Everybody is able to read them. Any future system is able to process them.
### My file name convention
All my files which do have a relation to a specific day or a time I start with a **date-stamp** or a **time-stamp** according to an adopted [ISO 8601][7].
Example file name with a date-stamp and two tags: `2014-05-09 Budget export for project 42 -- finance company.csv`
Example file name with a time-stamp (even including optional seconds) and two tags: `2014-05-09T22.19.58 Susan presenting her new shoes -- family clothing.jpg`
I have to use adopted ISO time-stamps because colons are not suitable for the Windows [file system NTFS][8]. Therefore, I replaced colons with dots for separating hours from minutes from the optional seconds.
In case of **time or date duration** , I separate the two date- or time-stamps with two minus signs: "`2014-05-09--2014-05-13 Jazz festival Graz -- folder tourism music.pdf`".
Time/date-stamps in file names have the advantage that they remain unchanged until I manually change them. Meta-data which is included in the file content itself (like [Exif][9]) tends to get lost when files are processed via tools that do not take care of those meta-data. Additionally, starting a file name with such a date/time-stamp ensures that files are displayed in a temporal order instead of alphabetic order. The alphabet is a [totally artificial sort order][10] and it is typically less practical for locating files by the user.
When I want to associate **tags** to a file name, I place them between the original file name and the [file name extension][11] separated by a space, two minus signs and an additional space: "`--`". My tags are lower case English words which do not contain spaces or special characters. Sometimes, I might use concatenated words like `quantifiedself` or `usergenerated`. I [tend to prefer general categories][12] instead of more (too) more specific describing tags. I re-use my tags on Twitter [hashtags][13], file names, folder names, bookmarks, blog entries like this one, and so forth.
Tags as part of the file name have several advantages. You are able to locate files with the help of tags by using your usual desktop search engine. Tags in file-names can not be lost because of copying on different storage media. This usually happens, whenever a system uses a different storage place than the file name: meta-data data-base, [dot-files][14], [alternate data streams][15], and so forth.
Of course, please do **avoid special characters** , umlauts, colons, and so forth in file and folder names in general. Especially when you synchronize files between different operating system platforms.
My **file name convention for folders** is the same as for files.
Note: Because of the clever [filenametimestamps][16]-module of [Memacs][17], all files and folders with a date/time-stamp appear on the very same time/day on my Org-mode calendar (agenda). This way, I get a very cool overview on what happened when on which day including all photographs I took.
### My general folder structure
In this section, I will describe my most important folders within my home folder. NOTE: this might get moved to an independent page somewhere in the future. Or not. Time will tell. :-)
Lots of stuff is only of interest for a certain period of time. These are things like downloads to quickly skim through its content, unpack ZIP files to examine the files contained, minor interesting stuff, and so forth. For **temporary stuff** I do have the `$HOME/tmp/` sub-hierarchy. New photographs are placed in `$HOME/tmp/digicam/`. Stuff I temporary copy from a CD, DVD, or USB memory stick are put in `$HOME/tmp/fromcd/`. Whenever a software tool needs temporary data within my user folder hierarchy, I use `$HOME/tmp/Tools/` as an starting point. A very frequent folder for me is `$HOME/tmp/2del/`: "2del" means "ready for being deleted any time". All my browser are using this folder as the default download folder for example. In case I need to free space on a machine, I firstly look at this `2del`-folder for stuff to delete.
In contrast to the temporary stuff described above, I certainly want to keep files **for a longer period of time** as well. Those files gets moved to my `$HOME/archive/` sub-hierarchy. Its got several sub-folders for backups, web/download-stuff I want to keep, binary files I want to archive, index files of removable media (CD, DVD, memory sticks, external hard drives), and a folder where I place stuff I want to archive (and look for a decent destination folder) in the near future. Sometimes, I am too busy or impatient for the moment to file things properly. Yes, that's me I even have a "do not bug me now"-folder. Is this weird to you? :-)
The most important sub-hierarchy within my archive is `$HOME/archive/events_memories/` and its sub/folders `2014/`, `2013/`, `2012/`, and so forth. As you might have guessed already, there is one **sub-folder per year**. Within each of them, there are single files and folders. The files are named according to my file name convention described in the previous section. Folder names start with an [ISO 8601][7] datestamp in the form "YYYY-MM-DD" followed by a hopefully descriptive name like `$HOME/archive/events_memories/2014/2014-05-08 Business marathon with colleagues/`. Within those date-related folders I keep all kinds of files which are related to a certain event: photographs, (scanned) PDF-files, text files, and so forth.
For **sharing data** , I maintain a `$HOME/share/` sub-hierarchy. There is my Dropbox folder, folders for important people I share data using all kinds of methods (like [unison][18]). I also share data among my set of devices: Mac Mini at home, GNU/Linux notebook at home, Android phone, root-server (my personal cloud), Windows-notebook at work. I don't want to elaborate on my synchronization set-up here. There might be another blog entry for this if you ask nicely. :-)
Within my `$HOME/templates_labels/` sub-hierarchy, I keep all kinds of **template files** ([LaTeX][19], scripts, ...), cliparts and **logos** , and so forth.
My **Org-mode** files, I mostly keep within `$HOME/org/`. I practice retentiveness and do not explain how much I love [Emacs/Org-mode][20] and how much I get out of it this time. You probably have read or heard me elaborating the awesome things I do with it. Just look out for [my `emacs` tag][21] on my blog and its [hashtag `#orgmode`][22] on twitter.
So far about my most important folder sub-hierarchies.
### My workflows
Tataaaa, after you learned about my folder structure and file name convention, here are my current workflows and tools I use for the requirements I described further up.
Please note that **you have to know, what you are doing**. My examples here contain folder paths and more that **only works on my machine or my set-up**. **You have to adopt stuff** like paths, file names, and so forth to meet your requirements!
#### Workflow: Moving files from my SD card to the laptop, rotating portrait images, and renaming files
When I want to move data from my digital camera to my GNU/Linux notebook, I take out its Mini-SD storage card and put it in my notebook. Then it gets mounted on `/media/digicam` automatically.
Then, I invoke [getdigicamdata.sh][23] which does several things: it moves the files from the SD card to a temporary folder for processing. The original file names are being converted to lower-case characters. All portrait photographs are rotated using [jhead][24]. Also with jhead, I generate file-name time-stamps from the Exif header time-stamps. Using [date2name][25] I add time-stamps also to the movie files. After processing all those files, they get moved to the destination folder for new digicam files: $HOME/tmp/digicam/tmp/~.
#### Workflow: Folder navigation, viewing, renaming, deleting image files
For skimming through my image and movie files, I prefer to use [geeqie][26] on GNU/Linux. It is a fairly lightweight image browser which has one big advantage other file browsers are missing: I can add external scripts/tools that can be invoked by a keyboard shortcut. This way, I am able to extend the feature-set of the image browser by arbitrary external commands.
Basic image management functionality is built-in to geeqie: navigating my folder hierarchy, viewing image files in window-mode and in full-screen more (shortcut `f`), renaming file names, deleting files, showing Exif meta-data (shortcut `Ctrl-e`).
On OS X, I use [Xee][27]. Unlike geeqie, it is not extendable by external commands. However, the basic navigation, viewing, and renaming functions are available as well.
#### Workflow: Adding and removing tags
I created a Python script called [filetags][28] which I use for adding and removing tags to single files as well as a set of files.
For digital photographs, I use tags like, e.g., `specialL` for landscape images that I consider suitable for desktop backgrounds and so forth, `specialP` for portrait photographs I would like to show to others, `sel` for a selection, and many more.
##### Initial set-up of filetags with geeqie
Adding filetags to geeqie is a manual step: `Edit > Preferences > Configure Editors ...`. Then create an additional entry with `New`. There, you can define a new desktop-file which looks like this:
add-tags.desktop
```
[Desktop Entry]
Name=filetags
GenericName=filetags
Comment=
Exec=/home/vk/src/misc/vk-filetags-interactive-adding-wrapper-with-gnome-terminal.sh %F
Icon=
Terminal=true
Type=Application
Categories=Application;Graphics;
hidden=false
MimeType=image/*;video/*;image/mpo;image/thm
Categories=X-Geeqie;
```
The wrapper-script `vk-filetags-interactive-adding-wrapper-with-gnome-terminal.sh` is necessary because I want a new terminal window to pop-up in order to add tags to my files:
vk-filetags-interactive-adding-wrapper-with-gnome-terminal.sh
```
#!/bin/sh
/usr/bin/gnome-terminal \
--geometry=85x15+330+5 \
--tab-with-profile=big \
--hide-menubar \
-x /home/vk/src/filetags/filetags.py --interactive "${@}"
#end
```
In geeqie, you can add a keyboard shortcut in `Edit > Preferences > Preferences ... > Keyboard`. I associated `t` with the `filetags` command.
The filetags script is also able to remove tags from a single file or a set of files. It basically uses the same method as described above. The only difference is the additional `--remove` parameter for the filetags script:
remove-tags.desktop
```
[Desktop Entry]
Name=filetags-remove
GenericName=filetags-remove
Comment=
Exec=/home/vk/src/misc/vk-filetags-interactive-removing-wrapper-with-gnome-terminal.sh %F
Icon=
Terminal=true
Type=Application
Categories=Application;Graphics;
hidden=false
MimeType=image/*;video/*;image/mpo;image/thm
Categories=X-Geeqie;
```
vk-filetags-interactive-removing-wrapper-with-gnome-terminal.sh
```
#!/bin/sh
/usr/bin/gnome-terminal \
--geometry=85x15+330+5 \
--tab-with-profile=big \
--hide-menubar \
-x /home/vk/src/filetags/filetags.py --interactive --remove "${@}"
#end
```
For removing tags, I created a keyboard shortcut for `T`.
##### Using filetags within geeqie
When I skim though image files in the geeqie file browser, I select files I want to tag (one to many) and press `t`. Then, a small window pops up and asks me for one or more tags. After confirming with `Return`, these tags gets added to the file names.
The same goes for removing tags: selecting multiple files, pressing `T`, entering tags to be removed, and confirming with `Return`. That's it. There is [almost no simpler way to add or remove tags to files][29].
#### Workflow: Advanced file renaming with appendfilename
##### Without appendfilename
Renaming a large set of files can be a tedious process. With original file names like `2014-04-20T17.09.11_p1100386.jpg`, the process to add a description to its file name is quite annoying. You are going to press `Ctrl-r` (rename) in geeqie which opens the file rename dialog. The base-name (file-name without the file extension) is marked by default. So if you do not want to delete/overwrite the file name (but append to it), you have to press the cursor key for `<right>`. Then, the cursor is placed between the base name and the extension. Type in your description (don't forget the initial space character) and confirm with `Return`.
##### Using appendfilename with geeqie
With [appendfilename][30], my process is simplified to gain maximum user experience for appending text to file names: When I press `a` (append) in geeqie, a dialog window pops up, asking for a text. After confirming with `Return`, the entered text gets placed between the time-stamp and the optional tags.
For example when I press `a` on `2014-04-20T17.09.11_p1100386.jpg` and I type `Pick-nick in Graz`, the file name gets changed to `2014-04-20T17.09.11_p1100386 Pick-nick in Graz.jpg`. When I press `a` once again and enter `with Susan`, the file name gets changed to `2014-04-20T17.09.11_p1100386 Pick-nick in Graz with Susan.jpg`. When the file name got tags as well, the appended text gets appended before the tag-separator.
This way, I do not have to be afraid to overwrite time-stamps or tags. The process for renaming gets much more enjoyable for me!
And the best part: when I want to add the same text to multiple selected files, this also works with appendfilename.
##### Initial set-up of appendfilename with geeqie
Add an additional editor to geeqie: `Edit > Preferences > Configure Editors ... > New`. Then enter the desktop file definition:
appendfilename.desktop
```
[Desktop Entry]
Name=appendfilename
GenericName=appendfilename
Comment=
Exec=/home/vk/src/misc/vk-appendfilename-interactive-wrapper-with-gnome-terminal.sh %F
Icon=
Terminal=true
Type=Application
Categories=Application;Graphics;
hidden=false
MimeType=image/*;video/*;image/mpo;image/thm
Categories=X-Geeqie;
```
Once again, I do use a wrapper-script that provides me the terminal window:
vk-appendfilename-interactive-wrapper-with-gnome-terminal.sh
```
#!/bin/sh
/usr/bin/gnome-terminal \
--geometry=90x5+330+5 \
--tab-with-profile=big \
--hide-menubar \
-x /home/vk/src/appendfilename/appendfilename.py "${@}"
#end
```
#### Workflow: Play movie files
On GNU/Linux, I use [mplayer][31] to play-back video files. Since geeqie does not play movie files on itself, I have to create a set-up where I can open a movie file in mplayer.
##### Initial set-up of mplayer with geeqie
I did already associate movie file extensions to mplayer using [xdg-open][32]. Therefore, I only had to create a general "open" command to geeqie which uses xdg-open to open any file with its associated application.
Once again, visit `Edit > Preferences > Configure Editors ...` in geeqie and add an entry for `open`:
open.desktop
```
[Desktop Entry]
Name=open
GenericName=open
Comment=
Exec=/usr/bin/xdg-open %F
Icon=
Terminal=true
Type=Application
hidden=false
NOMimeType=*;
MimeType=image/*;video/*
Categories=X-Geeqie;
```
When you also associate the shortcut `o` (see above) to geeqie, you are able to open video files (and other files) with their associated application.
##### Opening movie files (and others) with xdg-open
After the set-up process from above, you just have to press `o` when your geeqie cursor is above the file. That's it.
#### Workflow: Open in an external image editor
I rarely want to be able to quickly edit image files in the GIMP. Therefore, I added a shortcut `g` and associated it with the external editor "GNU Image Manipulation Program" (GIMP) which was already created by default by geeqie.
This way, only pressing `g` opens the current image file in the GIMP.
#### Workflow: Move to archive folder
Now that I have added comments to my file names, I want to move single files to `$HOME/archive/events_memories/2014/` or set of files to new folders within this folder like `$HOME/archive/events_memories/2014/2014-05-08 Business-Marathon After-Show-Party`.
The usual way is to select one or multiple files and move them to a folder with the shortcut `Ctrl-m`.
So booooring.
Therefore, I (again) wrote a Python script which does this job for me: [move2archive][33] (in short: `m2a`) expects one or more files as command line parameters. Then, a dialog appears where I am able to enter an optional folder name. When I do not enter anything at all but press `Return`, the files gets moved to the folder of the corresponding year. When I enter a folder name like `Business-Marathon After-Show-Party`, the date-stamp of the first image file is appended to the folder (`$HOME/archive/events_memories/2014/2014-05-08 Business-Marathon After-Show-Party`), the resulting folder gets created, and the files gets moved.
Once again: I am in geeqie, select one or more files, press `m` (move) and either press only `Return` (no special sub-folder) or enter a descriptive text which is the name of the sub-folder to be created (optionally without date-stamp).
**No image managing tool is as quick and as fun to use as my geeqie with appendfilename and move2archive via shotcuts.**
##### Initial set-up of m2a with geeqie
Once again, adding `m2a` to geeqie is a manual step: `Edit > Preferences > Configure Editors ...`. Then create an additional entry with `New`. There, you can define a new desktop-file which looks like this:
m2a.desktop
```
[Desktop Entry]
Name=move2archive
GenericName=move2archive
Comment=Moving one or more files to my archive folder
Exec=/home/vk/src/misc/vk-m2a-interactive-wrapper-with-gnome-terminal.sh %F
Icon=
Terminal=true
Type=Application
Categories=Application;Graphics;
hidden=false
MimeType=image/*;video/*;image/mpo;image/thm
Categories=X-Geeqie;
```
The wrapper-script `vk-m2a-interactive-wrapper-with-gnome-terminal.sh` is necessary because I want a new terminal window to pop-up in order to enter my desired destination folder for my files:
vk-m2a-interactive-wrapper-with-gnome-terminal.sh
```
#!/bin/sh
/usr/bin/gnome-terminal \
--geometry=157x56+330+5 \
--tab-with-profile=big \
--hide-menubar \
-x /home/vk/src/m2a/m2a.py --pauseonexit "${@}"
#end
```
In geeqie, you can add a keyboard shortcut in `Edit > Preferences > Preferences ... > Keyboard`. I associated `m` with the `m2a` command.
#### Workflow: Rotate images (loss-less)
Usually, portrait photographs are being marked automatically as portrait photographs by my digital camera. However, there are certain situations (like taking a photograph from above the motif) where my camera gets it wrong. In those **rare cases** , I have to manually fix the orientation.
You have to know that the JPEG file format is a lossy format which should be used only for photographs and not for computer-generated stuff like screen-shots or diagrams. Rotating a JPEG image file in the dumb way usually results in decompressing/visualizing the image file, rotating the resulting image, and re-encoding the result once again. This causes a resulting image with [much worse image quality than the original image][5].
Therefore, you should use a lossless method to rotate you JPEG image files.
Once again, I add an "external editor" to geeqie: `Edit > Preferences > Configure Editors ... > New`. There, I add two entries: one for rotating 270 degrees (which is 90 degrees counter-clock-wise) and one for rotating 90 degrees (clock-wise) using [exiftran][34]:
rotate-270.desktop
```
[Desktop Entry]
Version=1.0
Type=Application
Name=Losslessly rotate JPEG image counterclockwise
# call the helper script
TryExec=exiftran
Exec=exiftran -p -2 -i -g %f
# Desktop files that are usable only in Geeqie should be marked like this:
Categories=X-Geeqie;
OnlyShowIn=X-Geeqie;
# Show in menu "Edit/Orientation"
X-Geeqie-Menu-Path=EditMenu/OrientationMenu
MimeType=image/jpeg;
```
rotate-90.desktop
```
[Desktop Entry]
Version=1.0
Type=Application
Name=Losslessly rotate JPEG image clockwise
# call the helper script
TryExec=exiftran
Exec=exiftran -p -9 -i -g %f
# Desktop files that are usable only in Geeqie should be marked like this:
Categories=X-Geeqie;
OnlyShowIn=X-Geeqie;
# Show in menu "Edit/Orientation"
X-Geeqie-Menu-Path=EditMenu/OrientationMenu
# It can be made verbose
# X-Geeqie-Verbose=true
MimeType=image/jpeg;
```
I created geeqie keyboard shortcuts for `[` (counter-clock-wise) and `]` (clock-wise).
#### Workflow: Visualizing GPS coordinates
My digital camera has a GPS sensor which stores the current geographic location within the Exif meta-data of the JPEG files. The location data gets stored in [WGS 84][35] format like "47, 58, 26.73; 16, 23, 55.51" (latitude; longitude). This is not human-readable in the sense I would expect: either a map or a location name. Therefore, I added functionality to geeqie so, that I am able to see the location of a single image file on [OpenStreetMap][36]: `Edit > Preferences > Configure Editors ... > New`
photolocation.desktop
```
[Desktop Entry]
Name=vkphotolocation
GenericName=vkphotolocation
Comment=
Exec=/home/vk/src/misc/vkphotolocation.sh %F
Icon=
Terminal=true
Type=Application
Categories=Application;Graphics;
hidden=false
MimeType=image/bmp;image/gif;image/jpeg;image/jpg;image/pjpeg;image/png;image/tiff;image/x-bmp;image/x-gray;image/x-icb;image/x-ico;image/x-png;image/x-portable-anymap;image/x-portable-bitmap;image/x-portable-graymap;image/x-portable-pixmap;image/x-xbitmap;image/x-xpixmap;image/x-pcx;image/svg+xml;image/svg+xml-compressed;image/vnd.wap.wbmp;
```
This calls my wrapper-script named `vkphotolocation.sh` which uses [ExifTool][37] to extract the coordinates in a suitable format that [Marble][38] is able to read and visualize:
vkphotolocation.sh
```
#!/bin/sh
IMAGEFILE="${1}"
IMAGEFILEBASENAME=`basename ${IMAGEFILE}`
COORDINATES=`exiftool -c %.6f "${IMAGEFILE}" | awk '/GPS Position/ { print $4 " " $6 }'`
if [ "x${COORDINATES}" = "x" ]; then
zenity --info --title="${IMAGEFILEBASENAME}" --text="No GPS-location found in the image file."
else
/usr/bin/marble --latlon "${COORDINATES}" --distance 0.5
fi
#end
```
Mapped to the keyboard shortcut `G`, I can quickly get to the **map position of its location of a single image file**.
When I want to visualize the **positions of multiple JPEG image files as a path** , I am using [GpsPrune][39]. I was not able to derive a method where GpsPrune takes a set of files as command line parameters. And because of this, I have to manually start GpsPrune, select a set of files or a folder with `File > Add photos`.
This way, I get a dot for each JPEG location on a map of OpenStreetMap (if configured so). By clicking on such a dot, I get details of the corresponding image.
If you happen to be abroad while taking photographs, visualizing the GPS positions is a **great help for adding descriptions** to the file name!
#### Workflow: Filtering photographs according to their GPS coordinates
This is no workflow of mine. For the sake of completeness, I list features of tools that make this workflow possible. What I would like to do is looking for only those photographs out of a big pile of images, that are within a certain area (rectangle or point + distance).
So far, I found only [DigiKam][40] which is able to [filter according to a rectangle][41]. If you know another tool, please add it to the comments below or write an email.
#### Workflow: Showing a sub-set of a given set
As described in the requirements above, I want to be able to define a sub-set of files within a folder in order to present this small collection to other people.
The work-flow is pretty simple: I add a tag (via `t`/filetags) to the files of the selection. For this, I use the tag `sel` which is short for "selection". After I tagged the set of files, I can press `s` which I associated with a script that shows only the files tagged with `sel`.
Of course, this also works with any tag or tag combination. Therefore, with the same method, you are able to get a decent overview on all photos of your wedding that are tagged with "church" and "rings".
Nifty feature, isn't it? :-)
##### Initial set-up of filetags for filtering according to tags + geeqie
You have to define an additional "external editor": `Edit > Preferences > Configure Editors ... > New`:
filter-tags.desktop
```
[Desktop Entry]
Name=filetag-filter
GenericName=filetag-filter
Comment=
Exec=/home/vk/src/misc/vk-filetag-filter-wrapper-with-gnome-terminal.sh
Icon=
Terminal=true
Type=Application
Categories=Application;Graphics;
hidden=false
MimeType=image/*;video/*;image/mpo;image/thm
Categories=X-Geeqie;
```
This once again calls a wrapper-script I wrote:
vk-filetag-filter-wrapper-with-gnome-terminal.sh
```
#!/bin/sh
/usr/bin/gnome-terminal \
--geometry=85x15+330+5 \
--hide-menubar \
-x /home/vk/src/filetags/filetags.py --filter
#end
```
What `filetags` with parameter `--filter` does is basically the following: the user gets asked to enter one or more tags. Then, all matching files of the current folder are linked to `$HOME/.filetags_tagfilter/` using [symbolic links][42]. Then, a new geeqie instance is started which shows the linked files.
After quitting this new geeqie instance, you see the old geeqie instance, from where you invoked the selection process.
#### Summary with a real world example
Wow, this was a very long blog entry. No wonder that you might have lost the overview here and there. To sum up the things I am able to do within geeqie (that extends the standard feature set), I have this cool table below:
shortcut function `m` m2a `o` open (for non-images) `a` add text to file name `t` filetags (add) `T` filetags (remove) `s` filetags (filter) `g` gimp `G` show GPS position `[` lossless rotate counterclockwise `]` lossless rotate clockwise `Ctrl-e` EXIF `f` full-screen
Parts of a file name (including its path) and tools I use to manipulate the components accordingly:
```
/this/is/a/folder/2014-04-20T17.09 Pick-nick in Graz -- food graz.jpg
[ m2a ] [ date2name ] [ appendfilename ] [filetags]
```
In practice, I do the following steps to get my photographs from the camera to my archive: I put the SD memory card into my SD card reader of my computer. Then I start [getdigicamdata.sh][23]. After it is finished, I open `$HOME/tmp/digicam/tmp/` within geeqie. I skim through the photographs and delete the ones that did not work out. If there is an image with the wrong orientation, I correct it by `[` or `]`.
In a second run, I add descriptions to files I consider worth commenting on (`a`). Whenever I want to add tags I do so as well: I quickly mark all files that should share a tag (`Ctrl` \+ mouse-click) and tag them using [filetags][28] (`t`).
To combine files from a given event, I select corresponding files and move them to their "event-folder" within the yearly archive folder by typing the event description in [move2archive][33] (`m`). The rest (no special event folder) is moved to the yearly archive directly by move2archive (`m`) without stating a event description.
To finish my work-flow, I delete all files on my SD card, dismount it from the operating system, and put it back into my digital camera.
That's it.
Because this work-flow requires almost no overhead at all, commenting, tagging, and filing photographs is not a tedious job any more.
### Finally
So, this is a detailed description of my work-flows related to photographs and movies I make. You probably have found additional stuff I might be interested in. So please do not hesitate and leave a comment or email by using the links below.
I also would like to get feed-back if my work-flow works for you as well. And: if you have published your work-flows or find descriptions of other peoples work-flows, please do leave a comment as well!
Have fun and don't waste your time with the wrong tools or inefficient methods!
### Other Tools
Read about [gThumb in this article][43].
Please do suggest your tools of choice when you've got the feeling that they fit my requirements mentioned above.
### Email Comments
> Date: Sat, 26 Aug 2017 22:05:09 +0200
> Hello Karl,
> I like your articles and work with memacs and of course orgmode but I am not very familiar with python by the way... in your blog post of "managing-digital-photographs" you write about to open videos with [Geeqie][26].
> It works, but I cant see any video thumbnails in the browser of Geeqie. Do you have any suggestions to get this to work?
> Thank you, Thomas
Hi Thomas,
Thanks for your kind words. I always feel great when somebody finds my work useful in his/her life. Unfortunately most of the time, I never hear of them.
Yes, I sometimes use Geeqie to visualize folders that not only contain image files but movie files as well. In those cases, I don't see any thumbnail image of the video. You're absolutely right, there are many file browsers out there which are able to display some kind of preview image of a video.
Quite frankly, I never thought of video thumbnails and I don't miss them. A quick research in the preferences and with my search engine did not suggest that there is a possible way to enable video previews in Geeqie. So no luck here.
--------------------------------------------------------------------------------
via: http://karl-voit.at/managing-digital-photographs/
作者：[Karl Voit][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]:http://karl-voit.at
[1]:https://en.wikipedia.org/wiki/Jpeg
[2]:http://en.wikipedia.org/wiki/Vendor_lock-in
[3]:https://en.wikipedia.org/wiki/Raw_image_format
[4]:http://www.gimp.org/
[5]:http://petapixel.com/2012/08/14/why-you-should-always-rotate-original-jpeg-photos-losslessly/
[6]:https://en.wikipedia.org/wiki/Gps
[7]:https://en.wikipedia.org/wiki/Iso_date
[8]:https://en.wikipedia.org/wiki/Ntfs
[9]:https://en.wikipedia.org/wiki/Exif
[10]:http://www.isisinform.com/reinventing-knowledge-the-medieval-controversy-of-alphabetical-order/
[11]:https://en.wikipedia.org/wiki/File_name_extension
[12]:http://karl-voit.at/tagstore/en/papers.shtml
[13]:https://en.wikipedia.org/wiki/Hashtag
[14]:https://en.wikipedia.org/wiki/Dot-file
[15]:https://en.wikipedia.org/wiki/NTFS#Alternate_data_streams_.28ADS.29
[16]:https://github.com/novoid/Memacs/blob/master/docs/memacs_filenametimestamps.org
[17]:https://github.com/novoid/Memacs
[18]:http://www.cis.upenn.edu/~bcpierce/unison/
[19]:https://github.com/novoid/LaTeX-KOMA-template
[20]:http://orgmode.org/
[21]:http://karl-voit.at/tags/emacs
[22]:https://twitter.com/search?q%3D%2523orgmode&src%3Dtypd
[23]:https://github.com/novoid/getdigicamdata.sh
[24]:http://www.sentex.net/%3Ccode%3Emwandel/jhead/
[25]:https://github.com/novoid/date2name
[26]:http://geeqie.sourceforge.net/
[27]:http://xee.c3.cx/
[28]:https://github.com/novoid/filetag
[29]:http://karl-voit.at/tagstore/
[30]:https://github.com/novoid/appendfilename
[31]:http://www.mplayerhq.hu
[32]:https://wiki.archlinux.org/index.php/xdg-open
[33]:https://github.com/novoid/move2archive
[34]:http://manpages.ubuntu.com/manpages/raring/man1/exiftran.1.html
[35]:https://en.wikipedia.org/wiki/WGS84#A_new_World_Geodetic_System:_WGS_84
[36]:http://www.openstreetmap.org/
[37]:http://www.sno.phy.queensu.ca/~phil/exiftool/
[38]:http://userbase.kde.org/Marble/Tracking
[39]:http://activityworkshop.net/software/gpsprune/
[40]:https://en.wikipedia.org/wiki/DigiKam
[41]:https://docs.kde.org/development/en/extragear-graphics/digikam/using-kapp.html#idp7659904
[42]:https://en.wikipedia.org/wiki/Symbolic_link
[43]:http://karl-voit.at/2017/02/19/gthumb

119
sources/tech/20171207 Concurrent Servers- Part 5 - Redis case study.md Normal file
View File

@ -0,0 +1,119 @@
Concurrent Servers: Part 5 - Redis case study
======
This is part 5 in a series of posts on writing concurrent network servers. After discussing techniques for constructing concurrent servers in parts 1-4, this time we're going to do a case study of an existing production-quality server - [Redis][10].
![Redis logo](https://eli.thegreenplace.net/images/2017/redis_logo.png)
Redis is a fascinating project and I've been following it with interest for a while now. One of the things I admire most about Redis is the clarity of its C source code. It also happens to be a great example of a high-performance concurrent in-memory database server, so the opportunity to use it as a case study for this series was too good to ignore.
Let's see how the ideas discussed in parts 1-4 apply to a real-world application.
All posts in the series:
* [Part 1 - Introduction][3]
* [Part 2 - Threads][4]
* [Part 3 - Event-driven][5]
* [Part 4 - libuv][6]
* [Part 5 - Redis case study][7]
### Event-handling library
One of Redis's main claims to fame around the time of its original release in 2009 was its speed - the sheer number of concurrent client connections the server could handle. It was especially notable that Redis did this all in a single thread, without any complex locking and synchronization schemes on the data stored in memory.
This feat was achieved by Redis's own implementation of an event-driven library which is wrapping the fastest event loop available on a system (epoll for Linux, kqueue for BSD and so on). This library is called [ae][11]. ae makes it possible to write a fast server as long as none of the internals are blocking, which Redis goes to great lengths to guarantee [[1]][12].
What mainly interests us here is ae's support of file events - registering callbacks to be invoked when file descriptors (like network sockets) have something interesting pending. Like libuv, ae supports multiple event loops and - having read parts 3 and 4 in this series - the signature of aeCreateFileEvent shouldn't be surprising:
```
int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask,
aeFileProc *proc, void *clientData);
```
It registers a callback (proc) for new file events on fd, with the given event loop. When using epoll, it will call epoll_ctl to add an event on the file descriptor (either EPOLLIN, EPOLLOUT or both, depending on the mask parameter). ae's aeProcessEvents is the "run the event loop and dispatch callbacks" function, and it calls epoll_wait under the hood.
### Handling client requests
Let's trace through the Redis server code to see how ae is used to register callbacks for client events. initServer starts it by registering a callback for read events on the socket(s) being listened to, by calling aeCreateFileEvent with the callback acceptTcpHandler. This callback is invoked when new client connections are available. It calls accept [[2]][13] and then acceptCommonHandler, which in turn calls createClient to initialize the data structures required to track a new client connection.
createClient's job is to start listening for data coming in from the client. It sets the socket to non-blocking mode (a key ingredient in an asynchronous event loop) and registers another file event callback with aeCreateFileEvent - for read events - readQueryFromClient. This function will be invoked by the event loop every time the client sends some data.
readQueryFromClient does just what we'd expect - parses the client's command and acts on it by querying and/or manipulating data and sending a reply back. Since the client socket is non-blocking, this function has to be able to handle EAGAIN, as well as partial data; data read from the client is accumulated in a client-specific buffer, and the full query may be split across multiple invocations of the callback.
### Sending data back to clients
In the previous paragraph I said that readQueryFromClient ends up sending replies back to clients. This is logically true, because readQueryFromClient prepares the reply to be sent, but it doesn't actually do the physical sending - since there's no guarantee the client socket is ready for writing/sending data. We have to use the event loop machinery for that.
The way Redis does this is by registering a beforeSleep function to be called every time the event loop is about to go sleeping waiting for sockets to become available for reading/writing. One of the things beforeSleep does is call handleClientsWithPendingWrites. This function tries to send all available replies immediately by calling writeToClient; if some of the sockets are unavailable, it registers an event-loop callback to invoke sendReplyToClient when the socket is ready. This can be seen as a kind of optimization - if the socket is immediately ready for sending (which often is the case for TCP sockets), there's no need to register the event - just send the data. Since sockets are non-blocking, this never actually blocks the loop.
### Why does Redis roll its own event library?
In [part 4][14] we've discussed building asynchronous concurrent servers using libuv. It's interesting to ponder the fact that Redis doesn't use libuv, or any similar event library, and instead implements its own - ae, including wrappers for epoll, kqueue and select. In fact, antirez (Redis's creator) answered precisely this question [in a blog post in 2011][15]. The gist of his answer: ae is ~770 lines of code he intimately understands; libuv is huge, without providing additional functionality Redis needs.
Today, ae has grown to ~1300 lines, which is still trivial compared to libuv's 26K (this is without Windows, test, samples, docs). libuv is a far more general library, which makes it more complex and more difficult to adapt to the particular needs of another project; ae, on the other hand, was designed for Redis, co-evolved with Redis and contains only what Redis needs.
This is another great example of the dependencies in software projects formula I mentioned [in a post earlier this year][16]:
> The benefit of dependencies is inversely proportional to the amount of effort spent on a software project.
antirez referred to this, to some extent, in his post. He mentioned that dependencies that provide a lot of added value ("foundational" dependencies in my post) make more sense (jemalloc and Lua are his examples) than dependencies like libuv, whose functionality is fairly easy to implement for the particular needs of Redis.
### Multi-threading in Redis
[For the vast majority of its history][17], Redis has been a purely single-threaded affair. Some people find this surprising, but it makes total sense with a bit of thought. Redis is inherently network-bound - as long as the database size is reasonable, for any given client request, much more time is spent waiting on the network than inside Redis's data structures.
These days, however, things are not quite that simple. There are several new capabilities in Redis that use threads:
1. "Lazy" [freeing of memory][8].
2. Writing a [persistence journal][9] with fsync calls in a background thread.
3. Running user-defined modules that need to perform a long-running operation.
For the first two features, Redis uses its own simple bio library (the acronym stands for "Background I/O"). The library is hard-coded for Redis's needs and can't be used outside it - it runs a pre-set number of threads, one per background job type Redis needs.
For the third feature, [Redis modules][18] could define new Redis commands, and thus are held to the same standards as regular Redis commands, including not blocking the main thread. If a custom Redis command defined in a module wants to perform a long-running operation, it has to spin up a thread to run it in the background. src/modules/helloblock.c in the Redis tree provides an example.
With these features, Redis combines an event loop with threading to get both speed in the common case and flexibility in the general case, similarly to the work queue discussion in [part 4][19] of this series.
| [[1]][1] | A core aspect of Redis is its being an _in-memory_ database; therefore, queries should never take too long to execute. There are all kinds of complications, however. In case of partitioning, a server may end up routing the request to another instance; in this case async I/O is used to avoid blocking other clients. |
| [[2]][2] | Through anetAccept; anet is Redis's wrapper for TCP socket code. |
--------------------------------------------------------------------------------
via: https://eli.thegreenplace.net/2017/concurrent-servers-part-5-redis-case-study/
作者：[Eli Bendersky][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://eli.thegreenplace.net/pages/about
[1]:https://eli.thegreenplace.net/2017/concurrent-servers-part-5-redis-case-study/#id1
[2]:https://eli.thegreenplace.net/2017/concurrent-servers-part-5-redis-case-study/#id2
[3]:http://eli.thegreenplace.net/2017/concurrent-servers-part-1-introduction/
[4]:http://eli.thegreenplace.net/2017/concurrent-servers-part-2-threads/
[5]:http://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/
[6]:http://eli.thegreenplace.net/2017/concurrent-servers-part-4-libuv/
[7]:http://eli.thegreenplace.net/2017/concurrent-servers-part-5-redis-case-study/
[8]:http://antirez.com/news/93
[9]:https://redis.io/topics/persistence
[10]:https://redis.io/
[11]:https://redis.io/topics/internals-rediseventlib
[12]:https://eli.thegreenplace.net/2017/concurrent-servers-part-5-redis-case-study/#id4
[13]:https://eli.thegreenplace.net/2017/concurrent-servers-part-5-redis-case-study/#id5
[14]:http://eli.thegreenplace.net/2017/concurrent-servers-part-4-libuv/
[15]:http://oldblog.antirez.com/post/redis-win32-msft-patch.html
[16]:http://eli.thegreenplace.net/2017/benefits-of-dependencies-in-software-projects-as-a-function-of-effort/
[17]:http://antirez.com/news/93
[18]:https://redis.io/topics/modules-intro
[19]:http://eli.thegreenplace.net/2017/concurrent-servers-part-4-libuv/

3
sources/tech/20180130 tmux - A Powerful Terminal Multiplexer For Heavy Command-Line Linux User.md
View File

@ -1,3 +1,6 @@
erialin translating
tmux – A Powerful Terminal Multiplexer For Heavy Command-Line Linux User
======
tmux stands for terminal multiplexer, it allows users to create/enable multiple terminals (vertical & horizontal) in single window, this can be accessed and controlled easily from single window when you are working with different issues.

2
sources/tech/20180131 How to access-view Python help when using vim.md
View File

@ -1,3 +1,5 @@
translating---geekpi
How to access/view Python help when using vim
======

41
sources/tech/20180203 3 Ways to Extend the Power of Kubernetes.md
View File

@ -1,41 +0,0 @@
translating---geekpi
3 Ways to Extend the Power of Kubernetes
======
![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/chen-goldberg-kubecon.png?itok=WR_4i31u)
The ability to extend Kubernetes is its secret superpower, said Chen Goldberg, Director of Engineering at Google, speaking at the recent [KubeCon + CloudNativeCon][1] in Austin.
In the race to build tools that help engineers become more productive, Goldberg talked about how she once led a team that developed a platform that did just that. Despite the fact the platform initially worked, it was not extensible, and it was also difficult to modify.
Fortunately, said Goldberg, Kubernetes suffers from neither of these problems. To begin with, Kubernetes is self-healing system, as it uses controllers that implement what is called a " _Reconciliation Loop._ " In a reconciliation loop, a controller observes the current state of the system and compares it to its desired state. Once it has established the difference between each of these two states, it works towards achieving the desired state. This makes Kubernetes well-adapted to dynamic environments.
### 3 Ways to Extend Kubernetes
Goldberg then explained that to build the controllers, you need resources, that is, you need to extend Kubernetes. There are three ways to do that and, from the most flexible (but also more difficult) to the easiest they are: using a Kube aggregator, using an API server builder, or creating a Custom Resource Definition (or CRD).
The latter allows to extend Kubernetes' functionality even with minimal coding. To demonstrate how it is done, Goggle Software Engineer Anthony Yeh came on stage and showcased adding a stateful set to Kubernetes. (Stateful sets objects used to manage stateful applications, that is applications that need to store the state of the application, keeping track of, for example, a user's identity and their personal settings.) Using _catset_ , a CRD implemented in 100 lines of JavaScript in one single file, Yeh showed how you can add a stateful set to a Kubernetes deployment. A prior extension that was not a CRD, required 24 files and over 3,000 lines of code.
Addressing the issue of reliability of CRDs, Goldberg said Kubernetes had started a certification program that allows companies to register and certify their extensions for the Kubernetes community. In one month over 30 companies had signed up for the program.
Goldberg went on to explain how the extensibility of Kubernetes was a hot topic in this year's KubeCon, and how Google and IBM were building a platform to manage and secure microservices using CRDs. Or how some developers were bringing machine-learning to Kubernetes, and others were demonstrating open service broker and the consumption of services on hybrid settings.
In conclusion, Goldberg said, extensibility is about empowerment. And, the extensibility of Kubernetes makes it a general purpose and easy to use platform for developers, which allows them to run any application.
You can watch the entire video below:
https://www.youtube.com/embed/1kjgwXP_N7A?enablejsapi=1
--------------------------------------------------------------------------------
via: https://www.linux.com/blog/event/kubecon/2018/2/3-ways-extend-power-kubernetes
作者：[PAUL BROWN][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://www.linux.com/users/bro66
[1]:http://events17.linuxfoundation.org/events/kubecon-and-cloudnativecon-north-america

1
sources/tech/20180214 Protecting Code Integrity with PGP - Part 1- Basic Concepts and Tools.md
View File

@ -1,3 +1,4 @@
translating by kimii
Protecting Code Integrity with PGP — Part 1: Basic Concepts and Tools
======

4
sources/tech/20180224 How To Run A Command For A Specific Time In Linux.md
View File

@ -1,3 +1,7 @@
loujiaye Translating
How To Run A Command For A Specific Time In Linux
======
![](https://www.ostechnix.com/wp-content/uploads/2018/02/Run-A-Command-For-A-Specific-Time-In-Linux-1-720x340.png)

39
translated/tech/20180203 3 Ways to Extend the Power of Kubernetes.md Normal file
View File

@ -0,0 +1,39 @@
3种扩展 Kubernetes 能力的方式
======
![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/chen-goldberg-kubecon.png?itok=WR_4i31u)
Google 的工程总监 Chen Goldberg 在奥斯汀最近的[KubeCon 和 CloudNativeCon][1]上说，Kubernetes 的扩展能力是它的秘密武器。
在建立帮助工程师提高工作效率的工具的竞赛中，Goldberg 谈到他曾经领导过一个开发这样一个平台的团队。尽管平台最初有用，但它无法扩展，并且修改也很困难。
幸运的是，Goldberg 说，Kubernetes 没有这些问题。首先，Kubernetes 是一个自我修复系统，因为它使用的控制器实现了“_协调环_”（Reconciliation Loop）。在协调环中，控制器观察系统的当前状态并将其与所需状态进行比较。一旦它确定了这两个状态之间的差异，它就会努力实现所需的状态。这使得 Kubernetes 非常适合动态环境。
### 3种扩展 Kubernetes 的方式
Goldberg 然后解释说，要建立控制器，你需要资源，也就是说，你需要扩展 Kubernetes。有三种方法可以做到这一点，从最灵活（但也更困难）到最简单的是：使用 Kube 聚合器，使用 API​​ 服务器构建器或创建自定义资源定义（或 CRD）。
后者允许即使使用最少的编码来扩展 Kubernetes 的功能。为了演示它是如何完成的，Goggle 软件工程师 Anthony Yeh 出席并展示了为 Kubernetes 添加一个状态集。 （状态集对象用于管理有状态应用，即需要存储应用状态的程序，跟踪例如用户身份及其个人设置。）使用 _catset_，在一个 100 行 JavaScript 的文件中实现的 CRD，Yeh 展示了如何将状态集添加到 Kubernetes 部署中。之前的扩展不是 CRD，需要 24 个文件和 3000 多行代码。
为解决 CRD 可靠性问题，Goldberg 表示，Kubernetes 已经启动了一项认证计划，允许公司在 Kubernetes 社区注册和认证其扩展。在一个月内，已有 30 多家公司报名参加该计划。
Goldberg 继续解释 Kubernetes 的可扩展性如何成为今年 KubeCon 的热门话题，以及 Google 和 IBM 如何构建一个使用 CRD 管理和保护微服务的平台。或者一些开发人员如何将机器学习带入 Kubernetes，另外展示开放服务代理以及在混合设置上的服务消费。
Goldberg 总结说，可扩展性是种增能。而且，Kubernetes 的可扩展性使其成为开发者的通用平台，并且易于使用，这使得他们可以运行任何应用程序。
你可以在下面观看整个视频：
https://www.youtube.com/embed/1kjgwXP_N7A?enablejsapi=1
--------------------------------------------------------------------------------
via: https://www.linux.com/blog/event/kubecon/2018/2/3-ways-extend-power-kubernetes
作者：[PAUL BROWN][a]
译者：[geekpi](https://github.com/geekpi)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://www.linux.com/users/bro66
[1]:http://events17.linuxfoundation.org/events/kubecon-and-cloudnativecon-north-america