mirror of
https://github.com/LCTT/TranslateProject.git
synced 2024-12-26 21:30:55 +08:00
zEpoch
GitHub
commit
6ec33f9dbf
@ -2,15 +2,17 @@
|
||||
[#]: via: "https://jvns.ca/blog/2021/05/17/how-to-look-at-the-stack-in-gdb/"
|
||||
[#]: author: "Julia Evans https://jvns.ca/"
|
||||
[#]: collector: "lujun9972"
|
||||
[#]: translator: " "
|
||||
[#]: reviewer: " "
|
||||
[#]: publisher: " "
|
||||
[#]: url: " "
|
||||
[#]: translator: "amwps290"
|
||||
[#]: reviewer: "wxy"
|
||||
[#]: publisher: "wxy"
|
||||
[#]: url: "https://linux.cn/article-13550-1.html"
|
||||
|
||||
使用 GDB 查看程序中栈空间
|
||||
使用 GDB 查看程序的栈空间
|
||||
======
|
||||
|
||||
昨天我和一些人在闲聊的时候，他们说他们并不了解栈是如何工作的，而且也不知道如何去查看栈空间。
|
||||
|
||||
|
||||
昨天我和一些人在闲聊的时候，他们说他们并不真正了解栈是如何工作的，而且也不知道如何去查看栈空间。
|
||||
|
||||
这是一个快速教程，介绍如何使用 GDB 查看 C 程序的栈空间。我认为这对于 Rust 程序来说也是相似的。但我这里仍然使用 C 语言，因为我发现用它更简单，而且用 C 语言也更容易写出错误的程序。
|
||||
|
||||
@ -45,19 +47,19 @@ int main() {
|
||||
|
||||
我们使用 `gcc -g -O0 test.c -o test` 命令来编译这个程序。
|
||||
|
||||
`-g` 选项会在编译程序中将调式信息也编译进去。这将会使我们查看我们的变量更加容易。
|
||||
`-g` 选项会在编译程序中将调式信息也编译进去。这将会使我们查看我们的变量更加容易。
|
||||
|
||||
`-O0` 选项告诉 gcc 不要进行优化，我要确保我们的 `x` 变量不会被优化掉。
|
||||
|
||||
### 第一步：启动 GDB
|
||||
|
||||
像这样启动 GDB
|
||||
像这样启动 GDB：
|
||||
|
||||
```
|
||||
$ gdb ./test
|
||||
```
|
||||
|
||||
它打印出一些 GPL 东西，然后给出一个提示符。让我们在 `main` 函数这里设置一个断点
|
||||
它打印出一些 GPL 信息，然后给出一个提示符。让我们在 `main` 函数这里设置一个断点：
|
||||
|
||||
```
|
||||
(gdb) b main
|
||||
@ -81,8 +83,7 @@ Breakpoint 1, main () at test.c:4
|
||||
|
||||
### 第二步：查看我们变量的地址
|
||||
|
||||
让我们从了解我们的变量开始。 它们每个都在内存中有一个地址，我们可以像这样打印出来：
|
||||
|
||||
让我们从了解我们的变量开始。它们每个都在内存中有一个地址，我们可以像这样打印出来：
|
||||
|
||||
```
|
||||
(gdb) p &x
|
||||
@ -93,13 +94,13 @@ $2 = (char **) 0x7fffffffe280
|
||||
$4 = (char (*)[10]) 0x7fffffffe28e
|
||||
```
|
||||
|
||||
因此，如果我们查看栈的地址，那我们应该能够看到所有的这些变量！
|
||||
因此，如果我们查看那些地址的堆栈，那我们应该能够看到所有的这些变量！
|
||||
|
||||
### 概念：栈指针
|
||||
|
||||
我们将需要使用栈指针，因此我将尽力对其进行快速解释。
|
||||
|
||||
有一个名为 ESP 的 x86 寄存器，称为“栈指针”。 基本上，它是当前函数的栈起始地址。 在 GDB 中，您可以使用 `$sp` 来访问它。 当您调用新函数或从函数返回时，栈指针的值会更改。
|
||||
有一个名为 ESP 的 x86 寄存器，称为“<ruby>栈指针<rt>stack pointer</rt></ruby>”。 基本上，它是当前函数的栈起始地址。 在 GDB 中，你可以使用 `$sp` 来访问它。 当你调用新函数或从函数返回时，栈指针的值会更改。
|
||||
|
||||
### 第三步：在 `main` 函数开始的时候，我们查看一下在栈上的变量
|
||||
|
||||
@ -112,7 +113,7 @@ $7 = (void *) 0x7fffffffe270
|
||||
|
||||
因此，我们当前函数的栈起始地址是 `0x7fffffffe270`，酷极了。
|
||||
|
||||
现在，让我们使用 GDB 打印出当前函数堆栈开始后的前 40 个字（即160个字节）。 某些内存可能不是栈的一部分，因为我不太确定这里的堆栈有多大。 但是至少开始的地方是栈的一部分。
|
||||
现在，让我们使用 GDB 打印出当前函数堆栈开始后的前 40 个字（即 160 个字节）。 某些内存可能不是栈的一部分，因为我不太确定这里的堆栈有多大。 但是至少开始的地方是栈的一部分。
|
||||
|
||||
```
|
||||
(gdb) x/40x $sp
|
||||
@ -128,19 +129,17 @@ $7 = (void *) 0x7fffffffe270
|
||||
0x7fffffffe300: 0xf9ce816d 0x7533d7c8 0xa91a816d 0x7533c789
|
||||
```
|
||||
|
||||
我已粗体显示了 stack_string，heap_string 和 x 变量的位置，并改变了颜色(译者注：可以在原网页查看)：
|
||||
我已粗体显示了 `stack_string`，`heap_string` 和 `x` 变量的位置，并改变了颜色：
|
||||
|
||||
* `x` 是红色字体，并且起始地址是 `0x7fffffffe27c`
|
||||
* `heap_string` 是蓝色字体，起始地址是 `0x7fffffffe280`
|
||||
* `stack_string` 是紫色字体，起始地址是 `0x7fffffffe28e`
|
||||
|
||||
我想在此处将一些变量的位置加粗了一点点可能会有点偏差，但这大概是它们所在的位置。
|
||||
|
||||
您可能会在这里注意到的一件奇怪的事情是 x 的值是 0x5555，但是我们将 x 设置为 10！ 那是因为直到我们的 `main` 函数运行之后才真正设置 `x` ，而我们现在才到了 `main` 最开始的地方。
|
||||
你可能会在这里注意到的一件奇怪的事情是 `x` 的值是 0x5555，但是我们将 `x` 设置为 `10`！ 那是因为直到我们的 `main` 函数运行之后才真正设置 `x` ，而我们现在才到了 `main` 最开始的地方。
|
||||
|
||||
### 第三步：运行到第十行代码后，再次查看一下我们的堆栈
|
||||
|
||||
让我们跳过几行，等待变量实际设置为其初始化值。 到第 10 行时，`x` 应该设置为10。
|
||||
让我们跳过几行，等待变量实际设置为其初始化值。 到第 10 行时，`x` 应该设置为 `10`。
|
||||
|
||||
首先我们需要设置另一个断点：
|
||||
|
||||
@ -159,15 +158,13 @@ Breakpoint 2, main () at test.c:11
|
||||
11 printf("Enter a string for the stack: ");
|
||||
```
|
||||
|
||||
好的！ 让我们再来看看堆栈里的内容！ `gdb` 在这里格式化字节的方式略有不同，实际上我也不太关心这些(译者注:可以查看 GDB 手册中 x 命令，可以指定 c 来控制输出的格式)。 这里提醒一下您，我们的变量在栈上的位置：
|
||||
好的！ 让我们再来看看堆栈里的内容！ `gdb` 在这里格式化字节的方式略有不同，实际上我也不太关心这些（LCTT 译注:可以查看 GDB 手册中 `x` 命令，可以指定 `c` 来控制输出的格式）。 这里提醒一下你，我们的变量在栈上的位置：
|
||||
|
||||
|
||||
* `x` 是红色字体，并且起始地址是 `0x7fffffffe27c`
|
||||
* `heap_string` 是蓝色字体，起始地址是 `0x7fffffffe280`
|
||||
* `stack_string` 是紫色字体，起始地址是 `0x7fffffffe28e`
|
||||
|
||||
|
||||
|
||||
```
|
||||
(gdb) x/80x $sp
|
||||
0x7fffffffe270: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
|
||||
@ -186,9 +183,9 @@ Breakpoint 2, main () at test.c:11
|
||||
|
||||
### `stack_string` 在内存中是如何表示的
|
||||
|
||||
现在（第10行），`stack_string` 被设置为 “stack”。 让我们看看它在内存中的表示方式。
|
||||
现在（第 10 行），`stack_string` 被设置为字符串`stack`。 让我们看看它在内存中的表示方式。
|
||||
|
||||
我们可以像这样打印出字符串中的字节(译者注：可以通过 c 选项直接显示为字符)：
|
||||
我们可以像这样打印出字符串中的字节（LCTT 译注：可以通过 `c` 选项直接显示为字符）：
|
||||
|
||||
```
|
||||
(gdb) x/10x stack_string
|
||||
@ -196,8 +193,7 @@ Breakpoint 2, main () at test.c:11
|
||||
0x7fffffffe296: 0x00 0x00
|
||||
```
|
||||
|
||||
|
||||
"stack" 是一个长度为 5 的字符串，相对应 5 个 ASCII 码-`0x73`, `0x74`, `0x61`, `0x63`, 和 `0x6b`。`0x73` 是字符 `s` 的 ASCII 码。 `0x74` 是 `t` 的 ASCII 码。等等...
|
||||
`stack` 是一个长度为 5 的字符串，相对应 5 个 ASCII 码- `0x73`、`0x74`、`0x61`、`0x63` 和 `0x6b`。`0x73` 是字符 `s` 的 ASCII 码。 `0x74` 是 `t` 的 ASCII 码。等等...
|
||||
|
||||
同时我们也使用 `x/1s` 可以让 GDB 以字符串的方式显示：
|
||||
|
||||
@ -206,13 +202,11 @@ Breakpoint 2, main () at test.c:11
|
||||
0x7fffffffe28e: "stack"
|
||||
```
|
||||
|
||||
|
||||
|
||||
### `heap_string` 与 `stack_string` 有何不同
|
||||
|
||||
你已经注意到了 `stack_string` 和 `heap_string` 在栈上的表示非常不同：
|
||||
|
||||
* `stack_string` 是一段字符串内容("stack")
|
||||
* `stack_string` 是一段字符串内容（`stack`）
|
||||
* `heap_string` 是一个指针，指向内存中的某个位置
|
||||
|
||||
这里是 `heap_string` 变量在内存中的内容：
|
||||
@ -221,7 +215,7 @@ Breakpoint 2, main () at test.c:11
|
||||
0xa0 0x92 0x55 0x55 0x55 0x55 0x00 0x00
|
||||
```
|
||||
|
||||
这些字节实际上应该是从右向左读：因为 X86 是小端模式，因此，`heap_string` 中所存放的内存地址 `0x5555555592a0`
|
||||
这些字节实际上应该是从右向左读：因为 x86 是小端模式，因此，`heap_string` 中所存放的内存地址 `0x5555555592a0`
|
||||
|
||||
另一种方式查看 `heap_string` 中存放的内存地址就是使用 `p` 命令直接打印 ：
|
||||
|
||||
@ -234,9 +228,9 @@ $6 = 0x5555555592a0 ""
|
||||
|
||||
`x` 是一个 32 位的整数，可由 `0x0a 0x00 0x00 0x00` 来表示。
|
||||
|
||||
我们还是需要反向来读取这些字节（和我们读取 `heap_string` 需要反过来读是一样的），因此这个数表示的是 `0x000000000a` 或者是 `0x0a`, 它是一个 10;
|
||||
我们还是需要反向来读取这些字节（和我们读取 `heap_string` 需要反过来读是一样的），因此这个数表示的是 `0x000000000a` 或者是 `0x0a`，它是一个数字 `10`;
|
||||
|
||||
这就让我把把 x 设置成了 10；
|
||||
这就让我把把 `x` 设置成了 `10`。
|
||||
|
||||
### 第四步：从标准输入读取
|
||||
|
||||
@ -256,7 +250,6 @@ gets(heap_string);
|
||||
Breakpoint 3 at 0x555555555205: file test.c, line 16.
|
||||
```
|
||||
|
||||
|
||||
然后继续执行程序：
|
||||
|
||||
```
|
||||
@ -266,14 +259,14 @@ Continuing.
|
||||
|
||||
我们输入两个字符串，为栈上存储的变量输入 `123456789012` 并且为在堆上存储的变量输入 `bananas`;
|
||||
|
||||
### 让我们先来看一下 `stack_string` (这里有一个缓存区溢出)
|
||||
### 让我们先来看一下 `stack_string`（这里有一个缓存区溢出）
|
||||
|
||||
```
|
||||
(gdb) x/1s stack_string
|
||||
0x7fffffffe28e: "123456789012"
|
||||
```
|
||||
|
||||
这看起来相当正常，对吗？我们输入了 `12345679012`，然后现在它也被设置成了 `12345679012`（译者注：实测 gcc 8.3 环境下，会直接段错误）。
|
||||
这看起来相当正常，对吗？我们输入了 `12345679012`，然后现在它也被设置成了 `12345679012`（LCTT 译注：实测 gcc 8.3 环境下，会直接段错误）。
|
||||
|
||||
但是现在有一些很奇怪的事。这是我们程序的栈空间的内容。有一些紫色高亮的内容。
|
||||
|
||||
@ -290,7 +283,7 @@ Continuing.
|
||||
0x7fffffffe2b8: 0x00 0xa0 0xfc 0xf7 0x01 0x00 0x00 0x00
|
||||
```
|
||||
|
||||
令人奇怪的是 **stack_string 只支持 10 个字节**。但是现在当我们输入了 13 个字符以后，发生了什么？
|
||||
令人奇怪的是 **`stack_string` 只支持 10 个字节**。但是现在当我们输入了 13 个字符以后，发生了什么？
|
||||
|
||||
这是一个典型的缓冲区溢出，`stack_string` 将自己的数据写在了程序中的其他地方。在我们的案例中，这还没有造成问题，但它会使你的程序崩溃，或者更糟糕的是，使你面临非常糟糕的安全问题。
|
||||
|
||||
@ -313,7 +306,7 @@ fish: Job 1, './test' terminated by signal SIGABRT (Abort)
|
||||
|
||||
这里我猜是 `stack_string` 已经到达了这个函数栈的底部，因此额外的字符将会被写在另一块内存中。
|
||||
|
||||
当你故意去使用这个安全漏洞时，它被称为“堆栈粉碎”，我也不完全了解这是如何检测到的。
|
||||
当你故意去使用这个安全漏洞时，它被称为“堆栈粉碎”，而且不知何故有东西在检测这种情况的发生。
|
||||
|
||||
我也觉得这很有趣，虽然程序被杀死了，但是当缓冲区溢出发生时它不会立即被杀死——在缓冲区溢出之后再运行几行代码，程序才会被杀死。 好奇怪！
|
||||
|
||||
@ -348,7 +341,7 @@ fish: Job 1, './test' terminated by signal SIGABRT (Abort)
|
||||
|
||||
我们已经讨论过栈和堆是不同的内存区域，但是你怎么知道它们在内存中的位置呢？
|
||||
|
||||
每个进程都有一个名为 `/proc/$PID/maps` 的文件，它显示了每个进程的内存映射。 在这里您可以看到其中的栈和堆。
|
||||
每个进程都有一个名为 `/proc/$PID/maps` 的文件，它显示了每个进程的内存映射。 在这里你可以看到其中的栈和堆。
|
||||
|
||||
```
|
||||
$ cat /proc/24963/maps
|
||||
@ -364,19 +357,17 @@ $ cat /proc/24963/maps
|
||||
|
||||
这有点像旋风之旅，虽然我没有解释所有内容，但希望看到数据在内存中的实际情况可以使你更清楚地了解堆栈的实际情况。
|
||||
|
||||
我真的建议像这样来把玩一下 gdb —— 即使你不理解你在内存中看到的每一件事，我发现实际上像这样看到我程序内存中的数据会使抽象的概念，比如“栈”和“ 堆”和“指针”更容易理解。
|
||||
我真的建议像这样来把玩一下 gdb —— 即使你不理解你在内存中看到的每一件事，我发现实际上像这样看到我程序内存中的数据会使抽象的概念，比如“栈”和“堆”和“指针”更容易理解。
|
||||
|
||||
### 更多练习
|
||||
|
||||
一些关于思考栈的后续练习的想法（没有特定的顺序）：
|
||||
|
||||
* 尝试将另一个函数添加到 `test.c` 并在该函数的开头创建一个断点，看看是否可以从 `main` 中找到堆栈！ 他们说当你调用一个函数时“堆栈会变小”，你能在 gdb 中看到这种情况吗？
|
||||
* 尝试将另一个函数添加到 `test.c` 并在该函数的开头创建一个断点，看看是否可以从 `main` 中找到堆栈！ 他们说当你调用一个函数时“堆栈会变小”，你能在 gdb 中看到这种情况吗？
|
||||
* 从函数返回一个指向栈上字符串的指针，看看哪里出了问题。 为什么返回指向栈上字符串的指针是不好的？
|
||||
* 尝试在 C 中引起堆栈溢出，并尝试通过在 gdb 中查看堆栈溢出来准确理解会发生什么！
|
||||
* 查看 Rust 程序中的堆栈并尝试找到变量！
|
||||
* 在 [噩梦课程][1] 中尝试一些缓冲区溢出挑战。每个问题的答案写在 README 文件中，因此如果您不想被宠坏，请避免先去看答案。 所有这些挑战的想法是给你一个二进制文件，你需要弄清楚如何导致缓冲区溢出以使其打印出 “flag” 字符串。
|
||||
|
||||
|
||||
* 在 [噩梦课程][1] 中尝试一些缓冲区溢出挑战。每个问题的答案写在 README 文件中，因此如果你不想被宠坏，请避免先去看答案。 所有这些挑战的想法是给你一个二进制文件，你需要弄清楚如何导致缓冲区溢出以使其打印出 `flag` 字符串。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
@ -385,7 +376,7 @@ via: https://jvns.ca/blog/2021/05/17/how-to-look-at-the-stack-in-gdb/
|
||||
作者：[Julia Evans][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[amwps290](https://github.com/amwps290)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@ -3,32 +3,34 @@
|
||||
[#]: author: "Moshe Zadka https://opensource.com/users/moshez"
|
||||
[#]: collector: "lujun9972"
|
||||
[#]: translator: "zepoch"
|
||||
[#]: reviewer: " "
|
||||
[#]: publisher: " "
|
||||
[#]: url: " "
|
||||
[#]: reviewer: "wxy"
|
||||
[#]: publisher: "wxy"
|
||||
[#]: url: "https://linux.cn/article-13551-1.html"
|
||||
|
||||
使用Python解析配置文件
|
||||
使用 Python 解析配置文件
|
||||
======
|
||||
第一步是选择配置文件的格式：INI、JSON、YAML 或 TOML。
|
||||
|
||||
![Python programming language logo with question marks][1]
|
||||
> 第一步是选择配置文件的格式：INI、JSON、YAML 或 TOML。
|
||||
|
||||
有时，程序需要足够的参数，将它们全部作为命令行参数或环境变量既不让人愉快也不可行。 在这些情况下，您将需要使用配置文件。
|
||||
|
||||
|
||||
有几种流行的配置文件格式。 其中包括古老的（虽然偶尔定义不足）`INI`格式，虽然流行但有时难以手写的`JSON`格式，广泛但偶尔在细节方面令人惊讶的`YAML`格式，以及最新添加的`TOML `，很多人还没有听说过。
|
||||
有时，程序需要足够的参数，将它们全部作为命令行参数或环境变量既不让人愉快也不可行。 在这些情况下，你将需要使用配置文件。
|
||||
|
||||
您的首要任务是选择一种格式，然后记录该选择。 解决了这个简单的部分之后就是时候解析配置了。
|
||||
有几种流行的配置文件格式。其中包括古老的（虽然有时定义不明确）INI 格式，虽然流行但有时难以手写的 JSON 格式，使用广泛但有时在细节方面令人意外的 YAML 格式，以及很多人还没有听说过的最新出现的 TOML。
|
||||
|
||||
有时，在配置中拥有一个与“抽象“数据相对应的类是一个不错的想法。 因为这段代码不会对配置做任何事情，所以这是展示解析逻辑最简单的方式。
|
||||
你的首要任务是选择一种格式，然后记录该选择。解决了这个简单的部分之后就是时候解析配置了。
|
||||
|
||||
有时，在配置中拥有一个与“抽象“数据相对应的类是一个不错的想法。因为这段代码不会对配置做任何事情，所以这是展示解析逻辑最简单的方式。
|
||||
|
||||
想象一下文件处理器的配置：它包括一个输入目录、一个输出目录和要提取的文件。
|
||||
|
||||
配置类的抽象定义可能类似于：
|
||||
|
||||
```
|
||||
from __future__ import annotations
|
||||
```
|
||||
|
||||
```
|
||||
`from __future__ import annotations`[/code] [code]
|
||||
|
||||
import attr
|
||||
|
||||
@attr.frozen
|
||||
@ -44,8 +46,7 @@ class Configuration:
|
||||
parameters: Parameters
|
||||
```
|
||||
|
||||
为了使特定于格式的代码更简单，您还将编写一个函数来从字典中解析此类。 请注意，这假设配置将使用破折号，而不是下划线。 这种差异并不少见。
|
||||
|
||||
为了使特定于格式的代码更简单，你还需要编写一个函数来从字典中解析此类。请注意，这假设配置将使用破折号，而不是下划线。 这种差异并不少见。
|
||||
|
||||
```
|
||||
def configuration_from_dict(details):
|
||||
@ -68,7 +69,6 @@ JSON（JavaScript Object Notation）是一种类似于 JavaScript 的格式。
|
||||
|
||||
以下是 JSON 格式的示例配置：
|
||||
|
||||
|
||||
```
|
||||
json_config = """
|
||||
{
|
||||
@ -88,7 +88,6 @@ json_config = """
|
||||
|
||||
解析逻辑使用 `json` 模块将 JSON 解析为 Python 的内置数据结构（字典、列表、字符串），然后从字典中创建类：
|
||||
|
||||
|
||||
```
|
||||
import json
|
||||
def configuration_from_json(data):
|
||||
@ -102,7 +101,6 @@ INI 格式，最初只在 Windows 上流行，之后成为配置标准格式。
|
||||
|
||||
这是与 INI 相同的配置：
|
||||
|
||||
|
||||
```
|
||||
ini_config="""
|
||||
[files]
|
||||
@ -116,7 +114,6 @@ patterns = ['*.txt', '*.md']
|
||||
|
||||
Python 可以使用内置的 `configparser` 模块解析它。解析器充当类似 `dict` 的对象，因此可以直接传递给 `configuration_from_dict`：
|
||||
|
||||
|
||||
```
|
||||
import configparser
|
||||
|
||||
@ -128,11 +125,10 @@ def configuration_from_ini(data):
|
||||
|
||||
### YAML
|
||||
|
||||
YAML (Yet Another Markup Language) 是 JSON 的扩展，旨在更易于手动编写。 部分 YAML 需要通过具有较长的规范来实现这一点。
|
||||
YAML（Yet Another Markup Language）是 JSON 的扩展，旨在更易于手动编写。为了实现了这一点，部分原因是有一个很长的规范。
|
||||
|
||||
以下是 YAML 中的相同配置：
|
||||
|
||||
|
||||
```
|
||||
yaml_config = """
|
||||
files:
|
||||
@ -145,8 +141,7 @@ parameters:
|
||||
"""
|
||||
```
|
||||
|
||||
要让 Python 解析它，您需要安装第三方模块。 最受欢迎的是`PyYAML`（`pip install pyyaml`）。 YAML 解析器还返回可以传递给 `configuration_from_dict` 的内置 Python 数据类型。 但是，YAML 解析器需要一个字节流，因此您需要将字符串转换为字节流。
|
||||
|
||||
要让 Python 解析它，你需要安装第三方模块。最受欢迎的是`PyYAML`（`pip install pyyaml`）。 YAML 解析器还返回可以传递给 `configuration_from_dict` 的内置 Python 数据类型。但是，YAML 解析器需要一个字节流，因此你需要将字符串转换为字节流。
|
||||
|
||||
```
|
||||
import io
|
||||
@ -159,11 +154,10 @@ def configuration_from_yaml(data):
|
||||
|
||||
### TOML
|
||||
|
||||
TOML（Tom's Own Markup Language）旨在成为 YAML 的轻量级替代品。 规范比较短，已经在一些地方流行了（比如 Rust 的包管理器 Cargo 就用它来进行包配置）。
|
||||
TOML（Tom's Own Markup Language）旨在成为 YAML 的轻量级替代品。其规范比较短，已经在一些地方流行了（比如 Rust 的包管理器 Cargo 就用它来进行包配置）。
|
||||
|
||||
这是与 TOML 相同的配置：
|
||||
|
||||
|
||||
```
|
||||
toml_config = """
|
||||
[files]
|
||||
@ -175,8 +169,7 @@ patterns = [ "*.txt", "*.md",]
|
||||
"""
|
||||
```
|
||||
|
||||
为了解析 TOML，您需要安装第三方包。 最流行的一种被简单地称为 `toml`。 与 YAML 和 JSON 一样，它返回基本的 Python 数据类型。
|
||||
|
||||
为了解析 TOML，你需要安装第三方包。最流行的一种被简单地称为 `toml`。 与 YAML 和 JSON 一样，它返回基本的 Python 数据类型。
|
||||
|
||||
```
|
||||
import toml
|
||||
@ -185,9 +178,9 @@ def configuration_from_toml(data):
|
||||
return configuration_from_dict(parsed)
|
||||
```
|
||||
|
||||
### Summary总结
|
||||
### 总结
|
||||
|
||||
选择配置格式是一种微妙的权衡。 但是，一旦您做出决定，Python 就可以使用少量代码来解析大多数流行的格式。
|
||||
选择配置格式是一种微妙的权衡。但是，一旦你做出决定，Python 就可以使用少量代码来解析大多数流行的格式。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
@ -196,7 +189,7 @@ via: https://opensource.com/article/21/6/parse-configuration-files-python
|
||||
作者：[Moshe Zadka][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[zepoch](https://github.com/zepoch)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: translator: (zxy-wyx)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
@ -42,7 +42,7 @@ via: https://www.networkworld.com/article/3515314/data-center-spending-will-inch
|
||||
|
||||
作者：[Andy Patrizio][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
译者：[zxy-wyx](https://github.com/zxy-wy)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@ -1,47 +0,0 @@
|
||||
[#]: subject: (My weird jobs before tech)
|
||||
[#]: via: (https://opensource.com/article/21/5/weird-jobs-tech)
|
||||
[#]: author: (Chris Hermansen https://opensource.com/users/clhermansen)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (MM-BCY)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
My weird jobs before tech
|
||||
======
|
||||
You never know where you will travel from your first job.
|
||||
![Yellow plane flying in the air, Beechcraft D17S][1]
|
||||
|
||||
I had a few weird jobs before I hit tech.
|
||||
|
||||
I was a junior assistant in an aircraft repair shop, which meant tasks like cleaning dirty metal parts in solvent (wow, things were different back in the '70s). My most fun task there was ironing Dacron aircraft fabric onto the wooden ailerons and horizontal stabilizer on a beautiful old Beechcraft Staggerwing that was in the shop for a rebuild.
|
||||
|
||||
One summer during university, I worked at the same airport on the team that mixed the fire retardant and pumped it into the fire suppression aircraft ("[water bombers][2]"). That was probably the dirtiest job I ever had, but loading the aircraft was pretty cool. There was a small flap about two meters off the ground that you would stick your finger into after attaching the filling hose to the coupling. Then the person on the pump would start the pump. When you felt your finger get wet, you waved for the pump master to stop the pump. Meanwhile, the incredibly noisy right-side radial engine was running a few meters in front of you, with the propellers doing a great job of blowing off all the red dust that accumulated on you from mixing the retardant in the first place. If you screwed up and let the airplane get too full, they would have to taxi over to a patch of ground and dump the load right there, since they would be too heavy to take off otherwise.
|
||||
|
||||
Two other summers, I worked for the local Pepsi, 7-Up, and Orange Crush distributor delivering crates of soft drinks to stores and restaurants. That was definitely the most physically demanding job I ever had. Think of a five-high stack of wooden crates with each containing a dozen 750ml glass bottles of soft drinks on a hand truck. Think of pulling that up to a second-floor restaurant. Think of that restaurant getting 120 crates per week... 24 trips up those stairs and back down again with all the empties. A small truck would typically have 300 or so crates of soft drinks on board. We were paid by the load, not by the hour, so the goal was to get done early and hit the beach.
|
||||
|
||||
### My tech jobs
|
||||
|
||||
Delivering sodas was my last summer job during university. I graduated the next year with a degree in mathematics and a lot of computer courses, especially numerical analysis, under my belt. My first job in tech was working for a small computer services consultant. I used SPSS to do a bunch of analysis on some sport fishing surveys, wrote a few hundred lines of PL/1 to print concert tickets on the IBM 3800 laser printer in the service bureau where we rented time, and started working on some programs to analyze forest statistics. I eventually went to work for the client needing forestry statistics, becoming a partner in the mid-1980s. By then we were doing a lot more than measuring trees and no longer using a timesharing bureau to do our computations. We bought a Unix minicomputer, which we upgraded in the late 1980s to a network of Sun workstations.
|
||||
|
||||
I spent some time working on a big development project headquartered in Kuala Lumpur, Malaysia. Then we bought our first geographic information system, and I spent most of my time in the late 1980s and 1990s working with our customers who needed to customize that software to meet their business needs. By the early 2000s, my three older partners were getting ready to retire, and I was trying to understand how I fit into the long-term picture of our no-longer-small company of 200 or so employees. Our new employee-owners couldn't really figure that one out either, and in 2002, I found myself in Chile, looking to see if the Chile-Canada Free Trade Agreement provided a reasonable opportunity to move some of our business to Latin America.
|
||||
|
||||
That business started off formally in 2004. The Canadian parent, meanwhile, was badly sideswiped by a combination of some investments that, in the light of the 2007–2009 economic meltdown, no longer seemed so wise, and it was forced to close its doors in 2011. However, by that time, the Chilean subsidiary was a going concern, so our original employee and I became partners and purchased it from the asset sale. It's still going today, doing a lot of cool stuff in the social-environmental space, and I'm often a part of that, especially when my trusty mathematics and computational background are useful.
|
||||
|
||||
As a side hustle, I develop and support a horse racing information system for a wonderful man who has made a career out of buying and selling racehorses in India.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/5/weird-jobs-tech
|
||||
|
||||
作者：[Chris Hermansen][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[MM-BCY](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/clhermansen
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/yellow_plane_fly_air.jpg?itok=pEcrCVJT (Yellow plane flying in the air, Beechcraft D17S)
|
||||
[2]: https://worldairphotography.wordpress.com/2016/08/22/air-tanker-history-in-canada-part-one/amp/
|
||||
@ -0,0 +1,78 @@
|
||||
[#]: subject: (Pricing Yourself as a Contractor 101)
|
||||
[#]: via: (https://theartofmachinery.com/2021/07/04/pricing_as_contractor_101.html)
|
||||
[#]: author: (Simon Arneaud https://theartofmachinery.com)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
Pricing Yourself as a Contractor 101
|
||||
======
|
||||
|
||||
I’ve been self-employed for most of my career. Sometimes I talk to other people who are interested in leaving a full-time job to do some kind of contracting or service business. By far, the most common newbie mistake that we all seem to make is in pricing ourselves.
|
||||
|
||||
Take [this useful blog post that breaks down employee income vs freelancer income in the US][1]. It estimates that you need $140k revenue as a freelancer in the US to have the equivalent of $100k employee compensation. I remember finding calculations like that really useful when I first started a business. However, some people will look at the result and think, “Gee, I have to make 1.4x as much if I’m self employed. Can I really do that?”
|
||||
|
||||
No, no, no. That thinking is backwards.
|
||||
|
||||
### How to price yourself
|
||||
|
||||
Let’s make up an example. Suppose you’re a full-time-employed software engineer grossing that $100k p.a., and you’re thinking of switching to contracting.
|
||||
|
||||
When you’re self-employed, you have to think like a business because that’s literally how you’re making your living. So you have to add up all your costs and figure out how you’re going to recover them. Spreadsheets get a bad rap (for some good reasons), but they’re actually really useful for this stuff (and a lot of other calculations you’ll do as a business owner).
|
||||
|
||||
The first cost to add to the tally is that $100k. If that sounds weird, it’s what’s called “opportunity cost”. You could have made $100k by staying employed; not making that is effectively a cost you have to justify when planning your business. Mark that cost down, along with any other employment benefits you actually use. If your employer offers on-site lunches, add what it costs you to get lunch each workday of the year. If your employer offers employee discounts on its fitness software, but you don’t use that software anyway, don’t add that benefit as an opportunity cost.
|
||||
|
||||
Other costs depend on what you’re doing and where you live. Employee-provided health insurance isn’t as big a thing in Australia as in the US. On the other hand, compulsory superannuation payments (similar to the US 401(k)) are a big deal. I have my own company, and my major non-salary costs are insurance, accounting/filing, legal (for contract reviews, etc.), debt collection and various online service costs. If you’re counting something durable (like a desk) divide the cost by the estimated number of years you expect to use the thing.
|
||||
|
||||
Anyway, so far this is basically what was in Caleb’s blog post, so to keep things simple, I’ll assume the same $100k nominal salary and $140k equivalent business cost. (Scale everything to match your own circumstances, of course.) Now you need to figure out how to recover that cost. There are about 255 Australian working days in a year, so if you could contract them all out, you’d charge $550 a day (plus sales tax). In reality, you won’t be able to bill the entire year. I’ve taken a higher-risk approach and averaged about 60-70% in the past 6 years of my current stint of self employment. [Accenture’s annual financial reports][2] say they get about 90% “utilization” from their contractors, which I assume means they bill 90% of the total workdays. Let’s assume you’re moderate and bill 75% of work days. That means you recover $140k of costs in 75% of 255 days (or 191 days) by billing about $730 a day (plus sales tax).
|
||||
|
||||
### The mistake
|
||||
|
||||
People new to contracting often react to numbers like that and think, “WTF?! That’s huge!” That’s just one example calculation, but it’s normal for service prices to be around double or more what you might naïvely guess from equivalent full-time employee rates. However, that day rate came from a simple calculation of how much you need to charge to get the equivalent of a $100k salary. It’s the same thing. Thinking otherwise is the critical mistake.
|
||||
|
||||
New contractors are often still unsure. Won’t they sound _greedy_ asking for that much? If your clients have any clue, they’re doing pretty much the same calculation. “I could pay Gentle Blog Reader $730 a day for just as long as I want, or I could pay ~$140k for a full-timer who I won’t even really need every day.” A $100k salary isn’t actually $100k from the employer’s point of view, either. Basing prices on nominal base salaries just doesn’t make sense. Even if you’re selling B2C, your cluey competitors won’t be charging less, at least not sustainably.
|
||||
|
||||
### Why it matters
|
||||
|
||||
That specific example was for contracting, but it’s a basic rule of business economics: unless you’re trying some super-risky growth hacking (and we know how [Pets.com][3] turned out), you need to figure out your costs and set your price high enough to cover them.
|
||||
|
||||
Some people still feel uncomfortable with the price they need to set, and they rationalise dropping the price. Perhaps they think something like, “I’m a really nice person, and if I charge only $400 a day my clients will be even happier.” The problem is that you won’t get the same clients. Cluey clients who would pay $100k a year base salary for an employee won’t pay $400 a day for a contractor to do the same job. Instead, in practice, you might get a few good clients who just didn’t have the budget for $730 a day, but you _will_ get a whole bunch of really bad clients. Think about it. If a stranger offered you a fancy-looking diamond ring for $50, would you pay? Or would you rather buy another ring for a normal price?
|
||||
|
||||
Let me stress that I’m just taking the numbers from Caleb’s post and that everything is relative. Use your own numbers instead. In most parts of the world, $400 a day might be a fantastic rate. However, if you’re a senior fintech developer in Silicon Valley, charging $400 a day will just make you a magnet for terrible clients. Most of the good ones will know something isn’t adding up, and they’ll be scared away.
|
||||
|
||||
What do I mean by bad clients? Browse through [the Clients from Hell blog][4] for a bit. It ranges from a lot of basic annoyance like clients who are never satisfied, or who make unreasonable demands, or who waste your time, all the way to clients who are outright abusive, or who get you to do work to spec before arguing they shouldn’t have to pay because “I don’t want it”. Some clients simply don’t pay at all.
|
||||
|
||||
If _you_ don’t value your own product enough, don’t be shocked if you have customers who don’t value it enough, either.
|
||||
|
||||
It gets worse, though. Good clients tend to work with other good clients. If you’re always available when you say you will be, would you work with people who waste your time? If you treat others with respect, would you work with people who are unreasonable and abusive? On average, your good clients will tend to refer you to other good clients. The reverse is true of bad clients, if they’re even grateful enough to refer you to anyone at all. Therefore, if you charge a good price, your business will tend to grow as you build a reputation. If you undercharge, you’ll find yourself in a downward spiral where you’re not only losing money, but finding it harder and harder to get proper pay at all.
|
||||
|
||||
All of this is just a matter of averages, and if you’re lucky you’ll still get good clients even if you undercharge, and if you’re unlucky you’ll still get bad clients even if you charge the right price. However, if your revenue is already weak, each bad client will really hurt. Hoping to beat the averages isn’t a good plan.
|
||||
|
||||
### “But no one pays that much!”
|
||||
|
||||
Suppose you’re an experienced full-time engineer and you decide to try going independent. You’ll probably find that your calculated rate seems high compared to what you see on a freelancing website. That’s because it’s hard to build up a reputation on freelancing websites. Freelancing websites are most useful for casual buyers who primarily want a low price.
|
||||
|
||||
I think a lot of smart engineers assume career networking is hard and requires super high levels of extroversion, so they have to rely on freelancing websites for work. The bad news is that you need to build up a good reputation to get good pay. The good news is that most people can do it as long as they have skills that are in demand. Networking isn’t about going to so-called “networking events” (they’re actually mostly terrible for networking). Networking tips would make a whole new blog post, but the key is to find good clients in their natural habitats, and to do the things that make them keep coming back and maybe even refer you to other good clients.
|
||||
|
||||
In any case, don’t let freelancing websites or anything else set your price below the equivalent of what you could get from a full-time salary. In fact, [you might even get better than your current salary][5], which is why this is “Pricing 101”. Undercharging, however, will kill your self-employment career.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://theartofmachinery.com/2021/07/04/pricing_as_contractor_101.html
|
||||
|
||||
作者：[Simon Arneaud][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://theartofmachinery.com
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://calebporzio.com/making-100k-as-an-employee-versus-being-self-employed
|
||||
[2]: https://www.accenture.com/au-en/about/company/annual-report
|
||||
[3]: https://en.wikipedia.org/wiki/Pets.com
|
||||
[4]: https://clientsfromhell.net/
|
||||
[5]: https://theartofmachinery.com/2018/10/07/payrise_by_switching_jobs.html
|
||||
@ -1,351 +0,0 @@
|
||||
Cgo and Python
|
||||
============================================================
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
If you look at the [new Datadog Agent][8], you might notice most of the codebase is written in Go, although the checks we use to gather metrics are still written in Python. This is possible because the Datadog Agent, a regular Go binary, [embeds][9] a CPython interpreter that can be called whenever it needs to execute Python code. This process can be made transparent using an abstraction layer so that you can still write idiomatic Go code even when there’s Python running under the hood.
|
||||
|
||||
[video](https://youtu.be/yrEi5ezq2-c)
|
||||
|
||||
There are a number of reasons why you might want to embed Python in a Go application:
|
||||
|
||||
* It is useful during a port; gradually moving portions of an existing Python project to the new language without losing any functionality during the process.
|
||||
|
||||
* You can reuse existing Python software or libraries without re-implementing them in the new language.
|
||||
|
||||
* You can dynamically extend your software by loading and executing regular Python scripts, even at runtime.
|
||||
|
||||
The list could go on, but for the Datadog Agent the last point is crucial: we want you to be able to execute custom checks or change existing ones without forcing you to recompile the Agent, or in general, to compile anything.
|
||||
|
||||
Embedding CPython is quite easy and well documented. The interpreter itself is written in C and a C API is provided to programmatically perform operations at a very low level, like creating objects, importing modules, and calling functions.
|
||||
|
||||
In this article we’ll show some code examples, and we’ll focus on keeping the Go code idiomatic while interacting with Python at the same time, but before we proceed we need to address a small gap: the embedding API is C but our main application is Go, how can this possibly work?
|
||||
|
||||
|
||||
|
||||
### Introducing cgo
|
||||
|
||||
There are [a number of good reasons][10] why you might not want to introduce cgo in your stack, but embedding CPython is one of those cases where you must. [Cgo][11] is not a language nor a compiler. It’s a [Foreign Function Interface][12] (FFI), a mechanism we can use in Go to invoke functions and services written in a different language, specifically C.
|
||||
|
||||
When we say “cgo” we’re actually referring to a set of tools, libraries, functions, and types that are used by the go toolchain under the hood so we can keep doing `go build` to get our Go binaries. An absolutely minimal example of a program using cgo looks like this:
|
||||
|
||||
```
|
||||
package main
|
||||
|
||||
// #include <float.h>
|
||||
import "C"
|
||||
import "fmt"
|
||||
|
||||
func main() {
|
||||
fmt.Println("Max float value of float is", C.FLT_MAX)
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
The comment block right above the `import "C"` instruction is called a “preamble” and can contain actual C code, in this case an header inclusion. Once imported, the “C” pseudo-package lets us “jump” to the foreign code, accessing the `FLT_MAX` constant. You can build the example by invoking `go build`, the same as if it was plain Go.
|
||||
|
||||
If you want to have a look at all the work cgo does under the hood, run `go build -x`. You’ll see the “cgo” tool will be invoked to generate some C and Go modules, then the C and Go compilers will be invoked to build the object modules and finally the linker will put everything together.
|
||||
|
||||
You can read more about cgo on the [Go blog][13]. The article contains more examples and few useful links to get further into details.
|
||||
|
||||
Now that we have an idea of what cgo can do for us, let’s see how we can run some Python code using this mechanism.
|
||||
|
||||
|
||||
### Embedding CPython: a primer
|
||||
|
||||
A Go program that, technically speaking, embeds CPython is not as complicated as you might expect. In fact, at the bare minimum, all we have to do is initialize the interpreter before running any Python code and finalize it when we’re done. Please note that we’re going to use Python 2.x throughout all the examples but everything we’ll see can be applied to Python 3.x as well with very little adaptation. Let’s look at an example:
|
||||
|
||||
```
|
||||
package main
|
||||
|
||||
// #cgo pkg-config: python-2.7
|
||||
// #include <Python.h>
|
||||
import "C"
|
||||
import "fmt"
|
||||
|
||||
func main() {
|
||||
C.Py_Initialize()
|
||||
fmt.Println(C.GoString(C.Py_GetVersion()))
|
||||
C.Py_Finalize()
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
The example above does exactly what the following Python code would do:
|
||||
|
||||
```
|
||||
import sys
|
||||
print(sys.version)
|
||||
|
||||
```
|
||||
|
||||
You can see we put a `#cgo` directive in the preamble; those directives are passed to the toolchain to let you change the build workflow. In this case, we tell cgo to invoke “pkg-config” to gather the flags needed to build and link against a library called “python-2.7” and pass those flags to the C compiler. If you have the CPython development libraries installed in your system along with pkg-config, this would let you keep using a plain `go build` to compile the example above.
|
||||
|
||||
Back to the code, we use `Py_Initialize()` and `Py_Finalize()` to set up and shut down the interpreter and the `Py_GetVersion` C function to retrieve the string containing the version information for the embedded interpreter.
|
||||
|
||||
If you’re wondering, all the cgo bits we need to put together to invoke the C Python API are boilerplate code. This is why the Datadog Agent relies on [go-python][14] for all the embedding operations; the library provides a Go friendly thin wrapper around the C API and hides the cgo details. This is another basic embedding example, this time using go-python:
|
||||
|
||||
```
|
||||
package main
|
||||
|
||||
import (
|
||||
python "github.com/sbinet/go-python"
|
||||
)
|
||||
|
||||
func main() {
|
||||
python.Initialize()
|
||||
python.PyRun_SimpleString("print 'hello, world!'")
|
||||
python.Finalize()
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
This looks closer to regular Go code, no more cgo exposed and we can use Go strings back and forth while accessing the Python API. Embedding looks powerful and developer friendly. Time to put the interpreter to good use: let’s try to load a Python module from disk.
|
||||
|
||||
We don’t need anything complex on the Python side, the ubiquitous “hello world” will serve the purpose:
|
||||
|
||||
```
|
||||
# foo.py
|
||||
def hello():
|
||||
"""
|
||||
Print hello world for fun and profit.
|
||||
"""
|
||||
print "hello, world!"
|
||||
|
||||
```
|
||||
|
||||
The Go code is slightly more complex but still readable:
|
||||
|
||||
```
|
||||
// main.go
|
||||
package main
|
||||
|
||||
import "github.com/sbinet/go-python"
|
||||
|
||||
func main() {
|
||||
python.Initialize()
|
||||
defer python.Finalize()
|
||||
|
||||
fooModule := python.PyImport_ImportModule("foo")
|
||||
if fooModule == nil {
|
||||
panic("Error importing module")
|
||||
}
|
||||
|
||||
helloFunc := fooModule.GetAttrString("hello")
|
||||
if helloFunc == nil {
|
||||
panic("Error importing function")
|
||||
}
|
||||
|
||||
// The Python function takes no params but when using the C api
|
||||
// we're required to send (empty) *args and **kwargs anyways.
|
||||
helloFunc.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
Once built, we need to set the `PYTHONPATH` environment variable to the current working dir so that the import statement will be able to find the `foo.py`module. From a shell, the command would look like this:
|
||||
|
||||
```
|
||||
$ go build main.go && PYTHONPATH=. ./main
|
||||
hello, world!
|
||||
|
||||
```
|
||||
|
||||
|
||||
### The dreadful Global Interpreter Lock
|
||||
|
||||
Having to bring in cgo in order to embed Python is a tradeoff: builds will be slower, the Garbage Collector won’t help us managing memory used by the foreign system, and cross compilation will be non-trivial. Whether or not these are concerns for a specific project can be debated, but there’s something I deem not negotiable: the Go concurrency model. If we couldn’t run Python from a goroutine, using Go altogether would make very little sense.
|
||||
|
||||
Before playing with concurrency, Python, and cgo, there’s something we need to know: it’s the Global Interpreter Lock, also known as the GIL. The GIL is a mechanism widely adopted in language interpreters (CPython is one of those) preventing more than one thread from running at the same time. This means that no Python program executed by CPython will be ever able to run in parallel within the same process. Concurrency is still possible and in the end, the lock is a good tradeoff between speed, security, and implementation simplicity. So why should this pose a problem when it comes to embedding?
|
||||
|
||||
When a regular, non-embedded Python program starts, there’s no GIL involved to avoid useless overhead in locking operations; the GIL starts the first time some Python code requests to spawn a thread. For each thread, the interpreter creates a data structure to store information about the current state and locks the GIL. When the thread has finished, the state is restored and the GIL unlocked, ready to be used by other threads.
|
||||
|
||||
When we run Python from a Go program, none of the above happens automatically. Without the GIL, multiple Python threads could be created by our Go program. This could cause a race condition leading to fatal runtime errors, and most likely a segmentation fault bringing down the whole Go application.
|
||||
|
||||
The solution to this problem is to explicitly invoke the GIL whenever we run multithreaded code from Go; the code is not complex because the C API provides all the tools we need. To better expose the problem, we need to do something CPU bounded from Python. Let’s add these functions to our foo.py module from the previous example:
|
||||
|
||||
```
|
||||
# foo.py
|
||||
import sys
|
||||
|
||||
def print_odds(limit=10):
|
||||
"""
|
||||
Print odds numbers < limit
|
||||
"""
|
||||
for i in range(limit):
|
||||
if i%2:
|
||||
sys.stderr.write("{}\n".format(i))
|
||||
|
||||
def print_even(limit=10):
|
||||
"""
|
||||
Print even numbers < limit
|
||||
"""
|
||||
for i in range(limit):
|
||||
if i%2 == 0:
|
||||
sys.stderr.write("{}\n".format(i))
|
||||
|
||||
```
|
||||
|
||||
We’ll try to print odd and even numbers concurrently from Go, using two different goroutines (thus involving threads):
|
||||
|
||||
```
|
||||
package main
|
||||
|
||||
import (
|
||||
"sync"
|
||||
|
||||
"github.com/sbinet/go-python"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// The following will also create the GIL explicitly
|
||||
// by calling PyEval_InitThreads(), without waiting
|
||||
// for the interpreter to do that
|
||||
python.Initialize()
|
||||
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(2)
|
||||
|
||||
fooModule := python.PyImport_ImportModule("foo")
|
||||
odds := fooModule.GetAttrString("print_odds")
|
||||
even := fooModule.GetAttrString("print_even")
|
||||
|
||||
// Initialize() has locked the the GIL but at this point we don't need it
|
||||
// anymore. We save the current state and release the lock
|
||||
// so that goroutines can acquire it
|
||||
state := python.PyEval_SaveThread()
|
||||
|
||||
go func() {
|
||||
_gstate := python.PyGILState_Ensure()
|
||||
odds.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
python.PyGILState_Release(_gstate)
|
||||
|
||||
wg.Done()
|
||||
}()
|
||||
|
||||
go func() {
|
||||
_gstate := python.PyGILState_Ensure()
|
||||
even.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
python.PyGILState_Release(_gstate)
|
||||
|
||||
wg.Done()
|
||||
}()
|
||||
|
||||
wg.Wait()
|
||||
|
||||
// At this point we know we won't need Python anymore in this
|
||||
// program, we can restore the state and lock the GIL to perform
|
||||
// the final operations before exiting.
|
||||
python.PyEval_RestoreThread(state)
|
||||
python.Finalize()
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
While reading the example you might note a pattern, the pattern that will become our mantra to run embedded Python code:
|
||||
|
||||
1. Save the state and lock the GIL.
|
||||
|
||||
2. Do Python.
|
||||
|
||||
3. Restore the state and unlock the GIL.
|
||||
|
||||
The code should be straightforward but there’s a subtle detail we want to point out: notice that despite seconding the GIL mantra, in one case we operate the GIL by calling `PyEval_SaveThread()` and `PyEval_RestoreThread()`, in another (look inside the goroutines) we do the same with `PyGILState_Ensure()`and `PyGILState_Release()`.
|
||||
|
||||
We said when multithreading is operated from Python, the interpreter takes care of creating the data structure needed to store the current state, but when the same happens from the C API, we’re responsible for that.
|
||||
|
||||
When we initialize the interpreter with go-python, we’re operating in a Python context. So when `PyEval_InitThreads()` is called it initializes the data structure and locks the GIL. We can use `PyEval_SaveThread()` and `PyEval_RestoreThread()` to operate on already existing state.
|
||||
|
||||
Inside the goroutines, we’re operating from a Go context and we need to explicitly create the state and remove it when done, which is what `PyGILState_Ensure()` and `PyGILState_Release()` do for us.
|
||||
|
||||
|
||||
### Unleash the Gopher
|
||||
|
||||
At this point we know how to deal with multithreading Go code executing Python in an embedded interpreter but after the GIL, another challenge is right around the corner: the Go scheduler.
|
||||
|
||||
When a goroutine starts, it’s scheduled for execution on one of the `GOMAXPROCS`threads available—[see here][15] for more details on the topic. If a goroutine happens to perform a syscall or call C code, the current thread hands over the other goroutines waiting to run in the thread queue to another thread so they can have better chances to run; the current goroutine is paused, waiting for the syscall or the C function to return. When this happens, the thread tries to resume the paused goroutine, but if this is not possible, it asks the Go runtime to find another thread to complete the goroutine and goes to sleep. The goroutine is finally scheduled to another thread and it finishes.
|
||||
|
||||
With this in mind, let’s see what can happen to a goroutine running some Python code when a goroutine is moved to a new thread::
|
||||
|
||||
1. Our goroutine starts, performs a C call, and pauses. The GIL is locked.
|
||||
|
||||
2. When the C call returns, the current thread tries to resume the goroutine, but it fails.
|
||||
|
||||
3. The current thread tells the Go runtime to find another thread to resume our goroutine.
|
||||
|
||||
4. The Go scheduler finds an available thread and the goroutine is resumed.
|
||||
|
||||
5. The goroutine is almost done and tries to unlock the GIL before returning.
|
||||
|
||||
6. The thread ID stored in the current state is from the original thread and is different from the ID of the current thread.
|
||||
|
||||
7. Panic!
|
||||
|
||||
Luckily for us, we can force the Go runtime to always keep our goroutine running on the same thread by calling the LockOSThread function from the runtime package from within a goroutine:
|
||||
|
||||
```
|
||||
go func() {
|
||||
runtime.LockOSThread()
|
||||
|
||||
_gstate := python.PyGILState_Ensure()
|
||||
odds.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
python.PyGILState_Release(_gstate)
|
||||
wg.Done()
|
||||
}()
|
||||
|
||||
```
|
||||
|
||||
This will interfere with the scheduler and might introduce some overhead, but it’s a price that we’re willing to pay to avoid random panics.
|
||||
|
||||
### Conclusions
|
||||
|
||||
In order to embed Python, the Datadog Agent has to accept a few tradeoffs:
|
||||
|
||||
* The overhead introduced by cgo.
|
||||
|
||||
* The task of manually handling the GIL.
|
||||
|
||||
* The limitation of binding goroutines to the same thread during execution.
|
||||
|
||||
We’re happy to accept each of these for the convenience of running Python checks in Go. But by being conscious of the tradeoffs, we’re able to minimize their effect. Regarding other limitations introduced to support Python, we have few countermeasures to contain potential issues:
|
||||
|
||||
* The build is automated and configurable so that devs have still something very similar to `go build`.
|
||||
|
||||
* A lightweight version of the agent can be built stripping out Python support entirely simply using Go build tags.
|
||||
|
||||
* Such a version only relies on core checks hardcoded in the agent itself (system and network checks mostly) but is cgo free and can be cross compiled.
|
||||
|
||||
We’ll re-evaluate our options in the future and decide whether keeping around cgo is still worth it; we could even reconsider whether Python as a whole is still worth it, waiting for the [Go plugin package][16] to be mature enough to support our use case. But for now the embedded Python is working well and transitioning from the old Agent to the new one couldn’t be easier.
|
||||
|
||||
Are you a polyglot who loves mixing different programming languages? Do you love learning about the inner workings of languages to make your code more performant? [Join us at Datadog!][17]
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
|
||||
作者：[ Massimiliano Pippi][a]
|
||||
译者：[Zioyi](https://github.com/Zioyi)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://github.com/masci
|
||||
[1]:http://twitter.com/share?url=https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
[2]:http://www.reddit.com/submit?url=https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
[3]:https://www.linkedin.com/shareArticle?mini=true&url=https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
[4]:https://www.datadoghq.com/blog/category/under-the-hood
|
||||
[5]:https://www.datadoghq.com/blog/tag/agent
|
||||
[6]:https://www.datadoghq.com/blog/tag/golang
|
||||
[7]:https://www.datadoghq.com/blog/tag/python
|
||||
[8]:https://github.com/DataDog/datadog-agent/
|
||||
[9]:https://docs.python.org/2/extending/embedding.html
|
||||
[10]:https://dave.cheney.net/2016/01/18/cgo-is-not-go
|
||||
[11]:https://golang.org/cmd/cgo/
|
||||
[12]:https://en.wikipedia.org/wiki/Foreign_function_interface
|
||||
[13]:https://blog.golang.org/c-go-cgo
|
||||
[14]:https://github.com/sbinet/go-python
|
||||
[15]:https://morsmachine.dk/go-scheduler
|
||||
[16]:https://golang.org/pkg/plugin/
|
||||
[17]:https://www.datadoghq.com/careers/
|
||||
@ -1,91 +0,0 @@
|
||||
[#]: subject: (How to archive files on FreeDOS)
|
||||
[#]: via: (https://opensource.com/article/21/6/archive-files-freedos)
|
||||
[#]: author: (Jim Hall https://opensource.com/users/jim-hall)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
How to archive files on FreeDOS
|
||||
======
|
||||
There's a version of tar on FreeDOS, but the de facto standard archiver
|
||||
on DOS is Zip and Unzip.
|
||||
![Filing cabinet for organization][1]
|
||||
|
||||
On Linux, you may be familiar with the standard Unix archive command: `tar`. There's a version of `tar` on FreeDOS too (and a bunch of other popular archive programs), but the de facto standard archiver on DOS is Zip and Unzip. Both Zip and Unzip are installed in FreeDOS 1.3 RC4 by default.
|
||||
|
||||
The Zip file format was originally conceived in 1989 by Phil Katz of PKWARE, for the PKZIP and PKUNZIP pair of DOS archive utilities. Katz released the specification for Zip files as an open standard, so anyone could create Zip archives. As a result of the open specification, Zip became a standard archive on DOS. The [Info-ZIP][2] project implements an open source set of `ZIP` and `UNZIP` programs.
|
||||
|
||||
### Zipping files and directories
|
||||
|
||||
You can use `ZIP` at the DOS command line to create archives of files and directories. This is a handy way to make a backup copy of your work or to release a "package" to use in a future FreeDOS distribution. For example, let's say I wanted to make a backup of my project source code, which contains these source files:
|
||||
|
||||
![dir][3]
|
||||
|
||||
I'd like to archive these files
|
||||
(Jim Hall, [CC-BY SA 4.0][4])
|
||||
|
||||
`ZIP` sports a ton of command-line options to do different things, but the command line options I use most are `-r` to process directories and subdirectories _recursively_, and `-9` to provide the maximum compression possible. `ZIP` and `UNZIP` use a Unix-like command line, so you can combine options behind the dash: `-9r` will give maximum compression and include subdirectories in the Zip file.
|
||||
|
||||
![zip][5]
|
||||
|
||||
Zipping a directory tree
|
||||
(Jim Hall, [CC-BY SA 4.0][4])
|
||||
|
||||
In my example, `ZIP` was able to compress my source files from about 33 kilobytes down to about 22 kilobytes, saving me 11 kilobytes of valuable disk space. You might get different compression ratios depending on what options you give to `ZIP` or what files (and how many) you are trying to store in a Zip file. Generally, very long text files (such as source code) yield good compression—very small text files (like DOS "batch" files of only a few lines) are usually too short to compress well.
|
||||
|
||||
### Unzipping files and directories
|
||||
|
||||
Saving files into a Zip file is great, but you'll eventually need to extract those files somewhere. Let's start by examining what's inside the Zip file we just created. For this, use the `UNZIP` command. You can use a bunch of different options with `UNZIP`, but I find I use just a few common options.
|
||||
|
||||
To list the contents of a Zip file, use the `-l` ("list") option:
|
||||
|
||||
![unzip -l][6]
|
||||
|
||||
Listing the archive file contents with unzip
|
||||
(Jim Hall, [CC-BY SA 4.0][4])
|
||||
|
||||
The output allows me to see the 14 entries in the Zip file: 13 files plus the `SRC` directory entry.
|
||||
|
||||
If I want to extract the entire Zip file, I could just use the `UNZIP` command and provide the Zip file as a command-line option. That extracts the Zip file starting at my current working directory. Unless I'm restoring a previous version of something, I usually don't want to overwrite my current files. In that case, I will want to extract the Zip file to a new directory. You can specify the destination path with the `-d` ("destination") command-line option:
|
||||
|
||||
![unzip -d temp][7]
|
||||
|
||||
You can unzip into a destination path with -d
|
||||
(Jim Hall, [CC-BY SA 4.0][4])
|
||||
|
||||
Sometimes I want to extract a single file from a Zip file. In this example, let's say I wanted to extract `TEST.EXE`, a DOS executable program. To extract a single file, you specify the full path _from the Zip file_ that you want to extract. By default, `UNZIP` will extract this file using the path provided in the Zip file. To omit the path information, you can add the `-j` ("junk the path") option.
|
||||
|
||||
You can also combine options. Let's extract the `SRC\TEST.EXE` program from the Zip file, but omit the full path and save it in the `TEMP` directory:
|
||||
|
||||
![unzip -j][8]
|
||||
|
||||
Combining options with unzip
|
||||
(Jim Hall, [CC-BY SA 4.0][4])
|
||||
|
||||
Because Zip files are an open standard, we continue to see Zip files today. Every Linux distribution supports Zip files using the Info-ZIP programs. Your Linux file manager may also have Zip file support—on the GNOME file manager, you should be able to right-click on a folder and select "Compress" from the drop-down menu. You'll have the option to create a new archive file, including a Zip file.
|
||||
|
||||
Creating and managing Zip files is a key skill for any DOS user. You can learn more about `ZIP` and `UNZIP` at the Info-ZIP website, or use the `-h` ("help") option on the command line to print out a list of options.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/6/archive-files-freedos
|
||||
|
||||
作者：[Jim Hall][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/jim-hall
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/files_documents_organize_letter.png?itok=GTtiiabr (Filing cabinet for organization)
|
||||
[2]: http://infozip.sourceforge.net/
|
||||
[3]: https://opensource.com/sites/default/files/uploads/dir.png (I'd like to archive these files)
|
||||
[4]: https://creativecommons.org/licenses/by-sa/4.0/
|
||||
[5]: https://opensource.com/sites/default/files/uploads/zip-9r.png (Zipping a directory tree)
|
||||
[6]: https://opensource.com/sites/default/files/uploads/unzip-l.png (Listing the archive file contents with unzip)
|
||||
[7]: https://opensource.com/sites/default/files/uploads/unzip-d.png (You can unzip into a destination path with -d)
|
||||
[8]: https://opensource.com/sites/default/files/uploads/unzip-j.png (Combining options with unzip)
|
||||
@ -1,155 +0,0 @@
|
||||
[#]: subject: (How to parse Bash program configuration files)
|
||||
[#]: via: (https://opensource.com/article/21/6/bash-config)
|
||||
[#]: author: (David Both https://opensource.com/users/dboth)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
How to parse Bash program configuration files
|
||||
======
|
||||
Separating config files from code enables anyone to change their
|
||||
configurations without any special programming skills.
|
||||
![bash logo on green background][1]
|
||||
|
||||
Keeping program configurations separate from code is important. It enables non-programmers to alter configurations without having to modify the program's code. With compiled binary executables, that would be impossible for non-programmers because it not only requires access to source files (which we do have with open source programs) but also a programmer's skill set. Few people have that, and most people don't want to learn.
|
||||
|
||||
With shell languages such as Bash, source code access is available by definition since shell scripts are not compiled into binary formats. Despite that openness, it is not a particularly good idea for non-programmers to root around in shell scripts and alter them. Even knowledgeable developers and sysadmins can make accidental changes that cause errors or worse.
|
||||
|
||||
So placing configuration items into easily maintained text files provides separation and allows non-programmers to edit configuration elements without the danger of making unintentional changes to the code. Many developers do this for programs written in compiled languages because they don't expect the users to be developers. For many of the same reasons, it also makes sense to do this with interpreted shell languages.
|
||||
|
||||
### The usual way
|
||||
|
||||
As with many other languages, you can write code for a Bash program that reads and parses ASCII text configuration files, reads the variable name, and sets values as the program code executes. For example, a configuration file might look like this:
|
||||
|
||||
|
||||
```
|
||||
var1=LinuxGeek46
|
||||
var2=Opensource.com
|
||||
```
|
||||
|
||||
The program would read that file, parse each line, and set the values into each variable.
|
||||
|
||||
### Sourcing
|
||||
|
||||
Bash uses a much easier method for parsing and setting variables called _sourcing_. Sourcing an external file from an executable shell program is a simple method for including the content of that file into a shell program in its entirety. In one sense, this is very much like compiled language `include` statements that include library files at runtime. Such a file can include any type of Bash code, including variable assignments.
|
||||
|
||||
As usual, it is easier to demonstrate than to explain.
|
||||
|
||||
First, create a `~/bin` directory (if it does not already exist), and make it the present working directory (PWD). The [Linux Filesystem Hierarchical Standard][2] defines `~/bin` as the appropriate place for users to store their executable files.
|
||||
|
||||
Create a new file in this directory. Name it `main` and make it executable:
|
||||
|
||||
|
||||
```
|
||||
[dboth@david bin]$ touch main
|
||||
[dboth@david bin]$ chmod +x main
|
||||
[dboth@david bin]$
|
||||
```
|
||||
|
||||
Add the following content to this executable file:
|
||||
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
Name="LinuxGeek"
|
||||
echo $Name
|
||||
```
|
||||
|
||||
And execute this Bash program:
|
||||
|
||||
|
||||
```
|
||||
[dboth@david bin]$ ./main
|
||||
LinuxGeek
|
||||
[dboth@david bin]$
|
||||
```
|
||||
|
||||
Create a new file and call it `~/bin/data`. This file does not need to be executable. Add the following information to it:
|
||||
|
||||
|
||||
```
|
||||
# Sourced code and variables
|
||||
echo "This is the sourced code from the data file."
|
||||
FirstName="David"
|
||||
LastName="Both"
|
||||
```
|
||||
|
||||
Add three lines to the `main` program so that it looks like this:
|
||||
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
Name="LinuxGeek"
|
||||
echo $Name
|
||||
source ~/bin/data
|
||||
echo "First name: $FirstName"
|
||||
echo "LastName: $LastName"
|
||||
```
|
||||
|
||||
Rerun the program:
|
||||
|
||||
|
||||
```
|
||||
[dboth@david bin]$ ./main
|
||||
LinuxGeek
|
||||
This is the sourced code from the data file.
|
||||
First name: David
|
||||
LastName: Both
|
||||
[dboth@david bin]$
|
||||
```
|
||||
|
||||
There is one more really cool thing to know about sourcing. You can use a single dot (`.`) as a shortcut for the `source` command. Change the `main` file to substitute the `.` in place of `source`:
|
||||
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
Name="LinuxGeek"
|
||||
echo $Name
|
||||
. ~/bin/data
|
||||
echo "First name: $FirstName"
|
||||
echo "LastName: $LastName"
|
||||
```
|
||||
|
||||
And run the program again. The result should be exactly the same as the previous run.
|
||||
|
||||
### Starting Bash
|
||||
|
||||
Every Linux host that uses Bash—which is pretty much all of them since Bash is the default shell for all distributions—includes some excellent, built-in examples of sourcing.
|
||||
|
||||
Whenever a Bash shell starts, its environment must be configured so that it is usable. There are five main files and one directory that are used to configure the Bash environment. They are listed here along with their main functions:
|
||||
|
||||
* `/etc/profile`: System-wide environment and startup programs
|
||||
* `/etc/bashrc`: System-wide functions and aliases
|
||||
* `/etc/profile.d/`: Directory that contains system-wide scripts for configuring various command-line tools such as `vim` and `mc` and any custom configuration scripts a sysadmin creates
|
||||
* `~/.bash_profile`: User-specific environment and startup programs
|
||||
* `~/.bashrc`: User-specific aliases and functions
|
||||
* `~/.bash_logout`: User-specific commands to execute when the user logs out
|
||||
|
||||
|
||||
|
||||
Try to trace the execution sequence through these files and determine which sequence it uses for a non-login Bash initialization versus a log-in Bash initialization. I did this in Chapter 17 of Volume 1 in my Linux training series, [_Using and administering Linux: Zero to sysadmin_][3].
|
||||
|
||||
I'll give you one hint. It all starts with the `~/.bashrc` script.
|
||||
|
||||
### Conclusion
|
||||
|
||||
This article explored sourcing for pulling code and variable assignments into a Bash program. This method of parsing variables from a configuration file is fast, easy, and flexible. It provides a method for separating Bash code from variable assignments to allow non-programmers to set the values of those variables.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/6/bash-config
|
||||
|
||||
作者：[David Both][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/dboth
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/bash_command_line.png?itok=k4z94W2U (bash logo on green background)
|
||||
[2]: http://refspecs.linuxfoundation.org/fhs.shtml
|
||||
[3]: http://www.both.org/?page_id=1183
|
||||
@ -2,7 +2,7 @@
|
||||
[#]: via: (https://opensource.com/article/21/6/groovy-parse-json)
|
||||
[#]: author: (Chris Hermansen https://opensource.com/users/clhermansen)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
[#]: via: (https://opensource.com/article/21/7/open-source-plantnet)
|
||||
[#]: author: (Don Watkins https://opensource.com/users/don-watkins)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
@ -0,0 +1,111 @@
|
||||
[#]: subject: (7 guides about open source to keep your brain busy this summer)
|
||||
[#]: via: (https://opensource.com/article/21/7/open-source-guides)
|
||||
[#]: author: (Seth Kenlon https://opensource.com/users/seth)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
7 guides about open source to keep your brain busy this summer
|
||||
======
|
||||
Download one of our free guides to build a Python-based video game, get
|
||||
organized with open source tools, work on your home automation project,
|
||||
or experiment with your Raspberry Pi homelab.
|
||||
![One chair on a sandy beach][1]
|
||||
|
||||
### Start a new Raspberry Pi project
|
||||
|
||||
The Raspberry Pi has been capturing the hearts and hands of open source enthusiasts for nearly a decade. There are endless options with what you can do with a Raspberry Pi, whether it's [monitoring your garden][2], [setting parental controls][3] (especially handy on those long summer days), or even [tracking aircraft][4] from your own backyard. If any of these cool projects perked your interest but your Raspberry Pi is still collecting dust, then you need to download our guide to nudge you to get started. Before you know it, you'll be working with so many Raspberry Pis, you'll need to learn [how to manage them all][5]!
|
||||
|
||||
**Download: [How to get started with your Raspberry Pi][6]**
|
||||
|
||||
### Design your open source smart home
|
||||
|
||||
One clever and useful way to use your Raspberry Pi is to design your own smart home. Using Home Assistant and other open source tools, your home can be automated on your own terms without a third-party conglomerate. Author [Steve Ovens][7] guides you through every step of the way with a collection of home automation found in this handy eBook.
|
||||
|
||||
**Download: [A practical guide to home automation using open source tools][7]**
|
||||
|
||||
### Get organized
|
||||
|
||||
Maybe you're not quite ready to automate everything in your home yet but are interested in sprucing up the place—your brain space. Why not start with your to-do list? With contributor [Kevin Sonney's][8] productivity guide, you'll become familiar with 6 open source tools to help you stay organized. Once you're through with his tutorials, you'll be organized and have time to finally relax this summer.
|
||||
|
||||
**Download: [6 open source tools for staying organized][9]**
|
||||
|
||||
### Learn how to code
|
||||
|
||||
Computers are everywhere. Wouldn't it be great to be able to talk their language?
|
||||
|
||||
There are lots of languages designed for beginners. Bash is the same scripting language used in the Linux and MacOS terminals, so it makes for a great start if you're new to code. You can [learn it interactively][10], and then download our eBook to learn more.
|
||||
|
||||
**Download:** **[An introduction to programming with Bash][11]**
|
||||
|
||||
### Build a video game with Python
|
||||
|
||||
Another programming language that beginners love is Python. It's equally as popular among school children just learning to code as it is with professional programmers [developing websites][12] and [video editors][13] or [automating the cloud][14]. Regardless of what your eventual goal happens to be, a fun way to start with Python is by programming your own video game.
|
||||
|
||||
**Download: [A guide to building a video game with Python][15]**
|
||||
|
||||
### Discover clever ways to use Jupyter
|
||||
|
||||
To make Python interactive and easy to share, the Jupyter project provides a web-based development environment. You can write your code in "notebook" files, and then send them to other users to make it easy for them to replicate and visualize what you've done. It's the perfect combination of code, documentation, and presentation, and it's surprisingly flexible. Learn more by downloading Moshe Zadka's multifaceted tour through Jupyter.
|
||||
|
||||
**Download: [6 surprising ways to use Jupyter][16]**
|
||||
|
||||
### Experiment with Kubernetes in your homelab
|
||||
|
||||
So you've installed Linux on your Pi, you've logged in, you've set up a new user and [configured sudo][17] to make administration painless, and you're [running all the services][18] you need. Now what?
|
||||
|
||||
If you're comfortable with Linux and server administration, the next step for you could be The Cloud. Get Chris Collins' eBook to learn all about containers, pods, and clusters, all from the comfort of your own homelab.
|
||||
|
||||
**Download: [Running Kubernetes on your Raspberry Pi homelab][19]**
|
||||
|
||||
### Bonus: Book lists
|
||||
|
||||
All work and no play isn't healthy. Summer (or whatever season it is in your part of the world) vacation is meant to be relaxing, and there's nothing quite like sitting back on the porch or beach with a good book. Here are a few recent book lists to spark some ideas:
|
||||
|
||||
* [8 books for open source technologists (2021)][20]
|
||||
* [A dozen books for all ages][21]
|
||||
* [8 leadership books for self-improvement][22]
|
||||
* [6 must-read books for aspiring Cloud Architects][23]
|
||||
* Our very first [book list from 2010][24]
|
||||
|
||||
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/7/open-source-guides
|
||||
|
||||
作者：[Seth Kenlon][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/seth
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/chen-mizrach-unsplash.jpg?itok=S_wIO5e8 (One chair on a sandy beach)
|
||||
[2]: https://opensource.com/article/21/5/monitor-greenhouse-open-source
|
||||
[3]: https://opensource.com/article/21/3/raspberry-pi-parental-control
|
||||
[4]: https://opensource.com/article/21/3/tracking-flights-raspberry-pi
|
||||
[5]: https://opensource.com/article/21/5/raspberry-pi-cockpit
|
||||
[6]: https://opensource.com/downloads/raspberry-pi-guide
|
||||
[7]: https://opensource.com/downloads/home-automation-ebook
|
||||
[8]: https://opensource.com/users/ksonney
|
||||
[9]: https://opensource.com/downloads/organization-tools
|
||||
[10]: https://opensource.com/article/19/10/learn-bash-command-line-games#bashcrawl
|
||||
[11]: https://opensource.com/downloads/bash-programming-guide
|
||||
[12]: https://opensource.com/article/18/4/flask
|
||||
[13]: https://opensource.com/article/21/2/linux-python-video
|
||||
[14]: https://opensource.com/article/19/2/quickstart-guide-ansible
|
||||
[15]: https://opensource.com/downloads/python-gaming-ebook
|
||||
[16]: https://opensource.com/downloads/jupyter-guide
|
||||
[17]: https://opensource.com/article/19/10/know-about-sudo
|
||||
[18]: https://opensource.com/article/20/5/systemd-units
|
||||
[19]: https://opensource.com/downloads/kubernetes-raspberry-pi
|
||||
[20]: https://opensource.com/article/21/6/2021-opensourcecom-summer-reading-list
|
||||
[21]: https://opensource.com/article/20/6/summer-reading-list
|
||||
[22]: https://enterprisersproject.com/article/2021/5/8-leadership-books-self-improvement
|
||||
[23]: https://www.redhat.com/architect/books-cloud-architects
|
||||
[24]: https://opensource.com/life/10/8/open-books-opensourcecom-summer-reading-list
|
||||
@ -0,0 +1,158 @@
|
||||
[#]: subject: (Enter invisible passwords using this Python module)
|
||||
[#]: via: (https://opensource.com/article/21/7/invisible-passwords-python)
|
||||
[#]: author: (Seth Kenlon https://opensource.com/users/seth)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
Enter invisible passwords using this Python module
|
||||
======
|
||||
Give your password an extra layer of security with GPG and the Python
|
||||
getpass module.
|
||||
![Password lock][1]
|
||||
|
||||
Passwords are particularly problematic for programmers. You're not supposed to store them without encrypting them, and you're not supposed to reveal what's been typed when your user enters one. This became particularly important to me when I decided I wanted to boost security on my laptop. I encrypt my home directory—but once I log in, any password stored as plain text in a configuration file is potentially exposed to prying eyes.
|
||||
|
||||
Specifically, I use an application called [Mutt][2] as my email client. It lets me read and compose emails in my Linux terminal, but normally it expects a password in its configuration file. I restricted permissions on my Mutt config file so that only I can see it, but I'm the only user of my laptop, so I'm not really concerned about authenticated users inadvertently looking at my configs. Instead, I wanted to protect myself from absent-mindedly posting my config online, either for bragging rights or version control, with my password exposed. In addition, although I have no expectations of unwelcome guests on my system, I did want to ensure that an intruder couldn't obtain my password just by running `cat` on my config.
|
||||
|
||||
### Python GnuPG
|
||||
|
||||
The Python module `python-gnupg` is a Python wrapper for the `gpg` application. The module's name is `python-gnupg`, which you must not confuse with a module called `gnupg`.
|
||||
|
||||
[GnuPG][3] (GPG) is the default encryption system for Linux, and I've been using it since 2009 or so. I feel comfortable with it and have a high level of trust in its security.
|
||||
|
||||
I decided that the best way to get my password into Mutt was to store my password inside an encrypted GPG file, create a prompt for my GPG password to unlock the encrypted file, and hand the password over to Mutt (actually to the `offlineimap` command, which I use to synchronize my laptop with the email server).
|
||||
|
||||
[Getting user input with Python][4] is pretty easy. You make a call to `input`, and whatever the user types is stored as a variable:
|
||||
|
||||
|
||||
```
|
||||
print("Enter password: ")
|
||||
myinput = input()
|
||||
|
||||
print("You entered: ", myinput)
|
||||
```
|
||||
|
||||
My problem was when I typed a password into the terminal in response to my password prompt, everything I typed was visible to anyone looking over my shoulder or scrolling through my terminal history:
|
||||
|
||||
|
||||
```
|
||||
$ ./test.py
|
||||
Enter password: my-Complex-Passphrase
|
||||
```
|
||||
|
||||
### Invisible password entry with getpass
|
||||
|
||||
As is often the case, there's a Python module that's already solved my problem. The module is `getpass4`, and from the user's perspective, it behaves exactly like `input` except without displaying what the user is typing.
|
||||
|
||||
You can install both modules with [pip][5]:
|
||||
|
||||
|
||||
```
|
||||
$ python -m pip install --user \
|
||||
python-gnupg getpass4
|
||||
```
|
||||
|
||||
Here's my Python script to create a password prompt:
|
||||
|
||||
|
||||
```
|
||||
#!/usr/bin/env python
|
||||
# by Seth Kenlon
|
||||
# GPLv3
|
||||
|
||||
# install deps:
|
||||
# python3 -m pip install --user python-gnupg getpass4
|
||||
|
||||
import gnupg
|
||||
import getpass
|
||||
from pathlib import Path
|
||||
|
||||
def get_api_pass():
|
||||
homedir = str(Path.home())
|
||||
gpg = gnupg.GPG(gnupghome=os.path.join(homedir,".gnupg"), use_agent=True)
|
||||
passwd = getpass.getpass(prompt="Enter your GnuPG password: ", stream=None)
|
||||
|
||||
with open(os.path.join(homedir,'.mutt','pass.gpg'), 'rb') as f:
|
||||
apipass = (gpg.decrypt_file(f, passphrase=passwd))
|
||||
|
||||
f.close()
|
||||
|
||||
return str(apipass)
|
||||
|
||||
if __name__ == "__main__":
|
||||
apipass = get_api_pass()
|
||||
print(apipass)
|
||||
```
|
||||
|
||||
Save the file as `password_prompt.py` if you want to try it out. If you're using `offlineimap` and want to use this solution for your own password entry, then save it to some location you can point `offlineimap` to in your `.offlineimaprc` file (I use `~/.mutt/password_prompt.py`).
|
||||
|
||||
### Testing the password prompt
|
||||
|
||||
To see the script in action, you first must create an encrypted file (I'll assume that you already have GPG set up):
|
||||
|
||||
|
||||
```
|
||||
$ echo "hello world" > pass
|
||||
$ gpg --encrypt pass
|
||||
$ mv pass.gpg ~/.mutt/pass.gpg
|
||||
$ rm pass
|
||||
```
|
||||
|
||||
Now run the Python script:
|
||||
|
||||
|
||||
```
|
||||
$ python ~/.mutt/password_prompt.py
|
||||
Enter your GPG password:
|
||||
hello world
|
||||
```
|
||||
|
||||
Nothing displays as you type, but as long as you enter your GPG passphrase correctly, you will see the test message.
|
||||
|
||||
### Integrating the password prompt with offlineimap
|
||||
|
||||
I needed to integrate my new prompt with the `offlineimap` command. I chose Python for this script because I knew that `offlineimap` can make calls to Python applications. If you're an `offlineimap` user, you'll appreciate that the only "integration" required is changing two lines in your `.offlineimaprc` file.
|
||||
|
||||
First, add a line referencing the Python file:
|
||||
|
||||
|
||||
```
|
||||
`pythonfile = ~/.mutt/password_prompt.py`
|
||||
```
|
||||
|
||||
And then replace the `remotepasseval` line in `.offlineimaprc` with a call to the `get_api_pass()` function in `password_prompt.py`:
|
||||
|
||||
|
||||
```
|
||||
`remotepasseval = get_api_pass()`
|
||||
```
|
||||
|
||||
No more passwords in your config file!
|
||||
|
||||
### Security matters
|
||||
|
||||
It sometimes feels almost paranoid to think about security minutiae on your personal computer. Does your SSH config really need to be restricted to 600? Does it really matter that your email password is in an inconsequential config file buried within a hidden folder called, of all things, `.mutt`? Probably not.
|
||||
|
||||
And yet knowing that I don't have sensitive data quietly hidden away in my config files makes it a lot easier for me to commit files to public Git repositories, to copy and paste snippets into support forums, and to share my knowledge in the form of actual, known-good configuration files. For that alone, improved security has made my life easier. And with so many great Python modules available to help, it's easy to implement.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/7/invisible-passwords-python
|
||||
|
||||
作者：[Seth Kenlon][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/seth
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/password.jpg?itok=ec6z6YgZ (Password lock)
|
||||
[2]: http://www.mutt.org/
|
||||
[3]: https://gnupg.org/
|
||||
[4]: https://opensource.com/article/20/12/learn-python
|
||||
[5]: https://opensource.com/article/19/11/python-pip-cheat-sheet
|
||||
@ -0,0 +1,116 @@
|
||||
[#]: subject: (How I avoid breaking functionality when modifying legacy code)
|
||||
[#]: via: (https://opensource.com/article/21/7/legacy-code)
|
||||
[#]: author: (Alex Bunardzic https://opensource.com/users/alex-bunardzic)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
How I avoid breaking functionality when modifying legacy code
|
||||
======
|
||||
Extract methods give the biggest bang for the buck when it comes to
|
||||
modifying legacy code while avoiding the risk of breaking the
|
||||
functionality.
|
||||
![Coding on a computer][1]
|
||||
|
||||
Allow me a bit of introspection. I've been working in the software engineering field for 31 years. During those 31 years, I've modified a lot of legacy software.
|
||||
|
||||
Over time, I've formed certain habits when working with legacy code. Because on most projects I get paid to deliver working software that is easy to maintain, I cannot afford the luxury of taking my sweet time trying to fully understand the legacy code I am about to modify. So, I tend to skim. Skimming the code helps me quickly identify relevant portions in the repo. It is a race against time, and I don't have cycles at my disposal to dwell on less-relevant minutia. I'm always going for the most relevant area in the code. Once I find it, I slow down and start analyzing it.
|
||||
|
||||
I rely heavily on my power tools—integrated development environments (IDEs). It doesn't matter which power tool; these days, they're all pretty much capable of doing the same thing. What's important to me is having the ability to quickly find where functions are called and where variables are used.
|
||||
|
||||
Sooner or later, after I'm done skimming the code and analyzing the code segment I intend to change, I identify a place where I want to insert some code. Now that I understand the meaning of the classes, components, and objects involved in performing the function, I write a test first.
|
||||
|
||||
After that, I write code to make the test pass. I type the name of the object I intend to use and then press the dot key (**.**) and the IDE responds by giving me a full list of methods defined for that object. All those methods are callable from the location where my cursor is.
|
||||
|
||||
I then pick the method that makes sense to me. I fill in the blanks (that is, I supply values for the expected arguments/parameters), save the change, and run the test. If the test passes, I'm done with that micro change.
|
||||
|
||||
I typically repeat this activity many times per hour. Throughout the workday, it is not unusual to see it repeated dozens, even hundreds of times.
|
||||
|
||||
I believe the way I modify software is not unique to my work habits. I think it describes a typical flow that many (I'd even say most) software engineers adhere to.
|
||||
|
||||
### A few observations
|
||||
|
||||
The first thing apparent in this way of modifying legacy software is the absence of any work on documentation. Experience shows that software developers very rarely spend time reaching out for documentation. Time spent preparing the documentation and generating it to produce HTML-style online documents is often wasted.
|
||||
|
||||
Instead, most developers rely solely upon power tools. And rightly so—IDEs never lie, as they always offer the real-time picture of the system they are modifying, and documentation is usually stale.
|
||||
|
||||
Another thing is that developers don't read the source code the way it was written. When writing code from scratch (first pass), many developers tend to write long functions. Source code tends to bunch up. Bunching code up makes it easier to read and reason about on the first pass and debug. But after the first pass, people rarely, if ever, consume the code the way it was written. If we catch ourselves reading a whole function from beginning to end, it is most likely because we have exhausted all other options and have no choice but to slow down and read the code in a pedestrian way. However, in my experience, that slow and orderly reading of the code seldom happens.
|
||||
|
||||
### Problems caused by bunched-up code
|
||||
|
||||
If you were to leave the code as it was written during the first pass (i.e., long functions, a lot of bunched-up code for easy initial understanding and debugging), it would render IDEs powerless. If you cram all capabilities an object can offer into a single, giant function, later, when you're trying to utilize that object, IDEs will be of no help. IDEs will show the existence of one method (which will probably contain a large list of parameters providing values that enforce the branching logic inside that method). So, you won't know how to really use that object unless you open its source code and read its processing logic very carefully. And even then, your head will probably hurt.
|
||||
|
||||
Another problem with hastily cobbled-up, "bunched-up" code is that its processing logic is not testable. While you can still write an end-to-end test for that code (input values and the expected output values), you have no way of knowing if the bunched-up code is doing any other potentially risky processing. Also, you have no way of testing for edge cases, unusual scenarios, difficult-to-reproduce scenarios, etc. That renders your code untestable, which is a very bad thing to live with.
|
||||
|
||||
### Break up bunched-up code by extracting methods
|
||||
|
||||
Long functions or methods are always a sign of muddled thinking. When a block of code contains numerous statements, it usually means it is doing way too much processing. Cramming a lot of processing in one place typically means the developer hasn't carefully thought things through.
|
||||
|
||||
You don't need to look further than into how companies are typically organized. Instead of having hundreds of employees working in a single department, companies tend to break up into numerous smaller departments. That way, it is much clearer where responsibilities lie.
|
||||
|
||||
Software code is no different. An application exists to automate a lot of intricate processing. Processing gets broken into multiple smaller steps, so each step must be mapped onto a separate, isolated block of code. You create such separate, isolated, and autonomous blocks of code by extracting methods. You take a long, bulky block of code and break it up by extracting responsibilities into separate blocks of code.
|
||||
|
||||
### Extracted methods enable better naming
|
||||
|
||||
Developers write software code, but it is much more often consumed (i.e., read) by developers than written.
|
||||
|
||||
When consuming software code, it helps if the code is expressive. Expressiveness boils down to proper structure and proper naming. Consider the following statement:
|
||||
|
||||
|
||||
```
|
||||
`if((x && !y) && !b) || (b && y) && !(z >= 65))`
|
||||
```
|
||||
|
||||
It would be literally impossible to understand the meaning and the intention of this statement without running the code and stepping through it with a debugger. Such activity is called GAK (Geek at Keyboard). It is 100% unproductive and quite wasteful.
|
||||
|
||||
Here is where the extract method and proper naming practices come to the rescue. Take the complex statement contained within the `if` statement, extract it into its own method, and give that method a meaningful name. For example:
|
||||
|
||||
|
||||
```
|
||||
public bool IsEligible(bool b, bool x, bool y, int z) {
|
||||
return ((x && !y) && !b) || (b && y) && !(z >= 65);
|
||||
}
|
||||
```
|
||||
|
||||
Now replace the ugly `if` statement with a more readable statement:
|
||||
|
||||
|
||||
```
|
||||
`if(IsEligible(b, x, y, z))`
|
||||
```
|
||||
|
||||
Of course, you should also replace dumb one-character variable names with more meaningful names to improve readability.
|
||||
|
||||
### Reusing legacy code
|
||||
|
||||
Experience shows that any functionality that is not extracted and properly named and moved to the most reasonable class will never be reused. The extract method fosters frequent reuse, which goes a long way toward improving code quality.
|
||||
|
||||
### Testing legacy code
|
||||
|
||||
Writing tests for existing code is hard and feels less rewarding than doing [test-driven development][2] (TDD). Even after you determine that there should be several tests to ensure production code works as expected, when you realize production code must be changed to enable testing, you often decide to skip writing tests. In such situations, achieving your goal to deliver testable code, slowly but surely, keeps diminishing.
|
||||
|
||||
Writing tests for legacy code is tedious because it often requires a lot of time and code to set up the preconditions. That's the opposite of how you write tests when doing TDD, where time spent writing preconditions is minimal.
|
||||
|
||||
The best way to make legacy code testable is to practice the extract method approach. Locating a block of code nested in loops and conditionals and extracting it enables you to write small, precise tests. Such tests on extracted functions improve not only the testability of the code but also the understandability. If legacy code becomes more understandable thanks to extracting methods and writing legible tests, the chance of introducing defects is drastically reduced.
|
||||
|
||||
### Conclusion
|
||||
|
||||
Most of the discussion about extracting methods would not be necessary with TDD. Writing one test first, then making the test pass, then scanning that code for more insights into how the code should be structured and improved, making improvements, and finally making changes to part of the code base guarantees there will be no need to worry about extracting methods. Since legacy code usually means code that was not crafted with TDD methodology, you are forced to adopt a different approach. In my experience, extract methods give the biggest bang for the buck when it comes to modifying legacy code while avoiding the risk of breaking the functionality.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/7/legacy-code
|
||||
|
||||
作者：[Alex Bunardzic][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/alex-bunardzic
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/code_computer_laptop_hack_work.png?itok=aSpcWkcl (Coding on a computer)
|
||||
[2]: https://opensource.com/article/20/1/test-driven-development
|
||||
@ -0,0 +1,168 @@
|
||||
[#]: subject: (Things to do after installing Fedora 34 Workstation)
|
||||
[#]: via: (https://fedoramagazine.org/things-to-do-after-installing-fedora-34-workstation/)
|
||||
[#]: author: (Arman Arisman https://fedoramagazine.org/author/armanwu/)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
Things to do after installing Fedora 34 Workstation
|
||||
======
|
||||
|
||||
![][1]
|
||||
|
||||
Using a new operating system can be a lot of fun. But it often becomes confusing when we first use it, especially for new users who are not very familiar with computer systems. For those of you who are using Fedora for the first time and have successfully installed Fedora 34 Workstation, this article can be an initial guide. I’m sure that you want to feel more at home with your new fresh Fedora. These are several things to do after installing your Fedora 34 Workstation.
|
||||
|
||||
### System update
|
||||
|
||||
Maybe you think that you have installed the most recent version of Fedora 34 Workstation, so your Fedora should be up to date. But you still have to make sure that your Fedora Linux has all the updated packages. Because in every new release of an operating system, usually there are still many things that continue to be improved. You can use the terminal or GNOME software to run the update.
|
||||
|
||||
If you want to update via the terminal, then you just have to open a terminal and type the following command.
|
||||
|
||||
```
|
||||
$ sudo dnf update
|
||||
```
|
||||
|
||||
But if you want to do it with GNOME _Software_, open the application by selecting _Activities_ then locating and selecting the _Software_ item in the taskbar at the bottom of the screen. When it opens select the _Update_s tab at the top. After that you just click the _Download_ button. An update may require a restart afterwards and _Update_ will tell you that.
|
||||
|
||||
![GNOME Software location in the taskbar at the bottom of the screen][2]
|
||||
|
||||
_note: another way to select Activities is to press the super key on the keyboard. Super key is the button that has the Windows logo on most keyboards._
|
||||
|
||||
![Gnome Software showing Updates][3]
|
||||
|
||||
### System settings
|
||||
|
||||
You can view and configure your device’s system through _System Settings_. These include items like network, keyboard, mouse, sound, displays, etc. You can run it by pressing the _super_ key on your keyboard, clicking _Show Applications_ in the task bar at the bottom of the window, then selecting _Settings_. Configure it according to your needs.
|
||||
|
||||
![Settings menu showing Network selected][4]
|
||||
|
||||
### Additional repositories
|
||||
|
||||
Maybe some packages you need are not available to be installed from the official Fedora Repository. You can add software repositories with the _dnf config-manager_ command. Please be careful if you want to add other repositories besides the official Fedora repository.
|
||||
|
||||
The first thing you should do is define a new repository by adding a new file ending in _.repo_ to the _/etc/yum.repos.d/_ directory. Run the following command in the terminal.
|
||||
|
||||
```
|
||||
$ sudo dnf config-manager --add-repo /etc/yum.repos.d/file_name.repo
|
||||
```
|
||||
|
||||
_note: replace file_name with the repository file name._
|
||||
|
||||
Or you can use GNOME _Software_. Open it as described in the System Update section above. Now select the “hamburger” icon (three horizontal lines) on the top right and select _Software Repositories_. You can add the repository from there using the _Install_ option.
|
||||
|
||||
![GNOME Software showing location of Software Repositories menu][5]
|
||||
|
||||
Most people will enable RPM Fusion. It’s a third party repository. You can read about third party repositories in [Fedora Docs][6].
|
||||
|
||||
### Fastest mirror and Delta RPM
|
||||
|
||||
There are several things you can do to speed up your download times when using DNF to update your system. You can enable Fastest Mirror and Delta RPM. Edit _/etc/dnf/dnf.conf_ using a text editor, such as gedit or nano. Here’s the example to open _dnf.conf_ file with _nano_ in _terminal_.
|
||||
|
||||
```
|
||||
$ sudo nano /etc/dnf/dnf.conf
|
||||
```
|
||||
|
||||
Append the following line onto your _dnf.conf_ file.
|
||||
|
||||
```
|
||||
fastestmirror=true
|
||||
deltarpm=true
|
||||
```
|
||||
|
||||
Press _ctrl+o_ to save the file then _ctrl+x_ to quit from _nano_.
|
||||
|
||||
### Multimedia plugins for audio and video
|
||||
|
||||
You may need some plugins for your multimedia needs. You can install multimedia plugins by running this command in a terminal.
|
||||
|
||||
```
|
||||
$ sudo dnf group upgrade --with-optional Multimedia
|
||||
```
|
||||
|
||||
Please pay attention to the regulations and standards in your country regarding multimedia codecs. You can read about this in [Fedora Docs][7].
|
||||
|
||||
### Tweaks and Extentions
|
||||
|
||||
Fedora 34 Workstation comes with GNOME as the default Desktop Environment. We can do various configurations of GNOME by using Tweaks and Extensions, like changing themes, changing buttons in the window dialog, and many more.
|
||||
|
||||
Open your terminal and run this command to install GNOME Tweaks.
|
||||
|
||||
```
|
||||
$ sudo dnf install gnome-tweaks
|
||||
```
|
||||
|
||||
And run this command to install GNOME Extensions.
|
||||
|
||||
```
|
||||
$ sudo dnf install gnome-extensions-app
|
||||
```
|
||||
|
||||
Do the same way as above when you search for _GNOME Software_. Select _Activities_ or press the _super_ key then select _Show Applications_ to see a list of installed applications. You can find both applications in the list. You can do the same thing every time you want to search for installed applications. Then do the configuration with your preferences with _Tweaks_ and _Extensions_.
|
||||
|
||||
![GNOME Tweaks][8]
|
||||
|
||||
![GNOME Extensions][9]
|
||||
|
||||
### Install applications
|
||||
|
||||
When you first install Fedora, you will find several installed apps. You can add other applications according to your needs with GNOME Software. Do the same way to open GNOME Software as described earlier. Then find the application you want, select the application, and then press the Install button.
|
||||
|
||||
![GNOME Software][10]
|
||||
|
||||
Or you can do it with terminal. Here are the commands to find and install the application.
|
||||
|
||||
Command to search for available applications:
|
||||
|
||||
```
|
||||
$ sudo dnf search application_name
|
||||
```
|
||||
|
||||
The command to install the application:
|
||||
|
||||
```
|
||||
$ sudo dnf install application_name
|
||||
```
|
||||
|
||||
Commands to remove installed applications:
|
||||
|
||||
```
|
||||
$ sudo dnf remove application_name
|
||||
```
|
||||
|
||||
_note: replace application_name with the name of the application._
|
||||
|
||||
You can search for installed applications by viewing them in _Show Applications_. Select _Activities_ or press the _super_ key and select _Show Applications_. Then you can select the application you want to run from the list.
|
||||
|
||||
![Installed application list][11]
|
||||
|
||||
### Conclusion
|
||||
|
||||
Fedora Workstation is an easy-to-use and customizable operating system. There are many things you can do after installing Fedora 34 Workstation according to your needs. This article is just a basic guide for your first steps before you have more fun with your Fedora Linux system. You can read [Fedora Docs][12] for more detailed information. I hope you enjoy using Fedora Linux.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://fedoramagazine.org/things-to-do-after-installing-fedora-34-workstation/
|
||||
|
||||
作者：[Arman Arisman][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://fedoramagazine.org/author/armanwu/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://fedoramagazine.org/wp-content/uploads/2021/07/FedoraMagz-Cover_ThingsToDo.png
|
||||
[2]: https://fedoramagazine.org/wp-content/uploads/2021/07/GNOME_Software_location-1024x576.png
|
||||
[3]: https://fedoramagazine.org/wp-content/uploads/2021/07/Software_Updates-1024x735.png
|
||||
[4]: https://fedoramagazine.org/wp-content/uploads/2021/07/Settings-1024x764.png
|
||||
[5]: https://fedoramagazine.org/wp-content/uploads/2021/07/Software_Hamburger_-1-1024x685.png
|
||||
[6]: https://docs.fedoraproject.org/en-US/quick-docs/setup_rpmfusion/
|
||||
[7]: https://docs.fedoraproject.org/en-US/quick-docs/assembly_installing-plugins-for-playing-movies-and-music/
|
||||
[8]: https://fedoramagazine.org/wp-content/uploads/2021/07/Tweaks-1024x733.png
|
||||
[9]: https://fedoramagazine.org/wp-content/uploads/2021/07/GNOME_Extensions.png
|
||||
[10]: https://fedoramagazine.org/wp-content/uploads/2021/07/GNOME_Software-1-1024x687.png
|
||||
[11]: https://fedoramagazine.org/wp-content/uploads/2021/07/Show_Application-1024x576.png
|
||||
[12]: https://docs.fedoraproject.org/en-US/fedora/f34/
|
||||
@ -0,0 +1,158 @@
|
||||
[#]: subject: (Element: A Cross-Platform Decentralized Open-Source Messaging App)
|
||||
[#]: via: (https://itsfoss.com/element/)
|
||||
[#]: author: (Ankush Das https://itsfoss.com/author/ankush/)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
Element: A Cross-Platform Decentralized Open-Source Messaging App
|
||||
======
|
||||
|
||||
There are many open-source messaging applications available, especially if you are looking for [WhatsApp replacements][1] on both desktop and mobile.
|
||||
|
||||
Element is one of them, which is a decentralized alternative for private messaging that you can use to interact with individuals, communities, or businesses.
|
||||
|
||||
### Element: Privacy-Friendly Open-Source Messenger Built on Matrix Network
|
||||
|
||||
![][2]
|
||||
|
||||
Matrix is an open standard for secure and decentralized communication. And Element is the messaging client that uses that.
|
||||
|
||||
Element is also a part of the Matrix.org Foundation — so you will find most of the same team responsible for this.
|
||||
|
||||
Originally, it was known as [Riot][3], which we covered back then. But, after the [rebranding][4], it is now known as “Element”, which comes with an improved user experience and constantly focusing on making decentralized communication common for instant messaging.
|
||||
|
||||
![][5]
|
||||
|
||||
Element is not just another open-source messenger, it gives you the ability to do a lot of things.
|
||||
|
||||
Here, let me highlight some of the key features along with some details about it that follows as you read on.
|
||||
|
||||
### Features of Element
|
||||
|
||||
![][6]
|
||||
|
||||
Element is more of an all-in-one messenger than a replacement of something. You could choose it as an [open-source alternative to Slack][7] or a private alternative to any instant messenger like Telegram.
|
||||
|
||||
Some of the options that you get with it are:
|
||||
|
||||
* End-to-End encryption chat room
|
||||
* Public communities (may not be encrypted)
|
||||
* Direct voice call
|
||||
* Conference call in the community
|
||||
* Meet Jitsi integration (one of the [open-source alternatives to Zoom][8])
|
||||
* File sharing
|
||||
* Emoji and Sticker support
|
||||
* Moderation tools for managing communities
|
||||
* Extensive anti-spam options
|
||||
* Ability to bridge other services like Slack, Discord, IRC, and more
|
||||
* Offers paid managed hosting to have control over your data
|
||||
* Cross-signed device verification for message privacy/security
|
||||
* Fine grained notification settings
|
||||
* Email notifications
|
||||
* Ability to restore using encryption keys
|
||||
* Make yourself discoverable to the entire Matrix network using your email or number
|
||||
|
||||
|
||||
|
||||
The features offered by Element may sound to be overwhelming for a user who just wants private messaging.
|
||||
|
||||
But fortunately, all those features do not get in the way unless you explicitly access/configure them. So that’s a good thing.
|
||||
|
||||
First, let me address the installation instructions for Linux and I’ll give you some insights on how my experience with Element was (on both Linux desktop and Android).
|
||||
|
||||
### Installing Element in Linux
|
||||
|
||||
Element officially supports Debian/Ubuntu for installation. You can just add the package repository and install element.
|
||||
|
||||
The commands used for this is:
|
||||
|
||||
```
|
||||
sudo apt install -y wget apt-transport-https
|
||||
|
||||
sudo wget -O /usr/share/keyrings/riot-im-archive-keyring.gpg https://packages.riot.im/debian/riot-im-archive-keyring.gpg
|
||||
|
||||
echo "deb [signed-by=/usr/share/keyrings/riot-im-archive-keyring.gpg] https://packages.riot.im/debian/ default main" | sudo tee /etc/apt/sources.list.d/riot-im.list
|
||||
|
||||
sudo apt update
|
||||
|
||||
sudo apt install element-desktop
|
||||
```
|
||||
|
||||
Do note that they are still using Riot.im domain to host packages even after rebranding — so not to be confused with the older Riot messaging app.
|
||||
|
||||
You can also find it in AUR for Arch-based distros — but I’m not quite sure about how well it works.
|
||||
|
||||
Unfortunately, there’s no [Flatpak][9] or [Snap][10] package available. So, if you are using a distribution that isn’t officially supported by Element, the best place to explore solutions/raise issues will be their [GitHub page][11].
|
||||
|
||||
Now, before you get started using it, let me give you some heads up with my thoughts on it.
|
||||
|
||||
### Element on Linux and Android: Here’s What You Need to Know
|
||||
|
||||
To start with — the user experience is fantastic on both Android and desktop. I tried it on Linux Mint, and it worked flawlessly.
|
||||
|
||||
You do not need a mobile number to sign up. Just create a username and add an email account to it, and you’re done.
|
||||
|
||||
![][12]
|
||||
|
||||
One can opt for a paid homeserver (your own matrix network) or just join the free Matrix homeserver offered.
|
||||
|
||||
**Keep in mind,** if you are signing up for free, you may not get to experience all the features — like the ability to see who’s online. You can only do that with your own server, the free Matrix server restricts certain functionalities like that to be able to accommodate an unlimited number of free users.
|
||||
|
||||
When signing in to a mobile device, you will have to verify the session by scanning a QR code prompted on Element’s desktop app.
|
||||
|
||||
Once done, you can explore and join public communities available or create your own.
|
||||
|
||||
Most of the existing public communities do not have end-to-end encryption enabled. So make sure you know what you are doing before messaging in any of the public communities.
|
||||
|
||||
While Element supports bridging IRC, Slack, and others or adding bots to a community — it is just not supported for an encrypted community. So, you need to have an unencrypted community to be able to use bots and bridges.
|
||||
|
||||
![][13]
|
||||
|
||||
A **word of caution**:
|
||||
|
||||
Element is getting popular, and scammers/spammers are attracted to the platform because it does not need any valuable personal information to get started.
|
||||
|
||||
So **make sure that you do not trust anyone and keep your identity safe** by not using your real profile picture or work email, especially if you are joining the public communities.
|
||||
|
||||
Element is constantly improving and offers plenty of features for several use-cases. I don’t see a problem with it being an open-source Discord replacement as well (in some way).
|
||||
|
||||
I was impressed with the level of notification controls that it gives and an added email notification option (which is enabled by default). You can choose to have notifications based on the keywords that you find interesting, what an exciting feature to have!
|
||||
|
||||
![][14]
|
||||
|
||||
Overall, Element may not be the perfect replacement for everything you use right now but it is shaping up to be an all-in-one alternative to many proprietary options.
|
||||
|
||||
I’ve had a good experience with Element so far and I’m confident about its future. What do you think? Willing to try Element on Linux?
|
||||
|
||||
Feel free to let me know your thoughts on this.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://itsfoss.com/element/
|
||||
|
||||
作者：[Ankush Das][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://itsfoss.com/author/ankush/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://itsfoss.com/private-whatsapp-alternatives/
|
||||
[2]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2021/06/element-io.png?resize=800%2C531&ssl=1
|
||||
[3]: https://itsfoss.com/riot-desktop/
|
||||
[4]: https://itsfoss.com/riot-to-element/
|
||||
[5]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2021/06/element-ui.png?resize=800%2C602&ssl=1
|
||||
[6]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2021/06/element-settings.png?resize=800%2C673&ssl=1
|
||||
[7]: https://itsfoss.com/open-source-slack-alternative/
|
||||
[8]: https://itsfoss.com/open-source-video-conferencing-tools/
|
||||
[9]: https://itsfoss.com/what-is-flatpak/
|
||||
[10]: https://itsfoss.com/install-snap-linux/
|
||||
[11]: https://github.com/vector-im
|
||||
[12]: https://i2.wp.com/itsfoss.com/wp-content/uploads/2021/06/element-sign-in.png?resize=800%2C581&ssl=1
|
||||
[13]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2021/06/element-bridge-bots.png?resize=800%2C517&ssl=1
|
||||
[14]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2021/06/element-notifications.png?resize=800%2C547&ssl=1
|
||||
36
translated/talk/20210508 My weird jobs before tech.md
Normal file
36
translated/talk/20210508 My weird jobs before tech.md
Normal file
@ -0,0 +1,36 @@
|
||||
[#]: subject: (My weird jobs before tech)
|
||||
[#]: via: (https://opensource.com/article/21/5/weird-jobs-tech)
|
||||
[#]: author: (Chris Hermansen https://opensource.com/users/clhermansen)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (MM-BCY)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
我在科技行业之前的奇怪工作
|
||||
======
|
||||
你永远不会知道从你的第一份工作到现在你会去哪里。！
|
||||
在我加入技术部之前，我做过一些奇怪的工作。
|
||||
我是一家飞机修理店的初级助理，这意味着我的任务就像是清洗溶剂中的脏金属零件这样(哇，70年代的情况可不一样了)。我在那里最有趣的任务是在一架正在修理中的漂亮的老式比奇飞机倾斜机翼的木制副翼和水平稳定器上熨烫涤纶飞机的布料。
|
||||
在大学期间的一个夏天，，我在同一个机场的一个团队工作，混合了阻燃剂，然后把它注入灭火飞机(“[水弹][2]”)。那可能是我做过的最脏的工作了，但是给飞机装货还是挺酷的。有一个小皮瓣约两米离地面，你可以把你的手指插入后，连接灌装软管的耦合。然后泵上的人启动泵。当你觉得你的手指湿了，你挥手让泵主停止泵。与此同时，在你前方几米处，右侧的辐射状引擎噪音极大，螺旋桨吹掉了你身上因混合阻燃剂而积聚的红色粉尘。如果你搞砸了，让飞机装得太满，他们就得滑到一块地方，把货卸在那里，因为它们太重了，以至于无法在其他地方起飞。
|
||||
另外两个夏天，我在当地的百事可乐，七喜，橙色粉碎经销商那里工作，给商店和餐馆送一箱箱的软饮料。这绝对是我做过的体力要求最高的工作。想象一下一个五层高的木箱，每个木箱里装着一打750毫升的软饮料玻璃瓶，放在一辆手推车上。想象一下把它搬到二楼的餐厅，想象那家餐厅每周能拿到120箱... 24次爬楼梯，然后又带着空瓶子下来。一辆小卡车上通常会有300箱左右的软饮料。我们的工资是按负荷计算的，不是按小时计算的，所以我们的目标是早点完工，然后去海滩。
|
||||
我的技术工作
|
||||
送苏打水是我大学期间最后一份暑期工作。第二年我毕业了，获得了数学学位，还修了很多计算机课程，尤其是数值分析。我在技术部的第一份工作，是为一家小型电脑服务顾问公司工作。我用 社会科学统计套装软件 对一些钓鱼调查做了一些分析，写了几百行程序设计语言在我们按时间租来的服务局的 ibm 3800激光打印机上打印演唱会门票，并开始研究一些程序来分析森林统计。我最终为需要林业统计的客户工作，在20世纪80年代中期成为合伙人。那时我们已经不仅仅是测量树木，也不再使用分时局来进行计算了。我们买了一台 unix 小型计算机，我们在80年代后期升级到 sun 工作站网络
|
||||
我花了一些时间研究一个大型开发项目，总部设在马来西亚吉隆坡。然后我们买了第一个地理信息系统，在80年代末和90年代，我花了大部分时间和我们的客户一起工作，他们需要定制软件来满足他们的业务需求。到了21世纪初，我的三个老合伙人都准备退休了，我试图弄明白，我是如何融入我们这个不再是小公司的，大约200名员工的长期图景的。我们的新员工老板也不明白这一点，2002年，我来到智利，想看看智利-加拿大自由贸易协定，是否提供了一个合理的机会，把我们的部分业务转移到拉丁美洲。
|
||||
2004年正式成立。与此同时，这家加拿大母公司受到了一系列投资的严重影响，鉴于2007-2009年的经济衰退，这些投资似乎不再那么明智，它在2011年被迫关门。然而，那时候，智利子公司还在经营，所以我们原来的雇员和我成了合伙人，通过资产出售买下了它。直到今天，它仍在运行，在社会环境领域做了很多很酷的事情，我经常成为其中的一部分，特别是当我可靠的数学和计算背景非常有用的时候。
|
||||
作为一个副业，我为一个在印度买卖赛马的出色男人，开发和支持一个赛马信息系统。
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/5/weird-jobs-tech
|
||||
|
||||
作者：[Chris Hermansen][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[MM-BCY](https://github.com/MM-BCY)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/clhermansen
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/yellow_plane_fly_air.jpg?itok=pEcrCVJT (Yellow plane flying in the air, Beechcraft D17S)
|
||||
[2]: https://worldairphotography.wordpress.com/2016/08/22/air-tanker-history-in-canada-part-one/amp/
|
||||
349
translated/tech/20180416 Cgo and Python.md
Normal file
349
translated/tech/20180416 Cgo and Python.md
Normal file
@ -0,0 +1,349 @@
|
||||
Cgo 与 Python
|
||||
============================================================
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
如果你查看 [new Datadog Agent][8]，你可能会注意到大部分代码库是用 Go 编写的，但我们用来收集指标的检查仍然是用 Python 编写的。这大概是因为 Datadog Agent 是基于一个[嵌入][9] CPython 解释器的 GO，可以在任何时候按需执行 Python 代码。这个过程通过抽象层来透明化，使得你可以编写惯用的 Go 代码来运行后台的 Python。
|
||||
|
||||
[视频](https://youtu.be/yrEi5ezq2-c)
|
||||
|
||||
在 Go 应用程序中嵌入 Python 的原因有很多：
|
||||
|
||||
* 它在过渡期间很有用；逐步将现有 Python 项目的部分迁移到新语言，而不会在此过程中丢失任何功能。
|
||||
|
||||
* 你可以复用现有的 Python 软件或库，而无需用新语言重新实现。
|
||||
|
||||
* 你可以通过加载去执行常规 Python 脚本来动态扩展你软件，甚至在运行时也可以。
|
||||
|
||||
理由还可以列很多，但对于 Datadog Agent 来说，最后一点至关重要：我们希望做到无需重新编译 Agent，或者说编译任何内容就能够执行自定义检查或更改现有检查。
|
||||
|
||||
嵌入 CPython 非常简单，而且文档齐全。解释器本身是用 C 编写的，并且提供了一个 C API 以编程方式来执行底层操作，例如创建对象、导入模块和调用函数。
|
||||
|
||||
在本文中，我们将展示一些代码示例，我们将会在与 Python 交互的同时继续保持 Go 代码的惯用语，但在我们继续之前，我们需要解决一个间隙：嵌入 API 是 C 但我们的主要应用程序是 Go，这怎么可能工作？
|
||||
|
||||
|
||||
|
||||
### 介绍 cgo
|
||||
|
||||
有[很多好的理由][10] 说服你为什么不要在堆栈中引入 cgo，但嵌入 CPython 是你必须这样做的原因。[Cgo][11] 不是语言，也不是编译器。它是 [Foreign Function Interface][12] (FFI)，一种让我们可以在 Go 中使用来调用不同语言（特别是 C）编写的函数和服务的机制。
|
||||
|
||||
当我们提起“cgo”时，我们实际上指的是 Go 工具链在底层使用的一组工具、库、函数和类型，因此我们可以通过执行 `go build` 来获取我们的 Go 二进制文件。下面是使用 cgo 的示例程序：
|
||||
|
||||
```
|
||||
package main
|
||||
|
||||
// #include <float.h>
|
||||
import "C"
|
||||
import "fmt"
|
||||
|
||||
func main() {
|
||||
fmt.Println("Max float value of float is", C.FLT_MAX)
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
在这种包含标头情况下，`import "C"` 指令上方的注释块称为“前导码”，可以包含实际的 C 代码。导入后，我们可以通过“C”伪包来“跳转”到外部代码，访问常量`FLT_MAX`。你可以通过调用 `go build` 来构建，它就像普通的 Go 一样。
|
||||
|
||||
如果你想查看 cgo 在这背后到底做了什么，可以运行 `go build -x`。你将看到“cgo”工具将被调用以生成一些 C 和 Go 模块，然后将调用 C 和 Go 编译器来构建目标模块，最后链接器将所有内容放在一起。
|
||||
|
||||
你可以在 [Go Blog][13] 上阅读更多有关 cgo 的信息，文章包含更多的例子以及一些有用的链接来做进一步了解细节。
|
||||
|
||||
现在我们已经了解了 cgo 可以为我们做什么，让我们看看如何使用这种机制运行一些 Python 代码。
|
||||
|
||||
|
||||
### 嵌入 CPython: a primer
|
||||
|
||||
从技术上讲，嵌入 CPython 的 Go 程序并没有你想象的那么复杂。事实上，我们只需在运行 Python 代码之前初始化解释器，并在完成后关闭它。请注意，我们在所有示例中使用 Python 2.x，但我们只需做很少的调整就可以应用于 Python 3.x。让我们看一个例子：
|
||||
|
||||
```
|
||||
package main
|
||||
|
||||
// #cgo pkg-config: python-2.7
|
||||
// #include <Python.h>
|
||||
import "C"
|
||||
import "fmt"
|
||||
|
||||
func main() {
|
||||
C.Py_Initialize()
|
||||
fmt.Println(C.GoString(C.Py_GetVersion()))
|
||||
C.Py_Finalize()
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
上面的示例与以下 Python 代码完全相同：
|
||||
|
||||
```
|
||||
import sys
|
||||
print(sys.version)
|
||||
|
||||
```
|
||||
|
||||
你可以看到我们在开头加入了一个 `#cgo` 指令；这些指令被会被传递到工具链，你就改变了构建工作流程。在这种情况下，我们告诉 cgo 调用`pkg-config`来收集构建并链接名为“python-2.7”的库所需的标志，并将这些标志传递给 C 编译器。如果你的系统中安装了 CPython 开发库和 pkg-config，你只需要运行 `go build` 来编译上面的示例。
|
||||
|
||||
回到代码，我们使用`Py_Initialize()` 和`Py_Finalize()` 来初始化和关闭解释器，并使用`Py_GetVersion` C 函数来获取嵌入式解释器版本信息的字符串。
|
||||
|
||||
如果你更近一步，我们可以把所有调用 C Python API 的 cgo 代码一起，这就是 Datadog Agent 进行所有嵌入式操作所有依赖的 [go-python][14] 做的事情；该库为 C API 提供了一个 Go 友好的轻量级包，并隐藏了 cgo 细节。这是另一个基本的嵌入式示例，这次使用 go-python：
|
||||
```
|
||||
package main
|
||||
|
||||
import (
|
||||
python "github.com/sbinet/go-python"
|
||||
)
|
||||
|
||||
func main() {
|
||||
python.Initialize()
|
||||
python.PyRun_SimpleString("print 'hello, world!'")
|
||||
python.Finalize()
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
这看起来更接近普通 Go 代码，不再暴露 cgo，我们可以在访问 Python API 时来回使用 Go 字符串。嵌入式看起来功能强大且对开发人员友好，是时候充分利用解释器了：让我们尝试从磁盘加载 Python 模块。
|
||||
|
||||
在 Python 方面我们不需要任何复杂的东西，无处不在的“hello world”就可以达到目的：
|
||||
|
||||
```
|
||||
# foo.py
|
||||
def hello():
|
||||
"""
|
||||
Print hello world for fun and profit.
|
||||
"""
|
||||
print "hello, world!"
|
||||
|
||||
```
|
||||
|
||||
Go 代码稍微复杂一些，但仍然可读：
|
||||
|
||||
```
|
||||
// main.go
|
||||
package main
|
||||
|
||||
import "github.com/sbinet/go-python"
|
||||
|
||||
func main() {
|
||||
python.Initialize()
|
||||
defer python.Finalize()
|
||||
|
||||
fooModule := python.PyImport_ImportModule("foo")
|
||||
if fooModule == nil {
|
||||
panic("Error importing module")
|
||||
}
|
||||
|
||||
helloFunc := fooModule.GetAttrString("hello")
|
||||
if helloFunc == nil {
|
||||
panic("Error importing function")
|
||||
}
|
||||
|
||||
// The Python function takes no params but when using the C api
|
||||
// we're required to send (empty) *args and **kwargs anyways.
|
||||
helloFunc.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
首次构建，我们需要将 `PYTHONPATH` 环境变量设置为当前工作目录，以便导入语句能够找到 `foo.py` 模块。在 shell 中，该命令如下所示：
|
||||
|
||||
```
|
||||
$ go build main.go && PYTHONPATH=. ./main
|
||||
hello, world!
|
||||
|
||||
```
|
||||
|
||||
|
||||
### 可怕的全局解释器锁
|
||||
|
||||
为了嵌入 Python 必须引入 cgo 需要权衡：构建速度会变慢，垃圾收集器不会帮助我们管理外部系统使用的内存，交叉编译也很难。这些是否是针对特定项目的问题可以讨论，但我认为有一些不容商量的问题：Go 并发模型。如果我们不能从 goroutine 运行 Python，那么使用 Go 就没有意义了。
|
||||
|
||||
在处理并发、Python 和 cgo 之前，我们还需要知道一些事情：它就是全局解释器锁，也称为 GIL。GIL 是语言解释器（CPython 就是其中之一）中广泛采用的一种机制，可防止多个线程同时运行。这意味着 CPython 执行的任何 Python 程序都无法在同一进程中并行运行。并发仍然是可能的，锁是速度、安全性和实现之间的一个很好的权衡，那么，当涉及到嵌入时，为什么这会造成问题呢？
|
||||
|
||||
当一个常规的、非嵌入式的 Python 程序启动时，不涉及 GIL 以避免锁定操作中的无用开销；在某些 Python 代码请求生成线程时 GIL 首次启动。对于每个线程，解释器创建一个数据结构来存储当前的相关状态信息并锁定 GIL。当线程完成时，状态被恢复，GIL 被解锁，准备被其他线程使用。
|
||||
|
||||
当我们从 Go 程序运行 Python 时，上述情况都不会自动发生。如果没有 GIL，我们的 Go 程序可以创建多个 Python 线程，这可能会导致竞争条件，从而导致致命的运行时错误，并且很可能是分段错误导致整个 Go 应用程序瘫痪。
|
||||
|
||||
解决方案是在我们从 Go 运行多线程代码时显式调用 GIL；代码并不复杂，因为 C API 提供了我们需要的所有工具。为了更好地暴露这个问题，我们需要写一些受 CPU 限制的 Python 代码。让我们将这些函数添加到前面示例中的 foo.py 模块中：
|
||||
|
||||
```
|
||||
# foo.py
|
||||
import sys
|
||||
|
||||
def print_odds(limit=10):
|
||||
"""
|
||||
Print odds numbers < limit
|
||||
"""
|
||||
for i in range(limit):
|
||||
if i%2:
|
||||
sys.stderr.write("{}\n".format(i))
|
||||
|
||||
def print_even(limit=10):
|
||||
"""
|
||||
Print even numbers < limit
|
||||
"""
|
||||
for i in range(limit):
|
||||
if i%2 == 0:
|
||||
sys.stderr.write("{}\n".format(i))
|
||||
|
||||
```
|
||||
|
||||
我们将尝试从 Go 并发打印奇数和偶数，使用两个不同的 goroutines（因此涉及线程）：
|
||||
|
||||
```
|
||||
package main
|
||||
|
||||
import (
|
||||
"sync"
|
||||
|
||||
"github.com/sbinet/go-python"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// The following will also create the GIL explicitly
|
||||
// by calling PyEval_InitThreads(), without waiting
|
||||
// for the interpreter to do that
|
||||
python.Initialize()
|
||||
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(2)
|
||||
|
||||
fooModule := python.PyImport_ImportModule("foo")
|
||||
odds := fooModule.GetAttrString("print_odds")
|
||||
even := fooModule.GetAttrString("print_even")
|
||||
|
||||
// Initialize() has locked the the GIL but at this point we don't need it
|
||||
// anymore. We save the current state and release the lock
|
||||
// so that goroutines can acquire it
|
||||
state := python.PyEval_SaveThread()
|
||||
|
||||
go func() {
|
||||
_gstate := python.PyGILState_Ensure()
|
||||
odds.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
python.PyGILState_Release(_gstate)
|
||||
|
||||
wg.Done()
|
||||
}()
|
||||
|
||||
go func() {
|
||||
_gstate := python.PyGILState_Ensure()
|
||||
even.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
python.PyGILState_Release(_gstate)
|
||||
|
||||
wg.Done()
|
||||
}()
|
||||
|
||||
wg.Wait()
|
||||
|
||||
// At this point we know we won't need Python anymore in this
|
||||
// program, we can restore the state and lock the GIL to perform
|
||||
// the final operations before exiting.
|
||||
python.PyEval_RestoreThread(state)
|
||||
python.Finalize()
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
在阅读示例时，您可能会注意到一个模式，该模式将成为我们运行嵌入式 Python 代码的习惯写法：
|
||||
|
||||
1. 保存状态并锁定 GIL。
|
||||
|
||||
2. 执行 Python.
|
||||
|
||||
3. 恢复状态并解锁 GIL。
|
||||
|
||||
代码应该很简单，但我们想指出一个微妙的细节：请注意，尽管借用了 GIL 执行，有时我们通过调用 `PyEval_SaveThread()` 和 `PyEval_RestoreThread()` 来操作 GIL，有时（查看 goroutines）我们对 `PyGILState_Ensure()` 和 `PyGILState_Release()` 来做同样的事情。
|
||||
|
||||
我们说过当从 Python 操作多线程时，解释器负责创建存储当前状态所需的数据结构，但是当同样的事情发生在 C API 时，我们来负责处理。
|
||||
|
||||
当我们用 go-python 初始化解释器时，我们是在 Python 上下文中操作的。因此，当调用 `PyEval_InitThreads()` 时，它会初始化数据结构并锁定 GIL。我们可以使用 `PyEval_SaveThread()` 和 `PyEval_RestoreThread()` 对已经存在的状态进行操作。
|
||||
|
||||
在 goroutines 中，我们从 Go 上下文操作，我们需要显式创建状态并在完成后将其删除，这就是 `PyGILState_Ensure()` 和 `PyGILState_Release()` 为我们所做的。
|
||||
|
||||
|
||||
### 释放 Gopher
|
||||
|
||||
在这一点上，我们知道如何处理在嵌入式解释器中执行 Python 的多线程 Go 代码，但在 GIL 之后，另一个挑战即将来临：Go 调度程序。
|
||||
|
||||
当一个 goroutine 启动时，它被安排在可用的 `GOMAXPROCS` 线程之一上执行[参见此处][15] 了解有关该主题的更多详细信息。如果一个 goroutine 碰巧执行了系统调用或调用 C 代码，当前线程会将移交给另一个队列中等待运行的其他 goroutine ，以便它们有更好的机会运行； 当前 goroutine 被暂停，等待系统调用或 C 函数返回。当 C 函数返回时，线程会尝试恢复暂停的 goroutine，但如果这不可能，它会要求 Go runtime 找到另一个线程来完成 goroutine 并进入睡眠状态。
|
||||
|
||||
考虑到这一点，让我们看看当一个 goroutine 被移动到一个新线程时，运行一些 Python 代码的 goroutine 会发生什么：
|
||||
|
||||
1. 我们的 goroutine 启动，执行 C 调用，暂停，GIL 被锁定。
|
||||
|
||||
2. 当 C 调用返回时，当前线程尝试恢复 goroutine，但失败。
|
||||
|
||||
3. 当前线程告诉 Go 运行时寻找另一个线程来恢复我们的 goroutine。
|
||||
|
||||
4. Go scheduler 找到可用线程并恢复 goroutine。
|
||||
|
||||
5. goroutine 快完成了，并在返回之前尝试解锁 GIL。The goroutine is almost done and tries to unlock the GIL before returning.
|
||||
|
||||
6. 当前状态中存储的线程ID来自原始线程，与当前线程的ID不同。
|
||||
|
||||
7. Panic!
|
||||
|
||||
所幸，我们可以通过从 goroutine 中调用运行时包中的 LockOSThread 函数来强制 Go runtime 始终保持我们的 goroutine 在同一线程上运行：
|
||||
|
||||
```
|
||||
go func() {
|
||||
runtime.LockOSThread()
|
||||
|
||||
_gstate := python.PyGILState_Ensure()
|
||||
odds.Call(python.PyTuple_New(0), python.PyDict_New())
|
||||
python.PyGILState_Release(_gstate)
|
||||
wg.Done()
|
||||
}()
|
||||
|
||||
```
|
||||
|
||||
这会干扰 scheduler 并可能引入一些开销，但这是我们也愿意付出代价。
|
||||
|
||||
### 结论
|
||||
|
||||
为了嵌入 Python，Datadog Agent 必须接受一些权衡：
|
||||
* cgo 引入的开销。
|
||||
|
||||
* 手动处理 GIL 的任务。
|
||||
|
||||
* 在执行期间将 goroutine 绑定到同一线程的限制。
|
||||
|
||||
为了能方便在 Go 中运行 Python 检查代码，我们乐此不疲。意识到权衡，我们能够最大限度地减少它们的影响，除了为支持 Python 而引入的其他限制，我们没有对策来控制潜在问题：
|
||||
|
||||
* 构建是自动化和可配置的，因此开发人员仍然需要拥有与 `go build` 非常相似的东西。
|
||||
|
||||
* agent 的轻量级版本，可以完全剥离 Python 支持，只需使用 Go 构建标签。A lightweight version of the agent can be built stripping out Python support entirely simply using Go build tags.
|
||||
|
||||
* 这样的版本仅依赖于在代理本身中硬编码的核心检查（主要是系统和网络检查），但没有 cgo 并且可以交叉编译。Such a version only relies on core checks hardcoded in the agent itself (system and network checks mostly) but is cgo free and can be cross compiled.
|
||||
|
||||
我们将在未来重新评估我们的选择，并决定是否仍然值得保留 cgo； 我们甚至可以重新考虑 Python 作为一个整体是否仍然值得，等待 [Go plugin package][16] 成熟到足以支持我们的用例。但就目前而言，嵌入式 Python 运行良好，从旧代理过渡到新代理再简单不过了。
|
||||
|
||||
你是一个喜欢混合不同编程语言的多语言者吗？您喜欢了解语言的内部工作原理以提高您的代码性能吗？ [Join us at Datadog!][17]
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
|
||||
作者：[ Massimiliano Pippi][a]
|
||||
译者：[Zioyi](https://github.com/Zioyi)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://github.com/masci
|
||||
[1]:http://twitter.com/share?url=https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
[2]:http://www.reddit.com/submit?url=https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
[3]:https://www.linkedin.com/shareArticle?mini=true&url=https://www.datadoghq.com/blog/engineering/cgo-and-python/
|
||||
[4]:https://www.datadoghq.com/blog/category/under-the-hood
|
||||
[5]:https://www.datadoghq.com/blog/tag/agent
|
||||
[6]:https://www.datadoghq.com/blog/tag/golang
|
||||
[7]:https://www.datadoghq.com/blog/tag/python
|
||||
[8]:https://github.com/DataDog/datadog-agent/
|
||||
[9]:https://docs.python.org/2/extending/embedding.html
|
||||
[10]:https://dave.cheney.net/2016/01/18/cgo-is-not-go
|
||||
[11]:https://golang.org/cmd/cgo/
|
||||
[12]:https://en.wikipedia.org/wiki/Foreign_function_interface
|
||||
[13]:https://blog.golang.org/c-go-cgo
|
||||
[14]:https://github.com/sbinet/go-python
|
||||
[15]:https://morsmachine.dk/go-scheduler
|
||||
[16]:https://golang.org/pkg/plugin/
|
||||
[17]:https://www.datadoghq.com/careers/
|
||||
90
translated/tech/20210628 How to archive files on FreeDOS.md
Normal file
90
translated/tech/20210628 How to archive files on FreeDOS.md
Normal file
@ -0,0 +1,90 @@
|
||||
[#]: subject: (How to archive files on FreeDOS)
|
||||
[#]: via: (https://opensource.com/article/21/6/archive-files-freedos)
|
||||
[#]: author: (Jim Hall https://opensource.com/users/jim-hall)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
如何在 FreeDOS 上归档文件
|
||||
======
|
||||
有一个 FreeDOS 版的 tar，但 DOS 上事实上的标准归档工具是 Zip 和 Unzip。
|
||||
![Filing cabinet for organization][1]
|
||||
|
||||
在Linux上，你可能熟悉标准的 Unix 归档命令：`tar`。FreeDOS 上也有 `tar` 的版本（还有其他一些流行的归档程序），但 DOS 上事实上的标准归档程序是 Zip 和 Unzip。Zip 和 Unzip 都默认安装在 FreeDOS 1.3 RC4 中。
|
||||
|
||||
Zip 文件格式最初是由 PKWARE 的 Phil Katz 在 1989 年为 PKZIP 和 PKUNZIP 这对 DOS 归档工具构思的。Katz 将 Zip 文件的规范作为一个开放标准发布，因此任何人都可以创建 Zip 档案。作为开放规范的结果，Zip 成为 DOS 上的一个标准归档。[Info-ZIP][2] 项目实现了一套开源的 `ZIP` 和 `UNZIP` 程序。
|
||||
|
||||
### 对文件和目录进行压缩
|
||||
|
||||
你可以在 DOS 命令行中使用 `ZIP` 来创建文件和目录的归档。这是一个方便的方法，可以为你的工作做一个备份，或者发布一个“包”，在未来的 FreeDOS 发行中使用。例如，假设我想为我的项目源码做一个备份，其中包含这些源文件：
|
||||
|
||||
![dir][3]
|
||||
|
||||
我想把这些文件归档
|
||||
（Jim Hall,[CC-BY SA 4.0][4]）
|
||||
|
||||
`ZIP` 有大量的命令行选项来做不同的事情，但我最常使用的命令行选项是 `-r` 来处理目录和子目录_递归_，以及 `-9` 来提供可能的最大压缩。`ZIP` 和 `UNZIP` 使用类似 Unix 的命令行，所以你可以在破折号后面组合选项：`-9r` 将提供最大压缩并在 Zip 文件中包括子目录。
|
||||
|
||||
![zip][5]
|
||||
|
||||
压缩一个目录树
|
||||
（Jim Hall,[CC-BY SA 4.0][4]）
|
||||
|
||||
在我的例子中，`ZIP` 能够将我的源文件从大约 33KB 压缩到大约 22KB，为我节省了 11KB 的宝贵磁盘空间。你可能会得到不同的压缩率，这取决于你给 `ZIP` 的选项，或者你想在 Zip 文件中存储什么文件（以及有多少）。一般来说，非常长的文本文件（如源码）会产生良好的压缩效果，而非常小的文本文件（如只有几行的 DOS “批处理”文件）通常太短，无法很好地压缩。
|
||||
|
||||
### 解压文件和目录
|
||||
|
||||
将文件保存到 Zip 文件中是很好的，但你最终会需要将这些文件解压到某个地方。让我们首先检查一下我们刚刚创建的 Zip 文件里有什么。为此，使用 `UNZIP `命令。你可以在 `UNZIP`中使用一堆不同的选项，但我发现我只使用几个常用的选项。
|
||||
|
||||
要列出一个 Zip 文件的内容，使用 `-l` （“list”） 选项。
|
||||
|
||||
![unzip -l][6]
|
||||
|
||||
用 unzip 列出归档文件的内容
|
||||
(Jim Hall,[CC-BY SA 4.0][4])
|
||||
|
||||
该输出允让我看到 Zip 文件中的 14 个条目：13 个文件加上 `SRC` 目录。
|
||||
|
||||
如果我想提取整个 Zip 文件，我可以直接使用 `UNZIP` 命令并提供 Zip 文件作为命令行选项。这样就可以从我当前的工作目录开始提取 Zip 文件了。除非我正在恢复某个东西的先前版本，否则我通常不想覆盖我当前的文件。在这种情况下，我希望将 Zip 文件解压到一个新的目录。你可以用 `-d` （“destination”） 命令行选项指定目标路径。
|
||||
|
||||
![unzip -d temp][7]
|
||||
|
||||
你可以用 -d 来解压到目标路径
|
||||
（Jim Hall,[CC-BY SA 4.0][4]）
|
||||
|
||||
有时我想从一个 Zip 文件中提取一个文件。在这个例子中，假设我想提取一个 DOS 可执行程序 `TEST.EXE`。要提取单个文件，你要指定你想提取的 Zip 文件的完整路径。默认情况下，`UNZIP` 将使用 Zip 文件中提供的路径解压该文件。要省略路径信息，你可以添加 `-j`（“junk the path”） 选项。
|
||||
|
||||
你也可以合并选项。让我们从 Zip 文件中提取 `SRC\TEST.EXE` 程序，但省略完整路径并将其保存在 `TEMP` 目录下：
|
||||
|
||||
![unzip -j][8]
|
||||
|
||||
unzip 结合选项
|
||||
（Jim Hall,[CC-BY SA 4.0][4]）
|
||||
|
||||
因为 Zip 文件是一个开放的标准，所以我们会今天继续看到 Zip 文件。每个 Linux 发行版都支持使用 Info-ZIP 程序的 Zip 文件。你的 Linux 文件管理器可能也支持 Zip 文件。在 GNOME 文件管理器中，你应该可以右击一个文件夹并从下拉菜单中选择“压缩”。你可以选择创建一个新的归档文件，包括 Zip 文件。
|
||||
|
||||
创建和管理 Zip 文件是任何 DOS 用户的一项关键技能。你可以在 Info-ZIP 网站上了解更多关于 `ZIP` 和 `UNZIP` 的信息，或者在命令行上使用 `h`（“帮助”）选项来打印选项列表。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/6/archive-files-freedos
|
||||
|
||||
作者：[Jim Hall][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[geekpi](https://github.com/geekpi)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/jim-hall
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/files_documents_organize_letter.png?itok=GTtiiabr (Filing cabinet for organization)
|
||||
[2]: http://infozip.sourceforge.net/
|
||||
[3]: https://opensource.com/sites/default/files/uploads/dir.png (I'd like to archive these files)
|
||||
[4]: https://creativecommons.org/licenses/by-sa/4.0/
|
||||
[5]: https://opensource.com/sites/default/files/uploads/zip-9r.png (Zipping a directory tree)
|
||||
[6]: https://opensource.com/sites/default/files/uploads/unzip-l.png (Listing the archive file contents with unzip)
|
||||
[7]: https://opensource.com/sites/default/files/uploads/unzip-d.png (You can unzip into a destination path with -d)
|
||||
[8]: https://opensource.com/sites/default/files/uploads/unzip-j.png (Combining options with unzip)
|
||||
@ -0,0 +1,155 @@
|
||||
[#]: subject: (How to parse Bash program configuration files)
|
||||
[#]: via: (https://opensource.com/article/21/6/bash-config)
|
||||
[#]: author: (David Both https://opensource.com/users/dboth)
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
如何解析 Bash 程序的配置文件
|
||||
======
|
||||
将配置文件与代码分开，使任何人都可以改变他们的配置，而不需要任何特殊的编程技巧。
|
||||
![bash logo on green background][1]
|
||||
|
||||
将程序配置与代码分开是很重要的。它使非程序员能够改变配置而不需要修改程序的代码。如果是编译的二进制可执行文件，这对非程序员来说是不可能的，因为它不仅需要访问源文件（我们在开源程序中会这样），而且还需要程序员的技能组合。很少有人有这种能力，而且大多数人都不想学习。
|
||||
|
||||
对于像 Bash 这样的 shell 语言，由于 shell 脚本没有被编译成二进制格式，所以从定义上讲，源码是可以访问的。尽管有这种开放性，但对于非程序员来说，在 shell 脚本中钻研和修改它们并不是一个特别好的主意。即使是经验丰富的开发人员和系统管理员，也会意外地做出一些改变，导致错误或更糟。
|
||||
|
||||
因此，将配置项放在容易维护的文本文件中，提供了分离，并允许非程序员编辑配置，而不会有对代码进行意外修改的危险。许多开发者对用编译语言编写的程序都是这样做的，因为他们并不期望用户是开发者。由于许多相同的原因，对解释型 shell 语言这样做也是有意义的。
|
||||
|
||||
### 通常的方式
|
||||
|
||||
和其他许多语言一样, 你可以为 Bash 程序编写代码, 读取并解析 ASCII 文本的配置文件, 读取变量名称, 并在程序代码执行时设置值. 例如，一个配置文件可能看起来像这样：
|
||||
|
||||
|
||||
```
|
||||
var1=LinuxGeek46
|
||||
var2=Opensource.com
|
||||
```
|
||||
|
||||
程序将读取文件，解析每一行，并将值设置到每个变量中。
|
||||
|
||||
### Sourcing
|
||||
|
||||
Bash 使用一种更简单的方法来解析和设置变量, 叫做 _sourcing_. 从一个可执行的 shell 程序中获取一个外部文件是一种简单的方法，可以将该文件的内容完整地引入 shell 程序中。在某种意义上，这很像编译语言的 `include` 语句，在运行时包括库文件。这样的文件可以包括任何类型的 Bash 代码，包括变量赋值。
|
||||
|
||||
像往常一样，演示比解释更容易。
|
||||
|
||||
首先，创建一个 `~/bin` 目录（如果它还不存在的话），并将其作为当前工作目录 （PWD）。[Linux 文件系统分层标准][2]将 `~/bin` 定义为用户存储可执行文件的适当位置。
|
||||
|
||||
在这个目录下创建一个新文件。将其命名为 `main`，并使其可执行：
|
||||
|
||||
|
||||
```
|
||||
[dboth@david bin]$ touch main
|
||||
[dboth@david bin]$ chmod +x main
|
||||
[dboth@david bin]$
|
||||
```
|
||||
|
||||
在这个可执行文件中添加以下内容：
|
||||
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
Name="LinuxGeek"
|
||||
echo $Name
|
||||
```
|
||||
|
||||
并执行这个 Bash 程序：
|
||||
|
||||
|
||||
```
|
||||
[dboth@david bin]$ ./main
|
||||
LinuxGeek
|
||||
[dboth@david bin]$
|
||||
```
|
||||
|
||||
创建一个新的文件并命名为 `~/bin/data`。这个文件不需要是可执行的。在其中添加以下信息：
|
||||
|
||||
|
||||
```
|
||||
# Sourced code and variables
|
||||
echo "This is the sourced code from the data file."
|
||||
FirstName="David"
|
||||
LastName="Both"
|
||||
```
|
||||
|
||||
在 `main` 程序中增加三行，看起来像这样：
|
||||
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
Name="LinuxGeek"
|
||||
echo $Name
|
||||
source ~/bin/data
|
||||
echo "First name: $FirstName"
|
||||
echo "LastName: $LastName"
|
||||
```
|
||||
|
||||
重新运行该程序：
|
||||
|
||||
|
||||
```
|
||||
[dboth@david bin]$ ./main
|
||||
LinuxGeek
|
||||
This is the sourced code from the data file.
|
||||
First name: David
|
||||
LastName: Both
|
||||
[dboth@david bin]$
|
||||
```
|
||||
|
||||
关于 sourcing 还有一件非常酷的事情要知道。你可以使用一个单点（`.`）作为 `source` 命令的快捷方式。改变 `main` 文件，用 `.` 代替 `source`。
|
||||
|
||||
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
Name="LinuxGeek"
|
||||
echo $Name
|
||||
. ~/bin/data
|
||||
echo "First name: $FirstName"
|
||||
echo "LastName: $LastName"
|
||||
```
|
||||
|
||||
并再次运行该程序。其结果应该与之前的运行完全相同。
|
||||
|
||||
### 运行 Bash
|
||||
|
||||
每一台使用 Bash 的 Linux 主机（几乎所有主机都是，因为 Bash 是所有发行版的默认 shell），都包括一些优秀的、内置的 sourcing 示例。
|
||||
|
||||
每当 Bash shell 运行时，它的环境必须被配置成可以使用的样子。有五个主要文件和一个目录用于配置 Bash 环境。它们和它们的主要功能如下：
|
||||
|
||||
* `/etc/profile`: 全系统环境和启动程序
|
||||
* `/etc/bashrc`: 全系统的函数和别名
|
||||
* `/etc/profile.d/`: 包含全系统脚本的目录，用于配置各种命令行工具，如 `vim` 和 `mc` 以及系统管理员创建的任何自定义配置脚本
|
||||
* `~/.bash_profile`: 用户特定的环境和启动程序
|
||||
* `~/.bashrc`: 用户特定的别名和函数
|
||||
* `~/.bash_logout`: 用户特定的命令，在用户注销时执行
|
||||
|
||||
|
||||
|
||||
试着通过这些文件追踪执行顺序，确定它在非登录 Bash 初始化和登录 Bash 初始化中使用的顺序。我在我的 Linux 培训系列[_使用和管理 Linux：从零到系统管理员_][3]的第一卷第 17 章中做了这个工作。
|
||||
|
||||
给你一个提示。这一切都从 `~/.bashrc` 脚本开始。
|
||||
|
||||
### 总结
|
||||
|
||||
这篇文章探讨了在 Bash 程序中引用代码和变量的方法。这种从配置文件中解析变量的方法是快速、简单和灵活的。它提供了一种将 Bash 代码与变量赋值分开的方法，以使非程序员能够设置这些变量的值。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/21/6/bash-config
|
||||
|
||||
作者：[David Both][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[geekpi](https://github.com/geekpi)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/dboth
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/bash_command_line.png?itok=k4z94W2U (bash logo on green background)
|
||||
[2]: http://refspecs.linuxfoundation.org/fhs.shtml
|
||||
[3]: http://www.both.org/?page_id=1183
|
||||
Loading…
Reference in New Issue
Block a user