mirror of
https://github.com/LCTT/TranslateProject.git
synced 2025-03-24 02:20:09 +08:00
Merge branch 'master' of https://github.com/LCTT/TranslateProject.git
* 'master' of https://github.com/LCTT/TranslateProject.git: (274 commits) 选题: Using sudo to delegate permissions in Linux translating translating update at 2017年 12月 06日 星期三 19:54:17 CST Translating by qhwdw reformat translated 补充不完整的内容 PUB:20171006 Concurrent Servers Part 3 - Event-driven.md PRF:20171006 Concurrent Servers Part 3 - Event-driven.md PUB:20171128 How To Tell If Your Linux Server Has Been Compromised.md PRF:20171128 How To Tell If Your Linux Server Has Been Compromised.md Translated by qhwdw Delete 20171120 Mark McIntyre How Do You Fedora.md Add files via upload Delete 233 333 Update 20171120 Mark McIntyre How Do You Fedora.md Update 20171120 Mark McIntyre How Do You Fedora.md Update 20171120 Mark McIntyre How Do You Fedora.md ...
This commit is contained in:
commit
3c00ca3f51
published
20161216 Kprobes Event Tracing on ARMv8.md20170622 A users guide to links in the Linux filesystem.md20171006 Concurrent Servers Part 3 - Event-driven.md20171009 Examining network connections on Linux systems.md20171012 Linux Networking Hardware for Beginners Think Software.md20171029 A block layer introduction part 1 the bio layer.md
201711
20141028 When Does Your OS Run.md20170202 Understanding Firewalld in Multi-Zone Configurations.md20170227 Ubuntu Core in LXD containers.md20170418 INTRODUCING MOBY PROJECT A NEW OPEN-SOURCE PROJECT TO ADVANCE THE SOFTWARE CONTAINERIZATION MOVEMENT.md20170531 Understanding Docker Container Host vs Container OS for Linux and Windows Containers.md20170608 The Life-Changing Magic of Tidying Up Code.md20170706 Wildcard Certificates Coming January 2018.md20170825 Guide to Linux App Is a Handy Tool for Every Level of Linux User.md20170905 GIVE AWAY YOUR CODE BUT NEVER YOUR TIME.md20170928 3 Python web scrapers and crawlers.md20171002 Scaling the GitLab database.md20171003 PostgreSQL Hash Indexes Are Now Cool.md20171004 No the Linux desktop hasnt jumped in popularity.md20171007 Instant 100 command line productivity boost.md20171008 8 best languages to blog about.md20171009 CyberShaolin Teaching the Next Generation of Cybersecurity Experts.md20171010 Getting Started Analyzing Twitter Data in Apache Kafka through KSQL.md20171011 How to set up a Postgres database on a Raspberry Pi.md20171011 Why Linux Works.md20171013 6 reasons open source is good for business.md20171013 Best of PostgreSQL 10 for the DBA.md20171015 How to implement cloud-native computing with Kubernetes.md20171015 Monitoring Slow SQL Queries via Slack.md20171015 Why Use Docker with R A DevOps Perspective.md20171016 Introducing CRI-O 1.0.md20171017 A tour of Postgres Index Types.md20171017 Image Processing on Linux.md20171018 How containers and microservices change security.md20171018 Learn how to program in Python by building a simple dice game.md20171018 Tips to Secure Your Network in the Wake of KRACK.md20171019 3 Simple Excellent Linux Network Monitors.md20171019 How to manage Docker containers in Kubernetes with Java.md20171020 3 Tools to Help You Remember Linux Commands.md20171020 Running Android on Top of a Linux Graphics Stack.md20171024 Top 5 Linux pain points in 2017.md20171024 Who contributed the most to open source in 2017 Let s analyze GitHub’s data and find out.md20171024 Why Did Ubuntu Drop Unity Mark Shuttleworth Explains.md20171025 How to roll your own backup solution with BorgBackup, Rclone and Wasabi cloud storage.md20171026 But I dont know what a container is .md20171026 Why is Kubernetes so popular.md20171101 How to use cron in Linux.md20171101 We re switching to a DCO for source code contributions.md20171106 4 Tools to Manage EXT2 EXT3 and EXT4 Health in Linux.md20171106 Finding Files with mlocate.md20171106 Linux Foundation Publishes Enterprise Open Source Guides.md20171106 Most companies can t buy an open source community clue. Here s how to do it right.md20171107 AWS adopts home-brewed KVM as new hypervisor.md20171107 How I created my first RPM package in Fedora.md20171108 Build and test applications with Ansible Container.md20171110 File better bugs with coredumpctl.md20171114 Linux totally dominates supercomputers.md20171116 5 Coolest Linux Terminal Emulators.md20171117 How to Easily Remember Linux Commands.md20171118 Getting started with OpenFaaS on minikube.md20171128 tmate – Instantly Share Your Terminal Session To Anyone In Seconds.md
20171120 Containers and Kubernetes Whats next.md20171124 How to Install Android File Transfer for Linux.md20171124 Open Source Cloud Skills and Certification Are Key for SysAdmins.md20171124 Photon Could Be Your New Favorite Container OS.md20171128 How To Tell If Your Linux Server Has Been Compromised.md20171130 New Feature Find every domain someone owns automatically.md20171130 Search DuckDuckGo from the Command Line.md20171130 Translate Shell – A Tool To Use Google Translate From Command Line In Linux.md20171130 Wake up and Shut Down Linux Automatically.md20171201 How to Manage Users with Groups in Linux.md20171201 Linux Journal Ceases Publication.mdsources
talk
tech
20090701 The One in Which I Call Out Hacker News.md20130402 Dynamic linker tricks Using LD_PRELOAD to cheat inject features and investigate programs.md20160330 How to turn any syscall into an event Introducing eBPF Kernel probes.md20170209 INTRODUCING DOCKER SECRETS MANAGEMENT.md20170215 How to take screenshots on Linux using Scrot.md20170530 How to Improve a Legacy Codebase.md20170607 Why Car Companies Are Hiring Computer Security Experts.md20170622 A users guide to links in the Linux filesystem.md20170809 Designing a Microservices Architecture for Failure.md20170908 Betting on the Web.md20170921 How to answer questions in a helpful way.md20171005 Reasons Kubernetes is cool.md20171010 Operating a Kubernetes network.md20171011 LEAST PRIVILEGE CONTAINER ORCHESTRATION.md20171012 Linux Networking Hardware for Beginners Think Software.md20171020 How Eclipse is advancing IoT development.md20171029 A block layer introduction part 1 the bio layer.md20171031 How to use SVG as a Placeholder and Other Image Loading Techniques.md20171102 Dive into BPF a list of reading material.md20171107 GitHub welcomes all CI tools.md20171112 Love Your Bugs.md20171114 Sysadmin 101 Patch Management.md20171114 Take Linux and Run With It.md20171115 Security Jobs Are Hot Get Trained and Get Noticed.md20171115 Why and How to Set an Open Source Strategy.md20171116 Unleash Your Creativity – Linux Programs for Drawing and Image Editing.md20171118 Language engineering for great justice.md
@ -1,86 +1,82 @@
|
||||
# Kprobes Event Tracing on ARMv8
|
||||
ARMv8 上的 kprobes 事件跟踪
|
||||
==============
|
||||
|
||||

|
||||
|
||||
### 介绍
|
||||
|
||||
Kprobes 是一种内核功能,它允许通过在执行(或模拟)断点指令之前和之后,设置调用开发者提供例程的任意断点来检测内核。可参见 kprobes 文档[[1]][2] 获取更多信息。基本的 kprobes 功能可使用 CONFIG_KPROBEES 来选择。在 arm64 的 v4.8 发行版中, kprobes 支持被添加到主线。
|
||||
kprobes 是一种内核功能,它允许通过在执行(或模拟)断点指令之前和之后,设置调用开发者提供例程的任意断点来检测内核。可参见 kprobes 文档^注1 获取更多信息。基本的 kprobes 功能可使用 `CONFIG_KPROBEES` 来选择。在 arm64 的 v4.8 内核发行版中, kprobes 支持被添加到主线。
|
||||
|
||||
在这篇文章中,我们将介绍 kprobes 在 arm64 上的使用,通过在命令行中使用 debugfs 事件追踪接口来收集动态追踪事件。这个功能在一些架构(包括 arm32)上可用已经有段时间,现在在 arm64 上也能使用了。这个功能允许使用 kprobes 而无需编写任何代码。
|
||||
在这篇文章中,我们将介绍 kprobes 在 arm64 上的使用,通过在命令行中使用 debugfs 事件追踪接口来收集动态追踪事件。这个功能在一些架构(包括 arm32)上可用已经有段时间,现在在 arm64 上也能使用了。这个功能可以无需编写任何代码就能使用 kprobes。
|
||||
|
||||
### 探针类型
|
||||
|
||||
Kprbes 子系统提供了三种不同类型的动态探针,如下所述。
|
||||
kprobes 子系统提供了三种不同类型的动态探针,如下所述。
|
||||
|
||||
### Kprobes
|
||||
#### kprobes
|
||||
|
||||
基本探针是 kprobes 插入的一个软件断点,用以替代你正在探测的指令,当探测点被击中时,它为最终的单步执行(或模拟)保存下原始指令。
|
||||
基本探针是 kprobes 插入的一个软件断点,用以替代你正在探测的指令,当探测点被命中时,它为最终的单步执行(或模拟)保存下原始指令。
|
||||
|
||||
### Kretprobes
|
||||
#### kretprobes
|
||||
|
||||
Kretprobes 是 kprobes 的一部分,它允许拦截返回函数,而不必在返回点设置一个探针(或者可能有多个)。对于支持的架构(包括 ARMv8),只要选择 kprobes,就可以选择此功能。
|
||||
kretprobes 是 kprobes 的一部分,它允许拦截返回函数,而不必在返回点设置一个探针(或者可能有多个探针)。对于支持的架构(包括 ARMv8),只要选择 kprobes,就可以选择此功能。
|
||||
|
||||
### Jprobes
|
||||
#### jprobes
|
||||
|
||||
Jprobes 允许通过提供一个具有相同调用签名(call signature)的中间函数来拦截对一个函数的调用,这里中间函数将被首先调用。Jprobes 只是一个编程接口,它不能通过 debugfs 事件追踪子系统来使用。因此,我们将不会在这里进一步讨论 jprobes。如果你想使用 jprobes,请参考 kprobes 文档。
|
||||
jprobes 允许通过提供一个具有相同<ruby>调用签名<rt>call signature</rt></ruby>的中间函数来拦截对一个函数的调用,这里中间函数将被首先调用。jprobes 只是一个编程接口,它不能通过 debugfs 事件追踪子系统来使用。因此,我们将不会在这里进一步讨论 jprobes。如果你想使用 jprobes,请参考 kprobes 文档。
|
||||
|
||||
### 调用 Kprobes
|
||||
### 调用 kprobes
|
||||
|
||||
Kprobes 提供一系列能从内核代码中调用的 API 来设置探测点和当探测点被击中时调用的注册函数。在不往内核中添加代码的情况下,Kprobes 也是可用的,这是通过写入特定事件追踪的 debugfs 文件来实现的,需要在文件中设置探针地址和信息,以便在探针被击中时记录到追踪日志中。后者是本文将要讨论的重点。最后 Kprobes 可以通过 perl 命令来使用。
|
||||
kprobes 提供一系列能从内核代码中调用的 API 来设置探测点和当探测点被命中时调用的注册函数。在不往内核中添加代码的情况下,kprobes 也是可用的,这是通过写入特定事件追踪的 debugfs 文件来实现的,需要在文件中设置探针地址和信息,以便在探针被命中时记录到追踪日志中。后者是本文将要讨论的重点。最后 kprobes 可以通过 perl 命令来使用。
|
||||
|
||||
### Kprobes API
|
||||
#### kprobes API
|
||||
|
||||
内核开发人员可以在内核中编写函数(通常在专用的调试模块中完成)来设置探测点,并且在探测指令执行前和执行后立即执行任何所需操作。这在 kprobes.txt 中有很好的解释。
|
||||
|
||||
### 事件追踪
|
||||
#### 事件追踪
|
||||
|
||||
事件追踪子系统有自己的自己的文档[[2]][3],对于了解一般追踪事件的背景可能值得一读。事件追踪子系统是追踪点(tracepoints)和 kprobes 事件追踪的基础。事件追踪文档重点关注追踪点,所以请在查阅文档时记住这一点。Kprobes 与追踪点不同的是没有预定义的追踪点列表,而是采用动态创建的用于触发追踪事件信息收集的任意探测点。事件追踪子系统通过一系列 debugfs 文件来控制和监视。事件追踪(CONFIG_EVENT_TRACING)将在被如 kprobe 事件追踪子系统等需要时自动选择。
|
||||
事件追踪子系统有自己的自己的文档^注2 ,对于了解一般追踪事件的背景可能值得一读。事件追踪子系统是<ruby>追踪点<rt>tracepoints</rt></ruby>和 kprobes 事件追踪的基础。事件追踪文档重点关注追踪点,所以请在查阅文档时记住这一点。kprobes 与追踪点不同的是没有预定义的追踪点列表,而是采用动态创建的用于触发追踪事件信息收集的任意探测点。事件追踪子系统通过一系列 debugfs 文件来控制和监视。事件追踪(`CONFIG_EVENT_TRACING`)将在被如 kprobe 事件追踪子系统等需要时自动选择。
|
||||
|
||||
#### Kprobes 事件
|
||||
##### kprobes 事件
|
||||
|
||||
使用 kprobes 事件追踪子系统,用户可以在内核任意断点处指定要报告的信息,只需要指定任意现有可探测指令的地址以及格式化信息即可确定。在执行过程中遇到断点时,kprobes 将所请求的信息传递给事件追踪子系统的公共部分,这些部分将数据格式化并追加到追踪日志中,就像追踪点的工作方式一样。Kprobes 使用一个类似的但是大部分是独立的 debugfs 文件来控制和显示追踪事件信息。该功能可使用 CONFIG_KPROBE_EVENT 来选择。Kprobetrace文档[[3]][4] 提供了如何使用 kprobes 事件追踪的基本信息,并且应当被参考用以了解以下介绍示例的详细信息。
|
||||
使用 kprobes 事件追踪子系统,用户可以在内核任意断点处指定要报告的信息,只需要指定任意现有可探测指令的地址以及格式化信息即可确定。在执行过程中遇到断点时,kprobes 将所请求的信息传递给事件追踪子系统的公共部分,这些部分将数据格式化并追加到追踪日志中,就像追踪点的工作方式一样。kprobes 使用一个类似的但是大部分是独立的 debugfs 文件来控制和显示追踪事件信息。该功能可使用 `CONFIG_KPROBE_EVENT` 来选择。Kprobetrace 文档^ 注3 提供了如何使用 kprobes 事件追踪的基本信息,并且应当被参考用以了解以下介绍示例的详细信息。
|
||||
|
||||
### Kprobes 和 Perf
|
||||
#### kprobes 和 perf
|
||||
|
||||
Perf 工具为 Kprobes 提供了另一个命令行接口。特别地,“perf probe” 允许探测点除了由函数名加偏移量和地址指定外,还可由源文件和行号指定。Perf 接口实际上是使用 kprobes 的 debugfs 接口的封装器。
|
||||
perf 工具为 kprobes 提供了另一个命令行接口。特别地,`perf probe` 允许探测点除了由函数名加偏移量和地址指定外,还可由源文件和行号指定。perf 接口实际上是使用 kprobes 的 debugfs 接口的封装器。
|
||||
|
||||
### Arm64 Kprobes
|
||||
### Arm64 kprobes
|
||||
|
||||
上述所有 kprobes 的方面现在都在 arm64 上得到实现,然而实际上与其它架构上的有一些不同:
|
||||
|
||||
* 注册名称参数当然是依架构而特定的,并且可以在 ARM ARM 中找到。
|
||||
|
||||
* 目前不是所有的指令类型都可被探测。当前不可探测的指令包括 mrs/msr(除了 DAIF 读),异常生成指令,eret 和 hint(除了 nop 变体)。在这些情况下,只探测一个附近的指令来代替是最简单的。这些指令在探测的黑名单里是因为在 kprobes 单步执行或者指令模拟时它们对处理器状态造成的改变是不安全的,这是由于 kprobes 构造的单步执行上下文和指令所需要的不一致,或者是由于指令不能容忍在 kprobes 中额外的处理时间和异常处理(ldx/stx)。
|
||||
|
||||
* 目前不是所有的指令类型都可被探测。当前不可探测的指令包括 mrs/msr(除了 DAIF 读取)、异常生成指令、eret 和 hint(除了 nop 变体)。在这些情况下,只探测一个附近的指令来代替是最简单的。这些指令在探测的黑名单里是因为在 kprobes 单步执行或者指令模拟时它们对处理器状态造成的改变是不安全的,这是由于 kprobes 构造的单步执行上下文和指令所需要的不一致,或者是由于指令不能容忍在 kprobes 中额外的处理时间和异常处理(ldx/stx)。
|
||||
* 试图识别在 ldx/stx 序列中的指令并且防止探测,但是理论上这种检查可能会失败,导致允许探测到的原子序列永远不会成功。当探测原子代码序列附近时应该小心。
|
||||
|
||||
* 注意由于 linux ARM64 调用约定的具体信息,为探测函数可靠地复制栈帧是不可能的,基于此不要试图用 jprobes 这样做,这一点与支持 jprobes 的大多数其它架构不同。这样的原因是被调用者没有足够的信息来确定需要的栈数量。
|
||||
|
||||
* 注意当探针被击中时,一个探针记录的栈指针信息将反映出使用中的特定栈指针,它是内核栈指针或者中断栈指针。
|
||||
|
||||
* 注意当探针被命中时,一个探针记录的栈指针信息将反映出使用中的特定栈指针,它是内核栈指针或者中断栈指针。
|
||||
* 有一组内核函数是不能被探测的,通常因为它们作为 kprobes 处理的一部分被调用。这组函数的一部分是依架构特定的,并且也包含如异常入口代码等。
|
||||
|
||||
### 使用 Kprobes 事件追踪
|
||||
### 使用 kprobes 事件追踪
|
||||
|
||||
Kprobes 一个常用的例子是检测函数入口和/或出口。因为只需要使用函数名来作为探针地址,它安装探针特别简单。Kprobes 事件追踪将查看符号名称并且确定地址。ARMv8 调用标准定义了函数参数和返回值的位置,并且这些可以作为 kprobes 事件处理的一部分被打印出来。
|
||||
kprobes 的一个常用例子是检测函数入口和/或出口。因为只需要使用函数名来作为探针地址,它安装探针特别简单。kprobes 事件追踪将查看符号名称并且确定地址。ARMv8 调用标准定义了函数参数和返回值的位置,并且这些可以作为 kprobes 事件处理的一部分被打印出来。
|
||||
|
||||
### 例子: 函数入口探测
|
||||
#### 例子: 函数入口探测
|
||||
|
||||
检测 USB 以太网驱动程序复位功能:
|
||||
|
||||
```
|
||||
_$ pwd
|
||||
$ pwd
|
||||
/sys/kernel/debug/tracing
|
||||
$ cat > kprobe_events <<EOF
|
||||
p ax88772_reset %x0
|
||||
EOF
|
||||
$ echo 1 > events/kprobes/enable_
|
||||
$ echo 1 > events/kprobes/enable
|
||||
```
|
||||
|
||||
此时每次驱动器的 *ax8872_reset()* 函数被调用,追踪事件都将会被记录。这个事件将显示指向通过 作为此函数的唯一参数的 X0(按照 ARMv8 调用标准)传入的 _usbnet_ 结构的指针。插入需要以太网驱动程序的USB加密狗后,我们看见以下追踪信息:
|
||||
此时每次该驱动的 `ax8872_reset()` 函数被调用,追踪事件都将会被记录。这个事件将显示指向通过作为此函数的唯一参数的 `X0`(按照 ARMv8 调用标准)传入的 `usbnet` 结构的指针。插入需要以太网驱动程序的 USB 加密狗后,我们看见以下追踪信息:
|
||||
|
||||
```
|
||||
_$ cat trace
|
||||
$ cat trace
|
||||
# tracer: nop
|
||||
#
|
||||
# entries-in-buffer/entries-written: 1/1 #P:8
|
||||
@ -93,27 +89,27 @@ _$ cat trace
|
||||
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
|
||||
# | | | |||| | |
|
||||
kworker/0:0-4 [000] d… 10972.102939: p_ax88772_reset_0:
|
||||
(ax88772_reset+0x0/0x230) arg1=0xffff800064824c80_
|
||||
(ax88772_reset+0x0/0x230) arg1=0xffff800064824c80
|
||||
```
|
||||
|
||||
这里我们可以看见传入到我们的探测函数的指针参数的值。由于我们没有使用 kprobes 事件追踪的可选标签功能,我们需要的信息自动被标注为 _arg1_。注意这个指向我们需要 kprobes 记录这个探针的一组值的第一个,而不是函数参数的实际位置。在这个例子中它也只是碰巧是我们探测函数的第一个参数。
|
||||
这里我们可以看见传入到我们的探测函数的指针参数的值。由于我们没有使用 kprobes 事件追踪的可选标签功能,我们需要的信息自动被标注为 `arg1`。注意这指向我们需要 kprobes 记录这个探针的一组值的第一个,而不是函数参数的实际位置。在这个例子中它也只是碰巧是我们探测函数的第一个参数。
|
||||
|
||||
### 例子: 函数入口和返回探测
|
||||
#### 例子: 函数入口和返回探测
|
||||
|
||||
Kretprobe 功能专门用于探测函数返回。在函数入口 kprobes 子系统将会被调用并且建立钩子以便在函数返回时调用,钩子将记录需求事件信息。对最常见情况,返回信息通常在 X0 寄存器中,这是非常有用的。在 %x0 中返回值也可以被称为 _$retval_。以下例子也演示了如何提供一个可读的标签来展示有趣的信息。
|
||||
kretprobe 功能专门用于探测函数返回。在函数入口 kprobes 子系统将会被调用并且建立钩子以便在函数返回时调用,钩子将记录需求事件信息。对最常见情况,返回信息通常在 `X0` 寄存器中,这是非常有用的。在 `%x0` 中返回值也可以被称为 `$retval`。以下例子也演示了如何提供一个可读的标签来展示有趣的信息。
|
||||
|
||||
使用 kprobes 和 kretprobe 检测内核 *_do_fork()* 函数来记录参数和结果的例子:
|
||||
使用 kprobes 和 kretprobe 检测内核 `do_fork()` 函数来记录参数和结果的例子:
|
||||
|
||||
```
|
||||
_$ cd /sys/kernel/debug/tracing
|
||||
$ cd /sys/kernel/debug/tracing
|
||||
$ cat > kprobe_events <<EOF
|
||||
p _do_fork %x0 %x1 %x2 %x3 %x4 %x5
|
||||
r _do_fork pid=%x0
|
||||
EOF
|
||||
$ echo 1 > events/kprobes/enable_
|
||||
$ echo 1 > events/kprobes/enable
|
||||
```
|
||||
|
||||
此时每次对 _do_fork() 的调用都会产生两个记录到 “_trace_” 文件的 kprobe 事件,一个报告调用参数值,另一个报告返回值。返回值在 trace 文件中将被标记为“_pid_”。这里是三次 fork 系统调用执行后的 trace 文件的内容:
|
||||
此时每次对 `_do_fork()` 的调用都会产生两个记录到 trace 文件的 kprobe 事件,一个报告调用参数值,另一个报告返回值。返回值在 trace 文件中将被标记为 `pid`。这里是三次 fork 系统调用执行后的 trace 文件的内容:
|
||||
|
||||
```
|
||||
_$ cat trace
|
||||
@ -136,30 +132,30 @@ _$ cat trace
|
||||
bash-1671 [001] d..1 214.401975: r__do_fork_0: (SyS_clone+0x18/0x20 <- _do_fork) pid=0x726_
|
||||
```
|
||||
|
||||
### 例子: 解引用指针参数
|
||||
#### 例子: 解引用指针参数
|
||||
|
||||
对于指针值,kprobes 事件处理子系统也允许解引用和打印所需的内存内容,适用于各种基本数据类型。为了展示所需字段,手动计算结构的偏移量是必要的。
|
||||
|
||||
检测 `_do_wait()` 函数:
|
||||
|
||||
```
|
||||
_$ cat > kprobe_events <<EOF
|
||||
$ cat > kprobe_events <<EOF
|
||||
p:wait_p do_wait wo_type=+0(%x0):u32 wo_flags=+4(%x0):u32
|
||||
r:wait_r do_wait $retval
|
||||
EOF
|
||||
$ echo 1 > events/kprobes/enable_
|
||||
$ echo 1 > events/kprobes/enable
|
||||
```
|
||||
|
||||
注意在第一个探针中使用的参数标签是可选的,并且可用于更清晰地识别记录在追踪日志中的信息。带符号的偏移量和括号表明了寄存器参数是指向记录在追踪日志中的内存内容的指针。“_:u32_”表明了内存位置包含一个无符号的4字节宽的数据(在这个例子中指局部定义的结构中的一个 emum 和一个 int)
|
||||
注意在第一个探针中使用的参数标签是可选的,并且可用于更清晰地识别记录在追踪日志中的信息。带符号的偏移量和括号表明了寄存器参数是指向记录在追踪日志中的内存内容的指针。`:u32` 表明了内存位置包含一个无符号的 4 字节宽的数据(在这个例子中指局部定义的结构中的一个 emum 和一个 int)。
|
||||
|
||||
探针标签(冒号后)是可选的,并且将用来识别日志中的探针。对每个探针来说标签必须是独一无二的。如果没有指定,将从附近的符号名称自动生成一个有用的标签,如前面的例子所示。
|
||||
|
||||
也要注意“_$retval_”参数可以只是指定为“_%x0_”。
|
||||
也要注意 `$retval` 参数可以只是指定为 `%x0`。
|
||||
|
||||
这里是两次 fork 系统调用执行后的 “_trace_” 文件的内容:
|
||||
这里是两次 fork 系统调用执行后的 trace 文件的内容:
|
||||
|
||||
```
|
||||
_$ cat trace
|
||||
$ cat trace
|
||||
# tracer: nop
|
||||
#
|
||||
# entries-in-buffer/entries-written: 4/4 #P:8
|
||||
@ -174,23 +170,23 @@ _$ cat trace
|
||||
bash-1702 [001] d… 175.342074: wait_p: (do_wait+0x0/0x260) wo_type=0x3 wo_flags=0xe
|
||||
bash-1702 [002] d..1 175.347236: wait_r: (SyS_wait4+0x74/0xe4 <- do_wait) arg1=0x757
|
||||
bash-1702 [002] d… 175.347337: wait_p: (do_wait+0x0/0x260) wo_type=0x3 wo_flags=0xf
|
||||
bash-1702 [002] d..1 175.347349: wait_r: (SyS_wait4+0x74/0xe4 <- do_wait) arg1=0xfffffffffffffff6_
|
||||
bash-1702 [002] d..1 175.347349: wait_r: (SyS_wait4+0x74/0xe4 <- do_wait) arg1=0xfffffffffffffff6
|
||||
```
|
||||
|
||||
### 例子: 探测任意指令地址
|
||||
#### 例子: 探测任意指令地址
|
||||
|
||||
在前面的例子中,我们已经为函数的入口和出口插入探针,然而探测一个任意指令(除少数例外)是可能的。如果我们正在 C 函数中放置一个探针,第一步是查看代码的汇编版本以确定我们要放置探针的位置。一种方法是在 vmlinux 文件上使用 gdb,并在要放置探针的函数中展示指令。下面是一个在 arch/arm64/kernel/modules.c 中 _module_alloc_ 函数执行此操作的示例。在这种情况下,因为 gdb 似乎更喜欢使用弱符号定义,并且它是与这个函数关联的存根代码,所以我们从 System.map 中来获取符号值:
|
||||
在前面的例子中,我们已经为函数的入口和出口插入探针,然而探测一个任意指令(除少数例外)是可能的。如果我们正在 C 函数中放置一个探针,第一步是查看代码的汇编版本以确定我们要放置探针的位置。一种方法是在 vmlinux 文件上使用 gdb,并在要放置探针的函数中展示指令。下面是一个在 `arch/arm64/kernel/modules.c` 中 `module_alloc` 函数执行此操作的示例。在这种情况下,因为 gdb 似乎更喜欢使用弱符号定义,并且它是与这个函数关联的存根代码,所以我们从 System.map 中来获取符号值:
|
||||
|
||||
```
|
||||
_$ grep module_alloc System.map
|
||||
$ grep module_alloc System.map
|
||||
ffff2000080951c4 T module_alloc
|
||||
ffff200008297770 T kasan_module_alloc_
|
||||
ffff200008297770 T kasan_module_alloc
|
||||
```
|
||||
|
||||
在这个例子中我们使用了交叉开发工具,并且在我们的主机系统上调用 gdb 来检查指令包含我们感兴趣函数。
|
||||
|
||||
```
|
||||
_$ ${CROSS_COMPILE}gdb vmlinux
|
||||
$ ${CROSS_COMPILE}gdb vmlinux
|
||||
(gdb) x/30i 0xffff2000080951c4
|
||||
0xffff2000080951c4 <module_alloc>: sub sp, sp, #0x30
|
||||
0xffff2000080951c8 <module_alloc+4>: adrp x3, 0xffff200008d70000
|
||||
@ -218,37 +214,37 @@ _$ ${CROSS_COMPILE}gdb vmlinux
|
||||
0xffff200008095228 <module_alloc+100>: ldp x19, x20, [sp,#16] 0xffff20000809522c <module_alloc+104>: ldp x29, x30, [sp],#32
|
||||
0xffff200008095230 <module_alloc+108>: ret
|
||||
0xffff200008095234 <module_alloc+112>: mov sp, x29
|
||||
0xffff200008095238 <module_alloc+116>: mov x19, #0x0 // #0_
|
||||
0xffff200008095238 <module_alloc+116>: mov x19, #0x0 // #0
|
||||
```
|
||||
|
||||
在这种情况下,我们将在此函数中显示以下源代码行的结果:
|
||||
|
||||
```
|
||||
_p = __vmalloc_node_range(size, MODULE_ALIGN, VMALLOC_START,
|
||||
p = __vmalloc_node_range(size, MODULE_ALIGN, VMALLOC_START,
|
||||
VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_EXEC, 0,
|
||||
NUMA_NO_NODE, __builtin_return_address(0));_
|
||||
NUMA_NO_NODE, __builtin_return_address(0));
|
||||
```
|
||||
|
||||
…以及在此代码行的函数调用的返回值:
|
||||
……以及在此代码行的函数调用的返回值:
|
||||
|
||||
```
|
||||
_if (p && (kasan_module_alloc(p, size) < 0)) {_
|
||||
if (p && (kasan_module_alloc(p, size) < 0)) {
|
||||
```
|
||||
|
||||
我们可以在从调用外部函数的汇编代码中识别这些。为了展示这些值,我们将在目标系统上的0xffff20000809520c 和 0xffff20000809521c 处放置探针。
|
||||
我们可以在从调用外部函数的汇编代码中识别这些。为了展示这些值,我们将在目标系统上的 `0xffff20000809520c` 和 `0xffff20000809521c` 处放置探针。
|
||||
|
||||
```
|
||||
_$ cat > kprobe_events <<EOF
|
||||
$ cat > kprobe_events <<EOF
|
||||
p 0xffff20000809520c %x0
|
||||
p 0xffff20000809521c %x0
|
||||
EOF
|
||||
$ echo 1 > events/kprobes/enable_
|
||||
$ echo 1 > events/kprobes/enable
|
||||
```
|
||||
|
||||
现在将一个以太网适配器加密狗插入到 USB 端口后,我们看到以下写入追踪日志的内容:
|
||||
|
||||
```
|
||||
_$ cat trace
|
||||
$ cat trace
|
||||
# tracer: nop
|
||||
#
|
||||
# entries-in-buffer/entries-written: 12/12 #P:8
|
||||
@ -271,47 +267,45 @@ _$ cat trace
|
||||
modprobe-2097 [002] d… 78.030643: p_0xffff20000809520c: (module_alloc+0x48/0x98) arg1=0xffff2000011b8000
|
||||
modprobe-2097 [002] d… 78.030761: p_0xffff20000809521c: (module_alloc+0x58/0x98) arg1=0x0
|
||||
modprobe-2097 [002] d… 78.031132: p_0xffff20000809520c: (module_alloc+0x48/0x98) arg1=0xffff200001270000
|
||||
modprobe-2097 [002] d… 78.031187: p_0xffff20000809521c: (module_alloc+0x58/0x98) arg1=0x0_
|
||||
modprobe-2097 [002] d… 78.031187: p_0xffff20000809521c: (module_alloc+0x58/0x98) arg1=0x0
|
||||
```
|
||||
|
||||
Kprobes 事件系统的另一个功能是记录统计信息,这可在 inkprobe_profile 中找到。在以上追踪后,该文件的内容为:
|
||||
kprobes 事件系统的另一个功能是记录统计信息,这可在 `inkprobe_profile` 中找到。在以上追踪后,该文件的内容为:
|
||||
|
||||
```
|
||||
_$ cat kprobe_profile
|
||||
$ cat kprobe_profile
|
||||
p_0xffff20000809520c 6 0
|
||||
p_0xffff20000809521c 6 0_
|
||||
p_0xffff20000809521c 6 0
|
||||
```
|
||||
|
||||
这表明我们设置的两处断点每个共发生了 8 次击中,这当然与追踪日志数据是一致的。在 kprobetrace 文档中有更多 kprobe_profile 的功能描述。
|
||||
这表明我们设置的两处断点每个共发生了 8 次命中,这当然与追踪日志数据是一致的。在 kprobetrace 文档中有更多 kprobe_profile 的功能描述。
|
||||
|
||||
也可以进一步过滤 kprobes 事件。用来控制这点的 debugfs 文件在 kprobetrace 文档中被列出,然而他们内容的详细信息大多在 trace events 文档中被描述。
|
||||
也可以进一步过滤 kprobes 事件。用来控制这点的 debugfs 文件在 kprobetrace 文档中被列出,然而它们内容的详细信息大多在 trace events 文档中被描述。
|
||||
|
||||
### 总结
|
||||
|
||||
现在,Linux ARMv8 对支持 kprobes 功能也和其它架构相当。有人正在做添加 uprobes 和 systemtap 支持的工作。这些功能/工具和其他已经完成的功能(如: perf, coresight)允许 Linux ARMv8 用户像在其它更老的架构上一样调试和测试性能。
|
||||
现在,Linux ARMv8 对支持 kprobes 功能也和其它架构相当。有人正在做添加 uprobes 和 systemtap 支持的工作。这些功能/工具和其他已经完成的功能(如: perf、 coresight)允许 Linux ARMv8 用户像在其它更老的架构上一样调试和测试性能。
|
||||
|
||||
* * *
|
||||
|
||||
参考文献
|
||||
|
||||
[[1]][5] Jim Keniston, Prasanna S. Panchamukhi, Masami Hiramatsu. “Kernel Probes (Kprobes).” _GitHub_. GitHub, Inc., 15 Aug. 2016\. Web. 13 Dec. 2016.
|
||||
|
||||
[[2]][6] Ts’o, Theodore, Li Zefan, and Tom Zanussi. “Event Tracing.” _GitHub_. GitHub, Inc., 3 Mar. 2016\. Web. 13 Dec. 2016.
|
||||
|
||||
[[3]][7] Hiramatsu, Masami. “Kprobe-based Event Tracing.” _GitHub_. GitHub, Inc., 18 Aug. 2016\. Web. 13 Dec. 2016.
|
||||
- 注1: Jim Keniston, Prasanna S. Panchamukhi, Masami Hiramatsu. “Kernel Probes (kprobes).” _GitHub_. GitHub, Inc., 15 Aug. 2016\. Web. 13 Dec. 2016.
|
||||
- 注2: Ts’o, Theodore, Li Zefan, and Tom Zanussi. “Event Tracing.” _GitHub_. GitHub, Inc., 3 Mar. 2016\. Web. 13 Dec. 2016.
|
||||
- 注3: Hiramatsu, Masami. “Kprobe-based Event Tracing.” _GitHub_. GitHub, Inc., 18 Aug. 2016\. Web. 13 Dec. 2016.
|
||||
|
||||
|
||||
----------------
|
||||
|
||||
作者简介 : [David Long][8] David在 Linaro Kernel - Core Development 团队中担任工程师。 在加入 Linaro 之前,他在商业和国防行业工作了数年,既做嵌入式实时工作又为Unix提供软件开发工具。之后,在 Digital(又名 Compaq)公司工作了十几年,负责 Unix 标准,C 编译器和运行时库的工作。之后 David 又去了一系列初创公司做嵌入式 Linux 和安卓系统,嵌入式定制操作系统和 Xen 虚拟化。他拥有 MIPS,Alpha 和 ARM 平台的经验(等等)。他使用过从 1979 年贝尔实验室 V6 开始的大部分Unix操作系统,并且长期以来一直是 Linux 用户和倡导者。他偶尔也因使用烙铁和数字示波器调试设备驱动而知名。
|
||||
作者简介 : [David Long][8] 在 Linaro Kernel - Core Development 团队中担任工程师。 在加入 Linaro 之前,他在商业和国防行业工作了数年,既做嵌入式实时工作又为Unix提供软件开发工具。之后,在 Digital(又名 Compaq)公司工作了十几年,负责 Unix 标准,C 编译器和运行时库的工作。之后 David 又去了一系列初创公司做嵌入式 Linux 和安卓系统,嵌入式定制操作系统和 Xen 虚拟化。他拥有 MIPS,Alpha 和 ARM 平台的经验(等等)。他使用过从 1979 年贝尔实验室 V6 开始的大部分Unix操作系统,并且长期以来一直是 Linux 用户和倡导者。他偶尔也因使用烙铁和数字示波器调试设备驱动而知名。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: http://www.linaro.org/blog/kprobes-event-tracing-armv8/
|
||||
|
||||
作者:[ David Long][a]
|
||||
作者:[David Long][a]
|
||||
译者:[kimii](https://github.com/kimii)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
@ -0,0 +1,300 @@
|
||||
用户指南:Linux 文件系统的链接
|
||||
============================================================
|
||||
|
||||
> 学习如何使用链接,通过从 Linux 文件系统多个位置来访问文件,可以让日常工作变得轻松。
|
||||
|
||||

|
||||
|
||||
Image by : [Paul Lewin][8]. Modified by Opensource.com. [CC BY-SA 2.0][9]
|
||||
|
||||
在我为 opensource.com 写过的关于 Linux 文件系统方方面面的文章中,包括 [Linux 的 EXT4 文件系统的历史、特性以及最佳实践][10]; [在 Linux 中管理设备][11];[Linux 文件系统概览][12] 和 [用户指南:逻辑卷管理][13],我曾简要的提到过 Linux 文件系统一个有趣的特性,它允许用户从多个位置来访问 Linux 文件目录树中的文件来简化一些任务。
|
||||
|
||||
Linux 文件系统中有两种<ruby>链接<rt>link</rt></ruby>:<ruby>硬链接<rt>hard link</rt></ruby>和<ruby>软链接<rt>soft link</rt></ruby>。虽然二者差别显著,但都用来解决相似的问题。它们都提供了对单个文件的多个目录项(引用)的访问,但实现却大为不同。链接的强大功能赋予了 Linux 文件系统灵活性,因为[一切皆是文件][14]。
|
||||
|
||||
举个例子,我曾发现一些程序要求特定的版本库方可运行。 当用升级后的库替代旧库后,程序会崩溃,提示旧版本库缺失。通常,库名的唯一变化就是版本号。出于直觉,我仅仅给程序添加了一个新的库链接,并以旧库名称命名。我试着再次启动程序,运行良好。程序就是一个游戏,人人都明白,每个玩家都会尽力使游戏进行下去。
|
||||
|
||||
事实上,几乎所有的应用程序链接库都使用通用的命名规则,链接名称中包含了主版本号,链接所指向的文件的文件名中同样包含了小版本号。再比如,程序的一些必需文件为了迎合 Linux 文件系统规范,从一个目录移动到另一个目录中,系统为了向后兼容那些不能获取这些文件新位置的程序在旧的目录中存放了这些文件的链接。如果你对 `/lib64` 目录做一个长清单列表,你会发现很多这样的例子。
|
||||
|
||||
```
|
||||
lrwxrwxrwx. 1 root root 36 Dec 8 2016 cracklib_dict.hwm -> ../../usr/share/cracklib/pw_dict.hwm
|
||||
lrwxrwxrwx. 1 root root 36 Dec 8 2016 cracklib_dict.pwd -> ../../usr/share/cracklib/pw_dict.pwd
|
||||
lrwxrwxrwx. 1 root root 36 Dec 8 2016 cracklib_dict.pwi -> ../../usr/share/cracklib/pw_dict.pwi
|
||||
lrwxrwxrwx. 1 root root 27 Jun 9 2016 libaccountsservice.so.0 -> libaccountsservice.so.0.0.0
|
||||
-rwxr-xr-x. 1 root root 288456 Jun 9 2016 libaccountsservice.so.0.0.0
|
||||
lrwxrwxrwx 1 root root 15 May 17 11:47 libacl.so.1 -> libacl.so.1.1.0
|
||||
-rwxr-xr-x 1 root root 36472 May 17 11:47 libacl.so.1.1.0
|
||||
lrwxrwxrwx. 1 root root 15 Feb 4 2016 libaio.so.1 -> libaio.so.1.0.1
|
||||
-rwxr-xr-x. 1 root root 6224 Feb 4 2016 libaio.so.1.0.0
|
||||
-rwxr-xr-x. 1 root root 6224 Feb 4 2016 libaio.so.1.0.1
|
||||
lrwxrwxrwx. 1 root root 30 Jan 16 16:39 libakonadi-calendar.so.4 -> libakonadi-calendar.so.4.14.26
|
||||
-rwxr-xr-x. 1 root root 816160 Jan 16 16:39 libakonadi-calendar.so.4.14.26
|
||||
lrwxrwxrwx. 1 root root 29 Jan 16 16:39 libakonadi-contact.so.4 -> libakonadi-contact.so.4.14.26
|
||||
```
|
||||
|
||||
`/lib64` 目录下的一些链接
|
||||
|
||||
在上面展示的 `/lib64` 目录清单列表中,文件模式第一个字母 `l` (小写字母 l)表示这是一个软链接(又称符号链接)。
|
||||
|
||||
### 硬链接
|
||||
|
||||
在 [Linux 的 EXT4 文件系统的历史、特性以及最佳实践][15]一文中,我曾探讨过这样一个事实,每个文件都有一个包含该文件信息的 inode,包含了该文件的位置信息。上述文章中的[图2][16]展示了一个指向 inode 的单一目录项。每个文件都至少有一个目录项指向描述该文件信息的 inode ,目录项是一个硬链接,因此每个文件至少都有一个硬链接。
|
||||
|
||||
如下图 1 所示,多个目录项指向了同一 inode 。这些目录项都是硬链接。我曾在三个目录项中使用波浪线 (`~`) 的缩写,这是用户目录的惯例表示,因此在该例中波浪线等同于 `/home/user` 。值得注意的是,第四个目录项是一个完全不同的目录,`/home/shared`,可能是该计算机上用户的共享文件目录。
|
||||
|
||||

|
||||
|
||||
*图 1*
|
||||
|
||||
硬链接被限制在一个单一的文件系统中。此处的“文件系统” 是指挂载在特定挂载点上的分区或逻辑卷,此例中是 `/home`。这是因为在每个文件系统中的 inode 号都是唯一的。而在不同的文件系统中,如 `/var` 或 `/opt`,会有和 `/home` 中相同的 inode 号。
|
||||
|
||||
因为所有的硬链接都指向了包含文件元信息的单一 inode ,这些属性都是文件的一部分,像所属关系、权限、到该 inode 的硬链接数目,对每个硬链接来说这些特性没有什么不同的。这是一个文件所具有的一组属性。唯一能区分这些文件的是包含在 inode 信息中的文件名。链接到同一目录中的单一文件/ inode 的硬链接必须拥有不同的文件名,这是基于同一目录下不能存在重复的文件名的事实的。
|
||||
|
||||
文件的硬链接数目可通过 `ls -l` 来查看,如果你想查看实际节点号,可使用 `ls -li` 命令。
|
||||
|
||||
### 符号(软)链接
|
||||
|
||||
硬链接和软链接(也称为<ruby>符号链接<rt>symlink</rt></ruby>)的区别在于,硬链接直接指向属于该文件的 inode ,而软链接直接指向一个目录项,即指向一个硬链接。因为软链接指向的是一个文件的硬链接而非该文件的 inode ,所以它们并不依赖于 inode 号,这使得它们能跨越不同的文件系统、分区和逻辑卷起作用。
|
||||
|
||||
软链接的缺点是,一旦它所指向的硬链接被删除或重命名后,该软链接就失效了。软链接虽然还在,但所指向的硬链接已不存在。所幸的是,`ls` 命令能以红底白字的方式在其列表中高亮显示失效的软链接。
|
||||
|
||||
### 实验项目: 链接实验
|
||||
|
||||
我认为最容易理解链接用法及其差异的方法是动手搭建一个项目。这个项目应以非超级用户的身份在一个空目录下进行。我创建了 `~/temp` 目录做这个实验,你也可以这么做。这么做可为项目创建一个安全的环境且提供一个新的空目录让程序运作,如此以来这儿仅存放和程序有关的文件。
|
||||
|
||||
#### 初始工作
|
||||
|
||||
首先,在你要进行实验的目录下为该项目中的任务创建一个临时目录,确保当前工作目录(PWD)是你的主目录,然后键入下列命令。
|
||||
|
||||
```
|
||||
mkdir temp
|
||||
```
|
||||
|
||||
使用这个命令将当前工作目录切换到 `~/temp`。
|
||||
|
||||
```
|
||||
cd temp
|
||||
```
|
||||
|
||||
实验开始,我们需要创建一个能够链接到的文件,下列命令可完成该工作并向其填充内容。
|
||||
|
||||
```
|
||||
du -h > main.file.txt
|
||||
```
|
||||
|
||||
使用 `ls -l` 长列表命名确认文件正确地创建了。运行结果应类似于我的。注意文件大小只有 7 字节,但你的可能会有 1~2 字节的变动。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ls -l
|
||||
total 4
|
||||
-rw-rw-r-- 1 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
在列表中,文件模式串后的数字 `1` 代表存在于该文件上的硬链接数。现在应该是 1 ,因为我们还没有为这个测试文件建立任何硬链接。
|
||||
|
||||
#### 对硬链接进行实验
|
||||
|
||||
硬链接创建一个指向同一 inode 的新目录项,当为文件添加一个硬链接时,你会看到链接数目的增加。确保当前工作目录仍为 `~/temp`。创建一个指向 `main.file.txt` 的硬链接,然后查看该目录下文件列表。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln main.file.txt link1.file.txt
|
||||
[dboth@david temp]$ ls -l
|
||||
total 8
|
||||
-rw-rw-r-- 2 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
-rw-rw-r-- 2 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
目录中两个文件都有两个链接且大小相同,时间戳也一样。这就是有一个 inode 和两个硬链接(即该文件的目录项)的一个文件。再建立一个该文件的硬链接,并列出目录清单内容。你可以建立硬链接: `link1.file.txt` 或 `main.file.txt`。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln link1.file.txt link2.file.txt ; ls -l
|
||||
total 16
|
||||
-rw-rw-r-- 3 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
-rw-rw-r-- 3 dboth dboth 7 Jun 13 07:34 link2.file.txt
|
||||
-rw-rw-r-- 3 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
注意,该目录下的每个硬链接必须使用不同的名称,因为同一目录下的两个文件不能拥有相同的文件名。试着创建一个和现存链接名称相同的硬链接。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln main.file.txt link2.file.txt
|
||||
ln: failed to create hard link 'link2.file.txt': File exists
|
||||
```
|
||||
|
||||
显然不行,因为 `link2.file.txt` 已经存在。目前为止我们只在同一目录下创建硬链接,接着在临时目录的父目录(你的主目录)中创建一个链接。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln main.file.txt ../main.file.txt ; ls -l ../main*
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
上面的 `ls` 命令显示 `main.file.txt` 文件确实存在于主目录中,且与该文件在 `temp` 目录中的名称一致。当然它们不是不同的文件,它们是同一文件的两个链接,指向了同一文件的目录项。为了帮助说明下一点,在 `temp` 目录中添加一个非链接文件。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ touch unlinked.file ; ls -l
|
||||
total 12
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link2.file.txt
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
-rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
使用 `ls` 命令的 `i` 选项查看 inode 的硬链接号和新创建文件的硬链接号。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ls -li
|
||||
total 12
|
||||
657024 -rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link2.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
注意上面文件模式左边的数字 `657024` ,这是三个硬链接文件所指的同一文件的 inode 号,你也可以使用 `i` 选项查看主目录中所创建的链接的节点号,和该值相同。而那个只有一个链接的 inode 号和其他的不同,在你的系统上看到的 inode 号或许不同于本文中的。
|
||||
|
||||
接着改变其中一个硬链接文件的大小。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ df -h > link2.file.txt ; ls -li
|
||||
total 12
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link2.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
现在所有的硬链接文件大小都比原来大了,因为多个目录项都链接着同一文件。
|
||||
|
||||
下个实验在我的电脑上会出现这样的结果,是因为我的 `/tmp` 目录在一个独立的逻辑卷上。如果你有单独的逻辑卷或文件系统在不同的分区上(如果未使用逻辑卷),确定你是否能访问那个分区或逻辑卷,如果不能,你可以在电脑上挂载一个 U 盘,如果上述方式适合你,你可以进行这个实验。
|
||||
|
||||
试着在 `/tmp` 目录中建立一个 `~/temp` 目录下文件的链接(或你的文件系统所在的位置)。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln link2.file.txt /tmp/link3.file.txt
|
||||
ln: failed to create hard link '/tmp/link3.file.txt' => 'link2.file.txt':
|
||||
Invalid cross-device link
|
||||
```
|
||||
|
||||
为什么会出现这个错误呢? 原因是每一个单独的可挂载文件系统都有一套自己的 inode 号。简单的通过 inode 号来跨越整个 Linux 文件系统结构引用一个文件会使系统困惑,因为相同的节点号会存在于每个已挂载的文件系统中。
|
||||
|
||||
有时你可能会想找到一个 inode 的所有硬链接。你可以使用 `ls -li` 命令。然后使用 `find` 命令找到所有硬链接的节点号。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ find . -inum 657024
|
||||
./main.file.txt
|
||||
./link1.file.txt
|
||||
./link2.file.txt
|
||||
```
|
||||
|
||||
注意 `find` 命令不能找到所属该节点的四个硬链接,因为我们在 `~/temp` 目录中查找。 `find` 命令仅在当前工作目录及其子目录中查找文件。要找到所有的硬链接,我们可以使用下列命令,指定你的主目录作为起始查找条件。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ find ~ -samefile main.file.txt
|
||||
/home/dboth/temp/main.file.txt
|
||||
/home/dboth/temp/link1.file.txt
|
||||
/home/dboth/temp/link2.file.txt
|
||||
/home/dboth/main.file.txt
|
||||
```
|
||||
|
||||
如果你是非超级用户,没有权限,可能会看到错误信息。这个命令也使用了 `-samefile` 选项而不是指定文件的节点号。这个效果和使用 inode 号一样且更容易,如果你知道其中一个硬链接名称的话。
|
||||
|
||||
#### 对软链接进行实验
|
||||
|
||||
如你刚才看到的,不能跨越文件系统边界创建硬链接,即在逻辑卷或文件系统中从一个文件系统到另一个文件系统。软链接给出了这个问题的解决方案。虽然它们可以达到相同的目的,但它们是非常不同的,知道这些差异是很重要的。
|
||||
|
||||
让我们在 `~/temp` 目录中创建一个符号链接来开始我们的探索。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln -s link2.file.txt link3.file.txt ; ls -li
|
||||
total 12
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link2.file.txt
|
||||
658270 lrwxrwxrwx 1 dboth dboth 14 Jun 14 15:21 link3.file.txt ->
|
||||
link2.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
拥有节点号 `657024` 的那些硬链接没有变化,且硬链接的数目也没有变化。新创建的符号链接有不同的 inode 号 `658270`。 名为 `link3.file.txt` 的软链接指向了 `link2.file.txt` 文件。使用 `cat` 命令查看 `link3.file.txt` 文件的内容。符号链接的 inode 信息以字母 `l` (小写字母 l)开头,意味着这个文件实际是个符号链接。
|
||||
|
||||
上例中软链接文件 `link3.file.txt` 的大小只有 14 字节。这是文本内容 `link3.file.txt` 的大小,即该目录项的实际内容。目录项 `link3.file.txt` 并不指向一个 inode ;它指向了另一个目录项,这在跨越文件系统建立链接时很有帮助。现在试着创建一个软链接,之前在 `/tmp` 目录中尝试过的。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln -s /home/dboth/temp/link2.file.txt
|
||||
/tmp/link3.file.txt ; ls -l /tmp/link*
|
||||
lrwxrwxrwx 1 dboth dboth 31 Jun 14 21:53 /tmp/link3.file.txt ->
|
||||
/home/dboth/temp/link2.file.txt
|
||||
```
|
||||
|
||||
#### 删除链接
|
||||
|
||||
当你删除硬链接或硬链接所指的文件时,需要考虑一些问题。
|
||||
|
||||
首先,让我们删除硬链接文件 `main.file.txt`。注意指向 inode 的每个目录项就是一个硬链接。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ rm main.file.txt ; ls -li
|
||||
total 8
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 link2.file.txt
|
||||
658270 lrwxrwxrwx 1 dboth dboth 14 Jun 14 15:21 link3.file.txt ->
|
||||
link2.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
`main.file.txt` 是该文件被创建时所创建的第一个硬链接。现在删除它,仍然保留着原始文件和硬盘上的数据以及所有剩余的硬链接。要删除原始文件,你必须删除它的所有硬链接。
|
||||
|
||||
现在删除 `link2.file.txt` 硬链接文件。
|
||||
|
||||
```
|
||||
[dboth@david temp]$ rm link2.file.txt ; ls -li
|
||||
total 8
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
658270 lrwxrwxrwx 1 dboth dboth 14 Jun 14 15:21 link3.file.txt ->
|
||||
link2.file.txt
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
注意软链接的变化。删除软链接所指的硬链接会使该软链接失效。在我的系统中,断开的链接用颜色高亮显示,目标的硬链接会闪烁显示。如果需要修复这个损坏的软链接,你需要在同一目录下建立一个和旧链接相同名字的硬链接,只要不是所有硬链接都已删除就行。您还可以重新创建链接本身,链接保持相同的名称,但指向剩余的硬链接中的一个。当然如果软链接不再需要,可以使用 `rm` 命令删除它们。
|
||||
|
||||
`unlink` 命令在删除文件和链接时也有用。它非常简单且没有选项,就像 `rm` 命令一样。然而,它更准确地反映了删除的基本过程,因为它删除了目录项与被删除文件的链接。
|
||||
|
||||
### 写在最后
|
||||
|
||||
我用过这两种类型的链接很长一段时间后,我开始了解它们的能力和特质。我为我所教的 Linux 课程编写了一个实验室项目,以充分理解链接是如何工作的,并且我希望增进你的理解。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
作者简介:
|
||||
|
||||
戴维.布斯 - 戴维.布斯是 Linux 和开源倡导者,居住在北卡罗莱纳的罗列 。他在 IT 行业工作了四十年,为 IBM 工作了 20 多年的 OS/2。在 IBM 时,他在 1981 年编写了最初的 IBM PC 的第一个培训课程。他为 RedHat 教授过 RHCE 班,并曾在 MCI Worldcom、思科和北卡罗莱纳州工作。他已经用 Linux 和开源软件工作将近 20 年了。
|
||||
|
||||
---------------------------------
|
||||
|
||||
via: https://opensource.com/article/17/6/linking-linux-filesystem
|
||||
|
||||
作者:[David Both][a]
|
||||
译者:[yongshouzhang](https://github.com/yongshouzhang)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://opensource.com/users/dboth
|
||||
[1]:https://opensource.com/resources/what-is-linux?src=linux_resource_menu
|
||||
[2]:https://opensource.com/resources/what-are-linux-containers?src=linux_resource_menu
|
||||
[3]:https://developers.redhat.com/promotions/linux-cheatsheet/?intcmp=7016000000127cYAAQ
|
||||
[4]:https://developers.redhat.com/cheat-sheet/advanced-linux-commands-cheatsheet?src=linux_resource_menu&intcmp=7016000000127cYAAQ
|
||||
[5]:https://opensource.com/tags/linux?src=linux_resource_menu
|
||||
[6]:https://opensource.com/article/17/6/linking-linux-filesystem?rate=YebHxA-zgNopDQKKOyX3_r25hGvnZms_33sYBUq-SMM
|
||||
[7]:https://opensource.com/user/14106/feed
|
||||
[8]:https://www.flickr.com/photos/digypho/7905320090
|
||||
[9]:https://creativecommons.org/licenses/by/2.0/
|
||||
[10]:https://linux.cn/article-8685-1.html
|
||||
[11]:https://linux.cn/article-8099-1.html
|
||||
[12]:https://linux.cn/article-8887-1.html
|
||||
[13]:https://opensource.com/business/16/9/linux-users-guide-lvm
|
||||
[14]:https://opensource.com/life/15/9/everything-is-a-file
|
||||
[15]:https://linux.cn/article-8685-1.html
|
||||
[16]:https://linux.cn/article-8685-1.html#3_19182
|
||||
[17]:https://opensource.com/users/dboth
|
||||
[18]:https://opensource.com/article/17/6/linking-linux-filesystem#comments
|
@ -1,9 +1,9 @@
|
||||
并发服务器(3) —— 事件驱动
|
||||
并发服务器(三):事件驱动
|
||||
============================================================
|
||||
|
||||
这是《并发服务器》系列的第三节。[第一节][26] 介绍了阻塞式编程,[第二节 —— 线程][27] 探讨了多线程,将其作为一种可行的方法来实现服务器并发编程。
|
||||
这是并发服务器系列的第三节。[第一节][26] 介绍了阻塞式编程,[第二节:线程][27] 探讨了多线程,将其作为一种可行的方法来实现服务器并发编程。
|
||||
|
||||
另一种常见的实现并发的方法叫做 _事件驱动编程_,也可以叫做 _异步_ 编程 [^注1][28]。这种方法变化万千,因此我们会从最基本的开始,使用一些基本的 APIs 而非从封装好的高级方法开始。本系列以后的文章会讲高层次抽象,还有各种混合的方法。
|
||||
另一种常见的实现并发的方法叫做 _事件驱动编程_,也可以叫做 _异步_ 编程 ^注1 。这种方法变化万千,因此我们会从最基本的开始,使用一些基本的 API 而非从封装好的高级方法开始。本系列以后的文章会讲高层次抽象,还有各种混合的方法。
|
||||
|
||||
本系列的所有文章:
|
||||
|
||||
@ -13,13 +13,13 @@
|
||||
|
||||
### 阻塞式 vs. 非阻塞式 I/O
|
||||
|
||||
要介绍这个标题,我们先讲讲阻塞和非阻塞 I/O 的区别。阻塞式 I/O 更好理解,因为这是我们使用 I/O 相关 API 时的“标准”方式。从套接字接收数据的时候,调用 `recv` 函数会发生 _阻塞_,直到它从端口上接收到了来自另一端套接字的数据。这恰恰是第一部分讲到的顺序服务器的问题。
|
||||
作为本篇的介绍,我们先讲讲阻塞和非阻塞 I/O 的区别。阻塞式 I/O 更好理解,因为这是我们使用 I/O 相关 API 时的“标准”方式。从套接字接收数据的时候,调用 `recv` 函数会发生 _阻塞_,直到它从端口上接收到了来自另一端套接字的数据。这恰恰是第一部分讲到的顺序服务器的问题。
|
||||
|
||||
因此阻塞式 I/O 存在着固有的性能问题。第二节里我们讲过一种解决方法,就是用多线程。哪怕一个线程的 I/O 阻塞了,别的线程仍然可以使用 CPU 资源。实际上,阻塞 I/O 通常在利用资源方面非常高效,因为线程就等待着 —— 操作系统将线程变成休眠状态,只有满足了线程需要的条件才会被唤醒。
|
||||
|
||||
_非阻塞式_ I/O 是另一种思路。把套接字设成非阻塞模式时,调用 `recv` 时(还有 `send`,但是我们现在只考虑接收),函数返回地会很快,哪怕没有数据要接收。这时,就会返回一个特殊的错误状态 ^[注2][15] 来通知调用者,此时没有数据传进来。调用者可以去做其他的事情,或者尝试再次调用 `recv` 函数。
|
||||
_非阻塞式_ I/O 是另一种思路。把套接字设成非阻塞模式时,调用 `recv` 时(还有 `send`,但是我们现在只考虑接收),函数返回的会很快,哪怕没有接收到数据。这时,就会返回一个特殊的错误状态 ^注2 来通知调用者,此时没有数据传进来。调用者可以去做其他的事情,或者尝试再次调用 `recv` 函数。
|
||||
|
||||
证明阻塞式和非阻塞式的 `recv` 区别的最好方式就是贴一段示例代码。这里有个监听套接字的小程序,一直在 `recv` 这里阻塞着;当 `recv` 返回了数据,程序就报告接收到了多少个字节 ^[注3][16]:
|
||||
示范阻塞式和非阻塞式的 `recv` 区别的最好方式就是贴一段示例代码。这里有个监听套接字的小程序,一直在 `recv` 这里阻塞着;当 `recv` 返回了数据,程序就报告接收到了多少个字节 ^注3 :
|
||||
|
||||
```
|
||||
int main(int argc, const char** argv) {
|
||||
@ -69,8 +69,7 @@ hello # wait for 2 seconds after typing this
|
||||
socket world
|
||||
^D # to end the connection>
|
||||
```
|
||||
|
||||
The listening program will print the following:
|
||||
|
||||
监听程序会输出以下内容:
|
||||
|
||||
```
|
||||
@ -144,7 +143,6 @@ int main(int argc, const char** argv) {
|
||||
这里与阻塞版本有些差异,值得注意:
|
||||
|
||||
1. `accept` 函数返回的 `newsocktfd` 套接字因调用了 `fcntl`, 被设置成非阻塞的模式。
|
||||
|
||||
2. 检查 `recv` 的返回状态时,我们对 `errno` 进行了检查,判断它是否被设置成表示没有可供接收的数据的状态。这时,我们仅仅是休眠了 200 毫秒然后进入到下一轮循环。
|
||||
|
||||
同样用 `nc` 进行测试,以下是非阻塞监听器的输出:
|
||||
@ -183,19 +181,19 @@ Peer disconnected; I'm done.
|
||||
|
||||
作为练习,给输出添加一个时间戳,确认调用 `recv` 得到结果之间花费的时间是比输入到 `nc` 中所用的多还是少(每一轮是 200 ms)。
|
||||
|
||||
这里就实现了使用非阻塞的 `recv` 让监听者检查套接字变为可能,并且在没有数据的时候重新获得控制权。换句话说,这就是 _polling(轮询)_ —— 主程序周期性的查询套接字以便读取数据。
|
||||
这里就实现了使用非阻塞的 `recv` 让监听者检查套接字变为可能,并且在没有数据的时候重新获得控制权。换句话说,用编程的语言说这就是 <ruby>轮询<rt>polling</rt></ruby> —— 主程序周期性的查询套接字以便读取数据。
|
||||
|
||||
对于顺序响应的问题,这似乎是个可行的方法。非阻塞的 `recv` 让同时与多个套接字通信变成可能,轮询这些套接字,仅当有新数据到来时才处理。就是这样,这种方式 _可以_ 用来写并发服务器;但实际上一般不这么做,因为轮询的方式很难扩展。
|
||||
|
||||
首先,我在代码中引入的 200 ms 延迟对于记录非常好(监听器在我输入 `nc` 之间只打印几行 “Calling recv...”,但实际上应该有上千行)。但它也增加了多达 200 ms 的服务器响应时间,这几乎是意料不到的。实际的程序中,延迟会低得多,休眠时间越短,进程占用的 CPU 资源就越多。有些时钟周期只是浪费在等待,这并不好,尤其是在移动设备上,这些设备的电量往往有限。
|
||||
首先,我在代码中引入的 200ms 延迟对于演示非常好(监听器在我输入 `nc` 之间只打印几行 “Calling recv...”,但实际上应该有上千行)。但它也增加了多达 200ms 的服务器响应时间,这无意是不必要的。实际的程序中,延迟会低得多,休眠时间越短,进程占用的 CPU 资源就越多。有些时钟周期只是浪费在等待,这并不好,尤其是在移动设备上,这些设备的电量往往有限。
|
||||
|
||||
但是当我们实际这样来使用多个套接字的时候,更严重的问题出现了。想像下监听器正在同时处理 1000 个 客户端。这意味着每一个循环迭代里面,它都得为 _这 1000 个套接字中的每一个_ 执行一遍非阻塞的 `recv`,找到其中准备好了数据的那一个。这非常低效,并且极大的限制了服务器能够并发处理的客户端数。这里有个准则:每次轮询之间等待的间隔越久,服务器响应性越差;而等待的时间越少,CPU 在无用的轮询上耗费的资源越多。
|
||||
但是当我们实际这样来使用多个套接字的时候,更严重的问题出现了。想像下监听器正在同时处理 1000 个客户端。这意味着每一个循环迭代里面,它都得为 _这 1000 个套接字中的每一个_ 执行一遍非阻塞的 `recv`,找到其中准备好了数据的那一个。这非常低效,并且极大的限制了服务器能够并发处理的客户端数。这里有个准则:每次轮询之间等待的间隔越久,服务器响应性越差;而等待的时间越少,CPU 在无用的轮询上耗费的资源越多。
|
||||
|
||||
讲真,所有的轮询都像是无用功。当然操作系统应该是知道哪个套接字是准备好了数据的,因此没必要逐个扫描。事实上,就是这样,接下来就会讲一些API,让我们可以更优雅地处理多个客户端。
|
||||
讲真,所有的轮询都像是无用功。当然操作系统应该是知道哪个套接字是准备好了数据的,因此没必要逐个扫描。事实上,就是这样,接下来就会讲一些 API,让我们可以更优雅地处理多个客户端。
|
||||
|
||||
### select
|
||||
|
||||
`select` 的系统调用是轻便的(POSIX),标准 Unix API 中常有的部分。它是为上一节最后一部分描述的问题而设计的 —— 允许一个线程可以监视许多文件描述符 ^[注4][17] 的变化,不用在轮询中执行不必要的代码。我并不打算在这里引入一个关于 `select` 的理解性的教程,有很多网站和书籍讲这个,但是在涉及到问题的相关内容时,我会介绍一下它的 API,然后再展示一个非常复杂的例子。
|
||||
`select` 的系统调用是可移植的(POSIX),是标准 Unix API 中常有的部分。它是为上一节最后一部分描述的问题而设计的 —— 允许一个线程可以监视许多文件描述符 ^注4 的变化,而不用在轮询中执行不必要的代码。我并不打算在这里引入一个关于 `select` 的全面教程,有很多网站和书籍讲这个,但是在涉及到问题的相关内容时,我会介绍一下它的 API,然后再展示一个非常复杂的例子。
|
||||
|
||||
`select` 允许 _多路 I/O_,监视多个文件描述符,查看其中任何一个的 I/O 是否可用。
|
||||
|
||||
@ -209,30 +207,25 @@ int select(int nfds, fd_set *readfds, fd_set *writefds,
|
||||
`select` 的调用过程如下:
|
||||
|
||||
1. 在调用之前,用户先要为所有不同种类的要监视的文件描述符创建 `fd_set` 实例。如果想要同时监视读取和写入事件,`readfds` 和 `writefds` 都要被创建并且引用。
|
||||
|
||||
2. 用户可以使用 `FD_SET` 来设置集合中想要监视的特殊描述符。例如,如果想要监视描述符 2、7 和 10 的读取事件,在 `readfds` 这里调用三次 `FD_SET`,分别设置 2、7 和 10。
|
||||
|
||||
3. `select` 被调用。
|
||||
|
||||
4. 当 `select` 返回时(现在先不管超时),就是说集合中有多少个文件描述符已经就绪了。它也修改 `readfds` 和 `writefds` 集合,来标记这些准备好的描述符。其它所有的描述符都会被清空。
|
||||
|
||||
5. 这时用户需要遍历 `readfds` 和 `writefds`,找到哪个描述符就绪了(使用 `FD_ISSET`)。
|
||||
|
||||
作为完整的例子,我在并发的服务器程序上使用 `select`,重新实现了我们之前的协议。[完整的代码在这里][18];接下来的是代码中的高亮,还有注释。警告:示例代码非常复杂,因此第一次看的时候,如果没有足够的时间,快速浏览也没有关系。
|
||||
作为完整的例子,我在并发的服务器程序上使用 `select`,重新实现了我们之前的协议。[完整的代码在这里][18];接下来的是代码中的重点部分及注释。警告:示例代码非常复杂,因此第一次看的时候,如果没有足够的时间,快速浏览也没有关系。
|
||||
|
||||
### 使用 select 的并发服务器
|
||||
|
||||
使用 I/O 的多发 API 诸如 `select` 会给我们服务器的设计带来一些限制;这不会马上显现出来,但这值得探讨,因为它们是理解事件驱动编程到底是什么的关键。
|
||||
|
||||
最重要的是,要记住这种方法本质上是单线程的 ^[注5][19]。服务器实际上在 _同一时刻只能做一件事_。因为我们想要同时处理多个客户端请求,我们需要换一种方式重构代码。
|
||||
最重要的是,要记住这种方法本质上是单线程的 ^注5 。服务器实际上在 _同一时刻只能做一件事_。因为我们想要同时处理多个客户端请求,我们需要换一种方式重构代码。
|
||||
|
||||
首先,让我们谈谈主循环。它看起来是什么样的呢?先让我们想象一下服务器有一堆任务,它应该监视哪些东西呢?两种类型的套接字活动:
|
||||
|
||||
1. 新客户端尝试连接。这些客户端应该被 `accept`。
|
||||
|
||||
2. 已连接的客户端发送数据。这个数据要用 [第一节][11] 中所讲到的协议进行传输,有可能会有一些数据要被回送给客户端。
|
||||
|
||||
尽管这两种活动在本质上有所区别,我们还是要把他们放在一个循环里,因为只能有一个主循环。循环会包含 `select` 的调用。这个 `select` 的调用会监视上述的两种活动。
|
||||
尽管这两种活动在本质上有所区别,我们还是要把它们放在一个循环里,因为只能有一个主循环。循环会包含 `select` 的调用。这个 `select` 的调用会监视上述的两种活动。
|
||||
|
||||
这里是部分代码,设置了文件描述符集合,并在主循环里转到被调用的 `select` 部分。
|
||||
|
||||
@ -264,9 +257,7 @@ while (1) {
|
||||
这里的一些要点:
|
||||
|
||||
1. 由于每次调用 `select` 都会重写传递给函数的集合,调用器就得维护一个 “master” 集合,在循环迭代中,保持对所监视的所有活跃的套接字的追踪。
|
||||
|
||||
2. 注意我们所关心的,最开始的唯一那个套接字是怎么变成 `listener_sockfd` 的,这就是最开始的套接字,服务器借此来接收新客户端的连接。
|
||||
|
||||
3. `select` 的返回值,是在作为参数传递的集合中,那些已经就绪的描述符的个数。`select` 修改这个集合,用来标记就绪的描述符。下一步是在这些描述符中进行迭代。
|
||||
|
||||
```
|
||||
@ -298,7 +289,7 @@ for (int fd = 0; fd <= fdset_max && nready > 0; fd++) {
|
||||
}
|
||||
```
|
||||
|
||||
这部分循环检查 _可读的_ 描述符。让我们跳过监听器套接字(要浏览所有内容,[看这个代码][20]) 然后看看当其中一个客户端准备好了之后会发生什么。出现了这种情况后,我们调用一个叫做 `on_peer_ready_recv` 的 _回调_ 函数,传入相应的文件描述符。这个调用意味着客户端连接到套接字上,发送某些数据,并且对套接字上 `recv` 的调用不会被阻塞 ^[注6][21]。这个回调函数返回结构体 `fd_status_t`。
|
||||
这部分循环检查 _可读的_ 描述符。让我们跳过监听器套接字(要浏览所有内容,[看这个代码][20]) 然后看看当其中一个客户端准备好了之后会发生什么。出现了这种情况后,我们调用一个叫做 `on_peer_ready_recv` 的 _回调_ 函数,传入相应的文件描述符。这个调用意味着客户端连接到套接字上,发送某些数据,并且对套接字上 `recv` 的调用不会被阻塞 ^注6 。这个回调函数返回结构体 `fd_status_t`。
|
||||
|
||||
```
|
||||
typedef struct {
|
||||
@ -307,7 +298,7 @@ typedef struct {
|
||||
} fd_status_t;
|
||||
```
|
||||
|
||||
这个结构体告诉主循环,是否应该监视套接字的读取事件,写入事件,或者两者都监视。上述代码展示了 `FD_SET` 和 `FD_CLR` 是怎么在合适的描述符集合中被调用的。对于主循环中某个准备好了写入数据的描述符,代码是类似的,除了它所调用的回调函数,这个回调函数叫做 `on_peer_ready_send`。
|
||||
这个结构体告诉主循环,是否应该监视套接字的读取事件、写入事件,或者两者都监视。上述代码展示了 `FD_SET` 和 `FD_CLR` 是怎么在合适的描述符集合中被调用的。对于主循环中某个准备好了写入数据的描述符,代码是类似的,除了它所调用的回调函数,这个回调函数叫做 `on_peer_ready_send`。
|
||||
|
||||
现在来花点时间看看这个回调:
|
||||
|
||||
@ -464,37 +455,36 @@ INFO:2017-09-26 05:29:18,070:conn0 disconnecting
|
||||
INFO:2017-09-26 05:29:18,070:conn2 disconnecting
|
||||
```
|
||||
|
||||
和线程的情况相似,客户端之间没有延迟,他们被同时处理。而且在 `select-server` 也没有用线程!主循环 _多路_ 处理所有的客户端,通过高效使用 `select` 轮询多个套接字。回想下 [第二节中][22] 顺序的 vs 多线程的客户端处理过程的图片。对于我们的 `select-server`,三个客户端的处理流程像这样:
|
||||
和线程的情况相似,客户端之间没有延迟,它们被同时处理。而且在 `select-server` 也没有用线程!主循环 _多路_ 处理所有的客户端,通过高效使用 `select` 轮询多个套接字。回想下 [第二节中][22] 顺序的 vs 多线程的客户端处理过程的图片。对于我们的 `select-server`,三个客户端的处理流程像这样:
|
||||
|
||||

|
||||
|
||||
所有的客户端在同一个线程中同时被处理,通过乘积,做一点这个客户端的任务,然后切换到另一个,再切换到下一个,最后切换回到最开始的那个客户端。注意,这里没有什么循环调度,客户端在它们发送数据的时候被客户端处理,这实际上是受客户端左右的。
|
||||
|
||||
### 同步,异步,事件驱动,回调
|
||||
### 同步、异步、事件驱动、回调
|
||||
|
||||
`select-server` 示例代码为讨论什么是异步编程,它和事件驱动及基于回调的编程有何联系,提供了一个良好的背景。因为这些词汇在并发服务器的(非常矛盾的)讨论中很常见。
|
||||
`select-server` 示例代码为讨论什么是异步编程、它和事件驱动及基于回调的编程有何联系,提供了一个良好的背景。因为这些词汇在并发服务器的(非常矛盾的)讨论中很常见。
|
||||
|
||||
让我们从一段 `select` 的手册页面中引用的一句好开始:
|
||||
让我们从一段 `select` 的手册页面中引用的一句话开始:
|
||||
|
||||
> select,pselect,FD_CLR,FD_ISSET,FD_SET,FD_ZERO - 同步 I/O 处理
|
||||
> select,pselect,FD\_CLR,FD\_ISSET,FD\_SET,FD\_ZERO - 同步 I/O 处理
|
||||
|
||||
因此 `select` 是 _同步_ 处理。但我刚刚演示了大量代码的例子,使用 `select` 作为 _异步_ 处理服务器的例子。有哪些东西?
|
||||
|
||||
答案是:这取决于你的观查角度。同步常用作阻塞处理,并且对 `select` 的调用实际上是阻塞的。和第 1、2 节中讲到的顺序的、多线程的服务器中对 `send` 和 `recv` 是一样的。因此说 `select` 是 _同步的_ API 是有道理的。可是,服务器的设计却可以是 _异步的_,或是 _基于回调的_,或是 _事件驱动的_,尽管其中有对 `select` 的使用。注意这里的 `on_peer_*` 函数是回调函数;它们永远不会阻塞,并且只有网络事件触发的时候才会被调用。它们可以获得部分数据,并能够在调用过程中保持稳定的状态。
|
||||
答案是:这取决于你的观察角度。同步常用作阻塞处理,并且对 `select` 的调用实际上是阻塞的。和第 1、2 节中讲到的顺序的、多线程的服务器中对 `send` 和 `recv` 是一样的。因此说 `select` 是 _同步的_ API 是有道理的。可是,服务器的设计却可以是 _异步的_,或是 _基于回调的_,或是 _事件驱动的_,尽管其中有对 `select` 的使用。注意这里的 `on_peer_*` 函数是回调函数;它们永远不会阻塞,并且只有网络事件触发的时候才会被调用。它们可以获得部分数据,并能够在调用过程中保持稳定的状态。
|
||||
|
||||
如果你曾经做过一些 GUI 编程,这些东西对你来说应该很亲切。有个 “事件循环”,常常完全隐藏在框架里,应用的 “业务逻辑” 建立在回调上,这些回调会在各种事件触发后被调用,用户点击鼠标,选择菜单,定时器到时间,数据到达套接字,等等。曾经最常见的编程模型是客户端的 JavaScript,这里面有一堆回调函数,它们在浏览网页时用户的行为被触发。
|
||||
如果你曾经做过一些 GUI 编程,这些东西对你来说应该很亲切。有个 “事件循环”,常常完全隐藏在框架里,应用的 “业务逻辑” 建立在回调上,这些回调会在各种事件触发后被调用,用户点击鼠标、选择菜单、定时器触发、数据到达套接字等等。曾经最常见的编程模型是客户端的 JavaScript,这里面有一堆回调函数,它们在浏览网页时用户的行为被触发。
|
||||
|
||||
### select 的局限
|
||||
|
||||
使用 `select` 作为第一个异步服务器的例子对于说明这个概念很有用,而且由于 `select` 是很常见,可移植的 API。但是它也有一些严重的缺陷,在监视的文件描述符非常大的时候就会出现。
|
||||
使用 `select` 作为第一个异步服务器的例子对于说明这个概念很有用,而且由于 `select` 是很常见、可移植的 API。但是它也有一些严重的缺陷,在监视的文件描述符非常大的时候就会出现。
|
||||
|
||||
1. 有限的文件描述符的集合大小。
|
||||
|
||||
2. 糟糕的性能。
|
||||
|
||||
从文件描述符的大小开始。`FD_SETSIZE` 是一个编译期常数,在如今的操作系统中,它的值通常是 1024。它被硬编码在 `glibc` 的头文件里,并且不容易修改。它把 `select` 能够监视的文件描述符的数量限制在 1024 以内。曾有些分支想要写出能够处理上万个并发访问的客户端请求的服务器,这个问题很有现实意义。有一些方法,但是不可移植,也很难用。
|
||||
从文件描述符的大小开始。`FD_SETSIZE` 是一个编译期常数,在如今的操作系统中,它的值通常是 1024。它被硬编码在 `glibc` 的头文件里,并且不容易修改。它把 `select` 能够监视的文件描述符的数量限制在 1024 以内。曾有些人想要写出能够处理上万个并发访问的客户端请求的服务器,所以这个问题很有现实意义。有一些方法,但是不可移植,也很难用。
|
||||
|
||||
糟糕的性能问题就好解决的多,但是依然非常严重。注意当 `select` 返回的时候,它向调用者提供的信息是 “就绪的” 描述符的个数,还有被修改过的描述符集合。描述符集映射着描述符 就绪/未就绪”,但是并没有提供什么有效的方法去遍历所有就绪的描述符。如果只有一个描述符是就绪的,最坏的情况是调用者需要遍历 _整个集合_ 来找到那个描述符。这在监视的描述符数量比较少的时候还行,但是如果数量变的很大的时候,这种方法弊端就凸显出了 ^[注7][23]。
|
||||
糟糕的性能问题就好解决的多,但是依然非常严重。注意当 `select` 返回的时候,它向调用者提供的信息是 “就绪的” 描述符的个数,还有被修改过的描述符集合。描述符集映射着描述符“就绪/未就绪”,但是并没有提供什么有效的方法去遍历所有就绪的描述符。如果只有一个描述符是就绪的,最坏的情况是调用者需要遍历 _整个集合_ 来找到那个描述符。这在监视的描述符数量比较少的时候还行,但是如果数量变的很大的时候,这种方法弊端就凸显出了 ^注7 。
|
||||
|
||||
由于这些原因,为了写出高性能的并发服务器, `select` 已经不怎么用了。每一个流行的操作系统有独特的不可移植的 API,允许用户写出非常高效的事件循环;像框架这样的高级结构还有高级语言通常在一个可移植的接口中包含这些 API。
|
||||
|
||||
@ -541,30 +531,23 @@ while (1) {
|
||||
}
|
||||
```
|
||||
|
||||
通过调用 `epoll_ctl` 来配置 `epoll`。这时,配置监听的套接字数量,也就是 `epoll` 监听的描述符的数量。然后分配一个缓冲区,把就绪的事件传给 `epoll` 以供修改。在主循环里对 `epoll_wait` 的调用是魅力所在。它阻塞着,直到某个描述符就绪了(或者超时),返回就绪的描述符数量。但这时,不少盲目地迭代所有监视的集合,我们知道 `epoll_write` 会修改传给它的 `events` 缓冲区,缓冲区中有就绪的事件,从 0 到 `nready-1`,因此我们只需迭代必要的次数。
|
||||
通过调用 `epoll_ctl` 来配置 `epoll`。这时,配置监听的套接字数量,也就是 `epoll` 监听的描述符的数量。然后分配一个缓冲区,把就绪的事件传给 `epoll` 以供修改。在主循环里对 `epoll_wait` 的调用是魅力所在。它阻塞着,直到某个描述符就绪了(或者超时),返回就绪的描述符数量。但这时,不要盲目地迭代所有监视的集合,我们知道 `epoll_write` 会修改传给它的 `events` 缓冲区,缓冲区中有就绪的事件,从 0 到 `nready-1`,因此我们只需迭代必要的次数。
|
||||
|
||||
要在 `select` 里面重新遍历,有明显的差异:如果在监视着 1000 个描述符,只有两个就绪, `epoll_waits` 返回的是 `nready=2`,然后修改 `events` 缓冲区最前面的两个元素,因此我们只需要“遍历”两个描述符。用 `select` 我们就需要遍历 1000 个描述符,找出哪个是就绪的。因此,在繁忙的服务器上,有许多活跃的套接字时 `epoll` 比 `select` 更加容易扩展。
|
||||
|
||||
剩下的代码很直观,因为我们已经很熟悉 `select 服务器` 了。实际上,`epoll 服务器` 中的所有“业务逻辑”和 `select 服务器` 是一样的,回调构成相同的代码。
|
||||
剩下的代码很直观,因为我们已经很熟悉 “select 服务器” 了。实际上,“epoll 服务器” 中的所有“业务逻辑”和 “select 服务器” 是一样的,回调构成相同的代码。
|
||||
|
||||
这种相似是通过将事件循环抽象分离到一个库/框架中。我将会详述这些内容,因为很多优秀的程序员曾经也是这样做的。相反,下一篇文章里我们会了解 `libuv`,一个最近出现的更加受欢迎的时间循环抽象层。像 `libuv` 这样的库让我们能够写出并发的异步服务器,并且不用考虑系统调用下繁琐的细节。
|
||||
这种相似是通过将事件循环抽象分离到一个库/框架中。我将会详述这些内容,因为很多优秀的程序员曾经也是这样做的。相反,下一篇文章里我们会了解 libuv,一个最近出现的更加受欢迎的时间循环抽象层。像 libuv 这样的库让我们能够写出并发的异步服务器,并且不用考虑系统调用下繁琐的细节。
|
||||
|
||||
* * *
|
||||
|
||||
|
||||
[注1][1] 我试着在两件事的实际差别中突显自己,一件是做一些网络浏览和阅读,但经常做得头疼。有很多不同的选项,从“他们是一样的东西”到“一个是另一个的子集”,再到“他们是完全不同的东西”。在面临这样主观的观点时,最好是完全放弃这个问题,专注特殊的例子和用例。
|
||||
|
||||
[注2][2] POSIX 表示这可以是 `EAGAIN`,也可以是 `EWOULDBLOCK`,可移植应用应该对这两个都进行检查。
|
||||
|
||||
[注3][3] 和这个系列所有的 C 示例类似,代码中用到了某些助手工具来设置监听套接字。这些工具的完整代码在这个 [仓库][4] 的 `utils` 模块里。
|
||||
|
||||
[注4][5] `select` 不是网络/套接字专用的函数,它可以监视任意的文件描述符,有可能是硬盘文件,管道,终端,套接字或者 Unix 系统中用到的任何文件描述符。这篇文章里,我们主要关注它在套接字方面的应用。
|
||||
|
||||
[注5][6] 有多种方式用多线程来实现事件驱动,我会把它放在稍后的文章中进行讨论。
|
||||
|
||||
[注6][7] 由于各种非实验因素,它 _仍然_ 可以阻塞,即使是在 `select` 说它就绪了之后。因此服务器上打开的所有套接字都被设置成非阻塞模式,如果对 `recv` 或 `send` 的调用返回了 `EAGAIN` 或者 `EWOULDBLOCK`,回调函数就装作没有事件发生。阅读示例代码的注释可以了解更多细节。
|
||||
|
||||
[注7][8] 注意这比该文章前面所讲的异步 polling 例子要稍好一点。polling 需要 _一直_ 发生,而 `select` 实际上会阻塞到有一个或多个套接字准备好读取/写入;`select` 会比一直询问浪费少得多的 CPU 时间。
|
||||
- 注1:我试着在做网络浏览和阅读这两件事的实际差别中突显自己,但经常做得头疼。有很多不同的选项,从“它们是一样的东西”到“一个是另一个的子集”,再到“它们是完全不同的东西”。在面临这样主观的观点时,最好是完全放弃这个问题,专注特殊的例子和用例。
|
||||
- 注2:POSIX 表示这可以是 `EAGAIN`,也可以是 `EWOULDBLOCK`,可移植应用应该对这两个都进行检查。
|
||||
- 注3:和这个系列所有的 C 示例类似,代码中用到了某些助手工具来设置监听套接字。这些工具的完整代码在这个 [仓库][4] 的 `utils` 模块里。
|
||||
- 注4:`select` 不是网络/套接字专用的函数,它可以监视任意的文件描述符,有可能是硬盘文件、管道、终端、套接字或者 Unix 系统中用到的任何文件描述符。这篇文章里,我们主要关注它在套接字方面的应用。
|
||||
- 注5:有多种方式用多线程来实现事件驱动,我会把它放在稍后的文章中进行讨论。
|
||||
- 注6:由于各种非实验因素,它 _仍然_ 可以阻塞,即使是在 `select` 说它就绪了之后。因此服务器上打开的所有套接字都被设置成非阻塞模式,如果对 `recv` 或 `send` 的调用返回了 `EAGAIN` 或者 `EWOULDBLOCK`,回调函数就装作没有事件发生。阅读示例代码的注释可以了解更多细节。
|
||||
- 注7:注意这比该文章前面所讲的异步轮询的例子要稍好一点。轮询需要 _一直_ 发生,而 `select` 实际上会阻塞到有一个或多个套接字准备好读取/写入;`select` 会比一直询问浪费少得多的 CPU 时间。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
@ -572,7 +555,7 @@ via: https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/
|
||||
|
||||
作者:[Eli Bendersky][a]
|
||||
译者:[GitFuture](https://github.com/GitFuture)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@ -587,9 +570,9 @@ via: https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/
|
||||
[8]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/#id9
|
||||
[9]:https://eli.thegreenplace.net/tag/concurrency
|
||||
[10]:https://eli.thegreenplace.net/tag/c-c
|
||||
[11]:http://eli.thegreenplace.net/2017/concurrent-servers-part-1-introduction/
|
||||
[12]:http://eli.thegreenplace.net/2017/concurrent-servers-part-1-introduction/
|
||||
[13]:http://eli.thegreenplace.net/2017/concurrent-servers-part-2-threads/
|
||||
[11]:https://linux.cn/article-8993-1.html
|
||||
[12]:https://linux.cn/article-8993-1.html
|
||||
[13]:https://linux.cn/article-9002-1.html
|
||||
[14]:http://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/
|
||||
[15]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/#id11
|
||||
[16]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/#id12
|
||||
@ -598,10 +581,10 @@ via: https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/
|
||||
[19]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/#id14
|
||||
[20]:https://github.com/eliben/code-for-blog/blob/master/2017/async-socket-server/select-server.c
|
||||
[21]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/#id15
|
||||
[22]:http://eli.thegreenplace.net/2017/concurrent-servers-part-2-threads/
|
||||
[22]:https://linux.cn/article-9002-1.html
|
||||
[23]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/#id16
|
||||
[24]:https://github.com/eliben/code-for-blog/blob/master/2017/async-socket-server/epoll-server.c
|
||||
[25]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/
|
||||
[26]:http://eli.thegreenplace.net/2017/concurrent-servers-part-1-introduction/
|
||||
[27]:http://eli.thegreenplace.net/2017/concurrent-servers-part-2-threads/
|
||||
[26]:https://linux.cn/article-8993-1.html
|
||||
[27]:https://linux.cn/article-9002-1.html
|
||||
[28]:https://eli.thegreenplace.net/2017/concurrent-servers-part-3-event-driven/#id10
|
@ -1,24 +1,20 @@
|
||||
translating---geekpi
|
||||
|
||||
|
||||
Examining network connections on Linux systems
|
||||
检查 Linux 系统上的网络连接
|
||||
============================================================
|
||||
|
||||
### Linux systems provide a lot of useful commands for reviewing network configuration and connections. Here's a look at a few, including ifquery, ifup, ifdown and ifconfig.
|
||||
> Linux 系统提供了许多有用的命令来检查网络配置和连接。下面来看几个,包括 `ifquery`、`ifup`、`ifdown` 和 `ifconfig`。
|
||||
|
||||
Linux 上有许多可用于查看网络设置和连接的命令。在今天的文章中,我们将会通过一些非常方便的命令来看看它们是如何工作的。
|
||||
|
||||
There are a lot of commands available on Linux for looking at network settings and connections. In today's post, we're going to run through some very handy commands and see how they work.
|
||||
### ifquery 命令
|
||||
|
||||
### ifquery command
|
||||
|
||||
One very useful command is the **ifquery** command. This command should give you a quick list of network interfaces. However, you might only see something like this —showing only the loopback interface:
|
||||
一个非常有用的命令是 `ifquery`。这个命令应该会显示一个网络接口列表。但是,你可能只会看到类似这样的内容 - 仅显示回环接口:
|
||||
|
||||
```
|
||||
$ ifquery --list
|
||||
lo
|
||||
```
|
||||
|
||||
If this is the case, your **/etc/network/interfaces** file doesn't include information on network interfaces except for the loopback interface. You can add lines like the last two in the example below — assuming DHCP is used to assign addresses — if you'd like it to be more useful.
|
||||
如果是这种情况,那说明你的 `/etc/network/interfaces` 不包括除了回环接口之外的网络接口信息。在下面的例子中,假设你使用 DHCP 来分配地址,且如果你希望它更有用的话,你可以添加例子最后的两行。
|
||||
|
||||
```
|
||||
# interfaces(5) file used by ifup(8) and ifdown(8)
|
||||
@ -28,15 +24,13 @@ auto eth0
|
||||
iface eth0 inet dhcp
|
||||
```
|
||||
|
||||
### ifup and ifdown commands
|
||||
### ifup 和 ifdown 命令
|
||||
|
||||
The related **ifup** and **ifdown** commands can be used to bring network connections up and shut them down as needed provided this file has the required descriptive data. Just keep in mind that "if" means "interface" in these commands just as it does in the **ifconfig** command, not "if" as in "if I only had a brain".
|
||||
可以使用相关的 `ifup` 和 `ifdown` 命令来打开网络连接并根据需要将其关闭,只要该文件具有所需的描述性数据即可。请记住,“if” 在这里意思是<ruby>接口<rt>interface</rt></ruby>,这与 `ifconfig` 命令中的一样,而不是<ruby>如果我只有一个大脑<rt>if I only had a brain</rt></ruby> 中的 “if”。
|
||||
|
||||
<aside class="nativo-promo smartphone" id="" style="overflow: hidden; margin-bottom: 16px; max-width: 620px;"></aside>
|
||||
### ifconfig 命令
|
||||
|
||||
### ifconfig command
|
||||
|
||||
The **ifconfig** command, on the other hand, doesn't read the /etc/network/interfaces file at all and still provides quite a bit of useful information on network interfaces -- configuration data along with packet counts that tell you how busy each interface has been. The ifconfig command can also be used to shut down and restart network interfaces (e.g., ifconfig eth0 down).
|
||||
另外,`ifconfig` 命令完全不读取 `/etc/network/interfaces`,但是仍然提供了网络接口相当多的有用信息 —— 配置数据以及可以告诉你每个接口有多忙的数据包计数。`ifconfig` 命令也可用于关闭和重新启动网络接口(例如:`ifconfig eth0 down`)。
|
||||
|
||||
```
|
||||
$ ifconfig eth0
|
||||
@ -51,15 +45,13 @@ eth0 Link encap:Ethernet HWaddr 00:1e:4f:c8:43:fc
|
||||
Interrupt:21 Memory:fe9e0000-fea00000
|
||||
```
|
||||
|
||||
The RX and TX packet counts in this output are extremely low. In addition, no errors or packet collisions have been reported. The **uptime** command will likely confirm that this system has only recently been rebooted.
|
||||
输出中的 RX 和 TX 数据包计数很低。此外,没有报告错误或数据包冲突。或许可以用 `uptime` 命令确认此系统最近才重新启动。
|
||||
|
||||
The broadcast (Bcast) and network mask (Mask) addresses shown above indicate that the system is operating on a Class C equivalent network (the default) so local addresses will range from 192.168.0.1 to 192.168.0.254.
|
||||
上面显示的广播 (Bcast) 和网络掩码 (Mask) 地址表明系统运行在 C 类等效网络(默认)上,所以本地地址范围从 `192.168.0.1` 到 `192.168.0.254`。
|
||||
|
||||
### netstat command
|
||||
### netstat 命令
|
||||
|
||||
The **netstat** command provides information on routing and network connections. The **netstat -rn** command displays the system's routing table.
|
||||
|
||||
<aside class="nativo-promo tablet desktop" id="" style="overflow: hidden; margin-bottom: 16px; max-width: 620px;"></aside>
|
||||
`netstat` 命令提供有关路由和网络连接的信息。`netstat -rn` 命令显示系统的路由表。192.168.0.1 是本地网关 (Flags=UG)。
|
||||
|
||||
```
|
||||
$ netstat -rn
|
||||
@ -70,7 +62,7 @@ Destination Gateway Genmask Flags MSS Window irtt Iface
|
||||
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
|
||||
```
|
||||
|
||||
That **169.254.0.0** entry in the above output is only necessary if you are using or planning to use link-local communications. You can comment out the related lines in the **/etc/network/if-up.d/avahi-autoipd** file like this if this is not the case:
|
||||
上面输出中的 `169.254.0.0` 条目仅在你正在使用或计划使用本地链路通信时才有必要。如果不是这样的话,你可以在 `/etc/network/if-up.d/avahi-autoipd` 中注释掉相关的行:
|
||||
|
||||
```
|
||||
$ tail -12 /etc/network/if-up.d/avahi-autoipd
|
||||
@ -87,9 +79,9 @@ $ tail -12 /etc/network/if-up.d/avahi-autoipd
|
||||
#fi
|
||||
```
|
||||
|
||||
### netstat -a command
|
||||
### netstat -a 命令
|
||||
|
||||
The **netstat -a** command will display **_all_** network connections. To limit this to listening and established connections (generally much more useful), use the **netstat -at** command instead.
|
||||
`netstat -a` 命令将显示“所有”网络连接。为了将其限制为显示正在监听和已建立的连接(通常更有用),请改用 `netstat -at` 命令。
|
||||
|
||||
```
|
||||
$ netstat -at
|
||||
@ -105,21 +97,9 @@ tcp6 0 0 ip6-localhost:ipp [::]:* LISTEN
|
||||
tcp6 0 0 ip6-localhost:smtp [::]:* LISTEN
|
||||
```
|
||||
|
||||
### netstat -rn command
|
||||
### host 命令
|
||||
|
||||
The **netstat -rn** command displays the system's routing table. The 192.168.0.1 address is the local gateway (Flags=UG).
|
||||
|
||||
```
|
||||
$ netstat -rn
|
||||
Kernel IP routing table
|
||||
Destination Gateway Genmask Flags MSS Window irtt Iface
|
||||
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0
|
||||
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
|
||||
```
|
||||
|
||||
### host command
|
||||
|
||||
The **host** command works a lot like **nslookup** by looking up the remote system's IP address, but also provides the system's mail handler.
|
||||
`host` 命令就像 `nslookup` 一样,用来查询远程系统的 IP 地址,但是还提供系统的邮箱处理地址。
|
||||
|
||||
```
|
||||
$ host world.std.com
|
||||
@ -127,9 +107,9 @@ world.std.com has address 192.74.137.5
|
||||
world.std.com mail is handled by 10 smtp.theworld.com.
|
||||
```
|
||||
|
||||
### nslookup command
|
||||
### nslookup 命令
|
||||
|
||||
The **nslookup** also provides information on the system (in this case, the local system) that is providing DNS lookup services.
|
||||
`nslookup` 还提供系统中(本例中是本地系统)提供 DNS 查询服务的信息。
|
||||
|
||||
```
|
||||
$ nslookup world.std.com
|
||||
@ -141,9 +121,9 @@ Name: world.std.com
|
||||
Address: 192.74.137.5
|
||||
```
|
||||
|
||||
### dig command
|
||||
### dig 命令
|
||||
|
||||
The **dig** command provides quitea lot of information on connecting to a remote system -- including the name server we are communicating with and how long the query takes to respond and is often used for troubleshooting.
|
||||
`dig` 命令提供了很多有关连接到远程系统的信息 - 包括与我们通信的名称服务器以及查询需要多长时间进行响应,并经常用于故障排除。
|
||||
|
||||
```
|
||||
$ dig world.std.com
|
||||
@ -168,9 +148,9 @@ world.std.com. 78146 IN A 192.74.137.5
|
||||
;; MSG SIZE rcvd: 58
|
||||
```
|
||||
|
||||
### nmap command
|
||||
### nmap 命令
|
||||
|
||||
The **nmap** command is most frequently used to probe remote systems, but can also be used to report on the services being offered by the local system. In the output below, we can see that ssh is available for logins, that smtp is servicing email, that a web site is active, and that an ipp print service is running.
|
||||
`nmap` 经常用于探查远程系统,但是同样也用于报告本地系统提供的服务。在下面的输出中,我们可以看到登录可以使用 ssh、smtp 用于电子邮箱、web 站点也是启用的,并且 ipp 打印服务正在运行。
|
||||
|
||||
```
|
||||
$ nmap localhost
|
||||
@ -188,15 +168,15 @@ PORT STATE SERVICE
|
||||
Nmap done: 1 IP address (1 host up) scanned in 0.09 seconds
|
||||
```
|
||||
|
||||
Linux systems provide a lot of useful commands for reviewing their network configuration and connections. If you run out of commands to explore, keep in mind that **apropos network** might point you toward even more.
|
||||
Linux 系统提供了很多有用的命令用于查看网络配置和连接。如果你都探索完了,请记住 `apropos network` 或许会让你了解更多。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.networkworld.com/article/3230519/linux/examining-network-connections-on-linux-systems.html
|
||||
|
||||
作者:[Sandra Henry-Stocker][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
译者:[geekpi](https://github.com/geekpi)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
@ -0,0 +1,80 @@
|
||||
面向初学者的 Linux 网络硬件:软件思维
|
||||
===========================================================
|
||||
|
||||

|
||||
|
||||
> 没有路由和桥接,我们将会成为孤独的小岛,你将会在这个网络教程中学到更多知识。
|
||||
|
||||
[Commons Zero][3]Pixabay
|
||||
|
||||
上周,我们学习了本地网络硬件知识,本周,我们将学习网络互联技术和在移动网络中的一些很酷的黑客技术。
|
||||
|
||||
### 路由器
|
||||
|
||||
网络路由器就是计算机网络中的一切,因为路由器连接着网络,没有路由器,我们就会成为孤岛。图一展示了一个简单的有线本地网络和一个无线接入点,所有设备都接入到互联网上,本地局域网的计算机连接到一个连接着防火墙或者路由器的以太网交换机上,防火墙或者路由器连接到网络服务供应商(ISP)提供的电缆箱、调制调节器、卫星上行系统……好像一切都在计算中,就像是一个带着不停闪烁的的小灯的盒子。当你的网络数据包离开你的局域网,进入广阔的互联网,它们穿过一个又一个路由器直到到达自己的目的地。
|
||||
|
||||

|
||||
|
||||
*图一:一个简单的有线局域网和一个无线接入点。*
|
||||
|
||||
路由器可以是各种样式:一个只专注于路由的小巧特殊的小盒子,一个将会提供路由、防火墙、域名服务,以及 VPN 网关功能的大点的盒子,一台重新设计的台式电脑或者笔记本,一个树莓派计算机或者一个 Arduino,体积臃肿矮小的像 PC Engines 这样的单板计算机,除了苛刻的用途以外,普通的商品硬件都能良好的工作运行。高端的路由器使用特殊设计的硬件每秒能够传输最大量的数据包。它们有多路数据总线,多个中央处理器和极快的存储。(可以通过了解 Juniper 和思科的路由器来感受一下高端路由器书什么样子的,而且能看看里面是什么样的构造。)
|
||||
|
||||
接入你的局域网的无线接入点要么作为一个以太网网桥,要么作为一个路由器。桥接器扩展了这个网络,所以在这个桥接器上的任意一端口上的主机都连接在同一个网络中。一台路由器连接的是两个不同的网络。
|
||||
|
||||
### 网络拓扑
|
||||
|
||||
有多种设置你的局域网的方式,你可以把所有主机接入到一个单独的<ruby>平面网络<rt>flat network</rt></ruby>,也可以把它们划分为不同的子网。如果你的交换机支持 VLAN 的话,你也可以把它们分配到不同的 VLAN 中。
|
||||
|
||||
平面网络是最简单的网络,只需把每一台设备接入到同一个交换机上即可,如果一台交换上的端口不够使用,你可以将更多的交换机连接在一起。有些交换机有特殊的上行端口,有些是没有这种特殊限制的上行端口,你可以连接其中的任意端口,你可能需要使用交叉类型的以太网线,所以你要查阅你的交换机的说明文档来设置。
|
||||
|
||||
平面网络是最容易管理的,你不需要路由器也不需要计算子网,但它也有一些缺点。它们的伸缩性不好,所以当网络规模变得越来越大的时候就会被广播网络所阻塞。将你的局域网进行分段将会提升安全保障, 把局域网分成可管理的不同网段将有助于管理更大的网络。图二展示了一个分成两个子网的局域网络:内部的有线和无线主机,和一个托管公开服务的主机。包含面向公共的服务器的子网称作非军事区域 DMZ,(你有没有注意到那些都是主要在电脑上打字的男人们的术语?)因为它被阻挡了所有的内部网络的访问。
|
||||
|
||||

|
||||
|
||||
*图二:一个分成两个子网的简单局域网。*
|
||||
|
||||
即使像图二那样的小型网络也可以有不同的配置方法。你可以将防火墙和路由器放置在一台单独的设备上。你可以为你的非军事区域设置一个专用的网络连接,把它完全从你的内部网络隔离,这将引导我们进入下一个主题:一切基于软件。
|
||||
|
||||
### 软件思维
|
||||
|
||||
你可能已经注意到在这个简短的系列中我们所讨论的硬件,只有网络接口、交换机,和线缆是特殊用途的硬件。
|
||||
其它的都是通用的商用硬件,而且都是软件来定义它的用途。Linux 是一个真实的网络操作系统,它支持大量的网络操作:网关、虚拟专用网关、以太网桥、网页、邮箱以及文件等等服务器、负载均衡、代理、服务质量、多种认证、中继、故障转移……你可以在运行着 Linux 系统的标准硬件上运行你的整个网络。你甚至可以使用 Linux 交换应用(LISA)和VDE2 协议来模拟以太网交换机。
|
||||
|
||||
有一些用于小型硬件的特殊发行版,如 DD-WRT、OpenWRT,以及树莓派发行版,也不要忘记 BSD 们和它们的特殊衍生用途如 pfSense 防火墙/路由器,和 FreeNAS 网络存储服务器。
|
||||
|
||||
你知道有些人坚持认为硬件防火墙和软件防火墙有区别?其实是没有区别的,就像说硬件计算机和软件计算机一样。
|
||||
|
||||
### 端口聚合和以太网绑定
|
||||
|
||||
聚合和绑定,也称链路聚合,是把两条以太网通道绑定在一起成为一条通道。一些交换机支持端口聚合,就是把两个交换机端口绑定在一起,成为一个是它们原来带宽之和的一条新的连接。对于一台承载很多业务的服务器来说这是一个增加通道带宽的有效的方式。
|
||||
|
||||
你也可以在以太网口进行同样的配置,而且绑定汇聚的驱动是内置在 Linux 内核中的,所以不需要任何其他的专门的硬件。
|
||||
|
||||
### 随心所欲选择你的移动宽带
|
||||
|
||||
我期望移动宽带能够迅速增长来替代 DSL 和有线网络。我居住在一个有 25 万人口的靠近一个城市的地方,但是在城市以外,要想接入互联网就要靠运气了,即使那里有很大的用户上网需求。我居住的小角落离城镇有 20 分钟的距离,但对于网络服务供应商来说他们几乎不会考虑到为这个地方提供网络。 我唯一的选择就是移动宽带;这里没有拨号网络、卫星网络(即使它很糟糕)或者是 DSL、电缆、光纤,但却没有阻止网络供应商把那些我在这个区域从没看到过的 Xfinity 和其它高速网络服务的传单塞进我的邮箱。
|
||||
|
||||
我试用了 AT&T、Version 和 T-Mobile。Version 的信号覆盖范围最广,但是 Version 和 AT&T 是最昂贵的。
|
||||
我居住的地方在 T-Mobile 信号覆盖的边缘,但迄今为止他们给了最大的优惠,为了能够能够有效的使用,我必须购买一个 WeBoost 信号放大器和一台中兴的移动热点设备。当然你也可以使用一部手机作为热点,但是专用的热点设备有着最强的信号。如果你正在考虑购买一台信号放大器,最好的选择就是 WeBoost,因为他们的服务支持最棒,而且他们会尽最大努力去帮助你。在一个小小的 APP [SignalCheck Pro][8] 的协助下设置将会精准的增强你的网络信号,他们有一个功能较少的免费的版本,但你将一点都不会后悔去花两美元使用专业版。
|
||||
|
||||
那个小巧的中兴热点设备能够支持 15 台主机,而且还有拥有基本的防火墙功能。 但你如果你使用像 Linksys WRT54GL这样的设备,可以使用 Tomato、OpenWRT,或者 DD-WRT 来替代普通的固件,这样你就能完全控制你的防护墙规则、路由配置,以及任何其它你想要设置的服务。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linux.com/learn/intro-to-linux/2017/10/linux-networking-hardware-beginners-think-software
|
||||
|
||||
作者:[CARLA SCHRODER][a]
|
||||
译者:[FelixYFZ](https://github.com/FelixYFZ)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linux.com/users/cschroder
|
||||
[1]:https://www.linux.com/licenses/category/used-permission
|
||||
[2]:https://www.linux.com/licenses/category/used-permission
|
||||
[3]:https://www.linux.com/licenses/category/creative-commons-zero
|
||||
[4]:https://www.linux.com/files/images/fig-1png-7
|
||||
[5]:https://www.linux.com/files/images/fig-2png-4
|
||||
[6]:https://www.linux.com/files/images/soderskar-islandjpg
|
||||
[7]:https://www.linux.com/learn/intro-to-linux/2017/10/linux-networking-hardware-beginners-lan-hardware
|
||||
[8]:http://www.bluelinepc.com/signalcheck/
|
@ -0,0 +1,43 @@
|
||||
回复:块层介绍第一部分 - 块 I/O 层
|
||||
============================================================
|
||||
|
||||
### 块层介绍第一部分:块 I/O 层
|
||||
|
||||
回复:amarao 在[块层介绍第一部分:块 I/O 层][1] 中提的问题
|
||||
先前的文章:[块层介绍第一部分:块 I/O 层][2]
|
||||
|
||||

|
||||
|
||||
嗨,
|
||||
|
||||
你在这里描述的问题与块层不直接相关。这可能是一个驱动错误、可能是一个 SCSI 层错误,但绝对不是一个块层的问题。
|
||||
|
||||
不幸的是,报告针对 Linux 的错误是一件难事。有些开发者拒绝去看 bugzilla,有些开发者喜欢它,有些(像我这样)只能勉强地使用它。
|
||||
|
||||
另一种方法是发送电子邮件。为此,你需要选择正确的邮件列表,还有也许是正确的开发人员,当他们心情愉快,或者不是太忙或者不是假期时找到它们。有些人会努力回复所有,有些是完全不可预知的 - 这对我来说通常会发送一个补丁,包含一些错误报告。如果你只是有一个你自己几乎都不了解的 bug,那么你的预期响应率可能会更低。很遗憾,但这是是真的。
|
||||
|
||||
许多 bug 都会得到回应和处理,但很多 bug 都没有。
|
||||
|
||||
我不认为说没有人关心是公平的,但是没有人认为它如你想的那样重要是有可能的。如果你想要一个解决方案,那么你需要驱动它。一个驱动它的方法是花钱请顾问或者与经销商签订支持合同。我怀疑你的情况没有上面的可能。另一种方法是了解代码如何工作,并自己找到解决方案。很多人都这么做,但是这对你来说可能不是一种选择。另一种方法是在不同的相关论坛上不断提出问题,直到得到回复。坚持可以见效。你需要做好准备去执行任何你所要求的测试,可能包括建立一个新的内核来测试。
|
||||
|
||||
如果你能在最近的内核(4.12 或者更新)上复现这个 bug,我建议你邮件报告给 linux-kernel@vger.kernel.org、linux-scsi@vger.kernel.org 和我(neilb@suse.com)(注意你不必订阅这些列表来发送邮件,只需要发送就行)。描述你的硬件以及如何触发问题的。
|
||||
|
||||
包含所有进程状态是 “D” 的栈追踪。你可以用 “cat /proc/$PID/stack” 来得到它,这里的 “$PID” 是进程的 pid。
|
||||
|
||||
确保避免抱怨或者说这个已经坏了好几年了以及这是多么严重不足。没有人关心这个。我们关心的是 bug 以及如何修复它。因此只要报告相关的事实就行。
|
||||
|
||||
尝试在邮件中而不是链接到其他地方的链接中包含所有事实。有时链接是需要的,但是对于你的脚本,它只有 8 行,所以把它包含在邮件中就行(并避免像 “fuckup” 之类的描述。只需称它为“坏的”(broken)或者类似的)。同样确保你的邮件发送的不是 HTML 格式。我们喜欢纯文本。HTML 被所有的 @vger.kernel.org 邮件列表拒绝。你或许需要配置你的邮箱程序不发送 HTML。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://lwn.net/Articles/737655/
|
||||
|
||||
作者:[neilbrown][a]
|
||||
译者:[geekpi](https://github.com/geekpi)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://lwn.net/Articles/737655/
|
||||
[1]:https://lwn.net/Articles/737588/
|
||||
[2]:https://lwn.net/Articles/736534/
|
57
published/201711/20141028 When Does Your OS Run.md
Normal file
57
published/201711/20141028 When Does Your OS Run.md
Normal file
@ -0,0 +1,57 @@
|
||||
操作系统何时运行?
|
||||
============================================================
|
||||
|
||||
请各位思考以下问题:在你阅读本文的这段时间内,计算机中的操作系统在**运行**吗?又或者仅仅是 Web 浏览器在运行?又或者它们也许均处于空闲状态,等待着你的指示?
|
||||
|
||||
这些问题并不复杂,但它们深入涉及到系统软件工作的本质。为了准确回答这些问题,我们需要透彻理解操作系统的行为模型,包括性能、安全和除错等方面。在该系列文章中,我们将以 Linux 为主举例来帮助你建立操作系统的行为模型,OS X 和 Windows 在必要的时候也会有所涉及。对那些深度探索者,我会在适当的时候给出 Linux 内核源码的链接。
|
||||
|
||||
这里有一个基本认知,就是,在任意给定时刻,某个 CPU 上仅有一个任务处于活动状态。大多数情形下这个任务是某个用户程序,例如你的 Web 浏览器或音乐播放器,但它也可能是一个操作系统线程。可以确信的是,它是**一个任务**,不是两个或更多,也不是零个,对,**永远**是一个。
|
||||
|
||||
这听上去可能会有些问题。比如,你的音乐播放器是否会独占 CPU 而阻止其它任务运行?从而使你不能打开任务管理工具去杀死音乐播放器,甚至让鼠标点击也失效,因为操作系统没有机会去处理这些事件。你可能会愤而喊出,“它究竟在搞什么鬼?”,并引发骚乱。
|
||||
|
||||
此时便轮到**中断**大显身手了。中断就好比,一声巨响或一次拍肩后,神经系统通知大脑去感知外部刺激一般。计算机主板上的[芯片组][1]同样会中断 CPU 运行以传递新的外部事件,例如键盘上的某个键被按下、网络数据包的到达、一次硬盘读取的完成,等等。硬件外设、主板上的中断控制器和 CPU 本身,它们共同协作实现了中断机制。
|
||||
|
||||
中断对于记录我们最珍视的资源——时间——也至关重要。计算机[启动过程][2]中,操作系统内核会设置一个硬件计时器以让其产生周期性**计时中断**,例如每隔 10 毫秒触发一次。每当计时中断到来,内核便会收到通知以更新系统统计信息和盘点如下事项:当前用户程序是否已运行了足够长时间?是否有某个 TCP 定时器超时了?中断给予了内核一个处理这些问题并采取合适措施的机会。这就好像你给自己设置了整天的周期闹铃并把它们用作检查点:我是否应该去做我正在进行的工作?是否存在更紧急的事项?直到你发现 10 年时间已逝去……
|
||||
|
||||
这些内核对 CPU 周期性的劫持被称为<ruby>滴答<rt>tick</rt></ruby>,也就是说,是中断让你的操作系统滴答了一下。不止如此,中断也被用作处理一些软件事件,如整数溢出和页错误,其中未涉及外部硬件。**中断是进入操作系统内核最频繁也是最重要的入口**。对于学习电子工程的人而言,这些并无古怪,它们是操作系统赖以运行的机制。
|
||||
|
||||
说到这里,让我们再来看一些实际情形。下图示意了 Intel Core i5 系统中的一个网卡中断。图片中的部分元素设置了超链,你可以点击它们以获取更为详细的信息,例如每个设备均被链接到了对应的 Linux 驱动源码。
|
||||
|
||||

|
||||
|
||||
链接如下:
|
||||
|
||||
- network card : https://github.com/torvalds/linux/blob/v3.17/drivers/net/ethernet/intel/e1000e/netdev.c
|
||||
- USB keyboard : https://github.com/torvalds/linux/blob/v3.16/drivers/hid/usbhid/usbkbd.c
|
||||
- I/O APIC : https://github.com/torvalds/linux/blob/v3.16/arch/x86/kernel/apic/io_apic.c
|
||||
- HPET : https://github.com/torvalds/linux/blob/v3.17/arch/x86/kernel/hpet.c
|
||||
|
||||
让我们来仔细研究下。首先,由于系统中存在众多中断源,如果硬件只是通知 CPU “嘿,这里发生了一些事情”然后什么也不做,则不太行得通。这会带来难以忍受的冗长等待。因此,计算机上电时,每个设备都被授予了一根**中断线**,或者称为 IRQ。这些 IRQ 然后被系统中的中断控制器映射成值介于 0 到 255 之间的**中断向量**。等到中断到达 CPU,它便具备了一个完好定义的数值,异于硬件的某些其它诡异行为。
|
||||
|
||||
相应地,CPU 中还存有一个由内核维护的指针,指向一个包含 255 个函数指针的数组,其中每个函数被用来处理某个特定的中断向量。后文中,我们将继续深入探讨这个数组,它也被称作**中断描述符表**(IDT)。
|
||||
|
||||
每当中断到来,CPU 会用中断向量的值去索引中断描述符表,并执行相应处理函数。这相当于,在当前正在执行任务的上下文中,发生了一个特殊函数调用,从而允许操作系统以较小开销快速对外部事件作出反应。考虑下述场景,Web 服务器在发送数据时,CPU 却间接调用了操作系统函数,这听上去要么很炫酷要么令人惊恐。下图展示了 Vim 编辑器运行过程中一个中断到来的情形。
|
||||
|
||||

|
||||
|
||||
此处请留意,中断的到来是如何触发 CPU 到 [Ring 0][3] 内核模式的切换而未有改变当前活跃的任务。这看上去就像,Vim 编辑器直接面向操作系统内核产生了一次神奇的函数调用,但 Vim 还在那里,它的[地址空间][4]原封未动,等待着执行流返回。
|
||||
|
||||
这很令人振奋,不是么?不过让我们暂且告一段落吧,我需要合理控制篇幅。我知道还没有回答完这个开放式问题,甚至还实质上翻开了新的问题,但你至少知道了在你读这个句子的同时**滴答**正在发生。我们将在充实了对操作系统动态行为模型的理解之后再回来寻求问题的答案,对 Web 浏览器情形的理解也会变得清晰。如果你仍有问题,尤其是在这篇文章公诸于众后,请尽管提出。我将会在文章或后续评论中回答它们。下篇文章将于明天在 RSS 和 Twitter 上发布。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: http://duartes.org/gustavo/blog/post/when-does-your-os-run/
|
||||
|
||||
作者:[gustavo][a]
|
||||
译者:[Cwndmiao](https://github.com/Cwndmiao)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:http://duartes.org/gustavo/blog/about/
|
||||
[1]:http://duartes.org/gustavo/blog/post/motherboard-chipsets-memory-map
|
||||
[2]:http://duartes.org/gustavo/blog/post/kernel-boot-process
|
||||
[3]:http://duartes.org/gustavo/blog/post/cpu-rings-privilege-and-protection
|
||||
[4]:http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory
|
||||
[5]:http://feeds.feedburner.com/GustavoDuarte
|
||||
[6]:http://twitter.com/food4hackers
|
@ -0,0 +1,157 @@
|
||||
tmate:秒级分享你的终端会话
|
||||
=================
|
||||
|
||||
不久前,我们写过一篇关于 [teleconsole](https://www.2daygeek.com/teleconsole-share-terminal-session-instantly-to-anyone-in-seconds/) 的介绍,该工具可用于快速分享终端给任何人(任何你信任的人)。今天我们要聊一聊另一款类似的应用,名叫 `tmate`。
|
||||
|
||||
`tmate` 有什么用?它可以让你在需要帮助时向你的朋友们求助。
|
||||
|
||||
### 什么是 tmate?
|
||||
|
||||
[tmate](https://tmate.io/) 的意思是 `teammates`,它是 tmux 的一个分支,并且使用相同的配置信息(例如快捷键配置,配色方案等)。它是一个终端多路复用器,同时具有即时分享终端的能力。它允许在单个屏幕中创建并操控多个终端,同时这些终端还能与其他同事分享。
|
||||
|
||||
你可以分离会话,让作业在后台运行,然后在想要查看状态时重新连接会话。`tmate` 提供了一个即时配对的方案,让你可以与一个或多个队友共享一个终端。
|
||||
|
||||
在屏幕的地步有一个状态栏,显示了当前会话的一些诸如 ssh 命令之类的共享信息。
|
||||
|
||||
### tmate 是怎么工作的?
|
||||
|
||||
- 运行 `tmate` 时,会通过 `libssh` 在后台创建一个连接到 tmate.io (由 tmate 开发者维护的后台服务器)的 ssh 连接。
|
||||
- tmate.io 服务器的 ssh 密钥通过 DH 交换进行校验。
|
||||
- 客户端通过本地 ssh 密钥进行认证。
|
||||
- 连接创建后,本地 tmux 服务器会生成一个 150 位(不可猜测的随机字符)会话令牌。
|
||||
- 队友能通过用户提供的 SSH 会话 ID 连接到 tmate.io。
|
||||
|
||||
### 使用 tmate 的必备条件
|
||||
|
||||
由于 `tmate.io` 服务器需要通过本地 ssh 密钥来认证客户机,因此其中一个必备条件就是生成 SSH 密钥 key。
|
||||
记住,每个系统都要有自己的 SSH 密钥。
|
||||
|
||||
```shell
|
||||
$ ssh-keygen -t rsa
|
||||
Generating public/private rsa key pair.
|
||||
Enter file in which to save the key (/home/magi/.ssh/id_rsa):
|
||||
Enter passphrase (empty for no passphrase):
|
||||
Enter same passphrase again:
|
||||
Your identification has been saved in /home/magi/.ssh/id_rsa.
|
||||
Your public key has been saved in /home/magi/.ssh/id_rsa.pub.
|
||||
The key fingerprint is:
|
||||
SHA256:3ima5FuwKbWyyyNrlR/DeBucoyRfdOtlUmb5D214NC8 magi@magi-VirtualBox
|
||||
The key's randomart image is:
|
||||
+---[RSA 2048]----+
|
||||
| |
|
||||
| |
|
||||
| . |
|
||||
| . . = o |
|
||||
| *ooS= . + o |
|
||||
| . =.@*o.o.+ E .|
|
||||
| =o==B++o = . |
|
||||
| o.+*o+.. . |
|
||||
| ..o+o=. |
|
||||
+----[SHA256]-----+
|
||||
```
|
||||
|
||||
### 如何安装 tmate
|
||||
|
||||
`tmate` 已经包含在某些发行版的官方仓库中,可以通过包管理器来安装。
|
||||
|
||||
对于 Debian/Ubuntu,可以使用 [APT-GET 命令](https://www.2daygeek.com/apt-get-apt-cache-command-examples-manage-packages-debian-ubuntu-systems/)或者 [APT 命令](https://www.2daygeek.com/apt-command-examples-manage-packages-debian-ubuntu-systems/)to 来安装。
|
||||
|
||||
```shell
|
||||
$ sudo apt-get install software-properties-common
|
||||
$ sudo add-apt-repository ppa:tmate.io/archive
|
||||
$ sudo apt-get update
|
||||
$ sudo apt-get install tmate
|
||||
```
|
||||
|
||||
你也可以从官方仓库中安装 tmate。
|
||||
|
||||
```shell
|
||||
$ sudo apt-get install tmate
|
||||
```
|
||||
|
||||
对于 Fedora,使用 [DNF 命令](https://www.2daygeek.com/dnf-command-examples-manage-packages-fedora-system/) 来安装。
|
||||
|
||||
```shell
|
||||
$ sudo dnf install tmate
|
||||
```
|
||||
|
||||
对于基于 Arch Linux 的系统,使用 []()[Yaourt 命令](https://www.2daygeek.com/install-yaourt-aur-helper-on-arch-linux/)或 []()[Packer 命令](https://www.2daygeek.com/install-packer-aur-helper-on-arch-linux/) 来从 AUR 仓库中安装。
|
||||
|
||||
```shell
|
||||
$ yaourt -S tmate
|
||||
```
|
||||
或
|
||||
|
||||
```shell
|
||||
$ packer -S tmate
|
||||
```
|
||||
|
||||
对于 openSUSE,使用 [Zypper 命令](https://www.2daygeek.com/zypper-command-examples-manage-packages-opensuse-system/) 来安装。
|
||||
|
||||
```shell
|
||||
$ sudo zypper in tmate
|
||||
```
|
||||
|
||||
### 如何使用 tmate
|
||||
|
||||
成功安装后,打开终端然后输入下面命令,就会打开一个新的会话,在屏幕底部,你能看到 SSH 会话的 ID。
|
||||
|
||||
```shell
|
||||
$ tmate
|
||||
```
|
||||
|
||||

|
||||
|
||||
要注意的是,SSH 会话 ID 会在几秒后消失,不过不要紧,你可以通过下面命令获取到这些详细信息。
|
||||
|
||||
```shell
|
||||
$ tmate show-messages
|
||||
```
|
||||
|
||||
`tmate` 的 `show-messages` 命令会显示 tmate 的日志信息,其中包含了该 ssh 连接内容。
|
||||
|
||||

|
||||
|
||||
现在,分享你的 SSH 会话 ID 给你的朋友或同事从而允许他们观看终端会话。除了 SSH 会话 ID 以外,你也可以分享 web URL。
|
||||
|
||||
另外你还可以选择分享的是只读会话还是可读写会话。
|
||||
|
||||
### 如何通过 SSH 连接会话
|
||||
|
||||
只需要在终端上运行你从朋友那得到的 SSH 终端 ID 就行了。类似下面这样。
|
||||
|
||||
```shell
|
||||
$ ssh session: ssh 3KuRj95sEZRHkpPtc2y6jcokP@sg2.tmate.io
|
||||
```
|
||||
|
||||

|
||||
|
||||
### 如何通过 Web URL 连接会话
|
||||
|
||||
打开浏览器然后访问朋友给你的 URL 就行了。像下面这样。
|
||||
|
||||

|
||||
|
||||
|
||||
只需要输入 `exit` 就能退出会话了。
|
||||
|
||||
```
|
||||
[Source System Output]
|
||||
[exited]
|
||||
|
||||
[Remote System Output]
|
||||
[server exited]
|
||||
Connection to sg2.tmate.io closed by remote host。
|
||||
Connection to sg2.tmate.io closed。
|
||||
```
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.2daygeek.com/tmate-instantly-share-your-terminal-session-to-anyone-in-seconds/
|
||||
|
||||
作者:[Magesh Maruthamuthu][a]
|
||||
译者:[lujun9972](https://github.com/lujun9972)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
81
published/20171120 Containers and Kubernetes Whats next.md
Normal file
81
published/20171120 Containers and Kubernetes Whats next.md
Normal file
@ -0,0 +1,81 @@
|
||||
容器技术和 K8S 的下一站
|
||||
============================================================
|
||||
> 想知道容器编排管理和 K8S 的最新展望么?来看看专家怎么说。
|
||||
|
||||

|
||||
|
||||
如果你想对容器在未来的发展方向有一个整体把握,那么你一定要跟着钱走,看看钱都投在了哪里。当然了,有很多很多的钱正在投入容器的进一步发展。相关研究预计 2020 年容器技术的投入将占有 [27 亿美元][4] 的市场份额。而在 2016 年,容器相关技术投入的总额为 7.62 亿美元,只有 2020 年投入预计的三分之一。巨额投入的背后是一些显而易见的基本因素,包括容器化的迅速增长以及并行化的大趋势。随着容器被大面积推广和使用,容器编排管理也会被理所当然的推广应用起来。
|
||||
|
||||
来自 [The new stack][5] 的调研数据表明,容器的推广使用是编排管理被推广的主要的催化剂。根据调研参与者的反馈数据,在已经将容器技术使用到生产环境中的使用者里,有六成使用者正在将 Kubernetes(K8S)编排管理广泛的应用在生产环境中,另外百分之十九的人员则表示他们已经处于部署 K8S 的初级阶段。在容器部署初期的使用者当中,虽然只有百分之五的人员表示已经在使用 K8S ,但是百分之五十八的人员表示他们正在计划和准备使用 K8S。总而言之,容器和 Kubernetes 的关系就好比是鸡和蛋一样,相辅相成紧密关联。众多专家一致认为编排管理工具对容器的[长周期管理][6] 以及其在市场中的发展有至关重要的作用。正如 [Cockroach 实验室][7] 的 Alex Robinson 所说,容器编排管理被更广泛的拓展和应用是一个总体的大趋势。毫无疑问,这是一个正在快速演变的领域,且未来潜力无穷。鉴于此,我们对 Robinson 和其他的一些容器的实际使用和推介者做了采访,来从他们作为容器技术的践行者的视角上展望一下容器编排以及 K8S 的下一步发展。
|
||||
|
||||
### 容器编排将被主流接受
|
||||
|
||||
像任何重要技术的转型一样,我们就像是处在一个高崖之上一般,在经过了初期步履蹒跚的跋涉之后将要来到一望无际的广袤平原。广大的新天地和平实真切的应用需求将会让这种新技术在主流应用中被迅速推广,尤其是在大企业环境中。正如 Alex Robinson 说的那样,容器技术的淘金阶段已经过去,早期的技术革新创新正在减速,随之而来的则是市场对容器技术的稳定性和可用性的强烈需求。这意味着未来我们将不会再见到大量的新的编排管理系统的涌现,而是会看到容器技术方面更多的安全解决方案,更丰富的管理工具,以及基于目前主流容器编排系统的更多的新特性。
|
||||
|
||||
### 更好的易用性
|
||||
|
||||
人们将在简化容器的部署方面下大功夫,因为容器部署的初期工作对很多公司和组织来说还是比较复杂的,尤其是容器的[长期管理维护][8]更是需要投入大量的精力。正如 [Codemill AB][9] 公司的 My Karlsson 所说,容器编排技术还是太复杂了,这导致很多使用者难以娴熟驾驭和充分利用容器编排的功能。很多容器技术的新用户都需要花费很多精力,走很多弯路,才能搭建小规模的或单个的以隔离方式运行的容器系统。这种现象在那些没有针对容器技术设计和优化的应用中更为明显。在简化容器编排管理方面有很多优化可以做,这些优化和改造将会使容器技术更加具有可用性。
|
||||
|
||||
### 在混合云以及多云技术方面会有更多侧重
|
||||
|
||||
随着容器和容器编排技术被越来越多的使用,更多的组织机构会选择扩展他们现有的容器技术的部署,从之前的把非重要系统部署在单一环境的使用情景逐渐过渡到更加[复杂的使用情景][10]。对很多公司来说,这意味着他们必须开始学会在 [混合云][11] 和 [多云][12] 的环境下,全局化的去管理那些容器化的应用和微服务。正如红帽 [Openshift 部门产品战略总监][14] [Brian Gracely][13] 所说,“容器和 K8S 技术的使用使得我们成功的实现了混合云以及应用的可移植性。结合 Open Service Broker API 的使用,越来越多的结合私有云和公有云资源的新应用将会涌现出来。”
|
||||
据 [CloudBees][15] 公司的高级工程师 Carlos Sanchez 分析,联合服务(Federation)将会得到极大推动,使一些诸如多地区部署和多云部署等的备受期待的新特性成为可能。
|
||||
|
||||
**[ 想知道 CIO 们对混合云和多云的战略构想么? 请参看我们的这条相关资源, [Hybrid Cloud: The IT leader's guide][16]。 ]**
|
||||
|
||||
### 平台和工具的持续整合及加强
|
||||
|
||||
对任何一种科技来说,持续的整合和加强从来都是大势所趋;容器编排管理技术在这方面也不例外。来自 [Sumo Logic][17] 的首席分析师 Ben Newton 表示,随着容器化渐成主流,软件工程师们正在很少数的一些技术上做持续整合加固的工作,来满足他们的一些微应用的需求。容器和 K8S 将会毫无疑问的成为容器编排管理方面的主流平台,并轻松碾压其它的一些小众平台方案。因为 K8S 提供了一个相当清晰的可以摆脱各种特有云生态的途径,K8S 将被大量公司使用,逐渐形成一个不依赖于某个特定云服务的<ruby>“中立云”<rt>cloud-neutral</rt></ruby>。
|
||||
|
||||
### K8S 的下一站
|
||||
|
||||
来自 [Alcide][18] 的 CTO 和联合创始人 Gadi Naor 表示,K8S 将会是一个有长期和远景发展的技术,虽然我们的社区正在大力推广和发展 K8S,K8S 仍有很长的路要走。
|
||||
|
||||
专家们对[日益流行的 K8S 平台][19]也作出了以下一些预测:
|
||||
|
||||
**_来自 Alcide 的 Gadi Naor 表示:_** “运营商会持续演进并趋于成熟,直到在 K8S 上运行的应用可以完全自治。利用 [OpenTracing][20] 和诸如 [istio][21] 技术的 service mesh 架构,在 K8S 上部署和监控微应用将会带来很多新的可能性。”
|
||||
|
||||
**_来自 Red Hat 的 Brian Gracely 表示:_** “K8S 所支持的应用的种类越来越多。今后在 K8S 上,你不仅可以运行传统的应用程序,还可以运行原生的云应用、大数据应用以及 HPC 或者基于 GPU 运算的应用程序,这将为灵活的架构设计带来无限可能。”
|
||||
|
||||
**_来自 Sumo Logic 的 Ben Newton 表示:_** “随着 K8S 成为一个具有统治地位的平台,我预计更多的操作机制将会被统一化,尤其是 K8S 将和第三方管理和监控平台融合起来。”
|
||||
|
||||
**_来自 CloudBees 的 Carlos Sanchez 表示:_** “在不久的将来我们就能看到不依赖于 Docker 而使用其它运行时环境的系统,这将会有助于消除任何可能的 lock-in 情景“ [编辑提示:[CRI-O][22] 就是一个可以借鉴的例子。]“而且我期待将来会出现更多的针对企业环境的存储服务新特性,包括数据快照以及在线的磁盘容量的扩展。”
|
||||
|
||||
**_来自 Cockroach Labs 的 Alex Robinson 表示:_** “ K8S 社区正在讨论的一个重大发展议题就是加强对[有状态程序][23]的管理。目前在 K8S 平台下,实现状态管理仍然非常困难,除非你所使用的云服务商可以提供远程固定磁盘。现阶段也有很多人在多方面试图改善这个状况,包括在 K8S 平台内部以及在外部服务商一端做出的一些改进。”
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
via: https://enterprisersproject.com/article/2017/11/containers-and-kubernetes-whats-next
|
||||
|
||||
作者:[Kevin Casey][a]
|
||||
译者:[yunfengHe](https://github.com/yunfengHe)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://enterprisersproject.com/user/kevin-casey
|
||||
[1]:https://enterprisersproject.com/article/2017/11/kubernetes-numbers-10-compelling-stats
|
||||
[2]:https://enterprisersproject.com/article/2017/11/how-enterprise-it-uses-kubernetes-tame-container-complexity
|
||||
[3]:https://enterprisersproject.com/article/2017/11/5-kubernetes-success-tips-start-smart?sc_cid=70160000000h0aXAAQ
|
||||
[4]:https://451research.com/images/Marketing/press_releases/Application-container-market-will-reach-2-7bn-in-2020_final_graphic.pdf
|
||||
[5]:https://thenewstack.io/
|
||||
[6]:https://enterprisersproject.com/article/2017/10/microservices-and-containers-6-management-tips-long-haul
|
||||
[7]:https://www.cockroachlabs.com/
|
||||
[8]:https://enterprisersproject.com/article/2017/10/microservices-and-containers-6-management-tips-long-haul
|
||||
[9]:https://codemill.se/
|
||||
[10]:https://www.redhat.com/en/challenges/integration?intcmp=701f2000000tjyaAAA
|
||||
[11]:https://enterprisersproject.com/hybrid-cloud
|
||||
[12]:https://enterprisersproject.com/article/2017/7/multi-cloud-vs-hybrid-cloud-whats-difference
|
||||
[13]:https://enterprisersproject.com/user/brian-gracely
|
||||
[14]:https://www.redhat.com/en
|
||||
[15]:https://www.cloudbees.com/
|
||||
[16]:https://enterprisersproject.com/hybrid-cloud?sc_cid=70160000000h0aXAAQ
|
||||
[17]:https://www.sumologic.com/
|
||||
[18]:http://alcide.io/
|
||||
[19]:https://enterprisersproject.com/article/2017/10/how-explain-kubernetes-plain-english
|
||||
[20]:http://opentracing.io/
|
||||
[21]:https://istio.io/
|
||||
[22]:http://cri-o.io/
|
||||
[23]:https://opensource.com/article/17/2/stateful-applications
|
||||
[24]:https://enterprisersproject.com/article/2017/11/containers-and-kubernetes-whats-next?rate=PBQHhF4zPRHcq2KybE1bQgMkS2bzmNzcW2RXSVItmw8
|
||||
[25]:https://enterprisersproject.com/user/kevin-casey
|
@ -0,0 +1,75 @@
|
||||
如何在 Linux 下安装安卓文件传输助手
|
||||
===============
|
||||
|
||||
如果你尝试在 Ubuntu 下连接你的安卓手机,你也许可以试试 Linux 下的安卓文件传输助手。
|
||||
|
||||
本质上来说,这个应用是谷歌 macOS 版本的一个克隆。它是用 Qt 编写的,用户界面非常简洁,使得你能轻松在 Ubuntu 和安卓手机之间传输文件和文件夹。
|
||||
|
||||
现在,有可能一部分人想知道有什么是这个应用可以做,而 Nautilus(Ubuntu 默认的文件资源管理器)不能做的,答案是没有。
|
||||
|
||||
当我将我的 Nexus 5X(记得选择 [媒体传输协议 MTP][7] 选项)连接在 Ubuntu 上时,在 [GVfs][8](LCTT 译注: GNOME 桌面下的虚拟文件系统)的帮助下,我可以打开、浏览和管理我的手机,就像它是一个普通的 U 盘一样。
|
||||
|
||||
[][9]
|
||||
|
||||
但是*一些*用户在使用默认的文件管理器时,在 MTP 的某些功能上会出现问题:比如文件夹没有正确加载,创建新文件夹后此文件夹不存在,或者无法在媒体播放器中使用自己的手机。
|
||||
|
||||
这就是要为 Linux 系统用户设计一个安卓文件传输助手应用的原因,将这个应用当做将 MTP 设备安装在 Linux 下的另一种选择。如果你使用 Linux 下的默认应用时一切正常,你也许并不需要尝试使用它 (除非你真的很想尝试新鲜事物)。
|
||||
|
||||
|
||||

|
||||
|
||||
该 app 特点:
|
||||
|
||||
* 简洁直观的用户界面
|
||||
* 支持文件拖放功能(从 Linux 系统到手机)
|
||||
* 支持批量下载 (从手机到 Linux系统)
|
||||
* 显示传输进程对话框
|
||||
* FUSE 模块支持
|
||||
* 没有文件大小限制
|
||||
* 可选命令行工具
|
||||
|
||||
### Ubuntu 下安装安卓手机文件助手的步骤
|
||||
|
||||
以上就是对这个应用的介绍,下面是如何安装它的具体步骤。
|
||||
|
||||
这有一个 [PPA](个人软件包集)源为 Ubuntu 14.04 LTS、16.04 LTS 和 Ubuntu 17.10 提供可用应用。
|
||||
|
||||
为了将这一 PPA 加入你的软件资源列表中,执行这条命令:
|
||||
|
||||
```
|
||||
sudo add-apt-repository ppa:samoilov-lex/aftl-stable
|
||||
```
|
||||
|
||||
接着,为了在 Ubuntu 下安装 Linux版本的安卓文件传输助手,执行:
|
||||
|
||||
```
|
||||
sudo apt-get update && sudo apt install android-file-transfer
|
||||
```
|
||||
|
||||
这样就行了。
|
||||
|
||||
你会在你的应用列表中发现这一应用的启动图标。
|
||||
|
||||
在你启动这一应用之前,要确保没有其他应用(比如 Nautilus)已经挂载了你的手机。如果其它应用正在使用你的手机,就会显示“无法找到 MTP 设备”。要解决这一问题,将你的手机从 Nautilus(或者任何正在使用你的手机的应用)上移除,然后再重新启动安卓文件传输助手。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: http://www.omgubuntu.co.uk/2017/11/android-file-transfer-app-linux
|
||||
|
||||
作者:[JOEY SNEDDON][a]
|
||||
译者:[wenwensnow](https://github.com/wenwensnow)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://plus.google.com/117485690627814051450/?rel=author
|
||||
[1]:https://plus.google.com/117485690627814051450/?rel=author
|
||||
[2]:http://www.omgubuntu.co.uk/category/app
|
||||
[3]:http://www.omgubuntu.co.uk/category/download
|
||||
[4]:https://github.com/whoozle/android-file-transfer-linux
|
||||
[5]:http://www.omgubuntu.co.uk/2017/11/android-file-transfer-app-linux
|
||||
[6]:http://android.com/filetransfer?linkid=14270770
|
||||
[7]:https://en.wikipedia.org/wiki/Media_Transfer_Protocol
|
||||
[8]:https://en.wikipedia.org/wiki/GVfs
|
||||
[9]:http://www.omgubuntu.co.uk/wp-content/uploads/2017/11/browsing-android-mtp-nautilus.jpg
|
||||
[10]:https://launchpad.net/~samoilov-lex/+archive/ubuntu/aftl-stable
|
@ -0,0 +1,72 @@
|
||||
开源云技能认证:系统管理员的核心竞争力
|
||||
=========
|
||||
|
||||

|
||||
|
||||
> [2017年开源工作报告][1](以下简称“报告”)显示,具有开源云技术认证的系统管理员往往能获得更高的薪酬。
|
||||
|
||||
|
||||
报告调查的受访者中,53% 认为系统管理员是雇主们最期望被填补的职位空缺之一,因此,技术娴熟的系统管理员更受青睐而收获高薪职位,但这一职位,并没想象中那么容易填补。
|
||||
|
||||
系统管理员主要负责服务器和其他电脑操作系统的安装、服务支持和维护,及时处理服务中断和预防其他问题的出现。
|
||||
|
||||
总的来说,今年的报告指出开源领域人才需求最大的有开源云(47%),应用开发(44%),大数据(43%),开发运营和安全(42%)。
|
||||
|
||||
此外,报告对人事经理的调查显示,58% 期望招揽更多的开源人才,67% 认为开源人才的需求增长会比业内其他领域更甚。有些单位视开源人才为招聘最优选则,它们招聘的开源人才较上年增长了 2 个百分点。
|
||||
|
||||
同时,89% 的人事经理认为很难找到颇具天赋的开源人才。
|
||||
|
||||
### 为什么要获取认证
|
||||
|
||||
报告显示,对系统管理员的需求刺激着人事经理为 53% 的组织/机构提供正规的培训和专业技术认证,而这一比例去年为 47%。
|
||||
|
||||
对系统管理方面感兴趣的 IT 人才考虑获取 Linux 认证已成为行业规律。随便查看几个知名的招聘网站,你就能发现:[CompTIA Linux+][3] 认证是入门级 Linux 系统管理员的最高认证;如果想胜任高级别的系统管理员职位,获取[红帽认证工程师(RHCE)][4]和[红帽认证系统管理员(RHCSA)][5]则是不可或缺的。
|
||||
|
||||
戴士(Dice)[2017 技术行业薪资调查][6]显示,2016 年系统管理员的薪水为 79,538 美元,较上年下降了 0.8%;系统架构师的薪水为 125,946 美元,同比下降 4.7%。尽管如此,该调查发现“高水平专业人才仍最受欢迎,特别是那些精通支持产业转型发展所需技术的人才”。
|
||||
|
||||
在开源技术方面,HBase(一个开源的分布式数据库)技术人才的薪水在戴士 2017 技术行业薪资调查中排第一。在计算机网络和数据库领域,掌握 OpenVMS 操作系统技术也能获得高薪。
|
||||
|
||||
### 成为出色的系统管理员
|
||||
|
||||
出色的系统管理员须在问题出现时马上处理,这意味着你必须时刻准备应对可能出现的状况。这个职位追求“零责备的、精益的、流程或技术上交互式改进的”思维方式和善于自我完善的人格,成为一个系统管理员意味着“你必将与开源软件如 Linux、BSD 甚至开源 Solaris 等结下不解之缘”,Paul English ^译注1 在 [opensource.com][7] 上发文指出。
|
||||
|
||||
Paul English 认为,现在的系统管理员较以前而言,要更多地与软件打交道,而且要能够编写脚本来协助系统管理。
|
||||
|
||||
>译注1:Paul English,计算机科学学士,UNIX/Linux 系统管理员,PreOS Security Inc. 公司 CEO,2015-2017 年于为推动系统管理员发展实践的非盈利组织——<ruby>专业系统管理员联盟<rt>League of Professional System Administrator</rt></ruby>担任董事会成员。
|
||||
|
||||
### 展望 2018
|
||||
|
||||
[Robert Half 2018 年技术人才薪资导览][8]预测 2018 年北美地区许多单位将聘用大量系统管理方面的专业人才,同时个人软实力和领导力水平作为优秀人才的考量因素,越来越受到重视。
|
||||
|
||||
该报告指出:“良好的聆听能力和批判性思维能力对于理解和解决用户的问题和担忧至关重要,也是 IT 从业者必须具备的重要技能,特别是从事服务台和桌面支持工作相关的技术人员。”
|
||||
|
||||
这与[Linux基金会][9]^译注2 提出的不同阶段的系统管理员必备技能相一致,都强调了强大的分析能力和快速处理问题的能力。
|
||||
|
||||
>译注2:<ruby>Linux 基金会<rt>The Linux Foundation</rt></ruby>,成立于 2000 年,致力于围绕开源项目构建可持续发展的生态系统,以加速开源项目的技术开发和商业应用;它是世界上最大的开源非盈利组织,在推广、保护和推进 Linux 发展,协同开发,维护“历史上最大的共享资源”上功勋卓越。
|
||||
|
||||
如果想逐渐爬上系统管理员职位的金字塔上层,还应该对系统配置的结构化方法充满兴趣;且拥有解决系统安全问题的经验;用户身份验证管理的经验;与非技术人员进行非技术交流的能力;以及优化系统以满足最新的安全需求的能力。
|
||||
|
||||
- [下载][10]2017年开源工作报告全文,以获取更多信息。
|
||||
|
||||
|
||||
-----------------------
|
||||
|
||||
via: https://www.linux.com/blog/open-source-cloud-skills-and-certification-are-key-sysadmins
|
||||
|
||||
作者:[linux.com][a]
|
||||
译者:[wangy325](https://github.com/wangy325)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linux.com/blog/open-source-cloud-skills-and-certification-are-key-sysadmins
|
||||
[1]:https://www.linuxfoundation.org/blog/2017-jobs-report-highlights-demand-open-source-skills/
|
||||
[2]:https://www.linux.com/licenses/category/creative-commons-zero
|
||||
[3]:https://certification.comptia.org/certifications/linux?tracking=getCertified/certifications/linux.aspx
|
||||
[4]:https://www.redhat.com/en/services/certification/rhce
|
||||
[5]:https://www.redhat.com/en/services/certification/rhcsa
|
||||
[6]:http://marketing.dice.com/pdf/Dice_TechSalarySurvey_2017.pdf?aliId=105832232
|
||||
[7]:https://opensource.com/article/17/7/truth-about-sysadmins
|
||||
[8]:https://www.roberthalf.com/salary-guide/technology
|
||||
[9]:https://www.linux.com/learn/10-essential-skills-novice-junior-and-senior-sysadmins%20%20
|
||||
[10]:http://bit.ly/2017OSSjobsreport
|
@ -0,0 +1,134 @@
|
||||
Photon 也许能成为你最喜爱的容器操作系统
|
||||
============================================================
|
||||
|
||||

|
||||
|
||||
>Phonton OS 专注于容器,是一个非常出色的平台。 —— Jack Wallen
|
||||
|
||||
容器在当下的火热,并不是没有原因的。正如[之前][13]讨论的,容器可以使您轻松快捷地将新的服务与应用部署到您的网络上,而且并不耗费太多的系统资源。比起专用硬件和虚拟机,容器都是更加划算的,除此之外,他们更容易更新与重用。
|
||||
|
||||
更重要的是,容器喜欢 Linux(反之亦然)。不需要太多时间和麻烦,你就可以启动一台 Linux 服务器,运行[Docker][14],然后部署容器。但是,哪种 Linux 发行版最适合部署容器呢?我们的选择很多。你可以使用标准的 Ubuntu 服务器平台(更容易安装 Docker 并部署容器)或者是更轻量级的发行版 —— 专门用于部署容器。
|
||||
|
||||
[Photon][15] 就是这样的一个发行版。这个特殊的版本是由 [VMware][16] 于 2005 年创建的,它包含了 Docker 的守护进程,并可与容器框架(如 Mesos 和 Kubernetes )一起使用。Photon 经过优化可与 [VMware vSphere][17] 协同工作,而且可用于裸机、[Microsoft Azure][18]、 [Google Compute Engine][19]、 [Amazon Elastic Compute Cloud][20] 或者 [VirtualBox][21] 等。
|
||||
|
||||
Photon 通过只安装 Docker 守护进程所必需的东西来保持它的轻量。而这样做的结果是,这个发行版的大小大约只有 300MB。但这足以让 Linux 的运行一切正常。除此之外,Photon 的主要特点还有:
|
||||
|
||||
* 内核为性能而调整。
|
||||
* 内核根据[内核自防护项目][6](KSPP)进行了加固。
|
||||
* 所有安装的软件包都根据加固的安全标识来构建。
|
||||
* 操作系统在信任验证后启动。
|
||||
* Photon 的管理进程可以管理防火墙、网络、软件包,和远程登录在 Photon 机器上的用户。
|
||||
* 支持持久卷。
|
||||
* [Project Lightwave][7] 整合。
|
||||
* 及时的安全补丁与更新。
|
||||
|
||||
Photon 可以通过 [ISO 镜像][22]、[OVA][23]、[Amazon Machine Image][24]、[Google Compute Engine 镜像][25] 和 [Azure VHD][26] 安装使用。现在我将向您展示如何使用 ISO 镜像在 VirtualBox 上安装 Photon。整个安装过程大概需要五分钟,在最后您将有一台随时可以部署容器的虚拟机。
|
||||
|
||||
### 创建虚拟机
|
||||
|
||||
在部署第一台容器之前,您必须先创建一台虚拟机并安装 Photon。为此,打开 VirtualBox 并点击“新建”按钮。跟着创建虚拟机向导进行配置(根据您的容器将需要的用途,为 Photon 提供必要的资源)。在创建好虚拟机后,您所需要做的第一件事就是更改配置。选择新建的虚拟机(在 VirtualBox 主窗口的左侧面板中),然后单击“设置”。在弹出的窗口中,点击“网络”(在左侧的导航中)。
|
||||
|
||||
在“网络”窗口(图1)中,你需要在“连接”的下拉窗口中选择桥接。这可以确保您的 Photon 服务与您的网络相连。完成更改后,单击确定。
|
||||
|
||||

|
||||
|
||||
*图 1: 更改 Photon 在 VirtualBox 中的网络设置。[经许可使用][1]*
|
||||
|
||||
从左侧的导航选择您的 Photon 虚拟机,点击启动。系统会提示您去加载 ISO 镜像。当您完成之后,Photon 安装程序将会启动并提示您按回车后开始安装。安装过程基于 ncurses(没有 GUI),但它非常简单。
|
||||
|
||||
接下来(图2),系统会询问您是要最小化安装,完整安装还是安装 OSTree 服务器。我选择了完整安装。选择您所需要的任意选项,然后按回车继续。
|
||||
|
||||

|
||||
|
||||
*图 2: 选择您的安装类型。[经许可使用][2]*
|
||||
|
||||
在下一个窗口,选择您要安装 Photon 的磁盘。由于我们将其安装在虚拟机,因此只有一块磁盘会被列出(图3)。选择“自动”按下回车。然后安装程序会让您输入(并验证)管理员密码。在这之后镜像开始安装在您的磁盘上并在不到 5 分钟的时间内结束。
|
||||
|
||||

|
||||
|
||||
*图 3: 选择安装 Photon 的硬盘。[经许可使用][3]*
|
||||
|
||||
安装完成后,重启虚拟机并使用安装时创建的用户 root 和它的密码登录。一切就绪,你准备好开始工作了。
|
||||
|
||||
在开始使用 Docker 之前,您需要更新一下 Photon。Photon 使用 `yum` 软件包管理器,因此在以 root 用户登录后输入命令 `yum update`。如果有任何可用更新,则会询问您是否确认(图4)。
|
||||
|
||||

|
||||
|
||||
*图 4: 更新 Photon。[经许可使用][4]*
|
||||
|
||||
### 用法
|
||||
|
||||
正如我所说的,Photon 提供了部署容器甚至创建 Kubernetes 集群所需要的所有包。但是,在使用之前还要做一些事情。首先要启动 Docker 守护进程。为此,执行以下命令:
|
||||
|
||||
```
|
||||
systemctl start docker
|
||||
systemctl enable docker
|
||||
```
|
||||
|
||||
现在我们需要创建一个标准用户,以便我们可以不用 root 去运行 `docker` 命令。为此,执行以下命令:
|
||||
|
||||
```
|
||||
useradd -m USERNAME
|
||||
passwd USERNAME
|
||||
```
|
||||
|
||||
其中 “USERNAME” 是我们新增的用户的名称。
|
||||
|
||||
接下来,我们需要将这个新用户添加到 “docker” 组,执行命令:
|
||||
|
||||
```
|
||||
usermod -a -G docker USERNAME
|
||||
```
|
||||
|
||||
其中 “USERNAME” 是刚刚创建的用户的名称。
|
||||
|
||||
注销 root 用户并切换为新增的用户。现在,您已经可以不必使用 `sudo` 命令或者切换到 root 用户来使用 `docker` 命令了。从 Docker Hub 中取出一个镜像开始部署容器吧。
|
||||
|
||||
### 一个优秀的容器平台
|
||||
|
||||
在专注于容器方面,Photon 毫无疑问是一个出色的平台。请注意,Photon 是一个开源项目,因此没有任何付费支持。如果您对 Photon 有任何的问题,请移步 Photon 项目的 GitHub 下的 [Issues][27],那里可以供您阅读相关问题,或者提交您的问题。如果您对 Photon 感兴趣,您也可以在该项目的官方 [GitHub][28]中找到源码。
|
||||
|
||||
尝试一下 Photon 吧,看看它是否能够使得 Docker 容器和 Kubernetes 集群的部署更加容易。
|
||||
|
||||
欲了解 Linux 的更多信息,可以通过学习 Linux 基金会和 edX 的免费课程,[“Linux 入门”][29]。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linux.com/learn/intro-to-linux/2017/11/photon-could-be-your-new-favorite-container-os
|
||||
|
||||
作者:[JACK WALLEN][a]
|
||||
译者:[KeyLD](https://github.com/KeyLd)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linux.com/users/jlwallen
|
||||
[1]:https://www.linux.com/licenses/category/used-permission
|
||||
[2]:https://www.linux.com/licenses/category/used-permission
|
||||
[3]:https://www.linux.com/licenses/category/used-permission
|
||||
[4]:https://www.linux.com/licenses/category/used-permission
|
||||
[5]:https://www.linux.com/licenses/category/creative-commons-zero
|
||||
[6]:https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project
|
||||
[7]:http://vmware.github.io/lightwave/
|
||||
[8]:https://www.linux.com/files/images/photon0jpg
|
||||
[9]:https://www.linux.com/files/images/photon1jpg
|
||||
[10]:https://www.linux.com/files/images/photon2jpg
|
||||
[11]:https://www.linux.com/files/images/photon3jpg
|
||||
[12]:https://www.linux.com/files/images/photon-linuxjpg
|
||||
[13]:https://www.linux.com/learn/intro-to-linux/2017/11/how-install-and-use-docker-linux
|
||||
[14]:https://www.docker.com/
|
||||
[15]:https://vmware.github.io/photon/
|
||||
[16]:https://www.vmware.com/
|
||||
[17]:https://www.vmware.com/products/vsphere.html
|
||||
[18]:https://azure.microsoft.com/
|
||||
[19]:https://cloud.google.com/compute/
|
||||
[20]:https://aws.amazon.com/ec2/
|
||||
[21]:https://www.virtualbox.org/
|
||||
[22]:https://github.com/vmware/photon/wiki/Downloading-Photon-OS
|
||||
[23]:https://github.com/vmware/photon/wiki/Downloading-Photon-OS
|
||||
[24]:https://github.com/vmware/photon/wiki/Downloading-Photon-OS
|
||||
[25]:https://github.com/vmware/photon/wiki/Downloading-Photon-OS
|
||||
[26]:https://github.com/vmware/photon/wiki/Downloading-Photon-OS
|
||||
[27]:https://github.com/vmware/photon/issues
|
||||
[28]:https://github.com/vmware/photon
|
||||
[29]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux
|
@ -0,0 +1,149 @@
|
||||
如何判断 Linux 服务器是否被入侵?
|
||||
=========================
|
||||
|
||||
本指南中所谓的服务器被入侵或者说被黑了的意思,是指未经授权的人或程序为了自己的目的登录到服务器上去并使用其计算资源,通常会产生不好的影响。
|
||||
|
||||
免责声明:若你的服务器被类似 NSA 这样的国家机关或者某个犯罪集团入侵,那么你并不会注意到有任何问题,这些技术也无法发觉他们的存在。
|
||||
|
||||
然而,大多数被攻破的服务器都是被类似自动攻击程序这样的程序或者类似“脚本小子”这样的廉价攻击者,以及蠢蛋罪犯所入侵的。
|
||||
|
||||
这类攻击者会在访问服务器的同时滥用服务器资源,并且不怎么会采取措施来隐藏他们正在做的事情。
|
||||
|
||||
### 被入侵服务器的症状
|
||||
|
||||
当服务器被没有经验攻击者或者自动攻击程序入侵了的话,他们往往会消耗 100% 的资源。他们可能消耗 CPU 资源来进行数字货币的采矿或者发送垃圾邮件,也可能消耗带宽来发动 DoS 攻击。
|
||||
|
||||
因此出现问题的第一个表现就是服务器 “变慢了”。这可能表现在网站的页面打开的很慢,或者电子邮件要花很长时间才能发送出去。
|
||||
|
||||
那么你应该查看那些东西呢?
|
||||
|
||||
#### 检查 1 - 当前都有谁在登录?
|
||||
|
||||
你首先要查看当前都有谁登录在服务器上。发现攻击者登录到服务器上进行操作并不复杂。
|
||||
|
||||
其对应的命令是 `w`。运行 `w` 会输出如下结果:
|
||||
|
||||
```
|
||||
08:32:55 up 98 days, 5:43, 2 users, load average: 0.05, 0.03, 0.00
|
||||
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
|
||||
root pts/0 113.174.161.1 08:26 0.00s 0.03s 0.02s ssh root@coopeaa12
|
||||
root pts/1 78.31.109.1 08:26 0.00s 0.01s 0.00s w
|
||||
```
|
||||
|
||||
第一个 IP 是英国 IP,而第二个 IP 是越南 IP。这个不是个好兆头。
|
||||
|
||||
停下来做个深呼吸, 不要恐慌之下只是干掉他们的 SSH 连接。除非你能够防止他们再次进入服务器,否则他们会很快进来并踢掉你,以防你再次回去。
|
||||
|
||||
请参阅本文最后的“被入侵之后怎么办”这一章节来看找到了被入侵的证据后应该怎么办。
|
||||
|
||||
`whois` 命令可以接一个 IP 地址然后告诉你该 IP 所注册的组织的所有信息,当然就包括所在国家的信息。
|
||||
|
||||
#### 检查 2 - 谁曾经登录过?
|
||||
|
||||
Linux 服务器会记录下哪些用户,从哪个 IP,在什么时候登录的以及登录了多长时间这些信息。使用 `last` 命令可以查看这些信息。
|
||||
|
||||
输出类似这样:
|
||||
|
||||
```
|
||||
root pts/1 78.31.109.1 Thu Nov 30 08:26 still logged in
|
||||
root pts/0 113.174.161.1 Thu Nov 30 08:26 still logged in
|
||||
root pts/1 78.31.109.1 Thu Nov 30 08:24 - 08:26 (00:01)
|
||||
root pts/0 113.174.161.1 Wed Nov 29 12:34 - 12:52 (00:18)
|
||||
root pts/0 14.176.196.1 Mon Nov 27 13:32 - 13:53 (00:21)
|
||||
```
|
||||
|
||||
这里可以看到英国 IP 和越南 IP 交替出现,而且最上面两个 IP 现在还处于登录状态。如果你看到任何未经授权的 IP,那么请参阅最后章节。
|
||||
|
||||
登录后的历史记录会记录到二进制的 `/var/log/wtmp` 文件中(LCTT 译注:这里作者应该写错了,根据实际情况修改),因此很容易被删除。通常攻击者会直接把这个文件删掉,以掩盖他们的攻击行为。 因此, 若你运行了 `last` 命令却只看得见你的当前登录,那么这就是个不妙的信号。
|
||||
|
||||
如果没有登录历史的话,请一定小心,继续留意入侵的其他线索。
|
||||
|
||||
#### 检查 3 - 回顾命令历史
|
||||
|
||||
这个层次的攻击者通常不会注意掩盖命令的历史记录,因此运行 `history` 命令会显示出他们曾经做过的所有事情。
|
||||
一定留意有没有用 `wget` 或 `curl` 命令来下载类似垃圾邮件机器人或者挖矿程序之类的非常规软件。
|
||||
|
||||
命令历史存储在 `~/.bash_history` 文件中,因此有些攻击者会删除该文件以掩盖他们的所作所为。跟登录历史一样,若你运行 `history` 命令却没有输出任何东西那就表示历史文件被删掉了。这也是个不妙的信号,你需要很小心地检查一下服务器了。(LCTT 译注,如果没有命令历史,也有可能是你的配置错误。)
|
||||
|
||||
#### 检查 4 - 哪些进程在消耗 CPU?
|
||||
|
||||
你常遇到的这类攻击者通常不怎么会去掩盖他们做的事情。他们会运行一些特别消耗 CPU 的进程。这就很容易发现这些进程了。只需要运行 `top` 然后看最前的那几个进程就行了。
|
||||
|
||||
这也能显示出那些未登录进来的攻击者。比如,可能有人在用未受保护的邮件脚本来发送垃圾邮件。
|
||||
|
||||
如果你最上面的进程对不了解,那么你可以 Google 一下进程名称,或者通过 `losf` 和 `strace` 来看看它做的事情是什么。
|
||||
|
||||
使用这些工具,第一步从 `top` 中拷贝出进程的 PID,然后运行:
|
||||
|
||||
```
|
||||
strace -p PID
|
||||
```
|
||||
|
||||
这会显示出该进程调用的所有系统调用。它产生的内容会很多,但这些信息能告诉你这个进程在做什么。
|
||||
|
||||
```
|
||||
lsof -p PID
|
||||
```
|
||||
|
||||
这个程序会列出该进程打开的文件。通过查看它访问的文件可以很好的理解它在做的事情。
|
||||
|
||||
#### 检查 5 - 检查所有的系统进程
|
||||
|
||||
消耗 CPU 不严重的未授权进程可能不会在 `top` 中显露出来,不过它依然可以通过 `ps` 列出来。命令 `ps auxf` 就能显示足够清晰的信息了。
|
||||
|
||||
你需要检查一下每个不认识的进程。经常运行 `ps` (这是个好习惯)能帮助你发现奇怪的进程。
|
||||
|
||||
#### 检查 6 - 检查进程的网络使用情况
|
||||
|
||||
`iftop` 的功能类似 `top`,它会排列显示收发网络数据的进程以及它们的源地址和目的地址。类似 DoS 攻击或垃圾机器人这样的进程很容易显示在列表的最顶端。
|
||||
|
||||
#### 检查 7 - 哪些进程在监听网络连接?
|
||||
|
||||
通常攻击者会安装一个后门程序专门监听网络端口接受指令。该进程等待期间是不会消耗 CPU 和带宽的,因此也就不容易通过 `top` 之类的命令发现。
|
||||
|
||||
`lsof` 和 `netstat` 命令都会列出所有的联网进程。我通常会让它们带上下面这些参数:
|
||||
|
||||
```
|
||||
lsof -i
|
||||
```
|
||||
|
||||
```
|
||||
netstat -plunt
|
||||
```
|
||||
|
||||
你需要留意那些处于 `LISTEN` 和 `ESTABLISHED` 状态的进程,这些进程要么正在等待连接(LISTEN),要么已经连接(ESTABLISHED)。如果遇到不认识的进程,使用 `strace` 和 `lsof` 来看看它们在做什么东西。
|
||||
|
||||
### 被入侵之后该怎么办呢?
|
||||
|
||||
首先,不要紧张,尤其当攻击者正处于登录状态时更不能紧张。**你需要在攻击者警觉到你已经发现他之前夺回机器的控制权。**如果他发现你已经发觉到他了,那么他可能会锁死你不让你登陆服务器,然后开始毁尸灭迹。
|
||||
|
||||
如果你技术不太好那么就直接关机吧。你可以在服务器上运行 `shutdown -h now` 或者 `systemctl poweroff` 这两条命令之一。也可以登录主机提供商的控制面板中关闭服务器。关机后,你就可以开始配置防火墙或者咨询一下供应商的意见。
|
||||
|
||||
如果你对自己颇有自信,而你的主机提供商也有提供上游防火墙,那么你只需要以此创建并启用下面两条规则就行了:
|
||||
|
||||
1. 只允许从你的 IP 地址登录 SSH。
|
||||
2. 封禁除此之外的任何东西,不仅仅是 SSH,还包括任何端口上的任何协议。
|
||||
|
||||
这样会立即关闭攻击者的 SSH 会话,而只留下你可以访问服务器。
|
||||
|
||||
如果你无法访问上游防火墙,那么你就需要在服务器本身创建并启用这些防火墙策略,然后在防火墙规则起效后使用 `kill` 命令关闭攻击者的 SSH 会话。(LCTT 译注:本地防火墙规则 有可能不会阻止已经建立的 SSH 会话,所以保险起见,你需要手工杀死该会话。)
|
||||
|
||||
最后还有一种方法,如果支持的话,就是通过诸如串行控制台之类的带外连接登录服务器,然后通过 `systemctl stop network.service` 停止网络功能。这会关闭所有服务器上的网络连接,这样你就可以慢慢的配置那些防火墙规则了。
|
||||
|
||||
重夺服务器的控制权后,也不要以为就万事大吉了。
|
||||
|
||||
不要试着修复这台服务器,然后接着用。你永远不知道攻击者做过什么,因此你也永远无法保证这台服务器还是安全的。
|
||||
|
||||
最好的方法就是拷贝出所有的数据,然后重装系统。(LCTT 译注:你的程序这时已经不可信了,但是数据一般来说没问题。)
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://bash-prompt.net/guides/server-hacked/
|
||||
|
||||
作者:[Elliot Cooper][a]
|
||||
译者:[lujun9972](https://github.com/lujun9972)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://bash-prompt.net
|
@ -0,0 +1,48 @@
|
||||
使用 DNSTrails 自动找出每个域名的拥有者
|
||||
============================================================
|
||||
|
||||
今天,我们很高兴地宣布我们最近几周做的新功能。它是 Whois 聚合工具,现在可以在 [DNSTrails][1] 上获得。
|
||||
|
||||
在过去,查找一个域名的所有者会花费很多时间,因为大部分时间你都需要把域名翻译为一个 IP 地址,以便找到同一个人拥有的其他域名。
|
||||
|
||||
使用老的方法,在得到你想要的域名列表之前,你在一个工具和另外一个工具的一日又一日的研究和交叉比较结果中经常会花费数个小时。
|
||||
|
||||
感谢这个新工具和我们的智能 [WHOIS 数据库][2],现在你可以搜索任何域名,并获得组织或个人注册的域名的完整列表,并在几秒钟内获得准确的结果。
|
||||
|
||||
### 我如何使用 Whois 聚合功能?
|
||||
|
||||
第一步:打开 [DNSTrails.com][3]
|
||||
|
||||
第二步:搜索任何域名,比如:godaddy.com
|
||||
|
||||
第三步:在得到域名的结果后,如下所见,定位下面的 Whois 信息:
|
||||
|
||||

|
||||
|
||||
第四步:你会看到那里有有关域名的电话和电子邮箱地址。
|
||||
|
||||
第五步:点击右边的链接,你会轻松地找到用相同电话和邮箱注册的域名。
|
||||
|
||||

|
||||
|
||||
如果你正在调查互联网上任何个人的域名所有权,这意味着即使域名甚至没有指向注册服务商的 IP,如果他们使用相同的电话和邮件地址,我们仍然可以发现其他域名。
|
||||
|
||||
想知道一个人拥有的其他域名么?亲自试试 [DNStrails][5] 的 [WHOIS 聚合功能][4]或者[使用我们的 API 访问][6]。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://securitytrails.com/blog/find-every-domain-someone-owns
|
||||
|
||||
作者:[SECURITYTRAILS TEAM][a]
|
||||
译者:[geekpi](https://github.com/geekpi)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://securitytrails.com/blog/find-every-domain-someone-owns
|
||||
[1]:https://dnstrails.com/
|
||||
[2]:https://securitytrails.com/forensics
|
||||
[3]:https://dnstrails.com/
|
||||
[4]:http://dnstrails.com/#/domain/domain/ueland.com
|
||||
[5]:https://dnstrails.com/
|
||||
[6]:https://securitytrails.com/contact
|
@ -0,0 +1,97 @@
|
||||
在命令行中使用 DuckDuckGo 搜索
|
||||
=============
|
||||
|
||||

|
||||
|
||||
此前我们介绍了[如何在命令行中使用 Google 搜索][3]。许多读者反馈说他们平时使用 [Duck Duck Go][4],这是一个功能强大而且保密性很强的搜索引擎。
|
||||
|
||||
正巧,最近出现了一款能够从命令行搜索 DuckDuckGo 的工具。它叫做 ddgr(我把它读作 “dodger”),非常好用。
|
||||
|
||||
像 [Googler][7] 一样,ddgr 是一个完全开源而且非官方的工具。没错,它并不属于 DuckDuckGo。所以,如果你发现它返回的结果有些奇怪,请先询问这个工具的开发者,而不是搜索引擎的开发者。
|
||||
|
||||
### DuckDuckGo 命令行应用
|
||||
|
||||

|
||||
|
||||
[DuckDuckGo Bangs(DuckDuckGo 快捷搜索)][8] 可以帮助你轻易地在 DuckDuckGo 上找到想要的信息(甚至 _本网站 omgubuntu_ 都有快捷搜索)。ddgr 非常忠实地呈现了这个功能。
|
||||
|
||||
和网页版不同的是,你可以更改每页返回多少结果。这比起每次查询都要看三十多条结果要方便一些。默认界面经过了精心设计,在不影响可读性的情况下尽量减少了占用空间。
|
||||
|
||||
`ddgr` 有许多功能和亮点,包括:
|
||||
|
||||
* 更改搜索结果数
|
||||
* 支持 Bash 自动补全
|
||||
* 使用 DuckDuckGo Bangs
|
||||
* 在浏览器中打开链接
|
||||
* ”手气不错“选项
|
||||
* 基于时间、地区、文件类型等的筛选功能
|
||||
* 极少的依赖项
|
||||
|
||||
你可以从 Github 的项目页面上下载支持各种系统的 `ddgr`:
|
||||
|
||||
- [从 Github 下载 “ddgr”][9]
|
||||
|
||||
另外,在 Ubuntu 16.04 LTS 或更新版本中,你可以使用 PPA 安装 ddgr。这个仓库由 ddgr 的开发者维护。如果你想要保持在最新版本的话,推荐使用这种方式安装。
|
||||
|
||||
需要提醒的是,在本文创作时,这个 PPA 中的 ddgr _并不是_ 最新版本,而是一个稍旧的版本(缺少 -num 选项)。
|
||||
|
||||
使用以下命令添加 PPA:
|
||||
|
||||
```
|
||||
sudo add-apt-repository ppa:twodopeshaggy/jarun
|
||||
sudo apt-get update
|
||||
```
|
||||
|
||||
### 如何使用 ddgr 在命令行中搜索 DuckDuckGo
|
||||
|
||||
安装完毕后,你只需打开你的终端模拟器,并运行:
|
||||
|
||||
```
|
||||
ddgr
|
||||
```
|
||||
|
||||
然后输入查询内容:
|
||||
|
||||
```
|
||||
search-term
|
||||
```
|
||||
|
||||
你可以限制搜索结果数:
|
||||
|
||||
```
|
||||
ddgr --num 5 search-term
|
||||
```
|
||||
|
||||
或者自动在浏览器中打开第一条搜索结果:
|
||||
|
||||
|
||||
```
|
||||
ddgr -j search-term
|
||||
```
|
||||
|
||||
你可以使用参数和选项来提高搜索精确度。使用以下命令来查看所有的参数:
|
||||
|
||||
```
|
||||
ddgr -h
|
||||
```
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: http://www.omgubuntu.co.uk/2017/11/duck-duck-go-terminal-app
|
||||
|
||||
作者:[JOEY SNEDDON][a]
|
||||
译者:[yixunx](https://github.com/yixunx)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://plus.google.com/117485690627814051450/?rel=author
|
||||
[1]:https://plus.google.com/117485690627814051450/?rel=author
|
||||
[2]:http://www.omgubuntu.co.uk/category/download
|
||||
[3]:http://www.omgubuntu.co.uk/2017/08/search-google-from-the-command-line
|
||||
[4]:http://duckduckgo.com/
|
||||
[5]:http://www.omgubuntu.co.uk/2017/11/duck-duck-go-terminal-app
|
||||
[6]:https://github.com/jarun/ddgr
|
||||
[7]:https://github.com/jarun/googler
|
||||
[8]:https://duckduckgo.com/bang
|
||||
[9]:https://github.com/jarun/ddgr/releases/tag/v1.1
|
@ -0,0 +1,398 @@
|
||||
Translate Shell :一款在 Linux 命令行中使用谷歌翻译的工具
|
||||
============================================================
|
||||
|
||||
我对 CLI 应用非常感兴趣,因此热衷于使用并分享 CLI 应用。 我之所以更喜欢 CLI 很大原因是因为我在大多数的时候都使用的是字符界面(black screen),已经习惯了使用 CLI 应用而不是 GUI 应用。
|
||||
|
||||
我写过很多关于 CLI 应用的文章。 最近我发现了一些谷歌的 CLI 工具,像 “Google Translator”、“Google Calendar” 和 “Google Contacts”。 这里,我想在给大家分享一下。
|
||||
|
||||
今天我们要介绍的是 “Google Translator” 工具。 由于我的母语是泰米尔语,我在一天内用了很多次才理解了它的意义。
|
||||
|
||||
谷歌翻译为其它语系的人们所广泛使用。
|
||||
|
||||
### 什么是 Translate Shell
|
||||
|
||||
[Translate Shell][2] (之前叫做 Google Translate CLI) 是一款借助谷歌翻译(默认)、必应翻译、Yandex.Translate 以及 Apertium 来翻译的命令行翻译器。它让你可以在终端访问这些翻译引擎。 Translate Shell 在大多数 Linux 发行版中都能使用。
|
||||
|
||||
### 如何安装 Translate Shell
|
||||
|
||||
有三种方法安装 Translate Shell。
|
||||
|
||||
* 下载自包含的可执行文件
|
||||
* 手工安装
|
||||
* 通过包管理器安装
|
||||
|
||||
#### 方法 1 : 下载自包含的可执行文件
|
||||
|
||||
下载自包含的可执行文件放到 `/usr/bin` 目录中。
|
||||
|
||||
```
|
||||
$ wget git.io/trans
|
||||
$ chmod +x ./trans
|
||||
$ sudo mv trans /usr/bin/
|
||||
```
|
||||
|
||||
#### 方法 2 : 手工安装
|
||||
|
||||
克隆 Translate Shell 的 GitHub 仓库然后手工编译。
|
||||
|
||||
```
|
||||
$ git clone https://github.com/soimort/translate-shell && cd translate-shell
|
||||
$ make
|
||||
$ sudo make install
|
||||
```
|
||||
|
||||
#### 方法 3 : 通过包管理器
|
||||
|
||||
有些发行版的官方仓库中包含了 Translate Shell,可以通过包管理器来安装。
|
||||
|
||||
对于 Debian/Ubuntu, 使用 [APT-GET 命令][3] 或者 [APT 命令][4]来安装。
|
||||
|
||||
```
|
||||
$ sudo apt-get install translate-shell
|
||||
```
|
||||
|
||||
对于 Fedora, 使用 [DNF 命令][5] 来安装。
|
||||
|
||||
```
|
||||
$ sudo dnf install translate-shell
|
||||
```
|
||||
|
||||
对于基于 Arch Linux 的系统, 使用 [Yaourt 命令][6] 或 [Packer 明快][7] 来从 AUR 仓库中安装。
|
||||
|
||||
```
|
||||
$ yaourt -S translate-shell
|
||||
or
|
||||
$ packer -S translate-shell
|
||||
```
|
||||
|
||||
### 如何使用 Translate Shell
|
||||
|
||||
安装好后,打开终端闭关输入下面命令。 谷歌翻译会自动探测源文本是哪种语言,并且在默认情况下将之翻译成你的 `locale` 所对应的语言。
|
||||
|
||||
```
|
||||
$ trans [Words]
|
||||
```
|
||||
|
||||
下面我将泰米尔语中的单词 “நன்றி” (Nanri) 翻译成英语。 这个单词的意思是感谢别人。
|
||||
|
||||
```
|
||||
$ trans நன்றி
|
||||
நன்றி
|
||||
(Naṉṟi)
|
||||
|
||||
Thanks
|
||||
|
||||
Definitions of நன்றி
|
||||
[ தமிழ் -> English ]
|
||||
|
||||
noun
|
||||
gratitude
|
||||
நன்றி
|
||||
thanks
|
||||
நன்றி
|
||||
|
||||
நன்றி
|
||||
Thanks
|
||||
```
|
||||
|
||||
使用下面命令也能将英语翻译成泰米尔语。
|
||||
|
||||
```
|
||||
$ trans :ta thanks
|
||||
thanks
|
||||
/THaNGks/
|
||||
|
||||
நன்றி
|
||||
(Naṉṟi)
|
||||
|
||||
Definitions of thanks
|
||||
[ English -> தமிழ் ]
|
||||
|
||||
noun
|
||||
நன்றி
|
||||
gratitude, thanks
|
||||
|
||||
thanks
|
||||
நன்றி
|
||||
```
|
||||
|
||||
要将一个单词翻译到多个语种可以使用下面命令(本例中,我将单词翻译成泰米尔语以及印地语)。
|
||||
|
||||
```
|
||||
$ trans :ta+hi thanks
|
||||
thanks
|
||||
/THaNGks/
|
||||
|
||||
நன்றி
|
||||
(Naṉṟi)
|
||||
|
||||
Definitions of thanks
|
||||
[ English -> தமிழ் ]
|
||||
|
||||
noun
|
||||
நன்றி
|
||||
gratitude, thanks
|
||||
|
||||
thanks
|
||||
நன்றி
|
||||
|
||||
thanks
|
||||
/THaNGks/
|
||||
|
||||
धन्यवाद
|
||||
(dhanyavaad)
|
||||
|
||||
Definitions of thanks
|
||||
[ English -> हिन्दी ]
|
||||
|
||||
noun
|
||||
धन्यवाद
|
||||
thanks, thank, gratitude, thankfulness, felicitation
|
||||
|
||||
thanks
|
||||
धन्यवाद, शुक्रिया
|
||||
```
|
||||
|
||||
使用下面命令可以将多个单词当成一个参数(句子)来进行翻译。(只需要把句子应用起来作为一个参数就行了)。
|
||||
|
||||
```
|
||||
$ trans :ta "what is going on your life?"
|
||||
what is going on your life?
|
||||
|
||||
உங்கள் வாழ்க்கையில் என்ன நடக்கிறது?
|
||||
(Uṅkaḷ vāḻkkaiyil eṉṉa naṭakkiṟatu?)
|
||||
|
||||
Translations of what is going on your life?
|
||||
[ English -> தமிழ் ]
|
||||
|
||||
what is going on your life?
|
||||
உங்கள் வாழ்க்கையில் என்ன நடக்கிறது?
|
||||
```
|
||||
|
||||
下面命令单独地翻译各个单词。
|
||||
|
||||
```
|
||||
$ trans :ta curios happy
|
||||
curios
|
||||
|
||||
ஆர்வம்
|
||||
(Ārvam)
|
||||
|
||||
Translations of curios
|
||||
[ Română -> தமிழ் ]
|
||||
|
||||
curios
|
||||
ஆர்வம், அறிவாளிகள், ஆர்வமுள்ள, அறிய, ஆர்வமாக
|
||||
happy
|
||||
/ˈhapē/
|
||||
|
||||
சந்தோஷமாக
|
||||
(Cantōṣamāka)
|
||||
|
||||
Definitions of happy
|
||||
[ English -> தமிழ் ]
|
||||
|
||||
மகிழ்ச்சியான
|
||||
happy, convivial, debonair, gay
|
||||
திருப்தி உடைய
|
||||
happy
|
||||
|
||||
adjective
|
||||
இன்பமான
|
||||
happy
|
||||
|
||||
happy
|
||||
சந்தோஷமாக, மகிழ்ச்சி, இனிய, சந்தோஷமா
|
||||
```
|
||||
|
||||
简洁模式:默认情况下,Translate Shell 尽可能多的显示翻译信息。如果你希望只显示简要信息,只需要加上 `-b`选项。
|
||||
|
||||
```
|
||||
$ trans -b :ta thanks
|
||||
நன்றி
|
||||
```
|
||||
|
||||
字典模式:加上 `-d` 可以把 Translate Shell 当成字典来用。
|
||||
|
||||
```
|
||||
$ trans -d :en thanks
|
||||
thanks
|
||||
/THaNGks/
|
||||
|
||||
Synonyms
|
||||
noun
|
||||
- gratitude, appreciation, acknowledgment, recognition, credit
|
||||
|
||||
exclamation
|
||||
- thank you, many thanks, thanks very much, thanks a lot, thank you kindly, much obliged, much appreciated, bless you, thanks a million
|
||||
|
||||
Examples
|
||||
- In short, thanks for everything that makes this city great this Thanksgiving.
|
||||
|
||||
- many thanks
|
||||
|
||||
- There were no thanks in the letter from him, just complaints and accusations.
|
||||
|
||||
- It is a joyful celebration in which Bolivians give thanks for their freedom as a nation.
|
||||
|
||||
- festivals were held to give thanks for the harvest
|
||||
|
||||
- The collection, as usual, received a great response and thanks is extended to all who subscribed.
|
||||
|
||||
- It would be easy to dwell on the animals that Tasmania has lost, but I prefer to give thanks for what remains.
|
||||
|
||||
- thanks for being so helpful
|
||||
|
||||
- It came back on about half an hour earlier than predicted, so I suppose I can give thanks for that.
|
||||
|
||||
- Many thanks for the reply but as much as I tried to follow your advice, it's been a bad week.
|
||||
|
||||
- To them and to those who have supported the office I extend my grateful thanks .
|
||||
|
||||
- We can give thanks and words of appreciation to others for their kind deeds done to us.
|
||||
|
||||
- Adam, thanks for taking time out of your very busy schedule to be with us tonight.
|
||||
|
||||
- a letter of thanks
|
||||
|
||||
- Thank you very much for wanting to go on reading, and thanks for your understanding.
|
||||
|
||||
- Gerry has received a letter of thanks from the charity for his part in helping to raise this much needed cash.
|
||||
|
||||
- So thanks for your reply to that guy who seemed to have a chip on his shoulder about it.
|
||||
|
||||
- Suzanne, thanks for being so supportive with your comments on my blog.
|
||||
|
||||
- She has never once acknowledged my thanks , or existence for that matter.
|
||||
|
||||
- My grateful thanks go to the funders who made it possible for me to travel.
|
||||
|
||||
- festivals were held to give thanks for the harvest
|
||||
|
||||
- All you secretaries who made it this far into the article… thanks for your patience.
|
||||
|
||||
- So, even though I don't think the photos are that good, thanks for the compliments!
|
||||
|
||||
- And thanks for warning us that your secret service requires a motorcade of more than 35 cars.
|
||||
|
||||
- Many thanks for your advice, which as you can see, I have passed on to our readers.
|
||||
|
||||
- Tom Ryan was given a bottle of wine as a thanks for his active involvement in the twinning project.
|
||||
|
||||
- Mr Hill insists he has received no recent complaints and has even been sent a letter of thanks from the forum.
|
||||
|
||||
- Hundreds turned out to pay tribute to a beloved former headteacher at a memorial service to give thanks for her life.
|
||||
|
||||
- Again, thanks for a well written and much deserved tribute to our good friend George.
|
||||
|
||||
- I appreciate your doing so, and thanks also for the compliments about the photos!
|
||||
|
||||
See also
|
||||
Thanks!, thank, many thanks, thanks to, thanks to you, special thanks, give thanks, thousand thanks, Many thanks!, render thanks, heartfelt thanks, thanks to this
|
||||
```
|
||||
|
||||
使用下面格式可以使用 Translate Shell 来翻译文件。
|
||||
|
||||
```
|
||||
$ trans :ta file:///home/magi/gtrans.txt
|
||||
உங்கள் வாழ்க்கையில் என்ன நடக்கிறது?
|
||||
```
|
||||
|
||||
下面命令可以让 Translate Shell 进入交互模式。 在进入交互模式之前你需要明确指定源语言和目标语言。本例中,我将英文单词翻译成泰米尔语。
|
||||
|
||||
```
|
||||
$ trans -shell en:ta thanks
|
||||
Translate Shell
|
||||
(:q to quit)
|
||||
thanks
|
||||
/THaNGks/
|
||||
|
||||
நன்றி
|
||||
(Naṉṟi)
|
||||
|
||||
Definitions of thanks
|
||||
[ English -> தமிழ் ]
|
||||
|
||||
noun
|
||||
நன்றி
|
||||
gratitude, thanks
|
||||
|
||||
thanks
|
||||
நன்றி
|
||||
```
|
||||
|
||||
想知道语言代码,可以执行下面命令。
|
||||
|
||||
```
|
||||
$ trans -R
|
||||
```
|
||||
或者
|
||||
|
||||
```
|
||||
$ trans -T
|
||||
┌───────────────────────┬───────────────────────┬───────────────────────┐
|
||||
│ Afrikaans - af │ Hindi - hi │ Punjabi - pa │
|
||||
│ Albanian - sq │ Hmong - hmn │ Querétaro Otomi- otq │
|
||||
│ Amharic - am │ Hmong Daw - mww │ Romanian - ro │
|
||||
│ Arabic - ar │ Hungarian - hu │ Russian - ru │
|
||||
│ Armenian - hy │ Icelandic - is │ Samoan - sm │
|
||||
│ Azerbaijani - az │ Igbo - ig │ Scots Gaelic - gd │
|
||||
│ Basque - eu │ Indonesian - id │ Serbian (Cyr...-sr-Cyrl
|
||||
│ Belarusian - be │ Irish - ga │ Serbian (Latin)-sr-Latn
|
||||
│ Bengali - bn │ Italian - it │ Sesotho - st │
|
||||
│ Bosnian - bs │ Japanese - ja │ Shona - sn │
|
||||
│ Bulgarian - bg │ Javanese - jv │ Sindhi - sd │
|
||||
│ Cantonese - yue │ Kannada - kn │ Sinhala - si │
|
||||
│ Catalan - ca │ Kazakh - kk │ Slovak - sk │
|
||||
│ Cebuano - ceb │ Khmer - km │ Slovenian - sl │
|
||||
│ Chichewa - ny │ Klingon - tlh │ Somali - so │
|
||||
│ Chinese Simp...- zh-CN│ Klingon (pIqaD)tlh-Qaak Spanish - es │
|
||||
│ Chinese Trad...- zh-TW│ Korean - ko │ Sundanese - su │
|
||||
│ Corsican - co │ Kurdish - ku │ Swahili - sw │
|
||||
│ Croatian - hr │ Kyrgyz - ky │ Swedish - sv │
|
||||
│ Czech - cs │ Lao - lo │ Tahitian - ty │
|
||||
│ Danish - da │ Latin - la │ Tajik - tg │
|
||||
│ Dutch - nl │ Latvian - lv │ Tamil - ta │
|
||||
│ English - en │ Lithuanian - lt │ Tatar - tt │
|
||||
│ Esperanto - eo │ Luxembourgish - lb │ Telugu - te │
|
||||
│ Estonian - et │ Macedonian - mk │ Thai - th │
|
||||
│ Fijian - fj │ Malagasy - mg │ Tongan - to │
|
||||
│ Filipino - tl │ Malay - ms │ Turkish - tr │
|
||||
│ Finnish - fi │ Malayalam - ml │ Udmurt - udm │
|
||||
│ French - fr │ Maltese - mt │ Ukrainian - uk │
|
||||
│ Frisian - fy │ Maori - mi │ Urdu - ur │
|
||||
│ Galician - gl │ Marathi - mr │ Uzbek - uz │
|
||||
│ Georgian - ka │ Mongolian - mn │ Vietnamese - vi │
|
||||
│ German - de │ Myanmar - my │ Welsh - cy │
|
||||
│ Greek - el │ Nepali - ne │ Xhosa - xh │
|
||||
│ Gujarati - gu │ Norwegian - no │ Yiddish - yi │
|
||||
│ Haitian Creole - ht │ Pashto - ps │ Yoruba - yo │
|
||||
│ Hausa - ha │ Persian - fa │ Yucatec Maya - yua │
|
||||
│ Hawaiian - haw │ Polish - pl │ Zulu - zu │
|
||||
│ Hebrew - he │ Portuguese - pt │ │
|
||||
└───────────────────────┴───────────────────────┴───────────────────────┘
|
||||
```
|
||||
|
||||
想了解更多选项的内容,可以查看其 man 手册。
|
||||
|
||||
```
|
||||
$ man trans
|
||||
```
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.2daygeek.com/translate-shell-a-tool-to-use-google-translate-from-command-line-in-linux/
|
||||
|
||||
作者:[Magesh Maruthamuthu][a]
|
||||
译者:[lujun9972](https://github.com/lujun9972 )
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.2daygeek.com/author/magesh/
|
||||
[2]:https://github.com/soimort/translate-shell
|
||||
[3]:https://www.2daygeek.com/apt-get-apt-cache-command-examples-manage-packages-debian-ubuntu-systems/
|
||||
[4]:https://www.2daygeek.com/apt-command-examples-manage-packages-debian-ubuntu-systems/
|
||||
[5]:https://www.2daygeek.com/dnf-command-examples-manage-packages-fedora-system/
|
||||
[6]:https://www.2daygeek.com/install-yaourt-aur-helper-on-arch-linux/
|
||||
[7]:https://www.2daygeek.com/install-packer-aur-helper-on-arch-linux/
|
132
published/20171130 Wake up and Shut Down Linux Automatically.md
Normal file
132
published/20171130 Wake up and Shut Down Linux Automatically.md
Normal file
@ -0,0 +1,132 @@
|
||||
如何自动唤醒和关闭 Linux
|
||||
=====================
|
||||
|
||||

|
||||
|
||||
> 了解如何通过配置 Linux 计算机来根据时间自动唤醒和关闭。
|
||||
|
||||
|
||||
不要成为一个电能浪费者。如果你的电脑不需要开机就请把它们关机。出于方便和计算机宅的考虑,你可以通过配置你的 Linux 计算机实现自动唤醒和关闭。
|
||||
|
||||
### 宝贵的系统运行时间
|
||||
|
||||
有时候有些电脑需要一直处在开机状态,在不超过电脑运行时间的限制下这种情况是被允许的。有些人为他们的计算机可以长时间的正常运行而感到自豪,且现在我们有内核热补丁能够实现只有在硬件发生故障时才需要机器关机。我认为比较实际可行的是,像减少移动部件磨损一样节省电能,且在不需要机器运行的情况下将其关机。比如,你可以在规定的时间内唤醒备份服务器,执行备份,然后关闭它直到它要进行下一次备份。或者,你可以设置你的互联网网关只在特定的时间运行。任何不需要一直运行的东西都可以将其配置成在其需要工作的时候打开,待其完成工作后将其关闭。
|
||||
|
||||
### 系统休眠
|
||||
|
||||
对于不需要一直运行的电脑,使用 root 的 cron 定时任务(即 `/etc/crontab`)可以可靠地关闭电脑。这个例子创建一个 root 定时任务实现每天晚上 11 点 15 分定时关机。
|
||||
|
||||
```
|
||||
# crontab -e -u root
|
||||
# m h dom mon dow command
|
||||
15 23 * * * /sbin/shutdown -h now
|
||||
```
|
||||
|
||||
以下示例仅在周一至周五运行:
|
||||
|
||||
```
|
||||
15 23 * * 1-5 /sbin/shutdown -h now
|
||||
```
|
||||
|
||||
您可以为不同的日期和时间创建多个 cron 作业。 通过命令 `man 5 crontab` 可以了解所有时间和日期的字段。
|
||||
|
||||
一个快速、容易的方式是,使用 `/etc/crontab` 文件。但这样你必须指定用户:
|
||||
|
||||
```
|
||||
15 23 * * 1-5 root shutdown -h now
|
||||
```
|
||||
|
||||
### 自动唤醒
|
||||
|
||||
实现自动唤醒是一件很酷的事情;我大多数 SUSE (SUSE Linux)的同事都在纽伦堡,因此,因此为了跟同事能有几小时一起工作的时间,我不得不需要在凌晨五点起床。我的计算机早上 5 点半自动开始工作,而我只需要将自己和咖啡拖到我的桌子上就可以开始工作了。按下电源按钮看起来好像并不是什么大事,但是在每天的那个时候每件小事都会变得很大。
|
||||
|
||||
唤醒 Linux 计算机可能不如关闭它可靠,因此你可能需要尝试不同的办法。你可以使用远程唤醒(Wake-On-LAN)、RTC 唤醒或者个人电脑的 BIOS 设置预定的唤醒这些方式。这些方式可行的原因是,当你关闭电脑时,这并不是真正关闭了计算机;此时计算机处在极低功耗状态且还可以接受和响应信号。只有在你拔掉电源开关时其才彻底关闭。
|
||||
|
||||
### BIOS 唤醒
|
||||
|
||||
BIOS 唤醒是最可靠的。我的系统主板 BIOS 有一个易于使用的唤醒调度程序 (图 1)。对你来说也是一样的容易。
|
||||
|
||||

|
||||
|
||||
*图 1:我的系统 BIOS 有个易用的唤醒定时器。*
|
||||
|
||||
### 主机远程唤醒(Wake-On-LAN)
|
||||
|
||||
远程唤醒是仅次于 BIOS 唤醒的又一种可靠的唤醒方法。这需要你从第二台计算机发送信号到所要打开的计算机。可以使用 Arduino 或<ruby>树莓派<rt>Raspberry Pi</rt></ruby>发送给基于 Linux 的路由器或者任何 Linux 计算机的唤醒信号。首先,查看系统主板 BIOS 是否支持 Wake-On-LAN ,要是支持的话,必须先启动它,因为它被默认为禁用。
|
||||
|
||||
然后,需要一个支持 Wake-On-LAN 的网卡;无线网卡并不支持。你需要运行 `ethtool` 命令查看网卡是否支持 Wake-On-LAN :
|
||||
|
||||
```
|
||||
# ethtool eth0 | grep -i wake-on
|
||||
Supports Wake-on: pumbg
|
||||
Wake-on: g
|
||||
```
|
||||
|
||||
这条命令输出的 “Supports Wake-on” 字段会告诉你你的网卡现在开启了哪些功能:
|
||||
|
||||
* d -- 禁用
|
||||
* p -- 物理活动唤醒
|
||||
* u -- 单播消息唤醒
|
||||
* m -- 多播(组播)消息唤醒
|
||||
* b -- 广播消息唤醒
|
||||
* a -- ARP 唤醒
|
||||
* g -- <ruby>特定数据包<rt>magic packet</rt></ruby>唤醒
|
||||
* s -- 设有密码的<ruby>特定数据包<rt>magic packet</rt></ruby>唤醒
|
||||
|
||||
`ethtool` 命令的 man 手册并没说清楚 `p` 选项的作用;这表明任何信号都会导致唤醒。然而,在我的测试中它并没有这么做。想要实现远程唤醒主机,必须支持的功能是 `g` —— <ruby>特定数据包<rt>magic packet</rt></ruby>唤醒,而且下面的“Wake-on” 行显示这个功能已经在启用了。如果它没有被启用,你可以通过 `ethtool` 命令来启用它。
|
||||
|
||||
```
|
||||
# ethtool -s eth0 wol g
|
||||
```
|
||||
|
||||
这条命令可能会在重启后失效,所以为了确保万无一失,你可以创建个 root 用户的定时任务(cron)在每次重启的时候来执行这条命令。
|
||||
|
||||
```
|
||||
@reboot /usr/bin/ethtool -s eth0 wol g
|
||||
```
|
||||
|
||||
另一个选择是最近的<ruby>网络管理器<rt>Network Manager</rt></ruby>版本有一个很好的小复选框来启用 Wake-On-LAN(图 2)。
|
||||
|
||||

|
||||
|
||||
*图 2:启用 Wake on LAN*
|
||||
|
||||
这里有一个可以用于设置密码的地方,但是如果你的网络接口不支持<ruby>安全开机<rt>Secure On</rt></ruby>密码,它就不起作用。
|
||||
|
||||
现在你需要配置第二台计算机来发送唤醒信号。你并不需要 root 权限,所以你可以为你的普通用户创建 cron 任务。你需要用到的是想要唤醒的机器的网络接口和MAC地址信息。
|
||||
|
||||
```
|
||||
30 08 * * * /usr/bin/wakeonlan D0:50:99:82:E7:2B
|
||||
```
|
||||
|
||||
### RTC 唤醒
|
||||
|
||||
通过使用实时闹钟来唤醒计算机是最不可靠的方法。对于这个方法,可以参看 [Wake Up Linux With an RTC Alarm Clock][4] ;对于现在的大多数发行版来说这种方法已经有点过时了。
|
||||
|
||||
下周继续了解更多关于使用 RTC 唤醒的方法。
|
||||
|
||||
通过 Linux 基金会和 edX 可以学习更多关于 Linux 的免费 [Linux 入门][5]教程。
|
||||
|
||||
(题图:[The Observatory at Delhi][7])
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via:https://www.linux.com/learn/intro-to-linux/2017/11/wake-and-shut-down-linux-automatically
|
||||
|
||||
作者:[Carla Schroder][a]
|
||||
译者:[HardworkFish](https://github.com/HardworkFish)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linux.com/users/cschroder
|
||||
[1]:https://www.linux.com/files/images/bannerjpg
|
||||
[2]:https://www.linux.com/files/images/fig-1png-11
|
||||
[3]:https://www.linux.com/files/images/fig-2png-7
|
||||
[4]:https://www.linux.com/learn/wake-linux-rtc-alarm-clock
|
||||
[5]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux
|
||||
[6]:https://www.linux.com/licenses/category/creative-commons-attribution
|
||||
[7]:http://www.columbia.edu/itc/mealac/pritchett/00routesdata/1700_1799/jaipur/delhijantarearly/delhijantarearly.html
|
||||
[8]:https://www.linux.com/licenses/category/used-permission
|
||||
[9]:https://www.linux.com/licenses/category/used-permission
|
||||
|
163
published/20171201 How to Manage Users with Groups in Linux.md
Normal file
163
published/20171201 How to Manage Users with Groups in Linux.md
Normal file
@ -0,0 +1,163 @@
|
||||
如何在 Linux 系统中通过用户组来管理用户
|
||||
============================================================
|
||||
|
||||

|
||||
|
||||
> 本教程可以了解如何通过用户组和访问控制表(ACL)来管理用户。
|
||||
|
||||
当你需要管理一台容纳多个用户的 Linux 机器时,比起一些基本的用户管理工具所提供的方法,有时候你需要对这些用户采取更多的用户权限管理方式。特别是当你要管理某些用户的权限时,这个想法尤为重要。比如说,你有一个目录,某个用户组中的用户可以通过读和写的权限访问这个目录,而其他用户组中的用户对这个目录只有读的权限。在 Linux 中,这是完全可以实现的。但前提是你必须先了解如何通过用户组和访问控制表(ACL)来管理用户。
|
||||
|
||||
我们将从简单的用户开始,逐渐深入到复杂的访问控制表(ACL)。你可以在你所选择的 Linux 发行版完成你所需要做的一切。本文的重点是用户组,所以不会涉及到关于用户的基础知识。
|
||||
|
||||
为了达到演示的目的,我将假设:
|
||||
|
||||
你需要用下面两个用户名新建两个用户:
|
||||
|
||||
* olivia
|
||||
* nathan
|
||||
|
||||
你需要新建以下两个用户组:
|
||||
|
||||
* readers
|
||||
* editors
|
||||
|
||||
olivia 属于 editors 用户组,而 nathan 属于 readers 用户组。reader 用户组对 `/DATA` 目录只有读的权限,而 editors 用户组则对 `/DATA` 目录同时有读和写的权限。当然,这是个非常小的任务,但它会给你基本的信息,你可以扩展这个任务以适应你其他更大的需求。
|
||||
|
||||
我将在 Ubuntu 16.04 Server 平台上进行演示。这些命令都是通用的,唯一不同的是,要是在你的发行版中不使用 `sudo` 命令,你必须切换到 root 用户来执行这些命令。
|
||||
|
||||
### 创建用户
|
||||
|
||||
我们需要做的第一件事是为我们的实验创建两个用户。可以用 `useradd` 命令来创建用户,我们不只是简单地创建一个用户,而需要同时创建用户和属于他们的家目录,然后给他们设置密码。
|
||||
|
||||
```
|
||||
sudo useradd -m olivia
|
||||
sudo useradd -m nathan
|
||||
```
|
||||
|
||||
我们现在创建了两个用户,如果你看看 `/home` 目录,你可以发现他们的家目录(因为我们用了 `-m` 选项,可以在创建用户的同时创建他们的家目录。
|
||||
|
||||
之后,我们可以用以下命令给他们设置密码:
|
||||
|
||||
```
|
||||
sudo passwd olivia
|
||||
sudo passwd nathan
|
||||
```
|
||||
|
||||
就这样,我们创建了两个用户。
|
||||
|
||||
### 创建用户组并添加用户
|
||||
|
||||
现在我们将创建 readers 和 editors 用户组,然后给它们添加用户。创建用户组的命令是:
|
||||
|
||||
```
|
||||
addgroup readers
|
||||
addgroup editors
|
||||
```
|
||||
|
||||
(LCTT 译注:当你使用 CentOS 等一些 Linux 发行版时,可能系统没有 `addgroup` 这个命令,推荐使用 `groupadd` 命令来替换 `addgroup` 命令以达到同样的效果)
|
||||
|
||||

|
||||
|
||||
*图一:我们可以使用刚创建的新用户组了。*
|
||||
|
||||
创建用户组后,我们需要添加我们的用户到这两个用户组。我们用以下命令来将 nathan 用户添加到 readers 用户组:
|
||||
|
||||
```
|
||||
sudo usermod -a -G readers nathan
|
||||
```
|
||||
|
||||
用以下命令将 olivia 添加到 editors 用户组:
|
||||
|
||||
```
|
||||
sudo usermod -a -G editors olivia
|
||||
```
|
||||
|
||||
现在我们可以通过用户组来管理用户了。
|
||||
|
||||
### 给用户组授予目录的权限
|
||||
|
||||
假设你有个目录 `/READERS` 且允许 readers 用户组的所有成员访问这个目录。首先,我们执行以下命令来更改目录所属用户组:
|
||||
|
||||
```
|
||||
sudo chown -R :readers /READERS
|
||||
```
|
||||
|
||||
接下来,执行以下命令收回目录所属用户组的写入权限:
|
||||
|
||||
```
|
||||
sudo chmod -R g-w /READERS
|
||||
```
|
||||
|
||||
然后我们执行下面的命令来收回其他用户对这个目录的访问权限(以防止任何不在 readers 组中的用户访问这个目录里的文件):
|
||||
|
||||
```
|
||||
sudo chmod -R o-x /READERS
|
||||
```
|
||||
|
||||
这时候,只有目录的所有者(root)和用户组 reader 中的用户可以访问 `/READES` 中的文件。
|
||||
|
||||
假设你有个目录 `/EDITORS` ,你需要给用户组 editors 里的成员这个目录的读和写的权限。为了达到这个目的,执行下面的这些命令是必要的:
|
||||
|
||||
```
|
||||
sudo chown -R :editors /EDITORS
|
||||
sudo chmod -R g+w /EDITORS
|
||||
sudo chmod -R o-x /EDITORS
|
||||
```
|
||||
|
||||
此时 editors 用户组的所有成员都可以访问和修改其中的文件。除此之外其他用户(除了 root 之外)无法访问 `/EDITORS` 中的任何文件。
|
||||
|
||||
使用这个方法的问题在于,你一次只能操作一个组和一个目录而已。这时候访问控制表(ACL)就可以派得上用场了。
|
||||
|
||||
### 使用访问控制表(ACL)
|
||||
|
||||
现在,让我们把这个问题变得棘手一点。假设你有一个目录 `/DATA` 并且你想给 readers 用户组的成员读取权限,并同时给 editors 用户组的成员读和写的权限。为此,你必须要用到 `setfacl` 命令。`setfacl` 命令可以为文件或文件夹设置一个访问控制表(ACL)。
|
||||
|
||||
这个命令的结构如下:
|
||||
|
||||
```
|
||||
setfacl OPTION X:NAME:Y /DIRECTORY
|
||||
```
|
||||
|
||||
其中 OPTION 是可选选项,X 可以是 `u`(用户)或者是 `g` (用户组),NAME 是用户或者用户组的名字,/DIRECTORY 是要用到的目录。我们将使用 `-m` 选项进行修改。因此,我们给 readers 用户组添加读取权限的命令是:
|
||||
|
||||
```
|
||||
sudo setfacl -m g:readers:rx -R /DATA
|
||||
```
|
||||
|
||||
现在 readers 用户组里面的每一个用户都可以读取 `/DATA` 目录里的文件了,但是他们不能修改里面的内容。
|
||||
|
||||
为了给 editors 用户组里面的用户读写权限,我们执行了以下命令:
|
||||
|
||||
```
|
||||
sudo setfacl -m g:editors:rwx -R /DATA
|
||||
```
|
||||
|
||||
上述命令将赋予 editors 用户组中的任何成员读取权限,同时保留 readers 用户组的只读权限。
|
||||
|
||||
### 更多的权限控制
|
||||
|
||||
使用访问控制表(ACL),你可以实现你所需的权限控制。你可以添加用户到用户组,并且灵活地控制这些用户组对每个目录的权限以达到你的需求。如果想了解上述工具的更多信息,可以执行下列的命令:
|
||||
|
||||
* `man usradd`
|
||||
* `man addgroup`
|
||||
* `man usermod`
|
||||
* `man sefacl`
|
||||
* `man chown`
|
||||
* `man chmod`
|
||||
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linux.com/learn/intro-to-linux/2017/12/how-manage-users-groups-linux
|
||||
|
||||
作者:[Jack Wallen]
|
||||
译者:[imquanquan](https://github.com/imquanquan)
|
||||
校对:[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[1]:https://www.linux.com/files/images/group-people-16453561920jpg
|
||||
[2]:https://www.linux.com/files/images/groups1jpg
|
||||
[3]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux
|
||||
[4]:https://www.linux.com/licenses/category/creative-commons-zero
|
||||
[5]:https://www.linux.com/licenses/category/used-permission
|
34
published/20171201 Linux Journal Ceases Publication.md
Normal file
34
published/20171201 Linux Journal Ceases Publication.md
Normal file
@ -0,0 +1,34 @@
|
||||
Linux Journal 停止发行
|
||||
============================================================
|
||||
|
||||
EOF
|
||||
|
||||
伙计们,看起来我们要到终点了。如果按照计划而且没有什么其他的话,十一月份的 Linux Journal 将是我们的最后一期。
|
||||
|
||||
简单的事实是,我们已经用完了钱和期权。我们从来没有一个富有的母公司或者自己深厚的资金,从开始到结束,这使得我们变成一个反常的出版商。虽然我们在很长的一段时间内运营着,但当天平不可恢复地最终向相反方向倾斜时,我们在十一月份失去了最后一点支持。
|
||||
|
||||
虽然我们像看到出版业的过去那样看到出版业的未来 - 广告商赞助出版物的时代,因为他们重视品牌和读者 - 我们如今的广告宁愿追逐眼球,最好是在读者的浏览器中植入跟踪标记,并随时随地展示那些广告。但是,未来不是这样,过去的已经过去了。
|
||||
|
||||
我们猜想,有一个希望,那就是救世主可能会会来。但除了我们的品牌、我们的档案,我们的域名、我们的用户和读者之外,还必须是愿意承担我们一部分债务的人。如果你认识任何人能够提供认真的报价,请告诉我们。不然,请观看 LinuxJournal.com,并希望至少我们的遗留归档(可以追溯到 Linux Journal 诞生的 1994 年 4 月,当 Linux 命中 1.0 发布时)将不会消失。这里有很多很棒的东西,还有很多我们会痛恨世界失去的历史。
|
||||
|
||||
我们最大的遗憾是,我们甚至没有足够的钱回馈最看重我们的人:我们的用户。为此,我们不能更深刻或真诚地道歉。我们对订阅者而言有什么:
|
||||
|
||||
Linux Pro Magazine 为我们的用户提供了六本免费的杂志,我们在 Linux Journal 上一直赞叹这点。在我们需要的时候,他们是我们的第一批人,我们感谢他们的恩惠。我们今天刚刚完成了我们的 2017 年归档,其中包括我们曾经发表过的每一个问题,包括第一个和最后一个。通常我们以 25 美元的价格出售,但显然用户将免费获得。订阅者请注意有关两者的详细信息的电子邮件。
|
||||
|
||||
我们也希望在知道我们非常非常努力地让 Linux Journal 进行下去后能有一些安慰 ,而且我们已经用最精益、小的可能运营了很长一段时间。我们是一个大多数是自愿者的组织,有些员工已经几个月没有收到工资。我们还欠钱给自由职业者。这时一个限制发行商能够维持多长时间的限制,现在这个限制已经到头了。
|
||||
|
||||
伙计们,这是一个伟大的运营。乡亲。对每一个为我们的诞生、我们的成功和我们多年的坚持作出贡献的人致敬。我们列了一份名单,但是列表太长了,并且漏掉有价值的人的风险很高。你知道你是谁。我们再次感谢。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linuxjournal.com/content/linux-journal-ceases-publication
|
||||
|
||||
作者:[ Carlie Fairchild][a]
|
||||
译者:[geekpi](https://github.com/geekpi)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linuxjournal.com/users/carlie-fairchild
|
||||
[1]:https://www.linuxjournal.com/taxonomy/term/29
|
||||
[2]:https://www.linuxjournal.com/users/carlie-fairchild
|
@ -1,290 +0,0 @@
|
||||
GOOGLE CHROME–ONE YEAR IN
|
||||
========================================
|
||||
|
||||
|
||||
Four weeks ago, emailed notice of a free massage credit revealed that I’ve been at Google for a year. Time flies when you’re [drinking from a firehose][3].
|
||||
|
||||
When I mentioned my anniversary, friends and colleagues from other companies asked what I’ve learned while working on Chrome over the last year. This rambling post is an attempt to answer that question.
|
||||
|
||||
### NON-MASKABLE INTERRUPTS
|
||||
|
||||
While I _started_ at Google just over a year ago, I haven’t actually _worked_ there for a full year yet. My second son (Nate) was born a few weeks early, arriving ten workdays after my first day of work.
|
||||
|
||||
I took full advantage of Google’s very generous twelve weeks of paternity leave, taking a few weeks after we brought Nate home, and the balance as spring turned to summer. In a year, we went from having an enormous infant to an enormous toddler who’s taking his first steps and trying to emulate everything his 3 year-old brother (Noah) does.
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
I mention this because it’s had a huge impact on my work over the last year—_much_ more than I’d naively expected.
|
||||
|
||||
When Noah was born, I’d been at Telerik for [almost a year][4], and I’d been hacking on Fiddler alone for nearly a decade. I took a short paternity leave, and my coding hours shifted somewhat (I started writing code late at night between bottle feeds), but otherwise my work wasn’t significantly impacted.
|
||||
|
||||
As I pondered joining Google Chrome’s security team, I expected pretty much the same—a bit less sleep, a bit of scheduling awkwardness, but I figured things would fall into a good routine in a few months.
|
||||
|
||||
Things turned out somewhat differently.
|
||||
|
||||
Perhaps sensing that my life had become too easy, fate decided that 2016 was the year I’d get sick. _Constantly_. (Our theory is that Noah was bringing home germs from pre-school; he got sick a bunch too, but recovered quickly each time.) I was sick more days in 2016 than I was in the prior decade, including a month-long illness in the spring. _That_ ended with a bout of pneumonia that concluded with a doctor-mandated seven days away from the office. As I coughed my brains out on the sofa at home, I derived some consolation in thinking about Google’s generous life insurance package. But for the most part, my illnesses were minor—enough to keep me awake at night and coughing all day, but otherwise able to work.
|
||||
|
||||
Mathematically, you might expect two kids to be twice as much work as one, but in our experience, it hasn’t worked out that way. Instead, it varies between 80% (when the kids happily play together) to 400% (when they’re colliding like atoms in a runaway nuclear reactor). Thanks to my wife’s heroic efforts, we found a workable _daytime _routine. The nights, however, have been unexpectedly difficult. Big brother Noah is at an age where he usually sleeps through the night, but he’s sure to wake me up every morning at 6:30am sharp. Fortunately, Nate has been a pretty good sleeper, but even now, at just over a year old, he usually still wakes up and requires attention twice a night or so.
|
||||
|
||||
I can’t_ remember _the last time I had eight hours of sleep in a row. And that’s been _extremely _challenging… because I can’t remember _much else_ either. Learning new things when you don’t remember them the next day is a brutal, frustrating process.
|
||||
|
||||
When Noah was a baby, I could simply sleep in after a long night. Even if I didn’t get enough sleep, it wouldn’t really matter—I’d been coding in C# on Fiddler for a decade, and deadlines were few and far between. If all else failed, I’d just avoid working on any especially gnarly code and spend the day handling support requests, updating graphics, or doing other simple and straightforward grunt work from my backlog.
|
||||
|
||||
Things are much different on Chrome.
|
||||
|
||||
### ROLES
|
||||
|
||||
When I first started talking to the Chrome Security team about coming aboard, it was for a role on the Developer Advocacy team. I’d be driving HTTPS adoption across the web and working with big sites to unblock their migrations in any way I could. I’d already been doing the first half of that for fun (delivering [talks][5] at conferences like Codemash and [Velocity][6]), and I’d previously spent eight years as a Security Program Manager for the Internet Explorer team. I had _tons _of relevant experience. Easy peasy.
|
||||
|
||||
I interviewed for the Developer Advocate role. The hiring committee kicked back my packet and said I should interview as a Technical Program Manager instead.
|
||||
|
||||
I interviewed as a Technical Program Manager. The hiring committee kicked back my packet and said I should interview as a Developer Advocate instead.
|
||||
|
||||
The Chrome team resolved the deadlock by hiring me as a Senior Software Engineer (SWE).
|
||||
|
||||
I was initially _very _nervous about this, having not written any significant C++ code in over a decade—except for one [in-place replacement][7] of IE9’s caching logic which I’d coded as a PM because I couldn’t find a developer to do the work. But eventually I started believing in my own pep talk: _“I mean, how hard could it be, right? I’ve been troubleshooting code in web browsers for almost two decades now. I’m not a complete dummy. I’ll ramp up. It’ll be rough, but it’ll work out. Hell, I started writing Fiddler not knowing either C# nor HTTP, and _that _turned out pretty good. I’ll buy some books and get caught up. There’s no way that Google would have just hired me as a C++ developer without asking me any C++ coding questions if it wasn’t going to all be okay. Right? Right?!?”_
|
||||
|
||||
### THE FIREHOSE
|
||||
|
||||
I knew I had a lot to learn, and fast, but it took me a while to realize just how much else I didn’t know.
|
||||
|
||||
Google’s primary development platform is Linux, an OS that I would install every few years, play with for a day, then forget about. My new laptop was a Mac, a platform I’d used a bit more, but still one for which I was about a twentieth as proficient as I was on Windows. The Chrome Windows team made a half-hearted attempt to get me to join their merry band, but warned me honestly that some of the tooling wasn’t quite as good as it was on Linux and it’d probably be harder for me to get help. So I tried to avoid Windows for the first few months, ordering a puny Windows machine that took around four times longer to build Chrome than my obscenely powerful Linux box (with its 48 logical cores). After a few months, I gave up on trying to avoid Windows and started using it as my primary platform. I was more productive, but incredibly slow builds remained a problem for a few months. Everyone told me to just order _another_ obscenely powerful box to put next to my Linux one, but it felt wrong to have hardware at my desk that collectively cost more than my first car—especially when, at Microsoft, I bought all my own hardware. I eventually mentioned my cost/productivity dilemma to a manager, who noted I was getting paid a Google engineer’s salary and then politely asked me if I was just really terrible at math. I ordered a beastly Windows machine and now my builds scream. (To the extent that _any_ C++ builds can scream, of course. At Telerik, I was horrified when a full build of Fiddler slowed to a full 5 seconds on my puny Windows machine; my typical Chrome build today still takes about 15 minutes.)
|
||||
|
||||
Beyond learning different operating systems, I’d never used Google’s apps before (Docs/Sheets/Slides); luckily, I found these easy to pick up, although I still haven’t fully figured out how Google Drive file organization works. Google Docs, in particular, is so good that I’ve pretty much given up on Microsoft Word (which headed downhill after the 2010 version). Google Keep is a low-powered alternative to OneNote (which is, as far as I can tell, banned because it syncs to Microsoft servers) and I haven’t managed to get it to work well for my needs. Google Plus still hasn’t figured out how to support pasting of images via CTRL+V, a baffling limitation for something meant to compete in the space… hell, even _Microsoft Yammer _supports that, for gods sake. The only real downside to the web apps is that tab/window management on modern browsers is still a very much unsolved problem (but more on that in a bit).
|
||||
|
||||
But these speedbumps all pale in comparison to Gmail. Oh, Gmail. As a program manager at Microsoft, pretty much your _entire life _is in your inbox. After twelve years with Outlook and Exchange, switching to Gmail was a train wreck. “_What do you mean, there aren’t folders? How do I mark this message as low priority? Where’s the button to format text with strikethrough? What do you mean, I can’t drag an email to my calendar? What the hell does this Archive thing do? Where’s that message I was just looking at? Hell, where did my Gmail tab even go—it got lost in a pile of sixty other tabs across four top-level Chrome windows. WTH??? How does anyone get anything done?”_
|
||||
|
||||
### COMMUNICATION AND REMOTE WORK
|
||||
|
||||
While Telerik had an office in Austin, I didn’t interact with other employees very often, and when I did they were usually in other offices. I thought I had a handle on remote work, but I really didn’t. Working with a remote team on a daily basis is just _different_.
|
||||
|
||||
With communication happening over mail, IRC, Hangouts, bugs, document markup comments, GVC (video conferencing), G+, and discussion lists, it was often hard to [figure out which mechanisms to use][8], let alone which recipients to target. Undocumented pitfalls abounded (many discussion groups were essentially abandoned while others were unexpectedly broad; turning on chat history was deemed a “no-no” for document retention reasons).
|
||||
|
||||
It often it took a bit of research to even understand who various communication participants were and how they related to the projects at hand.
|
||||
|
||||
After years of email culture at Microsoft, I grew accustomed to a particular style of email, and Google’s is just _different._ Mail threads were long, with frequent additions of new recipients and many terse remarks. Many times, I’d reply privately to someone on a side thread, with a clarifying question, or suggesting a counterpoint to something they said. The response was often “_Hey, this just went to me. Mind adding on the main thread?_”
|
||||
|
||||
I’m working remotely, with peers around the world, so real-time communication with my team is essential. Some Chrome subteams use Hangouts, but the Security team largely uses IRC.
|
||||
|
||||
[
|
||||

|
||||
][9]
|
||||
|
||||
Now, I’ve been chatting with people online since BBSes were a thing (I’ve got a five digit ICQ number somewhere), but my knowledge of IRC was limited to the fact that it was a common way of taking over suckers’ machines with buffer overflows in the ‘90s. My new teammates tried to explain how to IRC repeatedly: “_Oh, it’s easy, you just get this console IRC client. No, no, you don’t run it on your own workstation, that’d be crazy. You wouldn’t have history! You provision a persistent remote VM on a machine in Google’s cloud, then SSH to that, then you run screens and then you run your IRC client in that. Easy peasy._”
|
||||
|
||||
Getting onto IRC remained on my “TODO” list for five months before I finally said “F- it”, installed [HexChat][10] on my Windows box, disabled automatic sleep, and called it done. It’s worked fairly well.
|
||||
|
||||
### GOOGLE DEVELOPER TOOLING
|
||||
|
||||
When an engineer first joins Google, they start with a week or two of technical training on the Google infrastructure. I’ve worked in software development for nearly two decades, and I’ve never even dreamed of the development environment Google engineers get to use. I felt like Charlie Bucket on his tour of Willa Wonka’s Chocolate Factory—astonished by the amazing and unbelievable goodies available at any turn. The computing infrastructure was something out of Star Trek, the development tools were slick and amazing, the _process_ was jaw-dropping.
|
||||
|
||||
While I was doing a “hello world” coding exercise in Google’s environment, a former colleague from the IE team pinged me on Hangouts chat, probably because he’d seen my tweets about feeling like an imposter as a SWE. He sent me a link to click, which I did. Code from Google’s core advertising engine appeared in my browser. Google’s engineers have access to nearly all of the code across the whole company. This alone was astonishing—in contrast, I’d initially joined the IE team so I could get access to the networking code to figure out why the Office Online team’s website wasn’t working. “Neat, I can see everything!” I typed back. “Push the Analyze button” he instructed. I did, and some sort of automated analyzer emitted a report identifying a few dozen performance bugs in the code. “Wow, that’s amazing!” I gushed. “Now, push the Fix button” he instructed. “Uh, this isn’t some sort of security red team exercise, right?” I asked. He assured me that it wasn’t. I pushed the button. The code changed to fix some unnecessary object copies. “Amazing!” I effused. “Click Submit” he instructed. I did, and watched as the system compiled the code in the cloud, determined which tests to run, and ran them. Later that afternoon, an owner of the code in the affected folder typed LGTM (Googlers approve changes by typing the acronym for Looks Good To Me) on the change list I had submitted, and my change was live in production later that day. I was, in a word, gobsmacked. That night, I searched the entire codebase for [misuse][11] of an IE cache control token and proposed fixes for the instances I found. I also narcissistically searched for my own name and found a bunch of references to blog posts I’d written about assorted web development topics.
|
||||
|
||||
Unfortunately for Chrome Engineers, the introduction to Google’s infrastructure is followed by a major letdown—because Chromium is open-source, the Chrome team itself doesn’t get to take advantage of most of Google’s internal goodies. Development of Chrome instead resembles C++ development at most major companies, albeit with an automatically deployed toolchain and enhancements like a web-based code review tool and some super-useful scripts. The most amazing of these is called [bisect-builds][12], and it allows a developer to very quickly discover what build of Chrome introduced a particular bug. You just give it a “known good” build number and a “known bad” build number and it automatically downloads and runs the minimal number of builds to perform a binary search for the build that introduced a given bug:
|
||||
|
||||

|
||||
|
||||
Firefox has [a similar system][13], but I’d’ve killed for something like this back when I was reproducing and reducing bugs in IE. While it’s easy to understand how the system functions, it works so well that it feels like magic. Other useful scripts include the presubmit checks that run on each change list before you submit them for code review—they find and flag various style violations and other problems.
|
||||
|
||||
Compilation itself typically uses a local compiler; on Windows, we use the MSVC command line compiler from Visual Studio 2015 Update 3, although work is underway to switch over to [Clang][14]. Compilation and linking all of Chrome takes quite some time, although on my new beastly dev boxes it’s not _too_ bad. Googlers do have one special perk—we can use Goma (a distributed compiler system that runs on Google’s amazing internal cloud) but I haven’t taken advantage of that so far.
|
||||
|
||||
For bug tracking, Chrome recently moved to [Monorail][15], a straightforward web-based bug tracking system. It works fairly well, although it is somewhat more cumbersome than it needs to be and would be much improved with [a few tweaks][16]. Monorail is open-source, but I haven’t committed to it myself yet.
|
||||
|
||||
For code review, Chrome presently uses [Rietveld][17], a web-based system, but this is slated to change in the near(ish) future. Like Monorail, it’s pretty straightforward although it would benefit from some minor usability tweaks; I committed one trivial change myself, but the pending migration to a different system means that it isn’t likely to see further improvements.
|
||||
|
||||
As an open-source project, Chromium has quite a bit of public [documentation for developers][18], including [Design Documents][19]. Unfortunately, Chrome moves so fast that many of the design documents are out-of-date, and it’s not always obvious what’s current and what was replaced long ago. The team does _value_ engineers’ investment in the documents, however, and various efforts are underway to update the documents and reduce Chrome’s overall architectural complexity. I expect these will be ongoing battles forever, just like in any significant active project.
|
||||
|
||||
### WHAT I’VE DONE
|
||||
|
||||
“That’s all well and good,” my reader asks, “but _what have you done_ in the last year?”
|
||||
|
||||
### I WROTE SOME CODE
|
||||
|
||||
My first check in to Chrome [landed][20] in February; it was a simple adjustment to limit Public-Key-Pins to 60 days. Assorted other checkins trickled in through the spring before I went on paternity leave. The most _fun_ fix I did cleaned up a tiny [UX glitch][21] that sat unnoticed in Chrome for almost a decade; it was mostly interesting because it was a minor thing that I’d tripped over for years, including back in IE. (The root cause was arguably that MSDN documentation about DWM lied; I fixed the bug in Chrome, sent the fix to IE, and asked MSDN to fix their docs).
|
||||
|
||||
I fixed a number of [minor][22] [security][23] [bugs][24], and lately I’ve been working on [UX issues][25] related to Chrome’s HTTPS user-experience. Back in 2005, I wrote [a blog post][26] complaining about websites using HTTPS incorrectly, and now, just over a decade later, Chrome and Firefox are launching UI changes to warn users when a site is collecting sensitive information on pages which are Not Secure; I’m delighted to have a small part in those changes.
|
||||
|
||||
Having written a handful of Internet Explorer Extensions in the past, I was excited to discover the joy of writing Chrome extensions. Chrome extensions are fun, simple, and powerful, and there’s none of the complexity and crashes of COM.
|
||||
|
||||
[
|
||||

|
||||
][27]
|
||||
|
||||
My first and most significant extension is the moarTLS Analyzer– it’s related to my HTTPS work at Google and it’s proven very useful in discovering sites that could improve their security. I [blogged about it][28] and the process of [developing it][29] last year.
|
||||
|
||||
Because I run several different Chrome instances on my PC (and they update daily or weekly), I found myself constantly needing to look up the Chrome version number for bug reports and the like. I wrote a tiny extension that shows the version number in a button on the toolbar (so it’s captured in screenshots too!):
|
||||
|
||||

|
||||
|
||||
More than once, I spent an hour or so trying to reproduce and reduce a bug that had been filed against Chrome. When I found out the cause, I’d jubilently add my notes to the issue in the Monorail bug tracker, click “Save changes” and discover that someone more familiar with the space had beaten me to the punch and figured it out while I’d had the bug open on my screen. Adding an “Issue has been updated” alert to the bug tracker itself seemed like the right way to go, but it would require some changes that I wasn’t able to commit on my own. So, instead I built an extension that provides such alerts within the page until the [feature][30] can be added to the tracker itself.
|
||||
|
||||
Each of these extensions was a joy to write.
|
||||
|
||||
### I FILED SOME BUGS
|
||||
|
||||
I’m a diligent self-hoster, and I run Chrome Canary builds on all of my devices. I submit crash reports and [file bugs][31] with as much information as I can. My proudest moment was in helping narrow down a bizarre and intermittent problem users had with Chrome on Windows 10, where Chrome tabs would crash on every startup until you rebooted the OS. My [blog post][32] explains the full story, and encourages others to file bugs as they encounter them.
|
||||
|
||||
### I TRIAGED MORE BUGS
|
||||
|
||||
I’ve been developing software for Windows for just over two decades, and inevitably I’ve learned quite a bit about it, including the undocumented bits. That’s given me a leg up in understanding bugs in the Windows code. Some of the most fun include issues in Drag and Drop, like this [gem][33] of a bug that means that you can’t drop files from Chrome to most applications in Windows. More meaningful [bugs][34] [relate][35] [to][36] [problems][37] with Windows’ Mark-of-the-Web security feature (about which I’ve [blogged][38] [about][39] [several][40] times).
|
||||
|
||||
### I TOOK SHERIFF ROTATIONS
|
||||
|
||||
Google teams have the notion of sheriffs—a rotating assignment that ensures that important tasks (like triaging incoming security bugs) always has a defined owner, without overwhelming any single person. Each Sheriff has a term of ~1 week where they take on additional duties beyond their day-to-day coding, designing, testing, etc.
|
||||
|
||||
The Sheriff system has some real benefits—perhaps the most important of which is creating a broad swath of people experienced and qualified in making triage decisions around security vulnerabilities. The alternative is to leave such tasks to a single owner, rapidly increasing their [bus factor][41] and thus the risk to the project. (I know this from first-hand experience. After IE8 shipped, I was on my way out the door to join another team. Then IE’s Security PM left, leaving a gaping hole that I felt obliged to stay around to fill. It worked out okay for me and the team, but it was tense all around.)
|
||||
|
||||
I’m on two sheriff rotations: [Enamel][42] (my subteam) and the broader Chrome Security Sheriff.
|
||||
|
||||
The Enamel rotation’s tasks are akin to what I used to do as a Program Manager at Microsoft—triage incoming bugs, respond to questions in the [Help Forums][43], and generally act as a point of contact for my immediate team.
|
||||
|
||||
In contrast, the Security Sheriff rotation is more work, and somewhat more exciting. The Security Sheriff’s [duties][44] include triaging all bugs of type “Security”, assigning priority, severity, and finding an owner for each. Most security bugs are automatically reported by [our fuzzers][45] (a tireless robot army!), but we also get reports from the public and from Chrome team members and [Project Zero][46] too.
|
||||
|
||||
At Microsoft, incoming security bug reports were first received and evaluated by the Microsoft Security Response Center (MSRC); valid reports were passed along to the IE team after some level of analysis and reproduction was undertaken. In general, all communication was done through MSRC, and the turnaround cycle on bugs was _typically _on the order of weeks to months.
|
||||
|
||||
In contrast, anyone can [file a security bug][47] against Chrome, and every week lots of people do. One reason for that is that Chrome has a [Vulnerability Rewards program][48] which pays out up to $100K for reports of vulnerabilities in Chrome and Chrome OS. Chrome paid out just under $1M USD in bounties [last year][49]. This is an _awesome _incentive for researchers to responsibly disclose bugs directly to us, and the bounties are _much _higher than those of nearly any other project.
|
||||
|
||||
In his “[Hacker Quantified Security][50]” talk at the O’Reilly Security conference, HackerOne CTO and Cofounder Alex Rice showed the following chart of bounty payout size for vulnerabilities when explaining why he was using a Chromebook. Apologies for the blurry photo, but the line at the top shows Chrome OS, with the 90th percentile line miles below as severity rises to Critical:
|
||||
|
||||
[
|
||||

|
||||
][51]
|
||||
|
||||
With a top bounty of $100000 for an exploit or exploit chain that fully compromises a Chromebook, researchers are much more likely to send their bugs to us than to try to find a buyer on the black market.
|
||||
|
||||
Bug bounties are great, except when they’re not. Unfortunately, many filers don’t bother to read the [Chrome Security FAQ][52] which explains what constitutes a security vulnerability and the great many things that do not. Nearly every week, we have at least one person (and often more) file a bug noting “_I can use the Developer Tools to read my own password out of a webpage. Can I have a bounty?_” or “_If I install malware on my PC, I can see what happens inside Chrome” _or variations of these.
|
||||
|
||||
Because we take security bug reports very seriously, we often spend a lot of time on what seem like garbage filings to verify that there’s not just some sort of communication problem. This exposes one downside of the sheriff process—the lack of continuity from week to week.
|
||||
|
||||
In the fall, we had one bug reporter file a new issue every week that was just a collection of security related terms (XSS! CSRF! UAF! EoP! Dangling Pointer! Script Injection!) lightly wrapped in prose, including screenshots, snippets from websites, console output from developer tools, and the like. Each week, the sheriff would investigate, ask for more information, and engage in a fruitless back and forth with the filer trying to figure out what claim was being made. Eventually I caught on to what was happening and started monitoring the sheriff’s queue, triaging the new findings directly and sparing the sheriff of the week. But even today we still catch folks who lookup old bug reports (usually Won’t Fixed issues), copy/paste the content into new bugs, and file them into the queue. It’s frustrating, but coming from a closed bug database, I’d choose the openness of the Chrome bug database every time.
|
||||
|
||||
Getting ready for my first Sherriff rotation, I started watching the incoming queue a few months earlier and felt ready for my first rotation in September. Day One was quiet, with a few small issues found by fuzzers and one or two junk reports from the public which I triaged away with pointers to the “_Why isn’t a vulnerability_” entries in the Security FAQ. I spent the rest of the day writing a fix for a lower-priority security [bug][53] that had been filed a month before. A pretty successful day, I thought.
|
||||
|
||||
Day Two was more interesting. Scanning the queue, I saw a few more fuzzer issues and [one external report][54] whose text started with “Here is a Chrome OS exploit chain.” The report was about two pages long, and had a forty-two page PDF attachment explaining the four exploits the finder had used to take over a fully-patched Chromebook.
|
||||
|
||||

|
||||
|
||||
Watching Luke’s X-wing take out the Death Star in Star Wars was no more exciting than reading the PDF’s tale of how a single byte memory overwrite in the DNS resolver code could weave its way through the many-layered security features of the Chromebook and achieve a full compromise. It was like the most amazing magic trick you’ve ever seen.
|
||||
|
||||
I hopped over to IRC. “So, do we see full compromises of Chrome OS every week?” I asked innocently.
|
||||
|
||||
“No. Why?” came the reply from several corners. I pasted in the bug link and a few moments later the replies started flowing in “OMG. Amazing!” Even guys from Project Zero were impressed, and they’re magicians who build exploits like this (usually for other products) all the time. The researcher had found one small bug and a variety of neglected components that were thought to be unreachable and put together a deadly chain.
|
||||
|
||||
The first patches were out for code review that evening, and by the next day, we’d reached out to the open-source owner of the DNS component with the 1-byte overwrite bug so he could release patches for the other projects using his code. Within a few days, fixes to other components landed and had been ported to all of the supported versions of Chrome OS. Two weeks later, the Chrome Vulnerability rewards team added the [reward-100000][55] tag, the only bug so far to be so marked. Four weeks after that, I had to hold my tongue when Alex mentioned that “no one’s ever claimed that $100000 bounty” during his “Hacker Quantified Security” talk. Just under 90 days from filing, the bug was unrestricted and made available for public viewing.
|
||||
|
||||
The remainder of my first Sheriff rotation was considerably less exciting, although still interesting. I spent some time looking through the components the researcher had abused in his exploit chain and filed a few bugs. Ultimately, the most risky component he used was removed entirely.
|
||||
|
||||
### OUTREACH AND BLOGGING
|
||||
|
||||
Beyond working on the Enamel team (focused on Chrome’s security UI surface), I also work on the “MoarTLS” project, designed to help encourage and assist the web as a whole in moving to HTTPS. This takes a number of forms—I help maintain the [HTTPS on Top Sites Report Card][56], I do consultations and HTTPS Audits with major sites as they enable HTTPS on their sites. I discover, reduce, and file bugs on Chrome’s and other browsers’ support of features like Upgrade-Insecure-Requests. I publish a [running list of articles][57] on why and how sites should enable TLS. I hassle teams all over Google (and the web in general) to enable HTTPS on every single hyperlink they emit. I responsibly disclosed security bugs in a number of products and sites, including [a vulnerability][58] in Hillary Clinton’s fundraising emails. I worked to send a notification to many many many thousands of sites collecting user information non-securely, warning them of the [UI changes in Chrome 56][59].
|
||||
|
||||
When I applied to Google for the Developer Advocate role, I expected I’d be delivering public talks _constantly_, but as a SWE I’ve only given a few talks, including my [Migrating to HTTPS talk][60] at the first O’Reilly Security Conference. I had a lot of fun at that conference, catching up with old friends from the security community (mostly ex-Microsofties). I also went to my first [Chrome Dev Summit][61], where I didn’t have a public talk (my colleagues did) but I did get to talk to some major companies about deploying HTTPS.
|
||||
|
||||
I also blogged [quite a bit][62]. At Microsoft, I started blogging because I got tired of repeating myself, and because our Exchange server and document retention policies had started making it hard or impossible to find old responses—I figured “Well, if I publish everything on the web, Google will find it, and Internet Archive will back it up.”
|
||||
|
||||
I’ve kept blogging since leaving Microsoft, and I’m happy that I have even though my reader count numbers are much lower than they were at Microsoft. I’ve managed to mostly avoid trouble, although my posts are not entirely uncontroversial. At Microsoft, they wouldn’t let me publish [this post][63] (because it was too frank); in my first month at Google, I got a phone call at home (during the first portion of my paternity leave) from a Google Director complaining that I’d written [something][64] that was too harsh about a change Microsoft had made. But for the most part, my blogging seems not to ruffle too many feathers.
|
||||
|
||||
### TIDBITS
|
||||
|
||||
* Food at Google is generally _really _good; I’m at a satellite office in Austin, so the selection is much smaller than on the main campuses, but the rotating menu is fairly broad and always has at least three major options. And the breakfasts! I gained about 15 pounds in my first few months, but my pneumonia took it off and I’ve restrained my intake since I came back.
|
||||
* At Microsoft, I always sneered at companies offering free food (“I’m an adult professional. I can pay for my lunch.”), but it’s definitely convenient to not have to hassle with payments. And until the government closes the loophole, it’s a way to increase employees’ compensation without getting taxed.
|
||||
* For the first three months, I was impressed and slightly annoyed that all of the snack options in Google’s micro-kitchens are healthy (e.g. fruit)—probably a good thing, since I sit about twenty feet from one. Then I saw someone open a drawer and pull out some M&Ms, and I learned the secret—all of the junk food is in drawers. The selection is impressive and ranges from the popular to the high end.
|
||||
* Google makes heavy use of the “open-office concept.” I think this makes sense for some teams, but it’s not at all awesome for me. I’d gladly take a 10% salary cut for a private office. I doubt I’m alone.
|
||||
* Coworkers at Google range from very smart to insanely off-the-scales-smart. Yet, almost all of them are humble, approachable, and kind.
|
||||
* Google, like Microsoft, offers gift matching for charities. This is an awesome perk, and one I aim to max out every year. I’m awed by people who go [far][1] beyond that.
|
||||
* **Window Management – **I mentioned earlier that one downside of web-based tools is that it’s hard to even _find _the right tab when I’ve got dozens of open tabs that I’m flipping between. The [Quick Tabs extension][2] is one great mitigation; it shows your tabs in a searchable, most-recently-used list in a convenient dropdown:
|
||||
|
||||
[
|
||||

|
||||
][65]
|
||||
|
||||
Another trick that I learned just this month is that you can instruct Chrome to open a site in “App” mode, where it runs in its own top-level window (with no other tabs), showing the site’s icon as the icon in the Windows taskbar. It’s easy:
|
||||
|
||||
On Windows, run chrome.exe –app=https://mail.google.com
|
||||
|
||||
While on OS X, run open -n -b com.google.Chrome –args –app=’[https://news.google.com][66]‘
|
||||
|
||||
_Tip: The easy way to create a shortcut to a the current page in app mode is to click the Chrome Menu > More Tools > Add to {shelf/desktop} and tick the Open as Window checkbox._
|
||||
|
||||
I now have [SlickRun][67] MagicWords set up for **mail**, **calendar**, and my other critical applications.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://textslashplain.com/2017/02/01/google-chrome-one-year-in/
|
||||
|
||||
作者:[ericlaw][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://textslashplain.com/author/ericlaw1979/
|
||||
[1]:https://www.jefftk.com/p/leaving-google-joining-wave
|
||||
[2]:https://chrome.google.com/webstore/detail/quick-tabs/jnjfeinjfmenlddahdjdmgpbokiacbbb
|
||||
[3]:https://textslashplain.com/2015/12/23/my-next-adventure/
|
||||
[4]:http://sdtimes.com/telerik-acquires-fiddler-debugger-along-with-its-creator/
|
||||
[5]:https://bayden.com/dl/Codemash2015-ericlaw-https-in-2015.pptx
|
||||
[6]:https://conferences.oreilly.com/velocity/devops-web-performance-2015/public/content/2015/04/16-https-stands-for-user-experience
|
||||
[7]:https://textslashplain.com/2015/04/09/on-appreciation/
|
||||
[8]:https://xkcd.com/1254/
|
||||
[9]:http://m.xkcd.com/1782/
|
||||
[10]:https://hexchat.github.io/
|
||||
[11]:https://blogs.msdn.microsoft.com/ieinternals/2009/07/20/internet-explorers-cache-control-extensions/
|
||||
[12]:https://www.chromium.org/developers/bisect-builds-py
|
||||
[13]:https://mozilla.github.io/mozregression/
|
||||
[14]:https://chromium.googlesource.com/chromium/src/+/lkgr/docs/clang.md
|
||||
[15]:https://bugs.chromium.org/p/monorail/adminIntro
|
||||
[16]:https://bugs.chromium.org/p/monorail/issues/list?can=2&q=reporter%3Aelawrence
|
||||
[17]:https://en.wikipedia.org/wiki/Rietveld_(software)
|
||||
[18]:https://www.chromium.org/developers
|
||||
[19]:https://www.chromium.org/developers/design-documents
|
||||
[20]:https://codereview.chromium.org/1733973004/
|
||||
[21]:https://codereview.chromium.org/2244263002/
|
||||
[22]:https://codereview.chromium.org/2323273003/
|
||||
[23]:https://codereview.chromium.org/2368593002/
|
||||
[24]:https://codereview.chromium.org/2347923002/
|
||||
[25]:https://codereview.chromium.org/search?closed=1&owner=elawrence&reviewer=&cc=&repo_guid=&base=&project=&private=1&commit=1&created_before=&created_after=&modified_before=&modified_after=&order=&format=html&keys_only=False&with_messages=False&cursor=&limit=30
|
||||
[26]:https://blogs.msdn.microsoft.com/ie/2005/04/20/tls-and-ssl-in-the-real-world/
|
||||
[27]:https://chrome.google.com/webstore/search/bayden?hl=en-US&_category=extensions
|
||||
[28]:https://textslashplain.com/2016/03/17/seek-and-destroy-non-secure-references-using-the-moartls-analyzer/
|
||||
[29]:https://textslashplain.com/2016/03/18/building-the-moartls-analyzer/
|
||||
[30]:https://bugs.chromium.org/p/monorail/issues/detail?id=1739
|
||||
[31]:https://bugs.chromium.org/p/chromium/issues/list?can=1&q=reporter%3Ame&colspec=ID+Pri+M+Stars+ReleaseBlock+Component+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=ids
|
||||
[32]:https://textslashplain.com/2016/08/18/file-the-bug/
|
||||
[33]:https://bugs.chromium.org/p/chromium/issues/detail?id=540547
|
||||
[34]:https://bugs.chromium.org/p/chromium/issues/detail?id=601538
|
||||
[35]:https://bugs.chromium.org/p/chromium/issues/detail?id=595844#c6
|
||||
[36]:https://bugs.chromium.org/p/chromium/issues/detail?id=629637
|
||||
[37]:https://bugs.chromium.org/p/chromium/issues/detail?id=591343
|
||||
[38]:https://textslashplain.com/2016/04/04/downloads-and-the-mark-of-the-web/
|
||||
[39]:https://blogs.msdn.microsoft.com/ieinternals/2011/03/23/understanding-local-machine-zone-lockdown/
|
||||
[40]:https://blogs.msdn.microsoft.com/ieinternals/2012/06/19/enhanced-protected-mode-and-local-files/
|
||||
[41]:https://en.wikipedia.org/wiki/Bus_factor
|
||||
[42]:https://www.chromium.org/Home/chromium-security/enamel
|
||||
[43]:https://productforums.google.com/forum/#!forum/chrome
|
||||
[44]:https://www.chromium.org/Home/chromium-security/security-sheriff
|
||||
[45]:https://blog.chromium.org/2012/04/fuzzing-for-security.html
|
||||
[46]:https://en.wikipedia.org/wiki/Project_Zero_(Google)
|
||||
[47]:https://bugs.chromium.org/p/chromium/issues/entry?template=Security%20Bug
|
||||
[48]:https://www.google.com/about/appsecurity/chrome-rewards/
|
||||
[49]:https://security.googleblog.com/2017/01/vulnerability-rewards-program-2016-year.html
|
||||
[50]:https://conferences.oreilly.com/security/network-data-security-ny/public/schedule/detail/53296
|
||||
[51]:https://textplain.files.wordpress.com/2017/01/image58.png
|
||||
[52]:https://dev.chromium.org/Home/chromium-security/security-faq
|
||||
[53]:https://bugs.chromium.org/p/chromium/issues/detail?id=639126#c11
|
||||
[54]:https://bugs.chromium.org/p/chromium/issues/detail?id=648971
|
||||
[55]:https://bugs.chromium.org/p/chromium/issues/list?can=1&q=label%3Areward-100000&colspec=ID+Pri+M+Stars+ReleaseBlock+Component+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=ids
|
||||
[56]:https://www.google.com/transparencyreport/https/grid/?hl=en
|
||||
[57]:https://whytls.com/
|
||||
[58]:https://textslashplain.com/2016/09/22/use-https-for-all-inbound-links/
|
||||
[59]:https://security.googleblog.com/2016/09/moving-towards-more-secure-web.html
|
||||
[60]:https://www.safaribooksonline.com/library/view/the-oreilly-security/9781491960035/video287622.html
|
||||
[61]:https://developer.chrome.com/devsummit/
|
||||
[62]:https://textslashplain.com/2016/
|
||||
[63]:https://blogs.msdn.microsoft.com/ieinternals/2013/10/16/strict-p3p-validation/
|
||||
[64]:https://textslashplain.com/2016/01/20/putting-users-first/
|
||||
[65]:https://chrome.google.com/webstore/detail/quick-tabs/jnjfeinjfmenlddahdjdmgpbokiacbbb
|
||||
[66]:https://news.google.com/
|
||||
[67]:https://bayden.com/slickrun/
|
@ -1,86 +0,0 @@
|
||||
translating by hopefully2333
|
||||
|
||||
# [The One in Which I Call Out Hacker News][14]
|
||||
|
||||
|
||||
> “Implementing caching would take thirty hours. Do you have thirty extra hours? No, you don’t. I actually have no idea how long it would take. Maybe it would take five minutes. Do you have five minutes? No. Why? Because I’m lying. It would take much longer than five minutes. That’s the eternal optimism of programmers.”
|
||||
>
|
||||
> — Professor [Owen Astrachan][1] during 23 Feb 2004 lecture for [CPS 108][2]
|
||||
|
||||
[Accusing open-source software of being a royal pain to use][5] is not a new argument; it’s been said before, by those much more eloquent than I, and even by some who are highly sympathetic to the open-source movement. Why go over it again?
|
||||
|
||||
On Hacker News on Monday, I was amused to read some people saying that [writing StackOverflow was hilariously easy][6]—and proceeding to back up their claim by [promising to clone it over July 4th weekend][7]. Others chimed in, pointing to [existing][8] [clones][9] as a good starting point.
|
||||
|
||||
Let’s assume, for sake of argument, that you decide it’s okay to write your StackOverflow clone in ASP.NET MVC, and that I, after being hypnotized with a pocket watch and a small club to the head, have decided to hand you the StackOverflow source code, page by page, so you can retype it verbatim. We’ll also assume you type like me, at a cool 100 WPM ([a smidge over eight characters per second][10]), and unlike me, _you_ make zero mistakes. StackOverflow’s *.cs, *.sql, *.css, *.js, and *.aspx files come to 2.3 MB. So merely typing the source code back into the computer will take you about eighty hours if you make zero mistakes.
|
||||
|
||||
Except, of course, you’re not doing that; you’re going to implement StackOverflow from scratch. So even assuming that it took you a mere ten times longer to design, type out, and debug your own implementation than it would take you to copy the real one, that already has you coding for several weeks straight—and I don’t know about you, but I am okay admitting I write new code _considerably_ less than one tenth as fast as I copy existing code.
|
||||
|
||||
_Well, okay_ , I hear you relent. *So not the whole thing. But I can do **most** of it.*
|
||||
|
||||
Okay, so what’s “most”? There’s simply asking and responding to questions—that part’s easy. Well, except you have to implement voting questions and answers up and down, and the questioner should be able to accept a single answer for each question. And you can’t let people upvote or accept their own answers, so you need to block that. And you need to make sure that users don’t upvote or downvote another user too many times in a certain amount of time, to prevent spambots. Probably going to have to implement a spam filter, too, come to think of it, even in the basic design, and you also need to support user icons, and you’re going to have to find a sanitizing HTML library you really trust and that interfaces well with Markdown (provided you do want to reuse [that awesome editor][11] StackOverflow has, of course). You’ll also need to purchase, design, or find widgets for all the controls, plus you need at least a basic administration interface so that moderators can moderate, and you’ll need to implement that scaling karma thing so that you give users steadily increasing power to do things as they go.
|
||||
|
||||
But if you do _all that_ , you _will_ be done.
|
||||
|
||||
Except…except, of course, for the full-text search, especially its appearance in the search-as-you-ask feature, which is kind of indispensable. And user bios, and having comments on answers, and having a main page that shows you important questions but that bubbles down steadily à la reddit. Plus you’ll totally need to implement bounties, and support multiple OpenID logins per user, and send out email notifications for pertinent events, and add a tagging system, and allow administrators to configure badges by a nice GUI. And you’ll need to show users’ karma history, upvotes, and downvotes. And the whole thing has to scale really well, since it could be slashdotted/reddited/StackOverflown at any moment.
|
||||
|
||||
But _then_ ! **Then** you’re done!
|
||||
|
||||
…right after you implement upgrades, internationalization, karma caps, a CSS design that makes your site not look like ass, AJAX versions of most of the above, and G-d knows what else that’s lurking just beneath the surface that you currently take for granted, but that will come to bite you when you start to do a real clone.
|
||||
|
||||
Tell me: which of those features do you feel you can cut and still have a compelling offering? Which ones go under “most” of the site, and which can you punt?
|
||||
|
||||
Developers think cloning a site like StackOverflow is easy for the same reason that open-source software remains such a horrible pain in the ass to use. When you put a developer in front of StackOverflow, they don’t really _see_ StackOverflow. What they actually _see_ is this:
|
||||
|
||||
```
|
||||
create table QUESTION (ID identity primary key,
|
||||
TITLE varchar(255), --- why do I know you thought 255?
|
||||
BODY text,
|
||||
UPVOTES integer not null default 0,
|
||||
DOWNVOTES integer not null default 0,
|
||||
USER integer references USER(ID));
|
||||
create table RESPONSE (ID identity primary key,
|
||||
BODY text,
|
||||
UPVOTES integer not null default 0,
|
||||
DOWNVOTES integer not null default 0,
|
||||
QUESTION integer references QUESTION(ID))
|
||||
```
|
||||
|
||||
If you then tell a developer to replicate StackOverflow, what goes into his head are the above two SQL tables and enough HTML to display them without formatting, and that really _is_ completely doable in a weekend. The smarter ones will realize that they need to implement login and logout, and comments, and that the votes need to be tied to a user, but that’s still totally doable in a weekend; it’s just a couple more tables in a SQL back-end, and the HTML to show their contents. Use a framework like Django, and you even get basic users and comments for free.
|
||||
|
||||
But that’s _not_ what StackOverflow is about. Regardless of what your feelings may be on StackOverflow in general, most visitors seem to agree that the user experience is smooth, from start to finish. They feel that they’re interacting with a polished product. Even if I didn’t know better, I would guess that very little of what actually makes StackOverflow a continuing success has to do with the database schema—and having had a chance to read through StackOverflow’s source code, I know how little really does. There is a _tremendous_ amount of spit and polish that goes into making a major website highly usable. A developer, asked how hard something will be to clone, simply _does not think about the polish_ , because _the polish is incidental to the implementation._
|
||||
|
||||
That is why an open-source clone of StackOverflow will fail. Even if someone were to manage to implement most of StackOverflow “to spec,” there are some key areas that would trip them up. Badges, for example, if you’re targeting end-users, either need a GUI to configure rules, or smart developers to determine which badges are generic enough to go on all installs. What will actually happen is that the developers will bitch and moan about how you can’t implement a really comprehensive GUI for something like badges, and then bikeshed any proposals for standard badges so far into the ground that they’ll hit escape velocity coming out the other side. They’ll ultimately come up with the same solution that bug trackers like Roundup use for their workflow: the developers implement a generic mechanism by which anyone, truly anyone at all, who feels totally comfortable working with the system API in Python or PHP or whatever, can easily add their own customizations. And when PHP and Python are so easy to learn and so much more flexible than a GUI could ever be, why bother with anything else?
|
||||
|
||||
Likewise, the moderation and administration interfaces can be punted. If you’re an admin, you have access to the SQL server, so you can do anything really genuinely administrative-like that way. Moderators can get by with whatever django-admin and similar systems afford you, since, after all, few users are mods, and mods should understand how the sites _work_ , dammit. And, certainly, none of StackOverflow’s interface failings will be rectified. Even if StackOverflow’s stupid requirement that you have to have and know how to use an OpenID (its worst failing) eventually gets fixed, I’m sure any open-source clones will rabidly follow it—just as GNOME and KDE for years slavishly copied off Windows, instead of trying to fix its most obvious flaws.
|
||||
|
||||
Developers may not care about these parts of the application, but end-users do, and take it into consideration when trying to decide what application to use. Much as a good software company wants to minimize its support costs by ensuring that its products are top-notch before shipping, so, too, savvy consumers want to ensure products are good before they purchase them so that they won’t _have_ to call support. Open-source products fail hard here. Proprietary solutions, as a rule, do better.
|
||||
|
||||
That’s not to say that open-source doesn’t have its place. This blog runs on Apache, [Django][12], [PostgreSQL][13], and Linux. But let me tell you, configuring that stack is _not_ for the faint of heart. PostgreSQL needs vacuuming configured on older versions, and, as of recent versions of Ubuntu and FreeBSD, still requires the user set up the first database cluster. MS SQL requires neither of those things. Apache…dear heavens, don’t even get me _started_ on trying to explain to a novice user how to get virtual hosting, MovableType, a couple Django apps, and WordPress all running comfortably under a single install. Hell, just trying to explain the forking vs. threading variants of Apache to a technically astute non-developer can be a nightmare. IIS 7 and Apache with OS X Server’s very much closed-source GUI manager make setting up those same stacks vastly simpler. Django’s a great a product, but it’s nothing _but_ infrastructure—exactly the thing that I happen to think open-source _does_ do well, _precisely_ because of the motivations that drive developers to contribute.
|
||||
|
||||
The next time you see an application you like, think very long and hard about all the user-oriented details that went into making it a pleasure to use, before decrying how you could trivially reimplement the entire damn thing in a weekend. Nine times out of ten, when you think an application was ridiculously easy to implement, you’re completely missing the user side of the story.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://bitquabit.com/post/one-which-i-call-out-hacker-news/
|
||||
|
||||
作者:[Benjamin Pollack][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://bitquabit.com/meta/about/
|
||||
[1]:http://www.cs.duke.edu/~ola/
|
||||
[2]:http://www.cs.duke.edu/courses/cps108/spring04/
|
||||
[3]:https://bitquabit.com/categories/programming
|
||||
[4]:https://bitquabit.com/categories/technology
|
||||
[5]:http://blog.bitquabit.com/2009/06/30/one-which-i-say-open-source-software-sucks/
|
||||
[6]:http://news.ycombinator.com/item?id=678501
|
||||
[7]:http://news.ycombinator.com/item?id=678704
|
||||
[8]:http://code.google.com/p/cnprog/
|
||||
[9]:http://code.google.com/p/soclone/
|
||||
[10]:http://en.wikipedia.org/wiki/Words_per_minute
|
||||
[11]:http://github.com/derobins/wmd/tree/master
|
||||
[12]:http://www.djangoproject.com/
|
||||
[13]:http://www.postgresql.org/
|
||||
[14]:https://bitquabit.com/post/one-which-i-call-out-hacker-news/
|
@ -0,0 +1,211 @@
|
||||
# Dynamic linker tricks: Using LD_PRELOAD to cheat, inject features and investigate programs
|
||||
|
||||
**This post assumes some basic C skills.**
|
||||
|
||||
Linux puts you in full control. This is not always seen from everyone’s perspective, but a power user loves to be in control. I’m going to show you a basic trick that lets you heavily influence the behavior of most applications, which is not only fun, but also, at times, useful.
|
||||
|
||||
#### A motivational example
|
||||
|
||||
Let us begin with a simple example. Fun first, science later.
|
||||
|
||||
|
||||
random_num.c:
|
||||
```
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
|
||||
int main(){
|
||||
srand(time(NULL));
|
||||
int i = 10;
|
||||
while(i--) printf("%d\n",rand()%100);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
Simple enough, I believe. I compiled it with no special flags, just
|
||||
|
||||
> ```
|
||||
> gcc random_num.c -o random_num
|
||||
> ```
|
||||
|
||||
I hope the resulting output is obvious – ten randomly selected numbers 0-99, hopefully different each time you run this program.
|
||||
|
||||
Now let’s pretend we don’t really have the source of this executable. Either delete the source file, or move it somewhere – we won’t need it. We will significantly modify this programs behavior, yet without touching it’s source code nor recompiling it.
|
||||
|
||||
For this, lets create another simple C file:
|
||||
|
||||
|
||||
unrandom.c:
|
||||
```
|
||||
int rand(){
|
||||
return 42; //the most random number in the universe
|
||||
}
|
||||
```
|
||||
|
||||
We’ll compile it into a shared library.
|
||||
|
||||
> ```
|
||||
> gcc -shared -fPIC unrandom.c -o unrandom.so
|
||||
> ```
|
||||
|
||||
So what we have now is an application that outputs some random data, and a custom library, which implements the rand() function as a constant value of 42\. Now… just run _random_num _ this way, and watch the result:
|
||||
|
||||
> ```
|
||||
> LD_PRELOAD=$PWD/unrandom.so ./random_nums
|
||||
> ```
|
||||
|
||||
If you are lazy and did not do it yourself (and somehow fail to guess what might have happened), I’ll let you know – the output consists of ten 42’s.
|
||||
|
||||
This may be even more impressive it you first:
|
||||
|
||||
> ```
|
||||
> export LD_PRELOAD=$PWD/unrandom.so
|
||||
> ```
|
||||
|
||||
and then run the program normally. An unchanged app run in an apparently usual manner seems to be affected by what we did in our tiny library…
|
||||
|
||||
###### **Wait, what? What did just happen?**
|
||||
|
||||
Yup, you are right, our program failed to generate random numbers, because it did not use the “real” rand(), but the one we provided – which returns 42 every time.
|
||||
|
||||
###### **But we *told* it to use the real one. We programmed it to use the real one. Besides, at the time we created that program, the fake rand() did not even exist!**
|
||||
|
||||
This is not entirely true. We did not choose which rand() we want our program to use. We told it just to use rand().
|
||||
|
||||
When our program is started, certain libraries (that provide functionality needed by the program) are loaded. We can learn which are these using _ldd_ :
|
||||
|
||||
> ```
|
||||
> $ ldd random_nums
|
||||
> linux-vdso.so.1 => (0x00007fff4bdfe000)
|
||||
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f48c03ec000)
|
||||
> /lib64/ld-linux-x86-64.so.2 (0x00007f48c07e3000)
|
||||
> ```
|
||||
|
||||
What you see as the output is the list of libs that are needed by _random_nums_ . This list is built into the executable, and is determined compile time. The exact output might slightly differ on your machine, but a **libc.so** must be there – this is the file which provides core C functionality. That includes the “real” rand().
|
||||
|
||||
We can have a peek at what functions does libc provide. I used the following to get a full list:
|
||||
|
||||
> ```
|
||||
> nm -D /lib/libc.so.6
|
||||
> ```
|
||||
|
||||
The _nm_ command lists symbols found in a binary file. The -D flag tells it to look for dynamic symbols, which makes sense, as libc.so.6 is a dynamic library. The output is very long, but it indeed lists rand() among many other standard functions.
|
||||
|
||||
Now what happens when we set up the environmental variable LD_PRELOAD? This variable **forces some libraries to be loaded for a program**. In our case, it loads _unrandom.so_ for _random_num_ , even though the program itself does not ask for it. The following command may be interesting:
|
||||
|
||||
> ```
|
||||
> $ LD_PRELOAD=$PWD/unrandom.so ldd random_nums
|
||||
> linux-vdso.so.1 => (0x00007fff369dc000)
|
||||
> /some/path/to/unrandom.so (0x00007f262b439000)
|
||||
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f262b044000)
|
||||
> /lib64/ld-linux-x86-64.so.2 (0x00007f262b63d000)
|
||||
> ```
|
||||
|
||||
Note that it lists our custom library. And indeed this is the reason why it’s code get’s executed: _random_num_ calls rand(), but if _unrandom.so_ is loaded it is our library that provides implementation for rand(). Neat, isn’t it?
|
||||
|
||||
#### Being transparent
|
||||
|
||||
This is not enough. I’d like to be able to inject some code into an application in a similar manner, but in such way that it will be able to function normally. It’s clear if we implemented open() with a simple “ _return 0;_ “, the application we would like to hack should malfunction. The point is to be **transparent**, and to actually call the original open:
|
||||
|
||||
inspect_open.c:
|
||||
```
|
||||
int open(const char *pathname, int flags){
|
||||
/* Some evil injected code goes here. */
|
||||
return open(pathname,flags); // Here we call the "real" open function, that is provided to us by libc.so
|
||||
}
|
||||
```
|
||||
|
||||
Hm. Not really. This won’t call the “original” open(…). Obviously, this is an endless recursive call.
|
||||
|
||||
How do we access the “real” open function? It is needed to use the programming interface to the dynamic linker. It’s simpler than it sounds. Have a look at this complete example, and then I’ll explain what happens there:
|
||||
|
||||
inspect_open.c:
|
||||
|
||||
```
|
||||
#define _GNU_SOURCE
|
||||
#include <dlfcn.h>
|
||||
|
||||
typedef int (*orig_open_f_type)(const char *pathname, int flags);
|
||||
|
||||
int open(const char *pathname, int flags, ...)
|
||||
{
|
||||
/* Some evil injected code goes here. */
|
||||
|
||||
orig_open_f_type orig_open;
|
||||
orig_open = (orig_open_f_type)dlsym(RTLD_NEXT,"open");
|
||||
return orig_open(pathname,flags);
|
||||
}
|
||||
```
|
||||
|
||||
The _dlfcn.h_ is needed for _dlsym_ function we use later. That strange _#define_ directive instructs the compiler to enable some non-standard stuff, we need it to enable _RTLD_NEXT_ in _dlfcn.h_ . That typedef is just creating an alias to a complicated pointer-to-function type, with arguments just as the original open – the alias name is _orig_open_f_type_ , which we’ll use later.
|
||||
|
||||
The body of our custom open(…) consists of some custom code. The last part of it creates a new function pointer _orig_open_ which will point to the original open(…) function. In order to get the address of that function, we ask _dlsym_ to find for us the next “open” function on dynamic libraries stack. Finally, we call that function (passing the same arguments as were passed to our fake “open”), and return it’s return value as ours.
|
||||
|
||||
As the “evil injected code” I simply used:
|
||||
|
||||
inspect_open.c (fragment):
|
||||
|
||||
```
|
||||
printf("The victim used open(...) to access '%s'!!!\n",pathname); //remember to include stdio.h!
|
||||
```
|
||||
|
||||
To compile it, I needed to slightly adjust compiler flags:
|
||||
|
||||
> ```
|
||||
> gcc -shared -fPIC inspect_open.c -o inspect_open.so -ldl
|
||||
> ```
|
||||
|
||||
I had to append _-ldl_ , so that this shared library is linked to _libdl_ , which provides the _dlsym_ function. (Nah, I am not going to create a fake version of _dlsym_ , though this might be fun.)
|
||||
|
||||
So what do I have in result? A shared library, which implements the open(…) function so that it behaves **exactly** as the real open(…)… except it has a side effect of _printf_ ing the file path :-)
|
||||
|
||||
If you are not convinced this is a powerful trick, it’s the time you tried the following:
|
||||
|
||||
> ```
|
||||
> LD_PRELOAD=$PWD/inspect_open.so gnome-calculator
|
||||
> ```
|
||||
|
||||
I encourage you to see the result yourself, but basically it lists every file this application accesses. In real time.
|
||||
|
||||
I believe it’s not that hard to imagine why this might be useful for debugging or investigating unknown applications. Please note, however, that this particular trick is not quite complete, because _open()_ is not the only function that opens files… For example, there is also _open64()_ in the standard library, and for full investigation you would need to create a fake one too.
|
||||
|
||||
#### **Possible uses**
|
||||
|
||||
If you are still with me and enjoyed the above, let me suggest a bunch of ideas of what can be achieved using this trick. Keep in mind that you can do all the above without to source of the affected app!
|
||||
|
||||
1. ~~Gain root privileges.~~ Not really, don’t even bother, you won’t bypass any security this way. (A quick explanation for pros: no libraries will be preloaded this way if ruid != euid)
|
||||
|
||||
2. Cheat games: **Unrandomize.** This is what I did in the first example. For a fully working case you would need also to implement a custom _random()_ , _rand_r()_ _, random_r()_ . Also some apps may be reading from _/dev/urandom_ or so, you might redirect them to _/dev/null_ by running the original _open()_ with a modified file path. Furthermore, some apps may have their own random number generation algorithm, there is little you can do about that (unless: point 10 below). But this looks like an easy exercise for beginners.
|
||||
|
||||
3. Cheat games: **Bullet time. **Implement all standard time-related functions pretend the time flows two times slower. Or ten times slower. If you correctly calculate new values for time measurement, timed _sleep_ functions, and others, the affected application will believe the time runs slower (or faster, if you wish), and you can experience awesome bullet-time action.
|
||||
Or go **even one step further** and let your shared library also be a DBus client, so that you can communicate with it real time. Bind some shortcuts to custom commands, and with some additional calculations in your fake timing functions you will be able to enable&disable the slow-mo or fast-forward anytime you wish.
|
||||
|
||||
4. Investigate apps: **List accessed files.** That’s what my second example does, but this could be also pushed further, by recording and monitoring all app’s file I/O.
|
||||
|
||||
5. Investigate apps: **Monitor internet access.** You might do this with Wireshark or similar software, but with this trick you could actually gain control of what an app sends over the web, and not just look, but also affect the exchanged data. Lots of possibilities here, from detecting spyware, to cheating in multiplayer games, or analyzing & reverse-engineering protocols of closed-source applications.
|
||||
|
||||
6. Investigate apps: **Inspect GTK structures.** Why just limit ourselves to standard library? Let’s inject code in all GTK calls, so that we can learn what widgets does an app use, and how are they structured. This might be then rendered either to an image or even to a gtkbuilder file! Super useful if you want to learn how does some app manage its interface!
|
||||
|
||||
7. **Sandbox unsafe applications.** If you don’t trust some app and are afraid that it may wish to _ rm -rf / _ or do some other unwanted file activities, you might potentially redirect all it’s file IO to e.g. /tmp by appropriately modifying the arguments it passes to all file-related functions (not just _open_ , but also e.g. removing directories etc.). It’s more difficult trick that a chroot, but it gives you more control. It would be only as safe as complete your “wrapper” was, and unless you really know what you’re doing, don’t actually run any malicious software this way.
|
||||
|
||||
8. **Implement features.** [zlibc][1] is an actual library which is run this precise way; it uncompresses files on the go as they are accessed, so that any application can work on compressed data without even realizing it.
|
||||
|
||||
9. **Fix bugs. **Another real-life example: some time ago (I am not sure this is still the case) Skype – which is closed-source – had problems capturing video from some certain webcams. Because the source could not be modified as Skype is not free software, this was fixed by preloading a library that would correct these problems with video.
|
||||
|
||||
10. Manually **access application’s own memory**. Do note that you can access all app data this way. This may be not impressive if you are familiar with software like CheatEngine/scanmem/GameConqueror, but they all require root privileges to work. LD_PRELOAD does not. In fact, with a number of clever tricks your injected code might access all app memory, because, in fact, it gets executed by that application itself. You might modify everything this application can. You can probably imagine this allows a lot of low-level hacks… but I’ll post an article about it another time.
|
||||
|
||||
These are only the ideas I came up with. I bet you can find some too, if you do – share them by commenting!
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://rafalcieslak.wordpress.com/2013/04/02/dynamic-linker-tricks-using-ld_preload-to-cheat-inject-features-and-investigate-programs/
|
||||
|
||||
作者:[Rafał Cieślak ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://rafalcieslak.wordpress.com/
|
||||
[1]:http://www.zlibc.linux.lu/index.html
|
@ -0,0 +1,361 @@
|
||||
How to turn any syscall into an event: Introducing eBPF Kernel probes
|
||||
============================================================
|
||||
|
||||
|
||||
TL;DR: Using eBPF in recent (>=4.4) Linux kernel, you can turn any kernel function call into a user land event with arbitrary data. This is made easy by bcc. The probe is written in C while the data is handled by python.
|
||||
|
||||
If you are not familiar with eBPF or linux tracing, you really should read the full post. It tries to progressively go through the pitfalls I stumbled unpon while playing around with bcc / eBPF while saving you a lot of the time I spent searching and digging.
|
||||
|
||||
### A note on push vs pull in a Linux world
|
||||
|
||||
When I started to work on containers, I was wondering how we could update a load balancer configuration dynamically based on actual system state. A common strategy, which works, it to let the container orchestrator trigger a load balancer configuration update whenever it starts a container and then let the load balancer poll the container until some health check passes. It may be a simple “SYN” test.
|
||||
|
||||
While this configuration works, it has the downside of making your load balancer waiting for some system to be available while it should be… load balancing.
|
||||
|
||||
Can we do better?
|
||||
|
||||
When you want a program to react to some change in a system there are 2 possible strategies. The program may _poll_ the system to detect changes or, if the system supports it, the system may _push_ events and let the program react to them. Wether you want to use push or poll depends on the context. A good rule of the thumb is to use push events when the event rate is low with respect to the processing time and switch to polling when the events are coming fast or the system may become unusable. For example, typical network driver will wait for events from the network card while frameworks like dpdk will actively poll the card for events to achieve the highest throughput and lowest latency.
|
||||
|
||||
In an ideal world, we’d have some kernel interface telling us:
|
||||
|
||||
> * “Hey Mr. ContainerManager, I’ve just created a socket for the Nginx-ware of container _servestaticfiles_ , maybe you want to update your state?”
|
||||
>
|
||||
> * “Sure Mr. OS, Thanks for letting me know”
|
||||
|
||||
While Linux has a wide range of interfaces to deal with events, up to 3 for file events, there is no dedicated interface to get socket event notifications. You can get routing table events, neighbor table events, conntrack events, interface change events. Just, not socket events. Or maybe there is, deep hidden in a Netlink interface.
|
||||
|
||||
Ideally, we’d need a generic way to do it. How?
|
||||
|
||||
### Kernel tracing and eBPF, a bit of history
|
||||
|
||||
Until recently the only way was to patch the kernel or resort on SystemTap. [SytemTap][5] is a tracing Linux system. In a nutshell, it provides a DSL which is then compiled into a kernel module which is then live-loaded into the running kernel. Except that some production system disable dynamic module loading for security reasons. Including the one I was working on at that time. The other way would be to patch the kernel to trigger some events, probably based on netlink. This is not really convenient. Kernel hacking come with downsides including “interesting” new “features” and increased maintenance burden.
|
||||
|
||||
Hopefully, starting with Linux 3.15 the ground was laid to safely transform any traceable kernel function into userland events. “Safely” is common computer science expression referring to “some virtual machine”. This case is no exception. Linux has had one for years. Since Linux 2.1.75 released in 1997 actually. It’s called Berkeley Packet Filter of BPF for short. As its name suggests, it was originally developed for the BSD firewalls. It had only 2 registers and only allowed forward jumps meaning that you could not write loops with it (Well, you can, if you know the maximum iterations and you manually unroll them). The point was to guarantee the program would always terminate and hence never hang the system. Still not sure if it has any use while you have iptables? It serves as the [foundation of CloudFlare’s AntiDDos protection][6].
|
||||
|
||||
OK, so, with Linux the 3.15, [BPF was extended][7] turning it into eBPF. For “extended” BPF. It upgrades from 2 32 bits registers to 10 64 bits 64 registers and adds backward jumping among others. It has then been [further extended in Linux 3.18][8] moving it out of the networking subsystem, and adding tools like maps. To preserve the safety guarantees, it [introduces a checker][9] which validates all memory accesses and possible code path. If the checker can’t guarantee the code will terminate within fixed boundaries, it will deny the initial insertion of the program.
|
||||
|
||||
For more history, there is [an excellent Oracle presentation on eBPF][10].
|
||||
|
||||
Let’s get started.
|
||||
|
||||
### Hello from from `inet_listen`
|
||||
|
||||
As writing assembly is not the most convenient task, even for the best of us, we’ll use [bcc][11]. bcc is a collection of tools based on LLVM and Python abstracting the underlying machinery. Probes are written in C and the results can be exploited from python allowing to easily write non trivial applications.
|
||||
|
||||
Start by install bcc. For some of these examples, you may require a recent (read >= 4.4) version of the kernel. If you are willing to actually try these examples, I highly recommend that you setup a VM. _NOT_ a docker container. You can’t change the kernel in a container. As this is a young and dynamic projects, install instructions are highly platform/version dependant. You can find up to date instructions on [https://github.com/iovisor/bcc/blob/master/INSTALL.md][12]
|
||||
|
||||
So, we want to get an event whenever a program starts to listen on TCP socket. When calling the `listen()` syscall on a `AF_INET` + `SOCK_STREAM` socket, the underlying kernel function is [`inet_listen`][13]. We’ll start by hooking a “Hello World” `kprobe` on it’s entrypoint.
|
||||
|
||||
```
|
||||
from bcc import BPF
|
||||
|
||||
# Hello BPF Program
|
||||
bpf_text = """
|
||||
#include <net/inet_sock.h>
|
||||
#include <bcc/proto.h>
|
||||
|
||||
// 1\. Attach kprobe to "inet_listen"
|
||||
int kprobe__inet_listen(struct pt_regs *ctx, struct socket *sock, int backlog)
|
||||
{
|
||||
bpf_trace_printk("Hello World!\\n");
|
||||
return 0;
|
||||
};
|
||||
"""
|
||||
|
||||
# 2\. Build and Inject program
|
||||
b = BPF(text=bpf_text)
|
||||
|
||||
# 3\. Print debug output
|
||||
while True:
|
||||
print b.trace_readline()
|
||||
|
||||
```
|
||||
|
||||
This program does 3 things: 1\. It attaches a kernel probe to “inet_listen” using a naming convention. If the function was called, say, “my_probe”, it could be explicitly attached with `b.attach_kprobe("inet_listen", "my_probe"`. 2\. It builds the program using LLVM new BPF backend, inject the resulting bytecode using the (new) `bpf()` syscall and automatically attaches the probes matching the naming convention. 3\. It reads the raw output from the kernel pipe.
|
||||
|
||||
Note: eBPF backend of LLVM is still young. If you think you’ve hit a bug, you may want to upgrade.
|
||||
|
||||
Noticed the `bpf_trace_printk` call? This is a stripped down version of the kernel’s `printk()`debug function. When used, it produces tracing informations to a special kernel pipe in `/sys/kernel/debug/tracing/trace_pipe`. As the name implies, this is a pipe. If multiple readers are consuming it, only 1 will get a given line. This makes it unsuitable for production.
|
||||
|
||||
Fortunately, Linux 3.19 introduced maps for message passing and Linux 4.4 brings arbitrary perf events support. I’ll demo the perf event based approach later in this post.
|
||||
|
||||
```
|
||||
# From a first console
|
||||
ubuntu@bcc:~/dev/listen-evts$ sudo /python tcv4listen.py
|
||||
nc-4940 [000] d... 22666.991714: : Hello World!
|
||||
|
||||
# From a second console
|
||||
ubuntu@bcc:~$ nc -l 0 4242
|
||||
^C
|
||||
|
||||
```
|
||||
|
||||
Yay!
|
||||
|
||||
### Grab the backlog
|
||||
|
||||
Now, let’s print some easily accessible data. Say the “backlog”. The backlog is the number of pending established TCP connections, pending to be `accept()`ed.
|
||||
|
||||
Just tweak a bit the `bpf_trace_printk`:
|
||||
|
||||
```
|
||||
bpf_trace_printk("Listening with with up to %d pending connections!\\n", backlog);
|
||||
|
||||
```
|
||||
|
||||
If you re-run the example with this world-changing improvement, you should see something like:
|
||||
|
||||
```
|
||||
(bcc)ubuntu@bcc:~/dev/listen-evts$ sudo python tcv4listen.py
|
||||
nc-5020 [000] d... 25497.154070: : Listening with with up to 1 pending connections!
|
||||
|
||||
```
|
||||
|
||||
`nc` is a single connection program, hence the backlog of 1\. Nginx or Redis would output 128 here. But that’s another story.
|
||||
|
||||
Easy hue? Now let’s get the port.
|
||||
|
||||
### Grab the port and IP
|
||||
|
||||
Studying `inet_listen` source from the kernel, we know that we need to get the `inet_sock` from the `socket` object. Just copy from the sources, and insert at the beginning of the tracer:
|
||||
|
||||
```
|
||||
// cast types. Intermediate cast not needed, kept for readability
|
||||
struct sock *sk = sock->sk;
|
||||
struct inet_sock *inet = inet_sk(sk);
|
||||
|
||||
```
|
||||
|
||||
The port can now be accessed from `inet->inet_sport` in network byte order (aka: Big Endian). Easy! So, we could just replace the `bpf_trace_printk` with:
|
||||
|
||||
```
|
||||
bpf_trace_printk("Listening on port %d!\\n", inet->inet_sport);
|
||||
|
||||
```
|
||||
|
||||
Then run:
|
||||
|
||||
```
|
||||
ubuntu@bcc:~/dev/listen-evts$ sudo /python tcv4listen.py
|
||||
...
|
||||
R1 invalid mem access 'inv'
|
||||
...
|
||||
Exception: Failed to load BPF program kprobe__inet_listen
|
||||
|
||||
```
|
||||
|
||||
Except that it’s not (yet) so simple. Bcc is improving a _lot_ currently. While writing this post, a couple of pitfalls had already been addressed. But not yet all. This Error means the in-kernel checker could prove the memory accesses in program are correct. See the explicit cast. We need to help is a little by making the accesses more explicit. We’ll use `bpf_probe_read` trusted function to read an arbitrary memory location while guaranteeing all necessary checks are done with something like:
|
||||
|
||||
```
|
||||
// Explicit initialization. The "=0" part is needed to "give life" to the variable on the stack
|
||||
u16 lport = 0;
|
||||
|
||||
// Explicit arbitrary memory access. Read it:
|
||||
// Read into 'lport', 'sizeof(lport)' bytes from 'inet->inet_sport' memory location
|
||||
bpf_probe_read(&lport, sizeof(lport), &(inet->inet_sport));
|
||||
|
||||
```
|
||||
|
||||
Reading the bound address for IPv4 is basically the same, using `inet->inet_rcv_saddr`. If we put is all together, we should get the backlog, the port and the bound IP:
|
||||
|
||||
```
|
||||
from bcc import BPF
|
||||
|
||||
# BPF Program
|
||||
bpf_text = """
|
||||
#include <net/sock.h>
|
||||
#include <net/inet_sock.h>
|
||||
#include <bcc/proto.h>
|
||||
|
||||
// Send an event for each IPv4 listen with PID, bound address and port
|
||||
int kprobe__inet_listen(struct pt_regs *ctx, struct socket *sock, int backlog)
|
||||
{
|
||||
// Cast types. Intermediate cast not needed, kept for readability
|
||||
struct sock *sk = sock->sk;
|
||||
struct inet_sock *inet = inet_sk(sk);
|
||||
|
||||
// Working values. You *need* to initialize them to give them "life" on the stack and use them afterward
|
||||
u32 laddr = 0;
|
||||
u16 lport = 0;
|
||||
|
||||
// Pull in details. As 'inet_sk' is internally a type cast, we need to use 'bpf_probe_read'
|
||||
// read: load into 'laddr' 'sizeof(laddr)' bytes from address 'inet->inet_rcv_saddr'
|
||||
bpf_probe_read(&laddr, sizeof(laddr), &(inet->inet_rcv_saddr));
|
||||
bpf_probe_read(&lport, sizeof(lport), &(inet->inet_sport));
|
||||
|
||||
// Push event
|
||||
bpf_trace_printk("Listening on %x %d with %d pending connections\\n", ntohl(laddr), ntohs(lport), backlog);
|
||||
return 0;
|
||||
};
|
||||
"""
|
||||
|
||||
# Build and Inject BPF
|
||||
b = BPF(text=bpf_text)
|
||||
|
||||
# Print debug output
|
||||
while True:
|
||||
print b.trace_readline()
|
||||
|
||||
```
|
||||
|
||||
A test run should output something like:
|
||||
|
||||
```
|
||||
(bcc)ubuntu@bcc:~/dev/listen-evts$ sudo python tcv4listen.py
|
||||
nc-5024 [000] d... 25821.166286: : Listening on 7f000001 4242 with 1 pending connections
|
||||
|
||||
```
|
||||
|
||||
Provided that you listen on localhost. The address is displayed as hex here to avoid dealing with the IP pretty printing but that’s all wired. And that’s cool.
|
||||
|
||||
Note: you may wonder why `ntohs` and `ntohl` can be called from BPF while they are not trusted. This is because they are macros and inline functions from “.h” files and a small bug was [fixed][14]while writing this post.
|
||||
|
||||
All done, one more piece: We want to get the related container. In the context of networking, that’s means we want the network namespace. The network namespace being the building block of containers allowing them to have isolated networks.
|
||||
|
||||
### Grab the network namespace: a forced introduction to perf events
|
||||
|
||||
On the userland, the network namespace can be determined by checking the target of `/proc/PID/ns/net`. It should look like `net:[4026531957]`. The number between brackets is the inode number of the network namespace. This said, we could grab it by scrapping ‘/proc’ but this is racy, we may be dealing with short-lived processes. And races are never good. We’ll grab the inode number directly from the kernel. Fortunately, that’s an easy one:
|
||||
|
||||
```
|
||||
// Create an populate the variable
|
||||
u32 netns = 0;
|
||||
|
||||
// Read the netns inode number, like /proc does
|
||||
netns = sk->__sk_common.skc_net.net->ns.inum;
|
||||
|
||||
```
|
||||
|
||||
Easy. And it works.
|
||||
|
||||
But if you’ve read so far, you may guess there is something wrong somewhere. And there is:
|
||||
|
||||
```
|
||||
bpf_trace_printk("Listening on %x %d with %d pending connections in container %d\\n", ntohl(laddr), ntohs(lport), backlog, netns);
|
||||
|
||||
```
|
||||
|
||||
If you try to run it, you’ll get some cryptic error message:
|
||||
|
||||
```
|
||||
(bcc)ubuntu@bcc:~/dev/listen-evts$ sudo python tcv4listen.py
|
||||
error: in function kprobe__inet_listen i32 (%struct.pt_regs*, %struct.socket*, i32)
|
||||
too many args to 0x1ba9108: i64 = Constant<6>
|
||||
|
||||
```
|
||||
|
||||
What clang is trying to tell you is “Hey pal, `bpf_trace_printk` can only take 4 arguments, you’ve just used 5.“. I won’t dive into the details here, but that’s a BPF limitation. If you want to dig it, [here is a good starting point][15].
|
||||
|
||||
The only way to fix it is to… stop debugging and make it production ready. So let’s get started (and make sure run at least Linux 4.4). We’ll use perf events which supports passing arbitrary sized structures to userland. Additionally, only our reader will get it so that multiple unrelated eBPF programs can produce data concurrently without issues.
|
||||
|
||||
To use it, we need to:
|
||||
|
||||
1. define a structure
|
||||
|
||||
2. declare the event
|
||||
|
||||
3. push the event
|
||||
|
||||
4. re-declare the event on Python’s side (This step should go away in the future)
|
||||
|
||||
5. consume and format the event
|
||||
|
||||
This may seem like a lot, but it ain’t. See:
|
||||
|
||||
```
|
||||
// At the begining of the C program, declare our event
|
||||
struct listen_evt_t {
|
||||
u64 laddr;
|
||||
u64 lport;
|
||||
u64 netns;
|
||||
u64 backlog;
|
||||
};
|
||||
BPF_PERF_OUTPUT(listen_evt);
|
||||
|
||||
// In kprobe__inet_listen, replace the printk with
|
||||
struct listen_evt_t evt = {
|
||||
.laddr = ntohl(laddr),
|
||||
.lport = ntohs(lport),
|
||||
.netns = netns,
|
||||
.backlog = backlog,
|
||||
};
|
||||
listen_evt.perf_submit(ctx, &evt, sizeof(evt));
|
||||
|
||||
```
|
||||
|
||||
Python side will require a little more work, though:
|
||||
|
||||
```
|
||||
# We need ctypes to parse the event structure
|
||||
import ctypes
|
||||
|
||||
# Declare data format
|
||||
class ListenEvt(ctypes.Structure):
|
||||
_fields_ = [
|
||||
("laddr", ctypes.c_ulonglong),
|
||||
("lport", ctypes.c_ulonglong),
|
||||
("netns", ctypes.c_ulonglong),
|
||||
("backlog", ctypes.c_ulonglong),
|
||||
]
|
||||
|
||||
# Declare event printer
|
||||
def print_event(cpu, data, size):
|
||||
event = ctypes.cast(data, ctypes.POINTER(ListenEvt)).contents
|
||||
print("Listening on %x %d with %d pending connections in container %d" % (
|
||||
event.laddr,
|
||||
event.lport,
|
||||
event.backlog,
|
||||
event.netns,
|
||||
))
|
||||
|
||||
# Replace the event loop
|
||||
b["listen_evt"].open_perf_buffer(print_event)
|
||||
while True:
|
||||
b.kprobe_poll()
|
||||
|
||||
```
|
||||
|
||||
Give it a try. In this example, I have a redis running in a docker container and nc on the host:
|
||||
|
||||
```
|
||||
(bcc)ubuntu@bcc:~/dev/listen-evts$ sudo python tcv4listen.py
|
||||
Listening on 0 6379 with 128 pending connections in container 4026532165
|
||||
Listening on 0 6379 with 128 pending connections in container 4026532165
|
||||
Listening on 7f000001 6588 with 1 pending connections in container 4026531957
|
||||
|
||||
```
|
||||
|
||||
### Last word
|
||||
|
||||
Absolutely everything is now setup to use trigger events from arbitrary function calls in the kernel using eBPF, and you should have seen most of the common pitfalls I hit while learning eBPF. If you want to see the full version of this tool, along with some more tricks like IPv6 support, have a look at [https://github.com/iovisor/bcc/blob/master/tools/solisten.py][16]. It’s now an official tool, thanks to the support of the bcc team.
|
||||
|
||||
To go further, you may want to checkout Brendan Gregg’s blog, in particular [the post about eBPF maps and statistics][17]. He his one of the project’s main contributor.
|
||||
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://blog.yadutaf.fr/2016/03/30/turn-any-syscall-into-event-introducing-ebpf-kernel-probes/
|
||||
|
||||
作者:[Jean-Tiare Le Bigot ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://blog.yadutaf.fr/about
|
||||
[1]:https://blog.yadutaf.fr/tags/linux
|
||||
[2]:https://blog.yadutaf.fr/tags/tracing
|
||||
[3]:https://blog.yadutaf.fr/tags/ebpf
|
||||
[4]:https://blog.yadutaf.fr/tags/bcc
|
||||
[5]:https://en.wikipedia.org/wiki/SystemTap
|
||||
[6]:https://blog.cloudflare.com/bpf-the-forgotten-bytecode/
|
||||
[7]:https://blog.yadutaf.fr/2016/03/30/turn-any-syscall-into-event-introducing-ebpf-kernel-probes/TODO
|
||||
[8]:https://lwn.net/Articles/604043/
|
||||
[9]:http://lxr.free-electrons.com/source/kernel/bpf/verifier.c#L21
|
||||
[10]:http://events.linuxfoundation.org/sites/events/files/slides/tracing-linux-ezannoni-linuxcon-ja-2015_0.pdf
|
||||
[11]:https://github.com/iovisor/bcc
|
||||
[12]:https://github.com/iovisor/bcc/blob/master/INSTALL.md
|
||||
[13]:http://lxr.free-electrons.com/source/net/ipv4/af_inet.c#L194
|
||||
[14]:https://github.com/iovisor/bcc/pull/453
|
||||
[15]:http://lxr.free-electrons.com/source/kernel/trace/bpf_trace.c#L86
|
||||
[16]:https://github.com/iovisor/bcc/blob/master/tools/solisten.py
|
||||
[17]:http://www.brendangregg.com/blog/2015-05-15/ebpf-one-small-step.html
|
110
sources/tech/20170209 INTRODUCING DOCKER SECRETS MANAGEMENT.md
Normal file
110
sources/tech/20170209 INTRODUCING DOCKER SECRETS MANAGEMENT.md
Normal file
@ -0,0 +1,110 @@
|
||||
INTRODUCING DOCKER SECRETS MANAGEMENT
|
||||
============================================================
|
||||
|
||||
Containers are changing how we view apps and infrastructure. Whether the code inside containers is big or small, container architecture introduces a change to how that code behaves with hardware – it fundamentally abstracts it from the infrastructure. Docker believes that there are three key components to container security and together they result in inherently safer apps.
|
||||
|
||||

|
||||
|
||||
A critical element of building safer apps is having a secure way of communicating with other apps and systems, something that often requires credentials, tokens, passwords and other types of confidential information—usually referred to as application secrets. We are excited to introduce Docker Secrets, a container native solution that strengthens the Trusted Delivery component of container security by integrating secret distribution directly into the container platform.
|
||||
|
||||
With containers, applications are now dynamic and portable across multiple environments. This made existing secrets distribution solutions inadequate because they were largely designed for static environments. Unfortunately, this led to an increase in mismanagement of application secrets, making it common to find insecure, home-grown solutions, such as embedding secrets into version control systems like GitHub, or other equally bad—bolted on point solutions as an afterthought.
|
||||
|
||||
### Introducing Docker Secrets Management
|
||||
|
||||
We fundamentally believe that apps are safer if there is a standardized interface for accessing secrets. Any good solution will also have to follow security best practices, such as encrypting secrets while in transit; encrypting secrets at rest; preventing secrets from unintentionally leaking when consumed by the final application; and strictly adhere to the principle of least-privilege, where an application only has access to the secrets that it needs—no more, no less.
|
||||
|
||||
By integrating secrets into Docker orchestration, we are able to deliver a solution for the secrets management problem that follows these exact principles.
|
||||
|
||||
The following diagram provides a high-level view of how the Docker swarm mode architecture is applied to securely deliver a new type of object to our containers: a secret object.
|
||||
|
||||

|
||||
|
||||
In Docker, a secret is any blob of data, such as a password, SSH private key, TLS Certificate, or any other piece of data that is sensitive in nature. When you add a secret to the swarm (by running `docker secret create`), Docker sends the secret over to the swarm manager over a mutually authenticated TLS connection, making use of the [built-in Certificate Authority][17] that gets automatically created when bootstrapping a new swarm.
|
||||
|
||||
```
|
||||
$ echo "This is a secret" | docker secret create my_secret_data -
|
||||
```
|
||||
|
||||
Once the secret reaches a manager node, it gets saved to the internal Raft store, which uses NACL’s Salsa20Poly1305 with a 256-bit key to ensure no data is ever written to disk unencrypted. Writing to the internal store gives secrets the same high availability guarantees that the the rest of the swarm management data gets.
|
||||
|
||||
When a swarm manager starts up, the encrypted Raft logs containing the secrets is decrypted using a data encryption key that is unique per-node. This key, and the node’s TLS credentials used to communicate with the rest of the cluster, can be encrypted with a cluster-wide key encryption key, called the unlock key, which is also propagated using Raft and will be required on manager start.
|
||||
|
||||
When you grant a newly-created or running service access to a secret, one of the manager nodes (only managers have access to all the stored secrets stored) will send it over the already established TLS connection exclusively to the nodes that will be running that specific service. This means that nodes cannot request the secrets themselves, and will only gain access to the secrets when provided to them by a manager – strictly for the services that require them.
|
||||
|
||||
```
|
||||
$ docker service create --name="redis" --secret="my_secret_data" redis:alpine
|
||||
```
|
||||
|
||||
The unencrypted secret is mounted into the container in an in-memory filesystem at /run/secrets/<secret_name>.
|
||||
|
||||
```
|
||||
$ docker exec $(docker ps --filter name=redis -q) ls -l /run/secrets
|
||||
total 4
|
||||
-r--r--r-- 1 root root 17 Dec 13 22:48 my_secret_data
|
||||
```
|
||||
|
||||
If a service gets deleted, or rescheduled somewhere else, the manager will immediately notify all the nodes that no longer require access to that secret to erase it from memory, and the node will no longer have any access to that application secret.
|
||||
|
||||
```
|
||||
$ docker service update --secret-rm="my_secret_data" redis
|
||||
|
||||
$ docker exec -it $(docker ps --filter name=redis -q) cat /run/secrets/my_secret_data
|
||||
|
||||
cat: can't open '/run/secrets/my_secret_data': No such file or directory
|
||||
```
|
||||
|
||||
Check out the [Docker secrets docs][18] for more information and examples on how to create and manage your secrets. And a special shout out to Laurens Van Houtven (https://www.lvh.io/[)][19] in collaboration with the Docker security and core engineering team to help make this feature a reality.
|
||||
|
||||
[Get safer apps for dev and ops w/ new #Docker secrets management][5]
|
||||
|
||||
[CLICK TO TWEET][6]
|
||||
|
||||
###
|
||||

|
||||
|
||||
### Safer Apps with Docker
|
||||
|
||||
Docker secrets is designed to be easily usable by developers and IT ops teams to build and run safer apps. Docker secrets is a container first architecture designed to keep secrets safe and used only when needed by the exact container that needs that secret to operate. From defining apps and secrets with Docker Compose through an IT admin deploying that Compose file directly in Docker Datacenter, the services, secrets, networks and volumes will travel securely, safely with the application.
|
||||
|
||||
Resources to learn more:
|
||||
|
||||
* [Docker Datacenter on 1.13 with Secrets, Security Scanning, Content Cache and More][7]
|
||||
|
||||
* [Download Docker][8] and get started today
|
||||
|
||||
* [Try secrets in Docker Datacenter][9]
|
||||
|
||||
* [Read the Documentation][10]
|
||||
|
||||
* Attend an [upcoming webinar][11]
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://blog.docker.com/2017/02/docker-secrets-management/
|
||||
|
||||
作者:[ Ying Li][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://blog.docker.com/author/yingli/
|
||||
[1]:http://www.linkedin.com/shareArticle?mini=true&url=http://dockr.ly/2k6gnOB&title=Introducing%20Docker%20Secrets%20Management&summary=Containers%20are%20changing%20how%20we%20view%20apps%20and%20infrastructure.%20Whether%20the%20code%20inside%20containers%20is%20big%20or%20small,%20container%20architecture%20introduces%20a%20change%20to%20how%20that%20code%20behaves%20with%20hardware%20-%20it%20fundamentally%20abstracts%20it%20from%20the%20infrastructure.%20Docker%20believes%20that%20there%20are%20three%20key%20components%20to%20container%20security%20and%20...
|
||||
[2]:http://www.reddit.com/submit?url=http://dockr.ly/2k6gnOB&title=Introducing%20Docker%20Secrets%20Management
|
||||
[3]:https://plus.google.com/share?url=http://dockr.ly/2k6gnOB
|
||||
[4]:http://news.ycombinator.com/submitlink?u=http://dockr.ly/2k6gnOB&t=Introducing%20Docker%20Secrets%20Management
|
||||
[5]:https://twitter.com/share?text=Get+safer+apps+for+dev+and+ops+w%2F+new+%23Docker+secrets+management+&via=docker&related=docker&url=http://dockr.ly/2k6gnOB
|
||||
[6]:https://twitter.com/share?text=Get+safer+apps+for+dev+and+ops+w%2F+new+%23Docker+secrets+management+&via=docker&related=docker&url=http://dockr.ly/2k6gnOB
|
||||
[7]:http://dockr.ly/AppSecurity
|
||||
[8]:https://www.docker.com/getdocker
|
||||
[9]:http://www.docker.com/trial
|
||||
[10]:https://docs.docker.com/engine/swarm/secrets/
|
||||
[11]:http://www.docker.com/webinars
|
||||
[12]:https://blog.docker.com/author/yingli/
|
||||
[13]:https://blog.docker.com/tag/container-security/
|
||||
[14]:https://blog.docker.com/tag/docker-security/
|
||||
[15]:https://blog.docker.com/tag/secrets-management/
|
||||
[16]:https://blog.docker.com/tag/security/
|
||||
[17]:https://docs.docker.com/engine/swarm/how-swarm-mode-works/pki/
|
||||
[18]:https://docs.docker.com/engine/swarm/secrets/
|
||||
[19]:https://lvh.io%29/
|
@ -1,331 +0,0 @@
|
||||
zpl1025
|
||||
How to take screenshots on Linux using Scrot
|
||||
============================================================
|
||||
|
||||
### On this page
|
||||
|
||||
1. [About Scrot][12]
|
||||
2. [Scrot Installation][13]
|
||||
3. [Scrot Usage/Features][14]
|
||||
1. [Get the application version][1]
|
||||
2. [Capturing current window][2]
|
||||
3. [Selecting a window][3]
|
||||
4. [Include window border in screenshots][4]
|
||||
5. [Delay in taking screenshots][5]
|
||||
6. [Countdown before screenshot][6]
|
||||
7. [Image quality][7]
|
||||
8. [Generating thumbnails][8]
|
||||
9. [Join multiple displays shots][9]
|
||||
10. [Executing operations on saved images][10]
|
||||
11. [Special strings][11]
|
||||
4. [Conclusion][15]
|
||||
|
||||
Recently, we discussed about the [gnome-screenshot][17] utility, which is a good screen grabbing tool. But if you are looking for an even better command line utility for taking screenshots, then you must give Scrot a try. This tool has some extra features that are currently not available in gnome-screenshot. In this tutorial, we will explain Scrot using easy to understand examples.
|
||||
|
||||
Please note that all the examples mentioned in this tutorial have been tested on Ubuntu 16.04 LTS, and the scrot version we have used is 0.8.
|
||||
|
||||
### About Scrot
|
||||
|
||||
[Scrot][18] (**SCR**eensh**OT**) is a screenshot capturing utility that uses the imlib2 library to acquire and save images. Developed by Tom Gilbert, it's written in C programming language and is licensed under the BSD License.
|
||||
|
||||
### Scrot Installation
|
||||
|
||||
The scrot tool may be pre-installed on your Ubuntu system, but if that's not the case, then you can install it using the following command:
|
||||
|
||||
sudo apt-get install scrot
|
||||
|
||||
Once the tool is installed, you can launch it by using the following command:
|
||||
|
||||
scrot [options] [filename]
|
||||
|
||||
**Note**: The parameters in [] are optional.
|
||||
|
||||
### Scrot Usage/Features
|
||||
|
||||
In this section, we will discuss how the Scrot tool can be used and what all features it provides.
|
||||
|
||||
When the tool is run without any command line options, it captures the whole screen.
|
||||
|
||||
[
|
||||

|
||||
][19]
|
||||
|
||||
By default, the captured file is saved with a date-stamped filename in the current directory, although you can also explicitly specify the name of the captured image when the command is run. For example:
|
||||
|
||||
scrot [image-name].png
|
||||
|
||||
### Get the application version
|
||||
|
||||
If you want, you can check the version of scrot using the -v command line option.
|
||||
|
||||
scrot -v
|
||||
|
||||
Here is an example:
|
||||
|
||||
[
|
||||

|
||||
][20]
|
||||
|
||||
### Capturing current window
|
||||
|
||||
Using the utility, you can limit the screenshot to the currently focused window. This feature can be accessed using the -u command line option.
|
||||
|
||||
scrot -u
|
||||
|
||||
For example, here's my desktop when I executed the above command on the command line:
|
||||
|
||||
[
|
||||

|
||||
][21]
|
||||
|
||||
And here's the screenshot captured by scrot:
|
||||
|
||||
[
|
||||

|
||||
][22]
|
||||
|
||||
### Selecting a window
|
||||
|
||||
The utility allows you to capture any window by clicking on it using the mouse. This feature can be accessed using the -s option.
|
||||
|
||||
scrot -s
|
||||
|
||||
For example, as you can see in the screenshot below, I have a screen with two terminal windows overlapping each other. On the top window, I run the aforementioned command.
|
||||
|
||||
[
|
||||

|
||||
][23]
|
||||
|
||||
Now suppose, I want to capture the bottom terminal window. For that, I will just click on that window once the command is executed - the command execution won't complete until you click somewhere on the screen.
|
||||
|
||||
Here's the screenshot captured after clicking on that terminal:
|
||||
|
||||
[
|
||||

|
||||
][24]
|
||||
|
||||
**Note**: As you can see in the above snapshot, whatever area the bottom window is covering has been captured, even if that includes an overlapping portion of the top window.
|
||||
|
||||
### Include window border in screenshots
|
||||
|
||||
The -u command line option we discussed earlier doesn't include the window border in screenshots. However, you can include the border of the window if you want. This feature can be accessed using the -b option (in conjunction with the -u option of course).
|
||||
|
||||
scrot -ub
|
||||
|
||||
Here is an example screenshot:
|
||||
|
||||
[
|
||||

|
||||
][25]
|
||||
|
||||
**Note**: Including window border also adds some of the background area to the screenshot.
|
||||
|
||||
### Delay in taking screenshots
|
||||
|
||||
You can introduce a time delay while taking screenshots. For this, you have to assign a numeric value to the --delay or -d command line option.
|
||||
|
||||
scrot --delay [NUM]
|
||||
|
||||
scrot --delay 5
|
||||
|
||||
Here is an example:
|
||||
|
||||
[
|
||||

|
||||
][26]
|
||||
|
||||
In this case, scrot will wait for 5 seconds and then take the screenshot.
|
||||
|
||||
### Countdown before screenshot
|
||||
|
||||
The tool also allows you to display countdown while using delay option. This feature can be accessed using the -c command line option.
|
||||
|
||||
scrot –delay [NUM] -c
|
||||
|
||||
scrot -d 5 -c
|
||||
|
||||
Here is an example screenshot:
|
||||
|
||||
[
|
||||

|
||||
][27]
|
||||
|
||||
### Image quality
|
||||
|
||||
Using the tool, you can adjust the quality of the screenshot image at the scale of 1-100\. High value means high size and low compression. Default value is 75, although effect differs depending on the file format chosen.
|
||||
|
||||
This feature can be accessed using --quality or -q option, but you have to assign a numeric value to this option ranging from 1-100.
|
||||
|
||||
scrot –quality [NUM]
|
||||
|
||||
scrot –quality 10
|
||||
|
||||
Here is an example snapshot:
|
||||
|
||||
[
|
||||

|
||||
][28]
|
||||
|
||||
So you can see that the quality of the image degrades a lot as the -q option is assigned value closer to 1.
|
||||
|
||||
### Generating thumbnails
|
||||
|
||||
The scrot utility also allows you to generate thumbnail of the screenshot. This feature can be accessed using the --thumb option. This option requires a NUM value, which is basically the percentage of the original screenshot size.
|
||||
|
||||
scrot --thumb NUM
|
||||
|
||||
scrot --thumb 50
|
||||
|
||||
**Note**: The --thumb option makes sure that the screenshot is captured and saved in original size as well.
|
||||
|
||||
For example, here is the original screenshot captured in my case:
|
||||
|
||||
[
|
||||

|
||||
][29]
|
||||
|
||||
And following is the thumbnail saved:
|
||||
|
||||
[
|
||||

|
||||
][30]
|
||||
|
||||
### Join multiple displays shots
|
||||
|
||||
In case your machine has multiple displays attached to it, scrot allows you to grab and join screenshots of these displays. This feature can be accessed using the -m command line option.
|
||||
|
||||
scrot -m
|
||||
|
||||
Here is an example snapshot:
|
||||
|
||||
[
|
||||

|
||||
][31]
|
||||
|
||||
### Executing operations on saved images
|
||||
|
||||
Using the tool, we can execute various operations on saved images - for example, open the screenshot in an image editor like gThumb. This feature can be accessed using the -e command line option. Here's an example:
|
||||
|
||||
scrot abc.png -e ‘gthumb abc.png’
|
||||
|
||||
Here, gthumb is an image editor which will automatically launch after we run the command.
|
||||
|
||||
Following is the snapshot of the command:
|
||||
|
||||
[
|
||||

|
||||
][32]
|
||||
|
||||
And here is the output of the above command:
|
||||
|
||||
[
|
||||

|
||||
][33]
|
||||
|
||||
So you can see that the scrot command grabbed the screenshot and then launched the gThumb image editor with the captured image as argument.
|
||||
|
||||
If you don’t specify a filename to your screenshot, then the snapshot will be saved with a date-stamped filename in your current directory - this, as we've already mentioned in the beginning, is the default behaviour of scrot.
|
||||
|
||||
Here's an -e command line option example where scrot uses the default name for the screenshot:
|
||||
|
||||
scrot -e ‘gthumb $n’
|
||||
|
||||
[
|
||||

|
||||
][34]
|
||||
|
||||
It's worth mentioning that $n is a special string, which provides access to the screenshot name. For more details on special strings, head to the next section.
|
||||
|
||||
### Special strings
|
||||
|
||||
The -e (or the --exec ) and filename parameters can take format specifiers when used with scrot. There are two types of format specifiers. First type is characters preceded by ‘%’ that are used for date and time formats, while the second type is internal to scrot and are prefixed by ‘$’
|
||||
|
||||
Several specifiers which are recognised by the --exec and filename parameters are discussed below.
|
||||
|
||||
**$f** – provides access to screenshot path (including filename).
|
||||
|
||||
For example,
|
||||
|
||||
scrot ashu.jpg -e ‘mv $f ~/Pictures/Scrot/ashish/’
|
||||
|
||||
Here is an example snapshot:
|
||||
|
||||
[
|
||||

|
||||
][35]
|
||||
|
||||
If you will not specify a filename, then scrot will by-default save the snapshot in a date stamped file format. This is the by-default date-stamped file format used in scrot : %yy-%mm-%dd-%hhmmss_$wx$h_scrot.png.
|
||||
|
||||
**$n** – provides snapshot name. Here is an example snapshot:
|
||||
|
||||
[
|
||||

|
||||
][36]
|
||||
|
||||
**$s** – gives access to the size of screenshot. This feature, for example, can be accessed in the following way.
|
||||
|
||||
scrot abc.jpg -e ‘echo $s’
|
||||
|
||||
Here is an example snapshot
|
||||
|
||||
[
|
||||

|
||||
][37]
|
||||
|
||||
Similarly, you can use the other special strings **$p**, **$w**, **$h**, **$t**, **$$** and **\n** that provide access to image pixel size, image width, image height, image format, $ symbol, and give access to new line respectively. You can, for example, use these strings in the way similar to the **$s** example we have discussed above.
|
||||
|
||||
### Conclusion
|
||||
|
||||
The utility is easy to install on Ubuntu systems, which is good for beginners. Scrot also provides some advanced features such as special strings that can be used in scripting by professionals. Needless to say, there is a slight learning curve associated in case you want to use them.
|
||||
|
||||

|
||||
[vie][16]
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/
|
||||
|
||||
作者:[Himanshu Arora][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/
|
||||
[1]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#get-the-applicationnbspversion
|
||||
[2]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#capturing-current-window
|
||||
[3]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#selecting-a-window
|
||||
[4]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#includenbspwindow-border-in-screenshots
|
||||
[5]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#delay-in-taking-screenshots
|
||||
[6]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#countdown-before-screenshot
|
||||
[7]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#image-quality
|
||||
[8]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#generating-thumbnails
|
||||
[9]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#join-multiple-displays-shots
|
||||
[10]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#executing-operations-on-saved-images
|
||||
[11]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#special-strings
|
||||
[12]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#about-scrot
|
||||
[13]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#scrot-installation
|
||||
[14]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#scrot-usagefeatures
|
||||
[15]:https://www.howtoforge.com/tutorial/how-to-take-screenshots-in-linux-with-scrot/#conclusion
|
||||
[16]:https://www.howtoforge.com/subscription/
|
||||
[17]:https://www.howtoforge.com/tutorial/taking-screenshots-in-linux-using-gnome-screenshot/
|
||||
[18]:https://en.wikipedia.org/wiki/Scrot
|
||||
[19]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/scrot.png
|
||||
[20]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/version.png
|
||||
[21]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/desktop.png
|
||||
[22]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/active.png
|
||||
[23]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/select1.png
|
||||
[24]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/select2.png
|
||||
[25]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/border-new.png
|
||||
[26]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/delay.png
|
||||
[27]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/countdown.png
|
||||
[28]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/img-quality.jpg
|
||||
[29]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/orig.png
|
||||
[30]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/thmb.png
|
||||
[31]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/multiple.png
|
||||
[32]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/exec1.png
|
||||
[33]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/exec2.png
|
||||
[34]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/exec3.png
|
||||
[35]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/f.png
|
||||
[36]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/n.png
|
||||
[37]:https://www.howtoforge.com/images/how-to-take-screenshots-in-linux-with-scrot/big/s.png
|
@ -1,108 +0,0 @@
|
||||
Translating by aiwhj
|
||||
# How to Improve a Legacy Codebase
|
||||
|
||||
|
||||
It happens at least once in the lifetime of every programmer, project manager or teamleader. You get handed a steaming pile of manure, if you’re lucky only a few million lines worth, the original programmers have long ago left for sunnier places and the documentation - if there is any to begin with - is hopelessly out of sync with what is presently keeping the company afloat.
|
||||
|
||||
Your job: get us out of this mess.
|
||||
|
||||
After your first instinctive response (run for the hills) has passed you start on the project knowing full well that the eyes of the company senior leadership are on you. Failure is not an option. And yet, by the looks of what you’ve been given failure is very much in the cards. So what to do?
|
||||
|
||||
I’ve been (un)fortunate enough to be in this situation several times and me and a small band of friends have found that it is a lucrative business to be able to take these steaming piles of misery and to turn them into healthy maintainable projects. Here are some of the tricks that we employ:
|
||||
|
||||
### Backup
|
||||
|
||||
Before you start to do anything at all make a backup of _everything_ that might be relevant. This to make sure that no information is lost that might be of crucial importance somewhere down the line. All it takes is a silly question that you can’t answer to eat up a day or more once the change has been made. Especially configuration data is susceptible to this kind of problem, it is usually not versioned and you’re lucky if it is taken along in the periodic back-up scheme. So better safe than sorry, copy everything to a very safe place and never ever touch that unless it is in read-only mode.
|
||||
|
||||
### Important pre-requisite, make sure you have a build process and that it actually produces what runs in production
|
||||
|
||||
I totally missed this step on the assumption that it is obvious and likely already in place but many HN commenters pointed this out and they are absolutely right: step one is to make sure that you know what is running in production right now and that means that you need to be able to build a version of the software that is - if your platform works that way - byte-for-byte identical with the current production build. If you can’t find a way to achieve this then likely you will be in for some unpleasant surprises once you commit something to production. Make sure you test this to the best of your ability to make sure that you have all the pieces in place and then, after you’ve gained sufficient confidence that it will work move it to production. Be prepared to switch back immediately to whatever was running before and make sure that you log everything and anything that might come in handy during the - inevitable - post mortem.
|
||||
|
||||
### Freeze the DB
|
||||
|
||||
If at all possible freeze the database schema until you are done with the first level of improvements, by the time you have a solid understanding of the codebase and the legacy code has been fully left behind you are ready to modify the database schema. Change it any earlier than that and you may have a real problem on your hand, now you’ve lost the ability to run an old and a new codebase side-by-side with the database as the steady foundation to build on. Keeping the DB totally unchanged allows you to compare the effect your new business logic code has compared to the old business logic code, if it all works as advertised there should be no differences.
|
||||
|
||||
### Write your tests
|
||||
|
||||
Before you make any changes at all write as many end-to-end and integration tests as you can. Make sure these tests produce the right output and test any and all assumptions that you can come up with about how you _think_ the old stuff works (be prepared for surprises here). These tests will have two important functions: they will help to clear up any misconceptions at a very early stage and they will function as guardrails once you start writing new code to replace old code.
|
||||
|
||||
Automate all your testing, if you’re already experienced with CI then use it and make sure your tests run fast enough to run the full set of tests after every commit.
|
||||
|
||||
### Instrumentation and logging
|
||||
|
||||
If the old platform is still available for development add instrumentation. Do this in a completely new database table, add a simple counter for every event that you can think of and add a single function to increment these counters based on the name of the event. That way you can implement a time-stamped event log with a few extra lines of code and you’ll get a good idea of how many events of one kind lead to events of another kind. One example: User opens app, User closes app. If two events should result in some back-end calls those two counters should over the long term remain at a constant difference, the difference is the number of apps currently open. If you see many more app opens than app closes you know there has to be a way in which apps end (for instance a crash). For each and every event you’ll find there is some kind of relationship to other events, usually you will strive for constant relationships unless there is an obvious error somewhere in the system. You’ll aim to reduce those counters that indicate errors and you’ll aim to maximize counters further down in the chain to the level indicated by the counters at the beginning. (For instance: customers attempting to pay should result in an equal number of actual payments received).
|
||||
|
||||
This very simple trick turns every backend application into a bookkeeping system of sorts and just like with a real bookkeeping system the numbers have to match, as long as they don’t you have a problem somewhere.
|
||||
|
||||
This system will over time become invaluable in establishing the health of the system and will be a great companion next to the source code control system revision log where you can determine the point in time that a bug was introduced and what the effect was on the various counters.
|
||||
|
||||
I usually keep these counters at a 5 minute resolution (so 12 buckets for an hour), but if you have an application that generates fewer or more events then you might decide to change the interval at which new buckets are created. All counters share the same database table and so each counter is simply a column in that table.
|
||||
|
||||
### Change only one thing at the time
|
||||
|
||||
Do not fall into the trap of improving both the maintainability of the code or the platform it runs on at the same time as adding new features or fixing bugs. This will cause you huge headaches because you now have to ask yourself every step of the way what the desired outcome is of an action and will invalidate some of the tests you made earlier.
|
||||
|
||||
### Platform changes
|
||||
|
||||
If you’ve decided to migrate the application to another platform then do this first _but keep everything else exactly the same_ . If you want you can add more documentation or tests, but no more than that, all business logic and interdependencies should remain as before.
|
||||
|
||||
### Architecture changes
|
||||
|
||||
The next thing to tackle is to change the architecture of the application (if desired). At this point in time you are free to change the higher level structure of the code, usually by reducing the number of horizontal links between modules, and thus reducing the scope of the code active during any one interaction with the end-user. If the old code was monolithic in nature now would be a good time to make it more modular, break up large functions into smaller ones but leave names of variables and data-structures as they were.
|
||||
|
||||
HN user [mannykannot][1] points - rightfully - out that this is not always an option, if you’re particularly unlucky then you may have to dig in deep in order to be able to make any architecture changes. I agree with that and I should have included it here so hence this little update. What I would further like to add is if you do both do high level changes and low level changes at least try to limit them to one file or worst case one subsystem so that you limit the scope of your changes as much as possible. Otherwise you might have a very hard time debugging the change you just made.
|
||||
|
||||
### Low level refactoring
|
||||
|
||||
By now you should have a very good understanding of what each module does and you are ready for the real work: refactoring the code to improve maintainability and to make the code ready for new functionality. This will likely be the part of the project that consumes the most time, document as you go, do not make changes to a module until you have thoroughly documented it and feel you understand it. Feel free to rename variables and functions as well as datastructures to improve clarity and consistency, add tests (also unit tests, if the situation warrants them).
|
||||
|
||||
### Fix bugs
|
||||
|
||||
Now you’re ready to take on actual end-user visible changes, the first order of battle will be the long list of bugs that have accumulated over the years in the ticket queue. As usual, first confirm the problem still exists, write a test to that effect and then fix the bug, your CI and the end-to-end tests written should keep you safe from any mistakes you make due to a lack of understanding or some peripheral issue.
|
||||
|
||||
### Database Upgrade
|
||||
|
||||
If required after all this is done and you are on a solid and maintainable codebase again you have the option to change the database schema or to replace the database with a different make/model altogether if that is what you had planned to do. All the work you’ve done up to this point will help to assist you in making that change in a responsible manner without any surprises, you can completely test the new DB with the new code and all the tests in place to make sure your migration goes off without a hitch.
|
||||
|
||||
### Execute on the roadmap
|
||||
|
||||
Congratulations, you are out of the woods and are now ready to implement new functionality.
|
||||
|
||||
### Do not ever even attempt a big-bang rewrite
|
||||
|
||||
A big-bang rewrite is the kind of project that is pretty much guaranteed to fail. For one, you are in uncharted territory to begin with so how would you even know what to build, for another, you are pushing _all_ the problems to the very last day, the day just before you go ‘live’ with your new system. And that’s when you’ll fail, miserably. Business logic assumptions will turn out to be faulty, suddenly you’ll gain insight into why that old system did certain things the way it did and in general you’ll end up realizing that the guys that put the old system together weren’t maybe idiots after all. If you really do want to wreck the company (and your own reputation to boot) by all means, do a big-bang rewrite, but if you’re smart about it this is not even on the table as an option.
|
||||
|
||||
### So, the alternative, work incrementally
|
||||
|
||||
To untangle one of these hairballs the quickest path to safety is to take any element of the code that you do understand (it could be a peripheral bit, but it might also be some core module) and try to incrementally improve it still within the old context. If the old build tools are no longer available you will have to use some tricks (see below) but at least try to leave as much of what is known to work alive while you start with your changes. That way as the codebase improves so does your understanding of what it actually does. A typical commit should be at most a couple of lines.
|
||||
|
||||
### Release!
|
||||
|
||||
Every change along the way gets released into production, even if the changes are not end-user visible it is important to make the smallest possible steps because as long as you lack understanding of the system there is a fair chance that only the production environment will tell you there is a problem. If that problem arises right after you make a small change you will gain several advantages:
|
||||
|
||||
* it will probably be trivial to figure out what went wrong
|
||||
|
||||
* you will be in an excellent position to improve the process
|
||||
|
||||
* and you should immediately update the documentation to show the new insights gained
|
||||
|
||||
### Use proxies to your advantage
|
||||
|
||||
If you are doing web development praise the gods and insert a proxy between the end-users and the old system. Now you have per-url control over which requests go to the old system and which you will re-route to the new system allowing much easier and more granular control over what is run and who gets to see it. If your proxy is clever enough you could probably use it to send a percentage of the traffic to the new system for an individual URL until you are satisfied that things work the way they should. If your integration tests also connect to this interface it is even better.
|
||||
|
||||
### Yes, but all this will take too much time!
|
||||
|
||||
Well, that depends on how you look at it. It’s true there is a bit of re-work involved in following these steps. But it _does_ work, and any kind of optimization of this process makes the assumption that you know more about the system than you probably do. I’ve got a reputation to maintain and I _really_ do not like negative surprises during work like this. With some luck the company is already on the skids, or maybe there is a real danger of messing things up for the customers. In a situation like that I prefer total control and an iron clad process over saving a couple of days or weeks if that imperils a good outcome. If you’re more into cowboy stuff - and your bosses agree - then maybe it would be acceptable to take more risk, but most companies would rather take the slightly slower but much more sure road to victory.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://jacquesmattheij.com/improving-a-legacy-codebase
|
||||
|
||||
作者:[Jacques Mattheij ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://jacquesmattheij.com/
|
||||
[1]:https://news.ycombinator.com/item?id=14445661
|
@ -0,0 +1,91 @@
|
||||
Why Car Companies Are Hiring Computer Security Experts
|
||||
============================================================
|
||||
|
||||
Photo
|
||||

|
||||
The cybersecurity experts Marc Rogers, left, of CloudFlare and Kevin Mahaffey of Lookout were able to control various Tesla functions from their physically connected laptop. They pose in CloudFlare’s lobby in front of Lava Lamps used to generate numbers for encryption.CreditChristie Hemm Klok for The New York Times
|
||||
|
||||
It started about seven years ago. Iran’s top nuclear scientists were being assassinated in a string of similar attacks: Assailants on motorcycles were pulling up to their moving cars, attaching magnetic bombs and detonating them after the motorcyclists had fled the scene.
|
||||
|
||||
In another seven years, security experts warn, assassins won’t need motorcycles or magnetic bombs. All they’ll need is a laptop and code to send driverless cars careering off a bridge, colliding with a driverless truck or coming to an unexpected stop in the middle of fast-moving traffic.
|
||||
|
||||
Automakers may call them self-driving cars. But hackers call them computers that travel over 100 miles an hour.
|
||||
|
||||
“These are no longer cars,” said Marc Rogers, the principal security researcher at the cybersecurity firm CloudFlare. “These are data centers on wheels. Any part of the car that talks to the outside world is a potential inroad for attackers.”
|
||||
|
||||
Those fears came into focus two years ago when two “white hat” hackers — researchers who look for computer vulnerabilities to spot problems and fix them, rather than to commit a crime or cause problems — successfully gained access to a Jeep Cherokee from their computer miles away. They rendered their crash-test dummy (in this case a nervous reporter) powerless over his vehicle and disabling his transmission in the middle of a highway.
|
||||
|
||||
The hackers, Chris Valasek and Charlie Miller (now security researchers respectively at Uber and Didi, an Uber competitor in China), discovered an [electronic route from the Jeep’s entertainment system to its dashboard][10]. From there, they had control of the vehicle’s steering, brakes and transmission — everything they needed to paralyze their crash test dummy in the middle of a highway.
|
||||
|
||||
“Car hacking makes great headlines, but remember: No one has ever had their car hacked by a bad guy,” Mr. Miller wrote on Twitter last Sunday. “It’s only ever been performed by researchers.”
|
||||
|
||||
Still, the research by Mr. Miller and Mr. Valasek came at a steep price for Jeep’s manufacturer, Fiat Chrysler, which was forced to recall 1.4 million of its vehicles as a result of the hacking experiment.
|
||||
|
||||
It is no wonder that Mary Barra, the chief executive of General Motors, called cybersecurity her company’s top priority last year. Now the skills of researchers and so-called white hat hackers are in high demand among automakers and tech companies pushing ahead with driverless car projects.
|
||||
|
||||
Uber, [Tesla][11], Apple and Didi in China have been actively recruiting white hat hackers like Mr. Miller and Mr. Valasek from one another as well as from traditional cybersecurity firms and academia.
|
||||
|
||||
Last year, Tesla poached Aaron Sigel, Apple’s manager of security for its iOS operating system. Uber poached Chris Gates, formerly a white hat hacker at Facebook. Didi poached Mr. Miller from Uber, where he had gone to work after the Jeep hack. And security firms have seen dozens of engineers leave their ranks for autonomous-car projects.
|
||||
|
||||
Mr. Miller said he left Uber for Didi, in part, because his new Chinese employer has given him more freedom to discuss his work.
|
||||
|
||||
“Carmakers seem to be taking the threat of cyberattack more seriously, but I’d still like to see more transparency from them,” Mr. Miller wrote on Twitter on Saturday.
|
||||
|
||||
Like a number of big tech companies, Tesla and Fiat Chrysler started paying out rewards to hackers who turn over flaws the hackers discover in their systems. GM has done something similar, though critics say GM’s program is limited when compared with the ones offered by tech companies, and so far no rewards have been paid out.
|
||||
|
||||
One year after the Jeep hack by Mr. Miller and Mr. Valasek, they demonstrated all the other ways they could mess with a Jeep driver, including hijacking the vehicle’s cruise control, swerving the steering wheel 180 degrees or slamming on the parking brake in high-speed traffic — all from a computer in the back of the car. (Those exploits ended with their test Jeep in a ditch and calls to a local tow company.)
|
||||
|
||||
Granted, they had to be in the Jeep to make all that happen. But it was evidence of what is possible.
|
||||
|
||||
The Jeep penetration was preceded by a [2011 hack by security researchers at the University of Washington][12] and the University of California, San Diego, who were the first to remotely hack a sedan and ultimately control its brakes via Bluetooth. The researchers warned car companies that the more connected cars become, the more likely they are to get hacked.
|
||||
|
||||
Security researchers have also had their way with Tesla’s software-heavy Model S car. In 2015, Mr. Rogers, together with Kevin Mahaffey, the chief technology officer of the cybersecurity company Lookout, found a way to control various Tesla functions from their physically connected laptop.
|
||||
|
||||
One year later, a team of Chinese researchers at Tencent took their research a step further, hacking a moving Tesla Model S and controlling its brakes from 12 miles away. Unlike Chrysler, Tesla was able to dispatch a remote patch to fix the security holes that made the hacks possible.
|
||||
|
||||
In all the cases, the car hacks were the work of well meaning, white hat security researchers. But the lesson for all automakers was clear.
|
||||
|
||||
The motivations to hack vehicles are limitless. When it learned of Mr. Rogers’s and Mr. Mahaffey’s investigation into Tesla’s Model S, a Chinese app-maker asked Mr. Rogers if he would be interested in sharing, or possibly selling, his discovery, he said. (The app maker was looking for a backdoor to secretly install its app on Tesla’s dashboard.)
|
||||
|
||||
Criminals have not yet shown they have found back doors into connected vehicles, though for years, they have been actively developing, trading and deploying tools that can intercept car key communications.
|
||||
|
||||
But as more driverless and semiautonomous cars hit the open roads, they will become a more worthy target. Security experts warn that driverless cars present a far more complex, intriguing and vulnerable “attack surface” for hackers. Each new “connected” car feature introduces greater complexity, and with complexity inevitably comes vulnerability.
|
||||
|
||||
Twenty years ago, cars had, on average, one million lines of code. The General Motors 2010 [Chevrolet Volt][13] had about 10 million lines of code — more than an [F-35 fighter jet][14].
|
||||
|
||||
Today, an average car has more than 100 million lines of code. Automakers predict it won’t be long before they have 200 million. When you stop to consider that, on average, there are 15 to 50 defects per 1,000 lines of software code, the potentially exploitable weaknesses add up quickly.
|
||||
|
||||
The only difference between computer code and driverless car code is that, “Unlike data center enterprise security — where the biggest threat is loss of data — in automotive security, it’s loss of life,” said David Barzilai, a co-founder of Karamba Security, an Israeli start-up that is working on addressing automotive security.
|
||||
|
||||
To truly secure autonomous vehicles, security experts say, automakers will have to address the inevitable vulnerabilities that pop up in new sensors and car computers, address inherent vulnerabilities in the base car itself and, perhaps most challenging of all, bridge the cultural divide between automakers and software companies.
|
||||
|
||||
“The genie is out of the bottle, and to solve this problem will require a major cultural shift,” said Mr. Mahaffey of the cybersecurity company Lookout. “And an automaker that truly values cybersecurity will treat security vulnerabilities the same they would an airbag recall. We have not seen that industrywide shift yet.”
|
||||
|
||||
There will be winners and losers, Mr. Mahaffey added: “Automakers that transform themselves into software companies will win. Others will get left behind.”
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.nytimes.com/2017/06/07/technology/why-car-companies-are-hiring-computer-security-experts.html
|
||||
|
||||
作者:[NICOLE PERLROTH ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.nytimes.com/by/nicole-perlroth
|
||||
[1]:https://www.nytimes.com/2016/06/09/technology/software-as-weaponry-in-a-computer-connected-world.html
|
||||
[2]:https://www.nytimes.com/2015/08/29/technology/uber-hires-two-engineers-who-showed-cars-could-be-hacked.html
|
||||
[3]:https://www.nytimes.com/2015/08/11/opinion/zeynep-tufekci-why-smart-objects-may-be-a-dumb-idea.html
|
||||
[4]:https://www.nytimes.com/by/nicole-perlroth
|
||||
[5]:https://www.nytimes.com/column/bits
|
||||
[6]:https://www.nytimes.com/2017/06/07/technology/why-car-companies-are-hiring-computer-security-experts.html?utm_source=wanqu.co&utm_campaign=Wanqu+Daily&utm_medium=website#story-continues-1
|
||||
[7]:http://www.nytimes.com/newsletters/sample/bits?pgtype=subscriptionspage&version=business&contentId=TU&eventName=sample&module=newsletter-sign-up
|
||||
[8]:https://www.nytimes.com/privacy
|
||||
[9]:https://www.nytimes.com/help/index.html
|
||||
[10]:https://bits.blogs.nytimes.com/2015/07/21/security-researchers-find-a-way-to-hack-cars/
|
||||
[11]:http://www.nytimes.com/topic/company/tesla-motors-inc?inline=nyt-org
|
||||
[12]:http://www.autosec.org/pubs/cars-usenixsec2011.pdf
|
||||
[13]:http://autos.nytimes.com/2011/Chevrolet/Volt/238/4117/329463/researchOverview.aspx?inline=nyt-classifier
|
||||
[14]:http://topics.nytimes.com/top/reference/timestopics/subjects/m/military_aircraft/f35_airplane/index.html?inline=nyt-classifier
|
||||
[15]:https://www.nytimes.com/2017/06/07/technology/why-car-companies-are-hiring-computer-security-experts.html?utm_source=wanqu.co&utm_campaign=Wanqu+Daily&utm_medium=website#story-continues-3
|
@ -1,314 +0,0 @@
|
||||
Translating by yongshouzhang
|
||||
|
||||
|
||||
A user's guide to links in the Linux filesystem
|
||||
============================================================
|
||||
|
||||
### Learn how to use links, which make tasks easier by providing access to files from multiple locations in the Linux filesystem directory tree.
|
||||
|
||||
|
||||

|
||||
Image by : [Paul Lewin][8]. Modified by Opensource.com. [CC BY-SA 2.0][9]
|
||||
|
||||
In articles I have written about various aspects of Linux filesystems for Opensource.com, including [An introduction to Linux's EXT4 filesystem][10]; [Managing devices in Linux][11]; [An introduction to Linux filesystems][12]; and [A Linux user's guide to Logical Volume Management][13], I have briefly mentioned an interesting feature of Linux filesystems that can make some tasks easier by providing access to files from multiple locations in the filesystem directory tree.
|
||||
|
||||
There are two types of Linux filesystem links: hard and soft. The difference between the two types of links is significant, but both types are used to solve similar problems. They both provide multiple directory entries (or references) to a single file, but they do it quite differently. Links are powerful and add flexibility to Linux filesystems because [everything is a file][14].
|
||||
|
||||
More Linux resources
|
||||
|
||||
* [What is Linux?][1]
|
||||
|
||||
* [What are Linux containers?][2]
|
||||
|
||||
* [Download Now: Linux commands cheat sheet][3]
|
||||
|
||||
* [Advanced Linux commands cheat sheet][4]
|
||||
|
||||
* [Our latest Linux articles][5]
|
||||
|
||||
I have found, for instance, that some programs required a particular version of a library. When a library upgrade replaced the old version, the program would crash with an error specifying the name of the old, now-missing library. Usually, the only change in the library name was the version number. Acting on a hunch, I simply added a link to the new library but named the link after the old library name. I tried the program again and it worked perfectly. And, okay, the program was a game, and everyone knows the lengths that gamers will go to in order to keep their games running.
|
||||
|
||||
In fact, almost all applications are linked to libraries using a generic name with only a major version number in the link name, while the link points to the actual library file that also has a minor version number. In other instances, required files have been moved from one directory to another to comply with the Linux file specification, and there are links in the old directories for backwards compatibility with those programs that have not yet caught up with the new locations. If you do a long listing of the **/lib64** directory, you can find many examples of both.
|
||||
|
||||
```
|
||||
lrwxrwxrwx. 1 root root 36 Dec 8 2016 cracklib_dict.hwm -> ../../usr/share/cracklib/pw_dict.hwm
|
||||
lrwxrwxrwx. 1 root root 36 Dec 8 2016 cracklib_dict.pwd -> ../../usr/share/cracklib/pw_dict.pwd
|
||||
lrwxrwxrwx. 1 root root 36 Dec 8 2016 cracklib_dict.pwi -> ../../usr/share/cracklib/pw_dict.pwi
|
||||
lrwxrwxrwx. 1 root root 27 Jun 9 2016 libaccountsservice.so.0 -> libaccountsservice.so.0.0.0
|
||||
-rwxr-xr-x. 1 root root 288456 Jun 9 2016 libaccountsservice.so.0.0.0
|
||||
lrwxrwxrwx 1 root root 15 May 17 11:47 libacl.so.1 -> libacl.so.1.1.0
|
||||
-rwxr-xr-x 1 root root 36472 May 17 11:47 libacl.so.1.1.0
|
||||
lrwxrwxrwx. 1 root root 15 Feb 4 2016 libaio.so.1 -> libaio.so.1.0.1
|
||||
-rwxr-xr-x. 1 root root 6224 Feb 4 2016 libaio.so.1.0.0
|
||||
-rwxr-xr-x. 1 root root 6224 Feb 4 2016 libaio.so.1.0.1
|
||||
lrwxrwxrwx. 1 root root 30 Jan 16 16:39 libakonadi-calendar.so.4 -> libakonadi-calendar.so.4.14.26
|
||||
-rwxr-xr-x. 1 root root 816160 Jan 16 16:39 libakonadi-calendar.so.4.14.26
|
||||
lrwxrwxrwx. 1 root root 29 Jan 16 16:39 libakonadi-contact.so.4 -> libakonadi-contact.so.4.14.26
|
||||
```
|
||||
|
||||
A few of the links in the **/lib64** directory
|
||||
|
||||
The long listing of the **/lib64** directory above shows that the first character in the filemode is the letter "l," which means that each is a soft or symbolic link.
|
||||
|
||||
### Hard links
|
||||
|
||||
In [An introduction to Linux's EXT4 filesystem][15], I discussed the fact that each file has one inode that contains information about that file, including the location of the data belonging to that file. [Figure 2][16] in that article shows a single directory entry that points to the inode. Every file must have at least one directory entry that points to the inode that describes the file. The directory entry is a hard link, thus every file has at least one hard link.
|
||||
|
||||
In Figure 1 below, multiple directory entries point to a single inode. These are all hard links. I have abbreviated the locations of three of the directory entries using the tilde (**~**) convention for the home directory, so that **~** is equivalent to **/home/user** in this example. Note that the fourth directory entry is in a completely different directory, **/home/shared**, which might be a location for sharing files between users of the computer.
|
||||
|
||||

|
||||
Figure 1
|
||||
|
||||
Hard links are limited to files contained within a single filesystem. "Filesystem" is used here in the sense of a partition or logical volume (LV) that is mounted on a specified mount point, in this case **/home**. This is because inode numbers are unique only within each filesystem, and a different filesystem, for example, **/var**or **/opt**, will have inodes with the same number as the inode for our file.
|
||||
|
||||
Because all the hard links point to the single inode that contains the metadata about the file, all of these attributes are part of the file, such as ownerships, permissions, and the total number of hard links to the inode, and cannot be different for each hard link. It is one file with one set of attributes. The only attribute that can be different is the file name, which is not contained in the inode. Hard links to a single **file/inode** located in the same directory must have different names, due to the fact that there can be no duplicate file names within a single directory.
|
||||
|
||||
The number of hard links for a file is displayed with the **ls -l** command. If you want to display the actual inode numbers, the command **ls -li** does that.
|
||||
|
||||
### Symbolic (soft) links
|
||||
|
||||
The difference between a hard link and a soft link, also known as a symbolic link (or symlink), is that, while hard links point directly to the inode belonging to the file, soft links point to a directory entry, i.e., one of the hard links. Because soft links point to a hard link for the file and not the inode, they are not dependent upon the inode number and can work across filesystems, spanning partitions and LVs.
|
||||
|
||||
The downside to this is: If the hard link to which the symlink points is deleted or renamed, the symlink is broken. The symlink is still there, but it points to a hard link that no longer exists. Fortunately, the **ls** command highlights broken links with flashing white text on a red background in a long listing.
|
||||
|
||||
### Lab project: experimenting with links
|
||||
|
||||
I think the easiest way to understand the use of and differences between hard and soft links is with a lab project that you can do. This project should be done in an empty directory as a _non-root user_ . I created the **~/temp** directory for this project, and you should, too. It creates a safe place to do the project and provides a new, empty directory to work in so that only files associated with this project will be located there.
|
||||
|
||||
### **Initial setup**
|
||||
|
||||
First, create the temporary directory in which you will perform the tasks needed for this project. Ensure that the present working directory (PWD) is your home directory, then enter the following command.
|
||||
|
||||
```
|
||||
mkdir temp
|
||||
```
|
||||
|
||||
Change into **~/temp** to make it the PWD with this command.
|
||||
|
||||
```
|
||||
cd temp
|
||||
```
|
||||
|
||||
To get started, we need to create a file we can link to. The following command does that and provides some content as well.
|
||||
|
||||
```
|
||||
du -h > main.file.txt
|
||||
```
|
||||
|
||||
Use the **ls -l** long list to verify that the file was created correctly. It should look similar to my results. Note that the file size is only 7 bytes, but yours may vary by a byte or two.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ls -l
|
||||
total 4
|
||||
-rw-rw-r-- 1 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
Notice the number "1" following the file mode in the listing. That number represents the number of hard links that exist for the file. For now, it should be 1 because we have not created any additional links to our test file.
|
||||
|
||||
### **Experimenting with hard links**
|
||||
|
||||
Hard links create a new directory entry pointing to the same inode, so when hard links are added to a file, you will see the number of links increase. Ensure that the PWD is still **~/temp**. Create a hard link to the file **main.file.txt**, then do another long list of the directory.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln main.file.txt link1.file.txt
|
||||
[dboth@david temp]$ ls -l
|
||||
total 8
|
||||
-rw-rw-r-- 2 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
-rw-rw-r-- 2 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
Notice that both files have two links and are exactly the same size. The date stamp is also the same. This is really one file with one inode and two links, i.e., directory entries to it. Create a second hard link to this file and list the directory contents. You can create the link to either of the existing ones: **link1.file.txt** or **main.file.txt**.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln link1.file.txt link2.file.txt ; ls -l
|
||||
total 16
|
||||
-rw-rw-r-- 3 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
-rw-rw-r-- 3 dboth dboth 7 Jun 13 07:34 link2.file.txt
|
||||
-rw-rw-r-- 3 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
Notice that each new hard link in this directory must have a different name because two files—really directory entries—cannot have the same name within the same directory. Try to create another link with a target name the same as one of the existing ones.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln main.file.txt link2.file.txt
|
||||
ln: failed to create hard link 'link2.file.txt': File exists
|
||||
```
|
||||
|
||||
Clearly that does not work, because **link2.file.txt** already exists. So far, we have created only hard links in the same directory. So, create a link in your home directory, the parent of the temp directory in which we have been working so far.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln main.file.txt ../main.file.txt ; ls -l ../main*
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
```
|
||||
|
||||
The **ls** command in the above listing shows that the **main.file.txt** file does exist in the home directory with the same name as the file in the temp directory. Of course, these are not different files; they are the same file with multiple links—directory entries—to the same inode. To help illustrate the next point, add a file that is not a link.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ touch unlinked.file ; ls -l
|
||||
total 12
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link2.file.txt
|
||||
-rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
-rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
Look at the inode number of the hard links and that of the new file using the **-i**option to the **ls** command.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ls -li
|
||||
total 12
|
||||
657024 -rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link1.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 link2.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 7 Jun 13 07:34 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
Notice the number **657024** to the left of the file mode in the example above. That is the inode number, and all three file links point to the same inode. You can use the **-i** option to view the inode number for the link we created in the home directory as well, and that will also show the same value. The inode number of the file that has only one link is different from the others. Note that the inode numbers will be different on your system.
|
||||
|
||||
Let's change the size of one of the hard-linked files.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ df -h > link2.file.txt ; ls -li
|
||||
total 12
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link2.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
The file size of all the hard-linked files is now larger than before. That is because there is really only one file that is linked to by multiple directory entries.
|
||||
|
||||
I know this next experiment will work on my computer because my **/tmp**directory is on a separate LV. If you have a separate LV or a filesystem on a different partition (if you're not using LVs), determine whether or not you have access to that LV or partition. If you don't, you can try to insert a USB memory stick and mount it. If one of those options works for you, you can do this experiment.
|
||||
|
||||
Try to create a link to one of the files in your **~/temp** directory in **/tmp** (or wherever your different filesystem directory is located).
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln link2.file.txt /tmp/link3.file.txt
|
||||
ln: failed to create hard link '/tmp/link3.file.txt' => 'link2.file.txt':
|
||||
Invalid cross-device link
|
||||
```
|
||||
|
||||
Why does this error occur? The reason is each separate mountable filesystem has its own set of inode numbers. Simply referring to a file by an inode number across the entire Linux directory structure can result in confusion because the same inode number can exist in each mounted filesystem.
|
||||
|
||||
There may be a time when you will want to locate all the hard links that belong to a single inode. You can find the inode number using the **ls -li** command. Then you can use the **find** command to locate all links with that inode number.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ find . -inum 657024
|
||||
./main.file.txt
|
||||
./link1.file.txt
|
||||
./link2.file.txt
|
||||
```
|
||||
|
||||
Note that the **find** command did not find all four of the hard links to this inode because we started at the current directory of **~/temp**. The **find** command only finds files in the PWD and its subdirectories. To find all the links, we can use the following command, which specifies your home directory as the starting place for the search.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ find ~ -samefile main.file.txt
|
||||
/home/dboth/temp/main.file.txt
|
||||
/home/dboth/temp/link1.file.txt
|
||||
/home/dboth/temp/link2.file.txt
|
||||
/home/dboth/main.file.txt
|
||||
```
|
||||
|
||||
You may see error messages if you do not have permissions as a non-root user. This command also uses the **-samefile** option instead of specifying the inode number. This works the same as using the inode number and can be easier if you know the name of one of the hard links.
|
||||
|
||||
### **Experimenting with soft links**
|
||||
|
||||
As you have just seen, creating hard links is not possible across filesystem boundaries; that is, from a filesystem on one LV or partition to a filesystem on another. Soft links are a means to answer that problem with hard links. Although they can accomplish the same end, they are very different, and knowing these differences is important.
|
||||
|
||||
Let's start by creating a symlink in our **~/temp** directory to start our exploration.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln -s link2.file.txt link3.file.txt ; ls -li
|
||||
total 12
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 link2.file.txt
|
||||
658270 lrwxrwxrwx 1 dboth dboth 14 Jun 14 15:21 link3.file.txt ->
|
||||
link2.file.txt
|
||||
657024 -rw-rw-r-- 4 dboth dboth 1157 Jun 14 14:14 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
The hard links, those that have the inode number **657024**, are unchanged, and the number of hard links shown for each has not changed. The newly created symlink has a different inode, number **658270**. The soft link named **link3.file.txt**points to **link2.file.txt**. Use the **cat** command to display the contents of **link3.file.txt**. The file mode information for the symlink starts with the letter "**l**" which indicates that this file is actually a symbolic link.
|
||||
|
||||
The size of the symlink **link3.file.txt** is only 14 bytes in the example above. That is the size of the text **link3.file.txt -> link2.file.txt**, which is the actual content of the directory entry. The directory entry **link3.file.txt** does not point to an inode; it points to another directory entry, which makes it useful for creating links that span file system boundaries. So, let's create that link we tried before from the **/tmp** directory.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ ln -s /home/dboth/temp/link2.file.txt
|
||||
/tmp/link3.file.txt ; ls -l /tmp/link*
|
||||
lrwxrwxrwx 1 dboth dboth 31 Jun 14 21:53 /tmp/link3.file.txt ->
|
||||
/home/dboth/temp/link2.file.txt
|
||||
```
|
||||
|
||||
### **Deleting links**
|
||||
|
||||
There are some other things that you should consider when you need to delete links or the files to which they point.
|
||||
|
||||
First, let's delete the link **main.file.txt**. Remember that every directory entry that points to an inode is simply a hard link.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ rm main.file.txt ; ls -li
|
||||
total 8
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 link2.file.txt
|
||||
658270 lrwxrwxrwx 1 dboth dboth 14 Jun 14 15:21 link3.file.txt ->
|
||||
link2.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
The link **main.file.txt** was the first link created when the file was created. Deleting it now still leaves the original file and its data on the hard drive along with all the remaining hard links. To delete the file and its data, you would have to delete all the remaining hard links.
|
||||
|
||||
Now delete the **link2.file.txt** hard link.
|
||||
|
||||
```
|
||||
[dboth@david temp]$ rm link2.file.txt ; ls -li
|
||||
total 8
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 link1.file.txt
|
||||
658270 lrwxrwxrwx 1 dboth dboth 14 Jun 14 15:21 link3.file.txt ->
|
||||
link2.file.txt
|
||||
657024 -rw-rw-r-- 3 dboth dboth 1157 Jun 14 14:14 main.file.txt
|
||||
657863 -rw-rw-r-- 1 dboth dboth 0 Jun 14 08:18 unlinked.file
|
||||
```
|
||||
|
||||
Notice what happens to the soft link. Deleting the hard link to which the soft link points leaves a broken link. On my system, the broken link is highlighted in colors and the target hard link is flashing. If the broken link needs to be fixed, you can create another hard link in the same directory with the same name as the old one, so long as not all the hard links have been deleted. You could also recreate the link itself, with the link maintaining the same name but pointing to one of the remaining hard links. Of course, if the soft link is no longer needed, it can be deleted with the **rm** command.
|
||||
|
||||
The **unlink** command can also be used to delete files and links. It is very simple and has no options, as the **rm** command does. It does, however, more accurately reflect the underlying process of deletion, in that it removes the link—the directory entry—to the file being deleted.
|
||||
|
||||
### Final thoughts
|
||||
|
||||
I worked with both types of links for a long time before I began to understand their capabilities and idiosyncrasies. It took writing a lab project for a Linux class I taught to fully appreciate how links work. This article is a simplification of what I taught in that class, and I hope it speeds your learning curve.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
作者简介:
|
||||
|
||||
David Both - David Both is a Linux and Open Source advocate who resides in Raleigh, North Carolina. He has been in the IT industry for over forty years and taught OS/2 for IBM where he worked for over 20 years. While at IBM, he wrote the first training course for the original IBM PC in 1981. He has taught RHCE classes for Red Hat and has worked at MCI Worldcom, Cisco, and the State of North Carolina. He has been working with Linux and Open Source Software for almost 20 years.
|
||||
|
||||
---------------------------------
|
||||
|
||||
via: https://opensource.com/article/17/6/linking-linux-filesystem
|
||||
|
||||
作者:[David Both ][a]
|
||||
译者:[runningwater](https://github.com/runningwater)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://opensource.com/users/dboth
|
||||
[1]:https://opensource.com/resources/what-is-linux?src=linux_resource_menu
|
||||
[2]:https://opensource.com/resources/what-are-linux-containers?src=linux_resource_menu
|
||||
[3]:https://developers.redhat.com/promotions/linux-cheatsheet/?intcmp=7016000000127cYAAQ
|
||||
[4]:https://developers.redhat.com/cheat-sheet/advanced-linux-commands-cheatsheet?src=linux_resource_menu&intcmp=7016000000127cYAAQ
|
||||
[5]:https://opensource.com/tags/linux?src=linux_resource_menu
|
||||
[6]:https://opensource.com/article/17/6/linking-linux-filesystem?rate=YebHxA-zgNopDQKKOyX3_r25hGvnZms_33sYBUq-SMM
|
||||
[7]:https://opensource.com/user/14106/feed
|
||||
[8]:https://www.flickr.com/photos/digypho/7905320090
|
||||
[9]:https://creativecommons.org/licenses/by/2.0/
|
||||
[10]:https://opensource.com/article/17/5/introduction-ext4-filesystem
|
||||
[11]:https://opensource.com/article/16/11/managing-devices-linux
|
||||
[12]:https://opensource.com/life/16/10/introduction-linux-filesystems
|
||||
[13]:https://opensource.com/business/16/9/linux-users-guide-lvm
|
||||
[14]:https://opensource.com/life/15/9/everything-is-a-file
|
||||
[15]:https://opensource.com/article/17/5/introduction-ext4-filesystem
|
||||
[16]:https://opensource.com/article/17/5/introduction-ext4-filesystem#fig2
|
||||
[17]:https://opensource.com/users/dboth
|
||||
[18]:https://opensource.com/article/17/6/linking-linux-filesystem#comments
|
@ -1,215 +0,0 @@
|
||||
yixunx翻译中
|
||||
|
||||
Designing a Microservices Architecture for Failure
|
||||
============================================================
|
||||
|
||||
|
||||
A Microservices architecture makes it possible to **isolate failures**through well-defined service boundaries. But like in every distributed system, there is a **higher chance** for network, hardware or application level issues. As a consequence of service dependencies, any component can be temporarily unavailable for their consumers. To minimize the impact of partial outages we need to build fault tolerant services that can **gracefully** respond to certain types of outages.
|
||||
|
||||
This article introduces the most common techniques and architecture patterns to build and operate a **highly available microservices** system based on [RisingStack’s Node.js Consulting & Development experience][3].
|
||||
|
||||
_If you are not familiar with the patterns in this article, it doesn’t necessarily mean that you do something wrong. Building a reliable system always comes with an extra cost._
|
||||
|
||||
### The Risk of the Microservices Architecture
|
||||
|
||||
The microservices architecture moves application logic to services and uses a network layer to communicate between them. Communicating over a network instead of in-memory calls brings extra latency and complexity to the system which requires cooperation between multiple physical and logical components. The increased complexity of the distributed system leads to a higher chance of particular **network failures**.
|
||||
|
||||
One of the biggest advantage of a microservices architecture over a monolithic one is that teams can independently design, develop and deploy their services. They have full ownership over their service's lifecycle. It also means that teams have no control over their service dependencies as it's more likely managed by a different team. With a microservices architecture, we need to keep in mind that provider **services can be temporarily unavailable** by broken releases, configurations, and other changes as they are controlled by someone else and components move independently from each other.
|
||||
|
||||
### Graceful Service Degradation
|
||||
|
||||
One of the best advantages of a microservices architecture is that you can isolate failures and achieve graceful service degradation as components fail separately. For example, during an outage customers in a photo sharing application maybe cannot upload a new picture, but they can still browse, edit and share their existing photos.
|
||||
|
||||

|
||||
|
||||
_Microservices fail separately (in theory)_
|
||||
|
||||
In most of the cases, it's hard to implement this kind of graceful service degradation as applications in a distributed system depend on each other, and you need to apply several failover logics _(some of them will be covered by this article later)_ to prepare for temporary glitches and outages.
|
||||
|
||||

|
||||
|
||||
_Services depend on each other and fail together without failover logics._
|
||||
|
||||
### Change management
|
||||
|
||||
Google’s site reliability team has found that roughly **70% of the outages are caused by changes** in a live system. When you change something in your service - you deploy a new version of your code or change some configuration - there is always a chance for failure or the introduction of a new bug.
|
||||
|
||||
In a microservices architecture, services depend on each other. This is why you should minimize failures and limit their negative effect. To deal with issues from changes, you can implement change management strategies and **automatic rollouts**.
|
||||
|
||||
For example, when you deploy new code, or you change some configuration, you should apply these changes to a subset of your instances gradually, monitor them and even automatically revert the deployment if you see that it has a negative effect on your key metrics.
|
||||
|
||||

|
||||
|
||||
_Change Management - Rolling Deployment_
|
||||
|
||||
Another solution could be that you run two production environments. You always deploy to only one of them, and you only point your load balancer to the new one after you verified that the new version works as it is expected. This is called blue-green, or red-black deployment.
|
||||
|
||||
**Reverting code is not a bad thing.** You shouldn’t leave broken code in production and then think about what went wrong. Always revert your changes when it’s necessary. The sooner the better.
|
||||
|
||||
#### Want to learn more about building reliable mircoservices architectures?
|
||||
|
||||
##### Check out our upcoming trainings!
|
||||
|
||||
[MICROSERVICES TRAININGS ][4]
|
||||
|
||||
### Health-check and Load Balancing
|
||||
|
||||
Instances continuously start, restart and stop because of failures, deployments or autoscaling. It makes them temporarily or permanently unavailable. To avoid issues, your load balancer should **skip unhealthy instances** from the routing as they cannot serve your customers' or sub-systems' need.
|
||||
|
||||
Application instance health can be determined via external observation. You can do it with repeatedly calling a `GET /health`endpoint or via self-reporting. Modern **service discovery** solutions continuously collect health information from instances and configure the load-balancer to route traffic only to healthy components.
|
||||
|
||||
### Self-healing
|
||||
|
||||
Self-healing can help to recover an application. We can talk about self-healing when an application can **do the necessary steps** to recover from a broken state. In most of the cases, it is implemented by an external system that watches the instances health and restarts them when they are in a broken state for a longer period. Self-healing can be very useful in most of the cases, however, in certain situations it **can cause trouble** by continuously restarting the application. This might happen when your application cannot give positive health status because it is overloaded or its database connection times out.
|
||||
|
||||
Implementing an advanced self-healing solution which is prepared for a delicate situation - like a lost database connection - can be tricky. In this case, you need to add extra logic to your application to handle edge cases and let the external system know that the instance is not needed to restart immediately.
|
||||
|
||||
### Failover Caching
|
||||
|
||||
Services usually fail because of network issues and changes in our system. However, most of these outages are temporary thanks to self-healing and advanced load-balancing we should find a solution to make our service work during these glitches. This is where **failover caching** can help and provide the necessary data to our application.
|
||||
|
||||
Failover caches usually use **two different expiration dates**; a shorter that tells how long you can use the cache in a normal situation, and a longer one that says how long can you use the cached data during failure.
|
||||
|
||||

|
||||
|
||||
_Failover Caching_
|
||||
|
||||
It’s important to mention that you can only use failover caching when it serves **the outdated data better than nothing**.
|
||||
|
||||
To set cache and failover cache, you can use standard response headers in HTTP.
|
||||
|
||||
For example, with the `max-age` header you can specify the maximum amount of time a resource will be considered fresh. With the `stale-if-error` header, you can determine how long should the resource be served from a cache in the case of a failure.
|
||||
|
||||
Modern CDNs and load balancers provide various caching and failover behaviors, but you can also create a shared library for your company that contains standard reliability solutions.
|
||||
|
||||
### Retry Logic
|
||||
|
||||
There are certain situations when we cannot cache our data or we want to make changes to it, but our operations eventually fail. In these cases, we can **retry our action** as we can expect that the resource will recover after some time or our load-balancer sends our request to a healthy instance.
|
||||
|
||||
You should be careful with adding retry logic to your applications and clients, as a larger amount of **retries can make things even worse** or even prevent the application from recovering.
|
||||
|
||||
In distributed system, a microservices system retry can trigger multiple other requests or retries and start a **cascading effect**. To minimize the impact of retries, you should limit the number of them and use an exponential backoff algorithm to continually increase the delay between retries until you reach the maximum limit.
|
||||
|
||||
As a retry is initiated by the client _(browser, other microservices, etc.)_ and the client doesn't know that the operation failed before or after handling the request, you should prepare your application to handle **idempotency**. For example, when you retry a purchase operation, you shouldn't double charge the customer. Using a unique **idempotency-key** for each of your transactions can help to handle retries.
|
||||
|
||||
### Rate Limiters and Load Shedders
|
||||
|
||||
Rate limiting is the technique of defining how many requests can be received or processed by a particular customer or application during a timeframe. With rate limiting, for example, you can filter out customers and microservices who are responsible for **traffic peaks**, or you can ensure that your application doesn’t overload until autoscaling can’t come to rescue.
|
||||
|
||||
You can also hold back lower-priority traffic to give enough resources to critical transactions.
|
||||
|
||||

|
||||
|
||||
_A rate limiter can hold back traffic peaks_
|
||||
|
||||
A different type of rate limiter is called the _concurrent request limiter_ . It can be useful when you have expensive endpoints that shouldn’t be called more than a specified times, while you still want to serve traffic.
|
||||
|
||||
A _fleet usage load shedder_ can ensure that there are always enough resources available to **serve critical transactions**. It keeps some resources for high priority requests and doesn’t allow for low priority transactions to use all of them. A load shedder makes its decisions based on the whole state of the system, rather than based on a single user’s request bucket size. Load shedders **help your system to recover**, since they keep the core functionalities working while you have an ongoing incident.
|
||||
|
||||
To read more about rate limiters and load shredders, I recommend checking out [Stripe’s article][5].
|
||||
|
||||
### Fail Fast and Independently
|
||||
|
||||
In a microservices architecture we want to prepare our services **to fail fast and separately**. To isolate issues on service level, we can use the _bulkhead pattern_ . You can read more about bulkheads later in this blog post.
|
||||
|
||||
We also want our components to **fail fast** as we don't want to wait for broken instances until they timeout. Nothing is more disappointing than a hanging request and an unresponsive UI. It's not just wasting resources but also screwing up the user experience. Our services are calling each other in a chain, so we should pay an extra attention to prevent hanging operations before these delays sum up.
|
||||
|
||||
The first idea that would come to your mind would be applying fine grade timeouts for each service calls. The problem with this approach is that you cannot really know what's a good timeout value as there are certain situations when network glitches and other issues happen that only affect one-two operations. In this case, you probably don’t want to reject those requests if there’s only a few of them timeouts.
|
||||
|
||||
We can say that achieving the fail fast paradigm in microservices by **using timeouts is an anti-pattern** and you should avoid it. Instead of timeouts, you can apply the _circuit-breaker_ pattern that depends on the success / fail statistics of operations.
|
||||
|
||||
#### Want to learn more about building reliable mircoservices architectures?
|
||||
|
||||
##### Check out our upcoming trainings!
|
||||
|
||||
[MICROSERVICES TRAININGS ][6]
|
||||
|
||||
### Bulkheads
|
||||
|
||||
Bulkhead is used in the industry to **partition** a ship **into sections**, so that sections can be sealed off if there is a hull breach.
|
||||
|
||||
The concept of bulkheads can be applied in software development to **segregate resources**.
|
||||
|
||||
By applying the bulkheads pattern, we can **protect limited resources** from being exhausted. For example, we can use two connection pools instead of a shared on if we have two kinds of operations that communicate with the same database instance where we have limited number of connections. As a result of this client - resource separation, the operation that timeouts or overuses the pool won't bring all of the other operations down.
|
||||
|
||||
One of the main reasons why Titanic sunk was that its bulkheads had a design failure, and the water could pour over the top of the bulkheads via the deck above and flood the entire hull.
|
||||
|
||||

|
||||
|
||||
_Bulkheads in Titanic (they didn't work)_
|
||||
|
||||
### Circuit Breakers
|
||||
|
||||
To limit the duration of operations, we can use timeouts. Timeouts can prevent hanging operations and keep the system responsive. However, using static, fine tuned timeouts in microservices communication is an **anti-pattern** as we’re in a highly dynamic environment where it's almost impossible to come up with the right timing limitations that work well in every case.
|
||||
|
||||
Instead of using small and transaction-specific static timeouts, we can use circuit breakers to deal with errors. Circuit breakers are named after the real world electronic component because their behavior is identical. You can **protect resources** and **help them to recover** with circuit breakers. They can be very useful in a distributed system where a repetitive failure can lead to a snowball effect and bring the whole system down.
|
||||
|
||||
A circuit breaker opens when a particular type of **error occurs multiple times** in a short period. An open circuit breaker prevents further requests to be made - like the real one prevents electrons from flowing. Circuit breakers usually close after a certain amount of time, giving enough space for underlying services to recover.
|
||||
|
||||
Keep in mind that not all errors should trigger a circuit breaker. For example, you probably want to skip client side issues like requests with `4xx` response codes, but include `5xx` server-side failures. Some circuit breakers can have a half-open state as well. In this state, the service sends the first request to check system availability, while letting the other requests to fail. If this first request succeeds, it restores the circuit breaker to a closed state and lets the traffic flow. Otherwise, it keeps it open.
|
||||
|
||||

|
||||
|
||||
_Circuit Breaker_
|
||||
|
||||
### Testing for Failures
|
||||
|
||||
You should continually **test your system against common issues** to make sure that your services can **survive various failures**. You should test for failures frequently to keep your team prepared for incidents.
|
||||
|
||||
For testing, you can use an external service that identifies groups of instances and randomly terminates one of the instances in this group. With this, you can prepare for a single instance failure, but you can even shut down entire regions to simulate a cloud provider outage.
|
||||
|
||||
One of the most popular testing solutions is the [ChaosMonkey][7]resiliency tool by Netflix.
|
||||
|
||||
### Outro
|
||||
|
||||
Implementing and running a reliable service is not easy. It takes a lot of effort from your side and also costs money to your company.
|
||||
|
||||
Reliability has many levels and aspects, so it is important to find the best solution for your team. You should make reliability a factor in your business decision processes and allocate enough budget and time for it.
|
||||
|
||||
### Key Takeways
|
||||
|
||||
* Dynamic environments and distributed systems - like microservices - lead to a higher chance of failures.
|
||||
|
||||
* Services should fail separately, achieve graceful degradation to improve user experience.
|
||||
|
||||
* 70% of the outages are caused by changes, reverting code is not a bad thing.
|
||||
|
||||
* Fail fast and independently. Teams have no control over their service dependencies.
|
||||
|
||||
* Architectural patterns and techniques like caching, bulkheads, circuit breakers and rate-limiters help to build reliable microservices.
|
||||
|
||||
To learn more about running a reliable service check out our free [Node.js Monitoring, Alerting & Reliability 101 e-book][8]. In case you need help with implementing a microservices system, reach out to us at [@RisingStack][9] on Twitter, or enroll in our upcoming [Building Microservices with Node.js][10].
|
||||
|
||||
|
||||
-------------
|
||||
|
||||
作者简介
|
||||
|
||||
[Péter Márton][2]
|
||||
|
||||
CTO at RisingStack, microservices and brewing beer with Node.js
|
||||
|
||||
[https://twitter.com/slashdotpeter][1]</footer>
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://blog.risingstack.com/designing-microservices-architecture-for-failure/
|
||||
|
||||
作者:[ Péter Márton][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://blog.risingstack.com/author/peter-marton/
|
||||
[1]:https://twitter.com/slashdotpeter
|
||||
[2]:https://blog.risingstack.com/author/peter-marton/
|
||||
[3]:https://risingstack.com/
|
||||
[4]:https://blog.risingstack.com/training-building-microservices-node-js/?utm_source=rsblog&utm_medium=roadblock-new&utm_content=/designing-microservices-architecture-for-failure/
|
||||
[5]:https://stripe.com/blog/rate-limiters
|
||||
[6]:https://blog.risingstack.com/training-building-microservices-node-js/?utm_source=rsblog&utm_medium=roadblock-new
|
||||
[7]:https://github.com/Netflix/chaosmonkey
|
||||
[8]:https://trace.risingstack.com/monitoring-ebook
|
||||
[9]:https://twitter.com/RisingStack
|
||||
[10]:https://blog.risingstack.com/training-building-microservices-node-js/
|
||||
[11]:https://blog.risingstack.com/author/peter-marton/
|
@ -1,3 +1,5 @@
|
||||
voidpainter is translating
|
||||
---
|
||||
[Betting on the Web][27]
|
||||
============================================================
|
||||
|
||||
|
@ -0,0 +1,172 @@
|
||||
How to answer questions in a helpful way
|
||||
============================================================
|
||||
|
||||
Your coworker asks you a slightly unclear question. How do you answer? I think asking questions is a skill (see [How to ask good questions][1]) and that answering questions in a helpful way is also a skill! Both of them are super useful.
|
||||
|
||||
To start out with – sometimes the people asking you questions don’t respect your time, and that sucks. I’m assuming here throughout that that’s not what happening – we’re going to assume that the person asking you questions is a reasonable person who is trying their best to figure something out and that you want to help them out. Everyone I work with is like that and so that’s the world I live in :)
|
||||
|
||||
Here are a few strategies for answering questions in a helpful way!
|
||||
|
||||
### If they’re not asking clearly, help them clarify
|
||||
|
||||
Often beginners don’t ask clear questions, or ask questions that don’t have the necessary information to answer the questions. Here are some strategies you can use to help them clarify.
|
||||
|
||||
* **Rephrase a more specific question** back at them (“Are you asking X?”)
|
||||
|
||||
* **Ask them for more specific information** they didn’t provide (“are you using IPv6?”)
|
||||
|
||||
* **Ask what prompted their question**. For example, sometimes people come into my team’s channel with questions about how our service discovery works. Usually this is because they’re trying to set up/reconfigure a service. In that case it’s helpful to ask “which service are you working with? Can I see the pull request you’re working on?”
|
||||
|
||||
A lot of these strategies come from the [how to ask good questions][2] post. (though I would never say to someone “oh you need to read this Document On How To Ask Good Questions before asking me a question”)
|
||||
|
||||
### Figure out what they know already
|
||||
|
||||
Before answering a question, it’s very useful to know what the person knows already!
|
||||
|
||||
Harold Treen gave me a great example of this:
|
||||
|
||||
> Someone asked me the other day to explain “Redux Sagas”. Rather than dive in and say “They are like worker threads that listen for actions and let you update the store!”
|
||||
> I started figuring out how much they knew about Redux, actions, the store and all these other fundamental concepts. From there it was easier to explain the concept that ties those other concepts together.
|
||||
|
||||
Figuring out what your question-asker knows already is important because they may be confused about fundamental concepts (“What’s Redux?”), or they may be an expert who’s getting at a subtle corner case. An answer building on concepts they don’t know is confusing, and an answer that recaps things they know is tedious.
|
||||
|
||||
One useful trick for asking what people know – instead of “Do you know X?”, maybe try “How familiar are you with X?”.
|
||||
|
||||
### Point them to the documentation
|
||||
|
||||
“RTFM” is the classic unhelpful answer to a question, but pointing someone to a specific piece of documentation can actually be really helpful! When I’m asking a question, I’d honestly rather be pointed to documentation that actually answers my question, because it’s likely to answer other questions I have too.
|
||||
|
||||
I think it’s important here to make sure you’re linking to documentation that actually answers the question, or at least check in afterwards to make sure it helped. Otherwise you can end up with this (pretty common) situation:
|
||||
|
||||
* Ali: How do I do X?
|
||||
|
||||
* Jada: <link to documentation>
|
||||
|
||||
* Ali: That doesn’t actually explain how to X, it only explains Y!
|
||||
|
||||
If the documentation I’m linking to is very long, I like to point out the specific part of the documentation I’m talking about. The [bash man page][3] is 44,000 words (really!), so just saying “it’s in the bash man page” is not that helpful :)
|
||||
|
||||
### Point them to a useful search
|
||||
|
||||
Often I find things at work by searching for some Specific Keyword that I know will find me the answer. That keyword might not be obvious to a beginner! So saying “this is the search I’d use to find the answer to that question” can be useful. Again, check in afterwards to make sure the search actually gets them the answer they need :)
|
||||
|
||||
### Write new documentation
|
||||
|
||||
People often come and ask my team the same questions over and over again. This is obviously not the fault of the people (how should _they_ know that 10 people have asked this already, or what the answer is?). So we’re trying to, instead of answering the questions directly,
|
||||
|
||||
1. Immediately write documentation
|
||||
|
||||
2. Point the person to the new documentation we just wrote
|
||||
|
||||
3. Celebrate!
|
||||
|
||||
Writing documentation sometimes takes more time than just answering the question, but it’s often worth it! Writing documentation is especially worth it if:
|
||||
|
||||
a. It’s a question which is being asked again and again b. The answer doesn’t change too much over time (if the answer changes every week or month, the documentation will just get out of date and be frustrating)
|
||||
|
||||
### Explain what you did
|
||||
|
||||
As a beginner to a subject, it’s really frustrating to have an exchange like this:
|
||||
|
||||
* New person: “hey how do you do X?”
|
||||
|
||||
* More Experienced Person: “I did it, it is done.”
|
||||
|
||||
* New person: ….. but what did you DO?!
|
||||
|
||||
If the person asking you is trying to learn how things work, it’s helpful to:
|
||||
|
||||
* Walk them through how to accomplish a task instead of doing it yourself
|
||||
|
||||
* Tell them the steps for how you got the answer you gave them!
|
||||
|
||||
This might take longer than doing it yourself, but it’s a learning opportunity for the person who asked, so that they’ll be better equipped to solve such problems in the future.
|
||||
|
||||
Then you can have WAY better exchanges, like this:
|
||||
|
||||
* New person: “I’m seeing errors on the site, what’s happening?”
|
||||
|
||||
* More Experienced Person: (2 minutes later) “oh that’s because there’s a database failover happening”
|
||||
|
||||
* New person: how did you know that??!?!?
|
||||
|
||||
* More Experienced Person: “Here’s what I did!”:
|
||||
1. Often these errors are due to Service Y being down. I looked at $PLACE and it said Service Y was up. So that wasn’t it.
|
||||
|
||||
2. Then I looked at dashboard X, and this part of that dashboard showed there was a database failover happening.
|
||||
|
||||
3. Then I looked in the logs for the service and it showed errors connecting to the database, here’s what those errors look like.
|
||||
|
||||
If you’re explaining how you debugged a problem, it’s useful both to explain how you found out what the problem was, and how you found out what the problem wasn’t. While it might feel good to look like you knew the answer right off the top of your head, it feels even better to help someone improve at learning and diagnosis, and understand the resources available.
|
||||
|
||||
### Solve the underlying problem
|
||||
|
||||
This one is a bit tricky. Sometimes people think they’ve got the right path to a solution, and they just need one more piece of information to implement that solution. But they might not be quite on the right path! For example:
|
||||
|
||||
* George: I’m doing X, and I got this error, how do I fix it
|
||||
|
||||
* Jasminda: Are you actually trying to do Y? If so, you shouldn’t do X, you should do Z instead
|
||||
|
||||
* George: Oh, you’re right!!! Thank you! I will do Z instead.
|
||||
|
||||
Jasminda didn’t answer George’s question at all! Instead she guessed that George didn’t actually want to be doing X, and she was right. That is helpful!
|
||||
|
||||
It’s possible to come off as condescending here though, like
|
||||
|
||||
* George: I’m doing X, and I got this error, how do I fix it?
|
||||
|
||||
* Jasminda: Don’t do that, you’re trying to do Y and you should do Z to accomplish that instead.
|
||||
|
||||
* George: Well, I am not trying to do Y, I actually want to do X because REASONS. How do I do X?
|
||||
|
||||
So don’t be condescending, and keep in mind that some questioners might be attached to the steps they’ve taken so far! It might be appropriate to answer both the question they asked and the one they should have asked: “Well, if you want to do X then you might try this, but if you’re trying to solve problem Y with that, you might have better luck doing this other thing, and here’s why that’ll work better”.
|
||||
|
||||
### Ask “Did that answer your question?”
|
||||
|
||||
I always like to check in after I _think_ I’ve answered the question and ask “did that answer your question? Do you have more questions?”.
|
||||
|
||||
It’s good to pause and wait after asking this because often people need a minute or two to know whether or not they’ve figured out the answer. I especially find this extra “did this answer your questions?” step helpful after writing documentation! Often when writing documentation about something I know well I’ll leave out something very important without realizing it.
|
||||
|
||||
### Offer to pair program/chat in real life
|
||||
|
||||
I work remote, so many of my conversations at work are text-based. I think of that as the default mode of communication.
|
||||
|
||||
Today, we live in a world of easy video conferencing & screensharing! At work I can at any time click a button and immediately be in a video call/screensharing session with someone. Some problems are easier to talk about using your voices!
|
||||
|
||||
For example, recently someone was asking about capacity planning/autoscaling for their service. I could tell there were a few things we needed to clear up but I wasn’t exactly sure what they were yet. We got on a quick video call and 5 minutes later we’d answered all their questions.
|
||||
|
||||
I think especially if someone is really stuck on how to get started on a task, pair programming for a few minutes can really help, and it can be a lot more efficient than email/instant messaging.
|
||||
|
||||
### Don’t act surprised
|
||||
|
||||
This one’s a rule from the Recurse Center: [no feigning surprise][4]. Here’s a relatively common scenario
|
||||
|
||||
* Human 1: “what’s the Linux kernel?”
|
||||
|
||||
* Human 2: “you don’t know what the LINUX KERNEL is?!!!!?!!!???”
|
||||
|
||||
Human 2’s reaction (regardless of whether they’re _actually_ surprised or not) is not very helpful. It mostly just serves to make Human 1 feel bad that they don’t know what the Linux kernel is.
|
||||
|
||||
I’ve worked on actually pretending not to be surprised even when I actually am a bit surprised the person doesn’t know the thing and it’s awesome.
|
||||
|
||||
### Answering questions well is awesome
|
||||
|
||||
Obviously not all these strategies are appropriate all the time, but hopefully you will find some of them helpful! I find taking the time to answer questions and teach people can be really rewarding.
|
||||
|
||||
Special thanks to Josh Triplett for suggesting this post and making many helpful additions, and to Harold Treen, Vaibhav Sagar, Peter Bhat Harkins, Wesley Aptekar-Cassels, and Paul Gowder for reading/commenting.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://jvns.ca/blog/answer-questions-well/
|
||||
|
||||
作者:[ Julia Evans][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://jvns.ca/about
|
||||
[1]:https://jvns.ca/blog/good-questions/
|
||||
[2]:https://jvns.ca/blog/good-questions/
|
||||
[3]:https://linux.die.net/man/1/bash
|
||||
[4]:https://jvns.ca/blog/2017/04/27/no-feigning-surprise/
|
149
sources/tech/20171005 Reasons Kubernetes is cool.md
Normal file
149
sources/tech/20171005 Reasons Kubernetes is cool.md
Normal file
@ -0,0 +1,149 @@
|
||||
Translating by qhwdw
|
||||
Reasons Kubernetes is cool
|
||||
============================================================
|
||||
|
||||
When I first learned about Kubernetes (a year and a half ago?) I really didn’t understand why I should care about it.
|
||||
|
||||
I’ve been working full time with Kubernetes for 3 months or so and now have some thoughts about why I think it’s useful. (I’m still very far from being a Kubernetes expert!) Hopefully this will help a little in your journey to understand what even is going on with Kubernetes!
|
||||
|
||||
I will try to explain some reason I think Kubenetes is interesting without using the words “cloud native”, “orchestration”, “container”, or any Kubernetes-specific terminology :). I’m going to explain this mostly from the perspective of a kubernetes operator / infrastructure engineer, since my job right now is to set up Kubernetes and make it work well.
|
||||
|
||||
I’m not going to try to address the question of “should you use kubernetes for your production systems?” at all, that is a very complicated question. (not least because “in production” has totally different requirements depending on what you’re doing)
|
||||
|
||||
### Kubernetes lets you run code in production without setting up new servers
|
||||
|
||||
The first pitch I got for Kubernetes was the following conversation with my partner Kamal:
|
||||
|
||||
Here’s an approximate transcript:
|
||||
|
||||
* Kamal: With Kubernetes you can set up a new service with a single command
|
||||
|
||||
* Julia: I don’t understand how that’s possible.
|
||||
|
||||
* Kamal: Like, you just write 1 configuration file, apply it, and then you have a HTTP service running in production
|
||||
|
||||
* Julia: But today I need to create new AWS instances, write a puppet manifest, set up service discovery, configure my load balancers, configure our deployment software, and make sure DNS is working, it takes at least 4 hours if nothing goes wrong.
|
||||
|
||||
* Kamal: Yeah. With Kubernetes you don’t have to do any of that, you can set up a new HTTP service in 5 minutes and it’ll just automatically run. As long as you have spare capacity in your cluster it just works!
|
||||
|
||||
* Julia: There must be a trap
|
||||
|
||||
There kind of is a trap, setting up a production Kubernetes cluster is (in my experience) is definitely not easy. (see [Kubernetes The Hard Way][3] for what’s involved to get started). But we’re not going to go into that right now!
|
||||
|
||||
So the first cool thing about Kubernetes is that it has the potential to make life way easier for developers who want to deploy new software into production. That’s cool, and it’s actually true, once you have a working Kubernetes cluster you really can set up a production HTTP service (“run 5 of this application, set up a load balancer, give it this DNS name, done”) with just one configuration file. It’s really fun to see.
|
||||
|
||||
### Kubernetes gives you easy visibility & control of what code you have running in production
|
||||
|
||||
IMO you can’t understand Kubernetes without understanding etcd. So let’s talk about etcd!
|
||||
|
||||
Imagine that I asked you today “hey, tell me every application you have running in production, what host it’s running on, whether it’s healthy or not, and whether or not it has a DNS name attached to it”. I don’t know about you but I would need to go look in a bunch of different places to answer this question and it would take me quite a while to figure out. I definitely can’t query just one API.
|
||||
|
||||
In Kubernetes, all the state in your cluster – applications running (“pods”), nodes, DNS names, cron jobs, and more – is stored in a single database (etcd). Every Kubernetes component is stateless, and basically works by
|
||||
|
||||
* Reading state from etcd (eg “the list of pods assigned to node 1”)
|
||||
|
||||
* Making changes (eg “actually start running pod A on node 1”)
|
||||
|
||||
* Updating the state in etcd (eg “set the state of pod A to ‘running’”)
|
||||
|
||||
This means that if you want to answer a question like “hey, how many nginx pods do I have running right now in that availabliity zone?” you can answer it by querying a single unified API (the Kubernetes API!). And you have exactly the same access to that API that every other Kubernetes component does.
|
||||
|
||||
This also means that you have easy control of everything running in Kubernetes. If you want to, say,
|
||||
|
||||
* Implement a complicated custom rollout strategy for deployments (deploy 1 thing, wait 2 minutes, deploy 5 more, wait 3.7 minutes, etc)
|
||||
|
||||
* Automatically [start a new webserver][1] every time a branch is pushed to github
|
||||
|
||||
* Monitor all your running applications to make sure all of them have a reasonable cgroups memory limit
|
||||
|
||||
all you need to do is to write a program that talks to the Kubernetes API. (a “controller”)
|
||||
|
||||
Another very exciting thing about the Kubernetes API is that you’re not limited to just functionality that Kubernetes provides! If you decide that you have your own opinions about how your software should be deployed / created / monitored, then you can write code that uses the Kubernetes API to do it! It lets you do everything you need.
|
||||
|
||||
### If every Kubernetes component dies, your code will still keep running
|
||||
|
||||
One thing I was originally promised (by various blog posts :)) about Kubernetes was “hey, if the Kubernetes apiserver and everything else dies, it’s ok, your code will just keep running”. I thought this sounded cool in theory but I wasn’t sure if it was actually true.
|
||||
|
||||
So far it seems to be actually true!
|
||||
|
||||
I’ve been through some etcd outages now, and what happens is
|
||||
|
||||
1. All the code that was running keeps running
|
||||
|
||||
2. Nothing _new_ happens (you can’t deploy new code or make changes, cron jobs will stop working)
|
||||
|
||||
3. When everything comes back, the cluster will catch up on whatever it missed
|
||||
|
||||
This does mean that if etcd goes down and one of your applications crashes or something, it can’t come back up until etcd returns.
|
||||
|
||||
### Kubernetes’ design is pretty resilient to bugs
|
||||
|
||||
Like any piece of software, Kubernetes has bugs. For example right now in our cluster the controller manager has a memory leak, and the scheduler crashes pretty regularly. Bugs obviously aren’t good but so far I’ve found that Kubernetes’ design helps mitigate a lot of the bugs in its core components really well.
|
||||
|
||||
If you restart any component, what happens is:
|
||||
|
||||
* It reads all its relevant state from etcd
|
||||
|
||||
* It starts doing the necessary things it’s supposed to be doing based on that state (scheduling pods, garbage collecting completed pods, scheduling cronjobs, deploying daemonsets, whatever)
|
||||
|
||||
Because all the components don’t keep any state in memory, you can just restart them at any time and that can help mitigate a variety of bugs.
|
||||
|
||||
For example! Let’s say you have a memory leak in your controller manager. Because the controller manager is stateless, you can just periodically restart it every hour or something and feel confident that you won’t cause any consistency issues. Or we ran into a bug in the scheduler where it would sometimes just forget about pods and never schedule them. You can sort of mitigate this just by restarting the scheduler every 10 minutes. (we didn’t do that, we fixed the bug instead, but you _could_ :) )
|
||||
|
||||
So I feel like I can trust Kubernetes’ design to help make sure the state in the cluster is consistent even when there are bugs in its core components. And in general I think the software is generally improving over time. The only stateful thing you have to operate is etcd
|
||||
|
||||
Not to harp on this “state” thing too much but – I think it’s cool that in Kubernetes the only thing you have to come up with backup/restore plans for is etcd (unless you use persistent volumes for your pods). I think it makes kubernetes operations a lot easier to think about.
|
||||
|
||||
### Implementing new distributed systems on top of Kubernetes is relatively easy
|
||||
|
||||
Suppose you want to implement a distributed cron job scheduling system! Doing that from scratch is a ton of work. But implementing a distributed cron job scheduling system inside Kubernetes is much easier! (still not trivial, it’s still a distributed system)
|
||||
|
||||
The first time I read the code for the Kubernetes cronjob controller I was really delighted by how simple it was. Here, go read it! The main logic is like 400 lines of Go. Go ahead, read it! => [cronjob_controller.go][4] <=
|
||||
|
||||
Basically what the cronjob controller does is:
|
||||
|
||||
* Every 10 seconds:
|
||||
* Lists all the cronjobs that exist
|
||||
|
||||
* Checks if any of them need to run right now
|
||||
|
||||
* If so, creates a new Job object to be scheduled & actually run by other Kubernetes controllers
|
||||
|
||||
* Clean up finished jobs
|
||||
|
||||
* Repeat
|
||||
|
||||
The Kubernetes model is pretty constrained (it has this pattern of resources are defined in etcd, controllers read those resources and update etcd), and I think having this relatively opinionated/constrained model makes it easier to develop your own distributed systems inside the Kubernetes framework.
|
||||
|
||||
Kamal introduced me to this idea of “Kubernetes is a good platform for writing your own distributed systems” instead of just “Kubernetes is a distributed system you can use” and I think it’s really interesting. He has a prototype of a [system to run an HTTP service for every branch you push to github][5]. It took him a weekend and is like 800 lines of Go, which I thought was impressive!
|
||||
|
||||
### Kubernetes lets you do some amazing things (but isn’t easy)
|
||||
|
||||
I started out by saying “kubernetes lets you do these magical things, you can just spin up so much infrastructure with a single configuration file, it’s amazing”. And that’s true!
|
||||
|
||||
What I mean by “Kubernetes isn’t easy” is that Kubernetes has a lot of moving parts learning how to successfully operate a highly available Kubernetes cluster is a lot of work. Like I find that with a lot of the abstractions it gives me, I need to understand what is underneath those abstractions in order to debug issues and configure things properly. I love learning new things so this doesn’t make me angry or anything, I just think it’s important to know :)
|
||||
|
||||
One specific example of “I can’t just rely on the abstractions” that I’ve struggled with is that I needed to learn a LOT [about how networking works on Linux][6] to feel confident with setting up Kubernetes networking, way more than I’d ever had to learn about networking before. This was very fun but pretty time consuming. I might write more about what is hard/interesting about setting up Kubernetes networking at some point.
|
||||
|
||||
Or I wrote a [2000 word blog post][7] about everything I had to learn about Kubernetes’ different options for certificate authorities to be able to set up my Kubernetes CAs successfully.
|
||||
|
||||
I think some of these managed Kubernetes systems like GKE (google’s kubernetes product) may be simpler since they make a lot of decisions for you but I haven’t tried any of them.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://jvns.ca/blog/2017/10/05/reasons-kubernetes-is-cool/
|
||||
|
||||
作者:[ Julia Evans][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://jvns.ca/about
|
||||
[1]:https://github.com/kamalmarhubi/kubereview
|
||||
[2]:https://jvns.ca/categories/kubernetes
|
||||
[3]:https://github.com/kelseyhightower/kubernetes-the-hard-way
|
||||
[4]:https://github.com/kubernetes/kubernetes/blob/e4551d50e57c089aab6f67333412d3ca64bc09ae/pkg/controller/cronjob/cronjob_controller.go
|
||||
[5]:https://github.com/kamalmarhubi/kubereview
|
||||
[6]:https://jvns.ca/blog/2016/12/22/container-networking/
|
||||
[7]:https://jvns.ca/blog/2017/08/05/how-kubernetes-certificates-work/
|
216
sources/tech/20171010 Operating a Kubernetes network.md
Normal file
216
sources/tech/20171010 Operating a Kubernetes network.md
Normal file
@ -0,0 +1,216 @@
|
||||
Operating a Kubernetes network
|
||||
============================================================
|
||||
|
||||
I’ve been working on Kubernetes networking a lot recently. One thing I’ve noticed is, while there’s a reasonable amount written about how to **set up** your Kubernetes network, I haven’t seen much about how to **operate** your network and be confident that it won’t create a lot of production incidents for you down the line.
|
||||
|
||||
In this post I’m going to try to convince you of three things: (all I think pretty reasonable :))
|
||||
|
||||
* Avoiding networking outages in production is important
|
||||
|
||||
* Operating networking software is hard
|
||||
|
||||
* It’s worth thinking critically about major changes to your networking infrastructure and the impact that will have on your reliability, even if very fancy Googlers say “this is what we do at Google”. (google engineers are doing great work on Kubernetes!! But I think it’s important to still look at the architecture and make sure it makes sense for your organization.)
|
||||
|
||||
I’m definitely not a Kubernetes networking expert by any means, but I have run into a few issues while setting things up and definitely know a LOT more about Kubernetes networking than I used to.
|
||||
|
||||
### Operating networking software is hard
|
||||
|
||||
Here I’m not talking about operating physical networks (I don’t know anything about that), but instead about keeping software like DNS servers & load balancers & proxies working correctly.
|
||||
|
||||
I have been working on a team that’s responsible for a lot of networking infrastructure for a year, and I have learned a few things about operating networking infrastructure! (though I still have a lot to learn obviously). 3 overall thoughts before we start:
|
||||
|
||||
* Networking software often relies very heavily on the Linux kernel. So in addition to configuring the software correctly you also need to make sure that a bunch of different sysctls are set correctly, and a misconfigured sysctl can easily be the difference between “everything is 100% fine” and “everything is on fire”.
|
||||
|
||||
* Networking requirements change over time (for example maybe you’re doing 5x more DNS lookups than you were last year! Maybe your DNS server suddenly started returning TCP DNS responses instead of UDP which is a totally different kernel workload!). This means software that was working fine before can suddenly start having issues.
|
||||
|
||||
* To fix a production networking issues you often need a lot of expertise. (for example see this [great post by Sophie Haskins on debugging a kube-dns issue][1]) I’m a lot better at debugging networking issues than I was, but that’s only after spending a huge amount of time investing in my knowledge of Linux networking.
|
||||
|
||||
I am still far from an expert at networking operations but I think it seems important to:
|
||||
|
||||
1. Very rarely make major changes to the production networking infrastructure (because it’s super disruptive)
|
||||
|
||||
2. When you _are_ making major changes, think really carefully about what the failure modes are for the new network architecture are
|
||||
|
||||
3. Have multiple people who are able to understand your networking setup
|
||||
|
||||
Switching to Kubernetes is obviously a pretty major networking change! So let’s talk about what some of the things that can go wrong are!
|
||||
|
||||
### Kubernetes networking components
|
||||
|
||||
The Kubernetes networking components we’re going to talk about in this post are:
|
||||
|
||||
* Your overlay network backend (like flannel/calico/weave net/romana)
|
||||
|
||||
* `kube-dns`
|
||||
|
||||
* `kube-proxy`
|
||||
|
||||
* Ingress controllers / load balancers
|
||||
|
||||
* The `kubelet`
|
||||
|
||||
If you’re going to set up HTTP services you probably need all of these. I’m not using most of these components yet but I’m trying to understand them, so that’s what this post is about.
|
||||
|
||||
### The simplest way: Use host networking for all your containers
|
||||
|
||||
Let’s start with the simplest possible thing you can do. This won’t let you run HTTP services in Kubernetes. I think it’s pretty safe because there are less moving parts.
|
||||
|
||||
If you use host networking for all your containers I think all you need to do is:
|
||||
|
||||
1. Configure the kubelet to configure DNS correctly inside your containers
|
||||
|
||||
2. That’s it
|
||||
|
||||
If you use host networking for literally every pod you don’t need kube-dns or kube-proxy. You don’t even need a working overlay network.
|
||||
|
||||
In this setup your pods can connect to the outside world (the same way any process on your hosts would talk to the outside world) but the outside world can’t connect to your pods.
|
||||
|
||||
This isn’t super important (I think most people want to run HTTP services inside Kubernetes and actually communicate with those services) but I do think it’s interesting to realize that at some level all of this networking complexity isn’t strictly required and sometimes you can get away without using it. Avoiding networking complexity seems like a good idea to me if you can.
|
||||
|
||||
### Operating an overlay network
|
||||
|
||||
The first networking component we’re going to talk about is your overlay network. Kubernetes assumes that every pod has an IP address and that you can communicate with services inside that pod by using that IP address. When I say “overlay network” this is what I mean (“the system that lets you refer to a pod by its IP address”).
|
||||
|
||||
All other Kubernetes networking stuff relies on the overlay networking working correctly. You can read more about the [kubernetes networking model here][10].
|
||||
|
||||
The way Kelsey Hightower describes in [kubernetes the hard way][11] seems pretty good but it’s not really viable on AWS for clusters more than 50 nodes or so, so I’m not going to talk about that.
|
||||
|
||||
There are a lot of overlay network backends (calico, flannel, weaveworks, romana) and the landscape is pretty confusing. But as far as I’m concerned an overlay network has 2 responsibilities:
|
||||
|
||||
1. Make sure your pods can send network requests outside your cluster
|
||||
|
||||
2. Keep a stable mapping of nodes to subnets and keep every node in your cluster updated with that mapping. Do the right thing when nodes are added & removed.
|
||||
|
||||
Okay! So! What can go wrong with your overlay network?
|
||||
|
||||
* The overlay network is responsible for setting up iptables rules (basically `iptables -A -t nat POSTROUTING -s $SUBNET -j MASQUERADE`) to ensure that containers can make network requests outside Kubernetes. If something goes wrong with this rule then your containers can’t connect to the external network. This isn’t that hard (it’s just a few iptables rules) but it is important. I made a [pull request][2] because I wanted to make sure this was resilient
|
||||
|
||||
* Something can go wrong with adding or deleting nodes. We’re using the flannel hostgw backend and at the time we started using it, node deletion [did not work][3].
|
||||
|
||||
* Your overlay network is probably dependent on a distributed database (etcd). If that database has an incident, this can cause issues. For example [https://github.com/coreos/flannel/issues/610][4] says that if you have data loss in your flannel etcd cluster it can result in containers losing network connectivity. (this has now been fixed)
|
||||
|
||||
* You upgrade Docker and everything breaks
|
||||
|
||||
* Probably more things!
|
||||
|
||||
I’m mostly talking about past issues in Flannel here but I promise I’m not picking on Flannel – I actually really **like** Flannel because I feel like it’s relatively simple (for instance the [vxlan backend part of it][12] is like 500 lines of code) and I feel like it’s possible for me to reason through any issues with it. And it’s obviously continuously improving. They’ve been great about reviewing pull requests.
|
||||
|
||||
My approach to operating an overlay network so far has been:
|
||||
|
||||
* Learn how it works in detail and how to debug it (for example the hostgw network backend for Flannel works by creating routes, so you mostly just need to do `sudo ip route list` to see whether it’s doing the correct thing)
|
||||
|
||||
* Maintain an internal build so it’s easy to patch it if needed
|
||||
|
||||
* When there are issues, contribute patches upstream
|
||||
|
||||
I think it’s actually really useful to go through the list of merged PRs and see bugs that have been fixed in the past – it’s a bit time consuming but is a great way to get a concrete list of kinds of issues other people have run into.
|
||||
|
||||
It’s possible that for other people their overlay networks just work but that hasn’t been my experience and I’ve heard other folks report similar issues. If you have an overlay network setup that is a) on AWS and b) works on a cluster more than 50-100 nodes where you feel more confident about operating it I would like to know.
|
||||
|
||||
### Operating kube-proxy and kube-dns?
|
||||
|
||||
Now that we have some thoughts about operating overlay networks, let’s talk about
|
||||
|
||||
There’s a question mark next to this one because I haven’t done this. Here I have more questions than answers.
|
||||
|
||||
Here’s how Kubernetes services work! A service is a collection of pods, which each have their own IP address (like 10.1.0.3, 10.2.3.5, 10.3.5.6)
|
||||
|
||||
1. Every Kubernetes service gets an IP address (like 10.23.1.2)
|
||||
|
||||
2. `kube-dns` resolves Kubernetes service DNS names to IP addresses (so my-svc.my-namespace.svc.cluster.local might map to 10.23.1.2)
|
||||
|
||||
3. `kube-proxy` sets up iptables rules in order to do random load balancing between them. Kube-proxy also has a userspace round-robin load balancer but my impression is that they don’t recommend using it.
|
||||
|
||||
So when you make a request to `my-svc.my-namespace.svc.cluster.local`, it resolves to 10.23.1.2, and then iptables rules on your local host (generated by kube-proxy) redirect it to one of 10.1.0.3 or 10.2.3.5 or 10.3.5.6 at random.
|
||||
|
||||
Some things that I can imagine going wrong with this:
|
||||
|
||||
* `kube-dns` is misconfigured
|
||||
|
||||
* `kube-proxy` dies and your iptables rules don’t get updated
|
||||
|
||||
* Some issue related to maintaining a large number of iptables rules
|
||||
|
||||
Let’s talk about the iptables rules a bit, since doing load balancing by creating a bajillion iptables rules is something I had never heard of before!
|
||||
|
||||
kube-proxy creates one iptables rule per target host like this: (these rules are from [this github issue][13])
|
||||
|
||||
```
|
||||
-A KUBE-SVC-LI77LBOOMGYET5US -m comment --comment "default/showreadiness:showreadiness" -m statistic --mode random --probability 0.20000000019 -j KUBE-SEP-E4QKA7SLJRFZZ2DD[b][c]
|
||||
-A KUBE-SVC-LI77LBOOMGYET5US -m comment --comment "default/showreadiness:showreadiness" -m statistic --mode random --probability 0.25000000000 -j KUBE-SEP-LZ7EGMG4DRXMY26H
|
||||
-A KUBE-SVC-LI77LBOOMGYET5US -m comment --comment "default/showreadiness:showreadiness" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-RKIFTWKKG3OHTTMI
|
||||
-A KUBE-SVC-LI77LBOOMGYET5US -m comment --comment "default/showreadiness:showreadiness" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-CGDKBCNM24SZWCMS
|
||||
-A KUBE-SVC-LI77LBOOMGYET5US -m comment --comment "default/showreadiness:showreadiness" -j KUBE-SEP-RI4SRNQQXWSTGE2Y
|
||||
|
||||
```
|
||||
|
||||
So kube-proxy creates a **lot** of iptables rules. What does that mean? What are the implications of that in for my network? There’s a great talk from Huawei called [Scale Kubernetes to Support 50,000 services][14] that says if you have 5,000 services in your kubernetes cluster, it takes **11 minutes** to add a new rule. If that happened to your real cluster I think it would be very bad.
|
||||
|
||||
I definitely don’t have 5,000 services in my cluster, but 5,000 isn’t SUCH a bit number. The proposal they give to solve this problem is to replace this iptables backend for kube-proxy with IPVS which is a load balancer that lives in the Linux kernel.
|
||||
|
||||
It seems like kube-proxy is going in the direction of various Linux kernel based load balancers. I think this is partly because they support UDP load balancing, and other load balancers (like HAProxy) don’t support UDP load balancing.
|
||||
|
||||
But I feel comfortable with HAProxy! Is it possible to replace kube-proxy with HAProxy! I googled this and I found this [thread on kubernetes-sig-network][15] saying:
|
||||
|
||||
> kube-proxy is so awesome, we have used in production for almost a year, it works well most of time, but as we have more and more services in our cluster, we found it was getting hard to debug and maintain. There is no iptables expert in our team, we do have HAProxy&LVS experts, as we have used these for several years, so we decided to replace this distributed proxy with a centralized HAProxy. I think this maybe useful for some other people who are considering using HAProxy with kubernetes, so we just update this project and make it open source: [https://github.com/AdoHe/kube2haproxy][5]. If you found it’s useful , please take a look and give a try.
|
||||
|
||||
So that’s an interesting option! I definitely don’t have answers here, but, some thoughts:
|
||||
|
||||
* Load balancers are complicated
|
||||
|
||||
* DNS is also complicated
|
||||
|
||||
* If you already have a lot of experience operating one kind of load balancer (like HAProxy), it might make sense to do some extra work to use that instead of starting to use an entirely new kind of load balancer (like kube-proxy)
|
||||
|
||||
* I’ve been thinking about where we want to be using kube-proxy or kube-dns at all – I think instead it might be better to just invest in Envoy and rely entirely on Envoy for all load balancing & service discovery. So then you just need to be good at operating Envoy.
|
||||
|
||||
As you can see my thoughts on how to operate your Kubernetes internal proxies are still pretty confused and I’m still not super experienced with them. It’s totally possible that kube-proxy and kube-dns are fine and that they will just work fine but I still find it helpful to think through what some of the implications of using them are (for example “you can’t have 5,000 Kubernetes services”).
|
||||
|
||||
### Ingress
|
||||
|
||||
If you’re running a Kubernetes cluster, it’s pretty likely that you actually need HTTP requests to get into your cluster so far. This blog post is already too long and I don’t know much about ingress yet so we’re not going to talk about that.
|
||||
|
||||
### Useful links
|
||||
|
||||
A couple of useful links, to summarize:
|
||||
|
||||
* [The Kubernetes networking model][6]
|
||||
|
||||
* How GKE networking works: [https://www.youtube.com/watch?v=y2bhV81MfKQ][7]
|
||||
|
||||
* The aforementioned talk on `kube-proxy` performance: [https://www.youtube.com/watch?v=4-pawkiazEg][8]
|
||||
|
||||
### I think networking operations is important
|
||||
|
||||
My sense of all this Kubernetes networking software is that it’s all still quite new and I’m not sure we (as a community) really know how to operate all of it well. This makes me worried as an operator because I really want my network to keep working! :) Also I feel like as an organization running your own Kubernetes cluster you need to make a pretty large investment into making sure you understand all the pieces so that you can fix things when they break. Which isn’t a bad thing, it’s just a thing.
|
||||
|
||||
My plan right now is just to keep learning about how things work and reduce the number of moving parts I need to worry about as much as possible.
|
||||
|
||||
As usual I hope this was helpful and I would very much like to know what I got wrong in this post!
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://jvns.ca/blog/2017/10/10/operating-a-kubernetes-network/
|
||||
|
||||
作者:[Julia Evans ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://jvns.ca/about
|
||||
[1]:http://blog.sophaskins.net/blog/misadventures-with-kube-dns/
|
||||
[2]:https://github.com/coreos/flannel/pull/808
|
||||
[3]:https://github.com/coreos/flannel/pull/803
|
||||
[4]:https://github.com/coreos/flannel/issues/610
|
||||
[5]:https://github.com/AdoHe/kube2haproxy
|
||||
[6]:https://kubernetes.io/docs/concepts/cluster-administration/networking/#kubernetes-model
|
||||
[7]:https://www.youtube.com/watch?v=y2bhV81MfKQ
|
||||
[8]:https://www.youtube.com/watch?v=4-pawkiazEg
|
||||
[9]:https://jvns.ca/categories/kubernetes
|
||||
[10]:https://kubernetes.io/docs/concepts/cluster-administration/networking/#kubernetes-model
|
||||
[11]:https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/11-pod-network-routes.md
|
||||
[12]:https://github.com/coreos/flannel/tree/master/backend/vxlan
|
||||
[13]:https://github.com/kubernetes/kubernetes/issues/37932
|
||||
[14]:https://www.youtube.com/watch?v=4-pawkiazEg
|
||||
[15]:https://groups.google.com/forum/#!topic/kubernetes-sig-network/3NlBVbTUUU0
|
174
sources/tech/20171011 LEAST PRIVILEGE CONTAINER ORCHESTRATION.md
Normal file
174
sources/tech/20171011 LEAST PRIVILEGE CONTAINER ORCHESTRATION.md
Normal file
@ -0,0 +1,174 @@
|
||||
# LEAST PRIVILEGE CONTAINER ORCHESTRATION
|
||||
|
||||
|
||||
The Docker platform and the container has become the standard for packaging, deploying, and managing applications. In order to coordinate running containers across multiple nodes in a cluster, a key capability is required: a container orchestrator.
|
||||
|
||||

|
||||
|
||||
Orchestrators are responsible for critical clustering and scheduling tasks, such as:
|
||||
|
||||
* Managing container scheduling and resource allocation.
|
||||
|
||||
* Support service discovery and hitless application deploys.
|
||||
|
||||
* Distribute the necessary resources that applications need to run.
|
||||
|
||||
Unfortunately, the distributed nature of orchestrators and the ephemeral nature of resources in this environment makes securing orchestrators a challenging task. In this post, we will describe in detail the less-considered—yet vital—aspect of the security model of container orchestrators, and how Docker Enterprise Edition with its built-in orchestration capability, Swarm mode, overcomes these difficulties.
|
||||
|
||||
Motivation and threat model
|
||||
============================================================
|
||||
|
||||
One of the primary objectives of Docker EE with swarm mode is to provide an orchestrator with security built-in. To achieve this goal, we developed the first container orchestrator designed with the principle of least privilege in mind.
|
||||
|
||||
In computer science,the principle of least privilege in a distributed system requires that each participant of the system must only have access to the information and resources that are necessary for its legitimate purpose. No more, no less.
|
||||
|
||||
> #### ”A process must be able to access only the information and resources that are necessary for its legitimate purpose.”
|
||||
|
||||
#### Principle of Least Privilege
|
||||
|
||||
Each node in a Docker EE swarm is assigned role: either manager or worker. These roles define a coarsegrained level of privilege to the nodes: administration and task execution, respectively. However, regardless of its role, a node has access only to the information and resources it needs to perform the necessary tasks, with cryptographically enforced guarantees. As a result, it becomes easier to secure clusters against even the most sophisticated attacker models: attackers that control the underlying communication networks or even compromised cluster nodes.
|
||||
|
||||
# Secure-by-default core
|
||||
|
||||
There is an old security maxim that states: if it doesn’t come by default, no one will use it. Docker Swarm mode takes this notion to heart, and ships with secure-by-default mechanisms to solve three of the hardest and most important aspects of the orchestration lifecycle:
|
||||
|
||||
1. Trust bootstrap and node introduction.
|
||||
|
||||
2. Node identity issuance and management.
|
||||
|
||||
3. Authenticated, Authorized, Encrypted information storage and dissemination.
|
||||
|
||||
Let’s look at each of these aspects individually
|
||||
|
||||
### Trust Bootstrap and Node Introduction
|
||||
|
||||
The first step to a secure cluster is tight control over membership and identity. Without it, administrators cannot rely on the identities of their nodes and enforce strict workload separation between nodes. This means that unauthorized nodes can’t be allowed to join the cluster, and nodes that are already part of the cluster aren’t able to change identities, suddenly pretending to be another node.
|
||||
|
||||
To address this need, nodes managed by Docker EE’s Swarm mode maintain strong, immutable identities. The desired properties are cryptographically guaranteed by using two key building-blocks:
|
||||
|
||||
1. Secure join tokens for cluster membership.
|
||||
|
||||
2. Unique identities embedded in certificates issued from a central certificate authority.
|
||||
|
||||
### Joining the Swarm
|
||||
|
||||
To join the swarm, a node needs a copy of a secure join token. The token is unique to each operational role within the cluster—there are currently two types of nodes: workers and managers. Due to this separation, a node with a copy of a worker token will not be allowed to join the cluster as a manager. The only way to get this special token is for a cluster administrator to interactively request it from the cluster’s manager through the swarm administration API.
|
||||
|
||||
The token is securely and randomly generated, but it also has a special syntax that makes leaks of this token easier to detect: a special prefix that you can easily monitor for in your logs and repositories. Fortunately, even if a leak does occur, tokens are easy to rotate, and we recommend that you rotate them often—particularly in the case where your cluster will not be scaling up for a while.
|
||||
|
||||

|
||||
|
||||
### Bootstrapping trust
|
||||
|
||||
As part of establishing its identity, a new node will ask for a new identity to be issued by any of the network managers. However, under our threat model, all communications can be intercepted by a third-party. This begs the question: how does a node know that it is talking to a legitimate manager?
|
||||
|
||||

|
||||
|
||||
Fortunately, Docker has a built-in mechanism for preventing this from happening. The join token, which the host uses to join the swarm, includes a hash of the root CA’s certificate. The host can therefore use one-way TLS and use the hash to verify that it’s joining the right swarm: if the manager presents a certificate not signed by a CA that matches the hash, the node knows not to trust it.
|
||||
|
||||
### Node identity issuance and management
|
||||
|
||||
Identities in a swarm are embedded in x509 certificates held by each individual node. In a manifestation of the least privilege principle, the certificates’ private keys are restricted strictly to the hosts where they originate. In particular, managers do not have access to private keys of any certificate but their own.
|
||||
|
||||
### Identity Issuance
|
||||
|
||||
To receive their certificates without sharing their private keys, new hosts begin by issuing a certificate signing request (CSR), which the managers then convert into a certificate. This certificate now becomes the new host’s identity, making the node a full-fledged member of the swarm!
|
||||
|
||||
####
|
||||

|
||||
|
||||
When used alongside with the secure bootstrapping mechanism, this mechanism for issuing identities to joining nodes is secure by default: all communicating parties are authenticated, authorized and no sensitive information is ever exchanged in clear-text.
|
||||
|
||||
### Identity Renewal
|
||||
|
||||
However, securely joining nodes to a swarm is only part of the story. To minimize the impact of leaked or stolen certificates and to remove the complexity of managing CRL lists, Swarm mode uses short-lived certificates for the identities. These certificates have a default expiration of three months, but can be configured to expire every hour!
|
||||
|
||||

|
||||
|
||||
This short certificate expiration time means that certificate rotation can’t be a manual process, as it usually is for most PKI systems. With swarm, all certificates are rotated automatically and in a hitless fashion. The process is simple: using a mutually authenticated TLS connection to prove ownership over a particular identity, a Swarm node generates regularly a new public/private key pair and sends the corresponding CSR to be signed, creating a completely new certificate, but maintaining the same identity.
|
||||
|
||||
### Authenticated, Authorized, Encrypted information storage and dissemination.
|
||||
|
||||
During the normal operation of a swarm, information about the tasks has to be sent to the worker nodes for execution. This includes not only information on which containers are to be executed by a node;but also, it includes all the resources that are necessary for the successful execution of that container, including sensitive secrets such as private keys, passwords, and API tokens.
|
||||
|
||||
### Transport Security
|
||||
|
||||
The fact that every node participating in a swarm is in possession of a unique identity in the form of a X509 certificate, communicating securely between nodes is trivial: nodes can use their respective certificates to establish mutually authenticated connections between one another, inheriting the confidentiality, authenticity and integrity properties of TLS.
|
||||
|
||||

|
||||
|
||||
One interesting detail about Swarm mode is the fact that it uses a push model: only managers are allowed to send information to workers—significantly reducing the surface of attack manager nodes expose to the less privileged worker nodes.
|
||||
|
||||
### Strict Workload Separation Into Security Zones
|
||||
|
||||
One of the responsibilities of manager nodes is deciding which tasks to send to each of the workers. Managers make this determination using a variety of strategies; scheduling the workloads across the swarm depending on both the unique properties of each node and each workload.
|
||||
|
||||
In Docker EE with Swarm mode, administrators have the ability of influencing these scheduling decisions by using labels that are securely attached to the individual node identities. These labels allow administrators to group nodes together into different security zones limiting the exposure of particularly sensitive workloads and any secrets related to them.
|
||||
|
||||

|
||||
|
||||
### Secure Secret Distribution
|
||||
|
||||
In addition to facilitating the identity issuance process, manager nodes have the important task of storing and distributing any resources needed by a worker. Secrets are treated like any other type of resource, and are pushed down from the manager to the worker over the secure mTLS connection.
|
||||
|
||||

|
||||
|
||||
On the hosts, Docker EE ensures that secrets are provided only to the containers they are destined for. Other containers on the same host will not have access to them. Docker exposes secrets to a container as a temporary file system, ensuring that secrets are always stored in memory and never written to disk. This method is more secure than competing alternatives, such as [storing them in environment variables][12]. Once a task completes the secret is gone forever.
|
||||
|
||||
### Storing secrets
|
||||
|
||||
On manager hosts secrets are always encrypted at rest. By default, the key that encrypts these secrets (known as the Data Encryption Key, DEK) is also stored in plaintext on disk. This makes it easy for those with minimal security requirements to start using Docker Swarm mode.
|
||||
|
||||
However, once you are running a production cluster, we recommend you enable auto-lock mode. When auto-lock mode is enabled, a newly rotated DEK is encrypted with a separate Key Encryption Key (KEK). This key is never stored on the cluster; the administrator is responsible for storing it securely and providing it when the cluster starts up. This is known as unlocking the swarm.
|
||||
|
||||
Swarm mode supports multiple managers, relying on the Raft Consensus Algorithm for fault tolerance. Secure secret storage scales seamlessly in this scenario. Each manager host has a unique disk encryption key, in addition to the shared key. Furthermore, Raft logs are encrypted on disk and are similarly unavailable without the KEK when in autolock mode.
|
||||
|
||||
### What happens when a node is compromised?
|
||||
|
||||

|
||||
|
||||
In traditional orchestrators, recovering from a compromised host is a slow and complicated process. With Swarm mode, recovery is as easy as running the docker node rm command. This removes the affected node from the cluster, and Docker will take care of the rest, namely re-balancing services and making sure other hosts know not to talk to the affected node.
|
||||
|
||||
As we have seen, thanks to least privilege orchestration, even if the attacker were still active on the host, they would be cut off from the rest of the network. The host’s certificate — its identity — is blacklisted, so the managers will not accept it as valid.
|
||||
|
||||
# Conclusion
|
||||
|
||||
Docker EE with Swarm mode ensures security by default in all key areas of orchestration:
|
||||
|
||||
* Joining the cluster. Prevents malicious nodes from joining the cluster.
|
||||
|
||||
* Organizing hosts into security zones. Prevents lateral movement by attackers.
|
||||
|
||||
* Scheduling tasks. Tasks will be issued only to designated and allowed nodes.
|
||||
|
||||
* Allocating resources. A malicious node cannot “steal” another’s workload or resources.
|
||||
|
||||
* Storing secrets. Never stored in plaintext and never written to disk on worker nodes.
|
||||
|
||||
* Communicating with the workers. Encrypted using mutually authenticated TLS.
|
||||
|
||||
As Swarm mode continues to improve, the Docker team is working to take the principle of least privilege orchestration even further. The task we are tackling is: how can systems remain secure if a manager is compromised? The roadmap is in place, with some of the features already available such as the ability of whitelisting only specific Docker images, preventing managers from executing arbitrary workloads. This is achieved quite naturally using Docker Content Trust.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://blog.docker.com/2017/10/least-privilege-container-orchestration/
|
||||
|
||||
作者:[Diogo Mónica ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://blog.docker.com/author/diogo/
|
||||
[1]:http://www.linkedin.com/shareArticle?mini=true&url=http://dockr.ly/2yZoNdy&title=Least%20Privilege%20Container%20Orchestration&summary=The%20Docker%20platform%20and%20the%20container%20has%20become%20the%20standard%20for%20packaging,%20deploying,%20and%20managing%20applications.%20In%20order%20to%20coordinate%20running%20containers%20across%20multiple%20nodes%20in%20a%20cluster,%20a%20key%20capability%20is%20required:%20a%20container%20orchestrator.Orchestrators%20are%20responsible%20for%20critical%20clustering%20and%20scheduling%20tasks,%20such%20as:%20%20%20%20Managing%20...
|
||||
[2]:http://www.reddit.com/submit?url=http://dockr.ly/2yZoNdy&title=Least%20Privilege%20Container%20Orchestration
|
||||
[3]:https://plus.google.com/share?url=http://dockr.ly/2yZoNdy
|
||||
[4]:http://news.ycombinator.com/submitlink?u=http://dockr.ly/2yZoNdy&t=Least%20Privilege%20Container%20Orchestration
|
||||
[5]:https://blog.docker.com/author/diogo/
|
||||
[6]:https://blog.docker.com/tag/docker-orchestration/
|
||||
[7]:https://blog.docker.com/tag/docker-secrets/
|
||||
[8]:https://blog.docker.com/tag/docker-security/
|
||||
[9]:https://blog.docker.com/tag/docker-swarm/
|
||||
[10]:https://blog.docker.com/tag/least-privilege-orchestrator/
|
||||
[11]:https://blog.docker.com/tag/tls/
|
||||
[12]:https://diogomonica.com/2017/03/27/why-you-shouldnt-use-env-variables-for-secret-data/
|
@ -1,79 +0,0 @@
|
||||
Translating by FelixYFZ
|
||||
|
||||
Linux Networking Hardware for Beginners: Think Software
|
||||
============================================================
|
||||
|
||||

|
||||
Without routers and bridges, we would be lonely little islands; learn more in this networking tutorial.[Creative Commons Zero][3]Pixabay
|
||||
|
||||
Last week, we learned about [LAN (local area network) hardware][7]. This week, we'll learn about connecting networks to each other, and some cool hacks for mobile broadband.
|
||||
|
||||
### Routers
|
||||
|
||||
Network routers are everything in computer networking, because routers connect networks. Without routers we would be lonely little islands. Figure 1 shows a simple wired LAN (local area network) with a wireless access point, all connected to the Internet. Computers on the LAN connect to an Ethernet switch, which connects to a combination firewall/router, which connects to the big bad Internet through whatever interface your Internet service provider (ISP) provides, such as cable box, DSL modem, satellite uplink...like everything in computing, it's likely to be a box with blinky lights. When your packets leave your LAN and venture forth into the great wide Internet, they travel from router to router until they reach their destination.
|
||||
|
||||
### [fig-1.png][4]
|
||||
|
||||

|
||||
Figure 1: A simple wired LAN with a wireless access point.[Used with permission][1]
|
||||
|
||||
A router can look like pretty much anything: a nice little specialized box that does only routing and nothing else, a bigger box that provides routing, firewall, name services, and VPN gateway, a re-purposed PC or laptop, a Raspberry Pi or Arduino, stout little single-board computers like PC Engines...for all but the most demanding uses, ordinary commodity hardware works fine. The highest-end routers use specialized hardware that is designed to move the maximum number of packets per second. They have multiple fat data buses, multiple CPUs, and super-fast memory. (Look up Juniper and Cisco routers to see what high-end routers look like, and what's inside.)
|
||||
|
||||
A wireless access point connects to your LAN either as an Ethernet bridge or a router. A bridge extends the network, so hosts on both sides of the bridge are on the same network. A router connects two different networks.
|
||||
|
||||
### Network Topology
|
||||
|
||||
There are multitudes of ways to set up your LAN. You can put all hosts on a single flat network. You can divide it up into different subnets. You can divide it into virtual LANs, if your switch supports this.
|
||||
|
||||
A flat network is the simplest; just plug everyone into the same switch. If one switch isn't enough you can connect switches to each other. Some switches have special uplink ports, some don't care which ports you connect, and you may need to use a crossover Ethernet cable, so check your switch documentation.
|
||||
|
||||
Flat networks are the easiest to administer. You don't need routers and don't have to calculate subnets, but there are some downsides. They don't scale, so when they get too large they get bogged down by broadcast traffic. Segmenting your LAN provides a bit of security, and makes it easier to manage larger networks by dividing it into manageable chunks. Figure 2 shows a simplified LAN divided into two subnets: internal wired and wireless hosts, and one for servers that host public services. The subnet that contains the public-facing servers is called a DMZ, demilitarized zone (ever notice all the macho terminology for jobs that are mostly typing on a computer?) because it is blocked from all internal access.
|
||||
|
||||
### [fig-2.png][5]
|
||||
|
||||

|
||||
Figure 2: A simplified LAN divided into two subnets.[Used with permission][2]
|
||||
|
||||
Even in a network as small as Figure 2 there are several ways to set it up. You can put your firewall and router on a single device. You could have a dedicated Internet link for the DMZ, divorcing it completely from your internal network. Which brings us to our next topic: it's all software.
|
||||
|
||||
### Think Software
|
||||
|
||||
You may have noticed that of the hardware we have discussed in this little series, only network interfaces, switches, and cabling are special-purpose hardware. Everything else is general-purpose commodity hardware, and it's the software that defines its purpose. Linux is a true networking operating system, and it supports a multitude of network operations: VLANs, firewall, router, Internet gateway, VPN gateway, Ethernet bridge, Web/mail/file/etc. servers, load-balancer, proxy, quality of service, multiple authenticators, trunking, failover...you can run your entire network on commodity hardware with Linux. You can even use Linux to simulate an Ethernet switch with LISA (LInux Switching Appliance) and vde2.
|
||||
|
||||
There are specialized distributions for small hardware like DD-WRT, OpenWRT, and the Raspberry Pi distros, and don't forget the BSDs and their specialized offshoots like the pfSense firewall/router, and the FreeNAS network-attached storage server.
|
||||
|
||||
You know how some people insist there is a difference between a hardware firewall and a software firewall? There isn't. That's like saying there is a hardware computer and a software computer.
|
||||
|
||||
### Port Trunking and Ethernet Bonding
|
||||
|
||||
Trunking and bonding, also called link aggregation, is combining two Ethernet channels into one. Some Ethernet switches support port trunking, which is combining two switch ports to combine their bandwidth into a single link. This is a nice way to make a bigger pipe to a busy server.
|
||||
|
||||
You can do the same thing with Ethernet interfaces, and the bonding driver is built-in to the Linux kernel, so you don't need any special hardware.
|
||||
|
||||
### Bending Mobile Broadband to your Will
|
||||
|
||||
I expect that mobile broadband is going to grow in the place of DSL and cable Internet. I live near a city of 250,000 population, but outside the city limits good luck getting Internet, even though there is a large population to serve. My little corner of the world is 20 minutes from town, but it might as well be the moon as far as Internet service providers are concerned. My only option is mobile broadband; there is no dialup, satellite Internet is sold out (and it sucks), and haha lol DSL, cable, or fiber. That doesn't stop ISPs from stuffing my mailbox with flyers for Xfinity and other high-speed services my area will never see.
|
||||
|
||||
I tried AT&T, Verizon, and T-Mobile. Verizon has the strongest coverage, but Verizon and AT&T are expensive. I'm at the edge of T-Mobile coverage, but they give the best deal by far. To make it work, I had to buy a weBoost signal booster and ZTE mobile hotspot. Yes, you can use a smartphone as a hotspot, but the little dedicated hotspots have stronger radios. If you're thinking you might want a signal booster, I have nothing but praise for weBoost because their customer support is superb, and they will do their best to help you. Set it up with the help of a great little app that accurately measures signal strength, [SignalCheck Pro][8]. They have a free version with fewer features; spend the two bucks to get the pro version, you won't be sorry.
|
||||
|
||||
The little ZTE hotspots serve up to 15 hosts and have rudimentary firewalls. But we can do better: get something like the Linksys WRT54GL, replace the stock firmware with Tomato, OpenWRT, or DD-WRT, and then you have complete control of your firewall rules, routing, and any other services you want to set up.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linux.com/learn/intro-to-linux/2017/10/linux-networking-hardware-beginners-think-software
|
||||
|
||||
作者:[CARLA SCHRODER][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linux.com/users/cschroder
|
||||
[1]:https://www.linux.com/licenses/category/used-permission
|
||||
[2]:https://www.linux.com/licenses/category/used-permission
|
||||
[3]:https://www.linux.com/licenses/category/creative-commons-zero
|
||||
[4]:https://www.linux.com/files/images/fig-1png-7
|
||||
[5]:https://www.linux.com/files/images/fig-2png-4
|
||||
[6]:https://www.linux.com/files/images/soderskar-islandjpg
|
||||
[7]:https://www.linux.com/learn/intro-to-linux/2017/10/linux-networking-hardware-beginners-lan-hardware
|
||||
[8]:http://www.bluelinepc.com/signalcheck/
|
@ -1,83 +0,0 @@
|
||||
apply for translating
|
||||
|
||||
How Eclipse is advancing IoT development
|
||||
============================================================
|
||||
|
||||
### Open source organization's modular approach to development is a good match for the Internet of Things.
|
||||
|
||||

|
||||
Image by : opensource.com
|
||||
|
||||
[Eclipse][3] may not be the first open source organization that pops to mind when thinking about Internet of Things (IoT) projects. After all, the foundation has been around since 2001, long before IoT was a household word, supporting a community for commercially viable open source software development.
|
||||
|
||||
September's Eclipse IoT Day, held in conjunction with RedMonk's [ThingMonk 2017][4] event, emphasized the big role Eclipse is taking in [IoT development][5]. It currently hosts 28 projects that touch a wide range of IoT needs and projects. While at the conference, I talked with [Ian Skerritt][6], who heads marketing for Eclipse, about Eclipse's IoT projects and how Eclipse thinks about IoT more broadly.
|
||||
|
||||
### What's new about IoT?
|
||||
|
||||
I asked Ian how IoT is different from traditional industrial automation, given that sensors and tools have been connected in factories for the past several decades. Ian notes that many factories still are not connected.
|
||||
|
||||
Additionally, he says, "SCADA [supervisory control and data analysis] systems and even the factory floor technology are very proprietary, very siloed. It's hard to change it. It's hard to adapt to it… Right now, when you set up a manufacturing run, you need to manufacture hundreds of thousands of that piece, of that unit. What [manufacturers] want to do is to meet customer demand, to have manufacturing processes that are very flexible, that you can actually do a lot size of one." That's a big piece of what IoT is bringing to manufacturing.
|
||||
|
||||
### Eclipse's approach to IoT
|
||||
|
||||
He describes Eclipse's involvement in IoT by saying: "There's core fundamental technology that every IoT solution needs," and by using open source, "everyone can use it so they can get broader adoption." He says Eclipse see IoT as consisting of three connected software stacks. At a high level, these stacks mirror the (by now familiar) view that IoT can usually be described as spanning three layers. A given implementation may have even more layers, but they still generally map to the functions of this three-layer model:
|
||||
|
||||
* A stack of software for constrained devices (e.g., the device, endpoint, microcontroller unit (MCU), sensor hardware).
|
||||
|
||||
* Some type of gateway that aggregates information and data from the different sensors and sends it to the network. This layer also may take real-time actions based on what the sensors are observing.
|
||||
|
||||
* A software stack for the IoT platform on the backend. This backend cloud stores the data and can provide services based on collected data, such as analysis of historical trends and predictive analytics.
|
||||
|
||||
The three stacks are described in greater detail in Eclipse's whitepaper "[The Three Software Stacks Required for IoT Architectures][7]."
|
||||
|
||||
Ian says that, when developing a solution within those architectures, "there's very specific things that need to be built, but there's a lot of underlying technology that can be used, like messaging protocols, like gateway services. It needs to be a modular approach to scale up to the different use cases that are up there." This encapsulates Eclipse's activities around IoT: Developing modular open source components that can be used to build a range of business-specific services and solutions.
|
||||
|
||||
### Eclipse's IoT projects
|
||||
|
||||
Of Eclipse's many IoT projects currently in use, Ian says two of the most prominent relate to [MQTT][8], a machine-to-machine (M2M) messaging protocol for IoT. Ian describes it as "a publish‑subscribe messaging protocol that was designed specifically for oil and gas pipeline monitoring where power-management network latency is really important. MQTT has been a great success in terms of being a standard that's being widely adopted in IoT." [Eclipse Mosquitto][9] is MQTT's broker and [Eclipse Paho][10] its client.
|
||||
|
||||
[Eclipse Kura][11] is an IoT gateway that, in Ian's words, "provides northbound and southbound connectivity [for] a lot of different protocols" including Bluetooth, Modbus, controller-area network (CAN) bus, and OPC Unified Architecture, with more being added all the time. One benefit, he says, is "instead of you writing your own connectivity, Kura provides that and then connects you to the network via satellite, via Ethernet, or anything." In addition, it handles firewall configuration, network latency, and other functions. "If the network goes down, it will store messages until it comes back up," Ian says.
|
||||
|
||||
A newer project, [Eclipse Kapua][12], is taking a microservices approach to providing different services for an IoT cloud platform. For example, it handles aspects of connectivity, integration, management, storage, and analysis. Ian describes it as "up and coming. It's not being deployed yet, but Eurotech and Red Hat are very active in that."
|
||||
|
||||
Ian says [Eclipse hawkBit][13], which manages software updates, is one of the "most intriguing projects. From a security perspective, if you can't update your device, you've got a huge security hole." Most IoT security disasters are related to non-updated devices, he says. "HawkBit basically manages the backend of how you do scalable updates across your IoT system."
|
||||
|
||||
Indeed, the difficulty of updating software in IoT devices is regularly cited as one of its biggest security challenges. IoT devices aren't always connected and may be numerous, plus update processes for constrained devices can be hard to consistently get right. For this reason, projects relating to updating IoT software are likely to be important going forward.
|
||||
|
||||
### Why IoT is a good fit for Eclipse
|
||||
|
||||
One of the trends we've seen in IoT development has been around building blocks that are integrated and applied to solve particular business problems, rather than monolithic IoT platforms that apply across industries and companies. This is a good fit with Eclipse's approach to IoT, which focuses on a number of modular stacks; projects that provide specific and commonly needed functions; and brokers, gateways, and protocols that can tie together the components needed for a given implementation.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
作者简介:
|
||||
|
||||
Gordon Haff - Gordon Haff is Red Hat’s cloud evangelist, is a frequent and highly acclaimed speaker at customer and industry events, and helps develop strategy across Red Hat’s full portfolio of cloud solutions. He is the author of Computing Next: How the Cloud Opens the Future in addition to numerous other publications. Prior to Red Hat, Gordon wrote hundreds of research notes, was frequently quoted in publications like The New York Times on a wide range of IT topics, and advised clients on product and...
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/17/10/eclipse-and-iot
|
||||
|
||||
作者:[Gordon Haff ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://opensource.com/users/ghaff
|
||||
[1]:https://opensource.com/article/17/10/eclipse-and-iot?rate=u1Wr-MCMFCF4C45IMoSPUacCatoqzhdKz7NePxHOvwg
|
||||
[2]:https://opensource.com/user/21220/feed
|
||||
[3]:https://www.eclipse.org/home/
|
||||
[4]:http://thingmonk.com/
|
||||
[5]:https://iot.eclipse.org/
|
||||
[6]:https://twitter.com/ianskerrett
|
||||
[7]:https://iot.eclipse.org/resources/white-papers/Eclipse%20IoT%20White%20Paper%20-%20The%20Three%20Software%20Stacks%20Required%20for%20IoT%20Architectures.pdf
|
||||
[8]:http://mqtt.org/
|
||||
[9]:https://projects.eclipse.org/projects/technology.mosquitto
|
||||
[10]:https://projects.eclipse.org/projects/technology.paho
|
||||
[11]:https://www.eclipse.org/kura/
|
||||
[12]:https://www.eclipse.org/kapua/
|
||||
[13]:https://eclipse.org/hawkbit/
|
||||
[14]:https://opensource.com/users/ghaff
|
||||
[15]:https://opensource.com/users/ghaff
|
||||
[16]:https://opensource.com/article/17/10/eclipse-and-iot#comments
|
@ -1,37 +0,0 @@
|
||||
A block layer introduction part 1: the bio layer
|
||||
============================================================
|
||||
|
||||
### A block layer introduction part 1: the bio layer
|
||||
|
||||
In reply to: [A block layer introduction part 1: the bio layer][1] by amarao
|
||||
Parent article: [A block layer introduction part 1: the bio layer][2]Hi,
|
||||
the problem you describe here is not directly related to the block layer. It is probably a driver bug, possible a SCSI-layer bug, but definitely not a block-layer problem.
|
||||
Reporting bugs against Linux is, unfortunately, a bit of a hit-and-miss affair. Some developers refused to touch bugzilla, some love it, and some (like me) only use it begrudgingly.
|
||||
The alternative is to send email. For that you need to choose the right list and maybe the right developer, and you need to catch them when they are in a good mood or aren't too busy or not on holidays. Some people will make an effort to respond to everything, others are completely unpredictable - and that is for me who usually sends a patch with any bug report. If you just have a bug that you barely understand yourself, your expected response rate is probably lower. Sad, but true.
|
||||
|
||||
Lots of bugs do get responded to and dealt with, but lots do not.
|
||||
|
||||
I don't think it is fair to say that nobody cares, but it probably is true that nobody sees it as being as important as you do. If you want a solution, then you need to drive it. One way to drive it is to spend money on a consultant or with a support contract from a distributor. I suspect that isn't possible in your situation. Another way is to learn how the code works and find a solution yourself. Lots of people do that, but again it might not be an option for you. Another way is to keep raising the issue on different relevant forums until you get a response. Persistence can bear fruit. You would need to be prepared to perform whatever testing is asked of you, possibly including building a new kernel to test.
|
||||
|
||||
If you are able to reproduce this problem on a recent kernel (4.12 or later) I suggest that you email a report to
|
||||
linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, and me (neilb@suse.com) (note that you do not need to subscribe to these lists to send mail, just send it). Describe the hardware and how to trigger the problem.
|
||||
Include the stack trace of any process in "D" state. You can get this with
|
||||
cat /proc/$PID/stack
|
||||
where "$PID" is the pid of the process.
|
||||
|
||||
Be sure to avoid complaining or saying how this has been broken for years and how it is grossly inadequate. Nobody cares about that. We do care about bugs and generally want to fix them. So just report the relevant facts.
|
||||
Try to include all facts in the mail rather than via links to somewhere else. Sometimes links are necessary, but in the case of your script, it is 8 lines long so just include it in the email (and avoid descriptions like "fuckup"; just call it "broken" or similar). Also make sure your email isn't sent as HTML. We like just plain text. HTML is rejected by all @vger.kernel.org mailing lists. You might need to configure your email program to not send HTML.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://lwn.net/Articles/737655/
|
||||
|
||||
作者:[ neilbrown][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://lwn.net/Articles/737655/
|
||||
[1]:https://lwn.net/Articles/737588/
|
||||
[2]:https://lwn.net/Articles/736534/
|
@ -1,239 +0,0 @@
|
||||
How to use SVG as a Placeholder, and Other Image Loading Techniques
|
||||
============================================================
|
||||
|
||||

|
||||
Generating SVGs from images can be used for placeholders. Keep reading!
|
||||
|
||||
I’m passionate about image performance optimisation and making images load fast on the web. One of the most interesting areas of exploration is placeholders: what to show when the image hasn’t loaded yet.
|
||||
|
||||
During the last days I have come across some loading techniques that use SVG, and I would like to describe them in this post.
|
||||
|
||||
In this post we will go through these topics:
|
||||
|
||||
* Overview of different types of placeholders
|
||||
|
||||
* SVG-based placeholders (edges, shapes and silhouettes)
|
||||
|
||||
* Automating the process.
|
||||
|
||||
### Overview of different types of placeholders
|
||||
|
||||
In the past [I have written about placeholders and lazy-load of images][28], and also [talked about it][29]. When doing lazy-loading of images it’s a good idea to think about what to render as a placeholder, since it can have a big impact in user’s perceived performance. In the past I described several options:
|
||||
|
||||
|
||||

|
||||
|
||||
Several strategies to fill the area of an image before it loads.
|
||||
|
||||
* Keeping the space empty for the image: In a world of responsive design, this prevents content from jumping around. Those layout changes are bad from a user’s experience point of view, but also for performance. The browser is forced to do layout re calculations every time it fetches the dimensions of an image, leaving space for it.
|
||||
|
||||
* Placeholder: Imagine that we are displaying a user’s profile image. We might want to display a silhouette in the background. This is shown while the main image is loaded, but also when that request failed or when the user didn’t set any profile picture at all. These images are usually vector-based, and due to their small size are a good candidate to be inlined.
|
||||
|
||||
* Solid colour: Take a colour from the image and use it as the background colour for the placeholder. This can be the dominant colour, the most vibrant… The idea is that it is based on the image you are loading and should help making the transition between no image to image loaded smoother.
|
||||
|
||||
* Blurry image: Also called blur-up technique. You render a tiny version of the image and then transition to the full one. The initial image is tiny both in pixels and kBs. To remove artifacts the image is scaled up and blurred. I have written previously about this on [How Medium does progressive image loading][1], [Using WebP to create tiny preview images][2], and [More examples of Progressive Image Loading][3] .
|
||||
|
||||
Turns out there are many other variations and lots of smart people are developing other techniques to create placeholders.
|
||||
|
||||
One of them is having gradients instead of solid colours. The gradients can create a more accurate preview of the final image, with very little overhead (increase in payload).
|
||||
|
||||
|
||||

|
||||
Using gradients as backgrounds. Screenshot from Gradify, which is not online anymore. Code [on GitHub][4].
|
||||
|
||||
Another technique is using SVGs based on the image, which is getting some traction with recent experiments and hacks.
|
||||
|
||||
### SVG-based placeholders
|
||||
|
||||
We know SVGs are ideal for vector images. In most cases we want to load a bitmap one, so the question is how to vectorise an image. Some options are using edges, shapes and areas.
|
||||
|
||||
#### Edges
|
||||
|
||||
In [a previous post][30] I explained how to find out the edges of an image and create an animation. My initial goal was to try to draw regions, vectorising the image, but I didn’t know how to do it. I realised that using the edges could also be innovative and I decided to animate them creating a “drawing” effect.
|
||||
|
||||
[Drawing images using edge detection and SVG animation
|
||||
Back in the days SVG was barely used and supported. Some time after we started using them as an alternative to classic…medium.com][31][][32]
|
||||
|
||||
#### Shapes
|
||||
|
||||
SVG can also be used to draw areas from the image instead of edges/borders. In a way, we would vectorise a bitmap image to create a placeholder.
|
||||
|
||||
Back in the days I tried to do something similar with triangles. You can see the result in my talks [at CSSConf][33] and [Render Conf][34].
|
||||
|
||||
|
||||
The codepen above is a proof of concept of a SVG-based placeholder composed of 245 triangles. The generation of the triangles is based on [Delaunay triangulation][35] using [Possan’s polyserver][36]. As expected, the more triangles the SVG uses, the bigger the file size.
|
||||
|
||||
#### Primitive and SQIP, a SVG-based LQIP technique
|
||||
|
||||
Tobias Baldauf has been working on another Low-Quality Image Placeholder technique using SVGs called [SQIP][37]. Before digging into SQIP itself I will give an overview of [Primitive][38], a library on which SQIP is based.
|
||||
|
||||
Primitive is quite fascinating and I definitely recommend you to check it out. It converts a bitmap image into a SVG composed of overlapping shapes. Its small size makes it suitable for inlining it straight into the page. One less roundtrip, and a meaningful placeholder within the initial HTML payload.
|
||||
|
||||
Primitive generates an image based on shapes like triangles, rectangles and circles (and a few others). In every step it adds a new one. The more steps, the resulting image looks closer to the original one. If your output is SVG it also means the size of the output code will be larger.
|
||||
|
||||
In order to understand how Primitive works, I ran it through a couple of images. I generated SVGs for the artwork using 10 shapes and 100 shapes:
|
||||
|
||||
** 此处有Canvas,请手动处理 **
|
||||
|
||||

|
||||
|
||||
|
||||

|
||||
|
||||

|
||||
Processing [this picture][5] using Primitive, using [10 shapes][6] and [100 shapes][7].
|
||||
|
||||
|
||||

|
||||
|
||||
|
||||
|
||||

|
||||
|
||||
|
||||

|
||||
Processing [this picture][8] using Primitive, using [10 shapes][9] and [100 shapes][10].
|
||||
|
||||
When using 10 shapes the images we start getting a grasp of the original image. In the context of image placeholders there is potential to use this SVG as the placeholder. Actually, the code for the SVG with 10 shapes is really small, around 1030 bytes, which goes down to ~640 bytes when passing the output through SVGO.
|
||||
|
||||
```
|
||||
<svg xmlns=”http://www.w3.org/2000/svg" width=”1024" height=”1024"><path fill=”#817c70" d=”M0 0h1024v1024H0z”/><g fill-opacity=”.502"><path fill=”#03020f” d=”M178 994l580 92L402–62"/><path fill=”#f2e2ba” d=”M638 894L614 6l472 440"/><path fill=”#fff8be” d=”M-62 854h300L138–62"/><path fill=”#76c2d9" d=”M410–62L154 530–62 38"/><path fill=”#62b4cf” d=”M1086–2L498–30l484 508"/><path fill=”#010412" d=”M430–2l196 52–76 356"/><path fill=”#eb7d3f” d=”M598 594l488–32–308 520"/><path fill=”#080a18" d=”M198 418l32 304 116–448"/><path fill=”#3f201d” d=”M1086 1062l-344–52 248–148"/><path fill=”#ebd29f” d=”M630 658l-60–372 516 320"/></g></svg>
|
||||
```
|
||||
|
||||
The images generated with 100 shapes are larger, as expected, weighting ~5kB after SVGO (8kB before). They have a great level of detail with a still small payload. The decision of how many triangles to use will depend largely on the type of image (eg contrast, amount of colours, complexity) and level of detail.
|
||||
|
||||
It would be possible to create a script similar to [cpeg-dssim][39] that tweaks the amount of shapes used until a [structural similarity][40] threshold is met (or a maximum number of shapes in the worst case).
|
||||
|
||||
These resulting SVGs are great also to use as background images. Being size-constrained and vector-based they are a good candidate for hero images and large backgrounds that otherwise would show artifacts.
|
||||
|
||||
#### SQIP
|
||||
|
||||
In [Tobias’ own words][41]:
|
||||
|
||||
> SQIP is an attempt to find a balance between these two extremes: it makes use of [Primitive][42] to generate a SVG consisting of several simple shapes that approximate the main features visible inside the image, optimizes the SVG using [SVGO][43] and adds a Gaussian Blur filter to it. This produces a SVG placeholder which weighs in at only ~800–1000 bytes, looks smooth on all screens and provides an visual cue of image contents to come.
|
||||
|
||||
The result is similar to using a tiny placeholder image for the blur-up technique (what [Medium][44] and [other sites][45] do). The difference is that instead of using a bitmap image, eg JPG or WebP, the placeholder is SVG.
|
||||
|
||||
If we run SQIP against the original images we’ll get this:
|
||||
|
||||
|
||||

|
||||
|
||||
|
||||
|
||||

|
||||
The output images using SQIP for [the first picture][11] and [the second one][12].
|
||||
|
||||
The output SVG is ~900 bytes, and inspecting the code we can spot the `feGaussianBlur` filter applied to the group of shapes:
|
||||
|
||||
```
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 2000 2000"><filter id="b"><feGaussianBlur stdDeviation="12" /></filter><path fill="#817c70" d="M0 0h2000v2000H0z"/><g filter="url(#b)" transform="translate(4 4) scale(7.8125)" fill-opacity=".5"><ellipse fill="#000210" rx="1" ry="1" transform="matrix(50.41098 -3.7951 11.14787 148.07886 107 194.6)"/><ellipse fill="#eee3bb" rx="1" ry="1" transform="matrix(-56.38179 17.684 -24.48514 -78.06584 205 110.1)"/><ellipse fill="#fff4bd" rx="1" ry="1" transform="matrix(35.40604 -5.49219 14.85017 95.73337 16.4 123.6)"/><ellipse fill="#79c7db" cx="21" cy="39" rx="65" ry="65"/><ellipse fill="#0c1320" cx="117" cy="38" rx="34" ry="47"/><ellipse fill="#5cb0cd" rx="1" ry="1" transform="matrix(-39.46201 77.24476 -54.56092 -27.87353 219.2 7.9)"/><path fill="#e57339" d="M271 159l-123–16 43 128z"/><ellipse fill="#47332f" cx="214" cy="237" rx="242" ry="19"/></g></svg>
|
||||
```
|
||||
|
||||
SQIP can also output an image tag with the SVG contents Base 64 encoded:
|
||||
|
||||
```
|
||||
<img width="640" height="640" src="example.jpg” alt="Add descriptive alt text" style="background-size: cover; background-image: url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAw…<stripped base 64>…PjwvZz48L3N2Zz4=);">
|
||||
```
|
||||
|
||||
#### Silhouettes
|
||||
|
||||
We just had a look at using SVGs for edges and primitive shapes. Another possibility is to vectorise the images “tracing” them. [Mikael Ainalem][47] shared [a codepen][48] a few days ago showing how to use a 2-colour silhouette as a placeholder. The result is really pretty:
|
||||
|
||||
|
||||

|
||||
|
||||
The SVGs in this case were hand drawn, but the technique quickly spawned integrations with tools to automate the process.
|
||||
|
||||
* [Gatsby][13], a static site generator using React supports these traced SVGs now. It uses [a JS PORT of potrace][14] to vectorise the images.
|
||||
|
||||
* [Craft 3 CMS][15], which also added support for silhouettes. It uses [a PHP port of potrace][16].
|
||||
|
||||
|
||||
* [image-trace-loader][17], a Webpack loader that uses potrace to process the images.
|
||||
|
||||
|
||||
It’s also interesting to see a comparison of the output between Emil’s webpack loader (based on potrace) and Mikael’s hand-drawn SVGs.
|
||||
|
||||
|
||||
I assume the output generated by potrace is using the default options. However, it’s possible to tweak them. Check [the options for image-trace-loader][49], which are pretty much [the ones passed down to potrace][50].
|
||||
|
||||
### Summary
|
||||
|
||||
We have seen different tools and techniques to generate SVGs from images and use them as placeholders. The same way [WebP is a fantastic format for thumbnails][51], SVG is also an interesting format to use in placeholders. We can control the level of detail (and thus, size), it’s highly compressible and easy to manipulate with CSS and JS.
|
||||
|
||||
#### Extra Resources
|
||||
|
||||
This post made it to [the top of Hacker News][52]. I’m very grateful for that, and for all the links to other resources that have been shared in the comments on that page. Here are a few of them!
|
||||
|
||||
* [Geometrize][18] is a port of Primitive written in Haxe. There is also [a JS implementation][19] that you can try out directly [on your browser][20].
|
||||
|
||||
* [Primitive.js][21], which is a port of Primitive in JS. Also, [primitive.nextgen][22], which is a port of the Primitive desktop app using Primitive.js and Electron.
|
||||
|
||||
* There are a couple of Twitter accounts where you can see examples of images generated with Primitive and Geometrize. Check out [@PrimitivePic][23] and [@Geometrizer][24].
|
||||
|
||||
* [imagetracerjs][25], which is a raster image tracer and vectorizer written in JavaScript. There are also ports for [Java][26] and [Android][27].
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://medium.freecodecamp.org/using-svg-as-placeholders-more-image-loading-techniques-bed1b810ab2c
|
||||
|
||||
作者:[ José M. Pérez][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://medium.freecodecamp.org/@jmperezperez?source=post_header_lockup
|
||||
[1]:https://medium.com/@jmperezperez/how-medium-does-progressive-image-loading-fd1e4dc1ee3d
|
||||
[2]:https://medium.com/@jmperezperez/using-webp-to-create-tiny-preview-images-3e9b924f28d6
|
||||
[3]:https://medium.com/@jmperezperez/more-examples-of-progressive-image-loading-f258be9f440b
|
||||
[4]:https://github.com/fraser-hemp/gradify
|
||||
[5]:https://jmperezperez.com/assets/images/posts/svg-placeholders/pexels-photo-281184-square.jpg
|
||||
[6]:https://jmperezperez.com/assets/images/posts/svg-placeholders/pexels-photo-281184-square-10.svg
|
||||
[7]:https://jmperezperez.com/assets/images/posts/svg-placeholders/pexels-photo-281184-square-100.svg
|
||||
[8]:https://jmperezperez.com/assets/images/posts/svg-placeholders/pexels-photo-618463-square.jpg
|
||||
[9]:https://jmperezperez.com/assets/images/posts/svg-placeholders/pexels-photo-618463-square-10.svg
|
||||
[10]:https://jmperezperez.com/assets/images/posts/svg-placeholders/pexels-photo-618463-square-100.svg
|
||||
[11]:https://jmperezperez.com/assets/images/posts/svg-placeholders/pexels-photo-281184-square-sqip.svg
|
||||
[12]:https://jmperezperez.com/svg-placeholders/%28/assets/images/posts/svg-placeholders/pexels-photo-618463-square-sqip.svg
|
||||
[13]:https://www.gatsbyjs.org/
|
||||
[14]:https://www.npmjs.com/package/potrace
|
||||
[15]:https://craftcms.com/
|
||||
[16]:https://github.com/nystudio107/craft3-imageoptimize/blob/master/src/lib/Potracio.php
|
||||
[17]:https://github.com/EmilTholin/image-trace-loader
|
||||
[18]:https://github.com/Tw1ddle/geometrize-haxe
|
||||
[19]:https://github.com/Tw1ddle/geometrize-haxe-web
|
||||
[20]:http://www.samcodes.co.uk/project/geometrize-haxe-web/
|
||||
[21]:https://github.com/ondras/primitive.js
|
||||
[22]:https://github.com/cielito-lindo-productions/primitive.nextgen
|
||||
[23]:https://twitter.com/PrimitivePic
|
||||
[24]:https://twitter.com/Geometrizer
|
||||
[25]:https://github.com/jankovicsandras/imagetracerjs
|
||||
[26]:https://github.com/jankovicsandras/imagetracerjava
|
||||
[27]:https://github.com/jankovicsandras/imagetracerandroid
|
||||
[28]:https://medium.com/@jmperezperez/lazy-loading-images-on-the-web-to-improve-loading-time-and-saving-bandwidth-ec988b710290
|
||||
[29]:https://www.youtube.com/watch?v=szmVNOnkwoU
|
||||
[30]:https://medium.com/@jmperezperez/drawing-images-using-edge-detection-and-svg-animation-16a1a3676d3
|
||||
[31]:https://medium.com/@jmperezperez/drawing-images-using-edge-detection-and-svg-animation-16a1a3676d3
|
||||
[32]:https://medium.com/@jmperezperez/drawing-images-using-edge-detection-and-svg-animation-16a1a3676d3
|
||||
[33]:https://jmperezperez.com/cssconfau16/#/45
|
||||
[34]:https://jmperezperez.com/renderconf17/#/46
|
||||
[35]:https://en.wikipedia.org/wiki/Delaunay_triangulation
|
||||
[36]:https://github.com/possan/polyserver
|
||||
[37]:https://github.com/technopagan/sqip
|
||||
[38]:https://github.com/fogleman/primitive
|
||||
[39]:https://github.com/technopagan/cjpeg-dssim
|
||||
[40]:https://en.wikipedia.org/wiki/Structural_similarity
|
||||
[41]:https://github.com/technopagan/sqip
|
||||
[42]:https://github.com/fogleman/primitive
|
||||
[43]:https://github.com/svg/svgo
|
||||
[44]:https://medium.com/@jmperezperez/how-medium-does-progressive-image-loading-fd1e4dc1ee3d
|
||||
[45]:https://medium.com/@jmperezperez/more-examples-of-progressive-image-loading-f258be9f440b
|
||||
[46]:http://www.w3.org/2000/svg
|
||||
[47]:https://twitter.com/mikaelainalem
|
||||
[48]:https://codepen.io/ainalem/full/aLKxjm/
|
||||
[49]:https://github.com/EmilTholin/image-trace-loader#options
|
||||
[50]:https://www.npmjs.com/package/potrace#parameters
|
||||
[51]:https://medium.com/@jmperezperez/using-webp-to-create-tiny-preview-images-3e9b924f28d6
|
||||
[52]:https://news.ycombinator.com/item?id=15696596
|
@ -0,0 +1,711 @@
|
||||
Dive into BPF: a list of reading material
|
||||
============================================================
|
||||
|
||||
* [What is BPF?][143]
|
||||
|
||||
* [Dive into the bytecode][144]
|
||||
|
||||
* [Resources][145]
|
||||
* [Generic presentations][23]
|
||||
* [About BPF][1]
|
||||
|
||||
* [About XDP][2]
|
||||
|
||||
* [About other components related or based on eBPF][3]
|
||||
|
||||
* [Documentation][24]
|
||||
* [About BPF][4]
|
||||
|
||||
* [About tc][5]
|
||||
|
||||
* [About XDP][6]
|
||||
|
||||
* [About P4 and BPF][7]
|
||||
|
||||
* [Tutorials][25]
|
||||
|
||||
* [Examples][26]
|
||||
* [From the kernel][8]
|
||||
|
||||
* [From package iproute2][9]
|
||||
|
||||
* [From bcc set of tools][10]
|
||||
|
||||
* [Manual pages][11]
|
||||
|
||||
* [The code][27]
|
||||
* [BPF code in the kernel][12]
|
||||
|
||||
* [XDP hooks code][13]
|
||||
|
||||
* [BPF logic in bcc][14]
|
||||
|
||||
* [Code to manage BPF with tc][15]
|
||||
|
||||
* [BPF utilities][16]
|
||||
|
||||
* [Other interesting chunks][17]
|
||||
|
||||
* [LLVM backend][18]
|
||||
|
||||
* [Running in userspace][19]
|
||||
|
||||
* [Commit logs][20]
|
||||
|
||||
* [Troubleshooting][28]
|
||||
* [Errors at compilation time][21]
|
||||
|
||||
* [Errors at load and run time][22]
|
||||
|
||||
* [And still more!][29]
|
||||
|
||||
_~ [Updated][146] 2017-11-02 ~_
|
||||
|
||||
# What is BPF?
|
||||
|
||||
BPF, as in **B**erkeley **P**acket **F**ilter, was initially conceived in 1992 so as to provide a way to filter packets and to avoid useless packet copies from kernel to userspace. It initially consisted in a simple bytecode that is injected from userspace into the kernel, where it is checked by a verifier—to prevent kernel crashes or security issues—and attached to a socket, then run on each received packet. It was ported to Linux a couple of years later, and used for a small number of applications (tcpdump for example). The simplicity of the language as well as the existence of an in-kernel Just-In-Time (JIT) compiling machine for BPF were factors for the excellent performances of this tool.
|
||||
|
||||
Then in 2013, Alexei Starovoitov completely reshaped it, started to add new functionalities and to improve the performances of BPF. This new version is designated as eBPF (for “extended BPF”), while the former becomes cBPF (“classic” BPF). New features such as maps and tail calls appeared. The JIT machines were rewritten. The new language is even closer to native machine language than cBPF was. And also, new attach points in the kernel have been created.
|
||||
|
||||
Thanks to those new hooks, eBPF programs can be designed for a variety of use cases, that divide into two fields of applications. One of them is the domain of kernel tracing and event monitoring. BPF programs can be attached to kprobes and they compare with other tracing methods, with many advantages (and sometimes some drawbacks).
|
||||
|
||||
The other application domain remains network programming. In addition to socket filter, eBPF programs can be attached to tc (Linux traffic control tool) ingress or egress interfaces and perform a variety of packet processing tasks, in an efficient way. This opens new perspectives in the domain.
|
||||
|
||||
And eBPF performances are further leveraged through the technologies developed for the IO Visor project: new hooks have also been added for XDP (“eXpress Data Path”), a new fast path recently added to the kernel. XDP works in conjunction with the Linux stack, and relies on BPF to perform very fast packet processing.
|
||||
|
||||
Even some projects such as P4, Open vSwitch, [consider][155] or started to approach BPF. Some others, such as CETH, Cilium, are entirely based on it. BPF is buzzing, so we can expect a lot of tools and projects to orbit around it soon…
|
||||
|
||||
# Dive into the bytecode
|
||||
|
||||
As for me: some of my work (including for [BEBA][156]) is closely related to eBPF, and several future articles on this site will focus on this topic. Logically, I wanted to somehow introduce BPF on this blog before going down to the details—I mean, a real introduction, more developed on BPF functionalities that the brief abstract provided in first section: What are BPF maps? Tail calls? What do the internals look like? And so on. But there are a lot of presentations on this topic available on the web already, and I do not wish to create “yet another BPF introduction” that would come as a duplicate of existing documents.
|
||||
|
||||
So instead, here is what we will do. After all, I spent some time reading and learning about BPF, and while doing so, I gathered a fair amount of material about BPF: introductions, documentation, but also tutorials or examples. There is a lot to read, but in order to read it, one has to _find_ it first. Therefore, as an attempt to help people who wish to learn and use BPF, the present article introduces a list of resources. These are various kinds of readings, that hopefully will help you dive into the mechanics of this kernel bytecode.
|
||||
|
||||
# Resources
|
||||
|
||||

|
||||
|
||||
### Generic presentations
|
||||
|
||||
The documents linked below provide a generic overview of BPF, or of some closely related topics. If you are very new to BPF, you can try picking a couple of presentation among the first ones and reading the ones you like most. If you know eBPF already, you probably want to target specific topics instead, lower down in the list.
|
||||
|
||||
### About BPF
|
||||
|
||||
Generic presentations about eBPF:
|
||||
|
||||
* [_Making the Kernel’s Networking Data Path Programmable with BPF and XDP_][53] (Daniel Borkmann, OSSNA17, Los Angeles, September 2017):
|
||||
One of the best set of slides available to understand quickly all the basics about eBPF and XDP (mostly for network processing).
|
||||
|
||||
* [The BSD Packet Filter][54] (Suchakra Sharma, June 2017):
|
||||
A very nice introduction, mostly about the tracing aspects.
|
||||
|
||||
* [_BPF: tracing and more_][55] (Brendan Gregg, January 2017):
|
||||
Mostly about the tracing use cases.
|
||||
|
||||
* [_Linux BPF Superpowers_][56] (Brendan Gregg, March 2016):
|
||||
With a first part on the use of **flame graphs**.
|
||||
|
||||
* [_IO Visor_][57] (Brenden Blanco, SCaLE 14x, January 2016):
|
||||
Also introduces **IO Visor project**.
|
||||
|
||||
* [_eBPF on the Mainframe_][58] (Michael Holzheu, LinuxCon, Dubin, October 2015)
|
||||
|
||||
* [_New (and Exciting!) Developments in Linux Tracing_][59] (Elena Zannoni, LinuxCon, Japan, 2015)
|
||||
|
||||
* [_BPF — in-kernel virtual machine_][60] (Alexei Starovoitov, February 2015):
|
||||
Presentation by the author of eBPF.
|
||||
|
||||
* [_Extending extended BPF_][61] (Jonathan Corbet, July 2014)
|
||||
|
||||
**BPF internals**:
|
||||
|
||||
* Daniel Borkmann has been doing an amazing work to present **the internals** of eBPF, in particular about **its use with tc**, through several talks and papers.
|
||||
* [_Advanced programmability and recent updates with tc’s cls_bpf_][30] (netdev 1.2, Tokyo, October 2016):
|
||||
Daniel provides details on eBPF, its use for tunneling and encapsulation, direct packet access, and other features.
|
||||
|
||||
* [_cls_bpf/eBPF updates since netdev 1.1_][31] (netdev 1.2, Tokyo, October 2016, part of [this tc workshop][32])
|
||||
|
||||
* [_On getting tc classifier fully programmable with cls_bpf_][33] (netdev 1.1, Sevilla, February 2016):
|
||||
After introducing eBPF, this presentation provides insights on many internal BPF mechanisms (map management, tail calls, verifier). A must-read! For the most ambitious, [the full paper is available here][34].
|
||||
|
||||
* [_Linux tc and eBPF_][35] (fosdem16, Brussels, Belgium, January 2016)
|
||||
|
||||
* [_eBPF and XDP walkthrough and recent updates_][36] (fosdem17, Brussels, Belgium, February 2017)
|
||||
|
||||
These presentations are probably one of the best sources of documentation to understand the design and implementation of internal mechanisms of eBPF.
|
||||
|
||||
The [**IO Visor blog**][157] has some interesting technical articles about BPF. Some of them contain a bit of marketing talks.
|
||||
|
||||
**Kernel tracing**: summing up all existing methods, including BPF:
|
||||
|
||||
* [_Meet-cute between eBPF and Kerne Tracing_][62] (Viller Hsiao, July 2016):
|
||||
Kprobes, uprobes, ftrace
|
||||
|
||||
* [_Linux Kernel Tracing_][63] (Viller Hsiao, July 2016):
|
||||
Systemtap, Kernelshark, trace-cmd, LTTng, perf-tool, ftrace, hist-trigger, perf, function tracer, tracepoint, kprobe/uprobe…
|
||||
|
||||
Regarding **event tracing and monitoring**, Brendan Gregg uses eBPF a lot and does an excellent job at documenting some of his use cases. If you are in kernel tracing, you should see his blog articles related to eBPF or to flame graphs. Most of it are accessible [from this article][158] or by browsing his blog.
|
||||
|
||||
Introducing BPF, but also presenting **generic concepts of Linux networking**:
|
||||
|
||||
* [_Linux Networking Explained_][64] (Thomas Graf, LinuxCon, Toronto, August 2016)
|
||||
|
||||
* [_Kernel Networking Walkthrough_][65] (Thomas Graf, LinuxCon, Seattle, August 2015)
|
||||
|
||||
**Hardware offload**:
|
||||
|
||||
* eBPF with tc or XDP supports hardware offload, starting with Linux kernel version 4.9 and introduced by Netronome. Here is a presentation about this feature:
|
||||
[eBPF/XDP hardware offload to SmartNICs][147] (Jakub Kicinski and Nic Viljoen, netdev 1.2, Tokyo, October 2016)
|
||||
|
||||
About **cBPF**:
|
||||
|
||||
* [_The BSD Packet Filter: A New Architecture for User-level Packet Capture_][66] (Steven McCanne and Van Jacobson, 1992):
|
||||
The original paper about (classic) BPF.
|
||||
|
||||
* [The FreeBSD manual page about BPF][67] is a useful resource to understand cBPF programs.
|
||||
|
||||
* Daniel Borkmann realized at least two presentations on cBPF, [one in 2013 on mmap, BPF and Netsniff-NG][68], and [a very complete one in 2014 on tc and cls_bpf][69].
|
||||
|
||||
* On Cloudflare’s blog, Marek Majkowski presented his [use of BPF bytecode with the `xt_bpf`module for **iptables**][70]. It is worth mentioning that eBPF is also supported by this module, starting with Linux kernel 4.10 (I do not know of any talk or article about this, though).
|
||||
|
||||
* [Libpcap filters syntax][71]
|
||||
|
||||
### About XDP
|
||||
|
||||
* [XDP overview][72] on the IO Visor website.
|
||||
|
||||
* [_eXpress Data Path (XDP)_][73] (Tom Herbert, Alexei Starovoitov, March 2016):
|
||||
The first presentation about XDP.
|
||||
|
||||
* [_BoF - What Can BPF Do For You?_][74] (Brenden Blanco, LinuxCon, Toronto, August 2016).
|
||||
|
||||
* [_eXpress Data Path_][148] (Brenden Blanco, Linux Meetup at Santa Clara, July 2016):
|
||||
Contains some (somewhat marketing?) **benchmark results**! With a single core:
|
||||
* ip routing drop: ~3.6 million packets per second (Mpps)
|
||||
|
||||
* tc (with clsact qdisc) drop using BPF: ~4.2 Mpps
|
||||
|
||||
* XDP drop using BPF: 20 Mpps (<10 % CPU utilization)
|
||||
|
||||
* XDP forward (on port on which the packet was received) with rewrite: 10 Mpps
|
||||
|
||||
(Tests performed with the mlx4 driver).
|
||||
|
||||
* Jesper Dangaard Brouer has several excellent sets of slides, that are essential to fully understand the internals of XDP.
|
||||
* [_XDP − eXpress Data Path, Intro and future use-cases_][37] (September 2016):
|
||||
_“Linux Kernel’s fight against DPDK”_ . **Future plans** (as of this writing) for XDP and comparison with DPDK.
|
||||
|
||||
* [_Network Performance Workshop_][38] (netdev 1.2, Tokyo, October 2016):
|
||||
Additional hints about XDP internals and expected evolution.
|
||||
|
||||
* [_XDP – eXpress Data Path, Used for DDoS protection_][39] (OpenSourceDays, March 2017):
|
||||
Contains details and use cases about XDP, with **benchmark results**, and **code snippets** for **benchmarking** as well as for **basic DDoS protection** with eBPF/XDP (based on an IP blacklisting scheme).
|
||||
|
||||
* [_Memory vs. Networking, Provoking and fixing memory bottlenecks_][40] (LSF Memory Management Summit, March 2017):
|
||||
Provides a lot of details about current **memory issues** faced by XDP developers. Do not start with this one, but if you already know XDP and want to see how it really works on the page allocation side, this is a very helpful resource.
|
||||
|
||||
* [_XDP for the Rest of Us_][41] (netdev 2.1, Montreal, April 2017), with Andy Gospodarek:
|
||||
How to get started with eBPF and XDP for normal humans. This presentation was also summarized by Julia Evans on [her blog][42].
|
||||
|
||||
(Jesper also created and tries to extend some documentation about eBPF and XDP, see [related section][75].)
|
||||
|
||||
* [_XDP workshop — Introduction, experience, and future development_][76] (Tom Herbert, netdev 1.2, Tokyo, October 2016) — as of this writing, only the video is available, I don’t know if the slides will be added.
|
||||
|
||||
* [_High Speed Packet Filtering on Linux_][149] (Gilberto Bertin, DEF CON 25, Las Vegas, July 2017) — an excellent introduction to state-of-the-art packet filtering on Linux, oriented towards DDoS protection, talking about packet processing in the kernel, kernel bypass, XDP and eBPF.
|
||||
|
||||
### About other components related or based on eBPF
|
||||
|
||||
* [_P4 on the Edge_][77] (John Fastabend, May 2016):
|
||||
Presents the use of **P4**, a description language for packet processing, with BPF to create high-performance programmable switches.
|
||||
|
||||
* If you like audio presentations, there is an associated [OvS Orbit episode (#11), called _**P4** on the Edge_][78] , dating from August 2016\. OvS Orbit are interviews realized by Ben Pfaff, who is one of the core maintainers of Open vSwitch. In this case, John Fastabend is interviewed.
|
||||
|
||||
* [_P4, EBPF and Linux TC Offload_][79] (Dinan Gunawardena and Jakub Kicinski, August 2016):
|
||||
Another presentation on **P4**, with some elements related to eBPF hardware offload on Netronome’s **NFP** (Network Flow Processor) architecture.
|
||||
|
||||
* **Cilium** is a technology initiated by Cisco and relying on BPF and XDP to provide “fast in-kernel networking and security policy enforcement for containers based on eBPF programs generated on the fly”. [The code of this project][150] is available on GitHub. Thomas Graf has been performing a number of presentations of this topic:
|
||||
* [_Cilium: Networking & Security for Containers with BPF & XDP_][43] , also featuring a load balancer use case (Linux Plumbers conference, Santa Fe, November 2016)
|
||||
|
||||
* [_Cilium: Networking & Security for Containers with BPF & XDP_][44] (Docker Distributed Systems Summit, October 2016 — [video][45])
|
||||
|
||||
* [_Cilium: Fast IPv6 container Networking with BPF and XDP_][46] (LinuxCon, Toronto, August 2016)
|
||||
|
||||
* [_Cilium: BPF & XDP for containers_][47] (fosdem17, Brussels, Belgium, February 2017)
|
||||
|
||||
A good deal of contents is repeated between the different presentations; if in doubt, just pick the most recent one. Daniel Borkmann has also written [a generic introduction to Cilium][80] as a guest author on Google Open Source blog.
|
||||
|
||||
* There are also podcasts about **Cilium**: an [OvS Orbit episode (#4)][81], in which Ben Pfaff interviews Thomas Graf (May 2016), and [another podcast by Ivan Pepelnjak][82], still with Thomas Graf about eBPF, P4, XDP and Cilium (October 2016).
|
||||
|
||||
* **Open vSwitch** (OvS), and its related project **Open Virtual Network** (OVN, an open source network virtualization solution) are considering to use eBPF at various level, with several proof-of-concept prototypes already implemented:
|
||||
|
||||
* [Offloading OVS Flow Processing using eBPF][48] (William (Cheng-Chun) Tu, OvS conference, San Jose, November 2016)
|
||||
|
||||
* [Coupling the Flexibility of OVN with the Efficiency of IOVisor][49] (Fulvio Risso, Matteo Bertrone and Mauricio Vasquez Bernal, OvS conference, San Jose, November 2016)
|
||||
|
||||
These use cases for eBPF seem to be only at the stage of proposals (nothing merge to OvS main branch) as far as I know, but it will be very interesting to see what comes out of it.
|
||||
|
||||
* XDP is envisioned to be of great help for protection against Distributed Denial-of-Service (DDoS) attacks. More and more presentations focus on this. For example, the talks from people from Cloudflare ( [_XDP in practice: integrating XDP in our DDoS mitigation pipeline_][83] ) or from Facebook ( [_Droplet: DDoS countermeasures powered by BPF + XDP_][84] ) at the netdev 2.1 conference in Montreal, Canada, in April 2017, present such use cases.
|
||||
|
||||
* [_CETH for XDP_][85] (Yan Chan and Yunsong Lu, Linux Meetup, Santa Clara, July 2016):
|
||||
**CETH** stands for Common Ethernet Driver Framework for faster network I/O, a technology initiated by Mellanox.
|
||||
|
||||
* [**The VALE switch**][86], another virtual switch that can be used in conjunction with the netmap framework, has [a BPF extension module][87].
|
||||
|
||||
* **Suricata**, an open source intrusion detection system, [seems to rely on eBPF components][88] for its “capture bypass” features:
|
||||
[_The adventures of a Suricate in eBPF land_][89] (Éric Leblond, netdev 1.2, Tokyo, October 2016)
|
||||
[_eBPF and XDP seen from the eyes of a meerkat_][90] (Éric Leblond, Kernel Recipes, Paris, September 2017)
|
||||
|
||||
* [InKeV: In-Kernel Distributed Network Virtualization for DCN][91] (Z. Ahmed, M. H. Alizai and A. A. Syed, SIGCOMM, August 2016):
|
||||
**InKeV** is an eBPF-based datapath architecture for virtual networks, targeting data center networks. It was initiated by PLUMgrid, and claims to achieve better performances than OvS-based OpenStack solutions.
|
||||
|
||||
* [_**gobpf** - utilizing eBPF from Go_][92] (Michael Schubert, fosdem17, Brussels, Belgium, February 2017):
|
||||
A “library to create, load and use eBPF programs from Go”
|
||||
|
||||
* [**ply**][93] is a small but flexible open source dynamic **tracer** for Linux, with some features similar to the bcc tools, but with a simpler language inspired by awk and dtrace, written by Tobias Waldekranz.
|
||||
|
||||
* If you read my previous article, you might be interested in this talk I gave about [implementing the OpenState interface with eBPF][151], for stateful packet processing, at fosdem17.
|
||||
|
||||

|
||||
|
||||
### Documentation
|
||||
|
||||
Once you managed to get a broad idea of what BPF is, you can put aside generic presentations and start diving into the documentation. Below are the most complete documents about BPF specifications and functioning. Pick the one you need and read them carefully!
|
||||
|
||||
### About BPF
|
||||
|
||||
* The **specification of BPF** (both classic and extended versions) can be found within the documentation of the Linux kernel, and in particular in file[linux/Documentation/networking/filter.txt][94]. The use of BPF as well as its internals are documented there. Also, this is where you can find **information about errors thrown by the verifier** when loading BPF code fails. Can be helpful to troubleshoot obscure error messages.
|
||||
|
||||
* Also in the kernel tree, there is a document about **frequent Questions & Answers** on eBPF design in file [linux/Documentation/bpf/bpf_design_QA.txt][95].
|
||||
|
||||
* … But the kernel documentation is dense and not especially easy to read. If you look for a simple description of eBPF language, head for [its **summarized description**][96] on the IO Visor GitHub repository instead.
|
||||
|
||||
* By the way, the IO Visor project gathered a lot of **resources about BPF**. Mostly, it is split between[the documentation directory][97] of its bcc repository, and the whole content of [the bpf-docs repository][98], both on GitHub. Note the existence of this excellent [BPF **reference guide**][99] containing a detailed description of BPF C and bcc Python helpers.
|
||||
|
||||
* To hack with BPF, there are some essential **Linux manual pages**. The first one is [the `bpf(2)` man page][100] about the `bpf()` **system call**, which is used to manage BPF programs and maps from userspace. It also contains a description of BPF advanced features (program types, maps and so on). The second one is mostly addressed to people wanting to attach BPF programs to tc interface: it is [the `tc-bpf(8)` man page][101], which is a reference for **using BPF with tc**, and includes some example commands and samples of code.
|
||||
|
||||
* Jesper Dangaard Brouer initiated an attempt to **update eBPF Linux documentation**, including **the different kinds of maps**. [He has a draft][102] to which contributions are welcome. Once ready, this document should be merged into the man pages and into kernel documentation.
|
||||
|
||||
* The Cilium project also has an excellent [**BPF and XDP Reference Guide**][103], written by core eBPF developers, that should prove immensely useful to any eBPF developer.
|
||||
|
||||
* David Miller has sent several enlightening emails about eBPF/XDP internals on the [xdp-newbies][152]mailing list. I could not find a link that gathers them at a single place, so here is a list:
|
||||
* [bpf.h and you…][50]
|
||||
|
||||
* [Contextually speaking…][51]
|
||||
|
||||
* [BPF Verifier Overview][52]
|
||||
|
||||
The last one is possibly the best existing summary about the verifier at this date.
|
||||
|
||||
* Ferris Ellis started [a **blog post series about eBPF**][104]. As I write this paragraph, the first article is out, with some historical background and future expectations for eBPF. Next posts should be more technical, and look promising.
|
||||
|
||||
* [A **list of BPF features per kernel version**][153] is available in bcc repository. Useful is you want to know the minimal kernel version that is required to run a given feature. I contributed and added the links to the commits that introduced each feature, so you can also easily access the commit logs from there.
|
||||
|
||||
### About tc
|
||||
|
||||
When using BPF for networking purposes in conjunction with tc, the Linux tool for **t**raffic **c**ontrol, one may wish to gather information about tc’s generic functioning. Here are a couple of resources about it.
|
||||
|
||||
* It is difficult to find simple tutorials about **QoS on Linux**. The two links I have are long and quite dense, but if you can find the time to read it you will learn nearly everything there is to know about tc (nothing about BPF, though). There they are: [_Traffic Control HOWTO_ (Martin A. Brown, 2006)][105], and the [_Linux Advanced Routing & Traffic Control HOWTO_ (“LARTC”) (Bert Hubert & al., 2002)][106].
|
||||
|
||||
* **tc manual pages** may not be up-to-date on your system, since several of them have been added lately. If you cannot find the documentation for a particular queuing discipline (qdisc), class or filter, it may be worth checking the latest [manual pages for tc components][107].
|
||||
|
||||
* Some additional material can be found within the files of iproute2 package itself: the package contains [some documentation][108], including some files that helped me understand better [the functioning of **tc’s actions**][109].
|
||||
**Edit:** While still available from the Git history, these files have been deleted from iproute2 in October 2017.
|
||||
|
||||
* Not exactly documentation: there was [a workshop about several tc features][110] (including filtering, BPF, tc offload, …) organized by Jamal Hadi Salim during the netdev 1.2 conference (October 2016).
|
||||
|
||||
* Bonus information—If you use `tc` a lot, here are some good news: I [wrote a bash completion function][111] for this tool, and it should be shipped with package iproute2 coming with kernel version 4.6 and higher!
|
||||
|
||||
### About XDP
|
||||
|
||||
* Some [work-in-progress documentation (including specifications)][112] for XDP started by Jesper Dangaard Brouer, but meant to be a collaborative work. Under progress (September 2016): you should expect it to change, and maybe to be moved at some point (Jesper [called for contribution][113], if you feel like improving it).
|
||||
|
||||
* The [BPF and XDP Reference Guide][114] from Cilium project… Well, the name says it all.
|
||||
|
||||
### About P4 and BPF
|
||||
|
||||
[P4][159] is a language used to specify the behavior of a switch. It can be compiled for a number of hardware or software targets. As you may have guessed, one of these targets is BPF… The support is only partial: some P4 features cannot be translated towards BPF, and in a similar way there are things that BPF can do but that would not be possible to express with P4\. Anyway, the documentation related to **P4 use with BPF** [used to be hidden in bcc repository][160]. This changed with P4_16 version, the p4c reference compiler including [a backend for eBPF][161].
|
||||
|
||||

|
||||
|
||||
### Tutorials
|
||||
|
||||
Brendan Gregg has produced excellent **tutorials** intended for people who want to **use bcc tools** for tracing and monitoring events in the kernel. [The first tutorial about using bcc itself][162] comes with eleven steps (as of today) to understand how to use the existing tools, while [the one **intended for Python developers**][163] focuses on developing new tools, across seventeen “lessons”.
|
||||
|
||||
Sasha Goldshtein also has some [_**Linux Tracing Workshops Materials**_][164] involving the use of several BPF tools for tracing.
|
||||
|
||||
Another post by Jean-Tiare Le Bigot provides a detailed (and instructive!) example of [using perf and eBPF to setup a low-level tracer][165] for ping requests and replies
|
||||
|
||||
Few tutorials exist for network-related eBPF use cases. There are some interesting documents, including an _eBPF Offload Starting Guide_ , on the [Open NFP][166] platform operated by Netronome. Other than these, the talk from Jesper, [_XDP for the Rest of Us_][167] , is probably one of the best ways to get started with XDP.
|
||||
|
||||

|
||||
|
||||
### Examples
|
||||
|
||||
It is always nice to have examples. To see how things really work. But BPF program samples are scattered across several projects, so I listed all the ones I know of. The examples do not always use the same helpers (for instance, tc and bcc both have their own set of helpers to make it easier to write BPF programs in C language).
|
||||
|
||||
### From the kernel
|
||||
|
||||
The kernel contains examples for most types of program: filters to bind to sockets or to tc interfaces, event tracing/monitoring, and even XDP. You can find these examples under the [linux/samples/bpf/][168]directory.
|
||||
|
||||
Also do not forget to have a look to the logs related to the (git) commits that introduced a particular feature, they may contain some detailed example of the feature.
|
||||
|
||||
### From package iproute2
|
||||
|
||||
The iproute2 package provide several examples as well. They are obviously oriented towards network programming, since the programs are to be attached to tc ingress or egress interfaces. The examples dwell under the [iproute2/examples/bpf/][169] directory.
|
||||
|
||||
### From bcc set of tools
|
||||
|
||||
Many examples are [provided with bcc][170]:
|
||||
|
||||
* Some are networking example programs, under the associated directory. They include socket filters, tc filters, and a XDP program.
|
||||
|
||||
* The `tracing` directory include a lot of example **tracing programs**. The tutorials mentioned earlier are based on these. These programs cover a wide range of event monitoring functions, and some of them are production-oriented. Note that on certain Linux distributions (at least for Debian, Ubuntu, Fedora, Arch Linux), these programs have been [packaged][115] and can be “easily” installed by typing e.g. `# apt install bcc-tools`, but as of this writing (and except for Arch Linux), this first requires to set up IO Visor’s own package repository.
|
||||
|
||||
* There are also some examples **using Lua** as a different BPF back-end (that is, BPF programs are written with Lua instead of a subset of C, allowing to use the same language for front-end and back-end), in the third directory.
|
||||
|
||||
### Manual pages
|
||||
|
||||
While bcc is generally the easiest way to inject and run a BPF program in the kernel, attaching programs to tc interfaces can also be performed by the `tc` tool itself. So if you intend to **use BPF with tc**, you can find some example invocations in the [`tc-bpf(8)` manual page][171].
|
||||
|
||||

|
||||
|
||||
### The code
|
||||
|
||||
Sometimes, BPF documentation or examples are not enough, and you may have no other solution that to display the code in your favorite text editor (which should be Vim of course) and to read it. Or you may want to hack into the code so as to patch or add features to the machine. So here are a few pointers to the relevant files, finding the functions you want is up to you!
|
||||
|
||||
### BPF code in the kernel
|
||||
|
||||
* The file [linux/include/linux/bpf.h][116] and its counterpart [linux/include/uapi/bpf.h][117] contain **definitions** related to eBPF, to be used respectively in the kernel and to interface with userspace programs.
|
||||
|
||||
* On the same pattern, files [linux/include/linux/filter.h][118] and [linux/include/uapi/filter.h][119] contain information used to **run the BPF programs**.
|
||||
|
||||
* The **main pieces of code** related to BPF are under [linux/kernel/bpf/][120] directory. **The different operations permitted by the system call**, such as program loading or map management, are implemented in file `syscall.c`, while `core.c` contains the **interpreter**. The other files have self-explanatory names: `verifier.c` contains the **verifier** (no kidding), `arraymap.c` the code used to interact with **maps** of type array, and so on.
|
||||
|
||||
* The **helpers**, as well as several functions related to networking (with tc, XDP…) and available to the user, are implemented in [linux/net/core/filter.c][121]. It also contains the code to migrate cBPF bytecode to eBPF (since all cBPF programs are now translated to eBPF in the kernel before being run).
|
||||
|
||||
* The **JIT compilers** are under the directory of their respective architectures, such as file[linux/arch/x86/net/bpf_jit_comp.c][122] for x86.
|
||||
|
||||
* You will find the code related to **the BPF components of tc** in the [linux/net/sched/][123] directory, and in particular in files `act_bpf.c` (action) and `cls_bpf.c` (filter).
|
||||
|
||||
* I have not hacked with **event tracing** in BPF, so I do not really know about the hooks for such programs. There is some stuff in [linux/kernel/trace/bpf_trace.c][124]. If you are interested in this and want to know more, you may dig on the side of Brendan Gregg’s presentations or blog posts.
|
||||
|
||||
* Nor have I used **seccomp-BPF**. But the code is in [linux/kernel/seccomp.c][125], and some example use cases can be found in [linux/tools/testing/selftests/seccomp/seccomp_bpf.c][126].
|
||||
|
||||
### XDP hooks code
|
||||
|
||||
Once loaded into the in-kernel BPF virtual machine, **XDP** programs are hooked from userspace into the kernel network path thanks to a Netlink command. On reception, the function `dev_change_xdp_fd()` in file [linux/net/core/dev.c][172] is called and sets a XDP hook. Such hooks are located in the drivers of supported NICs. For example, the mlx4 driver used for some Mellanox hardware has hooks implemented in files under the [drivers/net/ethernet/mellanox/mlx4/][173] directory. File en_netdev.c receives Netlink commands and calls `mlx4_xdp_set()`, which in turns calls for instance `mlx4_en_process_rx_cq()` (for the RX side) implemented in file en_rx.c.
|
||||
|
||||
### BPF logic in bcc
|
||||
|
||||
One can find the code for the **bcc** set of tools [on the bcc GitHub repository][174]. The **Python code**, including the `BPF` class, is initiated in file [bcc/src/python/bcc/__init__.py][175]. But most of the interesting stuff—to my opinion—such as loading the BPF program into the kernel, happens [in the libbcc **C library**][176].
|
||||
|
||||
### Code to manage BPF with tc
|
||||
|
||||
The code related to BPF **in tc** comes with the iproute2 package, of course. Some of it is under the[iproute2/tc/][177] directory. The files f_bpf.c and m_bpf.c (and e_bpf.c) are used respectively to handle BPF filters and actions (and tc `exec` command, whatever this may be). File q_clsact.c defines the `clsact` qdisc especially created for BPF. But **most of the BPF userspace logic** is implemented in[iproute2/lib/bpf.c][178] library, so this is probably where you should head to if you want to mess up with BPF and tc (it was moved from file iproute2/tc/tc_bpf.c, where you may find the same code in older versions of the package).
|
||||
|
||||
### BPF utilities
|
||||
|
||||
The kernel also ships the sources of three tools (`bpf_asm.c`, `bpf_dbg.c`, `bpf_jit_disasm.c`) related to BPF, under the [linux/tools/net/][179] or [linux/tools/bpf/][180] directory depending on your version:
|
||||
|
||||
* `bpf_asm` is a minimal cBPF assembler.
|
||||
|
||||
* `bpf_dbg` is a small debugger for cBPF programs.
|
||||
|
||||
* `bpf_jit_disasm` is generic for both BPF flavors and could be highly useful for JIT debugging.
|
||||
|
||||
* `bpftool` is a generic utility written by Jakub Kicinski, and that can be used to interact with eBPF programs and maps from userspace, for example to show, dump, pin programs, or to show, create, pin, update, delete maps.
|
||||
|
||||
Read the comments at the top of the source files to get an overview of their usage.
|
||||
|
||||
### Other interesting chunks
|
||||
|
||||
If you are interested the use of less common languages with BPF, bcc contains [a **P4 compiler** for BPF targets][181] as well as [a **Lua front-end**][182] that can be used as alternatives to the C subset and (in the case of Lua) to the Python tools.
|
||||
|
||||
### LLVM backend
|
||||
|
||||
The BPF backend used by clang / LLVM for compiling C into eBPF was added to the LLVM sources in[this commit][183] (and can also be accessed on [the GitHub mirror][184]).
|
||||
|
||||
### Running in userspace
|
||||
|
||||
As far as I know there are at least two eBPF userspace implementations. The first one, [uBPF][185], is written in C. It contains an interpreter, a JIT compiler for x86_64 architecture, an assembler and a disassembler.
|
||||
|
||||
The code of uBPF seems to have been reused to produce a [generic implementation][186], that claims to support FreeBSD kernel, FreeBSD userspace, Linux kernel, Linux userspace and MacOSX userspace. It is used for the [BPF extension module for VALE switch][187].
|
||||
|
||||
The other userspace implementation is my own work: [rbpf][188], based on uBPF, but written in Rust. The interpreter and JIT-compiler work (both under Linux, only the interpreter for MacOSX and Windows), there may be more in the future.
|
||||
|
||||
### Commit logs
|
||||
|
||||
As stated earlier, do not hesitate to have a look at the commit log that introduced a particular BPF feature if you want to have more information about it. You can search the logs in many places, such as on [git.kernel.org][189], [on GitHub][190], or on your local repository if you have cloned it. If you are not familiar with git, try things like `git blame <file>` to see what commit introduced a particular line of code, then `git show <commit>` to have details (or search by keyword in `git log` results, but this may be tedious). See also [the list of eBPF features per kernel version][191] on bcc repository, that links to relevant commits.
|
||||
|
||||

|
||||
|
||||
### Troubleshooting
|
||||
|
||||
The enthusiasm about eBPF is quite recent, and so far I have not found a lot of resources intending to help with troubleshooting. So here are the few I have, augmented with my own recollection of pitfalls encountered while working with BPF.
|
||||
|
||||
### Errors at compilation time
|
||||
|
||||
* Make sure you have a recent enough version of the Linux kernel (see also [this document][127]).
|
||||
|
||||
* If you compiled the kernel yourself: make sure you installed correctly all components, including kernel image, headers and libc.
|
||||
|
||||
* When using the `bcc` shell function provided by `tc-bpf` man page (to compile C code into BPF): I once had to add includes to the header for the clang call:
|
||||
|
||||
```
|
||||
__bcc() {
|
||||
clang -O2 -I "/usr/src/linux-headers-$(uname -r)/include/" \
|
||||
-I "/usr/src/linux-headers-$(uname -r)/arch/x86/include/" \
|
||||
-emit-llvm -c $1 -o - | \
|
||||
llc -march=bpf -filetype=obj -o "`basename $1 .c`.o"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
(seems fixed as of today).
|
||||
|
||||
* For other problems with `bcc`, do not forget to have a look at [the FAQ][128] of the tool set.
|
||||
|
||||
* If you downloaded the examples from the iproute2 package in a version that does not exactly match your kernel, some errors can be triggered by the headers included in the files. The example snippets indeed assume that the same version of iproute2 package and kernel headers are installed on the system. If this is not the case, download the correct version of iproute2, or edit the path of included files in the examples to point to the headers included in iproute2 (some problems may or may not occur at runtime, depending on the features in use).
|
||||
|
||||
### Errors at load and run time
|
||||
|
||||
* To load a program with tc, make sure you use a tc binary coming from an iproute2 version equivalent to the kernel in use.
|
||||
|
||||
* To load a program with bcc, make sure you have bcc installed on the system (just downloading the sources to run the Python script is not enough).
|
||||
|
||||
* With tc, if the BPF program does not return the expected values, check that you called it in the correct fashion: filter, or action, or filter with “direct-action” mode.
|
||||
|
||||
* With tc still, note that actions cannot be attached directly to qdiscs or interfaces without the use of a filter.
|
||||
|
||||
* The errors thrown by the in-kernel verifier may be hard to interpret. [The kernel documentation][129]may help, so may [the reference guide][130] or, as a last resort, the source code (see above) (good luck!). For this kind of errors it is also important to keep in mind that the verifier _does not run_ the program. If you get an error about an invalid memory access or about uninitialized data, it does not mean that these problems actually occurred (or sometimes, that they can possibly occur at all). It means that your program is written in such a way that the verifier estimates that such errors could happen, and therefore it rejects the program.
|
||||
|
||||
* Note that `tc` tool has a verbose mode, and that it works well with BPF: try appending `verbose`at the end of your command line.
|
||||
|
||||
* bcc also has verbose options: the `BPF` class has a `debug` argument that can take any combination of the three flags `DEBUG_LLVM_IR`, `DEBUG_BPF` and `DEBUG_PREPROCESSOR` (see details in [the source file][131]). It even embeds [some facilities to print output messages][132] for debugging the code.
|
||||
|
||||
* LLVM v4.0+ [embeds a disassembler][133] for eBPF programs. So if you compile your program with clang, adding the `-g` flag for compiling enables you to later dump your program in the rather human-friendly format used by the kernel verifier. To proceed to the dump, use:
|
||||
|
||||
```
|
||||
$ llvm-objdump -S -no-show-raw-insn bpf_program.o
|
||||
|
||||
```
|
||||
|
||||
* Working with maps? You want to have a look at [bpf-map][134], a very userful tool in Go created for the Cilium project, that can be used to dump the contents of kernel eBPF maps. There also exists [a clone][135] in Rust.
|
||||
|
||||
* There is an old [`bpf` tag on **StackOverflow**][136], but as of this writing it has been hardly used—ever (and there is nearly nothing related to the new eBPF version). If you are a reader from the Future though, you may want to check whether there has been more activity on this side.
|
||||
|
||||

|
||||
|
||||
### And still more!
|
||||
|
||||
* In case you would like to easily **test XDP**, there is [a Vagrant setup][137] available. You can also **test bcc**[in a Docker container][138].
|
||||
|
||||
* Wondering where the **development and activities** around BPF occur? Well, the kernel patches always end up [on the netdev mailing list][139] (related to the Linux kernel networking stack development): search for “BPF” or “XDP” keywords. Since April 2017, there is also [a mailing list specially dedicated to XDP programming][140] (both for architecture or for asking for help). Many discussions and debates also occur [on the IO Visor mailing list][141], since BPF is at the heart of the project. If you only want to keep informed from time to time, there is also an [@IOVisor Twitter account][142].
|
||||
|
||||
And come back on this blog from time to time to see if they are new articles [about BPF][192]!
|
||||
|
||||
_Special thanks to Daniel Borkmann for the numerous [additional documents][154] he pointed to me so that I could complete this collection._
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/
|
||||
|
||||
作者:[Quentin Monnet ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://qmonnet.github.io/whirl-offload/about/
|
||||
[1]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-bpf
|
||||
[2]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-xdp
|
||||
[3]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-other-components-related-or-based-on-ebpf
|
||||
[4]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-bpf-1
|
||||
[5]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-tc
|
||||
[6]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-xdp-1
|
||||
[7]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-p4-and-bpf
|
||||
[8]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#from-the-kernel
|
||||
[9]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#from-package-iproute2
|
||||
[10]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#from-bcc-set-of-tools
|
||||
[11]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#manual-pages
|
||||
[12]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#bpf-code-in-the-kernel
|
||||
[13]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#xdp-hooks-code
|
||||
[14]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#bpf-logic-in-bcc
|
||||
[15]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#code-to-manage-bpf-with-tc
|
||||
[16]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#bpf-utilities
|
||||
[17]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#other-interesting-chunks
|
||||
[18]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#llvm-backend
|
||||
[19]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#running-in-userspace
|
||||
[20]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#commit-logs
|
||||
[21]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#errors-at-compilation-time
|
||||
[22]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#errors-at-load-and-run-time
|
||||
[23]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#generic-presentations
|
||||
[24]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#documentation
|
||||
[25]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#tutorials
|
||||
[26]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#examples
|
||||
[27]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#the-code
|
||||
[28]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#troubleshooting
|
||||
[29]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#and-still-more
|
||||
[30]:http://netdevconf.org/1.2/session.html?daniel-borkmann
|
||||
[31]:http://netdevconf.org/1.2/slides/oct5/07_tcws_daniel_borkmann_2016_tcws.pdf
|
||||
[32]:http://netdevconf.org/1.2/session.html?jamal-tc-workshop
|
||||
[33]:http://www.netdevconf.org/1.1/proceedings/slides/borkmann-tc-classifier-cls-bpf.pdf
|
||||
[34]:http://www.netdevconf.org/1.1/proceedings/papers/On-getting-tc-classifier-fully-programmable-with-cls-bpf.pdf
|
||||
[35]:https://archive.fosdem.org/2016/schedule/event/ebpf/attachments/slides/1159/export/events/attachments/ebpf/slides/1159/ebpf.pdf
|
||||
[36]:https://fosdem.org/2017/schedule/event/ebpf_xdp/
|
||||
[37]:http://people.netfilter.org/hawk/presentations/xdp2016/xdp_intro_and_use_cases_sep2016.pdf
|
||||
[38]:http://netdevconf.org/1.2/session.html?jesper-performance-workshop
|
||||
[39]:http://people.netfilter.org/hawk/presentations/OpenSourceDays2017/XDP_DDoS_protecting_osd2017.pdf
|
||||
[40]:http://people.netfilter.org/hawk/presentations/MM-summit2017/MM-summit2017-JesperBrouer.pdf
|
||||
[41]:http://netdevconf.org/2.1/session.html?gospodarek
|
||||
[42]:http://jvns.ca/blog/2017/04/07/xdp-bpf-tutorial/
|
||||
[43]:http://www.slideshare.net/ThomasGraf5/clium-container-networking-with-bpf-xdp
|
||||
[44]:http://www.slideshare.net/Docker/cilium-bpf-xdp-for-containers-66969823
|
||||
[45]:https://www.youtube.com/watch?v=TnJF7ht3ZYc&list=PLkA60AVN3hh8oPas3cq2VA9xB7WazcIgs
|
||||
[46]:http://www.slideshare.net/ThomasGraf5/cilium-fast-ipv6-container-networking-with-bpf-and-xdp
|
||||
[47]:https://fosdem.org/2017/schedule/event/cilium/
|
||||
[48]:http://openvswitch.org/support/ovscon2016/7/1120-tu.pdf
|
||||
[49]:http://openvswitch.org/support/ovscon2016/7/1245-bertrone.pdf
|
||||
[50]:https://www.spinics.net/lists/xdp-newbies/msg00179.html
|
||||
[51]:https://www.spinics.net/lists/xdp-newbies/msg00181.html
|
||||
[52]:https://www.spinics.net/lists/xdp-newbies/msg00185.html
|
||||
[53]:http://schd.ws/hosted_files/ossna2017/da/BPFandXDP.pdf
|
||||
[54]:https://speakerdeck.com/tuxology/the-bsd-packet-filter
|
||||
[55]:http://www.slideshare.net/brendangregg/bpf-tracing-and-more
|
||||
[56]:http://fr.slideshare.net/brendangregg/linux-bpf-superpowers
|
||||
[57]:https://www.socallinuxexpo.org/sites/default/files/presentations/Room%20211%20-%20IOVisor%20-%20SCaLE%2014x.pdf
|
||||
[58]:https://events.linuxfoundation.org/sites/events/files/slides/ebpf_on_the_mainframe_lcon_2015.pdf
|
||||
[59]:https://events.linuxfoundation.org/sites/events/files/slides/tracing-linux-ezannoni-linuxcon-ja-2015_0.pdf
|
||||
[60]:https://events.linuxfoundation.org/sites/events/files/slides/bpf_collabsummit_2015feb20.pdf
|
||||
[61]:https://lwn.net/Articles/603983/
|
||||
[62]:http://www.slideshare.net/vh21/meet-cutebetweenebpfandtracing
|
||||
[63]:http://www.slideshare.net/vh21/linux-kernel-tracing
|
||||
[64]:http://www.slideshare.net/ThomasGraf5/linux-networking-explained
|
||||
[65]:http://www.slideshare.net/ThomasGraf5/linuxcon-2015-linux-kernel-networking-walkthrough
|
||||
[66]:http://www.tcpdump.org/papers/bpf-usenix93.pdf
|
||||
[67]:http://www.gsp.com/cgi-bin/man.cgi?topic=bpf
|
||||
[68]:http://borkmann.ch/talks/2013_devconf.pdf
|
||||
[69]:http://borkmann.ch/talks/2014_devconf.pdf
|
||||
[70]:https://blog.cloudflare.com/introducing-the-bpf-tools/
|
||||
[71]:http://biot.com/capstats/bpf.html
|
||||
[72]:https://www.iovisor.org/technology/xdp
|
||||
[73]:https://github.com/iovisor/bpf-docs/raw/master/Express_Data_Path.pdf
|
||||
[74]:https://events.linuxfoundation.org/sites/events/files/slides/iovisor-lc-bof-2016.pdf
|
||||
[75]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#about-xdp-1
|
||||
[76]:http://netdevconf.org/1.2/session.html?herbert-xdp-workshop
|
||||
[77]:https://schd.ws/hosted_files/2016p4workshop/1d/Intel%20Fastabend-P4%20on%20the%20Edge.pdf
|
||||
[78]:https://ovsorbit.benpfaff.org/#e11
|
||||
[79]:http://open-nfp.org/media/pdfs/Open_NFP_P4_EBPF_Linux_TC_Offload_FINAL.pdf
|
||||
[80]:https://opensource.googleblog.com/2016/11/cilium-networking-and-security.html
|
||||
[81]:https://ovsorbit.benpfaff.org/
|
||||
[82]:http://blog.ipspace.net/2016/10/fast-linux-packet-forwarding-with.html
|
||||
[83]:http://netdevconf.org/2.1/session.html?bertin
|
||||
[84]:http://netdevconf.org/2.1/session.html?zhou
|
||||
[85]:http://www.slideshare.net/IOVisor/ceth-for-xdp-linux-meetup-santa-clara-july-2016
|
||||
[86]:http://info.iet.unipi.it/~luigi/vale/
|
||||
[87]:https://github.com/YutaroHayakawa/vale-bpf
|
||||
[88]:https://www.stamus-networks.com/2016/09/28/suricata-bypass-feature/
|
||||
[89]:http://netdevconf.org/1.2/slides/oct6/10_suricata_ebpf.pdf
|
||||
[90]:https://www.slideshare.net/ennael/kernel-recipes-2017-ebpf-and-xdp-eric-leblond
|
||||
[91]:https://github.com/iovisor/bpf-docs/blob/master/university/sigcomm-ccr-InKev-2016.pdf
|
||||
[92]:https://fosdem.org/2017/schedule/event/go_bpf/
|
||||
[93]:https://wkz.github.io/ply/
|
||||
[94]:https://www.kernel.org/doc/Documentation/networking/filter.txt
|
||||
[95]:https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/tree/Documentation/bpf/bpf_design_QA.txt?id=2e39748a4231a893f057567e9b880ab34ea47aef
|
||||
[96]:https://github.com/iovisor/bpf-docs/blob/master/eBPF.md
|
||||
[97]:https://github.com/iovisor/bcc/tree/master/docs
|
||||
[98]:https://github.com/iovisor/bpf-docs/
|
||||
[99]:https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md
|
||||
[100]:http://man7.org/linux/man-pages/man2/bpf.2.html
|
||||
[101]:http://man7.org/linux/man-pages/man8/tc-bpf.8.html
|
||||
[102]:https://prototype-kernel.readthedocs.io/en/latest/bpf/index.html
|
||||
[103]:http://docs.cilium.io/en/latest/bpf/
|
||||
[104]:https://ferrisellis.com/tags/ebpf/
|
||||
[105]:http://linux-ip.net/articles/Traffic-Control-HOWTO/
|
||||
[106]:http://lartc.org/lartc.html
|
||||
[107]:https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/man/man8
|
||||
[108]:https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/tree/doc?h=v4.13.0
|
||||
[109]:https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/tree/doc/actions?h=v4.13.0
|
||||
[110]:http://netdevconf.org/1.2/session.html?jamal-tc-workshop
|
||||
[111]:https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/commit/bash-completion/tc?id=27d44f3a8a4708bcc99995a4d9b6fe6f81e3e15b
|
||||
[112]:https://prototype-kernel.readthedocs.io/en/latest/networking/XDP/index.html
|
||||
[113]:https://marc.info/?l=linux-netdev&m=147436253625672
|
||||
[114]:http://docs.cilium.io/en/latest/bpf/
|
||||
[115]:https://github.com/iovisor/bcc/blob/master/INSTALL.md
|
||||
[116]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/linux/bpf.h
|
||||
[117]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/bpf.h
|
||||
[118]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/linux/filter.h
|
||||
[119]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/filter.h
|
||||
[120]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/kernel/bpf
|
||||
[121]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/core/filter.c
|
||||
[122]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/x86/net/bpf_jit_comp.c
|
||||
[123]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/sched
|
||||
[124]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/kernel/trace/bpf_trace.c
|
||||
[125]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/kernel/seccomp.c
|
||||
[126]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/seccomp/seccomp_bpf.c
|
||||
[127]:https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
|
||||
[128]:https://github.com/iovisor/bcc/blob/master/FAQ.txt
|
||||
[129]:https://www.kernel.org/doc/Documentation/networking/filter.txt
|
||||
[130]:https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md
|
||||
[131]:https://github.com/iovisor/bcc/blob/master/src/python/bcc/__init__.py
|
||||
[132]:https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md#output
|
||||
[133]:https://www.spinics.net/lists/netdev/msg406926.html
|
||||
[134]:https://github.com/cilium/bpf-map
|
||||
[135]:https://github.com/badboy/bpf-map
|
||||
[136]:https://stackoverflow.com/questions/tagged/bpf
|
||||
[137]:https://github.com/iovisor/xdp-vagrant
|
||||
[138]:https://github.com/zlim/bcc-docker
|
||||
[139]:http://lists.openwall.net/netdev/
|
||||
[140]:http://vger.kernel.org/vger-lists.html#xdp-newbies
|
||||
[141]:http://lists.iovisor.org/pipermail/iovisor-dev/
|
||||
[142]:https://twitter.com/IOVisor
|
||||
[143]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#what-is-bpf
|
||||
[144]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#dive-into-the-bytecode
|
||||
[145]:https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#resources
|
||||
[146]:https://github.com/qmonnet/whirl-offload/commits/gh-pages/_posts/2016-09-01-dive-into-bpf.md
|
||||
[147]:http://netdevconf.org/1.2/session.html?jakub-kicinski
|
||||
[148]:http://www.slideshare.net/IOVisor/express-data-path-linux-meetup-santa-clara-july-2016
|
||||
[149]:https://cdn.shopify.com/s/files/1/0177/9886/files/phv2017-gbertin.pdf
|
||||
[150]:https://github.com/cilium/cilium
|
||||
[151]:https://fosdem.org/2017/schedule/event/stateful_ebpf/
|
||||
[152]:http://vger.kernel.org/vger-lists.html#xdp-newbies
|
||||
[153]:https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
|
||||
[154]:https://github.com/qmonnet/whirl-offload/commit/d694f8081ba00e686e34f86d5ee76abeb4d0e429
|
||||
[155]:http://openvswitch.org/pipermail/dev/2014-October/047421.html
|
||||
[156]:https://qmonnet.github.io/whirl-offload/2016/07/15/beba-research-project/
|
||||
[157]:https://www.iovisor.org/resources/blog
|
||||
[158]:http://www.brendangregg.com/blog/2016-03-05/linux-bpf-superpowers.html
|
||||
[159]:http://p4.org/
|
||||
[160]:https://github.com/iovisor/bcc/tree/master/src/cc/frontends/p4
|
||||
[161]:https://github.com/p4lang/p4c/blob/master/backends/ebpf/README.md
|
||||
[162]:https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md
|
||||
[163]:https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md
|
||||
[164]:https://github.com/goldshtn/linux-tracing-workshop
|
||||
[165]:https://blog.yadutaf.fr/2017/07/28/tracing-a-packet-journey-using-linux-tracepoints-perf-ebpf/
|
||||
[166]:https://open-nfp.org/dataplanes-ebpf/technical-papers/
|
||||
[167]:http://netdevconf.org/2.1/session.html?gospodarek
|
||||
[168]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/samples/bpf
|
||||
[169]:https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/examples/bpf
|
||||
[170]:https://github.com/iovisor/bcc/tree/master/examples
|
||||
[171]:http://man7.org/linux/man-pages/man8/tc-bpf.8.html
|
||||
[172]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/core/dev.c
|
||||
[173]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/mellanox/mlx4/
|
||||
[174]:https://github.com/iovisor/bcc/
|
||||
[175]:https://github.com/iovisor/bcc/blob/master/src/python/bcc/__init__.py
|
||||
[176]:https://github.com/iovisor/bcc/blob/master/src/cc/libbpf.c
|
||||
[177]:https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/tc
|
||||
[178]:https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/lib/bpf.c
|
||||
[179]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/tools/net
|
||||
[180]:https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/tree/tools/bpf
|
||||
[181]:https://github.com/iovisor/bcc/tree/master/src/cc/frontends/p4/compiler
|
||||
[182]:https://github.com/iovisor/bcc/tree/master/src/lua
|
||||
[183]:https://reviews.llvm.org/D6494
|
||||
[184]:https://github.com/llvm-mirror/llvm/commit/4fe85c75482f9d11c5a1f92a1863ce30afad8d0d
|
||||
[185]:https://github.com/iovisor/ubpf/
|
||||
[186]:https://github.com/YutaroHayakawa/generic-ebpf
|
||||
[187]:https://github.com/YutaroHayakawa/vale-bpf
|
||||
[188]:https://github.com/qmonnet/rbpf
|
||||
[189]:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git
|
||||
[190]:https://github.com/torvalds/linux
|
||||
[191]:https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
|
||||
[192]:https://qmonnet.github.io/whirl-offload/categories/#BPF
|
95
sources/tech/20171107 GitHub welcomes all CI tools.md
Normal file
95
sources/tech/20171107 GitHub welcomes all CI tools.md
Normal file
@ -0,0 +1,95 @@
|
||||
translating---geekpi
|
||||
|
||||
GitHub welcomes all CI tools
|
||||
====================
|
||||
|
||||
|
||||
[][11]
|
||||
|
||||
Continuous Integration ([CI][12]) tools help you stick to your team's quality standards by running tests every time you push a new commit and [reporting the results][13] to a pull request. Combined with continuous delivery ([CD][14]) tools, you can also test your code on multiple configurations, run additional performance tests, and automate every step [until production][15].
|
||||
|
||||
There are several CI and CD tools that [integrate with GitHub][16], some of which you can install in a few clicks from [GitHub Marketplace][17]. With so many options, you can pick the best tool for the job—even if it's not the one that comes pre-integrated with your system.
|
||||
|
||||
The tools that will work best for you depends on many factors, including:
|
||||
|
||||
* Programming language and application architecture
|
||||
|
||||
* Operating system and browsers you plan to support
|
||||
|
||||
* Your team's experience and skills
|
||||
|
||||
* Scaling capabilities and plans for growth
|
||||
|
||||
* Geographic distribution of dependent systems and the people who use them
|
||||
|
||||
* Packaging and delivery goals
|
||||
|
||||
Of course, it isn't possible to optimize your CI tool for all of these scenarios. The people who build them have to choose which use cases to serve best—and when to prioritize complexity over simplicity. For example, if you like to test small applications written in a particular programming language for one platform, you won't need the complexity of a tool that tests embedded software controllers on dozens of platforms with a broad mix of programming languages and frameworks.
|
||||
|
||||
If you need a little inspiration for which CI tool might work best, take a look at [popular GitHub projects][18]. Many show the status of their integrated CI/CD tools as badges in their README.md. We've also analyzed the use of CI tools across more than 50 million repositories in the GitHub community, and found a lot of variety. The following diagram shows the relative percentage of the top 10 CI tools used with GitHub.com, based on the most used [commit status contexts][19] used within our pull requests.
|
||||
|
||||
_Our analysis also showed that many teams use more than one CI tool in their projects, allowing them to emphasize what each tool does best._
|
||||
|
||||
[][20]
|
||||
|
||||
If you'd like to check them out, here are the top 10 tools teams use:
|
||||
|
||||
* [Travis CI][1]
|
||||
|
||||
* [Circle CI][2]
|
||||
|
||||
* [Jenkins][3]
|
||||
|
||||
* [AppVeyor][4]
|
||||
|
||||
* [CodeShip][5]
|
||||
|
||||
* [Drone][6]
|
||||
|
||||
* [Semaphore CI][7]
|
||||
|
||||
* [Buildkite][8]
|
||||
|
||||
* [Wercker][9]
|
||||
|
||||
* [TeamCity][10]
|
||||
|
||||
It's tempting to just pick the default, pre-integrated tool without taking the time to research and choose the best one for the job, but there are plenty of [excellent choices][21] built for your specific use cases. And if you change your mind later, no problem. When you choose the best tool for a specific situation, you're guaranteeing tailored performance and the freedom of interchangability when it no longer fits.
|
||||
|
||||
Ready to see how CI tools can fit into your workflow?
|
||||
|
||||
[Browse GitHub Marketplace][22]
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://github.com/blog/2463-github-welcomes-all-ci-tools
|
||||
|
||||
作者:[jonico ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://github.com/jonico
|
||||
[1]:https://travis-ci.org/
|
||||
[2]:https://circleci.com/
|
||||
[3]:https://jenkins.io/
|
||||
[4]:https://www.appveyor.com/
|
||||
[5]:https://codeship.com/
|
||||
[6]:http://try.drone.io/
|
||||
[7]:https://semaphoreci.com/
|
||||
[8]:https://buildkite.com/
|
||||
[9]:http://www.wercker.com/
|
||||
[10]:https://www.jetbrains.com/teamcity/
|
||||
[11]:https://user-images.githubusercontent.com/29592817/32509084-2d52c56c-c3a1-11e7-8c49-901f0f601faf.png
|
||||
[12]:https://en.wikipedia.org/wiki/Continuous_integration
|
||||
[13]:https://github.com/blog/2051-protected-branches-and-required-status-checks
|
||||
[14]:https://en.wikipedia.org/wiki/Continuous_delivery
|
||||
[15]:https://developer.github.com/changes/2014-01-09-preview-the-new-deployments-api/
|
||||
[16]:https://github.com/works-with/category/continuous-integration
|
||||
[17]:https://github.com/marketplace/category/continuous-integration
|
||||
[18]:https://github.com/explore?trending=repositories#trending
|
||||
[19]:https://developer.github.com/v3/repos/statuses/
|
||||
[20]:https://user-images.githubusercontent.com/7321362/32575895-ea563032-c49a-11e7-9581-e05ec882658b.png
|
||||
[21]:https://github.com/works-with/category/continuous-integration
|
||||
[22]:https://github.com/marketplace/category/continuous-integration
|
311
sources/tech/20171112 Love Your Bugs.md
Normal file
311
sources/tech/20171112 Love Your Bugs.md
Normal file
@ -0,0 +1,311 @@
|
||||
Love Your Bugs
|
||||
============================================================
|
||||
|
||||
In early October I gave a keynote at [Python Brasil][1] in Belo Horizonte. Here is an aspirational and lightly edited transcript of the talk. There is also a video available [here][2].
|
||||
|
||||
### I love bugs
|
||||
|
||||
I’m currently a senior engineer at [Pilot.com][3], working on automating bookkeeping for startups. Before that, I worked for [Dropbox][4] on the desktop client team, and I’ll have a few stories about my work there. Earlier, I was a facilitator at the [Recurse Center][5], a writers retreat for programmers in NYC. I studied astrophysics in college and worked in finance for a few years before becoming an engineer.
|
||||
|
||||
But none of that is really important to remember – the only thing you need to know about me is that I love bugs. I love bugs because they’re entertaining. They’re dramatic. The investigation of a great bug can be full of twists and turns. A great bug is like a good joke or a riddle – you’re expecting one outcome, but the result veers off in another direction.
|
||||
|
||||
Over the course of this talk I’m going to tell you about some bugs that I have loved, explain why I love bugs so much, and then convince you that you should love bugs too.
|
||||
|
||||
### Bug #1
|
||||
|
||||
Ok, straight into bug #1\. This is a bug that I encountered while working at Dropbox. As you may know, Dropbox is a utility that syncs your files from one computer to the cloud and to your other computers.
|
||||
|
||||
|
||||
|
||||
```
|
||||
+--------------+ +---------------+
|
||||
| | | |
|
||||
| METASERVER | | BLOCKSERVER |
|
||||
| | | |
|
||||
+-+--+---------+ +---------+-----+
|
||||
^ | ^
|
||||
| | |
|
||||
| | +----------+ |
|
||||
| +---> | | |
|
||||
| | CLIENT +--------+
|
||||
+--------+ |
|
||||
+----------+
|
||||
```
|
||||
|
||||
|
||||
Here’s a vastly simplified diagram of Dropbox’s architecture. The desktop client runs on your local computer listening for changes in the file system. When it notices a changed file, it reads the file, then hashes the contents in 4MB blocks. These blocks are stored in the backend in a giant key-value store that we call blockserver. The key is the digest of the hashed contents, and the values are the contents themselves.
|
||||
|
||||
Of course, we want to avoid uploading the same block multiple times. You can imagine that if you’re writing a document, you’re probably mostly changing the end – we don’t want to upload the beginning over and over. So before uploading a block to the blockserver the client talks to a different server that’s responsible for managing metadata and permissions, among other things. The client asks metaserver whether it needs the block or has seen it before. The “metaserver” responds with whether or not each block needs to be uploaded.
|
||||
|
||||
So the request and response look roughly like this: The client says, “I have a changed file made up of blocks with hashes `'abcd,deef,efgh'`”. The server responds, “I have those first two, but upload the third.” Then the client sends the block up to the blockserver.
|
||||
|
||||
|
||||
```
|
||||
+--------------+ +---------------+
|
||||
| | | |
|
||||
| METASERVER | | BLOCKSERVER |
|
||||
| | | |
|
||||
+-+--+---------+ +---------+-----+
|
||||
^ | ^
|
||||
| | 'ok, ok, need' |
|
||||
'abcd,deef,efgh' | | +----------+ | efgh: [contents]
|
||||
| +---> | | |
|
||||
| | CLIENT +--------+
|
||||
+--------+ |
|
||||
+----------+
|
||||
```
|
||||
|
||||
|
||||
|
||||
That’s the setup. So here’s the bug.
|
||||
|
||||
|
||||
|
||||
```
|
||||
+--------------+
|
||||
| |
|
||||
| METASERVER |
|
||||
| |
|
||||
+-+--+---------+
|
||||
^ |
|
||||
| | '???'
|
||||
'abcdldeef,efgh' | | +----------+
|
||||
^ | +---> | |
|
||||
^ | | CLIENT +
|
||||
+--------+ |
|
||||
+----------+
|
||||
```
|
||||
|
||||
Sometimes the client would make a weird request: each hash value should have been sixteen characters long, but instead it was thirty-three characters long – twice as many plus one. The server wouldn’t know what to do with this and would throw an exception. We’d see this exception get reported, and we’d go look at the log files from the desktop client, and really weird stuff would be going on – the client’s local database had gotten corrupted, or python would be throwing MemoryErrors, and none of it would make sense.
|
||||
|
||||
If you’ve never seen this problem before, it’s totally mystifying. But once you’d seen it once, you can recognize it every time thereafter. Here’s a hint: the middle character of each 33-character string that we’d often see instead of a comma was `l`. These are the other characters we’d see in the middle position:
|
||||
|
||||
|
||||
```
|
||||
l \x0c < $ ( . -
|
||||
```
|
||||
|
||||
The ordinal value for an ascii comma – `,` – is 44\. The ordinal value for `l` is 108\. In binary, here’s how those two are represented:
|
||||
|
||||
```
|
||||
bin(ord(',')): 0101100
|
||||
bin(ord('l')): 1101100
|
||||
```
|
||||
|
||||
You’ll notice that an `l` is exactly one bit away from a comma. And herein lies your problem: a bitflip. One bit of memory that the desktop client is using has gotten corrupted, and now the desktop client is sending a request to the server that is garbage.
|
||||
|
||||
And here are the other characters we’d frequently see instead of the comma when a different bit had been flipped.
|
||||
|
||||
|
||||
|
||||
```
|
||||
, : 0101100
|
||||
l : 1101100
|
||||
\x0c : 0001100
|
||||
< : 0111100
|
||||
$ : 0100100
|
||||
( : 0101000
|
||||
. : 0101110
|
||||
- : 0101101
|
||||
```
|
||||
|
||||
|
||||
### Bitflips are real!
|
||||
|
||||
I love this bug because it shows that bitflips are a real thing that can happen, not just a theoretical concern. In fact, there are some domains where they’re more common than others. One such domain is if you’re getting requests from users with low-end or old hardware, which is true for a lot of laptops running Dropbox. Another domain with lots of bitflips is outer space – there’s no atmosphere in space to protect your memory from energetic particles and radiation, so bitflips are pretty common.
|
||||
|
||||
You probably really care about correctness in space – your code might be keeping astronauts alive on the ISS, for example, but even if it’s not mission-critical, it’s hard to do software updates to space. If you really need your application to defend against bitflips, there are a variety of hardware & software approaches you can take, and there’s a [very interesting talk][6] by Katie Betchold about this.
|
||||
|
||||
Dropbox in this context doesn’t really need to protect against bitflips. The machine that is corrupting memory is a user’s machine, so we can detect if the bitflip happens to fall in the comma – but if it’s in a different character we don’t necessarily know it, and if the bitflip is in the actual file data read off of disk, then we have no idea. There’s a pretty limited set of places where we could address this, and instead we decide to basically silence the exception and move on. Often this kind of bug resolves after the client restarts.
|
||||
|
||||
### Unlikely bugs aren’t impossible
|
||||
|
||||
This is one of my favorite bugs for a couple of reasons. The first is that it’s a reminder of the difference between unlikely and impossible. At sufficient scale, unlikely events start to happen at a noticable rate.
|
||||
|
||||
### Social bugs
|
||||
|
||||
My second favorite thing about this bug is that it’s a tremendously social one. This bug can crop up anywhere that the desktop client talks to the server, which is a lot of different endpoints and components in the system. This meant that a lot of different engineers at Dropbox would see versions of the bug. The first time you see it, you can _really_ scratch your head, but after that it’s easy to diagnose, and the investigation is really quick: you look at the middle character and see if it’s an `l`.
|
||||
|
||||
### Cultural differences
|
||||
|
||||
One interesting side-effect of this bug was that it exposed a cultural difference between the server and client teams. Occasionally this bug would be spotted by a member of the server team and investigated from there. If one of your _servers_ is flipping bits, that’s probably not random chance – it’s probably memory corruption, and you need to find the affected machine and get it out of the pool as fast as possible or you risk corrupting a lot of user data. That’s an incident, and you need to respond quickly. But if the user’s machine is corrupting data, there’s not a lot you can do.
|
||||
|
||||
### Share your bugs
|
||||
|
||||
So if you’re investigating a confusing bug, especially one in a big system, don’t forget to talk to people about it. Maybe your colleagues have seen a bug shaped like this one before. If they have, you might save a lot of time. And if they haven’t, don’t forget to tell people about the solution once you’ve figured it out – write it up or tell the story in your team meeting. Then the next time your teams hits something similar, you’ll all be more prepared.
|
||||
|
||||
### How bugs can help you learn
|
||||
|
||||
### Recurse Center
|
||||
|
||||
Before I joined Dropbox, I worked for the Recurse Center. The idea behind RC is that it’s a community of self-directed learners spending time together getting better as programmers. That is the full extent of the structure of RC: there’s no curriculum or assignments or deadlines. The only scoping is a shared goal of getting better as a programmer. We’d see people come to participate in the program who had gotten CS degrees but didn’t feel like they had a solid handle on practical programming, or people who had been writing Java for ten years and wanted to learn Clojure or Haskell, and many other profiles as well.
|
||||
|
||||
My job there was as a facilitator, helping people make the most of the lack of structure and providing guidance based on what we’d learned from earlier participants. So my colleagues and I were very interested in the best techniques for learning for self-motivated adults.
|
||||
|
||||
### Deliberate Practice
|
||||
|
||||
There’s a lot of different research in this space, and one of the ones I think is most interesting is the idea of deliberate practice. Deliberate practice is an attempt to explain the difference in performance between experts & amateurs. And the guiding principle here is that if you look just at innate characteristics – genetic or otherwise – they don’t go very far towards explaining the difference in performance. So the researchers, originally Ericsson, Krampe, and Tesch-Romer, set out to discover what did explain the difference. And what they settled on was time spent in deliberate practice.
|
||||
|
||||
Deliberate practice is pretty narrow in their definition: it’s not work for pay, and it’s not playing for fun. You have to be operating on the edge of your ability, doing a project appropriate for your skill level (not so easy that you don’t learn anything and not so hard that you don’t make any progress). You also have to get immediate feedback on whether or not you’ve done the thing correctly.
|
||||
|
||||
This is really exciting, because it’s a framework for how to build expertise. But the challenge is that as programmers this is really hard advice to apply. It’s hard to know whether you’re operating at the edge of your ability. Immediate corrective feedback is very rare – in some cases you’re lucky to get feedback ever, and in other cases maybe it takes months. You can get quick feedback on small things in the REPL and so on, but if you’re making a design decision or picking a technology, you’re not going to get feedback on those things for quite a long time.
|
||||
|
||||
But one category of programming where deliberate practice is a useful model is debugging. If you wrote code, then you had a mental model of how it worked when you wrote it. But your code has a bug, so your mental model isn’t quite right. By definition you’re on the boundary of your understanding – so, great! You’re about to learn something new. And if you can reproduce the bug, that’s a rare case where you can get immediate feedback on whether or not your fix is correct.
|
||||
|
||||
A bug like this might teach you something small about your program, or you might learn something larger about the system your code is running in. Now I’ve got a story for you about a bug like that.
|
||||
|
||||
### Bug #2
|
||||
|
||||
This bug also one that I encountered at Dropbox. At the time, I was investigating why some desktop client weren’t sending logs as consistently as we expected. I’d started digging into the client logging system and discovered a bunch of interesting bugs. I’ll tell you only the subset of those bugs that is relevant to this story.
|
||||
|
||||
Again here’s a very simplified architecture of the system.
|
||||
|
||||
|
||||
```
|
||||
+--------------+
|
||||
| |
|
||||
+---+ +----------> | LOG SERVER |
|
||||
|log| | | |
|
||||
+---+ | +------+-------+
|
||||
| |
|
||||
+-----+----+ | 200 ok
|
||||
| | |
|
||||
| CLIENT | <-----------+
|
||||
| |
|
||||
+-----+----+
|
||||
^
|
||||
+--------+--------+--------+
|
||||
| ^ ^ |
|
||||
+--+--+ +--+--+ +--+--+ +--+--+
|
||||
| log | | log | | log | | log |
|
||||
| | | | | | | |
|
||||
| | | | | | | |
|
||||
+-----+ +-----+ +-----+ +-----+
|
||||
```
|
||||
|
||||
The desktop client would generate logs. Those logs were compress, encrypted, and written to disk. Then every so often the client would send them up to the server. The client would read a log off of disk and send it to the log server. The server would decrypt it and store it, then respond with a 200.
|
||||
|
||||
If the client couldn’t reach the log server, it wouldn’t let the log directory grow unbounded. After a certain point it would start deleting logs to keep the directory under a maximum size.
|
||||
|
||||
The first two bugs were not a big deal on their own. The first one was that the desktop client sent logs up to the server starting with the oldest one instead of starting with the newest. This isn’t really what you want – for example, the server would tell the client to send logs if the client reported an exception, so probably you care about the logs that just happened and not the oldest logs that happen to be on disk.
|
||||
|
||||
The second bug was similar to the first: if the log directory hit its maximum size, the client would delete the logs starting with the newest instead of starting with the oldest. Again, you lose log files either way, but you probably care less about the older ones.
|
||||
|
||||
The third bug had to do with the encryption. Sometimes, the server would be unable to decrypt a log file. (We generally didn’t figure out why – maybe it was a bitflip.) We weren’t handling this error correctly on the backend, so the server would reply with a 500\. The client would behave reasonably in the face of a 500: it would assume that the server was down. So it would stop sending log files and not try to send up any of the others.
|
||||
|
||||
Returning a 500 on a corrupted log file is clearly not the right behavior. You could consider returning a 400, since it’s a problem with the client request. But the client also can’t fix the problem – if the log file can’t be decrypted now, we’ll never be able to decrypt it in the future. What you really want the client to do is just delete the log and move on. In fact, that’s the default behavior when the client gets a 200 back from the server for a log file that was successfully stored. So we said, ok – if the log file can’t be decrypted, just return a 200.
|
||||
|
||||
All of these bugs were straightforward to fix. The first two bugs were on the client, so we’d fixed them on the alpha build but they hadn’t gone out to the majority of clients. The third bug we fixed on the server and deployed.
|
||||
|
||||
### 📈
|
||||
|
||||
Suddenly traffic to the log cluster spikes. The serving team reaches out to us to ask if we know what’s going on. It takes me a minute to put all the pieces together.
|
||||
|
||||
Before these fixes, there were four things going on:
|
||||
|
||||
1. Log files were sent up starting with the oldest
|
||||
|
||||
2. Log files were deleted starting with the newest
|
||||
|
||||
3. If the server couldn’t decrypt a log file it would 500
|
||||
|
||||
4. If the client got a 500 it would stop sending logs
|
||||
|
||||
A client with a corrupted log file would try to send it, the server would 500, the client would give up sending logs. On its next run, it would try to send the same file again, fail again, and give up again. Eventually the log directory would get full, at which point the client would start deleting its newest files, leaving the corrupted one on disk.
|
||||
|
||||
The upshot of these three bugs: if a client ever had a corrupted log file, we would never see logs from that client again.
|
||||
|
||||
The problem is that there were a lot more clients in this state than we thought. Any client with a single corrupted file had been dammed up from sending logs to the server. Now that dam was cleared, and all of them were sending up the rest of the contents of their log directories.
|
||||
|
||||
### Our options
|
||||
|
||||
Ok, there’s a huge flood of traffic coming from machines around the world. What can we do? (This is a fun thing about working at a company with Dropbox’s scale, and particularly Dropbox’s scale of desktop clients: you can trigger a self-DDOS very easily.)
|
||||
|
||||
The first option when you do a deploy and things start going sideways is to rollback. Totally reasonable choice, but in this case, it wouldn’t have helped us. The state that we’d transformed wasn’t the state on the server but the state on the client – we’d deleted those files. Rolling back the server would prevent additional clients from entering this state but it wouldn’t solve the problem.
|
||||
|
||||
What about increasing the size of the logging cluster? We did that – and started getting even more requests, now that we’d increased our capacity. We increased it again, but you can’t do that forever. Why not? This cluster isn’t isolated. It’s making requests into another cluster, in this case to handle exceptions. If you have a DDOS pointed at one cluster, and you keep scaling that cluster, you’re going to knock over its depedencies too, and now you have two problems.
|
||||
|
||||
Another option we considered was shedding load – you don’t need every single log file, so can we just drop requests. One of the challenges here was that we didn’t have an easy way to tell good traffic from bad. We couldn’t quickly differentiate which log files were old and which were new.
|
||||
|
||||
The solution we hit on is one that’s been used at Dropbox on a number of different occassions: we have a custom header, `chillout`, which every client in the world respects. If the client gets a response with this header, then it doesn’t make any requests for the provided number of seconds. Someone very wise added this to the Dropbox client very early on, and it’s come in handy more than once over the years. The logging server didn’t have the ability to set that header, but that’s an easy problem to solve. So two of my colleagues, Isaac Goldberg and John Lai, implemented support for it. We set the logging cluster chillout to two minutes initially and then managed it down as the deluge subsided over the next couple of days.
|
||||
|
||||
### Know your system
|
||||
|
||||
The first lesson from this bug is to know your system. I had a good mental model of the interaction between the client and the server, but I wasn’t thinking about what would happen when the server was interacting with all the clients at once. There was a level of complexity that I hadn’t thought all the way through.
|
||||
|
||||
### Know your tools
|
||||
|
||||
The second lesson is to know your tools. If things go sideways, what options do you have? Can you reverse your migration? How will you know if things are going sideways and how can you discover more? All of those things are great to know before a crisis – but if you don’t, you’ll learn them during a crisis and then never forget.
|
||||
|
||||
### Feature flags & server-side gating
|
||||
|
||||
The third lesson is for you if you’re writing a mobile or a desktop application: _You need server-side feature gating and server-side flags._ When you discover a problem and you don’t have server-side controls, the resolution might take days or weeks as you push out a new release or submit a new version to the app store. That’s a bad situation to be in. The Dropbox desktop client isn’t going through an app store review process, but just pushing out a build to tens of millions of clients takes time. Compare that to hitting a problem in your feature and flipping a switch on the server: ten minutes later your problem is resolved.
|
||||
|
||||
This strategy is not without its costs. Having a bunch of feature flags in your code adds to the complexity dramatically. You get a combinatoric problem with your testing: what if feature A is enabled and feature B, or just one, or neither – multiplied across N features. It’s extremely difficult to get engineers to clean up their feature flags after the fact (and I was also guilty of this). Then for the desktop client there’s multiple versions in the wild at the same time, so it gets pretty hard to reason about.
|
||||
|
||||
But the benefit – man, when you need it, you really need it.
|
||||
|
||||
# How to love bugs
|
||||
|
||||
I’ve talked about some bugs that I love and I’ve talked about why to love bugs. Now I want to tell you how to love bugs. If you don’t love bugs yet, I know of exactly one way to learn, and that’s to have a growth mindset.
|
||||
|
||||
The sociologist Carol Dweck has done a ton of interesting research about how people think about intelligence. She’s found that there are two different frameworks for thinking about intelligence. The first, which she calls the fixed mindset, holds that intelligence is a fixed trait, and people can’t change how much of it they have. The other mindset is a growth mindset. Under a growth mindset, people believe that intelligence is malleable and can increase with effort.
|
||||
|
||||
Dweck found that a person’s theory of intelligence – whether they hold a fixed or growth mindset – can significantly influence the way they select tasks to work on, the way they respond to challenges, their cognitive performance, and even their honesty.
|
||||
|
||||
[I also talked about a growth mindset in my Kiwi PyCon keynote, so here are just a few excerpts. You can read the full transcript [here][7].]
|
||||
|
||||
Findings about honesty:
|
||||
|
||||
> After this, they had the students write letters to pen pals about the study, saying “We did this study at school, and here’s the score that I got.” They found that _almost half of the students praised for intelligence lied about their scores_ , and almost no one who was praised for working hard was dishonest.
|
||||
|
||||
On effort:
|
||||
|
||||
> Several studies found that people with a fixed mindset can be reluctant to really exert effort, because they believe it means they’re not good at the thing they’re working hard on. Dweck notes, “It would be hard to maintain confidence in your ability if every time a task requires effort, your intelligence is called into question.”
|
||||
|
||||
On responding to confusion:
|
||||
|
||||
> They found that students with a growth mindset mastered the material about 70% of the time, regardless of whether there was a confusing passage in it. Among students with a fixed mindset, if they read the booklet without the confusing passage, again about 70% of them mastered the material. But the fixed-mindset students who encountered the confusing passage saw their mastery drop to 30%. Students with a fixed mindset were pretty bad at recovering from being confused.
|
||||
|
||||
These findings show that a growth mindset is critical while debugging. We have to recover from confusion, be candid about the limitations of our understanding, and at times really struggle on the way to finding solutions – all of which is easier and less painful with a growth mindset.
|
||||
|
||||
### Love your bugs
|
||||
|
||||
I learned to love bugs by explicitly celebrating challenges while working at the Recurse Center. A participant would sit down next to me and say, “[sigh] I think I’ve got a weird Python bug,” and I’d say, “Awesome, I _love_ weird Python bugs!” First of all, this is definitely true, but more importantly, it emphasized to the participant that finding something where they struggled an accomplishment, and it was a good thing for them to have done that day.
|
||||
|
||||
As I mentioned, at the Recurse Center there are no deadlines and no assignments, so this attitude is pretty much free. I’d say, “You get to spend a day chasing down this weird bug in Flask, how exciting!” At Dropbox and later at Pilot, where we have a product to ship, deadlines, and users, I’m not always uniformly delighted about spending a day on a weird bug. So I’m sympathetic to the reality of the world where there are deadlines. However, if I have a bug to fix, I have to fix it, and being grumbly about the existence of the bug isn’t going to help me fix it faster. I think that even in a world where deadlines loom, you can still apply this attitude.
|
||||
|
||||
If you love your bugs, you can have more fun while you’re working on a tough problem. You can be less worried and more focused, and end up learning more from them. Finally, you can share a bug with your friends and colleagues, which helps you and your teammates.
|
||||
|
||||
### Obrigada!
|
||||
|
||||
My thanks to folks who gave me feedback on this talk and otherwise contributed to my being there:
|
||||
|
||||
* Sasha Laundy
|
||||
|
||||
* Amy Hanlon
|
||||
|
||||
* Julia Evans
|
||||
|
||||
* Julian Cooper
|
||||
|
||||
* Raphael Passini Diniz and the rest of the Python Brasil organizing team
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: http://akaptur.com/blog/2017/11/12/love-your-bugs/
|
||||
|
||||
作者:[Allison Kaptur ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:http://akaptur.com/about/
|
||||
[1]:http://2017.pythonbrasil.org.br/#
|
||||
[2]:http://www.youtube.com/watch?v=h4pZZOmv4Qs
|
||||
[3]:http://www.pilot.com/
|
||||
[4]:http://www.dropbox.com/
|
||||
[5]:http://www.recurse.com/
|
||||
[6]:http://www.youtube.com/watch?v=ETgNLF_XpEM
|
||||
[7]:http://akaptur.com/blog/2015/10/10/effective-learning-strategies-for-programmers/
|
61
sources/tech/20171114 Sysadmin 101 Patch Management.md
Normal file
61
sources/tech/20171114 Sysadmin 101 Patch Management.md
Normal file
@ -0,0 +1,61 @@
|
||||
【翻译中 @haoqixu】Sysadmin 101: Patch Management
|
||||
============================================================
|
||||
|
||||
* [HOW-TOs][1]
|
||||
|
||||
* [Servers][2]
|
||||
|
||||
* [SysAdmin][3]
|
||||
|
||||
|
||||
A few articles ago, I started a Sysadmin 101 series to pass down some fundamental knowledge about systems administration that the current generation of junior sysadmins, DevOps engineers or "full stack" developers might not learn otherwise. I had thought that I was done with the series, but then the WannaCry malware came out and exposed some of the poor patch management practices still in place in Windows networks. I imagine some readers that are still stuck in the Linux versus Windows wars of the 2000s might have even smiled with a sense of superiority when they heard about this outbreak.
|
||||
|
||||
The reason I decided to revive my Sysadmin 101 series so soon is I realized that most Linux system administrators are no different from Windows sysadmins when it comes to patch management. Honestly, in some areas (in particular, uptime pride), some Linux sysadmins are even worse than Windows sysadmins regarding patch management. So in this article, I cover some of the fundamentals of patch management under Linux, including what a good patch management system looks like, the tools you will want to put in place and how the overall patching process should work.
|
||||
|
||||
### What Is Patch Management?
|
||||
|
||||
When I say patch management, I'm referring to the systems you have in place to update software already on a server. I'm not just talking about keeping up with the latest-and-greatest bleeding-edge version of a piece of software. Even more conservative distributions like Debian that stick with a particular version of software for its "stable" release still release frequent updates that patch bugs or security holes.
|
||||
|
||||
Of course, if your organization decided to roll its own version of a particular piece of software, either because developers demanded the latest and greatest, you needed to fork the software to apply a custom change, or you just like giving yourself extra work, you now have a problem. Ideally you have put in a system that automatically packages up the custom version of the software for you in the same continuous integration system you use to build and package any other software, but many sysadmins still rely on the outdated method of packaging the software on their local machine based on (hopefully up to date) documentation on their wiki. In either case, you will need to confirm that your particular version has the security flaw, and if so, make sure that the new patch applies cleanly to your custom version.
|
||||
|
||||
### What Good Patch Management Looks Like
|
||||
|
||||
Patch management starts with knowing that there is a software update to begin with. First, for your core software, you should be subscribed to your Linux distribution's security mailing list, so you're notified immediately when there are security patches. If there you use any software that doesn't come from your distribution, you must find out how to be kept up to date on security patches for that software as well. When new security notifications come in, you should review the details so you understand how severe the security flaw is, whether you are affected and gauge a sense of how urgent the patch is.
|
||||
|
||||
Some organizations have a purely manual patch management system. With such a system, when a security patch comes along, the sysadmin figures out which servers are running the software, generally by relying on memory and by logging in to servers and checking. Then the sysadmin uses the server's built-in package management tool to update the software with the latest from the distribution. Then the sysadmin moves on to the next server, and the next, until all of the servers are patched.
|
||||
|
||||
There are many problems with manual patch management. First is the fact that it makes patching a laborious chore. The more work patching is, the more likely a sysadmin will put it off or skip doing it entirely. The second problem is that manual patch management relies too much on the sysadmin's ability to remember and recall all of the servers he or she is responsible for and keep track of which are patched and which aren't. This makes it easy for servers to be forgotten and sit unpatched.
|
||||
|
||||
The faster and easier patch management is, the more likely you are to do it. You should have a system in place that quickly can tell you which servers are running a particular piece of software at which version. Ideally, that system also can push out updates. Personally, I prefer orchestration tools like MCollective for this task, but Red Hat provides Satellite, and Canonical provides Landscape as central tools that let you view software versions across your fleet of servers and apply patches all from a central place.
|
||||
|
||||
Patching should be fault-tolerant as well. You should be able to patch a service and restart it without any overall down time. The same idea goes for kernel patches that require a reboot. My approach is to divide my servers into different high availability groups so that lb1, app1, rabbitmq1 and db1 would all be in one group, and lb2, app2, rabbitmq2 and db2 are in another. Then, I know I can patch one group at a time without it causing downtime anywhere else.
|
||||
|
||||
So, how fast is fast? Your system should be able to roll out a patch to a minor piece of software that doesn't have an accompanying service (such as bash in the case of the ShellShock vulnerability) within a few minutes to an hour at most. For something like OpenSSL that requires you to restart services, the careful process of patching and restarting services in a fault-tolerant way probably will take more time, but this is where orchestration tools come in handy. I gave examples of how to use MCollective to accomplish this in my recent MCollective articles (see the December 2016 and January 2017 issues), but ideally, you should put a system in place that makes it easy to patch and restart services in a fault-tolerant and automated way.
|
||||
|
||||
When patching requires a reboot, such as in the case of kernel patches, it might take a bit more time, but again, automation and orchestration tools can make this go much faster than you might imagine. I can patch and reboot the servers in an environment in a fault-tolerant way within an hour or two, and it would be much faster than that if I didn't need to wait for clusters to sync back up in between reboots.
|
||||
|
||||
Unfortunately, many sysadmins still hold on to the outdated notion that uptime is a badge of pride—given that serious kernel patches tend to come out at least once a year if not more often, to me, it's proof you don't take security seriously.
|
||||
|
||||
Many organizations also still have that single point of failure server that can never go down, and as a result, it never gets patched or rebooted. If you want to be secure, you need to remove these outdated liabilities and create systems that at least can be rebooted during a late-night maintenance window.
|
||||
|
||||
Ultimately, fast and easy patch management is a sign of a mature and professional sysadmin team. Updating software is something all sysadmins have to do as part of their jobs, and investing time into systems that make that process easy and fast pays dividends far beyond security. For one, it helps identify bad architecture decisions that cause single points of failure. For another, it helps identify stagnant, out-of-date legacy systems in an environment and provides you with an incentive to replace them. Finally, when patching is managed well, it frees up sysadmins' time and turns their attention to the things that truly require their expertise.
|
||||
|
||||
______________________
|
||||
|
||||
Kyle Rankin is senior security and infrastructure architect, the author of many books including Linux Hardening in Hostile Networks, DevOps Troubleshooting and The Official Ubuntu Server Book, and a columnist for Linux Journal. Follow him @kylerankin
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linuxjournal.com/content/sysadmin-101-patch-management
|
||||
|
||||
作者:[Kyle Rankin ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linuxjournal.com/users/kyle-rankin
|
||||
[1]:https://www.linuxjournal.com/tag/how-tos
|
||||
[2]:https://www.linuxjournal.com/tag/servers
|
||||
[3]:https://www.linuxjournal.com/tag/sysadmin
|
||||
[4]:https://www.linuxjournal.com/users/kyle-rankin
|
68
sources/tech/20171114 Take Linux and Run With It.md
Normal file
68
sources/tech/20171114 Take Linux and Run With It.md
Normal file
@ -0,0 +1,68 @@
|
||||
Take Linux and Run With It
|
||||
============================================================
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
|
||||
"How do you run an operating system?" may seem like a simple question, since most of us are accustomed to turning on our computers and seeing our system spin up. However, this common model is only one way of running an operating system. As one of Linux's greatest strengths is versatility, Linux offers the most methods and environments for running it.
|
||||
|
||||
To unleash the full power of Linux, and maybe even find a use for it you hadn't thought of, consider some less conventional ways of running it -- specifically, ones that don't even require installation on a computer's hard drive.
|
||||
|
||||
### We'll Do It Live!
|
||||
|
||||
Live-booting is a surprisingly useful and popular way to get the full Linux experience on the fly. While hard drives are where OSes reside most of the time, they actually can be installed to most major storage media, including CDs, DVDs and USB flash drives.
|
||||
|
||||
When an OS is installed to some device other than a computer's onboard hard drive and subsequently booted instead of that onboard drive, it's called "live-booting" or running a "live session."
|
||||
|
||||
At boot time, the user simply selects an external storage source for the hardware to look for boot information. If found, the computer follows the external device's boot instructions, essentially ignoring the onboard drive until the next time the user boots normally. Optical media are increasingly rare these days, so by far the most typical form that an external OS-carrying device takes is a USB stick.
|
||||
|
||||
Most mainstream Linux distributions offer a way to run a live session as a way of trying them out. The live session doesn't save any user activity, and the OS resets to the clean default state after every shutdown.
|
||||
|
||||
Live Linux sessions can be used for more than testing a distro, though. One application is for executing system repair for critically malfunctioning onboard (usually also Linux) systems. If an update or configuration made the onboard system unbootable, a full system backup is required, or the hard drive has sustained serious file corruption, the only recourse is to start up a live system and perform maintenance on the onboard drive.
|
||||
|
||||
In these and similar scenarios, the onboard drive cannot be manipulated or corrected while also keeping the system stored on it running, so a live system takes on those burdens instead, leaving all but the problematic files on the onboard drive at rest.
|
||||
|
||||
Live sessions also are perfectly suited for handling sensitive information. If you don't want a computer to retain any trace of the operations executed or information handled on it, especially if you are using hardware you can't vouch for -- like a public library or hotel business center computer -- a live session will provide you all the desktop computing functions to complete your task while retaining no trace of your session once you're finished. This is great for doing online banking or password input that you don't want a computer to remember.
|
||||
|
||||
### Linux Virtually Anywhere
|
||||
|
||||
Another approach for implementing Linux for more on-demand purposes is to run a virtual machine on another host OS. A virtual machine, or VM, is essentially a small computer running inside another computer and contained in a single large file.
|
||||
|
||||
To run a VM, users simply install a hypervisor program (a kind of launcher for the VM), select a downloaded Linux OS image file (usually ending with a ".iso" file extension), and walk through the setup process.
|
||||
|
||||
Most of the settings can be left at their defaults, but the key ones to configure are the amount of RAM and hard drive storage to lease to the VM. Fortunately, since Linux has a light footprint, you don't have to set these very high: 2 GB of RAM and 16 GB of storage should be plenty for the VM while still letting your host OS thrive.
|
||||
|
||||
So what does this offer that a live system doesn't? First, whereas live systems are ephemeral, VMs can retain the data stored on them. This is great if you want to set up your Linux VM for a special use case, like software development or even security.
|
||||
|
||||
When used for development, a Linux VM gives you the solid foundation of Linux's programming language suites and coding tools, and it lets you save your projects right in the VM to keep everything organized.
|
||||
|
||||
If security is your goal, Linux VMs allow you to impose an extra layer between a potential hazard and your system. If you do your browsing from the VM, a malicious program would have to compromise not only your virtual Linux system, but also the hypervisor -- and _then_ your host OS, a technical feat beyond all but the most skilled and determined adversaries.
|
||||
|
||||
Second, you can start up your VM on demand from your host system, without having to power it down and start it up again as you would have to with a live session. When you need it, you can quickly bring up the VM, and when you're finished, you just shut it down and go back to what you were doing before.
|
||||
|
||||
Your host system continues running normally while the VM is on, so you can attend to tasks simultaneously in each system.
|
||||
|
||||
### Look Ma, No Installation!
|
||||
|
||||
Just as there is no one form that Linux takes, there's also no one way to run it. Hopefully, this brief primer on the kinds of systems you can run has given you some ideas to expand your use models.
|
||||
|
||||
The best part is that if you're not sure how these can help, live booting and virtual machines don't hurt to try!
|
||||

|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linuxinsider.com/story/Take-Linux-and-Run-With-It-84951.html
|
||||
|
||||
作者:[ Jonathan Terrasi ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linuxinsider.com/story/Take-Linux-and-Run-With-It-84951.html#searchbyline
|
||||
[1]:https://www.linuxinsider.com/story/Take-Linux-and-Run-With-It-84951.html#
|
||||
[2]:https://www.linuxinsider.com/perl/mailit/?id=84951
|
||||
[3]:https://www.linuxinsider.com/story/Take-Linux-and-Run-With-It-84951.html
|
||||
[4]:https://www.linuxinsider.com/story/Take-Linux-and-Run-With-It-84951.html
|
@ -0,0 +1,58 @@
|
||||
Security Jobs Are Hot: Get Trained and Get Noticed
|
||||
============================================================
|
||||
|
||||

|
||||
The Open Source Jobs Report, from Dice and The Linux Foundation, found that professionals with security experience are in high demand for the future.[Used with permission][1]
|
||||
|
||||
The demand for security professionals is real. On [Dice.com][4], 15 percent of the more than 75K jobs are security positions. “Every year in the U.S., 40,000 jobs for information security analysts go unfilled, and employers are struggling to fill 200,000 other cyber-security related roles, according to cyber security data tool [CyberSeek][5]” ([Forbes][6]). We know that there is a fast-increasing need for security specialists, but that the interest level is low.
|
||||
|
||||
### Security is the place to be
|
||||
|
||||
In my experience, few students coming out of college are interested in roles in security; so many people see security as niche. Entry-level tech pros are interested in business analyst or system analyst roles, because of a belief that if you want to learn and apply core IT concepts, you have to stick to analyst roles or those closer to product development. That’s simply not the case.
|
||||
|
||||
In fact, if you’re interested in getting in front of your business leaders, security is the place to be – as a security professional, you have to understand the business end-to-end; you have to look at the big picture to give your company the advantage.
|
||||
|
||||
### Be fearless
|
||||
|
||||
Analyst and security roles are not all that different. Companies continue to merge engineering and security roles out of necessity. Businesses are moving faster than ever with infrastructure and code being deployed through automation, which increases the importance of security being a part of all tech pros day to day lives. In our [Open Source Jobs Report with The Linux Foundation][7], 42 percent of hiring managers said professionals with security experience are in high demand for the future.
|
||||
|
||||
There has never been a more exciting time to be in security. If you stay up-to-date with tech news, you’ll see that a huge number of stories are related to security – data breaches, system failures and fraud. The security teams are working in ever-changing, fast-paced environments. A real challenge lies is in the proactive side of security, finding, and eliminating vulnerabilities while maintaining or even improving the end-user experience.
|
||||
|
||||
### Growth is imminent
|
||||
|
||||
Of any aspect of tech, security is the one that will continue to grow with the cloud. Businesses are moving more and more to the cloud and that’s exposing more security vulnerabilities than organizations are used to. As the cloud matures, security becomes increasingly important.
|
||||
|
||||
Regulations are also growing – Personally Identifiable Information (PII) is getting broader all the time. Many companies are finding that they must invest in security to stay in compliance and avoid being in the headlines. Companies are beginning to budget more and more for security tooling and staffing due to the risk of heavy fines, reputational damage, and, to be honest, executive job security.
|
||||
|
||||
### Training and support
|
||||
|
||||
Even if you don’t choose a security-specific role, you’re bound to find yourself needing to code securely, and if you don’t have the skills to do that, you’ll start fighting an uphill battle. There are certainly ways to learn on-the-job if your company offers that option, that’s encouraged but I recommend a combination of training, mentorship and constant practice. Without using your security skills, you’ll lose them fast with how quickly the complexity of malicious attacks evolve.
|
||||
|
||||
My recommendation for those seeking security roles is to find the people in your organization that are the strongest in engineering, development, or architecture areas – interface with them and other teams, do hands-on work, and be sure to keep the big-picture in mind. Be an asset to your organization that stands out – someone that can securely code and also consider strategy and overall infrastructure health.
|
||||
|
||||
### The end game
|
||||
|
||||
More and more companies are investing in security and trying to fill open roles in their tech teams. If you’re interested in management, security is the place to be. Executive leadership wants to know that their company is playing by the rules, that their data is secure, and that they’re safe from breaches and loss.
|
||||
|
||||
Security that is implemented wisely and with strategy in mind will get noticed. Security is paramount for executives and consumers alike – I’d encourage anyone interested in security to train up and contribute.
|
||||
|
||||
_[Download ][2]the full 2017 Open Source Jobs Report now._
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linux.com/blog/os-jobs-report/2017/11/security-jobs-are-hot-get-trained-and-get-noticed
|
||||
|
||||
作者:[ BEN COLLEN][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linux.com/users/bencollen
|
||||
[1]:https://www.linux.com/licenses/category/used-permission
|
||||
[2]:http://bit.ly/2017OSSjobsreport
|
||||
[3]:https://www.linux.com/files/images/security-skillspng
|
||||
[4]:http://www.dice.com/
|
||||
[5]:http://cyberseek.org/index.html#about
|
||||
[6]:https://www.forbes.com/sites/jeffkauflin/2017/03/16/the-fast-growing-job-with-a-huge-skills-gap-cyber-security/#292f0a675163
|
||||
[7]:http://media.dice.com/report/the-2017-open-source-jobs-report-employers-prioritize-hiring-open-source-professionals-with-latest-skills/
|
@ -0,0 +1,120 @@
|
||||
Why and How to Set an Open Source Strategy
|
||||
============================================================
|
||||
|
||||

|
||||
|
||||
This article explains how to walk through, measure, and define strategies collaboratively in an open source community.
|
||||
|
||||
_“If you don’t know where you are going, you’ll end up someplace else.” _ _—_ Yogi Berra
|
||||
|
||||
Open source projects are generally started as a way to scratch one’s itch — and frankly that’s one of its greatest attributes. Getting code down provides a tangible method to express an idea, showcase a need, and solve a problem. It avoids over thinking and getting a project stuck in analysis-paralysis, letting the project pragmatically solve the problem at hand.
|
||||
|
||||
Next, a project starts to scale up and gets many varied users and contributions, with plenty of opinions along the way. That leads to the next big challenge — how does a project start to build a strategic vision? In this article, I’ll describe how to walk through, measure, and define strategies collaboratively, in a community.
|
||||
|
||||
Strategy may seem like a buzzword of the corporate world rather something that an open source community would embrace, so I suggest stripping away the negative actions that are sometimes associated with this word (e.g., staff reductions, discontinuations, office closures). Strategy done right isn’t a tool to justify unfortunate actions but to help show focus and where each community member can contribute.
|
||||
|
||||
A good application of strategy achieves the following:
|
||||
|
||||
* Why the project exists?
|
||||
|
||||
* What the project looks to achieve?
|
||||
|
||||
* What is the ideal end state for a project is.
|
||||
|
||||
The key to success is answering these questions as simply as possible, with consensus from your community. Let’s look at some ways to do this.
|
||||
|
||||
### Setting a mission and vision
|
||||
|
||||
_“_ _Efforts and courage are not enough without purpose and direction.”_ — John F. Kennedy
|
||||
|
||||
All strategic planning starts off with setting a course for where the project wants to go. The two tools used here are _Mission_ and _Vision_ . They are complementary terms, describing both the reason a project exists (mission) and the ideal end state for a project (vision).
|
||||
|
||||
A great way to start this exercise with the intent of driving consensus is by asking each key community member the following questions:
|
||||
|
||||
* What drove you to join and/or contribute the project?
|
||||
|
||||
* How do you define success for your participation?
|
||||
|
||||
In a company, you’d ask your customers these questions usually. But in open source projects, the customers are the project participants — and their time investment is what makes the project a success.
|
||||
|
||||
Driving consensus means capturing the answers to these questions and looking for themes across them. At R Consortium, for example, I created a shared doc for the board to review each member’s answers to the above questions, and followed up with a meeting to review for specific themes that came from those insights.
|
||||
|
||||
Building a mission flows really well from this exercise. The key thing is to keep the wording of your mission short and concise. Open Mainframe Project has done this really well. Here’s their mission:
|
||||
|
||||
_Build community and adoption of Open Source on the mainframe by:_
|
||||
|
||||
* _Eliminating barriers to Open Source adoption on the mainframe_
|
||||
|
||||
* _Demonstrating value of the mainframe on technical and business levels_
|
||||
|
||||
* _Strengthening collaboration points and resources for the community to thrive_
|
||||
|
||||
At 40 words, it passes the key eye tests of a good mission statement; it’s clear, concise, and demonstrates the useful value the project aims for.
|
||||
|
||||
The next stage is to reflect on the mission statement and ask yourself this question: What is the ideal outcome if the project accomplishes its mission? That can be a tough one to tackle. Open Mainframe Project put together its vision really well:
|
||||
|
||||
_Linux on the Mainframe as the standard for enterprise class systems and applications._
|
||||
|
||||
You could read that as a [BHAG][1], but it’s really more of a vision, because it describes a future state that is what would be created by the mission being fully accomplished. It also hits the key pieces to an effective vision — it’s only 13 words, inspirational, clear, memorable, and concise.
|
||||
|
||||
Mission and vision add clarity on the who, what, why, and how for your project. But, how do you set a course for getting there?
|
||||
|
||||
### Goals, Objectives, Actions, and Results
|
||||
|
||||
_“I don’t focus on what I’m up against. I focus on my goals and I try to ignore the rest.”_ — Venus Williams
|
||||
|
||||
Looking at a mission and vision can get overwhelming, so breaking them down into smaller chunks can help the project determine how to get started. This also helps prioritize actions, either by importance or by opportunity. Most importantly, this step gives you guidance on what things to focus on for a period of time, and which to put off.
|
||||
|
||||
There are lots of methods of time bound planning, but the method I think works the best for projects is what I’ve dubbed the GOAR method. It’s an acronym that stands for:
|
||||
|
||||
* Goals define what the project is striving for and likely would align and support the mission. Examples might be “Grow a diverse contributor base” or “Become the leading project for X.” Goals are aspirational and set direction.
|
||||
|
||||
* Objectives show how you measure a goal’s completion, and should be clear and measurable. You might also have multiple objectives to measure the completion of a goal. For example, the goal “Grow a diverse contributor base” might have objectives such as “Have X total contributors monthly” and “Have contributors representing Y different organizations.”
|
||||
|
||||
* Actions are what the project plans to do to complete an objective. This is where you get tactical on exactly what needs done. For example, the objective “Have contributors representing Y different organizations” would like have actions of reaching out to interested organizations using the project, having existing contributors mentor new mentors, and providing incentives for first time contributors.
|
||||
|
||||
* Results come along the way, showing progress both positive and negative from the actions.
|
||||
|
||||
You can put these into a table like this:
|
||||
|
||||
| Goals | Objectives | Actions | Results |
|
||||
|:--|:--|:--|:--|
|
||||
| Grow a diverse contributor base | Have X total contributors monthly | Existing contributors mentor new mentors Providing incentives for first time contributors | |
|
||||
| | Have contributors representing Y different organizations | Reach out to interested organizations using the project | |
|
||||
|
||||
|
||||
In large organizations, monthly or quarterly goals and objectives often make sense; however, on open source projects, these time frames are unrealistic. Six- even 12-month tracking allows the project leadership to focus on driving efforts at a high level by nurturing the community along.
|
||||
|
||||
The end result is a rubric that provides clear vision on where the project is going. It also lets community members more easily find ways to contribute. For example, your project may include someone who knows a few organizations using the project — this person could help introduce those developers to the codebase and guide them through their first commit.
|
||||
|
||||
### What happens if the project doesn’t hit the goals?
|
||||
|
||||
_“I have not failed. I’ve just found 10,000 ways that won’t work.”_ — Thomas A. Edison
|
||||
|
||||
Figuring out what is within the capability of an organization — whether Fortune 500 or a small open source project — is hard. And, sometimes the expectations or market conditions change along the way. Does that make the strategy planning process a failure? Absolutely not!
|
||||
|
||||
Instead, you can use this experience as a way to better understand your project’s velocity, its impact, and its community, and perhaps as a way to prioritize what is important and what’s not.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.linuxfoundation.org/blog/set-open-source-strategy/
|
||||
|
||||
作者:[ John Mertic][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:https://www.linuxfoundation.org/author/jmertic/
|
||||
[1]:https://en.wikipedia.org/wiki/Big_Hairy_Audacious_Goal
|
||||
[2]:https://www.linuxfoundation.org/author/jmertic/
|
||||
[3]:https://www.linuxfoundation.org/category/blog/
|
||||
[4]:https://www.linuxfoundation.org/category/audience/c-level/
|
||||
[5]:https://www.linuxfoundation.org/category/audience/developer-influencers/
|
||||
[6]:https://www.linuxfoundation.org/category/audience/entrepreneurs/
|
||||
[7]:https://www.linuxfoundation.org/category/campaigns/membership/how-to/
|
||||
[8]:https://www.linuxfoundation.org/category/campaigns/events-campaigns/linux-foundation/
|
||||
[9]:https://www.linuxfoundation.org/category/audience/open-source-developers/
|
||||
[10]:https://www.linuxfoundation.org/category/audience/open-source-professionals/
|
||||
[11]:https://www.linuxfoundation.org/category/audience/open-source-users/
|
||||
[12]:https://www.linuxfoundation.org/category/blog/thought-leadership/
|
@ -0,0 +1,130 @@
|
||||
### Unleash Your Creativity – Linux Programs for Drawing and Image Editing
|
||||
|
||||
By: [chabowski][1]
|
||||
|
||||
The following article is part of a series of articles that provide tips and tricks for Linux newbies – or Desktop users that are not yet experienced with regard to certain topics. This series intends to complement the special edition #30 “[Getting Started with Linux][2]” based on [openSUSE Leap][3], recently published by the [Linux Magazine,][4] with valuable additional information.
|
||||
|
||||

|
||||
|
||||
This article has been contributed by Douglas DeMaio, openSUSE PR Expert at SUSE.
|
||||
|
||||
Both Mac OS or Window offer several popular programs for graphics editing, vector drawing and creating and manipulating Portable Document Format (PDF). The good news: users familiar with the Adobe Suite can transition with ease to free, open-source programs available on Linux.
|
||||
|
||||
Programs like [GIMP][5], [InkScape][6] and [Okular][7] are cross platform programs that are available by default in Linux/GNU distributions and are persuasive alternatives to expensive Adobe programs like [Photoshop][8], [Illustrator][9] and [Acrobat][10].
|
||||
|
||||
These creativity programs on Linux distributions are just as powerful as those for macOS or Window. This article will explain some of the differences and how the programs can be used to make your transition to Linux comfortable.
|
||||
|
||||
### Krita
|
||||
|
||||
The KDE desktop environment comes with tons of cool applications. [Krita][11] is a professional open source painting program. It gives users the freedom to create any artistic image they desire. Krita features tools that are much more extensive than the tool sets of most proprietary programs you might be familiar with. From creating textures to comics, Krita is a must have application for Linux users.
|
||||
|
||||

|
||||
|
||||
### GIMP
|
||||
|
||||
GNU Image Manipulation Program (GIMP) is a cross-platform image editor. Users of Photoshop will find the User Interface of GIMP to be similar to that of Photoshop. The drop down menu offers colors, layers, filters and tools to help the user with editing graphics. Rulers are located both horizontal and vertical and guide can be dragged across the screen to give exact measurements. The drop down menu gives tool options for resizing or cropping photos; adjustments can be made to the color balance, color levels, brightness and contrast as well as hue and saturation.
|
||||
|
||||

|
||||
|
||||
There are multiple filters in GIMP to enhance or distort your images. Filters for artistic expression and animation are available and are more powerful tool options than those found in some proprietary applications. Gradients can be applied through additional layers and the Text Tool offers many fonts, which can be altered in shape and size through the Perspective Tool.
|
||||
|
||||
The cloning tool works exactly like those in other graphics editors, so manipulating images is simple and acurrate given the selection of brush sizes to do the job.
|
||||
|
||||
Perhaps one of the best options available with GIMP is that the images can be saved in a variety of formats like .jpg, .png, .pdf, .eps and .svg. These image options provide high-quality images in a small file.
|
||||
|
||||
### InkScape
|
||||
|
||||
Designing vector imagery with InkScape is simple and free. This cross platform allows for the creation of logos and illustrations that are highly scalable. Whether designing cartoons or creating images for branding, InkScape is a powerful application to get the job done. Like GIMP, InkScape lets you save files in various formats and allows for object manipulation like moving, rotating and skewing text and objects. Shape tools are available with InkScape so making stars, hexagons and other elements will meet the needs of your creative mind.
|
||||
|
||||

|
||||
|
||||
InkScape offers a comprehensive tool set, including a drawing tool, a pen tool and the freehand calligraphy tool that allows for object creation with your own personal style. The color selector gives you the choice of RGB, CMYK and RGBA – using specific colors for branding logos, icons and advertisement is definitely convincing.
|
||||
|
||||
Short cut commands are similar to what users experience in Adobe Illustrator. Making layers and grouping or ungrouping the design elements can turn a blank page into a full-fledged image that can be used for designing technical diagrams for presentations, importing images into a multimedia program or for creating web graphics and software design.
|
||||
|
||||
Inkscape can import vector graphics from multiple other programs. It can even import bitmap images. Inkscape is one of those cross platform, open-source programs that allow users to operate across different operating systems, no matter if they work with macOS, Windows or Linux.
|
||||
|
||||
### Okular and LibreOffice
|
||||
|
||||
LibreOffice, which is a free, open-source Office Suite, allows users to collaborate and interact with documents and important files on Linux, but also on macOS and Window. You can also create PDF files via LibreOffice, and LibreOffice Draw lets you view (and edit) PDF files as images.
|
||||
|
||||

|
||||
|
||||
However, the Portable Document Format (PDF) is quite different on the three Operating Systems. MacOS offers [Preview][12] by default; Windows has [Edge][13]. Of course, also Adobe Reader can be used for both MacOS and Window. With Linux, and especially the desktop selection of KDE, [Okular][14] is the default program for viewing PDF files.
|
||||
|
||||

|
||||
|
||||
The functionality of Okular supports different types of documents, like PDF, Postscript, [DjVu][15], [CHM][16], [XPS][17], [ePub][18] and others. Yet the universal document viewer also offers some powerful features that make interacting with a document different from other programs on MacOS and Windows. Okular gives selection and search tools that make accessing the text in PDFs fluid for how users interact with documents. Viewing documents with Okular is also accommodating with the magnification tool that allows for a quick look at small text in a document.
|
||||
|
||||
Okular also provides users with the option to configure it to use more memory if the document is too large and freezes the Operating System. This functionality is convenient for users accessing high-quality print documents for example for advertising.
|
||||
|
||||
For those who want to change locked images and documents, it’s rather easy to do so with LibreOffice Draw. A hypothetical situation would be to take a locked IRS (or tax) form and change it to make the uneditable document editable. Imagine how much fun it could be to transform it to some humorous kind of tax form …
|
||||
|
||||
And indeed, the sky’s the limit on how creative a user wants to be when using programs that are available on Linux distributions.
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
(
|
||||
|
||||
_**2** votes, average: **5.00** out of 5_
|
||||
|
||||
)
|
||||
|
||||
_You need to be a registered member to rate this post._
|
||||
|
||||
Tags: [drawing][19], [Getting Started with Linux][20], [GIMP][21], [image editing][22], [Images][23], [InkScape][24], [KDE][25], [Krita][26], [Leap 42.3][27], [LibreOffice][28], [Linux Magazine][29], [Okular][30], [openSUSE][31], [PDF][32] Categories: [Desktop][33], [Expert Views][34], [LibreOffice][35], [openSUSE][36]
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.suse.com/communities/blog/unleash-creativity-linux-programs-drawing-image-editing/
|
||||
|
||||
作者:[chabowski ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[1]:https://www.suse.com/communities/blog/author/chabowski/
|
||||
[2]:http://www.linux-magazine.com/Resources/Special-Editions/30-Getting-Started-with-Linux
|
||||
[3]:https://en.opensuse.org/Portal:42.3
|
||||
[4]:http://www.linux-magazine.com/
|
||||
[5]:https://www.gimp.org/
|
||||
[6]:https://inkscape.org/en/
|
||||
[7]:https://okular.kde.org/
|
||||
[8]:http://www.adobe.com/products/photoshop.html
|
||||
[9]:http://www.adobe.com/products/illustrator.html
|
||||
[10]:https://acrobat.adobe.com/us/en/acrobat/acrobat-pro-cc.html
|
||||
[11]:https://krita.org/en/
|
||||
[12]:https://en.wikipedia.org/wiki/Preview_(macOS)
|
||||
[13]:https://en.wikipedia.org/wiki/Microsoft_Edge
|
||||
[14]:https://okular.kde.org/
|
||||
[15]:http://djvu.org/
|
||||
[16]:https://fileinfo.com/extension/chm
|
||||
[17]:https://fileinfo.com/extension/xps
|
||||
[18]:http://idpf.org/epub
|
||||
[19]:https://www.suse.com/communities/blog/tag/drawing/
|
||||
[20]:https://www.suse.com/communities/blog/tag/getting-started-with-linux/
|
||||
[21]:https://www.suse.com/communities/blog/tag/gimp/
|
||||
[22]:https://www.suse.com/communities/blog/tag/image-editing/
|
||||
[23]:https://www.suse.com/communities/blog/tag/images/
|
||||
[24]:https://www.suse.com/communities/blog/tag/inkscape/
|
||||
[25]:https://www.suse.com/communities/blog/tag/kde/
|
||||
[26]:https://www.suse.com/communities/blog/tag/krita/
|
||||
[27]:https://www.suse.com/communities/blog/tag/leap-42-3/
|
||||
[28]:https://www.suse.com/communities/blog/tag/libreoffice/
|
||||
[29]:https://www.suse.com/communities/blog/tag/linux-magazine/
|
||||
[30]:https://www.suse.com/communities/blog/tag/okular/
|
||||
[31]:https://www.suse.com/communities/blog/tag/opensuse/
|
||||
[32]:https://www.suse.com/communities/blog/tag/pdf/
|
||||
[33]:https://www.suse.com/communities/blog/category/desktop/
|
||||
[34]:https://www.suse.com/communities/blog/category/expert-views/
|
||||
[35]:https://www.suse.com/communities/blog/category/libreoffice/
|
||||
[36]:https://www.suse.com/communities/blog/category/opensuse/
|
@ -1,60 +0,0 @@
|
||||
Translating by ValoniaKim
|
||||
Language engineering for great justice
|
||||
============================================================
|
||||
|
||||
Whole-systems engineering, when you get good at it, goes beyond being entirely or even mostly about technical optimizations. Every artifact we make is situated in a context of human action that widens out to the economics of its use, the sociology of its users, and the entirety of what Austrian economists call “praxeology”, the science of purposeful human behavior in its widest scope.
|
||||
|
||||
This isn’t just abstract theory for me. When I wrote my papers on open-source development, they were exactly praxeology – they weren’t about any specific software technology or objective but about the context of human action within which technology is worked. An increase in praxeological understanding of technology can reframe it, leading to tremendous increases in human productivity and satisfaction, not so much because of changes in our tools but because of changes in the way we grasp them.
|
||||
|
||||
In this, the third of my unplanned series of posts about the twilight of C and the huge changes coming as we actually begin to see forward into a new era of systems programming, I’m going to try to cash that general insight out into some more specific and generative ideas about the design of computer languages, why they succeed, and why they fail.
|
||||
|
||||
In my last post I noted that every computer language is an embodiment of a relative-value claim, an assertion about the optimal tradeoff between spending machine resources and spending programmer time, all of this in a context where the cost of computing power steadily falls over time while programmer-time costs remain relatively stable or may even rise. I also highlighted the additional role of transition costs in pinning old tradeoff assertions into place. I described what language designers do as seeking a new optimum for present and near-future conditions.
|
||||
|
||||
Now I’m going to focus on that last concept. A language designer has lots of possible moves in language-design space from where the state of the art is now. What kind of type system? GC or manual allocation? What mix of imperative, functional, or OO approaches? But in praxeological terms his choice is, I think, usually much simpler: attack a near problem or a far problem?
|
||||
|
||||
“Near” and “far” are measured along the curves of falling hardware costs, rising software complexity, and increasing transition costs from existing languages. A near problem is one the designer can see right in front of him; a far problem is a set of conditions that can be seen coming but won’t necessarily arrive for some time. A near solution can be deployed immediately, to great practical effect, but may age badly as conditions change. A far solution is a bold bet that may smother under the weight of its own overhead before its future arrives, or never be adopted at all because moving to it is too expensive.
|
||||
|
||||
Back at the dawn of computing, FORTRAN was a near-problem design, LISP a far-problem one. Assemblers are near solutions. Illustrating that the categories apply to non-general-purpose languages, also roff markup. Later in the game, PHP and Javascript. Far solutions? Oberon. Ocaml. ML. XML-Docbook. Academic languages tend to be far because the incentive structure around them rewards originality and intellectual boldness (note that this is a praxeological cause, not a technical one!). The failure mode of academic languages is predictable; high inward transition costs, nobody goes there, failure to achieve community critical mass sufficient for mainstream adoption, isolation, and stagnation. (That’s a potted history of LISP in one sentence, and I say that as an old LISP-head with a deep love for the language…)
|
||||
|
||||
The failure modes of near designs are uglier. The best outcome to hope for is a graceful death and transition to a newer design. If they hang on (most likely to happen when transition costs out are high) features often get piled on them to keep them relevant, increasing complexity until they become teetering piles of cruft. Yes, C++, I’m looking at you. You too, Javascript. And (alas) Perl, though Larry Wall’s good taste mitigated the problem for many years – but that same good taste eventually moved him to blow up the whole thing for Perl 6.
|
||||
|
||||
This way of thinking about language design encourages reframing the designer’s task in terms of two objectives. (1) Picking a sweet spot on the near-far axis away from you into the projected future; and (2) Minimizing inward transition costs from one or more existing languages so you co-opt their userbases. And now let’s talk about about how C took over the world.
|
||||
|
||||
There is no more more breathtaking example than C than of nailing the near-far sweet spot in the entire history of computing. All I need to do to prove this is point at its extreme longevity as a practical, mainstream language that successfully saw off many competitors for its roles over much of its range. That timespan has now passed about 35 years (counting from when it swamped its early competitors) and is not yet with certainty ended.
|
||||
|
||||
OK, you can attribute some of C’s persistence to inertia if you want, but what are you really adding to the explanation if you use the word “inertia”? What it means is exactly that nobody made an offer that actually covered the transition costs out of the language!
|
||||
|
||||
Conversely, an underappreciated strength of the language was the low inward transition costs. C is an almost uniquely protean tool that, even at the beginning of its long reign, could readily accommodate programming habits acquired from languages as diverse as FORTRAN, Pascal, assemblers and LISP. I noticed back in the 1980s that I could often spot a new C programmer’s last language by his coding style, which was just the flip side of saying that C was damn good at gathering all those tribes unto itself.
|
||||
|
||||
C++ also benefited from having low transition costs in. Later, most new languages at least partly copied C syntax in order to minimize them.Notice what this does to the context of future language designs: it raises the value of being a C-like as possible in order to minimize inward transition costs from anywhere.
|
||||
|
||||
Another way to minimize inward transition costs is to simply be ridiculously easy to learn, even to people with no prior programming experience. This, however, is remarkably hard to pull off. I evaluate that only one language – Python – has made the major leagues by relying on this quality. I mention it only in passing because it’s not a strategy I expect to see a _systems_ language execute successfully, though I’d be delighted to be wrong about that.
|
||||
|
||||
So here we are in late 2017, and…the next part is going to sound to some easily-annoyed people like Go advocacy, but it isn’t. Go, itself, could turn out to fail in several easily imaginable ways. It’s troubling that the Go team is so impervious to some changes their user community is near-unanimously and rightly (I think) insisting it needs. Worst-case GC latency, or the throughput sacrifices made to lower it, could still turn out to drastically narrow the language’s application range.
|
||||
|
||||
That said, there is a grand strategy expressed in the Go design that I think is right. To understand it, we need to review what the near problem for a C replacement is. As I noted in the prequels, it is rising defect rates as systems projects scale up – and specifically memory-management bugs because that category so dominates crash bugs and security exploits.
|
||||
|
||||
We’ve now identified two really powerful imperatives for a C replacement: (1) solve the memory-management problem, and (2) minimize inward-transition costs from C. And the history – the praxeological context – of programming languages tells us that if a C successor candidate don’t address the transition-cost problem effectively enough, it almost doesn’t matter how good a job it does on anything else. Conversely, a C successor that _does_ address transition costs well buys itself a lot of slack for not being perfect in other ways.
|
||||
|
||||
This is what Go does. It’s not a theoretical jewel; it has annoying limitations; GC latency presently limits how far down the stack it can be pushed. But what it is doing is replicating the Unix/C infective strategy of being easy-entry and _good enough_ to propagate faster than alternatives that, if it didn’t exist, would look like better far bets.
|
||||
|
||||
Of course, the proboscid in the room when I say that is Rust. Which is, in fact, positioning itself as the better far bet. I’ve explained in previous installments why I don’t think it’s really ready to compete yet. The TIOBE and PYPL indices agree; it’s never made the TIOBE top 20 and on both indices does quite poorly against Go.
|
||||
|
||||
Where Rust will be in five years is a different question, of course. My advice to the Rust community, if they care, is to pay some serious attention to the transition-cost problem. My personal experience says the C to Rust energy barrier is _[nasty][2]_ . Code-lifting tools like Corrode won’t solve it if all they do is map C to unsafe Rust, and if there were an easy way to automate ownership/lifetime annotations they wouldn’t be needed at all – the compiler would just do that for you. I don’t know what a solution would look like, here, but I think they better find one.
|
||||
|
||||
I will finally note that Ken Thompson has a history of designs that look like minimal solutions to near problems but turn out to have an amazing quality of openness to the future, the capability to _be improved_ . Unix is like this, of course. It makes me very cautious about supposing that any of the obvious annoyances in Go that look like future-blockers to me (like, say, the lack of generics) actually are. Because for that to be true, I’d have to be smarter than Ken, which is not an easy thing to believe.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: http://esr.ibiblio.org/?p=7745
|
||||
|
||||
作者:[Eric Raymond ][a]
|
||||
译者:[译者ID](https://github.com/译者ID)
|
||||
校对:[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]:http://esr.ibiblio.org/?author=2
|
||||
[1]:http://esr.ibiblio.org/?author=2
|
||||
[2]:http://esr.ibiblio.org/?p=7711&cpage=1#comment-1913931
|
||||
[3]:http://esr.ibiblio.org/?p=7745
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user