document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2024-12-29 21:41:00 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'LCTT/master'

Browse Source
This commit is contained in:
Xingyu Wang 2019-05-29 06:47:05 +08:00
commit 4056324ac2
29 changed files with 2877 additions and 818 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

64
translated/tech/20190415 Kubernetes on Fedora IoT with k3s.md → published/20190415 Kubernetes on Fedora IoT with k3s.md
View File

@ -1,37 +1,37 @@
[#]: collector: (lujun9972) [#]: collector: (lujun9972)
[#]: translator: (StdioA) [#]: translator: (StdioA)
[#]: reviewer: ( ) [#]: reviewer: (wxy)
[#]: publisher: ( ) [#]: publisher: (wxy)
[#]: url: ( ) [#]: url: (https://linux.cn/article-10908-1.html)
[#]: subject: (Kubernetes on Fedora IoT with k3s) [#]: subject: (Kubernetes on Fedora IoT with k3s)
[#]: via: (https://fedoramagazine.org/kubernetes-on-fedora-iot-with-k3s/) [#]: via: (https://fedoramagazine.org/kubernetes-on-fedora-iot-with-k3s/)
[#]: author: (Lennart Jern https://fedoramagazine.org/author/lennartj/) [#]: author: (Lennart Jern https://fedoramagazine.org/author/lennartj/)
使用 k3s 在 Fedora IoT 上运行 Kubernetes 使用 k3s 在 Fedora IoT 上运行 K8S
====== ======
![][1] ![](https://img.linux.net.cn/data/attachment/album/201905/28/094048yrzlik9oek5rbs5s.jpg)
Fedora IoT 是一个即将发布的、面相物联网的 Fedora 版本。去年 Fedora Magazine 中的《如何使用 Fedora IOT 点亮 LED》一文，第一次介绍了它。从那以后，它与 Fedora Silverblue 一起不断改进，以提供针对面相容器的工作流程的不可变基础操作系统。 Fedora IoT 是一个即将发布的、面向物联网的 Fedora 版本。去年 Fedora Magazine 的《[如何使用 Fedora IoT 点亮 LED 灯][2]》一文第一次介绍了它。从那以后，它与 Fedora Silverblue 一起不断改进，以提供针对面向容器的工作流的不可变基础操作系统。
Kubernetes 是一个颇受欢迎的容器编排系统。它可能最常用在那些能够处理巨大负载的强劲硬件上。不过，它也能在像树莓派 3 这样轻量级的设备上运行。我们继续阅读，来了解如何运行它。 Kubernetes 是一个颇受欢迎的容器编排系统。它可能最常用在那些能够处理巨大负载的强劲硬件上。不过，它也能在像树莓派 3 这样轻量级的设备上运行。让我们继续阅读，来了解如何运行它。
### 为什么用 Kubernetes？ ### 为什么用 Kubernetes？
虽然 Kubernetes 在云计算领域风靡一时，但让它在小型单板机上运行可能并不是显而易见的。不过，我们有非常明确的理由来做这件事。首先，这是一个不需要昂贵硬件就可以学习并熟悉 Kubernetes 的好方法；其次，由于它的流行性，市面上有[大量应用][2]进行了预先打包，以用于在 Kubernetes 集群中运行。更不用说，当你遇到问题时，会有大规模的社区用户为你提供帮助。 虽然 Kubernetes 在云计算领域风靡一时，但让它在小型单板机上运行可能并不是常见的。不过，我们有非常明确的理由来做这件事。首先，这是一个不需要昂贵硬件就可以学习并熟悉 Kubernetes 的好方法；其次，由于它的流行性，市面上有[大量应用][3]进行了预先打包，以用于在 Kubernetes 集群中运行。更不用说，当你遇到问题时，会有大规模的社区用户为你提供帮助。
最后但同样重要的是，即使是在家庭实验室这样的小规模环境中，容器编排也确实能事情变得更加简单。虽然在学习曲线方面，这一点并不明显，但这些技能在你将来与任何集群打交道的时候都会有帮助。不管你面对的是一个单节点树莓派集群，还是一个大规模的机器学习场，它们的操作方式都是类似的。 最后但同样重要的是，即使是在家庭实验室这样的小规模环境中，容器编排也确实能够使事情变得更加简单。虽然在学习曲线方面，这一点并不明显，但这些技能在你将来与任何集群打交道的时候都会有帮助。不管你面对的是一个单节点树莓派集群，还是一个大规模的机器学习场，它们的操作方式都是类似的。
#### K3s - 轻量级的 Kubernetes #### K3s - 轻量级的 Kubernetes
一个 Kubernetes 的“正常”安装（如果有这么一说的话）对于物联网来说有点沉重。K8s 的推荐内存配置，是每台机器 2GB！不过，我们也有一些替代品，其中一个新人是 [k3s][4]——一个轻量级的 Kubernetes 发行版。 一个“正常”安装的 Kubernetes（如果有这么一说的话）对于物联网来说有点沉重。K8s 的推荐内存配置，是每台机器 2GB！不过，我们也有一些替代品，其中一个新人是 [k3s][4] —— 一个轻量级的 Kubernetes 发行版。
K3s 非常特殊，因为它将 etcd 替换成了 SQLite 以满足键值存储需求。还有一点，在于整个 k3s 将使用一个二进制文件分发，而不是每个组件一个。这减少了内存占用并简化了安装过程。基于上述原因，我们只需要 512MB 内存即可运行 k3s，简直适合小型单板电脑！ K3s 非常特殊，因为它将 etcd 替换成了 SQLite 以满足键值存储需求。还有一点，在于整个 k3s 将使用一个二进制文件分发，而不是每个组件一个。这减少了内存占用并简化了安装过程。基于上述原因，我们只需要 512MB 内存即可运行 k3s，极度适合小型单板电脑！
### 你需要的东西 ### 你需要的东西
1. 在虚拟机或实体设备中运行的 Fedora IoT。在[这里][5]可以看到优秀的入门指南。一台机器就足够了，不过两台可以用来测试向集群添加更多节点。 1. Fedora IoT 运行在虚拟机或实体设备中运行的。在[这里][5]可以看到优秀的入门指南。一台机器就足够了，不过两台可以用来测试向集群添加更多节点。
2. [配置防火墙][6]，允许 6443 和 8372 端口的通信。或者，你也可以简单地运行“systemctl stop firewalld”来为这次实验关闭防火墙。 2. [配置防火墙][6]，允许 6443 和 8372 端口的通信。或者，你也可以简单地运行 `systemctl stop firewalld` 来为这次实验关闭防火墙。
### 安装 k3s ### 安装 k3s
@ -49,14 +49,14 @@ kubectl get nodes
需要注意的是，有几个选项可以通过环境变量传递给安装脚本。这些选项可以在[文档][7]中找到。当然，你也完全可以直接下载二进制文件来手动安装 k3s。 需要注意的是，有几个选项可以通过环境变量传递给安装脚本。这些选项可以在[文档][7]中找到。当然，你也完全可以直接下载二进制文件来手动安装 k3s。
对于实验和学习来说，这样已经很棒了，不过单节点的集群也不是一个集群。幸运的是，添加另一个节点并不比设置第一个节点要难。只需要向安装脚本传递两个环境变量，它就可以找到第一个节点，避免运行 k3s 的服务器部分。 对于实验和学习来说，这样已经很棒了，不过单节点的集群也不能算一个集群。幸运的是，添加另一个节点并不比设置第一个节点要难。只需要向安装脚本传递两个环境变量，它就可以找到第一个节点，而不用运行 k3s 的服务器部分。
``` ```
curl -sfL https://get.k3s.io | K3S_URL=https://example-url:6443 \ curl -sfL https://get.k3s.io | K3S_URL=https://example-url:6443 \
K3S_TOKEN=XXX sh - K3S_TOKEN=XXX sh -
``` ```
上面的 example-url 应被替换为第一个节点的 IP 地址，或一个经过完全限定的域名。在该节点中，（用 XXX 表示的）令牌可以在 /var/lib/rancher/k3s/server/node-token 文件中找到。 上面的 `example-url` 应被替换为第一个节点的 IP 地址，或一个完全限定域名。在该节点中，（用 XXX 表示的）令牌可以在 `/var/lib/rancher/k3s/server/node-token` 文件中找到。
### 部署一些容器 ### 部署一些容器
@ -66,19 +66,19 @@ curl -sfL https://get.k3s.io | K3S_URL=https://example-url:6443 \
kubectl create deployment my-server --image nginx kubectl create deployment my-server --image nginx
``` ```
这会从名为“nginx”的容器镜像中创建出一个名叫“my-server”的 [Deployment][8]（镜像名默认使用 docker hub 注册中心，以及 latest 标签）。 这会从名为 `nginx` 的容器镜像中创建出一个名叫 `my-server` 的 [部署][8]（默认使用 docker hub 注册中心，以及 `latest` 标签）。
``` ```
kubectl get pods kubectl get pods
``` ```
为了接触到 pod 中运行的 nginx 服务器，首先将 Deployment 通过一个 [Service][9] 来进行暴露。以下命令将创建一个与 Deployment 同名的 Service。 为了访问到 pod 中运行的 nginx 服务器，首先通过一个 [服务][9] 来暴露该部署。以下命令将创建一个与该部署同名的服务。
``` ```
kubectl expose deployment my-server --port 80 kubectl expose deployment my-server --port 80
``` ```
Service 将作为一种负载均衡器和 Pod 的 DNS 记录来工作。比如，当运行第二个 Pod 时，我们只需指定 _my-server_（Service 名称）就可以通过 _curl_ 访问 nginx 服务器。有关如何操作，可以看下面的实例。 服务将作为一种负载均衡器和 Pod 的 DNS 记录来工作。比如，当运行第二个 Pod 时，我们只需指定 `my-server`（服务名称）就可以通过 `curl` 访问 nginx 服务器。有关如何操作，可以看下面的实例。
``` ```
# 启动一个 pod，在里面以交互方式运行 bash # 启动一个 pod，在里面以交互方式运行 bash
@ -90,15 +90,15 @@ curl my-server
### Ingress 控制器及外部 IP ### Ingress 控制器及外部 IP
默认状态下，一个 Service 只能获得一个 ClusterIP（只能从集群内部访问），但你也可以通过把它的类型设置为 [LoadBalancer][10] 为服务申请一个外部 IP。不过，并非所有应用都需要自己的 IP 地址。相反，通常可以通过基于 Host 请求头部或请求路径进行路由，从而使多个服务共享一个 IP 地址。你可以在 Kubernetes 使用 [Ingress][11] 完成此操作，而这也是我们要做的。Ingress 也提供了额外的功能，比如无需配置应用，即可对流量进行 TLS 加密。 默认状态下，一个服务只能获得一个 ClusterIP（只能从集群内部访问），但你也可以通过把它的类型设置为 [LoadBalancer][10] 为该服务申请一个外部 IP。不过，并非所有应用都需要自己的 IP 地址。相反，通常可以通过基于 Host 请求头部或请求路径进行路由，从而使多个服务共享一个 IP 地址。你可以在 Kubernetes 使用 [Ingress][11] 完成此操作，而这也是我们要做的。Ingress 也提供了额外的功能，比如无需配置应用即可对流量进行 TLS 加密。
Kubernetes 需要入口控制器来使 Ingress 资源工作，k3s 包含 [Traefik][12] 正是出于此目的。它还包含了一个简单的服务负载均衡器，可以为集群中的服务提供外部 IP。这篇[文档][13]描述了这种服务： Kubernetes 需要 Ingress 控制器来使 Ingress 资源工作，k3s 包含 [Traefik][12] 正是出于此目的。它还包含了一个简单的服务负载均衡器，可以为集群中的服务提供外部 IP。这篇[文档][13]描述了这种服务：
> k3s 包含一个使用可用主机端口的基础服务负载均衡器。比如，如果你尝试创建一个监听 80 端口的负载均衡器，它会尝试在集群中寻找一个 80 端口空闲的节点。如果没有可用端口，那么负载均衡器将保持在 Pending 状态。 > k3s 包含一个使用可用主机端口的基础服务负载均衡器。比如，如果你尝试创建一个监听 80 端口的负载均衡器，它会尝试在集群中寻找一个 80 端口空闲的节点。如果没有可用端口，那么负载均衡器将保持在 Pending 状态。
> >
> k3s README > k3s README
入口控制器已经通过这个负载均衡器暴露在外。你可以使用以下命令找到它正在使用的 IP 地址。 Ingress 控制器已经通过这个负载均衡器暴露在外。你可以使用以下命令找到它正在使用的 IP 地址。
``` ```
$ kubectl get svc --all-namespaces $ kubectl get svc --all-namespaces
@ -109,13 +109,13 @@ NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
kube-system traefik LoadBalancer 10.43.145.104 10.0.0.8 80:31596/TCP,443:31539/TCP 33d kube-system traefik LoadBalancer 10.43.145.104 10.0.0.8 80:31596/TCP,443:31539/TCP 33d
``` ```
找到名为 traefik 的 Service。在上面的例子中，我们感兴趣的 IP 是 10.0.0.8。 找到名为 `traefik` 的服务。在上面的例子中，我们感兴趣的 IP 是 10.0.0.8。
### 路由传入的请求 ### 路由传入的请求
让我们创建一个 Ingress，使它通过基于 Host 头部的路由规则将请求路由至我们的服务器。这个例子中我们使用 [xip.io][14] 来避免必要的 DNS 记录配置工作。它的工作原理是将 IP 地址作为子域包含，以使用10.0.0.8.xip.io的任何子域来达到IP 10.0.0.8。换句话说，my-server.10.0.0.8.xip.io 被用于访问集群中的入口控制器。你现在就可以尝试（使用你自己的 IP，而不是 10.0.0.8）。如果没有入口，你应该会访问到“默认后端”，只是一个写着“404 page not found”的页面。 让我们创建一个 Ingress，使它通过基于 Host 头部的路由规则将请求路由至我们的服务器。这个例子中我们使用 [xip.io][14] 来避免必要的 DNS 记录配置工作。它的工作原理是将 IP 地址作为子域包含，以使用 `10.0.0.8.xip.io` 的任何子域来达到 IP `10.0.0.8`。换句话说，`my-server.10.0.0.8.xip.io` 被用于访问集群中的 Ingress 控制器。你现在就可以尝试（使用你自己的 IP，而不是 10.0.0.8）。如果没有 Ingress，你应该会访问到“默认后端”，只是一个写着“404 page not found”的页面。
我们可以使用以下 Ingress 让入口控制器将请求路由到我们的 Web 服务器 Service。 我们可以使用以下 Ingress 让 Ingress 控制器将请求路由到我们的 Web 服务器的服务。
``` ```
apiVersion: extensions/v1beta1 apiVersion: extensions/v1beta1
@ -133,17 +133,17 @@ spec:
servicePort: 80 servicePort: 80
``` ```
将以上片段保存到 _my-ingress.yaml_ 文件中，然后运行以下命令将其加入集群： 将以上片段保存到 `my-ingress.yaml` 文件中，然后运行以下命令将其加入集群：
``` ```
kubectl apply -f my-ingress.yaml kubectl apply -f my-ingress.yaml
``` ```
你现在应该能够在你选择的完全限定域名中访问到 nginx 的默认欢迎页面了。在我的例子中，它是 my-server.10.0.0.8.xip.io。入口控制器会通过 Ingress 中包含的信息来路由请求。对 my-server.10.0.0.8.xip.io 的请求将被路由到 Ingress 中定义为“后端”的 Service 和端口（在本例中为 my-server 和 80）。 你现在应该能够在你选择的完全限定域名中访问到 nginx 的默认欢迎页面了。在我的例子中，它是 `my-server.10.0.0.8.xip.io`。Ingress 控制器会通过 Ingress 中包含的信息来路由请求。对 `my-server.10.0.0.8.xip.io` 的请求将被路由到 Ingress 中定义为 `backend` 的服务和端口（在本例中为 `my-server` 和 `80`）。
### 那么，物联网呢？ ### 那么，物联网呢？
想象如下场景：你的家伙农场周围有很多的设备。它是一个具有各种硬件功能，传感器和执行器的物联网设备的异构集合。也许某些设备拥有摄像头，天气或光线传感器。其它设备可能会被连接起来，用来控制通风、灯光、百叶窗或闪烁的LED。 想象如下场景：你的家或农场周围有很多的设备。它是一个具有各种硬件功能、传感器和执行器的物联网设备的异构集合。也许某些设备拥有摄像头、天气或光线传感器。其它设备可能会被连接起来，用来控制通风、灯光、百叶窗或闪烁的 LED。
这种情况下，你想从所有传感器中收集数据，在最终使用它来制定决策和控制执行器之前，也可能会对其进行处理和分析。除此之外，你可能还想配置一个仪表盘来可视化那些正在发生的事情。那么 Kubernetes 如何帮助我们来管理这样的事情呢？我们怎么保证 Pod 在合适的设备上运行？ 这种情况下，你想从所有传感器中收集数据，在最终使用它来制定决策和控制执行器之前，也可能会对其进行处理和分析。除此之外，你可能还想配置一个仪表盘来可视化那些正在发生的事情。那么 Kubernetes 如何帮助我们来管理这样的事情呢？我们怎么保证 Pod 在合适的设备上运行？
@ -155,13 +155,13 @@ kubectl label nodes <node-name> <label-key>=<label-value>
kubectl label nodes node2 camera=available kubectl label nodes node2 camera=available
``` ```
一旦它们被打上标签，我们就可以轻松地使用 [nodeSelector][15] 为你的工作负载选择合适的节点。拼图的最后一块：如果你想在_所有_合适的节点上运行 Pod，那应该使用 [DaemonSet][16] 而不是 Deployment。换句话说，应为每个使用唯一传感器的数据收集应用程序创建一个 DaemonSet，并使用 nodeSelectors 确保它们仅在具有适当硬件的节点上运行。 一旦它们被打上标签，我们就可以轻松地使用 [nodeSelector][15] 为你的工作负载选择合适的节点。拼图的最后一块：如果你想在*所有*合适的节点上运行 Pod，那应该使用 [DaemonSet][16] 而不是部署。换句话说，应为每个使用唯一传感器的数据收集应用程序创建一个 DaemonSet，并使用 nodeSelector 确保它们仅在具有适当硬件的节点上运行。
服务发现功能允许 Pod 通过 Service 名称来寻找彼此，这项功能使得这类分布式系统的管理工作变得易如反掌。你不需要为应用配置 IP 地址或自定义端口，也不需要知道它们。相反，它们可以通过集群中的具名 Service 轻松找到彼此。 服务发现功能允许 Pod 通过服务名称来寻找彼此，这项功能使得这类分布式系统的管理工作变得易如反掌。你不需要为应用配置 IP 地址或自定义端口，也不需要知道它们。相反，它们可以通过集群中的命名服务轻松找到彼此。
#### 充分利用空闲资源 #### 充分利用空闲资源
随着集群的启动并运行，收集数据并控制灯光和气候可能使你觉得你已经把它完成了。不过，集群中还有大量的计算资源可以用于其它项目。这才是 Kubernetes 真正出彩的地方。 随着集群的启动并运行，收集数据并控制灯光和气候，可能使你觉得你已经把它完成了。不过，集群中还有大量的计算资源可以用于其它项目。这才是 Kubernetes 真正出彩的地方。
你不必担心这些资源的确切位置，或者去计算是否有足够的内存来容纳额外的应用程序。这正是编排系统所解决的问题！你可以轻松地在集群中部署更多的应用，让 Kubernetes 来找出适合运行它们的位置（或是否适合运行它们）。 你不必担心这些资源的确切位置，或者去计算是否有足够的内存来容纳额外的应用程序。这正是编排系统所解决的问题！你可以轻松地在集群中部署更多的应用，让 Kubernetes 来找出适合运行它们的位置（或是否适合运行它们）。
@ -182,14 +182,14 @@ via: https://fedoramagazine.org/kubernetes-on-fedora-iot-with-k3s/
作者：[Lennart Jern][a] 作者：[Lennart Jern][a]
选题：[lujun9972][b] 选题：[lujun9972][b]
译者：[StdioA](https://github.com/StdioA) 译者：[StdioA](https://github.com/StdioA)
校对：[校对者ID](https://github.com/校对者ID) 校对：[wxy](https://github.com/wxy)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://fedoramagazine.org/author/lennartj/ [a]: https://fedoramagazine.org/author/lennartj/
[b]: https://github.com/lujun9972 [b]: https://github.com/lujun9972
[1]: https://fedoramagazine.org/wp-content/uploads/2019/04/k3s-1-816x345.png [1]: https://fedoramagazine.org/wp-content/uploads/2019/04/k3s-1-816x345.png
[2]: https://fedoramagazine.org/turnon-led-fedora-iot/ [2]: https://linux.cn/article-10380-1.html
[3]: https://hub.helm.sh/ [3]: https://hub.helm.sh/
[4]: https://k3s.io [4]: https://k3s.io
[5]: https://docs.fedoraproject.org/en-US/iot/getting-started/ [5]: https://docs.fedoraproject.org/en-US/iot/getting-started/

78
sources/talk/20190522 Experts- Enterprise IoT enters the mass-adoption phase.md Normal file
View File

@ -0,0 +1,78 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Experts: Enterprise IoT enters the mass-adoption phase)
[#]: via: (https://www.networkworld.com/article/3397317/experts-enterprise-iot-enters-the-mass-adoption-phase.html)
[#]: author: (Jon Gold https://www.networkworld.com/author/Jon-Gold/)
Experts: Enterprise IoT enters the mass-adoption phase
======
Dropping hardware prices, 5G boost business internet-of-things deployments; technical complexity encourages partnerships.
![Avgust01 / Getty Images][1]
[IoT][2] in general has taken off quickly over the past few years, but experts at the recent IoT World highlighted that the enterprise part of the market has been particularly robust of late – it’s not just an explosion of connected home gadgets anymore.
Donna Moore, chairwoman of the LoRa Alliance, an industry group that works to develop and scale low-power WAN technology for mass usage, said on a panel that she’s never seen growth this fast in the sector. “I’d say we’re now in the early mass adopters [stage],” she said.
**More on IoT:**
* [Most powerful Internet of Things companies][3]
* [10 Hot IoT startups to watch][4]
* [The 6 ways to make money in IoT][5]
* [What is digital twin technology? [and why it matters]][6]
* [Blockchain, service-centric networking key to IoT success][7]
* [Getting grounded in IoT networking and security][8]
* [Building IoT-ready networks must become a priority][9]
* [What is the Industrial IoT? [And why the stakes are so high]][10]
The technology itself has pushed adoption to these heights, said Graham Trickey, head of IoT for the GSMA, a trade organization for mobile network operators. Along with price drops for wireless connectivity modules, the array of upcoming technologies nestling under the umbrella label of [5G][11] could simplify the process of connecting devices to [edge-computing][12] hardware – and the edge to the cloud or [data center][13].
“Mobile operators are not just providers of connectivity now, they’re farther up the stack,” he said. Technologies like narrow-band IoT and support for highly demanding applications like telehealth are all set to be part of the final 5G spec.
### Partnerships needed to deal with IoT complexity**
**
That’s not to imply that there aren’t still huge tasks facing both companies trying to implement their own IoT frameworks and the creators of the technology underpinning them. For one thing, IoT tech requires a huge array of different sets of specialized knowledge.
“That means partnerships, because you need an expert in your [vertical] area to know what you’re looking for, you need an expert in communications, and you might need a systems integrator,” said Trickey.
Phil Beecher, the president and CEO of the Wi-SUN Alliance (the acronym stands for Smart Ubiquitous Networks, and the group is heavily focused on IoT for the utility sector), concurred with that, arguing that broad ecosystems of different technologies and different partners would be needed. “There’s no one technology that’s going to solve all these problems, no matter how much some parties might push it,” he said.
One of the central problems – [IoT security][14] – is particularly dear to Beecher’s heart, given the consequences of successful hacks of the electrical grid or other utilities. More than one panelist praised the passage of the EU’s General Data Protection Regulation, saying that it offered concrete guidelines for entities developing IoT tech – a crucial consideration for some companies that may not have a lot of in-house expertise in that area.
Join the Network World communities on [Facebook][15] and [LinkedIn][16] to comment on topics that are top of mind.
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3397317/experts-enterprise-iot-enters-the-mass-adoption-phase.html
作者：[Jon Gold][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Jon-Gold/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2019/02/iot_internet_of_things_mobile_connections_by_avgust01_gettyimages-1055659210_2400x1600-100788447-large.jpg
[2]: https://www.networkworld.com/article/3207535/what-is-iot-how-the-internet-of-things-works.html
[3]: https://www.networkworld.com/article/2287045/internet-of-things/wireless-153629-10-most-powerful-internet-of-things-companies.html
[4]: https://www.networkworld.com/article/3270961/internet-of-things/10-hot-iot-startups-to-watch.html
[5]: https://www.networkworld.com/article/3279346/internet-of-things/the-6-ways-to-make-money-in-iot.html
[6]: https://www.networkworld.com/article/3280225/internet-of-things/what-is-digital-twin-technology-and-why-it-matters.html
[7]: https://www.networkworld.com/article/3276313/internet-of-things/blockchain-service-centric-networking-key-to-iot-success.html
[8]: https://www.networkworld.com/article/3269736/internet-of-things/getting-grounded-in-iot-networking-and-security.html
[9]: https://www.networkworld.com/article/3276304/internet-of-things/building-iot-ready-networks-must-become-a-priority.html
[10]: https://www.networkworld.com/article/3243928/internet-of-things/what-is-the-industrial-iot-and-why-the-stakes-are-so-high.html
[11]: https://www.networkworld.com/article/3203489/what-is-5g-how-is-it-better-than-4g.html
[12]: https://www.networkworld.com/article/3224893/what-is-edge-computing-and-how-it-s-changing-the-network.html?nsdr=true
[13]: https://www.networkworld.com/article/3223692/what-is-a-data-centerhow-its-changed-and-what-you-need-to-know.html
[14]: https://www.networkworld.com/article/3269736/getting-grounded-in-iot-networking-and-security.html
[15]: https://www.facebook.com/NetworkWorld/
[16]: https://www.linkedin.com/company/network-world

55
sources/talk/20190523 Benchmarks of forthcoming Epyc 2 processor leaked.md Normal file
View File

@ -0,0 +1,55 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Benchmarks of forthcoming Epyc 2 processor leaked)
[#]: via: (https://www.networkworld.com/article/3397081/benchmarks-of-forthcoming-epyc-2-processor-leaked.html)
[#]: author: (Andy Patrizio https://www.networkworld.com/author/Andy-Patrizio/)
Benchmarks of forthcoming Epyc 2 processor leaked
======
Benchmarks of AMD's second-generation Epyc server briefly found their way online and show the chip is larger but a little slower than the Epyc 7601 on the market now.
![Gordon Mah Ung][1]
Benchmarks of engineering samples of AMD's second-generation Epyc server, code-named “Rome,” briefly found their way online and show a very beefy chip running a little slower than its predecessor.
Rome is based on the Zen 2 architecture, believed to be more of an incremental improvement over the prior generation than a major leap. It’s already known that Rome would feature a 64-core, 128-thread design, but that was about all of the details.
**[ Also read:[Who's developing quantum computers][2] ]**
The details came courtesy of SiSoftware's Sandra PC analysis and benchmarking tool. It’s very popular and has been used by hobbyists and benchmarkers alike for more than 20 years. New benchmarks are uploaded to the Sandra database all the time, and what I suspect happened is someone running a Rome sample ran the benchmark, not realizing the results would be uploaded to the Sandra database.
The benchmarks were from two different servers, a Dell PowerEdge R7515 and a Super Micro Super Server. The Dell product number is not on the market, so this would indicate a future server with Rome processors. The entry has since been deleted, but several sites, including the hobbyist site Tom’s Hardware Guide, managed to [take a screenshot][3].
According to the entry, the chip is a mid-range processor with a base clock speed of 1.4GHz, jumping up to 2.2GHz in turbo mode, with 16MB of Level 2 cache and 256MB of Level 3 cache, the latter of which is crazy. The first-generation Epyc had just 32MB of L3 cache.
That’s a little slower than the Epyc 7601 on the market now, but when you double the number of cores in the same space, something’s gotta give, and in this case, it’s electricity. The thermal envelope was not revealed by the benchmark. Previous Epyc processors ranged from 120 watts to 180 watts.
Sandra ranked the processor at #3 for arithmetic and #5 for multimedia processing, which makes me wonder what on Earth beat the Rome chip. Interestingly, the servers were running Windows 10, not Windows Server 2019.
**[[Get certified as an Apple Technical Coordinator with this seven-part online course from PluralSight.][4] ]**
Rome is expected to be officially launched at the massive Computex trade show in Taiwan on May 27 and will begin shipping in the third quarter of the year.
Join the Network World communities on [Facebook][5] and [LinkedIn][6] to comment on topics that are top of mind.
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3397081/benchmarks-of-forthcoming-epyc-2-processor-leaked.html
作者：[Andy Patrizio][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Andy-Patrizio/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2018/11/rome_2-100779395-large.jpg
[2]: https://www.networkworld.com/article/3275385/who-s-developing-quantum-computers.html
[3]: https://www.tomshardware.co.uk/amd-epyc-rome-processor-data-center,news-60265.html
[4]: https://pluralsight.pxf.io/c/321564/424552/7490?u=https%3A%2F%2Fwww.pluralsight.com%2Fpaths%2Fapple-certified-technical-trainer-10-11
[5]: https://www.facebook.com/NetworkWorld/
[6]: https://www.linkedin.com/company/network-world

74
sources/talk/20190523 Cisco ties its security-SD-WAN gear with Teridion-s cloud WAN service.md Normal file
View File

@ -0,0 +1,74 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Cisco ties its security/SD-WAN gear with Teridion’s cloud WAN service)
[#]: via: (https://www.networkworld.com/article/3396628/cisco-ties-its-securitysd-wan-gear-with-teridions-cloud-wan-service.html)
[#]: author: (Michael Cooney https://www.networkworld.com/author/Michael-Cooney/)
Cisco ties its security/SD-WAN gear with Teridion’s cloud WAN service
======
An agreement links Cisco Meraki MX Security/SD-WAN appliances and its Auto VPN technology to Teridion’s cloud-based WAN service that claims to accelerate TCP-based applications by up to 5X.
![istock][1]
Cisco and Teridion have tied the knot to deliver faster enterprise [software-defined WAN][2] services.
The agreement links [Cisco Meraki][3] MX Security/SD-WAN appliances and its Auto [VPN][4] technology which lets users quickly bring up and configure secure sessions between branches and data centers with [Teridion’s cloud-based WAN service][5]. Teridion’s service promises customers better performance and control over traffic running from remote offices over the public internet to the [data center][6]. The service features what Teridion calls “Curated Routing” which fuses WAN acceleration techniques with route optimization to speed traffic.
**More about SD-WAN**
* [How to buy SD-WAN technology: Key questions to consider when selecting a supplier][7]
* [How to pick an off-site data-backup method][8]
* [SD-Branch: What it is and why you’ll need it][9]
* [What are the options for security SD-WAN?][10]
For example, Teridion says its WAN service can accelerate TCP-based applications like file transfers, backups and page loads, by as much as three to five times.
“[The service] improves network performance for UDP based applications like voice, video, RDP, and VDI. Enterprises can get carrier grade performance over broadband and dedicated internet access. Depending on the locations of the sites, [customers] can expect to see a 15 to 30 percent reduction in latency. That’s the difference between a great quality video conference and an unworkable, choppy mess” Teridion [stated][11].
Teridion says the Meraki integration creates an IPSec connection from the Cisco Meraki MX to the Teridion edge. “Customers create locations in the Teridion portal and apply the preconfigured Meraki template to them, or just upload a csv file if you have a lot of locations. Then, from each Meraki MX, create a 3rd party IPSec tunnel to the Teridion edge IP addresses that are generated as part of the Teridion configuration.”
The combined Cisco Meraki and Teridion offering brings SD-WAN and security capabilities at the WAN edge that are tightly integrated with a WAN service delivered over cost-effective broadband or dedicated Internet access, said Raviv Levi, director of product management at Cisco Meraki in a statement. “This brings better reliability and consistency to the enterprise WAN across multiple sites, as well as high performance access to all SaaS applications and cloud workloads.”
Meraki’s MX family supports everything from SD-WAN and [Wi-Fi][12] features to next-generation [firewall][13] and intrusion prevention in a single package.
Some studies show that by 2021 over 75 percent of enterprise traffic will be SaaS-oriented, so giving branch offices SD-WAN's reliable, secure transportation options will be a necessity, Cisco said when it [upgraded the Meraki][3] boxes last year.
Cisco Meraki isn’t the only SD-WAN service Teridion supports. The company also has agreements Citrix, Silver Peak, VMware (VeloCloud). Teridion also has partnerships with over 25 cloud partners, including Google, Amazon Web Services and Microsoft Azure.
[Teridion for Cisco Meraki][14] is available now from authorized Teridion resellers. Pricing starts at $50 per site per month.
Join the Network World communities on [Facebook][15] and [LinkedIn][16] to comment on topics that are top of mind.
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3396628/cisco-ties-its-securitysd-wan-gear-with-teridions-cloud-wan-service.html
作者：[Michael Cooney][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Michael-Cooney/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2018/02/istock-820219662-100749695-large.jpg
[2]: https://www.networkworld.com/article/3031279/sd-wan-what-it-is-and-why-you-ll-use-it-one-day.html
[3]: https://www.networkworld.com/article/3301169/cisco-meraki-amps-up-throughput-wi-fi-to-sd-wan-family.html
[4]: https://www.networkworld.com/article/3138952/5-things-you-need-to-know-about-virtual-private-networks.html
[5]: https://www.networkworld.com/article/3284285/teridion-enables-higher-performing-and-more-responsive-saas-applications.html
[6]: https://www.networkworld.com/article/3223692/what-is-a-data-centerhow-its-changed-and-what-you-need-to-know.html
[7]: https://www.networkworld.com/article/3323407/sd-wan/how-to-buy-sd-wan-technology-key-questions-to-consider-when-selecting-a-supplier.html
[8]: https://www.networkworld.com/article/3328488/backup-systems-and-services/how-to-pick-an-off-site-data-backup-method.html
[9]: https://www.networkworld.com/article/3250664/lan-wan/sd-branch-what-it-is-and-why-youll-need-it.html
[10]: https://www.networkworld.com/article/3285728/sd-wan/what-are-the-options-for-securing-sd-wan.html?nsdr=true
[11]: https://www.teridion.com/blog/teridion-announces-deep-integration-with-cisco-meraki-mx/
[12]: https://www.networkworld.com/article/3318119/what-to-expect-from-wi-fi-6-in-2019.html
[13]: https://www.networkworld.com/article/3230457/what-is-a-firewall-perimeter-stateful-inspection-next-generation.html
[14]: https://www.teridion.com/meraki
[15]: https://www.facebook.com/NetworkWorld/
[16]: https://www.linkedin.com/company/network-world

74
sources/talk/20190523 Edge-based caching and blockchain-nodes speed up data transmission.md Normal file
View File

@ -0,0 +1,74 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Edge-based caching and blockchain-nodes speed up data transmission)
[#]: via: (https://www.networkworld.com/article/3397105/edge-based-caching-and-blockchain-nodes-speed-up-data-transmission.html)
[#]: author: (Patrick Nelson https://www.networkworld.com/author/Patrick-Nelson/)
Edge-based caching and blockchain-nodes speed up data transmission
======
Using a combination of edge-based data caches and blockchain-like distributed networks, Bluzelle claims it can significantly speed up the delivery of data across the globe.
![OlgaSalt / /getty][1]
The combination of a blockchain-like distributed network, along with the ability to locate data at the edge will massively speed up future networks, such as those used by the internet of things (IoT), claims Bluzelle in announcing what is says is the first decentralized data delivery network (DDN).
Distributed DDNs will be like content delivery networks (CDNs) that now cache content around the world to speed up the web, but in this case, it will be for data, the Singapore-based company explains. Distributed key-value (blockchain) networks and edge computing built into Bluzelle's system will provide significantly faster delivery than existing caching, the company claims in a press release announcing its product.
“The future of data delivery can only ever be de-centrally distributed,” says Pavel Bains, CEO and co-founder of Bluzelle. It’s because the world requires instant access to data that’s being created at the edge, he argues.
“But delivery is hampered by existing technology,” he says.
**[ Also read:[What is edge computing?][2] and [How edge networking and IoT will reshape data centers][3]. ]**
Bluzelle says decentralized caching is the logical next step to generalized data caching, used for reducing latency. “Decentralized caching expands the theory of caching,” the company writes in a [report][4] (Dropbox pdf) on its [website][5]. It says the cache must be expanded from simply being located at one unique location.
“Using a combination of distributed networks, the edge and the cloud, [it’s] thereby increasing the transactional throughput of data,” the company says.
This kind of thing is particularly important in consumer gaming now, where split-second responses from players around the world make or break a game experience, but it will likely be crucial for the IoT, higher-definition media, artificial intelligence, and virtual reality as they gain more of a role in digitization—including at critical enterprise applications.
“Currently applications are limited to data caching technologies that require complex configuration and management of 10-plus-year-old technology constrained to a few data centers,” Bains says. “These were not designed to handle the ever-increasing volumes of data.”
Bains says one of the key selling points of Bluzelle's network is that developers should be able to implement and run networks without having to also physically expand the networks manually.
“Software developers don’t want to react to where their customers come from. Our architecture is designed to always have the data right where the customer is. This provides a superior consumer experience,” he says.
Data caches are around now, but Bluzelle claims its system, written in C++ and available on Linux and Docker containers, among other platforms, is faster than others. It further says that if its system and a more traditional cache would connect to the same MySQL database in Virginia, say, their users will get the data three to 16 times faster than a traditional “non-edge-caching” network. Write updates to all Bluzelle nodes around the world takes 875 milliseconds (ms), it says.
The company has been concentrating its efforts on gaming, and with a test setup in Virginia, it says it was able to deliver data 33 times faster—at 22ms to Singapore—than a normal, cloud-based data cache. That traditional cache (located near the database) took 727ms in the Bluzelle-published test. In a test to Ireland, it claims 16ms over 223ms using a traditional cache.
An algorithm is partly the reason for the gains, the company explains. It “allows the nodes to make decisions and take actions without the need for masternodes,” the company says. Masternodes are the server-like parts of blockchain systems.
**More about edge networking**
* [How edge networking and IoT will reshape data centers][3]
* [Edge computing best practices][6]
* [How edge computing can help secure the IoT][7]
Join the Network World communities on [Facebook][8] and [LinkedIn][9] to comment on topics that are top of mind.
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3397105/edge-based-caching-and-blockchain-nodes-speed-up-data-transmission.html
作者：[Patrick Nelson][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Patrick-Nelson/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2019/02/blockchain_crypotocurrency_bitcoin-by-olgasalt-getty-100787949-large.jpg
[2]: https://www.networkworld.com/article/3224893/internet-of-things/what-is-edge-computing-and-how-it-s-changing-the-network.html
[3]: https://www.networkworld.com/article/3291790/data-center/how-edge-networking-and-iot-will-reshape-data-centers.html
[4]: https://www.dropbox.com/sh/go5bnhdproy1sk5/AAC5MDoafopFS7lXUnmiLAEFa?dl=0&preview=Bluzelle+Report+-+The+Decentralized+Internet+Is+Here.pdf
[5]: https://bluzelle.com/
[6]: https://www.networkworld.com/article/3331978/lan-wan/edge-computing-best-practices.html
[7]: https://www.networkworld.com/article/3331905/internet-of-things/how-edge-computing-can-help-secure-the-iot.html
[8]: https://www.facebook.com/NetworkWorld/
[9]: https://www.linkedin.com/company/network-world

80
sources/talk/20190523 Online performance benchmarks all companies should try to achieve.md Normal file
View File

@ -0,0 +1,80 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Online performance benchmarks all companies should try to achieve)
[#]: via: (https://www.networkworld.com/article/3397322/online-performance-benchmarks-all-companies-should-try-to-achieve.html)
[#]: author: (Zeus Kerravala https://www.networkworld.com/author/Zeus-Kerravala/)
Online performance benchmarks all companies should try to achieve
======
With digital performance more important than ever, companies must ensure their online performance meets customers’ needs. A new ThousandEyes report can help them determine that.
![Thinkstock][1]
There's no doubt about it: We have entered the experience economy, and digital performance is more important than ever.
Customer experience is the top brand differentiator, topping price and every other factor. And businesses that provide a poor digital experience will find customers will actively seek a competitor. In fact, recent ZK Research found that in 2018, about two-thirds of millennials changed loyalties to a brand because of a bad experience. (Note: I am an employee of ZK Research.)
To help companies determine if their online performance is leading, lacking, or on par with some of the top companies, ThousandEyes this week released its [2019 Digital Experience Performance Benchmark Report][2]. This document provides a comparative analysis of web, infrastructure, and network performance from the top 20 U.S. digital retail, travel, and media websites. Although this is a small sampling of companies, those three industries are the most competitive when it comes to using their digital platforms for competitive advantage. The aggregated data from this report can be used as an industry-agnostic performance benchmark that all companies should strive to meet.
**[ Read also:[IoT providers need to take responsibility for performance][3] ]**
The methodology of the study was for ThousandEyes to use its own platform to provide an independent view of performance. It uses active monitoring and a global network of monitoring agents to measure application and network layer performance for websites, applications, and services. The company collected data from 36 major cities scattered across the U.S. Six of the locations (Ashburn, Chicago, Dallas, Los Angeles, San Jose, and Seattle) also included vantage points connected to six major broadband ISPs (AT&T, CenturyLink, Charter, Comcast, Cox, and Verizon). This acts as a good proxy for what a user would experience.
The test involved page load tests against the websites of the major companies in retail, media, and travel and looked at several factors, including DNS response time, round-trip latency, network time (one-way latency), HTTP response time, and page load. The averages and median times can be seen in the table below. Those can be considered the average benchmarks that all companies should try to attain.
![][4]
### Choice of content delivery network matters by location
ThousandEyes' report also looked at how the various services that companies use impacts web performance. For example, the study measured the performance of the content delivery network (CDN) providers in the 36 markets. It found that in Albuquerque, Akamai and Fastly had the most latency, whereas Edgecast had the least. It also found that in Boston, all of the CDN providers were close. Companies can use this type of data to help them select a CDN. Without it, decision makers are essentially guessing and hoping.
### CDN performance is impacted by ISP
Another useful set of data was cross-referencing CDN performance by ISP, which lead to some fascinating information. With Comcast, Akamai, Cloudfront, Google and Incapula all had high amounts of latency. Only Edgecast and Fastly offered average latency. On the other hand, all of the CDNs worked great with CenturyLink. This tells a buyer, "If my customer base is largely in Comcast’s footprint, I should look at Edgecast or Fastly or my customers will be impacted."
### DNS and latency directly impact page load times
The ThousandEyes study also confirmed some points that many people believe as true but until now had no quantifiable evidence to support it. For example, it's widely accepted that DNS response time and network latency to the CDN edge correlate to web performance; the data in the report now supports that belief. ThousandEyes did some regression analysis and fancy math and found that in general, companies that were in the top quartile of HTTP performance had above-average DNS response time and network performance. There were a few exceptions, but in most cases, this is true.
Based on all the data, the below are the benchmarks for the three infrastructure metrics gathered and is what businesses, even ones outside the three verticals studied, should hope to achieve to support a high-quality digital experience.
* DNS response time 25 ms
* Round trip network latency 15 ms
* HTTP response time 250 ms
### Operations teams need to focus on digital performance
Benchmarking certainly provides value, but the report also offers some recommendations on how operations teams can use the data to improve digital performance. Those include:
* **Measure site from distributed user vantage points**. There is no single point that will provide a view of digital performance everywhere. Instead, measure from a range of ISPs in different regions and take a multi-layered approach to visibility (application, network and routing).
* **Use internet performance information as a baseline**. Compare your organization's data to the baselines, and if you’re not meeting it in some markets, focus on improvement there.
* **Compare performance to industry peers**. In highly competitive industries, it’s important to understand how you rank versus the competition. Don’t be satisfied with hitting the benchmarks if your key competitors exceed them.
* **Build a strong performance stack.** The data shows that solid DNS and HTTP response times and low latency are correlated to solid page load times. Focus on optimizing those factors and consider them foundational to digital performance.
Join the Network World communities on [Facebook][5] and [LinkedIn][6] to comment on topics that are top of mind.
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3397322/online-performance-benchmarks-all-companies-should-try-to-achieve.html
作者：[Zeus Kerravala][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Zeus-Kerravala/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2017/07/racing_speed_runners_internet-speed-100728363-large.jpg
[2]: https://www.thousandeyes.com/research/digital-experience
[3]: https://www.networkworld.com/article/3340318/iot-providers-need-to-take-responsibility-for-performance.html
[4]: https://images.idgesg.net/images/article/2019/05/thousandeyes-100797290-large.jpg
[5]: https://www.facebook.com/NetworkWorld/
[6]: https://www.linkedin.com/company/network-world

93
sources/talk/20190523 Study- Most enterprise IoT transactions are unencrypted.md Normal file
View File

@ -0,0 +1,93 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Study: Most enterprise IoT transactions are unencrypted)
[#]: via: (https://www.networkworld.com/article/3396647/study-most-enterprise-iot-transactions-are-unencrypted.html)
[#]: author: (Tim Greene https://www.networkworld.com/author/Tim-Greene/)
Study: Most enterprise IoT transactions are unencrypted
======
A Zscaler report finds 91.5% of IoT communications within enterprises are in plaintext and so susceptible to interference.
![HYWARDS / Getty Images][1]
Of the millions of enterprise-[IoT][2] transactions examined in a recent study, the vast majority were sent without benefit of encryption, leaving the data vulnerable to theft and tampering.
The research by cloud-based security provider Zscaler found that about 91.5 percent of transactions by internet of things devices took place over plaintext, while 8.5 percent were encrypted with [SSL][3]. That means if attackers could intercept the unencrypted traffic, they’d be able to read it and possibly alter it, then deliver it as if it had not been changed.
**[ For more on IoT security, see[our corporate guide to addressing IoT security concerns][4]. | Get regularly scheduled insights by [signing up for Network World newsletters][5]. ]**
Researchers looked through one month’s worth of enterprise traffic traversing Zscaler’s cloud seeking the digital footprints of IoT devices. It found and analyzed 56 million IoT-device transactions over that time, and identified the type of devices, protocols they used, the servers they communicated with, how often communication went in and out and general IoT traffic patterns.
The team tried to find out which devices generate the most traffic and the threats they face. It discovered that 1,015 organizations had at least one IoT device. The most common devices were set-top boxes (52 percent), then smart TVs (17 percent), wearables (8 percent), data-collection terminals (8 percent), printers (7 percent), IP cameras and phones (5 percent) and medical devices (1 percent).
While they represented only 8 percent of the devices, data-collection terminals generated 80 percent of the traffic.
The breakdown is that 18 percent of the IoT devices use SSL to communicate all the time, and of the remaining 82 percent, half used it part of the time and half never used it.
The study also found cases of plaintext HTTP being used to authenticate devices and to update software and firmware, as well as use of outdated crypto libraries and weak default credentials.
While IoT devices are common in enterprises, “many of the devices are employee owned, and this is just one of the reasons they are a security concern,” the report says. Without strict policies and enforcement, these devices represent potential vulnerabilities.
**[[Prepare to become a Certified Information Security Systems Professional with this comprehensive online course from PluralSight. Now offering a 10-day free trial!][6] ]**
Another reason employee-owned IoT devices are a concern is that many businesses don’t consider them a threat because no data is stored on them. But if the data they gather is transmitted insecurely, it is at risk.
### 5 tips to protect enterprise IoT
Zscaler recommends these security precautions:
* Change default credentials to something more secure. As employees bring in devices, encourage them to use strong passwords and to keep their firmware current.
* Isolate IoT devices on networks and restrict inbound and outbound network traffic.
* Restrict access to IoT devices from external networks and block unnecessary ports from external access.
* Apply regular security and firmware updates to IoT devices, and secure network traffic.
* Deploy tools to gain visibility of shadow-IoT devices already inside the network so they can be protected.
**More on IoT:**
* [What is edge computing and how it’s changing the network][7]
* [Most powerful Internet of Things companies][8]
* [10 Hot IoT startups to watch][9]
* [The 6 ways to make money in IoT][10]
* [What is digital twin technology? [and why it matters]][11]
* [Blockchain, service-centric networking key to IoT success][12]
* [Getting grounded in IoT networking and security][13]
* [Building IoT-ready networks must become a priority][14]
* [What is the Industrial IoT? [And why the stakes are so high]][15]
Join the Network World communities on [Facebook][16] and [LinkedIn][17] to comment on topics that are top of mind.
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3396647/study-most-enterprise-iot-transactions-are-unencrypted.html
作者：[Tim Greene][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Tim-Greene/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2019/05/network_security_network_traffic_scanning_by_hywards_gettyimages-673891964_2400x1600-100796830-large.jpg
[2]: https://www.networkworld.com/article/3207535/what-is-iot-how-the-internet-of-things-works.html
[3]: https://www.networkworld.com/article/3045953/5-things-you-need-to-know-about-ssl.html
[4]: https://www.networkworld.com/article/3269165/internet-of-things/a-corporate-guide-to-addressing-iot-security-concerns.html
[5]: https://www.networkworld.com/newsletters/signup.html
[6]: https://pluralsight.pxf.io/c/321564/424552/7490?u=https%3A%2F%2Fwww.pluralsight.com%2Fpaths%2Fcertified-information-systems-security-professional-cisspr
[7]: https://www.networkworld.com/article/3224893/internet-of-things/what-is-edge-computing-and-how-it-s-changing-the-network.html
[8]: https://www.networkworld.com/article/2287045/internet-of-things/wireless-153629-10-most-powerful-internet-of-things-companies.html
[9]: https://www.networkworld.com/article/3270961/internet-of-things/10-hot-iot-startups-to-watch.html
[10]: https://www.networkworld.com/article/3279346/internet-of-things/the-6-ways-to-make-money-in-iot.html
[11]: https://www.networkworld.com/article/3280225/internet-of-things/what-is-digital-twin-technology-and-why-it-matters.html
[12]: https://www.networkworld.com/article/3276313/internet-of-things/blockchain-service-centric-networking-key-to-iot-success.html
[13]: https://www.networkworld.com/article/3269736/internet-of-things/getting-grounded-in-iot-networking-and-security.html
[14]: https://www.networkworld.com/article/3276304/internet-of-things/building-iot-ready-networks-must-become-a-priority.html
[15]: https://www.networkworld.com/article/3243928/internet-of-things/what-is-the-industrial-iot-and-why-the-stakes-are-so-high.html
[16]: https://www.facebook.com/NetworkWorld/
[17]: https://www.linkedin.com/company/network-world

439
sources/tech/20170410 Writing a Time Series Database from Scratch.md
View File

@ -1,439 +0,0 @@
LuMing Translating
Writing a Time Series Database from Scratch
============================================================
I work on monitoring. In particular on [Prometheus][2], a monitoring system that includes a custom time series database, and its integration with [Kubernetes][3].
In many ways Kubernetes represents all the things Prometheus was designed for. It makes continuous deployments, auto scaling, and other features of highly dynamic environments easily accessible. The query language and operational model, among many other conceptual decisions make Prometheus particularly well-suited for such environments. Yet, if monitored workloads become significantly more dynamic, this also puts new strains on monitoring system itself. With this in mind, rather than doubling back on problems Prometheus already solves well, we specifically aim to increase its performance in environments with highly dynamic, or transient services.
Prometheus's storage layer has historically shown outstanding performance, where a single server is able to ingest up to one million samples per second as several million time series, all while occupying a surprisingly small amount of disk space. While the current storage has served us well, I propose a newly designed storage subsystem that corrects for shortcomings of the existing solution and is equipped to handle the next order of scale.
> Note: I've no background in databases. What I say might be wrong and mislead. You can channel your criticism towards me (fabxc) in #prometheus on Freenode.
### Problems, Problems, Problem Space
First, a quick outline of what we are trying to accomplish and what key problems it raises. For each, we take a look at Prometheus' current approach, what it does well, and which problems we aim to address with the new design.
### Time series data
We have a system that collects data points over time.
```
identifier -> (t0, v0), (t1, v1), (t2, v2), (t3, v3), ....
```
Each data point is a tuple of a timestamp and a value. For the purpose of monitoring, the timestamp is an integer and the value any number. A 64 bit float turns out to be a good representation for counter as well as gauge values, so we go with that. A sequence of data points with strictly monotonically increasing timestamps is a series, which is addressed by an identifier. Our identifier is a metric name with a dictionary of  _label dimensions_ . Label dimensions partition the measurement space of a single metric. Each metric name plus a unique set of labels is its own  _time series_  that has a value stream associated with it.
This is a typical set of series identifiers that are part of metric counting requests:
```
requests_total{path="/status", method="GET", instance=”10.0.0.1:80”}
requests_total{path="/status", method="POST", instance=”10.0.0.3:80”}
requests_total{path="/", method="GET", instance=”10.0.0.2:80”}
```
Let's simplify this representation right away: A metric name can be treated as just another label dimension — `__name__` in our case. At the query level, it might be be treated specially but that doesn't concern our way of storing it, as we will see later.
```
{__name__="requests_total", path="/status", method="GET", instance=”10.0.0.1:80”}
{__name__="requests_total", path="/status", method="POST", instance=”10.0.0.3:80”}
{__name__="requests_total", path="/", method="GET", instance=”10.0.0.2:80”}
```
When querying time series data, we want to do so by selecting series by their labels. In the simplest case `{__name__="requests_total"}` selects all series belonging to the `requests_total` metric. For all selected series, we retrieve data points within a specified time window.
In more complex queries, we may wish to select series satisfying several label selectors at once and also represent more complex conditions than equality. For example, negative (`method!="GET"`) or regular expression matching (`method=~"PUT|POST"`).
This largely defines the stored data and how it is recalled.
### Vertical and Horizontal
In a simplified view, all data points can be laid out on a two-dimensional plane. The  _horizontal_  dimension represents the time and the series identifier space spreads across the  _vertical_  dimension.
```
series
^
│ . . . . . . . . . . . . . . . . . . . . . . {__name__="request_total", method="GET"}
│ . . . . . . . . . . . . . . . . . . . . . . {__name__="request_total", method="POST"}
│ . . . . . . .
│ . . . . . . . . . . . . . . . . . . . ...
│ . . . . . . . . . . . . . . . . . . . . .
│ . . . . . . . . . . . . . . . . . . . . . {__name__="errors_total", method="POST"}
│ . . . . . . . . . . . . . . . . . {__name__="errors_total", method="GET"}
│ . . . . . . . . . . . . . .
│ . . . . . . . . . . . . . . . . . . . ...
│ . . . . . . . . . . . . . . . . . . . .
v
<-------------------- time --------------------->
```
Prometheus retrieves data points by periodically scraping the current values for a set of time series. The entity from which we retrieve such a batch is called a  _target_ . Thereby, the write pattern is completely vertical and highly concurrent as samples from each target are ingested independently.
To provide some measurement of scale: A single Prometheus instance collects data points from tens of thousands of  _targets_ , which expose hundreds to thousands of different time series each.
At the scale of collecting millions of data points per second, batching writes is a non-negotiable performance requirement. Writing single data points scattered across our disk would be painfully slow. Thus, we want to write larger chunks of data in sequence.
This is an unsurprising fact for spinning disks, as their head would have to physically move to different sections all the time. While SSDs are known for fast random writes, they actually can't modify individual bytes but only write in  _pages_  of 4KiB or more. This means writing a 16 byte sample is equivalent to writing a full 4KiB page. This behavior is part of what is known as [ _write amplification_ ][4], which as a bonus causes your SSD to wear out – so it wouldn't just be slow, but literally destroy your hardware within a few days or weeks.
For more in-depth information on the problem, the blog series ["Coding for SSDs" series][5] is a an excellent resource. Let's just consider the main take away: sequential and batched writes are the ideal write pattern for spinning disks and SSDs alike. A simple rule to stick to.
The querying pattern is significantly more differentiated than the write the pattern. We can query a single datapoint for a single series, a single datapoint for 10000 series, weeks of data points for a single series, weeks of data points for 10000 series, etc. So on our two-dimensional plane, queries are neither fully vertical or horizontal, but a rectangular combination of the two.
[Recording rules][6] mitigate the problem for known queries but are not a general solution for ad-hoc queries, which still have to perform reasonably well.
We know that we want to write in batches, but the only batches we get are vertical sets of data points across series. When querying data points for a series over a time window, not only would it be hard to figure out where the individual points can be found, we'd also have to read from a lot of random places on disk. With possibly millions of touched samples per query, this is slow even on the fastest SSDs. Reads will also retrieve more data from our disk than the requested 16 byte sample. SSDs will load a full page, HDDs will at least read an entire sector. Either way, we are wasting precious read throughput.
So ideally, samples for the same series would be stored sequentially so we can just scan through them with as few reads as possible. On top, we only need to know where this sequence starts to access all data points.
There's obviously a strong tension between the ideal pattern for writing collected data to disk and the layout that would be significantly more efficient for serving queries. It is  _the_  fundamental problem our TSDB has to solve.
#### Current solution
Time to take a look at how Prometheus's current storage, let's call it "V2", addresses this problem.
We create one file per time series that contains all of its samples in sequential order. As appending single samples to all those files every few seconds is expensive, we batch up 1KiB chunks of samples for a series in memory and append those chunks to the individual files, once they are full. This approach solves a large part of the problem. Writes are now batched, samples are stored sequentially. It also enables incredibly efficient compression formats, based on the property that a given sample changes only very little with respect to the previous sample in the same series. Facebook's paper on their Gorilla TSDB describes a similar chunk-based approach and [introduces a compression format][7] that reduces 16 byte samples to an average of 1.37 bytes. The V2 storage uses various compression formats including a variation of Gorilla’s.
```
┌──────────┬─────────┬─────────┬─────────┬─────────┐ series A
└──────────┴─────────┴─────────┴─────────┴─────────┘
┌──────────┬─────────┬─────────┬─────────┬─────────┐ series B
└──────────┴─────────┴─────────┴─────────┴─────────┘
. . .
┌──────────┬─────────┬─────────┬─────────┬─────────┬─────────┐ series XYZ
└──────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
chunk 1 chunk 2 chunk 3 ...
```
While the chunk-based approach is great, keeping a separate file for each series is troubling the V2 storage for various reasons:
* We actually need a lot more files than the number of time series we are currently collecting data for. More on that in the section on "Series Churn". With several million files, sooner or later way may run out of [inodes][1] on our filesystem. This is a condition we can only recover from by reformatting our disks, which is as invasive and disruptive as it could be. We generally want to avoid formatting disks specifically to fit a single application.
* Even when chunked, several thousands of chunks per second are completed and ready to be persisted. This still requires thousands of individual disk writes every second. While it is alleviated by also batching up several completed chunks for a series, this in return increases the total memory footprint of data which is waiting to be persisted.
* It's infeasible to keep all files open for reads and writes. In particular because ~99% of data is never queried again after 24 hours. If it is queried though though, we have to open up to thousands of files, find and read relevant data points into memory, and close them again. As this would result in high query latencies, data chunks are cached rather aggressively leading to problems outlined further in the section on "Resource Consumption".
* Eventually, old data has to be deleted and data needs to be removed from the front of millions of files. This means that deletions are actually write intensive operations. Additionally, cycling through millions of files and analyzing them makes this a process that often takes hours. By the time it completes, it might have to start over again. Oh yea, and deleting the old files will cause further write amplification for your SSD!
* Chunks that are currently accumulating are only held in memory. If the application crashes, data will be lost. To avoid this, the memory state is periodically checkpointed to disk, which may take significantly longer than the window of data loss we are willing to accept. Restoring the checkpoint may also take several minutes, causing painfully long restart cycles.
The key take away from the existing design is the concept of chunks, which we most certainly want to keep. The most recent chunks always being held in memory is also generally good. After all, the most recent data is queried the most by a large margin.
Having one file per time series is a concept we would like to find an alternative to.
### Series Churn
In the Prometheus context, we use the term  _series churn_  to describe that a set of time series becomes inactive, i.e. receives no more data points, and a new set of active series appears instead.
For example, all series exposed by a given microservice instance have a respective “instance” label attached that identifies its origin. If we perform a rolling update of our microservice and swap out every instance with a newer version, series churn occurs. In more dynamic environments those events may happen on an hourly basis. Cluster orchestration systems like Kubernetes allow continuous auto-scaling and frequent rolling updates of applications, potentially creating tens of thousands of new application instances, and with them completely new sets of time series, every day.
```
series
^
│ . . . . . .
│ . . . . . .
│ . . . . . .
│ . . . . . . .
│ . . . . . . .
│ . . . . . . .
│ . . . . . .
│ . . . . . .
│ . . . . .
│ . . . . .
│ . . . . .
v
<-------------------- time --------------------->
```
So even if the entire infrastructure roughly remains constant in size, over time there's a linear growth of time series in our database. While a Prometheus server will happily collect data for 10 million time series, query performance is significantly impacted if data has to be found among a billion series.
#### Current solution
The current V2 storage of Prometheus has an index based on LevelDB for all series that are currently stored. It allows querying series containing a given label pair, but lacks a scalable way to combine results from different label selections.
For example, selecting all series with label `__name__="requests_total"` works efficiently, but selecting all series with `instance="A" AND __name__="requests_total"` has scalability problems. We will later revisit what causes this and which tweaks are necessary to improve lookup latencies.
This problem is in fact what spawned the initial hunt for a better storage system. Prometheus needed an improved indexing approach for quickly searching hundreds of millions of time series.
### Resource consumption
Resource consumption is one of the consistent topics when trying to scale Prometheus (or anything, really). But it's not actually the absolute resource hunger that is troubling users. In fact, Prometheus manages an incredible throughput given its requirements. The problem is rather its relative unpredictability and instability in face of changes. By its architecture the V2 storage slowly builds up chunks of sample data, which causes the memory consumption to ramp up over time. As chunks get completed, they are written to disk and can be evicted from memory. Eventually, Prometheus's memory usage reaches a steady state. That is until the monitored environment changes —  _series churn_  increases the usage of memory, CPU, and disk IO every time we scale an application or do a rolling update.
If the change is ongoing, it will yet again reach a steady state eventually but it will be significantly higher than in a more static environment. Transition periods are often multiple hours long and it is hard to determine what the maximum resource usage will be.
The approach of having a single file per time series also makes it way too easy for a single query to knock out the Prometheus process. When querying data that is not cached in memory, the files for queried series are opened and the chunks containing relevant data points are read into memory. If the amount of data exceeds the memory available, Prometheus quits rather ungracefully by getting OOM-killed.
After the query is completed the loaded data can be released again but it is generally cached much longer to serve subsequent queries on the same data faster. The latter is a good thing obviously.
Lastly, we looked at write amplification in the context of SSDs and how Prometheus addresses it by batching up writes to mitigate it. Nonetheless, in several places it still causes write amplification by having too small batches and not aligning data precisely on page boundaries. For larger Prometheus servers, a reduced hardware lifetime was observed in the real world. Chances are that this is still rather normal for database applications with high write throughput, but we should keep an eye on whether we can mitigate it.
### Starting Over
By now we have a good idea of our problem domain, how the V2 storage solves it, and where its design has issues. We also saw some great concepts that we want to adapt more or less seamlessly. A fair amount of V2's problems can be addressed with improvements and partial redesigns, but to keep things fun (and after carefully evaluating my options, of course), I decided to take a stab at writing an entire time series database — from scratch, i.e. writing bytes to the file system.
The critical concerns of performance and resource usage are a direct consequence of the chosen storage format. We have to find the right set of algorithms and disk layout for our data to implement a well-performing storage layer.
This is where I take the shortcut and drive straight to the solution — skip the headache, failed ideas, endless sketching, tears, and despair.
### V3 — Macro Design
What's the macro layout of our storage? In short, everything that is revealed when running `tree` on our data directory. Just looking at that gives us a surprisingly good picture of what is going on.
```
$ tree ./data
./data
├── b-000001
│ ├── chunks
│ │ ├── 000001
│ │ ├── 000002
│ │ └── 000003
│ ├── index
│ └── meta.json
├── b-000004
│ ├── chunks
│ │ └── 000001
│ ├── index
│ └── meta.json
├── b-000005
│ ├── chunks
│ │ └── 000001
│ ├── index
│ └── meta.json
└── b-000006
├── meta.json
└── wal
├── 000001
├── 000002
└── 000003
```
At the top level, we have a sequence of numbered blocks, prefixed with `b-`. Each block obviously holds a file containing an index and a "chunk" directory holding more numbered files. The “chunks” directory contains nothing but raw chunks of data points for various series. Just as for V2, this makes reading series data over a time windows very cheap and allows us to apply the same efficient compression algorithms. The concept has proven to work well and we stick with it. Obviously, there is no longer a single file per series but instead a handful of files holds chunks for many of them.
The existence of an “index” file should not be surprising. Let's just assume it contains a lot of black magic allowing us to find labels, their possible values, entire time series and the chunks holding their data points.
But why are there several directories containing the layout of index and chunk files? And why does the last one contain a "wal" directory instead? Understanding those two questions, solves about 90% of our problems.
#### Many Little Databases
We partition our  _horizontal_  dimension, i.e. the time space, into non-overlapping blocks. Each block acts as a fully independent database containing all time series data for its time window. Hence, it has its own index and set of chunk files.
```
t0 t1 t2 t3 now
┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐
│ │ │ │ │ │ │ │ ┌────────────┐
│ │ │ │ │ │ │ mutable │ <─── write ──── ┤ Prometheus │
│ │ │ │ │ │ │ │ └────────────┘
└───────────┘ └───────────┘ └───────────┘ └───────────┘ ^
└──────────────┴───────┬──────┴──────────────┘ │
│ query
│ │
merge ─────────────────────────────────────────────────┘
```
Every block of data is immutable. Of course, we must be able to add new series and samples to the most recent block as we collect new data. For this block, all new data is written to an in-memory database that provides the same lookup properties as our persistent blocks. The in-memory data structures can be updated efficiently. To prevent data loss, all incoming data is also written to a temporary  _write ahead log_ , which is the set of files in our “wal” directory, from which we can re-populate the in-memory database on restart.
All these files come with their own serialization format, which comes with all the things one would expect: lots of flags, offsets, varints, and CRC32 checksums. Good fun to come up with, rather boring to read about.
This layout allows us to fan out queries to all blocks relevant to the queried time range. The partial results from each block are merged back together to form the overall result.
This horizontal partitioning adds a few great capabilities:
* When querying a time range, we can easily ignore all data blocks outside of this range. It trivially addresses the problem of  _series churn_  by reducing the set of inspected data to begin with.
* When completing a block, we can persist the data from our in-memory database by sequentially writing just a handful of larger files. We avoid any write-amplification and serve SSDs and HDDs equally well.
* We keep the good property of V2 that recent chunks, which are queried most, are always hot in memory.
* Nicely enough, we are also no longer bound to the fixed 1KiB chunk size to better align data on disk. We can pick any size that makes the most sense for the individual data points and chosen compression format.
* Deleting old data becomes extremely cheap and instantaneous. We merely have to delete a single directory. Remember, in the old storage we had to analyze and re-write up to hundreds of millions of files, which could take hours to converge.
Each block also contains a `meta.json` file. It simply holds human-readable information about the block to easily understand the state of our storage and the data it contains.
##### mmap
Moving from millions of small files to a handful of larger allows us to keep all files open with little overhead. This unblocks the usage of [`mmap(2)`][8], a system call that allows us to transparently back a virtual memory region by file contents. For simplicity, you might want to think of it like swap space, just that all our data is on disk already and no writes occur when swapping data out of memory.
This means we can treat all contents of our database as if they were in memory without occupying any physical RAM. Only if we access certain byte ranges in our database files, the operating system lazily loads pages from disk. This puts the operating system in charge of all memory management related to our persisted data. Generally, it is more qualified to make such decisions, as it has the full view on the entire machine and all its processes. Queried data can be rather aggressively cached in memory, yet under memory pressure the pages will be evicted. If the machine has unused memory, Prometheus will now happily cache the entire database, yet will immediately return it once another application needs it.
Therefore, queries can longer easily OOM our process by querying more persisted data than fits into RAM. The memory cache size becomes fully adaptive and data is only loaded once the query actually needs it.
From my understanding, this is how a lot of databases work today and an ideal way to do it if the disk format allows — unless one is confident to outsmart the OS from within the process. We certainly get a lot of capabilities with little work from our side.
#### Compaction
The storage has to periodically "cut" a new block and write the previous one, which is now completed, onto disk. Only after the block was successfully persisted, the write ahead log files, which are used to restore in-memory blocks, are deleted.
We are interested in keeping each block reasonably short (about two hours for a typical setup) to avoid accumulating too much data in memory. When querying multiple blocks, we have to merge their results into an overall result. This merge procedure obviously comes with a cost and a week-long query should not have to merge 80+ partial results.
To achieve both, we introduce  _compaction_ . Compaction describes the process of taking one or more blocks of data and writing them into a, potentially larger, block. It can also modify existing data along the way, e.g. dropping deleted data, or restructuring our sample chunks for improved query performance.
```
t0 t1 t2 t3 t4 now
┌────────────┐ ┌──────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐
│ 1 │ │ 2 │ │ 3 │ │ 4 │ │ 5 mutable │ before
└────────────┘ └──────────┘ └───────────┘ └───────────┘ └───────────┘
┌─────────────────────────────────────────┐ ┌───────────┐ ┌───────────┐
│ 1 compacted │ │ 4 │ │ 5 mutable │ after (option A)
└─────────────────────────────────────────┘ └───────────┘ └───────────┘
┌──────────────────────────┐ ┌──────────────────────────┐ ┌───────────┐
│ 1 compacted │ │ 3 compacted │ │ 5 mutable │ after (option B)
└──────────────────────────┘ └──────────────────────────┘ └───────────┘
```
In this example we have the sequential blocks `[1, 2, 3, 4]`. Blocks 1, 2, and 3 can be compacted together and the new layout is `[1, 4]`. Alternatively, compact them in pairs of two into `[1, 3]`. All time series data still exist but now in fewer blocks overall. This significantly reduces the merging cost at query time as fewer partial query results have to be merged.
#### Retention
We saw that deleting old data was a slow process in the V2 storage and put a toll on CPU, memory, and disk alike. How can we drop old data in our block based design? Quite simply, by just deleting the directory of a block that has no data within our configured retention window. In the example below, block 1 can safely be deleted, whereas 2 has to stick around until it falls fully behind the boundary.
```
|
┌────────────┐ ┌────┼─────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐
│ 1 │ │ 2 | │ │ 3 │ │ 4 │ │ 5 │ . . .
└────────────┘ └────┼─────┘ └───────────┘ └───────────┘ └───────────┘
|
|
retention boundary
```
The older data gets, the larger the blocks may become as we keep compacting previously compacted blocks. An upper limit has to be applied so blocks don’t grow to span the entire database and thus diminish the original benefits of our design.
Conveniently, this also limits the total disk overhead of blocks that are partially inside and partially outside of the retention window, i.e. block 2 in the example above. When setting the maximum block size at 10% of the total retention window, our total overhead of keeping block 2 around is also bound by 10%.
Summed up, retention deletion goes from very expensive, to practically free.
> _If you've come this far and have some background in databases, you might be asking one thing by now: Is any of this new? — Not really; and probably for the better._
>
> _The pattern of batching data up in memory, tracked in a write ahead log, and periodically flushed to disk is ubiquitous today._
> _The benefits we have seen apply almost universally regardless of the data's domain specifics. Prominent open source examples following this approach are LevelDB, Cassandra, InfluxDB, or HBase. The key takeaway is to avoid reinventing an inferior wheel, researching proven methods, and applying them with the right twist._
> _Running out of places to add your own magic dust later is an unlikely scenario._
### The Index
The initial motivation to investigate storage improvements were the problems brought by  _series churn_ . The block-based layout reduces the total number of series that have to be considered for serving a query. So assuming our index lookup was of complexity  _O(n^2)_ , we managed to reduce the  _n_  a fair amount and now have an improved complexity of  _O(n^2)_  — uhm, wait... damnit.
A quick flashback to "Algorithms 101" reminds us that this, in theory, did not buy us anything. If things were bad before, they are just as bad now. Theory can be depressing.
In practice, most of our queries will already be answered significantly faster. Yet, queries spanning the full time range remain slow even if they just need to find a handful of series. My original idea, dating back way before all this work was started, was a solution to exactly this problem: we need a more capable [ _inverted index_ ][9].
An inverted index provides a fast lookup of data items based on a subset of their contents. Simply put, I can look up all series that have a label `app=”nginx"` without having to walk through every single series and check whether it contains that label.
For that, each series is assigned a unique ID by which it can be retrieved in constant time, i.e. O(1). In this case the ID is our  _forward index_ .
> Example: If the series with IDs 10, 29, and 9 contain the label `app="nginx"`, the inverted index for the label "nginx" is the simple list `[10, 29, 9]`, which can be used to quickly retrieve all series containing the label. Even if there were 20 billion further series, it would not affect the speed of this lookup.
In short, if  _n_  is our total number of series, and  _m_  is the result size for a given query, the complexity of our query using the index is now  _O(m)_ . Queries scaling along the amount of data they retrieve ( _m_ ) instead of the data body being searched ( _n_ ) is a great property as  _m_  is generally significantly smaller.
For brevity, let’s assume we can retrieve the inverted index list itself in constant time.
Actually, this is almost exactly the kind of inverted index V2 has and a minimum requirement to serve performant queries across millions of series. The keen observer will have noticed, that in the worst case, a label exists in all series and thus  _m_  is, again, in  _O(n)_ . This is expected and perfectly fine. If you query all data, it naturally takes longer. Things become problematic once we get involved with more complex queries.
#### Combining Labels
Labels associated with millions of series are common. Suppose a horizontally scaling “foo” microservice with hundreds of instances with thousands of series each. Every single series will have the label `app="foo"`. Of course, one generally won't query all series but restrict the query by further labels, e.g. I want to know how many requests my service instances received and query `__name__="requests_total" AND app="foo"`.
To find all series satisfying both label selectors, we take the inverted index list for each and intersect them. The resulting set will typically be orders of magnitude smaller than each input list individually. As each input list has the worst case size O(n), the brute force solution of nested iteration over both lists, has a runtime of O(n^2). The same cost applies for other set operations, such as the union (`app="foo" OR app="bar"`). When adding further label selectors to the query, the exponent increases for each to O(n^3), O(n^4), O(n^5), ... O(n^k). A lot of tricks can be played to minimize the effective runtime by changing the execution order. The more sophisticated, the more knowledge about the shape of the data and the relationships between labels is needed. This introduces a lot of complexity, yet does not decrease our algorithmic worst case runtime.
This is essentially the approach in the V2 storage and luckily a seemingly slight modification is enough gain significant improvements. What happens if we assume that the IDs in our inverted indices are sorted?
Suppose this example of lists for our initial query:
```
__name__="requests_total" -> [ 9999, 1000, 1001, 2000000, 2000001, 2000002, 2000003 ]
app="foo" -> [ 1, 3, 10, 11, 12, 100, 311, 320, 1000, 1001, 10002 ]
intersection => [ 1000, 1001 ]
```
The intersection is fairly small. We can find it by setting a cursor at the beginning of each list and always advancing the one at the smaller number. When both numbers are equal, we add the number to our result and advance both cursors. Overall, we scan both lists in this zig-zag pattern and thus have a total cost of  _O(2n) = O(n)_  as we only ever move forward in either list.
The procedure for more than two lists of different set operations works similarly. So the number of  _k_  set operations merely modifies the factor ( _O(k*n)_ ) instead of the exponent ( _O(n^k)_ ) of our worst-case lookup runtime. A great improvement.
What I described here is a simplified version of the canonical search index used by practically any [full text search engine][10] out there. Every series descriptor is treated as a short "document", and every label (name + fixed value) as a "word" inside of it. We can ignore a lot of additional data typically encountered in search engine indices, such as word position and frequency data.
Seemingly endless research exists on approaches improving the practical runtime, often making some assumptions about the input data. Unsurprisingly, there are also plenty of techniques to compress inverted indices that come with their own benefits and drawbacks. As our "documents" are tiny and the “words” are hugely repetitive across all series, compression becomes almost irrelevant. For example, a real-world dataset of ~4.4 million series with about 12 labels each has less than 5,000 unique labels. For our initial storage version, we stick to the basic approach without compression, and just a few simple tweaks added to skip over large ranges of non-intersecting IDs.
While keeping the IDs sorted may sound simple, it is not always a trivial invariant to keep up. For instance, the V2 storage assigns hashes as IDs to new series and we cannot efficiently build up sorted inverted indices.
Another daunting task is modifying the indices on disk as data gets deleted or updated. Typically, the easiest approach is to simply recompute and rewrite them but doing so while keeping the database queryable and consistent. The V3 storage does exactly this by having a separate immutable index per block that is only modified via rewrite on compaction. Only the indices for the mutable blocks, which are held entirely in memory, need to be updated.
### Benchmarking
I started initial development of the storage with a benchmark based on ~4.4 million series descriptors extracted from a real world data set and generated synthetic data points to feed into those series. This iteration just tested the stand-alone storage and was crucial to quickly identify performance bottlenecks and trigger deadlocks only experienced under highly concurrent load.
After the conceptual implementation was done, the benchmark could sustain a write throughput of 20 million data points per second on my Macbook Pro — all while a dozen Chrome tabs and Slack were running. So while this sounded all great it also indicated that there's no further point in pushing this benchmark (or running it in a less random environment for that matter). After all, it is synthetic and thus not worth much beyond a good first impression. Starting out about 20x above the initial design target, it was time to embed this into an actual Prometheus server, adding all the practical overhead and flakes only experienced in more realistic environments.
We actually had no reproducible benchmarking setup for Prometheus, in particular none that allowed A/B testing of different versions. Concerning in hindsight, but [now we have one][11]!
Our tool allows us to declaratively define a benchmarking scenario, which is then deployed to a Kubernetes cluster on AWS. While this is not the best environment for all-out benchmarking, it certainly reflects our user base better than dedicated bare metal servers with 64 cores and 128GB of memory.
We deploy two Prometheus 1.5.2 servers (V2 storage) and two Prometheus servers from the 2.0 development branch (V3 storage). Each Prometheus server runs on a dedicated machine with an SSD. A horizontally scaled application exposing typical microservice metrics is deployed to worker nodes. Additionally, the Kubernetes cluster itself and the nodes are being monitored. The whole setup is supervised by yet another Meta-Prometheus, monitoring each Prometheus server for health and performance.
To simulate series churn, the microservice is periodically scaled up and down to remove old pods and spawn new pods, exposing new series. Query load is simulated by a selection of "typical" queries, run against one server of each Prometheus version.
Overall the scaling and querying load as well as the sampling frequency significantly exceed today's production deployments of Prometheus. For instance, we swap out 60% of our microservice instances every 15 minutes to produce series churn. This would likely only happen 1-5 times a day in a modern infrastructure. This ensures that our V3 design is capable of handling the workloads of the years ahead. As a result, the performance differences between Prometheus 1.5.2 and 2.0 are larger than in a more moderate environment.
In total, we are collecting about 110,000 samples per second from 850 targets exposing half a million series at a time.
After leaving this setup running for a while, we can take a look at the numbers. We evaluate several metrics over the first 12 hours within both versiones reached a steady state.
> Be aware of the slightly truncated Y axis in screen shots from the Prometheus graph UI.
![Heap usage GB](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/heap_usage.png)
> _Heap memory usage in GB_
Memory usage is the most troubling resource for users today as it is relatively unpredictable and it may cause the process to crash.
Obviously, the queried servers are consuming more memory, which can largely be attributed to overhead of the query engine, which will be subject to future optimizations. Overall, Prometheus 2.0's memory consumption is reduced by 3-4x. After about six hours, there is a clear spike in Prometheus 1.5, which aligns with the our retention boundary at six hours. As deletions are quite costly, resource consumption ramps up. This will become visible throughout various other graphs below.
![CPU usage cores](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/cpu_usage.png)
> _CPU usage in cores/second_
A similar pattern shows for CPU usage, but the delta between queried and non-queried servers is more significant. Averaging at about 0.5 cores/sec while ingesting about 110,000 samples/second, our new storage becomes almost negligible compared to the cycles spent on query evaluation. In total the new storage needs 3-10 times fewer CPU resources.
![Disk writes](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/disk_writes.png)
>_Disk writes in MB/second_
The by far most dramatic and unexpected improvement shows in write utilization of our disk. It clearly shows why Prometheus 1.5 is prone to wear out SSDs. We see an initial ramp-up as soon as the first chunks are persisted into the series files and a second ramp-up once deletion starts rewriting them. Surprisingly, the queried and non-queried server show a very different utilization.
Prometheus 2.0 on the other hand, merely writes about a single Megabyte per second to its write ahead log. Writes periodically spike when blocks are compacted to disk. Overall savings: staggering 97-99%.
![Disk usage](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/disk_usage.png)
> _Disk size in GB_
Closely related to disk writes is the total amount of occupied disk space. As we are using almost the same compression algorithm for samples, which is the bulk of our data, they should be about the same. In a more stable setup that would largely be true, but as we are dealing with high  _series churn_ , there's also the per-series overhead to consider.
As we can see, Prometheus 1.5 ramps up storage space a lot faster before both versions reach a steady state as the retention kicks in. Prometheus 2.0 seems to have a significantly lower overhead per individual series. We can nicely see how space is linearly filled up by the write ahead log and instantaneously drops as its gets compacted. The fact that the lines for both Prometheus 2.0 servers do not exactly match is a fact that needs further investigation.
This all looks quite promising. The important piece left is query latency. The new index should have improved our lookup complexity. What has not substantially changed is processing of this data, e.g. in `rate()` functions or aggregations. Those aspects are part of the query engine.
![Query latency](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/query_latency.png)
>_99th percentile query latency in seconds_
Expectations are completely met by the data. In Prometheus 1.5 the query latency increases over time as more series are stored. It only levels off once retention starts and old series are deleted. In contrast, Prometheus 2.0 stays in place right from the beginning.
Some caution must be taken on how this data was collected. The queries fired against the servers were chosen by estimating a good mix of range and instant queries, doing heavier and more lightweight computations, and touching few or many series. It does not necessarily represent a real-world distribution of queries. It is also not representative for queries hitting cold data and we can assume that all sample data is practically always hot in memory in either storage.
Nonetheless, we can say with good confidence, that the overall query performance became very resilient to series churn and improved by up to 4x in our straining benchmarking scenario. In a more static environment, we can assume query time to be mostly spent in the query engine itself and the improvement to be notably lower.
![Ingestion rate](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/ingestion_rate.png)
>_Ingested samples/second_
Lastly, a quick look into our ingestion rates of the different Prometheus servers. We can see that both servers with the V3 storage have the same ingestion rate. After a few hours it becomes unstable, which is caused by various nodes of the benchmarking cluster becoming unresponsive due to high load rather than the Prometheus instances. (The fact that both 2.0 lines exactly match is hopefully convincing enough.)
Both Prometheus 1.5.2 servers start suffering from significant drops in ingestion rate even though more CPU and memory resources are available. The high stress of series churn causes a larger amount of data to not be collected.
But what's the  _absolute maximum_  number of samples per second you could ingest now?
I don't know — and deliberately don't care.
There are a lot of factors that shape the data flowing into Prometheus and there is no single number capable of capturing quality. Maximum ingestion rate has historically been a metric leading to skewed benchmarks and neglect of more important aspects such as query performance and resilience to series churn. The rough assumption that resource usage increases linearly was confirmed by some basic testing. It is easy to extrapolate what could be possible.
Our benchmarking setup simulates a highly dynamic environment stressing Prometheus more than most real-world setups today. The results show we went way above our initial design goal, while running on non-optimal cloud servers. Ultimately, success will be determined by user feedback rather than benchmarking numbers.
> Note:  _At time of writing this, Prometheus 1.6 is in development, which will allow configuring the maximum memory usage more reliably and may notably reduce overall consumption in favor of slightly increased CPU utilization. I did not repeat the tests against this as the overall results still hold, especially when facing high series churn._
### Conclusion
Prometheus sets out to handle high cardinality of series and throughput of individual samples. It remains a challenging task, but the new storage seems to position us well for the hyper-scale, hyper-convergent, GIFEE infrastructure of the futu... well, it seems to work pretty well.
A [first alpha release of Prometheus 2.0][12] with the new V3 storage is available for testing. Expect crashes, deadlocks, and other bugs at this early stage.
The code for the storage itself can be found [in a separate project][13]. It's surprisingly agnostic to Prometheus itself and could be widely useful for a wider range of applications looking for an efficient local storage time series database.
> _There's a long list of people to thank for their contributions to this work. Here they go in no particular order:_
>
> _The groundlaying work by Bjoern Rabenstein and Julius Volz on the V2 storage engine and their feedback on V3 was fundamental to everything seen in this new generation._
>
> _Wilhelm Bierbaum's ongoing advice and insight contributed significantly to the new design. Brian Brazil's continous feedback ensured that we ended up with a semantically sound approach. Insightful discussions with Peter Bourgon validated the design and shaped this write-up._
>
> _Not to forget my entire team at CoreOS and the company itself for supporting and sponsoring this work. Thanks to everyone who listened to my ramblings about SSDs, floats, and serialization formats again and again._
--------------------------------------------------------------------------------
via: https://fabxc.org/blog/2017-04-10-writing-a-tsdb/
作者：[Fabian Reinartz ][a]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://twitter.com/fabxc
[1]:https://en.wikipedia.org/wiki/Inode
[2]:https://prometheus.io/
[3]:https://kubernetes.io/
[4]:https://en.wikipedia.org/wiki/Write_amplification
[5]:http://codecapsule.com/2014/02/12/coding-for-ssds-part-1-introduction-and-table-of-contents/
[6]:https://prometheus.io/docs/practices/rules/
[7]:http://www.vldb.org/pvldb/vol8/p1816-teller.pdf
[8]:https://en.wikipedia.org/wiki/Mmap
[9]:https://en.wikipedia.org/wiki/Inverted_index
[10]:https://en.wikipedia.org/wiki/Search_engine_indexing#Inverted_indices
[11]:https://github.com/prometheus/prombench
[12]:https://prometheus.io/blog/2017/04/10/promehteus-20-sneak-peak/
[13]:https://github.com/prometheus/tsdb

50
sources/tech/20190319 Blockchain 2.0- Blockchain In Real Estate -Part 4.md
View File

@ -1,50 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Blockchain 2.0: Blockchain In Real Estate [Part 4])
[#]: via: (https://www.ostechnix.com/blockchain-2-0-blockchain-in-real-estate/)
[#]: author: (EDITOR https://www.ostechnix.com/author/editor/)
Blockchain 2.0: Blockchain In Real Estate [Part 4]
======
![](https://www.ostechnix.com/wp-content/uploads/2019/03/Blockchain-In-Real-Estate-720x340.png)
### Blockchain 2.0: Smart‘er’ Real Estate
The [**previous article**][1] of this series explored the features of blockchain which will enable institutions to transform and interlace **traditional banking** and **financing systems** with it. This part will explore – **Blockchain in real estate**. The real estate industry is ripe for a revolution. It’s among the most actively traded most significant asset classes known to man. However, filled with regulatory hurdles and numerous possibilities of fraud and deceit, it’s also one of the toughest to participate in. The distributed ledger capabilities of the blockchain utilizing an appropriate consensus algorithm are touted as the way forward for the industry which is traditionally regarded as conservative in its attitude to change.
Real estate has always been a very conservative industry in terms of its myriad operations. Somewhat rightfully so as well. A major economic crisis such as the 2008 financial crisis or the great depression from the early half of the 20th century managed to destroy the industry and its participants. However, like most products of economic value, the real estate industry is resilient and this resilience is rooted in its conservative nature.
The global real estate market comprises an asset class worth **$228 trillion dollars** [1]. Give or take. Other investment assets such as stocks, bonds, and shares combined are only worth **$170 trillion**. Obviously, any and all transactions implemented in such an industry is naturally carefully planned and meticulously executed, for the most part. For the most part, because real estate is also notorious for numerous instances of fraud and devastating loses which ensue them. The industry because of the very conservative nature of its operations is also tough to navigate. It’s heavily regulated with complex laws creating an intertwined web of nuances that are just too difficult for an average person to understand fully. This makes entry and participation near impossible for most people. If you’ve ever been involved in one such deal, you’ll know how heavy and long the paper trail was.
This hard reality is now set to change, albeit a slow and gradual transformation. The very reasons the industry has stuck to its hardy tested roots all this while can finally give way to its modern-day counterpart. The backbone of the real estate industry has always been its paper records. Land deeds, titles, agreements, rental insurance, proofs, and declarations etc., are just the tip of the iceberg here. If you’ve noticed the pattern here, this should be obvious, the distributed ledger technology that is blockchain, fits in perfectly with the needs here. Forget paper records, conventional database systems are also points of major failure. They can be modified by multiple participants, is not tamper proof or un-hackable, has a complicated set of ever-changing regulatory parameters making auditing and verifying data a nightmare. The blockchain perfectly solves all of these issues and more.
Starting with a trivial albeit an important example to show just how bad the current record management practices are in the real estate sector, consider the **Title Insurance business** [2], [3]. Title Insurance is used to hedge against the possibility of the land’s titles and ownership records being inadmissible and hence unenforceable. An insurance product such as this is also referred to as an indemnity cover. It is by law required in many cases that properties have title insurance, especially when dealing with property that has changed hands multiple times over the years. Mortgage firms might insist on the same as well when they back real estate deals. The fact that a product of this kind has existed since the 1850s and that it does business worth at least **$1.5 trillion a year in the US alone** is a testament to the statement at the start. A revolution in terms of how these records are maintained is imperative to have in this situation and the blockchain provides a sustainable solution. Title fraud averages around $100k per case on average as per the **American Land Title Association** and 25% of all titles involved in transactions have an issue regarding their documents[4]. The blockchain allows for setting up an immutable permanent database that will track the property itself, recording each and every transaction or investment that has gone into it. Such a ledger system will make life easier for everyone involved in the real estate industry including one-time home buyers and make financial products such as Title Insurance basically irrelevant. Converting a physical asset such as real estate to a digital asset like this is unconventional and is extant only in theory at the moment. However, such a change is imminent sooner rather than later[5].
Among the areas in which blockchain will have the most impact within real estate is as highlighted above in maintaining a transparent and secure title management system for properties. A blockchain based record of the property can contain information about the property, its location, history of ownership, and any related public record of the same[6]. This will permit closing real estate deals fast and obliviates the need for 3rd party monitoring and oversight. Tasks such as real estate appraisal and tax calculations become matters of tangible objective parameters rather than subjective measures and guesses because of reliable historical data which is publicly verifiable. **UBITQUITY** is one such platform that offers customized blockchain-based solutions to enterprise customers. The platform allows customers to keep track of all property details, payment records, mortgage records and even allows running smart contracts that’ll take care of taxation and leasing automatically[7].
This brings us to the second biggest opportunity and use case of blockchains in real estate. Since the sector is highly regulated by numerous 3rd parties apart from the counterparties involved in the trade, due-diligence and financial evaluations can be significantly time-consuming. These processes are predominantly carried out using offline channels and paperwork needs to travel for days before a final evaluation report comes out. This is especially true for corporate real estate deals and forms a bulk of the total billable hours charged by consultants. In case the transaction is backed by a mortgage, duplication of these processes is unavoidable. Once combined with digital identities for the people and institutions involved along with the property, the current inefficiencies can be avoided altogether and transactions can take place in a matter of seconds. The tenants, investors, institutions involved, consultants etc., could individually validate the data and arrive at a critical consensus thereby validating the property records for perpetuity[8]. This increases the accuracy of verification manifold. Real estate giant **RE/MAX** has recently announced a partnership with service provider **XYO Network Partners** for building a national database of real estate listings in Mexico. They hope to one day create one of the largest (as of yet) decentralized real estate title registry in the world[9].
However, another significant and arguably a very democratic change that the blockchain can bring about is with respect to investing in real estate. Unlike other investment asset classes where even small household investors can potentially participate, real estate often requires large hands-down payments to participate. Companies such as **ATLANT** and **BitOfProperty** tokenize the book value of a property and convert them into equivalents of a cryptocurrency. These tokens are then put for sale on their exchanges similar to how stocks and shares are traded. Any cash flow that the real estate property generates afterward is credited or debited to the token owners depending on their “share” in the property[4].
However, even with all of that said, Blockchain technology is still in very early stages of adoption in the real estate sector and current regulations are not exactly defined for it to be either[8]. Concepts such as distributed applications, distributed anonymous organizations, smart contracts etc., are unheard of in the legal domain in many countries. A complete overhaul of existing regulations and guidelines once all the stakeholders are well educated on the intricacies of the blockchain is the most pragmatic way forward. Again, it’ll be a slow and gradual change to go through, however a much-needed one nonetheless. The next article of the series will look at how **“Smart Contracts”** , such as those implemented by companies such as UBITQUITY and XYO are created and executed in the blockchain.
--------------------------------------------------------------------------------
via: https://www.ostechnix.com/blockchain-2-0-blockchain-in-real-estate/
作者：[EDITOR][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.ostechnix.com/author/editor/
[b]: https://github.com/lujun9972
[1]: https://www.ostechnix.com/blockchain-2-0-redefining-financial-services/

177
sources/tech/20190329 How to manage your Linux environment.md
View File

@ -1,177 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: (robsean)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (How to manage your Linux environment)
[#]: via: (https://www.networkworld.com/article/3385516/how-to-manage-your-linux-environment.html#tk.rss_all)
[#]: author: (Sandra Henry-Stocker https://www.networkworld.com/author/Sandra-Henry_Stocker/)
How to manage your Linux environment
======
### Linux user environments help you find the command you need and get a lot done without needing details about how the system is configured. Where the settings come from and how they can be modified is another matter.
![IIP Photo Archive \(CC BY 2.0\)][1]
The configuration of your user account on a Linux system simplifies your use of the system in a multitude of ways. You can run commands without knowing where they're located. You can reuse previously run commands without worrying how the system is keeping track of them. You can look at your email, view man pages, and get back to your home directory easily no matter where you might have wandered off to in the file system. And, when needed, you can tweak your account settings so that it works even more to your liking.
Linux environment settings come from a series of files — some are system-wide (meaning they affect all user accounts) and some are configured in files that are sitting in your home directory. The system-wide settings take effect when you log in and local ones take effect right afterwards, so the changes that you make in your account will override system-wide settings. For bash users, these files include these system files:
```
/etc/environment
/etc/bash.bashrc
/etc/profile
```
And some of these local files:
```
~/.bashrc
~/.profile -- not read if ~/.bash_profile or ~/.bash_login
~/.bash_profile
~/.bash_login
```
You can modify any of the local four that exist, since they sit in your home directory and belong to you.
**[ Two-Minute Linux Tips:[Learn how to master a host of Linux commands in these 2-minute video tutorials][2] ]**
### Viewing your Linux environment settings
To view your environment settings, use the **env** command. Your output will likely look similar to this:
```
$ env
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;
01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:
*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:
*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:
*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;
31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:
*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:
*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:
*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:
*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:
*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:
*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:
*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:
*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:
*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:
*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:
*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:
*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.spf=00;36:
SSH_CONNECTION=192.168.0.21 34975 192.168.0.11 22
LESSCLOSE=/usr/bin/lesspipe %s %s
LANG=en_US.UTF-8
OLDPWD=/home/shs
XDG_SESSION_ID=2253
USER=shs
PWD=/home/shs
HOME=/home/shs
SSH_CLIENT=192.168.0.21 34975 22
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
SSH_TTY=/dev/pts/0
MAIL=/var/mail/shs
TERM=xterm
SHELL=/bin/bash
SHLVL=1
LOGNAME=shs
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
XDG_RUNTIME_DIR=/run/user/1000
PATH=/home/shs/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
LESSOPEN=| /usr/bin/lesspipe %s
_=/usr/bin/env
```
While you're likely to get a _lot_ of output, the first big section shown above deals with the colors that are used on the command line to identify various file types. When you see something like ***.tar=01;31:** , this tells you that tar files will be displayed in a file listing in red, while ***.jpg=01;35:** tells you that jpg files will show up in purple. These colors are meant to make it easy to pick out certain files from a file listing. You can learn more about these colors are defined and how to customize them at [Customizing your colors on the Linux command line][3].
One easy way to turn colors off when you prefer a simpler display is to use a command such as this one:
```
$ ls -l --color=never
```
That command could easily be turned into an alias:
```
$ alias ll2='ls -l --color=never'
```
You can also display individual settings using the **echo** command. In this command, we display the number of commands that will be remembered in our history buffer:
```
$ echo $HISTSIZE
1000
```
Your last location in the file system will be remembered if you've moved.
```
PWD=/home/shs
OLDPWD=/tmp
```
### Making changes
You can make changes to environment settings with a command like this, but add a line lsuch as "HISTSIZE=1234" in your ~/.bashrc file if you want to retain this setting.
```
$ export HISTSIZE=1234
```
### What it means to "export" a variable
Exporting a variable makes the setting available to your shell and possible subshells. By default, user-defined variables are local and are not exported to new processes such as subshells and scripts. The export command makes variables available to functions to child processes.
### Adding and removing variables
You can create new variables and make them available to you on the command line and subshells quite easily. However, these variables will not survive your logging out and then back in again unless you also add them to ~/.bashrc or a similar file.
```
$ export MSG="Hello, World!"
```
You can unset a variable if you need by using the **unset** command:
```
$ unset MSG
```
If the variable is defined locally, you can easily set it back up by sourcing your startup file(s). For example:
```
$ echo $MSG
Hello, World!
$ unset $MSG
$ echo $MSG
$ . ~/.bashrc
$ echo $MSG
Hello, World!
```
### Wrap-up
User accounts are set up with an appropriate set of startup files for creating a userful user environment, but both individual users and sysadmins can change the default settings by editing their personal setup files (users) or the files from which many of the settings originate (sysadmins).
Join the Network World communities on [Facebook][4] and [LinkedIn][5] to comment on topics that are top of mind.
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3385516/how-to-manage-your-linux-environment.html#tk.rss_all
作者：[Sandra Henry-Stocker][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Sandra-Henry_Stocker/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2019/03/environment-rocks-leaves-100792229-large.jpg
[2]: https://www.youtube.com/playlist?list=PL7D2RMSmRO9J8OTpjFECi8DJiTQdd4hua
[3]: https://www.networkworld.com/article/3269587/customizing-your-text-colors-on-the-linux-command-line.html
[4]: https://www.facebook.com/NetworkWorld/
[5]: https://www.linkedin.com/company/network-world

2
sources/tech/20190411 Be your own certificate authority.md
View File

@ -1,5 +1,5 @@
[#]: collector: (lujun9972) [#]: collector: (lujun9972)
[#]: translator: ( ) [#]: translator: (geekpi)
[#]: reviewer: ( ) [#]: reviewer: ( )
[#]: publisher: ( ) [#]: publisher: ( )
[#]: url: ( ) [#]: url: ( )

2
sources/tech/20190427 Monitoring CPU and GPU Temperatures on Linux.md
View File

@ -1,5 +1,5 @@
[#]: collector: (lujun9972) [#]: collector: (lujun9972)
[#]: translator: ( ) [#]: translator: (cycoe)
[#]: reviewer: ( ) [#]: reviewer: ( )
[#]: publisher: ( ) [#]: publisher: ( )
[#]: url: ( ) [#]: url: ( )

116
sources/tech/20190428 Installing Budgie Desktop on Ubuntu -Quick Guide.md
View File

@ -1,116 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: (geekpi)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Installing Budgie Desktop on Ubuntu [Quick Guide])
[#]: via: (https://itsfoss.com/install-budgie-ubuntu/)
[#]: author: (Atharva Lele https://itsfoss.com/author/atharva/)
Installing Budgie Desktop on Ubuntu [Quick Guide]
======
_**Brief: Learn how to install Budgie desktop on Ubuntu in this step-by-step tutorial.**_
Among all the [various Ubuntu versions][1], [Ubuntu Budgie][2] is the most underrated one. It looks elegant and it’s not heavy on resources.
Read this [Ubuntu Budgie review][3] or simply watch this video to see what Ubuntu Budgie 18.04 looks like.
[Subscribe to our YouTube channel for more Linux Videos][4]
If you like [Budgie desktop][5] but you are using some other version of Ubuntu such as the default Ubuntu with GNOME desktop, I have good news for you. You can install Budgie on your current Ubuntu system and switch the desktop environments.
In this post, I’m going to tell you exactly how to do that. But first, a little introduction to Budgie for those who are unaware about it.
Budgie desktop environment is developed mainly by [Solus Linux team.][6] It is designed with focus on elegance and modern usage. Budgie is available for all major Linux distributions for users to try and experience this new desktop environment. Budgie is pretty mature by now and provides a great desktop experience.
Warning
Installing multiple desktops on the same system MAY result in conflicts and you may see some issue like missing icons in the panel or multiple icons of the same program.
You may not see any issue at all as well. It’s your call if you want to try different desktop.
### Install Budgie on Ubuntu
This method is not tested on Linux Mint, so I recommend that you not follow this guide for Mint.
For those on Ubuntu, Budgie is now a part of the Ubuntu repositories by default. Hence, we don’t need to add any PPAs in order to get Budgie.
To install Budgie, simply run this command in terminal. We’ll first make sure that the system is fully updated.
```
sudo apt update && sudo apt upgrade
sudo apt install ubuntu-budgie-desktop
```
When everything is done downloading, you will get a prompt to choose your display manager. Select ‘lightdm’ to get the full Budgie experience.
![Select lightdm][7]
After the installation is complete, reboot your computer. You will be then greeted by the Budgie login screen. Enter your password to go into the homescreen.
![Budgie Desktop Home][8]
### Switching to other desktop environments
![Budgie login screen][9]
You can click the Budgie icon next to your name to get options for login. From there you can select between the installed Desktop Environments (DEs). In my case, I see Budgie and the default Ubuntu (GNOME) DEs.
![Select your DE][10]
Hence whenever you feel like logging into GNOME, you can do so using this menu.
[][11]
Suggested read Get Rid of 'snapd returned status code 400: Bad Request' Error in Ubuntu
### How to Remove Budgie
If you don’t like Budgie or just want to go back to your regular old Ubuntu, you can switch back to your regular desktop as described in the above section.
However, if you really want to remove Budgie and its component, you can follow the following commands to get back to a clean slate.
_**Switch to some other desktop environments before using these commands:**_
```
sudo apt remove ubuntu-budgie-desktop ubuntu-budgie* lightdm
sudo apt autoremove
sudo apt install --reinstall gdm3
```
After running all the commands successfully, reboot your computer.
Now, you will be back to GNOME or whichever desktop environment you had.
**What you think of Budgie?**
Budgie is one of the [best desktop environments for Linux][12]. Hope this short guide helped you install the awesome Budgie desktop on your Ubuntu system.
If you did install Budgie, what do you like about it the most? Let us know in the comments below. And as usual, any questions or suggestions are always welcome.
--------------------------------------------------------------------------------
via: https://itsfoss.com/install-budgie-ubuntu/
作者：[Atharva Lele][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://itsfoss.com/author/atharva/
[b]: https://github.com/lujun9972
[1]: https://itsfoss.com/which-ubuntu-install/
[2]: https://ubuntubudgie.org/
[3]: https://itsfoss.com/ubuntu-budgie-18-review/
[4]: https://www.youtube.com/c/itsfoss?sub_confirmation=1
[5]: https://github.com/solus-project/budgie-desktop
[6]: https://getsol.us/home/
[7]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_install_select_dm.png?fit=800%2C559&ssl=1
[8]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_homescreen.jpg?fit=800%2C500&ssl=1
[9]: https://i2.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_install_lockscreen.png?fit=800%2C403&ssl=1
[10]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_install_lockscreen_select_de.png?fit=800%2C403&ssl=1
[11]: https://itsfoss.com/snapd-error-ubuntu/
[12]: https://itsfoss.com/best-linux-desktop-environments/

120
sources/tech/20190520 Zettlr - Markdown Editor for Writers and Researchers.md Normal file
View File

@ -0,0 +1,120 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Zettlr – Markdown Editor for Writers and Researchers)
[#]: via: (https://itsfoss.com/zettlr-markdown-editor/)
[#]: author: (John Paul https://itsfoss.com/author/john/)
Zettlr – Markdown Editor for Writers and Researchers
======
There are quite a few [Markdown editors available for Linux][1], with more popping up all of the time. The problem is that like [Boostnote][2], most are designed for coders and may not be as welcoming to non-techie people. Let’s take a look at a Markdown editor that wants to replace Word and expensive word processors for the non-techies. Let’s take a look at Zettlr.
### Zettlr Markdown Editor
![Zettlr Light Mode][3]
I may have mentioned it a time or two on this site, but I prefer to write all of my documents in [Markdown][4]. It is simple to learn and does not leave you tied to a proprietary document format. I have also mentioned Markdown editor among my [list of open source tools for writers][5].
I have used a number of Markdown editors and am always interested to try out someone’s new take on the idea. Recently, I came across Zettlr, an open source markdown editor.
[Zettlr][6] is the creation of a German sociologist/political theorist named [Hendrik Erz][7]. Hendrik created Zettlr because he was frustrated by the current line up of word processors. He wanted something that would allow him to “focus on writing and reading only”.
After discovering Markdown, he tried several Markdown editors on different operating systems. But none of them had what he was looking for. [According to Hendrik][8], “But I had to realize that there are simply none written for the needs of organizing a huge amount of text efficiently. Most editors have been written by coders, therefore tailored to the needs of engineers and mathematicians. No luck for a student of social sciences, history or political science like me.”
So he decided to create his own. In November of 2017, he started to work on Zettlr.
![Zettlr About][9]
#### Zettlr Features
Zettlr has a number of neat features, including:
* Import sources from your [Zotero database][10] and cite them in your document
* Focus on your writing with the distraction free mode with optional line muting
* Support for code highlighting
* Use tags to sort information
* Ability to set writing goals for the session
* View writing stats over time
* Pomodoro Timer
* Light/Dark theme
* Create presentation using [reveal.js][11]
* Quick preview of a document
* Search all Markdown documents in a project folder with heatmap showing the density of word searched
* Export files to HTML, PDF, ODT, DOC, reStructuredText, LaTex, TXT, Emacs ORG, [TextBundle][12], and Textpack
* Add custom CSS to your document
[][13]
Suggested read Manage Your PDF Files In Style With Great Little Book Shelf
As I am writing this article, a dialog box popped up telling me about the recently released [1.3.0 beta][14]. This beta will include several new themes, as well as, a boatload of fixes, new features and under the hood improvements.
![Zettlr Night Mode][15]
#### Installing Zettlr
Currently, the only Linux repository that has Zettlr for you to install is the [AUR][16]. If your Linux distro is not Arch-based, you can [download an installer][17] from the website for macOS, Windows, Debian, and Fedora.
#### Final Thoughts on Zettlr
Note: In order to test Zettlr, I used it to write this article.
Zettlr has a number of neat features that I wish my Markdown editor of choice (ghostwriter) had, such as the ability to set a word count goal for the document. I also like the option to preview a document without having to open it.
![Zettlr Settings][18]
I did run into a couple issues, but they had more to do with the fact that Zettlr works a little bit different than ghostwriter. For example, when I tried to copy a quote or name from a web site, it pasted the in-line styling into Zettlr. Fortunately, there is an option to “Paste without Style”. A couple times I ran into a slight delay when I was trying to type. But that could because it is an Electron app.
Overall, I think that Zettlr is a good option for a first time Markdown user. It has features that many Markdown editors already have and adds a few more for those who only ever used word processors
As Hendrik says on the [Zettlr site][8], “Free yourselves from the fetters of word processors and see how your writing process can be improved by using technology that’s right at hand!”
If you do find Zettlr useful, please consider supporting [Hendrik][19]. As he says on the site, “And this free of any charge, because I do not believe in the fast-living, early-dying startup culture. I simply want to help.”
[][20]
Suggested read Calligra Office App Brings ODT Support To Android
Have you ever used Zettlr? What is your favorite Markdown editor? Please let us know in the comments below.
If you found this article interesting, please take a minute to share it on social media, Hacker News or [Reddit][21].
--------------------------------------------------------------------------------
via: https://itsfoss.com/zettlr-markdown-editor/
作者：[John Paul][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://itsfoss.com/author/john/
[b]: https://github.com/lujun9972
[1]: https://itsfoss.com/best-markdown-editors-linux/
[2]: https://itsfoss.com/boostnote-linux-review/
[3]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/05/Zettlr-light-mode.png?fit=800%2C462&ssl=1
[4]: https://daringfireball.net/projects/markdown/
[5]: https://itsfoss.com/open-source-tools-writers/
[6]: https://www.zettlr.com/
[7]: https://github.com/nathanlesage
[8]: https://www.zettlr.com/about
[9]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/05/zettlr-about.png?fit=800%2C528&ssl=1
[10]: https://www.zotero.org/
[11]: https://revealjs.com/#/
[12]: http://textbundle.org/
[13]: https://itsfoss.com/great-little-book-shelf-review/
[14]: https://github.com/Zettlr/Zettlr/releases/tag/v1.3.0-beta
[15]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/05/Zettlr-night-mode.png?fit=800%2C469&ssl=1
[16]: https://aur.archlinux.org/packages/zettlr-bin/
[17]: https://www.zettlr.com/download
[18]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/05/zettlr-settings.png?fit=800%2C353&ssl=1
[19]: https://www.zettlr.com/supporters
[20]: https://itsfoss.com/calligra-android-app-coffice/
[21]: http://reddit.com/r/linuxusersgroup

219
sources/tech/20190521 How to Disable IPv6 on Ubuntu Linux.md Normal file
View File

@ -0,0 +1,219 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (How to Disable IPv6 on Ubuntu Linux)
[#]: via: (https://itsfoss.com/disable-ipv6-ubuntu-linux/)
[#]: author: (Sergiu https://itsfoss.com/author/sergiu/)
How to Disable IPv6 on Ubuntu Linux
======
Are you looking for a way to **disable IPv6** connections on your Ubuntu machine? In this article, I’ll teach you exactly how to do it and why you would consider this option. I’ll also show you how to **enable or re-enable IPv6** in case you change your mind.
### What is IPv6 and why would you want to disable IPv6 on Ubuntu?
**[Internet Protocol version 6][1]** [(][1] **[IPv6][1]**[)][1] is the most recent version of the Internet Protocol (IP), the communications protocol that provides an identification and location system for computers on networks and routes traffic across the Internet. It was developed in 1998 to replace the **IPv4** protocol.
**IPv6** aims to improve security and performance, while also making sure we don’t run out of addresses. It assigns unique addresses globally to every device, storing them in **128-bits** , compared to just 32-bits used by IPv4.
![Disable IPv6 Ubuntu][2]
Although the goal is for IPv4 to be replaced by IPv6, there is still a long way to go. Less than **30%** of the sites on the Internet makes IPv6 connectivity available to users (tracked by Google [here][3]). IPv6 can also cause [problems with some applications at time][4].
Since **VPNs** provide global services, the fact that IPv6 uses globally routed addresses (uniquely assigned) and that there (still) are ISPs that don’t offer IPv6 support shifts this feature lower down their priority list. This way, they can focus on what matters the most for VPN users: security.
Another possible reason you might want to disable IPv6 on your system is not wanting to expose yourself to various threats. Although IPv6 itself is safer than IPv4, the risks I am referring to are of another nature. If you aren’t actively using IPv6 and its features, [having IPv6 enabled leaves you vulnerable to various attacks][5], offering the hacker another possible exploitable tool.
On the same note, configuring basic network rules is not enough. You have to pay the same level of attention to tweaking your IPv6 configuration as you do for IPv4. This can prove to be quite a hassle to do (and also to maintain). With IPv6 comes a suite of problems different to those of IPv4 (many of which can be referenced online, given the age of this protocol), giving your system another layer of complexity.
[][6]
Suggested read How To Remove Drive Icons From Unity Launcher In Ubuntu 14.04 [Beginner Tips]
### Disabling IPv6 on Ubuntu [For Advanced Users Only]
In this section, I’ll be covering how you can disable IPv6 protocol on your Ubuntu machine. Open up a terminal ( **default:** CTRL+ALT+T) and let’s get to it!
**Note:** _For most of the commands you are going to input in the terminal_ _you are going to need root privileges ( **sudo** )._
Warning!
If you are a regular desktop Linux user and prefer a stable working system, please avoid this tutorial. This is for advanced users who know what they are doing and why they are doing so.
#### 1\. Disable IPv6 using Sysctl
First of all, you can **check** if you have IPv6 enabled with:
```
ip a
```
You should see an IPv6 address if it is enabled (the name of your internet card might be different):
![IPv6 Address Ubuntu][7]
You have see the sysctl command in the tutorial about [restarting network in Ubuntu][8]. We are going to use it here as well. To **disable IPv6** you only have to input 3 commands:
```
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
```
You can check if it worked using:
```
ip a
```
You should see no IPv6 entry:
![IPv6 Disabled Ubuntu][9]
However, this only **temporarily disables IPv6**. The next time your system boots, IPv6 will be enabled again.
One method to make this option persist is modifying **/etc/sysctl.conf**. I’ll be using vim to edit the file, but you can use any editor you like. Make sure you have **administrator rights** (use **sudo** ):
![Sysctl Configuration][10]
Add the following lines to the file:
```
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6=1
```
For the settings to take effect use:
```
sudo sysctl -p
```
If IPv6 is still enabled after rebooting, you must create (with root privileges) the file **/etc/rc.local** and fill it with:
```
#!/bin/bash
# /etc/rc.local
/etc/sysctl.d
/etc/init.d/procps restart
exit 0
```
Now use [chmod command][11] to make the file executable:
```
sudo chmod 755 /etc/rc.local
```
What this will do is manually read (during the boot time) the kernel parameters from your sysctl configuration file.
[][12]
Suggested read 3 Ways to Check Linux Kernel Version in Command Line
#### 2\. Disable IPv6 using GRUB
An alternative method is to configure **GRUB** to pass kernel parameters at boot time. You’ll have to edit **/etc/default/grub**. Once again, make sure you have administrator privileges:
![GRUB Configuration][13]
Now you need to modify **GRUB_CMDLINE_LINUX_DEFAULT** and **GRUB_CMDLINE_LINUX** to disable IPv6 on boot:
```
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash ipv6.disable=1"
GRUB_CMDLINE_LINUX="ipv6.disable=1"
```
Save the file and run:
```
sudo update-grub
```
The settings should now persist on reboot.
### Re-enabling IPv6 on Ubuntu
To re-enable IPv6, you’ll have to undo the changes you made. To enable IPv6 until reboot, enter:
```
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=0
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=0
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=0
```
Otherwise, if you modified **/etc/sysctl.conf** you can either remove the lines you added or change them to:
```
net.ipv6.conf.all.disable_ipv6=0
net.ipv6.conf.default.disable_ipv6=0
net.ipv6.conf.lo.disable_ipv6=0
```
You can optionally reload these values:
```
sudo sysctl -p
```
You should once again see a IPv6 address:
![IPv6 Reenabled in Ubuntu][14]
Optionally, you can remove **/etc/rc.local** :
```
sudo rm /etc/rc.local
```
If you modified the kernel parameters in **/etc/default/grub** , go ahead and delete the added options:
```
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""
```
Now do:
```
sudo update-grub
```
**Wrapping Up**
In this guide I provided you ways in which you can **disable IPv6** on Linux, as well as giving you an idea about what IPv6 is and why you would want to disable it.
Did you find this article useful? Do you disable IPv6 connectivity? Let us know in the comment section!
--------------------------------------------------------------------------------
via: https://itsfoss.com/disable-ipv6-ubuntu-linux/
作者：[Sergiu][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://itsfoss.com/author/sergiu/
[b]: https://github.com/lujun9972
[1]: https://en.wikipedia.org/wiki/IPv6
[2]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/05/disable_ipv6_ubuntu.png?fit=800%2C450&ssl=1
[3]: https://www.google.com/intl/en/ipv6/statistics.html
[4]: https://whatismyipaddress.com/ipv6-issues
[5]: https://www.internetsociety.org/blog/2015/01/ipv6-security-myth-1-im-not-running-ipv6-so-i-dont-have-to-worry/
[6]: https://itsfoss.com/remove-drive-icons-from-unity-launcher-in-ubuntu/
[7]: https://i2.wp.com/itsfoss.com/wp-content/uploads/2019/05/ipv6_address_ubuntu.png?fit=800%2C517&ssl=1
[8]: https://itsfoss.com/restart-network-ubuntu/
[9]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/05/ipv6_disabled_ubuntu.png?fit=800%2C442&ssl=1
[10]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/05/sysctl_configuration.jpg?fit=800%2C554&ssl=1
[11]: https://linuxhandbook.com/chmod-command/
[12]: https://itsfoss.com/find-which-kernel-version-is-running-in-ubuntu/
[13]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/05/grub_configuration-1.jpg?fit=800%2C565&ssl=1
[14]: https://i2.wp.com/itsfoss.com/wp-content/uploads/2019/05/ipv6_address_ubuntu-1.png?fit=800%2C517&ssl=1

119
sources/tech/20190522 Convert Markdown files to word processor docs using pandoc.md Normal file
View File

@ -0,0 +1,119 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Convert Markdown files to word processor docs using pandoc)
[#]: via: (https://opensource.com/article/19/5/convert-markdown-to-word-pandoc)
[#]: author: (Scott Nesbitt https://opensource.com/users/scottnesbitt/users/jason-van-gumster/users/kikofernandez)
Convert Markdown files to word processor docs using pandoc
======
Living that plaintext life? Here's how to create the word processor
documents people ask for without having to work in a word processor
yourself.
![][1]
If you live your life in [plaintext][2], there invariably comes a time when someone asks for a word processor document. I run into this issue frequently, especially at the Day JobTM. Although I've introduced one of the development teams I work with to a [Docs Like Code][3] workflow for writing and reviewing release notes, there are a small number of people who have no interest in GitHub or working with [Markdown][4]. They prefer documents formatted for a certain proprietary application.
The good news is that you're not stuck copying and pasting unformatted text into a word processor document. Using **[pandoc][5]** , you can quickly give people what they want. Let's take a look at how to convert a document from Markdown to a word processor format in [Linux][6] using **pandoc.** ​​​​
Note that **pandoc** is also available for a wide variety of operating systems, ranging from two flavors of BSD ([NetBSD][7] and [FreeBSD][8]) to Chrome OS, MacOS, and Windows.
### Converting basics
To begin, [install **pandoc**][9] on your computer. Then, crack open a console terminal window and navigate to the directory containing the file that you want to convert.
Type this command to create an ODT file (which you can open with a word processor like [LibreOffice Writer][10] or [AbiWord][11]):
**pandoc -t odt filename.md -o filename.odt**
Remember to replace **filename** with the file's actual name. And if you need to create a file for that other word processor (you know the one I mean), replace **odt** on the command line with **docx**. Here's what this article looks like when converted to an ODT file:
![Basic conversion results with pandoc.][12]
These results are serviceable, but a bit bland. Let's look at how to add a bit more style to the converted documents.
### Converting with style
**pandoc** has a nifty feature enabling you to specify a style template when converting a marked-up plaintext file to a word processor format. In this file, you can edit a small number of styles in the document, including those that control the look of paragraphs, headings, captions, titles and subtitles, a basic table, and hyperlinks.
Let's look at the possibilities.
#### Creating a template
In order to style your documents, you can't just use _any_ template. You need to generate what **pandoc** calls a _reference_ template, which is the template it uses when converting text files to word processor documents. To create this file, type the following in a terminal window:
**pandoc -o custom-reference.odt --print-default-data-file reference.odt**
This command creates a file called **custom-reference.odt**. If you're using that other word processor, change the references to **odt** on the command line to **docx**.
Open the template file in LibreOffice Writer, and then press **F11** to open LibreOffice Writer's **Styles** pane. Although the [pandoc manual][13] advises against making other changes to the file, I change the page size and add headers and footers when necessary.
#### Using the template
So, how do you use that template you just created? There are two ways to do this.
The easiest way is to drop the template in your **/home** directory's **.pandoc** folder—you might have to create the folder first if it doesn't exist. When it's time to convert a document, **pandoc** uses this template file. See the next section on how to choose from multiple templates if you need more than one.
The other way to use your template is to type this set of conversion options at the command line:
**pandoc -t odt file-name.md --reference-doc=path-to-your-file/reference.odt -o file-name.odt**
If you're wondering what a converted file looks like with a customized template, here's an example:
![A document converted using a pandoc style template.][14]
#### Choosing from multiple templates
Many people only need one **pandoc** template. Some people, however, need more than one.
At my day job, for example, I use several templates—one with a DRAFT watermark, one with a watermark stating FOR INTERNAL USE, and one for a document's final versions. Each type of document needs a different template.
If you have similar needs, start the same way you do for a single template, by creating the file **custom-reference.odt**. Rename the resulting file—for example, to **custom-reference-draft.odt** —then open it in LibreOffice Writer and modify the styles. Repeat this process for each template you need.
Next, copy the files into your **/home** directory. You can even put them in the **.pandoc** folder if you want to.
To select a specific template at conversion time, you'll need to run this command in a terminal:
**pandoc -t odt file-name.md --reference-doc=path-to-your-file/custom-template.odt -o file-name.odt**
Change **custom-template.odt** to your template file's name.
### Wrapping up
To avoid having to remember a set of options I don't regularly use, I cobbled together some simple, very lame one-line scripts that encapsulate the options for each template. For example, I run the script **todraft.sh** to create a word processor document using the template with a DRAFT watermark. You might want to do the same.
Here's an example of a script using the template containing a DRAFT watermark:
`pandoc -t odt $1.md -o $1.odt --reference-doc=~/Documents/pandoc-templates/custom-reference-draft.odt`
Using **pandoc** is a great way to provide documents in the format that people ask for, without having to give up the command line life. This tool doesn't just work with Markdown, either. What I've discussed in this article also allows you to create and convert documents between a wide variety of markup languages. See the **pandoc** site linked earlier for more details.
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/5/convert-markdown-to-word-pandoc
作者：[Scott Nesbitt][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/scottnesbitt/users/jason-van-gumster/users/kikofernandez
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/computer_keyboard_laptop_development_code_woman.png?itok=vbYz6jjb
[2]: https://plaintextproject.online/
[3]: https://www.docslikecode.com/
[4]: https://en.wikipedia.org/wiki/Markdown
[5]: https://pandoc.org/
[6]: /resources/linux
[7]: https://www.netbsd.org/
[8]: https://www.freebsd.org/
[9]: https://pandoc.org/installing.html
[10]: https://www.libreoffice.org/discover/writer/
[11]: https://www.abisource.com/
[12]: https://opensource.com/sites/default/files/uploads/pandoc-wp-basic-conversion_600_0.png (Basic conversion results with pandoc.)
[13]: https://pandoc.org/MANUAL.html
[14]: https://opensource.com/sites/default/files/uploads/pandoc-wp-conversion-with-tpl_600.png (A document converted using a pandoc style template.)

223
sources/tech/20190523 Hardware bootstrapping with Ansible.md Normal file
View File

@ -0,0 +1,223 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Hardware bootstrapping with Ansible)
[#]: via: (https://opensource.com/article/19/5/hardware-bootstrapping-ansible)
[#]: author: (Mark Phillips https://opensource.com/users/markp/users/feeble/users/markp)
Hardware bootstrapping with Ansible
======
![computer servers processing data][1]
At a recent [Ansible London Meetup][2], I got chatting with somebody about automated hardware builds. _"It's all cloud now!"_ I hear you say. Ah, but for many large organisations it's not—they still have massive data centres full of hardware. Almost regularly somebody pops up on our internal mail list and asks, *"can Ansible do hardware provisioning?" *Well yes, you can provision hardware with Ansible…
### Requirements
Bootstrapping hardware is mostly about network services. Before we do any operating system (OS) installing then, we must set up some services. We will need:
* DHCP
* PXE
* TFTP
* Operating system media
* Web server
### Setup
Besides the DHCP configuration, everything else in this article is handled by the Ansible plays included in [this repository][3].
#### DHCP server
I'm writing here on the assumption you can control your DHCP configuration. If you don't have access to your DHCP server, you'll need to ask the owner to set two options. DHCP option 67 needs to be set to **pxelinux.0** and **next-server** (which is option 66—but you may not need to know that; often a DHCP server will have a field/option for 'next server') needs to be set to the IP address of your TFTP server.
If you can own the DHCP server, I'd suggest using dnsmasq. It's small and simple. I will not cover configuring it here, but look at [the man page][4] and the **\--enable-tftp** option.
#### TFTP
The **next-server** setting for our DHCP server, above, will point to a machine serving [TFTP][5]. Here I've used a [CentOS Linux][6] virtual machine, as it only takes one package (syslinux-tftpboot) and a service to start to have TFTP up and running. We'll stick with the default path, **/var/lib/tftpboot**.
#### PXE
If you're not already familiar with PXE, you might like to take a quick look at [the Wikipedia page][7]. For this article I'll keep it short—we will serve some files over TFTP, which DHCP guides our hardware to.
You'll want **images/pxeboot/{initrd.img,vmlinuz}** from the OS distribution media for pxeboot. These need to be copied to **/var/lib/tftpboot/pxeboot**. The referenced Ansible plays **do not do this step, **so you need to copy them over yourself.
We'll also need to serve the OS installation files. There are two approaches to this: 1) install, via HTTP, from the internet or 2) install, again via HTTP, from a local server. For my testing, since I'm on a private LAN (and I guess you are too), the fastest installation method is the second. The easiest way to prepare this is to mount the DVD image and rsync the `images`, **`Packages` **and `repodata` directories to your webserver location. The referenced Ansible plays will install **httpd** but won't copy over these files, so don't forget to do that after running [the play][8]. For this article, we'll once again stick with defaults for simplicity—so files need to be copied to Apache's standard docroot, **/var/www/html**.
#### Directories
We should end up with directory structures like this:
##### PXE/TFTP
```
[root@c7 ~]# tree /var/lib/tftpboot/pxe{b*,l*cfg}
/var/lib/tftpboot/pxeboot
└── 6
├── initrd.img
└── vmlinuz
```
##### httpd
```
[root@c7 ~]# tree -d /var/www/html/
/var/www/html/
├── 6 -> centos/6
├── 7 -> centos/7
├── centos
│ ├── 6
│ │ └── os
│ │ └── x86_64
│ │ ├── images
│ │ │ └── pxeboot
│ │ ├── Packages
│ │ └── repodata
│ └── 7
│ └── os
│ └── x86_64
│ ├── images
│ │ └── pxeboot
│ ├── Packages
│ └── repodata
└── ks
```
You'll notice my web setup appears a little less simple than the words above! I've pasted my actual structure to give you some ideas. The hardware I'm using is really old, and even getting CentOS 7 to work was horrible (if you're interested, it's due to the lack of [cciss][9] drivers for the HP Smart Array controller—yes, [there is an answer][10], but it takes a lot of faffing to make work), so all examples are of CentOS 6. I also wanted a flexible setup that could install many versions. Here I've done that using symlinks—this arrangement will work just fine for RHEL too, for example. The basic structure is present though—note the images, Packages and repodata directories.
These paths relate directly to [the PXE menu][11] file we'll serve up and [the kickstart file][12] too.
#### If you don't have DHCP
If you can't manage your own DHCP server or the owners of your infrastructure can't help, there is another option. In the past, I've used [iPXE][13] to create a boot image that I've loaded as virtual media. A lot of out-of-band/lights-out-management (LOM) interfaces on modern hardware support this functionality. You can make a custom embedded PXE menu in seconds with iPXE. I won't cover that here, but if it turns out to be a problem for you, then drop me a line [on Twitter][14] and I'll look at doing a follow-up blog post if enough people request it.
### Installing hardware
We've got our structure in place now, and we can [kickstart][15] a server. Before we do, we have to add some configuration to the TFTP setup to enable a given piece of hardware to pick up the PXE boot menu.
It's here we come across a small chicken/egg problem. We need a host's MAC address to create a link to the specific piece of hardware we want to kickstart. If the hardware is already running and we can access it with Ansible, that's great—we have a way of finding out the boot interface MAC address via the setup module (see [the reinstall play][16]). If it's a new piece of tin, however, we need to get the MAC address and tell our setup what to do with it. This probably means some manual intervention—booting the server and looking at a screen or maybe getting the MAC from a manifest or such like. Whichever way you get hold of it, we can tell our play about it via the inventory.
Let's put a custom variable into our simple INI format [inventory file][17], but run a play to set up TFTP…
```
(pip)iMac:ansible-hw-bootstrap$ ansible-inventory --host hp.box
{
"ilo_ip": "192.168.1.68",
"ilo_password": "administrator"
}
(pip)iMac:ansible-hw-bootstrap$ ansible-playbook plays/install.yml
PLAY [kickstart] *******************************************************************************************************
TASK [Host inventory entry has a MAC address] **************************************************************************
failed: [ks.box] (item=hp.box) => {
"assertion": "hostvars[item]['mac'] is defined",
"changed": false,
"evaluated_to": false,
"item": "hp.box",
"msg": "Assertion failed"
}
PLAY RECAP *************************************************************************************************************
ks.box : ok=0 changed=0 unreachable=0 failed=1
```
Uh oh, play failed. It [contains a check][18] that the host we're about to install actually has a MAC address added. Let's fix that and run the play again…
```
(pip)iMac:ansible-hw-bootstrap$ ansible-inventory --host hp.box
{
"ilo_ip": "192.168.1.68",
"ilo_password": "administrator",
"mac": "00:AA:BB:CC:DD:EE"
}
(pip)iMac:ansible-hw-bootstrap$ ansible-playbook plays/install.yml
PLAY [kickstart] *******************************************************************************************************
TASK [Host inventory entry has a MAC address] **************************************************************************
ok: [ks.box] => (item=hp.box) => {
"changed": false,
"item": "hp.box",
"msg": "All assertions passed"
}
TASK [Set PXE menu to install] *****************************************************************************************
ok: [ks.box] => (item=hp.box)
TASK [Reboot target host for PXE boot] *********************************************************************************
skipping: [ks.box] => (item=hp.box)
PLAY RECAP *************************************************************************************************************
ks.box : ok=2 changed=0 unreachable=0 failed=0
```
That worked! What did it do? Looking at the pxelinux.cfg directory under our TFTP root, we can see a symlink…
```
[root@c7 pxelinux.cfg]# pwd
/var/lib/tftpboot/pxelinux.cfg
[root@c7 pxelinux.cfg]# l
total 12
drwxr-xr-x. 2 root root 65 May 13 14:23 ./
drwxr-xr-x. 4 root root 4096 May 2 22:13 ../
-r--r--r--. 1 root root 515 May 2 12:22 00README
lrwxrwxrwx. 1 root root 7 May 13 14:12 01-00-aa-bb-cc-dd-ee -> install
-rw-r--r--. 1 root root 682 May 2 22:07 install
```
The **install** file is symlinked to a file named after our MAC address. This is the key, useful piece. It will ensure our hardware with MAC address **00-aa-bb-cc-dd-ee** is served a PXE menu when it boots from its network card.
So let's boot our machine.
Usefully, Ansible has some [remote management modules][19]. We're working with an HP server here, so we can use the [hpilo_boot][20] module to save us from having to interact directly with the LOM web interface.
Let's run the reinstall play on a booted server…
The neat thing about the **hpilo_boot** module, you'll notice, is it sets the boot medium to be the network. When the installation completes, the server restarts and boots from its hard drive. The eagle-eyed amongst you will have spotted the critical problem with this—what happens if the server boots to its network card again? It will pick up the PXE menu and promptly reinstall itself. I would suggest removing the symlink as a "belt and braces" step then. I will leave that as an exercise for you, dear reader. Hint: I would make the new server do a 'phone home' on boot, to somewhere, which runs a clean-up job. Since you wouldn't need the console open, as I had here to demonstrate what's going on in the background, a 'phone home' job would also give a nice indication that the process completed. Ansible, [naturally][21]. Good luck!
If you've any thoughts or comments on this process, please let me know.
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/5/hardware-bootstrapping-ansible
作者：[Mark Phillips][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/markp/users/feeble/users/markp
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/server_data_system_admin.png?itok=q6HCfNQ8 (computer servers processing data)
[2]: https://www.meetup.com/Ansible-London/
[3]: https://github.com/phips/ansible-hw-bootstrap
[4]: http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html
[5]: https://en.m.wikipedia.org/wiki/Trivial_File_Transfer_Protocol
[6]: https://www.centos.org
[7]: https://en.m.wikipedia.org/wiki/Preboot_Execution_Environment
[8]: https://github.com/phips/ansible-hw-bootstrap/blob/master/plays/kickstart.yml
[9]: https://linux.die.net/man/4/cciss
[10]: https://serverfault.com/questions/611182/centos-7-x64-and-hp-proliant-dl360-g5-scsi-controller-compatibility
[11]: https://github.com/phips/ansible-hw-bootstrap/blob/master/roles/kickstart/templates/pxe_install.j2#L10
[12]: https://github.com/phips/ansible-hw-bootstrap/blob/master/roles/kickstart/templates/local6.ks.j2#L3
[13]: https://ipxe.org
[14]: https://twitter.com/thismarkp
[15]: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/installation_guide/ch-kickstart2
[16]: https://github.com/phips/ansible-hw-bootstrap/blob/master/plays/reinstall.yml
[17]: https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html
[18]: https://github.com/phips/ansible-hw-bootstrap/blob/master/plays/install.yml#L9
[19]: https://docs.ansible.com/ansible/latest/modules/list_of_remote_management_modules.html
[20]: https://docs.ansible.com/ansible/latest/modules/hpilo_boot_module.html#hpilo-boot-module
[21]: https://github.com/phips/ansible-demos/tree/master/roles/phone_home

235
sources/tech/20190523 Run your blog on GitHub Pages with Python.md Normal file
View File

@ -0,0 +1,235 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Run your blog on GitHub Pages with Python)
[#]: via: (https://opensource.com/article/19/5/run-your-blog-github-pages-python)
[#]: author: (Erik O'Shaughnessy https://opensource.com/users/jnyjny/users/jasperzanjani/users/jasperzanjani/users/jasperzanjani/users/jnyjny/users/jasperzanjani)
Run your blog on GitHub Pages with Python
======
Create a blog with Pelican, a Python-based blogging platform that works
well with GitHub.
![Raspberry Pi and Python][1]
[GitHub][2] is a hugely popular web service for source code control that uses [Git][3] to synchronize local files with copies kept on GitHub's servers so you can easily share and back up your work.
In addition to providing a user interface for code repositories, GitHub also enables users to [publish web pages][4] directly from a repository. The website generation package GitHub recommends is [Jekyll][5], written in Ruby. Since I'm a bigger fan of [Python][6], I prefer [Pelican][7], a Python-based blogging platform that works well with GitHub.
Pelican and Jekyll both transform content written in [Markdown][8] or [reStructuredText][9] into HTML to generate static websites, and both generators support themes that allow unlimited customization.
In this article, I'll describe how to install Pelican, set up your GitHub repository, run a quickstart helper, write some Markdown files, and publish your first page. I'll assume that you have a [GitHub account][10], are comfortable with [basic Git commands][11], and want to publish a blog using Pelican.
### Install Pelican and create the repo
First things first, Pelican (and **ghp-import** ) must be installed on your local machine. This is super easy with [pip][12], the Python package installation tool (you have pip right?):
```
`$ pip install pelican ghp-import`
```
Next, open a browser and create a new repository on GitHub for your sweet new blog. Name it as follows (substituting your GitHub username for <username> here and throughout this tutorial):
```
`https://GitHub.com/username/username.github.io`
```
Leave it empty; we will fill it with compelling blog content in a moment.
Using a command line (you command line right?), clone your empty Git repository to your local machine:
```
$ git clone <https://GitHub.com/username/username.github.io> blog
$ cd blog
```
### That one weird trick…
Here's a not-super-obvious trick about publishing web content on GitHub. For user pages (pages hosted in repos named _username.github.io_ ), the content is served from the **master** branch.
I strongly prefer not to keep all the Pelican configuration files and raw Markdown files in **master** , rather just the web content. So I keep the Pelican configuration and the raw content in a separate branch I like to call **content**. (You can call it whatever you want, but the following instructions will call it **content**.) I like this structure since I can throw away all the files in **master** and re-populate it with the **content** branch.
```
$ git checkout -b content
Switched to a new branch 'content'
```
### Configure Pelican
Now it's time for content configuration. Pelican provides a great initialization tool called **pelican-quickstart** that will ask you a series of questions about your blog.
```
$ pelican-quickstart
Welcome to pelican-quickstart v3.7.1.
This script will help you create a new Pelican-based website.
Please answer the following questions so this script can generate the files
needed by Pelican.
> Where do you want to create your new web site? [.]
> What will be the title of this web site? Super blog
> Who will be the author of this web site? username
> What will be the default language of this web site? [en]
> Do you want to specify a URL prefix? e.g., <http://example.com> (Y/n) n
> Do you want to enable article pagination? (Y/n)
> How many articles per page do you want? [10]
> What is your time zone? [Europe/Paris] US/Central
> Do you want to generate a Fabfile/Makefile to automate generation and publishing? (Y/n) y
> Do you want an auto-reload & simpleHTTP script to assist with theme and site development? (Y/n) y
> Do you want to upload your website using FTP? (y/N) n
> Do you want to upload your website using SSH? (y/N) n
> Do you want to upload your website using Dropbox? (y/N) n
> Do you want to upload your website using S3? (y/N) n
> Do you want to upload your website using Rackspace Cloud Files? (y/N) n
> Do you want to upload your website using GitHub Pages? (y/N) y
> Is this your personal page (username.github.io)? (y/N) y
Done. Your new project is available at /Users/username/blog
```
You can take the defaults on every question except:
* Website title, which should be unique and special
* Website author, which can be a personal username or your full name
* Time zone, which may not be in Paris
* Upload to GitHub Pages, which is a "y" in our case
After answering all the questions, Pelican leaves the following in the current directory:
```
$ ls
Makefile content/ develop_server.sh*
fabfile.py output/ pelicanconf.py
publishconf.py
```
You can check out the [Pelican docs][13] to find out how to use those files, but we're all about getting things done _right now_. No, I haven't read the docs yet either.
### Forge on
Add all the Pelican-generated files to the **content** branch of the local Git repo, commit the changes, and push the local changes to the remote repo hosted on GitHub by entering:
```
$ git add .
$ git commit -m 'initial pelican commit to content'
$ git push origin content
```
This isn't super exciting, but it will be handy if we need to revert edits to one of these files.
### Finally getting somewhere
OK, now you can get bloggy! All of your blog posts, photos, images, PDFs, etc., will live in the **content** directory, which is initially empty. To begin creating a first post and an About page with a photo, enter:
```
$ cd content
$ mkdir pages images
$ cp /Users/username/SecretStash/HotPhotoOfMe.jpg images
$ touch first-post.md
$ touch pages/about.md
```
Next, open the empty file **first-post.md** in your favorite text editor and add the following:
```
title: First Post on My Sweet New Blog
date: <today's date>
author: Your Name Here
# I am On My Way To Internet Fame and Fortune!
This is my first post on my new blog. While not super informative it
should convey my sense of excitement and eagerness to engage with you,
the reader!
```
The first three lines contain metadata that Pelican uses to organize things. There are lots of different metadata you can put there; again, the docs are your best bet for learning more about the options.
Now, open the empty file **pages/about.md** and add this text:
```
title: About
date: <today's date>
![So Schmexy][my_sweet_photo]
Hi, I am <username> and I wrote this epic collection of Interweb
wisdom. In days of yore, much of this would have been deemed sorcery
and I would probably have been burned at the stake.
😆
[my_sweet_photo]: {filename}/images/HotPhotoOfMe.jpg
```
You now have three new pieces of web content in your content directory. Of the content branch. That's a lot of content.
### Publish
Don't worry; the payoff is coming!
All that's left to do is:
* Run Pelican to generate the static HTML files in **output** : [code]`$ pelican content -o output -s publishconf.py`
```
* Use **ghp-import** to add the contents of the **output** directory to the **master** branch: [code]`$ ghp-import -m "Generate Pelican site" --no-jekyll -b master output`
```
* Push the local master branch to the remote repo: [code]`$ git push origin master`
```
* Commit and push the new content to the **content** branch: [code] $ git add content
$ git commit -m 'added a first post, a photo and an about page'
$ git push origin content
```
### OMG, I did it!
Now the exciting part is here, when you get to view what you've published for everyone to see! Open your browser and enter:
```
`https://username.github.io`
```
Congratulations on your new blog, self-published on GitHub! You can follow this pattern whenever you want to add more pages or articles. Happy blogging.
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/5/run-your-blog-github-pages-python
作者：[Erik O'Shaughnessy][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/jnyjny/users/jasperzanjani/users/jasperzanjani/users/jasperzanjani/users/jnyjny/users/jasperzanjani
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/getting_started_with_python.png?itok=MFEKm3gl (Raspberry Pi and Python)
[2]: https://github.com/
[3]: https://git-scm.com
[4]: https://help.github.com/en/categories/github-pages-basics
[5]: https://jekyllrb.com
[6]: https://python.org
[7]: https://blog.getpelican.com
[8]: https://guides.github.com/features/mastering-markdown
[9]: http://docutils.sourceforge.net/docs/user/rst/quickref.html
[10]: https://github.com/join?source=header-home
[11]: https://git-scm.com/docs
[12]: https://pip.pypa.io/en/stable/
[13]: https://docs.getpelican.com

222
sources/tech/20190523 Testing a Go-based S2I builder image.md Normal file
View File

@ -0,0 +1,222 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Testing a Go-based S2I builder image)
[#]: via: (https://opensource.com/article/19/5/source-image-golang-part-3)
[#]: author: (Chris Collins https://opensource.com/users/clcollins)
Testing a Go-based S2I builder image
======
In the third article in this series on Source-to-Image for Golang
applications, build your application image and take it out for a spin.
![gopher illustrations][1]
In the first two articles in this series, we explored the general [requirements of a Source To Image (S2I) system][2] and [prepared an environment][3] specifically for a Go (Golang) application. Now let's give it a spin.
### Building the builder image
Once the Dockerfile and Source-to-Image (S2I) scripts are ready, the Golang builder image can be created with the **docker build** command:
```
`docker build -t golang-builder .`
```
This will produce a builder image named **golang-builder** with the context of our current directory.
### Building the application image
The golang-builder image is not much use without an application to build. For this exercise, we will build a simple **hello-world** application.
#### GoHelloWorld
Let's meet our test app, [GoHelloWorld][4]. Download the latest [version of Go][5] if you want to follow along. There are two important (for this exercise) files in this repository:
```
// goHelloWorld.go
package main
import "fmt"
func main() {
fmt.Println("Hello World!")
}
```
This is a very basic app, but it will work fine for testing the builder image. We also have a basic test for GoHelloWorld:
```
// goHelloWorld_test.go
package main
import "testing"
func TestMain(t *testing.T) {
t.Log("Hello World!")
}
```
#### Build the application image
Building the application image entails running the **s2i build** command with arguments for the repository containing the code to build (or **.** to build with code from the current directory), the name of the builder image to use, and the name of the resulting application image to create.
```
`$ s2i build https://github.com/clcollins/goHelloWorld.git golang-builder go-hello-world`
```
To build from a local directory on a filesystem, replace the Git URL with a period to represent the current directory. For example:
```
`$ s2i build . golang-builder go-hello-world`
```
_Note:_ If a Git repository is initialized in the current directory, S2I will fetch the code from the repository URL rather than using the local code. This results in local, uncommitted changes not being used when building the image (if you're unfamiliar with what I mean by "uncommitted changes," brush up on your [Git terminology over here][6]). Directories that are not Git-initialized repositories behave as expected.
#### Run the application image
Once the application image is built, it can be tested by running it with the **Docker** command. Source-to-Image has replaced the **CMD** in the image with the run script created earlier so it will execute the **/go/src/app/app** binary created during the build process:
```
$ docker run go-hello-world
Hello World!
```
Success! We now have a compiled Go application inside a Docker image created by passing the contents of a Git repo to S2I and without needing a special Dockerfile for our application.
The application image we just built includes not only the application, but also its source code, test code, the S2I scripts, Golang libraries, and _much of the Debian Linux distribution_ (because the Golang image is based on the Debian base image). The resulting image is not small:
```
$ docker images | grep go-hello-world
go-hello-world latest 75a70c79a12f 4 minutes ago 789 MB
```
For applications written in languages that are interpreted at runtime and depend on linked libraries, like Ruby or Python, having all the source code and operating system are necessary to run. The build images will be pretty large as a result, but at least we know it will be able to run. With these languages, we could stop here with our S2I builds.
There is the option, however, to more explicitly define the production requirements for the application.
Since the resulting application image would be the same image that would run the production app, I want to assure the required ports, volumes, and environment variables are added to the Dockerfile for the builder image. By writing these in a declarative way, our app is closer to the [Twelve-Factor App][7] recommended practice. For example, if we were to use the builder image to create application images for a Ruby on Rails application running [Puma][8], we would want to open a port to access the webserver. We should add the line **PORT 3000** in the builder Dockerfile so it can be inherited by all the images generated from it.
But for the Go app, we can do better.
### Build a runtime image
Since our builder image created a statically compiled Go binary with our application, we can create a final "runtime" image containing _only_ the binary and none of the other cruft.
Once the application image is created, the compiled GoHelloWorld app can be extracted and put into a new, empty image using the save-artifacts script.
#### Runtime files
Only the application binary and a Dockerfile are required to create the runtime image.
##### Application binary
Inside of the application image, the save-artifacts script is written to stream a tar archive of the app binary to stdout. We can check the files included in the tar archive created by save-artifacts with the **-vt** flags for tar:
```
$ docker run go-hello-world /usr/libexec/s2i/save-artifacts | tar -tvf -
-rwxr-xr-x 1001/root 1997502 2019-05-03 18:20 app
```
If this results in errors along the lines of "This does not appear to be a tar archive," the save-artifacts script is probably outputting other data in addition to the tar stream, as mentioned above. We must make sure to suppress all output other than the tar stream.
If everything looks OK, we can use **save-artifacts** to copy the binary out of the application image:
```
`$ docker run go-hello-world /usr/libexec/s2i/save-artifacts | tar -xf -`
```
This will copy the app file into the current directory, ready to be added to its own image.
##### Dockerfile
The Dockerfile is extremely simple, with only three lines. The **FROM scratch** source denotes that it uses an empty, blank parent image. The rest of the Dockerfile specifies copying the app binary into **/app** in the image and using that binary as the image **ENTRYPOINT** :
```
FROM scratch
COPY app /app
ENTRYPOINT ["/app"]
```
Save this Dockerfile as **Dockerfile-runtime**.
Why **ENTRYPOINT** and not **CMD**? We could do either, but since there is nothing else in the image (no filesystem, no shell), we couldn't run anything else anyway.
#### Building the runtime image
With the Dockerfile and binary ready to go, we can build the new runtime image:
```
`$ docker build -f Dockerfile-runtime -t go-hello-world:slim .`
```
The new runtime image is considerably smaller—just 2MB!
```
$ docker images | grep -e 'go-hello-world *slim'
go-hello-world slim 4bd091c43816 3 minutes ago 2 MB
```
We can test that it still works as expected with **docker run** :
```
$ docker run go-hello-world:slim
Hello World!
```
### Bootstrapping s2i with s2i create
While we hand-created all the S2I files in this example, the **s2i** command has a sub-command to help scaffold all the files we might need for a Source-to-Image build: **s2i create**.
Using the **s2i create** command, we can generate a new project, creatively named **go-hello-world-2** in the **./ghw2** directory:
```
$ s2i create go-hello-world-2 ./ghw2
$ ls ./ghw2/
Dockerfile Makefile README.md s2i test
```
The **create** sub-command creates a placeholder Dockerfile, a README.md with information about how to use Source-to-Image, some example S2I scripts, a basic test framework, and a Makefile. The Makefile is a great way to automate building and testing the Source-to-Image builder image. Out of the box, running **make** will build our image, and it can be extended to do more. For example, we could add steps to build a base application image, run tests, or generate a runtime Dockerfile.
### Conclusion
In this tutorial, we have learned how to use Source-to-Image to build a custom Golang builder image, create an application image using **s2i build** , and extract the application binary to create a super-slim runtime image.
In a future extension to this series, I would like to look at how to use the builder image we created with [OKD][9] to automatically deploy our Golang apps with **buildConfigs** , **imageStreams** , and **deploymentConfigs**. Please let me know in the comments if you are interested in me continuing the series, and thanks for reading.
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/5/source-image-golang-part-3
作者：[Chris Collins][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/clcollins
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/go-golang.png?itok=OAW9BXny (gopher illustrations)
[2]: https://opensource.com/article/19/5/source-image-golang-part-1
[3]: https://opensource.com/article/19/5/source-image-golang-part-2
[4]: https://github.com/clcollins/goHelloWorld.git
[5]: https://golang.org/doc/install
[6]: /article/19/2/git-terminology
[7]: https://12factor.net/
[8]: http://puma.io/
[9]: https://okd.io/

96
sources/tech/20190524 Choosing the right model for maintaining and enhancing your IoT project.md Normal file
View File

@ -0,0 +1,96 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Choosing the right model for maintaining and enhancing your IoT project)
[#]: via: (https://opensource.com/article/19/5/model-choose-embedded-iot-development)
[#]: author: (Drew Moseley https://opensource.com/users/drewmoseley)
Choosing the right model for maintaining and enhancing your IoT project
======
Learn more about these two models: Centralized Golden Master and
Distributed Build System
![][1]
In today's connected embedded device market, driven by the [Internet of things (IoT)][2], a large share of devices in development are based on Linux of one form or another. The prevalence of low-cost boards with ready-made Linux distributions is a key driver in this. Acquiring hardware, building your custom code, connecting the devices to other hardware peripherals and the internet as well as device management using commercial cloud providers has never been easier. A developer or development team can quickly prototype a new application and get the devices in the hands of potential users. This is a good thing and results in many interesting new applications, as well as many questionable ones.
When planning a system design for beyond the prototyping phase, things get a little more complex. In this post, we want to consider mechanisms for developing and maintaining your base [operating system (OS) image][3]. There are many tools to help with this but we won't be discussing individual tools; of interest here is the underlying model for maintaining and enhancing this image and how it will make your life better or worse.
There are two primary models for generating these images:
1. Centralized Golden Master
2. Distributed Build System
These categories mirror the driving models for [Source Code Management (SCM)][4] systems, and many of the arguments regarding centralized vs. distributed are applicable when discussing OS images.
### Centralized Golden Master
Hobbyist and maker projects primarily use the Centralized Golden Master method of creating and maintaining application images. This fact gives this model the benefit of speed and familiarity, allowing developers to quickly set up such a system and get it running. The speed comes from the fact that many device manufacturers provide canned images for their off-the-shelf hardware. For example, boards from such families as the [BeagleBone][5] and [Raspberry Pi][6] offer ready-to-use OS images and [flashing][7]. Relying on these images means having your system up and running in just a few mouse clicks. The familiarity is due to the fact that these images are generally based on a desktop distro many device developers have already used, such as [Debian][8]. Years of using Linux can then directly transfer to the embedded design, including the fact that the packaging utilities remain largely the same, and it is simple for designers to get the extra software packages they need.
There are a few downsides of such an approach. The first is that the [golden master image][9] is generally a choke point, resulting in lost developer productivity after the prototyping stage since everyone must wait for their turn to access the latest image and make their changes. In the SCM realm, this practice is equivalent to a centralized system with individual [file locking][10]. Only the developer with the lock can work on any given file.
![Development flow with the Centralized Golden Master model.][11]
The second downside with this approach is image reproducibility. This issue is usually managed by manually logging into the target systems, installing packages using the native package manager, configuring applications and dot files, and then modifying the system configuration files in place. Once this process is completed, the disk is imaged using the **dd** utility, or an equivalent, and then distributed.
Again, this approach creates a minefield of potential issues. For example, network-based package feeds may cease to exist, and the base software provided by the vendor image may change. Scripting can help mitigate these issues. However, these scripts tend to be fragile and break when changes are made to configuration file formats or the vendor's base software packages.
The final issue that arises with this development model is reliance on third parties. If the hardware vendor's image changes don't work for your design, you may need to invest significant time to adapt. To make matters even more complicated, as mentioned before, the hardware vendors often based their images on an upstream project such as Debian or Ubuntu. This situation introduces even more third parties who can affect your design.
### Distributed Build System
This method of creating and maintaining an image for your application relies on the generation of target images separate from the target hardware. The developer workflow here is similar to standard software development using an SCM system; the image is fully buildable by tooling and each developer can work independently. Changes to the system are made via edits to metadata files (scripting, recipes, configuration files, etc) and then the tooling is rerun to generate an updated image. These metadata files are then managed using an SCM system. Individual developers can merge the latest changes into their working copies to produce their development images. In this case, no golden master image is needed and developers can avoid the associated bottleneck.
Release images are then produced by a build system using standard SCM techniques to pull changes from all the developers.
![Development flow with the Distributed Build System model.][12]
Working in this fashion allows the size of your development team to increase without reducing productivity of individual developers. All engineers can work independently of the others. Additionally, this build setup ensures that your builds can be reproduced. Using standard SCM workflows can ensure that, at any future time, you can regenerate a specific build allowing for long term maintenance, even if upstream providers are no longer available. Similar to working with distributed SCM tools however, there is additional policy that needs to be in place to enable reproducible, release candidate images. Individual developers have their own copies of the source and can build their own test images but for a proper release engineering effort, development teams will need to establish merging and branching standards and ensure that all changes targeted for release eventually get merged into a well-defined branch. Many upstream projects already have well-defined processes for this kind of release strategy (for instance, using *-stable and *-next branches).
The primary downside of this approach is the lack of familiarity. For example, adding a package to the image normally requires creating a recipe of some kind and then updating the definitions so that the package binaries are included in the image. This is very different from running apt while logged into a running system. The learning curve of these systems can be daunting but the results are more predictable and scalable and are likely a better choice when considering a design for a product that will be mass produced.
Dedicated build systems such as [OpenEmbedded][13] and [Buildroot][14] use this model as do distro packaging tools such as [debootstrap][15] and [multistrap][16]. Newer tools such as [Isar][17], [debos][18], and [ELBE][19] also use this basic model. Choices abound, and it is worth the investment to learn one or more of these packages for your designs. The long term maintainability and reproducibility of these systems will reduce risk in your design by allowing you to generate reproducible builds, track all the source code, and remove your dependency on third-party providers continued existence.
#### Conclusion
To be clear, the distributed model does suffer some of the same issues as mentioned for the Golden Master Model; especially the reliance on third parties. This is a consequence of using systems designed by others and cannot be completely avoided unless you choose a completely roll-your-own approach which comes with a significant cost in development and maintenance.
For prototyping and proof-of-concept level design, and a team of just a few developers, the Golden Master Model may well be the right choice given restrictions in time and budget that are present at this stage of development. For low volume, high touch designs, this may be an acceptable trade-off for production use.
For general production use, the benefits in terms of team size scalability, image reproducibility and developer productivity greatly outweigh the learning curve and overhead of systems implementing the distributed model. Support from board and chip vendors is also widely available in these systems reducing the upfront costs of developing with them. For your next product, I strongly recommend starting the design with a serious consideration of the model being used to generate the base OS image. If you choose to prototype with the golden master model with the intention of migrating to the distributed model, make sure to build sufficient time in your schedule for this effort; the estimates will vary widely depending on the specific tooling you choose as well as the scope of the requirements and the out-of-the-box availability of software packages your code relies on.
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/5/model-choose-embedded-iot-development
作者：[Drew Moseley][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/drewmoseley
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/LAW-Internet_construction_9401467_520x292_0512_dc.png?itok=RPkPPtDe
[2]: https://en.wikipedia.org/wiki/Internet_of_things
[3]: https://en.wikipedia.org/wiki/System_image
[4]: https://en.wikipedia.org/wiki/Version_control
[5]: http://beagleboard.org/
[6]: https://www.raspberrypi.org/
[7]: https://en.wikipedia.org/wiki/Flash_memory
[8]: https://www.debian.org/
[9]: https://en.wikipedia.org/wiki/Software_release_life_cycle#RTM
[10]: https://en.wikipedia.org/wiki/File_locking
[11]: https://opensource.com/sites/default/files/uploads/cgm1_500.png (Development flow with the Centralized Golden Master model.)
[12]: https://opensource.com/sites/default/files/uploads/cgm2_500.png (Development flow with the Distributed Build System model.)
[13]: https://www.openembedded.org/
[14]: https://buildroot.org/
[15]: https://wiki.debian.org/Debootstrap
[16]: https://wiki.debian.org/Multistrap
[17]: https://github.com/ilbers/isar
[18]: https://github.com/go-debos/debos
[19]: https://elbe-rfs.org/

104
sources/tech/20190524 Dual booting Windows and Linux using UEFI.md Normal file
View File

@ -0,0 +1,104 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Dual booting Windows and Linux using UEFI)
[#]: via: (https://opensource.com/article/19/5/dual-booting-windows-linux-uefi)
[#]: author: (Alan Formy-Duval https://opensource.com/users/alanfdoss/users/ckrzen)
Dual booting Windows and Linux using UEFI
======
A quick rundown of setting up Linux and Windows to dual boot on the same
machine, using the Unified Extensible Firmware Interface (UEFI).
![Linux keys on the keyboard for a desktop computer][1]
Rather than doing a step-by-step how-to guide to configuring your system to dual boot, I’ll highlight the important points. As an example, I will refer to my new laptop that I purchased a few months ago. I first installed [Ubuntu Linux][2] onto the entire hard drive, which destroyed the pre-installed [Windows 10][3] installation. After a few months, I decided to install a different Linux distribution, and so also decided to re-install Windows 10 alongside [Fedora Linux][4] in a dual boot configuration. I’ll highlight some essential facts to get started.
### Firmware
Dual booting is not just a matter of software. Or, it is, but it involves changing your firmware, which among other things tells your machine how to begin the boot process. Here are some firmware-related issues to keep in mind.
#### UEFI vs. BIOS
Before attempting to install, make sure your firmware configuration is optimal. Most computers sold today have a new type of firmware known as [Unified Extensible Firmware Interface (UEFI)][5], which has pretty much replaced the other firmware known as [Basic Input Output System (BIOS)][6], which is often included through the mode many providers call Legacy Boot.
I had no need for BIOS, so I chose UEFI mode.
#### Secure Boot
One other important setting is Secure Boot. This feature detects whether the boot path has been tampered with, and stops unapproved operating systems from booting. For now, I disabled this option to ensure that I could install Fedora Linux. According to the Fedora Project Wiki [Features/Secure Boot ][7] Fedora Linux will work with it enabled. This may be different for other Linux distributions —I plan to revisit this setting in the future.
In short, if you find that you cannot install your Linux OS with this setting active, disable Secure Boot and try again.
### Partitioning the boot drive
If you choose to dual boot and have both operating systems on the same drive, you have to break it into partitions. Even if you dual boot using two different drives, most Linux installations are best broken into a few basic partitions for a variety of reasons. Here are some options to consider.
#### GPT vs MBR
If you decide to manually partition your boot drive in advance, I recommend using the [GUID Partition Table (GPT)][8] rather than the older [Master Boot Record (MBR)][9]. Among the reasons for this change, there are two specific limitations of MBR that GPT doesn’t have:
* MBR can hold up to 15 partitions, while GPT can hold up to 128.
* MBR only supports up to 2 terabytes, while GPT uses 64-bit addresses which allows it to support disks up to 8 million terabytes.
If you have shopped for hard drives recently, then you know that many of today’s drives exceed the 2 terabyte limit.
#### The EFI system partition
If you are doing a fresh installation or using a new drive, there are probably no partitions to begin with. In this case, the OS installer will create the first one, which is the [EFI System Partition (ESP)][10]. If you choose to manually partition your drive using a tool such as [gdisk][11], you will need to create this partition with several parameters. Based on the existing ESP, I set the size to around 500MB and assigned it the ef00 (EFI System) partition type. The UEFI specification requires the format to be FAT32/msdos, most likely because it is supportable by a wide range of operating systems.
![Partitions][12]
### Operating System Installation
Once you accomplish the first two tasks, you can install your operating systems. While I focus on Windows 10 and Fedora Linux here, the process is fairly similar when installing other combinations as well.
#### Windows 10
I started the Windows 10 installation and created a 20 Gigabyte Windows partition. Since I had previously installed Linux on my laptop, the drive had an ESP, which I chose to keep. I deleted all existing Linux and swap partitions to start fresh, and then started my Windows installation. The Windows installer automatically created another small partition—16 Megabytes—called the [Microsoft Reserved Partition (MSR)][13]. Roughly 400 Gigabytes of unallocated space remained on the 512GB boot drive once this was finished.
I then proceeded with and completed the Windows 10 installation process. I then rebooted into Windows to make sure it was working, created my user account, set up wi-fi, and completed other tasks that need to be done on a first-time OS installation.
#### Fedora Linux
I next moved to install Linux. I started the process, and when it reached the disk configuration steps, I made sure not to change the Windows NTFS and MSR partitions. I also did not change the EPS, but I did set its mount point to **/boot/efi**. I then created the usual ext4 formatted partitions, **/** (root), **/boot** , and **/home**. The last partition I created was Linux **swap**.
As with Windows, I continued and completed the Linux installation, and then rebooted. To my delight, at boot time the [GRand][14] [Unified Boot Loader (GRUB)][14] menu provided the choice to select either Windows or Linux, which meant I did not have to do any additional configuration. I selected Linux and completed the usual steps such as creating my user account.
### Conclusion
Overall, the process was painless. In past years, there has been some difficulty navigating the changes from UEFI to BIOS, plus the introduction of features such as Secure Boot. I believe that we have now made it past these hurdles and can reliably set up multi-boot systems.
I don’t miss the [Linux LOader (LILO)][15] anymore!
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/5/dual-booting-windows-linux-uefi
作者：[Alan Formy-Duval][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/alanfdoss/users/ckrzen
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/linux_keyboard_desktop.png?itok=I2nGw78_ (Linux keys on the keyboard for a desktop computer)
[2]: https://www.ubuntu.com
[3]: https://www.microsoft.com/en-us/windows
[4]: https://getfedora.org
[5]: https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface
[6]: https://en.wikipedia.org/wiki/BIOS
[7]: https://fedoraproject.org/wiki/Features/SecureBoot
[8]: https://en.wikipedia.org/wiki/GUID_Partition_Table
[9]: https://en.wikipedia.org/wiki/Master_boot_record
[10]: https://en.wikipedia.org/wiki/EFI_system_partition
[11]: https://sourceforge.net/projects/gptfdisk/
[12]: /sites/default/files/u216961/gdisk_screenshot_s.png
[13]: https://en.wikipedia.org/wiki/Microsoft_Reserved_Partition
[14]: https://en.wikipedia.org/wiki/GNU_GRUB
[15]: https://en.wikipedia.org/wiki/LILO_(boot_loader)

129
sources/tech/20190525 4 Ways to Run Linux Commands in Windows.md Normal file
View File

@ -0,0 +1,129 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (4 Ways to Run Linux Commands in Windows)
[#]: via: (https://itsfoss.com/run-linux-commands-in-windows/)
[#]: author: (Abhishek Prakash https://itsfoss.com/author/abhishek/)
4 Ways to Run Linux Commands in Windows
======
_**Brief: Want to use Linux commands but don’t want to leave Windows? Here are several ways to run Linux bash commands in Windows.**_
If you are learning Shell scripting probably as a part of your course curriculum, you need to use Linux commands to practice the commands and scripting.
Your school lab might have Linux installed but personally you don’t have a [Linux laptop][1] but the regular Windows computer like everyone else. Your homework needs to run Linux commands and you wonder how to run Bash commands and scripts on Windows.
You can [install Linux alongside Windows in dual boot mode][2]. This method allows you to choose either Linux or Windows when you start your computer. But taking all the trouble to mess with partitions for the sole purpose of running Linux command may not be for everyone.
You can also [use Linux terminals online][3] but your work won’t be saved here.
The good news is that there are several ways you can run Linux commands inside Windows, like any regular application. Isn’t it cool?
### Using Linux commands inside Windows
![][4]
As an ardent Linux user and promoter, I would like to see more and more people using ‘real’ Linux but I understand that at times, that’s not the priority. If you are just looking to practice Linux to pass your exams, you can use one of these methods for running Bash commands on Windows.
#### 1\. Use Linux Bash Shell on Windows 10
Did you know that you can run a Linux distribution inside Windows 10? The [Windows Subsystem for Linux (WSL)][5] allows you to run Linux inside Windows. The upcoming version of WSL will be using the real Linux kernel inside Windows.
This WSL, also called Bash on Windows, gives you a Linux distribution in command line mode running as a regular Windows application. Don’t be scared with the command line mode because your purpose is to run Linux commands. That’s all you need.
![Ubuntu Linux inside Windows][6]
You can find some popular Linux distributions like Ubuntu, Kali Linux, openSUSE etc in Windows Store. You just have to download and install it like any other Windows application. Once installed, you can run all the Linux commands you want.
[][7]
Suggested read 6 Non-Ubuntu Linux Distributions For Beginners
![Linux distributions in Windows 10 Store][8]
Please refer to this tutorial about [installing Linux bash shell on Windows][9].
#### 2\. Use Git Bash to run Bash commands on Windows
You probably know what [Git][10] is. It’s a version control system developed by [Linux creator Linus Torvalds][11].
[Git for Windows][12] is a set of tools that allows you to use Git in both command line and graphical interfaces. One of the tools included in Git for Windows is Git Bash.
Git Bash application provides and emulation layer for Git command line. Apart from Git commands, Git Bash also supports many Bash utilities such as ssh, scp, cat, find etc.
![Git Bash][13]
In other words, you can run many common Linux/Bash commands using the Git Bash application.
You can install Git Bash in Windows by downloading and installing the Git for Windows tool for free from its website.
[Download Git for Windows][12]
#### 3\. Using Linux commands in Windows with Cygwin
If you want to run Linux commands in Windows, Cygwin is a recommended tool. Cygwin was created in 1995 to provide a POSIX-compatible environment that runs natively on Windows. Cygwin is a free and open source software maintained by Red Hat employees and many other volunteers.
For two decades, Windows users use Cygwin for running and practicing Linux/Bash commands. Even I used Cygwin to learn Linux commands more than a decade ago.
![Cygwin | Image Credit][14]
You can download Cygwin from its official website below. I also advise you to refer to this [Cygwin cheat sheet][15] to get started with it.
[Download Cygwin][16]
#### 4\. Use Linux in virtual machine
Another way is to use a virtualization software and install Linux in it. This way, you install a Linux distribution (with graphical interface) inside Windows and run it like a regular Windows application.
This method requires that your system has a good amount of RAM, at least 4 GB but better if you have over 8 GB of RAM. The good thing here is that you get the real feel of using a desktop Linux. If you like the interface, you may later decide to [switch to Linux][17] completely.
![Ubuntu Running in Virtual Machine Inside Windows][18]
There are two popular tools for creating virtual machines on Windows, Oracle VirtualBox and VMware Workstation Player. You can use either of the two. Personally, I prefer VirtualBox.
[][19]
Suggested read 9 Simple Ways To Free Up Space On Ubuntu and Linux Mint
You can follow [this tutorial to learn how to install Linux in VirtualBox][20].
**Conclusion**
The best way to run Linux commands is to use Linux. When installing Linux is not an option, these tools allow you to run Linux commands on Windows. Give them a try and see which method is best suited for you.
--------------------------------------------------------------------------------
via: https://itsfoss.com/run-linux-commands-in-windows/
作者：[Abhishek Prakash][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://itsfoss.com/author/abhishek/
[b]: https://github.com/lujun9972
[1]: https://itsfoss.com/get-linux-laptops/
[2]: https://itsfoss.com/guide-install-linux-mint-16-dual-boot-windows/
[3]: https://itsfoss.com/online-linux-terminals/
[4]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/05/run-linux-commands-in-windows.png?resize=800%2C450&ssl=1
[5]: https://itsfoss.com/bash-on-windows/
[6]: https://i2.wp.com/itsfoss.com/wp-content/uploads/2016/08/install-ubuntu-windows-10-linux-subsystem-10.jpeg?resize=800%2C268&ssl=1
[7]: https://itsfoss.com/non-ubuntu-beginner-linux/
[8]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2016/08/install-ubuntu-windows-10-linux-subsystem-4.jpeg?resize=800%2C632&ssl=1
[9]: https://itsfoss.com/install-bash-on-windows/
[10]: https://itsfoss.com/basic-git-commands-cheat-sheet/
[11]: https://itsfoss.com/linus-torvalds-facts/
[12]: https://gitforwindows.org/
[13]: https://i2.wp.com/itsfoss.com/wp-content/uploads/2019/05/git-bash.png?ssl=1
[14]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/05/cygwin-shell.jpg?ssl=1
[15]: http://www.voxforge.org/home/docs/cygwin-cheat-sheet
[16]: https://www.cygwin.com/
[17]: https://itsfoss.com/reasons-switch-linux-windows-xp/
[18]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/05/ubuntu-running-in-virtual-machine-inside-windows.jpeg?resize=800%2C450&ssl=1
[19]: https://itsfoss.com/free-up-space-ubuntu-linux/
[20]: https://itsfoss.com/install-linux-in-virtualbox/

2
sources/tech/20190527 20- FFmpeg Commands For Beginners.md
View File

@ -1,5 +1,5 @@
[#]: collector: (lujun9972) [#]: collector: (lujun9972)
[#]: translator: ( ) [#]: translator: (robsean)
[#]: reviewer: ( ) [#]: reviewer: ( )
[#]: publisher: ( ) [#]: publisher: ( )
[#]: url: ( ) [#]: url: ( )

140
sources/tech/20190527 4 open source mobile apps for Nextcloud.md Normal file
View File

@ -0,0 +1,140 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (4 open source mobile apps for Nextcloud)
[#]: via: (https://opensource.com/article/19/5/mobile-apps-nextcloud)
[#]: author: (Scott Nesbitt https://opensource.com/users/scottnesbitt)
4 open source mobile apps for Nextcloud
======
Increase Nextcloud's value by turning it into an on-the-go information
hub.
![][1]
I've been using [Nextcloud][2] (and before that, ownCloud), an open source alternative to file syncing and storage services like Dropbox and Google Drive, for many years. It's been both reliable and useful, and it respects my privacy.
While Nextcloud is great at both syncing and storage, it's much more than a place to dump your files. Thanks to applications that you can fold into Nextcloud, it becomes more of an information hub than a storage space.
While I usually interact with Nextcloud using the desktop client or in a browser, I'm not always at my computer (or any computer that I trust). So it's important that I can work with Nextcloud using my [LineageOS][3]-powered smartphone or tablet.
To do that, I use several open source apps that work with Nextcloud. Let's take a look at four of them.
As you've probably guessed, this article looks at the Android version of those apps. I grabbed mine from [F-Droid][4], although you get them from other Android app markets. You might be able to get some or all of them from Apple's App Store if you're an iOS person.
### Working with files and folders
The obvious app to start with is the [Nextcloud sync client][5]. This little app links your phone or tablet to your Nextcloud account.
![Nextcloud mobile app][6]
Using the app, you can:
* Create folders
* Upload one or more files
* Sync files between your device and server
* Rename or remove files
* Make files available offline
You can also tap a file to view or edit it. If your device doesn't have an app that can open the file, then you're out of luck. You can still download it to your phone or tablet though.
### Reading news feeds
Remember all the whining that went on when Google pulled the plug on Google Reader in 2013? This despite Google giving users several months to find an alternative. And, yes, there are alternatives. One of them, believe it or not, is Nextcloud.
Nextcloud has a built-in RSS reader. All you need to do to get started is upload an [OPML][7] file containing your feeds or manually add a site's RSS feed to Nextcloud.
Going mobile is easy, too, with the Nextcloud [News Reader app][8].
![Nextcloud News app][9]
Unless you configure the app to sync when you start it up, you'll need to swipe down from the top of the app to load updates to your feeds. Depending on how many feeds you have, and how many unread items are in those feeds, syncing takes anywhere from a few seconds to half a minute.
From there, tap an item to read it in the News app.
![Nextcloud News app][10]
You can also add feeds or open what you're reading in your device's default web browser.
### Reading and writing notes
I don't use Nextcloud's [Notes][11] app all that often (I'm more of a [Standard Notes][12] person). That said, you might find the Notes app comes in handy.
How? By giving you a lightweight way to take [plain text][13] notes on your mobile device. The Notes app syncs any notes you have in your Nextcloud account and displays them in chronological order—newest or last-edited notes first.
![Nextcloud Notes app][14]
Tap a note to read or edit it. You can also create a note by tapping the **+** button, then typing what you need to type.
![Nextcloud Notes app][15]
There's no formatting, although you can add markup (like Markdown) to the note. Once you're done editing, the app syncs your note with Nextcloud.
### Accessing your bookmarks
Nextcloud has a decent bookmarking tool, and its [Bookmarks][16] app makes it easy to work with the tool on your phone or tablet.
![Nextcloud Bookmarks app][17]
Like the Notes app, Bookmarks displays your bookmarks in chronological order, with the newest appearing first in the list.
If you tagged your bookmarks in Nextcloud, you can swipe left in the app to see a list of those tags rather than a long list of bookmarks. Tap a tag to view the bookmarks under it.
![Nextcloud Bookmarks app][18]
From there, just tap a bookmark. It opens in your device's default browser.
You can also add a bookmark within the app. To do that, tap the **+** menu and add the bookmark.
![Nextcloud Bookmarks app][19]
You can include:
* The URL
* A title for the bookmark
* A description
* One or more tags
### Is that all?
Definitely not. There are apps for [Nextcloud Deck][20] (a personal kanban tool) and [Nextcloud Talk][21] (a voice and video chat app). There are also a number of third-party apps that work with Nextcloud. Just do a search for _Nextcloud_ or _ownCloud_ in your favorite app store to track them down.
--------------------------------------------------------------------------------
via: https://opensource.com/article/19/5/mobile-apps-nextcloud
作者：[Scott Nesbitt][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/scottnesbitt
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/OSDC_BUS_cloudagenda_20140341_jehb.png?itok=1NGs3_n4
[2]: https://nextcloud.com/
[3]: https://lineageos.org/
[4]: https://opensource.com/life/15/1/going-open-source-android-f-droid
[5]: https://f-droid.org/en/packages/com.nextcloud.client/
[6]: https://opensource.com/sites/default/files/uploads/nextcloud-app.png (Nextcloud mobile app)
[7]: http://en.wikipedia.org/wiki/OPML
[8]: https://f-droid.org/en/packages/de.luhmer.owncloudnewsreader/
[9]: https://opensource.com/sites/default/files/uploads/nextcloud-news.png (Nextcloud News app)
[10]: https://opensource.com/sites/default/files/uploads/nextcloud-news-reading.png (Nextcloud News app)
[11]: https://f-droid.org/en/packages/it.niedermann.owncloud.notes
[12]: https://opensource.com/article/18/12/taking-notes-standard-notes
[13]: https://plaintextproject.online
[14]: https://opensource.com/sites/default/files/uploads/nextcloud-notes.png (Nextcloud Notes app)
[15]: https://opensource.com/sites/default/files/uploads/nextcloud-notes-add.png (Nextcloud Notes app)
[16]: https://f-droid.org/en/packages/org.schabi.nxbookmarks/
[17]: https://opensource.com/sites/default/files/uploads/nextcloud-bookmarks.png (Nextcloud Bookmarks app)
[18]: https://opensource.com/sites/default/files/uploads/nextcloud-bookmarks-tags.png (Nextcloud Bookmarks app)
[19]: https://opensource.com/sites/default/files/uploads/nextcloud-bookmarks-add.png (Nextcloud Bookmarks app)
[20]: https://f-droid.org/en/packages/it.niedermann.nextcloud.deck
[21]: https://f-droid.org/en/packages/com.nextcloud.talk2

2
sources/tech/20190527 Dockly - Manage Docker Containers From Terminal.md
View File

@ -1,5 +1,5 @@
[#]: collector: (lujun9972) [#]: collector: (lujun9972)
[#]: translator: ( ) [#]: translator: (geekpi)
[#]: reviewer: ( ) [#]: reviewer: ( )
[#]: publisher: ( ) [#]: publisher: ( )
[#]: url: ( ) [#]: url: ( )

56
translated/talk/20190319 Blockchain 2.0- Blockchain In Real Estate -Part 4.md Normal file
View File

@ -0,0 +1,56 @@
[#]: collector: (lujun9972)
[#]: translator: (wxy)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Blockchain 2.0: Blockchain In Real Estate [Part 4])
[#]: via: (https://www.ostechnix.com/blockchain-2-0-blockchain-in-real-estate/)
[#]: author: (ostechnix https://www.ostechnix.com/author/editor/)
区块链 2.0：房地产区块链（四）
======
![](https://www.ostechnix.com/wp-content/uploads/2019/03/Blockchain-In-Real-Estate-720x340.png)
### 区块链 2.0：“更”智能的房地产
在本系列的[上一篇文章][1]中探讨了区块链的特征，这些区块链将使机构能够将**传统银行**和**融资系统**转换和交织在一起。这部分将探讨**房地产区块链**。房地产业正在走向革命。它是人类已知的最活跃、交易最重要的资产类别之一。然而，由于充满了监管障碍和欺诈、欺骗的无数可能性，它也是最难参与交易的之一。利用适当的共识算法的区块链的分布式分类账本功能被吹捧为这个行业的前进方向，而这个行业传统上被认为其面对变革是保守的。
就其无数的业务而言，房地产一直是一个非常保守的行业。这似乎也是理所当然的。2008 年金融危机或 20 世纪上半叶的大萧条等重大经济危机成功摧毁了该行业及其参与者。然而，与大多数具有经济价值的产品一样，房地产行业具有弹性，而这种弹性则源于其保守性。
全球房地产市场包括价值 228 万亿 [^1] 美元的资产类别。给予或接受。其他投资资产，如股票、债券和股票合计价值仅为 170 万亿美元。显然，在这样一个行业中实施的任何和所有交易在很大程度上都是自然精心策划和精心执行的。在大多数情况下，房地产也因许多欺诈事件而臭名昭着，并且随之而来的是毁灭性的损失。由于其运营非常保守，该行业也难以驾驭。它受到严格法律的严格监管，创造了一个交织在一起的细微差别网络，这对于普通人来说太难以完全理解。使得大多数人无法进入和参与。如果你曾参与过这样的交易，那么你就会知道纸质文件的重要性和长期性。
从一个微不足道的开始，虽然是一个重要的例子，以显示当前的记录管理实践在房地产行业有多糟糕，考虑一下[产权保险业务][2][^3]。产权保险用于对冲土地所有权和所有权记录不可接受且从而无法执行的可能性。诸如此类的保险产品也称为赔偿保险。在许多情况下，法律要求财产拥有产权保险，特别是在处理多年来多次易手的财产时。抵押贷款公司在支持房地产交易时也可能坚持同样的要求。事实上，这种产品自 19 世纪 50 年代就已存在，并且仅在美国每年至少有 1.5 万亿美元的商业价值这一事实证明了一开始的说法。在这种情况下，这些记录的维护方式必须进行改革，区块链提供了一个可持续解决方案。根据[美国土地产权协会][4]，平均每个案例的欺诈平均约为 10 万美元，并且涉及交易的所有产权中有 25％ 的文件存在问题。区块链允许设置一个不可变的永久数据库，该数据库将跟踪属性本身，记录已经进入的每个交易或投资。这样的分类帐本系统将使包括一次性购房者在内的房地产行业的每个人的生活更加轻松，并使诸如产权保险等金融产品基本上无关紧要。将诸如房地产之类的实物资产转换为这样的数字资产是非常规的，并且目前仅在理论上存在。然而，这种变化迫在眉睫，而不是迟到[^5]。
区块链在房地产中影响最大的领域如上所述，在维护透明和安全的产权管理系统方面。基于区块链的财产记录可以包含有关财产、其所在地、所有权历史以及相同公共记录的[信息][6]。这将允许快速完成房地产交易，并且无需第三方监控和监督。房地产评估和税收计算等任务成为有形的客观参数的问题，而不是主观测量和猜测，因为可靠的历史数据是可公开验证的。[UBITQUITY][7] 就是这样一个平台，为企业客户提供定制的基于区块链的解决方案。该平台允许客户跟踪所有房产细节、付款记录、抵押记录，甚至允许运行智能合约，自动处理税收和租赁。
这为我们带来了房地产区块链的第二大机遇和用例。由于该行业受到众多第三方的高度监管，除了参与交易的交易对手外，尽职调查和财务评估可能非常耗时。这些流程主要使用离线渠道进行，文书工作需要在最终评估报告出来之前进行数天。对于公司房地产交易尤其如此，这构成了顾问收取的总计费时间的大部分。如果交易由抵押支持，则这些过程的重复是不可避免的。一旦与所涉及的人员和机构的数字身份相结合，就可以完全避免当前的低效率，并且可以在几秒钟内完成交易。租户、投资者、相关机构、顾问等可以单独验证数据并达成一致的共识，从而验证永久性的财产记录[^8]。这提高了验证流形的准确性。房地产巨头 RE/MAX 最近宣布与服务提供商 XYO Network Partners 合作，[建立墨西哥房地产上市国家数据库][9]。他们希望有朝一日能够创建世界上最大的（截至目前）去中心化房地产登记处之一。
然而，区块链可以带来的另一个重要且可以说是非常民主的变化是投资房地产。与其他投资资产类别不同，即使是小型家庭投资者也可能参与其中，房地产通常需要大量的手工付款才能参与。诸如 ATLANT 和 BitOfProperty 之类的公司将房产的账面价值代币化并将其转换为加密货币的等价物。这些代币随后在交易所出售，类似于股票和股票的交易方式。[房地产后续产生的任何现金流都会根据其在财产中的“份额”记入贷方或借记给代币所有者][4]。
然而，尽管如此，区块链技术仍处于房地产领域的早期采用阶段，目前的法规还没有明确定义它。诸如分布式应用程序、分布式匿名组织（DAO）、智能合约等概念在许多国家的法律领域是闻所未闻的。一旦所有利益相关者充分接受了区块链复杂性的良好教育，就会彻底改革现有的法规和指导方针，这是最务实的前进方式。 同样，这将是一个缓慢而渐进的变化，但不过是一个急需的变化。本系列的下一篇文章将介绍 “智能合约”，例如由 UBITQUITY 和 XYO 等公司实施的那些是如何在区块链中创建和执行的。
[^1]: HSBC, “Global Real Estate,” no. April, 2008
[^3]: D. B. Burke, Law of title insurance. Aspen Law & Business, 2000.
[^5]: M. Swan, O’Reilly – Blockchain. Blueprint for a New Economy – 2015.
[^8]: Deloite, “Blockchain in commercial real estate The future is here ! Table of contents.”
--------------------------------------------------------------------------------
via: https://www.ostechnix.com/blockchain-2-0-blockchain-in-real-estate/
作者：[EDITOR][a]
选题：[lujun9972][b]
译者：[wxy](https://github.com/wxy)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.ostechnix.com/author/editor/
[b]: https://github.com/lujun9972
[1]: https://linux.cn/article-10689-1.html
[2]: https://www.forbes.com/sites/jordanlulich/2018/06/21/what-is-title-insurance-and-why-its-important/#1472022b12bb
[4]: https://www.cbinsights.com/research/blockchain-real-estate-disruption/#financing
[6]: https://www2.deloitte.com/us/en/pages/financial-services/articles/blockchain-in-commercial-real-estate.html
[7]: https://www.ubitquity.io/
[9]: https://www.businesswire.com/news/home/20181012005068/en/XYO-Network-Partners-REMAX-M%C3%A9xico-Bring-Blockchain

431
translated/tech/20170410 Writing a Time Series Database from Scratch.md Normal file
View File

@ -0,0 +1,431 @@
从零写一个时间序列数据库
============================================================
我从事监控工作。特别是在 [Prometheus][2] 上，监控系统包含一个自定义的时间序列数据库，并且集成在 [Kubernetes][3] 上。
在许多方面上 Kubernetes 展现出了所有 Prometheus 的设计用途。它使得<ruby>持续部署<rt>continuous deployments</rt></ruby>，<ruby>弹性伸缩<rt>auto scaling</rt></ruby>和其他<ruby>高动态环境<rt>highly dynamic environments</rt></ruby>下的功能可以轻易地访问。在众多概念上的决策中，查询语句和操作模型使得 Prometheus 特别适合这种环境。但是，如果监控的工作负载动态程度显著地增加，这就会给监控系统本身带来新的压力。记住了这一点，而不是回过头来看 Prometheus 已经解决的很好的问题，我们就可以明确目标去提升它高动态或<ruby>瞬态服务<rt>transient services</rt></ruby>环境下的表现。
Prometheus 的存储层在很长一段时间里都展现出卓越的性能，单一服务器就能够以每秒数百多万个时间序列的速度摄入多达一百万个样本，同时只占用了很少的磁盘空间。尽管当前的存储做的很好，但我依旧提出一个新设计的存储子系统，它更正了现存解决方案的缺点，并具备处理更大规模数据的能力。
注释：我没有数据库方面的背景。我说的东西可能是错的并让你误入歧途。你可以在 Freenode 的 #prometheus 频道上提出你的批评（fabxc）
### 问题，难题，问题域
首先，快速地概览一下我们要完成的东西和它的关键难题。我们可以先看一下 Prometheus 当前的做法 ，它为什么做的这么好，以及我们打算用新设计解决哪些问题。
### 时间序列数据
我们有一个收集一段时间数据的系统。
```
identifier -> (t0, v0), (t1, v1), (t2, v2), (t3, v3), ....
```
每个数据点是一个时间戳和值的元组。在监控中，时间戳是一个整数，值可以是任意数字。64 位浮点数对于计数器和测量值来说是一个好的表示方法，因此我们将会使用它。一系列严格单调递增的时间戳数据点是一个序列，它由标识符所引用。我们的标识符是一个带有<ruby>标签维度<rt>label dimensions</rt></ruby>字典的度量名称。标签维度分开了单一指标的测量空间。每一个指标名称加上一个独一无二的标签集就成了它自己的时间序列，它有一个与之关联的<ruby>数据流<rt>value stream</rt></ruby>。
这是一个典型的<ruby>序列标识符<rt>series identifiers </rt></ruby>集，它是统计请求指标的一部分：
```
requests_total{path="/status", method="GET", instance=”10.0.0.1:80”}
requests_total{path="/status", method="POST", instance=”10.0.0.3:80”}
requests_total{path="/", method="GET", instance=”10.0.0.2:80”}
```
让我们简化一下表示方法：度量名称可以当作另一个维度标签，在我们的例子中是 `__name__`。对于查询语句，可以对它进行特殊处理，但与我们存储的方式无关，我们后面也会见到。
```
{__name__="requests_total", path="/status", method="GET", instance=”10.0.0.1:80”}
{__name__="requests_total", path="/status", method="POST", instance=”10.0.0.3:80”}
{__name__="requests_total", path="/", method="GET", instance=”10.0.0.2:80”}
```
我们想通过标签来查询时间序列数据。在最简单的情况下，使用 `{__name__="requests_total"}` 选择所有属于 `requests_total` 指标的数据。对于所有选择的序列，我们在给定的时间窗口内获取数据点。
在更复杂的语句中，我们或许想一次性选择满足多个标签的序列，并且表示比相等条件更复杂的情况。例如，非语句（`method!="GET"`）或正则表达式匹配（`method=~"PUT|POST"`）。
这些在很大程度上定义了存储的数据和它的获取方式。
### 纵与横
在简化的视图中，所有的数据点可以分布在二维平面上。水平维度代表着时间，序列标识符域经纵轴展开。
```
series
^
│ . . . . . . . . . . . . . . . . . . . . . . {__name__="request_total", method="GET"}
│ . . . . . . . . . . . . . . . . . . . . . . {__name__="request_total", method="POST"}
│ . . . . . . .
│ . . . . . . . . . . . . . . . . . . . ...
│ . . . . . . . . . . . . . . . . . . . . .
│ . . . . . . . . . . . . . . . . . . . . . {__name__="errors_total", method="POST"}
│ . . . . . . . . . . . . . . . . . {__name__="errors_total", method="GET"}
│ . . . . . . . . . . . . . .
│ . . . . . . . . . . . . . . . . . . . ...
│ . . . . . . . . . . . . . . . . . . . .
v
<-------------------- time --------------------->
```
Prometheus 通过定期地抓取一组时间序列的当前值来获取数据点。我们获取到的实体称为目标。因此，写入模式完全地垂直且高度并发，因为来自每个目标的样本是独立摄入的。这里提供一些测量的规模：单一 Prometheus 实例从成千上万的目标中收集数据点，每个数据点都暴露在成百上千个不同的时间序列中。
在每秒采集数百万数据点这种规模下，批量写入是一个不能妥协的性能要求。在磁盘上分散地写入单个数据点会相当地缓慢。因此，我们想要按顺序写入更大的数据块。
对于旋转式磁盘，它的磁头始终得物理上地向不同的扇区上移动，这是一个不足为奇的事实。而我们都知道 SSD 具有快速随机写入的特点，但事实上它不能修改单独的字节，只能写入一页 4KiB 或更多的数据量。这就意味着写入 16 字节的样本相当于写入满满一个 4Kib 的页。这一行为部分上属于[写入放大][4]，这种特性会损耗你的 SSD。因此它不仅影响速度，而且还毫不夸张地在几天或几个周内破坏掉你的硬件。
关于此问题更深层次的资料，[“Coding for SSDs”系列][5]博客是极好的资源。让我们想想有什么收获：顺序写入和批量写入对于旋转式磁盘和 SSD 来说都是理想的写入模式。大道至简。
查询模式比起写入模式千差万别。我们可以查询单一序列的一个数据点，也可以为 10000 个序列查询一个数据点，还可以查询一个序列几个周的数据点，甚至是 10000 个序列几个周的数据点。因此在我们的二维平面上，查询范围不是完全水平或垂直的，而是二者形成矩形似的组合。
[记录规则][6]减轻了已知查询的问题，但对于<ruby>点对点<rt>ad-hoc</rt></ruby>查询来说并不是一个通用的解决方法。
我们知道自己想要批量地写入，但我们得到的仅仅是一系列垂直数据点的集合。当查询一段时间窗口内的数据点时，我们不仅很难弄清楚在哪才能找到这些单独的点，而且不得不从磁盘上大量随机的地方读取。也许一条查询语句会有数百万的样本，即使在最快的 SSD 上也会很慢。读入也会从磁盘上获取更多的数据而不仅仅是 16 字节的样本。SSD 会加载一整页，HDD 至少会读取整个扇区。不论哪一种，我们都在浪费宝贵的读吞吐量。
因此在理想上，相同序列的样本将按顺序存储，这样我们就能通过尽可能少的读取来扫描它们。在上层，我们仅需要知道序列的起始位置就能访问所有的数据点。
显然，将收集到的数据写入磁盘的理想模式与能够显著提高查询效率的布局之间存在着很强的张力。这是我们 TSDB 需要解决的一个基本问题。
#### 当前的解法
是时候看一下当前 Prometheus 是如何存储数据来解决这一问题的，让我们称它为“V2”。
我们创建一个时间序列的文件，它包含所有样本并按顺序存储。因为每几秒附加一个样本数据到所有文件中非常昂贵，我们打包 1Kib 样本序列的数据块在内存中，一旦打包完成就附加这些数据块到单独的文件中。这一方法解决了大部分问题。写入目前是批量的，样本也是按顺序存储的。它还支持非常高效的压缩格式，基于给定的同一序列的样本相对之前的数据仅发生非常小的改变这一特性。Facebook 在他们 Gorilla TSDB 上的论文中描述了一个相似的基于数据块的方法，并且[引入了一种压缩格式][7]，它能够减少 16 字节的样本到平均 1.37 字节。V2 存储使用了包含 Gorilla 等的各种压缩格式。
```
┌──────────┬─────────┬─────────┬─────────┬─────────┐ series A
└──────────┴─────────┴─────────┴─────────┴─────────┘
┌──────────┬─────────┬─────────┬─────────┬─────────┐ series B
└──────────┴─────────┴─────────┴─────────┴─────────┘
. . .
┌──────────┬─────────┬─────────┬─────────┬─────────┬─────────┐ series XYZ
└──────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
chunk 1 chunk 2 chunk 3 ...
```
尽管基于块存储的方法非常棒，但为每个序列保存一个独立的文件会给 V2 存储带来麻烦，因为：
* 我们实际上需要比当前收集的时间序列数目使用更多的文件。多出的部分在<ruby>序列分流<rt>Series Churn</rt></ruby>上。拥有几百万个文件，迟早会使用光文件系统中的 [inodes][1]。这种情况我们只可以通过重新格式化来恢复磁盘，这种方式是最具有破坏性的。我们通常想要避免为了适应一个应用程序而格式化磁盘。
* 即使是分块写入，每秒也会产生几千万块的数据块并且准备持久化。这依然需要每秒数千个次的磁盘写入量。尽管通过为每个序列打包好多个块来缓解，但反过来还是增加了等待持久化数据的总内存占用。
* 要保持所有文件的打开状态进行读写是不可行的。特别是因为 99% 的数据在 24 小时之后不再会被查询到。如果它还是被查询到，我们就得打开数千个文件，找到并读取相关的数据点到内存中，然后再关掉。这样做就会引起很高的查询延迟，数据块缓存加剧会导致新的问题，这一点在“资源消耗”一节另作讲述。
* 最终，旧的数据需要被删除并且数据需要从数百万文件的头部删除。这就意味着删除实际上是高强度的写入操作。此外，循环遍历数百万文件并且进行分析通常会导致这一过程花费数小时。当它完成时，可能又得重新来过。喔天，继续删除旧文件又会进一步导致 SSD 产生写入放大。
* 目前所积累的数据块仅维持在内存中。如果应用崩溃，数据就会丢失。为了避免这种情况，内存状态会定期的保存在磁盘上，这比我们能接受数据丢失的时间要长的多。恢复检查点也会花费数分钟，导致很长的重启周期。
我们能够从现有的设计中学到的关键部分是数据块的概念，这一点会依旧延续。最近一段时间的数据块会保持在内存中也大体上不错。毕竟，最近时间段的数据会大量的查询到。一个时间序列对应一个文件，这种概念是我们想要替换掉的。
### 序列分流
在 Prometheus 的<ruby>上下文<rt>context</rt></ruby>中，我们使用术语<ruby>序列分流<rt>series churn</rt></ruby>来描述不活越的时间序列集合，即不再接收数据点，取而代之的是出现一组新的活跃序列。
例如，由给定微服务实例产生的所有序列都有一个相对的“instance”标签来标识它的起源。如果我们为微服务执行了<ruby>滚动更新<rt>rolling update</rt></ruby>，并且为每个实例替换一个新的版本，序列分流便会发生。在更加动态的环境中，这些事情基本上每小时都会发生。像 Kubernetes 这样的<ruby>集群编排<rt>Cluster orchestration</rt></ruby>系统允许应用连续性的自动伸缩和频繁的滚动更新，这样也许会创建成千上万个新的应用程序实例，并且伴随着全新的时间序列集合，每天都是如此。
```
series
^
│ . . . . . .
│ . . . . . .
│ . . . . . .
│ . . . . . . .
│ . . . . . . .
│ . . . . . . .
│ . . . . . .
│ . . . . . .
│ . . . . .
│ . . . . .
│ . . . . .
v
<-------------------- time --------------------->
```
所以即便整个基础设施的规模基本保持不变，过一段时间后数据库内的时间序列还是会成线性增长。尽管 Prometheus 很愿意采集 1000 万个时间序列数据，但要想在 10 亿的序列中找到数据，查询效果还是会受到严重的影响。
#### 当前解法
当前 Prometheus 的 V2 存储系统对所有保存的序列拥有基于 LevelDB 的索引。它允许查询语句含有给定的<ruby>标签对<rt>label pair</rt></ruby>，但是缺乏可伸缩的方法来从不同的标签选集中组合查询结果。
例如，从所有的序列中选择标签 `__name__="requests_total"` 非常高效，但是选择  `instance="A" AND __name__="requests_total"` 就有了可伸缩性的问题。我们稍后会重新考虑导致这一点的原因和能够提升查找延迟的调整方法。
事实上正是这个问题才催生出了对更好的存储系统的最初探索。Prometheus 需要为查找亿万的时间序列改进索引方法。
### 资源消耗
当试图量化 Prometheus （或其他任何事情，真的）时，资源消耗是永恒不变的话题之一。但真正困扰用户的并不是对资源的绝对渴求。事实上，由于给定的需求，Prometheus 管理着令人难以置信的吞吐量。问题更在于面对变化时的相对未知性与不稳定性。由于自身的架构设计，V2 存储系统构建样本数据块相当缓慢，这一点导致内存占用随时间递增。当数据块完成之后，它们可以写到磁盘上并从内存中清除。最终，Prometheus 的内存使用到达平衡状态。直到监测环境发生了改变——每次我们扩展应用或者进行滚动更新，序列分流都会增加内存、CPU、磁盘 IO 的使用。如果变更正在进行，那么它最终还是会到达一个稳定的状态，但比起更加静态的环境，它的资源消耗会显著地提高。过渡时间通常为数个小时，而且难以确定最大资源使用量。
为每个时间序列保存一个文件这种方法也使得单一查询很容易崩溃 Prometheus 进程。当查询的数据没有缓存在内存中，查询的序列文件就会被打开，然后将含有相关数据点的数据块读入内存。如果数据量超出内存可用量，Prometheus 就会因 OOM 被杀死而退出。
在查询语句完成之后，加载的数据便可以被再次释放掉，但通常会缓存更长的时间，以便更快地查询相同的数据。后者看起来是件不错的事情。
最后，我们看看之前提到的 SSD 的写入放大，以及 Prometheus 是如何通过批量写入来解决这个问题的。尽管如此，在许多地方还是存在因为拥有太多小批量数据以及在页的边界上未精确对齐的数据而导致的写入放大。对于更大规模的 Prometheus 服务器，现实当中发现会缩减硬件寿命的问题。这一点对于数据库应用的高写入吞吐量来说仍然相当普遍，但我们应该放眼看看是否可以解决它。
### 重新开始
到目前为止我们对于问题域，V2 存储系统是如何解决它的，以及设计上的问题有了一个清晰的认识。我们也看到了许多很棒的想法，这些或多或少都可以拿来直接使用。V2 存储系统相当数量的问题都可以通过改进和部分的重新设计来解决，但为了好玩（当然，在我仔细的验证想法之后），我决定试着写一个完整的时间序列数据库——从头开始，即向文件系统写入字节。
性能与资源使用这种最关键的部分直接导致了存储格式的选取。我们需要为数据找到正确的算法和磁盘布局来实现一个高性能的存储层。
这就是我解决问题的捷径——跳过令人头疼，失败的想法，数不尽的草图，泪水与绝望。
### V3—宏观设计
我们存储系统的宏观布局是什么？简而言之，是当我们在数据文件夹里运行 `tree` 命令时显示的一切。看看它能给我们带来怎样一副惊喜的画面。
```
$ tree ./data
./data
├── b-000001
│ ├── chunks
│ │ ├── 000001
│ │ ├── 000002
│ │ └── 000003
│ ├── index
│ └── meta.json
├── b-000004
│ ├── chunks
│ │ └── 000001
│ ├── index
│ └── meta.json
├── b-000005
│ ├── chunks
│ │ └── 000001
│ ├── index
│ └── meta.json
└── b-000006
├── meta.json
└── wal
├── 000001
├── 000002
└── 000003
```
在最顶层，我们有一系列以 `b-` 为前缀编号的<ruby>块<rt>block</rt></ruby>。每个块中显然保存了索引文件和含有更多编号文件的 `chunk` 文件夹。`chunks` 目录只包含不同序列<ruby>数据点的原始块<rt>raw chunks of data points</rt><ruby>。与 V2存储系统一样，这使得通过时间窗口读取序列数据非常高效并且允许我们使用相同的有效压缩算法。这一点被证实行之有效，我们也打算沿用。显然，这里并不存在含有单个序列的文件，而是一堆保存着许多序列的数据块。
`index`文件的存在应不足为奇。让我们假设它拥有黑魔法，可以让我们找到标签、可能的值、整个时间序列和存放数据点的数据块。
但为什么这里有好几个文件夹都是索引和块文件的布局？并且为什么存在最后一个包含“wal”文件夹？理解这两个疑问便能解决九成的问题 。
#### 许多小型数据库
我们分割横轴，即将时间域分割为不重叠的块。每一块扮演者完全独立的数据库，它包含该时间窗口所有的时间序列数据。因此，它拥有自己的索引和一系列块文件。
```
t0 t1 t2 t3 now
┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐
│ │ │ │ │ │ │ │ ┌────────────┐
│ │ │ │ │ │ │ mutable │ <─── write ──── ┤ Prometheus │
│ │ │ │ │ │ │ │ └────────────┘
└───────────┘ └───────────┘ └───────────┘ └───────────┘ ^
└──────────────┴───────┬──────┴──────────────┘ │
│ query
│ │
merge ─────────────────────────────────────────────────┘
```
每一块的数据都是<ruby>不可变的<rt>immutable</rt></ruby>。当然，当我们采集新数据时，我们必须能向最近的块中添加新的序列和样本。对于该数据块，所有新的数据都将写入内存中的数据库中，它与我们的持久化的数据块一样提供了查找属性。内存中的数据结构可以高效地更新。为了防止数据丢失，所有预传入的数据同样被写入临时的<ruby>预写日志<rt>write ahead log</rt></ruby>中，这就是 `wal` 文件夹中的一些列文件，我们可以在重新启动时通过它们加载内存数据库。
所有这些文件都带有序列化格式，有我们所期望的所有东西：许多标志，偏移量，变体和 CRC32 校验。纸上得来终觉浅，绝知此事要躬行。
这种布局允许我们扩展查询范围到所有相关的块上。每个块上的部分结果最终合并成完整的结果。
这种横向分割增加了一些很棒的功能：
* 当查询一个时间范围，我们可以简单地忽略所有范围之外的数据块。通过减少需要检查的一系列数据，它可以初步解决序列分流的问题。
* 当完成一个块，我们可以通过顺序的写入大文件从内存数据库中保存数据。这样可以避免任何的写入放大，并且 SSD 与 HDD 均适用。
* 我们延续了 V2 存储系统的一个好的特性，最近使用而被多次查询的数据块，总是保留在内存中。
* 足够好了，我们也不再限定 1KiB 的数据块尺寸来使数据在磁盘上更好地对齐。我们可以挑选对单个数据点和压缩格式最合理的尺寸。
* 删除旧数据变得极为简单快捷。我们仅仅只需删除一个文件夹。记住，在旧的存储系统中我们不得不花数个小时分析并重写数亿个文件。
每个块还包含了 `meta.json` 文件。它简单地保存了关于块的存储状态和包含的数据以供人们简单的阅读。
##### mmap
将数百万个小文件合并为一个大文件使得我们用很小的开销就能保持所有的文件都打开。这就引出了 [`mmap(2)`][8] 的使用，一个允许我们通过文件透明地回传虚拟内存的系统调用。为了简便，你也许想到了<ruby>交换空间<rt>swap space</rt></ruby>，只是我们所有的数据已经保存在了磁盘上，并且当数据换出内存后不再会发生写入。
这意味着我们可以当作所有数据库的内容都保留在内存中却不占用任何物理内存。仅当我们访问数据库文件确定的字节范围时，操作系统从磁盘上<ruby>惰性加载<rt>lazy loads</rt></ruby>页数据。这使得我们将所有数据持久化相关的内存管理都交给了操作系统。大体上，操作系统已足够资格作出决定，因为它拥有整个机器和进程的视图。查询的数据可以相当积极的缓存进内存，但内存压力会使得页被逐出。如果机器拥有未使用的内存，Prometheus 目前将会高兴地缓存整个数据库，但是一旦其他进程需要，它就会立刻返回。
因此，查询不再轻易地使我们的进程 OOM，因为查询的是更多的持久化的数据而不是装入内存中的数据。内存缓存大小变得完全自适应，并且仅当查询真正需要时数据才会被加载。
就个人理解，如果磁盘格式允许，这就是当今大多数数据库的理想工作方式——除非有人自信的在进程中智胜操作系统。我们做了很少的工作但确实从外面获得了很多功能。
#### 压缩
存储系统需要定期的“切”出新块并写入之前完成的块到磁盘中。仅在块成功的持久化之后，写之前用来恢复内存块的日志文件（wal）才会被删除。
我们很乐意将每个块的保存时间设置的相对短一些（通常配置为 2 小时）以避免内存中积累太多的数据。当查询多个块，我们必须合并它们的结果为一个完成的结果。合并过程显然会消耗资源，一个周的查询不应该由 80 多个部分结果所合并。
为了实现两者，我们引入<ruby>压缩<rt>compaction</rt></ruby>。压缩描述了一个过程：取一个或更多个数据块并将其写入一个可能更大的块中。它也可以在此过程中修改现有的数据。例如，清除已经删除的数据，或为提升查询性能重建样本块。
```
t0 t1 t2 t3 t4 now
┌────────────┐ ┌──────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐
│ 1 │ │ 2 │ │ 3 │ │ 4 │ │ 5 mutable │ before
└────────────┘ └──────────┘ └───────────┘ └───────────┘ └───────────┘
┌─────────────────────────────────────────┐ ┌───────────┐ ┌───────────┐
│ 1 compacted │ │ 4 │ │ 5 mutable │ after (option A)
└─────────────────────────────────────────┘ └───────────┘ └───────────┘
┌──────────────────────────┐ ┌──────────────────────────┐ ┌───────────┐
│ 1 compacted │ │ 3 compacted │ │ 5 mutable │ after (option B)
└──────────────────────────┘ └──────────────────────────┘ └───────────┘
```
在这个例子中我们有一系列块`[1,2,3,4]`。块 1，2 ，3 可以压缩在一起，新的布局将会是 `[1,4]`。或者，将它们成对压缩为 `[1,3]`。所有的时间序列数据仍然存在，但现在整体上保存在更少的块中。这极大程度地缩减了查询时间的消耗，因为需要合并的部分查询结果变得更少了。
#### 保留
我们看到了删除旧的数据在 V2 存储系统中是一个缓慢的过程，并且消耗 CPU、内存和磁盘。如何才能在我们基于块的设计上清除旧的数据？相当简单，只要根据块文件夹下的配置的保留窗口里有无数据而删除该文件夹。在下面的例子中，块 1 可以被安全地删除，而块 2 则必须一直保持到界限后面。
```
|
┌────────────┐ ┌────┼─────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐
│ 1 │ │ 2 | │ │ 3 │ │ 4 │ │ 5 │ . . .
└────────────┘ └────┼─────┘ └───────────┘ └───────────┘ └───────────┘
|
|
retention boundary
```
得到越旧的数据，保存的块也就越大，因为我们会压缩之前的压缩块。因此必须为其设置一个上限，以防数据块扩展到整个数据库而损失我们设计的最初优势。
方便的是，这一点也限制了部分存在于保留窗口内部分存在于保留窗口外的总磁盘块的消耗。例如上面例子中的块 2。当设置了最大块尺寸为总保留窗口的 10% 后，我们保留块 2 的总开销也有了 10% 的上限。
总结一下，保留与删除从非常昂贵到了几乎没有成本。
> 如果你读到这里并有一些数据库的背景知识，现在你也许会问：这些都是最新的技术吗？——并不是。而且可能还会做的更好。
> 在内存中打包数据，定期的写入日志并刷新磁盘的模式在现在相当普遍。
> 我们看到的好处无论在什么领域的数据里都是适用的。遵循这一方法最著名的开源案例是 LevelDB，Cassandra，InfluxDB 和 HBase。关键是避免重复发明劣质的轮子，采用经得起验证的方法，并正确地运用它们。
> 这里仍有地方来添加你自己的黑魔法。
### 索引
研究存储改进的最初想法是解决序列分流的问题。基于块的布局减少了查询所要考虑的序列总数。因此假设我们索引查找的复杂度是 `O(n^2)`，我们就要设法减少 n 个相当数量的复杂度，之后就有了改进后 `O(n^2)` 的复杂度。——恩，等等...糟糕。
快速地想想“算法 101”课上提醒我们的，在理论上它并未带来任何好处。如果之前就很糟糕，那么现在也一样。理论是如此的残酷。
实际上，我们大多数的查询已经可以相当快地被相应。但是，跨越整个时间范围的查询仍然很慢，尽管只需要找到少部分数据。追溯到所有这些工作之前，最初我用来解决这个问题的想法是：我们需要一个更大容量的[倒排索引][9]。倒排索引基于数据项内容的子集提供了一种快速的查找方式。简单地说，我可以通过标签 `app=”nginx"` 查找所有的序列而无需遍历每个文件来看它是否包含该标签。
为此，每个序列被赋上一个唯一的 ID 来在常数时间内获取，例如 O(1)。在这个例子中 ID 就是 我们的正向索引。
> 示例：如果 ID 为 10，29 ，9 的序列包含标签 `app="nginx"`，那么 “nginx”的倒排索引就是简单的列表 `[10, 29, 9]`，它就能用来快速地获取所有包含标签的序列。即使有 200 多亿个数据也不会影响查找速度。
简而言之，如果 n 是我们序列总数，m 是给定查询结果的大小，使用索引的查询复杂度现在就是 O(m)。查询语句跟随它获取数据的数量 m 而不是被搜索的数据体 n 所扩展是一个很好的特性，因为 m 一般相当小。
为了简单起见，我们假设可以在常数时间内查找到倒排索引对应的列表。
实际上，这几乎就是 V2 存储系统已有的倒排索引，也是提供在数百万序列中查询性能的最低需求。敏锐的人会注意到，在最坏情况下，所有的序列都含有标签，因此 m 又成了 O(n)。这一点在预料之中也相当合理。如果你查询所有的数据，它自然就会花费更多时间。一旦我们牵扯上了更复杂的查询语句就会有问题出现。
#### 标签组合
数百万个带有标签的数据很常见。假设横向扩展着数百个实例的“foo”微服务，并且每个实例拥有数千个序列。每个序列都会带有标签`app="foo"`。当然，用户通常不会查询所有的序列而是会通过进一步的标签来限制查询。例如，我想知道服务实例接收到了多少请求，那么查询语句便是 `__name__="requests_total" AND app="foo"`。
为了找到适应所有标签选择子的序列，我们得到每一个标签的倒排索引列表并取其交集。结果集通常会比任何一个输入列表小一个数量级。因为每个输入列表最坏情况下的尺寸为 O(n)，所以在嵌套地为每个列表进行<ruby>暴力求解<rt>brute force solution</rt><ruby>下，运行时间为 O(n^2)。与其他的集合操作耗费相同，例如取并集 (`app="foo" OR app="bar"`)。当添加更多标签选择子在查询语句上，耗费就会指数增长到 O(n^3), O(n^4), O(n^5), ... O(n^k)。有很多手段都能通过改变执行顺序优化运行效率。越复杂，越是需要关于数据特征和标签之间相关性的知识。这引入了大量的复杂度，但是并没有减少算法的最坏运行时间。
这便是 V2 存储系统使用的基本方法，幸运的是，似乎稍微的改动就能获得很大的提升。如果我们假设倒排索引中的 ID 都是排序好的会怎么样？
假设这个例子的列表用于我们最初的查询：
```
__name__="requests_total" -> [ 9999, 1000, 1001, 2000000, 2000001, 2000002, 2000003 ]
app="foo" -> [ 1, 3, 10, 11, 12, 100, 311, 320, 1000, 1001, 10002 ]
intersection => [ 1000, 1001 ]
```
它的交集相当小。我们可以为每个列表的起始位置设置游标，每次从最小的游标处移动来找到交集。当二者的数字相等，我们就添加它到结果中并移动二者的游标。总体上，我们以锯齿形扫描两个列表，因此总耗费是 O(2n)=O(n)，因为我们总是在一个列表上移动。
两个以上列表的不同集合操作也类似。因此 k 个集合操作仅仅改变了因子 O(k*n) 而不是最坏查找运行时间下的指数 O(n^k)。
我在这里所描述的是任意一个[全文搜索引擎][10]使用的标准搜索索引的简化版本。每个序列描述符都视作一个简短的“文档”，每个标签（名称 + 固定值）作为其中的“单词”。我们可以忽略搜索引擎索引中很多附加的数据，例如单词位置和和频率。
似乎存在着无止境的研究来提升实际的运行时间，通常都是对输入数据做一些假设。不出意料的是，仍有大量技术来压缩倒排索引，其中各有利弊。因为我们的“文档”比较小，而且“单词”在所有的序列里大量重复，压缩变得几乎无关紧要。例如，一个真实的数据集约有 440 万个序列与大约 12 个标签，每个标签拥有少于 5000 个单独的标签。对于最初的存储版本，我们坚持基本的方法不使用压缩，仅做微小的调整来跳过大范围非交叉的 ID。
尽管维持排序好的 ID 听起来很简单，但实践过程中不是总能完成的。例如，V2 存储系统为新的序列赋上一个哈希值来当作 ID，我们就不能轻易地排序倒排索引。另一个艰巨的任务是当磁盘上的数据被更新或删除掉后修改其索引。通常，最简单的方法是重新计算并写入，但是要保证数据库在此期间可查询且具有一致性。V3 存储系统通过每块上独立的不可变索引来解决这一问题，仅通过压缩时的重写来进行修改。只有可变块上的索引需要被更新，它完全保存在内存中。
### 基准测试
我发起了一个最初版本的基准测试，它基于现实世界数据集中提取的大约 440 万个序列描述符，并生成合成数据点对应到这些序列中。这个方法仅仅测试单独的存储系统，快速的找到高并发负载场景下的运行瓶颈和触发死锁至关重要。
在概念性的运用完成之后，基准测试能够在我的 Macbook Pro 上维持每秒 2000 万的吞吐量—并且所有 Chrome 的页面和 Slack 都保持着运行。因此，尽管这听起来都很棒，它这也表明推动这项测试没有的进一步价值。（或者是没有在高随机环境下运行）。毕竟，它是合成的数据，因此在除了好的第一印象外没有多大价值。比起最初的设计目标高出 20 倍，是时候将它部署到真正的 Prometheus 服务器上了，为它添加更多现实环境中的开销和场景。
我们实际上没有可重复的 Prometheus 基准测试配置，特别是对于不同版本的 A/B 测试。亡羊补牢为时不晚，[现在就有一个了][11]！
工具可以让我们声明性地定义基准测试场景，然后部署到 AWS 的 Kubernetes 集群上。尽管对于全面的基准测试来说不是最好环境，但它肯定比 64 核 128GB 内存的专用<ruby>裸机服务器<rt>bare metal servers</rt></ruby>更能反映出用户基础。我们部署两个 Prometheus 1.5.2 服务器（V2 存储系统）和两个从 2.0 分支继续开发的 Prometheus （V3 存储系统） 。每个 Prometheus 运行在配备 SSD 的专用服务器上。我们将横向扩展的应用部署在了工作节点上，并且让其暴露典型的微服务量。此外，Kubernetes 集群本身和节点也被监控着。整个配置由另一个 Meta-Prometheus 所监督，它监控每个 Prometheus 的健康状况和性能。为了模拟序列分流，微服务定期的扩展和收缩来移除旧的 pods 并衍生新的 pods，生成新的序列。查询负载通过典型的查询选择来模拟，对每个 Prometheus 版本都执行一次。
总体上，伸缩与查询的负载和采样频率一样极大的超出了 Prometheus 的生产部署。例如，我们每隔 15 分钟换出 60% 的微服务实例去产生序列分流。在现代的基础设施上，一天仅大约会发生 1-5 次。这就保证了我们的 V3 设计足以处理未来几年的工作量。就结果而言，Prometheus 1.5.2 和 2.0 之间的性能差异在不温和的环境下会变得更大。
总而言之，我们每秒从 850 个暴露 50 万数据的目标里收集了大约 11 万份样本。
在此配置运行一段时间之后，我们可以看一下数字。我们评估了两个版本在 12 个小时之后到达稳定时的几个指标。
> 请注意从 Prometheus 图形界面的截图中轻微截断的 Y 轴
![Heap usage GB](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/heap_usage.png)
> 堆内存使用（GB）
内存资源使用对用户来说是最为困扰的问题，因为它相对的不可预测且能够导致进程崩溃。
显然，被查询的服务器正在消耗内存，这极大程度上归咎于查询引擎的开销，这一点可以当作以后优化的主题。总的来说，Prometheus 2.0 的内存消耗减少了 3-4 倍。大约 6 小时之后，在 Prometheus 1.5 上有一个明显的峰值，与我们设置 6 小时的保留边界相对应。因为删除操作成本非常高，所以资源消耗急剧提升。这一点在下面几张图中均有体现。
![CPU usage cores](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/cpu_usage.png)
> CPU 使用（核心/秒）
类似的模式展示 CPU 使用，但是查询的服务器与非查询的服务器之间的差异尤为明显。每秒获取大约 11 万个数据需要 0.5 核心/秒的 CPU 资源，比起评估查询所花费的时间，我们新的存储系统 CPU 消耗可忽略不计。
![Disk writes](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/disk_writes.png)
> 磁盘写入（MB/秒）
图片展示出的磁盘利用率取得了令人意想不到的提升。这就清楚的展示了为什么 Prometheus 1.5 很容易造成 SSD 损耗。我们看到最初的上升发生在第一个块被持久化到序列文件中的时期，然后一旦删除操作引发了重写就会带来第二个上升。令人惊讶的是，查询的服务器与非查询的服务器显示出了非常不同的利用率。
Prometheus 2.0 on the other hand, merely writes about a single Megabyte per second to its write ahead log. Writes periodically spike when blocks are compacted to disk. Overall savings: staggering 97-99%.Prometheus 2.0 在另一方面，每秒仅仅写入大约一兆字节的日志文件。当块压缩到磁盘之时，写入定期地出现峰值。这在总体上节省了：惊人的 97-99%。
![Disk usage](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/disk_usage.png)
> 磁盘大小（GB）
与磁盘写入密切相关的是总磁盘空间占用量。由于我们对样本几乎使用了相同的压缩算法，因此磁盘占用量应当相同。在更为稳定的配置中，这样做很大程度上是正确地，但是因为我们需要处理高序列分流，所以还要考虑每个序列的开销。
如我们所见，Prometheus 1.5 在两个版本达到稳定状态之前，使用的存储空间因保留操作而急速上升。Prometheus 2.0 似乎在每个序列上具有更少的消耗。我们可以清楚的看到写入日志线性地充满整个存储空间，然后当压缩完成后立刻掉下来。事实上对于两个 Prometheus 2.0 服务器，它们的曲线并不是完全匹配的，这一点需要进一步的调查。
前景大好。剩下最重要的部分是查询延迟。新的索引应当优化了查找的复杂度。没有实质上发生改变的是处理数据的过程，例如 `rate()` 函数或聚合。这些就是查询引擎要做的东西了。
![Query latency](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/query_latency.png)
> 第 99 个百分位查询延迟（秒）
数据完全符合预期。在 Prometheus 1.5 上，查询延迟随着存储的数据而增加。只有在保留操作开始且旧的序列被删除后才会趋于稳定。作为对比，Prometheus 从一开始就保持在合适的位置。
我们需要花一些心思在数据是如何被采集上，对服务器发出的查询请求通过估计以下方面被选中：查询范围和即时查询的组合，进行或轻或重的计算，访问或多或少的文件。它并不需要代表真实世界里查询的分布。也不能代表冷数据的查询性能，我们可以假设所有的样本数据都是保存在内存中的热数据。
尽管如此，我们可以相当自信地说，整体查询效果对序列分流变得非常有弹性，并且提升了高压基准测试场景下 4 倍的性能。在更为静态的环境下，我们可以假设查询时间大多数花费在了查询引擎上，改善程度明显较低。
![Ingestion rate](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/assets/ingestion_rate.png)
> 摄入的样本/秒
最后，快速地看一下不同 Prometheus 服务器的摄入率。我们可以看到搭载 V3 存储系统的两个服务器具有相同的摄入速率。在几个小时之后变得不稳定，这是因为不同的基准测试集群节点由于高负载变得无响应，与 Prometheus 实例无关。（两点之前的曲线完全匹配这一事实希望足够具有说服力）
尽管还有更多 CPU 和内存资源，两个 Prometheus 1.5.2 服务器的摄入率大大降低。序列分流高压导致了无法采集更多的数据。
那么现在每秒可以摄入的<ruby>绝对最大<rt>absolute maximum</rt></ruby>样本数是多少？
我不知道——而且故意忽略。
存在的很多因素都会影响 Prometheus 数据流量，而且没有一个单独的数字能够描述捕获质量。最大摄入率在历史上是一个导致基准出现偏差的度量量，并且忽视了更多重要的层面，例如查询性能和对序列分流的弹性。关于资源使用线性增长的大致猜想通过一些基本的测试被证实。很容易推断出其中的原因。
我们的基准测试模拟了高动态环境下 Prometheus 的压力，它比起真实世界中的更大。结果表明，虽然运行在没有优化的云服务器上，但是已经超出了预期的效果。
> 注意：在撰写本文的同时，Prometheus 1.6 正在开发当中，它允许更可靠地配置最大内存使用量，并且可能会显著地减少整体的消耗，提高 CPU 使用率。我没有重复进行测试，因为整体结果变化不大，尤其是面对高序列分流的情况。
### 总结
Prometheus 开始应对高基数序列与单独样本的吞吐量。这仍然是一项富有挑战性的任务，但是新的存储系统似乎向我们展示了未来的一些好东西：<ruby>超大规模<rt>hyper-scale</rt></ruby>，<ruby>高收敛度<rt>hyper-convergent</rt></ruby>，GIFEE 基础设施。好吧，它似乎运行的不错。
第一个配备 V3 存储系统的 [alpha 版本 Prometheus 2.0][12] 已经可以用来测试了。在早期阶段预计还会出现崩溃，死锁和其他 bug。
存储系统的代码可以在[这个单独的项目中找到][13]。Prometheus 对于寻找高效本地存储时间序列数据库的应用来说可能非常有用，之一点令人非常惊讶。
> 这里需要感谢很多人作出的贡献，以下排名不分先后：
> Bjoern Rabenstein 和 Julius Volz 在 V2 存储引擎上的打磨工作以及 V3 存储系统的反馈，这为新一代的设计奠定了基础。
> Wilhelm Bierbaum 对新设计不断的建议与见解作出了很大的贡献。Brian Brazil 不断的反馈确保了我们最终得到的是语义上合理的方法。与 Peter Bourgon 深刻的讨论验证了设计并形成了这篇文章。
> 别忘了我们整个 CoreOS 团队与公司对于这项工作的赞助与支持。感谢所有那些听我一遍遍唠叨 SSD，浮点数，序列化格式的同学。
--------------------------------------------------------------------------------
via: https://fabxc.org/blog/2017-04-10-writing-a-tsdb/
作者：[Fabian Reinartz ][a]
译者：[译者ID](https://github.com/LuuMing)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://twitter.com/fabxc
[1]:https://en.wikipedia.org/wiki/Inode
[2]:https://prometheus.io/
[3]:https://kubernetes.io/
[4]:https://en.wikipedia.org/wiki/Write_amplification
[5]:http://codecapsule.com/2014/02/12/coding-for-ssds-part-1-introduction-and-table-of-contents/
[6]:https://prometheus.io/docs/practices/rules/
[7]:http://www.vldb.org/pvldb/vol8/p1816-teller.pdf
[8]:https://en.wikipedia.org/wiki/Mmap
[9]:https://en.wikipedia.org/wiki/Inverted_index
[10]:https://en.wikipedia.org/wiki/Search_engine_indexing#Inverted_indices
[11]:https://github.com/prometheus/prombench
[12]:https://prometheus.io/blog/2017/04/10/promehteus-20-sneak-peak/
[13]:https://github.com/prometheus/tsdb

177
translated/tech/20190329 How to manage your Linux environment.md Normal file
View File

@ -0,0 +1,177 @@
[#]: collector: (lujun9972)
[#]: translator: (robsean)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (How to manage your Linux environment)
[#]: via: (https://www.networkworld.com/article/3385516/how-to-manage-your-linux-environment.html#tk.rss_all)
[#]: author: (Sandra Henry-Stocker https://www.networkworld.com/author/Sandra-Henry_Stocker/)
如何管理你的 Linux 环境
======
### Linux 用户环境变量帮助你找到你需要的命令，获取很多完成的细节，而不需要知道系统如何配置的。 设置来自哪里和如何被修改它们是另一个课题。
![IIP Photo Archive \(CC BY 2.0\)][1]
在 Linux 系统上的用户配置可以用多种方法简化你的使用。你可以运行命令，而不需要知道它们的位置。你可以重新使用先前运行的命令，而不用发愁系统是如何保持它们的踪迹。你可以查看你的电子邮件，查看手册页，并容易地回到你的 home 目录，而不管你在文件系统可能已经迷失方向。并且，当需要的时候，你可以调整你的账户设置，以便它向着你喜欢的方式来工作。
Linux 环境设置来自一系列的文件 — 一些是系统范围（意味着它们影响所有用户账户），一些是配置处于你的 home 目录中文件中。系统范围设置在你登陆时生效，本地设置在以后生效，所以，你在你账户中作出的更改将覆盖系统范围设置。对于 bash 用户，这些文件包含这些系统文件：
```
/etc/environment
/etc/bash.bashrc
/etc/profile
```
和其中一些本地文件：
```
~/.bashrc
~/.profile -- not read if ~/.bash_profile or ~/.bash_login
~/.bash_profile
~/.bash_login
```
你可以修改本地存在的四个文件的任何一个，因为它们处于你的 home 目录，并且它们是属于你的。
**[ 两分钟 Linux 提示：[学习如何在2分钟视频教程中掌握很多 Linux 命令][2] ]**
### 查看你的 Linux 环境设置
为查看你的环境设置，使用 **env** 命令。你的输出将可能与这相似：
```
$ env
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;
01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:
*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:
*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:
*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;
31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:
*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:
*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:
*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:
*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:
*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:
*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:
*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:
*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:
*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:
*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:
*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:
*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.spf=00;36:
SSH_CONNECTION=192.168.0.21 34975 192.168.0.11 22
LESSCLOSE=/usr/bin/lesspipe %s %s
LANG=en_US.UTF-8
OLDPWD=/home/shs
XDG_SESSION_ID=2253
USER=shs
PWD=/home/shs
HOME=/home/shs
SSH_CLIENT=192.168.0.21 34975 22
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
SSH_TTY=/dev/pts/0
MAIL=/var/mail/shs
TERM=xterm
SHELL=/bin/bash
SHLVL=1
LOGNAME=shs
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
XDG_RUNTIME_DIR=/run/user/1000
PATH=/home/shs/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
LESSOPEN=| /usr/bin/lesspipe %s
_=/usr/bin/env
```
虽然你可能会得到大量的输出，第一个大部分用颜色显示上面的细节，颜色被用于命令行上来识别各种各样文件类型。当你看到一些东西，像 ***.tar=01;31:** ，这告诉你 tar 文件将以红色显示在文件列表中，然而 ***.jpg=01;35:** 告诉你 jpg 文件将以紫色显现出来。这些颜色本意是使它易于从一个文件列表中分辨出某些文件。你可以在[在 Linux 命令行中自定义你的颜色][3]处学习更多关于这些颜色的定义，和如何自定义它们，
当你更喜欢一种不加装饰的显示时，一种简单关闭颜色方法是使用一个命令，例如这一个：
```
$ ls -l --color=never
```
这个命令可以简单地转换到一个别名：
```
$ alias ll2='ls -l --color=never'
```
你也可以使用 **echo** 命令来单独地显现设置。在这个命令中，我们显示在历史缓存区中将被记忆命令的数量：
```
$ echo $HISTSIZE
1000
```
如果你已经移动，你在文件系统中的最后位置将被记忆。
```
PWD=/home/shs
OLDPWD=/tmp
```
### 作出更改
你可以使用一个像这样的命令更改环境设置的，但是，如果你希望保持这个设置，在你的 ~/.bashrc 文件中添加一行代码，例如 "HISTSIZE=1234" 。
```
$ export HISTSIZE=1234
```
### "export" 一个变量的本意是什么
导出一个变量使设置可用于你的 shell 和可能的子shell。默认情况下，用户定义的变量是本地的，并不被导出到新的进程，例如，子 shell 和脚本。export 命令使变量可用于子进程的函数。
### 添加和移除变量
你可以创建新的变量，并使它们在命令行和子 shell 上非常容易地可用。然而，这些变量将不存活于你的登出和再次回来，除非你也添加它们到 ~/.bashrc 或一个类似的文件。
```
$ export MSG="Hello, World!"
```
如果你需要，你可以使用 **unset** 命令来消除一个变量：
```
$ unset MSG
```
如果变量被局部定义，你可以通过获得你的启动文件来简单地设置它回来。例如：
```
$ echo $MSG
Hello, World!
$ unset $MSG
$ echo $MSG
$ . ~/.bashrc
$ echo $MSG
Hello, World!
```
### 小结
用户账户是为创建一个有用的用户环境，而使用一组恰当的启动文件建立，但是，独立的用户和系统管理员都可以通过编辑他们的个人设置文件（对于用户）或很多来自设置起源的文件（对于系统管理员）来更改默认设置。
Join the Network World communities on 在 [Facebook][4] 和 [LinkedIn][5] 上加入网络世界社区来评论重要话题。
--------------------------------------------------------------------------------
via: https://www.networkworld.com/article/3385516/how-to-manage-your-linux-environment.html#tk.rss_all
作者：[Sandra Henry-Stocker][a]
选题：[lujun9972][b]
译者：[robsean](https://github.com/robsean)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.networkworld.com/author/Sandra-Henry_Stocker/
[b]: https://github.com/lujun9972
[1]: https://images.idgesg.net/images/article/2019/03/environment-rocks-leaves-100792229-large.jpg
[2]: https://www.youtube.com/playlist?list=PL7D2RMSmRO9J8OTpjFECi8DJiTQdd4hua
[3]: https://www.networkworld.com/article/3269587/customizing-your-text-colors-on-the-linux-command-line.html
[4]: https://www.facebook.com/NetworkWorld/
[5]: https://www.linkedin.com/company/network-world

116
translated/tech/20190428 Installing Budgie Desktop on Ubuntu -Quick Guide.md Normal file
View File

@ -0,0 +1,116 @@
[#]: collector: (lujun9972)
[#]: translator: (geekpi)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Installing Budgie Desktop on Ubuntu [Quick Guide])
[#]: via: (https://itsfoss.com/install-budgie-ubuntu/)
[#]: author: (Atharva Lele https://itsfoss.com/author/atharva/)
在 Ubuntu 上安装 Budgie 桌面 [快速指南]
======
_ **简介：在这一步步的教程中学习如何在 Ubuntu 上安装 Budgie 桌面。** _
在所有[各种 Ubuntu 版本][1]中，[Ubuntu Budgie][2] 是最被低估的版本。它看起来很优雅，而且需要的资源也不多。
阅读这篇 [Ubuntu Budgie 的评论][3]或观看此视频，了解 Ubuntu Budgie 18.04 的外观。
[Subscribe to our YouTube channel for more Linux Videos][4]
如果你喜欢 [Budgie 桌面][5]但你正在使用其他版本的 Ubuntu，例如默认的 GNOME 桌面的 Ubuntu，我有个好消息。你可以在当前的 Ubuntu 系统上安装 Budgie 并切换桌面环境。
在这篇文章中，我将告诉你到底该怎么做。但首先，对那些不了解 Budgie 的人进行一点介绍。
Budgie 桌面环境主要由 [Solus Linux 团队开发][6]。它的设计注重优雅和现代使用。Budgie 适用于所有主流 Linux 发行版，供用户尝试体验这种新的桌面环境。Budgie 现在非常成熟，并提供了出色的桌面体验。
警告
在同一系统上安装多个桌面可能会导致冲突，你可能会遇到一些问题，如面板中缺少图标或同一程序的多个图标。
你也许不会遇到任何问题。是否要尝试不同桌面由你决定。
### 在 Ubuntu 上安装 Budgie
此方法未在 Linux Mint 上进行测试，因此我建议你 Mint 上不要按照此指南进行操作。
对于正在使用 Ubuntu 的人，Budgie 现在默认是 Ubuntu 仓库的一部分。因此，我们不需要添加任何 PPA 来下载 Budgie。
要安装 Budgie，只需在终端中运行此命令即可。我们首先要确保系统已完全更新。
```
sudo apt update && sudo apt upgrade
sudo apt install ubuntu-budgie-desktop
```
下载完成后，你将看到选择显示管理器的提示。选择 “lightdm” 以获得完整的 Budgie 体验。
![Select lightdm][7]
安装完成后，重启计算机。然后，你会看到 Budgie 的登录页面。输入你的密码进入主屏幕。
![Budgie Desktop Home][8]
### 切换到其他桌面环境
![Budgie login screen][9]
你可以单击登录名旁边的 Budgie 图标获取登录选项。在那里，你可以在已安装的桌面环境 （DE） 之间进行选择。就我而言，我看到了 Budgie 和默认的 Ubuntu（GNOME）桌面。
![Select your DE][10]
因此，无论何时你想登录 GNOME，都可以使用此菜单执行此操作。
[][11]
建议阅读：在 Ubuntu中 摆脱 “snapd returned status code 400: Bad Request” 错误。
### 如何删除 Budgie
如果你不喜欢 Budgie 或只是想回到常规的以前的 Ubuntu，你可以如上节所述切换回常规桌面。
但是，如果你真的想要删除 Budgie 及其组件，你可以按照以下命令回到之前的状态。
_ **在使用这些命令之前先切换到其他桌面环境：** _
```
sudo apt remove ubuntu-budgie-desktop ubuntu-budgie* lightdm
sudo apt autoremove
sudo apt install --reinstall gdm3
```
成功运行所有命令后，重启计算机。
现在，你将回到 GNOME 或其他你有的桌面。
**你对Budgie有什么看法？**
Budgie 是[最佳 Linux 桌面环境][12]之一。希望这个简短的指南帮助你在 Ubuntu 上安装了很棒的 Budgie 桌面。
如果你安装了 Budgie，你最喜欢它的什么？请在下面的评论中告诉我们。像往常一样，欢迎任何问题或建议。
--------------------------------------------------------------------------------
via: https://itsfoss.com/install-budgie-ubuntu/
作者：[Atharva Lele][a]
选题：[lujun9972][b]
译者：[geekpi](https://github.com/geekpi)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://itsfoss.com/author/atharva/
[b]: https://github.com/lujun9972
[1]: https://itsfoss.com/which-ubuntu-install/
[2]: https://ubuntubudgie.org/
[3]: https://itsfoss.com/ubuntu-budgie-18-review/
[4]: https://www.youtube.com/c/itsfoss?sub_confirmation=1
[5]: https://github.com/solus-project/budgie-desktop
[6]: https://getsol.us/home/
[7]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_install_select_dm.png?fit=800%2C559&ssl=1
[8]: https://i1.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_homescreen.jpg?fit=800%2C500&ssl=1
[9]: https://i2.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_install_lockscreen.png?fit=800%2C403&ssl=1
[10]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/04/budgie_install_lockscreen_select_de.png?fit=800%2C403&ssl=1
[11]: https://itsfoss.com/snapd-error-ubuntu/
[12]: https://itsfoss.com/best-linux-desktop-environments/