mirror of
https://github.com/LCTT/TranslateProject.git
synced 2024-12-26 21:30:55 +08:00
Merge remote-tracking branch 'LCTT/master'
This commit is contained in:
Xingyu Wang
commit
2c82b7d446
131
published/20190826 Introduction to the Linux chown command.md
Normal file
131
published/20190826 Introduction to the Linux chown command.md
Normal file
@ -0,0 +1,131 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (wxy)
|
||||
[#]: reviewer: (wxy)
|
||||
[#]: publisher: (wxy)
|
||||
[#]: url: (https://linux.cn/article-11416-1.html)
|
||||
[#]: subject: (Introduction to the Linux chown command)
|
||||
[#]: via: (https://opensource.com/article/19/8/linux-chown-command)
|
||||
[#]: author: (Alan Formy-Duval https://opensource.com/users/alanfdoss)
|
||||
|
||||
chown 命令简介
|
||||
======
|
||||
|
||||
> 学习如何使用 chown 命令更改文件或目录的所有权。
|
||||
|
||||
|
||||
|
||||
Linux 系统上的每个文件和目录均由某个人拥有，拥有者可以完全控制更改或删除他们拥有的文件。除了有一个*拥有用户*外，文件还有一个*拥有组*。
|
||||
|
||||
你可以使用 `ls -l` 命令查看文件的所有权：
|
||||
|
||||
```
|
||||
[pablo@workstation Downloads]$ ls -l
|
||||
total 2454732
|
||||
-rw-r--r--. 1 pablo pablo 1934753792 Jul 25 18:49 Fedora-Workstation-Live-x86_64-30-1.2.iso
|
||||
```
|
||||
|
||||
该输出的第三和第四列是拥有用户和组，它们一起称为*所有权*。上面的那个 ISO 文件这两者都是 `pablo`。
|
||||
|
||||
所有权设置由 [chmod 命令][2]进行设置，控制允许谁可以执行读取、写入或运行的操作。你可以使用 `chown` 命令更改所有权（一个或两者）。
|
||||
|
||||
所有权经常需要更改。文件和目录一直存在在系统中，但用户不断变来变去。当文件和目录在系统中移动时，或从一个系统移动到另一个系统时，所有权也可能需要更改。
|
||||
|
||||
我的主目录中的文件和目录的所有权是我的用户和我的主要组，以 `user:group` 的形式表示。假设 Susan 正在管理 Delta 组，该组需要编辑一个名为 `mynotes` 的文件。你可以使用 `chown` 命令将该文件的用户更改为 `susan`，组更改为 `delta`：
|
||||
|
||||
```
|
||||
$ chown susan:delta mynotes
|
||||
ls -l
|
||||
-rw-rw-r--. 1 susan delta 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
当给该文件设置好了 Delta 组时，它可以分配回给我：
|
||||
|
||||
```
|
||||
$ chown alan mynotes
|
||||
$ ls -l mynotes
|
||||
-rw-rw-r--. 1 alan delta 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
给用户后添加冒号（`:`），可以将用户和组都分配回给我：
|
||||
|
||||
```
|
||||
$ chown alan: mynotes
|
||||
$ ls -l mynotes
|
||||
-rw-rw-r--. 1 alan alan 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
通过在组前面加一个冒号，可以只更改组。现在，`gamma` 组的成员可以编辑该文件：
|
||||
|
||||
```
|
||||
$ chown :gamma mynotes
|
||||
$ ls -l
|
||||
-rw-rw-r--. 1 alan gamma 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
`chown` 的一些附加参数都能用在命令行和脚本中。就像许多其他 Linux 命令一样，`chown` 有一个递归参数（`-R`），它告诉该命令进入目录以对其中的所有文件进行操作。没有 `-R` 标志，你就只能更改文件夹的权限，而不会更改其中的文件。在此示例中，假定目的是更改目录及其所有内容的权限。这里我添加了 `-v`（详细）参数，以便 `chown` 报告其工作情况：
|
||||
|
||||
```
|
||||
$ ls -l . conf
|
||||
.:
|
||||
drwxrwxr-x 2 alan alan 4096 Aug 5 15:33 conf
|
||||
|
||||
conf:
|
||||
-rw-rw-r-- 1 alan alan 0 Aug 5 15:33 conf.xml
|
||||
|
||||
$ chown -vR susan:delta conf
|
||||
changed ownership of 'conf/conf.xml' from alan:alan to susan:delta
|
||||
changed ownership of 'conf' from alan:alan to susan:delta
|
||||
```
|
||||
|
||||
根据你的角色，你可能需要使用 `sudo` 来更改文件的所有权。
|
||||
|
||||
在更改文件的所有权以匹配特定配置时，或者在你不知道所有权时（例如运行脚本时），可以使用参考文件（`--reference=RFILE`）。例如，你可以复制另一个文件（`RFILE`，称为参考文件）的用户和组，以撤消上面所做的更改。回想一下，点（`.`）表示当前的工作目录。
|
||||
|
||||
```
|
||||
$ chown -vR --reference=. conf
|
||||
```
|
||||
|
||||
### 报告更改
|
||||
|
||||
大多数命令都有用于控制其输出的参数。最常见的是 `-v`（`--verbose`）以启用详细信息，但是 `chown` 还具有 `-c`（`--changes`）参数来指示 `chown` 仅在进行更改时报告。`chown` 还会报告其他情况，例如不允许进行的操作。
|
||||
|
||||
参数 `-f`（`--silent`、`--quiet`）用于禁止显示大多数错误消息。在下一节中，我将使用 `-f` 和 `-c`，以便仅显示实际更改。
|
||||
|
||||
### 保持根目录
|
||||
|
||||
Linux 文件系统的根目录（`/`）应该受到高度重视。如果命令在此层级上犯了一个错误，则后果可能会使系统完全无用。尤其是在运行一个会递归修改甚至删除的命令时。`chown` 命令具有一个可用于保护和保持根目录的参数，它是 `--preserve-root`。如果在根目录中将此参数和递归一起使用，那么什么也不会发生，而是会出现一条消息：
|
||||
|
||||
```
|
||||
$ chown -cfR --preserve-root alan /
|
||||
chown: it is dangerous to operate recursively on '/'
|
||||
chown: use --no-preserve-root to override this failsafe
|
||||
```
|
||||
|
||||
如果不与 `--recursive` 结合使用，则该选项无效。但是，如果该命令由 `root` 用户运行，则 `/` 本身的权限将被更改，但其下的其他文件或目录的权限则不会更改：
|
||||
|
||||
```
|
||||
$ chown -c --preserve-root alan /
|
||||
chown: changing ownership of '/': Operation not permitted
|
||||
[root@localhost /]# chown -c --preserve-root alan /
|
||||
changed ownership of '/' from root to alan
|
||||
```
|
||||
|
||||
### 所有权即安全
|
||||
|
||||
文件和目录所有权是良好的信息安全性的一部分，因此，偶尔检查和维护文件所有权以防止不必要的访问非常重要。`chown` 命令是 Linux 安全命令集中最常见和最重要的命令之一。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/19/8/linux-chown-command
|
||||
|
||||
作者：[Alan Formy-Duval][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[wxy](https://github.com/wxy)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/alanfdoss
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/yearbook-haff-rx-linux-file-lead_0.png?itok=-i0NNfDC (Hand putting a Linux file folder into a drawer)
|
||||
[2]: https://opensource.com/article/19/8/introduction-linux-chmod-command
|
||||
@ -0,0 +1,101 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (wxy)
|
||||
[#]: reviewer: (wxy)
|
||||
[#]: publisher: (wxy)
|
||||
[#]: url: (https://linux.cn/article-11415-1.html)
|
||||
[#]: subject: (Copying large files with Rsync, and some misconceptions)
|
||||
[#]: via: (https://fedoramagazine.org/copying-large-files-with-rsync-and-some-misconceptions/)
|
||||
[#]: author: (Daniel Leite de Abreu https://fedoramagazine.org/author/dabreu/)
|
||||
|
||||
使用 rsync 复制大文件的一些误解
|
||||
======
|
||||
|
||||
![][1]
|
||||
|
||||
有一种观点认为，在 IT 行业工作的许多人经常从网络帖子里复制和粘贴。我们都干过，复制粘贴本身不是问题。问题是当我们在不理解它们的情况下这样干。
|
||||
|
||||
几年前，一个曾经在我团队中工作的朋友需要将虚拟机模板从站点 A 复制到站点 B。他们无法理解为什么复制的文件在站点 A 上为 10GB，但是在站点 B 上却变为 100GB。
|
||||
|
||||
这位朋友认为 `rsync` 是一个神奇的工具，应该仅“同步”文件本身。但是，我们大多数人所忘记的是了解 `rsync` 的真正含义、用法，以及我认为最重要的是它原本是用来做什么的。本文提供了有关 `rsync` 的更多信息，并解释了那件事中发生了什么。
|
||||
|
||||
### 关于 rsync
|
||||
|
||||
`rsync` 是由 Andrew Tridgell 和 Paul Mackerras 创建的工具，其动机是以下问题：
|
||||
|
||||
假设你有两个文件，`file_A` 和 `file_B`。你希望将 `file_B` 更新为与 `file_A` 相同。显而易见的方法是将 `file_A` 复制到 `file_B`。
|
||||
|
||||
现在，假设这两个文件位于通过慢速通信链接（例如，拨号 IP 链接）连接的两个不同的服务器上。如果`file_A` 大，将其复制到 `file_B` 将会很慢，有时甚至是不可能完成的。为了提高效率，你可以在发送前压缩 `file_A`，但这通常只会获得 2 到 4 倍的效率提升。
|
||||
|
||||
现在假设 `file_A` 和 `file_B` 非常相似，并且为了加快处理速度，你可以利用这种相似性。一种常见的方法是仅通过链接发送 `file_A` 和 `file_B` 之间的差异，然后使用这个差异列表在远程端重建文件。
|
||||
|
||||
问题在于，用于在两个文件之间创建一组差异的常规方法依赖于能够读取两个文件。因此，它们要求链接的一端预先提供两个文件。如果它们在同一台计算机上不是同时可用的，则无法使用这些算法。（一旦将文件复制过来，就不需要做对比差异了）。而这是 `rsync` 解决的问题。
|
||||
|
||||
`rsync` 算法有效地计算源文件的哪些部分与现有目标文件的部分匹配。这样，匹配的部分就不需要通过链接发送了；所需要的只是对目标文件部分的引用。只有源文件中不匹配的部分才需要发送。
|
||||
|
||||
然后，接收者可以使用对现有目标文件各个部分的引用和原始素材来构造源文件的副本。
|
||||
|
||||
另外，可以使用一系列常用压缩算法中的任何一种来压缩发送到接收器的数据，以进一步提高速度。
|
||||
|
||||
我们都知道，`rsync` 算法以一种漂亮的方式解决了这个问题。
|
||||
|
||||
在 `rsync` 的介绍之后，回到那件事！
|
||||
|
||||
### 问题 1：自动精简配置
|
||||
|
||||
有两件事可以帮助那个朋友了解正在发生的事情。
|
||||
|
||||
该文件在其他地方的大小变得越来越大的问题是由源系统上启用了<ruby>自动精简配置<rt>Thin Provisioning</rt></ruby>（TP）引起的，这是一种优化存储区域网络（SAN）或网络连接存储（NAS）中可用空间效率的方法。
|
||||
|
||||
由于启用了 TP，源文件只有 10GB，并且在不使用任何其他配置的情况下使用 `rsync` 进行传输时，目标位置将接收到全部 100GB 的大小。`rsync` 无法自动完成该（TP）操作，必须对其进行配置。
|
||||
|
||||
进行此工作的选项是 `-S`（或 `–sparse`），它告诉 `rsync` 有效地处理稀疏文件。它会按照它说的做！它只会发送该稀疏数据，因此源和目标将有一个 10GB 的文件。
|
||||
|
||||
### 问题 2：更新文件
|
||||
|
||||
当发送一个更新的文件时会出现第二个问题。现在目标仅接收 10GB 了，但始终传输的是整个文件（包含虚拟磁盘），即使只是在该虚拟磁盘上更改了一个配置文件。换句话说，只是该文件的一小部分发生了更改。
|
||||
|
||||
用于此传输的命令是：
|
||||
|
||||
```
|
||||
rsync -avS vmdk_file syncuser@host1:/destination
|
||||
```
|
||||
|
||||
同样，了解 `rsync` 的工作方式也将有助于解决此问题。
|
||||
|
||||
上面是关于 `rsync` 的最大误解。我们许多人认为 `rsync` 只会发送文件的增量更新，并且只会自动更新需要更新的内容。**但这不是 `rsync` 的默认行为**。
|
||||
|
||||
如手册页所述，`rsync` 的默认行为是在目标位置创建文件的新副本，并在传输完成后将其移动到正确的位置。
|
||||
|
||||
要更改 `rsync` 的默认行为，你必须设置以下标志，然后 `rsync` 将仅发送增量：
|
||||
|
||||
```
|
||||
--inplace 原地更新目标文件
|
||||
--partial 保留部分传输的文件
|
||||
--append 附加数据到更短的文件
|
||||
--progress 在传输时显示进度条
|
||||
```
|
||||
|
||||
因此，可以确切地执行我那个朋友想要的功能的完整命令是：
|
||||
|
||||
```
|
||||
rsync -av --partial --inplace --append --progress vmdk_file syncuser@host1:/destination
|
||||
```
|
||||
|
||||
注意，出于两个原因，这里必须删除稀疏选项 `-S`。首先是通过网络发送文件时，不能同时使用 `–sparse` 和 `–inplace`。其次，当你以前使用过 `–sparse` 发送文件时，就无法再使用 `–inplace` 进行更新。请注意，低于 3.1.3 的 `rsync` 版本将拒绝 `–sparse` 和 `–inplace` 的组合。
|
||||
|
||||
因此，即使那个朋友最终通过网络复制了 100GB，那也只需发生一次。以下所有更新仅复制差异，从而使复制非常高效。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://fedoramagazine.org/copying-large-files-with-rsync-and-some-misconceptions/
|
||||
|
||||
作者：[Daniel Leite de Abreu][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[wxy](https://github.com/wxy)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://fedoramagazine.org/author/dabreu/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://fedoramagazine.org/wp-content/uploads/2019/08/rsync-816x345.jpg
|
||||
@ -1,138 +0,0 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Introduction to the Linux chown command)
|
||||
[#]: via: (https://opensource.com/article/19/8/linux-chown-command)
|
||||
[#]: author: (Alan Formy-Duval https://opensource.com/users/alanfdosshttps://opensource.com/users/sethhttps://opensource.com/users/alanfdosshttps://opensource.com/users/sethhttps://opensource.com/users/greg-phttps://opensource.com/users/alanfdoss)
|
||||
|
||||
Introduction to the Linux chown command
|
||||
======
|
||||
Learn how to change a file or directory's ownership with chown.
|
||||
![Hand putting a Linux file folder into a drawer][1]
|
||||
|
||||
Every file and directory on a Linux system is owned by someone, and the owner has complete control to change or delete the files they own. In addition to having an owning _user_, a file has an owning _group_.
|
||||
|
||||
You can view the ownership of a file using the **ls -l** command:
|
||||
|
||||
|
||||
```
|
||||
[pablo@workstation Downloads]$ ls -l
|
||||
total 2454732
|
||||
-rw-r--r--. 1 pablo pablo 1934753792 Jul 25 18:49 Fedora-Workstation-Live-x86_64-30-1.2.iso
|
||||
```
|
||||
|
||||
The third and fourth columns of the output are the owning user and group, which together are referred to as _ownership_. Both are **pablo** for the ISO file above.
|
||||
|
||||
The ownership settings, set by the [**chmod** command][2], control who is allowed to perform read, write, or execute actions. You can change ownership (one or both) with the **chown** command.
|
||||
|
||||
It is often necessary to change ownership. Files and directories can live a long time on a system, but users can come and go. Ownership may also need to change when files and directories are moved around the system or from one system to another.
|
||||
|
||||
The ownership of the files and directories in my home directory are my user and my primary group, represented in the form **user:group**. Suppose Susan is managing the Delta group, which needs to edit a file called **mynotes**. You can use the **chown** command to change the user to **susan** and the group to **delta**:
|
||||
|
||||
|
||||
```
|
||||
$ chown susan:delta mynotes
|
||||
ls -l
|
||||
-rw-rw-r--. 1 susan delta 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
Once the Delta group is finished with the file, it can be assigned back to me:
|
||||
|
||||
|
||||
```
|
||||
$ chown alan mynotes
|
||||
$ ls -l mynotes
|
||||
-rw-rw-r--. 1 alan delta 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
Both the user and group can be assigned back to me by appending a colon (**:**) to the user:
|
||||
|
||||
|
||||
```
|
||||
$ chown alan: mynotes
|
||||
$ ls -l mynotes
|
||||
-rw-rw-r--. 1 alan alan 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
By prepending the group with a colon, you can change just the group. Now members of the **gamma** group can edit the file:
|
||||
|
||||
|
||||
```
|
||||
$ chown :gamma mynotes
|
||||
$ ls -l
|
||||
-rw-rw-r--. 1 alan gamma 0 Aug 1 12:04 mynotes
|
||||
```
|
||||
|
||||
A few additional arguments to chown can be useful at both the command line and in a script. Just like many other Linux commands, chown has a recursive argument ****(**-R**) which tells the command to descend into the directory to operate on all files inside. Without the **-R** flag, you change permissions of the folder only, leaving the files inside it unchanged. In this example, assume that the intent is to change permissions of a directory and all its contents. Here I have added the **-v** (verbose) argument so that chown reports what it is doing:
|
||||
|
||||
|
||||
```
|
||||
$ ls -l . conf
|
||||
.:
|
||||
drwxrwxr-x 2 alan alan 4096 Aug 5 15:33 conf
|
||||
|
||||
conf:
|
||||
-rw-rw-r-- 1 alan alan 0 Aug 5 15:33 conf.xml
|
||||
|
||||
$ chown -vR susan:delta conf
|
||||
changed ownership of 'conf/conf.xml' from alan:alan to susan:delta
|
||||
changed ownership of 'conf' from alan:alan to susan:delta
|
||||
```
|
||||
|
||||
Depending on your role, you may need to use **sudo** to change ownership of a file.
|
||||
|
||||
You can use a reference file (**\--reference=RFILE**) when changing the ownership of files to match a certain configuration or when you don't know the ownership (as might be the case when running a script). You can duplicate the user and group of another file (**RFILE**, known as a reference file), for example, to undo the changes made above. Recall that a dot (**.**) refers to the present working directory.
|
||||
|
||||
|
||||
```
|
||||
`$ chown -vR --reference=. conf`
|
||||
```
|
||||
|
||||
### Report Changes
|
||||
|
||||
Most commands have arguments for controlling their output. The most common is **-v** (-**-verbose**) to enable verbose, but chown also has a **-c** (**\--changes**) argument to instruct chown to only report when a change is made. Chown still reports other things, such as when an operation is not permitted.
|
||||
|
||||
The argument **-f** (**\--silent**, **\--quiet**) is used to suppress most error messages. I will use **-f** and the **-c** in the next section so that only actual changes are shown.
|
||||
|
||||
### Preserve Root
|
||||
|
||||
The root (**/**) of the Linux filesystem should be treated with great respect. If a mistake is made at this level, the consequences could leave a system completely useless. Particularly when you are running a recursive command that makes any kind of change or worse: deletions. The chown command has an argument that can be used to protect and preserve the root. The argument is **\--preserve-root**. If this argument is used with a recursive chown command on the root, nothing is done and a message appears instead.
|
||||
|
||||
|
||||
```
|
||||
$ chown -cfR --preserve-root alan /
|
||||
chown: it is dangerous to operate recursively on '/'
|
||||
chown: use --no-preserve-root to override this failsafe
|
||||
```
|
||||
|
||||
The option has no effect when not used in conjunction with **\--recursive**. However, if the command is run by the root user, the permissions of the **/** itself will be changed, but not of other files or directories within.
|
||||
|
||||
|
||||
```
|
||||
$ chown -c --preserve-root alan /
|
||||
chown: changing ownership of '/': Operation not permitted
|
||||
[root@localhost /]# chown -c --preserve-root alan /
|
||||
changed ownership of '/' from root to alan
|
||||
```
|
||||
|
||||
### Ownership is security
|
||||
|
||||
File and directory ownership is part of good information security, so it's important to occasionally check and maintain file ownership to prevent unwanted access. The chown command is one of the most common and important in the set of Linux security commands.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/19/8/linux-chown-command
|
||||
|
||||
作者：[Alan Formy-Duval][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/alanfdosshttps://opensource.com/users/sethhttps://opensource.com/users/alanfdosshttps://opensource.com/users/sethhttps://opensource.com/users/greg-phttps://opensource.com/users/alanfdoss
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/yearbook-haff-rx-linux-file-lead_0.png?itok=-i0NNfDC (Hand putting a Linux file folder into a drawer)
|
||||
[2]: https://opensource.com/article/19/8/introduction-linux-chmod-command
|
||||
@ -1,101 +0,0 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Copying large files with Rsync, and some misconceptions)
|
||||
[#]: via: (https://fedoramagazine.org/copying-large-files-with-rsync-and-some-misconceptions/)
|
||||
[#]: author: (Daniel Leite de Abreu https://fedoramagazine.org/author/dabreu/)
|
||||
|
||||
Copying large files with Rsync, and some misconceptions
|
||||
======
|
||||
|
||||
![][1]
|
||||
|
||||
There is a notion that a lot of people working in the IT industry often copy and paste from internet howtos. We all do it, and the copy-and-paste itself is not a problem. The problem is when we run things without understanding them.
|
||||
|
||||
Some years ago, a friend who used to work on my team needed to copy virtual machine templates from site A to site B. They could not understand why the file they copied was 10GB on site A but but it became 100GB on-site B.
|
||||
|
||||
The friend believed that _rsync_ is a magic tool that should just “sync” the file as it is. However, what most of us forget is to understand what _rsync_ really is, and how is it used, and the most important in my opinion is, where it come from. This article provides some further information about rsync, and an explanation of what happened in that story.
|
||||
|
||||
### About rsync
|
||||
|
||||
_rsync_ is a tool was created by Andrew Tridgell and Paul Mackerras who were motivated by the following problem:
|
||||
|
||||
Imagine you have two files, _file_A_ and _file_B_. You wish to update _file_B_ to be the same as _file_A_. The obvious method is to copy _file_A_ onto _file_B_.
|
||||
|
||||
Now imagine that the two files are on two different servers connected by a slow communications link, for example, a dial-up IP link. If _file_A_ is large, copying it onto _file_B_ will be slow, and sometimes not even possible. To make it more efficient, you could compress _file_A_ before sending it, but that would usually only gain a factor of 2 to 4.
|
||||
|
||||
Now assume that _file_A_ and _file_B_ are quite similar, and to speed things up, you take advantage of this similarity. A common method is to send just the differences between _file_A_ and _file_B_ down the link and then use such list of differences to reconstruct the file on the remote end.
|
||||
|
||||
The problem is that the normal methods for creating a set of differences between two files rely on being able to read both files. Thus they require that both files are available beforehand at one end of the link. If they are not both available on the same machine, these algorithms cannot be used. (Once you had copied the file over, you don’t need the differences). This is the problem that _rsync_ addresses.
|
||||
|
||||
The _rsync_ algorithm efficiently computes which parts of a source file match parts of an existing destination file. Matching parts then do not need to be sent across the link; all that is needed is a reference to the part of the destination file. Only parts of the source file which are not matching need to be sent over.
|
||||
|
||||
The receiver can then construct a copy of the source file using the references to parts of the existing destination file and the original material.
|
||||
|
||||
Additionally, the data sent to the receiver can be compressed using any of a range of common compression algorithms for further speed improvements.
|
||||
|
||||
The rsync algorithm addresses this problem in a lovely way as we all might know.
|
||||
|
||||
After this introduction on _rsync_, Back to the story!
|
||||
|
||||
### Problem 1: Thin provisioning
|
||||
|
||||
There were two things that would help the friend understand what was going on.
|
||||
|
||||
The problem with the file getting significantly bigger on the other size was caused by Thin Provisioning (TP) being enabled on the source system — a method of optimizing the efficiency of available space in Storage Area Networks (SAN) or Network Attached Storages (NAS).
|
||||
|
||||
The source file was only 10GB because of TP being enabled, and when transferred over using _rsync_ without any additional configuration, the target destination was receiving the full 100GB of size. _rsync_ could not do the magic automatically, it had to be configured.
|
||||
|
||||
The Flag that does this work is _-S_ or _–sparse_ and it tells _rsync_ to handle sparse files efficiently. And it will do what it says! It will only send the sparse data so source and destination will have a 10GB file.
|
||||
|
||||
### Problem 2: Updating files
|
||||
|
||||
The second problem appeared when sending over an updated file. The destination was now receiving just the 10GB, but the whole file (containing the virtual disk) was always transferred. Even when a single configuration file was changed on that virtual disk. In other words, only a small portion of the file changed.
|
||||
|
||||
The command used for this transfer was:
|
||||
|
||||
```
|
||||
rsync -avS vmdk_file syncuser@host1:/destination
|
||||
```
|
||||
|
||||
Again, understanding how _rsync_ works would help with this problem as well.
|
||||
|
||||
The above is the biggest misconception about rsync. Many of us think _rsync_ will simply send the delta updates of the files, and that it will automatically update only what needs to be updated. But this is not the default behaviour of _rsync_.
|
||||
|
||||
As the man page says, the default behaviour of _rsync_ is to create a new copy of the file in the destination and to move it into the right place when the transfer is completed.
|
||||
|
||||
To change this default behaviour of _rsync_, you have to set the following flags and then rsync will send only the deltas:
|
||||
|
||||
```
|
||||
--inplace update destination files in-place
|
||||
--partial keep partially transferred files
|
||||
--append append data onto shorter files
|
||||
--progress show progress during transfer
|
||||
```
|
||||
|
||||
So the full command that would do exactly what the friend wanted is:
|
||||
|
||||
```
|
||||
rsync -av --partial --inplace --append --progress vmdk_file syncuser@host1:/destination
|
||||
```
|
||||
|
||||
Note that the sparse flag _-S_ had to be removed, for two reasons. The first is that you can not use _–sparse_ and _–inplace_ together when sending a file over the wire. And second, when you once sent a file over with _–sparse_, you can’t updated with _–inplace_ anymore. Note that versions of rsync older than 3.1.3 will reject the combination of _–sparse_ and _–inplace_.
|
||||
|
||||
So even when the friend ended up copying 100GB over the wire, that only had to happen once. All the following updates were only copying the difference, making the copy to be extremely efficient.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://fedoramagazine.org/copying-large-files-with-rsync-and-some-misconceptions/
|
||||
|
||||
作者：[Daniel Leite de Abreu][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://fedoramagazine.org/author/dabreu/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://fedoramagazine.org/wp-content/uploads/2019/08/rsync-816x345.jpg
|
||||
@ -1,5 +1,5 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: translator: (HankChow)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
Loading…
Reference in New Issue
Block a user