document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2024-12-26 21:30:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'LCTT/master'

Browse Source
This commit is contained in:
Xingyu Wang 2020-04-28 22:00:59 +08:00
commit 96ba0e0346
4 changed files with 450 additions and 470 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

255
sources/tech/20200421 Using Python to visualize COVID-19 projections.md
View File

@ -1,255 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: ( )
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Using Python to visualize COVID-19 projections)
[#]: via: (https://opensource.com/article/20/4/python-data-covid-19)
[#]: author: (AnuragGupta https://opensource.com/users/999anuraggupta)
Using Python to visualize COVID-19 projections
======
I'll demonstrate how to create two visualizations of the spread of a
virus across the globe, provided open data and using open source
libraries.
![Colorful sound wave graph][1]
Using [Python][2] and some graphing libraries, you can project the total number of confirmed cases of COVID-19, and also display the total number of deaths for a country (this article uses India as an example) on a given date. Humans sometimes need help interpreting and processing the meaning of data, so this article also demonstrates how to create an animated horizontal bar graph for five countries, showing the variation of cases by date.
### Projecting confirmed cases and deaths for India
This is done in three steps.
#### 1\. Download data
Scientific data isn't always open, but fortunately, many modern science and healthcare organizations are eager to share information with each other and the public. Data about COVID-19 cases is available online, and it's updated frequently.
To parse the data, you first must download it: <https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv>
Load the data directly into a Pandas DataFrame. Pandas provides a function, **read_csv()**, which can take a URL and give back a DataFrame object, as shown below:
```
import pycountry
import plotly.express as px
import pandas as pd
URL_DATASET = r'<https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv>'
df1 = pd.read_csv(URL_DATASET)
print(df1.head(3))  # Get first 3 entries in the dataframe
print(df1.tail(3))  # Get last 3 entries in the dataframe
```
The top row of the data set contains column names:
1. Date
2. Country
3. Confirmed
4. Recovered
5. Deaths
The output of the **head** query includes a unique identifier (not listed as a column) plus an entry for each column:
```
0 2020-01-22 Afghanistan 0 0 0
1 2020-01-22 Albania 0 0 0
1 2020-01-22 Algeria 0 0 0
```
The output of the **tail** query is similar but contains the tail end of the data set:
```
12597 2020-03-31 West Bank and Gaza 119 18 1
12598 2020-03-31 Zambia 35 0 0
12599 2020-03-31 Zimbabwe 8 0 1
```
From the output, you can see that the DataFrame (**df1**) has the following columns:
1. Date
2. Country
3. Confirmed
4. Recovered
5. Dead
Further, you can see that the **Date** column has entries starting from January 22 to March 31. This database is updated daily, so you will have current values.
#### 2\. Select data for India
In this step, we will select only those rows in the DataFrame that include India. This is shown in the script below:
```
#### ----- Step 2 (Select data for India)----
df_india = df1[df1['Country'] == 'India']
print(df_india.head(3))
```
#### 3\. Plot data
Here we create a bar chart. We will put the dates on the X-axis and the number of confirmed cases and the number of deaths on the Y-axis. There are a few noteworthy things about this part of the script which are as follows:
* The line of code: **plt.rcParams["_figure.figsize"_]=20,20** is meant only for Jupyter. So remove it if you are using some other IDE.
* Notice the line of code: **ax1 = plt.gca()**. To ensure that both the plots i.e. for confirmed cases as well as for deaths are plotted on the same graph, we need to give to the second graph the **ax** object of the plot. So we use **gca()** to do this. (By the way, 'gca' stands for 'get current axis').
The complete script is given below:
```
#  Author:- Anurag Gupta # email:- [999.anuraggupta@gmail.com][3]
%matplotlib inline
import matplotlib.pyplot as plt
import pandas as pd
#### ----- Step 1 (Download data)----
URL_DATASET = r'<https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv>'
df1 = pd.read_csv(URL_DATASET)
# print(df1.head(3))  # Uncomment to see the dataframe
#### ----- Step 2 (Select data for India)----
df_india = df1[df1['Country'] == 'India']
print(df_india.head(3))
#### ----- Step 3 (Plot data)----
# Increase size of plot
plt.rcParams["figure.figsize"]=20,20  # Remove if not on Jupyter
# Plot column 'Confirmed'
df_india.plot(kind = 'bar', x = 'Date', y = 'Confirmed', color = 'blue')
ax1 = plt.gca()
df_india.plot(kind = 'bar', x = 'Date', y = 'Deaths', color = 'red', ax = ax1)
plt.show()
```
The entire script is [available on GitHub][4].
### Creating an animated horizontal bar graph for five countries
Note for Jupyter: To run this in Jupyter as a dynamic animation rather than as a static png, you need to add a magic command at the beginning of your cell, namely: **%matplotlib notebook**. This will keep the figure alive instead of displaying a static png file and can hence also show animations. If you are on another IDE, remove this line.
#### 1\. Download the data
This step is exactly the same as in the previous script, and therefore, it need not be repeated.
#### 2\. Create a list of all dates
If you examine the data you downloaded, you notice that it has a column **Date**. Now, this column has a date value for each country. So the same date is occurring a number of times. We need to create a list of dates with only unique values. This will be used on the X-axis of our bar charts. We have a line of code like: **list_dates = df[_‘Date’_].unique()**. The **unique()** method will pick up only the unique values for each date.
#### 3\. Pick five countries and create an **ax** object
Take a list of five countries. (You can choose whatever countries you prefer, or even increase or decrease the number of countries). I have also taken a list of five colors for the bars of each country. (You can change this too if you like). One important line of code here is: **fig, ax = plt.subplots(figsize=(15, 8))**. This is needed to create an **ax** object.
#### 4\. Write the call back function
If you want to do animation in Matplotlib, you need to create an object of a class called **matplotlib.animation.FuncAnimation**. The signature of this class is available online. The constructor of this class, apart from other parameters, also takes a parameter called **func**, and you have to give this parameter a callback function. So in this step, we will write the callback function, which is repeatedly called in order to render the animation.
#### 5\. Create **FuncAnimation** object
This step has partly been explained in the previous step.
Our code to create an object of this class is:
```
my_anim = animation.FuncAnimation(fig = fig, func = plot_bar,
                    frames= list_dates, blit=True,
                    interval=20)
```
The three important parameters to be given are:
* **fig**, which must be given a fig object, which we created earlier.
* **func**, which must be the call back function.
* **frames**, which must contain the variable on which the animation is to be done. Here in our case, it will be the list of dates we created earlier.
#### 6\. Save the animation to an mp4 file
You can save the animation created into an mp4 file. But for this you need **ffmpeg**. You can download this using pip by **pip install ffmpeg-python**, or using conda (on Jupyter) **install -c conda-forge ffmpeg**.
And finally, you can run the animation using **plt.show()**. Please note that on many platforms, the **ffmpeg** may not work properly and may require further "tweaking."
```
%matplotlib notebook
#  Author:- Anurag Gupta # email:- [999.anuraggupta@gmail.com][3]
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from time import sleep
#### ---- Step 1:- Download data
URL_DATASET = r'<https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv>'
df = pd.read_csv(URL_DATASET, usecols = ['Date', 'Country', 'Confirmed'])
# print(df.head(3)) # uncomment this to see output
#### ---- Step 2:- Create list of all dates
list_dates = df['Date'].unique()
# print(list_dates) # Uncomment to see the dates
#### --- Step 3:- Pick 5 countries. Also create ax object
fig, ax = plt.subplots(figsize=(15, 8))
# We will animate for these 5 countries only
list_countries = ['India', 'China', 'US', 'Italy', 'Spain']
# colors for the 5 horizontal bars
list_colors = ['black', 'red', 'green', 'blue', 'yellow']
### --- Step 4:- Write the call back function
# plot_bar() is the call back function used in FuncAnimation class object
def plot_bar(some_date):
    df2 = df[df['Date'].eq(some_date)]
    ax.clear()
    # Only take Confirmed column in descending order
    df3 = df2.sort_values(by = 'Confirmed', ascending = False)
    # Select the top 5 Confirmed countries
    df4 = df3[df3['Country'].isin(list_countries)]
    # print(df4)  # Uncomment to see that dat is only for 5 countries
    sleep(0.2)  # To slow down the animation
    # ax.barh() makes a horizontal bar plot.
    return ax.barh(df4['Country'], df4['Confirmed'], color= list_colors)
###----Step 5:- Create FuncAnimation object---------
my_anim = animation.FuncAnimation(fig = fig, func = plot_bar,
                    frames= list_dates, blit=True,
                    interval=20)
### --- Step 6:- Save the animation to an mp4
# Place where to save the mp4. Give your file path instead
path_mp4 = r'C:\Python-articles\population_covid2.mp4'  
# my_anim.save(path_mp4, fps=30, extra_args=['-vcodec', 'libx264'])
my_anim.save(filename = path_mp4, writer = 'ffmpeg',
             fps=30,
             extra_args= ['-vcodec', 'libx264', '-pix_fmt', 'yuv420p'])
plt.show()
```
The complete script is [available on GitHub][5].
--------------------------------------------------------------------------------
via: https://opensource.com/article/20/4/python-data-covid-19
作者：[AnuragGupta][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/999anuraggupta
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/colorful_sound_wave.png?itok=jlUJG0bM (Colorful sound wave graph)
[2]: https://opensource.com/resources/python
[3]: mailto:999.anuraggupta@gmail.com
[4]: https://raw.githubusercontent.com/ag999git/jupyter_notebooks/master/corona_bar_india
[5]: https://raw.githubusercontent.com/ag999git/jupyter_notebooks/master/corona_bar_animated

215
sources/tech/20200425 Inlining optimisations in Go.md
View File

@ -1,215 +0,0 @@
[#]: collector: (lujun9972)
[#]: translator: (lxbwolf)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Inlining optimisations in Go)
[#]: via: (https://dave.cheney.net/2020/04/25/inlining-optimisations-in-go)
[#]: author: (Dave Cheney https://dave.cheney.net/author/davecheney)
Inlining optimisations in Go
======
This is a post about how the Go compiler implements inlining and how this optimisation affects your Go code.
_n.b._ This article focuses on _gc_, the de facto Go compiler from [golang.org][1]. The concepts discussed apply broadly to other Go compilers like gccgo and llgo but may differ in implementation and efficacy.
### What is inlining?
Inlining is the act of combining smaller functions into their respective callers. In the early days of computing this optimisation was typically performed by hand. Nowadays inlining is one of a class of fundamental optimisations performed automatically during the compilation process.
### Why is inlining important?
Inlining is important for two reasons. The first is it removes the overhead of the function call itself. The second is it permits the compiler to more effectively apply other optimisation strategies.
#### Function call overhead
Calling a function[1][2] in any language carries a cost. There are the overheads of marshalling parameters into registers or onto the stack (depending on the ABI) and reversing the process on return. Invoking a function call involves jumping the program counter from one point in the instruction stream to another which can cause a pipeline stall. Once inside the function there is usually some preamble required to prepare a new stack frame for the function to execute and a similar epilogue needed to retire the frame before returning to the caller.
In Go a function call carries additional costs to support dynamic stack growth. On entry the amount of stack space available to the goroutine is compared to the amount required for the function. If insufficient stack space is available, the preamble jumps into the runtime logic that grows the stack by copying it to a new, larger, location. Once this is done the runtime jumps back to the start of the original function, the stack check is performed again, which now passes, and the call continues. In this way goroutines can start with a small stack allocation which grows only when needed.[2][3]
This check is cheap–only a few instructions–and because goroutine stacks grows geometrically the check rarely fails. Thus, the branch prediction unit inside a modern processor can hide the cost of the stack check by assuming it will always be successful. In the case where the processor mis-predicts the stack check and has to discard the work done while it was executing speculatively, the cost of the pipeline stall is relatively small compared to the cost of the work needed for the runtime to grow a goroutines stack.
While the overhead of the generic and Go specific components of each function call are well optimised by modern processors using speculative execution techniques, those overheads cannot be entirely eliminated, thus each function call carries with it a performance cost over and above the time it takes to perform useful work. As a function call’s overhead is fixed, smaller functions pay a larger cost relative to larger ones because they tend to do less useful work per invocation.
The solution to eliminating these overheads must therefore be to eliminate the function call itself, which the Go compiler does, under certain conditions, by replacing the call to a function with the contents of the function. This is known as _inlining_ because it brings the body of the function in line with its caller.
#### Improved optimisation opportunities
Dr. Cliff Click describes inlining as _the_ optimisation performed by modern compilers as it forms the basis for optimisations like constant proportion and dead code elimination. In effect, inlining allows the compiler to _see_ _furthe_r, allowing it to observe, in the context that a particular function is being called, logic that can be further simplified or eliminated entirely. As inlining can be applied recursively optimisation decisions can be made not only in the context of each individual function, but also applied to the chain of functions in a call path.
### Inlining in action
The effects of inlining can be demonstrated with this small example
```
package main
import "testing"
//go:noinline
func max(a, b int) int {
if a > b {
return a
}
return b
}
var Result int
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
r = max(-1, i)
}
Result = r
}
```
Running this benchmark gives the following result:[3][4]
```
% go test -bench=. 
BenchmarkMax-4   530687617         2.24 ns/op
```
The cost of `max(-1, i)` is around 2.24 nanoseconds on my 2015 MacBook Air. Now let’s remove the `//go:noinline` pragma and see the result:
```
% go test -bench=. 
BenchmarkMax-4   1000000000         0.514 ns/op
```
From 2.24 ns to 0.51 ns, or according to `benchstat`, a 78% improvement.
```
% benchstat {old,new}.txt
name   old time/op  new time/op  delta
Max-4  2.21ns ± 1%  0.49ns ± 6%  -77.96%  (p=0.000 n=18+19)
```
Where did these improvements come from?
First, the removal of the function call and associated preamble[4][5] was a major contributor. Pulling the contents of `max` into its caller reduced the number of instructions executed by the processor and eliminated several branches.
Now the contents of `max` are visible to the compiler as it optimises `BenchmarkMax` it can make some additional improvements. Consider that once `max` is inlined, this is what the body of `BenchmarkMax` looks like to the compiler:
```
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
if -1 > i {
r = -1
} else {
r = i
}
}
Result = r
}
```
Running the benchmark again we see our manually inlined version performs as well as the version inlined by the compiler
```
% benchstat {old,new}.txt
name   old time/op  new time/op  delta
Max-4  2.21ns ± 1%  0.48ns ± 3%  -78.14%  (p=0.000 n=18+18)
```
Now the compiler has access to the result of inlining `max` into `BenchmarkMax` it can apply optimisation passes which were not possible before. For example, the compiler has noted that `i` is initialised to `0` and only incremented so any comparison with `i` can assume `i` will never be negative. Thus, the condition `-1 > i` will never be true.[5][6]
Having proved that `-1 > i` will never be true, the compiler can simplify the code to
```
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
if false {
r = -1
} else {
r = i
}
}
Result = r
}
```
and because the branch is now a constant, the compiler can eliminate the unreachable path leaving it with
```
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
r = i
}
Result = r
}
```
Thus, through inlining and the optimisations it unlocks, the compiler has reduced the expression `r = max(-1, i)` to simply `r = i`.
### The limits of inlining
In this article I’ve discussed, so called, _leaf_ inlining; the act of inlining a function at the bottom of a call stack into its direct caller. Inlining is a recursive process, once a function has been inlined into its caller, the compiler may inline the resulting code into _its_ caller, as so on. For example, this code
```
func BenchmarkMaxMaxMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
r = max(max(-1, i), max(0, i))
}
Result = r
}
```
runs as fast as the previous example as the compiler is able to repeatedly apply the optimisations outlined above to reduce the code to the same `r = i` expression.
In the next article I’ll discuss an alternative inlining strategy when the Go compiler wishes to inline a function in the middle of a call stack. Finally I’ll discuss the limits that the compiler is prepared to go to to inline code, and which Go constructs are currently beyond its capability.
1. In Go, a method is just a function with a predefined formal parameter, the receiver. The relative costs of calling a free function vs a invoking a method, assuming that method is not called through an interface, are the same.[][7]
2. Up until Go 1.14 the stack check preamble was also used by the garbage collector to stop the world by setting all active goroutine’s stacks to zero, forcing them to trap into the runtime the next time they made a function call. This system was [recently replaced][8] with a mechanism which allowed the runtime to pause an goroutine without waiting for it to make a function call.[][9]
3. I’m using the `//go:noinline` pragma to prevent the compiler from inlining `max`. This is because I want to isolate the effects of inlining on `max` rather than disabling optimisations globally with `-gcflags='-l -N'`. I go into detail about the `//go:` comments in [this presentation][10].[][11]
4. You can check this for yourself by comparing the output of `go test -bench=. -gcflags=-S` with and without the `//go:noinline` annotation.[][12]
5. You can check this yourself with the `-gcflags=-d=ssa/prove/debug=on` flag.[][13]
#### Related posts:
1. [Five things that make Go fast][14]
2. [Why is a Goroutine’s stack infinite ?][15]
3. [How to write benchmarks in Go][16]
4. [Go’s hidden #pragmas][17]
--------------------------------------------------------------------------------
via: https://dave.cheney.net/2020/04/25/inlining-optimisations-in-go
作者：[Dave Cheney][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://dave.cheney.net/author/davecheney
[b]: https://github.com/lujun9972
[1]: https://github.com/golang/go
[2]: tmp.gBQ2tEtMHc#easy-footnote-bottom-1-4053 (In Go, a method is just a function with a predefined formal parameter, the receiver. The relative costs of calling a free function vs a invoking a method, assuming that method is not called through an interface, are the same.)
[3]: tmp.gBQ2tEtMHc#easy-footnote-bottom-2-4053 (Up until Go 1.14 the stack check preamble was also used by the garbage collector to stop the world by setting all active goroutine’s stacks to zero, forcing them to trap into the runtime the next time they made a function call. This system was <a href="https://github.com/golang/proposal/blob/master/design/24543-non-cooperative-preemption.md">recently replaced</a> with a mechanism which allowed the runtime to pause an goroutine without waiting for it to make a function call.)
[4]: tmp.gBQ2tEtMHc#easy-footnote-bottom-3-4053 (I’m using the <code>//go:noinline</code> pragma to prevent the compiler from inlining <code>max</code>. This is because I want to isolate the effects of inlining on <code>max</code> rather than disabling optimisations globally with <code>-gcflags='-l -N'</code>. I go into detail about the <code>//go:</code> comments in <a href="https://dave.cheney.net/2018/01/08/gos-hidden-pragmas">this presentation</a>.)
[5]: tmp.gBQ2tEtMHc#easy-footnote-bottom-4-4053 (You can check this for yourself by comparing the output of <code>go test -bench=. -gcflags=-S</code> with and without the <code>//go:noinline</code> annotation.)
[6]: tmp.gBQ2tEtMHc#easy-footnote-bottom-5-4053 (You can check this yourself with the <code>-gcflags=-d=ssa/prove/debug=on</code> flag.)
[7]: tmp.gBQ2tEtMHc#easy-footnote-1-4053
[8]: https://github.com/golang/proposal/blob/master/design/24543-non-cooperative-preemption.md
[9]: tmp.gBQ2tEtMHc#easy-footnote-2-4053
[10]: https://dave.cheney.net/2018/01/08/gos-hidden-pragmas
[11]: tmp.gBQ2tEtMHc#easy-footnote-3-4053
[12]: tmp.gBQ2tEtMHc#easy-footnote-4-4053
[13]: tmp.gBQ2tEtMHc#easy-footnote-5-4053
[14]: https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast (Five things that make Go fast)
[15]: https://dave.cheney.net/2013/06/02/why-is-a-goroutines-stack-infinite (Why is a Goroutine’s stack infinite ?)
[16]: https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go (How to write benchmarks in Go)
[17]: https://dave.cheney.net/2018/01/08/gos-hidden-pragmas (Go’s hidden #pragmas)

239
translated/tech/20200421 Using Python to visualize COVID-19 projections.md Normal file
View File

@ -0,0 +1,239 @@
[#]: collector: (lujun9972)
[#]: translator: (wxy)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Using Python to visualize COVID-19 projections)
[#]: via: (https://opensource.com/article/20/4/python-data-covid-19)
[#]: author: (AnuragGupta https://opensource.com/users/999anuraggupta)
使用 Python 来可视化 COVID-19 预测
======
> 我将演示如何使用开源库利用提供的全球病毒传播的开放数据来创建两个可视效果。
![Colorful sound wave graph][1]
使用 [Python][2] 和一些图形库，你可以预测出 COVID-19 确诊病例的总数，也可以显示一个国家（本文以印度为例）在给定日期的死亡总数。人们有时需要帮助解释和处理数据的意义，所以本文还演示了如何为五个国家创建一个动画横条形图，以显示按日期显示病例的变化。
### 印度的确诊病例和死亡人数预测
这要分三步来完成。
#### 1、下载数据
科学数据并不总是开放的，但幸运的是，许多现代科学和医疗机构都乐于相互之间及与公众共享信息。关于 COVID-19 病例的数据可以在网上查到，并且经常更新。
要解析这些数据，首先必须先下载。 <https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv>。
直接将数据加载到 Pandas `DataFrame` 中。Pandas 提供了一个函数 `read_csv()`，它可以获取一个 URL 并返回一个 `DataFrame` 对象，如下所示。
```
import pycountry
import plotly.express as px
import pandas as pd
URL_DATASET = r'https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv'
df1 = pd.read_csv(URL_DATASET)
print(df1.head(3)) # 获取数据帧中的前 3 项
print(df1.tail(3)) # 获取数据帧中的后 3 项
```
数据集的顶行包含列名。
1. `Date`
2. `Country`
3. `Confirmed`
4. `Recovered`
5. `Deaths`
`head` 查询的输出包括一个唯一的标识符（不作为列列出）和每个列的条目。
```
0 2020-01-22 Afghanistan 0 0 0
1 2020-01-22 Albania 0 0 0
1 2020-01-22 Algeria 0 0 0
```
`tail` 查询的输出类似，但包含数据集的尾端。
```
12597 2020-03-31 West Bank and Gaza 119 18 1
12598 2020-03-31 Zambia 35 0 0
12599 2020-03-31 Zimbabwe 8 0 1
```
从输出中，可以看到 DataFrame（`df1`）有以下几个列：
1. 日期
2. 国家
3. 确诊
4. 康复
5. 死亡
此外，你可以看到 `Date` 栏中的条目从 1 月 22 日开始到 3 月 31 日。这个数据库每天都会更新，所以你会有当前的值。
#### 2、选择印度的数据
在这一步中，我们将只选择 DataFrame 中包含印度的那些行。这在下面的脚本中可以看到。
```
#### ----- Step 2 (Select data for India)----
df_india = df1[df1['Country'] == 'India']
print(df_india.head(3))
```
#### 3、数据绘图
在这里，我们创建一个条形图。我们将把日期放在 X 轴上，把确诊的病例数和死亡人数放在 Y 轴上。这一部分的脚本有以下几个值得注意的地方。
* `plt.rcParams["_figure.figure.figsize"_]=20,20` 这一行代码只适用于 Jupyter。所以如果你使用其他 IDE，请删除它。
* 注意这行代码：`ax1 = plt.gca()`。为了确保两个图，即确诊病例和死亡病例的图都被绘制在同一个图上，我们需要给第二个图的 `ax` 对象。所以我们使用 `gca()` 来完成这个任务。（顺便说一下，`gca` 代表“get current axis”）
完整的脚本如下所示。
```
# Author:- Anurag Gupta # email:- 999.anuraggupta@gmail.com
%matplotlib inline
import matplotlib.pyplot as plt
import pandas as pd
#### ----- Step 1 (Download data)----
URL_DATASET = r'https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv'
df1 = pd.read_csv(URL_DATASET)
# print(df1.head(3)) # Uncomment to see the dataframe
#### ----- Step 2 (Select data for India)----
df_india = df1[df1['Country'] == 'India']
print(df_india.head(3))
#### ----- Step 3 (Plot data)----
# Increase size of plot
plt.rcParams["figure.figsize"]=20,20 # Remove if not on Jupyter
# Plot column 'Confirmed'
df_india.plot(kind = 'bar', x = 'Date', y = 'Confirmed', color = 'blue')
ax1 = plt.gca()
df_india.plot(kind = 'bar', x = 'Date', y = 'Deaths', color = 'red', ax = ax1)
plt.show()
```
整个脚本[可在 GitHub 上找到][4]。
#### 为五个国家创建一个动画水平条形图
关于 Jupyter 的注意事项：要在 Jupyter 中以动态动画的形式运行，而不是静态 png 的形式，你需要在单元格的开头添加一个神奇的命令，即: `%matplotlib notebook`。这将使图形保持动态，而不是显示静态的 png 文件，因此也可以显示动画。如果你在其他 IDE 上，请删除这一行。
#### 1、下载数据
这一步和前面的脚本完全一样，所以不需要重复。
#### 2、创建一个所有日期的列表
如果你检查你下载的数据，你会发现它有一列 `Date`。现在，这一列对每个国家都有一个日期值。因此，同一个日期会出现多次。我们需要创建一个只具有唯一值的日期列表。这会用在我们条形图的 X 轴上。我们有一行代码，如 `list_dates = df[_'Date'_].unique()`。`unique()` 方法将只提取每个日期的唯一值。
#### 3、挑选五个国家并创建一个 `ax` 对象。
做一个五个国家的名单。（你可以选择你喜欢的国家，甚至可以增加或减少国家的数量。）我也做了一个五个颜色的列表，每个国家的条形图的颜色对应一种。（如果你喜欢的话，也可以改一下。）这里有一行重要的代码是：`fig, ax = plt.subplots(figsize=(15, 8))`。这是创建一个 `ax` 对象所需要的。
#### 4、编写回调函数
如果你想在 Matplotlib 中做动画，你需要创建一个名为 `matplotlib.animation.FuncAnimation` 的类的对象。这个类的签名可以在网上查到。这个类的构造函数，除了其他参数外，还需要一个叫 `func` 的参数，你必须给这个参数一个回调函数。所以在这一步中，我们会写个回调函数，这个回调函数会被反复调用，以渲染动画。
#### 5、创建 `FuncAnimation` 对象
这一步在上一步中已经部分说明了。
我们创建这个类的对象的代码是：
```
my_anim = animation.FuncAnimation(fig = fig, func = plot_bar,
frames= list_dates, blit=True,
interval=20)
```
要给出的三个重要参数是：
* `fig`，必须给出一个 fig 对象，也就是我们之前创建的 fig 对象。
* `func`，必须是回调函数。
* `frames`，必须包含要做动画的变量。在我们这里，它是我们之前创建的日期列表。
#### 6、将动画保存为 mp4 文件
你可以将创建的动画保存为 mp4 文件。但是，你需要 `ffmpeg`。你可以用 `pip` 下载：`pip install ffmpeg-python`，或者用 conda（在 Jupyter 上）：`install -c conda-forge ffmpeg`。
最后，你可以使用 `plt.show()` 运行动画。请注意，在许多平台上，`ffmpeg` 可能无法正常工作，可能需要进一步“调整”。
```
%matplotlib notebook
# Author:- Anurag Gupta # email:- 999.anuraggupta@gmail.com
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from time import sleep
#### ---- Step 1:- Download data
URL_DATASET = r'https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv'
df = pd.read_csv(URL_DATASET, usecols = ['Date', 'Country', 'Confirmed'])
# print(df.head(3)) # uncomment this to see output
#### ---- Step 2:- Create list of all dates
list_dates = df['Date'].unique()
# print(list_dates) # Uncomment to see the dates
#### --- Step 3:- Pick 5 countries. Also create ax object
fig, ax = plt.subplots(figsize=(15, 8))
# We will animate for these 5 countries only
list_countries = ['India', 'China', 'US', 'Italy', 'Spain']
# colors for the 5 horizontal bars
list_colors = ['black', 'red', 'green', 'blue', 'yellow']
### --- Step 4:- Write the call back function
# plot_bar() is the call back function used in FuncAnimation class object
def plot_bar(some_date):
df2 = df[df['Date'].eq(some_date)]
ax.clear()
# Only take Confirmed column in descending order
df3 = df2.sort_values(by = 'Confirmed', ascending = False)
# Select the top 5 Confirmed countries
df4 = df3[df3['Country'].isin(list_countries)]
# print(df4) # Uncomment to see that dat is only for 5 countries
sleep(0.2) # To slow down the animation
# ax.barh() makes a horizontal bar plot.
return ax.barh(df4['Country'], df4['Confirmed'], color= list_colors)
###----Step 5:- Create FuncAnimation object---------
my_anim = animation.FuncAnimation(fig = fig, func = plot_bar,
frames= list_dates, blit=True,
interval=20)
### --- Step 6:- Save the animation to an mp4
# Place where to save the mp4. Give your file path instead
path_mp4 = r'C:\Python-articles\population_covid2.mp4'
# my_anim.save(path_mp4, fps=30, extra_args=['-vcodec', 'libx264'])
my_anim.save(filename = path_mp4, writer = 'ffmpeg',
fps=30,
extra_args= ['-vcodec', 'libx264', '-pix_fmt', 'yuv420p'])
plt.show()
```
完整的脚本[可以在 GitHub 上找到][5]。
--------------------------------------------------------------------------------
via: https://opensource.com/article/20/4/python-data-covid-19
作者：[AnuragGupta][a]
选题：[lujun9972][b]
译者：[wxy](https://github.com/wxy)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://opensource.com/users/999anuraggupta
[b]: https://github.com/lujun9972
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/colorful_sound_wave.png?itok=jlUJG0bM (Colorful sound wave graph)
[2]: https://opensource.com/resources/python
[3]: mailto:999.anuraggupta@gmail.com
[4]: https://raw.githubusercontent.com/ag999git/jupyter_notebooks/master/corona_bar_india
[5]: https://raw.githubusercontent.com/ag999git/jupyter_notebooks/master/corona_bar_animated

211
translated/tech/20200425 Inlining optimisations in Go.md Normal file
View File

@ -0,0 +1,211 @@
[#]: collector: (lujun9972)
[#]: translator: (lxbwolf)
[#]: reviewer: ( )
[#]: publisher: ( )
[#]: url: ( )
[#]: subject: (Inlining optimisations in Go)
[#]: via: (https://dave.cheney.net/2020/04/25/inlining-optimisations-in-go)
[#]: author: (Dave Cheney https://dave.cheney.net/author/davecheney)
Go 中的内联优化
======
本文讨论 Go 编译器是如何实现内联的以及这种优化方法如何影响你的 Go 代码。
*请注意：*本文重点讨论 *gc*，实际上是 [golang.org](https://github.com/golang/go) 的 Go 编译器。讨论到的概念可以广泛用于其他 Go 编译器，如 gccgo 和 llgo，但它们在实现方式和功能上可能有所差异。
### 内联是什么？
内联就是把简短的函数在调用它的地方展开。在计算机发展历程的早期，这个优化是由程序员手动实现的。现在，内联已经成为编译过程中自动实现的基本优化过程的其中一步。
### 为什么内联很重要？
有两个原因。第一个是它消除了函数调用本身的虚耗。第二个是它使得编译器能更高效地执行其他的优化策略。
#### 函数调用的虚耗
在任何语言中，调用一个函数 [1][2] 都会有消耗。把参数编组进寄存器或放入栈中（取决于 ABI），在返回结果时倒序取出时会有虚耗。引入一次函数调用会导致程序计数器从指令流的一点跳到另一点，这可能导致管道阻塞。函数内部通常有前置处理，需要为函数执行准备新的栈帧，还有与前置相似的后续处理，需要在返回给调用方之前释放栈帧空间。
在 Go 中函数调用会消耗额外的资源来支持栈的动态增长。在进入函数时，goroutine 可用的栈空间与函数需要的空间大小相等。如果可用空间不同，前置处理就会跳到把数据复制到一块新的、更大的空间的运行时逻辑，而这会导致栈空间变大。当这个复制完成后，运行时跳回到原来的函数入口，再执行栈空间检查，函数调用继续执行。这种方式下，goroutine 开始时可以申请很小的栈空间，在有需要时再申请更大的空间。[2][3]
这个检查消耗很小 — 只有几个指令 — 而且由于 goroutine 是成几何级数增长的，因此这个检查很少失败。这样，现代处理器的分支预测单元会通过假定检查肯定会成功来隐藏栈空间检查的消耗。当处理器预测错了栈空间检查，必须要抛弃它推测性执行的操作时，与为了增加 goroutine 的栈空间运行时所需的操作消耗的资源相比，管道阻塞的代价更小。
虽然现代处理器可以用预测性执行技术优化每次函数调用中的泛型和 Go 特定的元素的虚耗，但那些虚耗不能被完全消除，因此在每次函数调用执行必要的工作过程中都会有性能消耗。一次函数调用本身的虚耗是固定的，与更大的函数相比，调用小函数的代价更大，因为在每次调用过程中它们做的有用的工作更少。
消除这些虚耗的方法必须是要消除函数调用本身，Go 的编译器就是这么做的，在某些条件下通过用函数的内容来替换函数调用来实现。这个过程被称为*内联*，因为它在函数调用处把函数体展开了。
#### 改进的优化机会
Cliff Click 博士把内联描述为现代编译器做的优化措施，像常量传播（译注：此处作者笔误，原文为 constant proportion，修正为 constant propagation）和死码消除一样，都是编译器的基本优化方法。实际上，内联可以让编译器看得更深，使编译器可以观察调用的特定函数的上下文内容，可以看到能继续简化或彻底消除的逻辑。由于可以递归地执行内联，因此不仅可以在每个独立的函数上下文处进行这种优化，也可以在整个函数调用链中进行。
### 实践中的内联
下面这个例子可以演示内联的影响：
```go
package main
import "testing"
//go:noinline
func max(a, b int) int {
if a > b {
return a
}
return b
}
var Result int
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
r = max(-1, i)
}
Result = r
}
```
运行这个基准，会得到如下结果：[3][4]
```bash
% go test -bench=.
BenchmarkMax-4 530687617 2.24 ns/op
```
在我的 2015 MacBook Air 上 `max(-1, i)` 的耗时约为 2.24 纳秒。现在去掉 `//go:noinline` 编译指令，再看下结果：
```bash
% go test -bench=.
BenchmarkMax-4 1000000000 0.514 ns/op
```
从 2.24 纳秒降到了 0.51 纳秒，或者从 `benchstat` 的结果可以看出，有 78% 的提升。
```bash
% benchstat {old,new}.txt
name old time/op new time/op delta
Max-4 2.21ns ± 1% 0.49ns ± 6% -77.96% (p=0.000 n=18+19)
```
这个提升是从哪儿来的呢？
首先，移除掉函数调用以及与之关联的前置处理 [4][5] 是主要因素。把 `max` 函数的函数体在调用处展开，减少了处理器执行的指令数量并且消除了一些分支。
现在由于编译器优化了 `BenchmarkMax`，因此它可以看到 `max` 函数的内容，进而可以做更多的提升。当 `max` 被内联后，`BenchmarkMax` 呈现给编译器的样子，看起来是这样的：
```go
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
if -1 > i {
r = -1
} else {
r = i
}
}
Result = r
}
```
再运行一次基准，我们看一下手动内联的版本和编译器内联的版本的表现：
```bash
% benchstat {old,new}.txt
name old time/op new time/op delta
Max-4 2.21ns ± 1% 0.48ns ± 3% -78.14% (p=0.000 n=18+18)
```
现在编译器能看到在 `BenchmarkMax` 里内联 `max` 的结果，可以执行以前不能执行的优化措施。例如，编译器注意到 `i` 初始值为 `0`，仅做自增操作，因此所有与 `i` 的比较都可以假定 `i` 不是负值。这样条件表达式 `-1 > i` 永远不是 true。[5][6]
证明了 `-1 > i` 永远不为 true 后，编译器可以把代码简化为：
```go
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
if false {
r = -1
} else {
r = i
}
}
Result = r
}
```
并且因为分支里是个常量，编译器可以通过下面的方式移除不会走到的分支：
```go
func BenchmarkMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
r = i
}
Result = r
}
```
这样，通过内联和由内联解锁的优化过程，编译器把表达式 `r = max(-1, i))` 简化为 `r = i`。
### 内联的限制
本文中我论述的内联称作*叶子*内联；把函数调用栈中最底层的函数在调用它的函数处展开的行为。内联是个递归的过程，当把函数内联到调用它的函数 A 处后，编译器会把内联后的结果代码再内联到 A 的调用方，这样持续内联下去。例如，下面的代码：
```go
func BenchmarkMaxMaxMax(b *testing.B) {
var r int
for i := 0; i < b.N; i++ {
r = max(max(-1, i), max(0, i))
}
Result = r
}
```
与之前的例子中的代码运行速度一样快，因为编译器可以对上面的代码重复地进行内联，也把代码简化到 `r = i` 表达式。
下一篇文章中，我会论述当 Go 编译器想要内联函数调用栈中间的某个函数时选用的另一种内联策略。最后我会论述编译器为了内联代码准备好要达到的极限，这个极限 Go 现在的能力还达不到。
1. 在 Go 中，一个方法就是一个有预先定义的形参和接受者的函数。假设这个方法不是通过接口调用的，调用一个无消耗的函数所消耗的代价与引入一个方法是相同的。[][7]
2. 在 Go 1.14 以前，栈检查的前置处理也被 gc 用于 STW，通过把所有活跃的 goroutine 栈空间设为 0，来强制它们切换为下一次函数调用时的运行时状态。这个机制[最近被替换][8]为一种新机制，新机制下运行时可以不用等 goroutine 进行函数调用就可以暂停 goroutine。[][9]
3. 我用 `//go:noinline` 编译指令来阻止编译器内联 `max`。这是因为我想把内联 `max` 的影响与其他影响隔离开，而不是用 `-gcflags='-l -N'` 选项在全局范围内禁止优化。关于 `//go:` 注释在[这篇文章][10]中详细论述。[][11]
4. 你可以自己通过比较 `go test -bench=. -gcflags=-S`有无 `//go:noinline` 注释时的不同结果来验证一下。[][12]
5. 你可以用 `-gcflags=-d=ssa/prove/debug=on` 选项来自己验证一下。[][13]
#### 相关文章：
1. [使 Go 变快的 5 件事](https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast)
2. [为什么 Goroutine 的栈空间会无限增长？](https://dave.cheney.net/2013/06/02/why-is-a-goroutines-stack-infinite)
3. [Go 中怎么写基准测试](https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go)
4. [Go 中隐藏的编译指令](https://dave.cheney.net/2018/01/08/gos-hidden-pragmas)
--------------------------------------------------------------------------------
via: https://dave.cheney.net/2020/04/25/inlining-optimisations-in-go
作者：[Dave Cheney][a]
选题：[lujun9972][b]
译者：[lxbwolf](https://github.com/lxbwolf)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://dave.cheney.net/author/davecheney
[b]: https://github.com/lujun9972
[1]: https://github.com/golang/go
[2]: tmp.gBQ2tEtMHc#easy-footnote-bottom-1-4053 (In Go, a method is just a function with a predefined formal parameter, the receiver. The relative costs of calling a free function vs a invoking a method, assuming that method is not called through an interface, are the same.)
[3]: tmp.gBQ2tEtMHc#easy-footnote-bottom-2-4053 (Up until Go 1.14 the stack check preamble was also used by the garbage collector to stop the world by setting all active goroutine’s stacks to zero, forcing them to trap into the runtime the next time they made a function call. This system was <a href="https://github.com/golang/proposal/blob/master/design/24543-non-cooperative-preemption.md">recently replaced</a> with a mechanism which allowed the runtime to pause an goroutine without waiting for it to make a function call.)
[4]: tmp.gBQ2tEtMHc#easy-footnote-bottom-3-4053 (I’m using the <code>//go:noinline</code> pragma to prevent the compiler from inlining <code>max</code>. This is because I want to isolate the effects of inlining on <code>max</code> rather than disabling optimisations globally with <code>-gcflags='-l -N'</code>. I go into detail about the <code>//go:</code> comments in <a href="https://dave.cheney.net/2018/01/08/gos-hidden-pragmas">this presentation</a>.)
[5]: tmp.gBQ2tEtMHc#easy-footnote-bottom-4-4053 (You can check this for yourself by comparing the output of <code>go test -bench=. -gcflags=-S</code> with and without the <code>//go:noinline</code> annotation.)
[6]: tmp.gBQ2tEtMHc#easy-footnote-bottom-5-4053 (You can check this yourself with the <code>-gcflags=-d=ssa/prove/debug=on</code> flag.)
[7]: tmp.gBQ2tEtMHc#easy-footnote-1-4053
[8]: https://github.com/golang/proposal/blob/master/design/24543-non-cooperative-preemption.md
[9]: tmp.gBQ2tEtMHc#easy-footnote-2-4053
[10]: https://dave.cheney.net/2018/01/08/gos-hidden-pragmas
[11]: tmp.gBQ2tEtMHc#easy-footnote-3-4053
[12]: tmp.gBQ2tEtMHc#easy-footnote-4-4053
[13]: tmp.gBQ2tEtMHc#easy-footnote-5-4053
[14]: https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast (Five things that make Go fast)
[15]: https://dave.cheney.net/2013/06/02/why-is-a-goroutines-stack-infinite (Why is a Goroutine’s stack infinite ?)
[16]: https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go (How to write benchmarks in Go)
[17]: https://dave.cheney.net/2018/01/08/gos-hidden-pragmas (Go’s hidden #pragmas)