document/TranslateProject
2
0
Fork 0
mirror of https://github.com/LCTT/TranslateProject.git synced 2024-12-26 21:30:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

Translated

Browse Source 
tech/20230727 AI- A Few More Useful Python Libraries.md
This commit is contained in:
toknow-gh 2023-11-29 23:13:57 +08:00
parent cad2c3493e
commit fd2cda7c71
2 changed files with 200 additions and 188 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

188
sources/tech/20230727 AI- A Few More Useful Python Libraries.md
View File

@ -1,188 +0,0 @@
[#]: subject: "AI: A Few More Useful Python Libraries"
[#]: via: "https://www.opensourceforu.com/2023/07/ai-a-few-more-useful-python-libraries/"
[#]: author: "Deepu Benson https://www.opensourceforu.com/author/deepu-benson/"
[#]: collector: "lujun9972/lctt-scripts-1700446145"
[#]: translator: "toknow-gh"
[#]: reviewer: " "
[#]: publisher: " "
[#]: url: " "
AI: A Few More Useful Python Libraries
======
<https://www.opensourceforu.com/wp-content/uploads/2022/08/Python-ML.jpg>
_In the previous article in this series, we discussed the history of AI. We have also discussed matrices in detail earlier. In this third article in the series on AI, we will discuss more matrix operations. We will also get familiar with a few more Python libraries useful for developing AI._
Before we proceed any further, let us discuss a few significant terms often associated with AI and machine learning. Artificial neural networks (often called neural networks or NNs) are the core of machine learning and deep learning. But, what are they? By definition, they are computing models inspired by the biological neural networks that form human brains. I won’t be including the image of an NN here because the Internet nowadays is so full of it. But any person interested in AI has definitely seen one or more of those images, with an input layer on the left, one or more hidden layers in the middle, and an output layer on the right of the image. One important thing to understand is that the weights of edges between each layer, shown in the images we mentioned just now, keep on changing with training. This leads to successful machine learning and deep learning applications.
Supervised learning and unsupervised learning are two important models of machine learning. In the long run, any aspirant who wants to work in the field of AI or machine learning needs to learn about these models and the different techniques used to implement them. Hence, I think it is time for us to (informally) understand the difference between these two models. Consider two people named A and B who are supposed to classify apples and oranges into two separate groups. They have never seen apples or oranges. Both are given 100 photos of apples and oranges. But A has the additional information as to which are apples and which are oranges in the given photos. A is like a supervised algorithm. B does not have any additional information. All he/she has is the 100 photos. B is like an unsupervised algorithm. One day both start the classifications. Who will be better at his/her job? Conventional wisdom says A will outperform B. But machine learning says this may not be the case all the time. What if, out of the 100 images given to A, only 5 are of apples and the rest are all of oranges? He/she may not get familiar with apples at all. What if somebody intentionally cheated A by changing some of the labels of the photos, from apples to oranges and vice versa? In these scenarios, B could outperform A in the job.
But can this happen in a machine learning application for real? Yes! Imagine a situation where you are training your model with an inadequate or corrupted data set. This is just one of the many reasons why both these learning models, supervised and unsupervised, are still thriving in the fields of AI and machine learning. In the coming articles in this series, we will treat these topics more formally. Now let us learn to use JupyterLab, a very powerful tool available for developing AI based applications.
### Introduction to JupyterLab
In the previous articles in this series, we were using the Linux terminal to run Python code for simplicity. Now, it is time to introduce yet another powerful player in the world of AI, JupyterLab. Remember, in the first article in this series we considered a few other alternatives to JupyterLab. But after considering various pros and cons of each and every one of them, we have decided to stick with JupyterLab. As JupyterLab is more powerful than Jupyter Notebook, our choice is further justified. There are many reasons for preferring JupyterLab over the Linux terminal while developing AI based applications. For example, the library and packages we need to use are often available by default in JupyterLab. Another reason is the ease of collaboration with a group of programmers or researchers. There are many other reasons too, and we will explore these in due course.
We have already learned how to install JupyterLab in the first article in this series. Assuming you have installed JupyterLab in your system, with the instructions given earlier, use the command jupyter lab or jupyter-lab to open JupyterLab in your default browser, like Mozilla Firefox, Google Chrome, etc. Figure 1 shows the launcher of the JupyterLab (just a part of the window) opened in my default browser. A Python shell called IPython (Interactive Python) is used in JupyterLab. But IPython has an independent existence and can be executed from your Linux terminal with the command ipython.
![Figure 1: The launcher of the JupyterLab][1]
For the time being, we will start learning JupyterLab by using Jupyter Notebook alone, just one of the many features available with JupyterLab. In Figure 1, the button to be clicked to open a Jupyter Notebook is marked with a green rectangle. On clicking this button, you may be asked to select the kernel. If you have followed the installation steps given in this series, the sole choice would be Python 3 (ipykernel). But note that you can install kernels for programming languages like C++, R, MATLAB, Julia, etc, and many more in JupyterLab. Indeed, the complete list is quite large and can be found at <https://github.com/jupyter/jupyter/wiki/Jupyter-kernels>.
![Figure 2: A Jupyter Notebook window][2]
Now let us try to understand the working of a Jupyter Notebook very quickly. Figure 2 shows a Jupyter Notebook window. Notice that the extension of the open Jupyter Notebook file is .ipynb.
In Figure 2, you can see three different options, ‘code’, ‘markdown’, and ‘raw’ (marked with a red rectangle). These are the three types of cells you can use in a Jupyter Notebook. Code cells can be used for executable code. Markdown cells can be used for entering text data, and to write comments, explanations, etc. For example, if you are a computer trainer you can create interactive content of code and explanatory text, and share it with your students.
Unlike in a terminal, you can edit and rerun code in Jupyter Notebook, which is especially handy when you make simple typographical errors. Figure 3 shows how a few lines of Python code are executed in a Jupyter Notebook.
![Figure 3: Python code executed in Jupyter Notebook][3]
The button to execute the code in a cell is marked with a blue square. In order to execute the code in a cell, select that cell and then press this button. Figure 3 shows a markdown cell marked with a red rectangle, a code cell marked with a green rectangle, and the output of the code executed marked with a yellow rectangle. In this example, the Python code just prints the value of π (Pi).
As mentioned earlier, a number of libraries and packages are available in JupyterLab by default and you don’t need to install them in the beginning. You can import these libraries into your code by using the command import. The command !pip freeze will give you the complete list of libraries and packages currently available in your JupyterLab. If a library or package is not installed, then most of the time the command pip install package/library_name_in_all_lowercase_letters will work. For example, pip install tensorflow is the command to install the library TensorFlow in JupyterLab. I will explicitly mention the few times when there is a change in the installation command for a library. More powerful features of Jupyter Notebook and JupyterLab will be introduced as we proceed through this series.
### Some complex matrix operations
Let us learn about a few more complex matrix operations. Consider the code given below. To save some space, I am not showing the output. I have added line numbers for convenience and they are not part of the code.
```
1. import numpy as np
2. A = np.arr ay([[1,2,3],[4,5,6],[7,8,88]])
3. B = np.arr ay([[1,2,3],[4,5,6],[4,5,6]])
4. print(A.T)
5. print(A.T.T)
6. print(np.trace(A))
7. print(np.linalg.det(A))
8. C=np.linalg.inv(A)
9. print(C)
10. print(A@C)
```
First, the package NumPy is imported in line 1. Then, two matrices A and B are created in lines 2 and 3. Line 4 prints the transpose of matrix A. Compare matrix A with the transpose of A to understand how the transpose operation works. Line 5 prints the transpose of the transpose of A. You will see that A and A.T.T are equivalent. This gives you another hint about the working of the operation transpose. Line 6 prints the trace of matrix A. The value printed will be 94, because trace is the sum of the diagonal (also called main diagonal) elements of matrix A. Notice that the main diagonal elements of A are 1, 5, and 88. Line 7 prints the determinant of A. When you execute this code, you will see that the answer is -237.00000000000009 (the value may change slightly in your machine). Since the determinant is non-zero, A is called a non-singular matrix. Line 8 stores the inverse of matrix A into matrix C. Line 9 prints matrix C. Line 10 prints the product of matrices A and C. On careful observation, you will see that the product is an identity matrix, a matrix where all the diagonal elements are 1 and all the other elements are 0. Note that exactly 1s and 0s will not be printed in the answer. For example, in the answer I got, there are numbers like -3.81639165e-17. Note that this is the scientific notation for a floating-point number, which is -3.81639165 × 10-17. This is -0.0000000000000000381639165, in decimal notation, which is very close to zero. Similarly, you can verify the other numbers in the answer.
On a side note, going through a tutorial on floating-point number representation in computers and the complications involved in it will definitely help you a lot and I highly recommend it. Now, by the convention adopted in the first article, we will try to classify between basic Python code and Python code for AI. In this case, all the lines of code, except lines 1 and 9, can be considered as code for AI.
Now apply lines 4 to 10 on matrix B. Lines 4 to 7 work as before. However, the determinant of B is 0 and hence it is called a singular matrix. Further, line 8 gives you an error because inverse exists only for non-singular matrices whose determinant is non-zero. Now, apply the same operations on all the eight matrices we have introduced in the previous article in this series. On seeing the outputs, you will get the additional insight that matrix operations like determinant and inverse are applicable only to square matrices.
Square matrices are matrices with an equal number of rows and columns. Though we tried to understand the working of these operations based on the above examples, I haven’t explained anything about the theory behind them. So, it would be a good idea to learn more about matrix operations like transpose, inverse, determinant, etc, if you have forgotten them. You could also learn about the different types of matrices like identity matrix, diagonal matrix, triangular matrix, symmetric matrix, skew symmetric matrix, etc. Wikipedia articles on these terms are a good starting point.
Now let us learn about some matrix operations that are even more complex, called matrix decomposition (also known as matrix factorization). First, let us recall integer factorization to understand the need for matrix factorization. The number 15 is the product of 3 and 5. But why do we need factorization? Consider the semi-prime (an integer which is the product of exactly two prime numbers) 54079637 whose factors are 6869 and 7873. I think the code and the output shown in Figure 4 give an answer to our question about the need for factorization. The line of code %time in the beginning shows us the time taken to execute a cell in Jupyter Notebook. Elementary number theory tells us that both the variables a and b contain the answer to the mathematical expression 54079637100. However, Figure 4 shows us that the answer in variable b is calculated much faster. Further, this reduction in execution time will become better and better as the number and the exponent in the above calculation increases.
![Figure 4: Execution time of a Python code][4]
Now let us move on to matrix decomposition. Similar to integer factorization, in matrix decomposition a matrix is written as the product of some other matrices. There are a lot of matrix decomposition techniques, and in almost all of them a matrix is written as the product of matrices that are more sparse than the original matrix. A sparse matrix is a matrix that has a lot of elements with value zero. Thus, after decomposition, we can work with sparse matrices rather than the original dense matrix with a lot of non-zero elements. In this article, we will discuss three decomposition techniques — LUP decomposition, eigendecomposition, and singular value decomposition (SVD).
But, in order to perform matrix decomposition, we need the services of yet another powerful Python library called SciPy. SciPy has functions for performing operations in linear algebra, integration, differentiation, optimisation, etc. In a sense, SciPy works on top of NumPy. First, let us discuss LUP decomposition. Any square matrix has an LUP decomposition. There is a variant of LUP decomposition called LU decomposition. However, not every square matrix has an LU decomposition. Hence, we discuss LUP decomposition.
In LUP decomposition, a matrix A is written as the product of three matrices L, U, and P. However, these three matrices do have some properties. L is a lower triangular matrix. A square matrix with all the entries above the main diagonal as zero is called a lower triangular matrix. U is an upper triangular matrix. A square matrix with all the entries below the main diagonal as zero is called an upper triangular matrix. P is a permutation matrix. This is a square matrix which has exactly one element with value 1 in each row and each column, and every other element of it has the value 0.
Now consider the code given below to perform LUP decomposition. Again, the line numbers have been added for convenience and are not part of the code.
```
1. import numpy as np
2. import scipy as sp
3. A=np.rray([[11,22,33],[44,55,66],[77,88,888]])
4. P, L, U = sp.linalg.lu(A)
5. print(P)
6. print(L)
7. print(U)
8. print(P@L@U)
```
Figure 5 shows the output of the code. Lines 1 and 2 import packages NumPy and SciPy. A matrix A is created in line 3. Please keep in mind that we use matrix A throughout this section. Line 4 factorizes matrix A into three matrices — P, L, and U. Lines 5 to 7 print the matrices P, L, and U. From Figure 5, it is clear that P is a permutation matrix, L is a lower triangular matrix and U is an upper triangular matrix. Finally, line 10 multiplies the three matrices and prints the product matrix. Again, from Figure 5 it is clear that the product matrix P@L@U equals the original matrix A. Hence, the decomposition (factorization) property holds true. Further, you can verify from Figure 6 that matrices L, U, and P are more sparse (a lot of zero elements) than matrix A.
![Figure 5: Output of LUP decomposition with SciPy][5]
Now let us discuss eigendecomposition, in which a square matrix is represented in terms of its eigenvalues and eigenvectors. Though it is easy to calculate eigenvalues and eigenvectors using Python, a theoretical explanation about both is beyond the scope of our discussion. However, I encourage you to learn about these concepts (if you don’t know what they are already) so that you will have a clear idea about the actual operations you are performing. Again, the Wikipedia article on eigenvalues and eigenvectors is a good starting point. Now consider Figure 6, which shows the code for eigendecomposition.
![Figure 6: Eigendecomposition with SciPy][6]
In Figure 6, line 1 finds the eigenvalues and eigenvectors. Lines 2 and 3 print them. Notice that a similar result can also be obtained with NumPy using the line of code Lambda, Q = np.linalg.eig(A). This also tells us that there is some overlap between NumPy and SciPy functions. Line 4 reconstructs the original matrix A. The code snippet np.diag(Lambda) in line 4 converts the eigenvalues into a diagonal matrix (we call it Λ). A diagonal matrix is a matrix in which all the elements outside the main diagonal are zero. The code snippet sp.linalg.inv(Q) in line 4 finds the inverse of Q (we call it Q-1). Finally, the three matrices Q, Λ, and Q-1 are multiplied together to obtain the original matrix A. Thus, in eigendecomposition A=QΛQ-1.
Figure 6 also shows the output of the code executed. The eigenvalues are marked with a red rectangle, the eigenvectors are marked with a green rectangle, and the reconstructed original matrix A is marked with a blue rectangle. But what are numbers like 11.+0.j doing in the output? Well, 11.+0.j is the complex number representation of the integer 11.
Let us now move on to singular value decomposition (SVD). It is a generalisation of eigendecomposition. Figure 7 shows the code and output of SVD. Line 1 decomposes matrix A into three matrices U, S, and V. In line 2, the code snippet ‘np.diag(S)’ converts S to a diagonal matrix. Finally, the three matrices are multiplied together to reconstruct the original matrix A. The advantage of SVD is that it can even diagonalize non-square matrices. However, the code for singular value decomposition of a non-square matrix is slightly more complicated and we won’t discuss it here for the time being.
![Figure 7: Singular value decomposition with SciPy][7]
### A few more Python libraries for AI and machine learning
We now discuss two more libraries for developing AI and machine learning based applications. There is a reason why I didn’t say Python libraries. But we will come to that later. If you speak to a layperson about AI, what do you think will be the first image that will come to his/her mind? It will probably be a Terminator like scenario, where a machine can identify a person just by looking at him/her. Computer vision is one of the most important domains where AI and machine learning based applications are deployed a lot. So we are now going to get familiar with two libraries used in computer vision — OpenCV and Matplotlib. OpenCV (open source computer vision) is a library used mainly for real-time computer vision, and is developed using C and C++. C++ is the primary interface of OpenCV and that is the reason why I didn’t call it a Python library. However, Matplotlib is a plotting library for Python. One of my earlier articles in OSFY covers Matplotlib in a more detailed manner (<https://www.opensourceforu.com/2018/05/scientific-graphics-visualisation-an-introduction-to-matplotlib>).
There is an important reason for introducing these two libraries now. I have been preaching all this while that matrices are very important, but now I am going to give you a practical example of that. To illustrate this, let us read an image from our computer to a Jupyter Notebook, for further processing. We use Matplotlib for this task. Install Matplotlib with the command pip install matplotlib, if it is not available in your Jupyter Notebook. Figure 8 shows the code and the output that reads an image.
![Figure 8: Reading and displaying an image with Matplotlib][8]
In the figure, lines 1 and 2 import some functions from Matplotlib. Notice that, if required, you can import individual functions or packages from a library instead of importing the whole library. These two lines contain basic Python code. Line 3 reads an image titled ‘OSFY-Logo.jpg’ from my computer. I have downloaded this image from the home page of the OSFY portal. This image is of height 80 pixels and width 270 pixels. Lines 4 and 5 display the image on a Jupyter Notebook window. But what is more important are the two lines of code shown below the image (marked with a red rectangle). The output of these lines of code tells us that the variable named image is actually a NumPy array. Further, it is a three-dimensional array of shape 80 x 270 x 3.
The two-dimensional array (80 x 270) part is clear from the size of the image mentioned earlier. But what about the third dimension? This is because of the fact that the image we saw is a colour image. Colour images in computers are often stored using the RGB colour model, where one layer is used for each of the three primary colours — red, green, and blue. I am sure all of you remember the experiments during our school days where primary colours were mixed to form different colours. For example, red and green when combined give yellow. The varying shades of each colour are often represented with numbers ranging from 0 (darkest) to 255 (brightest). Thus, a pixel with value (255, 255, 255) represents pure white colour.
Now, execute the command print(image). Parts of a large array will be shown in the Jupyter Notebook and you will see a lot of 255s printed in the beginning of the array. What is the reason for this? If you look at the logo of OSFY, you will understand the reason. There is a lot of white colour in the borders of the logo and hence a lot of 255s are printed in the beginning. On a side note, it is really informative to learn more about RGB colour models as well as other colour models like CMY, CMYK, HSV, etc.
We now do the reverse process. We will create an image from an array. First, consider the code shown in Figure 9. It shows how two 3 x 3 matrices filled with random values between 0 and 255 are generated. Notice that though the same code is executed twice, the results are different. A pseudo-random number generator function of NumPy called randint does this. Indeed, the chances of my winning a lottery are far more than that of the two matrices being exactly equal.
![Figure 9: Two matrices with random entries][9]
Our next plan is to generate a three-dimensional array of shape 512 x 512 x 3 and then convert it into an image. But for that we are going to use the library OpenCV. Be careful while installing OpenCV. The command for installing it is pip install opencv-python. Now consider the code shown below:
```
1. import cv2
2. img = np.random.randint(0, 256, size=(512, 512, 3))
3. cv2.imwrite(‘img.jpg’, img)
```
Line 1 imports the library OpenCV. Again, be careful, as the library name for the import statement is not opencv, unlike most other packages. Line 3 converts the matrix img to an image called img.jpg. Figure 10 shows this image generated by OpenCV. If you run this code in your system, the image will be generated in the same directory where your Jupyter Notebook is locally stored. If you check the properties of this image, you will see that it has a height of 512 pixels and width of 512 pixels. From these examples, it is easy to see that any AI or machine learning application that deals with computer vision uses a lot of arrays, vectors and matrices as well as ideas from linear algebra. Hence, our extensive treatment of arrays, vectors and matrices in this series is well justified.
![Figure 10: The image generated by OpenCV][10]
Finally, consider the code shown below. What will the output image called image.jpg look like? I will give you two hints. The function zeros in lines 4 and 5 creates two 512 x 512 arrays green and blue filled with zeros. Lines 7 to 9 fill the three-dimensional array img1 with values from the arrays red, green, and blue.
```
1. import numpy as np
2. import cv2
3. red = np.random.randint(0, 256, size=(512, 512))
4. green = np.zeros([512, 512], dtype=np.uint8)
5. blue = np.zeros([512, 512], dtype=np.uint8)
6. img1 = np.zeros([512,512,3], dtype=np.uint8)
7. img1[:,:,0] = blue
8. img1[:,:,1] = green
9. img1[:,:,2] = red
10. cv2.imwrite(‘image.jpg’, img1)
```
We will wind up our discussion for now. In the next article in this series, we will begin by briefly learning about tensors, followed by installing and using a very powerful library called TensorFlow. TensorFlow is a serious player in the world of AI and machine learning.
Afterwards, we will take a break from matrices, vectors, and linear algebra for a short while and start learning a probability theory. If linear algebra is the brain of AI then probability is its heart, or vice versa.
--------------------------------------------------------------------------------
via: https://www.opensourceforu.com/2023/07/ai-a-few-more-useful-python-libraries/
作者：[Deepu Benson][a]
选题：[lujun9972][b]
译者：[译者ID](https://github.com/译者ID)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.opensourceforu.com/author/deepu-benson/
[b]: https://github.com/lujun9972
[1]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-1-The-launcher-of-the-JupyterLab-590x499.png
[2]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-2-Jupyter-Notebook-window.png
[3]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-3-Python-code-executed-in-Jupyter-Notebook.png
[4]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-4-Execution-time-of-a-Python-code-350x301.png
[5]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-5-LUP-decomposition-with-SciPy-1-350x186.png
[6]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-6-Eigendecomposition-with-SciPy-350x175.png
[7]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-7-Singular-value-decomposition-with-SciPy-350x168.png
[8]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-8-Reading-and-displaying-an-image-with-Matplotlib--350x278.png
[9]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-9-Two-matrices-with-random-entries-350x161.png
[10]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-10-The-image-generated-by-OpenCV-350x366.png

200
translated/tech/20230727 AI- A Few More Useful Python Libraries.md Normal file
View File

@ -0,0 +1,200 @@
[#]: subject: "AI: A Few More Useful Python Libraries"
[#]: via: "https://www.opensourceforu.com/2023/07/ai-a-few-more-useful-python-libraries/"
[#]: author: "Deepu Benson https://www.opensourceforu.com/author/deepu-benson/"
[#]: collector: "lujun9972/lctt-scripts-1700446145"
[#]: translator: "toknow-gh"
[#]: reviewer: " "
[#]: publisher: " "
[#]: url: " "
人工智能教程（三）：更多有用的 Python 库
======
<https://www.opensourceforu.com/wp-content/uploads/2022/08/Python-ML.jpg>
在本系列的[上一篇文章](https://linux.cn/article-16399-1.html)中，我们回顾了人工智能的历史，然后详细地讨论了矩阵。在本系列的第三篇文章中，我们将了解更多的矩阵操作，同时再介绍几个人工智能 Python 库。
在进入主题之前，我们先讨论几个人工智能和机器学习中常用的重要术语。<ruby>人工神经网络<rt>artificial neural network</rt></ruby>（通常简称为神经网络）是机器学习和深度学习的核心。它是受人脑的生物神经网络启发而设计的计算模型。本文中我没有插入神经网络模型的图片，因为在互联网上很容易找到它们。我相信任何对人工智能感兴趣的人应该都见过它们，左边是输入层，中间是一个或多个隐藏层，右边是输出层。各层之间的边上的<ruby>权重<rt>weight</rt></ruby>会随着训练不断变化。它是机器学习和深度学习应用成功的关键。
<ruby>监督学习<rt>supervised learning</rt></ruby>和<ruby>无监督学习<rt>unsupervised learning</rt></ruby>是两个重要的机器学习模型。从长远来看，任何立志于从事人工智能或机器学习领域工作的人都需要学习它们，并了解实现它们的各种技术。这里我认为有必要简单说明两种模型之间的区别了。假设有有两个用来识别苹果和橘子的机器模型，A 和 B。它们通过 100 张苹果和橘子的图片来学习这两种水果的特征（这个过程称为模型的训练）。不过模型 A 还有照片中哪些是苹果哪些是橘子的额外信息（这个额外信息称为标签）。这里 A 是监督学习模型，B 是无监督学习模型。你认为在是识别苹果和橘子的任务上，哪个模型效果更好呢？大多数人可能会认为模型 A 的效果更好。但是根据机器学习的理论，情况并非总是如此。如果这 100 张照片中只有 5 张是苹果，其它都是橘子呢？那么模型 A 可能根本就不熟悉苹果的特征。或者如果部分标签是错误的呢？在这些情况下，B 的表现可能比 A 更好。
在实际的机器学习应用中会发生这样的情况吗？是的！训练模型用的数据集可能是不充分的或者不完整的。这只是两种模型都仍然在人工智能和机器学习领域蓬勃发展的众多原因之一。在后续文章中，我们将更正式地讨论它们。下面我们开始学习使用 JupyterLab，它是一个用于开发人工智能程序的强大工具。
### JupyterLab 入门
在本系列的前几篇文章中，为了简单起见，我们一直使用 Linux 终端运行 Python 代码。现在要介绍另一个强大的人工智能工具——JupyterLab。在本系列的第一篇文章中，我们对比了几个候选项，最终决定使用 JupyterLab。它比 Jupyter Notebook 功能更强大，为我们预装了许多库和包，并且易于团队协作。还有一些其它原因，我们将在后续适时探讨它们。
在本系列的第一篇文章中，我们已经学习了如何安装 JupyterLab。假设你已经按文中的步骤安装好了 JupyterLab，使用 `jupyter lab` 或 `jupyter-lab` 命令在会默认浏览器（如 Mozilla Firefox、谷歌 Chrome 等）中打开 JupyterLab。（LCTT 译注：没有安装 JupyterLab 也不要紧，你可以先[在线试用 JupyterLab](https://jupyter.org/try-jupyter/lab/)）图 1 是在浏览器中打开的 JupyterLab 启动器的局部截图。JupyterLab 使用一个名为 IPython（交互式 Python）的 Python 控制台。注意，IPython 其实可以独立使用，在 Linux 终端运行 `ipython` 命令就可以启动它。
![图 1：JupyterLab 启动器][1]
现阶段我们使用 JupyterLab 中的 Jupyter Notebook 功能。点击图 1 中用绿框标记的按钮，打开 Jupyter Notebook。这时可能会要求你选择内核。如果你按照本系列第一篇的步骤安装 JupyterLab，那么唯一的可选项就是 Python 3（ipykernel）。请注意，你还可以在 JupyterLab 中安装其它编程语言的内核，比如 C++、R、MATLAB、Julia 等。事实上 Jupyter 的内核相当丰富，你可以访问[Jupyter 内核清单](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels)了解更多信息。
![图 2：Jupyter Notebook 窗口][2]
下面我们快速了解一下 Jupyter Notebook 的使用。图 2 显示的是一个在浏览器中打开的 Jupyter Notebook 窗口。从浏览器标签页的标题可以看出，Jupyter Notebook 打开的文件的扩展名是 .ipynb。
在图 2 红框处可以看到有三个选项，它们表示 Jupyter Notebook 中可以使用的三种类型的单元。code 表示代码单元，它是用来执行代码的。Markdown 单元可用于输入说明性的文本。如果你是一名计算机培训师，可以用代码单元和 Markdown 单元来创建交互式代码和解释性文本，然后分享给你的学员。raw 表示原始数据单元，其中的内容不会被格式化或转换。
和在终端中不同，在 Jupyter Notebook 中你可以编辑并重新运行代码，这在处理简单的拼写错误时特别方便。图 3 是在 Jupyter Notebook 中执行 Python 代码的截图。
![图 3：在 Jupyter Notebook 中执行 Python 代码][3]
要在执行单元中的代码，先选中该单元格，然后点击蓝框标记的按钮。图 3 中用红框标记的是 Markdown 单元，用绿框标记的是代码单元，用黄框标记的执行代码的输出。在这个例子中，Python 代码输出的是 π 的值。
前面提到，JupyterLab 默认安装了许多库和包，我们不用自己安装了。你可以使用 `import` 命令将这些库导入到代码中。使用 `!pip freeze` 命令可以列出 JupyterLab 中目前可用的所有库和包。如果有库或包没有安装，大多数情况下都可以通过 `pip install <全小写的库或者包的名称>` 来安装它们。例如安装 TensorFlow 的命令是 `pip install tensorflow`。如果后面有库的安装命令不遵循这个格式，我会进行特别说明。随着本系列的继续，我们还会看到 Jupyter Notebook 和 JupyterLab 更多强大的功能。
### 复杂的矩阵运算
通过下面的代码，我们来了解一些更复杂的矩阵运算或操作。为了节省空间，我没有展示代码的输出。并且为了方便起见，我添加了行号，它们不是代码的一部分。
```
1. import numpy as np
2. A = np.arr ay([[1,2,3],[4,5,6],[7,8,88]])
3. B = np.arr ay([[1,2,3],[4,5,6],[4,5,6]])
4. print(A.T)
5. print(A.T.T)
6. print(np.trace(A))
7. print(np.linalg.det(A))
8. C = np.linalg.inv(A)
9. print(C)
10. print(A@C)
```
下面我逐行来解释这些代码：
1. 导入 NumPy 包。
2. 创建矩阵 A。
3. 创建矩阵 B。
4. 打印矩阵 A 的<ruby>转置<rt>transpose</rt></ruby>。通过比较矩阵 A 与 A 的转置，你用该可以大致理解转置操作到底做了什么。
5. 打印 A 的转置的转置。可以看到它和矩阵 A 是相同的。这又提示了转置操作的含义。
6. 打印矩阵 A 的 <ruby>迹<rt>trace</rt></ruby>。迹是矩阵的对角线（也称为主对角线）元素的和。矩阵 A 的主对角线元素是 1、5 和 88，所以输出的值是 94。
7. 打印 A 的<ruby>行列式<rt>determinant</rt></ruby>。当执行代码的结果是 -237.00000000000009（在你的电脑中可能略有区别）。因为行列式不为 0，所以称 A 为<ruby>非奇异矩阵<rt>non-singular matrix</rt></ruby>。
8. 将矩阵 A 的<ruby>逆<rt>inverse</rt></ruby> 保存到矩阵 C 中。
9. 打印矩阵 C。
10. 打印矩阵 A 和 C 的乘积。仔细观察，你会看到乘积是一个<ruby>单位矩阵<rt>identity matrix</rt></ruby>，也就是一个所有对角线元素都为 1，所有其它元素都为 0 的矩阵。请注意，输出中打印出的不是精确的 1 和 0。在我得到的答案中，有像 -3.81639165e-17 这样的数字。这是浮点数的科学记数法，表示 -3.81639165 × 10<sup>-17</sup>， 即小数的 -0.0000000000000000381639165，它非常接近于零。同样输出中的其它数字也会有这种情况。我强烈建议你了解计算机是怎样表示浮点数的，这对你会有很大帮助。
根据第一篇文章中的惯例，可以将代码分成基本 Python 代码和人工智能代码。在这个例子中，除了第 1 行和第 9 行之外的所有代码行都可以被看作是人工智能代码。
现在将第 4 行到第 10 行的操作应用到矩阵 B 上。从第 4 行到第 6 行代码的输出没有什么特别之处。然而运行第 7 行时，矩阵 B 的行列式为 0，因此它被称为<ruby>奇异矩阵<rt>singular matrix</rt></ruby>。运行第 8 行代码会给产生一个错误，因为只有非奇异矩阵才存在逆矩阵。你可以尝试对本系列前一篇文章中的 8 个矩阵都应用相同的操作。通过观察输出，你会发现矩阵的行列式和求逆运算只适用于方阵。
方阵就是行数和列数相等的矩阵。在上面的例子中我只是展示了对矩阵执行各种操作，并没有解释它们背后的理论。如果你不知道或忘记了矩阵的转置、逆、行列式等知识的话，你最好自己学习它们。同时你也应该了解一下不同类型的矩阵，比如单位矩阵、对角矩阵、三角矩阵、对称矩阵、斜对称矩阵。维基百科上的相关文章是不错的入门。
现在让我们来学习<ruby>矩阵分解<rt>matrix decomposition</rt></ruby>，它是更复杂的矩阵操作。矩阵分解与整数的因子分解类似，就是把一个矩阵被写成其它矩阵的乘积。下面我通过图 4 中整数分解的例子来解释矩阵分解的必要性。代码单元开头的 `%time` 是 Jupyter Notebook 的<ruby>魔法命令<rt>magic command</rt></ruby>，它会打印代码运行所花费的时间。`**` 是 Python 的幂运算符。基本的代数知识告诉我们，变量 a 和 b 的值都等于 (6869 x 7873)<sup>100</sup>。但图 4 显示计算变量 b 的速度要快得多。事实上，随着底数和指数的增大，执行时间的减少会越来越明显。
![图 4：Python 代码的执行耗时][4]
在几乎所有的矩阵分解技术技术中，原始矩阵都会被写成更稀疏的矩阵的乘积。<ruby>稀疏矩阵<rt>sparse matrix</rt></ruby>是指有很多元素值为零的矩阵。在分解后，我们可以处理稀疏矩阵，而不是原始的具有大量非零元素的<ruby>密集矩阵<rt>dense matrix</rt></ruby>。在本文中将介绍三种矩阵分解技术——LUP 分解、<ruby>特征分解<rt>eigen decomposition</rt></ruby>和<ruby>奇异值分解<rt>singular value decomposition</rt></ruby>（SVD）。
为了执行矩阵分解，我们需要另一个强大的 Python 库，SciPy。SciPy 是基于 NumPy 库的科学计算库，它提供了线性代数、积分、微分、优化等方面的函数。首先，让我们讨论 LUP 分解。任何方阵都能进行 LUP 分解。LUP 分解有一种变体，称为 LU 分解。但并不是所有方阵都能 LU 分解。因此这里我们只讨论 LUP 分解。
在 LUP 分解中，矩阵 A 被写成三个矩阵 L、U 和 P 的乘积。其中 L 是一个<ruby>下三角矩阵<rt>lower triangular matrix</rt></ruby>，它是主对角线以上的所有元素都为零的方阵。U 是一个<ruby>上三角矩阵<rt>upper triangular matrix</rt></ruby>，它是主对角线以下所有元素为零的方阵。P 是一个<ruby>排列矩阵<rt>permutation matrix</rt></ruby>。这是一个方阵，它的每一行和每一列中都有一个元素为 1，其它元素的值都是 0。
现在看下面的 LUP 分解的代码。同样为了方便起见我添加了行号，它们不是代码的一部分。
```
1. import numpy as np
2. import scipy as sp
3. A=np.array([[11,22,33],[44,55,66],[77,88,888]])
4. P, L, U = sp.linalg.lu(A)
5. print(P)
6. print(L)
7. print(U)
8. print(P@L@U)
```
图 5 显示了代码的输出。第 1 行和第 2 行导入 NumPy 和 SciPy 包。在第 3 行创建矩阵A。请记住，我们在本节中会一直使用矩阵 A。第 4 行将矩阵 A 分解为三个矩阵——P、L 和 U。第 5 行到第 7 行打印矩阵P、L 和 U。从图 5 中可以清楚地看出，P 是一个置换矩阵，L 是一个下三角矩阵，U 是一个上三角矩阵。最后在第 8 行将这三个矩阵相乘并打印乘积矩阵。从图 5 可以看到乘积矩阵 `P@L@U` 等于原始矩阵 A，满足矩阵分解的性质。此外，图 5 也验证了矩阵 L、U 和 P 比矩阵 A 更稀疏。
![图 5：用 SciPy 进行 LUP 分解][5]
下面我们讨论特征分解，它是将一个方阵是用它的<ruby>特征值<rt>eigenvalue</rt></ruby>和<ruby>特征向量<rt>eigenvector</rt></ruby>来表示。用 Python 计算特征值和特征向量很容易。关于特征值和特征向量的理论解释超出了本文的讨论范围，如果你不知道它们是什么，我建议你通过维基百科等先了解它们，以便对正在执行的操作有一个清晰的概念。图 6 中是特征分解的代码。
![图6：用 SciPy 进行特征分解][6]
在图 6 中，第 1 行计算特征值和特征向量。第 2 行和第 3 行输出它们。注意，使用 NumPy 也能获得类似的效果，`Lambda, Q = np.linalg.eig(A)`。这也告诉我们 NumPy 和 SciPy 的功能之间有一些重叠。第 4 行重建了原始矩阵 A。第 4 行中的代码片段 `np.diag(Lambda)` 是将特征值转换为对角矩阵（记为 Λ）。对角矩阵是主对角线以外的所有元素都为 0 的矩阵。第 4 行的代码片段 `sp.linalg.inv(Q)` 是求 Q 的逆矩阵（记为 Q<sup>-1</sup>）。最后，将三个矩阵Q、Λ、Q<sup>-1</sup> 相乘得到原始矩阵 A。也就是在特征分解中 A=QΛQ<sup>-1</sup>。
图 6 还显示了执行的代码的输出。红框标记的是特征值，用绿框标记的是特征向量，重构的矩阵 A 用蓝框标记。你可能会感到奇怪，输出中像 11.+0.j 这样的数字是什么呢？其中的 j 是虚数单位。11.+0.j 其实就是 11.0+0.0j，即整数 11 的复数形式。
现在让我们来看奇异值分解（SVD），它是特征分解的推广。图 7 显示了 SVD 的代码和输出。第 1 行将矩阵 A 分解为三个矩阵 U、S 和 V。第 2 行中的代码片段 `np.diag(S)` 将 S 转换为对角矩阵。最后，将这三个矩阵相乘重建原始矩阵 A。奇异值分解的优点是它可以对角化非方阵。但非方阵的奇异值分解的代码稍微复杂一些，我们暂时不在这里讨论它。
![图 7：用 SciPy 进行 奇异值分解][7]
### 其它人工智能和机器学习的 Python 库
当谈到人工智能时，普通人最先想到的场景可能就是电影《终结者》里机器人通过视觉识别一个人。<ruby>计算机视觉<rt>computer vision</rt></ruby>是人工智能和机器学习技术被应用得最广泛的领域之一。下面我将介绍两个计算机视觉相关的库：OpenCV 和 Matplotlib。OpenCV (open source computer vision) 是一个主要用于实时计算机视觉的库，它由 C 和 C++ 开发。C++ 是 OpenCV 的主要接口，它通过 OpenCV-Python 向用户提供 Python 接口。Matplotlib 是基于 Python 的绘图库。我曾在 OSFY 上的一篇早期文章中详细介绍了 Matplotlib 的使用(https://www.opensourceforu.com/2018/05/scientific-graphics-visualisation-an-introduction-to-matplotlib)。
前面我一直在强调矩阵的重要性，现在我用一个实际的例子来加以说明。图 8 展示了在 Jupyter Notebook 中使用 Matplotlib 读取和显示图像的代码和输出。如果你没有安装 Matplotlib，使用 `pip install matplotlib` 命令安装 Matplotlib。
![图 8：用 Matplotlib 读取和显示图像][8]
在图 8 中，第 1 行和第 2 行从 Matplotlib 导入了一些函数。注意你可以从库中导入单个函数或包，而不用导入整个库。这两行是基本的 Python 代码。第 3 行从我的计算机中读取标题为 OSFY-Logo.jpg 的图像。我从 OSFY 门户网站的首页下载了这张图片。此图像高 80 像素，宽 270 像素。第 4 行和第 5 行在 Jupyter Notebook 窗口中显示图像。请注意图像下方用红框标记的两行代码，它的输出告诉我们变量 `image` 实际上是一个 NumPy 数组。具体来说，它是一个 80 x 270 x 3 的三维数组。
数组尺寸中的 80 x 270 就是图片的大小，这一点很容易理解。但是第三维度表示什么呢？这是因计算机像通常用 RGB 颜色模型来存储的彩色图。它有三层，分别用于表示红绿蓝三种原色。我相信你还记得学生时代的实验，把原色混合成不同的颜色。例如，红色和绿色混合在一起会得到黄色。在 RGB 模型中，每种颜色的亮度用 0 到 255 的数字表示。0 表示最暗，255 表示最亮。因此值为 (255,255,255) 的像素表示纯白色。
现在，执行代码 `print(image)`， Jupyter Notebook 会将整个数组的一部分部分打印出来。你可以看到数组的开头有许多 255。这是什么原因呢？如果你仔细看 OSFY 的图标会发现，图标的边缘有很多白色区域，因此一开始就印了很多 255。顺便说一句，你还可以了解一下其他颜色模型，如 CMY、CMYK、HSV 等。
现在我们反过来从一个数组创建一幅图像。首先看图 9 中所示的代码。它展示了如何生成两个 3 x 3 的随机矩阵，它的元素是 0 到 255 之间的随机值。注意，虽然相同的代码执行了两次，但生成的结果是不同的。这是通过调用 NumPy 的伪随机数生成器函数 `randint` 实现的。实际上，我中彩票的几率都比这两个矩阵完全相等的几率大得多。
![图 8：两个随机矩阵][9]
接下来我们要生成一个形状为 512 x 512 x 3 的三维数组，然后将它转换为图像。为此我们将用到 OpenCV。注意，安装 OpenCV 命令是 `pip install opencv-python`。看下面的代码：
```
1. import cv2
2. img = np.random.randint(0, 256, size=(512, 512, 3))
3. cv2.imwrite('img.jpg', img)
```
第 1 行导入库 OpenCV。注意导入语句是 `import cv2`，这与大多数其他包的导入不同。第 3 行将矩阵 img 转换为名为 img.jpg 的图像。图 10 显示了由 OpenCV 生成的图像。在系统中运行这段代码，将图像将被保存在 Jupyter Notebook 的同一目录下。如果你查看这张图片的属性，你会看到它的高度是 512 像素，宽度是 512 像素。通过这些例子，很容易看出，任何处理计算机视觉任务的人工智能和机器学习程序使用了大量的数组、向量、矩阵以及线性代数中的思想。这也是本系列用大量篇幅介绍数组、向量和矩阵的原因。
![图 10：OpenCV 生成的图像][10]
最后，考虑下面显示的代码。image.jpg 输出图像会是什么样子？我给你两个提示。函数`zeros`在第 4 行和第 5 行创建了两个512 x 512的数组，其中绿色和蓝色填充了零。第 7 行到第 9 行用来自数组 red、green 和 blue 的值填充三维数组 img1。
```
1. import numpy as np
2. import cv2
3. red = np.random.randint(0, 256, size=(512, 512))
4. green = np.zeros([512, 512], dtype=np.uint8)
5. blue = np.zeros([512, 512], dtype=np.uint8)
6. img1 = np.zeros([512,512,3], dtype=np.uint8)
7. img1[:,:,0] = blue
8. img1[:,:,1] = green
9. img1[:,:,2] = red
10. cv2.imwrite(‘image.jpg’, img1)
```
本期的内容就到此结束了。在下一篇文章中，我们将开始简单地学习<ruby>张量<rt>tensor</rt></ruby>，然后安装和使用 TensorFlow。TensorFlow 是人工智能和机器学习领域的重要参与者。之后，我们将暂时放下矩阵、向量和线性代数，开始学习概率论。概率论跟线性代数一样是人工智能的重要基石。
--------------------------------------------------------------------------------
via: https://www.opensourceforu.com/2023/07/ai-a-few-more-useful-python-libraries/
作者：[Deepu Benson][a]
选题：[lujun9972][b]
译者：[toknow-gh](https://github.com/toknow-gh)
校对：[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
[a]: https://www.opensourceforu.com/author/deepu-benson/
[b]: https://github.com/lujun9972
[1]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-1-The-launcher-of-the-JupyterLab-590x499.png
[2]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-2-Jupyter-Notebook-window.png
[3]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-3-Python-code-executed-in-Jupyter-Notebook.png
[4]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-4-Execution-time-of-a-Python-code-350x301.png
[5]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-5-LUP-decomposition-with-SciPy-1-350x186.png
[6]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-6-Eigendecomposition-with-SciPy-350x175.png
[7]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-7-Singular-value-decomposition-with-SciPy-350x168.png
[8]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-8-Reading-and-displaying-an-image-with-Matplotlib--350x278.png
[9]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-9-Two-matrices-with-random-entries-350x161.png
[10]: https://www.opensourceforu.com/wp-content/uploads/2022/08/Figure-10-The-image-generated-by-OpenCV-350x366.png