mirror of
https://github.com/LCTT/TranslateProject.git
synced 2025-02-03 23:40:14 +08:00
commit
358c755834
@ -1,4 +1,4 @@
|
|||||||
|
Translating by cposture 2016-08-02
|
||||||
### Making predictions
|
### Making predictions
|
||||||
|
|
||||||
Now that we have the preliminaries out of the way, we’re ready to make predictions. We’ll create a new file called predict.py that will use the train.csv file we created in the last step. The below code will:
|
Now that we have the preliminaries out of the way, we’re ready to make predictions. We’ll create a new file called predict.py that will use the train.csv file we created in the last step. The below code will:
|
||||||
|
150
sources/tech/20160309 Let’s Build A Web Server. Part 1.md
Normal file
150
sources/tech/20160309 Let’s Build A Web Server. Part 1.md
Normal file
@ -0,0 +1,150 @@
|
|||||||
|
Let’s Build A Web Server. Part 1.
|
||||||
|
=====================================
|
||||||
|
|
||||||
|
Out for a walk one day, a woman came across a construction site and saw three men working. She asked the first man, “What are you doing?” Annoyed by the question, the first man barked, “Can’t you see that I’m laying bricks?” Not satisfied with the answer, she asked the second man what he was doing. The second man answered, “I’m building a brick wall.” Then, turning his attention to the first man, he said, “Hey, you just passed the end of the wall. You need to take off that last brick.” Again not satisfied with the answer, she asked the third man what he was doing. And the man said to her while looking up in the sky, “I am building the biggest cathedral this world has ever known.” While he was standing there and looking up in the sky the other two men started arguing about the errant brick. The man turned to the first two men and said, “Hey guys, don’t worry about that brick. It’s an inside wall, it will get plastered over and no one will ever see that brick. Just move on to another layer.”1
|
||||||
|
|
||||||
|
The moral of the story is that when you know the whole system and understand how different pieces fit together (bricks, walls, cathedral), you can identify and fix problems faster (errant brick).
|
||||||
|
|
||||||
|
What does it have to do with creating your own Web server from scratch?
|
||||||
|
|
||||||
|
I believe to become a better developer you MUST get a better understanding of the underlying software systems you use on a daily basis and that includes programming languages, compilers and interpreters, databases and operating systems, web servers and web frameworks. And, to get a better and deeper understanding of those systems you MUST re-build them from scratch, brick by brick, wall by wall.
|
||||||
|
|
||||||
|
Confucius put it this way:
|
||||||
|
|
||||||
|
>“I hear and I forget.”
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbasi-part4/LSBAWS_confucius_hear.png)
|
||||||
|
|
||||||
|
>“I see and I remember.”
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbasi-part4/LSBAWS_confucius_see.png)
|
||||||
|
|
||||||
|
>“I do and I understand.”
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbasi-part4/LSBAWS_confucius_do.png)
|
||||||
|
|
||||||
|
I hope at this point you’re convinced that it’s a good idea to start re-building different software systems to learn how they work.
|
||||||
|
|
||||||
|
In this three-part series I will show you how to build your own basic Web server. Let’s get started.
|
||||||
|
|
||||||
|
First things first, what is a Web server?
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part1/LSBAWS_HTTP_request_response.png)
|
||||||
|
|
||||||
|
In a nutshell it’s a networking server that sits on a physical server (oops, a server on a server) and waits for a client to send a request. When it receives a request, it generates a response and sends it back to the client. The communication between a client and a server happens using HTTP protocol. A client can be your browser or any other software that speaks HTTP.
|
||||||
|
|
||||||
|
What would a very simple implementation of a Web server look like? Here is my take on it. The example is in Python but even if you don’t know Python (it’s a very easy language to pick up, try it!) you still should be able to understand concepts from the code and explanations below:
|
||||||
|
|
||||||
|
```
|
||||||
|
import socket
|
||||||
|
|
||||||
|
HOST, PORT = '', 8888
|
||||||
|
|
||||||
|
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||||
|
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||||
|
listen_socket.bind((HOST, PORT))
|
||||||
|
listen_socket.listen(1)
|
||||||
|
print 'Serving HTTP on port %s ...' % PORT
|
||||||
|
while True:
|
||||||
|
client_connection, client_address = listen_socket.accept()
|
||||||
|
request = client_connection.recv(1024)
|
||||||
|
print request
|
||||||
|
|
||||||
|
http_response = """\
|
||||||
|
HTTP/1.1 200 OK
|
||||||
|
|
||||||
|
Hello, World!
|
||||||
|
"""
|
||||||
|
client_connection.sendall(http_response)
|
||||||
|
client_connection.close()
|
||||||
|
```
|
||||||
|
|
||||||
|
Save the above code as webserver1.py or download it directly from GitHub and run it on the command line like this
|
||||||
|
|
||||||
|
```
|
||||||
|
$ python webserver1.py
|
||||||
|
Serving HTTP on port 8888 …
|
||||||
|
```
|
||||||
|
|
||||||
|
Now type in the following URL in your Web browser’s address bar http://localhost:8888/hello, hit Enter, and see magic in action. You should see “Hello, World!” displayed in your browser like this:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part1/browser_hello_world.png)
|
||||||
|
|
||||||
|
Just do it, seriously. I will wait for you while you’re testing it.
|
||||||
|
|
||||||
|
Done? Great. Now let’s discuss how it all actually works.
|
||||||
|
|
||||||
|
First let’s start with the Web address you’ve entered. It’s called an URL and here is its basic structure:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part1/LSBAWS_URL_Web_address.png)
|
||||||
|
|
||||||
|
This is how you tell your browser the address of the Web server it needs to find and connect to and the page (path) on the server to fetch for you. Before your browser can send a HTTP request though, it first needs to establish a TCP connection with the Web server. Then it sends an HTTP request over the TCP connection to the server and waits for the server to send an HTTP response back. And when your browser receives the response it displays it, in this case it displays “Hello, World!”
|
||||||
|
|
||||||
|
Let’s explore in more detail how the client and the server establish a TCP connection before sending HTTP requests and responses. To do that they both use so-called sockets. Instead of using a browser directly you are going to simulate your browser manually by using telnet on the command line.
|
||||||
|
|
||||||
|
On the same computer you’re running the Web server fire up a telnet session on the command line specifying a host to connect to localhost and the port to connect to 8888 and then press Enter:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ telnet localhost 8888
|
||||||
|
Trying 127.0.0.1 …
|
||||||
|
Connected to localhost.
|
||||||
|
```
|
||||||
|
|
||||||
|
At this point you’ve established a TCP connection with the server running on your local host and ready to send and receive HTTP messages. In the picture below you can see a standard procedure a server has to go through to be able to accept new TCP connections.
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part1/LSBAWS_socket.png)
|
||||||
|
|
||||||
|
In the same telnet session type GET /hello HTTP/1.1 and hit Enter:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ telnet localhost 8888
|
||||||
|
Trying 127.0.0.1 …
|
||||||
|
Connected to localhost.
|
||||||
|
GET /hello HTTP/1.1
|
||||||
|
|
||||||
|
HTTP/1.1 200 OK
|
||||||
|
Hello, World!
|
||||||
|
```
|
||||||
|
|
||||||
|
You’ve just manually simulated your browser! You sent an HTTP request and got an HTTP response back. This is the basic structure of an HTTP request:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part1/LSBAWS_HTTP_request_anatomy.png)
|
||||||
|
|
||||||
|
The HTTP request consists of the line indicating the HTTP method (GET, because we are asking our server to return us something), the path /hello that indicates a “page” on the server we want and the protocol version.
|
||||||
|
|
||||||
|
For simplicity’s sake our Web server at this point completely ignores the above request line. You could just as well type in any garbage instead of “GET /hello HTTP/1.1” and you would still get back a “Hello, World!” response.
|
||||||
|
|
||||||
|
Once you’ve typed the request line and hit Enter the client sends the request to the server, the server reads the request line, prints it and returns the proper HTTP response.
|
||||||
|
|
||||||
|
Here is the HTTP response that the server sends back to your client (telnet in this case):
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part1/LSBAWS_HTTP_response_anatomy.png)
|
||||||
|
|
||||||
|
Let’s dissect it. The response consists of a status line HTTP/1.1 200 OK, followed by a required empty line, and then the HTTP response body.
|
||||||
|
|
||||||
|
The response status line HTTP/1.1 200 OK consists of the HTTP Version, the HTTP status code and the HTTP status code reason phrase OK. When the browser gets the response, it displays the body of the response and that’s why you see “Hello, World!” in your browser.
|
||||||
|
|
||||||
|
And that’s the basic model of how a Web server works. To sum it up: The Web server creates a listening socket and starts accepting new connections in a loop. The client initiates a TCP connection and, after successfully establishing it, the client sends an HTTP request to the server and the server responds with an HTTP response that gets displayed to the user. To establish a TCP connection both clients and servers use sockets.
|
||||||
|
|
||||||
|
Now you have a very basic working Web server that you can test with your browser or some other HTTP client. As you’ve seen and hopefully tried, you can also be a human HTTP client too, by using telnet and typing HTTP requests manually.
|
||||||
|
|
||||||
|
Here’s a question for you: “How do you run a Django application, Flask application, and Pyramid application under your freshly minted Web server without making a single change to the server to accommodate all those different Web frameworks?”
|
||||||
|
|
||||||
|
I will show you exactly how in Part 2 of the series. Stay tuned.
|
||||||
|
|
||||||
|
BTW, I’m writing a book “Let’s Build A Web Server: First Steps” that explains how to write a basic web server from scratch and goes into more detail on topics I just covered. Subscribe to the mailing list to get the latest updates about the book and the release date.
|
||||||
|
|
||||||
|
--------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
via: https://ruslanspivak.com/lsbaws-part1/
|
||||||
|
|
||||||
|
作者:[Ruslan][a]
|
||||||
|
译者:[译者ID](https://github.com/译者ID)
|
||||||
|
校对:[校对者ID](https://github.com/校对者ID)
|
||||||
|
|
||||||
|
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||||
|
|
||||||
|
[a]: https://linkedin.com/in/ruslanspivak/
|
||||||
|
|
||||||
|
|
||||||
|
|
427
sources/tech/20160406 Let’s Build A Web Server. Part 2.md
Normal file
427
sources/tech/20160406 Let’s Build A Web Server. Part 2.md
Normal file
@ -0,0 +1,427 @@
|
|||||||
|
Let’s Build A Web Server. Part 2.
|
||||||
|
===================================
|
||||||
|
|
||||||
|
Remember, in Part 1 I asked you a question: “How do you run a Django application, Flask application, and Pyramid application under your freshly minted Web server without making a single change to the server to accommodate all those different Web frameworks?” Read on to find out the answer.
|
||||||
|
|
||||||
|
In the past, your choice of a Python Web framework would limit your choice of usable Web servers, and vice versa. If the framework and the server were designed to work together, then you were okay:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_before_wsgi.png)
|
||||||
|
|
||||||
|
But you could have been faced (and maybe you were) with the following problem when trying to combine a server and a framework that weren’t designed to work together:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_after_wsgi.png)
|
||||||
|
|
||||||
|
Basically you had to use what worked together and not what you might have wanted to use.
|
||||||
|
|
||||||
|
So, how do you then make sure that you can run your Web server with multiple Web frameworks without making code changes either to the Web server or to the Web frameworks? And the answer to that problem became the Python Web Server Gateway Interface (or WSGI for short, pronounced “wizgy”).
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_wsgi_idea.png)
|
||||||
|
|
||||||
|
WSGI allowed developers to separate choice of a Web framework from choice of a Web server. Now you can actually mix and match Web servers and Web frameworks and choose a pairing that suits your needs. You can run Django, Flask, or Pyramid, for example, with Gunicorn or Nginx/uWSGI or Waitress. Real mix and match, thanks to the WSGI support in both servers and frameworks:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_wsgi_interop.png)
|
||||||
|
|
||||||
|
So, WSGI is the answer to the question I asked you in Part 1 and repeated at the beginning of this article. Your Web server must implement the server portion of a WSGI interface and all modern Python Web Frameworks already implement the framework side of the WSGI interface, which allows you to use them with your Web server without ever modifying your server’s code to accommodate a particular Web framework.
|
||||||
|
|
||||||
|
Now you know that WSGI support by Web servers and Web frameworks allows you to choose a pairing that suits you, but it is also beneficial to server and framework developers because they can focus on their preferred area of specialization and not step on each other’s toes. Other languages have similar interfaces too: Java, for example, has Servlet API and Ruby has Rack.
|
||||||
|
|
||||||
|
It’s all good, but I bet you are saying: “Show me the code!” Okay, take a look at this pretty minimalistic WSGI server implementation:
|
||||||
|
|
||||||
|
```
|
||||||
|
# Tested with Python 2.7.9, Linux & Mac OS X
|
||||||
|
import socket
|
||||||
|
import StringIO
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
class WSGIServer(object):
|
||||||
|
|
||||||
|
address_family = socket.AF_INET
|
||||||
|
socket_type = socket.SOCK_STREAM
|
||||||
|
request_queue_size = 1
|
||||||
|
|
||||||
|
def __init__(self, server_address):
|
||||||
|
# Create a listening socket
|
||||||
|
self.listen_socket = listen_socket = socket.socket(
|
||||||
|
self.address_family,
|
||||||
|
self.socket_type
|
||||||
|
)
|
||||||
|
# Allow to reuse the same address
|
||||||
|
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||||
|
# Bind
|
||||||
|
listen_socket.bind(server_address)
|
||||||
|
# Activate
|
||||||
|
listen_socket.listen(self.request_queue_size)
|
||||||
|
# Get server host name and port
|
||||||
|
host, port = self.listen_socket.getsockname()[:2]
|
||||||
|
self.server_name = socket.getfqdn(host)
|
||||||
|
self.server_port = port
|
||||||
|
# Return headers set by Web framework/Web application
|
||||||
|
self.headers_set = []
|
||||||
|
|
||||||
|
def set_app(self, application):
|
||||||
|
self.application = application
|
||||||
|
|
||||||
|
def serve_forever(self):
|
||||||
|
listen_socket = self.listen_socket
|
||||||
|
while True:
|
||||||
|
# New client connection
|
||||||
|
self.client_connection, client_address = listen_socket.accept()
|
||||||
|
# Handle one request and close the client connection. Then
|
||||||
|
# loop over to wait for another client connection
|
||||||
|
self.handle_one_request()
|
||||||
|
|
||||||
|
def handle_one_request(self):
|
||||||
|
self.request_data = request_data = self.client_connection.recv(1024)
|
||||||
|
# Print formatted request data a la 'curl -v'
|
||||||
|
print(''.join(
|
||||||
|
'< {line}\n'.format(line=line)
|
||||||
|
for line in request_data.splitlines()
|
||||||
|
))
|
||||||
|
|
||||||
|
self.parse_request(request_data)
|
||||||
|
|
||||||
|
# Construct environment dictionary using request data
|
||||||
|
env = self.get_environ()
|
||||||
|
|
||||||
|
# It's time to call our application callable and get
|
||||||
|
# back a result that will become HTTP response body
|
||||||
|
result = self.application(env, self.start_response)
|
||||||
|
|
||||||
|
# Construct a response and send it back to the client
|
||||||
|
self.finish_response(result)
|
||||||
|
|
||||||
|
def parse_request(self, text):
|
||||||
|
request_line = text.splitlines()[0]
|
||||||
|
request_line = request_line.rstrip('\r\n')
|
||||||
|
# Break down the request line into components
|
||||||
|
(self.request_method, # GET
|
||||||
|
self.path, # /hello
|
||||||
|
self.request_version # HTTP/1.1
|
||||||
|
) = request_line.split()
|
||||||
|
|
||||||
|
def get_environ(self):
|
||||||
|
env = {}
|
||||||
|
# The following code snippet does not follow PEP8 conventions
|
||||||
|
# but it's formatted the way it is for demonstration purposes
|
||||||
|
# to emphasize the required variables and their values
|
||||||
|
#
|
||||||
|
# Required WSGI variables
|
||||||
|
env['wsgi.version'] = (1, 0)
|
||||||
|
env['wsgi.url_scheme'] = 'http'
|
||||||
|
env['wsgi.input'] = StringIO.StringIO(self.request_data)
|
||||||
|
env['wsgi.errors'] = sys.stderr
|
||||||
|
env['wsgi.multithread'] = False
|
||||||
|
env['wsgi.multiprocess'] = False
|
||||||
|
env['wsgi.run_once'] = False
|
||||||
|
# Required CGI variables
|
||||||
|
env['REQUEST_METHOD'] = self.request_method # GET
|
||||||
|
env['PATH_INFO'] = self.path # /hello
|
||||||
|
env['SERVER_NAME'] = self.server_name # localhost
|
||||||
|
env['SERVER_PORT'] = str(self.server_port) # 8888
|
||||||
|
return env
|
||||||
|
|
||||||
|
def start_response(self, status, response_headers, exc_info=None):
|
||||||
|
# Add necessary server headers
|
||||||
|
server_headers = [
|
||||||
|
('Date', 'Tue, 31 Mar 2015 12:54:48 GMT'),
|
||||||
|
('Server', 'WSGIServer 0.2'),
|
||||||
|
]
|
||||||
|
self.headers_set = [status, response_headers + server_headers]
|
||||||
|
# To adhere to WSGI specification the start_response must return
|
||||||
|
# a 'write' callable. We simplicity's sake we'll ignore that detail
|
||||||
|
# for now.
|
||||||
|
# return self.finish_response
|
||||||
|
|
||||||
|
def finish_response(self, result):
|
||||||
|
try:
|
||||||
|
status, response_headers = self.headers_set
|
||||||
|
response = 'HTTP/1.1 {status}\r\n'.format(status=status)
|
||||||
|
for header in response_headers:
|
||||||
|
response += '{0}: {1}\r\n'.format(*header)
|
||||||
|
response += '\r\n'
|
||||||
|
for data in result:
|
||||||
|
response += data
|
||||||
|
# Print formatted response data a la 'curl -v'
|
||||||
|
print(''.join(
|
||||||
|
'> {line}\n'.format(line=line)
|
||||||
|
for line in response.splitlines()
|
||||||
|
))
|
||||||
|
self.client_connection.sendall(response)
|
||||||
|
finally:
|
||||||
|
self.client_connection.close()
|
||||||
|
|
||||||
|
|
||||||
|
SERVER_ADDRESS = (HOST, PORT) = '', 8888
|
||||||
|
|
||||||
|
|
||||||
|
def make_server(server_address, application):
|
||||||
|
server = WSGIServer(server_address)
|
||||||
|
server.set_app(application)
|
||||||
|
return server
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if len(sys.argv) < 2:
|
||||||
|
sys.exit('Provide a WSGI application object as module:callable')
|
||||||
|
app_path = sys.argv[1]
|
||||||
|
module, application = app_path.split(':')
|
||||||
|
module = __import__(module)
|
||||||
|
application = getattr(module, application)
|
||||||
|
httpd = make_server(SERVER_ADDRESS, application)
|
||||||
|
print('WSGIServer: Serving HTTP on port {port} ...\n'.format(port=PORT))
|
||||||
|
httpd.serve_forever()
|
||||||
|
```
|
||||||
|
|
||||||
|
It’s definitely bigger than the server code in Part 1, but it’s also small enough (just under 150 lines) for you to understand without getting bogged down in details. The above server also does more - it can run your basic Web application written with your beloved Web framework, be it Pyramid, Flask, Django, or some other Python WSGI framework.
|
||||||
|
|
||||||
|
Don’t believe me? Try it and see for yourself. Save the above code as webserver2.py or download it directly from GitHub. If you try to run it without any parameters it’s going to complain and exit.
|
||||||
|
|
||||||
|
```
|
||||||
|
$ python webserver2.py
|
||||||
|
Provide a WSGI application object as module:callable
|
||||||
|
```
|
||||||
|
|
||||||
|
It really wants to serve your Web application and that’s where the fun begins. To run the server the only thing you need installed is Python. But to run applications written with Pyramid, Flask, and Django you need to install those frameworks first. Let’s install all three of them. My preferred method is by using virtualenv. Just follow the steps below to create and activate a virtual environment and then install all three Web frameworks.
|
||||||
|
|
||||||
|
```
|
||||||
|
$ [sudo] pip install virtualenv
|
||||||
|
$ mkdir ~/envs
|
||||||
|
$ virtualenv ~/envs/lsbaws/
|
||||||
|
$ cd ~/envs/lsbaws/
|
||||||
|
$ ls
|
||||||
|
bin include lib
|
||||||
|
$ source bin/activate
|
||||||
|
(lsbaws) $ pip install pyramid
|
||||||
|
(lsbaws) $ pip install flask
|
||||||
|
(lsbaws) $ pip install django
|
||||||
|
```
|
||||||
|
|
||||||
|
At this point you need to create a Web application. Let’s start with Pyramid first. Save the following code as pyramidapp.py to the same directory where you saved webserver2.py or download the file directly from GitHub:
|
||||||
|
|
||||||
|
```
|
||||||
|
from pyramid.config import Configurator
|
||||||
|
from pyramid.response import Response
|
||||||
|
|
||||||
|
|
||||||
|
def hello_world(request):
|
||||||
|
return Response(
|
||||||
|
'Hello world from Pyramid!\n',
|
||||||
|
content_type='text/plain',
|
||||||
|
)
|
||||||
|
|
||||||
|
config = Configurator()
|
||||||
|
config.add_route('hello', '/hello')
|
||||||
|
config.add_view(hello_world, route_name='hello')
|
||||||
|
app = config.make_wsgi_app()
|
||||||
|
```
|
||||||
|
|
||||||
|
Now you’re ready to serve your Pyramid application with your very own Web server:
|
||||||
|
|
||||||
|
```
|
||||||
|
(lsbaws) $ python webserver2.py pyramidapp:app
|
||||||
|
WSGIServer: Serving HTTP on port 8888 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
You just told your server to load the ‘app’ callable from the python module ‘pyramidapp’ Your server is now ready to take requests and forward them to your Pyramid application. The application only handles one route now: the /hello route. Type http://localhost:8888/hello address into your browser, press Enter, and observe the result:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_browser_pyramid.png)
|
||||||
|
|
||||||
|
You can also test the server on the command line using the ‘curl’ utility:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ curl -v http://localhost:8888/hello
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Check what the server and curl prints to standard output.
|
||||||
|
|
||||||
|
Now onto Flask. Let’s follow the same steps.
|
||||||
|
|
||||||
|
```
|
||||||
|
from flask import Flask
|
||||||
|
from flask import Response
|
||||||
|
flask_app = Flask('flaskapp')
|
||||||
|
|
||||||
|
|
||||||
|
@flask_app.route('/hello')
|
||||||
|
def hello_world():
|
||||||
|
return Response(
|
||||||
|
'Hello world from Flask!\n',
|
||||||
|
mimetype='text/plain'
|
||||||
|
)
|
||||||
|
|
||||||
|
app = flask_app.wsgi_app
|
||||||
|
```
|
||||||
|
|
||||||
|
Save the above code as flaskapp.py or download it from GitHub and run the server as:
|
||||||
|
|
||||||
|
```
|
||||||
|
(lsbaws) $ python webserver2.py flaskapp:app
|
||||||
|
WSGIServer: Serving HTTP on port 8888 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Now type in the http://localhost:8888/hello into your browser and press Enter:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_browser_flask.png)
|
||||||
|
|
||||||
|
Again, try ‘curl’ and see for yourself that the server returns a message generated by the Flask application:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ curl -v http://localhost:8888/hello
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Can the server also handle a Django application? Try it out! It’s a little bit more involved, though, and I would recommend cloning the whole repo and use djangoapp.py, which is part of the GitHub repository. Here is the source code which basically adds the Django ‘helloworld’ project (pre-created using Django’s django-admin.py startproject command) to the current Python path and then imports the project’s WSGI application.
|
||||||
|
|
||||||
|
```
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, './helloworld')
|
||||||
|
from helloworld import wsgi
|
||||||
|
|
||||||
|
|
||||||
|
app = wsgi.application
|
||||||
|
```
|
||||||
|
|
||||||
|
Save the above code as djangoapp.py and run the Django application with your Web server:
|
||||||
|
|
||||||
|
```
|
||||||
|
(lsbaws) $ python webserver2.py djangoapp:app
|
||||||
|
WSGIServer: Serving HTTP on port 8888 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Type in the following address and press Enter:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_browser_django.png)
|
||||||
|
|
||||||
|
And as you’ve already done a couple of times before, you can test it on the command line, too, and confirm that it’s the Django application that handles your requests this time around:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ curl -v http://localhost:8888/hello
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Did you try it? Did you make sure the server works with those three frameworks? If not, then please do so. Reading is important, but this series is about rebuilding and that means you need to get your hands dirty. Go and try it. I will wait for you, don’t worry. No seriously, you must try it and, better yet, retype everything yourself and make sure that it works as expected.
|
||||||
|
|
||||||
|
Okay, you’ve experienced the power of WSGI: it allows you to mix and match your Web servers and Web frameworks. WSGI provides a minimal interface between Python Web servers and Python Web Frameworks. It’s very simple and it’s easy to implement on both the server and the framework side. The following code snippet shows the server and the framework side of the interface:
|
||||||
|
|
||||||
|
```
|
||||||
|
def run_application(application):
|
||||||
|
"""Server code."""
|
||||||
|
# This is where an application/framework stores
|
||||||
|
# an HTTP status and HTTP response headers for the server
|
||||||
|
# to transmit to the client
|
||||||
|
headers_set = []
|
||||||
|
# Environment dictionary with WSGI/CGI variables
|
||||||
|
environ = {}
|
||||||
|
|
||||||
|
def start_response(status, response_headers, exc_info=None):
|
||||||
|
headers_set[:] = [status, response_headers]
|
||||||
|
|
||||||
|
# Server invokes the ‘application' callable and gets back the
|
||||||
|
# response body
|
||||||
|
result = application(environ, start_response)
|
||||||
|
# Server builds an HTTP response and transmits it to the client
|
||||||
|
…
|
||||||
|
|
||||||
|
def app(environ, start_response):
|
||||||
|
"""A barebones WSGI app."""
|
||||||
|
start_response('200 OK', [('Content-Type', 'text/plain')])
|
||||||
|
return ['Hello world!']
|
||||||
|
|
||||||
|
run_application(app)
|
||||||
|
```
|
||||||
|
|
||||||
|
Here is how it works:
|
||||||
|
|
||||||
|
1. The framework provides an ‘application’ callable (The WSGI specification doesn’t prescribe how that should be implemented)
|
||||||
|
2. The server invokes the ‘application’ callable for each request it receives from an HTTP client. It passes a dictionary ‘environ’ containing WSGI/CGI variables and a ‘start_response’ callable as arguments to the ‘application’ callable.
|
||||||
|
3. The framework/application generates an HTTP status and HTTP response headers and passes them to the ‘start_response’ callable for the server to store them. The framework/application also returns a response body.
|
||||||
|
4. The server combines the status, the response headers, and the response body into an HTTP response and transmits it to the client (This step is not part of the specification but it’s the next logical step in the flow and I added it for clarity)
|
||||||
|
|
||||||
|
And here is a visual representation of the interface:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_wsgi_interface.png)
|
||||||
|
|
||||||
|
So far, you’ve seen the Pyramid, Flask, and Django Web applications and you’ve seen the server code that implements the server side of the WSGI specification. You’ve even seen the barebones WSGI application code snippet that doesn’t use any framework.
|
||||||
|
|
||||||
|
The thing is that when you write a Web application using one of those frameworks you work at a higher level and don’t work with WSGI directly, but I know you’re curious about the framework side of the WSGI interface, too because you’re reading this article. So, let’s create a minimalistic WSGI Web application/Web framework without using Pyramid, Flask, or Django and run it with your server:
|
||||||
|
|
||||||
|
```
|
||||||
|
def app(environ, start_response):
|
||||||
|
"""A barebones WSGI application.
|
||||||
|
|
||||||
|
This is a starting point for your own Web framework :)
|
||||||
|
"""
|
||||||
|
status = '200 OK'
|
||||||
|
response_headers = [('Content-Type', 'text/plain')]
|
||||||
|
start_response(status, response_headers)
|
||||||
|
return ['Hello world from a simple WSGI application!\n']
|
||||||
|
```
|
||||||
|
|
||||||
|
Again, save the above code in wsgiapp.py file or download it from GitHub directly and run the application under your Web server as:
|
||||||
|
|
||||||
|
```
|
||||||
|
(lsbaws) $ python webserver2.py wsgiapp:app
|
||||||
|
WSGIServer: Serving HTTP on port 8888 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Type in the following address and press Enter. This is the result you should see:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_browser_simple_wsgi_app.png)
|
||||||
|
|
||||||
|
You just wrote your very own minimalistic WSGI Web framework while learning about how to create a Web server! Outrageous.
|
||||||
|
|
||||||
|
Now, let’s get back to what the server transmits to the client. Here is the HTTP response the server generates when you call your Pyramid application using an HTTP client:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_http_response.png)
|
||||||
|
|
||||||
|
The response has some familiar parts that you saw in Part 1 but it also has something new. It has, for example, four HTTP headers that you haven’t seen before: Content-Type, Content-Length, Date, and Server. Those are the headers that a response from a Web server generally should have. None of them are strictly required, though. The purpose of the headers is to transmit additional information about the HTTP request/response.
|
||||||
|
|
||||||
|
Now that you know more about the WSGI interface, here is the same HTTP response with some more information about what parts produced it:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_http_response_explanation.png)
|
||||||
|
|
||||||
|
I haven’t said anything about the ‘environ’ dictionary yet, but basically it’s a Python dictionary that must contain certain WSGI and CGI variables prescribed by the WSGI specification. The server takes the values for the dictionary from the HTTP request after parsing the request. This is what the contents of the dictionary look like:
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_environ.png)
|
||||||
|
|
||||||
|
A Web framework uses the information from that dictionary to decide which view to use based on the specified route, request method etc., where to read the request body from and where to write errors, if any.
|
||||||
|
|
||||||
|
By now you’ve created your own WSGI Web server and you’ve made Web applications written with different Web frameworks. And, you’ve also created your barebones Web application/Web framework along the way. It’s been a heck of a journey. Let’s recap what your WSGI Web server has to do to serve requests aimed at a WSGI application:
|
||||||
|
|
||||||
|
- First, the server starts and loads an ‘application’ callable provided by your Web framework/application
|
||||||
|
- Then, the server reads a request
|
||||||
|
- Then, the server parses it
|
||||||
|
- Then, it builds an ‘environ’ dictionary using the request data
|
||||||
|
- Then, it calls the ‘application’ callable with the ‘environ’ dictionary and a ‘start_response’ callable as parameters and gets back a response body.
|
||||||
|
- Then, the server constructs an HTTP response using the data returned by the call to the ‘application’ object and the status and response headers set by the ‘start_response’ callable.
|
||||||
|
- And finally, the server transmits the HTTP response back to the client
|
||||||
|
|
||||||
|
![](https://ruslanspivak.com/lsbaws-part2/lsbaws_part2_server_summary.png)
|
||||||
|
|
||||||
|
That’s about all there is to it. You now have a working WSGI server that can serve basic Web applications written with WSGI compliant Web frameworks like Django, Flask, Pyramid, or your very own WSGI framework. The best part is that the server can be used with multiple Web frameworks without any changes to the server code base. Not bad at all.
|
||||||
|
|
||||||
|
Before you go, here is another question for you to think about, “How do you make your server handle more than one request at a time?”
|
||||||
|
|
||||||
|
Stay tuned and I will show you a way to do that in Part 3. Cheers!
|
||||||
|
|
||||||
|
BTW, I’m writing a book “Let’s Build A Web Server: First Steps” that explains how to write a basic web server from scratch and goes into more detail on topics I just covered. Subscribe to the mailing list to get the latest updates about the book and the release date.
|
||||||
|
|
||||||
|
--------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
via: https://ruslanspivak.com/lsbaws-part2/
|
||||||
|
|
||||||
|
作者:[Ruslan][a]
|
||||||
|
译者:[译者ID](https://github.com/译者ID)
|
||||||
|
校对:[校对者ID](https://github.com/校对者ID)
|
||||||
|
|
||||||
|
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||||
|
|
||||||
|
[a]: https://github.com/rspivak/
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1,33 +1,36 @@
|
|||||||
An Introduction to Mocking in Python
|
Mock 在 Python 中的使用介绍
|
||||||
=====================================
|
=====================================
|
||||||
|
http://www.oschina.net/translate/an-introduction-to-mocking-in-python?cmp
|
||||||
|
本文讲述的是 Python 中 Mock 的使用
|
||||||
|
|
||||||
This article is about mocking in python,
|
**如何在避免测试你的耐心的情景下执行单元测试**
|
||||||
|
|
||||||
**How to Run Unit Tests Without Testing Your Patience**
|
通常,我们编写的软件会直接与我们称之为肮脏无比的服务交互。用外行人的话说:交互已设计好的服务对我们的应用程序很重要,但是这会带来我们不希望的副作用,也就是那些在我们自己测试的时候不希望的功能。例如:我们正在写一个社交 app,并且想要测试一下我们 "发布到 Facebook" 的新功能,但是不想每次运行测试集的时候真的发布到 Facebook。
|
||||||
|
|
||||||
More often than not, the software we write directly interacts with what we would label as “dirty” services. In layman’s terms: services that are crucial to our application, but whose interactions have intended but undesired side-effects—that is, undesired in the context of an autonomous test run.For example: perhaps we’re writing a social app and want to test out our new ‘Post to Facebook feature’, but don’t want to actually post to Facebook every time we run our test suite.
|
|
||||||
|
|
||||||
The Python unittest library includes a subpackage named unittest.mock—or if you declare it as a dependency, simply mock—which provides extremely powerful and useful means by which to mock and stub out these undesired side-effects.
|
Python 的单元测试库包含了一个名为 unittest.mock 或者可以称之为依赖的子包,简言之为 mock——其提供了极其强大和有用的方法,通过它们可以模拟和打桩我们不希望的副作用。
|
||||||
|
|
||||||
|
|
||||||
>Source | <http://www.toptal.com/python/an-introduction-to-mocking-in-python>
|
>Source | <http://www.toptal.com/python/an-introduction-to-mocking-in-python>
|
||||||
|
|
||||||
Note: mock is [newly included][1] in the standard library as of Python 3.3; prior distributions will have to use the Mock library downloadable via [PyPI][2].
|
注意:mock [最近收录][1]到了 Python 3.3 的标准库中;先前发布的版本必须通过 [PyPI][2] 下载 Mock 库。
|
||||||
|
|
||||||
|
###
|
||||||
### Fear System Calls
|
### Fear System Calls
|
||||||
|
|
||||||
To give you another example, and one that we’ll run with for the rest of the article, consider system calls. It’s not difficult to see that these are prime candidates for mocking: whether you’re writing a script to eject a CD drive, a web server which removes antiquated cache files from /tmp, or a socket server which binds to a TCP port, these calls all feature undesired side-effects in the context of your unit-tests.
|
再举另一个例子,思考一个我们会在余文讨论的系统调用。不难发现,这些系统调用都是主要的模拟对象:无论你是正在写一个可以弹出 CD 驱动的脚本,还是一个用来删除 /tmp 下过期的缓存文件的 Web 服务,这些调用都是在你的单元测试上下文中不希望的副作用。
|
||||||
|
|
||||||
>As a developer, you care more that your library successfully called the system function for ejecting a CD as opposed to experiencing your CD tray open every time a test is run.
|
> 作为一个开发者,你需要更关心你的库是否成功地调用了一个可以弹出 CD 的系统函数,而不是切身经历 CD 托盘每次在测试执行的时候都打开了。
|
||||||
|
|
||||||
As a developer, you care more that your library successfully called the system function for ejecting a CD (with the correct arguments, etc.) as opposed to actually experiencing your CD tray open every time a test is run. (Or worse, multiple times, as multiple tests reference the eject code during a single unit-test run!)
|
作为一个开发者,你需要更关心你的库是否成功地调用了一个可以弹出 CD 的系统函数(使用了正确的参数等等),而不是切身经历 CD 托盘每次在测试执行的时候都打开了。(或者更糟糕的是,很多次,在一个单元测试运行期间多个测试都引用了弹出代码!)
|
||||||
|
|
||||||
Likewise, keeping your unit-tests efficient and performant means keeping as much “slow code” out of the automated test runs, namely filesystem and network access.
|
同样,保持你的单元测试的效率和性能意味着需要让如此多的 "缓慢代码" 远离自动测试,比如文件系统和网络访问。
|
||||||
|
|
||||||
For our first example, we’ll refactor a standard Python test case from original form to one using mock. We’ll demonstrate how writing a test case with mocks will make our tests smarter, faster, and able to reveal more about how the software works.
|
对于我们首个例子,我们要从原始形式到使用 mock 地重构一个标准 Python 测试用例。我们会演示如何使用 mock 写一个测试用例使我们的测试更加智能、快速,并且能展示更多关于我们软件的工作原理。
|
||||||
|
|
||||||
### A Simple Delete Function
|
### 一个简单的删除函数
|
||||||
|
|
||||||
We all need to delete files from our filesystem from time to time, so let’s write a function in Python which will make it a bit easier for our scripts to do so.
|
有时,我们都需要从文件系统中删除文件,因此,让我们在 Python 中写一个可以使我们的脚本更加轻易完成此功能的函数。
|
||||||
|
|
||||||
```
|
```
|
||||||
#!/usr/bin/env python
|
#!/usr/bin/env python
|
||||||
@ -39,9 +42,9 @@ def rm(filename):
|
|||||||
os.remove(filename)
|
os.remove(filename)
|
||||||
```
|
```
|
||||||
|
|
||||||
Obviously, our rm method at this point in time doesn’t provide much more than the underlying os.remove method, but our codebase will improve, allowing us to add more functionality here.
|
很明显,我们的 rm 方法此时无法提供比相关 os.remove 方法更多的功能,但我们的基础代码会逐步改善,允许我们在这里添加更多的功能。
|
||||||
|
|
||||||
Let’s write a traditional test case, i.e., without mocks:
|
让我们写一个传统的测试用例,即,没有使用 mock:
|
||||||
|
|
||||||
```
|
```
|
||||||
#!/usr/bin/env python
|
#!/usr/bin/env python
|
||||||
@ -60,7 +63,7 @@ class RmTestCase(unittest.TestCase):
|
|||||||
def setUp(self):
|
def setUp(self):
|
||||||
with open(self.tmpfilepath, "wb") as f:
|
with open(self.tmpfilepath, "wb") as f:
|
||||||
f.write("Delete me!")
|
f.write("Delete me!")
|
||||||
|
|
||||||
def test_rm(self):
|
def test_rm(self):
|
||||||
# remove the file
|
# remove the file
|
||||||
rm(self.tmpfilepath)
|
rm(self.tmpfilepath)
|
||||||
@ -68,9 +71,11 @@ class RmTestCase(unittest.TestCase):
|
|||||||
self.assertFalse(os.path.isfile(self.tmpfilepath), "Failed to remove the file.")
|
self.assertFalse(os.path.isfile(self.tmpfilepath), "Failed to remove the file.")
|
||||||
```
|
```
|
||||||
|
|
||||||
Our test case is pretty simple, but every time it is run, a temporary file is created and then deleted. Additionally, we have no way of testing whether our rm method properly passes the argument down to the os.remove call. We can assume that it does based on the test above, but much is left to be desired.
|
我们的测试用例相当简单,但是当它每次运行的时候,它都会创建一个临时文件并且随后删除。此外,我们没有办法测试我们的 rm 方法是否正确地将我们的参数向下传递给 os.remove 调用。我们可以基于以上的测试认为它做到了,但还有很多需要改进的地方。
|
||||||
|
|
||||||
Refactoring with MocksLet’s refactor our test case using mock:
|
### 使用 Mock 重构
|
||||||
|
|
||||||
|
让我们使用 mock 重构我们的测试用例:
|
||||||
|
|
||||||
```
|
```
|
||||||
#!/usr/bin/env python
|
#!/usr/bin/env python
|
||||||
@ -82,7 +87,7 @@ import mock
|
|||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
class RmTestCase(unittest.TestCase):
|
class RmTestCase(unittest.TestCase):
|
||||||
|
|
||||||
@mock.patch('mymodule.os')
|
@mock.patch('mymodule.os')
|
||||||
def test_rm(self, mock_os):
|
def test_rm(self, mock_os):
|
||||||
rm("any path")
|
rm("any path")
|
||||||
@ -90,25 +95,30 @@ class RmTestCase(unittest.TestCase):
|
|||||||
mock_os.remove.assert_called_with("any path")
|
mock_os.remove.assert_called_with("any path")
|
||||||
```
|
```
|
||||||
|
|
||||||
With these refactors, we have fundamentally changed the way that the test operates. Now, we have an insider, an object we can use to verify the functionality of another.
|
使用这些重构,我们从根本上改变了该测试用例的运行方式。现在,我们有一个可以用于验证其他功能的内部对象。
|
||||||
|
|
||||||
### Potential Pitfalls
|
### 潜在陷阱
|
||||||
|
|
||||||
|
第一件需要注意的事情就是,我们使用了位于 mymodule.os 且用于模拟对象的 mock.patch 方法装饰器,并且将该 mock 注入到我们的测试用例方法。相比在 mymodule.os 引用它,那么只是模拟 os 本身,会不会更有意义呢?
|
||||||
One of the first things that should stick out is that we’re using the mock.patch method decorator to mock an object located at mymodule.os, and injecting that mock into our test case method. Wouldn’t it make more sense to just mock os itself, rather than the reference to it at mymodule.os?
|
One of the first things that should stick out is that we’re using the mock.patch method decorator to mock an object located at mymodule.os, and injecting that mock into our test case method. Wouldn’t it make more sense to just mock os itself, rather than the reference to it at mymodule.os?
|
||||||
|
|
||||||
Well, Python is somewhat of a sneaky snake when it comes to imports and managing modules. At runtime, the mymodule module has its own os which is imported into its own local scope in the module. Thus, if we mock os, we won’t see the effects of the mock in the mymodule module.
|
当然,当涉及到导入和管理模块,Python 的用法非常灵活。在运行时,mymodule 模块拥有被导入到本模块局部作用域的 os。因此,如果我们模拟 os,我们是看不到模拟在 mymodule 模块中的作用的。
|
||||||
|
|
||||||
The mantra to keep repeating is this:
|
这句话需要深刻地记住:
|
||||||
|
|
||||||
|
> 模拟测试一个项目,只需要了解它用在哪里,而不是它从哪里来。
|
||||||
> Mock an item where it is used, not where it came from.
|
> Mock an item where it is used, not where it came from.
|
||||||
|
|
||||||
|
如果你需要为 myproject.app.MyElaborateClass 模拟 tempfile 模块,你可能需要
|
||||||
If you need to mock the tempfile module for myproject.app.MyElaborateClass, you probably need to apply the mock to myproject.app.tempfile, as each module keeps its own imports.
|
If you need to mock the tempfile module for myproject.app.MyElaborateClass, you probably need to apply the mock to myproject.app.tempfile, as each module keeps its own imports.
|
||||||
|
|
||||||
|
先将那个陷阱置身事外,让我们继续模拟。
|
||||||
With that pitfall out of the way, let’s keep mocking.
|
With that pitfall out of the way, let’s keep mocking.
|
||||||
|
|
||||||
### Adding Validation to ‘rm’
|
### 向 ‘rm’ 中加入验证
|
||||||
|
|
||||||
|
之前定义的 rm 方法相当的简单。在盲目地删除之前,我们倾向于拿它来验证一个路径是否存在,并验证其是否是一个文件。让我们重构 rm 使其变得更加智能:
|
||||||
|
|
||||||
The rm method defined earlier is quite oversimplified. We’d like to have it validate that a path exists and is a file before just blindly attempting to remove it. Let’s refactor rm to be a bit smarter:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
#!/usr/bin/env python
|
#!/usr/bin/env python
|
||||||
@ -122,7 +132,7 @@ def rm(filename):
|
|||||||
os.remove(filename)
|
os.remove(filename)
|
||||||
```
|
```
|
||||||
|
|
||||||
Great. Now, let’s adjust our test case to keep coverage up.
|
很好。现在,让我们调整测试用例来保持测试的覆盖程度。
|
||||||
|
|
||||||
```
|
```
|
||||||
#!/usr/bin/env python
|
#!/usr/bin/env python
|
||||||
@ -134,33 +144,33 @@ import mock
|
|||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
class RmTestCase(unittest.TestCase):
|
class RmTestCase(unittest.TestCase):
|
||||||
|
|
||||||
@mock.patch('mymodule.os.path')
|
@mock.patch('mymodule.os.path')
|
||||||
@mock.patch('mymodule.os')
|
@mock.patch('mymodule.os')
|
||||||
def test_rm(self, mock_os, mock_path):
|
def test_rm(self, mock_os, mock_path):
|
||||||
# set up the mock
|
# set up the mock
|
||||||
mock_path.isfile.return_value = False
|
mock_path.isfile.return_value = False
|
||||||
|
|
||||||
rm("any path")
|
rm("any path")
|
||||||
|
|
||||||
# test that the remove call was NOT called.
|
# test that the remove call was NOT called.
|
||||||
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
||||||
|
|
||||||
# make the file 'exist'
|
# make the file 'exist'
|
||||||
mock_path.isfile.return_value = True
|
mock_path.isfile.return_value = True
|
||||||
|
|
||||||
rm("any path")
|
rm("any path")
|
||||||
|
|
||||||
mock_os.remove.assert_called_with("any path")
|
mock_os.remove.assert_called_with("any path")
|
||||||
```
|
```
|
||||||
|
|
||||||
Our testing paradigm has completely changed. We now can verify and validate internal functionality of methods without any side-effects.
|
我们的测试用例完全改变了。现在我们可以在没有任何副作用下核实并验证方法的内部功能。
|
||||||
|
|
||||||
### File-Removal as a Service
|
### 将文件删除作为服务
|
||||||
|
|
||||||
So far, we’ve only been working with supplying mocks for functions, but not for methods on objects or cases where mocking is necessary for sending parameters. Let’s cover object methods first.
|
到目前为止,我们只是对函数功能提供模拟测试,并没对需要传递参数的对象和实例的方法进行模拟测试。接下来我们将介绍如何对对象的方法进行模拟测试。
|
||||||
|
|
||||||
We’ll begin with a refactor of the rm method into a service class. There really isn’t a justifiable need, per se, to encapsulate such a simple function into an object, but it will at the very least help us demonstrate key concepts in mock. Let’s refactor:
|
首先,我们将rm方法重构成一个服务类。实际上将这样一个简单的函数转换成一个对象,在本质上,这不是一个合理的需求,但它能够帮助我们了解mock的关键概念。让我们开始重构:
|
||||||
|
|
||||||
```
|
```
|
||||||
#!/usr/bin/env python
|
#!/usr/bin/env python
|
||||||
@ -177,6 +187,7 @@ class RemovalService(object):
|
|||||||
os.remove(filename)
|
os.remove(filename)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 你会注意到我们的测试用例没有太大的变化
|
||||||
### You’ll notice that not much has changed in our test case:
|
### You’ll notice that not much has changed in our test case:
|
||||||
|
|
||||||
```
|
```
|
||||||
@ -189,29 +200,30 @@ import mock
|
|||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
class RemovalServiceTestCase(unittest.TestCase):
|
class RemovalServiceTestCase(unittest.TestCase):
|
||||||
|
|
||||||
@mock.patch('mymodule.os.path')
|
@mock.patch('mymodule.os.path')
|
||||||
@mock.patch('mymodule.os')
|
@mock.patch('mymodule.os')
|
||||||
def test_rm(self, mock_os, mock_path):
|
def test_rm(self, mock_os, mock_path):
|
||||||
# instantiate our service
|
# instantiate our service
|
||||||
reference = RemovalService()
|
reference = RemovalService()
|
||||||
|
|
||||||
# set up the mock
|
# set up the mock
|
||||||
mock_path.isfile.return_value = False
|
mock_path.isfile.return_value = False
|
||||||
|
|
||||||
reference.rm("any path")
|
reference.rm("any path")
|
||||||
|
|
||||||
# test that the remove call was NOT called.
|
# test that the remove call was NOT called.
|
||||||
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
||||||
|
|
||||||
# make the file 'exist'
|
# make the file 'exist'
|
||||||
mock_path.isfile.return_value = True
|
mock_path.isfile.return_value = True
|
||||||
|
|
||||||
reference.rm("any path")
|
reference.rm("any path")
|
||||||
|
|
||||||
mock_os.remove.assert_called_with("any path")
|
mock_os.remove.assert_called_with("any path")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
很好,我们知道 RemovalService 会如期工作。接下来让我们创建另一个服务,将其声明为一个依赖
|
||||||
Great, so we now know that the RemovalService works as planned. Let’s create another service which declares it as a dependency:
|
Great, so we now know that the RemovalService works as planned. Let’s create another service which declares it as a dependency:
|
||||||
|
|
||||||
```
|
```
|
||||||
@ -227,13 +239,13 @@ class RemovalService(object):
|
|||||||
def rm(self, filename):
|
def rm(self, filename):
|
||||||
if os.path.isfile(filename):
|
if os.path.isfile(filename):
|
||||||
os.remove(filename)
|
os.remove(filename)
|
||||||
|
|
||||||
|
|
||||||
class UploadService(object):
|
class UploadService(object):
|
||||||
|
|
||||||
def __init__(self, removal_service):
|
def __init__(self, removal_service):
|
||||||
self.removal_service = removal_service
|
self.removal_service = removal_service
|
||||||
|
|
||||||
def upload_complete(self, filename):
|
def upload_complete(self, filename):
|
||||||
self.removal_service.rm(filename)
|
self.removal_service.rm(filename)
|
||||||
```
|
```
|
||||||
@ -261,29 +273,29 @@ import mock
|
|||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
class RemovalServiceTestCase(unittest.TestCase):
|
class RemovalServiceTestCase(unittest.TestCase):
|
||||||
|
|
||||||
@mock.patch('mymodule.os.path')
|
@mock.patch('mymodule.os.path')
|
||||||
@mock.patch('mymodule.os')
|
@mock.patch('mymodule.os')
|
||||||
def test_rm(self, mock_os, mock_path):
|
def test_rm(self, mock_os, mock_path):
|
||||||
# instantiate our service
|
# instantiate our service
|
||||||
reference = RemovalService()
|
reference = RemovalService()
|
||||||
|
|
||||||
# set up the mock
|
# set up the mock
|
||||||
mock_path.isfile.return_value = False
|
mock_path.isfile.return_value = False
|
||||||
|
|
||||||
reference.rm("any path")
|
reference.rm("any path")
|
||||||
|
|
||||||
# test that the remove call was NOT called.
|
# test that the remove call was NOT called.
|
||||||
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
||||||
|
|
||||||
# make the file 'exist'
|
# make the file 'exist'
|
||||||
mock_path.isfile.return_value = True
|
mock_path.isfile.return_value = True
|
||||||
|
|
||||||
reference.rm("any path")
|
reference.rm("any path")
|
||||||
|
|
||||||
mock_os.remove.assert_called_with("any path")
|
mock_os.remove.assert_called_with("any path")
|
||||||
|
|
||||||
|
|
||||||
class UploadServiceTestCase(unittest.TestCase):
|
class UploadServiceTestCase(unittest.TestCase):
|
||||||
|
|
||||||
@mock.patch.object(RemovalService, 'rm')
|
@mock.patch.object(RemovalService, 'rm')
|
||||||
@ -291,13 +303,13 @@ class UploadServiceTestCase(unittest.TestCase):
|
|||||||
# build our dependencies
|
# build our dependencies
|
||||||
removal_service = RemovalService()
|
removal_service = RemovalService()
|
||||||
reference = UploadService(removal_service)
|
reference = UploadService(removal_service)
|
||||||
|
|
||||||
# call upload_complete, which should, in turn, call `rm`:
|
# call upload_complete, which should, in turn, call `rm`:
|
||||||
reference.upload_complete("my uploaded file")
|
reference.upload_complete("my uploaded file")
|
||||||
|
|
||||||
# check that it called the rm method of any RemovalService
|
# check that it called the rm method of any RemovalService
|
||||||
mock_rm.assert_called_with("my uploaded file")
|
mock_rm.assert_called_with("my uploaded file")
|
||||||
|
|
||||||
# check that it called the rm method of _our_ removal_service
|
# check that it called the rm method of _our_ removal_service
|
||||||
removal_service.rm.assert_called_with("my uploaded file")
|
removal_service.rm.assert_called_with("my uploaded file")
|
||||||
```
|
```
|
||||||
@ -338,39 +350,39 @@ import mock
|
|||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
class RemovalServiceTestCase(unittest.TestCase):
|
class RemovalServiceTestCase(unittest.TestCase):
|
||||||
|
|
||||||
@mock.patch('mymodule.os.path')
|
@mock.patch('mymodule.os.path')
|
||||||
@mock.patch('mymodule.os')
|
@mock.patch('mymodule.os')
|
||||||
def test_rm(self, mock_os, mock_path):
|
def test_rm(self, mock_os, mock_path):
|
||||||
# instantiate our service
|
# instantiate our service
|
||||||
reference = RemovalService()
|
reference = RemovalService()
|
||||||
|
|
||||||
# set up the mock
|
# set up the mock
|
||||||
mock_path.isfile.return_value = False
|
mock_path.isfile.return_value = False
|
||||||
|
|
||||||
reference.rm("any path")
|
reference.rm("any path")
|
||||||
|
|
||||||
# test that the remove call was NOT called.
|
# test that the remove call was NOT called.
|
||||||
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")
|
||||||
|
|
||||||
# make the file 'exist'
|
# make the file 'exist'
|
||||||
mock_path.isfile.return_value = True
|
mock_path.isfile.return_value = True
|
||||||
|
|
||||||
reference.rm("any path")
|
reference.rm("any path")
|
||||||
|
|
||||||
mock_os.remove.assert_called_with("any path")
|
mock_os.remove.assert_called_with("any path")
|
||||||
|
|
||||||
|
|
||||||
class UploadServiceTestCase(unittest.TestCase):
|
class UploadServiceTestCase(unittest.TestCase):
|
||||||
|
|
||||||
def test_upload_complete(self, mock_rm):
|
def test_upload_complete(self, mock_rm):
|
||||||
# build our dependencies
|
# build our dependencies
|
||||||
mock_removal_service = mock.create_autospec(RemovalService)
|
mock_removal_service = mock.create_autospec(RemovalService)
|
||||||
reference = UploadService(mock_removal_service)
|
reference = UploadService(mock_removal_service)
|
||||||
|
|
||||||
# call upload_complete, which should, in turn, call `rm`:
|
# call upload_complete, which should, in turn, call `rm`:
|
||||||
reference.upload_complete("my uploaded file")
|
reference.upload_complete("my uploaded file")
|
||||||
|
|
||||||
# test that it called the rm method
|
# test that it called the rm method
|
||||||
mock_removal_service.rm.assert_called_with("my uploaded file")
|
mock_removal_service.rm.assert_called_with("my uploaded file")
|
||||||
```
|
```
|
||||||
@ -426,7 +438,7 @@ To finish up, let’s write a more applicable real-world example, one which we m
|
|||||||
import facebook
|
import facebook
|
||||||
|
|
||||||
class SimpleFacebook(object):
|
class SimpleFacebook(object):
|
||||||
|
|
||||||
def __init__(self, oauth_token):
|
def __init__(self, oauth_token):
|
||||||
self.graph = facebook.GraphAPI(oauth_token)
|
self.graph = facebook.GraphAPI(oauth_token)
|
||||||
|
|
||||||
@ -444,7 +456,7 @@ import mock
|
|||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
class SimpleFacebookTestCase(unittest.TestCase):
|
class SimpleFacebookTestCase(unittest.TestCase):
|
||||||
|
|
||||||
@mock.patch.object(facebook.GraphAPI, 'put_object', autospec=True)
|
@mock.patch.object(facebook.GraphAPI, 'put_object', autospec=True)
|
||||||
def test_post_message(self, mock_put_object):
|
def test_post_message(self, mock_put_object):
|
||||||
sf = simple_facebook.SimpleFacebook("fake oauth token")
|
sf = simple_facebook.SimpleFacebook("fake oauth token")
|
||||||
@ -479,12 +491,3 @@ via: http://slviki.com/index.php/2016/06/18/introduction-to-mocking-in-python/
|
|||||||
[6]: http://www.voidspace.org.uk/python/mock/mock.html
|
[6]: http://www.voidspace.org.uk/python/mock/mock.html
|
||||||
[7]: http://www.toptal.com/qa/how-to-write-testable-code-and-why-it-matters
|
[7]: http://www.toptal.com/qa/how-to-write-testable-code-and-why-it-matters
|
||||||
[8]: http://www.toptal.com/python
|
[8]: http://www.toptal.com/python
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
@ -0,0 +1,216 @@
|
|||||||
|
使用 Python 创建你自己的 Shell:Part II
|
||||||
|
===========================================
|
||||||
|
|
||||||
|
在 [part 1][1] 中,我们已经创建了一个主要的 shell 循环、切分了的命令输入,以及通过 `fork` 和 `exec` 执行命令。在这部分,我们将会解决剩下的问题。首先,`cd test_dir2` 命令无法修改我们的当前目录。其次,我们仍无法优雅地从 shell 中退出。
|
||||||
|
|
||||||
|
### 步骤 4:内置命令
|
||||||
|
|
||||||
|
“cd test_dir2 无法修改我们的当前目录” 这句话是对的,但在某种意义上也是错的。在执行完该命令之后,我们仍然处在同一目录,从这个意义上讲,它是对的。然而,目录实际上已经被修改,只不过它是在子进程中被修改。
|
||||||
|
|
||||||
|
还记得我们 fork 了一个子进程,然后执行命令,执行命令的过程没有发生在父进程上。结果是我们只是改变了子进程的当前目录,而不是父进程的目录。
|
||||||
|
|
||||||
|
然后子进程退出,而父进程在原封不动的目录下继续运行。
|
||||||
|
|
||||||
|
因此,这类与 shell 自己相关的命令必须是内置命令。它必须在 shell 进程中执行而没有分叉(forking)。
|
||||||
|
|
||||||
|
#### cd
|
||||||
|
|
||||||
|
让我们从 `cd` 命令开始。
|
||||||
|
|
||||||
|
我们首先创建一个 `builtins` 目录。每一个内置命令都会被放进这个目录中。
|
||||||
|
|
||||||
|
```shell
|
||||||
|
yosh_project
|
||||||
|
|-- yosh
|
||||||
|
|-- builtins
|
||||||
|
| |-- __init__.py
|
||||||
|
| |-- cd.py
|
||||||
|
|-- __init__.py
|
||||||
|
|-- shell.py
|
||||||
|
```
|
||||||
|
|
||||||
|
在 `cd.py` 中,我们通过使用系统调用 `os.chdir` 实现自己的 `cd` 命令。
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
from yosh.constants import *
|
||||||
|
|
||||||
|
|
||||||
|
def cd(args):
|
||||||
|
os.chdir(args[0])
|
||||||
|
|
||||||
|
return SHELL_STATUS_RUN
|
||||||
|
```
|
||||||
|
|
||||||
|
注意,我们会从内置函数返回 shell 的运行状态。所以,为了能够在项目中继续使用常量,我们将它们移至 `yosh/constants.py`。
|
||||||
|
|
||||||
|
```shell
|
||||||
|
yosh_project
|
||||||
|
|-- yosh
|
||||||
|
|-- builtins
|
||||||
|
| |-- __init__.py
|
||||||
|
| |-- cd.py
|
||||||
|
|-- __init__.py
|
||||||
|
|-- constants.py
|
||||||
|
|-- shell.py
|
||||||
|
```
|
||||||
|
|
||||||
|
在 `constants.py` 中,我们将状态常量都放在这里。
|
||||||
|
|
||||||
|
```python
|
||||||
|
SHELL_STATUS_STOP = 0
|
||||||
|
SHELL_STATUS_RUN = 1
|
||||||
|
```
|
||||||
|
|
||||||
|
现在,我们的内置 `cd` 已经准备好了。让我们修改 `shell.py` 来处理这些内置函数。
|
||||||
|
|
||||||
|
```python
|
||||||
|
...
|
||||||
|
# Import constants
|
||||||
|
from yosh.constants import *
|
||||||
|
|
||||||
|
# Hash map to store built-in function name and reference as key and value
|
||||||
|
built_in_cmds = {}
|
||||||
|
|
||||||
|
|
||||||
|
def tokenize(string):
|
||||||
|
return shlex.split(string)
|
||||||
|
|
||||||
|
|
||||||
|
def execute(cmd_tokens):
|
||||||
|
# Extract command name and arguments from tokens
|
||||||
|
cmd_name = cmd_tokens[0]
|
||||||
|
cmd_args = cmd_tokens[1:]
|
||||||
|
|
||||||
|
# If the command is a built-in command, invoke its function with arguments
|
||||||
|
if cmd_name in built_in_cmds:
|
||||||
|
return built_in_cmds[cmd_name](cmd_args)
|
||||||
|
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
我们使用一个 python 字典变量 `built_in_cmds` 作为哈希映射(hash map),以存储我们的内置函数。我们在 `execute` 函数中提取命令的名字和参数。如果该命令在我们的哈希映射中,则调用对应的内置函数。
|
||||||
|
|
||||||
|
(提示:`built_in_cmds[cmd_name]` 返回能直接使用参数调用的函数引用的。)
|
||||||
|
|
||||||
|
我们差不多准备好使用内置的 `cd` 函数了。最后一步是将 `cd` 函数添加到 `built_in_cmds` 映射中。
|
||||||
|
|
||||||
|
```
|
||||||
|
...
|
||||||
|
# Import all built-in function references
|
||||||
|
from yosh.builtins import *
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
# Register a built-in function to built-in command hash map
|
||||||
|
def register_command(name, func):
|
||||||
|
built_in_cmds[name] = func
|
||||||
|
|
||||||
|
|
||||||
|
# Register all built-in commands here
|
||||||
|
def init():
|
||||||
|
register_command("cd", cd)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
# Init shell before starting the main loop
|
||||||
|
init()
|
||||||
|
shell_loop()
|
||||||
|
```
|
||||||
|
|
||||||
|
我们定义了 `register_command` 函数,以添加一个内置函数到我们内置的命令哈希映射。接着,我们定义 `init` 函数并且在这里注册内置的 `cd` 函数。
|
||||||
|
|
||||||
|
注意这行 `register_command("cd", cd)` 。第一个参数为命令的名字。第二个参数为一个函数引用。为了能够让第二个参数 `cd` 引用到 `yosh/builtins/cd.py` 中的 `cd` 函数引用,我们必须将以下这行代码放在 `yosh/builtins/__init__.py` 文件中。
|
||||||
|
|
||||||
|
```
|
||||||
|
from yosh.builtins.cd import *
|
||||||
|
```
|
||||||
|
|
||||||
|
因此,在 `yosh/shell.py` 中,当我们从 `yosh.builtins` 导入 `*` 时,我们可以得到已经通过 `yosh.builtins` 导入的 `cd` 函数引用。
|
||||||
|
|
||||||
|
我们已经准备好了代码。让我们尝试在 `yosh` 同级目录下以模块形式运行我们的 shell,`python -m yosh.shell`。
|
||||||
|
|
||||||
|
现在,`cd` 命令可以正确修改我们的 shell 目录了,同时非内置命令仍然可以工作。非常好!
|
||||||
|
|
||||||
|
#### exit
|
||||||
|
|
||||||
|
最后一块终于来了:优雅地退出。
|
||||||
|
|
||||||
|
我们需要一个可以修改 shell 状态为 `SHELL_STATUS_STOP` 的函数。这样,shell 循环可以自然地结束,shell 将到达终点而退出。
|
||||||
|
|
||||||
|
和 `cd` 一样,如果我们在子进程中 fork 和执行 `exit` 函数,其对父进程是不起作用的。因此,`exit` 函数需要成为一个 shell 内置函数。
|
||||||
|
|
||||||
|
让我们从这开始:在 `builtins` 目录下创建一个名为 `exit.py` 的新文件。
|
||||||
|
|
||||||
|
```
|
||||||
|
yosh_project
|
||||||
|
|-- yosh
|
||||||
|
|-- builtins
|
||||||
|
| |-- __init__.py
|
||||||
|
| |-- cd.py
|
||||||
|
| |-- exit.py
|
||||||
|
|-- __init__.py
|
||||||
|
|-- constants.py
|
||||||
|
|-- shell.py
|
||||||
|
```
|
||||||
|
|
||||||
|
`exit.py` 定义了一个 `exit` 函数,该函数仅仅返回一个可以退出主循环的状态。
|
||||||
|
|
||||||
|
```
|
||||||
|
from yosh.constants import *
|
||||||
|
|
||||||
|
|
||||||
|
def exit(args):
|
||||||
|
return SHELL_STATUS_STOP
|
||||||
|
```
|
||||||
|
|
||||||
|
然后,我们导入位于 `yosh/builtins/__init__.py` 文件的 `exit` 函数引用。
|
||||||
|
|
||||||
|
```
|
||||||
|
from yosh.builtins.cd import *
|
||||||
|
from yosh.builtins.exit import *
|
||||||
|
```
|
||||||
|
|
||||||
|
最后,我们在 `shell.py` 中的 `init()` 函数注册 `exit` 命令。
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
...
|
||||||
|
|
||||||
|
# Register all built-in commands here
|
||||||
|
def init():
|
||||||
|
register_command("cd", cd)
|
||||||
|
register_command("exit", exit)
|
||||||
|
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
到此为止!
|
||||||
|
|
||||||
|
尝试执行 `python -m yosh.shell`。现在你可以输入 `exit` 优雅地退出程序了。
|
||||||
|
|
||||||
|
### 最后的想法
|
||||||
|
|
||||||
|
我希望你能像我一样享受创建 `yosh` (**y**our **o**wn **sh**ell)的过程。但我的 `yosh` 版本仍处于早期阶段。我没有处理一些会使 shell 崩溃的极端状况。还有很多我没有覆盖的内置命令。为了提高性能,一些非内置命令也可以实现为内置命令(避免新进程创建时间)。同时,大量的功能还没有实现(请看 [公共特性](http://tldp.org/LDP/Bash-Beginners-Guide/html/x7243.html) 和 [不同特性](http://www.tldp.org/LDP/intro-linux/html/x12249.html))
|
||||||
|
|
||||||
|
我已经在 github.com/supasate/yosh 中提供了源代码。请随意 fork 和尝试。
|
||||||
|
|
||||||
|
现在该是创建你真正自己拥有的 Shell 的时候了。
|
||||||
|
|
||||||
|
Happy Coding!
|
||||||
|
|
||||||
|
--------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
via: https://hackercollider.com/articles/2016/07/06/create-your-own-shell-in-python-part-2/
|
||||||
|
|
||||||
|
作者:[Supasate Choochaisri][a]
|
||||||
|
译者:[cposture](https://github.com/cposture)
|
||||||
|
校对:[校对者ID](https://github.com/校对者ID)
|
||||||
|
|
||||||
|
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
|
||||||
|
|
||||||
|
[a]: https://disqus.com/by/supasate_choochaisri/
|
||||||
|
[1]: https://hackercollider.com/articles/2016/07/05/create-your-own-shell-in-python-part-1/
|
||||||
|
[2]: http://tldp.org/LDP/Bash-Beginners-Guide/html/x7243.html
|
||||||
|
[3]: http://www.tldp.org/LDP/intro-linux/html/x12249.html
|
||||||
|
[4]: https://github.com/supasate/yosh
|
Loading…
Reference in New Issue
Block a user