Merge pull request #3684 from fw8899/master

tranlation complete
This commit is contained in:
Xingyu.Wang 2015-12-30 01:23:04 +08:00
commit be9e252729
2 changed files with 195 additions and 196 deletions

View File

@ -1,196 +0,0 @@
translating by fw8899
Optimize Web Delivery with these Open Source Tools
================================================================================
Web proxy software forwards HTTP requests without modifying traffic in any way. They can be configured as a transparent proxy with no client-side configuration required. They can also be used as a reverse proxy front-end to websites; here the cache serves an unlimited number of clients for one or some web servers.
Web proxies are versatile tools. They have a wide variety of uses, from caching web, DNS and other lookups, to speeding up the delivery of a web server / reducing bandwidth consumption. Web proxy software can also harden security by filtering traffic and anonymizing connections, and offer media-range limitations. This software is used by high-profile, high-traffic websites such as The New York Times, The Guardian, and social media and content sites such as Twitter, Facebook, and Wikipedia.
Web caches have become a vital mechanism for optimising the amount of data that is delivered in a given period of time. Good web caches also help to minimise latency, serving pages as quickly as possible. This helps to prevent the end user from becoming impatient having to wait for content to be delivered. Web caches optimise the data flow between client and server. They also help to converse bandwidth by caching frequently-delivered content. If you need to reduce server load and improve delivery speed of your content, it is definitely worth exploring the benefits offered by web cache software.
To provide an insight into the quality of software available for Linux, I feature below 5 excellent open source web proxy tools. Some of the them are full-featured; a couple of them have very modest resource needs.
### Squid ###
Squid is a high-performance open source proxy caching server and web cache daemon. It supports FTP, Internet Gopher, HTTPS, TLS, and SSL. It handles all requests in a single, non-blocking, I/O-driven process over IPv4 or IPv6.
Squid consists of a main server program squid, a Domain Name System lookup program dnsserver, some optional programs for rewriting requests and performing authentication, together with some management and client tools.
Squid offers a rich access control, authorization and logging environment to develop web proxy and content serving applications.
Features include:
- Web proxy:
- Caching to reduce access time and bandwidth use
- Keeps meta data and especially hot objects cached in RAM
- Caches DNS lookups
- Supports non-blocking DNS lookups
- Implements negative chacking of failed requests
- Squid caches can be arranged in a hierarchy or mesh for additional bandwidth savings
- Enforce site-usage policies with extensive access controls
- Anonymize connections, such as disabling or changing specific header fields in a client's HTTP request
- Reverse proxy
- Media-range limitations
- Supports SSL
- Support for IPv6
- Error Page Localization - error pages presented by Squid may now be localized per-request to match the visitors local preferred language
- Connection Pinning (for NTLM Auth Passthrough) - a workaround which permits Web servers to use Microsoft NTLM Authentication instead of HTTP standard authentication through a web proxy
- Quality of Service (QoS) Flow support
- Select a TOS/Diffserv value to mark local hits
- Select a TOS/Diffserv value to mark peer hits
- Selectively mark only sibling or parent requests
- Allows any HTTP response towards clients to have the TOS value of the response coming from the remote server preserved
- Mask certain bits in the TOS received from the remote server, before copying the value to the TOS send towards clients
- SSL Bump (for HTTPS Filtering and Adaptation) - Squid-in-the-middle decryption and encryption of CONNECT tunneled SSL traffic, using configurable client- and server-side certificates
- eCAP Adaptation Module support
- ICAP Bypass and Retry enhancements - ICAP is now extended with full bypass and dynamic chain routing to handle multiple adaptation services.
- ICY streaming protocol support - commonly known as SHOUTcast multimedia streams
- Dynamic SSL Certificate Generation
- Support for the Internet Content Adaptation Protocol (ICAP)
- Full request logging
- Anonymize connections
- Website: [www.squid-cache.org][1]
- Developer: National Laboratory for Applied Networking Research (NLANR) and Internet volunteers
- License: GNU GPL v2
- Version Number: 4.0.1
### Privoxy ###
Privoxy (Privacy Enhancing Proxy) is a non-caching Web proxy with advanced filtering capabilities for enhancing privacy, modifying web page data and HTTP headers, controlling access, and removing ads and other obnoxious Internet junk. Privoxy has a flexible configuration and can be customized to suit individual needs and tastes. It supports both stand-alone systems and multi-user networks.
Privoxy uses the concept of actions in order to manipulate the data stream between the browser and remote sites.
Features include:
- Highly configurable - completely personalize your installation
- Ad blocking
- Cookie management
- Supports "Connection: keep-alive". Outgoing connections can be kept alive independently from the client
- Supports IPv6
- Tagging which allows to change the behaviour based on client and server headers
- Run as an "intercepting" proxy
- Sophisticated actions and filters for manipulating both server and client headers
- Can be chained with other proxies
- Integrated browser-based configuration and control utility. Browser-based tracing of rule and filter effects. Remote toggling
- Web page filtering (text replacements, removes banners based on size, invisible "web-bugs" and HTML annoyances, etc)
- Modularized configuration that allows for standard settings and user settings to reside in separate files, so that installing updated actions files won't overwrite individual user settings
- Support for Perl Compatible Regular Expressions in the configuration files, and a more sophisticated and flexible configuration syntax
- GIF de-animation
- Bypass many click-tracking scripts (avoids script redirection)
- User-customizable HTML templates for most proxy-generated pages (e.g. "blocked" page)
- Auto-detection and re-reading of config file changes
- Most features are controllable on a per-site or per-location basis
- Website: [www.privoxy.org][2]
- Developer: Fabian Keil (lead developer), David Schmidt, and many other contributors
- License: GNU GPL v2
- Version Number: 3.4.2
### Varnish Cache ###
Varnish Cache is a web accelerator written with performance and flexibility in mind. It's modern architecture offers significantly better performance. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture. Varnish stores web pages in memory so the web servers do not have to create the same web page repeatedly. The web server only recreates a page when it is changed. When content is served from memory this happens a lot faster then anything.
Additionally Varnish can serve web pages much faster then any application server is capable of - giving the website a significant speed enhancement.
For a cost-effective configuration, Varnish Cache uses between 1-16GB and a SSD disk.
Features include:
- Modern design
- VCL - a very flexible configuration language. The VCL configuration is translated to C, compiled, loaded and executed giving flexibility and speed
- Load balancing using both a round-robin and a random director, both with a per-backend weighting
- DNS, Random, Hashing and Client IP based Directors
- Load balance between multiple backends
- Support for Edge Side Includes including stitching together compressed ESI fragments
- Heavily threaded
- URL rewriting
- Cache multiple vhosts with a single Varnish
- Log data is stored in shared memory
- Basic health-checking of backends
- Graceful handling of "dead" backends
- Administered by a command line interface
- Use In-line C to extend Varnish
- Can be used on the same system as Apache
- Run multiple Varnish on the same system
- Support for HAProxy's PROXY protocol. This is a protocol adds a small header on each incoming TCP connection that describes who the real client is, added by (for example) an SSL terminating process
- Warm and cold VCL states
- Plugin support with Varnish Modules, called VMODs
- Backends defined through VMODs
- Gzip Compression and Decompression
- HTTP Streaming Pass & Fetch
- Saint and Grace mode. Saint Mode allows for unhealthy backends to be blacklisted for a period of time, preventing them from serving traffic when using Varnish as a load balancer. Grace mode allows Varnish to serve an expired version of a page or other asset in cases where Varnish is unable to retrieve a healthy response from the backend
- Experimental support for Persistent Storage, without LRU eviction
- Website: [www.varnish-cache.org][3]
- Developer: Varnish Software
- License: FreeBSD
- Version Number: 4.1.0
### Polipo ###
Polipo is an open source caching HTTP proxy which has modest resource needs.
It listens to requests for web pages from your browser and forwards them to web servers, and forwards the servers replies to your browser. In the process, it optimises and cleans up the network traffic. It is similar in spirit to WWWOFFLE, but the implementation techniques are more like the ones ones used by Squid.
Polipo aims at being a compliant HTTP/1.1 proxy. It should work with any web site that complies with either HTTP/1.1 or the older HTTP/1.0.
Features include:
- HTTP 1.1, IPv4 & IPv6, traffic filtering and privacy-enhancement
- Uses HTTP/1.1 pipelining if it believes that the remote server supports it, whether the incoming requests are pipelined or come in simultaneously on multiple connections
- Cache the initial segment of an instance if the download has been interrupted, and, if necessary, complete it later using Range requests
- Upgrade client requests to HTTP/1.1 even if they come in as HTTP/1.0, and up- or downgrade server replies to the client's capabilities
- Complete support for IPv6 (except for scoped (link-local) addresses)
- Use as a bridge between the IPv4 and IPv6 Internets
- Content-filtering
- Can use a technique known as Poor Man's Multiplexing to reduce latency
- SOCKS 4 and SOCKS 5 protocol support
- HTTPS proxying
- Behaves as a transparent proxy
- Run Polipo together with Privoxy or tor
- Website: [www.pps.univ-paris-diderot.fr/~jch/software/polipo/][4]
- Developer: Juliusz Chroboczek, Christopher Davis
- License: MIT License
- Version Number: 1.1.1
### Tinyproxy ###
Tinyproxy is a lightweight open source web proxy daemon. It is designed to be fast and yet small. It is useful for cases such as embedded deployments where a full featured HTTP proxy is required, but the system resources for a larger proxy are unavailable.
Tinyproxy is very useful in a small network setting, where a larger proxy would either be too resource intensive, or a security risk. One of the key features of Tinyproxy is the buffering connection concept. In effect, Tinyproxy will buffer a high speed response from a server, and then relay it to a client at the highest speed the client will accept. This feature greatly reduces the problems with sluggishness on the net.
Features:
- Easy to modify
- Anonymous mode - allows specification of individual HTTP headers that should be allowed through, and which should be blocked
- HTTPS support - Tinyproxy allows forwarding of HTTPS connections without modifying traffic in any way through the CONNECT method
- Remote monitoring - access proxy statistics from afar, letting you know exactly how busy the proxy is
- Load average monitoring - configure software to refuse connections after the server load reaches a certain point
- Access control - configure to only allow connections from certain subnets or IP addresses
- Secure - run without any special privileges, thus minimizing the chance of system compromise
- URL based filtering - allows domain and URL-based black- and whitelisting
- Transparent proxying - configure as a transparent proxy, so that a proxy can be used without any client-side configuration
- Proxy chaining - use an upstream proxy server for outbound connections, instead of direct connections to the target server, creating a so-called proxy chain
- Privacy features - restrict both what data comes to your web browser from the HTTP server (e.g., cookies), and to restrict what data is allowed through from your web browser to the HTTP server (e.g., version information)
- Small footprint - the memory footprint is about 2MB with glibc, and the CPU load increases linearly with the number of simultaneous connections (depending on the speed of the connection). Tinyproxy can be run on an old machine without affecting performance
- Website: [banu.com/tinyproxy][5]
- Developer: Robert James Kaes and contributors
- License: GNU GPL v2
- Version Number: 1.8.3
--------------------------------------------------------------------------------
via: http://www.linuxlinks.com/article/20151101020309690/WebDelivery.html
译者:[译者ID](https://github.com/译者ID)
校对:[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
[1]:http://www.squid-cache.org/
[2]:http://www.privoxy.org/
[3]:https://www.varnish-cache.org/
[4]:http://www.pps.univ-paris-diderot.fr/%7Ejch/software/polipo/
[5]:https://banu.com/tinyproxy/

View File

@ -0,0 +1,195 @@
使用开源工具优化Web响应
================================================================================
Web代理软件转发HTTP请求时并不会改变数据流量。它们经过配置后可以免客户端配置作为透明代理。它们还可以作为网站反向代理的前端缓存服务器在此能支撑一台或多台web服务器为海量用户提供服务。
网站代理功能多样有着宽泛的用途从页面缓存、DNS和其他查询到加速web服务器响应、降低带宽消耗。代理软件广泛用于大型高访问量的网站比如纽约时报、卫报 以及社交媒体网站如Twitter、Facebook和Wikipedia。
页面缓存已经成为优化单位时间内所能吞吐的数据量的至关重要的机制。好的Web缓存还能降低延迟尽可能快地响应页面让终端用户不至于因等待内容的时间过久而失去耐心。它们还能将频繁访问的内容缓存起来以节省带宽。如果你需要降低服务器负载并改善网站内容响应速度那缓存软件能带来的好处就绝对值得探索一番。
为深入探查Linux下可用的相关软件的质量我列出了下边5个优秀的开源web代理工具。它们中有些功能完备强大也有几个只需很低的资源就能运行。
### Squid ###
Squid是一个高性能、开源的代理缓存和Web缓存服务器支持FTP、Internet Gopher、HTTPS和SSL等多种协议。它通过一个非阻塞I/O事件驱动的单一进程处理所有IPV4或IPV6上的请求。
Squid由一个主服务程序squid和DNS查询程序dnsserver另外还有可选的请求重写、执行认证程序组件及一些管理和客户端工具构成。
Squid提供了丰富的访问控制、认证和日志环境 用于开发web代理和内容服务网站应用。
其特性包括:
- Web代理
- 通过缓存来降低访问时间和带宽使用
- 将元数据和特别热的对象缓存到内存中
- 缓存DNS查询
- 支持非阻塞的DNS查询
- 实现了失败请求的未果缓存
- Squid缓存可架设为层次结构或网状结构以节省额外的带宽
- 通过可扩展的访问控制来执行网站使用条款
- 隐匿请求如禁用或修改客户端HTTP请求头特定属性
- 反向代理
- 媒体范围限制
- 支持SSL
- 支持IPv6
- 错误页面的本地化 - Squid可以根据访问者的语言选项对每个请求展示本地化的错误页面
- 连接Pinning用于NTLM Auth Passthrough - 一种通过Web代理允许Web服务器使用Microsoft NTLM安全认证替代HTTP标准认证的方案
- 支持服务质量 (QoS, Quality of Service) 流
- 选择一个TOS/Diffserv值来标记本地命中
- 选择一个TOS/Diffserv值来标记邻居命中
- 选择性地仅标记同级或上级请求
- 允许任意发往客户端的HTTP响应保持由远程服务器处响应的TOS值
- 对收到的远程服务器的TOS值在复制之前对指定位进行掩码操作再发送到客户端
- SSL Bump (用于HTTPS过滤和适配) - Squid-in-the-middle在CONNECT方式的SSL隧道中用配置化的客户端和服务器端证书对流量进行解密和加密
- 支持适配模块
- ICAP旁路和重试增强 - 通过完全的旁路和动态链式路由扩展ICAP来处理多多个适应性服务。
- 支持ICY流式协议 - 俗称SHOUTcast多媒体流
- 动态SSL证书生产
- 支持ICAP协议(Internet Content Adaptation Protocol)
- 完整的请求日志记录
- 匿名连接
- 网站: [www.squid-cache.org][1]
- 开发: 美国国家应用网络研究实验室和网络志愿者
- 授权: GNU GPL v2
- 版本号: 4.0.1
### Privoxy ###
Privoxy(Privacy Enhancing Proxy)是一个非缓存类Web代理软件它自带的高级过滤功能用来增强隐私保护修改页面内容和HTTP头部信息访问控制以及去除广告和其它招人反感的互联网垃圾。Privoxy的配置非常灵活能充分定制已满足各种各样的需求和偏好。它支持单机和多用户网络两种模式。
Privoxy使用Actions规则来处理浏览器和远程站点间的数据流。
其特性包括:
- 高度配置化
- 广告拦截
- Cookie管理
- 支持"Connection: keep-alive"。可以无视客户端配置而保持持久连接
- 支持IPv6
- 标签化,允许按照客户端和服务器的请求头进行处理
- 作为拦截代理器运行
- 巧妙的手段和过滤机制用来处理服务器和客户端的HTTP头部
- 可以与其他代理软件链式使用
- 整合了基于浏览器的配置和控制工具,能在线跟踪规则和过滤效果,可远程开关
- 页面过滤(文本替换、根据尺寸大小删除广告栏, 隐藏的"web-bugs"元素和HTML容错等
- 模块化的配置使得标准配合和用户配置可以存放于不同文件中,这样安装更新就不会覆盖用户的个性化设置
- 配置文件支持Perl兼容的正则表达式以及更为精妙和灵活的配置语法
- GIF去动画
- 旁路处理大量click-tracking脚本避免脚本重定向
- 大多数代理生成的页面(例如 "访问受限" 页面可由用户自定义HTML模板
- 自动监测配置文件的修改并重新读取
- 最大特点是可以基于每个站点或每个位置来进行控制
- 网站: [www.privoxy.org][2]
- 开发: Fabian Keil开发领导者, David Schmidt, 和众多其他贡献者
- 授权: GNU GPL v2
- 版本号: 3.4.2
### Varnish Cache ###
Varnish Cache是一个为性能和灵活性而生的web加速器。它新颖的架构设计能带来显著的性能提升。根据你的架构通常情况下它能加速响应速度300-1000倍。Varnish将页面存储到内存这样web服务器就无需重复地创建相同的页面只需要在页面发生变化后重新生成。页面内容直接从内存中访问当然比其他方式更快。
此外Varnish能大大提升响应web页面的速度用任何应用服务器都能使网站访问速度大幅度地提升。
按按经验Varnish Cache比较经济的配置是1-16GB内存+SSD固态硬盘。
其特性包括:
- 新颖的设计
- VCL - 非常灵活的配置语言。VCL配置转换成C然后编译、加载、运行灵活且高效
- 能使用round-robin轮询和随机分发两种方式来负载均衡两种方式下后端服务器都可以设置权重
- 基于DNS、随机、散列和客户端IP的分发器
- 多台后端主机间的负载均衡
- 支持Edge Side Includes包括拼装压缩后的ESI片段
- 多线程并发
- URL重写
- 单Varnish缓存多个虚拟主机
- 日志数据存储在共享内存中
- 基本的后端服务器健康检查
- 优雅地处理后端服务器“挂掉”
- 命令行界面的管理控制台
- 使用内联C来扩展Varnish
- 可以与Apache用在相同的系统上
- 单系统可运行多个Varnish
- 支持HAProxy代理协议。该协议在每个收到的TCP请求例如SSL终止过程中附加小段头信息以记录客户端的真实地址
- 冷热VCL状态
- 用名为VMODs的Varnish模块来提供插件扩展
- 通过VMODs定义后端主机
- Gzip压缩及解压
- HTTP流通过和获取
- 神圣模式和优雅模式。用Varnish作为负载均衡器神圣模式下可以将不稳定的后端服务器在一段时间内打入黑名单阻止它们继续提供流量服务。优雅模式允许Varnish在获取不到后端服务器状态良好的响应时提供已过期版本的页面或其它内容。
- 实验性支持持久化存储无需LRU缓存淘汰
- 网站: [www.varnish-cache.org][3]
- 开发: Varnish Software
- 授权: FreeBSD
- 版本号: 4.1.0
### Polipo ###
Polipo是一个开源的HTTP缓存代理只需要非常低的资源开销。
它监听来自浏览器的web页面请求转发到web服务器然后将服务器的响应转发到浏览器。在此过程中它能优化和整形网络流量。从本质来讲Polipo与WWWOFFLE很相似但其实现技术更接近于Squid。
Polipo最开始的目标是作为一个兼容HTTP/1.1的代理理论它能在任何兼容HTTP/1.1或更早的HTTP/1.0的站点上运行。
其特性包括:
- HTTP 1.1、IPv4 & IPv6、流量过滤和隐私保护增强
- 如确认远程服务器支持则无论收到的请求是管道处理过的还是在多个连接上同时收到的都使用HTTP/1.1管道
- 下载被中断时缓存起始部分,当需要续传时用区间请求来完成下载
- 将HTTP/1.0的客户端请求升级为HTTP/1.1,然后按照客户端支持的级别进行升级或降级后回复
- 全面支持IPv6 (作用域(链路本地)地址除外)
- 作为IPv4和IPv6网络的网桥
- 内容过滤
- 能使用Poor Man多路复用技术降低延迟
- 支持SOCKS 4和SOCKS 5协议
- HTTPS代理
- 扮演透明代理的角色
- 可以与Privoxy或tor一起运行
- 网站: [www.pps.univ-paris-diderot.fr/~jch/software/polipo/][4]
- 开发: Juliusz Chroboczek, Christopher Davis
- 授权: MIT License
- 版本号: 1.1.1
### Tinyproxy ###
Tinyproxy是一个轻量级的开源web代理守护进程其设计目标是快而小。它适用于需要完整HTTP代理特性但系统资源又不足以运行大型代理的场景比如嵌入式部署。
Tinyproxy对小规模网络非常有用这样的场合下大型代理会使系统资源紧张或有安全风险。Tinyproxy的一个关键特性是其缓冲连接的理念。实质上Tinyproxy服务器的响应进行了高速缓冲然后按照客户端能够处理的最高速度进行响应。该特性极大的降低了网络延滞带来的问题。
特性:
- 易于修改
- 隐匿模式 - 定义哪些HTTP头允许通过哪些又会被拦截
- 支持HTTPS - Tinyproxy允许通过CONNECT方法转发HTTPS连接任何情况下都不会修改数据流量
- 远程监控 - 远程访问代理统计数据,让你能清楚了解代理服务当前的忙碌状态
- 平均负载监控 - 通过配置,当服务器的负载接近一定值后拒绝新连接
- 访问控制 - 通过配置仅允许指定子网或IP地址的访问
- 安全 - 运行无需额外权限,减小了系统受到威胁的概率
- 基于URL的过滤 - 允许基于域和URL的黑白名单
- 透明代理 - 配位为透明代理,这样客户端就无需任何配置
- 代理链 - 来流量出口处采用上游代理服务器,而不是直接转发到目标服务器,创建我们所说的代理链
- 隐私特性 - 限制允许从浏览器收到的来自HTTP服务器的数据例如cookies同时限制允许通过的从浏览器到HTTP服务器的数据例如版本信息
- 低开销 - 使用glibc内存开销只有2MBCPU负载按并发连接数线性增长取决于网络连接速度。 Tinyproxy可以运行在老旧的机器上而无需担心性能问题。
- 网站: [banu.com/tinyproxy][5]
- 开发: Robert James Kaes和其他贡献者
- 授权: GNU GPL v2
- 版本号: 1.8.3
--------------------------------------------------------------------------------
via: http://www.linuxlinks.com/article/20151101020309690/WebDelivery.html
译者:[fw8899](https://github.com/fw8899)
校对:[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
[1]:http://www.squid-cache.org/
[2]:http://www.privoxy.org/
[3]:https://www.varnish-cache.org/
[4]:http://www.pps.univ-paris-diderot.fr/%7Ejch/software/polipo/
[5]:https://banu.com/tinyproxy/