varnish测试

[root&＃64;localhost varnish]# rpm -ql varnish
/etc/logrotate.d/varnish
/etc/varnish
/etc/varnish/default.vcl
/etc/varnish/varnish.params
/run/varnish.pid
/usr/bin/varnishadm
/usr/bin/varnishhist
/usr/bin/varnishlog
/usr/bin/varnishncsa
/usr/bin/varnishstat
/usr/bin/varnishtest
/usr/bin/varnishtop
/usr/lib/systemd/system/varnish.service
/usr/lib/systemd/system/varnishlog.service
/usr/lib/systemd/system/varnishncsa.service
/usr/sbin/varnish_reload_vcl
/usr/sbin/varnishd
/usr/share/doc/varnish-4.0.5
/usr/share/doc/varnish-4.0.5/LICENSE
/usr/share/doc/varnish-4.0.5/README
/usr/share/doc/varnish-4.0.5/builtin.vcl
/usr/share/doc/varnish-4.0.5/changes.rst
/usr/share/doc/varnish-4.0.5/example.vcl
/usr/share/man/man1/varnishadm.1.gz
/usr/share/man/man1/varnishd.1.gz
/usr/share/man/man1/varnishhist.1.gz
/usr/share/man/man1/varnishlog.1.gz
/usr/share/man/man1/varnishncsa.1.gz
/usr/share/man/man1/varnishstat.1.gz
/usr/share/man/man1/varnishtest.1.gz
/usr/share/man/man1/varnishtop.1.gz
/usr/share/man/man3/vmod_directors.3.gz
/usr/share/man/man3/vmod_std.3.gz
/usr/share/man/man7/varnish-cli.7.gz
/usr/share/man/man7/varnish-counters.7.gz
/usr/share/man/man7/vcl.7.gz
/usr/share/man/man7/vsl-query.7.gz
/usr/share/man/man7/vsl.7.gz
/var/lib/varnish
/var/log/varnish
[root&＃64;localhost varnish]#

　　　　　　　1、varnishd应用程序的命令行参数&＃xff1b;监听的socket, 使用的存储类型等等&＃xff1b;额外的配置参数&＃xff1b;-p param&＃61;value -r param,param,... : 设定只读参数列表&＃xff1b;/etc/varnish/varnish.params2、-p选项指明的参数&＃xff1a;运行时参数&＃xff1a;也可在程序运行中&＃xff0c;通过其CLI进行配置&＃xff1b;3、vcl&＃xff1a;配置缓存系统的缓存机制&＃xff1b;通过vcl配置文件进行配置&＃xff1b;先编译&＃xff0c;后应用&＃xff1b;依赖于c编译器&＃xff1b;

Starting Varnish&＃xff08;启动varnish&＃xff09;假设varnishd在您的环境变量中&＃xff0c;您可能需要运行pkill varnishd来确定varnish没有运行。然后使用root执行下面的命令。varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:8080我添加了一些选项&＃xff0c;现在来详细分析他们&＃xff1a;-f /usr/local/etc/varnish/default.vcl这个 –f 选项指定varnishd使用哪个配置文件。-s malloc&＃xff0c;1G这个 –s 选项用来确定varnish使用的存储类型和存储容量&＃xff0c;我使用的是malloc类型&＃xff08;malloc是一个C函数&＃xff0c;用于分配内存空间&＃xff09;&＃xff0c; 1G 定义多少内存被malloced&＃xff0c;1G &＃61; 1gigabyte。-T 127.0.0.1&＃xff1a;2000Varnish有一个基于文本的管理接口&＃xff0c;启动它的话可以在不停止varnish的情况下来管理varnish。您可以指定管理软件监听哪个接口。当然您不能让全世界的人都能访问您的varnish管理接口&＃xff0c;因为他们可以很轻松的通过访问varnish管理接口来获得您的root访问权限。我推荐只让它监听本机端口。如果您的系统里有您不完全信任的用户&＃xff0c;您可以通过防火墙规则来限制他访问varnish的管理端口。-a 0.0.0.0&＃xff1a;8080这一句的意思是制定varnish监听所有IP发给8080端口的http请求&＃xff0c;如果在生产环境下&＃xff0c;您应该让varnish监听80&＃xff0c;这也是默认的。

*********#/etc/varnish/varnish.params 配置文件定义了默认的运行参数&＃xff0c;可以直接通过命令行指定&＃xff0c;也可以通过修改配置文件配置
*********#/etc/varnish/default.vcl 配置文件定义了缓存系统的配置

命令行工具&＃xff1a;varnishadm -S /etc/varnish/secret -T IP:PORTLog:varnishlogvarnishncsaStatisticsvarnishstatTop&＃xff1a;varnishtop

vcl各状态引擎的功用&＃xff1a;vcl_recvvcl_fetchvcl_pipe: 用于将请求直接发往后端主机&＃xff1b;vcl_hash: 自定义hash生成时的数据来源vcl_pass: 用于将请求直接传递至后端主机&＃xff1b;vcl_hit: 从缓存中查找到缓存对象时要执行的操作&＃xff1b;vcl_miss: 从缓存中款查找到缓存对象时要执行的操作&＃xff1b;vcl_deliver: 将用户请求的内容响应给客户端时用到的方法&＃xff1b; vcl_error: 在varnish端合成错误响应而时&＃xff1b;

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;# Default backend definition. Set this to point to your content server.
backend default { #定义后端服务器主机# .host &＃61; "127.0.0.1"; # .port &＃61; "8080";.host &＃61; "192.168.100.103";.port &＃61; "80";
}sub vcl_recv {# Happens before we check if we have this in cache already.## Typically you clean up the request here, removing COOKIEs you don&＃39;t need,
# rewriting the request, etc.if (req.method &＃61;&＃61; "PRI") {　　　　　　　　#自己添加官网部分/* We do not support SPDY or HTTP/2.0 */return (synth(405));}if (req.method !&＃61; "GET" &&req.method !&＃61; "HEAD" &&req.method !&＃61; "PUT" &&req.method !&＃61; "POST" &&req.method !&＃61; "TRACE" &&req.method !&＃61; "OPTIONS" &&req.method !&＃61; "DELETE") {/* Non-RFC2616 or CONNECT which is weird. */return (pipe);}if (req.method !&＃61; "GET" && req.method !&＃61; "HEAD") {/* We only deal with GET and HEAD by default */return (pass);}if (req.http.Authorization || req.http.COOKIE) {/* Not cacheable by default */return (pass);}return (hash);
}sub vcl_backend_response {# Happens after we have read the response headers from the backend.## Here you clean the response headers, removing silly Set-COOKIE headers# and other mistakes your backend does.
}sub vcl_deliver {# Happens when we have all the pieces we need, and are about to send the# response to the client.## You can do accounting or modifying the final object here.
if (obj.hits>0) { #定义在vcl_deliver中&＃xff0c;向响应给客户端的报文添加一个自定义首部X-Cache&＃xff1b;set resp.http.X-Cache &＃61; "HIT";} else {set resp.http.X-Cahce &＃61; "MISS";}
}

varnish> vcl.list
200
active 0 bootvarnish> vcl.load test1 default.vcl
200
VCL compiled.

varnish> vcl.use test1
200
VCL &＃39;test1&＃39; now activevarnish> vcl.list
200
available 0 boot
active 0 test1

varnish中的内置变量&＃xff1a;变量种类&＃xff1a;clientserverreqrespbereqberespobjstoragebereqbereq.http.HEADERS: 由varnish发往backend server的请求报文的指定首部&＃xff1b;bereq.request&＃xff1a;请求方法&＃xff1b;bereq.url&＃xff1a;bereq.proto&＃xff1a;bereq.backend&＃xff1a;指明要调用的后端主机&＃xff1b;berespberesp.protoberesp.status&＃xff1a;后端服务器的响应的状态码beresp.reason&＃xff1a;原因短语&＃xff1b;beresp.backend.ipberesp.backend.nameberesp.http.HEADER: 从backend server响应的报文的首部&＃xff1b;beresp.ttl&＃xff1a;后端服务器响应的内容的余下的生存时长&＃xff1b;objobj.ttl: 对象的ttl值&＃xff1b;obj.hits&＃xff1a;此对象从缓存中命中的次数&＃xff1b;serverserver.ipserver.hostnamereqresp

[root&＃64;localhost /]# curl -I http://192.168.100.100:6081/test1.html
HTTP/1.1 200 OK
Date: Sun, 10 Mar 2019 16:26:59 GMT
Server: Apache/2.4.6 (CentOS)
Last-Modified: Sun, 10 Mar 2019 16:26:16 GMT
ETag: "8-583bfea477f63"
Content-Length: 8
Content-Type: text/html; charset&＃61;UTF-8
X-Varnish: 327756
Age: 0
Via: 1.1 varnish-v4
X-Cahce: MISS192.168.100.100
Connection: keep-alive[root&＃64;localhost /]# curl -I http://192.168.100.100:6081/test1.html
HTTP/1.1 200 OK
Date: Sun, 10 Mar 2019 16:26:59 GMT
Server: Apache/2.4.6 (CentOS)
Last-Modified: Sun, 10 Mar 2019 16:26:16 GMT
ETag: "8-583bfea477f63"
Content-Length: 8
Content-Type: text/html; charset&＃61;UTF-8
X-Varnish: 131104 327757
Age: 2
Via: 1.1 varnish-v4
test-Cache: HIT192.168.100.100
Connection: keep-alive

vcl 4.0;backend b1 {.host &＃61; "...";.port &＃61; "...";
}backend b2 {.host &＃61; "...";.port &＃61; "...";
}sub vcl_init {new cluster1 &＃61; directors.round_robin();cluster1.add_backend(b1, 1.0);cluster1.add_backend(b2, 1.0);
}sub vcl_recv {set req.backend_hint &＃61; cluster1.backend();
}

backend server的定义&＃xff1a;backend name {.attribute &＃61; "value";} .host: BE主机的IP&＃xff1b;.port&＃xff1a;BE主机监听的PORT&＃xff1b;.probe: 对BE做健康状态检测&＃xff1b;.max_connections&＃xff1a;并连接最大数量&＃xff1b;后端主机的健康状态检测方式&＃xff1a;probe name {.attribute &＃61; "value";} .url: 判定BE健康与否要请求的url; .expected_response&＃xff1a;期望响应状态码&＃xff1b;默认为200&＃xff1b;示例1&＃xff1a;backend websrv1 {.host &＃61; "172.16.100.68";.port &＃61; "80";.probe &＃61; {.url &＃61; "/test1.html";}}backend websrv2 {.host &＃61; "172.16.100.69";.port &＃61; "80";.probe &＃61; {.url &＃61; "/test1.html";}}

[root&＃64;localhost varnish]# cat default.vcl
#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;# Default backend definition. Set this to point to your content server.
backend default { #定义后端服务器主机# .host &＃61; "127.0.0.1"; # .port &＃61; "8080";.host &＃61; "192.168.100.101";.port &＃61; "80";
}
backend web1 {.host &＃61; "192.168.100.101";.port &＃61; "80";.probe &＃61; {.url &＃61; "/tan.jpg";
}
}
backend web2 {.host &＃61; "192.168.100.103";.port &＃61; "80";.probe &＃61; {.url &＃61; "/tan.jpg";}}
import directors;sub vcl_init { #定义后端服务器集群new cluster1 &＃61; directors.round_robin();cluster1.add_backend(web1);cluster1.add_backend(web2);
}


*******************************************************************************************

vcl_recv是在Varnish完成对请求报文的解码为基本数据结构后第一个要执行的子例程&＃xff0c;它通常有四个主要用途&＃xff1a;
(1)修改客户端数据以减少缓存对象差异性&＃xff1b;比如删除URL中的www.等字符&＃xff1b;
(2)基于客户端数据选用缓存策略&＃xff1b;比如仅缓存特定的URL请求、不缓存POST请求等&＃xff1b;
(3)为某web应用程序执行URL重写规则&＃xff1b;
(4)挑选合适的后端Web服务器&＃xff1b;

可以使用下面的终止语句&＃xff0c;即通过return()向Varnish返回的指示操作&＃xff1a;
pass&＃xff1a;绕过缓存&＃xff0c;即不从缓存中查询内容或不将内容存储至缓存中&＃xff1b;
pipe&＃xff1a;不对客户端进行检查或做出任何操作&＃xff0c;而是在客户端与后端服务器之间建立专用“管道”&＃xff0c;并直接将数据在二者之间进行传送&＃xff1b;此时&＃xff0c;keep-alive连接中后续传送的数据也都将通过此管道进行直接传送&＃xff0c;并不会出现在任何日志中&＃xff1b;
lookup&＃xff1a;在缓存中查找用户请求的对象&＃xff0c;如果缓存中没有其请求的对象&＃xff0c;后续操作很可能会将其请求的对象进行缓存&＃xff1b;
error&＃xff1a;由Varnish自己合成一个响应报文&＃xff0c;一般是响应一个错误类信息、重定向类信息或负载均衡器返回的后端web服务器健康状态检查类信息&＃xff1b;

vcl_recv也可以通过精巧的策略完成一定意义上的安全功能&＃xff0c;以将某些特定的攻击扼杀于摇篮中。同时&＃xff0c;它也可以检查出一些拼写类的错误并将其进行修正等。

Varnish默认的vcl_recv专门设计用来实现安全的缓存策略&＃xff0c;它主要完成两种功能&＃xff1a;
(1)仅处理可以识别的HTTP方法&＃xff0c;并且只缓存GET和HEAD方法&＃xff1b;
(2)不缓存任何用户特有的数据&＃xff1b;

安全起见&＃xff0c;一般在自定义的vcl_recv中不要使用return()终止语句&＃xff0c;而是再由默认vcl_recv进行处理&＃xff0c;并由其做出相应的处理决策。

**********************************************************************************************************************************************
sub vcl_recv {

# Happens before we check if we have this in cache already.## Typically you clean up the request here, removing COOKIEs you don&＃39;t need,
# rewriting the request, etc.if (req.url ~ "(?i)\.(jpg|png|gif)$") {set req.backend_hint &＃61; web1;} else {set req.backend_hint &＃61; web2;} if (req.method &＃61;&＃61; "PRI") { #如果是http 2.0 不缓存/* We do not support SPDY or HTTP/2.0 */return (synth(405));}set req.backend_hint &＃61; cluster1.backend(); #设置默认的后端缓存集群if (req.method !&＃61; "GET" &&req.method !&＃61; "HEAD" &&req.method !&＃61; "PUT" &&req.method !&＃61; "POST" &&req.method !&＃61; "TRACE" &&req.method !&＃61; "OPTIONS" &&req.method !&＃61; "DELETE") {/* Non-RFC2616 or CONNECT which is weird. */return (pipe); #如果请求方法都不是上面这些&＃xff0c;就直接发到pipe,对客户端进行检查或做出任何操作&＃xff0c;而是在客户端与后端服务器之间建立专用“管道”&＃xff0c;并直接将数据在二者之间进行传送&＃xff1b;此时&＃xff0c;keep-alive连接中后续传送的数据也都将通过此管道进行直接传送&＃xff0c;并不会出现在任何日志中&＃xff1b;}if (req.method !&＃61; "GET" && req.method !&＃61; "HEAD") {/* We only deal with GET and HEAD by default */return (pass); #绕过缓存&＃xff0c;即不从缓存中查询内容或不将内容存储至缓存中&＃xff1b;}if (req.http.Authorization || req.http.COOKIE) {/* Not cacheable by default */return (pass); #绕过缓存&＃xff0c;即不从缓存中查询内容或不将内容存储至缓存中&＃xff1b;

}
return (hash); #交给hash 处理&＃xff0c;判断是否能够缓存
}
sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-COOKIE headers
# and other mistakes your backend does.
if (beresp.http.cache-control !~ "s-maxage") { #定义varnish 定义对于特定资源的缓存时间

if (bereq.url ~ "(?i)\.jpg$") {
set beresp.ttl &＃61; 3600s;
unset beresp.http.Set-COOKIE;
}
if (bereq.url ~ "(?i)\.css$") {
set beresp.ttl &＃61; 600s;
unset beresp.http.Set-COOKIE;
}
}
}
*******************************************************************************************

5、vcl_fetch

如前面所述&＃xff0c;相对于vcl_recv是根据客户端的请求作出缓存决策来说&＃xff0c;vcl_fetch则是根据服务器端的响应作出缓存决策。在任何VCL状态引擎中返回的pass操作都将由vcl_fetch进行后续处理。vcl_fetch中有许多可用的内置变量&＃xff0c;比如最常用的用于定义某对象缓存时长的beresp.ttl变量。通过return()返回给varnish的操作指示有&＃xff1a;
(1)deliver&＃xff1a;缓存此对象&＃xff0c;并将其发送给客户端(经由vcl_deliver)&＃xff1b;
(2)hit_for_pass&＃xff1a;不缓存此对象&＃xff0c;但可以导致后续对此对象的请求直接送达到vcl_pass进行处理&＃xff1b;
(3)restart&＃xff1a;重启整个VCL&＃xff0c;并增加重启计数&＃xff1b;超出max_restarts限定的最大重启次数后将会返回错误信息&＃xff1b;
(4)error code [reason]&＃xff1a;返回指定的错误代码给客户端并丢弃此请求&＃xff1b;

默认的vcl_fetch放弃了缓存任何使用了Set-COOKIE首部的响应。

***************************************************************************************************************




sub vcl_deliver {# Happens when we have all the pieces we need, and are about to send the# response to the client.## You can do accounting or modifying the final object here.
if (obj.hits>0) { #判断在响应客户端时&＃xff0c;如果命中缓存就响应HIT ,否则 MISSset resp.http.test-Cache &＃61; "HIT" &＃43; server.ip;} else {set resp.http.X-Cahce &＃61; "MISS" &＃43; server.ip;}
}[root&＃64;localhost varnish]#