作者:多米音乐_35677591 | 来源:互联网 | 2023-05-17 17:06
主要使用 swoole的子协程 和 协程channel 的两个特征 实现任务并发。
直接上代码说明:
$server = new SwooleHttpServer("0.0.0.0",20000,SWOOLE_PROCESS,SWOOLE_SOCK_TCP);
$server->on("request",function(SwooleHttpRequest $request,SwooleHttpResponse $response){
$fd = $request->fd;
if($request->server["request_uri"] == "/favicon.ico"){
$this->server->close($fd);
return false;
}
$urls = [
"http://www.estb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25885.html",
"http://www.sweb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_23660.html",
"http://www.daiinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25596.html",
"http://www.us764.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_16724.html",
"http://www.vieinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25571.html",
"http://www.estb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25885.html",
"http://www.sweb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_23660.html",
"http://www.daiinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25596.html",
"http://www.us764.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_16724.html",
"http://www.vieinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25571.html",
"http://www.estb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25885.html",
"http://www.sweb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_23660.html",
"http://www.daiinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25596.html",
"http://www.us764.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_16724.html",
"http://www.vieinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25571.html",
"http://www.estb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25885.html",
"http://www.sweb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_23660.html",
"http://www.daiinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25596.html",
"http://www.us764.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_16724.html",
"http://www.vieinfo.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25571.html",
"http://www.estb2b.com/product-detail/BPA-free-80--120-80mm-cash-register-paper-80--120-70mm-thermal-paper-rolls_25885.html",
......
];
$taskCount = count($urls);
$channel = new SwooleCoroutineChannel($taskCount); // 用于子协程与 主进程之间的通信
$start = time();
foreach ($urls as $url){
go(function()use($channel,$url){
// 在swoole协程中 http 客户端强烈推荐使用composer包 swlib/saber , 该包库功能强大,缺点是只能在 协程环境中使用
// 该包库是由 swoole 核心开发组开发,用于代替swoole原生协程 客户端
// 使用详情参考 https://github.com/swlib/saber
try{
$saberRespOnse= SwlibSaberGM::get($url); // 返回 SwlibSaberResponse 对象
$code = $saberResponse->getStatusCode();
$r = $code . " => " . strlen($saberResponse->getBody());
$channel->push($r);
}catch(Exception $e){
$channel->push("Exception : ".$e->getMessage());
}
});
}
for ($i = 1; $i <= $taskCount; $i++){
// 阻塞 等待处理结果
$r= $channel->pop();
var_dump($r);
}
$diff = time() - $start;
echo "全部完成需要时间差 diff={$diff}s n";
$response->end("返回响应数据到客户端");
});
$server->start();
经本人实测,在同时请求 任务多的情况下,性能尤为明显。以本案例爬取海外站点源码为例,性能比多进程process模型更高,所用时间只需要process模块1/3的时间。
重点掌握 #协程运行生命周期, #协程通信Channel, #协程Http客户端swlib/saber 的使用。
重点说明:协程模式环境下(也是使用协程的缺点),编写的代码与传统同步串行代码存在部分差异,某些pecl扩展和阻塞代码不能在协程环境下使用。
例如: curl扩展,sleep睡眠函数等不能在协程环境模式下使用(详细参考swoole文档)但是swoole官方开发组为之提供了对应的解决方案。使用swoole官方提供的函数或者库能很好解决对用的问题。(例如 mysql/redis客户端等在协程模式下必须使用协程客户端(最好使用基于协程连接池的pool),否则并不能体现很好的性能)