搜索到
26
篇与
Linux
的结果
-
阿里云服务器运维手册 通用服务管理查看进程CPU、内存占用情况，使用top命令toptop - 11:00:51 up 3 days, 1:36, 2 users, load average: 3.25, 3.42, 3.43 Tasks: 263 total, 2 running, 261 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.7 us, 1.1 sy, 0.0 ni, 95.8 id, 0.3 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem : 16266088 total, 1096464 free, 4306656 used, 10862968 buff/cache KiB Swap: 1049596 total, 1049596 free, 0 used. 10904716 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16387 elsearch 20 0 4444956 989524 26580 S 3.3 6.1 21:10.49 java 14530 root 20 0 329804 81656 10940 S 2.7 0.5 4:54.78 php 4178 mysql 20 0 1279992 382532 5948 S 2.3 2.4 458:37.19 mysqld 31286 www 20 0 481492 52736 35760 S 2.0 0.3 0:01.97 php-fpm 1411 root 10 -10 150048 31688 10532 R 1.7 0.2 99:16.20 AliYunDunMonito 3513 redis 20 0 353668 200220 1840 S 1.0 1.2 28:07.38 redis-server 25480 www 20 0 480508 56912 41792 S 1.0 0.3 0:20.98 php-fpm 1333 root 10 -10 101012 8580 6532 S 0.7 0.1 16:42.43 AliYunDun 4353 root 20 0 328612 88812 11256 S 0.7 0.5 1:16.44 php 286 root 20 0 0 0 0 S 0.3 0.0 3:27.25 jbd2/vda1-8 371 root 20 0 112692 59260 58896 S 0.3 0.4 3:04.46 systemd-journal 输入大写M按内存占用从大到小排序top - 11:02:56 up 3 days, 1:38, 2 users, load average: 2.12, 3.05, 3.30 Tasks: 262 total, 1 running, 261 sleeping, 0 stopped, 0 zombie %Cpu(s): 1.8 us, 0.9 sy, 0.0 ni, 96.3 id, 0.8 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem : 16266088 total, 1082872 free, 4322468 used, 10860748 buff/cache KiB Swap: 1049596 total, 1049596 free, 0 used. 10888904 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16387 elsearch 20 0 4445332 989696 26752 S 1.0 6.1 21:12.93 java 4178 mysql 20 0 1279992 382532 5948 S 4.3 2.4 458:53.16 mysqld 3513 redis 20 0 353668 200176 1840 S 0.7 1.2 28:08.05 redis-server 3859 www 20 0 6604368 164104 9256 S 0.0 1.0 3:01.53 jsvc 4182 mongo 20 0 501656 100108 10092 S 0.3 0.6 15:17.80 mongod 4353 root 20 0 328612 88812 11256 S 0.0 0.5 1:16.48 php 14530 root 20 0 329804 81832 10940 S 2.7 0.5 4:57.80 php 14773 www 20 0 483260 78600 59420 S 0.0 0.5 0:25.37 php-fpm 15172 www 20 0 483544 78304 59036 S 0.0 0.5 0:24.68 php-fpm 15498 www 20 0 482956 76880 59428 S 0.0 0.5 0:24.56 php-fpm 15073 www 20 0 482648 76444 59020 S 0.0 0.5 0:25.39 php-fpm 10644 www 20 0 480876 75548 58436 S 0.0 0.5 0:26.21 php-fpm 输入大写P按CPU占用从高到低排序top - 11:06:43 up 3 days, 1:42, 2 users, load average: 1.82, 2.38, 2.98 Tasks: 266 total, 1 running, 265 sleeping, 0 stopped, 0 zombie %Cpu(s): 5.5 us, 1.7 sy, 0.0 ni, 92.4 id, 0.3 wa, 0.0 hi, 0.2 si, 0.0 st MiB Mem : 15884.9 total, 1012.9 free, 4251.8 used, 10620.2 buff/cache MiB Swap: 1025.0 total, 1025.0 free, 0.0 used. 10601.0 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 24878 root 20 0 0.0t 0.0t 0.0t S 4.7 0.3 0:02.58 php 4178 mysql 20 0 0.0t 0.0t 0.0t S 2.3 2.4 459:03.77 mysqld 1411 root 10 -10 0.0t 0.0t 0.0t S 2.0 0.2 99:23.68 AliYunDunMonito 14530 root 20 0 0.0t 0.0t 0.0t S 2.0 0.5 5:03.22 php 599 root 20 0 0.0t 0.0t 0.0t S 1.0 0.2 13:08.92 exe 578 root 20 0 0.0t 0.0t 0.0t S 0.3 0.0 1:05.68 rngd 1059 root 20 0 0.0t 0.0t 0.0t S 0.3 0.0 3:15.33 AliYunDunUpdate 1333 root 10 -10 0.0t 0.0t 0.0t S 0.3 0.1 16:43.78 AliYunDun 3513 redis 20 0 0.0t 0.0t 0.0t S 0.3 1.2 28:09.43 redis-server 4182 mongo 20 0 0.0t 0.0t 0.0t S 0.3 0.6 15:18.59 mongod 4305 root 20 0 0.0t 0.0t 0.0t S 0.3 0.3 5:41.52 python 16387 elsearch 20 0 0.0t 0.0t 0.0t S 0.3 6.1 21:18.46 java 查看硬盘占用情况，使用df命令df -h[root@boass-cesi ~]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 276K 7.8G 1% /dev/shm tmpfs 7.8G 888K 7.8G 1% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup /dev/vda1 296G 75G 209G 27% / tmpfs 1.6G 0 1.6G 0% /run/user/0 tmpfs 1.6G 0 1.6G 0% /run/user/1010 tmpfs 1.6G 0 1.6G 0% /run/user/1003查看内存占用情况，使用free命令free -h[root@boass-cesi ~]# free -h total used free shared buff/cache available Mem: 15G 4.1G 1.0G 699M 10G 10G Swap: 1.0G 0B 1.0G查看当前文件夹硬盘空间占用情况du -h --max-depth=1[root@boass-cesi devtpro]# du -h --max-depth=1 1.8G ./src 5.1G ./public 2.5M ./extend 140K ./config 32K ./errpage 2.2M ./thinkphp 192K ./tests 76K ./route 110M ./runtime 13M ./application 8.0K ./hopo_shell 72K ./logs 67M ./vendor 7.0G .查看php相关进程ps aux | grep php[root@boass-cesi ~]# ps aux | grep php www 1817 0.0 0.4 480636 68232 ? S Jun29 0:22 php-fpm: pool www www 3480 0.0 0.1 267804 16644 ? S 10:04 0:04 php-fpm: pool www root 3528 0.0 0.0 458844 9972 ? Ss Jun27 0:10 php-fpm: master process (/www/server/php/73/etc/php-fpm.conf) root 3530 0.0 0.0 325748 11552 ? Ss Jun27 0:07 php-fpm: master process (/www/server/php/72/etc/php-fpm.conf) root 3534 0.0 0.0 259952 7912 ? Ss Jun27 0:08 php-fpm: master process (/www/server/php/71/etc/php-fpm.conf) www 3555 0.0 0.0 326008 14032 ? S Jun27 0:00 php-fpm: pool www root 4353 0.0 0.5 328612 88812 ? Ss Jun27 1:16 /www/server/php/71/bin/php /www/wwwroot/xiezuo.hopo.com.cn/think queue:work --daemon devops 4362 0.0 0.3 297168 52996 ? Ss Jun27 0:30 /www/server/php/71/bin/php /www/wwwroot/devtpro/src/cron/think queue:work --daemon --queue config_scheme_excel_to_image devops 4368 0.0 0.3 360992 57628 ? Ss Jun27 0:31 /www/server/php/72/bin/php /www/wwwroot/devtpro/src/cron/think queue:work --daemon --queue config_scheme_pdf_to_image平滑重启php-fpm找到php-fpm的主进程的pidps aux | grep "php-fpm: master process"[root@boass-cesi ~]# ps aux | grep "php-fpm: master process" root 3528 0.0 0.0 458844 9972 ? Ss Jun27 0:10 php-fpm: master process (/www/server/php/73/etc/php-fpm.conf) root 3530 0.0 0.0 325748 11552 ? Ss Jun27 0:07 php-fpm: master process (/www/server/php/72/etc/php-fpm.conf) root 3534 0.0 0.0 259952 7912 ? Ss Jun27 0:08 php-fpm: master process (/www/server/php/71/etc/php-fpm.conf) root 31597 0.0 0.0 112820 988 pts/0 S+ 11:25 0:00 grep --color=auto php-fpm: master process对指定进程pid(第二列)执行重启命令kill -USR2 3528服务器管理245服务器管理（门窗工匠、crm测试环境）启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload启动elasticsearch切换到elsearch用户su elsearch切换到elasticsearch的bin目录cd /usr/local/elasticsearch-6.5.0/bin/执行启动命令./elasticsearch -d重启elasticsearch首先停止elasticsearchps aux | grep elasticsearch | awk '{print $2}' | xargs kill然后执行上面提到的启动命令./elasticsearch -delasticsearch由于硬盘空间不足锁住写入不了数据的解决办法执行ifconfig命令获取本机内网IP[elsearch@boass-cesi elasticsearch-6.5.0]$ ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.18.100.239 netmask 255.255.240.0 broadcast 172.18.111.255 ether 00:16:3e:0a:11:23 txqueuelen 1000 (Ethernet) RX packets 278132727 bytes 98192582498 (91.4 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 270448689 bytes 106768594070 (99.4 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 63320337 bytes 8774142838 (8.1 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 63320337 bytes 8774142838 (8.1 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0通过curl解锁 curl -XPUT -H "Content-Type: application/json" http://172.18.100.239:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'清理门窗工匠日志切换到门窗工匠源码目录cd /www/wwwroot/devtpro/src/列出所有版本的文件夹[root@boass-cesi src]# ls 2.2.6 2.2.7 2.2.8 2.2.9 2.3.0 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 cron latest_version latest_version.conf logs runtime进入到某个版本的logs目录，例如2.3.5cd 2.3.5/logs/删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*启动异步任务队列进程service supervisor start停止异步任务队列进程service supervisor stop重载异步任务队列进程service supervisor reload启动workerman/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think worker:server -d重启workerman/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think worker:server reload停止workerman/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think worker:server stop127服务器管理（门窗工匠、CRM正式环境）启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload启动elasticsearch切换到elsearch用户su elsearch切换到elasticsearch的bin目录cd /usr/local/elasticsearch-6.5.0/bin/执行启动命令./elasticsearch -d重启elasticsearch首先停止elasticsearchps aux | grep elasticsearch | awk '{print $2}' | xargs kill然后执行上面提到的启动命令./elasticsearch -delasticsearch由于硬盘空间不足锁住写入不了数据的解决办法执行ifconfig命令获取本机内网IP[elsearch@boass-cesi elasticsearch-6.5.0]$ ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.18.100.241 netmask 255.255.240.0 broadcast 172.18.111.255 ether 00:16:3e:10:6a:c9 txqueuelen 1000 (Ethernet) RX packets 1578355690 bytes 834544632419 (777.2 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1494209189 bytes 236255611334 (220.0 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 7747730607 bytes 1193148967061 (1.0 TiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7747730607 bytes 1193148967061 (1.0 TiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0通过curl解锁 curl -XPUT -H "Content-Type: application/json" http://172.18.100.241:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'清理门窗工匠日志切换到门窗工匠源码目录cd /www/zhua/src列出所有版本的文件夹[root@boass-cesi src]# ls 2.2.6 2.2.7 2.2.8 2.2.9 2.3.0 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 cron latest_version latest_version.conf logs runtime进入到某个版本的logs目录，例如2.3.5cd 2.3.5/logs/删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*启动异步任务队列进程service supervisor start停止异步任务队列进程service supervisor stop重载异步任务队列进程service supervisor reload启动workerman/usr/local/php/bin/php /www/zhua/src/cron/think worker:server -d重启workerman/usr/local/php/bin/php /www/zhua/src/cron/think worker:server reload停止workerman/usr/local/php/bin/php /www/zhua/src/cron/think worker:server stop23服务器管理（gitlab、Jenkins、协同平台测试环境）清理协同平台日志进入到协同平台日志目录，包括log_new以及log_new/erpcd /www/xietong/log_new删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*清理完之后进入erp目录继续上面的命令清理cd /www/xietong/log_new/erp启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload重启gitlabsystemctl stop gitlab-runsvdir && systemctl start gitlab-runsvdir进入Jenkins docker容器docker exec -it 39 /bin/bash105服务器管理（协同平台正式环境）清理协同平台日志进入到协同平台日志目录，包括log_new以及log_new/erpcd /www/xietong/log_new删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*清理完之后进入erp目录继续上面的命令清理cd /www/xietong/log_new/erp启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload清理协同平台日志进入到协同平台日志目录，包括log_new以及log_new/erpcd /www/web/xt_admin_boass_com/public_html/log_new删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*清理完之后进入erp目录继续上面的命令清理cd /www/web/xt_admin_boass_com/public_html/log_new/erp启动redisservice redis_6379 start 重启redisservice redis_6379 restart启动Nginxservice nginxd start 重载Nignxservice nginxd reload -
使用thinkphp、think-queue、redis、supervisor搭建异步任务中心 为什么要搭建异步任务中心执行耗时任务作为web开发者，经常会遇到从用户操作开始到服务响应结束耗时较长的功能，例如导出导入大批量的Excel数据，给成千上万的用户发送邮件或信息，文件打包下载等等，如果采用同步的方式，响应时间太久，服务端或客户端（浏览器）连接会断开，导致任务中止，数据不一致，影响用户体验，采用异步的方式可以避免这个问题。统一管理异步任务，方便维护与开发将异步任务统一管理，有利于统一做异常处理，方便将代码进行组件化，提高开发效率。分担应用服务器的压力异步任务中心可以独立部署到另外一台服务器，与应用服务器分开，将耗时耗资源的任务从应用服务器剥离，减轻应用服务器压力。如何搭建异步任务中心流程 用户操作发起任务请求，应用服务接收到任务之后投递到异步任务中心，异步任务中心异步处理完任务之后，将结果反馈到应用服务，用户可以通过任务列表页面查看任务执行的结果。用到的技术和工具thinkphp5.1，用于编写接口与业务逻辑。think-queue2.0+redis，用于创建消息队列执行异步任务。supervisor，对think-queue创建的队列进程进行管理。MySQL，用于保存任务记录。步骤（以Excel导出任务为例）创建MySQL异步任务表CREATE TABLE `async_task` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key', `user_id` int(10) unsigned NOT NULL DEFAULT '0' COMMENT '用户ID（cor_admin_user表user_id）', `name` varchar(100) NOT NULL DEFAULT '' COMMENT '任务名称', `task_type` tinyint(4) NOT NULL DEFAULT '0' COMMENT '任务类型：0文件打包，1云盘打包下载，2Excel导出，3Excel导入', `identification` varchar(60) NOT NULL DEFAULT '' COMMENT '任务标识：详见AsyncTaskEnum文件', `task_params` text COMMENT '任务参数：例如导出列表的查询条件等等', `status` tinyint(4) NOT NULL DEFAULT '0' COMMENT '状态：0进行中，1完成，2失败', `out_file_name` varchar(120) NOT NULL DEFAULT '' COMMENT '输出文件名称', `download_url` varchar(255) NOT NULL DEFAULT '' COMMENT '下载路径', `exception_msg` varchar(255) NOT NULL DEFAULT '' COMMENT '异常信息', `create_time` datetime NOT NULL COMMENT '创建时间', `update_time` datetime DEFAULT NULL COMMENT '最后更新时间', `agency_user_id` int(10) unsigned NOT NULL DEFAULT '0' COMMENT '经销商的用户ID', PRIMARY KEY (`id`), KEY `idx_user_id` (`user_id`) ) ENGINE=InnoDB AUTO_INCREMENT=1810 DEFAULT CHARSET=utf8 COMMENT='异步任务表' ;定义Excel导出接口，后面需要导出的Excel服务类可以实现这个接口进行导出<?php namespace app\contract\excel; use think\queue\Job; interface ExportTaskInterface { /** * 开始导出任务 * @param array $params 创建任务的参数 * @return mixed */ public function startExportTask(array $params) : array; /** * 导出Excel * @param array $params * @return mixed */ public function doExportTask(array $params) : array; /** * 任务失败时的处理方法 * @param \Exception $e * @param array $params * @param Job $job * @return mixed */ public function exportTaskFailedHandle(\Exception $e, array $params, Job $job) : array; /** * 任务完成时的处理方法 * @param array $exportResult * @param Job $job * @return mixed */ public function taskFinishHandle(array $exportResult, Job $job) : array; }定义Excel导出Job,这个Job就会调用实现了ExportTaskInterface接口服务类相关的导出方法<?php namespace app\job\jobs; use app\common\MyLog; use app\contract\excel\ExportTaskInterface; use app\library\exception\ServiceLogicException; use think\queue\Job; class ExportExcelJob { public function fire(Job $job, $data) { if (!isset($data['handler'])){ $job->delete(); throw new ServiceLogicException('缺少处理类'); } try { $this->handleJob(new $data['handler'], $data, $job); }catch (\Exception $e){ var_dump($e->getFile() . $e->getLine() . $e->getMessage()); MyLog::errorLog('error', $e->getFile() . $e->getLine() . $e->getMessage(), 'export_excel_task'); $job->delete(); } } protected function handleJob(ExportTaskInterface $exportTask, $data, Job $job) { MyLog::infoLog('export_excel_task', '接收到Excel导出任务:' . json_encode($data), 'export_excel_task'); try { $result = $exportTask->doExportTask($data); $exportTask->taskFinishHandle($result, $job); MyLog::infoLog('export_excel_task_res', '执行成功:' . json_encode($result), 'export_excel_task'); }catch (\Exception $e){ MyLog::infoLog('export_excel_task_err', $e->getFile() . $e->getLine() . $e->getMessage(), 'export_excel_task'); $exportTask->exportTaskFailedHandle($e, $data, $job); } } public function fail(Job $job, $data) { $job->delete(); } }编写业务逻辑类，实现ExportTaskInterface接口，并补充导出相关的方法<?php namespace app\admin\service; use app\admin\model\UserInfo; use app\common\MyLog; use app\models\Order; use app\contract\excel\ExportTaskInterface; use app\library\enum\AsyncTaskEnum; use app\library\exception\ServiceLogicException; use app\library\traits\QueryBuilder; use app\models\AsyncTask; use app\service\common\XlsWriterService; use app\service\QueueService; use think\queue\Job; use third_library\LibTime; class FirstOrderCustomerService extends BaseService implements ExportTaskInterface { protected $mapField = 'firstOrderCustomerList'; public function index($params) { $returnType = $params['return_type'] ?? 'list'; if ($returnType == 'export'){ return $this->startExportTask($params); } $query = $this->getQuery($params); return $query->paginate($params['per_page'], null, ['page' => $params['page']]) ->toArray(); } protected function getQuery($params) { $query = Order::alias('o') ->leftJoin(UserInfo::getTable() . ' ui', 'o.user_id = ui.user_id') ->where('o.is_first_order', 1) ->append([ 'order_time', ])->field([ 'ui.company_name', 'o.add_time', 'o.goods_amount', 'o.order_sn', 'ui.salesman', 'ui.service_name' ]); $this->buildQuery(function ($field, $size, $keyword, $searchType) use ($query){ if ($searchType == 2){ // 下单时间 if ($size == '='){ $startTime = $keyword . ' ' . '00:00:00'; $endTime = $keyword . ' ' . '23:59:59'; $startTime = LibTime::getInstance()->local_strtotime($startTime); $endTime = LibTime::getInstance()->local_strtotime($endTime); $query->where($field, '>=', $startTime); $query->where($field, '<=', $endTime); }else{ $keyword = LibTime::getInstance()->local_strtotime($keyword); $query->where($field, $size, $keyword); } }else{ $query->where($field, $size, $keyword); } }, $params); return $query; } public function startExportTask(array $params): array { $order = Order::where('is_first_order', 1)->find(); if (empty($order)){ throw new ServiceLogicException('没有可导出的数据'); } $name = '今日下单客户' . date("YmdHis"); $data = [ 'user_id' => $params['admin_id'], 'name' => $name, 'task_type' => AsyncTaskEnum::TASK_TYPE_EXCEL_EXPORT, 'identification' => AsyncTaskEnum::IDENTIFICATION_EXCEL_EXPORT, 'task_params' => json_encode($params), 'out_file_name' => $name . '.xlsx', ]; $taskId = AsyncTask::createTask($data); $params['task_id'] = $taskId; $params['handler'] = self::class; QueueService::push('excel_export', $params); return ['msg' => '已创建导出任务，请在下载中心查看', 'task_id' => $taskId]; } public function doExportTask(array $params): array { $query = $this->getQuery($params); $now = time(); $filename = "今日下单客户{$params['admin_id']}_{$now}.xlsx"; $field = [ 'company_name' => ['name' =>'订单编号'], 'order_time' => ['name' =>'首次下单时间'], 'goods_amount' => ['name' =>'首次下单金额'], 'order_sn' => ['name' =>'订单号'], 'salesman' => ['name' =>'现销售对接人'], 'service_name' => ['name' =>'内勤名称'] ]; $fileObj = XlsWriterService::getInstance() ->constMemory($filename, null, false) ->field($field); $page = 1; while (true){ $orders = $query->page($page, 500) ->order('order_id', 'desc') ->select() ->toArray(); if (empty($orders)) break; $items = []; foreach ($orders as $val) { $item = []; foreach ($field as $key => $value) { $item[$key] = $val[$key] ?? ''; } $items[] = array_values($item); } !empty($items) && $fileObj->data($items); $page++; } $filePath = $fileObj->output(); return ['file_path' => $filePath, 'task_id' => $params['task_id']]; } public function exportTaskFailedHandle(\Exception $e, array $params, Job $job): array { AsyncTask::where('id', $params['task_id'])->update([ 'status' => AsyncTaskEnum::TASK_STATUS_FAIL, 'exception_msg' => $e->getMessage() ]); $job->delete(); return []; } public function taskFinishHandle(array $exportResult, Job $job): array { AsyncTask::where('id', $exportResult['task_id'])->update([ 'status' => AsyncTaskEnum::TASK_STATUS_FINISH, 'download_url' => $exportResult['file_path'] ]); $job->delete(); return []; } }为了能让异步任务顺利的运行，需要对startExportTask、doExportTask、exportTaskFailedHandle，taskFinishHandle四个方法补充相应的逻辑startExportTask，定义导出文件的名称，创建导出任务保存到数据库，并将任务投递到ExportExcelJob执行doExportTask，获取导出的数据，整理数据格式，调用xlswrite插件导出Excel文件exportTaskFailedHandle，任务失败后，更改任务的状态为失败，并保存失败的原因taskFinishHandle，任务成功后，更改状态为成功，并保存下载链接这四个方法对需要导出Excel的服务都适用，可以封装成trait，这样就不用在每个业务逻辑类重复写这四个方法，在后面的文章再详细讲讲。启动异步任务队列，在项目的根目录执行下面命令，也可以使用绝对路径执行，启动成功后，用户进行导出操作，接口把任务投递到当前队列，就可以执行导出任务了php think queue:work --daemon --queue excel_export使用supervisor对任务队列进程进行管理，以centos为例安装教程可以在网上参考，安装完之后在/etc/supervisord.d文件夹创建supervisor.ini文件，并写入以下内容，就可以使用supervisor进行管理了。;导出Excel [program:excel_export_task] command=/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think queue:work --daemon --queue excel_export directory=/www/wwwroot/devtpro/src/cron process_name=%(process_num)02d numprocs=1 autostart=true autorestart=true startsecs=1 startretries=20 redirect_stderr=true user=devops最后，编写接口查询async_task表记录，展示到前端页面让用户进行下载，至此，就完成了Excel异步导出功能的开发总结其它的异步任务与上面的Excel导出任务大同小异，参考上面的例子增加相应的服务即可。
-
debian10编译安装php7.0.33 下载源码cd /usr/local/src wget https://www.php.net/distributions/php-7.0.33.tar.gz解压源码包tar -zxvf php-7.0.33.tar.gz执行configure脚本，可以执行./configure --help查询相关配置./configure --prefix=/usr/local/php7.0 \ --enable-fpm \ --with-openssl \ --with-zlib \ --enable-bcmath \ --with-curl \ --enable-ftp \ --with-gd \ --enable-mbstring \ --with-mcrypt \ --with-mysqli \ --enable-pcntl \ --with-pdo-mysql \ --enable-soap \ --enable-zip \ --enable-mysqlnd \编译安装make && make install
-
-
Linux系统centos通过php、thinkphp5.1框架将HTML网页转为图片方法 思路使用PHP调用chrome浏览器来完成centos安装chrome浏览器添加源vim /etc/yum.repos.d/google-chrome.repo[google-chrome] name=google-chrome baseurl=https://dl.google.com/linux/chrome/rpm/stable/x86_64 enabled=1 gpgcheck=1 gpgkey=https://dl.google.com/linux/linux_signing_key.pub执行安装命令yum -y install google-chrome-stable如果安装失败则执行yum -y install google-chrome-stable通过chrome-php插件以及chrome的headless模式将HTML转为图片安装chrome-php插件composer require chrome-php/chromedemo$browserFactory = new BrowserFactory(Env::get('chrome.bin_path', '/opt/google/chrome/chrome')); $browserFactory->setOptions([ 'userAgent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36 Edg/101.0.1210.32', 'windowSize' => [1920, 1000], ]); $browser = $browserFactory->createBrowser(); try { $page = $browser->createPage(); $url = Env::get('webpage.customer_contacts_relation_graph_url') . "?company_id=$companyId"; $page->navigate($url)->waitForNavigation(Page::NETWORK_IDLE, 60000); $saveFilePath = $this->getSavePath($companyId); $page->screenshot([ 'captureBeyondViewport' => true, 'clip' => $page->getFullPageClip(), 'format' => 'jpeg', // default to 'png' - possible values: 'png', 'jpeg', 'quality' => 100 ])->saveToFile($saveFilePath); return $saveFilePath; }catch (\Exception $e){ $browser->close(); throw $e; } finally { $browser->close(); }
-
-
阿里云rds数据库mysql版cpu占用100%排查问题过程 背景某天去到公司，用户反馈说系统响应很慢，甚至有些功能基本用不了，严重影响工作效率。测试同事说阿里云RDS数据库的CPU占用了100%，我打开阿里云dms的监控页面，确实如此，接着开始排查问题。问题排查过程因为前段时间遇到过代码死循环查询数据库导致了CPU占用100%的问题，所以一开始就往这个方向排查。首先通过阿里云的dms控制台执行show processlist命令查询数据库正在执行的线程如下 发现大量状态为sending data的线程，通过查询资料，这个状态的意思是读取和过滤数据，如果一个表数据量很大且没有索引的话，就容易出现sending data状态，官方解释是，线程正在读取和处理 SELECT 的行记录，发送给客户端，由于在这个状态下的操作会执行大量的磁盘访问（读），因此它通常是查询生命周期最长的状态。但具体为什么会占用大量CPU，还有待学习了解。之前的死循环会出现在一些同步erp数据到业务系统的定时任务中，因为需要同步大量的数据而使用了循环。使用上面的sql语句（info字段）搜索代码，发现只有一个定时任务的方法使用了这个语句，因此接着登录服务器查询这个定时任务，但是没有找到相应的定时任务，后来了解到原来一直没有配置，但是同事在前一天手动执行过这个定时任务，通过如下监控信息 发现从前一天下午5点09分开始数据库的CPU就开始飙升到100%了。通过ps命令找不到相关的定时任务进程正在执行，但为什么那些语句一直存在呢？然后在运维同事的帮助下，使用lsof和ll /proc/pid命令以及上面show processlist结果的端口查询到发起那个语句请求的进程是php-fpm，工作目录是tp框架的public目录，印象中tp框架的php think命令运行的模式是cli，因此导致sending data状态有可能不是定时任务导致的。因为当时没有及时排查出原因，所以采取了同事的建议将服务器重启。 重启后数据库的CPU确实降下来了，但是过了一会又升上去了。确定了不是由于定时任务导致的，接着继续排查代码中调用这个定时任务方法的地方，后来确实发现还有一个接口调用了这个方法，通过查询nginx的access.log日志，发现当时一直有IP频繁地请求着这个接口，初步认定是由于这个接口调用了这个方法导致的问题。接着和同事一起去查看这个方法的代码，但是却没有发现这个方法的问题，逻辑上产生死循环的逻辑也没有被发现。在测试环境模拟复现这个问题，发现在死循环中获取一个数据量为三万的表的同一条数据时（查询条件有加索引并命中）并不会导致CPU飙升，但是没有索引或没有命中索引的情况下，CPU就会一直飙升。 从上面排查的过程中，没有发现死循环，考虑到有些单据数据行比较多，所以模拟了短时间内循环查询并修改大量不同单据的情景，在命中索引的情况下，CPU占用不高，但是没有命中索引的情况下，CPU占用升到百分之五十多。接着看了生产环境相关的表，数据量大概是十万行，而且用于查询的entry_id字段并没有设置索引，因此同步数据时，如果数据量较大，循环插入或者更新数据时会导致数据库CPU占用持续升高。通过下面的监控图 119.23.123.173 - - [21/Apr/2022:11:10:03 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194500 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.176 - - [21/Apr/2022:11:10:07 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194249 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.174 - - [21/Apr/2022:11:10:09 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194045 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.179 - - [21/Apr/2022:11:10:09 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194045 HTTP/1.1" 201 1003 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.174 - - [21/Apr/2022:11:10:14 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194552 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.167 - - [21/Apr/2022:11:10:14 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194552 HTTP/1.1" 201 1892 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.164 - - [21/Apr/2022:11:10:17 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194613 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.154 - - [21/Apr/2022:11:10:26 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194548 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.169 - - [21/Apr/2022:11:10:31 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194229 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 47.112.84.217 - - [21/Apr/2022:11:10:31 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194249 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.165 - - [21/Apr/2022:11:10:32 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194264 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.178 - - [21/Apr/2022:11:10:35 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194592 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.170 - - [21/Apr/2022:11:10:39 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194500 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.159 - - [21/Apr/2022:11:10:41 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194580 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.170 - - [21/Apr/2022:11:10:41 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194580 HTTP/1.1" 201 1125 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.154 - - [21/Apr/2022:11:10:46 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194576 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.170 - - [21/Apr/2022:11:10:46 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194576 HTTP/1.1" 201 1296 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.154 - - [21/Apr/2022:11:10:51 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194536 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.171 - - [21/Apr/2022:11:10:53 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194613 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 47.112.84.213 - - [21/Apr/2022:11:10:56 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194264 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.163 - - [21/Apr/2022:11:10:57 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194524 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.172 - - [21/Apr/2022:11:11:02 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194548 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 47.112.84.216 - - [21/Apr/2022:11:11:02 +0800] "GET /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194500 HTTP/1.1" 499 0 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400" 119.23.123.166 - - [21/Apr/2022:11:11:06 +0800] "OPTIONS /abc_admin/v2/sourceOrderPhoto/getODODetail?inquire_type=2&express_no=10111194508 HTTP/1.1" 200 5 "https://cadmin.hopo.com.cn/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 Core/1.77.103.400 QQBrowser/10.9.4624.400"结合同事在前一天手动执行了定时任务并且是全量获取数据以及nginx的access.log日志的情况，从前一天执行了定时任务开始一直到第二天11点08分的CPU都是占满的状态，然后重启了服务器，CPU降了下来，接着因为相关接口被频繁调用，所以又导致CPU重新占满。问题原因数据量大的表且用于查询的字段没有设置索引，导致查询效率低。执行定时任务时没有传入增量获取的参数，采用了全量获取数据的方式进行同步数据，数据量较大，一直占用着数据库资源。解决办法与预防措施需要给数据量较大的表频繁查询的字段添加索引同步大量数据时，采用增量的方式，如果是需要重新获取全量数据，则尽量在晚上执行，且要考虑同步的时长是否会影响白天系统的使用。优化同步数据的方法，尽量避免在循环中调用数据库，造成数据库资源占用过高，影响系统性能。
-
