搜索到
104
篇与
basil
的结果
-
-
-
-
Debian11安装Microsoft ODBC驱动 在Debian11系统中打开终端。使用root用户并执行以下命令更新软件包列表：sudo apt update执行以下命令通过APT安装unixodbc软件包：sudo apt install unixodbc unixodbc-dev安装Microsoft ODBC驱动程序。你可以从官方网站下载安装文件，或使用以下命令手动下载并安装：curl https://packages.microsoft.com/config/debian/11/prod.list > /etc/apt/sources.list.d/mssql-release.list curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -通过以下命令安装Microsoft ODBC驱动程序：sudo apt update sudo apt install msodbcsql17如果在安装过程中出现任何问题，请检查日志并按照提示进行解决。如果一切顺利，则可以为Microsoft SQL Server连接配置ODBC数据源。
-
阿里云服务器运维手册 通用服务管理查看进程CPU、内存占用情况，使用top命令toptop - 11:00:51 up 3 days, 1:36, 2 users, load average: 3.25, 3.42, 3.43 Tasks: 263 total, 2 running, 261 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.7 us, 1.1 sy, 0.0 ni, 95.8 id, 0.3 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem : 16266088 total, 1096464 free, 4306656 used, 10862968 buff/cache KiB Swap: 1049596 total, 1049596 free, 0 used. 10904716 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16387 elsearch 20 0 4444956 989524 26580 S 3.3 6.1 21:10.49 java 14530 root 20 0 329804 81656 10940 S 2.7 0.5 4:54.78 php 4178 mysql 20 0 1279992 382532 5948 S 2.3 2.4 458:37.19 mysqld 31286 www 20 0 481492 52736 35760 S 2.0 0.3 0:01.97 php-fpm 1411 root 10 -10 150048 31688 10532 R 1.7 0.2 99:16.20 AliYunDunMonito 3513 redis 20 0 353668 200220 1840 S 1.0 1.2 28:07.38 redis-server 25480 www 20 0 480508 56912 41792 S 1.0 0.3 0:20.98 php-fpm 1333 root 10 -10 101012 8580 6532 S 0.7 0.1 16:42.43 AliYunDun 4353 root 20 0 328612 88812 11256 S 0.7 0.5 1:16.44 php 286 root 20 0 0 0 0 S 0.3 0.0 3:27.25 jbd2/vda1-8 371 root 20 0 112692 59260 58896 S 0.3 0.4 3:04.46 systemd-journal 输入大写M按内存占用从大到小排序top - 11:02:56 up 3 days, 1:38, 2 users, load average: 2.12, 3.05, 3.30 Tasks: 262 total, 1 running, 261 sleeping, 0 stopped, 0 zombie %Cpu(s): 1.8 us, 0.9 sy, 0.0 ni, 96.3 id, 0.8 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem : 16266088 total, 1082872 free, 4322468 used, 10860748 buff/cache KiB Swap: 1049596 total, 1049596 free, 0 used. 10888904 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16387 elsearch 20 0 4445332 989696 26752 S 1.0 6.1 21:12.93 java 4178 mysql 20 0 1279992 382532 5948 S 4.3 2.4 458:53.16 mysqld 3513 redis 20 0 353668 200176 1840 S 0.7 1.2 28:08.05 redis-server 3859 www 20 0 6604368 164104 9256 S 0.0 1.0 3:01.53 jsvc 4182 mongo 20 0 501656 100108 10092 S 0.3 0.6 15:17.80 mongod 4353 root 20 0 328612 88812 11256 S 0.0 0.5 1:16.48 php 14530 root 20 0 329804 81832 10940 S 2.7 0.5 4:57.80 php 14773 www 20 0 483260 78600 59420 S 0.0 0.5 0:25.37 php-fpm 15172 www 20 0 483544 78304 59036 S 0.0 0.5 0:24.68 php-fpm 15498 www 20 0 482956 76880 59428 S 0.0 0.5 0:24.56 php-fpm 15073 www 20 0 482648 76444 59020 S 0.0 0.5 0:25.39 php-fpm 10644 www 20 0 480876 75548 58436 S 0.0 0.5 0:26.21 php-fpm 输入大写P按CPU占用从高到低排序top - 11:06:43 up 3 days, 1:42, 2 users, load average: 1.82, 2.38, 2.98 Tasks: 266 total, 1 running, 265 sleeping, 0 stopped, 0 zombie %Cpu(s): 5.5 us, 1.7 sy, 0.0 ni, 92.4 id, 0.3 wa, 0.0 hi, 0.2 si, 0.0 st MiB Mem : 15884.9 total, 1012.9 free, 4251.8 used, 10620.2 buff/cache MiB Swap: 1025.0 total, 1025.0 free, 0.0 used. 10601.0 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 24878 root 20 0 0.0t 0.0t 0.0t S 4.7 0.3 0:02.58 php 4178 mysql 20 0 0.0t 0.0t 0.0t S 2.3 2.4 459:03.77 mysqld 1411 root 10 -10 0.0t 0.0t 0.0t S 2.0 0.2 99:23.68 AliYunDunMonito 14530 root 20 0 0.0t 0.0t 0.0t S 2.0 0.5 5:03.22 php 599 root 20 0 0.0t 0.0t 0.0t S 1.0 0.2 13:08.92 exe 578 root 20 0 0.0t 0.0t 0.0t S 0.3 0.0 1:05.68 rngd 1059 root 20 0 0.0t 0.0t 0.0t S 0.3 0.0 3:15.33 AliYunDunUpdate 1333 root 10 -10 0.0t 0.0t 0.0t S 0.3 0.1 16:43.78 AliYunDun 3513 redis 20 0 0.0t 0.0t 0.0t S 0.3 1.2 28:09.43 redis-server 4182 mongo 20 0 0.0t 0.0t 0.0t S 0.3 0.6 15:18.59 mongod 4305 root 20 0 0.0t 0.0t 0.0t S 0.3 0.3 5:41.52 python 16387 elsearch 20 0 0.0t 0.0t 0.0t S 0.3 6.1 21:18.46 java 查看硬盘占用情况，使用df命令df -h[root@boass-cesi ~]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 276K 7.8G 1% /dev/shm tmpfs 7.8G 888K 7.8G 1% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup /dev/vda1 296G 75G 209G 27% / tmpfs 1.6G 0 1.6G 0% /run/user/0 tmpfs 1.6G 0 1.6G 0% /run/user/1010 tmpfs 1.6G 0 1.6G 0% /run/user/1003查看内存占用情况，使用free命令free -h[root@boass-cesi ~]# free -h total used free shared buff/cache available Mem: 15G 4.1G 1.0G 699M 10G 10G Swap: 1.0G 0B 1.0G查看当前文件夹硬盘空间占用情况du -h --max-depth=1[root@boass-cesi devtpro]# du -h --max-depth=1 1.8G ./src 5.1G ./public 2.5M ./extend 140K ./config 32K ./errpage 2.2M ./thinkphp 192K ./tests 76K ./route 110M ./runtime 13M ./application 8.0K ./hopo_shell 72K ./logs 67M ./vendor 7.0G .查看php相关进程ps aux | grep php[root@boass-cesi ~]# ps aux | grep php www 1817 0.0 0.4 480636 68232 ? S Jun29 0:22 php-fpm: pool www www 3480 0.0 0.1 267804 16644 ? S 10:04 0:04 php-fpm: pool www root 3528 0.0 0.0 458844 9972 ? Ss Jun27 0:10 php-fpm: master process (/www/server/php/73/etc/php-fpm.conf) root 3530 0.0 0.0 325748 11552 ? Ss Jun27 0:07 php-fpm: master process (/www/server/php/72/etc/php-fpm.conf) root 3534 0.0 0.0 259952 7912 ? Ss Jun27 0:08 php-fpm: master process (/www/server/php/71/etc/php-fpm.conf) www 3555 0.0 0.0 326008 14032 ? S Jun27 0:00 php-fpm: pool www root 4353 0.0 0.5 328612 88812 ? Ss Jun27 1:16 /www/server/php/71/bin/php /www/wwwroot/xiezuo.hopo.com.cn/think queue:work --daemon devops 4362 0.0 0.3 297168 52996 ? Ss Jun27 0:30 /www/server/php/71/bin/php /www/wwwroot/devtpro/src/cron/think queue:work --daemon --queue config_scheme_excel_to_image devops 4368 0.0 0.3 360992 57628 ? Ss Jun27 0:31 /www/server/php/72/bin/php /www/wwwroot/devtpro/src/cron/think queue:work --daemon --queue config_scheme_pdf_to_image平滑重启php-fpm找到php-fpm的主进程的pidps aux | grep "php-fpm: master process"[root@boass-cesi ~]# ps aux | grep "php-fpm: master process" root 3528 0.0 0.0 458844 9972 ? Ss Jun27 0:10 php-fpm: master process (/www/server/php/73/etc/php-fpm.conf) root 3530 0.0 0.0 325748 11552 ? Ss Jun27 0:07 php-fpm: master process (/www/server/php/72/etc/php-fpm.conf) root 3534 0.0 0.0 259952 7912 ? Ss Jun27 0:08 php-fpm: master process (/www/server/php/71/etc/php-fpm.conf) root 31597 0.0 0.0 112820 988 pts/0 S+ 11:25 0:00 grep --color=auto php-fpm: master process对指定进程pid(第二列)执行重启命令kill -USR2 3528服务器管理245服务器管理（门窗工匠、crm测试环境）启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload启动elasticsearch切换到elsearch用户su elsearch切换到elasticsearch的bin目录cd /usr/local/elasticsearch-6.5.0/bin/执行启动命令./elasticsearch -d重启elasticsearch首先停止elasticsearchps aux | grep elasticsearch | awk '{print $2}' | xargs kill然后执行上面提到的启动命令./elasticsearch -delasticsearch由于硬盘空间不足锁住写入不了数据的解决办法执行ifconfig命令获取本机内网IP[elsearch@boass-cesi elasticsearch-6.5.0]$ ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.18.100.239 netmask 255.255.240.0 broadcast 172.18.111.255 ether 00:16:3e:0a:11:23 txqueuelen 1000 (Ethernet) RX packets 278132727 bytes 98192582498 (91.4 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 270448689 bytes 106768594070 (99.4 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 63320337 bytes 8774142838 (8.1 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 63320337 bytes 8774142838 (8.1 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0通过curl解锁 curl -XPUT -H "Content-Type: application/json" http://172.18.100.239:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'清理门窗工匠日志切换到门窗工匠源码目录cd /www/wwwroot/devtpro/src/列出所有版本的文件夹[root@boass-cesi src]# ls 2.2.6 2.2.7 2.2.8 2.2.9 2.3.0 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 cron latest_version latest_version.conf logs runtime进入到某个版本的logs目录，例如2.3.5cd 2.3.5/logs/删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*启动异步任务队列进程service supervisor start停止异步任务队列进程service supervisor stop重载异步任务队列进程service supervisor reload启动workerman/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think worker:server -d重启workerman/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think worker:server reload停止workerman/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think worker:server stop127服务器管理（门窗工匠、CRM正式环境）启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload启动elasticsearch切换到elsearch用户su elsearch切换到elasticsearch的bin目录cd /usr/local/elasticsearch-6.5.0/bin/执行启动命令./elasticsearch -d重启elasticsearch首先停止elasticsearchps aux | grep elasticsearch | awk '{print $2}' | xargs kill然后执行上面提到的启动命令./elasticsearch -delasticsearch由于硬盘空间不足锁住写入不了数据的解决办法执行ifconfig命令获取本机内网IP[elsearch@boass-cesi elasticsearch-6.5.0]$ ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.18.100.241 netmask 255.255.240.0 broadcast 172.18.111.255 ether 00:16:3e:10:6a:c9 txqueuelen 1000 (Ethernet) RX packets 1578355690 bytes 834544632419 (777.2 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1494209189 bytes 236255611334 (220.0 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 7747730607 bytes 1193148967061 (1.0 TiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7747730607 bytes 1193148967061 (1.0 TiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0通过curl解锁 curl -XPUT -H "Content-Type: application/json" http://172.18.100.241:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'清理门窗工匠日志切换到门窗工匠源码目录cd /www/zhua/src列出所有版本的文件夹[root@boass-cesi src]# ls 2.2.6 2.2.7 2.2.8 2.2.9 2.3.0 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 cron latest_version latest_version.conf logs runtime进入到某个版本的logs目录，例如2.3.5cd 2.3.5/logs/删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*启动异步任务队列进程service supervisor start停止异步任务队列进程service supervisor stop重载异步任务队列进程service supervisor reload启动workerman/usr/local/php/bin/php /www/zhua/src/cron/think worker:server -d重启workerman/usr/local/php/bin/php /www/zhua/src/cron/think worker:server reload停止workerman/usr/local/php/bin/php /www/zhua/src/cron/think worker:server stop23服务器管理（gitlab、Jenkins、协同平台测试环境）清理协同平台日志进入到协同平台日志目录，包括log_new以及log_new/erpcd /www/xietong/log_new删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*清理完之后进入erp目录继续上面的命令清理cd /www/xietong/log_new/erp启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload重启gitlabsystemctl stop gitlab-runsvdir && systemctl start gitlab-runsvdir进入Jenkins docker容器docker exec -it 39 /bin/bash105服务器管理（协同平台正式环境）清理协同平台日志进入到协同平台日志目录，包括log_new以及log_new/erpcd /www/xietong/log_new删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*清理完之后进入erp目录继续上面的命令清理cd /www/xietong/log_new/erp启动redisservice redis start 重启redisservice redis restart启动Nginxservice nginx start 重载Nignxservice nginx reload清理协同平台日志进入到协同平台日志目录，包括log_new以及log_new/erpcd /www/web/xt_admin_boass_com/public_html/log_new删除某个日志rm -f 20230630_yunzhijia.log删除某一天的日志rm -f 20230630*删除某一个月的日志rm -f 202306*删除某一年的日志rm -f 2023*清理完之后进入erp目录继续上面的命令清理cd /www/web/xt_admin_boass_com/public_html/log_new/erp启动redisservice redis_6379 start 重启redisservice redis_6379 restart启动Nginxservice nginxd start 重载Nignxservice nginxd reload
-
使用thinkphp、think-queue、redis、supervisor搭建异步任务中心 为什么要搭建异步任务中心执行耗时任务作为web开发者，经常会遇到从用户操作开始到服务响应结束耗时较长的功能，例如导出导入大批量的Excel数据，给成千上万的用户发送邮件或信息，文件打包下载等等，如果采用同步的方式，响应时间太久，服务端或客户端（浏览器）连接会断开，导致任务中止，数据不一致，影响用户体验，采用异步的方式可以避免这个问题。统一管理异步任务，方便维护与开发将异步任务统一管理，有利于统一做异常处理，方便将代码进行组件化，提高开发效率。分担应用服务器的压力异步任务中心可以独立部署到另外一台服务器，与应用服务器分开，将耗时耗资源的任务从应用服务器剥离，减轻应用服务器压力。如何搭建异步任务中心流程 用户操作发起任务请求，应用服务接收到任务之后投递到异步任务中心，异步任务中心异步处理完任务之后，将结果反馈到应用服务，用户可以通过任务列表页面查看任务执行的结果。用到的技术和工具thinkphp5.1，用于编写接口与业务逻辑。think-queue2.0+redis，用于创建消息队列执行异步任务。supervisor，对think-queue创建的队列进程进行管理。MySQL，用于保存任务记录。步骤（以Excel导出任务为例）创建MySQL异步任务表CREATE TABLE `async_task` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key', `user_id` int(10) unsigned NOT NULL DEFAULT '0' COMMENT '用户ID（cor_admin_user表user_id）', `name` varchar(100) NOT NULL DEFAULT '' COMMENT '任务名称', `task_type` tinyint(4) NOT NULL DEFAULT '0' COMMENT '任务类型：0文件打包，1云盘打包下载，2Excel导出，3Excel导入', `identification` varchar(60) NOT NULL DEFAULT '' COMMENT '任务标识：详见AsyncTaskEnum文件', `task_params` text COMMENT '任务参数：例如导出列表的查询条件等等', `status` tinyint(4) NOT NULL DEFAULT '0' COMMENT '状态：0进行中，1完成，2失败', `out_file_name` varchar(120) NOT NULL DEFAULT '' COMMENT '输出文件名称', `download_url` varchar(255) NOT NULL DEFAULT '' COMMENT '下载路径', `exception_msg` varchar(255) NOT NULL DEFAULT '' COMMENT '异常信息', `create_time` datetime NOT NULL COMMENT '创建时间', `update_time` datetime DEFAULT NULL COMMENT '最后更新时间', `agency_user_id` int(10) unsigned NOT NULL DEFAULT '0' COMMENT '经销商的用户ID', PRIMARY KEY (`id`), KEY `idx_user_id` (`user_id`) ) ENGINE=InnoDB AUTO_INCREMENT=1810 DEFAULT CHARSET=utf8 COMMENT='异步任务表' ;定义Excel导出接口，后面需要导出的Excel服务类可以实现这个接口进行导出<?php namespace app\contract\excel; use think\queue\Job; interface ExportTaskInterface { /** * 开始导出任务 * @param array $params 创建任务的参数 * @return mixed */ public function startExportTask(array $params) : array; /** * 导出Excel * @param array $params * @return mixed */ public function doExportTask(array $params) : array; /** * 任务失败时的处理方法 * @param \Exception $e * @param array $params * @param Job $job * @return mixed */ public function exportTaskFailedHandle(\Exception $e, array $params, Job $job) : array; /** * 任务完成时的处理方法 * @param array $exportResult * @param Job $job * @return mixed */ public function taskFinishHandle(array $exportResult, Job $job) : array; }定义Excel导出Job,这个Job就会调用实现了ExportTaskInterface接口服务类相关的导出方法<?php namespace app\job\jobs; use app\common\MyLog; use app\contract\excel\ExportTaskInterface; use app\library\exception\ServiceLogicException; use think\queue\Job; class ExportExcelJob { public function fire(Job $job, $data) { if (!isset($data['handler'])){ $job->delete(); throw new ServiceLogicException('缺少处理类'); } try { $this->handleJob(new $data['handler'], $data, $job); }catch (\Exception $e){ var_dump($e->getFile() . $e->getLine() . $e->getMessage()); MyLog::errorLog('error', $e->getFile() . $e->getLine() . $e->getMessage(), 'export_excel_task'); $job->delete(); } } protected function handleJob(ExportTaskInterface $exportTask, $data, Job $job) { MyLog::infoLog('export_excel_task', '接收到Excel导出任务:' . json_encode($data), 'export_excel_task'); try { $result = $exportTask->doExportTask($data); $exportTask->taskFinishHandle($result, $job); MyLog::infoLog('export_excel_task_res', '执行成功:' . json_encode($result), 'export_excel_task'); }catch (\Exception $e){ MyLog::infoLog('export_excel_task_err', $e->getFile() . $e->getLine() . $e->getMessage(), 'export_excel_task'); $exportTask->exportTaskFailedHandle($e, $data, $job); } } public function fail(Job $job, $data) { $job->delete(); } }编写业务逻辑类，实现ExportTaskInterface接口，并补充导出相关的方法<?php namespace app\admin\service; use app\admin\model\UserInfo; use app\common\MyLog; use app\models\Order; use app\contract\excel\ExportTaskInterface; use app\library\enum\AsyncTaskEnum; use app\library\exception\ServiceLogicException; use app\library\traits\QueryBuilder; use app\models\AsyncTask; use app\service\common\XlsWriterService; use app\service\QueueService; use think\queue\Job; use third_library\LibTime; class FirstOrderCustomerService extends BaseService implements ExportTaskInterface { protected $mapField = 'firstOrderCustomerList'; public function index($params) { $returnType = $params['return_type'] ?? 'list'; if ($returnType == 'export'){ return $this->startExportTask($params); } $query = $this->getQuery($params); return $query->paginate($params['per_page'], null, ['page' => $params['page']]) ->toArray(); } protected function getQuery($params) { $query = Order::alias('o') ->leftJoin(UserInfo::getTable() . ' ui', 'o.user_id = ui.user_id') ->where('o.is_first_order', 1) ->append([ 'order_time', ])->field([ 'ui.company_name', 'o.add_time', 'o.goods_amount', 'o.order_sn', 'ui.salesman', 'ui.service_name' ]); $this->buildQuery(function ($field, $size, $keyword, $searchType) use ($query){ if ($searchType == 2){ // 下单时间 if ($size == '='){ $startTime = $keyword . ' ' . '00:00:00'; $endTime = $keyword . ' ' . '23:59:59'; $startTime = LibTime::getInstance()->local_strtotime($startTime); $endTime = LibTime::getInstance()->local_strtotime($endTime); $query->where($field, '>=', $startTime); $query->where($field, '<=', $endTime); }else{ $keyword = LibTime::getInstance()->local_strtotime($keyword); $query->where($field, $size, $keyword); } }else{ $query->where($field, $size, $keyword); } }, $params); return $query; } public function startExportTask(array $params): array { $order = Order::where('is_first_order', 1)->find(); if (empty($order)){ throw new ServiceLogicException('没有可导出的数据'); } $name = '今日下单客户' . date("YmdHis"); $data = [ 'user_id' => $params['admin_id'], 'name' => $name, 'task_type' => AsyncTaskEnum::TASK_TYPE_EXCEL_EXPORT, 'identification' => AsyncTaskEnum::IDENTIFICATION_EXCEL_EXPORT, 'task_params' => json_encode($params), 'out_file_name' => $name . '.xlsx', ]; $taskId = AsyncTask::createTask($data); $params['task_id'] = $taskId; $params['handler'] = self::class; QueueService::push('excel_export', $params); return ['msg' => '已创建导出任务，请在下载中心查看', 'task_id' => $taskId]; } public function doExportTask(array $params): array { $query = $this->getQuery($params); $now = time(); $filename = "今日下单客户{$params['admin_id']}_{$now}.xlsx"; $field = [ 'company_name' => ['name' =>'订单编号'], 'order_time' => ['name' =>'首次下单时间'], 'goods_amount' => ['name' =>'首次下单金额'], 'order_sn' => ['name' =>'订单号'], 'salesman' => ['name' =>'现销售对接人'], 'service_name' => ['name' =>'内勤名称'] ]; $fileObj = XlsWriterService::getInstance() ->constMemory($filename, null, false) ->field($field); $page = 1; while (true){ $orders = $query->page($page, 500) ->order('order_id', 'desc') ->select() ->toArray(); if (empty($orders)) break; $items = []; foreach ($orders as $val) { $item = []; foreach ($field as $key => $value) { $item[$key] = $val[$key] ?? ''; } $items[] = array_values($item); } !empty($items) && $fileObj->data($items); $page++; } $filePath = $fileObj->output(); return ['file_path' => $filePath, 'task_id' => $params['task_id']]; } public function exportTaskFailedHandle(\Exception $e, array $params, Job $job): array { AsyncTask::where('id', $params['task_id'])->update([ 'status' => AsyncTaskEnum::TASK_STATUS_FAIL, 'exception_msg' => $e->getMessage() ]); $job->delete(); return []; } public function taskFinishHandle(array $exportResult, Job $job): array { AsyncTask::where('id', $exportResult['task_id'])->update([ 'status' => AsyncTaskEnum::TASK_STATUS_FINISH, 'download_url' => $exportResult['file_path'] ]); $job->delete(); return []; } }为了能让异步任务顺利的运行，需要对startExportTask、doExportTask、exportTaskFailedHandle，taskFinishHandle四个方法补充相应的逻辑startExportTask，定义导出文件的名称，创建导出任务保存到数据库，并将任务投递到ExportExcelJob执行doExportTask，获取导出的数据，整理数据格式，调用xlswrite插件导出Excel文件exportTaskFailedHandle，任务失败后，更改任务的状态为失败，并保存失败的原因taskFinishHandle，任务成功后，更改状态为成功，并保存下载链接这四个方法对需要导出Excel的服务都适用，可以封装成trait，这样就不用在每个业务逻辑类重复写这四个方法，在后面的文章再详细讲讲。启动异步任务队列，在项目的根目录执行下面命令，也可以使用绝对路径执行，启动成功后，用户进行导出操作，接口把任务投递到当前队列，就可以执行导出任务了php think queue:work --daemon --queue excel_export使用supervisor对任务队列进程进行管理，以centos为例安装教程可以在网上参考，安装完之后在/etc/supervisord.d文件夹创建supervisor.ini文件，并写入以下内容，就可以使用supervisor进行管理了。;导出Excel [program:excel_export_task] command=/www/server/php/73/bin/php /www/wwwroot/devtpro/src/cron/think queue:work --daemon --queue excel_export directory=/www/wwwroot/devtpro/src/cron process_name=%(process_num)02d numprocs=1 autostart=true autorestart=true startsecs=1 startretries=20 redirect_stderr=true user=devops最后，编写接口查询async_task表记录，展示到前端页面让用户进行下载，至此，就完成了Excel异步导出功能的开发总结其它的异步任务与上面的Excel导出任务大同小异，参考上面的例子增加相应的服务即可。
-
thinkphp如何做接口日志监控 为什么要做接口日志记录方便快速排查问题 系统在运行过程中，偶尔会出现线上问题，通过接口日志可以快速地查询到请求的参数、运行过程中的异常以及响应的结果，有助于开发人员快速排查问题。方便操作审计 通过接口日志，可以快速查询到用户在什么时间对系统进行了什么操作，方便溯源和审计工作。接口请求频率与性能分析 通过日志记录，可以查询到接口在指定时间内请求的次数以及每次请求所耗费的时间，方便接口分析工作。如何做接口日志记录用到的技术与工具thinkphp框架异常处理、钩子和行为、think-queue队列supervisor进程管理工具PHP反射类流程 用户操作系统发起接口请求，系统处理请求过程中发生的异常以及处理正常的结果都通过抛出异常的方式响应给用户，这些异常会被预先定义好的异常处理器捕获，在异常处理器中获取本次请求的信息与结果，并推送到日志队列，日志队列再保存到日志数据库。步骤自定义异常基类<?php namespace app\library\exception; use app\common\MyLog; use think\Exception; /** * Class BaseException * 自定义异常类的基类 */ class BaseException extends Exception { public $code = 400; public $msg = '无效参数'; public $error_code = '999'; public $content = ''; public $request_url = ''; public $log_id = ''; /** * 构造函数，接收一个关联数组 * @param array $params 关联数组只应包含code、msg和error_code，且不应该是空值 */ public function __construct($params=[]) { if(!is_array($params)){ return; } if(array_key_exists('code',$params)){ $this->code = $params['code']; } if(array_key_exists('msg',$params)){ $this->msg = $params['msg']; } if(array_key_exists('error_code',$params)){ $this->error_code = (int)$params['error_code']; } if(array_key_exists('content',$params)){ $this->content = $params['content']; } if(array_key_exists('request_url',$params)){ $this->request_url = $params['request_url']; } $this->log_id = MyLog::setLogId(); } }定义成功和失败异常返回类,继承BaseException类<?php namespace app\library\exception; /** * 创建成功（如果不需要返回任何消息） */ class SuccessMessage extends BaseException { public $code = 201; public $msg = 'ok'; public $error_code = 0; public $content = ''; }<?php namespace app\library\exception; /** * 创建失败（如果不需要返回任何消息） */ class FailMessage extends BaseException { public $code = 201; public $msg = '操作失败'; public $error_code = 1; public $content = ''; }修改thinkphp异常处理器方法，捕获到接口抛出的SuccessMessage、FailMessage异常以及系统运行的异常，并在application/tags.php定义钩子行为，通过Hook::listen('before_response_result', $result)钩子传给日志处理类处理<?php namespace app\library\exception; use app\common\MyLog; use app\cooperation\service\EnDecryptDataService; use GuzzleHttp\Client; use GuzzleHttp\Exception\ClientException; use GuzzleHttp\Exception\RequestException; use think\exception\Handle; use Exception; use think\facade\Config; use think\facade\Env; use think\facade\Hook; use think\facade\Request; /* * 重写Handle的render方法，实现自定义异常消息 */ class ExceptionHandler extends Handle { private $code; private $msg; private $error_code; private $content; private $request_url; private $log_id; public function render(Exception $e) { if ($e instanceof BaseException) { //如果是自定义异常，则控制http状态码，不需要记录日志 //因为这些通常是因为客户端传递参数错误或者是用户请求造成的异常 不应当记录日志 $this->code = $e->code; $this->msg = $e->msg; $this->error_code = $e->error_code; $this->content = $e->content; $this->request_url = $e->request_url; $this->log_id = $e->log_id; $res = [ 'code' => $this->code, 'msg' => $this->msg, 'error_code' => $this->error_code, 'content' => $this->content, ]; MyLog::infoLog('[debug]',' [Url]:' . Request::url(true) . ' [Method]:' . Request::method() . ' [Header]:' . json_encode(Request::header(), JSON_UNESCAPED_UNICODE) . ' [Request]:'. json_encode(Request::param(),JSON_UNESCAPED_UNICODE) . ' [Response]:'. json_encode($res,JSON_UNESCAPED_UNICODE),'api'); } else{ $this->code = 500; $this->msg = 'sorry，这是一个未知错误. (^o^)Y'; $this->error_code = 999; $this->log_id = MyLog::setLogId(); if(config('app_debug')){ return parent::render($e); } } $result = [ 'msg' => $this->msg, 'error_code' => $this->error_code, 'content' => $this->content, 'scope' => session('scope'), 'log_id' => $this->log_id, ]; Hook::listen('before_response_result', $result); if ($e instanceof RpcResponseSuccess || $e instanceof RpcResponseFail){ return json(EnDecryptDataService::encryptData('hopo_share', json_encode($result, JSON_UNESCAPED_SLASHES|JSON_UNESCAPED_UNICODE))); } return json($result, $this->code); }<?php // +---------------------------------------------------------------------- // | ThinkPHP [ WE CAN DO IT JUST THINK ] // +---------------------------------------------------------------------- // | Copyright (c) 2006~2018 http://thinkphp.cn All rights reserved. // +---------------------------------------------------------------------- // | Licensed ( http://www.apache.org/licenses/LICENSE-2.0 ) // +---------------------------------------------------------------------- // | Author: liu21st <liu21st@gmail.com>1111 // +---------------------------------------------------------------------- // 应用行为扩展定义文件 return [ // 应用初始化 'app_init' => [ 'app\\admin\\behavior\\CORS', ], // 应用开始 'app_begin' => [], // 模块初始化 'module_init' => [], // 操作开始执行 'action_begin' => [ ], // 视图内容过滤 'view_filter' => [], // 日志写入 'log_write' => [], // 应用结束 'app_end' => [], // 返回结果前 'before_response_result' => [ \app\behavior\ApiRequestStatistics::class ] ];安装think-queue扩展，因为用的是thinkphp5.1框架，所以think-queue要选2.0版本 think-queue ，队列进程可以使用 supervisor 进行管理，使用队列来异步处理以及保存日志可以避免接口请求速度受到太大影响，安装后定义日志保存的Job，这里的例子是把日志保存到数据库，考虑到日志数量可能会很多，建议采用其它合适的方式存储composer require topthink/think-queue=^2.0 <?php namespace app\job\jobs; use app\common\MyLog; use app\models\ApiStatistics; use think\queue\Job; class ApiStatisticsJob { public function fire(Job $job, $data) { try { if (empty($data)){ $job->delete(); return false; } ApiStatistics::insertGetId($data); $job->delete(); return true; }catch (\Exception $e){ MyLog::errorLog('api_statistics_data', $data, 'api_statistics'); MyLog::errorLog('api_statistics', $e->getMessage() . $e->getFile() . $e->getLine(), 'api_statistics'); $job->delete(); } return true; } public function fail() { } }定义日志处理类，通过反射类以及thinkphp自带的方法获取接口相关信息，例如请求的模块、用户、接口地址、接口参数、请求时间、接口描述等等，并把这些信息推送到上面定义好的日志处理Job来保存日志<?php namespace app\behavior; use app\common\MyLog; use app\models\ClientElement; use app\service\LoginSv; use app\service\QueueService; use think\facade\App; use think\facade\Request; class ApiRequestStatistics { const MODULE_MAP = [ 'admin' => '总部管理端', 'agency' => '经销商管理端', 'api' => '客户端', 'saler' => 'CRM端', 'cooperation' => '开放接口' ]; public function run($response) { try { $this->saveApiInfo($response); $this->saveClientInfo(); $this->saveSearchInfo($response); }catch (\Exception $e){ MyLog::infoLog('debug', $e->getFile() . $e->getLine() . $e->getMessage(), 'api_statistics_and_client_element_track_debug'); } } protected function saveApiInfo($response) { $action = Request::action(true); $class = $this->parseClass(); $reflection = new \ReflectionClass($class); $classInfo = $this->getClassInfo($reflection); $methodInfo = $this->getMethodInfo($reflection, $action); $start = App::getBeginTime(); $end = microtime(true); $timeConsuming = bcsub($end, $start, 6); $userInfo = $this->getUserInfo(); $module = Request::module(); $data = [ 'user_id' => $userInfo['user_id'], 'user_name' => $userInfo['user_name'], 'user_agent' => Request::header('user-agent') ?: '', 'ip' => Request::ip() ?: '', 'module' => $module, 'module_name' => self::MODULE_MAP[$module] ?? '', 'class' => $classInfo['class'], 'class_name' => $classInfo['class_name'], 'method' => $methodInfo['method'], 'method_name' => $methodInfo['method_name'], 'url' => Request::path(), 'request_params' => json_encode(Request::param(), JSON_UNESCAPED_UNICODE|JSON_UNESCAPED_SLASHES), 'response_content' => json_encode($response, JSON_UNESCAPED_SLASHES|JSON_UNESCAPED_UNICODE), 'time_consuming' => bcmul($timeConsuming, 1000, 1), 'create_time' => time() ]; QueueService::push('api_statistics', $data); } protected function saveClientInfo() { $elementIds = Request::param('element_ids', ''); if (!$elementIds){ return true; } $elementIds = explode(',', $elementIds); if (!is_array($elementIds) || empty($elementIds)){ return true; } $elements = ClientElement::where('element_id', 'in', $elementIds)->select()->toArray(); $userInfo = $this->getUserInfo(); foreach ($elements as $element) { $data = [ 'element_id' => $element['id'], 'element_type' => $element['element_type'], 'element_title' => $element['element_title'], 'user_id' => $userInfo['user_id'], 'user_name' => $userInfo['user_name'], 'user_agent' => Request::header('user-agent') ?: '', 'ip' => Request::ip() ?: '', 'create_time' => time() ]; QueueService::push('client_element_track', $data); } return true; } protected function parseClass() { $module = Request::module(); $controller = Request::controller(); $controllerData = explode('.', $controller); $controllerDir = ''; if (count($controllerData) == 2){ $controllerDir = strtolower($controllerData[0]); } $className = end($controllerData); $class = "app\\$module\\controller\\"; $controllerDir && $class .= "$controllerDir\\"; $class .= "$className"; return $class; } protected function getClassInfo(\ReflectionClass $reflectionClass) { $class = $reflectionClass->getName(); $doc = $reflectionClass->getDocComment(); $className = ''; if(preg_match("/\/\*\*\s+\*(.*?)\s+\*/", $doc, $match)) { if(isset($match[1]) && $match[1]) { $className = $match[1]; } } return [ 'class' => trim($class), 'class_name' => trim($className) ]; } /** * 获取方法信息 * @param \ReflectionClass $reflectionClass * @param $action * @return array * @throws \ReflectionException */ protected function getMethodInfo(\ReflectionClass $reflectionClass, $action) { $method = $reflectionClass->getMethod($action); $name = $method->getName(); $document= $method->getDocComment(); preg_match("/\/\*\*\s+\*(.*?)\s+\*/", $document, $content); return [ 'method' => trim($name), 'method_name' => trim($content[1] ?? '') ]; } protected function getUserInfo() { $userInfo = [ 'user_id' => 0, 'user_name' => '未知' ]; $module = Request::module(); if (!isset(self::MODULE_MAP[$module])){ return $userInfo; } $authorization = Request()->header('Authorization'); if (!$authorization){ return $userInfo; } if ($module == 'admin'){ // 总部管理端 $token = $authorization; }else{ // 经销商管理端、客户端、CRM端 preg_match('/^Bearer\s+(.*?)$/', $authorization, $matches); $token = $matches[1] ?? ''; } if (empty($token)){ return $userInfo; } $loginSv = $loginService = new LoginSv($token); $user = $loginSv->getAllUsers(); if ($module == 'admin'){ // 总部管理端 $userInfo = [ 'user_id' => $user['admin_id'] ?? 0, 'user_name' => $user['admin_name'] ?? '未知' ]; }else{ // 经销商管理端、客户端、CRM端 $userInfo = [ 'user_id' => $user['staff_id'] ?? 0, 'user_name' => $user['staff_name'] ?? '未知' ]; } return $userInfo; } protected function saveSearchInfo($response){ if(Request::path()!='api/v2/search_goods'){ return false; } $keywords=Request::param('keywords'); $list= $response['content']['lists']['data'] ?? []; $content=[]; foreach ($list as $item){ $content[]=[ "goods_id"=> $item['goods_id']??'', "goods_name"=> $item['goods_name']??'', "goods_name_full"=> $item['goods_name_full']??'', "goods_sn"=> $item['goods_sn']??'', "goods_sn_full"=> $item['goods_sn_full']??'', ]; unset($item); } if(empty($keywords)){ return false; } $start = App::getBeginTime(); $end = microtime(true); $timeConsuming = bcsub($end, $start, 6); $userInfo = $this->getUserInfo(); $data = [ 'user_id' => $userInfo['user_id'], 'user_name' => $userInfo['user_name'], 'user_agent' => Request::header('user-agent') ?: '', 'ip' => Request::ip() ?: '', 'url' => Request::path(), 'keyword' => trim($keywords), 'content' => json_encode($content, JSON_UNESCAPED_SLASHES|JSON_UNESCAPED_UNICODE), 'time_consuming' => bcmul($timeConsuming, 1000, 1), 'create_time' => time() ]; QueueService::push('save_search_history', $data); } }编写日志查询接口以及前端页面，将日志展示出来。由于日志量大，采用MySQL等数据库储存日志查询性能不好，可采用elk,MongoDB,Zincsearch这些存储方案
-
PHP一些很实用但是被忽略的知识点 Exception的getTraceAsString、getTrace、getPrevious方法有时候程序出现的异常是前面的步骤引起的，例如thinkphp框架查询数据库时某个字段不存在，通过下面的方法只能够定位到抛出错误的文件、行以及错误信息，但是通过这些信息很难排查到在哪里设置了这个不存在的字段$e->getMessage(); $e->getLine(); $e->getFile();在文件D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\db\Connection.php 687行 发生错误：SQLSTATE[42S22]: Column not found: 1054 Unknown column 'idd' in 'where clause'这时候可以通过getTraceAsString、getTrace、getPrevious方法来捕获调用栈信息来判断异常的原因try { $res = DiyPackPartSku::where('idd', 3453)->find()->delete(); }catch (\Exception $e){ echo $e->getTraceAsString(); // var_dump($e->getTrace()); // echo $e->getPrevious(); }可以打印出调用栈来判断#0 D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\db\Connection.php(844): think\db\Connection->query('SELECT * FROM `...', Array, false, false) #1 D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\db\Query.php(3152): think\db\Connection->find(Object(think\db\Query)) #2 D:\phpstudy_pro\WWW\new_boass\application\service\repairdata\Demo.php(331): think\db\Query->find() #3 D:\phpstudy_pro\WWW\new_boass\application\service\repairdata\RepairDataFactory.php(30): app\service\repairdata\Demo->testDelete(NULL, NULL, NULL, NULL, NULL, NULL) #4 D:\phpstudy_pro\WWW\new_boass\application\command\RepairData.php(52): app\service\repairdata\RepairDataFactory->__call('testDelete', Array) #5 D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\console\Command.php(175): app\command\RepairData->execute(Object(think\console\Input), Object(think\console\Output)) #6 D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\Console.php(675): think\console\Command->run(Object(think\console\Input), Object(think\console\Output)) #7 D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\Console.php(261): think\Console->doRunCommand(Object(app\command\RepairData), Object(think\console\Input), Object(think\console\O utput)) #8 D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\Console.php(198): think\Console->doRun(Object(think\console\Input), Object(think\console\Output)) #9 D:\phpstudy_pro\WWW\new_boass\thinkphp\library\think\Console.php(115): think\Console->run() #10 D:\phpstudy_pro\WWW\new_boass\think(22): think\Console::init() #11 {main}
![PHP连接SQLserver报错：SQLSTATE[IMSSP]: This extension requires the Microsoft ODBC Driver for SQL Server to communicate with SQL Server. Access the followin](https://basil-blog-1252044342.cos.ap-guangzhou.myqcloud.com/npm/typecho-joe-next@6.0.0/assets/thumb/10.jpg)
