概述
公司所属产品服务器负载异常CPU接近满载,导致业务线无法运行.
排查
获取异常服务进程
使用 top 或 ps 查看异常进程PID
查询对应进程所运行服务
systemctl status 1655
查看所加载的配置文件信息
cat /etc/systemd/system/multi-user.target.wants/pmapx_start_2.service
停止并删除对应进程任务
systemctl stop pmapx_start_2
systemctl disable pmapx_start_2
rm -f /etc/systemd/system/multi-user.target.wants/pmapx_start_2.service
rm -f /usr/sbin/route_forbidden-clos
临时解决方法
封堵对应IP
iptables -I INPUT -s 139.99.124.170 -j DROP
iptables -I OUTPUT -s 139.99.124.170 -j DROP
其他
删除你看到的异常文件并kill掉对应进程
rm -rf /usr/lib/dev/systemd/systemd-dev
rm -rf /usr/lib/dev/systemd/systemd-udevd-run.sh
仅通过ssh登录执行的命令
[root@localhost ~]# systemctl status session-13.scope
● session-13.scope - Session 13 of user root
Loaded: loaded (/run/systemd/system/session-13.scope; static; vendor preset: disabled)
Drop-In: /run/systemd/system/session-13.scope.d
└─50-After-systemd-logind\x2eservice.conf, 50-After-systemd-user-sessions\x2eservice.conf, 50-Description.conf, 50-SendSIGHUP.conf, 50-Slice.conf, 50-TasksMax.conf
Active: active (abandoned) since 一 2021-12-06 23:02:21 CST; 3 days ago
CGroup: /user.slice/user-0.slice/session-13.scope
├─42258 ./systemdd
├─42269 /usr/bin/bsd-port/getty
└─42304 /usr/bin/.sshd
12月 06 23:02:21 localhost.localdomain systemd[1]: Started Session 13 of user root.
12月 06 23:02:21 localhost.localdomain sshd[42175]: pam_unix(sshd:session): session opened for user root by (uid=0)
12月 06 23:02:54 localhost.localdomain yum[42196]: Installed: wget-1.14-18.el7_6.1.x86_64
12月 06 23:02:54 localhost.localdomain yum[42196]: Installed: net-tools-2.0-0.25.20131004git.el7.x86_64
12月 06 23:04:55 localhost.localdomain sshd[42175]: pam_unix(sshd:session): session closed for user root
[root@localhost session-13.scope.d]# systemctl stop session-13.scope