So recently Abuse.ch, BrillantIT, and Proofpoint partnered up to take down the botnet known as EITest.
The group behind the botnet have been in operation since around 2011 according to the researchers. After the take down Spamhaus was given control of the Command and Control(C2) domains.
Update 4 (4/26/18) Now there is an Eitest cleanup part 2 with yara rule signatures to help with detection. But use in combination with the technique below to ensure you’ve found everything.
So somewhere in the neighborhood of 52,000 servers are now getting listed on the Spamhaus CBL blacklist, particularly an issue if you do have to send mail from the server still phoning home the the C2 domains. In the servers that I work on they are often hosting multiple domains on a server, and are often setup in a single server setup so no fancy network visibility setup.
Much of the malicious code also seems to be pretty stealthy against our standard malware scanning tools so even after running those you might get lucky and be able to find a compromised domain that overlaps with the EITest compromise, or you may miss it altogether. What we really need to be sure is to catch the code “in the act.” The code that contacts the botnet only triggers when the page with the malware is loaded in a browser.
One of my friends and associates whipped up the following script:
(Updated scripts later below)
Source http://sh.mdsc.info/eitest-connection-watch.sh
#!/bin/bash # Eitest investigation script # https://www.abuseat.org # https://sitecheck.sucuri.net/ # Watch for connections to sinkhole connect=$(ss -antp | grep 192.42.116.41); # If connection found then capture data if [[ $connect ]]; then # Get pid from connection PID=$(echo $connect | grep -Eio "pid=[0-9]{1,}" | cut -d= -f2); # Strace pid (strace -yrTfs 1024 -e trace=sendto,connect,open,write -o strace-$PID.out -p $PID &) # Log some basic info about the connection and process ps aux | awk "(\$2 ~ /$PID/)"'{print $0}' >> connect-log.txt; echo $connect >> connect-log.txt; fi
So the idea is you drop this into a script on the affected server.
[root@ex ~]# mkdir eitest-locate [root@ex ~]# cd eitest-locate/ [root@ex eitest-locate]# vim eitest-locate.sh [root@ex eitest-locate]# chmod 755 eitest-locate.sh [root@ex eitest-locate]# screen -S eitest-locate [root@ex eitest-locate]# watch -n.1 ./eitest-locate.sh
Let the screen run and once the botnet page is browsed by a user, crawler, or other entity you get 2 forms of output:
-rw-r--r-- 1 root root 18266 Apr 17 20:38 connect-log.txt -rw-r--r-- 1 root root 1888 Apr 17 20:37 strace-10031.out
The connect log confirms the connection to the Spamhaus sinkhole.
SYN-SENT 0 1 198.51.100.20:58908 192.42.116.41:80 users:(("php-cgi",pid=8084,fd=5))
example 10031 0.0 0.4 532084 34820 ? S 20:37 0:00 /opt/cpanel/ea-php56/root/usr/bin/php-cgi /home/example/public_html/info/index.php
And the 2nd part of the script grabs a trace of the program responsible for the connection.
99275 0.000028 write(1<pipe:[258386788]>, "\r\n", 2) = 2 <0.000010> 99275 0.000028 write(1<pipe:[258386788]>, "\r\n", 2) = 2 <0.000009> 99275 0.000026 write(1<pipe:[258386788]>, "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \r\n\r\n\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n<head \r\n\r\nprofile=\"http://gmpg.org/xfn/11\">\r\n<title>\r\nEXAM » Example</title>\r\n<meta \r\n\r\nhttp-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\r\n<meta name=\"generator\" content=\"WordPress 4.5.14\" />\r\n<!-- leave this for stats please -->\r\n<style type=\"text/css\" media=\"screen\">\r\n@import url( http://www.example.org/wp-content/themes/example/style.css );\r\n</style>\r\n<link rel=\"alternate\" type=\"application/rss+xml\" title=\"RSS 2.0\" href=\"http://www.example.org/feed\" />\r\n<link rel=\"alternate\" type=\"text/xml\" title=\"RSS .92\" href=\"http://www.example.org/feed/rss\" />\r\n<link rel=\"alternate\" type=\"application/atom+xml\" title=\"Atom 0.3\" href=\"http://www.example.org/feed/atom\" />\r\n<link rel=\"pingback\" href=\"http://www.example.org/xmlrpc.php\" />\r\n\t<link rel='archives' title='November 2015' href='http://www.example"..., 32678) = 32678 <0.000012> 99275 0.000947 sendto(4, "\1\0\0\0\1", 5, MSG_DONTWAIT, NULL, 0) = 5 <0.000065> 99275 0.006883 +++ exited with 0 +++ ~
So with these 2 pieces of information we have a positive ID on the file that houses the malicious code. From here we can report the findings to the server owner to allow for remediation to begin.
Now our environment is based on Cent6 and Cent7 servers and we did notice what appears to be different implementations of the ss tool on 6 vs 7, so on Cent6 we had to adjust the script slightly to use netstat:
#!/bin/bash # Eitest investigation script # https://www.abuseat.org # https://sitecheck.sucuri.net/ # Watch for connections to sinkhole connect=$(netstat -tpn | grep 192.42.116.41); # If connection found then capture data if [[ $connect ]]; then # Get pid from connection PID=$(echo $connect | awk '{print$7}' | cut -d '/' -f1); # Strace pid (strace -yrTfs 1024 -e trace=sendto,connect,open,write -o strace-$PID.out -p $PID &) # Log some basic info about the connection and process ps aux | awk "(\$2 ~ /$PID/)"'{print $0}' >> connect-log.txt; echo $connect >> connect-log.txt; fi
So have you been dealing with cleaning up this botnet too, hit me up on twitter @laskow26. Good luck, and happy hunting!
Update2
(4/23/18), after reviewing the CBL updated recommendations and their perl watch script, Mark whipped up a updated bash script, and I updated the netstat version drop in a screen without the need for the watch command and adds a lsof output for additional data points:
#!/bin/bash
# +----+----+----+----+
# | | | | |
# Author: Mark David Scott Cunningham | M | D | S | C |
# +----+----+----+----+
# Created: 2018-04-16
# Updated: 2018-04-23
#
# Purpose: Eitest investigation script
#
# Watch for connections to sinkhole
sinkhole="192.42.116.41"
# Repeat in loop until you stop the script
while true; do
connect=$(ss -antp | grep $sinkhole);
# If connection found then capture data
if [[ $connect ]]; then
# Get pid from connection
PID=$(echo $connect | grep -Eio "pid=[0-9]{1,}" | cut -d= -f2);
# Strace pid
(strace -yrTfs 1024 -e trace=sendto,connect,open,write -o eitest-trace-$PID.out -p $PID &)
# Get open files from lsof
(lsof -p $PID > eitest-files-$PID.log &)
# Log some basic info about the connection and process
ps aux | awk "(\$2 ~ /$PID/)"'{print $0}' >> eitest-connection-log.txt;
echo $connect >> eitest-connection-log.txt;
fi
sleep 0.01
done
#!/bin/bash
# Created: 2018-04-16
# Updated: 2018-04-23
#
# Purpose: Eitest investigation script
#
# Watch for connections to sinkhole
sinkhole="192.42.116.41"
# Repeat in loop until you stop the script
while true; do
connect=$(netstat -tpn | grep $sinkhole);
# If connection found then capture data
if [[ $connect ]]; then
# Get pid from connection
PID=$(echo $connect | awk '{print$7}' | cut -d '/' -f1);
# Strace pid
(strace -yrTfs 1024 -e trace=sendto,connect,open,write -o eitest-trace-$PID.out -p $PID &)
# Get open files from lsof
(lsof -p $PID > eitest-files-$PID.log &)
# Log some basic info about the connection and process
ps aux | awk "(\$2 ~ /$PID/)"'{print $0}' >> eitest-connection-log.txt;
echo $connect >> eitest-connection-log.txt;
fi
sleep 0.01
done
Update 3 (04/25/18)
I have now found 5 different cases where the injection was detectable by the following yara rule from the Yara github repo:
And see part 2 of the Eitest cleanup series.
/*
I first found this in May 2016, appeared in every PHP file on the
server, cleaned it with `sed` and regex magic. Second time was
in June 2016, same decoded content, different encoding/naming.
https://www.symantec.com/security_response/writeup.jsp?docid=2015-111911-4342-99
*/
rule php_anuna
{
meta:
author = "Vlad https://github.com/vlad-s"
date = "2016/07/18"
description = "Catches a PHP Trojan"
strings:
$a = /<\?php \$[a-z]+ = '/
$b = /\$[a-z]+=explode\(chr\(\([0-9]+[-+][0-9]+\)\)/
$c = /\$[a-z]+=\([0-9]+[-+][0-9]+\)/
$d = /if \(!function_exists\('[a-z]+'\)\)/
condition:
all of them
}
Update
(04/21/18):
Here’s some of the injections found using this method.
