download (1)

How I just got Hacked and Dealt with a Botnet/DDos Abuse of my Server

5.44 PM

So first of all, I got an email at 5.44pm from my Hosting Company, the Abuse department telling me to deal with the complaint and give a statement, so here is how the email looked: “Mass bruteforce attemps from your network”

Sehr geehrte(r) Herr Mikula Dedours,

wir haben einen Spam- bzw. Abuse-Hinweis von abuse@valuehost.ru erhalten.
Bitte treffen Sie alle noetigen Maßnahmen um dies kuenftig zu vermeiden.

Außerdem bitten wir Sie um die Abgabe einer kurzen Stellungnahme innerhalb von 24h an uns und an die Person, die diese Beschwerde eingereicht hat. Diese Stellungnahme soll Angaben enthalten, wie es zu dem Vorfall kommen konnte, bzw. was Sie dagegen unternehmen werden.

Weiteres Vorgehen:
– Problem beheben
– Stellungnahme an uns abgeben: Verwenden Sie dazu folgenden Link: http://abuse.hetzner.de/statements/?token=xxxxxxSECRETxxxxxxxx
– Stellungnahme per E-Mail an Beschwerenden abgeben

Die Daten werden anschliesend von einem Mitarbeiter überprüft der das weitere Vorgehen koordiniert. Sollten mehrere Beschwerden vorliegen, kann dies auch zu einer Sperrung des Servers fuehren.

Wichtiger Hinweis:
Wenn Sie uns antworten, lassen Sie bitte die Abuse-ID [AbuseID:XXXX] im Betreff unverändert.

Mit freundlichen Grüßen
Dominik P.

Apparently my server was sending wp-login.php attempts to some russian WordPress Blogs like hundreds in minutes, this is what they showed me:

188.40.69.206 intsumrak.ru – [11/Mar/2014:19:28:30 +0400] “POST /wp-login.php HTTP/1.0” 200 3572 “-” “-“
188.40.69.206 intsumrak.ru – [11/Mar/2014:19:28:41 +0400] “POST /wp-login.php HTTP/1.0” 200 3572 “-” “-“
188.40.69.206 intsumrak.ru – [11/Mar/2014:19:28:41 +0400] “POST /wp-login.php HTTP/1.0” 200 3572 “-” “-“
188.40.69.206 intsumrak.ru – [11/Mar/2014:19:28:46 +0400] “POST /wp-login.php HTTP/1.0” 200 3572 “-” “-“
188.40.69.206 intsumrak.ru – [11/Mar/2014:19:28:56 +0400] “POST /wp-login.php HTTP/1.0” 200 3572 “-” “-“

so I replied that I will look into it.

5.54 PM

After  10 Minutes i got a second email also an Abuse Report from valuehost, i replied the same again.
This is when I got scared that it might start getting me those abuse emails nonstop now and I didn’t know where to start with the cleanup or finding the source/problem of the attack from my server, i was destroyed and without answers.

Problem, I was at work and we had our cPanel/WHM Port blocked, so i could not get onto it.
I basically ignored the problem, because i didnt see much to do for the moment.

9.39 PM

While I was with 2 Working collegues at the bar, I checked my emails and saw this email, it was socially uncorrect to run home and do something about it because i had forgotten totally about it, so I stayed there.

11.30pm

We had finally left the place and we went back home, in  my hotel room in Malta I decided to go crazy about checking the problem source, the last email was not only again a abuse report, it also autogenerated a database blocklist for the IP of my server on some 3 sites (which is NOT GOOD AT ALL).

Those sites are/were:
http://www.blocklist.de/en/index.html
http://www.dnsblchile.org/
AND http://barracudacentral.org/rbl

So i had to go and find the problem so i can report a delisting and cleanup/removal of the ip in the networks.

First, let’s take a look at the email from the last abuse report: “Abuse-Message: badbot (First x 1)”

Sehr geehrte(r) Herr Mikula Dedours,

wir haben einen Spam- bzw. Abuse-Hinweis von autogenerated@blocklist.de erhalten.
Bitte treffen Sie alle noetigen Maßnahmen um dies kuenftig zu vermeiden.

Außerdem bitten wir Sie um die Abgabe einer kurzen Stellungnahme innerhalb von 24h an uns und an die Person, die diese Beschwerde eingereicht hat. Diese Stellungnahme soll Angaben enthalten, wie es zu dem Vorfall kommen konnte, bzw. was Sie dagegen unternehmen werden.

Weiteres Vorgehen:
– Problem beheben
– Stellungnahme an uns abgeben: Verwenden Sie dazu folgenden Link: http://abuse.hetzner.de/statements/?token=xxxxxxSECRETxxxxxx
– Stellungnahme per E-Mail an Beschwerenden abgeben

Die Daten werden anschliesend von einem Mitarbeiter überprüft der das weitere Vorgehen koordiniert. Sollten mehrere Beschwerden vorliegen, kann dies auch zu einer Sperrung des Servers fuehren.

Wichtiger Hinweis:
Wenn Sie uns antworten, lassen Sie bitte die Abuse-ID [AbuseID:XXXXXX] im Betreff unverändert.

Mit freundlichen Grüßen
Dominik P.

then, this time the same way my server was used for the attacks…

Mar 11 19:43:37 sjinks wplogin[25297]: 188.40.69.206 littlefox.ru [2014-03-11 19:43:37 0000] “POST wp-login.php HTTP/1.0” “-” “-” “admin”
Mar 11 19:43:48 sjinks wplogin[6830]: 188.40.69.206 littlefox.ru [2014-03-11 19:43:48 0000] “POST wp-login.php HTTP/1.0” “-” “-” “admin”
Mar 11 19:43:53 sjinks wplogin[21104]: 188.40.69.206 littlefox.ru [2014-03-11 19:43:53 0000] “POST wp-login.php HTTP/1.0” “-” “-” “admin”
Mar 11 19:43:54 sjinks wplogin[25297]: 188.40.69.206 littlefox.ru [2014-03-11 19:43:54 0000] “POST wp-login.php HTTP/1.0” “-” “-” “admin”
Mar 11 19:43:54 sjinks wplogin[9666]: 188.40.69.206 littlefox.ru [2014-03-11 19:43:54 0000] “POST wp-login.php HTTP/1.0” “-” “-” “admin”

so what now you might think?
it all escalated in my mind (stress)

00:30 AM

Here the steps I took…

  1. I Googled like crazy and didn’t find much information, only people tell me to like go crazy and yeah i am in deep shit and so on and that i need to download the entire website and then make a code check with the entire code from a backup, wahteverrr!
  2. I tried to look in WHM on several places in the backend where I could get a hang of the way to find out where exactly from my server it originates =(
  3. I looked at the ClamAV and it didnt help much…
  4. I then tried to see the CPU load, it was at about 250% and normally its at around 0.05% haha
  5. Then I found the solution finally….

The solution i found!

I found an article that seemed very helpful http://forums.cpanel.net/f34/troubleshooting-high-server-loads-linux-servers-319352.html and so I started SSH with putty, at this serverload you can imagine it took ages to load a single line in the command, anyways so I wanted to find out how to see the top load and which user caused it, i typed in

top c

that was the secret, now i saw this screen here:

download (1)

in this screenshot i blurred out the users but here it comes, the top user “user” 😉 was using 729% of CPU!!!!

1:00 AM

next thing i did is check the site itself and go to WHM and terminate the account like here:

download

then put it up new with fresh user on cPanel with the same settings and generated a password and uploaded the wordpress blog again fresh, used the content from the google index “site:domain.com” from the “cache:domain.com” to insert into the few pages i had in it, my guess the attack was from a script, sql injection or he found out the password to login, dont ask me! / But i do know that the domain is a big deal and there is alot of competition, not only was/is the domain ranking, but its lucrative.

1:30 AM

So now i could go to those “abuse” sites, the 3 that i listed and get them to delist my site, some i had to email, others just enter the IP and the last one did not let me because i did not own the postmaster@your-server.de of the DNS domain email.

I hope it helped! ALL DONE, if you have questions, drop me a  message in skype!

2:20 AM

I am done with writing this article for you now