Checklists – System is Compromised or Hacked – Part 1

Introduction

As in my previous Blog where i explained how i came to know if my system is hacked or compromized (link here). Here in this blog i will explain what basic things we can check on our system when we have doubt if our system is compromized .

This Blogs have 3 parts

• List of Checks which can determine if system is compromised or hacked – Part 1
• List of checks which can give a direction how system is compromised or hacked – Part 2
• What preventive steps (specially infra related) can be taken care to avoid hacking or to make system more secure – Part 3

Here , i am assuming system is Linux system with Centos installed .

List of Checks which can determine if system is compromised or hacked

• Generally when hacker break into a linux system it is high chance that it will alter you main packages like openssh,kernel etc.. , So first if of please check if these packages are altered or there are some changes in the files or binaries provided by these packages . Following are commands to check on Centos
  • sudo rpm -qa | grep openssh | xargs -I '{}' sudo rpm -V '{}'
  • If therr are files shown by above command in which you did not change anything then it means there is high chance your system is compromised
• Run rootkit Hunter to check if you system is compromised
  • Download rkhunter.tar.gz
  • copy it in /root and goto /root
  • tar zxvf rkhunter-1.4.2.tar.gz
  • cd rkhunter-1.4.2/
  • sh installer.sh --layout default --install
  • changes in /etc/rkhunter.conf ENABLE_TESTS="all" DISABLE_TESTS="none" HASH_CMD=SHA1 HASH_FLD_IDX=4 PKGMGR=RPM 7
  • /usr/local/bin/rkhunter --propupd
  • /usr/local/bin/rkhunter --update
  • /usr/local/bin/rkhunter -c -sk 10.
  • note output or check and copy /var/log/rkhunter.log
  • you can also check the link for using rkhunter
• Check /var/log/secure to check if there are many authentication failure requests and someone trying brute force to enter in to system
  • following will be the comand :
    • [root@localhost ~]# less /var/log/secure | grep 'authentication failures'
  • and output will be something like :
    • Apr 25 12:48:46 localhost sshd[2391]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.29.14 user=root
    • Apr 25 12:49:33 localhost sshd[2575]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.29.14 user=root
  • In above output you can see the rhost from where login attempt are made . If you see lots of entries like this then also check if at some point of time some login attempt will be successful from any of attempting rhosts . In secure logs accepted logs will looks something like as follows :
    • Apr 25 12:53:10 localhost sshd[3551]: Accepted password for root from 192.168.29.14 port 36362 ssh2

• Check in Processes if some unusual process is running and consuming high CPU using top and ps commands .
  • Command to list all process running in system : ps aux | less
  • Also check using top command if some unusual process trying to utiize high cpu
• Check if there is some unusual entry in crontab of all users made on system
  • crontab -u <user> -l . by default user is root
• Check if in id_rsa.pub , if some attacker has somehow made its entry in .ssh folder in every users’s home directory .

This was the Part 1 of the Blog , In later Parts i will explain some further checklist to ensure that you system will remain less hackable .

Thankyou .

Linux Machine Compromized/Broken -Power of Observation

Introduction :

In Debugging any issue or any dealing any problem or circumstance two things are important

• Observation — Observation not only at the time of issue but in general times also .
• Combining your general observations and Observations at the time of issue to conclude something .

In this Blog , I will explain the following :

• What was happening on my machine
• How i came to know my machine is broken into — Power of observation

What was happening on my machine

• Load on my machine is going very high
• On top command one process ./kwsapd0 is consuming around 3000% cpu

From here we get to know that kswapd is consuming process , The process kswapd0 is the process that manages virtual memory . So I thought that may be our some process is consuming more RAM and Virtual Memory is being used due to which kswapd process is doing its work but after hours of debugging we found no process is consuming RAM and around 80% RAM was free .

How i came to know my machine is broken into — Power of observation

There were two general observation which i observed and helped my geeting know what was the issue

• 1st is kswapd process looks in top command like [kswapd] not ./kswapd
• Kswapd0 can only consume 100% as it uses only one core in the machine .

From there I got to know that this kswapd0 is something unusual . On further debugging

I found ./.configrc/a/kswapd0 in root users directory .

Contents of this directory was :

$ find .configrc -type f
.configrc/dir2.dir
.configrc/a/kswapd0
.configrc/a/dir.dir
.configrc/a/a
.configrc/a/bash.pid
.configrc/a/run
.configrc/a/stop
.configrc/a/init0
.configrc/a/.procs
.configrc/a/upd
.configrc/cron.d
.configrc/b/sync
.configrc/b/dir.dir
.configrc/b/a
.configrc/b/run
.configrc/b/stop

There was also an entry in cron to run this .

So , From all of this i got to know that my system was compromised .

Yet I was unable to find out how my system was broken into . But in my future Blog i will explain what things one can check if your system is compromised and how it is compromised and what all security we can apply to our system to make it less hackable .