Search for a command to run...
A Deep Dive into Rootkit Detection via Linux Memory Forensics - HackTheBox Business CTF 2025
Currently working as a Senior Consultant at Netcompany spending my full-time job solving the SharePoint riddles. In the free time I'm expanding my understanding of cybersecurity through hacking activities. Git fanboy.
No comments yet. Be the first to comment.
💡 This write-up is a part of the HTB Sherlocks series. Sherlocks are investigative challenges that test defensive security skills. I encourage you to try them out if you like digital forensics, incident response, post-breach analysis and malware ana...
Hack The Box Sherlock write-up
💡 This write-up is a part of the HTB Sherlocks series. Sherlocks are investigative challenges that test defensive security skills. I encourage you to try them out if you like digital forensics, incident response, post-breach analysis and malware ana...
Hack The Box CTF Cyber Apocalypse 2025: Tales from Eldoria
A critical Linux server began behaving erratically under suspected Volnaya interference, so a full memory snapshot was captured for analysis. Stealthy components embedded in the dump are altering system behavior and hiding their tracks; your task is to sift through the snapshot to uncover these concealed elements and extract any hidden payloads.
Type: Forensics
Difficulty: Hard
Authors: c4n0pus
Event: Global Cyber Skills Benchmark CTF 2025: Operation Blackout (ctftime)
Official write-up:
TBA (but probably here)
volatility Linux symbols generation
ftrace hooks
rootkit concealment and persistence
Given: mem.elf
$ mem.elf (2.170.445.516 bytes)
ELF 64-bit LSB core file, x86-64, version 1 (SYSV)
62760d9d1ae0cc321c0ffa6a41fbfea30e280e51c38643eee0aee371d64ea12d mem.elf
As usual with any kind of memory dumps, the tool of choice is Volatility Framework.
volatility is via the Python virtual environment, so before I run any commands, I’m activating it . /opt/volatility3/venv/bin/activate. That way I can simply call volatility instead of python3 /opt/volatility/vol.pyGetting to know with memory dump of what image I’ll be working on:
$ vol -f ../mem.elf banners.Banners
Linux version 6.1.0-34-amd64 (debian-kernel@lists.debian.org)
(gcc-12 (Debian 12.2.0-14+deb12u1) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40)
#1 SMP PREEMPT_DYNAMIC Debian 6.1.135-1 (2025-04-25)
This is quite a recent kernel, and we haven’t been given a symbols file - it could mean that we may have to prepare a symbols file ourselves. Only one way to verify the hypothesis:
No symbols, no output. Let’s obtain them then.
ISF allows Volatility to
Understand kernel data structures like tasks, modules, file descriptors, etc.
Match the memory layout from the dump to the correct kernel version and configuration.
Work more efficiently, since the raw debug symbols don’t need to be parsed every time.
First, we have to get hands on the Linux distribution that the memory was dumped from.
I’m downloading Debian from the official site, installing it in the virtual machine. Now we have to match the same kernel.
$ sudo apt install linux-image-6.1.0-34-amd64
$ sudo shutdown -r now
After reboot, we can verify that we have the correct version of the system.
With that, we can proceed with dwarf2json tool (run on the same system).
$ cd /opt
$ git clone https://github.com/volatilityfoundation/dwarf2json.git
$ sudo apt install golang
$ go build -buildvcs=false
$ cd ~
$ sudo /opt/dwarf2json/dwarf2json linux --elf /usr/lib/debug/boot/vmlinux-6.1.0-34-amd64 \
--system-map /boot/System.map-6.1.0-34-amd64 > Debian12-6.1.0-34-amd64.json
Copy the JSON file to the system with volatility under volatility3/volatility3/symbols.
Challenge after challenge, I keep running the same plugins, so I’ve created a useful script that helps me automate these first steps.
import subprocess, os, sys
def main():
if len(sys.argv) != 2:
print(f"Usage: {sys.argv[0]} <memory_image>")
sys.exit(1)
memory_image = sys.argv[1]
if not os.path.exists(memory_image):
print(f"Error: Memory image '{memory_image}' not found.")
sys.exit(1)
# Configuration
venv_activate = "/opt/volatility3/venv/bin/activate"
output_dir = "./vol"
os.makedirs(output_dir, exist_ok=True)
plugins = [
"linux.bash.Bash",
"linux.elfs.Elfs",
"linux.envars.Envars",
"linux.ip.Link",
"linux.ip.Addr",
"linux.lsof.Lsof",
"linux.malfind.Malfind", # never found anything useful yet, but sounds useful
"linux.pagecache.Files",
"linux.psaux.PsAux",
"linux.pslist.PsList",
"linux.psscan.PsScan", # potential hidden processes that are not listed in pslist
"linux.pstree.PsTree",
"linux.sockstat.Sockstat",
"linux.lsmod.Lsmod"
]
# Construct the bash command to run all plugins
joined_plugins = " ".join(plugins)
bash_command = f'''
source "{venv_activate}" && \\
for plugin in {joined_plugins}; do
echo "[*] Running $plugin"
vol -f "{memory_image}" $plugin > "{output_dir}/$(basename $plugin).out"
done
deactivate
'''
# Execute the bash command
subprocess.run(bash_command, shell=True, executable='/bin/bash')
if __name__ == "__main__":
main()
At this point, I’m browsing and grepping through the output files to find some suspicious entries. But to keep this article tidy, let’s just go through the questions (in no particular order).
/dev directory. These rules define how the system should handle hardware events, such as assigning device names, setting permissions, or triggering scripts when devices are added or removed. udev rules are typically stored in /etc/udev/rules.d/ and are written using specific match and action syntax.In the output of linux.pagecache.Files we can see references to many /usr/lib/udev/rules.d/ and among them there is one that is definitely not a standard: 99-volnaya.rules. Its existence may potentially mean that the rootkit tries to integrate into the system’s normal device-handling workflow, making its actions blend in with legitimate operations.
Answer: 99-volnaya.rules
When browsing through the linux.sockstat.Sockstat output, a couple of connections stand out:
bash raised TCP connections should always give a warning.
Answer: 16.171.55.6
Because we already know that rootkit uses the volnaya string we can just search for its occurrences in all volatility outputs.
Answer: volnaya_xb127
There are two indicators of the suspicious bash process present in the volatility outputs.
In the aforementioned linux.sockstat.Sockstat potential revshells connections originate from one specific bash process.
In the linux.psscan output, we can identify multiple bash processes, but only one is a direct child of systemd and spawns an id command.
Answer: 2957
These two will be answered by outputs of two more volatility outputs.
volatility -f mem.elf linux.tracing.ftrace.CheckFtrace > vol/linux.tracing.ftrace.CheckFtrace.out
volatility -f mem.elf linux.check_syscall.Check_syscall > vol/linux.check_syscall.Check_syscall.out
ftrace is a kernel-level tracing framework used for debugging and performance profiling. But attackers (or rootkits) can abuse ftrace to hook syscall handlers without modifying the syscall table — making detection harder. So, for example rootkit can intercept a kill signal and perform own code in place of it (can still execute the original syscall making its action very difficult to detect for user). Or can hook to directory listing event and hide its own files before they are printed via ls. Ops, spoilers.
In the output of linux.check_syscall.Check_syscall:
Addresses of kill and getdents64: 0xffffb88b6bf0:0xffffb8b7c770 (answer).
And now the output of the second plugin:
These are all hooks created by the rootkit. Only two of them are syscalls, though.
Answer: kill:getdents64
Yes, one of the flags was included as an format hint to other question. I wonder how many players got that flag just by typing it because it was shown as example? New CTF solving routing discovered?
For the rest of the questions, we have to extract the binaries for both volnaya_usr (rootkit loader) and volnaya_xb127 (rootkit kernel module). As long as the binary is cached in the memory, we should be able to dump it with volatility. From the linux.pagecache.Files:
SuperblockAddr MountPoint Device InodeNum InodeAddr FileType InodePages CachedPages FileMode AccessTime ModificationTime ChangeTime FilePath InodeSize
0x9a097453c800 / 8:1 913940 0x9a0974b68128 REG 107 107 -rwxr-xr-x 2025-05-12 19:20:29.156000 UTC 2025-05-12 19:17:34.102003 UTC 2025-05-12 19:20:12.731680 UTC /usr/bin/volnaya_usr 437032
We are lucky and volnaya_usr is cached on 107 pages with inode address 0x9a0974b68128. Unfortunately volnaya_xb127 is not (0 pages, and 0 inode size) - but we will take care of it soon. For now, dump the volnaya_usr.
vol -f mem.elf linux.pagecache.InodePages --inode 0x9a0974b68128 --dump
Open it in Ghidra.
In the main function, there is a reference to xor_key. When navigated to the .data section select the label of xor_key so the whole memory segment is selected and choose “Copy special..” → “Byte String (No Spaces)".
Answer: 881ba50d42a430791ca2d9ce0630f5c9
In the same line of main we can see how the hostname is created. We can navigate to the .data section to copy the content of the hostname and decode the hostname rootkit connects to.
hostname = bytes.fromhex(
"eb7ac96120c5531232c1b7ad3408c4f8a66dca612cc5491832caadac"
)
xor_key = bytes.fromhex(
"881ba50d42a430791ca2d9ce0630f5c9"
)
decoded = bytearray()
for i in range(len(hostname)):
decoded.append(hostname[i] ^ xor_key[i % len(xor_key)])
print(decoded.decode('utf-8', errors='replace'))
Answer: callback.cnc2811.volnaya.htb
init_revshell there is callkill(0x41,local_c). It is usually used to send a signal to the process. But the thing is the maximum signal number is 64 and here 65 is sent.. Which doesn’t make sense until you recall the rootkit hooks to the kill syscall. This is one of the methods of communication between rootkit modules.Now the last two questions require looking into the previously unavailable volnaya_xb127. By analysing the code of the loader, especially install_module() function, we can see the following:
syscall is used to call sys_init_module (see linux.check_syscall.Check_syscall, index 0xAF=175) on some binary data located under local_18, of size 0×666c0, passing some argument. We can see that local_18 is a pointer to a deobf() function.
That’s definitely a rootkit kernel module binary. Here is the Python version (copy the elf_body and elf_hdr content from the .data sections.
elf_hdr = bytes.fromhex("7f4..00") # 64-byte ELF header
elf_body = bytes.fromhex("8c..c9") # ELF body
xor_key = bytes.fromhex("881ba50d42a430791ca2d9ce0630f5c9")
# Decrypt body
decrypted_body = bytearray()
for i in range(len(elf_body)):
decrypted_body.append(elf_body[i] ^ xor_key[i % 16])
# Reconstruct full ELF
with open("volnaya_xb127", "wb") as f:
f.write(elf_hdr)
f.write(decrypted_body)
Open volnaya_xb127 in Ghidra.
Here we have a nice listing of all hooks rootkit is setting and functions used to hide its activity.
In each of filldir hooks (that rootkit uses to manipulate directory listing) we can find a filename check. If it contains MAGIC_WORD the file is hidden.
Answer: volnaya
Now our focus is moved to the hook_sys_getdents64() function.
By analysing the vfs_fstatat documentation, we can determine that lVar10 contains a structure file metadata structure. Then in UID and GID of the file are stored respectively in iVar8 and iVar2 (through pointer offsetting). Later, those values are compared to the USER_HIDE and GROUP_HIDE which means those last two constants are UID and GID we are looking for.
By converting hexadecimals to decimals, we have the answer.
Answer: 1821:1992