Threat Hunting Exercise: Analyzing Linux LD_PRELOAD Rootkit Dataset
This hunt uses a simulated Linux auditd dataset (linux_ld_preload_rootkit_2022-05-28T033000.json) capturing T1014: Rootkit and T1574.006: Dynamic Linker Hijacking, where an adversary with root access installs a shared-object rootkit via /etc/ld.so.preload, hooking libc functions to hide malicious processes, files, and network connections from standard tools.
Step 1: Hypothesis Formation
Hypothesis: A shared library is written to disk (commonly under /usr/lib or /lib) and a reference to it is added to /etc/ld.so.preload, after which standard enumeration tools (ps, ls, netstat) on the host fail to show a process/connection that is independently confirmed present via kernel-level telemetry (auditd syscalls, eBPF). Indicators:
- auditd: write to
/etc/ld.so.preloadand creation of a new.sofile in a library directory, both by a root-context process outside package-manager activity (no correspondingdpkg/rpmtransaction). - Discrepancy between
ps auxoutput and/proc/*/statusenumeration, or betweennetstatand raw/proc/net/tcpparsing - a classic rootkit-hiding signature. - The new
.sofile exports symbols matching common libc-hooked functions (readdir,open,stat) per static analysis. - No software installation event (via package manager logs) corresponds to the new library file.
Null Hypothesis: The .so file and ld.so.preload entry are part of a legitimate performance/monitoring tool (e.g., an APM agent using LD_PRELOAD for instrumentation). Invalidate via checking the file against known vendor hashes and confirming a documented change ticket for the software installation.
Rationale: LD_PRELOAD rootkits defeat userspace enumeration entirely, making this one of the few Linux persistence techniques that specifically requires kernel-level or out-of-band telemetry to detect; this hunt validates the auditd-plus-cross-tool-discrepancy approach.
Step 2: Data Sources and Scope
- Sources: auditd (file writes, syscalls); package manager logs (dpkg/rpm history); kernel-level process enumeration (eBPF-based EDR or
/procdirect read) for cross-validation against userspaceps. - Scope: ~2022-05-28T03:30:00-04:15:00 UTC; Host: db02.internal (10.100.6.9); Actor: root (via prior privilege escalation).
- SIEM Queries (Splunk/ELK):
- Preload file write:
index=auditd path="/etc/ld.so.preload" type=PATH - New library without package install:
index=auditd path="*.so" type=PATH | NOT [search index=pkgmgr] - Enumeration discrepancy:
index=edr source="ebpf_proc" | join process_name [search index=edr source="ps_output"] | where isnull(ps_output.process_name)
Step 3: Key Findings
Parsed events (6 shown) confirm a rootkit library installed via ld.so.preload, with a confirmed discrepancy between kernel-level and userspace process visibility.
| Timestamp (UTC) | Host | Actor | Event | IOC/Why Suspicious? |
|---|---|---|---|---|
| 2022-05-28 03:32:15 | db02.internal | root | auditd WRITE: /usr/lib/libselinux-cache.so created | New-library IOC: file created by a root shell process, no corresponding dpkg transaction in package manager logs. |
| 2022-05-28 03:32:20 | db02.internal | root | auditd WRITE: /etc/ld.so.preload set to /usr/lib/libselinux-cache.so | Preload IOC: this file is empty by default on a hardened baseline; any content is a high-confidence rootkit indicator. |
| 2022-05-28 03:35:00 | db02.internal | - | eBPF telemetry shows process nc -lvp 4444 running (PID 8821) | Kernel-level visibility confirms a listening backdoor process. |
| 2022-05-28 03:35:00 | db02.internal | - | ps aux output (same timestamp) shows no PID 8821 | Hiding IOC: userspace enumeration fails to show a process that kernel-level telemetry confirms exists - textbook LD_PRELOAD hooking of readdir//proc parsing. |
| 2022-05-28 03:36:10 | db02.internal | - | Raw /proc/net/tcp parse shows a listener on port 4444 | netstat on the same host at the same time shows no such listener - second independent hiding confirmation. |
| - | db02.internal | (static analysis) | libselinux-cache.so exports hooks for readdir, fopen, stat | Function exports match exactly the libc calls typically hooked by userland rootkits to filter process/file listings. |
Validation:
- Timeline: Library drop → preload activation → immediate hiding of a listening backdoor process from standard tools - a complete rootkit installation-to-use chain within 5 minutes.
- False Positives: No package manager transaction and no matching vendor hash rules out a legitimate APM/monitoring agent.
- Correlation: Single host, single library, direct causal link between the preload write and the subsequent enumeration discrepancy - very high confidence.
Step 4: Recommendations & Next Steps
- Immediate Response: Do not rely on the compromised host's own tools for further investigation - boot from trusted media or use kernel-level/remote forensic collection; remove the
ld.so.preloadentry and the malicious library; treat the host as fully compromised at the root level and rebuild from a known-good image rather than attempting in-place cleanup. - Detection: Sigma-style rule:
title: Non-Empty ld.so.preload File→selection: path="/etc/ld.so.preload" AND file_size > 0- since this file is empty by default on virtually all hardened Linux baselines, any content at all warrants immediate investigation. - Pro Tip: Baseline
/etc/ld.so.preloadand/etc/ld.so.cache-related directories as "should always be empty/unchanged" assets, and monitor them with the same rigor as Windows Run keys - the emptiness of this file is itself the detection signal.
Hypothesis confirmed - an LD_PRELOAD rootkit was installed to hide a listening backdoor process from standard userspace enumeration tools!