Threat Hunting Exercise: Analyzing Linux SSH Port-Knocking Covert C2 Channel Dataset

This hunt uses a simulated Linux firewall/network dataset (linux_port_knock_2022-08-22T220000.json) capturing T1205.001: Traffic Signaling - Port Knocking, where an attacker configures a sequence-triggered firewall rule (via knockd or a custom iptables/nftables watcher) so that SSH access to a backdoor is only opened after a specific sequence of connection attempts to closed ports - keeping the backdoor invisible to routine port scans.

Step 1: Hypothesis Formation

Hypothesis: A host receives a rapid, sequential series of connection attempts to several closed/filtered ports from a single external source, immediately followed by that same source successfully establishing an SSH connection on a port that a prior scan showed as closed - indicating a port-knock sequence has opened access rather than a normally-listening service. Indicators:

• Firewall/IDS logs show 3-5 sequential SYN packets to different closed ports from one external IP within a few seconds.
• Immediately after the sequence, the same source IP successfully connects to the SSH port (or a non-standard port) that was previously logged as closed/filtered.
• A knockd process or custom iptables-watching script is running on the host, found via process/service enumeration, with no corresponding change-management record.
• The SSH session following the knock sequence authenticates successfully on the first attempt (suggesting key-based or pre-shared credential access set up during initial compromise), rather than a brute-force pattern.

Null Hypothesis: The sequential connection attempts are incidental background internet scanning noise (mass-scanning services routinely probe many ports), coincidentally followed by an unrelated legitimate SSH login. Invalidate by confirming the exact port sequence repeats consistently across multiple occurrences and matches a configured knockd rule on the host, rather than random scanner port ordering.

Rationale: Port knocking is specifically designed to keep a backdoor's listening port invisible to standard port scans and firewall audits, since the port only opens transiently after the correct sequence; this hunt validates sequential-connection-pattern detection as a control that doesn't depend on the backdoor port being open (and thus scannable) at audit time.

Step 2: Data Sources and Scope

• Sources: Host-based firewall logs (iptables/nftables LOG target) and network IDS; process/service enumeration (osquery) for knockd or custom watcher scripts; SSH auth logs (/var/log/auth.log) for the resulting session.
• Scope: ~2022-08-22T22:00:00-22:01:00 UTC, recurring nightly; Host: lnx-edge-node02 (10.270.1.8, public-facing); Source: 91.219.237.66.
• SIEM Queries (Splunk/ELK):
• index=firewall dest_ip="10.270.1.8" action=DROP | transaction src_ip maxspan=10s | where count>=3 (sequential closed-port hits from one source)
• index=firewall dest_ip="10.270.1.8" dest_port=22 action=ALLOW | search src_ip=<same_source_as_knock_sequence>
• Process check: index=osquery table="processes" name="knockd" OR cmdline="port-knock"

Step 3: Key Findings

Parsed events (5 shown) confirm a repeating nightly port-knock sequence from a single external IP that reliably preceded successful SSH access to a port otherwise shown as closed in routine scans.

Validation:

• Timeline: identical three-port knock sequence immediately followed by successful root SSH login, repeating nightly for 7 consecutive nights at the same source IP - a deliberate, scheduled backdoor access pattern.
• False Positives: the exact same port sequence and timing recurs across multiple nights, which is inconsistent with random internet background-noise scanning; knockd is actively configured on the host with a matching rule.
• Correlation: firewall log sequence, knockd configuration, and successful first-attempt root login together confirm a functioning port-knock-triggered backdoor, not incidental scan-then-login coincidence.

Step 4: Recommendations & Next Steps

• Immediate Response: Remove knockd and its configuration from lnx-edge-node02; rotate the SSH host key and all authorized keys; block 91.219.237.66 at the perimeter; treat the host as compromised at the root level and investigate for additional persistence given root access was available nightly.
• Detection: Sigma-style rule: title: Sequential Closed-Port Connections Followed by Successful SSH → selection: (firewall_action=DROP, 3+ distinct dest_ports, same src_ip, within 10s) followed by (dest_port=22 action=ALLOW, same src_ip, within 15s).
• Pro Tip: Treat any discovery of knockd or a custom port-sequence watcher on a production host as an immediate compromise indicator regardless of the sequence's specifics - legitimate infrastructure should use VPN or bastion-host access controls, not port knocking, which exists specifically to hide backdoor access from routine scanning and firewall audits.

Hypothesis confirmed - an attacker configured a port-knocking backdoor that opened root SSH access for 10 seconds following a specific three-port sequence, evading detection by routine port scans for at least 7 consecutive nights!