Lesson 1 of 5 · 25 min read

Linux Primer — syslog, journald, and auditd

Linux telemetry is essential for detection in mixed environments, cloud workloads, and container infrastructures. The tooling differs significantly from Windows, but the underlying goal is the same: knowing who did what on which system at what time.


syslog: The Legacy Foundation

syslog is the baseline logging system for Unix/Linux — it predates structured logging by decades. Every daemon that wants to log calls syslog(), and the message goes to a facility/severity combination:

Facilities: kern, user, mail, daemon, auth, syslog, lpr, news, uucp, cron, local0–local7 Severities: emerg(0), alert(1), crit(2), err(3), warning(4), notice(5), info(6), debug(7)

Mar 15 09:23:14 web-01 sshd[23421]: Accepted publickey for jdoe from 203.0.113.55 port 52341 ssh2

The format is: <timestamp> <hostname> <process>[<pid>]: <message>. Problems you already know from M02: no timezone, free-form message, format varies by application.


journald: Structured Logging Done Right

systemd-journald replaces syslog on most modern Linux systems. It stores log entries as binary key-value records — fully structured, no parsing required.

# Get SSH events as JSON
journalctl _SYSTEMD_UNIT=ssh.service -o json | jq '.MESSAGE, ._PID, ._UID, ._COMM'

Key journald fields for detection:

FieldContent
_COMMProcess name (comm from /proc)
_EXEFull executable path
_PIDProcess ID
_UIDReal UID of the process
_GIDReal GID
_SYSTEMD_UNITsystemd unit that owns the process
_TRANSPORTsyslog, journal, kernel, stdout, stderr
SYSLOG_IDENTIFIERApp-defined name (like syslog process field)
MESSAGEThe log message
{
  "__REALTIME_TIMESTAMP": "1710491994882341",
  "_HOSTNAME": "web-01",
  "_COMM": "sshd",
  "_EXE": "/usr/sbin/sshd",
  "_PID": "23421",
  "_UID": "0",
  "_SYSTEMD_UNIT": "ssh.service",
  "SYSLOG_IDENTIFIER": "sshd",
  "MESSAGE": "Accepted publickey for jdoe from 203.0.113.55 port 52341 ssh2"
}

auditd: Kernel-Level Syscall Audit

auditd is Linux’s answer to Windows Sysmon — kernel-level syscall interception with rich context. It’s controlled by two components:

Writing audit rules

# Record all process executions (execve syscall) for all users except root
-a always,exit -F arch=b64 -S execve -F auid>=1000 -F auid!=-1 -k exec_commands

# Monitor writes to /etc/passwd and /etc/shadow (credential file modification)
-w /etc/passwd -p wa -k credential_modification
-w /etc/shadow -p wa -k credential_modification

# Monitor sudo usage
-w /usr/bin/sudo -p x -k sudo_execution

# Detect privilege escalation via setuid binaries
-a always,exit -F arch=b64 -S setuid -S setreuid -S setgid -S setregid -k privilege_escalation

Reading an auditd record

type=EXECVE msg=audit(1710491994.882:84231): argc=3 a0="python3" a1="-c" a2="import socket,subprocess;s=socket.socket()"
type=SYSCALL msg=audit(1710491994.882:84231): arch=c000003e syscall=59 success=yes exit=0 a0=7f1234 a1=7f5678 items=2 ppid=1234 pid=5678 auid=1001 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=42 comm="python3" exe="/usr/bin/python3" key="exec_commands"
type=CWD msg=audit(1710491994.882:84231): cwd="/var/www/html"

Reading the record:

Neo23x0 auditd ruleset— GitHub auditd man page— Linux man pages