The best way to internalize schema differences is to look at the same real-world event in each format side by side. Here’s a single SSH login event — one user, one action, three schemas.
The event: jdoe successfully logs in to web-01.prod.corp.local from 203.0.113.55 via SSH with a public key at 09:23:14 UTC on 2024-03-15.
🔍 Design the schema yourself first
Before reading the worked examples below: design your own JSON schema for this event. What fields would you include? How would you name them? What types would you use?
Write it down. Then read the three representations below and compare your choices to ECS, OCSF, and what raw syslog gives you.
Attempt this before reading on. Getting stuck is the point.
Raw syslog (OpenSSH default)
Mar 15 09:23:14 web-01 sshd[23421]: Accepted publickey for jdoe from 203.0.113.55 port 52341 ssh2: RSA SHA256:abc123def456... What you get: a single line of free-form text with implicit structure. The format is:
<timestamp> <hostname> <program>[<pid>]: <message> Problems:
Timestamp has no year, no timezone — “Mar 15” is ambiguous
Fields are positional, not named — you need regex to extract jdoe, 203.0.113.55, publickey
A format change in OpenSSH breaks your parser
No structured event.outcome — you must pattern-match on “Accepted” vs “Failed”
To extract fields in Splunk SPL:
index = linux sourcetype = syslog "sshd" "Accepted" | rex field = _raw "Accepted (?<auth_method>\w+) for (?<user>\w+) from (?<src_ip>\S+)" | table _time, host , user , src_ip , auth_method If OpenSSH adds a field before “Accepted”, this regex fails silently.
Worked Example
Same event in Elastic ECS { "@timestamp" : "2024-03-15T09:23:14.000Z" , "event" : { "category" : [ "authentication" ], "type" : [ "start" ], "action" : "ssh_login" , "outcome" : "success" , "module" : "system" }, "host" : { "name" : "web-01.prod.corp.local" , "hostname" : "web-01" , "ip" : [ "10.0.1.50" ] }, "user" : { "name" : "jdoe" , "id" : "1001" }, "source" : { "ip" : "203.0.113.55" , "port" : 52341 }, "process" : { "name" : "sshd" , "pid" : 23421 }, "network" : { "transport" : "tcp" , "protocol" : "ssh" , "direction" : "ingress" }, "message" : "Accepted publickey for jdoe from 203.0.113.55 port 52341 ssh2: RSA SHA256:abc123def456..." } What you gain:
@timestamp in RFC 3339 UTC — no ambiguityevent.outcome: "success" — typed field, query directlyuser.name, source.ip — named, typed fields, no regex neededhost.name — explicit, queryable host identifierOriginal message preserved for forensics
ECS KQL query (Kibana):
event.category: "authentication" AND event.outcome: "success" AND network.protocol: "ssh" Same query works for Windows 4624, AWS CloudTrail, Okta — as long as they’re ECS-normalized.
Worked Example
Same event in OCSF { "class_uid" : 3002 , "class_name" : "Authentication" , "activity_id" : 1 , "activity_name" : "Logon" , "time" : 1710491994000 , "status_id" : 1 , "status" : "Success" , "auth_protocol_id" : 99 , "auth_protocol" : "SSH" , "logon_type_id" : 4 , "logon_type" : "Remote Interactive" , "actor" : { "user" : { "name" : "jdoe" , "uid" : "1001" , "type_id" : 1 }, "session" : { "uid" : "23421-ssh" , "is_remote" : true } }, "dst_endpoint" : { "hostname" : "web-01.prod.corp.local" , "ip" : "10.0.1.50" , "port" : 22 }, "src_endpoint" : { "ip" : "203.0.113.55" , "port" : 52341 }, "metadata" : { "product" : { "name" : "OpenSSH" , "version" : "8.9p1" , "vendor_name" : "OpenBSD" }, "version" : "1.1.0" } } What OCSF adds:
class_uid: 3002 — the event class is typed numerically, enabling schema-enforced queriesstatus_id: 1 — integer encoding of success (consistent across all OCSF Authentication events)logon_type_id: 4 — logon type as a typed enum (Remote Interactive)metadata.product — which software generated it, fully structured OCSF query in AWS Athena (Security Lake):
SELECT time , actor . user . name , src_endpoint . ip , dst_endpoint . hostname FROM security_lake . authentication_3002 WHERE status_id = 1 AND auth_protocol = 'SSH' AND time BETWEEN 1710490000000 AND 1710500000000 Note: OCSF uses millisecond epoch timestamps, which is less human-readable but ideal for SQL analytics at scale.
Side-by-Side Field Mapping
Concept Raw Syslog ECS OCSF Timestamp Mar 15 09:23:14@timestamp: "2024-03-15T09:23:14.000Z"time: 1710491994000Username parsed from message user.name: "jdoe"actor.user.name: "jdoe"Source IP parsed from message source.ip: "203.0.113.55"src_endpoint.ip: "203.0.113.55"Destination host web-01host.name: "web-01.prod.corp.local"dst_endpoint.hostname: "web-01.prod.corp.local"Outcome pattern match “Accepted” event.outcome: "success"status_id: 1Auth method parsed from message (not in core ECS) auth_protocol: "SSH"
The field mapping table is your detection engineering deliverable
When a new log source arrives, your first task is a field mapping exercise: identify the 10–15 fields you care about, find them in the source schema, and map them to your SIEM’s target schema (ECS or CIM). Document this as a table. If the source can’t provide a field (e.g., no parent process in a log), document that gap — it’s a detection coverage gap. The mapping table IS the deliverable, not a stepping stone to it.
Building the End-of-Module Challenge
This lesson sets up the module challenge: given 5 log samples in different formats, produce a field mapping table. Each sample is a different source:
Windows Security 4625 (EVTX/XML) — failed logon
Linux auditd (type=USER_LOGIN) — SSH login failure
AWS CloudTrail ConsoleLogin — AWS console authentication
Zeek conn.log — network connection record
Okta System Log — authentication event
For each, you’ll map to ECS fields: user.name, source.ip, host.name, event.outcome, event.action, @timestamp.
This exercise will surface exactly which sources are already ECS-normalized and which require custom field mapping work.
Elastic ECS Reference— Elastic
OCSF Authentication Class— OCSF
Retrieval Practice
A security architect says: 'We use OCSF in our AWS Security Lake but ECS in our on-prem Elastic cluster. We need to write one detection rule for SSH brute-force that covers both.' What's the correct approach, and what's the trade-off?
Reveal answer
The correct approach depends on where you want to write the rule. Option A: Write two rules (one ECS KQL for Elastic, one SQL for Athena) and accept the maintenance cost of keeping them logically equivalent. Option B: Normalize OCSF to ECS before it lands in Elastic (e.g., via a Lambda/Logstash transform), then write one rule in ECS. Trade-off: Option A has zero transform latency and preserves original OCSF fidelity; Option B simplifies rules but adds pipeline complexity and potential transform lag. For a brute-force detection with a time window, Option B's latency risk is real — a 5-minute buffering delay can push the alert past the lockout threshold.
Retrieval Practice
After building your field mapping table, you discover that your Linux auditd logs have no equivalent for ECS's 'process.parent.name' — auditd records the triggering syscall but not the full process tree. What are your options for this coverage gap?
Reveal answer
Three options: (1) Accept the gap — document it in your detection coverage map and note that parent-process-based detections can't apply to auditd sources; (2) Augment the log source — switch from auditd to a higher-fidelity source like eBPF (Falco/Tetragon), which does capture parent process context; (3) Correlate at the SIEM — correlate the auditd event with a preceding process creation record (if you have one from another source) by PID and timestamp to reconstruct the parent. Option 2 is the cleanest but requires infrastructure change; Option 3 is fragile and source-dependent.