Before you start
This page assumes the logs are already flowing — the Linux host observability guide sets up the filelog receiver for syslog and auth.log with a regex parser that promotes process_name, host_name, and the message into structured fields. Every rule below is a Telemtra log alert filtering on those fields plus a body substring.
Each table marks a pattern with one of three tiers: alert wire a rule, review look when it fires but don't page, context keep searchable for investigations. (Windows fleet too? The same tiers and structure live in the Windows Event IDs reference.)
Authentication logs /var/log/auth.log · /var/log/secure
auth.log (Debian/Ubuntu) or secure (RHEL family) answers "who got in, who tried, and who used privilege". Every pattern below arrives with a process_name the regex parser in our Linux guide promotes, so rules can filter on sshd vs sudo precisely.
| Message pattern | Process | Tier | What it means | What to do with it |
|---|---|---|---|---|
Failed password for | sshd | alert | SSH password failure — the brute-force signal. | Alert on count per host above ~10 in 5 minutes. The message carries the username and source IP: one IP hammering is a brute force, many IPs is a spray. |
maximum authentication attempts exceeded | sshd | alert | A single connection burned all its retries. | Alert on any — normal users don't do this; tools do. |
Failed password for invalid user | sshd | review | Logon attempt for a nonexistent account. | Constant internet background noise on public SSH. Review for your real usernames appearing; otherwise a Data Shaper drop candidate. |
Accepted password for / Accepted publickey for | sshd | context | Successful login. | The investigation baseline. Worth a dedicated rule only for root logins or known-sensitive accounts. |
user NOT in sudoers | sudo | alert | Privilege escalation attempt by an unauthorized account. | Alert on any. The message names the user, the host, and the command they tried. |
authentication failure | sudo, su | review | Wrong password on sudo or su. | Once is a typo; a burst on a service account is credential testing. |
COMMAND= | sudo | context | Every command run via sudo, with who and from where. | Keep searchable — this is the privileged-activity audit trail when something changed and nobody admits it. |
new user: / delete user / password changed | useradd, userdel, passwd | review | Account lifecycle on the host. | Same review as Windows 4720/4726: expected from provisioning tooling, suspicious by hand on a production box. |
session opened for user root | su, sshd | review | Someone became root. | On hosts where direct root access is banned by policy, make this an alert instead. |
Kernel messages kern.log · dmesg
The kernel announces dying hardware and resource exhaustion here long before applications notice. This is the availability channel — the Linux counterpart of the Windows System log.
| Message pattern | Process | Tier | What it means | What to do with it |
|---|---|---|---|---|
Out of memory: Killed process | kernel | alert | The OOM killer fired and names its victim. | Alert on any. This is the answer to "why did my service vanish without a log line" — the killed process, its PID, and memory stats are all in the message. |
I/O error, dev / Buffer I/O error | kernel | alert | A block device is returning errors. | Alert on any recurrence for the same device — the Linux twin of Windows disk events 7/51/153. Plan disk replacement, don't wait for SMART. |
ata… failed command / exception Emask | kernel | review | SATA-level command failures. | The early warning that precedes the I/O errors above; repeated entries for one port mean disk or cable. |
EXT4-fs error / XFS … Corruption | kernel | alert | Filesystem-level corruption detected. | Alert on any. Schedule an fsck/xfs_repair window before the kernel forces the issue. |
Remounting filesystem read-only | kernel | alert | The kernel gave up on a filesystem and froze writes. | Page immediately — every service writing to that mount is now failing. |
segfault at … in | kernel | review | A binary crashed; the message names the executable and library. | Group per executable, like Windows Application 1000. One segfault is a fluke; a series in the same library is a bug or bad RAM. |
blocked for more than 120 seconds | kernel | review | A task is stuck in uninterruptible sleep. | Almost always storage: a dying disk, an overloaded array, or a hung NFS mount. |
nf_conntrack: table full, dropping packet | kernel | alert | Connection-tracking table exhausted; new connections silently dropped. | Alert on any — pairs with the conntrack metrics in the host metrics reference for the early-warning version. |
TCP: Possible SYN flooding | kernel | review | SYN backlog overflowed on a listening port. | Either a real flood or an undersized listen backlog on a busy service — both worth knowing. |
link down / link up (NIC) | kernel | context | Network interface state changed. | A flapping link explains weird latency; correlate timestamps with system.network.errors rates. |
Service manager and cron syslog · journald
systemd and cron write to syslog/journal, which the filelog receiver in our Linux guide already tails. Unit lifecycle messages are the Linux equivalent of the Service Control Manager events on Windows.
| Message pattern | Process | Tier | What it means | What to do with it |
|---|---|---|---|---|
Failed to start / entered failed state | systemd | alert | A unit failed. | Alert grouped by unit name — the twin of Windows 7034. The preceding journal lines from the unit itself say why. |
Start request repeated too quickly | systemd | alert | Crash loop confirmed; systemd stopped retrying. | Alert on any. The service is down and will stay down until a human runs systemctl reset-failed. |
Watchdog timeout | systemd | review | A service stopped heartbeating but didn't exit. | Hung process — usually deadlock or saturated I/O. systemd kills and restarts it; recurrences are the signal. |
(CRON) ERROR / cron job failures | cron | review | A scheduled job failed or couldn't run. | Cron fails silently by design — this review queue is often the only place broken backups announce themselves. |
System clock wrong / time sync loss | chronyd, ntpd | review | The host lost time synchronization. | Clock drift quietly breaks TLS handshakes, Kerberos, and cross-host trace timestamps. Fix before debugging anything else weird. |
Properties to filter on
process_name— promoted by the regex parser:sshd,sudo,kernel,systemd,CRON. The first filter on every rule above, and what keeps a "Failed password" rule from matching an application log that happens to contain the words.host_name— the grouping key for per-host thresholds, shared with metrics and traces.- Username and source IP — sshd embeds both in the message body (
Failed password for root from [SERVER_IP]). Extract them with an extraregex_parseroperator if you want to group brute-force alerts by attacker IP rather than just count per host. severity— available when the source file preserves it and the Collector maps it. Follow the rsyslog severity setup; default text files may omit the level. ERROR and FATAL are useful additional filters, but do not replace message-pattern rules.
For compliance-grade auditing beyond syslog — every syscall, file access, and identity change — the next layer is auditd (/var/log/audit/audit.log), which the same filelog receiver can tail.
The alert rules to wire first
Five rules in Telemtra alerts, all on signal type logs:
- SSH brute force — body contains
Failed password,process_name = sshd, count perhost.nameabove ~10 in 5 minutes. - OOM kill — any
Out of memory: Killed process. Include the matched log line in the notification; it names the victim. - Disk failing — any
I/O errororBuffer I/O errorfromprocess_name = kernel. - Filesystem read-only — any
Remounting filesystem read-only. Page immediately. - Service crash loop — any
Start request repeated too quickly, or count ofentered failed stateper host above 3 in 10 minutes.
Noise control
Two patterns dominate auth.log volume: pam_unix(cron:session): session opened/closed firing every minute for every cron job, and the endless Invalid user scans against public SSH. Drop the cron session pairs at ingest with Data Shaper; keep the scans only if you report on them. For the metrics half of the same hosts — CPU, memory, disk, filesystem — see the host metrics reference.
New to Telemtra? The quickstart covers the minimum config for every supported stack, and creating a workspace takes about a minute.