Nginx monitoring with OpenTelemetry: logs and metrics

Before you start

Nginx observability has two halves. The logs half is covered by the Linux host observability guide: its filelog receiver tails access.log and error.log and promotes status, method, target, client_ip, and the error level to attributes — every rule below filters on those. The metrics half needs the stub_status module exposed locally and the Collector's nginx receiver pointed at it:

# nginx: add a local-only status endpoint
server {
    listen 127.0.0.1:8080;
    location /status {
        stub_status;
        allow 127.0.0.1;
        deny all;
    }
}

# collector: scrape it
receivers:
  nginx:
    endpoint: http://127.0.0.1:8080/status
    collection_interval: 30s

service:
  pipelines:
    metrics:
      receivers: [nginx]
      processors: [memory_limiter, resource/host, batch]
      exporters: [otlp/telemtra]

Tiers below: alert wire a rule, review look when it fires but don't page, context keep searchable.

Access log: status codes that matter

After the parser, status is a filterable attribute — these are the values and spikes worth rules:

Pattern	Tier	What it means	What to do with it
`status 502`	alert	Bad gateway — nginx couldn't get a valid response from the upstream.	Alert on rate per upstream. Almost always the backend is down or refusing connections; pairs with "connect() failed" in error.log.
`status 504`	alert	Gateway timeout — the upstream took longer than proxy_read_timeout.	Alert on rate. The backend is alive but too slow; the matching error.log line names the exact upstream and request.
`status 503`	alert	Service unavailable — upstream marked failed, maintenance, or limit_conn/limit_req with 503 configured.	Alert on rate. Check "no live upstreams" in error.log to distinguish dead backends from rate limiting.
`status 499`	review	Client closed the connection before nginx answered — an nginx-specific code.	The classic slow-backend symptom: users gave up waiting. A rising 499 ratio is an early warning that precedes the 504s.
`status 429`	review	Rate limit returned to the client (limit_req with default status).	Review which clients hit it: legitimate traffic bursts mean the zone is too tight, one IP means abuse working as intended.
`status 404 spike`	review	Burst of not-found responses.	A scanner probing paths (wp-login.php, .env, .git) or a broken deploy dropping assets. Group by target to tell which.
`status 401 / 403 spike`	review	Authentication or authorization failures.	On admin or API paths, treat a per-IP spike like SSH brute force on the web tier.
`status 413`	context	Request body exceeded client_max_body_size.	Usually a legitimate user hitting an undersized limit on an upload endpoint — worth knowing before they file a ticket.

Error log: messages that matter

The error log is where nginx explains the 5xx codes — the upstream address, the timeout that fired, the limit that tripped. Levels error and above are already promoted by the parser:

Message pattern	Tier	What it means	What to do with it
`no live upstreams`	alert	Every server in the upstream block is marked failed.	Page immediately — the service behind this route is fully down and clients are getting 502s.
`connect() failed … while connecting to upstream`	alert	The upstream refused or dropped the connection (111: refused, 110: timeout).	Alert on rate per upstream. Refused means process down or wrong port; timeout means host or network.
`upstream timed out`	alert	The backend exceeded proxy_connect/read/send_timeout.	The error.log twin of the 504s — it names the upstream address and the request, which the access log doesn't.
`worker_connections are not enough`	alert	nginx hit its connection capacity and is refusing new ones.	Alert on any. Raise worker_connections (and the worker_rlimit_nofile that bounds it) — and check what's holding connections open.
`Too many open files`	alert	The worker hit its file-descriptor limit (24: EMFILE).	Alert on any — every accept and proxied connection needs descriptors; nginx is dropping requests right now.
`limiting requests, excess`	review	limit_req is delaying or rejecting; the message names the zone and client IP.	The server-side view of the 429s — review which zones fire and against whom.
`client intended to send too large body`	context	Server-side record of the 413s.	Carries the actual attempted size — useful when deciding whether to raise the limit.
`SSL_do_handshake() failed`	context	TLS handshake errors.	Mostly scanner noise on public hosts; a spike right after a certificate or cipher change is the case that matters.

stub_status metrics

The nginx receiver emits four metrics, all enabled by default. They carry no per-route or per-status detail — that's what the access log is for — but they're the cheap, reliable capacity signals:

Metric	Kind	Unit	Attributes	Use
`nginx.requests`	Sum (monotonic)	`requests`	—	Total requests — chart as a rate for the traffic baseline every other alert is judged against.
`nginx.connections_accepted`	Sum (monotonic)	`connections`	—	Accepted client connections.
`nginx.connections_handled`	Sum (monotonic)	`connections`	—	Handled connections. Normally equals accepted — a growing gap means connections are being dropped at capacity (see worker_connections).
`nginx.connections_current`	Sum (non-monotonic)	`connections`	state (active, reading, writing, waiting)	Live connection levels. active trending toward your worker_connections setting is the capacity early-warning.

The alert rules to wire first

Five rules in Telemtra alerts — log rules filter on the parsed attributes, metric rules on the receiver's series:

Backend down — any error.log no live upstreams, plus rate of access-log status = 502 per host. Page immediately.
Backend slow — rate of status = 504, with rate of status = 499 as the earlier, softer version of the same problem.
Capacity — any worker_connections are not enough or Too many open files; for the early warning, threshold nginx.connections_current with state = active at ~80% of your worker_connections.
Error budget — ratio of status >= 500 to total requests above ~1% over 5 minutes.
Web brute force — count of status = 401 per client_ip on admin/API paths, same shape as the SSH rule in the Linux log patterns reference.

Noise control

Three drop candidates for Data Shaper before they reach storage: health-check requests (your own /healthz probes are usually the single largest source of access-log lines), scanner 404s against the well-known probe paths, and SSL_do_handshake() failed noise on public hosts. Keep everything else — the access log is the per-request truth that stub_status can't give you.

For the host underneath nginx — CPU, memory, disk, network — see the host metrics reference. New to Telemtra? The quickstart covers the minimum config for every supported stack, and creating a workspace takes about a minute.

Nginx monitoring:what to actually watch.