paulserban.eu

Writing Edition

Paul Serban

Writing, snippets & book notes

Vol. 2026

June 05, 2024

Grafana LGTM for Node.js: Shipping Pino Logs with Grafana Alloy and Loki in Docker Compose

A practical walkthrough for wiring an existing Pino-based Node.js service into Loki and Grafana using Alloy as the collection agent, entirely in Docker Compose

Introduction

If you already run a Node.js service that logs structured JSON with Pino, you are closer to a working observability stack than you might think. Pino was built around the idea that logging should be fast and machine-readable by default, which happens to be exactly what a log aggregation system like Grafana Loki wants to ingest. The gap most teams face isn't the logging library - it's the plumbing between "container writes JSON to stdout" and "engineer runs a query in Grafana." This article closes that gap using Grafana Alloy, the current generation collector in the Grafana observability ecosystem, running alongside Loki and Grafana in a single Docker Compose file.

By the end of this walkthrough you will have a four-service Compose stack - your existing Node.js app, Alloy, Loki, and Grafana - where every log line your app writes through Pino ends up parsed, labeled, and queryable in Grafana within seconds. The focus here is deliberately narrow: logs only, single-node, local-first. That scope is intentional. Most teams evaluating the Grafana LGTM stack don't need traces and metrics on day one; they need to stop grepping through docker logs output and start asking structured questions about their application. Once the log pipeline works, adding metrics and traces later is an incremental extension of the same Alloy configuration, not a rewrite.

What the Grafana LGTM Stack Actually Is

LGTM is Grafana Labs' shorthand for four components that share a common design philosophy: Loki for logs, Grafana for visualization and querying, Tempo for distributed traces, and Mimir for long-term, horizontally scalable metrics storage. The name is a marketing convenience more than an architectural mandate - you are not required to run all four to get value from any one of them. What ties them together is a consistent approach to indexing and querying: each system indexes on a small set of labels rather than the full payload, and each exposes its own query language (LogQL for Loki, TraceQL for Tempo, PromQL-compatible querying for Mimir) that Grafana understands natively. The result is that a single Grafana instance can correlate a spike in error logs with a matching latency increase in traces and a resource saturation signal in metrics, without switching tools.

This walkthrough uses exactly one letter and a half of that acronym: Loki for storage and query, and Grafana for the "G." Tempo and Mimir are absent, and that's fine - a logs-only deployment is a completely valid and common starting point for teams migrating off an ELK stack or off ad hoc docker logs inspection. If you later want to add request tracing, the same Alloy instance you're deploying here can run an OTLP receiver alongside the logging pipeline and forward spans to Tempo with no changes to the logging path.

The reason Loki specifically is worth adopting instead of a general-purpose text search engine comes down to indexing cost. Elasticsearch and similar systems index the full content of every log line, which is powerful for ad hoc full-text search but expensive at scale in both storage and compute. Loki borrows Prometheus's model: index a small set of labels (service name, log level, environment) and leave the log line itself as compressed, unindexed text in chunks. Queries filter first by label, then grep the matching chunks. For most operational log volumes - request logs, application errors, structured events - this trade produces dramatically lower storage costs and simpler operations, at the cost of full-text search performance across unindexed fields. That trade-off shapes almost every design decision later in this article, particularly around what you're allowed to turn into a label.

Why Grafana Alloy Instead of Promtail or Grafana Agent

If you've read older Loki tutorials, you've likely seen Promtail used as the log-shipping agent. That guidance is now outdated. Promtail was Loki's original, purpose-built shipping agent, and Grafana Agent later arrived as a broader collector for metrics, logs, and traces. Grafana Labs has since consolidated both into Grafana Alloy: Alloy replaced Promtail in the stack because Promtail is considered feature-complete, with future log-collection development happening in Alloy instead. The older Grafana Agent followed the same path - Grafana Agent reached end-of-life on November 1, 2025, and no longer receives support or security fixes, with Grafana Labs directing existing users to migrate to Alloy. Loki's own release notes confirm the same direction from the other side: Promtail is deprecated, with its code merged into Grafana Alloy. Practically, this means Alloy is not one option among several current agents - it is the only actively developed path Grafana Labs recommends for new deployments, whether you're shipping logs, metrics, or traces.

The more interesting shift, beyond deprecation notices, is architectural. Promtail and Grafana Agent's static mode used declarative YAML scrape configs - you described targets and pipelines as data. Alloy instead uses a component graph: each processing step (service discovery, log tailing, JSON parsing, relabeling, remote write) is a typed component with explicit inputs and outputs, wired together in a configuration language that resembles HCL. This has real practical benefits for a walkthrough like this one. Components can be composed and reused across logs, metrics, and traces in the same file, configuration changes apply without a restart, and Alloy ships a built-in web UI (on port 12345 by default) showing the live state of every component in the pipeline - which becomes the first place you look when a log stops showing up in Grafana.

Architecture Overview

The stack has four moving parts, and it's worth being precise about what each one owns. The Node.js application only does one job in this picture: write structured JSON log lines to stdout through Pino, the same way it already does in production today. Docker's logging driver captures that stdout stream as it always has. Grafana Alloy is the only component that changes: it mounts the Docker socket, discovers running containers, and tails each container's log stream directly through the Docker Engine API - the same mechanism docker logs -f uses internally. Loki receives parsed, labeled log entries pushed over HTTP and stores them, indexing only the labels Alloy attached. Grafana connects to Loki as a data source and lets you run LogQL queries and build dashboards against it.

This tail-via-socket approach is one of at least three reasonable ways to get Docker container logs into Loki, and it's worth naming the alternatives before committing to it. You could instead have Alloy (or Promtail) read the JSON log files Docker's json-file driver writes to disk, mounting /var/lib/docker/containers into the collector container - this avoids socket access but couples you to the json-file driver's on-disk format and rotation behavior. Alternatively, Loki ships a Docker logging driver plugin that lets Docker push logs directly to Loki without any collector in between, which removes Alloy from the picture entirely for pure logging but sacrifices the single-agent model that pays off once you add metrics or traces. Tailing through the Docker socket via discovery.docker and loki.source.docker, as this walkthrough does, sits in the middle: no dependency on log file formats, full compatibility with the rest of Alloy's pipeline, at the cost of granting the collector access to the Docker socket - a trade-off discussed in more depth later.

Nothing about this architecture is specific to Compose. The same Alloy configuration, or a near-identical one using Kubernetes service discovery instead of discovery.docker, runs as a DaemonSet in a cluster and tails container logs on every node. Compose is simply the fastest way to validate the pipeline end to end before deciding whether it's worth operating in production, and everything you configure here - the parsing stages, the label choices, the Loki schema - carries forward largely unchanged.

Implementation Walkthrough

Project Layout

A minimal version of this stack needs five files beyond your existing application code: a docker-compose.yml tying everything together, a loki-config.yaml for Loki's storage and schema settings, a config.alloy describing the collection pipeline, a Grafana provisioning file so the Loki data source exists automatically on first boot, and your application's own package.json/server code, which we assume already exists and already uses Pino. The layout below assumes your Node.js app lives in ./app with its own Dockerfile; adjust the build context if your existing project is structured differently.

.
├── docker-compose.yml
├── loki-config.yaml
├── config.alloy
├── grafana/
│   └── provisioning/
│       └── datasources/
│           └── loki.yaml
└── app/
    ├── Dockerfile
    ├── package.json
    └── src/
        └── server.js

The Node.js Service with Pino

The only change required in the application itself is making sure Pino's output is consistent and low-cardinality where it matters. If your service already logs JSON to stdout, you likely need to touch very little - but two settings are worth being deliberate about before they become load-bearing for your Grafana dashboards. First, Pino's default level output is a numeric severity code (10, 20, 30…), which is efficient but unreadable as a Loki label; overriding the level formatter to emit the string label makes downstream filtering and dashboarding far more intuitive. Second, attaching a small set of static fields - a service name and environment - through Pino's base option gives you exactly the low-cardinality identifiers you'll want as Loki labels, without tempting you to promote something like a request ID to a label later.

// src/server.js
import express from "express";
import pino from "pino";
import pinoHttp from "pino-http";

const logger = pino({
  level: process.env.LOG_LEVEL || "info",
  base: {
    service: "orders-api",
    env: process.env.NODE_ENV || "development",
  },
  formatters: {
    level(label) {
      return { level: label };
    },
  },
  redact: ["req.headers.authorization", "req.headers.cookie"],
});

const app = express();

app.use(
  pinoHttp({
    logger,
    customLogLevel: (req, res, err) => {
      if (res.statusCode >= 500 || err) return "error";
      if (res.statusCode >= 400) return "warn";
      return "info";
    },
  })
);

app.get("/orders/:id", (req, res) => {
  req.log.info({ orderId: req.params.id }, "Fetching order");
  res.json({ id: req.params.id, status: "processing" });
});

app.listen(3000, () => {
  logger.info("orders-api listening on port 3000");
});

Notice that orderId is logged as a structured field, not baked into the message string, and it is never promoted to a Loki label - it stays queryable through LogQL's JSON parser without inflating Loki's index. The redact option strips sensitive headers before they ever reach stdout, which matters once every log line is being centrally collected rather than sitting in a terminal only you can see.

Configuring Loki

Loki's configuration for a local walkthrough can be minimal: a single binary, filesystem storage, and the TSDB index format that's the current recommended default for new deployments. Authentication is disabled here for simplicity - multi-tenancy and auth are things to revisit before this leaves your laptop.

# loki-config.yaml
auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2024-01-01
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  reject_old_samples: true
  reject_old_samples_max_age: 168h

Configuring Grafana Alloy

This is the file that does the actual work, and it's worth reading component by component rather than copy-pasting blindly. discovery.docker connects to the Docker socket and enumerates running containers as discovery targets. discovery.relabel cleans up the container name label - Docker prefixes container names with a leading slash - so it reads naturally in Grafana. loki.source.docker takes those targets and tails each container's log stream, forwarding raw log lines downstream. loki.process is where Pino's JSON gets parsed: it extracts the fields we care about, promotes exactly two of them (level and service) to indexed labels, and sets the log's timestamp from Pino's own time field rather than the time Alloy happened to read the line. loki.write pushes the finished entries to Loki's HTTP push endpoint.

// config.alloy

discovery.docker "containers" {
  host = "unix:///var/run/docker.sock"
}

discovery.relabel "containers" {
  targets = discovery.docker.containers.targets

  rule {
    source_labels = ["__meta_docker_container_name"]
    regex         = "/(.*)"
    target_label  = "container"
  }
}

loki.source.docker "default" {
  host       = "unix:///var/run/docker.sock"
  targets    = discovery.relabel.containers.output
  forward_to = [loki.process.pino.receiver]
}

loki.process "pino" {
  forward_to = [loki.write.default.receiver]

  stage.json {
    expressions = {
      level     = "level",
      message   = "msg",
      service   = "service",
      timestamp = "time",
    }
  }

  stage.labels {
    values = {
      level   = "level",
      service = "service",
    }
  }

  stage.timestamp {
    source = "timestamp"
    format = "UnixMs"
  }
}

loki.write "default" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
  }
}

Two design choices in this file deserve emphasis because they're easy to get wrong later. First, only level and service become labels - everything else Pino logs remains part of the unindexed log body, queryable through LogQL's | json pipeline stage but never part of Loki's index. Second, stage.timestamp sources its value from Pino's own time field rather than the ingestion time; this matters when Alloy or the network introduces delay, since queries and alerts should reflect when the event actually happened in your application, not when the collector got around to processing it.

Provisioning Grafana

Rather than clicking through Grafana's UI to add Loki as a data source every time you tear down and rebuild the stack, Grafana's provisioning system lets you declare it as a file that gets picked up automatically on startup.

# grafana/provisioning/datasources/loki.yaml
apiVersion: 1

datasources:
  - name: Loki
    type: loki
    access: proxy
    url: http://loki:3100
    isDefault: true
    jsonData:
      maxLines: 1000

The Docker Compose File

With all four configuration files in place, the Compose file mostly wires ports, volumes, and startup order together. The one line that deserves attention is the Docker socket mount on the alloy service - mounted read-only here, though as the pitfalls section explains, read-only access to the socket still grants meaningful privilege.

# docker-compose.yml
services:
  app:
    build: ./app
    container_name: orders-api
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - LOG_LEVEL=info
    depends_on:
      - loki

  loki:
    image: grafana/loki:3.7.4
    container_name: loki
    command: -config.file=/etc/loki/loki-config.yaml
    ports:
      - "3100:3100"
    volumes:
      - ./loki-config.yaml:/etc/loki/loki-config.yaml
      - loki-data:/loki

  alloy:
    image: grafana/alloy:v1.19.2
    container_name: alloy
    command:
      - run
      - /etc/alloy/config.alloy
      - --server.http.listen-addr=0.0.0.0:12345
    ports:
      - "12345:12345"
    volumes:
      - ./config.alloy:/etc/alloy/config.alloy
      - /var/run/docker.sock:/var/run/docker.sock:ro
    depends_on:
      - loki

  grafana:
    image: grafana/grafana:12.4.11
    container_name: grafana
    ports:
      - "3001:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
    volumes:
      - ./grafana/provisioning:/etc/grafana/provisioning
      - grafana-data:/var/lib/grafana
    depends_on:
      - loki

volumes:
  loki-data:
  grafana-data:

Pin the image tags shown here to whatever the current stable releases are when you build this - Alloy, Loki, and Grafana all ship frequent patch releases, and floating on latest in a tutorial you'll revisit in six months is how you end up debugging an unrelated breaking change instead of your actual pipeline.

Bringing It Up and Querying

Run docker compose up -d, hit your application's endpoints a few times to generate log traffic, then open Grafana at http://localhost:3001 and navigate to Explore with the Loki data source selected. A query for {service="orders-api"} should immediately return every log line the app has written since startup, with level and service available as filterable label dropdowns. From there, LogQL lets you layer in the JSON fields that weren't promoted to labels: {service="orders-api"} | json | orderId="12345" filters on the parsed field without it ever having been part of the index, and sum(rate({service="orders-api", level="error"}[5m])) gives you an error-rate time series suitable for a dashboard panel or an alert rule, computed entirely from log volume rather than a separate metrics pipeline.

Trade-offs and Common Pitfalls

Mounting the Docker socket into the Alloy container is the single most consequential decision in this setup, and it's worth understanding exactly what it grants. Read access to /var/run/docker.sock is functionally equivalent to root access on the host, because the Docker API can be used to start privileged containers with arbitrary host mounts. Marking the mount :ro in Compose restricts what the file looks like from inside the container, but it does not restrict what the Docker Engine API allows once a process can talk to it - the read-only flag is frequently misunderstood this way. For a local walkthrough this is an acceptable trade for convenience; for anything internet-facing or multi-tenant, look at a socket proxy that exposes only the specific API endpoints Alloy needs (container list and logs), or run Alloy in an environment where you can grant it read-only log access through the container runtime's own permission model instead of the full socket.

The second pitfall shows up more slowly and is harder to diagnose after the fact: label cardinality. Loki's entire performance model depends on labels staying low-cardinality - a handful of distinct values per label, ideally in the tens or low hundreds, not the thousands. It's tempting, especially early on, to promote something like a request ID, user ID, or order ID to a label because it feels like it should be "indexed" for fast lookup. Doing so creates a new time series per unique value, and Loki's storage and query performance degrades sharply as the number of streams grows. The configuration in this article deliberately keeps only level and service as labels for exactly this reason; everything else stays inside the log line, parsed on query via LogQL's | json stage rather than at ingestion. This is slower per individual query than an indexed label lookup, but it scales linearly with the volume of matching logs rather than exploding the index, and for the query patterns most teams actually run - "show me errors for this service in the last hour, then filter by order ID" - it's the right trade.

Finally, this walkthrough's Loki configuration - single binary, filesystem storage, in-memory ring - is intentionally not representative of a production deployment. It has no replication, no object storage backend, and no separation between the write and read paths, all of which matter once log volume grows past what a single process comfortably handles. Loki supports a "simple scalable" deployment mode that splits read and write components while still being simpler to operate than full microservices mode, and it's the natural next step once this Compose setup has proven the pipeline works and you're ready to move it toward a real environment with S3-compatible object storage behind it.

The 80/20 Insight

Strip away the Compose YAML and the component graph syntax, and this entire setup earns most of its value from exactly two decisions made before a single log line reaches Loki. The first is disciplined structured logging at the source: consistent field names, a stable set of base fields like service and env, and level output as readable strings rather than numeric codes. The second is deliberate label design in the collection pipeline - choosing the two or three fields that genuinely have low cardinality and belong in Loki's index, and leaving everything else as parseable but unindexed content. Dashboards, alerts, and ad hoc debugging all become straightforward once those two things are right; almost every painful Loki deployment story traces back to getting one of them wrong.

This is also where a small amount of forward-looking discipline pays off disproportionately later. If you expect to add distributed tracing at some point - and Alloy is explicitly built to make that an incremental addition rather than a new agent - attaching a trace_id field to your Pino logger now, even before Tempo exists in your stack, costs nothing today and turns "correlate this error log with its trace" from a research project into a single LogQL-to-TraceQL link once Tempo is in place. The field sits unused in the log line until that day; when it matters, it's already there.

Best Practices for Production Readiness

Treat log volume as a cost center from the start, not something to optimize after a bill arrives. Set sensible log levels per environment - debug locally, info or warn in production for most services - and avoid logging full request or response bodies by default; Pino's redact option and selective field logging, as shown in the server example, keep sensitive data and payload bloat out of your log stream without requiring discipline at every call site. If a particular endpoint is genuinely noisy and valuable at debug level, consider sampling rather than either logging everything or nothing.

Operationally, don't treat Alloy as a black box once it's past the walkthrough stage. Alloy exposes its own internal metrics and a live component graph through its HTTP UI, which is the fastest way to confirm whether a missing log is a Pino problem, a Docker discovery problem, or a Loki ingestion problem - check that surface before assuming the pipeline is broken end to end. Set resource limits on the Alloy and Loki containers in Compose or your orchestrator of choice; a collector with no memory ceiling that suddenly receives a burst of log volume from a misbehaving service is a common way to turn a logging problem into a host-stability problem. On the Loki side, configure retention explicitly through limits_config and, once you're on object storage, lifecycle rules on the bucket - logs that nobody queries after 30 days shouldn't cost the same as logs from this morning.

Security deserves a pass before this setup leaves a developer's laptop. The Compose file above enables Grafana anonymous access and a static admin password for convenience during local development; neither belongs anywhere reachable outside a trusted network. Move credentials into environment files or a secrets manager, put a reverse proxy with TLS in front of Grafana for any shared or internet-facing deployment, and reconsider anonymous access entirely - Viewer-only anonymous access is a reasonable default for an internal, network-isolated dashboard, but it's not a substitute for real authentication once more than one team relies on the stack.

Key Takeaways

If you only take five things from this walkthrough, take these:

Conclusion

The distance between "we already log JSON with Pino" and "we can query and alert on our logs in Grafana" turns out to be shorter than most teams expect, and almost none of that distance is about learning a new logging library - it's about wiring an agent correctly and making a handful of deliberate choices about what gets indexed. Grafana Alloy's component model, while more upfront complexity than a flat Promtail YAML file, pays for itself the moment you want to add a second signal type or debug why a log stream stopped flowing, because the pipeline is inspectable rather than opaque.

What this Compose setup gives you is a real, working reference point rather than a diagram. From here, the path forward is incremental: swap filesystem storage for an object store and move to Loki's simple-scalable mode when volume demands it, add an OTLP receiver to the same Alloy configuration when tracing becomes worth the investment, and tighten the security posture - socket access, authentication, TLS - before any of this leaves a development machine. None of those steps require rethinking the architecture you just built; they extend it.

References