Agent Log Debugging

Overview

Query agent diagnostic logs directly via Postgres in Docker. No API auth needed — just docker exec + psql.

Quick Reference

Direct SQL (fastest)

docker exec breeze-postgres-dev psql -U breeze -d breeze -c "SQL HERE"

Common queries:

-- Recent errors for a device
SELECT timestamp, level, component, message, fields
FROM agent_logs
WHERE device_id = 'DEVICE_UUID'
  AND level IN ('error', 'warn')
ORDER BY timestamp DESC LIMIT 50;

-- All logs for a device in last hour
SELECT timestamp, level, component, message
FROM agent_logs
WHERE device_id = 'DEVICE_UUID'
  AND timestamp > now() - interval '1 hour'
ORDER BY timestamp DESC;

-- Search by message text
SELECT timestamp, level, component, message, fields
FROM agent_logs
WHERE message ILIKE '%connection%'
ORDER BY timestamp DESC LIMIT 30;

-- Logs by component (updater, heartbeat, websocket, main, etc.)
SELECT timestamp, level, message, fields
FROM agent_logs
WHERE device_id = 'DEVICE_UUID'
  AND component = 'websocket'
ORDER BY timestamp DESC LIMIT 50;

-- Find device_id from agent_id
SELECT id, hostname, agent_id, status
FROM devices
WHERE agent_id = 'AGENT_ID_STRING';

-- Log volume by level (health check)
SELECT level, count(*) FROM agent_logs
WHERE timestamp > now() - interval '24 hours'
GROUP BY level ORDER BY count DESC;

Enable Debug Log Shipping

Default shipping level is warn. To get all logs for debugging, send a command to the agent:

{
  "type": "set_log_level",
  "payload": { "level": "debug", "durationMinutes": 60 }
}

Auto-reverts to warn after the duration. Send via device commands endpoint or WebSocket.

Note: Log shipping is NOT enabled by default. The agent_logs table will be empty until the agent's log shipper is configured and the API endpoint is reachable. If logs are empty, ask the user for logs from the agent's local log file instead.

Dashboard API Endpoint

GET /api/v1/devices/:deviceId/diagnostic-logs (JWT or API key auth)

Query params: ?level=warn,error, ?component=updater, ?since=ISO, ?until=ISO, ?search=keyword, ?page=1&limit=100

Agent Log File Locations

Configurable via log_file in agent.yaml. Rotated at 50MB, 3 backups.

Log Shipping Pipeline

Agent slog handler → Shipper.Enqueue() → buffer (1000 entries) → batch every 60s → gzip POST to POST /api/v1/agents/:agentId/logs → agent_logs table

Key Tables

• agent_logs — shipped diagnostic logs (level, component, message, fields, agent_version)
• devices — lookup device_id from agent_id
• device_commands — command history (check if set_log_level was sent)

Common Debug Scenarios

Dev Push (Build & Deploy)

Zero-arg dev-push (recommended)

Defaults are read from /.env.dev (gitignored). Just run:

cd agent
make dev-push

.env.dev format

BREEZE_API_KEY=brz_...
BREEZE_DEV_DEVICE=<device-uuid>
BREEZE_API_URL=http://<your-dev-host>:3001

Override any default

make dev-push DEVICE=<other-device> AUTH_TOKEN=<other-key> API_URL=<other-url>

Auth: JWT or API key

The Makefile auto-detects brz_ prefix and sends via X-API-Key header. JWTs are sent via Authorization: Bearer. The dev-push API endpoint (POST /api/v1/dev/push) accepts both.

Current dev environment

Dev Iteration Loop: Fetch Logs → Fix → Deploy → Check

When debugging agent issues, use this cycle with dev push to iterate fast:

1. FETCH LOGS — identify the problem
   docker exec breeze-postgres-dev psql -U breeze -d breeze -c "
     SELECT timestamp, level, component, message
     FROM agent_logs
     WHERE device_id = '<device-uuid>'
       AND level IN ('error','warn')
     ORDER BY timestamp DESC LIMIT 30;"

   If agent_logs is empty, ask user for local log file contents.

2. FIX CODE — edit Go source in agent/internal/...

3. BUILD & DEPLOY — push new binary (agent restarts in ~5s)
   cd agent
   make dev-push

4. CHECK LOGS — verify the fix landed
   docker exec breeze-postgres-dev psql -U breeze -d breeze -c "
     SELECT timestamp, level, component, message, fields
     FROM agent_logs
     WHERE device_id = '<device-uuid>'
       AND timestamp > now() - interval '2 minutes'
     ORDER BY timestamp DESC;"

   → Confirm new agent_version shows 'dev-<timestamp>'
   → Confirm error/warn messages are gone
   → If not fixed, loop back to step 1

Tips:

• Before starting, bump log level: send set_log_level command with {level: "debug", durationMinutes: 30} so you get full detail
• After dev-push, auto_update is disabled on the agent — re-enable in agent.yaml when done
• The dev push binary includes the version string dev-<unix-timestamp> — filter by agent_version in logs to confirm which build is running
• Dev push only works in non-production (NODE_ENV !== 'production' or DEV_PUSH_ENABLED=true)