Aegis documentation

#collector — counters, snapshots and ingestion

The collector turns what the host is doing into numbers: interfaces and peers, nginx requests, mail deliveries. It is not a cube and has no application to build. It owns three independent submodules, each with its own raw snapshot, its own database and its own daemon:

SubmoduleReadsSnapshotDatabase
net/proc/net/dev, ssnet.jsonnet.db
nginxaccess_json.log, stub statusnginx.jsonnginx.db
mailPostfix mail.log, hook linesmail.jsonmail.db

Collection and ingestion are deliberately separate. A daemon writes a JSON snapshot on its own timer and never waits on SQLite; a keeper job folds that snapshot into the database afterwards. The raw file stays on disk, so what the collector saw can always be inspected without querying anything.

bash
aegis collector --helpaegis collector status

The group has no banner of its own: a bare aegis collector is an argparse error (exit 2), and --help is the discovery path. The commands are setup, status, collect, ingest, and the same three verbs under net, nginx and mail.

#How one pass works

  • collect reads the source and writes <raw_dir>/<name>.json atomically.
  • The snapshot replaces the previous one.

  • ingest reads that snapshot, inserts the rows, advances the state table and
  • records last_ingest.

  • A daemon repeats step 1 every interval seconds; a keeper job repeats step 2
  • on the ingestion interval.

    A failure in either half is reported and does not stop the other submodules.

    #Platform-wide commands

    #aegis collector setup

    Create the database of every enabled submodule and apply its migrations. It is idempotent: an up-to-date database is reported as such, and files from the two older layouts (a flat <name>.db, then storage/collector/<name>.db) are adopted on first use rather than recreated.

    bash
    aegis collector setup
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector setupaegis collector setup && aegis collector status
    text
    ✔ net: /aegis/database/storage/net/net.db (up to date); nginx: /aegis/database/storage/nginx/nginx.db (up to date); mail: /aegis/database/storage/mail/mail.db (up to date)

    An exception while creating a database prints the message with a ✖ marker and exits 1.

    #aegis collector status

    Print the whole collector on one screen: settings, daemon state, raw snapshot age and size, each database with its last ingest, and the data sections the submodules provide. Values come from the status files and from the SQLite databases, so the report cannot disagree with the data it describes.

    bash
    aegis collector status --limit <n>
    OptionDefaultMeaning
    --limit10rows in the "top peers", "ports", domains and hooks sections
    bash
    aegis collector statusaegis collector status --limit 5
    text
    Collector  raw dir        /aegis/data/collector  enabled        net, nginx, mail  interval       net 30s (ingest 30s)  interval       nginx 30s (ingest 30s)  interval       mail 30s (ingest 30s)  aegis-collector-net  running (pid 8123)    cycles       1842 (errors 0)    started      2026-11-16T21:00:03+00:00    last cycle   2026-11-17T06:42:01+00:00    interval     30s  aegis-collector-nginx  running (pid 8124)    cycles       1840 (errors 0)    started      2026-11-16T21:00:03+00:00    last cycle   2026-11-17T06:42:01+00:00    interval     30s  aegis-collector-mail  running (pid 8125)    cycles       1839 (errors 2)    started      2026-11-16T21:00:03+00:00    last cycle   2026-11-17T06:42:01+00:00    interval     30s  Raw files    net      184.2 kB   12s old  /aegis/data/collector/net.json    nginx     42.6 kB   12s old  /aegis/data/collector/nginx.json    mail       9.1 kB   12s old  /aegis/data/collector/mail.json  net.db  1.2 MB  (9 tables)    /aegis/database/storage/net/net.db    last ingest  2026-11-17T06:42:00+00:00    last collect 2026-11-17T06:41:58+00:00  nginx.db  412.0 kB  (7 tables)    /aegis/database/storage/nginx/nginx.db    last ingest  2026-11-17T06:42:00+00:00    last collect 2026-11-17T06:41:58+00:00  mail.db  84.3 kB  (6 tables)    /aegis/database/storage/mail/mail.db    last ingest  2026-11-17T06:42:00+00:00    last collect 2026-11-17T06:41:58+00:00  Interfaces    name       state    mtu         rx         tx        rx/s       tx/s    eth0       up      1500    12.4 GB    3.1 GB    1.2 MB/s  84.0 kB/s    lo         unknown 65536   214.0 MB   214.0 MB  12.0 kB/s  12.0 kB/s  Top peers (by rx rate, 2)    ip                conns         rx         tx        rx/s       tx/s  ports    203.0.113.24         18   284.1 MB   12.2 MB   84.0 kB/s  9.1 kB/s  443, 80    198.51.100.7          4    18.4 MB    2.1 MB   12.0 kB/s  1.2 kB/s  443  Ports (2)    proto   port  listen  conns         rx         tx    tcp      443  yes        18   284.1 MB   12.2 MB    tcp       80  yes         4    18.4 MB    2.1 MB  nginx (bucket 2026-11-17T06:41:00+00:00)    requests     184    bytes        12.4 MB    status       171 2xx, 6 3xx, 7 4xx, 0 5xx    connections  active max 12 (reading 1, writing 2, waiting 9)  mail (bucket 2026-11-17T06:41:00+00:00)    deliveries   42 sent, 1 deferred, 0 bounced, 0 rejected    example.com      38 sent      0 deferred      0 bounced    hook queue         out         42 runs  0 failed  avg 12 ms  max 88 ms

    A daemon that never started shows a line instead of counters:

    text
      aegis-collector-mail  not started (no status file)

    A leftover status file from a crashed process is reported honestly:

    text
      aegis-collector-net  dead (stale status file)

    status always exits 0. A missing database, a missing snapshot or a section with no rows simply shortens the report; nginx and mail sections appear only when their submodule is enabled and their database has a bucket.

    #aegis collector collect

    Run one collection pass for every submodule, in the order net, nginx, mail. Each result is printed on its own line; a failing submodule prints an error and the rest still run. The exit code is 1 when any submodule failed.

    bash
    aegis collector collect
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector collectaegis collector collect && aegis collector ingest
    text
    ✔ net: 4 interfaces, 23 peers, 12 ports✔ nginx: 184 requests, 65126400 bytes, 184 stored✔ mail: 42 deliveries, 57 events, 3 hook runs

    A submodule that raises prints its name and the error on stderr, marked ✖, and makes the command exit 1; the remaining submodules still run.

    collect writes every submodule it is asked for, enabled or not. The enabled flag decides what the keeper supervises, not what an explicit command may run.

    #aegis collector ingest

    Fold every snapshot into its database, in the same order and with the same per-submodule error handling. An ingest with no snapshot yet is not an error for the report, but the submodule returns its own message.

    bash
    aegis collector ingest
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector ingestaegis collector ingest && aegis collector status
    text
    ✔ net: 4 interfaces, 23 peers, 12 ports✔ nginx: 184 requests, 184 stored✔ mail: 42 deliveries, 57 events

    A submodule that cannot write its database prints its name and the error, marked ✖, and the exit code is 1.

    #The net collector

    net reads /proc/net/dev for cumulative interface counters and the socket table for peers and service ports. It uses ss -tin because that is the only source of per-socket bytes_sent / bytes_received without a kernel module; when ss is absent it falls back to /proc/net/tcp* and /proc/net/udp*, which give counts but no bytes.

    Every pass replaces the snapshot atomically, including one row per socket so the ingest can account traffic that disappears when a socket closes.

    #aegis collector net daemon

    Run the net collector in the foreground, one pass every collector.net.interval seconds. The keeper runs exactly this command as the service aegis-collector-net; start it by hand only when debugging.

    bash
    aegis collector net daemon
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector net daemonaegis collector net daemon &
    text
    2026-11-17T06:40:00+00:00 aegis-collector-net started, every 30s2026-11-17T06:40:00+00:00 4 interfaces, 23 peers, 12 ports in 0.09s2026-11-17T06:40:30+00:00 4 interfaces, 23 peers, 12 ports in 0.08s

    The loop is drift-free, never stacks missed cycles and handles signals:

    SignalEffect
    SIGTERM, SIGINTfinish the current cycle, remove the pid file, exit 0
    SIGHUPre-read the configuration and apply a new interval

    A second copy refuses to start, logs already running (pid 8123) and exits 1. The pid file is run/collector/net.pid; the status file and the log are described under Files and paths.

    #aegis collector net collect

    Run one net pass and print what it found. This is the same code the daemon calls, so it is the quickest way to prove the host data is readable.

    bash
    aegis collector net collect
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector net collectaegis collector net collect && aegis collector net ingest
    text
    ✔ net: 4 interfaces, 23 peers, 12 ports

    #aegis collector net ingest

    Read net.json and write its rows into net.db: interfaces with their rates, peers with connection counts and traffic, and service ports. The pass also stores the last collect and ingest stamps in the database state table.

    bash
    aegis collector net ingest
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector net ingestaegis collector net ingest && aegis collector status --limit 5
    text
    ✔ net: 4 interfaces, 23 peers, 12 ports

    #The nginx collector

    nginx reads access_json.log from the byte offset it stored last time, so a pass counts only the requests that happened since the previous one. Rotation is detected by inode, size and the first line of the file, which also survives copytruncate. The nginx stub status page supplies the live connection counts that a log cannot show.

    Each parsed request is also copied to a per-cycle spool file, then renamed to .jsonl; the ingest reads whole files only, so a request is never stored twice or cut in half.

    #aegis collector nginx daemon

    Run the nginx collector in the foreground, every collector.nginx.interval seconds. The keeper supervises it as aegis-collector-nginx.

    bash
    aegis collector nginx daemon
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector nginx daemonaegis collector nginx daemon &
    text
    2026-11-17T06:40:00+00:00 aegis-collector-nginx started, every 30s2026-11-17T06:40:00+00:00 184 requests, 65126400 bytes, 184 stored in 0.04s

    The signal handling, the pid file and the refusal of a second copy are the same as the net daemon.

    #aegis collector nginx collect

    Run one nginx pass. With no access log yet (nginx installed but not serving, or access_json.log not created) the counters are zero and the pass still writes a snapshot.

    bash
    aegis collector nginx collect
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector nginx collectaegis collector nginx collect && aegis collector nginx ingest
    text
    ✔ nginx: 184 requests, 65126400 bytes, 184 stored
    text
    ✔ nginx: 0 requests, 0 bytes, 0 stored

    #aegis collector nginx ingest

    Write the snapshot's counters and the spooled requests into nginx.db.

    bash
    aegis collector nginx ingest
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector nginx ingestaegis collector nginx ingest && aegis collector status
    text
    ✔ nginx: 184 requests, 184 stored

    #The mail collector

    mail reads Postfix's mail.log from the last offset. Delivery lines carry status=sent|deferred|bounced|rejected, the lines around them carry the sender, size and SMTP client, and the platform's own hook runner writes aegis-hook name=… direction=… status=… ms=… lines. Per-message events go to a spool file the ingest reads, exactly like nginx requests.

    #aegis collector mail daemon

    Run the mail collector in the foreground. The keeper supervises it as aegis-collector-mail.

    bash
    aegis collector mail daemon
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector mail daemonaegis collector mail daemon &
    text
    2026-11-17T06:40:00+00:00 aegis-collector-mail started, every 30s2026-11-17T06:40:00+00:00 42 deliveries, 57 events, 3 hook runs in 0.02s

    #aegis collector mail collect

    Read new lines from mail.log and write the snapshot plus the event spool.

    bash
    aegis collector mail collect
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector mail collectaegis collector mail collect && aegis collector mail ingest
    text
    ✔ mail: 42 deliveries, 57 events, 3 hook runs

    The Postfix log path comes from postfix.log_file and defaults to /aegis/logs/postfix/mail.log.

    #aegis collector mail ingest

    Write the delivery counters, per-domain totals, hook runs and message events into mail.db.

    bash
    aegis collector mail ingest
    Argument / optionDefaultMeaning
    ——none; the command takes no arguments or options
    bash
    aegis collector mail ingestaegis collector mail ingest && aegis collector status
    text
    ✔ mail: 42 deliveries, 57 events

    #Files and paths

    text
    /aegis/storage/logs/collector/<name>.json          snapshot/aegis/storage/logs/collector/<name>.status.json   pid, cycles, last error/aegis/database/storage/<name>/<name>.db           SQLite database/aegis/database/migrations/core/<name>/*.sql       migrations/aegis/run/collector/<name>.pid                    pid file/aegis/logs/collector/<name>.log                   one line per cycle

    The paths follow aegis.root; on this platform it is /aegis. The snapshot directory is collector.raw_dir when set (absolute or relative to the root), and the default above otherwise. A database left by an older layout is moved into place on first use, so an upgraded host needs no manual step.

    #Configuration

    KeyDefaultMeaning
    collector.raw_dirstorage/logs/collectorwhere snapshots are written
    collector.interval60fallback collection interval, seconds
    collector.ingest_intervalcollection intervalfallback ingestion interval
    collector.keep_days0 (keep everything)fallback retention for append-only rows
    collector.net.enabledtrueenable the net submodule
    collector.nginx.enabledtrueenable the nginx submodule
    collector.mail.enabledtrueenable the mail submodule
    collector.<name>.intervalcollector.intervalcollection interval for one submodule
    collector.<name>.ingest_intervalits collection intervalingestion interval for one submodule
    collector.net.geoip_ttl_hours48how long one cached GeoIP answer is reused; 0 looks it up again

    collector.<name>.keep_days prunes the append-only tables of that submodule on every ingest (request rows, mail events, rollups, cached GeoIP answers); 0 disables pruning for that submodule. The global collector.keep_days is the fallback, and collector.<name>.enabled decides which submodules the keeper supervises. status prints the nginx and mail data sections only when their enabled key is explicitly true, so set all three in config/aegis.toml when you want the report to match what runs.

    Set collector.<name>.ingest_interval explicitly when the ingestion rate matters: when neither it nor a collection interval is configured, the keeper registers the ingest job with a 10-second fallback.

    The keeper registers one continuous daemon per enabled submodule and one collector-<name>-ingest job per enabled submodule:

    bash
    aegis keeper servicesaegis keeper cron

    #Checking it is alive

    bash
    aegis collector status           # daemon state, snapshot age, last ingestaegis collector net collect      # can the source be read at all?aegis check                      # collector rows in the platform health report
    SymptomLikely cause
    not started (no status file)the keeper never started the daemon, or it is disabled
    dead (stale status file)the process died; the status file is from the previous run
    a snapshot age that keeps growingthe daemon is hung or the iterator stopped
    no collector database yetaegis collector setup has not run
    nginx counters stay at zeroaccess_json.log is missing, or its format is not JSON

    There is nothing destructive in this group: no command deletes a database or a snapshot. collect replaces the current snapshot, which is its purpose.