QueueKeeperCheck.Config (queue_keeper_check v0.2.1)

Copy Markdown View Source

Thresholds and safety limits. All check durations are in seconds. Pass overrides as keyword lists:

QueueKeeperCheck.run(MyApp.Repo, backlog: [max_depth: 5_000], hanging: [default_budget: 120])

Check defaults:

checkoptiondefaultmeaning
:orphanedafter3600executing longer than this, far beyond any expected runtime (matches Oban Lifeline's default rescue_after of 60 min). Measures execution age only: the job may be orphaned by a node crash, or may still be running
:hangingdefault_budget300executing longer than the worker's budget
:hangingbudgets%{}per-worker budgets, e.g. %{"MyApp.VideoWorker" => 1800}
:backlogmax_depth1000due jobs (available, or scheduled/retryable whose time has passed) per queue
:backlogmax_latency300age of the oldest due job per queue
:stalledwindow60due jobs older than backlog.max_latency and no job attempted in this queue within the window
:stalledlookback3600completed jobs count as recent activity only if they were scheduled at most this many seconds before the queue's oldest due job (bounds the query; see QueueKeeperCheck.Checks.stalled/4)
:retry_stormwindow300look-back window
:retry_stormmin_retries20retrying/retried jobs per worker (see below)
:retry_storminclude_completedfalsealso count completed jobs on attempt >= 2 attempted in the window. Costs a full scan of oban_jobs on large tables (no index on attempted_at), so it is off by default
:discardedwindow / max3600 / 5jobs discarded in the window, per worker
:cancelledwindow / max3600 / 10jobs cancelled in the window, per worker
:cron_missingexpected%{} (off)worker => max seconds between insertions, e.g. %{"MyApp.Nightly" => 90_000}

:retry_storm counts, per worker, within the window:

  • retryable jobs whose last (failed) attempt started in the window (attempted_at; the query also bounds scheduled_at, which Oban sets after the attempt, so it can use the composite index),
  • executing jobs on attempt >= 2 attempted in the window (attempted_at),
  • discarded / cancelled jobs on attempt >= 2 discarded / cancelled in the window (discarded_at / cancelled_at),
  • with include_completed: true, completed jobs on attempt >= 2 attempted in the window (attempted_at).

A job that failed before the window and is still waiting out a long backoff is not counted. (Up to v0.2.0 every retryable job counted, whatever the window.)

A finding is :critical when the value is at least twice its threshold; :orphaned, :stalled and :cron_missing findings are always :critical.

Safety options (top level, not per check):

optiondefaultmeaning
:statement_timeout5000ms. Per statement server-side limit (Postgres statement_timeout, MySQL max_execution_time); also the basis of the client-side timeout on every engine
:lock_timeout1000ms. Postgres lock_timeout: give up rather than queue behind a lock (e.g. a migration)
:check_timeout30000ms. Client-side budget for one whole check (Postgres transaction / MySQL checkout holding one connection); the connection is dropped if exceeded
:read_onlytruePostgres: run each check in a transaction marked transaction_read_only, so the server rejects any write
:prefix"public" (Postgres), none elsewhereschema of oban_jobs (a database name on MySQL). Must be a plain identifier, [A-Za-z_][A-Za-z0-9_]{0,62}, or false for no prefix (use the connection's search_path)

Invalid or unknown options raise ArgumentError before any query runs.

Summary

Functions

Merge user options over the defaults, validating every value. Raises ArgumentError.

Safety options (:statement_timeout, :lock_timeout, :read_only) merged over defaults.

Validate an Oban prefix. Ecto already quotes it as an identifier (and rejects quote characters); this is a stricter allow-list so a prefix can only ever be a plain name.

Functions

build(opts)

Merge user options over the defaults, validating every value. Raises ArgumentError.

defaults()

guard(opts)

Safety options (:statement_timeout, :lock_timeout, :read_only) merged over defaults.

guard_defaults()

valid_prefix?(prefix)

Validate an Oban prefix. Ecto already quotes it as an identifier (and rejects quote characters); this is a stricter allow-list so a prefix can only ever be a plain name.