Logotipo Hedhog

Get Hedhog updates in your inbox

New releases, fresh recipes, and breaking changes — no spam.

Queue Providers

Hedhog processes background jobs — email sending, report generation, file processing, scheduled tasks — through @hed-hog-pro/queue. Only the Database provider is actually implemented today. RabbitMQ and BullMQ exist as settings and a provider enum value, but there is no working RabbitMQ or BullMQ worker behind them yet — picking either one in settings will not move jobs.

This isn't a configuration mistake on your part if it doesn't work; it's the current state of the library. Build on Database, and treat the RabbitMQ/BullMQ settings as reserved for a future release rather than something to debug.

Status: ⚠️ Partial — Database provider is fully implemented; RabbitMQ and BullMQ are reserved settings with no provider behind them yet.

queue_job table polled by QueueWorkerProcessor, with RabbitMQ and BullMQ shown as not-yet-implemented

Quick Setup

  1. Leave queue.provider set to database — it's the only one that works
  2. Register a job handler via QueueModule.register({ handlers: {...} })
  3. Enqueue a job and confirm a queue_job row appears, then flips from pending to done

How It Works (Database provider)

  1. Application code enqueues a job, e.g. QueueService.add('send-email', payload)
  2. The job is written as a row in the queue_job table with status = 'pending'
  3. QueueWorkerProcessor polls that table on an interval, atomically claims a batch of jobs via an optimistic-lock updateMany (so multiple API instances can run workers without double-processing the same job), and invokes the matching registered handler
  4. On success the job is marked done; on failure it's retried up to queue.default_max_attempts, then marked failed
  5. Jobs stuck in processing past queue.database.lock_timeout_seconds (e.g. a worker crashed mid-job) are automatically released back for retry — you don't need to manually unstick them

Real Setting Keys

Unlike most other recipes, queue settings use dot-separated keys, not hyphens — don't second-guess yourself if you see queue.database.poll_interval_seconds instead of queue-database-poll-interval-seconds, that's correct.

Provider selection & defaults

Setting keyDescription
queue.providerActive provider — leave this as database, the only one that works
queue.default_max_attemptsDefault retries before a job is marked failed (default 3)
queue.default_retry_delay_secondsDelay before retrying a failed job (default 300)
queue.default_timeout_secondsMax time a single job run gets before being considered stuck (default 900)

Database provider (working)

Setting keyDescription
queue.database.enabledEnable the DB-backed provider (default true)
queue.database.poll_interval_secondsHow often the worker checks for new jobs (default 5)
queue.database.batch_sizeJobs claimed per poll (default 20)
queue.database.lock_timeout_secondsHow long a job can stay processing before being treated as stale and released (default 300)

Worker behavior (applies regardless of provider)

Setting keyDescription
queue.worker.enabledWhether this instance runs a worker at all — set to false on instances that should only enqueue, never process
queue.worker.heartbeat_interval_secondsHow often the worker reports liveness (default 30)
queue.worker.dynamic_scope_enabledAllow the set of queues this worker processes to change at runtime without a restart (default true)
queue.worker.scope_refresh_interval_secondsHow often the worker re-checks its assigned queue scope (default 5)
queue.worker.scope_drain_timeout_secondsGrace period to finish in-flight jobs when a queue is removed from scope (default 30)

RabbitMQ / BullMQ (reserved, not implemented)

Setting keyStatus
queue.rabbitmq.enabled, queue.rabbitmq.url, queue.rabbitmq.exchange, queue.rabbitmq.default_queueSettings exist; no consuming code exists yet
queue.bullmq.enabled, queue.bullmq.redis_urlSettings exist; no consuming code exists yet

Don't spin up RabbitMQ or Redis expecting Hedhog to pick them up — there's nothing on the other end listening yet.

Defining Job Handlers

Register handlers in your NestJS module:

import { QueueModule } from '@hed-hog-pro/queue';

@Module({
  imports: [
    QueueModule.register({
      handlers: {
        'send-email': SendEmailHandler,
        'process-report': ProcessReportHandler,
      },
    }),
  ],
})
export class AppModule {}

Each handler is a class with a handle(job) method:

@Injectable()
export class SendEmailHandler implements JobHandler {
  async handle(job: Job<{ to: string; subject: string; body: string }>) {
    await this.mail.send(job.data);
  }
}

Throw a NonRetryableError (from @hed-hog-pro/queue) instead of a regular error if a failure should not be retried — for example, a job that failed because the input data was permanently invalid, not because of a transient outage.

Scaling the Database Provider

Since the worker uses an atomic claim (updateMany with a status filter as an optimistic lock), it's safe to run multiple API instances with queue.worker.enabled = true simultaneously — they won't double-process the same job. Throughput is bounded by queue.database.poll_interval_seconds and queue.database.batch_size; lower the poll interval or raise the batch size for higher volume, at the cost of more frequent polling load on PostgreSQL.


Verify It Worked

  • Enqueue a test job and confirm a row appears in queue_job with status = 'pending', then flips to done within queue.database.poll_interval_seconds
  • Force a handler to throw once and confirm the job retries instead of disappearing, up to queue.default_max_attempts

Troubleshooting

SymptomLikely cause
Jobs enqueue but never runqueue.worker.enabled is false on every running instance
queue.provider set to rabbitmq or bullmq and nothing happensExpected — neither provider is implemented; switch back to database
A job is stuck processing indefinitelyThe worker that claimed it crashed before queue.database.lock_timeout_seconds elapsed — it will self-release after that timeout, not before
Same job processed twiceShould not happen with the Database provider's optimistic lock — if you see this, check for a custom handler that isn't idempotent rather than the queue layer itself