Clustering

How an occurrence is claimed

Every replica arms a timer for the same occurrence. When it fires, a cluster job goes through two claims in the store before it runs:

1. The occurrence claim, occ:<job>:<epoch ms>. The first replica to create it runs the occurrence, the others record skipped-claimed. It is never released: a replica whose clock is a few seconds behind, waking up after the winner already finished, still finds it and stays out.
2. The overlap lock, active:<job>, only with overlap: 'skip'. It is held for the whole run and extended by a heartbeat every third of lockTtl. While it is held, the next occurrence is skipped-overlap on every replica.

Instance jobs skip both. Their overlap is checked locally.

What the guarantee is

A cluster job runs at most once per occurrence in normal operation, and at least once as long as one replica is up when it is due (with misfire: 'run-once' covering restarts). The edge is a lost lock: when the heartbeat cannot extend the overlap lock, the run's ctx.signal is aborted with reason lock-lost, because another replica may take over. Write handlers so that running twice does no harm, using ctx.occurrenceId as the idempotency key, and check ctx.signal between steps.

Stores

RedisStore

A claim is a key created with SET NX PX. Extending and releasing go through scripts that check the owner first, so a replica can never touch a claim it lost. Every operation uses a single key, which keeps Redis Cluster working without hash tags.

store: defineSchedulerStore({
  store: RedisStore,
  initOptions: { url: process.env.REDIS_URL, prefix: 'myapp:scheduler' },
}),

Pass client instead of url to share a connection you already have, for example the one BullMQ uses. The store never closes a client it was given.

PostgresStore

Claims and state live in two tables, created on first start unless createTables: false. Expiry is always computed with the database's now(), so pods whose clocks drift apart still agree on when a claim expires. Claims that expired more than an hour ago are deleted every ten minutes.

store: defineSchedulerStore({
  store: PostgresStore,
  initOptions: { connectionString: process.env.DATABASE_URL, schema: 'ops' },
}),

Advisory locks were left out on purpose: they belong to a session, which breaks behind a connection pool. The table definitions are in the PostgresStore source for teams whose migrations own the schema.

MemoryStore in production

MemoryStore only coordinates the process it lives in. In development the scheduler logs one warning and carries on. With NODE_ENV=production, cluster jobs on it refuse to start, since every replica would run them. Set allowLocalCluster: true only for a deployment that really is a single process.

Misfires

An occurrence is missed when no replica was up at the time, or when a timer fires later than misfireGrace because the event loop was blocked, the laptop slept or the pod was throttled.

• misfire: 'skip' records it as missed and waits for the next one.
• misfire: 'run-once' runs the latest missed occurrence, with ctx.trigger === 'misfire'. On start, each cluster job compares the store's record of its last run with its schedule and catches up on anything missed in the last 24 hours, through the normal claim, so twenty pods starting together run it once.

A job the store has no record of is never caught up on, so a new daily job does not run the moment it is first deployed.

Kubernetes and multi-server setups

Run as many replicas as you like with the same image and the same store. Nothing needs to know about the others.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 4
  template:
    spec:
      terminationGracePeriodSeconds: 30
      containers:
        - name: api
          image: registry.example.com/api:1.4.0
          env:
            - name: REDIS_URL
              value: redis://redis:6379

The replica id defaults to HOSTNAME, which Kubernetes sets to the pod name, followed by the process id and a random suffix. It shows up in the store, the devtools and every run event, so you can tell which pod ran what.

To keep scheduled work off the pods that serve traffic, set autoStart: false on the web deployment and run a second deployment of the same image with it on.

Graceful shutdown

stop() stops arming timers, waits for runs in flight, and once shutdownTimeout is reached aborts them with reason shutdown. Overlap locks are released, occurrence claims stay so no other replica runs those occurrences again.

SchedulerPlugin closes the scheduler with the application, so turning on signals in the core config is enough: on SIGTERM the HTTP server stops first, then the scheduler drains, then the process exits.

export default defineConfig({
  shutdown: { handleSignals: true },
  plugins: [[SchedulerPlugin, { store, shutdownTimeout: '25s' }]],
});

The scheduler can also stop on its own signals with handleSignals: true in its options. It never exits the process then, other parts of the app may still be closing. Either way, keep shutdownTimeout below the pod's terminationGracePeriodSeconds (30 seconds by default), or Kubernetes kills the process before the runs were aborted cleanly.

External triggers

On platforms without a long-lived process, or when you prefer Kubernetes CronJobs, an external scheduler can fire the jobs instead. Set external: true on jobs that should never run from an in-process timer. Both triggers below claim the occurrence the job's schedule is at, so a CronJob that fires twice for 03:00 still runs the job once.

From the CLI

With SchedulerPlugin listed in vercube.config.ts, the CLI gets two commands. They import the built app without starting its server or its timers.

$ vercube scheduler list
NAME               SCHEDULE   TIMEZONE       SCOPE    TARGET  NEXT RUNS
billing.close-day  0 3 * * *  Europe/Warsaw  cluster  inline  2026-01-02T02:00:00.000Z

$ vercube scheduler run billing.close-day
{
  "name": "billing.close-day",
  "occurrenceId": "billing.close-day@2026-01-01T02:00:00.000Z",
  "outcome": "completed",
  ...
}

run exits with 0 when the job completed, was enqueued, or was skipped because another replica has it, and with 1 when it failed. --force skips the occurrence claim, --entry points at another built entry, --build builds the app first.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: billing-close-day
spec:
  schedule: '0 3 * * *'
  timeZone: Europe/Warsaw
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: run
              image: registry.example.com/api:1.4.0
              command: ['npx', 'vercube', 'scheduler', 'run', 'billing.close-day']

Over HTTP

The http option mounts GET and POST <path>/run/<job>. It refuses to start without a secret, and every request has to send it as a bearer token.

[
  SchedulerPlugin,
  { store, autoStart: false, http: { secret: process.env.CRON_SECRET } },
];

The response is sent once the job finished, because a serverless function stops as soon as it has answered: 200 with { outcome, occurrenceId, durationMs }, 401 for a wrong secret, 404 for an unknown job, 500 when it failed. Vercel Cron sends GET with Authorization: Bearer $CRON_SECRET, so this is all it needs:

{
  "crons": [{ "path": "/_scheduler/run/billing.close-day", "schedule": "0 3 * * *" }]
}

Queue targets

For heavy work, or work that needs retries with backoff, hand the occurrence to @vercube/queue. The replica that claims the occurrence only enqueues it, and whichever worker consumes the queue runs the method.

@Cron('0 * * * *', {
  name: 'reports.build',
  target: {
    queue: 'reports',
    attempts: 5,
    backoff: { type: 'exponential', delay: 1000 },
  },
})
public async build(ctx: SchedulerTypes.JobContext): Promise<void> {
  // runs on a worker, ctx.trigger is 'queue' and ctx.attempt counts the queue's
  // attempts
}

The occurrence is still claimed before it is enqueued: only some transports deduplicate on the job id. Overlap is enforced on the worker, with the same lock as an inline run. A run that fails is rethrown to the queue, so its attempts and backoff apply, and the job's own retries are ignored. The scheduler and QueuePlugin both have to be registered, and a queue target needs cluster scope.

Observability

Every run is a scheduler.run <job> span carrying the job, scope, trigger, occurrence, replica, attempt and outcome. A queue target's enqueue runs inside that span, so the worker's run lands in the same trace. Three metrics come with it:

With @vercube/devtools enabled, the Scheduler panel lists every job with its next run, its last run anywhere in the cluster and the replica that ran it, and runs a job on demand.