Skip to main content

MarketDataBackfillProgress

Aggregated backfill/upfill progress for one runner. Drives the dashboard's "Data availability" panel: the progress bar (readyRefs / configuredRefs), chips for pending/running/failed, a split between backfill (history extension) and upfill (catch-up), and a per-exchange rollup.

type MarketDataBackfillProgress {
  runnerID: ID!
  totalPending: Int!
  totalRunning: Int!
  totalFailed: Int!
  totalDone: Int!
  configuredRefs: Int!
  readyRefs: Int!
  backfillPending: Int!
  upfillPending: Int!
  oldestPendingCreatedAt: Float!
  byExchange: [MarketDataBackfillProgressByExchange!]!
}

Fields​

MarketDataBackfillProgress.runnerID ● ID! non-null scalar​

The runner this progress summary applies to.

MarketDataBackfillProgress.totalPending ● Int! non-null scalar​

Queued backfill jobs (rows) not yet claimed by a worker.

MarketDataBackfillProgress.totalRunning ● Int! non-null scalar​

Backfill jobs (rows) a worker is executing right now.

MarketDataBackfillProgress.totalFailed ● Int! non-null scalar​

Configured refs that are NOT ready and whose most recent download attempt failed (a terminal failure since their last success). Counts refs, not job rows.

MarketDataBackfillProgress.totalDone ● Int! non-null scalar​

Backfill jobs (rows) that finished successfully within the job table's retention window (about a day). Recent activity, not a lifetime total — use readyRefs for completion.

MarketDataBackfillProgress.configuredRefs ● Int! non-null scalar​

Refs the runner is configured to hold, excluding refs that can never become ready (unencodable pairs, refs in the permanent invalid-ref cache). The denominator of the progress bar.

MarketDataBackfillProgress.readyRefs ● Int! non-null scalar​

Configured refs that have a file in the data lake. The numerator of the progress bar.

MarketDataBackfillProgress.backfillPending ● Int! non-null scalar​

Jobs classified as history extension (startMs earlier than existing data) that are still pending or running. Used for the "Fetching historical data" section header count.

MarketDataBackfillProgress.upfillPending ● Int! non-null scalar​

Jobs classified as incremental catch-up (window ends after existing data's last candle) that are still pending or running. Drives the "Catching up to current time" section header count.

MarketDataBackfillProgress.oldestPendingCreatedAt ● Float! non-null scalar​

UNIX ms of the oldest pending row's createdAt. Zero when no rows are pending. The dashboard uses this to highlight queues that have been stuck for an unusually long time.

MarketDataBackfillProgress.byExchange ● [MarketDataBackfillProgressByExchange!]! non-null object​

Alphabetical rollup by exchange for per-exchange chips.

Returned By​

marketDataBackfillProgress query