AIModelScorecard

One model's scorecard over a window. A ratio with no denominator is null ("no data"), never zero.

type AIModelScorecard {
  model: String!
  calls: Int!
  failedRuns: Int!
  failurePerCall: Float
  turns: Int!
  costPerTurn: Float
  missionSteps: Int!
  costPerMissionStep: Float
  feedbackUp: Int!
  feedbackDown: Int!
  feedbackRatio: Float
  evalPassRate: Float
  evalPassed: Boolean
  evalRecordedAt: Time
}

Fields

AIModelScorecard.model ● String! non-null scalar

Model id (OpenRouter), as ai_usage_event and the catalogue record it.

AIModelScorecard.calls ● Int! non-null scalar

Model calls recorded (ai_usage_event rows).

AIModelScorecard.failedRuns ● Int! non-null scalar

Runs on this model that ended failed, timeout or empty_answer.

AIModelScorecard.failurePerCall ● Float scalar

Provider failures per attempted call: failedRuns / (calls + failedRuns).

AIModelScorecard.turns ● Int! non-null scalar

Chat turns (interactive and background) the model served.

AIModelScorecard.costPerTurn ● Float scalar

Average credits per turn, markup included.

AIModelScorecard.missionSteps ● Int! non-null scalar

Mission steps the model served.

AIModelScorecard.costPerMissionStep ● Float scalar

Average credits per Mission step, markup included.

AIModelScorecard.feedbackUp ● Int! non-null scalar

AIModelScorecard.feedbackDown ● Int! non-null scalar

AIModelScorecard.feedbackRatio ● Float scalar

Thumbs up over rated answers.

AIModelScorecard.evalPassRate ● Float scalar

Scenarios passed over scenarios in the model's latest eval record (not windowed).

AIModelScorecard.evalPassed ● Boolean scalar

Whether the latest eval record passed the release bar.

AIModelScorecard.evalRecordedAt ● Time scalar

Returned By

aiModelScorecard query