Class Judge

Judge implementation that handles evaluation functionality and conversation management.

According to the AIEval spec, judges are AI Configs with mode: "judge" that evaluate other AI Configs using structured output.

Index

Constructors

Accessors

Methods

evaluate evaluateMessages getAIConfig getRunner

Constructors

constructor

new Judge(
    _aiConfig: LDAIJudgeConfig,
    _runner: Runner,
    _sampleRate?: number,
    logger?: LDLogger,
): Judge
Parameters
- _aiConfig: LDAIJudgeConfig
- _runner: Runner
- _sampleRate: number = 1.0
- Optionallogger: LDLogger
Returns Judge
- Defined in api/judge/Judge.ts:56

Accessors

sampleRate

get sampleRate(): number
The default sampling rate baked in at construction. Used by evaluate / evaluateMessages when no per-call rate is supplied.

Returns number
- Defined in api/judge/Judge.ts:69

Methods

evaluate

evaluate(
    input: string,
    output: string,
    samplingRate?: number,
): Promise<LDJudgeResult>
Evaluates an AI response using the judge's configuration.
Parameters
- input: string
  The input prompt or question that was provided to the AI
- output: string
  The AI-generated response to be evaluated
- OptionalsamplingRate: number
  Sampling rate (0-1) to determine if evaluation should be processed. When omitted, the Judge's constructor-default rate is used. An explicit 0 overrides the default — only undefined falls through.
Returns Promise<LDJudgeResult>
Promise that resolves to evaluation results
- Defined in api/judge/Judge.ts:98

evaluateMessages

evaluateMessages(
    messages: LDMessage[],
    response: RunnerResult,
    samplingRatio?: number,
): Promise<LDJudgeResult>
Evaluates an AI response from chat messages and a runner result.

Each message is rendered as <role>: <content> so the judge model can distinguish speakers in the message history. Messages are joined with a single newline.
Parameters
- messages: LDMessage[]
  Array of messages representing the conversation history
- response: RunnerResult
  The runner result containing the AI-generated content to evaluate
- OptionalsamplingRatio: number
  Sampling ratio (0-1). When omitted, the Judge's constructor-default rate is used.
Returns Promise<LDJudgeResult>
Promise that resolves to evaluation results
- Defined in api/judge/Judge.ts:171

getAIConfig

getAIConfig(): LDAIJudgeConfig
Returns the AI Config used by this judge.

Returns LDAIJudgeConfig
- Defined in api/judge/Judge.ts:186

getRunner

getRunner(): Runner
Returns the runner used by this judge.

Returns Runner
- Defined in api/judge/Judge.ts:193