Class Judge

Judge implementation that handles evaluation functionality and conversation management.

According to the AIEval spec, judges are AI Configs with mode: "judge" that evaluate other AI Configs using structured output.

Hierarchy

Judge

Index

Constructors

constructor

new Judge(_aiConfig, _aiConfigTracker, _aiProvider, logger?): Judge
Parameters
- _aiConfig: LDAIJudgeConfig
- _aiConfigTracker: LDAIConfigTracker
- _aiProvider: AIProvider
- Optional logger: LDLogger
  Optional
Returns Judge
- Defined in sdk/server-ai/src/api/judge/Judge.ts:22

Properties

`Private` `Readonly` _aiConfig

_aiConfig: LDAIJudgeConfig

`Private` `Readonly` _aiConfigTracker

_aiConfigTracker: LDAIConfigTracker

`Private` `Readonly` _aiProvider

_aiProvider: AIProvider

`Private` `Readonly` _evaluationResponseStructure

_evaluationResponseStructure: Record<string, unknown>

`Private` `Optional` `Readonly` _logger

_logger?: LDLogger

Methods

`Private` _constructEvaluationMessages

_constructEvaluationMessages(input, output): LDMessage[]
Constructs evaluation messages by combining judge's config messages with input/output.
Parameters
- input: string
- output: string
Returns LDMessage[]
- Defined in sdk/server-ai/src/api/judge/Judge.ts:148

`Private` _interpolateMessage

_interpolateMessage(content, variables): string
Interpolates message content with variables using Mustache templating.
Parameters
- content: string
- variables: Record<string, string>
Returns string
- Defined in sdk/server-ai/src/api/judge/Judge.ts:163

`Private` _parseEvaluationResponse

_parseEvaluationResponse(data): Record<string, EvalScore>
Parses the structured evaluation response from the AI provider.
Parameters
- data: Record<string, unknown>
Returns Record<string, EvalScore>
- Defined in sdk/server-ai/src/api/judge/Judge.ts:170

evaluate

evaluate(input, output, samplingRate?): Promise<undefined | JudgeResponse>
Evaluates an AI response using the judge's configuration.
Parameters
- input: string
  
  The input prompt or question that was provided to the AI
- output: string
  
  The AI-generated response to be evaluated
- samplingRate: number = 1
  
  Sampling rate (0-1) to determine if evaluation should be processed (defaults to 1)
Returns Promise<undefined | JudgeResponse>
Promise that resolves to evaluation results or undefined if not sampled
- Defined in sdk/server-ai/src/api/judge/Judge.ts:42

evaluateMessages

evaluateMessages(messages, response, samplingRatio?): Promise<undefined | JudgeResponse>
Evaluates an AI response from chat messages and response.
Parameters
- messages: LDMessage[]
  
  Array of messages representing the conversation history
- response: ChatResponse
  
  The AI response to be evaluated
- samplingRatio: number = 1
  
  Sampling ratio (0-1) to determine if evaluation should be processed (defaults to 1)
Returns Promise<undefined | JudgeResponse>
Promise that resolves to evaluation results or undefined if not sampled
- Defined in sdk/server-ai/src/api/judge/Judge.ts:113

getAIConfig

getAIConfig(): LDAIJudgeConfig
Returns the AI Config used by this judge.

Returns LDAIJudgeConfig
- Defined in sdk/server-ai/src/api/judge/Judge.ts:127

getProvider

getProvider(): AIProvider
Returns the AI provider used by this judge.

Returns AIProvider
- Defined in sdk/server-ai/src/api/judge/Judge.ts:141

getTracker

getTracker(): LDAIConfigTracker
Returns the tracker associated with this judge.

Returns LDAIConfigTracker
- Defined in sdk/server-ai/src/api/judge/Judge.ts:134

Class Judge

Hierarchy

Index

Constructors

Properties

Methods

Constructors

constructor

Parameters

_aiConfig: LDAIJudgeConfig

_aiConfigTracker: LDAIConfigTracker

_aiProvider: AIProvider

Optional logger: LDLogger

Returns Judge

Properties

Private Readonly _aiConfig

Private Readonly _aiConfigTracker

Private Readonly _aiProvider

Private Readonly _evaluationResponseStructure

Private Optional Readonly _logger

Methods

Private _constructEvaluationMessages

Parameters

input: string

output: string

Returns LDMessage[]

Private _interpolateMessage

Parameters

content: string

variables: Record<string, string>

Returns string

Private _parseEvaluationResponse

Parameters

data: Record<string, unknown>

Returns Record<string, EvalScore>

evaluate

Parameters

input: string

output: string

samplingRate: number = 1

Returns Promise<undefined | JudgeResponse>

evaluateMessages

Parameters

messages: LDMessage[]

response: ChatResponse

samplingRatio: number = 1

Returns Promise<undefined | JudgeResponse>

getAIConfig

Returns LDAIJudgeConfig

getProvider

Returns AIProvider

getTracker

Returns LDAIConfigTracker

Settings

Member Visibility

Theme

On This Page

`Optional` logger: LDLogger

`Private` `Readonly` _aiConfig

`Private` `Readonly` _aiConfigTracker

`Private` `Readonly` _aiProvider

`Private` `Readonly` _evaluationResponseStructure

`Private` `Optional` `Readonly` _logger

`Private` _constructEvaluationMessages

`Private` _interpolateMessage

`Private` _parseEvaluationResponse