Interface ILdAiConfigTracker

A utility capable of generating events related to a specific AI model configuration.

Namespace: LaunchDarkly.Sdk.Server.Ai.Interfaces

Assembly: LaunchDarkly.ServerSdk.Ai.dll

Syntax

public interface ILdAiConfigTracker

Properties

Config

The AI model configuration retrieved from LaunchDarkly, or a default value if unable to retrieve.

Declaration

LdAiConfig Config { get; }

Property Value

Type	Description
LdAiConfig

Methods

TrackDuration(float)

Tracks a duration metric related to this config. For example, if a particular operation related to usage of the AI model takes 100ms, this can be tracked and made available in LaunchDarkly.

Declaration

void TrackDuration(float durationMs)

Parameters

Type	Name	Description
float	durationMs	the duration in milliseconds

TrackDurationOfTask<T>(Task<T>)

Tracks the duration of a task, and returns the result of the task.

If the provided task throws, then this method will also throw.

In the case the provided function throws, this function will still record the duration.

Declaration

Task<T> TrackDurationOfTask<T>(Task<T> task)

Parameters

Type	Name	Description
Task<T>	task	the task

Returns

Type	Description
Task<T>	the task

Type Parameters

Name	Description
T	type of the task's result

TrackError()

Tracks an unsuccessful generation event related to this config.

Declaration

void TrackError()

TrackFeedback(Feedback)

Tracks feedback (positive or negative) related to the output of the model.

Declaration

void TrackFeedback(Feedback feedback)

Parameters

Type	Name	Description
Feedback	feedback	the feedback

Exceptions

Type	Condition
ArgumentOutOfRangeException	thrown if the feedback value is not Positive or Negative

TrackRequest(Task<Response>)

Tracks a request to a provider. The request is a task that returns a Response, which contains information about the request such as token usage and metrics.

This function will track the duration of the operation, the token usage, and the success or error status.

If the provided function throws, then this method will also throw.

In the case the provided function throws, this function will record the duration and an error.

A failed operation will not have any token usage data.

It is the responsibility of the caller to fill in the Response object with any details that should be tracked.

Example:

var response = tracker.TrackRequest(Task.Run(() => new Response {
    // 1. Make a request to the AI provider
    // 2. Identify relevant statistics returned in the response
    // 3. Return a Response object containing the relevant statistics
    Usage = new Usage { Total = 1, Input = 1, Output = 1 },
    Metrics = new Metrics { LatencyMs = 100 }
}));

If no latency statistic is explicitly returned in the Response, then the duration of the Task is automatically measured and recorded as the latency metric associated with this request.

Declaration

Task<Response> TrackRequest(Task<Response> request)

Parameters

Type	Name	Description
Task<Response>	request	a task representing the request

Returns

Type	Description
Task<Response>	the task

TrackSuccess()

Tracks a generation event related to this config.

Declaration

void TrackSuccess()

TrackTimeToFirstToken(float)

Tracks the time it takes for the first token to be generated.

Declaration

void TrackTimeToFirstToken(float timeToFirstTokenMs)

Parameters

Type	Name	Description
float	timeToFirstTokenMs	the duration in milliseconds

TrackTokens(Usage)

Tracks token usage related to this config.

Declaration

void TrackTokens(Usage usage)

Parameters

Type	Name	Description
Usage	usage	the token usage