Constructor Detail

Method Detail

• id

   final String id()

  A unique identifier for the experiment event. If you don't provide one, BrainTrust will generate one for you

• _xactId

   final String _xactId()

  The transaction id of an event is unique to the network operation that processed the event insertion. Transaction ids are monotonically increasing over time and can be used to retrieve a versioned snapshot of the experiment (see the version parameter)

• created

   final OffsetDateTime created()

  The timestamp the experiment event was created

• experimentId

   final String experimentId()

  Unique identifier for the experiment

• projectId

   final String projectId()

  Unique identifier for the project that the experiment belongs under

• rootSpanId

   final String rootSpanId()

  A unique identifier for the trace this experiment event belongs to

• spanId

   final String spanId()

  A unique identifier used to link different experiment events together as part of a full trace. See the tracing guide for full details on tracing

• context

   final Optional<ExperimentEvent.Context> context()

  Context is additional information about the code that produced the experiment event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the experiment event

• _error

   final JsonValue _error()

  The error that occurred, if any.

• _expected

   final JsonValue _expected()

  The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate your experiments while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models

• _input

   final JsonValue _input()

  The arguments that uniquely define a test case (an arbitrary, JSON serializable object). Later on, Braintrust will use the input to know whether two test cases are the same between experiments, so they should not contain experiment-specific state. A simple rule of thumb is that if you run the same experiment twice, the input should be identical

• isRoot

   final Optional<Boolean> isRoot()

  Whether this span is a root span

• metadata

   final Optional<ExperimentEvent.Metadata> metadata()

  A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings

• metrics

   final Optional<ExperimentEvent.Metrics> metrics()

  Metrics are numerical measurements tracking the execution of the code that produced the experiment event. Use "start" and "end" to track the time span over which the experiment event was produced

• origin

   final Optional<ObjectReference> origin()

  Indicates the event was copied from another object.

• _output

   final JsonValue _output()

  The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question

• scores

   final Optional<ExperimentEvent.Scores> scores()

  A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare experiments

• spanAttributes

   final Optional<SpanAttributes> spanAttributes()

  Human-identifying attributes of the span, such as name, type, etc.

• spanParents

   final Optional<List<String>> spanParents()

  An array of the parent span_ids of this experiment event. This should be empty for the root span of a trace, and should most often contain just one parent element for subspans

• tags

   final Optional<List<String>> tags()

  A list of tags to log

• _id

   final JsonField<String> _id()

  Returns the raw JSON value of id.

  Unlike id, this method doesn't throw if the JSON field has an unexpected type.

• __xactId

   final JsonField<String> __xactId()

  Returns the raw JSON value of _xactId.

  Unlike _xactId, this method doesn't throw if the JSON field has an unexpected type.

• _created

   final JsonField<OffsetDateTime> _created()

  Returns the raw JSON value of created.

  Unlike created, this method doesn't throw if the JSON field has an unexpected type.

• _experimentId

   final JsonField<String> _experimentId()

  Returns the raw JSON value of experimentId.

  Unlike experimentId, this method doesn't throw if the JSON field has an unexpected type.

• _projectId

   final JsonField<String> _projectId()

  Returns the raw JSON value of projectId.

  Unlike projectId, this method doesn't throw if the JSON field has an unexpected type.

• _rootSpanId

   final JsonField<String> _rootSpanId()

  Returns the raw JSON value of rootSpanId.

  Unlike rootSpanId, this method doesn't throw if the JSON field has an unexpected type.

• _spanId

   final JsonField<String> _spanId()

  Returns the raw JSON value of spanId.

  Unlike spanId, this method doesn't throw if the JSON field has an unexpected type.

• _context

   final JsonField<ExperimentEvent.Context> _context()

  Returns the raw JSON value of context.

  Unlike context, this method doesn't throw if the JSON field has an unexpected type.

• _isRoot

   final JsonField<Boolean> _isRoot()

  Returns the raw JSON value of isRoot.

  Unlike isRoot, this method doesn't throw if the JSON field has an unexpected type.

• _metadata

   final JsonField<ExperimentEvent.Metadata> _metadata()

  Returns the raw JSON value of metadata.

  Unlike metadata, this method doesn't throw if the JSON field has an unexpected type.

• _metrics

   final JsonField<ExperimentEvent.Metrics> _metrics()

  Returns the raw JSON value of metrics.

  Unlike metrics, this method doesn't throw if the JSON field has an unexpected type.

• _origin

   final JsonField<ObjectReference> _origin()

  Returns the raw JSON value of origin.

  Unlike origin, this method doesn't throw if the JSON field has an unexpected type.

• _scores

   final JsonField<ExperimentEvent.Scores> _scores()

  Returns the raw JSON value of scores.

  Unlike scores, this method doesn't throw if the JSON field has an unexpected type.

• _spanAttributes

   final JsonField<SpanAttributes> _spanAttributes()

  Returns the raw JSON value of spanAttributes.

  Unlike spanAttributes, this method doesn't throw if the JSON field has an unexpected type.

• _spanParents

   final JsonField<List<String>> _spanParents()

  Returns the raw JSON value of spanParents.

  Unlike spanParents, this method doesn't throw if the JSON field has an unexpected type.

• _tags

   final JsonField<List<String>> _tags()

  Returns the raw JSON value of tags.

  Unlike tags, this method doesn't throw if the JSON field has an unexpected type.

• _additionalProperties

   final Map<String, JsonValue> _additionalProperties()

• toBuilder

   final ExperimentEvent.Builder toBuilder()

• validate

   final ExperimentEvent validate()

• isValid

   final Boolean isValid()

• equals

   Boolean equals(Object other)

• hashCode

   Integer hashCode()

• toString

   String toString()

• builder

   final static ExperimentEvent.Builder builder()

  Returns a mutable builder for constructing an instance of ExperimentEvent.

  The following fields are required:

  .id()
  ._xactId()
  .created()
  .experimentId()
  .projectId()
  .rootSpanId()
  .spanId()