Constructor Detail

Method Detail

• id

   final InsertExperimentEvent.Builder id(String id)

  A unique identifier for the experiment event. If you don't provide one, BrainTrust will generate one for you

• id

   final InsertExperimentEvent.Builder id(Optional<String> id)

  Alias for calling Builder.id with id.orElse(null).

• id

   final InsertExperimentEvent.Builder id(JsonField<String> id)

  Sets Builder.id to an arbitrary JSON value.

  You should usually call Builder.id with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• _isMerge

   final InsertExperimentEvent.Builder _isMerge(Boolean _isMerge)

  The _is_merge field controls how the row is merged with any existing row with the same id in the DB. By default (or when set to false), the existing row is completely replaced by the new row. When set to true, the new row is deep-merged into the existing row, if one is found. If no existing row is found, the new row is inserted as is.

  For example, say there is an existing row in the DB {"id": "foo", "input": {"a": 5, "b": 10}}. If we merge a new row as {"_is_merge": true, "id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"a": 5, "b": 11, "c": 20}}. If we replace the new row as {"id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"b": 11, "c": 20}}

• _isMerge

   final InsertExperimentEvent.Builder _isMerge(Boolean _isMerge)

  Alias for Builder._isMerge.

  This unboxed primitive overload exists for backwards compatibility.

• _isMerge

   final InsertExperimentEvent.Builder _isMerge(Optional<Boolean> _isMerge)

  Alias for calling Builder._isMerge with _isMerge.orElse(null).

• _isMerge

   final InsertExperimentEvent.Builder _isMerge(JsonField<Boolean> _isMerge)

  Sets Builder._isMerge to an arbitrary JSON value.

  You should usually call Builder._isMerge with a well-typed Boolean value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• _mergePaths

   final InsertExperimentEvent.Builder _mergePaths(List<List<String>> _mergePaths)

  The _merge_paths field allows controlling the depth of the merge, when _is_merge=true. _merge_paths is a list of paths, where each path is a list of field names. The deep merge will not descend below any of the specified merge paths.

  For example, say there is an existing row in the DB {"id": "foo", "input": {"a": {"b": 10}, "c": {"d": 20}}, "output": {"a": 20}}. If we merge a new row as {"_is_merge": true, "_merge_paths": [["input", "a"], ["output"]], "input": {"a": {"q": 30}, "c": {"e": 30}, "bar": "baz"}, "output": {"d": 40}}, the new row will be {"id": "foo": "input": {"a": {"q": 30}, "c": {"d": 20, "e": 30}, "bar": "baz"}, "output": {"d": 40}}. In this case, due to the merge paths, we have replaced input.a and output, but have still deep-merged input and input.c.

• _mergePaths

   final InsertExperimentEvent.Builder _mergePaths(Optional<List<List<String>>> _mergePaths)

  Alias for calling Builder._mergePaths with _mergePaths.orElse(null).

• _mergePaths

   final InsertExperimentEvent.Builder _mergePaths(JsonField<List<List<String>>> _mergePaths)

  Sets Builder._mergePaths to an arbitrary JSON value.

  You should usually call Builder._mergePaths with a well-typed List<List<String>> value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• addMergePath

   final InsertExperimentEvent.Builder addMergePath(List<String> mergePath)

  Adds a single List<String> to _mergePaths.

• _objectDelete

   final InsertExperimentEvent.Builder _objectDelete(Boolean _objectDelete)

  Pass _object_delete=true to mark the experiment event deleted. Deleted events will not show up in subsequent fetches for this experiment

• _objectDelete

   final InsertExperimentEvent.Builder _objectDelete(Boolean _objectDelete)

  Alias for Builder._objectDelete.

  This unboxed primitive overload exists for backwards compatibility.

• _objectDelete

   final InsertExperimentEvent.Builder _objectDelete(Optional<Boolean> _objectDelete)

  Alias for calling Builder._objectDelete with _objectDelete.orElse(null).

• _objectDelete

   final InsertExperimentEvent.Builder _objectDelete(JsonField<Boolean> _objectDelete)

  Sets Builder._objectDelete to an arbitrary JSON value.

  You should usually call Builder._objectDelete with a well-typed Boolean value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• _parentId

   final InsertExperimentEvent.Builder _parentId(String _parentId)

  DEPRECATED: The _parent_id field is deprecated and should not be used. Support for _parent_id will be dropped in a future version of Braintrust. Log span_id, root_span_id, and span_parents explicitly instead.

  Use the _parent_id field to create this row as a subspan of an existing row. Tracking hierarchical relationships are important for tracing (see the guide for full details).

  For example, say we have logged a row {"id": "abc", "input": "foo", "output": "bar", "expected": "boo", "scores": {"correctness": 0.33}}. We can create a sub-span of the parent row by logging {"_parent_id": "abc", "id": "llm_call", "input": {"prompt": "What comes after foo?"}, "output": "bar", "metrics": {"tokens": 1}}. In the webapp, only the root span row "abc" will show up in the summary view. You can view the full trace hierarchy (in this case, the "llm_call" row) by clicking on the "abc" row.

  If the row is being merged into an existing row, this field will be ignored.

• _parentId

   final InsertExperimentEvent.Builder _parentId(Optional<String> _parentId)

  Alias for calling Builder._parentId with _parentId.orElse(null).

• _parentId

   final InsertExperimentEvent.Builder _parentId(JsonField<String> _parentId)

  Sets Builder._parentId to an arbitrary JSON value.

  You should usually call Builder._parentId with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• context

   final InsertExperimentEvent.Builder context(InsertExperimentEvent.Context context)

  Context is additional information about the code that produced the experiment event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the experiment event

• context

   final InsertExperimentEvent.Builder context(Optional<InsertExperimentEvent.Context> context)

  Alias for calling Builder.context with context.orElse(null).

• context

   final InsertExperimentEvent.Builder context(JsonField<InsertExperimentEvent.Context> context)

  Sets Builder.context to an arbitrary JSON value.

  You should usually call Builder.context with a well-typed Context value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• created

   final InsertExperimentEvent.Builder created(OffsetDateTime created)

  The timestamp the experiment event was created

• created

   final InsertExperimentEvent.Builder created(Optional<OffsetDateTime> created)

  Alias for calling Builder.created with created.orElse(null).

• created

   final InsertExperimentEvent.Builder created(JsonField<OffsetDateTime> created)

  Sets Builder.created to an arbitrary JSON value.

  You should usually call Builder.created with a well-typed OffsetDateTime value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• error

   final InsertExperimentEvent.Builder error(JsonValue error)

  The error that occurred, if any.

• expected

   final InsertExperimentEvent.Builder expected(JsonValue expected)

  The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate your experiments while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models

• input

   final InsertExperimentEvent.Builder input(JsonValue input)

  The arguments that uniquely define a test case (an arbitrary, JSON serializable object). Later on, Braintrust will use the input to know whether two test cases are the same between experiments, so they should not contain experiment-specific state. A simple rule of thumb is that if you run the same experiment twice, the input should be identical

• metadata

   final InsertExperimentEvent.Builder metadata(InsertExperimentEvent.Metadata metadata)

  A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings

• metadata

   final InsertExperimentEvent.Builder metadata(Optional<InsertExperimentEvent.Metadata> metadata)

  Alias for calling Builder.metadata with metadata.orElse(null).

• metadata

   final InsertExperimentEvent.Builder metadata(JsonField<InsertExperimentEvent.Metadata> metadata)

  Sets Builder.metadata to an arbitrary JSON value.

  You should usually call Builder.metadata with a well-typed Metadata value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• metrics

   final InsertExperimentEvent.Builder metrics(InsertExperimentEvent.Metrics metrics)

  Metrics are numerical measurements tracking the execution of the code that produced the experiment event. Use "start" and "end" to track the time span over which the experiment event was produced

• metrics

   final InsertExperimentEvent.Builder metrics(Optional<InsertExperimentEvent.Metrics> metrics)

  Alias for calling Builder.metrics with metrics.orElse(null).

• metrics

   final InsertExperimentEvent.Builder metrics(JsonField<InsertExperimentEvent.Metrics> metrics)

  Sets Builder.metrics to an arbitrary JSON value.

  You should usually call Builder.metrics with a well-typed Metrics value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• origin

   final InsertExperimentEvent.Builder origin(ObjectReference origin)

  Indicates the event was copied from another object.

• origin

   final InsertExperimentEvent.Builder origin(Optional<ObjectReference> origin)

  Alias for calling Builder.origin with origin.orElse(null).

• origin

   final InsertExperimentEvent.Builder origin(JsonField<ObjectReference> origin)

  Sets Builder.origin to an arbitrary JSON value.

  You should usually call Builder.origin with a well-typed ObjectReference value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• output

   final InsertExperimentEvent.Builder output(JsonValue output)

  The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question

• rootSpanId

   final InsertExperimentEvent.Builder rootSpanId(String rootSpanId)

  Use span_id, root_span_id, and span_parents instead of _parent_id, which is now deprecated. The span_id is a unique identifier describing the row's place in the a trace, and the root_span_id is a unique identifier for the whole trace. See the guide for full details.

  For example, say we have logged a row {"id": "abc", "span_id": "span0", "root_span_id": "root_span0", "input": "foo", "output": "bar", "expected": "boo", "scores": {"correctness": 0.33}}. We can create a sub-span of the parent row by logging {"id": "llm_call", "span_id": "span1", "root_span_id": "root_span0", "span_parents": ["span0"], "input": {"prompt": "What comes after foo?"}, "output": "bar", "metrics": {"tokens": 1}}. In the webapp, only the root span row "abc" will show up in the summary view. You can view the full trace hierarchy (in this case, the "llm_call" row) by clicking on the "abc" row.

  If the row is being merged into an existing row, this field will be ignored.

• rootSpanId

   final InsertExperimentEvent.Builder rootSpanId(Optional<String> rootSpanId)

  Alias for calling Builder.rootSpanId with rootSpanId.orElse(null).

• rootSpanId

   final InsertExperimentEvent.Builder rootSpanId(JsonField<String> rootSpanId)

  Sets Builder.rootSpanId to an arbitrary JSON value.

  You should usually call Builder.rootSpanId with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• scores

   final InsertExperimentEvent.Builder scores(InsertExperimentEvent.Scores scores)

  A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare experiments

• scores

   final InsertExperimentEvent.Builder scores(Optional<InsertExperimentEvent.Scores> scores)

  Alias for calling Builder.scores with scores.orElse(null).

• scores

   final InsertExperimentEvent.Builder scores(JsonField<InsertExperimentEvent.Scores> scores)

  Sets Builder.scores to an arbitrary JSON value.

  You should usually call Builder.scores with a well-typed Scores value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• spanAttributes

   final InsertExperimentEvent.Builder spanAttributes(SpanAttributes spanAttributes)

  Human-identifying attributes of the span, such as name, type, etc.

• spanAttributes

   final InsertExperimentEvent.Builder spanAttributes(Optional<SpanAttributes> spanAttributes)

  Alias for calling Builder.spanAttributes with spanAttributes.orElse(null).

• spanAttributes

   final InsertExperimentEvent.Builder spanAttributes(JsonField<SpanAttributes> spanAttributes)

  Sets Builder.spanAttributes to an arbitrary JSON value.

  You should usually call Builder.spanAttributes with a well-typed SpanAttributes value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• spanId

   final InsertExperimentEvent.Builder spanId(String spanId)

  Use span_id, root_span_id, and span_parents instead of _parent_id, which is now deprecated. The span_id is a unique identifier describing the row's place in the a trace, and the root_span_id is a unique identifier for the whole trace. See the guide for full details.

  For example, say we have logged a row {"id": "abc", "span_id": "span0", "root_span_id": "root_span0", "input": "foo", "output": "bar", "expected": "boo", "scores": {"correctness": 0.33}}. We can create a sub-span of the parent row by logging {"id": "llm_call", "span_id": "span1", "root_span_id": "root_span0", "span_parents": ["span0"], "input": {"prompt": "What comes after foo?"}, "output": "bar", "metrics": {"tokens": 1}}. In the webapp, only the root span row "abc" will show up in the summary view. You can view the full trace hierarchy (in this case, the "llm_call" row) by clicking on the "abc" row.

  If the row is being merged into an existing row, this field will be ignored.

• spanId

   final InsertExperimentEvent.Builder spanId(Optional<String> spanId)

  Alias for calling Builder.spanId with spanId.orElse(null).

• spanId

   final InsertExperimentEvent.Builder spanId(JsonField<String> spanId)

  Sets Builder.spanId to an arbitrary JSON value.

  You should usually call Builder.spanId with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• spanParents

   final InsertExperimentEvent.Builder spanParents(List<String> spanParents)

  Use span_id, root_span_id, and span_parents instead of _parent_id, which is now deprecated. The span_id is a unique identifier describing the row's place in the a trace, and the root_span_id is a unique identifier for the whole trace. See the guide for full details.

  For example, say we have logged a row {"id": "abc", "span_id": "span0", "root_span_id": "root_span0", "input": "foo", "output": "bar", "expected": "boo", "scores": {"correctness": 0.33}}. We can create a sub-span of the parent row by logging {"id": "llm_call", "span_id": "span1", "root_span_id": "root_span0", "span_parents": ["span0"], "input": {"prompt": "What comes after foo?"}, "output": "bar", "metrics": {"tokens": 1}}. In the webapp, only the root span row "abc" will show up in the summary view. You can view the full trace hierarchy (in this case, the "llm_call" row) by clicking on the "abc" row.

  If the row is being merged into an existing row, this field will be ignored.

• spanParents

   final InsertExperimentEvent.Builder spanParents(Optional<List<String>> spanParents)

  Alias for calling Builder.spanParents with spanParents.orElse(null).

• spanParents

   final InsertExperimentEvent.Builder spanParents(JsonField<List<String>> spanParents)

  Sets Builder.spanParents to an arbitrary JSON value.

  You should usually call Builder.spanParents with a well-typed List<String> value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• addSpanParent

   final InsertExperimentEvent.Builder addSpanParent(String spanParent)

  Adds a single String to spanParents.

• tags

   final InsertExperimentEvent.Builder tags(List<String> tags)

  A list of tags to log

• tags

   final InsertExperimentEvent.Builder tags(Optional<List<String>> tags)

  Alias for calling Builder.tags with tags.orElse(null).

• tags

   final InsertExperimentEvent.Builder tags(JsonField<List<String>> tags)

  Sets Builder.tags to an arbitrary JSON value.

  You should usually call Builder.tags with a well-typed List<String> value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• addTag

   final InsertExperimentEvent.Builder addTag(String tag)

  Adds a single String to tags.

• additionalProperties

   final InsertExperimentEvent.Builder additionalProperties(Map<String, JsonValue> additionalProperties)

• putAdditionalProperty

   final InsertExperimentEvent.Builder putAdditionalProperty(String key, JsonValue value)

• putAllAdditionalProperties

   final InsertExperimentEvent.Builder putAllAdditionalProperties(Map<String, JsonValue> additionalProperties)

• removeAdditionalProperty

   final InsertExperimentEvent.Builder removeAdditionalProperty(String key)

• removeAllAdditionalProperties

   final InsertExperimentEvent.Builder removeAllAdditionalProperties(Set<String> keys)

• build

   final InsertExperimentEvent build()

  Returns an immutable instance of InsertExperimentEvent.

  Further updates to this Builder will not mutate the returned instance.