Constructor Detail

Method Detail

• body

   final EvalCreateParams.Builder body(EvalCreateParams.Body body)

  Sets the entire request body.

  This is generally only useful if you are already constructing the body separately. Otherwise, it's more convenient to use the top-level setters instead:

  • data

  • projectId

  • scores

  • task

  • baseExperimentId

  • etc.

• data

   final EvalCreateParams.Builder data(EvalCreateParams.Data data)

  The dataset to use

• data

   final EvalCreateParams.Builder data(JsonField<EvalCreateParams.Data> data)

  Sets Builder.data to an arbitrary JSON value.

  You should usually call Builder.data with a well-typed Data value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• data

   final EvalCreateParams.Builder data(EvalCreateParams.Data.DatasetId datasetId)

  Alias for calling data with Data.ofDatasetId(datasetId).

• data

   final EvalCreateParams.Builder data(EvalCreateParams.Data.ProjectDatasetName projectDatasetName)

  Alias for calling data with Data.ofProjectDatasetName(projectDatasetName).

• data

   final EvalCreateParams.Builder data(EvalCreateParams.Data.DatasetRows datasetRows)

  Alias for calling data with Data.ofDatasetRows(datasetRows).

• projectId

   final EvalCreateParams.Builder projectId(String projectId)

  Unique identifier for the project to run the eval in

• projectId

   final EvalCreateParams.Builder projectId(JsonField<String> projectId)

  Sets Builder.projectId to an arbitrary JSON value.

  You should usually call Builder.projectId with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• scores

   final EvalCreateParams.Builder scores(List<EvalCreateParams.Score> scores)

  The functions to score the eval on

• scores

   final EvalCreateParams.Builder scores(JsonField<List<EvalCreateParams.Score>> scores)

  Sets Builder.scores to an arbitrary JSON value.

  You should usually call Builder.scores with a well-typed List<Score> value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• addScore

   final EvalCreateParams.Builder addScore(EvalCreateParams.Score score)

  Adds a single Score to scores.

• addScore

   final EvalCreateParams.Builder addScore(EvalCreateParams.Score.FunctionId functionId)

  Alias for calling addScore with Score.ofFunctionId(functionId).

• addScore

   final EvalCreateParams.Builder addScore(EvalCreateParams.Score.ProjectSlug projectSlug)

  Alias for calling addScore with Score.ofProjectSlug(projectSlug).

• addScore

   final EvalCreateParams.Builder addScore(EvalCreateParams.Score.GlobalFunction globalFunction)

  Alias for calling addScore with Score.ofGlobalFunction(globalFunction).

• addScore

   final EvalCreateParams.Builder addScore(EvalCreateParams.Score.PromptSessionId promptSessionId)

  Alias for calling addScore with Score.ofPromptSessionId(promptSessionId).

• addScore

   final EvalCreateParams.Builder addScore(EvalCreateParams.Score.InlineCode inlineCode)

  Alias for calling addScore with Score.ofInlineCode(inlineCode).

• addScore

   final EvalCreateParams.Builder addScore(EvalCreateParams.Score.InlinePrompt inlinePrompt)

  Alias for calling addScore with Score.ofInlinePrompt(inlinePrompt).

• task

   final EvalCreateParams.Builder task(EvalCreateParams.Task task)

  The function to evaluate

• task

   final EvalCreateParams.Builder task(JsonField<EvalCreateParams.Task> task)

  Sets Builder.task to an arbitrary JSON value.

  You should usually call Builder.task with a well-typed Task value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• task

   final EvalCreateParams.Builder task(EvalCreateParams.Task.FunctionId functionId)

  Alias for calling task with Task.ofFunctionId(functionId).

• task

   final EvalCreateParams.Builder task(EvalCreateParams.Task.ProjectSlug projectSlug)

  Alias for calling task with Task.ofProjectSlug(projectSlug).

• task

   final EvalCreateParams.Builder task(EvalCreateParams.Task.GlobalFunction globalFunction)

  Alias for calling task with Task.ofGlobalFunction(globalFunction).

• task

   final EvalCreateParams.Builder task(EvalCreateParams.Task.PromptSessionId promptSessionId)

  Alias for calling task with Task.ofPromptSessionId(promptSessionId).

• task

   final EvalCreateParams.Builder task(EvalCreateParams.Task.InlineCode inlineCode)

  Alias for calling task with Task.ofInlineCode(inlineCode).

• task

   final EvalCreateParams.Builder task(EvalCreateParams.Task.InlinePrompt inlinePrompt)

  Alias for calling task with Task.ofInlinePrompt(inlinePrompt).

• baseExperimentId

   final EvalCreateParams.Builder baseExperimentId(String baseExperimentId)

  An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

• baseExperimentId

   final EvalCreateParams.Builder baseExperimentId(Optional<String> baseExperimentId)

  Alias for calling Builder.baseExperimentId with baseExperimentId.orElse(null).

• baseExperimentId

   final EvalCreateParams.Builder baseExperimentId(JsonField<String> baseExperimentId)

  Sets Builder.baseExperimentId to an arbitrary JSON value.

  You should usually call Builder.baseExperimentId with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• baseExperimentName

   final EvalCreateParams.Builder baseExperimentName(String baseExperimentName)

  An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

• baseExperimentName

   final EvalCreateParams.Builder baseExperimentName(Optional<String> baseExperimentName)

  Alias for calling Builder.baseExperimentName with baseExperimentName.orElse(null).

• baseExperimentName

   final EvalCreateParams.Builder baseExperimentName(JsonField<String> baseExperimentName)

  Sets Builder.baseExperimentName to an arbitrary JSON value.

  You should usually call Builder.baseExperimentName with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• experimentName

   final EvalCreateParams.Builder experimentName(String experimentName)

  An optional name for the experiment created by this eval. If it conflicts with an existing experiment, it will be suffixed with a unique identifier.

• experimentName

   final EvalCreateParams.Builder experimentName(JsonField<String> experimentName)

  Sets Builder.experimentName to an arbitrary JSON value.

  You should usually call Builder.experimentName with a well-typed String value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• gitMetadataSettings

   final EvalCreateParams.Builder gitMetadataSettings(EvalCreateParams.GitMetadataSettings gitMetadataSettings)

  Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.

• gitMetadataSettings

   final EvalCreateParams.Builder gitMetadataSettings(Optional<EvalCreateParams.GitMetadataSettings> gitMetadataSettings)

  Alias for calling Builder.gitMetadataSettings with gitMetadataSettings.orElse(null).

• gitMetadataSettings

   final EvalCreateParams.Builder gitMetadataSettings(JsonField<EvalCreateParams.GitMetadataSettings> gitMetadataSettings)

  Sets Builder.gitMetadataSettings to an arbitrary JSON value.

  You should usually call Builder.gitMetadataSettings with a well-typed GitMetadataSettings value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• isPublic

   final EvalCreateParams.Builder isPublic(Boolean isPublic)

  Whether the experiment should be public. Defaults to false.

• isPublic

   final EvalCreateParams.Builder isPublic(Boolean isPublic)

  Alias for Builder.isPublic.

  This unboxed primitive overload exists for backwards compatibility.

• isPublic

   final EvalCreateParams.Builder isPublic(Optional<Boolean> isPublic)

  Alias for calling Builder.isPublic with isPublic.orElse(null).

• isPublic

   final EvalCreateParams.Builder isPublic(JsonField<Boolean> isPublic)

  Sets Builder.isPublic to an arbitrary JSON value.

  You should usually call Builder.isPublic with a well-typed Boolean value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• maxConcurrency

   final EvalCreateParams.Builder maxConcurrency(Double maxConcurrency)

  The maximum number of tasks/scorers that will be run concurrently. Defaults to undefined, in which case there is no max concurrency.

• maxConcurrency

   final EvalCreateParams.Builder maxConcurrency(Double maxConcurrency)

  Alias for Builder.maxConcurrency.

  This unboxed primitive overload exists for backwards compatibility.

• maxConcurrency

   final EvalCreateParams.Builder maxConcurrency(Optional<Double> maxConcurrency)

  Alias for calling Builder.maxConcurrency with maxConcurrency.orElse(null).

• maxConcurrency

   final EvalCreateParams.Builder maxConcurrency(JsonField<Double> maxConcurrency)

  Sets Builder.maxConcurrency to an arbitrary JSON value.

  You should usually call Builder.maxConcurrency with a well-typed Double value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• metadata

   final EvalCreateParams.Builder metadata(EvalCreateParams.Metadata metadata)

  Optional experiment-level metadata to store about the evaluation. You can later use this to slice & dice across experiments.

• metadata

   final EvalCreateParams.Builder metadata(JsonField<EvalCreateParams.Metadata> metadata)

  Sets Builder.metadata to an arbitrary JSON value.

  You should usually call Builder.metadata with a well-typed Metadata value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• parent

   final EvalCreateParams.Builder parent(EvalCreateParams.Parent parent)

  Options for tracing the evaluation

• parent

   final EvalCreateParams.Builder parent(JsonField<EvalCreateParams.Parent> parent)

  Sets Builder.parent to an arbitrary JSON value.

  You should usually call Builder.parent with a well-typed Parent value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• parent

   final EvalCreateParams.Builder parent(EvalCreateParams.Parent.SpanParentStruct spanParentStruct)

  Alias for calling parent with Parent.ofSpanParentStruct(spanParentStruct).

• parent

   final EvalCreateParams.Builder parent(String string)

  Alias for calling parent with Parent.ofString(string).

• repoInfo

   final EvalCreateParams.Builder repoInfo(RepoInfo repoInfo)

  Metadata about the state of the repo when the experiment was created

• repoInfo

   final EvalCreateParams.Builder repoInfo(Optional<RepoInfo> repoInfo)

  Alias for calling Builder.repoInfo with repoInfo.orElse(null).

• repoInfo

   final EvalCreateParams.Builder repoInfo(JsonField<RepoInfo> repoInfo)

  Sets Builder.repoInfo to an arbitrary JSON value.

  You should usually call Builder.repoInfo with a well-typed RepoInfo value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• stream

   final EvalCreateParams.Builder stream(Boolean stream)

  Whether to stream the results of the eval. If true, the request will return two events: one to indicate the experiment has started, and another upon completion. If false, the request will return the evaluation's summary upon completion.

• stream

   final EvalCreateParams.Builder stream(JsonField<Boolean> stream)

  Sets Builder.stream to an arbitrary JSON value.

  You should usually call Builder.stream with a well-typed Boolean value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• timeout

   final EvalCreateParams.Builder timeout(Double timeout)

  The maximum duration, in milliseconds, to run the evaluation. Defaults to undefined, in which case there is no timeout.

• timeout

   final EvalCreateParams.Builder timeout(Double timeout)

  Alias for Builder.timeout.

  This unboxed primitive overload exists for backwards compatibility.

• timeout

   final EvalCreateParams.Builder timeout(Optional<Double> timeout)

  Alias for calling Builder.timeout with timeout.orElse(null).

• timeout

   final EvalCreateParams.Builder timeout(JsonField<Double> timeout)

  Sets Builder.timeout to an arbitrary JSON value.

  You should usually call Builder.timeout with a well-typed Double value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• trialCount

   final EvalCreateParams.Builder trialCount(Double trialCount)

  The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.

• trialCount

   final EvalCreateParams.Builder trialCount(Double trialCount)

  Alias for Builder.trialCount.

  This unboxed primitive overload exists for backwards compatibility.

• trialCount

   final EvalCreateParams.Builder trialCount(Optional<Double> trialCount)

  Alias for calling Builder.trialCount with trialCount.orElse(null).

• trialCount

   final EvalCreateParams.Builder trialCount(JsonField<Double> trialCount)

  Sets Builder.trialCount to an arbitrary JSON value.

  You should usually call Builder.trialCount with a well-typed Double value instead. This method is primarily for setting the field to an undocumented or not yet supported value.

• additionalBodyProperties

   final EvalCreateParams.Builder additionalBodyProperties(Map<String, JsonValue> additionalBodyProperties)

• putAdditionalBodyProperty

   final EvalCreateParams.Builder putAdditionalBodyProperty(String key, JsonValue value)

• putAllAdditionalBodyProperties

   final EvalCreateParams.Builder putAllAdditionalBodyProperties(Map<String, JsonValue> additionalBodyProperties)

• removeAdditionalBodyProperty

   final EvalCreateParams.Builder removeAdditionalBodyProperty(String key)

• removeAllAdditionalBodyProperties

   final EvalCreateParams.Builder removeAllAdditionalBodyProperties(Set<String> keys)

• additionalHeaders

   final EvalCreateParams.Builder additionalHeaders(Headers additionalHeaders)

• additionalHeaders

   final EvalCreateParams.Builder additionalHeaders(Map<String, Iterable<String>> additionalHeaders)

• putAdditionalHeader

   final EvalCreateParams.Builder putAdditionalHeader(String name, String value)

• putAdditionalHeaders

   final EvalCreateParams.Builder putAdditionalHeaders(String name, Iterable<String> values)

• putAllAdditionalHeaders

   final EvalCreateParams.Builder putAllAdditionalHeaders(Headers additionalHeaders)

• putAllAdditionalHeaders

   final EvalCreateParams.Builder putAllAdditionalHeaders(Map<String, Iterable<String>> additionalHeaders)

• replaceAdditionalHeaders

   final EvalCreateParams.Builder replaceAdditionalHeaders(String name, String value)

• replaceAdditionalHeaders

   final EvalCreateParams.Builder replaceAdditionalHeaders(String name, Iterable<String> values)

• replaceAllAdditionalHeaders

   final EvalCreateParams.Builder replaceAllAdditionalHeaders(Headers additionalHeaders)

• replaceAllAdditionalHeaders

   final EvalCreateParams.Builder replaceAllAdditionalHeaders(Map<String, Iterable<String>> additionalHeaders)

• removeAdditionalHeaders

   final EvalCreateParams.Builder removeAdditionalHeaders(String name)

• removeAllAdditionalHeaders

   final EvalCreateParams.Builder removeAllAdditionalHeaders(Set<String> names)

• additionalQueryParams

   final EvalCreateParams.Builder additionalQueryParams(QueryParams additionalQueryParams)

• additionalQueryParams

   final EvalCreateParams.Builder additionalQueryParams(Map<String, Iterable<String>> additionalQueryParams)

• putAdditionalQueryParam

   final EvalCreateParams.Builder putAdditionalQueryParam(String key, String value)

• putAdditionalQueryParams

   final EvalCreateParams.Builder putAdditionalQueryParams(String key, Iterable<String> values)

• putAllAdditionalQueryParams

   final EvalCreateParams.Builder putAllAdditionalQueryParams(QueryParams additionalQueryParams)

• putAllAdditionalQueryParams

   final EvalCreateParams.Builder putAllAdditionalQueryParams(Map<String, Iterable<String>> additionalQueryParams)

• replaceAdditionalQueryParams

   final EvalCreateParams.Builder replaceAdditionalQueryParams(String key, String value)

• replaceAdditionalQueryParams

   final EvalCreateParams.Builder replaceAdditionalQueryParams(String key, Iterable<String> values)

• replaceAllAdditionalQueryParams

   final EvalCreateParams.Builder replaceAllAdditionalQueryParams(QueryParams additionalQueryParams)

• replaceAllAdditionalQueryParams

   final EvalCreateParams.Builder replaceAllAdditionalQueryParams(Map<String, Iterable<String>> additionalQueryParams)

• removeAdditionalQueryParams

   final EvalCreateParams.Builder removeAdditionalQueryParams(String key)

• removeAllAdditionalQueryParams

   final EvalCreateParams.Builder removeAllAdditionalQueryParams(Set<String> keys)

• build

   final EvalCreateParams build()

  Returns an immutable instance of EvalCreateParams.

  Further updates to this Builder will not mutate the returned instance.

  The following fields are required:

  .data()
  .projectId()
  .scores()
  .task()