Constructor Detail

Method Detail

• data

   final EvalCreateParams.Data data()

  The dataset to use

• projectId

   final String projectId()

  Unique identifier for the project to run the eval in

• scores

   final List<EvalCreateParams.Score> scores()

  The functions to score the eval on

• task

   final EvalCreateParams.Task task()

  The function to evaluate

• baseExperimentId

   final Optional<String> baseExperimentId()

  An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

• baseExperimentName

   final Optional<String> baseExperimentName()

  An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

• experimentName

   final Optional<String> experimentName()

  An optional name for the experiment created by this eval. If it conflicts with an existing experiment, it will be suffixed with a unique identifier.

• gitMetadataSettings

   final Optional<EvalCreateParams.GitMetadataSettings> gitMetadataSettings()

  Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.

• isPublic

   final Optional<Boolean> isPublic()

  Whether the experiment should be public. Defaults to false.

• maxConcurrency

   final Optional<Double> maxConcurrency()

  The maximum number of tasks/scorers that will be run concurrently. Defaults to undefined, in which case there is no max concurrency.

• metadata

   final Optional<EvalCreateParams.Metadata> metadata()

  Optional experiment-level metadata to store about the evaluation. You can later use this to slice & dice across experiments.

• parent

   final Optional<EvalCreateParams.Parent> parent()

  Options for tracing the evaluation

• repoInfo

   final Optional<RepoInfo> repoInfo()

  Metadata about the state of the repo when the experiment was created

• stream

   final Optional<Boolean> stream()

  Whether to stream the results of the eval. If true, the request will return two events: one to indicate the experiment has started, and another upon completion. If false, the request will return the evaluation's summary upon completion.

• timeout

   final Optional<Double> timeout()

  The maximum duration, in milliseconds, to run the evaluation. Defaults to undefined, in which case there is no timeout.

• trialCount

   final Optional<Double> trialCount()

  The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.

• _data

   final JsonField<EvalCreateParams.Data> _data()

  Returns the raw JSON value of data.

  Unlike data, this method doesn't throw if the JSON field has an unexpected type.

• _projectId

   final JsonField<String> _projectId()

  Returns the raw JSON value of projectId.

  Unlike projectId, this method doesn't throw if the JSON field has an unexpected type.

• _scores

   final JsonField<List<EvalCreateParams.Score>> _scores()

  Returns the raw JSON value of scores.

  Unlike scores, this method doesn't throw if the JSON field has an unexpected type.

• _task

   final JsonField<EvalCreateParams.Task> _task()

  Returns the raw JSON value of task.

  Unlike task, this method doesn't throw if the JSON field has an unexpected type.

• _baseExperimentId

   final JsonField<String> _baseExperimentId()

  Returns the raw JSON value of baseExperimentId.

  Unlike baseExperimentId, this method doesn't throw if the JSON field has an unexpected type.

• _baseExperimentName

   final JsonField<String> _baseExperimentName()

  Returns the raw JSON value of baseExperimentName.

  Unlike baseExperimentName, this method doesn't throw if the JSON field has an unexpected type.

• _experimentName

   final JsonField<String> _experimentName()

  Returns the raw JSON value of experimentName.

  Unlike experimentName, this method doesn't throw if the JSON field has an unexpected type.

• _gitMetadataSettings

   final JsonField<EvalCreateParams.GitMetadataSettings> _gitMetadataSettings()

  Returns the raw JSON value of gitMetadataSettings.

  Unlike gitMetadataSettings, this method doesn't throw if the JSON field has an unexpected type.

• _isPublic

   final JsonField<Boolean> _isPublic()

  Returns the raw JSON value of isPublic.

  Unlike isPublic, this method doesn't throw if the JSON field has an unexpected type.

• _maxConcurrency

   final JsonField<Double> _maxConcurrency()

  Returns the raw JSON value of maxConcurrency.

  Unlike maxConcurrency, this method doesn't throw if the JSON field has an unexpected type.

• _metadata

   final JsonField<EvalCreateParams.Metadata> _metadata()

  Returns the raw JSON value of metadata.

  Unlike metadata, this method doesn't throw if the JSON field has an unexpected type.

• _parent

   final JsonField<EvalCreateParams.Parent> _parent()

  Returns the raw JSON value of parent.

  Unlike parent, this method doesn't throw if the JSON field has an unexpected type.

• _repoInfo

   final JsonField<RepoInfo> _repoInfo()

  Returns the raw JSON value of repoInfo.

  Unlike repoInfo, this method doesn't throw if the JSON field has an unexpected type.

• _stream

   final JsonField<Boolean> _stream()

  Returns the raw JSON value of stream.

  Unlike stream, this method doesn't throw if the JSON field has an unexpected type.

• _timeout

   final JsonField<Double> _timeout()

  Returns the raw JSON value of timeout.

  Unlike timeout, this method doesn't throw if the JSON field has an unexpected type.

• _trialCount

   final JsonField<Double> _trialCount()

  Returns the raw JSON value of trialCount.

  Unlike trialCount, this method doesn't throw if the JSON field has an unexpected type.

• _additionalProperties

   final Map<String, JsonValue> _additionalProperties()

• toBuilder

   final EvalCreateParams.Body.Builder toBuilder()

• validate

   final EvalCreateParams.Body validate()

• isValid

   final Boolean isValid()

• equals

   Boolean equals(Object other)

• hashCode

   Integer hashCode()

• toString

   String toString()

• builder

   final static EvalCreateParams.Body.Builder builder()

  Returns a mutable builder for constructing an instance of Body.

  The following fields are required:

  .data()
  .projectId()
  .scores()
  .task()