com.databricks.sdk.service.serving

Class ServingEndpointsAPI

java.lang.Object

com.databricks.sdk.service.serving.ServingEndpointsAPI

@Generated
public class ServingEndpointsAPI
extends Object

The Serving Endpoints API allows you to create, update, and delete model serving endpoints.

You can use a serving endpoint to serve models from the Databricks Model Registry or from Unity Catalog. Endpoints expose the underlying models as scalable REST API endpoints using serverless compute. This means the endpoints and associated compute resources are fully managed by Databricks and will not appear in your cloud account. A serving endpoint can consist of one or more MLflow models from the Databricks Model Registry, called served entities. A serving endpoint can have at most ten served entities. You can configure traffic settings to define how requests should be routed to your served entities behind an endpoint. Additionally, you can configure the scale of resources that should be applied to each served entity.

Constructor Summary

Constructors

Constructor and Description
ServingEndpointsAPI(ApiClient apiClient) Regular-use constructor
ServingEndpointsAPI(ServingEndpointsService mock) Constructor for mocks

Method Summary

Modifier and Type	Method and Description
BuildLogsResponse	buildLogs(BuildLogsRequest request) Retrieves the build logs associated with the provided served model.
BuildLogsResponse	buildLogs(String name, String servedModelName)
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	create(CreateServingEndpoint request) Create a new serving endpoint.
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	create(String name)
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	createProvisionedThroughputEndpoint(CreatePtEndpointRequest request) Create a new PT serving endpoint.
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	createProvisionedThroughputEndpoint(String name, PtEndpointCoreConfig config)
void	delete(DeleteServingEndpointRequest request) Delete a serving endpoint.
void	delete(String name)
ExportMetricsResponse	exportMetrics(ExportMetricsRequest request) Retrieves the metrics associated with the provided serving endpoint in either Prometheus or OpenMetrics exposition format.
ExportMetricsResponse	exportMetrics(String name)
ServingEndpointDetailed	get(GetServingEndpointRequest request) Retrieves the details for a single serving endpoint.
ServingEndpointDetailed	get(String name)
GetOpenApiResponse	getOpenApi(GetOpenApiRequest request) Get the query schema of the serving endpoint in OpenAPI format.
GetOpenApiResponse	getOpenApi(String name)
GetServingEndpointPermissionLevelsResponse	getPermissionLevels(GetServingEndpointPermissionLevelsRequest request) Gets the permission levels that a user can have on an object.
GetServingEndpointPermissionLevelsResponse	getPermissionLevels(String servingEndpointId)
ServingEndpointPermissions	getPermissions(GetServingEndpointPermissionsRequest request) Gets the permissions of a serving endpoint.
ServingEndpointPermissions	getPermissions(String servingEndpointId)
HttpRequestResponse	httpRequest(ExternalFunctionRequest request) Make external services call using the credentials stored in UC Connection.
HttpRequestResponse	httpRequest(String connectionName, ExternalFunctionRequestHttpMethod method, String path)
ServingEndpointsService	impl()
Iterable<ServingEndpoint>	list() Get all serving endpoints.
ServerLogsResponse	logs(LogsRequest request) Retrieves the service logs associated with the provided served model.
ServerLogsResponse	logs(String name, String servedModelName)
EndpointTags	patch(PatchServingEndpointTags request) Used to batch add and delete tags from a serving endpoint with a single API call.
EndpointTags	patch(String name)
PutResponse	put(PutRequest request) Deprecated: Please use AI Gateway to manage rate limits instead.
PutResponse	put(String name)
PutAiGatewayResponse	putAiGateway(PutAiGatewayRequest request) Used to update the AI Gateway of a serving endpoint.
PutAiGatewayResponse	putAiGateway(String name)
QueryEndpointResponse	query(QueryEndpointInput request) Query a serving endpoint.
QueryEndpointResponse	query(String name)
ServingEndpointPermissions	setPermissions(ServingEndpointPermissionsRequest request) Sets permissions on an object, replacing existing permissions if they exist.
ServingEndpointPermissions	setPermissions(String servingEndpointId)
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	updateConfig(EndpointCoreConfigInput request) Updates any combination of the serving endpoint's served entities, the compute configuration of those served entities, and the endpoint's traffic config.
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	updateConfig(String name)
ServingEndpointPermissions	updatePermissions(ServingEndpointPermissionsRequest request) Updates the permissions on a serving endpoint.
ServingEndpointPermissions	updatePermissions(String servingEndpointId)
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	updateProvisionedThroughputEndpointConfig(String name, PtEndpointCoreConfig config)
Wait<ServingEndpointDetailed,ServingEndpointDetailed>	updateProvisionedThroughputEndpointConfig(UpdateProvisionedThroughputEndpointConfigRequest request) Updates any combination of the pt endpoint's served entities, the compute configuration of those served entities, and the endpoint's traffic config.
ServingEndpointDetailed	waitGetServingEndpointNotUpdating(String name)
ServingEndpointDetailed	waitGetServingEndpointNotUpdating(String name, Duration timeout, Consumer<ServingEndpointDetailed> callback)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ServingEndpointsAPI

public ServingEndpointsAPI(ApiClient apiClient)

Regular-use constructor

ServingEndpointsAPI

public ServingEndpointsAPI(ServingEndpointsService mock)

Constructor for mocks

Method Detail

waitGetServingEndpointNotUpdating

public ServingEndpointDetailed waitGetServingEndpointNotUpdating(String name)
                                                          throws TimeoutException

Throws:

TimeoutException

waitGetServingEndpointNotUpdating

public ServingEndpointDetailed waitGetServingEndpointNotUpdating(String name,
                                                                 Duration timeout,
                                                                 Consumer<ServingEndpointDetailed> callback)
                                                          throws TimeoutException

Throws:

TimeoutException

buildLogs

public BuildLogsResponse buildLogs(String name,
                                   String servedModelName)

buildLogs

public BuildLogsResponse buildLogs(BuildLogsRequest request)

Retrieves the build logs associated with the provided served model.

create

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> create(String name)

create

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> create(CreateServingEndpoint request)

Create a new serving endpoint.

createProvisionedThroughputEndpoint

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> createProvisionedThroughputEndpoint(String name,
                                                                                                 PtEndpointCoreConfig config)

createProvisionedThroughputEndpoint

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> createProvisionedThroughputEndpoint(CreatePtEndpointRequest request)

Create a new PT serving endpoint.

delete

public void delete(String name)

delete

public void delete(DeleteServingEndpointRequest request)

Delete a serving endpoint.

exportMetrics

public ExportMetricsResponse exportMetrics(String name)

exportMetrics

public ExportMetricsResponse exportMetrics(ExportMetricsRequest request)

Retrieves the metrics associated with the provided serving endpoint in either Prometheus or OpenMetrics exposition format.

get

public ServingEndpointDetailed get(String name)

get

public ServingEndpointDetailed get(GetServingEndpointRequest request)

Retrieves the details for a single serving endpoint.

getOpenApi

public GetOpenApiResponse getOpenApi(String name)

getOpenApi

public GetOpenApiResponse getOpenApi(GetOpenApiRequest request)

Get the query schema of the serving endpoint in OpenAPI format. The schema contains information for the supported paths, input and output format and datatypes.

getPermissionLevels

public GetServingEndpointPermissionLevelsResponse getPermissionLevels(String servingEndpointId)

getPermissionLevels

public GetServingEndpointPermissionLevelsResponse getPermissionLevels(GetServingEndpointPermissionLevelsRequest request)

Gets the permission levels that a user can have on an object.

getPermissions

public ServingEndpointPermissions getPermissions(String servingEndpointId)

getPermissions

public ServingEndpointPermissions getPermissions(GetServingEndpointPermissionsRequest request)

Gets the permissions of a serving endpoint. Serving endpoints can inherit permissions from their root object.

httpRequest

public HttpRequestResponse httpRequest(String connectionName,
                                       ExternalFunctionRequestHttpMethod method,
                                       String path)

httpRequest

public HttpRequestResponse httpRequest(ExternalFunctionRequest request)

Make external services call using the credentials stored in UC Connection.

list

public Iterable<ServingEndpoint> list()

Get all serving endpoints.

logs

public ServerLogsResponse logs(String name,
                               String servedModelName)

logs

public ServerLogsResponse logs(LogsRequest request)

Retrieves the service logs associated with the provided served model.

patch

public EndpointTags patch(String name)

patch

public EndpointTags patch(PatchServingEndpointTags request)

Used to batch add and delete tags from a serving endpoint with a single API call.

put

public PutResponse put(String name)

put

public PutResponse put(PutRequest request)

Deprecated: Please use AI Gateway to manage rate limits instead.

putAiGateway

public PutAiGatewayResponse putAiGateway(String name)

putAiGateway

public PutAiGatewayResponse putAiGateway(PutAiGatewayRequest request)

Used to update the AI Gateway of a serving endpoint. NOTE: External model, provisioned throughput, and pay-per-token endpoints are fully supported; agent endpoints currently only support inference tables.

query

public QueryEndpointResponse query(String name)

query

public QueryEndpointResponse query(QueryEndpointInput request)

Query a serving endpoint.

setPermissions

public ServingEndpointPermissions setPermissions(String servingEndpointId)

setPermissions

public ServingEndpointPermissions setPermissions(ServingEndpointPermissionsRequest request)

Sets permissions on an object, replacing existing permissions if they exist. Deletes all direct permissions if none are specified. Objects can inherit permissions from their root object.

updateConfig

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> updateConfig(String name)

updateConfig

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> updateConfig(EndpointCoreConfigInput request)

Updates any combination of the serving endpoint's served entities, the compute configuration of those served entities, and the endpoint's traffic config. An endpoint that already has an update in progress can not be updated until the current update completes or fails.

updatePermissions

public ServingEndpointPermissions updatePermissions(String servingEndpointId)

updatePermissions

public ServingEndpointPermissions updatePermissions(ServingEndpointPermissionsRequest request)

Updates the permissions on a serving endpoint. Serving endpoints can inherit permissions from their root object.

updateProvisionedThroughputEndpointConfig

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> updateProvisionedThroughputEndpointConfig(String name,
                                                                                                       PtEndpointCoreConfig config)

updateProvisionedThroughputEndpointConfig

public Wait<ServingEndpointDetailed,ServingEndpointDetailed> updateProvisionedThroughputEndpointConfig(UpdateProvisionedThroughputEndpointConfigRequest request)

Updates any combination of the pt endpoint's served entities, the compute configuration of those served entities, and the endpoint's traffic config. Updates are instantaneous and endpoint should be updated instantly

impl

public ServingEndpointsService impl()