Options passed to the SageMaker client.
The content handler class that provides an input and output transform functions to handle formats between LLM and the endpoint.
The name of the endpoint from the deployed SageMaker model. Must be unique within an AWS Region.
Optional
cacheOptional
callbackUse callbacks
instead
Optional
callbacksOptional
concurrencyUse maxConcurrency
instead
Optional
endpointOptional attributes passed to the InvokeEndpointCommand
Optional
maxThe maximum number of concurrent calls that can be made.
Defaults to Infinity
, which means no limit.
Optional
maxThe maximum number of retries that can be made for a single call, with an exponential backoff between each attempt. Defaults to 6.
Optional
metadataOptional
modelKey word arguments to pass to the model.
Optional
onCustom handler to handle failed attempts. Takes the originally thrown error object as input, and should itself throw an error if the input error is not retryable.
Optional
streamingOptional
tagsOptional
verboseGenerated using TypeDoc
The SageMakerEndpointInput interface defines the input parameters for the SageMakerEndpoint class, which includes the endpoint name, client options for the SageMaker client, the content handler, and optional keyword arguments for the model and the endpoint.