Return an endpoint from CloudPredictor.deploy()

README.md

@@ -76,7 +76,7 @@ cloud_predictor.fit(
 result = cloud_predictor.predict(test_data)
 
 # Real-time inference endpoint
-cloud_predictor.deploy()
-result = cloud_predictor.predict_real_time(test_data)
-cloud_predictor.cleanup_deployment()
+endpoint = cloud_predictor.deploy()
+result = endpoint.predict(test_data)
+endpoint.delete_endpoint()
 ```

docs/index.md

@@ -109,9 +109,9 @@ cloud_predictor.fit(
 result = cloud_predictor.predict(test_data)
 
 # Real-time inference endpoint
-cloud_predictor.deploy()
-result = cloud_predictor.predict_real_time(test_data)
-cloud_predictor.cleanup_deployment()
+endpoint = cloud_predictor.deploy()
+result = endpoint.predict(test_data)
+endpoint.delete_endpoint()
 ```
 
 → [Full walkthrough](tutorials/predictor-tabular.md)
@@ -142,9 +142,9 @@ cloud_predictor.fit(
 result = cloud_predictor.predict(data)
 
 # Real-time inference endpoint
-cloud_predictor.deploy()
-result = cloud_predictor.predict_real_time(data)
-cloud_predictor.cleanup_deployment()
+endpoint = cloud_predictor.deploy()
+result = endpoint.predict(data)
+endpoint.delete_endpoint()
 ```
 
 → [Full walkthrough](tutorials/predictor-timeseries.md)
@@ -173,7 +173,6 @@ hidden:
 Setup <api/setup>
 Tabular <api/tabular>
 Time Series <api/timeseries>
-Multimodal <api/multimodal>
 ```
 
 ```{toctree}

docs/tutorials/predictor-tabular.md

@@ -54,47 +54,45 @@ A rough guideline: if you need predictions less often than once an hour and can
 
 ### Real-time inference
 
-Deploy the predictor as a SageMaker endpoint with {py:meth}`~autogluon.cloud.TabularCloudPredictor.deploy`:
+Deploy the predictor as a SageMaker endpoint with {py:meth}`~autogluon.cloud.TabularCloudPredictor.deploy`, then send requests through the returned {py:class}`~autogluon.cloud.TabularEndpoint`:
 
 ```python
-cloud_predictor.deploy(
-    instance_type="ml.m5.2xlarge",
-)
-```
-
-Optionally, you can also attach to a deployed endpoint with {py:meth}`~autogluon.cloud.TabularCloudPredictor.attach_endpoint`:
-
-```python
-cloud_predictor.attach_endpoint(endpoint="ENDPOINT_NAME")
+endpoint = cloud_predictor.deploy(instance_type="ml.m5.2xlarge")  # takes a few minutes
 ```
 
-Send requests to the endpoint with {py:meth}`~autogluon.cloud.TabularCloudPredictor.predict_real_time`, which returns a pandas Series of predictions:
+{py:meth}`~autogluon.cloud.TabularEndpoint.predict` returns a pandas Series of predictions:
 
 ```python
-result = cloud_predictor.predict_real_time("test.csv")  # DataFrame, local path, or S3 URL
+result = endpoint.predict("test.csv")  # DataFrame, local path, or S3 URL
 # 0      dog
 # 1      cat
 # 2      cat
 # Name: label, dtype: object
 ```
 
-For class probabilities, use {py:meth}`~autogluon.cloud.TabularCloudPredictor.predict_proba_real_time`, which returns a DataFrame with one column per class:
+For class probabilities, use {py:meth}`~autogluon.cloud.TabularEndpoint.predict_proba`. Like the batch version, it returns a `(predictions, probabilities)` tuple by default; pass `include_predict=False` to get the probabilities DataFrame alone:
 
 ```python
-result = cloud_predictor.predict_proba_real_time("test.csv")
+probabilities = endpoint.predict_proba("test.csv", include_predict=False)
 #         dog       cat
 # 0  0.682754  0.317246
 # 1  0.195782  0.804218
 # 2  0.372283  0.627717
 ```
 
-Make sure you clean up the endpoint with {py:meth}`~autogluon.cloud.TabularCloudPredictor.cleanup_deployment`:
+The endpoint stays active — and billed — until you delete it:
 
 ```python
-cloud_predictor.cleanup_deployment()
+endpoint.delete_endpoint()
 ```
 
-To check whether an endpoint is currently attached, call {py:meth}`~autogluon.cloud.TabularCloudPredictor.info` and look for the `endpoint` key in the returned dict.
+To send requests to an endpoint that's already running (e.g. from a previous session), build a {py:class}`~autogluon.cloud.TabularEndpoint` directly from the endpoint name:
+
+```python
+from autogluon.cloud import TabularEndpoint
+
+endpoint = TabularEndpoint(endpoint_name="ENDPOINT_NAME")
+```
 
 #### Invoke the endpoint without AutoGluon-Cloud
 The deployed endpoint is a normal SageMaker endpoint, and you can invoke it through other methods. For example, to invoke it with boto3 directly:

docs/tutorials/predictor-timeseries.md

@@ -77,24 +77,16 @@ A rough guideline: if you need predictions less often than once an hour and can
 
 ### Real-time inference
 
-Deploy the predictor as a SageMaker endpoint with {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.deploy`:
+Deploy the predictor as a SageMaker endpoint with {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.deploy`, then send requests through the returned {py:class}`~autogluon.cloud.TimeSeriesEndpoint`:
 
 ```python
-cloud_predictor.deploy(
-    instance_type="ml.m5.2xlarge",
-)
-```
-
-Optionally, you can also attach to a deployed endpoint with {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.attach_endpoint`:
-
-```python
-cloud_predictor.attach_endpoint(endpoint="ENDPOINT_NAME")
+endpoint = cloud_predictor.deploy(instance_type="ml.m5.2xlarge")  # takes a few minutes
 ```
 
-Send requests to the endpoint with {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.predict_real_time`. It takes the historical observations to forecast from, plus optional `known_covariates` (required when `known_covariates_names` was set at fit time) and `static_features`. The result is a DataFrame with one row per `(item_id, future timestamp)` pair and a column for each predicted quantile (plus the `mean`):
+{py:meth}`~autogluon.cloud.TimeSeriesEndpoint.predict` takes the historical observations to forecast from, plus optional `known_covariates` (required when `known_covariates_names` was set at fit time) and `static_features`. The result is a DataFrame with one row per `(item_id, future timestamp)` pair and a column for each predicted quantile (plus the `mean`):
 
 ```python
-forecasts = cloud_predictor.predict_real_time(
+forecasts = endpoint.predict(
     "train.csv",  # historical observations — forecasts start from the last timestamp per item
     known_covariates="known_covariates.csv",  # required if known_covariates_names was set
     static_features="static_features.csv",    # optional
@@ -107,13 +99,19 @@ forecasts = cloud_predictor.predict_real_time(
 # ...
 ```
 
-Make sure you clean up the endpoint with {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.cleanup_deployment`:
+The prediction length, quantiles, and target are the ones set at fit time; requests that set them to different values are rejected. The endpoint stays active — and billed — until you delete it:
 
 ```python
-cloud_predictor.cleanup_deployment()
+endpoint.delete_endpoint()
 ```
 
-To check whether an endpoint is currently attached, call {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.info` and look for the `endpoint` key in the returned dict.
+To send requests to an endpoint that's already running (e.g. from a previous session), build a {py:class}`~autogluon.cloud.TimeSeriesEndpoint` directly from the endpoint name:
+
+```python
+from autogluon.cloud import TimeSeriesEndpoint
+
+endpoint = TimeSeriesEndpoint(endpoint_name="ENDPOINT_NAME")
+```
 
 #### Invoke the endpoint without AutoGluon-Cloud
 The deployed endpoint is a normal SageMaker endpoint, so you can invoke it from any AWS SDK. The simplest payload is the historical observations as CSV — forecasts are generated starting from the last timestamp of each item:
@@ -141,7 +139,7 @@ The CSV format only carries the historical observations. To pass `static_feature
 :animate: fade-in-slide-down
 :color: secondary
 
-**Option 1: AutoGluon-Cloud's native `application/x-autogluon` envelope.** Each DataFrame is serialized as base64-encoded parquet and bundled in a single JSON object. This is what {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.predict_real_time` sends under the hood:
+**Option 1: AutoGluon-Cloud's native `application/x-autogluon` envelope.** Each DataFrame is serialized as base64-encoded parquet and bundled in a single JSON object. This is what {py:meth}`~autogluon.cloud.TimeSeriesEndpoint.predict` sends under the hood:
 
 ```python
 import base64
@@ -162,7 +160,7 @@ payload = {
     "data": df_to_b64(train_data),
     "known_covariates": df_to_b64(known_covariates),
     "static_features": df_to_b64(static_features),
-    "inference_kwargs": {},  # prediction_length / quantile_levels are baked in at fit time
+    "inference_kwargs": {},  # prediction_length / quantile_levels / target are fixed at fit time
 }
 
 client = boto3.client("sagemaker-runtime")

pyproject.toml

@@ -47,7 +47,7 @@ dependencies = [
     "PyYAML~=6.0",
     "Pillow>=10.2,<13",
     "huggingface_hub>=0.20,<2",
-    "typing_extensions>=4.0,<5",
+    "typing_extensions>=4.5,<5",  # 4.5 adds @deprecated (PEP 702)
     # CLI dependencies (autogluon-cloud command)
     "click>=8.0,<9",
     "rich>=13.0,<15",

src/autogluon/cloud/__init__.py

@@ -3,6 +3,7 @@
 from autogluon.common.utils.log_utils import _add_stream_handler
 
 from .cloud_setup import bootstrap, register, status, teardown
+from .endpoint.multimodal_endpoint import MultiModalEndpoint
 from .endpoint.tabular_endpoint import TabularEndpoint
 from .endpoint.timeseries_endpoint import TimeSeriesEndpoint
 from .model.foundation_model import TabularFoundationModel, TimeSeriesFoundationModel
@@ -13,6 +14,7 @@
 
 __all__ = [
     "MultiModalCloudPredictor",
+    "MultiModalEndpoint",
     "TabularCloudPredictor",
     "TabularEndpoint",
     "TabularFoundationModel",

src/autogluon/cloud/backend/multimodal_sagemaker_backend.py

@@ -156,7 +156,7 @@ def predict(
         """
         Predict using SageMaker batch transform.
         When minimizing latency isn't a concern, then the batch transform functionality may be easier, more scalable, and more appropriate.
-        If you want to minimize latency, use `predict_real_time()` instead.
+        If you want to minimize latency, deploy an endpoint with `deploy()` instead.
         To learn more: https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html
 
         Parameters
@@ -197,7 +197,7 @@ def predict_proba(
         """
         Predict proba using SageMaker batch transform.
         When minimizing latency isn't a concern, then the batch transform functionality may be easier, more scalable, and more appropriate.
-        If you want to minimize latency, use `predict_real_time()` instead.
+        If you want to minimize latency, deploy an endpoint with `deploy()` instead.
         To learn more: https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html
 
         Parameters

src/autogluon/cloud/backend/sagemaker_backend.py

@@ -489,9 +489,6 @@ def deploy(
             :meth:`FoundationModel.cache_model_artifact`) to skip the round-trip. Ignored when ``predictor_path`` is
             None.
         """
-        assert self.endpoint_name is None, (
-            "There is an endpoint already attached. Either detach it with `detach` or clean it up with `cleanup_deployment`"
-        )
         if inference_mode not in ("realtime", "serverless"):
             raise ValueError(f"Unsupported inference_mode={inference_mode!r}")
         overrides = check_override_keys(backend_overrides, DEPLOY_OVERRIDE_KEYS)
@@ -811,7 +808,7 @@ def predict(
         """
         Predict using SageMaker batch transform.
         When minimizing latency isn't a concern, then the batch transform functionality may be easier, more scalable, and more appropriate.
-        If you want to minimize latency, use `predict_real_time()` instead.
+        If you want to minimize latency, deploy an endpoint with `deploy()` instead.
         To learn more: https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html
         This method creates a SageMaker model with the trained predictor and runs a transform job with it.
 
@@ -888,7 +885,7 @@ def predict_proba(
         """
         Predict using SageMaker batch transform.
         When minimizing latency isn't a concern, then the batch transform functionality may be easier, more scalable, and more appropriate.
-        If you want to minimize latency, use `predict_real_time()` instead.
+        If you want to minimize latency, deploy an endpoint with `deploy()` instead.
         To learn more: https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html
         This method creates a SageMaker model with the trained predictor and runs a transform job with it.
 
@@ -1190,7 +1187,8 @@ def _predict_real_time(self, test_data, accept, split_pred_proba=True, inference
                     "The invocation of endpoint failed with Error Code 413. This is likely due to pay load size being too large."
                 )
                 logger.warning(
-                    "SageMaker endpoint could only take maximum 5MB. Please consider reduce test data size or use `predict()` instead."
+                    "SageMaker endpoints accept at most 6 MB per request (4 MB for serverless endpoints). Please "
+                    "send smaller batches, or use `predict()` for batch inference instead."
                 )
             raise e
 
@@ -1269,7 +1267,7 @@ def _predict(
             # either a file to a dataframe, or a file to an image
             if is_image_file(test_data):
                 logger.warning(
-                    "Are you sure you want to do batch inference on a single image? You might want to try `deploy()` and `predict_real_time()` instead"
+                    "Are you sure you want to do batch inference on a single image? You might want to try `deploy()` and `endpoint.predict()` instead"
                 )
             elif original_features is not None:
                 # Loading is only needed for the column check below — skip it for predictors that don't track

src/autogluon/cloud/backend/timeseries_sagemaker_backend.py

@@ -135,7 +135,7 @@ def predict(
         """
         Predict using SageMaker batch transform.
         When minimizing latency isn't a concern, then the batch transform functionality may be easier, more scalable, and more appropriate.
-        If you want to minimize latency, use `predict_real_time()` instead.
+        If you want to minimize latency, deploy an endpoint with `deploy()` instead.
         To learn more: https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html
 
         Parameters

src/autogluon/cloud/endpoint/endpoint.py

@@ -0,0 +1,47 @@
+import logging
+from typing import Any
+
+import boto3
+
+from ..utils.aws_utils import setup_sagemaker_session
+from ..utils.sagemaker_api import delete_endpoint
+
+logger = logging.getLogger(__name__)
+
+
+class Endpoint:
+    """Base class for handles to SageMaker endpoints deployed through AutoGluon-Cloud."""
+
+    def __init__(self, endpoint_name: str, session: boto3.Session | None = None):
+        """
+        Parameters
+        ----------
+        endpoint_name: str
+            Name of an existing SageMaker endpoint deployed through AutoGluon-Cloud, either with ``deploy()`` of a
+            cloud predictor (e.g. :meth:`autogluon.cloud.TabularCloudPredictor.deploy`) or of a foundation model
+            (e.g. :meth:`autogluon.cloud.TabularFoundationModel.deploy`).
+        session: boto3.Session | None, default = None
+            ``boto3.Session`` used to invoke and delete the endpoint. If ``None``, the default ambient session is used.
+        """
+        self._endpoint_name = endpoint_name
+        self._session = setup_sagemaker_session(boto_session=session)
+
+    @property
+    def endpoint_name(self) -> str:
+        return self._endpoint_name
+
+    @staticmethod
+    def _pop_as_pandas(inference_kwargs: dict[str, Any]) -> None:
+        # The serve scripts always pass as_pandas=True, so forwarding it would raise a duplicate-keyword TypeError.
+        if inference_kwargs.pop("as_pandas", True) is not True:
+            logger.warning("as_pandas must be True for real-time prediction; ignoring it.")
+
+    def delete_endpoint(self) -> None:
+        """Delete the endpoint and its backing model + endpoint config.
+
+        SageMaker API
+        -------------
+        * :sm-api:`DeleteEndpoint`, :sm-api:`DeleteEndpointConfig` and :sm-api:`DeleteModel`: delete the endpoint and
+          the endpoint config and model created with it.
+        """
+        delete_endpoint(self._endpoint_name, self._session)