radiant_mlhub.client package
Submodules
radiant_mlhub.client.catalog_downloader module
- class radiant_mlhub.client.catalog_downloader.AssetRecord(*, rowid: int = None, asset_key: str = None, asset_save_path: str = None, asset_url: str = None, bbox_json: str = None, collection_id: str = None, common_asset: bool = False, single_datetime: datetime.datetime = None, start_datetime: datetime.datetime = None, end_datetime: datetime.datetime = None, filtered: bool = False, geometry_json: str = None, item_id: str = None)[source]
Bases:
pydantic.main.BaseModel
A stac_assets db record.
- end_datetime: Optional[datetime.datetime]
- single_datetime: Optional[datetime.datetime]
- start_datetime: Optional[datetime.datetime]
- radiant_mlhub.client.catalog_downloader.COMMON_ASSET_NAMES = ['documentation', 'readme', 'test_split', 'train_split', 'validation_split']
Common assets will be put into _common and only downloaded once.
- class radiant_mlhub.client.catalog_downloader.CatalogDownloader(config: radiant_mlhub.client.catalog_downloader.CatalogDownloaderConfig)[source]
Bases:
object
- catalog_file: pathlib.Path
- db_conn: sqlite3.Connection
- db_cur: sqlite3.Cursor
- err_report: _io.TextIOWrapper
- err_report_path: pathlib.Path
- work_dir: pathlib.Path
- class radiant_mlhub.client.catalog_downloader.CatalogDownloaderConfig(*, api_key: str = None, bbox: Optional[Union[Tuple[float], List[float]]] = None, catalog_only: bool = False, collection_filter: Dict[str, List[str]] = None, dataset_id: str, if_exists: radiant_mlhub.if_exists.DownloadIfExistsOpts = DownloadIfExistsOpts.resume, intersects: Dict[str, Any] = None, output_dir: pathlib.Path, profile: str = None, mlhub_api_session: radiant_mlhub.session.Session, temporal_query: Optional[Union[datetime.datetime, Tuple[datetime.datetime, datetime.datetime]]] = None)[source]
Bases:
pydantic.main.BaseModel
Configuration model & validator for CatalogDownloader.
- mlhub_api_session: radiant_mlhub.session.Session
Requests session for mlhub api calls.
- output_dir: pathlib.Path
- temporal_query: Optional[Union[datetime.datetime, Tuple[datetime.datetime, datetime.datetime]]]
radiant_mlhub.client.collections module
- radiant_mlhub.client.collections.get_collection(collection_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /collections/{p1}
endpoint.See the MLHub API docs for details.
- Parameters
- Returns
collection
- Return type
- Raises
EntityDoesNotExist – If a 404 response code is returned by the API
MLHubException – If any other response code is returned
- radiant_mlhub.client.collections.get_collection_item(collection_id: str, item_id: str, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /collections/{p1}/items/{p2}
endpoint.- Parameters
collection_id (str) – The ID of the Collection to which the Item belongs.
item_id (str) – The ID of the Item.
api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Returns
item
- Return type
- radiant_mlhub.client.collections.list_collection_items(collection_id: str, *, page_size: Optional[int] = None, extensions: Optional[List[str]] = None, limit: int = 10, api_key: Optional[str] = None, profile: Optional[str] = None) Iterator[Dict[str, Any]] [source]
Yields JSON-like dictionaries representing STAC Item objects returned by the Radiant MLHub
GET /collections/{collection_id}/items
endpoint.Note
Because some collections may contain hundreds of thousands of items, this function limits the total number of responses to
10
by default. You can change this value by increasing the value of thelimit
keyword argument, or setting it toNone
to list all items. Be aware that trying to list all items in a large collection may take a very long time.- Parameters
collection_id (str) – The ID of the collection from which to fetch items
page_size (int) – The number of items to return in each page. If set to
None
, then this parameter will not be passed to the API and the default API value will be used (currently30
).extensions (list) – If provided, then only items that support all of the extensions listed will be returned.
limit (int) – The maximum total number of items to yield. Defaults to
10
.api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Yields
item (dict) – JSON-like dictionary representing a STAC Item associated with the given collection.
- radiant_mlhub.client.collections.list_collections(*, api_key: Optional[str] = None, profile: Optional[str] = None) List[Dict[str, Any]] [source]
Gets a list of JSON-like dictionaries representing STAC Collection objects returned by the Radiant MLHub
GET /collections
endpoint.See the MLHub API docs for details.
- Parameters
- Returns
collections – List of JSON-like dictionaries representing STAC Collection objects.
- Return type
List[dict]
radiant_mlhub.client.datasets module
- radiant_mlhub.client.datasets.download_collection_archive(archive_id: str, output_dir: Optional[Union[str, pathlib.Path]] = None, *, if_exists: radiant_mlhub.if_exists.DownloadIfExistsOpts = DownloadIfExistsOpts.resume, api_key: Optional[str] = None, profile: Optional[str] = None) pathlib.Path [source]
Downloads the archive with the given ID to an output location (current working directory by default).
The
if_exists
argument determines how to handle an existing archive file in the output directory. The default behavior (defined byif_exists="resume"
) is to resume the download by requesting a byte range starting at the size of the existing file. If the existing file is the same size as the file to be downloaded (as determined by theContent-Length
header), then the download is skipped. You can automatically skip download usingif_exists="skip"
(this may be faster if you know the download was not interrupted, since no network request is made to get the archive size). You can also overwrite the existing file usingif_exists="overwrite"
.- Parameters
archive_id (str) – The ID of the archive to download. This is the same as the Collection ID.
output_dir (Path) – Path to which the archive will be downloaded. Defaults to the current working directory.
if_exists (str, optional) – How to handle an existing archive at the same location. If
"skip"
, the download will be skipped. If"overwrite"
, the existing file will be overwritten and the entire file will be re-downloaded. If"resume"
(the default), the existing file size will be compared to the size of the download (using theContent-Length
header). If the existing file is smaller, then only the remaining portion will be downloaded. Otherwise, the download will be skipped.api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Returns
output_path – The full path to the downloaded archive file.
- Return type
Path
- Raises
ValueError – If
if_exists
is not one of"skip"
,"overwrite"
, or"resume"
.
- radiant_mlhub.client.datasets.get_catalog_info(dataset_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Gets info for the given archive from the
/catalog/{dataset_id}/info
endpoint as a JSON-like dictionary.The JSON object returned by the API has the following properties:
dataset
: ID of the dataset that this archive’s Collection belongs to.stac_catalog_size
: size of the dataset_id.tar.gz STAC archive (in bytes)estimated_dataset_size
: size in bytes of entire dataset (estimated)
- Parameters
- Returns
archive_info – JSON-like dictionary representing the API response.
- Return type
- radiant_mlhub.client.datasets.get_collection_archive_info(archive_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Gets info for the given archive from the
/archive/{archive_id}/info
endpoint as a JSON-like dictionary.The JSON object returned by the API has the following properties:
collection
: The ID of the Collection that this archive is associated with.dataset
: The ID of the dataset that this archive’s Collection belongs to.size
: The size of the archive (in bytes)types
: The types associated with this archive’s Collection. Will be one of"source_imagery"
or"label"
.
- Parameters
- Returns
archive_info – JSON-like dictionary representing the API response.
- Return type
- radiant_mlhub.client.datasets.get_dataset(dataset_id_or_doi: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing a dataset by first trying to look up the dataset by ID, then falling back to finding the dataset by DOI.
See the MLHub API docs for details.
- radiant_mlhub.client.datasets.get_dataset_by_doi(dataset_doi: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /datasets/doi/{dataset_id}
endpoint.See the MLHub API docs for details.
- radiant_mlhub.client.datasets.get_dataset_by_id(dataset_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /datasets/{dataset_id}
endpoint.See the MLHub API docs for details.
- radiant_mlhub.client.datasets.list_datasets(*, tags: Optional[Union[str, Iterable[str]]] = None, text: Optional[Union[str, Iterable[str]]] = None, api_key: Optional[str] = None, profile: Optional[str] = None) List[Dict[str, Any]] [source]
Gets a list of JSON-like dictionaries representing dataset objects returned by the Radiant MLHub
GET /datasets
endpoint.See the MLHub API docs for details.
- Parameters
tags (A tag or list of tags to filter datasets by. If not
None
, only datasets) – containing all provided tags will be returned.text (A text phrase or list of text phrases to filter datasets by. If not
None
,) – only datasets containing all phrases will be returned.api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Returns
datasets
- Return type
List[dict]
radiant_mlhub.client.ml_models module
- radiant_mlhub.client.ml_models.get_model_by_id(model_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /models/{model_id}
endpoint.See the MLHub API docs for details.
radiant_mlhub.client.resumable_downloader module
- class radiant_mlhub.client.resumable_downloader.ResumableDownloader(url: str, out_file: pathlib.Path, desc: Optional[str] = None, session: Optional[requests.sessions.Session] = None, if_exists: radiant_mlhub.if_exists.DownloadIfExistsOpts = DownloadIfExistsOpts.overwrite, disable_progress_bar: bool = True, chunk_size: int = 1024, chunk_unit: str = 'KB')[source]
Bases:
object
Resumable downloader, for a single file.
Similar to datasets._download_collection_archive_chunked(), but this is not parallelized.
Supports DownloadIfExistsOpts.
Displays progress bar (optional).
- out_file: pathlib.Path
- session: requests.sessions.Session
Module contents
Low-level functions for making requests to MLHub API and Blob Storage endpoints.
- class radiant_mlhub.client.CatalogDownloader(config: radiant_mlhub.client.catalog_downloader.CatalogDownloaderConfig)[source]
Bases:
object
- catalog_file: pathlib.Path
- db_conn: sqlite3.Connection
- db_cur: sqlite3.Cursor
- err_report: _io.TextIOWrapper
- err_report_path: pathlib.Path
- work_dir: pathlib.Path
- class radiant_mlhub.client.CatalogDownloaderConfig(*, api_key: str = None, bbox: Optional[Union[Tuple[float], List[float]]] = None, catalog_only: bool = False, collection_filter: Dict[str, List[str]] = None, dataset_id: str, if_exists: radiant_mlhub.if_exists.DownloadIfExistsOpts = DownloadIfExistsOpts.resume, intersects: Dict[str, Any] = None, output_dir: pathlib.Path, profile: str = None, mlhub_api_session: radiant_mlhub.session.Session, temporal_query: Optional[Union[datetime.datetime, Tuple[datetime.datetime, datetime.datetime]]] = None)[source]
Bases:
pydantic.main.BaseModel
Configuration model & validator for CatalogDownloader.
- mlhub_api_session: radiant_mlhub.session.Session
Requests session for mlhub api calls.
- output_dir: pathlib.Path
- temporal_query: Optional[Union[datetime.datetime, Tuple[datetime.datetime, datetime.datetime]]]
- radiant_mlhub.client.download_collection_archive(archive_id: str, output_dir: Optional[Union[str, pathlib.Path]] = None, *, if_exists: radiant_mlhub.if_exists.DownloadIfExistsOpts = DownloadIfExistsOpts.resume, api_key: Optional[str] = None, profile: Optional[str] = None) pathlib.Path [source]
Downloads the archive with the given ID to an output location (current working directory by default).
The
if_exists
argument determines how to handle an existing archive file in the output directory. The default behavior (defined byif_exists="resume"
) is to resume the download by requesting a byte range starting at the size of the existing file. If the existing file is the same size as the file to be downloaded (as determined by theContent-Length
header), then the download is skipped. You can automatically skip download usingif_exists="skip"
(this may be faster if you know the download was not interrupted, since no network request is made to get the archive size). You can also overwrite the existing file usingif_exists="overwrite"
.- Parameters
archive_id (str) – The ID of the archive to download. This is the same as the Collection ID.
output_dir (Path) – Path to which the archive will be downloaded. Defaults to the current working directory.
if_exists (str, optional) – How to handle an existing archive at the same location. If
"skip"
, the download will be skipped. If"overwrite"
, the existing file will be overwritten and the entire file will be re-downloaded. If"resume"
(the default), the existing file size will be compared to the size of the download (using theContent-Length
header). If the existing file is smaller, then only the remaining portion will be downloaded. Otherwise, the download will be skipped.api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Returns
output_path – The full path to the downloaded archive file.
- Return type
Path
- Raises
ValueError – If
if_exists
is not one of"skip"
,"overwrite"
, or"resume"
.
- radiant_mlhub.client.get_catalog_info(dataset_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Gets info for the given archive from the
/catalog/{dataset_id}/info
endpoint as a JSON-like dictionary.The JSON object returned by the API has the following properties:
dataset
: ID of the dataset that this archive’s Collection belongs to.stac_catalog_size
: size of the dataset_id.tar.gz STAC archive (in bytes)estimated_dataset_size
: size in bytes of entire dataset (estimated)
- Parameters
- Returns
archive_info – JSON-like dictionary representing the API response.
- Return type
- radiant_mlhub.client.get_collection(collection_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /collections/{p1}
endpoint.See the MLHub API docs for details.
- Parameters
- Returns
collection
- Return type
- Raises
EntityDoesNotExist – If a 404 response code is returned by the API
MLHubException – If any other response code is returned
- radiant_mlhub.client.get_collection_archive_info(archive_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Gets info for the given archive from the
/archive/{archive_id}/info
endpoint as a JSON-like dictionary.The JSON object returned by the API has the following properties:
collection
: The ID of the Collection that this archive is associated with.dataset
: The ID of the dataset that this archive’s Collection belongs to.size
: The size of the archive (in bytes)types
: The types associated with this archive’s Collection. Will be one of"source_imagery"
or"label"
.
- Parameters
- Returns
archive_info – JSON-like dictionary representing the API response.
- Return type
- radiant_mlhub.client.get_collection_item(collection_id: str, item_id: str, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /collections/{p1}/items/{p2}
endpoint.- Parameters
collection_id (str) – The ID of the Collection to which the Item belongs.
item_id (str) – The ID of the Item.
api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Returns
item
- Return type
- radiant_mlhub.client.get_dataset(dataset_id_or_doi: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing a dataset by first trying to look up the dataset by ID, then falling back to finding the dataset by DOI.
See the MLHub API docs for details.
- radiant_mlhub.client.get_dataset_by_doi(dataset_doi: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /datasets/doi/{dataset_id}
endpoint.See the MLHub API docs for details.
- radiant_mlhub.client.get_dataset_by_id(dataset_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /datasets/{dataset_id}
endpoint.See the MLHub API docs for details.
- radiant_mlhub.client.get_model_by_id(model_id: str, *, api_key: Optional[str] = None, profile: Optional[str] = None) Dict[str, Any] [source]
Returns a JSON-like dictionary representing the response from the Radiant MLHub
GET /models/{model_id}
endpoint.See the MLHub API docs for details.
- radiant_mlhub.client.list_collection_items(collection_id: str, *, page_size: Optional[int] = None, extensions: Optional[List[str]] = None, limit: int = 10, api_key: Optional[str] = None, profile: Optional[str] = None) Iterator[Dict[str, Any]] [source]
Yields JSON-like dictionaries representing STAC Item objects returned by the Radiant MLHub
GET /collections/{collection_id}/items
endpoint.Note
Because some collections may contain hundreds of thousands of items, this function limits the total number of responses to
10
by default. You can change this value by increasing the value of thelimit
keyword argument, or setting it toNone
to list all items. Be aware that trying to list all items in a large collection may take a very long time.- Parameters
collection_id (str) – The ID of the collection from which to fetch items
page_size (int) – The number of items to return in each page. If set to
None
, then this parameter will not be passed to the API and the default API value will be used (currently30
).extensions (list) – If provided, then only items that support all of the extensions listed will be returned.
limit (int) – The maximum total number of items to yield. Defaults to
10
.api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Yields
item (dict) – JSON-like dictionary representing a STAC Item associated with the given collection.
- radiant_mlhub.client.list_collections(*, api_key: Optional[str] = None, profile: Optional[str] = None) List[Dict[str, Any]] [source]
Gets a list of JSON-like dictionaries representing STAC Collection objects returned by the Radiant MLHub
GET /collections
endpoint.See the MLHub API docs for details.
- Parameters
- Returns
collections – List of JSON-like dictionaries representing STAC Collection objects.
- Return type
List[dict]
- radiant_mlhub.client.list_datasets(*, tags: Optional[Union[str, Iterable[str]]] = None, text: Optional[Union[str, Iterable[str]]] = None, api_key: Optional[str] = None, profile: Optional[str] = None) List[Dict[str, Any]] [source]
Gets a list of JSON-like dictionaries representing dataset objects returned by the Radiant MLHub
GET /datasets
endpoint.See the MLHub API docs for details.
- Parameters
tags (A tag or list of tags to filter datasets by. If not
None
, only datasets) – containing all provided tags will be returned.text (A text phrase or list of text phrases to filter datasets by. If not
None
,) – only datasets containing all phrases will be returned.api_key (str) – An API key to use for this request. This will override an API key set in a profile on using an environment variable
profile (str) – A profile to use when making this request.
- Returns
datasets
- Return type
List[dict]