file_io

class sentence_transformers.util.file_io.disabled_tqdm(*_, **__)[source]

Class to override disable argument in case progress bars are globally disabled.

Taken from https://github.com/tqdm/tqdm/issues/619#issuecomment-619639324.

sentence_transformers.util.file_io.http_get(url: str, path: str) → None[source]

Download a URL to a local file with a progress bar.

The content is streamed in chunks and first written to a temporary "<path>_part" file, which is atomically moved to path once the download has completed successfully. Parent directories of path are created automatically if they do not exist.

Parameters:

  • url (str) – The HTTP(S) URL to download.

  • path (str) – Destination file path on the local filesystem.

Raises:

  • ImportError – If the optional httpx dependency is not installed.

  • httpx.HTTPStatusError – If the HTTP request returns a non-success status code.

  • OSError – If the file cannot be written to path.

Returns:

None

sentence_transformers.util.file_io.is_sentence_transformer_model(model_name_or_path: str, token: bool | str | None = None, cache_folder: str | None = None, revision: str | None = None, local_files_only: bool = False) → bool[source]

Checks if the given model name or path corresponds to a SentenceTransformer model.

Parameters:

  • model_name_or_path (str) – The name or path of the model.

  • token (Optional[Union[bool, str]]) – The token to be used for authentication. Defaults to None.

  • cache_folder (Optional[str]) – The folder to cache the model files. Defaults to None.

  • revision (Optional[str]) – The revision of the model. Defaults to None.

  • local_files_only (bool) – Whether to only use local files for the model. Defaults to False.

Raises:

Exception – Propagates errors from the Hub that are not “file not found” (e.g. authentication, rate-limit, or network errors). The probe deliberately surfaces these instead of returning False, so callers don’t mistake a flaky Hub for “this isn’t a SentenceTransformer”.

Returns:

True if the model is a SentenceTransformer model, False if it exists but does not contain a modules.json.

Return type:

bool

sentence_transformers.util.file_io.load_dir_path(model_name_or_path: str, subfolder: str, token: bool | str | None = None, cache_folder: str | None = None, revision: str | None = None, local_files_only: bool = False) → str | None[source]

Loads the subfolder path for a given model name or path.

Parameters:

  • model_name_or_path (str) – The name or path of the model.

  • subfolder (str) – The subfolder to load.

  • token (Optional[Union[bool, str]]) – The token for authentication.

  • cache_folder (Optional[str]) – The folder to cache the downloaded files.

  • revision (Optional[str], optional) – The revision of the model. Defaults to None.

  • local_files_only (bool, optional) – Whether to only use local files. Defaults to False.

Raises:

Exception – Errors that are not unambiguously “this is not on the Hub” propagate to the caller. HFValidationError and LocalEntryNotFoundError are converted to a None return. Other failures (auth, rate-limit, network) trigger a single cache-only retry; if the cache also lacks the file, the original exception is re-raised (a cache miss would mask the real cause).

Returns:

The subfolder path, or None if the parent is a local directory without the subfolder, or no valid repo id was given.

Return type:

Optional[str]

sentence_transformers.util.file_io.load_file_path(model_name_or_path: str, filename: str | Path, subfolder: str = '', token: bool | str | None = None, cache_folder: str | None = None, revision: str | None = None, local_files_only: bool = False) → str | None[source]

Loads a file from a local or remote location.

Parameters:

  • model_name_or_path (str) – The model name or path.

  • filename (str) – The name of the file to load.

  • subfolder (str) – The subfolder within the model subfolder (if applicable).

  • token (Optional[Union[bool, str]]) – The token to access the remote file (if applicable).

  • cache_folder (Optional[str]) – The folder to cache the downloaded file (if applicable).

  • revision (Optional[str], optional) – The revision of the file (if applicable). Defaults to None.

  • local_files_only (bool, optional) – Whether to only consider local files. Defaults to False.

Raises:

Exception – Errors that are not unambiguously “this file is not on the Hub” propagate to the caller (e.g. authentication, rate-limit, network). Only EntryNotFoundError, HFValidationError, and LocalEntryNotFoundError are converted to a None return.

Returns:

The path to the loaded file, or None if the file is not present locally and the Hub confirms it is not in the repo (or no valid repo id was given).

Return type:

Optional[str]