MultiVectorEncoder

class sentence_transformers.multi_vector_encoder.model.MultiVectorEncoder(model_name_or_path: str | None = None, *, modules: list[Module] | None = None, device: str | None = None, prompts: dict[str, str] | None = None, default_prompt_name: str | None = None, cache_folder: str | None = None, trust_remote_code: bool = False, revision: str | None = None, local_files_only: bool = False, token: bool | str | None = None, model_kwargs: dict[str, Any] | None = None, processor_kwargs: dict[str, Any] | None = None, config_kwargs: dict[str, Any] | None = None, model_card_data: MultiVectorEncoderModelCardData | None = None, backend: Literal['torch', 'onnx', 'openvino'] = 'torch', similarity_fn_name: str | SimilarityFunction | None = None)[source]

Loads or creates a multi-vector / late-interaction (ColBERT-style) embedding model.

Unlike SentenceTransformer which produces a single vector per input, MultiVectorEncoder produces a sequence of vectors per input, one per token. Scoring between queries and documents is done with the MaxSim late-interaction operator: for each query token, take the max similarity to any document token, then sum across query tokens.

Parameters:

model_name_or_path (str, optional) – If a filepath on disk, loads the model from that path. Otherwise, tries to download a pre-trained MultiVectorEncoder model. If that fails, tries to construct a model from the Hugging Face Hub with that name. Defaults to None.
modules (list[nn.Module], optional) – A list of torch modules that are called sequentially. Can be used to create custom MultiVectorEncoder models from scratch. Defaults to None.
device (str, optional) – Device (like "cuda", "cpu", "mps", "npu") that should be used for computation. If None, checks if a GPU can be used. Defaults to None.
prompts (dict[str, str], optional) – Standard ST prompts dict, prepended to inputs by the encode methods. For ColBERT-style models supply {"query": "[Q] ", "document": "[D] "} (or whatever the model’s prefix tokens are). Legacy PyLate / Stanford-NLP checkpoints stored these as separate query_prefix / document_prefix fields and are auto-promoted on load.
default_prompt_name (str, optional) – The name of the prompt that should be used by default. If not set, no prompt will be applied. Defaults to None.
cache_folder (str, optional) – Path to store models. Can also be set by the SENTENCE_TRANSFORMERS_HOME environment variable. Defaults to None.
trust_remote_code (bool, optional) – Whether to allow for custom models defined on the Hub in their own modeling files. Defaults to False.
revision (str, optional) – The specific model version to use. Defaults to None.
local_files_only (bool, optional) – Whether to only look at local files. Defaults to False.
token (bool or str, optional) – Hugging Face authentication token. Defaults to None.
model_kwargs (dict[str, Any], optional) – Keyword arguments passed to the underlying Hugging Face Transformers model. Defaults to None.
processor_kwargs (dict[str, Any], optional) – Keyword arguments passed to the Hugging Face Transformers processor / tokenizer. Defaults to None.
config_kwargs (dict[str, Any], optional) – Keyword arguments passed to the Hugging Face Transformers config. Defaults to None.
model_card_data (MultiVectorEncoderModelCardData, optional) – A model card data object. Defaults to None.
backend (str, optional) – The backend to use for inference. Can be "torch" (default), "onnx", or "openvino". Defaults to "torch".
similarity_fn_name (str or SimilarityFunction, optional) – The name of the similarity function, either "maxsim" or "meanmaxsim" (MaxSim divided by the query’s token count). Defaults to "maxsim".

Note

Length / expansion / masking knobs (query_length, document_length, query_expansion, skiplist_words, …) live on the underlying modules (Transformer and MultiVectorMask). Saved checkpoints carry them in their config.

Example

from sentence_transformers import MultiVectorEncoder

# 1. Load a pretrained MultiVectorEncoder model
model = MultiVectorEncoder("lightonai/LateOn")

queries = ["What is the capital of France?"]
documents = [
    "Paris is the capital of France.",
    "Berlin is the capital of Germany.",
]

# 2. Encode queries and documents (note the asymmetric encode_query / encode_document split)
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)

# Each entry is a 2D tensor of shape (num_tokens_i, embedding_dim), variable-length per input.
print(query_embeddings[0].shape)
# torch.Size([10, 128])

# 3. Score with MaxSim
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[9.1129, 8.8769]], device='cuda:0')

Initialize a BaseModel instance.

Parameters:

model_name_or_path (str, optional) – If a filepath on disk, loads the model from that path. Otherwise, tries to download a pre-trained model. If that fails, tries to construct a model from the Hugging Face Hub with that name. Defaults to None.
modules (list[nn.Module], optional) – A list of torch modules that are called sequentially. Can be used to create custom models from scratch. Defaults to None.
device (str, optional) – Device (like "cuda", "cpu", "mps", "npu") that should be used for computation. If None, checks if a GPU can be used. If a device_map is provided via model_kwargs, that controls device placement and this argument is ignored. Defaults to None.
prompts (dict[str, str], optional) – A dictionary with prompts for the model. The key is the prompt name, the value is the prompt text. The prompt text will be prepended before any text during inference. For example: {"query": "query: ", "passage": "passage: "}. If a model has saved prompts, you can override them by passing your own, or pass {"query": "", "document": ""} to disable them. Defaults to None.
default_prompt_name (str, optional) – The name of the prompt that should be used by default. If not set, no prompt will be applied. Defaults to None.
cache_folder (str, optional) – Path to store models. Can also be set by the SENTENCE_TRANSFORMERS_HOME environment variable. Defaults to None.
trust_remote_code (bool, optional) – Whether to allow for custom models defined on the Hub in their own modeling files. Only set to True for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine. Defaults to False.
revision (str, optional) – The specific model version to use. It can be a branch name, a tag name, or a commit id, for a stored model on Hugging Face. Defaults to None.
local_files_only (bool, optional) – Whether to only look at local files (i.e., do not try to download the model). Defaults to False.
token (bool or str, optional) – Hugging Face authentication token to download private models. Defaults to None.
model_kwargs (dict[str, Any], optional) –
Keyword arguments passed to the underlying Hugging Face Transformers model via AutoModel.from_pretrained. Particularly useful options include:
- torch_dtype: Override the default torch.dtype and load the model under a specific dtype. Can be torch.float16, torch.bfloat16, torch.float32, or "auto" to use the dtype from the model’s config.json.
- attn_implementation: The attention implementation to use. For example "eager", "sdpa", or "flash_attention_2". If you pip install kernels, then "flash_attention_2" should work without having to install flash_attn. It is frequently the fastest option. Defaults to "sdpa" when available (torch>=2.1.1).
- device_map: Controls how the model is placed across devices, e.g. "auto" for model parallelism, or a single device like "cuda:1" to load the backbone directly onto one GPU (useful when serving multiple models in one process). When set, it takes precedence over the device argument.
- provider: For backend="onnx", the ONNX execution provider (e.g. "CUDAExecutionProvider").
- file_name: For backend="onnx" or "openvino", the filename to load (e.g. for optimized or quantized models).
- export: For backend="onnx" or "openvino", whether to export the model to the backend format. Also set automatically if the exported file doesn’t exist.
See the PreTrainedModel.from_pretrained documentation for more details. Defaults to None.
processor_kwargs (dict[str, Any], optional) – Keyword arguments passed to the Hugging Face Transformers processor/tokenizer via AutoProcessor.from_pretrained. See the AutoTokenizer.from_pretrained documentation for more details. Defaults to None.
config_kwargs (dict[str, Any], optional) – Keyword arguments passed to the Hugging Face Transformers config via AutoConfig.from_pretrained. See the AutoConfig.from_pretrained documentation for more details. Defaults to None.
model_card_data (CardData, optional) – A model card data object that contains information about the model. Used to generate a model card when saving the model. If not set, a default model card data object is created. Defaults to None.
backend (str, optional) – The backend to use for inference. Can be "torch" (default), "onnx", or "openvino". Defaults to "torch".

active_adapters() → list[str][source]

If you are not familiar with adapters and PEFT methods, we invite you to read more about them on the PEFT official documentation: https://huggingface.co/docs/peft

Gets the current active adapters of the model. In case of multi-adapter inference (combining multiple adapters for inference) returns the list of all active adapters so that users can deal with them accordingly.

For previous PEFT versions (that does not support multi-adapter inference), module.active_adapter will return a single string.

add_adapter(*args, **kwargs) → None[source]

Adds a fresh new adapter to the current model for training purposes. If no adapter name is passed, a default name is assigned to the adapter to follow the convention of PEFT library (in PEFT we use “default” as the default adapter name).

Requires peft as a backend to load the adapter weights and the underlying model to be compatible with PEFT.

Parameters:

*args – Positional arguments to pass to the underlying AutoModel add_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.add_adapter
**kwargs – Keyword arguments to pass to the underlying AutoModel add_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.add_adapter

bfloat16() → Self

Casts all floating point parameters and buffers to bfloat16 datatype.

Note

This method modifies the module in-place.

Returns:: self
Return type:: Module

compile(*args, **kwargs) → None[source]

Compile the model’s forward pass with torch.compile() to speed up inference.

All arguments are forwarded to torch.compile(). Inference (e.g. encode() or predict()) runs the forward pass by calling the model, so compiling the model speeds up these calls.

Tip

Pass dynamic=True for inputs with variable sequence lengths, so one compiled kernel handles any length. Explicit dynamic=False recompiles the model for every new sequence length, which adds significant overhead.

property config: PretrainedConfig | None

The transformers PretrainedConfig of the underlying model.

Several integrations (most notably Deepspeed and transformers.Trainer) read model.config.hidden_size directly. Without this delegation those integrations crash with AttributeError: 'SentenceTransformer' object has no attribute 'config' because BaseModel is an nn.Sequential rather than a PreTrainedModel.

Returns the config of transformers_model, or None when no underlying transformers model can be located (e.g. a pure StaticEmbedding-only setup).

cpu() → Self

Move all model parameters and buffers to the CPU.

Note

This method modifies the module in-place.

Returns:: self
Return type:: Module

cuda(device: int | device | None = None) → Self

Move all model parameters and buffers to the GPU.

This also makes associated parameters and buffers different objects. So it should be called before constructing the optimizer if the module will live on GPU while being optimized.

Note

This method modifies the module in-place.

Parameters:: device (int, optional) – if specified, all parameters will be copied to that device
Returns:: self
Return type:: Module

delete_adapter(*args, **kwargs) → None[source]

If you are not familiar with adapters and PEFT methods, we invite you to read more about them on the PEFT official documentation: https://huggingface.co/docs/peft

Delete an adapter’s LoRA layers from the underlying model.

Parameters:

*args – Positional arguments to pass to the underlying AutoModel delete_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.delete_adapter
**kwargs – Keyword arguments to pass to the underlying AutoModel delete_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.delete_adapter

property device: device: Get torch.device from module, assuming that the whole module has one device. In case there are no PyTorch parameters, fall back to CPU.

disable_adapters() → None[source]: Disable all adapters that are attached to the model. This leads to inferring with the base model only.

double() → Self

Casts all floating point parameters and buffers to double datatype.

Note

This method modifies the module in-place.

Returns:: self
Return type:: Module

property dtype: dtype | None

The dtype of the module (assuming that all the module parameters have the same dtype).

Type:: torch.dtype

enable_adapters() → None[source]: Enable adapters that are attached to the model. The model will use self.active_adapter()

Compute multi-vector token-level embeddings.

Tip

Prefer encode_query() and encode_document() for retrieval tasks. They set the task for you and route through the correct prefix / length / masking. Use encode() directly only when you want to override the task explicitly.

Parameters:

inputs – The inputs to embed. Can be a string, a list of strings, or multimodal inputs (dicts, images, arrays).
prompt_name (str, optional) – The name of the prompt to use for encoding.
prompt (str, optional) – A prompt string to prepend to each input. Overrides prompt_name.
batch_size (int, optional) – Batch size for the forward pass. Defaults to 32.
show_progress_bar (bool, optional) – Whether to show a progress bar. Defaults to None (auto).
output_value (str, optional) – "token_embeddings" (default) returns per-input token embeddings, sliced by the scoring mask. None returns the raw per-input module output dicts instead: every feature key (token_embeddings, attention_mask, and any extra keys custom modules wrote), unsliced and padded to each batch’s longest input. With None, normalization and the convert_to_* options do not apply. Per-call token_pooling does apply: it rewrites token_embeddings and attention_mask in the dicts.
convert_to_numpy (bool, optional) – If True, returns a list of numpy.ndarray and moves each batch to the CPU as it is encoded. Defaults to False, so embeddings stay on device: scoring them with similarity() then needs no transfer, which is worth multiples on an accelerator. Set it for corpora too large to keep in device memory. Multi-process encoding (a pool, or a list of ``device``s) always returns on the CPU, since embeddings are moved there to cross the process boundary.
device (str, torch.device, list, or None) – Device(s) for computation. Defaults to None.
normalize_embeddings (bool, optional) – If True, L2-normalize each per-token embedding before returning. Use this when the loaded pipeline does not include a Normalize module but you still want unit-norm vectors. No-op when a token-level Normalize already ran. Defaults to False.
pool (dict, optional) – A multi-process pool created via start_multi_process_pool().
chunk_size (int, optional) – Chunk size for multi-process encoding.
token_pooling (BaseTokenPooling, optional) – Per-call token pooling applied after the pipeline to embeddings whose task is in the pooling’s tasks (by default only documents). If the model already bakes a pooling into its pipeline, this compounds on top of it (pooling further). A one-time note is logged so the case is discoverable. With output_value=None, applied to the raw dicts (token_embeddings and attention_mask are rewritten). Defaults to None.
task (str, optional) – One of "query", "document". Sets the prefix / length / masking strategy.

Returns:

By default, a list of per-input 2D tensors of shape (num_tokens_i, embedding_dim) (variable-length) on device, or numpy arrays with convert_to_numpy=True. With output_value=None, a list of per-input feature dicts (including each input’s real attention_mask). If a single string is passed, the outer list is unwrapped (e.g. a bare 2D tensor for the default).

Return type:

list[Tensor] | list[ndarray] | Tensor | ndarray

Compute document embeddings. Uses the first available of "document" / "passage" / "corpus" prompts and routes through the document side.

See encode() for the full parameter documentation. This method differs only by:

If no prompt_name or prompt is provided, it uses the first available of "document" / "passage" / "corpus" from the model’s prompts dictionary.
It sets the task to "document": the document prefix token is inserted, the max sequence length is document_length, and skiplist tokens (e.g. punctuation) are excluded from the output.

Compute query embeddings. Uses the “query” prompt if available and routes through the query side.

See encode() for the full parameter documentation. This method differs only by:

If no prompt_name or prompt is provided, it uses the predefined "query" prompt when one exists in the model’s prompts dictionary.
It sets the task to "query": the query prefix token is inserted, the max sequence length is query_length, and (when query_expansion is set) the input is extended with expansion tokens.

eval() → Self

Set the module in evaluation mode.

This has an effect only on certain modules. See the documentation of particular modules for details of their behaviors in training/evaluation mode, i.e. whether they are affected, e.g. Dropout, BatchNorm, etc.

This is equivalent with self.train(False).

See Locally disabling gradient computation for a comparison between .eval() and several similar mechanisms that may be confused with it.

Returns:: self
Return type:: Module

evaluate(evaluator: BaseEvaluator, output_path: str | None = None) → dict[str, float] | float[source]

Evaluate the model based on an evaluator

Parameters:

evaluator (BaseEvaluator) – The evaluator used to evaluate the model.
output_path (str, optional) – The path where the evaluator can write the results. Defaults to None.

Returns:

The evaluation results.

extend(sequential: Iterable[Module]) → Self

Extends the current Sequential container with layers from another Sequential container.

Parameters:: sequential (Sequential) – A Sequential container whose layers will be added to the current container.

Example:

>>> import torch.nn as nn
>>> n = nn.Sequential(nn.Linear(1, 2), nn.Linear(2, 3))
>>> other = nn.Sequential(nn.Linear(3, 4), nn.Linear(4, 5))
>>> n.extend(other) # or `n + other`
Sequential(
    (0): Linear(in_features=1, out_features=2, bias=True)
    (1): Linear(in_features=2, out_features=3, bias=True)
    (2): Linear(in_features=3, out_features=4, bias=True)
    (3): Linear(in_features=4, out_features=5, bias=True)
)

float() → Self

Casts all floating point parameters and buffers to float datatype.

Note

This method modifies the module in-place.

Returns:: self
Return type:: Module

get_adapter_state_dict(*args, **kwargs) → dict[source]

If you are not familiar with adapters and PEFT methods, we invite you to read more about them on the PEFT official documentation: https://huggingface.co/docs/peft

Gets the adapter state dict that should only contain the weights tensors of the specified adapter_name adapter. If no adapter_name is passed, the active adapter is used.

Parameters:

*args – Positional arguments to pass to the underlying AutoModel get_adapter_state_dict function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.get_adapter_state_dict
**kwargs – Keyword arguments to pass to the underlying AutoModel get_adapter_state_dict function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.get_adapter_state_dict

get_backend() → Literal['torch', 'onnx', 'openvino'][source]

Return the backend used for inference, which can be one of “torch”, “onnx”, or “openvino”.

Returns:: The backend used for inference.
Return type:: str

get_embedding_dimension() → int | None[source]: The dimensionality of each token vector returned by encode().

get_max_seq_length() → int | None[source]

Deprecated since version Use: the max_seq_length property instead.

Returns the maximal sequence length that the first module of the model accepts. Longer inputs will be truncated.

Returns:: The maximal sequence length that the model accepts, or None if it is not defined.
Return type:: Optional[int]

get_model_kwargs() → list[str][source]

Get the keyword arguments specific to this model for inference methods like encode or predict.

Example

>>> from sentence_transformers import SentenceTransformer, SparseEncoder
>>> SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2").get_model_kwargs()
[]
>>> SentenceTransformer("jinaai/jina-embeddings-v4", trust_remote_code=True).get_model_kwargs()
['task', 'truncate_dim']
>>> SparseEncoder("opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill").get_model_kwargs()
['task']

Returns:: A list of keyword arguments for the forward pass.
Return type:: list[str]

gradient_checkpointing_enable(gradient_checkpointing_kwargs: dict[str, Any] | None = None) → None[source]: Enable gradient checkpointing for the model.

half() → Self

Casts all floating point parameters and buffers to half datatype.

Note

This method modifies the module in-place.

Returns:: self
Return type:: Module

insert(index: int, module: Module) → Self

Inserts a module into the Sequential container at the specified index.

Parameters:

index (int) – The index to insert the module.
module (Module) – The module to be inserted.

Example:

>>> import torch.nn as nn
>>> n = nn.Sequential(nn.Linear(1, 2), nn.Linear(2, 3))
>>> n.insert(0, nn.Linear(3, 4))
Sequential(
    (0): Linear(in_features=3, out_features=4, bias=True)
    (1): Linear(in_features=1, out_features=2, bias=True)
    (2): Linear(in_features=2, out_features=3, bias=True)
)

is_singular_input(inputs: Any) → TypeIs[str | Image | ndarray | Tensor | AudioDict | None | VideoDict | MessageDict | list[MessageDict] | dict[Literal['text', 'image', 'audio', 'video'], str | Image | ndarray | Tensor | AudioDict | None | VideoDict]][source]: Check if the input is a single example rather than a batch. Redeclared with TypeIs so type checkers narrow the branches in encode().

load_adapter(*args, **kwargs) → None[source]

Load adapter weights from file or remote Hub folder.” If you are not familiar with adapters and PEFT methods, we invite you to read more about them on PEFT official documentation: https://huggingface.co/docs/peft

Requires peft as a backend to load the adapter weights and the underlying model to be compatible with PEFT.

Parameters:

*args – Positional arguments to pass to the underlying AutoModel load_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.load_adapter
**kwargs – Keyword arguments to pass to the underlying AutoModel load_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.load_adapter

property max_seq_length: int | None

Returns the maximal input sequence length for the model. Longer inputs will be truncated.

Returns:: The maximal input sequence length, or None if not defined.
Return type:: Optional[int]

property modalities: list[Literal['text', 'image', 'audio', 'video', 'message'] | tuple[Literal['text', 'image', 'audio', 'video'], ...]]: Return the list of modalities supported by this model, e.g. ["text"] or ["text", "image", "message"].

model_card_data_class[source]: alias of MultiVectorEncoderModelCardData

mtia(device: int | device | None = None) → Self

Move all model parameters and buffers to the MTIA.

This also makes associated parameters and buffers different objects. So it should be called before constructing the optimizer if the module will live on MTIA while being optimized.

Note

This method modifies the module in-place.

Parameters:: device (int, optional) – if specified, all parameters will be copied to that device
Returns:: self
Return type:: Module

pop(key: int | slice) → Module: Pop key from self.

Preprocesses the inputs for the model.

Parameters:

inputs (Sequence[SingleInput | PairInput]) – A list of inputs to be preprocessed. Each input can be a string, dict, tuple, PIL Image, numpy array, torch Tensor, or other supported modality. If a single input is provided, it must be wrapped in a list.
prompt (str, optional) – A prompt string to prepend to text inputs. Defaults to None. If the model supports the message modality, the prompt will be added as a system message to the input messages instead of being prepended to text.
**kwargs – Forwarded to the input module’s preprocess. For Transformer input modules this notably includes processing_kwargs, which overrides the processor kwargs configured on the module for this call only (see Transformer.preprocess).

Returns:

A dictionary of tensors with the preprocessed inputs.

Return type:

dict[str, Tensor | Any]

property processor: Any: Property to get the processor that is used by this model

push_to_hub(repo_id: str, token: str | None = None, private: bool | None = None, safe_serialization: bool = True, commit_message: str | None = None, local_model_path: str | None = None, exist_ok: bool = False, replace_model_card: bool = False, train_datasets: list[str] | None = None, revision: str | None = None, create_pr: bool = False) → str[source]

Uploads all elements of this model to a HuggingFace Hub repository, creating it if it doesn’t exist.

Parameters:

repo_id (str) – Repository name for your model in the Hub, including the user or organization.
token (str, optional) – An authentication token (See https://huggingface.co/settings/token)
private (bool, optional) – Set to true, for hosting a private model
safe_serialization (bool, optional) – If true, save the model using safetensors. If false, save the model the traditional PyTorch way
commit_message (str, optional) – Message to commit while pushing.
local_model_path (str, optional) – Path of the model locally. If set, this file path will be uploaded. Otherwise, the current model will be uploaded
exist_ok (bool, optional) – If true, saving to an existing repository is OK. If false, saving only to a new repository is possible
replace_model_card (bool, optional) – If true, replace an existing model card in the hub with the automatically created model card. If false (default), keep the existing model card if one exists in the repository.
train_datasets (List[str], optional) – Datasets used to train the model. If set, the datasets will be added to the model card in the Hub.
revision (str, optional) – Branch to push the uploaded files to
create_pr (bool, optional) – If True, create a pull request instead of pushing directly to the main branch

Returns:

The url of the commit of your model in the repository on the Hugging Face Hub.

Return type:

str

register_load_state_dict_pre_hook(hook)

Register a pre-hook to be run before module’s load_state_dict() is called.

It should have the following signature::: hook(module, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs) -> None # noqa: B950

Parameters:: hook (Callable) – Callable hook that will be invoked before loading the state dict.

register_state_dict_post_hook(hook)

Register a post-hook for the state_dict() method.

It should have the following signature::: hook(module, state_dict, prefix, local_metadata) -> None

The registered hooks can modify the state_dict inplace.

save_pretrained(path: str, model_name: str | None = None, create_model_card: bool = True, train_datasets: list[str] | None = None, safe_serialization: bool = True) → None[source]

Saves a model and its configuration files to a directory, so that it can be loaded again.

Parameters:

path (str) – Path on disk where the model will be saved.
model_name (str, optional) – Optional model name.
create_model_card (bool, optional) – If True, create a README.md with basic information about this model.
train_datasets (List[str], optional) – Optional list with the names of the datasets used to train the model.
safe_serialization (bool, optional) – If True, save the model using safetensors. If False, save the model the traditional (but unsafe) PyTorch way.

set_adapter(*args, **kwargs) → None[source]

Sets a specific adapter by forcing the model to use that adapter and disable the other adapters.

Parameters:

*args – Positional arguments to pass to the underlying AutoModel set_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.set_adapter
**kwargs – Keyword arguments to pass to the underlying AutoModel set_adapter function. More information can be found in the transformers documentation https://huggingface.co/docs/transformers/main/en/main_classes/peft#transformers.integrations.PeftAdapterMixin.set_adapter

set_submodule(target: str, module: Module, strict: bool = False) → None

Set the submodule given by target if it exists, otherwise throw an error.

Note

If strict is set to False (default), the method will replace an existing submodule or create a new submodule if the parent module exists. If strict is set to True, the method will only attempt to replace an existing submodule and throw an error if the submodule does not exist.

For example, let’s say you have an nn.Module A that looks like this:

A(
    (net_b): Module(
        (net_c): Module(
            (conv): Conv2d(3, 3, 3)
        )
        (linear): Linear(3, 3)
    )
)

(The diagram shows an nn.Module A. A has a nested submodule net_b, which itself has two submodules net_c and linear. net_c then has a submodule conv.)

To override the Conv2d with a new submodule Linear, you could call set_submodule("net_b.net_c.conv", nn.Linear(1, 1)) where strict could be True or False

To add a new submodule Conv2d to the existing net_b module, you would call set_submodule("net_b.conv", nn.Conv2d(1, 1, 1)).

In the above if you set strict=True and call set_submodule("net_b.conv", nn.Conv2d(1, 1, 1), strict=True), an AttributeError will be raised because net_b does not have a submodule named conv.

Parameters:

target – The fully-qualified string name of the submodule to look for. (See above example for how to specify a fully-qualified string.)
module – The module to set the submodule to.
strict – If False, the method will replace an existing submodule or create a new submodule if the parent module exists. If True, the method will only attempt to replace an existing submodule and throw an error if the submodule doesn’t already exist.

Raises:

ValueError – If the target string is empty or if module is not an instance of nn.Module.
AttributeError – If at any point along the path resulting from the target string the (sub)path resolves to a non-existent attribute name or an object that is not an instance of nn.Module.

Compute the all-pairs score matrix between two collections of multi-vector embeddings, using this model’s similarity_fn_name.

Parameters:

embeddings1 (Union[Tensor, ndarray, list]) – Query embeddings, as a list of [num_tokens_i, embedding_dim]-shaped tensors or arrays, a padded [num_embeddings_1, num_tokens, embedding_dim]-shaped tensor, or a single [num_tokens, embedding_dim]-shaped tensor scored as a batch of one.
embeddings2 (Union[Tensor, ndarray, list]) – Document embeddings, in the same forms.
**kwargs –
Forwarded to the scoring function, maxsim() or mean_maxsim(). Particularly useful options include:
- device: Run the scoring on this device. The returned scores stay on the documents’ device either way.
- chunk_elements: Cap how much of the corpus is scored at once. The budget is an element count over this function’s own intermediates, so a value tuned here does not carry over to similarity_pairwise(), which packs pairs instead.
- length_normalize: Divide each score by the number of real query tokens (True scores MeanMaxSim, False plain MaxSim).

Returns:

A [num_embeddings_1, num_embeddings_2]-shaped torch tensor with scores, on the documents’ device.

Return type:

Tensor

Example:

>>> model = MultiVectorEncoder("lightonai/GTE-ModernColBERT-v1")
>>> query_embeddings = model.encode_query(["What is the capital of France?"])
>>> document_embeddings = model.encode_document(["Paris is the capital of France.", "Berlin is the capital of Germany."])
>>> model.similarity(query_embeddings, document_embeddings)
tensor([[..., ...]])

property similarity_fn_name: Literal['maxsim', 'meanmaxsim']: The similarity function used by similarity() and similarity_pairwise(). Set it to "meanmaxsim" for a model trained with length-normalized scoring, so evaluators and the model card score the way training did. Defaults to "maxsim" on first access if not explicitly set.

Compute the pairwise score vector between matched query / document pairs, using this model’s similarity_fn_name.

Parameters:

embeddings1 (Union[Tensor, ndarray, list]) – Query embeddings, as a list of [num_tokens_i, embedding_dim]-shaped tensors or arrays, a padded [num_embeddings, num_tokens, embedding_dim]-shaped tensor, or a single [num_tokens, embedding_dim]-shaped tensor scored as a batch of one.
embeddings2 (Union[Tensor, ndarray, list]) – Document embeddings, in the same forms.
**kwargs –
Forwarded to the scoring function, maxsim_pairwise() or mean_maxsim_pairwise(). Particularly useful options include:
- device: Run the scoring on this device. The returned scores stay on the documents’ device either way.
- chunk_elements: Cap how many pairs are scored at once. The budget is an element count over this function’s own intermediates (each pair also carries a padded query), so a value tuned on similarity() does not carry over.
- length_normalize: Divide each score by the number of real query tokens (True scores MeanMaxSim, False plain MaxSim).

Returns:

A [num_embeddings]-shaped torch tensor with pairwise scores, on the documents’ device.

Return type:

Tensor

start_multi_process_pool(target_devices: list[str] | None = None) → dict[Literal['input', 'output', 'processes'], Any][source]

Starts a multi-process pool to infer with several independent processes.

This method is recommended if you want to predict on multiple GPUs or CPUs. It is advised to start only one process per GPU. This method works together with predict and stop_multi_process_pool.

Parameters:: target_devices (List[str], optional) – PyTorch target devices, e.g. [“cuda:0”, “cuda:1”, …], [“npu:0”, “npu:1”, …], or [“cpu”, “cpu”, “cpu”, “cpu”]. If target_devices is None and CUDA/NPU is available, then all available CUDA/NPU devices will be used. If target_devices is None and CUDA/NPU is not available, then 4 CPU devices will be used.
Returns:: A dictionary with the target processes, an input queue, and an output queue.
Return type:: Dict[str, Any]

static stop_multi_process_pool(pool: dict[Literal['input', 'output', 'processes'], Any]) → None[source]

Stops all processes started with start_multi_process_pool.

Parameters:: pool (Dict[str, object]) – A dictionary containing the input queue, output queue, and process list.
Returns:: None

supports(modality: Literal['text', 'image', 'audio', 'video', 'message'] | tuple[Literal['text', 'image', 'audio', 'video'], ...]) → bool[source]

Check if the model supports the given modality.

A modality is supported if:

It is directly listed in modalities (including tuple modalities that are explicitly listed), or
It is a tuple of modalities (e.g. ("image", "text")) where each part is individually supported and the model also supports "message" format, which is used to combine multiple modalities into a single input.

Parameters:: modality – A single modality string (e.g. "text", "image") or a tuple of modality strings (e.g. ("image", "text")).
Returns:: Whether the model supports the given modality.
Return type:: bool

Example:

>>> from sentence_transformers import SentenceTransformer
>>> model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
>>> model.supports("text")
True
>>> model.supports("image")
False

to(*args, **kwargs)

Move and/or cast the parameters and buffers.

This can be called as

to(device=None, dtype=None, non_blocking=False)

to(dtype, non_blocking=False)

to(tensor, non_blocking=False)

to(memory_format=torch.channels_last)

Its signature is similar to torch.Tensor.to(), but only accepts floating point or complex dtypes. In addition, this method will only cast the floating point or complex parameters and buffers to dtype (if given). The integral parameters and buffers will be moved device, if that is given, but with dtypes unchanged. When non_blocking is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.

See below for examples.

Note

This method modifies the module in-place.

Parameters:

device (torch.device) – the desired device of the parameters and buffers in this module
dtype (torch.dtype) – the desired floating point or complex dtype of the parameters and buffers in this module
tensor (torch.Tensor) – Tensor whose dtype and device are the desired dtype and device for all parameters and buffers in this module
memory_format (torch.memory_format) – the desired memory format for 4D parameters and buffers in this module (keyword only argument)

Returns:

self

Return type:

Module

Examples:

>>> # xdoctest: +IGNORE_WANT("non-deterministic")
>>> linear = nn.Linear(2, 2)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
        [-0.5113, -0.2325]])
>>> linear.to(torch.double)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
        [-0.5113, -0.2325]], dtype=torch.float64)
>>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA1)
>>> gpu1 = torch.device("cuda:1")
>>> linear.to(gpu1, dtype=torch.half, non_blocking=True)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
        [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1')
>>> cpu = torch.device("cpu")
>>> linear.to(cpu)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
        [-0.5112, -0.2324]], dtype=torch.float16)

>>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble)
>>> linear.weight
Parameter containing:
tensor([[ 0.3741+0.j,  0.2382+0.j],
        [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128)
>>> linear(torch.ones(3, 2, dtype=torch.cdouble))
tensor([[0.6122+0.j, 0.1150+0.j],
        [0.6122+0.j, 0.1150+0.j],
        [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)

tokenize(texts: list[str] | list[dict] | list[tuple[str, str]], **kwargs) → dict[str, Tensor][source]: Deprecated since version `tokenize`: is deprecated. Use preprocess instead.

property tokenizer: Any: Property to get the tokenizer that is used by this model

train(mode: bool = True) → Self

Set the module in training mode.

This has an effect only on certain modules. See the documentation of particular modules for details of their behaviors in training/evaluation mode, i.e., whether they are affected, e.g. Dropout, BatchNorm, etc.

Parameters:: mode (bool) – whether to set training mode (True) or evaluation mode (False). Default: True.
Returns:: self
Return type:: Module

property transformers_model: PreTrainedModel | None

Property to get the underlying transformers PreTrainedModel instance, if it exists. Note that it’s possible for a model to have multiple underlying transformers models, but this property will return the first one it finds in the module hierarchy.

Note

This property can also return e.g. ORTModelForFeatureExtraction or OVModelForFeatureExtraction instances from the optimum-intel and optimum-onnx libraries, if the model is loaded using backend="onnx" or backend="openvino".

Returns:: The underlying transformers model or None if not found.
Return type:: PreTrainedModel or None

MultiVectorEncoderModelCardData

class sentence_transformers.multi_vector_encoder.model_card.MultiVectorEncoderModelCardData(language: str | list[str] | None = <factory>, license: str | None = None, model_name: str | None = None, model_id: str | None = None, train_datasets: list[dict[str, str]] = <factory>, eval_datasets: list[dict[str, str]] = <factory>, task_name: str | None = None, tags: list[str] = <factory>, local_files_only: bool = False, generate_widget_examples: bool = True)[source]

A dataclass storing data used in the model card for MultiVectorEncoder models.

Parameters:

language (Optional[Union[str, List[str]]]) – The model language, either a string or a list, e.g. “en” or [“en”, “de”, “nl”]
license (Optional[str]) – The license of the model, e.g. “apache-2.0”, “mit”, or “cc-by-nc-sa-4.0”
model_name (Optional[str]) – The pretty name of the model, e.g. “MultiVectorEncoder based on answerdotai/ModernBERT-base”.
model_id (Optional[str]) – The model ID when pushing the model to the Hub, e.g. “tomaarsen/mve-modernbert-base-ms-marco”.
train_datasets (List[Dict[str, str]]) – A list of the names and/or Hugging Face dataset IDs of the training datasets, e.g. [{"name": "MS MARCO", "id": "microsoft/ms_marco"}].
eval_datasets (List[Dict[str, str]]) – A list of the names and/or Hugging Face dataset IDs of the evaluation datasets.
task_name (str) – The human-readable task the model is trained on, e.g. “semantic search with late interaction”.
tags (Optional[List[str]]) – A list of tags for the model, e.g. ["sentence-transformers", "multi-vector", "colbert", "late-interaction"].
local_files_only (bool) – If True, don’t attempt to find dataset or base model information on the Hub.
generate_widget_examples (bool) – If True, generate widget examples from the evaluation or training dataset.

Tip

Install codecarbon to automatically track carbon emission usage and include it in your model cards.

generate_usage_snippet() → str[source]

Generate the Python usage code snippet for the model card.

Returns the code block (including ``` delimiters) showing how to use this model. Called after run_usage_snippet() has set usage_examples and similarities.

Subclasses can override this to generate snippets for different model types (e.g. IR models, cross-encoders) or multimodal inputs.

pipeline_tag: str = None

task_name: str | None = None