Skip to main content
An embedding vector is a numerical representation of a text, such that different numerical components of the vector capture different dimensions of the text’s meaning. Embeddings can be used, for example, to calculate the semantic similarity between pairs of texts. See link_embeddings, for example, to create a network of texts connected by similarity. In this step, embeddings of texts are calculated using pre-trained neural language models, especially those using the popular transformer architecture (e.g. Bert-based models).

​
Things to keep in mind

  • Unlike embed_text, which uses a different, appropriate spaCy model for each language in the text column, this step will always use a single model only to calculate embeddings. This means the model should be multilingual if you have mixed languages, and that otherwise you need to choose the correct model for your (single) language.
  • Each model will be downloaded on the fly before processing the text. This adds a little lag to its execution time (the bigger the model the longer the download), though for a sufficient number of texts the time spent downloading should not be significant. Note also, however, that the download, and therefore this step, may fail if the servers of its publisher are not responsive.
  • Since this step potentially supports tens if not hundreds of different models, we cannot provide support or advice on specific models.

​
Usage

The following example shows how the step can be used in a recipe.

Examples

  • Example 1
  • Signature
To calculate embeddings using a multilingual sentence-bert model (from sentence-transformers):
embed_text_with_model(ds.text, {"collection": "SBERT", "name": "distiluse-base-multilingual-cased-v2"}) -> (ds.embedding)

​
Inputs & Outputs

The following are the inputs expected by the step and the outputs it produces. These are generally columns (ds.first_name), datasets (ds or ds[["first_name", "last_name"]]) or models (referenced by name e.g. "churn-clf").
​
text
column[text]
required
A text column to calculate embeddings for.
​
embedding
column[list[number]]
required
A column of embedding vectors capturing the meaning of each input text.

​
Configuration

The following parameters can be used to configure the behaviour of the step by including them in a json object as the last “input” to the step, i.e. step(..., {"param": "value", ...}) -> (output).

Parameters

  • Sentence-BERT
  • Hugging Face
​
collection
string
default:"SBERT"
required
Embed texts using a Sentence-BERT model. Models in this collection (also known as sentence-transformers) have been trained specifically for semantic similarity, i.e. for the purpose of comparing the meaning of texts. Individual models in this collection can be found here: https://www.sbert.net/docs/pretrained_models.html. They differ in terms of the language they have been trained on; their size (the bigger the better usually, but also the slower); as well as their purpose or intended area of application (e.g. it has a specific model to embed scientific publications).
​
name
string
required
A specific Sentence-BERT model name. To find a model appropriate for your data or task, check the website of the Sentence-BERT model collection.
  • paraphrase-MiniLM-L6-v2
  • distiluse-base-multilingual-cased-v2
​
normalize
boolean
default:"true"
Whether text embedding vectors should be normalized (to lengths of 1.0). This may make similarity calculations easier. E.g. we can then use the dot product as a similarity “metric”, instead of the usual cosine angle (which not all downstream functions may support).
​
batch_size
integer
default:"32"
How many texts to push through the model at the same time. Greater values usually mean faster processing (if supported by the model), but also greater use of memory.Values must be in the following range:
1 ≤ batch_size < inf
⌘I