Skip to main content
Uses Spotify’s Annoy to perform approximate nearest neighbour search.

​
Usage

The following examples show how the step can be used in a recipe.

Examples

To link similar embeddings with default configuration

​
Inputs & Outputs

The following are the inputs expected by the step and the outputs it produces. These are generally columns (ds.first_name), datasets (ds or ds[["first_name", "last_name"]]) or models (referenced by name e.g. "churn-clf").
​
column[list[number]]
required
A categorical column containing embeddings (numerical vectors/lists). Usually the result of previously executing a step embed_[entity].
​
column
required
A column containing for each row a list of IDs (row numbers) identfying other rows it will be linked to.
​
column
required
A column containing for each row a list of weights identfying the “importance” of each link to targets identified in the targets column.

​
Configuration

The following parameters can be used to configure the behaviour of the step by including them in a json object as the last “input” to the step, i.e. step(..., {"param": "value", ...}) -> (output).

Parameters

​
integer
default:"15"
Number of nearest neighbours to connect to.Values must be in the following range:
​
number
default:"0"
Minimum similarity for connecting two nodes.Values must be in the following range:
​
number
default:"0"
Minimum similarity for connecting two nodes, expressed as a quantile of the similarity distribution.Values must be in the following range:
​
integer
default:"30"
Number of trees. Affects the build time and the index size. A larger value will give more accurate results, but will take longer to create a larger index.
​
integer
default:"2"
Accuracy multipler. A larger value will give more accurate results, but will take longer time to return.
​
string
default:"angular"
Metric to use, only angular supported for now. Annoy’s angular metric is equivalent to sqrt(2*(1-cos(u,v))), whose max. is sqrt(2*2) = 2. I.e. the distance between (1,0) and (-1,0), at maximum angular separation, should be exactly 2 Note that for the weights of the resulting network links Annoy’s distances are converted to similarities in the interval [0,1].Values must be one of the following:
  • angular
  • euclidean
  • manhattan
  • hamming
  • dot
​
number
Used to seed the random number generator, creating deterministic results.