> ## Documentation Index
> Fetch the complete documentation index at: https://docs.graphext.com/llms.txt
> Use this file to discover all available pages before exploring further.

# filter_duplicate_nodes

> Remove duplicate nodes in a network. 

For each pair of nodes connected by a link that indicates a similarity greater than a specified threshold, keeps only
one of the two nodes and rewires the deleted node's incoming and outgoing links to point to the "surviving" node.

## Usage

The following example shows how the step can be used in a recipe.

<Accordion title="Examples" icon="code" defaultOpen="true">
  <Tabs>
    <Tab title="Example 1">
      To de-duplicate pairs of nodes with a link weight (similarity) greater than 0.9

      ```stan theme={null}
      filter_duplicate_nodes(network, {
        "duplicate_threshold": 0.9
      }) -> (network_filtered)
      ```
    </Tab>

    <Tab title="Signature">
      General syntax for using the step in a recipe. Shows the inputs and outputs the step is expected to receive and will produce respectively. For futher details see sections below.

      ```stan theme={null}
      filter_duplicate_nodes(network: dataset, {
          "param": value,
          ...
      }) -> (network_flt: dataset)
      ```
    </Tab>
  </Tabs>
</Accordion>

## Inputs & Outputs

The following are the inputs expected by the step and the outputs it produces. These are generally
columns (`ds.first_name`), datasets (`ds` or `ds[["first_name", "last_name"]]`) or models (referenced
by name e.g. `"churn-clf"`).

<Accordion title="Inputs" icon="right-to-bracket">
  <ParamField path="network" type="dataset" required>
    dataset containing the *nodes* (rows) to de-duplicate and the *links* between nodes of the input dataset.
  </ParamField>
</Accordion>

<Accordion title="Outputs" icon="right-from-bracket">
  <ParamField path="network_flt" type="dataset" required>
    A new dataset containing the same columns as the input `data`, but without duplicate rows and having connections rewired such that none
    points to a deleted node.
  </ParamField>
</Accordion>

## Configuration

The following parameters can be used to configure the behaviour of the step by including them in
a json object as the last "input" to the step, i.e. `step(..., {"param": "value", ...}) -> (output)`.

<Accordion title="Parameters" defaultOpen="true" icon="sliders">
  <ParamField path="duplicate_threshold" type="[number, null]" required>
    Similarity threshold for candidate nodes to be eliminated.
    Any node linked to another node with a weight (usually similarity) greater than this value
    will be eliminated. Default (`null`) corresponds to positive infinity (no de-duplication).
  </ParamField>
</Accordion>