> ## Documentation Index > Fetch the complete documentation index at: https://docs.graphext.com/llms.txt > Use this file to discover all available pages before exploring further. # link_similar_columns > Calculates all pair-wise column dependencies (by default mutual information). Create a new network dataset where nodes (rows) represent the original dataset's variables (its columns), and links represent dependencies between variables (i.e. associations/correlations). By default the step measures ["mutual information"](https://en.wikipedia.org/wiki/Mutual_information) between variables. In effect, all pair-wise "correlations" between the original dataset's columns are calculated. A threshold is then applied to extract only the largest (most interesting) "correlations". These are then translated into network links between nodes representing the original variables. Each node/row in the new dataset will also contain information about its correlation with all other nodes (variables). Note: in this first version, only quantitative and categorical variables will be analyzed (but not tags, lists, embeddings etc.). You can pass a dataset with arbitrary column types, but those not supported by the selected correlation method will be ignored in the result. ## Usage The following examples show how the step can be used in a recipe. The following example calculates all correlations without filtering, leading to a fully connected correlation network unless some correlations are exactly 0. ```stan theme={null} links_similar_columns(ds) -> (corrs) ``` The following example removes correlations below the 75th percentile, creating a network only connecting the most correlated variables. ```stan theme={null} links_similar_columns(ds, { "min_similarity_quantile": 0.75, }) -> (corrs) ``` General syntax for using the step in a recipe. Shows the inputs and outputs the step is expected to receive and will produce respectively. For futher details see sections below. ```stan theme={null} link_similar_columns(ds_in: dataset, { "param": value, ... }) -> (ds_out: dataset) ``` ## Inputs & Outputs The following are the inputs expected by the step and the outputs it produces. These are generally columns (`ds.first_name`), datasets (`ds` or `ds[["first_name", "last_name"]]`) or models (referenced by name e.g. `"churn-clf"`). Input dataset containing arbitrary columns to calculate "correlations" for. A dataset containing M rows and M+2 columns (where M is the number of columns in the input dataset). Each row represents a variable in the original dataset, and the columns contain the "correlations" with the remaining variables. An additional 2 columns ("targets" and "weights") contain links connecting original variables to other variables they're correlated with. ## Configuration The following parameters can be used to configure the behaviour of the step by including them in a json object as the last "input" to the step, i.e. `step(..., {"param": "value", ...}) -> (output)`. Correlation method/statistic. Which statistic to use to measure variable association/correlation. The default is `"mutual_information"`, which applies scikit-learn\`s k-nearest neighbors implementations (see [here](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.mutual_info_classif.html) and [here](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.mutual_info_regression.html)). It supports both categorical and quantitative variables, is relatively fast, but doesn't have a natural upper bound (i.e. values are not in the range \[0, 1]). The `"distance_correlation"` [method](https://en.wikipedia.org/wiki/Distance_correlation) also supports both categorical and quantitative variables, and has a natural upper bound of 1. It's relatively slow though, so make sure to select a reasonable value for `n_samples`. `"distance_correlation_fast"` uses an [optimized implementation](https://dcor.readthedocs.io/en/stable/index.html) of distance correlation, but only supports quantitative variables. `"pearson"` is the standard [Pearson correlation coefficient](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient), which also only supports quantitative variables. Lastly, `"predictive_power"` calculates a version of the [predictive power score (PPS)](https://github.com/8080labs/ppscore/). This essentially fits a decision tree to predict variable y using only variable x as a predictor, and measures it performance relative to a dummy/baseline prediction. It supports both categorical and quantitative variables. Values must be one of the following: * `mutual_information` * `distance_correlation` * `distance_correlation_fast` * `pearson` * `predictive_power` Absolute similarity threshold. The minimum "correlation" for the creation of a link between two variables. Similarity threshold expressed as a quantile. E.g. a value of 0.6 means the bottom 60% of "correlations" will be discarded. Both minima (absolute and quantile) must be exceeded for a link to be created. Values must be in the following range: ```javascript theme={null} 0.0 ≤ min_similarity_quantile ≤ 1 ``` Replacement for discarded correlations. The weight of links whose correlations don't pass the minimum threshold. Links with weights of `null` will be discarded (the default behavior). Can be set e.g. to 0, to generate all possible links. Number of Samples. It represents the maximum number of samples to use when measuring "correlations". Maximum number of links per node. Ranks the links by weight and keeps only the most similar targets. The random seed used if applicable for selected `method`.