Skip to content

Derive column

fast step  derive

Derive new columns with a custom JS script.

Supported any JS script using ECMAScript 2020 syntax. The script should have a return clause returning either a value or null / undefined. The script has access to a row object that represent a row in the dataset and have the column names as keys. Lists are supported both as inputs and outputs.

Usage


The following are the step's expected inputs and outputs and their specific types.

Step signature
derive_column(ds_in: dataset, {"param": value}) -> (new_col: column)

where the object {"param": value} is optional in most cases and if present may contain any of the parameters described in the corresponding section below.

Example

The following example joins all values in a list of numbers with '|' as separator:

Example call (in recipe editor)
derive_column(ds_in, {
  "script": "return row.numCol?.join(' | ');"
}) -> (new_col)
More examples

The following example computes the sum for a list of numbers:

Example call (in recipe editor)
derive_column(ds_in, {
  "script": "return row.numCol?.reduce((sum, n) => sum + n, 0);"
}) -> (new_col)

The following example adds a prefix to a category:

Example call (in recipe editor)
derive_column(ds_in, {
  "script": "return row.cat != null ? `Prefix_${row.cat}` : null;"
}) -> (new_col)

The following example extracts a regex from a text:

Example call (in recipe editor)
derive_column(ds_in, {
  "script": "return row.text?.match(/\d+/);"
}) -> (new_col)

The following example extracts the domain from a URL column:

Example call (in recipe editor)
derive_column(ds_in, {
  "script": "return row.url != null ? new URL(row.url).hostname } : null;"
}) -> (new_col)

The following example extracts the year component from a Date column:

Example call (in recipe editor)
derive_column(ds_in, {
  "script": "return row.dateCol != null ? new Date(row.dateCol).getUTCFullYear() : null;"
}) -> (new_col)

Inputs


ds_in: dataset

An input dataset.

Outputs


new_col: column

The column resulting from evaluating the script.

Parameters


script: string

The javascript code to execute.

Example parameter values:

  • For example, to multiply by 2 every row with a value:

    "return row.num * 2;"
    

type: string

Output column type. Select the desired type using a shortened yet fully specified name.

Must be one of: "category", "date", "number", "text", "url", "list[number]", "list[category]", "list[url]"