Derive column

fast step  derive

Derive a new column with a custom JS script.

Supports any JS script using ECMAScript 2020 syntax. The script should have a return clause returning either a value or null / undefined. The script has access to a row object that represent a row in the dataset and have the column names as keys. Lists are supported both as inputs and outputs.

Usage

---

The following are the step's expected inputs and outputs and their specific types.

Step signature

derive_column(ds_in: dataset, {"param": value}) -> (new_col: column)

where the object {"param": value} is optional in most cases and if present may contain any of the parameters described in the corresponding section below.

Example

The following example joins all values in a list of numbers with '|' as separator:

Example call (in recipe editor)

derive_column(ds_in, {
  "script": "return row.numCol?.join(' | ');",
  "type": "text"
}) -> (new_col)

More examples

The following example computes the sum for a list of numbers:

Example call (in recipe editor)

derive_column(ds_in, {
  "script": "return row.numCol?.reduce((sum, n) => sum + n, 0);",
  "type": "category"
}) -> (new_col)

The following example adds a prefix to a category:

Example call (in recipe editor)

derive_column(ds_in, {
  "script": "return row.cat != null ? `Prefix_${row.cat}` : null;"
}) -> (new_col)

The following example extracts a regex from a text:

Example call (in recipe editor)

derive_column(ds_in, {
  "script": "return row.text?.match(/\d+/);",
  "type": "category"
}) -> (new_col)

The following example extracts the domain from a URL column:

Example call (in recipe editor)

derive_column(ds_in, {
  "script": "return row.url != null ? new URL(row.url).hostname } : null;",
  "type": "category"
}) -> (new_col)

The following example extracts the year component from a Date column:

Example call (in recipe editor)

derive_column(ds_in, {
  "script": "return row.dateCol != null ? new Date(row.dateCol).getUTCFullYear() : null;",
  "type": "category"
}) -> (new_col)

Inputs

---

ds_in: dataset

An input dataset.

Outputs

---

new_col: column

The column resulting from evaluating the script.

Parameters

---

script: string

The javascript code to execute.

Example parameter values:

• For example, to multiply by 2 every row with a value:

  "return row.num * 2;"
  

---

type: string

Output column type. Select the desired type using a shortened yet fully specified name.

Must be one of: "boolean", "category", "date", "number", "text", "url", "list[number]", "list[category]", "list[url]"