Derive column¶
fast step derive
Derive a new column with a custom JS script.
Supports any JS script using ECMAScript 2020 syntax. The script should have a return
clause returning either a value or null / undefined.
The script has access to a row
object that represent a row in the dataset and have the column names as keys.
Lists are supported both as inputs and outputs.
It's important to correctly manage null values by checking for null (e.g. if (row.col != null) { ... }
) or using the JS optional chaining operator (?
).
Usage¶
The following are the step's expected inputs and outputs and their specific types.
derive_column(ds: dataset, {"param": value}) -> (new_col: column)
where the object {"param": value}
is optional in most cases and if present may contain any of the parameters described in the
corresponding section below.
Example¶
The following example joins all values in a list of numbers with '|' as separator:
derive_column(ds, {
"script": "return row.numCol?.join(' | ');",
"type": "text"
}) -> (ds.new_col)
More examples
The following example computes the sum for a list of numbers:
derive_column(ds, {
"script": "return row.numCol?.reduce((sum, n) => sum + n, 0);",
"type": "category"
}) -> (ds.new_col)
The following example adds a prefix to a category:
derive_column(ds, {
"script": "return row.cat != null ? `Prefix_${row.cat}` : null;"
}) -> (ds.new_col)
The following example extracts a regex from a text:
derive_column(ds, {
"script": "return row.text?.match(/\d+/);",
"type": "category"
}) -> (ds.new_col)
The following example extracts the domain from a URL column:
derive_column(ds, {
"script": "return row.url != null ? new URL(row.url).hostname } : null;",
"type": "category"
}) -> (ds.new_col)
The following example extracts the year component from a Date column:
derive_column(ds, {
"script": "return row.dateCol != null ? new Date(row.dateCol).getUTCFullYear() : null;",
"type": "number"
}) -> (ds.new_col)
Inputs¶
ds: dataset
An input dataset.
Outputs¶
new_col: column
The column resulting from evaluating the script.
Parameters¶
script: string
The javascript code to execute.
Example parameter values:
-
For example, to multiply by 2 every row with a value:
"return row.num * 2;"
type: string
Output column type. Select the desired type using a shortened yet fully specified name.
Must be one of:
"boolean"
,
"category"
,
"date"
,
"number"
,
"text"
,
"url"
,
"list[number]"
,
"list[category]"
,
"list[url]"