replace_regex
Replace parts of text detected with a regular expression.
A regular expression (or regex, regex pattern) is a sequence of characters that forms a search pattern. This
pattern is compared against texts, and any matches are substituted by a desired replacement. The replacement
can be a simple (constant) text string, or a formatting pattern referencing all or parts of the matched character
sequence.
Simple replacement of fixed text strings with another fixed text string can be performed easily. E.g., to
replace all occurrences of “hi” with “hello”, you’d simply use
{"pattern": "hi", "replacement": "hello"}.
However, using capturing groups in pattern and replacement parameters allows for much greater flexibility.
For example, if a column of texts includes twitter mentions of the form “@abc”, the regular expression
"pattern": "@(\\w*)" will match these mentions and save the actual name without the ”@” character in a capturing
group. Using the replacement string "replacement": "{1}" will then replace all matched mentions with only the name
part of the twitter handle, effectively removing the ”@” tags from all mentions (without removing other occurrences of
the ”@” character).
To further familiarize yourself with the regex language also see these references:
Usage
The following example shows how the step can be used in a recipe.Examples
Examples
To change the way dates are formatted in a column of texts from “2019-04-15” to “15.04.2019”:
The specified pattern will match 3 numbers separated by the minus sign, and will replace such occurences by the same
three numbers in reverse order and separated with a period.
Inputs & Outputs
The following are the inputs expected by the step and the outputs it produces. These are generally columns (ds.first_name), datasets (ds or ds[["first_name", "last_name"]]) or models (referenced
by name e.g. "churn-clf").
Inputs
Inputs
A column containing text-like values.
Outputs
Outputs
The output column’s data type will depend on the input and specified parameters:
text: if input is text and parameter"as_category": falsecategory: if input is not a column of lists and"as_category": truelist: if input is a column of lists.
Configuration
The following parameters can be used to configure the behaviour of the step by including them in a json object as the last “input” to the step, i.e.step(..., {"param": "value", ...}) -> (output).
Parameters
Parameters
Regular expression to be matched in your input texts.
The regex
pattern may include (numbered) regex capturing groups,
which allows this method to use parts of a match to format the way matches are then replaced in the output via the replacement
parameter (see below).A format string determining how matches will be replaced in the text.
The replacement string allows for use of Google-RE2 style string replacement with curly braces and numerical identifiers, e.g. "" instead
of the usual regex syntax using backslashes (“\1”). Numerical identifiers refer to capturing groups in the regex pattern (named groups are
not supported), where
- 0 is the whole match
- 1 is the 1st capturing group
- 2 is the 2nd capturing group
- etc…
Regex configuration flags.
Uses Python-style regex flags determining how to perform matches, e.g. [“a”, “ignorecase”]
Note that flags not present in Google-RE2 implementation will be ignored.
Array items
Array items
Each item in array.Values must be one of the following:
a ascii debug i ignorecase l locale m multiline s dotall x verboseWhether to cast result to category data type when otherwise it would be texts.