replace_regex
Replace parts of text detected with a regular expression.
A regular expression (or regex, regex pattern) is a sequence of characters that forms a search pattern. This pattern is compared against texts, and any matches are substituted by a desired replacement. The replacement can be a simple (constant) text string, or a formatting pattern referencing all or parts of the matched character sequence.
Simple replacement of fixed text strings with another fixed text string can be performed easily. E.g., to
replace all occurrences of “hi” with “hello”, you’d simply use {"pattern": "hi", "replacement": "hello"}.
However, using capturing groups in pattern and replacement parameters allows for much greater flexibility.
For example, if a column of texts includes twitter mentions of the form “@abc”, the regular expression
"pattern": "@(\\w*)" will match these mentions and save the actual name without the ”@” character in a capturing
group. Using the replacement string "replacement": "{1}" will then replace all matched mentions with only the name
part of the twitter handle, effectively removing the ”@” tags from all mentions (without removing other occurrences of
the ”@” character).
To further familiarize yourself with the regex language also see these references:
Usage
The following example shows how the step can be used in a recipe.
To change the way dates are formatted in a column of texts from “2019-04-15” to “15.04.2019”: The specified pattern will match 3 numbers separated by the minus sign, and will replace such occurences by the same three numbers in reverse order and separated with a period.
To change the way dates are formatted in a column of texts from “2019-04-15” to “15.04.2019”: The specified pattern will match 3 numbers separated by the minus sign, and will replace such occurences by the same three numbers in reverse order and separated with a period.
General syntax for using the step in a recipe. Shows the inputs and outputs the step is expected to receive and will produce respectively. For futher details see sections below.
Inputs & Outputs
The following are the inputs expected by the step and the outputs it produces. These are generally
columns (ds.first_name), datasets (ds or ds[["first_name", "last_name"]]) or models (referenced
by name e.g. "churn-clf").
Configuration
The following parameters can be used to configure the behaviour of the step by including them in
a json object as the last “input” to the step, i.e. step(..., {"param": "value", ...}) -> (output).
Regular expression to be matched in your input texts.
The regex pattern may include (numbered) regex capturing groups,
which allows this method to use parts of a match to format the way matches are then replaced in the output via the replacement
parameter (see below).
A format string determining how matches will be replaced in the text. The replacement string allows for use of Google-RE2 style string replacement with curly braces and numerical identifiers, e.g. "" instead of the usual regex syntax using backslashes (“\1”). Numerical identifiers refer to capturing groups in the regex pattern (named groups are not supported), where
- 0 is the whole match
- 1 is the 1st capturing group
- 2 is the 2nd capturing group
- etc…
The default is the empty string "", i.e. matched parts will be removed from the text.
Regex configuration flags. Uses Python-style regex flags determining how to perform matches, e.g. [“a”, “ignorecase”] Note that flags not present in Google-RE2 implementation will be ignored.
Whether to cast result to category data type when otherwise it would be texts.