A regular expression (or regex, regex pattern) is a sequence of characters that forms a search pattern. This pattern is compared against texts, and any matches are substituted by a desired replacement. The replacement can be a simple (constant) text string, or a formatting pattern referencing all or parts of the matched character sequence.

Simple replacement of fixed text strings with another fixed text string can be performed easily. E.g., to replace all occurrences of “hi” with “hello”, you’d simply use {"pattern": "hi", "replacement": "hello"}. However, using capturing groups in pattern and replacement parameters allows for much greater flexibility. For example, if a column of texts includes twitter mentions of the form “@abc”, the regular expression "pattern": "@(\\w*)" will match these mentions and save the actual name without the ”@” character in a capturing group. Using the replacement string "replacement": "{1}" will then replace all matched mentions with only the name part of the twitter handle, effectively removing the ”@” tags from all mentions (without removing other occurrences of the ”@” character).

To further familiarize yourself with the regex language also see these references:

pattern
string
required

Regular expression to be matched in your input texts. The regex pattern may include (numbered) regex capturing groups, which allows this method to use parts of a match to format the way matches are then replaced in the output via the replacement parameter (see below).

replacement
string

A format string determining how matches will be replaced in the text. The replacement string allows for use of Google-RE2 style string replacement with curly braces and numerical identifiers, e.g. "" instead of the usual regex syntax using backslashes (“\1”). Numerical identifiers refer to capturing groups in the regex pattern (named groups are not supported), where

  • 0 is the whole match
  • 1 is the 1st capturing group
  • 2 is the 2nd capturing group
  • etc…

The default is the empty string "", i.e. matched parts will be removed from the text.

flags
array[string]

Regex configuration flags. Uses Python-style regex flags determining how to perform matches, e.g. [“a”, “ignorecase”] Note that flags not present in Google-RE2 implementation will be ignored.

as_category
boolean
default: "true"

Whether to cast result to category data type when otherwise it would be texts.

Was this page helpful?