bat365 Homepage

regexprep

Replace text in words of documents using regular expression

Syntax

newDocuments = regexprep(documents,expression,replace)

Description

Text Analytics Toolbox™ provides functions for common text preprocessing steps. For example, to remove punctuation and symbol characters, use erasePunctuation or to remove stem words using the Porter stemmer, use normalizeWords. For more information, see Text Data Preparation.

example

newDocuments = regexprep(documents,expression,replace) replaces all occurrences of the regular expression expression in the words of documents with the text in replace.

The function matches each word independently. The match does not have to span the whole word.

Examples

collapse all

Update Text in Words

Open Live Script

Replace words that begin with "s", end "e", and have at least one character between them. To match whole words, use "^" to match the start of a word and "$" to match the end of the word.

documents = tokenizedDocument([ ...
    "an example of a short sentence" 
    "a second short sentence"])

documents = 
  2x1 tokenizedDocument:

    6 tokens: an example of a short sentence
    4 tokens: a second short sentence

expression = "^s(\w+)e$";
replace = "thing";
newDocuments = regexprep(documents,expression,replace)

newDocuments = 
  2x1 tokenizedDocument:

    6 tokens: an example of a short thing
    4 tokens: a second short thing

If you do not use "^" and "$", then you can match substrings of the words. Replace all vowels with "_".

expression = "[aeiou]";
replace = "\_";
newDocuments = regexprep(documents,expression,replace)

newDocuments = 
  2x1 tokenizedDocument:

    6 tokens: _n _x_mpl_ _f _ sh_rt s_nt_nc_
    4 tokens: _ s_c_nd sh_rt s_nt_nc_

Include Captured Tokens in Word Replacement

Open Live Script

Replace variations of the word "walk" by capturing the letters that follow "walk".

documents = tokenizedDocument([
    "I walk"
    "they walked"
    "we are walking"])

documents = 
  3x1 tokenizedDocument:

    2 tokens: I walk
    2 tokens: they walked
    3 tokens: we are walking

expression = "walk(\w*)";
replace = "ascend$1";
newDocuments = regexprep(documents,expression,replace)

newDocuments = 
  3x1 tokenizedDocument:

    2 tokens: I ascend
    2 tokens: they ascended
    3 tokens: we are ascending

Input Arguments

collapse all

`documents` — Input documents
`tokenizedDocument` array

Input documents, specified as a tokenizedDocument array.

`expression` — Regular expression
character vector | cell array of character vectors | string array

Regular expression, specified as a character vector, a cell array of character vectors, or a string array. Each expression can contain characters, metacharacters, operators, tokens, and flags that specify patterns to match in str.

The following tables describe the elements of regular expressions.

Metacharacters

Metacharacters represent letters, letter ranges, digits, and space characters. Use them to construct a generalized pattern of characters.

Metacharacter	Description	Example
`.`	Any single character, including white space	`'..ain'` matches sequences of five consecutive characters that end with `'ain'`.
`[c₁c₂c₃]`	Any character contained within the square brackets. The following characters are treated literally: `$ \| . * + ?` and `-` when not used to indicate a range.	`'[rp.]ain'` matches `'rain'` or `'pain'` or `'.ain'`.
`[^c₁c₂c₃]`	Any character not contained within the square brackets. The following characters are treated literally: `$ \| . * + ?` and `-` when not used to indicate a range.	`'[^rp]ain'` matches all four-letter sequences that end in `'ain'`, except `'rain'` and `'pain'` and `'ain'`. For example, it matches `'gain'`, `'lain'`, or `'vain'`.
`[c`₁`-c`₂`]`	Any character in the range of `c`₁ through `c`₂	`'[A-G]'` matches a single character in the range of `A` through `G`.
`\w`	Any alphabetic, numeric, or underscore character. For English character sets, `\w` is equivalent to `[a-zA-Z_0-9]`	`'\w*'` identifies a word comprised of any grouping of alphabetic, numeric, or underscore characters.
`\W`	Any character that is not alphabetic, numeric, or underscore. For English character sets, `\W` is equivalent to `[^a-zA-Z_0-9]`	`'\W*'` identifies a term that is not a word comprised of any grouping of alphabetic, numeric, or underscore characters.
`\s`	Any white-space character; equivalent to `[ \f\n\r\t\v]`	`'\w*n\s'` matches words that end with the letter `n`, followed by a white-space character.
`\S`	Any non-white-space character; equivalent to `[^ \f\n\r\t\v]`	`'\d\S'` matches a numeric digit followed by any non-white-space character.
`\d`	Any numeric digit; equivalent to `[0-9]`	`'\d*'` matches any number of consecutive digits.
`\D`	Any nondigit character; equivalent to `[^0-9]`	`'\w*\D\>'` matches words that do not end with a numeric digit.
`\oN` or `\o{N}`	Character of octal value `N`	`'\o{40}'` matches the space character, defined by octal `40`.
`\xN` or `\x{N}`	Character of hexadecimal value `N`	`'\x2C'` matches the comma character, defined by hex `2C`.

Character Representation

Operator	Description
`\a`	Alarm (beep)
`\b`	Backspace
`\f`	Form feed
`\n`	New line
`\r`	Carriage return
`\t`	Horizontal tab
`\v`	Vertical tab
`\char`	Any character with special meaning in regular expressions that you want to match literally (for example, use `\\` to match a single backslash)

Quantifiers

Quantifiers specify the number of times a pattern must occur in the matching text.

Quantifier	Number of Times Expression Occurs	Example
`expr*`	0 or more times consecutively.	`'\w*'` matches a word of any length.
`expr?`	0 times or 1 time.	`'\w*(\.m)?'` matches words that optionally end with the extension `.m`.
`expr+`	1 or more times consecutively.	`'<img src="\w+\.gif">'` matches an `<img>` HTML tag when the file name contains one or more characters.
`expr{m,n}`	At least `m` times, but no more than `n` times consecutively. `{0,1}` is equivalent to `?`.	`'\S{4,8}'` matches between four and eight non-white-space characters.
`expr{m,}`	At least `m` times consecutively. `{0,}` and `{1,}` are equivalent to `*` and `+`, respectively.	`'<a href="\w{1,}\.html">'` matches an `<a>` HTML tag when the file name contains one or more characters.
`expr{n}`	Exactly `n` times consecutively. Equivalent to `{n,n}`.	`'\d{4}'` matches four consecutive digits.

Quantifiers can appear in three modes, described in the following table. q represents any of the quantifiers in the previous table.

Mode	Description	Example
`expr`q	Greedy expression: match as many characters as possible.	Given the text `'<tr><td><p>text</p></td>'`, the expression `'</?t.*>'` matches all characters between `<tr` and `/td>`: `'<tr><td><p>text</p></td>'`
`expr`q`?`	Lazy expression: match as few characters as necessary.	Given the text`'<tr><td><p>text</p></td>'`, the expression `'</?t.*?>'` ends each match at the first occurrence of the closing angle bracket (`>`): `'<tr>' '<td>' '</td>'`
`expr`q+	Possessive expression: match as much as possible, but do not rescan any portions of the text.	Given the text`'<tr><td><p>text</p></td>'`, the expression `'</?t.+>'` does not return any matches, because the closing angle bracket is captured using `.`, and is not rescanned.

Mode

Description

Example

exprq

Greedy expression: match as many characters as possible.

Given the text '<tr><td>text</td>', the expression '</?t.*>' matches all characters between <tr and /td>:

'<tr><td><p>text</p></td>'

exprq?

Lazy expression: match as few characters as necessary.

Given the text'<tr><td>text</td>', the expression '</?t.*?>' ends each match at the first occurrence of the closing angle bracket (>):

'<tr>'   '<td>'   '</td>'

exprq+

Possessive expression: match as much as possible, but do not rescan any portions of the text.

Given the text'<tr><td>text</td>', the expression '</?t.*+>' does not return any matches, because the closing angle bracket is captured using .*, and is not rescanned.

Grouping Operators

Grouping operators allow you to capture tokens, apply one operator to multiple elements, or disable backtracking in a specific group.

Grouping Operator	Description	Example
`(expr)`	Group elements of the expression and capture tokens.	`'Joh?n\s(\w*)'` captures a token that contains the last name of any person with the first name `John` or `Jon`.
`(?:expr)`	Group, but do not capture tokens.	`'(?:[aeiou][^aeiou]){2}'` matches two consecutive patterns of a vowel followed by a nonvowel, such as `'anon'`. Without grouping, `'[aeiou][^aeiou]{2}'`matches a vowel followed by two nonvowels.
`(?>expr)`	Group atomically. Do not backtrack within the group to complete the match, and do not capture tokens.	`'A(?>.)Z'` does not match `'AtoZ'`, although `'A(?:.)Z'` does. Using the atomic group, `Z` is captured using `.*` and is not rescanned.
`(expr1\|expr2)`	Match expression `expr1` or expression `expr2`. If there is a match with `expr1`, then `expr2` is ignored. You can include `?:` or `?>` after the opening parenthesis to suppress tokens or group atomically.	`'(let\|tel)\w+'` matches words that contain, but do not end, with `let` or `tel`.

Anchors

Anchors in the expression match the beginning or end of the input text or word.

Anchor	Matches the...	Example
`^expr`	Beginning of the input text.	`'^M\w*'` matches a word starting with `M` at the beginning of the text.
`expr$`	End of the input text.	`'\w*m$'` matches words ending with `m` at the end of the text.
`\<expr`	Beginning of a word.	`'\<n\w*'` matches any words starting with `n`.
`expr\>`	End of a word.	`'\w*e\>'` matches any words ending with `e`.

Lookaround Assertions

Lookaround assertions look for patterns that immediately precede or follow the intended match, but are not part of the match.

The pointer remains at the current location, and characters that correspond to the test expression are not captured or discarded. Therefore, lookahead assertions can match overlapping character groups.

Lookaround Assertion	Description	Example
`expr(?=test)`	Look ahead for characters that match `test`.	`'\w*(?=ing)'` matches terms that are followed by `ing`, such as `'Fly'` and `'fall'` in the input text `'Flying, not falling.'`
`expr(?!test)`	Look ahead for characters that do not match `test`.	`'i(?!ng)'` matches instances of the letter `i` that are not followed by `ng`.
`(?<=test)expr`	Look behind for characters that match `test`.	`'(?<=re)\w*'` matches terms that follow `'re'`, such as `'new'`, `'use'`, and `'cycle'` in the input text `'renew, reuse, recycle'`
`(?<!test)expr`	Look behind for characters that do not match `test`.	`'(?<!\d)(\d)(?!\d)'` matches single-digit numbers (digits that do not precede or follow other digits).

If you specify a lookahead assertion before an expression, the operation is equivalent to a logical AND.

Operation	Description	Example
`(?=test)expr`	Match both `test` and `expr`.	`'(?=[a-z])[^aeiou]'` matches consonants.
`(?!test)expr`	Match `expr` and do not match `test`.	`'(?![aeiou])[a-z]'` matches consonants.

Logical and Conditional Operators

Logical and conditional operators allow you to test the state of a given condition, and then use the outcome to determine which pattern, if any, to match next. These operators support logical OR, and if or if/else conditions.

Conditions can be tokens, lookaround operators, or dynamic expressions of the form (?@cmd). Dynamic expressions must return a logical or numeric value.

Conditional Operator	Description	Example
`expr1\|expr2`	Match expression `expr1` or expression `expr2`. If there is a match with `expr1`, then `expr2` is ignored.	`'(let\|tel)\w+'` matches words that start with `let` or `tel`.
`(?(cond)expr)`	If condition `cond` is `true`, then match `expr`.	`'(?(?@ispc)[A-Z]:\\)'` matches a drive name, such as `C:\`, when run on a Windows^® system.
`(?(cond)expr1\|expr2)`	If condition `cond` is `true`, then match `expr1`. Otherwise, match `expr2`.	`'Mr(s?)\..?(?(1)her\|his) \w'` matches text that includes `her` when the text begins with `Mrs`, or that includes `his` when the text begins with `Mr`.

Conditional Operator

Description

Example

expr1|expr2

Match expression expr1 or expression expr2.

If there is a match with expr1, then expr2 is ignored.

'(let|tel)\w+' matches words that start with let or tel.

(?(cond)expr)

If condition cond is true, then match expr.

'(?(?@ispc)[A-Z]:\\)' matches a drive name, such as C:\, when run on a Windows^® system.

(?(cond)expr1|expr2)

If condition cond is true, then match expr1. Otherwise, match expr2.

'Mr(s?)\..*?(?(1)her|his) \w*' matches text that includes her when the text begins with Mrs, or that includes his when the text begins with Mr.

Token Operators

Tokens are portions of the matched text that you define by enclosing part of the regular expression in parentheses. You can refer to a token by its sequence in the text (an ordinal token), or assign names to tokens for easier code maintenance and readable output.

Ordinal Token Operator	Description	Example
`(expr)`	Capture in a token the characters that match the enclosed expression.	`'Joh?n\s(\w*)'` captures a token that contains the last name of any person with the first name `John` or `Jon`.
`\N`	Match the `N`th token.	`'<(\w+).>.</\1>'` captures tokens for HTML tags, such as `'title'` from the text `'<title>Some text</title>'`.
`(?(N)expr1\|expr2)`	If the `N`th token is found, then match `expr1`. Otherwise, match `expr2`.	`'Mr(s?)\..?(?(1)her\|his) \w'` matches text that includes `her` when the text begins with `Mrs`, or that includes `his` when the text begins with `Mr`.

Named Token Operator	Description	Example
`(?<name>expr)`	Capture in a named token the characters that match the enclosed expression.	`'(?<month>\d+)-(?<day>\d+)-(?<yr>\d+)'` creates named tokens for the month, day, and year in an input date of the form `mm-dd-yy`.
`\k<name>`	Match the token referred to by `name`.	`'<(?<tag>\w+).>.</\k<tag>>'` captures tokens for HTML tags, such as `'title'` from the text `'<title>Some text</title>'`.
`(?(name)expr1\|expr2)`	If the named token is found, then match `expr1`. Otherwise, match `expr2`.	`'Mr(?<sex>s?)\..?(?(sex)her\|his) \w'` matches text that includes `her` when the text begins with `Mrs`, or that includes `his` when the text begins with `Mr`.

Note

If an expression has nested parentheses, MATLAB^® captures tokens that correspond to the outermost set of parentheses. For example, given the search pattern '(and(y|rew))', MATLAB creates a token for 'andrew' but not for 'y' or 'rew'.

Dynamic Regular Expressions

Dynamic expressions allow you to execute a MATLAB command or a regular expression to determine the text to match.

The parentheses that enclose dynamic expressions do not create a capturing group.

Operator	Description	Example
`(??expr)`	Parse `expr` and include the resulting term in the match expression. When parsed, `expr` must correspond to a complete, valid regular expression. Dynamic expressions that use the backslash escape character (`\`) require two backslashes: one for the initial parsing of `expr`, and one for the complete match.	`'^(\d+)((??\\w{$1}))'` determines how many characters to match by reading a digit at the beginning of the match. The dynamic expression is enclosed in a second set of parentheses so that the resulting match is captured in a token. For instance, matching `'5XXXXX'` captures tokens for `'5'` and `'XXXXX'`.
`(??@cmd)`	Execute the MATLAB command represented by `cmd`, and include the output returned by the command in the match expression.	`'(.{2,}).?(??@fliplr($1))'` finds palindromes that are at least four characters long, such as `'abba'`.
`(?@cmd)`	Execute the MATLAB command represented by `cmd`, but discard any output the command returns. (Helpful for diagnosing regular expressions.)	`'\w?(\w)(?@disp($1))\1\w'` matches words that include double letters (such as `pp`), and displays intermediate results.

Operator

Description

Example

(??expr)

Parse expr and include the resulting term in the match expression.

When parsed, expr must correspond to a complete, valid regular expression. Dynamic expressions that use the backslash escape character (\) require two backslashes: one for the initial parsing of expr, and one for the complete match.

'^(\d+)((??\\w{$1}))' determines how many characters to match by reading a digit at the beginning of the match. The dynamic expression is enclosed in a second set of parentheses so that the resulting match is captured in a token. For instance, matching '5XXXXX' captures tokens for '5' and 'XXXXX'.

(??@cmd)

Execute the MATLAB command represented by cmd, and include the output returned by the command in the match expression.

'(.{2,}).?(??@fliplr($1))' finds palindromes that are at least four characters long, such as 'abba'.

(?@cmd)

Execute the MATLAB command represented by cmd, but discard any output the command returns. (Helpful for diagnosing regular expressions.)

'\w*?(\w)(?@disp($1))\1\w*' matches words that include double letters (such as pp), and displays intermediate results.

Within dynamic expressions, use the following operators to define replacement text.

Replacement Operator	Description
`$&` or `$0`	Portion of the input text that is currently a match
$`	Portion of the input text that precedes the current match
`$'`	Portion of the input text that follows the current match (use `$''` to represent `$'`)
`$N`	`N`th token
`$<name>`	Named token
`${cmd}`	Output returned when MATLAB executes the command, `cmd`

Comments

Characters	Description	Example
`(?#comment)`	Insert a comment in the regular expression. The comment text is ignored when matching the input.	`'(?# Initial digit)\<\d\w+'` includes a comment, and matches words that begin with a number.

Search Flags

Search flags modify the behavior for matching expressions. An alternative to using a search flag within an expression is to pass an option input argument.

Flag	Description
`(?-i)`	Match letter case (default for `regexp` and `regexprep`).
`(?i)`	Do not match letter case (default for `regexpi`).
`(?s)`	Match dot (`.`) in the pattern with any character (default).
`(?-s)`	Match dot in the pattern with any character that is not a newline character.
`(?-m)`	Match the `^` and `$` metacharacters at the beginning and end of text (default).
`(?m)`	Match the `^` and `$` metacharacters at the beginning and end of a line.
`(?-x)`	Include space characters and comments when matching (default).
`(?x)`	Ignore space characters and comments when matching. Use `'\ '` and `'\#'` to match space and `#` characters.

The expression that the flag modifies can appear either after the parentheses, such as

(?i)\w*

or inside the parentheses and separated from the flag with a colon (:), such as

(?i:\w*)

The latter syntax allows you to change the behavior for part of a larger expression.

Data Types: char | cell | string

`replace` — Replacement text
character vector | cell array of character vectors | string array

Replacement text, specified as a character vector, a cell array of character vectors, or a string array, as follows:

If replace is a single character vector and expression is a cell array of character vectors, then regexprep uses the same replacement text for each expression.
If replace is a cell array of N character vectors and expression is a single character vector, then regexprep attempts N matches and replacements.
If both replace and expression are cell arrays of character vectors, then they must contain the same number of elements. regexprep pairs each replace element with its corresponding element in expression.

The replacement text can include regular characters, special characters (such as tabs or new lines), or replacement operators, as shown in the following tables.

Replacement Operator	Description
`$&` or `$0`	Portion of the input text that is currently a match
$`	Portion of the input text that precedes the current match
`$'`	Portion of the input text that follows the current match (use `$''` to represent `$'`)
`$N`	`N`th token
`$<name>`	Named token
`${cmd}`	Output returned when MATLAB executes the command, `cmd`

Operator	Description
`\a`	Alarm (beep)
`\b`	Backspace
`\f`	Form feed
`\n`	New line
`\r`	Carriage return
`\t`	Horizontal tab
`\v`	Vertical tab
`\char`	Any character with special meaning in regular expressions that you want to match literally (for example, use `\\` to match a single backslash)

Data Types: char | cell | string

Output Arguments

collapse all

`newDocuments` — Output documents
`tokenizedDocument` array

Output documents, returned as a tokenizedDocument array.

Tips

Text Analytics Toolbox provides functions for common text preprocessing steps. For example, to remove punctuation and symbol characters, use erasePunctuation or to remove stem words using the Porter stemmer, use normalizeWords. For more information, see Text Data Preparation.

Version History

Introduced in R2017b

regexprep

Syntax

Description

Examples

Update Text in Words

Include Captured Tokens in Word Replacement

Input Arguments

`documents` — Input documents
`tokenizedDocument` array

`expression` — Regular expression
character vector | cell array of character vectors | string array

`replace` — Replacement text
character vector | cell array of character vectors | string array

Output Arguments

`newDocuments` — Output documents
`tokenizedDocument` array

Tips

Version History

See Also

Topics

regexprep

Syntax

Description

Examples

Update Text in Words

Include Captured Tokens in Word Replacement

Input Arguments

documents — Input documents tokenizedDocument array

expression — Regular expression character vector | cell array of character vectors | string array

replace — Replacement text character vector | cell array of character vectors | string array

Output Arguments

newDocuments — Output documents tokenizedDocument array

Tips

Version History

See Also

Topics

`documents` — Input documents
`tokenizedDocument` array

`expression` — Regular expression
character vector | cell array of character vectors | string array

`replace` — Replacement text
character vector | cell array of character vectors | string array

`newDocuments` — Output documents
`tokenizedDocument` array