Page Body

Manipulating Text With Bash String Operators

For complicated text operations, Bash has powerful external commands available (e.g., cut, sed, awk). Also, Bash has built-in string operators that can handle simpler text operations with greater efficiency.

Note: If you are not familiar with the GNU/Linux command line interface, review the Conventions page before proceeding.

String Operators

Bash's string operators give you a built-in resource-light way to manipulate the values of variables in several ways. The syntax of a Bash string operator involves inserting a special character that denotes an operator between a variable's name and the right curly bracket (}). Any argument that the operator may need to work on is inserted to the operator's right.

Substitution Operators

Substitution operators can be used to deal with variables' default values, undefined variables, variable existence, and variables' substrings.

:-

${example_variable:-example_word}
If example_variable exists and is not null, return its value. Otherwise, return example_word.

This operator lets you return a default value if a variable is undefined.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:-How are you?}"
Hi there!

$ greeting=''
$ echo "${greeting:-How are you?}"
How are you?

:=

${example_variable:=example_word}
If example_variable exists and is not null, return its value. Otherwise, set it to example_word, and the return its value. If you omit the colon, the assignment takes place only if example_variable does not exist (i.e., an existing variable with no set value is left unchanged).

This operator lets you set a variable to a default value if it is undefined, and return that value. Keep in mind, Bash's positional parameters cannot be set in this way.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:=How are you?}"
Hi there!

$ greeting=''
$ echo "${greeting:=How are you?}"
How are you?

$ echo "$greeting"
How are you?

:?

${example_variable:?example_message}
If example_variable exists and is not null, return its value. Otherwise, print example_variable:, followed by example_message, and abort the current command or script. If you omit example_message, the default message parameter null or not set is displayed via both the Standard Error and the Standard Output (if the shell is not interactive, it exits). If the colon is omitted, the message will only appear if the variable does not exist.

This operator lets you catch errors that result from variables being undefined.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:?Make sure you set the \'greeting\' variable.}"
Hi there!

$ greeting=''
$ echo "${greeting:?Make sure you set the \'greeting\' variable.}"
greeting: Make sure you set the 'greeting' variable.

:+

${example_variable:+example_word}
If example_variable exists and is not null, return example_word. Otherwise, return null.

This operator lets you test for the existence of a variable.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:+The 'greeting' variable exists.}"
The 'greeting' variable exists.

$ greeting=''
$ echo "${greeting:+The 'greeting' variable exists.}"

$

:

${example_variable:example_offset:example_length}
Returns the substring of example_variable starting at example_offset and up to example_length characters (example_offset and example_length are considered arithmetic expressions). Bash strings are zero-based, so the first character in example_variable is at position 0. If example_length is omitted, the substring starts at example_offset and continues to the end of example_variable.
If example_offset is negative, counting begins from the end of example_variable, and a space must be placed between the : and example_offset. If example_length is negative, it is interpreted as an offset in characters from the end of example_variable, rather than a number of characters, and the expansion is the characters between example_offset and example_length.
If example_variable is @ or an indexed array subscripted by @ or *, then example_length is the number of positional parameters you'd like returned, starting at example_offset.

This string operator lets you return parts of a string. It performs substring expansion.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:0:2}"
Hi

$ echo "${greeting:3}"
there!

$ echo "${greeting: -6}"
there!

$ echo "${greeting:0:-7}"
Hi

$ cat example.bash
#!/usr/bin/env bash

echo "${@:1:2}"

exit 0

$ example.bash one two three
one two

$ my_array=(one two three)
$ echo "${my_array[@]:1:2}"
two three

Pattern Matching Operators

Pattern matching operators can be used to match portions of a variable's string value against patterns. These patterns can contain shell wildcard characters.

The pattern matching operators do not change the contents of variables. They work on-the-fly and can be used with other commands (e.g., echo) to return your desired output.

#

${example_variable#example_pattern}
If the pattern matches the beginning of the variable's value, delete the shortest part that matches example_pattern and return the rest.

Example:

$ my_file='/home/user/Documents/test.txt'
$ echo "${my_file#/*/}"
user/Documents/test.txt

##

${example_variable##example_pattern}
If the pattern matches the beginning of the variable's value, delete the longest part that matches example_pattern and return the rest.

Example:

$ my_file='/home/user/Documents/test.txt'
$ echo "${my_file##/*/}"
test.txt

%

${example_variable%example_pattern}
If the pattern matches the end of the variable's value, delete the shortest part that matches example_pattern and return the rest.

Example:

$ my_file='/home/user/Documents/test.txt'
$ echo "${my_file%/*}"
/home/user/Documents

%%

${example_variable%%example_pattern}
If the pattern matches the end of the variable's value, delete the longest part example_pattern that matches and return the rest.

Example:

$ my_file='/home/user/Documents/test.txt'
$ echo "${my_file%%/*}"

$

/ and //

${example_variable/example_pattern/example_string} and ${example_variable//example_pattern/example_string}
The longest match to example_pattern in example_variable is replaced by example_string. In the ${example_variable/example_pattern/example_string} form, only the first match is replaced. In the ${example_variable//example_pattern/example_string} form, all matches are replaced. If example_string is null, the matches are deleted.
If example_pattern begins with a #, it must match at the start of the variable. If example_pattern begins with a %, it must match with the end of the variable.

Examples:

$ my_file='/home/user/Documents/user/test.txt'
$ echo "${my_file/user/new_user}"
/home/new_user/Documents/user/test.txt

$ echo "${my_file//user/new_user}"
/home/new_user/Documents/new_user/test.txt

$ echo "${my_file/user/}"
/home//Documents/user/test.txt

$ echo "${my_file//user/}"
/home//Documents//test.txt

$ my_string='stuff like that is good stuff'
$ "${my_string/#stuff/junk}"
junk like that is good stuff

$ "${my_string/%stuff/junk}"
stuff like that is good junk

$ cat example.bash
#!/usr/bin/env bash

echo "${@/two/2}"

exit 0

$ example.bash one two three
one 2 three

For each of the pattern matching operators above, if example_variable is @ or *, the operation is applied to each positional parameter in turn, and the expansion is the output. If example_variable is an array variable subscripted with @ or *, the operation is applied to each member of the array in turn, and the expansion is the output.

Extended Pattern Matching Operators

If the shopt option extglob is enabled, Bash provides a further set of pattern matching operators. Each of these operators takes one or more patterns, usually strings, that can be separated by the vertical bar (|).

*(example_pattern_list)
Matches zero or more occurrences of the given pattern(s).

Example:

$ ls -hl
total 32K
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Documents
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Downloads
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Music
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Pictures
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Public
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Templates
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Videos

$ ls -dhl *(Desktop|Documents|Defaults)
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Documents
+(example_pattern_list)
Matches one or more occurrences of the given pattern(s).
?(example_pattern_list)
Matches zero or one occurrences of the given pattern(s).
@(example_pattern_list)
Matches exactly one of the given pattern(s).
!(example_pattern_list)
Matches anything except one of the given pattern(s).

The patterns provided to these operators can contain shell wildcards, and the patterns can be nested.

Example:

$ ls -hl
total 32K
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Documents
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Downloads
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Music
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Pictures
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Public
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Templates
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Videos

$ ls -dhl !(Do+(c*|w*))
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Music
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Pictures
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Public
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Templates
drwxr-xr-x 2 user user 4.0K Aug  9 20:29 Videos

Length Operator

${#example_variable}
Returns the length of the value of the variable as a character string.

Example:

$ greeting='Hi there!'
$ echo ${#greeting}
9

Documentation

You can find out more about these operators in the Linux User's Manual under man bash and online via the Linux Documentation Project and GNU's Bash Reference Manual (1, 2).

Avatar

Enjoyed this post?

Subscribe to the feed for the latest updates.