Silicon Soul

For complicated text operations, Bash has powerful external commands available (e.g., cut, sed, awk). Also, Bash has built-in string operators that can handle simpler text operations with greater efficiency.

Note: If you are not familiar with the GNU/Linux command line interface, review the Conventions page before proceeding.

String Operators

Bash's string operators give you a built-in resource-light way to manipulate the values of variables in several ways. The syntax of a Bash string operator involves inserting a special character that denotes an operator between a variable's name and the right curly bracket (}). Any argument that the operator may need to work on is inserted to the operator's right.

Substitution Operators

Substitution operators are used to deal with variables' default values, undefined variables, variable existence, and variables' substrings.

:-

${ex_variable:-ex_word}

If ex_variable exists and is not null, return its value. Otherwise, return ex_word.

This operator lets you return a default value if a variable is undefined.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:-How are you?}"
Hi there!

$ greeting=''
$ echo "${greeting:-How are you?}"
How are you?

:=

${ex_variable:=ex_word}

If ex_variable exists and is not null, return its value. Otherwise, set it to ex_word, and then return its value. If you omit the colon, the assignment takes place only if ex_variable does not exist (i.e., an existing variable with no set value is left unchanged).

This operator lets you set a variable to a default value if it is undefined, and return that value. Keep in mind, Bash's positional parameters cannot be set in this way.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:=How are you?}"
Hi there!

$ greeting=''
$ echo "${greeting:=How are you?}"
How are you?

$ echo "$greeting"
How are you?

:?

${ex_variable:?ex_message}

If ex_variable exists and is not null, return its value. Otherwise, print ex_variable:, followed by ex_message, and abort the current command or script.

If you omit ex_message, the default message parameter null or not set is displayed via both the standard error and the standard output (if the shell is not interactive, it exits). If the colon is omitted, the message will only appear if the variable does not exist.

This operator lets you catch errors that result from variables being undefined.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:?Make sure you set the \'greeting\' variable.}"
Hi there!

$ greeting=''
$ echo "${greeting:?Make sure you set the \'greeting\' variable.}"
greeting: Make sure you set the 'greeting' variable.

:+

${ex_variable:+ex_word}

If ex_variable exists and is not null, return ex_word. Otherwise, return null.

This operator lets you test for the existence of a variable.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:+The 'greeting' variable exists.}"
The 'greeting' variable exists.

$ greeting=''
$ echo "${greeting:+The 'greeting' variable exists.}"

$

:

${ex_variable:ex_offset:ex_length}

Returns the substring of ex_variable starting at ex_offset and up to ex_length characters (ex_offset and ex_length are considered arithmetic expressions). Bash strings are zero-based, so the first character in ex_variable is at position 0. If ex_length is omitted, the substring starts at ex_offset and continues to the end of ex_variable.

If ex_offset is negative, counting begins from the end of ex_variable, and a space must be placed between the : and ex_offset. If ex_length is negative, it is interpreted as an offset in characters from the end of ex_variable, rather than a number of characters, and the expansion is the characters between ex_offset and ex_length.

If ex_variable is @ or an indexed array subscripted by @ or *, then ex_length is the number of positional parameters you'd like returned, starting at ex_offset.

This string operator lets you return parts of a string. It performs substring expansion.

Examples:

$ greeting='Hi there!'
$ echo "${greeting:0:2}"
Hi

$ echo "${greeting:3}"
there!

$ echo "${greeting: -6}"
there!

$ echo "${greeting:0:-7}"
Hi

$ cat example.bash
#!/usr/bin/env bash

echo "${@:1:2}"

exit 0

$ example.bash one two three
one two

$ my_array=(one two three)
$ echo "${my_array[@]:1:2}"
two three

Pattern Matching Operators

Pattern matching operators are used to match portions of a variable's string value against patterns. These patterns can contain shell wildcard characters.

The pattern matching operators do not change the contents of variables. They work on-the-fly and can be used with other commands (e.g., echo) to return your desired output.

#

${ex_variable#ex_pattern}

If the pattern matches the beginning of the variable's value, delete the shortest part that matches ex_pattern and return the rest.

Example:

$ my_file='/home/amnesia/Documents/test.txt'
$ echo "${my_file#/*/}"
amnesia/Documents/test.txt

##

${ex_variable##ex_pattern}

If the pattern matches the beginning of the variable's value, delete the longest part that matches ex_pattern and return the rest.

Example:

$ my_file='/home/amnesia/Documents/test.txt'
$ echo "${my_file##/*/}"
test.txt

%

${ex_variable%ex_pattern}

If the pattern matches the end of the variable's value, delete the shortest part that matches ex_pattern and return the rest.

Example:

$ my_file='/home/amnesia/Documents/test.txt'
$ echo "${my_file%/*}"
/home/amnesia/Documents

%%

${ex_variable%%ex_pattern}

If the pattern matches the end of the variable's value, delete the longest part ex_pattern that matches and return the rest.

Example:

$ my_file='/home/amnesia/Documents/test.txt'
$ echo "${my_file%%/*}"

$

/ and //

${ex_variable/ex_pattern/ex_string} and ${ex_variable//ex_pattern/ex_string}

The longest match to ex_pattern in ex_variable is replaced by ex_string. In the ${ex_variable/ex_pattern/ex_string} form, only the first match is replaced. In the ${ex_variable//ex_pattern/ex_string} form, all matches are replaced. If ex_string is null, the matches are deleted.

If ex_pattern begins with a #, it must match at the start of the variable. If ex_pattern begins with a %, it must match with the end of the variable.

Examples:

$ my_file='/home/amnesia/Documents/amnesia/test.txt'
$ echo "${my_file/amnesia/new_amnesia}"
/home/new_amnesia/Documents/amnesia/test.txt

$ echo "${my_file//amnesia/new_amnesia}"
/home/new_amnesia/Documents/new_amnesia/test.txt

$ echo "${my_file/amnesia/}"
/home//Documents/amnesia/test.txt

$ echo "${my_file//amnesia/}"
/home//Documents//test.txt

$ my_string='stuff like that is good stuff'
$ echo "${my_string/#stuff/junk}"
junk like that is good stuff

$ echo "${my_string/%stuff/junk}"
stuff like that is good junk

$ cat example.bash
#!/usr/bin/env bash

echo "${@/two/2}"

exit 0

$ example.bash one two three
one 2 three

For each of the pattern matching operators above, if ex_variable is @ or *, the operation is applied to each positional parameter in turn, and the expansion is the output. If ex_variable is an array variable subscripted with @ or *, the operation is applied to each member of the array in turn, and the expansion is the output.

Extended Pattern Matching Operators

If the shopt option extglob is enabled, Bash provides a further set of pattern matching operators. Each of these operators takes one or more patterns, usually strings, that can be separated by the vertical bar (|).

*(ex_pattern_list)

Matches zero or more occurrences of the given pattern(s).

Example:

$ ls -hl
total 32K
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Documents
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Downloads
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Music
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Pictures
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Public
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Templates
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Videos

$ ls -dhl *(Desktop|Documents|Defaults)
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Documents

+(ex_pattern_list)

Matches one or more occurrences of the given pattern(s).

?(ex_pattern_list)

Matches zero or one occurrences of the given pattern(s).

@(ex_pattern_list)

Matches exactly one of the given pattern(s).

!(ex_pattern_list)

Matches anything except one of the given pattern(s).

The patterns provided to these operators can contain shell wildcards, and the patterns can be nested.

Example:

$ ls -hl
total 32K
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Documents
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Downloads
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Music
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Pictures
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Public
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Templates
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Videos

$ ls -dhl !(Do+(c*|w*))
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Desktop
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Music
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Pictures
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Public
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Templates
drwxr-xr-x 2 amnesia amnesia 4.0K Aug  9 20:29 Videos

Length Operator

${#ex_variable}

Returns the length of the value of the variable as a character string.

Example:

$ greeting='Hi there!'
$ echo ${#greeting}
9

Documentation

You can find out more about these operators in the Linux User's Manual under man 1 bash and online via the Linux Documentation Project and GNU's Bash Reference Manual (¹, ²).