Parameter Substitution (Bash)

Parameter substitution, parameter expansion or shell parameter expansion means that in a Bash expression a parameter or variable is replaced by the contents of that variable, possibly in combination with an operation.

Parameter expansion is indicated by $, often in combination with curly braces: ${}, or even more complete: ${parameter}.

Judging by [1], all cases are covered in this article.

→ This list is not complete. Check https://stackoverflow.com/questions/5406858/difference-between-unset-and-empty-variables-in-bash#5406887 for some more amazing things.

${}

Something as simple as displaying the value of a variable, is already an example of parameter substitution:

$ a=12
$ echo $a

12

Adding braces might make the concept of expansion even clearer. In particular, that $ is the operator. It's not part of the variable's name: The part inside the curly braces doesn't get a $ sign:

$ a=12
$ echo ${a}

12

No nesting

You can't nest ${} expressions: The syntax expander doesn't know how to handle them [2]. Also see below at Find & replace.

Illustration:

$ echo ${${HELLO}WORLD}

-bash: ${${HELLO}WORLD}: bad substitution

Null substitution - ${parameter:-word}

Use this syntax to replace parameter with literal word if the former is empty. Something like ifnull(a, b) in SQL:

# The parameter has a value:

$ i1="Hello, world!"
$ echo ${i1:-De parameter is leeg!}

Hello, world!

# De parameter bestaat niet - De literal wordt weergegeven

$ i1="Hello, world!"
$ echo ${i3:-De parameter is leeg!}

De parameter is leeg!

Wil je in plaats van een literal een parameter meegeven? Geen probleem: Daar gaat dit artikel over:

$ i1="Hello, world!"
$ i2="Vervanging"
$ echo ${i3:-${i2}}

Vervanging

Als de parameter wel bestaat, maar leeg is:

$ i1=
$ echo ${i1:-Leeg}

Leeg

Substring replacement - ${parameter/pattern/string}

You can use parameter expansion for find-&-replace:

$ i1="Hello, world!"
$ echo ${i1/world/moon}

Hello, moon!

Only the first occurence of pattern will be replaced:

$ i1="Hello, world1, world2 & world3!"
$ echo ${i1/world/moon}

Hello, moon1, world2 & world3!

Substring replacement - ${parameter//pattern/string}

Use a double / after parameter, to replace all occurences of pattern.

$ i1="Hello, world1, world2 & world3!"
$ echo ${i1//world/moon}

Hello, moon1, moon2 & moon3!

Meer:

The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the 
longest match of pattern against its value is replaced with string. The match is performed according to 
the rules described below (see Pattern Matching). If pattern begins with ‘/’, all matches of pattern are 
replaced with string. Normally only the first match is replaced. If pattern begins with ‘#’, it must match 
at the beginning of the expanded value of parameter. If pattern begins with ‘%’, it must match at the end 
of the expanded value of parameter. If string is null, matches of pattern are deleted and the / following 
pattern may be omitted. If the nocasematch shell option (see the description of shopt in The Shopt 
Builtin) is enabled, the match is performed without regard to the case of alphabetic characters. If 
parameter is ‘@’ or ‘*’, the substitution operation is applied to each positional parameter in turn, and 
the expansion is the resultant list. If parameter is an array variable subscripted with ‘@’ or ‘*’, the 
substitution operation is applied to each member of the array in turn, and the expansion is the resultant 
list.

Some inspiration:

String pattern filtering - ${parameter/pattern}

This is a variant of two chapters before. It uses a pattern which is a bit similar (but not really!) to regex
As string is empty, the pattern will be deleted from the input:

$ i="hoi123hallo"
$ echo ${i/[()a-zA-Z]}

oi123hallo

String pattern filtering - ${parameter//pattern}

Use a double / between parameter and pattern, to filter out all occurences of the pattern - Now you can really filter out stuff:

i="hoi123hallo"; echo ${i/[()a-zA-Z]}   # 123
i="howarexxdxxy 12 (diameter)"; echo ">>"${i//[()a-zA-Z]}"<<"   # >> 12 << - Spaces get preserved
i="howarexxdxxy 12 (diameter)"; echo ">>"${i//[()a-zA-Z ]}"<<"  # >>12<< - Spaces get also filtered out

Convert case

[3]:

$ name="vivek"
$ echo $name
vivek

$ echo ${name^}
Vivek

$ echo ${name^^}
VIVEK

String length - ${#string}

This syntax is not only for arrays, but also for scalars:

i="hoihallo"
echo ${#i}

8

Substring index extraction - ${string:index:length}

i="1234567890"
echo ${i:3}     # Select from index 3 (base 0 → 4th character)
echo ${i:3:5}   # Select 5 characters

4567890
45678

Substring removal - ${string#substring}

${string#substring} deletes the shortest match of substring (as a pattern expression) from the front of $string

I'm not sure what kind of pattern expression is being used. It doesn't seem like regular expressions, does it?
The trick is, that it deletes from the front of the string - See the examples below
This is probably what SQL's substring_index does.

Examples:

i="1234567890"

echo $i
echo ${i#12}	# First 2 characters deleted
echo ${i#34}    # Nothing deleted: String doesn't start with "34"
echo ${i#*34}   # First 4 characters deleted
echo ${i#34*}   # Nothing deleted: String doesn't start with "34"

The wildcard can also be used inside the substring:

i="12345-12345"

echo $i          # 12345-12345
echo ${i#14}     # 12345-12345
echo ${i#1*4}    # 5-12345
echo ${i#25}     # 12345-12345
echo ${i#*25}    # 12345-12345
echo ${i#*2*5}   # -12345

echo ${i#*2*}    # 2345-12345 - Trailing "*" has no meaning

I still don't know what kind of pattern matching is used, but bracket expressions as in regular expressions, surely work:

# Bracket expressions
########################################
#
i="12345-12345"

echo $i              # 12345-12345
echo ${i#[14]}       # 2345-12345 - "1" matches and is deleted
echo ${i#[14]2}      # 345-12345 - "12" matches
echo ${i#[1234]}     # 2345-12345 - Only 1 character is picked
echo ${i#[234]}      # 12345-12345 - Nothing happens
echo ${i#[1234]*}    # 2345-12345
echo ${i#*[1234]}    # 2345-12345 - Smallest interval is chosen
echo ${i#*[4321]}    # 2345-12345 - Smallest interval is chosen
echo ${i#1*[34]}     # 45-12345 - Smallest interval is chosen
echo ${i#1*[43]}     # 45-12345 - Smallest interval is chosen
echo ${i#*[34]}      # 45-12345
echo ${i#*[43]}      # 45-12345
echo ${i#*[34]5}     # -12345 - Surprise. Maybe *45 is smallest interval?
echo ${i#*[43]5}     # -12345 - Surprise. *45 is selected as interval
echo ${i#*[34]2}     # Nothing deleted
echo ${i#*[43]2}     # Nothing deleted

Here's some more regex stuff. It's a bit of a mixed bag:

# Some more regex stuff
########################################
#
i="12345-12345"

echo $i                # 12345-12345
echo ${i#[0-9]}        # 2345-12345
echo ${i#[0-9]{2}}     # 12345-12345}
echo ${i#[0-9]+}       # 2345-12345
echo ${i#.....}        # 12345-12345
echo ${i#[0-9][0-9]}   # 345-12345

Substring removal - ${string##substring}

As before, except that the pattern matching is done as inclusive as possible:

i="12345-12345"

echo $i           # 12345-12345
echo ${i##14}     # 12345-12345
echo ${i##1*4}    # 5 - Only the last character left

echo ${i##25}     # 12345-12345
echo ${i##*25}    # 12345-12345
echo ${i##*2*5}   # Evertything removed

echo ${i##*5}     # Everything removed
echo ${i##*4}     # 5 - Last character left

Substring removal - ${string%substring}

As ${string#substring} but now starting at the end of the string. Like SQL's substring_index with a negative index:

i="1234567890"

echo $i
echo ${i%90}	# Last 2 characters deleted
echo ${i%89}	# Nothing happened: String doesn't end at "89"
echo ${i%89*}	# Last 3 characters deleted
echo ${i%6*9*}	# Last 5 characters deleted

Substring removal - ${string%%substring}

As before, but with a pattern match that is as broadly as possible:

i="12345-12345"

echo $i           # 12345-12345
echo ${i%%45}     # 12345-123
echo ${i%%4*5}    # 123
echo ${i%%34}     # 12345-12345
echo ${i%%34*}    # 12 - Only first two characters remain
echo ${i%%3*4*}   # 12 - Only first two characters remain

Casus: Extra line breaks product_cat (aug. 2021)

WordPress product_cat-taxon-beschrijvingen worden via WP-CLI en Bash bijgewerkt. Ik gebruik parameter substitution for find-&-replace, met code zoals dit:

i1=$(wp wc product_cat --user=4 get 18869 --field="description")
wp wc product_cat --user=4 update 18869 --description="${i1/Alle widgets voor/All widgets for}"

Er wordt echter een karakter 0a (line feed) toegevoegd aan het begin van de string. Hoe kan dat? Hoe verhinder ik dat?

Zes line feeds vóór de eigenlijke tekst (ik heb deze routine vermoedelijk al zes keer uitgevoerd)

Het gaat inderdaad om line feeds: 0a

Sources