Parameter Substitution (Bash)

Uit De Vliegende Brigade
Naar navigatie springen Naar zoeken springen
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Parameter substitution, parameter expansion or shell parameter expansion means that in a Bash expression a parameter or variable is replaced by the contents of that variable, possibly in combination with an operation.

Parameter expansion is indicated by $, often in combination with curly braces: ${}, or even more complete: ${parameter}.

Judging by [1], all cases are covered in this article.

→ This list is not complete. Check https://stackoverflow.com/questions/5406858/difference-between-unset-and-empty-variables-in-bash#5406887 for some more amazing things.

For trim, rather see Trim (Bash).

${}

Something as simple as displaying the value of a variable, is already an example of parameter substitution: 

$ a=12
$ echo $a

12

Adding braces might make the concept of expansion even clearer. In particular, that $ is the operator. It's not part of the variable's name: The part inside the curly braces doesn't get a $ sign: 

$ a=12
$ echo ${a}

12

No nesting

You can't nest ${} expressions: The syntax expander doesn't know how to handle them [2]. Also see below at Find & replace.

Illustration: 

$ echo ${${HELLO}WORLD}

-bash: ${${HELLO}WORLD}: bad substitution

Null substitution - ${parameter:-word}

Use this syntax to replace parameter with literal word if the former is empty. Something like ifnull(a, b) in SQL: 

# The parameter has a value:

$ i1="Hello, world!"
$ echo ${i1:-De parameter is leeg!}

Hello, world!

# De parameter bestaat niet - De literal wordt weergegeven

$ i1="Hello, world!"
$ echo ${i3:-De parameter is leeg!}

De parameter is leeg!

Wil je in plaats van een literal een parameter meegeven? Geen probleem: Daar gaat dit artikel over: 

$ i1="Hello, world!"
$ i2="Vervanging"
$ echo ${i3:-${i2}}

Vervanging

Als de parameter wel bestaat, maar leeg is: 

$ i1=
$ echo ${i1:-Leeg}

Leeg

Substring replacement - ${parameter/pattern/string}

You can use parameter expansion for find-&-replace: 

$ i1="Hello, world!"
$ echo ${i1/world/moon}

Hello, moon!

Only the first occurence of pattern will be replaced: 

$ i1="Hello, world1, world2 & world3!"
$ echo ${i1/world/moon}

Hello, moon1, world2 & world3!

Substring replacement - ${parameter//pattern/string}

Use a double / after parameter, to replace all occurences of pattern. 

$ i1="Hello, world1, world2 & world3!"
$ echo ${i1//world/moon}

Hello, moon1, moon2 & moon3!

Meer: 

The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the 
longest match of pattern against its value is replaced with string. The match is performed according to 
the rules described below (see Pattern Matching). If pattern begins with ‘/’, all matches of pattern are 
replaced with string. Normally only the first match is replaced. If pattern begins with ‘#’, it must match 
at the beginning of the expanded value of parameter. If pattern begins with ‘%’, it must match at the end 
of the expanded value of parameter. If string is null, matches of pattern are deleted and the / following 
pattern may be omitted. If the nocasematch shell option (see the description of shopt in The Shopt 
Builtin) is enabled, the match is performed without regard to the case of alphabetic characters. If 
parameter is ‘@’ or ‘*’, the substitution operation is applied to each positional parameter in turn, and 
the expansion is the resultant list. If parameter is an array variable subscripted with ‘@’ or ‘*’, the 
substitution operation is applied to each member of the array in turn, and the expansion is the resultant 
list.

Some inspiration:

String pattern filtering - ${parameter/pattern}

  • This is a variant of two chapters before. It uses a pattern which is a bit similar (but not really!) to regex
  • As string is empty, the pattern will be deleted from the input:
$ i="hoi123hallo"
$ echo ${i/[()a-zA-Z]}

oi123hallo

String pattern filtering - ${parameter//pattern}

Use a double / between parameter and pattern, to filter out all occurences of the pattern - Now you can really filter out stuff: 

i="hoi123hallo"; echo ${i/[()a-zA-Z]}   # 123
i="howarexxdxxy 12 (diameter)"; echo ">>"${i//[()a-zA-Z]}"<<"   # >> 12 << - Spaces get preserved
i="howarexxdxxy 12 (diameter)"; echo ">>"${i//[()a-zA-Z ]}"<<"  # >>12<< - Spaces get also filtered out

Convert case

[3]: 

$ name="vivek"
$ echo $name
vivek

$ echo ${name^}
Vivek

$ echo ${name^^}
VIVEK

String length - ${#string}

This syntax is not only for arrays, but also for scalars: 

i="hoihallo"
echo ${#i}

8

Substring index extraction - ${string:index:length}

i="1234567890"
echo ${i:3}     # Select from index 3 (base 0 → 4th character)
echo ${i:3:5}   # Select 5 characters

4567890
45678

Trim leading characters - ${string#substring}

${string#substring} deletes the shortest match of substring (as a pattern expression) from the front of $string

Examples

i="1234567890"

echo $i
echo ${i#12}	# First 2 characters deleted
echo ${i#34}    # Nothing deleted: String doesn't start with "34"
echo ${i#*34}   # First 4 characters deleted
echo ${i#34*}   # Nothing deleted: String doesn't start with "34"

The wildcard can also be used inside the substring: 

i="12345-12345"

echo $i          # 12345-12345
echo ${i#14}     # 12345-12345
echo ${i#1*4}    # 5-12345
echo ${i#25}     # 12345-12345
echo ${i#*25}    # 12345-12345
echo ${i#*2*5}   # -12345

echo ${i#*2*}    # 2345-12345 - Trailing "*" has no meaning

This pattern matching syntax is called Pattern Matching [4], [5]. It's not the same as regular expressions. At times, I call it bracket expressions, which I still find quite a good name 

# Bracket expressions
########################################
#
i="12345-12345"

echo $i              # 12345-12345
echo ${i#[14]}       # 2345-12345 - "1" matches and is deleted
echo ${i#[14]2}      # 345-12345 - "12" matches
echo ${i#[1234]}     # 2345-12345 - Only 1 character is picked
echo ${i#[234]}      # 12345-12345 - Nothing happens
echo ${i#[1234]*}    # 2345-12345
echo ${i#*[1234]}    # 2345-12345 - Smallest interval is chosen
echo ${i#*[4321]}    # 2345-12345 - Smallest interval is chosen
echo ${i#1*[34]}     # 45-12345 - Smallest interval is chosen
echo ${i#1*[43]}     # 45-12345 - Smallest interval is chosen
echo ${i#*[34]}      # 45-12345
echo ${i#*[43]}      # 45-12345
echo ${i#*[34]5}     # -12345 - Surprise. Maybe *45 is smallest interval?
echo ${i#*[43]5}     # -12345 - Surprise. *45 is selected as interval
echo ${i#*[34]2}     # Nothing deleted
echo ${i#*[43]2}     # Nothing deleted

Here's some more regex stuff. It's a bit of a mixed bag: 

# Some more regex stuff
########################################
#
i="12345-12345"

echo $i                # 12345-12345
echo ${i#[0-9]}        # 2345-12345
echo ${i#[0-9]{2}}     # 12345-12345}
echo ${i#[0-9]+}       # 2345-12345
echo ${i#.....}        # 12345-12345
echo ${i#[0-9][0-9]}   # 345-12345

Trim leading characters - ${string##substring}

As before, except that the pattern matching is done as inclusive (or greedy) as possible.

  • This command seems intended to remove substrings that consists of different characters
  • You can use it to remove multiple instances of the same character, but you need a (slight) trick for this.

Intro

i="12345-12345"

echo $i           # 12345-12345
echo ${i##14}     # 12345-12345
echo ${i##1*4}    # 5 - Only the last character left

echo ${i##25}     # 12345-12345
echo ${i##*25}    # 12345-12345
echo ${i##*2*5}   # Evertything removed

echo ${i##*5}     # Everything removed
echo ${i##*4}     # 5 - Last character left

Remove multiple identical characters

i="111-2345"
echo ${i#1}	# 11-2345
echo ${i#11} 	#  1-2345
echo ${i#111}   #   -2345
echo ${i##1}    # 11-2345 # Now what you unexpected?
echo ${i##1*1}	#   -2345 # This is how to do it!

Remove multiple spaces

Quite suprisingly, it works just for spaces as for regular characters, but displaying might be tricky and it's problematic when there are also trailing spaces 

# 3 leading spaces
# 8 characters in total
#
i="   -2345"

# Output: > -2345< - 8
# Whitespace gets stripped from output! 
# Length is still 8, but you don't see more than one space
#
echo ">"$i"< - ${#i}"

# This goes all fine:
#
             echo ">${i}< - ${#i}" # >   -2345< - 8
j=${i# };    echo ">${j}< - ${#j}" # >  -2345< - 7
j=${i#  };   echo ">${j}< - ${#j}" # > -2345< - 6
j=${i#   };  echo ">${j}< - ${#j}" # >-2345< - 5
j=${i## };   echo ">${j}< - ${#j}" # >  -2345< - 7
j=${i## * }; echo ">${j}< - ${#j}" # >-2345< - 5

Problem when there are also trailing spaces - Everything now gets removed: 

# Remove leading spaces when there are also trailing spaces
########################################
#
# This doesn't work: The whole string now gets removed
#
i="  abcde   "; echo ">>>${i}<<< - ${#i}"
i=${i## * };	echo ">>>${i}<<< - ${#i}"

Substring removal - ${string%substring}

As ${string#substring} but now starting at the end of the string. Like SQL's substring_index with a negative index: 

i="1234567890"

echo $i
echo ${i%90}	# Last 2 characters deleted
echo ${i%89}	# Nothing happened: String doesn't end at "89"
echo ${i%89*}	# Last 3 characters deleted
echo ${i%6*9*}	# Last 5 characters deleted

Substring removal - ${string%%substring}

As before, but with a pattern match that is as broadly as possible:

Intro

i="12345-12345"

echo $i           # 12345-12345
echo ${i%%45}     # 12345-123
echo ${i%%4*5}    # 123
echo ${i%%34}     # 12345-12345
echo ${i%%34*}    # 12 - Only first two characters remain
echo ${i%%3*4*}   # 12 - Only first two characters remain

Removing spaces

See an earlier chapter about trimming multiple leading spaces for details: 

i="123     " # 3 characters, 5 spaces

             echo ">"$i"< - ${#i}"
             echo ">${i}< - ${#i}"
j=${i% };    echo ">${j}< - ${#j}"
j=${i%  };   echo ">${j}< - ${#j}"
j=${i%   };  echo ">${j}< - ${#j}"
j=${i%% };   echo ">${j}< - ${#j}"
j=${i% * };  echo ">${j}< - ${#j}"
j=${i%% * }; echo ">${j}< - ${#j}"   # The only correct version

Removing tabs

The trick: \t is not the same as an actual tab: → ←

With \t: 

i="\t\t\t123"
echo ">${i}< - ${#i}" # Output: \t\t\t123< -9
echo ">"${i}"< - ${#i}" # Output:            123< -9
j=${i##'\t'*'\t'};    echo ">${j}< - ${#j}"
Etc.

Anyhow, this works: 

j=${i##	*	};   # Use actual tabs

Casus: Extra line breaks product_cat (aug. 2021)

WordPress product_cat-taxon-beschrijvingen worden via WP-CLI en Bash bijgewerkt. Ik gebruik parameter substitution for find-&-replace, met code zoals dit: 

i1=$(wp wc product_cat --user=4 get 18869 --field="description")
wp wc product_cat --user=4 update 18869 --description="${i1/Alle widgets voor/All widgets for}"

Er wordt echter een karakter 0a (line feed) toegevoegd aan het begin van de string. Hoe kan dat? Hoe verhinder ik dat?

Zes line feeds vóór de eigenlijke tekst (ik heb deze routine vermoedelijk al zes keer uitgevoerd)
Het gaat inderdaad om line feeds: 0a

See also

Sources

Pattern Matching c.q. Bracket Matching

Overgenomen van "https://wiki.devliegendebrigade.nl/index.php?title=Parameter_Substitution_(Bash)&oldid=35493"