Replace all String regex matches with environment variables in bash - regex

Question

I am trying to figure out how to replace matches of regex in a file with the value of an environment variable called a substring of that matched string. Each matched String also has a default along with it, separated by a colon. What I mean is if I have a file called myFile.properties that contains:
varWithDefault=${envVar1:default1}
varSetNoDefault=${envVar2}
varEmptyValue=${emptyEnvVar}
varEmptyValueWithDefault=${emptyEnvVar:3}
varNotSetWithDefault=${notSetEnvVar:I have : a colon}
# required but no match will prob be an error message
varNotSetNoDefault=${notSetEnvVar}
And I set the following environment variables:
export envVar1=value1
export envVar2=value2
export emptyEnvVar=""
then the file should update to be:
varWithDefault=value1
varSetNoDefault=value2
varEmptyValue=
varEmptyValueWithDefault=
varNotSetWithDefault=I have : a colon
# required but no match will prob be an error message
varNotSetNoDefault=${notSetEnvVar}
I have something that for the most part works... but doesn't handle when an environment is set to empty string, nor does it handle defaults properly.
# get any variables we would like to replace in the myFile.properties file
# after the colon I say "anything except for a closing bracket" so that it will allow for colons in the value
varStrings=$( grep -Po '\${([_a-zA-Z][_a-zA-Z0-9]*)(:[^}]*)*}' myFile.properties )
# loop through the variables found and replace them with the value of the corresponding environment variable
# varString is a value that looks like: "${my_variable:my_default}"
for varString in ${varStrings[#]} ; do
# ideally grab these values from the matched regex, but I can't seem to figure out how to do that
# propName would be: "my_variable"
# defaultValue would be: "my_default"
propName=$varString[0]
defaultValue=$varString[1]
# this technically gets the values, but I would also need to remove the "${}"
propName="$( cut -d ':' -f 1- <<< "$varString" )"
defaultValue="$( cut -d ':' -f 2- <<< "$varString" )"
# $varString will be a String in the format '${my_variable:my_default}' so I need to strip the default chunk from it before doing this
# but... to get the environment variable value if there was no default:
envValue=`eval echo $varString`
# if there is a matching environment variable, do the replacement; otherwise, spit out a warning
# the -z will also fail the if check if the value is an empty string, which I don't want. I only want it to go to the else clause if it was not set. Not sure how to do that.
if [ ! -z "$envValue" ]; then
echo "Replacing $varString with environment variable value '$envValue'"
sed -i "s|$varString|$envValue|g" myFile.properties
else
# set the default value
if [[ noDefaultValueGiven ]] ; then
echo "Warning: No environment variable defined for $envVarName. String not replaced."
else
echo "Warning: No environment variable '$envVarName' defined. Using default value '$defaultValue'."
sed -i "s|$varString|$defaultValue|g" myFile.properties
fi
fi
done
The two big issues I'm having with this is:
1. How to loop through each regex match and have access to both regex groups (the sections surrounded by parenthesis)
2. How to check if an environment variable exists based on a string representation of said variable (i.e. check if "${my_variable}" is set, not just if it is empty)
Does anyone know how this should be done? I'm used to SpringBoot doing this for me.

Answer 1

You may do it in pure BASH regex:
re='^([^=]+=)\$\{([_a-zA-Z][_a-zA-Z0-9]*)(:([^}]*))?\}'
while IFS= read -r line; do
if [[ $line =~ $re ]]; then # match regex
#declare -p BASH_REMATCH
var="${BASH_REMATCH[2]}" # var name in group #2
if [[ -n ${!var+set} ]]; then # if var is set in env
line="${BASH_REMATCH[1]}${!var}" # use var value
elif [[ -n ${BASH_REMATCH[4]} ]]; then. # if default value is set
line="${BASH_REMATCH[1]}${BASH_REMATCH[4]}" # use default string
fi
fi
echo "$line" # print each line
done < file.properties
Output:
varWithDefault=value1
varSetNoDefault=value2
varEmptyValue=
varEmptyValueWithDefault=
varNotSetWithDefault=I have : a colon
# required but no match will prob be an error message
varNotSetNoDefault=${notSetEnvVar}
Code Demo