How to use sed in shell script to replace all environment value occurrences with their current values - regex

Question

I would like to have a shell script to iterate over all the occurrences of environment variable names in a file and replace them with their current values. I am not sure how this can be done by using sed command.
The file content:
values:
value1:
name: "something"
value: "$ENV_VAR1" # this could be any variable name
value2:
name: "something"
value: "$ENV_VAR2"
...
First, I need to find all occurrences of any variable (Using regex "\$(.*?)" ). Then, somehow, I need to replace it with the variable value from the shell. I am not sure how I can use the sed command to achieve the second part as the variable name is specified in the file itself.
Something like the following command:
sed -i "s/\"\$(.*?)\"/${Some_How_Get_Var_Name}/g" file.yaml

Answer 1

This is a problem that comes up often. envsubst is commonly given as a solution, but I find it's easier to just stick with perl and do something like:
perl -pe 'while (my ($k, $v) = each %ENV) { s/\$$k/$v/g }'
This is almost certainly not a robust solution (it will replace $FOO, but it won't do replacements of the form ${FOO}), but I find I'm always disappointed that envsubst doesn't do ${FOO-bar}, and envsubst seems less ubiquitous than perl.
Or, rather than doing the replacement for everything in the environment, you might prefer something like:
perl -pe 's/\$([[:alpha:]_][_[:alnum:]]+)/$ENV{$1}/g'
or
perl -pe 's/\$([[:alpha:]_][\w]+)/$ENV{$1}/g'
These last two will replace '$FOO' with the empty string if FOO is not defined, while the first leaves it unreplaced. Which behavior you desire may drive the decision as to which to use.
I won't claim these are completely correct, but they are a reasonable approximation.

Answer 2

If You are using bash and the envsubst command is avaiable you can do:
envsubst < inputfile
E.g. (creating a temp input for demonstrating it:
$ env | tail -2 | sed 's_^_$_'
$MANPATH=/home/linuxbrew/.linuxbrew/share/man:
$INFOPATH=/home/linuxbrew/.linuxbrew/share/info:
Then running this through envsubst:
$ env | tail -2 | sed 's_^_$_' | envsubst
/home/linuxbrew/.linuxbrew/share/man:=/home/linuxbrew/.linuxbrew/share/man:
/home/linuxbrew/.linuxbrew/share/info:=/home/linuxbrew/.linuxbrew/share/info:

Answer 3

This might work for you (GNU sed):
sed '/value:/{y/"/\n/;s/^.*/printf "&"/e;y/\n/"/}' file
On any line containing the string value: convert any "'s to newlines, use printf to convert the environmental variables to their real values and reconvert the introduced newlines back to "'s.
N.B. If the environmental variable can contain "'s, these will need to be quoted following the printf command, i.e. insert s/"/\\"/g before the last y command.