How do I use command substition in Dockerfile - dockerfile

Question

In my Dockerfile I need to use command substition to add some environment variables. I want to set
ENV PYTHONPATH /usr/local/$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')
but it doesn't work. The result is
foo#bar:~$ echo $PYTHONPATH
/usr/local/$(python3 -c from distutils import sysconfig; print(sysconfig.get_python_lib()))
What's wrong?

Answer 1

What went wrong
The $( ... ) command substitution you attempted is for Bash, whereas the Dockerfile is not Bash. So docker doesn't know what to do with that, it's just plain text to docker, docker just spews out what you wrote as-is.
Recommendation
To avoid hard-coding values into a Dockerfile, and instead, to dynamically change a setting or custom variable as PYTHONPATH during the build, perhaps the ARG ... , --build-arg docker features might be most helpful, in combination with ENV ... to ensure it persists.
Within your Dockerfile:
ARG PYTHON_PATH_ARG
ENV PYTHONPATH ${PYTHON_PATH_ARG}
In Bash where you build your container:
python_path="/usr/local$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"
docker build --build-arg PYTHON_PATH_ARG=$python_path .
Explanation
According to documentation, ARG:
The ARG instruction defines a variable that users can pass at build-time to the builder with the docker build command using the --build-arg <varname>=<value> flag.
So, in Bash we first:
python_path="/usr/local$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"
$(...) Bash command substitution is used to dynamically put together a Python path value
this value is stored temporarily in a Bash variable $python_path for clarity
docker build --build-arg PYTHON_PATH_ARG=$python_path .
Bash variable $python_path value is passed to docker's --build-arg PYTHON_PATH_ARG
Within the Dockerfile:
ARG PYTHON_PATH_ARG
so PYTHON_PATH_ARG stores the value from --build-arg PYTHON_PATH_ARG...
ARG variables are not equivalent to ENV variables, so we couldn't merely do ARG PYTHONPATH and be done with it. According to documentation about Using arg variables:
ARG variables are not persisted into the built image as ENV variables are.
So finally:
ENV PYTHONPATH ${PYTHON_PATH_ARG}
We use Dockerfile's ${...} convention to get the value of PYTHON_PATH_ARG, and save it to your originally named PYTHONPATH environment variable
Differences from original code
You originally wrote:
ENV PYTHONPATH /usr/local/$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')
I re-wrote the Python path finding portion as a Bash command, and tested on my machine:
$ python_path="/usr/local/$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"
$ echo $python_path
/usr/local//usr/lib/python3/dist-packages
Notice there is a double forward slash ... local//usr ... , not sure if that will break anything for you, depends on how you use it in your code.
Instead, I changed it to:
$ python_path="/usr/local$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"
Result:
$ echo $python_path
/usr/local/usr/lib/python3/dist-packages
So this new code will have no double forward slashes.

Answer 2

You should use ARG if possible. But sometimes you really need to use command substitution for a dynamic variable. As long as you put all the commands in the same RUN statement, then you can still access the value.
RUN foo=$(date) && \
echo $foo