Sagemaker lifecycle config: could not find conda environment conda_python3 - amazon-web-services

Question

the below script should run a notebook called prepTimePreProcessing whenever a AWS notebook instance starts runing.
however I am getting "could not find conda environment conda_python3" error from the lifecycle config file.
set -e
ENVIRONMENT=python3
NOTEBOOK_FILE="/home/ec2-user/SageMaker/prepTimePreProcessing.ipynb"
echo "Activating conda env"
source /home/ec2-user/anaconda3/bin/activate "$ENVIRONMENT"
echo "Starting notebook"
nohup jupyter nbconvert --to notebook --inplace --ExecutePreprocessor.timeout=600 --ExecutePreprocessor.kernel_name=python3 --execute "$NOTEBOOK_FILE" &
Any help whould be appreciated.

Answer 1

Assuming no environment problems, if you open a terminal in the instance in use and run:
conda env list
the result should also contain this line:
python3 /home/ec2-user/anaconda3/envs/python3
After that, you can create a .sh script inside /home/ec2-user/SageMaker containing all the code to run. This way it also becomes versionable by being a persisted file in the instance space and not inside an external configuration.
The on-start.sh/on-create.sh (from this point I will simply call it script.sh) file becomes trivially:
# PARAMETERS
ENVIRONMENT=python3
# conda env
source /home/ec2-user/anaconda3/bin/activate "$ENVIRONMENT";
echo "'$ENVIRONMENT' env activated"
In the lifecycle config, on the other hand, just write a few lines to invoke the previously created script.sh:
#!/bin/bash
set -e
SETUP_FILE=/home/ec2-user/SageMaker/script.sh
echo "Run setup script"
sh "$SETUP_FILE"
echo "Setup completed!"
Extra
If you want to add a safety check so that the .sh file is read correctly regardless of line breaks, I would also add a conversion:
#!/bin/bash
set -e
SETUP_FILE=/home/ec2-user/SageMaker/script.sh
# convert script to unix format
echo "Converting setup script into unix format"
sudo yum -y install dos2unix > /dev/null 2>&1
dos2unix "$SETUP_FILE" > /dev/null 2>&1
echo "Run setup script"
sh "$SETUP_FILE"
echo "Setup completed!"