fbprophet failed to build wheel in google cloud function - google-cloud-platform

Question

I am hoping to use Fbprophet on my cloud function in a Python 3.7 environment, but it fails to build and gives me the following error.
Build failed: `pip_download_wheels` had stderr output:
ERROR: Command errored out with exit status 1:
command: /opt/python3.7/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-srnqu7b5/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-srnqu7b5/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-3_5khs54
cwd: /tmp/pip-wheel-srnqu7b5/fbprophet/
Complete output (40 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/fbprophet
creating build/lib/fbprophet/stan_model
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-wheel-srnqu7b5/fbprophet/setup.py", line 148, in <module>
"""
File "/opt/python3.7/lib/python3.7/site-packages/setuptools/__init__.py", line 140, in setup
return distutils.core.setup(**attrs)
File "/opt/python3.7/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/python3.7/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/opt/python3.7/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/opt/python3.7/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 202, in run
self.run_command('build')
File "/opt/python3.7/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/python3.7/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/opt/python3.7/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/python3.7/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/python3.7/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-wheel-srnqu7b5/fbprophet/setup.py", line 48, in run
build_models(target_dir)
File "/tmp/pip-wheel-srnqu7b5/fbprophet/setup.py", line 36, in build_models
from fbprophet.models import StanBackendEnum
File "/tmp/pip-wheel-srnqu7b5/fbprophet/fbprophet/__init__.py", line 8, in <module>
from fbprophet.forecaster import Prophet
File "/tmp/pip-wheel-srnqu7b5/fbprophet/fbprophet/forecaster.py", line 14, in <module>
import numpy as np
ModuleNotFoundError: No module named 'numpy'
----------------------------------------
ERROR: Failed building wheel for fbprophet
ERROR: Failed to build one or more wheels
error: `pip_download_wheels` returned code: 1; Error ID: 618AA8E7
This is what my requirements.txt file looks like:
cython
pystan
numpy
pandas==1.0.3
google-cloud-storage==1.29.0
fbprophet
geopy==1.22.0
google-cloud-bigquery==1.25.0
Everything works perfectly fine locally in a Python 3.7 virtual environment on jupyter notebook. Would appreciate any help because I've spent almost an entire day trying to fix this but to no avail.

Answer 1

I'm having similar issues to this. My goal is to deploy a function that, when passed some input, will feed this into a Prophet model in order to make a prediction before passing the prediction to another part of my system.
As far as I can tell, there are a few things that make this complicated.
First, there is the issue of build dependencies that #mgoya mentioned in the comment above. This manifests both when installing Prophet and when installing Pystan (a dependency of Prophet). In my cloudbuild.yaml I'm attempting to circumvent this by installing the dependencies in sequence, like this:
steps:
- name: 'docker.io/library/python:3.9'
entrypoint: /bin/sh
# Run pip install and pytest in the same build step
# (pip packages won't be preserved in future steps!)
args: [-c, 'pip uninstall pystan; pip install convertdate==2.1.2 lunarcalendar==0.0.9 holidays==0.10.3 pgen tqdm cython pandas numpy setuptools; pip install pystan==2.19.1.1; pip install -r requirements.txt']
timeout: 1200s
This first uninstalls Pystan (if one exists locally), then installs the build dependencies for Pystan (a build dependency for Prophet), then installs Prophet (which is included in my requirements.txt).
Note the semicolons. I ran into issues when using double-ampersands instead (i.e. pip install ... && pip install ... etc.. I believe this may be due to the way that the installed packages make their way to the local file system and are made available to the following commands. By running them with && this seems to try and run them all at once, which won't give the earlier parts of the installation time to propagate and be discoverable by later steps. Using semicolons seems to help with this but it's anecdotal evidence at best.
You could also try using python -m pip install ... rather than pip install though I'm not familiar enough with Python to tell you the difference off the top of my head.
Secondly, there are the memory requirements for installing Pystan. I've read somewhere that Pystan requires 4GB of memory to install. The default machine_type for cloudbuild does not have that much. You can try increasing the size of the machine by changing its type.
Finally, there's the resources allocated for function execution. Again, I read somewhere that Prophet / Pystan require 2GB of memory just to run one of the models. So, if your cloud function doesn't have this, it may run into memory issues when trying to execute. In my experience so far, memory issues are not that transparent within Google Cloud.
--
My current thinking (and perhaps what I'd recommend to anyone else reading this post) is to consider whether Cloud Functions is the right tool for this, given the weight of the dependencies. Pystan and Prophet are rather special-case dependencies, given their build and runtime resource requirements.
What I've opted to do is build (locally) a container with these dependencies baked in and pushing this to Google's Container Registry. My plan from here is to use that container as the base image for a Cloud Run application, which is significantly easier to deploy. This has obvious drawbacks but if my base image changes infrequently (which it will) I think this approach would be fine. Unfortunately, this model ("bring your own container") is not supported by Cloud Functions - it's designed where all you bring is code.