I have been using AWS MWAA for a while, but recently had some issues with conflicting packages. So I decided to investigate this a little further.
For this purpose:
I created an MWAA v2.0.2 instance without additional Python packages. The idea here was to grab the packages they install by default and try to replicate it in my local.
Here are the packages they install by default:
alembic==1.5.8
amqp==2.6.1
apache-airflow==2.0.2
apache-airflow-providers-amazon==1.4.0
apache-airflow-providers-celery==1.0.1
apache-airflow-providers-cncf-kubernetes==1.0.0
apache-airflow-providers-databricks==1.0.1
apache-airflow-providers-docker==1.2.0
apache-airflow-providers-ftp==1.0.1
apache-airflow-providers-http==1.1.1
apache-airflow-providers-imap==1.0.1
apache-airflow-providers-oracle==1.1.0
apache-airflow-providers-postgres==1.0.2
apache-airflow-providers-presto==1.0.2
apache-airflow-providers-sftp==1.2.0
apache-airflow-providers-slack==3.0.0
apache-airflow-providers-sqlite==1.0.2
apache-airflow-providers-ssh==1.3.0
apache-airflow-providers-tableau==1.0.0
apispec==3.3.2
argcomplete==1.12.3
attrs==20.3.0
Babel==2.9.1
bcrypt==3.2.0
billiard==3.6.4.0
bleach==4.1.0
blinker==1.4
boto3==1.9.199
botocore==1.12.253
cached-property==1.5.2
cachetools==4.2.4
cattrs==1.5.0
cchardet==2.1.7
celery==4.4.7
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==7.1.2
clickclick==20.10.2
colorama==0.4.4
colorlog==5.0.1
commonmark==0.9.1
connexion==2.7.0
croniter==0.3.37
cryptography==3.4.7
cx-Oracle==8.3.0
defusedxml==0.7.1
dill==0.3.2
Django==2.2.24
dnspython==1.16.0
docker==3.7.3
docker-pycreds==0.4.0
docutils==0.15.2
email-validator==1.1.2
Flask==1.1.2
Flask-AppBuilder==3.3.2
Flask-Babel==1.0.0
Flask-Caching==1.10.1
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
flower==0.9.7
google-auth==2.3.3
graphviz==0.16
gunicorn==19.10.0
humanize==3.4.1
idna==2.10
importlib-metadata==1.7.0
importlib-resources==1.5.0
inflection==0.3.1
iso8601==0.1.14
isodate==0.6.0
itsdangerous==1.1.0
Jinja2==2.11.3
jinjasql==0.1.8
jmespath==0.10.0
jsonschema==3.2.0
kombu==4.6.11
kubernetes==11.0.0
lazy-object-proxy==1.4.3
lockfile==0.12.2
Mako==1.1.4
Markdown==3.3.4
MarkupSafe==1.1.1
marshmallow==3.11.1
marshmallow-enum==1.5.1
marshmallow-oneofschema==2.1.0
marshmallow-sqlalchemy==0.23.1
natsort==7.1.1
numpy==1.21.0
oauthlib==3.1.1
openapi-schema-validator==0.1.5
openapi-spec-validator==0.3.0
packaging==21.3
pandas==1.3.4
paramiko==2.8.1
pendulum==2.1.2
presto-python-client==0.7.0
prison==0.1.3
prometheus-client==0.8.0
psutil==5.8.0
psycopg2==2.9.2
psycopg2-binary==2.7.7
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pycurl==7.43.0.5
Pygments==2.8.1
PyJWT==1.7.1
PyNaCl==1.4.0
pyparsing==3.0.6
pyrsistent==0.17.3
pysftp==0.2.9
python-daemon==2.3.0
python-dateutil==2.8.1
python-editor==1.0.4
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2021.1
pytzdata==2020.1
PyYAML==6.0
requests==2.25.1
requests-oauthlib==1.3.0
rich==9.2.0
rsa==4.7.2
s3transfer==0.2.1
sentry-sdk==0.19.4
setproctitle==1.2.2
six==1.15.0
slack-sdk==3.12.0
SQLAlchemy==1.3.24
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.37.0
sqlparse==0.4.2
sshtunnel==0.1.5
statsd==3.3.0
swagger-ui-bundle==0.0.8
tableauserverclient==0.17.0
tabulate==0.8.9
tenacity==6.2.0
termcolor==1.1.0
text-unidecode==1.3
tornado==6.1
tracer-utils==0.0.1.dev32+g26636a5.d20220107
typing-extensions==3.7.4.3
unicodecsv==0.14.1
urllib3==1.25.11
vine==1.3.0
watchtower==1.0.6
webencodings==0.5.1
websocket-client==1.2.1
Werkzeug==1.0.1
WTForms==2.3.3
zipp==3.4.1
I tried to install these on my local, on the same Python version they are using - Python 3.7.10
This resulted in a conflict itself:
ERROR: Cannot install -r requirements-direct.txt (line 4) and watchtower==1.0.6 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested watchtower==1.0.6
apache-airflow-providers-amazon 1.4.0 depends on watchtower~=0.7.3
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
AWS should have used the constraints-3.7.txt file they mention in their documentation here https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-faqs.html#python-version but it doesn't look like it looking at installed packages.
How is it possible to install these conflicting packages and does the apache-airflow-providers-amazon package even work since it depends on watchtower~=0.7.3
Related
After running eb deploy & opening up my elastic beanstalk django website I get a module error of No module named 'turtle'. It all run perfect locally and I have PythonTurtle installed & in my requirements.txt. Is there a missing pip I haven't accounted for in my requirements.txt?
I've run pip3 freeze > requirements.txt many times with the same result.
How can I get past this error?
appdirs==1.4.4
asgiref==3.3.4
astroid==2.4.2
attrs==21.2.0
autopep8==1.5.7
awsebcli==3.20.2
bcrypt==3.2.0
blessed==1.18.1
botocore==1.21.65
bugsnag==4.2.0
cached-property==1.5.2
cement==2.8.2
certifi==2021.5.30
cffi==1.14.6
chardet==4.0.0
colorama==0.4.3
cryptography==3.4.7
distlib==0.3.1
dj-places==4.0.0
Django==3.1.6
django-crispy-forms==1.11.0
django-filter==21.1
django-mathfilters==1.0.0
djangorestframework==3.12.4
djangorestframework-simplejwt==5.0.0
docker==4.4.4
docker-compose==1.25.5
dockerpty==0.4.1
docopt==0.6.2
filelock==3.0.12
future==0.16.0
idna==2.10
importlib-metadata==4.8.2
isort==5.7.0
jmespath==0.10.0
jsonschema==3.2.0
lazy-object-proxy==1.4.3
Markdown==3.3
mccabe==0.6.1
paramiko==2.7.2
pathspec==0.5.9
psycopg2-binary==2.8.6
pycodestyle==2.7.0
pycparser==2.20
PyJWT==2.3.0
pylint==2.6.0
PyNaCl==1.4.0
pyrsistent==0.18.0
python-dateutil==2.8.1
PythonTurtle==0.3.2
pytz==2021.1
PyYAML==5.4.1
requests==2.25.1
s3transfer==0.5.0
selenium==3.141.0
semantic-version==2.8.5
six==1.14.0
sqlparse==0.4.1
termcolor==1.1.0
texttable==1.6.3
toml==0.10.2
tqdm==4.62.3
urllib3==1.26.7
virtualenv==20.4.2
wcwidth==0.1.9
websocket-client==0.59.0
wrapt==1.12.1
zipp==3.6.0
When I try to deploy my application to pythonanywhere, the following error
it is returned.
Could not find a version that satisfies the requirement ItsDangerous==1.0.0
How can I solve it? I tried to change the versions to a higher one or add other packages (as per the internet tips), but to no avail.
My requirements file.
absl-py==0.7.1
aiourllib==0.1.3
appdirs==1.4.3
asn1crypto==0.24.0
astor==0.7.1
beautifulsoup4==4.6.3
bleach==3.0.2
boto==2.49.0
boto3==1.9.103
botocore==1.12.103
bs4==0.0.1
certifi==2018.10.15
cffi==1.12.2
chardet==3.0.4
civis==1.9.0
civis-jupyter-extensions==0.1.3
civis-jupyter-notebook==0.4.2
click==6.7
cloudpickle==0.6.1
colorama==0.4.0
confusable-homoglyphs==3.2.0
crypto==1.4.1
cryptography==2.6.1
cssselect==1.0.3
cycler==0.10.0
decorator==4.3.0
defusedxml==0.5.0
Django==2.0
django-bootstrap3==11.0.0
django-bower==5.2.0
django-ckeditor==5.6.1
django-cors-headers==2.4.0
django-crispy-forms==1.7.2
django-debug-toolbar==1.9.1
django-filter==2.0.0
django-filters==0.2.1
django-js-asset==1.2.2
django-multiselectfield==0.1.8
django-registration==2.4.1
django-scheduler==0.8.7
django-storages==1.7.1
django-taggit==0.23.0
django-tinymce==2.8.0
django-widget-tweaks==1.4.3
djangorestframework==3.9.0
docutils==0.14
entrypoints==0.2.3
fake-useragent==0.1.11
Flask==1.0.2
gast==0.2.2
gitdb2==2.0.5
GitPython==2.1.11
grpcio==1.19.0
h5py==2.9.0
httplib2==0.11.3
icalendar==4.0.3
idna==2.7
ipykernel==5.1.0
ipython==6.2.1
ipython-genutils==0.2.0
ipywidgets==7.4.2
ItsDangerous==1.0.0
jedi==0.13.1
Jinja2==2.10
jmespath==0.9.4
joblib==0.11
jsonref==0.1
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.3
jupyter-console==6.0.0
jupyter-core==4.4.0
Keras-Applications==1.0.7
Keras-Preprocessing==1.0.9
kiwisolver==1.0.1
lxml==4.2.5
Markdown==3.0.1
MarkupSafe==1.0
matplotlib==3.0.3
mistune==0.8.4
Naked==0.1.31
nbconvert==5.4.0
nbformat==4.4.0
notebook==5.7.0
numpy==1.15.2
pandas==0.23.4
pandocfilters==1.4.2
parse==1.9.0
parso==0.3.1
pickleshare==0.7.5
Pillow==5.3.0
prometheus-client==0.4.2
prompt-toolkit==1.0.15
protobuf==3.7.0
psycopg2==2.7.5
pubnub==4.1.2
PyAutoGUI==0.9.41
pybase62==0.4.0
pycparser==2.19
pycrypto==2.6.1
pycryptodome==3.8.0
pycryptodomex==3.6.6
pyee==5.0.0
PyGetWindow==0.0.3
Pygments==2.2.0
PyMsgBox==1.0.6
pyparsing==2.2.2
pyppeteer==0.0.25
pyquery==1.4.0
PyRect==0.1.4
PyScreeze==0.1.19
python-dateutil==2.7.3
python-decouple==3.1
PyTweening==1.0.3
pytz==2018.5
pywinpty==0.5.4
PyYAML==3.13
pyzmq==17.1.2
qtconsole==4.4.2
requests==2.20.0
requests-html==0.9.0
s3transfer==0.2.0
scipy==1.1.0
seaborn==0.9.0
Send2Trash==1.5.0
shellescape==3.4.1
simplegeneric==0.8.1
six==1.11.0
smmap2==2.0.5
sqlparse==0.2.4
tensorboard==1.12.2
tensorflow==1.12.0
termcolor==1.1.0
terminado==0.8.1
testpath==0.4.2
tornado==5.1.1
tqdm==4.27.0
traitlets==4.3.2
Unidecode==1.0.23
urllib3==1.24
virtualenv==16.0.0
w3lib==1.19.0
wcwidth==0.1.7
webencodings==0.5.1
websockets==6.0
Werkzeug==0.14.1
widgetsnbextension==3.4.2
win-unicode-console==0.5
According to the CHANGES file, the 1.0.0 release was yanked, and is no longer available.
Try switching to 1.1.0, which is available on PyPI:
itsdangerous==1.1.0
I am trying to integrate Google Firebase with Django App. The error what i am facing is the makemigrations file get created in site packages of fcm_django and while migrating the model is not created in DB.
The Package I am using are below.
Babel==2.6.0
cairocffi==0.8.1
CairoSVG==2.1.3
certifi==2018.1.18
cffi==1.11.5
chardet==3.0.4
cssselect2==0.2.1
defusedxml==0.5.0
Django==1.11
django-cors-headers==2.2.0
django-haystack==2.8.1
django-oauth-toolkit==1.0.0
django-oauth2-provider==0.2.6.1
django-phonenumber-field==2.0.1
django-rest-framework-docs==0.1.7
django-uuidfield==0.5.0
django-vinaigrette==1.1.1
djangorestframework==3.7.7
drfdocs==0.0.11
elasticsearch==6.3.0
fcm-django==0.2.19
future==0.16.0
gcloud==0.17.0
html5lib==1.0.1
httplib2==0.11.3
idna==2.6
ipaddress==1.0.22
jsonfield==2.0.2
jsonpickle==0.9.6
jws==0.1.3
oauth2client==3.0.0
oauthlib==2.0.7
pdfrw==0.4
phonenumbers==8.9.11
Pillow==5.1.0
pkg-resources==0.0.0
protobuf==3.6.1
psycopg2==2.7.4
psycopg2-binary==2.7.4
pyasn1==0.4.4
pyasn1-modules==0.2.2
pycparser==2.18
pyfcm==1.4.5
PyJWT==1.6.4
Pyphen==0.9.4
PySocks==1.6.8
python-dateutil==2.7.3
python-jwt==2.0.1
pytz==2018.3
requests==2.19.1
requests-toolbelt==0.7.0
rsa==4.0
sendotp==0.0.4
shortuuid==0.5.0
simplejson==3.16.0
six==1.11.0
tinycss2==0.6.1
twilio==6.16.3
urllib3==1.22
WeasyPrint==0.42.3
webencodings==0.5.1
Could Someone please let me know Why isn't this making the db structure?
Thanks in advance
I have a project that was running normally until now when the code base has developed, now running very slow. I don't know what is wrong with it
Command like python manage.py runserver takes 3 minutes to load
Project dependencies
amqp==2.2.2
anyjson==0.3.3
appdirs==1.4.3
autopep8==1.3.5
billiard==3.5.0.3
cached-property==1.4.2
celery==4.2.0rc3
certifi==2018.4.16
chardet==3.0.4
defusedxml==0.5.0
dj-database-url==0.5.0
Django==2.0.6
django-annoying==0.10.3
django-celery-beat==1.1.1
django-cors-headers==2.2.0
django-heroku==0.3.1
django-redis==4.9.0
django-silk==3.0.0
djangorestframework==3.8.2
djangorestframework-xml==1.3.0
gprof2dot==2016.10.13
gunicorn==19.8.1
idna==2.6
isodate==0.6.0
Jinja2==2.10
kombu==4.1.0
lxml==4.2.1
Markdown==2.6.11
MarkupSafe==1.0
psycopg2==2.7.4
psycopg2-binary==2.7.4
pycodestyle==2.4.0
PyConfigure==0.5.9
Pygments==2.2.0
PyStaticConfiguration==0.10.3
python-dateutil==2.7.3
pytz==2018.4
PyYAML==3.12
redis==2.10.6
requests==2.18.4
requests-toolbelt==0.8.0
six==1.11.0
sqlparse==0.2.4
structlog==18.1.0
urllib3==1.22
vine==1.1.4
whitenoise==3.3.1
zeep==2.5.0
UPDATE
I found that i am using zeep client on my init.py
wsdl = 'http: . . . '
client = zeep.Client(wsdl)
How can i use it so that it doesn't hold server from starting
I am trying to deploy a django app to AWS using elastic beanstalk but it is saying that my requirements.txt is invalid when I run eb deploy. Can someone please help?
Do I need to create eb virtual environment inside conda virtual env ? Or i can use any one ? Is it the same thing ?
Error :
requirements.txt
alabaster==0.7.10
anaconda-client==1.6.9
anaconda-navigator==1.7.0
anaconda-project==0.8.2
asn1crypto==0.24.0
astroid==1.6.1
astropy==2.0.3
attrs==17.4.0
Babel==2.5.3
backports.shutil-get-terminal-size==1.0.0
beautifulsoup4==4.6.0
bitarray==0.8.1
bkcharts==0.2
blaze==0.11.3
bleach==2.1.2
bokeh==0.12.13
boto==2.48.0
botocore==1.10.11
Bottleneck==1.2.1
certifi==2018.1.18
cffi==1.11.4
chardet==3.0.4
click==6.7
cloudpickle==0.5.2
clyent==1.2.2
colorama==0.3.7
comtypes==1.1.4
conda==4.5.2
conda-build==3.4.1
conda-verify==2.0.0
contextlib2==0.5.5
cryptography==2.1.4
cycler==0.10.0
Cython==0.27.3
cytoolz==0.9.0
dask==0.16.1
datashape==0.5.4
decorator==4.2.1
distributed==1.20.2
Django==2.0.4
django-appconf==1.0.2
django-bootstrap3==10.0.1
django-bootstrap4==0.0.6
django-braces==1.13.0
django-crispy-forms==1.7.2
django-environ==0.4.4
django-forms-bootstrap==3.1.0
django-multiselectfield==0.1.8
django-rest-framework==0.1.0
django-select2==6.0.2
djangorestframework==3.8.2
docutils==0.14
entrypoints==0.2.3
et-xmlfile==1.0.1
fastcache==1.0.2
filelock==2.0.13
Flask==0.12.2
Flask-Cors==3.0.3
gevent==1.2.2
glob2==0.6
greenlet==0.4.12
h5py==2.7.1
heapdict==1.0.0
html5lib==1.0.1
idna==2.6
imageio==2.2.0
imagesize==0.7.1
ipython==6.2.1
ipython-genutils==0.2.0
ipywidgets==7.1.1
isort==4.2.15
itsdangerous==0.24
jdcal==1.3
jedi==0.11.1
Jinja2==2.10
jmespath==0.9.3
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.2
jupyter-console==5.2.0
jupyter-core==4.4.0
jupyterlab==0.31.4
jupyterlab-launcher==0.10.2
lazy-object-proxy==1.3.1
llvmlite==0.21.0
locket==0.2.0
lxml==4.1.1
MarkupSafe==1.0
matplotlib==2.1.2
mccabe==0.6.1
menuinst==1.4.11
mistune==0.8.3
mpmath==1.0.0
msgpack-python==0.5.1
multipledispatch==0.4.9
navigator-updater==0.1.0
nbconvert==5.3.1
nbformat==4.4.0
networkx==2.1
nltk==3.2.5
nose==1.3.7
notebook==5.4.0
numba==0.36.2
numexpr==2.6.4
numpy==1.14.0
numpydoc==0.7.0
odo==0.5.1
olefile==0.45.1
openpyxl==2.4.10
packaging==16.8
pandas==0.22.0
pandocfilters==1.4.2
parso==0.1.1
partd==0.3.8
path.py==10.5
pathlib2==2.3.0
patsy==0.5.0
pep8==1.7.1
pickleshare==0.7.4
Pillow==5.1.0
pkginfo==1.4.1
pluggy==0.6.0
ply==3.10
prompt-toolkit==1.0.15
psutil==5.4.3
py==1.5.2
pyasn1==0.4.2
pycodestyle==2.3.1
pycosat==0.6.3
pycparser==2.18
pycrypto==2.6.1
pycurl==7.43.0.1
pyflakes==1.6.0
Pygments==2.2.0
pylint==1.8.2
pyodbc==4.0.22
pyOpenSSL==17.5.0
pyparsing==2.2.0
PySocks==1.6.7
pytest==3.3.2
python-dateutil==2.7.2
pytz==2018.4
PyWavelets==0.5.2
pywin32==222
pywinpty==0.5
PyYAML==3.12
pyzmq==16.0.3
QtAwesome==0.4.4
qtconsole==4.3.1
QtPy==1.3.1
requests==2.18.4
rope==0.10.7
rsa==3.4.2
ruamel-yaml==0.15.35
s3transfer==0.1.13
scikit-image==0.13.1
scikit-learn==0.19.1
scipy==1.0.0
seaborn==0.8.1
Send2Trash==1.4.2
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.11.0
snowballstemmer==1.2.1
sortedcollections==0.5.3
sortedcontainers==1.5.9
south==1.0.2
Sphinx==1.6.6
sphinxcontrib-websupport==1.0.1
spyder==3.2.6
SQLAlchemy==1.2.1
statsmodels==0.8.0
sympy==1.1.1
tables==3.4.2
tblib==1.3.2
terminado==0.8.1
testpath==0.3.1
toolz==0.9.0
tornado==4.5.3
traitlets==4.3.2
typing==3.6.2
unicodecsv==0.14.1
urllib3==1.22
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.14.1
widgetsnbextension==3.1.0
win-inet-pton==1.0.1
win-unicode-console==0.5
wincertstore==0.2
wrapt==1.10.11
xlrd==1.1.0
XlsxWriter==1.0.2
xlwings==0.11.5
xlwt==1.3.0
zict==0.1.3