Airflow installation failure beam[gcp] - google-cloud-platform

Airflow installation with command is failing
sudo pip3 install apache-airflow[gcp_api]
Everything was working fine yesterday. Today I see the following error:
Could not find a version that satisfies the requirement apache-beam[gcp]==2.3.0 (from google-cloud-dataflow->apache-airflow[gcp_api]) (from versions: 0.6.0, 2.0.0, 2.1.0, 2.1.1, 2.2.0)
No matching distribution found for apache-beam[gcp]==2.3.0 (from google-cloud-dataflow->apache-airflow[gcp_api])
Can someone help me on this?
Thanks in advance

I faced the same problem :(
Why?
most likely it happened because in the new version(2.3.0) of apache-beam they actually added the restriction for python3
https://pypi.python.org/pypi/apache-beam/2.3.0
Requires Python: >=2.7,<3.0
the previous packages didn't have this restriction, that is why it was working before(if you didn't use dataflow from gcp).
probably you have the last version of https://pypi.python.org/pypi/google-cloud-dataflow/2.3.0 which has updated apache-beam package
How to fix?
uninstall google-cloud-dataflow
pip3 uninstall google-cloud-dataflow
and install version 2.2.0 which has the old version of apache beam
pip install google-cloud-dataflow==2.2.0
Fixed this problem for me, I hope it will help you as well

This has been resolved in the master branch at Apache Airflow Github by Pull Request #3273 .
You can install the latest development branch using the below commands.
pip install git+https://github.com/apache/incubator-airflow
pip install git+https://github.com/apache/incubator-airflow#egg=apache-airflow[gcp_api]

Related

How can I upgrade AWS SAM to a certain version?

I'm trying to upgrade SAM CLI, version 1.18.2 to 1.22.0 on Ubuntu 20.04. I used brew command:
~brew upgrade aws-sam-cli#1.22.0
but didn't work. Showed the next message:
Error: No available formula or cask with the name "aws-sam-cli#1.22".
Also I'm trying to find a downloadable to install a certain version.
Thanks for your help!
https://github.com/aws/aws-sam-cli/issues/1767
For anyone in the future, use pip.
pip3 install --user 'aws-sam-cli==1.12.0'

Cannot install "psycopg2" on Windows 10 with Python 3.8

Yesterday I uninstalled python 3.7 version by mistake. Then I install python again (this time 3.8 version) and again set up my environment. But I could not start my Django project which has Postgres connection. Actually I cannot install "psycopg2" in my environment. I searched for hours and implement every solutions I get from online but it does not work. Let me tell you what I did so far.
First it said to add Postgres in my PATH so I added C:\Program Files\PostgreSQL\12\bin\ in my PATH.
A new problem then arise with a huge ERROR report with 2 vital Error.
ERROR: Failed building wheel for psycopg2
..........................
Running setup.py install for psycopg2 ... error
I try to upgrade wheel but it says,
Requirement already up-to-date
http://initd.org/psycopg/docs/install.html#install-from-source
I learned from this site that psycopg2 requires python2 version. So I installed python 2.7 also.
I reinstalled PostgreSQL but it does not work.
I deleted my virtual environment and create again but it does not work.
Some says they solve this problem by running pip install psycopg2-binary But it does not work for me.
Please help me to get rid of this. I stuck for hours.
When you asked this question, Python 3.8 had been released very recently so there were not any wheels for Python 3.8 yet.
At the time, my suggestion was to install Python 3.7.X and install the binary wheel with:
pip install psycopg2-binary
Since then, binary wheels have been released for Python 3.8, so the above command should work with Python 3.8.X as well.
I wouldn't try to build from source on Windows if it can be avoided.
Finally, you misunderstood the section of the docs about Python 2. You only need Python 2.7 if you are running Python 2. For Python 3, which you should be using for all new projects, it currently supports Python 3.4 to 3.8.
use:
sudo apt install python3-dev libpq-dev
then try doing:
pip3 install psycopg2
Hope it works for you!!
This problem mainly occurs due to this -- " error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/"
Now to install psycopg2, first you need to install visual studio from Microsoft - https://visualstudio.microsoft.com/visual-cpp-build-tools/
now you need to install the c++ desktop development tool with all its default components selected.
After successful visual studio c++ desktop development kit installation, you can now install psycopg2 successfully in your machine.

Can not install SAM CLI on mac os

I am trying to follow instructions mentioned in here to setup my first Amazon Lambda -based app. However when I type "pip install aws-sam-cli" I get the following error :
"Collecting aws-sam-cli
Could not find any downloads that satisfy the requirement aws-sam-cli
No distributions at all found for aws-sam-cli"
I am using python 2.7.10.
Any help would be appreciated.
So the problem was with python versions. I had an corrupted old version of python 2.7 ... I installed 3.6 but it did not work. So I deleted and uninstalled EVERYTHING related to pythons and installed 2.7 from scratch.After that I was able to install SAM cli smoothly without any problem.
So conclusion , make sure your python is working properly and is up-to-date. Make sure you do not have version conflicts.

pip installation error for tensorflow

I was trying to install tensorflow from source on ubuntu 14.04, python 2.7.
I followed the steps from "tensorflow.org" for source installation.
I had completed all the steps, such as bazel installation, python dependencies installation.
In the final step for sudo pip installation; the command was as follows:-
$sudo pip install /tmp/tensorflow_pkg/tensorflow-1.3.0rc0-cp27-none-linux_x86_64.whl
but i am getting error as follows:
Unpacking /tmp/tensorflow_pkg/tensorflow-1.3.0rc0-cp27-none-linux_x86_64.whl
Downloading/unpacking tensorflow-tensorboard (from tensorflow==1.3.0rc0)
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement tensorflow-tensorboard (from tensorflow==1.3.0rc0)
Cleaning up...
No distributions at all found for tensorflow-tensorboard (from tensorflow==1.3.0rc0)
Storing debug log for failure in /home/ubuntu/.pip/pip.log
i also checked tensorflow_pkg wheel :but the above package was not available there.
so can a different .whl can be insatlled using pip such as -tensorflow-1.2.1-cp27-none-linux_x86_64.whl
which have downloaded in my desktop.
Please tell me how i can resolve this issue.
Thanks and regards
I didn't figure out the root of the problem, but for what its worth I skipped the tensorboard issue by installing an older version (1.2.0 worked for me)
pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.2.0-cp27-none-linux_x86_64.whl
Try:
pip install tensorflow
If that doesn't work:
sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.2.1-cp27-none-linux_x86_64.whl
More info here and Common problems here.
My issue was resolved by following steps.
I updated the pip using
Also exported proxy settings.
e.g export https_proxy=...
This was able to resolve my issue.
I also tried installing other .whl file for tensorflow and yes tensorflow can be installed using the downloaded .whl file also.
Thank you

How To Download django-mssql On Windows

I can't download django-mssql from Google Code there doesn't seem to be a direct download link and the svn wont work for me.
Anybody know a direct download link?
You can install the latest release from PyPi using pip install django-mssql or easy_install django-mssql. If you aren't using pip already then I would recommend you start.
The source is under a mercurial repository. If you have a mercurial client installed, you can check it out with the following command:
hg clone https://bitbucket.org/Manfre/django-mssql/ django-mssql