sqlalchemy showing an error related to os.getenv() - flask

I am working on sqlalchemy and there's command in it engine -
create_engine(os.getenv("DATABASE_URL")).
When I run the program it shows an error saying -
"Could not parse rfc1738 URL from string 'C:\Program Files\ PostgreSQL\10\data"
Code
import os
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
#engine = create_engine('sqlite://')
engine = create_engine(os.getenv("DATABASE_URL"))
db = scoped_session(sessionmaker(bind=engine))
def main():
flights = db.execute("SELECT origin, destinaiton, duration FROM flights").fetchall()
for flights in flights:
print(f"{flight.origin} to {flight.destination}, {flight.duration} minutes.")
if __name__ == "__main__":
main()
Error
Traceback (most recent call last): File "list.py", line 7, in
engine = create_engine(os.getenv("DATABASE_URL")) File "C:\Users\Amber
Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine__init__.py",
line 424, in create_engine
return strategy.create(*args, **kwargs) File "C:\Users\Amber Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine\strategies.py",
line 50, in create
u = url.make_url(name_or_url) File "C:\Users\Amber Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine\url.py",
line 211, in make_url
return _parse_rfc1738_args(name_or_url) File "C:\Users\Amber Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine\url.py",
line 270, in _parse_rfc1738_args
"Could not parse rfc1738 URL from string '%s'" % name) sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string
'C:\Program Files\PostgreSQL\10\data'
Please someone help me resolving this issue.

The engine URL shouldn't be a path to your local Postgres installation but a sting that tells SQLAlchemy how to connect to the database. It has the following format:
postgresql://username:password#server:port/databasename

I'm also watching course video of cs50 now. You should try to just use create_engine("your_database_url").
If you still want to use create_engine(os.getenv("DATABASE_URL")) tp process, make sure you can log in using the command line like this psql $DATABASE_URL.

using psql $DATABASE_URL works for me but I'm still getting the same error on create_engine("your_database_url")
(project1) C:\Users\Gregory Goufan\Downloads\CS50s Web Programming with Python and JavaScript\project1\project1\project1>psql postgresql://openpg:openpgpwd#localhost:5432/postgres
psql (9.5.8)
WARNING: Console code page (437) differs from Windows code page (1252)
8-bit characters might not work correctly. See psql reference
page "Notes for Windows users" for details.
Type "help" for help.
postgres=#
But it is true that by putting the url direct in create_engine funciton works strange thing because it is indeed a string which is returned
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string ' 'postgresql://openpg:openpgpwd#localhost:5432/postgres''

Related

Fetchingdata from Cloud Firstore results in OSError when SUBPROCESS.POPEN.COMMUNICATE() method is used

I have sample script that starts by opening a connection to firebase cloud database and retrieving the information from the database using get() method. Script then moves to a infinite loop that calls subprocess.popen.communicate() method.
Output -
1. Data is retrieved successfully from the database .
2. Infinite loop runs successfully for 4 minutes and then terminates with OSError: Bad File Descriptor
import sys, os, time, subprocess
import google.auth.transport.requests
import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore
import datetime
def firebase_retrieve_data():
cred =firebase_admin.credentials.Certificate('path/to/serviceAccount.json')
app=firebase_admin.initialize_app(cred)
db= firestore.client()
doc_ref = db.collection(u'collection_name').document(u'doc_name')
try:
doc = doc_ref.get()
firebase_admin.delete_app(app)
except Exception as e:
firebase_admin.delete_app(app)
print(e)
print(u'No such document!')
firebase_retrieve_data()
while(True):
x=subprocess.Popen("pwd", stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds= True, shell=True).communicate()
Although both pieces seem independent of each other and OSError is generated due to subprocess.popen.communicate() method, I suspect error is related to get() function that retrieves the data from the database. Reason I say this is because, while the script was running I ran following command to get list of active connections in the background
"sudo lsof -i -n | grep -E "(LISTEN|ESTABLISHED)"
I found out that get() function opens a connection to database that remains open for about 4-5 minutes(I assume thats the timeout for the database connection). As soon as this database connection closes, the script fails with OSError: [Errno 9] Bad file descriptor on line
x=subprocess.Popen("pwd", stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds= True, shell=True).communicate()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in init
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
I suspect that it has something to do with improper closing of database connection but all the firebase examples or python firebase documentation I see has no mention of proper closing of database connection()
Plus, I am failing to understand the reason of how firebase database connection should effect call to subprocess.popen.communciate() method
Used
os.system("pwd")
instead of
subprocess.Popen("pwd", stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds= True, shell=True).communicate()
This Works . However, still don't know the reason why subprocess.Popen.communicate() and os.popen() throw an error

httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com

When i try to fetch the details of gmail using google-api's (googleapiclient,oauth2client), i am getting below error:
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/google_api_python_client-1.7.8-py2.7.egg/googleapiclient/_helpers.py", line 130, in positional_wrapper
File "/usr/local/lib/python2.7/site-packages/google_api_python_client-1.7.8-py2.7.egg/googleapiclient/discovery.py", line 224, in build
File "/usr/local/lib/python2.7/site-packages/google_api_python_client-1.7.8-py2.7.egg/googleapiclient/discovery.py", line 274, in _retrieve_discovery_doc
File "/usr/local/lib/python2.7/site-packages/httplib2-0.12.1-py2.7.egg/httplib2/__init__.py", line 2135, in request
cachekey,
File "/usr/local/lib/python2.7/site-packages/httplib2-0.12.1-py2.7.egg/httplib2/__init__.py", line 1796, in _request
conn, request_uri, method, body, headers
File "/usr/local/lib/python2.7/site-packages/httplib2-0.12.1-py2.7.egg/httplib2/__init__.py", line 1707, in _conn_request
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com
but it is working fine in my pc but not from remote location.
code:
from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
credentials = ServiceAccountCredentials.from_json_keyfile_name(
"quickstart-1551349397232-e8bcb3368ae1.json", scopes=
['https://www.googleapis.com/auth/admin.directory.group', 'https://www.googleapis.com/auth/admin.directory.user', 'https://www.googleapis.com/auth/admin.directory.domain', 'https://www.googleapis.com/auth/gmail.readonly'])
delegated_credentials = credentials.create_delegated('jango#carbonitedepot.com')
DIRECOTORY = build('admin', 'directory_v1', credentials=delegated_credentials)
try:
results = DIRECOTORY.users().list(customer='my_customer').execute()
users = results.get('users', [])
res = []
for info in users:
print(info)
res.append(info.get("primaryEmail"))
print(res)
except Exception as e:
print(e)
Any help would be much appreciated.
Thanks in advance.
I had the same issue and I searched a lot to get it fixed, turns out I was at fault here.
I don't know from where there was podman installed instead of docker and that caused the problem.
I suggest you check your docker version and make sure the latest is running at the server, otherwise, the library works fine! Please let me know if you still face this issue, would like to dig deep!
The httplib2 is likely scanning through network interface DHCP DNS nameservers (in your registry or visible to docker) and then trying to connect through a stale DNS or possible an IPv6. Patching your socket in the main routine may solve it:
# Monkey patch to force IPv4, since FB seems to hang on IPv6
import socket
old_getaddrinfo = socket.getaddrinfo
def new_getaddrinfo(*args, **kwargs):
responses = old_getaddrinfo(*args, **kwargs)
return [response
for response in responses
if response[0] == socket.AF_INET]
socket.getaddrinfo = new_getaddrinfo

Missing environment variables in OpenShift

I am trying to set up a Python 2.7 project in Openshift. The goal is to use Tornado Websockets. I have tried a number of examples, but I keep getting stuck on a number of missing environment variables that are shown in the examples.
My current app.py:
import imp
import os
try:
zvirtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'],
'virtenv', 'bin', 'activate_this.py')
execfile(zvirtenv, dict(__file__ = zvirtenv) )
except IOError:
pass
if __name__ == '__main__':
ip = os.environ['OPENSHIFT_PYTHON_IP']
port = int(os.environ['OPENSHIFT_PYTHON_PORT'])
app = imp.load_source('application', 'main.py')
app.application.listen(port , ip)
app.ioloop.IOLoop.instance().start()
My install requires:
install_requires=['tornado', 'requests', 'beautifulsoup4']
This results in the following error:
---> Running application from Python script (app.py) ...
Traceback (most recent call last):
File "app.py", line 14, in <module>
zvirtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'],
File "/opt/app-root/lib64/python2.7/UserDict.py", line 40, in __getitem__
raise KeyError(key)
KeyError: 'OPENSHIFT_PYTHON_DIR'
Can anybody help me out?
You are looking for environment variables set when using OpenShift 2, but are using OpenShift 3. Under OpenShift 3 you do not need to activate the Python virtual environment, it is done for you. Your app.py should listen on all interfaces and on port 8080. That address doesn't change so long as using the default Python S2I builder, so environment variables not used to pass it in.

Setting Spotify credentials using Spotipy

I am trying out spotipy with python 2.7.10 preinstalled on my mac 10.10, specifically [add_a_saved_track.py][1] Here is the code as copied from github:
# Add tracks to 'Your Collection' of saved tracks
import pprint
import sys
import spotipy
import spotipy.util as util
scope = 'user-library-modify'
if len(sys.argv) > 2:
username = sys.argv[1]
tids = sys.argv[2:]
else:
print("Usage: %s username track-id ..." % (sys.argv[0],))
sys.exit()
token = util.prompt_for_user_token(username, scope)
if token:
sp = spotipy.Spotify(auth=token)
sp.trace = False
results = sp.current_user_saved_tracks_add(tracks=tids)
pprint.pprint(results)
else:
print("Can't get token for", username)
I registered the application with developer.spotify.com/my-applications and received client_id and client_secret. I am a bit unclear about selection of redirect_uri so I set that to 'https://play.spotify.com/collection/songs'
Running this from terminal I get an error that says:
You need to set your Spotify API credentials. You can do this by
setting environment variables like so:
export SPOTIPY_CLIENT_ID='your-spotify-client-id'
export SPOTIPY_CLIENT_SECRET='your-spotify-client-secret'
export SPOTIPY_REDIRECT_URI='your-app-redirect-url'
I put that into my code with the id, secret, and url as strings, just following the imports but above the util.prompt_for_user_token method.
That caused a traceback:
File "add-track.py", line 8
export SPOTIPY_CLIENT_ID='4f...6'
^
SyntaxError: invalid syntax
I noticed that Text Wrangler does not recognize 'export' as a special word. And I searched docs.python.org for 'export' and came up with nothing helpful. What is export? How am I using it incorrectly?
I next tried passing the client_id, client_secret, and redirect_uri as arguments in the util.prompt_for_user_token method like so:
util.prompt_for_user_token(username,scope,client_id='4f...6',client_secret='xxx...123',redirect_uri='https://play.spotify.com/collection/songs')
When I tried that, this is what happens in terminal:
User authentication requires interaction with your
web browser. Once you enter your credentials and
give authorization, you will be redirected to
a url. Paste that url you were directed to to
complete the authorization.
Opening https://accounts.spotify.com/authorize?scope=user-library-modify&redirect_uri=https%3A%2F%2Fplay.spotify.com%2Fcollection%2Fsongs&response_type=code&client_id=4f...6 in your browser
Enter the URL you were redirected to:
I entered https://play.spotify.com/collection/songs and then got this traceback:
Traceback (most recent call last):
File "add-track.py", line 21, in <module>
token = util.prompt_for_user_token(username, scope, client_id='4f...6', client_secret='xxx...123', redirect_uri='https://play.spotify.com/collection/songs')
File "/Library/Python/2.7/site-packages/spotipy/util.py", line 86, in prompt_for_user_token
token_info = sp_oauth.get_access_token(code)
File "/Library/Python/2.7/site-packages/spotipy/oauth2.py", line 210, in get_access_token
raise SpotifyOauthError(response.reason)
spotipy.oauth2.SpotifyOauthError: Bad Request
It seems like I am missing something, perhaps another part of Spotipy needs to be imported, or some other python module. It seems I am missing the piece that sets client credentials. How do I do that? I am fairly new at this (if that wasn't obvious). Please help.
UPDATE: I changed redirect_uri to localhost:8888/callback. That causes a Firefox tab to open with an error -- "unable to connect to server." (Since I do not have a server running. I thought about installing node.js as in the Spotify Web API tutorial, but I have not yet). The python script then asks me to copy and paste the URL I was redirected to. Even though FF could not open a page, I got this to work by copying the entire URL including the "code=BG..." that follows localhost:8888/callback? I am not sure this is an ideal setup, but at least it works.
Does it matter if I set up node.js or not?
The process you've followed (including your update) is exactly as the example intends and you are not missing anything! Obviously, it is a fairly simple tutorial, but it sets you up with a token and you should be able to get the information you need.
For the credentials, you can set these directly in your Terminal by running each of the export commands. Read more about EXPORT here: https://www.cyberciti.biz/faq/linux-unix-shell-export-command/

Error using api.update_status method in tweepy using Oauth2

Here is my code:-
I have double checked all the auth parameters.
import tweepy
CONSUMER_KEY ='#Omitted - you should not publish your actual key'
CONSUMER_SECRET ='#Omitted - you should not publish your actual secret'
ACCESS_KEY='#Omitted - you should not publish your access key'
ACCESS_SECRET = '#Omitted - you should not publish your access secret'
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth)
api.update_status('Tweeting from command line')
Saved the file in home folder as status.py
after running python status.py follwing error comes:-
Traceback (most recent call last):
File "status.py", line 14, in <module>
api.update_status('Tweeting from command line')
File "/usr/local/lib/python2.7/dist-packages/tweepy-1.10-py2.7.egg/tweepy/binder.py", line 185, in _call
return method.execute()
File "/usr/local/lib/python2.7/dist-packages/tweepy-1.10-py2.7.egg/tweepy/binder.py", line 168, in execute
raise TweepError(error_msg, resp)
tweepy.error.TweepError: Could not authenticate with OAuth.
Please, help me out
I received this error under the same conditions - using tweepy, all of my keys/secrete were copy and pasted correctly. The problem was the time on my server. After running ntpdate -b pool.ntp.org I was to use tweepy just fine.
I am able to authenticate using tweepy, I have an extra line in my code though, it might help for you to change your code to this:
import tweepy
from tweepy import OAuthHandler
then proceede with the rest of your code. Also add a line in your code to print out to the shell to show your connect as follows:
print api.me().name
Make sure the line you see above this is right after api = tweepy.API(auth)
Try api.update_status(status='Tweeting from command line'). It helped me.