I am trying to set up a Python 2.7 project in Openshift. The goal is to use Tornado Websockets. I have tried a number of examples, but I keep getting stuck on a number of missing environment variables that are shown in the examples.
My current app.py:
import imp
import os
try:
zvirtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'],
'virtenv', 'bin', 'activate_this.py')
execfile(zvirtenv, dict(__file__ = zvirtenv) )
except IOError:
pass
if __name__ == '__main__':
ip = os.environ['OPENSHIFT_PYTHON_IP']
port = int(os.environ['OPENSHIFT_PYTHON_PORT'])
app = imp.load_source('application', 'main.py')
app.application.listen(port , ip)
app.ioloop.IOLoop.instance().start()
My install requires:
install_requires=['tornado', 'requests', 'beautifulsoup4']
This results in the following error:
---> Running application from Python script (app.py) ...
Traceback (most recent call last):
File "app.py", line 14, in <module>
zvirtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'],
File "/opt/app-root/lib64/python2.7/UserDict.py", line 40, in __getitem__
raise KeyError(key)
KeyError: 'OPENSHIFT_PYTHON_DIR'
Can anybody help me out?
You are looking for environment variables set when using OpenShift 2, but are using OpenShift 3. Under OpenShift 3 you do not need to activate the Python virtual environment, it is done for you. Your app.py should listen on all interfaces and on port 8080. That address doesn't change so long as using the default Python S2I builder, so environment variables not used to pass it in.
Related
I am working on sqlalchemy and there's command in it engine -
create_engine(os.getenv("DATABASE_URL")).
When I run the program it shows an error saying -
"Could not parse rfc1738 URL from string 'C:\Program Files\ PostgreSQL\10\data"
Code
import os
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
#engine = create_engine('sqlite://')
engine = create_engine(os.getenv("DATABASE_URL"))
db = scoped_session(sessionmaker(bind=engine))
def main():
flights = db.execute("SELECT origin, destinaiton, duration FROM flights").fetchall()
for flights in flights:
print(f"{flight.origin} to {flight.destination}, {flight.duration} minutes.")
if __name__ == "__main__":
main()
Error
Traceback (most recent call last): File "list.py", line 7, in
engine = create_engine(os.getenv("DATABASE_URL")) File "C:\Users\Amber
Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine__init__.py",
line 424, in create_engine
return strategy.create(*args, **kwargs) File "C:\Users\Amber Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine\strategies.py",
line 50, in create
u = url.make_url(name_or_url) File "C:\Users\Amber Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine\url.py",
line 211, in make_url
return _parse_rfc1738_args(name_or_url) File "C:\Users\Amber Bhanarkar\AppData\Local\Programs\Python\Python36\lib\site-packages\sqlalchemy\engine\url.py",
line 270, in _parse_rfc1738_args
"Could not parse rfc1738 URL from string '%s'" % name) sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string
'C:\Program Files\PostgreSQL\10\data'
Please someone help me resolving this issue.
The engine URL shouldn't be a path to your local Postgres installation but a sting that tells SQLAlchemy how to connect to the database. It has the following format:
postgresql://username:password#server:port/databasename
I'm also watching course video of cs50 now. You should try to just use create_engine("your_database_url").
If you still want to use create_engine(os.getenv("DATABASE_URL")) tp process, make sure you can log in using the command line like this psql $DATABASE_URL.
using psql $DATABASE_URL works for me but I'm still getting the same error on create_engine("your_database_url")
(project1) C:\Users\Gregory Goufan\Downloads\CS50s Web Programming with Python and JavaScript\project1\project1\project1>psql postgresql://openpg:openpgpwd#localhost:5432/postgres
psql (9.5.8)
WARNING: Console code page (437) differs from Windows code page (1252)
8-bit characters might not work correctly. See psql reference
page "Notes for Windows users" for details.
Type "help" for help.
postgres=#
But it is true that by putting the url direct in create_engine funciton works strange thing because it is indeed a string which is returned
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string ' 'postgresql://openpg:openpgpwd#localhost:5432/postgres''
I'm trying to run Django tests ( version 1.8 )
But I get this error
from django.test import TestCase
class JobTypesResourceTest (TestCase):
def setUp(self):
TestCase.setUp(self)
def test_basicGet(self):
return True
Traceback (most recent call last):
File "C:\Users\user\.p2\pool\plugins\org.python.pydev_4.4.0.201510052309\pysrc\runfiles.py", line 234, in <module>
main()
File "C:\Users\user\.p2\pool\plugins\org.python.pydev_4.4.0.201510052309\pysrc\runfiles.py", line 78, in main
return pydev_runfiles.main(configuration) # Note: still doesn't return a proper value.
File "C:\Users\user\.p2\pool\plugins\org.python.pydev_4.4.0.201510052309\pysrc\pydev_runfiles.py", line 835, in main
PydevTestRunner(configuration).run_tests()
File "C:\Users\user\.p2\pool\plugins\org.python.pydev_4.4.0.201510052309\pysrc\pydev_runfiles.py", line 793, in run_tests
MyDjangoTestSuiteRunner(run_tests).run_tests([])
File "C:\Users\user\.p2\pool\plugins\org.python.pydev_4.4.0.201510052309\pysrc\pydev_runfiles.py", line 813, in run_tests
raise AssertionError("Unable to run suite with DjangoTestSuiteRunner because it couldn't be imported.")
AssertionError: Unable to run suite with DjangoTestSuiteRunner because it couldn't be imported.
Am I missing a python library ?
Don't use context menu 'Run as' -> 'Python unit-test'. Use project's context menu 'Django' -> 'Run Django Tests (manage.py test)' instead.
To create launch configuration for Django Tests copy launch configuration that runs your Django project (automatically created after 'Run as' -> 'PyDev: Django')
and change program arguments from 'runserver' to 'test'.
Your TEST_RUNNER setting is set to django.test.simple.DjangoTestSuiteRunner or a subclass of it.
django.test.simple.DjangoTestSuiteRunner was deprecated in Django 1.6 and removed in Django 1.8.
Since you are using Eclipse, I think this is accurate for you.
Thank you for the help. Looks like I was running the tests wrong. I was using Eclipse-> run as pyunit option which looks like was using older code. Was working when I ran using manage.py test
I'm using Django served by uWSGI and NGINX.
Ubuntu 14.04.1 LTS 64-bit
Python 3.4
Django 1.7.4
uWSGI 1.9.17.1-debian (64bit)
NGINX 1.4.6
python-pdfkit 0.5.0
wkhtmltopdf 0.12.2.1
OpenLayers v3.0.0
When I try running pdfkit.from_url(...) to print a map to pdf the request times out.
More specifically it hangs in python's subprocess.py communicate, self._communicate:
with _PopenSelector() as selector:
if self.stdin and input:
selector.register(self.stdin, selectors.EVENT_WRITE)
if self.stdout:
selector.register(self.stdout, selectors.EVENT_READ)
if self.stderr:
selector.register(self.stderr, selectors.EVENT_READ)
while selector.get_map():
...
selector.get_map() always returns a valid result, ensuring an infinite loop.
If I run this in the Django development server (instead of uWSGI+NGINX) everything runs fine.
in my view:
wkhtmltopdfBinLocationString = '/usr/local/bin/wkhtmltopdf'
wkhtmltopdfBinLocationBytes = wkhtmltopdfBinLocationString.encode('utf-8')
#this fixes some leftover python2 assumptions about strings
config = pdfkit.configuration(wkhtmltopdf=wkhtmltopdfBinLocationBytes)
pdfkit.from_url(reportPdfUrl, reportPdfFile, configuration=config, options={
'javascript-delay': 1500
})
Several places I have seen answers along the line of "set the close-on-exec flag on the socket" solving similar issues.
Is this something I can set from my "from_url" options (wkhtmltopdf does not accept it by that name) or can I configure uWSGI to assume 'close-on-exec'? I have not been able to make either of these work, but maybe I just need help with changing my uWSGI customization file:
[uwsgi]
workers = 1
chdir = [...]
plugins = python34
wsgi-file = [...]/wsgi.py
pythonpath = [...]
I tried something like
close-on-exec = true
but that didn't seem to do anything.
NOTE: the wsgi.py file is simple:
"""
WSGI config for dst project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/
"""
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "[my_project].settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Any thoughts?
In my uwsgi config, I have these options:
[uwsgi]
chmod-socket = 777
socket = 127.0.0.1:9031
plugins = python
pythonpath = /adminserver/
callable = app
master = True
processes = 4
reload-mercy = 8
cpu-affinity = 1
max-requests = 2000
limit-as = 512
reload-on-as = 256
reload-on-rss = 192
no-orphans
vacuum
My app structure looks like this:
/adminserver
app.py
...
My app.py has these bits of code:
app = Flask(__name__)
...
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5003, debug=True)
The result is that when I try to curl my server, I get this error:
Wed Sep 11 23:28:56 2013 - added /adminserver/ to pythonpath.
Wed Sep 11 23:28:56 2013 - *** no app loaded. going in full dynamic mode ***
Wed Sep 11 23:28:56 2013 - *** uWSGI is running in multiple interpreter mode ***
What do the module and callable options do? The docs say:
module, wsgi Argument: string
Load a WSGI module as the application. The module (sans .py) must be
importable, ie. be in PYTHONPATH.
This option may be set with -w from the command line.
callable Argument: string Default: application
Set default WSGI callable name.
Module
A module in Python maps to a file on disk - when you have a directory like this:
/some-dir
module1.py
module2.py
If you start up a python interpreter while the current working directory is /some-dir you will be able to import each of the modules:
some-dir$ python
>>> import module1, module2
# Module1 and Module2 are now imported
Python searches sys.path (and a few other things, see the docs on import for more information) for a file that matches the name you are trying to import. uwsgi uses Python's import process under the covers to load the module that contains your WSGI application.
Callable
The WSGI PEPs (333 and 3333) specify that a WSGI application is a callable that takes two arguments and returns an iterable that yields bytestrings:
# simple_wsgi.py
# The simplest WSGI application
HELLO_WORLD = b"Hello world!\n"
def simple_app(environ, start_response):
"""Simplest possible application object"""
status = '200 OK'
response_headers = [('Content-type', 'text/plain')]
start_response(status, response_headers)
return [HELLO_WORLD]
uwsgi needs to know the name of a symbol inside of your module that maps to the WSGI application callable, so it can pass in the environment and the start_response callable - essentially, it needs to be able to do the following:
wsgi_app = getattr(simple_wsgi, 'simple_app')
TL;PC (Too Long; Prefer Code)
A simple parallel of what uwsgi is doing:
# Use `module` to know *what* to import
import simple_wsgi
# construct request environment from user input
# create a callable to pass for start_response
# and then ...
# use `callable` to know what to call
wsgi_app = getattr(simple_wsgi, 'simple_app')
# and then call it to respond to the user
response = wsgi_app(environ, start_response)
For anyone else having this problem, if you are sure your configuration is correct, you should check your uWSGI version.
Ubuntu 12.04 LTS provides 1.0.3. Removing that and using pip to install 2.0.4 resolved my issues.
First, check your configuration whether is correct.
my uwsgi.ini configuration:
[uwsgi]
chdir=/home/air/repo/Qiy
uid=nobody
gid=nobody
module=Qiy.wsgi:application
socket=/home/air/repo/Qiy/uwsgi.sock
master=true
workers=5
pidfile=/home/air/repo/Qiy/uwsgi.pid
vacuum=true
thunder-lock=true
enable-threads=true
harakiri=30
post-buffering=4096
daemonize=/home/air/repo/Qiy/uwsgi.log
then use uwsgi --ini uwsgi.ini to run uwsgi.
if not work, you rm -rf the venv directory, and re-initial the venv, and re-try my step.
I re-initial the venv solved my issue, seems the problem is when I pip3 install some packages of requirements.txt, and upgrade the pip, then install uwsgi package. so, I delete the venv, and re-initial my virtual environment.
I'd like to use Buildout to get django-registration with Django 1.5, and I have a custom user using MyUser(AbstractUser). I used to get it from the recipe v0.8 and it was great. Since, I switched to 1.5, remove my UserProfile and use this custom MyUser.
django-registration no longer work. I've been told I should get it from the repository, and that's what I'm trying to do. I've in the past used mr.developer to get a newer version of django-tastypie, and I tried to reproduce the same with django-registration. I have an error while calling bin/buildout tho. Let's first check the buildout config:
[buildout]
extensions = mr.developer
parts = myquivers
eggs =
django-registration
include-site-packages = false
versions = versions
sources = sources
auto-checkout =
django-registration
[sources]
django-registration = hg https://bitbucket.org/ubernostrum/django-registration
[versions]
django = 1.5
[myquivers]
recipe = djangorecipe
settings = development
eggs = ${buildout:eggs}
project = myquivers
Pretty simple config. It used to work with tastypie like I said, and I'm trying to do the same steps:
- python2.7 bootstrap.py
- bin/buildout
- bin/develop activate django-registration
- bin/develop checkout django-registration
- bin/myquivers syncdb
- bin/myquivers runservser
But it fails at the bin/buildout steps:
$ bin/buildout
Getting distribution for 'mr.developer'.
Got mr.developer 1.25.
mr.developer: Creating missing sources dir /home/damien/Documents/projects/myquivers/src.
mr.developer: Queued 'django-registration' for checkout.
mr.developer: Cloned 'django-registration' with mercurial.
Develop: '/home/damien/Documents/projects/myquivers/src/django-registration'
Traceback (most recent call last):
File "/tmp/tmpzLDggG", line 13, in <module>
exec(compile(open('/home/damien/Documents/projects/myquivers/src/django-registration/setup.py').read(), '/home/damien/Documents/projects/myquivers/src/django-registration/setup.py', 'exec'))
File "/home/damien/Documents/projects/myquivers/src/django-registration/setup.py", line 30, in <module>
version=get_version().replace(' ', '-'),
File "/home/damien/Documents/projects/myquivers/src/django-registration/registration/__init__.py", line 5, in get_version
from django.utils.version import get_version as django_get_version
ImportError: No module named django.utils.version
While:
Installing.
Processing develop directory '/home/damien/Documents/projects/myquivers/src/django-registration'.
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/home/damien/Documents/projects/myquivers/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 1923, in main
getattr(buildout, command)(args)
File "/home/damien/Documents/projects/myquivers/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 466, in install
installed_develop_eggs = self._develop()
File "/home/damien/Documents/projects/myquivers/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/buildout.py", line 707, in _develop
zc.buildout.easy_install.develop(setup, dest)
File "/home/damien/Documents/projects/myquivers/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 871, in develop
call_subprocess(args)
File "/home/damien/Documents/projects/myquivers/eggs/zc.buildout-2.1.0-py2.7.egg/zc/buildout/easy_install.py", line 129, in call_subprocess
% repr(args)[1:-1])
Exception: Failed to run command:
'/usr/bin/python2.7', '/tmp/tmpzLDggG', '-q', 'develop', '-mxN', '-d', '/home/damien/Documents/projects/myquivers/develop-eggs/tmpYM_dR9build'
Checking at the error, first Django seems not to be in the system, and that's right, when entering python2.7 and try >>> import django, it fails. But that's normal and that's why I'm using buildout, to not install system-wide Django, just locally for my project.
Any idea how to fix this? Is there a better alternative than taking this repo version? Please let me know, again, custom user/django 1.5/django-registration.
Thanks!