I'm working on a website with Python 3 and Django 1.6. I wanted to create a view, which renders a particular element from my site to a png. Therefore I use Selenium to remote control Firefox to get a specific page from my webserver and the fetching a screenshot for the response. The following code is written for this purpose:
def do_get(self, *args, **kwargs):
from pyvirtualdisplay import Display
from selenium import webdriver
import base64
display = Display(visible=0, size=(200, 100))
display.start()
browser = webdriver.Firefox()
browser.get('http://www.google.com')
response = base64.b64decode(browser.get_screenshot_as_base64())
browser.quit()
display.stop()
return HttpResponse(content=response, mimetype='image/png')
The problem is, that if I'm running the code via the python3 console directly as root everything is working fine. But my apache works with a different user without root permissions. When I try to execute the code with this user, the code is stuck in the line
browser = webdriver.Firefox()
It seems that Selenium can't connect to Firefox, when I interrupt the exection following traceback occurs:
Traceback (most recent call last):
File "test.py", line 6, in <module>
browser = webdriver.Firefox()
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/firefox/webdriver.py", line 59, in __init__
self.binary, timeout),
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/firefox/extension_connection.py", line 47, in __init__
self.binary.launch_browser(self.profile)
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 60, in launch_browser
self._start_from_profile_path(self.profile.path)
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 83, in _start_from_profile_path
env=self._firefox_env).communicate()
File "/usr/lib/python3.4/subprocess.py", line 936, in communicate
stdout = _eintr_retry_call(self.stdout.read)
File "/usr/lib/python3.4/subprocess.py", line 487, in _eintr_retry_call
return func(*args)
It seems to be a user permission issue, therefore I changed the owner of the installed firefox to my apache user. I can run firefox from the console with this user, but with the webserver the problem persists. My server is running a Ubuntu 14.04 LTS instance.
Any ideas?
Are you on the latest version of Firefox? I upgraded to version 32 this morning and the webdriver plugin is no longer supported.
Mozilla support suggests using Chrome instead of downgrading :-(
Related
I'm building a PWA with django and have this issue with django-allauth Google login. Everything works well on a Windows PC with Chrome and Edge. But when trying to login with Google on iOS devices, I get the social network login failure with code: 'unknown', exception: 'PermissionDenied()'
Any ideas what could be the reason or where to search for it?
EDIT
I tried to do some more debugging and got these errors (only on iOS devices!):
Internal Server Error: /accounts/google/login/callback/ Traceback
(most recent call last): File
"/home/kava/.virtualenvs/django3/lib/python3.8/site-packages/allauth/socialaccount/providers/oauth2/views.py",
line 141, in dispatch
login.state = SocialLogin.verify_and_unstash_state( File "/home/kava/.virtualenvs/django3/lib/python3.8/site-packages/allauth/socialaccount/models.py",
line 325, in verify_and_unstash_state
raise PermissionDenied() django.core.exceptions.PermissionDenied
I managed to solve the problem.
The error was caused by the line that follows if "socialaccount_state" not in request.session: in this function:
#classmethod
def verify_and_unstash_state(cls, request, verifier):
if "socialaccount_state" not in request.session:
raise PermissionDenied()
state, verifier2 = request.session.pop("socialaccount_state")
if verifier != verifier2:
raise PermissionDenied()
return state
So the problem was with the session data, more precisely, the 'socialaccount_state' was missing.
After Googling this issue in connection with iOS I found this thread.
I changed the SESSION COOKIE_SAMESITE and CSRF_COOKIE_SAMESITE to None as suggested there and it works!
Btw, if 'None' was between apostrophes, it didn't work on iOS 12 devices.
I have built an api with flask-restful and flask-jwt-extended and have correctly configured the validation passages for token expiration and invalidation. However, even though it has built the token expiration and invalid validation callbacks, api does not process correctly and reports the error: Signature has expired
On the server in the cloud, we have a Centos 7 x64 of 16gb ram, running the application using gunicorn in version 19.9.0. Using the miniconda to create the applications' python environments.
In tests in the production environment, the application complains of the expired token. However in a test environment, using Ubuntu 18.04.2, x64 with 16 gb ram, using the same settings with miniconda and gunicorn, the application has no problems executing it, returning the correct message when the token expires.
My jwt.py
from flask import Blueprint, Response, json, request
from flask_jwt_extended import (JWTManager, create_access_token,
create_refresh_token, get_jwt_identity,
jwt_required)
from app.models.core import User
from .schemas import UserSchema
from .utils import send_reponse, user_roles
def configure_jwt(app):
JWT = JWTManager(app)
#JWT.expired_token_loader
def my_expired_token_callback(expired_token):
return Response(
response=json.dumps({
"message": "Expired token"
}),
status=401,
mimetype='application/json'
)
#JWT.invalid_token_loader
def my_invalid_token_callback(invalid_token):
return Response(
response=json.dumps({
"message": "Invalid token"
}),
status=422,
mimetype='application/json'
)
Error log:
[2019-05-23 15:42:02 -0300] [3745] [ERROR] Exception on /api/company [POST]
Traceback (most recent call last):
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_restful/__init__.py", line 458, in wrapper
resp = resource(*args, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask/views.py", line 88, in view
return self.dispatch_request(*args, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_restful/__init__.py", line 573, in dispatch_request
resp = meth(*args, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 102, in wrapper
verify_jwt_in_request()
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 31, in verify_jwt_in_request
jwt_data = _decode_jwt_from_request(request_type='access')
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 266, in _decode_jwt_from_request
decoded_token = decode_token(encoded_token, csrf_token)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/utils.py", line 107, in decode_token
allow_expired=allow_expired
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/tokens.py", line 138, in decode_jwt
leeway=leeway, options=options)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/jwt/api_jwt.py", line 104, in decode
self._validate_claims(payload, merged_options, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/jwt/api_jwt.py", line 134, in _validate_claims
self._validate_exp(payload, now, leeway)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/jwt/api_jwt.py", line 175, in _validate_exp
raise ExpiredSignatureError('Signature has expired')
jwt.exceptions.ExpiredSignatureError: Signature has expired
I'm trying to understand why the application is able to correctly return the token expiration message in the test environment, where in the production environment it returns the error code 500 Internal Server Error. In addition to fixing this problem in our application.
Based on this link found inside the project repository, I discovered that the problem is related to the flask configuration option called PROPAGATE_EXCEPTIONS, which must be True.
The issue in the flask-jwt-extended repository that helped me find the answer.
This comment states that Flask Restful needs to ignore JWT and JWT Extended Exceptions and provides a simple snippet that solves the issue.
Copying the code from above link,
from flask_jwt_extended.exceptions import JWTExtendedException
from jwt.exceptions import PyJWTError
class FixedApi(Api):
def error_router(self, original_handler, e):
if not isinstance(e, PyJWTError) and not isinstance(e, JWTExtendedException) and self._has_fr_route():
try:
return self.handle_error(e)
except Exception:
pass # Fall through to original handler
return original_handler(e)
I started with the following basic python script for using selenium:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select, WebDriverWait
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
path_bin='/usr/bin/firefox'
path_dr='/usr/local/bin/geckodriver'
profile = webdriver.FirefoxProfile()
binary=FirefoxBinary(path_bin)
self.driver = webdriver.Firefox(executable_path=path_dr,firefox_profile=profile,firefox_binary=binary)
self.driver.implicitly_wait(30)
Then, I tried executing it twice (first time without sudo and second time with sudo) as follows:
user4#pc-4:~/Scripts$ python test.py
Traceback (most recent call last):
File "test.py", line 2049, in <module>
self.driver = webdriver.Firefox(executable_path=path_dr,firefox_profile=profile,firefox_binary=binary)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 144, in __init__
self.service = Service(executable_path, log_path=log_path)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/service.py", line 44, in __init__
log_file = open(log_path, "a+") if log_path is not None and log_path != "" else None
IOError: [Errno 13] Permission denied: 'geckodriver.log'
user4#pc-4:~/Scripts$ sudo python test.py
Traceback (most recent call last):
File "test.py", line 2049, in <module>
self.driver = webdriver.Firefox(executable_path=path_dr,firefox_profile=profile,firefox_binary=binary)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 155, in __init__
keep_alive=True)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 92, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 183, in start_session
self.capabilities = response['value']
KeyError: 'value'
So, I have to understand the following:
As observed IOError: [Errno 13] Permission denied: 'geckodriver.log' is being observed when I run the program without a sudo. Is that really required?
Even though the browser window is opened, in spite of having resolved all the dependencies (I'm using Ubuntu 16.04) like geckodriver for Firefox, I'm getting the above error KeyError: 'value'. How can I resolve this? Do I have to make any changes in my code to avoid this?
I could use some helpful advice/tips/tweaks to get my program running.
P.S.: Digging into the web to learn on selenium, points to some architecture-specific issues as mentioned in here and here and none of them seems to have been solved and that concerns me. Would chrome be a better option?
Update: The following is the output of ls -alh:
-rw-r--r-- 1 root root 238K Aug 22 17:12 geckodriver.log
-rwxrwxrwx 1 user4 user 82K Aug 22 17:08 test.py
As you were trying to pass a couple of arguments when invoking webdriver.Firefox(), we need to consider that executable_path and firefox_binary both these arguments takes string type of arguments. So you need to pass the absolute path of geckodriver binary to executable_path and the absolute path of firefox binary to firefox_binary. Further you don't need to use binary=FirefoxBinary(path_bin). The following code block works fine on my Windows 8 machine:
Code is based on Python 3.6.1
from selenium import webdriver
path_bin=r'C:\Program Files\Mozilla Firefox\firefox.exe'
path_dr=r'C:\Utility\BrowserDrivers\geckodriver.exe'
profile = webdriver.FirefoxProfile()
driver = webdriver.Firefox(executable_path=path_dr,firefox_profile=profile,firefox_binary=path_bin)
driver.get('https://www.google.co.in')
As you are on ubuntu the following code block must work for you:
from selenium import webdriver
path_bin='/usr/bin/firefox'
path_dr='/usr/local/bin/geckodriver'
profile = webdriver.FirefoxProfile()
driver = webdriver.Firefox(executable_path=path_dr,firefox_profile=profile,firefox_binary=path_bin)
driver.get('https://www.google.co.in')
I had this same issue with
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 183, in start_session
self.capabilities = response['value']
KeyError: 'value'
So I have updated selenium and geckodriver to the latest version. This was the solution in my case.
I am using Python 2.7.5 with web.py (v0.38) installed on a Linux machine. Below is my code in the most basic form (webhooks.py)
#!/usr/bin/python
import web
urls = ('/.*','WebHooks')
app = web.application(urls, globals())
class WebHooks:
def POST(self):
raw_payload = web.data()
json_encode = json.loads(raw_payload)
if __name__ == '__main__':
app.run()
I execute python webhooks.py 9999
It opens up a local port http://0.0.0.0:9999/
My issue: I have read the documentation located here and I am stumped. Would somebody be able to help me open an HTTPS URL? https://0.0.0.0:9999/
What I have tried
Add the following into my code for testing:
response = app.request("/.*", https=True)
I would get an error: AttributeError: 'module' object has no attribute 'request'
I solved that issue with pip install urllib.py and then adding import urllib to the top of my code but I ended up with a bunch of errors:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/web/application.py", line 239, in process
return self.handle()
File "/usr/lib/python2.7/site-packages/web/application.py", line 230, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/lib/python2.7/site-packages/web/application.py", line 461, in _delegate
cls = fvars[f]
KeyError: u'WebHooks'
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/web/application.py", line 239, in process
return self.handle()
File "/usr/lib/python2.7/site-packages/web/application.py", line 229, in handle
fn, args = self._match(self.mapping, web.ctx.path)
AttributeError: 'ThreadedDict' object has no attribute 'path'
You're headed down the wrong path, but not to worry. The response = app.request("/.*", https=True) bit you're trying has to do with your application making an https request, rather then handling an https request.
See http://webpy.org/cookbook/ssl
Internally, web.py uses a CherryPyWSGIServer. To handle https, you need to provide the server with an ssl_certificate and ssl_key. Very simply, add a few lines before you invoke app.run():
if __name__ == '__main__':
from web.wsgiserver import CherryPyWSGIServer
ssl_cert = '/path-to-cert.crt'
ssl_key = '/path-to-cert.key'
CherryPyWSGIServer.ssl_certificate = ssl_cert
CherryPyWSGIServer.ssl_private_key = ssl_key
app.run()
Of course, in a full solution, you'll probably want apache or nginx to handle the https portion, but the above is perfect for small applications and testing.
I'm new to django (using python 2.7) and I was just trying to use heroku for the first time but I always get the following error:
remote: -----> Preparing static assets
remote: Collectstatic configuration error. To debug, run:
remote: $ heroku run python manage.py collectstatic --noinput
When I run that command, an import error regarding the django registration redux library shows up. I've had this problem before in Django and I fixed it by placing RequestSite under 'requests' and Site under 'models'. That solved the problem but the error still shows up in Heroku.
(venv) C:\Users\Carolina\Desktop\Coding\venv\project1>heroku run python manage.p
y collectstatic --noinput
Running python manage.py collectstatic --noinput on acla-acla... up, run.3645
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/_
_init__.py", line 350, in execute_from_command_line
utility.execute()
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/_
_init__.py", line 324, in execute
django.setup()
File "/app/.heroku/python/lib/python2.7/site-packages/django/__init__.py", lin
e 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/app/.heroku/python/lib/python2.7/site-packages/django/apps/registry.py"
, line 115, in populate
app_config.ready()
File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/admin/app
s.py", line 22, in ready
self.module.autodiscover()
File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/admin/__i
nit__.py", line 26, in autodiscover
autodiscover_modules('admin', register_to=site)
File "/app/.heroku/python/lib/python2.7/site-packages/django/utils/module_load
ing.py", line 50, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File "/app/.heroku/python/lib/python2.7/importlib/__init__.py", line 37, in im
port_module
__import__(name)
File "/app/.heroku/python/lib/python2.7/site-packages/registration/admin.py",
line 2, in <module>
from django.contrib.sites.models import RequestSite
ImportError: cannot import name RequestSite
The thing is - that line doesn't exist. I went to venv/lib/site-packages/registration/admin.py and line 2 is this one:
from django.contrib import admin
from django.contrib.sites.requests import RequestSite # HERE
from django.contrib.sites.models import Site
from django.utils.translation import ugettext_lazy as _
from registration.models import RegistrationProfile
from registration.users import UsernameField
class RegistrationAdmin(admin.ModelAdmin):
actions = ['activate_users', 'resend_activation_email']
list_display = ('user', 'activation_key_expired')
raw_id_fields = ['user']
search_fields = ('user__{0}'.format(UsernameField()), 'user__first_name', 'user__last_name')
def activate_users(self, request, queryset):
"""
Activates the selected users, if they are not already
activated.
"""
for profile in queryset:
RegistrationProfile.objects.activate_user(profile.activation_key)
activate_users.short_description = _("Activate users")
def resend_activation_email(self, request, queryset):
"""
Re-sends activation emails for the selected users.
Note that this will *only* send activation emails for users
who are eligible to activate; emails will not be sent to users
whose activation keys have expired or who have already
activated.
"""
if Site._meta.installed:
site = Site.objects.get_current()
else:
site = RequestSite(request)
for profile in queryset:
if not profile.activation_key_expired():
profile.send_activation_email(site)
resend_activation_email.short_description = _("Re-send activation emails")
admin.site.register(RegistrationProfile, RegistrationAdmin)
This is what I get with pip freeze, just in case:
Django==1.9
django-crispy-forms==1.5.2
django-registration==2.0.3
django-registration-redux==1.2
django-tinymce==2.2.0
Pillow==3.0.0
requests==2.9.0
South==1.0.2
stripe==1.27.1
wheel==0.24.0
Anyone knows why this is happening? Thanks in advance!
EDIT ----
Ok, so the problem was the one mentioned by Daniel Roseman. The library is broken in pypi and I had to tell heroku to install it from github (where the package is fixed).
So, I went to my requirements.txt file and replaced this line:
django-registration-redux==1.2
with this one:
-e git://github.com/macropin/django-registration.git#egg=django-registration==1.2
(I also removed 'django-registration==2.0.3' because it is an old version of django-registration-redux and was creating problems).
Hope this helps people with the same issue!
It sounds like you edited the code for your locally installed copy of django-registration-redux. But that won't have any effect on Heroku, since the library will be installed directly from PyPI, according to the version in your requirements.txt.
If the library is really broken, you will need to fork it and point your requirements.txt to your fixed version. However, looking at the code on GitHub, it doesn't actually seem to be broken; you just need to update the version you are pointing to.