Open Edx Third Party Login Integration with Keycloak - django

I am using edx-ironwood.2-6 in ubuntu 18.08. I am also running keycloak 9.0.0. To enable third-party login using Keycloak I am using the python-social-auth library suggested in edx documentation. Since by default keycloak.py was not available in the edx-ironwood, I copied this keycloak.js file at the location
edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/social_core/backends
Followed all steps written in comments to setup keycloak and added following information in keycloak.py
SECRET = 'client secret'
PUBLIC_KEY = 'publick key from keycloak'
AUTHORIZATION_URL = 'http://localhost:8080/auth/realms/myrealm/protocol/openid-connect/auth'
ACCESS_TOKEN_URL = 'http://localhost:8080/auth/realms/myrealm/protocol/openid-connect/token'
USER_DETAILS_URL = 'http://localhost:8080/auth/realms/myrealm/protocol/openid-connect/userinfo'
Added following line in the lms.env.json
"THIRD_PARTY_AUTH_BACKENDS":[
"social_core.backends.keycloak.KeycloakOAuth2",
"social_core.backends.google.GoogleOAuth2",
"social_core.backends.linkedin.LinkedinOAuth2"
]
In the Django Admin App, Added a Provider
Name: Keycloak
slug: keycloak
site: localhost:81
backend: keycloak
client Id: 'mooc'
Client Secret: 'secret key'
Also Added client secret in lms.auth.json
"SOCIAL_AUTH_OAUTH_SECRETS": {
"keycloak":"14f89ef1-02ff-48ad-825f-8160e515ec8e"
}
In Keycloak client settings, added access type 'confidential', and redirect uri 'http://localhost:81/auth/complete/keycloak/'
After server restart, In edx login page, login button for keycloak appearing but when I am clicking on it in the browser a message is coming There has been a 500 error on the Open Edx Server
In the apache2 log file following error is coming
[Sat Apr 18 17:09:21.212377 2020] [:error] [pid 8143] Traceback (most recent call last):
[Sat Apr 18 17:09:21.212419 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
[Sat Apr 18 17:09:21.212442 2020] [:error] [pid 8143] response = get_response(request)
[Sat Apr 18 17:09:21.212462 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
[Sat Apr 18 17:09:21.212485 2020] [:error] [pid 8143] response = self._get_response(request)
[Sat Apr 18 17:09:21.212506 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
[Sat Apr 18 17:09:21.212526 2020] [:error] [pid 8143] response = self.process_exception_by_middleware(e, request)
[Sat Apr 18 17:09:21.212548 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
[Sat Apr 18 17:09:21.212569 2020] [:error] [pid 8143] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Sat Apr 18 17:09:21.212589 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/decorators.py", line 185, in inner
[Sat Apr 18 17:09:21.212610 2020] [:error] [pid 8143] return func(*args, **kwargs)
[Sat Apr 18 17:09:21.212630 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/views/decorators/cache.py", line 57, in _wrapped_view_func
[Sat Apr 18 17:09:21.212651 2020] [:error] [pid 8143] response = view_func(request, *args, **kwargs)
[Sat Apr 18 17:09:21.212671 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/social_django/utils.py", line 49, in wrapper
[Sat Apr 18 17:09:21.212697 2020] [:error] [pid 8143] return func(request, backend, *args, **kwargs)
[Sat Apr 18 17:09:21.212720 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/social_django/views.py", line 23, in auth
[Sat Apr 18 17:09:21.212742 2020] [:error] [pid 8143] return do_auth(request.backend, redirect_name=REDIRECT_FIELD_NAME)
[Sat Apr 18 17:09:21.212762 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/social_core/actions.py", line 29, in do_auth
[Sat Apr 18 17:09:21.212783 2020] [:error] [pid 8143] return backend.start()
[Sat Apr 18 17:09:21.212803 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/social_core/backends/base.py", line 35, in start
[Sat Apr 18 17:09:21.212823 2020] [:error] [pid 8143] return self.strategy.redirect(self.auth_url())
[Sat Apr 18 17:09:21.212844 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/social_django/strategy.py", line 88, in redirect
[Sat Apr 18 17:09:21.212864 2020] [:error] [pid 8143] return redirect(url)
[Sat Apr 18 17:09:21.212884 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/shortcuts.py", line 56, in redirect
[Sat Apr 18 17:09:21.212904 2020] [:error] [pid 8143] return redirect_class(resolve_url(to, *args, **kwargs))
[Sat Apr 18 17:09:21.212925 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/shortcuts.py", line 147, in resolve_url
[Sat Apr 18 17:09:21.212945 2020] [:error] [pid 8143] return reverse(to, args=args, kwargs=kwargs)
[Sat Apr 18 17:09:21.212965 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/src/django-wiki/wiki/models/__init__.py", line 90, in reverse
[Sat Apr 18 17:09:21.212986 2020] [:error] [pid 8143] url = original_django_reverse(*args, **kwargs)
[Sat Apr 18 17:09:21.213006 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/urls/base.py", line 91, in reverse
[Sat Apr 18 17:09:21.213026 2020] [:error] [pid 8143] return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs)))
[Sat Apr 18 17:09:21.213047 2020] [:error] [pid 8143] File "/opt/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/urls/resolvers.py", line 497, in _reverse_with_prefix
[Sat Apr 18 17:09:21.213067 2020] [:error] [pid 8143] raise NoReverseMatch(msg)
[Sat Apr 18 17:09:21.213089 2020] [:error] [pid 8143] NoReverseMatch: Reverse for 'None?scope=profile+email&state=yst8UI8KZihrluHg9R0oUFexFIx0QYGM&redirect_uri=http%3A%2F%2Flocalhost%3A81%2Fauth%2Fcomplete%2Fkeycloak%2F%3Fredirect_state%3Dyst8UI8KZihrluHg9R0oUFexFIx0QYGM&response_type=code&client_id=mooc' not found. 'None?scope=profile+email&state=yst8UI8KZihrluHg9R0oUFexFIx0QYGM&redirect_uri=http%3A%2F%2Flocalhost%3A81%2Fauth%2Fcomplete%2Fkeycloak%2F%3Fredirect_state%3Dyst8UI8KZihrluHg9R0oUFexFIx0QYGM&response_type=code&client_id=mooc' is not a valid view function or pattern name.
I tried same third party integration with same python-social-auth for LinkedIn and that is working.
Since I am just a beginner in the Django, Can anyone help me identify the issue from the above details.

After finding and reading many official resources, I haven't found any solution which worked. So I decided to try my own custom file for keycloak.
Here is the link of the file
https://github.com/ranjeet692/python-social-auth-keycloak,
For open edx users, place this file into this directory
/edx-ironwood.2-6/apps/edx/venvs/edxapp/lib/python2.7/site-packages/social_core/backends
Follow the rest instruction as mentioned here
Social Auth Documentation
and restart the server. You should be able to login with keycloak.

There is now an official backend for keycloak in python-social-auth:
https://github.com/python-social-auth/social-core/blob/master/social_core/backends/keycloak.py
Just add
"THIRD_PARTY_AUTH_BACKENDS": [
"social_core.backends.keycloak.KeycloakOAuth2"
],
to your lms.env.json
together with the keycloak config:
"SOCIAL_AUTH_KEYCLOAK_KEY": "openedx",
"SOCIAL_AUTH_KEYCLOAK_SECRET": "...",
"SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY": "...",
"SOCIAL_AUTH_KEYCLOAK_AUTHORIZATION_URL": "https://YOUR_KEYCLOAK/realms/YOUR_REALM/protocol/openid-connect/auth",
"SOCIAL_AUTH_KEYCLOAK_ACCESS_TOKEN_URL": "https://YOUR_KEYCLOAK/realms/YOUR_REALM/protocol/openid-connect/token",
Restart your installation, then go to:
https://YOUR.LMS/admin/third_party_auth/oauth2providerconfig/
and configure according to https://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/latest/configuration/tpa/tpa_integrate_open/tpa_oauth.html#additional-oauth2-providers-advanced

Related

[SOLVED]Django smtp gmail not working in production

I'm using django to send email through gmail smtp. It only works, however, before deployment. In production or deployment whatever you call, when I try to send email it keep being loaded forever and only says 'Server Error 500'.
Below is part of my settings.py
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_PORT = 587
EMAIL_USE_TLS = True
EMAIL_HOST_USER = config['EMAIL_USER']
EMAIL_HOST_PASSWORD = config['EMAIL_PASS']
Below is .../django_project/users/views.py
from django.shortcuts import render, redirect
from django.contrib import messages
from django.contrib.auth.decorators import login_required
from .forms import UserRegisterForm, UserUpdateForm, ProfileUpdateForm
def register(request):
if request.method == 'POST':
form = UserRegisterForm(request.POST)
if form.is_valid():
form.save()
username = form.cleaned_data.get('username')
messages.success(request, f'Your account has been created! You are now able to log in.')
return redirect('login')
else:
form = UserRegisterForm()
return render(request, 'users/register.html', {'form': form})
#login_required()
def profile(request):
if request.method == 'POST':
u_form = UserUpdateForm(request.POST, instance=request.user)
p_form = ProfileUpdateForm(request.POST,
request.FILES,
instance=request.user.profile)
if u_form.is_valid() and p_form.is_valid():
u_form.save()
p_form.save()
messages.success(request, f'Your account has been updated!')
return redirect('profile')
else:
u_form = UserUpdateForm(instance=request.user)
p_form = ProfileUpdateForm(instance=request.user.profile)
context = {
'u_form': u_form,
'p_form': p_form
}
return render(request, 'users/profile.html', context)
What's confusing is my tutorial videos didn't put any from django.core.mail import send_mail or something. So I'm not sure whether the view.py above is the one that should be shown.
Perhaps this issue is with Gmail itself. Please let me know how can I solve this. Thanks.
EDITED
I briefly set DEBUG = True and saw error messages from error.log.
[Tue Feb 18 12:10:04.400741 2020] [core:notice] [pid 11826:tid 139852027644992] AH00094: Command line: '/usr/sbin/apache2'
[Tue Feb 18 12:15:05.589207 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] Internal Server Error: /password-reset/
[Tue Feb 18 12:15:05.589265 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] Traceback (most recent call last):
[Tue Feb 18 12:15:05.589270 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
[Tue Feb 18 12:15:05.589274 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] response = get_response(request)
[Tue Feb 18 12:15:05.589278 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
[Tue Feb 18 12:15:05.589282 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] response = self.process_exception_by_middleware(e, request)
[Tue Feb 18 12:15:05.589286 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
[Tue Feb 18 12:15:05.589290 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Tue Feb 18 12:15:05.589294 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view
[Tue Feb 18 12:15:05.589298 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] return self.dispatch(request, *args, **kwargs)
[Tue Feb 18 12:15:05.589301 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/utils/decorators.py", line 43, in _wrapper
[Tue Feb 18 12:15:05.589305 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] return bound_method(*args, **kwargs)
[Tue Feb 18 12:15:05.589309 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
[Tue Feb 18 12:15:05.589313 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] response = view_func(request, *args, **kwargs)
[Tue Feb 18 12:15:05.589316 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/contrib/auth/views.py", line 222, in dispatch
[Tue Feb 18 12:15:05.589320 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] return super().dispatch(*args, **kwargs)
[Tue Feb 18 12:15:05.589324 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/views/generic/base.py", line 97, in dispatch
[Tue Feb 18 12:15:05.589327 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] return handler(request, *args, **kwargs)
[Tue Feb 18 12:15:05.589331 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/views/generic/edit.py", line 142, in post
[Tue Feb 18 12:15:05.589335 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] return self.form_valid(form)
[Tue Feb 18 12:15:05.590890 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/contrib/auth/views.py", line 235, in form_valid
[Tue Feb 18 12:15:05.590901 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] form.save(**opts)
[Tue Feb 18 12:15:05.590905 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/contrib/auth/forms.py", line 324, in save
[Tue Feb 18 12:15:05.590909 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] user_email, html_email_template_name=html_email_template_name,
[Tue Feb 18 12:15:05.590913 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/contrib/auth/forms.py", line 272, in send_mail
[Tue Feb 18 12:15:05.590916 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] email_message.send()
[Tue Feb 18 12:15:05.590920 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/core/mail/message.py", line 276, in send
[Tue Feb 18 12:15:05.590923 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] return self.get_connection(fail_silently).send_messages([self])
[Tue Feb 18 12:15:05.590927 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/core/mail/backends/smtp.py", line 102, in send_messages
[Tue Feb 18 12:15:05.590930 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] new_conn_created = self.open()
[Tue Feb 18 12:15:05.590934 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/home/djtu/django_project/venv/lib/python3.7/site-packages/django/core/mail/backends/smtp.py", line 62, in open
[Tue Feb 18 12:15:05.590942 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] self.connection = self.connection_class(self.host, self.port, **connection_params)
[Tue Feb 18 12:15:05.590946 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/usr/lib/python3.7/smtplib.py", line 251, in __init__
[Tue Feb 18 12:15:05.590949 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] (code, msg) = self.connect(host, port)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
[Tue Feb 18 12:15:05.590953 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/usr/lib/python3.7/smtplib.py", line 336, in connect
[Tue Feb 18 12:15:05.590956 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] self.sock = self._get_socket(host, port, self.timeout)
[Tue Feb 18 12:15:05.590960 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/usr/lib/python3.7/smtplib.py", line 307, in _get_socket
[Tue Feb 18 12:15:05.590963 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] self.source_address)
[Tue Feb 18 12:15:05.590966 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/usr/lib/python3.7/socket.py", line 727, in create_connection
[Tue Feb 18 12:15:05.590970 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] raise err
[Tue Feb 18 12:15:05.590973 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] File "/usr/lib/python3.7/socket.py", line 716, in create_connection
[Tue Feb 18 12:15:05.590976 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] sock.connect(sa)
[Tue Feb 18 12:15:05.590980 2020] [wsgi:error] [pid 11827:tid 139851910838016] [remote 121.131.97.11:49984] TimeoutError: [Errno 110] Connection timed out
I hope this log will give some clarification on what went wrong. Can somebody point me out to right direction?
EDITED #2
Thanks to Sachin. I tried as suggested, but my server machine failed to connect. Instead it show this messages.
(venv) myusername#hostname:~$ telnet smtp.gmail.com 587
Trying 2404:6800:4008:c00::6d...
Trying 108.177.97.109...
telnet: Unable to connect to remote host: Connection timed out
Still the issue remains unfortunately.
After some time I realized what went wrong. I use 2-step verification for my gmail account, which means my password won't work in external program like django, unless I replace it with app password.
For more info, please visit official page for app password:
https://support.google.com/accounts/answer/185833?hl=en
And if you're using linode for web hosting company, they might block email port for some reasons, so it could be external problem too.

Encountering error getting Spacy to load models

I've deployed a Django web app in Elastic Beanstalk and I'm trying to implement spacy, yet encountering errors.
I've listed the download link for the en_core_web_sm in my requirements.txt as https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz
I'm importing it in my code my calling import en_core_web_sm and later trying to load it with:
nlp = en_core_web_sm.load()
However in my logs from eb I'm getting errors indicating the model isn't loading properly. Am I missing something here? My instance is configured for 2gb of RAM, that might not be enough, but it shouldn't effect the loading of the model, right? Why would it load the tokenizer but not the model?
[Fri Sep 01 04:33:49.276836 2017] [:error] [pid 23201] \x1b[93m Warning: no model found for 'en'\x1b[0m
[Fri Sep 01 04:33:49.276841 2017] [:error] [pid 23201]
[Fri Sep 01 04:33:49.276849 2017] [:error] [pid 23201]
[Fri Sep 01 04:33:49.276851 2017] [:error] [pid 23201] Only loading the 'en' tokenizer.
[Fri Sep 01 04:33:49.276855 2017] [:error] [pid 23201]
[Fri Sep 01 04:33:49.333165 2017] [:error] [pid 23201] Internal Server Error: /webhookmb/
[Fri Sep 01 04:33:49.333187 2017] [:error] [pid 23201] Traceback (most recent call last):
[Fri Sep 01 04:33:49.333189 2017] [:error] [pid 23201] File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner
[Fri Sep 01 04:33:49.333192 2017] [:error] [pid 23201] response = get_response(request)
[Fri Sep 01 04:33:49.333194 2017] [:error] [pid 23201] File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response
[Fri Sep 01 04:33:49.333196 2017] [:error] [pid 23201] response = self.process_exception_by_middleware(e, request)
[Fri Sep 01 04:33:49.333198 2017] [:error] [pid 23201] File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response
[Fri Sep 01 04:33:49.333201 2017] [:error] [pid 23201] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Fri Sep 01 04:33:49.333203 2017] [:error] [pid 23201] File "/opt/python/run/venv/lib/python3.4/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
[Fri Sep 01 04:33:49.333205 2017] [:error] [pid 23201] return view_func(*args, **kwargs)
[Fri Sep 01 04:33:49.333216 2017] [:error] [pid 23201] File "/opt/python/run/venv/lib/python3.4/site-packages/django/views/decorators/http.py", line 40, in inner
[Fri Sep 01 04:33:49.333218 2017] [:error] [pid 23201] return func(request, *args, **kwargs)
[Fri Sep 01 04:33:49.333220 2017] [:error] [pid 23201] File "/opt/python/current/app/webhookmb/views.py", line 81, in webhookmb
[Fri Sep 01 04:33:49.333222 2017] [:error] [pid 23201] interpreter = Interpreter.load(metadata, RasaNLUConfig("rasa_nlu/config_spacy.json"))
[Fri Sep 01 04:33:49.333224 2017] [:error] [pid 23201] File "/opt/python/current/app/rasa_nlu/model.py", line 213, in load
[Fri Sep 01 04:33:49.333226 2017] [:error] [pid 23201] component_name, model_metadata.model_dir, model_metadata, **context)
[Fri Sep 01 04:33:49.333228 2017] [:error] [pid 23201] File "/opt/python/current/app/rasa_nlu/components.py", line 310, in load_component
[Fri Sep 01 04:33:49.333229 2017] [:error] [pid 23201] model_metadata, cached_component, **context)
[Fri Sep 01 04:33:49.333231 2017] [:error] [pid 23201] File "/opt/python/current/app/rasa_nlu/registry.py", line 128, in load_component_by_name
[Fri Sep 01 04:33:49.333233 2017] [:error] [pid 23201] return component_clz.load(model_dir, metadata, cached_component, **kwargs)
[Fri Sep 01 04:33:49.333235 2017] [:error] [pid 23201] File "/opt/python/current/app/rasa_nlu/utils/spacy_utils.py", line 99, in load
[Fri Sep 01 04:33:49.333237 2017] [:error] [pid 23201] cls.ensure_proper_language_model(nlp)
[Fri Sep 01 04:33:49.333239 2017] [:error] [pid 23201] File "/opt/python/current/app/rasa_nlu/utils/spacy_utils.py", line 113, in ensure_proper_language_model
[Fri Sep 01 04:33:49.333240 2017] [:error] [pid 23201] "Make sure you have downloaded the correct model (https://spacy.io/docs/usage/).")
[Fri Sep 01 04:33:49.333244 2017] [:error] [pid 23201] Exception: Failed to load spacy language model for lang 'en'. Make sure you have downloaded the correct model (https://spacy.io/docs/usage/).

AWS S3 - failed to identify the profile information

I am using boto3 for communicating with AWS S3. I first tried with my local system. It successfully got connected and was working fine. I moved the code to the server(flask-apache2). Now the code is not working. It shows ProfileNotFound: The config profile (dev) could not be found exception even though I set the the parameters over ~/.aws/credentials and ~/.aws/config
~/.aws/credentials file:
[dev]
aws_access_key_id = *****************
aws_secret_access_key = *********************
~/.aws/config file:
[default]
region = us-west-2
output = json
[profile dev]
region = us-west-2
output = json
This is how I am accessing the profile:
session = boto3.Session(profile_name='dev')
s3 = session.resource('s3')
Error Log:
session = boto3.Session(profile_name='dev')
[Wed Jun 21 05:50:56.912720 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/boto3/session.py", line 80, in __init__
[Wed Jun 21 05:50:56.912836 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] self._setup_loader()
[Wed Jun 21 05:50:56.912866 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/boto3/session.py", line 120, in _setup_loader
[Wed Jun 21 05:50:56.912893 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] self._loader = self._session.get_component('data_loader')
[Wed Jun 21 05:50:56.912910 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/botocore/session.py", line 701, in get_component
[Wed Jun 21 05:50:56.913104 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] return self._components.get_component(name)
[Wed Jun 21 05:50:56.913121 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/botocore/session.py", line 897, in get_component
[Wed Jun 21 05:50:56.913134 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] self._components[name] = factory()
[Wed Jun 21 05:50:56.913145 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/botocore/session.py", line 181, in <lambda>
[Wed Jun 21 05:50:56.913154 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] lambda: create_loader(self.get_config_variable('data_path')))
[Wed Jun 21 05:50:56.913163 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/botocore/session.py", line 265, in get_config_variable
[Wed Jun 21 05:50:56.913172 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] elif self._found_in_config_file(methods, var_config):
[Wed Jun 21 05:50:56.913181 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/botocore/session.py", line 286, in _found_in_config_file
[Wed Jun 21 05:50:56.913190 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] return var_config[0] in self.get_scoped_config()
[Wed Jun 21 05:50:56.913198 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] File "/usr/local/lib/python2.7/dist-packages/botocore/session.py", line 358, in get_scoped_config
[Wed Jun 21 05:50:56.913206 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] raise ProfileNotFound(profile=profile_name)
[Wed Jun 21 05:50:56.913225 2017] [wsgi:error] [pid 21586] [remote 103.227.98.117:37564] ProfileNotFound: The config profile (dev) could not be found
Finally, I found the solution!
I added the AWS Credentials & Config file paths to apache2 envvars file. And the problem got solved.
Steps:
$sudo nano /etc/apache2/envvars
# Added the following lines
export AWS_CONFIG_FILE=/home/ubuntu/.aws/config
export AWS_SHARED_CREDENTIALS_FILE=/home/ubuntu/.aws/credentials
sudo service apache2 reload | sudo service apache2 restart
Done! It started working fine!!
I initially tried adding like AWS_CONFIG_FILE=~/.aws/credentials. This didn't work. Have to give the complete path.

Internal Server Error Django/apache

My Django app was working fine, but now I'm getting a server error, the last thing I did was a reboot to my Centos7 Server and now I'm getting this error:
and this are my logs I'm using apache:
[Thu Mar 30 14:04:10.571843 2017] [:error] [pid 11335] [remote 10.221.50.100:24] mod_wsgi (pid=11335): Target WSGI script '/opt/btsystem/BTSystem/wsgi.py' cannot be loaded as Python module.
[Thu Mar 30 14:04:10.571991 2017] [:error] [pid 11335] [remote 10.221.50.100:24] mod_wsgi (pid=11335): Exception occurred processing WSGI script '/opt/btsystem/BTSystem/wsgi.py'.
[Thu Mar 30 14:04:10.572263 2017] [:error] [pid 11335] [remote 10.221.50.100:24] Traceback (most recent call last):
[Thu Mar 30 14:04:10.572544 2017] [:error] [pid 11335] [remote 10.221.50.100:24] File "/opt/btsystem/BTSystem/wsgi.py", line 16, in <module>
[Thu Mar 30 14:04:10.572795 2017] [:error] [pid 11335] [remote 10.221.50.100:24] application = get_wsgi_application()
[Thu Mar 30 14:04:10.572919 2017] [:error] [pid 11335] [remote 10.221.50.100:24] File "/usr/lib64/python2.7/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
[Thu Mar 30 14:04:10.573054 2017] [:error] [pid 11335] [remote 10.221.50.100:24] django.setup(set_prefix=False)
[Thu Mar 30 14:04:10.573107 2017] [:error] [pid 11335] [remote 10.221.50.100:24] File "/usr/lib64/python2.7/site-packages/django/__init__.py", line 22, in setup
[Thu Mar 30 14:04:10.573170 2017] [:error] [pid 11335] [remote 10.221.50.100:24] configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
[Thu Mar 30 14:04:10.573261 2017] [:error] [pid 11335] [remote 10.221.50.100:24] File "/usr/lib64/python2.7/site-packages/django/utils/log.py", line 75, in configure_logging
[Thu Mar 30 14:04:10.573312 2017] [:error] [pid 11335] [remote 10.221.50.100:24] logging_config_func(logging_settings)
[Thu Mar 30 14:04:10.573369 2017] [:error] [pid 11335] [remote 10.221.50.100:24] File "/usr/lib64/python2.7/logging/config.py", line 803, in dictConfig
[Thu Mar 30 14:04:10.573409 2017] [:error] [pid 11335] [remote 10.221.50.100:24] dictConfigClass(config).configure()
[Thu Mar 30 14:04:10.573455 2017] [:error] [pid 11335] [remote 10.221.50.100:24] File "/usr/lib64/python2.7/logging/config.py", line 585, in configure
[Thu Mar 30 14:04:10.573501 2017] [:error] [pid 11335] [remote 10.221.50.100:24] '%r: %s' % (name, e))
[Thu Mar 30 14:04:10.573567 2017] [:error] [pid 11335] [remote 10.221.50.100:24] ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/opt/btsystem-logs/btsystem-debug.log'
Any idea?
The last line shows that Django doesn't have permission to write to the configured log file.
[Thu Mar 30 14:04:10.573567 2017] [:error] [pid 11335] [remote 10.221.50.100:24] ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/opt/btsystem-logs/btsystem-debug.log'
Make sure the file /opt/btsystem-logs/btsystem-debug.log is writeable by the user running the Django server.
The main problem was that I needed to do the following to enable the virtual host from Apache:
firewall-cmd --get-active-zones
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --reload
Needed to open the ports.

504 Gateway Timeout sending mail

I have a problem sending mail from a contact form with Django. I have set smtp config in settings.py and I can send it from local. But in production server, I get a 504 Gateway Timeout Error. This is the Error Log content.
[Sun Jul 17 18:17:46.142782 2016] [wsgi:error] [pid 16049] [client XXXXX:37298] Timeout when reading response headers from daemon process 'domain.gal': /home/wsgi/web/domain.gal/private/django/domain.gal/domain/wsgi.py, referer: http://domain.gal/contacto/
[Sun Jul 17 18:17:49.120971 2016] [wsgi:error] [pid 16046] Internal Server Error: /contacto/
[Sun Jul 17 18:17:49.121004 2016] [wsgi:error] [pid 16046] Traceback (most recent call last):
[Sun Jul 17 18:17:49.121008 2016] [wsgi:error] [pid 16046] File "/home/wsgi/web/domain.gal/private/django/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 149, in get_response
[Sun Jul 17 18:17:49.121012 2016] [wsgi:error] [pid 16046] response = self.process_exception_by_middleware(e, request)
[Sun Jul 17 18:17:49.121015 2016] [wsgi:error] [pid 16046] File "/home/wsgi/web/domain.gal/private/django/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 147, in get_response
[Sun Jul 17 18:17:49.121018 2016] [wsgi:error] [pid 16046] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Sun Jul 17 18:17:49.121021 2016] [wsgi:error] [pid 16046] File "/home/wsgi/web/domain.gal/private/django/domain.gal/home/views.py", line 43, in contacto
[Sun Jul 17 18:17:49.121023 2016] [wsgi:error] [pid 16046] send_mail(subject, message, from_email, ['info#domain.es'])
[Sun Jul 17 18:17:49.121026 2016] [wsgi:error] [pid 16046] File "/home/wsgi/web/domain.gal/private/django/env/lib/python3.4/site-packages/django/core/mail/__init__.py", line 61, in send_mail
[Sun Jul 17 18:17:49.121029 2016] [wsgi:error] [pid 16046] return mail.send()
[Sun Jul 17 18:17:49.121032 2016] [wsgi:error] [pid 16046] File "/home/wsgi/web/domain.gal/private/django/env/lib/python3.4/site-packages/django/core/mail/message.py", line 292, in send
[Sun Jul 17 18:17:49.121034 2016] [wsgi:error] [pid 16046] return self.get_connection(fail_silently).send_messages([self])
[Sun Jul 17 18:17:49.121037 2016] [wsgi:error] [pid 16046] File "/home/wsgi/web/domain.gal/private/django/env/lib/python3.4/site-packages/django/core/mail/backends/smtp.py", line 100, in send_messages
[Sun Jul 17 18:17:49.121040 2016] [wsgi:error] [pid 16046] new_conn_created = self.open()
[Sun Jul 17 18:17:49.121043 2016] [wsgi:error] [pid 16046] File "/home/wsgi/web/domain.gal/private/django/env/lib/python3.4/site-packages/django/core/mail/backends/smtp.py", line 58, in open
[Sun Jul 17 18:17:49.121060 2016] [wsgi:error] [pid 16046] self.connection = connection_class(self.host, self.port, **connection_params)
[Sun Jul 17 18:17:49.121063 2016] [wsgi:error] [pid 16046] File "/usr/lib64/python3.4/smtplib.py", line 242, in __init__
[Sun Jul 17 18:17:49.121066 2016] [wsgi:error] [pid 16046] (code, msg) = self.connect(host, port)
[Sun Jul 17 18:17:49.121069 2016] [wsgi:error] [pid 16046] File "/usr/lib64/python3.4/smtplib.py", line 321, in connect
[Sun Jul 17 18:17:49.121071 2016] [wsgi:error] [pid 16046] self.sock = self._get_socket(host, port, self.timeout)
[Sun Jul 17 18:17:49.121074 2016] [wsgi:error] [pid 16046] File "/usr/lib64/python3.4/smtplib.py", line 292, in _get_socket
[Sun Jul 17 18:17:49.121076 2016] [wsgi:error] [pid 16046] self.source_address)
[Sun Jul 17 18:17:49.121079 2016] [wsgi:error] [pid 16046] File "/usr/lib64/python3.4/socket.py", line 512, in create_connection
[Sun Jul 17 18:17:49.121081 2016] [wsgi:error] [pid 16046] raise err
[Sun Jul 17 18:17:49.121084 2016] [wsgi:error] [pid 16046] File "/usr/lib64/python3.4/socket.py", line 503, in create_connection
[Sun Jul 17 18:17:49.121086 2016] [wsgi:error] [pid 16046] sock.connect(sa)
[Sun Jul 17 18:17:49.121091 2016] [wsgi:error] [pid 16046] TimeoutError: [Errno 110] Connection timed out
[Sun Jul 17 18:17:49.121097 2016] [wsgi:error] [pid 16046]
Any idea plz?
Your app can't connect to the specified SMTP server. Check for the following issues:
Is your SMTP details correct?
Do you have a running SMTP server at the configured host/port?
Is the connection being blocked by a firewall?
From the log, it's evident that the SMTP connection is timing out. Please provide more details (code/configuration) to speculate better.
Ok masnum is right. I have changed the port and dissabled TLS use and it works. But I have two questions. My configuration was the follow:
EMAIL_PORT = 587
EMAIL_USE_TLS = True
Is this correct? I can use this in localhost, why not in production server? Busy port maybe?
And the other question. I use this:
try:
send_mail(subject, message, from_email, ['info#domain.com'])
except BadHeaderError:
messages.warning(request, 'Ha ocurrido un fallo. Inténtelo de nuevo más tarde.')
return render(request, "contacto.html", {'form': form})
except TimeoutError:
messages.warning(request, 'Ha ocurrido un fallo. Inténtelo de nuevo más tarde.')
return render(request, "contacto.html", {'form': form})
Why the error is not catched by TimeoutError instead of 504 Apache?
Txh