I'm using a Flask REST API for my application and I've noticed that when I send requests from outside my own network, it's sometimes very, very slow. Most calls get completed within 150ms, but some take 8 seconds. The database connection is to a MySQL database using DBUtils.PersistentDB
The code for the endpoint:
#app.route("/name", methods=["POST"])
#jwt_refresh_token_required
def get_name_and_company():
user = get_jwt_identity()
response_object = server_functions.get_name_and_company(user)
return response_object
The function it uses:
def get_name_and_company(user):
sql = "SELECT fysios.firstname, fysios.lastname, companies.name FROM
fysios " +\
"INNER JOIN companies ON fysios.companyID = companies.id WHERE fysios.email = %s"
cursor = flask_server.get_db().cursor()
cursor.execute(sql, user)
data = cursor.fetchall()
first_name = data[0]['firstname']
last_name = data[0]['lastname']
company = data[0]['name']
response_object = name_and_company(first_name, last_name, company)
return make_response(jsonify(response_object)), 200
Here are the timestamps on the Flask server (it is the internal dev server, but I am running it with threaded=True):
[08/Mar/2019 22:16:54] "OPTIONS /login HTTP/1.1" 200 -
[08/Mar/2019 22:16:55] "POST /login HTTP/1.1" 200 -
[08/Mar/2019 22:16:55] "OPTIONS /clients HTTP/1.1" 200 -
[08/Mar/2019 22:16:55] "OPTIONS /verifyLogin HTTP/1.1" 200 -
[08/Mar/2019 22:16:55] "POST /clients HTTP/1.1" 200 -
[08/Mar/2019 22:16:57] "POST /verifyLogin HTTP/1.1" 200 -
[08/Mar/2019 22:16:57] "OPTIONS /name HTTP/1.1" 200 -
[08/Mar/2019 22:16:58] "POST /clients HTTP/1.1" 200 -
[08/Mar/2019 22:17:05] "POST /name HTTP/1.1" 200 -
As you can see, /name takes a total of 8 seconds and I can't find out why. This call to /name is just an example, it can happen on any of the calls. Is there a way to find out where the Flask application is actually stuck on?
Deploying to AWS Beanstalk solved the problem. I don't know if the limitations of the build-in dev server is to blame, but that's what did it for me.
Related
How can I get my GraphQL API to show more query/post data in the console? I'm running a Django app that is powered by GraphQL and served via a react frontend. With regular Django paths I would see something like this in the development server:
[04/Sep/2020 11:53:08] "GET /my_app/2020/09/01/5cc4e7cc-7.png HTTP/1.1" 200 11330
But with GraphQL all I see is this:
[04/Sep/2020 11:53:18] "POST /graphql HTTP/1.1" 200 32
[04/Sep/2020 11:53:18] "POST /graphql HTTP/1.1" 200 2993
[04/Sep/2020 11:53:29] "POST /graphql HTTP/1.1" 200 11635
Any ideas?
I highly suggest checking out Silky. It's a profiling tool that can show you
the request body - that's where you'll find the graphql
speed of the response
all the DB queries sent during your request
(if you set it up) cprofiler for the request
I am using Django 1.9, Python 3, running locally on Docker (for testing)
Trying to integrate django-saml2-auth into my application.
Pretty much followed all the steps in the docs:
1) All installations were successful
2) New URLs were imported above the rest
3) Installed apps includes 'django_saml2_auth'
4) 'SAML2_AUTH' dict was placed in settings (and all attributes were mapped)
5) In the SAML2 identity provider (using OneLogin), the Single-sign-on URL and Audience URI(SP Entity ID) was set to http://127.0.0.1:8000/saml2_auth/acs/
What happens is that when I get to http://127.0.0.1:8000/admin the browser goes into an infinite redirect loop:
...
[02/May/2018 15:43:06] "GET /admin/ HTTP/1.1" 302 0
[02/May/2018 15:43:06] "GET /admin/login/?next=/admin/ HTTP/1.1" 302 0
[02/May/2018 15:43:07] "POST /saml2_auth/acs/ HTTP/1.1" 302 0
[02/May/2018 15:43:07] "GET /admin/ HTTP/1.1" 302 0
[02/May/2018 15:43:07] "GET /admin/login/?next=/admin/ HTTP/1.1" 302 0
[02/May/2018 15:43:08] "POST /saml2_auth/acs/ HTTP/1.1" 302 0
[02/May/2018 15:43:08] "GET /admin/ HTTP/1.1" 302 0
...
When I disable django-saml2-auth I see that a staff user was created.
In the OneLogin interface I can see that I logged in successfully.
Overriding django_saml2_auth.views.signin(r), where r is a django.core.handlers.wsgi.WSGIRequest, for <WSGIRequest: GET '/admin/login/?next=/admin/'>, and in the request, the user is set to AnonymousUser, COOKIES contain sessionid and csrftoken.
I would expect that a session would start for the user that was created/fetched, and that I will get to an /admin/<whatever> page.
I will appreciate any help in debugging this, thank you!
EDIT: I was able to get it to work by removing AUTHENTICATION_BACKENDS from settings.py- I have 3 other backends that I use. It seems like they conflict with django-saml2-auth.
Is there any way to get django-saml2-auth to work with other backends?
EDIT 2: Will try to integrate django-saml2-pro-auth, which has a backend so will not conflict. I would really appreciate some insight though.
EDIT 3: back to EDIT 2, when I remove all the backends and they don't conflict, the log flow looks like that:
[04/May/2018 15:24:26] "GET /admin/ HTTP/1.1" 302 0
[04/May/2018 15:24:27] "GET /admin/login/?next=/admin/ HTTP/1.1" 302
[04/May/2018 15:26:27] "POST /saml2_auth/acs/ HTTP/1.1" 302 0
[04/May/2018 15:26:27] "GET /admin/ HTTP/1.1" 200 38398
Where the last GET does not get redirected, with 200.
Issue resolved:
After taking a deeper dive- it seems like this code is the issue:
In django_saml2_auth/views.py, acs():
if target_user.is_active:
target_user.backend = 'django.contrib.auth.backends.ModelBackend'
login(r, target_user)
else:
return HttpResponseRedirect(get_reverse([denied, 'denied', 'django_saml2_auth:denied']))
It seems like the default ModelBackend is necessary.
When other backends are used, the default is no longer used by Django, and hence the infinite loop.
If the default backend is added to the list of backends, everything works as intended.
I've been having an issue with a HTTP 500 error. It doesn't seem to cause any real problems until a few hours after the error has been shown. After the error shows up the script begins to act abnormally, giving hangups on controlling a relay and degrading in response time.
I checked my logs both in the pi and the server side and below is what I found but I can't figure out what my issue is since it seems so random, the HTTP request typically returns a 200 status and runs fine.
On the pi:
tail applocation/errors.log
2018-03-05 06:38:33 [HTTP Error] While sending data to remote Server: HTTP Error 500: Internal Server Error
2018-03-05 09:08:50 [HTTP Error] While sending data to remote Server: HTTP Error 500: Internal Server Error
on the server:
This is where it gets weird because most of the time it returns 200 but like I said every so often it gives a 500 error.
cat /var/log/ngnix/access.log | grep 9:0
47.176.12.130 - - [05/Mar/2018:09:08:29 -0800] "POST /api/1.0/access/add/ HTTP/1.1" 200 43 "-" "Python-urllib/3.4"
47.176.12.130 - - [05/Mar/2018:09:08:50 -0800] "POST /api/1.0/access/add/ HTTP/1.1" 500 38 "-" "Python-urllib/3.4"
47.176.12.130 - - [05/Mar/2018:09:09:28 -0800] "POST /api/1.0/access/add/ HTTP/1.1" 200 43 "-" "Python-urllib/3.4"
cat /var/log/ngnix/access.log | grep 500
raspberry.pi.ip - - [05/Mar/2018:06:38:33 -0800] "POST /api/1.0/access/add/ HTTP/1.1" 500 38 "-" "Python-urllib/3.4"
raspberry.pi.ip - - [05/Mar/2018:09:08:50 -0800] "POST /api/1.0/access/add/ HTTP/1.1" 500 38 "-" "Python-urllib/3.4"
in my urls.py inside my API:
from django.conf.urls import url
from . import views
urlpatterns = [
url(r'^1.0/access/add/$', views.access_add, name='access_add'),
url(r'^1.0/employees/active/$', views.employees_active, name='employees_active'),
]
in my views.py inside my API:
#csrf_exempt
def access_add(request):
context = {
'status': 'error',
'msg' : ''
}
if request.method != 'POST':
context["msg"] = "This method only supports POST"
return JsonResponse(context)
try:
content = json.loads(request.body.decode('utf-8'))
except Exception as e:
context['msg'] = "Failed to load JSON parameters."
return JsonResponse(context)
key = content.get('key')
gate = content.get('gate')
code = content.get('code')
if key != KEY:
context['msg'] = "Please provide a valid API key."
return JsonResponse(context)
if gate not in ['entry','exit']:
context['msg'] = "Please provide a valid gate name."
return JsonResponse(context)
gate = 1 if gate == "entry" else 2
if not code:
context['msg'] = "Please provide an access code."
return JsonResponse(context)
context['status'] = "success"
try:
employee = Employee.objects.get(code=code)
Eventlog.objects.create(event=gate, employee=employee, status=True)
except Employee.DoesNotExist:
try:
Eventlog.objects.create(event=gate, status=False)
except:
pass
return JsonResponse(context)
I don't understand what is wrong?
callback uri:
google - http://127.0.0.1:5000/auth/google/
twitter - http://127.0.0.1:5000/auth/tw/
config:
from authomatic.providers import oauth2, oauth1
SECRET_KEY = '####'
AUTH_CONFIG = {
'google': {
'class_': oauth2.Google,
'consumer_key': '####',
'consumer_secret': ####',
'scope': ['email',],
},
'tw': {
'class_': oauth1.Twitter,
'consumer_key': '####',
'consumer_secret': '####',
},
}
controller:
from authomatic.adapters import WerkzeugAdapter
from authomatic import Authomatic
from app import app, db
from app.models.users import User
authomatic = Authomatic(
app.config.get('AUTH_CONFIG'),
app.config.get('SECRET_KEY'),
report_errors=True
)
#app.route('/auth/<provider>/', methods=['GET', 'POST'])
def auth(provider):
print "REQUEST: ", request.args
response = make_response()
result = authomatic.login(WerkzeugAdapter(request, response), provider)
if result:
if result.user:
result.user.update()
if result.user.email:
user = User.query.filter(User.email == result.user.email).first()
if user is None:
user = User(nickname=result.user.name, email=result.user.email)
db.session.add(user)
db.session.commit()
flash('A new user profile has been created for you.')
return redirect(url_for('index'))
flash('Your provider return empty data, try again later.')
return redirect(url_for('index'))
return response
and after I accept access to app in google or twitter, I have redirected to index.html page with flash massage "Your provider return empty data, try again later"
in console I see:
google:
127.0.0.1 - - [29/Nov/2014 00:41:26] "GET /login/ HTTP/1.1" 200 -
REQUEST: ImmutableMultiDict([])
127.0.0.1 - - [29/Nov/2014 00:41:27] "GET /auth/google/ HTTP/1.1" 302 -
REQUEST: ImmutableMultiDict([('state', u'bbee8547ff97e001sdss61e6'), ('code', u'4/ZJRhjCqEzAVep9UL2epaTzYI')])
127.0.0.1 - - [29/Nov/2014 00:41:30] "GET /auth/google/?state=bbee8547ff97e001d3d77161e6&code=4/ZJRhjCqEzAVep9UL2epaTzYI HTTP/1.1" 302 -
127.0.0.1 - - [29/Nov/2014 00:41:30] "GET / HTTP/1.1" 200 -
twitter:
127.0.0.1 - - [29/Nov/2014 00:43:38] "GET /login/ HTTP/1.1" 200 -
REQUEST: ImmutableMultiDict([])
127.0.0.1 - - [29/Nov/2014 00:43:42] "GET /auth/tw/ HTTP/1.1" 302 -
REQUEST: ImmutableMultiDict([('oauth_token', u'KmF9L1m5CYUY9O6joIh0'), ('oauth_verifier', u'95sGsiRz5sTxZua88G')])
127.0.0.1 - - [29/Nov/2014 00:43:44] "GET /auth/tw/?oauth_token=KmF9L1m5CYUY9O6joIh0&oauth_verifier=95sGsiRz5sTxZua88G HTTP/1.1" 302 -
127.0.0.1 - - [29/Nov/2014 00:43:44] "GET / HTTP/1.1" 200 -
May be some thing wrong if i get 302 - on response???
Please help me!
I believe this is a setup issue on the Google Developer Console.
Under the APIs & auth settings of your project you need to authorize the API's you want access to, otherwise nothing will be returned in the results var.
If you inspect it you will see an error message from Google telling you to authorize API's.
Add the following API's from here: APIs & auth > APIs, select Google+ API, this will return the result dictionary with the values you are looking for.
from authomatic.providers import openid, oauth2
CONFIG = {
'oi': {
# OpenID provider dependent on the python-openid package.
'class_': openid.OpenID,
},
'google' : {
'class_': oauth2.Google,
'consumer_key': 'GOOGLE DEVELOPER CLIENT ID',
'consumer_secret': 'GOOGLE DEVELOPER CLIENT SECRET',
'scope': ['profile', 'email']
}
}
on your index.html you will want to trigger the call like:
Sign In With Google
That should trigger the call to the google oauth login.
#app.route('/login/<provider_name>/', methods=['GET', 'POST'])
def login(provider_name):
"""
Login handler, must accept both GET and POST to be able to use OpenID.
"""
# We need response object for the WerkzeugAdapter.
response = make_response()
# Log the user in, pass it the adapter and the provider name.
result = authomatic.login(WerkzeugAdapter(request, response), provider_name)
# If there is no LoginResult object, the login procedure is still pending.
if result:
if result.user:
# We need to update the user to get more info.
result.user.update()
# The rest happens inside the template.
#result.user.data.x will return further user data
return render_template('login.html', email=result.user.email, name=result.user.name)
# Don't forget to return the response.
return response
I'm new to django and bootstrap3 and I've been trying to implement the typeahead by twitter.
my urls.py
url(r'^name_autocomplete/?q=$', 'home.views.name_autocomplete'),
my script:
<script type="text/javascript">
$(document).ready(function() {
$("#navPersonSearch").typeahead({
name: 'people',
remote: 'name_autocomplete/?q=%QUERY'
});
});
my view
def name_autocomplete(request):
query = request.GET.get('query','')
if(len(query) > 0):
print "hello"
results = Person.objects.filter(short__istartswith=query)
result_list = []
for item in results:
result_list.append(item.short)
else:
result_list = []
response_text = json.dumps(result_list, separators=(',',':'))
return HttpResponse(response_text, content_type="application/json")
The requests seem to be getting through from the console output:
[05/Oct/2013 01:34:12] "GET / HTTP/1.1" 200 2007
[05/Oct/2013 01:34:12] "GET /static/bootstrap/css/bootstrap.min.css HTTP/1.1" 304 0
[05/Oct/2013 01:34:12] "GET /static/bootstrap/css/typeahead.js-bootstrap.css HTTP/1.1" 304 0
[05/Oct/2013 01:34:12] "GET /static/bootstrap/js/typeahead.js HTTP/1.1" 304 0
[05/Oct/2013 01:34:15] "GET /name_autocomplete/?q=J HTTP/1.1" 200 2
[05/Oct/2013 01:34:30] "GET /name_autocomplete/?q=Jac HTTP/1.1" 200 2
But the typeahead isn't working.
Replace the urlpattern as follow (you should not specify query string part):
url(r'^name_autocomplete/$', 'home.views.name_autocomplete'),
And in the view, you get query, while the javascript send query string as q=; so replace the following line:
query = request.GET.get('query','')
with
query = request.GET.get('q','')