The following code works with aws_apigateway.LambdaRestApi + Lambda.
import sys;
import awsgi;
from flask import Flask,request,Response
app = Flask(__name__)
def lambda_handler(event,context):
print("start lambda");
try:
return awsgi.response(app,event,context);
except Exception as e:
print("event",event);
print(sys.exc_info());
#app.route("/",methods=["GET"])
def index():
return "hello";
However, when I ran the same code with Lambda + FunctionURL without using APIgateway, I got an error as in the following log of Lambda. It seems that Flask fails to handle the change in the event object. We cannot use Flask with FunctionURL? If we can, how to do it? If we cannot, what kind of alternative framework do we have that works with FunctionURL?
event {'version': '2.0', 'routeKey': '$default', 'rawPath': '/', 'rawQueryString': '', 'headers': {'sec-fetch-mode': 'navigate', 'x-amzn-tls-version': 'TLSv1.2', 'sec-fetch-site': 'cross-site', 'accept-language': 'ja,en-US;q=0.9,en;q=0.8', 'x-forwarded-proto': 'https', 'x-forwarded-port': '443', ...
(<class 'KeyError'>, KeyError('httpMethod'), <traceback object at 0x7f945ddac230>)
Related
I have the following method in Flask that return a jsonify response, so when hitting http://127.0.0.1:5000/status route, it should renderize in a browser
[
{
user: "admin"
},
{
result: "OK - Healthy"
}
]
The method is this:
#app.route('/status')
def health_check():
response = [
{'user': 'admin'},
{'result': 'OK - Healthy'}
]
return jsonify(response)
I am trying to build a test case that examine the content of the jsonify(response) object returned:
class HealthStatusCase(unittest.TestCase):
def test_health_check(self):
response = health_check()
self.assertEqual(response, ['200 OK'])
But I don't know how to check the content of a jsonify output, the above test is misleading.
When I check the value of the jsonify(response) I get
pdb> jsonify(response)
<Response 71 bytes [200 OK]>
ipdb>
But that I am interested in is to access to the content of the response list, such as:
{'result': 'OK - Healthy'} and compare that key value pair.
UPDATE
I've followed the pytest approach suggested, by working with a fixture to make a request to /status endpoint, so the test case is now like this:
#pytest.fixture
def test_health_check(client):
response = client.get('/status')
assert response.json == [
{'user': 'admin'},
{'result': 'OK - Healthy'}
]
When I execute python -m pytest tests/test_health_check.py the test pass:
> python -m pytest tests/test_health_check.py
============================================================== test session starts ===============================================================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/../../
plugins: flask-1.2.0
collected 1 item
tests/test_health_check.py . [100%]
=============================================================== 1 passed in 0.11s ================================================================
But then something that I miss is that if I modified the assert response.json content to let's say like this:
#pytest.fixture
def test_health_check(client):
response = client.get('/status')
assert response.json == [
{'user': 'admin'},
{'result': 'OKdsdsd - Healthy'}
]
The test also pass
I know a feature is intended to ilustrate a behavior and run it inside a test, but is there a way to make a relationship with the original values of the json in my orginal response list? I feel that this test is non meaningful.
Better use pytest library for testing.
With it your code will be super simple
def test_health_check(client):
response = client.get('/status')
assert response.json == [
{'user': 'admin'},
{'result': 'OK - Healthy'}
]
Based on Flask docs article about testing: https://flask.palletsprojects.com/en/2.2.x/testing/
Docs for pytest: https://docs.pytest.org/
Update:
In order to make the test above to work you should add this to tests/conftest.py
import pytest
from my_project import create_app
#pytest.fixture()
def app():
app = create_app()
app.config.update({
"TESTING": True,
})
yield app
#pytest.fixture()
def client(app):
return app.test_client()
#pytest.fixture()
def runner(app):
return app.test_cli_runner()
Or if you not have smth as a create_app function
import pytest
from my_project import app
#pytest.fixture()
def client(app):
return app.test_client()
#pytest.fixture()
def runner(app):
return app.test_cli_runner()
Copied from: https://flask.palletsprojects.com/en/2.2.x/testing/#fixtures
Environment:
Ubuntu 16.04.6
conda 4.12.0
Apache/2.4.18 (Ubuntu)
python==3.8.1
Django==4.0.3
channels==3.0.5
asgi-redis==1.4.3
asgiref==3.4.1
daphne==3.0.2
I am attempting to create a websocket service that only relays messages from redis to an authenticated user. Users do not communicate with each other, therefore I don't need Channel layers and my understanding is that Channel layers are an entirely optional part of Channels.
I'm simply trying to broadcast messages specific to a user that has been authenticated through custom middleware. I have custom auth middleware that takes an incoming session id and returns the authenticated user dictionary along with a user_id. I'm also attaching this user_id to the scope.
I have elected to route all traffic through daphne via Apache2 using ProxyPass on port 8033. This is preferred since I'm using a single domain for this service.
However, I'm really struggling to maintain a connection to the websocket server, particularly if I refresh the browser. It will work on the first request, and fail after with the following message in journalctl:
Application instance <Task pending name='Task-22' coro=<ProtocolTypeRouter.__call__() running at /root/miniconda2/lib/python3.8/site-packages/channels/routing.py:71> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f658c03f670>()]>> for connection <WebSocketProtocol client=['127.0.0.1', 46010] path=b'/ws/userupdates/'> took too long to shut down and was killed.
After spending hours on the github for channels, and trying many of the suggestions (particularly found on https://github.com/django/channels/issues/1119), I'm still at a loss. The version of the code below is the best working version so far that at least establishes an initial connection, sends back the connection payload {"success": true, "user_id": XXXXXX, "message": "Connected"} and relays all redis messages successfully. But, if I refresh the browser, or close and reopen, it fails to establish a connection and posts the above journalctl message until I restart apache and daphne.
My hunch is that I'm not properly disconnecting the consumer or I'm missing proper use of async await. Any thoughts?
Relevant apache config
RewriteEngine On
RewriteCond %{HTTP:Connection} Upgrade [NC]
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteRule /(.*) ws://127.0.0.1:8033/$1 [P,L]
<Location />
ProxyPass http://127.0.0.1:8033/
ProxyPassReverse /
</Location>
app/settings.py
[...]
ASGI_APPLICATION = 'app.asgi.application'
ASGI_THREADS = 1000
CHANNEL_LAYERS = {}
[...]
app/asgi.py
import os
from django.core.asgi import get_asgi_application
from django.urls import path
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'app.settings')
import websockets.routing
from user.models import UserAuthMiddleware
application = ProtocolTypeRouter({
'http': get_asgi_application(),
"websocket": AllowedHostsOriginValidator(
UserAuthMiddleware(
URLRouter(websockets.routing.websocket_urlpatterns)
)
),
})
user/models.py::UserAuthMiddleWare
Pulls user_id from custom authentication layer and attaches user_id to scope.
class UserAuthMiddleware(CookieMiddleware):
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
# Check this actually has headers. They're a required scope key for HTTP and WS.
if "headers" not in scope:
raise UserSessionError(
"UserAuthMiddleware was passed a scope that did not have a headers key "
+ "(make sure it is only passed HTTP or WebSocket connections)"
)
# Go through headers to find the cookie one
for name, value in scope.get("headers", []):
if name == b"cookie":
cookies = parse_cookie(value.decode("latin1"))
break
else:
# No cookie header found - add an empty default.
cookies = {}
# now gather user data from session
try:
req = HttpRequest()
req.GET = QueryDict(query_string=scope.get("query_string"))
setattr(req, 'COOKIES', cookies)
setattr(req, 'headers', scope.get("headers")),
session = UserSession(req)
scope['user_id'] = session.get_user_id()
except UserSessionError as e:
raise e
return await self.app(scope, receive, send)
websockets/routing.py
from django.urls import re_path
from . import consumers
websocket_urlpatterns = [
re_path(r'ws/userupdates/', consumers.UserUpdatesConsumer.as_asgi())
]
websockets/consumers.py::UserUpdatesConsumer
from channels.generic.websocket import JsonWebsocketConsumer
import json, redis
class UserUpdatesConsumer(JsonWebsocketConsumer):
def connect(self):
self.accept()
self.redis = redis.Redis(host='127.0.0.1', port=6379, db=0, decode_responses=True)
self.p = self.redis.pubsub()
if 'user_id' not in self.scope:
self.send_json({
'success': False,
'message': 'No user_id present'
})
self.close()
else:
self.send_json({
'success': True,
'user_id': self.scope['user_id'],
'message': 'Connected'
})
self.p.psubscribe(f"dip_alerts")
self.p.psubscribe(f"userupdates_{self.scope['user_id']}*")
for message in self.p.listen():
if message.get('type') == 'psubscribe' and message.get('data') in [1,2]:
continue
if message.get('channel') == "dip_alerts":
self.send_json({
"key": "dip_alerts",
"event": "dip_alert",
"data": json.loads(message.get('data'))
})
else:
self.send(message.get('data'))
I have tried several things but I am not able to figure this one out. I have a back-end flask app and a front-end client written in Angular. When I submit my register user form I get a cors error. I have read the documentation for flask_cors and have tried to get it to work but I still get the same error below:
Access to XMLHttpRequest at 'http://localhost:5000/v1/auth/register' from origin 'http://localhost:4200' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Here is my app.py from the flask app.
Any help is greatly appreciated!
import os, sys
from flask import Flask
import pathlib
from flask_cors import CORS
from flask_restplus import Api, Resource, fields
from werkzeug.middleware.proxy_fix import ProxyFix
import coloredlogs, logging as log
coloredlogs.install()
from main.apis.user import api as User
from main.apis.auth import api as Auth
from main import create_app
from flask_pymongo import PyMongo
# Init app
app = Flask(__name__)
#cors = CORS(app, resources={r"*": {"origins": "*"}})
CORS(app, origins="http://localhost:4200", allow_headers=[
"Content-Type", "Authorization", "Access-Control-Allow-Credentials","Access-Control-Allow-Origin"],
supports_credentials=True, intercept_exceptions=False)
authorizations = {
'token': {
'type': 'apiKey',
'in': 'header',
'name': 'Authorization'
}
}
config_name = os.getenv('FLASK_CONFIG')
app = create_app(config_name)
api = Api(app, authorizations=authorizations, version='1.0', title='API docs',
description='A simple REST API with JWT authentication.',
doc='/docs'
)
app.config['jwt']._set_error_handler_callbacks(api)
app.config['ROOT_DIR'] = pathlib.Path(__file__).parent.absolute()
# #app.before_first_request
# this function is to init the db and realted models
# def create_tables():
# print("Before first statement")
# db.create_all()
# Endpoints
api.add_namespace(Auth, path='/v1')
api.add_namespace(User, path='/v1')
# Run Server
if __name__ == '__main__':
app.run()
adding this seemed to fix my issue.
#app.after_request
def after_request(response):
response.headers.add('Access-Control-Allow-Origin', '*')
response.headers.add('Access-Control-Allow-Headers', 'Content-Type')
return response
Looks like in your code you are creating app = Flask(__name__), applying the CORS to that variable and then over writing app by using app = create_app(config_name) a few lines later. So that causes your CORS setup on the first app to be lost.
I have set up a server with flask CORS and had it working for sending data to a React web app that I built, but when I went to test the POST method it stoped working and now it is broken for sending and receiving. The error log in the console of the web app is: "Origin http://localhost:3000 is not allowed by Access-Control-Allow-Origin. Fetch API cannot load http://127.0.0.1:5000/ due to access control checks. "
I ran into this issue earlier and added flask_cors and it worked for a while. Here is my server code:
from flask_cors import CORS, cross_origin
app = FlaskAPI(__name__)
app.config['SECRET_KEY'] = 'the quick brown fox jumps over the lazy dog'
app.config['CORS_HEADERS'] = 'Content-Type'
cors = CORS(app, resources={r"/": {"origins": "http://localhost:port"}})
# Also fails with this variation
# cors = CORS(app, resources={r"/api/*": {"origins": "*"}})
#app.route("/", methods=['GET', 'POST'])
#cross_origin(origin='localhost',headers=['Content- Type','Authorization'])
# Also fails with these variations
# #cross_origin(origin='http://127.0.0.1:5000/',headers=['Content- Type','Authorization'])
# #cross_origin(origin='http://localhost:3000',headers=['Content- Type','Authorization'])
def job_api():
with app.app_context():
job_data = get_job_data()
json_data = jsonify(eqtls=[job.data for job in job_data])
return json_data
if __name__ == "__main__":
app.run(debug=True)
Here is my client code:
componentDidMount() {
fetch('http://127.0.0.1:5000/')
.then(res => res.json())
.then((data) => {
this.setState({ job_data: data.eqtls })
})
.catch(console.log)
}
You need to enable CORS policy on your API, so it can accept requests from across different hosts.
Just do a google Flask cors, and make sure you are Accepting '*' or specifically your URL.
If you accept cors though you should be able to accept all CORS and then make your API robust enough so that no nasty data can be requested
Try:
from flask_cors import CORS, cross_origin
app = FlaskAPI(__name__)
app.config['SECRET_KEY'] = 'the quick brown fox jumps over the lazy dog'
app.config['CORS_HEADERS'] = 'Content-Type'
#app.route("/", methods=['GET', 'POST'])
#cross_origin()
def job_api():
with app.app_context():
job_data = get_job_data()
json_data = jsonify(eqtls=[job.data for job in job_data])
return json_data
if __name__ == "__main__":
app.run(debug=True)
I read the documenttation, and can just add #cross_origin() as a simple decorator: https://flask-cors.readthedocs.io/en/latest/#route-specific-cors-via-decorator
I'm trying to set a Authentication middleware for django channels. I want this middleware to be active only for websocket requests.
Seems like that in this case i don't get a full middleware functionality. For example i can't get response = self.get_response(scope) working:
'TokenAuthMiddleware' object has no attribute 'get_response'
Everything is allright with this middleware now (it is activated only for websocket requests and not registered in settings.py), except that i need a means to modify a response status codes (block anonymous users and set the error code for ExpiredSignatureError). Any help appreciated. I use Django 2.0.6 and channels 2.1.1. jwt authentication by djangorestframework-jwt
middleware:
import jwt, re
import traceback
import logging
from channels.auth import AuthMiddlewareStack
from django.contrib.auth.models import AnonymousUser
from django.conf import LazySettings
from jwt import InvalidSignatureError, ExpiredSignatureError, DecodeError
from project.models import MyUser
settings = LazySettings()
logger = logging.getLogger(__name__)
class TokenAuthMiddleware:
"""
Token authorization middleware for Django Channels 2
"""
def __init__(self, inner):
self.inner = inner
def __call__(self, scope):
headers = dict(scope['headers'])
auth_header = None
if b'authorization' in headers:
auth_header = headers[b'authorization'].decode()
else:
try:
auth_header = _str_to_dict(headers[b'cookie'].decode())['X-Authorization']
except:
pass
logger.info(auth_header)
if auth_header:
try:
user_jwt = jwt.decode(
auth_header,
settings.SECRET_KEY,
)
scope['user'] = MyUser.objects.get(
id=user_jwt['user_id']
)
except (InvalidSignatureError, KeyError, ExpiredSignatureError, DecodeError):
traceback.print_exc()
pass
except Exception as e: # NoQA
logger.error(scope)
traceback.print_exc()
return self.inner(scope)
TokenAuthMiddlewareStack = lambda inner: TokenAuthMiddleware(AuthMiddlewareStack(inner))
def _str_to_dict(str):
return {k: v.strip('"') for k, v in re.findall(r'(\S+)=(".*?"|\S+)', str)}
routing.py
application = ProtocolTypeRouter({
# (http->django views is added by default)
'websocket': TokenAuthMiddlewareStack(
URLRouter(
cmonitorserv.routing.websocket_urlpatterns
)
),
})
Wasn't able to find a solution using middleware.
For now solved by handling auth permissions in consumers.py
def _is_authenticated(self):
if hasattr(self.scope, 'auth_error'):
return False
if not self.scope['user'] or self.scope['user'] is AnonymousUser:
return False
return True
Another important thing which doesn't seem to be documented anywhere - to reject a connection with the custom error code, we need to accept it first.
class WebConsumer(WebsocketConsumer):
def connect(self):
self.accept()
if self._is_authenticated():
....
else:
logger.error("ws client auth error")
self.close(code=4003)