I am trying to setup a Flask API limiter for each user. The following code limits an IP Address to 3 request per minute.
from flask_limiter import Limiter
from flask_limiter.util import get_remote_address
limiter = Limiter(
key_func=get_remote_address, #limit by IP Address
storage_uri="redis://localhost:6379",
strategy="moving-window"
)
#api.route('/api/submit-code')
#limiter.limit('3 per minute')
def submit_code():
user_id = session.get("user_id")
if not user_id:
return jsonify({"error": "Unauthorized"}), 401
How can I change this to limit the user instead of IP address? I am using server sessions so I'm not sure how to include user_id in the limiter decorator.
I ended up modifying the decorator to the following:
#limiter.limit('3 per minute', key_func = lambda : session.get("user_id"))
The limiter will not apply to users that are not logged in. You can simply add a check in the function for this like the OP.
Related
REST API service has a limit of requests (say a maximum of 100 requests per minute). In Django, I am trying to allow USERs to access such API and retrieve data in real-time to update SQL tables. Therefore there is a problem that if multiple users are trying to access the API, the limit of requests is likely to be exceeded.
Here is a code snippet as an example of how I currently perform requests - each user will add a list of objects he wants to request and run request_engine().start(object_list) to access the API. I use multithreading to speed up requests. I also allow retrying failed API requests via setting a limit on the number of requests for each request object upper_limit.
As I understand there should be some queue for API requests. I anticipate there must be a more elegant solution for this, however, I could not find any similar examples. How can one implement/rewrite this for multiUSER usage with Django?
import requests
from multiprocessing.dummy import Pool as ThreadPool
N=50 # number of threads
upper_limit=1 # limit on the number of requests for a single object
class request_engine():
def __init__(self):
pass
def start(self,objs):
self.objs={obj:{'status':0,'data':None} for obj in objs}
done=False
while not done:
self.parallel_requests()
done=all(_['status']>upper_limit or _['status']==-1 for obj,_ in self.objs.items())
return dict(self.objs)
def single_request(self,request_obj):
URL = f"https://reqres.in/api/users?page={request_obj}"
r = requests.get(url = URL)
if r.ok:
res = r.json()
self.objs[request_obj]['status']=-1
self.objs[request_obj]['data']=res
else:
self.objs[request_obj]['status']+=1
def parallel_requests(self):
objs=[obj for obj,_ in self.objs.items() if _['status']!=-1 and _['status']<=upper_limit]
pool = ThreadPool(N)
pool.map(self.single_request, objs)
pool.close()
pool.join()
objs=[1,2,3,4,5,6,7,7,8,234,124,24,535,6,234,24,4,1,3,4,5,4,3,5,3,1,5,2,3,5,3]
result=request_engine().start(objs)
print([_['status'] for obj,_ in result.items()])
# status corresponds to the number of unsuccessful requests
# status=-1 implies success of the request
Thanks in advance.
I have two containers (docker) running the application and I'm trying to redirect the request from one of the container to another. The container where I'm redirecting from has this code and the 172.17.0.3 is the IP of the second container. I have seen that it can be pinged. In the other container I don't have the else part and no if condition check. When I run a curl request to this container using another client container in the same network curl http://172.17.0.2:3333?count=100, it should ideally redirect but I get Internal Server Error as the response. However, when I login to the container 2 and run curl, I get redirected to ... response.
from flask_restful import Resource, Api
app = Flask(__name__)
api = Api(app)
class Greeting (Resource):
def get(self):
offload = True
if offload == False:
count = request.args.get('count')
count = int(count)
for i in range(count):
continue
return count
else:
count = request.args.get('count')
redirect_str = "http://172.17.0.3:3333?count=" + count
return redirect(redirect_str, code=302)
api.add_resource(Greeting, '/') # Route_1
if __name__ == '__main__':
app.run('0.0.0.0', '3333')
I want to be able to wait for the response back from the server 172.17.0.3 and once I receive the message, send the response back to the client. Can anyone tell me how it can be done?
You need to send a request to the other container instead of redirect to it if you want to wait for its response. Using e.g. the requests library this would look something like
import requests
resp = requests.get('http://172.17.0.3:3333?count=' + count)
return resp.text
Check request's Quickstart guide for more info: https://requests.readthedocs.io/en/master/user/quickstart.
So I'm able to create aa JWT on login using flask-jwt-extended
I set the expiration to 5 minutes.
So I have routes in my VueJS app and when they are called(beforeeach) I make a call to "/api/check" to make sure that the token is valid.
All other API calls are via axios to the backend.
What I'm not understanding that since the cookie is HTTP only I can't check the expiation with javascript...does that mean I have to ping the backend every X minutes along with with every axios call to refresh the cookie then make the actual API call?
Seems like a lot of overhead.
Lots of code out there on the nuts and bolts however not much on the actual steps behind the issue I'm having...understanding...
Thanks
You could have the backend automatically refresh the cookie if it is close to expiring without having to do anything extra on the frontend. Something like this (untested)
#app.after_request
def refresh_jwt_if_near_expiring(response):
expires_time = get_raw_jwt().get('exp')
if not expires_time:
return response
# Change the time delta based on your app and exp config.
target_time = datetime.datetime.utcnow() + datetime.timedelta(minutes=5)
if (target_time > expires_time):
access_token = create_access_token(identity=get_jwt_identity())
set_access_cookies(response, access_token)
return response
Then if the user is inactive for X many minutes they get logged out, otherwise they keep getting a new token as needed. If they do wait too long and get logged out, the backend returns a 401 and your frontend can prompt them to log in again.
A backend service which decomposes the jwt and returns the expiration time would do the trick.
in my opinion you don't need route "/api/check" to make sure tokens are valid. just add a decorator #jwt_required
#api.route('/api/user')
#jwt_required
def api_user():
result = #your logic
return jsonify(result), 200
Below is my working code to get twitter followers for certain accounts (#hudsonci in this case).
My issue is the time that it is taking to pull in all of these followers. This account specifically has approx 1,000 followers ... I can only get to 300 at a time with the rate limiting restrictions. So, it is taking > an hour to get all the followers for this account. I can imagine this will become a huge pain in the ass for large accounts.
I am looking for some suggestions for how I can improve this. I feel like I am not taking full advantage of the pagination cursor, but I can't be sure.
any help is appreciated.
#!/usr/bin/env python
# encoding: utf-8
import tweepy
import time
#Twitter API credentials
consumer_key = "mine"
consumer_secret = "mine"
access_key = "mine"
access_secret = "mine"
#authorize twitter, initialize tweepy
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)
def handle_errors(cursor):
while True:
try:
yield cursor.next()
except tweepy.TweepError:
time.sleep(20 * 60)
for user in handle_errors(tweepy.Cursor(api.followers,screen_name='hudsonci').items()):
print user.screen_name
As per the Twitter documentation for followers you need to use the count parameter.
Specifies the number of IDs attempt retrieval of, up to a maximum of 5,000 per distinct request.
So, adding count=5000 should help you.
You are getting 300 followers at a time because getting the followers object (as opposed to IDs only) has a page limit 20. With 15 requests per window, that comes out to be 300 followers.
Here are the docs for followers: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-followers-list
First off, if there is a true, official way of having flash/flex's NetConnections usurp the session/cookie state of the surrounding web page, so that if the user has already logged in, they don't need to provide credentials again just to set up an AMF connection, please stop me now and post the official answer.
Barring that, I'm assuming there is not, as I have searched and it seems to not exist. I've concocted a means of doing this, but want some feedback as to whether it is secure.
Accessing a wrapper-page for a flash object will always go to secure https due to django middleware
When the page view is loaded in Django, it creates a "session alias" object with a unique key that points to the current session in play (in which someone ostensibly logged in)
That session alias model is saved, and that key is placed into a cookie whose key is another random string, call it randomcookie
That randomcookie key name is passed as a context variable and written into the html as a flashvar to the swf
The swf is also loaded only via https
The flash application uses ExternalInterface to call java to grab the value at that randomcookie location, and also deletes the cookie
It then creates a NetConnection to a secure server https location, passing that randomcookie as an argument (data, not in the url) to a login-using-cookie rpc
At the gateway side, pyamf looks up the session alias and gets the session it points to, and logs in the user based on that (and deletes the alias, so it can't be reused)
(And the gateway request could also set the session cookie and session.session_key to the known session ID, but I could let it make a whole new session key... I'm assuming that doing so should affect the response properly so that it contains the correct session key)
At this point, the returned cookie values on the flash side should stick to the NetConnection so that further calls are authenticated (if a connection is authenticated using username and password the normal way, this definitely works, so I think this is a safe bet, testing will soon prove or disprove this)
So, is this unsafe, or will this work properly? As far as I know, since the html page is guaranteed to be over ssl, the key and cookie data should be encrypted and not steal-able. Then, the info therein should be safe to use one-time as basically a temporary password, sent again over ssl because the gateway is also https. After that, it's using the normal pyAMF system over https and not doing anything out of the ordinary.
No responses on this so far, so the best I can do is confirm that it does in fact physically work. For details on how to set up Flex Builder to write html-wrappers that communicate with Django pages templates, see my other post. The above was accomplished using a combination of the aforementioned, plus:
Made a SessionAlias model:
class SessionAlias(models.Model):
alias = models.CharField( max_length=40, primary_key=True )
session = models.ForeignKey( Session )
created = models.DateTimeField( auto_now_add=True )
Flex points to a Django page that loads via a view containing:
s = SessionAlias()
s.alias = SessionStore().session_key // generates new 40-char random
s.session = Session.objects.get( session_key=request.session.session_key )
s.save();
randomcookie = SessionStore().session_key // generates new 40-char random
kwargs['extra_context']['randomcookie'] = randomcookie
response = direct_to_template( request, **kwargs )
response.set_cookie( randomcookie, value=alias )
In the flex html-wrapper, where randomcookie is the location to look for the alias:
<param name="flashVars" value="randomcookie={{randomcookie}}" />
In applicationComplete, where we get randomcookie and find the alias, and log on using that:
var randomcookie:String = this.parameters["randomcookie"];
// randomcookie is something like "abc123"
var js:String = "function get_cookie(){return document.cookie;}";
var cookies:String = ExternalInterface.call(js).toString();
// cookies looks like "abc123=def456; sessionid=ghi789; ..."
var alias:String = // strip out the "def456"
mynetconnection.call( "loginByAlias", alias, successFunc, failureFunc );
Which in turn access this pyamf gateway rpc:
from django.contrib.auth import SESSION_KEY, load_backend
from django.contrib.auth.models import User
from django.contrib import auth
from django.conf import settings
def loginByAlias( request, alias ):
a = SessionAlias.objects.get( alias=alias )
session_engine = __import__( settings.SESSION_ENGINE, {}, {}, [''] )
session_wrapper = session_engine.SessionStore( a.session.session_key )
user_id = session_wrapper.get( SESSION_KEY )
user = User.objects.get( id=user_id )
user.backend='django.contrib.auth.backends.ModelBackend'
auth.login( request, user )
a.delete()
return whateverToFlash
And at that point, on the flash/flex side, that particular mynetconnection retains the session cookie state that can make future calls such that, inside the gateway, request.user is the properly-authenticated user that logged onto the webpage in the first place.
Note again that the run/debug settings for flex must use https, as well as the gateway settings for NetConnection. And when releasing this, I have to make sure that authenticated users stay on https.
Any further info from people would be appreciated, especially if there's real feedback on the security aspects of this...
IE doesn't give access to cookies in local development but if you publish the SWF and put on a domain, it should pickup the session just like ever other browser. Use Firefox 3.6 to build your flex apps locally.
Tested in IE8, Firefox using a pyamf gateway on Flex 3 with NetConnection. The gateway function was decorated with #login_required