I have the following three files.
app.py
from flask_restful import Api
from lib import globals
from flask import Flask
from flask.ext.cache import Cache
globals.algos_app = Flask(__name__)
#cache in file system
globals.cache = Cache(globals.algos_app, config={'CACHE_TYPE': 'filesystem', 'CACHE_DIR': '/tmp'})
api = Api(globals.algos_app)
api.add_resource(Test, '/test')
if __name__ == '__main__':
globals.algos_app.run(host='0.0.0.0', debug=True)
globals.py
global algos_app
global cache
Test.py
from flask_restful import Resource
from lib import globals
from flask_restful import Resource
import time
class Test(Resource):
def get(self):
return self.someMethod()
def post(self):
globals.cache.clear()
return self.someMethod()
#globals.cache.cached()
def someMethod(self):
return str(time.ctime())
I have a GET method which needs to the value from the cache and a POST method which updates the cache by first clearing the cache.
However, no matter I call the GET or the POST method, it always gives me the value from the cache.
PS: At the moment I am simply testing on the development server however I do need to deploy it using WSGI later.
I am not sure if it is the best way, but I did it using the following way.
class Test(Resource):
def get(self):
return globals.cache.get('curr_time')
def post(self):
result = self.someMethod()
globals.cache.set('curr_time', result, timeout=3600)
def someMethod(self):
return str(time.ctime())
Related
I want to use scrapy spider in Django views and I tried using CrawlRunner and CrawlProcess but there are problems, views are synced and further crawler does not return a response directly
I tried a few ways:
# Core imports.
from scrapy.crawler import CrawlerProcess
from scrapy.utils.project import get_project_settings
# Third-party imports.
from rest_framework.views import APIView
from rest_framework.response import Response
# Local imports.
from scrapy_project.spiders.google import GoogleSpider
class ForFunAPIView(APIView):
def get(self, *args, **kwargs):
process = CrawlerProcess(get_project_settings())
process.crawl(GoogleSpider)
process.start()
return Response('ok')
is there any solution to handle that and run spider directly in other scripts or projects without using DjangoItem pipeline?
you didn't really specify what the problems are, however, I guess the problem is that you need to return the Response immediately, and leave the heavy call aka function to run in the background, you can alter your code as following, to use the Threading module
from threading import Thread
class ForFunAPIView(APIView):
def get(self, *args, **kwargs):
process = CrawlerProcess(get_project_settings())
process.crawl(GoogleSpider)
thread = Thread(target=process.start)
thread.start()
return Response('ok')
after a while of searching for this topic, I found a good explanation here: Building a RESTful Flask API for Scrapy
My Flask application is not recognizing/using the two defined routes in auth.py, how come?
File structure
Error Msg:
Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
Routes
http://127.0.0.1:5000/home (WORKS)
http://127.0.0.1:5000/profile (WORKS)
http://127.0.0.1:5000/login (DOES NOT WORK)
http://127.0.0.1:5000/register (DOES NOT WORK)
app.py
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/home")
def home():
return render_template("index.html")
#app.route("/profile")
def profile():
return render_template("profile.html")
auth.py
from flask import current_app as app, render_template
#app.route("/login")
def login():
return render_template("login.html")
#app.route("/register")
def register():
return render_template("register.html")
You can't register routes to current_app, instead you have to use a class called Blueprint which is built exactly for this purpose (splitting application into multiple files).
app.py
from flask import Flask, render_template
from auth import auth_bp
app = Flask(__name__)
# Register the blueprint
app.register_blueprint(auth_bp)
#app.route("/home")
def home():
return render_template("index.html")
#app.route("/profile")
def profile():
return render_template("profile.html")
auth.py
from flask import Blueprint, render_template
# Initialize the blueprint
auth_bp = Blueprint('auth', __name__)
#auth_bp.route("/login")
def login():
return render_template("login.html")
#auth_bp.route("/register")
def register():
return render_template("register.html")
See https://flask.palletsprojects.com/en/2.0.x/blueprints/ for more information.
It seems like you have at least two files in which you have these routes. In your app.py file you have /home and /profile, they both work. They work because you initialised the Flask application over there.
Flask offers Blueprints to split up your application. You could create a blueprint called auth for example.
There is a specific tutorial on this subject as well.
I suggest moving the initialisation of the app variable to the __init__.py file and creating a create_app() method that returns app. In this method you can register your blueprints as well.
This method would look like:
def create_app():
app = Flask(__name__)
from . import app as application, auth
app.register_blueprint(auth.bp)
app.register_blueprint(application.bp)
return app
Your auth.py file, for example, would look like:
from flask import Blueprint, render_template
bp = Blueprint('auth', __name__)
#bp.route("/login")
def login():
return render_template("login.html")
#bp.route("/register")
def register():
return render_template("register.html")
I am attempting to create a Flask middleware in order to make py2neo transactions atomic. First I got a working outside of application context error, and I tried to apply this solution, as seen in the following code:
from flask import g
from py2neo import Graph
def get_db():
return Graph(password="secret")
class TransactionMiddleware(object):
def __init__(self, app):
self.app = app
with app.app_context(): # Error raises here.
g.graph = get_db()
g.transaction = g.graph.begin()
def __call__(self, environ, start_response):
try:
app_status = self.app(environ, start_response)
# some code
return app_status
except BaseException:
g.transaction.rollback()
raise
else:
g.transaction.commit()
But I got this error: AttributeError: 'function' object has no attribute 'app_context'.
I don't know if the solution I'm trying is not suitable for my case, or what is the problem.
You are in a WSGI middleware, so what gets passed in as an app is actually a method called Flask.wsgi_app the results of which you a later supposed to return from your __call__ handler.
Perhaps you can simply import your actual flask app and use app_context on that, as in:
from app import app # or whatever module it's in
# and then
class TransactionMiddleware(object):
...
def __call__(self, environ, start_response):
with app.app_context():
...
I would also consider just instantiating your Graph somehere on a module level. Perhaps next to your flask app instantiation and not attaching it to g. This way you can use it without application context:
# app.py
flask = Flask(__main__)
db = Graph(password="secret")
You can use it by directly importing:
# middleware.py
from app import db
In my Flask server app, I wanted to split up my routes into separate files so I used Blueprint. However this caused logging to fail within the constructor function used by a route. Can anyone see what I might have done wrong to cause this?
Simplified example ...
main.py ...
#!/usr/bin/python
import logging
import logging.handlers
from flask import Flask, Blueprint
from my_routes import *
logger = logging.getLogger("")
logger.setLevel(logging.DEBUG)
handler = logging.handlers.RotatingFileHandler("flask.log",
maxBytes=3000000, backupCount=2)
formatter = logging.Formatter(
'[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
logging.getLogger().addHandler(logging.StreamHandler())
logging.debug("started app")
app = Flask(__name__)
app.register_blueprint(api_v1_0)
if __name__ == '__main__':
logging.info("Starting server")
app.run(host="0.0.0.0", port=9000, debug=True)
my_routes.py ...
import logging
import logging.handlers
from flask import Flask, Blueprint
class Class1():
def __init__(self):
logging.debug("Class1.__init__()") # This statement does not get logged
self.prop1=11
def method1(self):
logging.debug("Class1.method1()")
return self.prop1
obj1 = Class1()
api_v1_0 = Blueprint('api_v1_0', __name__)
#api_v1_0.route("/route1", methods=["GET"])
def route1():
logging.debug("route1()")
return(str(obj1.method1()))
You create an instance of Class1 in the global scope of module my_routes.py, so the constructor runs at the time you import that module, the from my_routes import * line in main.py. This is before your logging handler is configured, so there is nowhere to log at that time.
The solution is simple, move your import statement below the chunk of code that sets up the logging handler.
I am starting to write tests for a Django app, which relies on several environment variables. When I am testing it in the shell, I can import os and specify the variables and my tests work just fine. However, when I put them into tests.py, I still get a key error because those variables are not found. here's what my test looks like:
from django.utils import unittest
from django.test.utils import setup_test_environment
from django.test.client import Client
import os
os.environ['a'] = 'a'
os.environ['b'] = 'b'
class ViewTests(unittest.TestCase):
def setUp(self):
setup_test_environment()
def test_login_returning_right_template(self):
""" get / should return login.html template """
c = Client()
resp = c.get('/')
self.assertEqual(resp.templates[0].name, 'login.html')
Is this the wrong place to initialize those variables? I tried to do it on setUp, but with the same result - they are not found. Any suggestions on how to initialize environment variables before running a test suite?
Thanks!
Luka
You should not relay on os.envior in your views. If you have to, do It in your settings.py
MY_CUSTOM_SETTING = os.environ.get('a', 'default_value')
And in views use settings variable:
from django.conf.settings import MY_CUSTOM_SETTING
print MY_CUSTOM_SETTING
Then in your test you can set this setting:
from django.test import TestCase
class MyTestCase(TestCase):
def test_something(self):
with self.settings(MY_CUSTOM_SETTING='a'):
c = Client()
resp = c.get('/')
self.assertEqual(resp.templates[0].name, 'login.html')