I'm using flask blueprints just as written here:
http://flask.pocoo.org/docs/0.12/patterns/appdispatch/
Specifically, I'm running the following piece of code:
from werkzeug.wsgi import DispatcherMiddleware
from frontend_app import application as frontend
from backend_app import application as backend
application = DispatcherMiddleware(frontend, {
'/backend': backend
})
Flask has an awesome feature of reloading application as it detects change in code. However, in my case it is not desirable for both apps: backend app is too heavy to be reloaded every few minutes. When I reload fronend, backend gets reloaded as well and it takes too long. Is there any way to reload only app that was changed?
I realize it might be difficult, because as far as I understand at this point
application = DispatcherMiddleware(frontend, {
'/backend': backend
})
applications are merged into a single one and they are not separate anymore.
Anyway, maybe there is a different way to achieve what I want?
Thanks.
I am not sure if there is anything direct for doing such thing. Only thing I can think if you preventing a reinitialization of your module.
See a below example of how one can achieve this
reloadme.py
import sys
if 'reloadme' in sys.modules and hasattr(sys.modules['reloadme'], '__MODULE_LOADED'):
print ("module is being reloaded")
mod = sys.modules['reloadme']
x = mod.x
else:
print ("Loading module for first time")
__MODULE_LOADED = True
x = 2
def printx():
print (x)
reloadtest.py
from importlib import reload
import reloadme
reloadme.printx()
reloadme.x = 10
reloadme.printx()
print("Reloading reloadme to check if x is changed back to 2")
reload(reloadme)
reloadme.printx()
The output of running reloadtest.py will be as below
Loading module for first time
2
10
Reloading reloadme to check if x is changed back to 2
Reloading module
10
Related
I'm trying to organize my Flask app, as it's getting quite big in length at close to 1000 lines
I am trying to separate the REST API from my main app, by using the approach shown here: https://flask-restx.readthedocs.io/en/latest/scaling.html#multiple-apis-with-reusable-namespaces
What remains in my main.py is something like
from apiv1 import blueprint as api1
REST_API = Flask(__name__)
REST_API.wsgi_app = ProxyFix(REST_API.wsgi_app, x_for=1)
REST_API.register_blueprint(api1)
However in my app, I am using the flask limiter
# Very basic DOS prevention
try:
limiter = Limiter(
REST_API,
key_func=get_remote_address,
storage_uri="redis://localhost:6379/1",
# storage_options={"connect_timeout": 30},
strategy="fixed-window", # or "moving-window"
default_limits=["90 per minute"]
)
# Allow local workatation run
except:
limiter = Limiter(
REST_API,
key_func=get_remote_address,
default_limits=["90 per minute"]
)
This is likewise placed in a decorator to my various API functions
decorators = [limiter.limit("30/minute")]
def post(self, server_id = ''):
# [..]
Now that I am splitting my REST api from the same file that declaring my endpoints, I don't know how to pass its object. The REST_API var exists only in my main.py
How should I handle passing the limiter variable, or any other global objects for that matter?
I worked for a few hours yesterday but I finally understood the pythonic way to do this sort of thing.
I just couldn't wrap my head around how imports function so I was struggling with questions like "how do I pass the variable during import" etc.
Finally it clicked for me that I need to follow a "pull" method with my imports, instead of trying to push variables into them. I.e. I setup the center location in my package's __init__ which will import my logger module, and then my other modules will import THAT logger variable from there.
So in my app's __init__, I have
from .limiter import limiter
And in the app/apis/v1.py I have
from .. import limiter
And this seems to finally work. I don't know if this is the expected way, meaning to play with relative module paths, so if there;s a more elegant way, please let me know
I built a small code to find analogies using word2vec and it runs fine as stand alone application. Here is the working code
import numpy as np
# Get the interactive Tools for Matplotlib
%matplotlib notebook
from gensim.test.utils import datapath, get_tmpfile
from gensim.models import KeyedVectors
from gensim.scripts.glove2word2vec import glove2word2vec
import os
glove_file = os.path.abspath('glove.6B/glove.6B.100d.txt')
word2vec_glove_file = get_tmpfile("glove.6B.100d.word2vec.txt")
glove2word2vec(glove_file, word2vec_glove_file)
model = KeyedVectors.load_word2vec_format(word2vec_glove_file)
def analogy(x1, x2, y1):
result = model.most_similar(positive=[y1, x2], negative=[x1])
return result[0][0]
analogy('woman', 'queen', 'man')
Now, I plan to use flask to create a small web application, so that users can find analogies via the webpage. For this I have a basic question
I assume I need to save the model and then load it when I start the server. Please correct me I am I am wrong.
Here is the code that using Flask, it is working, but can you please suggest if saving model is required here?
2. Any suggestions to improve this code are welcome!
import numpy as np
from gensim.test.utils import datapath, get_tmpfile
from gensim.models import KeyedVectors
from gensim.scripts.glove2word2vec import glove2word2vec
import os
from flask import Flask, request
app = Flask(__name__)
#app.route("/", methods=['GET'])
def welcome():
return "Welcome to our Machine Learning REST API!"
#app.route("/analogy", methods=['GET'])
def analogy_route():
word1 = request.args.get("word1")
word2 = request.args.get("word2")
word3 = request.args.get("word3")
result = model.most_similar(positive=[word3, word2], negative=[word1])
return str(result[0][0])
if __name__ == "__main__":
glove_file = os.path.abspath('glove.6B/glove.6B.100d.txt')
word2vec_glove_file = get_tmpfile("glove.6B.100d.word2vec.txt")
glove2word2vec(glove_file, word2vec_glove_file)
model = KeyedVectors.load_word2vec_format(word2vec_glove_file)
app.run(host='0.0.0.0', port=5000, debug=True)
You probably don't want to be doing the GLoVe-to-word2vec format conversion, into a temporary file, every time you start your service. (It probably takes a noticeable amount of time, and may be filling a temp directory with redundant copies of the same data.)
Instead, perform the conversion only once, into a non-temporary location. Then, ignore the original glove.6B.100d.txt file entirely – it's no longer needed. Instead, just ensure the converted file is available to your web service in a stable location.
Very roughly, that means:
Run once, anywhere:
glove2word2vec('glove.6B/glove.6B.100d.txt', `glove.6B.100d.word2vec.txt`)
(Note that neither the use of absfile() for get_tmpfile() are strictly necessary – you can supply string paths directly to the glove2word2vec() function.)
Ensure that the new file glove.6B.100d.word2vec.txt is available in the working directory of your web service.
Have your web service's __main__ branch just load the already-converted file, avoiding redundant repeated conversion work:
if __name__ == "__main__":
model = KeyedVectors.load_word2vec_format('glove.6B.100d.word2vec.txt')
app.run(host='0.0.0.0', port=5000, debug=True)
(The exact path 'glove.6B.100d.word2vec.txt' might be slightly different depending on where you choose to place the full file.)
I recently started learning flask and created a simple webapp which randomly generates kids' math work sheets in PDF based on user input.
The PDF opens automatically in a browser and can be viewed. But when I try downloading it both on a PC and in Chrome iOS, I get error messages (Chrome PC: Failed - Network error / Chrome iOS:the file could not be downloaded at this time).
You can try it out here: kidsmathsheets.com
I suspect it has something to do with the way I'm generating and returning the PDF file. FYI I'm using ReportLab to generate the PDF. My code below (hosted on pythonanywhere):
from reportlab.lib.pagesizes import A4, letter
from reportlab.pdfgen import canvas
from reportlab.platypus import Table
from flask import Flask, render_template, request, Response
import io
from werkzeug import FileWrapper
# Other code to take in input and generate data
filename=io.BytesIO()
if letter_size:
c = canvas.Canvas(filename, pagesize=letter)
else:
c = canvas.Canvas(filename, pagesize=A4)
pdf_all(c, p_set, answer=answers, letter=letter_size)
c.save()
filename.seek(0)
wrapped_file = FileWrapper(filename)
return Response(wrapped_file, mimetype="application/pdf", direct_passthrough=True)
else:
return render_template('index.html')
Any idea what's causing the issue? Help is much appreciated!
Please check whether you are using an ajax POST request for invoking the endpoint to generate your data and display the PDF respectively. If this is the case - quite probably this causes the behaviour our observe. You might want to try invoking the endpoint with a GET request to /my-endpoint/some-hashed-non-reusable-id-of-my-document where some-hashed-non-reusable-id-of-my-documentwill tell the endpoint which document to serve without allowing users to play around with guesstimates about what other documents you might have. You might try it first like:
#app.route('/display-document/<document_id>'):
def display_document(document_id):
document = get_my_document_from_wherever_it_is(document_id)
binary = get_binary_data_from_document(document)
.........
Prepare response here
.......
return send_file(binary, mimetype="application/pdf")
Kind note: a right click and 'print to pdf' will work but this is not the solution we want
I have a Dash app that makes some graphs based on data drawn from an API, and I'd like to give the user an option to change a parameter, pull new data based on this, and then redraw the graphs. It could be through a form, but I figured the simplest method would be to use the <pathname> route system from Flask. Dash allows me to do this:
import dash
import dash_core_components as dcc
import dash_html_components as html
import plotly.express as px
app = dash.Dash(__name__)
app.layout = html.Div(children=[
dcc.Location(id='url', refresh=False),
html.Div(id='page-content'),
])
#app.callback(dash.dependencies.Output('page-content', 'children'),
[dash.dependencies.Input('url', 'pathname')])
def display_page(pathname):
if pathname == '/':
return html.Div('Please append a pathname to the route')
else:
data = get_data_from_api(int(pathname))
fig_1 = px.line(data, x="time", y="price")
fig_2 = px.line(data, x="time", y="popularity")
return html.Div(children=[
dcc.Graph(id='fig_1',figure=fig_1),
dcc.Graph(id='fig_2',figure=fig_2),
])
if __name__ == '__main__':
app.run_server(debug=True)
But the problem is that the API call takes a minute or two, and it seems to be constantly polling it, such that the request times out and and the graphs never redraw. What I need is something which doesn't auto-refresh, which can run the API call, update the underlying data, and then tell the app to refresh its state.
I did consider a Dash-within-Flask hybrid like this, but it seems excessively complicated for my use-case. Is there a simpler way to do this?
I think you can add a html.Button to your layout.
html.Button('Update', id='update-button')
To your Callback you can add:
#app.callback(dash.dependencies.Output('page-content', 'children'),
[dash.dependencies.Input('url', 'pathname'),
dash.dependencies.Input('update-button', 'n_clicks')])
def display_page(pathname, n_clicks):
....
No need to process the variabel n_clicks in anyway. The Callback is always triggerd.
Cheers
Selenium driver.get (url) wait till full page load. But a scraping page try to load some dead JS script. So my Python script wait for it and doesn't works few minutes. This problem can be on every pages of a site.
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.cortinadecor.com/productos/17/estores-enrollables-screen/estores-screen-corti-3000')
# It try load: https://www.cetelem.es/eCommerceCalculadora/resources/js/eCalculadoraCetelemCombo.js
driver.find_element_by_name('ANCHO').send_keys("100")
How to limit the time wait, block AJAX load of a file, or is other way?
Also I test my script in webdriver.Chrome(), but will use PhantomJS(), or probably Firefox(). So, if some method uses a change in browser settings, then it must be universal.
When Selenium loads a page/url by default it follows a default configuration with pageLoadStrategy set to normal. To make Selenium not to wait for full page load we can configure the pageLoadStrategy. pageLoadStrategy supports 3 different values as follows:
normal (full page load)
eager (interactive)
none
Here is the code block to configure the pageLoadStrategy :
Firefox :
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities().FIREFOX
caps["pageLoadStrategy"] = "normal" # complete
#caps["pageLoadStrategy"] = "eager" # interactive
#caps["pageLoadStrategy"] = "none"
driver = webdriver.Firefox(desired_capabilities=caps, executable_path=r'C:\path\to\geckodriver.exe')
driver.get("http://google.com")
Chrome :
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities().CHROME
caps["pageLoadStrategy"] = "normal" # complete
#caps["pageLoadStrategy"] = "eager" # interactive
#caps["pageLoadStrategy"] = "none"
driver = webdriver.Chrome(desired_capabilities=caps, executable_path=r'C:\path\to\chromedriver.exe')
driver.get("http://google.com")
Note : pageLoadStrategy values normal, eager and none is a requirement as per WebDriver W3C Editor's Draft but pageLoadStrategy value as eager is still a WIP (Work In Progress) within ChromeDriver implementation. You can find a detailed discussion in “Eager” Page Load Strategy workaround for Chromedriver Selenium in Python
#undetected Selenium answer works well but for the chrome, part its not working use the below answer for chrome
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
capa = DesiredCapabilities.CHROME
capa["pageLoadStrategy"] = "none"
browser= webdriver.Chrome(desired_capabilities=capa,executable_path='PATH',options=options)