I've tried using both CherryPy 3.5 and Tornado 4 to host my Flask WSGI application. Regardless of the WSGI server, every 2s my browser tries to make a CONNECT/GET request to myserver:9485.
Chrome's network view looks like this:
Firefox's network view looks like this:
Finally, IE's network view doesn't show anything, but I do see this in the console:
Everything about the site works fine, but it kills me knowing this is happening in the background. How do I stop the madness?
Oddly enough, I don't see these messages when I run the code shown below on my development machine.
Additional details:
Python 2.7.6, Flask 0.10.1, Angular 1.2.19
CherryPy code:
static_handler = tools.staticdir.handler(section='/', dir=app.static_folder)
d = wsgiserver.WSGIPathInfoDispatcher({'/': app, '/static': static_handler})
server = wsgiserver.CherryPyWSGIServer(('127.0.0.1', 9000), d)
server.start()
Probably unrelated, but I am using an EventSource polyfill on one of my pages (I see these messages regardless if I hit that page or not).
Related
I have a baffling issue with cookie handling in a Blazor server app (.NET Core 6) using openid (Keycloak). Actually, more than a couple which are may or may not linked. It’s a typical (?) reverse proxy architecture:
A central nginx receives queries for services like Jenkins, JypyterHub, SonarQube, Discourse etc. These are mapped through aliases in internal IPs where the nginx can access them. This nginx intercepts URL like: https://hub.domain.eu
A reverse proxy which resolves to https://dsc.domain.eu. This forwards request to a Blazor app running in Kestrel in port 5001. Both Kestrel and nginx under SSL – required to get the websockets working.
Some required background: the Blazor app is essentially a ‘hub’ where its various razor pages ‘host’ in iframe-like the above mentioned services. How it works: When the user asks for the root path (https://hub.domain.eu) it opens the root page of the Blazor app (/).
The nav menu contains the links to razor pages which contain the iframes for the abovementioned services. For example:
The relative path is intercepted by the ‘central’ nginx which loads Jenkins. Everything is under the same Keycloak OpenID server. Note that everything works fine without the Blazor app.
Scenarios that cause the same problem
Assume the user logins in my app using the login page of Keycloak (NOT the REST API) through redirection. Then proceeds to link and he is indeed logged in as well. The controls in the App change accordingly to indicate that the user is indeed authenticated. If you close the tab and open a new one, the Blazor app will act as if it’s not logged in while the other services (e.g Jenkins) will show the logged in user from before. When you press the Login link, you’ll be greeted with a 502 nginx error. If you clean the cookies from browser (or in private / stealth mode) everything works again. Or of you just log off e.g. from Jenkins.
Assume that the user is now in a service such as Jenkins, SonarQube, etc. if you press F5 now you have two problems: you get a 404 Error but only on SOME services such as Sonarcube but not in others. This is a side problem for another post. The thing is that Blazor app appears not logged in again by pressing Back / Refresh
The critical part of Program.cs looks like the following:
This class handles the login / logoff:
Side notes:
SaveTokens = false still causes large header errors and results in empty token (shown in the above code with the Warning: Token received was null). I’m still able to obtain user details though from httpContext.
No errors show up in the reverse proxy error.log and in Kestrel (all deployed in Linux)
MOST important: if I copy-paste the failed login link (the one that produced the 502 error) to a "clean" browser, it works fine.
There are lots of properties affecting the OpenID connect, it could also be an nginx issue but I’ve run out of ideas the last five days. The nginx config has been accommodated for large headers and websockets.
Any clues as to where I should at least focus my research to track the error??
The 502 error shows an error at NGINX's side. The reverse proxy had proper configuration but as it turned out, not the front one. Once we set the header size to suggested size, everything played out.
I have a very simple flask app that has been working for years, but last week requests from the built app return a 500, and from the Flask side, I can't even see the request. I am not seeing an OPTIONS request.
The below lines worked previously to keep CORS happy.
#app.after_request
def after_request(response):
response.headers.add('Access-Control-Allow-Origin', '*')
response.headers.add('Access-Control-Allow-Headers', 'Content-Type,Authorization,Auth-Token')
response.headers.add('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE')
return response
I have tried in a few browsers and all of them fail to successfully make any requests. Since the server doesn't even acknowledge the request has been made, I am not sure where to trouble shoot. I did confirm the app returns data as expected when I use Postman to make the request, as well as confirming that if I use the app locally (gulp serve on my computer) that the requests are successful. I have to believe its CORS, but what might I have to add / do to get the browser to be happy? Thanks.
The solution to my problem was that chrome started to "restrict the ability of websites to communicate with devices on the local network"
Communicating from Chrome 94+ with LAN devices that do not support HTTPS from a web app
I have a Flask site deployed to IIS via wfastcgi configuration.
When I use chrome or firefox developer tools to analyse the loading time of the homepage, I find many seconds (ranging from 6 to 10 in average) as waiting time to receive the first byte.
It was even 30 seconds before, but then I "optimized" my python code to avoid any db sql operation at loading time. Then I've followed the hints of this blog of nspointers, and now from the taskbar of the server I see the w3wp.exe for my app pool identity
w3wp.exe – It is the IIS worker process for the application pool
staying up and running even during idle time. But that is not true for the other
python.exe – The main FastCGI process for the Django or flask
applications.
and I'm not sure if this is a problem and just in case what I am supposed to do, aside from the step 4 described in the mentioned post.
Now in the “Edit FastCGI Application” dialog under “Process Model”
edit the “Idle Timeout” and set it to 2592000 which is the max value
for that field in seconds
I've also looked at the log written by the Flask app and compared it to the log written by IIS and this is the most important point in making me believe that the issue is in the wfastcgi part, before the execution of the python code.
Because I see that the time-taken of the IIS log matches with the client time reported by chrome or firefox as TTFB and the log written by python at the start of the execution is logged at almost the same time of the time written by IIS, that
corresponds to the time that the request finished
(as I thought indeed and as I find it's confirmed by this answer)
So in conclusion, based on what I tried and what I understand, I suspect that IIS is "wasting" many seconds to "prepare" the python wfascgi command, before actually starting to execute my app code to produce a response for the web request. It is really too much in my opinion, since other applications I've developed (for example in F# WebSharper) under IIS without this mechanism of wfastcgi load immediately in the browser and the difference in the response time between them and the python Flask app is quite noticeable. Is there anything else I can do to improve the response time?
Ok, now I have the proof I was searching and I know where the server is actually spending the time.
So I've researched a bit about the wfastcgi and finally opened the script itself under venv\Lib\site-packages.
Skimming over the 900 lines, you can spot the relevant log part:
def log(txt):
"""Logs messages to a log file if WSGI_LOG env var is defined."""
if APPINSIGHT_CLIENT:
try:
APPINSIGHT_CLIENT.track_event(txt)
except:
pass
log_file = os.environ.get('WSGI_LOG')
if log_file:
with open(log_file, 'a+', encoding='utf-8') as f:
txt = txt.replace('\r\n', '\n')
f.write('%s: %s%s' % (datetime.datetime.now(), txt, '' if txt.endswith('\n') else '\n'))
Now, well knowing how to set the environment variables, I defined a specific WSGI_LOG path, and here we go, now I see those 5 seconds TTFB from chrome (as well as the same 5 seconds from IIS log with time 11:23:26 and time-taken 5312) in the wfastcgi.py log.
2021-02-01 12:23:21.452634: wfastcgi.py 3.0.0 initializing
2021-02-01 12:23:26.624620: wfastcgi.py 3.0.0 started
So, of course, wfastcgi.py is the script one would possibly try to optimize...
BTW, after digging into it, that time is due to importing the main flask app
handler = __import__(module_name, fromlist=[name_list[0][0]])
What remains to be verified is the behavior of rerunning the process (and the import of the main flask module, that is time consuming) for each request.
In conclusion, I guess it is a BUG, but I have solved it by deleting the "monitoring changes to file" FastCGI settings as per the screenshot below.
The response time is under a second.
I have a different answer to you by suggesting you try to switch over to HTTP Platform Handler for your IIS fronted Flask app.
Config Reference
This is also the recommended option by Microsoft:
Your app's web.config file instructs the IIS (7+) web server running on Windows about how it should handle Python requests through either HttpPlatform (recommended) or FastCGI.
https://learn.microsoft.com/en-us/visualstudio/python/configure-web-apps-for-iis-windows?view=vs-2019
Example config can be:
<configuration>
<system.webServer>
<handlers>
<add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified"/>
</handlers>
<httpPlatform processPath="c:\inetpub\wwwroot\run.cmd"
arguments="%HTTP_PLATFORM_PORT%"
stdoutLogEnabled="true"
stdoutLogFile="c:\inetput\Logs\logfiles\python_app.log"
processPerApplication="2"
startupTimeLimit="60"
requestTimeout="00:01:05"
forwardWindowsAuthToken="True"
>
<environmentVariables>
<environmentVariable name="FLASK_RUN_PORT" value="%HTTP_PLATFORM_PORT%" />
</environmentVariables>
</httpPlatform>
</system.webServer>
</configuration>
With run.cmd being something like
cd %~dp0
.venv\scripts\waitress-serve.exe --host=127.0.0.1 --port=%1 myapp:wsgifunc
Note that the HTTP Platform handler will dynamically set on a port and passing that into the python process via the FLASK_RUN_PORT env var which flask will automatically take as a port configuration.
Security notes:
Make sure you bind your flask app to localhost only, so it's not visible directly from the outside - especially if you are using authentication via IIS
In the above example the forwardWindowsAuthToken is being set which then can be used to rely on Windows Integrated authentication done by IIS then the token passed over to Python and you can get the authenticated user name from Python. I have documented that here. I actually use that for single-sign on with Kerberos and AD group based authorization, so it works really nice.
Example to only listen on localhost / loopback adapter to avoid external requests hitting the python app directly. In case you want all requests to go via IIS.
if __name__ == "__main__":
app.run(host=127.0.0.1)
I am writing a single page app with React for educational purposes. My React-Router v4 BrowserRouter handles client side routing correctly on CodeSandbox but not locally. In this case, the local server is the webstorm built-in devserver. HashRouter works locally but BrowserRouter does not.
Functioning properly: https://codesandbox.io/s/j71nwp9469
You are likely serving your app on the built-in webserver (localhost:63342), right? Internal web server returns 404 when using 'absolute' URLs (the ones starting with slash) as it serves files from localhost:port/project_name and not from localhost:port. That's why you have to make sure to change all URLs from absolute to the relative ones.
There is no way to set up the internal webserver to use project root as server document root. But you can configure it to use URLs like http://<host name>:<port> where the 'host name' is a name specified in hosts file, like 127.0.0.1 myhostName. See https://youtrack.jetbrains.com/issue/WEB-8988#comment=27-577559.
The solution was to understand how push state routing and the history API works. It is necessary to proxy requests through the index page when serving Single Page Applications that utilize the HTML5 History API.
The Webstorm dev server is not expected to include this feature, therefore the mention of Webstorm in this thread was a mistake.
There are multiple libraries of < 20 lines which do this for us, or it can easily be hand coded.
I am trying to write unittests for my Flask API endpoints. I want the test cases to connect to the dev server which is running on a different port 5555.
Here is what I am doing in setUp() to make a test_client.
import flask_app
flask_app.app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+mysqldb://root:#localhost/mvp_test_db'
flask_app.app.config['TESTING'] = True
flask_app.app.config['SERVER_NAME'] = '192.168.2.2' //the IP of the dev server
flask_app.app.config['SERVER_PORT'] = 5555
self.app_client = flask_app.app.test_client()
Then when I make a request using app_client like -
r = self.app_client.post('/API/v1/dummy_api', data = {'user_id' : 1})
I get a 404 when I print r and the request never comes to the dev server (no logs printed). I am not able to inspect the URL to which the connection is being attempted above. Any ideas?
It (app.test_client) does not send requests through network interfaces. All requests are simulated. These are processed inside werkzeug routing system. To process "/API/v1/dummy_api" url you need register a view for. If it is registered, connect it in the import section. Application settings and test settings almost always have almost equal settings.