Funny error using cherrypy wsgi server - django

I am serving my django project using cherrypy's wsgi server. The server serves all static and media files from django project. The code is the following
#create basic django WSGI app
app = WSGIHandler()
path = {'/': app}
#create app that handles static files from static root and put it in path
path[static_url] = StaticFileWsgiApplication(static_root)
#create app that handles /media and put it in path
path[media_url] = StaticFileWsgiApplication(media_root)
#dispatch the applications
dispatcher = wsgiserver.WSGIPathInfoDispatcher(path)
if __name__ == '__main__':
print 'Close window to exit'
#create wsgi cherrypy server
server = wsgiserver.CherryPyWSGIServer(
('0.0.0.0', 8000),
dispatcher,
server_name='cherrypy-django',
numthreads=20
)
try:
server.start()
except KeyboardInterrupt:
server.stop()
and the static file app
class StaticFileWsgiApplication(object):
#init function to get the directory of collectstatic (STATIC_ROOT)
def __init__(self, static_file_root):
self.static_file_root = os.path.normpath(static_file_root)
#Every wsgi app must be callable. It needs 2 arguments
#eviron: wsgi environ path
#start_response:Function creates the response
def __call__(self, environ, start_respone):
def done(status, headers, output):
#usefull for debugging
#returns the output(actual static file)
#also it produces the response using the start_response()
start_respone(status, headers.items())
return output
#get the path_info see PEP 333
path_info = environ['PATH_INFO']
#remove leading '/' from path (URI)
if path_info[0]=='/':
path_info = path_info[1:]
#actual file path in filesystem
file_path = os.path.normpath((os.path.join(self.static_file_root, path_info)))
#prevent escaping out of paths bellow media root (e.g via '..')
if not file_path.startswith(self.static_file_root):
status = '401 UNAUTHORIZED'
headers = {'Content_type':'text/plain'}
output = ['Permission denied. Illegal Path']
return done(status, headers, output)
#Only allow GET and HEAD requests not PUT, POST, DELETE
if not (environ['REQUEST_METHOD'] == 'GET' or environ['REQUEST_METHOD'] == 'HEAD'):
status = '405 METHOD NOT ALLOWED'
headers = {'Content_type':'text/plain'}
output = SimpleResponse(['405:Method not allowed'])
output.status_code = 405
return done(status, headers, output)
if not (os.path.exists(file_path)):
status = "404 NOT FOUND"
headers = {'Content_type':'text\plain'}
output = SimpleResponse(['Page not found %s' %file_path])
output.status_code = 404
return done(status, headers, output)
try:
fp = open(file_path, 'rb')
except IOError:
status = '401 UNAUTHORIZED'
headers = {'Content_type':'text/plain'}
output = SimpleResponse(['Permission denied %s' %file_path])
output.status_code = 401
return done(status, headers, output)
#This is a very simple implementation of conditional GET with
#the Last-Modified header. It makes media files a bit speedier
#because the files are only read off disk for the first request
#(assuming the browser/client supports conditional GET).
#mimetype needs to be ascii not uinicode, as django is all unicode, need to do conversion
mtime = http_date(os.stat(file_path)[stat.ST_MTIME]).encode('ascii','ignore')
if environ.get('HTTP_IF_MODIFIED_SINCE', None) == mtime:
headers = {'Last-Modified':mtime}
status = '304 NOT MODIFIED'
output = SimpleResponse()
output.status_code = 304
else:
status = '200 OK'
mime_type = mimetypes.guess_type(file_path)[0]
if mime_type:
headers = {'Content_type':mime_type}
output = BlockIteratorResponse(fp)
return done(status, headers,output)
I get an error in call about not initialising headers before using it...probably because all headers are inside if's. The error is
UnboundLocalError:Local variable "headers" referenced before assignment
The funny thing is that i get this error only on Mozilla but web works fine, everything is served correctly, and after i sign in to my webpage it doesn't show the error anymore.
Should I initialize headers variable in beginning of call like
headers = {}
And why is it happening only in firefox?

If you want to serve static content using CherryPy, you should use built-in tools -- see Serving Static Content. I guess you need to create a CherryPy app serving static content only and then combine it with your Django app with WSGIPathInfoDispatcher(...).
BUT what you should really consider is using nginx + uWSGI -- see Setting up Django and your web server with uWSGI and nginx.

Related

Why is my hosted Django app showing my username in the URL?

I've written a Django app which I'm trying to get set up on shared web hosting (A2). It's working, except that when I go to:
http://example.com/terms/
the URL changes in the browser bar to:
http://example.com/home/myusername/myappfolder/myappname/terms/
showing the full path to where my app is on disk.
This doesn't happen with static files - e.g. http://example.com.com/static/image.png works normally.
The app is running in a virtual environment. I'm using python 3.6.8 and Django 2.1.4.
I followed these instructions to set up my app, which include setting up this passenger.wsgi file, that looks like this:
import myapp.wsgi
SCRIPT_NAME = '/home/username/myapp'
class PassengerPathInfoFix(object):
"""
Sets PATH_INFO from REQUEST_URI because Passenger doesn't provide it.
"""
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
from urllib.parse import unquote
environ['SCRIPT_NAME'] = SCRIPT_NAME
request_uri = unquote(environ['REQUEST_URI'])
script_name = unquote(environ.get('SCRIPT_NAME', ''))
offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
return self.app(environ, start_response)
application = myapp.wsgi.application
application = PassengerPathInfoFix(application)
I'd be grateful for any pointers as to where to look to solve this.
Got it working!
In my modified passenger_wsgi.py, I changed the line
SCRIPT_NAME = os.getcwd()
to
SCRIPT_NAME = ''
One thing I should point out is that the absolute path was getting inserted on redirects - so if I visited
http://example.com/terms
it would redirect to
http://example.com/terms/
and insert the path in the URL.
As you're debugging I recommend disabling the cache, as that threw me for several loops when changes I made didn't seem to take effect.
Thanks to this question for getting me on the right track.

django view didn't return an HttpResponse object when retrieving file

I've run into an issue when creating a custom view for processing requests for media file. I've customized it to be able to do custom permissions checks and overrides.
url(r'^media/(?P<path>.*)$', views.document_view, {'document_root': settings.MEDIA_ROOT}),
#login_required
def document_view(request, path, document_root):
name = os.path.join(settings.MEDIA_ROOT, path)
if not os.path.isfile(name):
raise Http404("File not found.")
if request.user.is_staff:
print "staff"
serve_document(request, path)
else:
print "FORBIDDEN"
raise PermissionDenied
def serve_document(rcvd_request, path):
print "Send " + path
# set PRIVATE_MEDIA_USE_XSENDFILE in your deployment-specific settings file
# should be false for development, true when your webserver supports xsendfile
# This is currently zero
if settings.PRIVATE_MEDIA_USE_XSENDFILE:
response = HttpResponse(mimetype='application/force-download')
response['X-Accel-Redirect'] = filename # Nginx
response['X-Sendfile'] = filename # Apache 2 with mod-xsendfile
del response['Content-Type'] # let webserver regenerate this
return response
else:
print "Fallback serve"
# fallback method
return serve(rcvd_request, path, settings.MEDIA_ROOT)
return serve(request, path, path, settings.MEDIA_ROOT)
When I click on a link that has the file linked to, I get
The view portal.views.dashboard.document_view didn't return an HttpResponse object. It returned None instead.
This doesn't make much sense since I know the serve gets called with the right path, the file exists (in fact a previous version without staff check worked in downloading the files.
I've seen people have issues due to indentation and function exiting with None as default, but this isn't the case.
The problem is in the following line of your document_view:
serve_document(request, path)
Django view expects an HttpResponse returned. But this line does not return anything. The response returned by serve_document is not assigned to anything nor returned.
Thus, you should change it to:
return serve_document(request, path)
or, make it more explicit:
http_response = serve_document(request, path)
return http_response

Simple HTTP client and server in python but client making atleast 50 request simultaneously with different IP address for each request

I want to know how , HTTP CLIENT making 100 REQUEST to server at same time but with different IP address for each request or with range of IP ADDRESSES, code in python or using a CURL function in code by configuring different IP Addresses or range of IP Addresses
Please help me editing the code to make several request to sever using different IP address requesting for a Html or web page on server and reading it contents.
Luckily, python provide us an HTTP server module, it's called BaseHTTPServer. We use two classes from this module, namely BaseHTTPRequestHandler and HTTPServer. We also need to use os module to access file in your system.
First, we need to import those modules
#!/usr/bin/env python
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import os
Next, write a custom class namely KodeFunHTTPRequestHandler that implement BaseHTTPRequestHandler.
#Create custom HTTPRequestHandler class
class KodeFunHTTPRequestHandler(BaseHTTPRequestHandler):
I just implement GET command from HTTP. To do that, we need to override do_GET() function. Our objective is to request an HTML file from server using our client (explained int the next step). Script inside do_GET() function is written to handle our objective
#handle GET command
def do_GET(self):
rootdir = 'c:/xampp/htdocs/' #file location
try:
if self.path.endswith('.html'):
f = open(rootdir + self.path) #open requested file
#send code 200 response
self.send_response(200)
#send header first
self.send_header('Content-type','text-html')
self.end_headers()
#send file content to client
self.wfile.write(f.read())
f.close()
return
except IOError:
self.send_error(404, 'file not found')
Finally, add following script to run the server
def run():
print('http server is starting...')
#ip and port of servr
#by default http server port is 80
server_address = ('127.0.0.1', 80)
httpd = HTTPServer(server_address, KodeFunHTTPRequestHandler)
print('http server is running...')
httpd.serve_forever()
if __name__ == '__main__':
run()
Save it and name it whatever you want (e.g, kodefun-httpserver.py). Here is the complete script of http server:
kodefun-httpserver.py
#!/usr/bin/env python
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import os
#Create custom HTTPRequestHandler class
class KodeFunHTTPRequestHandler(BaseHTTPRequestHandler):
#handle GET command
def do_GET(self):
rootdir = 'c:/xampp/htdocs/' #file location
try:
if self.path.endswith('.html'):
f = open(rootdir + self.path) #open requested file
#send code 200 response
self.send_response(200)
#send header first
self.send_header('Content-type','text-html')
self.end_headers()
#send file content to client
self.wfile.write(f.read())
f.close()
return
except IOError:
self.send_error(404, 'file not found')
def run():
print('http server is starting...')
#ip and port of servr
#by default http server port is 80
server_address = ('127.0.0.1', 80)
httpd = HTTPServer(server_address, KodeFunHTTPRequestHandler)
print('http server is running...')
httpd.serve_forever()
if __name__ == '__main__':
run()
Write a simple HTTP client
To check if our server is working fine, we need an HTTP client. You can use your web browser to do this. But, It'll be cool if you make your own client. To accomplish this, let's make another script, let's name it kodefun-httpclient.py. Copy following script to your file:
kodefun-httpclient.py
#!/usr/bin/env python (CLIENT CODE)
import httplib
import sys
#get http server ip
http_server = sys.argv[1]
#create a connection
conn = httplib.HTTPConnection(http_server)
while 1:
cmd = raw_input('input command (ex. GET index.html): ')
cmd = cmd.split()
if cmd[0] == 'exit': #tipe exit to end it
break
#request command to server
conn.request(cmd[0], cmd[1])
#get response from server
rsp = conn.getresponse()
#print server response and data
print(rsp.status, rsp.reason)
data_received = rsp.read()
print(data_received)
conn.close
Test using GET command

Django makes too many processes (on fastCGI)

I have a Django app with fastCGI.
The following view starts lots of proccesses named "/usr/bin/python index.fcgi". They suddenly appear and just stay there forever.
def serveMusicFile(request,filename):
from django.conf import settings
file = open('/my/private/file/location/'+filename)
response = HttpResponse(file.read(), mimetype="audio/mpeg")
response['Accept-Ranges'] = 'bytes'
response['Content-Length'] = os.path.getsize('/my/private/file/location/'+filename)
response['Content-Disposition'] = 'filename='+filename
return response
So, why does Django/fastCGI start this proccess? How can I stop them being pilled?

Django with mod_XSENDFILE unable to download complete file

Attached is the code which downloads a file from browser using django 1.3 and Apache 2.2 with mod_xsendfile
#login_required
def sendfile(request, productid):
path = settings.RESOURCES_DIR
filepath = os.path.join('C:/workspace/y/src/y/media/audio/','sleep_away.mp3')
print "filepath",filepath
filename = 'sleep_away.mp3' # Select your file here.
print "Within sendfile size", os.path.getsize(filepath)
wrapper = FileWrapper(open(filepath,'r'))
content_type = mimetypes.guess_type(filename)[0]
response = HttpResponse(wrapper, content_type = content_type)
print "Within wrapper"
from django.utils.encoding import smart_str
response['X-Sendfile'] = smart_str(filepath)
response['Content-Length'] = os.path.getsize(filepath)
from django.utils.encoding import smart_str
response['Content-Disposition'] = 'attachment; filename=%s/' % smart_str(filename)
return response
The console shows the following filesize which is the right size
Within sendfile size 4842585
But when I download/save the file it shows 107 KB...i.e 109,787 bytes.Where am I going wrong. Why isnt it downloading the complete file?
I consider your new to django or python. Try to put the import statements at the beginning of the method. Once imported it can be used through the method no need import every time you use. In windows you should use "rb" (read binary) to serve anything other than text files. Try not to use variable names that might conflict with method names or other keywords of the language. Your method should be like this
#login_required
def sendfile(request, productid):
from django.utils.encoding import smart_str
##set path and filename
resource_path = settings.RESOURCES_DIR # resource dir ie /workspace/y/src/y/media
filename = "sleep_away.mp3" #file to be served
##add it to os.path
filepath = os.path.join(resource_path,"audio",filename)
print "complete file path: ", filepath
##filewrapper to server in size of 8kb each until whole file is served
file_wrapper = FileWrapper(file(filepath,'rb')) ##windows needs rb (read binary) for non text files
##get file mimetype
file_mimetype = mimetypes.guess_type(filepath)
##create response with file_mimetype and file_wrapper
response = HttpResponse(content_type=file_mimetype, file_wrapper)
##set X-sendfile header with filepath
response['X-Sendfile'] = filepath ##no need for smart_str here.
##get filesize
print "sendfile size", os.stat(filepath).st_size
response['Content-Length'] = os.stat(filepath).st_size ##set content length
response['Content-Disposition'] = 'attachment; filename=%s/' % smart_str(filename) ##set disposition
return response ## all done, hurray!! return response :)
Hope that helps
You could have a look at the django-private-files project. Haven't tested it myself, but it looks promissing.
link to the docs --> http://readthedocs.org/docs/django-private-files/en/latest/usage.html
cheers