How to keep Bottle routes handling images from gridfs in mod_wsgi? - python-2.7

Desired Behaviour
Allow Bottle routes to handle requests for images in the format:
<img src="/gridfs/img/my_image.jpg">
On the server, this is being handled by a Bottle route and returning the image as expected:
#route('/gridfs/img/<filename>')
def server_gridfs_img(filename):
# get the image
# return it
Current Behaviour
In a local environment however, with mod_wsgi set up, I am getting the error:
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator at webmaster#localhost to inform them of the time this error occurred, and the actions you performed just before this error.
More information about this error may be available in the server error log.
Apache/2.4.7 (Ubuntu) Server at localhost Port 80
What I've Tried
This is my local mod_wsgi configuration:
WSGIPythonHome /var/www/html/ENV
WSGIPythonPath /var/www/html:/var/www/html/ENV/lib/python2.7/site-packages:/var/www/html/wsgi
<VirtualHost *:80>
# for all content in static folder - css, js, img, fonts
Alias /static/ /var/www/html/wsgi/static/
# for rockmongo administration
Alias /rockmongo /var/www/html/rockmongo
<Directory /var/www/html/rockmongo>
Order deny,allow
Allow from all
</Directory>
ServerAdmin webmaster#localhost
DocumentRoot /var/www/html
WSGIScriptAlias / /var/www/html/wsgi/application
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
This is my /var/www/html/wsgi/application:
from mybottleapp import application
Directory
Question
The rest of the routes in my application seem to be being honored.
It doesn't seem that Bottle is handling the requests for images like /gridfs/img/my_image.jpg though.
How can I make that happen?
UPDATE
Looked at Apache error log and saw this:
CorruptGridFile: no chunk #1
I am troubleshooting that now...

I posted the solution to the updated situation mentioned above here:
How to mongodump from OpenShift and mongorestore locally on MongoDB 2.4.9?
The data corruption was caused from exporting and importing with RockMongo - apparently sometimes there can be issues with this approach and larger data sets.

Related

Apache throws ERROR 500: Internal Server Error when GET from localhost/internal network

I have a production server with apache and django installed using mod_wsgi.
The django application has a REST API that serves some info when a GET request is sent.
This has always worked fine on the develop server, were we ran django using manage.py in a screen. Now we created a production server with apache running django but this API returns Error 500 when running wget from localhost or other machines in the same network (using 192.168.X.X IP).
Here's the output from wget:
~$ wget localhost:80/someinfo
--2020-04-02 16:26:59-- http://localhost/someinfo
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2020-04-02 16:26:59 ERROR 500: Internal Server Error.
It seems that the connection succeeds, so I guess it's not an apache problem. The error comes from the API response.
The error in apache error.log looks like this:
127.0.0.1 - - [02/Apr/2020:14:24:36 +0000] "GET /someinfo HTTP/1.1" 500 799 "-" "Wget/1.19.4 (linux-gnu)"
question: what is the number after 500? Sometimes is 799 and other times is 803.
But if the request is done using the public IP of the server from outside (i.e. from the browser) the API works fine and I see the correct information.
I already checked django's allowed hosts and it was accepting localhost, and the 192.168.X.X IP of the other machine. In the end I left django's settings.py like this:
#ALLOWED_HOSTS = ['localhost', '127.0.0.1', '192.168.1.101']
ALLOWED_HOSTS = ['*']
Note: 192.168.1.101 is the machine that tries to make the GET request.
The final goal of all this is to be able to make a GET request from a python script running in that machine (which already works if django runs via manage.py).
My apache.conf:
<VirtualHost *:80>
ServerAdmin webmaster#localhost
#DocumentRoot /var/www/html
Alias /static /home/myuser/myproject/django/static_root
<Directory /home/myuser/myproject/django/static_root>
Require all granted
</Directory>
<Directory /home/myuser/myproject/django/myproject_django>
<Files wsgi.py>
Require all granted
</Files>
</Directory>
WSGIDaemonProcess myproject python-home=/home/myuser/env python-path=/home/myuser/myproject/django
WSGIProcessGroup myproject
WSGIScriptAlias / /home/myuser/myproject/django/myproject_django/wsgi.py
</VirtualHost>
I tried running django via manage.py and the wget from localhost works just fine. The problem only appears when django is ran by apache.
I also tried the solution given in this post, but changing the line does not fix the error.
I have some doubts concerning this error:
how does apache run django?
does restarting apache2 service also restart django? (thus, reading again the settings.py)
Is there any other django settings file rather than the one I'm editing?
how can I see django logs? I don't have the console now so I can't see real time prints.
I appreciate a lot any help.
I finally managed to solve it myself.
It turns out wsgi handles requests from localhost or external IPs as different instance groups. So all I had to do is put
WSGIApplicationGroup %{GLOBAL}
in /etc/apache2/sites-available/000-default.conf

Timeout when reading response headers from daemon process even after setting WSGIApplication group to Global

I am hosting a Django based webpage locally using Apache. However, I am getting the following error :
Timeout when reading response headers from daemon process 'office':var/www/office/office/wsgi.py.
I tried adding the line WSGIApplicationGroup %{GLOBAL} to the conf file, but still getting the same error.
This is my .conf file.
WSGIPythonPath /var/www/office
ServerName office.org
LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so
WSGIApplicationGroup %{GLOBAL}
<VirtualHost 0.0.0.0:80>
ServerAlias www.office.org
DocumentRoot /var/www/example.com/public_html
<Directory /var/www/example.com>
Require all granted
</Directory>
<Directory /var/www/office/office>
<Files wsgi.py>
Require all granted
</Files>
</Directory>
WSGIDaemonProcess office python-home=/var/www/venv python-path=/var/www/office
WSGIProcessGroup office
WSGIScriptAlias /verify /var/www/office/office/wsgi.py process-group=office
ErrorLog /var/www/logs/error.log
CustomLog /var/www/logs/custom.log combined
</VirtualHost>
This is wsgi.py file:
"""
WSGI config for office project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/2.0/howto/deployment/wsgi/
"""
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "office.settings")
application = get_wsgi_application()
I've got the same problem and after a lot of struggling and trying different things, this tutorial helped me to solve the problems.
I created and successfully run the demo project. The problem for me were the paths in the config file and also using the server python version instead of creating a virtual environment for the project.
I hope it will help you
I have got the same problem "Timeout when reading response headers" in Flask framework
I resolved it by adding TimeOut 600 in httpd.conf/app.conf
Reference : https://ubiq.co/tech-blog/increase-request-timeout-apache/
if this happened on a working server, You can try to force renew Your ssl certificate.
the reason could be found in apache error logs:
Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
helped for me.

apache + django + mod_wsgi conversion to https keeps going back to http

I've had a django (satchmo) site using Apache and mod_wsgi running fine for a couple of years now. Until now it has only served http, and I'm trying to convert the entire site to https. I have a signed ssl certificate which I believe is fine.
I have adapted my Apache configuration according to my understanding of the docs.
When I try to connect to the site using https, the browser connects fine on port 443, and the Apache server responds with the correct ssl certificate followed by the TLS key exchange etc (according to what I see in Wire Shark). At that point everything looks fine and there are no errors. But..
Once the ssl connection is established, the browser then initiates a "GET / HTTP/1.1" in a new TCP connection to port 80 (i.e. http). It's like it knew nothing about the https connection already in place.
Is it possible that django is at fault? I have not changed the django configuration at all, as I was under the impression that only Apache needs to know about it? (I don't use nginx - Apache handles all of the content.)
I can't "see" what is going on in the ssl conversation, but presumably django is telling the browser client to connect on port 80 somehow? Is that possible?
To make things simple, I now have a plain index.html page when you connect to http, and I've moved all of the django & mod_wsgi to port 443. If I connect straight to the http address, I get the simple index page, no problem.
When I try to connect to the https address, the browser effectively gets redirected to the index.html page. (I don't have any Redirect or Rewrite commands in Apache though.)
Here is my Apache configuration:
<VirtualHost *:80>
ServerName demo.pasta.co.za
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
DocumentRoot /var/www/http_site
</VirtualHost>
<VirtualHost *:443>
SSLEngine on
SSLProtocol all -SSLv2
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM
SSLCertificateFile /etc/ssl/private/pasta.co.za.crt
SSLCertificateKeyFile /etc/ssl/private/pasta.co.za.key
SSLCertificateChainFile /etc/ssl/private/root_bundle.crt
ServerName demo.pasta.co.za
Alias /favicon.ico /usr/local/django/pasta/static/favicon.ico
Alias /robots.txt /usr/local/django/pasta/static/robots.txt
AliasMatch ^/([^/]*\.css) /usr/local/django/pasta/store/static/$1
WSGIDaemonProcess demo.pasta.co.za processes=2 threads=25 display-name=%{GROUP}
WSGIProcessGroup demo.pasta.co.za
WSGIScriptAlias / /usr/local/django/pasta/apache/django.wsgi
<Directory /usr/local/django/pasta/apache>
Order allow,deny
Allow from all
</Directory>
Alias /static/admin/ /usr/share/pyshared/django/contrib/admin/static/admin/
Alias /static/images/ /usr/local/django/pasta/store/static/images/
Alias /static/ /usr/local/django/pasta/store/static/
Alias /media/ /usr/local/django/pasta/store/static/
<Directory /usr/local/django/pasta/store/static>
Order deny,allow
Options -Indexes
Allow from all
</Directory>
<Directory /usr/share/pyshared/django/contrib/admin/static/admin>
Order deny,allow
Allow from all
</Directory>
</VirtualHost>
And here is my mod_wsgi file:
import os, sys
sys.path.insert (0,"/usr/local/django/pasta/store")
import settings
import django.core.management
django.core.management.setup_environ(settings)
utility = django.core.management.ManagementUtility()
command = utility.fetch_command('runserver')
command.validate()
import django.conf
import django.utils
django.utils.translation.activate(django.conf.settings.LANGUAGE_CODE)
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
There are dozens of examples on SO where people accomplish what I'm trying to do with more or less what I have above.
What am I missing? It feels like I've left out something really obvious.
I have Debian stable running django v1.4.5 python v2.7.3 and apache v2.2.22 with mod-wsgi v3.3-4.
Many thanks!
Accepted answer for satchmo nginx redirect to https then to http and back by mipadi ...
Satchmo includes a piece of middleware called satchmo_store.shop.SSLMiddleware.SSLRedirect, which automatically does redirecting to SSL/non-SSL portions of the site. You have to set up URLs to be served via SSL if you want them to be served via SSL, otherwise the middleware redirects to a non-SSL page. From the docs:
This middleware answers the problem of redirecting to (and from) a SSL secured path by stating what paths should be secured in urls.py file. To secure a path, add the additional view_kwarg 'SSL':True to the view_kwargs.
For example
urlpatterns = patterns('some_site.some_app.views',
(r'^test/secure/$','test_secure',{'SSL':True}),
)
All paths where 'SSL':False or where the kwarg of 'SSL' is not specified are routed to an unsecure path.
For example
urlpatterns = patterns('some_site.some_app.views',
(r'^test/unsecure1/$','test_unsecure',{'SSL':False}),
(r'^test/unsecure2/$','test_unsecure'),
)
In your case, since you're serving the entire site via SSL, you can probably just disable that middleware in your settings.py file.
(From my experiance you need to change quite a few urlpatterns in a few files.)

Deploy Django on Apache virtual host: 404

I know there are a lot of articles talking about this issue, but I keep getting a 404 error when deploying a Django site through Apache virtual host. Here is my .conf file:
Listen 8001
<VirtualHost *:8001>
ServerName www.myhostname.com/basic
ServerAdmin caisj#example.com
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
DocumentRoot /home/browser/BASIC/basic
Alias /static /home/browser/BASIC/basic/static
Alias /media /home/browser/BASIC/_py/lib/python2.6/site-packages/Django-1.4-py2.6.egg/django/contrib/admin/media
WSGIScriptAlias /basic /home/browser/BASIC/basic/basic.wsgi
ErrorLog /home/browser/BASIC/basic/log/basic.error.log
LogLevel info
CustomLog /home/browser/BASIC/basic/log/basic.access.log combined
</VirtualHost>
I have tried different combinations of the "ServerName" and other parameters, but when visiting www.myhostname.com/basic, I keep getting a 404 error. The Apache restarts successfully and the log file contains no clues.
Or could anybody help to tell where to find errors? Thanks.
You have set this up to listen on port 8001 and rooted at /basic, so you need to access www.myhostname.com:8001/basic.
Note that you shouldn't set DocumentRoot to the location of your Django files. It isn't necessary to serve a Django app, and it could potentially be a security risk - a misconfiguration could mean that your code files, including any db passwords, might be served.

Django + mod_wsgi: Can someone advise me on my setup and rewrite rules

This is my first time deploying Django to a recently acquired Linode server and I'm curious if someone can look over my deployment and help me fix some nagging issues and advise me whether i'm doing things incorrectly.
Directory Structure
home\
-public\
-example.com\
-public\
-.htaccess
-index.html
-log\
-application\
-mysite\
-mysite\
-manage.py
-static\
-myapp\
-logs\
How is this for deployment structure for Django?
Incorrect URL Naming
I've hosted the Django application called 'myapp' on my domain 'example.com'. Following the instructions on the Django website I've made it so that the urls.py for the app must begin with '/myapp'. This has resulted in the domain for the app becoming 'example.com/myapp'.
How can I set it so that example.com is simply the Django app I've written?
I'd like to simply navigate to example.com and it load my app instead of example.com/myapp.
Even weirder is that I would've thought that example.com would load my index.html file however it tries to find a URL mapping for Django instead...
Django Log File Writing Permissions
Whenever I SSH onto my machine to either 'syncdb' or 'collectstatic', the logging module creates the log file I've named in my settings.py file. This causes problems for me because I am the owner of the file and apache2 (www-data) cannot write to it. It's just annoying having to manually delete the log file after every command before I restart the apache server.
Here is my /etc/apache2/sites-available/example.com file:
# domain: example.com
# public: /home/setheron/public/example.com/
WSGIPythonPath /home/setheron/public/example.com/applications/mysite:/home/setheron/env/lib/python2.7/site-packages
<VirtualHost *:80>
# Admin email, Server Name (domain name), and any aliases
ServerAdmin setheron#setheron.com
ServerName www.example.example.com
ServerAlias example.com
WSGIScriptAlias / /home/setheron/public/example.com/applications/mysite/mysite/wsgi.py
Alias /static/ /home/setheron/public/example.com/applications/mysite/static/
<Directory /home/setheron/public/example.com/applications/mysite/static/>
Order deny,allow
Allow from all
</Directory>
<Directory /home/setheron/public/example.com/applications/mysite/mysite>
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>
# Index file and Document Root (where the public files are located)
DirectoryIndex index.html index.php
DocumentRoot /home/setheron/public/example.com/public
# Log file locations
LogLevel warn
ErrorLog /home/setheron/public/example.com/log/error.log
CustomLog /home/setheron/public/example.com/log/access.log combined
</VirtualHost>
If you want Django serving the entire site, get rid of your public directory, indexes and whatnot. Other than /static, you should only need your WSGIScriptAlias directive. Fix the urls.py to say that your site should be coming from /, rather than /myapp.