Django 1.4 form posting forcing method to GET - django

When I submit a form that has method="POST" the view that handles it is getting method GET, and I can't seem to get it to be a POST, which is necessary for file uploads. Short of just canning my existing (was working a week ago, before upgrade to 1.4) and trying to use the form upload the sample from https://docs.djangoproject.com/en/dev/topics/http/file-uploads/, can you see what is going wrong here? I tried a fresh session using Chrome Incognito with the same results. When I don't include method="POST" it submits with method GET and the fields are included in the request.GET dictionary in the email. However, when I change to POST/post, it remains method GET and the fields are not in the request.POST or the request.GET dictionaries. Is there a bug in Django 1.4 that is causing this behavior? Is there something I'm missing? I tried to make a simple example to rule out all extraneous potential variables.
Simple test scenario:
urls.py:
url(r'^upload_test/+$', 'views.presentation.UploadTest'),
url(r'^uploaded_test/+$', 'views.presentation.UploadedTest'),
views/presentation.py:
def Notify(subject, body):
"""
Send an email to <to_email>, from <from_email> with the
subject and body supplied.
"""
send_mail(subject, body, '<from_email>', ['<to_email>'], fail_silently=False)
def UploadTest(request):
Notify("UploadTest", "METHOD: " + str(request.method) + """
""" + str(request))
return render_to_response('upload.htm', context_instance=RequestContext(request))
def UploadedTest(request):
Notify("UploadedTest", "METHOD: " + str(request.method) + """
""" + str(request) + """
FILES:
""" + str(request.FILES))
return render_to_response('upload.htm', context_instance=RequestContext(request))
upload.htm (note method is POST, I tried lowercase just in case as well):
<form
id="upload_file_form"
action="/uploaded_test"
method="POST"
enctype="multipart/form-data"
target="_blank"
>
{% csrf_token %}
<input type="text" name="extra_field" value="extra post field" />
<input type="file" name="upload_file" id="upload_file" />
<input type="submit" name="upload" value="Submit" />
</form>
Results of simple test scenario
The emails I get (note the first is the form pre-submit, the second is the posted form page):
FIRST EMAIL (subject "UploadTest"):
METHOD: GET
<WSGIRequest
path:/upload_test/,
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{
'csrftoken': '...',
'sessionid': '...'},
META:{'CSRF_COOKIE': '...',
'DOCUMENT_ROOT': '...',
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTPS': '1',
'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
'HTTP_ACCEPT_LANGUAGE': 'en-us,en;q=0.5',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_COOKIE': '... csrftoken=...; CFID=16005; CFTOKEN=...; sessionid=...',
'HTTP_HOST': '...',
'HTTP_USER_AGENT': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:11.0) Gecko/20100101 Firefox/11.0',
'PATH_INFO': u'/upload_test/',
'PATH_TRANSLATED': '...site.wsgi/upload_test/',
'QUERY_STRING': '',
'REMOTE_ADDR': '10.1.0.100',
'REMOTE_PORT': '56037',
'REQUEST_METHOD': 'GET',
'REQUEST_URI': '/upload_test/',
'SCRIPT_FILENAME': '...site.wsgi',
'SCRIPT_NAME': u'',
'SERVER_ADDR': '10.1.10.12',
'SERVER_ADMIN': '...',
'SERVER_NAME': '...',
'SERVER_PORT': '443',
'SERVER_PROTOCOL': 'HTTP/1.1',
'SERVER_SIGNATURE': '',
'SERVER_SOFTWARE': 'Apache',
'SSL_TLS_SNI': '...',
'mod_wsgi.application_group': '...|',
'mod_wsgi.callable_object': 'application',
'mod_wsgi.handler_script': '',
'mod_wsgi.input_chunked': '0',
'mod_wsgi.listener_host': '',
'mod_wsgi.listener_port': '443',
'mod_wsgi.process_group': '',
'mod_wsgi.request_handler': 'wsgi-script',
'mod_wsgi.script_reloading': '1',
'mod_wsgi.version': (3, 3),
'wsgi.errors': <mod_wsgi.Log object at 0x1962048>,
'wsgi.file_wrapper': <built-in method file_wrapper of mod_wsgi.Adapter object at 0xb73d2770>,
'wsgi.input': <mod_wsgi.Input object at 0x1962020>,
'wsgi.multiprocess': True,
'wsgi.multithread': True,
'wsgi.run_once': False,
'wsgi.url_scheme': 'https',
'wsgi.version': (1, 1)}>
SECOND EMAIL (subject "UploadedTest", method is STILL "GET", notice HTTP_REFERER is correct, and request.FILES exists, but is empty.):
METHOD: GET
<WSGIRequest
path:/uploaded_test/,
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{
'csrftoken': '...',
'sessionid': '...'
},
META:{'CSRF_COOKIE': '...',
'DOCUMENT_ROOT': '...',
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTPS': '1',
'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
'HTTP_ACCEPT_LANGUAGE': 'en-us,en;q=0.5',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_COOKIE': 'csrftoken=...; sessionid=...',
'HTTP_HOST': '...',
'HTTP_REFERER': '.../upload_test/',
'HTTP_USER_AGENT': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:11.0) Gecko/20100101 Firefox/11.0',
'PATH_INFO': u'/uploaded_test/',
'PATH_TRANSLATED': '...site.wsgi/uploaded_test/',
'QUERY_STRING': '',
'REMOTE_ADDR': '10.1.0.100',
'REMOTE_PORT': '56040',
'REQUEST_METHOD': 'GET',
'REQUEST_URI': '/uploaded_test/',
'SCRIPT_FILENAME': '...site.wsgi',
'SCRIPT_NAME': u'',
'SERVER_ADDR': '10.1.10.12',
'SERVER_ADMIN': '...',
'SERVER_NAME': '...',
'SERVER_PORT': '443',
'SERVER_PROTOCOL': 'HTTP/1.1',
'SERVER_SIGNATURE': '',
'SERVER_SOFTWARE': 'Apache',
'SSL_TLS_SNI': '...',
'mod_wsgi.application_group': '...|',
'mod_wsgi.callable_object': 'application',
'mod_wsgi.handler_script': '',
'mod_wsgi.input_chunked': '0',
'mod_wsgi.listener_host': '',
'mod_wsgi.listener_port': '443',
'mod_wsgi.process_group': '',
'mod_wsgi.request_handler': 'wsgi-script',
'mod_wsgi.script_reloading': '1',
'mod_wsgi.version': (3, 3),
'wsgi.errors': <mod_wsgi.Log object at 0x1956b88>,
'wsgi.file_wrapper': <built-in method file_wrapper of mod_wsgi.Adapter object at 0x21cf0f8>,
'wsgi.input': <mod_wsgi.Input object at 0x1961048>,
'wsgi.multiprocess': True,
'wsgi.multithread': True,
'wsgi.run_once': False,
'wsgi.url_scheme': 'https',
'wsgi.version': (1, 1)}>
FILES:
<MultiValueDict: {}>
Doubtful, but perhaps it is something wrong in my wsgi file, or in my Apache site directive. Here are those, just in case:
...wsgi:
import os
import sys
path = '/var'
if path not in sys.path:
sys.path.append(path)
path = '/var/www'
if path not in sys.path:
sys.path.append(path)
path = '/var/www/site'
if path not in sys.path:
sys.path.append(path)
os.environ['DJANGO_SETTINGS_MODULE'] = 'site.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
Apache sites-enabled directive:
<VirtualHost 10.1.10.12:80>
ServerName ...
RewriteEngine on
RewriteCond %{HTTPS} !=on
RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [R,L]
</VirtualHost>
<Directory /var/www/site>
RewriteEngine on
RewriteCond 1 =1
RewriteRule ^/(.*) https://%{SERVER_NAME}
</Directory>
<VirtualHost 10.1.10.12:443>
ServerName ...
DocumentRoot /var/www/site
ServerAlias ...
ErrorLog /var/log/apache2/site-error.log
LogLevel warn
CustomLog /var/log/apache2/site-access.log combined
WSGIScriptAlias / /var/www/site/site.wsgi
SSLEngine on
SSLCertificateFile ...
SSLCertificateKeyFile ...
# Media serving, must specify which files to serve and not
# hand their urls to the django framework.
Alias /robots.txt .../robots.txt
Alias /favicon.ico .../favicon.ico
Alias /templates/ .../django_templates/
Alias /static/ .../django_static/
Alias /images/ .../django_static/images/
<Directory /usr/local/lib/python2.6/dist-packages/django/contrib/admin/media/>
Order deny,allow
Allow from all
</Directory>
<Directory .../django_static/>
Order deny,allow
Allow from all
</Directory>
<Directory /usr/local/lib/python2.6/dist-packages/django/contrib/admin/media/js>
Order allow,deny
Allow from all
</Directory>
<Directory .../uploads/>
Order allow,deny
Allow from all
</Directory>
XSendFile on

Perhaps I need to read more about the urls.py or regular expression. I accidentally changed the form to post to action="uploaded_test" without the opening slash, and received a 404 error report which has the request.method as "POST," (what the ?!##$*&)...??? This meant that it had to be something with my posting url! I had it as:
'^uploaded_test/+$'
I changed it to:
'^uploaded_test$'
and verified the form was posting to action="/uploaded_test" and it worked! I changed urls.py to:
'^uploaded_test/$'
and changed the form to post to action="/uploaded_test/" and it also worked! I had the /+ in most of my urls entries so that I could accept either /loc or /loc/ for many entries. I think I will change them back, now. I don't know if this is a bug or something I don't understand about the regular expression /+ syntax.

Have you tried an if-else statement in your presentation layer?.
for example if request.method == 'POST': and then the rest of your code.

Related

Redirect is not allowed for a preflight request

I have this problem where i get the response when trying to use a rest api: "Access to fetch at 'https://kollektivet.app:8082/api/login/' from origin 'https://kollektivet.app' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request."
Picture of response when trying to fetch
This happens when i try any of the rest api's I am using. From what i have read, this error means I am trying to re-direct, which I am not.
The backend is Django and looks like this:
#csrf_exempt
#api_view(["POST"])
#permission_classes((AllowAny,))
def register(request,):
password = request.data.get("password", "")
email = request.data.get("email", "")
if not email and not password and not email:
return Response(
data={
"message": "username, password and email is required to register a user"
},
status=status.HTTP_400_BAD_REQUEST
)
new_user = User.objects.create_user(
email=email, password=password
)
return Response(status=status.HTTP_201_CREATED)
And the front-end is in react which looks like this:
createUser(event) {
event.preventDefault();
let data = {
name: this.state.name,
password: this.state.password,
repeatPassword: this.state.repeatPassword,
email: this.state.email
};
if (this.state.name !== '' && this.state.password !== '' && this.state.email !== '' && this.checkPasswords()) {
console.log('name', this.state.name, 'password ', this.state.password, 'email ', this.state.email);
fetch("https://kollektivet.app:8082/api/register/", {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json',
},
mode: "cors",
body: JSON.stringify(data)
})
.then(response => response.json())
.then(data => console.log(data))
.catch(error => console.log(error));
this.setState({message: "Du er nå registrert! For å aktivere din konto trykk på linken som vi har sendt til deg på epost"});
this.setState({name: ""});
this.setState({password: ""});
this.setState({repeatPassword: ""});
this.setState({email: ""});
}
}
I Do have this is the Django settings file:
CORS_ORIGIN_ALLOW_ALL = True
CORS_ALLOW_HEADERS = (
'accept',
'accept-encoding',
'authorization',
'content-type',
'dnt',
'origin',
'user-agent',
'x-csrftoken',
'x-requested-with',
)
I am running this on apache2 if this is relevant.
The port 8082 is also closed. Does this need to be open when it is on the same server?
Thanks!
You're being redirected to site.site.comapi/register/
Do you have some other middleware that does this? Maybe in Apache config?
Note it's a 301 so your browser has cached this response and will now always redirect there even if your rove the code that resulted in this redirect, or even if you stop Django from running.
So you will need to also clear your redirect cache in the browser.
This is why I don't like 301 responses. 302 are much more polite.
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
"corsheaders.middleware.CorsMiddleware",
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
In settings.py, if I arrange my MIDDLEWARE array like this, it will work fine.
But if I move corsheaders.middleware.CorsMiddleware to the last line, that error will occur
Just writing my story here in case anybody may have the same issue as mine.
Basically, this is a server end issue, so no need to change anything at the front end.
For this error message, it indicated that the OPTIONS requests got a
response with the status code in either 301 Moved Permanently", "307
Temporary Redirect", or "308 Permanent Redirect".
Please check your requested URL with the 'location' attribute from the OPTIONS response, see if they are the same or not.
For me, the issue was my request URL is **"https://server.com/getdata"**
but the URL I set on the server was **"https://server.com/getdata/"**
then the server give me an 308 to correct it with the '/', this works for POST, GET and HEAD, but not OPTIONS.
I'm using flask, flask_restful and the flask_cors.
use this will also solve this redirect issue for me.
app.url_map.strict_slashes = False
I had the same problem until I discovered that the redirect was caused by the Django internationalization framework, where all urls get a i18n url extension such as /en/path_to_resource when actually path_to_resource was requested. The internationalization framework achieves this through a 302 redirect.
The solution to this problem is to keep the rest-api urls outside the section with the i18n_patterns. The resulting urls.py might then look like
urlpatterns = [
path('i18n/', include('django.conf.urls.i18n')),
path('rest/', include(('rest.urls', 'rest'), namespace='rest')),
]
urlpatterns += i18n_patterns(
path('admin/', admin.site.urls),
path('jsi18n/', JavaScriptCatalog.as_view(), name='javascript-catalog'),
path('my_app/', include(('my_app.urls', 'my_app'), namespace='my_app')),
)

Scrapy post request getting redirected to a wrong page

I am trying to get to details page of this site
To get there from the web one should click 1. Consula Titlulo 2. Select ORO from Minerals dropdown and 3. click Buscar. 4. Then click the very first item in the list.
Dev tools and Fiddler show that I should make POST request with item id as a payload and this POST request is then redirected to details page.
In my case Im being redirected to homepage. What do I miss?
Here is my Scrapy spider.
# -*- coding: utf-8 -*-
import scrapy
from scrapy.shell import inspect_response
class CodeSpider(scrapy.Spider):
name = "col"
start_urls =['http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/index.cmc']
headers ={
"Connection": "keep-alive",
"Cache-Control": "max-age=0",
"Origin": "http://www.cmc.gov.co:8080",
"Upgrade-Insecure-Requests": "1",
"DNT": "1",
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1, AppleWebKit/537.36 (KHTML, like Gecko, Chrome/68.0.3440.106 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Referer":"http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/busqueda.cmc",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "en-US,en;q=0.9,ru;q=0.8,uk;q=0.7",
}
def parse(self, response):
inspect_response(response, self)
payload = {'expediente': '29', 'tipoSolicitud': ''}
url = 'http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/busqueda.cmc'
yield scrapy.FormRequest(url, formdata = payload, headers=self.headers, callback = self.parse, dont_filter=True)
Here is the log with redirect.
2018-08-23 13:58:05 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/index.cmc> from <POST http://
www.cmc.gov.co:8080/CmcFrontEnd/consulta/busqueda.cmc>
2018-08-23 13:58:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/index.cmc> (referer: http://www.cmc.gov.co:8080/CmcFron
tEnd/consulta/busqueda.cmc)
From what I see scrapy also assigns correct Cookie before sending request.
In [2]: request.headers
Out[2]:
{'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'en-US,en;q=0.9,ru;q=0.8,uk;q=0.7',
'Cache-Control': 'max-age=0',
'Connection': 'keep-alive',
'Cookie': 'PHPSESSID=1um6r67md5qpdcqs9g2n15g605',
'Dnt': '1',
'Origin': 'http://www.cmc.gov.co:8080',
'Referer': 'http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/busqueda.cmc',
'Upgrade-Insecure-Requests': '1',
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1, AppleWebKit/537.36 (KHTML, like Gecko, Chrome/68.0.3440.106 Safari/537.36'}
What do I miss?
Moreover if I use Postman code with GET for details page it works fine and returns the page.
Same code in Scrapy redirects.
In [1]: url = "http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/detalleExpedienteTitulo.cmc"^M
...: ^M
...: headers = {^M
...: 'upgrade-insecure-requests': "1",^M
...: 'user-agent': "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",^M
...: 'dnt': "1",^M
...: 'accept': "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",^M
...: 'referer': "http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/busqueda.cmc",^M
...: 'accept-encoding': "gzip, deflate",^M
...: 'accept-language': "en-US,en;q=0.9,ru;q=0.8,uk;q=0.7",^M
...: 'cookie': "PHPSESSID=2ba8dsre6l42un95qu33k09ud6",^M
...: 'cache-control': "no-cache",^M
...: ^M
...: }^M
...:
In [2]: fetch(url, headers=headers)
2018-08-23 14:47:13 [scrapy.core.engine] INFO: Spider opened
2018-08-23 14:47:13 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/index.cmc> from <GET http://w
ww.cmc.gov.co:8080/CmcFrontEnd/consulta/detalleExpedienteTitulo.cmc>
2018-08-23 14:47:13 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://www.cmc.gov.co:8080/CmcFrontEnd/consulta/index.cmc> (referer: http://www.cmc.gov.co:8080/CmcFron
tEnd/consulta/busqueda.cmc)
It appears that I missed POST request in the very beggining. This post request generates correct session ID which is to be new for every other search.

getting empty post request with Django1.10 + Nginx + Supervisor + Gunicorn

I'm making posts requests with curl using this, and even more simple than this:
curl -u usr:pwd -H "Content-Type: multipart/form-data" -i --form "file=#/path/to/myfile/myfile.zip" -X POST http://midominio.co/api/mypostrequest/
but I'm always getting an empty body, this what I'm getting:
{
'session': <django.contrib.sessions.backends.db.SessionStore object at 0x7f118e502b90>,
'_post': <QueryDict: {}>,
'content_params': {'boundary': '------------------------axxxxxxxxx'},
'_post_parse_error': False,
'_messages': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x7f118e502d90>,
'resolver_match': ResolverMatch(func=piston.resource.Resource, args=(), kwargs={}, url_name=None, app_names=[], namespaces=[]),
'GET': <QueryDict: {}>,
'_stream': <django.core.handlers.wsgi.LimitedStream object at 0x7f118e502b50>,
'COOKIES': {},
'_files': <MultiValueDict: {}>,
'_read_started': False,
'META': {'HTTP_AUTHORIZATION': 'Basic Y2FpbjpjYWlu',
'SERVER_SOFTWARE': 'gunicorn/19.7.1',
'SCRIPT_NAME': u'',
'REQUEST_METHOD': 'POST',
'PATH_INFO': u'/api/mypostrequest/',
'SERVER_PROTOCOL': 'HTTP/1.0',
'QUERY_STRING': '',
'CONTENT_LENGTH': '180',
'HTTP_USER_AGENT': 'curl/7.51.0',
'HTTP_CONNECTION': 'close',
'SERVER_NAME': 'midominio.co',
'REMOTE_ADDR': '',
'wsgi.url_scheme': 'http',
'SERVER_PORT': '80',
'wsgi.input': <gunicorn.http.body.Body object at 0x7f118e5029d0>,
'HTTP_HOST': 'midominio.co',
'wsgi.multithread': False,
'HTTP_ACCEPT': '*/*',
'wsgi.version': (1, 0),
'RAW_URI': '/api/mypostrequest/',
'wsgi.run_once': False,
'wsgi.errors': <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f118e502950>,
'wsgi.multiprocess': True,
'gunicorn.socket': <socket._socketobject object at 0x7f118e4f6de0>,
'CONTENT_TYPE': 'multipart/form-data; boundary=------------------------axxxxxxxxx',
'HTTP_X_FORWARDED_FOR': '186.00.00.000',
'wsgi.file_wrapper': <class 'gunicorn.http.wsgi.FileWrapper'>},
'environ': {'HTTP_AUTHORIZATION': 'Basic Y2FpbjpjYWlu',
'SERVER_SOFTWARE': 'gunicorn/19.7.1',
'SCRIPT_NAME': u'',
'REQUEST_METHOD': 'POST',
'PATH_INFO': u'/api/mypostrequest/',
'SERVER_PROTOCOL': 'HTTP/1.0',
'QUERY_STRING': '',
'CONTENT_LENGTH': '180',
'HTTP_USER_AGENT': 'curl/7.51.0',
'HTTP_CONNECTION': 'close',
'SERVER_NAME': 'midominio.co',
'REMOTE_ADDR': '',
'wsgi.url_scheme': 'http',
'SERVER_PORT': '80',
'wsgi.input': <gunicorn.http.body.Body object at 0x7f118e5029d0>,
'HTTP_HOST': 'midominio.co',
'wsgi.multithread': False,
'HTTP_ACCEPT': '*/*',
'wsgi.version': (1, 0),
'RAW_URI': '/api/mypostrequest/',
'wsgi.run_once': False,
'wsgi.errors': <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f118e502950>,
'wsgi.multiprocess': True,
'gunicorn.socket': <socket._socketobject object at 0x7f118e4f6de0>,
'CONTENT_TYPE': 'multipart/form-data; boundary=------------------------axxxxxxxxx',
'HTTP_X_FORWARDED_FOR': '186.00.00.000',
'wsgi.file_wrapper': <class 'gunicorn.http.wsgi.FileWrapper'>},
'path_info': u'/api/mypostrequest/',
'content_type': 'multipart/form-data; boundary=------------------------axxxxxxxxx',
'path': u'/api/mypostrequest/',
'data': <QueryDict: {}>,
'method': 'POST',
'user': <User: cain>
}
I'm using the default nginx configuration, I also have a web version of this part with an "upload button" and everything is working fine, there is not errors in the .log files of nginx or supervisor and the django installation is okay, any ideas? thanks
I was a mistake trying to take this as a normal request, this is a WSGIRequest so that means it need to be processed different I solved using this, I hope this help to someone else:
environ = request.environ
form = cgi.FieldStorage(fp=environ['wsgi.input'], environ=environ)
f = form['file'].file

Below POST Method is not working in scrapy

I have tried with headers, cookies, Formdata and body too, but i got 401 and 500 status code. In this site First Page is in GET method & gives HTML response and further pages are in POST method & gives JSON response. But these status codes arrives for unauthorisation but i have searched and i couldn't find any CSRF token or auth token in web page headers.
import scrapy
from SouthShore.items import Product
from scrapy.http import Request, FormRequest
class OjcommerceDSpider(scrapy.Spider):
handle_httpstatus_list = [401,500]
name = "ojcommerce_d"
allowed_domains = ["ojcommerce.com"]
#start_urls = ['http://www.ojcommerce.com/search?k=south%20shore%20furniture']
def start_requests(self):
return [FormRequest('http://www.ojcommerce.com/ajax/search.aspx/FetchDataforPaging',
method ="POST",
body = '''{"searchTitle" : "south shore furniture","pageIndex" : '2',"sortBy":"1"}''',
headers={'Content-Type': 'application/json; charset=UTF-8', 'Accept' : 'application/json, text/javascript, */*; q=0.01',
'Cookie' :'''vid=eAZZP6XwbmybjpTWQCLS+g==;
_ga=GA1.2.1154881264.1480509732;
ASP.NET_SessionId=rkklowbpaxzpp50btpira1yp'''},callback=self.parse)]
def parse(self,response):
with open("ojcommerce.json","wb") as f:
f.write(response.body)
I got it working with the following code:
import json
from scrapy import Request, Spider
class OjcommerceDSpider(Spider):
name = "ojcommerce"
allowed_domains = ["ojcommerce.com"]
custom_settings = {
'LOG_LEVEL': 'DEBUG',
'COOKIES_DEBUG': True,
'DEFAULT_REQUEST_HEADERS': {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36',
},
}
def start_requests(self):
yield Request(
url='http://www.ojcommerce.com/search?k=furniture',
callback=self.parse_search_page,
)
def parse_search_page(self, response):
yield Request(
url='http://www.ojcommerce.com/ajax/search.aspx/FetchDataforPaging',
method='POST',
body=json.dumps({'searchTitle': 'furniture', 'pageIndex': '2', 'sortBy': '1'}),
callback=self.parse_json_page,
headers={
'Content-Type': 'application/json; charset=UTF-8',
'Accept': 'application/json, text/javascript, */*; q=0.01',
'X-Requested-With': 'XMLHttpRequest',
},
)
def parse_json_page(self,response):
data = json.loads(response.body)
with open('ojcommerce.json', 'wb') as f:
json.dump(data, f, indent=4)
Two observations:
a previous request to another site page is needed to get a "fresh" ASP.NET_SessionId cookie
I couldn't make it work using FormRequest, use Request instead.

Different environment in wsgi request object

After some hours of reading manuals and other helps, finally i got my nginx+uwsgi1.9+django1.6+python3.3 server.
But now I have problems (mb problem in my understanding something) - how to get POST data from request ? I mean how to get it correctly.
The code in django view:
def info(request):
print(request)
return HttpResponse(request)
request to server:
http://127.0.0.1:8000/info/
{
"test":"test"
}
and interesting part - output in uwsgi log (POST and GET dicts):
<WSGIRequest
path:/info/,
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{},
META:{'CONTENT_LENGTH': '24',
'CONTENT_TYPE': 'application/json',
'CSRF_COOKIE': 'upJxA8TWO0nhKACr0dfU46Qyu0DzzUTR',
'DOCUMENT_ROOT': '/usr/share/nginx/html',
'HTTP_ACCEPT': '*/*',
'HTTP_ACCEPT_ENCODING': 'gzip,deflate,sdch',
'HTTP_ACCEPT_LANGUAGE': 'ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_CONTENT_LENGTH': '24',
'HTTP_CONTENT_TYPE': 'application/json',
'HTTP_COOKIE': '',
'HTTP_HOST': '127.0.0.1:8000',
'HTTP_ORIGIN': 'chrome-extension://hgmloofddffdnphfgcellkdfbfbjeloo',
'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36',
'PATH_INFO': '/info/',
'QUERY_STRING': '',
'REMOTE_ADDR': '127.0.0.1',
'REMOTE_PORT': '53315',
'REQUEST_METHOD': 'POST',
'REQUEST_URI': '/info/',
'SCRIPT_NAME': '',
'SERVER_NAME': 'django',
'SERVER_PORT': '8000',
'SERVER_PROTOCOL': 'HTTP/1.1',
'uwsgi.node': b'',
'uwsgi.version': b'1.9.18.2',
'wsgi.errors': <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>,
'wsgi.file_wrapper': <built-in function uwsgi_sendfile>,
'wsgi.input': <uwsgi._Input object at 0x7fa4ec6b8ee8>,
'wsgi.multiprocess': True,
'wsgi.multithread': False,
'wsgi.run_once': False,
'wsgi.url_scheme': 'http',
'wsgi.version': (1, 0)}>
and response in browser (same as request to server):
{
"test":"test"
}
What I'm doing wrong ?
request.POST
and
request.POSt.dict()
returns empty dicts.
So questions is how to get POST data in code and why it looks so different when I use environment ?
UPD:
return HttpResponse(str(request)) - returns WSGIRequest object instead of POST data to me. But still dont know how to get POST data in code.
UPD2:
uwsgi config:
[uwsgi]
module = mysite.wsgi
master = true
processes = 5
socket = :8001
daemonize = /var/log/uwsgi/mysite.log
touch-reload = /tmp/uwsgi-touch
post-buffering = 1
UPD3:
versions of software:
Python 3.3.2
Django 1.6.2
uWSGI 1.9.18.2
UPD4:
Final code:
if request.method == "POST":
if request.META["CONTENT_TYPE"] == "application/json":
result = json.loads(request.body.decode())
else: result = request.POST.dict()
return HttpResponse(json.dumps(result), content_type="application/json")
Thats what I exactly want.
When I send data to server with
POST
Content-Type: application/x-www-form-urlencoded
a=1
or:
POST
Content-Type: application/json
{"a":"1"}
I see same response (and in code variables):
Content-Type: application/json
{"a": "1"}
Looks like you a POSTing JSON data, not HTML form data.
In that case you are looking for the raw post data, which is accessed like this:
request.body
See HttpRequest.body in the documentation.
If you want to parse that JSON string, use this:
import json
data = json.loads(request.body)
The request.POST dictionary is only populated when the request contains form data. This is when the Content-Type header is application/x-www-form-urlencoded or multipart/form-data.