I'm trying to POST some JSON and a binary file from an iPhone to a Django server running django-piston using ASIHTTPRequest
I know how to get it to work if I am ONLY sending JSON strings, and I know how to make it work if I am ONLY sending a file, but doing both is tricky.
So we'll start with ASIHTTPRequest code
ASIFormDataRequest *request = [[ASIFormDataRequest alloc] initWithURL:url];
[request setRequestMethod:#"POST"];
[request setPostFormat:ASIMultipartFormDataPostFormat];
[request appendPostData:[#"{\"save\":{\"name\":\"iostest\"}}" dataUsingEncoding:NSUTF8StringEncoding]];
[request addData:UIImageJPEGRepresentation([UIImage imageNamed:#"test.jpg"], 1.0f)
withFileName:#"test.jpg"
andContentType:#"image/jpeg"
forKey:#"data"];
[request setDelegate:self];
[request startAsynchronous];
My best idea here is that adding raw string data directly to the POST body and then adding a file just doesn't work.
But if I instead try
[request setPostValue:#"{\"name\":\"iostest\"}" forKey:#"save"];
Then the piston data dictionary will store ['save'] as a string instead of a deserialized object, so it will literally deliver the string
"{\"name\":\"iostest\"}"
Here's my Piston handler code
def create(self, request):
data = request.data
print(data['save']) #{\"name\":\"iostest\"}"
print("Files: " + request.FILES['data'].name) #test.jpg
print("Data Save Name: " + data['save']['name']) #crash, interprets this as a string indeces lookup
Ideas are welcome.
I have basically hacked my way around this.
The basic problem is that the request format in which Django expects files to be submitted to the server is one which django-piston literally just drops the ball on.
When it encounters multipart requests, it simply doesn't try to parse the data.
The solution to this problem is to manually call the parsing engine, which, in the case of JSON, is straight out of django.utils (which is kind of disappointing).
You achieve this by using ASIHTTPRequest (or the request module of your choice) to set a standard post value by key, and then access it the old fashioned way.
from django.utils import simplejson
data = simplejson.loads(request.POST['save'])
Which basically just reduces this handler method at this point to nothing more than a regular old Django view in terms of the steps you have to take to get it going.
So clearly, django-piston is not built to deal with files apparently?
My best idea here is that adding raw
string data directly to the POST body
and then adding a file just doesn't
work.
That wouldn't work, no. If you're POSTing form data using 'application/x-www-form-urlencoded' format, or 'multipart/form-data' you're not going to be able to just tack some extra data on the end - it needs to go in as part of the form data. Something like this I guess...
[request setPostValue:#"{\"save\":{\"name\":\"iostest\"}}" forKey:#"data"];
But if I remove the string data and only post the file it still doesn't work.
Is more problematic...
or if it's Piston erroneously misreading the data.
I probably wouldn't look in that direction first - piston doesn't really mess with the request object, so it seems more likely that the ASI request isn't quite right.
I think the place to start would be to inspect the incoming request and check that it really is a valid formPOST request:
Check that request["CONTENT_TYPE"] is set to 'multipart/form-data'
Inspect the request.raw_post_data and make sure that it is valid form data as specified in http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2 - check that the key names are as you expected and that the file content is present. (Obviously you'll want to use a small text file when you're testing this!)
Check which keys actually are present in request.FILES, if any, in case it's as simple as something like a misnamed field.
Failing all that I'd try to narrow down if it's a problem on the client or server side by trying to write a plain python client and seeing if you have the same issue then. Looking around, something like this: http://atlee.ca/software/poster/ might be useful.
Related
I am trying to pass an encrypted token from an external application into Application Express. I want to read and work with this token in a custom authentication scheme as a way to authenticate the user into the application.
What is the best way to do this? At first, I was trying to just append the token onto the URL, eg:
/pls/apex/f?p=999:1&Token=XXXXXXXX
But then Apex returns a 404.
So then, I was trying to use the Application Express session values to send in the token, creating a URL like this:
f?p=999:1:::::TOKEN:XXXXXXXX
And then my sentry function I would do something like:
v_token := V('TOKEN')
To get it. However, this isn't working either, and I think's because the session isn't established yet when the sentry function executes? And is it even possible to do it this way? (Since there would be no item with this name, and no page yet to create it on...)
Is there a better approach to doing what I'm trying to do? If I had this added as a HTTP Header upstream, can I read that somehow in the sentry function? Maybe with owa_util.get_cgi_env? Does that work to read HTTP Headers from the request?
Thank you
If anyone else runs into something like this - I figured out a workaround.
Just put the token in the "value" session variables section of the URL, like so
f?p=999:1::::::XXXXXXXX
Then in the "sentry function" I can get the entire query string like this:
v_query_str := owa_util.get_cgi_env('QUERY_STRING');
And then I can split v_query_str by : and get the 8th token, which is what I need.
I found some examples using apex_util.string_to_table to split the string, which works nicely.
I'm using django_haystack with Solr 4.9. I've amended the /select request handler so that all requests use dismax by default.
The problem is that sometimes I would like to query specific fields, but I can't find a way to get the SearchQuerySet api to get it to play nicely with dismax. So basically I want to send the following (or equivalent) request to Solr:q=hello&qf=content_auto
I've tried the following aproaches:
Standard Api
SearchQuerySet().filter(content_auto='hello')
# understandably results in the following being sent to solr:
q=content_auto:hello
AltParser
query = AltParser('dismax', 'hello', qf="content_auto")
sqs = SearchQuerySet().filter(content=query)
# Results in
q=(_query_:"{!dismax+qf%3Dcontent_auto}hello")
Raw
query = Raw('hello&qf=content_auto')
# results in
q=hello%26qf%3Dcontent_auto
The last approach was so close, but since it escaped the = and & it doesn't seem to process the query correctly.
What is the best approach to dealing with this? I have no need for non-dismax querying so it would be preferable to keep the /select request handler the same rather than having to wrap every query in a Raw or AltParser.
In short, the answer is that it can't be done without creating a custom backend and SearchQuerySet. In the end I had to just revert back to a standard configuration and specifying dismax with an AltParser, slightly annoying because it affects your spelling suggestions.
I have a Django server responsible for serving JSON formatted files to a controller machine. My server gets data from POST requests and serve JSON files on GET requests. I'm wondering what is the most RESTful and efficient way of creating this bridge on the server?
I'm considering creating a JSON model and storing instances on my database for each POST request and pack the instances into JSON files dynamically and serve them on GET requests. The alternative would be creating JSON files on POST requests, saving them in a folder on the server, and serving these files on GET requests.
Which way is better and why? Or am I failing to see a completely better method?
Why are you creating files? You can let a Django view return a JSON response instead of a HTML response:
import json
# in your view:
data = {}
return HttpResponse(json.dumps(data), mimetype="application/json")
Construct the JSON data dynamically from the available data and if the JSON responses are big, add caching (e.g., memcached or Varnish).
This will prevent certain problems (what to do if a GET request is done but there is no JSON file yet?). This way you can generate the JSON based on the data that you have or return JSON with an error message instead.
I looked into json serialization with natural keys and dependencies etc. to control the fields that are serialized. I also tried using the middleware wodofstuff to allow for deeper foreign key serialization. However, I decided to use templates to render the JSON response.
Some pitfalls are
I am responsible for formatting the JSON (more prone to mistakes like a missing semi-colon)
I am responsible for escaping characters
Renders slower than buit-in serialization?
Some advantages are
I have control over what is serialized (even if the underlying models is changed)
I can format many to many or foreign key relationships on the JSON file however I like
TLDL; In my case, the format of the JSON file I needed was very custom. It had dictionary in list in dictionary. Some fields are iterative so I have for loops in the templates to render them. However, the format requires some of the fields in the iterated object to be encapsulated in a list as oppose to a dictionary.
This was an obstacle I ran into while considering simplejson. By using something like
import simplejson as json
def encode_complex(obj):
if isinstance(obj, complex):
return [obj.real, obj.imag]
raise TypeError(repr(o) + " is not JSON serializable")
json.dumps(2 + 1j, default=encode_complex)
'[2.0, 1.0]'
I could manage to return iterated data; however, I needed iteration in iteration and custom object types (list or dict) to encapsulate certain iterations. Finally, (may it be lack of knowledge or lack of patience) I decided to just do it in the templates.
I feel like rendering it through templates is not the most scalable or 'smartest' way of doing it, could it possibly be done in a better way? Please feel free to prove me right or wrong.
I have stumbled on a strange issue that I can't resolve:
In my Django app there is a method which gets hit by a POST from a java applet, which sends it a JSON object. Django method parses it like so:
req = json.loads(request.raw_post_data)
and based on the results returns a value. I haven't written this code, but yesterday I was sent to investigate an error triggered in this method. It was saying there was "ValueError: Expecting property name: line 1 column 1 (char 1)".
What I discovered is that my raw post data looks like this:
{#012#011"ImmutableMachineFactors": #012#011{#012#011#011"machineName": "lukka",#012#011#011"osName": "MacOS"}}
The type of it was string, however, my attempts to replace these weird characters with spaces or nothing failed. It would just ignore the sub() command. I know that raw_post_data returns a bytestring, but when I tried to convert it to a regular string using:
mystring.decode('utf-8')
it did add the u'' notation, but didn't remove those weird characters. Stranger still, in many cases (on my personal machine), Django would happily convert this kind of data into JSON, it only fails sometimes, which led me to believe that the JSON which triggered the error was malformed, but when I would strip out all the #011 and #012 characters, it parsed perfectly.
My questions are:
1) What are those crazy things? (#011, #012). I tried to google around, but these are very common things to find in a search, so I couldn't find anything relevant.
2) How can I turn this bytestring into a regular string so that I can replace those characters? Or is it the wring way to approach this problem?
Thanks!
Luka
This may be way too late to help, but since QueryDict instances (request.POST or request.DATA) are immutable, it's reasonable to expect that request.raw_post_data is also immutable. You'd have to make a copy before changing it.
I have some existing python code that uses django-piston which returns a dictionary as its response. For example:
from piston.handler import BaseHandler
class FooHandler(BaseHandler):
allowed_methods = ('GET',)
#classmethod
def create(self, request):
return { 'foo': 'bar' }
This code works fine, and is serialized into JSON with the appropriate HTTP header set (I'm assuming this works by some piston magic involving emitters; for bonus points, feel free to clarify how this behavior works as well, as I'm still getting to know django-piston).
I need to be able to modify the response in arbitrary ways, e.g. setting headers, status codes, etc. without using some pre-baked solution designed for a specific purpose. Is there a convenient way to access the response object in the context of this code and manipulate it, or has the response object not yet been created? In order to get access to a response object, will I have to construct it manually (a la vanilla django), serialize the dictionary, and set the appropriate headers by hand, and thus lose out on some of the useful magic of django-piston?
Every django-piston method returns an HTTPResponse.
When you return that dictionary, django-piston is just serializing it and sticking it in an HTTPResponse that it has crafted of some variety.
Kind of surprised you didn't pick up on that given that those "return rc.CREATED" lines in all the django-piston examples in the wiki are just hyper-simplistic responses with an HTTP code and response body.
Take a look here: https://bitbucket.org/jespern/django-piston/src/c4b2d21db51a/piston/utils.py
at the rc_factory class, which creates a variety of simple HTTPResponse objects for use with Piston.
At the very least you can observe how they do it, and then craft your own.
But the answer to the essence of your question "can I make a custom HTTPResponse" is yes, you can. Of course, that's what web servers do.
It is possible to set a custom response code by returning an HttpResponse object from your handler methods.
return HttpResponse({'foo': 'bar'}, status=404)
Unfortunately, you cannot set headers in the same way. For this, you have to write a custom Emitter that sets the headers you need. Something along these lines might work:
class CustomEmitter(JSONEmitter):
def render(self, request):
content = super(CustomEmitter, self).render(request)
response = HttpResponse(content)
response['Cache-Control'] = 'max-age=60'
Emitter.register('json', CustomEmitter, 'application/json; charset=utf-8')
You are quite right that django-piston uses emitters to serialize/deserialize requests and responses. You can even register your own emitters with piston and have it use those.
There are several ways that you can determine what the format should be.
1. mime-type
2. <format> in your URL
3. .json at the end of your URL
Which particular headers are you wanting to manipulate? There may be other solutions then just hacking away at the response object.