DRF DataTables GET Request Is Too Long - django

I am using DRF to serialize a model, and serve the model to an endpoint. I am using django-rest-framework-datatables to format that endpoint into the JSON DataTables requires with ?format=datatables. I have a large DataTable that I am trying to render, and once I get to a certain number of columns, a nondescript 404 error occurs (https://datatables.net/manual/tech-notes/7). The request URL for the ajax data is 2k+ characters long, and I think this is the issue since it's a GET request.
My query string looks like this:
format=datatables&draw=1&columns%5B0%5D%5Bdata%5D=id&columns%5B0%5D%5Bname%5D=&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=triggerDate&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=entities&columns%5B2%5D%5Bname%5D=entities.entityDescription&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=analyticCategory&columns%5B3%5D%5Bname%5D=analyticCategory.analyticCategory&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=analytic&columns%5B4%5D%5Bname%5D=analytic.analyticDescription&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=risk&columns%5B5%5D%5Bname%5D=risk.risk&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=logic&columns%5B6%5D%5Bname%5D=logic.logic&columns%5B6%5D%5Bsearchable%5D=true&columns%5B6%5D%5Borderable%5D=true&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=description&columns%5B7%5D%5Bname%5D=&columns%5B7%5D%5Bsearchable%5D=true&columns%5B7%5D%5Borderable%5D=true&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=disposition&columns%5B8%5D%5Bname%5D=&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=true&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=asc&start=0&length=10&search%5Bvalue%5D=&search%5Bregex%5D=false&_=1554909043682
If I remove one column (any column), it works perfectly.
How do I get around this limitation without using POST since it's a REST API?

Related

Django rest-auth PUT method returning JSON Unexpected token

I've been trying to try to do this for a couple of days for now. I am trying to update the user's data through put method in Django using rest framework. However, I've been receiving this error message in the console.
This is the fetch function that is used to send the data to the backend. As it's shown, I'm even trying to compare it to other fetch methods and I think nothing is wrong with the body of the fetch.
Below are my codes used to update the data.
views.py
serializers.py
urls.py
So, the error is
Unexpected token < in JSON
which means that it tried to parse a response containing a character <. If I had to guess, the response is a HTML document, because it would start with < (<html>...).
For instance, it can be a 404 error page because the URL was not found. The best way to know is, instead of logging response.json(), just log response.text(). Until you do that, hard to know what really happens.

How to make a request in Postman

Arabam is an e-commerce site that I am attempting to query. As an example, given the automobile page, you can add query parameters to the page such as days and sort as follows:
https://www.arabam.com/ikinci-el/otomobil?days=30&sort=startedAt.desc
I will be accessing the data via their API, however, which lives at:
https://api.arabam.com/listing/v2/search
And here is the API Key I'm using:
_V85Kref7xGZHc1XRpUmOhDDd07zhZTOvUSIbJe_sSNHSDV79EjODA==
I am able to make the request using Postman:
But whichever query parameters I pass, the total number of keys in the response remains the same. How do I pass parameters correctly? Either I am not passing them correctly, or these are not the correct parameters. How do I find correct parameters?
I'm relatively new to this so need a bit of guidance.
Have you tried posting the params as a json body instead

convert JSON input to form-data in AWS application gateway

Our old legacy APIs accept the data only in form-data format, but I am required to send my data as JSON in body of request. So, how I can convert my JSON (application/json) input to form data in AWS Application gateaway.
I have input parameter like this
{"key1": "val1", "key2": "val2"}
I tried many solution with template mapping and query string parameter but they didn't work for me maybe I am doing something wrong. Above configuration is fully supporting form-data.
Note: Due to some reason I don't want to change my legacy django code to handle JSON input instead of form data.
I would suggest looking at the structure of a raw POST form-data request. Then you should be able to structure the Integration Request to fit the form-data format using the right Content-Type and mapping template.
There is an example for GET and an example for POST on this page (with the caption "the HTTP request looks like this") https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Forms/Sending_and_retrieving_form_data

Facebook url scrape failing with internal 500 error

According to https://developers.facebook.com/docs/sharing/opengraph/using-objects#update it is possible to do a 'POST' on ?id=url&scrape=true to refresh the url object in Facebook cache.
This is working fine and the object attributes are getting refreshed with the last scraped time changing when the url is viewed thru the Facebook debugger.
However the API call to scrape always fails with a 500 internal error and I am keen to avoid this error from happening. I have tried to several combination of using POST fields and with/without encoding the query parameters. But nothing has helped.
Anybody else having this problem and any help is highly appreciated.
I am using graph api version - v2.5

HTSQL shell's Get Request Not compatible with Gunicorn+Nginx for long queries

I am using HTSQL with Django. I use HTSQL shell to check/generate my queries and then use them for rendering data in json and raw formats.
so like, my HTSQL shell url is:
http://127.0.0.1:8000/htsql
so when I want to access data from a table in the HTSQL shell environment, I do,
http://127.0.0.1:8000/htsql/table_name
and to get JSON data,
http://127.0.0.1:8000/htsql/table_name/:json
In background, HTSQL shell fetches this data by using a GET request. So from my client-side Javascipt/jQuery, I initiate a GET request with its URL in above format and get my desired JSON data directly.
Everything was fine when I was using local Django server, but when I deployed my project using Gunicorn and Nginx, it naturally started to block some of my long(actually, pretty long) queries in the GET requests. I searched this problem and found out that Gunicorn allows GET request values ranging from 0 to 8190 characters. So I tweaked my Gunicorn settings for the maximum limit but still the same problem. This was because my queries, when used with several filter values, are exceeding 8190 limit.
So I thought to use POST request as its normally preferred for secure and long requests. So I changed my GET request to POST request and pointed it to the same URL as mentioned above and tried it on my local Django Server(i.e without Gunicorn and Nginx). But now I get "400 BAD REQUEST". With firebug, I checked that the response was "POST requests are not permitted."
I also noticed that the HTSQL_Django Module routes all the request to htsql_django.views.gateway. I had a look to this gateway function in the views.py of htsql_django module but couldn't find any clue.
Is it so, that the HTSQL doesn't accepts POST requests?? How can I fetch/access JSON data from HTSQL using POST request?