I am trying to upload a large json data from android retrofit client to django rest api view.
For example(Json Data):
{'post_data': [{'x':'asdkdadlf fdsfsd','y':'This is a paragraph........ Ending paragraph', 'created_on':'2018-22-08'},
{'x':'asdkdadlf fdsfsd','y':'This is a paragraph........ Ending paragraph', 'created_on':'2018-22-08'},
{'x':'asdkdadlf fdsfsd','y':'This is a paragraph........ Ending paragraph', 'created_on':'2018-22-08'},
....
{'x':'asdkdadlf fdsfsd','y':'This is a paragraph........ Ending paragraph', 'created_on':'2018-22-08'},
{'x':'asdkdadlf fdsfsd','y':'This is a paragraph........ Ending paragraph', 'created_on':'2018-22-08'},
]}`
API Response output:
08-22 13:21:12.964 15628-16104/com.mml.wapp D/Abhay: Group type: 1 group name: REAL ESTATE PROPERTYfrom:+234 818 230 9054
08-22 13:21:12.995 15628-16104/com.mml.wapp D/Abhay: Total messages in db before adding current message: 100
08-22 13:21:13.062 15628-16104/com.mml.wapp D/deleteMessage: Deleted all messages
08-22 13:21:13.200 15628-16109/com.mml.wapp D/OkHttp: --> POST https://api.multiplymyleads.com/api/upload_whatsapp_msgs/ http/1.1
Content-Type: application/x-www-form-urlencoded
08-22 13:21:13.204 15628-16109/com.mml.wapp D/OkHttp: Content-Length: 73304
08-22 13:21:13.211 15628-16109/com.mml.wapp D/OkHttp: postdata=%7B%22%E2%80%AA%2B234%20818%20230%209054%E2%80%AC%22%3A%5B%7B%22group_type%22%3A%221%22%2C%22time%22%3A%222018-08-
... followed by a lot more of that
As far as I understand, data in above response seems huge, may be that's why its getting timeout error? As its not able to upload all data in one API call in given timeout time in server side.
What can we do to resolve above issue ?
I have checked API returning success response if reduce the json post_data size.
I am using retrofit 2.0 client in android side for API call.
After lot of research I solved it from android side using below spinet of code.
OkHttpClient client = new OkHttpClient();
client.newBuilder()
.connectTimeout(60, TimeUnit.SECONDS)
.readTimeout(60L, TimeUnit.SECONDS)
.writeTimeout(60L, TimeUnit.SECONDS);;
Which increases timeout for response.
Related
As I see from the answer for this question: Karate will automatically send any cookies returned by the server in the next request.
But when I send the request I see two sets of cookies in Set-Cookie of response: one is auto-created and another is real, that returned from the server.
When I printed responseCookies, I saw there only automatic cookies
and for the next request new cookies are generated and sent.
For my test I need to use cookies returned after the first request because it is a call to login service.
Feature: Using cookies in next request
Background:
Given url baseUrl
And path LOGOUT_SERVICE_ENDPOINT
And configure headers = read('classpath:headers.js')
And def filename = 'classpath:resources/users/' + brand.toLowerCase() + '/user.json'
And json user = read(filename)
Scenario: Login
When def login = callonce read('classpath:features/login_service/login.feature') user
* print login.responseCookies
And request { arg1: '#(brand)'}
And method post
Then status 200
What is wrong in my feature or it is Karate issue?
two sets of cookies in Set-Cookie of response:
Maybe that is a bug in the server ?
Also try using "shared scope", because cookies also will be part of the "global" variables etc.
* callonce read('classpath:features/login_service/login.feature') user
* request { arg1: '#(brand)'}
If you are still stuck, please follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I am trying to load a web page and monitor it for XHR (XMLHttpRequests). To do this I am using Ghost.py with Python2.7. I can see the XHR being made and can read the URL and response, however I would like to read the request body so that I can recreate these requests at a later date.
from ghost import Ghost, Session
ghost = Ghost()
with ghost.start():
session = Session(ghost, download_images=False, display=False)
page, rs = session.open("https://www.example.com/", timeout=60)
assert page.http_status == 200
result, resources = session.wait_while_selector('[class=loading]:not([style="display: none;"])', timeout=60)
for resource in resources:
print resource.url
print resource.content
I have searched within the documentation, but cannot find any references to XHR request body and I have searched within the returned resources object for references to the request, but can only find request.headers (which does not include the POST body) and the _reply data.
Can you save the POST body of a XHR request as well as the response?
I'll first explain the architecture of my system and then move to the question:
I have a REST API which is used as my API gateway. This server is build using Flask. I also have RabbitMQ cluster, and a client I wrote that listens to a specific queue and executes the tasks its getting.
Until now, all of my requests were asynchronous, so once a request has reached to the API gateway, a callback_uri field with URL to POST the results to provided as part of the request, and the API gateway was just responsible for sending the task to RabbitMQ and the worker processed the task, and at the end POSTed the results back to the callback URL.
My question is:
I want a new endpoint to be synchronous in the sense of, that the processing will be done still by the same worker as before, but I want to get the results back to the API gateway to return to the user and release the connection.
My current solution:
I'm sending a unique callback_uri as part of the request to the worker as before, but now the specific endpoint is implemented by my API gateway and allow both POST and GET methods, so the worker can POST the results once it finished, and my API gateway keeps polling the callback URL until a result is available and then return the result to the client.
Is there any other preferred option other than having a busy-waiting HTTP worker polling its own endpoint to get the results? but still be synchronous so the connection released only when the results become available?
Code for illustration only:
#app.route('/long_task', methods=['POST'])
#sync_request
def long_task():
try:
if request.get_json() is None:
return ERROR_MSG_NO_JSON, 400
create_and_send_request_to_rabbitmq()
return '', 200
except Exception as ex:
return ERROR_MSG_NO_DATA, 400
def sync_request(func):
def call(*args, **kwargs):
create_callback_uri()
result = func(*args, **kwargs)
status_code = result[1]
if status_code == 200:
result = get_callback_result()
return result
return call
def get_callback_result():
callback_uri = request.get_json()['callback_uri']
has_answer = False
headers = {'content-type': 'application/json'}
empty_response = {}
content = json.dumps(empty_response)
try:
with Timeout(seconds=SYNC_REQUEST_TIMEOUT_SECONDS):
while not has_answer:
response = requests.get(callback_uri, headers=headers)
if response.status_code == 200:
has_answer = True
content = response.content
else:
time.sleep(0.2)
except TimeoutException:
log.debug('Timed out on sync request for request %s ' % request)
return content, 200
So if I understand you correctly you want your backend to wait for the response from some worker (via RabbitMQ). You can achieve that by implementing rpc over rabbitmq. The key idea is to use the correlation id.
But definitely the most efficient way would be to run the client over websockets (or raw tcp socket if it is not a browser) and notify him directly when the job is done. That way you don't lock resources (client connection, rabbitmq queues) and you avoid performance hit (rpc).
I am trying to send mails with attachment by using Amazon SES
HttpRequest httpReq = new HttpRequest();
httpReq.setMethod('POST');
httpReq.setEndpoint('https://email.us-east-1.amazonaws.com');
Blob bsig = Crypto.generateMac('HmacSHA256', Blob.valueOf(awsFormattedNow), Blob.valueOf(secret));
httpReq.setHeader('X-Amzn-Authorization','AWS3-HTTPS AWSAccessKeyId='+key+', Algorithm=HmacSHA256, Signature='+EncodingUtil.base64Encode(bsig));
httpReq.setHeader('Date',awsFormattedNow);
httpReq.setHeader('From','sample#gmail.com');
httpReq.setHeader('To','sample#gmail.com');
httpReq.setHeader('Subject','Hello');
httpReq.setHeader('Accept-Language','en-US');
httpReq.setHeader('Content-Language','en-US');
httpReq.setHeader('Content-Type','multipart/mixed;boundary=\"_003_97DCB304C5294779BEBCFC8357FCC4D2\"');
httpReq.setHeader('MIME-Version','1.0');
// httpReq.setHeader('Action','SendRawEmail');
String email = 'Action=SendRawEmail';
email += '--_003_97DCB304C5294779BEBCFC8357FCC4D2 \n Content-Type: text/plain; charset="us-ascii" \n Content-Transfer-Encoding: quoted-printable \n';
email +='Hi Andrew. Here are the customer service names and telephone numbers I promised you.';
httpReq.setBody(email);
System.debug(httpReq.getBody());
Http http = new Http();
HttpResponse response = http.send(httpReq);
I am getting error like
<AccessDeniedException>
<Message>Unable to determine service/operation name to be authorized</Message>
</AccessDeniedException>
Kindly please help me where i am doing wrong .Thanks in advance
Take another look at the documentation. There are several issues with your code.
SES expects an HTTP POST with all of the parameters strung together consistent with application/x-www-form-urlencoded POST requests.
Your HTTP request needs to be Content-type: application/x-www-form-urlencoded, not multipart/mixed... -- that's part of the raw message you're trying to send.
You are mixing up things that should be in the body, and setting HTTP request headers, instead. For example, these are also incorrect:
httpReq.setHeader('From','sample#gmail.com');
httpReq.setHeader('To','sample#gmail.com');
httpReq.setHeader('Subject','Hello');
These should go in the request body, not in the HTTP request headers. Also, the values are urlencoded. From the example code:
Action=SendEmail
&Source=user%40example.com
&Destination.ToAddresses.member.1=allan%40example.com
The line breaks were added for clarity.
Your interests might be best served by trying to successfully send a simple e-mail, first, and then later attempting to modify your code to support attachments, because you have numerous errors that will need to be corrected before this code will work properly.
http://docs.aws.amazon.com/ses/latest/DeveloperGuide/query-interface-requests.html
http://docs.aws.amazon.com/ses/latest/APIReference/API_SendRawEmail.html
I'm currently trying to get a POST request using multipart/form-data running to the Django REST framework. I've successfully run through some test requests via the interactive API screens, which work fine. I've then tried to convert these over to using a non-Session based auth strategy, and I've consistently got errors. The requests I've sent are of the form:
POST /api/logs/ HTTP/1.1
Host: host:8080
Connection: keep-alive
Content-Length: 258
Accept: application/json
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryTOhRsMbL8ak9EMQB
Authorization: Token -token-
------WebKitFormBoundaryx6ThtBDZxZNUCkKl
Content-Disposition: form-data; name="SubmittedAt"
2014-01-23T10:39:00
------WebKitFormBoundaryx6ThtBDZxZNUCkKl
Content-Disposition: form-data; name="Device"
CheeseDevice
------WebKitFormBoundaryx6ThtBDZxZNUCkKl--
Sadly, the result has been (for all the requests I've run):
{"Device": ["This field is required."], "SubmittedAt": ["This field is required."], "LogFile": ["This field is required."]}
Interestingly, I've been able to send chunks of JSON through to the endpoint, and they're accepted as expected, eg:
POST /api/logs/ HTTP/1.1
Content-Type: application/json
Host: host:8080
Connection: keep-alive
Content-Length: 35
Accept: application/json
Authorization: Token -token-
{
"Device": "CheeseDevice"
}
Returns:
{"SubmittedAt": ["This field is required."], "LogFile": ["This field is required."]}
As expected - it actually accepts the Device argument and only raises errors on the missing items. I'd switch to using JSON, but sadly cannot upload files with it...
Thanks in advance for any help!
Edit:
Further investigation (ie: writing a view method that returns the request data shows that request.DATA isn't getting populated, for some reason. Method I'm using to debug follows:
def test_create(self, request, pk=None):
return Response(request.DATA)
Edit 2:
Even further investigation (and dropping code chunks into the framework for debugging) indicates that the requests are getting caught up in _perform_form_overloading and never hitting the MultiPartParser. Not sure why this is occurring but I'll try and trace it further.
After delving down every level I could find...
Looks like the problem stems from the line endings - ie: the libs and request senders I've been using send the content through with "\n" (LF) endings, while the HTTP spec requires "\r\n" endings (CR,LF)
This hinges on the following code in the Django core, within http/multipartparser.py - in parse_boundary_stream:
header_end = chunk.find(b'\r\n\r\n')
For dev purposes (and because it's going to be way easier to patch at the Django end than in the clients...) I've switched the above line to:
header_end = chunk.replace("\r\n","\n").find(b'\n\n')
This updated code follows the recommendations in Section 19.3 of the HTTP/1.1 spec regarding Tolerant Applications and accepting LF instead of just CRLF - I'll try and get around to seeing if this is suitable for inclusion in the Django core.
Edit:
For reference, the patch is up on GitHub: https://github.com/tr00st/django/commit/9cf6075c113dd27e3743626ab0e18c6616488bd9
This could be due to malformed multipart post data.
Also possible that you don't have MultiPartParser installed, but I don't think that'll be it as you'd normally expect to see a 415 Unsupported Media Type response in that case.