I have developed a stored process to receive an XML file as a response to a web service call and deployed it as a web service - something similar to the example here.
It is successful and working fine in receiving an XML file which is of ~100KB but failing to receive similar file which is about 3MB. The other system sending the response seems to throw the below error
HTTP Response Code 413 for 'https://mystoredprocessURL'. I understand that this is related payload too large in size.
Could you suggest me how to configure the length of payload size that has to be received so that the stored process can receive a larger file. Tried to research but could not find anything relevant.
my first idea. I hope this helps, but that's just a hint:
SAS SMC-->plug-ins tab-->Application manager-->configuration manager-->
SAS Application infrastucture-->BI web services for java 9.4-->WebServiceMaker
-->Settings tab-->Attachemnt optimized threshold block.
Maybe the default size is 2048.
Related
I have .net core MVC API implementation. In my controller I try to query for 800 records from DB. In result my response body size is abound 6MB. In that case response time is over 6s. My service is in AWS cloud.
I made several tests to make diagnostic of service. In all these scenarios I still ready 800 records from DB. Here is list of my experiments:
Return only 10 records - my response time were under 800ms always and size of response body 20kB.
Return only 100 records - my response time were over 800ms but not timeouts and size of response body 145kB
Try to use my custom json serialization in controller await JsonSerializer.SerializeAsync(HttpContext.Response.Body, limitedResult); - a bit better result but like 10% only
Return only 850 records - my response time were over 6s but and size of response body 6MB
In service I don't have problem with memory or with restart service.
Looks like for Kastrel the problem is to serve large response data.
My objections are connected with buffers I/O which for large response will use disk what can affect performance of AWS docker image.
Question is how to optimize Kastrel to serve large response size?
UPDATE:
I enabled zip compression on server side. My files are compressed quite good because of json format. But result is exact the SAME. Network bandwidth is not a problem. So looks like between my controller and compression is bottleneck. Any suggestion how to configure .net core service to handle large response (>1 MB)?
Since you are already sure that network is not an issue - this mostly points to time being spent in serialization. You can try running the application on local machine and using a profiler such as PerfView to see where the time is spent most for big json.
Question on why pubsub requests seem to trigger such a high number of 503 errors? Is this something common? It seems other people see something similar but a majority of my requests end up that way
Similar to
Google Pubsub: UNAVAILABLE: The service was unable to fulfill your request
Catch error code from GCP pub/sub
This is expected behavior. Streaming pull, which is used by the client libraries, creates a bidirectional stream for receiving messages and sending back acknowledgements. These streams stay open for long periods of time and don't close with a successful response code when messages are received, they terminate with an error condition when the stream disconnects, perhaps due to a restart on the part of the server receiving the request or because of brief network blip. Therefore, even if you are receiving messages successfully, you'll still see error response codes for all of the streams themselves. The new streaming pull docs address this question directly.
I am developing a CICS web service requestor application to consume a distributed web service.
I used the web services assistant DFHWS2LS to transform the wsdl to copybooks successfully.
I have no problem issuing the PUT CONTAINER and INVOKE SERVICE api commands, but when I issue GET CONTAINER I am not receiving any containers or data. No response codes or error messages, but no data. Any ideas on how to debug this would be greatly appreciated.
Thanks,
I have never seen RESP be DFHRESP(NORMAL) and RESP2 be zero and have nothing returned by the server.
Verify the WSDL specifies that something is, in fact, returned by the web service.
Check the RESP and RESP2 values returned by the INVOKE SERVICE API. You don't mention these explicitly, and I presume the former is DFHRESP(NORMAL) and the latter is 0, but you might have coded NOHANDLE so I thought I'd ask.
Take a look in the TD queue mapped to CSSL (the default is the MSGUSR DD) for your CICS region. This is where CICS logs messages when it runs into an error while processing your SOAP request. Look for messages prefixed DFHPI.
Try pinging the endpoint from a TSO session running on the same LPAR as your CICS region, it's possible you're being stopped by a firewall.
In your comment you indicate the requestor is "seeing whitespace on the <SOAP-ENV:Envelope tag>". This isn't something under your direct control. The CICS "plumbing" code takes care of formatting the SOAP message. You may want to ask your CICS Systems Programmer to look for APARs related to the problem and install any associated PTFs.
You could verify the requestor's claim by using the transport handler in Appendix A.3 of this redbook. You'll have to modify your pipeline configuration file to execute the handler.
I have a mobile device that is constantly recording information. The information is stored on the device's local database. Every few minutes, the device will upload the data to a server through a REST API - sometimes the uploaded data corresponds to dozens of records from the same table. Right now, the server responds with
{status: "SAVED"}
if the data is saved to the server.
In the interest of being 100% sure that the data is actually uploaded (so the device won't attempt to upload it again), is that simple response enough? Or should I be hashing the incoming data and responding with it, or something similar? Perhaps I should send back the local row ids of the device's table's rows?
I think it's fine to have a very simple "SUCCESS" response if the entire request did indeed successfully save.
However, I think that when there is a problem, your response needs to include the IDs (or some other unique identifier) of the records that failed to save so that they can be queued to be resent.
If the same records fail multiple times, you might need to log the error or display it so that further action can be taken.
A successful response could be something as simple as:
<response>
<status>1</status>
</response>
An error response could be something like:
<response>
<status>0</status>
<errorRecords>
<id>441</id>
<id>8462</id>
<id>12</id>
</errorRecords>
</response>
You could get fancy and have different status codes that mean different, more specific messages.
I'm working with IMAP and attempting to find a way to find the size of an IMAP message prior to actually uploading it to a server from a client (not a server-to-server transfer).
Once the message is actually on the server, finding its size is relatively easy - you can just use a BODY structure then use mail_fetchstructure, as in:
BODY *bodyStructure;
mail_fetchstructure(MailStream, msgno, &bodyStructure);
printf("Message size: %u\n", bodyStructure->size.bytes);
However, that will only work after you upload the message to the server. I'm trying to find the size of the IMAP message, as it will be, once it has been uploaded to the server. Does anyone have any ideas?
Which C++ framework are you using? In python you upload the message as a string, so you could just take the length of the string...