Error 414 When sending invoice to Amazon MWS with _UPLOAD_VAT_INVOICE_ - amazon-web-services

I'm trying to send invoices to amazon mws through _UPLOAD_VAT_INVOICE_ following the java example in this guide:
Link
pdf file is a simple invoice of 85 kb
The error is status code 414 that is "Uri too long"
Debugging original amazon class MarketplaceWebServiceClient I see this:
if( request instanceof SubmitFeedRequest ) {
// For SubmitFeed, HTTP body is reserved for the Feed Content and the function parameters
// are contained within the HTTP header
SubmitFeedRequest sfr = (SubmitFeedRequest)request;
method = new HttpPost( config.getServiceURL() + "?" + getSubmitFeedUrlParameters( parameters ) );
getSubmitFeedUrlParameters method takes every parameter and add it to querystring. One of these parameters is contentMD5 from:
String contentMD5 = Base64.encodeBase64String(pdfDocument);
So there is a very large string representing pdf file passed as parameter. This causes error 414
But that class is the original one taken from MaWSJavaClientLibrary-1.1.jar
Can anybody help me please?
Thanks

For the last 2 days I was working on the same problem,
I changed like this and it works now
InputStream contentStream = new ByteArrayInputStream(pdfDocument);
String contentMD5 =computeContentMD5Header(new ByteArrayInputStream(pdfDocument));
public static String computeContentMD5Header(InputStream inputStream) {
// Consume the stream to compute the MD5 as a side effect.
DigestInputStream s;
try {
s = new DigestInputStream(inputStream,
MessageDigest.getInstance("MD5"));
// drain the buffer, as the digest is computed as a side-effect
byte[] buffer = new byte[8192];
while (s.read(buffer) > 0)
;
return new String(
org.apache.commons.codec.binary.Base64.encodeBase64(s
.getMessageDigest().digest()), "UTF-8");
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
} catch (IOException e) {
throw new RuntimeException(e);
}
}

Related

S3AbortableInputStream : Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. Warning when reading only ObjectMetadata

I am using <aws.java.sdk>1.11.637</aws.java.sdk> with Spring boot 2.1.4.RELEASE.
Code : which causes S3 Warning (Here i have to access Only getUserMetadata from S3Object and not whole Object Content)
private Map<String, String> getUserHeaders(String key) throws IOException {
Map<String, String> userMetadata = new HashMap<>();
S3Object s3Object = null;
try {
s3Object = s3Client.getObject(new GetObjectRequest(bucketName, key));
userMetadata.putAll(s3Object.getObjectMetadata().getUserMetadata());
} finally {
if (s3Object != null) {
s3Object.close();
}
}
return userMetadata;
}
Output : Whenever s3Object.close(); is invoked , then i see warning on console with below message
{"msg":"Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.","logger":"com.amazonaws.services.s3.internal.S3AbortableInputStream","level":"WARN","component":"demo-app"}
My Investigation for error cause:
I further checked https://github.com/aws/aws-sdk-java/blob/c788ca832287484c327c8b32c0e2b0090a74c23d/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/internal/S3AbortableInputStream.java#L173-L187 and it says if _readAllBytes() is not true (in my case where i am using S3Object just to getUserMetadata and not whole stream content) then there would be warning always.
Questions:
a) How S3Object.close is leading to invoke S3AbortableInputStream.close
as I assume code inside S3Object.close https://github.com/aws/aws-sdk-java/blob/c788ca832287484c327c8b32c0e2b0090a74c23d/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/model/S3Object.java#L222 only invoke SdkFilterInputStream.close via is.close();
b) How should I get rid of these warnings when i want use S3Object only to read meta data and not whole object content.
Maybe try using this API function designed to retrieve only the metadata for an S3 object:
ObjectMetadata getObjectMetadata(GetObjectMetadataRequest getObjectMetadataRequest)
throws SdkClientException,
AmazonServiceException
So change your code to:
ObjectMetadata s3ObjectMeta = null;
s3ObjectMeta = s3Client.getObjectMetadata(new GetObjectMetadataRequest(bucketName, key));
userMetadata.putAll(s3ObjectMeta.getUserMetadata());

Response not coming back from one service to another service - microservices

I am facing strange issue where response as string not coming back from one service to another service. we have created microservices where one service is calling another service. i can see response printed in logs . after that line immediately i am returning that response but its coming back as null.
I created similar method with same code and it works fine. I have put code for calling service and service method from which i am returning response.
controller:
#RequestMapping(value = "/test/save", method = RequestMethod.POST)
#ResponseBody
public String save(#RequestBody Calculation calculation,
HttpServletRequest request) {
logger.info("In .save");
String result = "false";
try {
result = CalService.save(calculation);
logger.info("Response from service is :" + result);
} catch (Exception e) {
logger.error("Exception occured in save:", e);
}
return result;
}
method call client :
public String saveCal(Calculation calculation) {
String result = null;
try {
logger.info("In save");
MultiValueMap<String, String> headers = new LinkedMultiValueMap<String, String>();
headers.add("REMOTE_USER", "test");
HttpEntity<Calculation> request = new HttpEntity<Calculation>(Calculation, headers);
RestTemplate template = new RestTemplate();
template.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
result = template.postForObject(url+"/test/save", request, String.class);
} catch (Exception e) {
logger.error("Exception occured in SaveMddMd", e);
result = "fail";
}
logger.info("Save"+result);
return result;
}
result returned is success or failure.
I can see result printed in controller as success but when it comes back to client it prints as null. I created exact same method with different signature which returns result as success. we are using microservices here.
Jordan

Whats the Efficient way to call http request and read inputstream in spark MapTask

Please see the below code sample
JavaRDD<String> mapRDD = filteredRecords
.map(new Function<String, String>() {
#Override
public String call(String url) throws Exception {
BufferedReader in = null;
URL formatURL = new URL((url.replaceAll("\"", ""))
.trim());
try {
HttpURLConnection con = (HttpURLConnection) formatURL
.openConnection();
in = new BufferedReader(new InputStreamReader(con
.getInputStream()));
return in.readLine();
} finally {
if (in != null) {
in.close();
}
}
}
});
here url is http GET request. example
http://ip:port/cyb/test?event=movie&id=604568837&name=SID&timestamp_secs=1460494800&timestamp_millis=1461729600000&back_up_id=676700166
This piece of code is very slow . IP and port are random and load is distributed so ip can have 20 different value with port so I dont see bottleneck .
When I comment
in = new BufferedReader(new InputStreamReader(con
.getInputStream()));
return in.readLine();
The code is too fast.
NOTE: Input data to process is 10GB. Using spark to read from S3.
is there anything wrong I am doing with BufferedReader or InputStreamReader any alternative .
I cant use foreach in spark as I have to get the response back from server and need to save JAVARdd as textFile on HDFS.
if we use mappartition code something as below
JavaRDD<String> mapRDD = filteredRecords.mapPartitions(new FlatMapFunction<Iterator<String>, String>() {
#Override
public Iterable<String> call(Iterator<String> tuple) throws Exception {
final List<String> rddList = new ArrayList<String>();
Iterable<String> iterable = new Iterable<String>() {
#Override
public Iterator<String> iterator() {
return rddList.iterator();
}
};
while(tuple.hasNext()) {
URL formatURL = new URL((tuple.next().replaceAll("\"", ""))
.trim());
HttpURLConnection con = (HttpURLConnection) formatURL
.openConnection();
try(BufferedReader br = new BufferedReader(new InputStreamReader(con
.getInputStream()))) {
rddList.add(br.readLine());
} catch (IOException ex) {
return rddList;
}
}
return iterable;
}
});
here also for each record we are doing same .. isnt it ?
Currently you are using
map function
which creates a url request for each row in the partition.
You can use
mapPartition
Which will make the code run faster as it creates connection to the server only once , that is only one connection per partition.
A big cost here is setting up TCP/HTTPS connections. This is exacerbated by the fact that Even if you only read the first (short) line of a large file, in an attempt to re-use HTTP/1.1 connections better, modern HTTP clients try to read() to the end of the file, so avoiding aborting the connection. This is a good strategy for small files, but not for those in MB.
There is a solution there: set the content-length on the read, so that only a smaller block is read in, reducing the cost of the close(); the connection recycling then reduces HTTPS setup costs. This is what the latest Hadoop/Spark S3A client does if you set fadvise=random on the connection: requests blocks rather than the entire multi-GB file. Be aware though: that design is actually really bad if you are going byte-by-byte through a file...

How to get value of 'CARBON_HOME' in java code

I was trying to implement an Axis2 service that receives user requests and publishes them as events to a CEP using carbon databridge thrift (via 'org.wso2.carbon.databridge.agent.thrift.DataPublisher')
I followed the code sample provided in wso2cep-3.1.0/samples/producers/activity-monitor
please see the following code snippet
public class GatewayServiceSkeleton{
private static Logger logger = Logger.getLogger(GatewayServiceSkeleton.class);
public RequestResponse request(Request request)throws AgentException,
MalformedStreamDefinitionException,StreamDefinitionException,
DifferentStreamDefinitionAlreadyDefinedException,
MalformedURLException,AuthenticationException,DataBridgeException,
NoStreamDefinitionExistException,TransportException, SocketException,
org.wso2.carbon.databridge.commons.exception.AuthenticationException
{
final String GATEWAY_SERVICE_STREAM = "gateway.cep";
final String VERSION = "1.0.0";
final String PROTOCOL = "tcp://";
final String CEPHOST = "cep.gubnoi.com";
final String CEPPORT = "7611";
final String CEPUSERNAME = "admin";
final String CEPPASSWORD = "admin";
Object[] metadata = { request.getDeviceID(), request.getViewID()};
Object[] correlationdata = { request.getSessionID()};
Object[] payloaddata = {request.getBucket()};
KeyStoreUtil.setTrustStoreParams();
KeyStoreUtil.setKeyStoreParams();
DataPublisher dataPublisher = new DataPublisher(PROTOCOL + CEPHOST + ":" + CEPPORT, CEPUSERNAME, CEPPASSWORD);
//create event
Event event = new Event (GATEWAY_SERVICE_STREAM + ":" + VERSION, System.currentTimeMillis(), metadata, correlationdata, payloaddata);
//Publish event for a valid stream
dataPublisher.publish(event);
//stop
dataPublisher.stop();
RequestResponse response = new RequestResponse();
response.setSessionID(request.getSessionID());
response.setDeviceID(request.getDeviceID());
response.setViewID(request.getViewID());
response.setBucket(request.getBucket());
return response;
}
there is also a utility class that set the key store parameters as following
public class KeyStoreUtil {
static File filePath = new File("../../../repository/resources/security");
public static void setTrustStoreParams() {
String trustStore = filePath.getAbsolutePath();
System.setProperty("javax.net.ssl.trustStore", trustStore + "/client-truststore.jks");
System.setProperty("javax.net.ssl.trustStorePassword", "wso2carbon");
}
public static void setKeyStoreParams() {
String keyStore = filePath.getAbsolutePath();
System.setProperty("Security.KeyStore.Location", keyStore + "/wso2carbon.jks");
System.setProperty("Security.KeyStore.Password", "wso2carbon");
}
}
I uploaded the service into a wso2as-5.2.1, and called the service using SOAPUI
the request returned an error message "cannot borrow client for TCP"
I debug, and found out the problem might lies with the class 'KeyStoreUtil',
where the 'filePath' somehow retuned a 'null',
static File filePath = new File("../../../repository/resources/security");
and caused the failure on this line
DataPublisher dataPublisher = new DataPublisher(PROTOCOL + CEPHOST + ":" + CEPPORT, CEPUSERNAME, CEPPASSWORD);
I guess it could be a better idea if I use the value of "CARBON_HOME" to figure out the location of Key Store
so my question is :
How may I be able to get the value of 'CARBON_HOME' in the Java code?
that said. If you think a bit more:
the service will be called numerous time; whileas the 'setTrustStoreParams' and the 'setKeyStoreParams' will only be needed to executed once at the server/service initiate.
So, are there any even better ways to remove 'setTrustStoreParams' and 'setKeyStoreParams' out of the service code, or implement as configurable items?
Please advise
thanks
so my question is :
How may I be able to get the value of 'CARBON_HOME' in the Java code?
You should use the property carbon.home like following which will retrieve the WSO2 product's home directory.
System.getProperty("carbon.home");

Decoded response in Java ME (Nokia Asha)

I am implementing small Java ME app. This app gets some data from 3rd patty resource and needs to be authenticated before. I do first call for get cookies (it was easy), and the second call with this cookies for get data. I googled a little how to do it, and found next solution - Deal with cookie with J2ME
I have changed this code to next for my purpose:
public void getData(String url,String cookie) {
HttpConnection hpc = null;
InputStream is = null;
try {
hpc = (HttpConnection) Connector.open(url);
hpc.setRequestProperty("cookie", cookie);
hpc.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
hpc.setRequestProperty("Accept-Encoding", "gzip, deflate");
hpc.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
is = hpc.openInputStream();
int length = (int) hpc.getLength();
byte[] response = new byte[length];
is.read(response);
String strResponse = new String(response);
} catch (Exception e) {
System.out.println(e.getMessage() + " " + e.toString());
} finally {
try {
if (is != null)
is.close();
if (hpc != null)
hpc.close();
} catch (Exception e) {}
}
}
I get something like to the next
??ÑÁNÃ0à;O±(²§M}A-?#
.?PYS¨Ôe¥Í#\üìde??XÊo}Vâ]hk?­6ëµóA|µvÞz'Íà?wAúêmw4í0?ÐÆ?ÚMW=?òêz CÛUa:6Ö7¼T?<oF?nh6[_0?l4?äê&)?çó³?ÅÕúf¨ä(.? ªDÙ??§?ÊP+??(:?Á,Si¾ïA¥ã-jJÅÄ8ÊbBçL)gs.S.þG5ÌÀÆéX}CÁíÑ-þ?BDK`²?\¶?ó3I÷ô±e]°6¬c?q?Ó?¼?Y.¯??Y?%?ÏP1è?ìw;?È Ò??e
|ôh0?
How can I decode this?
Stupid me. I didn't take to consideration next code: hpc.setRequestProperty("Accept-Encoding", "gzip, deflate"); I get coded in ZIP response and everything that I need it decode it.