We're new to Amazon Seller Partner-API. Need to invoke certain Amazon SP-APIs for an integration workflow. For some internal reasons, using Amazon SDKs is a secondary option. With our conventional approach, we're able to interact with most APIs, in this case the AWS Request signing & Signature generation is where we're stuck.
As per Amazon using SDK handles it all internally. Is it possible to use a command line utility like - AWS CLI to interact with SP-APIs? Not sure if this is feasible. Found this - amazon-sp-api but not sure if it is stable / reliable.
I believe there should be ways to interact with SP-API from command line. If not, atleast there should be a tool that is able to produce AWS Request signature (given the request info, key etc...).
Kindly share your experience and expertise. We're new to AWS, so if I'm confusing AWS with SP-API (esp for Request signing - I believe both use the same mechanism) pls point it out.
The link you shared to amz.tools does not look like a command line interface. It is just an SDK generated in NodeJS. There is not way to connect to the API via command line. You can use Postman if you want to avoid SDKs.
And yes, AWS is not the same thing as SP API.
You can search github for SDKs generated on other languages; some seem to have a lot of use.
We generated our own SDK in C# because others didn't fit out criteria.
I have a query related to the google cloud storage for julia application.
Currently, I am hosting a julia application (docker container) on GCP and would like to allow the app to utilize cloud storage buckets to write and read the data.
I have explored few packages which promise to do this operation.
GoogleCloud.jl
This package in the docs show a clear and concise representation of the implementation. However, adding this package result in incremental compilation warning with many of the packages failing to compile. I have opened an issue on their github page : https://github.com/JuliaCloud/GoogleCloud.jl/issues/41
GCP.jl
The scope is limited, currently the only support is for BigQuery
Python package google
This is quite informative and operational but will take a toll on the code's performance. But do advise if this is the only viable option.
I would like to know are there other methods which can be used to configure a julia app to work with google storage?
Thanks look forward to the suggestions!
GCP.jl is promising plus you may be able to do with gRPC if Julia support gRPC (see below).
Discovery
Google has 2 types of SDK (aka Client Library). API Client Libraries are available for all Google's APIs|services.
Cloud Client Libraries are newer, more language idiosyncratic but only available for Cloud. Google Cloud Storage (GCS) is part of Cloud but, in this case, I think an API Client Library is worth pursuing...
Google's API (!) Client Libraries are auto-generated from a so-called Discovery document. Interestingly, GCP.jl specifically describes using Discovery to generate the BigQuery SDK and mentions that you can use the same mechanism for any other API Client Library (i.e. GCS).
NOTE Explanation of Google Discovery
I'm unfamiliar with Julia but, if you can understand enough of that repo to confirm that it's using the Discovery document to generate APIs and, if you can work out how to reconfigure it for GCS, this approach would provide you with a 100% fidelity SDK for Cloud Storage (and any other Google API|service).
Someone else tried to use the code to generate an SDK for Sheets and had an issue so it may not be perfect.
gRPC
Google publishes for the subset of its services that support gRPC. If you'd prefer to use gRPC, it ought be possible to use the Protobufs in Google's repo to define a gRPC client for Cloud Storage
I checked github code for transcribe streaming options and it looks like there is no transcribe streaming mentions neither in docs nor in config file: src/botocore/botocore/data/transcribe/2017-10-26/service-2.json.
But I see documentation for Ruby: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/TranscribeStreamingService.html
This is why I believe it makes sense to do it using scripting (not compiled) language.
Are there any plans to add it in the future? If yes, when approximately?
Or am I simply missing it?
I saw documentation describes low level communication for this kind of API, but I want to save dev time re-using the lib.
python sdk for transcribe streaming is not available yet.
Are there any mocks for AWS SWF or EMR available anywhere? I tried looking at some other AWS API mocks such as https://github.com/atlassian/localstack/ or https://github.com/treelogic-swe/aws-mock but they don't have SWF or EMR which are the things that would be really painful to reproduce. Just not sure if anyone has heard of a way to locally test things that use dependencies on those services.
The "moto" project (https://github.com/spulec/moto) groups mocks for the "boto" library (the official python sdk for AWS), and it has mocks for basic things in SWF (disclaimer: I'm the author who contributed them) and EMR.
If you happen to work in Python they're ready to use via a #mock_swf decorator (use 0.4.x for boto 2.x or 1.x for boto 3.x). If you work with another language, moto supports a server mode that mimics an AWS endpoint. The SWF service is not provided out of the box yet, but with a minor change in "moto/backends.py" you should be able to try using it. I think the EMR service works out of the box.
Should you have any issue with the SWF mocks in this project, you can file an issue on the Github project, don't hesitate to cc me directly (#jbbarth), I can probably help improving this.
I'm quite interested in beginning some development using Amazon SQS, perhaps SimpleDB too, my question is this, are there any open source solutions that mimic the functionality, just for the purposes of development. I've already encountered the Eucalyptus project (http://open.eucalyptus.com) for creating an EC-esque cloud.
I've not had any success with google, I suspect it's because the cost of entry is so inexpensive, but still, does anyone know of anything like this?
For SQS I wrote ElasticMQ, which you can run either embedded (it's written in Scala, so runs on the JVM) or stand-alone. It has both persistent and in-memory modes, the first being good for dev, second for testing.
If you need a test double for more than just SQS, you can try LocalStack.
To simulate SQS, it internally uses ElasticMQ mentioned by adamw.
You can start LocalStack via Docker, for example, and it will start the following services:
API Gateway at http://localhost:4567
Kinesis at http://localhost:4568
DynamoDB at http://localhost:4569
DynamoDB Streams at http://localhost:4570
Elasticsearch at http://localhost:4571
S3 at http://localhost:4572
Firehose at http://localhost:4573
Lambda at http://localhost:4574
SNS at http://localhost:4575
SQS at http://localhost:4576
Redshift at http://localhost:4577
ES (Elasticsearch Service) at http://localhost:4578
SES at http://localhost:4579
Route53 at http://localhost:4580
CloudFormation at http://localhost:4581
CloudWatch at http://localhost:4582
SSM at http://localhost:4583
Some of the Amazon SDKs have "mock" mode, which is:
The mock service is an alternate way
to use the sample code. The service
doesn't call AWS, but instead returns
a set response that you can modify to
suit your needs (the XML response
files are in the Mock directory). The
mock service makes it easy for you to
test how your application handles
different responses.
For SQS, it appears the Perl and PHP SDKs have mock mode. I know that the .NET SDK for Amazon RDS also has the mock mode.
The Java SDK doesn't contain mock implementations:
The client mock implementations have been removed. Instead, developers
are encouraged to use more flexible and full featured mock libraries,
such as EasyMock, jMock
If the SDK you will be using doesn't have the mock mode available, you could probably create your own similar type of thing which returns the preconfigured responses instead of actually hitting up the service.
See here for more info
GoAws - https://github.com/p4tin/goaws - was just released as beta. (disclaimer - I am the developer).
Regarding the Java SDK, it does no longer contain mock implementations:
The client mock implementations have been removed. Instead, developers
are encouraged to use more flexible and full featured mock libraries,
such as EasyMock, jMock
If you are in .NET or Mono you can try Stratosphere. It has local implementations that mimic SimpleDB, SQS and S3. For SimpleDB mock implementation it uses SQLite, for SQS and S3 it stores messages/objects in file system.
if you need to simulate SNS as well as SQS you can check out: Yopa