I want to interact with Amazon Web Services DynamoDB in a Google Sheet via code in a GoogleScript. However, I cannot figure out how to integrate the AWS SDK. I am hoping to avoid having to write a library to handle the integration via the AWS HTTP API, as there are JavaScript and Java SDKs available for the SDK. Help?
(I've done some pretty extensive Google and Stack Overflow searches. This is the closest thing I've found to an answer, but that is the Google App Engine, not the Google Apps Script.)
Thanks!
I just made a function that does the basic authentication for any api request you want to make. You still might have to get some of the headers in order but it does most of the hard work for you.
For example:
function myFunction() {
AWS.init("MY_ACCESS_KEY", "MY_SECRET_KEY");
var instanceXML = AWS.request('ec2', 'us-east-1', 'DescribeInstances', {"Version":"2015-10-01"});
...
}
I put it in a repo with some documentation for use. Here's the link: https://github.com/smithy545/aws-apps-scripts
I recently added my own derivative of the "AWS via Google Apps Script" here: https://github.com/neilobremski/gas/blob/main/GasAWS.js
The JavaScript code here is a bit more modern and specifically uses the Utilities available in Google Apps Script for the SHA-256 stuff rather than the abandoned (but excellent!) Crypto-JS library.
Usage is the same except I moved the region parameter and have a global default. I did this because often resources are in the same region and repeatedly specifying it is a pain.
Responding if anyone else finds this useful, one of the responses above gave a good hint for me to get started but not able to add comment on it.
Steps in below tutorial walks through setting up API Gateway, Lambda and DynamoDb. Once setup, you can directly use URLFetch in GAS to your API Gateway.
http://docs.aws.amazon.com/lambda/latest/dg/with-on-demand-https-example.html
Code in AWS link is complete and needs no additional setup to work directly using URL Fetch. You may however want to enable security on API Gateway.
On GAS side -
var headers = {
"x-api-key" : AWS_KEY
};
var event = {};
event.operation = "create";
event.tableName = AWS_DB_NAME;
event.payload.Item = yourDataObjwithKey;
var options = {
"method":"POST",
"headers":headers,
"payload":JSON.stringify(event)
};
var url = AWS_GW;
var response = UrlFetchApp.fetch(url, options);
I am about to implement this and it looks like the best way is to use a Lambda function to query the DynamoDB via API Gateway.
It's really not that hard if you know NodeJS.
Related
We're new to Amazon Seller Partner-API. Need to invoke certain Amazon SP-APIs for an integration workflow. For some internal reasons, using Amazon SDKs is a secondary option. With our conventional approach, we're able to interact with most APIs, in this case the AWS Request signing & Signature generation is where we're stuck.
As per Amazon using SDK handles it all internally. Is it possible to use a command line utility like - AWS CLI to interact with SP-APIs? Not sure if this is feasible. Found this - amazon-sp-api but not sure if it is stable / reliable.
I believe there should be ways to interact with SP-API from command line. If not, atleast there should be a tool that is able to produce AWS Request signature (given the request info, key etc...).
Kindly share your experience and expertise. We're new to AWS, so if I'm confusing AWS with SP-API (esp for Request signing - I believe both use the same mechanism) pls point it out.
The link you shared to amz.tools does not look like a command line interface. It is just an SDK generated in NodeJS. There is not way to connect to the API via command line. You can use Postman if you want to avoid SDKs.
And yes, AWS is not the same thing as SP API.
You can search github for SDKs generated on other languages; some seem to have a lot of use.
We generated our own SDK in C# because others didn't fit out criteria.
I am looking for a language / framework or a method by which I can build API / web application code such that it can run on Serverless compute's like aws lambda and the same code runs on a dedicated compute system like lightsail or EC2.
First I thought of using Docker to do this but AWS Lambda entry point is a specific function signature which is very different than Spring Controllers. Is there a solution available currently?
So basically when I run it in lambda - it will have cold start issue, later when the app is ready or get popular I would like to move it to a EC2 instance for better performance and higher traffic load.
I want to start right in this situation so that later it can be easy to port and resolve the performance issue's
I'd say; no this is not possible easily.
When you are building an api that you'd want to run on lambda's you most likely will be using an API Gateway which takes care of your routing to different lambda functions (best practice). So the moment you would me working on an api like this migrating to EC2 would be a nightmare as you would need to rebuild the whole application a more of a monolith application which could run on EC2.
I would honestly commit to either run it on EC2/Containers or run it on Lambda, if cold start is your main issue with Lambda's you might wanna look into Lambda Snapstart for Java or use another language like Typescript/Python.
After some correct keywords in google I finally got what I was looking for, checkout this blog and code library shared by AWS which helps you convert the request and response of the request as per the framework required http request
Running APIs Written in Java on AWS Lambda: https://aws.amazon.com/blogs/opensource/java-apis-aws-lambda/
Repo Code: https://github.com/awslabs/aws-serverless-java-container
Thanks Ricardo for your response - will do check out Lambda Snapstart for sure and try it as well. I have not tested out this completely but it looks promising to some extent.
at my app, I was able to track all the lambda, APIGateway and DynamoDB requests through AWS-X-Ray.
I am doing the same as the answer in this question:
Adding XRAY Tracing to non-rest functions e.g., SQS, Cognito Triggers etc
However, how would be the case of S3, SQS or other services/non-rest functions ??
I saw some old code that does not even use aws-sdk, the dependencies are import direct like:
import {S3Client, S3Address, RoleService, SQSService} from '#sws/aws-bridge';
So, in these cases, how to integrate/activate AWS-XRay?
Thank you very much in advance!
Cheers,
Marcelo
At the moment Lambda doesn't support continuing traces from triggers other than REST APIs or direct invocation
The upstream service can be an instrumented web application or another Lambda function. Your service can invoke the function directly with an instrumented AWS SDK client, or by calling an API Gateway API with an instrumented HTTP client.
https://docs.aws.amazon.com/xray/latest/devguide/xray-services-lambda.html
In every other case it will create its own, new Trace ID and use that instead.
You can work around this yourself by creating a new AWS-Xray segment inside the Lambda Function and using the incoming TraceID from the event. This will result in two Segments for your lambda invocation. One which Lambda itself creates, and one which you create to extend the existing trace. Whether that's acceptable or not for your use case is something you'll have to decide for yourself!
If you're working with Python you can do it with aws-xray-lambda-segment-shim.
If you're working with NodeJS you can follow this guide on dev.to.
If you're working with .NET there are some examples on this GitHub issue.
I've got a piece of code that I need to make available over the 'Net. It's a perfect fit for an AWS Lambda with an HTTP API on top - a stateless, side effect free, rather CPU intensive function, blob in, blob out. It's written in C#/.NET, but it's not pure .NET, it makes use of the UWP API, therefore requires Windows Server 2016.
AWS Lambdas only run on Linux hosts, even C# ones. Is there any way to deploy this piece in the Amazon cloud in serverless manner - maybe something other than a Lambda? I know I can go with a EC2 VM, but this is the very kind of thing serverless architecture was invented for.
Lambda is the only option for serverless computing on AWS and Lambda functions run only on Linux machines.
If you need to run serverless functions in a Windows machine, try Azure Functions. That's the Lambda equivalent in the Microsoft cloud. I'm not sure if it runs in a Windows Server 2016 machine and couldn't find any reference to the platform, but I would expect that, as a brand new service, they are using their own edge tech.
To confirm if the platform is what you need, try this function:
using System.Management;
using System.Net;
using System.Threading.Tasks;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
// Get OS friendly name
// http://stackoverflow.com/questions/577634/how-to-get-the-friendly-os-version-name
var caption = (from x in new ManagementObjectSearcher("SELECT Caption FROM Win32_OperatingSystem").Get().Cast<ManagementObject>()
select x.GetPropertyValue("Caption")).FirstOrDefault();
string name = caption != null ? caption.ToString() : "Unknown";
// the function response
return req.CreateResponse(HttpStatusCode.OK, name);
}
I think yoy can achieve this via combination of CodeDeploy service and AWS CodePipeline.
Refer to this article:
http://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-windows.html
to learn how to deploy code via CodeDeploy. Later see this article:
http://docs.aws.amazon.com/codepipeline/latest/userguide/getting-started-4.html
to learn how you can configure aws Pipline to call Code Deploy and later execute your batch job on created windows machine (note: you will probably want to use
S3 instead of Github - which is possible with CodePipeline).
I would consider to bootstrap whole such configuration via script - using aws cli - this way you can clean up easily your resources like this:
:aws codepipeline delete-pipeline --name "MyJob"
Of course you can configure the pipeline via aws web console and leave the pipeline configured to run your code on regular basis.
I'm quite interested in beginning some development using Amazon SQS, perhaps SimpleDB too, my question is this, are there any open source solutions that mimic the functionality, just for the purposes of development. I've already encountered the Eucalyptus project (http://open.eucalyptus.com) for creating an EC-esque cloud.
I've not had any success with google, I suspect it's because the cost of entry is so inexpensive, but still, does anyone know of anything like this?
For SQS I wrote ElasticMQ, which you can run either embedded (it's written in Scala, so runs on the JVM) or stand-alone. It has both persistent and in-memory modes, the first being good for dev, second for testing.
If you need a test double for more than just SQS, you can try LocalStack.
To simulate SQS, it internally uses ElasticMQ mentioned by adamw.
You can start LocalStack via Docker, for example, and it will start the following services:
API Gateway at http://localhost:4567
Kinesis at http://localhost:4568
DynamoDB at http://localhost:4569
DynamoDB Streams at http://localhost:4570
Elasticsearch at http://localhost:4571
S3 at http://localhost:4572
Firehose at http://localhost:4573
Lambda at http://localhost:4574
SNS at http://localhost:4575
SQS at http://localhost:4576
Redshift at http://localhost:4577
ES (Elasticsearch Service) at http://localhost:4578
SES at http://localhost:4579
Route53 at http://localhost:4580
CloudFormation at http://localhost:4581
CloudWatch at http://localhost:4582
SSM at http://localhost:4583
Some of the Amazon SDKs have "mock" mode, which is:
The mock service is an alternate way
to use the sample code. The service
doesn't call AWS, but instead returns
a set response that you can modify to
suit your needs (the XML response
files are in the Mock directory). The
mock service makes it easy for you to
test how your application handles
different responses.
For SQS, it appears the Perl and PHP SDKs have mock mode. I know that the .NET SDK for Amazon RDS also has the mock mode.
The Java SDK doesn't contain mock implementations:
The client mock implementations have been removed. Instead, developers
are encouraged to use more flexible and full featured mock libraries,
such as EasyMock, jMock
If the SDK you will be using doesn't have the mock mode available, you could probably create your own similar type of thing which returns the preconfigured responses instead of actually hitting up the service.
See here for more info
GoAws - https://github.com/p4tin/goaws - was just released as beta. (disclaimer - I am the developer).
Regarding the Java SDK, it does no longer contain mock implementations:
The client mock implementations have been removed. Instead, developers
are encouraged to use more flexible and full featured mock libraries,
such as EasyMock, jMock
If you are in .NET or Mono you can try Stratosphere. It has local implementations that mimic SimpleDB, SQS and S3. For SimpleDB mock implementation it uses SQLite, for SQS and S3 it stores messages/objects in file system.
if you need to simulate SNS as well as SQS you can check out: Yopa