I am wondering how one creates many "things" in the AWS IoT solution via API without using the AWS web interface since this is not realistic in case I want thousands or millions of things. I guess you could write some script utilizing the "aws" client described here "http://docs.aws.amazon.com/iot/latest/developerguide/thing-registry.html" but thats not optimal if I want to control it from another service.
I assumed there would be a RESTish API to do this but it doesn't seem like it if I read the docs: "You use the AWS IoT console or the AWS CLI to interact with the registry."
Anyone who created thousands/millions of things - how did you interact with AWS IoT?
Ok, so I found it. Here it is possible to manage all the AWS IoT things:
http://docs.aws.amazon.com/iot/latest/apireference/API_Operations.html
anujdeshpande has given some headsup on this area.
"Automatically" means you can look at the following steps:
When do you want to create a thing automatically? What is that triggers that create-thing process?
Check this link for Just In Time registration of the devices
You can create a Lambda function mapped to the AWS API Gateway which can invoke the AWS-IOT SDK to create a thing.
Look at this page where they have created 50+ things using Javascript using aws-sdk library using serverless architecture approach
Another interesting way to do this is to use Just-in-Time registration - basically you upload the CA certificate that you used to sign keys while manufacturing.
This has some awesome advantages -
Your thing won't be created till your device first tries to connect to AWS IoT. So no blanks on your AWS account.
You won't have to mix your AWS IoT thing creation with the manufacturing process. Which is great because you don't want to give your Chinese (most likely scenario) manufacturers direct access to your AWS account
More about this
Link below is an example of creating 50 Iot devices, and implementing a Serverless AWS IoT Backend with AWS Lambda and Amazon DynamoDB. You can vary the devices to the number you like; It is done using the earlier version of AWS Iot Platform. Have a look.
https://aws.amazon.com/blogs/compute/implementing-a-serverless-aws-iot-backend-with-aws-lambda-and-amazon-dynamodb/
Here is the code to create 50 Iot Things
var AWS = require('aws-sdk');
AWS.config.region = 'ap-northeast-1';
var crypto = require('crypto');
var endpoint = "<endpoint prefix>.iot.<region>.amazonaws.com";
var iot = new AWS.Iot();
var iotdata = new AWS.IotData({endpoint: endpoint});
var topic = "registration";
var type = "MySmartIoTDevice"
//Create 50 AWS IoT Things
for(var i = 1; i < 51; i++) {
var serialNumber = "SN-"+crypto.randomBytes(Math.ceil(12/2)).toString('hex').slice(0,15).toUpperCase();
var clientId = "ID-"+crypto.randomBytes(Math.ceil(12/2)).toString('hex').slice(0,12).toUpperCase();
var activationCode = "AC-"+crypto.randomBytes(Math.ceil(20/2)).toString('hex').slice(0,20).toUpperCase();
var thing = "myThing"+i.toString();
var thingParams = {
thingName: thing
};
iot.createThing(thingParams).on('success', function(response) {
//Thing Created!
}).on('error', function(response) {
console.log(response);
}).send();
//Checking all devices were created
iot.listThings().on('success', function(response) {
var things = response.data.things;
var myThings = [];
for(var i = 0; i < things.length; i++) {
if (things[i].thingName.includes("myThing")){
myThings[i]=things[i].thingName;
}
}
if (myThings.length == 50){
console.log("myThing1 to 50 created and registered!");
}
}).on('error', function(response) {
console.log(response);
}).send();
You can do that by using AWS IoT SDKs. You can find supported AWS IoT SDKs here. For doing that you will need proper credentials and policies which the simplest one would be AWS account credentials (Access key along with secret key from IAM or Cognito user credentials, ...).
For creating things automatically, you can use AWS IoT Device SDK for Python which is Boto3. You can find more information about it here. The following codes show the example of creating a thing in AWS IoT automatically using python and AWS credentials:
import boto3
client = boto3.client('iot')
response = client.create_thing(
thingName=[NameOfThing],
thingTypeName=[ThingType],
attributePayload={
'attributes': {
'string': 'string'
},
'merge': True|False
},
billingGroupName='string'
)
Related
I'm trying to find a resource in AWS, but I don't know what type of resource it was provisioned as.
I don't know AWS very well, but I'm having trouble finding the page in the portal that lists all of our resources in our account. How do I find it?
You can list all resources for all regions using the tag editor https://resources.console.aws.amazon.com/r/tags
You should select all regions individually from the 'Regions*' dropdown and select only 'All resource types' from the Resource types* dropdown. Leave Tags empty and press 'Find Resources'
You could use AWS Config, which provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.
An alternate approach, that I've used in the past, is to find resources that are not tagged. You can search for untagged resources using the tag editor, and that should find all resources (including ones that might not show up in AWS Config). This approach allows you to search across all regions whereas AWS Config works on a region basis.
As this is StackOverflow, I'm assuming it's a programming related question.
To do this programatically, create an app to read resources of specific types
Python Example
import boto3
ec2 = boto3.resource('ec2')
instances = ec2.instances
for instance in instances:
print(instance.id, instance.instance_type)
NodeJS Example
var AWS = require('aws-sdk');
AWS.config.update({region: 'us-west-2'});
var ec2 = new AWS.EC2({apiVersion: '2016-11-15'});
var params = { DryRun: false };
ec2.describeInstances(params, function(err, data) {
if (err) {
console.log("Error", err.stack);
} else {
console.log("Success", JSON.stringify(data));
}
});
I am using aws-amplify and aws-sdk in Angular JS/Typescript.
I can successfully access my AWS DynamoDB as an authenticated user.
I am also trying to add guest user access for a certain table in Dynamo, but I am struggling to understand how I would get a reference to the DynamoDB without any credentials.
My code looks like this at the moment
getDocumentClient() {
return Auth.currentCredentials()
.then(credentials => new AWS.DynamoDB.DocumentClient({ credentials: credentials }))
.catch(err => logger.debug('error getting document client', err));
How would I do something similar to get access to the DynamoDB as an unauthenticated guest user?
Cheers
Lee
Try makeUnauthenticatedRequest.
Here's an example with S3 - I've shown this because I know you can make requests to S3 from the AWS SDK as an unauthenticated user. I'm assuming that this will also work for DynamoDB but have not tested it.
var s3 = new AWS.S3();
var params = {
Bucket: 'mybucket'
};
s3.makeUnauthenticatedRequest('listObjects', params, callback);
The more strategic approach would be Amazon Cognito Identity Pools which support unauthenticated/guest identities. Cognito vends an identity and AWS credentials, and you can configure an IAM role allowing DynamoDB read access for unauthenticated identity types.
I think you can refer to what is mentioned in the blog post below.
https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/
The basic idea is to use API Gateway as a proxy for DynamoDB API. Permission to access DynamoDB is granted to API Gateway via execution role, and API Gateway is configured to open to public. In doing so, the flow will be as follows:
Web Browser <----- HTTPS -----> API Gateway <----- AWS Service Proxy integration -----> DynamoDB
You would think two of Amazon's products would integrate nicely together. You'd also think that Amazon would have proper documentation on their services. Both are horribly wrong.
I'm using Account Linking with Alexa and using AWS Cognito as my oauth provider. I am able to successfully link my account just fine, and with every subsequent alexa invocation I get an access token returned back to me. Now what?
I need to be able to access other AWS services such as dynamo, lambda, and iot from my alexa lambda function. Thought it would be as easy as something like this:
var accessToken = event.session.user.accessToken;
var params = {
IdentityPoolId: "{identity_pool_id}",
Logins : {
'cognito-idp.us-east-1.amazonaws.com/{user_pool_id}' : accessToken
}
};
AWS.config.region = 'us-east-1';
AWS.config.credentials = new AWS.CognitoIdentityCredentials(params);
AWS.config.credentials.get(function(err) {
if (err) {
console.log(err);
} else {
console.log('success');
console.log("Cognito Identity Id: " + AWS.config.credentials.identityId);
}
});
But of course it's not that simple. Can anyone help me out?
You need to assign appropriate "Policies" for the created "Role" under IAM. All of the AWS services works on policy based access permissions and these must be explicitly attached with the IAM role for that role to be able to access/run the underlying service on the behalf of that user role.
In summary you need to assign policies from IAM related to "IOT", "DynamoDB", "Lambda", etc.
Can someone tell me how to get access to AWS credentials in an AWS Lambda function?
I've searched the internet thoroughly but still haven't found anything that has helped me.
Im writing the function in Java. I think I should have access to the credentials with the context object in the HandleRequest method.
If it helps, I want to invoke a DynamoDB client and upload a record to the database.
I came into the same problem myself recently.
This certainly is a blind spot in AWS's Lambda documentation for Java, in my opinion.
This snippet in Java should work for you, assuming you're using the AWS SDK for Java Document API :
DynamoDB dynamodb = new DynamoDB(
new AmazonDynamoDBClient(new EnvironmentVariableCredentialsProvider()));
The main takeaway is to use the EnvironmentVariableCredentialsProvider to access the required credentials to access your other AWS resources within the AWS Lambda container. The Lambda containers are shipped with credentials as environment variables, and this is sufficient in retrieving them.
Note: This creates a DynamoDB instance that only sees resources in the default region. To create one for a specific region, use this (assuming you want to access DynamoDB's in the ap-northeast-1 region):
DynamoDB dynamodb = new DynamoDB(
Regions.getRegion(Regions.AP_NORTHEAST_1).createClient(
AmazonDynamoDBClient.class,
new EnvironmentVariableCredentialsProvider(),
new ClientConfiguration()));
Your Lambda function's permissions are controlled by the IAM Role it executes as. Either add Dynamo PutItem permission to the current role, or create a new role for this purpose.
After giving permissions to the Role, you don't need to write special code to handle credentials, just use the AWS SDK normally. For example:
var AWS = require("aws-sdk");
exports.handler = function(event, context) {
var dynamodb = new AWS.DynamoDB();
var putItemParams = {
"TableName": "Scratch",
"Item": {
"Id": {
"S": "foo"
},
"Text": {
"S": "bar"
}
}
};
dynamodb.putItem(putItemParams, function (err, response) {
if (err) {
context.fail("dynamodb.putItem failed: " + err);
} else {
context.succeed("dynamodb.putItem succeeded");
}
});
};
Is sufficient to put an item in a DynamoDB table, with the correct Role permissions.
Adding to #Gordon Tai's answer, using the current api using AmazonDynamoDBClientBuilder this looks like:
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()
.withCredentials(new EnvironmentVariableCredentialsProvider())
.withRegion(Regions.US_EAST_1)
.build();
DynamoDB dynamoDB = new DynamoDB(client);
I have an Amazon Lambda instance and an Amazon SNS instance. The Lambda code watches for changes in our database and I would like it to make a call to Amazon SNS to send pushes to our users. For example:
When a user on one of our forums gets a new message, the Lambda code recognizes this change every time it is run (every 10 minutes) and should send a push to the user's smartphone via SNS.
I'm running into a wall when it comes to the documentation; Amazon's docs only talk about how to trigger Lambda code via SNS, but not the reverse. Does anyone have an example of how I can accomplish this?
There is nothing special about pushing SNS notifications in the context of Lambda. I would think of it as just another external service that you interact with.
What you could do is pull in the AWS SDK in your lambda code and after that use the code to make the SNS calls. You will need to inject the right credentials to be able to call the Amazon SNS API (but you probably do something similar for getting the database endpoint and credentials if you are talking to the database)
Yes, you can use AWS Lambda to achieve what you want. You also need to give proper IAM Permissions allowing your Lambda IAM Role to publish messages to you SNS Topic.
Example SNS Publish IAM Policy:
{
"Statement":[ {
"Effect":"Allow",
"Action":"sns:Publish",
"Resource":"arn:aws:sns:*:<your account id>:<your topic id>"
} ]
}
You can use the lambda below to push an SNS message to a user, but you must know what the endpoint ARN is for that user. For example, if in an Android app, when the user logs in you will have the app send a GCM (Google Cloud Messaging) token to your backend (via an API call that triggers a lambda, for example). Your backend, which is connected to GCM, can then use this token to lookup which endpoint ARN corresponds to such user and put that in the lambda below. Alternatively, you can have the app send the endpoint ARN directly to your backend, though I think it might be a bit less secure. Make sure you give IAM permissions to publish to your app via SNS. You can use the lambda below to push the message:
var AWS = require('aws-sdk');
var sns = new AWS.SNS({apiVersion: '2010-03-31'});
exports.handler = (event, context, callback) => {
console.log(JSON.stringify(event))
var payload = {
"default": "The message string.",
"GCM":"{"+
"\"notification\":{"+
"\"body\":\"PUT NOTIFICATION BODY HERE\","+
"\"title\":\"PUT NOTIFICATION TITLE HERE\""+
"}"+
"}"
};
payload = JSON.stringify(payload);
var params = {
TargetArn: 'PUT THE ENDPOINT ARN HERE',
Subject: 'foo2',
MessageStructure: 'json',
Message: payload
}
sns.publish(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
};