Can someone demystify EMR/EC2/Lambda/SNS/SQ in AWS services for me? - amazon-web-services

I have a reason for this question. So I would appreciate positive responses.
AWS is a black box to me.
In my mind all services offered by AWS are plug play. So AWS has a user interface where you register a bunch of info and you click a button and AWS takes care of it.
But clearly that's not the case because there are AWS services based interview questions.
So I am not sure if there is a coding element to it. Can someone demystify AWS and all its services on a higher level to me ?
Like AWS EMR/EC2/Lambda/SNS/SQS - Just a one word response is fine. Right now to me its a button clicking services with zero coding or skill. But clearly I am wrong.

Right now to me its a button clicking services with zero coding or skill. But clearly I am wrong.
I'm going to assume that your question is something along the lines of why they're called "web services", rather than what each service does. If the latter, then you've gotten a lot of links in the comments. I'd also Google for Amazon's Cloud Practitioner videos.
The reason why AWS is called a web service is because that's exactly how they work: when you want to do something, you make a GET/POST/PUT/DELETE to the endpoint for a service. For example, to send a message to an SQS queue in the us-east-1 region, you make a POST request to https://sqs.us-east-2.amazonaws.com.
Each of these requests is signed, using an access key and secret key. If you don't properly sign a request, it will be rejected.
While you can construct these GET and POST requests yourself, AWS provides a bunch of language-specific SDKs that do it for you. For example, if you want to send a message to an SQS queue using Java, you would write something like this:
AmazonSQS client = AmazonSQSClientBuilder.defaultClient()
// this is available from the console, or you could make another call to retrieve it if you know the queue name
String myQueueUrl = "...";
client.sendMessage(myQueueUrl, "an example message");

Related

How to send parameters to "Open in Cloud Shell" URL?

I want to create a button that will open GCP cloud shell and run code that create some resources in the account.
I am trying to use "Open in Cloud Shell" (https://cloud.google.com/shell/docs/open-in-cloud-shell) URL and adding my GIT repo to the URL, but the problem is that my code should get different arguments in every run. There is a way to send arguments with this URL? Or maybe there is another solution for running code with arguments in GCP cloud shell via URL?
This is NOT a direct answer to your original question however it might be useful for an overall answer. If we don't like this answer, simply let me know and we'll delete it.
From you clarification in the comments, what I now sense is that you want to create GCP resources that the user can work with. For example, a PubSub topic. We'll use that as an illustration. The first thing I want to do is disavow us of the notion that there is anything "special" about a resource and the identity that it used to create that resource other than the identity must have authority to create it. For example, if user "john" creates a topic, that doesn't mean that the topic is "owned" by john. A GCP resource "just exists" after it is created. In order for a user to "use" a resource, it (the resource) must authorize the sets of users to work with it. This is where GCP IAM comes into play. Separate your goal into two parts.
Upon request, a new GCP topic is created
Once the GCP topic is created, you grant permissions on the topic to be worked with by named identities (users/groups)
Don't think "The user who creates the topic is immediately the one who can work with it".
For example, you may wish to grant your users the ability to subscribe to a topic but may not want those users to be able to "manipulate" topics such as creation/update/delete.
I am assuming that the solution you are working against is for end users rather than internal developers?
Off the top of my head, I'm tempted to suggest that you review the following very short video:
How to authenticate calls to your Google Cloud Run service
This is just a teaser but it does give us a clue. It alludes to the notion that a request from an authenticated (to Google) user can be received by a Cloud Run instance and Cloud Run can then know who the user is. With that in mind, in the code of your Cloud Run, you can then make a "yes/no" decision as to whether to proceed. If yes to proceed, then Cloud Run (which is indeed running as a single user and we won't change that) creates the topic and then assigns subscription (or publication or other) permissions to the topic on behalf of the identity that came in with the request.

Is pubsub suitable to be used by client desktop applications?

If I were to create a client desktop application, I'm trying to find a reliable way to notify client applications of new data that needs to be queried from the server. Would pubsub be a good use for this? Most of the documentation I see for it seems to be focused on server to server communication, and is a bit ambiguous if this would work well for server to client notifications.
If it should work, would I be able to properly authenticate subscribers to limit the topics they could subscribe to? This application would be potentially downloadable by anyone, and I would need to ensure that information intended for one client couldn't end up in the hands of another client.
Cloud Pub/Sub is not going to be a good choice for this use case. First of all, note that each topic and project is limited to 10,000 subscriptions. Therefore, if you intend to have more than that, you will run out of subscriptions. Secondly, note that a subscription only receives messages published after the subscription is created. If you only need messages to be delivered that were published after the user came to the website, this may be okay. However, with these two issues combined, you'll need to consider lifetime of your subscriptions. Do they get deleted when a user logs out? If not, when a user comes back, do you expect them to get all of the messages published since the last time they visited?
Additionally, as discussed in the comments, there is the issue of authentication. Your client-side app would have to have the credentials to subscribe. This would require you to essentially leak those credentials into your client-side code, which could be a vulnerability in your application.
The service designed to deliver notifications of this nature is Firebase Cloud Messaging.
If you want to open the application to anyone on the internet, you can't rely on the IAM service that only works with Google identity -> You can't ask your user to have a Google Account, the user experience will be bad.
Thus, you can't use IAM service to secure the PubSub access, and thus to use PubSub because anyone could access it.
In your use case, the first step is to ask the user to register (create an account, validate email, maybe use payment method,...). Then, you have an identity, but managed by you, not by IAM. You know which messages are for this user and which aren't.
If you want to be notified "in real time", I propose you to use long polling method or streaming to push data to the user. Cloud Run is now capable to do this and I recommend you to have a look on that.

AWS S3 Static Website Push Capability

Is there a way for me to have a static website on S3 that uses lambda and api gateway for services but to have messages pushed from lambda (that originated from a scheduled run) instead of polling for new information?
I was thinking of using IOT, but im not too familiar with it im not sure if a website can be a IOT thing, the documentation doesnt really help much with configuring your things.
Any other ideas?
Thanks
Answer from MaiKay looks like exactly what i wanted to achieve, blog post even has a small demo where messages can be pushed to a session running in another tab:
You are able to use IoT for it. A nice article can be found here: https://serverless.com/blog/serverless-notifications-on-aws – MaiKaY

Amazon Echo - Push a message to the device

I have integrated my amazon echo device with the amazon portal associated to my account. I was able to create my own custom question with the Alexa Skills Kit and process with an AWS Lambda function to generate a response.
My question is: is it possible to programatically "push" a message to the echo device? For example, I would like for it to speak without having to ask it a question. I'd like it to do something at a specific moment.
If it is possible, could you please share any sample code to achieve this?
It is not currently possible, but it is an oft requested feature on the AWS forums.
http://forums.developer.amazon.com/forums/thread.jspa;jsessionid=EC0D457A400B594DD0F0561EEB43A8FA?messageID=17713&#17713
I've not done this myself but it seems using the Alexa Voice Service could do the trick. It allows processing of voice from any type of audio capture and sends it to the Alexa Service. It seems possible you could record the proper phrase into a sound file and send that to AVS, thus triggering the Alexa service.
I know it's capable but Amazon hasn't offered it as a feature yet. If you go to the Echo web site http://alexa.amazon.com/spa/index.html#cards, Settings, Connected Home and select Discover devices, the echo will perform a command triggered from the web site and will speak when completed. I didn't have to say a word.
From what I read on an article about notifications here, they were going to do something that meant you still had to ask Alexa to tell you your notifications, which would build up throughout the day and cause your device to light up and chime to let you know you had a notification to listen to.
Rather than allowing Alexa to randomly blurt stuff out any time she feels like it.
Which kinda seems pointless if you don't get the notification at the exact time it was sent, for example a smart home connected device triggering an alert to let you know you left your door open too long or the app automatically reading you the weather when you get up and turn on the coffee pot in a morning, things like that. If you have to ask for the notifications you may as well just do the speaking and request stuff from the beginning.
Maybe I'm wrong but that's how it reads to me.
Surely we want, with opt-in permission per skill, to allow Alexa to just say whatever she likes whenever she likes from the skills we have set up receiving such commands, without us having to say oh I see I have notifications let me ask what those are.
Just for people who stumble across this question in 2021:
The solution is to use Amazon Proactive Events
Your Skill has to request for the notification permission, subscribe for a specific event and then you can generate a access token and POST events to the amazon API.
Took me some hours to find out because Amazon offers different things which all sound quite similar but some are deprecated (ASK CLI v1) and others are just for Alexa device manufacturers.
Wuhu! Sounds like Amazon may enable push notifications this Fall.
Digital Trends Article
Amazon is expected to establish guidelines for developers and manufacturers so that Alexa remains classy and doesn’t become an interrupting nag.

Amazon Mechanical Turk task retrieval API

I'm writing an app wherein the premise is that people who can't directly fund a charity out of their pocket could automatically work on Amazon AWS HITs in order to bring clean water to the 3rd world - I'd known there was an Amazon AWS API but is very unclear on how to retrieve a HIT to be worked on, rather than just consume some data about some tasks I'm trying to complete.
Is there any way to retrieve a HIT to be worked on through the AWS API or otherwise?
Thanks ahead of time.
As far as I know, the only way to work on a HIT is through the mTurk website. i.e. - not via API.
There is a site that is trying to do something very similar to what you have described. http://www.sparked.com/