I'm building an Alexa Smart Home skill.
Each of my users owns a hub device in their homes which is maintains an permanently open connection to my servers.
Now, Smart Home Skills require the usage of AWS Lambda, which I actually do not want to use, since I already have my own servers.
The idea is to use Lambda to just forward every incoming event to my servers via https and answering all requests asynchronously.
Since I imagine there are other people that want to forward lambda calls to their own servers, I wonder if there is already open source software for doing just that?
You can certainly use your Lambda as a pass-through or proxy, send the request to your server to send an asynchronous response.
However, another approach could be to handle the Alexa specific logic in your Lambda, and then call your server/API via HTTP to just get what it needs (state/device info) for any request from Alexa. This decouples your logic better IMO, with separation of concerns.
There's no open source software to do this for you either, as essentially you'll need to map your device/state data to the format for Alexa. The docs are pretty good though.
Related
I am a newbie to AWS and cloud computing in general, so I apologize if this question is foolish.
I am currently working on developing an app for Amazon Echo that would allow it to remotely control a PC (i.e. change volume, pause a movie, etc.). My problem is that I do not know how to communicate between my Amazon Lambda service and my Windows Application.
Any ideas?
There are potentially some problems with the way you have posed the question -- how to communicate between a Lambda Function and a Windows machine could involve a number of different solutions, but what you are looking for (as far as I can tell) is a more specific -- yet simultaneously more generalizable -- solution.
Are you trying to actually make an Alexa skill that users could use, or just something for yourself? It makes a big difference, because for just yourself there are a number of hacky solutions you could implement, like port forwarding and dynamic DNS, which fail dramatically if you try to do them in the real world. You need another component -- some kind of real-time push messaging -- that bridges between an "agent" in your Windows app and requests emitted by your Lambda code.
Your actual problem to solve is not so much how to communicate between AWS Lambda and a Windows Application, but rather one of a need for understanding how a platform like Alexa needs to communicate with a "smart home" device, specifically an entertainment device.
It is a relatively complicated undertaking, because -- fundamentally -- there is no way of communicating directly between Lambda and an arbitrary device out on the Internet. Dynamic IP addresses, network address translation (NAT), firewalls, security considerations, and other factors make it impossible to reliably initiate a connection from a Lambda function (or indeed from any Internet connected device) to any other arbitrary destination device. Most devices (my phone, my Alexa-controlled light switch, my Windows laptop) are running behind a boundary that assumes requests are initiated behind the boundary. When I open web sites, stream video, etc., I initiate the request and the response returns on the channel (often a TCP connection) that I have created, from behind my boundary (e.g. the router in my cable modem) that doesn't allow external initiation of TCP connections. They are bidirectional once established, but must be initiated from inside.
Of course, you can statically "poke a hole" in your router configuration by forwarding a specific TCP port to a specific internal (usually private) IP address, which works as long as your Internet provider doesn't change your IP address, and your internal device doesn't get a new IP address... and there'a UPnP NAT Traversal, which seems like a good solution until you realize that it is also terrible (though for a "hobbyist" application, it could work).
While this is a long and complex topic, the short answer is that Alexa, via Lambda code, is only capable of initiating connections, and your device, wherever it may be, is only capable of initiating connections -- not receiving them... and thus you need some kind of "meet in the middle" solution: something that allows the device to maintain its "connection" to a central "service" that can coordinate the interactions on demand.
For example:
AWS IoT Core is a managed cloud platform that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely. With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected.
https://aws.amazon.com/iot-core/
The client initiates the connection (e.g. via a web socket) to the IoT platform, and maintains it, so that when a message arrives at IoT, the service knows how to deliver that message to the client when it's received. ("even when they aren't online" refers to the "device shadow" capability, which allows you to programmatically interact with a proxy for the device, e.g. knowing the last temperature setting of a thermostat, and asking the thermostat to change its set point when the connection is re-established at some future point).
Or, potentially something like this:
Firebase Cloud Messaging (FCM) is a cross-platform messaging solution that lets you reliably deliver messages at no cost.
Using FCM, you can notify a client app that new email or other data is available to sync.
https://firebase.google.com/docs/cloud-messaging/
Both of these potential solutions solve the problem by "knowing how to contact" arbitrary devices, wherever they may be... and I would suggest that this is the core of your actual need.
There are a lot of alternatives for such a "service," including roll-your-own websocket or HTML EventSource implementations with servers... the purpose of this is not product recommendations but rather to give you an idea of what you would need for such a scenario -- an intermediate platform that can be interacted with by the Lambda code, which also knows how to communicate with "agent" code running on the device... because both Lambda and the agent need to initiate the communication channels and thus additional components are required to bridge them together.
Lets say I have web applicatons/services:
API
Set of Applications
API is used for managing some resources (simple CRUD operations). Now what I need is to subscribe Applications for changes of different API resources. Applications would do some background work on a change.
I came up to idea of callbacks. So that Applications can oauthorise and post to the API a callback config.
I think that this config should look like this:
{
'callback_url': 'http://3rdpartyservice.com/callback',
'resources': ['foo1', 'foo2'],
'ref_data': { 'token': 'abcd1234' }
}
resources is array of the resources that 3rd party service is interested in
ref_data is custom json for 3rd party usage (e.g. for auth)
This way on specified resource change the API would send a request to callback_url. This request would contain resource data, action(create/update/delete) and ref_data.
The intention here is to make this generic enough to allow 3rd party clients configure such callbacks.
So the question are:
Are there any best practices?
What about security potential issues?
Are there any real world examples on the web?
Tx
Sounds very similar as WebHooks or Service Hooks.
Check out the Web Hooks on GitHub, to get a good idea what they are and how they work. See also last alinea Service Hooks, as it explains how github handles these WebHooks. This would be similar for your application. The OAuth explains why and how it is done.
See also Webhooks, REST and the Open Web, from API User Experience.
There is even RestHooks.
The general solution to this requirement is usually called "publish/subscribe". There are dozens of solutions to this - google "publish subscribe REST" for some examples. You can also read "Enterprise Integration Patterns".
They key challenge in this kind of solution is "real-time versus queue".
For instance, if you have an API with a million clients, who are all interested in the same event, you cannot guarantee that in real time you can reach all of those clients within whatever timeframe their application demands. You also have to worry about the network going away, or clients being temporarily down. In this case, you application might define an event queue, and clients look in that queue for events they're interested in. Once you go down that route, you're probably going to use some off-the-shelf software rather than building your own. Apache Camel is a good open source implementation.
In your example, for instance, what happens if you cannot reach 3rdpartyservice.com? Or if http://3rdpartyservice.com/callback throws an error when posting an update to foo1, but not to foo2? Or if http://3rdpartyservice.com/ uses a different flavour of OAuth than you're used to? How do you guarantee http://3rdpartyservice.com/ that it's you who is posting an update, not a hacker?
Your choices really tend to come down to your non-functional requirements, rather than the functional ones - things like uptime, guarantee of notification, guarantee of delivery, etc. are more important than the specifics of how you pass across the parameters, and whether it's "resource-based" or some other protocol.
In my website, I'd like to create a public API that would allow clients (unknown people) to interact with my services. A classic REST API would work well in that case.
However, I need to be able to send events to the clients too. These events are not related to client HTTP requests. I saw "webhooks" are a way to deal with this. If I understood well, with webhooks, my service would send HTTP POST requests to a URL specified by the client, with event data inside this request.
I think websocket can be used too as a solution for this full-duplex communication need.
What I want to know, is which method would be the simplest for clients to implement to talk to my services? Simplicity is the key point here.
The hard thing is that my clients can use various technologies (full websites with HTTP servers, iOS/Android apps without server, etc.)
What are implications for clients if I use REST API + webhooks? Websockets? etc?
How to make a choice?
Hope it's clear (but not sure). Thanks :)
I would consider webhooks a simpler solution. And yes, you understood it well, that with webhooks, a developer using your API would register a URL where your backend would POST event data. It's a common pattern that's used in APIs.
A great benefit of using a webhooks design is that a client/server connection does not need to stay open. After all, if events occur infrequently (i.e. only a few times per hour, or per day) or keeping a consistent connection open is a challenge, establishing a connection only when it's needed is rather efficient.
The challenge of using webhooks for you, the API provider, is designing an evented backend system that deals with change of state detection and reliable webhook calling mechanisms (i.e. dealing with webhook receiver URLs that are unresponsive or throw errors).
The challenge of using webhooks on the developer end is that they need to stand up a reliable web server that listens for the event POST data from your server.
Realtime APIs (i.e. based on Websockets, Bayeux/CometD) are really swell because that live connection means that new connections do not have to be established, which is particularly useful with very chatty sessions. Additionally, there are a lot of projects and companies out there that have taken care of the heavy lifting on the server and client with fully-baked libraries. One of those is Fanout.io which makes pushing messages between the client/server possible with just a few lines of code, utilizing XMPP, Bayeux, and Websockets when possible.
(I am not affiliated with Fanout, but I have used it)
So, to sum it up, webhooks are simple mostly because you are already familiar with the architecture needed to implement them, and the pattern is a well traveled one. If you are leaning toward a persistent connection approach, I would look at tools/platforms like Fanout because it takes care of the heavy lifting (i.e. subscribe/publish, concurrent connection scale, client/server libraries).
I want to enable event notifications for my customers. There are many possible ways to send notifications: emails, sms, XMPP/other IM, pre-recorded voice messages over SIP, phone-specific message push services, REST callbacks etc.
I don't want to develop all these transports myself, so I need a web service that can manage those notifications for customers. Also I don't want to store emails/phones/other personally identifiable information.
The notifications are transactional (i.e. it's not mass delivering same message to everyone). Paid solutions are welcome.
There is http://pagerduty.com but it is
designed to work within enterprise and not with outside customers
focused on full cycle of incident response as opposed to simple message delivery
So it puts more burden on respondents and I want something that requires zero effort for the users to setup.
Monitis is another example. It has multiple transports including Twitter, but again it's designed for insiders and not for service subscribers coming in bulk numbers.
Amazon SNS seems to be too low-level as it only manages delivery of push notifications, but for diplaying them I have to write a mobile app which I don't want.
XMPP servers as described in How best to deliver notifications to various IM / notification services? have traditionally supported the idea of different transports, but I'd like a third-party hosted service.
Twilio has only 2 transports: SMS and voice call and more oriented on full 2-side communications.
I cannot even find the right google keywords to search for the service/SaaS I want.
The question is, are there any such services? A sample of a few would give me an idea of what to look for.
This comes very late, perhaps too late but...
You should not need to implement any of the transport but you may be required to build some of the gateways and you will most likely need to assembly the application which talks to each of the gateways. You are not likely going to find a single service for this.
You've already outlined the strategy. You basically have these pieces:
transports
gateways
application
Each of the transports is accessed through some client via either an API or a CLI - so you'll need to figure out what your environment is. Java is probably a good choice but other cross-platform environments would likely work. Existing infrastructure like Apache ServiceMix has support for some of these transports:
https://cwiki.apache.org/confluence/display/SM/Components+list
and there may be other middle-ware with similar, distinct transports.
You will likely want a gateway for each provider for each transport type. You may be able to find a provider which adequately services multiple transports, e.g. Twilio's SMS and voice, but that will likely be the exception. You may also find that because of the differences in transports (and therefore functionality), it's more convenient to build a gateway for each transport type. So, you might have two configured providers in your SMS gateway, one for Twilio and one for Kannel, and you might have your Twilio account used in the SMS gateway and in the SIP gateway.
The final step is assembling your application into something meaningful. This might be something like:
sent.......: "Thanks for your purchase, ${username}!"
sent to the channels (i.e., provider-transport pair) configured, perhaps, by the user and being able to collect the response from the user:
response...: "It was a pleasure! --Bob"
You will need to store the basics of the each transport's endpoint, e.g., phone number for SMS, username for chat, etc., so if you have PII security issues to address you'll need to think though that. One option may be to turn all the PII over to each provider but you'll still need to keep each account for your users in each provider, and you will likely need to know something about the user, like "${username}" above, to personalize your notification appropriately within your application. So, removing all PII from your application seems unlikely.
I'm not sure how much this help but perhaps it gives you some ideas.
I would like to use Amazon SQS in my application to queue requests from other external systems that don't belong to me.
What is the better way of doing this, directly expose the SQS Queue and the required messageformat OR publish a web service (WCF) that queues the request.
Also I read that SQS is relative slow for a singe access, but am I right that it can handle easyly a lot of concurrent accesses from different clients?
Best
Thomas
This is largely a matter of preference and depends a bit on your situation. But my recommendation would be to wrap it with your own web-service.
Building your web-service allows you to do things like validation, throttling, schema versioning etc. E.g. you can reject invalid messages with immediate synchronous feedback to the sender. If the external systems are publishing directly to your queue, then invalid messages become your problem not theirs, and if you revise your schema and want to reject old-schema messages then you either have to drop them or set up a separate back-channel to feed back information to the publisher. That adds unnecessary complexity to your system. Having a web-service would even let you switch to other queuing technologies later if you need to.
But building your own web-service has downsides too: will your own service be able to handle the same load as the SQS API with the same low latency? It won't scale infinitely like SQS, so how responsive will you need to be to changes in load? Have you got the resources to manage a separate service? And it's more work than just giving a client's AWS account permission to publish to your queue.
If you're happy with the extra work involved, and you want a more future-proof system, IMHO it's worth building the web-service wrapper.