I am a newbie to AWS and cloud computing in general, so I apologize if this question is foolish.
I am currently working on developing an app for Amazon Echo that would allow it to remotely control a PC (i.e. change volume, pause a movie, etc.). My problem is that I do not know how to communicate between my Amazon Lambda service and my Windows Application.
Any ideas?
There are potentially some problems with the way you have posed the question -- how to communicate between a Lambda Function and a Windows machine could involve a number of different solutions, but what you are looking for (as far as I can tell) is a more specific -- yet simultaneously more generalizable -- solution.
Are you trying to actually make an Alexa skill that users could use, or just something for yourself? It makes a big difference, because for just yourself there are a number of hacky solutions you could implement, like port forwarding and dynamic DNS, which fail dramatically if you try to do them in the real world. You need another component -- some kind of real-time push messaging -- that bridges between an "agent" in your Windows app and requests emitted by your Lambda code.
Your actual problem to solve is not so much how to communicate between AWS Lambda and a Windows Application, but rather one of a need for understanding how a platform like Alexa needs to communicate with a "smart home" device, specifically an entertainment device.
It is a relatively complicated undertaking, because -- fundamentally -- there is no way of communicating directly between Lambda and an arbitrary device out on the Internet. Dynamic IP addresses, network address translation (NAT), firewalls, security considerations, and other factors make it impossible to reliably initiate a connection from a Lambda function (or indeed from any Internet connected device) to any other arbitrary destination device. Most devices (my phone, my Alexa-controlled light switch, my Windows laptop) are running behind a boundary that assumes requests are initiated behind the boundary. When I open web sites, stream video, etc., I initiate the request and the response returns on the channel (often a TCP connection) that I have created, from behind my boundary (e.g. the router in my cable modem) that doesn't allow external initiation of TCP connections. They are bidirectional once established, but must be initiated from inside.
Of course, you can statically "poke a hole" in your router configuration by forwarding a specific TCP port to a specific internal (usually private) IP address, which works as long as your Internet provider doesn't change your IP address, and your internal device doesn't get a new IP address... and there'a UPnP NAT Traversal, which seems like a good solution until you realize that it is also terrible (though for a "hobbyist" application, it could work).
While this is a long and complex topic, the short answer is that Alexa, via Lambda code, is only capable of initiating connections, and your device, wherever it may be, is only capable of initiating connections -- not receiving them... and thus you need some kind of "meet in the middle" solution: something that allows the device to maintain its "connection" to a central "service" that can coordinate the interactions on demand.
For example:
AWS IoT Core is a managed cloud platform that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely. With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected.
https://aws.amazon.com/iot-core/
The client initiates the connection (e.g. via a web socket) to the IoT platform, and maintains it, so that when a message arrives at IoT, the service knows how to deliver that message to the client when it's received. ("even when they aren't online" refers to the "device shadow" capability, which allows you to programmatically interact with a proxy for the device, e.g. knowing the last temperature setting of a thermostat, and asking the thermostat to change its set point when the connection is re-established at some future point).
Or, potentially something like this:
Firebase Cloud Messaging (FCM) is a cross-platform messaging solution that lets you reliably deliver messages at no cost.
Using FCM, you can notify a client app that new email or other data is available to sync.
https://firebase.google.com/docs/cloud-messaging/
Both of these potential solutions solve the problem by "knowing how to contact" arbitrary devices, wherever they may be... and I would suggest that this is the core of your actual need.
There are a lot of alternatives for such a "service," including roll-your-own websocket or HTML EventSource implementations with servers... the purpose of this is not product recommendations but rather to give you an idea of what you would need for such a scenario -- an intermediate platform that can be interacted with by the Lambda code, which also knows how to communicate with "agent" code running on the device... because both Lambda and the agent need to initiate the communication channels and thus additional components are required to bridge them together.
Related
I want to create a C++ application that can be used (in part) to communicate between users on a local area network using UDP. Each instance of the application will have a thread dedicated to listening for other instances of the application and broadcasting its presence to other instances.
Is there a reliable way to perform this type of broadcast/listening on the LAN using pure C++ and POSIX system calls? I know there's no generally reliable way to find all IPs on a LAN, but I assume this is only because other devices are not willing to respond to pings.
Don't re-invent the wheel. There are two existing technologies, that, when combined, solve your problem in a standardized, well-designed, proven manner:
RFC6762 mDNS (Multicast DNS) is a protocol that works almost exactly like DNS, except it works using IP multicast. Instead of sending your DNS request to a unicast address, you send your DNS request to a multicast group, and any member of that group can answer your request (so you may get multiple answers).
RFC6763 DNS-SD (DNS-based Service Discovery) is a way to encode Services as DNS entries, which allows you then to retrieve Services from DNS using specially encoded hostnames. (For example, looking up the hostname _ipp._tcp.example.com would return a list of all printers that support the Internet Printing Protocol over TCP within the domain example.com)
So, we have one protocol that allows us to ask DNS about Services, and we have one protocol that allows us to ask a group of hosts to answer DNS queries … when we put the two together, we get a way of asking hosts for services! The combination of those two protocols is sometimes called Zeroconf networking, and is already implemented in macOS, iOS, tvOS, and watchOS (where it is called Bonjour), Android, most Unices (via Avahi, a portable implementation of those two protocols), and many home devices such as TVs. E.g. Spotify Connect, ChromeCast, Philips Hue and many others are also based on it. It's how iTunes devices find each other on the local network, for example.
Unfortunately, support in Windows is still limited, at the moment it seems to only exist for Windows 10 apps implemented in ECMAScript. (However, nothing stops you from shipping your own implementation with your app, and AFAIK, both Avahi and Apple's mDNSResponder work on Windows.)
So, what you would basically do is to send an mDNS query to the mDNS multicast group and ask for _myprotocol._udp.local. Then, assuming that your application registers itself with the OS's mDNS service (i.e. Bonjour on macOS, Avahi on Unices, …), you would get back a list of all hosts on the local network that support your protocol.
I am running an Openfire server on a AWS EC2 instance and am able to connect to the server from my mobile devices and send messages back and forth. Of course, since XMPP is a client-server based protocol, I incur costs for running this traffic over the AWS server. However, for most use cases, this cost is not very high at all, as normal XMPP stanzas rarely seem to go above ca. 1 KB, so from this end all is ok.
I would now, however, like to include the ability to send images from one client to another. One way would be to use an HTTP server, to which user A uploads the picture and then sends the URL of the image to user B via XMPP, so that the user can now get the image via HTTP. There are also several other methods for sending images via XMPP. However, I am interested in doing this via Jingle.
As far as as I understand, Jingle is an out of band peer-to-peer extension to XMPP. My simple question is, since Jingle communicates peer-to-peer, i.e. without the use of a server, for the multimedia aspect of the session, will I even incur any data cost on AWS for transferring multimedia from one client to another using Jingle? Or put differently, if Jingle is peer-to-peer, does any data go via my AWS server using Jingle (except the session initiate, ack, session terminate stanzas)? If not, what kind of route does this data take, and how can anyone be billed for this traffic cost, if it is peer-to-peer?
Jingle is a negotiation mechanism, and there are a couple of different transports it could negotiate for file transfer. The most common transport is peer to peer bytestreams defined in http://xmpp.org/extensions/xep-0260.html - here the only traffic you'd see via the server would be the jingle negotiation, which is a similar sort of volume to other XMPP traffic). There is also an in-band bytestream transport defined in http://xmpp.org/extensions/xep-0261.html that some clients will use - typically for smaller transfers as it's inefficient, but has the advantage of working in hostile networks with NAT and firewalls. If you control the clients, simply not supporting IBB would be your best bet for ensuring the traffic doesn't travel via the server. If you don't, I'd suggest configuring your server to block IBB traffic.
I note as well that running a server-side proxy will drastically increase the odds of the out-of-band mechanism in 260 working in the face of hostile networks, at the cost of server bandwidth.
There is also the not-widely-deployed http://xmpp.org/extensions/xep-0343.html out of band transport.
I've been reading around on the www but just can't get the most important basics of P2P.
The diagram is like this:
[peer1]<-->[dsl-router1]<-->[central server]<-->[dsl-router2]<-->[peer2]
I'm developing a chat software on the central server. Chat messages being transfered thru' the central server well by now, however, I need to make the p2p file sharing feature because the bandwidth (the cable bandwith, not the transfer limit) of the server supposed for transfering chat messages only.
The problem is that, my software on central server knows the IPs and ports of router1 and router2, but not the peer1 and peer2 as these peers are behind the routers and don't have IP addresses.
How to actually transfer some data from peer1 to peer2 and vice versa without having this data passing thru' central server?
(and the worst case is that there is a wireless router between peer and dsl-router)
There are two basic ways of doing this. The new way is to use IGDP (opening a port via uPnP). This is described quite well here:
http://www.codeproject.com/Articles/13285/Using-UPnP-for-Programmatic-Port-Forwardings-and-N
If neither of the two nodes have a router supporting uPnP then another alternative is TCP hole punching, which is not perfect but works quite well in practice. This is described here:
http://www.brynosaurus.com/pub/net/p2pnat/
During some situations, "routers" supplied by the ISP may run on bridge mode, which directly exposes the peer computer on the internet (the computer gets a public internet address). If at least one side has this configuration (or in a similar situation that the peer client is not behind another device), then things should be rather straight forward: simply assign the central server's job to whoever that have this privilege.
In the other case where both peers only have a local address (e.g. 192.168.0.2) assigned to their computers, it would then be rather difficult to get through the routers; clients behind routers are for the most part unreachable from the outside unless they originated the request. Then, one solution to the problem is port forwarding. By doing port forwarding, either through explicitly written rules or UPnP, some ports on the peer computer is exposed to the public internet, as in the first situation where instead of only some ports the entire computer is exposed.
If you are without either of these, then there is no simple way to avoid sending through the central server. Though you could, potentially, find other peers who have the capability to transfer for others.
I have a C++ application that currently uses a simple TCP/IP client/server model to communicate between 2 instances of itself. This works fine on a local network, but I would like this to be used across an external network. Currently, maybe due to firewall issues, it is not able to connect across an external network.
I am not an expert on networking, but I was thinking about having a dedicated server in the middle acting as a hub for communications. Will this mitigate firewall issues?
How do networked games communicate with each other? Is there usually a server in the middle or is it peer-to-peer?
In any case, I'd appreciate any advice on protocols and infrastructure to implement a network enabled application.
Regards
I think the problem is dedicated to the NAT as mentioned by cnicutar.
Maybe you want to have a look at libupnp for automatic port forwarding in the hardware firewalls (your router at home)
There is no de facto architecture for multiplayer network games. Both client-server (most MMOs, most PC FPS's and RTS's) and Peer-to-Peer (most console games) are valid approaches.
Juoni Smed's survey in his book "Algorithms and Networking for Computer Games" is a pretty good overview of the different architectures in the wild.
For the specific issues you're talking about, your need for a proxy server, as others have noted, is probably down to NAT issues - the two machines you're trying to get talking do not have public IP addresses. If you want to pursue a Peer-to-Peer architecture (or to have one of your clients act as the server, as many modern Client-Server games do) you will need your clients to talk directly to each other. This can be achieved with NAT Traversal, unfortunately this is a fiddly process.
Luckily you can use a modern framework like the excellent Raknet which includes State Synchronisation, Remote Procedure Calls AND NAT Traversal out of the box. It's free for hobbyist use and is incorporated in to several modern industrial-grade game engines.
The bane of modern internet communications is NAT. Due to NAT (which shouldn't be confused with a simple firewall) a large portion of hosts on the internet don't have a public address and thus can't (easily) accept incoming connections. NAT breaks the internet so badly that people are moving to a totally different scheme, with slightly different semantics, just to get rid of it.
There are basically two class of solutions
NAT traversal which is sometimes used for peer-to-peer communication. Usually NAT traversal schemes require some publicly accessible server for brokering the initial connection, but the actual communication is done peer-to-peer
Client-server communication. This is easier (since the server generally should have a publicly accessible address) but also kind of inefficient. For instance, say you've got a peer on the same 10Gb LAN. If you want to send him a file through the server (which happens to be in another country) it's going to take ages instead of seconds.
I'm not sure which one is "generally used". But think of it this way:
If there is the logical need for a "controller" (say 8 people are playing a strategy game) then you probably need a server
If any two peers can logically interact without a "controller", you probably want peer-to-peer communication
If you need to transfer LOTS of data fast (file transfer), you almost surely want p2p.
The easiest way to accomplish what you want is by using sockets(in case you are doing it differently). The way you are connecting your app is usually how it's done. Also if it work sin a local network and it does not over the Internet it must be a firewall issue so try opening ports in your router configuration.
You will have to give more info about your program in order to explain if you should go with peer-to-peer or with a server.
I'm working on the design of a remote control application. From my iPhone or a web browser, I'll send a few commands. Soon my home computer will perform the commands and send back results. I know there are remote desktop apps, but I want something programmable, something simpler, and something that I wrote.
My current direction is to use Amazon Simple Queue Service (SQS) as the message bus. The iPhone places some messages in a queue. My local Java/JRuby program notices the messages on the queue, performs the work and sends back status via a different queue.
This will be a very low-volume application. At $1.00 for a million requests (plus a handful of data transfer charges), Amazon SQS looks a lot more affordable than having my own server of any type. And super reliable, that's important for me too.
Are there better/standard toolkits or architectures for this kind of remote control? Cost is not a big issue, but I prefer the tons I learn by doing it myself.
I'm moderately concerned about security, but doubt it will be a problem. The list of commands recognized will be very short, and only recognized in specific contexts. No "erase hard drive" stuff.
update: I'll probably distribute these programs to some other people who want the same function, but who don't have Amazon SQS accounts. For now, they'll use anonymous access to my queues, with random 80-character queue names.
Well, I think it's a clever approach -- and as you said, the costs for your little traffic aren't even worth mentioning.
As I mentioned in the comment, it's a good way to leave your home machine behind your firewall and not have an open port on the internet.
I would suggest using OnlineMQ.com as a start; they have a free package.