We're trying to set up a Windows Continuous Integration server in the cloud that has software with a particularly difficult licensing scheme (a certain embedded software compiler). This compiler either lets the user run with a dongle, or node locks the software to run on a specific set of hardware. Seeing as how we can't exactly ship off a dongle to Amazon to plug in to our machine for us, we want to use the node locked licensing scheme with this software.
However, in order for this license to remain valid, my EC2 instance needs to have a set of "static" hardware IDs. We don't actually know which hardware IDs this software will look for (and they won't tell us), but I'd like to know which hardware IDs I should expect to change with instance restarts, and which hardware IDs, if any, I can lock down through proper configuration of my EC2 instance.
For example, I already know that I can lock down the MAC address of the Network Interface using Amazon's Elastic Network Interface (ENI). Is there any other configuration I can do, or need to do, in order to get a set of static hardware IDs?
Related
Can we use AWS Device Farm to test anti-virus application by installing real malware on rented devices?
AWS's AUP includes the following wording:
You may not use, or facilitate or allow others to use, the Services or the AWS Site: ... to violate the security, integrity, or availability of any user, network, computer or communications system, software application, or network or computing device;
So with regards to installing malware or exposing the devices to live virus or malware payload, the answer is an unequivocal no.
However, I would imagine that as with standard EICAR test strings, you could test file contents for various pattern matches and remain within the bounds of the AUP, ie by including a substring of a particular known malicious payload to test detection. The moment you allowed known malicious code to execute you would be in violation of the AUP, and depending on AWS's inbuilt scanning they may detect and block your access proactively regardless.
So to summarize, the answer is no, you'd need to build your own lab on your own equipment to do testing of this nature.
I'm thinking about creating EC2 instances that run industrial software. This EC2 instance should be able to receive data from Serial COM port. Is it possible to connect things like this, with eventually a serial tunneling or anything else ?
There are two parts to the answer to this question.
Yes, absolutely, this is possible in theory.
...but it might not be practical or possible for the specific purpose/application/device that you have in mind.
Terminal servers are a simple illustration of the principle. Note that in this sense, I'm not referring to a Windows Remote Desktop "terminal server," but to the generic device commonly called a terminal server which, in its simplest form, is a physical device that speaks a serial protocol such as RS-232 on one side, and TCP on the other side. RS-232 is the wire protocol of a PC "COM" port.
But a computer, configured with appropriate drivers, can have a "virtual" COM port that appears -- to software on the computer -- as an interface just like a physical COM port, but the driver for that COM port, instead of providing the software with access to literal hardware is in fact managing a TCP connection, tunneling the send and receive bits from the COM port back to the physical port on the terminal server.
The terminal server could be an actual, dedicated terminal server device, could be a full size PC, a microcontroller with network capability, a Raspberry Pi with a USB-to-Serial adapter... there are a lot of potential variations.
Problems to consider:
A lot of older industrial software seems very poorly written. This is my impression, anyway. The implementers, perhaps working at the edges of their expertise, having made certain design assumptions that work on physical COM ports but may not be compatible with such a deployment. This is less likely to be true if the systems are newer, and run on a modern OS, but back in the early days of Windows and before, this was a real mess.
License key dongles, if present, might rely on peculiarities of physical COM ports that are difficult or impossible to virtualize. The terms of the software license may constrain you from doing this.
Some equipment may be unnecessarily sensitive to the delay that is introduced by the distances involved. There are, for example, 22 ms of round-trip time in the Internet path between one of my facilities and the nearest AWS region. At 9600-8-N-1 that is the time to transmit ~23 bytes, and that's best case -- the virtualization layer and encryption will add more. The machine or the software may or may not be well-written enough to accept that kind of delay, which does not occur on a physical, local COM port.
If the machine is being controlled (not simply observed) then you need to absolutely ensure that you are not creating a safety hazard by separating the machine from its software through a virtualization layer.
"Sharing" control of a machine by software on multiple EC2 instances seems like it might be implied by your illustration, but this potentially adds an entirely different set of complexity.
In theory, yes, it's possible. I mean, even Windows remote desktop allows you to share a local COM port with the remote system, allowing remote software to access and control your local device.
In practice, this potentially requires a lot of attention to a large number of factors that vary by operating system as well as by the specific peculiarities of the devices and software in question.
EC2 connectivity is through the network interface, so I don't think a direct connection like the one you are asking is possible.
However, if you are talking about industrial devices supporting automation, chances are you can use MQTT or at least a MQTT bridge. If that's the case, you can take a look at AWS IOT Core
With AWS IOT Core you don't only get bidirectional connectivity between your devices and AWS, but also powerful analytics, device registry and management, and full integration with the AWS ecosystem.
I am a newbie to AWS and cloud computing in general, so I apologize if this question is foolish.
I am currently working on developing an app for Amazon Echo that would allow it to remotely control a PC (i.e. change volume, pause a movie, etc.). My problem is that I do not know how to communicate between my Amazon Lambda service and my Windows Application.
Any ideas?
There are potentially some problems with the way you have posed the question -- how to communicate between a Lambda Function and a Windows machine could involve a number of different solutions, but what you are looking for (as far as I can tell) is a more specific -- yet simultaneously more generalizable -- solution.
Are you trying to actually make an Alexa skill that users could use, or just something for yourself? It makes a big difference, because for just yourself there are a number of hacky solutions you could implement, like port forwarding and dynamic DNS, which fail dramatically if you try to do them in the real world. You need another component -- some kind of real-time push messaging -- that bridges between an "agent" in your Windows app and requests emitted by your Lambda code.
Your actual problem to solve is not so much how to communicate between AWS Lambda and a Windows Application, but rather one of a need for understanding how a platform like Alexa needs to communicate with a "smart home" device, specifically an entertainment device.
It is a relatively complicated undertaking, because -- fundamentally -- there is no way of communicating directly between Lambda and an arbitrary device out on the Internet. Dynamic IP addresses, network address translation (NAT), firewalls, security considerations, and other factors make it impossible to reliably initiate a connection from a Lambda function (or indeed from any Internet connected device) to any other arbitrary destination device. Most devices (my phone, my Alexa-controlled light switch, my Windows laptop) are running behind a boundary that assumes requests are initiated behind the boundary. When I open web sites, stream video, etc., I initiate the request and the response returns on the channel (often a TCP connection) that I have created, from behind my boundary (e.g. the router in my cable modem) that doesn't allow external initiation of TCP connections. They are bidirectional once established, but must be initiated from inside.
Of course, you can statically "poke a hole" in your router configuration by forwarding a specific TCP port to a specific internal (usually private) IP address, which works as long as your Internet provider doesn't change your IP address, and your internal device doesn't get a new IP address... and there'a UPnP NAT Traversal, which seems like a good solution until you realize that it is also terrible (though for a "hobbyist" application, it could work).
While this is a long and complex topic, the short answer is that Alexa, via Lambda code, is only capable of initiating connections, and your device, wherever it may be, is only capable of initiating connections -- not receiving them... and thus you need some kind of "meet in the middle" solution: something that allows the device to maintain its "connection" to a central "service" that can coordinate the interactions on demand.
For example:
AWS IoT Core is a managed cloud platform that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely. With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected.
https://aws.amazon.com/iot-core/
The client initiates the connection (e.g. via a web socket) to the IoT platform, and maintains it, so that when a message arrives at IoT, the service knows how to deliver that message to the client when it's received. ("even when they aren't online" refers to the "device shadow" capability, which allows you to programmatically interact with a proxy for the device, e.g. knowing the last temperature setting of a thermostat, and asking the thermostat to change its set point when the connection is re-established at some future point).
Or, potentially something like this:
Firebase Cloud Messaging (FCM) is a cross-platform messaging solution that lets you reliably deliver messages at no cost.
Using FCM, you can notify a client app that new email or other data is available to sync.
https://firebase.google.com/docs/cloud-messaging/
Both of these potential solutions solve the problem by "knowing how to contact" arbitrary devices, wherever they may be... and I would suggest that this is the core of your actual need.
There are a lot of alternatives for such a "service," including roll-your-own websocket or HTML EventSource implementations with servers... the purpose of this is not product recommendations but rather to give you an idea of what you would need for such a scenario -- an intermediate platform that can be interacted with by the Lambda code, which also knows how to communicate with "agent" code running on the device... because both Lambda and the agent need to initiate the communication channels and thus additional components are required to bridge them together.
If it possible to provide a service to multiple clients whereby if the server providing this service goes down, another one takes it's place- without some sort of centralised "control" which detects whether the main server has gone down and to redirect the clients to the new server?
Is it possible to do without having a centralised interface/gateway?
In other words, its a bit like asking can you design a node balancer without having a centralised control to direct clients?
Well, you are not giving much information about the "service" you are asking about, so I'll answer in a generic way.
For the first part of my answer, I'll assume you are talking about a "centralized interface/gateway" involving ip addresses. For this, there's CARP (Common Address Redundancy Protocol), quoted from the wiki:
The Common Address Redundancy Protocol or CARP is a protocol which
allows multiple hosts on the same local network to share a set of IP
addresses. Its primary purpose is to provide failover redundancy,
especially when used with firewalls and routers. In some
configurations CARP can also provide load balancing functionality. It
is a free, non patent-encumbered alternative to Cisco's HSRP. CARP is
mostly implemented in BSD operating systems.
Quoting the netbsd's "Introduction to CARP":
CARP works by allowing a group of hosts on the same network segment to
share an IP address. This group of hosts is referred to as a
"redundancy group". The redundancy group is assigned an IP address
that is shared amongst the group members. Within the group, one host
is designated the "master" and the rest as "backups". The master host
is the one that currently "holds" the shared IP; it responds to any
traffic or ARP requests directed towards it. Each host may belong to
more than one redundancy group at a time.
This might solve your question at the network level, by having the slaves takeover the ip address in order, without a single point of failure.
Now, for the second part of the answer (the application level), with distributed erlang, you can have several nodes (a cluster) that will give you fault tolerance and redundancy (so you would not use ip addresses here, but "distributed erlang" -a cluster of erlang nodes- instead).
You would have lots of nodes lying around with your Distributed Applciation started, and your application resource file would contain a list of nodes (ordered) where the application can be run.
Distributed erlang will control which of the nodes is "the master" and will automagically start and stop your application in the different nodes, as they go up and down.
Quoting (as less as possible) from http://www.erlang.org/doc/design_principles/distributed_applications.html:
In a distributed system with several Erlang nodes, there may be a need
to control applications in a distributed manner. If the node, where a
certain application is running, goes down, the application should be
restarted at another node.
The application will be started at the first node, specified by the
distributed configuration parameter, which is up and running. The
application is started as usual.
For distribution of application control to work properly, the nodes
where a distributed application may run must contact each other and
negotiate where to start the application.
When started, the node will wait for all nodes specified by
sync_nodes_mandatory and sync_nodes_optional to come up. When all
nodes have come up, or when all mandatory nodes have come up and the
time specified by sync_nodes_timeout has elapsed, all applications
will be started. If not all mandatory nodes have come up, the node
will terminate.
If the node where the application is running goes down, the
application is restarted (after the specified timeout) at the first
node, specified by the distributed configuration parameter, which is
up and running. This is called a failover
distributed = [{Application, [Timeout,] NodeDesc}]
If a node is started, which has higher priority according to
distributed, than the node where a distributed application is
currently running, the application will be restarted at the new node
and stopped at the old node. This is called a takeover.
Ok, that was meant as a general overview, since it can be a long topic :)
For the specific details, it is highly recommended to read the Distributed OTP Applications chapter for learnyousomeerlang (and of course the previous link: http://www.erlang.org/doc/design_principles/distributed_applications.html)
Also, your "service" might depend on other external systems like databases, so you should consider fault tolerance and redundancy there, too. The whole architecture needs to be fault tolerance and distributed for "the service" to work in this way.
Hope it helps!
This answer is a general overview to high availability for networked applications, not specific to Erlang. I don't know too much about what is available in the OTP framework yet because I am new to the language.
There are a few different problems here:
Client connection must be moved to the backup machine
The session may contain state data
How to detect a crash
Problem 1 - Moving client connection
This may be solved in many different ways and on different layers of the network architecture. The easiest thing is to code it right into the client, so that when a connection is lost it reconnects to another machine.
If you need network transparency you may use some technology to sync TCP states between different machines and then reroute all traffic to the new machine, which may be entirely invisible for the client. This is much harder to do than the first suggestion.
I'm sure there are lots of things to do in-between these two.
Problem 2 - State data
You obviously need to transfer the session state from the crashed machine unto the backup machine. This is really hard to do in a reliable way and you may lose the last few transactions because the crashed machine may not be able to send the last state before the crash. You can use a synchronized call in this way to be really sure about not losing state:
Transaction/message comes from the client into the main machine.
Main machine updates some state.
New state is sent to backup machine.
Backup machine confirms arrival of the new state.
Main machine confirms success to the client.
This may potentially be expensive (or at least not responsive enough) in some scenarios since you depend on the backup machine and the connection to it, including latency, before even confirming anything to the client. To make it perform better you can let the client check with the backup machine upon connection what transactions it received and then resend the lost ones, making it the client's responsibility to queue the work.
Problem 3 - Detecting a crash
This is an interesting problem because a crash is not always well-defined. Did something really crash? Consider a network program that closes the connection between the client and server, but both are still up and connected to the network. Or worse, makes the client disconnect from the server without the server noticing. Here are some questions to think about:
Should the client connect to the backup machine?
What if the main server updates some state and send it to the backup machine while the backup have the real client connected - will there be a data race?
Can both the main and backup machine be up at the same time or do you need to shut down work on one of them and move all sessions?
Do you need some sort of authority on this matter, some protocol to decide which one is master and which one is slave? Who is that authority? How do you decentralise it?
What if your nodes loses their connection between them but both continue to work as expected (called network partitioning)?
See Google's paper "Chubby lock server" (PDF) and "Paxos made live" (PDF) to get an idea.
Briefly,this solution involves using a consensus protocol to elect a master among a group of servers that handles all the requests. If the master fails, the protocol is used again to elect the next master.
Also, see gen_leader for an example in leader election which works with detecting failures and transferring service ownership.
What would be the simplest way for an application I'm writing to block all Internet access on a Windows machine?
More details:
Windows: XP or higher
Application: A basic Win32 app written in C/C++.
Blocking: It needs to be able to block and unblock at will, ideally in a way that the user can't easily reverse. (By, say, right clicking on a network connection icon in the system tray.) Also, ideally, I'd like the method it uses to allow access to be restored should the user restart Windows or reset the machine, though I'd also be willing to have the app auto launch with Windows and unblock access upon startup if the machine was reset while in a blocked state.
Internet access: Primarily, I'd like to block conventional browsers from hitting conventional http/https sites. Secondarily, it would be nice to block IM clients and client-side social networking apps. It would also be nice, but not required, to still allow local networking for file sharing, etc. (Note that only the first requirement is absolute.)
Final notes: This is not meant to be a security utility, nor will its relationship to the user be adversarial (as, for example, with a parental control utility) so it's not important for it to use a scheme that can't be worked around by a determined user. (Consider that I intend for a reboot or reset to clear the blocking. This means that any workaround a user might discover that would take more effort than this is okay.)
Thanks!
p.s. I suspect that the Windows Firewall API won't work for me because this needs to work for users that haven't enabled the firewall or don't have admin privileges, but I'll be thrilled if I'm corrected on this.
It sounds like you're intending to run applications that you don't want to access the internet. Perhaps you could run them inside a virtual machine such as VirtualBox with networking disabled.
You could do it with a Winsock SPI. The Windows SDK has a sample (under Samples\netds\winsock\lsp) which implements what is called a layered service provider which allows you to hook all the user mode functions provided by Winsock and reject/modify the calls to block network access or redirect traffic to different locations. All installed winsock applications will be affected, so in your code you could have policys for what applications can go out and the like and disabled/enable on the fly. Now a determined person could find ways around this but it would be a pain.
That said this isn't trivial to do but the sample should get you most of the way there.
You cannot effectively or practically write your tool with only a user mode application.
What you need to write is a network I/O stack filter driver. This done by writing a Windows Driver. This is different from a Windows Win32 application. Drivers run in kernel mode and applications run in user mode.
On Windows Vista and later, the kernel mode Network Programming Interface (NPI) is designed for this. This is the same API that Windows Firewalls use. These are sometimes called the Winsock kernel (WSK) APIs.
In effect, you are writing a network firewall (more or less)
here are some links
Introduction to Winsock Kernel (WSK)
Windows Core Networking Blog
The Network Programming Interface Docs on MSDN
Note, your will likely need at least two components
Your driver
A Graphical application that a person can use to control your tool
If you want to do any monitoring, you will likely need a user mode service that collects data from your driver. This works better than trying to do this in the driver. In general, you should do the minimal amount of work in the driver.
A few notes:
You need to be very conscious of security when writing this kind of software. This is very much non trivial. Software that is network facing has the highest security requirements.
Be cognizant of performance.
Your driver and/or service must be aware of the context of a calling application. This is also a security boundary. For example, an application not running as administrator should not be able to control your driver.
take a look at firewall sourcecodes