The "Event Capture" feature in Azure Event Hubs pipes event data to an Azure data store such as Azure BLOB Storage or an Azure Data Lake. When configured to do this, will Event Capture encrypt the data in transit to the storage location?
I can find information online in Microsoft's Azure doco that indicates encryption at rest can be achieved (both in storage and in the message stores within Azure Event Hubs) however I am unclear as to whether data in transit is also encrypted. Anyone know if this is the case or not?
My assumption is Azure Event Hub and Azure Storage will talk to each other via HTTPS protocol which ensures secure communication i.e. encryption in transit.
You can enforce this encryption in transit further by enabling Secure transfer required. This will make sure all clients (in this case Azure Event Hub) use HTTPS.
Related
I am working on a project wherein a mobile app (Flutter app) should create a unique identity (IOT thing) of my IOT device machine (on AWS) as soon as the user logins onto the app and also it should establish a connection (and subsequent communications) with the isolated IOT device after its configuration (Somewhat similar to Google Home configuration process).
The process starts with user logging onto the app (using AWS Amplify/Congnito to automatically add the user account in the cloud). Subsequently, the app needs to configure the IOT device (ESP32) and send Wifi credentials of home network by connecting to ESP's WiFi AP. However, since both the app and IOT device are isolated from each other (except during the configuration process), I don't have much idea on how exactly will mobile app connect (creating a unique IOT thing automatically) and communicate with the isolated IOT device over AWS IOT cloud.
I have worked with some of the AWS services like AWS Lambda, DynamoDB, AWS IOT for quite a while, hence I am aware of the basics. One of the possible solutions for the above maybe Fleet Provisioning by a Trusted User feature provided by AWS IOT (I did read it's documentation, but still confused). We can also use ESP32's Bluetooth feature for the app to scan nearby devices with which it needs to connect with, but I am not sure how will the connection establish through AWS cloud.
Since I am a beginner to AWS cloud services, it'd be great if someone can provide a possible solution for the above in detail. Also if possible, please provide a solution which would work incase of a large-scale deployment of the above project.
The parts you may be missing are missing are 1) MQTT messaging, which is the IoT message broker service provided by IoT Core, and 2) the need for an application program interface to handle creating and activating devices.
MQTT is a lightweight and widely adopted messaging protocol that is designed for constrained devices.
Your devices (things) will need to communicate back to AWS via MQTT messages, which are captured and processed by the IoT Rules service.
These rules typically trigger Lambda functions, which implement the process logic you need for your application.
See https://mqtt.org/ and https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html
2} you will want to implement a simple API to handle device activation. The AWS API Gateway service makes it easy to implement APIs. For example, you can implement an endpoint for POST /device/ to create a device. That endpoint can trigger a Lambda handles device provisionsing.
I am trying to setup the backend services for consumer IoT device which includes the following activities (4 main high level services). These services are ultimately consumed by a web application or from mobile application through REST API's.
Instead of reinventing the wheel, I am evaluating AWS offerings which should partially or fully match these services. With my limited knowledge and research, I was able to find few AWS offering that matches, however they may not be complete and the most appropriate ones, and hence need advice. Below are the services and my findings.
Self user registration - Cognito with User Pools
Device provisioning (Adding/attaching one or more devices to a user) - No idea on how to handle this with AWS. Can we add the devices as user attributes ? or should we handle them at AWS IoT offering ?
Authentication & Authorization of users and devices - Again Cognito
Secure MQTT broker service for communication between device and users - AWS IoT seems to be matching, however I am not very much aware on configuring them and setting up user level access controls through API's. Can we use AWS IoT here ?
Device provisioning can be generally done in three ways:
Manually, by defining an IoT things in IoT Services. Certificates should be placed on IoT devices.
By using AWS IoT SDKs. AWS Credentials (or Temporary Security Credentials) is needed.
Using AWS CLI.
In IoT services, you can put devices together by adding them in a same group. User can access to the IoT device's data in two ways:
IoT devices save data in Databases and user can access databases and retrieve data.
Defining the application(Mobile App, ...) as an IoT device and putting it in the same group, so the application can subscribe/publish to the topics of different devices. Consider that, all IoT devices have credentials (CA, private and public key) which should be place on IoT devices and it will be needed during communication with AWS Broker. Also the policies for each device should be attached correctly.
Brief Solution:
I am storing IOT Device location and Service Provider Cars location in Dynamo DB via AWS Kinesis.
I am mapping One User with a specific service providers Cars and dispatching the Car to user’s shared location.
Problem: I need to track these mapped (service provider car and user location) on real time on service provider dashboard.
1. Does DynamoDB offers any direct API to publish and track these location real time?
2. Do I need to expose these mapped location and track then on dashboard via AWS Kinesis? Does AWS Kinesis offer such APIs?
Any suggestions?
If you are using MQTT IoT Core from AWS you have other options.
Subscribe to MQTT topic directly via WebSockets by AWS JS library.
You can Use Cognito to manage users and to have some acces to AWS services, this way you can get credentials to subscribe to the topic in MQTT using WebSockets like this link describe.
https://docs.aws.amazon.com/iot/latest/developerguide/protocols.html#mqtt-ws
Having own Users manager and using a EC2 Instance.
You can create an EC2 instance and put an AppServer with NodeJs for example and use SocketIO to comunicate to the WebBrowser of users with your own manager, this way they can receive realtime info, whatever you want. You can then use an AWS MQTT IoT NodeJS client and subscribe to the topics you want and just send it by WebSockets of SocketIO server (All this in the same WebApp). This way you need to consider number of users and data in traffic. You can have topics by users for example topics like this "car/tracking/mycarId" where mycarId is an unique Id that identifies this user/car and you canonly subscribe to these you want at the same time, this way you are not subscribed to all topics of your cars and you are not receiving innecesary data and you dont need so much proccess.
I have implemented the second one but now Im migrating my realtime block to Cognito and MQTT via WebSockets.
Regards, Héctor
I am building real-time dashboard using aws services, currently my application using MySQL database(RDS), which service and how would be designed real-time dashboard using Amazon web service, currently my approach is to use kinesis with redshift and connect my application via JDBC connector or use kinesis with s3 and use Athena to show real-time aggregators.
please help.
Thanks in advance.
Although the Amazon Kinesis Docs mention that Kinesis streams can be used to send updates to dashboards, Kinesis has no native mechanism to do this alone. For some very good security reasons it's unwise to allow clients (i.e. from a webpage you serve) to access backend services like Kinesis directly.
Instead, you'll want to set up your application layer (Java, in this case) to listen to the Kinesis streams and expose any relevant events to your client, storing any changes you need to keep track of in your database (RDS, in this case). For a real-time dashboard I'd recommend using something like WebSockets to send events in real time to your webpage from your server, as they're widely supported and easy to use. Heres a tutorial on how to implement WebSockets on GlassFish
When we upload data to S3, is it protected in transit by default (via HTTPS maybe)?
I found this article which, if I understand correctly, states S3 does not use HTTPS:
Amazon Simple Storage Service: You can still use HTTP with Amazon S3
and securely make authenticated requests. The service uses a different
secure signing protocol.
Should we in this case protect the data in transit with Client-Side Encryption?
The article you cited is obsolete. It was originally written in 2008, and apparently when updated in 2015, some of the outdated information was left in place.
The version refers to the particular algorithm for signing the request. These AWS services have deprecated the older, less-secure methods (signature versions 0 and 1) and will no longer allow them after September 2009.
Indeed, versions 0 and 1 are not supported.
A few AWS services don't support signature version 2:
Amazon Simple Storage Service: You can still use HTTP with Amazon S3 and securely make authenticated requests. The service uses a different secure signing protocol.
This is also inaccurate. S3 supports signature version 2 in all regions where signature version 2 was deployed. Regions launched in 2014 or later do not support V2 at all, they require Signature Version 4, and in those regions, S3 also requires Signature Version 4.
Importantly, though, none of this has anything at all to do with HTTPS.
From the same document:
Most AWS services accept HTTPS requests, including:
...
Amazon Simple Storage Service
Okay, so, let's revisit this line:
The service uses a different secure signing protocol.
This statement is not about encryption, or security of the payload. This is a statement about the secuity of the request authentication and authorization process -- its resistance to forgery and reverse-engineering -- whether or not the request is sent encrypted.
HTTPS is supported by S3, to protect data in transit.
Quoting from the Security section of the S3 FAQs:
You can securely upload/download your data to Amazon S3 via SSL
endpoints using the HTTPS protocol.
If you're using the https:// endpoint for S3, then your data in transit should be encrypted properly. The quote that you referred to in the question means that it's also possible to access S3 using http:// protocol, in which case the data wouldn't be encrypted in transit. See this related question.
If you were asking specifically about whether AWS CLI encrypts data in transit, then the answer is yes. See this question.
Also, please note that the primary purpose of using client-side encryption would be to encrypt data at rest, and to use an encryption algorithm of your own choosing. If you use client-side encryption but still use the http:// endpoint, your communication over the wire would still be unencrypted, technically speaking, because the cyphertexts being passed over the wire could be extracted by an attacker for analysis.
Update:
If you were asking specifically about AWS Java SDK, the default protocol is again https. Quoting from javadocs for AWS Java SDK:
By default, all service endpoints in all regions use the https
protocol. To use http instead, specify it in the ClientConfiguration
supplied at construction.
And from the javadocs for ClientConfiguration.getProtocol:
The default configuration is to use HTTPS for all requests for
increased security.
Client-side/server-side encryption's primary purpose is to secure data at rest. If anyone was to break open your cloud provider's data center somehow and steal the disks that had your data, you're making it difficult for them to get hold of your data in plaintext by encrypting it either client-side/server-side. Doing it client-side gives you the benefit of having more control on the encryption algorithm with the seemingly additional side-effect of your data not being transmitted in plaintext over the wire. However, the communication channel itself is not encrypted. If you were using a weak encryption algorithm for example, an attacker could still sniff the encrypted data over the wire and decrypt it. Also, it's important to know that using SSL means:
You as the client can be sure you're talking to AWS
Your communication with AWS is encrypted, so others can't intercept it
You have verification that the message received is the same as the
message sent
In essence, you definitely want to use SSL irrespective of whether you want to use client-side encryption or not.