I have created one broker on Amazon MQ and got a SSL endpoint on port 61617. I was looking for one non SSL endpoint as well (like we can have tcp on 61616 on Active MQ). Does Amazon MQ provides only SSL? Is there anyway, we can get the TCP endpoint as well?
AmazonMQ only provides SSL endpoint, it does not expose and TCP endpoint like activeMQ. But it works and connect equally well when you switch from activeMQ tcp endpoint to amazonMQ SSL.
For example:
activemq.broker.url =
failover:(tcp://abc1.gogole.com:61616,tcp://abc1.gogole.com:61616)?randomize=false&maxReconnectAttempts=10
amazonmq.broker.url =
failover:(ssl://efg-1.mq.us-west-2.amazonaws.com:61617,ssl://efg-2.mq.us-west-2.amazonaws.com:61617)?randomize=false&maxReconnectAttempts=5
Any specific use case on why are you looking for TCP endpoint?
Related
In MSK to communicate with brokers by using TLS(SSL) we have used 9094 port and able to communicate MSK cluster with the required keystore and truststore configurations.
And to communicate with brokers by using SASL_SCRAM we have used 9096 port and able to communicate MSK cluster with username and password configurations.
And now we want to configure both SASL_SSL to authenticate client while connecting to broker nodes as a mandatory configurations at client side.
NOTE: We are able to configure both SASL_SSL security protocol on 9096 port but the client configuration considered as the TLS configurations are optional because if we remove TLS properties, still it is able to connect broker nodes.
Im testing this connection with kafka console producer like below:
./bin/kafka-console-producer.sh --broker-list b-1.xxx:9096,b-2xx:9096 --producer.config ./bin/client_sasl_ssl.properties --topic TestTopicSASLSSL
File: client_sasl_ssl.properties
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
#ssl.truststore.location=/path/to/kafka.client.truststore.jks
#ssl.keystore.location=/path/to/kafka.client.keystore.jks
#ssl.keystore.password=psswd
#ssl.key.password=psswd
Please note that im able to produce data with or without the above commented ssl properties, which seems optional.
Any option to mandate both SASL and TLS auth protocol to restrict client connection.
I have a Java application configured with some self signed certificates that communicates with ActiveMQ version 5.13.0 over SSL/TLS version 1.2. The relevant self signed certificates reside in their respective keystores and truststores. This connection over TLS works just fine on my local Windows machine, clients without the proper certificates are unable to communicate with the broker and clients with the proper certificates can.
However, this does not work when the same code and keystores are used on an AWS EC2 instance. I have the same version of ActiveMQ installed there and am using the very same keystores and truststores on the broker and client side. Clients without any certificates configured are able to connect to the broker and communicate.
I would like to understand if SSL/TLS for ActiveMQ must be configured differently on a Linux machine or if there is something else that I am missing.
Snippets from the activemq.xml file that enable activemq to use SSL/TLS:
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<sslContext>
<sslContext keyStore="file:${activemq.base}/conf/broker.ks"
keyStorePassword="changeit" trustStore="file:${activemq.base}/conf/broker.ts"
trustStorePassword="changeit"/>
</sslContext>
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxInactivityDuration=300000&wireFormat.maxFrameSize=104857600&jms.messagePrioritySupported=false"/>
<transportConnector name ="ssl" uri="ssl://0.0.0.0:61714?transport.enabledProtocols=TLSv1.2"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
Answering my own query.
I handle the Java client and that client connects to port 61714 that is designated for SSL.
The folks dealing with the IoT device side told me that these devices default to port 1883 for MQTT connections and port 8883 for secure MQTT connections.
This can be configured by adding the below line to the transport connectors :
<transportConnector name="mqtt+ssl" uri="mqtt+ssl://0.0.0.0:8883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
The device has some constraints due to which it cannot connect to an SSL port and publish MQTT messages. The Java client on the other hand has no issues connecting to the SSL port and publishing and consuming MQTT messages, so adding the above line resolved this.
If needed, one could comment out the transport connector for port 1883 so that no clients without the needed certificates are able to connect to the MQTT broker.
According to this document, using MQTT on port 443 in AWS IoT requires configuring the ALPN extension with the x-amzn-mqtt-ca protocol.
I use the AWS IoT C++ SDK with OpenSSL. When creating the network::OpenSSLConnection object, I configured the endpoint_port parameter to 443 and enable_alpn to true. Where can I set the protocol to x-amzn-mqtt-ca? The connection seems to work that way, and TCPView shows that it actually uses the HTTPS port.
Did you try setting the following two protected variables too during network::OpenSSLConnection(..)?
SSL_CTX *p_ssl_context_; ///< SSL Context instance
SSL *p_ssl_handle_; ///< SSL Handle
More info on how to set specific ALPN protocols is on OpenSSL docs # here.
Hope that helps!
I have enabled Private Google API access for a VPC and I use this HTTP proxy solution described to connect my offsite datacenter to the Google Cloud backend.
Using the solution, I have verified that the Google object storage api's work, by using gsutil to move files across the offsite network.
However I am unable to connect to mqtt.googleapis.com that is required for cloud IOT.
I think this is because the MQTT broker running at mqtt.googleapis.com cannot be accessed via a private network unless it is also proxied like the HTTP proxy solution described above.
Meanwhile actual gsutil IOT commands work fine because I presume they are running over the Google HTTP API.
To solve this I see we'd need any one of the below, unless someone has different way to do this?
Run an MQTT broker proxy in the private VPC and route MQTT packets to the mqtt.googleapis.com . Is there a suitable MQTT proxy broker that we can use in this case?
If we get a range of public IP's that the mqtt bridge (mqtt.googleapis.com) is running at then we can simply build the network routes for this one use case. Is this available?
Would it work via the HTTP protocol bridge in IoT Core? Is using HTTP instead of MQTT an option?
I managed to get this to work using NGINX as a reverse proxy and stream the TCP traffic directly to mqtt.googleapis.com. Here are the steps to achieve this
Install Nginx with the --with-stream configuration flag . This builds Nginx with the functionality of a TCP streaming proxy
My Nginx conf file contains the following to point to Google's broker. The Nginx server is running in an instance in the VPC
/etc/nginx/nginx.conf
stream {
upstream google_mqtt {
server mqtt.googleapis.com:8883;
}
server {
listen 8883;
proxy_pass google_mqtt;
}
}
The internal private VPC has a DNS Server that resolves mqtt.googleapis.com to the IP of the Nginx server
I'm scaling up TCP service on AWS now. I investigated for a while and find
"sticky sessions" is using cookie to attach all requests from one user to specific instance via HTTP protocol. However, for TCP connection, is there any solution to open up "stickness" in TCP connection?
Very Thanks.
No. AWS does not provide any mechanism for TCP sticky sessions. That's the price you pay for being deeper in the network stack.