Basically can I have Azure Function or web job with App Service environment listening to event hub without service endpoint. This way I will be able to accept data over public internet and also listen to in vnet?
Adding probable Security architecture:
Service Endpoints saves your time by doing IP firewall and NAT and gateway configuration changes in your VNET. If you will do those changes on your own, then yes you can access any public endpoint from your VNET.
Related
My architecture is the following:
Web app that a user can connect via the internet and that connects to an API that retrieves the data for the application;
API that receives the request from the web app, retrieves the data from the database and sends it back to the web app.
I'm running both of them in AWS app runner but with all the configurations set to public, but this is not ideal since my API should be kept from the internet and accessed only by the web app.
What I tried to do is set the API with a VPC connector in the outgoing network and a VPC endpoint as incoming:
It, indeed, is not accessible via the internet, but it works from inside the VPC (I'm able to connect to it using my VPN).
For the web app, I set the incoming as public (so I can access it via the internet) and the outgoing as a VPC connector (the same as the API one):
But this doesn't work. I can connect to the web app, but it cannot connect to the API. What's more confusing to me is that when I set the incoming for the web app to be a VPC endpoint, I could access it from inside the VPC using the VPN, and it could connect to the API without problems.
Does anyone know what I'm doing wrong?
I have an Apache server running the front end (Angular) which relies on an API which is hosted on the same instance as the Apache. I don't want my API (Express) open to public yet but need access to it with my front end which shares the same IP. Things I've tried,
Setting API base url as 'localhost' doesn't seem to work.
Adding a security rule in AWS security groups to allow connections only to the same IP (to itself) doesn't work.
Is there any workaround for this?
Connections to same IP are always open by default. You may need to add private IP of the ec2 instance as your api base URL. (Port you know better). Cors too should be enabled for that private IP.
First of all, using Angular as the front-end means needing an API that can access publicly you just need to implement securities, because you just serve the UI to the client user and their browsers are the one accessing the API not the server of the angular.
You can setup another API which can be deploy on the same server of your UI and same url which will serve as controller of your "Private API" that you can manage using Security groups in AWS
Replaced ${IP} to 172.17.0.1 so it can connect to the same EC2 after restarting. Add a rule for the inbound connection from the same SG
I have enabled Private Google API access for a VPC and I use this HTTP proxy solution described to connect my offsite datacenter to the Google Cloud backend.
Using the solution, I have verified that the Google object storage api's work, by using gsutil to move files across the offsite network.
However I am unable to connect to mqtt.googleapis.com that is required for cloud IOT.
I think this is because the MQTT broker running at mqtt.googleapis.com cannot be accessed via a private network unless it is also proxied like the HTTP proxy solution described above.
Meanwhile actual gsutil IOT commands work fine because I presume they are running over the Google HTTP API.
To solve this I see we'd need any one of the below, unless someone has different way to do this?
Run an MQTT broker proxy in the private VPC and route MQTT packets to the mqtt.googleapis.com . Is there a suitable MQTT proxy broker that we can use in this case?
If we get a range of public IP's that the mqtt bridge (mqtt.googleapis.com) is running at then we can simply build the network routes for this one use case. Is this available?
Would it work via the HTTP protocol bridge in IoT Core? Is using HTTP instead of MQTT an option?
I managed to get this to work using NGINX as a reverse proxy and stream the TCP traffic directly to mqtt.googleapis.com. Here are the steps to achieve this
Install Nginx with the --with-stream configuration flag . This builds Nginx with the functionality of a TCP streaming proxy
My Nginx conf file contains the following to point to Google's broker. The Nginx server is running in an instance in the VPC
/etc/nginx/nginx.conf
stream {
upstream google_mqtt {
server mqtt.googleapis.com:8883;
}
server {
listen 8883;
proxy_pass google_mqtt;
}
}
The internal private VPC has a DNS Server that resolves mqtt.googleapis.com to the IP of the Nginx server
EDIT
I now realise that I need to install a certificate on the server and validate the client certificate separately. I'm looking at https://github.com/xavierjohn/ClientCertificateMiddleware
I believe the certificate has to be from one of the CA's listed in AWS doco - http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-supported-certificate-authorities-for-http-endpoints.html
This certificate allows API Gateway to establish a HTTPS connection to the instance and it passes along the client certificate that can be validated.
ORIGINAL POST
I am trying to configure a new microservices environment and I'm having a few issues.
Here is what I'm trying to achieve:
Angular website connects to backend API via API Gateway (URL: gateway.company.com.au)
API Gateway is configured with 4 stages - DEV, UAT, PreProd and PROD
The resources in API Gateway are a HTTP Proxy to the back-end services via a Network Load Balancer. Each service in each stage will get a different port allocated: i.e. 30,000, 30,001, etc for DEV, 31,000, 31,000, etc for UAT
The network load balancer has a DNS of services.company.com.au
AWS ECS hosts the docker containers for the back-end services. These services are .NET Core 2.0 Web API projects
The ECS task definition specifies the container image to use and has a port mapping configured - Host Port: 0 Container Port: 4430. A host port of 0 is dynamically allocated by ECS.
The network load balancer has a listener for each microservice port and forwards the request to a target group. There is a target group for each service for each environment.
The target group includes both EC2 instances in the ECS cluster and ports are dynamically assigned by ECS
This port is then mapped by ECS/Docker to the container port of 4430
In order to prevent clients from calling services.company.com.au directly, the API Gateway is configured with a Client Certificate.
In my Web API, I'm building the web host as follows:
.UseKestrel(options =>
{
options.Listen(new IPEndPoint(IPAddress.Any, 4430), listenOptions =>
{
const string certBody = "-----BEGIN CERTIFICATE----- Copied from API Gateway Client certificate -----END CERTIFICATE-----";
var cert = new X509Certificate2(Encoding.UTF8.GetBytes(certBody));
var httpsConnectionAdapterOptions = new HttpsConnectionAdapterOptions
{
ClientCertificateMode = ClientCertificateMode.AllowCertificate,
SslProtocols = System.Security.Authentication.SslProtocols.Tls,
ServerCertificate = cert
};
listenOptions.UseHttps(httpsConnectionAdapterOptions);
});
})
My DockerFile is:
FROM microsoft/aspnetcore:2.0
ARG source
WORKDIR /app
EXPOSE 80 443
COPY ${source:-obj/Docker/publish} .
ENTRYPOINT ["dotnet", "microservice.company.com.au.dll"]
When I use Postman to try and access the service, I get a 504 Gateway timeout. The CloudWatch log shows:
(e4d594b7-c8f3-11e7-8458-ef6f94e65b64) Sending request to http://microservice.company.com.au:30000/service
(e4d594b7-c8f3-11e7-8458-ef6f94e65b64) Execution failed due to an internal error
(e4d594b7-c8f3-11e7-8458-ef6f94e65b64) Method completed with status: 504
I've been able to get the following architecture working:
API Gateway
Application Load Balancer - path-based routing to direct to the right container
ECS managing ports on the load balancer
The container listening on HTTP port 80
Unfortunately, this leaves the services open on the DNS of the Application Load Balancer due to API Gateway being able to only access public load balancers.
I'm not sure where it's failing but I suspect I've not configured .NET Core/Kestrel correctly to terminate the SSL using the Client Certificate.
In relation to this overall architecture, it would make things easier if:
The public Application Load Balancer could be used with a HTTPS listener using the Client Certificate of API Gateway to terminate the SSL connection
API Gateway could connect to internal load balancers without using Lambda as a proxy
Any tips or suggestions will be considered but at the moment, the main goal is to get the first architecture working.
I more information is required let me know and I will update the question.
The problem was caused by the security group attached to the EC2 instances that formed the ECS cluster not allowing the correct port range. The security group for each EC2 instance in the cluster needs to allow the ECS dynamic port range.
i created an android application that requires use of web service
i want it to be able to access the app everywhere therefore i need
my web services to be public with an external ip so i can access
what is the best way to do it?
I have an Amazon Web Services account i dont know if created an instance and run the web services there will be the best solution
My big problem with Amazon instance is that it takes a while to show in the app the result of the web service
Any ideas in how to make my web service public?
It appears that your requirement is:
Expose a public API endpoint for use by your Android application
Run some code when the API is called
There are two ways you could expose an API:
Use Amazon API Gateway, which that can publish, maintain, monitor, and secure APIs. It takes care of security and throttling. A DNS name is provided, which should be used for API calls. When a request is receive, API Gateway can pass the request to a web server or can trigger an AWS Lambda function to execute code without requiring a server.
Or, run an Amazon EC2 instance with your application. Assign an Elastic IP Address to the instance, which is a static IP address. Create an A record in Amazon Route 53 (or your own DNS server) that points a DNS name to that IP address.