Problem:
Goal - Integrate Bitbucket server which is running in a container/EC2 from AWS-1 account with [ APIGateway-->Lambda-->S3-->Codepipeline ] in AWS-2 account for autodeploying of code whenever there is push in to the bitbucket.
I had setup the infra on AWS-2 account by running cloudforamtion. And while testing / invoking the API gateway from Bitbucket in webhook, it is giving the 'forbidden' error
[
Response details
HTTP status:403
Headers
x-amz-apigw-id: xxxxxxxxxxxxxx
Server: Server
Connection: keep-alive
x-amzn-RequestId: xxxxxxxxxxxxxxxxxxxxx
x-amzn-ErrorType: ForbiddenException
Content-Length: 23
Date: xxxxxxxxxxxxxx
Via: 1.1 localhost (Apache-HttpClient/4.5.5 (cache))
Content-Type: application/json
Body
**{"message":"Forbidden"}**
]
I have installed bitbucket server on my laptop to test.
However, when I invoke api gateurl from the postman or bitbucket server on my laptop I am getting expected exception from lambda/api gateway
[
Response details
HTTP status:502
Headers
x-amz-apigw-id: xxxxxxxxxxxxxxxxxxxxx
Connection: keep-alive
x-amzn-RequestId: xxxxxxxxxxxxxxxxxxxxx
x-amzn-ErrorType: InternalServerErrorException
Content-Length: 36
Date: Tue, 14 Jul 2020 16:25:21 GMT
X-Amzn-Trace-Id: Root=xxxxxxxxxxxxxxxxxxxxx
Via: 1.1 localhost (Apache-HttpClient/4.5.5 (cache))
Content-Type: application/json
Body
{"message": "Internal server error"}
]
The owner of AWSaccount-1 mentioning that resource policy for bit bucket allows all inbound and outbound calls. However he did run curl command from bitbucket hosting container/ec2, got same forbidden error.
I am so confused here, can you please help me or let me know I need to provide more information.
note - this is my first question in stackoverflow :)
Related
In our GKE Kubernetes cluster we have a deployment running a pod with a third-party unmodifiable legacy application exposing an HTTP service (port :80) protected with based authentication. The cluster is exposed to the outside Internet through a GCP load balancer that takes care of terminating HTTPS connections.
When, from the Internet, I try to access the service using CURL, it works perfectly, I got an HTTP status 200.
curl -vvv 'https://username:password#portale-ambrogio-test.sigemi.cloud/admin'
* Trying 34.107.254.133...
* Connected to portale-ambrogio-test.sigemi.cloud (34.107.254.133) port 443 (#0)
* found 129 certificates in /etc/ssl/certs/ca-certificates.crt
* found 516 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: *.sigemi.cloud (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=*.sigemi.cloud
* start date: Tue, 24 May 2022 00:00:00 GMT
* expire date: Wed, 24 May 2023 23:59:59 GMT
* issuer: C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
* compression: NULL
* ALPN, server accepted to use http/1.1
* Server auth using Basic with user 'username'
> GET /admin HTTP/1.1
> Host: portale-ambrogio-test.sigemi.cloud
> Authorization: Basic aR.......................==
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Ntrip-Version: Ntrip/2.0
< Access-Control-Allow-Origin: *
< Server: NTRIP BKG Caster/2.0.39
< Content-Type: text/html
< Date: Tue, 06 Sep 2022 15:53:09 GMT
< Via: 1.1 google
< Transfer-Encoding: chunked
< Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
When I access the same resource through the browser, it fails instead.
Do you know why that could happen? I think it is something related to some security policy implemented by the browser but what?
[EDIT] Now with the browser I'm explicitly typing username and password in the authentication pop up to avoid the problem pointed out by #john-hanley
The problem was caused by few facts.
Our web application was not fully compliant with any of the HTTP specs (1.1, 2 and 3) regarding the name of the HTTP headers. In particular HTTP/1.1 states that header names should be considered case insensitive, while HTTP/2 and HTTP/3 states that headers should be lower case. Our application expected header names to be capitalized, so it always looked for the "Authentication" header.
When modern browsers connect to an HTTP web server, they usually use HTTP/1.1 and in this case, they tend to capitalize header names, so browsers actually send a header which name is spelled "Authorization".
However when modern browsers connect to HTTPS, they upgrade to HTTP/2 or HTTP/3 id supported by the web server. If they upgrade to such protocols, they obey it, so the send the authorization header in lower case, so "authorization".
Our reverse proxy is provided by GCP, that indeed supports HTTP/3. So, when the browser connects to our GCP reverse proxy it switches to HTTP/3 and start sending lower case headers.
The reverse proxy the forward the request to our application in lower case.
And because of 1) the app is unable to authenticate.
Hope this report can help, because I spend a whole day trying to figure it out (and I actually learnt about HTTP/2 and HTTP/3).
The solution was obviously to fix our application and made it fully HTTP compliant.
The CLI curl is converting username:password from the URI into Authorization: Basic ....
Chrome stopped supporting URI credentials in the address bar after version 52. Internally, this is still supported via JavaScript.
I'm developing a lambda function and using AWS SAM to build my stack. I've noticed that when I test locally using sam local start-api, the API responds with HTTP 1.0 for HTTP 1.1 requests. Here's a sample output from curl:
* Trying 127.0.0.1:3000...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET / HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 403 FORBIDDEN
< Content-Type: application/json
< Content-Length: 43
< Server: Werkzeug/1.0.1 Python/3.7.10
< Date: Fri, 03 Dec 2021 04:21:40 GMT
<
{ [43 bytes data]
* Closing connection 0
{
"message": "Missing Authentication Token"
}
As you can see in the output above, an HTTP 1.1 request is sent to the API but the API responds with HTTP 1.0. This happens for all requests.
I assume that when I push this out to AWS, the API will also use HTTP 1.0. Does anyone know if I can configure which HTTP protocol version gets used for API gateway proxy to lambda function? Also, how do I set a minimum TLS version of 1.2?
Tell curl to use http 1.1:
curl http://api-url -vvv --http1.1
After deploying a blazor server side project to an Amazon Web Services environment via AWS Elastic Beanstalk, the website experiences frequent disconnects that I do not understand.
These disconnects do not happen locally when testing.
Errors:
[2020-04-30T16:29:18.326Z] Error: Connection disconnected with error 'Error'.
and
Failed to load resource: the server responded with a status of 504 ()
The request causing the 504 errors has a header like so:
Request URL: https://mywebserver/_blazor?id=UOPQELxzuEcaGbpNUQA01Q&_=1588264098305
Request Method: GET
Status Code: 504
Remote Address: 3.11.236.203:443
Referrer Policy: no-referrer-when-downgrade
content-length: 550
content-type: text/html
date: Thu, 30 Apr 2020 16:29:20 GMT
server: awselb/2.0
status: 504
:authority: mywebserver
:method: GET
:path: /_blazor?id=UOPQELxzuEcaGbpNUQA01Q&_=1588264098305
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-GB,en-US;q=0.9,en;q=0.8
content-type: text/plain;charset=UTF-8
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36
x-requested-with: XMLHttpRequest
This issue seems remarkably similar to a Github issue posted here, that is currently being investigated by Microsoft, although this issue is not AWS-related: https://github.com/dotnet/aspnetcore/issues/19094
Any help would be vastly appreciated!
i managed to fix this in the end.
Apparently the issue was caused by the use of LongPolling. LongPolling issues requests and waits 100 seconds before cancelling the request. By setting the timeout to be greater than the default of 60 seconds in the Elastic Load Balancer in my AWS settings, the disconnects stopped. 110 seconds was recommended as a safe timeout value.
Issue on Asp.NET github that I opened for those interested: https://github.com/dotnet/aspnetcore/issues/21369
I am having trouble establishing a websocket connection. I have an aws ec2 instance that is running an SSL websocket server on a subdomain: api.example.com, on port 9999. My client application is running at www.example.com, the client sets up a listener:
new WebSocket('wss://api.example.com:9999');
This results in the browser giving me:
net::ERR_CONNECTION_REFUSED
So my next idea was since my application load balancer already has an https listener setup on port 443 going to my api server, I changed my websocket listener setup to:
new WebSocket('wss://api.example.com/socket');
Then I made my nodejs (restify) webserver on the ec2 instance catch /socket with plans to redirect it to localhost:9999... However, I put a console.log in that /socket endpoint prior to redirecting, and there is no logging happening-- I also tried wss://api.example.com:443/socket, neither seems to hit the endpoint in my webserver.
Yet if I use curl, it works fine:
~>curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Host: api.example.com" -H "Origin: https://www.example.com" https://api.example.com/socket
HTTP/2 301
date: Tue, 26 Jun 2018 02:27:07 GMT
content-type: application/json
content-length: 4
location: https://localhost:9999/localhost
server: restify
access-control-allow-origin: https://www.example.com
vary: origin
access-control-expose-headers: api-version, content-length, content-md5, content-type, date, request-id, response-time
although https://localhost:9999/localhost seems a little weird-- but regardless, when I use curl, it hits the endpoint and I see the log.
I came across a situation where I have to invoke a SOAP webservice (deployed in another server) from one of the production server manually and check whether everything is fine.
as these are all live servers there are no network tools like wget, curl and nc are available. I tried checking for a solution is google but no luck.
As a workaround I can write a java client socket and invoke the service but even that is not allowed in here.
telnet is there but am not sure how to make it work for my case.
Is there any other way to invoke remote services without these tools?
After trying few hours finally, I was able to invoke SOAP service with telnet as below
first open a TCP connection to the remote server as below.
$> telnet hostname portname
Once it is connected, frame a request as one of the below methods and paste on the screen and press enter key two times.
There are two ways we can call a service.
Method 1: instead of mentioning endpoint path in POST header, we can give it in SOAPAction header.
POST / HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "<endpoint URL from WSDL>"
Content-Length: <number of bytes you are sending in body section>
Host: <hostname>:<port>
Proxy-Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<SOAP Request payload>
Method 2: mentioning endpoint path in the request header itself, so we can give empty value in "" SOAPAction header (it means request path itself is the Endpoint path).
POST /soap/server HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: ""
Content-Length: <payload size>
Host: hostname:port
Proxy-Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<SOAP Request payload>
Response: Once the call invoked successfully, the response will be printed as below
HTTP/1.1 200 OK
Content-Type: text/xml;charset=UTF-8
Content-Length: <response payload size>
<SOAP response payload>
For more information on SOAPHeader check this link
Note: Make sure the length of the request payload is correct before sending it.