I have successfully deployed a Python (FastAPI) API on the Digital Oceans App Platform. There is a POST API endpoint that returns an Excel file as a response. When the file size is 7 KB, it is returning fine. When the file size is 80 KB, a 0 bytes file gets downloaded.
The cURL I am using is similar to
curl -X 'POST' \
'https://example.com/api' \
-H 'accept: */*' \
-H 'header: value' \
-H 'Content-Type: multipart/form-data' \
-F 'myfile=#myfile.pdf;type=application/pdf' \
--output output.xlsx
When the output.xlsx is less than 10 KB, it is getting saved just fine, however when it is more, the output is a 0 bytes file.
Is there a limit on the response size limit on the DO App Platform? Could not find any such thing mentioned on https://docs.digitalocean.com/products/app-platform/details/limits/
The API code is similar to https://fastapi.tiangolo.com/advanced/custom-response/#fileresponse
Not using any NGINX proxy as well. Just a simple Docker based deployment as given in https://fastapi.tiangolo.com/deployment/docker/#build-a-docker-image-for-fastapi
Related
I am trying to make a curl request to get ECR Authorization token instead of using aws cli.
I referred https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_GetAuthorizationToken.html
And to generate signature, I referred this : https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
curl -v --location --request POST 'https://ecr.us-east-1.amazonaws.com:443' -H 'Accept-Encoding: identity' -H 'Content-Length: 1590' -H 'X-Amz-Target: AmazonEC2ContainerRegistry_V20150921.GetAuthorizationToken' -H 'X-Amz-Date: 20151129T221940Z' -H 'User-Agent: aws-cli/1.17.14 Python/2.7.5 Linux/3.10.0-957.1.3.el7.x86_64 botocore/1.14.14' -H 'Content-Type: application/x-amz-json-1.1' -H 'Authorization: AUTHPARAMS {AWS4-HMAC-SHA256 Credential=AKIA*******/20200514/ap-south-1/ecr/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=ffe96d25bba760d7502993a1dbf0*********************}'
Response returned after few minutes is :
* Empty reply from server
* Connection #0 to host ecr.ap-south-1.amazonaws.com left intact
curl: (52) Empty reply from server
I basically want to use this token in making a curl request to return ECR image tags:
curl -i -s -H "Authorization: Basic TOKEN" https://90********.dkr.ecr.ap-south-1.amazonaws.com/v2/image-name/tags/list
If i can request the image tags using above command without Token generation....please provide the answer.
this error means that you are receiving a 502 response error, bad gateway. I had a similar when tried to run a curl command via ssh inside an instance hosted on EC2, so I figured out that I had deployed my image with the port mappings incorrectly. I would recommend you to check the following:
Check if your security group allows traffic on port 433
Check if you have deployed your container with the port mappings correctly to ECR, like this:
suppose you have an image called list
To run in your local machine you should execute the following command
docker run -d -p 80:433 list
make sure you have mapped these ports correctly on your Dockerfile, then tag your image
docker tag list 90********.dkr.ecr.ap-south-1.amazonaws.com/v2/image-name/tags/list
and push to ECR
docker push 90********.dkr.ecr.ap-south-1.amazonaws.com/v2/image-name/tags/list
once you have your image pushed to ECR you can run it
docker run -d -p 80:433 90********.dkr.ecr.ap-south-1.amazonaws.com/v2/image-name/tags/list
and then you can curl your image
curl http://ecr-image/your/path
I tried to create AWS Device farm remote session, which i am able to do it successfully. The response JSON of the created remote session has an endpoint (wss) and the hostAddress(IP). Rather then login into AWS device farm to interact with the device. I wanted to provide remote access directly on my own web page (I am not sure whether its possible). Hoping it can be rendered under a canvas tag.
Though i do not have experience on the socket, i just tried some sample code to connect with the received Web Socket URL.
var wsUri = "wss://devicefarm-interactive.us-west-2.amazonaws.com/?X-Amz-Date=*&X-Amz-Credential=*&X-Amz-Algorithm=*&X-Amz-SignedHeaders=host&X-Amz-Signature=*&X-Amz-Security-Token=*";
var websocket = new WebSocket(wsUri);
websocket.send('ping');
Below is the error the console.
Connection closed before receiving a handshake response.
Any sample links to implement would be helpful
I'm able to reproduce that error. I looked for examples of using the web socket connection using the networking tab after inspecting the page in chrome(since it has a web socket debugger built into it) which will show the following:
Basically how that appears to be working is it's sending us a constant stream of images.
When I copy as cURL command this is the result
curl 'wss://devicefarm-interactive.us-west-2.amazonaws.com/?X-Amz-Date=20190518T211708Z&X-Amz-Credential=ASIAIGY76PSQXN5NZ3UA%2F20190518%2Fus-west-2%2Fdevicefarm%2Faws4_request&X-Amz-Algorithm=AWS4-HMAC-SHA256&arn=arn%3Aaws%3Adevicefarm%3Aus-west-2%111122223333%3Asession%3A8f4af46d-8f86-4dcc-8324-e691ce3723f3%2F6b4dc632-188f-45b9-be32-3e9aa6881ed3%2F00000&X-Amz-SignedHeaders=host&X-Amz-Signature=39b45e1489d44dfcb904d58e59e985844416061a828aa75750d5a67db36c55dd&X-Amz-Security-Token=someTokenValue&path=video' -H 'Pragma: no-cache' -H 'Origin: https://us-west-2.console.aws.amazon.com' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'Sec-WebSocket-Key: 60RGfCwWuULib6NmeoC2fA==' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36' -H 'Upgrade: websocket' -H 'Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits' -H 'Cache-Control: no-cache' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' --compressed
When I did the following CLI command I get the endpoint back:
aws devicefarm get-remote-access-session --arn arn:aws:devicefarm:us-west-2:111122223333:session:8f4af46d-8f86-4dcc-8324-e691ce3723f3/6b4dc632-188f-45b9-be32-3e9aa6881ed3/00000 --region us-west-2 --query remoteAccessSession.endpoint
"wss://devicefarm-interactive.us-west-2.amazonaws.com/?X-Amz-Date=20190518T212323Z&X-Amz-Credential=ASIAIU5CA7GBP5IEBR6Q%2F20190518%2Fus-west-2%2Fdevicefarm%2Faws4_request&X-Amz-Algorithm=AWS4-HMAC-SHA256&arn=arn%3Aaws%3Adevicefarm%3Aus-west-2%3A111122223333%3Asession%3A8f4af46d-8f86-4dcc-8324-e691ce3723f3%2F6b4dc632-188f-45b9-be32-3e9aa6881ed3%2F00000&X-Amz-SignedHeaders=host&X-Amz-Signature=aaaaqaqbbbb4e9f09b7312715c295a11b77bc0d9e7b21dcb61422a61f78a1f&X-Amz-Security-Token=FQoGZXIvYXdzEFcaDD2E90%2Bsp3i%2F%2F8cBbyKDA2EGKkFYSvXDR%2Fb7LfS%2FpQEPCWFVhe9eCOTSussvshjldx69CEFvVgV3JYtOvm2yu0UMVAxlDYlujvpMfSNwLx7FH%2B42k9qGYuvy5dQbVLg%2F%2BCRuyK9OjCxpD5pUfQ9b81U6LawcI2I1CekXeTgapRuTK9tCPcGtNOlxAvWQUVlyDGTtmqjz7vRlostquMoenNr9UB1v8jx0NSo1YIlrgY8YvZV2o5pcbYiI9I9CBD0%2F3snJZAyQtmPZkMT9gr9hI0jgX1X5MlOuarFmm%2F2Sn%2FH8L3ewMQXhvuho3OTNZTISBmUgJAbZSmQcazuDmjXqPkoNpYYcUb92vd2w5MbRfFSa5SHHXUMVcE5Wsop3BzwJyj%2FNyl59BdjFWdo82NgSFP6OBjYLjiux3hR2dx86ILJ9tfNMNfq0WXzL3Z%2BqecwMTxlxrLfZmPftsUDaO5RPtOP9uuI%2BPjfIOWOV7uFy9GjKG4HKFY%2BZVGgWhb1fVVG7%2BYHbPxgMaAKI3YJqmM9IIy8%2FdCjL74HnBQ%3D%3D%7CMjA1LjI1MS4yMzMuMTc5"
which appears to be signed. Using wscat I tried to connect to it.
endpoint=$(aws devicefarm get-remote-access-session --arn arn:aws:devicefarm:us-west-2:111122223333:session:8f4af46d-8f86-4dcc-8324-e691ce3723f3/6b4dc632-188f-45b9-be32-3e9aa6881ed3/00000 --region us-west-2 --query remoteAccessSession.endpoint)
wscat -c $endpoint
Error:
/usr/local/lib/node_modules/wscat/node_modules/ws/lib/websocket.js:455
throw new Error(Invalid URL: ${this.url});
I'll need to spend some more time on this later but I think some of this content is helpful so I posted it.
I have followed the instructions from https://github.com/spring-cloud-services-samples/cook and managed to install and run Config Server in PCF environment (SERVICE: Config Server, PLAN: standard).
I'm now trying to hit /encrypt endpoint of the p-config-server service, in order to encrypt new value. I'm following the instructions at http://docs.run.pivotal.io/spring-cloud-services/config-server/configuring-with-git.html#encryption-and-encrypted-values:
TOKEN=$(curl -k ACCESS_TOKEN_URI -u CLIENT_ID:CLIENT_SECRET -d
grant_type=client_credentials | jq -r .access_token); curl -k -H
"Authorization: bearer $TOKEN" -H "Accept: application/json"
URI/encrypt -d 'VALUE'
...but I always get:
{
"error": "access_denied",
"error_description": "Access is denied"
}
On the other side, if I try to get standard endpoint, to get config for an app, I'm able to retrieve JSON containing app properties. E.g.
TOKEN=$(curl -k ACCESS_TOKEN_URI -u CLIENT_ID:CLIENT_SECRET -d
grant_type=client_credentials | jq -r .access_token); curl -k -H
"Authorization: bearer $TOKEN" -H "Accept: application/json"
URI/my-app/default
... gives result like:
{"name":"my-app","profiles":["default"],"label":null,"version":"bb6e64592ced731ebba272430291a595e0f14a77","state":null,"propertySources":[{"name":"https://github.com/some-user/config/my-app.yml","source":{"my-property.name":"Test123"}}]}
Any idea why I can not access /encrypt endpoint?
Thank you.
Btw, I'm executing the command in CentOS Linux release 7.4.1708, with installed jq (command-line JSON processor).
I've got the answer from CloundFoundry IT support. In my CF environment, "encrypt" endpoint should have a trailing slash (/). So it should be ...URI/encrypt/ -d 'VALUE'. Maybe it helps someone.
One more hint I've got: There is a CF CLI plugin for the Spring-Cloud-Services which you could use for convenience.
https://github.com/pivotal-cf/spring-cloud-services-cli-plugin
cf install-plugin -r CF-Community "Spring Cloud Services"
cf help config-server-encrypt-value
Hi Actually you need to hit cf env command first and take note of configuration values from that which for sample looks like below:
{
"VCAP_SERVICES": {
"p-config-server": [
{
"credentials": {
"access_token_uri": "https://p-spring-cloud-services.uaa.cf.wise.com/oauth/token",
"client_id": "p-config-server-876cd13b-1564-4a9a-9d44-c7c8a6257b73",
"client_secret": "rU7dMUw6bQjR",
"uri": "https://config-86b38ce0-eed8-4c01-adb4-1a651a6178e2.apps.wise.com"
},
[...]
and then use those values in your curl bash script. for example
TOKEN=$(curl -k https://config-86b38ce0-eed8-4c01-adb4-1a651a6178e2.apps.wise.com -u p-config-server-876cd13b-1564-4a9a-9d44-c7c8a6257b73:rU7dMUw6bQjR -d
grant_type=client_credentials | jq -r .access_token); curl -k -H
"Authorization: bearer $TOKEN" -H "Accept: application/json"
URI/ENDPOINT | jq
Basically following values are required:
ACCESS_TOKEN_URI with the value of credentials.access_token_uri
CLIENT_ID with the value of credentials.client_id
CLIENT_SECRET with the value of credentials.client_secret
URI with the value of credentials.uri
Replace ENDPOINT with the relevant endpoint:
application/profile to retrieve configuration from a Config Server service instance
eureka/apps to retrieve the registry from a Service Registry service instance
Then I think you will no more get access denied response.
I'm trying to POST the simple data to the Parse server using REST, but not able to do so, here is the code
#include<curl/curl.h>
curl -X POST \
-H "X-Parse-Application-Id: xxxxxxxxxx" \
-H "X-Parse-REST-API-Key: xxxxxxxxxxxx" \
-H "Content-Type: application/json" \
-d '{"score":1337,"playerName":"Sean Plott","cheatMode":false}' \
https://api.parse.com/1/classes/GameScore
what is X here? do we need to define x? what connection settings need to be used to post the data successfully? and where is the return object is Json, where is is stored?
thanks.
You are trying to use a terminal command in C++, this won't work. X is a command line option for curl.
Below is a basic example of an HTTP POST request submitted with Curl using C. You should be able to adapt this to your needs.
http://curl.haxx.se/libcurl/c/http-post.html
I am building a web service for a web application, and I would like a simple tool to test this as I am developing. I have tried some firefox plug-ins (Poster, 'REST Client'), and even though these work fine I have been unable to upload files with them.
Also, I would rather have a command-line tool that I can use to easily write a set of integration tests for this web service and that I can send to consumers of this web service as an example.
I know that curl can work for this but would like a few examples, especially around authentication (using HTTP Basic) and file uploads.
Answering my own question.
curl -X GET --basic --user username:password \
https://www.example.com/mobile/resource
curl -X DELETE --basic --user username:password \
https://www.example.com/mobile/resource
curl -X PUT --basic --user username:password -d 'param1_name=param1_value' \
-d 'param2_name=param2_value' https://www.example.com/mobile/resource
POSTing a file and additional parameter
curl -X POST -F 'param_name=#/filepath/filename' \
-F 'extra_param_name=extra_param_value' --basic --user username:password \
https://www.example.com/mobile/resource
In addition to existing answers it is often desired to format the REST output (typically JSON and XML lacks indentation). Try this:
$ curl https://api.twitter.com/1/help/configuration.xml | xmllint --format -
$ curl https://api.twitter.com/1/help/configuration.json | python -mjson.tool
Tested on Ubuntu 11.0.4/11.10.
Another issue is the desired content type. Twitter uses .xml/.json extension, but more idiomatic REST would require Accept header:
$ curl -H "Accept: application/json"
From the documentation on http://curl.haxx.se/docs/httpscripting.html :
HTTP Authentication
curl --user name:password http://www.example.com
Put a file to a HTTP server with curl:
curl --upload-file uploadfile http://www.example.com/receive.cgi
Send post data with curl:
curl --data "birthyear=1905&press=%20OK%20" http://www.example.com/when.cgi