I have deployed a LAMP stack application on AWS. I need to monitor that using CloudWatch.
Can someone guide me on how to use the CloudWatch API for GetMetrics for CPU utilization? The AWS documentation is very scarce.
I see that the putmetrics call will let me create my own metrics.
My requirement is that I need to display those metric results in a mobile app.
My app monitors a project deployed on AWS. The alerts and metrics that come in must stream into the app.
I don't want just the metrics data in the AWS console,
I want it viewable in my mobile app. The app is developed in MEAN stack.
I must also add that the app is deployed on AWS and the application that is
being monitored is also in there(its a LAMP stack). I have managed to set 2 endpoints(HTTP and DB) and I have written
simple scripts in Javascript to monitor them. But ideally they should happen via Cloudwatch.
Providing a piece of code that replicates the issue that you are seeing normally allows who sees the question to help you better than guessing what you're doing.
Are you using an SDK to do this? What language/version?
here are links to the API docs:
http://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricStatistics.html
http://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_ListMetrics.html
The pattern is to list the metrics and after that use the result and feed it into getmetricsstatistics.
In your specific case, googling the issue a bit before might answer the question before you ask it on SO. For example:
https://forums.aws.amazon.com/thread.jspa?messageID=295740
This can happen when you are hitting the wrong endpoint. Check if you are hitting endpoint of the right AWS service.
For example, trying to hit DynamoDB's endpoint when you want to access CloudWatch APIs.
Related
I have several applications deployed on AWS EKS as microservices.
They are also deployed across different AWS accounts and have dependencies on each other.
I would like some kind of dashboard that says where exactly a request failed in a long flow of request across say 10 different microservices (m1 calls m2 and so on till m5 and say 1 request fails at m2 and another at m4, i would like to see a dashboard that shows where this flow got interrupted for each request).
How could I achieve to get this dashboard?
FOund this tool named ZIkpin which provides pretty much what I am looking for.
Any alternatives available? DOes ELK provide this dashboard? How about Kiali?
I am using istio for service mesh. Is any dashboard available that works best with istio for distributed tracing?
To cover the scenario you mention here, firstly make sure to have a centralized logging. I have used Elk and found it to be good covering logs from multiple services and it comes with a good dashboard view to debug the logs.
You can have different source types for logs across the micro services to differentiate while debugging. use something like a request-id which flows across all the 10 different services which the request hits in the path. This would make the identification easier, there are other ways too to handle it but for someone new to the flow could debug faster
You can use filebeat to push the logs with different log levels to elk from the log files generated at every ms.
Kibana dashboard is good for monitoring and comes with multiple search options as basic as http status code 500 which would directly give all internal server errors.
To improve further monitoring use alerts, graphs to get triggers.
I am using AWS Managed Prometheus service and setup a Prometheus server on my EKS cluster to collect and write metrics on my AMP workspace, using the helm chart, as per tutorial from AWS. All works fine, I am also connecting to a cluster run Grafana and I can see the metrics no problem.
However, my use case is to query metrics from my web application which runs on the cluster and to display the said metrics using my own diagram widgets. In other words, I don't want to use Grafana.
So I was thinking to use the AWS SDK (Java in my case, https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/amp/model/package-summary.html), which works fine (I can list my workspaces etc...), except it doesn't have any method for querying metrics!?
The documentation indeed mentions that this is not out of the box (!) and basically redirects to Grafana...
This seems fairly odd to me as the basic use case would be to run some queries no? Am I missing something here? do I need to create my own HTTP requests for this?
FYI, I ended up doing the query manually, creating an SdkHttpFullRequest and using an AWS4Signer to sign it. Works OK but I wonder why it couldn't be included in the SDK directly... The only gotcha was to make sure to specify the "aps" for the signing name in the Aws4SignerParams creation.
We have couple of application running on AWS. Currently we are redirecting all our logs to single bucket. However for ease of access to users, I am thinking to install ELK Stack on EC2 instance.
Would want to check if there is alternate way available where I don't have to maintain this stack.
Scaling won't be an issue, as this is only for logs generated through application running on AWS, so not ingestion or processing is required. mostly log4j logs.
You can go for either the managed Elasticsearch available in AWS or setup your own in an EC2 instance
It usually comes down to the price involved and the amount of time you have in hand in setting up and maintaining your own setup
With your own setup, you can do a lot more configurations than that provided by the managed service and also helps in reducing the cost
You can find more info on this blog
I'm trying to figure out what AWS services I need for the mobile application I'm working on with my startup. The application we're working on should go into the app-/play-store later this year, so we need a "best-practice" solution for our case. It must be high scaleable so if there are thousands of requests to the server it should remain stable and fast. Also we maybe want to deploy a website on it.
Actually we are using Uberspace (link) servers with an Node.js application and MongoDB running on it. Everything works fine, but for the release version we want to go with AWS. What we need is something we can run Node.js / MongoDB (or something similar to MongoDB) on and something to store images like profile pictures that can be requested by the user.
I have already read some informations about AWS on their website but that didn't help a lot. There are so many services and we don't know which of these fit our needs perfectly.
A friend told me to just use AWS EC2 for the Node.js server + MongoDB and S3 to store images, but on some websites I have read that it is better to use this architecture:
We would be glad if there is someone who can share his/her knowledge with us!
To run code: you can use lambda, but be careful: the benefit you
don't have to worry about server, the downside is lambda sometimes
unreasonably slow. If you need it really fast then you need it on EC2
with auto-scaling. If you tune it up properly it works like a charm.
To store data: DynamoDB if you want it really fast (single digits
milliseconds regardless of load and DB size) and according to best
practices. It REQUIRES proper schema or will cost you a fortune,
otherwise use MongoDB on EC2.
If you need RDBMS then RDS (benefits:
scalability, availability, no headache with maintenance)
Cache: they have both Redis and memcached.
S3: to store static assets.
I do not suggest CloudFront, there are another CDN on market with better
price/possibilities.
API gateway: yes, if you have an API.
Depending on your app, you may need SQS.
Cognito is a good service if you want to authenticate your users at using google/fb/etc.
CloudWatch: if you're metric-addict then it's not for you, perhaps standalone EC2
will be better. But, for most people CloudWatch is abcolutely OK.
Create all necessary alarms (CPU overload etc).
You should use roles
to allow access to your S3/DB from lambda/AWS.
You should not use the root account but create a separate user instead.
Create billing alarm: you'll know if you're going to break budget.
Create lambda functions to backup your EBS volumes (and whatever else you may need to backup). There's no problem if backup starts a second later, so
Lambda is ok here.
Run Trusted Adviser now and then.
it'd be better for you to set it up using CloudFormation stack: you'll be able to deploy the same infrastructure with ease in another region if/when needed, also it's relatively easier to manage Infrastructure-as-a-code than when it built manually.
If you want a very high scalable application, you may be need to use a serverless architecture with AWS lambda.
There is a framework called serverless that helps you to manage and organize all your lambda function and put them behind AWS Gateway.
For the storage you can use AWS EC2 and install MongoDB or you can go with AWS DynamODB as your NoSql storage.
If you want a frontend, both web and mobile, you may be want to visit the react native approach.
I hope I've been helpful.
I tried searching for this topic on Google and after many failed attempts I decided to post this as questions here.
What I want to achieve: Monitoring my aws limits using Nagios.
As I have understood aws cli can be used to get the limits of only few aws services, for more in depth cost management and service limit management one has to opt for trusted advisor. Unfortunately it's quite expensive.
So I was wondering if there's a much simpler way with Nagios in which I could get notified if any of the aws services for my account is hitting a limit?
What kind of service limit notification strategy is used by organizations(That can't afford to buy a subscription of trusted advisor) that use Aws?
You're right: only few services can show their limit (and current usage) through CLI or API. I don't like it either :) We have three options here:
Create a parser that grabs information from AWS Console (there is an example code here: https://forrestbrazeal.com/2015/07/20/adventures-in-aws-automating-service-limit-checks/).
Buy Trusted Advisor (btw, you can get a Trusted Advisor report with API call).
Try using awslimitchecker. Cause someone already tried to solve this problem.
https://awslimitchecker.readthedocs.io/en/latest/