I am new to Grafana. I am setting it up to view data from Cloudwatch for a Custom Metrics. Custom Metrics Namespace Name is JVMStats, Metric is JVMHeapUsed, Dimension is instance Id. If I configure these data, I am not able to get the graph. Can you please advice me on how to get the data?
Regards
Karthik
I want to do the same.
As far as I can tell, it's not possible out of the box with the latest Grafana (2.6 at time of writing). See this issue.
This pull request implements it. It's currently tagged as 3.0-beta1. So I expect we'll both be able to do what we want come version 3.0.
EDIT: inserting proof of 3.0-beta-1 working
I installed 3.0-beta-1, and was able to use Custom Metrics, as evidenced by this image:
I managed to add my custom metrics now, only issue I had was that I listed my custom metrics in the "Data Source" configurations with commmas and spaces:
Custom1, Custom2
but it must be only commas:
Custom1,Custom2
And it works for me. The preview in the text box shows this, but I missed it.
Another option is to configure AWS CloudWatch job to collect data into Axibase Time Series Database where CustomMetrics namespace is enabled out-of-the-box:
Disclosure: I work for Axibase.
Related
I was trying to use EventArc to monitor Firestore changes, which will trigger a Cloud Run service.
It works by listening to Any Resource or specific resource name.
Resource name
projects/PROJECT_Id/databases/(default)
It works, but it will listen to all changes in firestore. However, I want to filter the event to a specific collection. I have tried a few combinations to the pattern, and none of them works. E.g.
projects/PROJECT_Id/databases/(default)/users/*
projects/PROJECT_Id/databases/(default)/users/{user}
Any ideas? Thanks :)
I have seen in the image you shared, under the “Resource” section you selected “specific resource” instead can you try to change it to a “path pattern” and you will be able to write a specific path that the trigger object needs to respect.
You can refer to this documentation for Applying a path pattern when filtering in Eventarc.
You can also check out the AuditLog step of Trigger Cloud Run with events from Eventarc codelab for an example on how to use path patterns.
I deleted all the items in the DataTemplate table but when I query them again with the searchDataTemplates endpoint on the app or in AppSync it returns the old data, but when I use the listDataTemplates it returns nothing which is correct. Needed to repopulate the data in the table.
data template table
search endpoint
list endpoint
when I updated items individually it worked just fine but when i deleted all the items from the console (around 700 items) the search endpoint stopped working. Just the search
UPDATE:
I repopulated the data hoping it'd reset but now the listDataTemplates shows the new data and the search still shows the old data, is there some cache that needs to be reset?
SECOND UPDATE:
I removed the table and the appsync functions are gone however when i recreated the table (with no data) the testing out the function still returns the old data. I'm guessing the opensearch stuff hasn't been updated?
If you are using AppSync with Amplify CLI, #searchable will automatically create the followings:
An OpenSearch Domain
A Lambda Function that will be attached to the DynamoDB Streams and push the changes (create/update/delete) over to your OpenSearch Domain.
And the problem that you're facing is most likely due to the Lambda Function created failed to push the changes from DynamoDB Streams to OpenSearch. A quick suggestion is to check on the created Lambda Function first.
Reference: #searchable
This issue can only happen if caching is enabled in your application.
I am not sure what's the infrastructure you are using, so i would go ahead with some educated guess. Please feel free to correct me if i overstepped.
From your description of question, you have an AppSync as API layer and DynamoDb as primary database.
If these are the only two resources you have, please check the AppSync cache configuration.
Open AppSync console
from left panel select APIs -> your api -> caching
Validate Caching behavior is set to None
In case if you have AWS OpenSearch enabled for search query (i could be wrong, however picking up from previous comment). Then validate the cluster configuration.
Open AWS Open Search Service console
From left panel select Domains and click on the openserch domain that you are using
scroll to the bottom right and look for Advanced cluster settings and ensure the attribute Fielddata cache allocation is set to 0
If Fielddata cache allocation is not 0, update the cluster configuration and modify the advanced cluster setting to set the Fielddata cache allocation field to 0.
Wait for a few minutes (I would suggest 5 minutes) and then retry your use-case.
I hope this would help resolve your issue.
I am new to Hygieia. I have setup Hygieia (https://github.com/capitalone/Hygieia) as per the documentation and I could start my application and see the dashboard successfully. In Hygieia-Dashboard, I could see values coming for all widget items (say utilization, instances stoped, and so on.) except the IP Utilization widget. When i see the mongodb that i configured with this (after I ran AWS module) i could see data for most of the documents except the cloud_subnetwork and cloud_virtual_network. Is this causing the issue?
Can anyone please assist me for populating IPUtilization widget ?
if I am still missing with anything? Anyfurther configuration to be made for this?
I found a solution for this issue., i.e., we need to fetch values for subnets and virtual_networks by using the AmazonEC2Client's instances by calling its describeSubnets() and describeVpcs() respectively and save these to database. After this, we could see values in the IPUtilization widgets. Hope this helps!
Simple question, but I suspect it doesn't have a simple or easy answer. Still, worth asking.
We're creating an implementation for push notifications using AWS with our Web Server running on EC2, sending messages to a queue on SQS, which is dealt with using Lambda, which is sent finally to SNS to be delivered to the iOS/Android apps.
The question I have is this: is there a way to query SNS endpoints based on the custom user data that you can provide on creation? The only way I see to do this so far is to list all the endpoints in a given platform application, and then search through that list for the user data I'm looking for... however, a more direct approach would be far better.
Why I want to do this is simple: if I could attach a User Identifier to these Device Endpoints, and query based on that, I could avoid completely having to save the ARN to our DynamoDB database. It would save a lot of implementation time and complexity.
Let me know what you guys think, even if what you think is that this idea is impractical and stupid, or if searching through all of them is the best way to go about this!
Cheers!
There isn't the ability to have a "where" clause in ListTopics. I see two possibilities:
Create a new SNS topic per user that has some identifiable id in it. So, for example, the ARN would be something like "arn:aws:sns:us-east-1:123456789:know-prefix-user-id". The obvious downside is that you have the potential for a boat load of SNS topics.
Use a service designed for this type of usage like PubNub. Disclaimer - I don't work for PubNub or own stock but have successfully used it in multiple projects. You'll be able to target one or many users this way.
According the the [AWS documentation][1] if you try and create a new Platform Endpoint with the same User Data you should get a response with an exception including the ARN associated with the existing PlatformEndpoint.
It's definitely not ideal, but it would be a round about way of querying the User Data Endpoint attributes via exception.
//Query CustomUserData by exception
CreatePlatformEndpointRequest cpeReq = new CreatePlatformEndpointRequest().withPlatformApplicationArn(applicationArn).withToken("dummyToken").withCustomUserData("username");
CreatePlatformEndpointResult cpeRes = client.createPlatformEndpoint(cpeReq);
You should get an exception with the ARN if an endpoint with the same withCustomUserData exists.
Then you just use that ARN and away you go.
I have a Lambda function and its logs in Cloudwatch (Log group and Log Stream). Is it possible to filter (in Cloudwatch Management Console) all logs that contain "error"? For example logs containing "Process exited before completing request".
In Log Groups there is a button "Search Events". You must click on it first.
Then it "changes" to "Filter Streams":
Now you should just type your filter and select the beginning date-time.
So this is kind of a side issue, but it was relevant for us. (I posted this to another answer on StackOverflow but thought it would be relevant to this conversation too)
We've noticed that tailing and searching logs gets really slow after a log group has a lot of Log Streams in it, like when an AWS Lambda Function has had a lot of invocations. This is because "tail" type utilities and searching need to connect to each log stream to run. Log Events get expired and deleted due to the policy you set on the Log Group itself, but the Log Streams never get cleaned up. I made a few little utility scripts to help with that:
https://github.com/four43/aws-cloudwatch-log-clean
Hopefully that save you some agony over waiting for those logs to get searched.
You can also use CloudWatch Insights (https://aws.amazon.com/about-aws/whats-new/2018/11/announcing-amazon-cloudwatch-logs-insights-fast-interactive-log-analytics/) which is an AWS extension to CloudWatch logs that gives a pretty powerful query and analytics tool. However it can be slow. Some of my queries take up to a minute. Okay, if you really need that data.
You could also use a tool I created called SenseLogs. It downloads CloudWatch data to your browser where you can do queries like you ask about. You can use either full text and search for "error" or if your log data is structured (JSON), you can use a Javascript like expression language to filter by field, eg:
error == 'critical'
Posting an update as CloudWatch has changed since 2016:
In the Log Groups there is a Search all button for a full-text search
Then just type your search: