I downloaded the "wmi_exporter-0.7.0-386.exe" from the link wmi exporter and ran the '.exe' file using command prompt.
Also i have followed the guidance link prometheus wmi guidance to run the command in the command prompt ,it executed as expected and i am able to check the metrics in "http://localhost:9182/metrics".
My problem here is , if i run the command "./wmi_exporter-0.7.0-386.exe" alone using the command prompt , i am able to check the memory utilization of my system.For example it is showing the details of the memory utilization like "wmi_cs_physical_memory_bytes 3.4673408e+10" and all the memory details as well in browser metrics.
But if i run the command ".\wmi_exporter.exe --collectors.enabled "process" --collector.process.processes-where "Name LIKE 'firefox%'"" , in metrics it is filtering only the "firefox" browser related things , i am not able to check the system memory utilization details like "wmi_cs_physical_memory_bytes 3.4673408e+10".
Kindly suggest me how to get the system utilization details as well as firefox related details in metrics.
You just need to add collector query, for all the services which you want to monitor.Like,
msiexec /i C:\Users\Administrator\Downloads\wmi_exporter-0.4.3-amd64 ENABLED_COLLECTORS="cpu,cs,logical_disk,os,net,system,process,service,memory" --% EXTRA_FLAGS="--collector.service.services-where ""Name LIKE 'sisense%'"""
Querying is very simple , just need to add the collector in the query itself,
.\wmi_exporter.exe --collectors.enabled "process,cs,logical_disk" --collector.process.processes-where "Name LIKE 'firefox%'"
where 'cs' collects local system details , 'logical_disk' collects local drives memory details (such as 'C:\','D:\')
and the "Name LIKE 'firefox'" part collects the fixefox details
Related
I'd like to know how to check if there is any EC2 Instance has not been rebooted for over 90 days.
I know I can check it in CloudTrail event history if it is less than 90 days, and I can also check it using linux command if I am using linux AMI. But is there any way I can do it inside AWS console?
Thank you all in advance.
The best way to have 100% sure is looking at uptime inside your OS.
You can use AWS Systems Manager Run Command to get the result of uptime from all your instances remotely. The link below explain how to do it.
https://aws.amazon.com/getting-started/hands-on/remotely-run-commands-ec2-instance-systems-manager/
For doing inside the console you can check under Monitor and Troubleshoot -> Get System Logs
You can use get-console-output aws cli command and get the system logs which are actually just the output of dmesg command.
you'll get something like this
[153.460957] cloud-init[2765]: RUNNING HANDLER [_system : restart network] ************************************
Then [153.460957] this is dmesg specific time stamp you can convert that to human readable by following How do I convert dmesg timestamp to custom date format?
Open the CloudTrail console, and then choose Event history.
In Filter, select the drop-down menu, and then choose Event name.
In the Enter event name text box, enter (StopInstances), and then choose the event name.
In Time range, enter the desired time range (up to 9o days), and then choose Apply.
I need your help , actually i need to create a dasboard in GCP to show TTR Time , TTR response via fetching logs from GCP logging that i am writting using script but unable to achieve it.
Below is command i am using:
gcloud logging write logging/user/TTR4 '{"Gremblin_correlation_exec_id": "correlation_id","SenerioName": "Senerio1","ServiceName": "Service1","SubsystemName": "subsystem1","TTRTime": 500,"EndTimestamp": "2020-11-30 06:06:56+00:00","Node_ipfirst": "10.128.0.55:80","node_ipsecound": "10.128.0.6:80","starttimestamp": "2020-11-30 05:58:08+00:00" }' --payload-type=json
i am getting jason data but not able to show it on dasboard like TTRTime above 500 using filter based upon service name and subsystemName.
The question: Imagine I run a very simple Python script on EMR - assert 1 == 2. This script will fail with an AssertionError. The log the contains the traceback containing that AssertionError will be placed (if logs are enabled) in an S3 bucket that I specified on setup, and then I can read the log containing the AssertionError when those logs get dropped into S3. However, where do those logs exist before they get dropped into S3?
I presume they would exist on the EC2 instance that the particular script ran on. Let's say I'm already connected to that EC2 instance and the EMR step that the script ran on had the ID s-EXAMPLE. If I do:
[n1c9#mycomputer cwd]# gzip -d /mnt/var/log/hadoop/steps/s-EXAMPLE/stderr.gz
[n1c9#mycomputer cwd]# cat /mnt/var/log/hadoop/steps/s-EXAMPLE/stderr
Then I'll get an output with the typical 20/01/22 17:32:50 INFO Client: Application report for application_1 (state: ACCEPTED) that you can see in the stderr log file you can access on EMR:
So my question is: Where is the log (stdout) to see the actual AssertionError that was raised? It gets placed in my S3 bucket indicated for logging about 5-7 minutes after the script fails/completes, so where does it exist in EC2 before that? I ask because getting to these error logs before they are placed on S3 would save me a lot of time - basically 5 minutes each time I write a script that fails, which is more often than I'd like to admit!
What I've tried so far: I've tried checking the stdout on the EC2 machine in the paths in the code sample above, but the stdout file is always empty:
What I'm struggling to understand is how that stdout file can be empty if there's an AssertionError traceback available on S3 minutes later (am I misunderstanding how this process works?). I also tried looking in some of the temp folders that PySpark builds, but had no luck with those either. Additionally, I've printed the outputs of the consoles for the EC2 instances running on EMR, both core and master, but none of them seem to have the relevant information I'm after.
I also looked through some of the EMR methods for boto3 and tried the describe_step method documented here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr.html#EMR.Client.describe_step - which, for failed steps, have a FailureDetails json dict response. Unfortunately, this only includes a LogFile key which links to the stderr.gz file on S3 (even in that file doesn't exist yet) and a Message key which contain a generic Exception in thread.. message, not the stdout. Am I misunderstanding something about the existence of those logs?
Please feel free to let me know if you need any more information!
It is quite normal that with log collecting agents, the actual logs files doesn't actually grow, but they just intercept stdout to do what they need.
Most probably when you configure to use S3 for the logs, the agent is configured to either read and delete your actual log file, or maybe create a symlink of the log file to somewhere else, so that file is actually never writen when any process open it for write.
maybe try checking if there is any symlink there
find -L / -samefile /mnt/var/log/hadoop/steps/s-EXAMPLE/stderr
but it can be something different from a symlink to achieve the same logic, and I ddint find anything in AWS docs, so most probably is not intended that you will have both S3 and files at the same time and maybe you wont find it
If you want to be able to check your logs more frequently, you may want to think about installing a third party logs collector (logstash, beats, rsyslog,fluentd) and ship logs to SolarWinds Loggly, logz.io, or set up a ELK (Elastic search, logstash, kibana)
You can check this article from Loggly, or create a free acount in logz.io and check the lots of free shippers that they support
I am looking for a way to see how many concurrent invocations there are active at any point in time, e.g. in a minute range. I am looking for this as I received the error:
Forbidden: 403 Exceeded rate limits: too many concurrent queries for
this project_and_region. For more information, see
https://cloud.google.com/bigquery/
The quotas are listed here: https://cloud.google.com/functions/quotas
I am fine with having quotas, but I would like to see this number in a chart. Where can I find this?
Currently there is no way of seeing that information directly. There is a workaround though. You can do as follows:
Go to Google Cloud Console > Stackdriver Logging
At the text box that says "Filter by label or text search", click on the small arrow at the end of the text box.
Choose "Convert to advanced filter"
Type that query inside:
resource.type="cloud_function"
resource.labels.function_name="[GOOGLE_CLOUD_FUNCTION_NAME]"
"Function execution started"
At "Last hour" drop down menu, choose "Custom"
Fix the start and end time
This will list all the times that the Cloud Function was executed in the time range. If it was executed multiple times, instead of counting one by one you can use the following Python script:
Open Google Cloud Shell
Install Google Cloud Logging Library $ pip install google-cloud-logging
Create a main.py file using my GitHub code example. (I have tested it and it is working as expected)
Change the date_a_str and set it as start date.
Change the date_b_str and set it as end date.
In function_name = "[CLOUD_FUNCTION_NAME]" change [CLOUD_FUNCTION_NAME] to the name of your Cloud Function.
Execute the Python code $ python main.py
You should see a response as follows:
Found entries: [XX]
Waiting up to 5 seconds.
Sent all pending logs.
Tidal - How to get output of web services job on to hard disk? I a trying to output the results of web services job on to hard disk. Please let me know who I can achieve this using Tidal.
Create a second job that depends on the web service job. In this second job, use the "output" command from the Tidal command line interface to save the output of the previous job to a file. It would look something like this:
command: /path/to/tidal/sacmd
parameters:
-i <JobID..1234> -o /path/to/output/file.out
For the job id parameter, you select the predecessor web service job and the JobID reference to the predecessor gets filled in as above.
sacmd may be called tescmd depending on the OS and the version of Tidal you are using.