How to calculate normalized %CPU per process using Prometheus wmi_exporter using wmi_process_cpu_time_total and wmi_cs_logical_processors - wmi

Using the following promql I'm succeeded to get per process %CPU:
sum by (process, process_id) (rate(wmi_process_cpu_time_total{instance="my_instance", process="my_process"}[5m]))*100/(amount_of_cores)
This returns correct vector if amount_of_cores is hardcoded scalar (in my case - 32). Unfortunately I cannot use hardcoded values here because I have many instances with different amount of cores...
In the other hand, the following query returns correct amount of cores on the instance:
wmi_cs_logical_processors{instance="my_instance"}
Unfortunately when I use this query as a divider - it returns nothing. As I understand the issue is that first query has several labels, but second - only one "instance", so direct dividing returns nothing.
What can I do here?

During writing the question I found the answer. So, in case that someone interesting...
To divide one vector to another with different labels you should use ignoring(labels_not_existing_in_divider), group_left and some aggregator on divider (I used sum).
((sum by (process, process_id) (rate(wmi_process_cpu_time_total{instance=~"$hostname.*"}[5m]))*100))
/
ignoring(process, process_id) group_left sum (wmi_cs_logical_processors{instance=~"$hostname.*"})

Related

Getting huge number after using simple subtraction in DAX measure

I'm kind of new to DAX and I'm basically learning as I'm using it in my work. We are building reports in PowerBI and we have data model that gets data from Oracle database. So I'm using DAX to create measures in this data model.
I need to substract 2 numbers from each other. So I created simple measure which looked like this:
[MEASURE1] - [MEASURE2]
Whether it works or it doesn't depends on my Period filter which uses another table. I don't know how could period be related to any of this. So when I change filter to some values, I get normal number. However, when I switch it to different values, I get numbers like 2,27483058473905E-13.
Weird thing is that if I check those two measures that I'm subtracting, they have exactly the same numbers, so the difference should be 0.
I know this is not the best explanation, but it is impossible to describe entire data model here. So I'm just looking for some ideas what could possibly be causing this and what should I check.
I have literally no idea what could be causing this.
Floating point precision.
Either use fixed decimal data types, specify the format string of the measure, or wrap your measure in ROUND, e.g.:
Diff =
ROUND (
[Measure 1] - [Measure 2] ,
2
)
2,27483058473905E-13 is not a huge number, but as close as a decimal calculator can get to zero.

AWS Quicksight calculated fields gives incorrect result for simple division

I have a dataset with fields targeted and opens and I need to add calculated field opens per targeted which essentially means doing simple devision of those 2 values.
My calculated field is as follows
{opens}/{targeted}
but then displaying simple table with values they are completely incorrect
If I try any other operator like + * etc calculations are correct.
I'm completely out of ideas on how to debug this. I've simplified the dataset to just columns of targeted and opens, can't get any simpler.
Had the same problem, I fixed it by wrapping the columns with the sum() function. Like this:
sum({opens})/sum({targeted})
I think you need to make AWS understand that you are working with float numbers.
1.0*{opens}/{targeted}
if still not working try also
(1.0*{opens})/({targeted}*1.0)
it should give you the desired output (not tested, let me know if it doesnt work)

Correct values for SsaSpikeEstimator's pvalueHistoryLength

In the creation of a SsaSpikeEstimator instance by the DetectSpikeBySsa method, there is a parameter called pvalueHistoryLength - could anybody please help me understand, for any given time series with X points, which is the optimal value for this parameter?
I got similar issue, when I try to read the paper, https://arxiv.org/pdf/1206.6910.pdf, I notice one paragraph
Also, simulations and theory (Golyandina, 2010) show that it is
better to choose window length L smaller than half of the time series length
N. One of the recommended values is N/3.
Maybe that's why in the ML.Net Power Anomaly example, the value is chosen to be 30 for the 90 points dataset.

Countif and ArrayFormula with multiple levels

I have a formula. It works - but feels like it could be made much simpler.
I have many departments across several columns. Each row has an item that we're tracking and each column has a status text that changes as we do the work.
'queue' - it's in line waiting to be done and weighs down the average
'active' - in process and provides a half value across the average
'done', 'ok'd', 'rcvd' - finished and contributes to the final average
'none' - denotes a department that's inactive on this job and should not count in the final average.
The formula is:
=iferror(((ArrayFormula(sum(countif(B3:O3,{"done","ok'd","rcvd"}))))+(countif(B3:O3,"active")/2))/(counta(B3:O3)-(countif(B3:O3,"none"))),)
The formula works but I'm looking to see if there's an easier way to approach it. Would a query or array modification work better in this scenario?
What if I wanted to add other text strings based on syntax for my current application?
Here's a link to a sample sheet with it in context.
https://docs.google.com/spreadsheets/d/1zPFAcSxM7tYjZmlATYde7qKsDoeH6AW_xjFooOZFOf4/edit#gid=0
EDIT:
As a followup question - how do I get the same thing to work across the columns?
I did some reverse engineering to the solution and can see the formula working across the top of my sheet - but it's giving me an error:
"MMULT has incompatible matrix sizes. Number of columns in first matrix (13) must equal number of rows in second matrix (1)."
Here's the formula I've added (it's also in the linked sheet).
=ARRAYFORMULA(IF(LEN(B4:N4), MMULT(IFERROR(( N(REGEXMATCH(B4:N9, "ok'd|done|ready|rcvd"))+ N(REGEXMATCH(B4:N9, "active"))/2)/MMULT(N(REGEXMATCH(B4:N9, "[^none]")),TRANSPOSE(ROW(B4:B9)^0)), 0), TRANSPOSE(ROW(B4:B9)^0)),))
As a followup question - how do I get the same thing to work across the columns?
=ARRAYFORMULA(TRANSPOSE(IF(LEN(TRANSPOSE(B4:N4)), MMULT(IFERROR((
N(REGEXMATCH(TRANSPOSE(B4:N16), "ok'd|done|ready|rcvd"))+
N(REGEXMATCH(TRANSPOSE(B4:N16), "active"))/2)/MMULT(
N(REGEXMATCH(TRANSPOSE(B4:N16), "[^none]")),
(ROW(B4:B16)^0)), 0),
(ROW(B4:B16)^0)), )))
=ARRAYFORMULA(IF(LEN(B3:B9), MMULT(IFERROR((
N(REGEXMATCH(B3:N9, "ok'd|done|ready|rcvd"))+
N(REGEXMATCH(B3:N9, "active"))/2)/MMULT(
N(REGEXMATCH(B3:N9, "[^none]")),
TRANSPOSE(COLUMN(B3:N3)^0)), 0),
TRANSPOSE(COLUMN(B3:N3)^0)), ))

C++, determine the part that have the highest zero crosses

I’m not specialist in signal processing. I’m doing simple processing on 1D signal using c++. I want really to know how I can determine the part that have the highest zero cross rate (highest frequency!). Is there a simple way or method to tell the beginning and the end of this part.
This image illustrate the form of my signal, and this image is what I need to do (two indexes of beginning and end)
Edited:
Actually I have no prior idea about the width of the beginning and the end, it's so variable.
I could calculate the number of zero crossing, but I have no idea how to define it's range
double calculateZC(vector<double> signals){
int ZC_counter=0;
int size=signals.size();
for (int i=0; i<size-1; i++){
if((signals[i]>=0 && signals[i+1]<0) || (signals[i]<0 && signals[i+1]>=0)){
ZC_counter++;
}
}
return ZC_counter;
}
Here is a fairly simple strategy which might give you some point to start. The outline of the algorithm is as follows
Input: Vector of your data points {y0,y1,...}
Parameters:
Window size sigma.
A threshold 0<p<1 defining when to start looking for a region.
Output: The start- and endpoint {t0,t1} of the region with the most zero-crossings
I won't give any C++ code, but the method should be easy to implement. As example let us use the following function
What we desire is the region between about 480 and 600 where the zero density higher than in the front. First step in the algorithm is to calculate the positions of zeros. You can do this by what you already have but instead of counting, you store the values for i where you met a zero.
This will give you a list of zero positions
From this list (you can do this directly in the above for-loop!) you create a list having the same size as your input data which looks like {0,0,0,...,1,0,..,1,0,..}. Every zero-crossing position in your input data is marked with a 1.
The next step is to smooth this list with a smoothing filter of size sigma. Here, you can use what you like; in the simplest case a moving average or a Gaussian filter. The higher you choose sigma the bigger becomes your look around window which measures how many zero-crossings are around a certain point. Let me give the output of this filter together with the original zero positions. Note that I used a Gaussian filter of size 10 here
In a next step, you go through the filtered data find the maximum value. In this case it is about 0.15. Now you choose your second parameter which is some percentage of this maximum. Lets say p=0.6.
The final step is to go through the filtered data and when the value is greater than p you start to remember a new region. As soon as the value drops below p, you end this region and remember start and endpoint. Once you are finished walking through the data, you are left with a list of regions, each defined by a start and an endpoint. Now you choose the region with the biggest extend and you are done.
(Optionally, you could add the filter size to each end of the final region)
For the above example, I get 11 regions as follows
{{164,173},{196,205},{220,230},{241,252},{259,271},{278,290},
{297,309},{318,327},{341,350},{458,468},{476,590}}
where the one with the biggest extend is the last one {476,590}. The final result looks (with 1/2 filter region padding)
Conclusion
Please don't be discouraged by the length of my answer. I tried to explain everything in detail. The implementation is really just some loops:
one loop to create the zero-crossings list {0,0,..,1,0,...}
one nested loop for the moving average filter (or you use some library Gaussian filter). Here you can at the same time extract the maximum value
one loop to extract all regions
one loop to extract the largest region if you haven't already extracted it in the above step