I am trying to help a public school here, but I have very limited knowledge in Power Bi so I hope your guys could enlight me on this case:
we have a very simple report with a table and a kpi
Kpi counts all students
table shows studants grades
Student Math Portuguese History Science
StD A 6 6 7 8
StD B 6 7 6 7
StD C 8 9 7 8
StD D 6 6 6 6
StD E 6 7 8 8
StD F 8 6 7 7
the rule that must be applied to the kpi (count(Students)) and to the table is to show studenst only if:
at least 2 subjects are equal or under 6
portuguese is equal or under 6
math is under 6
all the rest should not be showed in the table or counted in the KPI. In this case I would see/count only students A, B, D,E & F
any help would be very appreciated
To tackle your task try the following:
Create a calculated column in your table with the following DAX code:
isValid =
VAR cond_2_subjects = (('Table'[Math] <= 6 ) + ('Table'[Portuguese] <= 6) + ('Table'[History] <= 6) + ('Table'[Science] <= 6)) >= 2
VAR cond_portuguese = 'Table'[Portuguese] <= 6
VAR cond_math = 'Table'[Math] < 6
RETURN
-- This will check if any of the given conditions is true
IF(
cond_2_subjects || cond_portuguese || cond_math,
TRUE(),
FALSE()
)
The table should then look like this:
The KPI (measure) can then be written like so:
# Students =
CALCULATE(
COUNT('Table'[Student]),
-- only count Students where conditions are true (calculated column isValid = True)
'Table'[isValid] = TRUE()
)
The final result should then look like this:
The table on the left has specified 'Table'[isValid] = TRUE() as filter on visual
Related
I have a csv with one of the columns that contains periods:
timespan (string): PnYnMnD, where P is a literal value that starts the expression, nY is the number of years followed by a literal Y, nM is the number of months followed by a literal M, nD is the number of days followed by a literal D, where any of these numbers and corresponding designators may be absent if they are equal to 0, and a minus sign may appear before the P to specify a negative duration.
I want to return a data frame that contains all the data in the csv with parsed timespan column.
So far I have a code that parses periods:
import re
timespan_regex = re.compile(r'P(?:(\d+)Y)?(?:(\d+)M)?(?:(\d+)D)?')
def parse_timespan(timespan):
# check if the input is a valid timespan
if not timespan or 'P' not in timespan:
return None
# check if timespan is negative and skip initial 'P' literal
curr_idx = 0
is_negative = timespan.startswith('-')
if is_negative:
curr_idx = 1
# extract years, months and days with the regex
match = timespan_regex.match(timespan[curr_idx:])
years = int(match.group(1) or 0)
months = int(match.group(2) or 0)
days = int(match.group(3) or 0)
timespan_days = years * 365 + months * 30 + days
return timespan_days if not is_negative else -timespan_days
print(parse_timespan(''))
print(parse_timespan('P2Y11M20D'))
print(parse_timespan('-P2Y11M20D'))
print(parse_timespan('P2Y'))
print(parse_timespan('P0Y'))
print(parse_timespan('P2Y4M'))
print(parse_timespan('P16D'))
Output:
None
1080
-1080
730
0
850
16
How do I apply this code to the whole csv column while running the function processing csv?
def do_process_citation_data(f_path):
global my_ocan
my_ocan = pd.read_csv(f_path, names=['oci', 'citing', 'cited', 'creation', 'timespan', 'journal_sc', 'author_sc'],
parse_dates=['creation', 'timespan'])
my_ocan = my_ocan.iloc[1:] # to remove the first row
my_ocan['creation'] = pd.to_datetime(my_ocan['creation'], format="%Y-%m-%d", yearfirst=True)
my_ocan['timespan'] = parse_timespan(my_ocan['timespan']) #I tried like this, but sure it is not working :)
return my_ocan
Thank you and have a lovely day :)
Like with Python's builtin map, Pandas also has that method. You can check its documentation here. Since you already have your function ready which takes a single parameter and returns a value, you just need this:
my_ocan['timespan'] = my_ocan['timespan'].map(parse_timespan) #This will take each value in the column "timespan", pass it to your function 'parse_timespan', and update the specific row with the returned value
And here is a generic demo:
import pandas as pd
def demo_func(x):
#Takes an int or string, prefixes with 'A' and returns a string.
return "A" + str(x)
df = pd.DataFrame({"Column_1": [1, 2, 3, 4], "Column_2": [10, 9, 8, 7]})
print(df)
df['Column_1'] = df['Column_1'].map(demo_func)
print("After mapping:\n{}".format(df))
Output:
Column_1 Column_2
0 1 10
1 2 9
2 3 8
3 4 7
After mapping:
Column_1 Column_2
0 A1 10
1 A2 9
2 A3 8
3 A4 7
I have got below table and want to add calculated column Rank (oldest top-3) that ranks only when Status is "O". Note that **Rank (oldest top-3)**is the desired result.
Status Days open Rank (oldest top-3)
C 1
O 1 4
O 2 3
C 3
C 4
C 5
O 6 2
O 7 1
C 8
C 9
I have got below code but they do not work for me.
Rank = IF(order[Status] = "C", BLANK(),
RANKX(FILTER(order, order[Status] = "O"),
order[Days open], , 1, Dense))
I get top 3 and not the botom one. Also, with filter it filter out any other data. I tried to replace FILTER with ALLSELECTED but it did not work.
Input
I have created a table named order with the following data:
Status Days open
C 1
O 1
O 2
C 3
C 4
C 5
O 6
O 7
C 8
C 9
Code
Then I have added a calculated column with the following DAX:
Rank =
IF('order'[Status] = "C",
BLANK(),
RANKX(
FILTER('order', 'order'[Status] = "O"),
'order'[Days open],
,
0,
Dense
)
)
The only difference compared to your DAX (apart from formatting) is that the second to last option of the RANKX function is 0 instead of 1.
The documentation of RANKX indicates that 0 ranks the series in descending order.
Output
FILTER('order', 'order'[Status] = "O"),change to FILTER(all('order'), 'order'[Status] = "O"),if not, your resutls may the all the same in one table.
I've a dataframe which looks like this:
wave mean median mad
0 4050.32 -0.016182 -0.011940 0.008885
1 4208.98 0.023707 0.007189 0.032585
2 4508.28 3.662293 0.001414 7.193139
3 4531.62 -15.459313 -0.001523 30.408377
4 4551.65 0.009028 0.007581 0.005247
5 4554.46 0.001861 0.010692 0.027969
6 6828.60 -10.604568 -0.000590 21.084799
7 6839.84 -0.003466 -0.001870 0.010169
8 6842.04 -32.751551 -0.002514 65.118329
9 6842.69 18.293519 -0.002158 36.385884
10 6843.66 0.006386 -0.002468 0.034995
11 6855.72 0.020803 0.000886 0.040529
As it's clearly evident in the above table that some of the values in the column mad and median are very big(outliers). So i want to remove the rows which have these very big values.
For example in row3 the value of mad is 30.408377 which very big so i want to drop this row. I know that i can use one line
to remove these values from the columns but it doesn't removes the complete row
df[np.abs(df.mad-df.mad.mean()) <= (3*df.mad.std())]
But i want to remove the complete row.
How can i do that?
Predicates like what you've given will remove entire rows. But none of your data is outside of 3 standard deviations. If you tone it down to just one standard deviation, rows are removed with your example data.
Here's an example using your data:
import pandas as pd
import numpy as np
columns = ["wave", "mean", "median", "mad"]
data = [
[4050.32, -0.016182, -0.011940, 0.008885],
[4208.98, 0.023707, 0.007189, 0.032585],
[4508.28, 3.662293, 0.001414, 7.193139],
[4531.62, -15.459313, -0.001523, 30.408377],
[4551.65, 0.009028, 0.007581, 0.005247],
[4554.46, 0.001861, 0.010692, 0.027969],
[6828.60, -10.604568, -0.000590, 21.084799],
[6839.84, -0.003466, -0.001870, 0.010169],
[6842.04, -32.751551, -0.002514, 65.118329],
[6842.69, 18.293519, -0.002158, 36.385884],
[6843.66, 0.006386, -0.002468, 0.034995],
[6855.72, 0.020803, 0.000886, 0.040529],
]
df = pd.DataFrame(np.array(data), columns=columns)
print("ORIGINAL: ")
print(df)
print()
res = df[np.abs(df['mad']-df['mad'].mean()) <= (df['mad'].std())]
print("REMOVED: ")
print(res)
this outputs:
ORIGINAL:
wave mean median mad
0 4050.32 -0.016182 -0.011940 0.008885
1 4208.98 0.023707 0.007189 0.032585
2 4508.28 3.662293 0.001414 7.193139
3 4531.62 -15.459313 -0.001523 30.408377
4 4551.65 0.009028 0.007581 0.005247
5 4554.46 0.001861 0.010692 0.027969
6 6828.60 -10.604568 -0.000590 21.084799
7 6839.84 -0.003466 -0.001870 0.010169
8 6842.04 -32.751551 -0.002514 65.118329
9 6842.69 18.293519 -0.002158 36.385884
10 6843.66 0.006386 -0.002468 0.034995
11 6855.72 0.020803 0.000886 0.040529
REMOVED:
wave mean median mad
0 4050.32 -0.016182 -0.011940 0.008885
1 4208.98 0.023707 0.007189 0.032585
2 4508.28 3.662293 0.001414 7.193139
3 4531.62 -15.459313 -0.001523 30.408377
4 4551.65 0.009028 0.007581 0.005247
5 4554.46 0.001861 0.010692 0.027969
6 6828.60 -10.604568 -0.000590 21.084799
7 6839.84 -0.003466 -0.001870 0.010169
10 6843.66 0.006386 -0.002468 0.034995
11 6855.72 0.020803 0.000886 0.040529
Observe that rows indexed 8 and 9 are now gone.
Be sure you're reassigning the output of df[np.abs(df['mad']-df['mad'].mean()) <= (df['mad'].std())] as shown above. The operation is not done in place.
Doing df[np.abs(df.mad-df.mad.mean()) <= (3*df.mad.std())] will not change the dataframe.
But assign it back to df, so that:
df = df[np.abs(df.mad-df.mad.mean()) <= (3*df.mad.std())]
I need some help with creating a new variable. I feel like the egen function is what I need to use but I can't figure it out.
I have 3 variables for cancer treatment - radiotherapy, chemotherapy and surgery - given with the number of times each patient has received each treatment.
I would like to create a new "Treatment" variable where 1 = radiotherapy, 2 = chemotherapy, 3 = surgery, 4 = combination (with 1 or more for any of the above 3), 5 = none
You should always show code you have tried and give example data. See https://stackoverflow.com/help/mcve for guidance.
Assume variables radio chemo surgery with values 0 or positive.
gen treatment = 5
replace treatment = 1 if radio & !chemo & !surgery
replace treatment = 2 if chemo & !radio & !surgery
replace treatment = 3 if surgery & !chemo & !radio
replace treatment - 4 if ((surgery > 0) + (radio > 0) + (chemo > 0)) > 1
using the facts that non-zero is true and its negation is false. See this FAQ
Another way to do it:
gen treatment = 5
replace treatment = 1 if radio
replace treatment = cond(treatment == 1, 4, 2) if chemo
replace treatment = cond(inlist(treatment, 1, 2), 4, 3) if surgery
In similar circumstances I would code your none category 0, not 5. That's likely to yield more sensible graphs and tables.
Code not tested.
i trying to assign a row number and a Set-number for List, but Set Number containing wrong number of rows in one set.
var objx = new List<x>();
var i = 0;
var r = 1;
objY.ForEach(x => objx .Add(new x
{
RowNumber = ++i,
DatabaseID= x.QuestionID,
SetID= i == 5 ? r++ : i % 5 == 0 ? r += 1 : r
}));
for Above code like objY Contains 23 rows, and i want to break 23 rows in 5-5 set.
so above code will give the sequence like[Consider only RowNumber]
[1 2 3 4 5][6 7 8 9][ 10 11 12 13 14 ].......
its a valid as by the logic
and if i change the logic for Setid as
SetID= i % 5 == 0 ? r += 1 : r
Result Will come Like
[1 2 3 4 ][5 6 7 8 9][10 11 12 13 14].
Again correct output of code
but expected for set of 5.
[1 2 3 4 5][ 6 7 8 9 10].........
What i missing.............
i should have taken my Maths class very Serious.
I think you want something like this:
var objX = objY.Select((x, i) => new { ObjX = x, Index = i })
.GroupBy(x => x.Index / 5)
.Select((g, i) =>
g.Select(x => new objx
{
RowNumber = x.Index + 1
DatabaseID = x.ObjX.QuestionID,
SetID = i + 1
}).ToList())
.ToList();
Note that i'm grouping by x.Index / 5 to ensure that every group has 5 items.
Here's a demo.
Update
it will be very helpful,if you can explain your logic
Where should i start? I'm using Linq methods to select and group the original list to create a new List<List<ObjX>> where every inner list has maximum 5 elements(less in the last if the total-count is not dividable by 5).
Enumerable.Select enables to project something from the input sequence to create something new. This method is comparable to a variable in a loop. In this case i project an anonymous type with the original object and the index of it in the list(Select has an overload that incorporates the index). I create this anonymous type to simply the query and because i need the index later in the GroupBy``.
Enumerable.GroupBy enables to group the elements in a sequence by a specified key. This key can be anything which is derivable from the element. Here i'm using the index two build groups of a maximum size of 5:
.GroupBy(x => x.Index / 5)
That works because integer division in C# (or C) results always in an int, where the remainder is truncated(unlike VB.NET btw), so 3/4 results in 0. You can use this fact to build groups of the specified size.
Then i use Select on the groups to create the inner lists, again by using the index-overload to be able to set the SetId of the group:
.Select((g, i) =>
g.Select(x => new objx
{
RowNumber = x.Index + 1
DatabaseID = x.ObjX.QuestionID,
SetID = i + 1
}).ToList())
The last step is using ToList on the IEnumerable<List<ObjX>> to create the final List<List<ObX>>. That also "materializes" the query. Have a look at deferred execution and especially Jon Skeets blog to learn more.