I want to set a variable to 0 if the "Independent" column is equal to 0,1,or null. I have been trying something like this:
df["Iflag"] = df.Independent.where((df.Independent == 0) | (df.Independent == 1 )|(df.Independent.isnull())).astype(int)
Iflag = df[df.Iflag == 0]
pd.DataFrame(Iflag, columns=["LocIdent","Independent"]).to_csv(Targcsv, mode='ab')
I get an error that says I cannot convert NA to integer. This code works when I drop the check to see if Independent is null. My question is, what is the best way to write an if statement that includes null values in Pandas?
I'd just fill the nan values first and then your code would work, NaN cannot be represented using ints hence the error.
So something like
# fill the nan values
df.Independent = df.Independent.fillna(0)
# set any values that are 1 to 0
df.loc[df.Indepedent == 1, 'Independent'] = 0
# take a view of the df where the value is 0
Iflag = df[df.Independent == 0]
pd.DataFrame(Iflag, columns=["LocIdent","Independent"]).to_csv(Targcsv, mode='ab')
It's redundant to check where a value is 0 if all you're going to is set it to 0 again anyway so all you need to do is find the rows where it's 1 already, set these to 0 and then take a view of the df where the condition is satisfied.
Related
I have data like:
Folder Replied Complied
1 testing 1 1
2 /complete/ 0 1
3 none 1 1
4 Incomplete 0 1
5 complete// 0 0
6 Incomplete 1 0
7 ABCcomplete 1 1
I like a measure to calculate the average of Complied (sum divided by count), only where Folder contains the string complete AND Replied is 0 (both conditions simultaneously).
Therefore rows 2, 4, 5 should be used in the count, resulting in 0.66... (1 + 1 + 0)/3
i've tried several things but the formula either results in an error, or returns the wrong result
i.e.
Measure = CALCULATE (
Average( [Complied]),
CONTAINSSTRING([Folder],"complete") && [replied] = 0
)
DAX is very confusing to me. Thanks in advance
edit:
I've seen examples like
`
= CALCULATE(AVERAGE([col]), CONTAINSSTRING([Folder],"complete") , [replied] = 0)
note the , instead of && but that doesn't work for some reason either. Neither does AND(condition1, condition2).
This dax measure should be the one you are looking for:
Measure = CALCULATE(AVERAGE(Sheet1[Complied]),
CONTAINSSTRING(Sheet1[Folder],"complete") && Sheet1[Replied]=0)
So how is it working?
ContainString to check about "complete", work like VBA instr function
&& in order to meet both condition
Calculate(method, expression) to filter all the value
Scorecard
You may first test with the following measure to check if statement is working in your case first:
IF(CONTAINSSTRING(Sheet1[Folder],"complete") && Sheet1[Replied]=0,"True","False")
Only three row is True here:
I wanted to find out the number of 0's at end of integer.
Eg for 2020 it should count 1
for 2000 it should count 3
for 3010000 it should count 4
I have no idea to do it without counting all the zeros and not just the ending ones!
someone please help :)
Go to Power Query Editor and add a Custom Colum with this below code-
if Number.Mod([number],100000) = 0 then 5
else if Number.Mod([number],10000) = 0 then 4
else if Number.Mod([number],1000) = 0 then 3
else if Number.Mod([number],100) = 0 then 2
else if Number.Mod([number],10) = 0 then 1
else 0
Considered highst possibility of trailing 0 is 5. You can add more if/else case following the above logic if you predict more numbers of consecutive 0 at the end.
Here is sample output using above logic-
Take advantage of the fact, that text "00123" converted to number will be 2 characters shorter.
= let
TxtRev = Text.Reverse(Number.ToText([num]))&"1", /*convert to text and reverse, add 1 to handle num being 0*/
TxtNoZeroes = Number.ToText(Number.FromText(TxtRev)) /*convert to number to remove starting zeroes and then back to text*/
in
Text.Length(TxtRev)-Text.Length(TxtNoZeroes) /*compare length of original value with length without zeroes*/
This will work for any number of trailing zeroes (up to Int64 capacity of course, minus space for &"1"). Assuming that the column is of number type; if it's a text then just remove Number.ToText in TxtRev. If you have negative numbers or decimals, replace characters not being a digit after converting to text. For initial number being 0 it shows 1, but if it should show 0 just remove &"1".
You can do it as general string manipulation:
= Text.Length(Text.From([number])) - Text.Length(Text.TrimEnd(Text.From(number]), "0"))
We convert the column to string, strip of the zeroes, count take that away from the total length, giving you the amount of stripped zeroes.
Edit: I messed up my first answer, this one should in fact be correct
I noticed that the following two for-loop cases behave differently sometimes while most of the time they are the same. I couldn't figure out the pattern, does anyone have any idea? Thanks!
case 1:
for (i <- myList.length - 1 to 0 by -1) { ... }
case 2:
for (i <- myList.length - 1 to 0) { ...}
Well, they definitely don't do the same things. n to 0 by -1 means "start at n and go to 0, counting backwards by 1. So:
5 to 0 by -1
// res0: scala.collection.immutable.Range = Range(5, 4, 3, 2, 1, 0)
Whereas n to 0 means "start at n and got to 0 counting forward by 1". But you'll notice that if n > 0, then there will be nothing in that list, since there is no way to count forward to 0 from anything greater than zero.
5 to 0
// res1: scala.collection.immutable.Range.Inclusive = Range()
The only way that they would produce the same result is if n=0 since counting from 0 to 0 is the same forwards and backwards:
0 to 0 by -1 // Range(0)
0 to 0 // Range(0)
In your case, since you're starting at myList.length - 1, they will produce the same result when the length of myList is 1.
In summary, the first version makes sense, because you want to count down to 0 by counting backward (by -1). And the second version doesn't make sense because you're not going to want to count forward to 0 from a length (which is necessarily non-negative).
First, we need to learn more about how value members to and by works.
to - Click Here for API documentation
to is a value member that appears in classes like int, double etc.
scala> 1 to 3
res35: scala.collection.immutable.Range.Inclusive = Range(1, 2, 3)
Honestly, you don't have to use start to end by step and start.to(end, step) will also work if you are more comfortable working with in this world. Basically, to will return you a Range.Inclusive object if we are talking about integer inputs.
by - Click Here for API documentation
Create a new range with the start and end values of this range and a new step
scala> Range(1,8) by 3
res54: scala.collection.immutable.Range = Range(1, 4, 7)
scala> Range(1,8).by(3)
res55: scala.collection.immutable.Range = Range(1, 4, 7)
In the end, lets spend some time looking at what happens when the step is on a different direction from start to end. Like 1 to 3 by -1
Here is the source code of the Range class and it is actually pretty straightforward to read:
def by(step: Int): Range = copy(start, end, step)
So by is actually calling a function copy, so what is copy?
protected def copy(start: Int, end: Int, step: Int): Range = new Range(start, end, step)
So copy is literally recreate a new range with different step, then lets look at the constructor or Range itself.
Reading this paragraph of code
override final val isEmpty = (
(start > end && step > 0)
|| (start < end && step < 0)
|| (start == end && !isInclusive)
)
These cases will trigger the exception and your result will be a empty Range in cases like 1 to 3 by -1..etc.
Sorry the length of my post is getting out of control since I am also learning Scala now.
Why don't you just read the source code of Range, it is written by Martin Odersky and it is only 500 lines including comments :)
>>> movement = ((x,y)for x in range(-1,2) for y in range (-1,2) if (x,y)!=(0,0))
>>> for x,y in movement:
print x,y
-1 -1
-1 0
-1 1
0 -1
0 1
1 -1
1 0
1 1
>>> for a,b in movement:
print a,b
>>> for x,y in movement:
print x,y
>>>
THis is happening in terminal and idle both (Am i doing something wrong or is this supposed to happen?)
You are creating a generator object (https://wiki.python.org/moin/Generators). This means that you can iterate over it exactly once - that's why the first time it works as advertised.
Change the movement 'maker' to this and it works; now it creates a list, not a generator.
movement = [(x,y)for x in range(-1,2) for y in range (-1,2) if (x,y)!=(0,0)]
To elaborate a bit; on the linked page it sais this:
In fact, we can turn a list comprehension into a generator expression
by replacing the square brackets ("[ ]") with parentheses.
i = 0;
while (fscanf(fp, "%f %f %d", &x[i], &y[i], &outputs[i]) != EOF) {
if (outputs[i] == 0) {
outputs[i] = -1;
}
i++;
}
patternCount = i;
I dont understand the meaning of this line from the above code:
if (outputs[i] == 0) {
outputs[i] = -1;
What does it represent. The Output is a matrix or vector.??
The refrence of the code is:
Perceptron learning algorithm not converging to 0
I have a output file that has 3 columns:
1 0 0
0 0 1
0 1 0
So it a vector file??
outputs is defined as a one-dimensional array containing integer values..
float x[208], y[208];
int outputs[208];
Each index in the array can be seen as corresponding to a line read in the data file.
i x y outputs
--------------------------------------
0 | -8.818681 3.025210 1
1 | 3.653846 -2.969188 0
2 | ... ... .
.. | ... ... .
208 | -6.565934 -4.649860 1
Where if i == 0 then
x[0] == -8.818681
y[0] == 3.025210
outputs[0] == 1
The wonderful code and information posted by user Amro explain the limits and function of outputs.
"...bias term, i.e. a third weight component connected to an input of value 1. (+1/-1)
"
Values for outputs in the data file have been assigned one/zero values.
Therefore the code in question checks to see if the value for outputs read in from the file is equal to zero and re-assigns to a -1.
if (outputs[i] == 0)
outputs[i] = -1;
as far as I can tell, the code is reading from a file, and the file is supposed to have a repeated pattern, each pattern consists of 3 numbers.
Your loop copy the first number in each pattern to x, the second to y, and the last to outputs. However whenever the third number is a zero, it is changed to -1.
patternCount will store the number of pattern read in the file
A perceptron is a term from artificial intelligence/neural networks. It operates in much the same way as a single neuron is supposed to operate in the brain.
It has a number of inputs and a single output.
All this file is doing is specifying what the output should be for a given set of inputs. That's why the x/y and output are named differently.
As to why it's morphing the output from 0 to -1 (that's all it's doing by the way: changing zeros in the third file column into negative one), I'm not sure. The outputs of perceptrons almost invariably feed into other perceptrons so passing a -1 to something that expects 0 or 1 is an ... interesting ... idea.