I'm trying to do this: if the last column is negative number from 1-5 then write second and last column to a file "neg.txt". If a last column is positive number, second and last column need to be written to "pos.txt". My both output files end up empty after execution. I don't know what's wrong with the code, when I think if statement can handle multiple conditions. I also tried with regular expressions but it did't work so I made it as simple as possible to see what is not working.
The input file looks like this:
abandon odustati -2
abandons napusta -2
abandoned napusten -2
absentee odsutne -1
absentees odsutni -1
aboard na brodu 1
abducted otet -2
accepted prihvaceno 1
My code is:
from urllib.request import urlopen
import re
pos=open('lek_pos.txt','w')
neg=open('lek_neg.txt','w')
allCondsAreOK1 = ( parts[2]=='1' and parts[2]=='2' and
parts[2]=='3' and parts[2]=='4' and parts[2]=='5' )
allCondsAreOK2 = ( parts[2]=='-1' and parts[2]=='-2' and
parts[2]=='-3' and parts[2]=='-4' and parts[2]=='-5' )
with open('leksicki_resursi.txt') as pos:
for line in pos:
parts=line.split() # split line into parts
if len(parts) > 1: # if at least 2 columns (parts)
if allCondsAreOK:
pos.write(parts[1]+parts[2])
elif allCondsAreOK2:
neg.write(parts[1]+parts[2])
else:
print("nothing matches")
You don't need a regex, you just need an if/elif checking if after casting to int the last value falls between -5 and -1, if it does you write to the neg file or if the value is any non negative number you write to the pos file:
with open('leksicki_resursi.txt') as f, open('lek_pos.txt','w')as pos, open('lek_neg.txt','w') as neg:
for row in map(str.split, f):
a, b = row[1], int(row[-1])
if b >= 0:
pos.write("{},{}\n".format(a, b))
elif -5 <= b <= -1:
neg.write("{},{}\n".format(a, b))
If the positive nums must also be between 1-5 then you can do something similar to the negative condition:
if 5 >= int(b) >= 0:
pos.write("{},{}\n".format(a, b))
elif -5 <= int(b) <= -1:
neg.write("{},{}\n".format(a, b))
Also if you have empty lines you can filter them out:
for row in filter(None,map(str.split, f)):
I have a data frame and I'm trying to loop through the data frame to identify those columns which contain a special character or which are all capital letters.
I have tried a few things but nothing where I'm apple to catch the column names within the loop.
data = data.frame(one=c(1,3,5,1,3,5,1,3,5,1,3,5), two=c(1,3,5,1,3,5,1,3,5,1,3,5),
thr=c("A","B","D","E","F","G","H","I","J","H","I","J"),
fou=c("A","B","D","A","B","D","A","B","D","A","B","D"),
fiv=c(1,3,5,1,3,5,1,3,5,1,3,5),
six=c("A","B","D","E","F","G","H","I","J","H","I","J"),
sev=c("A","B","D","A","B","D","A","B","D","A","B","D"),
eig=c("A","B","D","A","B","D","A","B","D","A","B","D"),
nin=c(1.24,3.52,5.33,1.44,3.11,5.33,1.55,3.66,5.33,1.32,3.54,5.77),
ten=c(1:12),
ele=rep(1,12),
twe=c(1,2,1,2,1,2,1,2,1,2,1,2),
thir=c("THiS","THAT34","T(&*(", "!!!","#$#","$Q%J","who","THIS","this","this","this","this"),
stringsAsFactors = FALSE)
data
colls <- c()
spec=c("$","%","&")
for( col in names(data) ) {
if( length(strings[stringr::str_detect(data[,col], spec)]) >= 1 ){
print("HORRAY")
colls <- c(collls, col)
}
else print ("NOOOOOOOOOO")
}
for( col in names(data) ) {
if( any(data[,col]) %in% spec ){
print("HORRAY")
colls <- c(collls, col)
}
else print ("NOOOOOOOOOO")
}
Can anyone shed light on a good way to tackle this problem.
EDIT:
The end goal is to have a vector with a name of column names which meet that criteria. Sorry for my poor SO question, but hopefully this will help with what I'm trying to do
I would use grep() to search for the pattern you are interested in. See here.
[:upper:] Matches any upper case letters.
Combining it with anchors (^,$) and match one or more times (+) gives ^[[:upper:]]+$ and should only match entries completely in capitals.
The following would match the special characters in your toy data set (but is not guaranteed to match all special characters in your real data set i.e form feeds, carriage returns)
[:punct:] #Matches punctuation - ! " # $ % & ' ( ) * + , - . / : ; < = > ? # [ \ ] ^ _ ` { | } ~.
Note that rather than use [:punct:] you could define your special characters manually.
We can try the resultant code on the first row of your data set:
#Using grepl() rather than grep() so that we return a list of logical values.
grepl(x= data[1,], pattern = "^[[:upper:]]+$|[[:punct:]]")
[1] FALSE FALSE TRUE TRUE FALSE TRUE TRUE TRUE TRUE FALSE FALSE FALSE FALSE
This gives us our expected response except for column nine which has the value 1.24. Here the decimal point is being recognised as punctuation and is being flagged as a match.
We can add a "negative lookahead assertion" - (?!\\.) - to remove any periods from consideration, before they are even tested for being punctuation characters. Note we use \ to escape the period.
grepl(x= data[1,], perl = TRUE, pattern = "(?!\\.)(^[[:upper:]]+$|[[:punct:]])")
[1] FALSE FALSE TRUE TRUE FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE
This returns a better response - it now no longer matches decimal places. NOTE: This might not be what you want as this pattern also won't match any fullstops in character fields. You would need to refine the pattern further.
Rather than use a 'for loop' to reiterate this code across every row in your dataframe I would use vectorization instead which is 'more R like'.
To do this we must convert our script into a function which we will call with apply()
myFunction <- function(x){
matches <- grepl(x= x, perl = TRUE, pattern = "(?!\\.)(^[[:upper:]]+$|[[:punct:]])")
#Given a set of logical vectors 'matches', is at least one of the values true? using any()
return(any(matches))
}
apply(X = data, 1, myFunction)
The 1 above instructs apply() to reiterate across rows rather than columns.
[1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
In your example data set all rows have an entry containing a special character or a string of all capital letters. This is unsurprising as many columns in your example data set are a list of single capital letters.
If you are just interested in which values in column thirteen fit the stated criteria you can use:
matches <- grepl(x= data$thir, perl = TRUE, pattern = "(?!\\.)(^[[:upper:]]+$|[[:punct:]])")
matches
[1] FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE
To subset your dataframe on matching rows:
data[matches,]
one two thr fou fiv six sev eig nin ten ele twe thir
3 5 5 D D 5 D D D 5.33 3 1 1 T(&*(
4 1 1 E A 1 E A A 1.44 4 1 2 !!!
5 3 3 F B 3 F B B 3.11 5 1 1 #$#
6 5 5 G D 5 G D D 5.33 6 1 2 $Q%J
8 3 3 I B 3 I B B 3.66 8 1 2 THIS
To subset your dataframe on non-matching rows:
data[!matches,]
one two thr fou fiv six sev eig nin ten ele twe thir
1 1 1 A A 1 A A A 1.24 1 1 1 THiS
2 3 3 B B 3 B B B 3.52 2 1 2 THAT34
7 1 1 H A 1 H A A 1.55 7 1 1 who
9 5 5 J D 5 J D D 5.33 9 1 1 this
10 1 1 H A 1 H A A 1.32 10 1 2 this
11 3 3 I B 3 I B B 3.54 11 1 1 this
12 5 5 J D 5 J D D 5.77 12 1 2 this
Note that the regular expression used doesn't match THAT34 as it isn't composed wholly of capitalised letters, having the number 34 at the end.
EDIT:
To get a list of column names identifying columns that fulfill the criteria in your edit use myFunction described above with:
colnames(data)[apply(X = data, 2, myFunction)]
"thr" "fou" "six" "sev" "eig" "thir"
The number in apply() changes from 1 to 2 to reiterate across columns rather than rows. We pass the output from apply(), a list of logical matches (TRUE or FALSE), to colnames(data) - this returns the matching column names via subsetting.
I would collapse the data into strings (one string per row)
strings = apply(data, 1, paste, collapse = "")
contains_only_caps = strings == toupper(strings)
strings[contains_only_caps]
# [1] "33BB3BBB3.52 212THAT34" "55DD5DDD5.33 311T(&*(" "11EA1EAA1.44 412!!!" "33FB3FBB3.11 511#$#"
# [5] "55GD5GDD5.33 612$Q%J" "33IB3IBB3.66 812THIS"
# escaping special characters
spec=c("\\$","%","\\&")
contains_spec = stringr::str_detect(strings, pattern = paste(spec, collapse = "|"))
strings[contains_spec]
# [1] "55DD5DDD5.33 311T(&*(" "33FB3FBB3.11 511#$#" "55GD5GDD5.33 612$Q%J"
You could also use which on contains_spec or contains_only_caps to get the corresponding row numbers for the original data frame. I think that using strings rather than row-wise data frame elements will by much faster - as long as you want to search the whole strings, not certain columns for certain conditions.
Good evening,
I have a list "a" which I successfully subset using regular expressions.
a=a[grep("Macy*|Nors*", a$Geography, perl=TRUE),]
a=a[grep("Levis*|Diesel*|Replay*", a$Brand.Name, perl=TRUE), ]
a=a[grep("Week*", a$Time, perl=TRUE), ]
I created the function clean below but when I apply it to my list "a"nothing happens
clean=function(x){
x=x[grep("Macy*|Nors*", x$Geography, perl=TRUE),]
x=x[grep("Levis*|Diesel*|Replay*", x$Brand.Name, perl=TRUE), ]
x=x[grep("Week*", x$Time, perl=TRUE), ]
return (x)
}
clean(a) just returns original "a"
I tried printing each individual step but literally nothing happens.
Thank you for your help
a = data.frame(Geography = c(paste('Macy',1:5, sep = " "),'john'),stringsAsFactors = FALSE)
a
Geography
1 Macy 1
2 Macy 2
3 Macy 3
4 Macy 4
5 Macy 5
6 john
clean = function(x){
x = x[grep("Macy*|Nors*", x[['Geography']], perl = TRUE),]
return(x)
}
clean(a)
[1] "Macy 1" "Macy 2" "Macy 3" "Macy 4" "Macy 5"
A<-c(1,2,3,4,5,6)
B<-c("L124","L234","Not","R23","NIMT","Lreg")
DF<-data.frame(A,B)
I would like to create a third column of values from Column B that begin with L. Any other values that do not begin with L would be listed as "Not L"
The final result would look like:
A B C
1 1 L124 L124
2 2 L234 L234
3 3 Not Not L
4 4 R23 Not L
5 5 NIMT Not L
6 6 Lreg Lreg
Using ifelse and grepl for example:
ifelse(grepl('^L',B),B,"Not L")
[1] "L124" "L234" "Not L" "Not L" "Not L" "Lreg"