data.Hotel_Address.head(10)
0 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
1 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
2 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
3 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
4 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
5 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
6 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
7 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
8 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
9 s Gravesandestraat 55 Oost 1092 AA Amsterdam ...
How can I extract country name after last space till the end of string with regex in pandas?
Regex is not necessary, use split and select last lists by str[-1]:
data['new'] = data.Hotel_Address.str.split().str[-1]
Related
In SAS, I have a dataset(have) as below, I need to add a group variable based on test and visitnum. When visitnum is 101 and 108, they need to be in the same group. The desired as shown as data want.
data have:
test visitnum ord seq
aa 101 0 0
aa 101 0 1
aa 108 1 0
aa 108 1 1
aa 108 2 0
aa 108 2 1
aa 115 1 0
aa 115 1 1
aa 115 2 0
aa 115 2 1
bb 101 0 0
bb 101 0 1
bb 108 1 0
bb 108 1 1
bb 108 2 0
bb 108 2 1
bb 115 1 0
bb 115 1 1
bb 115 2 0
bb 115 2 1
data want:
test visitnum ord seq group
aa 101 0 0 1
aa 101 0 1 1
aa 108 1 0 1
aa 108 1 1 1
aa 108 2 0 1
aa 108 2 1 1
aa 115 1 0 2
aa 115 1 1 2
aa 115 2 0 2
aa 115 2 1 2
bb 101 0 0 3
bb 101 0 1 3
bb 108 1 0 3
bb 108 1 1 3
bb 108 2 0 3
bb 108 2 1 3
bb 115 1 0 4
bb 115 1 1 4
bb 115 2 0 4
bb 115 2 1 4
First sort your data by test and visitnum. There are two cases when we want to increment the group number:
When it's the start of a test group and the visitnum is 101 or 108
When it's the start of a visitnum group and it's not 101 or 108
Here's how this looks:
proc sort data=have;
by test visitnum;
run;
data want;
set have;
by test visitnum;
if( first.test AND visitnum IN(101, 108)
OR (first.visitnum AND visitnum NOT IN(101, 108) )
)
then group+1;
run;
Output:
test visitnum ord seq group
aa 101 0 0 1
aa 101 0 1 1
aa 108 1 0 1
aa 108 1 1 1
aa 108 2 0 1
aa 108 2 1 1
aa 115 1 0 2
aa 115 1 1 2
aa 115 2 0 2
.. ... .. .. ..
bb 115 2 0 4
bb 115 2 1 4
This is my first foray into using SAS macros, and I'm following this page from the amazing UCLA Stats Consulting Group. I'm interested in using macro variables in PROC MIXED to avoid copying and pasting blocks of code (my actual data set has ~400 variables).
My example modifies the UCLA example to have students in many schools.
data hsb3;
input id school female race ses prog
read write math science socst;
datalines;
1 1 0 4 1 1 57 52 41 47 57
2 1 1 4 2 3 68 59 53 63 61
3 1 0 2 3 1 44 33 54 58 31
4 1 0 4 3 3 63 44 47 53 56
5 1 0 4 2 2 47 51 43 50 61
6 1 1 4 2 2 44 52 51 50 61
7 1 0 3 2 1 50 59 60 56 52
8 1 0 1 2 2 34 46 52 53 57
9 1 0 4 2 2 63 57 51 63 61
19 2 0 3 1 2 57 63 41 63 61
20 2 1 4 2 2 60 57 51 58 31
21 2 0 4 3 2 57 55 51 53 56
22 2 0 4 3 2 73 46 71 50 61
23 2 0 4 2 1 54 65 57 50 61
24 2 1 4 2 2 45 60 50 56 52
25 2 0 3 2 1 42 63 43 53 57
26 2 0 1 1 2 34 57 51 63 61
27 2 0 4 2 2 63 49 60 55 31
10 3 1 3 2 2 57 55 51 55 31
11 3 1 4 3 3 60 46 71 31 56
12 3 1 4 2 2 57 66 57 55 61
13 3 0 3 3 2 50 60 50 31 61
14 3 0 4 3 2 57 57 57 55 46
15 3 0 3 3 3 68 55 50 31 56
16 3 0 4 1 2 34 46 43 50 56
17 3 0 4 3 2 34 65 51 50 56
18 3 0 4 1 2 63 60 60 47 57
28 4 1 3 2 2 57 52 52 53 61
29 4 1 4 2 3 60 57 51 63 61
30 4 1 1 2 2 57 65 51 55 46
31 4 0 4 3 2 73 60 71 31 56
32 4 0 4 3 2 54 63 57 55 46
33 4 0 3 1 2 45 57 50 31 56
34 4 0 1 1 1 42 49 43 50 56
35 4 0 4 3 2 47 52 51 50 56
36 4 0 4 2 1 57 57 60 56 52
;
run;
The UCLA example shows how to use macro variables with proc reg to do several simple linear regression models to predict reading score with any of the other variables:
%let indvars = write math female socst;
proc reg data = hsb3;
model read = &indvars;
run;
quit;
To do this taking school into account, we can use PROC MIXED instead:
proc mixed data = hsb3;
class school;
model read = &indvars;
random school;
run;
quit;
But what I really want to do is to see if any of the scores differ by gender (still taking school into account).
%let scores = read write math science socst;
proc mixed data = hsb3;
class school;
model &scores = female;
random school;
run;
quit;
Now I get the error:
NOTE: The SAS System stopped processing this step because of errors.
167 class school;
168 model &indvars = female;
-
22
200
NOTE: Line generated by the macro variable "INDVARS".
1 write math female socst
----
73
ERROR 22-322: Syntax error, expecting one of the following: a name, ;, (, *, -, /, :, #,
_CHARACTER_, _CHAR_, _NUMERIC_, |.
ERROR 200-322: The symbol is not recognized and will be ignored.
ERROR 73-322: Expecting an =.
Somehow the macro variable is not working. Is there a problem with using macro variables as a response variable in PROC MIXED? They work as a response variable in PROC REG....
proc reg data = hsb3;
model &scores = female;
run;
quit;
Your problem doesn't have anything to do with macro variables or macro code. Instead you are not creating a valid MODEL statement to use in PROC MIXED.
The MODEL statement names a single dependent variable ...
Try transforming the data perhaps?
%let scores = read write math science socst;
data want ; set hsb3 ;
array scores &scores ;
do i=1 to dim(scores);
score=scores(i);
name=vname(scores(i));
output;
end;
run;
proc sort; by name ; run;
proc mixed data = want;
by name;
class school;
model score = female;
random school;
run;
I have two data frames:
df1 =
Id ColA ColB ColC
1 aa bb cc
3 11 ww 55
5 11 bb cc
df2 =
Id ColD ColE ColF
1 ff ee rr
2 ww rr 55
3 hh 11 22
4 11 11 cc
5 cc bb aa
I need to merge these two data frames to get the following result:
result =
Id ColA ColB ColC ColD ColE ColF
1 aa bb cc ff ee rr
2 NaN NaN NaN ww rr 55
3 11 ww 55 hh 11 22
4 NaN NaN NaN 11 11 cc
5 11 bb cc cc bb aa
I do the merging this way:
import pandas as pd
result = pd.merge(df1,df2,on='Id')
However my result looks as follows instead of the expected above-shown result:
result =
Id ColA ColB ColC ColD ColE ColF
1 aa bb cc ff ee rr
3 11 ww 55 hh 11 22
5 11 bb cc cc bb aa
According to the documentation of merge, you need to specify the 'how' parameter as outer (the default is inner, which is consistent with what you're getting):
outer: use union of keys from both frames (SQL: full outer join)
inner: use intersection of keys from both frames (SQL: inner join)
data test;
infile datalines;
input k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
array a(*) k1-k10;
do i=1 to 10;
if a(i) eq . then stop;
line=a(i);
input #line k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
output;
end;
stop;
datalines;
5 9 2 4 6 3 . . . .
29 57 32 9 2 29 2 0 23 1
83 34 28 1 43 3 24 2 6 2
0 84 62 75 3 52 65 1 5 2
0 2 12 45 92 3 60 24 6 2
47 24 87 2 52 36 1 17 3 1
90 93 2 1 40 20 75 2 5 14
78 27 27 2 4 1 12 21 4 2
21 40 3 21 3 19 3 2 4 2
84 2 5 3 13 6 23 98 1 2
;
run;
I want to read only the observations whose numbers are in the first row. The expected result:
0 2 12 45 92 3 60 24 6 2
21 40 3 21 3 19 3 2 4 2
29 57 32 9 2 29 2 0 23 1
0 84 62 75 3 52 65 1 5 2
47 24 87 2 52 36 1 17 3 1
83 34 28 1 43 3 24 2 6 2
The error I get after running my code:
ERROR: Old line 3387 wanted but SAS is at line 3391.
Use: INFILE N=X; , with a suitable value of x.
RULE: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0
3391 47 24 87 2 52 36 1 17 3 1
k1=0 k2=2 k3=12 k4=45 k5=92 k6=3 k7=60 k8=24 k9=6 k10=2 i=2 line=2 _ERROR_=1 _N_=1
What does "a suitable value of x" mean? What should I change in my code?
You are overwriting the values in your array with your second input statement. Here they are read into different variables so as not to be overwritten.
data test;
infile datalines n=100;
input h1 h2 h3 h4 h5 h6 h7 h8 h9 h10;
array h{*} h1-h10;
do i = 1 to 10;
line = h[i];
if line then do;
input #line k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
output;
end;
end;
keep k:;
datalines;
5 9 2 4 6 3 . . . .
29 57 32 9 2 29 2 0 23 1
83 34 28 1 43 3 24 2 6 2
0 84 62 75 3 52 65 1 5 2
0 2 12 45 92 3 60 24 6 2
47 24 87 2 52 36 1 17 3 1
90 93 2 1 40 20 75 2 5 14
78 27 27 2 4 1 12 21 4 2
21 40 3 21 3 19 3 2 4 2
84 2 5 3 13 6 23 98 1 2
;
run;
SAS is telling you that you need to amend your infile statement to allow it to read a sufficient number of lines ahead. For your code as written, n=10 should be ok, as none of variables you're using to get the line number have values greater than 10.
data test;
/*Add the n= option to the infile statement as suggested by log message*/
infile datalines n= 10;
input k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
array a(*) k1-k10;
array b(*) b1-b10;
/*Make a copy of the first row
that won't get overwritten by subsequent input statements*/
do i=1 to 10;
b(i) = a(i);
end;
do i=1 to 10;
if b(i) eq . then stop;
line=b(i);
input #line k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
output;
end;
stop;
datalines;
5 9 2 4 6 3 . . . .
29 57 32 9 2 29 2 0 23 1
83 34 28 1 43 3 24 2 6 2
0 84 62 75 3 52 65 1 5 2
0 2 12 45 92 3 60 24 6 2
47 24 87 2 52 36 1 17 3 1
90 93 2 1 40 20 75 2 5 14
78 27 27 2 4 1 12 21 4 2
21 40 3 21 3 19 3 2 4 2
84 2 5 3 13 6 23 98 1 2
;
run;
I have this data set:
data a1q1;
input pid los age gender $ temp wbc anti service $ ;
cards;
1 5 30 F 99 82 2 M
2 10 73 F 98 52 1 M
3 6 40 F 99 122 2 S
4 11 47 F 98 42 2 S
5 5 25 F 99 112 2 S
6 14 82 M 97 61 2 S
7 30 60 M 100 81 1 M
8 11 56 F 99 72 2 M
9 17 43 F 98 72 2 M
10 3 50 M 98 122 1 S
11 9 59 F 98 72 1 M
12 3 4 M 98 32 2 S
13 8 22 F 100 111 2 S
14 8 33 F 98 141 1 S
15 5 20 F 98 112 1 S
16 5 32 M 99 92 2 S
17 7 36 M 99 61 2 S
18 4 69 M 98 62 2 S
19 3 47 M 97 51 2 M
20 7 22 M 98 62 2 S
21 9 11 M 98 102 2 S
22 11 19 M 99 141 2 S
23 11 67 F 98 42 2 M
24 9 43 F 99 52 2 S
25 4 41 F 98 52 2 M
;
I need to use PROC SGPLOT to output an identical, if not, similar barchart that would be outputted from the following PROC:
proc gchart data = a1q1;
vbar wbc / group = gender;
run;
I need PROC SGPLOT to group the two genders together and not stack them. I have tried coding this way but to no avail:
proc sgplot data = a1q1;
vbar wbc / group= gender response =wbc stat=freq nostatlabel;
run;
How would I go about coding to get the output I need?
Thank you for your time!
Sounds like you should use SGPANEL, not SGPLOT. SGPLOT can make grouped bar charts, but not automatically make histogram bins without using a format (you could do that if you want) and doesn't support group with the histogram plot. However, SGPANEL can handle that.
proc sgpanel data=a1q1;
panelby gender;
histogram wbc;
run;