I have data set as below. Can someone help me to import data to hdfs using sqoop boundary query, Using the column (id) which is having duplicate keys.
mysql> select id,name,age from employee;
id name age
1 A 30
2 B 35
3 C 40
4 D 23
5 E 26
1 A 24
2 B 16
3 C 78
4 G 66
3 H 56
4 A 63
20 C 58
13 F 47
2 A 49
3 B 60
Related
I am Looking for SAS coding to compute scores for a whole cohort based on scores calculated for a subgroup
I can create scores in the whole population by itself as my whole dataset but have no experience in using the fitted values of a subgroup dataset to compute scores for the whole population
I work with SAS coding
NA
Welcome to stackoverflow! If I understand your question, this will do what you want.
I grabbed some data from sas support:
Data Neuralgia;
input Treatment $ Sex $ Age Duration Pain $ ##;
datalines;
P F 68 1 No B M 74 16 No P F 67 30 No
P M 66 26 Yes B F 67 28 No B F 77 16 No
A F 71 12 No B F 72 50 No B F 76 9 Yes
A M 71 17 Yes A F 63 27 No A F 69 18 Yes
B F 66 12 No A M 62 42 No P F 64 1 Yes
A F 64 17 No P M 74 4 No A F 72 25 No
P M 70 1 Yes B M 66 19 No B M 59 29 No
A F 64 30 No A M 70 28 No A M 69 1 No
B F 78 1 No P M 83 1 Yes B F 69 42 No
B M 75 30 Yes P M 77 29 Yes P F 79 20 Yes
A M 70 12 No A F 69 12 No B F 65 14 No
B M 70 1 No B M 67 23 No A M 76 25 Yes
P M 78 12 Yes B M 77 1 Yes B F 69 24 No
P M 66 4 Yes P F 65 29 No P M 60 26 Yes
A M 78 15 Yes B M 75 21 Yes A F 67 11 No
P F 72 27 No P F 70 13 Yes A M 75 6 Yes
B F 65 7 No P F 68 27 Yes P M 68 11 Yes
P M 67 17 Yes B M 70 22 No A M 65 15 No
P F 67 1 Yes A M 67 10 No P F 72 11 Yes
A F 74 1 No B M 80 21 Yes A F 69 3 No
;
run;
Then subsetted down to build a model using only the males:
data males;
set Neuralgia;
where sex = "M";
run;
Then I built a model and saved the model details, into the work library, in a file called theMaleModel.
proc logistic data=males outmodel=work.theMaleModel;
class Treatment;
model Pain = Treatment Age Duration ;
run;
Then I apply the male model to the full dataset and save the scored results into a dataset, in the work library, called scoreEverybody:
proc logistic inmodel=work.theMaleModel;
score data=Neuralgia out=scoreEverybody;
run;
You can see more examples like this if you look here. If that answers your question please click the check next to this answer.
This is my first foray into using SAS macros, and I'm following this page from the amazing UCLA Stats Consulting Group. I'm interested in using macro variables in PROC MIXED to avoid copying and pasting blocks of code (my actual data set has ~400 variables).
My example modifies the UCLA example to have students in many schools.
data hsb3;
input id school female race ses prog
read write math science socst;
datalines;
1 1 0 4 1 1 57 52 41 47 57
2 1 1 4 2 3 68 59 53 63 61
3 1 0 2 3 1 44 33 54 58 31
4 1 0 4 3 3 63 44 47 53 56
5 1 0 4 2 2 47 51 43 50 61
6 1 1 4 2 2 44 52 51 50 61
7 1 0 3 2 1 50 59 60 56 52
8 1 0 1 2 2 34 46 52 53 57
9 1 0 4 2 2 63 57 51 63 61
19 2 0 3 1 2 57 63 41 63 61
20 2 1 4 2 2 60 57 51 58 31
21 2 0 4 3 2 57 55 51 53 56
22 2 0 4 3 2 73 46 71 50 61
23 2 0 4 2 1 54 65 57 50 61
24 2 1 4 2 2 45 60 50 56 52
25 2 0 3 2 1 42 63 43 53 57
26 2 0 1 1 2 34 57 51 63 61
27 2 0 4 2 2 63 49 60 55 31
10 3 1 3 2 2 57 55 51 55 31
11 3 1 4 3 3 60 46 71 31 56
12 3 1 4 2 2 57 66 57 55 61
13 3 0 3 3 2 50 60 50 31 61
14 3 0 4 3 2 57 57 57 55 46
15 3 0 3 3 3 68 55 50 31 56
16 3 0 4 1 2 34 46 43 50 56
17 3 0 4 3 2 34 65 51 50 56
18 3 0 4 1 2 63 60 60 47 57
28 4 1 3 2 2 57 52 52 53 61
29 4 1 4 2 3 60 57 51 63 61
30 4 1 1 2 2 57 65 51 55 46
31 4 0 4 3 2 73 60 71 31 56
32 4 0 4 3 2 54 63 57 55 46
33 4 0 3 1 2 45 57 50 31 56
34 4 0 1 1 1 42 49 43 50 56
35 4 0 4 3 2 47 52 51 50 56
36 4 0 4 2 1 57 57 60 56 52
;
run;
The UCLA example shows how to use macro variables with proc reg to do several simple linear regression models to predict reading score with any of the other variables:
%let indvars = write math female socst;
proc reg data = hsb3;
model read = &indvars;
run;
quit;
To do this taking school into account, we can use PROC MIXED instead:
proc mixed data = hsb3;
class school;
model read = &indvars;
random school;
run;
quit;
But what I really want to do is to see if any of the scores differ by gender (still taking school into account).
%let scores = read write math science socst;
proc mixed data = hsb3;
class school;
model &scores = female;
random school;
run;
quit;
Now I get the error:
NOTE: The SAS System stopped processing this step because of errors.
167 class school;
168 model &indvars = female;
-
22
200
NOTE: Line generated by the macro variable "INDVARS".
1 write math female socst
----
73
ERROR 22-322: Syntax error, expecting one of the following: a name, ;, (, *, -, /, :, #,
_CHARACTER_, _CHAR_, _NUMERIC_, |.
ERROR 200-322: The symbol is not recognized and will be ignored.
ERROR 73-322: Expecting an =.
Somehow the macro variable is not working. Is there a problem with using macro variables as a response variable in PROC MIXED? They work as a response variable in PROC REG....
proc reg data = hsb3;
model &scores = female;
run;
quit;
Your problem doesn't have anything to do with macro variables or macro code. Instead you are not creating a valid MODEL statement to use in PROC MIXED.
The MODEL statement names a single dependent variable ...
Try transforming the data perhaps?
%let scores = read write math science socst;
data want ; set hsb3 ;
array scores &scores ;
do i=1 to dim(scores);
score=scores(i);
name=vname(scores(i));
output;
end;
run;
proc sort; by name ; run;
proc mixed data = want;
by name;
class school;
model score = female;
random school;
run;
data test;
infile datalines;
input k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
array a(*) k1-k10;
do i=1 to 10;
if a(i) eq . then stop;
line=a(i);
input #line k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
output;
end;
stop;
datalines;
5 9 2 4 6 3 . . . .
29 57 32 9 2 29 2 0 23 1
83 34 28 1 43 3 24 2 6 2
0 84 62 75 3 52 65 1 5 2
0 2 12 45 92 3 60 24 6 2
47 24 87 2 52 36 1 17 3 1
90 93 2 1 40 20 75 2 5 14
78 27 27 2 4 1 12 21 4 2
21 40 3 21 3 19 3 2 4 2
84 2 5 3 13 6 23 98 1 2
;
run;
I want to read only the observations whose numbers are in the first row. The expected result:
0 2 12 45 92 3 60 24 6 2
21 40 3 21 3 19 3 2 4 2
29 57 32 9 2 29 2 0 23 1
0 84 62 75 3 52 65 1 5 2
47 24 87 2 52 36 1 17 3 1
83 34 28 1 43 3 24 2 6 2
The error I get after running my code:
ERROR: Old line 3387 wanted but SAS is at line 3391.
Use: INFILE N=X; , with a suitable value of x.
RULE: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0
3391 47 24 87 2 52 36 1 17 3 1
k1=0 k2=2 k3=12 k4=45 k5=92 k6=3 k7=60 k8=24 k9=6 k10=2 i=2 line=2 _ERROR_=1 _N_=1
What does "a suitable value of x" mean? What should I change in my code?
You are overwriting the values in your array with your second input statement. Here they are read into different variables so as not to be overwritten.
data test;
infile datalines n=100;
input h1 h2 h3 h4 h5 h6 h7 h8 h9 h10;
array h{*} h1-h10;
do i = 1 to 10;
line = h[i];
if line then do;
input #line k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
output;
end;
end;
keep k:;
datalines;
5 9 2 4 6 3 . . . .
29 57 32 9 2 29 2 0 23 1
83 34 28 1 43 3 24 2 6 2
0 84 62 75 3 52 65 1 5 2
0 2 12 45 92 3 60 24 6 2
47 24 87 2 52 36 1 17 3 1
90 93 2 1 40 20 75 2 5 14
78 27 27 2 4 1 12 21 4 2
21 40 3 21 3 19 3 2 4 2
84 2 5 3 13 6 23 98 1 2
;
run;
SAS is telling you that you need to amend your infile statement to allow it to read a sufficient number of lines ahead. For your code as written, n=10 should be ok, as none of variables you're using to get the line number have values greater than 10.
data test;
/*Add the n= option to the infile statement as suggested by log message*/
infile datalines n= 10;
input k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
array a(*) k1-k10;
array b(*) b1-b10;
/*Make a copy of the first row
that won't get overwritten by subsequent input statements*/
do i=1 to 10;
b(i) = a(i);
end;
do i=1 to 10;
if b(i) eq . then stop;
line=b(i);
input #line k1 k2 k3 k4 k5 k6 k7 k8 k9 k10;
output;
end;
stop;
datalines;
5 9 2 4 6 3 . . . .
29 57 32 9 2 29 2 0 23 1
83 34 28 1 43 3 24 2 6 2
0 84 62 75 3 52 65 1 5 2
0 2 12 45 92 3 60 24 6 2
47 24 87 2 52 36 1 17 3 1
90 93 2 1 40 20 75 2 5 14
78 27 27 2 4 1 12 21 4 2
21 40 3 21 3 19 3 2 4 2
84 2 5 3 13 6 23 98 1 2
;
run;
I have this data set:
data a1q1;
input pid los age gender $ temp wbc anti service $ ;
cards;
1 5 30 F 99 82 2 M
2 10 73 F 98 52 1 M
3 6 40 F 99 122 2 S
4 11 47 F 98 42 2 S
5 5 25 F 99 112 2 S
6 14 82 M 97 61 2 S
7 30 60 M 100 81 1 M
8 11 56 F 99 72 2 M
9 17 43 F 98 72 2 M
10 3 50 M 98 122 1 S
11 9 59 F 98 72 1 M
12 3 4 M 98 32 2 S
13 8 22 F 100 111 2 S
14 8 33 F 98 141 1 S
15 5 20 F 98 112 1 S
16 5 32 M 99 92 2 S
17 7 36 M 99 61 2 S
18 4 69 M 98 62 2 S
19 3 47 M 97 51 2 M
20 7 22 M 98 62 2 S
21 9 11 M 98 102 2 S
22 11 19 M 99 141 2 S
23 11 67 F 98 42 2 M
24 9 43 F 99 52 2 S
25 4 41 F 98 52 2 M
;
I need to use PROC SGPLOT to output an identical, if not, similar barchart that would be outputted from the following PROC:
proc gchart data = a1q1;
vbar wbc / group = gender;
run;
I need PROC SGPLOT to group the two genders together and not stack them. I have tried coding this way but to no avail:
proc sgplot data = a1q1;
vbar wbc / group= gender response =wbc stat=freq nostatlabel;
run;
How would I go about coding to get the output I need?
Thank you for your time!
Sounds like you should use SGPANEL, not SGPLOT. SGPLOT can make grouped bar charts, but not automatically make histogram bins without using a format (you could do that if you want) and doesn't support group with the histogram plot. However, SGPANEL can handle that.
proc sgpanel data=a1q1;
panelby gender;
histogram wbc;
run;
For example, I have a matrix M of size 10x10 and I have a column matrix ind of length 5
How can I assign A(ind,:) to a new matrix B in C++ with OpenCV?
Below is how I do in Matlab:
A = [ 41 8 33 36 22 14 38 43 18 4
46 49 2 2 20 34 13 13 42 3
7 48 43 14 39 33 26 41 30 27
46 25 47 3 40 9 35 13 28 39
32 41 34 5 10 6 45 47 46 47
5 8 38 42 25 25 48 18 15 7
14 22 38 35 23 48 28 10 38 29
28 46 20 16 33 18 7 13 38 24
48 40 33 48 36 30 8 31 20 1
49 48 9 2 38 12 13 24 29 17]
ind = [2; 8; 4; 6; 2]
B = A(ind, :);
B = [ 46 49 2 2 20 34 13 13 42 3
28 46 20 16 33 18 7 13 38 24
46 25 47 3 40 9 35 13 28 39
5 8 38 42 25 25 48 18 15 7
46 49 2 2 20 34 13 13 42 3]
Can anyone tell me how to do this in C++ with OpenCV without using for loop
There is no direct way to extract a random ordering of rows/columns without iterating in some way. The simplest method is to extract rows and push them into the target matrix one by one. Given you have your matrix A declared and its data set:
cv::Mat B;
B.push_back(A(cv::Range(2,3),cv::Range::all()));
B.push_back(A(cv::Range(8,9),cv::Range::all()));
B.push_back(A(cv::Range(4,5),cv::Range::all()));
B.push_back(A(cv::Range(6,7),cv::Range::all()));
B.push_back(A(cv::Range(2,3),cv::Range::all()));
should do what you want. This uses the overloaded operator()(cv::rowRange, cv::colRange) to extract the selected rows.
I don't think this is possible without using for loop but the fastest way of doing this is by using memcpy. You can see the complete code here