Say I have a dataset that consists of schoolID SAT_code and student_name. What I want to classify is that for those schoolID with 'ABC' would have same SAT_code of 'East'. While those with schoolID with 'XYZ' would have same SAT_code of 'Midwest'.
For a dataset like this:
schoolID SAT_code student_name
ABC Jasmine Smith
ABC Michael Jordan
ABC Madison Trump
XYZ Sarah Potter
XYZ Jim Fowler
XYZ Jack Black
. .
. .
. .
There are more than 30 schoolID besides that.
The easiest, but notorious one I could think of was to use if-then for 30 times.
data stateSAT;
set statestats;
if schoolID eq 'ABC' then SAT_code 'East';
else if schoolID eq 'XYZ' then SAT_code 'Midwest';
else if schoolID eq 'MNO' then SAT_code 'East';
and so forth.....
run;
Are there more efficient way, possibly having some kind of for-loop to resolve this issue?
Thanks.
Lookup table
This is best done using a lookup table:
Create a table with schoolID and SAT_code and then perform a merge with your source table:
data schoolsat;
infile datalines delimiter=',';
input schoolID $3 SAT_code $25;
datalines;
ABC,East
XYZ,Midwest
MNO,East
;
run;
This creates a table which maps the schoolID values to SAT_code values. Add all of the required combinations to this table.
Once you've done that, there are two ways of merging data in the SAS world. These are both examples of 'LEFT JOINs', which will keep all records from your statestats table, regardless of whether there's a corresponding row in your new schoolsat mapping table created above. If there's no corresponding schoolID value in schoolsat, SAS will leave a missing value for SAT_code in the resulting table.
SQL
proc sql;
create table stateSAT as
select a.*,
b.SAT_code
from statestats a
left join schoolsat b
on a.schoolID = b.schoolID;
quit;
DATA Step
proc sort data=schoolsat;
by schoolID;
run;
proc sort data=statestats;
by schoolID;
run;
data stateSAT;
merge statestats (in=a)
schoolsat;
by schoolID;
if a;
run;
Your long sequence of if/then statements can be more cleanly stated with a SELECT statement. It is similar to SQL CASE or C switch
length SAT_code $20;
select (schoolID);
when ("ABC") SAT_code = 'East';
when ("XYZ") SAT_code = 'Midwest';
when ("MNO") SAT_code = 'East';
…
otherwise SAT_code = '???';
end;
However, with 30+ schoolIDs, you would be better off following #mjsqu advice of moving those schoolID mappings to another data structure.
Related
I am having two data sets. The first data set has airport codes (JFK, LGA, EWR) in a variable 'airport'. The second dataset has the list of all major airports in the world. This dataset has two variables 'faa' holding the FAA Code (like JFG, LGA, EWR) and 'name' holding the actual name of the airport (John. F Kennedy, Le Guardia etc.).
My requirement is to create value labels for in the first data set, so that instead of airport code, the actual name of the airport comes up. I know I can use custom formats to achieve this. But can I write SAS code which can read the unique airport codes, then get the names from another data set and create a value label automatically?
PS: Other wise, the only option I see is to use MS Excel to get the unique list of FAA codes in dataset 1, and then use VLOOKUP to get the names of the airports. And then create one custom format by listing each unique FAA code and the airport name.
I think "value label" is SPSS terminology. Looks like you want to create a format. Just use your lookup table to create an input dataset for PROC FORMAT.
So if your second table looks like this:
data table2;
length FAA $4 Name $40 ;
input FAA Name $40. ;
cards;
JFK John F. Kennedy (NYC)
LGA Laguardia (NYC)
EWR Newark (NJ)
;
You can use this code to convert it into a dataset that PROC FORMAT can use to create a format.
data fmt ;
fmtname='$FAA';
hlo=' ';
set table2 (rename=(faa=start name=label));
run;
proc format cntlin=fmt lib=work.formats;
run;
Now you can use that format with your other data.
proc freq data=table1 ;
tables airport ;
format airport faa. ;
run;
Firstly, consider if it is really a format what is needed. For example, you may just do a left join to retrieve the column (airport) name from table2 (FAA-Name table).
Anyway, I believe the following macro does the trick:
Create auxiliary tables:
data have1;
input airport $;
datalines;
a
d
e
;
run;
data have2;
input faa $ name $;
datalines;
a aaaa
b bbbb
c cccc
d dddd
;
run;
Macro to create Format:
%macro create_format;
*count number of faa;
proc sql noprint;
select distinct count(faa) into:n
from have2;
quit;
*create macro variables for each faa and name;
proc sql noprint;
select faa, name
into:faa1-:faa%left(&n),:name1-:name%left(&n)
from have2;
quit;
*create format;
proc format;
value $airport
%do i=1 %to &n;
"&faa%left(&i)" = "&name%left(&i)"
%end;
other = "Unknown FAA code";
run;
%mend create_format;
%create_format;
Apply format:
data want;
set have1;
format airport $airport.;
run;
I have a dataset with many columns like this:
ID Indicator Name C1 C2 C3....C90
A 0001 Black 0 1 1.....0
B 0001 Blue 1 0 0.....1
B 0002 Blue 1 0 0.....1
Some of the IDs are duplicates because the indicator is different, but they're essentially the same record. To find duplicates, I want to select distinct ID, Name and then C1 through C90 to check because some claims who have the same Id and indicator have different C1...C90 values.
Is there a way to select c1...c90 either through proc sql or a sas data step? It seems the only way I can think of is to set the dataset and then drop the non essential columns, but in the actual dataset, it's not only Indicator but at least 15 other columns.
It would be nice if PROC SQL used the : variable name wildcard like other Procs do. When no other alternative is reasonable, I usually use a macro to select bulk columns. This might work for you:
%macro sel_C(n);
%do i=1 %to %eval(&n.-1);
C&i.,
%end;
C&n.
%mend sel_C;
proc sql;
select ID,
Indicator,
Name,
%sel_C(90)
from have_data;
quit;
If I understand the question properly, the easiest way would be to concatenate the columns to one. RETAIN that value from row to row, and you can compare it across rows to see if it's the same or not.
data want;
set have;
by id indicator;
retain last_cols;
length last_cols $500;
cols = catx('|',of c1-c90);
if first.id then call missing(last_cols);
else do;
identical = (cols = last_cols); *or whatever check you need to perform;
end;
output;
last_cols = cols;
run;
There are a few different ways you can do this and it will be much easier if the actual column names are C1 - C90. If you're just looking to remove anything that you know is a duplicate you can use proc sort.
proc sort data=dups out=nodups nodupkey;
by ID Name C1-C90;
run;
The nodupkey option will automatically remove any duplicates in the by statement.
Alternatively, if you want to know which records contain duplicates, you could use proc summary.
proc summary data=dups nway missing;
class ID Name C1-C90;
output out=onlydups(where=(_freq_ > 1));
run;
proc summary creates two new variables, _type_ and _freq_. If you specify _freq_ > 1 you will only output the duplicate records. Also, note that this will remove the Indicator variable.
The following inherited simplified code is meant to replace missing values of a column with the values of not missing entries in a group:
DATA WORK.TOYDATA;
INPUT Category $ PRICE;
DATALINES;
Cat1 2
Cat1 .
Cat1 .
Cat2 .
Cat2 3
Cat2 .
;
DATA WORK.OUTTOYDATA;
SET WORK.TOYDATA;
BY Category ;
RETAIN _PRICE;
IF FIRST.Category THEN _PRICE=PRICE;
IF NOT MISSING(PRICE) THEN _PRICE=PRICE;
ELSE PRICE=_PRICE;
DROP _PRICE;
RUN;
Unfortunately, this will not work if the first entry in a group is missing. How could this be fixed?
As SAS works row by row through the dataset there is no value to replace if the first value is missing.
You could sort the data by Category and Price DESCENDING to circumvent this.
proc sort data= WORK.TOYDATA; by Category DESCENDING PRICE; run;
Or if there is only one NON-missing value by category you could use a sql join e.g.
proc sql;
create table WORK.OUTTOYDATA as
select a.Category, coalesce(a.PRICE, b.PRICE) as PRICE
from WORK.TOYDATA a
left join (select distinct Category, PRICE
from WORK.TOYDATA
where PRICE ne .
) b
on a.Category eq b.Category
;
quit;
As #Jetzler pointed out, the easiest way is just to sort the data. However, if you have multiple columns with missing values then you'd need to do multiple sorts, which isn't efficient.
Another option from doing a join is proc stdize which can be used to replace missing values with a simple measure (mean, median, sum etc). The default method will suffice in your example, you just need to add the reponly option which only replaces missing values and does not standardize the data.
DATA WORK.TOYDATA;
INPUT Category $ PRICE;
DATALINES;
Cat1 2
Cat1 .
Cat1 .
Cat2 .
Cat2 3
Cat2 .
;
run;
proc stdize data=TOYDATA out=want reponly;
by category;
var price;
run;
Just a general question lets say I have two datasets called dataset1 and dataset2 and If I want to compare the rows of dataset1 with the complete dataset2 so essentially compare each row of dataset1 with dataset2. Below is just an example of the two datasets
Dataset1
EmployeeID Name Employeer
12345 John Microsoft
1234567 Alice SAS
1234565 Jim IBM
Dataset1
EmployeeID2 Name DateAbsent
12345 John 25/06/2009
12345 John 26/06/2009
1234567 Alice 27/06/2010
1234567 Alice 30/06/2011
1234567 Alice 2/8/2012
12345 John 28/06/2009
12345 John 25/07/2009
12345 John 25/08/2009
1234565 Jim 26/08/2009
1234565 Jim 27/08/2010
1234565 Jim 28/08/2011
1234565 Jim 29/08/2012
I have written some programming logic its not sas code, this is just my logic
for item in dataset1:
for item2 in dataset2:
if item.EmployeeID=item2.EmployeeID2 and item.Name=item2.Name then output newSet
This is an inner join.
proc sql noprint;
create table output as
select a.EmployeeId,
a.Name,
a.Employeer,
b.DateAbsent
from dataset1 as a
inner join
dataset2 as b
on a.EmployeeID = b.EmployeeID2
and a.Name = b.name;
quit;
I recommend reading the SAS documentation on PROC SQL if you are unfamiliar with the syntax
To do this in a Data step, the data sets need to be sorted by the variables to join on (or indexed). Also the variable names need to be the same, so I will assume both variables are EmployeeID.
/*sort*/
proc sort data=dataset1;
by EmployeeID Name;
run;
proc sort data=dataset2;
by EmployeeID Name;
run;
data output;
merge dataset1 (in=ds1) dataset2 (inds2);
by EmployeeID Name;
if ds1 and ds2;
run;
The data step does the loop for you. It needs sorted sets because it only takes 1 pass over the data sets. The if clause checks to make sure you are getting a value from both data sets.
Is your goal to compare the two dataset and see where there are differences? Proc Compare will do this for you. You can compare specific columns or the entire dataset.
Trying to make a more simple unique identifier from already existing identifier. Starting with just and ID column I want to make a new, more simple, id column so the final data looks like what follows. There are 1million + id's, so it isnt an option to do if thens, maybe a do statement?
ID NEWid
1234 1
3456 2
1234 1
6789 3
1234 1
A trivial data step solution not using monotonic().
proc sort data=have;
by id;
run;
data want;
set have;
by id;
if first.id then newid+1;
run;
using proc sql..
(you can probably do this without the intermediate datasets using subqueries, but sometimes monotonic doesn't act the way you'd think in a subquery)
proc sql noprint;
create table uniq_id as
select distinct id
from original
order by id
;
create table uniq_id2 as
select id, monotonic() as newid
from uniq_id
;
create table final as
select a.id, b.newid
from original_set a, uniq_id2 b
where a.id = b.id
;
quit;