Is there a SAS function to delete negative and missing values from a variable in a dataset? - sas

Variable name is PRC. This is what I have so far. First block to delete negative values. Second block is to delete missing values.
data work.crspselected;
set work.crspraw;
where crspyear=2016;
if (PRC < 0)
then delete;
where ticker = 'SKYW';
run;
data work.crspselected;
set work.crspraw;
where ticker = 'SKYW';
where crspyear=2016;
where=(PRC ne .) ;
run;

Instead of using a function to remove negative and missing values, it can be done more simply when inputting or outputting the data. It can also be done with only one data step:
data work.crspselected;
set work.crspraw(where = (PRC >= 0 & PRC ^= .)); * delete values that are negative and missing;
where crspyear = 2016;
where ticker = 'SKYW';
run;
The section that does it is:
(where = (PRC >= 0 & PRC ^= .))
Which can be done for either the input dataset (work.crspraw) or the output dataset (work.crspselected).
If you must use a function, then the function missing() includes only missing values as per this answer. Hence ^missing() would do the opposite and include only non-missing values. There is not a function for non-negative values. But I think it's easier and quicker to do both together simultaneously without a function.

You don't need more than your first test to remove negative and missing values. SAS treats all 28 missing values (., ._, .A ... .Z) as less than any actual number.

Related

How to correct this sas function in order to have the jaccard distance?

I created a SAS function using fcmp to calculate the jaccard distance between two strings. I do not want to use macros, as I'm going to use it through a large dataset for multiples variables. the substrings I have are missing others.
proc fcmp outlib=work.functions.func;
function distance_jaccard(string1 $, string2 $);
n = length(string1);
m = length(string2);
ngrams1 = "";
do i = 1 to (n-1);
ngrams1 = cats(ngrams1, substr(string1, i, 2) || '*');
end;
/*ngrams1= ngrams1||'*';*/
put ngrams1=;
ngrams2 = "";
do j = 1 to (m-1);
ngrams2 = cats(ngrams2, substr(string2, j, 2) || '*');
end;
endsub;
options cmplib=(work.functions);
data test;
string1 = "joubrel";
string2 = "farjoubrel";
jaccard_distance = distance_jaccard(string1, string2);
run;
I expected ngrams1 and ngrams2 to contain all the substrings of length 2 instead I got this
ngrams1=jo*ou*ub
ngrams2=fa*ar*rj
If you want real help with your algorithm you need to explain in words what you want to do.
I suspect your problem is that you never defined how long you new character variables NGRAM1 and NGRAM2 should be. From the output you show it appears that FCMP defaulted them to length $8.
To define a variable you need use a LENGTH statement (or an ATTRIB statement with the LENGTH= option) before you start referencing the variable.

NOT+IN SAS operators combined, is this valid? Can't find documentation

I'm trying to understand how to code something along the lines of "NOT IN the LIST" type of logic in SAS.
I figured I could do "NOT" + "IN" as something like below.
Data work.OUT;
Set work.IN;
If VAR=1 then OUTPUT=1;
else if VAR=2 then OUTPUT=2;
else if VAR NOT in (1,2) then OUTPUT=3;
else OUTPUT=4;
run;
When I export the dataset all I see is OUTPUT=3 for all records. So something is happening in the derivation and it's transforming all VAR values into OUTPUT 3 values for some reason. Even though I know for a fact that other values exist in the VAR.
I don't understand what the problem is? Can we not combine NOT+IN operators? Alternatively, do you have any other ways of coding this type of logic in SAS? I rather not code each bit of code since I have more than 300 unique values for VAR
Welcome to Stack Overflow Alejandro. Your code assigns values 1 2 or 3 depending on what values are in the variable called var:
data in;
do var = 1 to 5;
output;
end;
run;
Data work.OUT;
set work.IN;
If VAR=1 then OUTPUT=1;
else if VAR=2 then OUTPUT=2;
else if VAR NOT in (1,2) then OUTPUT=3;
else OUTPUT=4;
run;
Your code says check for var = 1 then check for var = 2 and then check if it is not 1 or 2. The final else is never checked because a var will be 1 or 2 or not 1 or 2.
If you have a pile of if checks, you can use a select/when/otherwise/end block. It will check a series of rules (in the order you type them) and then will do something based on whichever rule is true first.
data out;
set in;
select;
when(var = 1) output = 1;
when(var = 2) output = 2;
when(var < 5) output = 3;
when(.) output = -9999999;
otherwise output = 42;
end;
run;
I hope that helps. If not please send up another flare.

How to format dates for use in SAS?

I am trying to adapt Method 4 in this paper to calculate the duration of many observations, but discounting overlapping dates: https://support.sas.com/resources/papers/proceedings/proceedings/sugi31/048-31.pdf
For example, two rows of observations for subject 101 lasting from 2017-03-02 to 2017-03-16 and 2017-03-04 to 2017-03-17 respectively should return a value of only 16 days.
I am getting an error with the dates being 'Invalid numeric data', though, resulting in later errors. I have tried format startdate yyyymmdd10.; and format stopdate yyyymmdd10.; with no success.
Can anyone help me properly format my dates for use here, or identify any further errors?
Edit: Line 80 refers to do xdate = startdate to stopdate;.
I am still unable to convert or create the date variables as numeric/date values. I have used the following code:
data sasuser.Mdm;
set sasuser.Mdm;
do xdate = input(Startdate,yymmdd10.) to input(stopdate,yymmdd10.);
put xdate= yymmdd10.;
output;
end;
run;
To get this output:
1 data sasuser.Mdm;
2 set sasuser.Mdm;
3 do xdate = input(Startdate,yymmdd10.) to input(stopdate,yymmdd10.);
4 put xdate= yymmdd10.;
5 output;
6 end;
7 run;
xdate=2017-03-02
xdate=2017-03-03
xdate=2017-03-04
xdate=2017-03-05
xdate=2017-03-06
xdate=2017-03-07
xdate=2017-03-08
xdate=2017-03-09
xdate=2017-03-10
xdate=2017-03-11
xdate=2017-03-12
xdate=2017-03-13
xdate=2017-03-14
xdate=2017-03-15
xdate=2017-03-16
xdate=2017-03-04
xdate=2017-03-05
xdate=2017-03-06
xdate=2017-03-07
xdate=2017-03-08
xdate=2017-03-09
xdate=2017-03-10
xdate=2017-03-11
xdate=2017-03-12
xdate=2017-03-13
xdate=2017-03-14
xdate=2017-03-15
xdate=2017-03-16
xdate=2017-03-17
xdate=2017-03-07
xdate=2017-03-08
xdate=2017-03-09
xdate=2017-03-10
xdate=2017-03-11
xdate=2017-03-12
xdate=2017-03-13
xdate=2017-03-14
xdate=2017-03-15
xdate=2017-03-16
xdate=2017-03-17
xdate=2017-03-18
xdate=2017-03-19
xdate=2017-03-20
xdate=2017-03-21
xdate=2017-02-08
xdate=2017-02-09
xdate=2017-02-10
xdate=2017-02-11
xdate=2017-02-12
xdate=2017-02-13
xdate=2017-02-14
xdate=2017-02-15
xdate=2017-02-16
xdate=2017-02-17
xdate=2017-02-18
xdate=2017-02-19
xdate=2017-02-20
xdate=2017-02-21
xdate=2017-02-22
xdate=2017-02-23
xdate=2017-02-24
xdate=2017-02-23
xdate=2017-02-24
xdate=2017-02-25
xdate=2017-02-26
xdate=2017-02-27
xdate=2017-02-28
xdate=2017-03-01
xdate=2017-03-02
xdate=2017-03-03
xdate=2017-03-04
xdate=2017-03-05
xdate=2017-03-06
xdate=2017-03-07
xdate=2017-03-08
xdate=2017-02-26
xdate=2017-02-28
xdate=2017-03-13
xdate=2017-03-17
xdate=2017-03-25
xdate=2017-03-28
xdate=2017-03-23
xdate=2017-03-24
xdate=2017-03-25
xdate=2017-03-26
xdate=2017-03-27
xdate=2017-03-28
xdate=2017-03-29
xdate=2017-03-30
xdate=2017-03-29
xdate=2017-04-03
xdate=2017-04-04
xdate=2017-04-03
xdate=2017-04-04
xdate=2017-04-05
xdate=2017-04-05
xdate=2017-04-06
xdate=2017-04-06
xdate=2017-04-07
xdate=2017-03-25
xdate=2017-03-26
xdate=2017-03-30
xdate=2017-04-01
xdate=2017-04-02
xdate=2017-04-03
xdate=2017-04-04
xdate=2017-04-08
xdate=2017-04-09
xdate=2017-04-10
xdate=2017-04-11
xdate=2017-04-12
xdate=2017-04-12
xdate=2017-04-13
xdate=2017-04-13
xdate=2017-04-14
xdate=2017-04-15
xdate=2017-04-16
xdate=2017-04-17
xdate=2017-04-18
xdate=2017-04-19
xdate=2017-04-20
xdate=2017-04-21
xdate=2017-04-22
xdate=2017-04-19
xdate=2017-04-23
xdate=2017-04-24
xdate=2017-04-25
xdate=2017-04-26
xdate=2017-04-26
xdate=2017-04-27
xdate=2017-04-28
xdate=2017-05-05
xdate=2017-05-06
xdate=2017-05-16
xdate=2017-05-19
xdate=2017-05-20
xdate=2017-05-21
xdate=2017-05-22
xdate=2017-05-19
xdate=2017-05-20
xdate=2017-05-21
xdate=2017-05-22
xdate=2017-05-23
xdate=2017-05-24
xdate=2017-05-25
xdate=2017-05-26
xdate=2017-05-22
xdate=2017-05-23
xdate=2017-05-24
xdate=2017-05-25
xdate=2017-05-26
xdate=2017-05-27
xdate=2017-05-28
xdate=2017-05-29
xdate=2017-05-30
xdate=2017-05-31
xdate=2017-06-01
xdate=2017-06-02
xdate=2017-06-03
xdate=2017-06-04
xdate=2017-06-05
xdate=2017-06-06
xdate=2017-06-07
xdate=2017-06-08
xdate=2017-06-09
xdate=2017-06-10
xdate=2017-06-11
xdate=2017-06-12
xdate=2017-06-13
xdate=2017-06-14
xdate=2017-06-15
xdate=2017-06-16
xdate=2017-06-17
xdate=2017-06-18
xdate=2017-06-19
xdate=2017-06-20
xdate=2017-06-21
xdate=2017-06-22
xdate=2017-06-23
xdate=2017-06-24
xdate=2017-06-25
xdate=2017-06-26
xdate=2017-06-27
xdate=2017-06-28
xdate=2017-06-29
xdate=2017-06-30
xdate=2017-07-01
xdate=2017-07-02
xdate=2017-07-03
xdate=2017-07-04
xdate=2017-07-05
xdate=2017-07-06
xdate=2017-07-07
xdate=2017-07-08
xdate=2017-07-09
xdate=2017-07-10
xdate=2017-07-11
xdate=2017-07-12
xdate=2017-07-13
xdate=2017-07-14
xdate=2017-07-15
xdate=2017-07-16
xdate=2017-07-17
xdate=2017-07-18
xdate=2017-07-19
xdate=2017-07-20
xdate=2017-07-21
xdate=2017-07-22
xdate=2017-07-23
xdate=2017-07-24
xdate=2017-07-25
xdate=2017-07-26
xdate=2017-07-27
xdate=2017-07-28
xdate=2017-07-29
xdate=2017-07-30
xdate=2017-07-31
xdate=2017-08-01
xdate=2017-08-02
xdate=2017-08-03
xdate=2017-08-04
xdate=2017-08-05
xdate=2017-08-06
xdate=2017-08-07
xdate=2017-08-08
xdate=2017-08-09
xdate=2017-08-10
xdate=2017-08-11
xdate=2017-08-12
xdate=2017-08-13
xdate=2017-08-14
xdate=2017-08-15
xdate=2017-08-16
xdate=2017-08-17
xdate=2017-08-18
xdate=2017-08-19
xdate=2017-08-20
xdate=2017-08-21
xdate=2017-08-22
xdate=2017-08-23
xdate=2017-08-24
xdate=2017-08-25
xdate=2017-08-26
xdate=2017-08-27
xdate=2017-08-28
xdate=2017-08-29
xdate=2017-08-30
xdate=2017-08-31
xdate=2017-09-01
xdate=2017-05-27
xdate=2017-05-28
xdate=2017-05-29
xdate=2017-05-30
xdate=2017-05-31
xdate=2017-06-01
xdate=2017-06-02
xdate=2017-06-03
xdate=2017-06-04
xdate=2017-06-05
xdate=2017-06-06
xdate=2017-06-07
xdate=2017-06-08
xdate=2017-06-09
xdate=2017-06-10
xdate=2017-06-11
xdate=2017-06-12
xdate=2017-06-13
xdate=2017-06-14
xdate=2017-06-15
xdate=2017-06-16
xdate=2017-06-17
xdate=2017-06-18
xdate=2017-06-19
xdate=2017-06-20
xdate=2017-06-21
xdate=2017-06-22
xdate=2017-06-23
xdate=2017-06-24
xdate=2017-06-25
xdate=2017-06-26
xdate=2017-06-27
xdate=2017-06-28
xdate=2017-06-29
xdate=2017-06-30
xdate=2017-07-01
xdate=2017-07-02
xdate=2017-07-03
xdate=2017-07-04
xdate=2017-07-05
xdate=2017-07-06
xdate=2017-07-07
xdate=2017-07-08
xdate=2017-07-09
xdate=2017-07-10
xdate=2017-07-11
xdate=2017-07-12
xdate=2017-07-13
xdate=2017-07-14
xdate=2017-07-15
xdate=2017-07-16
xdate=2017-07-17
xdate=2017-07-18
xdate=2017-07-19
xdate=2017-07-20
xdate=2017-07-21
xdate=2017-07-22
xdate=2017-07-23
xdate=2017-07-24
xdate=2017-07-25
xdate=2017-07-26
xdate=2017-07-27
xdate=2017-07-28
xdate=2017-07-29
xdate=2017-07-30
xdate=2017-07-31
xdate=2017-08-01
xdate=2017-08-02
xdate=2017-08-03
xdate=2017-08-04
xdate=2017-08-05
xdate=2017-08-06
xdate=2017-08-07
xdate=2017-08-08
xdate=2017-08-09
xdate=2017-08-10
xdate=2017-08-11
xdate=2017-08-12
xdate=2017-08-13
xdate=2017-08-14
xdate=2017-08-15
xdate=2017-08-16
xdate=2017-08-17
xdate=2017-08-18
xdate=2017-08-19
xdate=2017-08-20
xdate=2017-08-21
xdate=2017-08-22
xdate=2017-08-23
xdate=2017-08-24
xdate=2017-08-25
xdate=2017-08-26
xdate=2017-08-27
xdate=2017-08-28
xdate=2017-08-29
xdate=2017-08-30
xdate=2017-08-31
xdate=2017-09-01
xdate=2017-06-14
xdate=2017-06-15
xdate=2017-06-16
xdate=2017-06-17
xdate=2017-06-18
xdate=2017-06-19
xdate=2017-06-20
xdate=2017-06-21
xdate=2017-06-22
xdate=2017-06-23
xdate=2017-06-24
xdate=2017-06-25
xdate=2017-06-26
xdate=2017-06-27
xdate=2017-06-28
xdate=2017-06-29
xdate=2017-06-14
xdate=2017-06-15
xdate=2017-06-16
xdate=2017-06-17
xdate=2017-06-18
xdate=2017-06-19
xdate=2017-06-20
xdate=2017-06-21
xdate=2017-06-22
xdate=2017-06-23
xdate=2017-06-24
xdate=2017-06-25
xdate=2017-06-26
xdate=2017-06-27
xdate=2017-06-28
xdate=2017-06-29
xdate=2017-03-27
xdate=2017-04-02
xdate=2017-04-07
xdate=2017-04-08
xdate=2017-04-09
xdate=2017-04-13
xdate=2017-04-14
xdate=2017-04-15
xdate=2017-04-16
xdate=2017-04-17
xdate=2017-04-19
xdate=2017-04-20
xdate=2017-04-21
xdate=2017-04-22
xdate=2017-04-23
xdate=2017-04-24
xdate=2017-04-20
xdate=2017-04-21
xdate=2017-04-22
xdate=2017-04-23
xdate=2017-04-24
xdate=2017-04-25
xdate=2017-04-26
xdate=2017-04-27
xdate=2017-04-28
xdate=2017-04-29
xdate=2017-04-30
xdate=2017-05-01
xdate=2017-05-02
xdate=2017-04-24
xdate=2017-04-25
xdate=2017-04-26
xdate=2017-04-27
xdate=2017-04-28
xdate=2017-04-29
xdate=2017-04-30
xdate=2017-05-01
xdate=2017-05-02
xdate=2017-05-03
xdate=2017-05-04
xdate=2017-05-05
xdate=2017-05-06
xdate=2017-05-07
xdate=2017-05-08
xdate=2017-05-09
xdate=2017-05-10
xdate=2017-05-11
xdate=2017-05-12
xdate=2017-05-13
xdate=2017-05-14
xdate=2017-05-15
xdate=2017-05-16
ERROR: Invalid DO loop control information, either the INITIAL or TO expression is missing or the
BY expression is missing, zero, or invalid.
SUBJID=106 KEY=106-9 OBS=9 TOTAL=12 STARTDATE=2017-04-25 STOPDATE= CLASS=Steroid / Diuretic
xdate=20934 _ERROR_=1 _N_=52
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 52 observations read from the data set SASUSER.MDM.
WARNING: The data set SASUSER.MDM may be incomplete. When this step was stopped there were 431
observations and 8 variables.
WARNING: Data set SASUSER.MDM was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
real time 0.38 seconds
cpu time 0.29 seconds```
I don't understand why input doesn't appear to be working. Dates are still listed as character strings under column attributes. The do part also isn't working as intended. I'd be grateful for any further guidance.
Do not use the same name in the DATA and SET statement. Then you're always having to rebuild from the start.
Convert your start and stop date to SAS dates
Remove PUT
Add formats to see them displayed as desired
Drop old variables to avoid confusion.
Your two code steps, the data step and SQL do not appear related. Not sure why you would even need a list of dates for intervals or anything. There are much better ways to calculate an overlap. I think you're putting us through an xy problem where it would be significantly easier to show us what you're attempting to do and people would be able to provide a much better solution.
data sasuser.Mdm2; /*1*/
set sasuser.Mdm;
/*2*/
start_date = input(startdate, yymmdd10.);
end_date = input(stopdate, yymmdd10.);
do xdate = start_date to stop_date;
output; /*3*/
end;
/*4*/
format start_date end_date xDate yymmdd10.;
/*5*/
drop startdate stopdate;
run;
*check;
proc contents data=sasuser.mdm2;
run;
EDIT: Also, if you had some sort of grouping variable to indicate that these were part of the same episode you could then just take the min/max of the dates and subtract them to get the interval duration for starters. Grouping via a data step is trivial.
data want;
set have;
by id;
retain episode;
start_date = input(start_date, yymmdd10.);
end_date = input(stopdate, yymmdd10.);
prev_stop_date = lag(stopDate);
if first.id then do;
episode = 0;
call missing(prev_stop_date);
end;
if not (start_date <=prev_stop_date <= end_date) then episode+1;
*could add in logic to calculate dates and durations as well depending....;
run;
It sounds like your SAS log is complaining about this statement.
do xdate=startdate to stopdate;
Because STARTDATE and STOPDATE are character strings instead of dates.
Make sure to create your date values as dates instead of character strings.
Tom's correct, of course, the startdate and stopdate seem to be characters.
To properly use this, do something like this (only the do loop is relevant for you, the rest is to show it working):
data _null_;
startdate = '2017-03-02';
stopdate = '2017-03-16';
do xdate = input(Startdate,yymmdd10.) to input(stopdate,yymmdd10.);
put xdate= yymmdd10.; *just put to the log to see what you are getting;
end;
run;
input will convert the text to a numeric value. Do realize you have to format that xdate as a date format if you want to be able to view it - if you're just using it as an input, though, you can leave the formatting off.

SAS refer to a single digit out of a whole number

Do I have a way to refer a value in the following way =>
value=23032017 and I want to refer to the digit in the forth place of this number, i.e 2
If it is a number then use some simple arithmetic.
data _null_;
value=23032017 ;
do position=1 to 9;
digit = mod(int(value/10**(position-1)),10);
put position= digit= ;
end;
run;
position=1 digit=7
position=2 digit=1
position=3 digit=0
position=4 digit=2
position=5 digit=3
position=6 digit=0
position=7 digit=3
position=8 digit=2
position=9 digit=0

Error surrounding use of scan(&varlist) + Comparison of macro variables

As a follow up to this question, for which my existing answer appears to be best:
Extracting sub-data from a SAS dataset & applying to a different dataset
Given a dataset dsn_in, I currently have a set of macro variables max_1 - max_N that contain numeric data. I also have a macro variable varlist containing a list of variables. The two sets of macros are related such that max_1 is associated with scan(&varlist, 1), etc. I am trying to do compare the data values within dsn_in for each variable in varlist to the associated comparison values max_1 - max_N. I would like to output the updated data to dsn_out. Here is what I have so far:
data dsn_out;
set dsn_in;
/* scan list of variables and compare to decision criteria.
if > decision criteria, cap variable */
do i = 1 by 1 while(scan(&varlist, i) ~= '');
if scan("&varlist.", i) > input(symget('max_' || left(put(i, 2.))), best12.) then
scan("&varlist.", i) = input(symget('max_' || left(put(i, 2.))), best12.);
end;
run;
However, I'm getting the following error, which I don't understand. options mprint; shown. SAS appears to be interpreting scan as both an array and a variable, when it's a SAS function.
ERROR: Undeclared array referenced: scan.
MPRINT(OUTLIERS_MAX): if scan("var1 var2 var3 ... varN", i) > input(symget('max_'
|| left(put(i, 2.))), best12.) then scan("var1 var2 var3 ... varN", i) =
input(symget('max_' || left(put(i, 2.))), best12.);
ERROR: Variable scan has not been declared as an array.
MPRINT(OUTLIERS_MAX): end;
MPRINT(OUTLIERS_MAX): run;
Any help you can provide would be greatly appreciated.
The specific issue you have here is that you place SCAN on the left side of an equal sign. That is not allowed; SUBSTR is allowed to be used in this fashion, but not SCAN.