I created a SAS function using fcmp to calculate the jaccard distance between two strings. I do not want to use macros, as I'm going to use it through a large dataset for multiples variables. the substrings I have are missing others.
proc fcmp outlib=work.functions.func;
function distance_jaccard(string1 $, string2 $);
n = length(string1);
m = length(string2);
ngrams1 = "";
do i = 1 to (n-1);
ngrams1 = cats(ngrams1, substr(string1, i, 2) || '*');
end;
/*ngrams1= ngrams1||'*';*/
put ngrams1=;
ngrams2 = "";
do j = 1 to (m-1);
ngrams2 = cats(ngrams2, substr(string2, j, 2) || '*');
end;
endsub;
options cmplib=(work.functions);
data test;
string1 = "joubrel";
string2 = "farjoubrel";
jaccard_distance = distance_jaccard(string1, string2);
run;
I expected ngrams1 and ngrams2 to contain all the substrings of length 2 instead I got this
ngrams1=jo*ou*ub
ngrams2=fa*ar*rj
If you want real help with your algorithm you need to explain in words what you want to do.
I suspect your problem is that you never defined how long you new character variables NGRAM1 and NGRAM2 should be. From the output you show it appears that FCMP defaulted them to length $8.
To define a variable you need use a LENGTH statement (or an ATTRIB statement with the LENGTH= option) before you start referencing the variable.
I'm trying to understand how to code something along the lines of "NOT IN the LIST" type of logic in SAS.
I figured I could do "NOT" + "IN" as something like below.
Data work.OUT;
Set work.IN;
If VAR=1 then OUTPUT=1;
else if VAR=2 then OUTPUT=2;
else if VAR NOT in (1,2) then OUTPUT=3;
else OUTPUT=4;
run;
When I export the dataset all I see is OUTPUT=3 for all records. So something is happening in the derivation and it's transforming all VAR values into OUTPUT 3 values for some reason. Even though I know for a fact that other values exist in the VAR.
I don't understand what the problem is? Can we not combine NOT+IN operators? Alternatively, do you have any other ways of coding this type of logic in SAS? I rather not code each bit of code since I have more than 300 unique values for VAR
Welcome to Stack Overflow Alejandro. Your code assigns values 1 2 or 3 depending on what values are in the variable called var:
data in;
do var = 1 to 5;
output;
end;
run;
Data work.OUT;
set work.IN;
If VAR=1 then OUTPUT=1;
else if VAR=2 then OUTPUT=2;
else if VAR NOT in (1,2) then OUTPUT=3;
else OUTPUT=4;
run;
Your code says check for var = 1 then check for var = 2 and then check if it is not 1 or 2. The final else is never checked because a var will be 1 or 2 or not 1 or 2.
If you have a pile of if checks, you can use a select/when/otherwise/end block. It will check a series of rules (in the order you type them) and then will do something based on whichever rule is true first.
data out;
set in;
select;
when(var = 1) output = 1;
when(var = 2) output = 2;
when(var < 5) output = 3;
when(.) output = -9999999;
otherwise output = 42;
end;
run;
I hope that helps. If not please send up another flare.
Variable name is PRC. This is what I have so far. First block to delete negative values. Second block is to delete missing values.
data work.crspselected;
set work.crspraw;
where crspyear=2016;
if (PRC < 0)
then delete;
where ticker = 'SKYW';
run;
data work.crspselected;
set work.crspraw;
where ticker = 'SKYW';
where crspyear=2016;
where=(PRC ne .) ;
run;
Instead of using a function to remove negative and missing values, it can be done more simply when inputting or outputting the data. It can also be done with only one data step:
data work.crspselected;
set work.crspraw(where = (PRC >= 0 & PRC ^= .)); * delete values that are negative and missing;
where crspyear = 2016;
where ticker = 'SKYW';
run;
The section that does it is:
(where = (PRC >= 0 & PRC ^= .))
Which can be done for either the input dataset (work.crspraw) or the output dataset (work.crspselected).
If you must use a function, then the function missing() includes only missing values as per this answer. Hence ^missing() would do the opposite and include only non-missing values. There is not a function for non-negative values. But I think it's easier and quicker to do both together simultaneously without a function.
You don't need more than your first test to remove negative and missing values. SAS treats all 28 missing values (., ._, .A ... .Z) as less than any actual number.
I have a column of format DATETIME23. like this:
14.02.2017 13:00:25
I want to conver it to a string, so later, i would be able to modern it, so, for example, the final version would look like:
2017-02-14 13:00:25.000
Problem occures, when i try to convert date to char format: in result i have a string of smth like 1802700293 - which is the number of seconds.
I tried:
format date $23.0
or
date = put(date, $23.0)
P.S This is nother try:
data a;
format d date9.;
d = '12jan2016'd;
dtms = cat(day(d),'-',month(d),'-',year(d),' 00:00:00.000');
/* если нужно обязательно двухзначные день и месяц, то такой колхоз: */
if day(d) < 10 then dd=cat('0',put(day(d),$1.));
else ddday=put(day(d),$2.);
if month(d) < 10 then mm=cat('0',put(month(d),$1.));
else mm=put(month(d),$2.);
yyyy=put(year(d),$4.);
/*dtms2 = cat(dd,'-',mm,'-',yyyy,' 00:00:00.000');*/
dtms2 = cat(dd,'-',mm,'-',yyyy,' 00:00:00.000');
dtms = cat(day(d),'-',month(d),'-',year(d),' 00:00:00.000');
run;
BUT, abnormally, the dtms2 concat destroys the zero in the month element
If your datetime is stored as a SAS datetime, just use the appropriate format :
data test ;
dt = '09feb2017:13:53:26'dt ; /* specify a datetime constant */
new_dt = put(dt,E8601DT23.3) ; /* ISO datetime format */
run ;
Output
dt new_dt
1802267606 2017-02-09T13:53:26.000
If you need to replace the 'T' with a space, simply add a translate function around the put().
For your dtms solution you can use put and the Z2. format to keep the leading zero when you concatenate:
dtms = cat(day(d),'-', put(month(d),z2.),'-',year(d),' 00:00:00.000');
You should be able to just use put(date, datetime23.) for your problem though instead of $23, which is converting the number of seconds to a string with length 23. However, as a comment has mentioned datetime23. is not the format from your example.
I have a dataset with string variables and I am trying to generate a new binary variable based on the first two characters. All strings are 5 characters long, but I'm only concerned with the first two in order to sort.
For example, I could have 22001 and 22005. Since both are of the form 22XXX, I want to assign value 1 for both in the variable type_A. And if I have 25001 and 25005, since both are not of the form 22XXX, I want to assign value 0 for both in the variable type_A.
This should do the job:
clear
set obs 4
generate str5 var1 = "22001" in 1
replace var1 = "22005" in 2
replace var1 = "25001" in 3
replace var1 = "25005" in 4
gen type_A = substr(var1, 1, 2) == "22"
Please note that as you explain your problem it looks like you you are storing 22005 as text - which may not necessarily be the best idea..