Using PRXMATCH to match strings from another sas dataset - sas

Question

Need your assistance and guidance. Please see below
*rsubmit;proc sql;
connect to teradata(user=&user_id. password=&user_pwd.);
create table mylib.DWH_table as select * from connection to teradata(
select distinct nm from DWH_table
);
quit;*endrsubmit;
*rsubmit;
DATA mylib.out_sas1;
set mylib.DWH_table;
if prxmatch ("m/studio/i",nm) > 0;
run;*endrsubmit;
So the above code checks for the word "studio" in the column nm and returns the results. However, this is a manual process that needs to be automated. I have another dataset that contains just one column named "KEYWORDS". Some of the sample data I have given below
KEYWORDS:
apple
mango
banana
grapes
The goal is that SAS should take the word in the column and compare it to the value in the database and create a separate output table.
So for example:
*rsubmit;
DATA mylib.out_sas2;
set mylib.DWH_table;
if prxmatch ("m/apple/i",nm) > 0;
run;*endrsubmit;
*rsubmit;
DATA mylib.out_sas3;
set mylib.DWH_table;
if prxmatch ("m/mango/i",nm) > 0;
run;*endrsubmit;
Can this be done in SAS?

Answer 1

Put your keywords in macro vairables
proc sql;
select count(distinct KEYWORDS)
into :no_keys
from mylib.MY_KEYWORDS;
select distinct KEYWORDS
into :key_1-key_&no_keys
from mylib.MY_KEYWORDS;
quit;
Now use those macro variables
%macro find_keywords;
data
%do key_nr = 1 %to &no_keys;
mylib.out_sas&key_nr (drop = UP_nm)
%end;
;
set mylib.DWH_table;
UP_nm : upcase(nm);
%do key_nr = 1 %to &no_keys;
keyword = "&key.";
if prxmatch ("m/&&key_&key_nr/i",UP_nm) > 0 then output out_sas&key_nr;
%end;
run;
%mend;
%find_keywords;
You need to embed this in a macro, because you cannot use %do ... %end; in "open" code. && resolves to &, which makes it a delayed &, that is resolved after resolving &key_nr.
Disclaimer: this code is not tested. If you have trouble getting it running, please respond.

Answer 2

Consider a macro call via a data step using CALL EXECUTE:
%macro subset_data(key);
%let name_unquoted = %qsysfunc(compress(&key., %str(%")));
data mylib.out_&name_unquoted.;
set mylib.DWH_table;
if prxmatch ("m/"||trim(&key.)||"/i",nm) > 0;
run;
%mend;
data _null_;
set mydata;
call execute('%nrstr(%subset_data("'||KEYWORDS||'"))');
run;
Alternatively, instead of call execute, create a SAS script file of macro calls, then run with %include:
data _null_;
set mydata;
file "Temp.sas" ;
put '%subset_data("' KEYWORDS '") ;' ;
run;
%include "Temp.sas";
But if keywords are many (i.e., tens to hundreds to thousands), consider #Richard's comment above to develop an indicator column in a concatenated dataset via a helper, temp dataset:
%macro subset_data(key);
*** BUILD temp WITH INDICATOR;
data temp;
set mylib.DWH_table;
if prxmatch ("m/"||trim(&key.)||"/i",nm) > 0;
keyword = &key.;
run;
*** CONCATENATE temp;
data mylib.subset_data;
set mylib.subset_data
temp;
run;
%mend;
Reproducible Example (using sashelp.class dataset)
proc contents data = sashelp.class; run;
%macro subset_data(key);
%let name_unquoted = %qsysfunc(compress(&key.,%str(%")));
data &name_unquoted.;
set sashelp.class;
if prxmatch("m/"||trim(&key.)||"/i", Name) > 0;
run;
%mend;
data keywords;
input id keyword $;
datalines;
1 w
2 u
3 y
;
data _null_;
set keywords;
call execute('%nrstr(%subset_data("'||keyword||'"))');
run;
proc sql version
%macro subset_data(key);
%let name_unquoted = %qsysfunc(compress(&key., %str(%")));
proc sql;
create table &name_unquoted. as
select * from mylib.DWH_table
where nm like "%" || trim(&key.) || "%";
-- where nm index(nm, trim(&key.)) > 0;
quit;
%mend;
proc sql (with SAS## datasets)
data keywords;
set keywords;
dname = cat("", "sas", _n_);
run;
%macro subset_data(key, dname);
%let name_unquoted = %qsysfunc(compress(&dname.,%str(%")));
proc sql;
create table &name_unquoted. as
select * from mylib.DWH_table
where nm like "%" || trim(&key.) || "%";
-- where nm index(nm, trim(&key.)) > 0;
quit;
%mend;
data _null_;
set keywords;
call execute('%nrstr(%subset_data("'||keyword||'", "'||dname||'"))');
run;

Answer 3

One idea is to perform an cross join on an is match criteria. The result is one table with one row per name noun match.
Sample data and code:
data names;
length name $80;
infile cards length=L;
input name $varying. L;
datalines;
Bob
Bob's Burgers
Angel
Angle iron city
Chad
Chadwicks town council
Dutch
Edward
run;
data nouns;
length noun $10;
infile cards length=L;
input noun $varying. L;
datalines;
chad
own
ward
burger
run;
/*
* might want to pre lowercase the data being matched up
data lower_names;
set names;
lower_name = lower(name);
data lower_nouns;
lower_noun = lower(noun);
run;
*/
proc sql;
create table want as
select name, noun
from names as NAME
cross join nouns as NOUN
where index(lowcase(NAME),lowcase(trim(NOUN))) >= 1 /* SAS INDEX() result: 1 or higher means noun is present */
;
quit;
Regardless of your approach there will be a lot of activity. Suppose there are 100 nouns to be checked against all names, that's 26M names x 100 nouns = 2.6B is match evaluations. The system that is the most powerful and resource available will usually get you the fastest answer.
Case 1: SAS installation better
Download names to SAS
cross join names to nouns in SAS
Case 2: Teradata installation is better
Upload nouns to Teradata
cross join names to nouns in Teradata (via passthrough SQL)
Case 1 code:
Proc SQL;
connect to (user=&user_id. password=&user_pwd.);
* download names;
create table mylib.DWH_names as
select * from connection to Teradata (
select distinct nm from DWH_table
);
create table work.NameNounMatches as
select
nm,
noun
from
mylib.dwh_names as NAMES
cross join
mylib.nouns as NOUNS
where
INDEX(lowcase(NAMES.nm),lowcase(trim(NOUNS.noun))) >= 1
;
Case 2 code:
Teradata temp table -- Upload (connection=global) from Tom on https://communities.sas.com/t5/SAS-Enterprise-Guide/SAS-Access-to-Teradata-How-to-create-Temporary-tables-in/td-p/228852
libname tdwork teradata username=&username password=&password server=&server
connection=global dbmstemp=yes
;
data tdwork.NOUNS_UPLOADED;
set mylib.nouns;
run;
* cross join in Teradata via passthrough;
proc sql;
connect using tdwork;
create table work.NameNounMatches as
select * from connection to tdwork
( select Cust.UNIQUE_ID,IP.IP_NAME
from TABLE_DWH as NAMES_LIST
cross join NOUNS_UPLOADED as NOUNS_LIST
where POSITION(NAMES_LIST.nm,NOUNS_LIST.noun) >= 1
);
quit;