I am trying to exclude procedure codes from the data:
Excludes antipsychotic meds (Long-Acting Injections) dispensed
DATA CLMSPULL_NEW2;
SET CLMSPULL_NEW;
Excludes hospice codes
DATA CLMSPULL_NEW3;
SET CLMSPULL_NEW2;
These are the codes built into macros:
/*Exclusions*/
%LET HOSPICE =
('99377','99378','G0182','G9473','G9474','G9475','G9476',
'G9477','G9478','G9479','Q5003','Q5004','Q5005','Q5006','Q5007','Q5008','Q5010','S9126','T2042',
'T2043','T2044','T2045','T2046','0143', '0144', '0149', '0150', '0151', '0152', '0153', '0154',
'0159', '0160', '0164', '0167', '0169', '0200', '0201', '0202', '0203', '0204', '0206', '0207',
'0208', '0209', '0210', '0211', '0212', '0213', '0214', '0219', '0720', '0721', '0722', '0723',
'0724', '0729', '0987', '99221','99222','99223','99231','99232','99233',
'99238','99239','99251','99252','99253','99254','99255','99291');
/*Long-Acting Injections*/
%LET LAINJ = ('J0401','J1631','J2358','J2426','J2680','J2794');
Not very clear what you're asking, but here's an attemp, assuming you have in your dataset the two columns hospice and lainj:
proc sql;
create table CLMSPULL_cleaned as
select *
from CLMSPULL_NEW
where hospice not in &hospice
and lainj not in &laing;
quit;
Related
I have the below graph created by Proc sgplot. Since the labels of values on the xaxis is too long, I want to put them in two lines. For example, instead of "T2 to T3 (Pre-pandemic to initial pandemic)", I'd like to have
line 1: T2 to T3
line 2: (Pre-pandemic to initial pandemic)" .
I have created these labels using proc format. Thanks for your help.
Consider the options FITPOLICY and SPLITCHAR for the XAXIS statement.
For you, this would be
xaxis ... fitpolicy=split splitchar="(";
For further details, consult the documentation.
In my SAS Code, I am trying to Report a table into a Word file using SAS ODS statement. And I am using the Company's standard styles. When the job runs, I can see the borders of Proc report output table in the output window. But these are gone in the Word file. Please see code below:
ods word file="&outfolder\Tabulations and graphs for ¤t_month. ¤t_year. report.docx" style=custom_style;
options orientation=landscape nodate nonumber center topmargin=0.1in bottommargin=0.15in rightmargin=0.2in leftmargin=0.2in papersize=letter;
ods escapechar="^";
ods graphics on/ outputfmt=png height = 6.75in;
%let footnote_style = ^S = {just=l font_size=7pt font_weight = light};
proc report data=out.sample nowd spanrows split="$" ;
/*There are multiple define statements here*/
run;
ods word close;
There is a long list of customized styles, So I can't remove it.
Also, If I use ODS PDF then the borders appear in PDF but somehownot in docx file.
Custom Style code:
proc template;
define style Styles.custom_styles;
style fonts/
'TitleFont2'=("Arial",12pt,bold)
'TitleFont'=("Arial",12pt,bold)
'headingFont'=("Arial",10pt,bold)
'docFont'=("Arial",10pt)
'footFont'=("Arial",8pt)
'StrongFont'=("Arial",14pt,bold)
'EmphasisFont'=("Arial",14pt,italic)
'FixedEmphasisFont'=("<monospace>, Courier, monospace",10pt,italic)
'FixedStrongFont'=("<monospace>, Courier, monospace",10pt,bold)
'FixedHeadingFont'=("<monospace>, Courier, monospace",10pt)
'BatchFixedFont'=("<monospace>, Courier, monospace",10pt)
'FixedFont'=("<monospace>, Courier, monospace",10pt)
'headingEmphasisFont'=("Arial",14pt,bold italic);
style GraphFonts/
'GraphDataFont'=("Arial",7pt)
'GraphValueFont'=("Arial",9pt)
'GraphLabel2Font'=("Arial",10pt)
'GraphLabelFont'=("Arial",10pt)
'GraphFootnoteFont'=("Arial",8pt)
'GraphTitleFont'=("Arial",12pt,bold)
'GraphTitle1Font'=("Arial",12pt,bold)
'GraphAnnoFont'=("Arial",8pt)
'GraphUnicodeFont'=("<MTsans-serif-unicode>",9pt);
style color_list "Colors used in the default style"/
'bgA'=cxffffff
'bgA1'=cx005293
'bgb1'=cxe05206
'fgA'=cxffffff
'fgB1'=cx000000
'grey'=cxCCCCCC
'fgD1'=cx005293
'blue1'=cx005293
'orange1'=cxe05206
'red1'=cxd52b1e
'green2'=cx007582
'grey2'=cx6f7c7d
'moss2'=cx476822
'yellow2'=cxf6a800
'black2'=cx31261d
'blue2'=cx009cde
'black'=cx000000;
style colors "Abstract colors used in the default style"/
'headerfg'=color_list('bgA')
'headerbg'=color_list('bgA1')
'tablebg'=cxcccccc
'headerfgemph'=color_list('bgA')
'headerbgemph'=color_list('bgA1')
'headerfgstrong'=color_list('bgA')
'headerbgstrong'=color_list('bgA1')
'datafgemph'=color_list('fgB1')
'databgemph'=color_list('bgA')
'datafgstrong'=color_list('fgB1')
'databgstrong'=color_list('bgA')
'datafg'=color_list('fgB1')
'databg'=color_list('bgA')
'batchfg'=color_list('blue1')
'batchbg'=color_list('bgA')
'tableborder'=color_list('fgD1')
'notefg'=color_list('fgB1')
'notebg'=color_list('bgA')
'bylinefg'=color_list('blue1')
'bylinebg'=color_list('bgA')
'captionfg'=color_list('fgB1')
'captionbg'=color_list('bgA')
'proctitlefg'=color_list('fgB1')
'proctitlebg'=color_list('bgA')
'titlefg'=color_list('fgB1')
'titlebg'=color_list('bgA')
'systitlefg'=color_list('fgB1')
'systitlebg'=color_list('bgA')
'contentfg'=color_list('fgB1')
'contentbg'=color_list('bgA')
'docfg'=color_list('fgB1')
'docbg'=color_list('bgA');
style graphcolors/
'gdata1'=color_list('blue1') 'gcdata1'=color_list('blue1')
'gdata2'=color_list('orange1') 'gcdata2'=color_list('orange1')
'gdata3'=color_list('red1') 'gcdata3'=color_list('red1')
'gdata4'=color_list('green2') 'gcdata4'=color_list('green2')
'gdata5'=color_list('grey2') 'gcdata5'=color_list('grey2')
'gdata6'=color_list('moss2') 'gcdata6'=color_list('moss2')
'gdata7'=color_list('yellow2') 'gcdata7'=color_list('yellow2')
'gdata8'=color_list('black2') 'gcdata8'=color_list('black2')
'gdata9'=color_list('blue2') 'gcdata9'=color_list('blue2')
'gcdata12'=cxF7AC4E 'gdata12'=cxF7AC4E
'gcdata11'=cxB38EF3 'gdata11'=cxB38EF3
'gcdata10'=cx47A82A 'gdata10'=cx47A82A
'gcmiss'=cx979797
'gmiss'=cxc3c3c2
'gablock'=colors('docbg')
'gblock'=colors('docbg')
'gcclipping'=cxDC531F
'gclipping'=cxE7774F
'gcstars'=cx000000
'gstars'=cxB9CFE7
'gcruntest'=cxBF4D4D
'gruntest'=cxCAE3FF
'gccontrollim'=cxBFC7D9
'gcontrollim'=cxE6F2FF
'gcerror'=cx000000
'gerror'=cxB9CFE7
'gcpredictlim'=cx003178
'gpredictlim'=cxB9CFE7
'gcpredict'=cx003178
'gpredict'=cx003178
'gcconfidence2'=cx003178
'gcconfidence'=cx003178
'gconfidence2'=cxB9CFE7
'gconfidence'=cxB9CFE7
'gcfit2'=cx003178
'gcfit'=cx003178
'gfit2'=cx003178
'gfit'=cx003178
'gcoutlier'=cx000000
'goutlier'=cxB9CFE7
'gcdata'=cx000000
'gdata'=cxB9CFE7
'ginsetheader'=colors('docbg')
'ginset'=cxFFFFFF
'greferencelines'=cx808080
'gheader'=colors('docbg')
'gconramp3cend'=cxFF0000
'gconramp3cneutral'=cxFF00FF
'gconramp3cstart'=cx0000FF
'gramp3cend'=cxDD6475
'gramp3cneutral'=cxFFFFFF
'gramp3cstart'=cx967CD0
'gconramp2cend'=cx99CCFF
'gconramp2cstart'=cxFFFFFF
'gramp2cend'=cx967CD0
'gramp2cstart'=cxFFFFFF
'gtext'=color_list('black')
'glabel'=color_list('black')
'gborderlines'=color_list('black')
'goutlines'=color_list('black')
'ggrid'=cxECECEC
'gaxis'=color_list('grey')
'gshadow'=cx000000
'glegend'=cxFFFFFF
'gfloor'=cxFFFFFF
'gwalls'=cxFFFFFF;
style StartUpFunction "Controls the StartUp Function. TAGATTR is only element used.";
style ShutDownFunction "Controls the Shut-Down Function. TAGATTR is only element used.";
style Container "Abstract. Controls all container oriented elements." /
font=Fonts('DocFont')
color=colors('docfg')
backgroundcolor=colors('docbg');
style Index from Container "Abstract. Controls Contents and Pages." /
color=colors('contentfg')
backgroundcolor=colors('contentbg');
style Document from Container "Abstract. Controls the various document bodies." /
doctype="<!DOCTYPE HTML PUBLIC ""-//W3C//DTD HTML 3.2 Final//EN"">"
contenttype="text/html"
protectspecialchars=auto;
style Output from Container "Abstract. Controls basic output forms." /
backgroundcolor=colors('tablebg')
rules=NONE
frame=BOX
cellpadding=7
borderspacing=1
bordercollapse=separate;
style HeadersAndFooters from Cell "Abstract. Controls table headers and footers." /
font=fonts('HeadingFont')
color=colors('headerfg')
backgroundcolor=colors('headerbg');
style Caption from HeadersAndFooters "Abstract. Controls caption field in proc tabulate." /
borderspacing=0
cellpadding=0
color=colors('captionfg')
backgroundcolor=colors('captionbg');
end;
run;
Any help will be appreciated If someone can help me figure out which ODS styles control the PROC REPORT output? So I can add borderwidth or other relevant options.
Thank You!
Resolved finally. Turns out ODS Word doesn't work properly with custom styles. So instead used ODS RTF and used .doc instead of .docX
I need to control the data type when reading XML data in SAS. The XML data are written and accessed using the XML libname engine in SAS.
XML File :
<Test>
<origin>YYYY</origin>
<NumToUse>50503</NumToUse>
<AcctNum>3-219HHJLJ</AcctNum>
<Status>1</Status>
<TADIG>AUSVF</TADIG>
<LocationNumber>1234567891011</LocationNumber>
<Phnumber>1234567890</Phnumber>
<ReferenceNumber>0044E71146</ReferenceNumber>
Map File :
<COLUMN name="LocationNumber">
<PATH syntax="XPath">/Test/LocationNumber</PATH>
<TYPE>character</TYPE>
<DATATYPE>string</DATATYPE>
<LENGTH>11</LENGTH>
</COLUMN>
<COLUMN name="PhNumber">
<PATH syntax="XPath">/Test/PhNumber</PATH>
<TYPE>character</TYPE>
<DATATYPE>string</DATATYPE>
<LENGTH>15</LENGTH>
</COLUMN>
<COLUMN name="ReferenceNumber">
<PATH syntax="XPath">/Test/ReferenceNumber</PATH>
<TYPE>numeric</TYPE>
<DATATYPE>double</DATATYPE>
</COLUMN>
Since the Reference Number is treated as Numeric am not able get the value for that particular column .its giving me
ERROR: Data contains invalid content for float datatype. Invalid content is 0044E71146
How to read the data into SAS dataset ? suggestion pls
The map file created by the XMLV2 library can be modified before using the libref to copy data into your SAS session.
There are many ways to process the engine generated map file (which is an xml file itself)
XSL transform (Proc XSL)
Someone (not me) well versed in XSLT language could likely write a short program to perform the modifications
Programmatic manipulation of the parsed xml document
Textual manipulation of the xml file
Hand editing
Text processing program
The map file :
defines which xml nodes are to become data set columns
defines the type, length, format, label, etc, of the column
defines which tables are to be created
defines which tables are to contain which columns
For the case of wanting to change the XMLV2 engine's interpretation of numeric columns to character columns, the mapfile needs to be modified (aka transformed).
<COLUMN> nodes have children nodes that, at a minimum, need to be changed
from
<TYPE>numeric</TYPE>
to
<TYPE>character</TYPE>
Considerations
There are other considerations you might need to make when transforming numbers to character enmasse, such as:
Is this numeric a date ?
Should the number be rendered through it's format first ?
SAS users are comfortable with the concept of single 'data set' which contains both data and metadata (in the header). For XML data, the data is in one file and the metadata (the mapfile) is in another.
When a SAS data that is exported and only the data xml is kept, the metadata gets lost. Making the round trip back to SAS means the metadata must be guessed at via automap.
Example code for "Programmatic manipulation of the parsed xml document"
This example changes all column definitions for numeric columns to character. The XPath /SXLEMAP/TABLE/COLUMN[not(#class='ORDINAL') and ./TYPE[text()='numeric'] is used to identify the column definitions that will be changed. No further special considerations are made.
Create xml data file to be processed
%macro createXmlV2(data=,folder=);
%local lib mem;
%let syslast = &data;
%let lib = %scan(&syslast,1,.);
%let mem = %scan(&syslast,2,.);
FILENAME XMLOUT "&folder./&data..xml";
LIBNAME XMLOUT XMLV2;
proc copy in=&lib out=xmlout;
select &mem;
run;
LIBNAME XMLOUT clear;
FILENAME XMLOUT clear;
%mend;
%* Something to play with;
%* Create an XMLV2 generated xml file containing the data set;
%createXmlV2 (data=sashelp.citiday, folder=/temp)
%createXmlV2 (data=sashelp.citimon, folder=/temp)
%createXmlV2 (data=sashelp.baseball, folder=/temp)
%*;
Transform automap file so numeric columns become character columns
%macro prepXmlRefsFor(file=);
FILENAME XMLFILE "&file";
FILENAME MAPOUT "&file..map";
FILENAME MAPOUT2 "&file..map.transformed";
%mend;
%prepXmlRefsFor(file=/temp/sashelp.citiday.xml)
%prepXmlRefsFor(file=/temp/sashelp.baseball.xml)
%prepXmlRefsFor(file=/temp/sashelp.citimon.xml)
%*;
%* create an automap file using XMLV2 library engine;
LIBNAME XMLFILE XMLV2 XMLTYPE=XMLMAP XMLMAP=MAPOUT AUTOMAP=REPLACE ;
%* parse and rewrite the generated map file ;
%* change ALL non-ordinal, non-character COLUMN nodes to indicate character type wanted;
proc groovy;
submit
"%sysfunc(pathname(MAPOUT))"
"%sysfunc(pathname(MAPOUT2))"
"20"
;
import javax.xml.parsers.*;
import javax.xml.xpath.*;
import javax.xml.transform.*;
import javax.xml.transform.dom.*;
import javax.xml.transform.stream.*;
// get parameter from submit line;
map_in=args[0]; // the automap
map_out=args[1]; // the automap transformed
length=args[2]; // length of character value for columns previously considered numeric
// parse mapfile;
doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(map_in);
xPath = XPathFactory.newInstance().newXPath();
// select the set of automap nodes that define non-ordinal, numeric columns
columns = xPath.evaluate(
"/SXLEMAP/TABLE/COLUMN" +
"[" +
"not(#class='ORDINAL')" +
" and " +
"./TYPE[text()='numeric']" +
"]", doc, XPathConstants.NODESET);
for (column in columns) {
type = xPath.evaluate("TYPE", column, XPathConstants.NODE);
dtyp = xPath.evaluate("DATATYPE", column, XPathConstants.NODE);
leng = xPath.evaluate("LENGTH", column, XPathConstants.NODE);
type.setTextContent("character");
dtyp.setTextContent("string");
if (leng == null)
column.appendChild(leng = doc.createElement("LENGTH"));
leng.setTextContent(length);
}
// rewrite mapfile with updated nodes
TransformerFactory.newInstance().newTransformer().transform(
new DOMSource(doc), new StreamResult(new File(map_out))
);
println "Programmatic transformation of mapfile completed.";
endsubmit;
quit;
* resubmit libname so libref uses transformed mapfile;
LIBNAME XMLFILE XMLV2 XMLTYPE=XMLMAP XMLMAP=MAPOUT2 AUTOMAP=REUSE;
proc copy in=xmlfile out=work;
run;
LIBNAME XMLFILE clear;
FILENAME XMLFILE clear;
FILENAME MAPOUT clear;
FILENAME MAPOUT2 clear;
One thing that became obvious after examining the 'round-trip' outcome is that xml files created by XMLV2, when re-read, will create separate column-named tables for any columns that contain missing values. These tables would have to be merged to recreate the original data set.
You might understand that the auto-mapping feature built into XMLV2 engine is choosing to define the ReferenceNumber as number, instead of as a character, is because the only one value the parser is examining is 0044E71146 and is presuming the #E# is scientific (or exponent) notation for a number.
The solution is to let the libname automap the data xml file and then update the map file xml to meet your requirements.
Example code:
XMLV2 engine creates MAPFILE, and Proc GROOVY is used to XML parse and rewrite the mapfile.
FILENAME XMLFILE "/temp/test.xml" ;
FILENAME MAPFILE "/temp/test.xml.map" ;
* parse data test.xml and write mapfile test.xml.map;
LIBNAME XMLFILE XMLV2 XMLTYPE=XMLMAP XMLMAP=MAPFILE AUTOMAP=REPLACE ;
* parse and rewrite mapfile;
* change desired column nodes to be string/character of a specified length;
proc groovy;
submit "%sysfunc(pathname(mapfile))";
import java.io.File;
import java.io.IOException;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.xpath.XPathFactory;
import javax.xml.xpath.XPathConstants;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;
* get parameter from submit line;
mapfile=args[0];
* parse mapfile;
doc = DocumentBuilderFactory
.newInstance()
.newDocumentBuilder()
.parse(
mapfile
)
;
xPath = XPathFactory
.newInstance()
.newXPath()
;
void setCharacter(column,length) {
* find column node and child nodes important to XMLV2 mapfile usage;
node = xPath.evaluate("/SXLEMAP/TABLE/COLUMN[#name='"+column+"']", doc, XPathConstants.NODE);
type = xPath.evaluate("TYPE", node, XPathConstants.NODE);
dtyp = xPath.evaluate("DATATYPE", node, XPathConstants.NODE);
leng = xPath.evaluate("LENGTH", node, XPathConstants.NODE);
if (type != null && !type.getTextContent().equals("character")) { type.setTextContent("character") }
if (dtyp != null && !dtyp.getTextContent().equals("string")) { dtyp.setTextContent("string") }
if (leng == null) {
leng = doc.createElement("LENGTH");
leng.setTextContent(length.toString());
node.appendChild(leng);
}
else
if (!length.getTextContent().equals(length.toString())) {
leng.setTextContent(length.toString());
}
}
// Make sure these two columns will be character, if not already
setCharacter("ReferenceNumber",25);
setCharacter("Phnumber", 20);
// rewrite mapfile with updated nodes
TransformerFactory
.newInstance()
.newTransformer()
.transform(
new DOMSource(doc),
new StreamResult(new File(mapfile))
);
endsubmit;
quit;
* resubmit libname so libref uses now updated mapfile;
LIBNAME XMLFILE XMLV2 XMLTYPE=XMLMAP XMLMAP=MAPFILE;
proc copy in=xmlfile out=work;
run;
Note: You could textually parse and rewrite the map file, however, there is a small outside chance the mapfile may not meet your 'text-parsing' expectations.
Let SAS make the map file.
FILENAME XMLFILE "/v/temp/test.xml" ;
FILENAME MAPFILE "/v/temp/test.xml.map" ;
LIBNAME XMLFILE XMLV2 XMLTYPE=XMLMAP XMLMAP=MAPFILE AUTOMAP=REUSE ;
Edit the file and fix the definition
<COLUMN name="ReferenceNumber">
<PATH syntax="XPath">/Test/ReferenceNumber</PATH>
<TYPE>character</TYPE>
<DATATYPE>string</DATATYPE>
<LENGTH>15</LENGTH>
</COLUMN>
You probably will want to save this in permanent and not a temporary location. Now use the fixed file to re-define the libref.
FILENAME XMLFILE "/v/temp/test.xml" ;
FILENAME MAPFILE "/v/permanent/fixed_xml.map" ;
LIBNAME XMLFILE XMLV2 XMLTYPE=XMLMAP XMLMAP=MAPFILE AUTOMAP=REUSE ;
Here is the code I am using for variables selection:
proc glmselect data=abct;
where incex1=1;
title 'GLMSELECT with Elastic Net';
model devmood_c = asetot age yrseduc sex employyn cohabyn caucyn asitot penntot
anxdis ahealthuse ahospit ventxpwk acmn nhospit bmi comorb
aqllimmn aqlsubmn aqlsympmn aqlemotmn aqlenvirmn aqltotmn
smoke3gp nalcwkcurr
/selection=elasticnet(steps=120 L2=0.001 choose=validate);
run;
The problem is that, when I run it, it tells me:
ERROR: Variable incex1 is not on file WORK.ABCT.
This incex1 variable is used to exclude people in our database that have score too high on a particular question. It works with LASSO, but even though the code is similar, doesn't seem to work with elasticnet.
Does anyone know how I could use it or if there is another way to exclude the patients who scored under a certain threshold on a questionnaire?
This is how incex1 has been coded:
if devmood_c = 0 then incex1=1;
if devmood_c = 1 then incex1=1;
if devmood_c = . then incex1=0;
if bdisev > 2 then incex1=0;
label incex1 = "1=no mood at baseline or BDI > 20, 0=excluded";
This works in test data, so it is likely an issue with your source data not having the characteristics you expect. For example,
ods graphics on;
proc glmselect data=sashelp.Leutrain valdata=sashelp.Leutest
plots=coefficients;
where x1>0;
model y = x2-x7129/
selection=elasticnet(steps=120 l2=0.001 choose=validate);
run;
That works as expected.
I have a table
id Attribute Other
1 Written Jan 20 File: 78yt8fgkje ....
2 12/22/2004 File: 3Bsdffsdf85 ....
3 12/17/2004 File: 5Osdfdsf58384 ....
4 Some May File: 0w98ejcj ....
5 10/24/2001 File: 2Ddsfsdfd1429 ....
....................
I need to remove everything that goes after the File: word in the Attribute variable
How can I do this?
I tried this solution from internet. It does not work and I do not understand what is 32767
data newDataSet;
set oldDataSet;
regex1 = prxparse("/ File:.*? /");
call prxchange(rx1, 32767, Attribute);
run;
PRX is probably overkill for this.
data want;
set have;
filepos = find(attribute,'File:');
if filepos>0 then attribute=substr(Attribute,1,filepos+5);
run;
Filepos+5 is to keep "File:" as you say "after". IF you want to get rid of "File:" as well, just get rid of the +5.