I want to perform the standard likelihood ratio test in logsitic regression using SAS. I will have a full logistic model, containing all variables, named A and a nested logistic model B, which is derived by dropping out one variable from A.
If I want to test whether that drop out variable is significant or not, I shall perform a likelihood ratio test of model A and B. Is there an easy way to perform this test (essentially a chi-square test) in SAS using a PROC? Thank you very much for the help.
If you want to perform likelihood ratio tests that are full model v.s. one variable dropped model, you can use the GENMOD procedure with the type3 option.
Script:
data d1;
do z = 0 to 2;
do y = 0 to 1;
do x = 0 to 1;
input n ##;
output;
end; end; end;
cards;
100 200 300 400
50 100 150 200
50 100 150 200
;
proc genmod data = d1;
class y z;
freq n;
model x = y z / error = bin link = logit type3;
run;
Output:
LR Statistics For Type 3 Analysis
Chi-
Source DF Square Pr > ChiSq
y 1 16.09 <.0001
z 2 0.00 1.0000
I'm not sure about a PROC statement that can specifically perform LRT but you can compute the test for nested models.
Script
proc logistic data = full_model;
model dependent_var = independent_var(s);
ods output GlobalTests = GlobalTests_full;
run;
data _null_;
set GlobalTests_full;
if test = "Likelihood Ratio" then do;
call symput("ChiSq_full", ChiSq);
call symput("DF_full", DF);
end;
run;
proc logistic data = reduced_model;
model dependent_var = independent_var(s);
ods output GlobalTests = GlobalTests_reduced;
run;
data _null_;
set GlobalTests_reduced;
if test = "Likelihood Ratio" then do;
call symput("ChiSq_reduced", ChiSq);
call symput("DF_reduced", DF);
end;
run;
data LRT_result;
LR = &ChiSq_full - &ChiSq_reduced;
DF = &DF_full - &DF_reduced;
p = 1 - probchi(ChiSq,DF);
run;
I'm no expert on logistic regression, but I think what you are trying to accomplish can be done with PROC LOGISTIC, using the "SELECTION=SCORE" option on the MODEL statement. There are other SELECTION options available, such as STEPWISE, but I think SCORE matches closest to what you are looking for. I would suggest reading up on it though, because there are some associated options (BEST=, START= STOP=) that you might benefit from too.
Related
I can't find a way to summarize the same variable using different weights.
I try to explain it with an example (of 3 records):
data pippo;
a=10;
wgt1=0.5;
wgt2=1;
wgt3=0;
output;
a=3;
wgt1=0;
wgt2=0;
wgt3=1;
output;
a=8.9;
wgt1=1.2;
wgt2=0.3;
wgt3=0.1;
output;
run;
I tried the following:
proc summary data=pippo missing nway;
var a /weight=wgt1;
var a /weight=wgt2;
var a /weight=wgt3;
output out=pluto (drop=_freq_ _type_) sum()=;
run;
Obviously it gives me a warning because I used the same variable "a" (I can't rename it!).
I've to save a huge amount of data and not so much physical space and I should construct like 120 field (a0-a6,b0-b6 etc) that are the same variables just with fixed weight (wgt0-wgt5).
I want to store a dataset with 20 columns (a,b,c..) and 6 weight (wgt0-wgt5) and, on demand, processing a "summary" without an intermediate datastep that oblige me to create 120 fields.
Due to the huge amount of data (more or less 55Gb every month) I'd like also not to use proc sql statement:
proc sql;
create table pluto
as select sum(db.a * wgt1) as a0, sum(db.a * wgt1) as a1 , etc.
quit;
There is a "Super proc summary" that can summarize the same field with different weights?
Thanks in advance,
Paolo
I think there are a few options. One is the data step view that data_null_ mentions. Another is just running the proc summary however many times you have weights, and either using ods output with the persist=proc or 20 output datasets and then setting them together.
A third option, though, is to roll your own summarization. This is advantageous in that it only sees the data once - so it's faster. It's disadvantageous in that there's a bit of work involved and it's more complicated.
Here's an example of doing this with sashelp.baseball. In your actual case you'll want to use code to generate the array reference for the variables, and possibly for the weights, if they're not easily creatable using a variable list or similar. This assumes you have no CLASS variable, but it's easy to add that into the key if you do have a single (set of) class variable(s) that you want NWAY combinations of only.
data test;
set sashelp.baseball;
array w[5];
do _i = 1 to dim(w);
w[_i] = rand('Uniform')*100+50;
end;
output;
run;
data want;
set test end=eof;
i = .;
length varname $32;
sumval = 0 ;
sum=0;
if _n_ eq 1 then do;
declare hash h_summary(suminc:'sumval',keysum:'sum',ordered:'a');;
h_summary.defineKey('i','varname'); *also would use any CLASS variable in the key;
h_summary.defineData('i','varname'); *also would include any CLASS variable in the key;
h_summary.defineDone();
end;
array w[5]; *if weights are not named in easy fashion like this generate this with code;
array vars[*] nHits nHome nRuns; *generate this with code for the real dataset;
do i = 1 to dim(w);
do j = 1 to dim(vars);
varname = vname(vars[j]);
sumval = vars[j]*w[i];
rc = h_summary.ref();
if i=1 then put varname= sumval= vars[j]= w[i]=;
end;
end;
if eof then do;
rc = h_summary.output(dataset:'summary_output');
end;
run;
One other thing to mention though... if you're doing this because you're doing something like jackknife variance estimation or that sort of thing, or anything that uses replicate weights, consider using PROC SURVEYMEANS which can handle replicate weights for you.
You can SCORE your data set using a customized SCORE data set that you can generate
with a data step.
options center=0;
data pippo;
retain a 10 b 1.75 c 5 d 3 e 32;
run;
data score;
if 0 then set pippo;
array v[*] _numeric_;
retain _TYPE_ 'SCORE';
length _name_ $32;
array wt[3] _temporary_ (.5 1 .333);
do i = 1 to dim(v);
call missing(of v[*]);
do j = 1 to dim(wt);
_name_ = catx('_',vname(v[i]),'WGT',j);
v[i] = wt[j];
output;
end;
end;
drop i j;
run;
proc print;[enter image description here][1]
run;
proc score data=pippo score=score;
id a--e;
var a--e;
run;
proc print;
run;
proc means stackods sum;
ods exclude summary;
ods output summary=summary;
run;
proc print;
run;
enter image description here
I am comparing the evolution of plasma concentrations over time for different treatments of patients.
We applied each treatment to different subjects and for each treatment we want a graph with the evolution for each subject in black, as well as for the the mean in red.
It should look like this
but it does look like this
My data has variable
trtan and trta for treatment number and name
subjid for the patient receiving that treatment
ATPT for timepoint
AVAL for Individual Concentrations
MEAN for average Concentrations
I am using SGPLOT to produce this line plot. y axis has concentrations while x axis has time points, I am sorting data by treatment, subject and timepoint before passing to Proc SGPLOT.
Lines for indivizual subjects are fine, Issue is with mean line plot, Since dataset is sorted by subject i am getting multiple mean plots by subject as well.
My requirement is to have multiple indivizual plots and an overlaying mean plot. Can anyone advise how can i solve this.
I am using below code. How can I repair it?
proc sort data = pc2;
by trtan trta subjid atptn atpt;
run;
proc sgplot data = pc2 dattrmap = anno pad = (bottom = 20%) NOAUTOLEGEND ;
by trtan trta;
series x = atptn y = aval/ group = trta
lineattrs = (color = black thickness = 1 pattern = solid );
series x = atptn y = mean/ group = trta attrid = trtcolor
lineattrs = (thickness = 2 pattern = solid );
xaxis label= "Actual Time (h)"
labelattrs = (size = 10)
values = (0 12 24 36 48 72 96 120 168)
valueattrs = (size = 10)
grid;
yaxis label= "Plasma Concentration (ng/mL)"
labelattrs = (size = 10)
valueattrs = (size = 10)
grid;
run;
This is not a problem with the mean only.
Leave out the mean, ass min=-20 to your yaxis specification, and you will see the same problem.
Alternatively run this code
data pc2;
do subj = 1 to 3;
do time = 1 to 25;
value = 2*sin(time/3) + rand('Normal');
output;
end;
end;
run;
proc sgplot data=pc2;
series x=time y=value;
run;
and you will get
The solution is to have one plot for each subject, so first sort the data by time and transpose it to have one variable subj_1 etc. for each subject.
proc sort data=pc2 out=SORTED;
by time subj;
run;
proc transpose data=TEST out=TRANS prefix=subj_;
by time;
id subj;
run;
I leave it as an exercise for you to add the mean to this dataset.
Then run sgplot with a series statement per subject. To build these statements, we interrogate the meta data in dataset WORK.TRANS
proc sql;
select distinct 'series x=time y='|| name ||'/lineattrs = (color=black)'
into :series_statements separated by ';'
from sasHelp.vColumn
where libname eq 'WORK' and memName eq 'TRANS'
and (name like 'subj%' or name = mean;
quit;
proc sgplot data=TRANS;
&series_statements;
run;
The result, without the mean, looks like this for my example:
Of course, you will have to do some graphical fine tuning.
We can achive it simply by taking the mean by ATPT and then instead of merging the mean record to the PK data by ATPT, you need to append the records and then you can run your code and it will give you the result you are expecting, please let me know if it does not work, it seems to have worked for me.
I am attempting to replicate Excel's Goal Seek in SAS.
I would like to find a constant number that when added to the initial data the overall average of the data equals the target. This gets a bit tricky when a transformation is involved.
So my three data points (var1) are 0.78, 0.8, 0.85. The target is 0.87.
I would like to find x where AVERAGE(1/(1+EXP(-(LN(var1/(1+var1)) + x))) = 0.87
This is the code I currently have, but it gets x = 0.4803 when it should be 0.4525 (found via Excel).
data aa;
input var1 target;
datalines;
0.78 0.87
0.8 0.87
0.85 0.87
;
run;
proc model data=aa outparms=parm;
target = 1/(1+EXP(-(log(var1/(1-var1)) + x)));
fit target;
run;
I think this isn't working bc it doesn't include an average of all 3 data points. I'm not sure how to do this. Ideally I'd just be able to change the second line in the proc model node to this:
target = Avg(1/(1+EXP(-(log(var1/(1-var1)) + x))));
But that doesn't work.
proc model is primarily designed for time-series, and doesn't do well with using summary functions vertically; however, it does great when doing it horizontally. One way to resolve it would be by transposing the problem:
proc transpose data=aa out=aa_trans;
by target;
var var1;
run;
proc model data=aa_trans;
endo x;
exo COL1-COL3 target;
target = mean(1/(1+EXP(-(log(COL1/(1-COL1)) + x)))
, 1/(1+EXP(-(log(COL2/(1-COL2)) + x)))
, 1/(1+EXP(-(log(COL3/(1-COL3)) + x))) );
solve / out=solution solveprint ;
run;
We get an answer of 0.4531398172. This can be checked by directly plugging in the value:
data _null_;
set aa_trans;
x = 0.4531398172;
check = mean(1/(1+EXP(-(log(COL1/(1-COL1)) + x)))
, 1/(1+EXP(-(log(COL2/(1-COL2)) + x)))
, 1/(1+EXP(-(log(COL3/(1-COL3)) + x))) );
put '*********** ' check;
run;
This method requires additional macro programming to generalize, and may be very computationally expensive if you have many observations to transpose. To generalize it for any given number of columns, you could use the following macro program:
%macro generateEquation;
%global eq;
%let eq = ;
proc sql noprint;
select count(*)
into :total
from aa
;
quit;
%do i = 1 %to &total.;
%let eq = %cmpres(&eq 1/(1+EXP(-(log(COL&i/(1-COL&i))+x))));
%end;
%let eq = mean(%sysfunc(tranwrd(&eq, %str( ), %str(,) ) ) );
%put &eq;
%mend;
%generateEquation;
proc model data=aa_trans;
endo x;
exo COL1-COL3 target;
target = &eq.;
solve / out=solution solveprint ;
run;
Instead, you might want to reframe this problem as an optimization problem with no objective function. proc optmodel, if available at your site, lets you do this matrix manipulation. The resulting code is more complex and manual, but will give you a more generalized and computationally feasible result.
You will need to add two new variables and separate the target to a new dataset.
data aa;
input targetid obs var1;
datalines;
1 1 0.78
1 2 0.8
1 3 0.85
;
run;
data bb;
input targetid target;
datalines;
1 0.87
;
run;
proc optmodel;
set id;
set obs;
set <num,num> id_obs;
/* Constants */
number target{id};
number var1{id_obs};
read data bb into id=[targetid]
target;
read data aa into id_obs=[targetid obs]
var1;
/* Parameter of interest */
var x{id};
/* Force the solver to seek the required goal */
con avg {i in id}: target[i] = sum{<j,n> in id_obs: j=i} (1/(1+EXP(-(log(var1[j, n]/(1-var1[j, n])) + x[i]))) )
/ sum{<j,n> in id_obs: j=i} 1;
/* Check if it's the equation that we want */
expand;
/* Solve using the non-linear programming solver with no objective */
solve with nlp noobjective;
/* Output */
create data solution from [targetid] = {i in id}
x[i];
quit;
optmodel returns a similar answer: 0.4531395426, which differs by 0.0000002746 decimal places. The answers are not identical due to differing methods and optimality tolerances; however, the solution checks out.
proc sql;
select Avg(1/(1+EXP(-(log(var1/(1-var1)) + 0.4531395426))))
from aa;
quit;
I'm looking for a macro or something in SAS that can help me in isolating the outliers from a dataset. I define an outlier as: Upperbound: Q3+1.5(IQR) Lowerbound: Q1-1.5(IQR). I have the following SAS code:
title 'Fall 2015';
proc univariate data = fall2015 freq;
var enrollment_count;
histogram enrollment_count / vscale = percent vaxis = 0 to 50 by 5 midpoints = 0 to 300 by 5;
inset n mean std max min range / position = ne;
run;
I would like to get rid of the outliers from fall2015 dataset. I found some macros, but no luck in working the macro. Several have a class variable which I don't have. Any ideas how to separate my data?
Here's a macro I wrote a while ago to do this, under slightly different rules. I've modified it to meet your criteria (1.5).
Use proc means to calculate Q1/Q3 and IQR (QRANGE)
Build Macro to cap based on rules
Call macro using call execute and boundaries set, using the output from step 1.
*Calculate IQR and first/third quartiles;
proc means data=sashelp.class stackods n qrange p25 p75;
var weight height;
ods output summary=ranges;
run;
*create data with outliers to check;
data capped;
set sashelp.class;
if name='Alfred' then weight=220;
if name='Jane' then height=-30;
run;
*macro to cap outliers;
%macro cap(dset=,var=, lower=, upper=);
data &dset;
set &dset;
if &var>&upper then &var=&upper;
if &var<&lower then &var=&lower;
run;
%mend;
*create cutoffs and execute macro for each variable;
data cutoffs;
set ranges;
lower=p25-1.5*qrange;
upper=p75+1.5*qrange;
string = catt('%cap(dset=capped, var=', variable, ", lower=", lower, ", upper=", upper ,");");
call execute(string);
run;
SAS newbie here.
My question is about PROC REG in SAS; let's assume I have already created a model and now I would like to use this model, and known predictor variables to estimate a response value.
Is there a clean and easy way of doing this in SAS? So far I've been manually grabbing the intercept and the coefficients from the output of my model to calculate the response variable but as you can imagine it can get pretty nasty when you have a lot of covariates. Their user's guide is pretty cryptic...
Thanks in advance.
#Reese is correct. Here is some sample code to get you up the learning curve faster:
/*Data to regress*/
data test;
do i=1 to 100;
x1 = rannor(123);
x2 = rannor(123)*2 + 1;
y = 1*x1 + 2*x2 + 4*rannor(123);
output;
end;
run;
/*Data to score*/
data to_score;
_model_ = "Y_on_X";
y = .;
x1 = 1.5;
x2 = -1;
run;
/*Method 1: just put missing values on the input data set and
PROC REG will do it for you*/
data test_2;
set test to_score;
run;
proc reg data=test_2 alpha=.01 outest=est;
Y_on_X: model y = x1 x2;
output out=test2_out(where=(y=.)) p=predicted ucl=UCL_Pred lcl=LCL_Pred;
run;
quit;
proc print data=test2_out;
run;
/*Method 2: Use the coefficients and the to_score data with
PROC SCORE*/
proc score data=to_score score=est out=scored type=parms;
var x1 x2;
run;
proc print data=scored;
var Y_on_X X1 X2;
run;
2 ways:
Append the data you want into the data set you're going to use to get estimates but leave the y value blank. Grab the estimates using the output statement from proc reg.
Use Proc Score
http://support.sas.com/documentation/cdl/en/statug/63347/HTML/default/viewer.htm#statug_score_sect018.htm