This might sound awkward but I do have a requirement to be able to concatenate all the values of a char column from a dataset, into one single string. For example:
data person;
input attribute_name $ dept $;
datalines;
John Sales
Mary Acctng
skrill Bish
;
run;
Result : test_conct = "JohnMarySkrill"
The column could vary in number of rows in the input dataset.
So, I tried the code below but it errors out when the length of the combined string (samplkey) exceeds 32K in length.
DATA RECKEYS(KEEP=test_conct);
length samplkey $32767;
do until(eod);
SET person END=EOD;
if lengthn(attribute_name) > 0 then do;
test_conct = catt(test_conct, strip(attribute_name));
end;
end;
output; stop;
run;
Can anyone suggest a better way to do this, may be break down a column into chunks of 32k length macro vars?
Regards
It would very much help if you indicated what you're trying to do but a quick method is to use SQL
proc sql NOPRINT;
select name into :name_list separated by ""
from sashelp.class;
quit;
%put &name_list.;
As you've indicated macro variables do have a size limit (64k characters) in most installations now. Depending on what you're doing, a better method may be to build a macro that puts the entire list as needed into where ever it needs to go dynamically but you would need to explain the usage for anyone to suggest that option. This answers your question as posted.
Try this, using the VARCHAR() option. If you're on an older version of SAS this may not work.
data _null_;
set sashelp.class(keep = name) end=eof;
length long_var varchar(1000000);
length want $256.;
retain long_var;
long_var = catt(long_var, name);
if eof then do;
want = md5(long_var);
put want;
end;
run;
Related
Suppose I have the following dataset:
Name Option
---- ------
A X
A
B
C X
B
E X
C
I want to delete all lines in which in column "Name" there is a letter that in column Option as an X
In the previous example for instance I would like to delete all lines in which in Name there is A, C and E.
How could I do?
I am a beginner in Sas
Use delete.
data want;
set have;
if(option = 'X') then delete;
run;
An important note about delete: no other code will run after this statement. If you have code after this conditional then it will not execute. This is a unique feature of delete.
You can optionally use remove instead, in which case additional code will run after the statement.
Since you are a beginning let's explain some basic terminology. A DATESET consists of OBSERVATIONS (what you might call a row or a line) and VARIABLES (what you might call a column).
If you want to select observations that contain particular values then you probably want to use the IN operator.
data want;
set have;
where name not in ('A','B','C');
run;
If you want to select observations where the value of the variable NAME contains a particular letter then you probably want to use INDEXC() function.
data want;
set have;
where not indexc(name,'ABC');
run;
If you do not care about the case of the letters then you could convert the values to uppercase and test. Or switch to the FINDC() function instead, which has more options, including one to ignore the case when checking for letter matches.
data want;
set have;
where not findc(name,'ABC','i');
run;
Here is a SQL solution if you want to delete ALL rows corresponding to a name that has any row with OPTION='X'
data have;
infile datalines missover;
input name $ option $;
datalines;
A X
A
B
C X
B
E X
C
;
proc sql;
create table remove as
select distinct(name) from have
where option = 'X'
;
create table want as
select * from have
where name not in (select name from remove)
;
quit;
The following code is an old-school SAS technique of SORT and MERGE and produces the same result.
proc sort data=have;
by name;
data filter;
set have;
by name;
where option='X';
if first.name;
data want;
merge have filter(in=residue);
by name;
if not residue;
run;
Now the question I have is I have a bigger problem as I am getting "this range is repeated or overlapped"... To be specific my values of label are repeating I mean my format has repeated values like a=aa b=aa c=as kind of. How do I resolve this error. When I use the hlo=M as muntilqbel option it gives double the data...
I am mapping like below.
Santhan=Santhan
Chintu=Santhan
Please suggest a solution.
To convert data to a FORMAT use the CNTLIN= option on PROC FORMAT. But first make sure the data describes a valid format. So read the data from the file.
data myfmt ;
infile 'myfile.txt' dsd truncover ;
length fmtname $32 start $100 value $200 ;
fmtname = '$MYFMT';
input start value ;
run;
Make sure to set the lengths of START and VALUE to be long enough for any actual values your source file might have.
Then make sure it is sorted and you do not have duplicate codes (START values).
proc sort data=myfmt out=myfmt_clean nodupkey ;
by start;
run;
The SAS log will show if any observations were deleted because of duplicate START values.
If you do have duplicate values then examine the dataset or original text file to understand why and determine how you want to handle the duplicates. The PROC SORT step above will keep just one of the duplicates. You might just has exact duplicates, in which case keeping only one is fine. Or you might want to collapse the duplicate observations into a single observation and concatenate the multiple decodes into one long decode.
If you want you can add a record that will add the functionality of the OTHER keyword of the VALUE statement in PROC FORMAT. You can use that to set a default value, like 'Value not found', to decode any value you might encounter that was not in your original source file.
data myfmt_final;
set myfmt_clean end=eof;
output;
if eof then do;
start = ' ';
label = 'Value not found';
hlo = 'O' ;
output;
end;
run;
Then use PROC FORMAT to make the format from the cleaned up data file.
proc format cntlin = myfmt_final;
run;
To convert a FORMAT to a dataset use the CNTLOUT= option on PROC FORMAT.
For example if you had created this format previously.
proc format ;
value $myfmt 'ABC'='ABC' 'BCD'='BCD' 'BCD1'='BCD' 'BCD2'='BCD' ;
run;
then you can use another PROC FORMAT step to make a dataset. Use the SELECT statement if you format catalog has more than one format defined and you just want one (or some) of them.
proc format cntlout=myfmt ;
select $myfmt ;
run;
Then you can use that dataset to easily make a text file. For example a comma delimited file.
data _null_;
set myfmt ;
file 'myfmt.txt' dsd ;
put start label;
run;
The result would be a text file that looks like this:
ABC,ABC
BCD,BCD
BCD1,BCD
BCD2,BCD
You get this error because you have the same code that maps to two different categories. I'm going to guess you likely did not import your data correctly from your text file and ended up getting some values truncated but without the full process it's an educated guess.
This will work fine:
proc format;
value $ test
'a'='aa' 'b'='aa' 'c'='as'
;
run;
This version will not work, because a is mapped to two different values, so SAS will not know which one to use.
proc format;
value $ badtest
'a'='aa'
'a' = 'ba'
'b' = 'aa'
'c' = 'as';
run;
This generates the error regarding overlaps in your data.
The way to fix this is to find the duplicates and determine which code they should actually map to. PROC SORT can be used to get your duplicate records.
I've output 'Moments' from Proc Univariate to datasets. Many.
Example: Moments_001.sas7bdat through to Moments_237.sas7bdat
For the first column of each dataset (new added first column, and probably new dataset, as opposed to the original) I would like to have a particular text in every cell going down to bottom row.
The exact text would be the name of the respective dataset file: say, "Moments_001".
I do not have to 'grab' the filename, per se, if that's not possible. As I know what the names are already, I can put that text into the procedure. However, grabbing the filenames, if possible, would be easier from my standpoint.
I'd greatly appreciate any help anyone could provide to accomplish this.
Thanks,
Nicholas Kormanik
Are you looking for the INDSNAME option of the SET statement? You need to define two variables because the one generated by the option is automatically dropped.
data want;
length moment dsn $41 ;
set Moments_001 - Moments_237 indsname=dsn ;
moment=dsn;
run;
I think something along these lines should be what you're after. Assuming you have a list of moments, you can loop through it and add a new variable as the first column of each dataset.
%let list_of_moments = moments_001 moments_002 ... moments_237;
%macro your_macro;
%do i = 1 %to %sysfunc(countw(&list_of_moments.));
%let this_moment = %scan(&list_of_moments., &i.);
data &this_moment._v2;
retain new_variable;
set &this_moment.;
new_variable = "&this_moment.";
run;
%end;
%mend your_macro;
%your_macro;
The brute force entering of text into column 1 looks like this:
data moments_001;
length text $ 16;
set moments_001;
text="Moments_001";
run;
You could also write a macro that would loop through all 237 data sets and insert the text.
UNTESTED CODE
%macro do_all;
%do i=1 %to 237;
%let num = %sysfunc(putn(&i,z3.));
data moments_#
length text & 16;
set moments_#
text="Moments_&num";
run;
%end;
%mend
%do_all
It seems to me (not knowing your problem) that if you use PROC UNIVARIATE with the BY option, then you wouldn't need 237 different data sets, all of your output would be in one data set and the BY variable would also be in the data set. Does that solve your problem?
Dataset: Have
F1 F2
Student Section
Name No
Dataset "Have". Data has new line character.
I need to compress the newline character from the data.
I want to do this dynamically as sometimes the "Have" dataset may contain new variables like F3,F4,F5 etc.,
I have written as macro to do this.. However it is not working as expected.
When i execute the below code, first time I am getting error as invalid reference newcnt. If i execute for second time in the same session, i am not getting error.
PFB my code:
%macro update_2(newcnt);
data HAVE;
set HAVE;
%do i= 1 %to &newcnt;
%let colname = F&i;
&colname=compress(&colname,,'c');
%end;
run;
%mend update_2;
%macro update_1();
proc sql noprint;
select count(*) into :cnt from dictionary.columns where libname="WORK" and memname="HAVE";
quit;
%update_2(&cnt)
%mend update_1;
Note: All the variables have name as F1,F2,F3,F4.,
Please tell me what is going wrong..
If there is any other procedures, please help me.
In your macro %update_1 you're creating a macro variable called &cnt, but when you call %update_2 you refer to another macro variable, &colcnt. Try fixing this reference and see if your code behaves as expected.
We created our own function to clean unwanted characters from strings using proc fcmp. In this case, our function cleans tab characters, line feeds, and carriage returns.
proc fcmp outlib=common.funcs.funcs; /* REPLACE TARGET DESTINATION AS NECESSARY */
function clean(iField $) $200;
length cleaned $200;
bad_char_list = byte(10) || byte(9) || byte(13);
cleaned = translate(iField," ",bad_char_list);
return (cleaned );
endsub;
run;
Create some test data with a new line character in the middle of it, then export it and view the results. You can see the string has been split across lines:
data x;
length employer $200;
employer = cats("blah",byte(10),"diblah");
run;
proc export data=x outfile="%sysfunc(pathname(work))\x.csv" dbms=csv replace;
run;
Run our newly created clean() function against the string and export it again. You can see it is now on a single line as desired:
data y;
set x;
employer = clean(employer);
run;
proc export data=y outfile="%sysfunc(pathname(work))\y.csv" dbms=csv replace;
run;
Now to apply this method to all character variables in our desired dataset. No need for macros, just define an array referencing all the character variables, and iterate over them applying the clean() function as we go:
data cleaned;
set x;
array a[*] _char_;
do cnt=lbound(a) to hbound(a);
a[cnt] = clean(a[cnt]);
end;
run;
EDIT : Also note that fcmp may have some performance considerations to consider. If you are working with very large amounts of data, there may be other solutions that will perform better.
EDIT 6/15/2020 : Corrected missing length statement that could result in truncated responses.
Here's an example of Robert Penridge's function, as a call routine with an array as an argument. This probably only works in 9.4+ or possibly later updates of 9.3, when permanent arrays began being allowed to be used as arguments in this way.
I'm not sure if this could be done flexibly with an array as a function; without using macros (which require recompilation of the function constantly) I don't know how one could make the right size of array be returned without doing it as a call routine.
I added 'Z' to the drop list so it's obvious that it works.
options cmplib=work.funcs;
proc fcmp outlib=work.funcs.funcs;
sub clean(iField[*] $);
outargs iField;
bad_char_list = byte(11)|| byte(10) || byte(9) || byte(13)||"Z";
do _i = 1 to dim(iField);
iField[_i] = translate(iField[_i],trimn(" "),bad_char_list);
end;
endsub;
quit;
data y;
length employer1-employer5 $20;
array employer[4] $;
do _i = 1 to dim(employer);
employer[_i] = "Hello"||byte(32)||"Z"||"Goodbye";
end;
employer5 = "Hello"||byte(32)||"Z"||"Goodbye";
call clean(employer);
run;
proc print data=y;
run;
Here is another alternative. If newline is the only thing you want to remove, then we are talking about Char only, you may leverage implicit array and Do over,
data want;
set have;
array chr _character_;
do over chr;
chr=compress(chr,,'c');
end;
run;
I currently have a dataset with 200 variables. From those variables, I created 100 new variables. Now I would like to drop the original 200 variables. How can I do that?
Slightly better would be, how I can drop variables 3-200 in the new dataset.
sorry if I was vague in my question but basically I figured out I need to use --.
If my first variable is called first and my last variable is called last, I can drop all the variables inbetween with (drop= first--last);
Thanks for all the responses.
As with most SAS tasks, there are several alternatives. The easiest and safest way to drop variables from a SAS data set is with PROC SQL. Just list the variables by name, separated by a comma:
proc sql;
alter table MYSASDATA
drop name, age, address;
quit;
Altering the table with PROC SQL removes the variables from the data set in place.
Another technique is to recreate the data set using a DROP option:
data have;
set have(drop=name age address);
run;
And yet another way is using a DROP statement:
data have;
set have;
drop name age address;
run;
Lots of options - some 'safer', some less safe but easier to code. Let's imagine you have a dataset with variables ID, PLNT, and x1-x200 to start with.
data have;
id=0;
plnt=0;
array x[200];
do _t = 1 to dim(x);
x[_t]=0;
end;
run;
data want;
set have;
*... create new 100 variables ... ;
*option 1:
drop x1-x200; *this works when x1-x200 are numerically consecutive;
*option 2:
drop x1--x200; *this works when they are physically in order on the dataset -
only the first and last matter;
run;
*Or, do it this way. This would also work with SQL ALTER TABLE. This is
the safest way to do it.;
proc sql;
select name into :droplist separated by ' ' from dictionary.columns
where libname='WORK' and memname='HAVE' and name not in ('ID','PRNT');
quit;
proc datasets lib=work;
modify want;
drop &droplist.;
quit;
If all of the variables you want to drop are named so they all start the same (like old_var_1, old_var_2, ..., old_var_n), you could do this (note the colon in drop option):
data have;
set have(drop= old_var:);
run;
data want;
set have;
drop VAR1--VARx;
run;
Would love to know if you can do this by position.
Definitely works with variable names separated by double dash (--).
I have some macros that would allow this here
You could run that whole set of macros, or just run list_vars(), is_blank(), num_words, find_word, remove_word, remove_words , nth_word().
Using these it would be:
%let keep_vars = keep_this and_this also_this;
%let drop_vars = %list_vars(old_dataset);
%let drop_vars = %remove_words(&drop_vars , &keep_vars);
data new_dataset (drop = &drop_vars );
set old_dataset;
/*stuff happens*/
run;
This will keep the three variables keep_this and_this also_this but drop everything else in the old dataset.