SAS String to Numeric - '+0000000xx.xxx' - sas

Question

I am extracting data from a database that has all values posted in strings, in the format +000000xx.xxx or -00000xx.xxx . I need to convert these to numeric to operate on.
data want;
set have;
numeric_var = string_var*1;
run;
works fine, but, to save compute time and resources on the final running, which will be over a much larger dataset, and in the interest of doing things properly I'd rather do that with a format or informat statement.
data want;
set have;
numeric_var = input(string_var, best8.);
run;
seems to output wrong values and to round everything to 0.
Any ideas?

Answer 1

Using best8. is telling SAS to only consider the first 8 characters of the string, so that's never going to work. You should use just best. or possibly best32. if you feel you have to pre-specify the length.
However, make sure you run some benchmarks before changing your current simple solution. SAS is already doing a character-to-numeric conversion as part of the numeric_var = string_var*1; statement, and is apparently doing it correctly; changing the code to use an informat will not automatically be any faster.
It would be cool if you benchmarked both methods and reported the results back here.
EDIT:
I did some benchmarking on this, out of curiosity. The code and log are below but TL;DR - the informat seems to be very slightly but consistently faster - 7.58 seconds vs 7.83 seconds in the run below on a 50 million observation data set. So the informat method is the way to go, but the 3% performance gain wouldn't be worth refactoring a large program, particularly if you don't have good test coverage to be sure of avoiding regressions.
483 * Set small for testing, big for benchmarking;
484 %let obs = 50000000;
485
486 * Generate test data;
487 data testdata;
488 do i = 1 to &obs;
489 numeric = round(ranuni(0)*100, 0.001);
490 char = '+' || put(numeric, z12.3-L);
491 output;
492 end;
493 run;
NOTE: The data set WORK.TESTDATA has 50000000 observations and 3 variables.
NOTE: DATA statement used (Total process time):
real time 12.55 seconds
user cpu time 11.41 seconds
system cpu time 0.84 seconds
memory 4375.18k
OS Memory 20784.00k
Timestamp 12/10/2019 10:36:11 AM
Step Count 51 Switch Count 0
494
495 %macro charToNum(in=, method=, obs=);
496
497 * Convert back to numeric;
498 data converted;
499 set ∈
500 %if "&method" = "MULT-BY-ONE" %then %do;
501 converted = char * 1;
502 %end; %else %if "&method" = "INFORMAT" %then %do;
503 converted = input(char, 32.);
504 %end;
505 if converted ne numeric then do;
506 put "ERROR: Conversion failed: " numeric= char= converted=;
507 end;
508 run;
509
510 %mend;
511
512 %charToNum(in = testdata, method = MULT-BY-ONE, obs = &obs);
NOTE: Character values have been converted to numeric values at the places given by:
(Line):(Column).
3:20
NOTE: There were 50000000 observations read from the data set WORK.TESTDATA.
NOTE: The data set WORK.CONVERTED has 50000000 observations and 4 variables.
NOTE: DATA statement used (Total process time):
real time 7.83 seconds
user cpu time 5.92 seconds
system cpu time 1.88 seconds
memory 14642.84k
OS Memory 31036.00k
Timestamp 12/10/2019 10:36:18 AM
Step Count 52 Switch Count 0
513 %charToNum(in = testdata, method = INFORMAT, obs = &obs);
NOTE: There were 50000000 observations read from the data set WORK.TESTDATA.
NOTE: The data set WORK.CONVERTED has 50000000 observations and 4 variables.
NOTE: DATA statement used (Total process time):
real time 7.58 seconds
user cpu time 5.36 seconds
system cpu time 2.15 seconds
memory 14646.18k
OS Memory 31036.00k
Timestamp 12/10/2019 10:36:26 AM
Step Count 53 Switch Count 0

Answer 2

If you want to keep only the numbers, use the code below.
Using compress this way the numbers in the string will keeped.
The first parameter is the name of variable. The second is optional, this case the caracters to be keeped. Third is "k" that means keep.
data want;
set have;
numeric_var = input(compress(string_var,"0123456789","k"), best8.);
run;