I am having trouble running a foreach loop. The loop runs without error but gives no output. Can someone tell me what they think might be going on? Many thanks in advance!
Here is the code:
cd "O:\RESEARCH\ikhilko\Subway Big Data project"
local datafiles : dir . files "*.txt"
foreach file in `datafiles' {
insheet using `file',
clear
insheet using `file',
drop v9-v43
save date1, replace
}
UPDATE:
Interestingly, the code runs when I just type it into the command line, rather than doing it from the .do file, any idea what might be going on there?
It is important to note that local macros are precisely that, i.e. defined and visible only locally.
Locally means within
the same interactive session
or
the same program
or
the same do file (or do file editor contents)
or
the same part of the do file (or ...) executed by selection
Locality is, it seems, biting you here. A local macro defined in one place is not visible in another. A local macro reference will evaluate to missing, i.e. an empty string, if the macro is not visible.
Some code for the debugging. display the contents of your local datafiles to see what's going into the loop:
local datafiles : dir . files "*.txt"
display `"`datafiles'"'
local wordx : word 1 of `datafiles'
display `"`wordx'"'
foreach file in `datafiles' {
display "`file'"
}
(The code does not format well in the comments section.)
Related
I need to create 215 empty csv files with Stata and save them on my computer.
Since this is a repetitive task, a loop would be perfect. Each file would have a similar but different name (for example Data_Australia, Data_Austria and so on).
How do I create a loop to generate several empty csv datasets with Stata?
I tried the community-contributed command touch but it works well when you only need to generate one empty dataset.
Assuming you want a completely empty file (no header row or anything), just open a file to write to, and immediately close it again.
cd "C:\Users\My\Directory"
local country_names Australia Austria "Republic of Korea" // add all the names here
foreach country_name in `country_names' {
file open f1 using "Data_`country_name'.csv", write replace
file close f1
}
If you have the names stored as a string variable, say country, you can instead loop through the values in that variable (in this case stopping when it reaches the end or an empty row).
local row = 1
while country[`row'] != "" {
file open f1 using "Data_`=country[`row']'.csv", write replace
file close f1
local ++row
}
I would like to use a variable (its value) as file name. Any ideas? Im using stata 14
Thanks a Lot in advance!
Per the comment from #toonice, please do give more details and I can better address your question.
However, you can use local macros to input into file names. Let's say you have a data set of a single variable x taking values of your filenames. You could loop through the data to save different files with the values of x. For example:
local N = _N
forvalues i = 1/`N' {
local myfilename x[`i']
// Insert code that changes data in some way to make files different
save ../output/`myfilename'_staticfilename.dta, replace
}
Give me more context and I am happy to provide more help.
Using Stata, I define a local macro (macro_name) as a variable (macro_variable) in one data file.
After reading in a new file (in the same do file), I'm no longer able to reference that macro.
Instead, I receive the error:
. di `macro_name'
macro_variable not found
I am learning how to use macros, so please bear with me. However, shouldn't I be able to still display or call on that macro in a single do file even if I load in a new data set?
For example:
use "newdata.dta", clear
This problem occurs regardless of whether I define the macro as a global or local. Additionally, I attempted to solve the problem by creating a separate locals.do file that include in the preamble of my master do file as:
include locals.do
But, I still receive the error listed above.
Do macros (local or global) disappear immediately upon reading in a new file? That doesn't seem right based on what I've read.
Thanks in advance for any clarification.
Consider the following, which points to the source of your problem, and in the last command, reproduces precisely the error message you received.
. do "/var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//SD08491.000000"
. local macro_name macro_variable
. macro list _macro_name
_macro_name: macro_variable
. display "`macro_name'"
macro_variable
. display `macro_name'
macro_variable not found
r(111);
end of do-file
Added in edit: The above was run from the do-file editor window. When I instead launch Stata and paste the four commands into the command window, running them a line at a time, the following are what results.
. local macro_name macro_variable
. macro list _macro_name
_macro_name: macro_variable
. display "`macro_name'"
macro_variable
. display `macro_name'
macro_variable not found
r(111);
.
At the risk of over-explaining, the point to my original answer is that the error message the displayed in the original post, and in the final command in both of my examples, was due to the failure to include quotation marks in the display command, which caused display to believe that "macro_variable", which was the value assigned to the local macro "macro_name" was not a character string constant, but rather a variable name or scalar, and display was unable to locate a variable or scalar by that name.
Let me add as a bonus explanation that the use of locals.do described in the original post has no hope of working, because local macros are local to the do-file in which they are executed, and vanish at the termination of that do-file. In particular, if you submit a local command by selecting a subset of the lines in the do-file editor window, those lines are copied into a temporary do-file and the values of the local macros vanish at the termination of the temporary do-file.
Generalizing what I wrote in my comment above to Nick:
Macros only maintain the connection between the variable/varlist assigned to a macro name and, therefore, the variable/varlist to which the macro's name refers to must be in memory (i.e. the dataset that contains the variable/varlist has to be in memory) in order to access it via the macro.
Assigning a variable/varlist to a macro does not persist the actual value(s)/element(s) in memory, but rather maintain the connection between the variable/varlist and the macro name assigned to it/them.
I have data in a .txt in which the variables are delimited by the symbol | and the first row contains the variable names. I have successfully insheeted the data as:
insheet using "filename.txt", delim("|") clear
However, I would like to insheet only one variable from the data set. When I try to insheet only the one variable in, I have tried:
insheet variable using "filename.txt", delim("|") clear
Unfortunately, it does not work, and using a reduced down version of the .txt, I receive an error:
too few variables specified
error in line 2 of file
The .txt looks as follows:
V1|V2
123|456
Note that there are more variables and more rows but I've reduced it for ease of exposition. In addition, the .txt is formatted with an automatic return after each row.
I would greatly appreciate any help that you can provide to do this task. Please let me know whether there is any further information that I can provide about the to make the issue clearer.
It's difficult for me to say why that doesn't work, but insheet is old code that seems a little more fragile than other import commands.
Did you try import excel?
Is it out of the question to insheet everything and drop what you don't want?
Did you think of using filefilter to change the | to spaces or commas?
The Stata command insheet does not have this option. Use insheet and keep varname.
I've just started learning SAS because I'm required to use it for a statistics course. For this course, the university provides SAS 9.2 through their virtual-machine setup: I make a reservation in their system, they generate a VM on one of their servers, and I connect to the VM using Microsoft's Remote Desktop client. The virtual machines are generated and erased per session; settings are reset every time, and files must be stored on my client computer (which is accessible in the VM by a UNC path).
Within this setup, when I open a program file stored on my laptop, I've only been able to access the accompanying data files (each stored in the same folder as the program) either by hardcoding the full path or by updating the "current folder" setting at the beginning of each session. The first is problematic because it means the program won't run anywhere else - in particular, when I email it to the professor. The second is inconvenient, because browsing to this particular UNC path is time consuming, and I already have to browse to the same path to open the program file.
I want to make this easier by programmatically setting the current folder to the folder containing the program. Then I could just open the file and get to work. I've found some examples of getting the filename of the program file, of getting the path to a fileref, and of (link limit exceeded) setting the current folder, but I haven't been able to combine them in the right way. Please connect the dots for me.
To programmatically change the Windows current directory from SAS, you can use the X command, which is what really happens when you use the "Change current folder" dialog box:
x 'cd "\\computername\share name\folder"';
You can also do this using the SYSTEM data step function, a method I prefer because you get a return code (but more typing of course):
data _null_;
rc = system( 'cd "\\computername\share name\folder"' );
if rc = 0
then putlog 'Command successful';
else putlog 'Command failed';
run;
Note the UNC path is surrounded with double-quotes, which is necessary if the path contains blanks.
Of course, this still requires you to manually type in the command, but it might be something you could add to the program source code. If your VM environment allowed you to maintain some permanent presence on the server, you could save this command into a start-up file.
I would ask your professor for advice; if you are working with data given to you as part of your class, you may only need to send just the source code. On the other hand, if you are creating output data as part of your assignment, your professor might want your to deliver source code and SAS data sets. Surely he or she will have some procedure.
Complete Answer:
SAS's obtuse notation requires some strange delimiter fiddling to combine my partial solution (finding the path) with #Bob Duell's partial solution (setting the current folder). There seem to be two key rules involved:
&var is expanded in double-quoted strings ("&var"), but not single-quoted strings ('&var')
Quotes in &var are not treated as delimiters after expansion
So the solution is to compute a string of the quoted path (where the quotes are part of the string), and expand that within a double-quoted parameter to X or SYSTEM:
%let qsrc=%str(%")&src%str(%");
X "cd &qsrc"
It's not required to store the string, both &src and &qsrc can be expanded in-place, which yields a single statement solution:
X "cd %str(%")%substr(%sysget(SAS_EXECFILEPATH),1,%eval(%length(%sysget(SAS_EXECFILEPATH))-%length(%sysget(SAS_EXECFILENAME))))%str(%")";
This executes correctly, but breaks the syntax coloring in the GUI. Within a string, %str(%") and "" both expand to ", so replacing %str(%") with "" both executes correctly and is colored correctly in the GUI:
X "cd ""%substr(%sysget(SAS_EXECFILEPATH),1,%eval(%length(%sysget(SAS_EXECFILEPATH))-%length(%sysget(SAS_EXECFILENAME))))""";
This inherits the limitation that it only works when SAS_EXECFILEPATH and SAS_EXECFILENAME are defined, which is the case when running from within the Windows GUI editor. It's also subject to any limitations on in the "cd" command, which SAS intercepts rather than invoking the Windows command line. I expect it will fail on paths containing quotes.
A partial answer: One way to get the containing folder from the filename of the program file
Spread out & logging steps:
/* Find PathName of folder containing program */
%let FullName=%sysget(SAS_EXECFILEPATH);
%put FullName: &FullName.;
%let FullLen=%length(&FullName);
%put FullLen: &FullLen.;
%let BaseName=%sysget(SAS_EXECFILENAME);
%put BaseName: &BaseName.;
%let BaseLen=%length(&BaseName);
%put BaseLen: &BaseLen.;
%let PathLen=%eval(&FullLen.-&BaseLen.);
%put PathLen: &PathLen.;
%let PathName=%substr(&FullName,1,&PathLen);
%put PathName: &PathName.;
Consolidated & silent:
/* Find src folder */
%let src=%substr(%sysget(SAS_EXECFILEPATH),1,%eval(%length(%sysget(SAS_EXECFILEPATH))-%length(%sysget(SAS_EXECFILENAME))));
This only works when SAS_EXECFILEPATH and SAS_EXECFILENAME are defined, and it's not clear when that is. It does work when using the Windows GUI editor.