SAS can take a very long time to load an output table after you click on it from the process flow, is there a way to cancel it? I've wasted hours waiting for tables to load, I am hoping there is a way to exit the "Input Data" or "Output Data" tabs and return to the process flow window.
I don't know of a way to directly stop the table from loading; in my brief tests, ctrl+break and esc don't stop it, which are the usual possibilities, and both do work in some places in EG. I will say that this is somewhat of a weakness in EG in general; it's not perfect at handling things in the background and allowing you to interrupt, though it's improved greatly over the years.
What you can do to avoid this being a problem, at least possibly, is to go to the Tools->Options->Data->Performance option screen, and limit the Maximum dimensions to display in the data grid value to something well under the default 1,000,000. Change it to 50,000 or something else that is a reasonable compromise between your needs and how long it takes to connect to datasets.
Alternately, you can prevent datasets from appearing on the process flow by altering the option in Tools->Options->Results->Results General Maximum number of output data sets to add to the project to zero. That doesn't prevent you from browsing datasets; you would just have to do it through the Servers tab on the lower left (in the default setup).
If I am not wrong then you would "NOT" want the Input and Output datasets to open when the process is run. Then uncheck the highlighted options in SAS EG, Goto Tools--> Options --> Results --> Results General and uncheck the options shown in image below.
Related
I have asked a few questions related to this personal project of mine already on this platform, and this should be the last one since I am so close to finishing. Below is the link to a mock example spreadsheet I've created, which mimics what my actual project does but it contains less sensitive information and is also smaller in size.
Mock Spreadsheet
Basic rundown of the spreadsheet:
Pulls data from a master schedule which is controlled/edited by another party into the Master Schedule tab.
In the columns adjacent to the imported data, an array formula expands the master schedule by classroom in case some of the time slots designate multiple rooms. Additional formulas adjust the date, start time, and end time to be capped within the current day's 24-hour period. The start time of each class is also made to be an hour earlier.
In the Room Schedule tab, an hourly calendar is created based on the room number in the first column, and only corresponds to the current day.
I have tested the spreadsheet extensively with multiple scenarios, and I'm happy with how everything works except for the calculation time. I figured the two volatile functions I use would take some processing time just by themselves, and I certainly didn't expect this to be lightning-fast especially without using a script, but the project that I am actually implementing this method for is much larger and takes a very long time to update. The purpose of this spreadsheet is to allow users to find an open room and "reserve" it by clicking the checkbox next to it (which will consequently color the entire row red) allowing everyone else to know that it is now taken.
I'd like to know if there is any way to optimize / speed up my spreadsheet, or to not update it every time a checkbox is clicked and instead update it "manually", similar to what OP is asking here. I am not familiar with Apps Script nor am I well-versed in writing code overall, but I am willing to learn - I just need a push in the right direction since I am going into this blind. I know the number of formulas in the Room Schedule tab is probably working against me yet I am so close to what I wanted the final product to be, so any help or insight is greatly appreciated!
Feel free to ask any questions if I didn't explain this well enough.
to speed up things you should avoid usage of the same formulae per each row and make use of arrayformulas. for example:
=IF(AND(TEXT(K3,"m/d")<>$A$1,(M3-L3)<0),K3+1,K3+0)
=ARRAYFORMULA(IF(K3:K<>"",
IF((TEXT(K3:K, "m/d")<>$A$1)*((M3:M-L3:L)<0), K3:K+1, K3:K+0), ))
=IF(AND(TEXT(K3,"m/d")=$A$1,(M3-L3)<0),TIMEVALUE("11:59:59 PM"),M3+0)
=ARRAYFORMULA(IF(K3:K<>"",
IF((TEXT(K3,"m/d")=$A$1)*((M3-L3)<0), TIMEVALUE("11:59:59 PM"), M3:M+0), ))
My client does not want to see intermediate work tables in SAS workflow and as a workaround, i have set option --> Results --> results general --> Maximum number of O/P data sets to add to the project AS 0.
Now, the issue is, i get this note for each of my program in workflow - "data set limit reached". Now i understand why, but can someone help me in suppressing it?? I do not want these notes to be generated in my workflow.
TBH, i am just using Proc SQl in my programs and creating tables out of it.
Thanks in advance!!!!!
As far as I know, the only true solution here would be to use PROC DATASETS to clean up your intermediate data tables (which obviously is not ideal if you're testing). In cases where I wanted to have a clean workflow, I did what you did and just lived with the notes.
Another possibility would be to use SAS Studio instead of Enterprise Guide. If you have 3.7 (or possibly 3.6?) you have a workflow mode ("Visual") instead of just the "Programmer" mode which has only single programs; it's a much more simple version of the workflow than EG has, but that is somewhat of a benefit in some situations.
As i run more commands in Stata, the earlier output disappears from the window (i.e, if i scroll to the top, the earlier output is no longer there, suggesting that there is a set 'height' or number of rows of the output window).
Is it possible to change this setting, i.e., to increase the amount of output that is displayed?
Thanks to the suggestion in the comments - in case of relevance to anyone else, this can be achieved with the command:
set scrollbufsize 2000000
(or any value up to 2000000) - this takes effect the next time Stata is opened.
I am working on a war game project using legacy C++ code. Attacks appear to be functioning strangely in "extreme cases" (e.g., a combat in one game "tile" in which 5000 arrows are fired at a group of 3 regiments, one of which can be the target for any of the ~5000 potential "hits" in the course of one game turn). My task is to collate data on what is happening during a game turn.
I need to record how the value for one particular Local variable changes during the course of one game turn for any unit that comes under fire (restricting to a 'test scenario' in which only regiments in one tile are under attack).
My Visual Studio "Watch window" at the beginning of a debug session looks about like this:
Name----------------------Value------------Type
targ-----------------------011-------------short
(gUnit[011]).damage--------0---------------short
(gUnit[047]).damage--------0---------------short
(gUnit[256]).damage--------0---------------short
What this mean is: at the first breakpoint, the regiment represented in nested array gUnit[011]) has been determined to be the target ("targ") for a "hit" and the "damage" for that regiment prior to this hit is = 0.
After one loop that may change to this:
Name----------------------Value------------Type
targ-----------------------256-------------short
(gUnit[011]).damage--------3---------------short
(gUnit[047]).damage--------0---------------short
(gUnit[256]).damage--------0---------------short
Which means: after one "hit" Regiment 011 has suffered 3 damage, and now for the next hit that has been determined to occur Regiment 047 is the target. So for each cycle of the call loop, any of the potential targets can be reassigned as the target and the value of 'damage' can change. For this example of 5000 arrows fired, there could be 5000 hits (though it would be unlikely to be > 1300 hits for that many shots fired). For each hit that occurs in the span of one game turn, I need to record:
target and damage
I have breakpoints setup and I can do what I need to do manually meaning:
Advance to first call to the function that determines a hit ["MissileHit()"]
Advance again and note that value for (gUnit[011]).damage has changed from 0 to 3. /* i.e., the Regiment with the ID "011" has taken "3" damage . . . */
Repeat 5000 times for experiment ONE (and noting that, with each loop targ can change to any of the three arrays that represent the units that are under attack in the tile.
Repeat for another 7 experiments (with slightly different conditions) for phase one of testing.
Repeat for perhaps 3 or 4 more phases of testing . . .
In sum, THOUSANDS of rows with at least two columns:
targ.......damage
(as a side note, this csv will get additional columns plugged in to it after each test cycle, so the final spreadsheet will look more like this:
atkrID....Terrain....atkrExpr....atkrFatig....targID....targDamage
The end result of all this data collection being: descriptive statistics can be calculated on the various in-game variables that mediate the outcomes of attacks)
Obviously, doing this "manually" would be ridiculous.
Is there an easy way to get Visual Studio to generate csv files that will record this data for me?
I can certainly go down the path of writing scripts to run when these breakpoints execute, and fstreaming stuff into a file. But, I'm still wet-behind the ears (both with Visual Studio, C++ and developing in general!) so, I thought it worth asking if there are built in features of Visual Studio that would facilitate this.
I think the VS extension like Drop's comment would be a better path for this issue, but I think it would require the high extension knowledge, so I just provide another workaround for this issue: Add the data breakpoint and output the value to the output windows, even if you also need to copy and past the value to your files, but I think it is also convenient for you to analyze the value during debugging.
Visual Studio. Debug. How to save to a file all the values a variable has had during the duration of a run?
I ran the following code and an hour later, just as the code was finishing a sort execute error occurred. Is there something wrong with my code or is my computer processor and Ram insufficient
proc sql;
create table today as
select a.account_number, a.client_type, a.device ,a.entry_date_est,
a.entry_time_est, a.duration_seconds, a.channel_name, b.esn, b.service_start_date,
b.service_end_date, b.product_name, b.billing_frequency_fee, b.plan_category,
b.plan_subtype, b.plan_type
from listen_nomiss a inner join service_nomiss b
on (a.account_number = b.account_number)
order by account_number;
quit;
That error is most commonly seen when you run out of utility space to perform the sort. A few suggestions for troubleshooting are available in this SAS KB post; the most useful suggestions:
options fullstimer msglevel=i ; will give you a lot more information about what's going on behind the scenes, so you can troubleshoot what is causing the issue
proc options option=utilloc; run; will tell you where the utility directory is that your temporary files will be created in for the sort. Verify that about 3 times the space needed for the final table is available - sorting requires roughly 3 times the space in order to properly sort the dataset due to how the sort is processed.
OPTIONS COMPRESS; will save some (possibly a lot of) space if not already enabled.
options memsize; and options sortsize; will tell you how much memory is allocated to SAS, and at what size a sort is done in memory versus on disk. sortsize should be about 1/3 of memsize (given the requirement of 3x space to process it). If your final table is around but just over sortsize, you may be better off trying to increase sortsize if the default is too low (same for memsize).
You could also have some issues with permissions; some of the other suggestions in the kb article relate to verifying you actually have permission to write to the utility directory, or that it exists at all.
I've had a project in the past where resources was an issue as well.
A couple of ways around it when sorting were:
Don't forget that proc sort has a TAGSORT option, which will make it first only sort on the by statement variables and attach everything else afterwards. Useful when having many columns not involved in the by statement.
Indexes: if you build an index of exactly the variables in your by-statement, you can use a by statement without sorting, it will rely on the index.
Split it up: you can split up the dataset in multiple chunks and sort each chunk separately. Then you do a data step where you put them all in the set statement. When you use a by statement there as well, SAS will weave the records so that the result is also according to the by-statement.
Note that these approaches have a performance hit (maybe the third one only to a lesser extent) and indexes can give you headaches if you don't take them into account later on (or destroy them intentionally).
One note if/when you would rewrite the whole join as a SAS merge: keep in mind that SAS merge does not by itself mimic many-to-many joins. (it does one-to-one, one-to-many and many-to-one) Probably not the case here (it rarely is), but i mention it to be on the safe side.