I was trying to practice with a dataset having more than 100K rows and my SAS UE shows error as out of space while trying statistical analysis,after some google search I found some solutions like extending disk space in VM and cleaning work libraries(I did clean the work library using "proc datasets library=WORK kill; run; quit;" but the issue remains same) but I am not sure how to increase the disk space, or redirecting work library to local storage in my Mac. There are no proper guidelines I have seen/understood from google search. Please help.
You can modify the cores on the VM to be 2 and increase the RAM space in the Oracle VB settings. You cannot increase the size of the VM and 100K rows should not be problematic unless you're not cleaning up processes.
Yes, SAS UE does have a tendency to not clean up after crashes so eventually if you've crashed it multiple times you'll have to reinstall to clean up. You can get around this by reassigning the work library. A quick way to do this is in projects that will be affecting it set the USER library to your myfolders or another space on your computer.
libname user '/folders/myfolders/tempWSpace';
Make sure you first create the folder under myfolders. Then any single level data set (no libname) will automatically be stored in user library and you should be ok to run your code.
Related
Basically when I do sorting or join table in sas, the sas will use resources / space from local disk C: to process the code, but since I only have 100GB left on local disk C:, It will result in error whenever SAS was out of resources.
My question is how to configure / change the setting in SAS to use resources from Local Disk E: instead, since I have larger space there.
I already looking through the forum, but found no similiar question.
Please Help.
Assuming you are talking about desktop SAS, or a server that you administer, you can control where the work and utility folders are stored in a few ways.
The best way is to use the -work and -utilloc options in your sasv9.cfg file. That file can be in a few places, but often the SAS Shortcut you open SAS with specifies it with the -CONFIG option. You can also set the option in that shortcut with -WORK or -UTILLOC command line options. The article How SAS Finds and Processes Configuration Files can help you decide the location of the sasv9.cfg you want to modify; if you are using a personal copy on your own laptop, you may change the one in the Program Files folder, but if not, or if you don 't have administrative rights, you have other places you can place a config file that will override that one.
A paper that discusses a few of these options is one by Peter Eberhardt and Mengting Wang.
One way is to set up a library named user for projects that will be time intensive and this way you get it to be dynamic as needed. When you have a library called user, that becomes the default workspace instead of work. But, you need to clean up that library manually, it won't delete data sets automatically when you're done with it.
libname user '/folders/myfolders/demo';
As #Tom indicates, you can also set an option to use a library that already exists if desired.
options user = myLib;
An advantage of this method over the config file method as it only does it for projects where it's needed, rather than your full system.
I am a transmission planning engineer and trying to automate the execution of PSSE 100 times or more at one go through a Python code. I already runs, change loads, reruns psse and write bus based summary report to *.csv file. What I really want to do is select the first active power load variable of a PSSE case and increase it by 1 MW. Then run psse, write results to a csv file. Change the selected load back to its original value and move on to the next active load to do the same again and again until I have done same for all load busses.
This will help me to calculate transmission loss factors for entire network with one go.
Thanks
#dsmtlk, if you're experienced in Python, you can readily find the information you need in the PSSE API Manual located in your PSSE program folder (mine is in C:\Program Files (x86)\PTI\PSSE33\DOCS). The API routines for getting bus data are in section 8.6. The routine for changing bus data—viz., psspy.load_data_4()—is in section 2.21.
If you're new to Python, here are a couple links I found helpful when I first started:
https://docs.python.org/2/tutorial/
http://www.tutorialspoint.com/python/
I'm using Pentaho Data Integration to create a transformation from xlsx files to mysql, but I can't import data from large files with Excel 2007 xlsx(apache POI Straiming). It gives me out of memory errors.
Did you try this option ?
Advanced settings -> Generation mode -> Less memory consumed for large excel(Event mode
(You need to check "Read excel2007 file format" first)
I would recommend you to increase jvm memory allocation before running the transformation. By default, pentaho data integration aka kettle comes with low memory allocation which would cause issues with running ETLs involving large files. You would need to modify the -Xmx value so that it specifies a larger upper memory limit in spoon.bat accordingly.
If you are using spoon in windows and edit spoon.bat in the line show below.
if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xmx512m" "-XX:MaxPermSize=256m"
If you are using kitchen or pan, edit in those pan.bat or kitchen.bat accordingly. If you are using in linux, change in .sh files.
We have default SAS workspace of x TB. We also have alternate 10X TB workspace on same server at different folder location.
Can anyone please help me with syntax that can be used in SAS EG to point to the alternate workspace instead of default one?
The SAS work directory can be changed for individuals by creating a $HOME/sasv9.cfg file and placing one line in it:
-WORK {full path to the SAS work directory}
if you are running in unix, you can change the work directory in the execution. nohup sas -work /myworkdirectory mypgm.sas &
Are you referring to the SAS work library, which is the location where SAS lets you store temporary data sets?
If so, then it depends. Are you using EG to in a client/server setup? In that setup you will have to get your SAS Admin to make changes on the server or in the SAS Metadata to point the work library for all Workspace Servers that start to use the other location that has more available space.
Would you not define SAS libraries out of these workspaces?
i.e. libname mydata '/folders/myfolders/'
This will then assign each library to your active SAS session.
Use this as precode to any manipulation your doing.
If you have Management Console, or Using PROC METADATA you can create permanent libraries.
You mentioned workspace, so I assume you need to control the WORK library.
Use the SAS system option
options work=library-specification
In the SAS documentation it states: specifies the libref or physical name of the storage space where all data sets with one-level names are stored. This library must exist.
Make sure the the file space is "close" to where the processing is done or file transfer will be a bottleneck.
I am running SAS EG which is installed in Windows but have access to GRID in AIX. Now I want to drop some of the files generated in a program (mostly csv) to a windows location.
My first question, is it possible without using fancy X or call system (guessing) routine with just one libname or filename.
Second question is will it be possible to drop generated files from EG session in the windows directory.
If you have SAS/CONNECT licensed, and are able to run a local SAS session as well as the connection to UNIX, you can use PROC UPLOAD or DOWNLOAD, or even better, you can define a libname like so:
libname ulib server=unix slibref=work;
server=unix defines which remote session you are accessing (you may have multiple simultaneously) and slibref=work defines which server-side libname you are accessing. Change both values as needed. This requires having an open connection to the server (via SAS/CONNECT), and would be executed in your local session.
If you do NOT have SAS/CONNECT licensed, and/or cannot start a local session on your desktop, you have a few options.
First off, if the UNIX server can mount the windows location as a drive, then you can simply write to it as if it were on the UNIX machine. This is likely if it is on a shared drive on your NAS or similar; however, if by 'windows location' you mean 'on my desktop machine', it's unlikely.
Second, you may be able to use an EG add-on that allows you to do this. If you're able to install it, it's a simple point and click interface. It is NOT scriptable in SAS (it may be scriptable via automation, I'm not sure). Chris Hemedinger made it available in his blog (currelt(http://blogs.sas.com/content/sasdummy/2012/12/06/copy-files-in-sas-eg/); it is called "Copy Files" if that link breaks and you need to search for it. The blog post has the add-on and instructions for installing it.