I have a script that generates command line invocations of rrdtool xport based on input provided in a domain specific language. This works well, until the number of DEFs in the command line exceed a certain number - it seems to be around 50. At that point the command simply returns without any output or error information.
Is there a limit on the number of DEFs in rrdtool export? If so, then can it be raised or circumvented?
The issue turned out to be the character limit on the command line sent to the shell via Python's os.system method call. The issue can be worked around by creating a temporary executable script, writing the command line to the script and executing it.
Related
I am basically new to the qgis and pyqgis, I have been given a task to write a python script where I should print the value of measured by the measureline tool into python console. So far I am able to get function iface.actionMeasure().trigger() this can be used to trigger the measure line tool after measuring the distance I want to print the value into the python console.
I would like to have same option as mentioned in this question: How to display full output in Jupyter, not only last result? but for AWS EMR's jupyterhub's pyspark kernel (Spark 2.4.4). It works with python3 (python3.6) kernel.
It works if I use print statements, but in that case, it doesn't work if last step is failed, then it will only show result for the failed step as shown in the image below.
Also, to note, not sure if it is related, but, below code doesn't run in sync i.e. print wait print wait...., but, it just prints everything at once at the end.
import time
for i in range(0,10):
print(i)
time.sleep(2)
Just adding the question from the referred post, if in case the referred question/post gets deleted or changes.
I want Jupyter to print all the interactive output without resorting to print, not only the last result. How to do it?
Example :
a=3
a
a+1
I would like to display
3
4
The print statement output goes to the stdout or stderr on the computer that is running a spark executor.
Considering you have a big cluster having n workers(each storing partition of an RDD or DataFrame). Is is hard to expect the ordered output in a job (for instance map). This may be considered a design choice as well for spark itself. Where will that data be printed out? since nodes are running code in parallel, which of them will be printed first?
So, we dont have interactive print statements inside jobs. These whole thing can also remind you about why we had accumulators and broadcast variables.
So, I will advice you to use logs generated by steps instead and work with logs. To view logs in Amazon S3, cluster logging must be enabled (which is the default for new clusters). View Log Files Archived to Amazon S3.
For your second question about sleep() and print, python is line buffered, which forces it to wait for a newline before printing to stdout. If the output is not a console, then even newline won't trigger a flush.
You can force the behaviour as
import time
for i in range(0,10):
print(i,flush=True)
time.sleep(2)
I am running a huge program in SAS and am using proc printto in order to save the log elsewhere. This works for about ten hours, and then SAS switches itself back to the normal log window. It was printing to a notepad file, which only maxed out at 6718KB.
Anyone know why SAS reverts to printing in the log window? Is it just me?
Thanks!!
One workaround for this would be to execute your program in batch. In batch mode, there are no log limits, as the log is automatically printed out to a file (by default, a file that is in the same location as your .sas program, with the same name but .log). Listing similarly will go to the same location .lst.
This would avoid log length limitations. If you want your log printed to a specific place, you can use the -altlog startup option.
I have 1000+ .sas files that I am trying to run in batch (all the code in each file is on one line), but SAS truncates the line to 256 characters and the code fails. Running each file individually (outside the batch) works fine.
Is there a way around this that would not force me to open each file and manually change the length of each line to something SAS can handle?
Prior to SAS 9.2, you can't extend the line limit in a batch file. You can extend the line limit in an %include, though, using the LRECL option; one option is to do so and have your batch submission all in one file with a bunch of %includes.
If you are running SAS 9.2 or later, you have the LRECL system option, which allows up to 32767 characters per line. See this page for more information.
Does anyone know how to use the "execute selection" function in the do-file editor of Stata for code that spans multiple lines?
Currently I can't find a way to do this without using the #delimit ; system which requires repeating "delimit ;" at the beginning of every block I want to run.
Any suggestions appreciated!
I believe that you might be understanding the #delimit ; command wrongly: this is useful when you are coding a do-file to execute it in its entirety afterwards. I also assume that you are using Stata 11, since previous versions behave differently (if I recall well, Stata 10 SE for Mac does not support // comments and delimiting, for example).
If you are executing only a fraction of the code, use /// at the end of a line to continue its command on the next one.
Basic example (that will clear any open data, so beware):
sysuse lifeexp, clear
sc lexp safewater, ///
mlab(country);
This should run flawlessly even if you execute the sysuse command and the sc (scatter) commands separately. The sc command has the mlab option (to add labels to the data points) on a different line, but both lines will be interpreted as only one command due to the /// indication.
Hope this helps!