I have an iPython notebook that has blocks of code of which I want to be able to have the opportunity to stop at points whilst it is executing.
The code downloads a query form an API and on the first return determines the maximum rows and whether it contains sampled data.
Depending if it has sampled data, and to what extent, I would like the option to stop the code prior to the while loop which will iteratively download the full dataset.
The 'break' code I have is as below:
print "Contains sampled data? " + str(metadata["containsSampledData"])
print "Break query? -> Y"
bi = raw_input()
if bi == "Y":
break
I have tried return and break, which both give errors (though they do stop the code). In addition, I have tired quit() however I am not wanting to reset the kernal and re-import modules/create variables.
Is there any way, outside of a loop and not within a function that you can stop the code, without quitting the instance?
I think you need to rethink and rewrite the logic of your program.
print "Contains sampled data? " + str(metadata["containsSampledData"])
while True:
# Check the condition with your API stuff here
# and exit or ask everytime if the condition is true as shown below
print "Break query? -> Y"
bi = raw_input()
if bi == "Y":
break
# Continue executing your code here.
The other option is to use sys.exit() as you already tried but it will exit the program.
Related
I'm writing a program which updates MySQL tables every few seconds as new data is received. The function has an infinite loop to update constantly, with another loop inside it that iterates through table rows and picks data accordingly.
Unfortunately, no matter what I do, I encounter the most pathetic error:
Can't create more than max_prepared_stmt_count statements
I've tried simply deleting all mysql related values (except con and driver) after each iteration with statements seen in the example code here.
I've tried adding
stmt = NULL;
stmt->close();
Just in case before every value.
I've tried rewriting the program with auto_ptr, which allegedly clean up automatically, based on this answer.
I have even tried closing and reestablishing connection to the MySQL server after each iteration, but even that did not help me.
Please help.
More information on your loop may be required.
But it sounds like your stmt = con->createStatement(); is inside your loop. You only need one connection, before your loop starts.
Then you can execute a statment multiple times in your loop, res = stmt->executeQuery("SELECT 'Hello World!' AS _message");
When your application terminates then you close your stmt, delete stmt;
From mysql.com "The default value is 16,382." mysql.com-sysvar_max_prepared_stmt_count Which it sounds like you only require one.
I have some code that calculates the price of a stock option using Monte Carlo and returns a discounted price. The final few lines of the relevant method look like this:
if(payoffType == pt.LongCall or payoffType == pt.LongPut):
discountedPrice=discountedValue
elif(payoffType == pt.ShortCall or payoffType == pt.ShortPut):
discountedPrice=(-1.0)*discountedValue
else:
raise Exception
#endif
print "dv:", discountedValue, " px:", discountedPrice
return discountedPrice
At a higher level of the program, I create four pricers, which are passed to instances of a portfolio class that calls the price() method on the pricer it has received.
When I set the breakpoint on the if statement or the print statement, the breakpoints work as expected. When I set the breakpoint on the return statement, the breakpoint is interpreted correctly on the first pass through the pricing code, but then skipped on subsequent passes.
On occasion, if I have set a breakpoint somewhere in the flow of execution between the first pass through the pricing code and the second pass, the breakpoint will be picked up.
I obviously have a workaround, but I'm curious if anyone else has observed this behavior in the PyDev debugger, and if so, does anyone know the root cause?
The issues I know of are:
If you have a StackOverflowError anywhere in the code, Python will disable the tracing that the debugger uses.
I know there are some issues with asynchronous code which could make the debugger break.
A workaround is using a programmatic breakpoint (i.e.: pydevd.settrace -- the remote debugger session: http://www.pydev.org/manual_adv_remote_debugger.html has more details on it) -- it resets the tracing even if Python broke it in a stack overflow error and will always be hit to (the issue on asynchronous code is that the debugger tries to run with untraced threads, but sometimes it's not able to restore it on some conditions when dealing with asynchronous code).
I'm trying to write a GDB script to do instruction tracing in a bounded maner (i.e start addr and stop addr). Perhaps I'm failing at google but I cant seem to find this in existence already.
Here is my stab at it:
python
def start_logging():
gdb.execute("set logging on")
gdb.execute("while $eip != 0xBA10012E9")
gdb.execute("x/1i $eip")
gdb.execute("stepi")
gdb.execute(" end")
gdb.execute("set logging off")
gdb.execute("set pagination off")
gdb.execute("break *0xBA19912CF")
gdb.execute("command 1 $(start_logging())")
gdb.execute("continue")
In my mind this should set up a breakpoint then set the command to run when it hits. When the breakpoint hits it should single step through the code until the end address is hit and then it will turn off logging.
When I run this with gdb the application will break at the correct point but no commands are run.
What am I doing wrong? Sorry if this is the wrong way to go about this please let me know. I'm new to gdb scripting
I see a few odd things in here.
First, it looks like you are trying to split multi-line gdb commands across multiple calls to gdb.execute. I don't believe this will work. Certainly it isn't intended to work.
Second, there's no reason to try to do a "while" loop via gdb.execute. It's better to just do it directly in Python.
Third, I think the "command" line seems pretty wrong as well. I don't really get what it is trying to do, I guess call start_logging when the breakpoint is hit? And then continue? Well, it won't work as written.
What I would suggest is something like:
gdb.execute('break ...')
gdb.execute('run')
while gdb.parse_and_eval('$eip') != 0x...:
gdb.execute('stepi')
If you really want logging, either do the 'set logging' business or just instruct gdb.execute to return a string and log it from Python.
I'm developing a Windows Console Application, using Visual Studio 2010, which prints progress percentage on screen such that the last number gets overwritten with the current number. I'm using carriage return to achieve this like so:
std::wcout
<< "[Running: "
<< std::setw(3)
<< std::setprecision(3)
<< percent
<< "%]"
<< '\r'
;
When I execute this program through PowerShell on a remote host using PSSesion:
Enter-PSSession WWW.XXX.YYY.ZZZ -credential <UserName>
[WWW.XXX.YYY.ZZZ]: PS C:\> .\test.exe
it behaves differently compared to when its executed locally either in PowerShell or in cmd.exe. I face following three problems:
1) The console output of the program is different, in that, instead of overwriting [Running xx%] its printing in the next line like so:
[Running 1%]
[Running 2%]
[Running 3%]
[Running 4%]
...
This is similar to what happens when output of a program is redirected to a file (lone carriage returns or newlines are replaced with carriage returns+newlines combinations).
2) The output doesn't shows up as and when the program writes something to cout. It comes at once at the end of execution. Does powershell caches the output of remote program and sends to the caller only once? If there is significant time difference between two lines (line meaning all the output between two newlines) then the first line gets printed as if it waits for some time for another newline and if received, it again goes to waiting state, if not received within certain time period (~500ms), it sends the output till last newline (and not all the accumulated output) to the caller. As can be seen from my code, there is no newline resulting in all the [Running xx%] being printed at once at the end.
3) At certain point in the program I need user's confirmation, but cin.fail() returns true in remote execution. So, is there any way to take user input in such an execution environment? Or is there any way to detect that the program is being executed remotely through Powershell (eg. some env variable)?
Any help related to any point will be greatly appreciated. :)
With respect to 1), the WinRM protocol for PowerShell Remoting is really just an input/output stream for commands and their serialized output. It has no terminal emulation capabilities, so when you run things remotely, their output is fed back sequentially one line at a time. If you used SSH or Telnet to a remote shell of powershell.exe, you would likely see what you expect.
As for 2), the default output buffermode appears to be "Block" so this would explain the behaviour you're seeing. You can change this with the New-PSSessionOption cmdlet to create a configuration for a session created by New-PSSession that can be passed to the -Session parameter on remoting aware cmdlets. The option is OutputBufferingMode. If you set it to None, you should get the desired behaviour.
# default output buffering mode is "none" for default options
$s = new-pssession localhost -sessionoption (new-pssessionoption)
enter-pssession $s
For 3), you should be able to accept input interactively IIRC. Looks like interactive stuff doesn't work right - I may have completely misremembered this.
I'm running several simulations using Condor and have coded the program so that it outputs a progress status in the console. This is done at the end of a loop where it simply prints the current time (this can also be percentage or elapsed time). The code looks something like this:
printf("START");
while (programNeedsToRum) {
// Run code repetitive code...
// Print program status update
printf("[%i:%i:%i]\r\n", hours, minutes, seconds);
}
printf("FINISH");
When executing normally (i.e. in the terminal/cmd/bash) this works fine, but the condor nodes don't seem to printf() the status. Only once the simulation has finished, all the status updates have been outputted to the file but then it's no longer of use. My *.sub file that I submit to condor looks like this:
universe = vanilla
executable = program
output = out/out-$(Process)
error = out/err-$(Process)
queue 100
When submitted the program executes (this is confirmed in condor_q) and the output files contain this:
START
Only once the program has finished running its corresponding output file shows (example):
START
[0:3:4]
[0:8:13]
[0:12:57]
[0:18:44]
FINISH
Whilst the program executes, the output file only contains the START text. So I came to the conclusion that the file is not updated if the node executing program is busy. So my question is, is there a way of updating the output files manually or gather any information on the program's progress in a better way?
Thanks already
Max
What you want to do is use the streaming output options. See the stream_error and stream_output options you can pass to condor_submit as outlined here: http://research.cs.wisc.edu/htcondor/manual/current/condor_submit.html
By default, HTCondor stores stdout and stderr locally on the execute node and transfers them back to the submit node on job completion. Setting stream_output to TRUE will ask HTCondor to instead stream the output as it occurs back to the submit node. You can then inspect it as it happens.
Here's something I used a few years ago to solve this problem. It uses condor_chirp which is used to transfer files from the execute host to the submitter. I have a python script that executes the program I really want to run, and redirects its output to a file. Then, periodically, I send the output file back to the submit host.
Here's the Python wrapper, stream.py:
#!/usr/bin/python
import os,sys,time
os.environ['PATH'] += ':/bin:/usr/bin:/cygdrive/c/condor/bin'
# make sure the file exists
open(sys.argv[1], 'w').close()
pid = os.fork()
if pid == 0:
os.system('%s >%s' % (' '.join (sys.argv[2:]), sys.argv[1]))
else:
while True:
time.sleep(10)
os.system('condor_chirp put %s %s' % (sys.argv[1], sys.argv[1]))
try:
os.wait4(pid, os.WNOHANG)
except OSError:
break
And my submit script. The problem ran sh hello.sh, and redirected the output to myout.txt:
universe = vanilla
executable = C:\cygwin\bin\python.exe
requirements = Arch=="INTEL" && OpSys=="WINNT60" && HAS_CYGWIN==TRUE
should_transfer_files = YES
transfer_input_files = stream.py,hello.sh
arguments = stream.py myout.txt sh hello.sh
transfer_executable = false
It does send the output in its entirety, so take that in to account if you have a lot of jobs running at once. Currently, its sending the output every 10 seconds .. you may want to adjust that.
with condor_tail you can view the output of a running process.
to see stdout just add the job-ID (and -f if you want to follow the output and see the updates immediately. Example:
condor_tail 314.0 -f