I am trying to replicate the ventilator/workers/sink paradigm described in the ZMQ guide. I have the same Python Ventilator, the same C++ worker as, and the same Python Sink as was described in the ZMQ examples. I want to launch the ventilator, workers, and sink from one main python script, so I created "class" wrappers around the ventilator & sink, and both of those classes subclass the Python module "multiprocessing.Process." Since the C++ is a binary, I launch it with Python's subprocess.Popen call.
The order of starting all of this up is as follows:
h = subprocess.Popen('test') # test is the name of the binary
time.sleep(1)
s = sinkObj.start()
time.sleep(1)
v = ventObj.start()
What I am finding is that no data is getting through the system when I start up the components like this. However, if I start the C++ binary in its own shell, and only start the sinkObj and ventObj from the main python script, it works fine.
I apologize in advance if this is more of a Python question than a ZMQ question, but I haven't run into issues like this w/ Python's subprocess. I have also tried using os.system() instead of the subprocess... but same issue. I put all the code on this website: https://github.com/kkarrancsu/zmqtest if anybody is curious to test it out. There is a readme on that git which tells you what the files are.
Any ideas on why this could be happening?
------------------------- UPDATE --------------------
I found that if I create a shell script which simply launches the C binary, and call that shell script w/ os.system('run_the_shell_script') it works! So this means that there is something wrong with the way that I am using subprocess.Popen(...), but can't seem to pinpoint what the issue is. I tried w/ the shell=True flag, but it still hangs with that...
It's the name of the worker binary file that causes the problem.
There two solutions:
Chang the name of the binary file test to test_new and do the same in your All.py file, and then it will work as you desire.
Substitute subprocess.Popen('./test', shell=True) for subprocess.Popen('test', shell=True).
test is Linux command. If you type the following in your shell
$ echo $PATH
You may see that . is at the last position. It means that until shell couldn't find the binary file to be executed in the directories that $PATH indicates, it will try to search for it in the current directory .
When you execute subprocess.Popen('test', shell=True), it could find it before trying the . directory and so it won't execute the workers.
As I see, the ventilator and sink bind() to ports 6557 and 6558, and the C++ app connect() to these ports. In this case, if you start the cpp app first, it will try to connect() to the endpoints, but as nothing is bound there, it will drop silently.
In ZeroMQ the basic principle is "First Bind, then Connect". So you should not connect() before you bind() something on the socket. Imagine bind() is the 'Server', and connect() is the client. You cannot connect client to non-existing server. Also, in ZeroMQ every socket can be 'Server', but you SHOULD HAVE only 1 bind()-ing socket per URL. And you can have multiple 'connect()'s.
Related
I am working with a command line tool called 'ideviceinfo' (see https://github.com/libimobiledevice) to help me to quickly get back serial, IMEI and battery health information from the iOS device I work with daily. It executes much quicker than Apple's own 'cfgutil' tools.
Up to know I have been able to develop a more complicated script than the one shown below in PyCharm (my main IDE) to assign specific values etc to individual variables and then to use something like to pyclip and pyautogui to help automatically paste these into the fields of the database app we work with. I have also been able to use the simplified version of the script both in Mac OS X terminal and in the python shell without any hiccups.
I am looking to use AppleScript to help make running the script as easy as possible.
When I try to use Applescript's "do shell script 'python script.py'" I just get back a string of lenght zero when I call 'ideviceinfo'. The exact same thing happens when I try to build an Automator app with a 'Run Shell Script' component for "python script.py".
I have tried my best to isolate the problem down. When other more basic commands such as 'date' are called within the script they return valid strings.
#!/usr/bin/python
import os
ideviceinfoOutput = os.popen('ideviceinfo').read()
print ideviceinfoOutput
print len (ideviceinfoOutput)
boringExample = os.popen('date').read()
print boringExample
print len (boringExample)
I am running Mac OS X 10.11 and am on Python 2.7
Thanks.
I think I've managed to fix it on my own. I just need to be far more explicit about where the 'ideviceinfo' binary (I hope that's the correct term) was stored on the computer.
Changed one line of code to
ideviceinfoOutput = os.popen('/usr/local/bin/ideviceinfo').read()
and all seems to be OK again.
I have an application that I turned into a simple Native Client App a year ago, and I've been trying to get it running again. However, when I try to run it, or any of the example VS projects, the web server fails to start, giving me usage hints for httpd.py, and saying "httpd.py: error: unrecognized arguments: 5103".
I wasn't able to find anything about this on the NaCL guide or on the net. I could probably troubleshoot the issue if I could see the script that starts the webserver, but I have no idea where this is stored.
The script that start the server is 'nacl_sdk\pepper_43\tools\httpd.py'. The problem is with the port argument being formated incorrectly.
The expected format is:
httpd.py [-h] [-C SERVE_DIR] [-p PORT] [--no-dir-check]
But, the received arguments formatted by the add-in is:
['--no_dir_check', '5103']
where the port prefix is missing and should be '-p 5103'
For a quick fix, add the following line
parser.add_argument('args', nargs=argparse.REMAINDER)
before the parse_args(args) in the main(args) method in httpd.py.
This will keep the unknown arguments from being parsed and will use the default value for port instead (5103).
I have a software made of two halves: one is python running on a first pc, the other is cpp running on a second one.
They communicate through the serial port (tty).
I would like to test the python side on my pc, feeding it with the proper data and see if it behaves as expected.
I started using subprocess but then came the problem: which stdin and stdout should I supply?
cStringIO does not work because there is no fileno()
PIPE doesn't work either because select.select() says there is something to read even if nothing it's actually sent
Do you have any hints? Is there a fake tty module I can use?
Ideally you should mock that out and just test the behavior, without relying too much on terminal IO. You can use mock.patch for that. Say you want to test t_read:
#mock.patch.object(stdin, 'fileno')
#mock.patch.object(stdin, 'read')
def test_your_behavior(self, mock_read, mock_fileno):
# this should make select.select return what you expect it to return
mock_fileno.return_value = 'your expected value'
# rest of the test goes here...
If you can post at least part of the code you're trying to test, I can maybe give you a better example.
I'm trying to create a new dir via SSH with a python script. When i try my commands by using the Python command line it just works. But when I try to do the same by a script it does not create the new 'test' folder (I even copy/paste the commands in the script into the Python cmd to verify they are right and there they work). So any ideas why it does not work by script?
The used code:
child = pexpect.spawn('ssh 192.168.56.101 -oStrictHostKeyChecking=no')
child.expect=('password:')
child.sendline('MyPwd')
child.sendline('mkdir /home/myUser/Desktop/test')
Seems to work when I just add another line
for example
child.sendline('\n')
so the entire script is
child = pexpect.spawn('ssh 192.168.56.101 -oStrictHostKeyChecking=no')
child.expect=('password:')
child.sendline('MyPwd')
child.sendline('mkdir /home/myUser/Desktop/test')
child.sendline('\n')
What I usually do to solve this issue is sync-ing with host machine. After I send something to the machine, I expect an answer, which usually translates in the machine's prompt. So, in your case, I would go for something like this:
child = pexpect.spawn('ssh 192.168.56.101 -oStrictHostKeyChecking=no')
child.expect('password:')
child.sendline('MyPwd')
child.expect('YourPromptHere')
child.sendline('mkdir /home/myUser/Desktop/test')
child.expect('YourPromptHere')
You can just replace YourPromptHere with the prompt of the machine, if you are running the script on a single target, or with a regular expression (eg. "(\$ )|(# )|(> )").
tl;dr : To summarize what I said, you need to wait until the previous action was finished until sending a new one.
I have been having problems trying to run an external program from a python program that was generated from a trigger in a postgres 9.2 database. The trigger works. It writes to a file. I had tried just running the external program but the permissions would not allow it to run. I was able to create a folder (using os.system(“mkdir”) ). The owner of the folder is NETWORK SERVICE.
I need to run a program called sdktest. When I try to run it no response happens so I think that means that the python program does not have enough permissions (with an owner of NETWORK SERVICE) to run it.
I have been having my program copy files that it needs into that directory so they would have the correct permissions and that has worked to some degree but the program that I need to run is the last one and it is not running because it does not have enough permissions.
My python program runs a C++ program called PG_QB_Connector which calls sdktest.
Is there any way I can change the owner of the process to be a “normal” owner? Is there a better way to do this? Basically I just need to have this C++ program have eniough perms to run correctly.
BTW, when I run the C++ program by hand, the line that runs the sdktest program runs correctly, however, when I run it from the postgres/python it does not do anything...
I have Windows 7, python 3.2. The other 2 questions that I asked about this are located here and here
The python program:
CREATE or replace FUNCTION scalesmyone (thename text)
RETURNS int
AS $$
a=5
f = open('C:\\JUNK\\frompython.txt','w')
f.write(thename)
f.close()
import os
os.system('"mkdir C:\\TEMPWITHOWNER"')
os.system('"mkdir C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\JUNK\\junk.txt C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\BATfiles\\junk6.txt C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\BATfiles\\run_addcust.bat C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\Workfiles\\PG_QB_Connector.exe C:\\TEMPWITHOWNER\\addcustomer"')
os.system('"copy C:\\Workfiles\\sdktest.exe C:\\TEMPWITHOWNER\\addcustomer"')
import subprocess
return_code = subprocess.call(["C:\\TEMPWITHOWNER\\addcustomer\\PG_QB_Connector.exe", '"hello"'])
$$ LANGUAGE plpython3u;
The C++ program that is called from the python program and calls sdktest.exe is below
command = "copy C:\\Workfiles\\AddCustomerFROMWEB.xml C:\\TEMPWITHOWNER\\addcustomer\\AddCustomerFROMWEB.xml";
system(command.c_str());
//everything except for the qb file is in my local folder
command = "C:\\TEMPWITHOWNER\\addcustomer\\sdktest.exe \"C:\\Users\\Public\\Documents\\Intuit\\QuickBooks\\Company Files\\Shain Software.qbw\" C:\\TEMPWITHOWNER\\addcustomer\\AddCustomerFROMWEB.xml C:\\TEMPWITHOWNER\\addcustomer\\outputfromsdktestofaddcust.xml";
system(command.c_str());
It sounds like you want to invoke a command-line program from within a PostgreSQL trigger or function.
A usually-better alternative is to have the trigger send a NOTIFY and have a process with a PostgreSQL connection LISTENing for notifications. When a notification comes in, the process can start your program. This is the approach I would recommend; it's a lot cleaner and it means your program doesn't have to run under PostgreSQL's user ID. See NOTIFY and LISTEN.
If you really need to run commands from inside Pg:
You can use PL/Pythonu with os.system or subprocess.check_call; PL/Perlu with system(); etc. All these can run commands from inside Pg if you need to. You can't invoke programs directly from PostgreSQL, you need to use one of the 'untrusted' (meaning fully privileged, not sandboxed) procedural languages to invoke external executables. PL/TCL can probably do it too.
Update:
Your Python code as shown above has several problems:
Using os.system in Python to copy files is just wrong. Use the shutil library: http://docs.python.org/3/library/shutil.html to copy files, and the simple os.mkdir command to create directories.
The double-layered quoting looks wrong; didn't you mean to quote only each argument not the whole command? You should be using subprocess.call instead of os.system anyway.
Your final subprocess.call invocation appears OK, but fails to check the error code so you'll never know if it went wrong; you should use subprocess.check_call instead.
The C++ code also appears to fail to check for errors from the system() invocations so you'll never know if the command it runs fails.
Like the Python code, copying files in C++ by using the copy shell command is generally wrong. Microsoft Windows provides the CopyFile function for this; equivalents or alternatives exist on other platforms and you can use portable-but-less-efficient stream copying too.