Can't download a file via os.system('scp (...)') whithin a python script launched via a init script - python-2.7

I'm trying to download a remote file using the scp command called by os.system() in a Python script (scp.py) started as a service by a procd init script.
This script is running on OpenWrt 15.05.1 (which uses BusyBox to implement the shell environment). SCP itself is handled by dropbear SSH (https://matt.ucc.asn.au/dropbear/dropbear.html)
The code is quite simple :
import os
import logging
logging.basicConfig(level=logging.DEBUG, filename='/usr/local/www/log/scp_test.log', filemode = 'w')
rc = os.system('scp -i /root/.ssh/id_rsa root#vps500141.ovh.net:/root/simon/test /tmp/hello')
error_code, signal = rc >> 8, rc & 0xFF
logging.debug('error_code -> {}'.format(error_code))
logging.debug('signal -> {}'.format(signal))
When launched in the terminal using the command "python scp.py", the program's behave well. The download is successful and produces the following output :
root#S096C08:/usr/local/bin# python scp.py
test 100% 24 0.0KB/s 00:00
root#S096C08:/usr/local/bin# cat /usr/local/www/log/scp_test.log
DEBUG:root:error_code -> 0
DEBUG:root:signal -> 0
But this program fails to download anything when launched as a service using the command /etc/init.d/scp_test start
root#S096C08:/usr/local/bin# /etc/init.d/scp_test start
root#S096C08:/usr/local/bin# cat /usr/local/www/log/scp_test.log
DEBUG:root:error_code -> 1
DEBUG:root:signal -> 0
Furthermore, a quick investigation of the syslogs using the logread command shows this :
Wed May 15 10:24:58 2019 daemon.err python[3156]: Host 'vps500141.ovh.net' is not in the trusted hosts file.
Wed May 15 10:24:58 2019 daemon.err python[3156]: (ssh-rsa fingerprint md5 41:aa:2b:57:48:be:01:81:48:a3:d0:ac:b6:56:16:34)
Wed May 15 10:24:58 2019 daemon.err python[3156]: Do you want to continue connecting? (y/n)
Wed May 15 10:24:58 2019 daemon.err python[3156]: /usr/bin/dbclient: Connection to root#vps500141.ovh.net:22 exited: Didn't validate host key
vps500141.ovh.net already have an entry in /root/.ssh/known_hosts so this is quite surprising that it is not found when the script is running.
I've also tried to add the "-o StrictHostKeyChecking=no" option to the scp command but it doesn't seems to work either.
NB : this is the content of the init script /etc/init.d/scp_test
#!/bin/sh /etc/rc.common
# Copyright (C) 2015 CZ.NIC z.s.p.o. (http://www.nic.cz/)
START=99
STOP=0
USE_PROCD=1
SCRIPT="/usr/local/bin/scp.py"
start_service() {
procd_open_instance
procd_set_param user root
procd_set_param command python "$SCRIPT"
procd_set_param stderr 1
procd_close_instance
}

Problem resolved after moving /root/.ssh/known_hosts to /.ssh/known_hosts

Related

Got a SIGABRT while executing native code - error when running xsp4 on AWS during Cloudformation

I'm getting an error when starting the xsp4 server on an AWS Ubuntu AMI. The error is initialize: mono_thread_create_internal () failed, and then:
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
The same command executes without errors on my own computer. Any idea what the error is - maybe a missing package, some configuration or something like that?
Edit: Huh, that's weird, it's working when I log in via ssh and manually run the command. I need to run it during CloudFormation however, and it fails there. Maybe something isn't fully setup yet during CloudFormation, or the user account is different? But on second thought, doesn't CloudFormation run as root?
Full log:
+ xsp4 --port 80
Cloud-init v. 0.7.9 running 'modules:final' at Sun, 22 Oct 2017 17:18:30 +0000. Up 12.78 seconds.
Cloud-init v. 0.7.9 finished at Sun, 22 Oct 2017 17:19:27 +0000. Datasource DataSourceEc2. Up 69.50 seconds
xsp4
Listening on address: 0.0.0.0
Root directory: /var/www/HelloWebApi
Listening on port: 80 (non-secure)
Hit Return to stop the server.
The server is not started.
at Mono.WebServer.ApplicationServer.Stop () in :0
at (wrapper remoting-invoke-with-check) Mono.WebServer.ApplicationServer:Stop ()
at Mono.WebServer.XSP.Server.DebugMain (System.String[] args, Boolean root, IApplicationHost ext_apphost, Boolean quiet) in :0
initialize: mono_thread_create_internal () failed
Stacktrace:
at
at (wrapper managed-to-native) System.Net.Sockets.Socket.socket_pool_queue (System.Net.Sockets.SocketAsyncCallback,System.Net.Sockets.SocketAsyncResult)
at System.Net.Sockets.Socket.QueueSocketAsyncResult (System.Collections.Generic.Queue`1,System.Net.Sockets.SocketAsyncWorker,System.Net.Sockets.SocketAsyncResult)
at System.Net.Sockets.Socket.AcceptAsync (System.Net.Sockets.SocketAsyncEventArgs)
at Mono.WebServer.ApplicationServer.RunServer ()
at System.Threading.ThreadHelper.ThreadStart_Context (object)
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool)
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool)
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object)
at System.Threading.ThreadHelper.ThreadStart ()
at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr)
Native stacktrace:
/usr/bin/mono() [0x49ff2f]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7f39c549b390]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7f39c50f5428]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7f39c50f702a]
/usr/bin/mono() [0x630409]
/usr/bin/mono() [0x63069c]
/usr/bin/mono() [0x63076f]
/usr/bin/mono() [0x58e6e5]
[0x4044ec36]
Debug info from gdb:
=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
If it works after SSH and not directly they most probably it is a TTY missing issue. If your server requires TTY then it won't start this way
As you mentioned in your response that xsp4 waits for enter key to exit and take --nonstop if you don't want the behavior, then you should use that arg. It will make sure the TTY is not required.
In case if a executable actually still requires TTY then you can look at the script command. Which does allocate a TTY and execute the command. See below for more details
http://man7.org/linux/man-pages/man1/script.1.html

remote core file analysis with eclipse

I want to use eclipse IDE as front-end GUI to debug a core file of another OS that is docker'ed in my host. With 'gdbserver' in the docker i can only debug a running process. But i need post-mortem debugging of core files.
I tried some /home/me/bin/remote_gdb wrapper that uses
docker exec -it $DOCKER_CONTAINER bash -c "gdb /usr/bin/executable /opt/crash/executable.core
This works using in a shell. But it fails using in 'eclipse' because of
Could not determine GDB version using command: remote_gdb --version
even though 'remote_gdb --version' prints out exactly the same as the original 'gdb --version'.
I could make something work with 'ddd' as a front-end - let me exercise:
> cd $HOME/SRC
> ls -l
total 4
-rw-rw-r-- 1 frank frank 96 Aug 24 18:04 test.c
> cat test.c
#include <stdio.h>
int main(int argc, char** argv)
{
printf("hello, world!\n");
return 0;
}
> docker run -v $HOME/SRC:/SRC -t fedora /bin/bash
// IN DOCKER NOW:
>> yum install gcc
[...]
>> yum install gdb
[...]
>> cd /SRC
>> gcc -g3 -O0 -o test test.c
>> ls -l
total 40
-rwxr-xr-x 1 root root 34864 Aug 24 16:26 test
-rw-rw-r-- 1 1000 1000 96 Aug 24 16:04 test.c
Back in Host:
> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e501d8384060 fedora "/bin/bash" 4 minutes ago Up 4 minutes stupefied_murdock
> cat $HOME/bin/remote_gdb
#!/bin/bash
docker exec -it e501d8384060 gdb /SRC/test
> remote_gdb # $HOME/bin/ is in $PATH
GNU gdb (GDB) Fedora 8.0-20.fc26
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /SRC/test...done.
(gdb) list
1 #include <stdio.h>
2
3 int main(int argc, char** argv)
4 {
5 printf("hello, world!\n");
6 return 0;
7 }
8
(gdb)
Switching back to the shell, where i am in the docker - see 'gdb' is running:
>> yum install procps-ng
[...]
>> ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 16:22 ? 00:00:00 /bin/bash
root 16 0 0 16:23 ? 00:00:00 bash
root 116 0 0 16:28 ? 00:00:00 gdb /SRC/test
root 131 16 0 16:30 ? 00:00:00 ps -ef
Back again on the host shell (after quit gdb session in docker):
> ddd --debugger remote_gdb
The 'ddd' plays nicely with gdb started in docker.
And it also works with core-files (add a "sleep(100);" and kill the process).
But 'ddd' doesn't have a really good GUI.
So what's the way to make such work with 'eclipse'?
I also couldn't make it work with other options: 'qtcreator', 'kdevelop', 'nemiver', 'kdbg', etc
Addendum:
It also doesn't work with 'ssh' into container.
But when i execute the same 'remote_gdb' wrapper script as 'External Tool' it can be executed(?!)
I partially could solve the issue.
The main problem was: my docker command does not provide a tty to eclipse.
As a workaround i switched over to 'ssh' into the docker image.
Moreover i had to align the file locations (executable, core-file and source-files) between host and docker image. Because using a 'gdb' wrapper script, 'eclipse' is not aware that i am debugging remotely and verifies all files for local availability.
This works for post-mortem debugging (core file analysis) and attaching a running process in docker (in general on the remote system).
However, it does not work for starting up a process in gdb via eclipse.
So used 'strace' to check for the activities between 'eclipse' and 'gdb' and found out, that for starting up a new process in 'gdb', 'eclipse' tries to apply some '--interpreter mi2 -nx' command:
inferior-tty-set --thread-group i1 /dev/pts/27
Does it mean it tries to bind a local tty to gdb?
That of course does not work remotely.
What is the reason to use "inferior-tty-set" by 'eclipse'?
It doesn't use this command for doing post-mortem debugging or attaching a running process - while for both i even have a gdb console operable in 'eclipse'
?

vtkXOpenGLRenderWindow error in a terminal in VNC

I run a vnc server on my workstation and connect with another computer. Both of the server and client are running Debian Jessie:
$ uname -a
Linux debian-VAIO 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2+deb8u2 (2016-06-25) x86_64 GNU/Linux
Xfce is installed for the vnc server and xstartup is:
$ cat ~/.vnc/xstartup
#!/bin/sh
xrdb $HOME/.Xresources
xsetroot -solid grey
export XKL_XMODMAP_DISABLE=1
exec startxfce4
In the ~/.bashrc, I have added the lines below:
export LIBGL_ALWAYS_INDIRECT=y
export LD_PRELOAD='/usr/lib/x86_64-linux-gnu/libstdc++.so.6'
Currently, when I ssh to the server with ssh -X, the $DISPLAY is localhost:10.0, and the vtk related command will launch an X window. However, if a vnc server is launched in the ssh terminal, logging into xfce and opening a terminal in it, the $DISPLAY is :1.0 and vtk fails to launch an X window, with the following message:
Error: In /home/orobix/Desktop/vmtk-build/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx, line 394
vtkXOpenGLRenderWindow (0x2c30f10): Could not find a decent visual
Error: In /home/orobix/Desktop/vmtk-build/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx, line 394
vtkXOpenGLRenderWindow (0x2c30f10): Could not find a decent visual
Error: In /home/orobix/Desktop/vmtk-build/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx, line 394
vtkXOpenGLRenderWindow (0x2c30f10): Could not find a decent visual
Error: In /home/orobix/Desktop/vmtk-build/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx, line 613
vtkXOpenGLRenderWindow (0x2c30f10): GLX not found. Aborting.
I think it might be relate to the missed config of X server in xstartup of vnc server. But I don't know how to do it. Could anyone help me debug it? Any further information will be provided if needed. Thanks!
20160823 Update
I accepted the suggestion of VirtualGL + TurboVNC and installed the two components. A simple config using vglserver_config was done according to http://www.virtualgl.org/vgldoc/2_1_1/#hd009001 part 6.1. Then I made vncserver of TurboVNC to run xfce with xstartup.turbovnc:
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
OS=`uname -s`
exec startxfce4
On the client, I use vncviewer of TurboVNC to connect the server. The desktop looks quite different from that in a default vncserver. Then I tried
/opt/VirtualGL/bin/vglrun vmtkimageviewer -ifile image_volume_voi.vti
in which the vmtkimageviewer should open up a window, but finally it gives out an error:
Executing vmtkimageviewer ...
X Error of failed request: GLXBadContext
Major opcode of failed request: 156 (GLX)
Minor opcode of failed request: 6 (X_GLXIsDirect)
Serial number of failed request: 17
Current serial number in output stream: 16
And unfortunately, I find that with ssh -X, I cannot launch the X window as I did before, although $DISPLAY is still localhost:10.0. The output is:
Executing vmtkimageviewer ...
X Error of failed request: BadValue (integer parameter out of range for operation)
Major opcode of failed request: 156 (GLX)
Minor opcode of failed request: 3 (X_GLXCreateContext)
Value in failed request: 0x0
Serial number of failed request: 37
Current serial number in output stream: 38
But I think I am closer to the fact, as it looks like VirtualGL works(?). What to do for the next?
Plain and simple the X server variant that's being used by Xrdp (either Xvnc or Xvfb) simply doesn't support OpenGL / GLX and thus programs that need OpenGL will not work in that configuration.
Fallback to Mesa swrast or llvmpipe is possible though: http://www.mesa3d.org/llvmpipe.html
Finally, I got everything done. VirtualGL only is enough for my purpose. Just install and config as the section 6.2 & 6.3 in its user's guide. Use the default vncserver and vncviewer as usual, TurboVNC is not needed. Use vglrun when you need an OpenGL support. One modification is that I should remove export LIBGL_ALWAYS_INDIRECT=y in my .bashrc.
It runs fluent in my case. #datenwolf: Thank you again!

Ubuntu, libftdi priviliges, seg fault when non-root user run program

I am trying to write simple program, that uses libftdi, and I have come across a strange problem.
When running the program as non-root it looks like this:
./BoxDriver
Naruszenie ochrony pamięci (zrzut pamięci)
And in Dmesg the last line I see is:
[ 3320.467864] BoxDriver[4205]: segfault at 0 ip 00007f05c2821f7a sp 00007ffd9c6c9c00 error 4 in libftdi.so.1.20.0[7f05c2820000+7000]
I am using Ubuntu:
Linux AdamsPC 3.19.0-30-generic #34-Ubuntu SMP Fri Oct 2 22:08:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
And FTDI device:
Bus 001 Device 011: ID 0403:6010 Future Technology Devices International, Ltd FT2232C Dual USB-UART/FIFO IC
The program runs fine, when it is running as a root:
sudo ./BoxDriver
OK, started
I have also tried to debug the source code (in eclipse ang gcc, no seg fault occurred), and the problem is with:
ftdi_usb_open(ftc, 0x0403, 0x6010)
It always returns -8, that stands for : "get product description failed"
I have searched for any answer, most are about adding user to dialout group, or adding rule to udev, but none of answers I have founded worked.
Any suggestions are highly appreciated. Thanks
There is a few different ways to fix this but generally it sounds like your user account does not have permission to interface with the USB device. You could add the user to the appropriate group I believe for ubuntu that is dialout so.
sudo usermod -a -G dialout user
Alternatively if multiple accounts need to use it but they all should have permissions you could change the filesystem it's mounted to to have full permissions.
sudo chmod 777 /media/drive_name

running vssadmin from a program

I'm trying to run the following command from a c++ program:
system("vssadmin list writers > tmp.txt ");
I'm getting this output:
"vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool (C) Copyright 2001-2005 Microsoft Corp.
Error: A Volume Shadow Copy Service component encountered an unexpected error. Check the Application event log for more information.
in the event log there are 2 errors: 22 and 8193.
if I run it directly from the command line it works fine.
Thanks.
The problem was that I was running a 32 bit application on a 64 bit machine thus windows was redirecting the app to the wrong vssadmin.
http://msdn.microsoft.com/en-us/library/aa384187
http://msdn.microsoft.com/en-us/library/aa365743