Eclipse PTP hangs starting debug-session of an OpenMPI application - c++

Question

I want to use eclipse to debug some mpi code. The Eclipse PTP version looked well for the job, but I have problems with the debugger. I failed to get the debugger working (as bugfree program i used the included example of "MPI Pi C Project" for testing and try to debug it with two threads on my linux desktop). The actual symptom is that it hangs after the application was compiled showing me a "Operation in progress ..." message in the parallel debugging perspective.
My steps so far (or my actual try to get it running):
Installing new Eclipse PTP
Installing ISP (www.cs.utah.edu/formal_verification/ISP-release/ )
Installing PTP SDM (this should be not necessary, because the PTP-plugin has an
integrated SDM)
Then i configured MPI, SDM and ISP in the
"Preferences->Parallel Tools" dialog
Now the project compiles without
errors.
I add a "Open MPI-Generic-Interactive" run configuration with
an environment variable LD_LIBRARY_PATH=/usr/lib64/mpi/gcc/openmpi/lib (the variable is not set on the workstation and later it could be removed easy this way)
Now the program runs without errors with the expected output started by eclipse.
As a
next step i start the debugger with two threads and the program will
be compiled and eclipse ask me if i want to switch perspective. Then
a message window appears telling me "Operation in progress ..." and a
progress bar hangs at 20% for more than an hour (in one case).
So far I tried different solutions and debugging approachs:
I used Eclipse PTP-wiki and help.eclipse.org/kepler/topic/org.eclipse.ptp.doc.user/html/toc.html to get a little bit information on it.
I removed the LD_LIBRARY_PATH variable, now the executable wont run (if you run it inside of eclipse you get a "error while loading shared libraries: libmpi.so.1: cannot open shared object file" message and the program ends). In this case the message is not thrown when you start the debugger, so it should be that mpirun is never called by the debugger.
When i cancel the "Operation in progress ..."-message i get an error "Error completing debug job launch. Reason: Cannot connect to debugger." So i tried to google that error message, but i found only some threads like this "dev.eclipse.org/mhonarc/lists/ptp-user/msg01183.html", most of them to old and with the solution that ssh needs to be installed (this should not be the problem, i am able to connect from other hosts to the workstation using ssh, so server and client must work fine. Additional they use much older versions of PTP, because the problem to configure resource managers is now in the run/debug dialog)
I installed the optional RDT-Server and RDT-Tools but no reaction (i think this should not be the problem, but it was worth a try)
I found some questions here, but no solution.
Any suggestions how to solve that?
As appendix the debugger configuration (out of eclipse):
25524=RUNNING
25659=RUNNING
allAttributes=
allocator=
allocator_base_verbose=0
backtrace=
backtrace_base_verbose=0
bml=bucket
bml_base_verbose=67108864
bml_r2_show_unreach_errors=0
btl=0
btl_base_verbose=
btl_base_warn_component_unused=0
btl_self_bandwidth=4
btl_self_eager_limit=5
btl_self_exclusivity=5
btl_self_flags=3
btl_self_free_list_inc=64
btl_self_free_list_max=-1
btl_self_free_list_num=4
btl_self_latency=-1
btl_self_max_send_size=4
btl_self_min_rdma_pipeline_size=bucket
btl_self_rdma_pipeline_frag_size=128
btl_self_rdma_pipeline_send_length=4
btl_self_rndv_eager_limit=4
btl_sm_bandwidth=4
btl_sm_eager_limit=5
btl_sm_exclusivity=64
btl_sm_flags=3
btl_sm_have_knem_support=64
btl_sm_knem_dma_min=4
btl_sm_knem_max_simultaneous=-1
btl_sm_latency=128
btl_sm_max_send_size=4
btl_sm_rndv_eager_limit=4
btl_sm_use_knem=10
btl_tcp_bandwidth=stderr
btl_tcp_eager_limit=4
btl_tcp_endpoint_cache=4
btl_tcp_exclusivity=3
btl_tcp_flags=4
btl_tcp_if_exclude=0
btl_tcp_if_include=bucket
btl_tcp_if_seq=
btl_tcp_latency=0
btl_tcp_max_send_size=4
btl_tcp_min_rdma_pipeline_size=-1
btl_tcp_port_min_v4=64
btl_tcp_port_range_v4=20
btl_tcp_rdma_pipeline_frag_size=bucket
btl_tcp_rdma_pipeline_send_length=128
btl_tcp_rndv_eager_limit=5
btl_tcp_use_nagle=-1
byNodeOption=false
bySlotOption=false
carto=40
carto_auto_detect_priority=0
carto_base_verbose=
carto_file_path=
carto_file_priority=11
coll=0
coll_base_verbose=
coll_basic_crossover=50
coll_basic_priority=0
coll_hierarch_bcast_alg=
coll_hierarch_detection_alg=0
coll_hierarch_ignore_sm=30
coll_hierarch_priority=
coll_hierarch_segment_size=0
coll_hierarch_use_rdma=0
coll_hierarch_verbose=0
coll_inter_priority=10
coll_inter_verbose=4
coll_sm_comm_in_use_flags=0
coll_sm_comm_num_segments=2
coll_sm_control_size=0
coll_sm_fragment_size=0
coll_sm_info_num_procs=32768
coll_sm_priority=0
coll_sm_shared_mem_used_data=40
coll_sm_tree_degree=4
coll_sync_barrier_after=0
coll_sync_barrier_before=75
coll_sync_priority=0
coll_tuned_init_chain_fanout=8
coll_tuned_init_tree_fanout=2
coll_tuned_pre_allocate_memory_comm_size_limit=8192
coll_tuned_priority=4096
coll_tuned_use_dynamic_rules=4
control.address=localhost
control.user.name=Sebastian
control.working.dir=/home/Sebastian
current_controller=Basic.Options
debuggerArgs=--port=51091 --host=localhost --debugger=gdb-mi --debugger_path=/usr/bin/gdm --debug=0
debuggerExecutablePath=/usr/local/bin/sdm
debuggerId=org.eclipse.ptp.debug.sdm
disabled=true
dpm=0
dpm_base_verbose=none
enabled_Basic.Options=byNodeOption bySlotOption hostFileOption hostListOption mpiNumberOfProcesses noLocalOption noOversubscribeOption prefixOption
errmgr=
errmgr_base_verbose=0
errmgr_default_priority=0
ess=
ess_base_verbose=0
ess_env_priority=0
ess_hnp_priority=0
ess_singleton_priority=0
ess_slave_priority=0
ess_slurm_priority=0
ess_slurmd_priority=0
ess_tool_priority=0
executableDirectory=/home/Sebastian/testW/Test/Debug
executablePath=/home/Sebastian/testW/Test/Debug/Test
extraArgs=
filem=
filem_base_verbose=0
filem_rsh_cp=cp
filem_rsh_max_incomming=10
filem_rsh_max_outgoing=10
filem_rsh_priority=20
filem_rsh_rcp=scp
filem_rsh_rsh=ssh
filem_rsh_verbose=0
grpcomm=
grpcomm_bad_priority=0
grpcomm_base_verbose=0
grpcomm_basic_priority=0
grpcomm_hier_priority=0
hostFileOption=false
hostListOption=false
hwloc=0
hwloc_base_mem_alloc_policy=
hwloc_base_mem_bind_failure_action=0
hwloc_base_verbose=0
invalid_Basic.Options=hostFile hostList prefix script_path
io=1000
io_base_freelist_increment=50
io_base_freelist_initial_size=4
io_base_freelist_max_size=548864
io_base_verbose=0
io_romio_complete_configure_params=0
io_romio_delete_priority=32768
io_romio_priority=30
io_romio_user_configure_params=4
io_romio_version=4
iof=262144
iof_base_verbose=2147483647
launchMode=run
local_prefix=
maffinity=30
maffinity_base_verbose=40
maffinity_first_use_priority=0
maffinity_hwloc_priority=
mca_base_param_file_path=/usr/lib64/mpi/gcc/openmpi/share/openmpi/amca-param-sets:/home/Sebastian
mca_base_param_file_path_force=
mca_base_param_file_prefix=
mca_component_disable_dlopen=0
mca_component_path=/usr/lib64/mpi/gcc/openmpi/lib64/openmpi:/home/Sebastian/.openmpi/components
mca_component_show_load_errors=1
mca_param_files=/home/Sebastian/.openmpi/mca-params.conf:/etc/openmpi-mca-params.conf
mca_verbose=stderr
memchecker=0
memory=
memory_base_verbose=
memory_linux_disable=-1
memory_linux_ptmalloc2_available=0
memory_linux_ptmalloc2_enable=0
memory_linux_ummunotify_available=1
memory_linux_ummunotify_enable=-1
mpiNumberOfProcesses=2
mpi_abort_delay=0
mpi_abort_print_stack=0
mpi_event_tick_rate=-1
mpi_have_sparse_group_storage=0
mpi_keep_peer_hostnames=1
mpi_leave_pinned=-1
mpi_leave_pinned_pipeline=0
mpi_no_free_handles=0
mpi_notify_init_finalize=1
mpi_paffinity_alone=0
mpi_param_check=1
mpi_preconnect_all=0
mpi_preconnect_mpi=0
mpi_show_handle_leaks=0
mpi_show_mca_params=
mpi_show_mca_params_file=
mpi_show_mpi_alloc_mem_leaks=0
mpi_use_sparse_group_storage=0
mpi_warn_on_fork=1
mpi_yield_when_idle=-1
mpool=16
mpool_base_verbose=64
mpool_rdma_print_stats=0
mpool_rdma_rcache_name=16
mpool_rdma_rcache_size_limit=
mpool_sm_allocator=10
mpool_sm_min_size=10
mpool_sm_verbose=from MPICH2 v1.3.1 with an additional patch from romio-maint#mcs.anl.gov about an attribute issue
mtl=1
mtl_base_verbose=0
noLocalOption=false
noOversubscribeOption=false
notifier=
notifier_base_verbose=0
notifier_command_cmd=/sbin/initlog -f $s -n "Open MPI" -s "$S: $m (errorcode: $e)"
notifier_command_priority=10
notifier_command_timeout=30
notifier_syslog_priority=0
notifier_threshold_severity=critical
odls=
odls_base_sigkill_timeout=1
odls_base_verbose=0
odls_default_priority=0
ompi_minor_version=6
oob=2147483647
oob_base_verbose=0
oob_tcp_connect_sleep=1
oob_tcp_debug=0
oob_tcp_disable_family=0
oob_tcp_dynamic_ports=
oob_tcp_if_exclude=
oob_tcp_if_include=
oob_tcp_listen_mode=event
oob_tcp_listen_thread_max_queue=10
oob_tcp_listen_thread_wait_time=10
oob_tcp_peer_limit=0
oob_tcp_peer_retries=0
oob_tcp_priority=0
oob_tcp_rcvbuf=0
oob_tcp_sndbuf=0
oob_tcp_static_ports=
oob_tcp_verbose=100
op_base_verbose=131072
opal_event_include=poll
opal_net_private_ipv4=10.0.0.0/8;172.16.0.0/12;192.168.0.0/16;169.254.0.0/16
opal_paffinity_alone=0
opal_profile=0
opal_profile_file=
opal_set_max_sys_limits=0
opal_signal=6,7,8,11
org.eclipse.debug.core.SOURCE_PATH_COMPUTER_ID=org.eclipse.ptp.debug.core.sourcePathComputer
org.eclipse.debug.core.appendEnvironmentVariables=true
org.eclipse.debug.core.environmentVariables={LD_LIBRARY_PATH=/usr/lib64/mpi/gcc/openmpi/lib}
org.eclipse.ptp.launch.ARGUMENT_ATTR=
org.eclipse.ptp.launch.ATTR_AUTO_RUN_COMMAND=true
org.eclipse.ptp.launch.ATTR_CONSOLE=true
org.eclipse.ptp.launch.ATTR_COPY_EXECUTABLE_FROM_LOCAL=false
org.eclipse.ptp.launch.ATTR_DEBUGGER_HOST=localhost
org.eclipse.ptp.launch.ATTR_REMOTE_EXECUTABLE_PATH=/home/Sebastian/testW/Test/Debug/Test
org.eclipse.ptp.launch.ATTR_SYNC_AFTER=false
org.eclipse.ptp.launch.ATTR_SYNC_BEFORE=false
org.eclipse.ptp.launch.ATTR_SYNC_RULES=[]
org.eclipse.ptp.launch.CONFIGURATION_NAME=Open MPI-Generic-Interactive
org.eclipse.ptp.launch.CONNECTION_NAME=Local
org.eclipse.ptp.launch.DEBUGGER_ARGS=--port=51091 --host=localhost --debugger=gdb-mi --debugger_path=/usr/bin/gdm --debug=0
org.eclipse.ptp.launch.DEBUGGER_EXECUTABLE_PATH=/usr/local/bin/sdm
org.eclipse.ptp.launch.DEBUGGER_ID=org.eclipse.ptp.debug.sdm
org.eclipse.ptp.launch.DEBUGGER_SDM_BACKEND=gdb-mi
org.eclipse.ptp.launch.DEBUGGER_SDM_BACKEND_PATH=/usr/bin/gdm
org.eclipse.ptp.launch.DEBUGGER_SDM_EXECUTABLE=/usr/local/bin/sdm
org.eclipse.ptp.launch.DEBUGGER_START_MODE=run
org.eclipse.ptp.launch.DEBUGGER_USE_BUILTIN_SDM=true
org.eclipse.ptp.launch.PROJECT_ATTR=Test
org.eclipse.ptp.launch.REMOTE_SERVICES_ID=org.eclipse.ptp.remote.LocalServices
org.eclipse.ptp.launch.RESOURCE_MANAGER_NAME=25e8e9bf-c561-344e-9abc-4cd12336be73
org.eclipse.ptp.launch.STOP_IN_MAIN=true
orte_abort_timeout=1
orte_allocation_required=0
orte_assume_same_shell=1
orte_base_help_aggregate=1
orte_base_user_debugger=totalview #mpirun# -a #mpirun_args# : ddt -n #np# -start #executable# #executable_argv# #single_app# : fxp #mpirun# -a #mpirun_args#
orte_cpu_set=
orte_daemon_bootstrap=0
orte_daemon_fail=-1
orte_daemon_fail_delay=0
orte_daemon_spin=0
orte_debug=0
orte_debug_daemons=0
orte_debug_daemons_file=0
orte_debug_verbose=-1
orte_debugger_check_rate=0
orte_debugger_test_attach=0
orte_debugger_test_daemon=
orte_default_hostfile=/usr/lib64/mpi/gcc/openmpi/etc/openmpi-default-hostfile
orte_do_not_launch=0
orte_forward_job_control=0
orte_heartbeat_rate=0
orte_hetero_apps=0
orte_keep_fqdn_hostnames=0
orte_launch_agent=orted
orte_leave_session_attached=0
orte_no_session_dirs=
orte_num_boards=1
orte_num_cores=0
orte_num_sockets=0
orte_output_debugger_proctable=0
orte_output_filename=
orte_process_binding=
orte_rankfile=
orte_report_bindings=0
orte_report_events=
orte_report_launch_progress=0
orte_rsh_agent=ssh : rsh
orte_send_profile=0
orte_show_resolved_nodenames=0
orte_startup_timeout=0
orte_tag_output=0
orte_timeout_step=1000
orte_timestamp_output=0
orte_timing=0
orte_timing_details=0
orte_tmpdir_base=
orte_use_regexp=0
orte_xml_file=
orte_xml_output=0
orte_xterm=
osc=0
osc_base_verbose=0
osc_pt2pt_eager_limit=0
osc_pt2pt_no_locks=
osc_rdma_eager_send=-1
osc_rdma_no_locks=131072
osc_rdma_rdma_completion_wait=10
osc_rdma_use_buffers=32
osc_rdma_use_rdma=65536
paffinity=0
paffinity_base_verbose=0
paffinity_hwloc_priority=0
plm=
plm_base_verbose=0
plm_rsh_agent=ssh : rsh
plm_rsh_assume_same_shell=1
plm_rsh_daemonize_llspawn=0
plm_rsh_daemonize_qrsh=0
plm_rsh_delay=1
plm_rsh_disable_llspawn=0
plm_rsh_disable_qrsh=0
plm_rsh_force_rsh=0
plm_rsh_num_concurrent=128
plm_rsh_priority=10
plm_rsh_tree_spawn=0
plm_slurm_args=
plm_slurm_priority=0
pml= CFLAGS='-DNDEBUG -fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -finline-functions -fno-strict-aliasing -pthread' CPPFLAGS=' -I/home/abuild/rpmbuild/BUILD/openmpi-1.6/opal/mca/hwloc/hwloc132/hwloc/include -I/usr/include/infiniband -I/usr/include/infiniband' FFLAGS='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g' LDFLAGS=' ' --enable-shared --disable-static --prefix=/usr/lib64/mpi/gcc/openmpi --with-mpi=open_mpi --disable-aio
pml_base_verbose=
pml_bfo_allocator=
pml_cm_free_list_inc=vma
pml_cm_free_list_max=0
pml_cm_free_list_num=0
pml_cm_priority=0
pml_csum_allocator=0
pml_ob1_allocator=0
prefixOption=false
progArgs=
ptpDirectory=/home/Sebastian/.eclipsesettings
pubsub=error
pubsub_base_verbose=
pubsub_orte_priority=0
ras=
ras_base_display_alloc=0
ras_base_display_devel_alloc=0
ras_base_verbose=0
ras_cm_priority=0
ras_loadleveler_priority=90
ras_slurm_priority=75
rcache=0
rcache_base_verbose=0
rmaps=
rmaps_base_cpus_per_proc=1
rmaps_base_cpus_per_rank=1
rmaps_base_display_devel_map=0
rmaps_base_display_map=0
rmaps_base_loadbalance=0
rmaps_base_n_perboard=-1
rmaps_base_n_pernode=-1
rmaps_base_n_persocket=-1
rmaps_base_no_oversubscribe=0
rmaps_base_no_schedule_local=0
rmaps_base_pernode=0
rmaps_base_schedule_policy=slot
rmaps_base_slot_list=
rmaps_base_stride=1
rmaps_base_verbose=0
rmaps_load_balance_priority=0
rmaps_rank_file_path=
rmaps_rank_file_priority=0
rmaps_resilient_fault_grp_file=
rmaps_resilient_priority=0
rmaps_round_robin_priority=0
rmaps_seq_priority=0
rmaps_topo_priority=0
rml=
rml_base_verbose=0
rml_oob_priority=0
rml_wrapper=
routed=
routed_base_verbose=0
routed_binomial_priority=0
routed_cm_priority=0
routed_direct_priority=0
routed_linear_priority=0
routed_radix_priority=0
routed_slave_priority=0
shmem=10
shmem_base_verbose=
shmem_mmap_backing_file_base_dir=50
shmem_mmap_enable_nfs_warning=0
shmem_mmap_priority=
shmem_mmap_relocate_backing_file=1
shmem_posix_priority=0
shmem_sysv_priority=/dev/shm
sysinfo=
sysinfo_base_verbose=0
timer=10
timer_base_verbose=40
topo=
topo_base_verbose=0
valid_Basic.Options=25524 25659 allAttributes allocator allocator_base_verbose backtrace backtrace_base_verbose bml bml_base_verbose bml_r2_show_unreach_errors btl btl_base_verbose btl_base_warn_component_unused btl_self_bandwidth btl_self_eager_limit btl_self_exclusivity btl_self_flags btl_self_free_list_inc btl_self_free_list_max btl_self_free_list_num btl_self_latency btl_self_max_send_size btl_self_min_rdma_pipeline_size btl_self_rdma_pipeline_frag_size btl_self_rdma_pipeline_send_length btl_self_rndv_eager_limit btl_sm_bandwidth btl_sm_eager_limit btl_sm_exclusivity btl_sm_flags btl_sm_have_knem_support btl_sm_knem_dma_min btl_sm_knem_max_simultaneous btl_sm_latency btl_sm_max_send_size btl_sm_rndv_eager_limit btl_sm_use_knem btl_tcp_bandwidth btl_tcp_eager_limit btl_tcp_endpoint_cache btl_tcp_exclusivity btl_tcp_flags btl_tcp_if_exclude btl_tcp_if_include btl_tcp_if_seq btl_tcp_latency btl_tcp_max_send_size btl_tcp_min_rdma_pipeline_size btl_tcp_port_min_v4 btl_tcp_port_range_v4 btl_tcp_rdma_pipeline_frag_size btl_tcp_rdma_pipeline_send_length btl_tcp_rndv_eager_limit btl_tcp_use_nagle byNodeOption bySlotOption carto carto_auto_detect_priority carto_base_verbose carto_file_path carto_file_priority coll coll_base_verbose coll_basic_crossover coll_basic_priority coll_hierarch_bcast_alg coll_hierarch_detection_alg coll_hierarch_ignore_sm coll_hierarch_priority coll_hierarch_segment_size coll_hierarch_use_rdma coll_hierarch_verbose coll_inter_priority coll_inter_verbose coll_sm_comm_in_use_flags coll_sm_comm_num_segments coll_sm_control_size coll_sm_fragment_size coll_sm_info_num_procs coll_sm_priority coll_sm_shared_mem_used_data coll_sm_tree_degree coll_sync_barrier_after coll_sync_barrier_before coll_sync_priority coll_tuned_init_chain_fanout coll_tuned_init_tree_fanout coll_tuned_pre_allocate_memory_comm_size_limit coll_tuned_priority coll_tuned_use_dynamic_rules control.address control.user.name control.working.dir current_controller debuggerId disabled dpm dpm_base_verbose enabled_Basic.Options errmgr errmgr_base_verbose errmgr_default_priority ess ess_base_verbose ess_env_priority ess_hnp_priority ess_singleton_priority ess_slave_priority ess_slurm_priority ess_slurmd_priority ess_tool_priority executableDirectory executablePath extraArgs filem filem_base_verbose filem_rsh_cp filem_rsh_max_incomming filem_rsh_max_outgoing filem_rsh_priority filem_rsh_rcp filem_rsh_rsh filem_rsh_verbose grpcomm grpcomm_bad_priority grpcomm_base_verbose grpcomm_basic_priority grpcomm_hier_priority hostFileOption hostListOption hwloc hwloc_base_mem_alloc_policy hwloc_base_mem_bind_failure_action hwloc_base_verbose invalid_Basic.Options io io_base_freelist_increment io_base_freelist_initial_size io_base_freelist_max_size io_base_verbose io_romio_complete_configure_params io_romio_delete_priority io_romio_priority io_romio_user_configure_params io_romio_version iof iof_base_verbose launchMode local_prefix maffinity maffinity_base_verbose maffinity_first_use_priority maffinity_hwloc_priority mca_base_param_file_path mca_base_param_file_path_force mca_base_param_file_prefix mca_component_disable_dlopen mca_component_path mca_component_show_load_errors mca_param_files mca_verbose memchecker memory memory_base_verbose memory_linux_disable memory_linux_ptmalloc2_available memory_linux_ptmalloc2_enable memory_linux_ummunotify_available memory_linux_ummunotify_enable mpiNumberOfProcesses mpi_abort_delay mpi_abort_print_stack mpi_event_tick_rate mpi_have_sparse_group_storage mpi_keep_peer_hostnames mpi_leave_pinned mpi_leave_pinned_pipeline mpi_no_free_handles mpi_notify_init_finalize mpi_paffinity_alone mpi_param_check mpi_preconnect_all mpi_preconnect_mpi mpi_show_handle_leaks mpi_show_mca_params mpi_show_mca_params_file mpi_show_mpi_alloc_mem_leaks mpi_use_sparse_group_storage mpi_warn_on_fork mpi_yield_when_idle mpool mpool_base_verbose mpool_rdma_print_stats mpool_rdma_rcache_name mpool_rdma_rcache_size_limit mpool_sm_allocator mpool_sm_min_size mpool_sm_verbose mtl mtl_base_verbose noLocalOption noOversubscribeOption notifier notifier_base_verbose notifier_command_cmd notifier_command_priority notifier_command_timeout notifier_syslog_priority notifier_threshold_severity odls odls_base_sigkill_timeout odls_base_verbose odls_default_priority ompi_minor_version oob oob_base_verbose oob_tcp_connect_sleep oob_tcp_debug oob_tcp_disable_family oob_tcp_dynamic_ports oob_tcp_if_exclude oob_tcp_if_include oob_tcp_listen_mode oob_tcp_listen_thread_max_queue oob_tcp_listen_thread_wait_time oob_tcp_peer_limit oob_tcp_peer_retries oob_tcp_priority oob_tcp_rcvbuf oob_tcp_sndbuf oob_tcp_static_ports oob_tcp_verbose op_base_verbose opal_event_include opal_net_private_ipv4 opal_paffinity_alone opal_profile opal_profile_file opal_set_max_sys_limits opal_signal orte_abort_timeout orte_allocation_required orte_assume_same_shell orte_base_help_aggregate orte_base_user_debugger orte_cpu_set orte_daemon_bootstrap orte_daemon_fail orte_daemon_fail_delay orte_daemon_spin orte_debug orte_debug_daemons orte_debug_daemons_file orte_debug_verbose orte_debugger_check_rate orte_debugger_test_attach orte_debugger_test_daemon orte_default_hostfile orte_do_not_launch orte_forward_job_control orte_heartbeat_rate orte_hetero_apps orte_keep_fqdn_hostnames orte_launch_agent orte_leave_session_attached orte_no_session_dirs orte_num_boards orte_num_cores orte_num_sockets orte_output_debugger_proctable orte_output_filename orte_process_binding orte_rankfile orte_report_bindings orte_report_events orte_report_launch_progress orte_rsh_agent orte_send_profile orte_show_resolved_nodenames orte_startup_timeout orte_tag_output orte_timeout_step orte_timestamp_output orte_timing orte_timing_details orte_tmpdir_base orte_use_regexp orte_xml_file orte_xml_output orte_xterm osc osc_base_verbose osc_pt2pt_eager_limit osc_pt2pt_no_locks osc_rdma_eager_send osc_rdma_no_locks osc_rdma_rdma_completion_wait osc_rdma_use_buffers osc_rdma_use_rdma paffinity paffinity_base_verbose paffinity_hwloc_priority plm plm_base_verbose plm_rsh_agent plm_rsh_assume_same_shell plm_rsh_daemonize_llspawn plm_rsh_daemonize_qrsh plm_rsh_delay plm_rsh_disable_llspawn plm_rsh_disable_qrsh plm_rsh_force_rsh plm_rsh_num_concurrent plm_rsh_priority plm_rsh_tree_spawn plm_slurm_args plm_slurm_priority pml pml_base_verbose pml_bfo_allocator pml_cm_free_list_inc pml_cm_free_list_max pml_cm_free_list_num pml_cm_priority pml_csum_allocator pml_ob1_allocator prefixOption progArgs ptpDirectory pubsub pubsub_base_verbose pubsub_orte_priority ras ras_base_display_alloc ras_base_display_devel_alloc ras_base_verbose ras_cm_priority ras_loadleveler_priority ras_slurm_priority rcache rcache_base_verbose rmaps rmaps_base_cpus_per_proc rmaps_base_cpus_per_rank rmaps_base_display_devel_map rmaps_base_display_map rmaps_base_loadbalance rmaps_base_n_perboard rmaps_base_n_pernode rmaps_base_n_persocket rmaps_base_no_oversubscribe rmaps_base_no_schedule_local rmaps_base_pernode rmaps_base_schedule_policy rmaps_base_slot_list rmaps_base_stride rmaps_base_verbose rmaps_load_balance_priority rmaps_rank_file_path rmaps_rank_file_priority rmaps_resilient_fault_grp_file rmaps_resilient_priority rmaps_round_robin_priority rmaps_seq_priority rmaps_topo_priority rml rml_base_verbose rml_oob_priority rml_wrapper routed routed_base_verbose routed_binomial_priority routed_cm_priority routed_direct_priority routed_linear_priority routed_radix_priority routed_slave_priority shmem shmem_base_verbose shmem_mmap_backing_file_base_dir shmem_mmap_enable_nfs_warning shmem_mmap_priority shmem_mmap_relocate_backing_file shmem_posix_priority shmem_sysv_priority sysinfo sysinfo_base_verbose timer timer_base_verbose topo topo_base_verbose valid_Basic.Options visible_Basic.Options working.directory
visible_Basic.Options=byNodeOption bySlotOption hostFile hostFileOption hostList hostListOption mpiNumberOfProcesses noLocalOption noOversubscribeOption prefix prefixOption
working.directory=/home/Sebastian

Answer 1

OK, i updated the PTP Version to 7.0.4 and now it is working, do not know why.