Call python-nmap PortScanner(), nmap not found - python-2.7

In my virtualenv I installed python-nmap and nmap is installed (OS X).
But if I call mmap like (virtualenv activated...):
import sys
sys.path.append('/usr/local/bin')
import nmap
nm = nmap.PortScanner()
I get the following error:
Raise PortScannerError('nmap program was not found in path')
nmap.nmap.PortScannerError: 'nmap program was not found in path'
Is there still another way to enter the path to nmap?

Portet the project to Python 3 and update nmap to 0.3.3 now it works.
Thanks a lot!

install nmap with homebrew
brew install nmap
Then your install will work correctly.

sys.path governs where the Python interpreter looks for importing modules. The "path" in your error is the OS's PATH environment variable, which tells the OS where to look for Nmap. You can either set this directly with os.putenv or you can pass the full path to the nmap binary in the nmap.PortScanner constructor:
nm = nmap.PortScanner('/usr/local/bin/nmap')

I have tried these methods but not worked then I further looked into it.
Error:
>>> import nmap
>>> nm = nmap.PortScanner()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/nmap/nmap.py", line 137, in __init__
raise PortScannerError('nmap program was not found in path')
nmap.nmap.PortScannerError: 'nmap program was not found in path'
so I looked into file "/usr/local/lib/python2.7/dist-packages/nmap/nmap.py". The code where they are checking for nmap is not working for some reason.
code:
# regex used to detect nmap
regex = re.compile('Nmap version [0-9]*\.[0-9]*[^ ]* \( http://nmap\.org \)')
# launch 'nmap -V', we wait after 'Nmap version 5.0 ( http://nmap.org )'
p = subprocess.Popen(['nmap', '-V'], bufsize=10000, stdout=subprocess.PIPE)
self._nmap_last_output = p.communicate()[0] # store stdout
for line in self._nmap_last_output.split('\n'):
if regex.match(line) is not None:
is_nmap_found = True
# Search for version number
regex_version = re.compile('[0-9]+')
regex_subversion = re.compile('\.[0-9]+')
rv = regex_version.search(line)
rsv = regex_subversion.search(line)
if rv is not None and rsv is not None:
# extract version/subversion
self._nmap_version_number = int(line[rv.start():rv.end()])
self._nmap_subversion_number = int(line[rsv.start()+1:rsv.end()])
break
if is_nmap_found == False:
raise PortScannerError('nmap program was not found in path')
I found out that first regex is not working so I changed this:
from :
regex = re.compile('Nmap version [0-9]*\.[0-9]*[^ ]* \( http://nmap\.org \)')
to:
regex = re.compile('Nmap version [0-9]*\.[0-9]*)
Now it is working just like it suppose to work!

Related

`nix-build` fails because Python wants something that's unavailable, without saying what it wants

(At least I think that's the problem.)
The error
When I run nix-build -A serialosc from the root of nixpkgs, nix prints the following:
[jeff#jbb-dell:~/nix/nixpkgs-fork]$ nix-build -A serialosc
these derivations will be built:
/nix/store/p40wvn28grb8nrcd6scbxhhjqw4495kk-serialosc.drv
building '/nix/store/p40wvn28grb8nrcd6scbxhhjqw4495kk-serialosc.drv'...
unpacking sources
unpacking source archive /nix/store/5zj802wfjd0ima92lpzzsqdjqvrnrwf9-serialosc
source root is serialosc
patching sources
configuring
configure flags: --prefix=/nix/store/b6qzvzmwjdi29nnzc769904wc0mj1cds-serialosc configure
Setting top to : /build/serialosc
Setting out to : /build/serialosc/build
Checking for 'gcc' (C compiler) : gcc
Checking for working poll() : yes
Checking for libudev : yes
Checking for libmonome : yes
Checking for liblo : yes
Checking for function strdup : yes
Checking for function _strdup : not found
Checking for function strndup : yes
Checking for function strcasecmp : yes
Checking for header unistd.h : yes
Checking for header dns_sd.h : yes
Checking for library dl : yes
Traceback (most recent call last):
File "/build/serialosc/.waf-1.8.5-3fc7b0f5070c04bfd3f2489448471a84/waflib/Scripting.py", line 103, in waf_entry_point
run_commands()
File "/build/serialosc/.waf-1.8.5-3fc7b0f5070c04bfd3f2489448471a84/waflib/Scripting.py", line 164, in run_commands
ctx=run_command(cmd_name)
File "/build/serialosc/.waf-1.8.5-3fc7b0f5070c04bfd3f2489448471a84/waflib/Scripting.py", line 155, in run_command
ctx.execute()
File "/build/serialosc/.waf-1.8.5-3fc7b0f5070c04bfd3f2489448471a84/waflib/Configure.py", line 92, in execute
super(ConfigurationContext,self).execute()
File "/build/serialosc/.waf-1.8.5-3fc7b0f5070c04bfd3f2489448471a84/waflib/Context.py", line 92, in execute
self.recurse([os.path.dirname(g_module.root_path)])
File "/build/serialosc/.waf-1.8.5-3fc7b0f5070c04bfd3f2489448471a84/waflib/Context.py", line 133, in recurse
user_function(self)
File "/build/serialosc/wscript", line 259, in configure
stderr=devnull).decode().strip()
File "/nix/store/lbrpma3528hq7gwn8ffbnwaxvsqah0bb-python-2.7.17/lib/python2.7/subprocess.py", line 216, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/nix/store/lbrpma3528hq7gwn8ffbnwaxvsqah0bb-python-2.7.17/lib/python2.7/subprocess.py", line 394, in __init__
errread, errwrite)
File "/nix/store/lbrpma3528hq7gwn8ffbnwaxvsqah0bb-python-2.7.17/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
builder for '/nix/store/p40wvn28grb8nrcd6scbxhhjqw4495kk-serialosc.drv' failed with exit code 2
error: build of '/nix/store/p40wvn28grb8nrcd6scbxhhjqw4495kk-serialosc.drv' failed
[jeff#jbb-dell:~/nix/nixpkgs-fork]$
My suspicion
The problem appears to arise in lib/python2.7/subprocess.py. I found a version of that file online, and the definition of _execute_child seems to make reference to some absolute paths -- for instance, in this passage:
if shell:
args = ["/bin/sh", "-c"] + args
if executable:
args[0] = executable
Therefore my guess is that Nix is building in a sandbox that doesn't offer whatever Python expects. But the error doesn't tell me what Python was hoping to find.
The context
(You might not need this information.)
I'm trying to build serialosc on NixOS 19.09. Here's my fork of nixpkgs, including my broken recipe for serialosc. The fork also includes my libmonome package, which works, and which serialosc depends on, and which is not yet part of the official nixpkgs repo. I've given hard links to the specific commit; they shouldn't change when I update the fork.
Cross-posting from
https://discourse.nixos.org/t/nix-build-fails-because-python-wants-something-thats-unavailable-without-saying-what-it-wants/5675/2
so that it could help people who only find this, answer might continue on NixOS discourse. I tried to keep it helpful for general NixOS writing a derivation debugging, with some Python bits
fetchgit
First, let's help with the fetchgit part in your recipe
src = fetchgit {
# Once, it seemed to finish, and I don't know how! Then it crashed with:
# Switched to a new branch 'fetchgit'
# removing `.git'...
# hash mismatch in fixed-output derivation '/nix/store/5zj802wfjd0ima92lpzzsqdjqvrnrwf9-serialosc':
# wanted: sha256:1vqcxi32wc4pklbddllflkaigkfvd4ykwrjqccayvrk10dx1sna3
# got: sha256:1zmzjasv21ix7i7s58a31k0025ji32hv2jm2ww6s0xhjmr5ax34j
# This way it [[gets pretty far]].
# The fetchSubmodules value I set again seems to have no effect.
url = https://github.com/monome/serialosc.git;
rev = "v1.4.1";
sha256 = "1zmzjasv21ix7i7s58a31k0025ji32hv2jm2ww6s0xhjmr5ax34j";
Perhaps there was a force push? Not sure why it would stop working with the hash
I found that the fetchSubmodules has no effect if you have the hash of the one without the submodules, use nix-prefetch-git --fetch-submodules https://github.com/monome/serialosc which gives
{
"url": "https://github.com/monome/serialosc",
"rev": "cec0ea76b2d5f69afa74d3ffc14a0950e32a7914",
"date": "2019-06-09T21:46:13+02:00",
"sha256": "03qkzslhih72idwafgfxmkwp5v3x048njh0c682phw2ks11plmbp",
"fetchSubmodules": true
}
so use
fetchgit {
url = https://github.com/monome/serialosc;
rev = "cec0ea76b2d5f69afa74d3ffc14a0950e32a7914";
# or rev = version; but that might break with force pushes
sha256 = "03qkzslhih72idwafgfxmkwp5v3x048njh0c682phw2ks11plmbp";
fetchSubmodules = true;
}
wafHook
There is a (slight) wafHook documentation in https://nixos.org/nixpkgs/manual/
Debugging error
It seems to want to do git stuff, here is how I debugged it:
$ nix-shell https://github.com/JeffreyBenjaminBrown/nixpkgs/archive/2bbd131811991bec9ae6b296b5cd1441640c306c.tar.gz -A serialosc --pure
nix-shell $ mkdir /tmp/delme-serialosc
nix-shell $ cd !$
# To find out what a build would do
nix-shell $ type genericBuild
...
# Doing the `if [ -z "${phases:-}" ]; then ...` bit gives us `echo $phases`
# unpackPhase patchPhase configurePhase buildPhase checkPhase installPhase fixupPhase installCheckPhase distPhase
nix-shell $ ${unpackPhase:-unpackPhase}
nix-shell $ cd $sourceRoot
nix-shell $ ${patchPhase:-patchPhase}
nix-shell $ ${configurePhase:-configurePhase}
# Error happens here
nix-shell $ type ${configurePhase:-configurePhase}
# Do the steps until you find it stops in `python ...`
# Python debug info: https://docs.python.org/3/library/pdb.html
nix-shell $ python -m pdb "$wafPath" "${flagsArray[#]}";
(Pdb) continue # or cont, or just c
...
(Pdb) break /nix/store/lbrpma3528hq7gwn8ffbnwaxvsqah0bb-python-2.7.17/lib/python2.7/subprocess.py:1047
(Pdb) continue
> /nix/store/lbrpma3528hq7gwn8ffbnwaxvsqah0bb-python-2.7.17/lib/python2.7/subprocess.py(1047)_execute_child()
-> raise child_exception
(Pdb) up
> /nix/store/lbrpma3528hq7gwn8ffbnwaxvsqah0bb-python-2.7.17/lib/python2.7/subprocess.py(394)__init__()
-> errread, errwrite)
(Pdb) # empty is equivalent to do last command, i.e. up in this case
> /nix/store/lbrpma3528hq7gwn8ffbnwaxvsqah0bb-python-2.7.17/lib/python2.7/subprocess.py(216)check_output()
-> process = Popen(stdout=PIPE, *popenargs, **kwargs)
(Pdb) print(popenargs)
(['git', 'rev-parse', '--verify', '--short', 'HEAD'],)
But you will find you might need to figure a way to avoid waf using git as it cannot access the network unless it's a fixed-output derivation (e.g. those fetching sources). Hopefully the bit about fetchSubmodules will fix it!
Edit: I didn’t read past that it tries to use git, it tries to figure out what version it is, you should somehow make it avoid using git to do that, e.g. grep -r 'rev-parse' suggests editing the wscript file
Edit 2: E.g. something like
--- wscript-old 2020-01-29 15:55:54.100976818 +0000
+++ wscript 2020-01-29 15:55:58.128977024 +0000
## -257,7 +257,7 ##
conf.env.GIT_COMMIT = subprocess.check_output(
["git", "rev-parse", "--verify", "--short", "HEAD"],
stderr=devnull).decode().strip()
- except subprocess.CalledProcessError:
+ except (subprocess.CalledProcessError, OSError):
conf.env.GIT_COMMIT = ''
conf.define("VERSION", VERSION)
perhaps it could be upstreamed too (this leaves the git commit ID as empty in this case)? Or just use substituteInPlace or something to put in the actual commit ID to be
--- wscript-old 2020-01-29 15:55:54.100976818 +0000
+++ wscript 2020-01-29 15:57:56.177983054 +0000
## -249,16 +249,7 ##
conf.env.VERSION = VERSION
- try:
- import os
-
- devnull = open(os.devnull, 'w')
-
- conf.env.GIT_COMMIT = subprocess.check_output(
- ["git", "rev-parse", "--verify", "--short", "HEAD"],
- stderr=devnull).decode().strip()
- except subprocess.CalledProcessError:
- conf.env.GIT_COMMIT = ''
+ conf.env.GIT_COMMIT = 'cec0ea76b2d5f69afa74d3ffc14a0950e32a7914'
conf.define("VERSION", VERSION)
conf.define("_GNU_SOURCE", 1)

I don't understand how this "os.join" function is working? I am getting errors constantly and no reading on os functions is helping me

Here's the code
sys.path.append( "../tools/" )
from parse_out_email_text import parseOutText #(its just another .py file that has a function I wrote)
from_sara = open("from_sara.txt", "r")
from_chris = open("from_chris.txt", "r")
from_data = []
word_data = []
temp_counter = 0
for name, from_person in [("sara", from_sara), ("chris", from_chris)]:
for path in from_person:
### only look at first 200 emails when developing
### once everything is working, remove this line to run over full dataset
temp_counter += 1
if temp_counter < 200:
path = os.path.join('..', path[:-1]) #(THIS IS THE PART I CAN'T GET MY HEAD AROUND)
print path
email = open(path, "r")
email.close()
print "emails processed"
from_sara.close()
from_chris.close()
When I run this, it gives me an error as shown below:
Traceback (most recent call last):
..\maildir/bailey-s/deleted_items/101.
File "C:/Users/AmitSingh/Desktop/Data/Udacity/Naya_attempt/vectorize_text.py", line 47, in <module>
email = open(path, "r")
IOError: [Errno 2] No such file or directory: '..\\maildir/bailey-s/deleted_items/101.'
I don't even have this """'..\maildir/bailey-s/deleted_items/101.'""" directory path on my laptop, I tried to change the path by replacing the '..' in the code by the actual path name to the folder where I keep all the files, and nothing changes.
path = os.path.join('..', path[:-1])
This code is part of an online course on machine learning and I have been stuck at this point for 3 hours now. Any help would be really appreciated.
(P.S. This is not a homework question and there are no grades attached to this, its a free online course)
your test data is not there so it cannot find it. you should run start-up code again and make sure the necessary maildir are all there.
Go to tools inside your udacity project directory and run startup.py.
It is about 400 Mb so sit back and relax!
I know this is extremely late, but I found this post after having the exact same problem.
All the answers that I found here and on other sites, even the issue requests in the original github, were just "run startup.py" I already did that. However, it was telling me:
Traceback (most recent call last):
File "K:\documents\Udacity\Mini-Projects\ud120-projects\text_learning\vectorize_text.py", line 48, in <module>
email = open(path, "r")
FileNotFoundError: [Errno 2] No such file or directory: '..\\maildir/bailey-s/deleted_items/101.'
Just like yours. I then found where this file was located and it was indeed on my computer
I added 'tools' to the os.path.join() line as you can see here:
for name, from_person in [("sara", from_sara), ("chris", from_chris)]:
for path in from_person:
### only look at first 200 emails when developing
### once everything is working, remove this line to run over full dataset
temp_counter += 1
if temp_counter < 200:
#path = os.path.join('..', path[:-1]) <---original
path = os.path.join('..','tools', path[:-1])
print(path)
email = open(path, "r")
This worked for me finally. So, I hope it helps anyone else that stumbles on this problem in the future.
Also, I noticed on some examples I found of other repos of the lessons. Their 'tools' folder was named 'utils'.
Here is an example, this is a repo that someone tweaked to use jupyter notebooks to run the lessons So, use the one that you have.
In your Udacity course folder, first go to tools directory, check if you have maildir folder present and if it has got subfolders in it, if they are present then go back to text_learning/vectorize_text.py, find this line of code path = os.path.join('..', path[:-1]), change it to path = os.path.join('../tools/', path[:-1]),
On terminal, cd text_learning , then python vectorize_text.py, this should solve the issue.
If this does not solve the issue, then Go to tools inside your udacity project directory and run startup.py. Wait till the process is complete
Repeat step 1.

maya python subprocess error

in python 2.7 i try this code to get data from Deadline software. Its return some data from server...
import subprocess
path = 'C:/Program Files/Thinkbox/Deadline7/bin/'
p1 = subprocess.Popen([path + 'deadlinecommand.exe', 'pools'], stdout=subprocess.PIPE)
p1.communicate()
and see result:
('none\r\npool_01\r\npool_02\r\npool_03\r\npool_04\r\npool_05\r\npoolhalf\r\n', None)
but when i copy that code to python in maya 2014 i get error:
p1 = subprocess.Popen(['path + 'deadlinecommand.exe', 'pools'], stdout=subprocess.PIPE)
# Error: WindowsError: file C:\PROGRA~1\Autodesk\maya2014\bin\python27.zip\subprocess.py line 826: 6 #
run this exe file - is the only option dedline communication. but it pays to stdout data and how it is necessary to pull out. subprocess options except I have not found, but if there are other options will be glad to try them
anyone else encountered this problem? strange that in pure Python 2.7 running in windows all works, and there is no Maya 2014
i use:
Windows 7 + Python 2.7.9
Maya 2014 (Python 2.7.3)
I was just trying something similar couple of days back, connecting to Deadline via the command line submitter and getting
# File "C:\Program Files\Autodesk\Maya2013\bin\python26.zip\subprocess.py", line 786, in _make_inheritable
# WindowsError: [Error 6] The handle is invalid
error in Maya 2013.5. One workaround found here which does fix this issue is to pipe all the handles
p1 = subprocess.Popen([path + 'deadlinecommand.exe', 'pools'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
Hope it helps.

How to set a compiler for codepy.toolchain?

These two lines were taken from the ThrustInterop.py found in PyCuda/Examples
import codepy.toolchain
codepy.toolchain.guess_toolchain()
I understand that this should return the Toolchain instance for a C++ compiler found in $HOME/.aksetup-defaults.py.
BOOST_COMPILER = 'gcc'
BOOST_INC_DIR = ['/usr/local/Cellar/boost/1.57.0/include/']
BOOST_LIB_DIR = ['/usr/local/Cellar/boost/1.57.0/lib/']
BOOST_PYTHON_LIBNAME = ['boost_python']
BOOST_THREAD_LIBNAME = ['boost_thread']
CUDADRV_LIBNAME = ['cuda']
CUDADRV_LIB_DIR = ['${CUDA_ROOT}/lib']
CUDART_LIBNAME = ['cudart']
CUDART_LIB_DIR = ['${CUDA_ROOT}/lib']
CUDA_ENABLE_CURAND = True
CUDA_ENABLE_GL = True
CUDA_INC_DIR = ['${CUDA_ROOT}/include']
CUDA_TRACE = False
CURAND_LIBNAME = ['curand']
CURAND_LIB_DIR = ['${CUDA_ROOT}/lib']
USE_SHIPPED_BOOST = False
Under Python 2.7.6, I get the following exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/codepy/toolchain.py", line 433, in guess_toolchain
raise ToolchainGuessError("unknown compiler")
codepy.toolchain.ToolchainGuessError: unknown compiler
On a Cluster with Python 3.4.1:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/cluster/home03/math/echeverl/local2/lib/python3.4/site-packages/codepy/toolchain.py", line 439, in guess_nvcc_toolchain
gcc_kwargs = _guess_toolchain_kwargs_from_python_config()
File "/cluster/home03/math/echeverl/local2/lib/python3.4/site-packages/codepy/toolchain.py", line 398, in _guess_toolchain_kwargs_from_python_config
so_ext=make_vars["SO"],
KeyError: 'SO'
After looking at the library I found that it doesn't recognize clang so I modified 'guess_toolchain' in toolchain.py
def guess_toolchain():
"""Guess and return a :class:`Toolchain` instance.
Raise :exc:`ToolchainGuessError` if no toolchain could be found.
"""
kwargs = _guess_toolchain_kwargs_from_python_config()
from pytools.prefork import call_capture_output
result, version, stderr = call_capture_output([kwargs["cc"], "--version"])
if result != 0:
raise ToolchainGuessError("compiler version query failed: "+stderr)
#Original Code
# if "Free Sofware Foundation" in version:
#Modified line to recognize clang
if ("Free Software Foundation" in version) or ("clang" in version):
if "-Wstrict-prototypes" in kwargs["cflags"]:
kwargs["cflags"].remove("-Wstrict-prototypes")
if "darwin" in version:
# Are we running in 32-bit mode?
# The python interpreter may have been compiled as a Fat binary
# So we need to check explicitly how we're running
# And update the cflags accordingly
import sys
if sys.maxint == 0x7fffffff:
kwargs["cflags"].extend(['-arch', 'i386'])
return GCCToolchain(**kwargs)
else:
raise ToolchainGuessError("unknown compiler")
How can I tell codepy.toolchain which compiler to use?
Setting the CC environment variable to an appropriate value might help. Also check your Python Makefile, stored under config in your Python lib directory. That's where the config information comes from in the first place.
I modified the file ~/.aksetup-defaults.py to:
#For CUDA
CUDA_ROOT=['/Developer/NVIDIA/CUDA-5.0/']
CUDADRV_LIB_DIR=['/Developer/NVIDIA/CUDA-5.0/lib']
CUDA_INC_DIR=['/Developer/NVIDIA/CUDA-5.0/include']
#For Boost
BOOST_INC_DIR=['${HOME}/pool/include/']
BOOST_LIB_DIR=['${HOME}/pool/lib']
USE_SHIPPED_BOOST=False
This solved the issue with toolchain, but I'm still trying to figure out how to compile.

Crash on Nite initialisation in python program written using primesense2.2.0.30-5

I am using Ubunut 14.04 and have installed OpenNI 2.2, NITE 2.2 and primesense2.2.0.30-5 (https://pypi.python.org/pypi/primesense/2.2.0.30-5)
The sample program mentioned at the above site works quite well however when I am trying to use initialize nite2 the program crashes.
Here is the code:
from primesense import openni2, nite2
openni2.initialize() # can also accept the path of the OpenNI redistribution
nite2.initialize()
if (nite2.is_initialized()):
print "nite2 initialized"
else:
print "nite2 not initialized"
dev = openni2.Device.open_any()
print dev.get_sensor_info(openni2.SENSOR_DEPTH)
depth_stream = dev.create_depth_stream()
depth_stream.start()
frame = depth_stream.read_frame()
frame_data = frame.get_buffer_as_uint16()
depth_stream.stop()
openni2.unload()
Error Report :
Traceback (most recent call last):
File "test.py", line 4, in <module>
nite2.initialize()
File "/usr/local/lib/python2.7/dist-packages/primesense/nite2.py", line 81, in initialize
("\n ".join("%s: %s" % (dir, ex) for dir, ex in exceptions)),)
primesense.utils.InitializationError: NiTE2 could not be loaded:
/home/user/project/kinect/NiTE-Linux-x64-2.2/Samples/Bin/libNiTE2.so: /home/user/project/kinect/NiTE-Linux-x64-2.2/Samples/Bin/libNiTE2.so: undefined symbol: niteDumpUserTrackerCalibrationDataToFile
OpenNI2-FreenectDriver: Closing device freenect://0
I want nite python bindings for user tracking and hand tracking . Did anyone done it before successfully
A workaround is to comment the following lines in file _openni2.py from the primesense python bindings:
global _niteDumpUserTrackerCalibrationDataToFile
_niteDumpUserTrackerCalibrationDataToFile = dll.niteDumpUserTrackerCalibrationDataToFile
_niteDumpUserTrackerCalibrationDataToFile.restype = NiteStatus
_niteDumpUserTrackerCalibrationDataToFile.argtypes = [NiteUserTrackerHandle, NiteUserId, ctypes.c_char_p]
Also, make sure you have a copy/symlink of both the NiTE2 library and folder (Redist/NiTE2 in the archive) in the same directory as your python script.