How to set a compiler for codepy.toolchain? - python-2.7

These two lines were taken from the ThrustInterop.py found in PyCuda/Examples
import codepy.toolchain
codepy.toolchain.guess_toolchain()
I understand that this should return the Toolchain instance for a C++ compiler found in $HOME/.aksetup-defaults.py.
BOOST_COMPILER = 'gcc'
BOOST_INC_DIR = ['/usr/local/Cellar/boost/1.57.0/include/']
BOOST_LIB_DIR = ['/usr/local/Cellar/boost/1.57.0/lib/']
BOOST_PYTHON_LIBNAME = ['boost_python']
BOOST_THREAD_LIBNAME = ['boost_thread']
CUDADRV_LIBNAME = ['cuda']
CUDADRV_LIB_DIR = ['${CUDA_ROOT}/lib']
CUDART_LIBNAME = ['cudart']
CUDART_LIB_DIR = ['${CUDA_ROOT}/lib']
CUDA_ENABLE_CURAND = True
CUDA_ENABLE_GL = True
CUDA_INC_DIR = ['${CUDA_ROOT}/include']
CUDA_TRACE = False
CURAND_LIBNAME = ['curand']
CURAND_LIB_DIR = ['${CUDA_ROOT}/lib']
USE_SHIPPED_BOOST = False
Under Python 2.7.6, I get the following exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/codepy/toolchain.py", line 433, in guess_toolchain
raise ToolchainGuessError("unknown compiler")
codepy.toolchain.ToolchainGuessError: unknown compiler
On a Cluster with Python 3.4.1:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/cluster/home03/math/echeverl/local2/lib/python3.4/site-packages/codepy/toolchain.py", line 439, in guess_nvcc_toolchain
gcc_kwargs = _guess_toolchain_kwargs_from_python_config()
File "/cluster/home03/math/echeverl/local2/lib/python3.4/site-packages/codepy/toolchain.py", line 398, in _guess_toolchain_kwargs_from_python_config
so_ext=make_vars["SO"],
KeyError: 'SO'
After looking at the library I found that it doesn't recognize clang so I modified 'guess_toolchain' in toolchain.py
def guess_toolchain():
"""Guess and return a :class:`Toolchain` instance.
Raise :exc:`ToolchainGuessError` if no toolchain could be found.
"""
kwargs = _guess_toolchain_kwargs_from_python_config()
from pytools.prefork import call_capture_output
result, version, stderr = call_capture_output([kwargs["cc"], "--version"])
if result != 0:
raise ToolchainGuessError("compiler version query failed: "+stderr)
#Original Code
# if "Free Sofware Foundation" in version:
#Modified line to recognize clang
if ("Free Software Foundation" in version) or ("clang" in version):
if "-Wstrict-prototypes" in kwargs["cflags"]:
kwargs["cflags"].remove("-Wstrict-prototypes")
if "darwin" in version:
# Are we running in 32-bit mode?
# The python interpreter may have been compiled as a Fat binary
# So we need to check explicitly how we're running
# And update the cflags accordingly
import sys
if sys.maxint == 0x7fffffff:
kwargs["cflags"].extend(['-arch', 'i386'])
return GCCToolchain(**kwargs)
else:
raise ToolchainGuessError("unknown compiler")
How can I tell codepy.toolchain which compiler to use?

Setting the CC environment variable to an appropriate value might help. Also check your Python Makefile, stored under config in your Python lib directory. That's where the config information comes from in the first place.

I modified the file ~/.aksetup-defaults.py to:
#For CUDA
CUDA_ROOT=['/Developer/NVIDIA/CUDA-5.0/']
CUDADRV_LIB_DIR=['/Developer/NVIDIA/CUDA-5.0/lib']
CUDA_INC_DIR=['/Developer/NVIDIA/CUDA-5.0/include']
#For Boost
BOOST_INC_DIR=['${HOME}/pool/include/']
BOOST_LIB_DIR=['${HOME}/pool/lib']
USE_SHIPPED_BOOST=False
This solved the issue with toolchain, but I'm still trying to figure out how to compile.

Related

pyomo mindtpy example program when run becomes unfeasible for binary variable

So I installed pyomo, glpk, and ipopt with anaconda,
When I run the example code here: https://pyomo.readthedocs.io/en/stable/contributed_packages/mindtpy.html
from pyomo.environ import *
model = ConcreteModel()
model.x = Var(bounds=(1.0,10.0),initialize=5.0)
model.y = Var(within=Binary)
model.c1 = Constraint(expr=(model.x-3.0)**2 <= 50.0*(1-model.y))
model.c2 = Constraint(expr=model.x*log(model.x)+5.0 <= 50.0*(model.y))
model.objective = Objective(expr=model.x, sense=minimize)
SolverFactory('mindtpy').solve(model, mip_solver='glpk', nlp_solver='ipopt',tee=True)
model.objective.display()
model.display()
model.pprint()
I get the output that the binary variable has apparently become infeasible:
python minlpex.py
INFO: ---Starting MindtPy---
INFO: Original model has 2 constraints (2 nonlinear) and 0 disjunctions, with
2 variables, of which 1 are binary, 0 are integer, and 1 are continuous.
INFO: NLP 1: Solve relaxed integrality
INFO: NLP 1: OBJ: 1.0 LB: 1.0 UB: inf
INFO: ---MindtPy Master Iteration 0---
INFO: MIP 1: Solve master problem.
WARNING: Empty constraint block written in LP format - solver may error
Traceback (most recent call last):
File "minlpex.py", line 13, in <module>
op.SolverFactory('mindtpy').solve(model, mip_solver='glpk', nlp_solver='ipopt',tee=True)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/mindtpy/MindtPy.py", line 370, in solve
MindtPy_iteration_loop(solve_data, config)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/mindtpy/iterate.py", line 30, in MindtPy_iteration_loop
handle_master_mip_optimal(master_mip, solve_data, config)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/mindtpy/mip_solve.py", line 62, in handle_master_mip_optimal
config)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/gdpopt/util.py", line 199, in copy_var_list_values
v_to.set_value(value(v_from, exception=False))
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/core/base/var.py", line 173, in set_value
if valid or self._valid_value(val):
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/core/base/var.py", line 185, in _valid_value
"domain %s" % (val, type(val), self.domain))
ValueError: Numeric value `0.22709088987977885` (<class 'float'>) is not in domain Binary
So I was a little confused, since this was a code provided, I would not expect it to error like this. So I feel like I'm messing something up or I am missing some required library?
Thanks a lot.
Looks like something must be wrong with the conda pyomo install or ipopt install.
When I reinstalled using pip for ipopt and compiling pyomo from github source everything works fine.

cx_Freeze: 'The system cannot find the file specified' error during build [win10] [PyQt4] [python2.7]

I'm trying to create a .exe file from a python script (which use PyQt4 GUI and matplotlib). I'm using cx_Freeze version 5.1.1 for 64-bit windows with the following setup.py:
import cx_Freeze
import sys
import matplotlib
base = "Win32GUI"
includes = ["atexit"]
buildOptions = dict(
#create_shared_zip=False,
#append_script_to_exe=True,
includes=includes
)
executables = [cx_Freeze.Executable(script = "main.py", base = base)] # icon = "chart32.jpg")]
cx_Freeze.setup(
name= "1ChPlotGUI",
options = dict(build_exe=buildOptions), # {"build_exe": {"packages": ["matplotlib"], "include_files":["chart32.jpg"]}},
version = "0.01",
description = "1 Channel Plotting app with GUI",
executables = executables
)
after running
python setup.py build
in the cmd from the position of
C:\Users\Us.Er\Pyth-examples\Qt\UI-examples\ChannelplotGUI-to-exe
I have something like this:
running build
running build_exe
copying c:\users\Us.Er\appdata\local\enthought\canopy\user\lib\site-
packages\cx_Freeze\bases\Win32GUI.exe -> build\exe.win-amd64-2.7\main.exe
copying
c:\users\Us.Er\appdata\local\enthought\canopy\user\scripts\python27.dll ->
build\exe.win-amd64-2.7\python27.dll
Traceback (most recent call last):
File "setup.py", line 23, in <module>
executables = executables
File "c:\users\Us.Er\appdata\local\enthought\canopy\user\lib\site-packages\cx_Freeze\dist.py", line 349, in setup
distutils.core.setup(**attrs)
File "C:\Users\Us.Er\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.7.4.3348.win-x86_64\lib\distutils\core.py", line 151, in setup
dist.run_commands()
File "C:\Users\Us.Er\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.7.4.3348.win-x86_64\lib\distutils\dist.py", line 953, in run_commands
self.run_command(cmd)
File "C:\Users\Us.Er\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.7.4.3348.win-x86_64\lib\distutils\dist.py", line 972, in run_command
cmd_obj.run()
File "C:\Users\Us.Er\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.7.4.3348.win-x86_64\lib\distutils\command\build.py", line 127, in run
self.run_command(cmd_name)
File "C:\Users\Us.Er\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.7.4.3348.win-x86_64\lib\distutils\cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "C:\Users\Us.Er\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.7.4.3348.win-x86_64\lib\distutils\dist.py", line 972, in run_command
cmd_obj.run()
File "c:\users\Us.Er\appdata\local\enthought\canopy\user\lib\site-packages\cx_Freeze\dist.py", line 219, in run
freezer.Freeze()
File "c:\users\Us.Er\appdata\local\enthought\canopy\user\lib\site-packages\cx_Freeze\freezer.py", line 626, in Freeze
self._FreezeExecutable(executable)
File "c:\users\Us.Er\appdata\local\enthought\canopy\user\lib\site-packages\cx_Freeze\freezer.py", line 232, in _FreezeExecutable
self._AddVersionResource(exe)
File "c:\users\Us.Er\appdata\local\enthought\canopy\user\lib\site-packages\cx_Freeze\freezer.py", line 172, in _AddVersionResource
stamp(fileName, versionInfo)
File "c:\users\Us.Er\appdata\local\enthought\canopy\user\lib\site-packages\win32\lib\win32verstamp.py", line 159, in stamp
h = BeginUpdateResource(pathname, 0)
pywintypes.error: (2, 'BeginUpdateResource', 'The system cannot find the file specified.')
What is the possible solution for above problem?
EDIT
To be clear: I don't want to add any icon for now. I am happy with just a simple, working .exe
I have added the target in this way:
executables = [cx_Freeze.Executable(script = "main.py", base = base, targetName="main.exe")]
I have tried to add
targetDir = "C:\Users\Us.Er\Pyth-examples\Qt\UI-examples\ChannelplotGUI-to-exe"
anyway, the return is:
TypeError: __init__() got an unexpected keyword argument 'targetDir'
The main problem is as before - the error with last lines as:
h = BeginUpdateResource(pathname, 0)
pywintypes.error: (2, 'BeginUpdateResource', 'The system cannot find the file specified.')
In my case the issue was solved by changing
'build_exe': 'build_folder'
to
'build_exe': './/build_folder'
This is because open(pathname) would work on the pathname='build_folder\\executable.exe', allowing the code to advance during the check at the begginning of the stamp method, but BeginUpdateResource(pathname, 0) would only work on './/build_folder\\executable.exe'
Solution
options = {
'build_exe': {
'build_exe': './/build'
}
}
Note: .//build here, build is the folder name where you want to build your project.
Thanks to help of user jpeg I managed to successfully freeze the script.
To eliminate the problem with path I have added a line print(pathname) in win32verstamp.py before line 159.
The print statement worked fine showing the relative path to the newly made .exe file.
Despite showing the correct path, the error was still present. I went to the stamp() definition in win32verstamp.py and found a try - except block, and have inserted print(pathname) over there.
The part is:
def stamp(pathname, options):
# For some reason, the API functions report success if the file is open
# but doesnt work! Try and open the file for writing, just to see if it is
# likely the stamp will work!
#print("Current path is " + pathname)
try:
f = open(pathname, "a+b")
f.close()
print("Possible to open" + pathname) #<---line added
except IOError, why:
print "WARNING: File %s could not be opened - %s" % (pathname, why)r code here
....
Since that the freeze was possible. (Not sure why tho, I'm happy to add some info if explanation is known)
I had the same issue tying to freeze my PyQt5 app (using python 3.7 under windows 10) with the following message :
h = BeginUpdateResource(pathname, 0)
pywintypes.error: (2, 'BeginUpdateResource', 'Le fichier spécifié est introuvable.')
So I choose to use cx_freeze Sample provided as a first step.
(This samples are availables after cx_freeze installation under MyVENVpath\Lib\site-packages\cx_Freeze\samples)
The very basic PyQt5 exemple was working, but my case was way more complex as I used several sub Python class I have developed.
I honestly don't really understand why but I was able to make it work So here is all modifications I have performed :
In setup.py identify sub python files
add my personnal packages and class like this :
include = ["PyQt5", "PySerial", "myclass1", "myclass2", "myclass3", "PandasModel"]
packages = ["PySerial", "myclass1", "myclass2", "myclass3", "PandasModel", "pandas","math"]
Use relative path in setup.py
executables = [cx_Freeze.Executable(os.getcwd() + r'\..\..\MyFileName.py', base=base)]
include_files = [(os.getcwd() + r'\..\..\CalibrationTool.ui', r'Inputs\GUI\CalibrationTool.ui')
The reason for "...." is to jump two directories higher level, as my python.exe is in sub folders VENV\Script\
Add python Files containing "Myclass1" at same level as setup.py.
Even if in my application, class files are in a subfolder Input\Packages\Myclass1File.py, I had to copy-paste it at the same level as setup.py. This way the include and package line of setup.py work and was able to generate the executable.
Last but not least,the terminal command
I have no idea why but the only command who did suceed was:
to use full path to the python.exe and full path to the setup.py with the "build" at the end.
cmd:>C:\Folder1\Folder2\MyVENV\Scripts\python.exe D:\Folder1\Folder2\setup.py build
This finaly created the executable in a build folder under C:\Folder1\Folder2\MyVENV\Scripts\

Can't close file, being used by another process, windows error 32?

I'm trying to have a script that will automatically download files and install them in the correct directories. However When i try to delete the downloaded file after everything has been extracted correctly, I get an OS error that its being used by another process. I use with so the file should close correctly, but even if i put manual closes for source and zip_file it still gives this error. Does anyone know why?
It is most definitely being used by the python interpreter and not any other program, i tested this using "Unlocker". So my only conclusion is the file isn't closing after i open it.
# Downloads, updates, and installs the most recent version of flippy. Only tested on Windows operating systems.
import os
import shutil
import zipfile
import urllib
from os.path import expanduser
import maya.cmds as mc
import maya.mel as mel
download_dir = expanduser("~")
scripts_path = mc.internalVar(usd=True)
prefs_path = mc.internalVar(upd=True)
urllib.urlretrieve(r"https://drive.google.com/uc?export=download&id=0BwF-kFX824PuXzZVQmJja3JzNkE", r"%s/flippy.zip" %(download_dir))
with zipfile.ZipFile("%s/flippy.zip" %(download_dir)) as zip_file:
for member in zip_file.namelist():
with zip_file.open(member) as source:
filename = os.path.basename(member)
if filename == 'flippy.py' or filename == 'flippy_invert_attrs.txt':
target = file(os.path.join(scripts_path, filename), "wb")
copy = True
elif filename == 'flippy_icon.png':
target = file(os.path.join("%s/icons" %(prefs_path), filename), "wb")
copy = True
elif filename == 'shelf_NeoTools.mel':
target = file(os.path.join("%s/shelves" %(prefs_path), filename), "wb")
copy = True
elif filename == 'CHANGELOG.txt':
copy = False
for line in source:
print line
else:
copy = False
# copy file (taken from zipfile's extract)
if copy:
with source, target:
shutil.copyfileobj(source, target)
os.remove("%s/flippy.zip" %(download_dir))
This code gives the following error:
# Error: 32
# Traceback (most recent call last):
# File "<maya console>", line 40, in <module>
# WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'F:/My Documents/flippy.zip' #

Creating a single executable .exe from Python script that uses PuLP

I have been struggling with this for a while. I have used py2exe and cx_freeze to package everything. I am using a 32 bit machine and Everything works fine and the interface opens up and everything just that I know the entire puLP package is not being copied correctly into the package. I know this because the solver does not work. Inside both library zips in the packages created by py2exe and cx_freeze, there are only .pyc files included where PuLP has cbc.exe and other file types that make the solver work.
Is there any work around this? I have tried copying the actual PuLP package into the library.zip as well as into the dist folder and that didn't work.
Here is the setup I used for py2exe:
import sys
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need fine tuning.
build_exe_options = {"packages": ["pulp"],
"icon": "icon.ico",
"include_files": ["icon.ico","cbc.exe","cbc-32","cbc-64","cbc-osx-64","CoinMP.dll"]}
# GUI applications require a different base on Windows (the default is for a
# console application).
base = None
if sys.platform == "win32":
base = "Win32GUI"
setup( name = "my_app",
version = "0.1",
options = {"build_exe": build_exe_options},
executables = [Executable("my_app.py", base=base)])
I received the following error:
Exception in Tkinter callback
Traceback (most recent call last):
File "Tkinter.pyc", line 1470, in __call__
File "my_app.py", line 796, in <lambda>
File "my_app.py", line 467, in branchAndBound
File "pulp\pulp.pyc", line 1619, in solve
AttributeError: 'NoneType' object has no attribute 'actualSolve'
EDIT
I tried to change the paths to cbc.exe and CoinMP.dll but that didn't really work either. I am probably missing something.
I changed the following inside solvers.py in the PuLP package:
try:
coinMP_path = config.get("locations", "CoinMPPath").split(', ')
except configparser.Error:
coinMP_path = ['/Users/al/Desktop/my_app/build/exe.win32-2.7']
try:
cbc_path = config.get("locations", "CbcPath")
except configparser.Error:
cbc_path = '/Users/al/Desktop/my_app/build/exe.win32-2.7'
try:
pulp_cbc_path = config.get("locations", "PulpCbcPath")
except configparser.Error:
pulp_cbc_path = '/Users/al/Desktop/my_app/build/exe.win32-2.7'
What am I missing or doing wrong?

Call python-nmap PortScanner(), nmap not found

In my virtualenv I installed python-nmap and nmap is installed (OS X).
But if I call mmap like (virtualenv activated...):
import sys
sys.path.append('/usr/local/bin')
import nmap
nm = nmap.PortScanner()
I get the following error:
Raise PortScannerError('nmap program was not found in path')
nmap.nmap.PortScannerError: 'nmap program was not found in path'
Is there still another way to enter the path to nmap?
Portet the project to Python 3 and update nmap to 0.3.3 now it works.
Thanks a lot!
install nmap with homebrew
brew install nmap
Then your install will work correctly.
sys.path governs where the Python interpreter looks for importing modules. The "path" in your error is the OS's PATH environment variable, which tells the OS where to look for Nmap. You can either set this directly with os.putenv or you can pass the full path to the nmap binary in the nmap.PortScanner constructor:
nm = nmap.PortScanner('/usr/local/bin/nmap')
I have tried these methods but not worked then I further looked into it.
Error:
>>> import nmap
>>> nm = nmap.PortScanner()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/nmap/nmap.py", line 137, in __init__
raise PortScannerError('nmap program was not found in path')
nmap.nmap.PortScannerError: 'nmap program was not found in path'
so I looked into file "/usr/local/lib/python2.7/dist-packages/nmap/nmap.py". The code where they are checking for nmap is not working for some reason.
code:
# regex used to detect nmap
regex = re.compile('Nmap version [0-9]*\.[0-9]*[^ ]* \( http://nmap\.org \)')
# launch 'nmap -V', we wait after 'Nmap version 5.0 ( http://nmap.org )'
p = subprocess.Popen(['nmap', '-V'], bufsize=10000, stdout=subprocess.PIPE)
self._nmap_last_output = p.communicate()[0] # store stdout
for line in self._nmap_last_output.split('\n'):
if regex.match(line) is not None:
is_nmap_found = True
# Search for version number
regex_version = re.compile('[0-9]+')
regex_subversion = re.compile('\.[0-9]+')
rv = regex_version.search(line)
rsv = regex_subversion.search(line)
if rv is not None and rsv is not None:
# extract version/subversion
self._nmap_version_number = int(line[rv.start():rv.end()])
self._nmap_subversion_number = int(line[rsv.start()+1:rsv.end()])
break
if is_nmap_found == False:
raise PortScannerError('nmap program was not found in path')
I found out that first regex is not working so I changed this:
from :
regex = re.compile('Nmap version [0-9]*\.[0-9]*[^ ]* \( http://nmap\.org \)')
to:
regex = re.compile('Nmap version [0-9]*\.[0-9]*)
Now it is working just like it suppose to work!