Installing CLD libary on windows and bind to Python - c++

I have a need to make use of Chromium's Compact Language Detector library within a Python script.
AFAIK, there are two projects that leverage this library, but I have been having troubles with getting either of them set up on a Windows 7 machine.
I had some similar problems with Mike McCandless Original Project (GoogleCode), but I then spotted Matt Sanford fork on the same Project (github). For the purpose of this question, I will focus on Matts project, as it seems to have been updated more often (but happy to get an answer that works for either project).
Downloaded the project as a zip, and extracted to my local drive.
Ran vcvarsall.bat from CMD
Then ran the build.win.cmd batch file. This ran ok.
Copied the resulting libcld.lib to ports/python/cld.lib
Tried running the setup.py but it returns with the following against line 12.
exceptions.TypeError: init() keywords must be strings
++EDIT++
Found this issue:
setup.py fails on Windows. The hackish solution to fix this is to make
pkgconfig return the dict {'define_macros': [('WIN32',None)],
'libraries': packages}
Im a bit of a rookie, but not entirely sure on the steps to implement this hack. Can anyone give me some slightly more verbose steps?

Try this patch (works for me):
diff --git a/ports/python/setup.py b/ports/python/setup.py
index e1950c3..889f21a 100644
--- a/ports/python/setup.py
+++ b/ports/python/setup.py
## -9,7 +9,10 ## def pkgconfig(*packages, **kw):
module = Extension('cld',
['pycldmodule.cc'],
- **pkgconfig('cld'))
+ define_macros=[('WIN32', None)],
+ libraries=['libcld'],
+ include_dirs=['..\\..\\'],
+ library_dirs=['..\\..\\'])
setup(name='cld',
version='0.031415',

Related

Python2 required as dependencie but already installed

First of all, sorry for my english and sorry for taking your time, I'm pretty sure the answers are online but it seems like I can't find the right keywords for this issue.
The problem: Python2 is installed but still asked as a dependency for every python2 module I want to install with my package manager.
I built python2 from source on Archlinux in a proot environment
(because I use termux on a non rooted phone and it's probably why yay did not worked as expected to install python2) and I think I did it well because "python2" open the python command line and "python2 -V" return me "Python 2.7.18", i can execute python scripts with it, etc.
I built python by downloading and uncompressing python2 from python.org, then in the uncompressed file ./configure --enable-optimization , make -s and make install.
I'm a noob so i don't know but i probably just need a way to handle python2 with pacman or a way to tell pacman that python2 is indeed installed.
repo.
I know to handle manualy built pkgs with pacman, but not software built from source. So i'v found a PKGBUILD for python2 but (again, probably because of the proot) when i use makepkg here what's happen :
[...
...
...]
==> Extracting sources...
-> Extracting Python-2.7.18.tar.xz with bsdtar
==> Starting prepare()...
bsdtar: Removing leading '/' from member names
patching file Makefile.pre.in
patch: setting attribute security.selinux for security.selinux: Permission denied
==> ERROR: A failure occurred in prepare().
Aborting...
So if anyone know how could i make makepkg works as intended or how could i tell pacman that python2 is already installed, it would totaly make my day.
PS : I know python2 is deprecated and as it's not updated anymore the security is getting worse and worse, but it's not for my main setup so don't worry. I also think i could install modules manually but it is not something i wish to do since i'd like to install the all BlackArch repo.

Can't build RDCOMClient using rtools40 and R 4.0

A while back, I created a fork of the RDCOMClient package to keep it working with R 3.6 (https://github.com/dkyleward/RDCOMClient). People are now running into issues again because it won't work with R 4.0. The problem doesn't seem as easy to fix, and I'm hoping for some help.
If I flip Rstudio back to R 3.6 (and rtools35), I can use the package after installing with devtools::install_github(). When I try in R 4.0 (and rtools40), the package builds and I can connect over COM to an application. The first line of code below works, and xl is a COM pointer; however, trying to do anything with it (like set Excel to visible) will crash R.
xl <- RDCOMClient::COMCreate("Excel.Application")
xl[["Visible"]] <- TRUE
Again, the above works in R 3.6.
Is there is a way to continue building with the previous rtools? I came across https://github.com/r-windows/rtools-backports#readme, which talks about using rtools35 to keep building packages, so I have hope, but I don't understand how to make it happen.
Alternatively, if there are minor changes I can make to the R or cpp code that will solve my problem, I'm all ears. I'm a cpp novice, though.
This was a quick fix :
install.packages("RDCOMClient", repos = "http://www.omegahat.net/R")
Install R-4.0.0
Install Rtools35
Edit $R_HOME/etc/x64/Makeconf (for R-4.0.0-x64)
Rcmd INSTALL RDCOMClient
Rik's answer was incredibly helpful and got a version working; however, after spending a day on it, I was able to improve on it. I want to put that here in case I have to do it again. The main improvement is being able to build a working package for both 32- and 64-bit architectures. By default, R installs both, and this makes things easier when installing dependent packages.
The first two steps are the same:
Install R-4.0.0 (https://cran.r-project.org/bin/windows/base/old/4.0.0/R-4.0.0-win.exe)
Install Rtools35 (https://cran.r-project.org/bin/windows/Rtools/Rtools35.exe) in directory c:\Rtools
If (like me) you had already installed rtools40, a system environment variable named RTOOLS40_HOME is created. The first step is to change that to:
C:\rtools
If you don't have rtools40 installed, then create the RTOOLS40_HOME system environment variable.
Two changes are still needed in the make files. These are found in your R installation directory.
In etc\x64\Makeconf, add underscores to match the rtools35 directory structure by setting these values:
MINGW_PREFIX = /mingw_$(WIN)
BINPREF ?= "$(RTOOLS40_ROOT)/mingw_64/bin/"
Do the same in etc\i386\Makeconf:
MINGW_PREFIX = /mingw_$(WIN)
BINPREF ?= "$(RTOOLS40_ROOT)/mingw_32/bin/"
Do not set BINPREF as an environment variable, or this will overwrite the makefile changes (like RTOOLS40_HOME does). With these complete, finish off with the same steps that Rik outlined:
Open windows command prompt and change to the directory that contains the RDCOMClient subdirectory and type:
R CMD INSTALL RDCOMClient –-build RDCOMClient.zip
This installs RDCOMClient in the local installation of R-4.0.0 and additionally creates the file RDCOMClient_0.94-0.zip that can be installed on other systems using the following command:
install.packages("RDCOMClient_0.94-0.zip", repos = NULL, type = "win.binary")
I can confirm that the procedure delineated in the answer above leads in the right direction but a few extra steps may be required. I can also confirm that the procedure below produces a Windows binary file that can be installed and will run under R-4.0.0:
Install R-4.0.0 (https://cran.r-project.org/bin/windows/base/old/4.0.0/R-4.0.0-win.exe)
Install Rtools35 (https://cran.r-project.org/bin/windows/Rtools/Rtools35.exe) in directory c:\Rtools
Edit $R_HOME/etc/x64/Makeconf (for R-4.0.0-x64) by changing
## The rtools40 installer sets RTOOLS40_HOME, default to standard install path
RTOOLS40_HOME ?= c:/rtools40
to
## The rtools40 installer sets RTOOLS40_HOME, default to standard install path
RTOOLS40_HOME ?= c:/rtools
Download RDCOMClient-master.zip from https://github.com/omegahat/RDCOMClient (click the green Clone button and select download zip)
Unpack to a directory named RDCOMClient
Ensure that the following PATH variables are set:
C:\Program Files\R\R-4.0.0\bin\x64 (assuming this is the location where R is installed)
C:\Rtools\bin
C:\Rtools\mingw_64\bin
Add environment variable BINPREF with the following value (the final slash is important):
C:/Rtools/mingw_64/bin/
Open windows command prompt and change to the directory that contains the RDCOMClient subdirectory and type:
R CMD INSTALL RDCOMClient –-build RDCOMClient.zip
This installs RDCOMClient in the local installation of R-4.0.0 and additionally creates the file RDCOMClient_0.94-0.zip that can be installed on other systems using the following command:
install.packages("RDCOMClient_0.94-0.zip", repos = NULL, type = "win.binary")
I am using R 4.1.2 and I found RDCOMClient will crash the R Session and the above solutions were not working.
Then, I further check with the source owner and found out the solution.
https://github.com/omegahat/RDCOMClient/issues/36
Duncantl gave the solution and it works.
dir.create("MyTemp")
remotes::install_github("BSchamberger/RDCOMClient", ref = "main", lib = "MyTemp")
If that is successful, we can then load the newly installed package with
library("RDCOMClient", lib.loc = "MyTemp")

Pabot - Unable to run parallel robotframework tests

So, I'm working on a robotframework test project, and the goal is to run several test suites in parallel. For this purpose, pabot was chosen as the solution. I am trying to implement it, but with little success.
My issue is: after installing Pabot (which, I might say, I did by cloning the project and running "setup.py install", instead of using pip, since the corporate proxy I'm behind has proven an obstacle I can't overcome), I created a new directory in the project tree, moved some suites there, and ran:
pabot --processes 2 --outputdir pabot_results Login*.robot
Doing so results in the following error message:
2018-10-10 10:27:30.449000 [PID:9676] [0] EXECUTING Suites.LoginAdmin
2018-10-10 10:27:30.449000 PID:400 EXECUTING Suites.LoginUser
2018-10-10 10:27:30.777000 PID:400 FAILED Suites.LoginUser
2018-10-10 10:27:30.777000 [PID:9676] [0] FAILED Suites.LoginAdmin
WARN: No output files in "pabot_results\pabot_results"
Output:
[ ERROR ] Reading XML source '' failed: invalid mode ('rb') or filename
Try --help for usage information.
Elapsed time: 0 minutes 0.578 seconds
Upon inspecting the stderr file that was generated, I have this message:
Traceback (most recent call last):
File "C:\Python27\Lib\site-packages\robotframework-3.1a2.dev1-py2.7.egg\robot\running\runner.py", line 22, in
from .context import EXECUTION_CONTEXTS
ValueError: Attempted relative import in non-package
Apparently, this has to do with something from the runner.py script, which, if I'm not mistaken, came with the installation of robotframework. Since manually modifying that script does not seem to me the optimal solution, my question is, what am I missing here? Did I forget to do anything while setting this up? Or is this an issue of compatibility between versions?
This project is using Maven as the tool to manage dependencies. The version I am running is 3.5.4. I am using a Windows 10, 64bit system; I have Python 2.7.14, and Robot Framework 3.1a2.dev1. The Pabot version is 0.44. Obviously, I added C:\Python27 and C:\Python27\Scripts to the PATH environment variable.
Edit: I am also using robotframework-maven-plugin version 1.4.0.8, if that happens to be relevant.
Edit 2: added the error messages in text format.
I believe I've come across an issue similar when setting up parallel execution on my machine. Firstly I would confirm that pabot is installed using pip show robotframework-pabot.
Then you should define the directory your results are going to using -d.
I then modified the name of the -o to Output.xml to make it easy to identify.
This is a copy of the code I use. Runs optimally with 8 processes
pabot --processes 8 -d results -o Output.xml Tests
Seems that you stumbled on a bug in the prerelease version of robot framework (3.1a2.dev1).
Please install a release version of robot framework. For example 3.0.4.
Just in case anyone happens to stumble upon this issue in the future:
Since I can't use pip, and I tried a good deal of workarounds that eventually made things more unstable, I ended up saving my project and removing everything Python-related from my system, so as to allow me to install everything from scratch. In a Windows 10, 64bit system, I used:
Python 2.7.14
wxPython 2.8.12.1, win64, unicode, for py27
setuptools 40.2.0 (to allow me to use the easy_install command)
Robot Framework 3.0.4
robotremoteserver 1.1
Selenium2Library 3.0.0
and Pabot version 0.45.
I might add that, when installing the Selenium2Library the way I described above, it eventually tries to download some things from the pip repositories - which, if you have a proxy, will cause you trouble. I solved this problem by browsing https://pypi.org/simple/selenium/, manually downloading the 2.53.6 .tar.gz file, then extracting it and running setup.py install on the command line.
PS: Ideally, though, anyone should be able to use proxy settings from the command line (--proxy http://user:password#server:port) to get pip and then use it; however, for some reason, probably related to network security configurations that I didn't want to lose time with, this didn't work in my case.

Install pycairo or py2cairo using buildout

Does anyone have any experience with trying to install cairo for django using buildout?
It will install pycairo or py2cairo (the first is for python >3.0, the latest is for python 2.6, which I Am using)
I found 2 recipes to use for installation, both of them gives me errors, in buildout.cfg I have them as 2 parts (of course tried them seperately, not thogether).
The errors are totally different, but the result is the same: I can't get cairo installed using buildout.......
based on: http://pypi.python.org/pypi/tl.buildout_gtk/
[cairo_tl]
recipe = tl.buildout_gtk
#pycairo-url = http://cairographics.org/releases/py2cairo-1.10.0.tar.bz2
pycairo-url = http://cairographics.org/releases/py2cairo-1.8.10.tar.gz
pycairo-md5sum = http://cairographics.org/releases/py2cairo-1.8.10.tar.gz.md5
based on: https://bitbucket.org/lgs/yaco.recipe.pycairo
[cairo_yaco]
recipe = yaco.recipe.pycairo
find-links = http://pypi.python.org/pypi/yaco.recipe.pycairo/0.1.1
#find-links = http://pypi.python.org/pypi/yaco.recipe.pycairo/0.1
# pkg-config-path ?
Both need a local install of cairo (pycairo and py2cairo depend on this):
so use on the server:
sudo apt-get install libcairo2-dev
The part jusing tl.buildout_gtk:
Either using the py2cairo-1.10.0 or pycairo-1.8.2 gives me a MD5 checksum mismatch
(see for available releases: http://cairographics.org/releases/)
e.g:
Error: MD5 checksum mismatch downloading 'http://cairographics.org/releases/py2cairo-1.10.0.tar.bz2'
or:
Error: MD5 checksum mismatch downloading 'http://cairographics.org/releases/py2cairo-1.8.10.tar.gz'
The part jusing yaco:
I Am pretty sure this part is not configured correctly (pkg-config-path refering to the local cairo package), but I do not even get to that point.
It gives an error:
DistributionNotFound: zc.recipe.cmmi
However, this is installed in the eggs
By the way, I had the same kind of headache trying to install PIL by buildout, which I solved combining a lot of options and posts on the internet, together with a lot of time and trials and errors.
If anyone want to know how I finally got it working, just ask, and I will publish it.
(I consider myselve still a django starter, so I do not know for sure if anyone is interested in the solution)
PIL does not have all the options I found in cairo, so I started using cairo. On my local PC everything works fine (ubuntu desktop), on my server (Ubuntu server) I can't getting it to work......
Any other options for making drawings on the fly..... (like matplotlib...) let me know.
Pretty sure you have to give the real md5 hash to the pycairo-md5 option instead of a string with some url in it.
I would avoid specialized recipes like those two whenever possible. There are some cases when a specialized recipe really is needed, but those are cases where the dependency is so egregiously eccentric that no common build/install pattern is usable. I don't know pycairo so that may be the case.
But before assuming it is, try the following. Always try using an egg first by adding it to the eggs option of the relevant buildout part. If that doesn't work, update your question with those details, and then try adding a separate zc.recipe.egg part with build options that may help the distribution build successfully. If that doesn't work or the distribution uses a ./configure && make && make install (AKA "CMMI") build system, update your question with those details, and then try using zc.recipe.cmmi to build the package after which you can use the extra-paths option of your zc.recipe.egg part to include the relevant bits of the CMMI part (extra-paths = ${cmmi-part:location}/weird/path/to/python/modules).
This is the general dance for getting poorly/oddly behaved distributions. If this doesn't work, then a specialized recipe may be in order but in general there's a way to get it to work with the above and that will be much more maintainable.

Installing GDAL via buildout on MacOSX

I'm trying to install GDAL via buildout on MacOSX. I have following config:
[gdal]
recipe = zc.recipe.cmmi
url = http://download.osgeo.org/gdal/gdal-1.7.2.tar.gz
extra_options =
--with-geos=${geos:location}/bin/geos-config
--with-static-proj4=${proj:location}
--with-jpeg={libjpeg:location}
--with-spatialite=${spatialite-lib:location}
--with-sqlite=${spatialite-lib:location}
I tried many other configurations but all I've got is the following error:
http://pastebin.com/s0WxfnUu
Any clues?
I've managed to resolve the problem - description below.
Just for the record - proper configuration for building GDAL on MacOSX (for example purposes I've cut off additional stuff):
[libiconv]
# ...
[gdal]
recipe = hexagonit.recipe.cmmi
url = http://download.osgeo.org/gdal/gdal-1.7.3.tar.gz
configure-options =
--with-python
--with-jpeg=internal
--with-spatialite=${spatialite:location}
CFLAGS=-I${libiconv:location}/include
LIBS="${libiconv:location}/lib/libiconv.la ${libiconv:location}/lib/libcharset.la"
You need to specify libiconv (in my example I'm building it from source) - it will prevent charset errors.
Also my problem was in the version of jpeglib, so I've definied --with-jpeg=internal so the GDAL will use it's own libs. Depending on the error you can use --with-png=internal, --with-tiff=internal etc.
Good luck!
Buildout is great for a lot of things, but even I tend to hand over some responsibility to the OS.
The whole whopping geo stack is one of those things. Gdal, geos, libproj4, mapnik, I don't know what. Compiling everything from source is just too cumbersome, especially as you have a lot of dependencies.
In your case, it looks like there's a mismatch in some of the libraries that gdal compiles against. Wrong length of arguments.
My tip is to install gdal and friends via your OS (so: windows installer, apt-get, rpm, whatever).