at the moment I'm facing some problems compiling (and running) a (huge) own project with support of Tensorflow. On my own system (Ubuntu 16.04 LTS) everything works fine. Same procedure on a cluster leads to a compile error and I'm not able to find a solution yet.
System information
Have I written custom code: YES
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS 7.4.1708
TensorFlow installed from (source or binary): source (using the git repo)
TensorFlow version (use command below): 1.9
Python version: 2.7.15
Bazel version (if compiling from source): 0.16
GCC/Compiler version (if compiling from source): 7.30
CUDA/cuDNN version: not used
GPU model and memory: Tesla K20m
Exact command to reproduce:
Cloned tensorflow repo from github
Configure Bazel (./configure in tensorflow repo)
Built libtensorflow_cc.so with bazel (worked fine!!!)
Downloaded dependencies with delivered script tensorflow/contrib/makefile/download_dependencies.sh
(tried installing protobuf & eigen manually, too)!
Installed protobuf with ./autogen.sh && ./configure && make && make install
Installed Eigen from downloaded dependencies
Copied libraries, headers and includes into own project:
$ cp bazel-bin/tensorflow/libtensorflow_cc.so ../tf_project/lib/
$ cp bazel-bin/tensorflow/libtensorflow_framework.so ../tf_project/lib/
$ cp /tmp/proto/lib/libprotobuf.a ../tf_project/lib/
$ mkdir -p ../tf_project/include/tensorflow
$ cp -r bazel-genfiles/ * ../tf_project/include/
$ cp -r tensorflow/cc ../tf_project/include/tensorflow
$ cp -r tensorflow/core ../tf_project/include/tensorflow
$ cp -r third_party ../tf_project/include
$ cp -r /tmp/proto/include/ * ../tf_project/include
$ cp -r /tmp/eigen/include/eigen3/ * ../tf_project/include
Remark: On my own system it works this way for weeks now. I can use tensorflow in my own project with an exported model trained with keras inside a python project. I make predictions using client_sessions and many other functions of the tensorflow framework and it works properly.
Problem: At the cluster I can compile tensorflow as dynamic library, install protobuf and eigen.
When I try to compile my project (similar process regarding my own system) without significant changes it doesn't work and stops with following error message:
.../tensorflow/include/tensorflow/core/framework/tensor.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^~~~~
.../include/tensorflow/core/framework/tensor.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update
^~~~~
.../include/tensorflow/core/framework/tensor.pb.h:14:2: error: #error your headers.
#error your headers.
^~~~~
.../include/tensorflow/core/framework/tensor.pb.h:27:10: fatal error: google/protobuf/inlined_string_field.h: No such file or directory
#include <google/protobuf/inlined_string_field.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
So, clearly this should be the issue:
This file was generated by a newer version of protoc which is incompatible with your Protocol Buffer headers. Please update your headers.
and
fatal error: google/protobuf/inlined_string_field.h: No such file or directory
Tried solutions:
I tried different versions of protobuf but there was an error every single try.
I tried installing protobuf and eigen manually (without the download_dependencies.sh script)
I'm wondering because my own installation following exact the same steps works properly. Maybe there is an issue with one of the components unless I tried different versions to make sure that these aren't "new" issues.
Can someone help me solving this error that I can compile and run this project at the other machine?
Looking forward to get helpful solutions :) Thank you very much for the support!
Best regards from Germany!
First, I checked on github, tensorflow r1.9 requires protobuf >=3.6.0. With download_dependencies.sh script, you always get protobuf 3.5.0, in which lack of inlined_string_field.h and some other headers.
Second, some errors that ask you to update protobuf version. I tried many versions of protobuf too. Only version 3.6.0 works well, rather than version 3.6.1 or older ones.
For these errors,
error This file was generated by a newer version of protoc which is
fatal error: google/protobuf/inlined_string_field.h: No such file or directory
It is a mismatch version problem. My solutions is that manually download protobuf 3.6.0 from
https://github.com/protocolbuffers/protobuf/releases/download/v3.6.0/protoc-3.6.0-linux-x86_64.zip
install it and cp /usr/local/include/google to somewhere/tf/include. It works quite well for me. I don't know if you tried version 3.6.0.
download_dependencies.sh script would provide a mismatch version of
protobuf. See the issue I posted on github
https://github.com/tensorflow/tensorflow/issues/22536
Also, I noticed your copied files are different from mine. I did in this way.
sudo mkdir /usr/local/tensorflow/include
sudo cp -r tensorflow/contrib/makefile/downloads/eigen/Eigen /usr/local/tensorflow/include/
sudo cp -r tensorflow/contrib/makefile/downloads/eigen/unsupported /usr/local/tensorflow/include/
sudo cp -r tensorflow/contrib/makefile/gen/protobuf/include/google /usr/local/tensorflow/include/
sudo cp tensorflow/contrib/makefile/downloads/nsync/public/* /usr/local/tensorflow/include/
sudo cp -r bazel-genfiles/tensorflow /usr/local/tensorflow/include/
sudo cp -r tensorflow/cc /usr/local/tensorflow/include/tensorflow
sudo cp -r tensorflow/core /usr/local/tensorflow/include/tensorflow
sudo mkdir /usr/local/tensorflow/include/third_party
sudo cp -r third_party/eigen3 /usr/local/tensorflow/include/third_party/
sudo mkdir /usr/local/tensorflow/lib
sudo cp bazel-bin/tensorflow/libtensorflow_*.so /usr/local/tensorflow/lib
Btw, I run build_all_linux.sh instead of download_dependencies.sh.
I hope it will be helpful for you.
I meet this problem before, following is my solution:
I successfully built tensorflow-r1.8 with bazel. And I found protoc in following path is 3.5.0:
/home/zsb/.cache/bazel/_bazel_zsb/1372f28eb0671f692e7ac38330377d8c/execroot/org_tensorflow/bazel-out/host/bin/external/protobuf_archive/protoc --version
but actually my system config is using protoc version 3.4.0
I confirmed this by type "protoc --version" directly.
So finally I update system protobuffer version to 3.5.0 and this problem fixed.
When trying to install shogun im getting this error. Can some one suggest a solution?
N: Ignoring file '50unattended-upgrades.ucf-dist' in directory '/etc/apt/apt.conf.d/' as it has an invalid filename extension
E: Unable to locate package libshogun17
Im using ubuntu operating system.
See this explanation of the origin and purpose of .ucf-dist files. This means you can either ignore the notice (that's what the N: prefix stands for) or remove said file.
sudo rm /etc/apt/apt.conf.d/50unattended-upgrades.ucf-dist
libshogun17 is not part of ubuntu by default. Currently we supply two different ppa's for xenial and trusty:
for nightly builds: https://launchpad.net/~shogun-toolbox/+archive/ubuntu/nightly
for stable releases: https://launchpad.net/~shogun-toolbox/+archive/ubuntu/stable
please follow the instruction either the given page or http://shogun.ml/install#ubuntu how to add the ppa to your system.
Note, as mentioned above those PPAs only supply packages for ubuntu trusty or xenial.
Okay so i'm trying to install Homebrew, so that I can I can install nodejs and npm. However I'm using this command from brew.sh;
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Once I install Homebrew It keeps returning this error
It appears Homebrew is already installed. If your intent is to reinstall you
should do the following before running this installer again:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
The current contents of /usr/local are bin CODEOFCONDUCT.md etc lib libexec Library LICENSE.txt README.md sbin share .git .github .gitignore
omz_urlencode:42: failed to load module `zsh/regex': dlopen(/usr/local/Cellar/zsh/5.2/lib/zsh/regex.so, 9): image not found
omz_urlencode:42: -regex-match not available for regex'
I honestly have no idea what's happening here. I'm not sure If I have to symlink Homebrew up or what. But I have tried everything I know so far (which isn't much). If anyone could be kind enough to give me instructions as what too look for to solve the issue, I'm all ears.
Thank you for listening!
It seems it's a bug that have been corrected in recent version of OMZ, you should update it.
You get the output:
zsh: command not found: homebrew
Because homebrew is not a command. Try brew -v instead.
To resolve:
Warning: node-5.10.1 already installed, it's just not linked
Use brew link node
Okay so I think I have made some progress, It may have been for some weird reason I didn't have permission writes to run the 'brew link < package > ' so i ran the following command:
'sudo chown -R /usr/local/share/systemtap/tapset && brew doctor'
Once I ran 'brew link node' it successfully created 7 symlinks for the node directory on the following path;
'Linking /usr/local/Cellar/node/5.10.1... 7 symlinks created'
However, when I installed live-server via npm; I typed 'live-server -v' and it again returned 'zsh: command not found: live-server'. In addition to this it keeps telling me I have 'unbrewed dylibs':
Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected dylibs:
/usr/local/lib/libociei.dylib
So im still unsure what the exact issue is.
P.S. Apologies for not posting this correctly, as im trying to see where I can seperate it into commands like you corrected in the first post, if you could link me to a post where it tells me how to use it properly i'll be more than happy to read it.
Thanks,
I am experiencing very annoying problems with the application apktool problem.
I do not understand what i am doing wrong, or what the problem is.
I tried this on debian , and on linux mint. I used different versions of apktool,
resulting in the same error:
I: Checking whether sources has changed...
I: Checking whether resources has changed...
I: Building resources...
Exception in thread "main" brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [aapt, p, -F, /tmp/APKTOOL3630495287059303807.tmp, -I, /home/awesomename/apktool/framework/1.apk, -S, /home/awesomename/out/./res, -M, /home/awesomename/out/./AndroidManifest.xml]
at brut.androlib.res.AndrolibResources.aaptPackage(Unknown Source)
at brut.androlib.Androlib.buildResourcesFull(Unknown Source)
at brut.androlib.Androlib.buildResources(Unknown Source)
at brut.androlib.Androlib.build(Unknown Source)
at brut.androlib.Androlib.build(Unknown Source)
at brut.apktool.Main.cmdBuild(Unknown Source)
at brut.apktool.Main.main(Unknown Source)
Caused by: brut.common.BrutException: could not exec command: [aapt, p, -F, /tmp/APKTOOL3630495287059303807.tmp, -I, /home/windows/apktool/framework/1.apk, -S, /home/windows/out/./res, -M, /home/windows/out/./AndroidManifest.xml]
at brut.util.OS.exec(Unknown Source)
... 7 more
Caused by: java.io.IOException: Cannot run program "aapt": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:485)
... 8 more
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:135)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
... 10 more
It seems it can not use aapt , but i read about apktool.
And it seems that aapt is build inside apktool , why is it not working ?
It seems there's some problem in building the resources while recompiling the apk.
what you can do is, when you decompile your apk use this command
apktool d -f -r apkfilename.apk
here -f is to replace previous decompiled apk's code and -r is to ignore the decompiling of resources.
this would prevent the resources from being decompiled and will simply copy the same resources when you recompile the apk.
In case you've been using v1 and now upgraded to v2, try manually deleting the framework file.
On windows 8 it's normally at C:\Users\YourName\apktool\framework\1.apk.
The file should be regenerated once you try to build something.
My problem was solved by deleting the \framework\1.apk, making a backup on the files I modified, ereasing the dir and decompiling the *.apk again, etc... (on linux, the path is home/[user]/apktool/...). After the update, apktool always loaded the old resource table. N
For me, I solved this problem by first clearing apktool's framework directory by typing in the terminal.
$ apktool empty-framework-dir
Afterwards I uninstalled apktool and related files by typing
$ sudo apt purge apktool
Then i went to https://bitbucket.org/iBotPeaches/apktool/downloads/ to get the latest jar file for apktool(apktool_2.5.0.jar as at the time of writing this).
On first run
$ java -jar apktool_2.5.0.jar b <MyAPP.apk> #Without ><
it works.
since I work with apktool most of the times I needed a situation where I can run apktool from anywhere so I gave the jar file execute permissions by typing
$ sudo chmod +x apktool_2.5.0.jar
Afterwards I moved it /usr/bin/ by typing
$ sudo apktool_2.5.0.jar /usr/bin/
Definitely seems like the aapt PATH problem I had awhile back. Have you added aapt to PATH? If you still have problems, I have made a good apk kit in bash to avoid all these dependency problems. It supports apktool, signapk, zipalign,adb, fastboot, and heimdall. Check it out. All you need is a current java install.
http://forum.xda-developers.com/android/development/toolkit-apk-munky-rench-t3026757/post58747626#post58747626
There isn’t really enough information to give you a definite answer.
How ever you mentioned using different versions but the aapt issue was solved in version 2.4. Dependencies have been reduced to java version 1.8 or greater and the framework.
I use Debian and have the following:
Apktool 2.4
java version 11
Android framework
That’s all it took to get rid of the aapt path error.
The last error I came across was unrelated to aapt but was on the framework so I ran this command
apktool empty-framework-dir
And it solved it.
try to put the dir which include aapt file to your PATH. for example, export PATH=$PATH:./ ./apktool b
try to install ia32-libs and update latest version of apktool. (if possible restart)
apktool requires "ia32-libs" which is not available after Ubuntu 12.04. install ia32-libs
sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 lib32stdc++6
Download latest version of apktools.jar - https://bitbucket.org/iBotPeaches/apktool/downloads
apktool complete installation guide - http://ibotpeaches.github.io/Apktool/install/
I just encounter same problem when run apktool d foo.apk(decompiled success) and then apktool b foo(recompile failed with similar error).
The apktool tool above was installed via sudo apt-get install apktool on Kali Linux.
So, the solution was visits apktool's official site, e.g. https://connortumbleson.com/2017/01/23/apktool-v2-2-2-released/ (it's latest version at this time of writing), download it, md5sum it, e.g. md5sum apktool_2.2.2.jar to verify, then rename that apktool_2.2.2.jar to apktool.jar.
Then do java -jar ./apktool.jar b foo to recompile, it success without error (the generated apk located at ./foo/dist/foo.apk).
The main issue is apktool version you need 2.4.0
You must manually install it from ibotpeaches git hub
here some good info
https://www.youtube.com/watch?v=kB6s10Uwpcs
and a automated script for kali
https://github.com/catenatedgoose?tab=repositories
In my mind the problem is how you install apktool...
I had the same problem and I did this and it worked very well:
For installation you first have to remove any installed apktool by the command:
sudo apt purge apktool
Then you'll have to install apktool but in a different way.
To continue save the link bellow as apktool in a directory.
[https://raw.githubusercontent.com/iBotPeaches/Apktool/master/scripts/linux/apktool]
Then open this link below and download the latest apktool.jar file: https://bitbucket.org/iBotPeaches/apktool/downloads/
Then rename the file as apktool.jar
After that give both files the permission by the command:
Sudo chmod -x apktool.jar
And for the saved script:
Sudo chmod -x apktool
At the end copy both files in the directory:
/usr/local/bin
By the command:
Sudo cp apktool.jar /usr/local/bin
And the script file:
Sudo cp apktool /usr/local/bin
After that try running apktoolin the terminal.
The solution is to include your apktool directory into your system PATH.
I installed ColdFusion 9.0.0 today and I'm trying to upgrade to 9.0.1.
Linux 64bit environment.
I don't have root access.
/tmp doesn't seem to have enough room to install 9.0.1
I found the following errors in /opt/coldfusion9/Adobe_ColdFusion_9.0.1_InstallLog.log
Install File: /tmp/515453.tmp/cf-multi-startup
Status: ERROR
Additional Notes: ERROR - ZeroGlq: No space left on device
Install File: /tmp/515453.tmp/cf-init.sh
Status: ERROR
Additional Notes: ERROR - ZeroGlq: No space left on device
I tried -D IATEMPDIR=/opt/temp, but installer says it's trying to use /home/coldfusion due to lack of space in /tmp. It tries to use /tmp anyway.
Any ideas?
Here's my final script that worked. I had to fill /tmp and /home to 100% to get a decent error response out of the bin file. I'm sure my Linux admins were very pleased.
cd /opt/coldfusion9/bin
IATEMPDIR=/home/coldfusion
export IATEMPDIR
LAX_DEBUG=1
export LAX_DEBUG
./ColdFusion_update_901_WWEJ_linux64.bin