In our organization, we use conan for our c++ package management. We have our local artifactory server which is acting as our remote for day-to-day usage. When we use a new package, we normally first download it from one of the global conan servers and upload it to our local server. I thought that this way we can work with stations that are not connected to the internet or that the connection is less stable there.
However, I encountered an issue with some of the packages. In specific, when I tried to use boost/1.68.0#conan/stable (that was downloaded and installed on our local artifactory server in the past) I received the following error:
ERROR: Error 403 downloading file https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.gz
Waiting 5 seconds to retry...
ERROR: boost/1.68.0#conan/stable: Error in source() method, line 75
tools.get(url, sha256=sha256)
ConanException: Error 403 downloading file https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.gz
After verifying that my remote is only the local artifactory server (by using conan remote list) - I have searched the dl.bintray.com in the mentioned conan package's folder. I found it in the recipe file: ~/.conan/data/boost/1.68.0/conan/stable/export/conanfile.py in the following function:
def source(self):
if tools.os_info.is_windows:
sha256 = "3b1db0b67079266c40b98329d85916e910bbadfc3db3e860c049056788d4d5cd"
extension = ".zip"
else:
sha256 = "da3411ea45622579d419bfda66f45cd0f8c32a181d84adfa936f5688388995cf"
extension = ".tar.gz"
zip_name = "%s%s" % (self.folder_name, extension)
url = "https://dl.bintray.com/boostorg/release/%s/source/%s" % (self.version, zip_name)
tools.get(url, sha256=sha256)
I thought that the location of the package & the download is managed by conan. Specifying the system to download from another hard-coded location quite miss the point for package management system - doesn't it? Is it a problem with the boost package? Am I using it wrong?
Thanks
Related
I try to build the Bazel C++ Build Tutorial from the Bazel homepage (Getting Started) with this command but without any connection to the internet: bazel build //main:hello-world
The Jenkins Server will not have any connection to the internet so I can't do prefetching or similar. Is there a way how to prepare my C++ project on another computer and transfer the dependencies to the Jenkins Server to do it offline? How? I would just need what I got from the following error message respectively get the tutorial running:
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Repository rules_cc instantiated at:
/DEFAULT.WORKSPACE.SUFFIX:267:6: in <toplevel>
C:/users/XXX/_bazel_XXX/dq2p42jq/external/bazel_tools/tools/build_defs/repo/utils.bzl:201:18: in maybe
Repository rule http_archive defined at:
C:/users/XXX/_bazel_XXX/dq2p42jq/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
WARNING: Download from https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip failed: class java.io.IOException Unable to tunnel through proxy. Proxy returns "HTTP/1.1 501 Not Implemented"
WARNING: Download from https://github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip failed: class java.io.IOException Unable to tunnel through proxy. Proxy returns "HTTP/1.1 501 Not Implemented"
ERROR: An error occurred during the fetch of repository 'rules_cc':
Traceback (most recent call last):
File "C:/users/XXX/_bazel_XXX/dq2p42jq/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl
download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip, https://github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip] to C:/users/XXX/_bazel_XXX/dq2p42jq/external/rules_cc/temp1363696983472254851/b1c40e1de81913a3c40e5948f78719c28152486d.zip: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 501 Not Implemented"
ERROR: Error fetching repository: Traceback (most recent call last):
File "C:/users/XXX/_bazel_XXX/dq2p42jq/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl
download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip, https://github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip] to C:/users/XXX/_bazel_XXX/dq2p42jq/external/rules_cc/temp1363696983472254851/b1c40e1de81913a3c40e5948f78719c28152486d.zip: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 501 Not Implemented"
ERROR: Skipping '//main:hello-world': no such package '#rules_cc//cc': java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip, https://github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip] to C:/users/XXX/_bazel_XXX/dq2p42jq/external/rules_cc/temp1363696983472254851/b1c40e1de81913a3c40e5948f78719c28152486d.zip: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 501 Not Implemented"
WARNING: Target pattern parsing failed.
ERROR: no such package '#rules_cc//cc': java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip, https://github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip] to C:/users/XXX/_bazel_XXX/dq2p42jq/external/rules_cc/temp1363696983472254851/b1c40e1de81913a3c40e5948f78719c28152486d.zip: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 501 Not Implemented"
INFO: Elapsed time: 30.974s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
currently loading: main
Update:
I tried prefetching and it doesn't work. What I did:
run bazel fetch //...
Copy the prefetched data from bazel info output_base directory from local to jenkins server (I had to recreate one sym link to the install dir).
I run bazel build --fetch=false //main:hello-world on jenkins without refatching. Now I get following error:
Loading:
Loading: 0 packages loaded
WARNING: /DEFAULT.WORKSPACE:1:17: External repository 'bazel_tools' is not up-to-date and fetching is disabled. To update, run the build without the '--nofetch' command line option.
ERROR: error loading package '': Every .bzl file must have a corresponding package, but '#bazel_tools//tools/build_defs/repo:http.bzl' does not have one. Please create a BUILD file in the same or any parent directory. Note that this BUILD file does not need to do anything except exist.
INFO: Elapsed time: 0.298s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
I found one solution for offline building a simple C++ projects with bazel.
First you have to download the rules_cc and rules_java libs, because these where the external dependent libs in the simple Bazel C++ tutorial project. I found no release for rules_cc so I downloaded the zip file from the url which was mentioned in the error message on the console when I was trying to use bazel build offline: https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/b1c40e1de81913a3c40e5948f78719c28152486d.zip. The sha key was also mentioned in the error message when you try to use the lib locally with a wrong sha key (see below). The rules_java lib can be downloaded here: https://github.com/bazelbuild/rules_java/releases. The WORKSPACE specification is also mentioned there.
Then you have to add the following to your WORKSPACE file. Be aware, that you have to point to your local copy of the archive files for rules_cc and rules_java. And for rules_cc you have to mention in strip_prefix the root path of the archive file (the first and only root directory in the .zip file). The rules_java has no root directory:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "rules_cc",
urls = ["file:C:/tmp/b1c40e1de81913a3c40e5948f78719c28152486d.zip"],
strip_prefix = "rules_cc-b1c40e1de81913a3c40e5948f78719c28152486d",
sha256 = "d0c573b94a6ef20ef6ff20154a23d0efcb409fb0e1ff0979cec318dfe42f0cdd",
)
http_archive(
name = "rules_java",
url = "file:C:/tmp/rules_java-4.0.0.tar.gz",
sha256 = "34b41ec683e67253043ab1a3d1e8b7c61e4e8edefbcad485381328c934d072fe",
)
You can run bazel fetch //... to fetch data locally on the machine, which has access to the internet. Then you can just copy prefetched data: from bazel info output_base directory on your local PC to bazel info output_base in the jenkins job.
I have an Elastic Beanstalk application that I'm trying to configure to connect to a FileMaker Pro database, over JDBC. The code I'm using is:
import jaydebeapi as jdp
jdbc_driver_location = '/tmp/fmjdbc.jar'
conn = jdb.connect(jdbc_driver_class,
jdbc_connection_type + '://' + db_url + '/' + db_name,
[user_name, password], jdbc_driver_location,)
When I attempt this, I get the following error:
java.sql.SQLException: No suitable driver found for jdbc:filemaker://10.120.120.108/carecord-<class 'jpype._jexception.java.sql.SQLExceptionPyRaisable'>
To try and solve to problem, I've added the jdbc.jar to both the /tmp folder of the Ec2 instance, as well as included it in the project directory. When if I SSH into the EC2 instance and issue the command:
JAVA_HOME=/tmp/fmjdbc.jar
The program will run the next time it's prompted, without issue. After a few hours it will give the original error and need to be issued the above command again to work. To fix this I tried adding the following to /.ebextensions, to copy the .jar into the tmp folder from the project directory and issue the above command to the server from the start:
commands:
command01:
command: sudo cp /opt/python/current/app/fmjdbc.jar /tmp/fmjdbc.jar
command02:
command: JAVA_HOME=/tmp/fmjdbc.jar
But the project still gives the error. Any thoughts on how I can add this driver to the classpath such that the job will run consistently?
To help folks who have this issue in the future, the answer to this that I found was at the end of this thread.
I appended the following:
if jpype.isJVMStarted() and not jpype.isThreadAttachedToJVM():
jpype.attachThreadToJVM()
jpype.java.lang.Thread.currentThread().setContextClassLoader(jpype.java.lang.ClassLoader.getSystemClassLoader())
Just above the
jdbc_driver_location = '/tmp/fmjdbc.jar'
section of my original code above. This allows the application to loop and successfully find the necessary driver.
JAVA_HOME is supposed to point to the location where Java is installed on the server. You don't use JAVA_HOME to add libraries to the classpath. You shouldn't have to set any environment variables for your code to work.
The root of your problem is that you are copying the file to /tmp/fmjdbc.jar but you are setting jdbc_driver_location to be /tmp/jdbc.jar. Notice how those file names are different. To fix your code change it to this:
jdbc_driver_location = '/tmp/fmjdbc.jar'
I have a SUSE ES 11 SP3 Amazon EC2 instance and I need to install the php5-openssl package, but I'm getting in trouble when using "zypper install". I should have upgraded the insance infrastructure update (more details here), but I missed the deadline and I am now with a useless instance, because I can not install anything .
Whenever I use zypper install I got:
Refreshing service 'susecloud'.
Problem retrieving the repository index file for service 'susecloud':
Download (curl) error for 'http://sa-east-1-ec2-update.susecloud.net/repo/repoindex.xml?cookies=0':
Error code: Connection failed
Error message: Couldn't resolve host 'sa-east-1-ec2-update.susecloud.net'
Check if the URI is valid and accessible.
Error building the cache:
[|] Valid metadata not found at specified URL(s)
Warning: Disabling repository 'php' because of the above error.
Download (curl) error for 'http://sa-east-1-ec2-update.susecloud.net/repo/update/SLE11-SDK-SP3-Pool/sle-11-x86_64/repodata/repomd.xml':
Error code: Connection failed
Error message: Couldn't resolve host 'sa-east-1-ec2-update.susecloud.net'
Abort, retry, ignore? [a/r/i/? shows all options] (a): a
Problem retrieving files from 'SLE11-SDK-SP3-Pool'.
Download (curl) error for 'http://sa-east-1-ec2-update.susecloud.net/repo/update/SLE11-SDK-SP3-Pool/sle-11-x86_64/repodata/repomd.xml':
Error code: Connection failed
Error message: Couldn't resolve host 'sa-east-1-ec2-update.susecloud.net'
Please see the above error message for a hint.
Warning: Disabling repository 'SLE11-SDK-SP3-Pool' because of the above error.
Download (curl) error for 'http://sa-east-1-ec2-update.susecloud.net/repo/update/SLE11-SDK-SP3-Updates/sle-11-x86_64/repodata/repomd.xml':
Error code: Connection failed
Error message: Couldn't resolve host 'sa-east-1-ec2-update.susecloud.net'
If I try updating the update infrascructure I got:
Adding repository 'tmp_instance_infrastructure_upgrade' [done]
Repository 'tmp_instance_infrastructure_upgrade' successfully added
Enabled: Yes
Autorefresh: No
GPG check: Yes
URI: dir:///usr/share/instance_infrastructure_upgrade/repo
Repository 'tmp_instance_infrastructure_upgrade' priority has been set to 1.
Retrieving repository 'tmp_instance_infrastructure_upgrade' metadata [done]
Building repository 'tmp_instance_infrastructure_upgrade' cache [done]
Specified repositories have been refreshed.
Refreshing service 'susecloud'.
Problem retrieving the repository index file for service 'susecloud':
Download (curl) error for 'http://sa-east-1-ec2-update.susecloud.net/repo/repoindex.xml?cookies=0':
Error code: Connection failed
Error message: Couldn't resolve host 'sa-east-1-ec2-update.susecloud.net'
Check if the URI is valid and accessible.
Error building the cache:
[|] Valid metadata not found at specified URL(s)
Warning: Disabling repository 'php' because of the above error.
Download (curl) error for 'http://sa-east-1-ec2-update.susecloud.net/repo/update/SLE11-SDK-SP3-Pool/sle-11-x86_64/repodata/repomd.xml':
Error code: Connection failed
Error message: Couldn't resolve host 'sa-east-1-ec2-update.susecloud.net'
...
--> Updating packages failed
I tried to install each php5-openssl dependency, downloading each RPM and installing with rpm -i, but every time I resolve one dependency another appears. I also tried the suse forum (post here) but with no success until now.
So my questions are:
Is there some way to fix zypper repositories manually? Even if only to install the php5-openssl package.
Is there some way to use RPM to manage the dependencies for php5-openssl?
Is there another alternative to install php5-openssl in my suse instance?
in short:
you cannot fix these repositories; since they don't exist anymore (try browsing to http://sa-east-1-ec2-update.susecloud.net/).
rpm cannot look for dependencies because rpm does not use repositories (that's where zypper comes in). You can look for rpms on the internet, and then install the manually using rpm; but rpm won't locate the missing rpms.
Yes; as said in point 2; you can look for the rpms; download them and install them manually; or else you can even look for repositories that contain the rpms you want, and add that repository. However be careful with that; since those repositories probably aren't made for Amazon EC2. If you do this; try to find a repository as close as possible to suse 11 EC2...
I have an EC2 instance on Amazon Web Services and, in the same VPC there's also an instance of ElastiCache (Redis).
My EC2 instance have installed Amazon Linux AMI 2015.09. I have also configured it to work with mpm_worker_module and not prefork. The configuration is correctly working and I can access pages from my browser.
I have then installed redis client, and the redis-cli command is perfectly working from ssh.
I have then installed phpredis (https://github.com/nicolasff/phpredis/zipball/master -O phpredis.zip) and I have created a simple PHP script to check if it's working properly (a simple index.php).
If I launch this script from ssh console (i.e. php index.php) it works great. If I launch the same script trying to open the page from my browser I get this error:
Fatal error: Class 'Redis' not found
My php.conf file inside apache is the following:
<IfModule prefork.c>
LoadModule php5_module modules/libphp-5.6.so
</IfModule>
<IfModule !prefork.c>
LoadModule php5_module modules/libphp-zts-5.6.so
</IfModule>
In folder /etc/php-zts-5.6.d I have added the file "redis.ini" with inside
extension = redis.so
I have also noticed that redis installation adds redis.so only to this folder
/usr/lib64/php/5.6/modules/
And not to /usr/lib64/php-zts/5.6/modules/ folder, so I copied it from one folder to the other. But it's still not working, with same error:
Fatal error: Class 'Redis' not found
Instead, if I use Predis, with "include" of autoload.php, it works fine. But I'd like to have installed phpredis, cause our system will make thousands of calls each second to Redis Server, and phpredis will be much faster since it's compiled.
What am I doing wrong?
Edit
In my redis error log I get this message
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/5.6/modules/redis.so' - /usr/lib64/php/5.6/modules/redis.so: undefined symbol: compiler_globals in Unknown on line 0
Plus, if I use the prefork module, it works great. If I use the worker, I got the above error.
The solution was doing the following steps to install phpredis:
download and unzip phpredis
command zts-phpize
command ./configure --with-php-config=/usr/bin/zts-php-config
command make && make install
I was doing phpize and not zts-phpize and I was not adding the --with-php-config option: this was the problem
Of course then you have to add redis.ini into /etc/php-zts-5.6.d/ folder with, inside the file:
extension=redis.so
Restart apache and it should work
Hope that will help someone
P.S. If you want it to work from command line (i.e. "php script.php"), you have to install it also with phpize and simple ./configure command.
I am running a flask app on an AWS EC2 server, and have been using boto to access data stored in dynamoDB. After accidentally adding boto.conf to a git commit (and push and pull on the server), I have found that my python code can no longer locate the boto.conf file. I rolled back the changes with git, but the problem remains.
The python module and boto.conf file exist in the same directory, but when the module calls
boto.config.load_credential_file('boto.conf')
I get the flask error IOError: [Errno 2] No such file or directory: 'boto.conf'.
As per Documentation:
I'm not really sure why you are using boto.config_load_credential_file. In general you can pick up the config in a file called either ~/.boto or /etc/boto.cfg.
You can also look at this questions from SO that also answers how to get the configuration for boto: Getting Credentials File in the boto.cfg for Python