Adding JDBC jar driver to classpath for AWS Elastic Beanstalk job - amazon-web-services

I have an Elastic Beanstalk application that I'm trying to configure to connect to a FileMaker Pro database, over JDBC. The code I'm using is:
import jaydebeapi as jdp
jdbc_driver_location = '/tmp/fmjdbc.jar'
conn = jdb.connect(jdbc_driver_class,
jdbc_connection_type + '://' + db_url + '/' + db_name,
[user_name, password], jdbc_driver_location,)
When I attempt this, I get the following error:
java.sql.SQLException: No suitable driver found for jdbc:filemaker://10.120.120.108/carecord-<class 'jpype._jexception.java.sql.SQLExceptionPyRaisable'>
To try and solve to problem, I've added the jdbc.jar to both the /tmp folder of the Ec2 instance, as well as included it in the project directory. When if I SSH into the EC2 instance and issue the command:
JAVA_HOME=/tmp/fmjdbc.jar
The program will run the next time it's prompted, without issue. After a few hours it will give the original error and need to be issued the above command again to work. To fix this I tried adding the following to /.ebextensions, to copy the .jar into the tmp folder from the project directory and issue the above command to the server from the start:
commands:
command01:
command: sudo cp /opt/python/current/app/fmjdbc.jar /tmp/fmjdbc.jar
command02:
command: JAVA_HOME=/tmp/fmjdbc.jar
But the project still gives the error. Any thoughts on how I can add this driver to the classpath such that the job will run consistently?

To help folks who have this issue in the future, the answer to this that I found was at the end of this thread.
I appended the following:
if jpype.isJVMStarted() and not jpype.isThreadAttachedToJVM():
jpype.attachThreadToJVM()
jpype.java.lang.Thread.currentThread().setContextClassLoader(jpype.java.lang.ClassLoader.getSystemClassLoader())
Just above the
jdbc_driver_location = '/tmp/fmjdbc.jar'
section of my original code above. This allows the application to loop and successfully find the necessary driver.

JAVA_HOME is supposed to point to the location where Java is installed on the server. You don't use JAVA_HOME to add libraries to the classpath. You shouldn't have to set any environment variables for your code to work.
The root of your problem is that you are copying the file to /tmp/fmjdbc.jar but you are setting jdbc_driver_location to be /tmp/jdbc.jar. Notice how those file names are different. To fix your code change it to this:
jdbc_driver_location = '/tmp/fmjdbc.jar'

Related

Unable to execute a step on a running EMR

I have an EMR cluster 5.28.1 running in AWS but I forgot to install from python libraries as part of the bootstrap action. Now that the cluster is running, I was simply attempting to add a step via the EMR console. Here are my settings
JAR: s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar
Main class: None
Arguments: s3://xxxx/install_python_libraries.sh
Unfortunately, I get the following error.
Cannot run program "s3://xxxxx/install_python_libraries.sh" (in directory "."): error=2, No such file or directory
I am not sure what I am doing wrong. The shell script looks like this.
#!/bin/bash -xe
# Non-standard and non-Amazon Machine Image Python modules:
sudo pip-3.6 install boto3
sudo pip-3.6 install xmltodict
I also tried this by simply using 'command-runner.jar' but I get the same error. Can you please help me figure out the problem so I do this via the console? I would like to install the libraries on all nodes - master and core.
Thanks
The issue is the xxx.sh files EOL/carriage return type.
In other words, if it is Windows ("\r\n") then it will not work and return the ./ file not found error.
Convert it to unix type ("\n") using something like notepad++ and it will run fine.
(In notepad++ edit>EOL Conversion>Unix(LF) hit save and try again)

Elastic Beanstalk .ebextensions ignored (Windows)

This is Driving me nuts.
I had a working .ebextensions config file in my Project which was working fine.
Recently my single instance failed and a new one got initiated. My configuration failed to run so i tried to troubleshoot what went wrong. I didn't find anything suspicious so i just created a new .config with a very simple command but it still fails!!
I validated my config file with an online yaml validator.
I Connected to the instance through remote desktop and saw that .ebextensions folder is actually created within the wwwroot and then it disappears meaning that it got successfully picked up by elastic beanstalk.
I also granted all permissions to everyone on the test folder just to make sure this is not the reason.
Whichever i tried the old configuration or this test command it just does not work and elastic beanstalk just ignores it!
Any info of what might be wrong is appreciated.
commands:
01_Dowork:
command: mkdir kakarot
cwd: c:\\testdir
waitForCompletion: 0
I think everything under 01_DoWork needs to be indented (command, cwd, waitForCompletion). Also, make sure you're using spaces and not tabs.
Check the properties on your config file in VS. It should be (I think) both 'Content' and 'Copy if Newer'. Also, make sure that it gets packaged into the msdeploy package. It's a .zip file in/below your obj directory.
The command will error-out of it's already succeeded, so you would want to either ignore errors or add this. I found this syntax on another SO post but don't know who to credit for it :-/. The errorlevel will cause your command to not run if the directory already exists.
test: test ! -d c:\\testdir\\kakarot
If you're creating a package.zip (that inside has a deploy manifest json file plus the actual site.zip content) for a Windows deployment, it appears the .ebextensions directory needs to be inside package.zip, alongside the manifest json, not inside the site.zip, contrary to the current documentation.

Elastic Beanstalk ebextension config files using VS toolkit

Im trying to set Folder Permissions for creating additional folders/files inside Temp directory in my .Net Project.
I know there a lot of references for a similar question like this (as given)
Running a .config file on Elastic Beanstalk?
How To Set Folder Permissions in Elastic Beanstalk Using YAML File?
Im having trouble verifying if its given the permission, im not able to create any folders/files in the Temp folder; i cant find any errors either during deployment in the Elastic Beanstalk (last 100 lines) related to permission setting.
Im using the following code in my config file
command: icacls \"C:/inetpub/wwwroot/myapp_deploy/Temp\" /grant DefaultAppPool:(OI)(CI)
(i have replaced myapp with the EB application name)
Please help anyone.
i was having trouble posting the code and a general unknown exception error was being generated with no additional details.
Figured out the mistake i was making:
ICACLS command requires the directory path to have double slash '\'.
Verified command for above would be:
command: "icacls \"C:\inetpub\wwwroot\Temp\" /grant DefaultAppPool:(OI)(CI)F"

redis.so module is not loaded

I have an EC2 instance on Amazon Web Services and, in the same VPC there's also an instance of ElastiCache (Redis).
My EC2 instance have installed Amazon Linux AMI 2015.09. I have also configured it to work with mpm_worker_module and not prefork. The configuration is correctly working and I can access pages from my browser.
I have then installed redis client, and the redis-cli command is perfectly working from ssh.
I have then installed phpredis (https://github.com/nicolasff/phpredis/zipball/master -O phpredis.zip) and I have created a simple PHP script to check if it's working properly (a simple index.php).
If I launch this script from ssh console (i.e. php index.php) it works great. If I launch the same script trying to open the page from my browser I get this error:
Fatal error: Class 'Redis' not found
My php.conf file inside apache is the following:
<IfModule prefork.c>
LoadModule php5_module modules/libphp-5.6.so
</IfModule>
<IfModule !prefork.c>
LoadModule php5_module modules/libphp-zts-5.6.so
</IfModule>
In folder /etc/php-zts-5.6.d I have added the file "redis.ini" with inside
extension = redis.so
I have also noticed that redis installation adds redis.so only to this folder
/usr/lib64/php/5.6/modules/
And not to /usr/lib64/php-zts/5.6/modules/ folder, so I copied it from one folder to the other. But it's still not working, with same error:
Fatal error: Class 'Redis' not found
Instead, if I use Predis, with "include" of autoload.php, it works fine. But I'd like to have installed phpredis, cause our system will make thousands of calls each second to Redis Server, and phpredis will be much faster since it's compiled.
What am I doing wrong?
Edit
In my redis error log I get this message
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/5.6/modules/redis.so' - /usr/lib64/php/5.6/modules/redis.so: undefined symbol: compiler_globals in Unknown on line 0
Plus, if I use the prefork module, it works great. If I use the worker, I got the above error.
The solution was doing the following steps to install phpredis:
download and unzip phpredis
command zts-phpize
command ./configure --with-php-config=/usr/bin/zts-php-config
command make && make install
I was doing phpize and not zts-phpize and I was not adding the --with-php-config option: this was the problem
Of course then you have to add redis.ini into /etc/php-zts-5.6.d/ folder with, inside the file:
extension=redis.so
Restart apache and it should work
Hope that will help someone
P.S. If you want it to work from command line (i.e. "php script.php"), you have to install it also with phpize and simple ./configure command.

Elastic Beanstalk and Cron tasks not sure if its working with Yii

I am having a problem with getting a simple cron task set up on Elastic Beanstalk. I have found some of the other questions on here useful, but i still can't seem to get the cron to execute. I am unsure if it is an AWS issue, or if the script itself is not executing. The script is set up inside YII as a Console Command. I am not finding any PHP errors, and the ec2 instance is loaded without errors. Here is what i have done so far:
I have created a folder on the root of my application called .ebextensions.
Within that folder i have created a configuration file with the contents
# Installing dos2unix in case files are edited on windows PC
packages:
yum:
dos2unix: []
container_commands:
01-command:
command: dos2unix -k cron_setup.sh
02-command:
command: chmod 700 cron_setup.sh
03-command:
command: "cat .ebextensions/cron_task.txt > /etc/cron.d/cron_task && chmod 644 /etc/cron.d/cron_task"
# leader_only prevents problems when EB auto-scales
leader_only: true
the file cron_task.txt exists inside the .ebextensions folder with the contents
# The newline at the end of this file is extremely important. Cron won't run without it.
* * * * * /bin/php /var/www/html/crons.php test > /dev/null
Crons.php is a file at the root of the application that includes the Yii framework
defined('YII_DEBUG') or define('YII_DEBUG',true);
// including Yii
require_once(dirname(__FILE__).$yii.'/yii.php');
// we'll use a separate config file
$configFile=dirname(__FILE__).'/protected/config/cron.php';
// creating and running console application
Yii::createConsoleApplication($configFile)->run();
the config/cron.php file is a setup file for the framework, includes database connection and model inclusions, etc
and the cron script being referenced in the cron_task.txt file is a console command that looks like this
class TestCommand extends CConsoleCommand {
public function run($args) {
$message = new Crontasks();
$message->timestamp = date("Y-m-d H:i:s");
$message->message = "test";
$message->insert();
}
}
here i am just trying to get a record into the database to prove the cron was executed successfully. And i can't seem to get a record added.
The problem is, i don't know where this is failing. I am not getting any instance errors. And i took a snapshot log and cant seem to find any relevant errors in there either. Should php errors be logged here? OR do i have to set it up myself to log errors? The problem, i am also having trouble getting into ec2 via SSH. I am getting a permission denied (public key) error!! Even though i have set up the security group/key pair and using the correct public DNS for the instance!
If anyone can see anything obvious is what im doing wrong here, please let me know! Otherwise could you give any advice on where to look for any errors that might be preventing this cron task to execute? Many thanks!!