postgresql-10 : The program "initdb" is needed by pg_ctl but was not found in the same directory as "pg_ctl" Check your Installation - centos7

I am new to this forum, i am facing this issue, where i get the below error.
sudo -u shahid ./pg_ctl -D /root/pgsql10x/data/ initdb
invalid binary "/root/pgsql10x/bin/pg_ctl"
invalid binary "/root/pgsql10x/bin/pg_ctl"
invalid binary "/root/pgsql10x/bin/pg_ctl"
The program "initdb" is needed by pg_ctl but was not found in the same directory as "pg_ctl".
Check your installation.
I am trying to run this from root.
I have built from the source. I am trying version 10. on CentOS7.
I downloaded direct from postgres site.
I am not facing this problem when I run as non-root user.
I have all the files in the bin directory, as shared in image here:

Finally, I was able to resolve the issue.
The problem was due to the built source was directly placed in "/root/" directory as
/root/pgsql10x/ in root login.
when i placed it in "/app/" directory as /app/pgsql10x/ things started working normally, database got created and database got working without any problem (in root login).
-Shahid

The error comes from the fact that you are trying to start the cluster as the root user you'll have to switch to postgres and try restarting the cluster again
su - postgres
pg_ctl start

Related

how to run a one-off task on cloudfoundry to upload data before starting Python app

Hi this is my first experience trying to deploy a Python app on cloud using CF. I am having issues deploying my app; I sincerely appreciate if anyone can help me or point me to the right direction to solve the issue.
The main problem is the app that I am trying to deploy is large size due to a lot of python dependencies. The size of my app directory is 200 Kb. The first error I observed was: Staging fails due to "Failed to upload payload for droplet" . I think the reason is when all Python dependencies are downloaded from requirements.txt file and finally the droplet is created its size is too large for upload. The droplet size=982. 3 Mb.
The first solution I tried was vendoring app where I created a vendor directory containing all python dependencies but the size of vendor directory was greater that 1Gb, which causes the upload size exceed 1Gb limit and leads to failure in uploading app files.
The second solution I am working on is to upload all installed Python libraries on an object store (in my case S3 bucket which is bounded to my app) and then download the dependencies folder called Pypackages to the app's root directory: /home/vcap/app, so I want to have /home/vcap/app/Pypackages exist before my app starts on the cloud. But I couldn't do it successfully yet. I have included a python script in my app directory which downloads files from S3 bucket successfully. (I have put the correct absolute path for download in downloadS3.py script ie, /home/vcap/app/Pypackages) I want to run this script using "python downloadS3.py" as a one-off task. First I tried the solution here : Can I have multiple commands run in a manifest.yml file?
and although I can see the status of the task is SUCCEED via '$cf tasks my-app-name' , /home/vcap/app/Pypackages does not exist.
I also tried to run one-off task as the steps below:
1-
$ cf push -c 'python downloadS3.py && sleep infinity' -i 1 --no-route
2-
$ cf push -c 'null'
I have printed the contents of /home/vcap/app on my app, ie when app is started and I enter the url in my browser (I don't know what is the right way to see the contents of root directory). Anyway, the problem is Pypackages are not downloaded to the correct root directory. I am not sure if I am running the one-off task in a wrong way or if there is a better solution to make my app work.
I appreciate any helps! (edited)
Diego Cells stage apps and upload droplet to blobstore via cloud controller, the max file can be uploaded is configurable at Ops Manager > TAS for VMs > Application Developer Control > Maximum File Upload Size (MB), default is 1024MB. Seems this is causing restriction, if you can get it increased with your admin help...
Tasks run in their own containers so possibly not an option. I think Python buildpack collects and install the packages before creating the droplet, so don't think copying packages directly to /app directory will be of much help.
If you have data files then you can use .profile file and do some scripting to copy them from S3 or server/NFS location into the /app directory. Something like
wget http://s3.location.com/data_files
cp data_files /home/vcap/app/
But if all these are packages and increasing the size is not feasible then you may need to look to break the app..

Divio App trouble to create project directory and clone repository

Please help I have trouble with Divio App trying to making it work.
When I press "set up project"
it gives me this
*
Creating workspace
cloning project repository
Cloning into '/c/Users/Ubisoft/Documents/iloveit'...
Bad owner or permissions on /home/divio/.ssh/config
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
There was an error trying to run a command. This is most likely
not an issue with divio-cli, but the called program itself.
Try checking the output of the command above.
The command was:
git clone git#git.divio.com:iloveit.git /c/Users/Ubisoft/Documents/iloveit
*
and in windows power shell it gives me this
Creating workspace
cloning project repository
Cloning into '/c/Users/Ubisoft/Documents/iloveit'...
Bad owner or permissions on /home/divio/.ssh/config
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
------------------------------------------------------------------------------------------------------------------------
There was an error trying to run a command. This is most likely
not an issue with divio-cli, but the called program itself.
Try checking the output of the command above.
The command was:
git clone git#git.divio.com:iloveit.git /c/Users/Ubisoft/Documents/iloveit
divio#app-1.0.0 /c/Users/Ubisoft/Documents
$
I also tried this from virtual MacOS and getting this message:
https://i.stack.imgur.com/QccvY.png
I also tried to mess arouond with creating SSH keys but didn't work out.
Can someone provide me step by step explanation how to make this wonderful app work?
In the Windows examples you show, I see:
Bad owner or permissions on /home/divio/.ssh/config
I am not sure how that has happened, but that is what is preventing your local environment from providing the expected key to the Divio Control Panel.
In the Macintosh example, the environment doesn't have a key that the Control Panel knows about.
You will need to add the key (probably from ~/.ssh/rsa_id.pub) to https://control.divio.com/account/ssh-keys/. If you don't already have a key in the Macintosh environment, you will need to set one up.
For anyone who may face this issue. On my MacOS virtual machine I managed to find solution. Looks like it is Internet Service Provider blocking port 22 something like that. Okay looks like problem resolved. I used VPN and without any hastle with SSH I got different result. Looks like it is working now not finished creating project yet but promising:
Creating workspace
cloning project repository
Cloning into '/Users/johnwick/Documents/best-project'...
Locking the website...
remote: Counting objects: 785, done.
remote: Compressing objects: 100% (739/739), done.
Unlocking the website...(385/785), 1.05 MiB | 524.00 KiB/s
remote: Total 785 (delta 112), reused 0 (delta 0)
Receiving objects: 100% (785/785), 1.77 MiB | 448.00 KiB/s, done.
Resolving deltas: 100% (112/112), done.
Checking out files: 100% (615/615), done.
downloading remote docker images
Pulling db ... done
Pulling web ... done
building local docker images
db uses an image, skipping
Building web
Step 1/7 : FROM divio/base:4.15-py3.6-slim-stretch
4.15-py3.6-slim-stretch: Pulling from divio/base

Cannot chmod file on Openshift online v3 : Operation not permitted

I am migrating a Django application from Openshift v2 to v3 (In case you don't know, RedHat is shutting down v2 on September 30th, see: https://blog.openshift.com/migrate-to-v3-v2-eol/)
So, I am following this blog post to help me: https://blog.openshift.com/migrating-django-applications-openshift-3/ . I am new to all these Docker / Kubernetes concepts the new version is build upon.
I was able to make some progress : I managed to get a successful build of my app. Yet it crashes at deployment time:
---> Running application from script (app.sh) ...
/usr/libexec/s2i/run: line 42: /opt/app-root/src/app.sh: Permission denied
Indeed, app.sh has lost its x permission. I log into the failing container as debug and see it:
> oc debug dc/<my app>
> (app-root)sh-4.2$ ls -l /opt/app-root/src/app.sh
-rw-rw-r--. 1 default root 127 Sep 6 21:20 /opt/app-root/src/app.sh
The blog posts states "Ensure that the app.sh file is executable by running chmod +x app.sh.", which I did on my local repo. Whatever, I want to do it again directly in the pod, but it doesn't work:
(app-root)sh-4.2$ chmod +x /opt/app-root/src/app.sh
chmod: changing permissions of ‘/opt/app-root/src/app.sh’: Operation not permitted
So, how can I set the x permission to app.sh ? Thank you
Without looking into more details, any S2I builder image will gladly use your custom supplied run script to start the application in an alternative way.
Create .s2i/bin/ (mind the dot) in your source code directory, place the run script into it and rebuild the app in OpenShift - it will automatically use your custom run script upon deployment.
This is the preferred way of starting applications using custom commands in OpenShift.
Regarding your immediate problem, there is a very simple reason why you can not change the permissions of the script: you were trying to modify the permissions in the deployed pod, and not the builder pod. Deployed pods run using different UIDs, usually somewhere in the range of 100000000, and definitely do not match the file ownership as generated by the build. Hence permission denied.
The root cause of your problem (app.sh losing executable permissions) must be in the way the build process installs those files, and indeed looking at the /usr/libexec/s2i/assemble script in the base image does seem to reveal the culprit. The last two lines are:
# set permissions for any installed artifacts
fix-permissions /opt/app-root
If you wanted to change this part of the build instead of using a custom run script, I suggest you then create .s2i/bin/assemble in your project's source code and make it look sort of like this:
#!/bin/bash
echo "Running stock build:"
${STI_SCRIPTS_PATH}/assemble
echo "Fixing the mess:"
chmod 755 /opt/app-root/src/app.sh
This will fix whatever the stock build process does to file permissions, and will do it using the same UID as the rest of the build, so file ownership shouldn't be an issue.
as I stumbled upon this issue myself I've found a way to resolve it.
You have to make your file app.sh executable and push it in your repo as such.
If git does not track this modification as it did for me, you have to use: git update-index --chmod=+x app.sh for it to work.

Adding JDBC jar driver to classpath for AWS Elastic Beanstalk job

I have an Elastic Beanstalk application that I'm trying to configure to connect to a FileMaker Pro database, over JDBC. The code I'm using is:
import jaydebeapi as jdp
jdbc_driver_location = '/tmp/fmjdbc.jar'
conn = jdb.connect(jdbc_driver_class,
jdbc_connection_type + '://' + db_url + '/' + db_name,
[user_name, password], jdbc_driver_location,)
When I attempt this, I get the following error:
java.sql.SQLException: No suitable driver found for jdbc:filemaker://10.120.120.108/carecord-<class 'jpype._jexception.java.sql.SQLExceptionPyRaisable'>
To try and solve to problem, I've added the jdbc.jar to both the /tmp folder of the Ec2 instance, as well as included it in the project directory. When if I SSH into the EC2 instance and issue the command:
JAVA_HOME=/tmp/fmjdbc.jar
The program will run the next time it's prompted, without issue. After a few hours it will give the original error and need to be issued the above command again to work. To fix this I tried adding the following to /.ebextensions, to copy the .jar into the tmp folder from the project directory and issue the above command to the server from the start:
commands:
command01:
command: sudo cp /opt/python/current/app/fmjdbc.jar /tmp/fmjdbc.jar
command02:
command: JAVA_HOME=/tmp/fmjdbc.jar
But the project still gives the error. Any thoughts on how I can add this driver to the classpath such that the job will run consistently?
To help folks who have this issue in the future, the answer to this that I found was at the end of this thread.
I appended the following:
if jpype.isJVMStarted() and not jpype.isThreadAttachedToJVM():
jpype.attachThreadToJVM()
jpype.java.lang.Thread.currentThread().setContextClassLoader(jpype.java.lang.ClassLoader.getSystemClassLoader())
Just above the
jdbc_driver_location = '/tmp/fmjdbc.jar'
section of my original code above. This allows the application to loop and successfully find the necessary driver.
JAVA_HOME is supposed to point to the location where Java is installed on the server. You don't use JAVA_HOME to add libraries to the classpath. You shouldn't have to set any environment variables for your code to work.
The root of your problem is that you are copying the file to /tmp/fmjdbc.jar but you are setting jdbc_driver_location to be /tmp/jdbc.jar. Notice how those file names are different. To fix your code change it to this:
jdbc_driver_location = '/tmp/fmjdbc.jar'

Neo4j on Heroku with Python

I'm trying to get the Neo4j on Heroku and Getting Started with Python on Heroku tutorials up and going. The Neo4j one works fine, but the Python one has problems.
For anyone else trying to follow this tutorial, I've recorded the
problems and my solutions to help you out as well.
This is all done on a Win7 x64 dev machine.
Q1) "virtualenv venv --distribute" - errors with:
'virtualenv' is not recognized as an internal or external command, operable program or batch file.
A1) The workaround is to fully qualify the path to:
"C:\Python27\Scripts\virtualenv venv --distribute"
Q2a) "foreman start" - errors with:
'foreman' is not recognized as an internal or external command, operable program or batch file.
A2) Looks like a path issue so I ran the line:
"set PATH=%PATH%;C:\Program Files (x86)\Heroku\ruby-1.9.2\bin\"
Q2b) "foreman start" now errors:
Bad file descriptor
{Ruby paths...}
A2b) Help?
So I can't run the app locally, but maybe still on the server, so moving on...
Q3) .gitignore - can't create this file on Windows.
A3) Clone another project and copy that file and edit.
Q4) "git push heroku master" - errors with:
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
A4) Apparently need to create new SSH keys. Managing Your SSH Keys Note again fully quality the path such as below and select the new key to add to heroku.
"c:\Program Files (x86)\Git\bin\ssh-keygen.exe" -t rsa"
Q5) Try "git push heroku master" the same with the Neo4j test app "flask-py2neo" - errors during the compile. Is this example current?
A5) Remove distribute from requirements.txt.
Any ideas?