Bitnami Redmine Windows Installer Upgrade to 3.3.1 - redmine

Most of the documentation examples are for Linux:
https://docs.bitnami.com/installer/apps/redmine/#how-to-upgrade-redmine
I would like to see one for the Windows Server variety. Trying to upgrade 3.2.2 to 3.3.1. Client wants to keep it on local Windows only. No cloud.

Bitnami Developer here. Thanks for your comment, we will update soon the documentation of bitnami to add more guides of windows.
I have been able to migrate to redmine 3.2.2 to 3.3.1, these are the steps you have to follow:
Go to the manager-windows (C:\Bitnami\redmine-3.2.2-0\manager-windows.exe) and stop all the services. Then start again mysql. You should have something like this:
manager-windows
Do a dump of your mysql database. You can use the use-redmine console to do this (C:\Bitnami\redmine-3.2.2-0\use_redmine.exe) and execute the following:
mysqldump -u root -p --databases bitnami_redmine > backup.sql
Save that backup and download the last version of redmine stack installer (3.3.1-0): Bitnami redmine installers
Install it in your machine and open the manager-windows (C:\Bitnami\redmine-3.3.1-0\manager-windows.exe). Stop all services and start again the mysql service to restore the backup.
Start the use_redmine console(C:\Bitnami\redmine-3.3.1-0\use_redmine.exe)
Execute the following in the use_redmine console:
mysql -u root -p
Password: ****
mysql> drop database bitnami_redmine;
mysql> create database bitnami_redmine;
mysql> grant all privileges on bitnami_redmine.* to 'bn_redmine'#'localhost' identified by 'DATABASE_PASSWORD';
Restore the new database:
`mysql -u root -p bitnami_redmine < /path/to/your/backup.sql`
Edit the Redmine configuration file to update the database user password (the same that you set previously) at
C:\Bitnami\redmine-3.3.1-0\apps\redmine\htdocs\config\database.yml:
production:
adapter: mysql2
database: bitnami_redmine
host: localhost
username: bn_redmine
password: "DATABASE_PASSWORD"
encoding: utf8
In the use_redmine console migrate the database to the latest version:
bundle exec rake db:migrate RAILS_ENV=production
After this, you should be able to start all the services again in the C:\Bitnami\redmine-3.3.1-0\manager-windows.exe and log in in the application as always.

Related

How to connect apache superset directly with google BigQuery?

I am running Apache superset on GCP instance and it works fine with Sqlite database which is default in superset and I don't need to configure so many things. But my requirement is that I need superset to connect directly with BigQuery instead of Sqlite and I don't have developer background. So, is there an easy way to do that without heavy codes?
Connecting to BigQuery is very well documented here in Preset's Superset user documentation https://docs.preset.io/docs/big-query-database
Following the steps mentioned at the official Google Cloud page here, you need to do the following
Install pybigquery
pip install pybigquery
Download your Google Cloud authorization json key file
From your terminal instance, set GOOGLE_APPLICATION_CREDENTIALS env. var to the path of your json key file
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/[json_file].json"
Elbehery is right. I don't have enough rep to comment, but I wanted to note that Apache has created docs for this.
FWIW, I couldn't use the UI for importing credentials.json so I set it as an env var in my Docker image. Here are the commands and steps I run locally:
# Setup virtual environment (exit by typing "deactivate")
pip3 install virtualenv
python3 -m virtualenv ./.venv
source ./.venv/bin/activate
​
# Download Superset
git clone https://github.com/apache/superset.git
cd superset/
​
# Create a copy of your credentials for docker to use
cp ~/.config/gcloud/application_default_credentials.json docker/credentials.json
echo "GOOGLE_APPLICATION_CREDENTIALS=docker/credentials.json" >> docker/.env-non-dev
​
# Run Superset
docker-compose -f docker-compose-non-dev.yml pull
docker-compose -f docker-compose-non-dev.yml up
​
Now that Superset is running locally:
visit http://0.0.0.0:8088/ in your web browser
In the top right of UI, click the +DATABASE button (or + > Data > Connect Database)
In popup window Click Supported Databases then (at the bottom) Other
Set DISPLAY NAME: BigQuery
Set SQLALCHEMY URI: bigquery://my_project_id
Click Test Connection
Click Connect
​
Now that BigQuery is integrated into Superset:
In the top of page Click SQL Lab > SQL Editor

Redmine: fetching emails from POP3 under Windows

I have Redmine running locally under Windows 10.
Environment:
Redmine version 3.4.6.stable
Ruby version 2.4.4-p296 (2018-03-28) [x64-mingw32]
Rails version 4.2.8
Environment production
Database adapter SQLite
SCM:
Git 2.16.2
Filesystem
Redmine plugins:
redmine_issues_tree 0.0.12
I try to fetch emails from a POP3 account. I use the following command:
rake redmine:email:receive_pop3 host=xxx.xxx.com port=995 ssl=1 username=uuu#yyy.com password=zzz RAILS_ENV=production unknown_user=accept no_permission_check=1 project=MyProject allow_override=project,tracker,status,priority move_on_success=read move_on_failure=failed --trace
The content of the email I have sent to the above mailbox is:
Project: MyProject
Tracker: Bug
Status: New
Priority: Normal
Executing the rake command as shown above I get:
** Invoke redmine:email:receive_pop3 (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute redmine:email:receive_pop3
C:\redmine-3.4.6>
The connection to the mailbox seem to work. But there is nothing happening in Redmine. No issue is added, nor is there any other changes. What could I try to find the problem?
My second question concerns the
move_on_success=read
and
move_on_failure=failed
options. Are these options only for IMAP accounts or do they work for POP3 as well? If they work for POP3 in which mailboxes or folders are the emails moved?

InfluxDB Cannot see databases from localhost:8083 + Cannot access Command Line Interface

Please feel free to redirect me to any other place if this isn't the right one for this question.
Problem: When I log to the administration panel : "localhost:8083" with "root" "root" I cannot see the existing databases nor the data in it. Also, I have no way to access InfluxDB from the command line.
Also the line sudo /etc/init.d/influxdb start does not work for my setup. I have to go into /etc/init.d/ and run sudo ./influxdb start -config=config.toml in order to get the server running.
I've installed influxDB v0.8 from https://influxdb.com/docs/v0.8/introduction/installation.html for Ubuntu 14.04.
I've been developing a Clojure program using the Capacitor API just to get started and interact with InfluxDB. It runs well, I can create delete, insert and query a database without problems.
netstat -anp | grep LISTEN confirms me that ports 8083 8086 8090 and 8099 are listening.
I've been Googling all around but cannot manage to get a solution.
Thanks for the support and enjoy building things !
Problem solved: the database weren't visible in firefox but everything is visible in Chromium!
Why couldn't I access the CLI ? I was expecting the v0.8 to behave exactly like the v0.9.
You help was appreciated anyway !
For InfluxDB 0.9 the CLI could be started with:
/opt/influxdb/influx
then you can display available databases:
Connected to http://localhost:8086 version 0.9.1
InfluxDB shell 0.9.1
> show databases
name: databases
---------------
name
collectd
graphite
> use collectd
Using database collectd
> show series limit 5
You can try creating new database from CLI:
> CREATE DATABASE mydb
or with curl command:
curl -G 'http://localhost:8086/query' --data-urlencode "q=CREATE DATABASE mydb"
Web UI should be available on http://localhost:8083

How can i install puppet cluster on Amazon EC2 instances?

I'm using ubuntu 12.04 AMI in EC2 for creating puppet cluster and i'm facing problems while configuring it.
The problem is that the master is not able to recognize the slaves.
Do i need more packages other than mysql
/etc/mysql/my.cnf
what changes do i need in the above file?
Puppet is a configuration management tool that allows automating the process of defining and maintaining consistent state of several developer workstations. It is a descriptive, centralized and client-server based system. The central server is configured and the clients synchronize themselves to it to ensure that all systems end in the described state. For instance, the task of ensuring the same development environment on all developer systems in a project can be easily accomplished using Puppet.
Here is a quick procedure to set up a Puppet server and one Puppet client on Amazon EC2 instance having Ubuntu OS, and also installing Puppet Dashboard on server to view the status of the clients.
Prerequisites
Two ec2 instances set up with Ubuntu ami.
One instance named as puppetserver and other as puppetclient.
Procedure
Puppet server and client set up
Configuring hosts files View the /etc/hostname file on puppetserver and puppetclient. These are the Puppet server and client hostnames respectively
Edit /etc/hosts file on both the systems. Add server and client IPs and corresponding hostnames.
Setting up the Puppet Server
Enabling the Puppet Labs Package repository
Download the "puppetlabs-release" package for the OS (here, Ubuntu 12.04) on Puppet server
Install the package by running
dpkg -i
Run apt-get update to get new list of available packages.
For example, to enable the repository for Ubuntu 12.04, Precise Pangolin:
wget https://apt.puppetlabs.com/puppetlabs-release-precise.deb
sudo dpkg -i puppetlabs-release-precise.deb
sudo apt-get updateInstall Puppet
Install Puppet
Install puppetmaster
sudo apt-get update sudo apt-get install puppetmaster
Setting up the Puppet Client
Install Puppet on the puppet client(s)
sudo apt-get update sudo apt-get install puppet
Specify the Puppet server domain name on the client. To do this, modify the
/etc/puppet/puppet.conf
file and add the line
server=.
The client can now connect to the Puppet master.
Start the Puppet agent service for establishing first communication between server and client.
sudo puppet agent --verbose --no-daemonize --onetime
This starts a connection to the Puppet master process that is listening on port 8140 on the Puppet server. The output will be verbose, and the agent will not continue running in the background as a daemon. Also, it will run only one time, that is, after the connection is closed, the agent process will exit. The output looks like:
The client has made itself known to the server by sending an SSL certificate request. The server needs to certify the client.
To view the list of yet-to-be signed certificates on the server
sudo puppet cert --list
This lists the following
Sign the client node's SSL certificate
sudo puppet cert --sign <puppet client name>
Client can now establish full connection to the server and poll the Puppet master for any configuration updations.
Defining Configurations
We have set up puppet on both Puppet server and client and have also established communication between the two machines. Next step is to define the configuration for the target systems using puppet manifest. These manifests are specified in site.pp file.
As an example, we define a manifest that will create a helloworld.txt file on the client.
Defining manifest
Put the following manifest definition in /etc/puppet/manifests/site.pp file,
node "<puppet client hostname>" { file { "/home/ubuntu/helloworld.txt": content => "This is test content", ensure => file, owner => "ubuntu", group => "ubuntu", mode => 0644 } }
This manifest defines that the puppet client must have a helloworld.txt file
in /home/ubuntu/ folder with content, This is test content.
Getting changes on client
On puppet client, run the following command.
sudo puppet agent -t
The puppet client pulls the manifests defined in the site.pp file on the puppet server. It learned that a file named helloworld.txt with defined specifications, is expected to exist at location /home/ubuntu. Since, no such file exists on the client, the agent takes action and creates the file.
View the 'helloworld.txt' file
To verify that the client exists in a state defined by the Puppet server, run the following command
sudo vi /home/ubuntu/helloworld.txt
The file contents are same as defined in the manifest definition on the server.
Installing Puppet Dashboard
Overview
Puppet Dashboard is a GUI that interfaces with Puppet. It can be used to view and report the status of all the client nodes. Puppet dashboard runs on port 3000 on the puppet server.
Following are the steps for set up
Installing external dependencies
Dashboard is a Ruby on Rails web app and thus requires certain software to be installed
RubyGems
Rake version 0.8.3 or newer
MySQL database server version 5.x
Ruby-MySQL bindings version 2.7.x or 2.8.x
Install the packages
sudo apt-get install -y build-essential irb libmysql-ruby libmysqlclient-dev libopenssl-ruby libreadline-ruby mysql-server rake rdoc ri ruby ruby-dev
Install RubyGems package system
( URL="http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz" PACKAGE=$(echo $URL | sed "s/\.[^\.]*$//; s/^.*\///") cd $(mktemp -d /tmp/install_rubygems.XXXXXXXXXX) && \ wget -c -t10 -T20 -q $URL && \ tar xfz $PACKAGE.tgz && \ cd $PACKAGE && \ sudo ruby setup.rb )
Create gem as an alternative name for gem1.8
sudo update-alternatives --install /usr/bin/gem gem /usr/bin/gem1.8 1
Installing Puppet Dashboard
Install puppet-dashboard from puppetlabs package repository
sudo apt-get update sudo apt-get install puppet-dashboard
Configuring Dashboard
Modify the database.yml file. It can be found at /usr/share/puppet-dashboard/config/database.yml.
Under the key-value pairs for production environment, the database value 'dashboard_production' specifies the dashboard database name, and username value 'dashboard' specifies the user for this database. In the next step, we will create both the database and the user. password value is the password for MySQL.
Creating and Configuring MySQL database
Create the user and database for puppet-dashboard. Navigate to MySQL command line
CREATE DATABASE dashboard_production CHARACTER SET utf8; CREATE USER 'dashboard'#'localhost' IDENTIFIED BY 'my_password'; GRANT ALL PRIVILEGES ON dashboard_production.* TO 'dashboard'#'localhost';
Configure MySQL's maximum packet size to permit larger rows in database
set global max_allowed_packet = 33554432;
Also modify the mysql configuration file /etc/mysql/my.cnf
Allowing 32MB allows an occasional 17MB row with plenty of spare room
max_allowed_packet = 32M
To create dashboard tables, run the following command in the puppet-dashboard folder
cd /usr/share/puppet-dashboard rake RAILS_ENV=production db:migrate
Testing that Dashboard is working
Start the dashboard using Ruby’s built-in WEBrick server
cd /usr/share/puppet-dashboard
sudo ./script/server -e production
Dashboard instance starts on port 3000 using the “production” environment. Dashboard’s UI can be viewed at :3000
Configure puppet
Both the puppet server and client need to be configured for the dashboard to receive reports.
Configure agent nodes to submit reports to master by turning their reporting ON.
puppet.conf (on each agent)
[agent]
report = true
Configure the server. Add the http report handler to puppet server's reports setting and set reporturl to Dashboard instance’s reports/upload URL
puppet.conf (on puppet master)
[master]
reports = store, http
reporturl = http://<server hostname>:3000/reports/upload
For enabling dashboard's external node classifier(ENC),
puppet.conf (on puppet master)
[master]
node_terminus = exec
external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=http://<server hostname>:3000 /usr/share/puppet-dashboard/bin/external_node
Testing Puppet's connection to Dashboard
Restart the puppet master
Run one of the puppet agents to test the configurations
sudo puppet agent -t
The output will be:
This means that the report has arrived. To process it, we will activate the delayed_job workers.
Starting delayed_job workers
Run the following command
cd /usr/share/puppet-dashboard
sudo env RAILS_ENV=production script/delayed_job -p dashboard -n 1 -m start
This starts the delayed_job workers, and completes the pending task.
Thus, puppet is now installed on two EC2 instances, out of which one is server and the other is client. Also, puppet-dashboard is installed to view the status of the client nodes.

Where is data_directory for homebrew? or for postgresql or how do I solve error, ""global/pg_filenode.map": No such file or directory"?

I'm having difficulty with postgresql after installing via homebrew. I'm on Mac OS X 10.9.5 (just updated today and did restart).
Note I didn't deliberately change anything from a default postgresql installation via homebrew
PROBLEM
I can't uninstall postgresql
I can't log in to view tables / start the "command line" of postgres
createdb returns an error about pg_filenode.map "no such file or directory"
initdb returns error wondering where data_directory is (I don't know)
ANY OF THESE WOULD BE SUCCESS
Log in to psql (so I can create a database or table like I can do in mysql, browse existing tables)
Uninstall postgres altogether (so I can re-try with postgresql.app)
I've spent almost 10 hours troubleshooting this and come up short.
I've looked at loads of answers already (including SO questions 24132418 and 14510237) though I can't include all links due to not having reputation points yet
Any pointers would be really appreciated!
Below seems lengthy because I have pasted all the logs, please let me know if I have missed some important info though
Thanks, -Mark
DETAILS
I have been working my way through an introduction to Heroku, and they said installing postgresql was a requirment, so I did that using homebrew
brew install postgresql
I didn't check it, I proceeded building my app, but the next day I run the webapp locally and get error,
Request URL: http://localhost:5000/admin/
Django Version: 1.6.5
Exception Type: OperationalError
Exception Value:
FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
Exception Location: /Users/macuser/Dropbox/code/heroku/awe01/lib/python2.7/site-packages/psycopg2/__init__.py in connect, line 164
So I want to investigate.
Now I've never used postgresql before, so I want to "get to the command line" for postgresql
Please note, I'm a complete newbie and have never used postgresql before.
In mysql I know this would be
mysql -u root -p
(and then enter password)
I'm unable to do the equivalent in postgresql
moriartymacbookair13:~ macuser$ postgres
postgres does not know where to find the server configuration file.
You must specify the --config-file or -D invocation option or set the PGDATA environment variable.
moriartymacbookair13:~ macuser$ postgres -V
postgres (PostgreSQL) 9.3.5
The more detailed heroku docs recommend installing postgresql using postgres.app
I hadn't done this, I used homebrew, so I tried uninstalling postgresql
moriartymacbookair13:~ macuser$ brew uninstall postgres
Uninstalling /usr/local/Cellar/postgresql/9.3.5_1...
Error: Permission denied - /usr/local/bin/clusterdb
That didn't work, so back to where I was. To figure out what path to specify, I ran
brew info postgresql
If you scroll down to the bottom of this, you can see it advises me of the path
moriartymacbookair13:~ macuser$ brew info postgres
postgresql: stable 9.3.5 (bottled), devel 9.4beta2
http://www.postgresql.org/
Conflicts with: postgres-xc
/usr/local/Cellar/postgresql/9.3.5_1 (2927 files, 38M) *
Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/postgresql.rb
==> Dependencies
Required: openssl ✔, readline ✔
Recommended: ossp-uuid ✔
==> Options
--32-bit
Build 32-bit only
--enable-dtrace
Build with DTrace support
--no-perl
Build without Perl support
--no-tcl
Build without Tcl support
--with-python
Build with python support
--without-ossp-uuid
Build without ossp-uuid support
--devel
install development version 9.4beta2
==> Caveats
If builds of PostgreSQL 9 are failing and you have version 8.x installed,
you may need to remove the previous version first. See:
https://github.com/Homebrew/homebrew/issues/issue/2510
To migrate existing data from a previous major version (pre-9.3) of PostgreSQL, see:
http://www.postgresql.org/docs/9.3/static/upgrading.html
When installing the postgres gem, including ARCHFLAGS is recommended:
ARCHFLAGS="-arch x86_64" gem install pg
To install gems without sudo, see the Homebrew wiki.
To reload postgresql after an upgrade:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Or, if you don't want/need launchctl, you can just run:
postgres -D /usr/local/var/postgres
Note: I have ignored the comment at the top:
Conflicts with: postgres-xc
/usr/local/Cellar/postgresql/9.3.5_1 (2927 files, 38M) *
Anyway there we have it, toward the end of that large output it confirms the path for me is /usr/local/var/postgres, so I tried -D
moriartymacbookair13:~ macuser$ postgres -D /usr/local/postgres
postgres cannot access the server configuration file "/usr/local/postgres/postgresql.conf": No such file or directory
So this time I specified the config file:
moriartymacbookair13:~ macuser$ postgres --config-file=/usr/local/var/postgres/postgresql.conf
postgres does not know where to find the database system data.
This can be specified as "data_directory" in "/usr/local/var/postgres/postgresql.conf", or by the -D invocation option, or by the PGDATA environment variable.
(Out of desperation, I even tried deleting it and going again)
moriartymacbookair13:~ macuser$ initdb /usr/local/var/postgres
The files belonging to this database system will be owned by user "macuser".
This user must also own the server process.
The database cluster will be initialized with locale "en_GB.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
initdb: directory "/usr/local/var/postgres" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "/usr/local/var/postgres" or run initdb
with an argument other than "/usr/local/var/postgres".
moriartymacbookair13:~ macuser$ rm -rf /usr/local/var/postgres
moriartymacbookair13:~ macuser$ initdb /usr/local/var/postgres -E utf8
The files belonging to this database system will be owned by user "macuser".
This user must also own the server process.
The database cluster will be initialized with locale "en_GB.UTF-8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
creating configuration files ... ok
creating template1 database in /usr/local/var/postgres/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
postgres -D /usr/local/var/postgres
or
pg_ctl -D /usr/local/var/postgres -l logfile start
moriartymacbookair13:~ macuser$ postgres -D /usr/local/postgres
postgres cannot access the server configuration file "/usr/local/postgres/postgresql.conf": No such file or directory
moriartymacbookair13:~ macuser$ postgres --config-file=/usr/local/var/postgres/postgresql.conf
postgres does not know where to find the database system data.
This can be specified as "data_directory" in "/usr/local/var/postgres/postgresql.conf", or by the -D invocation option, or by the PGDATA environment variable.
So I took a look at that file:
In that in the conf file itself, the data variable is not defined
moriartymacbookair13:~ macuser$ ls /usr/local/var/postgres/
PG_VERSION pg_ident.conf pg_stat/ pg_xlog/
base/ pg_multixact/ pg_stat_tmp/ postgresql.conf
global/ pg_notify/ pg_subtrans/
pg_clog/ pg_serial/ pg_tblspc/
pg_hba.conf pg_snapshots/ pg_twophase/
moriartymacbookair13:~ macuser$ vim /usr/local/var/postgres/postgresql.conf
shows that the value is commented out (see line 41... almost the whole file is commented out)
35 # FILE LOCATIONS
36 #----------------------------------------------------------------------------- -
37
38 # The default values of these variables are driven from the -D command-line
39 # option or PGDATA environment variable, represented here as ConfigDir.
40
41 #data_directory = 'ConfigDir' # use data in another directory
42 # (change requires restart)
43 #hba_file = 'ConfigDir/pg_hba.conf' # host-based authentication file
44 # (change requires restart)
45 #ident_file = 'ConfigDir/pg_ident.conf' # ident configuration file
46 # (change requires restart)
47
48 # If external_pid_file is not explicitly set, no extra PID file is written.
49 #external_pid_file = '' # write an extra PID file
50 # (change requires restart)
HELP: DOES ANYONE KNOW WHAT I SHOULD SPECIFY THAT 'ConfigDir' AS for where data_directory is?
In case helpful
here are contents of the Cellar folder
moriartymacbookair13:~ macuser$ ls /usr/local/Cellar/postgresql/9.3.5_1/
COPYRIGHT homebrew.mxcl.postgresql.plist
HISTORY include/
INSTALL_RECEIPT.json lib/
README share/
bin/
(I don't understand why I have some postgres files in Cellar and some in usr/local/var)
I tried some other things too:
moriartymacbookair13:~ macuser$ sudo -u postgres pg_ctl -D /usr/local/var/postgres -w start
sudo: unknown user: postgres
That's based on other SO questions
moriartymacbookair13:~ macuser$ ls -ld /usr/local/var/postgres/
drwx------ 19 macuser admin 646B 20 Sep 19:46 /usr/local/var/postgres//
moriartymacbookair13:~ macuser$ chown -R postgres /usr/local/var/postgres
chown: postgres: illegal user name
moriartymacbookair13:~ macuser$ brew uninstall postgresql
Uninstalling /usr/local/Cellar/postgresql/9.3.5_1...
Error: Permission denied - /usr/local/bin/clusterdb
moriartymacbookair13:~ macuser$ brew uninstall postgres
Uninstalling /usr/local/Cellar/postgresql/9.3.5_1...
Error: Permission denied - /usr/local/bin/clusterdb
moriartymacbookair13:~ macuser$
As I say, I installed Postgresql only to try out heroku, so here is the error Chrome gives me from the code when I run foreman start locally.
(Note "the django app" is just a demo app https://devcenter.heroku.com/articles/getting-started-with-python#prepare-the-app)
It works for localhost:5000 home, but when I add /admin (the page that would interact with a database), I get the following error:
OperationalError at /admin/
FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
Request Method: GET
Request URL: http://localhost:5000/admin/
Django Version: 1.6.5
Exception Location: /Users/macuser/Dropbox/code/heroku/awe01/lib/python2.7/site-packages/psycopg2/__init__.py in connect, line 164
Python Executable: /Users/macuser/Dropbox/code/heroku/awe01/bin/python
Full error traceback log here: http://dpaste.com/3CREGVQ
Note this answer https://stackoverflow.com/a/5053003/870121
recommends createdb not just initdb
And the above error is the same as if I run createdb:
moriartymacbookair13:~ macuser$ createdb /usr/local/var/postgres/
createdb: could not connect to database template1: FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
Steps advised here
Postgres is failing with 'could not open relation mapping file "global/pg_filenode.map" '
are
Remove and re-add the launch agent
Kill the processes for <postgresql version number>
Initialize the db initdb /usr/local/var/postgres
Restart my computer
If this is relevant someone please tell me how to "Remove and re-add the launch agent"?
Is is this? (based on the output of the brew info postgresql from earlier)
moriartymacbookair13:~ macuser$ launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
moriartymacbookair13:~ macuser$ killall postgresql
No matching processes belonging to you were found
moriartymacbookair13:~ macuser$ killall postgres
No matching processes belonging to you were found
moriartymacbookair13:~ macuser$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
FINALLY
In case it inspires anyone,
moriartymacbookair13:~ macuser$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry and just ignore them. Thanks!
Warning: The /usr/local directory is not writable.
Even if this directory was writable when you installed Homebrew, other
software may change permissions on this directory. Some versions of the
"InstantOn" component of Airfoil are known to do this.
You should probably change the ownership and permissions of /usr/local
back to your user account.
Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected dylibs:
/usr/local/lib/libguide.dylib
/usr/local/lib/libKLF_OGL.dylib
Warning: Some directories in your path end in a slash.
Directories in your path should not end in a slash. This can break other
doctor checks. The following directories should be edited:
$/Users/macuser/Dropbox/code/aws/ElasticBeanstalkCommandLineTools/AWS-ElasticBeanstalk-CLI-2.6.3/AWSDevTools.sh/macosx/python2.7/ /Users/macuser/Dropbox/code/aws/ElasticBeanstalkCommandLineTools/AWS-ElasticBeanstalk-CLI-2.6.3/AWSDevTools/eb/macosx/python2.7/
Warning: /usr/bin occurs before /usr/local/bin
This means that system-provided programs will be used instead of those
provided by Homebrew. The following tools exist at both paths:
git
git-cvsserver
git-receive-pack
git-shell
git-upload-archive
git-upload-pack
Consider setting your PATH so that /usr/local/bin
occurs before /usr/bin. Here is a one-liner:
echo export PATH='/usr/local/bin:$PATH' >> ~/.bash_profile
moriartymacbookair13:~ macuser$
Earlier today I had this error
createdb: could not connect to database postgres: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
as per http://www.postgresql.org/docs/9.3/interactive/tutorial-createdb.html
but that error seems to have gone away now
Any advice on un or re-installing postgresql, resolving the unknown user: postgres error, setting location of #data_directory = 'ConfigDir' or any other tips... I would really appreciate it. It's been a full day lost to this now so I would appreciate anything at all.
FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
You had a PostgreSQL install, but deleted or moved the data directory while the server was running. Or you installed it on a removable disk that you then removed. Or some application on your system is messing with how other apps change files.
moriartymacbookair13:~ macuser$ postgres
postgres does not know where to find the server configuration file.
postgres is the server application. The client is psql. However, if global/pg_filenode.map is missing you're not going to have any more luck with psql.
moriartymacbookair13:~ macuser$ brew uninstall postgres
sudo brew uninstall postgres
Anyway there we have it, toward the end of that large output it
confirms the path for me is /usr/local/var/postgres, so I tried -D
moriartymacbookair13:~ macuser$ postgres -D /usr/local/postgres
postgres cannot access the server configuration file "/usr/local/postgres/postgresql.conf": No such file or directory
You got the path wrong. Note what "brew info" said:
Or, if you don't want/need launchctl, you can just run:
postgres -D /usr/local/var/postgres
As for the postgres user:
moriartymacbookair13:~ macuser$ sudo -u postgres pg_ctl -D /usr/local/var/postgres -w start
sudo: unknown user: postgres
I think some Mac OS X installs use _postgres or postgres_. I don't have a mac, so I can't check. But it doesn't matter in your case anyway, because the datadir is owned by your user, not postgres:
moriartymacbookair13:~ macuser$ ls -ld /usr/local/var/postgres/
drwx------ 19 macuser admin 646B 20 Sep 19:46 /usr/local/var/postgres//
It looks to me like you or some tool on your system has mangled your PostgreSQL data directory, corrupting it. This shouldn't happen, so I'd be making an effort to find out what removed pg_filenode.map and how/when.
If there is no data you care about in the data directory, you should delete it and re-initdb.
I don't have a Mac and I can't see how Homebrew configures your PostgreSQL install's startup, but it sounds like it probably runs postgres as user macuser. In which case, just:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
rm -rf /usr/local/var/postgres
initdb -D /usr/local/var/postgres -U postgres --auth-local peer --auth-host md5 -E utf-8l
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Finally, re:
Note, when I tried to use the postgres.app installer now it told me
"Could not start on Port 5432" and froze during installation
at a guess, when you tried that you had the postgres server from Homebrew already running on 5432, but nonfunctional because of the damaged data directory.
So you really have two things here:
Re-initdb the datadir to get PostgreSQL working again; and
Figure out how it got damaged in the first place
SOLUTION to the problem I posted
sudo brew uninstall postgresql (literally, sudo solved my problem... I had read that sudo brew can be dangerous but it worked well in this case)
Restart
Tried to run postgres.app ...got error
Restart
Fully uninstalled postgres.app (following instructions on site: quit, drag to the Trash, and then "Finder > Empty Trash..." as per http://postgresapp.com/documentation/install.html)
Restart
Delelted data folder as per trouble shooting instructions of postgresapp
i.e., the 5 steps here:
Resetting Postgres.app
http://postgresapp.com/documentation/troubleshooting.html (I may have done this earlier, I forget the order of steps 5/7)
Restart
Re-installed postgresql via postgres.app
Added the path to my /etc/profile file... adding to ~/.bashrc or ~/.bash_profile doesn't work for me for some reason. Here in an
Extract from /etc/profile for me
It might be from another file for you, see https://apple.stackexchange.com/questions/12993/why-doesnt-bashrc-run-automatically#comment13715_13019
# 1:
# For postgresql postgres postgres.app:
# Otherwise psql isn't a recognised command
# Based on http://postgresapp.com/documentation/cli-tools.html
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.3/bin
# 2:
# And this from https://devcenter.heroku.com/articles/heroku-postgresql#local-setup
export PATH="/Applications/Postgres93.app/Contents/MacOS/bin:$PATH:$PATH"
# 3:
# And now this based on error of file not existing
# from http://stackoverflow.com/questions/13868730/socket-file-var-pgsql-socket-s-pgsql-5432-missing-in-mountain-lion-os-x-ser -- don't think this made a difference though
export PGHOST=localhost
I think this is overkill; the first change is definitely useful as it allows me to go straight to psql just by typing "psql" in terminal (I personally didn't even need -h localhost); I'm less sure of the need of the other two additions
That's it.
HOW IT GOT DAMAGED IN THE FIRST PLACE
I haven't figured this out.
OTHER TIPS FOR ANY MAC USER WHO'S USING DJANGO ON HEROKU WITH POSTGRESQL
In case relevant, now that it's working, here are some more steps I've finished to get fully set up and comfortable with postgres and my new heroku app. This has been my first experience with heroku, django and postgresql so I have tried to note all details which struck me!
Note: I had locally finished the tutorial for the "polls" sample app (from https://docs.djangoproject.com/en/1.6/intro/tutorial01/). My goal was to add this /polls app to the "getting started" django sample app (from https://devcenter.heroku.com/articles/getting-started-with-python)
(Note: I chose django 1.6 since it seems to be the latest version supported by heroku right now.)
I am running Mac OS X 10.9.5 but some of my comments may be helpful for anyone confused about local/heroku postgres integration
Create a local postgres db to use with django app
Once postgres app is running (launched by double-clicking the program in Applications -> Postgres, then look out for the elephant in the notification bar if you're on a Mac), I can log in to psql. But first straight from the command line, I made a database by command
createdb cool01db
Note if you get this,
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432? or similar error, then you haven't set up the postgres app/server correctly, so don't proceed through my instructions
Now I could log in to psql as per instructions on http://postgresapp.com/documentation/cli-tools.html
psql cool01db
Running the psql command \d shows me they're nothing in this database yet; tables will be created automatically by django when I do my syncdb command
Note: Since I had already completed the polls tutorial elsewhere, I had copied the "polls" folder (which contains things like models.py) into the folder of my heroku sample app, so "polls" was a folder alongside "gettingstarted" and "hello". Therefore I also added "polls" to the list of "installed apps" in the gettingstarted/settings.py as per https://docs.djangoproject.com/en/1.6/intro/tutorial01/#activating-models
I edited the gettingstarted/settings.py file as per instructions on the django tutorial
https://docs.djangoproject.com/en/1.6/intro/tutorial01/#database-setup and https://docs.djangoproject.com/en/1.6/ref/settings/#std:setting-DATABASES
For me, this looked like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'cool01db',
'USER': '',
'PASSWORD': '',
'HOST': 'localhost', # '127.0.0.1' probably works also
'PORT': '5432',
}
}
To use the newly-created local postgres DB and not the heroku one
I commented out this link in gettingstarted/settings.py
# DATABASES['default'] = dj_database_url.config()
This is as per excellent comment, http://stackoverflow.com/a/16422414/870121
I navigated into the project folder (which contains the above three folders and files like manage.py) and ran (locally, not on heroku) python manage.py syncdb
Here I was prompted to create a superuser with a password
Note, now the tables for the heroku sample app (tables like hello_greeting) and the django polls app (tables like polls_choice and polls_question) are all automatically created in the local postgres db, and I can view them if I log in to psql (psql cool01db from the terminal) and type \d (once you're in)
I ran foreman start and navigated to http://localhost:5000/admin/ and entered my superuser username and password I had just created. This successfully logged me in to the /admin site powered by my LOCAL postgres database
I saw the admin panel with the opportunity to add a Question as per django tutorial, so I added a question with some answers as per https://docs.djangoproject.com/en/1.6/intro/tutorial02/#make-the-poll-app-modifiable-in-the-admin
Now I visited http://localhost:5000/polls and sure enough it was working
If in the terminal I ran psql cool01db and then the sql command \d I could see the tables, and select * from question_choice showed me the updates just made. Very satisfying.
To use the heroku postgres db in development
I uncommented this line in gettingstarted/settings.py
DATABASES['default'] = dj_database_url.config()
I ran heroku config
Knew to do this from https://devcenter.heroku.com/articles/getting-started-with-python#define-config-vars
Output was this:
(awe01)moriartymacbookair13:getstartapp macuser$ heroku config
=== awe01 Config Vars
DATABASE_URL: postgres://aqmorelettersqz:uqry_pnFmorelettersYf#ec2-XX-XXX-XX-XX.compute-1.amazonaws.com:5432/dbmorelettersce
HEROKU_POSTGRESQL_YELLOW_URL: postgres://aqmorelettersqz:uqry_pnFmorelettersYf#ec2-XX-XXX-XXX-XX.compute-1.amazonaws.com:5432/dbmorelettersce
PAPERTRAIL_API_TOKEN: kmorelettersx
TIMES: 2
(awe01)moriartymacbookair13:getstartapp macuser$
Then I pasted all of that into the hidden file .env which lies in the same folder as manage.py
Syntax note: this means the file .env looked like this:
TIMES=2
DATABASE_URL=postgres://aqmorelettersqz:uqry_pnFmorelettersYf#ec2-XX-XXX-XX-XX.compute-1.amazonaws.com:5432/dbmorelettersce
HEROKU_POSTGRESQL_YELLOW_URL=pos.... etc
...i.e., with = signs but no spaces around them (and no comments in this file)
Note, this involved deleting the original "DATABASE_URL=" entry from the .env file
As far as I understand, the .env file is what your local machine sees, but the output of "heroku config" is what the heroku machine sees (corrections welcome), so I wanted them to match.
NB: Once Postgres is installed and you can connect, you’ll need to export the DATABASE_URL environment variable for your app to connect to it when running locally.
Run command export DATABASE_URL=postgres:///$(whoami)
This tells Postgres to connect locally to the database matching your user account name (which is set up as part of installation).
That's from https://devcenter.heroku.com/articles/heroku-postgresql#local-setup
Note (to self): I don't understand what happens here (I can't see a file that's changed as a result of running this, or what exactly happens). But I did a restart of computer and everything still works, I didn't need to run this again.
Now on a heroku dyno I ran "syncdb"
heroku run python manage.py syncdb
(You will be prompted to create a superuser only if you haven't already done so)
I went to myappname.herokuapp.com/admin and logged in with the superuser account created for heroku postgresql (which I have the same as local for convenience). There I was able to add a question, and this time this question is added to the heroku postgresql db. Visiting myappname.herokuapp.com/polls shows me that question available for votes
I think that's it
Now if I do foreman start, what I see at /db or /polls is the same as if I visit my locallhost:5000 or myappname.herokuapp.com, since both are using the heroku postgresdb (I don't even need my postgres server running)
To view tables and run commants like \dt or SELECT * FROM... I can do a command heroku pg:psql as per https://devcenter.heroku.com/articles/getting-started-with-python#provision-a-database
Thanks Craig Ringer for the original answer.