ModuleNotFoundError psycopg2 when deploying in heroku even though psycopg2 is installed - django

Question

I migrated my website from sqlite3 to postgresql. I'm deploying my website in heroku but this error came up when I'm executing command
push heroku master
-----> Python app detected
! Python has released a security update! Please consider upgrading to python-3.7.3
Learn More: https://devcenter.heroku.com/articles/python-runtimes
-----> Installing python-3.7.2
-----> Installing pip
-----> Installing dependencies with Pipenv 2018.5.18…
Installing dependencies from Pipfile…
-----> Installing SQLite3
-----> $ python manage.py collectstatic --noinput
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 20, in <module>
import psycopg2 as Database
ModuleNotFoundError: No module named 'psycopg2'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/app/.heroku/python/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/app/.heroku/python/lib/python3.7/site-packages/django/core/management/__init__.py", line 357, in execute
django.setup()
File "/app/.heroku/python/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/app/.heroku/python/lib/python3.7/site-packages/django/apps/registry.py", line 114, in populate
app_config.import_models()
File "/app/.heroku/python/lib/python3.7/site-packages/django/apps/config.py", line 211, in import_models
self.models_module = import_module(models_module_name)
File "/app/.heroku/python/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/tmp/build_bda0f9a0e468e1cd92eba305080085ba/home/models.py", line 3, in <module>
from wagtail.core.models import Page
File "/app/.heroku/python/lib/python3.7/site-packages/wagtail/core/models.py", line 8, in <module>
from django.contrib.auth.models import Group, Permission
File "/app/.heroku/python/lib/python3.7/site-packages/django/contrib/auth/models.py", line 2, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/app/.heroku/python/lib/python3.7/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
class AbstractBaseUser(models.Model):
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/models/base.py", line 117, in __new__
new_class.add_to_class('_meta', Options(meta, app_label))
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/models/base.py", line 321, in add_to_class
value.contribute_to_class(cls, name)
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/models/options.py", line 204, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/__init__.py", line 28, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/utils.py", line 201, in __getitem__
backend = load_backend(db['ENGINE'])
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/utils.py", line 110, in load_backend
return import_module('%s.base' % backend_name)
File "/app/.heroku/python/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/app/.heroku/python/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 24, in <module>
raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2'
! Error while running '$ python manage.py collectstatic --noinput'.
See traceback above for details.
You may need to update application code to resolve this error.
Or, you can disable collectstatic for this application:
$ heroku config:set DISABLE_COLLECTSTATIC=1
https://devcenter.heroku.com/articles/django-assets
! Push rejected, failed to compile Python app.
! Push failed
I don't know why it says import error since I already installed psycopg2 in my virtual environment. The python shell does not even throw some errors when I import it and I can run my server successfully in my local machine
In my requirements.txt file, psycopg2==2.8.3 is listed there.
My Pipfile
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
django = "<2.3,>=2.2"
wagtail = "<2.6,>=2.5"
[requires]
python_version = "3.7"
My database configuration in base.py of settings folder
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'personalsite',
}
}
in production.py of my settings folder
DATABASES['default'] = dj_database_url.config()
in requirements.txt
psycopg2==2.8.2
beautifulsoup4==4.6.0
certifi==2019.6.16
chardet==3.0.4
dj-database-url==0.5.0
dj-static==0.0.6
Django==2.2.2
django-modelcluster==4.4
django-taggit==0.24.0
django-toolbelt==0.0.1
django-treebeard==4.3
djangorestframework==3.9.4
draftjs-exporter==2.1.6
gunicorn==19.9.0
html5lib==1.0.1
idna==2.8
Pillow==5.4.1
pytz==2019.1
requests==2.22.0
six==1.12.0
sqlparse==0.3.0
static3==0.7.0
Unidecode==1.1.0
urllib3==1.25.3
wagtail==2.5.1
webencodings==0.5.1
Willow==1.1

Answer 1

It seems like you installed all those packages to your local Python installation and not to your virtual environment. Your Pipfile doesn't reflect the packages such as psycopg2, dj_database_url etc. Please install the packages to your virtual environment.
First, activate your virtual environment,
pipenv shell
Then install the following packages,
pipenv install psycopg2-binary
pipenv install dj-database-url
Run pipenv lock to generate the appropriate Pipfile.lock. And try to push it to Heroku once again using the following command.
push heroku master