I've been developing a web application under the docker environment. On that way, the approach that views.py in the container access to an other file failed. Following is the logs when the container named ‘web’ run
And, demo/demo_app_views.py is
class TestViews(TemplateView):
template_name='top.html'
with open('usr/src/demo/data/a', 'rb') as data:
b = pickle.load(data)
elder_brother=b['1']
def get_context_data(self, **kwargs):
context=super().get_context_data(**kwargs)
context['brother']=elder_brother
return context
top=TestViews.as_view()
No matter of course, I confirmed the file ‘a’ stored on ‘data’ folder in ‘web’ container
And Dockerfile is
FROM python:3.7-alpine
WORKDIR /usr/src/demo
RUN mkdir /usr/src/demo/staticfiles
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
RUN apk update \ && apk add postgresql-dev gcc python3-dev musl-dev
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY ./entrypoint.sh .
COPY . .
ENTRYPOINT ["/usr/src/demo/entrypoint.sh"]
and... Docker-compose.yml is
version: '3.7'
services:
web:
build: ./demo
command: gunicorn demo.wsgi:application --bind 0.0.0.0:8000
volumes:
- ./demo/:/usr/src/demo:cached
- static_volume:/usr/src/demo/staticfiles
expose:
- 8000
env_file:
- ./base.env
depends_on:
- db
db:
image: postgres:11.4-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=project_demo
- POSTGRES_PASSWORD=project_demo
- POSTGRES_DB=project_demo_dev
nginx:
build: ./nginx
volumes:
- static_volume:/usr/src/demo/staticfiles
ports:
- 1337:80
depends_on:
- web
volumes:
postgres_data:
static_volume:
and the displaced log after docker-compose up -d --build
web_1 | res = instance.__dict__[self.name] = self.func(instance)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 564, in urlconf_module
web_1 | return import_module(self.urlconf_name)
web_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
web_1 | return _bootstrap._gcd_import(name[level:], package, level)
web_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
web_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
web_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
web_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
web_1 | File "/usr/src/demo/demo/urls.py", line 21, in <module>
web_1 | path('demo_app/', include('demo_app.urls')),
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
web_1 | urlconf_module = import_module(urlconf_module)
web_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
web_1 | return _bootstrap._gcd_import(name[level:], package, level)
web_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
web_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
web_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
web_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
web_1 | File "/usr/src/demo/demo_app/urls.py", line 18, in <module>
web_1 | from . import views
web_1 | File "/usr/src/demo/demo_app/views.py", line 27, in <module>
web_1 | class TestViews(TemplateView):
web_1 | File "/usr/src/demo/demo_app/views.py", line 30, in TestViews
web_1 | with open('usr/src/demo/data/a', 'rb') as data:
web_1 | FileNotFoundError: [Errno 2] No such file or directory: 'usr/src/demo/data/a'
web_1 | Traceback (most recent call last):
web_1 | File "manage.py", line 21, in <module>
web_1 | main()
web_1 | File "manage.py", line 17, in main
web_1 | execute_from_command_line(sys.argv)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
web_1 | utility.execute()
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
web_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
web_1 | self.execute(*args, **cmd_options)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 361, in execute
web_1 | self.check()
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
web_1 | include_deployment_checks=include_deployment_checks,
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 65, in _run_checks
web_1 | issues.extend(super()._run_checks(**kwargs))
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
web_1 | return checks.run_checks(**kwargs)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
web_1 | new_errors = check(app_configs=app_configs)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 13, in check_url_config
web_1 | return check_resolver(resolver)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 23, in check_resolver
web_1 | return check_method()
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 398, in check
web_1 | for pattern in self.url_patterns:
web_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
web_1 | res = instance.__dict__[self.name] = self.func(instance)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 571, in url_patterns
web_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
web_1 | res = instance.__dict__[self.name] = self.func(instance)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 564, in urlconf_module
web_1 | return import_module(self.urlconf_name)
web_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
web_1 | return _bootstrap._gcd_import(name[level:], package, level)
web_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
web_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
web_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
web_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
web_1 | File "/usr/src/demo/demo/urls.py", line 21, in <module>
web_1 | path('demo_app/', include('demo_app.urls')),
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
web_1 | urlconf_module = import_module(urlconf_module)
web_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
web_1 | return _bootstrap._gcd_import(name[level:], package, level)
web_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
web_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
web_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
web_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
web_1 | File "/usr/src/demo/demo_app/urls.py", line 18, in <module>
web_1 | from . import views
web_1 | File "/usr/src/demo/demo_app/views.py", line 27, in <module>
web_1 | class TestViews(TemplateView):
web_1 | File "/usr/src/demo/demo_app/views.py", line 30, in TestViews
web_1 | with open('usr/src/demo/data/a', 'rb') as data:
web_1 | FileNotFoundError: [Errno 2] No such file or directory: 'usr/src/demo/data/a'
web_1 | [2020-07-04 07:52:18 +0000] [1] [INFO] Starting gunicorn 20.0.4
web_1 | [2020-07-04 07:52:18 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
web_1 | [2020-07-04 07:52:18 +0000] [1] [INFO] Using worker: sync
web_1 | [2020-07-04 07:52:18 +0000] [10] [INFO] Booting worker with pid: 10
I’m not good in docker, so please tell me how to access the file ‘a’, which is dictionary type file, not database
Thank you
David's advisable comment led the answer and it is to change
open('usr/src/demo...')
to
open('/usr/src/demo...')
Thank David
Related
I am doing a docker django project with celery in which the project's name for main container is main_config. the main_ms\main_config\celery.py looks like
import os
from celery import Celery
from django.conf import settings
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
includedTasks=[]
CeleryApp = Celery ('main_config', broker=settings.CELERY_BROKER_URL, backend=settings.CELERY_RESULT_BACKEND,include=includedTasks)
CeleryApp.config_from_object('django.conf:settings', namespace='CELERY')
CeleryApp.autodiscover_tasks()
in docker-compose this containers is like:
main_django_ser:
container_name: main_django_container
build:
context: ./main_ms
dockerfile: Dockerfile.main
command: sh -c "
celery -A main_config.celery worker --loglevel=info &&
python manage.py runserver 0.0.0.0:8000"
by starting docker-compose gave the error below:
main_django_container | Usage: celery [OPTIONS] COMMAND [ARGS]...
main_django_container | Try 'celery --help' for help.
main_django_container |
main_django_container | Error: Invalid value for '-A' / '--app':
main_django_container | Unable to load celery application.
main_django_container | While trying to load the module main_config.celery the following error occurred:
main_django_container | Traceback (most recent call last):
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 57, in convertmain_django_container | return find_app(value)
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/app/utils.py", line 384, in find_app
main_django_container | sym = symbol_by_name(app, imp=imp)
main_django_container | File "/usr/local/lib/python3.9/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
main_django_container | module = imp(module_name, package=package, **kwargs)
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/utils/imports.py", line 105, in import_from_cwd
main_django_container | return imp(module, package=package)
main_django_container | File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
main_django_container | return _bootstrap._gcd_import(name[level:], package, level)
main_django_container | File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
main_django_container | File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
main_django_container | File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
main_django_container | File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
main_django_container | File "<frozen importlib._bootstrap_external>", line 850, in exec_module
main_django_container | File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
main_django_container | File "/main_ms/main_config/__init__.py", line 1, in <module>
main_django_container | from .celery import CeleryApp as celery_app
main_django_container | File "/main_ms/main_config/celery.py", line 6, in <module>
main_django_container | CeleryApp = Celery ('main_config', broker=settings.CELERY_BROKER_URL, backend=settings.CELERY_RESULT_BACKEND,include=includedTasks)
main_django_container | File "/usr/local/lib/python3.9/site-packages/django/conf/__init__.py", line 87, in __getattr__
main_django_container | self._setup(name)
main_django_container | File "/usr/local/lib/python3.9/site-packages/django/conf/__init__.py", line 74, in _setup
main_django_container | self._wrapped = Settings(settings_module)
main_django_container | File "/usr/local/lib/python3.9/site-packages/django/conf/__init__.py", line 183, in __init__
main_django_container | mod = importlib.import_module(self.SETTINGS_MODULE)
main_django_container | File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
main_django_container | return _bootstrap._gcd_import(name[level:], package, level)
main_django_container | ModuleNotFoundError: No module named 'config'
so I changed config to main_config in line 4 of main_ms\main_config\celery.py
and this time gave error:
main_django_container | /usr/local/lib/python3.9/site-packages/celery/platforms.py:840: SecurityWarning: You're running the worker with superuser privileges: this is
main_django_container | absolutely not recommended!
main_django_container |
main_django_container | Please specify a different user using the --uid option.
main_django_container |
main_django_container | User information: uid=0 euid=0 gid=0 egid=0
main_django_container |
main_django_container | warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(
main_django_container | Traceback (most recent call last):
main_django_container | File "/usr/local/bin/celery", line 8, in <module>
main_django_container | sys.exit(main())
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/__main__.py", line 15, in main
main_django_container | sys.exit(_main())
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 217, in main
main_django_container | return celery(auto_envvar_prefix="CELERY")
main_django_container | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
main_django_container | return self.main(*args, **kwargs)
main_django_container | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
main_django_container | rv = self.invoke(ctx)
main_django_container | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
main_django_container | return _process_result(sub_ctx.command.invoke(sub_ctx))
main_django_container | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
main_django_container | return ctx.invoke(self.callback, **ctx.params)
main_django_container | File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
main_django_container | return __callback(*args, **kwargs)
main_django_container | File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_funcmain_django_container | return f(get_current_context(), *args, **kwargs)
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/bin/base.py", line 134, in caller
main_django_container | return f(ctx, *args, **kwargs)
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/bin/worker.py", line 343, in workermain_django_container | worker = app.Worker(
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/worker/worker.py", line 99, in __init__
main_django_container | self.setup_instance(**self.prepare_args(**kwargs))
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/worker/worker.py", line 120, in setup_instance
main_django_container | self._conninfo = self.app.connection_for_read()
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 808, in connection_for_read
main_django_container | return self._connection(url or self.conf.broker_read_url, **kwargs)
main_django_container | File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 867, in _connection
main_django_container | return self.amqp.Connection(
main_django_container | File "/usr/local/lib/python3.9/site-packages/kombu/connection.py", line 181, in __init__
main_django_container | if not get_transport_cls(transport).can_parse_url:
main_django_container | File "/usr/local/lib/python3.9/site-packages/kombu/transport/__init__.py", line 85, in
get_transport_cls
main_django_container | _transport_cache[transport] = resolve_transport(transport)
main_django_container | File "/usr/local/lib/python3.9/site-packages/kombu/transport/__init__.py", line 70, in
resolve_transport
main_django_container | return symbol_by_name(transport)
main_django_container | File "/usr/local/lib/python3.9/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
main_django_container | module = imp(module_name, package=package, **kwargs)
main_django_container | File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
main_django_container | return _bootstrap._gcd_import(name[level:], package, level)
main_django_container | File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
main_django_container | File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
main_django_container | File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
main_django_container | File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
main_django_container | File "<frozen importlib._bootstrap_external>", line 850, in exec_module
main_django_container | File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
main_django_container | File "/usr/local/lib/python3.9/site-packages/kombu/transport/redis.py", line 262, in <module>
main_django_container | class PrefixedStrictRedis(GlobalKeyPrefixMixin, redis.Redis):
main_django_container | AttributeError: 'NoneType' object has no attribute 'Redis'
I should say the CELERY_BROKER_URL and CELERY_RESULT_BACKEND are redis://redis:6380/0 in main_ms\main_config\settings.py
update (may be considered as an answer for some of errors above)
I realized that I had not added celery[redis]==5.2.7 in requirements.txt which installs python packages. also didnt have redis container in docker-compose.yml and I also checked to have celery -A main_config.celery worker --loglevel=info && specially _config part in command section in django conatainers. now I dont get errors above but a new error pops up: [2022-11-06 13:54:42,190: ERROR/MainProcess] consumer: Cannot connect to redis://redis:6379/0: Error -3 connecting to redis:6379. Try again..
I've split my django environment as per this post.
In settings/base.py I have BASE_DIR specified:
BASE_DIR = Path(__file__).resolve().parent.parent.parent
In settings/__init__.py I have:
from .base import *
env = os.environ['ENV']
...
if env == 'test':
from .test import *
...
In settings/test.py I have DATA_VOLUME_BASE_DIR = BASE_DIR / 'scanned_images'.
I expect that Django loads the settings module, imports settings/base.py, checks the environment ('test'), imports settings/test.py (which works) and the variable is available (which isn't). My stacktrace:
sid_test_web | Traceback (most recent call last):
sid_test_web | File "/code/manage.py", line 22, in <module>
sid_test_web | main()
sid_test_web | File "/code/manage.py", line 18, in main
sid_test_web | execute_from_command_line(sys.argv)
sid_test_web | File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_
sid_test_web | utility.execute()
sid_test_web | File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 363, in execute
sid_test_web | settings.INSTALLED_APPS
sid_test_web | File "/usr/local/lib/python3.10/site-packages/django/conf/__init__.py", line 82, in __getattr__
sid_test_web | self._setup(name)
sid_test_web | File "/usr/local/lib/python3.10/site-packages/django/conf/__init__.py", line 69, in _setup
sid_test_web | self._wrapped = Settings(settings_module)
sid_test_web | File "/usr/local/lib/python3.10/site-packages/django/conf/__init__.py", line 170, in __init__
sid_test_web | mod = importlib.import_module(self.SETTINGS_MODULE)
sid_test_web | File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
sid_test_web | return _bootstrap._gcd_import(name[level:], package, level)
sid_test_web | File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
sid_test_web | File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
sid_test_web | File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
sid_test_web | File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
sid_test_web | File "<frozen importlib._bootstrap_external>", line 883, in exec_module
sid_test_web | File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
sid_test_web | File "/code/my_project/settings/__init__.py", line 9, in <module>
sid_test_web | from .test import *
sid_test_web | File "/code/my_project/settings/test.py", line 39, in <module>
sid_test_web | DATA_VOLUME_BASE_DIR = BASE_DIR / 'scanned_images'
sid_test_web | NameError: name 'BASE_DIR' is not defined
How do I make BASE_DIR variable available?
You have to import the .base setting in your test settings to make them available in test.py
from .base import *
# ... your other test settings
The second thing is to choose which settings you like in the __init__.py.
env = os.environ['ENV']
if env == 'test':
from .test import *
else:
from .base import *
The main question is: What is the rest framework ViewSet friendly way to make the reverse routes?
In a django cookiecutter site I have api_router.py with this
router.register(r"users", UserViewSet)
router.register(r"userprofile", UserProfileViewSet, basename="userprofile")
router.register(r"student", StudentViewSet)
urlpatterns = router.urls
app_name = "api"
print(f"we have routes")
for route in router.urls:
print(route)
print(f"{reverse('student-list') = }")
which errors out saying that reverse can't find student-list
the student model.py has a StudentViewSet with the standard
class StudentViewSet(
RetrieveModelMixin,
CreateModelMixin,
ListModelMixin,
UpdateModelMixin,
GenericViewSet,
):
so it should have the stuff needed to create the student-list
the output of that print statement is showing student-list
**** edited ....***
athenaeum_local_django | <URLPattern '^student/$' [name='student-list']>
athenaeum_local_django | <URLPattern '^student\.(?P<format>[a-z0-9]+)/?$' [name='student-list']>
athenaeum_local_django | <URLPattern '^student/(?P<userprofile__user__username>[^/.]+)/$' [name='student-detail']>
athenaeum_local_django | <URLPattern '^student/(?P<userprofile__user__username>[^/.]+)\.(?P<format>[a-z0-9]+)/?$' [name='student-detail']>
The end of the error is as follows:
ahenaeum_local_django | File "/app/config/urls.py", line 29, in <module>
athenaeum_local_django | path("api/", include("config.api_router")),
athenaeum_local_django | File "/usr/local/lib/python3.9/site-packages/django/urls/conf.py", line 34, in include
athenaeum_local_django | urlconf_module = import_module(urlconf_module)
athenaeum_local_django | File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
athenaeum_local_django | return _bootstrap._gcd_import(name[level:], package, level)
athenaeum_local_django | File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
athenaeum_local_django | File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
athenaeum_local_django | File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
athenaeum_local_django | File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
athenaeum_local_django | File "<frozen importlib._bootstrap_external>", line 850, in exec_module
athenaeum_local_django | File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
athenaeum_local_django | File "/app/config/api_router.py", line 22, in <module>
athenaeum_local_django | print(f"{reverse('student-list') = }")
athenaeum_local_django | File "/usr/local/lib/python3.9/site-packages/django/urls/base.py", line 86, in reverse
athenaeum_local_django | return resolver._reverse_with_prefix(view, prefix, *args, **kwargs)
athenaeum_local_django | File "/usr/local/lib/python3.9/site-packages/django/urls/resolvers.py", line 698, in _reverse_with_prefix
athenaeum_local_django | raise NoReverseMatch(msg)
athenaeum_local_django | django.urls.exceptions.NoReverseMatch: Reverse for 'student-list' not found. 'student-list' is not a valid view function or pattern name.
Unexpected API error for athenaeum_local_django (HTTP code 500)
Since you have set an app_name, you will have to use it in reverse, so:
reverse('api:student-list')
I have a cookiecutter-django application with docker that works fine if I run it locally with docker compose -f local.yml up. Now I am trying to deploy it so first I created a docker-machine in my computer (using macOS Catalina) and activated it.
Now, inside the docker-machine, the docker-compose build works fine, but when I run it, the application crashes.
Any idea what can be happening? I have been trying to solve this for almost a week now...
This are my logs when I do docker-compose up in the docker-machine:
Creating network "innovacion_innsai_default" with the default driver
Creating innovacion_innsai_postgres_1 ... done
Creating innovacion_innsai_django_1 ... done
Creating innovacion_innsai_node_1 ... done
Attaching to innovacion_innsai_postgres_1, innovacion_innsai_django_1, innovacion_innsai_node_1
postgres_1 | 2020-03-16 08:41:12.472 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-03-16 08:41:12.472 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-03-16 08:41:12.473 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-03-16 08:41:12.494 UTC [21] LOG: database system was shut down at 2020-03-16 08:31:09 UTC
postgres_1 | 2020-03-16 08:41:12.511 UTC [1] LOG: database system is ready to accept connections
django_1 | PostgreSQL is available
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | psycopg2.errors.UndefinedTable: relation "innovation_sector" does not exist
django_1 | LINE 1: ...n_sector"."id", "innovation_sector"."sector" FROM "innovatio...
django_1 | ^
django_1 |
django_1 |
django_1 | The above exception was the direct cause of the following exception:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "manage.py", line 30, in <module>
django_1 | execute_from_command_line(sys.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
django_1 | utility.execute()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
django_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
django_1 | self.execute(*args, **cmd_options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 361, in execute
django_1 | self.check()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
django_1 | include_deployment_checks=include_deployment_checks,
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 65, in _run_checks
django_1 | issues.extend(super()._run_checks(**kwargs))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
django_1 | return checks.run_checks(**kwargs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
django_1 | new_errors = check(app_configs=app_configs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
django_1 | all_namespaces = _load_all_namespaces(resolver)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
django_1 | url_patterns = getattr(resolver, 'url_patterns', [])
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 584, in url_patterns
django_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 577, in urlconf_module
django_1 | return import_module(self.urlconf_name)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/config/urls.py", line 18, in <module>
django_1 | path("", include("innovacion_innsai.innovation.urls", namespace="innovation")),
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
django_1 | urlconf_module = import_module(urlconf_module)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/app/innovacion_innsai/innovation/urls.py", line 2, in <module>
django_1 | from innovacion_innsai.innovation import views
django_1 | File "/app/innovacion_innsai/innovation/views.py", line 9, in <module>
django_1 | from .analytics import alimentacion_cases, agro_cases, turismo_cases, movilidad_cases
django_1 | File "/app/innovacion_innsai/innovation/analytics.py", line 17, in <module>
django_1 | for case in Case.objects.filter(sector__sector=sectors[0]):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 308, in __getitem__
django_1 | qs._fetch_all()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
django_1 | self._result_cache = list(self._iterable_class(self))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 55, in __iter__
django_1 | results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1133, in execute_sql
django_1 | cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
django_1 | return super().execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
django_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
django_1 | return executor(sql, params, many, context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
django_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | django.db.utils.ProgrammingError: relation "innovation_sector" does not exist
django_1 | LINE 1: ...n_sector"."id", "innovation_sector"."sector" FROM "innovatio...
django_1 | ^
django_1 |
innovacion_innsai_django_1 exited with code 1
node_1 |
node_1 | > innovacion_innsai#1.1.0 dev /app
node_1 | > gulp
node_1 |
node_1 | [08:41:22] Using gulpfile /app/gulpfile.js
node_1 | [08:41:22] Starting 'default'...
node_1 | [08:41:22] Starting 'styles'...
node_1 | [08:41:22] Starting 'scripts'...
node_1 | [08:41:22] Starting 'imgCompression'...
node_1 | [08:41:22] gulp-imagemin: Minified 0 images
This is my local.yml:
version: '3'
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: innovacion_innsai_local_django
depends_on:
- postgres
volumes:
- .:/app
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: innovacion_innsai_production_postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data
- local_postgres_data_backups:/backups
env_file:
- ./.envs/.local/.postgres
#Estas dos siguientes lineas las he añadido yo luego
ports:
- "5432:5432"
node:
build:
context: .
dockerfile: ./compose/local/node/Dockerfile
image: innovacion_innsai_local_node
depends_on:
- django
volumes:
- .:/app
# http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html
- /app/node_modules
command: npm run dev
ports:
- "3000:3000"
# Expose browsersync UI: https://www.browsersync.io/docs/options/#option-ui
- "3001:3001"
And this is my .postgres file inside the .envs/local folder:
# PostgreSQL
# ------------------------------------------------------------------------------
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=innovacion_innsai
POSTGRES_USER=debug
POSTGRES_PASSWORD=debug
File "/app/innovacion_innsai/innovation/analytics.py", line 17, in <module>
django_1 | for case in Case.objects.filter(sector__sector=sectors[0]):
It means that when django is initializing (the moment it loads the modules) it's doing DB queries on database that has no migrations applied. You need to transform whatever your code is doing on line 17 to run it lazy.
I am new to django, docker and scrapy and I am trying to run a django app that also uses scrapy (I basically create a django app that is also a scrapy app and try to call a spider from a django view). Despite specifying this scrapy in requirements.txt and running pip from the Dockerfile, the dependencies are not installed in the container prior to running python manage.py runserver 0.0.0.0:8000 and the django app fails during the system checks, resulting in the web container exiting because of the following exception:
| Exception in thread django-main-thread:
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
web_1 | self.run()
web_1 | File "/usr/local/lib/python3.7/threading.py", line 870, in run
web_1 | self._target(*self._args, **self._kwargs)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", line 54, in wrapper
web_1 | fn(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
web_1 | self.check(display_num_errors=True)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
web_1 | include_deployment_checks=include_deployment_checks,
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
web_1 | return checks.run_checks(**kwargs)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
web_1 | new_errors = check(app_configs=app_configs)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
web_1 | all_namespaces = _load_all_namespaces(resolver)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
web_1 | url_patterns = getattr(resolver, 'url_patterns', [])
web_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
web_1 | res = instance.__dict__[self.name] = self.func(instance)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 579, in url_patterns
web_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
web_1 | res = instance.__dict__[self.name] = self.func(instance)
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 572, in urlconf_module
web_1 | return import_module(self.urlconf_name)
web_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
web_1 | return _bootstrap._gcd_import(name[level:], package, level)
web_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
web_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
web_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
web_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
web_1 | File "/code/composeexample/urls.py", line 21, in <module>
web_1 | path('scrapy/', include('scrapy_app.urls')),
web_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
web_1 | urlconf_module = import_module(urlconf_module)
web_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
web_1 | return _bootstrap._gcd_import(name[level:], package, level)
web_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
web_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
web_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
web_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
web_1 | File "/code/scrapy_app/urls.py", line 4, in <module>
web_1 | from scrapy_app import views
web_1 | File "/code/scrapy_app/views.py", line 1, in <module>
web_1 | from scrapy.crawler import CrawlerProcess
web_1 | ModuleNotFoundError: No module named 'scrapy'
I tried using pip3 instead of pip, pip install --no-cache-dir -r requirements.txt, changing the order of the statements in the Dockerfile and I also checked that Scrapy==1.7.3 appears in requirements.txt. Nothing seems to work.
This is my Dockerfile:
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
And this is my docker-compose.yml:
version: '3'
services:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
A little late, but I came across this issue and eventually figured it out (and am putting this here for anyone else who has the same issue).
The first time I tried building my docker image, my requirements.txt didn't contain the required modules. Of course, I added the required modules but nothing seemed to happen, this is because we need to rebuild the container from scratch, otherwise we are just attempting to build the same version again and again.
To rebuild the container with our updated files, we write:
docker-compose rm -f
docker-compose pull
docker-compose up
If that doesn't work, try the same but replace final line with docker-compose up --build -d.
I got this from this answer.
It seems like you are lacking scrapy in your requirements.txt !
I tried to build a minimal version with all of your components. Hope it helps.
test.py
import scrapy
from time import sleep
def main():
while True:
print(scrapy)
sleep(1)
if __name__ == "__main__":
main()
requirements.txt
Scrapy==1.7.3
Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED 1
WORKDIR /code
COPY requirements.txt .
RUN pip3 install -r requirements.txt
COPY . ./
CMD [ "python3", "test.py" ]
docker-compose.yml
version: '3'
services:
db:
image: postgres
web:
build: .
ports:
- "8000:8000"
depends_on:
- db