Pytest and coverage: why do coverage results vary with directory structure? - django

I have a working test suite using Pytest on a fairly large django project. Problem is I am unable to achieve proper results using coverage, and I am wondering if it may be because of the projects directory structure.
Consider the following sample of the directory tree:
.
├── apps
│   ├── api
│   │   ├── __init__.py
│   │   ├── tests
│   │   │   ├── __init__.py
│   │   │   └── views
│   │   │   ├── __init__.py
│   │   │   └── test_tickets.py
│   │   └── views
│   │   ├── __init__.py
│   │   ├── tickets.py
│   ├── support
│   │   ├── __init__.py
│   │   ├── tests
│   │   │   ├── __init__.py
│   │   │   └── utils
│   │   │   ├── __init__.py
│   │   │   ├── test_management_commands.py
│   │   ├── utils
│   │   │   ├── __init__.py
│   │   │   ├── management_commands.py
And a sample output of the coverage report:
coverage run --source apps/ -m py.test apps/
coverage report
Name Stmts Miss Cover
---------------------------------------------------------------
apps/api/views/tickets.py 42 18 57%
apps/support/utils/management_commands.py 135 100 26%
Looking at the html report I can see many statements executed by the tests are not considered covered, even though they should be. I believe this coverage data to be incomplete, it seems to only be considering imports, definitions, and docstrings as covered.
Unable to determine why the coverage appeared incorrect, I tried running a single test module, with positive results:
coverage run --source apps/support/utils/management_commands.py -m py.test apps/support/tests/utils/test_management_commands.py
coverage report
Name Stmts Miss Cover
---------------------------------------------------------------
apps/support/utils/management_commands.py 135 68 50%
This is more accurate, the HTML report shows the statement I have tests for are indicated as covered this time. Unable to figure out why running a single test module yields accurate results, I modified the directory structure by moving the tests under a single parent folder.
.
├── apps
│   ├── api
│   │   ├── __init__.py
│   │   └── views
│   │   ├── __init__.py
│   │   ├── tickets.py
│   ├── support
│   │   ├── __init__.py
│   │   ├── utils
│   │   │   ├── __init__.py
│   │   │   ├── management_commands.py
├── tests
│   │   ├── __init__.py
│   ├── api
│   │   ├── __init__.py
│   │   └── views
│   │   ├── __init__.py
│   │   └── test_tickets.py
│   ├── support
│   │   ├── __init__.py
│   │   ├── utils
│   │   │   ├── __init__.py
│   │   │   ├── test_management_commands.py
Re-running coverage with this directory structure yields more accurate results:
coverage run --source apps/ -m py.test tests/
coverage report
Name Stmts Miss Cover
---------------------------------------------------------------
apps/api/views/tickets.py 42 0 100%
apps/support/utils/management_commands.py 135 68 50%
Can anyone explain why running coverage with py.test was yielding in complete coverage under the original directory structure? Was the directory structure actually the problem or am I missing something else here?
Additional info:
# pytest.ini
[pytest]
addopts = --nomigrations
markers =
slowtest: mark a test as being slow
integration: mark a test as being an integration test
INSTALLED_APPS += ('django_coverage', )
TEST_DISCOVER_PATTERN = 'test_*'
COVERAGE_MODULE_EXCLUDES = [
'settings',
'urls$',
'locale$',
'tests$',
'django',
'migrations',
'compressor',
'templates?$',
'fixtures$',
'static$',
]
ROOT_PATH = os.path.abspath('%s/' % os.path.dirname(__file__))
The .coveragerc
[run]
source = apps
omit =
apps/*/templates?/*
apps/*/migrations/*
apps/*/factories/*
apps/*/tests/*
[html]
directory = coverage
Module versions (some may be unrelated):
pytest==2.9.0
pytest-cov==2.2.1
pytest-django==2.9.1
django-coverage==1.2.4
coverage==4.0.3

I see I'm a little late to the party (3-year-old question with no accepted answer yet), but since I've just had this same question with a seemingly obvious answer: coverage will only report on code that is actually run. So if your tests don't call a bit of code and it doesn't get run during normal loading of the application, coverage will not show a report for that code. Code that doesn't run does not cause bugs :)

Related

Chef::Exceptions::FileNotFound: template[/var/www/html/index.html]

I am new to the chef I could not understand what is an issue. Following is my default script
apt_update 'Update the apt cache daily' do
frequency 86_400
action :periodic
end
package 'apache2'
service 'apache2' do
supports status: true
action [:enable, :start]
end
template '/var/www/html/index.html' do
source 'index.html.erb'
end
this is the error I am getting
[2020-04-25T12:57:00+00:00] FATAL: Stacktrace dumped to /home/vagrant/.chef/local-mode-cache/cache/chef-stacktrace.out
[2020-04-25T12:57:00+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-04-25T12:57:00+00:00] FATAL: Chef::Exceptions::FileNotFound: template[/var/www/html/index.html] (learn_chef_apache2::default line 18) had an error: Chef::Exceptions::FileNotFound: Cookbook 'learn_chef_apache2' (0.1.0) does not contain a file at any of these locations:
templates/host-vagrant.vm/index.html.erb
templates/ubuntu-18.04/index.html.erb
templates/ubuntu/index.html.erb
templates/default/index.html.erb
templates/index.html.erb
and this is my cookbooks tree
cookbooks
├── learn_chef_apache2
│   ├── Berksfile
│   ├── CHANGELOG.md
│   ├── chefignore
│   ├── LICENSE
│   ├── metadata.rb
│   ├── README.md
│   ├── recipes
│   │   └── default.rb
│   ├── spec
│   │   ├── spec_helper.rb
│   │   └── unit
│   │   └── recipes
│   │   └── default_spec.rb
│   └── test
│   └── integration
│   └── default
│   └── default_test.rb
├── learn_chef_appache2
│   └── templates
│   ├── default
│   └── index.html.erb
└── templates
└── index.html.erb
Can someone please help me what wrong I am doing and it will be great if you can share a link or explain it for my understanding.
what I did wrong was my template was created outside learn_chef_apache2 whereas it should be inside as follwing
cookbooks
└── learn_chef_apache2
├── Berksfile
├── CHANGELOG.md
├── chefignore
├── index.html.erb
├── LICENSE
├── metadata.rb
├── README.md
├── recipes
│   └── default.rb
├── spec
│   ├── spec_helper.rb
│   └── unit
│   └── recipes
│   └── default_spec.rb
├── templates
│   └── index.html.erb
└── test
└── integration
└── default
└── default_test.rb

List apps of a django project

I'm completely new for django and I'd want to list the apps of a django project, for example:
FeinCMS
I know that startapp creates the directory structure for an app. I wonder if there is either a function or a file to get the app list.
For example taking FeinCMS as an example, the repo contains:
feincms
├── AUTHORS
├── CHANGELOG.rst
├── CONTRIBUTING.rst
├── docs
│   ├── admin.rst
│   ├── advanced
│   ├── conf.py
│   ├── contenttypes.rst
│   ├── contributing.rst
│   ├── deprecation.rst
│   ├── extensions.rst
│   ├── faq.rst
│   ├── images
│   ├── index.rst
│   ├── installation.rst
│   ├── integration.rst
│   ├── Makefile
│   ├── medialibrary.rst
│   ├── migrations.rst
│   ├── page.rst
│   ├── releases
│   ├── settings.rst
│   ├── templatetags.rst
│   └── versioning.rst
├── feincms
│   ├── admin
│   ├── apps.py
│   ├── content
│   ├── contents.py
│   ├── context_processors.py
│   ├── contrib
│   ├── default_settings.py
│   ├── extensions
│   ├── __init__.py
│   ├── _internal.py
│   ├── locale
│   ├── management
│   ├── models.py
│   ├── module
│   ├── shortcuts.py
│   ├── signals.py
│   ├── static
│   ├── templates
│   ├── templatetags
│   ├── translations.py
│   ├── urls.py
│   ├── utils
│   └── views
├── LICENSE
├── MANIFEST.in
├── README.rst
├── setup.cfg
├── setup.py
└── tests
├── cov.sh
├── manage.py
├── requirements.txt
├── testapp
└── tox.ini
How can I use django to list the apps in the repo?
settings.INSTALLED_APPS is the list of apps. You can start a django shell (manage.py shell) to query the settings:
from django.conf import settings
print(settings.INSTALLED_APPS)
>>> ['user', 'django.contrib.auth', 'django.contrib.sites', ...]
Query django.apps ...
from django.apps import apps
for app in apps.get_app_configs():
print(app, app.name, app.label)
Results in ...
<ContentTypesConfig: contenttypes> django.contrib.contenttypes contenttypes
<AdminConfig: admin> django.contrib.admin admin

Eclipse: unresolved import

Eclipse Neon (4.6.0).
PyDev for Eclipse 5.1.2.201606231256
I have created a Django project:
File / New / Project / PyDev Django project
Selected "Add project directory to the PYTHONPATH".
Now I have this folder structure.
(django_comments) michael#ThinkPad:~/workspace/formsets$ tree
.
└── formsets
├── db.sqlite3
├── formsets
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-35.pyc
│   │   ├── settings.cpython-35.pyc
│   │   ├── urls.cpython-35.pyc
│   │   └── wsgi.cpython-35.pyc
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── home_page
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   ├── __init__.py
│   │   └── __pycache__
│   │   └── __init__.cpython-35.pyc
│   ├── models.py
│   ├── __pycache__
│   │   ├── admin.cpython-35.pyc
│   │   ├── __init__.cpython-35.pyc
│   │   ├── models.cpython-35.pyc
│   │   └── views.cpython-35.pyc
│   ├── templates
│   │   └── home_page
│   │   └── home_page.html
│   ├── tests.py
│   └── views.py
└── manage.py
In prjoect properties in PyDev-PYTHONPATH at the tab Source Folders I have:
/${PROJECT_DIR_NAME}
In home_page/views.py I have created HomePageView. And in urls.py I would like to import it:
from home_page.views import HomePageView
The problem is:
1) HomePageView is underlined with red line. Error is Unresolved import: HomePageView.
2) Code completion is not working.
By the way, if I run the project, it works. That HomePageView shows what was expected.
Could you help me understand what have I done wrongly.
Try to add a path to the directory where your manage.py file is located. So I'm guessing it to add:
/${PROJECT_DIR_NAME}/formsets

Django relative import from external app

I'm a 2 Scoops of Django 1.8 reader. Chapter 29 (what about those random utilities) suggests to create a core app to store commonly used code. It also suggests that you can use this syntax to import code from it:
e.g.
from core.models import TimeStampedModel
How ever it seems that this relative import does not work. I'm using cookiecutter-django and I needed to do:
from projectname.apps.core.models import TimeStampedModel
I tried adding my APPS_DIR to the path:
sys.path.insert(str(APPS_DIR))
But that resulted in import conflicts given that now there were 2 modules with the same name, new_app and projectname.apps.new_app.
I just want to avoid explicit imports. Is there a way to include the Installed Apps in the python path without creating import conflicts? what are best practices regarding external apps imports?
edit: adds project structure
.
├── README.rst
├── manage.py
├── config
│   ├── __init__.py
│   ├── settings
│   │   ├── __init__.py
│   │   ├── common.py
│   │   ├── local.py
│   │   ├── test.py
│   │   ├── production.py
│   │   └── staging.py
│   ├── urls.py
│   ├── views.py
│   └── wsgi.py
├── projectname
│   ├── __init__.py
│   ├── apps
│   │   ├── __init__.py
│   │   ├── core
│   │   │   └── __init__.py
│   │   └── new_app
│   │ └── __init__.py
│   ├── static
│   │   └── ...
│   └── templates
│   └── ...
├── requirements
│   ├── base.txt
│   ├── local.txt
│   ├── production.txt
│   └── test.txt
└── tests
└── ...
If you want to use relative imports you need to import it this way
from .core.models import TimeStampedModel
This would take the relative path from which the code is being executed, unlike absolute imports which are not supported in Django 1.8

Override django userena views

I know there is a plethora of questions related to this, like this one, but I does not seem to help.
My question is simple, given this function signature, how do I override the view profile_detail? I seem to be doing everything exactly as both resources say, but my templates are still not receiving the extra context. Here's what I am trying to do:
# This overrides and extends userena's view named 'profile_detail'.
def profileDetailView(request, username):
# Do some logic, etc.
foo_list = Some.objects.filter(id=username)
extra_context = {'foo_list': foo_list}
# I have also tried the following for extra_context:
extra_context['foo_list'] = foo_list
return userena_views.profile_detail(request, username, template_name='userena/profile_detail.html',
extra_context=extra_context)
I am not sure where I am going wrong. I have my URL's correct, as it is loading my views without error. Also, I have tested through the shell to create foo_list by import objects - it worked. I have also done a test to make sure the username being passed in actually results in an object being found. Thus - it seems like the only problem could be in the extra_context, at least I hope! Any help would be much appreciated!
EDIT: File structure Request
├── project
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── models.py
│   ├── models.pyc
│   ├── settings.py
│   ├── settings.pyc
│   ├── static
│   │   ├── css
│   │   │   └─
│   │   └── js
│   ├── test.py
│   ├── urls.py
│   ├── urls.pyc
│   ├── wsgi.py
│   └── wsgi.pyc
├── manage.py
├── media
├── templates
│   ├── admin
│   │   └── base_site.html
│   ├── base_site.html
│   ├── errors
│   │   └── not_authorized.html
│   └── userena
│   ├── base.html
│   ├── base_userena.html
│   ├── profile_detail.html
│   ├── profile_details.html
│   └── signup_form.html
└── utils
TEMPLATE_DIRS = (
'/home/username/project/project/templates',
)