errors while running python scripts triggered by django app - django

Question

I am building a fully open source testrunner for my needs but I am running into some problems. the test runner parses a yaml file for a set of scripts in various paths and executed the scripts and uses a lib that i will be creating to return the outcome. right now i have a simple ping script that im working to get running and testing as i progress but i am getting a lot of errors. the errors are below and all the source code is also shown below the errors.
The github repo for this is here. feel free to pull it in and test the issues i am seeing.
https://github.com/castaway2000/testrunner
The issue:
I am trying to use the testrunner i built to parse a yaml file for paths to scripts i am writing for projects im using.
For example if want to use a group of certain tests on a target, i can make a yaml file for each set of the types of tests.
There is a certain problem I am seeing with this however, the relative path and exact path of the files are not able to use the django libraries, cause its unable to find the path of the libraries unless its running from the top level of the django app (ie. ./ping_google.py vs ./testcases/ping_google.py)
but on top of that, the django app says is not running when the independent libraries are referencing models.py and admin.py cant import models from the same directory. I need help fixing and understanding this issue.
Here is the rundown(stacktrace):
Enterprize:testrunner xwing$ python3 ping_google.py
Traceback (most recent call last):
File "ping_google.py", line 1, in <module>
from testrunnerlib.test import HostInterface
File "/Users/xwing/PycharmProjects/testrunner/testrunnerlib/test.py", line 11, in <module>
from testrunner.models import Host, TestSuite
File "/Users/xwing/PycharmProjects/testrunner/testrunner/models.py", line 5, in <module>
class Host(models.Model):
File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 105, in __new__
app_config = apps.get_containing_app_config(module)
File "/usr/local/lib/python3.6/site-packages/django/apps/registry.py", line 237, in get_containing_app_config
self.check_apps_ready()
File "/usr/local/lib/python3.6/site-packages/django/apps/registry.py", line 124, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
after i put import django and django.setup() in the django settings file the above error goes away but i get the following error:
Enterprize:testrunner xwing$ python3 ping_google.py
Traceback (most recent call last):
File "ping_google.py", line 1, in <module>
from testrunnerlib.test import HostInterface
File "/Users/xwing/PycharmProjects/testrunner/testrunnerlib/test.py", line 11, in <module>
from testrunner.models import Host, TestSuite
File "/Users/xwing/PycharmProjects/testrunner/testrunner/models.py", line 5, in <module>
class Host(models.Model):
File "/Users/xwing/PycharmProjects/testrunner/testrunner/models.py", line 6, in Host
ip_address = models.CharField(max_length=16)
File "/usr/local/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 1043, in __init__
super(CharField, self).__init__(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 166, in __init__
self.db_tablespace = db_tablespace or settings.DEFAULT_INDEX_TABLESPACE
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 97, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/Users/xwing/PycharmProjects/testrunner/testrunner/settings.py", line 133, in <module>
django.setup()
File "/usr/local/lib/python3.6/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.6/site-packages/django/apps/registry.py", line 115, in populate
app_config.ready()
File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/apps.py", line 23, in ready
self.module.autodiscover()
File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/__init__.py", line 26, in autodiscover
autodiscover_modules('admin', register_to=site)
File "/usr/local/lib/python3.6/site-packages/django/utils/module_loading.py", line 50, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/Users/xwing/PycharmProjects/testrunner/testrunner/admin.py", line 3, in <module>
from testrunner.models import Host, TestSuite
ImportError: cannot import name 'Host'
Fixing this will help with testing the rest of the build out scenarios for the testrunner but i will still need advice on the relative path and environment python needs to use to know where to look for these libraries. if possible i can put the libs in the root python directory so the libs are irrelevant to the problem.
problem file:
from testrunnerlib.test import HostInterface
from testrunnerlib.outcomes import Outcomes
from ping3 import ping
def pinger(host):
result = Outcomes()
try:
ping_google = ping(host)
print(ping_google)
if ping_google:
return result.passed()
msg = 'ping had an issue, the following is all we know %s' % ping_google
return result.failed(msg)
except Exception as e:
return result.aborted(exception=e)
if __name__ == '__main__':
pinger(HostInterface().target)
only lib with django imports:
import yaml
import subprocess
from testrunner.models import Host, TestSuite
class HostInterface(object):
def __init__(self):
self._target = 'not set'
#property
def target(self):
return self._target
#target.setter
def target(self, value):
print("setter of target called", value)
self._target = value
#target.deleter
def target(self):
print("deleter of target called")
del self._target
def host(self):
out = Host.objects.get(id=self.target).name
return out
class YamlInterface:
def __init__(self, yamlfile):
self.file = yamlfile
def handle_yaml(self):
data = TestSuite.objects.get(id=self.file)
yamldata = yaml.safe_load(data.text)
for i in yamldata['testsuite']:
status = subprocess.call('python3 %s' % i, shell=True)
print(status)
def run_tests(host, yaml):
h_interface = HostInterface()
h_interface.target = host
h_interface.host()
yaml = YamlInterface(yaml)
yaml.handle_yaml()
the models:
from __future__ import unicode_literals
from django.db import models
class Host(models.Model):
ip_address = models.CharField(max_length=16)
port = models.IntegerField()
name = models.CharField(max_length=256)
class TestSuite(models.Model):
name = models.CharField(max_length=256)
text = models.TextField()
is_active = models.BooleanField(default=True)
created = models.DateTimeField(auto_now_add=True, auto_now=False)
updated = models.DateTimeField(auto_now_add=False, auto_now=True)
def __str__(self):
return "%s" % self.name
admin.py
from django.contrib import admin
from django import forms
from testrunner.models import Host, TestSuite
class HostAdmin(admin.ModelAdmin):
list_display = ['name']
fields = ('name', 'ip_address', 'port')
def __str__(self):
return '%s' % self.name
pass
admin.site.register(Host, HostAdmin)
class TestSuiteAdmin(admin.ModelAdmin):
def formfield_for_dbfield(self, db_field, **kwargs):
formfield = super(TestSuiteAdmin, self).formfield_for_dbfield(db_field, **kwargs)
if db_field.name == 'text':
formfield.widget = forms.Textarea(attrs=formfield.widget.attrs)
return formfield
admin.site.register(TestSuite, TestSuiteAdmin)

Answer 1

You need to make a Django Management Command. This will let you create scripts that will allow you to use all of Django's features.
And you would run this command as python3 manage.py ping_google
To create a management command,
In your apps folder, create a module called management (make a folder called management and place init.py file in it)
Inside the management folder, create a commands module (folder and init.py file)
Inside the commands folder create your ping_google.py file.
Commands are written like this,
from django.core.management.base import BaseCommand, CommandError
class Command(BaseCommand):
help = 'Desc of your command'
def handle(self, *args, **options):
# Your logic goes here
You can read more on custom django commands here