How can I load two webcontexts programmitically?
Here is the directory structure. and What I want is that I want to code the Starter.java programatically, not with the xml configuration.
Jetty Version : jetty-9.1.5.v20140505
├── java
│ └── com
│ └── embed
│ └── jetty
│ └── server
│ ├── Starter.java
| ...
├── resources
│ ├── log4j.properties
│ └── version.properties
└── webapps
├── webapps1
│ ├── WEB-INF
│ │ ├── classess
│ │ ├── lib
│ │ └── web.xml
│ ├── index.jsp
├── webapps2
│ ├── WEB-INF
│ │ ├── classess
│ │ ├── lib
│ │ └── web.xml
│ └─ index.jsp
Related
I have a django (v4.1) project with no tests defined. Before I started writing tests, I ran python manage.py test, expecting to get something like:
Found 0 test(s).
System check identified no issues (0 silenced).
....
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
But instead I got this:
Found 1 test(s).
System check identified no issues (0 silenced).
E
======================================================================
ERROR: django.config (unittest.loader._FailedTest.django.config)
----------------------------------------------------------------------
ImportError: Failed to import test module: django.config
Traceback (most recent call last):
File "/usr/local/lib/python3.11/unittest/loader.py", line 440, in _find_test_path
package = self._get_module_from_name(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/unittest/loader.py", line 350, in _get_module_from_name
__import__(name)
ModuleNotFoundError: No module named 'django.config'
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
I've checked this question but it seems more to do with a migration problem from Django 2 to 3 and how to run tests.
Clarifications:
When I say there are no tests defined, I mean that there are the auto-generated test.py files created by manage.py startapp in three applications, but I've commented out the standard import statement # from django.test import TestCase.
I tried searching for any instances of django.config using grep -r "django.config" . from the repo directory (which is above django's BASE_DIRECTORY and there are no instances of that in any of the files.
I have named the "project" directory (the one containing settings) "config" so wondered if there was come import confusion stemming from that. Did a search for all occurrences of "config":
$ grep -r "config" .
./templates/base.html: using the 'builtins' key of OPTIONS in settings (in ./config/base/templates.py).
./config/admin.py: default_site = 'config.admin.CustomAdminSite'
./config/asgi.py:ASGI config for server project.
./config/settings/base/databases.py:from config.utilities import dprint
./config/settings/base/databases.py: "NAME": str(BASE_DIR / "config/db.sqlite3"),
./config/settings/base/core.py:ROOT_URLCONF = "config.urls"
./config/settings/base/core.py:WSGI_APPLICATION = "config.wsgi.application"
./config/settings/base/authentication.py:# authall configuration settings
./config/settings/base/applications.py: # NB see ./authentication.py for social account config of allauth
./config/settings/base/applications.py: "config.admin.CustomAdminConfig", # replaces django.contrib.admin
./config/settings/__init__.py:from config.utilities import dprint
./config/settings/__init__.py:SECRETS_DIR = BASE_DIR / 'config/secrets/'
./config/wsgi.py:WSGI config for server project.
./apps/accounts/admin.py:from config.utilities import dprint
./apps/core/views.py:from config.utilities import dprint
./manage.py: os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
Project structure beneath django's BASE_DIRECTORY:
├── __init__.py
├── apps
│ ├── accounts
│ │ ├── __init__.py
│ │ ├── admin.py
│ │ ├── apps.py
│ │ ├── forms.py
│ │ ├── managers.py
│ │ ├── models.py
│ │ ├── signals.py
│ │ ├── tests.py
│ │ ├── urls.py
│ │ └── views.py
│ ├── core
│ │ ├── __init__.py
│ │ ├── admin.py
│ │ ├── apps.py
│ │ ├── models.py
│ │ ├── urls.py
│ │ └── views.py
│ └── recipes
│ ├── __init__.py
│ ├── admin.py
│ ├── apps.py
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── config
│ ├── __init__.py
│ ├── admin.py
│ ├── asgi.py
│ ├── db.sqlite3
│ ├── secrets
│ │ ├── development.secret
│ │ └── secret.template
│ ├── settings
│ │ ├── __init__.py
│ │ ├── base
│ │ └── development
│ ├── tests.py
│ ├── urls.py
│ ├── utilities.py
│ └── wsgi.py
├── manage.py
├── static
│ ├── css
│ │ └── styles.css
│ └── js
└── templates
├── allauth
│ └── account
├── base.html
├── base_with_nav.html
├── core
│ ├── index.html
│ └── item1.html
├── navbar.html
└── navbar_rhs_items.html
Hi I have a problem in getting async function return value.
this is my views.py code
Directory is djangoweb/views.py
def preprocess_log2(request,uuid):
data = get_object_or_404(Adata, uuid=uuid)
if request.method == "POST":
result = log2_transform_task.delay(data.raw_path)
test = result.get()
data.mat_data = test
data.save()
return redirect("raptorlite:home")
return render(request, 'raptorlite/detail.html',{"data":data})
this is my task code
Directory is djangoweb/bin/log2_transform.py
#shared_task
def log2_transform_task(raw:str, out_dir = None) -> str:
return out_name
sorry for not uploading my source code but I checked result it work perfectly
and it is my settings.py
CELERY_BROKER_URL = 'redis://redis:6379/0'
result_extended = True
accept_content = ['application/json']
result_serializer = 'json'
task_serializer = 'json'
timezone = 'Asia/Seoul'
CELERY_RESULT_BACKEND = 'redis://redis:6379/0'
enter image description here
when I run code if not use result.get() it works well but I want to get return value so I followed celery docs however after using get to return value suddenly that error happended..
please help me..
I followed celery docs and I read about similar error case in here but I checked that function name do not duplicate
my apps file tree is
├── djangoweb
│ ├── admin.py
│ ├── apps.py
│ ├── bin
│ │ ├── add_annotation.py
│ │ ├── adjust_qqnorm.py
│ │ ├── entrez_id2symbol.py
│ │ ├── generate_id.py
│ │ ├── gpl_probe2other.py
│ │ ├── __init__.py
│ │ ├── log2_transform.py
│ │ ├── outlier_fitting.py
│ │ ├── __pycache__
│ │ ├── renovate_entrez_gene.py
│ │ ├── rescale_1to2.py
│ │ ├── run_rescale_1to2.py
│ │ └── stats_onco_lite.py
│ ├── forms.py
│ ├── __init__.py
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
So long story short, I'm trying to use docker to take my Django project to production through Heroku. This is my first ever Django project, first time using Docker as well.
Currently I'm getting an error when I try to run my image, this is my first attempt at testing that it runs just fine. I did check that it would run with gunicorn before I put it into the docker image. That worked just fine, although my website didn't look like what it was supposed to, but that's the whole point of docker right?
As of right now my website looks just fine with the classic:
(try_django) bash-3.2$ python manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
October 15, 2020 - 18:31:48
Django version 2.2, using settings 'try_django.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Now I'm just trying to get this project deployed.
I can see that the image is created just fine:
simple-dj-docker latest 11554361b373 17 minutes ago 1.24GB
My next step is to run:
docker run -it -p 80:8888 simple-dj-docker
In return I get:
[2020-10-15 18:13:15 +0000] [6] [INFO] Starting gunicorn 20.0.4
[2020-10-15 18:13:15 +0000] [6] [INFO] Listening at: http://0.0.0.0:8888 (6)
[2020-10-15 18:13:15 +0000] [6] [INFO] Using worker: sync
[2020-10-15 18:13:15 +0000] [8] [INFO] Booting worker with pid: 8
[2020-10-15 18:13:15 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app
mod = importlib.import_module(module)
File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'try_django'
[2020-10-15 18:13:15 +0000] [8] [INFO] Worker exiting (pid: 8)
[2020-10-15 18:13:15 +0000] [6] [INFO] Shutting down: Master
[2020-10-15 18:13:15 +0000] [6] [INFO] Reason: Worker failed to boot.
My Tree:
(try_django) bash-3.2$ tree
.
├── Dockerfile
├── Pipfile
├── Pipfile.lock
├── README.md
├── app
│ ├── blog
│ │ ├── __init__.py
│ │ ├── admin.py
│ │ ├── apps.py
│ │ ├── forms.py
│ │ ├── migrations
│ │ │ ├── 0001_initial.py
│ │ │ ├── 0002_contactmessage.py
│ │ │ ├── 0003_auto_20200910_1608.py
│ │ │ ├── 0004_auto_20200911_0410.py
│ │ │ ├── 0005_homecontactmessage.py
│ │ │ ├── 0006_auto_20200922_1940.py
│ │ │ └── __init__.py
│ │ ├── models.py
│ │ ├── templates
│ │ │ └── blog
│ │ │ ├── create.html
│ │ │ ├── delete.html
│ │ │ ├── detail.html
│ │ │ ├── form.html
│ │ │ ├── list-inline.html
│ │ │ ├── list.html
│ │ │ └── update.html
│ │ ├── tests.py
│ │ ├── urls.py
│ │ └── views.py
│ ├── db.sqlite3
│ ├── manage.py
│ ├── rs_blog
│ │ ├── __init__.py
│ │ ├── admin.py
│ │ ├── apps.py
│ │ ├── forms.py
│ │ ├── migrations
│ │ │ ├── 0001_initial.py
│ │ │ ├── 0002_contactmessage.py
│ │ │ ├── 0003_auto_20200911_0432.py
│ │ │ ├── 0004_auto_20200922_1940.py
│ │ │ └── __init__.py
│ │ ├── models.py
│ │ ├── templates
│ │ │ ├── form_studio.html
│ │ │ └── rs_blog
│ │ │ ├── create.html
│ │ │ ├── delete.html
│ │ │ ├── detail.html
│ │ │ ├── form.html
│ │ │ ├── list-inline.html
│ │ │ ├── list.html
│ │ │ └── update.html
│ │ ├── tests.py
│ │ ├── urls.py
│ │ └── views.py
│ ├── searches
│ │ ├── __init__.py
│ │ ├── admin.py
│ │ ├── apps.py
│ │ ├── migrations
│ │ │ ├── 0001_initial.py
│ │ │ └── __init__.py
│ │ ├── models.py
│ │ ├── templates
│ │ │ └── searches
│ │ │ └── view.html
│ │ ├── tests.py
│ │ └── views.py
│ ├── static
│ │ ├── css
│ │ │ ├── style_basestudio.css
│ │ │ ├── style_basetech.css
│ │ │ └── style_home.css
│ │ ├── images
│ │ │ ├── 1120.jpg
│ │ │ ├── Colorado.jpg
│ │ │ ├── IMG_9646.jpg
│ │ │ ├── beer-film.jpg
│ │ │ ├── clouds-sm.jpg
│ │ │ ├── clouds.jpg
│ │ │ ├── clouds.png
│ │ │ ├── frosty-range.jpg
│ │ │ ├── logo-dark.png
│ │ │ ├── logo.png
│ │ │ ├── logo2dark.png
│ │ │ ├── moon-sm.jpg
│ │ │ ├── moon.jpg
│ │ │ ├── moon.png
│ │ │ └── stars.jpg
│ │ └── js
│ │ └── home.js
│ ├── templates
│ │ ├── about.html
│ │ ├── about_studio.html
│ │ ├── about_tech.html
│ │ ├── base.html
│ │ ├── basestudio.html
│ │ ├── basetech.html
│ │ ├── contact_us.html
│ │ ├── form.html
│ │ ├── form_studio.html
│ │ ├── form_tech.html
│ │ ├── gallery.html
│ │ ├── home.html
│ │ ├── home_navbar.html
│ │ ├── js.html
│ │ ├── reesonstudio.html
│ │ ├── reesontech.html
│ │ └── rs_navbar.html
│ └── try_django
│ ├── __init__.py
│ ├── forms.py
│ ├── settings.py
│ ├── urls.py
│ ├── views.py
│ └── wsgi.py
├── static_cdn_test
│ ├── blank
│ │ └── blank.txt
│ ├── media
│ │ └── image
│ │ ├── Desk\ Edited.jpg
│ │ ├── Desk_Edited.jpg
│ │ ├── IMG_0581.jpg
│ │ ├── IMG_3435.HEIC
│ │ ├── IMG_9305.jpg
│ │ ├── IMG_9379.jpg
│ │ ├── IMG_9427.jpg
│ │ ├── IMG_9925.JPG
│ │ ├── Screen_Shot_2020-08-11_at_11.43.03_PM.png
│ │ ├── Screen_Shot_2020-08-11_at_11.43.03_PM_CCdYJ6B.png
│ │ ├── Screen_Shot_2020-08-11_at_11.43.03_PM_CCdYJ6B_FmfxW03.png
│ │ ├── Screen_Shot_2020-08-11_at_11.43.03_PM_CCdYJ6B_St4aVdj.png
│ │ └── Screen_Shot_2020-08-11_at_11.43.14_PM.png
│ └── static
│ ├── admin
│ │ ├── css
│ │ │ ├── autocomplete.css
│ │ │ ├── base.css
│ │ │ ├── changelists.css
│ │ │ ├── dashboard.css
│ │ │ ├── fonts.css
│ │ │ ├── forms.css
│ │ │ ├── login.css
│ │ │ ├── responsive.css
│ │ │ ├── responsive_rtl.css
│ │ │ ├── rtl.css
│ │ │ ├── vendor
│ │ │ │ └── select2
│ │ │ │ ├── LICENSE-SELECT2.md
│ │ │ │ ├── select2.css
│ │ │ │ └── select2.min.css
│ │ │ └── widgets.css
│ │ ├── fonts
│ │ │ ├── LICENSE.txt
│ │ │ ├── README.txt
│ │ │ ├── Roboto-Bold-webfont.woff
│ │ │ ├── Roboto-Light-webfont.woff
│ │ │ └── Roboto-Regular-webfont.woff
│ │ ├── img
│ │ │ ├── LICENSE
│ │ │ ├── README.txt
│ │ │ ├── calendar-icons.svg
│ │ │ ├── gis
│ │ │ │ ├── move_vertex_off.svg
│ │ │ │ └── move_vertex_on.svg
│ │ │ ├── icon-addlink.svg
│ │ │ ├── icon-alert.svg
│ │ │ ├── icon-calendar.svg
│ │ │ ├── icon-changelink.svg
│ │ │ ├── icon-clock.svg
│ │ │ ├── icon-deletelink.svg
│ │ │ ├── icon-no.svg
│ │ │ ├── icon-unknown-alt.svg
│ │ │ ├── icon-unknown.svg
│ │ │ ├── icon-viewlink.svg
│ │ │ ├── icon-yes.svg
│ │ │ ├── inline-delete.svg
│ │ │ ├── search.svg
│ │ │ ├── selector-icons.svg
│ │ │ ├── sorting-icons.svg
│ │ │ ├── tooltag-add.svg
│ │ │ └── tooltag-arrowright.svg
│ │ └── js
│ │ ├── SelectBox.js
│ │ ├── SelectFilter2.js
│ │ ├── actions.js
│ │ ├── actions.min.js
│ │ ├── admin
│ │ │ ├── DateTimeShortcuts.js
│ │ │ └── RelatedObjectLookups.js
│ │ ├── autocomplete.js
│ │ ├── calendar.js
│ │ ├── cancel.js
│ │ ├── change_form.js
│ │ ├── collapse.js
│ │ ├── collapse.min.js
│ │ ├── core.js
│ │ ├── inlines.js
│ │ ├── inlines.min.js
│ │ ├── jquery.init.js
│ │ ├── popup_response.js
│ │ ├── prepopulate.js
│ │ ├── prepopulate.min.js
│ │ ├── prepopulate_init.js
│ │ ├── timeparse.js
│ │ ├── urlify.js
│ │ └── vendor
│ │ ├── jquery
│ │ │ ├── LICENSE.txt
│ │ │ ├── jquery.js
│ │ │ └── jquery.min.js
│ │ ├── select2
│ │ │ ├── LICENSE.md
│ │ │ ├── i18n
│ │ │ │ ├── ar.js
│ │ │ │ ├── az.js
│ │ │ │ ├── bg.js
│ │ │ │ ├── ca.js
│ │ │ │ ├── cs.js
│ │ │ │ ├── da.js
│ │ │ │ ├── de.js
│ │ │ │ ├── el.js
│ │ │ │ ├── en.js
│ │ │ │ ├── es.js
│ │ │ │ ├── et.js
│ │ │ │ ├── eu.js
│ │ │ │ ├── fa.js
│ │ │ │ ├── fi.js
│ │ │ │ ├── fr.js
│ │ │ │ ├── gl.js
│ │ │ │ ├── he.js
│ │ │ │ ├── hi.js
│ │ │ │ ├── hr.js
│ │ │ │ ├── hu.js
│ │ │ │ ├── id.js
│ │ │ │ ├── is.js
│ │ │ │ ├── it.js
│ │ │ │ ├── ja.js
│ │ │ │ ├── km.js
│ │ │ │ ├── ko.js
│ │ │ │ ├── lt.js
│ │ │ │ ├── lv.js
│ │ │ │ ├── mk.js
│ │ │ │ ├── ms.js
│ │ │ │ ├── nb.js
│ │ │ │ ├── nl.js
│ │ │ │ ├── pl.js
│ │ │ │ ├── pt-BR.js
│ │ │ │ ├── pt.js
│ │ │ │ ├── ro.js
│ │ │ │ ├── ru.js
│ │ │ │ ├── sk.js
│ │ │ │ ├── sr-Cyrl.js
│ │ │ │ ├── sr.js
│ │ │ │ ├── sv.js
│ │ │ │ ├── th.js
│ │ │ │ ├── tr.js
│ │ │ │ ├── uk.js
│ │ │ │ ├── vi.js
│ │ │ │ ├── zh-CN.js
│ │ │ │ └── zh-TW.js
│ │ │ ├── select2.full.js
│ │ │ └── select2.full.min.js
│ │ └── xregexp
│ │ ├── LICENSE.txt
│ │ ├── xregexp.js
│ │ └── xregexp.min.js
│ └── blank.txt
├── try_django.sublime-project
└── try_django.sublime-workspace
My Dockerfile:
# Base Image
FROM python:3.8
# create and set working directory
RUN mkdir /app
WORKDIR /app
# Add current directory code to working directory
ADD . /app/
# set default environment variables
ENV PYTHONUNBUFFERED 1
ENV LANG C.UTF-8
ENV DEBIAN_FRONTEND=noninteractive
# set project environment variables
# grab these via Python's os.environ
# these are 100% optional here
ENV PORT=8888
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
tzdata \
python3-setuptools \
python3-pip \
python3-dev \
python3-venv \
git \
&& \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# install environment dependencies
RUN pip3 install --upgrade pip
RUN pip3 install pipenv
# Install project dependencies
RUN pipenv install --skip-lock --system --dev
EXPOSE 8888
CMD gunicorn try_django.wsgi:application --bind 0.0.0.0:$PORT
Somebody please save my life, lol, I feel like I'm so close to getting this up and running as my personal website and one of the first projects among my portfolio of coding but wow the last week of trying to figure out deployment has been brutal.
I searched for this for 3 weeks but didn't find any real answer.
The main goal is to save time to test dev Chef cookbooks locally before deploying on production on AWS.
All I found is some hints using Ubuntu with Vagrant :
Chef - How to run a cookbook locally
http://pixelcog.com/blog/2015/simplify-opsworks-dev-with-packer/
Have anyone experienced to run kitchen locally with a Centos guest with a repository of Chef cookbooks with a JSON (Chef node configuration) as the node environment (like in opsworks) ?
My .kitchen.yml file and tree directory :
---
driver:
# specifies the software that manages the machine. We're using the Vagrant Test Kitchen driver
name: vagrant
provisioner:
# specifies how to run Chef. We use chef_zero because it enables you to mimic a Chef server environment on your local machine. This allows us to work with node attributes and other Chef server feature
name: chef_zero
environments_path: './env' # JSON file (node config) is not used !: env/preprod.json
client_rb:
environment: preprod
verifier:
# specifies which application to use when running automated tests. You'll learn more about automated testing in a future module.
name: inspec
platforms:
- name: centos-7
suites:
- name: default
run_list:
# list of cookbooks
- recipe[nginx::default]
attributes:
Tree without of the repository content without all files, just dir names :
(minified)
├── foobar-cookbooks
│ ├── agent_version
│ ├── apache2
│ │ └── templates
│ │ ├── default
│ ├── foobar
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── recipes
│ │ └── templates
│ │ └── default
│ ├── foobar_app_akeneo
│ │ ├── definitions
│ │ ├── metadata.rb
│ │ └── templates
│ │ └── default
│ ├── foobar_app_drupal
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── templates
│ ├── foobar_app_joomla
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── metadata.rb
│ │ └── recipes
│ ├── Config
│ ├── dependencies
│ │ ├── attributes
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── deploy
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── ebs
│ │ ├── attributes
│ │ ├── files
│ │ │ └── default
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── gem_support
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── haproxy
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── README.rdoc
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── LICENSE
│ ├── memcached
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── mod_php5_apache2
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── mysql
│ │ ├── attributes
│ │ ├── files
│ │ │ └── default
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── nginx
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── metadata.rb
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── opsworks_agent_monit
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ ├── default
│ ├── opsworks_aws_flow_ruby
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── templates
│ │ └── default
│ ├── opsworks_berkshelf
│ │ ├── attributes
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── providers
│ │ ├── recipes
│ │ └── resources
│ ├── opsworks_bundler
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── opsworks_cleanup
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── opsworks_commons
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── providers
│ │ ├── recipes
│ │ └── resources
│ ├── opsworks_custom_cookbooks
│ │ ├── attributes
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── opsworks_ecs
│ │ ├── attributes
│ │ ├── files
│ │ │ └── default
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── templates
│ │ └── default
│ ├── opsworks_ganglia
│ │ ├── attributes
│ │ ├── files
│ │ │ └── default
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── opsworks_initial_setup
│ │ ├── attributes
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── opsworks_java
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ ├── amazon
│ │ ├── default
│ ├── opsworks_nodejs
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── opsworks_rubygems
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── opsworks_shutdown
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── opsworks_stack_state_sync
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── templates
│ │ └── default
│ ├── packages
│ │ ├── attributes
│ │ ├── libraries
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── passenger_apache2
│ │ ├── attributes
│ │ ├── definitions
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── php
│ │ ├── attributes
│ │ │ └── default.rb
│ │ ├── recipes
│ │ ├── specs
│ │ └── templates
│ │ └── default
│ ├── Rakefile
│ ├── README.md
│ ├── ruby
│ │ ├── attributes
│ │ ├── metadata.rb
│ │ ├── recipes
│ │ └── specs
│ ├── scm_helper
│ ├── ssh_host_keys
│ ├── ssh_users
│ ├── test_suite
├── attributes
├── Berksfile
├── chefignore
├── definitions
├── env
├── layers.json
├── metadata.rb
├── recipes
├── spec
├── specs
├── test
"Using Test Kitchen to Manage Test Environments
Although you can test Chef code on your private server before launching it in a production environment, node that Chef will make changes to the configuration of the server when you run the code.
A smart way to go about testing Chef code during development is to set up a sandbox environment that closely resembles a production environment. This will give you a safe place to test your Chef recipes. Chef comes with Test Kitchen, which helps you create a sandbox environment for testing. Test Kitchen utilizes Vagrant and Virtual Box to get its job done.
Test Kitchen runs on Vagrant, and you create your sandbox environment on top of Test Kitchen. Test Kitchen is installed as part of the Chef Development Kit, and you need to install it separately if you’re using Chef Client.
In order to create a virtual environment using Test Kitchen, you use the kitchen create command:
$ kitchen create default-centos65
This example shows how to create a virtual environment running CentOs. This command downloads the Vagrant base box and configure and boot the VM instance. Test Kitchen will pull base boxes that Chef Software makes available on the Internet via VagrantCloud The CentOS instance created by the last command will set a up a barebones CentOS installation with just enough stuff to let Chef run.
You can login to the CentOS VM by doing this:
$ kitchen login default-centos65
Last login: Fri May 28 10:41:48 2016
from 10.0.1.1
Welcome to your Packer-built virtual machine.
Run all your test Chef code in this Test Kitchen supported sandbox environment.
Test Kitchen uses YAML file format for its configuration files. YAML files work with two types of data – key-value pairs and lists." - Alapati, S. (March 2018). Modern Linux Administration.
I recently purchased an HTML/CSS/Js admin template based on the Bootstrap framework. It basically covered all my needs for an MVP and my plan was to customize it a bit and then plug my already developed back-end through flask.
I am quite inexperienced in this field, so I was quite impressed by the automatic workflow that was used by this admin template.
The basic structure is the following one:
root/
├── dist/
│ └── html/
│ ├── assets/
│ └── all_pages.html
├── grunt/
│ └── tasks/
├── node_modules/
├── src/
│ ├── assets/
│ ├── html/
│ ├── js/
│ └── sass/
├── Gruntfile.js
└── package.json
Thanks to grunt tasks and npm management, handling assets is very easy, after an npm install you can handle everything with grunt.
The sass are compiled in a css style for production and all code gets minified and copied to the dist folder depending on the settings.
You can easily develop on the src path, and use the grunt task "server" to both watch for changes and directly display them before sending everything to production folder "dist".
My problems arise when I try to keep this behavior with a flask application interacting with it.
My flask application uses this structure:
root/
├── __init__.py
├── templates/
│ ├── layout.html
│ └── bp1/
│ │ ├── layout.html
│ │ └── other_pages.html
│ └── bp2/
│ ├── layout.html
│ └── other_pages.html
├── views/
│ ├── __init__.py
│ ├── bp1.py.py
│ └── bp2.py.py
├── static/
│ ├── css/
│ ├── js/
│ └── img/
├── Dockerfile
└── requirements.txt
Basically, there is no difference between development and production version, and the web-app gets deployed through its docker image.
My question here is, how on earth should I approach the merging of these two guys? How to have a flask project with src-dist separation and a workflow similar to the one I described above?
I would like to keep all the good features (I managed to notice with my skills) of the admin template and have something with:
src and dist folders separation...so that all sass items, unused/discarded js code and html pages are only in the development "src" folder and will not be used in production
grunt automation for compiling sass, cleaning lib directories, watching for changes, npmcopy (to install packages with npm and move only required files to production), notifications, minification, etc...
Docker image based deployment that is based only on the "dist-generated" resource and ignores the "src-development" stuff.
Alright, I came up with a setup that works pretty neatly and that I think is worth sharing for anyone else stuck or doubtful when in a similar scenario.
Structure
root/
├── src/
│ ├── __init__.py
│ ├── models.py
│ ├── database.py
│ ├── static/
│ │ ├── css/
│ │ │ └── app.css
│ │ ├── js/
│ │ ├── img
│ │ └── lib
│ ├── templates/
│ │ ├── layout.html
│ │ ├── bp1/
│ │ │ ├── layout.html
│ │ │ └── other_pages.html
│ │ └── bp2/
│ │ ├── layout.html
│ │ └── other_pages.html
│ ├── views/
│ │ ├── __init__.py
│ │ ├── bp1.py
│ │ └── bp2.py
│ └── sass/
├── dist/
│ ├── __init__.py
│ ├── models.py
│ ├── database.py
│ ├── static/
│ │ ├── css/
│ │ │ └── app.css
│ │ ├── js/
│ │ ├── img
│ │ └── lib
│ ├── templates/
│ │ ├── layout.html
│ │ ├── bp1/
│ │ │ ├── layout.html
│ │ │ └── other_pages.html
│ │ └── bp2/
│ │ ├── layout.html
│ │ └── other_pages.html
│ └── views/
│ ├── __init__.py
│ ├── bp1.py
│ └── bp2.py
├── templates/
│ ├── layout.html
│ └── bp1/
│ │ ├── layout.html
│ │ └── other_pages.html
│ └── bp2/
│ ├── layout.html
│ └── other_pages.html
├── views/
│ ├── __init__.py
│ ├── bp1.py.py
│ └── bp2.py.py
├── static/
│ ├── css/
│ ├── js/
│ └── img/
├── instance/
│ └── flask.cfg
├── grunt/
│ └── tasks/
├── static/
├── node_modules/
├── venv/
├── Gruntfile.js
├── package.json
├── Dockerfile
├── .gitignore
└── requirements.txt
Workflow
packages are installed with npm and the package.json (node_modules gets generated).
a python virtualenv is configured using the 'requirements.txt' and linked to 'venv'.
a grunt tasks is called and uses npmcopy to move only the required files to src/static/lib that gets used by flasks' templates as: static/lib in order to keep src-dist compatibility.
a grunt task is able to compile sass parts and create 'app.css' within static/css.
several other grunt tasks do other useful things like minification.
grunt's default task performs concurrently a 'watch task' and launches flask run to let the development continue smoothly (more on this later).
grunt dist creates in the dist folder a production-ready flask project with all packages, styles and pages developed in the previous steps.
Grunt's flask task
This simple piece of code manages to launch a flask server locally to start development.
// Launch flask's server
grunt.registerTask('flask', 'Run flask server.', function() {
var spawn = require('child_process').spawn;
grunt.log.writeln('Starting Flask.');
var PIPE = {
stdio: 'inherit',
env: {
FLASK_APP: './src/__init__.py:create_app()',
FLASK_ENV: 'development',
LC_ALL: 'C.UTF-8',
LANG: 'C.UTF-8'
}
};
// more on venv later
spawn('venv/bin/flask', ['run'], PIPE);
});
Flask setup for development
In order for the flask run command to work properly in development mode, the following are configured:
venv: symbolic link to the python virtualenv used for the project.
instance/flask.cfg: flask instance folder
Gitignore
Other than the whole 'dist' folder, these are excluded from VCS:
venv;
instance folder;
lib folder within the src one;
node_modules;
Conclusion
This setup is pretty handy and is quite easy to share. Local, easy, configurations let everything work neatly for development.
Production code can be generated and then deployed/configured quickly depending on the strategies (k8s, server deployments, ...).