Plone 4.3.x - grokcore.view - UserWarning: Found the following unassociated template after configuration - templates

On a vanilla Plone 4.3.3 site (Unified Installer on Ubuntu 14.04.1LTS), and after updating buildout.cfg with the zopeskel and paster boiler plate stuff and running buildout, I successfully created a dexterity package in my src folder:
$ cd src
$ ../bin/zopeskel dexterity my.package
After updating buildout.cfg (adding my.package to the eggs section and src/my.package to the develop section) and running buildout, I added content to my new package:
$ cd my.package
$ ../../bin/paster addcontent dexterity_content
I called the new content type mytype, resulting in mytype.py, a templates folder called mytype_templates, etc.
Restarting Plone and.... so far, so good....
Then I add templates to the mytype_templates folder:
add.pt
edit.pt
view.pt
In the mytype.py file I added all the necessary imports, schema definition
Class Imytype(form.Schema, IImageScaleTraversable):
....
....
, etc, etc, and obviously also the view, add and edit classes:
class View(dexterity.DisplayForm):
grok.context(Imytype)
grok.require('zope2.View')
# Disable turn fieldsets to tabs behavior
enable_form_tabbing = False
def update(self):
super(View, self).update()
class Add(dexterity.AddForm):
grok.name('my.package.mytype')
# Disable turn fieldsets to tabs behavior
enable_form_tabbing = False
def __init__(self, context, request):
super(Add, self).__init__(context, request)
......
......
class Edit(dexterity.EditForm):
grok.context(Imytype)
# Disable turn fieldsets to tabs behavior
enable_form_tabbing = False
def update(self):
super(Edit, self).update()
......
......
When I restart my Plone site in foreground mode, I get the following messages:
2015-02-06 12:52:41 INFO ZServer HTTP server started at Fri Feb 6 12:52:41 2015
Hostname: 0.0.0.0
Port: 8080
/home/Plone434_site/buildout-cache/eggs/grokcore.view-2.8-py2.7.egg/grokcore/view/templatereg.py:261: UserWarning: Found the following unassociated template after configuration: /home/Plone434_site/zinstance/src/my.package/my/package/mytype_templates/edit.pt
warnings.warn(msg, UserWarning, 1)
/home/Plone434_site/buildout-cache/eggs/grokcore.view-2.8-py2.7.egg/grokcore/view/templatereg.py:261: UserWarning: Found the following unassociated template after configuration: /home/Plone434_site/zinstance/src/my.package/my/package/mytype_templates/add.pt
warnings.warn(msg, UserWarning, 1)
2015-02-06 12:52:46 INFO Zope Ready to handle requests
Seemingly grok successfully picks up the view.pt, but not the add.pt and edit.pt
This is confirmed by customizing the templates. Changes to view.pt renders fine, changes to add.pt and edit.pt have no results. Plone falls back on the default dexterity templates, as the add.pt and edit.pt are not grokked.
I found a work-around by adding the following:
....
from Products.Five.browser.pagetemplatefile import ViewPageTemplateFile
....
and to the Add class:
template = ViewPageTemplateFile('mytype_templates/add.pt')
and to the Edit class:
template = ViewPageTemplateFile('mytype_templates/edit.pt')
Obviously the error messages as listed above are still there, but at least it works and I can customize the add.pt and edit.pt.
Although I can live with this workaround, I am wondering why only the view.pt is grokked and not the add.pt and edit.pt.
Please notice that this (weird?) behavior was also duplicated using Plone 4.3.1, 4.3.2, 4.3.3 and 4.3.4
Any suggestions?

You have to declare the name, context, layer and schema of the views; use something like this (note the grok.layer method which maybe you're missing):
class AddForm(dexterity.AddForm):
grok.name('my.package.mytype')
grok.layer(Imylayer)
grok.context(Imytype)
schema = Imytype
def update(self):
super(AddForm, self).update()
...
class EditForm(dexterity.EditForm):
grok.context(Imytype)
grok.layer(Imylayer)
schema = Imytype
def update(self):
super(EditForm, self).update()
...
Alternatively you may skip the use of Grok at all and register everything via ZCML.
An example of this can be found in the collective.nitf package. There's a branch using Grok and a pull request removing it.

Related

Django, Apache2 on Google Kubernetes Engine writing Opencensus Traces to Stackdriver Trace

I have a Django web app served from Apache2 with mod_wsgi in docker containers running on a Kubernetes cluster in Google Cloud Platform, protected by Identity-Aware Proxy. Everything is working great, but I want to send GCP Stackdriver traces for all requests without writing one for each view in my project. I found middleware to handle this, using Opencensus. I went through this documentation, and was able to manually generate traces that exported to Stackdriver Trace in my project by specifying the StackdriverExporter and passing the project_id parameter as the Google Cloud Platform Project Number for my project.
Now to make this automatic for ALL requests, I followed the instructions to set up the middleware. In settings.py, I added the module to INSTALLED_APPS, MIDDLEWARE, and set up the OPENCENSUS_TRACE dictionary of options. I also added the OPENCENSUS_TRACE_PARAMS. This works great with the default exporter 'opencensus.trace.exporters.print_exporter.PrintExporter', as I can see the Trace and Span information, including Trace ID and all details in my Apache2 web server logs. However, I want to send these to my Stackdriver Trace processor for analysis.
I tried setting the EXPORTER parameter to opencensus.trace.exporters.stackdriver_exporter.StackdriverExporter, which works when run manually from the shell, as long as you supply the project number.
When it is set up to use StackdriverExporter, the web page will not respond load, the health check starts to fail, and ultimately the web page comes back with a 502 error, stating I should try again in 30 seconds (I believe the Identity-Aware Proxy is generating this error, once it detects the failed health check), but the server generates no errors, and there are no logs in access or errors for Apache2.
There is another dictionary in settings.py named OPENCENSUS_TRACE_PARAMS, which I presume is needed to determine which project number the exporter should be using. The example has GCP_EXPORTER_PROJECT set as None, and SERVICE_NAME set as 'my_service'.
What options do I need to set to get the exporter to send back to Stackdriver instead of printing to logs? Do you have any idea about how I can set this up?
settings.py
MIDDLEWARE = (
...
'opencensus.trace.ext.django.middleware.OpencensusMiddleware',
)
INSTALLED_APPS = (
...
'opencensus.trace.ext.django',
)
OPENCENSUS_TRACE = {
'SAMPLER': 'opencensus.trace.samplers.probability.ProbabilitySampler',
'EXPORTER': 'opencensus.trace.exporters.stackdriver_exporter.StackdriverExporter', # This one just makes the server hang with no response or error and kills the health check.
'PROPAGATOR': 'opencensus.trace.propagation.google_cloud_format.GoogleCloudFormatPropagator',
# 'EXPORTER': 'opencensus.trace.exporters.print_exporter.PrintExporter', # This one works to print the Trace and Span with IDs and details in the logs.
}
OPENCENSUS_TRACE_PARAMS = {
'BLACKLIST_PATHS': ['/health'],
'GCP_EXPORTER_PROJECT': 'my_project_number', # Should this be None like the example, or Project ID, or Project Number?
'SAMPLING_RATE': 0.5,
'SERVICE_NAME': 'my_service', # Not sure if this is my app name or some other service name.
'ZIPKIN_EXPORTER_HOST_NAME': 'localhost', # Are the following even necessary, or are they causing a failure that is not detected by Apache2?
'ZIPKIN_EXPORTER_PORT': 9411,
'ZIPKIN_EXPORTER_PROTOCOL': 'http',
'JAEGER_EXPORTER_HOST_NAME': None,
'JAEGER_EXPORTER_PORT': None,
'JAEGER_EXPORTER_AGENT_HOST_NAME': 'localhost',
'JAEGER_EXPORTER_AGENT_PORT': 6831
}
Here's an example (I prettified the format for readability) of the Apache2 log when it is set to use the PrintExporter:
[Fri Feb 08 09:00:32.427575 2019]
[wsgi:error]
[pid 1097:tid 139801302882048]
[client 10.48.0.1:43988]
[SpanData(
name='services.views.my_view',
context=SpanContext(
trace_id=e882f23e49e34fc09df621867d753532,
span_id=None,
trace_options=TraceOptions(enabled=True),
tracestate=None
),
span_id='bcbe7b96906a482a',
parent_span_id=None,
attributes={
'http.status_code': '200',
'http.method': 'GET',
'http.url': '/',
'django.user.name': ''
},
start_time='2019-02-08T17:00:29.845733Z',
end_time='2019-02-08T17:00:32.427455Z',
child_span_count=0,
stack_trace=None,
time_events=[],
links=[],
status=None,
same_process_as_parent_span=None,
span_kind=1
)]
Thanks in advance for any tips, assistance, or troubleshooting advice!
Edit 2019-02-08 6:56 PM UTC:
I found this in the middleware:
# Initialize the exporter
transport = convert_to_import(settings.params.get(TRANSPORT))
if self._exporter.__name__ == 'GoogleCloudExporter':
_project_id = settings.params.get(GCP_EXPORTER_PROJECT, None)
self.exporter = self._exporter(
project_id=_project_id,
transport=transport)
elif self._exporter.__name__ == 'ZipkinExporter':
_service_name = self._get_service_name(settings.params)
_zipkin_host_name = settings.params.get(
ZIPKIN_EXPORTER_HOST_NAME, 'localhost')
_zipkin_port = settings.params.get(
ZIPKIN_EXPORTER_PORT, 9411)
_zipkin_protocol = settings.params.get(
ZIPKIN_EXPORTER_PROTOCOL, 'http')
self.exporter = self._exporter(
service_name=_service_name,
host_name=_zipkin_host_name,
port=_zipkin_port,
protocol=_zipkin_protocol,
transport=transport)
elif self._exporter.__name__ == 'TraceExporter':
_service_name = self._get_service_name(settings.params)
_endpoint = settings.params.get(
OCAGENT_TRACE_EXPORTER_ENDPOINT, None)
self.exporter = self._exporter(
service_name=_service_name,
endpoint=_endpoint,
transport=transport)
elif self._exporter.__name__ == 'JaegerExporter':
_service_name = self._get_service_name(settings.params)
self.exporter = self._exporter(
service_name=_service_name,
transport=transport)
else:
self.exporter = self._exporter(transport=transport)
The exporter is now named StackdriverExporter, instead of GoogleCloudExporter. I set up a class in my app named GoogleCloudExporter that inherits StackdriverExporter, and updated my settings.py to use GoogleCloudExporter, but it didn't seem to work, I wonder if there is other code referencing these old naming schemes, possibly for the transport. I'm searching the source code for clues... This at least tells me I can get rid of the ZIPKIN and JAEGER param options, as this is determined on the EXPORTER param.
Edit 2019-02-08 11:58 PM UTC:
I scrapped Apache2 to isolate the problem and just set my docker image to use Django's built in webserver CMD ["python", "/path/to/manage.py", "runserver", "0.0.0.0:80"] and it works! When I go to the site, it writes traces to Stackdriver Trace for each request, the Span name is the module and method being executed.
Somehow Apache2 is not being allowed to send these, but I can do so from the shell when running as root. I'm adding Apache2 and mod-wsgi tags to the question, because I have a funny feeling this has to do with forking child processes in Apache2 and mod-WSGI. Would it be the child process being unable to be created as apache2's child process is sandboxed, or could this be a permissions thing? It seems strange, because it is just calling python modules, no external system OS binaries, that I am aware of. Any other ideas would be greatly appreciated!
I had this problem while using gunicorn with gevent as the worker class. To resolve and get cloud traces working the solution was to monkey patch grpc like so
from gevent import monkey
monkey.patch_all()
import grpc.experimental.gevent as grpc_gevent
grpc_gevent.init_gevent()
See https://github.com/grpc/grpc/issues/4629#issuecomment-376962677

Sphinx builtin themes not found

In my conf.py it says I should see the documentation for a list of builtin-themes. Now my first google hit leads me to http://www.sphinx-doc.org/en/stable/theming.html#builtin-themes . There are a bunch of themes in there which my sphinx does not know about. For example 'classic'.
in conf.py
html_theme = 'classic'
On my shell I do: sphinx-build -b html source build
writing output... [100%] index
Exception occurred:
File "~\appdata\local\programs\python\python35\lib\site-packages\sphinx\jinja2glue.py", line 200, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: about.html
sphinx version: 1.6.3.
I had same problem because I used alabaster theme before. There are a few lines in conf.py that are needed for alabaster and break other themes. You need to comment them out.
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
#
# This is required for the alabaster theme
# refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars
# html_sidebars = {
# '**': [
# 'about.html',
# 'navigation.html',
# 'relations.html', # needs 'show_related': True theme option to display
# 'searchbox.html',
# 'donate.html',
# ]
# }
It is an issue after changing alabaster theme to another one that doesn't use about.html. Laurent said the workaround.
Issue:
https://github.com/sphinx-doc/sphinx/issues/3987

How to customize TRAC plugin template python file

I am currently modifying our TRAC instance to Bootstrap 3.1. However, some templating needs to be done on the .py files. I only know how to customize .html files... just add classes, customize DOM structure a little bit then put it in templates folder of our TRAC instance.
NOW WHAT ABOUT customizing .py files from plugins? I tried putting them in templates folder but nothing happened.
I had no experience with Python, but it's easy just to hack around and add a bootstrap class e.g adding "col-sm-2 control-label" in a label in milestone.py
def __edit_project(self, data, req):
milestone = data.get('milestone').name
all_projects = self.__SmpModel.get_all_projects_filtered_by_conditions(req)
id_project_milestone = self.__SmpModel.get_id_project_milestone(milestone)
if id_project_milestone != None:
id_project_selected = id_project_milestone[0]
else:
id_project_selected = None
return tag.div(
tag.label(
class_="col-sm-2 control-label",
'Project',
tag.br(),
tag.select(
tag.option(),
[tag.option(row[1], selected=(id_project_selected == row[0] or None), value=row[0]) for row in sorted(all_projects, key=itemgetter(1))],
name="project")
),
class_="field")
Compiling the plugin again worked for me. After adding bootstrap classes on specific .py files, here are the steps/commands I did:
In our TRAC environment plugins directory where specific setup.py of the plugin I'm editing is located, build the .egg file e.g
tracproject/plugins_source/sampleplugin: python setup.py bdist_egg
Then I renamed the plugin's original .egg file in the plugins directory e.g
tracproject/plugins/sampleplugin/: mv sampleplugin.egg sampleplugin.egg.old
After that, I copied the newly .egg file generated to the plugins directory e.g
tracproject/plugins_source/sampleplugin/dist: mv sampleplugin.egg ../../../plugins/
Lastly, I restarted our server e.g (however, there were cases, no restart was needed since changes were instantly reflected)
sudo service apache2 restart
Thanks #falkb! I see that you're the author of SimpleMultiProject plugin I was trying to put bootstrap classes. :)
Here's a snippet of simplemultiprojectplugin milestone.py where I added styling
def __edit_project(self, data, req):
milestone = data.get('milestone').name
all_projects = self.__SmpModel.get_all_projects_filtered_by_conditions(req)
id_project_milestone = self.__SmpModel.get_id_project_milestone(milestone)
if id_project_milestone != None:
id_project_selected = id_project_milestone[0]
else:
id_project_selected = None
return tag.div(
tag.label('Project', class_="control-label col-sm-2"),
tag.div(
tag.select(
tag.option(),
[tag.option(row[1], selected=(id_project_selected == row[0] or None), value=row[0]) for row in sorted(all_projects, key=itemgetter(1))],
name="project",
class_="form-control"),
class_="col-sm-5"),
class_="form-group")

Sphinx documentation and autodoc-skip-member

I'm building my sphinx doc for a django project the following way:
sphinx-apidoc app -o docs/source/app --force
Now it includes all of the South migrations which I don't want to have in my documentation. I now tried to exclude them the following way:
conf.py:
def skip_migrations(app, what, name, obj, skip, options):
return skip or (what == 'module' and name.find('Migration') != -1)\
or str(obj).find('migrations') != -1
def setup(app):
app.connect('autodoc-skip-member', skip_migrations)
Now they aren't documented anymore, but are still listed under modules. How can I exclude them?
You may exclude the rst files created for the migrations by adding them to the exclude_pattern in your conf.py file:
exclude_patterns = ["**/*.migrations.rst",]
Just avoid generating the .rst files with sphinx-apidoc in the first place:
sphinx-apidoc app -o docs/source/app --force */migrations/*
Patterns added after the module name are understood as excluded paths.

How do I ignore static files of a particular app only with collectstatic?

Just to get this out of the way, if at all possible, I'd like to do this without nesting them all inside a directory with the app's name inside the app's static folder, it feels redundant. If it's the only way then such is life.
I am using:
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
and:
STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
which compiles the JS and SASS when running collectstatic. These are both located in the APP_ROOT/static/ directory.
The only 'problem' is that it brings along all the source sass and js files alongside it. At some point I'm going to be pushing this all to S3 and I'd like to avoid that if possible.
I found that if you run:
python manage.py collectstatic -i sass -i js
It still compiles the JS and CSS files I specified, while leaving the rest of the 'source' files out. Unfortunately, it also ignores every js file in /admin/ as it matches /admin/js/ etc. I don't even know if that's likely to be a problem for this particular project, but I can foresee in the future other apps where I definitely will want to include static js/css kept in an app.
What I'd like to be able to do is something like:
python manage.py collectstatic -i app_name/sass -i app_name/js
And as I mentioned at the top the easy solution is just to prefix my static files in the folder with app_name/, much like how django.contrib.admin does it. At this point however, you end up with a directory structure of PROJECT_ROOT/app_name/static/app_name/[js|sass|img|data]/ and I think it should be possible to avoid the redundancy.
Then again, maybe it's the best option, so as to guarantee avoiding conflict with other apps?
I've looked into writing custom storages and finders, and I think it's possible to roll my own. I wanted to check here though first, to see if this is a problem someone else has solved, or, to get a reality check if the overwhelming response is to just add the prefix directory.
If I was to roll my own, the path I think I would take would be extending django.contrib.staticfiles.finders.AppDirectoriesFinder and overriding list(). I'm not yet positive this approach would work, I need to more trace how things progress from the collectstatic management command, so if anyone has done this or something simlilar before, or knows why it will/won't work, any help is appreciated.
Thanks!
I managed to solve this by subclassing Django finders like this:
PYTHON 2.X
from django.contrib.staticfiles import finders
from django.conf import settings
def add_ignores(ignore_patterns):
ignore = settings.STATICFILES_FINDERS_IGNORE
if ignore:
if ignore_patterns:
ignore_patterns.extend(ignore)
else:
ignore_patterns = ignore
return ignore_patterns
class FileSystemFinderIgnore(finders.FileSystemFinder):
def list(self, ignore_patterns):
return super(FileSystemFinderIgnore, self).list(add_ignores(ignore_patterns))
class AppDirectoriesFinderIgnore(finders.AppDirectoriesFinder):
def list(self, ignore_patterns):
return super(AppDirectoriesFinderIgnore, self).list(add_ignores(ignore_patterns))
class DefaultStorageFinderIgnore(finders.DefaultStorageFinder):
def list(self, ignore_patterns):
return super(DefaultStorageFinderIgnore, self).list(add_ignores(ignore_patterns))
PYTHON 3.X
from django.contrib.staticfiles import finders
from django.conf import settings
def add_ignores(ignore_patterns):
ignore = settings.STATICFILES_FINDERS_IGNORE
if ignore:
if ignore_patterns:
ignore_patterns.extend(ignore)
else:
ignore_patterns = ignore
return ignore_patterns
class FileSystemFinderIgnore(finders.FileSystemFinder):
def list(self, ignore_patterns):
return super().list(add_ignores(ignore_patterns))
class AppDirectoriesFinderIgnore(finders.AppDirectoriesFinder):
def list(self, ignore_patterns):
return super().list(add_ignores(ignore_patterns))
class DefaultStorageFinderIgnore(finders.DefaultStorageFinder):
def list(self, ignore_patterns):
return super().list(add_ignores(ignore_patterns))
and adding this to my settings:
STATICFILES_FINDERS_IGNORE = [
'*.scss',
'*.js',
]
I'm new to django-pipeline, but I believe it now has pipeline.finders.FileSystemFinder and pipeline.finders.AppDirectoriesFinder to do exactly this.
See the section "If you want to exclude Pipelinable content from your collected static files" on https://django-pipeline.readthedocs.org/en/latest/storages.html.
Also, from the source code of 1.5:
class AppDirectoriesFinder(PatternFilterMixin, DjangoAppDirectoriesFinder):
"""
Like AppDirectoriesFinder, but doesn't return any additional ignored
patterns.
This allows us to concentrate/compress our components without dragging
the raw versions in via collectstatic.
"""
Note that at the time of writing, this results in empty compressed/minified content when DEBUG==True, see https://github.com/cyberdelia/django-pipeline/issues/418. I assume this will be fixed in future versions of django-pipeline.