In our Django project we are using Gulp to compile our assets, then UglifyJS to minify them. During this whole process we are generating sourcemaps, which appear to be working correctly.
The problem comes when we use the Django static template tag to include our minified files. Say we have a minified JS file called ourapp.min.js. In our template we would write:
<script src="{% static 'ourapp.min.js %}"></script>
which would be compiled into something like:
<script src="/ourstaticroot/ourapp.min.0123456789ab.js"></script>
(where 0123456789ab is a hash of the file contents)
The problem now is that, although the file has been renamed, our sourcemap still points to the old filename, so suddenly becomes invalid. If we then need to debug this page (say, using Sentry) it cannot find the source file and we are left to debug the uglified file instead, which becomes much more of a task.
Does anyone know of a good way to get around this? We would like to continue using Gulp for our assets, and also continue using the hashed filenames, as this prevents issues caused by caching of stale asset files.
it does not really matter, since the original files are being kept. So if your file points to a map without a hash it should be served by Django. Of course you need to be careful with long term expiration headers. Should you be using whitenoise are are fine, since they handle expiration properly and long term expiration headers are only set on hashed files.
Cheers
-Joe
I see two options available to you. It's not clear where exactly the cache busting suffix gets added to your files, but at that point you could:
Use some string processing, to manipulate the assets and set their sourceMapUrl with a url that is generated by your system and includes the expected suffix //# sourceMappingURL=/path/to/file.js.<suffix>.map. This could be a simple bash command in a gulp step (if that's where it happens)
Alternatively, browsers will also accept an http header SourceMap: /path/to/file.js.map which you could instruct your asset server to set for files with headers. This may be more difficult depending on your infrastructure.
Related
When I run python manage.py collectstatic, it makes a copy of each image, JavaScript, and CSS file with a hash in the filename:
Post-processed 'css/theme.css' as 'css/theme.afeb1fc222a9.css'
Post-processed 'css/custom.css' as 'css/custom.585e1b29ff9a.css'
...
I'm assuming this is just a way of making a versioned filename for better caching; the client or CDN can be told to cache this file indefinitely, because if I make a change, the hash will differ, and I'll just reference the new version by the new name.
However, I'm not clear on how I'm supposed to reference this URL. The documentation on serving static files just says,
In your templates, either hardcode the url like /static/my_app/example.jpg or, preferably, use the static template tag to build the URL for the given relative path by using the configured STATICFILES_STORAGE storage (this makes it much easier when you want to switch to a content delivery network (CDN) for serving static files).
I went through my templates and dutifully switched every static resource (including the CSS files) from a hardcoded URL to a {% static "..." %} template tag, assuming it would map to the versioned filename where appropriate. But it doesn't.
I'm also using WhiteNoise for serving the resources, and I'm not entirely sure how it affects things, but it also says,
Want forever-cacheable files and compression support? Just add this to your settings.py: STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
But I have that in my settings file and it also doesn't seem to do anything with these versioned filenames.
If DEBUG is True then the static url will be 'css/theme.css' instead of
'css/theme.afeb1fc222a9.css'
In a Django project, I would like to include a static file in a template. The following is the the project structure:
proj/
main/
static/
main/
js/
main.js
news/
templates/
news/
news.html
In news.html, I would like to include the main.js with the following hypothetical format:
{% load staticfiles %}
...
{% include {% static 'main/js/main.js' %} %}
How do I do it?
The include directive only searches file in template directories. What you could do (although I wouldn't) is to change your settings to also include the static files:
TEMPLATE_DIRS = [
...
"path/to/your/statics",
]
I have a couple of ideas.
The easiest thing to do would be to ensure you have the filesystem loader enabled, and you include the relevant directory containing static files in TEMPLATES[0]['DIRS'] (previously TEMPLATE_DIRS). The caveats are that it will not pick up static files contained inside your installed applications automatically, unless you list them in DIRS, too. Another option would be to use STATIC_ROOT, but that will only work if you run collectstatic every time you change the static file, which you normally don't have to do in development. Another pitfall is that it will only work with local static storage, not if you use any CDN or otherwise host your static files on a different host / network / whatever.
The other thing you can do is to write your own template loader which will use the static files API to load statics as templates. This is a bit more involved, but it should be able to access static files regardless of how they are stored and served.
Should you choose either option, you still have to be careful. For instance, you'll have to ensure that the static files you're including as templates are safe to include into HTML or whatever other context you're using them in. There will be no escaping going on.
In case you're trying to optimize the number of requests the client has to make, there are better ways. You're most likely better off implementing some pipeline that will pre-process and minify your static files. Including any significant chunk of CSS / JS into the HTML will make it impossible for clients to take advantage of caching, forcing them to re-download the same static content over and over again, likely impacting the performance negatively.
Edit: Since you want to just include some dynamic JavaScript variables, that's not a static file at all. What you really want is to create a template containing JavaScript code, and not mess with handling static files as templates. There is no rule that would say every javascript needs to be a static file. If it's dynamic, it's not a static file.
For the record, this was a typical instance of the XY problem. Problem X was dynamically assigning values to JavaScript variables when rendering templates. You came up with including static files in templates as a potential solution, but then got stuck with problem Y, which was that you didn't know how to include static files in templates.
I'm making a "this day in history" sort of site in gh-pages, using javascript to pull the day's entry for the front page and a collection to store all the other entries indexed by date.
The entries are text files.
I've made markdown files as stubs to pull in the text files. I don't want to replicate the text files if possible, because then any typos I would have to remember to fix in two places.
As far as I can see, there are two ways to include the text files in the template:
{% include date.txt %} which requires the txt files to be in the _includes directory, thus not generated into the site and not available to the javascript on the front page
{% include_relative date.txt %} which requires the txt file to be in the collection folder, which is also not generated unless it has a yaml header, in which case it would be difficult to extract the text from the generated html.
Is there another way I'm missing for jekyll to include plain text files without them having to be in special _folders?
I'm using github pages, so plugins are out.
I think there is no other way to include text files through liquid. It is part of the way it separates published files from the pieces that go together to make the files.
The way forward is to adapt the javascript to read the text from a raw blob from the github repository.
But using an http request to raw.githubusercontent.com gives a Cross Origin Resource Sharing error.
So next way is to make a new collection with a new layout to output the input files as they are.
I'm serving a Django app on top of Apache, and after my recent deployments, I noticed that a lot of users were being served the new version of the main HTML file, but they were still picking up the older JS files from their Cache.
I tried stuff like patch_cache_control(response, no_cache=True, no_store=True, must_revalidate=True), but this doesn't seem to work for static files.
Is there a way to always force every static to be re-fetched from the server when a page is accessed?
You can do a lot of stuff with your headers.
Or you could use something like {% static 'js/jquery.js' %}?v={{ settings.CURRENT_VERSION }}, this way the browser would think that you are using a completely new file, because the name of the file would be something like jquery.js?v=1. I would not really recommend doing this, but it is a really quick hack.
I've got a Django project spinning up where it would be great to have the UI in lots of color schemes based on the users' school colors. I have this fantasy of having a base variables.less file along with a bunch of other .less files that compile into style.css.
But once a user sets their school colors it generates a blue.variables.less file (if they've selected the blue preset) or school123.variables.less file (if they got all fancy and used the color picker to make their own color scheme) and then compiles everything to blue.style.css or school123.style.css and then that's the .css file we load when we serve the page.
I can imagine lots of ways for this to fall down. Like how do I reprocess all those files when I push an update to forms.less or layout.less.
Is there a better way to do this? I Googled my fingers raw but haven't found anyone attempting this madness.
There are quite a few ways to accomplish your goal of being able to have user specific color schemes, but they each have their advantages / disadvantages.
I'm assuming you are using some framework like Bootstrap with the files that you name.
Option 1: Inline CSS for color-specific styles (Preferred)
This is my preferred option due to performance and simplicity. You can store each of the customized colors for each user, or even creating a model so you can reuse colors that represent a specific school. It's stored in the database and can scale to an very large number of color schemes without generating a lot of very similar files.
Create a snippet in your template code that has any style that uses the color variable.
base.html
{% include "color-snippet.css" with main-color="{{ user's main color }}" alt-color="{{ user's alt color }}" %}
color-snippet.css (note this file will be in your templates directory as it's being handled by your template engine
<style>
.some-style {
color: {{ main-color }} !important;
}
</style>
So the big downside to this is you'll need to customize Bootstrap beyond the variables.less. You'll need to grep through the less files to find all the classes that would be generated, and copy the style to your snippet in css and not less. It'll take some investment up front and work when you want to upgrade to a newer Bootstrap, but it'll allow you to separate the color part of the styles to be derived dynamically at runtime.
I prefer this approach since you don't have to deal with compilation of less outside your collectstatic step.
Option 2: Client side compilation of LESS
You can have Django serve a file that is dynamically created and returns the variables you want. Then you can have less.js compile it on the client.
This would involve adding to your base template a url path that is handled by Django that isn't part of your static path (e.g. /style/variables), creating a handler in views and then returning text content that would be your less file variables.
Option 3: Server side compilation of LESS
I use Django Pipeline to do my server side compilation of less to css. It takes some setup to get working with your Django application. In development mode, Django Pipeline will compile on every request the associated less files into CSS files. In production mode, it'll point to the appropriate file path to the compiled file. It hooks into collectstatic so your less gets compiled when you collectstatic.
The biggest problem with this approach is that the mapping for your static files (what less + css files compile to css) is defined in your settings file. This requires a server restart when you update this. You could base your own server side less compilation off how Django Pipeline works but have logic for the mapping instead of defining it in something that requires a server restart.
It's a lot of work and the less compilation of Bootstrap is non-trivial to have to do on every request.
If you created your own mapping that doesn't require you to restart your Django server process, you could always just run collectstatic to create the new css files. This would avoid compilation at every request.
While this last approach is closest to what you mentioned, it seems a lot more work and error prone than just separating the color-specific styles and using django templating to customize it.
The last approach as well works well if your number of schemes is rather low since you can just create all the mappings ahead of time and not let people generate their own at runtime. They can suggest them and you can just update them at some regular cadence.