Serve static HTML in Django - django

I’m pretty new to Django so forgive me if this is something I shouldn’t even consider, but I’d like my app to be able to link to a large number of static HTML pages (enough that creating URL paths to each one would be unreasonable) and there would constantly be more being uploaded (by me via FTP).
I’ve got this working on the the development server by adding the path to those HTML files to my STATICFILES_DIRS [] but that doesn’t seem to work when I push my code to production.
I tried setting up a STATIC_ROOT and running collectstatic but that didn’t help, and I’m also worried that even if I got that working, I would have to run collectstatic each time I uploaded new files.
So my question is, is this even reasonable? Should I just avoid hosting these static HTML files alongside my project and leave them where they are now, on a separate server under a separate domain?
The only reason I wanted to host them together initially is because along with the static HTML files, there is an SQL LITE database that my Django app displays data from (this is the whole purpose of my app). However, I could just implement another method of getting that SQL LITE file like using ftlib. Note that I don’t need that database to connect to Django at all - I just needs to read data from it.

You don't need to write urls for every page. You can "capture" the requested page name from the url and render the page according to its value.
# urls.py
url(r'^page/(?P<page_name>\w+)/$', my_view)
# views.py
import os
from django.http import HttpResponse, Http404
FTP_UPLOAD_DIR = '/path/to/directory/where/you/upload/files/'
def my_view(request, page_name):
# check if requested page exists
if os.path.exists(FTP_UPLOAD_DIR + page_name):
# if yes, then serve the page
with open(FTP_UPLOAD_DIR + page_name) as f:
response = HttpResponse(f.read())
return response
else:
raise Http404
Above, we are reading the file directly from the upload folder, so there's no need for you to run collectstatic.

Related

How do I get the css templates to update automatically? The issue is with the browser cache I think

from flask import Flask
from flask import render_template
app = Flask(__name__)
app.config["TEMPLATES_AUTO_RELOAD"] = True
#app.route("/")
def index():
return render_template("index.html")
#app.route("/hello/<name>")
def hello(name):
return render_template("hello.html", name = name)
if (__name__ == "__main__"):
app.run(debug = True)
So in order to see the most up to date css I have to hold shift and then refresh. Obviously, this isn't ideal, I want the user to see the most up-to-date version automatically. I was told that the config line would od that, but it isn't working properly. How do I actually get this to work?
This has more to do with the browser cache than with flask it self. So what you can do is modify your css files names every time you modify them.
So you say is way too much work?
Well there is a flask extension called Flask Static Digest that will keep track of your static files (You can config what it keeps track of) and will compile them with an md5 tag dependig on the files content. So every time you update statics files and compile, the names of the files will change as well and the browser will be forced to reload the resources.

Django, render with link or remove template dirs when render in view

hai i am newbie in django and sorry for grammar mistake
first i have some project to upload file extension .html to AWS S3, so in views.py i want to render a link i've already uploaded to AWS S3. ex: render(request, 'somelink.com', context) , it possible? or any other solution? and also i want to send context parameters
why i not using media_url upload to local? cause i have limited disk, and other problem when i do production i cant load media_url, ignore this case, cause i already try many solution
Assuming that you want to render it server side, you will need to fetch the template (the .html file) and render it.
You can do it using the requests library to fetch the url template. And the Django template render engine to render the context: https://docs.djangoproject.com/en/4.1/ref/templates/api/#rendering-a-context
import requests
from django.template import Template, Context
template = requests.get('somelink.com').content
t = Template(template)
c = Context(context)
return HttpResponse(t.render(c))
Notes:
It can worse the response time from your server.
As you are concerned about space, you can try to use a front-end solution to render data from your server via API.
Hope it can help you.

Unit Test: How To Mock MEDIA_ROOT But Still Access A File That Is Stored In My Normal MEDIA Folder

Im testing one of my webpages POST functions. if request.method == 'POST' it generates a pdf and attaches it to the user. Every time I ran tests I was generating pdf files which were building up in my MEDIA folder and I had to manually delete them. I decided to look for ways to prevent this from happening and what seems to me to be the best way is to override my MEDIA_ROOT with a tempfile.
I am getting the following error message
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Acer\\AppData\\Local\\Temp\\profile_pics\\default.jpg'
I am sure this is because I must create and log-in a user prior to running the test (I dont want to do this in setup as I must generate different types of users for different tests). Every time a user is created their profile picture is set to profile_pics\\default.jpg'.
My understanding is that the mocked MEDIA_ROOT is empty and doesnt contain the default profile pic, so the error is thrown.
My question is how can I circumvent this (ideally mocking the default.jpg image)? I found other solutions besides mocking the MEDIA_ROOT folder, but they seem quite hacky and id prefer to do it properly and hopefully learn from this.
tests.py
from django.test import override_settings
import tempfile
#override_settings(MEDIA_ROOT=tempfile.gettempdir())
def test_redirects_after_POST(self):
user = User.objects.create_superuser('username')
self.client.force_login(user)
response = self.client.post(
reverse('final_question'), data={
'answer': 'this has me totally stumped'}
)
self.assertRedirects(response, reverse('page_2'))
Thank you.
The problem:
PDF files are building up in your MEDIA_ROOT folder because there is nothing automatically deleting them.
Recommended solution:
I actually ran into this problem a few months ago myself, and the solution I found was the django-cleanup tool. It's very easy to install and integrate into your app (simply add it to your INSTALLED_APPS in settings.py).
Note: This is assuming that your PDF files are in a FileField of one of your models. If that's not the case, then this solution won't work for you.

How to tell Django to put media (pdf) files urls in sitemap.xml?

I've put static views and model views in my Django generated sitemap.xml file but i do not know how to tell Django to put all of the media files in to it? I have a hundred of PDF files with seo friendly links and i want them in my sitemap.xml, but as they are not in correlation with any of my models i don't know how to manage this?
EDIT: I almost forgot one important thing - my media (pdf) files are served through CloudFront so even if i manage somehow to list them in my Django Sitemap.xml i'll have additional problem because they have 'something.cloudfront.com' in their url's and not on my web site's url 'example.com'.
Is this even possible to solve? How does this reflect on SEO?
SOLVED:
#kb, thanks for a great answer! I've used RewriteRule in my htaccess as you suggested in the first part of your answer, and it works fine.
As of second part, instead of creating model for my media files (which would work just fine, but only downside would be manual adding of every new pdf file)
i decided to add some lines to my items() method so i could list bucket content and filter pdf files. With that i can have all of my files up-to-date all the time, easily:
#sitemap.py
import boto
from boto.s3.key import Key
from boto.s3.connection import S3Connection
import re
def items(self):
AWS_ACCESS_KEY_ID = #'my_access_key_number'
AWS_SECRET_ACCESS_KEY = #'my_secret_access_key'
Bucketname = #'my_bucket_name'
conn = boto.s3.connect_to_region('eu-central-1', aws_access_key_id=AWS_ACCESS_KEY_ID, aws_secret_access_key=AWS_SECRET_ACCESS_KEY, is_secure=False, calling_format = boto.s3.connection.OrdinaryCallingFormat())
bucket = conn.get_bucket(Bucketname)
new_list = []
regex = re.compile(r'bucketsubfolder/media.*\.pdf$', re.I) #i hold my media files in bucketsubfolder so url is for example somedomain.cloudfront.net/bucketsubfolder/media/somefile.pdf
for item in bucket.list():
if regex.match(item.name):
new_list.append(item.name)
return new_list
You are not allowed to use external urls in your sitemap (or rather, they won't have the desired effect being indexed by Google as part of your site content).
I think your best option is to dedicate a path on your site like /hosted/pdf/xxxx.pdf that rewrites everything to cloudfront.com/pdf/xxxx.pdf or similar using mod_rewrite/location patterns/regex.
That way you can use a local site URL in your sitemap but still have the browser sent to the cloudfront served content directly, I think this might even be a good use of the 302 HTTP status code.
In the Sitemap class there is an items() method that returns what is to be included in the sitemap.xml, and you could create your own class that extends it and adds additional data.
You can either manually add the data hardcoded in the method but I think the preferred option is to create a Model that represents each remote hosted file and that contains the information necessary to output it in the sitemap. (This also lets you add properties such as visibility on a per file basis and lets you manage it via admin assuming you set up a ModelAdmin for it.)
I think you might be able do something similar to what they show in http://docs.djangoproject.com/en/1.9/ref/contrib/sitemaps with the BlogSitemap class that extends Sitemap. Be sure to check the heading "Sitemap for static views" on that page as well.
My suggestion is that you chose the model approach to represent the files, so you have your hosted PDFs (or other CDN content) as a model called StaticHostedFile or similar and you iterate through all of them in the items() section. It does require you to index all the current PDFs to create models for them as well as create a new model whenever a new PDF is added (but that could be automated).
It can be good to know that you can add "includes" in a sitemap.xml so you might be able to split the site content into two sitemaps (content+pdfs) and include both in sitemap.xml, for instance:
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>http://www.example.com/original_sitemap.xml</loc>
<lastmod>2016-07-12T09:12Z</lastmod>
</sitemap>
<sitemap>
<loc>http://www.example.com/pdf_sitemap.xml</loc>
<lastmod>2016-07-15T08:55Z</lastmod>
</sitemap>
</sitemapindex>
This still requires local URLs and rewrites as per above though, but it can be a nifty trick for when you have several separate sitemaps to combine. (For instance if running a Django site under one subdir and a Wordpress site under another or whatnot.)

Reversing urls in Django / Celery

I have been asked to send an email on friday with a list of projects matching a set of conditions. The database front end is written in Django and its an in house application. I would like to link to the admin page of the project in the email.
On my development machine I am trying
reverse("admin:index")
and I notice that it is only returning
admin
from the Celery task, whereas in standard Django views this would return
127.0.0.1:8000/admin/
Is there a way around this? I would prefer not to hard code the start of the urls.
I have set up Celery as described in the "first steps with Django" tutorial, and have a
myproject/myproject/celery.py
file defining the app (alongside settings.py and urls.py) and
project/myapp/tasks.py
file with the actual tasks.
reverse() always returns a domain-relative url, i.e. /admin/. One option to increase portability is to use the sites framework.
As you don't have a request object in your Celery task, you have to explicitly set the SITE_ID in your settings and set the site's name and domain in your database. Then you can do:
from django.contrib.sites.models import Site
url = "%s%s" % (Site.objects.get_current().domain, reverse('admin:index'))
Usually URLs in django are relative - i.e. the "127.0.0.1:8000" part you see when you hover the link comes from the fact that you are browsing the site from that location. Try looking at the href value of the URLs generated.
A simple solution to your problem is to concatenate the reverse output with the domain, i.e.
'http://example.com/'+reverse('admin:index')