I have several instances of my project running on my server, like so:
http://0.0.0.0/one
http://0.0.0.0/two
I also have an activation view that is accessible via:
http://0.0.0.0/one/activate/u/1/c/123
When I do reverse() on this view from django shell, the url given to me as:
/activate/u/1/c/123
So it does not include the /one server path. However, when I use reverse() to look up the path of the page to send in an email somewhere else in the project, reverse() seems to return the full server path + the view path, like so:
/one/activate/u/1/c/123
Does anyone have any idea of why this is happening?
reverse() is supposed to include this server path, so that you can just use it in a link and it'll work without having to change anything else in your code. But manage.py shell doesn't set the appropriate path prefix; that code happens in the wsgi/etc handler. This is Django bug #16734 (which I incidentally reported :p).
You can work around this by calling django.core.management.base.set_script_prefix manually, presumably in your settings.py. For example:
# when running through wsgi, this will get overriden
# but it's needed for manage.py
from django.core.urlresolvers import set_script_prefix
set_script_prefix('/one/')
Related
I am developing a django web application where a user can modify the code of certain classes, in the application itself, through UI using ace editor (think of as gitlab/github where you can change code online). But these classes are ran by django and celery worker at some point.
Once code changes are saved, the changes are not picked by django due to gunicorn but works fine with celery because its different process. (running it locally using runserver works fine and changes are picked by both django and celery).
Is there a way to make gunicorn reflects the changes of certain directory that contain the classes without reloading the whole application? and if reloading is necessary, is there a way to reload gunicorn's workers one-by-one without having any downtime?
the gunicron command:
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
The wsgi configuration file:
import os
import sys
from django.core.wsgi import get_wsgi_application
app_path = os.path.abspath(os.path.join(
os.path.dirname(os.path.abspath(__file__)), os.pardir))
sys.path.append(os.path.join(app_path, 'an_application'))
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production")
application = get_wsgi_application()
The reload option is "intended for development". There's no strong wording saying you shouldn't use it in production. The reason you shouldn't use it in production is because people make typos, change in one file, may need several other changes in others, etc etc. So, you can make your site inaccessible and then you don't have a working app to fix it again.
For a dev, that's no problem as you look at the logs/output in your shell and restart it. This is why #Krzysztof's suggestion is the best one. Push the code changes to your repo, make it go through the CI/CD and switch over the pod. If CI fails, then CD won't happen so you're good.
Of course, that's a scope far too large for a Q&A site.
Why not save the code in a separate text file or database and the relevant method can simply load the code dynamically as a string and execute it using exec()?
Let say you have a function function1 which can be edited by a user. When the user submits the changes, process the input (separate out the functions so that you know which function has what definition), and save them all individually, like function1, function2 etc., in a database or a text file as strings.
One you need to execute function1, just load its value that you saved and use exec to execute the code.
This way, you won't need to reload gunicorn since all workers will always fetch the updated function definition at run time!
Something in the lines of:
def function1_original():
# load function definition
f = open("function1.txt", "r")
# execute the string
exec(f.read()) # this will just load the function definition
function1() # this will execute the user defined function
So the user will define:
def function1():
# user defined code
# blah blah
...
I was able to solve this by changing the extension of the python scripts to anything but .py
Then I loaded these files using the following function:
from importlib import util
from immportlib.machinary import SourceFileLoader
def load_module(module_name, modele_path):
module_path = path.join(path.dirname(__file__), "/path/to/your/files{}.anyextension".format(module_name))
spec = util.spec_from_loader(module_name,
SourceFileLoader(module_name, module_path))
module = util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
In this case, they are not loaded by Gunicorn in RAM and I was able to apply the changes on fly without the need to apply eval or exec functiong.
TL;DR: Added custom location type to environment.js then ember serve -> open browser to route /foo -> cannot GET /foo
Followed the instructions at https://www.emberjs.com/api/classes/Ember.Location.html#toc_custom-implementation and copied the code exactly as it appeared into a file called app/locations/history-url-logging.js, added a line to config/environment.js that said:
ENV.locationType = 'history-url-logging';
For reference, the code given in the docs is simply:
import Ember from 'ember';
export default Ember.HistoryLocation.extend({
implementation: 'history-url-logging',
pushState: function (path) {
console.log(path);
this._super.apply(this, arguments);
}
});
I decided to restart the server, did the usual CTRL+C to ember s then did ember s again. I went back to my browser sitting on one of the routes, hit F5, and received the cryptic error:
Cannot GET /contacts
So, after MUCH Googling and trial and error (and posting a previous question here which I just edited with this text you're reading), I discovered that to FIX that error, all I had to do remove the config line ENV.locationType = 'history-url-logging';, restart the server (ember s), and suddenly the app worked fine!
What's even more odd is that if I start the app without that line in environment.js, then once the app is running (and the browser window reloads just fine, etc), then I re-add the line that says ENV.locationType = 'history-url-logging'; (which triggers a live reload), and the app still works fine! (E.g. hitting F5 to reload the page doesn't vie me the "Cannot GET /contacts" (or whatever the route is) error.) And, of course, the console gives me the "console.log" output as expected from the code above.
So, long and short of it, using a custom location totally seems to screw up ember serve - which is really sad and frustrating! Any ideas how to fix this?
Ember built-in server looks at the environment.js locationType property to figure out if it must serve routes after the rootURL path. By default, if the locationType is history it will do it. It uses string matching.
In your case you wrote your own location, inheriting from HistoryLocation therefor the locationType property in the environement.js is now history-url-logging. The built-in server doesn't recognize it as a history based form of location just by the name. It will default to hash location. It doesn't analyze your code.
For this scenario, we have to help the built-in server to understand that the locationType is equivalent to a history location.
You need to add historySupportMiddleware: true in your environment.js file right after the locationType property.
I am using emmedded jetty in my java project. For some reason, the path I am sending into resourceHandler is c:\ (lower case) and it is being aliased to C:\ (upper case). Because of this my static content is not being served.
I read some docs that indicated that jetty compares the absolute path and canonical path to detect aliases. In the log I see:
[qtp15485575-19] INFO org.eclipse.jetty.server.handler.ResourceHandler - file:/c:/filepath aliased to file:/C:/filepath
Anyone have any ideas on how to resolve?
Update: logged bug to eclipse for this: https://bugs.eclipse.org/bugs/show_bug.cgi?id=471526
Here's their response:
"it is indeed very annoying, but is forced on us by the poor security model of the servlet spec.
IF the spec had said that all URIs were denied unless explicitly allowed, we would not need to check for aliases. But instead it has a model where it allows all URIs except those that are specifically denied.
Thus if a security constraint is put on /secretfile.txt we have to make sure that any aliases of that file are also constrained... and do so in an FS independent way. That means on various operating systems we might need to block:
/sEcRetFile.TXT
/secretfile.txt
/SECRE~01.TXT
/secretfile.txt##0
etc. etc. etc.
So to be sure, we have implemented the alias system.
Normally we don't get problems with c: and C: because that should be normalized when configuring the context, so the correct one should be used. But filesystems do change their behaviour between releases, so it can be very annoying.
I think this is handled somewhat better in jetty 9.3 where we can use the Path classes to better examine parts of the path."
So probably the best bet is to use Jetty 9 if you can and use Joakin's fix if it is still an issue.
Give your ResourceHandler a full, absolute, and real path.
package jetty.resource;
import java.io.File;
import java.nio.file.Path;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.ResourceHandler;
import org.eclipse.jetty.util.resource.PathResource;
public class ResourceHandlerFromFSExample
{
public static void main(String[] args) throws Exception
{
Server server = new Server(8080);
Path webRootPath = new File("src/test/webroot").toPath().toRealPath();
System.err.println("WebRoot is " + webRootPath);
ResourceHandler handler = new ResourceHandler();
handler.setBaseResource(new PathResource(webRootPath));
handler.setDirectoriesListed(true);
server.setHandler(handler);
server.start();
server.join();
}
}
And btw, DefaultServlet is still a better choice for static file serving.
I am using Jetty-9 in embedded mode and need only one web application. Consequently I would like the root URL to go to the homepage of that application, i.e. something like
http://localhost:4444/
should end up in a servlet. I start out with:
ServletContextHandler scContext =
new ServletContextHandler(ServletContextHandler.SESSIONS);
scContext.setContextPath("/");
None of the following worked, neither
scContext.addServlet(ListsServlet.class, "/");
nor
scContext.setWelcomeFiles(new String[]{"/lists})
where /lists is mapped to the ListsServlet servlet. All I get is a 403 (Forbidden).
I do not use the DefaultServlet, which seems to handle welcome files. But since the ServletContextHandler has setWelcomeFiles I expected it to contain the logic to use them.
Any ideas?
For the 403 Forbidden error, you have some security setup that is not allowing you to access the handlers/servlets.
Eliminate that security (for now), verify that the rest is working, then add security a bit later to lock down specifics.
If you want to see some the suggestions below at work, consider looking at the code example in the answer from another stackoverflow: How to correctly support html5 <video> sources with jetty.
Welcome files are appended to the incoming request path if there is nothing present at that location. For example requesting a directory and then a welcome-file of 'index.html' is appended to the request path.
While this would work ...
scContext.setWelcomeFiles(new String[]{"lists"})
// Add Default Servlet (must be named "default")
ServletHolder holderDefault = new ServletHolder("default",DefaultServlet.class);
holderDefault.setInitParameter("resourceBase",baseDir.getAbsolutePath());
holderDefault.setInitParameter("dirAllowed","true");
holderDefault.setInitParameter("welcomeServlets","true");
holderDefault.setInitParameter("redirectWelcome","true");
scContext.addServlet(holderDefault,"/");
It's likely not what you are aiming for, as you said the root path only.
The above would also make changes to requests like /foo/ to /foo/lists
Instead, it might make more sense to use a Rewrite rule + handler instead of the welcome-files approach.
RewriteHandler rewrite = new RewriteHandler();
rewrite.setHandler(scContext);
RewritePatternRule rootRule = new RewritePatternRule();
rootRule.setPattern("/");
rootRule.setReplacement("/list");
rootRule.setTerminating(true);
rewrite.addRule(rootRule);
server.setHandler(rewrite);
This RewritePatternRule simply changes any request path / to /list and then forwards that request to the wrapped ssContext (if you want to see the /list on the browser, change it to a RedirectPatternRule instead.
I have a bug in my 404 setup. I know that because, when I try to reach some page which doesn't exist, I get my server error template. But that templates is useless because it doesn't give me any debug info. In order to get django's debug page, I need to set DEBUG=True in settings file. But if I do that, bug doesn't appear because django doesn't try to access my buggy 404 setup. So what do you guys think? This is in my root urls file:
handler404 = 'portal.blog.views.handlenotfound' And this is in portal.blog.views.handlenotfound:
def handlenotfound(request):
global common_data
datas = {
'tags' : Tag.objects.all(),
'date_list' : Post.objects.filter(yayinlandi=True).dates("pub_date","year")
}
data.update(common_data)
return render_to_response("404.html",datas)
Edit:
I guess I also need to return a HttpResponseNotFound right?
If I had to debug this kind of errors, I would either
temporarily turn the handler into a simple view served by a custom url, so that django's internal mechanisms don't get into the way, or
(temporarily) wrap the handler code in a try..except block to log any error you may have missed
Anyway, are you sure your handler doesn't get called if DEBUG=true?
data.update(common_data) should be datas.update(common_data).
(Incidentally, data is already plural: the singular is datum.)