I am using the JQuery load function to load part of my page. Can I access the variables from that page in the page that loads it. e.g.
Page A uses JQuery load function to load B
Page B loads and sets a variable in context called pageB_var which holds a django object
Page A can then access this variable by doing {{pageB_var}} since it was added to the context
If not what is the best way of doing this?
Thanks
No. Page B's rendering context is irrelevant and unreachable by the time you get B's response.
Here's what happens:
Page A is rendered in the server. during this time, its context exists. when the server is done rendering it, it sends the rendered page to the client. the client web browser then runs the javascript including your jquery load() to call the server again and tell it to render B. at this point the process that rendered page A doesn't exist anymore, so for page B to send stuff to page A's rendering you would have to make time travel....
The way to do this, is for page B to return a JSON object, and then use the (javascript) callback function given to load() to render changes to the page based on this JSON response from B.
It sounds like you're using an asynchronous request to update part of a page, receiving a partial HTML response and replacing part of the markup of your page with this response.
You've found that this obviously limits you to updating information contained within that partial page.
Consider instead providing a JSON response to your asynchronous request, containing all the information you need, and then updating the necessary HTML in any part of the page by manipulating the DOM client-side, with JavaScript. Think of the JSON as being your context for this purpose.
Once page A is sent to the browser it's mostly fixed in memory. You'd have to use the DOM functions in JavaScript in order to modify what it shows. Having said that, you could return JSON from the view for the call to page B and then decode it and insert it into page A.
Related
Is there a way in django to update the same page with the response without totally rendering it. I am trying to create a code editor to test. But when I am returning the results my contents are removed. I understand it is because I am rendering the page . I need the contents to retain. How can I do it using redirect? I am including the render statement I used and a screenshot of how it looks here:
Steps:
Handle post request
Program execution code
Save the result in a variable called "message". Then I used
return render(request, 'editor.html', {'message': message})
I want to redirect the message to the same page without rendering a new page.
[Before submission][1]
[After submission][2]
[1]: https://i.stack.imgur.com/BxoLU.png
[2]: https://i.stack.imgur.com/uiEOU.png
Any help will be appreciated. Thank you.
it is possible. using ajax in front-end and render or render_to_string in back-end(Django). using ajax you're able to call a url (with/without data you want to send), then in the views.py you can do anything needed and return render(request,'template.html, context). then in ajax you have success: function (res) { $('#id_of_element').append(res)}. this function in ajax will append the recived response which is a rendered HTML template to the target element.
For that, you have to switch to a different web software paradigm called "single page application", which implies that both, backend and frontend, are functional software components on their own, instead of having a "dumb" HTML frontend that only displays what the backend renders.
In a regular web application, the front end is served from a backend with all the information that is going to display. In a Single Page Application, the front end is served by a server independent of the backend server, and the frontend and backend interact through an API served by the backend.
With this architecture, the frontend component is responsible for requesting and providing data from and to the backend, as well as for displaying the data and getting user's interaction, and the mean for interchanging data with the backend is called an ajax, that is an asynchronous request.
The only language accepted by web browsers is javascript, but there are many frameworks and second level languages that can render a javascript application, like React, Angular, Vue, and many others.
I have just started with Python Web scraping through Requests. This could be a broad question, I will try to make it as brief as possible.
I came through situation where sometimes an entire page source can be downloaded with r.content (where r is a response object of requests's get call)
Sometimes some part of the data is stored in json format... In files that can be accessed by deeply observing the get and post calls made.
However, I even found websites where the entire content is in DOM but part of it is neither in Page source nor in Json files.
I am wondering how many of such places can a website store a data in?
(Just the names, I am not looking for how to get there)
For these last type of websites, I have observed almost every requests call made, but couldn't find where the data is.
So are there any other place except the 2 mentioned above? Or those are the only two indicating I am not doing my job right of observing the requests call?
You may answer it in brief bullet points and I can take my study from there.
Thanks in advance.
Lets assume we are talking only about HTML data. A web server could serve you data in many other formats (JSON/XML, etc.)
Please note that what I have described is generalisation, and like most generalisations, you could find exceptions that do not fit in it.
Broadly we could divide the type of data displayed (for the end user) into two categories
Pre render
Post render
Pre render
The entire HTML page is constructed at server-side and sent across to the client. Here, the JS side is concerned with the user interaction, and not with the structure of the data.
We are slowly moving away from this type of structure, but currently a large majority of all web pages uses this.
Web scraping is relatively easy here, as we can programatically pull the html page, and not bother about the javascript code that accompanies it.
a combination of requests and beautifulsoup should work in almost all of the cases (assuming that you could identify the general structure of the document).
Post render
Here the HTML page that is returned from the server is just a "skeleton" or placeholders for the actual data. The data is rendered by the accompanying JS code.
In such cases, if you fetch the source file via for eg., requests, you will get an empty shell, with no data in it.
for this if you inspect the calls made by a browser while rendering, (chrome's network tab or firefox's inspect tool or the more popular firebug), you will most likely see ajax requests that brings back the actual data from the server)
depending on how the requests are made, you could hit that ajax endpoint, and get the data in JSON.
you could use response.json() function to extract it into python-dicts.
In certain (rare) cases, there would not be an ajax call, but the HTML served from the server will still be a shell. The actual data is part of that file served, but stored as part of the JS code itself. This could be done for a variety of reasons, for example for dynamic data to be sent to static js files, or just to deter simple attempts of scraping the page.
One approach to scraping such pages would be to 'render' the page in a headless browser, which executes the JS code and returns an HTML that could be parsed via parsers like beautifulsoup
beautifulsoup has the ability to work with many parsers, one of which is html5lib, which could solve this issue.
you could also look at selenium or mechanize
or you could try parsing the js code yourself which might be faster.
Arriving at a conclusion as to what to use requires careful inspection of how the page is rendered on a browser. Even if you don't see an ajax request, the html that is served by the server need not be how the browser displays it.
A good way to start is by looking at the bare-html that is being served, by either downloading the page via curl or requests.get or simply rendering it in your browser with javascript disabled.
Good luck.
We currently have a large implementation done in Django which mostly spit out static HTML. We are planning to implement a javascript modular framework to hook to django's API and define needed routes..etc
But, we do not want to scratch off what we did so far as the site is completely functional for non-js users. The more I read about JavaScript MVC designs, the more I find it impossible to implement the two to work together.
Any ideas of how to proceed in that direction? is there a best practice that someone can follow?
Keep what you got right now as a fail-safe. Create new, separate views meant for Ajax requests that handle the processing of data and respond with appropriately formatted respond data (XML, JSON etc.).
Basically, Django is your model. Any action defined by your Javascript controller can either modify the model or the view.
If it modifies the view, there are two options: you either change just the display of data, and thus you just call a Javascript function that alters the HTML, or you display new data. In that case, you have to create a Django view that supplies that data in a format that Javascript can process easily. In your Ajax request, supply a callback (the view part of your Javascript) that handles displaying that data.
If the action modifies the model, you also need to create a Django view that processes the POST data sent by the Ajax request as needed, and returns either the error messages or a success message in a format that is, again, easily processed by Javascript. Again, you should register a callback to your Ajax request that handles the display of the messages returned by Django.
So basically, the only change to your current views would be that you supply the appropriate Javascript code to map the actions to Ajax requests and to handle the data returned by Django for your Ajax requests. Everything else should be done in separate Django views.
I have a form in a plugin that on submit I need to redirect to another page.
What's the best way to accomplish this?
So possible solutions are:
Use an App-Hook
Throw an exception in your plugin render method that would be caught by a middleware class and do the redirect from there.
Create a middleware class and during the "process_response" method check for a value on the request object that was added during the render method of the plugin then do the redirect.
Plugins are not really suitable for processing POST requests and there's no way to influence the HTTP Response object from a plugin (other than the content of that response).
The reason Plugins have no hook to process POST requests is that a single page is usually composed of several plugins, and figuring out which plugin should handle the POST request would be very hard. For the same reason they can't change the response, since two plugins could try to change the response in an incompatible fashion.
The solution for this is to have a dedicated POST endpoint for your plugins (either static via urlpatterns or via an Apphook). That endpoint would then redirect to another page, or the page the form is on, so the plugin would send some data with it. Alternatively the plugin submits the form via AJAX to that endpoint and redirects/acts in javascript.
I'm writing a simple angularJS layer to a django app. On initial page load, the view content is already populated in the ng-view node. When I "manually" execute a route change (via ng-click callback), I do a request to the server to fetch the chromeless html content and overwrite the content of the ng-view node.
My issue is that on initial page load, the route change is executed and thus makes another call back to the server to fetch the same content that was pre-populated in the ng-view node.
I'm not sure if this is just the way Angular works or if it is that django uses trailing slashes and angular does not and thus thinks it is a different route?
Is there a way to prevent this from happening? Perhaps canceling the change in $routeChangeStart event?
Thanks! long time SO reader, first time poster.