i try to scrape a website, for choosing every Category, i need to check a check box
<ul class="mvSrcLk">
<li>
<label class="mvNavSel mvNavLvl1">
Marché frais
<input type="checkbox" value="f/1/1p" name="selectedNavigationCategoryPath">
</label>
</li>
<li>
<label class="mvNavSel mvNavLvl1">
Épicerie sucrée
<input type="checkbox" value="f/1/1o" name="selectedNavigationCategoryPath">
</label>
how can do this by Scrapy, i read this but it doesn't helped me
Is the code part of a <form>? If so then you may need to make a request with the "check box selected", as well as anything else required for the form submission, and then scrape what you need from that request.
If however, the checkbox executes some sort of Javascript that changes the HTML content dynamically, then you'll need to use other tools in combination with Scrapy. There are projects that combine Scrapy and Selenium or Splash that should execute Javascript and return the response.
References
Scraping dynamic content using python-Scrapy
selenium with scrapy for dynamic page
Related
I am using Bootstrap-select for multi-select items with Django. It works fine on desktop, but when the native mobile drop-down is enabled, the selected values of the dropdown do not populate.
HTML
<!-- start product brand info -->
<div class="row">
<div class="col-md-6 col-xs-*">
<div>
<label for="id_brand-name" class="form-label">Products</label>
</div>
<select multiple class="form-control selectpicker mb-3" id="id_brand-name" name="brand-name" mobile="true" multiple required>
{% for product in products %}
<option value="{{product.id}}" id="optiion">{{ product.brand.name | title }} - {{product.name | title}}</option>
{%endfor%}
</select>
<div class="invalid-feedback">
Select at least one product.
</div>
</div>
</div>
<!-- end product info -->
<script>
//Enble native drop down on mobile
window.onload = function () {
$('#id_brand-name').selectpicker({
container: 'body'
});
if( /Android|webOS|iPhone|iPad|iPod|BlackBerry/i.test(navigator.userAgent) ) {
$('#id_brand-name').selectpicker('mobile')
}};
</script>
No matter how many items are selected, the selector always shows Nothing selected. When the data is posted, it is apart of the form though.
I like to use bootstrap-select, but this problem has been bothering me for a long time
Similar questions can be found on github, but no perfect answer.
The reason is that it will always refresh his title, no matter how remove() is done externally, and you can't change it's style on iphone Safari and Firefox.
So my solution is: if you can't remove it, then join it.
You have to change the original file: bootstrap-select.js
Search for: bs-title-option, next line you will find:
this.selectpicker.view.titleOption.value = '';
And add two lines down:
this.selectpicker.view.titleOption.value = '';
// new add
this.selectpicker.view.titleOption.disabled='true';
this.selectpicker.view.titleOption.textContent=this.options.title;
//
Done
When you use $('#id_brand-name').selectpicker('mobile')
The title text show on first empty option, and it can't be choose.
try it!
Lets say I have a model with 2 fields. With one field being a choice field of radio button Choice1, Choice2 and Other, the next being Other which is a textfield I want the "other" textbox to appar / enabled only when "Other" is selected in the radio button.
This question is not from the django-forms category. This applies more to the front-end category. Everything that is sent from the django server is static. Sure, you can write a form class and override the template for it. Also, you can connect the js script directly to the form class. It is convenient, but not canonical. Just, write the JS script or using JQuery, which will activate the field when you select a particular option.
I wrote for you a small example of how this can be do it.
I hope this helps you.
$('input[type=radio][name=choices]').change(function() {
$('input[type=text][name=other]').prop(
'disabled',
function(i, v) {
return !v;
});
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<form>
<p><input name="choices" type="radio" value="choice1">Choice 1</p>
<p><input name="choices" type="radio" value="choice2">Choice 2</p>
<p><input name="other" type="text" disabled></p>
</form>
I'm stuck using django-markdownx to automatically update page and to submit changes.
I followed this question and answer and managed to get django-markdownx working in admin, and within my view. However in my view editing the textarea does not automatically update the page.
The admin page with django-markdownx is exactly what I want, updating the textarea updates the page, but not the underlying database field until you hit save.
So I then tried to rip out the admin code into my own view.
In my view/template I have a form, textarea similar to admin one. I also included "/static/markdownx/js/markdownx.js" and set my form to mostly be similar to the admin page:
<form method="POST" action="">{% csrf_token %}
<div class="markdownx">
<textarea name="myfield" rows="10" cols="40" required="" data-markdownx-upload-urls-path="/markdownx/upload/" data-markdownx-editor-resizable="" class="markdownx-editor" id="id_myfield" data-markdownx-urls-path="/markdownx/markdownify/" data-markdownx-latency="500" data-markdownx-init="" style="transition: opacity 1s ease;">
{{ note.myfield }}
</textarea>
</div>
<div class="markdownx-preview">
{{ note.formatted_markdown|safe }}
</div>
</form>
This didn't work.
I see periodically there is requests to /markdownx/markdownify/ when you edit in admin, but not mine. I'm not sure if I should aim to do the same or just do some timed javascript page refresh and pass all the data from within my form back to my view to then re-render the page again.
I can't quite get my head around the django-markdownx documentation.
UPDATE:
The Documentation seems to suggest that a call to MarkdownX() should do the initialisation.
<script src="/static/markdownx/js/markdownx.js"></script>
...
<script type="text/javascript">
let parent = document.getElementsByClassName('markdownx');
let md = new MarkdownX( element, element.querySelector('.markdownx-editor'), element.querySelector('.markdownx-preview'));
</script>
But when I try this I get.
Uncaught ReferenceError: MarkdownX is not defined
Also I don't see any initialisation like this within the admin page.
Is there an example of using the django-markdownx in your own views similar to the usage within admin?
Thanks
LB
The following is a broken solution.
The correct method would be to use the MarkdownX's built-in Javascript, but I just can't get it to work, yet. So, I wrote my own. It may be of use to others.
In template html, include js.cookie.min.js in order to get the django csrftoken. Then a callback function which will be called when a change is made to the textarea. We then update the preview div with HTML code we received back from MarkdownX's markdownify call.
<script src="https://cdn.jsdelivr.net/npm/js-cookie#2/src/js.cookie.min.js"></script>
...
<script type="text/javascript">
function myMDFunc( elem ) {
input = elem.value;
var csrftoken = Cookies.get('csrftoken');
$.ajax(
{
type: "POST",
url: "/markdownx/markdownify/",
data: { CSRF: csrftoken, csrfmiddlewaretoken: csrftoken, content: input}
})
.done(function(data, status){
document.getElementById("markdownx-preview").innerHTML = data;
});
}
</script>
Still in the template html, in the form, call this function both for onchange and onkeyup.
<form method="POST" action=""> {% csrf_token %}
{{ note.title }}
<div class="markdownx">
<textarea onchange="myMDFunc(this)" onkeyup="myMDFunc(this)" cols="60" rows="5" name="text" >
{{ note.myfield }}
</textarea>
</div>
<div class="markdownx-preview" id="markdownx-preview">
{{ note.formatted_markdown|safe }}
</div>
<input type="submit" id="submit" name="submit">
</form>
In summary, a change to the textarea means we invoke the 'onchange' or 'onkeyup', which calls myMDFunc. Then myMDFunc does an ajax call with data which is the raw MarkDown code, the response to this call is the pretty HTML data. The callback within myMDFunc updates the preview with that pretty HTML.
It kinda works. I'm sure the real MarkdownX code will handle drag'n'drop of images and pacing the ajax calls to be nice to the server.
How can i create a html form that action attribute have 2 destination.I want when user click on submit bottom , check if user entered wrong data the page goes to another pages with window.location and if user insert the correct input goes to main page with the same instruction.
First of all, what do you mean by correct input?
Main form data validation occurs in server side, not client side. you'd better use client side just for simple verification, like for typos.
There is no need for 2 destination pages (as you call it so).
You may use the standard action attribute which is the page on the server to which you are sending your form data.
there, You have the option to decide which condition needs what action and send the data (and then the user) to the desired page / action.
Sample code for the form
<form id='myform' action='action.php' method='POST' target='formresponse'>
<label for='name' >Your Full Name*: </label><br/>
<input type='text' name='name' id='name' maxlength="50" /><br/>
<label for='email' >Email Address*:</label><br/>
<input type='text' name='email' id='email' maxlength="50" /><br/>
<input type='button' name='Submit' value='Submit' />
</form>
<iframe name='formresponse' width='300' height='200'></frame>
Multiple action
function SubmitForm()
{
document.forms['contactus'].action='action1.php';
document.forms['contactus'].target='frame_result1';
document.forms['contactus'].submit();
document.forms['contactus'].action='action2.php';
document.forms['contactus'].target='frame_result2';
document.forms['contactus'].submit();
return true;
}
When working with Coldfusion 9 and cfform with a HTML format, I place a cfinput on a page with a label, it displays the label to the right of the text box. I have tried using the tag, with and without it but no matter what I do, the label is always to the right of the box.
<cfform method="post" name="mfForm" >
<label for="campaign">Mailfile ID:</label>
<cfinput type="text" name="campaign" id="campaign">
<cfinput type="submit" name="submit" value="Submit" id="submit">
</cfform>
Don't ever remember having this problem before recently. I would just use an HTML form, but want to take advantage of cf's autosuggest.
I hate to say it, but frankly quirks like this are why many people suggest ditching the built-in ajax features and using the underlying libraries (or some jQuery alternative) directly. You will have greater control, more choices, not to mention you will not be tied to whatever version ships with ColdFusion. Most of these libraries are updated frequently, so within a year the ones bundled with CF are often out of date. ExtJS is a good example. The public version is already up to version 4.2.1, but CF9 still uses 3.1.0.
Anyway, getting back to your question ... if you do a view source you will see CF generates several div tags, one of which contains the style="float:left" directive, which could explain the behavior you are seeing.
I did a quick search and happened upon a note in the the CF8 docs which suggest a hack for datefields which may also apply here:
To correctly display label text next to the control in both Internet Explorer and Firefox, you must surround the label text in a
<div style="float:left;"> tag and put three <br> tags between each
line.
Simply adding the div seems to work for me with the sample you posted:
<cfform method="post" name="mfForm" >
<div style="float:left;">
<label for="campaign">Mailfile ID:</label>
</div>
<cfinput type="text" name="campaign" id="campaign" autosuggest="AA,BBB,CCC,DDD">
<cfinput type="submit" name="submit" value="Submit" id="submit">
</cfform>
But again, you might want to consider using the javascript libraries directly instead of relying on the built-in ajax features, so you can avoid weirdness like this.