I have a api url.
http://10.236.220.130:5056/api/v1/viewentities?where={"EntityId":"srv-1499839282322"}
my search query variable/key is where={"EntityId":"srv-1499839282322"}
this is my code in python 2.7
tgt="srv-1499839282322"
baseurl="http://10.236.220.130:5056/api/v1/viewentities?"`enter code here`
values={"where": '{"EntityId": "srv-1499839282322"}'}
(if i hardcode this srv stuff above it works)
encode=urllib.urlencode(values)
url=baseurl + encode
print encode
Note the url which am getting is working but my condition is, i want that srv stuff to be a variable that will come as one of my function parameter.
================
I tried this these
tgt="srv-1499839282322"
baseurl="http://10.236.220.130:5056/api/v1/viewentities?"
value={"where": {}}
value["where"]["EntityId"]=tgt
encode=urllib.urlencode(values)
url=baseurl + encode
print encode
But this encode url is not working...what i found is the
the urlparse.urlencode takes the query as a dictionary and my value as itself is a dictionary. (nested)
if print values(working) and value(not working)
i find the single quotes are the one missing from value.
{'where': {'EntityId': 'srv-1499839282322'}}-----> not working
{'where': '{"EntityId": "srv-1499839282322"}'} ----> working
Related
I have this url encoded:
Started PUT "/path/thing/9812/close?status=close&shutdown_on=2018-12-05%2010%3A08%3A06&affected_external_id=15027&fqdns%5B0%5D=150.212.3.249"
which decoded is this:
"/path/thing/9812/close?status=close&shutdown_on=2018-12-05 10:08:06&affected_external_id=15027&fqdns[0]=150.212.3.249"
I get this parameters:
Parameters: {"status"=>"close", "shutdown_on"=>"2018-12-05 10:08:06", "affected_external_id"=>"15027", "fqdns"=>{"0"=>"150.212.3.249"}, "id"=>"9812"}
How can get fqdn as a array? on Rails 4
You should do the following:
params[:fqdns].to_a
Doing this, will produce the following:
{['0', '150.212.3.249', ...]}
If you wnat only the values, may you can try:
params[:fqdns].values
Doing this, will give you the following:
['150.212.3.249', ...]
But for this, you have to do it inside a ruby class, i strongly recommends you to do it inside your controller. Hope i can help.
UPDATE
After a recommends, you can do it with strong parameters, permiting the param fqdns as a hash (because you route receiving a hash):
def resource_params
params.permit(....., fqdns: {})
end
After this, you already have to execute the solutions above to get fqsnd as a array
For educational purposes I am trying to scrape this page using lxml and requests in Python.
Specifically I just want to print the research areas of all the professors on the page.
This is what I have done till now
import requests
from lxml import html
response=requests.get('http://cse.iitkgp.ac.in/index.php?secret=d2RkOUgybWlNZzJwQXdLc28wNzh6UT09')
parsed_body=html.fromstring(response.content)
for row in parsed_body.xpath('//div[#id="maincontent"]//tr[position() mod 2 = 1]'):
for column in row.xpath('//td[#class="fcardcls"]/tr[2]/td/font/text()'):
print column.strip()
But it is not printing anything. I was struggling quite a bit with xpaths and was intially using the copy xpath feature in chrome. I followed what was done in the following SO questions/answers and cleaned up my code quite a bit and got rid of ' tbody ' in the xpaths. Still the code returns a blank.
1. Empty List Returned
2. Python-lxml-xpath problem
First of all, the main content with the desired data inside is loaded from a different endpoint via an XHR request - simulate that in your code.
Here is the complete working code printing names and a list of research areas per name:
import requests
from lxml import html
response = requests.get('http://cse.iitkgp.ac.in/faculty4.php?_=1450503917634')
parsed_body = html.fromstring(response.content)
for row in parsed_body.xpath('.//td[#class="fcardcls"]'):
name = row.findtext(".//a[#href]/b")
name = ' '.join(name.split()) # getting rid of multiple spaces
research_areas = row.xpath('.//*[. = "Research Areas: "]/following-sibling::text()')[0].split(", ")
print(name, research_areas)
The idea here is use the fact that all "professor blocks" are located in td elements with class="fcardcls". For every block, get the name from the bold link text and research areas from the following string after Research Areas: bold text.
am getting the following post data in my django app
POST
Variable Value
csrfmiddlewaretoken u'LHM3nkrrrrrrrrrrrrrrrrrrrrrrrrrdd'
id u'{"docs":[],"dr":1, "id":4, "name":"Group", "proj":"/al/p1/proj/2/", "resource":"/al/p1/dgroup/4/","route":"group", "parent":null'
am trying to get the id value in variable id i.e "id":4 (the value 4). When I do request.POST.get('id')I get the whole json string. u'{"docs":[],"dr":1, "id":4, "name":"Group", "proj":"/al/p1/proj/2/", "resource":"/al/p1/dgroup/4/","route":"group", "parent":null' How can I get the "id" in the string?
The data you are sending is simply a json string.
You have to parse that string before you can access data within it. For this you can use Python's json module (it should be available if you're using Python 2.7).
import json
data = json.loads( request.POST.get('id') )
id = data["id"]
If you somehow don't have the json module, you can get the simplejson module.
For more details, refer this question : best way to deal with JSON in django
That's happening because id is string, not dict as it should be. Please provide your template and view code to find source of problem.
I have the following XML file :
<user-login-permission>true</user-login-permission>
<total-matched-record-number>15000</total-matched-record-number>
<total-returned-record-number>15000</total-returned-record-number>
<active-user-records>
<active-user-record>
<active-user-name>username</active-user-name>
<authentication-realm>realm</authentication-realm>
<user-roles>Role</user-roles>
<user-sign-in-time>date</user-sign-in-time>
<events>0</events>
<agent-type>text</agent-type>
<login-node>node</login-node>
</active-user-record>
There are many records
I'm trying to get values from tags and save them in a different text file using the following code :
soup = BeautifulSoup(open("path/to/xmlfile"), features="xml")
with open('path/to/outputfile', 'a') as f:
for i in range(len(soup.findall('active-user-name'))):
f.write ('%s\t%s\t%s\t%s\n' % (soup.findall('active-user-name')[i].text, soup.findall('authentication-realm')[i].text, soup.findall('user-roles')[i].text, soup.findall('login-node')[i].text))
I get the error TypeError : 'NoneType' object not callable Python with BeautifulSoup XML for line : for i in range(len(soup.findall('active-user-name'))):
Any idea what could be causing this?
Thanks!
There are a number of issues that need to be addressed with this, the first is that the XML file you provided is not valid XML - a root element is required.
Try something like this as the XML:
<root>
<user-login-permission>true</user-login-permission>
<total-matched-record-number>15000</total-matched-record-number>
<total-returned-record-number>15000</total-returned-record-number>
<active-user-records>
<active-user-record>
<active-user-name>username</active-user-name>
<authentication-realm>realm</authentication-realm>
<user-roles>Role</user-roles>
<user-sign-in-time>date</user-sign-in-time>
<events>0</events>
<agent-type>text</agent-type>
<login-node>node</login-node>
</active-user-record>
</active-user-records>
</root>
Now onto the python. First off there is not a findall method, it's either findAll or find_all. findAll and find_all are equivalent, as documented here
Next up I would suggest altering your code so you aren't making use of the find_all method quite so often - using find instead will improve the efficiency, especially for large XML files. Additionally the code below is easier to read and debug:
from bs4 import BeautifulSoup
xml_file = open('./path_to_file.xml', 'r')
soup = BeautifulSoup(xml_file, "xml")
with open('./path_to_output_f.txt', 'a') as f:
for s in soup.findAll('active-user-record'):
username = s.find('active-user-name').text
auth = s.find('authentication-realm').text
role = s.find('user-roles').text
node = s.find('login-node').text
f.write("{}\t{}\t{}\t{}\n".format(username, auth, role, node))
Hope this helps. Let me know if you require any further assistance!
The solution is simple: don't use findall method - use find_all.
Why? Because there is no findall method at all, there are findAll and find_all, which are equivalent. See docs for more information.
Though, I agree, error message is confusing.
Hope that helps.
The fix for my version of this problem is to coerce the BeautifulSoup instance into a type string. You do this following:
https://groups.google.com/forum/#!topic/comp.lang.python/ymrea29fMFI
you use the following pythonic:
From python manual
str( [object])
Return a string containing a nicely printable representation of an
object. For strings, this returns the string itself. The difference
with repr(object) is that str(object) does not always attempt to
return a string that is acceptable to eval(); its goal is to return a
printable string. If no argument is given, returns the empty string,
I'm trying to encode a Python array into json using simplejson.dumps:
In [30]: s1 = ['test', '<script>']
In [31]: simplejson.dumps(s1)
Out[31]: '["test", "<script>"]'
Works fine.
But I want to escape the strings first (using escapejs from Django) before calling simplejson.dumps:
In [35]: s_esc
Out[35]: [u'test', u'\\u003Cscript\\u003E']
In [36]: print simplejson.dumps(s_esc)
["test", "\\u003Cscript\\u003E"]
My problem is: I want the escaped string to be: ["test", "\u003Cscript\u003E"] instead of ["test", "\\u003Cscript\\u003E"]
I can use replace:
In [37]: print simplejson.dumps(s_esc).replace('\\\\', '\\')
["test", "\u003Cscript\u003E"]
But is this a good approach? I just want to escape the strings first before encoding them to json. So there will be no syntax errors when I use them in template.
Thanks. :)
simplejson 2.1.0 and later include a JSONEncoderForHTML encoder that does exactly what you want. To use it in your example:
>>> s1 = ['test', '<script>']
>>> simplejson.dumps(s1, cls=simplejson.encoder.JSONEncoderForHTML)
'["test", "\\u003cscript\\u003e"]'
I ran into this recently where I didn't have control over the code that was generating the data structures, so I couldn't escape the strings as they were being assembled. JSONEncoderForHTML solved the problem neatly at the point of output.
Of course, you'll need to have simplejson 2.1.0 or later. (Django used to come with an older version, and Django 1.5 deprecated django.utils.simplejson entirely.) If you can't upgrade for some reason, the JSONEncoderForHTML code is relatively small and could probably be pulled into earlier code or used with Python 2.6+'s json package -- though I haven't tried this myself
You're doing the operations in the wrong order. You should dump your data to a JSON string, then escape that string. You can do the escaping with the addslashes Django filter.