I need to convert the following cUrl command to Python 2.7:
curl -X POST --data-binary #myfile http://127.0.0.1/process/my/data
I googled for this, but I only found examples of posting fields or files, instead of sending the content of a file into the bodyof the request.
For those interested, here's the answer:
# -*- coding: utf-8 -*-
import httplib2
h = httplib2.Http(".cache")
resp, content = h.request("http://127.0.0.1/requests",
"POST", body="Here is the content of the file you want to post",
headers={'Content-type': 'application/json'}
)
print resp
Related
I am trying to publish to a pub/sub topic via HTTP Bridge using python3 & CURL.
**Python3**
import json
import logging
import os
import socket
import sys
import time
import requests
URL = 'https://cloudiotdevice.googleapis.com/v1/projects/{}/locations/{}/registries/{}/devices/{}:publishEvent'
JWT = 'JWT'
def main():
if not URL or not JWT:
sys.exit("Are the Environment Variables set?")
get_sensor_data(socket.gethostname())
def get_sensor_data(device_id):
while True:
print("in get_sensor data")
payload = {'device': str('asd'),
'type': str('adssaff'),
'timestamp': str(time.time()),
'data': json.dumps({'temperature': str('23'),
'humidity': str('442')})}
post_data(payload)
print("data printed")
time.sleep(5)
def post_data(payload):
payload = json.dumps(payload)
headers = {
'Content-Type': 'application/json; charset=utf-8',
'Authorization': JWT
}
try:
req = requests.post(URL, json=str(payload), headers=headers)
print("request Successfull "+str(req))
except requests.exceptions.ConnectionError:
logging.error('Error posting data to Cloud Function!')
except requests.exceptions.MissingSchema:
logging.error('Error posting data to Cloud Function! Are Environment Variables set?')
if __name__ == '__main__':
This is giving an error 400 because i think i havent described the subfolder.
Now i am confuse that where can i define the subfolder(Topic name) in my code?
and is there only subfolder is missing? or i am doing something else wrong too?
CURL
i also tried using the CURL command described in
https://cloud.google.com/iot/docs/how-tos/http-bridge
The command is
curl -X POST -H 'authorization: Bearer JWT' -H 'content-type: application/json' --data '{"binary_data": "DATA", "sub_folder": "SUBFOLDER"}' -H 'cache-control: no-cache' 'https://cloudiotdevice.googleapis.com/v1/projects/{project-id}/locations/{cloud-region}/registries/{registry-id}/devices/{device-id}:publishEvent'
It triggers my cloud function which means the authorization works but i am not able to see "DATA" in my logs. which i assume i am not providing the right format for the binary_data. whyt would be the right format if i would like to publish 'payload' described above using curl too?
It looks like you are using a JSON payload with the data field set to an object, rather than binary string. Try to either json.dumps the object in the 'data' field or send the 'data' field as a string.
From this documentation.
https://cloud.google.com/iot/docs/reference/cloudiotdevice/rest/v1/projects.locations.registries.devices/publishEvent
I found out that my payload request body was not correct.
so payload should look like this below ..
s= json.jumps('json object')
payload = {"subFolder": 'Sub_FOLDER_NAME', "binaryData": base64.b64encode(s.encode('utf-8'))}
I need to save the SSL keys in a file, in order to decrypt the TCP packet via Wireshark later.
What should I do?
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import urllib2
import json
data={}
data_json = json.dumps(data, encoding='UTF-8', ensure_ascii=False)
requrl = "https://52.31.41.56/test" # look, the protocol is https
req = urllib2.Request(url=requrl, data=data_json)
req.add_header('Content-Type', 'application/json')
# how can I record the SSL keys in a file, for Wireshark decryption
rsp_fp = urllib2.urlopen(req)
rsp_data = rsp_fp.read()
print(rsp_data)
Use sslkeylogfile
Example Usage
Use sslkeylog, which is compatible with both Python2 and Python3. I'm modifying your code to save the SSL key logs while making a connection to Stack Overflow.
import urllib2
import sslkeylog
# Save SSL keys to "sslkeylog.txt" in this directory
# Note that you only have to do this once while this is in scope
sslkeylog.set_keylog("sslkeylog.txt")
# Make an HTTPS connection to Stack Overflow
requrl = "https://stackoverflow.com"
req = urllib2.Request(url=requrl)
rsp_fp = urllib2.urlopen(req)
Verification
Then if we check sslkeylog.txt, we can see that there is now an entry:
bash$ cat sslkeylogfile.txt
CLIENT_RANDOM a655a2e200ddc96c1571fe29af1962013ccbab1b9e9b865db112a9c1492c449a 3280c9fbee32df623074f80519f278420971aaa6eb91ab0f1f973d505a03ddbcc4fba2ca83f6d733addebdb0358e606d
I have a curl request that I want to convert to urllib in python2.
curl which works and gives son response:
curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" \
-H "Accept: application/json" \
"https://api.xyz.com/apiv1.2/reports/nodes?start_date=2014-04-01&end_date=2014-04-21"
I tried the following code and it keeps on redirecting me to login html page as response. How can I convert the above curl request to urllib?
headers = {"Content-Type": "application/json", "AUTH_TOKEN":'1234yyzxx'}
data = urllib.urlencode(values)
request = urllib2.Request(ENDPOINT + '?' + data, headers=headers)
response = urllib2.urlopen(request)
text = response.read()
print text
Found it using requests library.
import requests
import json
response = requests.get(ENDPOINT, headers=headers, params=values)
text = json.loads(response.text)
However, I wouldn't mind answers in all non-deprecated libraries (urllib, urllib2, urllib3 etc).
Which one is faster?
I have taken a look at other questions related to multipart/form POST requests in Python but unfortunately, they don't seem to address my exact question. Basically, I normally use CURL in order to hit an API service that allows me to upload zip files in order to create HTML5 assets. The CURL command I use looks like this:
curl -X POST -H "Authorization: api: 222111" --form "type=html" --form "file=Folder1/Folder2/example.zip" "https://example.api.com/upload?ins_id=123"
I am trying to use a python script to iterate through a folder of zip files in order to upload all of these files and receive a "media ID" back. This is what my script looks like:
import os
import requests
import json
ins_id = raw_input("Please enter your member ID: ")
auth = raw_input("Please enter your API authorization token: ")
for filename in os.listdir("zips"):
if filename.endswith(".zip"):
file_path = os.path.abspath(filename)
url = "https://example.api.com/upload?
ins_id="+str(ins_id)
header = {"Authorization": auth}
response = requests.post(url, headers=header, files={"form_type":
(None, "html"), "form_file_upload": (None, str(file_path))})
api_response = response.json()
print api_response
This API service requires the file path to be included when submitting the POST. However, when I use this script, the response indicates that "file not provided". Am I including this information correctly in my script?
Thanks.
Update:
I think I am heading in the right direction now (thanks to the answer provided) but now, I receive an error message stating that there is "no such file or directory". My thinking is that I am not using os.path correctly but even if I change my code to use "relpath" I still get the same message. My script is in a folder and I have a completely different folder called "zips" (in the same directory) which is where all of my zip files are stored.
To upload files with the request library, you can include the file handler directly in the JSON as described in the documentation. This is the corresponding example that I have taken from there:
url = 'http://httpbin.org/post'
files = {'file': open('path_to_your_file', 'rb')}
r = requests.post(url, files=files)
If we integrate this in your script, it would look as follows (I also made it slightly more pythonic):
import os
import requests
import json
folder = 'zips'
ins_id = raw_input("Please enter your member ID: ")
auth = raw_input("Please enter your API authorization token: ")
url = "https://example.api.com/upload?"
header = {"Authorization": auth}
for filename in os.listdir(folder):
if not filename.endswith(".zip"):
continue
file_path = os.path.abspath(os.path.join(folder, filename))
ins_id="+str(ins_id)"
response = requests.post(
url, headers=header,
files={"form_type": (None, "html"),
"form_file_upload": open(file_path, 'rb')}
)
api_response = response.json()
print api_response
As I don't have the API end point, I can't actually test this code block - but it should be something along these lines.
During the test for my Django 1.9 project I get an error:
Python swears on this code:
def test_students_list(self):
# make request to the server to get homepage page
response = self.client.get(self.url)
# do we have student name on a page?
self.assertIn('Vitaliy', response.content)
How to set the same encoding for the arguments in function assertIn?
I tried so:
self.assertIn(u"Vitaliy", response.content.decode('utf8'))
The result is the same...
P.S. I have Python 2.7.6 on Ubuntu 14.04
Did you try to define your Python source code encoding using:
# -- coding: utf-8 --
As suggest in PEP 0263.
See https://docs.djangoproject.com/en/1.9/ref/request-response/#django.http.HttpResponse.content
The HTTPResponse.content is noted in the docs as being a bytestring (https://github.com/django/django/blob/master/django/http/response.py#L225), it should be encoded as DEFAULT_CHARSET which by default is utf-8 but in both our cases this doesn't seem to get through to the test.
My solution is to tell Python request.content should have unicode encoding:
def test_students_list(self):
# make request to the server to get homepage page
response = self.client.get(self.url)
# do we have student name on a page?
self.assertIn('Vitaliy', unicode(response.content, encoding='utf-8'))
Use self.assertContains(response, 'Vitaliy') instead.