Python: Accessing HTTPS url over Proxy using Requests - python-2.7

import shutil
import requests
import json
proxy = {
'user' : 'user',
'pass' : 'password',
'host' : "test.net",
'port' : 8080
}
url = 'https://github.com/timeline.json'
response = requests.get(url,verify=True, proxies={"https" : \
"http://%(user)s:%(pass)s#%(host)s:%(port)d" % proxy})
with open(r'..\test.json','wb') as out_file:
out_file.write(response.text)
print response
I'm trying to access a HTTPS link (e.g https://github.com/timeline.json) over proxy in office environment using Requests.
Accessing HTTP link seems to be working fine. Getting SSL error in HTTPS.
Please suggest what's missing in the code. Thanks!
Error received:
raise SSLError(e)
requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

I am using almost the same code you provided, and searched an proxy server. Everything is OK with me.
Try to take a look at document here requests proxies. And notice that https://github.com/timeline.json is deprecated by Github,try https://api.github.com/events
According to doc:
To use HTTP Basic Auth with your proxy, use the http://user:password#host/ syntax:
proxies = {
"http": "http://user:pass#10.10.1.10:3128/",
}
Are you missing a / at the end? Take a try.
import requests
url = 'https://api.github.com/events'
proxy = {
"http" : "http://211.162.xxx.xxx:80"
}
response = requests.get(url, verify=True, proxies=proxy)
print response.status_code
if response.status_code == requests.codes.ok:
response.encoding = 'utf-8'
jsontxt = response.json()
print jsontxt

Related

Problems with flask and bad request

I was programming myself a pretty nice api to get some json data from my gameserver to my webspace using json,
but everytime i am sending a request using angular i am getting this:
127.0.0.1 - - [20/Mar/2018 17:07:33] code 400, message Bad request version
("▒\x9c▒▒{▒'\x12\x99▒▒▒\xadH\x00\x00\x14▒+▒/▒,▒0▒\x13▒\x14\x00/\x005\x00")
127.0.0.1 - - [20/Mar/2018 17:07:33] "▒\x9dtTc▒\x93▒4▒M▒▒▒▒▒\x9c▒▒{▒'\x99▒▒▒▒H▒+▒/▒,▒0▒▒/5"
HTTPStatus.BAD_REQUEST -
127.0.0.1 - - [20/Mar/2018 17:07:33] code 400, message Bad request syntax
('\x16\x03\x01\x00▒\x01\x00\x00\x9d\x03\x03▒k,&▒▒ua\x8c\x82\x17\x05▒QwQ$▒0▒▒\x9f▒B1\x98\x19W▒▒▒▒\x00\x00\x14▒+▒/▒,▒0▒\x13▒\x14\x00/\x005\x00')
127.0.0.1 - - [20/Mar/2018 17:07:33] "▒\x9d▒k,&▒▒ua\x8c\x82▒QwQ$▒0▒▒\x9f▒B1\x98W▒▒▒▒▒+▒/▒,▒0▒▒/5"
HTTPStatus.BAD_REQUEST -
127.0.0.1 - - [20/Mar/2018 17:07:33] code 400, message Bad request syntax
('\x16\x03\x01\x00▒\x01\x00\x00▒\x03\x03)▒▒\x1e\xa0▒\t\r\x14g%▒▒\x17▒▒\x80\x8d}▒F▒▒\x08U▒ġ▒▒\x06▒\x00\x00\x1c▒+▒/▒,▒0▒')
g%▒▒▒▒\x80\x8d}▒F▒U▒ġ▒▒▒▒+▒/▒,▒0▒" HTTPStatus.BAD_REQUEST -
My api
from flask import Flask, jsonify
from flaskext.mysql import MySQL
from flask_cors import CORS, cross_origin
app = Flask(__name__)
CORS(app)
app.config['CORS_HEADERS'] = 'Content-Type'
cors = CORS(app, resources={r"/punishments": {"origins": "http://localhost:5000" "*"}})
mysql = MySQL()
# MySQL configurations
app.config['MYSQL_DATABASE_USER'] = 'test'
app.config['MYSQL_DATABASE_PASSWORD'] = 'Biologie1'
app.config['MYSQL_DATABASE_DB'] = 'test'
app.config['MYSQL_DATABASE_HOST'] = 'localhost'
mysql.init_app(app)
#app.route('/punishments', methods=['GET'])
#cross_origin(origin='localhost:5000',headers=['Content- Type','Authorization'])
def get():
cur = mysql.connect().cursor()
cur.execute('''select * from test.punishments''')
r = [dict((cur.description[i][0], value)
for i, value in enumerate(row)) for row in cur.fetchall()]
return jsonify({'punishments' : r})
if __name__ == '__main__':
app.run()
My client function
export class ApiUserService {
private _postsURL = "https://localhost:5000/punishments";
constructor(private http: HttpClient) {
}
getPosts(): Observable<Punishments[]> {
let headers = new HttpHeaders();
headers = headers.set('Content-Type', 'application/json; charset=utf-8');
return this.http
.get(this._postsURL,{
headers: {'Content-Type':'application/json; charset=utf-8'}
})
.map((response: Response) => {
return <Punishments[]>response.json();
})
.catch(this.handleError);
}
private handleError(error: Response) {
return Observable.throw(error.statusText);
}
}
I had the same error as yours.
My flask server was installed inside respberry-pi and I was trying to access it using https://ip:5000.
The problem was I was using https instead of http.
When I changed it to http://ip:5000, it worked.
I also faced same problem
use only http not https :-
http://ip:portnumber
Recently I also faced this, the problem came from the SSL config on my app.
In my .env I set the SSL_DISABLE to False, then I change it to True.
change SSL_DISABLE=False to SSL_DISABLE=True
So, the point here is are: check your URL, maybe be it something like: https://127.0.0.1:5000, just change it to http://127.0.0.1:5000.
Hope it helps to someone who also facing this issue in the future.
in my case, i was trying to debug SocketIo server running on flask. I was trying to access the server using wss:// which was causing the bad request. Changing it to ws:// resolved the issue.
In my case i resolve the problem reverting flask version to Flask==1.1.4 and jinja dependency to Jinja2==3.0.3

Python telnet connection refuse exception

I'm using following function to make telnet connection verification
telnetlib.Telnet("172.28.5.240", "8080")
When the connection refused it shows exception message. Is it possible to hide the message and detect as success or failed through if condition?
You can use try-except-finally blocks
try:
#
#
response = 'Success'
except:
response = 'Failed'
finally:
print response
Based on Suku's answer I develop my code. that is a working answer. And following is my script for reference.
try:
conn = telnetlib.Telnet("172.28.5.240", "80")
response = 'Success'
except:
response = 'Failed'
finally:
print response
None of the options helped me.
Maybe someone will come in handy.
100% working version:
Used to check the availability of the RDP server in ZABBIX:
import telnetlib
response = ''
HOST = '192.168.1.201'
PORT = 3389
tn = telnetlib.Telnet()
try:
tn.open(HOST, PORT, 3)
response = '2'
except Exception:
response = '0'
finally:
tn.close()
print(response)

How can I create https connection on python 2.7?

The situation
I'm trying to connect to a server on https protocol with python script. Could someone give me the working example that sends a GET request to https server, or web resource to how to create https connection with python?
An attempt so far
I have learned that the module httplib on python supports creation of http connection, but not https connection?
import httplib
conn = httplib.HTTPConnection('https://adsche.skplanet.com/api/startNewTurn')
header = {"Content-type" : "application/json"}
conn.request('GET', '/announce?info_hash=%da', '', header)
r1 = conn.getresponse()
print r1.status, r1.reason
data1 = r1.read()
print data1

Python requests 503 erros when trying to access localhost:8000

I am facing a bit of a situation,
Scenario: I got a django rest api running on my localhost:8000 and I want to access the api using my command line. I have tried urllib2 and python requests libs to talk to the api but failed(i'm getting a 503 error). But when I pass google.com as the url, I am getting the expected response. So I believe my approach is correct but I'm doing something wrong. please see the code below :
import urllib, urllib2, httplib
url = 'http://localhost:8000'
httplib.HTTPConnection.debuglevel = 1
print "urllib"
data = urllib.urlopen(url);
print "urllib2"
request = urllib2.Request(url)
opener = urllib2.build_opener()
feeddata = opener.open(request).read()
print "End\n"
Envioroments:
OS Win7
python v2.7.5
Django==1.6
Markdown==2.3.1
colorconsole==0.6
django-filter==0.7
django-ping==0.2.0
djangorestframework==2.3.10
httplib2==0.8
ipython==1.0.0
jenkinsapi==0.2.14
names==0.3.0
phonenumbers==5.8b1
requests==2.1.0
simplejson==3.3.1
termcolor==1.1.0
virtualenv==1.10.1
Thanks
I had a similar problem, but found that it was the company's proxy that was preventing from pinging myself.
503 Reponse when trying to use python request on local website
Try:
>>> import requests
>>> session = requests.Session()
>>> session.trust_env = False
>>> r = session.get("http://localhost:5000/")
>>> r
<Response [200]>
>>> r.content
'Hello World!'
If you are registering your serializers with DefaultRouter then your api will appear at
http://localhost:8000/api/ for an html view of the index
http://localhost:8000/api/.json for a JSON view of the index
http://localhost:8000/api/appname for an html view of the individual resource
http://localhost:8000/api/appname/.json for a JSON view of the individual resource
you can check the response in your browser to make sure your URL is working as you expect.

XMLRPC (client/proxy) use behind a firewall - Python 2.7

I'm new to XMLRPC but I need to use it (xmlrpclib in Python 2.7) to communicate with a server (www.neos-server.org) which accepts xml files. I'm behind a firewall that severely restricts outgoing and incoming traffic, but I'm able to browse the web mostly unimpeded using a webproxy.
Test 1 and 2 (below) work, but Test 3 results in an HTTP 502 error (cantconnect)
import urllib2
import xmlrpclib
import httplib
class ProxyTransport(xmlrpclib.Transport):
def request(self, host, handler, request_body, verbose):
self.verbose = verbose
url = 'http://' + host + handler
if self.verbose: "ProxyTransport URL: [%s]" % url
request = urllib2.Request(url)
request.add_data(request_body)
request.add_header("User-Agent", self.user_agent)
request.add_header("Content-Type", "text/xml")
proxy_handler = urllib2.ProxyHandler({"http":"MYPROXY:8080"})
opener = urllib2.build_opener(proxy_handler)
f = opener.open(request)
return(self.parse_response(f))
# TEST 1 - HTML fetching
def test1():
html = urllib2.urlopen("http://www.google.com").read() # note no proxy setup here
print html
# TEST 2 - XMLRPC sample server fetching
def test2():
p = ProxyTransport()
test_url = "http://betty.userland.com"
#test_server = xmlrpclib.Server(test_url) # gives <ProtocolError for betty.userland.com/RPC2: 403 WebTrap>
test_server = xmlrpclib.Server(test_url, transport=p)
test_api = "examples.getStateName(9)"
print "API: %s" % test_api
r = eval("test_server.%s" % test_api)
print "Result: ", r
# TEST 3 - XMLRPC server (NEOS)
def test3():
# Setup proxy and server
p = ProxyTransport()
NEOS_HOST = "www.neos-server.org"
NEOS_PORT = 3332
neos = xmlrpclib.Server("http://%s:%d" % (NEOS_HOST, NEOS_PORT), transport = p)
# Talk
print "Ping Neos..."
neos.ping()
test1()
test2()
test3()
I've tried a couple different solutions (https://gist.github.com/nathforge/980961, https://mail.python.org/pipermail/python-list/2006-July/367049.html) but they don't seem to work. I assume I need to be able to ping before I can send/receive xml files and results. What should I try next?