How to add proxy settings in Paho-MQTT? - python-2.7

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Client paho-mqtt MqttServer
# main.py
import paho.mqtt.publish as publish
from json import dumps
from ssl import PROTOCOL_TLSv1
import urllib2
class MqttClient():
host = 'mqtt.xyz.com'
port = '1883'
auth = {}
topic = '%s/streams'
tls = None
def __init__(self, auth, tls=None):
self.auth = auth
self.topic = '%s/streams' % auth['username']
if tls:
self.tls = tls
self.port = '8883'
def publish(self, msg):
try:
publish.single (topic=self.topic,payload=msg,hostname=self.host,
tls=self.tls,port=self.port)
except Exception, ex:
print ex
if __name__ == '__main__':
auth = {'username': 'your username', 'password': ''}
#tls_dict = {'ca_certs': 'ca_certs.crt', 'tls_version': PROTOCOL_TLSv1} # sslvers.
msg_dict={'protocol':'v2','device':'Development Device','at':'now','data':{'temp':21,'hum':58}}
client_mqtt =MqttClient(auth=auth) # non ssl version
#client_mqtt =MqttClient(auth=auth, tls=tls_dict) # ssl version
client_mqtt.publish(dumps(msg_dict))
I guess my organization's proxy settings are blocking the request, so please guide me in including the proxy settings in the above code.
For instance if address is 'proxy.xyz.com' and port number is '0000' and my network username is 'xyz' and password is 'abc'

You haven't mentioned what sort of proxy you are talking about, but assuming you want to use a HTTP proxy.
You can not use a HTTP proxy to forward raw MQTT traffic as the two protocols are not compatible.
If the broker you want to connect to supports MQTT over Websockets then you should be able connect vai a modern HTTP proxy, but this will not work with the code you have posted.
As noted in the comments, The Python Paho client added SOCKS and HTTP proxy support under pull request 315.

Paho community added this feature on 2018.
https://github.com/eclipse/paho.mqtt.python/pull/315
I am adding the code snippet since no one still added it. Just set proxy options for the client object.
client = mqtt.Client()
client.proxy_set(proxy_type=socks.HTTP, proxy_addr=proxy_host, proxy_port=proxy_port)

Your code should something as follow
import socks
import socket
import paho.mqtt.client as mqtt
client = mqtt.Client(client_id='the-client-id')
client.username_pw_set('username', 'password')
# set proxy ONLY after client build but after connect
socks.setdefaultproxy(proxy_type=socks.PROXY_TYPE_HTTP, addr="proxy.xyz.com", port=0000, rdns=True,username="xyz", password="abc")
socket.socket = socks.socksocket
# connect
client.connect('mqtt.host', port=1883, keepalive=60, bind_address="")
client.loop_forever()

I solved it while using a socks proxy and using PySocks
import socks
import socket
import paho.mqtt.client as mqtt
client = mqtt.Client(client_id='the-client-id')
client.username_pw_set('username', 'password')
# set proxy ONLY after client build but after connect
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4, "socks.proxy.host", 1080)
socket.socket = socks.socksocket
# connect
client.connect('mqtt.host', port=1883, keepalive=60, bind_address="")
client.loop_forever()

Related

How to stop double message send in Flask based Slack application

I'm currently working on a custom bot for a Slack workspace to send messages when specific entries are given on a Google Sheet. I'm using a simple Flask setup with the Slack SDK and pydrive. I'm running into an issue where each message that is sent "asynchronusly" (i.e. not triggered by a specific Slack event) gets sent twice when running the Flask application. If I disable the Flask app, I only get one instance of the "Restarting Money Bot" sent. If the server is running, I get two instances, back to back. If the server is running and I respond to a slash command (test_command()), I only get the single response as required. I've attempted to alter the send method (using a raw API call rather than the SDK method) to no avail; I'm not sure where else to go with this issue.
Here's the code that's causing the issues. slack_handle just has simple wrapper functions for the API calls so I can keep Slack tokens and signatures local to one file:
from flask import Flask, request, redirect, make_response, render_template, send_from_directory
import os
import logging
import threading
import time
import json
from flask.wrappers import Response
import slack_handle
import drive_handle
logging.basicConfig(level=logging.DEBUG)
bot = Flask(__name__)
drive_access = drive_handle.gdrive()
#bot.route('/slack/test', methods=["POST"])
def test_command():
if not slack_handle.verifier.is_valid_request(request.get_data(), request.headers):
return make_response("Invalid request", 403)
td_test = threading.Thread(target = test_command_thread)
td_test.start()
return Response(status=200)
def test_command_thread():
slack_handle.sendMoneyBot("Money Bot is active :heavy_dollar_sign:")
slack_handle.sendMoneyBot("Last refreshed " + str(drive_access.access_time) + " minute(s) ago")
slack_handle.sendMoneyBot("Good connection: (" + str(drive_access.access_status) + ")")
# To run prior to server start
slack_handle.sendTreasurer("Restarting Money Bot")
td = threading.Thread(target = drive_access.mainDaemon)
td.daemon = True
td.start()
# Local server debug
if __name__ == "__main__":
port = int(os.environ.get('PORT', 5000))
bot.run(host='0.0.0.0', port=port, debug=True)
Any help would be awesome

Why flask doesn't work with gunicorn and telegram bot api?

I am using telegram bot api (telebot), flask and gunicorn.
When I use command python app.py everything is work fine but when I use python wsgi.py flask is stated on http://127.0.0.1:5000/ and bot doesn't answer and if I am using gunicorn --bind 0.0.0.0:8443 wsgi:app webhook is setting but telegram bot doesn't answer. I tried to add app.run from app.py to wsgi.py but it doesn't work
app.py
import logging
import time
import flask
import telebot
API_TOKEN = '111111111:token_telegram'
WEBHOOK_HOST = 'droplet ip'
WEBHOOK_PORT = 8443 # 443, 80, 88 or 8443 (port need to be 'open')
WEBHOOK_LISTEN = '0.0.0.0' # In some VPS you may need to put here the IP addr
WEBHOOK_SSL_CERT = 'webhook_cert.pem' # Path to the ssl certificate
WEBHOOK_SSL_PRIV = 'webhook_pkey.pem' # Path to the ssl private key
WEBHOOK_URL_BASE = "https://%s:%s" % (WEBHOOK_HOST, WEBHOOK_PORT)
WEBHOOK_URL_PATH = "/%s/" % (API_TOKEN)
logger = telebot.logger
telebot.logger.setLevel(logging.DEBUG)
bot = telebot.TeleBot(API_TOKEN)
app = flask.Flask(__name__)
# Empty webserver index, return nothing, just http 200
#app.route('/', methods=['GET', 'HEAD'])
def index():
return ''
# Process webhook calls
#app.route(WEBHOOK_URL_PATH, methods=['POST'])
def webhook():
if flask.request.headers.get('content-type') == 'application/json':
json_string = flask.request.get_data().decode('utf-8')
update = telebot.types.Update.de_json(json_string)
bot.process_new_updates([update])
return ''
else:
flask.abort(403)
# Handle all other messages
#bot.message_handler(func=lambda message: True, content_types=['text'])
def echo_message(message):
bot.reply_to(message, message.text)
# Remove webhook, it fails sometimes the set if there is a previous webhook
bot.remove_webhook()
#
time.sleep(1)
# Set webhook
bot.set_webhook(url=WEBHOOK_URL_BASE + WEBHOOK_URL_PATH,
certificate=open(WEBHOOK_SSL_CERT, 'r'))
if __name__ == "__main__":
# Start flask server
app.run(host=WEBHOOK_LISTEN,
port=WEBHOOK_PORT,
ssl_context=(WEBHOOK_SSL_CERT, WEBHOOK_SSL_PRIV),
debug=True)
wsgi.py
from app import app
if __name__ == "__main__":
app.run()
It's a bad practice to show python webservers to the world. It's not secure.
Good practice - using a reverse proxy, e.g. nginx
Chain
So the chain shoud be:
api.telegram.org -> your_domain -> your_nginx -> your_webserver -> your_app
SSL
Your ssl certificates shoud be checked on nginx level. On success just pass request to your webserver (flask or something else). It's also a tip about how to use infitity amount of bots on one host/port :)
How to configure nginx reverse proxy - you can find in startoverflow or google.
Telegram Webhooks
telebot is so complicated for using webhooks.
Try to use aiogram example. It's pretty simple:
from aiogram import Bot, Dispatcher, executor
from aiogram.types import Message
WEBHOOK_HOST = 'https://your.domain'
WEBHOOK_PATH = '/path/to/api'
bot = Bot('BOT:TOKEN')
dp = Dispatcher(bot)
#dp.message_handler()
async def echo(message: Message):
return message.answer(message.text)
async def on_startup(dp: Dispatcher):
await bot.set_webhook(f"{WEBHOOK_HOST}{WEBHOOK_PATH}")
if __name__ == '__main__':
executor.start_webhook(dispatcher=dp, on_startup=on_startup,
webhook_path=WEBHOOK_PATH, host='localhost', port=3000)
app.run(host=0.0.0.0, port=5000, debug=True)
Change port number and debug arguments based on your case.

Sending Emails on GAE through smtp.gmail.com in Python

After reading Google's documentation it should be possible to send an email via smtp.gmail.com on port 465 or 587 on GAE standard.
Reference: https://cloud.google.com/appengine/docs/standard/python/sockets/#limitations_and_restrictions_if_lang_is_java_java_7_runtime_only_endif
What is not documented is how to use the socket library.
I am able to send an email via smtplib running the python script locally.
server = smtplib.SMTP_SSL("smtp.gmail.com", 587)
server.ehlo()
server.login(gmail_access["email"], gmail_access["password"])
server.sendmail(gmail_access["email"], report.owner, msg.as_string())
server.close()
When trying to run the code with GAE's dev_appserver I get the nondescript error "[Errno 13] Permission denied"
Any assistance would be greatly appreciated.
Oddly enough the error only occurs when trying to run the code locally with dev_appserver.py. After deploying the code to App Engine it worked.
import socket
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
msg = MIMEMultipart("alternative")
msg["Subject"] = subject
msg["From"] = gmail_access["email"]
msg["To"] = report.owner
msg.attach(MIMEText(body, "html"))
server = smtplib.SMTP_SSL("smtp.gmail.com", 465)
server.ehlo()
server.login(gmail_access["email"], gmail_access["password"])
server.sendmail(gmail_access["email"], report.owner, msg.as_string())
server.close()

Why can't my django application send an email using Google App Engine and specified SMTP settings?

I wrote a django app, and it was sending email fine. I am now porting it to use it with Google App Engine, but it won't send for some reason. What's going on?
settings.py
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_USER = 'xxxx#gmail.com'
EMAIL_HOST_PASSWORD = 'xxx'
EMAIL_PORT = 587
EMAIL_USE_TLS = True
*views.py*
from google.appengine.api import mail
import webapp2
...
mail.send_mail(to="xxx#gmail.com", sender="test#test.com", subject="test", body="Hello World")
Nothing shows up! It's strange
You can't just use, nor specify any old smtp server with app engine. You have to use the provided mail api. The emails will also appear to come from the admin of the application.
The development web server can send email messages if configured to do
so with command-line options. It can use an SMTP server or the
Sendmail application, if available. When your application is running
on App Engine, it uses the App Engine mail service to send email
messages. See the Development Web Server for more information.
https://developers.google.com/appengine/docs/python/mail/sendingmail
Here's the example they give:
import webapp2
from google.appengine.api import mail
from google.appengine.api import users
class InviteFriendHandler(webapp2.RequestHandler):
def post(self):
user = users.get_current_user()
if user is None:
login_url = users.create_login_url(self.request.path)
self.redirect(login_url)
return
to_addr = self.request.get("friend_email")
if not mail.is_email_valid(to_addr):
# Return an error message...
pass
message = mail.EmailMessage()
message.sender = user.email()
message.to = to_addr
message.body = """
I've invited you to Example.com!
To accept this invitation, click the following link,
or copy and paste the URL into your browser's address
bar:
%s
""" % generate_invite_link(to_addr)
message.send()
Sending emails on Google App Engine is restricted, only email addresses which belongs to project owners or developers could be used.
and Django has it's own mechanism to send out email, for better integration, I will suggest you to use some Django library such as Django Rocket Engine.
download the source code and put it under you App Engine project, deploy it and put this line in settings.py:
EMAIL_BACKEND = 'rocket_engine.mail.EmailBackend'
after integrated that, you don't need to use GAE Email API by yourself, you just need to use the Django email API as usual, then emails got sent.

Simple HTTP client and server in python but client making atleast 50 request simultaneously with different IP address for each request

I want to know how , HTTP CLIENT making 100 REQUEST to server at same time but with different IP address for each request or with range of IP ADDRESSES, code in python or using a CURL function in code by configuring different IP Addresses or range of IP Addresses
Please help me editing the code to make several request to sever using different IP address requesting for a Html or web page on server and reading it contents.
Luckily, python provide us an HTTP server module, it's called BaseHTTPServer. We use two classes from this module, namely BaseHTTPRequestHandler and HTTPServer. We also need to use os module to access file in your system.
First, we need to import those modules
#!/usr/bin/env python
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import os
Next, write a custom class namely KodeFunHTTPRequestHandler that implement BaseHTTPRequestHandler.
#Create custom HTTPRequestHandler class
class KodeFunHTTPRequestHandler(BaseHTTPRequestHandler):
I just implement GET command from HTTP. To do that, we need to override do_GET() function. Our objective is to request an HTML file from server using our client (explained int the next step). Script inside do_GET() function is written to handle our objective
#handle GET command
def do_GET(self):
rootdir = 'c:/xampp/htdocs/' #file location
try:
if self.path.endswith('.html'):
f = open(rootdir + self.path) #open requested file
#send code 200 response
self.send_response(200)
#send header first
self.send_header('Content-type','text-html')
self.end_headers()
#send file content to client
self.wfile.write(f.read())
f.close()
return
except IOError:
self.send_error(404, 'file not found')
Finally, add following script to run the server
def run():
print('http server is starting...')
#ip and port of servr
#by default http server port is 80
server_address = ('127.0.0.1', 80)
httpd = HTTPServer(server_address, KodeFunHTTPRequestHandler)
print('http server is running...')
httpd.serve_forever()
if __name__ == '__main__':
run()
Save it and name it whatever you want (e.g, kodefun-httpserver.py). Here is the complete script of http server:
kodefun-httpserver.py
#!/usr/bin/env python
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import os
#Create custom HTTPRequestHandler class
class KodeFunHTTPRequestHandler(BaseHTTPRequestHandler):
#handle GET command
def do_GET(self):
rootdir = 'c:/xampp/htdocs/' #file location
try:
if self.path.endswith('.html'):
f = open(rootdir + self.path) #open requested file
#send code 200 response
self.send_response(200)
#send header first
self.send_header('Content-type','text-html')
self.end_headers()
#send file content to client
self.wfile.write(f.read())
f.close()
return
except IOError:
self.send_error(404, 'file not found')
def run():
print('http server is starting...')
#ip and port of servr
#by default http server port is 80
server_address = ('127.0.0.1', 80)
httpd = HTTPServer(server_address, KodeFunHTTPRequestHandler)
print('http server is running...')
httpd.serve_forever()
if __name__ == '__main__':
run()
Write a simple HTTP client
To check if our server is working fine, we need an HTTP client. You can use your web browser to do this. But, It'll be cool if you make your own client. To accomplish this, let's make another script, let's name it kodefun-httpclient.py. Copy following script to your file:
kodefun-httpclient.py
#!/usr/bin/env python (CLIENT CODE)
import httplib
import sys
#get http server ip
http_server = sys.argv[1]
#create a connection
conn = httplib.HTTPConnection(http_server)
while 1:
cmd = raw_input('input command (ex. GET index.html): ')
cmd = cmd.split()
if cmd[0] == 'exit': #tipe exit to end it
break
#request command to server
conn.request(cmd[0], cmd[1])
#get response from server
rsp = conn.getresponse()
#print server response and data
print(rsp.status, rsp.reason)
data_received = rsp.read()
print(data_received)
conn.close
Test using GET command