pyhs2 error connective to kerberos enabled hive - python-2.7

I am trying to connect to hive using python 2(miniconda2 installation ).
Below is the code I am trying-
connection = hive.connect(host='psvlxihpnn1', port= '10000', authMechanism='KERBEROS', user='***',password='****', configuration={'krb_host': 'psvlxihpnn1', 'krb_service': 'ITEDM'} )
kerberos host is installed on the same host machine and has service name 'ITEDM'
Strangely, I am getting below error-
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/export/home/itedm/miniconda2/lib/python2.7/site-packages/pyhs2-0.6.0-py2.7.egg/pyhs2/__init__.py", line 7, in connect
File "/export/home/itedm/miniconda2/lib/python2.7/site-packages/pyhs2-0.6.0-py2.7.egg/pyhs2/connections.py", line 46, in __init__
File "/export/home/itedm/miniconda2/lib/python2.7/site-packages/pyhs2-0.6.0-py2.7.egg/pyhs2/cloudera/thrift_sasl.py", line 66, in open
thrift.transport.TTransport.TTransportException: Could not start SASL: Error in sasl_client_start (-1) SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/INFORMATICA.COM#ITEDM not found in Kerberos database)
I am not passing 'krbtgt' as user, not sure why this error is coming.
Thanks for any help.

This connection string will work as long as the user running the script has a valid kerberos ticket:
import pyhs2
with pyhs2.connect(host='beeline_host',
port=10000,
authMechanism="KERBEROS") as conn:
with conn.cursor() as cur:
print cur.getDatabases()
Username, password and any other configuration parameters are not passed through the KDC.

Related

Redash v7 on Docker can't connect to RDS MySQL: "SSL connection error: unknown error number"

My Redash v7 server can't connect to Amazon RDS MySQL 8.0.28.
The error says "SSL connection error: unknown error number".
How can I solve this?
The server is running on Docker. Its image is redash/redash:7.0.0.b18042.
I tried replacing ca certificates file to https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem but it didn't work.
Here's the Python script reproduces the error.
Run this file on a Redash v7 container and you can see the error message.
import MySQLdb
ssl_params = {'ca': '/app/redash/query_runner/files/rds-combined-ca-bundle.pem'}
connection = MySQLdb.connect(host='<snip>',
user='<snip>',
passwd='<snip>',
db='<snip>',
port=3306,
charset='utf8',
use_unicode=True,
ssl=ssl_params,
connect_timeout=60)
cursor = connection.cursor()
cursor.execute('select count(1) from users')
data = cursor.fetchall()
print(data)
$ python example.py
Traceback (most recent call last):
File "example.py", line 13, in <module>
connect_timeout=60)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 193, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2026, 'SSL connection error: unknown error number')
This error doesn't occur on a Redash v10 container but upgrading to v10 is not an option for now.
I found a workaround by myself.
Upgrading libmysqlclient20 package solved my issue.
libmysqlclient20 package in redash/redash:7.0.0.b18042 image are statically linked to yaSSL which doesn't support TLS v1.2+.
Upgraded libmysqlclient20 package are compiled using OpenSSL which supports TLS v1.2+.
MySQL 8.0.28 dropped support for TLS v1.1. That's why I need the latest libmysqlclient20.
SEE ALSO: https://dev.mysql.com/doc/refman/5.7/en/ssl-libraries.html

Read user email contents using imaplib2

I'm writing a python code to read email from mail server(neither google nor outlook) using imaplib2 package. I'm getting the following error with my code. Can anybody tell me how could I fix it. Do I need to use only port=993 and host=imap, if so how can I connect to the server I need to
import imaplib2
import base64
email_user = 'mailid'
email_pass = 'password'
email_server = 'ns21.interactivedns.com'
Mail = imaplib2.IMAP4_SSL(email_server, 995) # used this server and port since I'm trying to read mail contents from this server
Mail.login(email_user, email_pass)
Mail.select('Inbox')
type, messages = Mail.search(None, 'ALL')
Error I'm gertting:
Traceback (most recent call last):
File "C:/Users/user01/PycharmProjects/MongoAlchemy/client.py", line 6, in <module>
M = imaplib2.IMAP4_SSL('ns21.interactivedns.com', 995)
File "C:\Users\user01\PycharmProjects\MongoAlchemy\venv\lib\site-packages\imaplib2\imaplib2.py", line 2073, in __init__
IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
File "C:\Users\user01\PycharmProjects\MongoAlchemy\venv\lib\site-packages\imaplib2\imaplib2.py", line 392, in __init__
raise self.error('unrecognised server welcome message: %s' % repr(self.welcome))
imaplib2.imaplib2.error: unrecognised server welcome message: None

WebDriverException() in Python2.7 using xvfb-run for chrome on Amazon Linux AMI Red-hat 7.2

Trying to run selenium for chrome through
/usr/bin/xvfb-run -a -e /tmp/selenium.log -w 0 /usr/bin/python2.7
and getting error like
(<class 'selenium.common.exceptions.WebDriverException'>, WebDriverException(), <traceback object at 0x7fc7cfc9c128>)
the strange thing is i am not even getting complete stack trace and the -e option for xvfb-run also is not returning any errors in file .
Here is my code
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
f = webdriver.ChromeOptions()
browser =webdriver.Chrome(executable_path='/home/local/chromedriver',chrome_options=f)
browser.get(url)
browser.quit()
i also tried with
executable_path='/usr/bin/chromedriver' and without executable_path
but getting the same error .
it was initially working with python 2.6 with firefox .
i checked maybe around 10-15 + similar questions even about how to enable more trace or debugging but none of the solutions seems to be doing the trick for me here.
selenium version 3.141.0
Edit :
i updated some code to be able to get complete stack trace and i got this error ,
WebDriverException: Message: The browser appears to have exited before
we could connect.
a bit more digging and saw its related to selenium/browser
installed old selenium version 2.53
thus above error was gone but now i am getting connection reset by peer error for everything , below is stack
(<class 'socket.error'>, error(104, 'Connection reset by peer'), <traceback object at 0x1e1e320>)
Traceback (most recent call last):
File "/var/www/Goose/src/goose/network.py", line 137, in get_html
browser.get(url)
File "/var/www/Goose/src/selenium/webdriver/remote/webdriver.py", line 245, in get
self.execute(Command.GET, {'url': url})
File "/var/www/Goose/src/selenium/webdriver/remote/webdriver.py", line 231, in execute
response = self.command_executor.execute(driver_command, params)
File "/var/www/Goose/src/selenium/webdriver/remote/remote_connection.py", line 395, in execute
return self._request(command_info[0], url, body=data)
File "/var/www/Goose/src/selenium/webdriver/remote/remote_connection.py", line 426, in _request
resp = self._conn.getresponse()
File "/usr/lib64/python2.6/httplib.py", line 1049, in getresponse
response.begin()
File "/usr/lib64/python2.6/httplib.py", line 433, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python2.6/httplib.py", line 389, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib64/python2.6/socket.py", line 479, in readline
data = self._sock.recv(self._rbufsize)
error: [Errno 104] Connection reset by peer
even requests [security] is installed as i had seen one cause to be that .
solved it ,
the issue was the chrome browser and drivers were not properly installed , and their versions not compatible ,
installed latest chrome to run on the ami server using a script i got from
https://intoli.com/blog/installing-google-chrome-on-centos/
installed perfectly ,
added few key arguments ,
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('path to driver',chrome_options=chrome_options)
and viola it started working ,

Sample connection for HIVE and Python

I am trying to connect python with hive data base .
Both are on different server.
hive resides on host xx.xxx.xxx.x and python is on my local system.
i am trying to use the below code , however its not working
import pyhive
from pyhive import hive
conn = hive.connect(host = 'xx.xxx.xx.xx', port = 8888, auth = 'KERBEROS', kerberos_service_name='adsuedscaihen01.aipcore.local', username ='user1', database = 'database1'
)
cur = conn.cursor()
cur.execute('SELECT * from table1')
result = cur.fetchall()
print(result)
while i run the above code , i am facing the below error:
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/pyhive/hive.py", line 64, in connect
return Connection(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pyhive/hive.py", line 162, in init
self._transport.open()
File "/usr/lib/python2.7/site-packages/thrift_sasl/init.py", line 79, in open
message=("Could not start SASL: %s" % self.sasl.getError()))
thrift.transport.TTransport.TTransportException: Could not start SASL: Error in sasl_client_start (-1) SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server not found in Kerberos database)

Connect to Django test database

Is there a way to connect to django test database using connections?
I have tried
cursor = connections['test_name_of_main_db'].cursor()
and also specified the test db name in settings but I still receive errors:
Traceback (most recent call last):
File "/home/tets/4d/lib/python3.5/site-packages/django/db/utils.py", line 176, in ensure_defaults
conn = self.databases[alias]
KeyError: 'auto_tests'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/tets/healthware_server/component/medical/tests.py", line 72, in test_model_observation
cursor = connections['auto_tests'].cursor()
File "/home/tets/4d/lib/python3.5/site-packages/django/db/utils.py", line 208, in __getitem__ self.ensure_defaults(alias)
File "/home/tets/4d/lib/python3.5/site-packages/django/db/utils.py", line 178, in ensure_defaults
raise ConnectionDoesNotExist("The connection %s doesn't exist" % alias)
django.db.utils.ConnectionDoesNotExist: The connection auto_tests doesn't exist
According to the docs:
The default test database names are created by prepending test_ to the value of each NAME in DATABASES.
But in your stacktrace the error says:
django.db.utils.ConnectionDoesNotExist: The connection auto_tests doesn't exist
Are you sure you are using the right alias? According to the error, it is looking for a database named auto_tests.
As another debugging step, you could print all the aliases during the test to see which ones are available
for alias in connections:
print(alias)
Of course, you could also try to use the default database and see if that works for you during the tests:
# note that it is 'connection' and not 'connections'
from django.db import connection
cursor = connection.cursor()