Can't connect superset to dremio - apache-superset

Question

I am running apache-superset using docker-compose by following the instructions here (https://superset.apache.org/docs/installation/installing-superset-using-docker-compose/) using docker-compose-non-dev.yml.
I have also added sqlalchemy-dremio to superset/docker/requirements-local.txt, in order to add dremio support as mentioned here (https://superset.apache.org/docs/databases/docker-add-drivers)
For dremio, I have a seperate container running on dremio/dremio-oss image using
docker run -p 9047:9047 -p 31010:31010 -p 45678:45678 -p 32010:32010 dremio/dremio-oss
and then made an account in dremio using the web interface at localhost:9047
But when I try to add dremio as a database in superset I get the following errors
on pressing test connection I get the following error
The connection string I'm using is
dremio+flight://dremio:dremio123#host.docker.internal:32010/dremio;SSL=0
At first I thought it might be a network error or an error in dremio, but I can connect to dremio using the python script here https://github.com/dremio-hub/arrow-flight-client-examples/blob/main/python/example.py
python example.py -host host.docker.internal -query 'SELECT 1'
This script runs successfully from both outside the container from host_os using localhost and from inside the superset_app container using host.docker.internal as host. Therefore I don't think its a network configuration problem, also this confirms that the sqlalchemy-dremio package was installed properly inside the superset containers.
Here is the docker logs for this error from superset_app container
2022-09-30 16:34:09,635:WARNING:superset.views.base:SupersetErrorsException
Traceback (most recent call last):
File "/app/superset/databases/commands/test_connection.py", line 123, in run
raise DBAPIError(None, None, None)
sqlalchemy.exc.DBAPIError: (builtins.NoneType) None
(Background on this error at: https://sqlalche.me/e/14/dbapi)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 114, in wraps
raise ex
File "/app/superset/views/base_api.py", line 111, in wraps
duration, response = time_function(f, self, *args, **kwargs)
File "/app/superset/utils/core.py", line 1572, in time_function
response = func(*args, **kwargs)
File "/app/superset/utils/log.py", line 244, in wrapper
value = f(*args, **kwargs)
File "/app/superset/views/base_api.py", line 84, in wraps
return f(self, *args, **kwargs)
File "/app/superset/databases/api.py", line 708, in test_connection
TestConnectionDatabaseCommand(item).run()
File "/app/superset/databases/commands/test_connection.py", line 148, in run
raise DatabaseTestConnectionFailedError(errors) from ex
superset.databases.commands.exceptions.DatabaseTestConnectionFailedError: [SupersetError(message='(builtins.NoneType) None\n(Background on this error at: https://sqlalche.me/e/14/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Dremio', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
***************
['UID=dremio', 'PWD=dremio123', 'HOST=host.docker.internal', 'PORT=32010', 'Schema=dremio', 'SSL=0']
***************

Answer 1

Ensure you are installing the latest version of sqlalchemy_dremio. You may need to install from source as setup.py wasn't updated accordingly (around time of writing). You will also need to add some SQLAlchemy base functions to sqlalchemy_dremio. Have a look at the following issue: https://github.com/narendrans/sqlalchemy_dremio/issues/20