I am running Apache Superset on AWS-ECS to facilitate a connection directly with our RDS. This connection works, but has to be configured manually.
Is there a way to programmatically configure source databases with Apache Superset?
I have tried setting SQLALCHEMY_DATABASE_URI, but that is only for the Superset back-end configuration and settings.
Superset has an API, so you can create/update/delete databases via HTTP requests. Authorization is non-trivial, since you need login first to get a CSRF token and store cookies:
import requests
from bs4 import BeautifulSoup
from yarl import URL
superset_url = URL('https://superset.example.org/')
session = requests.Session()
# get CSRF token
response = session.get(superset_url / "login/")
soup = BeautifulSoup(response.text, "html.parser")
csrf_token = soup.find("input", {"id": "csrf_token"})["value"]
# get cookies
session.post(
superset_url / "login/",
data=dict(username=username, password=password, csrf_token=csrf_token),
)
# create database
database = {
"database_name": "my db",
"sqlalchemy_uri": "gsheets://",
...
}
session.post(
superset_url / "api/v1/database/",
json=database,
headers={"X-CSRFToken": csrf_token},
)
Related
I would like to implement Facebook login using Flask.
Here is the function which is called to request service from Facebook
#users_view.route('/facebook/')
def facebook():
credentials = current_app.config['OAUTH_CREDENTIALS']['facebook']
f_client = WebApplicationClient(credentials['id'])
authorization_endpoint = 'https://www.facebook.com/dialog/oauth'
request_uri = f_client.prepare_request_uri(
authorization_endpoint,
redirect_uri=request.base_url + "/auth",
scope=["email"],
auth_type = "reauthenticate",
)
print("REQUEST: {}".format(request_uri))
return redirect(request_uri)
#users_view.route("/facebook/auth")
def facebook_callback():
credentials = current_app.config['OAUTH_CREDENTIALS']['facebook']
f_client = WebApplicationClient(credentials['id'])
token_endpoint = 'https://graph.facebook.com/oauth/access_token'
code = request.args.get("code")
token_url, headers, body = f_client.prepare_token_request(
token_endpoint,
authorization_response=request.url,
redirect_url=request.base_url,
code=code
)
print("ALL: url:{} headers:{} url:{} ".format(token_url, headers, body))
Which forwards me to this URL:
https://www.facebook.com/dialog/oauth?response_type=code&client_id=5453357158093262&redirect_uri=https%3A%2F%2F127.0.0.1%3A5000%2Fuser%2Ffacebook%2F%2Fauth&scope=email&auth_type=reauthenticate&ret=login&fbapp_pres=0&logger_id=1cc03c7d-9a19-43ba-978c-4ed8cb7aa559&tp=unspecified&cbt=1663931173196&ext=1663941992&hash=AeaYsntT-4HEQj4ZtfI
That throws the following Error:
In my Facebook developers account, I have following redirect URL configuration:
Kindly, advice how can I fix this issue.
Facebook API is expecting the requester to use HTTPS as you've set in your Facebook Developer Account.
Probably you are running your flask app using HTTP protocol (not HTTPS) while on your Facebook Developer account you did white-list only HTTPS, but since you didn't specify HTTP it will be rejected.
A) You cannot allow HTTP(without S) from Facebook Panel because Oauth2 need HTTPS.
Try:
B) Install pyOpenSSL
pip3 install pyOpenSSL
Create ssl_my_app.py and run with ssl_context:
from flaskr import create_app
from flask import current_app, g
import sqlite3 # if using databases
app = create_app()
with app.app_context():
g.db = sqlite3.connect(
current_app.config['DATABASE'],
detect_types=sqlite3.PARSE_DECLTYPES
)
g.db.row_factory = sqlite3.Row
with current_app.open_resource('schema.sql') as f:
g.db.executescript(f.read().decode('utf-8'))
app.run(ssl_context='adhoc')
run the app using ssl_my_app.py:
python3 ssl_my_app.py
This will run the app using HTTPS (self-signed certificate).
So when you call Facebook API your application's request will be in the white-list.
I want to watch updates on telegram messages in an django application and interact with django orm.
I found telethon library, it works with user api which is what I want.
Below code simply works on its own.
from telethon import TelegramClient
from telethon import events
api_id = 231232131
api_hash = '32131232312312312edwq'
client = TelegramClient('anon', api_id, api_hash)
#client.on(events.NewMessage)
async def my_event_handler(event):
if 'hello' in event.raw_text:
await event.reply('hi!')
client.start()
But telethon requires phone message verification and it needs to work in a seperate thread.
I couldn't find a way to put this code in a django application. And when django starts, I dont know how to bypass phone verification.
It should always work in an seperate loop and interact with django orm. Which is very confusing for me.
You can simply use django-telethon and use the API endpoints for signing bot and user session.
run the following command to start the server:
python manage.py runserver
run the following command to start telegram client:
python manage.py runtelegram
go to admin panel and telegram app section. create a new app. get data from the your Telegram account.
request code from telegram:
import requests
import json
url = "127.0.0.1:8000/telegram/send-code-request/"
payload = json.dumps({
"phone_number": "+12345678901",
"client_session_name": "name of the client session"
})
headers = {
'Content-Type': 'application/json'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
send this request for sign in:
import requests
import json
url = "127.0.0.1:8000/telegram/login-user-request/"
payload = json.dumps({
"phone_number": "+12345678901",
"client_session_name": "name of the client session",
"code": "1234",
"password": "1234"
})
headers = {
'Content-Type': 'application/json'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
It's not the answer I wanted initially.
But, I think this is better approach.
Instead of trying to put all of this in django application. It's better to run it seperately and let django application communicate this with rest framework.
My app has a Django 3.1 backend with django-allauth and dj-rest-auth (actively supported fork of django-rest-auth).
My mobile and web frontends can already sign in using Facebook and Google via REST. I'm now trying to add Salesforce as a 3rd REST social login method, but am running into issues.
I've followed the django-allauth instructions for Salesforce:
Created a Salesforce Connected App with id and openid scopes (along with some others), and set the callback URL to https://www.mywebdomain.com/accounts/salesforce/login/callback/
Created a SocialApplication in Django with client ID, secret, and login URL in the "Key" field (https://login.salesforce.com/)
Included allauth.socialaccount.providers.salesforce in INSTALLED_APPS
I've been using client-side JSforce to kick off the Salesforce auth request in the frontend, but I'm open to other methods if they are simpler/better/etc.
Running jsforce.browser.login() in my clients' JS code opens a Salesforce login popup. After entering Salesforce login credentials, the Salesforce system redirected to my defined callback URL, resulting in a page that shows the following text:
Social Network Login Failure
An error occurred while attempting to login via your social network account.
The URL in the address bar on that page looks something like this:
https://www.mywebdomain.com/accounts/salesforce/login/callback/#access_token=00D3t000004QWRm%21ARwAQPfHWiM6jdB43dlyW6qjEw._34mjzGi_Jv6YCXp0QssT.9F9lCge5_YaH8gqTy3Od6SywCs8X9zOGv145SyviBVeGdn0&instance_url=https%3A%2F%2Fna123.salesforce.com&id=https%3A%2F%2Flogin.salesforce.com%2Fid%2F00D3t000004QWRmEAO%2F0053t000008QBetAAG&issued_at=1606802917608&signature=KvxAX0WBCFQYY%2BO25id9%2FXxpbh2q2d2vWdQ%2FFV5FCBw%3D&state=jsforce0.popup.c0ockgct29g&scope=id+api+web+refresh_token+openid&token_type=Bearer
I tried to debug and print the error in my backend, but both auth_error.code and auth_error.exception were blank/empty.
I also tried sending the access_token from that URL's hash to my Salesforce API endpoint (see below), but that resulted in a 400 error ("Incorrect value").
Here is how I've defined my SocialLoginViews in my views.py, based on dj-rest-auth's social auth documentation:
from dj_rest_auth.registration.views import SocialLoginView
from allauth.socialaccount.providers.facebook.views import FacebookOAuth2Adapter
from allauth.socialaccount.providers.google.views import GoogleOAuth2Adapter
from allauth.socialaccount.providers.salesforce.views import SalesforceOAuth2Adapter
class FacebookLogin(SocialLoginView):
adapter_class = FacebookOAuth2Adapter
class GoogleLogin(SocialLoginView):
adapter_class = GoogleOAuth2Adapter
class SalesforceLogin(SocialLoginView):
adapter_class = SalesforceOAuth2Adapter
My urls.py:
from .views import FacebookLogin, GoogleLogin, SalesforceLogin
urlpatterns = [
...
# Sending access_token to the Facebook and Google REST endpoints works,
# but doing the same for the Salesforce REST endpoint does not (400 error: "Incorrect value")
url(r'^api/rest-auth/facebook/$', FacebookLogin.as_view(), name='fb_login'),
url(r'^api/rest-auth/google/$', GoogleLogin.as_view(), name='google_login'),
url(r'^api/rest-auth/salesforce/$', SalesforceLogin.as_view(), name='salesforce_login'),
...
]
How can Salesforce social auth be made to work in this app?
I figured it out and got it working: when posting to my dj-rest-auth Salesforce API endpoint, I was only including access_token in my POST body. I actually need both access_token and key, where key is the Salesforce login URL ("https://login.salesforce.com").
It was indeed in the django-allauth instructions for Salesforce, but I misinterpreted the wording. I now know that it says to require both access_token and key in the POST body.
I have an app written with Flask and try to use Flask-Dance (Flask-Dance Docs - Google Example) to enable Google OAuth. I got the following setup:
from flask import redirect, url_for, jsonify, Blueprint
from flask_dance.contrib.google import make_google_blueprint, google
from server.app import app
# Internal auth blueprint
auth = Blueprint('auth', __name__, url_prefix='/auth')
# Google auth blueprint
google_login = make_google_blueprint(
client_id=app.config['GOOGLE_CLIENT_ID'],
client_secret=app.config['GOOGLE_CLIENT_SECRET'],
scope=['profile', 'email']
)
def auth_google_view():
"""
Authenticate user with google
"""
# Not authorized
print(google.authorized)
if not google.authorized:
return redirect(url_for('google.login'))
# Authorized - check data
user_info = google.get('/oauth2/v2/userinfo')
if user_info.ok:
return jsonify({'status': 'ok', 'email': user_info.json() .['email']}), 200
return jsonify({'status': 'failed'})
# Add urls
auth.add_url_rule('/google', view_func=auth_google_view)
And then in the app/__init__.py:
from server.app.auth import auth, google_login
app.register_blueprint(auth)
app.register_blueprint(google_login, url_prefix='/google_login')
By clicking on button in the app I go to /auth/google and there (after redirects) I can see a google accounts list to choose from. When I select an account in the Network dev tools I see the following routing (url parameters missing):
https://accounts.google.com/_/signin/oauth?authuser=
http://127.0.0.1:8001/google_login/google/authorized?state=
http://127.0.0.1:8001/google_login/google
And then:
https://accounts.google.com/o/oauth2/auth?response_type=
...
all starts from the beginning and I see a "choose account" screen.
In the Google API account I have a redirect url:
http://127.0.0.1:8001/google_login/google/authorized
In the development environment I set OAUTHLIB_INSECURE_TRANSPORT=1 and OAUTHLIB_RELAX_TOKEN_SCOPE=1
It seems like the third URL in routing should be /auth/google and try to resolve google.authorized once again but it does not and I see result of print(google.authorized) # False only once when click on a google button inside the app.
The blueprint generated by make_google_blueprint defaults to redirecting towards / when the authentication cycle has ended; you can configure this using the parameters redirect_url or redirect_to. In your case:
google_login = make_google_blueprint(
client_id=app.config['GOOGLE_CLIENT_ID'],
client_secret=app.config['GOOGLE_CLIENT_SECRET'],
scope=['profile', 'email'],
redirect_to='auth.auth_google_view'
)
EDIT: Also make sure your app has a good secret_key set.
I wrote a django app, and it was sending email fine. I am now porting it to use it with Google App Engine, but it won't send for some reason. What's going on?
settings.py
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_USER = 'xxxx#gmail.com'
EMAIL_HOST_PASSWORD = 'xxx'
EMAIL_PORT = 587
EMAIL_USE_TLS = True
*views.py*
from google.appengine.api import mail
import webapp2
...
mail.send_mail(to="xxx#gmail.com", sender="test#test.com", subject="test", body="Hello World")
Nothing shows up! It's strange
You can't just use, nor specify any old smtp server with app engine. You have to use the provided mail api. The emails will also appear to come from the admin of the application.
The development web server can send email messages if configured to do
so with command-line options. It can use an SMTP server or the
Sendmail application, if available. When your application is running
on App Engine, it uses the App Engine mail service to send email
messages. See the Development Web Server for more information.
https://developers.google.com/appengine/docs/python/mail/sendingmail
Here's the example they give:
import webapp2
from google.appengine.api import mail
from google.appengine.api import users
class InviteFriendHandler(webapp2.RequestHandler):
def post(self):
user = users.get_current_user()
if user is None:
login_url = users.create_login_url(self.request.path)
self.redirect(login_url)
return
to_addr = self.request.get("friend_email")
if not mail.is_email_valid(to_addr):
# Return an error message...
pass
message = mail.EmailMessage()
message.sender = user.email()
message.to = to_addr
message.body = """
I've invited you to Example.com!
To accept this invitation, click the following link,
or copy and paste the URL into your browser's address
bar:
%s
""" % generate_invite_link(to_addr)
message.send()
Sending emails on Google App Engine is restricted, only email addresses which belongs to project owners or developers could be used.
and Django has it's own mechanism to send out email, for better integration, I will suggest you to use some Django library such as Django Rocket Engine.
download the source code and put it under you App Engine project, deploy it and put this line in settings.py:
EMAIL_BACKEND = 'rocket_engine.mail.EmailBackend'
after integrated that, you don't need to use GAE Email API by yourself, you just need to use the Django email API as usual, then emails got sent.