I tried to send an email after an user object gets created. I use celery to perform the task. My application was deployed to AWS Elastic Beanstalk (Linux2 Python3.7). I have one RDS configured for my environment. Whenever I created a new user, my celery worker got the same error as follow:
This is the error I got in celery-worker.log
[2020-08-17 10:51:20,667: INFO/MainProcess] Received task: account.tasks.send_user_email_when_user_created_by_admin[0c202693-43b7-44b7-a9b9-0362bc38afac]
[2020-08-17 10:51:20,675: WARNING/ForkPoolWorker-1] Sending email for new usr
[2020-08-17 10:51:20,693: ERROR/ForkPoolWorker-1] Task account.tasks.send_user_email_when_user_created_by_admin[0c202693-43b7-44b7-a9b9-0362bc38afac] raised unexpected: OperationalError('could not translate host name "aa1rm1r9klym9tk.ce9nktrqundw.us-west-2.rds.amazonaws.com" to address: Name or service not known\n')
Traceback (most recent call last):
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
self.connect()
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/db/backends/base/base.py", line 194, in connect
self.connection = self.get_new_connection(conn_params)
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 178, in get_new_connection
connection = Database.connect(**conn_params)
File "/var/app/venv/staging-LQM1lest/lib64/python3.7/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "aa1rm1r9klym9tk.ce9nktrqundw.us-west-2.rds.amazonaws.com" to address: Name or service not known
This is the code I used for sending an email to user
#receiver(post_save, sender=User)
def create_profile_handler(sender, instance, created, **kwargs):
if created:
profile = models.Profile(user=instance)
profile.save()
transaction.on_commit(tasks.send_user_email_when_user_created_by_admin.delay(instance.id))
I don't know where this host name came from as I couldn't find such rds under my account. I did the nslookup aa1rm1r9klym9tk.ce9nktrqundw.us-west-2.rds.amazonaws.com 8.8.8.8 and it couldn't find such domain. What caused this issue? How to fix it?
The database you're trying to reach doesn't exist under that hostname. There are no DNS records for that domain name: https://www.nslookup.io/dns-records/aa1rm1r9klym9tk.ce9nktrqundw.us-west-2.rds.amazonaws.com/cloudflare/. You probably misconfigured your database settings in Django.
Related
I want to list all the Google Cloud Schedule jobs within a project, but to use the ListJobsRequest() class the parent parameter is required: projects/PROJECT_ID/locations/LOCATION_ID. Since I have jobs in different locations I would like to list all jobs, is it possible to do?
I already tried projects/PROJECT_ID/locations/* and projects/PROJECT_ID/locations.
In both cases I got the following error:
Traceback (most recent call last):
File "C:\Users\user\Desktop\TI\GCP\bq-to-scheduler\main.py", line 97, in <module>
list_scheduler_jobs()
File "C:\Users\user\Desktop\TI\GCP\bq-to-scheduler\main.py", line 51, in list_scheduler_jobs
page_result = client.list_jobs(request=request)
File "C:\Users\user\Desktop\TI\GCP\bq-to-scheduler\venv\lib\site-packages\google\cloud\scheduler_v1\services\cloud_scheduler\client.py", line 548, in list_jobs
response = rpc(
File "C:\Users\user\Desktop\TI\GCP\bq-to-scheduler\venv\lib\site-packages\google\api_core\gapic_v1\method.py", line 154, in __call__
return wrapped_func(*args, **kwargs)
File "C:\Users\user\Desktop\TI\GCP\bq-to-scheduler\venv\lib\site-packages\google\api_core\retry.py", line 283, in retry_wrapped_func
return retry_target(
File "C:\Users\user\Desktop\TI\GCP\bq-to-scheduler\venv\lib\site-packages\google\api_core\retry.py", line 190, in retry_target
return target()
File "C:\Users\user\Desktop\TI\GCP\bq-to-scheduler\venv\lib\site-packages\google\api_core\grpc_helpers.py", line 52, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.PermissionDenied: 403 The principal (user or service account) lacks IAM permission "cloudscheduler.jobs.list" for the resource "projects/projeto1-358102/locations/*" (or the resource may not exist).
Help?
You can list all your available locations in your project using the code below. I also included calling list_jobs() to send a request to list the available jobs on the location.
I got the code on listing location in this document but I edited the authentication to use google.auth library instead of oauth2client.client since this is already deprecated.
from googleapiclient import discovery
import google.auth
from google.cloud import scheduler_v1
def get_project_locations():
credentials, project = google.auth.default(scopes=['https://www.googleapis.com/auth/cloud-platform'])
service = discovery.build('cloudscheduler', 'v1', credentials=credentials)
name = f'projects/{project}'
loc_arr = []
request = service.projects().locations().list(name=name)
while True:
response = request.execute()
for location in response.get('locations', []):
loc_arr.append(location['labels']['cloud.googleapis.com/region'])
request = service.projects().locations().list_next(previous_request=request, previous_response=response)
if request is None:
break
return project,loc_arr
def list_jobs(project_id,location_list):
client = scheduler_v1.CloudSchedulerClient()
for location in location_list:
request = scheduler_v1.ListJobsRequest(parent = f"projects/{project_id}/locations/{location}")
page_result = client.list_jobs(request=request)
for response in page_result:
print(response.name)
print(response.http_target)
print(response.schedule)
project_id,location_list = get_project_locations()
list_jobs(project_id,location_list)
Output:
From GCP scheduler:
I recently deployed a site that I developed in Django onto a production server running Ubuntu 19.10 and Apache/2.4.41. I've been able to get my site functioning in every respect with the exception of sending emails using SMTP. When running my site on the development server on my local machine Django is able to send emails without issue, but on the production server any attempt by the site to send an email hangs and eventually results in a server error. In Apache's error.log I'm finding this at the bottom of the traceback: TimeoutError: [Errno 110] Connection timed out. Curiously, this problem persists if I turn on port 8000 and run this site with Django's development server from the remote machine.
I've made sure that my settings.py is configured properly (as I mentioned this works fine on my local computer).
settings.py
...
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_PORT = 587
EMAIL_USE_TLS = True
EMAIL_HOST_USER = site_config.get('EMAIL_USER')
EMAIL_HOST_PASSWORD = site_config.get('EMAIL_PASS')
My suspicion has been that the issue must be related to my firewall blocking outgoing traffic on port 587, but I double checked my settings and couldn't see how this could be the case. I even manually set rules to allow outgoing traffic on port 587 and 25 to be sure, but did not have any success.
ufw status
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip
To Action From
-- ------ ----
22/tcp ALLOW IN Anywhere
80/tcp ALLOW IN Anywhere
22/tcp (v6) ALLOW IN Anywhere (v6)
80/tcp (v6) ALLOW IN Anywhere (v6)
587 ALLOW OUT Anywhere
25/tcp ALLOW OUT Anywhere
587 (v6) ALLOW OUT Anywhere (v6)
25/tcp (v6) ALLOW OUT Anywhere (v6)
I've tried testing that I'm able to establish SMTP connections at all using the advice offered in this post and got a timeout error there as well.
All of this seems to point to some sort of error in the way I've configured my firewall and/or Apache, but, after reading a number of posts on SO and elsewhere, I feel I've run out of ideas.
Lastly, here's what I'm seeing in the error log:
/var/log/apache2/error.log
log: ERROR - Internal Server>
Traceback (most recent call last):
File "/home/user/mysite/venv>
response = get_response(request)
File "/home/user/mysite/venv>
response = self.process_exception_by_middleware(e,>
File "/home/user/mysite/venv>
response = wrapped_callback(request, *callback_arg>
File "/home/user/mysite/venv>
return self.dispatch(request, *args, **kwargs)
File "/home/user/mysite/venv>
return bound_method(*args, **kwargs)
File "/home/user/mysite/venv>
response = view_func(request, *args, **kwargs)
File "/home/user/mysite/venv>
return super().dispatch(*args, **kwargs)
File "/home/user/mysite/venv>
return handler(request, *args, **kwargs)
File "/home/user/mysite/venv>
return self.form_valid(form)
File "/home/user/mysite/venv>
form.save(**opts)
File "/home/user/mysite/venv>
user_email, html_email_template_name=html_email_te>
File "/home/user/mysite/venv>
email_message.send()
File "/home/user/mysite/venv>
return self.get_connection(fail_silently).send_mes>
File "/home/user/mysite/venv>
new_conn_created = self.open()
File "/home/user/mysite/venv>
self.connection = self.connection_class(self.host,>
File "/usr/lib/python3.7/smtplib.py", line 251, in _>
(code, msg) = self.connect(host, port)
File "/usr/lib/python3.7/smtplib.py", line 336, in c>
self.sock = self._get_socket(host, port, self.time>
File "/usr/lib/python3.7/smtplib.py", line 307, in _>
self.source_address)
File "/usr/lib/python3.7/socket.py", line 727, in cr>
raise err
File "/usr/lib/python3.7/socket.py", line 716, in cr>
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
I'd happily add more information to this if there is anything important that I've left out. This is my first attempt at deploying a site to a remote server so keep in mind this is all a bit new to me.
If you use Linode to deploy your website, note that the port 587 is blocked by default.
It took me 2 days to figure it out.
I'm using google api, googleapiclient.discovery and build.instances().list() to extract all instances information under my gcp project. But I'm having permission error.
I can run compute.instances().list() fine on local and compute.instances().list() and by using Application Default Credentials. But the default credentials doesn't work when I publish app to server, and it raise exception:
raise exceptions.DefaultCredentialsError(_HELP_MESSAGE)
2019-07-17T00:55:56.682326031Z google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials.
So I need to set credential explicitly. And I downloaded service account key as "credential.json" and tried to set GOOGLE_APPLICATION_CREDENTIALS as following
import googleapiclient.discovery
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]= "credential.json"
compute = googleapiclient.discovery.build('compute', 'v1')
compute.instances().list(project="projectname", zone = "us-west1-a").execute()
But I'm getting the permission error
Traceback (most recent call last):
File "lst_temp.py", line 6, in <module>
compute.instances().list(project="project-name", zone = "us-west1-a").execute()
File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 851, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/compute/v1/projects/project-name/zones/us-west1-a/instances?alt=json returned "Required 'compute.instances.list' permission for 'projects/projectname'">
I have built an api with flask-restful and flask-jwt-extended and have correctly configured the validation passages for token expiration and invalidation. However, even though it has built the token expiration and invalid validation callbacks, api does not process correctly and reports the error: Signature has expired
On the server in the cloud, we have a Centos 7 x64 of 16gb ram, running the application using gunicorn in version 19.9.0. Using the miniconda to create the applications' python environments.
In tests in the production environment, the application complains of the expired token. However in a test environment, using Ubuntu 18.04.2, x64 with 16 gb ram, using the same settings with miniconda and gunicorn, the application has no problems executing it, returning the correct message when the token expires.
My jwt.py
from flask import Blueprint, Response, json, request
from flask_jwt_extended import (JWTManager, create_access_token,
create_refresh_token, get_jwt_identity,
jwt_required)
from app.models.core import User
from .schemas import UserSchema
from .utils import send_reponse, user_roles
def configure_jwt(app):
JWT = JWTManager(app)
#JWT.expired_token_loader
def my_expired_token_callback(expired_token):
return Response(
response=json.dumps({
"message": "Expired token"
}),
status=401,
mimetype='application/json'
)
#JWT.invalid_token_loader
def my_invalid_token_callback(invalid_token):
return Response(
response=json.dumps({
"message": "Invalid token"
}),
status=422,
mimetype='application/json'
)
Error log:
[2019-05-23 15:42:02 -0300] [3745] [ERROR] Exception on /api/company [POST]
Traceback (most recent call last):
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_restful/__init__.py", line 458, in wrapper
resp = resource(*args, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask/views.py", line 88, in view
return self.dispatch_request(*args, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_restful/__init__.py", line 573, in dispatch_request
resp = meth(*args, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 102, in wrapper
verify_jwt_in_request()
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 31, in verify_jwt_in_request
jwt_data = _decode_jwt_from_request(request_type='access')
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 266, in _decode_jwt_from_request
decoded_token = decode_token(encoded_token, csrf_token)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/utils.py", line 107, in decode_token
allow_expired=allow_expired
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/flask_jwt_extended/tokens.py", line 138, in decode_jwt
leeway=leeway, options=options)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/jwt/api_jwt.py", line 104, in decode
self._validate_claims(payload, merged_options, **kwargs)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/jwt/api_jwt.py", line 134, in _validate_claims
self._validate_exp(payload, now, leeway)
File "/home/company/miniconda3/envs/api_ms/lib/python3.6/site-packages/jwt/api_jwt.py", line 175, in _validate_exp
raise ExpiredSignatureError('Signature has expired')
jwt.exceptions.ExpiredSignatureError: Signature has expired
I'm trying to understand why the application is able to correctly return the token expiration message in the test environment, where in the production environment it returns the error code 500 Internal Server Error. In addition to fixing this problem in our application.
Based on this link found inside the project repository, I discovered that the problem is related to the flask configuration option called PROPAGATE_EXCEPTIONS, which must be True.
The issue in the flask-jwt-extended repository that helped me find the answer.
This comment states that Flask Restful needs to ignore JWT and JWT Extended Exceptions and provides a simple snippet that solves the issue.
Copying the code from above link,
from flask_jwt_extended.exceptions import JWTExtendedException
from jwt.exceptions import PyJWTError
class FixedApi(Api):
def error_router(self, original_handler, e):
if not isinstance(e, PyJWTError) and not isinstance(e, JWTExtendedException) and self._has_fr_route():
try:
return self.handle_error(e)
except Exception:
pass # Fall through to original handler
return original_handler(e)
I created a program for accessing a server via Paramiko and sockets.
#make imports
from socket import *
from datetime import datetime
from pickle import load, dump
from Crypto.Hash import SHA256
from subprocess import check_output as exeCMD
from sqlite3 import connect as SQLconnect
import paramiko, sys, threading, os
#get password from file
pasword = load(open("usrData/pswd.txt", "rb"))
#class for initiating server connection with client
class Server(paramiko.ServerInterface):
#initialize object
def __init__(self):
self.event = threading.Event()
#check password for user entry
def check_auth_password(self, username, password):
#where the error is
givenpswdHash = SHA256.new(password)
print(givenpswdHash.hexdigest())
if (username in unameList) and (givenpswdHash.hexdigest() == pasword):
return paramiko.AUTH_SUCCESSFUL
return paramiko.AUTH_FAILED
#what to execute in command line
def terminal(hostIP, hostPort, hostKeyPath, hostKeyPswd):
#create sockets before this etc...
#create server instance
server = Server()
#get server onto session
#where we call out server function
session.start_server(server=server)
#continue talking to client
When I launch the server, and get a client to connect to it, I get this error :
No handlers could be found for logger "paramiko.transport"
Traceback (most recent call last):
File "./terminalServer.py", line 212, in <module>
main()
File "./terminalServer.py", line 209, in main
terminal(ip, port, keyPath, keyPswd)
File "./terminalServer.py", line 142, in terminal
session.start_server(server=server)
File "/usr/local/lib/python2.7/dist-packages/paramiko/transport.py", line 471, in start_server
raise e
ValueError: CTR mode needs counter parameter, not IV
It has something to do with the Crypto I added for password authentication. If anyone knows howto solve this issue, please
leave a comment. Thank you in advance.
All I had to do is replace all the alpha versions of pycrypto with the stable version. The current stable version (Sept. 1st 2015) for pycrypto is 2.6.1 and for paramiko it's 1.14.2.