Related
I'm trying to run localstack for the first time. I am using link mint 17. However, I keep getting this error:
Starting local dev environment. CTRL-C to quit.
Starting mock ES service (http port 4578)...
Starting local Elasticsearch (http port 4571)...
Starting mock S3 (http port 4572)...
Starting mock SNS (http port 4575)...
Starting mock SQS (http port 4576)...
Starting mock SES (http port 4579)...
Starting mock SSM (http port 4583)...
Starting mock API Gateway (http port 4567)...
Starting mock DynamoDB (http port 4569)...
Starting mock DynamoDB Streams service (http port 4570)...
Starting mock Firehose service (http port 4573)...
Starting mock Lambda service (http port 4574)...
Starting mock Kinesis (http port 4568)...
Starting mock Redshift (http port 4577)...
Starting mock Route53 (http port 4580)...
Starting mock CloudFormation (http port 4581)...
Starting mock CloudWatch (http port 4582)...
2018-10-29T22:06:03:WARNING:infra.py: Service "elasticsearch" not yet available, retrying...
2018-10-29T22:06:11:ERROR:localstack.services.generic_proxy: Error forwarding request: HTTPConnectionPool(host='127.0.0.1', port=4564): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f050959e5c0>: Failed to establish a new connection: [Errno 111] Connection refused',)) Traceback (most recent call last):
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
raise err
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
conn = self._new_conn()
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f050959e5c0>: Failed to establish a new connection: [Errno 111] Connection refused
What should i do to fix this? I already installed the dependencies, like java, maven, npm, pip, and python
My short question, how to setup Django-channels v2 on AWS Elasticbeanstalk? The long and less inviting questions underneath.
Thank you in advance!
I am trying to set up an elasticbeanstalk instance using Django Channels 2.02. I followed this https://hackernoon.com/setting-up-django-channels-on-aws-elastic-beanstalk-716fd5a49c4a tutorial. Tutorial made for Channels 1.
The Elasticbeanstalk instance was created with an application load balancer. The Redis 4.0 instance is running 0.0.0.0:6379, confirmed by netstat. Custom TCP, Port Range: 6379, Source: 0.0.0.0/0 on redis instance.
alb_listener.config in .ebekstension:
option_settings: aws:elbv2:listener:80:
DefaultProcess: http
ListenerEnabled: 'true'
Protocol: HTTP aws:elasticbeanstalk:environment:process:http:
Port: '8000'
Protocol: HTTP
Daphne server startet with command "Daphne .asgi:application". Listening on TCP address 0.0.0.0:8000. It returns:
2018-03-12 08:59:30,778 ERROR Exception inside application: [Errno -2] Name or service not known
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels/consumer.py", line 54, in __call__
await await_many_dispatch([receive, self.channel_receive], self.dispatch)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels/utils.py", line 47, in await_many_dispatch
result = task.result()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 167, in receive
task.result()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 187, in receive_loop
real_channel, message = await self.receive_single(general_channel)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 224, in receive_single
async with self.connection(index) as connection:
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 403, in __aenter__
self.conn = await aioredis.create_redis(**self.kwargs)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/aioredis/commands/__init__.py", line 174, in create_redis
loop=loop)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/aioredis/connection.py", line 107, in create_connection
timeout, loop=loop)
File "/usr/lib64/python3.6/asyncio/tasks.py", line 339, in wait_for
return (yield from fut)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/aioredis/stream.py", line 19, in open_connection
lambda: protocol, host, port, **kwds)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 733, in create_connection
infos = f1.result()
File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib64/python3.6/socket.py", line 743, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
[Errno -2] Name or service not known
The Websocket returns no error in browser console.
It works on my local runserver setup. What is the problem? Can you please help me write a config script so that I don't need to start Daphne manually?
Very late reply here.
The problem is with config:hosts URL in the tutorial. Remove the redis:// portion from the URL.
For instance,
REDIS_PUBLIC_DNS = "YOUR-REDIS-DNS.amazonaws.com"
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [(REDIS_PUBLIC_DNS, 6379)],
}
}
}
You can check the connection using redis-cli,
redis-cli -h REDIS-PUBLIC-DNS-url ping ;you should expect PONG in response.
Currently, I have tried both plain-old Django SMTP and a few different api-based Django libraries for my transactional email provider (Postmark).
When I run my development server, everything works perfectly. Emails send via the Postmark API with no problem.
When I deploy to AWS with Zappa, visit my website, and do a task that is supposed to send an email (Ex. Resetting a user's password) the page continually loads until it says Endpoint request timed out.
I have tried setting the timeout of my AWS Lambda function to a longer duration in case Django decides to throw an error.
Here is the error that was thrown. Just keep in mind this error only happens in production. I created a custom management command in able to retrieve this error.
HTTPSConnectionPool(host='api.postmarkapp.com', port=443): Max retries exceeded with url: /email/batch (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6cfbd5dd30>: Failed to establish a new connection: [Errno 110] Connection timed out',)): ConnectionError
Traceback (most recent call last):
File "/var/task/handler.py", line 509, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 240, in lambda_handler
return handler.handler(event, context)
File "/var/task/handler.py", line 376, in handler
management.call_command(*event['manage'].split(' '))
File "/var/task/django/core/management/__init__.py", line 131, in call_command
return command.execute(*args, **defaults)
File "/var/task/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/var/task/users/management/commands/sendemail.py", line 13, in handle
fail_silently=False,
File "/var/task/django/core/mail/__init__.py", line 62, in send_mail
return mail.send()
File "/var/task/django/core/mail/message.py", line 348, in send
return self.get_connection(fail_silently).send_messages([self])
File "/var/task/postmarker/django/backend.py", line 66, in send_messages
responses = self.client.emails.send_batch(*prepared_messages, TrackOpens=self.get_option('TRACK_OPENS'))
File "/var/task/postmarker/models/emails.py", line 332, in send_batch
return self.EmailBatch(*emails).send(**extra)
File "/var/task/postmarker/models/emails.py", line 247, in send
responses = [self._manager._send_batch(*batch) for batch in chunks(emails, self.MAX_SIZE)]
File "/var/task/postmarker/models/emails.py", line 247, in <listcomp>
responses = [self._manager._send_batch(*batch) for batch in chunks(emails, self.MAX_SIZE)]
File "/var/task/postmarker/models/emails.py", line 276, in _send_batch
return self.call('POST', '/email/batch', data=emails)
File "/var/task/postmarker/models/base.py", line 72, in call
return self.client.call(*args, **kwargs)
File "/var/task/postmarker/core.py", line 106, in call
**kwargs
File "/var/task/postmarker/core.py", line 129, in _call
method, url, json=data, params=kwargs, headers=default_headers, timeout=self.timeout
File "/var/task/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/var/task/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/var/task/requests/adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.postmarkapp.com', port=443): Max retries exceeded with url: /email/batch (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6cfbd5dd30>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I have allowed all incoming and outgoing traffic to my AWS security group in an attempt to fix this. Still to no avail.
Any help would be greatly, greatly appreciated. Cheers.
The explanation is simple: A Lambda instance running in a VPC cannot access the internet:
When you add VPC configuration to a Lambda function, it can only access resources in that VPC. If a Lambda function needs to access both VPC resources and the public Internet, the VPC needs to have a Network Address Translation (NAT) instance inside the VPC.
The solution is also simple, if annoying: run a NAT Instance or NAT Gateway in the VPC. (An alternate solution is to take your Lambda out of the VPC, but that is a much bigger change.)
I am running Django / Zappa in Lambda with a NAT instance for connecting to Amazon Simple Email Service and it works fine.
I'm running a GAE python application with 3 modules, one module run every half an hour and is in charge of fetching a file form an FTP server, processing it and saving it to CloudStorage.
Recently the communication between my application and the FTP server start failing more often, increasingly until it became unusable.
The FTP server is Highwinds, they did some changes recently, asked me some questions like the ping and traceroute from the instance running the code to their server, but I'm not able to provide that given the GAE restrictions.
I was able to overcome some errors by just repeating the same operation multiple times.
I'm including some error traces at the end of this message.
I would appreciate some help troubleshooting this issue.
files = ftp_service.nlst()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 509, in nlst
self.retrlines(cmd, files.append)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 432, in retrlines
conn = self.transfercmd(cmd)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 371, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 330, in ntransfercmd
conn = socket.create_connection((host, port), self.timeout)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/socket.py", line 569, in create_connection
raise err
error: [Errno 110] Connection timed out
ftp_service.retrbinary("RETR %s" % file_path, callback=handle_binary)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 409, in retrbinary
conn = self.transfercmd(cmd, rest)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 371, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 334, in ntransfercmd
resp = self.sendcmd(cmd)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 244, in sendcmd
return self.getresp()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ftplib.py", line 217, in getresp
raise error_temp, resp
error_temp: 425 Rejected data connection from foreign address 74.125.183.23:53274.
Why google adwords api stops on call this link:
https://adwords.google.com/api/adwords/mcm/v201502/CustomerService?wsdl
With this error - should I load some certificate before and how?
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
Using Python 2.7.10.
Full source code:
create_adwords_client_without_yaml.py
Full error code:
Traceback (most recent call last):
File "C:/Users/Crezary Wagner/PycharmProjects/learn-adwords/src/examples/create_adwords_client_without_yaml.py", line 56, in <module>
CLIENT_CUSTOMER_ID)
File "C:/Users/Crezary Wagner/PycharmProjects/learn-adwords/src/examples/create_adwords_client_without_yaml.py", line 50, in main
customer = adwords_client.GetService('CustomerService').get()
File "C:\root\Python27\lib\site-packages\googleads\adwords.py", line 256, in GetService
proxy=proxy_option, cache=self.cache, timeout=3600)
File "C:\root\Python27\lib\site-packages\suds\client.py", line 115, in __init__
self.wsdl = reader.open(url)
File "C:\root\Python27\lib\site-packages\suds\reader.py", line 150, in open
d = self.fn(url, self.options)
File "C:\root\Python27\lib\site-packages\suds\wsdl.py", line 136, in __init__
d = reader.open(url)
File "C:\root\Python27\lib\site-packages\suds\reader.py", line 74, in open
d = self.download(url)
File "C:\root\Python27\lib\site-packages\suds\reader.py", line 92, in download
fp = self.options.transport.open(Request(url))
File "C:\root\Python27\lib\site-packages\suds\transport\https.py", line 62, in open
return HttpTransport.open(self, request)
File "C:\root\Python27\lib\site-packages\suds\transport\http.py", line 67, in open
return self.u2open(u2request)
File "C:\root\Python27\lib\site-packages\suds\transport\http.py", line 132, in u2open
return url.open(u2request, timeout=tm)
File "C:\root\Python27\lib\urllib2.py", line 431, in open
response = self._open(req, data)
File "C:\root\Python27\lib\urllib2.py", line 449, in _open
'_open', req)
File "C:\root\Python27\lib\urllib2.py", line 409, in _call_chain
result = func(*args)
File "C:\root\Python27\lib\urllib2.py", line 1240, in https_open
context=self._context)
File "C:\root\Python27\lib\urllib2.py", line 1197, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
Python uses certificates from 'system ssl certificate store' to certify https connection, if there is not any appropriate ssl cert in the store error occurs like this.
Download ssl certificate (Open your https link in browser and click lock icon in address bar > More Information > View Certificate > Details > Export) and install it on your system as stated this link http://windows.microsoft.com/en-us/windows/import-export-certificates-private-keys#1TC=windows-7
Not sure if that's the problem here, but worth checking it.
Python 2.7.9 enabled certificate validation by default for HTTP connections.
The server you're connecting to does not have a certificate that is trusted by your client. pysphere should configure SSL appropriately for this use case.
Try making your request like:
requests.get('https://adwords.google.com/api/adwords/mcm/v201502/CustomerService?wsdl', verify=False)
Try this, it helped me:
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
I encountered this issue. I had my phone setup using the same DNS block list and it wasn't immediately apparent after I'd enabled the tool and resumed work on this particular project. I suggest scrutinizing your setup and verify that there aren't any adblockers (DNS level in my case ala NextDNS/hosted PiHole) enabled. Hours upon hours spent trying out python versions, certificates, reinstalling things. Hope this helps someone!