I'm trying to run localstack for the first time. I am using link mint 17. However, I keep getting this error:
Starting local dev environment. CTRL-C to quit.
Starting mock ES service (http port 4578)...
Starting local Elasticsearch (http port 4571)...
Starting mock S3 (http port 4572)...
Starting mock SNS (http port 4575)...
Starting mock SQS (http port 4576)...
Starting mock SES (http port 4579)...
Starting mock SSM (http port 4583)...
Starting mock API Gateway (http port 4567)...
Starting mock DynamoDB (http port 4569)...
Starting mock DynamoDB Streams service (http port 4570)...
Starting mock Firehose service (http port 4573)...
Starting mock Lambda service (http port 4574)...
Starting mock Kinesis (http port 4568)...
Starting mock Redshift (http port 4577)...
Starting mock Route53 (http port 4580)...
Starting mock CloudFormation (http port 4581)...
Starting mock CloudWatch (http port 4582)...
2018-10-29T22:06:03:WARNING:infra.py: Service "elasticsearch" not yet available, retrying...
2018-10-29T22:06:11:ERROR:localstack.services.generic_proxy: Error forwarding request: HTTPConnectionPool(host='127.0.0.1', port=4564): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f050959e5c0>: Failed to establish a new connection: [Errno 111] Connection refused',)) Traceback (most recent call last):
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
raise err
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
conn = self._new_conn()
File "/home/user/venv35/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f050959e5c0>: Failed to establish a new connection: [Errno 111] Connection refused
What should i do to fix this? I already installed the dependencies, like java, maven, npm, pip, and python
Related
I am struggling in sending a email from cloud-composer 2 environment(composer-2.0.11-airflow-2.2.3) via smtp.gmail.com.
Please find my configs below :
Note I tried using smtp_password_cmd and smtp_password_secret but both give the same error.
I have also setup a network connectivity test which shows smtp.gmail.com is reachable from my VPC
Error Msg:
File "/opt/python3.8/lib/python3.8/smtplib.py", line 310, in _get_socket
return socket.create_connection((host, port), timeout,
File "/opt/python3.8/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/opt/python3.8/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address
Any help/pointers would be appreciated.
Thanks
This is the error I keep getting
[DEBUG] 2018-11-09T16:56:03.543Z eaa6c2df-e43f-11e8-ae27-cf0081fe803c ConnectionError received when sending HTTP request.
Traceback (most recent call last):
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connection.py", line 134, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/var/runtime/botocore/vendored/requests/packages/urllib3/util/connection.py", line 88, in create_connection
raise err
File "/var/runtime/botocore/vendored/requests/packages/urllib3/util/connection.py", line 78, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connection.py", line 204, in connect
conn = self._new_conn()
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connection.py", line 139, in _new_conn
(self.host, self.timeout))
botocore.vendored.requests.packages.urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPSConnection object at 0x7f77b6b3fd30>, 'Connection to dynamodb.eu-west-2.amazonaws.com timed out. (connect timeout=60)')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/runtime/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 597, in urlopen
_stacktrace=sys.exc_info()[2])
File "/var/runtime/botocore/vendored/requests/packages/urllib3/util/retry.py", line 271, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
botocore.vendored.requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='dynamodb.eu-west-2.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7f77b6b3fd30>, 'Connection to dynamodb.eu-west-2.amazonaws.com timed out. (connect timeout=60)'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/runtime/botocore/endpoint.py", line 222, in _get_response
proxies=self.proxies, timeout=self.timeout)
File "/var/runtime/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/var/runtime/botocore/vendored/requests/adapters.py", line 419, in send
raise ConnectTimeout(e, request=request)
botocore.vendored.requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='dynamodb.eu-west-2.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7f77b6b3fd30>, 'Connection to dynamodb.eu-west-2.amazonaws.com timed out. (connect timeout=60)'))
The crazy part is my code worked just fine yesterday. And now all of a sudden my lambda function seems not to be able to access the dynamodb. They are connected via a VPC endpoint and it worked fine yesterday. This has happened to me before and when I just left it for the night it ended up working the next day. This is unacceptable behaviour though and I need to find out what is causing it.
Any ideas?
I couldn't get this to work simply using the new vpc endpoint feature so I went for the old way of doing things which is using a nat gateway. Here's a good link on how to do it:
https://gist.github.com/reggi/dc5f2620b7b4f515e68e46255ac042a7
My short question, how to setup Django-channels v2 on AWS Elasticbeanstalk? The long and less inviting questions underneath.
Thank you in advance!
I am trying to set up an elasticbeanstalk instance using Django Channels 2.02. I followed this https://hackernoon.com/setting-up-django-channels-on-aws-elastic-beanstalk-716fd5a49c4a tutorial. Tutorial made for Channels 1.
The Elasticbeanstalk instance was created with an application load balancer. The Redis 4.0 instance is running 0.0.0.0:6379, confirmed by netstat. Custom TCP, Port Range: 6379, Source: 0.0.0.0/0 on redis instance.
alb_listener.config in .ebekstension:
option_settings: aws:elbv2:listener:80:
DefaultProcess: http
ListenerEnabled: 'true'
Protocol: HTTP aws:elasticbeanstalk:environment:process:http:
Port: '8000'
Protocol: HTTP
Daphne server startet with command "Daphne .asgi:application". Listening on TCP address 0.0.0.0:8000. It returns:
2018-03-12 08:59:30,778 ERROR Exception inside application: [Errno -2] Name or service not known
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels/consumer.py", line 54, in __call__
await await_many_dispatch([receive, self.channel_receive], self.dispatch)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels/utils.py", line 47, in await_many_dispatch
result = task.result()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 167, in receive
task.result()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 187, in receive_loop
real_channel, message = await self.receive_single(general_channel)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 224, in receive_single
async with self.connection(index) as connection:
File "/opt/python/run/venv/local/lib/python3.6/site-packages/channels_redis/core.py", line 403, in __aenter__
self.conn = await aioredis.create_redis(**self.kwargs)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/aioredis/commands/__init__.py", line 174, in create_redis
loop=loop)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/aioredis/connection.py", line 107, in create_connection
timeout, loop=loop)
File "/usr/lib64/python3.6/asyncio/tasks.py", line 339, in wait_for
return (yield from fut)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/aioredis/stream.py", line 19, in open_connection
lambda: protocol, host, port, **kwds)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 733, in create_connection
infos = f1.result()
File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib64/python3.6/socket.py", line 743, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
[Errno -2] Name or service not known
The Websocket returns no error in browser console.
It works on my local runserver setup. What is the problem? Can you please help me write a config script so that I don't need to start Daphne manually?
Very late reply here.
The problem is with config:hosts URL in the tutorial. Remove the redis:// portion from the URL.
For instance,
REDIS_PUBLIC_DNS = "YOUR-REDIS-DNS.amazonaws.com"
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [(REDIS_PUBLIC_DNS, 6379)],
}
}
}
You can check the connection using redis-cli,
redis-cli -h REDIS-PUBLIC-DNS-url ping ;you should expect PONG in response.
Currently, I have tried both plain-old Django SMTP and a few different api-based Django libraries for my transactional email provider (Postmark).
When I run my development server, everything works perfectly. Emails send via the Postmark API with no problem.
When I deploy to AWS with Zappa, visit my website, and do a task that is supposed to send an email (Ex. Resetting a user's password) the page continually loads until it says Endpoint request timed out.
I have tried setting the timeout of my AWS Lambda function to a longer duration in case Django decides to throw an error.
Here is the error that was thrown. Just keep in mind this error only happens in production. I created a custom management command in able to retrieve this error.
HTTPSConnectionPool(host='api.postmarkapp.com', port=443): Max retries exceeded with url: /email/batch (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6cfbd5dd30>: Failed to establish a new connection: [Errno 110] Connection timed out',)): ConnectionError
Traceback (most recent call last):
File "/var/task/handler.py", line 509, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 240, in lambda_handler
return handler.handler(event, context)
File "/var/task/handler.py", line 376, in handler
management.call_command(*event['manage'].split(' '))
File "/var/task/django/core/management/__init__.py", line 131, in call_command
return command.execute(*args, **defaults)
File "/var/task/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/var/task/users/management/commands/sendemail.py", line 13, in handle
fail_silently=False,
File "/var/task/django/core/mail/__init__.py", line 62, in send_mail
return mail.send()
File "/var/task/django/core/mail/message.py", line 348, in send
return self.get_connection(fail_silently).send_messages([self])
File "/var/task/postmarker/django/backend.py", line 66, in send_messages
responses = self.client.emails.send_batch(*prepared_messages, TrackOpens=self.get_option('TRACK_OPENS'))
File "/var/task/postmarker/models/emails.py", line 332, in send_batch
return self.EmailBatch(*emails).send(**extra)
File "/var/task/postmarker/models/emails.py", line 247, in send
responses = [self._manager._send_batch(*batch) for batch in chunks(emails, self.MAX_SIZE)]
File "/var/task/postmarker/models/emails.py", line 247, in <listcomp>
responses = [self._manager._send_batch(*batch) for batch in chunks(emails, self.MAX_SIZE)]
File "/var/task/postmarker/models/emails.py", line 276, in _send_batch
return self.call('POST', '/email/batch', data=emails)
File "/var/task/postmarker/models/base.py", line 72, in call
return self.client.call(*args, **kwargs)
File "/var/task/postmarker/core.py", line 106, in call
**kwargs
File "/var/task/postmarker/core.py", line 129, in _call
method, url, json=data, params=kwargs, headers=default_headers, timeout=self.timeout
File "/var/task/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/var/task/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/var/task/requests/adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.postmarkapp.com', port=443): Max retries exceeded with url: /email/batch (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6cfbd5dd30>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I have allowed all incoming and outgoing traffic to my AWS security group in an attempt to fix this. Still to no avail.
Any help would be greatly, greatly appreciated. Cheers.
The explanation is simple: A Lambda instance running in a VPC cannot access the internet:
When you add VPC configuration to a Lambda function, it can only access resources in that VPC. If a Lambda function needs to access both VPC resources and the public Internet, the VPC needs to have a Network Address Translation (NAT) instance inside the VPC.
The solution is also simple, if annoying: run a NAT Instance or NAT Gateway in the VPC. (An alternate solution is to take your Lambda out of the VPC, but that is a much bigger change.)
I am running Django / Zappa in Lambda with a NAT instance for connecting to Amazon Simple Email Service and it works fine.
Why google adwords api stops on call this link:
https://adwords.google.com/api/adwords/mcm/v201502/CustomerService?wsdl
With this error - should I load some certificate before and how?
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
Using Python 2.7.10.
Full source code:
create_adwords_client_without_yaml.py
Full error code:
Traceback (most recent call last):
File "C:/Users/Crezary Wagner/PycharmProjects/learn-adwords/src/examples/create_adwords_client_without_yaml.py", line 56, in <module>
CLIENT_CUSTOMER_ID)
File "C:/Users/Crezary Wagner/PycharmProjects/learn-adwords/src/examples/create_adwords_client_without_yaml.py", line 50, in main
customer = adwords_client.GetService('CustomerService').get()
File "C:\root\Python27\lib\site-packages\googleads\adwords.py", line 256, in GetService
proxy=proxy_option, cache=self.cache, timeout=3600)
File "C:\root\Python27\lib\site-packages\suds\client.py", line 115, in __init__
self.wsdl = reader.open(url)
File "C:\root\Python27\lib\site-packages\suds\reader.py", line 150, in open
d = self.fn(url, self.options)
File "C:\root\Python27\lib\site-packages\suds\wsdl.py", line 136, in __init__
d = reader.open(url)
File "C:\root\Python27\lib\site-packages\suds\reader.py", line 74, in open
d = self.download(url)
File "C:\root\Python27\lib\site-packages\suds\reader.py", line 92, in download
fp = self.options.transport.open(Request(url))
File "C:\root\Python27\lib\site-packages\suds\transport\https.py", line 62, in open
return HttpTransport.open(self, request)
File "C:\root\Python27\lib\site-packages\suds\transport\http.py", line 67, in open
return self.u2open(u2request)
File "C:\root\Python27\lib\site-packages\suds\transport\http.py", line 132, in u2open
return url.open(u2request, timeout=tm)
File "C:\root\Python27\lib\urllib2.py", line 431, in open
response = self._open(req, data)
File "C:\root\Python27\lib\urllib2.py", line 449, in _open
'_open', req)
File "C:\root\Python27\lib\urllib2.py", line 409, in _call_chain
result = func(*args)
File "C:\root\Python27\lib\urllib2.py", line 1240, in https_open
context=self._context)
File "C:\root\Python27\lib\urllib2.py", line 1197, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
Python uses certificates from 'system ssl certificate store' to certify https connection, if there is not any appropriate ssl cert in the store error occurs like this.
Download ssl certificate (Open your https link in browser and click lock icon in address bar > More Information > View Certificate > Details > Export) and install it on your system as stated this link http://windows.microsoft.com/en-us/windows/import-export-certificates-private-keys#1TC=windows-7
Not sure if that's the problem here, but worth checking it.
Python 2.7.9 enabled certificate validation by default for HTTP connections.
The server you're connecting to does not have a certificate that is trusted by your client. pysphere should configure SSL appropriately for this use case.
Try making your request like:
requests.get('https://adwords.google.com/api/adwords/mcm/v201502/CustomerService?wsdl', verify=False)
Try this, it helped me:
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
I encountered this issue. I had my phone setup using the same DNS block list and it wasn't immediately apparent after I'd enabled the tool and resumed work on this particular project. I suggest scrutinizing your setup and verify that there aren't any adblockers (DNS level in my case ala NextDNS/hosted PiHole) enabled. Hours upon hours spent trying out python versions, certificates, reinstalling things. Hope this helps someone!