I'm using tweepy and trying to run the basic script as shown by this video. I was previously receiving 401 errors (unsynchronized time zones) but am using the provided keys. I fixed that problem and now I'm getting this result:
Traceback (most recent call last):
File "algotest.py", line 25, in <module>
twitterStream.filter(track=["North"])
File "/usr/local/lib/python2.7/dist-packages/tweepy-2.3-py2.7.egg/tweepy/streaming.py", line 313, in filter
File "/usr/local/lib/python2.7/dist-packages/tweepy-2.3-py2.7.egg/tweepy/streaming.py", line 235, in _start
File "/usr/local/lib/python2.7/dist-packages/tweepy-2.3-py2.7.egg/tweepy/streaming.py", line 151, in _run
File "/usr/local/lib/python2.7/dist-packages/requests-2.2.1-py2.7.egg/requests/sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-2.2.1-py2.7.egg/requests/sessions.py", line 438, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-2.2.1-py2.7.egg/requests/adapters.py", line 327, in send
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='stream.twitter.com', port=443): Max retries exceeded with url: /1.1/statuses/filter.json?track=North&delimited=length (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)
Any way around this? Is there some sort of reset option I can trigger?
Thanks in advance
Turns out the solution is simply to wait a day. Who would've thought!
I was also getting same error while using the python-twitter module in my script but it got resolved automatically when I tried after an interval. As there is limit for the number of try at a particular interval hence we get this error when we exceed that maximum try limit.
Related
I have a Google Cloud Function in Python 3.7 reading from a Pub/Sub subscription in synchronous pull mode.
After running fine 1/hour for 24 hours, it threw this exception stack trace:
Traceback (most recent call last): File
"/env/local/lib/python3.7/site-packages/google/api_core/grpc_helpers.py",
line 57, in error_remapped_callable
return callable_(*args, **kwargs) File "/env/local/lib/python3.7/site-packages/grpc/_channel.py", line 824,
in call
return _end_unary_response_blocking(state, call, False, None) File "/env/local/lib/python3.7/site-packages/grpc/_channel.py", line
726, in _end_unary_response_blocking
raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status =
StatusCode.DEADLINE_EXCEEDED details = "Deadline Exceeded"
debug_error_string =
"{"created":"#1580454091.145703535","description":"Error received from
peer
ipv4:74.125.202.95:443","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Deadline
Exceeded","grpc_status":4}"
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File
"/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py",
line 346, in run_http_function
result = _function_handler.invoke_user_function(flask.request) File
"/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py",
line 217, in invoke_user_function
return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py",
line 210, in call_user_function
return self._user_function(request_or_event) File "/user_code/main.py", line 39, in iteration
response = sub.pull(sub_path, MAX_MESSAGES) File "/env/local/lib/python3.7/site-packages/google/cloud/pubsub_v1/_gapic.py",
line 40, in
fx = lambda self, *a, **kw: wrapped_fx(self.api, *a, **kw) # noqa File
"/env/local/lib/python3.7/site-packages/google/cloud/pubsub_v1/gapic/subscriber_client.py",
line 1005, in pull
request, retry=retry, timeout=timeout, metadata=metadata File "/env/local/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py",
line 143, in call
return wrapped_func(*args, **kwargs) File "/env/local/lib/python3.7/site-packages/google/api_core/retry.py",
line 286, in retry_wrapped_func
on_error=on_error, File "/env/local/lib/python3.7/site-packages/google/api_core/retry.py",
line 184, in retry_target
return target() File "/env/local/lib/python3.7/site-packages/google/api_core/timeout.py",
line 214, in func_with_timeout
return func(*args, **kwargs) File "/env/local/lib/python3.7/site-packages/google/api_core/grpc_helpers.py",
line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc) File "", line 3, in raise_from
google.api_core.exceptions.DeadlineExceeded: 504 Deadline Exceeded
What is this about? Is it to be expected or a result of some configuration problem? If to be expected, how should it be handled?
The documentation ( view-source:https://googleapis.dev/python/pubsub/latest/subscriber/api/client.html ) on pull has nothing about this being a possible exception.
I ack the messages immediately after the pull completes. I only permit one function execution at a time. I have a 600 second acknowledgement deadline. A block of messages pulled at one time seem to be less than 100 in number. If this is about failing to ack a message, it seems like the error could be done much better.
This exception is raised by the client when there's no messages to read in the subscription. It is a known issue from the latest PubSub library versions >= 1.0.0. If necessary, you can downgrade to the version 0.45.0 where this issue was not present.
However, as a workaround you can catch the DeadlineExceeded exception and retry the operation again. Also, based on the comment of Hemang, here's a small monkeypatch that you can add to your running code, which might help to get the same behavior as in version 0.45.0.
from google.cloud.pubsub_v1.gapic import subscriber_client_config as sub_config
sub_config.config['interfaces']['google.pubsub.v1.Subscriber']['retry_params']['messaging']['initial_rpc_timeout_millis'] = 25000
Finally, keep in mind that when using synchronous pull, having many outstanding pull requests helps lower the delivery latency, which in turn might result in higher latency pull requests (and DeadlineExceeded errors). Although, if latency is crucial for the application, you could consider using StreamingPull
This is the error I keep getting
[DEBUG] 2018-11-09T16:56:03.543Z eaa6c2df-e43f-11e8-ae27-cf0081fe803c ConnectionError received when sending HTTP request.
Traceback (most recent call last):
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connection.py", line 134, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/var/runtime/botocore/vendored/requests/packages/urllib3/util/connection.py", line 88, in create_connection
raise err
File "/var/runtime/botocore/vendored/requests/packages/urllib3/util/connection.py", line 78, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connection.py", line 204, in connect
conn = self._new_conn()
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connection.py", line 139, in _new_conn
(self.host, self.timeout))
botocore.vendored.requests.packages.urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPSConnection object at 0x7f77b6b3fd30>, 'Connection to dynamodb.eu-west-2.amazonaws.com timed out. (connect timeout=60)')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/runtime/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/var/runtime/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 597, in urlopen
_stacktrace=sys.exc_info()[2])
File "/var/runtime/botocore/vendored/requests/packages/urllib3/util/retry.py", line 271, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
botocore.vendored.requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='dynamodb.eu-west-2.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7f77b6b3fd30>, 'Connection to dynamodb.eu-west-2.amazonaws.com timed out. (connect timeout=60)'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/runtime/botocore/endpoint.py", line 222, in _get_response
proxies=self.proxies, timeout=self.timeout)
File "/var/runtime/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/var/runtime/botocore/vendored/requests/adapters.py", line 419, in send
raise ConnectTimeout(e, request=request)
botocore.vendored.requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='dynamodb.eu-west-2.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7f77b6b3fd30>, 'Connection to dynamodb.eu-west-2.amazonaws.com timed out. (connect timeout=60)'))
The crazy part is my code worked just fine yesterday. And now all of a sudden my lambda function seems not to be able to access the dynamodb. They are connected via a VPC endpoint and it worked fine yesterday. This has happened to me before and when I just left it for the night it ended up working the next day. This is unacceptable behaviour though and I need to find out what is causing it.
Any ideas?
I couldn't get this to work simply using the new vpc endpoint feature so I went for the old way of doing things which is using a nat gateway. Here's a good link on how to do it:
https://gist.github.com/reggi/dc5f2620b7b4f515e68e46255ac042a7
I'm trying to finish the setup of django-allauth for my site (in development).
Using Django==1.6.5 and django-allauth==0.17.0.
After following the documentation, I have been able to get the FB dialog. When I click OK, it hangs on localhost:8000/accounts/facebook/login/token/ for about 2 minutes, before returning with an error. The console is showing:
Error accessing FB user profile
Traceback (most recent call last):
File "/home/amir/claudius/lib/python2.7/site-packages/allauth/socialaccount/providers/facebook/views.py", line 73, in login_by_token
login = fb_complete_login(request, app, token)
File "/home/amir/claudius/lib/python2.7/site-packages/allauth/socialaccount/providers/facebook/views.py", line 26, in fb_complete_login
params={'access_token': token.token})
File "/home/amir/claudius/lib/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs)
File "/home/amir/claudius/lib/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/home/amir/claudius/lib/python2.7/site-packages/requests/sessions.py", line 456, in request
resp = self.send(prep, **send_kwargs)
File "/home/amir/claudius/lib/python2.7/site-packages/requests/sessions.py", line 559, in send
r = adapter.send(request, **kwargs)
File "/home/amir/claudius/lib/python2.7/site-packages/requests/adapters.py", line 375, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPSConnectionPool(host='graph.facebook.com', port=443): Max retries exceeded with url: /me?access_token=CAAUi8RJCRZAkBAPdHFKhckONnLwjOExZCeVXpW39GZAZBLdD5rTsukQqTPi9KP6neMDxtwdhZAQvmzCS92rxR0rIZCNlzenQ2jHiyANvToy6tOWrOh5ZAYFmJFYeYvbXGNc9fuPIa0hAUqGfPzFtZB0tepoxoO7bpt01izuTYBkmS9NJChXaX9iDZAQlDTDvtLTZBvLesjFtSfwp6RusbArRzH (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)
[26/Jul/2014 06:14:36] "POST /accounts/facebook/login/token/ HTTP/1.1" 200 1205
Anyone knows the cause for this?
Well, I found out that Facebook Graph API requires IPv6 in a PHP post by Etienne Rached, of all places. Tethering through my mobile phone solved it straight away after that for the development process, and this will be a non-issue when our website is deployed.
I hope this will help someone out there.
So i have a django site that is giving me this AmbiguousTimeError. I have a job activates when a product is saved that is given a brief timeout before updating my search index. Looks like an update was made in the Daylight Savings Time hour, and pytz cannot figure out what to do with it.
How can i prevent this from happening the next time the hour shifts for DST?
[2012-11-06 14:22:52,115: ERROR/MainProcess] Unrecoverable error: AmbiguousTimeError(datetime.datetime(2012, 11, 4, 1, 11, 4, 335637),)
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/celery/worker/__init__.py", line 353, in start
component.start()
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 369, in start
self.consume_messages()
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 842, in consume_messages
self.connection.drain_events(timeout=10.0)
File "/usr/local/lib/python2.6/dist-packages/kombu/connection.py", line 191, in drain_events
return self.transport.drain_events(self.connection, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/kombu/transport/virtual/__init__.py", line 760, in drain_events
self._callbacks[queue](message)
File "/usr/local/lib/python2.6/dist-packages/kombu/transport/virtual/__init__.py", line 465, in _callback
return callback(message)
File "/usr/local/lib/python2.6/dist-packages/kombu/messaging.py", line 485, in _receive_callback
self.receive(decoded, message)
File "/usr/local/lib/python2.6/dist-packages/kombu/messaging.py", line 457, in receive
[callback(body, message) for callback in callbacks]
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 560, in receive_message
self.strategies[name](message, body, message.ack_log_error)
File "/usr/local/lib/python2.6/dist-packages/celery/worker/strategy.py", line 25, in task_message_handler
delivery_info=message.delivery_info))
File "/usr/local/lib/python2.6/dist-packages/celery/worker/job.py", line 120, in __init__
self.eta = tz_to_local(maybe_iso8601(eta), self.tzlocal, tz)
File "/usr/local/lib/python2.6/dist-packages/celery/utils/timeutils.py", line 52, in to_local
dt = make_aware(dt, orig or self.utc)
File "/usr/local/lib/python2.6/dist-packages/celery/utils/timeutils.py", line 211, in make_aware
return localize(dt, is_dst=None)
File "/usr/local/lib/python2.6/dist-packages/pytz/tzinfo.py", line 349, in localize
raise AmbiguousTimeError(dt)
AmbiguousTimeError: 2012-11-04 01:11:04.335637
EDIT: I fixed it temporarily with this code in celery:
celery/worker/job.py # line 120
try:
self.eta = tz_to_local(maybe_iso8601(eta), self.tzlocal, tz)
except:
self.eta = None
I don't want to have changes in a pip installed app, so i need to fix what i can in my code:
This runs when i save my app:
self.task_cls.apply_async(
args=[action, get_identifier(instance)],
countdown=15
)
I'm assuming that i need to somehow detect if i'm in the ambiguous time and adjust countdown.
I think i'm going to have to clear the tasks to fix this, but how can i prevent this from happening the next time the hour shifts for DST?
It's not clear what you're doing (you haven't shown any code), but basically you need to take account for the way the world works. You can't avoid having ambiguous times when you convert from local time to UTC (or to a different zone's local time) when the time goes back an hour.
Likewise you ought to be aware that there are "gap" or "impossible" times, where a reasonable-sounding local time simply doesn't occur.
I don't know what options Python gives you, but ideally an API should let you resolve ambiguous times however you want - whether that's throwing an error, giving you the earlier occurrence, the later occurrence, or something else.
Apparently, Celery solved this issue:
https://github.com/celery/celery/issues/1061
We've got a Django-based web application that is used for receiving POST data from iOS devices (push notification tokens).
All in all, the application seems to be working fine, and we're receiving a 1000-2000 POSTs with valid data every hour. However, I'm occasionally receiving error logs from Django with the following data:
Traceback (most recent call last):
File "/opt/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/opt/local/lib/python2.7/site-packages/django/views/decorators/vary.py", line 19, in inner_func
response = func(*args, **kwargs)
File "/opt/local/lib/python2.7/site-packages/django_piston-0.2.3-py2.7.egg/piston/resource.py", line 160, in __call__
request.data = request.POST
File "/opt/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 180, in _get_post
self._load_post_and_files()
File "/opt/local/lib/python2.7/site-packages/django/http/__init__.py", line 372, in _load_post_and_files
self._post, self._files = QueryDict(self.body, encoding=self._encoding), MultiValueDict()
File "/opt/local/lib/python2.7/site-packages/django/http/__init__.py", line 328, in body
self._body = self.read()
File "/opt/local/lib/python2.7/site-packages/django/http/__init__.py", line 384, in read
return self._stream.read(*args, **kwargs)
File "/opt/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 98, in read
result = self.buffer + self._read_limited()
File "/opt/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 92, in _read_limited
result = self.stream.read(size)
UnreadablePostError: request data read error
And the WSGIRequest dump says POST: <could not parse>
I've been trying to find more information on this error, and a lot of what I'm seeing points to this error being caused by a user canceling a POST request before the post completes. Is this an error that I should be concerned about, or should I just set up the server to filter out these error messages? I'd say that I get maybe 8-10 automated emails per day about this.
I'm not sure whether you're still waiting for the answer but basically the comments contain most of the information. So, just to close the question up...
These errors mean that a malformed request arrived to the server. Someone might have cancelled their request or it got mangled on its way (e.g. bad internet connection), etc.
You can't solve these errors directly. However you can take a look at the page (if it's always the same one) and check if for example the loading doesn't take too long. You can also try and cause the error manually to see when exactly it happens and why. Unless you find something relevant, I don't think you need to worry about it much.