ADL LRS saves statement but responds with HTTP 500 - django

I've installed ADL LRS following README.txt on https://github.com/adlnet/ADL_LRS.
I'm testing it with TincanJS (https://github.com/RusticiSoftware/TinCanJS) with following code, statement gets saved but response code is 500.
lrs = new TinCan.LRS(
{
////// This works well
endpoint: "https://cloud.scorm.com/tc/public/",
username: "<Test User>",
password: "<Test Password>",
allowFail: false
////// This returns Internal Error(500)
endpoint: "my.server:8000/xapi/",
username: "<My User>",
password: "<My Password>",
allowFail: false
}
);
var statement = new TinCan.Statement(
{
actor: {
"name": "Ac Tor",
"account": {
"homePage": "http://act.or",
"name": "actor"
}
},
verb: {
id: "http://adlnet.gov/expapi/verbs/experienced"
},
target: {
id: "http://rusticisoftware.github.com/TinCanJS"
}
}
);
lrs.saveStatement(
statement,
{
callback: function (err, xhr) {
//...
}
}
);
Log at /ADL_LRS/../logs/lrs.log:
2017-03-15 10:33:31,248 [ERROR] lrs.views: 500 === /xapi/statements
Traceback (most recent call last):
File "/home/kursat/ADL_LRS/lrs/views.py", line 214, in handle_request
return processors[path][req_dict['method']](req_dict)
File "/home/kursat/ADL_LRS/lrs/utils/req_process.py", line 164, in statements_put
check_activity_metadata.delay(stmt_ids)
File "/home/kursat/env/local/lib/python2.7/site-packages/celery/app/task.py", line 453, in delay
return self.apply_async(args, kwargs)
File "/home/kursat/env/local/lib/python2.7/site-packages/celery/app/task.py", line 560, in apply_async
**dict(self._get_exec_options(), **options)
File "/home/kursat/env/local/lib/python2.7/site-packages/celery/app/base.py", line 354, in send_task
reply_to=reply_to or self.oid, **options
File "/home/kursat/env/local/lib/python2.7/site-packages/celery/app/amqp.py", line 305, in publish_task
**kwargs
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/messaging.py", line 172, in publish
routing_key, mandatory, immediate, exchange, declare)
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/connection.py", line 470, in _ensured
interval_max)
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/connection.py", line 382, in ensure_connection
interval_start, interval_step, interval_max, callback)
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/utils/__init__.py", line 246, in retry_over_time
return fun(*args, **kwargs)
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/connection.py", line 250, in connect
return self.connection
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/connection.py", line 756, in connection
self._connection = self._establish_connection()
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/connection.py", line 711, in _establish_connection
conn = self.transport.establish_connection()
File "/home/kursat/env/local/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 116, in establish_connection
conn = self.Connection(**opts)
File "/home/kursat/env/local/lib/python2.7/site-packages/amqp/connection.py", line 183, in __init__
return self._x_open(virtual_host)
File "/home/kursat/env/local/lib/python2.7/site-packages/amqp/connection.py", line 628, in _x_open
(10, 41), # Connection.open_ok
File "/home/kursat/env/local/lib/python2.7/site-packages/amqp/abstract_channel.py", line 67, in wait
self.channel_id, allowed_methods, timeout)
File "/home/kursat/env/local/lib/python2.7/site-packages/amqp/connection.py", line 241, in _wait_method
channel, method_sig, args, content = read_timeout(timeout)
File "/home/kursat/env/local/lib/python2.7/site-packages/amqp/connection.py", line 330, in read_timeout
return self.method_reader.read_method()
File "/home/kursat/env/local/lib/python2.7/site-packages/amqp/method_framing.py", line 189, in read_method
raise m
error: [Errno 104] Connection reset by peer
Thanks in advance.

There are log files in the log/ folder created during installation of the LRS. Inside there is an lrs.log file that will tell you what is causing the 500 error. I was able to send statements to a local LRS instance with no 500 error. Please respond back with the error so I can help. Thanks

Unless this is just a quirk of TinCanJS, I would think target should be object.

Related

how to fix error with ssl certificate in aws lambda

I am using this guide on the AWS deeplens. I am experimenting with the code and at one point it stopped working for the second standalone lambda function. At some point, the code started getting the exception and I only recently noticed it.
Here is the code in question:
def sendMessageToIoTTopic(iotMessage):
# this is a default topic if its not specified in the lambda envirionment
topicName = "worker-safety"
if "iot_topic" in os.environ:
topicName = os.environ['iot_topic']
print("Send message to topic: " + topicName + " before")
iotClient = boto3.client('iot-data', region_name='us-east-1')
response = iotClient.publish(
topic=topicName,
qos=1,
payload=json.dumps(iotMessage)
)
print("Send message to topic: " + topicName + "after")
print("using this topic **{}** response is {}".format(topicName, response))
The guide says to modify the default timeout from 1 second to 3 seconds. Here is the log for this code with a 3 second timeout.
2021-01-31T14:28:35.579-05:00 Before we send to iot MQTT?
2021-01-31T14:28:35.579-05:00 Send message to topic: worker-safety-demo-cloud before
2021-01-31T14:28:36.202-05:00 END RequestId: 89794691-a149-4480-b0bd-fd1a69522774
2021-01-31T14:28:36.202-05:00 REPORT RequestId: 89794691-a149-4480-b0bd-fd1a69522774 Duration: 3003.48 ms Billed Duration: 3000 ms Memory Size: 128 MB Max Memory Used: 35 MB
2021-01-31T14:28:36.202-05:00 2021-01-31T19:28:36.201Z 89794691-a149-4480-b0bd-fd1a69522774 Task timed out after 3.00 seconds
2021-01-31T14:28:38.016-05:00 START RequestId: 71562d34-28d6-471a-8a43-b1e0bab1d621 Version: $LATEST
Adjusting the timeout for the lambda function from 3 seconds to 15 seconds allows us to see the timeout error in detail. It shows that it is an SSL error.
This is the resultant log with more info:
2021-01-31T14:36:08.775-05:00 Before we send to iot MQTT?
2021-01-31T14:36:08.775-05:00 Send message to topic: worker-safety-demo-cloud before
2021-01-31T14:36:17.295-05:00 [ERROR] SSLError: SSL validation failed for https://data.iot.us-east-1.amazonaws.com/topics/worker-safety-demo-cloud?qos=1 [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091) Traceback (most recent call last): File "/var/task/lambda_function.py", line 177, in lambda_handler sendMessageToIoTTopic(iotMessage) File "/var/task/lambda_function.py", line 119, in sendMessageToIoTTopic payload=json.dumps(iotMessage) File "/var/runtime/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/var/runtime/botocore/client.py", line 663, in _make_api_call operation_model, request_dict, request_context) File "/var/runtime/botocore/client.py", line 682, in _make_request return self._endpoint.make_request(operation_model, request_dict) File "/var/runtime/botocore/endpoint.py", line 102, in make_request return self._send_request(request_dict, operation_model) File "/var/runtime/botocore/endpoint.py", line 137, in _send_request success_response, exception): File "/var/runtime/botocore/endpoint.py", line 256, in _needs_retry caught_exception=caught_exception, request_dict=request_dict) File "/var/runtime/botocore/hooks.py", line 356, in emit return self._emitter.emit(aliased_event_name, **kwargs) File "/var/runtime/botocore/hooks.py", line 228, in emit return self._emit(event_name, kwargs) File "/var/runtime/botocore/hooks.py", line 211, in _emit response = handler(**kwargs) File "/var/runtime/botocore/retryhandler.py", line 183, in __call__ if self._checker(attempts, response, caught_exception): File "/var/runtime/botocore/retryhandler.py", line 251, in __call__ caught_exception) File "/var/runtime/botocore/retryhandler.py", line 277, in _should_retry return self._checker(attempt_number, response, caught_exception) File "/var/runtime/botocore/retryhandler.py", line 317, in __call__ caught_exception) File "/var/runtime/botocore/retryhandler.py", line 223, in __call__ attempt_number, caught_exception) File "/var/runtime/botocore/retryhandler.py", line 359, in _check_caught_exception raise caught_exception File "/var/runtime/botocore/endpoint.py", line 200, in _do_get_response http_response = self._send(request) File "/var/runtime/botocore/endpoint.py", line 269, in _send return self.http_session.send(request) File "/var/runtime/botocore/httpsession.py", line 281, in send raise SSLError(endpoint_url=request.url, error=e)
2021-01-31T14:36:17.296-05:00 END RequestId: 0da11ea2-c9d3-42ce-a8d5-cd3e709bf42e
The SSL error in detail is this:
[ERROR] SSLError: SSL validation failed for https://data.iot.us-east-1.amazonaws.com/topics/worker-safety-demo-cloud?qos=1 [SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)Traceback (most recent call last):  
File "/var/task/lambda_function.py", line 177, in lambda_handler    sendMessageToIoTTopic(iotMessage)  
File "/var/task/lambda_function.py", line 119, in sendMessageToIoTTopic    payload=json.dumps(iotMessage)  
File "/var/runtime/botocore/client.py", line 357, in _api_call    return self._make_api_call(operation_name, kwargs)  
File "/var/runtime/botocore/client.py", line 663, in _make_api_call    operation_model, request_dict, request_context)  
File "/var/runtime/botocore/client.py", line 682, in _make_request    return self._endpoint.make_request(operation_model, request_dict)  
File "/var/runtime/botocore/endpoint.py", line 102, in make_request    return self._send_request(request_dict, operation_model)  
File "/var/runtime/botocore/endpoint.py", line 137, in _send_request    success_response, exception):  
File "/var/runtime/botocore/endpoint.py", line 256, in _needs_retry    caught_exception=caught_exception, request_dict=request_dict)  
File "/var/runtime/botocore/hooks.py", line 356, in emit    return self._emitter.emit(aliased_event_name, **kwargs)  
File "/var/runtime/botocore/hooks.py", line 228, in emit    return self._emit(event_name, kwargs)  
File "/var/runtime/botocore/hooks.py", line 211, in _emit    response = handler(**kwargs)  File "/var/runtime/botocore/retryhandler.py", line 183, in __call__    if self._checker(attempts, response, caught_exception):  
File "/var/runtime/botocore/retryhandler.py", line 251, in __call__    caught_exception)  File "/var/runtime/botocore/retryhandler.py", line 277, in _should_retry    return self._checker(attempt_number, response, caught_exception)  
File "/var/runtime/botocore/retryhandler.py", line 317, in __call__    caught_exception)  File "/var/runtime/botocore/retryhandler.py", line 223, in __call__    attempt_number, caught_exception)  
File "/var/runtime/botocore/retryhandler.py", line 359, in _check_caught_exception    raise caught_exception  
File "/var/runtime/botocore/endpoint.py", line 200, in _do_get_response    http_response = self._send(request)  
File "/var/runtime/botocore/endpoint.py", line 269, in _send    return self.http_session.send(request) 
 File "/var/runtime/botocore/httpsession.py", line 281, in send    raise SSLError(endpoint_url=request.url, error=e)

Temporary failure in name resolution channel 2 django

I have implemented channel2 in my django project , after some time i start it i have received this error on my server , i have apache on ubuntu for my code.
2019-02-12 10:30:51,863 ERROR Exception inside application: [Errno -3] Temporary failure in name resolution
File "/var/www/RPG/python/lib/python3.6/site-packages/channels/consumer.py", line 59, in _call_
[receive, self.channel_receive], self.dispatch
File "/var/www/RPG/python/lib/python3.6/site-packages/channels/utils.py", line 59, in await_many_dispatch
await task
File "/var/www/RPG/python/lib/python3.6/site-packages/channels/utils.py", line 51, in await_many_dispatch
result = task.result()
File "/var/www/RPG/python/lib/python3.6/site-packages/channels_redis/core.py", line 429, in receive
real_channel
File "/var/www/RPG/python/lib/python3.6/site-packages/channels_redis/core.py", line 484, in receive_single
index, channel_key, timeout=self.brpop_timeout
File "/var/www/RPG/python/lib/python3.6/site-packages/channels_redis/core.py", line 324, in _brpop_with_clean
async with self.connection(index) as connection:
File "/var/www/RPG/python/lib/python3.6/site-packages/channels_redis/core.py", line 820, in _aenter_
self.conn = await self.pool.pop()
File "/var/www/RPG/python/lib/python3.6/site-packages/channels_redis/core.py", line 70, in pop
conns.append(await aioredis.create_redis(**self.host, loop=loop))
File "/var/www/RPG/python/lib/python3.6/site-packages/aioredis/commands/__init__.py", line 178, in create_redis
loop=loop)
File "/var/www/RPG/python/lib/python3.6/site-packages/aioredis/connection.py", line 108, in create_connection
timeout, loop=loop)
File "/usr/lib/python3.6/asyncio/tasks.py", line 339, in wait_for
return (yield from fut)
File "/var/www/RPG/python/lib/python3.6/site-packages/aioredis/stream.py", line 19, in open_connection
lambda: protocol, host, port, **kwds)
File "/usr/lib/python3.6/asyncio/base_events.py", line 739, in create_connection
infos = f1.result()
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
[Errno -3] Temporary failure in name resolution
Here i attach a image which solve the problem but i need to find better solution.I don't think it's a properway to do this.

celerybeat send task fails with ssl error

I am using celerybeat to send periodic tasks to rabbitmq queue. It works as expected for some time and then send celery.backend_cleanup fails with SSL error.
[2016-10-24 04:00:02,309: DEBUG/MainProcess] Channel open
[2016-10-24 04:00:02,443: ERROR/MainProcess] Message Error: Couldn't apply scheduled task celery.backend_cleanup: [Errno 1] _ssl.c:1309: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry
[' File "/local/mnt/apps/ipcat/venvs/django16/bin/celery", line 9, in <module>\n load_entry_point(\'celery-ipcat==3.1.23\', \'console_scripts\', \'celery\')()\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/__main__.py", line 30, in main\n main()\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/celery.py", line 81, in main\n cmd.execute_from_commandline(argv)\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/celery.py", line 793, in execute_from_commandline\n super(CeleryCommand, self).execute_from_commandline(argv)))\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/base.py", line 311, in execute_from_commandline\n return self.handle_argv(self.prog_name, argv[1:])\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/celery.py", line 785, in handle_argv\n return self.execute(command, argv)\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/celery.py", line 717, in execute\n ).run_from_argv(self.prog_name, argv[1:], command=argv[0])\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/base.py", line 315, in run_from_argv\n sys.argv if argv is None else argv, command)\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/base.py", line 377, in handle_argv\n return self(*args, **options)\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/base.py", line 274, in __call__\n ret = self.run(*args, **kwargs)\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/bin/beat.py", line 79, in run\n return beat().run()\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/apps/beat.py", line 83, in run\n self.start_scheduler()\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/apps/beat.py", line 112, in start_scheduler\n beat.start()\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/beat.py", line 479, in start\n interval = self.scheduler.tick()\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/beat.py", line 221, in tick\n next_time_to_run = self.maybe_due(entry, self.publisher)\n', ' File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/beat.py", line 207, in maybe_due\n exc, traceback.format_stack(), exc_info=True)\n']
Traceback (most recent call last):
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/beat.py", line 204, in maybe_due
result = self.apply_async(entry, publisher=publisher)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/beat.py", line 260, in apply_async
entry, exc=exc)), sys.exc_info()[2])
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/beat.py", line 252, in apply_async
**entry.options)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/app/task.py", line 565, in apply_async
**dict(self._get_exec_options(), **options)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/app/base.py", line 354, in send_task
reply_to=reply_to or self.oid, **options
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/celery/app/amqp.py", line 310, in publish_task
**kwargs
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/messaging.py", line 172, in publish
routing_key, mandatory, immediate, exchange, declare)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/connection.py", line 436, in _ensured
return fun(*args, **kwargs)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/messaging.py", line 184, in _publish
[maybe_declare(entity) for entity in declare]
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/messaging.py", line 111, in maybe_declare
return maybe_declare(entity, self.channel, retry, **retry_policy)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/common.py", line 113, in maybe_declare
return _maybe_declare(entity, declared, ident, channel)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/common.py", line 120, in _maybe_declare
entity.declare()
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/entity.py", line 521, in declare
self.exchange.declare(nowait)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/kombu/entity.py", line 174, in declare
nowait=nowait, passive=passive,
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/amqp/channel.py", line 615, in exchange_declare
self._send_method((40, 10), args)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/amqp/abstract_channel.py", line 56, in _send_method
self.channel_id, method_sig, args, content,
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/amqp/method_framing.py", line 221, in write_method
write_frame(1, channel, payload)
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/amqp/transport.py", line 182, in write_frame
frame_type, channel, size, payload, 0xce,
File "/local/mnt/apps/ipcat/venvs/django16/lib/python2.7/site-packages/amqp/transport.py", line 254, in _write
n = write(s)
File "/usr/lib64/python2.7/ssl.py", line 172, in write
return self._sslobj.write(data)
SchedulingError: Couldn't apply scheduled task celery.backend_cleanup: [Errno 1] _ssl.c:1309: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry
[2016-10-24 04:00:02,743: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.
After this failure all the subsequent send task fails with error as,
[2016-10-24 12:30:03,026: DEBUG/MainProcess] beat: Synchronizing schedule...
[2016-10-24 12:30:03,253: ERROR/MainProcess] Message Error: Couldn't apply scheduled task aggregate_hw_errata_sign_off_metrics_schedule: 'NoneType' object has no attribute 'do_handshake'
I could disable celery.backend_cleanjob using CELERY_TASK_RESULT_EXPIRES = None but I am afraid, it might fails on other tasks configured.
Any help or guidance is appreciated.
I think, I found the problem. I was creating connection and was passing it to exchange and queue instance. I modified the code to let Queue and Exchange create connection to broker. so far no issues after this change.
Thanks
Old code:
from kombu import Connection, Queue, Exchange
conn = Connection(
hostname=settings.BROKER_HOST,
port=settings.BROKER_PORT,
userid=settings.BROKER_USER,
password=settings.BROKER_PASSWORD,
virtual_host=settings.BROKER_VHOST,
connect_timeout=BROKER_CONNECTION_TIMEOUT,
ssl=BROKER_USE_SSL,
transport='pyamqp')
conn.connect()
channel = conn.channel()
exchange = Exchange('my.exchange', type='direct', passive=True, channel=channel)
CELERY_QUEUES = (
Queue('my.tasks', exchange=exchange, routing_key='my.metrics', channel=conn, passive=True),
)
New Code (fix):
from kombu import Queue, Exchange
exchange = Exchange('my.exchange', type='direct', passive=True)
CELERY_QUEUES = (
Queue('my.tasks', exchange=exchange, routing_key='my.metrics', passive=True),
)

python-selenium on aws lambda throws error when switch_back_to.default_content and try to select element

python-selenium using phantomjs used to open an iFrame inside the page, then after switching back to the default content(driver.switch_to.default_content()) does not work.
The following is a pseudo code snippet to explain the scenario.
frame = WebDriverWait(driver, wait_time).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe.class_name")))
......
driver.switch_to.default_content()
print('Switched back to default content')
# Error occurs when line below is interpreted
driver.find_element_by_css_selector("div[title^='Test %s']" %(array_of_content[1]))
Following error occurs:-
Traceback (most recent call last):
File "/var/task/main.py", line 217, in test_method
driver.find_element_by_css_selector("div[title^='Test %s']" %(array_of_content[1]))
File "/var/task/selenium/webdriver/support/wait.py", line 71, in until
value = method(self._driver)
File "/var/task/selenium/webdriver/support/expected_conditions.py", line 213, in __call__
element = visibility_of_element_located(self.locator)(driver)
File "/var/task/selenium/webdriver/support/expected_conditions.py", line 78, in __call__
return _element_if_visible(_find_element(driver, self.locator))
File "/var/task/selenium/webdriver/support/expected_conditions.py", line 307, in _find_element
return driver.find_element(*by)
File "/var/task/selenium/webdriver/remote/webdriver.py", line 752, in find_element
'value': value})['value']
File "/var/task/selenium/webdriver/remote/webdriver.py", line 234, in execute
response = self.command_executor.execute(driver_command, params)
File "/var/task/selenium/webdriver/remote/remote_connection.py", line 401, in execute
return self._request(command_info[0], url, body=data)
File "/var/task/selenium/webdriver/remote/remote_connection.py", line 471, in _request
resp = opener.open(request, timeout=self._timeout)
File "/usr/lib64/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 1229, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib64/python2.7/urllib2.py", line 1202, in do_open
r = h.getresponse(buffering=True)
File "/var/task/raven/breadcrumbs.py", line 328, in getresponse
rv = real_getresponse(self, *args, **kwargs)
File "/usr/lib64/python2.7/httplib.py", line 1132, in getresponse
response.begin()
File "/usr/lib64/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python2.7/httplib.py", line 417, in _read_status
raise BadStatusLine(line)
BadStatusLine: ''
Any ideas, what is happening.
As I am able to see the same error, even if I write driver.save_screenshot('test.png').
But it is not throwing any error on local machines, also tried to use AMI instance to replicate if something is there in concern Amazon Linux.

Python: Urllib2 | [Errno 54] Connection reset by peer

I'm calling a list of urls from the same domain and returning a snip of their html for a few thousand domains but am getting this error about 1,000 rows or so in.
Is there anything I can do to avoid this error?
Does it make sense to create a wait step after every row? every few hundred rows?
Is there a better way to get around this?
File "/Users.../ap.py", line 144, in <module> simpleProg()
File "/Users.../ap.py", line 21, in simpleProg()
File "/Users.../ap.py", line 57, in first_step()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 400, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 418, in _open
'_open', req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1207, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1180, in do_open
r = h.getresponse(buffering=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1030, in getresponse
response.begin()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 407, in begin
version, status, reason = self._read_status()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 365, in _read_status
line = self.fp.readline()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 447, in readline
data = self._sock.recv(self._rbufsize)
socket.error: [Errno 54] Connection reset by peer