verifying mount points on an Ec2 instance - amazon-web-services

i am mounting multiple ebs volumes on an ec2 instance to perform some validations and check-sums but the problem is that before mounting volumes on the target mount point , i need to check whether that mount is not occupied, since i am creating mount points dynamically e.g
test_lambda.py (e.g)
device_prefixes = ['f','g','h','i','j','k','l','m','n','o','p']
attached_volumes = []
try:
for index,volume_id in enumerate(created_volumes):
MOUNT_POINT = "/dev/sd{}".format(device_prefixes[index])
I have tried commands using SSM like /proc/mounts, findmnt, but they are displaying partitions and other volumes but not target like (/dev/sde,/dev/sdf etc)

Thanks to the hint provided by jordan, i was able to clear my understanding about block device mapping , boto3 ec2 client has a method associated
`# where attribute can be one of the following` `"""'|'kernel'|'ramdisk'|'userData'|'disableApiTermination'|'instanceInitiatedShutdownBehavior'|'rootDeviceName'|'blockDeviceMapping'|'productCodes'|'sourceDestCheck'|'groupSet'|'ebsOptimized'|'sriovNetSupport'|'enaSupport'|'enclaveOptions'|'disableApiStop',z"""`
print(instance.describe_attribute(
instance.describe_attribute(
Attribute='blockDeviceMapping',
DryRun=False,
)
it does give us info about certain resources, apologies if my question was not clear in the first place,
{'BlockDeviceMappings': [{'DeviceName': '/dev/xvda', 'Ebs':
{'AttachTime': datetime.datetime(2022, 5, 27, 12, 4, 1,
tzinfo=tzlocal()), 'DeleteOnTermination': True, 'Status': 'attached',
'VolumeId': 'vol-05bb41df0e6bc4bf3'}}, {'DeviceName': '/dev/sdl',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 51,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-00be0e0f001fea1a9'}}, {'DeviceName': '/dev/sdm',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 51,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-09c552e77ba2a563f'}}, {'DeviceName': '/dev/sdn',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 52,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-08d28a4ffe5e71dff'}}, {'DeviceName': '/dev/sdo',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 53,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-0e63fa17bd6185ee3'}}, {'DeviceName': '/dev/sdf',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 36, 42,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-058de095fcfb1ef05'}}, {'DeviceName': '/dev/sdg',
'Ebs': {'Atta

Related

Truncate consecutive values in a Django (TimescaleDB) QuerySet

I have a QuerySet consisting of a timestamp and a boolean value. When there are consecutive repeating values, I'd like to truncate them to only show the first (and maybe the last one too, but not mandatory).
e.g. the given queryet
<TimescaleQuerySet [
{'time': datetime.datetime(2023, 1, 23, 10, 57, 7, 971506, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 10, 53, 11, 787303, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 10, 53, 20, 646474, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 27, 7, 971506, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 23, 20, 646474, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 23, 11, 787303, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 9, 57, 7, 971506, tzinfo=<UTC>), 'is_on': True}]
should truncate to
<TimescaleQuerySet [
{'time': datetime.datetime(2023, 1, 23, 10, 57, 7, 971506, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 10, 53, 20, 646474, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 23, 11, 787303, tzinfo=<UTC>), 'is_on': True}]
I'm breaking my head over this. Is there an elegant way to achieve this? I want to avoid looping over the whole queryset, it's just too slow when theres >1000 results.

Getting BulkwriteError when using MongoDb with djangorestframework-simplejwt?

I am using MongoDB and SimpleJWT in DjangoREST to authenticate and authorize users. I tried to implement user logout, whereby in SimpleJWT it's basically blacklisting a user token. When the first user logs in, everything seems okay and their refresh token is added to the Outstanding token table. But when I try to log in a second user, I get the below error:
raise BulkWriteError(full_result)
pymongo.errors.BulkWriteError: batch op errors occurred, full error: {'writeErrors': [{'index': 0, 'code': 11000, 'keyPattern': {'jti_hex': 1}, 'keyValue': {'jti_hex': None}, 'errmsg': 'E11000 duplicate key error collection: fsm_database.token_blacklist_outstandingtoken index: token_blacklist_outstandingtoken_jti_hex_d9bdf6f7_uniq dup key: { jti_hex: null
}', 'op': {'id': 19, 'user_id': 7, 'jti': '43bccc686fc648f5b60b22df3676b434', 'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoicmVmcmVzaCIsImV4cCI6MTY1OTY1NDUzNCwiaWF0IjoxNjU5NTY4MTM0LCJqdGkiOiI0M2JjY2M2ODZmYzY0OGY1YjYwYjIyZGYzNjc2YjQzNCIsInVzZXJfaWQiOjd9.aQmt5xAyncfpv_kDD2pF7iS98Hld98LhG6ng-rCW23M', 'created_at': datetime.datetime(2022,
8, 3, 23, 8, 54, 125539), 'expires_at': datetime.datetime(2022, 8, 4, 23, 8, 54), '_id': ObjectId('62eb00064621b38109bbae16')}}], 'writeConcernErrors': [], 'nInserted': 0, 'nUpserted': 0, 'nMatched': 0, 'nModified': 0, 'nRemoved': 0, 'upserted': []}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\MR.Robot\.virtualenvs\fsm-GjGxZg3c\lib\site-packages\djongo\cursor.py", line 51, in execute
self.result = Query(
File "C:\Users\MR.Robot\.virtualenvs\fsm-GjGxZg3c\lib\site-packages\djongo\sql2mongo\query.py", line 784, in __init__
self._query = self.parse()
File "C:\Users\MR.Robot\.virtualenvs\fsm-GjGxZg3c\lib\site-packages\djongo\sql2mongo\query.py", line 869, in parse
raise exe from e
djongo.exceptions.SQLDecodeError:
Keyword: None
Sub SQL: None
FAILED SQL: INSERT INTO "token_blacklist_outstandingtoken" ("user_id", "jti", "token", "created_at", "expires_at") VALUES (%(0)s, %(1)s, %(2)s, %(3)s, %(4)s)
Params: [7, '43bccc686fc648f5b60b22df3676b434', 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoicmVmcmVzaCIsImV4cCI6MTY1OTY1NDUzNCwiaWF0IjoxNjU5NTY4MTM0LCJqdGkiOiI0M2JjY2M2ODZmYzY0OGY1YjYwYjIyZGYzNjc2YjQzNCIsInVzZXJfaWQiOjd9.aQmt5xAyncfpv_kDD2pF7iS98Hld98LhG6ng-rCW23M', datetime.datetime(2022, 8, 3, 23, 8, 54, 125539), datetime.datetime(2022, 8, 4, 23, 8, 54)]
Version: 1.3.6
MongoDB seems to have a problem inserting the token for the second user in the outstanding table.
How can I fix this?
So I asked the library maintainers and they said that they don't support MongoDB. Check out this issue.

Using wso2mb message broker server with a php client

I trying to connect a php client (using php-amqplib lib) to a server based on wso2mb (wso2 message broker version 3.1.0).
I couldnt succed in that when using the amqp_consumer.php and always locked with this error :
INFO {org.wso2.andes.server.protocol.AMQProtocolEngine} - Unable to create SASL Server:AMQPLAIN whilst processing:[ConnectionStartOkBodyImpl: clientProperties={product=[LONG_STRING: AMQPLib], platform=[LONG_STRING: PHP], version=[LONG_STRING: 2.6], information=[LONG_STRING: ], copyright=[LONG_STRING: ], capabilities=[FIELD_TABLE: {authentication_failure_close=[BOOLEAN: true], publisher_confirms=[BOOLEAN: true], consumer_cancel_notify=[BOOLEAN: true], exchange_exchange_bindings=[BOOLEAN: true], basic.nack=[BOOLEAN: true], connection.blocked=[BOOLEAN: true]}]}, mechanism=AMQPLAIN, response=[5, 76, 79, 71, 73, 78, 83, 0, 0, 0, 5, 97, 100, 109, 105, 110, 8, 80, 65, 83, 83, 87, 79, 82, 68, 83, 0, 0, 0, 5, 97, 100, 109, 105, 110], locale=en_US]
[2016-11-04 08:05:26,901] INFO {org.wso2.andes.server.protocol.AMQProtocolEngine} - Closing connection due to: org.wso2.andes.AMQConnectionException: Unable to create SASL Server:AMQPLAIN [error code 506: resource error]
I using thos params in confing.php as conexions params
require_once __DIR__ . '/../vendor/autoload.php';
define('HOST', 'localhost');
define('PORT', 5692);
define('USER', 'admin');
define('PASS', 'admin');
define('VHOST', '/');
My questions :
1. Could you recommand any php library / tutorial to establish communication between some php code and wso2mb ?
2. What are the connexion method that are allowed by wso2mb ? (PLAIN, AMQPLAIN...?)
3. Help plz :)

How do i get request and response count in scrapyd?

I am trying to get request and response count in scrapyd,while running multiple spider means 8 spider dynamically.I am try to get those count using python.
following counts:
enter code here{'downloader/request_bytes': 130427,
'downloader/request_count': 273,
'downloader/request_method_count/GET': 273,
'downloader/response_bytes': 2169984,
'downloader/response_count': 273,
'downloader/response_status_count/200': 271,
'downloader/response_status_count/404': 2,
'dupefilter/filtered': 416,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2015, 7, 21, 14, 21, 38, 13000),
'item_scraped_count': 119,
'log_count/DEBUG': 406,
'log_count/INFO': 44,
'offsite/domains': 9,
'offsite/filtered': 5865,
'request_depth_max': 14,
'response_received_count': 273,
'scheduler/dequeued': 273,
'scheduler/dequeued/memory': 273,
'scheduler/enqueued': 273,
'scheduler/enqueued/memory': 273,
'start_time': datetime.datetime(2015, 7, 21, 14, 16, 41, 144000)}
enter code here
Thanks,
Use the Stats Collection from Scrapy.
With this you can access the statistics dumped at the end to the console and if you write your own middleware you can combine the results of your 8 spiders together -- like in the example of the documentation.

Regroup by day, then regroup again by the hour

I have a dictionary list of items. For example
activities = [
{'start': datetime.datetime(2015, 9, 21, 8, 0), 'end': datetime.datetime(2015, 9, 21, 9, 0),
u'name': u'First Activity', 'description': 'desc of first activity...'},
{'start': datetime.datetime(2015, 9, 21, 9, 0), 'end': datetime.datetime(2015, 9, 21, 10, 0),
u'name': u'Second Activity', 'description': 'desc of next activity...'},
{'start': datetime.datetime(2015, 9, 22, 8, 0), 'end': datetime.datetime(2015, 9, 21, 9, 0),
u'name': u'Third Activity', 'description': 'desc of third activity...'},
{'start': datetime.datetime(2015, 9, 23, 8, 0), 'end': datetime.datetime(2015, 9, 21, 9, 0),
u'name': u'Next Activity', 'description': 'desc of next activity...'},
{'start': datetime.datetime(2015, 9, 23, 9, 0), 'end': datetime.datetime(2015, 9, 21, 10, 0),
u'name': u'Another Activity', 'description': 'desc of another activity...'},
{'start': datetime.datetime(2015, 9, 21, 10, 0), 'end': datetime.datetime(2015, 9, 21, 11, 0),
u'name': u'New Activity', 'description': 'desc of new activity...'}
]
I would like to do a regroup, first by the day, then perform a second regroup by the hour.
I'm having trouble enough just getting a regroup to work by either the day or by hour.
For example:
{% regroup activities by start|date:"l, n/d" as activity_list_by_day %}
{% for session in activity_list_by_day %}
{{ session.grouper }}
{% endfor %}
Isn't working out well. I don't see how to specify that I want the regroup to work by the day.
Is there some way to manipulate my 'start' item for these regroupings? Or might I need to create some extra items to work with?
Edited to add: I see now I can start out by sorting my activities list with:
activities.sort(key=lambda item:item['start'])
This helps tremendously. Actually might have solved my own problem...