Regroup by day, then regroup again by the hour - django

I have a dictionary list of items. For example
activities = [
{'start': datetime.datetime(2015, 9, 21, 8, 0), 'end': datetime.datetime(2015, 9, 21, 9, 0),
u'name': u'First Activity', 'description': 'desc of first activity...'},
{'start': datetime.datetime(2015, 9, 21, 9, 0), 'end': datetime.datetime(2015, 9, 21, 10, 0),
u'name': u'Second Activity', 'description': 'desc of next activity...'},
{'start': datetime.datetime(2015, 9, 22, 8, 0), 'end': datetime.datetime(2015, 9, 21, 9, 0),
u'name': u'Third Activity', 'description': 'desc of third activity...'},
{'start': datetime.datetime(2015, 9, 23, 8, 0), 'end': datetime.datetime(2015, 9, 21, 9, 0),
u'name': u'Next Activity', 'description': 'desc of next activity...'},
{'start': datetime.datetime(2015, 9, 23, 9, 0), 'end': datetime.datetime(2015, 9, 21, 10, 0),
u'name': u'Another Activity', 'description': 'desc of another activity...'},
{'start': datetime.datetime(2015, 9, 21, 10, 0), 'end': datetime.datetime(2015, 9, 21, 11, 0),
u'name': u'New Activity', 'description': 'desc of new activity...'}
]
I would like to do a regroup, first by the day, then perform a second regroup by the hour.
I'm having trouble enough just getting a regroup to work by either the day or by hour.
For example:
{% regroup activities by start|date:"l, n/d" as activity_list_by_day %}
{% for session in activity_list_by_day %}
{{ session.grouper }}
{% endfor %}
Isn't working out well. I don't see how to specify that I want the regroup to work by the day.
Is there some way to manipulate my 'start' item for these regroupings? Or might I need to create some extra items to work with?
Edited to add: I see now I can start out by sorting my activities list with:
activities.sort(key=lambda item:item['start'])
This helps tremendously. Actually might have solved my own problem...

Related

Truncate consecutive values in a Django (TimescaleDB) QuerySet

I have a QuerySet consisting of a timestamp and a boolean value. When there are consecutive repeating values, I'd like to truncate them to only show the first (and maybe the last one too, but not mandatory).
e.g. the given queryet
<TimescaleQuerySet [
{'time': datetime.datetime(2023, 1, 23, 10, 57, 7, 971506, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 10, 53, 11, 787303, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 10, 53, 20, 646474, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 27, 7, 971506, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 23, 20, 646474, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 23, 11, 787303, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 9, 57, 7, 971506, tzinfo=<UTC>), 'is_on': True}]
should truncate to
<TimescaleQuerySet [
{'time': datetime.datetime(2023, 1, 23, 10, 57, 7, 971506, tzinfo=<UTC>), 'is_on': True},
{'time': datetime.datetime(2023, 1, 23, 10, 53, 20, 646474, tzinfo=<UTC>), 'is_on': False},
{'time': datetime.datetime(2023, 1, 23, 10, 23, 11, 787303, tzinfo=<UTC>), 'is_on': True}]
I'm breaking my head over this. Is there an elegant way to achieve this? I want to avoid looping over the whole queryset, it's just too slow when theres >1000 results.

Getting BulkwriteError when using MongoDb with djangorestframework-simplejwt?

I am using MongoDB and SimpleJWT in DjangoREST to authenticate and authorize users. I tried to implement user logout, whereby in SimpleJWT it's basically blacklisting a user token. When the first user logs in, everything seems okay and their refresh token is added to the Outstanding token table. But when I try to log in a second user, I get the below error:
raise BulkWriteError(full_result)
pymongo.errors.BulkWriteError: batch op errors occurred, full error: {'writeErrors': [{'index': 0, 'code': 11000, 'keyPattern': {'jti_hex': 1}, 'keyValue': {'jti_hex': None}, 'errmsg': 'E11000 duplicate key error collection: fsm_database.token_blacklist_outstandingtoken index: token_blacklist_outstandingtoken_jti_hex_d9bdf6f7_uniq dup key: { jti_hex: null
}', 'op': {'id': 19, 'user_id': 7, 'jti': '43bccc686fc648f5b60b22df3676b434', 'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoicmVmcmVzaCIsImV4cCI6MTY1OTY1NDUzNCwiaWF0IjoxNjU5NTY4MTM0LCJqdGkiOiI0M2JjY2M2ODZmYzY0OGY1YjYwYjIyZGYzNjc2YjQzNCIsInVzZXJfaWQiOjd9.aQmt5xAyncfpv_kDD2pF7iS98Hld98LhG6ng-rCW23M', 'created_at': datetime.datetime(2022,
8, 3, 23, 8, 54, 125539), 'expires_at': datetime.datetime(2022, 8, 4, 23, 8, 54), '_id': ObjectId('62eb00064621b38109bbae16')}}], 'writeConcernErrors': [], 'nInserted': 0, 'nUpserted': 0, 'nMatched': 0, 'nModified': 0, 'nRemoved': 0, 'upserted': []}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\MR.Robot\.virtualenvs\fsm-GjGxZg3c\lib\site-packages\djongo\cursor.py", line 51, in execute
self.result = Query(
File "C:\Users\MR.Robot\.virtualenvs\fsm-GjGxZg3c\lib\site-packages\djongo\sql2mongo\query.py", line 784, in __init__
self._query = self.parse()
File "C:\Users\MR.Robot\.virtualenvs\fsm-GjGxZg3c\lib\site-packages\djongo\sql2mongo\query.py", line 869, in parse
raise exe from e
djongo.exceptions.SQLDecodeError:
Keyword: None
Sub SQL: None
FAILED SQL: INSERT INTO "token_blacklist_outstandingtoken" ("user_id", "jti", "token", "created_at", "expires_at") VALUES (%(0)s, %(1)s, %(2)s, %(3)s, %(4)s)
Params: [7, '43bccc686fc648f5b60b22df3676b434', 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoicmVmcmVzaCIsImV4cCI6MTY1OTY1NDUzNCwiaWF0IjoxNjU5NTY4MTM0LCJqdGkiOiI0M2JjY2M2ODZmYzY0OGY1YjYwYjIyZGYzNjc2YjQzNCIsInVzZXJfaWQiOjd9.aQmt5xAyncfpv_kDD2pF7iS98Hld98LhG6ng-rCW23M', datetime.datetime(2022, 8, 3, 23, 8, 54, 125539), datetime.datetime(2022, 8, 4, 23, 8, 54)]
Version: 1.3.6
MongoDB seems to have a problem inserting the token for the second user in the outstanding table.
How can I fix this?
So I asked the library maintainers and they said that they don't support MongoDB. Check out this issue.

verifying mount points on an Ec2 instance

i am mounting multiple ebs volumes on an ec2 instance to perform some validations and check-sums but the problem is that before mounting volumes on the target mount point , i need to check whether that mount is not occupied, since i am creating mount points dynamically e.g
test_lambda.py (e.g)
device_prefixes = ['f','g','h','i','j','k','l','m','n','o','p']
attached_volumes = []
try:
for index,volume_id in enumerate(created_volumes):
MOUNT_POINT = "/dev/sd{}".format(device_prefixes[index])
I have tried commands using SSM like /proc/mounts, findmnt, but they are displaying partitions and other volumes but not target like (/dev/sde,/dev/sdf etc)
Thanks to the hint provided by jordan, i was able to clear my understanding about block device mapping , boto3 ec2 client has a method associated
`# where attribute can be one of the following` `"""'|'kernel'|'ramdisk'|'userData'|'disableApiTermination'|'instanceInitiatedShutdownBehavior'|'rootDeviceName'|'blockDeviceMapping'|'productCodes'|'sourceDestCheck'|'groupSet'|'ebsOptimized'|'sriovNetSupport'|'enaSupport'|'enclaveOptions'|'disableApiStop',z"""`
print(instance.describe_attribute(
instance.describe_attribute(
Attribute='blockDeviceMapping',
DryRun=False,
)
it does give us info about certain resources, apologies if my question was not clear in the first place,
{'BlockDeviceMappings': [{'DeviceName': '/dev/xvda', 'Ebs':
{'AttachTime': datetime.datetime(2022, 5, 27, 12, 4, 1,
tzinfo=tzlocal()), 'DeleteOnTermination': True, 'Status': 'attached',
'VolumeId': 'vol-05bb41df0e6bc4bf3'}}, {'DeviceName': '/dev/sdl',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 51,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-00be0e0f001fea1a9'}}, {'DeviceName': '/dev/sdm',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 51,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-09c552e77ba2a563f'}}, {'DeviceName': '/dev/sdn',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 52,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-08d28a4ffe5e71dff'}}, {'DeviceName': '/dev/sdo',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 15, 53,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-0e63fa17bd6185ee3'}}, {'DeviceName': '/dev/sdf',
'Ebs': {'AttachTime': datetime.datetime(2022, 5, 27, 15, 36, 42,
tzinfo=tzlocal()), 'DeleteOnTermination': False, 'Status': 'attached',
'VolumeId': 'vol-058de095fcfb1ef05'}}, {'DeviceName': '/dev/sdg',
'Ebs': {'Atta

How do i get request and response count in scrapyd?

I am trying to get request and response count in scrapyd,while running multiple spider means 8 spider dynamically.I am try to get those count using python.
following counts:
enter code here{'downloader/request_bytes': 130427,
'downloader/request_count': 273,
'downloader/request_method_count/GET': 273,
'downloader/response_bytes': 2169984,
'downloader/response_count': 273,
'downloader/response_status_count/200': 271,
'downloader/response_status_count/404': 2,
'dupefilter/filtered': 416,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2015, 7, 21, 14, 21, 38, 13000),
'item_scraped_count': 119,
'log_count/DEBUG': 406,
'log_count/INFO': 44,
'offsite/domains': 9,
'offsite/filtered': 5865,
'request_depth_max': 14,
'response_received_count': 273,
'scheduler/dequeued': 273,
'scheduler/dequeued/memory': 273,
'scheduler/enqueued': 273,
'scheduler/enqueued/memory': 273,
'start_time': datetime.datetime(2015, 7, 21, 14, 16, 41, 144000)}
enter code here
Thanks,
Use the Stats Collection from Scrapy.
With this you can access the statistics dumped at the end to the console and if you write your own middleware you can combine the results of your 8 spiders together -- like in the example of the documentation.

import openlayers3 in ember.js

I am starting up an ember.js app aimed at drawing and displaying maps.
I am using ember.js v1.11.0 and ol3 v3.4.0
I managed to install ol3 via bower and import it using Brocfile.js:
app.import('bower_components/ol3/build/ol.js');
app.import('bower_components/ol3/css/ol.css');
I can use it as well without problem in my views, etc. What I would like if possible is to get rid of the server errors:
views/map.js: line 6, col 22, 'ol' is not defined.
views/map.js: line 7, col 19, 'ol' is not defined.
views/map.js: line 10, col 21, 'ol' is not defined.
views/map.js: line 11, col 19, 'ol' is not defined.
views/map.js: line 14, col 19, 'ol' is not defined.
views/map.js: line 17, col 17, 'ol' is not defined.
And if possible get autocompletion in my Intellij IDEA (make it recognise the ol library)
If somebody could give me a hand, that owuld be much apreciated.
add ol to your .jshintrc file. in the predef array:
"predef": [
"document",
"window",
"-Promise",
"ol"
]
....
And for the intellij-idea you should be able to get your answer from the docs here:
https://www.jetbrains.com/idea/help/configuring-javascript-libraries.html