NDB - Most Efficient Way to Delete a List of Keys - python-2.7

Question

I believe I need to use ndb.delete_multi but I am confused as to how to make it work for a specific set of keys and whether it is the most efficient way to go. Using Python 2.7 with Google App Engine.
First, I am collecting the keys that I want to delete. I don't want to delete everything, only those entries that are 1 hour or more old. Toward that end I first collect the list of keys that meet this criteria.
cs = ChannelStore()
delMsgKeys = []
for x in cs.allMessages():
current = datetime.datetime.now()
recordTime = x.channelMessageCreated
timeDiffSecs = (current - recordTime).total_seconds()
timeDiff = (timeDiffSecs/60)/60
if timeDiff >=1:
delMsgKeys.append(x.key.id())
ndb.delete_multi(?????)
The definition for cs.allMessages():
def allMessages(self):
return ChannelStore.query().fetch()
First, is this overall the most efficient approach? Second, how do I use the list of keys I created with the ndb.delete_multi statement?
---Update----
The issue with the ndb.delete_multi has to do with the keys I was passing it. In the code I posted above the keys should have been stored as follows:
delMsgKeys.append(x.key)
With the above ndb.delete_multi works.

Answer 1

Per the NDB documentation, you can just pass in a list of keys to ndb.delete_multi, so based on your code, this should work:
ndb.delete_multi(delMsgKeys)
I'm not sure what the limit is for the number of keys that you can pass in a single ndb.delete_multi() call, though.
For this query:
ChannelStore.query().fetch()
You can add an entity property to store a timestamp when you create/update the entity by adding auto_now = True (more documentation here). Then with the timestamp property you can query for something like this:
sixty_mins_ago = datetime.datetime.now()- datetime.timedelta(minutes = 60)
qry = ChannelStore.query()
list_of_keys = qry.filter(ChannelStore.timestamp < sixty_mins_ago).fetch(keys_only = True)
Since you don't need the entities, a keys_only fetch will be cheaper. Of course this code is assuming your ChannelStore model has a timestamp property, so your model will have to be something like this:
class ChannelStore(ndb.model):
#other properties go here
timestamp = ndb.DateTimeProperty(auto_now = True)
Putting it all together, something like this could work for the code block you have above:
from models import ChannelStore
from google.appengine.ext import ndb
from datetime import datetime, timedelta
# other imports
def delete_old_entities():
sixty_mins_ago = datetime.now() - timedelta(minutes = 60)
qry = ChannelStore.query()
qry = qry.filter(ChannelStore.timestamp < sixty_mins_ago)
list_of_keys = qry.fetch(keys_only = True)
ndb.delete_multi(list_of_keys)
In case you have to delete a lot of keys and are running into some kind of API limit with the ndb.delete_multi call, you can change the delete_old_entities() method to the following:
def delete_old_entities():
sixty_mins_ago = datetime.datetime.now() - datetime.timedelta(minutes = 60)
qry = ChannelStore.query()
qry = qry.filter(ChannelStore.timestamp < sixty_mins_ago)
list_of_keys = qry.fetch(keys_only = True)
while list_of_keys:
# delete 100 at a time
ndb.delete_multi(list_of_keys[:100])
list_of_keys = list_of_keys[100:]