gaetk2.datastore module¶
gaetk2.datastore tries to codify a common set of expectations and usages for gaetk2.
Inherit from gaetk2.datastore.gaetkModel
instead of ndb.Model to get some
added functionality. The rationale there is that e common interface and thus
admin- and programmer-time is more important than savings on space and and
processing time. To we add possible unneded database fields. You can
remove them on a case by case basis in derivered classes.
query_iterator()
- helps to iterate over big query resultsget_or_insert_if_new()
helps you to see if a new Entity was created.copy_entity()
- can write an entity with a different key to the datastoreupdate_obj()
- basically implements conditionalput()
reload_obj()
- forces an object to be re-read from diskapply_to_all_entities()
- iterates over a table executing a function (“mapper”)
Data Model Conventions¶
- url
- created_at, updated_at
- name, nicename
- designator
Module contents¶
-
gaetk2.datastore.
query_iterator
(query, limit=50)[source]¶ Iterates over a datastore query while avoiding timeouts via a cursor.
Especially helpful for usage in backend-jobs.
-
gaetk2.datastore.
copy_entity
(e, **extra_args)[source]¶ Copy ndb entity but change values in kwargs.
- Usage::
- b = copy_entity(a, id=’new_id_here’) b.put()
-
gaetk2.datastore.
get_or_insert_if_new
(cls, id, **kwds)[source]¶ Like ndb.get_or_insert()` but returns (entity, new).
This allows you to see if something has been created or if there was an already existing entity:
>>> get_or_insert_if_new(Model, 'newid') (<instance>, True) >>> get_or_insert_if_new(Model, 'newid') (<instance>, False)
-
gaetk2.datastore.
write_on_change2
(obj, data)[source]¶ Apply new data to an entity and write to datastore if anything changed.
This should save you money since reads are 3 times cheaper than writes. It also helps you do leave not given attributes unchanged.
Usage:
instance = ndb.Model...get() dirty = write_on_change2(instance, ..., dict(id=123, amout_open=500, score=5, ...)
-
gaetk2.datastore.
update_obj
(obj, **kwargs)[source]¶ More modern Interface to
write_on_change2()
.
-
gaetk2.datastore.
apply_to_all_entities
(func, model, batch_size=0, num_updated=0, num_processed=0, cursor=None)[source]¶ Appliy a certain task all entities of model.
It scans every entity in the datastore for the model, exectues func(entity) on it and re-saves it if func trturns true. Tries to keep updated_at and updated_by unchanged.
Example
- def _fixup_MyModel_updatefunc(obj):
- if obj.wert_eur is not None:
- obj.wert_eur = int(obj.wert_eur) return True
return False
- def fixup_MyModel():
- apply_to_all_entities(_fixup_app_angebotspos_updatefunc, MyModel)
# or
- def execute(_now):
- datastore.apply_to_all_entities(
- _fixup_bestandsbuch_updatefunc, ic_bestandsbuch.ic_BestandsbuchEintrag)
- def _fixup_bestandsbuch_updatefunc(obj):
changed = False # Attribute, die es als string und text in der datebnbank gibt normalisieren for attrname in ‘’‘ausloeser vorhergehender_bestandsbucheintrag info’‘’.split():
- if getattr(obj, attrname, None) is not None:
- setattr(obj, attrname, unicode(getattr(obj, attrname))) changed = True
return changed