gaetk2.tools Package¶
Thees package contains functionality mostly used intenally.
Contents
Todo
- ids.py
- hujson2.py
- http.py
- config.py
- auth0tools.py
- sentry.py
- structured.py
gaetk2.tools.caching - smart caching¶
Caching on Google App Engine makes your application faster and cheaper. While for key.get() operations ndp provides caching for you, queries are nwver cached by the datastore infrastructure.
After years of experimentation we come to the conclusion that you should always use some time-based cache invalidation. This will result in “eventual consitency” even if you do not get your cache invalidation strategy perfectly right.
We provide lru_cache()
with a default TTL of 12 hours. It does local instance memory caching and is an extension of functools
from
Python 3.3.
lru_cache_memcache()
is an extension using a two-level strategy:
content which is not found in the local instance cache is pulled from the
shared memcache. Cache entries are not shared between different versions of
your application.
It is suggested, that you use a relatively small maxsize with lru_cache_memcache()
to save on instance memory.
-
gaetk2.tools.caching.
lru_cache
(maxsize=64, typed=False, ttl=43200)[source]¶ Least-recently-used cache decorator.
Parameters: - maxsize (int or None) – if None, the LRU features are disabled and the cache can grow without bound.
- typed (boolean) – if True, arguments of different types will be cached separately. For example, f(3.0) and f(3) will be treated as distinct calls with distinct results.
- ttl (int or None) – if set, cache entries are only served for ttl seconds.
Arguments to the cached function must be hashable.
View the cache statistics named tuple (hits, misses, maxsize, currsize) with f.cache_info(). Clear the cache and statistics with f.cache_clear(). Access the underlying function with f.__wrapped__.
See: http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used
Example
@lru_cache(maxsize=6) def _fetchbrands(): query = mk_models.mk_Brand.query() return [brand.name for brand in query.iter() if not brand.deleted]
-
class
gaetk2.tools.caching.
lru_cache_memcache
(maxsize=8, typed=False, ttl=43200)[source]¶ Use
lru_cache()
with memcache as an fallback.Arguments are the same as
lru_cache()
.Example
@lru_cache_memcache(ttl=3600) def _fetchbrands(): query = mk_models.mk_Brand.query() return [brand.name for brand in query.iter() if not brand.deleted]
gaetk2.tools.datetools¶
Todo
- Explain what gaetk2.tools.datetools is for
-
gaetk2.tools.datetools.
tertial
(date)[source]¶ Wandelt ein Date oder Datetime-Objekt in einen Tertial-String
-
gaetk2.tools.datetools.
rfc3339_date
(date=None)[source]¶ Formates a datetime object according to RfC 3339.
-
gaetk2.tools.datetools.
rfc3339_date_parse
(date)[source]¶ Parses an RfC 3339 timestamp into a datetime object.
-
gaetk2.tools.datetools.
convert_to_date
(date)[source]¶ Converts argument into a date object.
Assumes argument to be a RfC 3339 coded date or a date(time) object.
-
gaetk2.tools.datetools.
convert_to_datetime
(date)[source]¶ Converts argument into a datetime object.
Assumes argument to be a RfC 3339 coded date or a date(time) object.
gaetk2.tools.unicode - string handling¶
This are functions which help to handle data from a pre-Unicode world. Much of this code is acient and has no use in a worl where JSON and XML ensure somewhat clean encoding. But still there are so many places where you are allowd to send only ASCII subsets.
Data Cleanup¶
de_noise()
- removed Unicode Characters which normally have no place in buiseness documents (eg street names). This includes Emojii but also protected spaces unusual quotation marks etc. This data is usually included dut to cut and paste errors. Read source to see what is replaced.de_umlaut()
- converts data to plain ASCII while converting german Umlauts to something reasonable.de_utf8()
- “repair” wrongly decoded UTF-8.
Number Conversion¶
num_encode()
and num_decode()
convert arbitrary long numbers to strings and back again. Works nice for datastore IDs. Uses base 62 (lowwer and upper letters and numbers) to get a compact representation.
num_encode_uppercase()
uses base36 which is less compact but case insensitive.
You can use these functions to getsomewhat easy to tipe compact datastore ids:
class SomeEntity(ndb.Model):
nr = ndb.ComputedProperty(lambda num_encode(self: self.key.id()) if self.key.id() else '?')
Module contents¶
-
gaetk2.tools.unicode.
de_utf8
(data)[source]¶ This is meant to help with utf-8 data appearing where unicode should apperar.
-
gaetk2.tools.unicode.
de_umlaut
(data)[source]¶ Converts a text to ASCII acting smart about Umlauts.
>>> de_umlaut('1 Über Hügel saß René äöüÄÖÜß') '1 Ueber Huegel sass Rene aeoeueAeOeUess'
-
gaetk2.tools.unicode.
de_noise
(data)[source]¶ Removes all stuff which should not appear in normal Western Text.
>>> de_noise(u'»Susie`s Giga\Super-Markt®¿«') u">Susie's Giga/Super-Markt(R)?<" >>> de_noise(u"ümlaut eins:") u'\xfcmlaut eins:' >>> de_noise(u'«A» {C} ¿D? „E“ ›F‹') u'<A> (C) ?D? "E" >F<' >>> de_noise(u'`A´') u"'A'" >>> de_noise(u'«😎» Umlaute kann doctest !gut {®} ¿👩👩👧👦? „👨❤️💋👨“ ›🎅🏻🎅🏼🎅🏽🎅🏾🎅🏿‹') u'<> Umlaute kann doctest !gut ((R)) ?? "" ><' >>> de_noise(u'DE37 330 5 13 50 0 010 4414 22') u'DE37330513500010441422'
gaetk2.tools.structured_xls package¶
Todo
- Explain what gaetk2.tools.structured_xls is for
-
class
gaetk2.tools.structured_xls.
XLSwriter
(output=None, sheetname='This Sheet')[source]¶ csv
- Module compatible Interface to generate excel files.… but you have to call
save()
odergetvalue()
to generate the final XLS file.Parameters: Uses the deprecated xlwt.
Usage:
xlswriter = XLSwriter() xlswriter.writerow(['foo', 1, 2]) xlswriter.writerow(['bar', 3, datetime.date.today()]) xlswriter.save(open('test.xls')