Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 10267 over 11 years Aaron Marcuse-Kubitza backups/Makefile: %.backup/restore: documented ...
  bin 10348 over 11 years Aaron Marcuse-Kubitza bugfix: bin/repl: text mode: repurpose this to ...
  config 7801 over 11 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: mk_db: Added command...
  exports 9928 over 11 years Aaron Marcuse-Kubitza added exports/_archive/
  inputs 10361 over 11 years Aaron Marcuse-Kubitza inputs/REMIB/Specimen/postprocess.sql: map_null...
  lib 10302 over 11 years Aaron Marcuse-Kubitza lib/sql_io.py: put_table(): documented that Pos...
  mappings 10289 over 11 years Aaron Marcuse-Kubitza mappings/VegCore.htm: regenerated from wiki. Sp...
  planning 10311 over 11 years Aaron Marcuse-Kubitza planning/timeline/timeline.2013.xls: moved Indi...
  schemas 10359 over 11 years Aaron Marcuse-Kubitza schemas/util.sql: map_nulls(): documented that ...
  web 10306 over 11 years Aaron Marcuse-Kubitza web/links/index.htm: updated to Firefox bookmar...
.htaccess 326 Bytes 8771 over 11 years Aaron Marcuse-Kubitza /.htaccess: use canonical URL without symlinks
.rsync_filter.upload 33 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
.rsync_ignore 12 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
Makefile 12.6 KB 10223 over 11 years Aaron Marcuse-Kubitza /Makefile: mysql-Linux: also install mysql-work...
README.TXT 23.2 KB 10286 over 11 years Aaron Marcuse-Kubitza /README.TXT: Maintenance: regenerate mappings/V...
fix_perms 97 Bytes 7560 almost 12 years Aaron Marcuse-Kubitza Added root fix_perms
map 1001 Bytes 6949 almost 12 years Aaron Marcuse-Kubitza vegbien_dest: Changed default $prefix to "", so...
new_terms.csv 38.1 KB 7222 almost 12 years Aaron Marcuse-Kubitza new_terms.csv: Regenerated
run 433 Bytes 9916 over 11 years Aaron Marcuse-Kubitza /run: geoscrub_input/make(): documented runtime...
unmapped_terms.csv 13.1 KB 7201 almost 12 years Aaron Marcuse-Kubitza **/new_terms.csv, **/unmapped_terms.csv: Regene...

Latest revisions

# Date Author Comment
10361 07/20/2013 01:27 AM Aaron Marcuse-Kubitza

inputs/REMIB/Specimen/postprocess.sql: map_nulls__*(): turned off STRICT to allow dynamic inlining, which speeds up the mk_derived_col() statements by 5x (342799.823 ms -> 71533.252 ms (6 min -> 1 min) for latitude_sec)

10360 07/19/2013 07:23 PM Aaron Marcuse-Kubitza

inputs/REMIB/Specimen/postprocess.sql: runtimes: updated for vegbiendev, before dynamic inlining. the times are about twice as fast as on starscream, so vegbiendev is faster at whatever is the limiting speed factor (probably not CPU, based on other benchmarks).

10359 07/19/2013 07:05 PM Aaron Marcuse-Kubitza

schemas/util.sql: map_nulls(): documented that due to dynamic inlining, this is just as fast as util._map() which it wraps. dynamic inlining now brings altogether a 40x speed improvement to map_nulls() (4000 ms -> 100 ms), and would likely bring a comparable improvement for other functions that are run repeatedly and call other user-defined functions.

10358 07/19/2013 06:35 PM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: map_nulls(): updated to use hstore(text[], anyelement), which has replaced hstore(anyarray, anyelement)

10357 07/19/2013 06:30 PM Aaron Marcuse-Kubitza

schemas/util.sql: removed hstore(anyarray, anyelement), which did not support dynamic inlining, to avoid confusion over which hstore() function to use. use new hstore(text[], anyelement) instead (with explicit cast on the keys array if needed).

10356 07/19/2013 06:23 PM Aaron Marcuse-Kubitza

schemas/util.sql: added hstore(text[], anyelement), which dynamically inlines properly, unlike hstore(anyarray, anyelement). this can be selected by explicitly casting the keys array to text[], which now provides a 6x speed improvement (380 ms -> 60 ms) for map_nulls().

10355 07/19/2013 05:31 PM Aaron Marcuse-Kubitza

schemas/util.sql: fix_array(): turned off STRICT to allow dynamic inlining, which speeds up util.map_nulls() by 3x (1500 ms -> 500 ms)

10354 07/19/2013 05:15 PM Aaron Marcuse-Kubitza

schemas/util.sql: array_length(anyarray), array_length(anyarray, dimension integer): turned off STRICT to allow dynamic inlining, which speeds up util.map_nulls(). this requires adding a `CASE WHEN $1 IS NULL THEN NULL` statement to array_length(anyarray, dimension integer) to replace the functionality provided by STRICT.

10353 07/19/2013 04:41 PM Aaron Marcuse-Kubitza

schemas/util.sql: map_nulls(): turned off STRICT to allow dynamic inlining, which causes a 2x speed improvement1. (see r10352 for an explanation of dynamic inlining.) note that turning off STRICT disables NULL-skipping (avoiding running a function when all its params are NULL), so it should only be used when the NULL-skipping optimization is needed less than dynamic inlining....

10352 07/19/2013 04:23 PM Aaron Marcuse-Kubitza

schemas/util.sql: inlinable IMMUTABLE functions: avoid using config params (e.g. `SET search_path TO util`) because these prevent dynamic inlining (i.e. inlining of a function call with variable instead of constant arguments, by substituting the arguments into the function's body). dynamic inlining can speed up function evaluation significantly, because a (slow) call to a user-defined SQL function is avoided.

View all revisions | View revisions

Also available in: Atom