Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 10267 over 11 years Aaron Marcuse-Kubitza backups/Makefile: %.backup/restore: documented ...
  bin 10348 over 11 years Aaron Marcuse-Kubitza bugfix: bin/repl: text mode: repurpose this to ...
  config 7801 almost 12 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: mk_db: Added command...
  exports 9928 over 11 years Aaron Marcuse-Kubitza added exports/_archive/
  inputs 10350 over 11 years Aaron Marcuse-Kubitza inputs/REMIB/Specimen/postprocess.sql: runtimes...
  lib 10302 over 11 years Aaron Marcuse-Kubitza lib/sql_io.py: put_table(): documented that Pos...
  mappings 10289 over 11 years Aaron Marcuse-Kubitza mappings/VegCore.htm: regenerated from wiki. Sp...
  planning 10311 over 11 years Aaron Marcuse-Kubitza planning/timeline/timeline.2013.xls: moved Indi...
  schemas 10356 over 11 years Aaron Marcuse-Kubitza schemas/util.sql: added hstore(text[], anyeleme...
  web 10306 over 11 years Aaron Marcuse-Kubitza web/links/index.htm: updated to Firefox bookmar...
.htaccess 326 Bytes 8771 over 11 years Aaron Marcuse-Kubitza /.htaccess: use canonical URL without symlinks
.rsync_filter.upload 33 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
.rsync_ignore 12 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
Makefile 12.6 KB 10223 over 11 years Aaron Marcuse-Kubitza /Makefile: mysql-Linux: also install mysql-work...
README.TXT 23.2 KB 10286 over 11 years Aaron Marcuse-Kubitza /README.TXT: Maintenance: regenerate mappings/V...
fix_perms 97 Bytes 7560 almost 12 years Aaron Marcuse-Kubitza Added root fix_perms
map 1001 Bytes 6949 almost 12 years Aaron Marcuse-Kubitza vegbien_dest: Changed default $prefix to "", so...
new_terms.csv 38.1 KB 7222 almost 12 years Aaron Marcuse-Kubitza new_terms.csv: Regenerated
run 433 Bytes 9916 over 11 years Aaron Marcuse-Kubitza /run: geoscrub_input/make(): documented runtime...
unmapped_terms.csv 13.1 KB 7201 almost 12 years Aaron Marcuse-Kubitza **/new_terms.csv, **/unmapped_terms.csv: Regene...

Latest revisions

# Date Author Comment
10356 07/19/2013 06:23 PM Aaron Marcuse-Kubitza

schemas/util.sql: added hstore(text[], anyelement), which dynamically inlines properly, unlike hstore(anyarray, anyelement). this can be selected by explicitly casting the keys array to text[], which now provides a 6x speed improvement (380 ms -> 60 ms) for map_nulls().

10355 07/19/2013 05:31 PM Aaron Marcuse-Kubitza

schemas/util.sql: fix_array(): turned off STRICT to allow dynamic inlining, which speeds up util.map_nulls() by 3x (1500 ms -> 500 ms)

10354 07/19/2013 05:15 PM Aaron Marcuse-Kubitza

schemas/util.sql: array_length(anyarray), array_length(anyarray, dimension integer): turned off STRICT to allow dynamic inlining, which speeds up util.map_nulls(). this requires adding a `CASE WHEN $1 IS NULL THEN NULL` statement to array_length(anyarray, dimension integer) to replace the functionality provided by STRICT.

10353 07/19/2013 04:41 PM Aaron Marcuse-Kubitza

schemas/util.sql: map_nulls(): turned off STRICT to allow dynamic inlining, which causes a 2x speed improvement1. (see r10352 for an explanation of dynamic inlining.) note that turning off STRICT disables NULL-skipping (avoiding running a function when all its params are NULL), so it should only be used when the NULL-skipping optimization is needed less than dynamic inlining....

10352 07/19/2013 04:23 PM Aaron Marcuse-Kubitza

schemas/util.sql: inlinable IMMUTABLE functions: avoid using config params (e.g. `SET search_path TO util`) because these prevent dynamic inlining (i.e. inlining of a function call with variable instead of constant arguments, by substituting the arguments into the function's body). dynamic inlining can speed up function evaluation significantly, because a (slow) call to a user-defined SQL function is avoided.

10351 07/19/2013 04:15 PM Aaron Marcuse-Kubitza

schemas/vegbien.my.sql: updated for new bin/repl text mode matching, which also affects non-regexps. this causes the replacement of a few more occurrences of PostgreSQL-only one-word typenames with their MySQL equivalents.

10350 07/19/2013 02:26 PM Aaron Marcuse-Kubitza

inputs/REMIB/Specimen/postprocess.sql: runtimes: documented the machine the times are from

10349 07/19/2013 01:52 PM Aaron Marcuse-Kubitza

inputs/REMIB/: switched to new-style import, using the steps at wiki.vegpath.org/Switching_to_new-style_import#stage-I-source-specific > "run the following for each datasource"

10348 07/19/2013 11:40 AM Aaron Marcuse-Kubitza

bugfix: bin/repl: text mode: repurpose this to match SQL identifiers, for use by inputs/input.Makefile %/postprocess.sql. %/postprocess.sql is the only place currently using this mode, so this will not affect other scripts.

10347 07/19/2013 10:51 AM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: %/postprocess.sql: need to run bin/repl in text mode (text=1) so that values to match are treated as literal strings rather than regular expressions. this difference is important for column names with spaces or special characters.

View all revisions | View revisions

Also available in: Atom