Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 10267 over 11 years Aaron Marcuse-Kubitza backups/Makefile: %.backup/restore: documented ...
  bin 10348 over 11 years Aaron Marcuse-Kubitza bugfix: bin/repl: text mode: repurpose this to ...
  config 7801 over 11 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: mk_db: Added command...
  exports 9928 over 11 years Aaron Marcuse-Kubitza added exports/_archive/
  inputs 10376 over 11 years Aaron Marcuse-Kubitza inputs/REMIB/Specimen/postprocess.sql: map_null...
  lib 10372 over 11 years Aaron Marcuse-Kubitza lib/runscripts/table.run: postprocess(): added ...
  mappings 10289 over 11 years Aaron Marcuse-Kubitza mappings/VegCore.htm: regenerated from wiki. Sp...
  planning 10311 over 11 years Aaron Marcuse-Kubitza planning/timeline/timeline.2013.xls: moved Indi...
  schemas 10375 over 11 years Aaron Marcuse-Kubitza schemas/util.sql: map_nulls(): documented that ...
  web 10306 over 11 years Aaron Marcuse-Kubitza web/links/index.htm: updated to Firefox bookmar...
.htaccess 326 Bytes 8771 over 11 years Aaron Marcuse-Kubitza /.htaccess: use canonical URL without symlinks
.rsync_filter.upload 33 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
.rsync_ignore 12 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
Makefile 12.6 KB 10223 over 11 years Aaron Marcuse-Kubitza /Makefile: mysql-Linux: also install mysql-work...
README.TXT 23.2 KB 10286 over 11 years Aaron Marcuse-Kubitza /README.TXT: Maintenance: regenerate mappings/V...
fix_perms 97 Bytes 7560 almost 12 years Aaron Marcuse-Kubitza Added root fix_perms
map 1001 Bytes 6949 almost 12 years Aaron Marcuse-Kubitza vegbien_dest: Changed default $prefix to "", so...
new_terms.csv 38.1 KB 7222 almost 12 years Aaron Marcuse-Kubitza new_terms.csv: Regenerated
run 433 Bytes 9916 over 11 years Aaron Marcuse-Kubitza /run: geoscrub_input/make(): documented runtime...
unmapped_terms.csv 13.1 KB 7201 almost 12 years Aaron Marcuse-Kubitza **/new_terms.csv, **/unmapped_terms.csv: Regene...

Latest revisions

# Date Author Comment
10376 07/20/2013 05:07 AM Aaron Marcuse-Kubitza

inputs/REMIB/Specimen/postprocess.sql: map_nulls() derived cols: updated runtimes for map_nulls() inlining, which created a speed improvement of 7x for the numeric columns and 2.5x for the text columns (292563.362->41929.772 ms and 83640.424->35690.797 ms, respectively). note that the map_nulls__coord__*() calls could be optimized further by combining the successive map_nulls() calls into one, with the hstores merged.

10375 07/20/2013 04:37 AM Aaron Marcuse-Kubitza

schemas/util.sql: map_nulls(): documented that inputs/REMIB/Specimen/postprocess.sql > country also shows that inlining is now happening properly. note that the speed improvement due to inlining is not as much, %wise, when the values util._map() is run on are long strings instead of the short strings used in the initial profiling. this is because a greater % of the time is spent in system functions such as hstore>text, which are not affected by the inlining because they are run either way.

10374 07/20/2013 04:18 AM Aaron Marcuse-Kubitza

schemas/util.sql: map_nulls(): use new nulls_map(). proper inlining (i.e. same runtime before and after change) has been verified with the following profiling query:
SELECT util.map_nulls(array[1, 2, 3]::text[], v) FROM unnest(array_fill(1, array100000)) f (v)

10373 07/20/2013 04:05 AM Aaron Marcuse-Kubitza

schemas/util.sql: added nulls_map(), for use with _map()

10372 07/20/2013 03:39 AM Aaron Marcuse-Kubitza

lib/runscripts/table.run: postprocess(): added remake action that calls trim_table()

10371 07/20/2013 03:37 AM Aaron Marcuse-Kubitza

lib/runscripts/table.run: added trim_table(), which calls util.trim(regclass, regclass)

10370 07/20/2013 03:23 AM Aaron Marcuse-Kubitza

lib/runscripts/table.run: map_table(): added remake action that calls reset_col_names()

10369 07/20/2013 03:21 AM Aaron Marcuse-Kubitza

lib/runscripts/table.run: added reset_col_names(), which calls util.reset_col_names()

10368 07/20/2013 03:19 AM Aaron Marcuse-Kubitza

bugfix: lib/runscripts/table.run: map_table(): moved $map_table to global var so it can be used by other functions

10367 07/20/2013 03:09 AM Aaron Marcuse-Kubitza

bugfix: lib/runscripts/table.run: postprocess(): don't propagate $remake to remake_VegBIEN_mappings(), since this will cause map.csv to be remade, which is not related to the postprocessing.

View all revisions | View revisions

Also available in: Atom