Project

General

Profile

Statistics
| Revision:
  • svn:ignore: *.pyc tnrs.url

# Date Author Comment
14855 10/19/2014 03:15 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: $sudo_cmd_wrapper: use esc() so this does not need to be embedded with [@] array notation

14854 10/19/2014 03:09 PM Aaron Marcuse-Kubitza

bugfix: lib/runscripts/util.run: $wrap_fn: this always needs to contain `is_runscript=1` because $is_runscript is not (and cannot be) exported

14853 10/19/2014 03:06 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: sudo(): when don't need sudo: do need `command` to have effects of invoking external command

14852 10/19/2014 03:00 PM Aaron Marcuse-Kubitza

fix: lib/sh/db.sh: pg_snapshot(): first need to run pg_stop_backup() in case a backup was not stopped cleanly

14851 10/19/2014 03:00 PM Aaron Marcuse-Kubitza

fix: lib/sh/db.sh: pg_stop_backup(): ignore benign "a backup is not in progress" error

14850 10/19/2014 01:19 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: sudo(): only run sudo if needed

14849 10/19/2014 01:10 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: sudo(): factored PATH-preserving code out into separate wrapper var that can also be used separately, eg. with `sudo -u`

14848 10/19/2014 01:03 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: sudo(): moved addition of -E (which should always happen) to separate, overridden sudo() function

14847 10/19/2014 12:51 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: moved -E to is_callable section because it just applies to that

14842 10/18/2014 04:24 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: added become_sudo alias

14841 10/18/2014 04:21 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: added $is_root

14826 10/14/2014 12:36 PM Aaron Marcuse-Kubitza

lib/sql_io.py: cleanup_table(): trim(): documented that this also converts character varying fields to text

14822 10/14/2014 10:09 AM Aaron Marcuse-Kubitza

bugfix: lib/sql_io.py: null_strs_str_default: removed "NA" because this is the abbr for a Spanish province (Navarra). this fixes the 2nd bug of #955, geovalidation duplicated rows.

14821 10/14/2014 10:00 AM Aaron Marcuse-Kubitza

lib/sql_io.py: cleanup_table(): debug-print null_strs

14820 10/14/2014 09:56 AM Aaron Marcuse-Kubitza

lib/sql_io.py: null_strs: made it customizable from an env var, since the same list of null_strs doesn't work for all datasources (see #957)

14817 10/14/2014 09:39 AM Aaron Marcuse-Kubitza

fix: *Makefile: changed line endings to \n so that `patch` can work with pasted input. use `svn di --extensions --ignore-eol-style` to verify no diff.

14816 10/14/2014 08:35 AM Aaron Marcuse-Kubitza

lib/sql_io.py: null_strs: made it customizable from an env var, since the same list of null_strs doesn't work for all datasources (see #957)

14803 10/07/2014 12:01 AM Aaron Marcuse-Kubitza

bugfix: lib/tnrs.py: encode_map: also need to encode + because TNRS removes it from the morphospecies (vegpath.org/wiki/CVS_validation#Bobs-revised-document > issue #4)

14785 09/30/2014 07:36 AM Aaron Marcuse-Kubitza

lib/sql_io.py: null_strs: added N/A and NA (this will remove a common abbr for North America, but we don't use the continent, so this is OK)

14776 09/26/2014 10:21 PM Aaron Marcuse-Kubitza

lib/runscripts/table.run: added check_headers()

14775 09/26/2014 10:21 PM Aaron Marcuse-Kubitza

bugfix: lib/runscripts/table.run: srcs: missing []

14774 09/26/2014 10:08 PM Aaron Marcuse-Kubitza

lib/runscripts/table.run: added header() and use it in header.txt()

14773 09/26/2014 09:50 PM Aaron Marcuse-Kubitza

lib/runscripts/table.run: 1st_src(): use a variable for this instead, to avoid needing to run this function each time it's used, and to make @srcs available

14772 09/26/2014 09:22 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: wildcard.(): clarified that it only removes . .. when at then beginning of the list

14768 09/26/2014 02:31 AM Aaron Marcuse-Kubitza

lib/runscripts/table.run: added header.txt()

14767 09/26/2014 02:31 AM Aaron Marcuse-Kubitza

lib/runscripts/table.run: added 1st_src()

14765 09/26/2014 02:30 AM Aaron Marcuse-Kubitza

lib/runscripts/subdir.run: subdir_make(): use new $datasrc_dir

14764 09/26/2014 02:29 AM Aaron Marcuse-Kubitza

lib/runscripts/subdir.run: added $datasrc_dir

14763 09/26/2014 02:29 AM Aaron Marcuse-Kubitza

lib/sh/make.sh: make(): added support for $output_data mode which uses --silent

14762 09/26/2014 12:36 AM Aaron Marcuse-Kubitza

fix: lib/sh/util.sh: already_exists_msg(): changed calling convention to avoid it seeming like `return 0` is run if already_exists_msg() throws an error, when in fact already_exists_msg() is just a command that should be run before returning/errexiting

14760 09/25/2014 03:28 PM Aaron Marcuse-Kubitza

bugfix: lib/common.Makefile: $(wildcard/): need final pass with $(wildcard) to support inputs without wildcard chars

14737 09/24/2014 07:50 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: $sync_remote_url: jupiter user is always aaronmk, not the local user

14736 09/24/2014 07:41 PM Aaron Marcuse-Kubitza

*{.sh,run}: use standard WARNING syntax for warning labels

14734 09/24/2014 07:26 PM Aaron Marcuse-Kubitza

*{.sh,run}: use standard WARNING syntax for warning labels

14733 09/24/2014 07:25 PM Aaron Marcuse-Kubitza

fix: lib/sh/util.sh: verbosity_compat(): documented that this should not be run until right before executing an external command, so that it doesn't mess up the logging mechanism

14731 09/24/2014 06:55 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: functions called by pst(): commented out/removed logging calls that would have caused infinite recursion when pst()'ing a logging function

14730 09/24/2014 06:38 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: echo_func(): get call context before wrapper(s), which is more useful for debugging. this uses skip_stack_frames()'s lookahead=1 mode.

14729 09/24/2014 06:37 PM Aaron Marcuse-Kubitza

fix: lib/sh/util.sh: skip_stack_frames() callers: updated for new skip_stack_frames rather than get_stack_frame behavior

14728 09/24/2014 06:32 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: skip_stack_frames(): added lookahead support, which looks at entry after current to deterine whether to skip current. this is useful for skipping wrappers, by looking at the calling function's name.

14727 09/24/2014 06:30 PM Aaron Marcuse-Kubitza

fix: lib/sh/util.sh: skip_stack_frames(): for new skip_stack_frames rather than get_stack_frame behavior, stack frames must be skipped in the caller to preserve the stack frame pointer

14726 09/24/2014 06:26 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: get_stack_frame_after(): renamed to skip_stack_frames() for clarity

14725 09/24/2014 06:25 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: added skip_stack_frame_in_caller, unskip_stack_frame_in_caller

14724 09/24/2014 06:24 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: added prev_stack_frame

14723 09/24/2014 07:04 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: echo_func(): use new format_stack_frame, which adds call context information to what was provided by func_loc

14722 09/24/2014 07:02 AM Aaron Marcuse-Kubitza

fix: lib/sh/util.sh: format_stack_frame(): need to hide canon_rel_path() info using log+

14721 09/24/2014 07:01 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: added get_stack_frame_after()

14720 09/24/2014 06:59 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: added matches()

14719 09/24/2014 06:58 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: next_stack_frame: documented usage

14718 09/24/2014 04:24 AM Aaron Marcuse-Kubitza

fix: lib/runscripts/util.run: runscript template: all(): moved example commands to target(), where they would more likely be located

14717 09/24/2014 02:35 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: format_stack_frame(): support including args

14716 09/24/2014 12:08 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: debugging: added pst() (print_stack_trace)

14715 09/24/2014 12:05 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: added stack_trace(), print_stack_trace()

14714 09/24/2014 12:04 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: added format_stack_frame()

14713 09/24/2014 12:03 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: added get_stack_frame() and helpers

14712 09/23/2014 10:29 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: terminal: moved before errors so it can be used by it

14711 09/23/2014 10:25 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: errors, debugging: moved after datatype sections so their functions can use these

14710 09/22/2014 05:45 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: canon_rel_path() stub: proper no-op requires passing through original path

14709 09/22/2014 05:42 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: canon_rel_path(): fall back to original path if can't resolve, instead of errexiting

14708 09/22/2014 05:39 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: canon_rel_path(): don't re-localize $path because this clears it

14707 09/22/2014 05:36 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: canon_rel_path(): import $1 to $path before function body, so that the function body can be moved to a nested function

14706 09/22/2014 03:48 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: added canon_rel_path() stub for use by debugging functions

14704 09/22/2014 03:36 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: moved func_loc() to before debugging section so it can be used by debugging functions

14703 09/20/2014 02:34 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: command__exec(): need to restore $verbosity before calling die_e

14694 09/16/2014 04:25 PM Aaron Marcuse-Kubitza

fix: lib/sh/local.sh: $sync_remote_url: need $USER so user can be overridden when running as root

14686 09/16/2014 02:46 PM Aaron Marcuse-Kubitza

lib/Firefox_bookmarks.reformat.csv: label page's self-description as such: also support quotations enclosed in '

14682 09/10/2014 04:48 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: echo_vars(): merge repeated flags so there aren't flags in between the vars (which is also not valid declare syntax)

14681 09/10/2014 04:15 PM Aaron Marcuse-Kubitza

lib/sh/db.sh: pg_cmd(): log vars on same line to avoid clutter

14680 09/10/2014 03:17 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: echo_vars(): put all the vars on the same line so they don't clutter up the call graph generated at the default verbosity

14622 08/28/2014 08:13 PM Aaron Marcuse-Kubitza

lib/tnrs.py single_tnrs_request(), bin/tnrs_client: use_tnrs_export: default to False because this mode uses incorrect selected matches (vegpath.org/issues/943), and the JSON mode that fixes this is now available

14620 08/28/2014 07:57 PM Aaron Marcuse-Kubitza

bugfix: lib/csvs.py: JsonReader: need to pass col_order to row_dict_to_list_reader

14618 08/28/2014 07:12 PM Aaron Marcuse-Kubitza

bugfix: lib/tnrs.py: JSON output: need to stringify arrays so they match what is output in TSV-export mode

14617 08/28/2014 07:10 PM Aaron Marcuse-Kubitza

lib/csvs.py: JsonReader: added support for values that are arrays

14616 08/28/2014 07:05 PM Aaron Marcuse-Kubitza

lib/csvs.py: MultiFilter: inherit from WrapReader instead of Filter to avoid needing to define a no-op filter_() function

14615 08/28/2014 06:49 PM Aaron Marcuse-Kubitza

bugfix: lib/csvs.py: row_dict_to_list_reader: need to override next() directly instead of just using Filter, because Filter doesn't support returning multiple rows for one input row (in this case, prepending a header row). this caused the 1st data row to be missing.

14614 08/28/2014 06:47 PM Aaron Marcuse-Kubitza

lib/csvs.py: Filter: inherit from WrapReader, which separates out the CSV-reader API code

14613 08/28/2014 06:43 PM Aaron Marcuse-Kubitza

lib/csvs.py: added WrapReader

14612 08/28/2014 06:43 PM Aaron Marcuse-Kubitza

lib/csvs.py: added Reader

14600 08/28/2014 03:10 AM Aaron Marcuse-Kubitza

lib/csvs.py: JsonReader: factored out row-dict-to-list into new row_dict_to_list_reader so that JSON-specific preprocessing is kept separate from the row format translation

14599 08/27/2014 03:17 PM Aaron Marcuse-Kubitza

lib/csvs.py: added MultiFilter, which enables applying multiple filters by nesting

14598 08/26/2014 07:57 PM Aaron Marcuse-Kubitza

lib/tnrs.py: single_tnrs_request(): JSON mode: implemented output of JSON data

14597 08/26/2014 07:53 PM Aaron Marcuse-Kubitza

lib/tnrs.py: single_tnrs_request(): factored out wrapping in TnrsOutputStream, since this is done for both modes

14596 08/26/2014 07:47 PM Aaron Marcuse-Kubitza

fix: lib/tnrs.py: JSON mode: TSV export columns: need to translate these to JSON column names before they can be used with the JSON data

14595 08/26/2014 07:44 PM Aaron Marcuse-Kubitza

lib/csvs.py: added JsonReader, which reads parsed JSON data as row tuples

14594 08/26/2014 07:43 PM Aaron Marcuse-Kubitza

lib/csvs.py: added row_dict_to_list(), which translates a CSV dict-based row to a list-based one

14593 08/26/2014 07:43 PM Aaron Marcuse-Kubitza

lib/csvs.py: RowNumFilter: added support for filtering the header row as well

14592 08/26/2014 07:42 PM Aaron Marcuse-Kubitza

lib/csvs.py: ColInsertFilter: added support for filtering the header row as well

14591 08/26/2014 05:12 PM Aaron Marcuse-Kubitza

lib/csvs.py: InputRewriter: documented that this is also a stream (in addition to inheriting from StreamFilter)

14590 08/26/2014 05:11 PM Aaron Marcuse-Kubitza

bugfix: lib/csvs.py: InputRewriter: accept a reader, as would be expected, instead of a custom stream whose lines are tuples

14589 08/26/2014 05:08 PM Aaron Marcuse-Kubitza

fix: lib/sql_io.py: append_csv(): use new csvs.ProgressInputFilter instead of streams.ProgressInputStream(csvs.StreamFilter(__)), so that the input to csvs.InputRewriter is a reader, not a stream. this avoids the need for csvs.InputRewriter to accept a stream whose lines are tuples, instead of the expected reader.

14586 08/26/2014 04:49 PM Aaron Marcuse-Kubitza

lib/csvs.py: added ProgressInputFilter, analogous to streams.ProgressInputStream

14585 08/26/2014 04:46 PM Aaron Marcuse-Kubitza

lib/sql_io.py: added commented-out debug statement used to troubleshoot copy_expert() errors

14584 08/26/2014 04:45 PM Aaron Marcuse-Kubitza

lib/dicts.py: added pair_keys(), pair_values()

14583 08/26/2014 04:15 PM Aaron Marcuse-Kubitza

bugfix: lib/streams.py: CaptureStream: end_idx must also be > start_idx

14578 08/25/2014 10:17 PM Aaron Marcuse-Kubitza

lib/tnrs.py: single_tnrs_request(): use_tnrs_export=False: need to obtain export columns

14577 08/25/2014 10:16 PM Aaron Marcuse-Kubitza

lib/csvs.py: added header(stream)

14576 08/25/2014 10:16 PM Aaron Marcuse-Kubitza

fix: lib/tnrs.py: single_tnrs_request(): need to `assert name_ct >= 1`, because with no names, TNRS hangs indefinitely

14545 08/21/2014 12:40 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/archives.sh: compress(): don't include dir prefix in zip archive

14544 08/21/2014 12:40 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: cd(): use echo_run instead of a manual echo_cmd call

14543 08/21/2014 12:35 PM Aaron Marcuse-Kubitza

fix: lib/sh/util.sh: cd(): indent after running cd rather than before

14542 08/21/2014 12:32 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: cd(): support rebasing path vars for the new dir