Project

General

Profile

Statistics
| Revision:

# Date Author Comment
7601 02/20/2013 01:43 AM Aaron Marcuse-Kubitza

schemas/vegbien.sql: analytical_stem_view: Removed deprecated taxonNameWithMorphospecies now that we have speciesBinomialWithMorphospecies

7600 02/20/2013 01:17 AM Aaron Marcuse-Kubitza

schemas/vegbien.sql: analytical_stem_view: speciesBinomial: Added morphospecies suffix to create speciesBinomialWithMorphospecies

7599 02/20/2013 12:15 AM Aaron Marcuse-Kubitza

inputs/import.stats.xls: Updated import times

7598 02/20/2013 12:06 AM Aaron Marcuse-Kubitza

README.TXT: Full database import: Check that unscrubbed_taxondetermination_view returns no rows: Documented that this takes 90 s with LIMIT 1

7597 02/19/2013 11:16 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: _taxon_family_require_std(): Also allow non-aceae families accepted by TNRS

7596 02/19/2013 09:38 PM Aaron Marcuse-Kubitza

Added inputs/SALVIAS/_archive/salvias_plots.*.sql.zip.md5

7595 02/19/2013 09:35 PM Aaron Marcuse-Kubitza

Added inputs/VegBank/_archive/vegbank_for_bien.tar.gz.url

7594 02/19/2013 09:29 PM Aaron Marcuse-Kubitza

Added inputs/U/UtrechtHerbarium.csv.tar.gz.url

7593 02/19/2013 09:28 PM Aaron Marcuse-Kubitza

Added inputs/TEAM/_archive/ci-team_extract.tar.gz.url

7592 02/19/2013 09:27 PM Aaron Marcuse-Kubitza

Added inputs/SpeciesLink/_archive/specieslink*.txt.gz.url

7591 02/19/2013 09:22 PM Aaron Marcuse-Kubitza

Added inputs/REMIB/_archive/remib_raw.csv.tar.gz.url

7590 02/19/2013 09:19 PM Aaron Marcuse-Kubitza

Added inputs/NY/NYSpecimenDataAmericas.csv.tar.gz.url

7589 02/19/2013 09:17 PM Aaron Marcuse-Kubitza

Added inputs/NCU/_archive/NCU-NCSC_2010-02-12.csv.tar.gz.url

7588 02/19/2013 09:14 PM Aaron Marcuse-Kubitza

Added inputs/MO/mo_digirexport.tar.gz.url

7587 02/19/2013 09:13 PM Aaron Marcuse-Kubitza

Added inputs/Madidi/_archive/2010-1-2/madidi_plots_original_12jan2010.zip.url

7586 02/19/2013 09:11 PM Aaron Marcuse-Kubitza

Added inputs/GBIF/gbif_extract.tar.gz.url

7585 02/19/2013 09:10 PM Aaron Marcuse-Kubitza

Added inputs/FIA/fia_extract.tar.gz.url

7584 02/19/2013 09:08 PM Aaron Marcuse-Kubitza

Added inputs/CVS/_archive/CVS-allTaxonOccurrences_2010-01-12.txt.tar.gz.url

7583 02/19/2013 09:04 PM Aaron Marcuse-Kubitza

Added inputs/ARIZ/ARIZ_DiGIR_21012010.csv.tar.gz.url

7582 02/19/2013 08:55 PM Aaron Marcuse-Kubitza

Added inputs/UNCC/Specimen/UNCC.csv.url, UNCC.csv.md5

7581 02/19/2013 08:45 PM Aaron Marcuse-Kubitza

Added inputs/XAL/_src/digir.xml.gz.md5

7580 02/19/2013 08:39 PM Aaron Marcuse-Kubitza

Added inputs/UNCC/_src/ with UNCC.csv.zip.md5

7579 02/19/2013 08:23 PM Aaron Marcuse-Kubitza

Added inputs/SpeciesLink/_src

7578 02/16/2013 08:24 AM Aaron Marcuse-Kubitza

README.TXT: Datasource setup: MySQL inputs: .sql exports: Use new mysql_bien to connect to the MySQL DB created for the datasource

7577 02/16/2013 08:22 AM Aaron Marcuse-Kubitza

Added mysql_bien, which runs a MySQL command on the local MySQL server

7576 02/16/2013 08:06 AM Aaron Marcuse-Kubitza

Added inputs/GBIF/_src/GBIFPortalDB-2012-12-11.dump.md5 (md5sum of the expanded file)

7575 02/16/2013 08:02 AM Aaron Marcuse-Kubitza

root Makefile: MySQL: mysql-Linux: Also install phpMyAdmin

7574 02/16/2013 08:01 AM Aaron Marcuse-Kubitza

root Makefile: MySQL: mysql-Linux: Split apt-get dependencies into separate commands, like for other apt-get commands, to avoid having one failed dependency prevent the following dependencies from being installed

7573 02/16/2013 07:57 AM Aaron Marcuse-Kubitza

root Makefile: MySQL: *mysql_users: Also add bien_read user

7572 02/16/2013 07:49 AM Aaron Marcuse-Kubitza

root Makefile: MySQL: Renamed *mysql_user to *mysql_users because there can be multiple users

7571 02/16/2013 06:51 AM Aaron Marcuse-Kubitza

inputs/: Added .md5 files for all .zip, .gz

7570 02/16/2013 06:47 AM Aaron Marcuse-Kubitza

Added inputs/HVAA/Specimen/Herbario_occur_1360871068.csv.url

7569 02/16/2013 06:39 AM Aaron Marcuse-Kubitza

lib/common.Makefile: rsync: $(rsync*): Use --no-group because the file group is different depending on the machine

7568 02/16/2013 06:10 AM Aaron Marcuse-Kubitza

input.Makefile: SVN: $(_svnFilesGlob): Also add .md5 files. This allows svn to track where unversioned files should be in the directory tree.

7567 02/16/2013 06:07 AM Aaron Marcuse-Kubitza

input.Makefile: SVN: $(_svnFilesGlob): .url, .pdf, and README.TXT in the top-level dir: Fixed bug where had extra / after brace expr

7566 02/16/2013 06:00 AM Aaron Marcuse-Kubitza

input.Makefile: SVN: $(_svnFilesGlob): Also add .url, .pdf, and README.TXT in the top-level dir

7565 02/16/2013 05:53 AM Aaron Marcuse-Kubitza

input.Makefile: SVN: $(_svnFilesGlob): Add .url, .pdf, and README.TXT files in all subdirs, not just _src

7564 02/16/2013 05:25 AM Aaron Marcuse-Kubitza

lib/common.Makefile: remote server: Use jupiter instead of vegbiendev, to ensure that all files get uploaded there rather than only to vegbiendev. This involves adding an extra database import step to download the uploaded files from jupiter onto vegbiendev.

7563 02/16/2013 02:50 AM Aaron Marcuse-Kubitza

inputs/FIA/_src/Makefile: all: Extract zip files before running tables target, because it requires the created dirs

7562 02/16/2013 02:40 AM Aaron Marcuse-Kubitza

schemas/vegbien.ERD.mwb: Fixed table sizes

7561 02/16/2013 01:17 AM Aaron Marcuse-Kubitza

Removed no longer used fix_permissions. Use root fix_perms instead.

7560 02/16/2013 01:16 AM Aaron Marcuse-Kubitza

Added root fix_perms

7559 02/15/2013 11:58 PM Aaron Marcuse-Kubitza

Moved Checksums from backups/Makefile to lib/common.Makefile so all dirs (including inputs/) can use md5sum testing

7558 02/15/2013 11:08 PM Aaron Marcuse-Kubitza

lib/common.Makefile: $(remote): Made remote basepath configurable in $(remote_basepath)

7557 02/15/2013 11:04 PM Aaron Marcuse-Kubitza

lib/common.Makefile: Renamed $(src_server) to $(remote_host) and $(src_user) to $(remote_user) for clarity

7556 02/15/2013 10:16 PM Aaron Marcuse-Kubitza

inputs/GBIF/: Added refresh metadata

7555 02/14/2013 11:49 AM Aaron Marcuse-Kubitza

Added inputs/HVAA/

7554 02/14/2013 11:14 AM Aaron Marcuse-Kubitza

Added inputs/ARIZ/_archive

7553 02/14/2013 11:13 AM Aaron Marcuse-Kubitza

inputs/ARIZ/: Removed previous data now that it has been refreshed

7552 02/14/2013 11:08 AM Aaron Marcuse-Kubitza

inputs/ARIZ/: Mapped refresh

7551 02/14/2013 09:48 AM Aaron Marcuse-Kubitza

Added inputs/ARIZ/import_order.txt

7550 02/14/2013 09:22 AM Aaron Marcuse-Kubitza

Added inputs/NY/_archive/

7549 02/14/2013 09:20 AM Aaron Marcuse-Kubitza

inputs/NY/: Removed tables from previous extract

7548 02/14/2013 08:59 AM Aaron Marcuse-Kubitza

inputs/NY/: Mapped refresh

7547 02/14/2013 08:58 AM Aaron Marcuse-Kubitza

inputs/*/*/VegBIEN.csv: Regenerated from mappings/VegCore-VegBIEN.csv

7546 02/14/2013 08:52 AM Aaron Marcuse-Kubitza

Added inputs/NY/import_order.txt

7545 02/14/2013 02:51 AM Aaron Marcuse-Kubitza

inputs/ARIZ/: Added SQL export for refresh

7544 02/14/2013 02:33 AM Aaron Marcuse-Kubitza

my2pg.data: Translate indefinite (zero) months which have a definite day. This is unusual, but does appear in some data such as the ARIZ DB.

7543 02/14/2013 02:28 AM Aaron Marcuse-Kubitza

my2pg.data: Translate indefinite dates (dates with 0 as the month or day)

7542 02/14/2013 02:23 AM Aaron Marcuse-Kubitza

my2pg: Use my2pg.data to perform data-only replacements, instead of duplicating them in both my2pg and my2pg.data

7541 02/14/2013 02:01 AM Aaron Marcuse-Kubitza

my2pg: named UNIQUE KEYs: Comment out the name because PostgreSQL requires it to be globally unique, but MySQL only requires it to be unique within the table

7540 02/14/2013 01:53 AM Aaron Marcuse-Kubitza

my2pg: Translate UNIQUE KEYs instead of removing them

7539 02/14/2013 01:49 AM Aaron Marcuse-Kubitza

my2pg*: Removed KEYs: Comment out the definition rather than removing it

7538 02/14/2013 01:45 AM Aaron Marcuse-Kubitza

my2pg*: Remove FOREIGN KEYs because MySQL does not dump tables in dependency order, which prevents PostgreSQL from creating tables whose fkeys refer to a later table

7537 02/14/2013 01:33 AM Aaron Marcuse-Kubitza

my2pg*: Replacing invalid table elements to remove them: Use a dummy CHECK constraint instead of a boolean field to avoid adding fields to the table. The elements can't always simply be removed because sed can't remove the trailing comma of the previous element, and removing the following comma doesn't work for the last element in the table.

7536 02/14/2013 12:11 AM Aaron Marcuse-Kubitza

my2pg*: Replace '0000-00-00 00:00:00' with '-infinity'

7535 02/14/2013 12:04 AM Aaron Marcuse-Kubitza

my2pg: Replace datetime with timestamp

7534 02/13/2013 11:59 PM Aaron Marcuse-Kubitza

my2pg: Remove COLLATE field attribute

7533 02/13/2013 11:56 PM Aaron Marcuse-Kubitza

lib/MySQL.*.sql.make: Documented that $server user/host are for ssh, not the DB

7532 02/13/2013 11:55 PM Aaron Marcuse-Kubitza

lib/MySQL.*.sql.make: Documented that $server can also contain a username (which will be used by ssh)

7531 02/13/2013 11:51 PM Aaron Marcuse-Kubitza

my2pg_export: Use the --quick option to facilitate exporting large tables (it avoids retrieving all rows before outputting any of them)

7530 02/13/2013 11:00 PM Aaron Marcuse-Kubitza

README.TXT: Datasource setup: Added instructions for MS Access databases

7529 02/13/2013 10:43 PM Aaron Marcuse-Kubitza

README.TXT: Datasource setup: MySQL inputs: Added instruction to skip the Add input data for each table section

7528 02/13/2013 10:40 PM Aaron Marcuse-Kubitza

inputs/NY/: Added SQL export for refresh

7527 02/12/2013 01:08 PM Aaron Marcuse-Kubitza

mappings/VegCore.htm: Regenerated from wiki. Brad's new DwC ID terms spreadsheet has now been added, and a number of the ID terms clarified, disambiguated, and recategorized. In particular, institutionCode has now been split into the custodialInstitutions and collectingInstitution, to differentiate between which institution has the specimen vs. stamped the specimen. This distinction is important because the catalogNumber, stamped on the specimen, is only unique within the collectingInstitution. Most datasources don't unambiguously specify which institution their institutionCode is referring to, so it has been assumed to be custodialInstitutions unless a data dictionary says otherwise (as is the case for UNCC). In addition, a MatchedTaxonDetermination table has been added with the *_matched fields from TNRS.

7526 02/12/2013 12:15 PM Aaron Marcuse-Kubitza

inputs/CVS/observation_/map.csv: baseSaturation: Resolved ambiguous term

7525 02/12/2013 12:09 PM Aaron Marcuse-Kubitza

mappings/Makefile: VegCore.vocab.csv: Ignore leading ? when sorting so that ambiguous terms sort alphabetically with other terms. This prevents terms from moving from their previous location when they become ambiguous.

7524 02/12/2013 12:07 PM Aaron Marcuse-Kubitza

Added sort_ci to sort a spreadsheet, ignoring leading punctuation

7523 02/12/2013 12:05 PM Aaron Marcuse-Kubitza

mappings/VegCore.vocab.csv: Changed line endings to \r\n in preparation for having a Python script run on it (which changes the line endings)

7522 02/12/2013 11:47 AM Aaron Marcuse-Kubitza

mappings/Makefile: VegCore.vocab.csv: Added back ambiguous terms, so that the vocabulary contains all terms defined by VegCore, regardless of whether they are ambiguous or unambiguous terms

7521 02/12/2013 11:44 AM Aaron Marcuse-Kubitza

mappings/Makefile: VegCore.vocab.csv: Added back synonyms, so that the vocabulary contains all terms defined by VegCore, regardless of whether they are synonyms or primary terms. This also prevents VegCore.vocab.csv from losing entries when terms are renamed, which made it difficult to verify that no terms were lost when refactoring.

7520 02/12/2013 05:50 AM Aaron Marcuse-Kubitza

inputs/MO/Specimen/postprocess.sql: Remove frameshifted rows by detecting InstitutionCodes without any letters

7519 02/12/2013 04:59 AM Aaron Marcuse-Kubitza

inputs/ARIZ/Specimen/map.csv: CollectorNumber/FieldNumber: Use /_first to map these identical fields to the same location

7518 02/12/2013 04:54 AM Aaron Marcuse-Kubitza

inputs/ARIZ/Specimen/map.csv: Fixed bug where the column names for InstitutionCode and CollectionCode were reversed in the source data

7517 02/12/2013 04:14 AM Aaron Marcuse-Kubitza

inputs/*/Specimen/map.csv for Canadensys sources: Remapped institutionID to UNUSED

7516 02/09/2013 07:45 AM Aaron Marcuse-Kubitza

mappings/VegCore.htm: Regenerated from wiki. The original*, accepted*, and verbatim* Taxon fields have now been moved to separate OriginalTaxonDetermination, AcceptedTaxonDetermination, and TaxonVerbatim tables.

7515 02/09/2013 06:52 AM Aaron Marcuse-Kubitza

mappings/VegCore.htm: Regenerated from wiki

7514 02/09/2013 06:34 AM Aaron Marcuse-Kubitza

mappings/VegCore.htm: Regenerated from wiki

7513 02/09/2013 04:08 AM Aaron Marcuse-Kubitza

README.TXT: Maintenance: VegCore data dictionary: Replaced VegCore.*.csv with VegCore.htm because now that VegCore.*.csv are sorted alphabetically, they generally don't change when VegCore.htm changes

7512 02/09/2013 04:04 AM Aaron Marcuse-Kubitza

mappings/VegCore.*.csv: Regenerated from wiki. A plain text label is now used for Replace with, which fixes a bug where the PRIVATE permalink pointed to its Replace with in realLatitude instead of its definition.

7511 02/09/2013 03:55 AM Aaron Marcuse-Kubitza

redmine_synonyms: Support plain text labels other than Alternative, such as Replace with

7510 02/09/2013 03:13 AM Aaron Marcuse-Kubitza

mappings/VegCore.*.csv: Regenerated from wiki. Alternatives now contain the "Alternative" label as plain text rather than as an image title, thus avoiding an HTML anchor conflict with the definition and allowing ambiguous terms to be placed before their alternatives as well as after.

7509 02/09/2013 03:11 AM Aaron Marcuse-Kubitza

README.TXT: Maintenance: VegCore data dictionary: Updated VegCore.csv filename to VegCore.*.csv

7508 02/09/2013 02:57 AM Aaron Marcuse-Kubitza

redmine_synonyms: Support alternatives which contain the "Alternative" label as plain text rather than as an image title. This is done to include the "Alternative" label in the HTML anchor and thus prevent the anchor from conflicting with the actual definition of the alternative (which would otherwise have the same anchor text). This allows ambiguous terms to be placed before their alternatives as well as after, because there won't be anchor conflicts that need to be resolved with careful ordering.

7507 02/09/2013 02:48 AM Aaron Marcuse-Kubitza

mappings/VegCore.csv: Regenerated from wiki. Taxon terms with prefixes for other TaxonDeterminations now indicate the analogous term in an "analogous to" label next to the term

7506 02/09/2013 02:47 AM Aaron Marcuse-Kubitza

mappings/VegCore.csv: Regenerated from wiki. Taxon terms with prefixes for other TaxonDeterminations now indicate the analogous term in an "analogous to" label next to the term

7505 02/09/2013 02:47 AM Aaron Marcuse-Kubitza

mappings/VegCore.csv: Regenerated from wiki. Taxon terms with prefixes for other TaxonDeterminations now indicate the analogous term in an "analogous to" label next to the term

7504 02/07/2013 01:57 PM Aaron Marcuse-Kubitza

mappings/VegCore-VegBIEN.csv: datasourceRecordID: Fixed bug where also need to add datasourceRecordID next to occurrenceID for an institutionCode remap switch

7503 02/07/2013 01:57 PM Aaron Marcuse-Kubitza

inputs/bien_web/observation/test.xml.ref: Regenerated

7502 02/07/2013 01:48 PM Aaron Marcuse-Kubitza

inputs/import.stats.xls: Updated import times using the import_times bugfix for times longer than a day