backups/*retention_policy*: on jupiter: changed to avoid retaining backups further back, as these consume significant disk space on jupiter and are not useful for anything
backups/*retention_policy*: added explanations
backups/*retention_policy*: on jupiter: backups further back: removed "if disk space permits" because this is already labeled "optionally"
backups/*retention_policy*: changed to require retaining *.backup of the last 2 successful imports on all machines
backups/*retention_policy*: allow keeping *.backup of the last 2 successful imports on all machines, not just jupiter
: renamed 2TB drive's BIEN3 partition to BIEN3.**SAVE since one might not see the SAVE file in it
/"**DO_NOT_DELETE": renamed to shorter SAVE
added backups/*retention_policies*/ with retention policy files for each partition
backups/README.TXT: renamed to retention_policy to match the naming convention of the retention policy files in the various partitions
backups/TNRS.*: removed no longer needed old TNRS backups, which are part of the respective full-database backups in any case
backups/vegbien.r14089.backup.md5: updated
backups/TNRS.backup.md5: updated
added backups/vegbien.r14089.backup.md5
backups/README.TXT: added backup retention policy
backups/pg_snapshot: documented runtime for initial backup (4 h)
backups/mysql_snapshot: documented initial vegbiendev->jupiter upload time for GBIF/raw_occurrence_record.MYD (7 h for 91 GB = 3.7 MB/s)
added backups/mysql_snapshot, pg_snapshot
added backups/vegbien.r13002.backup.md5, vegbien.r13160.backup.md5
added backups/users.sql.run
added backups/vegbien.r12025.backup.md5
moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).
updated backups/TNRS.backup.md5
added backups/vegbien.r11786.backup.md5
backups/TNRS.backup: saved copy backups/TNRS.2013-11-18.backup
added backups/vegbien.r11549.backup.md5
added backups/vegbien.r10848.backup.md5
bugfix: backups/fix_perms: just make the backups themselves private, since the other files are in svn, and their permissions should match their accessibility through Redmine
added backups/vegbien.r10548.backup.md5
bugfix: backups/: svn:ignore: removed *.md5, which should be under version control
updated backups/TNRS.backup
backups/TNRS.backup.md5: updated for last backup
backups/Makefile: %.backup/restore: documented runtime (11 h; ~5 h to insert data). note that this is still much shorter than re-running column-based import.
backups/Makefile: $(restore): added --verbose to display pg_restore's incremental progress
bugfix: backups/.rsync_filter.download: need to prevent existing backups from being deleted on the local side, too, by changing hide patterns to exclude
backups/Makefile: upload: use bin/sync_upload
added backups/vegbien.r9897.backup.md5
backups/.rsync_ignore: replaced with .rsync_filter.upload to allow uploading new backups but not deleting existing backups if they don't exist on the local (rsync-invoking) side; and .rsync_filter.download to avoid downloading backups to the local side. this allows storing older backups just on jupiter, where there is much more disk space. note that this change must be made on the remote side (jupiter) for it to be effective, because these are remote-side rules and are only processed by the remote-side rsync instance.
added backups/TNRS.2013-6-17.backup.md5, TNRS.2013-6-22.backup.md5
added backups/.rsync_ignore with filters from /README.TXT > Maintenance > to synchronize vegbiendev, jupiter, and your local machine. these filters will now be used with bin/sync_upload in addition to the periodic backup commands.
added backups/TNRS.backup.md5, vegbien.r9459.backup.md5
added backups/*.md5
added backups/TNRS.2013-5-21.backup.md5
lib/common.Makefile: remote server: Use jupiter instead of vegbiendev, to ensure that all files get uploaded there rather than only to vegbiendev. This involves adding an extra database import step to download the uploaded files from jupiter onto vegbiendev.
Moved Checksums from backups/Makefile to lib/common.Makefile so all dirs (including inputs/) can use md5sum testing
lib/common.Makefile: Renamed $(src_server) to $(remote_host) and $(src_user) to $(remote_user) for clarity
backups/Makefile: %.md5/test: Added comment to run with `make -s` to avoid echoing make commands
backups/Makefile: Synchronization: %/download: Download the .md5 file first, so that the user is prompted right away for their password rather than after the main file has finished downloading, by which time the password prompt has timed out
backups/Makefile: Removed no longer used $(psqlVerbose)
backups/Makefile: Removed %.backup/rm_indexes, which is no longer needed because archived imports are now backed up instead of being stored without indexes in the live DB
backups/Makefile: %.backup/remove: Fixed bug where need to use $no_search_path option to psql_script_vegbien
backups/Makefile: Factored $(isMac) out into lib/common.Makefile
backups/Makefile; $(public*): Don't add a "public." prefix to get the name of the public schema
backups/Makefile: Removed no longer used $(rmSchema)
backups/Makefile: Use \$(rmSchemaCmd) from lib/common.Makefile instead of \$(rmSchema)
backups/Makefile: Removed no longer used $(dateFmt), $(mtime)
backups/Makefile: Removed %.backup/rotate, because this incorrectly causes the current time rather than the version to be used in the backup filename. The version should instead be specified in the backup filename when it's created.
Makefiles: Factored out common vars/functions into lib/common.Makefile
backups/Makefile: Synchronization: $(remote): Fixed bug where need trailing / at end of path
backups/Makefile: Synchronization: $(remote): Updated path to backups
backups/fix_perms: Removed world read permissions from backups dir. Note that this will require superuser permissions to view archived backups on jupiter, because the bien group is not set up with the same members as on vegbiendev. (On jupiter, it contains only stri,regetz,donoghue,naiamh.)
Added backups/fix_perms
backups/Makefile: Synchronization: %/download: Also download any .md5 file for the file
backups/Makefile: Synchronization: Replaced download target, which downloads all backups, with %/download, which downloads just a specific backup, because you would generally only want to extract a single backup from the archive for reinstallation
backups/Makefile: Synchronization: Sync with jupiter instead of vegbiendev. This requires running `make backups/upload` on vegbiendev to archive the files, instead of `make backups/download` to download them to your local machine.
backups/Makefile: Checksums: %.md5/test: Only use md5sum's -v option on Mac, because it's not supported on Linux (there, verbose mode is the default)
backups/Makefile: SQL: Full DB: vegbien.%.backup: Also generate MD5 sum
backups/Makefile: Support removing public schema versions based on the version of a full DB backup
backups/Makefile: Synchronization: Also sync *.md5
backups/Makefile: Checksums: %.md5/test: Made it an _always target
backups/Makefile: Checksums: Added %.md5/test to test generated checksums
backups/Makefile: Moved md5-related targets to separate Checksums section
backups/Makefile: %.md5: Removed not applicable comment which had been copied from %.sql
backups/Makefile: %.md5: Fixed bug where md5sum does not have a -q option like md5
backups/Makefile: %.md5: Fixed bug where need to use md5sum instead of md5 on Linux
backups/Makefile: Added md5s target to generate .md5 files for all backups
backups/Makefile: %.md5: Run with `nice -n +5` to avoid slowing down the UI
backups/: svn:ignore: Added *.md5. Removed no longer applicable *.log.
backups/Makefile: Changed paths to be relative to the Makefile rather than the current directory, so this Makefile can be used in other directories as well (such as jupiter:/aaronmk/VegBIEN.backups/)
backups/Makefile: %.backup: Also create MD5 of backup
