backups/TNRS.backup.md5: updated
backups/pg_snapshot: documented runtime for initial backup (4 h)
backups/mysql_snapshot: documented initial vegbiendev->jupiter upload time for GBIF/raw_occurrence_record.MYD (7 h for 91 GB = 3.7 MB/s)
added backups/mysql_snapshot, pg_snapshot
added backups/vegbien.r13002.backup.md5, vegbien.r13160.backup.md5
added backups/users.sql.run
added backups/vegbien.r12025.backup.md5
moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).
updated backups/TNRS.backup.md5
added backups/vegbien.r11786.backup.md5
backups/TNRS.backup: saved copy backups/TNRS.2013-11-18.backup
added backups/vegbien.r11549.backup.md5
added backups/vegbien.r10848.backup.md5
bugfix: backups/fix_perms: just make the backups themselves private, since the other files are in svn, and their permissions should match their accessibility through Redmine
added backups/vegbien.r10548.backup.md5
bugfix: backups/: svn:ignore: removed *.md5, which should be under version control
updated backups/TNRS.backup
backups/TNRS.backup.md5: updated for last backup
backups/Makefile: %.backup/restore: documented runtime (11 h; ~5 h to insert data). note that this is still much shorter than re-running column-based import.
backups/Makefile: $(restore): added --verbose to display pg_restore's incremental progress
bugfix: backups/.rsync_filter.download: need to prevent existing backups from being deleted on the local side, too, by changing hide patterns to exclude
backups/Makefile: upload: use bin/sync_upload
added backups/vegbien.r9897.backup.md5
backups/.rsync_ignore: replaced with .rsync_filter.upload to allow uploading new backups but not deleting existing backups if they don't exist on the local (rsync-invoking) side; and .rsync_filter.download to avoid downloading backups to the local side. this allows storing older backups just on jupiter, where there is much more disk space. note that this change must be made on the remote side (jupiter) for it to be effective, because these are remote-side rules and are only processed by the remote-side rsync instance.
added backups/TNRS.2013-6-17.backup.md5, TNRS.2013-6-22.backup.md5
added backups/.rsync_ignore with filters from /README.TXT > Maintenance > to synchronize vegbiendev, jupiter, and your local machine. these filters will now be used with bin/sync_upload in addition to the periodic backup commands.
added backups/TNRS.backup.md5, vegbien.r9459.backup.md5
added backups/*.md5
added backups/TNRS.2013-5-21.backup.md5
lib/common.Makefile: remote server: Use jupiter instead of vegbiendev, to ensure that all files get uploaded there rather than only to vegbiendev. This involves adding an extra database import step to download the uploaded files from jupiter onto vegbiendev.
Moved Checksums from backups/Makefile to lib/common.Makefile so all dirs (including inputs/) can use md5sum testing
lib/common.Makefile: Renamed $(src_server) to $(remote_host) and $(src_user) to $(remote_user) for clarity
backups/Makefile: %.md5/test: Added comment to run with `make -s` to avoid echoing make commands
backups/Makefile: Synchronization: %/download: Download the .md5 file first, so that the user is prompted right away for their password rather than after the main file has finished downloading, by which time the password prompt has timed out
backups/Makefile: Removed no longer used $(psqlVerbose)
backups/Makefile: Removed %.backup/rm_indexes, which is no longer needed because archived imports are now backed up instead of being stored without indexes in the live DB
backups/Makefile: %.backup/remove: Fixed bug where need to use $no_search_path option to psql_script_vegbien
backups/Makefile: Factored $(isMac) out into lib/common.Makefile
backups/Makefile; $(public*): Don't add a "public." prefix to get the name of the public schema
backups/Makefile: Removed no longer used $(rmSchema)
backups/Makefile: Use \$(rmSchemaCmd) from lib/common.Makefile instead of \$(rmSchema)
backups/Makefile: Removed no longer used $(dateFmt), $(mtime)
backups/Makefile: Removed %.backup/rotate, because this incorrectly causes the current time rather than the version to be used in the backup filename. The version should instead be specified in the backup filename when it's created.
Makefiles: Factored out common vars/functions into lib/common.Makefile
backups/Makefile: Synchronization: $(remote): Fixed bug where need trailing / at end of path
backups/Makefile: Synchronization: $(remote): Updated path to backups
backups/fix_perms: Removed world read permissions from backups dir. Note that this will require superuser permissions to view archived backups on jupiter, because the bien group is not set up with the same members as on vegbiendev. (On jupiter, it contains only stri,regetz,donoghue,naiamh.)
Added backups/fix_perms
backups/Makefile: Synchronization: %/download: Also download any .md5 file for the file
backups/Makefile: Synchronization: Replaced download target, which downloads all backups, with %/download, which downloads just a specific backup, because you would generally only want to extract a single backup from the archive for reinstallation
backups/Makefile: Synchronization: Sync with jupiter instead of vegbiendev. This requires running `make backups/upload` on vegbiendev to archive the files, instead of `make backups/download` to download them to your local machine.
backups/Makefile: Checksums: %.md5/test: Only use md5sum's -v option on Mac, because it's not supported on Linux (there, verbose mode is the default)
backups/Makefile: SQL: Full DB: vegbien.%.backup: Also generate MD5 sum
backups/Makefile: Support removing public schema versions based on the version of a full DB backup
backups/Makefile: Synchronization: Also sync *.md5
backups/Makefile: Checksums: %.md5/test: Made it an _always target
backups/Makefile: Checksums: Added %.md5/test to test generated checksums
backups/Makefile: Moved md5-related targets to separate Checksums section
backups/Makefile: %.md5: Removed not applicable comment which had been copied from %.sql
backups/Makefile: %.md5: Fixed bug where md5sum does not have a -q option like md5
backups/Makefile: %.md5: Fixed bug where need to use md5sum instead of md5 on Linux
backups/Makefile: Added md5s target to generate .md5 files for all backups
backups/Makefile: %.md5: Run with `nice -n +5` to avoid slowing down the UI
backups/: svn:ignore: Added *.md5. Removed no longer applicable *.log.
backups/Makefile: Changed paths to be relative to the Makefile rather than the current directory, so this Makefile can be used in other directories as well (such as jupiter:/aaronmk/VegBIEN.backups/)
backups/Makefile: %.backup: Also create MD5 of backup
backups/Makefile: Added %.md5 target to create checksums of each backup
backups/Makefile: $(rsyncBackups): Added *.csv
backups/: Ignore _* and *.csv
backups/Makefile: Backups: Full DB: Specify the date suffix of the backup when it's created rather than adding it afterwards. This allows the user to specify a suffix that matches the corresponding public-schema backup.
backups/: svn:ignore: Also ignore .*, which includes temp files generated by rsync
backups/Makefile: Added synchronization of backups with vegbiendev. Added downloading backups to After a new import steps.
backups/Makefile: Refactored to include lib/common.Makefile
backups/Makefile: Full DB: Added vegbien.backup/all to run both test and rotate
backups/Makefile: .sql: When testing, turn it off so make won't skip `.sql: %` in favor of it
backups/Makefile: Split %.backup and %.sql into separate targets for clarity
backups/Makefile: Added %.backup/rotate
backups/Makefile: Rearranged sections so that backup targets, which apply to both Archived imports and Full DB, are at the top of a common Backups section
backups/Makefile: Added %.backup/test
backups/Makefile: Full DB backups: Added vegbien.backup
backups/Makefile: Factored backup command into $(backup) for later use by full DB backups. Made Backups, Archived imports sections subsections of Archived imports so Full DB backups can have its own section.
backups/Makefile: Fixed bug where $(SHELL) needed to be set to bash so that $'\n' would be interpreted correctly
backups/Makefile: Fixed bug where *.sql files needed to be restored using psql because pg_restore only supports "non-plain-text formats"
backups/Makefile: Don't log stderr or run the command verbosely and instead just output the command and run time to the terminal. This matches what we do for pg_dump, which works better because it just prints the useful information when it's done running.
backups/Makefile: Remove log files after successful restore/extraction because they are only useful for tail -f when the restore operation is running in the background
backups/Makefile: Backups: Fixed bug where `%.sql: %` needed to come before %.sql with no prerequisites to be matched first
Moved archived imports and make targets to maintain them to new backups dir
