Project

General

Profile

Statistics
| Revision:
  • svn:ignore: .* _* *.backup *.sql *.csv

# Date Author Comment
13563 05/30/2014 02:49 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

13472 05/17/2014 06:01 PM Aaron Marcuse-Kubitza

backups/pg_snapshot: documented runtime for initial backup (4 h)

13294 04/23/2014 12:09 PM Aaron Marcuse-Kubitza

backups/mysql_snapshot: documented initial vegbiendev->jupiter upload time for GBIF/raw_occurrence_record.MYD (7 h for 91 GB = 3.7 MB/s)

13272 04/22/2014 04:00 AM Aaron Marcuse-Kubitza

added backups/mysql_snapshot, pg_snapshot

13194 04/19/2014 10:13 PM Aaron Marcuse-Kubitza

added backups/vegbien.r13002.backup.md5, vegbien.r13160.backup.md5

13193 04/19/2014 10:12 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

13167 04/18/2014 06:57 PM Aaron Marcuse-Kubitza

added backups/users.sql.run

13008 04/01/2014 11:26 AM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

12176 02/13/2014 12:54 PM Aaron Marcuse-Kubitza

added backups/vegbien.r12025.backup.md5

12174 02/13/2014 12:53 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

11970 01/20/2014 11:33 AM Aaron Marcuse-Kubitza

moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).

11899 12/11/2013 07:56 PM Aaron Marcuse-Kubitza

updated backups/TNRS.backup.md5

11898 12/11/2013 07:56 PM Aaron Marcuse-Kubitza

added backups/vegbien.r11786.backup.md5

11824 12/04/2013 09:44 PM Aaron Marcuse-Kubitza

backups/TNRS.backup: saved copy backups/TNRS.2013-11-18.backup

11697 11/18/2013 11:38 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

11620 11/09/2013 06:04 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

11577 11/05/2013 11:08 PM Aaron Marcuse-Kubitza

added backups/vegbien.r11549.backup.md5

11575 11/05/2013 11:06 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

11355 10/18/2013 06:27 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

10873 09/05/2013 01:00 AM Aaron Marcuse-Kubitza

added backups/vegbien.r10848.backup.md5

10872 09/05/2013 12:59 AM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

10867 09/04/2013 11:32 PM Aaron Marcuse-Kubitza

bugfix: backups/fix_perms: just make the backups themselves private, since the other files are in svn, and their permissions should match their accessibility through Redmine

10752 08/27/2013 01:20 PM Aaron Marcuse-Kubitza

updated backups/TNRS.backup.md5

10750 08/27/2013 01:03 PM Aaron Marcuse-Kubitza

added backups/vegbien.r10548.backup.md5

10749 08/27/2013 01:03 PM Aaron Marcuse-Kubitza

bugfix: backups/: svn:ignore: removed *.md5, which should be under version control

10745 08/27/2013 11:42 AM Aaron Marcuse-Kubitza

updated backups/TNRS.backup

10609 08/07/2013 07:44 AM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated for last backup

10267 07/13/2013 10:35 PM Aaron Marcuse-Kubitza

backups/Makefile: %.backup/restore: documented runtime (11 h; ~5 h to insert data). note that this is still much shorter than re-running column-based import.

10254 07/11/2013 05:54 AM Aaron Marcuse-Kubitza

backups/Makefile: $(restore): added --verbose to display pg_restore's incremental progress

10031 06/25/2013 02:21 PM Aaron Marcuse-Kubitza

bugfix: backups/.rsync_filter.download: need to prevent existing backups from being deleted on the local side, too, by changing hide patterns to exclude

10027 06/25/2013 01:21 PM Aaron Marcuse-Kubitza

backups/Makefile: upload: use bin/sync_upload

10016 06/25/2013 08:50 AM Aaron Marcuse-Kubitza

added backups/vegbien.r9897.backup.md5

10014 06/23/2013 08:56 PM Aaron Marcuse-Kubitza

backups/.rsync_ignore: replaced with .rsync_filter.upload to allow uploading new backups but not deleting existing backups if they don't exist on the local (rsync-invoking) side; and .rsync_filter.download to avoid downloading backups to the local side. this allows storing older backups just on jupiter, where there is much more disk space. note that this change must be made on the remote side (jupiter) for it to be effective, because these are remote-side rules and are only processed by the remote-side rsync instance.

10012 06/23/2013 07:48 PM Aaron Marcuse-Kubitza

updated backups/TNRS.backup.md5

10011 06/23/2013 07:48 PM Aaron Marcuse-Kubitza

added backups/TNRS.2013-6-17.backup.md5, TNRS.2013-6-22.backup.md5

10003 06/23/2013 03:40 PM Aaron Marcuse-Kubitza

added backups/.rsync_ignore with filters from /README.TXT > Maintenance > to synchronize vegbiendev, jupiter, and your local machine. these filters will now be used with bin/sync_upload in addition to the periodic backup commands.

9926 06/19/2013 10:15 AM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

9897 06/13/2013 08:13 AM Aaron Marcuse-Kubitza

added backups/TNRS.backup.md5, vegbien.r9459.backup.md5

9496 05/21/2013 08:26 PM Aaron Marcuse-Kubitza

added backups/*.md5

9495 05/21/2013 08:22 PM Aaron Marcuse-Kubitza

added backups/TNRS.2013-5-21.backup.md5

7564 02/16/2013 05:25 AM Aaron Marcuse-Kubitza

lib/common.Makefile: remote server: Use jupiter instead of vegbiendev, to ensure that all files get uploaded there rather than only to vegbiendev. This involves adding an extra database import step to download the uploaded files from jupiter onto vegbiendev.

7559 02/15/2013 11:58 PM Aaron Marcuse-Kubitza

Moved Checksums from backups/Makefile to lib/common.Makefile so all dirs (including inputs/) can use md5sum testing

7557 02/15/2013 11:04 PM Aaron Marcuse-Kubitza

lib/common.Makefile: Renamed $(src_server) to $(remote_host) and $(src_user) to $(remote_user) for clarity

7326 01/22/2013 02:49 PM Aaron Marcuse-Kubitza

backups/Makefile: %.md5/test: Added comment to run with `make -s` to avoid echoing make commands

7037 01/03/2013 02:21 AM Aaron Marcuse-Kubitza

backups/Makefile: Synchronization: %/download: Download the .md5 file first, so that the user is prompted right away for their password rather than after the main file has finished downloading, by which time the password prompt has timed out

7026 01/02/2013 06:36 PM Aaron Marcuse-Kubitza

backups/Makefile: Removed no longer used $(psqlVerbose)

7025 01/02/2013 06:36 PM Aaron Marcuse-Kubitza

backups/Makefile: Removed %.backup/rm_indexes, which is no longer needed because archived imports are now backed up instead of being stored without indexes in the live DB

7024 01/02/2013 06:31 PM Aaron Marcuse-Kubitza

backups/Makefile: %.backup/remove: Fixed bug where need to use $no_search_path option to psql_script_vegbien

6965 12/20/2012 09:25 AM Aaron Marcuse-Kubitza

backups/Makefile: Factored $(isMac) out into lib/common.Makefile

6940 12/19/2012 11:56 AM Aaron Marcuse-Kubitza

backups/Makefile; $(public*): Don't add a "public." prefix to get the name of the public schema

6939 12/19/2012 11:40 AM Aaron Marcuse-Kubitza

backups/Makefile: Removed no longer used $(rmSchema)

6938 12/19/2012 11:39 AM Aaron Marcuse-Kubitza

backups/Makefile: Use \$(rmSchemaCmd) from lib/common.Makefile instead of \$(rmSchema)

6924 12/19/2012 09:07 AM Aaron Marcuse-Kubitza

backups/Makefile: Removed no longer used $(dateFmt), $(mtime)

6923 12/19/2012 08:59 AM Aaron Marcuse-Kubitza

backups/Makefile: Removed %.backup/rotate, because this incorrectly causes the current time rather than the version to be used in the backup filename. The version should instead be specified in the backup filename when it's created.

6915 12/19/2012 06:15 AM Aaron Marcuse-Kubitza

Makefiles: Factored out common vars/functions into lib/common.Makefile

6890 12/17/2012 04:59 PM Aaron Marcuse-Kubitza

backups/Makefile: Synchronization: $(remote): Fixed bug where need trailing / at end of path

6889 12/17/2012 04:32 PM Aaron Marcuse-Kubitza

backups/Makefile: Synchronization: $(remote): Updated path to backups

6818 12/12/2012 09:40 PM Aaron Marcuse-Kubitza

backups/fix_perms: Removed world read permissions from backups dir. Note that this will require superuser permissions to view archived backups on jupiter, because the bien group is not set up with the same members as on vegbiendev. (On jupiter, it contains only stri,regetz,donoghue,naiamh.)

6725 12/10/2012 07:52 PM Aaron Marcuse-Kubitza

Added backups/fix_perms

6724 12/10/2012 07:45 PM Aaron Marcuse-Kubitza

backups/Makefile: Synchronization: %/download: Also download any .md5 file for the file

6667 12/06/2012 09:56 PM Aaron Marcuse-Kubitza

backups/Makefile: Synchronization: Replaced download target, which downloads all backups, with %/download, which downloads just a specific backup, because you would generally only want to extract a single backup from the archive for reinstallation

6666 12/06/2012 09:47 PM Aaron Marcuse-Kubitza

backups/Makefile: Synchronization: Sync with jupiter instead of vegbiendev. This requires running `make backups/upload` on vegbiendev to archive the files, instead of `make backups/download` to download them to your local machine.

6655 12/06/2012 06:07 PM Aaron Marcuse-Kubitza

backups/Makefile: Checksums: %.md5/test: Only use md5sum's -v option on Mac, because it's not supported on Linux (there, verbose mode is the default)

6648 12/06/2012 04:33 PM Aaron Marcuse-Kubitza

backups/Makefile: SQL: Full DB: vegbien.%.backup: Also generate MD5 sum

6645 12/05/2012 10:56 AM Aaron Marcuse-Kubitza

backups/Makefile: Support removing public schema versions based on the version of a full DB backup

6595 12/04/2012 11:52 PM Aaron Marcuse-Kubitza

backups/Makefile: Synchronization: Also sync *.md5

6573 12/04/2012 07:07 PM Aaron Marcuse-Kubitza

backups/Makefile: Checksums: %.md5/test: Made it an _always target

6572 12/04/2012 07:05 PM Aaron Marcuse-Kubitza

backups/Makefile: Checksums: Added %.md5/test to test generated checksums

6571 12/04/2012 07:01 PM Aaron Marcuse-Kubitza

backups/Makefile: Moved md5-related targets to separate Checksums section

6570 12/04/2012 06:59 PM Aaron Marcuse-Kubitza

backups/Makefile: %.md5: Removed not applicable comment which had been copied from %.sql

6536 12/03/2012 02:47 PM Aaron Marcuse-Kubitza

backups/Makefile: %.md5: Fixed bug where md5sum does not have a -q option like md5

6535 12/03/2012 02:43 PM Aaron Marcuse-Kubitza

backups/Makefile: %.md5: Fixed bug where need to use md5sum instead of md5 on Linux

6526 12/03/2012 11:06 AM Aaron Marcuse-Kubitza

backups/Makefile: Added md5s target to generate .md5 files for all backups

6524 12/03/2012 10:48 AM Aaron Marcuse-Kubitza

backups/Makefile: %.md5: Run with `nice -n +5` to avoid slowing down the UI

6523 12/03/2012 10:46 AM Aaron Marcuse-Kubitza

backups/: svn:ignore: Added *.md5. Removed no longer applicable *.log.

6522 12/03/2012 10:42 AM Aaron Marcuse-Kubitza

backups/Makefile: Changed paths to be relative to the Makefile rather than the current directory, so this Makefile can be used in other directories as well (such as jupiter:/aaronmk/VegBIEN.backups/)

6521 12/03/2012 10:34 AM Aaron Marcuse-Kubitza

backups/Makefile: %.backup: Also create MD5 of backup

6520 12/03/2012 10:31 AM Aaron Marcuse-Kubitza

backups/Makefile: Added %.md5 target to create checksums of each backup

6469 11/27/2012 12:27 PM Aaron Marcuse-Kubitza

backups/Makefile: $(rsyncBackups): Added *.csv

6467 11/26/2012 06:10 PM Aaron Marcuse-Kubitza

backups/: Ignore _* and *.csv

4751 09/17/2012 11:58 AM Aaron Marcuse-Kubitza

backups/Makefile: Backups: Full DB: Specify the date suffix of the backup when it's created rather than adding it afterwards. This allows the user to specify a suffix that matches the corresponding public-schema backup.

4200 08/23/2012 05:00 PM Aaron Marcuse-Kubitza

backups/: svn:ignore: Also ignore .*, which includes temp files generated by rsync

3701 07/31/2012 09:05 PM Aaron Marcuse-Kubitza

backups/Makefile: Added synchronization of backups with vegbiendev. Added downloading backups to After a new import steps.

3699 07/31/2012 08:54 PM Aaron Marcuse-Kubitza

backups/Makefile: Refactored to include lib/common.Makefile

3546 07/23/2012 06:39 PM Aaron Marcuse-Kubitza

backups/Makefile: Full DB: Added vegbien.backup/all to run both test and rotate

3544 07/23/2012 06:19 PM Aaron Marcuse-Kubitza

backups/Makefile: .sql: When testing, turn it off so make won't skip `.sql: %` in favor of it

3543 07/23/2012 06:07 PM Aaron Marcuse-Kubitza

backups/Makefile: Split %.backup and %.sql into separate targets for clarity

3438 07/18/2012 03:31 PM Aaron Marcuse-Kubitza

backups/Makefile: Added %.backup/rotate

3437 07/18/2012 02:58 PM Aaron Marcuse-Kubitza

backups/Makefile: Rearranged sections so that backup targets, which apply to both Archived imports and Full DB, are at the top of a common Backups section

3409 07/17/2012 03:47 PM Aaron Marcuse-Kubitza

backups/Makefile: Added %.backup/test

3407 07/16/2012 08:23 PM Aaron Marcuse-Kubitza

backups/Makefile: Full DB backups: Added vegbien.backup

3404 07/16/2012 07:37 PM Aaron Marcuse-Kubitza

backups/Makefile: Factored backup command into $(backup) for later use by full DB backups. Made Backups, Archived imports sections subsections of Archived imports so Full DB backups can have its own section.

3403 07/16/2012 07:16 PM Aaron Marcuse-Kubitza

backups/Makefile: Fixed bug where $(SHELL) needed to be set to bash so that $'\n' would be interpreted correctly

3402 07/16/2012 07:06 PM Aaron Marcuse-Kubitza

backups/Makefile: Fixed bug where *.sql files needed to be restored using psql because pg_restore only supports "non-plain-text formats"

3399 07/16/2012 06:19 PM Aaron Marcuse-Kubitza

backups/Makefile: Don't log stderr or run the command verbosely and instead just output the command and run time to the terminal. This matches what we do for pg_dump, which works better because it just prints the useful information when it's done running.

3398 07/16/2012 05:28 PM Aaron Marcuse-Kubitza

backups/Makefile: Remove log files after successful restore/extraction because they are only useful for tail -f when the restore operation is running in the background

3395 07/16/2012 04:39 PM Aaron Marcuse-Kubitza

backups/Makefile: Backups: Fixed bug where `%.sql: %` needed to come before %.sql with no prerequisites to be matched first

3394 07/16/2012 04:14 PM Aaron Marcuse-Kubitza

Moved archived imports and make targets to maintain them to new backups dir

3393 07/16/2012 04:08 PM Aaron Marcuse-Kubitza

Moved archived imports and make targets to maintain them to new backups dir