Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  branches 11969 almost 11 years Aaron Marcuse-Kubitza added standard svn dirs branches, tags, trunk
  tags 11969 almost 11 years Aaron Marcuse-Kubitza added standard svn dirs branches, tags, trunk
  trunk 13028 over 10 years Aaron Marcuse-Kubitza /README.TXT: Full database import: disk space: ...
  • svn:ignore: extern

Latest revisions

# Date Author Comment
13028 04/02/2014 09:01 AM Aaron Marcuse-Kubitza

/README.TXT: Full database import: disk space: documented that the entire disk again gets used long after the beginning of the import, when only a few datasources are running (ie. it definitely seems to be a recent bug in Postgres, and not a latent problem)

13027 04/01/2014 05:40 PM Aaron Marcuse-Kubitza

/README.TXT: Maintenance: added task to regularly re-run full-database import so that bugs in it don't pile up. it needs to be kept in working order so that it works when it's needed.

13026 04/01/2014 04:24 PM Aaron Marcuse-Kubitza

/README.TXT: Full database import: added steps to manually reimport the applicable datasources if there are errors due to exceeding available disk space

13025 04/01/2014 04:13 PM Aaron Marcuse-Kubitza

/README.TXT: Full database import: removed extra `ssh -t vegbiendev.nceas.ucsb.edu` before "upload logs", because the previous steps also occur on vegbiendev

13024 04/01/2014 04:04 PM Aaron Marcuse-Kubitza

/README.TXT: Notes on system stability: added recommendation to maintain a snapshot copy of the VM as it was at the last successful import, for fallback use if a system upgrade breaks anything. system upgrades on the snapshot VM should be disabled completely, and because this will also disable security fixes, the snapshot VM should be disconnected from the internet and all networking interfaces. (this is an unfortunate consequence of modern OSes being written in non-memory-safe languages such as C and C++.)

13023 04/01/2014 03:43 PM Aaron Marcuse-Kubitza

/README.TXT: Full database import: disk space: documented that a higher high-water mark actually occurs later in the import, so that the disk usage issue actually remains a problem after the very beginning

13022 04/01/2014 03:37 PM Aaron Marcuse-Kubitza

fix: /README.TXT: Full database import: disk space: increased the minimum free space recommendation to 1 TB, because analysis of the disk usage during the beginning of the import shows that actually close to the entire amount is being used. however, this problem is normally undetectable unless the disk space is specifically checked, because it only manifests itself if the available disk space is exceeded completely.

13021 04/01/2014 02:04 PM Aaron Marcuse-Kubitza

/README.TXT: Full database import: documented that the beginning of the import should be scheduled at a time when the DB will not be needed for other uses, because vegbiendev will be slow for the first few hours of the import due to the import using all the available cores

13020 04/01/2014 01:36 PM Aaron Marcuse-Kubitza

/README.TXT: Full database import: documented that CPU load warning e-mails can safely be ignored. they happen because the parallel imports use all the available cores.

13019 04/01/2014 01:31 PM Aaron Marcuse-Kubitza

fix: lib/common.Makefile: $(nice): use an increment of +10 instead of +5 because +5 still leaves the shell sluggish

View all revisions | View revisions

Also available in: Atom