BIEN links

Quick links

links
VegPath
BIEN
BIEN3
markup
meetings
issues
to do
pub
valid
VegCore
VegBIEN
FAQ
vegbiendev
"
nimoy
starscream
DwC
VegBank
TNRS
SALVIAS
Postgres
MySQL
SQL
mod

BIEN

DataONE compatibility

KNB | Knowledge Network for Biocomplexity: share
page's self-description: Metacat UI
Contribute Data | DataONE
page's self-description: Member Nodes are the key to getting data available through DataONE. As shown in the Depositing Data into DataONE video, data gets deposited into one of the current Member Nodes, through that Member Node's submission process. Once the data is published through that Member Node, it is available in the DataONE catalog and should work with the various items in the Investigator Toolkit.

open-sourcing

licenses

Creative Commons — CC0 1.0 Universal
licensing - Difference between MIT/BSD and CC-BY licenses - Stack Overflow
"CC licenses aren't meant to be used for software" (because it's incompatible with the GPL) "The license you choose for your software should not have confusion and debate: don't use CC for software"
Various Licenses and Comments about Them - GNU Project - Free Software Foundation: Licenses for Works of Practical Use Besides Software and Documentation
"Creative Commons Attribution 2.0 license (a.k.a. CC BY) (#ccby) This is a non-copyleft free license that is good for art and entertainment works, and educational works. Please don't use it for software or documentation, since it is incompatible with the GNU GPL and with the GNU FDL."
(GNU Verbatim Copying License) Information for Maintainers of GNU Software: License Notices for Other Files
page's self-description: Information for Maintainers of GNU Software: License Notices for Other Files

UCSB

Standing Order 100. Officers of the University | Board of Regents
"(mm) The President is authorized to develop and implement policies and procedures on matters pertaining to intellectual property, including patents, copyrights, trademarks, and tangible research products, and to execute documents necessary for the administration of intellectual property, including those which may contain commitments existing longer than seven years. The President annually shall report to the Board on matters pertaining to intellectual property." "(dd) Except as otherwise specifically provided in the Bylaws and Standing Orders, the President is authorized to execute on behalf of the Corporation all contracts and other documents necessary in the exercise of the President's duties"
Current List of Delegations | UCOP
"2201 Authority to execute documents for administration of University patent and other intellectual property matters Provost and Executive Vice President - Academic Affairs 5/9/08"
DA2201, Execution of Documents for Administration of University Patent and Other Intellectual Property Matters - da2201.pdf
"PROVOST AND EXECUTIVE VICE PRESIDENT—ACADEMIC AND HEALTH AFFAIRS [...] Effective August 13, 2007, the authority granted to the President as written above is delegated to you, subject to conditions specified elsewhere in the Bylaws and Standing Orders of The Regents and subject also to terms of applicable Regental and Presidential policies which are or may be established from time to time"
Academic Affairs | UCOP
"Aimée Dorr Provost and Executive Vice President"
** Frequently Asked Questions | UCSB Office of Technology & Industry Alliances: Can I Open Source Software?
"If your UCSB research group is interested in releasing code under an open source license, please complete and forward the “Request to Open Source Software” form to the TIA Office"
*** UCSB: Request to Open Source Software
Open Source Software | UCSB Office of Technology & Industry Alliances
"because version three of the GPL (GPLv3) includes terms and conditions that are not acceptable to the University of California, UCSB will not authorize the release of any software that incorporates code acquired under the GPLv3"

funding sources

iPlant/NCEAS Collaboration to Build the BIEN and Environment & Organisms Working Groups' Informatics Frameworks | Marine Science Institute, UCSB
" Award Period: Tuesday, March 1, 2011 to Sunday, June 30, 2013 Award Amount: $281 483 Agency Name: University of Arizona Award Number: Y562553 PI First Name: Mark PI last name: Schildhauer "
NatureLab: NCEAS core support and capacity building | Marine Science Institute, UCSB
" Award Period: Thursday, December 20, 2012 to Monday, August 31, 2015 Award Amount: $2 440 941 Agency Name: Gordon and Betty Moore Foundation Award Number: 3530 PI First Name: Frank PI last name: Davis "
^ Grants List | Gordon and Betty Moore Foundation: Creating a global center of excellence focused on knowledge generation to sustain nature and human well-being.
"Grant ID: GBMF3530"
NCEAS: National Center for Ecological Analysis and Synthesis | Marine Science Institute, UCSB
" Award Period: Sunday, October 1, 2006 to Monday, September 30, 2013 Award Amount: $18 402 599 Agency Name: National Science Foundation Award Number: DBI-0553768 PI First Name: Frank PI last name: Davis Co-PI: Stephanie Hampton "

U.S. Copyright Office - Definitions (FAQ): Who is an author?
page's self-description: U.S. Copyright Office is an office of public record for copyright registration and deposit of copyright material. "In cases of works made for hire, the employer or commissioning party is considered to be the author"

UArizona

About Intellectual Property at The University of Arizona | Office of Technology Transfer
page's self-description: If you are a University Of Arizona Faculty member, please read the Office of Technology Transfer Practise Statement Procedures For Developing And Securing University Intellectual Property "Generally, federal agency funding contracts are governed by the terms of the Bayh-Dole Act which allows the University to own and exploit inventions created with federal funding"
Arizona Board of Regents: Intellectual Property Policy: Microsoft Word - 6-908-Intellectual Property Policy - abor_6-908-intellectual_property_policy.pdf
"The Board claims ownership of intellectual property in each of the following categories: (1) Any intellectual property created by a university or Board employee in the course and scope of employment, and (2) Any intellectual property created with the significant use of Board or university resources, unless otherwise provided in an authorized agreement for the use of those resources" (p. 1) "“Course and Scope of Employment" s hall include any activity that is listed or described in the employee' s job description or is within the employee's field of employment, in cluding research, instruction, or any activities assigned to the empl oyee that involve the creation of intellectual property. Except as se t forth in Article I, Section B, copyrightable works created by an employee in the course and scope of employment are considered to be works made for hire under U.S. Copyright Law, with owner ship vested in the Board." (p. 14) "The Board does not construe the use of office space, library resources, personal workstations, or personal computers as constituting significant use of university resources." (p. 16)

iPlant

TNRS

UA-OTT Notifications - Library - iPlant Collaborative Wiki
"2011 06 15 Muir TNRS"
TNRS/LICENSE at master · iPlantCollaborativeOpenSource/TNRS · GitHub
page's self-description: TNRS - The Taxonomic Name Resolution Service is a free utility for correcting and standardizing plant names.

Applying iPlant License to Code - Library - iPlant Collaborative Wiki
Checklist for open source code release - Library - iPlant Collaborative Wiki
Current OTT contact for open source and iPlant - Library - iPlant Collaborative Wiki
Open Source | iPlant Collaborative
"From the beginning, iPlant has had a commitment to release all code we produce with an open source software license. This is enshrined in our agreement with the National Science Foundation"
Staff & Collaborators | iPlant Collaborative Web Portal
"Brad Boyle Research Associate University of Arizona"

SALVIAS

The SALVIAS Project Mission Statement
"SALVIAS is based at the University of Arizona in cooperation with the Center for Applied Biodiversity Science at Conservation International"

VegBank

GNU General Public License v2.0 - GNU Project - Free Software Foundation
"You must cause any work that you distribute or publish, that in whole or in part *contains or* is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License." [+emph] (the viral clause, which also applies to aggregated works) "If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works"
(for comparison to GPLv2) The GNU General Public License v3.0 - GNU Project - Free Software Foundation
"You may convey a work based on the Program, *or the modifications to produce it from the Program,* in the form of source code under the terms of section 4" i.e. a diff of your changes would apparently be GPL-ed, even though the line #s in the diff ought to constitute fair use

TWiki

default site-wide preferences - TWikiPreferences < TWiki < TWiki
Local customisations of site-wide preferences - TWikiPreferences < Main < TWiki
BlackListPlugin < TWiki < TWiki
WebNotify < Main < TWiki

use cases

research using VegBank

2006: Spatial variation of climatic and non-climatic controls on species distribution: the range limit of Tsuga heterophylla
page's self-description: jbi_1509 1384..1396 - gavin_jbi06.pdf "In the USA, known occurrences in vegetation plots were obtained from the VegBank database (VEGBANK Contributors, 2004)"
2007: Merging disparate datasets: - stohlgren-proposal_508.pdf
"Modified: 2/16/07" "The datasets we will be focusing on integrating with GODM to address hypothesis 1 include VegBank" "VegBank ( http://vegbank.org ) is the vegetation plot database of the Ecological Society of America's (ESA's) Panel on Vegetation Classi fication. Most plots archived in VegBank contain complete floristi c records, precise geocoordinates, and some measure of taxon importance. At present ove r 20,000 plots spanning 27 states and containing well in excess of 6,000 species ar e archived in VegBank, and in excessive of 10,000 more plots are scheduled to be added in the near future."
2009: Temperature dependence, spatial scale, and tree species diversity in eastern Asia and North America
"using data o f1,370 forest plots in the 2 continents (398 plots for eastern Asia, see ref. 23; and 972 plots for North America, see http://www.vegbank.org/)." (p. 4)
2012: Hotter? Colder? Wetter? Drier? Elevational range shifts in response to 50 years of climatic change across western North America
page's self-description: Working Memory in the Two Cerebral Hemispheres M.G. Funnell, M.K. Colvin, & M.S. Gazzaniga Center for Cognitive Neuroscience, Dartmouth College - Harsch.pdf "Modified: 10/18/12" "Last-Modified: Tue, 23 Oct 2012 22:57:51 GMT" "Species occurrence data were obtained from four online databases: VegBank http://vegbank.org/vegbank/index.jsp"

 ‐ GBIF use cases | GBIF.ORG

**** iPlant/NCEAS Collaboration to Build the BIEN and Environment & Organisms Working Groups' Informatics Frameworks | Marine Science Institute, UCSB
"Award Period: Tuesday, March 1, 2011 to Sunday, June 30, 2013 Award Amount: $281 483" "Key deliverables will include: developing an integrated information resource by merging several well-established plant occurrence information resources, including specimen data from various natural history and botanical museums, as well as plots data;"
/load-geoscrub-input.sh - BIEN Geo - NCEAS Projects
page's self-description: Redmine
/README.txt - BIEN Geo - NCEAS Projects
page's self-description: Redmine
11-02-24 Kickoff Meeting - Scientific Opportunities Team - iPlant Collaborative Wiki
Activity Participants: botanical information network Working Group 24th—28th October 2011 | NCEAS
BI_Project_Charter - iPToL - iPlant Collaborative Wiki
BIEN - / - Repository - NCEAS Projects
page's self-description: Redmine
BIEN - Wiki - NCEAS Projects
page's self-description: Redmine
BIEN Database 3.0 - SciPlant - iPlant Collaborative Wiki
BIEN links
BIEN Project | iPlant Collaborative Web Portal
bien.pdf
Botanical Information and Ecology Network
Botanical Information and Ecology Network
page's self-description: NCEAS Project 12290: Developing an integrated botanical information network to investigate the ecological impacts of global climate change on plant biodiversity
Home Page | iPlant Collaborative
Munin :: nceas.ucsb.edu :: vegbiendev.nceas.ucsb.edu
NCEAS Project 12290: botanical information network | NCEAS
nimoy
nimoy phpPgAdmin
Overview of NCEAS Computing and GIS Support
publishable datasources
Sources « Botanical Information and Ecology Network
Strategic Initiatives | iPlant Collaborative
"Consumer Projects. iPlant provides cyberinfrastructure services for a large number of projects, including [...], BIEN"
Taxonomic Name Resolution Service

vegetation data

Canadensys

Canadensys IPT
Norms for data use and publication – Canadensys

Darwin Core

Darwin Core
Taxon - darwincore - Darwin Core Project site for discussion and development - Google Project Hosting
DwC hist

data management

Data management plan - Wikipedia, the free encyclopedia
"By deciding on an archive ahead of time, the data collector can format data during collection to make its future submission to a database easier" "It also allows the data collector to direct requests for data to the database, rather than address requests individually" "Funding agencies are beginning to require data management plans as part of the proposal and evaluation process" "management and preservation costs may be considerable, depending on the nature of the project"
US NSF - About: Data Management & Sharing Frequently Asked Questions (FAQs)
"Usually, preserving the data in its most raw form is desirable, although data derivatives and products can also be preserved" "The expectation is that all data will be made available after a reasonable length of time" "One standard of timeliness is to make the data or samples accessible immediately after publication" "It is NSF’s strong expectation that investigators will share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants"

DiGIR

DiGIR - Table of Contents
DiGIR web-browser client helper scripts.
GBIF Data Access and Database Interoperability
http://splink.cria.org.br/digir/
http://www.digir.net/prot/release/searchRequestDiGIR1.5.xml
search.vm - gbif-occurrencestore - Research into a scalable and feature-rich occurrence store and index - Google Project Hosting
DiGIR Messaging
Biodiversity Collections Index
simple-digir/SimpleDigir.php at master · diogok/simple-digir · GitHub
page's self-description: simple-digir - Tool (class and UI) to simplify search on DIGIR server
BCI: Search:
BCI: Search: digir

GBIF

licenses

licenses.csv
guidelines
page's self-description: gbif-data-licenses - Standardizing licenses of GBIF mediated data

Disclaimers and agreements ‐ Terms relating to information and data accessed, used and shared through GBIF: Data sharing
"Responsibility regarding the restriction of access to sensitive data resides with the Data Publisher." "The Data Publisher warrants that it has made the necessary agreements with the original owners of the data that it can make the data available through the GBIF network."
Disclaimers and agreements ‐ Terms relating to information and data accessed, used and shared through GBIF: Data use agreement
GBIF Newsroom ‐ News and events from around the GBIF community
GBIF Newsroom ‐ News and events from around the GBIF community: Uses of data
Index of ftp://ftp.gbif.org/
Memorandum of Understanding for the Global Biodiversity Information Facility
"This document is not legally binding and has no effect as a legal or political precedent."
Plantae (Plants) - GBIF Portal
page's self-description: Observational and specimen data for Plantae "79,725,738 records with coordinates"
Resources ‐ Tools and information to support the GBIF community: GBIF Memorandum of Understanding 2012
"non-binding Memorandum of Understanding (MoU)"

growth forms

Growth Habits Codes and Definitions | USDA PLANTS
page's self-description: Growth Habits Codes and Definitions
Appendix E (Normative): Growth Form Names, Codes, and Definitions < National Vegetation Classification Standard (Version 2)

NCBI

Home - Taxonomy - NCBI
Taxonomy - Site Guide - NCBI

NVS

What is the NVS Databank?
The National Vegetation Survey Databank (NVS)

SALVIAS

What is SALVIAS?
SALVIAS TaxonScrubber

sPlot

sPlot | German Centre for Integrative Biodiversity Research
sPlot database | German Centre for Integrative Biodiversity Research
sPlot-ERD_2013-09-09.jpg (JPEG Image, 1298 × 843 pixels)

TAPIR

TAPIR metadata

TNRS

Taxonomic Name Resolution Service - API
Taxonomic Name Resolution Service - Home
Taxonomic Name Resolution Service - Instructions
Taxonomic Name Resolution Service - Instructions
Sources

Tropicos

Tropicos - Home
Web Services - Tropicos

VegBank

reinstalling VegBank on VegBankVM
VegBank
VegBank Data Dictionary: Menu
vegbank repository
VegBank XML
vegbank_erd.pdf
VegBranch tools : Normalizer

Atlas of Living Australia
page's self-description: The Atlas of Living Australia provides tools to enable users of biodiversity information to find, access, combine and visualise data on Australian plants and animals
Bien-db Info Page
Biodiversity Collections Index
Carnegie Spectranomics
Dashboard - iPlant Collaborative Wiki
Enquist Lab in action on PBS NOVA < PBS Nova - Fractals - Hunting the Hidden Dimension. - YouTube
page's self-description: Mysteriously beautiful fractals are shaking up the world of mathematics and deepening our understanding of nature. A fractal is "a rough or fragmented geomet...
FIA DataMart 5.1 :: Home
GIVD: List of Databases
Home - iPToL - iPlant Collaborative Wiki
Index Herbariorum :: NYBG.org
iTOL: Interactive Tree Of Life
Jellyfish Blooms of the Global Ocean: Magnitude, Causes and Consequences for Marine Food Webs, Carbon Cycling and Socioeconomics | NCEAS
Mission — TraitNet
National Vegetation Classification Standard (Version 2)
- contains field names for plots data
PaleoLab | Pollen database
Phylomatic (v3)
RAINFOR
smartphone apps for field biologists
Smithsonian Tropical Research Institute-Center for Tropical Forest Science
speciesLink: networkManager
Survey | Qualtrics Survey Software | Current Progress 21%
page's self-description: Survey Software, Enterprise Survey software for enterprise feedback management and CRM solutions. Enables high-quality data collection, panel management and results analysis. Perfect for market research or CRM solution (Customer Relationship Management) integration. Free trial and consultation. "Which of the following do you use to access or submit data? (Choose all that apply.) DAAC (A Distributed Active-Archive Center) DataONE (Data Observation Network for Earth) Dryad ESA (Ecological Society of America) GBIF (A Global Biodiversity Information Facility) ILTER (International Long-term Ecological Research Network) KNB (Knowledge Network of Biocomplexity) LTER (Long-term Ecological Research Network) NEON (National Ecological Observatory Network) ORNL DAAC (Oak Ridge National Laboratory Distributed Active Archive Center) PISCO (Partnership for Interdisciplinary Study of Coastal Oceans) SAEON (South African Environmental Observation Network) SanParks (South Africa National Parks) TERN (Taiwan Ecological Research Network) UC3 (University of California Curation Center) USGS CSC (USGS Core Sciences Clearinghouse) "
TDWG: Taxonomic Concept Transfer Schema
Turboveg for Windows
Veg-X

NCEAS

Welcome to NCEAS | NCEAS
main_page [NCEAS Computing Services Knowledge Base]
NCEAS Identity Manager (to change password)
user_accounts [NCEAS Computing Services Knowledge Base]
Staff Computer Setup - NCEAS Knowledge Base
NCEAS webmail
Phone List | NCEAS
support
Connect - NCEAS Web IRC
Suggested Acknowledgment Formats | NCEAS
"New Moore Foundation Funding Supports UCSB Ecology Synthesis Center Embarking on a New Era " - UC Santa Barbara News Release
page's self-description: New Moore Foundation Funding Supports UCSB Ecology Synthesis Center Embarking on a New Era
Identity :: People Finder
Factor Leave Accrual | Business & Financial Services - UCSB
Off-Campus Access: Login

projects

SNAP

Announcing SNAP 2014 Call for Proposals and Six New SNAP Working Groups | NCEAS
SNAP: Integrating natural defenses into coastal disaster risk reduction @ NCEAS Project 12678 | NCEAS

Ocean Health Index 2013: Results Summary
page's self-description: The Ocean Health Index calculates an annual global score that assesses the condition of marine ecosystems and provides a means to advance ocean policy and compare future progress worldwide.

Roundtable

*** virtual collaboration

* Facing-Fears-Virtual-Meetings.pdf
"Reduce the number of participants" "Reduce the scope for the actual meeting" "Make sure you have a note - taker and a time - keeper (but shouldn’t be you)" "Ask that all stay OFF mute. Everyone will be more likely to hear key clicks" but "people can jump into the conversation without delay." however, this also increases the risk that there will be echo problems, or typing noises, etc. "conditions where remote meetings may not be the best s olution [...] people come from different time zones and must participate at same time with all others for extended periods of time" but for short meetings, this can be mitigated by scheduling at a time that is daytime for everyone "Have small meeting pods - people in same room joining in together rather than from their own desks." I actually think this is a bad idea, because it causes the people within the pods to communicate with each other rather than with the entire group
Forbes-Insight-F2F.pdf
- note that the chart on face-to-face vs. technology-enabled meetings (p3) does not includes *disadvantages* of either
Using virtual collaboration to replace or supplement carbon-intensive research travel: barriers, best practices, and opportunities for innovation | Roundtable

thinking preferences

Thinking Preferences

Dr. David Goodstein -- Recent Articles

Index of /~aaronmk

iPlant

Atmosphere

Atmosphere
Atmosphere Manual Table of Contents - Atmosphere Manual - iPlant Collaborative Wiki
RealVNC - Download VNC® Viewer
page's self-description: Download VNC® Viewer for Windows, Mac or UNIX.

Data Store

Data Store Quick Start - 1 Getting Started with iPlant - iPlant Collaborative Wiki
Storing and Accessing Your Data in the Data Store - 1 Getting Started with iPlant - iPlant Collaborative Wiki

TNRS

(deprecated) TNRS - The iPlant Collaborative JIRA
iPlantCollaborativeOpenSource/TNRS · GitHub
page's self-description: TNRS - The Taxonomic Name Resolution Service is a free utility for correcting and standardizing plant names.
Issues · iPlantCollaborativeOpenSource/TNRS · GitHub
page's self-description: TNRS - The Taxonomic Name Resolution Service is a free utility for correcting and standardizing plant names.
Taxonomic Name Resolution Service - Instructions

WebEx

Cisco_WebEx_Removal_Tool.dmg
Join a Test Meeting: WebEx
page's self-description: Test your browser by joining a test meeting.
Webex crashing | Apple Support Communities
"You could try deleting ~/Library/Application Support/WebEx folder which will clean out the (myriad) old versions that WebEx keeps around, and then reinstalling."

Big Data: The View From The Classroom « A Smarter Planet Blog A Smarter Planet Blog
page's self-description: by Nirav Merchant, iPlant Collaborative, Director, BioComputing at the Arizona Research Laboratories (ARL), The University of Arizona Today researchers in
cheatsheet.pdf @ Adobe Connect File Share
Global call-in numbers
page's self-description: 85
iPlant Collaborative Contributor License Agreement
iPlant User Management
Notation Guide - iPlant Collaborative Wiki
Seed Projects | iPlant Collaborative Web Portal
System Dashboard - The iPlant Collaborative JIRA

resources

copyright

authority to bind

Authority to Bind – “You did what?” | Pazar Law
"For officers (and perhaps directors) it should be expressed in writing as part of the corporate bylaws"
The High-touch Legal Services® Blog…for Startups!Who Can Sign Contracts for a Corporation? » The High-touch Legal Services® Blog...for Startups!

Derivative work - Wikipedia, the free encyclopedia
"A “derivative work” is a work based upon one or more preexisting works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which a work may be recast, transformed, or adapted. A work consisting of editorial revisions, annotations, elaborations, or other modifications which, as a whole, represent an original work of authorship, is a “derivative work”."
Let’s Make Copyright Opt-OUT
page's self-description: If you oppose IP and don’t want a patent–just don’t apply. Unfortunately, someone else might independently invent the same thing, patent it, and shut you down, since your having invented it first, or independently invented it, is no defense. So, […] "But if you don’t want copyright, you’re out of luck. Under the current law, copyright is received automatically. Contrary to popular wisdom, you don’t “copyright” something. It’s not a verb. You don’t need to put a copyright notice on your work. You don’t need to register it. It’s automatic. And there’s nothing you can do about it. (See “Copyright Is Very Sticky!“) This makes it often impossible to know who the owner of a given work is or who to approach to ask permission; it contributes to the “orphan works” problem as well–where older works still apparently under copyright fall into obscurity because the copyright owner is unknown or cannot be found for permission to republish."
Scènes à faire - Wikipedia, the free encyclopedia
"variable declarations are generally not considered protected elements of a program"
U.S. Copyright Office - Fair Use
page's self-description: Factsheet on fair use of copyrighted works. "Copyright [...] does not extend to any ideas, systems, or factual information conveyed in a work"

databases

relational

modeling

*** The Enhanced Entity-Relationship (EER) Model @ Ch08.pdf
"If all the subclasses of a specialization/ generalization have few specific attributes and no specific relationships  Can be merged into the superclass  Replace with one or more type attributes that specify the subclass or subclasses that each entity belongs to " "Identification [...]  Needed at two levels  To distinguish among database objects and classes  To identify database objects and to relate them to their real-world counterparts "
Anomalies: Insert, Update and Delete - Access wiki - Access Help and How-to - Microsoft Office by UtterAccess.com
page's self-description: Microsoft Access help database discussion forum and wiki
Crow's Feet Are Best
page's self-description: The Data Administration Newsletter delivers business intelligence, data warehousing and analytics resources provided by Claudia Imhoff, Bill Inmon and other experts. Additional topics include data quality, data integration, CRM, data marts, data mining, business performance management, BPM, data modeling, enterprise application management, ERP, RFID, storage, supply chain and others. "Peter Chen invented data modeling with his innovative way of showing data structures in terms of entities and relationships"
Enhanced entity–relationship model - Wikipedia, the free encyclopedia
"MySQL Workbench offers creating, editing and exporting EER Models"
GUI Database Design Tools - PostgreSQL wiki

MDB

Linux MDB driver - alpha version
page's self-description: OpenOffice.org: The Free, Open Source Office Suite
The Electrofriend: How to compile mdbtools on Mac OS X 10.4 and 10.5
MDB Tools - Unlocking Your Data
How to Open and Edit MS Access Databases from Mac OS X
page's self-description: Information on how to read and query MS Access databases from Mac OS X.

SQL

inheritance

sql - How can you represent inheritance in a database? - Stack Overflow
xkcd • View topic - Inheritance in SQL
"One table per class with type field: Same as above, with a column in the parent class table indicating the sub-class." "Or... use Postgres http://www.postgresql.org/docs/8.1/stat ... herit.html"

optimization

Afraid of SSD?
page's self-description: Should a SQL performance consultant be afraid of SSD? Some surprising facts why I'm not afraid of SSD. "If you want to get the factor 50 speed-up of SSDs, you’d better avoid reading large chunks of sequential data, because that’s where you can only gain a factor five improvement. To put that into database context: if you are doing many full table scans, you won’t get the full potential of SSD. On the other hand, index lookups have a tendency to cause many random IO operations and can thus benefit from the fast response time of SSDs"
Hash Aggregate - Craig Freedman's SQL Server Blog - Site Home - MSDN Blogs
Indexed Group By | SQL Performance Explained
"the sort/group algorithm can use an index to avoid the sort operation, thus enabling a pipelined group by."
PostgreSQL EXPLAIN | Understand each Operation
page's self-description: Explaining the difference between "Bitmap Index Scan" and "Index Scan" in the PostgreSQL database. "A plain Index Scan fetches one tuple-pointer at a time from the index, and immediately visits that tuple in the table. A bitmap scan fetches all the tuple-pointers from the index in one go, sorts them using an in-memory "bitmap" data structure, and then visits the table tuples in physical tuple-location order."

PostgreSQL

backups

online backups: pg_start_backup() method: (more reliable and Postgres-recommended, but requires putting the DB server into a special mode during backup) # configure: ## in postgresql.conf, set wal_level = hot_standby ## $ sudo service postgresql restart # run: ## $ sudo -u postgres psql <<<"SELECT pg_start_backup('backup', true);" ## copy files using rsync, etc. ## $ sudo -u postgres psql <<<"SELECT pg_stop_backup();" wal_keep_segments method: (less reliable, but the backup is completely independent of the DB server) # configure: ## in postgresql.conf: ### set wal_level = hot_standby ### set wal_keep_segments to long enough that the WAL chain spans the duration of the backup #### to determine this, put the DB into peak usage and measure how many WAL segments are created in the time it takes an rsync with minimal changes to complete (about a few seconds) ## $ sudo service postgresql restart # run: ## copy files using rsync, etc. ## copy again to get changes that occurred during the copy operation ## copy again to make sure there are no more changes offline backups: # copy files using rsync, etc. # copy again to get changes that occurred during the copy operation # $ sudo service postgresql stop # copy again to make the copy a consistent snapshot # $ sudo service postgresql start

pg_clog, pg_xlog

** pg_log, pg_xlog, and pg_clog
page's self-description: ... so I deleted several GB of log files from the pg_xlog directory to free up some disk space. Now my database won't start. Oh-oh. Um ... when ... "Should you ever delete files from pg_clog, you might as well delete the entire database directory. There is no recovery from a missing clog."
KB Parallels: How to recreate postgres pg_clog/xxxx file
page's self-description: Symptoms Sometimes the transaction log files become corrupted or deleted due to hordware crash, orpower failure, usually the error is like folowing: hsphere=# select * from TABLE; ERROR:&nbsp; could not access status of transaction 1918986094 DETAIL:&nbsp; could not open file &quot;/var/lib/pgsql/data/pg_clog/0726&quot;: No such file or directory ... "The needed file should be reacreated manually with blanked inside, but filesize should be the same"
pg_clog and pg_xlog empty, postgresql refuses to start - PostgreSQL Database
page's self-description: pg_clog and pg_xlog empty, postgresql refuses to start. PostgreSQL Database Forums on Bytes. "I'd put back the clog files if at all possible. If not you can substitute zero-filled 256K files for each clog file the server complains about not finding, but realize that you *will* be losing transactions in whole or in part." "As for pg_xlog, you can cons up dummy xlog contents with pg_resetxlog, but again you'd be better off with the real thing. The dummy files will not be capable of replaying whatever happened since your last checkpoint."
Re: what are the pg_clog and pg_xlog directories ?

pg_dump

pg_restore

*** Re: Why is pg_restore trying to create tables in pg_catalog? — Postgresql General Discussion
page's self-description: Postgresql Admin: Re: Why is pg_restore trying to create tables in pg_catalog? - you need to *create* the schema before you restore into it, or else you will get a cryptic error message that it's using pg_catalog instead: "what pg_dump emits is SET search_path = myschema, pg_catalog; CREATE TABLE ... and at the moment, if myschema doesn't exist, the creation target devolves to pg_catalog"

PostgreSQL - general - pg_dump not including custom CAST?
page's self-description: pg_dump not including custom CAST?. All, I did this in my database: CREATE CAST (VARCHAR AS BYTEA) WITHOUT FUNCTION; But when I use pg_dump to dump the database and use pg_restore to bring... "It's just casts. They're a bit of a problem since they have neither owners nor schemas, so there's not anything very concrete to base a dump-or-don't-dump decision on. The rule pg_dump uses is to dump it if at least one of the three underlying objects (source type, dest type, or function) is dumpable."

** PostgreSQL: Documentation: 9.3: Continuous Archiving and Point-in-Time Recovery (PITR)
"To prepare for low level standalone hot backups, set wal_level to archive (or hot_standby), archive_mode to on, and set up an archive_command that performs archiving only when a switch file exists." "When wal_level is minimal some SQL commands are optimized to avoid WAL logging" this is really the only reason that wal_level must be archive, since the server normally keeps around lots of WAL segments anyway. "The file identified by pg_stop_backup's result is the last segment that is required to form a complete set of backup files" * actually, this is only needed if parts of base/ were copied *after* the backup finishes copying pg_xlog/. most backup mechanisms copy in alphabetical order, so that pg_xlog/ will be the last subdir copied, and the last segment in it will be copied after base/ is already done.
** PostgreSQL: Documentation: 9.3: Write Ahead Log
"But minimal WAL does not contain enough information to reconstruct the data from a base backup and the WAL logs, so either archive or hot_standby level must be used to enable WAL archiving (archive_mode) and streaming replication." "In many situations, turning off synchronous_commit for noncritical transactions can provide much of the potential performance benefit of turning off fsync, without the attendant risks of data corruption." "it is only advisable to turn off fsync if you can easily recreate your entire database from external data" (which is the case for us, actually, but we currently turn off synchronous_commit instead) ** note that turning fsync on protects against power failure, but not hard disk failure or human error. for that, you need continuous archiving or point-in-time recovery, respectively. "In hot_standby level, the same information is logged as with archive, plus information needed to reconstruct the status of running transactions from the WAL. [...] It is thought that there is little measurable difference in performance between using hot_standby and archive levels" ie, if you are going to use archive anyway, might as well go straight to hot_standby
*** PostgreSQL: Documentation: 9.3: File System Level Backup
"The database server must be shut down in order to get a usable backup. Half-way measures such as disallowing all connections will not work (in part [...] because of internal buffering within the server)." ** in practice, this is not necessarily the case. it is entirely possible to get a usable filesystem-level backup, using the approaches described at http://vegpath.org/links/#backups . this is important because filesystem-level backups have several significant advantages over traditional database-level backups with pg_dump: 1) only changed files will be backed up each time, rather than the entire database 2) no CPU time needs to be spent on intensive compression of the data 3) most importantly, the backup is instantaneously restorable simply by swapping the backup and primary database directories. by comparison, a pg_dump backup can take many hours to restore. in general, to get a consistent filesystem-level backup, you need a WAL chain that's at least long enough (time-wise) for the WAL blocks created when it starts copying base/ to still be around when it finishes copying pg_xlog/ (it must also copy in alphabetical order, and not in parallel)
*** PostgreSQL: Documentation: 9.3: Replication: wal_keep_segments
"wal_keep_segments (integer) Specifies the minimum number of past log file segments kept in the pg_xlog directory"
Postgres 9.3 feature highlight: parallel pg_dump
page's self-description: Postgres 9.3 feature highlight: parallel pg_dump
PostgreSQL: Documentation: 9.3: CHECKPOINT
"A checkpoint is a point in the transaction log sequence at which all data files have been updated to reflect the information in the log"
PostgreSQL: Documentation: 9.3: pg_basebackup

disk space

quotas

PostgreSQL - admin - Best way to limit database sizes
page's self-description: Best way to limit database sizes. Hi,         I have a single Postgres server which will be hosting multiple databases belonging to different users. What would be the recommendation to limit the size... "put each tablespace on a LVM partition of a control sized, extensible" "But LVM/tablespace should work" "How about a trigger that for every say 1000 or 10000 update or inserts (or timely if cron job) will use pg_stats or pg_statistics to decide if a role (user) privileges should be modified to read only (or not)"

PostgreSQL: Disk space not released after TRUNCATE - Database Administrators
[#JENA-28] Drop Temporary Tables in Postgres (PATCH) - ASF JIRA
PostgreSQL: Documentation: 8.3: CLUSTER
Re: ERROR: could not write block 196261 of temporary file: No space left
PostgreSQL: PostgreSQL: In place UPDATE
disk suddenly full, postgres using all space - Untangle Forums
page's self-description: A customer box that's never used more than a few gigs of space suddenly ran out of disk space so bad the captive portal page wouldn't load properly.

doc

** PostgreSQL: Documentation: 9.3: Function Volatility Categories
- *** "A common error is to label a function IMMUTABLE when its results depend on a configuration parameter. For example, a function that manipulates timestamps might well have results that depend on the TimeZone setting. For safety, such functions should be labeled STABLE instead." this especially includes functions that depend on the search_path! (ie. that don't have schema qualifiers on all invoked functions.) however, this effect will only be noticeable if the function is called on only *constant* values in a *PL/pgSQL* function, in which case the wrong search_path (the one in effect at *create* time) will be used. ** "There is relatively little difference between STABLE and IMMUTABLE categories when considering simple interactive queries that are planned and immediately executed: it doesn't matter a lot whether a function is executed once during planning or once during query execution startup. But there is a big difference if the plan is saved and reused later. Labeling a function IMMUTABLE when it really isn't might allow it to be prematurely folded to a constant during planning, resulting in a stale value being re-used during subsequent uses of the plan. This is a hazard when using prepared statements or when using function languages that cache plans (such as PL/pgSQL)." ie. if you are going to use your STABLE function in a PL/pgSQL function, be sure to label it STABLE so that it does not get constant-folded until call time (if labeled IMMUTABLE, it would be folded right away when the function is created)
*** PostgreSQL: Documentation: 9.3: Controlling the Planner with Explicit JOIN Clauses
**** PostgreSQL: Documentation: 9.3: Query Planning
- **** "join_collapse_limit (integer) The planner will rewrite explicit JOIN constructs (except FULL JOINs) into lists of FROM items whenever a list of no more than this many items would result. Smaller values reduce planning time but might yield inferior query plans." you definitely want to turn this *off* in all cases! joins are written in a particular order for a reason (because the programmer knows which indexes are available and therefore which order is correct). the query planner is not smarter than the programmer! incorrect query plans are an ongoing bug in Postgres, because it does not support index hints and by default does not follow the join order. specifically, Postgres often does the following things in query plans which should normally never be done: * performs a sequential scan when an index is available (because it incorrectly thinks there are too many dead rows in the index, or because it incorrectly assumes the result of a nested SELECT (used as an index key) will cover most of the rows in the table) * uses an in-memory hash join when an index is available (because it incorrectly thinks that accessing an index on disk will be too slow) * sorts *both* input tables before performing a merge join (if really no index is available, a hash join is much faster than two sorts and a merge join) * does the entire join sequence in the opposite order (because it incorrectly thinks that the last table in the join will be filtered to fewer rows than the first table, and ignores the programmer's join order) * joins a pair of tables in the wrong order (the smaller table should be iterated over, with an index or hash lookup being used for the larger table, not vice versa) for these reasons, the following must always be off: SET enable_seqscan = off; for queries that use an explicit join order, they will often be faster if you turn off join_collapse_limit to force using your join order. however, if you do this, you may need to turn off other options as well: SET join_collapse_limit = 1; -- turn it off SET enable_sort = off; SET enable_hashjoin = off; -- better to keep this on if possible SET enable_mergejoin = off; fortunately, the combination of all of these is usually sufficient to get Postgres to use the right query plan.
PostgreSQL: Documentation: 8.3: Array Functions and Operators
PostgreSQL: Documentation: 8.3: Control Structures
PostgreSQL: Documentation: 8.3: pg_enum
PostgreSQL: Documentation: 8.3: PL/Python Functions
PostgreSQL: Documentation: 8.3: ROLLBACK TO SAVEPOINT
PostgreSQL: Documentation: 8.3: Row and Array Comparisons
PostgreSQL: Documentation: 8.3: The Statistics Collector
PostgreSQL: Documentation: 8.4: Function Overloading
PostgreSQL: Documentation: 8.4: Query Language (SQL) Functions
PostgreSQL: Documentation: 8.4: WITH Queries (Common Table Expressions)
PostgreSQL: Documentation: 9.0: Calling Functions
PostgreSQL: Documentation: 9.1: ALTER TABLE
"As an exception, if the USING clause does not change the column contents and the old type is either binary coercible to the new type or an unconstrained domain over the new type, a table rewrite is not needed, but any indexes on the affected columns must still be rebuilt" "If the constraint is marked NOT VALID, the potentially-lengthy initial check to verify that all rows in the table satisfy the constraint is skipped. *The constraint will still be enforced against subsequent inserts or updates* (that is, they'll fail unless there is a matching row in the referenced table)." [+emph] it has been experimentally verified that the constraint *is* enforced against new rows, so this unfortunately *cannot* be used to temporarily disable an fkey "VALIDATE CONSTRAINT This form validates a foreign key constraint that was previously created as NOT VALID, by scanning the table to ensure there are no unmatched rows. Nothing happens if the constraint is already marked valid." "Disabling or enabling internally generated constraint triggers requires superuser privileges" if you try to disable a foreign key constraint's trigger, you will get an error, even if it is marked NOT VALID: "ERROR: permission denied: "RI_ConstraintTrigger_..." is a system trigger SQL state: 42501" "To force an immediate rewrite of the table, you can use VACUUM FULL, CLUSTER or one of the forms of ALTER TABLE that forces a rewrite. This results in no semantically-visible change in the table, but gets rid of no-longer-useful data."
PostgreSQL: Documentation: 9.1: ANALYZE
"the autovacuum daemon (see Section 23.1.5) takes care of automatic analyzing of tables when they are first loaded with data, and as they change throughout regular operation"
PostgreSQL: Documentation: 9.1: Basic Statements: 39.5.4. Executing Dynamic Commands
"In situations where the best plan depends strongly on the parameter values, EXECUTE can be significantly faster; while when the plan is not sensitive to parameter values, re-planning will be a waste."
PostgreSQL: Documentation: 9.1: Binary Data Types
PostgreSQL: Documentation: 9.1: Character Set Support
PostgreSQL: Documentation: 9.1: COMMENT
PostgreSQL: Documentation: 9.1: Constraints: 5.3.6. Exclusion Constraints
"EXCLUDE USING gist (c WITH &&)"
PostgreSQL: Documentation: 9.1: COPY
- text format: "Backslash characters (\) can be used in the COPY data to quote data characters that might otherwise be taken as row or column delimiters." "the following characters must be preceded by a backslash if they appear as part of a column value: backslash itself, newline, carriage return, and the current delimiter character." "special backslash sequences are recognized by COPY FROM" (only in text mode) "The following special backslash sequences are recognized by COPY FROM: [...] \digits Backslash followed by one to three octal digits specifies the character with that numeric code " this includes \0 CSV format: "any occurrence within the value of a QUOTE character or the ESCAPE character is preceded by the escape character" with ESCAPE=\ : \->\\, "->\" with ESCAPE=" : "->""
PostgreSQL: Documentation: 9.1: CREATE CONVERSION
PostgreSQL: Documentation: 9.1: CREATE FUNCTION
"SET FROM CURRENT saves the session's current value of the parameter as the value to be applied when the function is entered." This is referring to the session when the function is *defined*, not when it's called.
PostgreSQL: Documentation: 9.1: CREATE INDEX
"Only B-tree currently supports unique indexes" but see exclusion constraints (http://www.postgresql.org/docs/9.1/static/sql-createtable.html#SQL-CREATETABLE-EXCLUDE) for a way to do this for gist indexes
PostgreSQL: Documentation: 9.1: CREATE OPERATOR
"The operator name is a sequence of up to NAMEDATALEN-1 (63 by default) characters from the following list: + - * / < > = ~ ! @ # % ^ & | ` ? "
PostgreSQL: Documentation: 9.1: CREATE OPERATOR CLASS
- an operator class is equivalent to a comparator in other programming languages (with the exception that the same comparator cannot be used for both trees and hashes)
PostgreSQL: Documentation: 9.1: CREATE RULE
"You can create the illusion of an updatable view by defining ON INSERT, ON UPDATE, and ON DELETE rules (or any subset of those that's sufficient for your purposes) to replace update actions on the view with appropriate updates on other tables. If you want to support INSERT RETURNING and so on, then be sure to put a suitable RETURNING clause into each of these rules. Alternatively, an updatable view can be implemented using INSTEAD OF triggers (see CREATE TRIGGER)."
PostgreSQL: Documentation: 9.1: CREATE TABLE: EXCLUDE
"The access method must support amgettuple (see Chapter 52); at present this means GIN cannot be used. [...] So in practice the access method will always be GiST."
PostgreSQL: Documentation: 9.1: CREATE TRIGGER
"a constraint trigger. This is the same as a regular trigger except that the timing of the trigger firing can be adjusted using SET CONSTRAINTS" "A column-specific trigger (one defined using the UPDATE OF column_name syntax) will fire when any of its columns are listed as targets in the UPDATE command's SET list. It is possible for a column's value to change even when the trigger is not fired, because changes made to the row's contents by BEFORE UPDATE triggers are not considered. Conversely, a command such as UPDATE ... SET x = x ... will fire a trigger on column x, even though the column's value did not change." "For UPDATE events, it is possible to specify a list of columns" "It is possible for a column's value to change even when the trigger is not fired, because changes made to the row's contents by BEFORE UPDATE triggers are not considered"
PostgreSQL: Documentation: 9.1: CREATE TYPE
PostgreSQL: Documentation: 9.1: Date/Time Functions and Operators
PostgreSQL: Documentation: 9.1: Default Values
PostgreSQL: Documentation: 9.1: hstore
PostgreSQL: Documentation: 9.1: Index Types
"The query planner will consider using a hash index whenever an indexed column is involved in a comparison using the = operator"
PostgreSQL: Documentation: 9.1: Inheritance
PostgreSQL: Documentation: 9.1: Lexical Structure
"A variant of quoted identifiers allows including escaped Unicode characters identified by their code points. This variant starts with U& (upper or lower case U followed by ampersand) immediately before the opening double quote, without any spaces in between, for example U&"foo"." "the OPERATOR syntax, as for example in: SELECT 3 OPERATOR(pg_catalog.+) 4; "
PostgreSQL: Documentation: 9.1: NOTIFY
"When NOTIFY is used to signal the occurrence of changes to a particular table, a useful programming technique is to put the NOTIFY in a rule that is triggered by table updates. In this way, notification happens automatically when the table is changed, and the application programmer cannot accidentally forget to do it."
PostgreSQL: Documentation: 9.1: Object Identifier Types
PostgreSQL: Documentation: 9.1: PL/Python - Python Procedural Language
PostgreSQL: Documentation: 9.1: Procedural Languages
PostgreSQL: Documentation: 9.1: Query Language (SQL) Functions
PostgreSQL: Documentation: 9.1: Query Planning: 18.7.1. Planner Method Configuration (enable_seqscan, etc.)
PostgreSQL: Documentation: 9.1: Routine Vacuuming: 23.1.5. The Autovacuum Daemon
PostgreSQL: Documentation: 9.1: SELECT: Data-Modifying Statements in WITH
"PostgreSQL allows INSERT, UPDATE, and DELETE to be used as WITH queries"
PostgreSQL: Documentation: 9.1: Server-side Functions
PostgreSQL: Documentation: 9.1: SET CONSTRAINTS
"PostgreSQL does not require constraint names to be unique within a schema (but only per-table)" this is INCORRECT: UNIQUE constraints must be globally unique because their implicit indexes must be globally unique. trying to create an inter-table duplicate UNIQUE constraint gives the error: -- NOTICE: CREATE TABLE / UNIQUE will create implicit index "x" for table "b" ERROR: relation "x" already exists --
PostgreSQL: Documentation: 9.1: String Functions and Operators
PostgreSQL: Documentation: 9.1: Subquery Expressions
PostgreSQL: Documentation: 9.1: System Administration Functions
- includes set_config(), current_setting()
PostgreSQL: Documentation: 9.1: System Information Functions
"current_schemas(boolean) returns an array of the names of all schemas presently in the search path. The Boolean option determines whether or not implicitly included system schemas such as pg_catalog are included in the returned search path." "In Unix parlance, the session user is the "real user" and the current user is the "effective user"."
PostgreSQL: Documentation: 9.1: System Information Functions: has_table_privilege()
"`SELECT has_table_privilege('joe', 'mytable', 'INSERT, SELECT WITH GRANT OPTION');`"
PostgreSQL: Documentation: 9.2: Control Structures: 39.6.6.1. Obtaining information about an error
"PG_EXCEPTION_DETAIL text the text of the exception's detail message, if any" " GET STACKED DIAGNOSTICS text_var1 = MESSAGE_TEXT, text_var2 = PG_EXCEPTION_DETAIL, text_var3 = PG_EXCEPTION_HINT; "
PostgreSQL: Documentation: 9.3: Aggregate Functions
PostgreSQL: Documentation: 9.3: Client Connection Defaults: statement_timeout
"statement_timeout (integer) Abort any statement that takes more than the specified number of milliseconds, starting from the time the command arrives at the server from the client."
PostgreSQL: Documentation: 9.3: Conditional Expressions: CASE
PostgreSQL: Documentation: 9.3: CREATE CAST
PostgreSQL: Documentation: 9.3: CREATE DOMAIN
PostgreSQL: Documentation: 9.3: CREATE VIEW
"Simple views are automatically updatable: the system will allow INSERT, UPDATE and DELETE statements to be used on the view in the same way as on a regular table." (this is only available in 9.3+)
PostgreSQL: Documentation: 9.3: Error Reporting and Logging
"Valid values are TERSE, DEFAULT, and VERBOSE, each adding more fields to displayed messages. TERSE excludes the logging of DETAIL, HINT, QUERY, and CONTEXT error information. VERBOSE output includes the SQLSTATE error code (see also Appendix A) and the source code file name, function name, and line number that generated the error. Only superusers can change this setting [but it also exists in psql]." to hide stack traces in psql: \set VERBOSITY terse there is unfortunately *no* option to do this in pgAdmin
PostgreSQL: Documentation: 9.3: Error Reporting and Logging: 18.8.2. When To Log
"client_min_messages [...] Valid values are DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1, LOG, NOTICE, WARNING, ERROR, FATAL, and PANIC. Each level includes all the levels that follow it."
PostgreSQL: Documentation: 9.3: Geometric Functions and Operators
PostgreSQL: Documentation: 9.3: Interfacing Extensions To Indexes
PostgreSQL: Documentation: 9.3: Operator Classes and Operator Families
"Note that you should also create an index with the default operator class if you want queries involving ordinary <, <=, >, or >= comparisons to use an index. Such queries cannot use the xxx_pattern_ops operator classes. (Ordinary equality comparisons can use these operator classes, however.)"
PostgreSQL: Documentation: 9.3: Pattern Matching: 9.7.3.5. Regular Expression Matching Rules
"If newline-sensitive matching is specified, . and bracket expressions using ^ will never match the newline character (so that matches will never cross newlines unless the RE explicitly arranges it) and ^ and $ will match the empty string after and before a newline respectively" "non-newline-sensitive matching (default)" "If partial newline-sensitive matching is specified, this affects . and bracket expressions as with newline-sensitive matching, but not ^ and $." this is the (nonintuitive) default in most other regexp implementations "If inverse partial newline-sensitive matching is specified, this affects ^ and $ as with newline-sensitive matching, but not . and bracket expressions. This isn't very useful but is provided for symmetry."
PostgreSQL: Documentation: 9.3: pg_dump
PostgreSQL: Documentation: 9.3: pg_dump: Options
"--jobs [...] You can only use this option with the directory output format because this is the only output format where multiple processes can write their data at the same time. [...] Requesting exclusive locks on database objects while running a parallel dump could cause the dump to fail. [...] For a consistent backup, the database server needs to support synchronized snapshots, a feature that was introduced in PostgreSQL 9.2. With this feature, database clients can ensure they see the same data set even though they use different connections. pg_dump -j uses multiple database connections; it connects to the database once with the master process and once again for each worker job. Without the synchronized snapshot feature, the different worker jobs wouldn't be guaranteed to see the same data in each connection, which could lead to an inconsistent backup. "
PostgreSQL: Documentation: 9.3: pg_dumpall
"--globals-only Dump only global objects (roles and tablespaces), no databases. "
PostgreSQL: Documentation: 9.3: pg_settings
PostgreSQL: Documentation: 9.3: pg_upgrade
PostgreSQL: Documentation: 9.3: PostgreSQL Error Codes
"For some types of errors, the server reports the name of a database object (a table, table column, data type, or constraint) associated with the error; for example, the name of the unique constraint that caused a unique_violation error. Such names are supplied in separate fields of the error report message so that applications need not try to extract them from the possibly-localized human-readable text of the message. As of PostgreSQL 9.3, complete coverage for this feature exists only for errors in SQLSTATE class 23 (integrity constraint violation), but this is likely to be expanded in future." this is especially useful for error parsing in column-based import [wiki.vegpath.org/Column-based_import] (and may even have been motivated by it!)
PostgreSQL: Documentation: 9.3: psql
PostgreSQL: Documentation: 9.3: RESET
"The default value is defined as the value that the parameter would have had, if no SET had ever been issued for it in the current session. The actual source of this value might be a compiled-in default, *the configuration file* [+emph]" ie., *not* the compiled default
PostgreSQL: Documentation: 9.3: Resource Consumption: 18.4.4. Cost-based Vacuum Delay
"During the execution of VACUUM and ANALYZE commands, the system maintains an internal counter that keeps track of the estimated cost of the various I/O operations that are performed. When the accumulated cost reaches a limit (specified by vacuum_cost_limit), the process performing the operation will sleep for a short period of time, as specified by vacuum_cost_delay. Then it will reset the counter and continue execution." "This feature is disabled by default for manually issued VACUUM commands." however, *autovacuum*_vacuum_cost_delay is *enabled* by default
PostgreSQL: Documentation: 9.3: Schemas
"pg_catalog is always effectively part of the search path. If it is not named explicitly in the path then it is implicitly searched before searching the path's schemas. This ensures that built-in names will always be findable. However, you can explicitly place pg_catalog at the end of your search path if you prefer to have user-defined names override built-in names."
PostgreSQL: Documentation: 9.3: System Columns
"ctid The physical location of the row version within its table. Note that although the ctid can be used to locate the row version very quickly, a row's ctid will change if it is updated or moved by VACUUM FULL. Therefore ctid is useless as a long-term row identifier [unless your DB is select-optimized and rarely updated]. The OID, or even better a user-defined serial number, should be used to identify logical rows." "tableoid The OID of the table containing this row. This column is particularly handy for queries that select from inheritance hierarchies (see Section 5.8), since without it, it's difficult to tell which individual table a row came from. The tableoid can be joined against the oid column of pg_class to obtain the table name."
PostgreSQL: Documentation: 9.3: VALUES
PostgreSQL: Documentation: 9.3: VALUES Lists
PostgreSQL: Documentation: 9.3: WITH Queries (Common Table Expressions)
PostgreSQL: Documentation: Manuals: Log File Maintenance
PostgreSQL: Documentation: Manuals: psql
PostgreSQL: Documentation: Manuals: System Administration Functions
PostgreSQL: Documentation: Manuals: Template Databases

from/to MySQL

Converting from other Databases to PostgreSQL - PostgreSQL wiki
Converting MySQL to PostgreSQL - Wikibooks, open books for an open world
my2pg.pl
MySQL vs PostgreSQL - WikiVS
"PostgreSQL will feature the MERGE clause in a future version [5], which follows the SQL:2008 standard. This can do the same as MySQL's non-standard 'INSERT IGNORE','REPLACE' and 'INSERT ... ON DUPLICATE UPDATE' statements but with more granularity." "MySQL triggers are also not activated by cascading updates and deletes even when caused by a SQL statement"
sql - How can you represent inheritance in a database? - Stack Overflow
Comparison of database column types in MySQL, PostgreSQL, and SQLite? (Cross-Mapping) - Stack Overflow
postgresql - convert MySQL SET data type to Postgres - Stack Overflow

guest account

[PostgreSQL] prevent user change password? - Grokbase
page's self-description: (4 replies) Is it possible to prevent a user from changing their password? I have a database with a 'Guest' account, that will have limited access. I don't want any of my guests to change the Guest account password. tia Richard "Putting: host my_database guest 0.0.0.0 0.0.0.0 trust ahead of other entries in pg_hba.conf seems to do the trick. Even if guest is given a password, or it gets changed, guest can connect without being asked for it."

hash indexes

"Hash index" vs. "b-tree index" (PostgreSQL 8.0) - Grokbase
page's self-description: (33 replies) Greetings, We are working on speeding up the queries by creating indexes. We have queries with searching criteria such as "select ... where *col1='...'*". This is a simple query with only "=" operation. As a result I setup hash index on column "col1". While, in postgreSQL 8 doc, it is wirttern: *Note: * Testing has shown PostgreSQL's hash indexes to perform no better than B-tree indexes, and the index size and build time for hash indexes is much worse. For these reasons, hash index use is
* sql - Postgres hashing user defined types - Stack Overflow
"ERROR: data type urlpair has no default operator class for access method "hash"" ie. a user-defined type needs a special `hash` operator class
Re: why hash on the primary key?
"A probe into an in-memory hashtable is a lot cheaper than a probe into an index, so this type of plan makes plenty of sense if the hashtable will fit in RAM and there are going to be a lot of probes"
select * from depesz; » Blog Archive » Should you use HASH index?
select * from depesz; » Blog Archive » Waiting for 8.4 – hash based DISTINCT
sql - PostgreSQL HASH index - Stack Overflow
"It should be noted that as of version 8.4 the issue with Hash indexes being less efficient and slower than b-tree indexes is resolved"

installing 9.3

after installing, you need to restore config values that the upgrade reset: - in pgAdmin > Preferences > Query tool > Query editor, set Max characters per column back to -1 (to avoid cells being truncated). this is *not* a bug in PostgreSQL, only in pgAdmin, and does *not* signal a need to downgrade.

PostgreSQL - bugs - 9.3beta2 unrecognized node type in join alias vars when retrieving view definition
page's self-description: 9.3beta2 unrecognized node type in join alias vars when retrieving view definition. Hi all, While testing the 9.3 beta I am unable to get the view definition for some views. Because of this I...
Re: Problems with apt.postgresql.org and precise
Apt - PostgreSQL wiki
- on Linux: $ sudo apt-get --yes install postgresql-9.3 postgresql-contrib-9.3 postgresql-plpython-9.3 postgresql-plpython3-9.3 libpq-dev postgresql-9.3-postgis phppgadmin
installing postgresql 9.3.2 on ubuntu 12.04 . with error fix: ummet dependencies
page's self-description: installing postgresql 9.3.2 on ubuntu 12.04 . with error fix: ummet dependencies - Gist is a simple way to share snippets of text and code with others.

upgrading DB to 9.3

Re: Problem with data migration from 9.1 to 9.2
- on Linux: $ sudo -u postgres /usr/lib/postgresql/9.3/bin/initdb -D /var/lib/postgresql/9.3/main/ -E UTF8 --lc-collate=en_US.UTF-8 on Mac: $ sudo chmod go+w /Library/PostgreSQL/9.3/ $ sudo -u postgres initdb -D /Library/PostgreSQL/9.3/data/ -E UTF8 --lc-collate=en_US.UTF-8 $ sudo chmod go-w /Library/PostgreSQL/9.3/
Ubuntu 12.04, PostgreSQL-9.1 - Can't access $libdir/postgis-2.0 - Stack Overflow
- on Linux: $ sd ln -s postgis-2.1.so /usr/lib/postgresql/9.3/lib/postgis-2.0.so
PostgreSQL: Documentation: 9.3: pg_upgrade
- on Linux: $ sudo service postgresql stop $ for i in /etc/postgresql/9.1/main/*.conf*; do sudo ln -sf "$i" /var/lib/postgresql/9.1/main/; done $ for i in /etc/postgresql/9.3/main/*.conf*; do sudo ln -sf "$i" /var/lib/postgresql/9.3/main/; done $ sudo -u postgres -s $ mkdir /tmp/postgres_install $ cd /tmp/postgres_install $ /usr/lib/postgresql/9.3/bin/pg_upgrade --user=postgres -b /usr/lib/postgresql/9.1/bin/ -B /usr/lib/postgresql/9.3/bin/ -d /var/lib/postgresql/9.1/main/ -D /var/lib/postgresql/9.3/main/ # runtime: 1 h ("04:47:01" - ~"05:50") for a 312 GB cluster (staging tables+import) $ sudo service postgresql start $ sudo -u postgres psql -p 5433 # try connecting to the new DB $ sudo apt-get remove postgresql-9.1 $ ./delete_old_cluster.sh $ ./analyze_new_cluster.sh # runtime: 30 min ("31m6.298s") on Mac: # stop the PostgreSQL server $ sudo -u postgres -s $ mkdir /tmp/postgres_install $ cd /tmp/postgres_install $ pg_upgrade --user=postgres -b /Library/PostgreSQL/9.1/bin/ -B /Library/PostgreSQL/9.3/bin/ -d /Library/PostgreSQL/9.1/data/ -D /Library/PostgreSQL/9.3/data/ # runtime: 15 min ("06:28:35" - "06:14:41") for a 30 GB cluster (staging tables only) # start the PostgreSQL server # try connecting to the new DB $ ./delete_old_cluster.sh $ ./analyze_new_cluster.sh # runtime: 30 min ("07:10:59" - "06:40:23") for a 30 GB cluster (staging tables only)

pgAdmin: Download - Mac OSX
page's self-description: pgAdmin is the leading graphical Open Source management, development and administration tool for PostgreSQL, running on Windows, Linux, Solaris, FreeBSD and Mac OSX
Psycopg2 on Mac OS X (Snow Leopard) - Google Groups
page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations. on Mac: '(2) Create symbolic links in /usr/lib : sudo ln -s /Library/PostgreSQL/9.3/lib/libpq.5.dylib /usr/lib/libpq.5.dylib sudo ln -s /Library/PostgreSQL/9.3/lib/libssl.1.0.0.dylib /usr/lib/libssl.1.0.0.dylib sudo ln -s /Library/PostgreSQL/9.3/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.1.0.0.dylib '
python - psycopg2 installation error - Library not loaded: libssl.dylib - Stack Overflow
- with homebrew: " My solution is this: install openssl from brew $ brew install openssl copy libssl.1.0.0.dylib and libcrypto.1.0.0.dylib from /usr/local/Cellar/openssl/1.0.1c to /usr/lib/ $ cd /usr/local/Cellar/openssl/1.0.1c/lib $ sudo cp libssl.1.0.0.dylib libcrypto.1.0.0.dylib /usr/lib/ in /usr/lib directory, make a softlink libssl.dylib and libcrypto.dylib. You may have to remove the existing links. $ sudo rm libssl.dylib libcrypto.dylib $ sudo ln -s libssl.1.0.0.dylib libssl.dylib $ sudo ln -s libcrypto.1.0.0.dylib libcrypto.dylib "

invalid byte sequence errors

postgresql - How to solve UTF8 invalid byte sequence copy errors on a restore, when the source database is encoded in UTF8? - Database Administrators Stack Exchange
invalid byte sequence for encoding "UTF8": 0x00 - Grokbase
page's self-description: (6 replies) Hi, I've got another problem. I sometimes get the following SQLException when doing an insert: ERROR: invalid byte sequence for encoding "UTF8": 0x00 Exception: org.postgresql.util.PSQLException org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1525) org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1309) org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:188)
PostgreSQL - hackers - Making TEXT NUL-transparent
page's self-description: Making TEXT NUL-transparent. Occasionally, we get bitten by embedded NUL bytes in TEXT values. We take care of generating proper UTF-8, but this additional restriction sometimes slips by. It would... - Lack of support for NUL bytes is a critical limitation of PostgreSQL when dealing with arbitrary input data which should be stored verbatim

oids

OID vs SERIAL
"SERIAL: Pros: [...] * Easier to reproduce all or part of a db on a dump/restore " the oid value cannot be set by the user, and therefore a table with an fkey to an oid column is **not reimportable**
PostgreSQL - OID System Column - Table with OIDs (Identity, Autoincrement) - SQLines
page's self-description: OID is auto-incrementing integer value, unique within a PostgreSQL database (not just a table) that can be automatically assigned to each row of a table created WITH OIDS option. Although OID can be used as an identity (auto-increment) primary key column, it is recommended to use SERIAL data type instead. For more information, see Generating IDs in PostgreSQL. "Uniqueness Across all tables until the max value is reached" (which happens after "232-1 (4 billion)" rows)

PostGIS

installing

Linux

Install Postgres 9.1, PostGIS and create PostGIS template on Ubuntu 11.10 Oneiric Ocelot — Gist
Installing PostGIS on Ubuntu

Mac

Installing PostGIS on Mac OS X and Ubuntu
page's self-description: Installing PostGIS on Mac OS X and Ubuntu - Gist is a simple way to share snippets of text and code with others. "`brew install postgis`"
PostgreSQL [KyngChaos Wiki]
page's self-description: Mac OS X installers for PostgreSQL, and GIS extensions PostGIS and pgRouting. Built for Lion and Snow Leopard, 64bit only. NOTE: I do not support any other Postgres distribution in my extensions. Please do not try to install PostGIS in another Postgres and then ask why it won't install.

Moving PostGIS to another schema with Extensions - Postgres OnLine Journal
page's self-description: One of the things people have complained about for quite some time is that postgis is installed in the public schema by default and it's difficult to move after the fact. With now over 900 functions types, etc, in the 2.0.0 release that is a lot of cluttering of workspace. Now that postgis 2.0.0 is packaged as an extension, you can move all those functions etc. to another schema with the ALTER EXTENSION command. PgAdmin even throws a nice GUI on top to allow you to do this with some mouse maneuvering if you prefer the guided way. This might very well be my most favorite usability feature, because if things don't work out you can just move it back to public. I've been hesitant to do this before because well it was harder and I have a lot of 3rd party apps I work with and fear one of them hard-coded public.geometry somewhere. With extensions I can easily revert if it doesn't work out.

* Lin.ear th.inking: Quirks of the "Contains" Spatial Predicate
** Chapter 13. PostGIS Special Functions Index: 13.3. PostGIS Geography Support Functions
"&& - Returns TRUE if A's 2D *bounding box* intersects B's 2D bounding box [+emph]"
Chapter 4. Using PostGIS: Data Management and Queries: What about the date-line and the poles?
"What about the date-line and the poles? All the calculations have no conception of date-line or poles, the coordinates are spherical (longitude/latitude) so a shape that crosses the dateline is, from a calculation point of view, no different from any other shape." "All our code assumes that the points are joined by the *shorter* of the two paths along the great circle. As a consequence, shapes that have arcs of more than 180 degrees will not be correctly modelled."
postgis - What is the Difference between Geometric and Geographic columns? - Geographic Information Systems Stack Exchange
"geography, which uses geodetic measurement instead of Cartesian measurement. Coordinate points in the geography type are always represented in WGS 84 lon lat degrees" in general, you always want to use geography for point-in-polygon calculations because it handles date line wraparound properly
ST_GeogFromText
ST_MakeEnvelope

recovery mode

because of Linux out-of-memory (OOM) killer

*** postgresql - psql seems to timeout with long queries - Stack Overflow
"the Linux out-of-memory (OOM) killer triggers and terminates the process. In the latter case you will see reports of OOM killer activity in the kernel's dmesg output. I expect this is what you'll see in your case. PostgreSQL servers should be configured without virtual memory overcommit so that the OOM killer does not run and PostgreSQL can handle out-of-memory conditions its self. See the PostgreSQL documentation on Linux memory overcommit."
** The database system is in recovery mode
"Could this be the linux kernel randomly killing processes under heavy load issue?" "Here's the logs: [...] NOTICE: Message from PostgreSQL backend: The Postmaster has informed me that some other backend died abnormally and possibly corrupted shared memory. " our logs show this problem, too: " 2013-12-02 23:51:11 PST LOG: server process (PID 22312) was terminated by signal 9: Killed 2013-12-02 23:51:11 PST DETAIL: Failed process was running: /*NCBI.nodes*/ INSERT INTO "_taxonlabel_set_parent_id(parent_id=taxonlabel_pkeys.out.taxonl" ("taxonID", "result") SELECT * FROM "pg_temp"."_taxonlabel_set_parent_id(parent_id=taxonlabel_.out.taxonl_wrap"() /* EXPLAIN: Function Scan on "_taxonlabel_set_parent_id(parent_id=taxonlabel_.out.taxonl_wrap" (cost=0.25..10.25 rows=1000 width=8) */ 2013-12-02 23:51:11 PST LOG: terminating any other active server processes 2013-12-02 23:51:11 PST WARNING: terminating connection because of crash of another server process 2013-12-02 23:51:11 PST DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2013-12-02 23:51:11 PST HINT: In a moment you should be able to reconnect to the database and repeat your command. 2013-12-02 23:51:11 PST WARNING: terminating connection because of crash of another server process 2013-12-02 23:51:11 PST DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2013-12-02 23:51:11 PST HINT: In a moment you should be able to reconnect to the database and repeat your command. 2013-12-02 23:51:11 PST WARNING: terminating connection because of crash of another server process 2013-12-02 23:51:11 PST DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2013-12-02 23:51:11 PST HINT: In a moment you should be able to reconnect to the database and repeat your command. 2013-12-02 23:51:12 PST LOG: all server processes terminated; reinitializing 2013-12-02 23:51:14 PST FATAL: the database system is in recovery mode 2013-12-02 23:51:14 PST LOG: database system was interrupted; last known up at 2013-12-02 23:50:28 PST 2013-12-02 23:51:14 PST LOG: database system was not properly shut down; automatic recovery in progress 2013-12-02 23:51:14 PST FATAL: the database system is in recovery mode 2013-12-02 23:51:14 PST LOG: redo starts at 1DDB/54FA3B30 "
** Re: The database system is in recovery mode
"the kernel uses signal 9, not 11, when it wants to kill something" (!!!) apparently, the kernel doesn't even give the process a chance to shut down cleanly
How to Configure the Linux Out of Memory Killer
page's self-description: This article describes the Linux out-of-memory (OOM) killer and how to find out why it killed a particular process. It also provides methods for configuring the OOM killer to better suit the needs of many different environments. "We can set valid ranges for oom_adj from -16 to +15, and a setting of -17 exempts a process entirely from the OOM killer. The higher the number, the more likely our process will be selected for termination if the system encounters an OOM condition" "The contents of /proc/2592/oom_score can also be viewed to determine how likely a process is to be killed by the OOM killer. A score of 0 is an indication that our process is exempt from the OOM killer. The higher the OOM score, the more likely a process will be killed in an OOM condition."

Re: FATAL: the database system is in recovery mode
"unless you had extremely high settings for both checkpoint_segments and checkpoint_timeout, it shouldn't take an hour to recover from a crash. Does it appear that the startup subprocess is making progress at all? (Use "ps" to find the postmaster's startup process child, then see if it's doing anything using "strace" or some such.)"

SHOW GRANTS FOR

PostgreSQL - novice - "SHOW GRANTS FOR username" or why \z is not enough for me
page's self-description: :"SHOW GRANTS FOR username" or why \z is not enough for me. Hello As a newbie Postgres admin I like to double check that my users have all necessary rights and more important only those and... " >I somehow think that there must be a more elegant solution > like "SHOW GRANTS FOR foo" in MySQL. Any ideas? has_table_privilege() and sibling functions might help you with that "

table modified time

How to get last access/modification date of a PostgreSQL database? - Stack Overflow
sql - Latest table modified time in PostgreSQL - Stack Overflow
Re: Last insert/update/delete time for a table
Re: Last insert/update/delete time for a table

TIDs

PostgreSQL - performance - Using ctid column changes plan drastically
page's self-description: Using ctid column changes plan drastically. Hi, I was testing a query to delete duplicates to see how well using ctid works if the table doesn't have a unique identifier available. The table... "Joins on tid columns just aren't supported very well at the moment. [...] There are gotchas if any of the rows receive concurrent updates." (Tom Lane) "type tid doesn't have any hash support" (Tom Lane)
PostgreSQL: Documentation: 9.3: System Columns
"ctid The physical location of the row version within its table. Note that although the ctid can be used to locate the row version very quickly, a row's ctid will change if it is updated or moved by VACUUM FULL. Therefore ctid is useless as a long-term row identifier [unless your DB is select-optimized and rarely updated]. The OID, or even better a user-defined serial number, should be used to identify logical rows."
Re: tid scan - is it ever used?
" Currently it is the fastest possible access to a single row. So an application that selects data and wants to update rows is optimized for PostgreSQL if it knows about that fact and qualifies the updates by ctid. "
Re: What is Tid Scan
"It is scanning table by TupleID's. A tuple id is a 6-byte entity which consists of 4-byte page number and 2-byte tuple index inside page. So if you know the TID you can directly get the corresponding tuple."
Using the PostgreSQL System Columns | 2ndQuadrant
"One thing you can use these for is finding duplicate data in a table. Let’s say you’re trying to add a unique constraint, but one row in the table is duplicated 3 times, which blocks the unique index from being created. When rows are identical in every column, you can’t write any simple SELECT statement to uniquely identify them. That means deleting all of them but one copy requires some annoying and fragile SQL code, combining DELETE with LIMIT and/or OFFSET–which is always scary. If you use the ctid instead, the implementation will be PostgreSQL specific, but it will also be faster and cleaner. See Deleting Duplicate Records in a Table for an example of how that can be done."

vacuuming

* postgresql - How to monitor or do vacuum without stop all? - Database Administrators Stack Exchange
"It is safe to cancel the VACUUM FULL; doing so will undo all it's work, though"
* VACUUM FULL - PostgreSQL wiki
"ALTER TABLE .. SET DATA TYPE (relevant for 8.4 and below only) [...] This trick will not work in PostgreSQL versions 9.1 or later, as it detects that the change in data type is degenerate and so no rewrite is necessary." however, adding a USING clause could prevent this from being optimized away, allowing this to work in 9.1+ as well
** PostgreSQL - admin - How can I tell if I'm autovacuuming?
page's self-description: How can I tell if I'm autovacuuming?. From what I see in the docs, these three settings in postgresql.conf should be enough for PostgreSQL (8.2) to autovacuum with the default settings: autovacuum... "> (b) How can I verify that my databases are being vacuumed by autovacuum? pg_stat_user_tables should have a last_autovac column." "you can see autovacuum related messages like: DEBUG: autovacuum: processing database "postgres" DEBUG: autovacuum: processing database "template1" in your server log, by setting log_min_messages to 'debug1' in your postgresql.conf."
Don’t stop PostgreSQL’s autovacuum with your application | gocept weblog
page's self-description: The problem Some weeks ago, we received a complaint from a customer about bad PostgreSQL performance for a specific application. I took a look into the database and found strange things going on: the query planner was executing "interesting" query plans, tables were bloated with lots of dead rows (one was 6 times as big… "Vacuuming (whether automatic or manual) stops at the oldest transaction id that is still in use." (as opposed to skipping past any active transactions to completed ones) "autovacuum will abort if it is not able to obtain a table lock within one second" "postgres[13251]: [40-1] user=,db= ERROR: canceling autovacuum task" "I have installed a nightly cron job to force-vacuum the database. PostgreSQL has shown much improved query responses since then. Some queries’ completion times even improved by a factor of 10."
PostgreSQL - hackers - Lock problem with autovacuum truncating heap
page's self-description: Lock problem with autovacuum truncating heap. We have run across a problem with autovacuum that occurs when it can truncate off a large amount of empty blocks. It behaves different in version 9.0... "In 9.0, the autovacuum process will be aborted about 1 second after another transaction starts waiting for a lock. The result is that even a simple INSERT will take 1 second." this means that autovacuum will never complete if clients keep making INSERTs :(
postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault
"pg_stat_user_tables has columns *last_autovacuum* and *last_autoanalyze*. They will be updated whenever it does something."

» PostgreSQL – Change Default Encoding of New Databases To UTF-8 (Optional) Journal
[GENERAL] ROLLBACK in a function - Google Groups
[PostgreSQL-Hackers] DOMAINs and CASTs - Grokbase
page's self-description: (33 replies) Hi, If i create a DOMAIN an then want to create a CAST from that domain to another type it gives an error. Consider this example: """ create domain datetime as timestamp with time zone check (value between '1753-01-01 00:00:00' and '9999-12-31 23:59:59'); create function datetime2int(datetime) returns int language sql stable strict as $$ select $1::date - '1753-01-01'::date; $$; create cast(datetime as int) with function datetime2int(datetime); """ if i try to cast, get this error: select
[SOLVED] PostgreSQL Can't Rename Table Constraints
page's self-description: ' I had to rename a table yesterday due to a contact / group name change in the office and sadly I'm the only person who knows how to barely interact
** PostgreSQL - novice - Determine the name of the calling function
page's self-description: Determine the name of the calling function. Postgres 9.2 / pgAdmin III Linux / Windows XP Hello, Within a sub-function, I would like to programmatically determine the name of the... "the result of pg_exception_context clearly does contain the info you want"
** Robert Haas: Security Barrier Views
"By creating a function with a very small cost, bob has tricked the optimizer into evaluating leak(text) before checking whether organization = 'CIA', and thus Tom's name gets spit out as a side effect while executing the query, even though it still doesn't show up in the query results"
/hw/hw1/student/Hw1/postgresql-8.0.3/src/backend/access/transam/xact.c – cs186 – Trac
#1018728: postgres NAMEDATALEN limitation create problems | drupal.org
A ragged bee in the bonnet - Andrew's PostgreSQL blog
Constraint name update in PostgreSQL - Stack Overflow
database - how to emulate "insert ignore" and "on duplicate key update" (sql merge) with postgresql? - Stack Overflow
How can I change 'shmmax'
page's self-description: I have installed Oracle 9i on RedHat 9. Current value of 'shmmax' is 33554432. Some people recommend value of appr. 50% of the RAM. I tried to change the value of 'shmmax' in two ways: 1. cat 268435456 > /proc/sys/kernel/shmmax 2. sysctl -w kernel.shmmax=268435456
How Do I Enable remote access to PostgreSQL database server?
page's self-description: A step by step guide for enabling remote access to your PostgreSQL database server under Linux or UNIX operating system so that client can connect server remotely.
How to edit system catalogs in PostgreSQL 8.1? - Server Fault
ilines -- universal newlines from any data source « Python recipes « ActiveState Code
logging - Where postgresql logs are on mac os x? - Stack Overflow
Maximum columns problem - dBforums
page's self-description: Hi all, I have roughly 30,000 objects (tissue samples) for which I would like to store 1500 attributes for each sample. Each attribute is a numerical
MLA Wire: Postgresql: Indexes on Foreign Keys
MySQL and Postgres command equivalents (mysql vs psql) | End Point Blog
page's self-description: End Point developers blog about ecommerce, databases, PostgreSQL, MySQL, Interchange, Ruby on Rails, Spree, Django, PHP, Android, hosting, Git, version control, Perl, Python, Java, and SEO.
New in PostgreSQL 9.3: New in Functions | Database Technologies
page's self-description: In the series of blogging about new features in 9.3, today, I thought about blogging new functions and improvements coming in PostgreSQL. Lets look whats new in 9.3, in terms of in build functions: 1. New in one array functions for one dimensional array PostgreSQL 9.3, is coming with new functions which can help users… "5. Improvement in pg_get_viewdef() to print new line after each SELECT and FROM entry "
Omega Glory - PostgreSQL Slow Count Workaround
petere/plsh · GitHub
page's self-description: plsh - PL/sh is a procedural language handler for PostgreSQL that allows you to write stored procedures in a shell of your choice.
pgAdmin Options - Tab 3 (Query)
page's self-description: pgAdmin is the leading graphical Open Source management, development and administration tool for PostgreSQL, running on Windows, Linux, Solaris, FreeBSD and Mac OSX
phpPgAdmin - Wikipedia, the free encyclopedia
postgres foreign key to a view - Google Search
Postgres Session Variables – Neat. | An Oak In The Fall
page's self-description: Postgres Session Variables – Neat.
Postgresql - concat_ws like function? - Stack Overflow
PostgreSQL - general - [GENERAL] Update on tables when the row doesn't change
page's self-description: [GENERAL] Update on tables when the row doesn't change. Hi all, maybe it's a very silly question, but why does Postgres perform an update on the table even if no data changes? I recognized this... "Because testing for this would almost surely be a net loss for the vast majority of applications. Checking to see if the new row value exactly equals the old is hardly a zero-cost operation; if you pay that on every update, that's a lot of overhead that you are hoping to make back by sometimes avoiding the physical store of the new tuple."
PostgreSQL - general - avoiding CONTEXT messages
page's self-description: avoiding CONTEXT messages. Hi, I am using PostgreSQL 9.0 and I want to suppress CONTEXT part of error message for selected user defined Pl/PgSQL functions. Is this... 'in psql, you do it by setting: \set VERBOSITY terse'
PostgreSQL - general - Quiet "CONTEXT"?
page's self-description: Quiet "CONTEXT"?. Hi all, I'm in the process of debugging some PL/pgSQL functions. I have function A which calls function B (e.g. on Line 22). The debugging info I'm interested in is in... "In psql, "\set VERBOSITY terse" might approximate what you want." (tom lane)
PostgreSQL - general - Short-circuit boolean evaluation
page's self-description: Short-circuit boolean evaluation. Hi, Does Postgresql perform short-circuit boolean evaluation both in SQL and PL/pgSQL functions? As an example, suppose I have a function called "do_stuff" which... "SQL is a somewhat declarative language. There is no "order" to evaluation as such. So you can't talk about short circuiting either. This applies to any SQL database. You can somewhat enforce order with subselects and CASE and other such constructs."
postgresql - Getting the encoding of a Postgres database using the command line? - Stack Overflow
postgresql - Hash a column in postgres using sha-256 - Stack Overflow
PostgreSQL - How to Create User-Defined Type (UDT) - SQLines
page's self-description: You can create a user-defined type using CREATE DOMAIN and CREATE TYPE statements. Quick Example: -- Define a type using CREATE DOMAIN CREATE DOMAIN addr VARCHAR(90) NOT NULL DEFAULT 'N/A'; -- Define a type using CREATE TYPE CREATE TYPE address AS (city VARCHAR(90), street VARCHAR(90)); -- Use them in a table CREATE TABLE location (ship_address addr, full_address address);
postgresql - How to include files relative to the current executing script in psql? - Stack Overflow
postgresql - How to restore public schema into different schema - Stack Overflow
postgresql - Putting EXPLAIN results into a table? - Stack Overflow
PostgreSQL - sql - foreign key on pg_shadow
page's self-description: foreign key on pg_shadow. Dear list, Upon designing our application we thought that the following setup would be a good idea to implement security: Using the pg_shadow table as it is: |...
PostgreSQL 8.4 on Mac OS X 10.7 (Lion) | Jonathan Dean
page's self-description: Web Software Engineer
PostgreSQL and bloat
page's self-description: /dev/dim
PostgreSQL on Mac OS X
PostgreSQL services Mac OS X Lion start/stop | blog.hersoncruz.com
postgresql, DROP FUNCTION without knowing the number/type of parameters? - Stack Overflow
PostgreSQL: assigning ROWNUM (row number) to each returned row | EXPLAIN EXTENDED
page's self-description: How to create fast database queries
PostgreSQL: Documentation: 9.3: CREATE OPERATOR
pqc - PostgreSQL Query Cache - Google Project Hosting
psql -f COPY from STDIN - Grokbase
page's self-description: (5 replies) The following command works fine when pasing it to psql via the -c option: cat event.csv | \ psql -c "COPY (event_id, event_name) FROM STDIN DELIMITER AS ',' NULL AS ''" When executed from a file via -f, it does nothing (no error messages either): event.sql: COPY (event_id, event_name) FROM STDIN DELIMITER AS ',' NULL AS '' cat event.csv | psql -f event.sql What's the problem? Many thanks in advance. -- Best Regards, Tarlika Elisabeth Schmitz
Psycopg
Re: 1600 column limit per table
Re: generic options for explain
Re: How to start a database in ReadOnly mode?
Re: Restoring right side of the display
Re: Server crash... trying to figure it out
"you can easily allocate up to shared_buffers + (user_connections [~=max_connections] * work_mem)"
Re: why hash on the primary key?
"A probe into an in-memory hashtable is a lot cheaper than a probe into an index, so this type of plan makes plenty of sense if the hashtable will fit in RAM and there are going to be a lot of probes"
security - How can I hash passwords in postgresql? - Stack Overflow
set isolation level for postgresql stored procedures - Stack Overflow
"In PG your procedures aren't separate transactions"
Slow queries after convertinf DB from SQL_ASCII to UTF8 - Dev Shed
page's self-description: Slow queries after convertinf DB from SQL_ASCII to UTF8- PostgreSQL Help. Visit Dev Shed to discuss Slow queries after convertinf DB from SQL_ASCII to UTF8 "my search queries take 5 times longer to give me results" [in UTF8 than in SQL_ASCII]
SourceForge.net: phpPgAdmin: Detail: 3072065 - "Auto login" doesn't work
page's self-description: phpPgAdmin is a fully functional web-based administration utility for a PostgreSQL database server. It handles all the basic functionality as well as some advanced features such as triggers, views and functions (stored procs)
sql - Foreign key to a pg_catalog table in PostgreSQL - Stack Overflow
sql - Postgres: define a default value for CAST failures? - Stack Overflow
sql - PostgreSQL HASH index - Stack Overflow
"It should be noted that as of version 8.4 the issue with Hash indexes being less efficient and slower than b-tree indexes is resolved"
sql - What is the maximum number of columns in a PostgreSQL select query - Stack Overflow
string - PostgreSQL: Difference between text and varchar (character varying) - Stack Overflow
String Functions and Operators Compatibility - PostgreSQL wiki
ubuntu - How to configure postgresql so it accepts login+password auth? - Stack Overflow
"if it is telling you the database does not exist, then the login was successful"
Updateable Views in PostgreSQL 9.1 using INSTEAD OF Trigger | Database Technologies
page's self-description: About updateable views user ask many times. Is it supported in PostgreSQL? Can we write Complex updateable views? Answer for above is yes. Till 9.0, we have to use RULE for implementing updateable view. Again, RULE Implementation used to be a bit tidious, since user has to write multiple RULES to implement this feature. Following… "Till 9.0, we have to use RULE for implementing updateable view"

MySQL

`mysql`

for interactive formatting (including echoed queries) when inputting from a file: (this format is slightly different than the actual interactive format because the queries are echoed with lines before and after) $ mysql --table --verbose --verbose <file for raw formatting when inputting from the console: (unfortunately does not display prompt) $ mysql --batch

doc

MySQL :: MySQL 5.1 Reference Manual :: 13.2.6 LOAD DATA INFILE Syntax
"file_name cannot be an existing file" "The file is created on the server host, so you must have the FILE privilege to use this syntax" "if the MySQL client software is installed on the remote machine, you can instead use a client command such as mysql -e "SELECT ..." > file_name to generate the file on the client host"
MySQL :: MySQL 5.1 Reference Manual :: 13.2.8.1 SELECT ... INTO Syntax
MySQL :: MySQL 5.5 Reference Manual :: 10.1.9.2 CONVERT() and CAST()
MySQL :: MySQL 5.5 Reference Manual :: 11.3 Date and Time Types
MySQL :: MySQL 5.5 Reference Manual :: 11.4.2 The BINARY and VARBINARY Types
MySQL :: MySQL 5.5 Reference Manual :: 13.1.13 CREATE INDEX Syntax
" Storage Engine Permissible Index Types MyISAM BTREE InnoDB BTREE MEMORY/HEAP HASH, BTREE NDB BTREE, HASH (see note in text) " note that HASH is *not* available for either MyISAM or InnoDB
MySQL :: MySQL 5.5 Reference Manual :: 13.1.15 CREATE PROCEDURE and CREATE FUNCTION Syntax
MySQL :: MySQL 5.5 Reference Manual :: 13.5 SQL Syntax for Prepared Statements
MySQL :: MySQL 5.5 Reference Manual :: 13.6.7.3 RESIGNAL Syntax
MySQL :: MySQL 5.5 Reference Manual :: 13.6.7.4 SIGNAL Syntax
MySQL :: MySQL 5.5 Reference Manual :: 13.7.1.3 GRANT Syntax
MySQL :: MySQL 5.5 Reference Manual :: 13.7.5.11 SHOW CREATE PROCEDURE Syntax
MySQL :: MySQL 5.5 Reference Manual :: 13.7.6.4 KILL Syntax
MySQL :: MySQL 5.5 Reference Manual :: 14.3.15 Limits on InnoDB Tables
MySQL :: MySQL 5.5 Reference Manual :: 14.4.2.1 Overview of Fast Index Creation
MySQL :: MySQL 5.5 Reference Manual :: 19.5.2 View Processing Algorithms
"For UNDEFINED, MySQL chooses which algorithm to use" "A reason to choose TEMPTABLE explicitly is that locks can be released on underlying tables after the temporary table has been created and before it is used to finish processing the statement"
MySQL :: MySQL 5.5 Reference Manual :: 19.5.3 Updatable and Insertable Views
"For a view to be updatable, there must be a one-to-one relationship between the rows in the view and the rows in the underlying table. There are also certain other constructs that make a view nonupdatable" "The WITH CHECK OPTION clause can be given for an updatable view to prevent inserts or updates to rows except those for which the WHERE clause in the select_statement is true." this refers to views that include only a subset of rows (via WHERE), and requires rows inserted into it to be part of that subset. without this option, the WHERE clause is ignored when inserting.
MySQL :: MySQL 5.5 Reference Manual :: 4.5.1.1 mysql Options
MySQL :: MySQL 5.5 Reference Manual :: 4.5.1.2 mysql Commands
MySQL :: MySQL 5.5 Reference Manual :: 5.1.7 Server SQL Modes > POSTGRESQL
"Equivalent to PIPES_AS_CONCAT, ANSI_QUOTES, IGNORE_SPACE, NO_KEY_OPTIONS, NO_TABLE_OPTIONS, NO_FIELD_OPTIONS."
MySQL :: MySQL 5.5 Reference Manual :: 8.3.8 Comparison of B-Tree and Hash Indexes
MySQL :: MySQL 5.5 Reference Manual :: 9.2 Schema Object Names
MySQL :: MySQL 5.5 Reference Manual :: C.5.4.1 How to Reset the Root Password
- to add a user if you are not root but have sudo access: sudo service mysql stop sudo mysqld --skip-grant-tables --skip-networking & sudo mysql > INSERT INTO mysql.user (Host, User, Password) VALUES ('localhost', 'user', PASSWORD('password')); sudo kill "$(cat /var/run/mysqld/mysqld.pid)" sudo service mysql start mysql > GRANT SELECT, SHOW VIEW ON `db`.* TO 'user'@'localhost' WITH GRANT OPTION;
MySQL :: MySQL 5.5 Reference Manual :: MySQL Glossary :: clustered index
"Because modifying the columns of the clustered index is an expensive operation, choose primary columns that are rarely or never updated."
MySQL :: MySQL 5.6 Reference Manual :: 13.1.1 ALTER DATABASE Syntax
MySQL :: MySQL 5.6 Reference Manual :: 4.5.4 mysqldump — A Database Backup Program

dynamic SQL

How To have Dynamic SQL in MySQL Stored Procedure - Stack Overflow
MySQL – Writing dynamic SQL in stored procedures « IT Integrated Business Solutions
MySQL :: Re: Dynamic SQl in Triggers/Procdures
"Stored procedures called from triggers operate under the same restrictions as triggers, i.e., no dynamic SQL is allowed."

enum

8 Reasons Why MySQL's ENUM Data Type Is Evil - Chris Komlenic
page's self-description: The pros and (mostly) cons of MySQL's ENUM data-type (as compared to reference tables), as well as when it is okay to use and how to use it if you must. "With an ENUM we're actually storing pieces of data in a place that was only intended to hold crucial information about the model" "in large systems the effect of normalization can already tax the limits of human comprehension and complicate queries" "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered." [Donald Knuth]
Kawoolutions - SQL Database Design - 8. User-Defined Types - 8.3 Enumerated Types
page's self-description: Kawoolutions,,SQL Database Design,8. User-Defined Types,8.3 Enumerated Types "some people claim, that MySQL ENUMs are much faster than separate tables, especially on large databases. Furthermore, enums can give you some savings in database size, depending on the implementation. For example, a MySQL ENUM occupies only one byte for up to 255 values and two bytes for up to 65535 values."

missing

CHECK constraints

MySQL: How to do a (simple/complex) CHECK CONSTRAINT? « Christosoft Blog
"you can execute it in MySQL without an error. But also without any effect whatsoever. I tried it in InnoDB (MySQL 5.5) and it did not check the constraint at all." "So we go with a trigger instead. For constraints, we use a BEFORE trigger."
phpmyadmin -- create table with 'check' function [Archive] - CodingForums.com
page's self-description: [Archive] phpmyadmin -- create table with 'check' function MySQL "mysql itself does not support CHECK constraints"

MySQL Workbench

MySQL :: Download MySQL Workbench
MySQL Bugs: #40167: no method to show hidden relationships again
MySQL Bugs: #46626: Unable to copy-paste selected tables between two instances of Workbench
MySQL Bugs: #67322: MySQL Workbench not installing Ubuntu 12.10
MySQL Workbench Loses Saved Connections - Database Administrators Stack Exchange
mysql - ERROR: Error 1005: Can't create table (errno: 121) - Stack Overflow

exporting indexes separately

this happens in /mysql-workbench-gpl-5.2.47-src/modules/db.mysql/src/module_db_mysql.cpp . it does *not* use mysqldump, unfortunately.

resetting ibdata

you need to do this after manually renaming a database with InnoDB tables (but you should not manually rename a database with InnoDB tables, unless you have set innodb_file_per_table in /etc/mysql/my.cnf!)

database - How to shrink/purge ibdata1 file in MySQL - Stack Overflow
" to setup your server to use separate files for each table you need to change my.cnf in order to enable this: [mysqld] innodb_file_per_table "
MySQL :: MySQL 5.5 Reference Manual :: 14.3.6 Adding, Removing, or Resizing InnoDB Data and Log Files

string primary keys

mysql - performance penalty of strings as primary keys? - Stack Overflow
"we use varchar(32) for primary keys (GUIDs) and we haven't met performance issues of this. Our product is a web site with extreme overload and is critical to be stable. We use SQL Server 2005."

vs. other SQL DBs

PostgreSQL vs. MySQL: A Comparison of Speed, Integrity and Popularity » oBlog
"MySQL is still the world's most popular open source database, with over 50,000 downloads per day. Its accessible attitude fostered high popularity and rapid growth of the MySQL community"
The Codex » Do Not Pass This Way Again
- arguments for MySQL: better UI (phpMyAdmin): "as a user, a company head, and a person who [...] has no time [for] front-ends [which were] ignored because the backend is *so amazingly awesome*" (may be referring to lack of sufficiently-good phpMyAdmin equivalents for other DBs?)

ariejan.net
page's self-description: a blog by ariejan de vroom about software engineering and craftsmanship.
casting - How to use the CAST function correctly in a MySql SELECT statement? - Stack Overflow
"VARCHAR isn't a valid type for the CAST function, but CHAR is."
Creating a MySQL View directly in phpMyAdmin | Sky on Tech
Extend phpMyAdmin Session Timeout Value (Expired Session)
Getting out of MySQL Character Set Hell < Blue Box Blog
How Do I Enable Remote Access To MySQL Database Server?
page's self-description: MySQL Remote Access - A step-by-step guide to enable remote access to a MySQL database server under Linux, UNIX and BSD operating systems.
How do I quickly rename a mysql database (change schema name)? - Stack Overflow
linux - How to output MySQL query results in csv format? - Stack Overflow
mysql - #1071 - Specified key was too long; max key length is 767 bytes - Stack Overflow
mysql - can't remove "GRANT USAGE" - Database Administrators Stack Exchange
mysql - mysqldump returning unknown variable sql-mode error - Stack Overflow
MySQL – Extract or Restore a Single Table From a Huge mysqldump File
page's self-description: A while back, I wrote an article that explained some awk magic for extracting the restore information for a single table from a mysqldump file.  Since then I've created a Perl script that accomplishes the same thing, but is a little more flexible.
MySQL :: MySQL 5.5 Reference Manual :: 13.7.5 SHOW Syntax
MySQL :: Re: How to Restore a particular table from the mysqldump of a whole database?
MySQL :: Re: latin1_bin vs latin1_general_cs
MySQL :: Re: utf8_unicode_ci vs utf8_general_ci
MySQL Bugs: #44732: mysql client should printing warnings on stderr in batch mode
MySQL Bugs: #62207: Table comments are truncated to 66 characters rather than the 2048 the db allows
MySQL Lists: mysql: Re: Why does mysql drop index very very slow in a large table?
MySQL Lists: mysql: RE: Why does mysql drop index very very slow in a large table?
MySQL on Mac OS X
MySQLdb
Need MySQL 4 to ignore ALTER TABLE errors - Stack Overflow
What's the opposite of mysqldump to load data back into a database? | drupal.org

SQLite

Implementation Limits For SQLite
Quick example comparing SQLite vs PostgreSQL timings < Task #363: Assemble all GHCN data into a single database - Environment and organisms - NCEAS Projects
page's self-description: Redmine "SQLite takes nearly twice as long even though it doesn't have the precip records and is thus less than half the size."
sql - Can I update New in before insert trigger in sqlite? - Stack Overflow
"The only solution I've found: [in trigger body] insert into test values (New.id, now()); select raise(ignore); " "No, you can't update NEW. What I tend to do is use a VIEW with an INSTEAD OF trigger"
SQLite Query Language: CREATE TRIGGER

XPath to SQL

http://publib.boulder.ibm.com/infocenter/iadthelp/v7r0/index.jsp?topic=/com.ibm.etools.logging.ui.doc.user/ref/rmapping_XPath_SQL.html
xpath to sql translation
page's self-description: Are there any tools (java classes, tag libraries) which can translate xpath statements into a SQL query? Given an xpath query which has a predicate that filters node values or "I've never seen an Xpath to SELECT [] FROM [] WHERE clause translation. And it is not easy to do that, because you have to have rules in how you store xml in a relational database. I think that's also the reason there are no tools to do that." " We have found a couple of products on IBM's Alphaworks site. One of them uses the "DB2 XML Extenders" mapping of XML<=>SQL. The other seems to create a default mapping based on SQL metadata. Both of them are limited to Linux & Windows. While we may use these, our main RDB is DB2 for ISeries. So we are still checking. If you are curious, the links are: http://www.alphaworks.ibm.com/tech/xtable http://www.alphaworks.ibm.com/tech/db2uriaccess "

Hierarchical and recursive queries in SQL - Wikipedia, the free encyclopedia
"Recursive CTEs (or "recursive subquery factoring"[15] in Oracle lingo) can be used to traverse relations (as graphs or trees)" recursive queries function like queues: the 2nd half (after UNION ALL) fetches its input from the next row at the beginning of the queue, and appends its result row(s) to the end of the queue

Object-relational database - Wikipedia, the free encyclopedia

semantic

How should a semantic inheritance chain look like? - ANSWERS
page's self-description: During our work with a triple store we noticed a strange behaviour related to inheritance chaining and linking properties to classes. Consider the following example: 3 types defined - GrandParent, Parent and Child. 3 properties defined - GrandParentsProp, ParentsProp and ChildsProp. Parent inh... "The owl inference rule for domains is: if (A engineSize B) then because the domain of engineSize is Car, then A is a Car." but the same field name can have different meanings, depending on which class it's defined in

hardware

Intel Core i5 vs. AMD Phenom X4

From experience, the 27-inch iMac's "2.66 GHz Intel Core i5" (System Information.app) is much faster than vegbiendev's 2.44 GHz "AMD Phenom(tm) 9550 Quad-Core" (/proc/cpuinfo).

Intel Core i5 3570K vs AMD Phenom X4 9550
page's self-description: We put the 3.4 GHz 3570K to the test against the older 2.2 GHz 9550 to find out which you should buy, the Intel or the AMD. "CPUBoss recommends the Intel Core i5 3570K based on its performance, single-core performance and overclocking."

Does the magic trackpad work better than the magic mouse? - Apple Store (U.S.)
page's self-description: Well, for me personally, I never unpacked my Magic Mouse after working with my Magic Trackpad for a few hours. However, that's subjective and merely my own opinion. I have read that some people like to use both, depending on what they do on their Mac's. Battery life is also not that great with the Trackpad. I purchased the Apple battery charg... "The problem with the trackpad is that you can't touch it, unless you want the mouse to move. You end up with your hand poised in the air above the trackpad - which as already tilted up - and this is actually much worse ergonomically than a mouse where your hand can rest on the mouse" this is less true for a laptop trackpad because you can move your fingers to the side more easily
What Is WiMAX Broadband Wireless Networking?
page's self-description: WiMAX is the industry term for standard network technologies supporting long-distance wireless broadband. The WiMAX standard provides both fixed and mobile wireless connectivity options.
Wireless Standards - 802.11a, 802.11b/g/n and 802.11ac - Which Is Best?
page's self-description: Which of the popular wireless home networking standards like 802.11n or 802.11ac Wi-Fi is right for you? This article explains the pros and cons of each of these wireless standards.

Java

java - Getting the name of the current executing method - Stack Overflow

LDAP

LDAPv3 Distinguished Names
Appendix E: LDAP core.schema

OS

Unix

Linux

backups

Easy Automated Snapshot-Style Backups with Rsync
page's self-description: How to make easy, automated snapshot-style backups with rsync and UNIX
Linux equivalent to "Time Machine"
page's self-description: Is there a polished and working equivilent to Mac's time machine? I've googled and found some of the alternatives, and installed them and used them, ""Backintime" will work, but has a simple interface"
Taking a Backup Using Snapshots
"A snapshot volume [...] presents all the data that was in the volume at the time the snapshot was created"

grub

* Booting once-only - GNU GRUB Manual 0.97
page's self-description: GNU GRUB Manual 0.97
* During a chroot attempt, I got this error: "chroot: failed to run command '/bin/bash': Exec format error" - Ask Ubuntu
"Verify that you are using the right Live CD. For example, verify you are not using a 32bit CD instead of a 64bit CD. You need a 64bit kernel to run 64bit code, so check your architecture." also make sure you are using the AMD install CD if your CPU is AMD
Booting fallback systems - GNU GRUB Manual 0.97
page's self-description: GNU GRUB Manual 0.97
http://uutils.googlecode.com/svn/trunk/bin/regrub
Invoking grub-set-default - GNU GRUB Manual 0.97
page's self-description: GNU GRUB Manual 0.97
savedefault - GNU GRUB Manual 0.97
page's self-description: GNU GRUB Manual 0.97

system updates

[ubuntu] How can I uninstall / rollback the most recent update?
page's self-description: I was prompted to install a host of updates last night, and one (or more?) have rendered my system unusable unless I boot to low graphics recovery mode (failsafex). The problem I'm seeing when I boot normally is the video rate is unbearably slow (~5 FPS and worse when moving windows). I suspect the problem maybe related to the SandyBridge chipset my laptop uses. Prior to the update 3d acceleration didn't work, and now 2d is also broken. I tried selecting 'previous version of linux' and "I believe you can roll back individual packages using Synaptic Package Manager"

Ubuntu

GVFS

Bug #227808 “file permissions destroyed by vim/gvfs/fuse” : Bugs : “gvfs” package : Ubuntu
page's self-description: This bug is a little difficult to explain, but I will do my best. I'm using Ubuntu Hardy Heron with no third party repositories. Every package is up to date as of this posting. It goes like this. When I edit files with vim or gvim through gvfs/fuse/sftp the file permissions on the real remote file system get destroyed. Here is a procedure you can follow to duplicate this bug. The only non-default package you need installed is vim or gvim. 1 - Open a terminal and ssh to a remote machine. 2 -... "If you attempt to chmod any files, it will result in a chmod of the actual permissions on the remote file system, but it will not change the permissions of the file in the local ~/.gvfs directory." "using the chmod command on files through gvfs/fuse/sftp does not work in an expected fashion" "When something is mounted in .gvfs, the virtual file system applies a umask of 077" actually, it just doesn't support the g/o permissions at all
gvfs/daemon/gvfsbackendobexftp-fl-parser.c at master · gicmo/gvfs
page's self-description: gvfs - Virtual filesystem for the GNOME desktop - Ck development branch "G_FILE_ATTRIBUTE_ACCESS_CAN_READ" it doesn't use Unix-style u/g/o permissions at all :-(

Ubuntu 14.04 upgrade

Apache

**WARNING**: this breaks MultiViews redirects with the filename "index"; do not upgrade if you depend on that! after upgrading: # $ sudo a2dismod php5_cgi # $ sudo apache2ctl restart # for any redirects that use MultiViews to append an extension to files named "index", rewrite these to avoid using MultiViews

phpMyAdmin

after upgrading: # $ sudo apt-get install phpmyadmin ## whenever asked which version of a file to use, enter Y # $ sudo apache2ctl restart # *must* be done or phpMyAdmin will display a blank page!

Postgres

after upgrading, switch to the official Ubuntu version: # $ sudo rm /etc/apt/sources.list.d/pgdg.list # $ sudo apt-get update # $ sudo apt-get remove postgresql-9.3 ## **IMPORTANT**: make note of all the additional packages that will be removed # $ sudo apt-get install postgresql-9.3 # for each additional package displayed by `apt-get remove`: ## $ sudo apt-get install [package]

rsync

**WARNING**: this breaks use of ~ to reference the remote user's home dir; do not upgrade if you depend on that! after upgrading: # for any remote paths that use ~, rewrite these to avoid using it

14 Things To Do After Installing Ubuntu 14.04
page's self-description: Confused what to do after Installing Ubuntu 14.04? Here are some essential things to do after installing Ubuntu 14.04 Trusty Tahr.

VirtualBox

Access Shared Folders in a VirtualBox Ubuntu 11.04 Virtual Machine
page's self-description: Have you installed Ubuntu 11.04 as a virtual machine in VirtualBox but have had problems getting the Shared Folders feature to work? We were able to add a shared folder, but were unable to access it.

[ubuntu] On login 5 packages can be updated - Ubuntu Forums
page's self-description: [ubuntu] On login 5 packages can be updated Server Platforms
[ubuntu] remove old linux kernels
page's self-description: How does one remove the extra entries of the old linux kernels after upgrade of a new version. Remember Synaptic used to do that but it was the case when Startup manager used to do a good job. Please help if i'm missing something. this needs to be done periodically, because apt-get unfortunately does not do it automatically: $ sudo apt-get autoremove if that doesn't work: " dpkg-query -l | awk '/linux-image-*/ {print $2}' sudo apt-get --purge remove <Kernel> sudo update-grub "
Index of ftp://ftp.ucsb.edu/pub/mirrors/linux/ubuntu/
KVM/VirtManager - Community Ubuntu Documentation

liblockfile 1.08, lockfile_8h.html
Linux Find Out CPU Architecture Information
page's self-description: Explains how to use lscpu command to find out CPU Architecture information under Linux operating systems.
Linux: Delete user password
lockfile_create(3): manage lockfiles - Linux man page
page's self-description: The lockfile_create function creates a lockfile in an NFS safe way.
Revert system changes after apt-get upgrade on Linux Mint/Ubuntu - Stack Overflow
"You can use ppa-purge - install with sudo apt-get install ppa-purge"
What is used to create the shadow password hash??
page's self-description: Hello, When you create a new user thats able to login etc etc, you also create there password for them, this obviously works with the users password
HowTo: Linux Show List Of Network Cards
page's self-description: Explains how to use command line to find information about Linux network interface names.

Mac

"App Store is temporarily unavailable" error

*** Impossible to perform software updates: Apple Support Communities
"I had disabled Spotlight and although updates worked for many months, they stopped working. By re-enabling Spotlight, I was able to see OSX and App updates, download and install them." "My Spotlight index wasn't borked, it just wasn't there. I don't use spotlight so I keep it turned off. Odd that they would magically start requiring it to be turned on to perform updates." "It's the new paradigm. Updaates are tied to the Mac App Store which needs the Spotlight index to determine what's on your computer."
"The App Store is temporarily unavailable. Please try again - JustAnswer
page's self-description: Question - "The App Store is temporarily unavailable. Please try again - 2R. Find the answer to this and other Mac questions on JustAnswer. "I can sign into the store if I open it from the Dock and sign in manually"
Another App Store Updates problem: Apple Support Communities
"Dec 13, 2013" a recent problem "Fortunately, I am able to go to the command line and use the softwareupdate program and do my updates manually so I know that the problem is at the App Store leve."

10.8 Mountain Lion upgrade

**WARNING**: **DO NOT** upgrade unless you are prepared to fix several programs broken by the upgrade: svn, Apache PHP, Apache ~/Sites dirs, PostgreSQL, Python psycopg2, Python OrderedDict, X11, Xcode instructions are in the corresponding subdirs. these programs will be COMPLETELY UNAVAILABLE until they are manually fixed!

whether to upgrade

primary advantages of upgrading: Time Machine can now back up to multiple disks *at once*, allowing you to have multiple backups without needing to manually switch back and forth between the disks

Apple - OS X Mountain Lion - See everything the new OS X can do.
page's self-description: Play iOS gamers in Game Center. Share right from your apps. Send iMessages. And more. Over 200 more features, in fact. All part of OS X Mountain Lion.
Should I Upgrade to Mountain Lion?
page's self-description: Dear Lifehacker, Mountain Lion is out, and I'm seeing it has some pretty cool new features! Should I upgrade?

Mountain Lion 10.8 Updating requirements: Apple Support Communities
"the installer takes 3 minutes before rebooting itself, and 33 minutes after rebooting to complete the installation on an iMac 11,2"

Xcode

note: upgrading this requires a (free) App Store account also needed for svn

Mac App Store - Xcode
page's self-description: Read reviews, get customer ratings, see screenshots and learn more about Xcode on the Mac App Store. Download Xcode and enjoy it on your Mac.

X11

XQuartz

svn

note: upgrading this requires a (free) App Store account and a reinstall of Xcode

SVN Missing in Mac OS X 10.8 – Mountain Lion | Grapii
page's self-description: Having recently upgraded to Mac OS X 10.8 - Mountain Lion, I was surprised to learn that SVN (Subversion) is not included in this release of the OS. This was a

Apache

PHP

in /etc/apache2/httpd.conf : uncomment 'LoadModule php5_module libexec/apache2/libphp5.so'

~/Sites dirs

in /etc/apache2/extra/httpd-userdir.conf : after 'UserDir Sites', add: --- <Directory /Users/*/Sites> AllowOverride all Order Deny,Allow Allow from all </Directory> ---

Python

psycopg2

note: `ln -f` will overwrite the existing symlink to the previous libssl version $ sudo ln -fs /usr/lib/libssl.1.0.0.dylib /usr/lib/libssl.dylib $ sudo ln -fs /usr/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.dylib $ sudo easy_install psycopg2

postgresql - Update to latest libssl for Mountain Lion without MacPorts? - Ask Different
" sudo ln -fs /usr/lib/libssl.1.0.0.dylib /usr/lib/libssl.dylib sudo ln -fs /usr/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.dylib "

OrderedDict

change all scripts to import OrderedDict from collections instead of ordereddict

8.3. collections — High-performance container datatypes — Python v2.7.5 documentation

PostgreSQL

after upgrading to 10.8, you need to modify PostgreSQL: change the postgres user to _postgres: $ sudo chown -R _postgres /Library/PostgreSQL/9.1/data/ $ sudo chown -R _postgres /Library/PostgreSQL/9.1/postgres.log $ sudo -u _postgres /Library/PostgreSQL/9.1/bin/pg_ctl -D /Library/PostgreSQL/9.1/data -l /Library/PostgreSQL/9.1/postgres.log restart

Where is the Lion version of Property List Editor: Apple Support Communities
"The Property List Editor has been dropped as a standalone application. It is now fully integrated in XCode"

10.9 Mavericks upgrade

**WARNING**: **DO NOT** upgrade, it is much slower than the previous version according to Brad

whether to upgrade

conclusion: *don't*, it breaks existing programs

After a month of use, we dish out our biggest gripes with OS X Mavericks | Ars Technica
page's self-description: No new software is bug-free, and OS X 10.9 doesn't buck the trend. "When Mavericks' new responsive scrolling feature is working as designed, it draws sections of your window that aren't yet on-screen so that they show up more quickly when you scroll down (or up). [...] Unfortunately, the feature seems to have broken scrolling for other programs. I notice the problem most often in Chrome, Outlook 2011, and the Limechat IRC client [...] scrolling at higher speed becomes quite jerky and inconsistent" "The problem is not an app not making use of a new feature. It is that an OS upgrade is breaking functionality of a perfectly working application." "more recent MacBook Airs and the Retina MacBook Pros have integrated their power buttons into the keyboard, putting it up in the upper-right corner of the keyboard" "In Mavericks, pushing that power button automatically puts the computer to sleep, no questions asked. You don't need to push down on the button for any particular length of time, and there's no dialog box to ask you what you wanted to do when you pushed the button (or if you meant to push it at all). And unlike some of our multi-monitor gripes, there's no way to revert the button behavior back to its pre-Mavericks state." "Mavericks totally loses its mind if a network file share vanishes without being properly unmounted"
5 things to do before upgrading to OS X Mavericks | MacFixIt - CNET Reviews
page's self-description: If you are considering installing OS X Mavericks, think about not only system requirements, but also your current OS configuration and any installed software you use. Read this article by Topher Kessler on CNET. "All too often, people upgrade their systems only to find that their programs now do not load, or regularly crash, only because they have not been updated" i.e. this means Mavericks will *break* existing programs "unplug your backups from your system to prevent them from potentially being tampered with during the OS installation"
30 days with OS X 10.9 'Mavericks' | ZDNet
page's self-description: After 30 days with Apple's new operating system, I can sum up my experience in three words: It just works. "the new memory compression feature is helping to make switching between applications smoother, especially system resource heavy Adobe programs such as Photoshop, Lightroom, Premiere Pro and After Effects" "One of the things that I really like about OS X 10.9 is how it feels in many ways like 10.8." "Battery life Much better. I'd say that I'm getting an extra 45 minutes from a charge, which isn't bad at all."

MacFusion

OPTIONS - macfuse - Commonly useful mount-time options available in MacFUSE - The Easiest and Fastest Way to Create File Systems for Mac OS X - Google Project Hosting
- defer_permissions: "if the file system retrieves file information from another computer and reports user/group IDs /as is/, the kernel will not see any alien IDs as belonging to the user that mounted the volume. (This can happen if user ID translation is not enabled or sometimes doesn't work with sshfs.) The defer_permissions (formerly defer_auth) option is useful in such cases. It causes MacFUSE to assume that all accesses are allowed"
Issue 257 - macfusion - SSHFS | Finder | copy | Administrator password requested | fails | Finder Error code -43 - General GUI for Fuse Filesystems to be used with MacFUSE - Google Project Hosting
"-o defer_permissions"
"Could not mount filesystem: Remote host has disconnected." < More specific error message · Issue #9 · mgorbach/macfusion2 · GitHub
page's self-description: After having my Mac woken up from sleep I got the error message Could not mount filesystem: Remote host has disconnected. Which didn't exactly help me because the host was reachable fine with ssh. Investigating the log revealed this message: mount_fusefs: mount point /Volumes/username@hostname is itself on a MacFUSE volume A quick look into finder showed that the volume was still mounted while the connection was lost. Unmounting it solved this issue quickly, but I'm pretty sure this error can be ... "the volume was still mounted while the connection was lost. Unmounting it solved this issue quickly"
Stefan Hayden » MacFusion Error: Could not mount filesystem: Remote host has disconnected.
page's self-description: Stefan Hayden - Graphic Artist, Web Designer, Blog Designer. Look for his yellow backpack in real life. (applies only to OSX 10.6)

quitting programs

note that `kill -TERM` often does an *unclean* shutdown. to instead simulate File > Quit or clicking the red X in the corner: $ osascript -e 'tell application "<name (eg. Firefox)>" to quit'

recovery partition

How can I create or recreate a Lion recovery partition? - Ask Different
Lion Recovery Update v1.0

sleep

wake for network access

test using the following on another machine: (sleep 5; wakeonlan; ping <hostname>)

PrioritizeNetworkReachabilityOverSleep

Wake on Demand doesn't work - mactips - Helpful configuration tips for OS X users
page's self-description: How can I configure the iMac to wake up when I try to connect to it from another computer?

About Wake on Demand
page's self-description:

The Wake on Demand feature of Mac OS X v10.6 or later,  lets your Mac continue to share items (such as music, printers, files, or your screen) when it is asleep.

Access Your Computer Anytime and Save Energy with Wake-on-LAN
page's self-description: You want access to your home computer wherever you are, whatever you're doing, whether that's via a remote desktop connection, SSH, FTP, web interface, or any other remote access you've set up. The catch is, you don't like throwing money away to an always-on system. Luckily you can have your digital cake and eat it, too, and today I'll show you how to boot and shut down your system remotely so that it's ready for you when you need it and it's not wasting energy when you don't.
pmset(1) Mac OS X Manual Page
page's self-description: HTML versions of the OS X, OS X Server, Xcode Tools, and CHUD man pages. 'The -a, -b, -c, -u flags determine whether the settings apply to battery ( -b ), charger (wall power) ( -c ), UPS ( -u ) or all ( -a ).' 'autorestart - automatic restart on power loss' (needs to be set for UPS because the computer will always be in UPS mode at the time of a power failure) 'womp - wake on ethernet magic packet' 'ttyskeepawake - prevent idle system sleep when any tty (e.g. remote login session) is 'active'.' use `pmset -g custom` to verify settings
Wake a sleeping Mac from the network - Mac OS X Hints
" Actually your python script also needs to set SO_BROADCAST: #!/usr/bin/env python import socket; s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1); s.sendto('\xff'*6+'\x00\x24\x36\xf0\xc4\x69'*16, ('10.0.0.255', 80)); " (note that this script does not actually work, although it should)

Display Sleep, Wake-Up and Boot time From Terminal in Mac OSx | SEO, SEM, Web, eCommerce, iOS, Cloud, HTML5, Chrome Apps and Android
page's self-description: If there is a need to know exactly when your Mac was last booted up or put to sleep, or woke from sleep, You can get precise information directly from the "sysctl -a |grep waketime"
Indicator of iMac sleep status - Mac OS X Hints
pmset(1) Mac OS X Manual Page
SmartSleep
Why hibernate or 'safe sleep' mode is no longer necessary in OS X Lion (Updated) | TUAW - The Unofficial Apple Weblog
page's self-description: Update: Several commenters expressed concern that disabling safe sleep could expose you to the possibility of drive corruption if you lose power or

Time Machine

Can I boot from an time machine backup...: Apple Support Communities
How to restore data from Time Machine in Mountain Lion | Macworld
page's self-description: If you use Time Machine regularly, you're ready for whatever technological mishaps life throws at you. Restoring data from Time Machine is just as easy as backing things up in the first place. "start up your Mac from the Mountain Lion recovery partition by pressing (and holding down) Command-R at startup. This launches Recovery Mode, a portion of your drive that Mountain Lion treats as a separate volume"
mac - How to make TimeMachine back up contents of any path or mounted volume - Super User
"open the time machine settings file, found here: /Library/Preferences/com.apple.TimeMachine.plist"
Time Machine: Delete Files or Folders from Backup | a Tech-Recipes Tutorial
page's self-description: Time Machine does an amazing job of backing up all files on your system. However, sometimes you do not want every file saved forever and ever. Here is how you remove a backed up file or folder from the time machine. I really will not judge you for your huge file collections. Images, videos, programs […]
Time Machine: How to transfer backups from the current backup drive to a new backup drive
page's self-description:

If you want to switch to a different TIme Machine backup drive, use this article to learn how to transfer your existing Time Machine backups to the new backup drive, and then use it for regular Time Machine backups.

** Homebrew — The missing package manager for OS X
Dan Bishop » How to mount SFTP/SSH shares in OS X NOW UPDATED to include OS X 10.8
Disable the Dock’s ‘bounce to alert’ behavior | Macworld
page's self-description: Learn how to prevent icons from bouncing in the Dock to get your attention
firewall blocks ssh since Sept 12 update: Apple Support Communities
Flush DNS cache changed in Lion - MacRumors Forums
page's self-description: Flush DNS cache changed in Lion Mac OS X 10.7 Lion "What works under 10.7 is: sudo killall -HUP mDNSResponder "
Force every folder to be arranged by kind and sorted by name? - MacRumors Forums
page's self-description: Force every folder to be arranged by kind and sorted by name? Mac OS X 10.7 Lion
How do I remove the "extended attributes" on a file in mac osx - Stack Overflow
"Use the xattr command" "you can also use the -c option to remove all extended attributes"
How to change "Require password after sleep or screen saver" from command-line in 10.6.x - Google Groups
page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations. see ~/Library/Preferences/com.apple.screensaver.plist > askForPasswordDelay (units: s)
How to Stop Icons From Bouncing | eHow.com
page's self-description: How to Stop Icons From Bouncing. The dock is a distinctive feature of the Mac operating system. It keeps icons of commonly used and currently open programs visible at the bottom of your screen for quick access. When programs first open, or need to alert the user of some new update, the icons in the dock bounce. Many people find this helpful when...
HOWTO: Fix file sorting in Finder’s column view on Mac OS X Lion « inner.geek
Install htop on Mac OS X - opinion, technology, culture, politics - themainframe.ca
Linux Time Capsule Server - Mac Guides
Lock the keychain when the screensaver activates - Mac OS X Hints
MacBook "Core 2 Duo" 2.0 13" (White/06) Specs (Late 2006, MA700LL/A, MacBook2,1, A1181, 2121) @ EveryMac.com
page's self-description: Technical specifications for the MacBook "Core 2 Duo" 2.0 13" (White/06). Dates sold, processor type, memory info, hard drive details, price and more. "a 1280 by 800 native resolution" (example small-screen computer) visible browser area for the small-screen computer: 1203x716px
OS X: How to reset the DNS cache
page's self-description:

Learn how to reset (flush) the DNS cache.

osx - List all members of a group (Mac OS X) - Super User
Prevent .DS_Store file creation on network volumes - Mac OS X Hints
Quick Look Helper process eating virtual memory on Mac - Operating Systems
page's self-description: Quick Look Helper process eating virtual memory on Mac, Operating Systems, Computer end-user technical support troubleshooting for Windows, Mac, and Linux. Operating system tips and tweaks as well..
What is the Mac terminal command to remove ACL? - Super User
Will Time Machine backup portions of my file system that come from mounted drives? - Ask Different
"you could have unintentionally told the system to secure and ignore this volume by having an obscurely documented empty file called no_log in /path/to/your/fs/.fseventsd/no_log"

\0

Brian Dessent - Re: OT: grep for \x00 = NUL
" $ perl -ne 'print if m/\000/' or $ awk '/\000/ { print }' "
text processing - Multiline pattern match using sed, awk or grep - Unix and Linux

bash

** modifying a running shell script

you need to first unlink the file, and *then* upload the modified version. in jEdit, this can be done by enabling two-stage save. `svn up` and rsync (without --inplace) use two-stage save automatically. to prevent this from occurring accidentally, always make all your shell scripts read-only. this will force applications to use two-stage save (where supported) to replace these files.

**** Emacs workflow to edit Bash scripts while they run - Stack Overflow
"The reason deletion is safe but modification is not is that in Unix, a deleted file gets inaccessible from the file system, but the processes that had opened the file (here, /bin/bash) can still read it" so you can fix the problem by first unlinking the file, and *then* uploading the modified version
** Could replacing a bash script with a new version cause a running instance of the script to fail - Stack Overflow
"However, this would require that the running bash script is reading from the disk as it gets to each new step. Is this how it works?" yes, unfortunately incorrect: "bash will also check whether the source file is updated while the script is running. If it has been, bash will reload it and continue running it from the current position reopen the file, seek to the current position of the script, and continue running the script from that point" no, it just reads from disk: "mob's explanation is wrong, stating that it reads all at invocation and checks updates. See my answer to the question Emacs workflow to edit Bash scripts while they run [bookmarked above]" "if you need to prevent bash to reaload script you must to call it as stream."

doc

Bash Reference Manual
page's self-description: Bash Reference Manual
Bash Reference Manual: Aliases
page's self-description: Bash Reference Manual "Aliases are expanded when a command is read, not when it is executed" "Aliases are expanded when a function definition is read, not when the function is executed, because a function definition is itself a compound command" note that if statements are also "compound commands", which have aliases expanded when their definition is read (i.e. at the beginning of the if statement, for the entire if statement), not when they are executed, and any aliases for function names in them need to be unaliased *before* the if statement containing the function definition. this is necessary even if the if statement is *not* executed, because the expansion of such aliases will cause syntax errors in the parsing of the if statement itself. "If the last character of the alias value is a space or tab character, then the next command word following the alias is also checked for alias expansion."
Bash Reference Manual: Bash Variables
page's self-description: Bash Reference Manual "BASH_COMMAND The command currently being executed or about to be executed, unless the shell is executing a command as the result of a trap, in which case it is the command executing at the time of the trap." useful with `trap cmd DEBUG`
Bash Reference Manual: Filename Expansion
page's self-description: Bash Reference Manual "If GLOBIGNORE is set, each matching filename that also matches one of the patterns in GLOBIGNORE is removed from the list of matches. The filenames . and .. are always ignored when GLOBIGNORE is set and not null."
Bash Reference Manual: GNU Parallel
page's self-description: Bash Reference Manual
Bash Reference Manual: The Set Builtin
page's self-description: Bash Reference Manual -e: "The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test in an if statement, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command’s return status is being inverted with !. A trap on ERR, if set, is executed before the shell exits."

** C-Left and C-Right
"For bash, I added these bindings to my ~/.bashrc: bind '"\e[5C": forward-word' bind '"\e[5D": backward-word' # a commenter on the internet recommended these additional escapes bind '"\e[1;5C": forward-word' bind '"\e[1;5D": backward-word' "
bash - How to determine function name from inside a function - Stack Overflow
bash - How to run an alias in a shell script? - Ask Ubuntu
bash - Trap, ERR, and echoing the error line - Unix & Linux Stack Exchange
Bash Associative Arrays | Linux Journal
page's self-description: The bash man page has long had the following bug listed: "It's too big and too slow" (at the very bottom of the man page). If you agree with that, then you probably won't want to read about the "new" associative arrays that were added in version 4.0 of bash.
Bash: Detect if variable is an array - FVue
Detaching processes from the current bash session | Sébastien Wains
How do I rename a bash function? - Stack Overflow
How to reverse array in bash onliner FOR loop? - Stack Overflow
Is there any mechanism in Shell script alike "include guard" in C++? - Stack Overflow
Linux command to repeat a string n times - Super User
Mivok.net - Bash function renaming and overriding
Process Substitution
"read -a list < <( od -Ad -w24 -t u2 /dev/urandom )" this can be used to pipe the substituted command to a numbered port instead of passing it as an argument
shell script - How can I get bash to exit on backtick failure in a similar way to pipefail? - Unix & Linux Stack Exchange
"a way to take the exit status into account is to use the command substitution in a simple assignment: then the exit status of the assignment is the exit status of the last command substitution in the assignment(s)." the following doesn't work: "Only other solution that came to mind at this moment would be to use read: ls -l ghost_under_bed | read name" "Another case to watch for is explicit subshells: (somecommand). According to the interpretation above, the subshell may return a nonzero status, but since this is not a simple command in the parent shell, the parent shell should continue." this is why errexit doesn't work properly for subshells on Mac
string - How to trim whitespace from bash variable? - Stack Overflow
- the following *doesn't* work: "read -r var << eof $var eof"
TMOUT – Automatically Exit Unix Shell When there is No Activity
page's self-description: Question: I would like to terminate my Unix command line shell, when I don't execute any command for N number of seconds. i.e How to automatically log out if
where is the rehash command
page's self-description: I was trying to install gEda which installed fine from their CD, but after setting the path variables I ran rehash as the instructions told me and I "Bash doesn't have the rehash builtin command. Use "hash -r" instead which is equivalent."

gnome-keyring

linux - store svn password in gnome-keyring - Super User
ubuntu - use of gnome-keyring-daemon without X - Super User
GNOME Keyring - ArchWiki
GNOME Keyring Tool
Kenneth Holter » Using Svn client and Gnome keyring in SSH sessions
page's self-description: Subversion client software have traditionally stored (i.e. cached) plaintext user passwords, meaning that you password is accessible by anyone who can access files in your ~/.subversion/auth folder. With Subversion 1.6,

make

GNU `make'
page's self-description: GNU `make'
Override target in makefile to add more commands? - Stack Overflow

rsync

"Input/output error"

Rsync input/output error - bad disk or corrupt file? - Dedicated Server - Web Hosting Talk

rsync --append behavior
rsync ignore files
page's self-description: Hello colleague I have little problem with my rsync's.I make sync from our country to other and everything is working.When sync finish donwloading or "--exclude-from=FILE exclude patterns listed in FILE"
Rsync include/exclude quirks
page's self-description: Issues with rsync's include and exclude options

screen

scrollback

note that viewing scrollback is much more difficult to do in screen than in the surrounding terminal program (which usually has graphical scrollbars). this is because screen provided scrolling before GUIs were available, so it had to rely on terminal rendering and keyboard commands. the ability to reconnect to a running shell (which is what screen is primarily used for nowadays) was actually an extra feature, rather than the primary purpose (providing scrollback), so it is not as well-engineered for that particular purpose as today's GUI users would like. for a better way to support reconnecting, see persistent shells.

GNU Screen: Working with the Scrollback Buffer : Samsarin
- "Entering Scrollback Mode and Navigating": "To enter scrollback hit C-a [" use arrow keys to scroll, like in less
Screen > Height
"Set the display height to a specified number of lines" this will *not* change the scrollback height
persistent shells < Shell commands - BIEN - NCEAS Projects
page's self-description: Redmine

Screen
screen - The Terminal Multiplexer
Screen > Colon
"C-a : Allows you to enter `.screenrc' command lines"
Screen > The `.screenrc' file

sed

Getting "sed error - illegal byte sequence" (in bash) - Stack Overflow
" `unset LANG` "
sed : using colons as separators instead of forward slashes | commandlinefu.com
page's self-description: sed : using colons as separators instead of forward slashes - sed \"s:/old/direcory/:/new/directory/:\" <file> - Having to escape forwardslashes when using sed can be a pain. However, it's possible to instead of using / as the separator to use : . I found this by trying to substitute $PWD into my pattern, like so $ sed \"s/~.*/$PWD/\" file.txt Of course, $PWD will expand to a character string that begins with a / , which will make sed spit out an error such as \"sed: -e expression #1, char 8: unknown option to `s'\". So simply changing it to $ sed \"s:~.*:$PWD:\" file.txt did the trick.

* Monitor Progress with Pipe Viewer
page's self-description: For sys admins, running a process can feel like driving cross-country without a road map. Pipe Viewer is one way to answer the question, 'are we there yet?' "apt-get install pv" "pv -p -s ${SIZE}k"
** bash - How can I ssh directly to a particular directory? - Stack Overflow
" $ ssh -t xxx.xxx.xxx.xxx "cd /directory_wanted && exec \$SHELL" "
/dev/fd | Advanced Programming in the UNIX® Environment: UNIX File I/O | InformIT
page's self-description: Learn about the traditional UNIX I/O functions. You will see how to apply these functions when using your UNIX system.
BREs Regular Expressions
Clam AntiVirus
- to scan your home directory: $ clamscan --recursive --infected ~ &
Command to wait for a process (not a child process) - Ubuntu Forums
page's self-description: Command to wait for a process (not a child process) Programming Talk
du question: sort by size & show human-readable sizes - Ubuntu Forums
page's self-description: du question: sort by size & show human-readable sizes Programming Talk
Fink - Source Release Download
page's self-description: The Fink Team wants to bring the full world of Unix Open Source software to Darwin and Mac OS X. We modify Unix software so that it compiles and runs on Mac OS X and make it available for download as a coherent distribution
GNU `ed' Manual
page's self-description: GNU `ed' Manual
How to Convert Uppercase to Lowercase With SED | eHow.com
page's self-description: How to Convert Uppercase to Lowercase With SED. Stream editor, or sed, is a powerful Linux program that you can use to programmatically perform regular edits upon batches of text files. One example of a systematic edit that sed is well-suited for is converting all the letters in a batch of text files from uppercase to lowercase.
Killing zombie process
linux - How can I zip/compress a symlink? - Server Fault
linux - How to clear the scrollback in the screen command? - Stack Overflow
Nice Value of a Process
page's self-description: Hi, All, What is the command to display the nice value of a process? Thanks, August
Overview of Single Root I/O Virtualization (SR-IOV) (Windows Drivers)
Setup SSH to run on a non-standard port
UNIX / Linux: Send E-mail When sudo Runs
page's self-description: Explains how to send email log when sudo runs and configure sudo log file to keep track of all user commands. to turn off incorrect password e-mails: IMPORTANT: open a second root shell to use for recovery if /etc/sudoers gets messed up. otherwise, you may lock yourself out of root! $ nano /etc/sudoers add line "Defaults !mail_badpass" save $ sudo -s # test sudo
uutils - Revision 126: /trunk
uutils - Unix utilities - Google Project Hosting
What is SR-IOV? - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers
page's self-description: What is SR-IOV? Single Root I/O Virtualization is a specification that is designed to allow physical PCIe devices to present multiple virtual instances of themselves to an OS instance or hypervisor. Read the full article for all the details.
What is Umask and How To Setup Default umask Under Linux?
page's self-description: Explains how to read and setup umask (he file mode creation mask of the current process) for file under Linux operating systems using bash shell.
Xterm Control Sequences

VirtualBox

shrinking disk image

* How To Shrink Your Virtualbox VM And Free Up Space For Your Hard Disk - Make Tech Easier
"this tutorial assumes you are using Ubuntu and is running on ext2/ext3 filesystem. It does not work on ext4" note that there are other ways to zero out free space that don't require zerofree and ext2/ext3; *however*, these methods typically will create an empty file that fills the apparent disk space, thus stretching out the VM size even farther to the numerical limit "If you are still using the older Virtualbox 2.0.x version, you can easily shrink the hard disk with the following command: VboxManage modifyvdi /path/to/your/VM.vdi compact If you have upgraded your Virtualbox to the 2.1.x version, you will find that the above command will no longer work " "clonehd did indeed shrink my image. I would recommend following all of the steps including defragging and zeroing out the empty space to ensure a successful shrink."
** Shrinking a Dynamic VirtualBox Disk Image | The Linux Daily
"Keep in mind that zerofree does in fact support ext4 (I think this article [the maketecheasier.com bookmark] is a tiny bit dated, but it’s a great one)."

#12258 (Failed to load VMMR0.r0 (VERR_SYMBOL_NOT_FOUND)) – Oracle VM VirtualBox
"re-install VirtualBox 4.3"
virtualbox.org • View topic - Shared folder permissions

Windows

username URLs

Internet Explorer does not support user names and passwords in Web site addresses (HTTP or HTTPS URLs)
page's self-description: Discusses the default behavior of Internet Explorer for handling user information, such as user name and password, in HTTP and in HTTPS URLs. "05-07-2009" "By default, versions of Windows Internet Explorer that were released starting with the release of security update 832894 do not support handling user names and passwords in HTTP and HTTP with Secure Sockets Layer (SSL) or HTTPS URLs. The following URL syntax is not supported in Internet Explorer or in Windows Explorer: http(s)://username:password@server/resource.ext" ie. username URLs (containing @) don't work *at all*, not even as a hyperlink with http:// prepended note that this is actually a security fix to prevent phishing: "For example, the following URL appears to open http://www.wingtiptoys.com but actually opens http://example.com: http://www.wingtiptoys.com@example.com" however, they could have just restricted usernames containing ".com" or "www."; for the security fix, they didn't need to ban all username URLs "APPLIES TO Windows Internet Explorer 8 Windows Internet Explorer 7 Microsoft Internet Explorer 6.0 Microsoft Internet Explorer 5.5 "
[SOLVED] How to Put log + password into url directly ?
page's self-description: Hi, I'm trying to figure out if it's possible to put my log and my password into shared calendar's url directly in order to avoid the popup asking for " How to disable the new default behavior for handling user information in HTTP or HTTPS URLs To disable the new default behavior in Windows Explorer and Internet Explorer, create iexplore.exe and explorer.exe DWORD values in one of the following registry keys and set their value data to 0. * For all users of the program, set the value in the following registry key: HKEY_LOCAL_MACHINE\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME _PASSWORD_DISABLE * For the current user of the program only, set the value in the following registry key: HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME _PASSWORD_DISABLE "

Windows Update

Error code 800706BE and 8008005 - Microsoft Community
page's self-description: I have Windows Vista and I'm attempting to do automatic updates but the error messages 800706BE and 80080005 have both come up. I haven't been able to update since the 14th of January 2009 and the " 1. First, please verify that there is no any third-party firewalls preventing Windows update to communicate with "update.microsoft.com'. 2. Check whether the Windows firewall is blocking the Windows update. Please review the following KB article to follow for connection errors: http://support.microsoft.com/kb/836941 "
Temporary connection-related errors may occur when you use Windows Update or Microsoft Update
page's self-description: Describes a problem that occurs when the Windows Update client does not receive a response from the Windows Update or Microsoft Update Web site. Provides a resolution for this problem.
MicrosoftFixit50123.msi

Python

doc

13.1. csv — CSV File Reading and Writing — Python v2.7.3 documentation
19.5. XML Processing Modules — Python v2.7.4 documentation
19.6. xml.dom — The Document Object Model API — Python v2.7.2 documentation
20.6. urllib2 — extensible library for opening URLs — Python v2.7.2 documentation
26.4. The Python Profilers — Python v2.7.2 documentation
26.7. trace — Trace or track Python statement execution — Python v2.7.5 documentation
27.6. warnings — Warning control — Python v2.7.2 documentation
3. Data model — Python v2.7.3 documentation
5. Built-in Types — Python v2.7.3 documentation
5. Expressions — Python v2.7.2 documentation
7.2. re — Regular expression operations — Python v2.7.2 documentation
9.7. itertools — Functions creating iterators for efficient looping — Python v2.7.2 documentation
Overview — Python v2.7.2 documentation
regex 2014.02.19 : Python Package Index
page's self-description: Alternative regular expression module, to replace re. "POSIX character classes [[:alpha:]]; [[:^alpha:]] POSIX character classes are supported. This is actually treated as an alternative form of \p{...}. "

parallel

Documentation for processing-0.50
ParallelProcessing - PythonInfo Wiki
16.2. threading — Higher-level threading interface — Python v2.7.3 documentation
Parallel Python - Home
page's self-description: Parallel Python

3.4.1 Basic customization
3.6.2 String Formatting Operations
compiled Jython vs. CPython | Python | Python
page's self-description: compiled Jython vs. CPython Python Python "the benchmarks comparing Jython+JIT to CPython are mostly in favour of CPython" "An advantage of jython is that it profits from speedups of the JVM's. Since Java is more widespread than CPython, there is in general more interest in speeding up JVM's than in speeding up the CPython byte code interpreter."
Differences between CPython and Jython
Django | GeoDjango | Django documentation
page's self-description: Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.
How do you convert a hex representation into a string? - Python
page's self-description: How do you convert a hex representation into a string?. Python Forums on Bytes. " >>> def numToWord(n): ... hexvalue = hex(n)[2:] ... return "".join([chr(int(s, 16)) for s in [hexvalue[i:i+2] for i in range(0,len(hexvalue),2)]]) ... >>> numToWord(1650811246) 'bean' >>> "
How to determine if the variable is a function in Python? - Stack Overflow
Issue 1519638: Unmatched Group issue - workaround - Python tracker
MySQLdb User's Guide
PEP 0263 -- Defining Python Source Code Encodings
page's self-description: None
PEP 3132 -- Extended Iterable Unpacking
page's self-description: None
Psycopg
psycopg2.extensions.querycancelederror: copy from stdin failed: error in .read() call - Google Search
Python Exception Handling Techniques - Doug Hellmann
urllib2 - The Missing Manual
page's self-description: Website of Michael Foord. Python programming articles, projects and technical blog.

science

open-access data

Data reuse and the open data citation advantage [PeerJ]
page's self-description: Background. Attribution to the original contributor upon reuse of published data is important both as a reward for data creators and to document the provenance of research findings. Previous studies have found that papers with publicly available datasets receive a higher number of citations than similar studies without available data. However, few previous analyses have had the statistical power to control for the many variables known to predict citation rate, which has led to uncertain estimates of the “citation benefit”. Furthermore, little is known about patterns in data reuse over time and across datasets. Method and Results. Here, we look at citation rates while controlling for many known citation predictors and investigate the variability of data reuse. In a multivariate regression on 10,555 studies that created gene expression microarray data, we found that studies that made data available in a public repository received 9% (95% confidence interval: 5% to 13%) more citations than similar studies for which the data was not made available. Date of publication, journal impact factor, open access status, number of authors, first and last author publication history, corresponding author country, institution citation history, and study topic were included as covariates. The citation benefit varied with date of dataset deposition: a citation benefit was most clear for papers published in 2004 and 2005, at about 30%. Authors published most papers using their own datasets within two years of their first publication on the dataset, whereas data reuse papers published by third-party investigators continued to accumulate for at least six years. To study patterns of data reuse directly, we compiled 9,724 instances of third party data reuse via mention of GEO or ArrayExpress accession numbers in the full text of papers. The level of third-party data use was high: for 100 datasets deposited in year 0, we estimated that 40 papers in PubMed reused a dataset by year 2, 100 by year 4, and more than 150 data reuse papers had been published by year 5. Data reuse was distributed across a broad base of datasets: a very conservative estimate found that 20% of the datasets deposited between 2003 and 2007 had been reused at least once by third parties. Conclusion. After accounting for other factors affecting citation rate, we find a robust citation benefit from open data, although a smaller one than previously reported. We conclude there is a direct effect of third-party data reuse that persists for years beyond the time when researchers have published most of the papers reusing their own data. Other factors that may also contribute to the citation benefit are considered. We further conclude that, at least for gene expression microarray data, a substantial fraction of archived datasets are reused, and that the intensity of dataset reuse has been steadily increasing since 2003. "papers with publicly available datasets receive a higher number of citations than similar studies without available data" "There is evidence that studies that make their data available do indeed receive more citations than similar studies that do not"

QGIS [KyngChaos Wiki]
page's self-description: Mac OS X installers for QGIS. For OS X Lion, Mt Lion & Mavericks. QGIS Standard Requirements: * GDAL Complete 1.11 framework package * NOTE: GDAL now requires Apple's Java 6, and QGIS will trigger the system to ask to install it if it is not installed first install links under Requirements

software concepts

APIs

**** How to Design a Good API and Why it Matters
"Public APIs are forever - one chance to get it right" "Code lives on as examples, unit tests" "Your job is to extract true requirements _ Should take the form of use-cases" "Write multiple plug-ins before release [...] If you write three, it will work fine" "You won't be able to please everyone _ Aim to displease everyone equally" "Expect to evolve API" "You can always add, but you can never remove" "Names Should Be Largely Self-Explanatory _ Avoid cryptic abbreviations" "Parameter: indicate units, form, ownership" "Dimension is mutable • Each getSize call must allocate Dimension • Causes millions of needless object allocations" "Classes should be immutable unless there ’ s a good reason to do otherwise" "Reduce need for boilerplate code _ Generally done via cut-and-paste" "Overload With Care [...] no two with same number of args" "Three or fewer parameters is ideal" "Create helper class to hold parameters"

Dynamic scoping < Scope (computer science) - Wikipedia, the free encyclopedia
"Dynamic scoping also voids all the benefits of referential transparency" "Dynamic scoping provides an excellent abstraction for thread local storage, but if it is used that way it cannot be based on saving and restoring a global variable"
Hamming distance - Wikipedia, the free encyclopedia
Birthday problem - Wikipedia, the free encyclopedia > Probability table

sync

BitTorrent Sync

SECURITY WARNING: There is an important security hole in the Linux version of BitTorrent Sync: when btsync is run with no arguments, it makes all your files suddenly *world-readable and world-writable*, via a publicly-accessible, unprotected web interface (the WebUI). The WebUI by default allows anyone on the web to create a sync folder to view and edit any files in directories writable by you. To fix this, you apparently *must* use a config file (with the --config option) which is appropriately secured by setting webui > password. If you are currently running btsync without a secured config file, you should run `killall btsync` to turn off the WebUI. You can use `lsof -i` to verify that the WebUI is not running. The security hole has been reported at http://forum.bittorrent.com/topic/18612-security-hole-in-linux-webui-makes-all-your-files-publicly-accessible/ , but it's not clear when/whether they will fix it.

WebUI

located at http://hostname:8888/gui/

BitTorrent WebUI - Videos & Guides - Help - BitTorrent - Delivering the World's Content
page's self-description: BitTorrent is a fast and easy client for Windows and Mac with many features.
Question about the webUI - BitTorrent Sync - BitTorrent Forums
page's self-description: Question about the webUI - posted in BitTorrent Sync: Because I'd like to run this on a headless linux server, I'm obviously not going to be connecting from localhost. When I connect from a LAN interface on it's LAN IP I get "Invalid request" as shown below:- However, if I create an SSH tunnel so I'm connecting from localhost (In this case, 0.0.0.0:8889 --> localhost:8888) it works:- Is there anyway to get BitTorrent Sync to accept LAN requests? Keeping an S...
security hole in Linux WebUI makes all your files publicly accessible - BitTorrent Sync - BitTorrent Forums
page's self-description: security hole in Linux WebUI makes all your files publicly accessible - posted in BitTorrent Sync: Running btsync on Linux will by default create a *publicly-accessible, unprotected* WebUI, allowing anyone on the web to create a sync folder to view and edit files your files (i.e. files in directories writable by you). Could the defaults (used when running btsync without a config file) be changed to prevent this unintended data leak? A temporary workaround is to run `killall btsync` to tur...

.SyncIgnore is great but... - BitTorrent Sync - BitTorrent Forums
page's self-description: .SyncIgnore is great but... - posted in BitTorrent Sync: .SyncIgnore is a file where you can filter out files you don't want synced.  However it seems that when I put files in here bittorrent decides to delete those files rather than leave them un-touched.  That is unexpected.  Is this a bug?  Expected behavior?  Am I using it wrong? Cheers
Install on Linux - BitTorrent Sync - BitTorrent Forums
page's self-description: Page 1 of 2 - Install on Linux - posted in BitTorrent Sync: I just got my invite for Sync really like it so far, but i am not able to install the Linux version (SyncApp_i386-1.0.63). my Linux expert says it is missing its header. do we do something wrong or is this an bug?
Move synced directory - BitTorrent Sync - BitTorrent Forums
page's self-description: Move synced directory - posted in BitTorrent Sync: I would like to move where a directory is synced to without re-syncing it. Suppose I have given a read-only key to 10 other people, and now I want it stored in a different place on my machine. Is there any way of doing that without having them all delete the old and add a new key?

Dropbox

upgrade notification

when you use an out-of-date Dropbox client to look up a file on the Dropbox website, it just gives you links to download Dropbox, but doesn't tell you *why* you need to re-download Dropbox. many users would falsely assume they don't need to download it because they already have. however, they do in fact need to *re*-download it in order to *upgrade* their client to be compatible with the website. this page should instead say: "your version of Dropbox is out-of-date, and links to the Dropbox website will not work. please download the latest version of Dropbox below to restore this functionality." I spent a long time assuming that the website links were just broken, until I finally realized that this was because my client was out-of-date. (I had always thought that Dropbox would prompt me to upgrade if a new version were available, the way most other software does.)

exclude the .svn-Folder from Dropbox < svn - Using Subversion with DropBox - Stack Overflow
" exclude the .svn-Folder from Dropbox: [...] Go to your Folder an Copy the .svn-Folder somewhere else Go to your Dropbox-Preferences > Advanced > Selective Sync and deactivate to .svn-Folder The Folder should now be deleted from your HD and Dropbox, you can check that on the website. Put your copy of the .svn-Folder back into the directory. You should see a little gray sign like (-) "

Google Drive

do not use: unstable

** The Google Drive folder selected is not empty. Please select an empty folder. - Google Product Forums
page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations. "once you log out of GDrive, there is absolutely, positively no way at all to reconnect a GDrive login back to a local folder that already has files in it" "Google Drive only works properly if you a) have only one account and b) never, ever ... for any reason what so ever ... log out of that one account." "I selected all, dragged o my desktop, started a sync, then put everything back in my Drive folder. I was asked if I wanted to replace the file/folder and I said yes. I think this did the trick. I have a mac" but: "I did that once before and got duplicates of everything."
Google Product Forums › Google Drive › Follow Symbolic Links
"Dropbox let's me link other folders on my computer to my Synced folder using Symbolic Links. It let's me link my Documents folder to my Dropbox and is very convenient. This doesn't seem to work with Google Drive."
Symlink workaround for mac - Google Product Forums
page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations.

EasySVN < Subversion Guide | Subversion Clients, Subversion Servers & more
page's self-description: A subversion guide to source code management including subversion tutorials, reviews for svn clients and servers, free snv client downloads, and more.
iqbox-svn - Dropbox-like syncing Client for Windows using SVN - Google Project Hosting

version control

svn

'svn upgrade' fails with "svn: E155016: Insufficient NODES rows"
- a .*-revert file is indeed present, but deleting it does not fix the problem. instead, check out a new working copy: $ find . -name .svn -exec rm -rf {} \; $ svn co --force repo_url .
** svn - How to escape @ characters in Subversion managed file names? - Stack Overflow
"You need only append an at sign to the end of the path [to escape any @s in it], such as news@11@. svn cares only about the last at sign in the argument, and it is not considered illegal to omit a literal peg revision specifier after that at sign" note that the trailing-@ escape for paths containing @ should only be used on paths where @ has a special meaning (usually only the *input* path)
** svn switch (sw)
- eg. after moving everything into /trunk/, do the following on all other working copies: $ time svn switch --force https://<old-repo-root>/trunk # runtime: <1 min ("0m28.465s") # if there are errors after running svn switch, run svn switch on each broken file to restore the correct repository URL
*** Creating an SVN backup with svnsync : CloudForge Support
page's self-description: Creating an SVN backup with svnsync. SVN. Code and Deploy note: you must run *all* of the commands in this page, including creating hooks/pre-revprop-change
Apache Subversion FAQ
Executable Files in SVN
High Fibre Programming » SVN: Entry has unexpectedly changed special status
- actually, you should force-replace the item: $ svn rm --keep-local <item> $ svn add <item>
Version Control with Subversion: Ignoring Unversioned Items
"[ Begins a character class definition terminated by ], used for matching a subset of characters"

git

git-svn

to migrate from svn

*IMPORTANT*: first put your repository into the standard svn layout (branches, tags, trunk) $ cd <parent_dir_of_checkout> $ time git svn clone --stdlayout https://<svn-url> & # runtime: 15 min ("12m24.338s") # note that it appears to spend 10 min thinking about it, and then at the very end, very quickly downloads your entire repo to `svn update`: $ git svn fetch

(some steps in this article are incorrect) Converting a Subversion repository to Git, (7 steps to migrate a complete mirror of svn in git) | JohnAlbin

** GitSvnCrashCourse - Git SCM Wiki
"here's what you need to use an SVN project: git svn clone --stdlayout url # svn checkout url git svn rebase # svn update git commit -a && git svn dcommit # svn commit "
git svn - git svn repository changed domain name, how to fix? - Stack Overflow
"I changed the .git/config SVN url" note that the URL must be a web URL, not file://

* git - Add another user to project owners in Github - Stack Overflow
"You can only add collaborators to your repository. It cannot be "co-owned"."
** How do I simply create a patch from my latest git commit? - Stack Overflow
"`git show HEAD`" "`git format-patch HEAD^ --stdout`" "`git log -1 -p --pretty='%b'`"
A Gaggle Of Git Tips | Viget
"I use git rebase -i to make sure every commit is clean. For example, when I have two commits with broken code followed by a third clean commit"
Effectively Using Git With Subversion | Viget
Emmanuel Bernard: How to install Git and git-svn on Mac OS X
Git - git-apply Documentation

Concurrent Versions System - Wikipedia, the free encyclopedia
Revision Control System - Wikipedia, the free encyclopedia

web

client side

color profiles

Gfx.color management.mode - MozillaZine Knowledge Base
" 1: Enable color management for rendered graphics. 2: Enable color management for tagged graphics only. (Default) " "Without a properly calibrated monitor and a correct color profile, color management may actually make colors look worse."
Browsers, sRGB and monitor profiling | duncanmartin.com
page's self-description: Duncan Martin's Blog "Firefox can apply monior profiles, but there’s a complication. By default it will only apply the monitor profile if the image is tagged with a colour space." "Chrome doesn’t apply monitor profiles."

CSS

CSS white-space property
page's self-description: Free HTML CSS JavaScript DOM jQuery XML AJAX RSS ASP .NET PHP SQL tutorials, references, examples for web building.

Firefox

** About:config entries - MozillaZine Knowledge Base
- **IMPORTANT**: to enable security.password_lifetime, you need to set security.ask_for_password=2: "security. password_lifetime Integer Determines how long (in minutes) to go without asking for the master password in Mozilla Mail when security.ask_for_password is 2. Default value is 30." "security. ask_for_password Integer Determines when Mozilla Mail should ask for the master password. 0 (default): Only the first time it's needed 1: Every time it's needed 2: Every n minutes, where n is the value in security.password_lifetime. "
How to make all tabs reload on open? | Firefox Support Forum | Mozilla Support
" See Tab preferences and settings Use Firefoxbutton-> Options -> |Tabs|-> [] don't load tabs until selected (clear the tick) "

Flash

How Fight My Monster Actually Works - 推酷
page's self-description: How Fight My Monster Actually Works http://www.fightmymonster.com/ "We have a Flash client with a difference. It is full screen, uses a flowed layout [no it doesn't], and looks and behaves like a normal website" "The client has a complex state, but if the browser window is refreshed, the client is able to restore its previous state from the server"

Java applets

Should I use Java applets or JavaScript/PHP to make my site more interactive? - Programmers Stack Exchange
"I can do it with JAVA much faster (know it better than javascript and php). But I am afraid of browser incompatibility, security options for applets and similar things." "While Applets are ridiculously out of fashion, that does not matter to end users. On a modern machine, JVM startup time is not a problem, and *browser incompatibility less of a factor than with JavaScript*." [+emph] "Applet-specific security is irrelevant as long as you only draw stuff and don't want to do things like access the user's harddisk." (but you can use a desktop app to get around that) "they don't run in all browsers" which ones? "Libraries like JQuery and ext_js make gui development as structured as java" but without the all-important type safety "Applets [...] require client installs of java" but most clients already have this installed "Almost everyone has Flash installed, and every modern browser version can handle HTML Canvas. But while most people will have Java installed, many wont." who? "the user needs to have a Java browser plug-in installed but that's true for nearly everyone nowadays" "The main benefit is that Java gives you a huge number of libraries / tools for layout and graphics etc" "Java applets can achieve very good performance, certainly well ahead of Javascript. May not be a real issue in your case, but you get all the usual advantages of the JVM JIT-compiling down to native code" "the Java layer means you don't have to worry about browser differences / incompatibilities so much" "the Google Web Toolkit. Your client code can be written in Java that gets compiled into browser-specific JS"

Roundcube

change password

#1483886 (Roundcube doesn't work when password contains escape character "\") – Roundcube Webmail
roundcubemail/plugins/password at master · roundcube/roundcubemail · GitHub
page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
roundcubemail/plugins/password/config.inc.php.dist at master · roundcube/roundcubemail · GitHub
page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
Backslash Character in IMAP Password? - Chilkat Q&A Forum
page's self-description: I do have a question though as a user is experiencing a bug: their password has a backslash in it and the library is not able to connect to the IMAP server because the string is not properly escaped. Is there a best practice for this?
28941 – backslash and Double Quote " not escaped for IMAP username Login
roundcubemail/plugins/password/drivers at master · roundcube/roundcubemail · GitHub
page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
roundcubemail/plugins/password/drivers/ldap_simple.php at master · roundcube/roundcubemail · GitHub
page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
roundcubemail/plugins/password/drivers/chpasswd.php at master · roundcube/roundcubemail · GitHub
page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
rcube_utils.php in subversion/trunk/roundcubemail/program/include – Roundcube Webmail

wikis

Redmine

** exporting data

*** REST API

** you can also use .json to get data in JSON format

Rest api - Redmine
page's self-description: Redmine "Most of the time, the API requires authentication. To enable the API-style authentication, you have to check Enable REST API in Administration -> Settings -> Authentication"
Rest WikiPages - Redmine
page's self-description: Redmine

** Export your HostedRedmine.com projects - HostedRedmine.com Support - Online Project Management :: Free Redmine Hosting
page's self-description: Redmine
** Export your HostedRedmine.com projects - HostedRedmine.com Support - Online Project Management :: Free Redmine Hosting: Exporter.zip
page's self-description: Redmine see files.php

HTML

Comments in Wiki pages - Redmine
page's self-description: Redmine
Defect #6272: Raw HTML Formating not working - Redmine
page's self-description: Redmine
Insert HTML fragments - Redmine
page's self-description: Redmine
Question Defense » Outline Every Cell In Every Row Of A Table Using Textile Formatting
page's self-description: Outline Every Cell In Every Row Of A Table Using Textile Formatting | Question Defense

WYSIWYG

New WYSIWYG for Wiki - Redmine
page's self-description: Redmine
PeterLawrence/redmine_wysiwyg_textile · GitHub
page's self-description: redmine_wysiwyg_textile - Redmine Wysiwyg Textile Editor using tinymce

Anchors in wiki pages - Redmine
page's self-description: Redmine
Feature #4482: Cache textile rendering - Redmine
page's self-description: Redmine
Introducing the Redmine Google Docs Plugin | Evolving Web
possible to move/copy a project to new server - Redmine
page's self-description: Redmine 'it may be easier to copy my whole db over, then remove all the other projects just leaving the one I need, then add the new projects on top of it'
Redmine - How to make wiki public - Redmine
page's self-description: Redmine
Redmine - RedmineTextFormatting - Redmine
page's self-description: Redmine "%{background:lightgreen}Green Background% displays Green Background"
RedmineWikiFormatting
Using anchors in redmine - Alexey Bokov’s weblog » Blog Archive » Redmine as corporate wikipedia

TWiki

when moving the TWiki install dir, you need to update the absolute paths in lib/LocalSite.cfg

BlackListPlugin < Plugins < TWiki
DontNotify < TWiki < TWiki
ForceNewRevisionCheckBox < Codev < TWiki
HowToEmptyTrash < Support < TWiki
Is there a way to list all the child topics in a parent topic < Support < TWiki
ManagingUsers < TWiki < TWiki
Option To Control inline_styles TinyMCE Setting [#1093736] | Drupal
StrikeThroughShortHand < Codev < TWiki
TextFormattingRules < TWiki < TWiki
TinyMCEPlugin < Plugins < TWiki
TreePlugin < TWiki < TWiki
TWiki on Ubuntu 10.04 (Lucid) – Linode Library
page's self-description: Install and configure a structured wiki with TWiki.
TWiki-5.1.0 How to Disable Registration < Support < TWiki
"If you move the TWiki.TWikiRegistration topic I recommend to move it to an access controlled web"
TWikiDocumentation < TWiki < TWiki
TWikiOnUbuntu < Codev < TWiki
TWikiPreferences < TWiki < TWiki
TWikiVariables < TWiki < TWiki

Confluence Licensing | Atlassian
Pricing | Atlassian Confluence
page's self-description: Confluence pricing starts at $10 for 10 users and scales with your growing business, from startup to enterprise. Download Confluence or try hosted.

Convert HTML to XHTML
page's self-description: Online HTML to XHTML converter tool, copy and paste your HTML source code into the box below.
How do I force a favicon refresh - Stack Overflow
How to Block Certain Sites in Google Search Results without Extensions | Web Talk
page's self-description: Google users are accustomed to modify the  Google official home page with a lot of  extensions able to modify the design of the Google search engine page as
html - Making an svg image object clickable with onclick, avoiding absolute positioning - Stack Overflow
Mozilla — Plugin Check & Updates — mozilla.org
weird square favicon | Firefox Support Forum | Mozilla Support

domains

AdamsNames

IMPORTANT - AdamsNames Changes
Breaking: Registry Operator AdamsNames Hacked Locking Up .TC, .GD & .VC
page's self-description: According to a press release we just received by* KSregistry GmbH, which is owned by Key-Systems GmbH, “a third party has executed a transfer of the domain name adamsnames.net and now operates a shadow registry under this domain”. Adams Names is the Registry for the ccTLD’s of .TC, .GD and .VG and uses KSregistry as its back end to operate the registries. “According to the CEO of AdamsNames Ltd., Mr. Carsten Pauli, this transfer was not authorized by the registry operator”. “KSregistry
IMPORTANT - AdamsNames Changes (.TC - .GD - .VG) | Hexonet

DNS

Network Tools: DNS,IP,Email
page's self-description: DNS and Network troubleshooting and diagnostic tools integrated into one sweet interface.
Traceroute, Ping, Domain Name Server (DNS) Lookup, WHOIS
Deploying DNSSEC | ICANN
DNS: DNAME is useless - davidb dives in
page's self-description: DNAME, if you are not aware, is a DNS record type defined in RFC 2672. If you are familiar with the DNS’s CNAME record, then think of DNAME as …
BIND - Wikipedia, the free encyclopedia
BIND 9's New Resource Records | DNAME, Domain Alias | InformIT
page's self-description: DNS&#039;s new version of BIND, BIND 9, implements two new Resource Records to help you take DNS to an advanced level. Learn about them quickly in this article.
Trailing Dots in Domain Names
"If you're a web site administrator setting up a web site using Apache "VirtualHost" directives or similar, you need to have a ServerAlias line listing all the things the user might type to get to that web site (typically the first label, the whole name without a trailing dot, and the whole name with a trailing dot, as shown in the example above)."

subdomain delegation

How to delegate a sub-domain to other DNS servers - Simple DNS Plus
HOWTO - Delegate a Subdomain (a.k.a. subzone)

Basic Guide to DNS - Google Apps Help
page's self-description: This guide is a list of common terms you may encounter when managing the DNS settings of your domain. Click a term below to jump to the description of the term and its practical use with Google Apps

TLDs

GTLD – Trusted Information on Generic Top Level Domains – Verisign
page's self-description: Verisign brings extensive gTLD expertise. We provide premium consulting and technical services to help you apply for, manage, and monetize your new gTLD.
List of Internet top-level domains - Wikipedia, the free encyclopedia
When will the new gTLDs be released? : Marcaria.com Support
page's self-description: When will the new gTLDs be released?. New gTLDs. Domain Names
New TLD List 2012 - New Top Level Domains - New TLDs 2012
page's self-description: Comprehensive list of the new Top Level Domains (TLD's) for the New TLD program from ICANN in 2012.

Free Website Uptime Monitoring - Uptime Robot
Linux Basics - Set A Static IP On Ubuntu | HowtoForge - Linux Howtos and Tutorials
page's self-description: Linux Basics - Set A Static IP On Ubuntu
subdomain - what are the allowed characters in a sub-domain - Stack Overflow
10 Free Services to Monitor Your Site's Uptime
page's self-description: This series is supported by Rackspace, the better way to do hosting. Learn more about Rackspace's hosting solutions here. You can't sit staring at your monitor 24 hours a day to...
Three Letter Domain names
page's self-description: 3 letter domain names
.ORG Registrars | .ORG, The Public Interest Registry| Buy, Sell, Register .ORG Domain Names

e-mail

Gmail

Gmvault: gmail backup
page's self-description: backup and restore your gmail account To do full backup: /Applications/gmvault-v1.8.1-beta/bin/gmvault sync --multiple-db-owner user@gmail.com To backup only new e-mails: /Applications/gmvault-v1.8.1-beta/bin/gmvault sync --multiple-db-owner --type quick user@gmail.com In Mac OS X (including 10.8), you need to run `.../gmvault-v1.8.1-beta/bin/gmvault`. This is in the README.txt under "Linux and Mac OS X install". However, other methods don't work: gmv_runner.app just opened and closed, `pip install gmvault` gave me an error "pkg_resources.DistributionNotFound: pip==1.1", and `sudo easy_install gmvault` gave me an error "ImportError: No module named gmv.gmv_cmd".
Using Web Notifications - WebAPI | MDN
page's self-description: The Web Notifications API allows a web page to send notifications that are displayed outside the page at the system level. This allows web apps to send information to a user even if the application is idle. One of the main obvious use cases is a webmail application that would notify the user each time a new e-mail is received, even if the user is doing something else with another application. "Firefox and Safari close the notifications automatically after a few moments, e.g. 4 seconds"

server side

Apache

2.4

[ubuntu] Apache 2.4 is ready for you!
page's self-description: ! Please move thread to Server Platforms ! (my mistake) The latest stable apache web server 2.4.2 is available for both 12.04 LTS and Quantal Quetzal ! Test away! Current apache 2.2 users please backup your stuff beforehand! apt-add-repository ppa:ptn107/apache apt-get update apt-get install apache2 https://launchpad.net/~ptn107/+archive/apache
Alias 403 Forbidden with Apache - Stack Overflow
Expressions in Apache HTTP Server - Apache HTTP Server
Question #224526 : Questions : Ubuntu
page's self-description: As company compliance to Requirement 10 of PCI DSS, I recently installed a File Integrity Monitoring system using OSSEC HIDSi and it's been up and running. We are also required to conduct internal vulnerability scan with our network to comply with Requirement 11 and so I used Nexpose Rapid7. After scanning, report says that Linux server running OSSEC is vulnerable through its outdated Apache server (version 2.22) and suggest to upgrade it to version 2.4. Apache server is required for the OSSE...
Upgrading to 2.4 from 2.2 - Apache HTTP Server

auth

digest auth

mod_auth_digest - Apache HTTP Server
TracStandalone – The Trac Project
apache - How do you htdigest 400 user accounts? - Stack Overflow

htpasswd - Manage user files for basic authentication - Apache HTTP Server
"The SHA encryption format does not use salting"
Password Formats - Apache HTTP Server

doc

mod_rewrite

Advanced Techniques with mod_rewrite - Apache HTTP Server
RewriteRule Flags - Apache HTTP Server
mod_rewrite - Apache HTTP Server

Apache
Apache Core Features ErrorDocument directive
mod_alias - Apache HTTP Server
mod_autoindex - Apache HTTP Server
mod_userdir - Apache HTTP Server
mod_authz_host - Apache HTTP Server

mod_rewrite

Crazy Advanced Mod_Rewrite Debug Tutorial
page's self-description: Crazy Advanced Mod_Rewrite Debug Tutorial
Bug 38642 – mod_rewrite adds path info postfix after a substitution occured
mod rewrite - .htaccess RewriteRule - DOCUMENT_ROOT and RewriteBase - Stack Overflow

Apache vs Lighttpd - WikiVS
"In Lighttpd you're restricted to rewriting via the config level only, which means for things like Wordpress you will need to hard code the rewrite rule in."
apache2 - enabling and disabling sites - Apache - Snipplr Social Snippet Repository
page's self-description: enable and disable sites and modules in apache 2
Bug 31210 – Problematic competition between mod_rewrite and mod_dir
" > /* if someone used the PASSTHROUGH flag in per-dir > * context we just ignore it. It is only useful > * in per-server context > */ "
How can I check which version of apache is installed on a Debian machine? - Stack Overflow
Setting up an SSL server with Apache2
page's self-description: Tips for a Debian GNU/Linux System Administrator.
Setting up Self-Signed SSL Certificate to use with Apache Web Server | Knowledge by Experience
page's self-description: In this article, we will see, how to setup a self-signed SSL certificate and configure it to use with Apache Web Server in Ubuntu Linux platform. This i...
Where are the Apache Configuration files on Mac? – httpd.conf | Smart Web Developer
page's self-description: Here's where to find httpd.conf on Mac. This is the httpd.conf location on Mac OS X 10.6 Snow Leopard and Mac OS X 10.7 Lion. This article also shows how to

FastCGI

Integrating FastCGI with Java
Apache module mod_fastcgi

Java frameworks

Tomcat

How to change Tomcat default port ?

Why isn't Java used for modern web application development? - Programmers Stack Exchange
"Twitter for example was built using Rails/Ruby but once it became unscalable, they migrated to the JVM." "There are over 38 different web frameworks for java according to wikipedia ( http://en.wikipedia.org/wiki/Comparison_of_web_application_frameworks#Java ) and 23 different ORM's ( http://en.wikipedia.org/wiki/List_of_object-relational_mapping_software#Java )" "it seems that dynamic languages tend to have the tools and frameworks that remove a lot of boilerplate code. (To make up for our lack of powerful IDEs that generate boilerplate for us.)" "The steps to become proficient with Java are complex and/or expensive" "Applets extinguished maybe decade ago" (but they are still used for self-contained UIs, such as games) "Flash did what Applets tried to be. Quick start up, fast download, low memory footprint" (but they lack Java's type safety and large library) "We implemented our own dynamic class loader which means we can change Java classes on the fly without restarting the server"

URLs

What Every Developer Should Know About URLs
page's self-description: I have recently written about the value of fundamentals in software development. I am still firmly of the opinion that you need to have your fundamentals down
PURL Home Page
bitly
URL Decoder/Encoder
Not your typical links- URL Encoding at Dreamweaver FAQ.com
page's self-description: Some characters are not safe to use in a URL without first being encoded. The most common URL escape code is %20, which represents a space. Listed here are some of the most common encoding characters with examples of their use.

Data URI scheme - Wikipedia, the free encyclopedia

XML

XSLT Tutorial
page's self-description: Free HTML XHTML CSS JavaScript jQuery XML DOM XSL XSLT RSS AJAX ASP .NET PHP SQL tutorials, references, examples for web building.
Download Old Versions - <oXygen/> XML Editor
page's self-description: Download previous versions of oXygen XML Editor. this version (13.2) works with NCEAS's license

Bug #788256 “inkscape cannot dock toolbars” : Bugs : “inkscape” package : Ubuntu
page's self-description: Binary package hint: inkscape I installed Inkscape for the first time on Ubuntu 11.04 and I see that the main toolbar and the toolbar for sticking lines and segments where on the right side (later noticed the "wide" and "standard" views). 1) I tried to move the main toolbar and it worked like expected in vertical mode. I could however not dock the toolbar on top, horizontally. 2) When I tried to move the other toolbar it would not dock anywhere, not even where it was at the beginning. Pro...
Compatibility - Microsoft Access 2000 - CodeWeavers
Converting UTF8 to Latin1
Creating Hyperlinks in Adobe Acrobat « Acrobat for Legal Professionals
page's self-description: The Acrolaw Blog is a resource for lawyers, law firms, paralegals, legal IT pros and anyone interested in the use of Acrobat in the legal community. Rick Borstein– the author of the blog– is the Business Development Manager for Acrobat in the Legal Market for Adobe Systems.
cryptography - Is SHA1 better than md5 only because it generates a hash of 160 bits? - IT Security Stack Exchange
Download data for the whole world | Global Administrative Areas
English to French, Italian, German & Spanish Dictionary - WordReference.com
page's self-description: Free online Oxford dictionaries - Spanish, French, Italian, German and more. Conjugations, audio pronunciations and forums for your questions.
How To Create An Image Map Using GIMP
page's self-description: Have you ever wanted to add an image map to your web site? An image map is a graphic that contains hotspots that link to URLs. They are used as a means t
How to Import/Place an Image in a PDF
"Choose Tools—>Advanced Editing—>Touchup Object Tool Right-click and choose Place Image…"
java - What is a regex "independent capturing group"? - Stack Overflow
Password Strength Test
PDF/A - Wikipedia, the free encyclopedia
PHP: $_SERVER - Manual
Safari Books Online - Home
utf8-decoder
WARNING: <file> failed verification -- update discarded (will try again).

Google
Google Calendar
remember password
Recent Tags
Recently Bookmarked