BIEN links
Quick links
links¶
BIEN¶
BIEN3¶
markup¶
meetings¶
pub¶
vegbiendev¶
"¶
nimoy¶
starscream¶
DwC¶
VegBank¶
TNRS¶
SALVIAS¶
Postgres¶
MySQL¶
mod¶
BIEN
DataONE compatibility
KNB | Knowledge Network for Biocomplexity: share¶
- page's self-description: Metacat UI
Contribute Data | DataONE¶
- page's self-description: Member Nodes are the key to getting data available through DataONE. As shown in the Depositing Data into DataONE video, data gets deposited into one of the current Member Nodes, through that Member Node's submission process. Once the data is published through that Member Node, it is available in the DataONE catalog and should work with the various items in the Investigator Toolkit.
open-sourcing
licenses
Creative Commons — CC0 1.0 Universal¶
licensing - Difference between MIT/BSD and CC-BY licenses - Stack Overflow¶
- "CC licenses aren't meant to be used for software" (because it's incompatible with the GPL) "The license you choose for your software should not have confusion and debate: don't use CC for software"
Various Licenses and Comments about Them - GNU Project - Free Software Foundation: Licenses for Works of Practical Use Besides Software and Documentation¶
- "Creative Commons Attribution 2.0 license (a.k.a. CC BY) (#ccby) This is a non-copyleft free license that is good for art and entertainment works, and educational works. Please don't use it for software or documentation, since it is incompatible with the GNU GPL and with the GNU FDL."
(GNU Verbatim Copying License) Information for Maintainers of GNU Software: License Notices for Other Files¶
- page's self-description: Information for Maintainers of GNU Software: License Notices for Other Files
UCSB
- Standing Order 100. Officers of the University | Board of Regents¶
- "(mm) The President is authorized to develop and implement policies and procedures on matters pertaining to intellectual property, including patents, copyrights, trademarks, and tangible research products, and to execute documents necessary for the administration of intellectual property, including those which may contain commitments existing longer than seven years. The President annually shall report to the Board on matters pertaining to intellectual property." "(dd) Except as otherwise specifically provided in the Bylaws and Standing Orders, the President is authorized to execute on behalf of the Corporation all contracts and other documents necessary in the exercise of the President's duties"
- Current List of Delegations | UCOP¶
- "2201 Authority to execute documents for administration of University patent and other intellectual property matters Provost and Executive Vice President - Academic Affairs 5/9/08"
- DA2201, Execution of Documents for Administration of University Patent and Other Intellectual Property Matters - da2201.pdf¶
- "PROVOST AND EXECUTIVE VICE PRESIDENT—ACADEMIC AND HEALTH AFFAIRS [...] Effective August 13, 2007, the authority granted to the President as written above is delegated to you, subject to conditions specified elsewhere in the Bylaws and Standing Orders of The Regents and subject also to terms of applicable Regental and Presidential policies which are or may be established from time to time"
- Academic Affairs | UCOP¶
- "Aimée Dorr Provost and Executive Vice President"
** Frequently Asked Questions | UCSB Office of Technology & Industry Alliances: Can I Open Source Software?¶
- "If your UCSB research group is interested in releasing code under an open source license, please complete and forward the “Request to Open Source Software” form to the TIA Office"
- *** UCSB: Request to Open Source Software¶
- "because version three of the GPL (GPLv3) includes terms and conditions that are not acceptable to the University of California, UCSB will not authorize the release of any software that incorporates code acquired under the GPLv3"
funding sources
iPlant/NCEAS Collaboration to Build the BIEN and Environment & Organisms Working Groups' Informatics Frameworks | Marine Science Institute, UCSB¶
- " Award Period: Tuesday, March 1, 2011 to Sunday, June 30, 2013 Award Amount: $281 483 Agency Name: University of Arizona Award Number: Y562553 PI First Name: Mark PI last name: Schildhauer "
- NatureLab: NCEAS core support and capacity building | Marine Science Institute, UCSB¶
- " Award Period: Thursday, December 20, 2012 to Monday, August 31, 2015 Award Amount: $2 440 941 Agency Name: Gordon and Betty Moore Foundation Award Number: 3530 PI First Name: Frank PI last name: Davis "
^ Grants List | Gordon and Betty Moore Foundation: Creating a global center of excellence focused on knowledge generation to sustain nature and human well-being.¶
- "Grant ID: GBMF3530"
- NCEAS: National Center for Ecological Analysis and Synthesis | Marine Science Institute, UCSB¶
- " Award Period: Sunday, October 1, 2006 to Monday, September 30, 2013 Award Amount: $18 402 599 Agency Name: National Science Foundation Award Number: DBI-0553768 PI First Name: Frank PI last name: Davis Co-PI: Stephanie Hampton "
U.S. Copyright Office - Definitions (FAQ): Who is an author?¶
- page's self-description: U.S. Copyright Office is an office of public record for copyright registration and deposit of copyright material. "In cases of works made for hire, the employer or commissioning party is considered to be the author"
UArizona
About Intellectual Property at The University of Arizona | Office of Technology Transfer¶
- page's self-description: If you are a University Of Arizona Faculty member, please read the Office of Technology Transfer Practise Statement Procedures For Developing And Securing University Intellectual Property "Generally, federal agency funding contracts are governed by the terms of the Bayh-Dole Act which allows the University to own and exploit inventions created with federal funding"
- Arizona Board of Regents: Intellectual Property Policy: Microsoft Word - 6-908-Intellectual Property Policy - abor_6-908-intellectual_property_policy.pdf¶
- "The Board claims ownership of intellectual property in each of the following categories: (1) Any intellectual property created by a university or Board employee in the course and scope of employment, and (2) Any intellectual property created with the significant use of Board or university resources, unless otherwise provided in an authorized agreement for the use of those resources" (p. 1) "“Course and Scope of Employment" s hall include any activity that is listed or described in the employee' s job description or is within the employee's field of employment, in cluding research, instruction, or any activities assigned to the empl oyee that involve the creation of intellectual property. Except as se t forth in Article I, Section B, copyrightable works created by an employee in the course and scope of employment are considered to be works made for hire under U.S. Copyright Law, with owner ship vested in the Board." (p. 14) "The Board does not construe the use of office space, library resources, personal workstations, or personal computers as constituting significant use of university resources." (p. 16)
iPlant
TNRS
UA-OTT Notifications - Library - iPlant Collaborative Wiki¶
- "2011 06 15 Muir TNRS"
TNRS/LICENSE at master · iPlantCollaborativeOpenSource/TNRS · GitHub¶
- page's self-description: TNRS - The Taxonomic Name Resolution Service is a free utility for correcting and standardizing plant names.
Open Source | iPlant Collaborative¶
- "From the beginning, iPlant has had a commitment to release all code we produce with an open source software license. This is enshrined in our agreement with the National Science Foundation"
Staff & Collaborators | iPlant Collaborative Web Portal¶
- "Brad Boyle Research Associate University of Arizona"
SALVIAS
- The SALVIAS Project Mission Statement¶
- "SALVIAS is based at the University of Arizona in cooperation with the Center for Applied Biodiversity Science at Conservation International"
VegBank
- GNU General Public License v2.0 - GNU Project - Free Software Foundation¶
- "You must cause any work that you distribute or publish, that in whole or in part *contains or* is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License." [+emph] (the viral clause, which also applies to aggregated works) "If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works"
- (for comparison to GPLv2) The GNU General Public License v3.0 - GNU Project - Free Software Foundation¶
- "You may convey a work based on the Program, *or the modifications to produce it from the Program,* in the form of source code under the terms of section 4" i.e. a diff of your changes would apparently be GPL-ed, even though the line #s in the diff ought to constitute fair use
TWiki
default site-wide preferences - TWikiPreferences < TWiki < TWiki¶
Local customisations of site-wide preferences - TWikiPreferences < Main < TWiki¶
use cases
research using VegBank
2006: Spatial variation of climatic and non-climatic controls on species distribution: the range limit of Tsuga heterophylla¶
- page's self-description: jbi_1509 1384..1396 - gavin_jbi06.pdf "In the USA, known occurrences in vegetation plots were obtained from the VegBank database (VEGBANK Contributors, 2004)"
2007: Merging disparate datasets: - stohlgren-proposal_508.pdf¶
- "Modified: 2/16/07" "The datasets we will be focusing on integrating with GODM to address hypothesis 1 include VegBank" "VegBank ( http://vegbank.org ) is the vegetation plot database of the Ecological Society of America's (ESA's) Panel on Vegetation Classi fication. Most plots archived in VegBank contain complete floristi c records, precise geocoordinates, and some measure of taxon importance. At present ove r 20,000 plots spanning 27 states and containing well in excess of 6,000 species ar e archived in VegBank, and in excessive of 10,000 more plots are scheduled to be added in the near future."
2009: Temperature dependence, spatial scale, and tree species diversity in eastern Asia and North America¶
- "using data o f1,370 forest plots in the 2 continents (398 plots for eastern Asia, see ref. 23; and 972 plots for North America, see http://www.vegbank.org/)." (p. 4)
2012: Hotter? Colder? Wetter? Drier? Elevational range shifts in response to 50 years of climatic change across western North America¶
- page's self-description: Working Memory in the Two Cerebral Hemispheres M.G. Funnell, M.K. Colvin, & M.S. Gazzaniga Center for Cognitive Neuroscience, Dartmouth College - Harsch.pdf "Modified: 10/18/12" "Last-Modified: Tue, 23 Oct 2012 22:57:51 GMT" "Species occurrence data were obtained from four online databases: VegBank http://vegbank.org/vegbank/index.jsp"
‐ GBIF use cases | GBIF.ORG¶
- **** iPlant/NCEAS Collaboration to Build the BIEN and Environment & Organisms Working Groups' Informatics Frameworks | Marine Science Institute, UCSB¶
- "Award Period: Tuesday, March 1, 2011 to Sunday, June 30, 2013 Award Amount: $281 483" "Key deliverables will include: developing an integrated information resource by merging several well-established plant occurrence information resources, including specimen data from various natural history and botanical museums, as well as plots data;"
/load-geoscrub-input.sh - BIEN Geo - NCEAS Projects¶
- page's self-description: Redmine
- page's self-description: Redmine
Activity Participants: botanical information network Working Group 24th—28th October 2011 | NCEAS¶
BI_Project_Charter - iPToL - iPlant Collaborative Wiki¶
- page's self-description: Redmine
- page's self-description: Redmine
- bien.pdf¶
Botanical Information and Ecology Network¶
- page's self-description: NCEAS Project 12290: Developing an integrated botanical information network to investigate the ecological impacts of global climate change on plant biodiversity
- Munin :: nceas.ucsb.edu :: vegbiendev.nceas.ucsb.edu¶
nimoy¶
nimoy phpPgAdmin¶
Overview of NCEAS Computing and GIS Support¶
publishable datasources¶
- "Consumer Projects. iPlant provides cyberinfrastructure services for a large number of projects, including [...], BIEN"
- Taxonomic Name Resolution Service¶
vegetation data
Canadensys
Canadensys IPT¶
Norms for data use and publication – Canadensys¶
Darwin Core
- Darwin Core¶
Taxon - darwincore - Darwin Core Project site for discussion and development - Google Project Hosting¶
DwC hist¶
data management
Data management plan - Wikipedia, the free encyclopedia¶
- "By deciding on an archive ahead of time, the data collector can format data during collection to make its future submission to a database easier" "It also allows the data collector to direct requests for data to the database, rather than address requests individually" "Funding agencies are beginning to require data management plans as part of the proposal and evaluation process" "management and preservation costs may be considerable, depending on the nature of the project"
- US NSF - About: Data Management & Sharing Frequently Asked Questions (FAQs)¶
- "Usually, preserving the data in its most raw form is desirable, although data derivatives and products can also be preserved" "The expectation is that all data will be made available after a reasonable length of time" "One standard of timeliness is to make the data or samples accessible immediately after publication" "It is NSF’s strong expectation that investigators will share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants"
DiGIR
- DiGIR - Table of Contents¶
DiGIR web-browser client helper scripts.¶
GBIF Data Access and Database Interoperability¶
http://splink.cria.org.br/digir/¶
http://www.digir.net/prot/release/searchRequestDiGIR1.5.xml¶
DiGIR Messaging¶
Biodiversity Collections Index¶
- page's self-description: simple-digir - Tool (class and UI) to simplify search on DIGIR server
- BCI: Search:¶
BCI: Search: digir¶
GBIF
licenses
licenses.csv¶
- page's self-description: gbif-data-licenses - Standardizing licenses of GBIF mediated data
- "Responsibility regarding the restriction of access to sensitive data resides with the Data Publisher." "The Data Publisher warrants that it has made the necessary agreements with the original owners of the data that it can make the data available through the GBIF network."
Index of ftp://ftp.gbif.org/¶
Memorandum of Understanding for the Global Biodiversity Information Facility¶
- "This document is not legally binding and has no effect as a legal or political precedent."
Plantae (Plants) - GBIF Portal¶
- page's self-description: Observational and specimen data for Plantae "79,725,738 records with coordinates"
- "non-binding Memorandum of Understanding (MoU)"
growth forms
Growth Habits Codes and Definitions | USDA PLANTS¶
- page's self-description: Growth Habits Codes and Definitions
Appendix E (Normative): Growth Form Names, Codes, and Definitions < National Vegetation Classification Standard (Version 2)¶
NCBI
Home - Taxonomy - NCBI¶
NVS
- What is the NVS Databank?¶
The National Vegetation Survey Databank (NVS)¶
SALVIAS
- What is SALVIAS?¶
SALVIAS TaxonScrubber¶
sPlot
sPlot | German Centre for Integrative Biodiversity Research¶
sPlot-ERD_2013-09-09.jpg (JPEG Image, 1298 × 843 pixels)¶
TAPIR
- TAPIR metadata¶
TNRS
- Taxonomic Name Resolution Service - API¶
Taxonomic Name Resolution Service - Home¶
Taxonomic Name Resolution Service - Instructions¶
Taxonomic Name Resolution Service - Instructions¶
Sources¶
Tropicos
Tropicos - Home¶
Web Services - Tropicos¶
VegBank
reinstalling VegBank on VegBankVM¶
VegBank¶
VegBank Data Dictionary: Menu¶
VegBank XML¶
vegbank_erd.pdf¶
VegBranch tools : Normalizer¶
Atlas of Living Australia¶
- page's self-description: The Atlas of Living Australia provides tools to enable users of biodiversity information to find, access, combine and visualise data on Australian plants and animals
- Bien-db Info Page¶
Biodiversity Collections Index¶
Carnegie Spectranomics¶
Enquist Lab in action on PBS NOVA < PBS Nova - Fractals - Hunting the Hidden Dimension. - YouTube¶
- page's self-description: Mysteriously beautiful fractals are shaking up the world of mathematics and deepening our understanding of nature. A fractal is "a rough or fragmented geomet...
- FIA DataMart 5.1 :: Home¶
GIVD: List of Databases¶
Index Herbariorum :: NYBG.org¶
iTOL: Interactive Tree Of Life¶
Mission — TraitNet¶
- - contains field names for plots data
- PaleoLab | Pollen database¶
Phylomatic (v3)¶
RAINFOR¶
smartphone apps for field biologists¶
Smithsonian Tropical Research Institute-Center for Tropical Forest Science¶
Survey | Qualtrics Survey Software | Current Progress 21%¶
- page's self-description: Survey Software, Enterprise Survey software for enterprise feedback management and CRM solutions. Enables high-quality data collection, panel management and results analysis. Perfect for market research or CRM solution (Customer Relationship Management) integration. Free trial and consultation. "Which of the following do you use to access or submit data? (Choose all that apply.) DAAC (A Distributed Active-Archive Center) DataONE (Data Observation Network for Earth) Dryad ESA (Ecological Society of America) GBIF (A Global Biodiversity Information Facility) ILTER (International Long-term Ecological Research Network) KNB (Knowledge Network of Biocomplexity) LTER (Long-term Ecological Research Network) NEON (National Ecological Observatory Network) ORNL DAAC (Oak Ridge National Laboratory Distributed Active Archive Center) PISCO (Partnership for Interdisciplinary Study of Coastal Oceans) SAEON (South African Environmental Observation Network) SanParks (South Africa National Parks) TERN (Taiwan Ecological Research Network) UC3 (University of California Curation Center) USGS CSC (USGS Core Sciences Clearinghouse) "
TDWG: Taxonomic Concept Transfer Schema¶
Turboveg for Windows¶
Veg-X¶
NCEAS
Welcome to NCEAS | NCEAS¶
main_page [NCEAS Computing Services Knowledge Base]¶
NCEAS Identity Manager (to change password)¶
user_accounts [NCEAS Computing Services Knowledge Base]¶
Staff Computer Setup - NCEAS Knowledge Base¶
NCEAS webmail¶
support¶
Connect - NCEAS Web IRC¶
"New Moore Foundation Funding Supports UCSB Ecology Synthesis Center Embarking on a New Era " - UC Santa Barbara News Release¶
- page's self-description: New Moore Foundation Funding Supports UCSB Ecology Synthesis Center Embarking on a New Era
Identity :: People Finder¶
Factor Leave Accrual | Business & Financial Services - UCSB¶
Off-Campus Access: Login¶
projects
SNAP
Announcing SNAP 2014 Call for Proposals and Six New SNAP Working Groups | NCEAS¶
- Ocean Health Index 2013: Results Summary¶
- page's self-description: The Ocean Health Index calculates an annual global score that assesses the condition of marine ecosystems and provides a means to advance ocean policy and compare future progress worldwide.
Roundtable
*** virtual collaboration
* Facing-Fears-Virtual-Meetings.pdf¶
- "Reduce the number of participants" "Reduce the scope for the actual meeting" "Make sure you have a note - taker and a time - keeper (but shouldn’t be you)" "Ask that all stay OFF mute. Everyone will be more likely to hear key clicks" but "people can jump into the conversation without delay." however, this also increases the risk that there will be echo problems, or typing noises, etc. "conditions where remote meetings may not be the best s olution [...] people come from different time zones and must participate at same time with all others for extended periods of time" but for short meetings, this can be mitigated by scheduling at a time that is daytime for everyone "Have small meeting pods - people in same room joining in together rather than from their own desks." I actually think this is a bad idea, because it causes the people within the pods to communicate with each other rather than with the entire group
- - note that the chart on face-to-face vs. technology-enabled meetings (p3) does not includes *disadvantages* of either
- Using virtual collaboration to replace or supplement carbon-intensive research travel: barriers, best practices, and opportunities for innovation | Roundtable¶
thinking preferences
- Thinking Preferences¶
- Dr. David Goodstein -- Recent Articles¶
iPlant
Atmosphere¶
Big Data: The View From The Classroom « A Smarter Planet Blog A Smarter Planet Blog¶
- page's self-description: by Nirav Merchant, iPlant Collaborative, Director, BioComputing at the Arizona Research Laboratories (ARL), The University of Arizona Today researchers in
Global call-in numbers¶
- page's self-description: 85
- iPlant Collaborative Contributor License Agreement¶
iPlant User Management¶
- page's self-description: TNRS - The Taxonomic Name Resolution Service is a free utility for correcting and standardizing plant names.
Martha Narro, Ph.D. | iPlant Collaborative Web Portal¶
System Dashboard - The iPlant Collaborative JIRA¶
Taxonomic Name Resolution Service - Instructions¶
TNRS - The iPlant Collaborative JIRA¶
resources
copyright
authority to bind
Authority to Bind – “You did what?” | Pazar Law¶
- "For officers (and perhaps directors) it should be expressed in writing as part of the corporate bylaws"
The High-touch Legal Services® Blog…for Startups!Who Can Sign Contracts for a Corporation? » The High-touch Legal Services® Blog...for Startups!¶
- "A “derivative work” is a work based upon one or more preexisting works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which a work may be recast, transformed, or adapted. A work consisting of editorial revisions, annotations, elaborations, or other modifications which, as a whole, represent an original work of authorship, is a “derivative work”."
Let’s Make Copyright Opt-OUT¶
- page's self-description: If you oppose IP and don’t want a patent–just don’t apply. Unfortunately, someone else might independently invent the same thing, patent it, and shut you down, since your having invented it first, or independently invented it, is no defense. So, […] "But if you don’t want copyright, you’re out of luck. Under the current law, copyright is received automatically. Contrary to popular wisdom, you don’t “copyright” something. It’s not a verb. You don’t need to put a copyright notice on your work. You don’t need to register it. It’s automatic. And there’s nothing you can do about it. (See “Copyright Is Very Sticky!“) This makes it often impossible to know who the owner of a given work is or who to approach to ask permission; it contributes to the “orphan works” problem as well–where older works still apparently under copyright fall into obscurity because the copyright owner is unknown or cannot be found for permission to republish."
- "variable declarations are generally not considered protected elements of a program"
- page's self-description: Factsheet on fair use of copyrighted works. "Copyright [...] does not extend to any ideas, systems, or factual information conveyed in a work"
databases
relational
modeling
- *** The Enhanced Entity-Relationship (EER) Model @ Ch08.pdf¶
- "If all the subclasses of a specialization/ generalization have few specific attributes and no specific relationships Can be merged into the superclass Replace with one or more type attributes that specify the subclass or subclasses that each entity belongs to " "Identification [...] Needed at two levels To distinguish among database objects and classes To identify database objects and to relate them to their real-world counterparts "
- Anomalies: Insert, Update and Delete - Access wiki - Access Help and How-to - Microsoft Office by UtterAccess.com¶
- page's self-description: Microsoft Access help database discussion forum and wiki
Crow's Feet Are Best¶
- page's self-description: The Data Administration Newsletter delivers business intelligence, data warehousing and analytics resources provided by Claudia Imhoff, Bill Inmon and other experts. Additional topics include data quality, data integration, CRM, data marts, data mining, business performance management, BPM, data modeling, enterprise application management, ERP, RFID, storage, supply chain and others. "Peter Chen invented data modeling with his innovative way of showing data structures in terms of entities and relationships"
- "MySQL Workbench offers creating, editing and exporting EER Models"
- GUI Database Design Tools - PostgreSQL wiki¶
MDB
- Linux MDB driver - alpha version¶
- page's self-description: OpenOffice.org: The Free, Open Source Office Suite
The Electrofriend: How to compile mdbtools on Mac OS X 10.4 and 10.5¶
MDB Tools - Unlocking Your Data¶
How to Open and Edit MS Access Databases from Mac OS X¶
- page's self-description: Information on how to read and query MS Access databases from Mac OS X.
SQL
inheritance
xkcd • View topic - Inheritance in SQL¶
- "One table per class with type field: Same as above, with a column in the parent class table indicating the sub-class." "Or... use Postgres http://www.postgresql.org/docs/8.1/stat ... herit.html"
optimization
Afraid of SSD?¶
- page's self-description: Should a SQL performance consultant be afraid of SSD? Some surprising facts why I'm not afraid of SSD. "If you want to get the factor 50 speed-up of SSDs, you’d better avoid reading large chunks of sequential data, because that’s where you can only gain a factor five improvement. To put that into database context: if you are doing many full table scans, you won’t get the full potential of SSD. On the other hand, index lookups have a tendency to cause many random IO operations and can thus benefit from the fast response time of SSDs"
Hash Aggregate - Craig Freedman's SQL Server Blog - Site Home - MSDN Blogs¶
- "the sort/group algorithm can use an index to avoid the sort operation, thus enabling a pipelined group by."
- page's self-description: Explaining the difference between "Bitmap Index Scan" and "Index Scan" in the PostgreSQL database. "A plain Index Scan fetches one tuple-pointer at a time from the index, and immediately visits that tuple in the table. A bitmap scan fetches all the tuple-pointers from the index in one go, sorts them using an in-memory "bitmap" data structure, and then visits the table tuples in physical tuple-location order."
PostgreSQL
backups
- online backups: pg_start_backup() method: (more reliable and Postgres-recommended, but requires putting the DB server into a special mode during backup) # configure: ## in postgresql.conf, set wal_level = hot_standby ## $ sudo service postgresql restart # run: ## $ sudo -u postgres psql <<<"SELECT pg_start_backup('backup', true);" ## copy files using rsync, etc. ## $ sudo -u postgres psql <<<"SELECT pg_stop_backup();" wal_keep_segments method: (less reliable, but the backup is completely independent of the DB server) # configure: ## in postgresql.conf: ### set wal_level = hot_standby ### set wal_keep_segments to long enough that the WAL chain spans the duration of the backup #### to determine this, put the DB into peak usage and measure how many WAL segments are created in the time it takes an rsync with minimal changes to complete (about a few seconds) ## $ sudo service postgresql restart # run: ## copy files using rsync, etc. ## copy again to get changes that occurred during the copy operation ## copy again to make sure there are no more changes offline backups: # copy files using rsync, etc. # copy again to get changes that occurred during the copy operation # $ sudo service postgresql stop # copy again to make the copy a consistent snapshot # $ sudo service postgresql start
pg_clog, pg_xlog
** pg_log, pg_xlog, and pg_clog¶
- page's self-description: ... so I deleted several GB of log files from the pg_xlog directory to free up some disk space. Now my database won't start. Oh-oh. Um ... when ... "Should you ever delete files from pg_clog, you might as well delete the entire database directory. There is no recovery from a missing clog."
KB Parallels: How to recreate postgres pg_clog/xxxx file¶
- page's self-description: Symptoms Sometimes the transaction log files become corrupted or deleted due to hordware crash, orpower failure, usually the error is like folowing: hsphere=# select * from TABLE; ERROR: could not access status of transaction 1918986094 DETAIL: could not open file "/var/lib/pgsql/data/pg_clog/0726": No such file or directory ... "The needed file should be reacreated manually with blanked inside, but filesize should be the same"
pg_clog and pg_xlog empty, postgresql refuses to start - PostgreSQL Database¶
- page's self-description: pg_clog and pg_xlog empty, postgresql refuses to start. PostgreSQL Database Forums on Bytes. "I'd put back the clog files if at all possible. If not you can substitute zero-filled 256K files for each clog file the server complains about not finding, but realize that you *will* be losing transactions in whole or in part." "As for pg_xlog, you can cons up dummy xlog contents with pg_resetxlog, but again you'd be better off with the real thing. The dummy files will not be capable of replaying whatever happened since your last checkpoint."
- "To prepare for low level standalone hot backups, set wal_level to archive (or hot_standby), archive_mode to on, and set up an archive_command that performs archiving only when a switch file exists." "When wal_level is minimal some SQL commands are optimized to avoid WAL logging" this is really the only reason that wal_level must be archive, since the server normally keeps around lots of WAL segments anyway. "The file identified by pg_stop_backup's result is the last segment that is required to form a complete set of backup files" * actually, this is only needed if parts of base/ were copied *after* the backup finishes copying pg_xlog/. most backup mechanisms copy in alphabetical order, so that pg_xlog/ will be the last subdir copied, and the last segment in it will be copied after base/ is already done.
- "But minimal WAL does not contain enough information to reconstruct the data from a base backup and the WAL logs, so either archive or hot_standby level must be used to enable WAL archiving (archive_mode) and streaming replication." "In many situations, turning off synchronous_commit for noncritical transactions can provide much of the potential performance benefit of turning off fsync, without the attendant risks of data corruption." "it is only advisable to turn off fsync if you can easily recreate your entire database from external data" (which is the case for us, actually, but we currently turn off synchronous_commit instead) ** note that turning fsync on protects against power failure, but not hard disk failure or human error. for that, you need continuous archiving or point-in-time recovery, respectively. "In hot_standby level, the same information is logged as with archive, plus information needed to reconstruct the status of running transactions from the WAL. [...] It is thought that there is little measurable difference in performance between using hot_standby and archive levels" ie, if you are going to use archive anyway, might as well go straight to hot_standby
- "The database server must be shut down in order to get a usable backup. Half-way measures such as disallowing all connections will not work (in part [...] because of internal buffering within the server)." ** in practice, this is not necessarily the case. it is entirely possible to get a usable filesystem-level backup, using the approaches described at http://vegpath.org/links/#backups . this is important because filesystem-level backups have several significant advantages over traditional database-level backups with pg_dump: 1) only changed files will be backed up each time, rather than the entire database 2) no CPU time needs to be spent on intensive compression of the data 3) most importantly, the backup is instantaneously restorable simply by swapping the backup and primary database directories. by comparison, a pg_dump backup can take many hours to restore. in general, to get a consistent filesystem-level backup, you need a WAL chain that's at least long enough (time-wise) for the WAL blocks created when it starts copying base/ to still be around when it finishes copying pg_xlog/ (it must also copy in alphabetical order, and not in parallel)
- "wal_keep_segments (integer) Specifies the minimum number of past log file segments kept in the pg_xlog directory"
Postgres 9.3 feature highlight: parallel pg_dump¶
- page's self-description: Postgres 9.3 feature highlight: parallel pg_dump
- "A checkpoint is a point in the transaction log sequence at which all data files have been updated to reflect the information in the log"
disk space
quotas
PostgreSQL - admin - Best way to limit database sizes¶
- page's self-description: Best way to limit database sizes. Hi, I have a single Postgres server which will be hosting multiple databases belonging to different users. What would be the recommendation to limit the size... "put each tablespace on a LVM partition of a control sized, extensible" "But LVM/tablespace should work" "How about a trigger that for every say 1000 or 10000 update or inserts (or timely if cron job) will use pg_stats or pg_statistics to decide if a role (user) privileges should be modified to read only (or not)"
PostgreSQL: Disk space not released after TRUNCATE - Database Administrators¶
[#JENA-28] Drop Temporary Tables in Postgres (PATCH) - ASF JIRA¶
Re: ERROR: could not write block 196261 of temporary file: No space left¶
PostgreSQL: PostgreSQL: In place UPDATE¶
disk suddenly full, postgres using all space - Untangle Forums¶
- page's self-description: A customer box that's never used more than a few gigs of space suddenly ran out of disk space so bad the captive portal page wouldn't load properly.
doc
- - *** "A common error is to label a function IMMUTABLE when its results depend on a configuration parameter. For example, a function that manipulates timestamps might well have results that depend on the TimeZone setting. For safety, such functions should be labeled STABLE instead." this especially includes functions that depend on the search_path! (ie. that don't have schema qualifiers on all invoked functions.) however, this effect will only be noticeable if the function is called on only *constant* values in a *PL/pgSQL* function, in which case the wrong search_path (the one in effect at *create* time) will be used. ** "There is relatively little difference between STABLE and IMMUTABLE categories when considering simple interactive queries that are planned and immediately executed: it doesn't matter a lot whether a function is executed once during planning or once during query execution startup. But there is a big difference if the plan is saved and reused later. Labeling a function IMMUTABLE when it really isn't might allow it to be prematurely folded to a constant during planning, resulting in a stale value being re-used during subsequent uses of the plan. This is a hazard when using prepared statements or when using function languages that cache plans (such as PL/pgSQL)." ie. if you are going to use your STABLE function in a PL/pgSQL function, be sure to label it STABLE so that it does not get constant-folded until call time (if labeled IMMUTABLE, it would be folded right away when the function is created)
- - **** "join_collapse_limit (integer) The planner will rewrite explicit JOIN constructs (except FULL JOINs) into lists of FROM items whenever a list of no more than this many items would result. Smaller values reduce planning time but might yield inferior query plans." you definitely want to turn this *off* in all cases! joins are written in a particular order for a reason (because the programmer knows which indexes are available and therefore which order is correct). the query planner is not smarter than the programmer! incorrect query plans are an ongoing bug in Postgres, because it does not support index hints and by default does not follow the join order. specifically, Postgres often does the following things in query plans which should normally never be done: * performs a sequential scan when an index is available (because it incorrectly thinks there are too many dead rows in the index, or because it incorrectly assumes the result of a nested SELECT (used as an index key) will cover most of the rows in the table) * uses an in-memory hash join when an index is available (because it incorrectly thinks that accessing an index on disk will be too slow) * sorts *both* input tables before performing a merge join (if really no index is available, a hash join is much faster than two sorts and a merge join) * does the entire join sequence in the opposite order (because it incorrectly thinks that the last table in the join will be filtered to fewer rows than the first table, and ignores the programmer's join order) * joins a pair of tables in the wrong order (the smaller table should be iterated over, with an index or hash lookup being used for the larger table, not vice versa) for these reasons, the following must always be off: SET enable_seqscan = off; for queries that use an explicit join order, they will often be faster if you turn off join_collapse_limit to force using your join order. however, if you do this, you may need to turn off other options as well: SET join_collapse_limit = 1; -- turn it off SET enable_sort = off; SET enable_hashjoin = off; -- better to keep this on if possible SET enable_mergejoin = off; fortunately, the combination of all of these is usually sufficient to get Postgres to use the right query plan.
- "As an exception, if the USING clause does not change the column contents and the old type is either binary coercible to the new type or an unconstrained domain over the new type, a table rewrite is not needed, but any indexes on the affected columns must still be rebuilt" "If the constraint is marked NOT VALID, the potentially-lengthy initial check to verify that all rows in the table satisfy the constraint is skipped. *The constraint will still be enforced against subsequent inserts or updates* (that is, they'll fail unless there is a matching row in the referenced table)." [+emph] it has been experimentally verified that the constraint *is* enforced against new rows, so this unfortunately *cannot* be used to temporarily disable an fkey "VALIDATE CONSTRAINT This form validates a foreign key constraint that was previously created as NOT VALID, by scanning the table to ensure there are no unmatched rows. Nothing happens if the constraint is already marked valid." "Disabling or enabling internally generated constraint triggers requires superuser privileges" if you try to disable a foreign key constraint's trigger, you will get an error, even if it is marked NOT VALID: "ERROR: permission denied: "RI_ConstraintTrigger_..." is a system trigger SQL state: 42501" "To force an immediate rewrite of the table, you can use VACUUM FULL, CLUSTER or one of the forms of ALTER TABLE that forces a rewrite. This results in no semantically-visible change in the table, but gets rid of no-longer-useful data."
- "the autovacuum daemon (see Section 23.1.5) takes care of automatic analyzing of tables when they are first loaded with data, and as they change throughout regular operation"
- "In situations where the best plan depends strongly on the parameter values, EXECUTE can be significantly faster; while when the plan is not sensitive to parameter values, re-planning will be a waste."
- "EXCLUDE USING gist (c WITH &&)"
- - text format: "Backslash characters (\) can be used in the COPY data to quote data characters that might otherwise be taken as row or column delimiters." "the following characters must be preceded by a backslash if they appear as part of a column value: backslash itself, newline, carriage return, and the current delimiter character." "special backslash sequences are recognized by COPY FROM" (only in text mode) "The following special backslash sequences are recognized by COPY FROM: [...] \digits Backslash followed by one to three octal digits specifies the character with that numeric code " this includes \0 CSV format: "any occurrence within the value of a QUOTE character or the ESCAPE character is preceded by the escape character" with ESCAPE=\ : \->\\, "->\" with ESCAPE=" : "->""
- "SET FROM CURRENT saves the session's current value of the parameter as the value to be applied when the function is entered." This is referring to the session when the function is *defined*, not when it's called.
- "Only B-tree currently supports unique indexes" but see exclusion constraints (http://www.postgresql.org/docs/9.1/static/sql-createtable.html#SQL-CREATETABLE-EXCLUDE) for a way to do this for gist indexes
- "The operator name is a sequence of up to NAMEDATALEN-1 (63 by default) characters from the following list: + - * / < > = ~ ! @ # % ^ & | ` ? "
- - an operator class is equivalent to a comparator in other programming languages (with the exception that the same comparator cannot be used for both trees and hashes)
- "You can create the illusion of an updatable view by defining ON INSERT, ON UPDATE, and ON DELETE rules (or any subset of those that's sufficient for your purposes) to replace update actions on the view with appropriate updates on other tables. If you want to support INSERT RETURNING and so on, then be sure to put a suitable RETURNING clause into each of these rules. Alternatively, an updatable view can be implemented using INSTEAD OF triggers (see CREATE TRIGGER)."
- "The access method must support amgettuple (see Chapter 52); at present this means GIN cannot be used. [...] So in practice the access method will always be GiST."
- "a constraint trigger. This is the same as a regular trigger except that the timing of the trigger firing can be adjusted using SET CONSTRAINTS" "A column-specific trigger (one defined using the UPDATE OF column_name syntax) will fire when any of its columns are listed as targets in the UPDATE command's SET list. It is possible for a column's value to change even when the trigger is not fired, because changes made to the row's contents by BEFORE UPDATE triggers are not considered. Conversely, a command such as UPDATE ... SET x = x ... will fire a trigger on column x, even though the column's value did not change." "For UPDATE events, it is possible to specify a list of columns" "It is possible for a column's value to change even when the trigger is not fired, because changes made to the row's contents by BEFORE UPDATE triggers are not considered"
- "The query planner will consider using a hash index whenever an indexed column is involved in a comparison using the = operator"
- "A variant of quoted identifiers allows including escaped Unicode characters identified by their code points. This variant starts with U& (upper or lower case U followed by ampersand) immediately before the opening double quote, without any spaces in between, for example U&"foo"." "the OPERATOR syntax, as for example in: SELECT 3 OPERATOR(pg_catalog.+) 4; "
- "When NOTIFY is used to signal the occurrence of changes to a particular table, a useful programming technique is to put the NOTIFY in a rule that is triggered by table updates. In this way, notification happens automatically when the table is changed, and the application programmer cannot accidentally forget to do it."
- "PostgreSQL does not require constraint names to be unique within a schema (but only per-table)" this is INCORRECT: UNIQUE constraints must be globally unique because their implicit indexes must be globally unique. trying to create an inter-table duplicate UNIQUE constraint gives the error: -- NOTICE: CREATE TABLE / UNIQUE will create implicit index "x" for table "b" ERROR: relation "x" already exists --
- - includes set_config(), current_setting()
- "current_schemas(boolean) returns an array of the names of all schemas presently in the search path. The Boolean option determines whether or not implicitly included system schemas such as pg_catalog are included in the returned search path." "In Unix parlance, the session user is the "real user" and the current user is the "effective user"."
- "`SELECT has_table_privilege('joe', 'mytable', 'INSERT, SELECT WITH GRANT OPTION');`"
- "PG_EXCEPTION_DETAIL text the text of the exception's detail message, if any" " GET STACKED DIAGNOSTICS text_var1 = MESSAGE_TEXT, text_var2 = PG_EXCEPTION_DETAIL, text_var3 = PG_EXCEPTION_HINT; "
- "statement_timeout (integer) Abort any statement that takes more than the specified number of milliseconds, starting from the time the command arrives at the server from the client."
- "Simple views are automatically updatable: the system will allow INSERT, UPDATE and DELETE statements to be used on the view in the same way as on a regular table." (this is only available in 9.3+)
- "Valid values are TERSE, DEFAULT, and VERBOSE, each adding more fields to displayed messages. TERSE excludes the logging of DETAIL, HINT, QUERY, and CONTEXT error information. VERBOSE output includes the SQLSTATE error code (see also Appendix A) and the source code file name, function name, and line number that generated the error. Only superusers can change this setting [but it also exists in psql]." to hide stack traces in psql: \set VERBOSITY terse there is unfortunately *no* option to do this in pgAdmin
- "client_min_messages [...] Valid values are DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1, LOG, NOTICE, WARNING, ERROR, FATAL, and PANIC. Each level includes all the levels that follow it."
- "Note that you should also create an index with the default operator class if you want queries involving ordinary <, <=, >, or >= comparisons to use an index. Such queries cannot use the xxx_pattern_ops operator classes. (Ordinary equality comparisons can use these operator classes, however.)"
- "If newline-sensitive matching is specified, . and bracket expressions using ^ will never match the newline character (so that matches will never cross newlines unless the RE explicitly arranges it) and ^ and $ will match the empty string after and before a newline respectively" "non-newline-sensitive matching (default)" "If partial newline-sensitive matching is specified, this affects . and bracket expressions as with newline-sensitive matching, but not ^ and $." this is the (nonintuitive) default in most other regexp implementations "If inverse partial newline-sensitive matching is specified, this affects ^ and $ as with newline-sensitive matching, but not . and bracket expressions. This isn't very useful but is provided for symmetry."
- "--jobs [...] You can only use this option with the directory output format because this is the only output format where multiple processes can write their data at the same time. [...] Requesting exclusive locks on database objects while running a parallel dump could cause the dump to fail. [...] For a consistent backup, the database server needs to support synchronized snapshots, a feature that was introduced in PostgreSQL 9.2. With this feature, database clients can ensure they see the same data set even though they use different connections. pg_dump -j uses multiple database connections; it connects to the database once with the master process and once again for each worker job. Without the synchronized snapshot feature, the different worker jobs wouldn't be guaranteed to see the same data in each connection, which could lead to an inconsistent backup. "
- "--globals-only Dump only global objects (roles and tablespaces), no databases. "
- "For some types of errors, the server reports the name of a database object (a table, table column, data type, or constraint) associated with the error; for example, the name of the unique constraint that caused a unique_violation error. Such names are supplied in separate fields of the error report message so that applications need not try to extract them from the possibly-localized human-readable text of the message. As of PostgreSQL 9.3, complete coverage for this feature exists only for errors in SQLSTATE class 23 (integrity constraint violation), but this is likely to be expanded in future." this is especially useful for error parsing in column-based import [wiki.vegpath.org/Column-based_import] (and may even have been motivated by it!)
- "The default value is defined as the value that the parameter would have had, if no SET had ever been issued for it in the current session. The actual source of this value might be a compiled-in default, *the configuration file* [+emph]" ie., *not* the compiled default
- "During the execution of VACUUM and ANALYZE commands, the system maintains an internal counter that keeps track of the estimated cost of the various I/O operations that are performed. When the accumulated cost reaches a limit (specified by vacuum_cost_limit), the process performing the operation will sleep for a short period of time, as specified by vacuum_cost_delay. Then it will reset the counter and continue execution." "This feature is disabled by default for manually issued VACUUM commands." however, *autovacuum*_vacuum_cost_delay is *enabled* by default
- "pg_catalog is always effectively part of the search path. If it is not named explicitly in the path then it is implicitly searched before searching the path's schemas. This ensures that built-in names will always be findable. However, you can explicitly place pg_catalog at the end of your search path if you prefer to have user-defined names override built-in names."
- "ctid The physical location of the row version within its table. Note that although the ctid can be used to locate the row version very quickly, a row's ctid will change if it is updated or moved by VACUUM FULL. Therefore ctid is useless as a long-term row identifier [unless your DB is select-optimized and rarely updated]. The OID, or even better a user-defined serial number, should be used to identify logical rows." "tableoid The OID of the table containing this row. This column is particularly handy for queries that select from inheritance hierarchies (see Section 5.8), since without it, it's difficult to tell which individual table a row came from. The tableoid can be joined against the oid column of pg_class to obtain the table name."
from/to MySQL
Converting from other Databases to PostgreSQL - PostgreSQL wiki¶
Converting MySQL to PostgreSQL - Wikibooks, open books for an open world¶
my2pg.pl¶
MySQL vs PostgreSQL - WikiVS¶
- "PostgreSQL will feature the MERGE clause in a future version [5], which follows the SQL:2008 standard. This can do the same as MySQL's non-standard 'INSERT IGNORE','REPLACE' and 'INSERT ... ON DUPLICATE UPDATE' statements but with more granularity." "MySQL triggers are also not activated by cascading updates and deletes even when caused by a SQL statement"
hash indexes
"Hash index" vs. "b-tree index" (PostgreSQL 8.0) - Grokbase¶
- page's self-description: (33 replies) Greetings, We are working on speeding up the queries by creating indexes. We have queries with searching criteria such as "select ... where *col1='...'*". This is a simple query with only "=" operation. As a result I setup hash index on column "col1". While, in postgreSQL 8 doc, it is wirttern: *Note: * Testing has shown PostgreSQL's hash indexes to perform no better than B-tree indexes, and the index size and build time for hash indexes is much worse. For these reasons, hash index use is
* sql - Postgres hashing user defined types - Stack Overflow¶
- "ERROR: data type urlpair has no default operator class for access method "hash"" ie. a user-defined type needs a special `hash` operator class
- "A probe into an in-memory hashtable is a lot cheaper than a probe into an index, so this type of plan makes plenty of sense if the hashtable will fit in RAM and there are going to be a lot of probes"
select * from depesz; » Blog Archive » Should you use HASH index?¶
- "It should be noted that as of version 8.4 the issue with Hash indexes being less efficient and slower than b-tree indexes is resolved"
installing 9.3
- after installing, you need to restore config values that the upgrade reset: - in pgAdmin > Preferences > Query tool > Query editor, set Max characters per column back to -1 (to avoid cells being truncated). this is *not* a bug in PostgreSQL, only in pgAdmin, and does *not* signal a need to downgrade.
- page's self-description: 9.3beta2 unrecognized node type in join alias vars when retrieving view definition. Hi all, While testing the 9.3 beta I am unable to get the view definition for some views. Because of this I...
- - on Linux: $ sudo apt-get --yes install postgresql-9.3 postgresql-contrib-9.3 postgresql-plpython-9.3 postgresql-plpython3-9.3 libpq-dev postgresql-9.3-postgis phppgadmin
installing postgresql 9.3.2 on ubuntu 12.04 . with error fix: ummet dependencies¶
- page's self-description: installing postgresql 9.3.2 on ubuntu 12.04 . with error fix: ummet dependencies - Gist is a simple way to share snippets of text and code with others.
upgrading DB to 9.3
- - on Linux: $ sudo -u postgres /usr/lib/postgresql/9.3/bin/initdb -D /var/lib/postgresql/9.3/main/ -E UTF8 --lc-collate=en_US.UTF-8 on Mac: $ sudo chmod go+w /Library/PostgreSQL/9.3/ $ sudo -u postgres initdb -D /Library/PostgreSQL/9.3/data/ -E UTF8 --lc-collate=en_US.UTF-8 $ sudo chmod go-w /Library/PostgreSQL/9.3/
- - on Linux: $ sd ln -s postgis-2.1.so /usr/lib/postgresql/9.3/lib/postgis-2.0.so
- - on Linux: $ sudo service postgresql stop $ for i in /etc/postgresql/9.1/main/*.conf*; do sudo ln -sf "$i" /var/lib/postgresql/9.1/main/; done $ for i in /etc/postgresql/9.3/main/*.conf*; do sudo ln -sf "$i" /var/lib/postgresql/9.3/main/; done $ sudo -u postgres -s $ mkdir /tmp/postgres_install $ cd /tmp/postgres_install $ /usr/lib/postgresql/9.3/bin/pg_upgrade --user=postgres -b /usr/lib/postgresql/9.1/bin/ -B /usr/lib/postgresql/9.3/bin/ -d /var/lib/postgresql/9.1/main/ -D /var/lib/postgresql/9.3/main/ # runtime: 1 h ("04:47:01" - ~"05:50") for a 312 GB cluster (staging tables+import) $ sudo service postgresql start $ sudo -u postgres psql -p 5433 # try connecting to the new DB $ sudo apt-get remove postgresql-9.1 $ ./delete_old_cluster.sh $ ./analyze_new_cluster.sh # runtime: 30 min ("31m6.298s") on Mac: # stop the PostgreSQL server $ sudo -u postgres -s $ mkdir /tmp/postgres_install $ cd /tmp/postgres_install $ pg_upgrade --user=postgres -b /Library/PostgreSQL/9.1/bin/ -B /Library/PostgreSQL/9.3/bin/ -d /Library/PostgreSQL/9.1/data/ -D /Library/PostgreSQL/9.3/data/ # runtime: 15 min ("06:28:35" - "06:14:41") for a 30 GB cluster (staging tables only) # start the PostgreSQL server # try connecting to the new DB $ ./delete_old_cluster.sh $ ./analyze_new_cluster.sh # runtime: 30 min ("07:10:59" - "06:40:23") for a 30 GB cluster (staging tables only)
Psycopg2 on Mac OS X (Snow Leopard) - Google Groups¶
- page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations. on Mac: '(2) Create symbolic links in /usr/lib : sudo ln -s /Library/PostgreSQL/9.3/lib/libpq.5.dylib /usr/lib/libpq.5.dylib sudo ln -s /Library/PostgreSQL/9.3/lib/libssl.1.0.0.dylib /usr/lib/libssl.1.0.0.dylib sudo ln -s /Library/PostgreSQL/9.3/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.1.0.0.dylib '
invalid byte sequence errors
- page's self-description: (6 replies) Hi, I've got another problem. I sometimes get the following SQLException when doing an insert: ERROR: invalid byte sequence for encoding "UTF8": 0x00 Exception: org.postgresql.util.PSQLException org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1525) org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1309) org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:188)
- page's self-description: Making TEXT NUL-transparent. Occasionally, we get bitten by embedded NUL bytes in TEXT values. We take care of generating proper UTF-8, but this additional restriction sometimes slips by. It would... - Lack of support for NUL bytes is a critical limitation of PostgreSQL when dealing with arbitrary input data which should be stored verbatim
oids
- "SERIAL: Pros: [...] * Easier to reproduce all or part of a db on a dump/restore " the oid value cannot be set by the user, and therefore a table with an fkey to an oid column is **not reimportable**
- PostgreSQL - OID System Column - Table with OIDs (Identity, Autoincrement) - SQLines¶
- page's self-description: OID is auto-incrementing integer value, unique within a PostgreSQL database (not just a table) that can be automatically assigned to each row of a table created WITH OIDS option. Although OID can be used as an identity (auto-increment) primary key column, it is recommended to use SERIAL data type instead. For more information, see Generating IDs in PostgreSQL. "Uniqueness Across all tables until the max value is reached" (which happens after "232-1 (4 billion)" rows)
PostGIS
- Installing PostGIS on Ubuntu¶
PostgreSQL [KyngChaos Wiki]¶
- page's self-description: Mac OS X installers for PostgreSQL, and GIS extensions PostGIS and pgRouting. Built for Lion and Snow Leopard, 64bit only. NOTE: I do not support any other Postgres distribution in my extensions. Please do not try to install PostGIS in another Postgres and then ask why it won't install.
postgresql - Struggling with enabling a PostGIS database on Mac - GIS¶
recovery mode
because of Linux out-of-memory (OOM) killer
- "the Linux out-of-memory (OOM) killer triggers and terminates the process. In the latter case you will see reports of OOM killer activity in the kernel's dmesg output. I expect this is what you'll see in your case. PostgreSQL servers should be configured without virtual memory overcommit so that the OOM killer does not run and PostgreSQL can handle out-of-memory conditions its self. See the PostgreSQL documentation on Linux memory overcommit."
- "Could this be the linux kernel randomly killing processes under heavy load issue?" "Here's the logs: [...] NOTICE: Message from PostgreSQL backend: The Postmaster has informed me that some other backend died abnormally and possibly corrupted shared memory. " our logs show this problem, too: " 2013-12-02 23:51:11 PST LOG: server process (PID 22312) was terminated by signal 9: Killed 2013-12-02 23:51:11 PST DETAIL: Failed process was running: /*NCBI.nodes*/ INSERT INTO "_taxonlabel_set_parent_id(parent_id=taxonlabel_pkeys.out.taxonl" ("taxonID", "result") SELECT * FROM "pg_temp"."_taxonlabel_set_parent_id(parent_id=taxonlabel_.out.taxonl_wrap"() /* EXPLAIN: Function Scan on "_taxonlabel_set_parent_id(parent_id=taxonlabel_.out.taxonl_wrap" (cost=0.25..10.25 rows=1000 width=8) */ 2013-12-02 23:51:11 PST LOG: terminating any other active server processes 2013-12-02 23:51:11 PST WARNING: terminating connection because of crash of another server process 2013-12-02 23:51:11 PST DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2013-12-02 23:51:11 PST HINT: In a moment you should be able to reconnect to the database and repeat your command. 2013-12-02 23:51:11 PST WARNING: terminating connection because of crash of another server process 2013-12-02 23:51:11 PST DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2013-12-02 23:51:11 PST HINT: In a moment you should be able to reconnect to the database and repeat your command. 2013-12-02 23:51:11 PST WARNING: terminating connection because of crash of another server process 2013-12-02 23:51:11 PST DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2013-12-02 23:51:11 PST HINT: In a moment you should be able to reconnect to the database and repeat your command. 2013-12-02 23:51:12 PST LOG: all server processes terminated; reinitializing 2013-12-02 23:51:14 PST FATAL: the database system is in recovery mode 2013-12-02 23:51:14 PST LOG: database system was interrupted; last known up at 2013-12-02 23:50:28 PST 2013-12-02 23:51:14 PST LOG: database system was not properly shut down; automatic recovery in progress 2013-12-02 23:51:14 PST FATAL: the database system is in recovery mode 2013-12-02 23:51:14 PST LOG: redo starts at 1DDB/54FA3B30 "
- "the kernel uses signal 9, not 11, when it wants to kill something" (!!!) apparently, the kernel doesn't even give the process a chance to shut down cleanly
How to Configure the Linux Out of Memory Killer¶
- page's self-description: This article describes the Linux out-of-memory (OOM) killer and how to find out why it killed a particular process. It also provides methods for configuring the OOM killer to better suit the needs of many different environments. "We can set valid ranges for oom_adj from -16 to +15, and a setting of -17 exempts a process entirely from the OOM killer. The higher the number, the more likely our process will be selected for termination if the system encounters an OOM condition" "The contents of /proc/2592/oom_score can also be viewed to determine how likely a process is to be killed by the OOM killer. A score of 0 is an indication that our process is exempt from the OOM killer. The higher the OOM score, the more likely a process will be killed in an OOM condition."
- "unless you had extremely high settings for both checkpoint_segments and checkpoint_timeout, it shouldn't take an hour to recover from a crash. Does it appear that the startup subprocess is making progress at all? (Use "ps" to find the postmaster's startup process child, then see if it's doing anything using "strace" or some such.)"
SHOW GRANTS FOR
- page's self-description: :"SHOW GRANTS FOR username" or why \z is not enough for me. Hello As a newbie Postgres admin I like to double check that my users have all necessary rights and more important only those and... " >I somehow think that there must be a more elegant solution > like "SHOW GRANTS FOR foo" in MySQL. Any ideas? has_table_privilege() and sibling functions might help you with that "
table modified time
TIDs
- page's self-description: Using ctid column changes plan drastically. Hi, I was testing a query to delete duplicates to see how well using ctid works if the table doesn't have a unique identifier available. The table... "Joins on tid columns just aren't supported very well at the moment. [...] There are gotchas if any of the rows receive concurrent updates." (Tom Lane) "type tid doesn't have any hash support" (Tom Lane)
- "ctid The physical location of the row version within its table. Note that although the ctid can be used to locate the row version very quickly, a row's ctid will change if it is updated or moved by VACUUM FULL. Therefore ctid is useless as a long-term row identifier [unless your DB is select-optimized and rarely updated]. The OID, or even better a user-defined serial number, should be used to identify logical rows."
- " Currently it is the fastest possible access to a single row. So an application that selects data and wants to update rows is optimized for PostgreSQL if it knows about that fact and qualifies the updates by ctid. "
- "It is scanning table by TupleID's. A tuple id is a 6-byte entity which consists of 4-byte page number and 2-byte tuple index inside page. So if you know the TID you can directly get the corresponding tuple."
Using the PostgreSQL System Columns | 2ndQuadrant¶
- "One thing you can use these for is finding duplicate data in a table. Let’s say you’re trying to add a unique constraint, but one row in the table is duplicated 3 times, which blocks the unique index from being created. When rows are identical in every column, you can’t write any simple SELECT statement to uniquely identify them. That means deleting all of them but one copy requires some annoying and fragile SQL code, combining DELETE with LIMIT and/or OFFSET–which is always scary. If you use the ctid instead, the implementation will be PostgreSQL specific, but it will also be faster and cleaner. See Deleting Duplicate Records in a Table for an example of how that can be done."
vacuuming
* postgresql - How to monitor or do vacuum without stop all? - Database Administrators Stack Exchange¶
- "It is safe to cancel the VACUUM FULL; doing so will undo all it's work, though"
- "ALTER TABLE .. SET DATA TYPE (relevant for 8.4 and below only) [...] This trick will not work in PostgreSQL versions 9.1 or later, as it detects that the change in data type is degenerate and so no rewrite is necessary." however, adding a USING clause could prevent this from being optimized away, allowing this to work in 9.1+ as well
- page's self-description: How can I tell if I'm autovacuuming?. From what I see in the docs, these three settings in postgresql.conf should be enough for PostgreSQL (8.2) to autovacuum with the default settings: autovacuum... "> (b) How can I verify that my databases are being vacuumed by autovacuum? pg_stat_user_tables should have a last_autovac column." "you can see autovacuum related messages like: DEBUG: autovacuum: processing database "postgres" DEBUG: autovacuum: processing database "template1" in your server log, by setting log_min_messages to 'debug1' in your postgresql.conf."
Don’t stop PostgreSQL’s autovacuum with your application | gocept weblog¶
- page's self-description: The problem Some weeks ago, we received a complaint from a customer about bad PostgreSQL performance for a specific application. I took a look into the database and found strange things going on: the query planner was executing "interesting" query plans, tables were bloated with lots of dead rows (one was 6 times as big… "Vacuuming (whether automatic or manual) stops at the oldest transaction id that is still in use." (as opposed to skipping past any active transactions to completed ones) "autovacuum will abort if it is not able to obtain a table lock within one second" "postgres[13251]: [40-1] user=,db= ERROR: canceling autovacuum task" "I have installed a nightly cron job to force-vacuum the database. PostgreSQL has shown much improved query responses since then. Some queries’ completion times even improved by a factor of 10."
- page's self-description: Lock problem with autovacuum truncating heap. We have run across a problem with autovacuum that occurs when it can truncate off a large amount of empty blocks. It behaves different in version 9.0... "In 9.0, the autovacuum process will be aborted about 1 second after another transaction starts waiting for a lock. The result is that even a simple INSERT will take 1 second." this means that autovacuum will never complete if clients keep making INSERTs :(
postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault¶
- "pg_stat_user_tables has columns *last_autovacuum* and *last_autoanalyze*. They will be updated whenever it does something."
» PostgreSQL – Change Default Encoding of New Databases To UTF-8 (Optional) Journal¶
- page's self-description: (33 replies) Hi, If i create a DOMAIN an then want to create a CAST from that domain to another type it gives an error. Consider this example: """ create domain datetime as timestamp with time zone check (value between '1753-01-01 00:00:00' and '9999-12-31 23:59:59'); create function datetime2int(datetime) returns int language sql stable strict as $$ select $1::date - '1753-01-01'::date; $$; create cast(datetime as int) with function datetime2int(datetime); """ if i try to cast, get this error: select
[SOLVED] PostgreSQL Can't Rename Table Constraints¶
- page's self-description: ' I had to rename a table yesterday due to a contact / group name change in the office and sadly I'm the only person who knows how to barely interact
- page's self-description: Determine the name of the calling function. Postgres 9.2 / pgAdmin III Linux / Windows XP Hello, Within a sub-function, I would like to programmatically determine the name of the... "the result of pg_exception_context clearly does contain the info you want"
** Robert Haas: Security Barrier Views¶
- "By creating a function with a very small cost, bob has tricked the optimizer into evaluating leak(text) before checking whether organization = 'CIA', and thus Tom's name gets spit out as a side effect while executing the query, even though it still doesn't show up in the query results"
/hw/hw1/student/Hw1/postgresql-8.0.3/src/backend/access/transam/xact.c – cs186 – Trac¶
#1018728: postgres NAMEDATALEN limitation create problems | drupal.org¶
A ragged bee in the bonnet - Andrew's PostgreSQL blog¶
How can I change 'shmmax'¶
- page's self-description: I have installed Oracle 9i on RedHat 9. Current value of 'shmmax' is 33554432. Some people recommend value of appr. 50% of the RAM. I tried to change the value of 'shmmax' in two ways: 1. cat 268435456 > /proc/sys/kernel/shmmax 2. sysctl -w kernel.shmmax=268435456
How Do I Enable remote access to PostgreSQL database server?¶
- page's self-description: A step by step guide for enabling remote access to your PostgreSQL database server under Linux or UNIX operating system so that client can connect server remotely.
ilines -- universal newlines from any data source « Python recipes « ActiveState Code¶
Maximum columns problem - dBforums¶
- page's self-description: Hi all, I have roughly 30,000 objects (tissue samples) for which I would like to store 1500 attributes for each sample. Each attribute is a numerical
MLA Wire: Postgresql: Indexes on Foreign Keys¶
MySQL and Postgres command equivalents (mysql vs psql) | End Point Blog¶
- page's self-description: End Point developers blog about ecommerce, databases, PostgreSQL, MySQL, Interchange, Ruby on Rails, Spree, Django, PHP, Android, hosting, Git, version control, Perl, Python, Java, and SEO.
New in PostgreSQL 9.3: New in Functions | Database Technologies¶
- page's self-description: In the series of blogging about new features in 9.3, today, I thought about blogging new functions and improvements coming in PostgreSQL. Lets look whats new in 9.3, in terms of in build functions: 1. New in one array functions for one dimensional array PostgreSQL 9.3, is coming with new functions which can help users… "5. Improvement in pg_get_viewdef() to print new line after each SELECT and FROM entry "
Omega Glory - PostgreSQL Slow Count Workaround¶
petere/plsh · GitHub¶
- page's self-description: plsh - PL/sh is a procedural language handler for PostgreSQL that allows you to write stored procedures in a shell of your choice.
pgAdmin Options - Tab 3 (Query)¶
- page's self-description: pgAdmin is the leading graphical Open Source management, development and administration tool for PostgreSQL, running on Windows, Linux, Solaris, FreeBSD and Mac OSX
- Postgres Session Variables – Neat. | An Oak In The Fall¶
- page's self-description: Postgres Session Variables – Neat.
- page's self-description: [GENERAL] Update on tables when the row doesn't change. Hi all, maybe it's a very silly question, but why does Postgres perform an update on the table even if no data changes? I recognized this... "Because testing for this would almost surely be a net loss for the vast majority of applications. Checking to see if the new row value exactly equals the old is hardly a zero-cost operation; if you pay that on every update, that's a lot of overhead that you are hoping to make back by sometimes avoiding the physical store of the new tuple."
- page's self-description: avoiding CONTEXT messages. Hi, I am using PostgreSQL 9.0 and I want to suppress CONTEXT part of error message for selected user defined Pl/PgSQL functions. Is this... 'in psql, you do it by setting: \set VERBOSITY terse'
- page's self-description: Quiet "CONTEXT"?. Hi all, I'm in the process of debugging some PL/pgSQL functions. I have function A which calls function B (e.g. on Line 22). The debugging info I'm interested in is in... "In psql, "\set VERBOSITY terse" might approximate what you want." (tom lane)
- page's self-description: Short-circuit boolean evaluation. Hi, Does Postgresql perform short-circuit boolean evaluation both in SQL and PL/pgSQL functions? As an example, suppose I have a function called "do_stuff" which... "SQL is a somewhat declarative language. There is no "order" to evaluation as such. So you can't talk about short circuiting either. This applies to any SQL database. You can somewhat enforce order with subselects and CASE and other such constructs."
- PostgreSQL - How to Create User-Defined Type (UDT) - SQLines¶
- page's self-description: You can create a user-defined type using CREATE DOMAIN and CREATE TYPE statements. Quick Example: -- Define a type using CREATE DOMAIN CREATE DOMAIN addr VARCHAR(90) NOT NULL DEFAULT 'N/A'; -- Define a type using CREATE TYPE CREATE TYPE address AS (city VARCHAR(90), street VARCHAR(90)); -- Use them in a table CREATE TABLE location (ship_address addr, full_address address);
- page's self-description: foreign key on pg_shadow. Dear list, Upon designing our application we thought that the following setup would be a good idea to implement security: Using the pg_shadow table as it is: |...
PostgreSQL 8.4 on Mac OS X 10.7 (Lion) | Jonathan Dean¶
- page's self-description: Web Software Engineer
- PostgreSQL and bloat¶
- page's self-description: /dev/dim
PostgreSQL on Mac OS X¶
PostgreSQL services Mac OS X Lion start/stop | blog.hersoncruz.com¶
PostgreSQL: assigning ROWNUM (row number) to each returned row | EXPLAIN EXTENDED¶
- page's self-description: How to create fast database queries
- page's self-description: (5 replies) The following command works fine when pasing it to psql via the -c option: cat event.csv | \ psql -c "COPY (event_id, event_name) FROM STDIN DELIMITER AS ',' NULL AS ''" When executed from a file via -f, it does nothing (no error messages either): event.sql: COPY (event_id, event_name) FROM STDIN DELIMITER AS ',' NULL AS '' cat event.csv | psql -f event.sql What's the problem? Many thanks in advance. -- Best Regards, Tarlika Elisabeth Schmitz
Psycopg¶
Re: generic options for explain¶
Re: Restoring right side of the display¶
- "you can easily allocate up to shared_buffers + (user_connections [~=max_connections] * work_mem)"
- "A probe into an in-memory hashtable is a lot cheaper than a probe into an index, so this type of plan makes plenty of sense if the hashtable will fit in RAM and there are going to be a lot of probes"
- "In PG your procedures aren't separate transactions"
Slow queries after convertinf DB from SQL_ASCII to UTF8 - Dev Shed¶
- page's self-description: Slow queries after convertinf DB from SQL_ASCII to UTF8- PostgreSQL Help. Visit Dev Shed to discuss Slow queries after convertinf DB from SQL_ASCII to UTF8 "my search queries take 5 times longer to give me results" [in UTF8 than in SQL_ASCII]
SourceForge.net: phpPgAdmin: Detail: 3072065 - "Auto login" doesn't work¶
- page's self-description: phpPgAdmin is a fully functional web-based administration utility for a PostgreSQL database server. It handles all the basic functionality as well as some advanced features such as triggers, views and functions (stored procs)
- "It should be noted that as of version 8.4 the issue with Hash indexes being less efficient and slower than b-tree indexes is resolved"
sql - What is the maximum number of columns in a PostgreSQL select query - Stack Overflow¶
- "if it is telling you the database does not exist, then the login was successful"
- page's self-description: About updateable views user ask many times. Is it supported in PostgreSQL? Can we write Complex updateable views? Answer for above is yes. Till 9.0, we have to use RULE for implementing updateable view. Again, RULE Implementation used to be a bit tidious, since user has to write multiple RULES to implement this feature. Following… "Till 9.0, we have to use RULE for implementing updateable view"
MySQL
`mysql`
- for interactive formatting (including echoed queries) when inputting from a file: (this format is slightly different than the actual interactive format because the queries are echoed with lines before and after) $ mysql --table --verbose --verbose <file for raw formatting when inputting from the console: (unfortunately does not display prompt) $ mysql --batch
doc
- "file_name cannot be an existing file" "The file is created on the server host, so you must have the FILE privilege to use this syntax" "if the MySQL client software is installed on the remote machine, you can instead use a client command such as mysql -e "SELECT ..." > file_name to generate the file on the client host"
- " Storage Engine Permissible Index Types MyISAM BTREE InnoDB BTREE MEMORY/HEAP HASH, BTREE NDB BTREE, HASH (see note in text) " note that HASH is *not* available for either MyISAM or InnoDB
- "For UNDEFINED, MySQL chooses which algorithm to use" "A reason to choose TEMPTABLE explicitly is that locks can be released on underlying tables after the temporary table has been created and before it is used to finish processing the statement"
- "For a view to be updatable, there must be a one-to-one relationship between the rows in the view and the rows in the underlying table. There are also certain other constructs that make a view nonupdatable" "The WITH CHECK OPTION clause can be given for an updatable view to prevent inserts or updates to rows except those for which the WHERE clause in the select_statement is true." this refers to views that include only a subset of rows (via WHERE), and requires rows inserted into it to be part of that subset. without this option, the WHERE clause is ignored when inserting.
- "Equivalent to PIPES_AS_CONCAT, ANSI_QUOTES, IGNORE_SPACE, NO_KEY_OPTIONS, NO_TABLE_OPTIONS, NO_FIELD_OPTIONS."
- - to add a user if you are not root but have sudo access: sudo service mysql stop sudo mysqld --skip-grant-tables --skip-networking & sudo mysql > INSERT INTO mysql.user (Host, User, Password) VALUES ('localhost', 'user', PASSWORD('password')); sudo kill "$(cat /var/run/mysqld/mysqld.pid)" sudo service mysql start mysql > GRANT SELECT, SHOW VIEW ON `db`.* TO 'user'@'localhost' WITH GRANT OPTION;
- "Because modifying the columns of the clustered index is an expensive operation, choose primary columns that are rarely or never updated."
dynamic SQL
MySQL – Writing dynamic SQL in stored procedures « IT Integrated Business Solutions¶
MySQL :: Re: Dynamic SQl in Triggers/Procdures¶
- "Stored procedures called from triggers operate under the same restrictions as triggers, i.e., no dynamic SQL is allowed."
enum
- 8 Reasons Why MySQL's ENUM Data Type Is Evil - Chris Komlenic¶
- page's self-description: The pros and (mostly) cons of MySQL's ENUM data-type (as compared to reference tables), as well as when it is okay to use and how to use it if you must. "With an ENUM we're actually storing pieces of data in a place that was only intended to hold crucial information about the model" "in large systems the effect of normalization can already tax the limits of human comprehension and complicate queries" "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered." [Donald Knuth]
- Kawoolutions - SQL Database Design - 8. User-Defined Types - 8.3 Enumerated Types¶
- page's self-description: Kawoolutions,,SQL Database Design,8. User-Defined Types,8.3 Enumerated Types "some people claim, that MySQL ENUMs are much faster than separate tables, especially on large databases. Furthermore, enums can give you some savings in database size, depending on the implementation. For example, a MySQL ENUM occupies only one byte for up to 255 values and two bytes for up to 65535 values."
missing
CHECK constraints
MySQL: How to do a (simple/complex) CHECK CONSTRAINT? « Christosoft Blog¶
- "you can execute it in MySQL without an error. But also without any effect whatsoever. I tried it in InnoDB (MySQL 5.5) and it did not check the constraint at all." "So we go with a trigger instead. For constraints, we use a BEFORE trigger."
phpmyadmin -- create table with 'check' function [Archive] - CodingForums.com¶
- page's self-description: [Archive] phpmyadmin -- create table with 'check' function MySQL "mysql itself does not support CHECK constraints"
MySQL Workbench
- MySQL :: Download MySQL Workbench¶
MySQL Bugs: #40167: no method to show hidden relationships again¶
exporting indexes separately
- this happens in /mysql-workbench-gpl-5.2.47-src/modules/db.mysql/src/module_db_mysql.cpp . it does *not* use mysqldump, unfortunately.
resetting ibdata
- you need to do this after manually renaming a database with InnoDB tables (but you should not manually rename a database with InnoDB tables, unless you have set innodb_file_per_table in /etc/mysql/my.cnf!)
- " to setup your server to use separate files for each table you need to change my.cnf in order to enable this: [mysqld] innodb_file_per_table "
string primary keys
- "we use varchar(32) for primary keys (GUIDs) and we haven't met performance issues of this. Our product is a web site with extreme overload and is critical to be stable. We use SQL Server 2005."
vs. other SQL DBs
PostgreSQL vs. MySQL: A Comparison of Speed, Integrity and Popularity » oBlog¶
- "MySQL is still the world's most popular open source database, with over 50,000 downloads per day. Its accessible attitude fostered high popularity and rapid growth of the MySQL community"
- The Codex » Do Not Pass This Way Again¶
- - arguments for MySQL: better UI (phpMyAdmin): "as a user, a company head, and a person who [...] has no time [for] front-ends [which were] ignored because the backend is *so amazingly awesome*" (may be referring to lack of sufficiently-good phpMyAdmin equivalents for other DBs?)
ariejan.net¶
- page's self-description: a blog by ariejan de vroom about software engineering and craftsmanship.
- "VARCHAR isn't a valid type for the CAST function, but CHAR is."
Creating a MySQL View directly in phpMyAdmin | Sky on Tech¶
Extend phpMyAdmin Session Timeout Value (Expired Session)¶
Getting out of MySQL Character Set Hell < Blue Box Blog¶
- page's self-description: MySQL Remote Access - A step-by-step guide to enable remote access to a MySQL database server under Linux, UNIX and BSD operating systems.
mysql - mysqldump returning unknown variable sql-mode error - Stack Overflow¶
MySQL – Extract or Restore a Single Table From a Huge mysqldump File¶
- page's self-description: A while back, I wrote an article that explained some awk magic for extracting the restore information for a single table from a mysqldump file. Since then I've created a Perl script that accomplishes the same thing, but is a little more flexible.
MySQL Lists: mysql: Re: Why does mysql drop index very very slow in a large table?¶
MySQLdb¶
SQLite
Implementation Limits For SQLite¶
- page's self-description: Redmine "SQLite takes nearly twice as long even though it doesn't have the precip records and is thus less than half the size."
- "The only solution I've found: [in trigger body] insert into test values (New.id, now()); select raise(ignore); " "No, you can't update NEW. What I tend to do is use a VIEW with an INSTEAD OF trigger"
SQLite Query Language: CREATE TRIGGER¶
XPath to SQL
http://publib.boulder.ibm.com/infocenter/iadthelp/v7r0/index.jsp?topic=/com.ibm.etools.logging.ui.doc.user/ref/rmapping_XPath_SQL.html¶
xpath to sql translation¶
- page's self-description: Are there any tools (java classes, tag libraries) which can translate xpath statements into a SQL query? Given an xpath query which has a predicate that filters node values or "I've never seen an Xpath to SELECT [] FROM [] WHERE clause translation. And it is not easy to do that, because you have to have rules in how you store xml in a relational database. I think that's also the reason there are no tools to do that." " We have found a couple of products on IBM's Alphaworks site. One of them uses the "DB2 XML Extenders" mapping of XML<=>SQL. The other seems to create a default mapping based on SQL metadata. Both of them are limited to Linux & Windows. While we may use these, our main RDB is DB2 for ISeries. So we are still checking. If you are curious, the links are: http://www.alphaworks.ibm.com/tech/xtable http://www.alphaworks.ibm.com/tech/db2uriaccess "
- "Recursive CTEs (or "recursive subquery factoring"[15] in Oracle lingo) can be used to traverse relations (as graphs or trees)" recursive queries function like queues: the 2nd half (after UNION ALL) fetches its input from the next row at the beginning of the queue, and appends its result row(s) to the end of the queue
semantic
How should a semantic inheritance chain look like? - ANSWERS¶
- page's self-description: During our work with a triple store we noticed a strange behaviour related to inheritance chaining and linking properties to classes. Consider the following example: 3 types defined - GrandParent, Parent and Child. 3 properties defined - GrandParentsProp, ParentsProp and ChildsProp. Parent inh... "The owl inference rule for domains is: if (A engineSize B) then because the domain of engineSize is Car, then A is a Car." but the same field name can have different meanings, depending on which class it's defined in
hardware
Intel Core i5 vs. AMD Phenom X4
- From experience, the 27-inch iMac's "2.66 GHz Intel Core i5" (System Information.app) is much faster than vegbiendev's 2.44 GHz "AMD Phenom(tm) 9550 Quad-Core" (/proc/cpuinfo).
Intel Core i5 3570K vs AMD Phenom X4 9550¶
- page's self-description: We put the 3.4 GHz 3570K to the test against the older 2.2 GHz 9550 to find out which you should buy, the Intel or the AMD. "CPUBoss recommends the Intel Core i5 3570K based on its performance, single-core performance and overclocking."
Does the magic trackpad work better than the magic mouse? - Apple Store (U.S.)¶
- page's self-description: Well, for me personally, I never unpacked my Magic Mouse after working with my Magic Trackpad for a few hours. However, that's subjective and merely my own opinion. I have read that some people like to use both, depending on what they do on their Mac's. Battery life is also not that great with the Trackpad. I purchased the Apple battery charg... "The problem with the trackpad is that you can't touch it, unless you want the mouse to move. You end up with your hand poised in the air above the trackpad - which as already tilted up - and this is actually much worse ergonomically than a mouse where your hand can rest on the mouse" this is less true for a laptop trackpad because you can move your fingers to the side more easily
What Is WiMAX Broadband Wireless Networking?¶
- page's self-description: WiMAX is the industry term for standard network technologies supporting long-distance wireless broadband. The WiMAX standard provides both fixed and mobile wireless connectivity options.
- page's self-description: Which of the popular wireless home networking standards like 802.11n or 802.11ac Wi-Fi is right for you? This article explains the pros and cons of each of these wireless standards.
Java
LDAP
LDAPv3 Distinguished Names¶
Appendix E: LDAP core.schema¶
OS
Unix
Linux
backups
Easy Automated Snapshot-Style Backups with Rsync¶
- page's self-description: How to make easy, automated snapshot-style backups with rsync and UNIX
- page's self-description: Is there a polished and working equivilent to Mac's time machine? I've googled and found some of the alternatives, and installed them and used them, ""Backintime" will work, but has a simple interface"
Taking a Backup Using Snapshots¶
- "A snapshot volume [...] presents all the data that was in the volume at the time the snapshot was created"
grub
- page's self-description: GNU GRUB Manual 0.97
- page's self-description: GNU GRUB Manual 0.97
- page's self-description: GNU GRUB Manual 0.97
- page's self-description: GNU GRUB Manual 0.97
system updates
[ubuntu] How can I uninstall / rollback the most recent update?¶
- page's self-description: I was prompted to install a host of updates last night, and one (or more?) have rendered my system unusable unless I boot to low graphics recovery mode (failsafex). The problem I'm seeing when I boot normally is the video rate is unbearably slow (~5 FPS and worse when moving windows). I suspect the problem maybe related to the SandyBridge chipset my laptop uses. Prior to the update 3d acceleration didn't work, and now 2d is also broken. I tried selecting 'previous version of linux' and "I believe you can roll back individual packages using Synaptic Package Manager"
Ubuntu
Ubuntu 14.04 upgrade
Apache
- **WARNING**: this breaks MultiViews redirects with the filename "index"; do not upgrade if you depend on that! after upgrading: # $ sudo a2dismod php5_cgi # $ sudo apache2ctl restart # for any redirects that use MultiViews to append an extension to files named "index", rewrite these to avoid using MultiViews
phpMyAdmin
- after upgrading: # $ sudo apt-get install phpmyadmin ## whenever asked which version of a file to use, enter Y # $ sudo apache2ctl restart # *must* be done or phpMyAdmin will display a blank page!
Postgres
- after upgrading, switch to the official Ubuntu version: # $ sudo rm /etc/apt/sources.list.d/pgdg.list # $ sudo apt-get update # $ sudo apt-get remove postgresql-9.3 ## **IMPORTANT**: make note of all the additional packages that will be removed # $ sudo apt-get install postgresql-9.3 # for each additional package displayed by `apt-get remove`: ## $ sudo apt-get install [package]
rsync
- **WARNING**: this breaks use of ~ to reference the remote user's home dir; do not upgrade if you depend on that! after upgrading: # for any remote paths that use ~, rewrite these to avoid using it
14 Things To Do After Installing Ubuntu 14.04¶
- page's self-description: Confused what to do after Installing Ubuntu 14.04? Here are some essential things to do after installing Ubuntu 14.04 Trusty Tahr.
- page's self-description: [ubuntu] On login 5 packages can be updated Server Platforms
- page's self-description: How does one remove the extra entries of the old linux kernels after upgrade of a new version. Remember Synaptic used to do that but it was the case when Startup manager used to do a good job. Please help if i'm missing something. this needs to be done periodically, because apt-get unfortunately does not do it automatically: $ sudo apt-get autoremove if that doesn't work: " dpkg-query -l | awk '/linux-image-*/ {print $2}' sudo apt-get --purge remove <Kernel> sudo update-grub "
Access Shared Folders in a VirtualBox Ubuntu 11.04 Virtual Machine¶
- page's self-description: Have you installed Ubuntu 11.04 as a virtual machine in VirtualBox but have had problems getting the Shared Folders feature to work? We were able to add a shared folder, but were unable to access it.
- Index of ftp://ftp.ucsb.edu/pub/mirrors/linux/ubuntu/¶
KVM/VirtManager - Community Ubuntu Documentation¶
- liblockfile 1.08, lockfile_8h.html¶
lockfile_create(3): manage lockfiles - Linux man page¶
- page's self-description: The lockfile_create function creates a lockfile in an NFS safe way.
- "You can use ppa-purge - install with sudo apt-get install ppa-purge"
- page's self-description: Hello, When you create a new user thats able to login etc etc, you also create there password for them, this obviously works with the users password
Mac
"App Store is temporarily unavailable" error
*** Impossible to perform software updates: Apple Support Communities¶
- "I had disabled Spotlight and although updates worked for many months, they stopped working. By re-enabling Spotlight, I was able to see OSX and App updates, download and install them." "My Spotlight index wasn't borked, it just wasn't there. I don't use spotlight so I keep it turned off. Odd that they would magically start requiring it to be turned on to perform updates." "It's the new paradigm. Updaates are tied to the Mac App Store which needs the Spotlight index to determine what's on your computer."
"The App Store is temporarily unavailable. Please try again - JustAnswer¶
- page's self-description: Question - "The App Store is temporarily unavailable. Please try again - 2R. Find the answer to this and other Mac questions on JustAnswer. "I can sign into the store if I open it from the Dock and sign in manually"
Another App Store Updates problem: Apple Support Communities¶
- "Dec 13, 2013" a recent problem "Fortunately, I am able to go to the command line and use the softwareupdate program and do my updates manually so I know that the problem is at the App Store leve."
10.8 Mountain Lion upgrade
- **WARNING**: **DO NOT** upgrade unless you are prepared to fix several programs broken by the upgrade: svn, Apache PHP, Apache ~/Sites dirs, PostgreSQL, Python psycopg2, Python OrderedDict, X11, Xcode instructions are in the corresponding subdirs. these programs will be COMPLETELY UNAVAILABLE until they are manually fixed!
whether to upgrade
- primary advantages of upgrading: Time Machine can now back up to multiple disks *at once*, allowing you to have multiple backups without needing to manually switch back and forth between the disks
Apple - OS X Mountain Lion - See everything the new OS X can do.¶
- page's self-description: Play iOS gamers in Game Center. Share right from your apps. Send iMessages. And more. Over 200 more features, in fact. All part of OS X Mountain Lion.
Should I Upgrade to Mountain Lion?¶
- page's self-description: Dear Lifehacker, Mountain Lion is out, and I'm seeing it has some pretty cool new features! Should I upgrade?
- "the installer takes 3 minutes before rebooting itself, and 33 minutes after rebooting to complete the installation on an iMac 11,2"
Xcode
- note: upgrading this requires a (free) App Store account also needed for svn
Mac App Store - Xcode¶
- page's self-description: Read reviews, get customer ratings, see screenshots and learn more about Xcode on the Mac App Store. Download Xcode and enjoy it on your Mac.
X11
XQuartz¶
svn
- note: upgrading this requires a (free) App Store account and a reinstall of Xcode
SVN Missing in Mac OS X 10.8 – Mountain Lion | Grapii¶
- page's self-description: Having recently upgraded to Mac OS X 10.8 - Mountain Lion, I was surprised to learn that SVN (Subversion) is not included in this release of the OS. This was a
Apache
PHP
- in /etc/apache2/httpd.conf : uncomment 'LoadModule php5_module libexec/apache2/libphp5.so'
~/Sites dirs
- in /etc/apache2/extra/httpd-userdir.conf : after 'UserDir Sites', add: --- <Directory /Users/*/Sites> AllowOverride all Order Deny,Allow Allow from all </Directory> ---
Python
psycopg2
- note: `ln -f` will overwrite the existing symlink to the previous libssl version $ sudo ln -fs /usr/lib/libssl.1.0.0.dylib /usr/lib/libssl.dylib $ sudo ln -fs /usr/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.dylib $ sudo easy_install psycopg2
postgresql - Update to latest libssl for Mountain Lion without MacPorts? - Ask Different¶
- " sudo ln -fs /usr/lib/libssl.1.0.0.dylib /usr/lib/libssl.dylib sudo ln -fs /usr/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.dylib "
OrderedDict
- change all scripts to import OrderedDict from collections instead of ordereddict
8.3. collections — High-performance container datatypes — Python v2.7.5 documentation¶
PostgreSQL
- after upgrading to 10.8, you need to modify PostgreSQL: change the postgres user to _postgres: $ sudo chown -R _postgres /Library/PostgreSQL/9.1/data/ $ sudo chown -R _postgres /Library/PostgreSQL/9.1/postgres.log $ sudo -u _postgres /Library/PostgreSQL/9.1/bin/pg_ctl -D /Library/PostgreSQL/9.1/data -l /Library/PostgreSQL/9.1/postgres.log restart
- "The Property List Editor has been dropped as a standalone application. It is now fully integrated in XCode"
10.9 Mavericks upgrade
- **WARNING**: **DO NOT** upgrade, it is much slower than the previous version according to Brad
whether to upgrade
- conclusion: *don't*, it breaks existing programs
After a month of use, we dish out our biggest gripes with OS X Mavericks | Ars Technica¶
- page's self-description: No new software is bug-free, and OS X 10.9 doesn't buck the trend. "When Mavericks' new responsive scrolling feature is working as designed, it draws sections of your window that aren't yet on-screen so that they show up more quickly when you scroll down (or up). [...] Unfortunately, the feature seems to have broken scrolling for other programs. I notice the problem most often in Chrome, Outlook 2011, and the Limechat IRC client [...] scrolling at higher speed becomes quite jerky and inconsistent" "The problem is not an app not making use of a new feature. It is that an OS upgrade is breaking functionality of a perfectly working application." "more recent MacBook Airs and the Retina MacBook Pros have integrated their power buttons into the keyboard, putting it up in the upper-right corner of the keyboard" "In Mavericks, pushing that power button automatically puts the computer to sleep, no questions asked. You don't need to push down on the button for any particular length of time, and there's no dialog box to ask you what you wanted to do when you pushed the button (or if you meant to push it at all). And unlike some of our multi-monitor gripes, there's no way to revert the button behavior back to its pre-Mavericks state." "Mavericks totally loses its mind if a network file share vanishes without being properly unmounted"
5 things to do before upgrading to OS X Mavericks | MacFixIt - CNET Reviews¶
- page's self-description: If you are considering installing OS X Mavericks, think about not only system requirements, but also your current OS configuration and any installed software you use. Read this article by Topher Kessler on CNET. "All too often, people upgrade their systems only to find that their programs now do not load, or regularly crash, only because they have not been updated" i.e. this means Mavericks will *break* existing programs "unplug your backups from your system to prevent them from potentially being tampered with during the OS installation"
30 days with OS X 10.9 'Mavericks' | ZDNet¶
- page's self-description: After 30 days with Apple's new operating system, I can sum up my experience in three words: It just works. "the new memory compression feature is helping to make switching between applications smoother, especially system resource heavy Adobe programs such as Photoshop, Lightroom, Premiere Pro and After Effects" "One of the things that I really like about OS X 10.9 is how it feels in many ways like 10.8." "Battery life Much better. I'd say that I'm getting an extra 45 minutes from a charge, which isn't bad at all."
MacFusion
- - defer_permissions: "if the file system retrieves file information from another computer and reports user/group IDs /as is/, the kernel will not see any alien IDs as belonging to the user that mounted the volume. (This can happen if user ID translation is not enabled or sometimes doesn't work with sshfs.) The defer_permissions (formerly defer_auth) option is useful in such cases. It causes MacFUSE to assume that all accesses are allowed"
- "-o defer_permissions"
- page's self-description: After having my Mac woken up from sleep I got the error message Could not mount filesystem: Remote host has disconnected. Which didn't exactly help me because the host was reachable fine with ssh. Investigating the log revealed this message: mount_fusefs: mount point /Volumes/username@hostname is itself on a MacFUSE volume A quick look into finder showed that the volume was still mounted while the connection was lost. Unmounting it solved this issue quickly, but I'm pretty sure this error can be ... "the volume was still mounted while the connection was lost. Unmounting it solved this issue quickly"
Stefan Hayden » MacFusion Error: Could not mount filesystem: Remote host has disconnected.¶
- page's self-description: Stefan Hayden - Graphic Artist, Web Designer, Blog Designer. Look for his yellow backpack in real life. (applies only to OSX 10.6)
quitting programs
- note that `kill -TERM` often does an *unclean* shutdown. to instead simulate File > Quit or clicking the red X in the corner: $ osascript -e 'tell application "<name (eg. Firefox)>" to quit'
recovery partition
Lion Recovery Update v1.0¶
sleep
wake for network access
- test using the following on another machine: (sleep 5; wakeonlan; ping <hostname>)
PrioritizeNetworkReachabilityOverSleep
Wake on Demand doesn't work - mactips - Helpful configuration tips for OS X users¶
- page's self-description: How can I configure the iMac to wake up when I try to connect to it from another computer?
- page's self-description:
The Wake on Demand feature of Mac OS X v10.6 or later, lets your Mac continue to share items (such as music, printers, files, or your screen) when it is asleep.
Access Your Computer Anytime and Save Energy with Wake-on-LAN¶
- page's self-description: You want access to your home computer wherever you are, whatever you're doing, whether that's via a remote desktop connection, SSH, FTP, web interface, or any other remote access you've set up. The catch is, you don't like throwing money away to an always-on system. Luckily you can have your digital cake and eat it, too, and today I'll show you how to boot and shut down your system remotely so that it's ready for you when you need it and it's not wasting energy when you don't.
pmset(1) Mac OS X Manual Page¶
- page's self-description: HTML versions of the OS X, OS X Server, Xcode Tools, and CHUD man pages. 'The -a, -b, -c, -u flags determine whether the settings apply to battery ( -b ), charger (wall power) ( -c ), UPS ( -u ) or all ( -a ).' 'autorestart - automatic restart on power loss' (needs to be set for UPS because the computer will always be in UPS mode at the time of a power failure) 'womp - wake on ethernet magic packet' 'ttyskeepawake - prevent idle system sleep when any tty (e.g. remote login session) is 'active'.' use `pmset -g custom` to verify settings
Wake a sleeping Mac from the network - Mac OS X Hints¶
- " Actually your python script also needs to set SO_BROADCAST: #!/usr/bin/env python import socket; s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1); s.sendto('\xff'*6+'\x00\x24\x36\xf0\xc4\x69'*16, ('10.0.0.255', 80)); " (note that this script does not actually work, although it should)
- SmartSleep¶
Why hibernate or 'safe sleep' mode is no longer necessary in OS X Lion (Updated) | TUAW - The Unofficial Apple Weblog¶
- page's self-description: Update: Several commenters expressed concern that disabling safe sleep could expose you to the possibility of drive corruption if you lose power or
Time Machine
How to restore data from Time Machine in Mountain Lion | Macworld¶
- page's self-description: If you use Time Machine regularly, you're ready for whatever technological mishaps life throws at you. Restoring data from Time Machine is just as easy as backing things up in the first place. "start up your Mac from the Mountain Lion recovery partition by pressing (and holding down) Command-R at startup. This launches Recovery Mode, a portion of your drive that Mountain Lion treats as a separate volume"
mac - How to make TimeMachine back up contents of any path or mounted volume - Super User¶
- "open the time machine settings file, found here: /Library/Preferences/com.apple.TimeMachine.plist"
- page's self-description:
If you want to switch to a different TIme Machine backup drive, use this article to learn how to transfer your existing Time Machine backups to the new backup drive, and then use it for regular Time Machine backups.
- Dan Bishop » How to mount SFTP/SSH shares in OS X NOW UPDATED to include OS X 10.8¶
- page's self-description: Learn how to prevent icons from bouncing in the Dock to get your attention
Flush DNS cache changed in Lion - MacRumors Forums¶
- page's self-description: Flush DNS cache changed in Lion Mac OS X 10.7 Lion "What works under 10.7 is: sudo killall -HUP mDNSResponder "
- page's self-description: Force every folder to be arranged by kind and sorted by name? Mac OS X 10.7 Lion
- "Use the xattr command" "you can also use the -c option to remove all extended attributes"
- page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations. see ~/Library/Preferences/com.apple.screensaver.plist > askForPasswordDelay (units: s)
How to Stop Icons From Bouncing | eHow.com¶
- page's self-description: How to Stop Icons From Bouncing. The dock is a distinctive feature of the Mac operating system. It keeps icons of commonly used and currently open programs visible at the bottom of your screen for quick access. When programs first open, or need to alert the user of some new update, the icons in the dock bounce. Many people find this helpful when...
- HOWTO: Fix file sorting in Finder’s column view on Mac OS X Lion « inner.geek¶
Install htop on Mac OS X - opinion, technology, culture, politics - themainframe.ca¶
Linux Time Capsule Server - Mac Guides¶
MacBook "Core 2 Duo" 2.0 13" (White/06) Specs (Late 2006, MA700LL/A, MacBook2,1, A1181, 2121) @ EveryMac.com¶
- page's self-description: Technical specifications for the MacBook "Core 2 Duo" 2.0 13" (White/06). Dates sold, processor type, memory info, hard drive details, price and more. "a 1280 by 800 native resolution" (example small-screen computer) visible browser area for the small-screen computer: 1203x716px
- page's self-description:
Learn how to reset (flush) the DNS cache.
osx - List all members of a group (Mac OS X) - Super User¶
Quick Look Helper process eating virtual memory on Mac - Operating Systems¶
- page's self-description: Quick Look Helper process eating virtual memory on Mac, Operating Systems, Computer end-user technical support troubleshooting for Windows, Mac, and Linux. Operating system tips and tweaks as well..
Will Time Machine backup portions of my file system that come from mounted drives? - Ask Different¶
- "you could have unintentionally told the system to secure and ignore this volume by having an obscurely documented empty file called no_log in /path/to/your/fs/.fseventsd/no_log"
\0
Brian Dessent - Re: OT: grep for \x00 = NUL¶
- " $ perl -ne 'print if m/\000/' or $ awk '/\000/ { print }' "
text processing - Multiline pattern match using sed, awk or grep - Unix and Linux¶
bash
** modifying a running shell script
- you need to first unlink the file, and *then* upload the modified version. in jEdit, this can be done by enabling two-stage save. `svn up` and rsync (without --inplace) use two-stage save automatically. to prevent this from occurring accidentally, always make all your shell scripts read-only. this will force applications to use two-stage save (where supported) to replace these files.
- "The reason deletion is safe but modification is not is that in Unix, a deleted file gets inaccessible from the file system, but the processes that had opened the file (here, /bin/bash) can still read it" so you can fix the problem by first unlinking the file, and *then* uploading the modified version
- "However, this would require that the running bash script is reading from the disk as it gets to each new step. Is this how it works?" yes, unfortunately incorrect: "bash will also check whether the source file is updated while the script is running. If it has been, bash will reload it and continue running it from the current position reopen the file, seek to the current position of the script, and continue running the script from that point" no, it just reads from disk: "mob's explanation is wrong, stating that it reads all at invocation and checks updates. See my answer to the question Emacs workflow to edit Bash scripts while they run [bookmarked above]" "if you need to prevent bash to reaload script you must to call it as stream."
doc
- page's self-description: Bash Reference Manual
- page's self-description: Bash Reference Manual "Aliases are expanded when a command is read, not when it is executed" "Aliases are expanded when a function definition is read, not when the function is executed, because a function definition is itself a compound command" note that if statements are also "compound commands", which have aliases expanded when their definition is read (i.e. at the beginning of the if statement, for the entire if statement), not when they are executed, and any aliases for function names in them need to be unaliased *before* the if statement containing the function definition. this is necessary even if the if statement is *not* executed, because the expansion of such aliases will cause syntax errors in the parsing of the if statement itself. "If the last character of the alias value is a space or tab character, then the next command word following the alias is also checked for alias expansion."
- page's self-description: Bash Reference Manual "BASH_COMMAND The command currently being executed or about to be executed, unless the shell is executing a command as the result of a trap, in which case it is the command executing at the time of the trap." useful with `trap cmd DEBUG`
- page's self-description: Bash Reference Manual "If GLOBIGNORE is set, each matching filename that also matches one of the patterns in GLOBIGNORE is removed from the list of matches. The filenames . and .. are always ignored when GLOBIGNORE is set and not null."
- page's self-description: Bash Reference Manual
- page's self-description: Bash Reference Manual -e: "The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test in an if statement, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command’s return status is being inverted with !. A trap on ERR, if set, is executed before the shell exits."
bash - How to run an alias in a shell script? - Ask Ubuntu¶
Bash Associative Arrays | Linux Journal¶
- page's self-description: The bash man page has long had the following bug listed: "It's too big and too slow" (at the very bottom of the man page). If you agree with that, then you probably won't want to read about the "new" associative arrays that were added in version 4.0 of bash.
Bash: Detect if variable is an array - FVue¶
Detaching processes from the current bash session | Sébastien Wains¶
Mivok.net - Bash function renaming and overriding¶
Process Substitution¶
- "read -a list < <( od -Ad -w24 -t u2 /dev/urandom )" this can be used to pipe the substituted command to a numbered port instead of passing it as an argument
- "a way to take the exit status into account is to use the command substitution in a simple assignment: then the exit status of the assignment is the exit status of the last command substitution in the assignment(s)." the following doesn't work: "Only other solution that came to mind at this moment would be to use read: ls -l ghost_under_bed | read name" "Another case to watch for is explicit subshells: (somecommand). According to the interpretation above, the subshell may return a nonzero status, but since this is not a simple command in the parent shell, the parent shell should continue." this is why errexit doesn't work properly for subshells on Mac
- - the following *doesn't* work: "read -r var << eof $var eof"
TMOUT – Automatically Exit Unix Shell When there is No Activity¶
- page's self-description: Question: I would like to terminate my Unix command line shell, when I don't execute any command for N number of seconds. i.e How to automatically log out if
where is the rehash command¶
- page's self-description: I was trying to install gEda which installed fine from their CD, but after setting the path variables I ran rehash as the instructions told me and I "Bash doesn't have the rehash builtin command. Use "hash -r" instead which is equivalent."
gnome-keyring
GNOME Keyring - ArchWiki¶
GNOME Keyring Tool¶
Kenneth Holter » Using Svn client and Gnome keyring in SSH sessions¶
- page's self-description: Subversion client software have traditionally stored (i.e. cached) plaintext user passwords, meaning that you password is accessible by anyone who can access files in your ~/.subversion/auth folder. With Subversion 1.6,
make
- page's self-description: GNU `make'
rsync
"Input/output error"
Rsync input/output error - bad disk or corrupt file? - Dedicated Server - Web Hosting Talk¶
rsync --append behavior¶
- page's self-description: Hello colleague I have little problem with my rsync's.I make sync from our country to other and everything is working.When sync finish donwloading or "--exclude-from=FILE exclude patterns listed in FILE"
Rsync include/exclude quirks¶
- page's self-description: Issues with rsync's include and exclude options
screen
scrollback
- note that viewing scrollback is much more difficult to do in screen than in the surrounding terminal program (which usually has graphical scrollbars). this is because screen provided scrolling before GUIs were available, so it had to rely on terminal rendering and keyboard commands. the ability to reconnect to a running shell (which is what screen is primarily used for nowadays) was actually an extra feature, rather than the primary purpose (providing scrollback), so it is not as well-engineered for that particular purpose as today's GUI users would like. for a better way to support reconnecting, see persistent shells.
- GNU Screen: Working with the Scrollback Buffer : Samsarin¶
- - "Entering Scrollback Mode and Navigating": "To enter scrollback hit C-a [" use arrow keys to scroll, like in less
Screen > Height¶
- "Set the display height to a specified number of lines" this will *not* change the scrollback height
- page's self-description: Redmine
screen - The Terminal Multiplexer¶
- "C-a : Allows you to enter `.screenrc' command lines"
sed
- " `unset LANG` "
sed : using colons as separators instead of forward slashes | commandlinefu.com¶
- page's self-description: sed : using colons as separators instead of forward slashes - sed \"s:/old/direcory/:/new/directory/:\" <file> - Having to escape forwardslashes when using sed can be a pain. However, it's possible to instead of using / as the separator to use : . I found this by trying to substitute $PWD into my pattern, like so $ sed \"s/~.*/$PWD/\" file.txt Of course, $PWD will expand to a character string that begins with a / , which will make sed spit out an error such as \"sed: -e expression #1, char 8: unknown option to `s'\". So simply changing it to $ sed \"s:~.*:$PWD:\" file.txt did the trick.
* Monitor Progress with Pipe Viewer¶
- page's self-description: For sys admins, running a process can feel like driving cross-country without a road map. Pipe Viewer is one way to answer the question, 'are we there yet?' "apt-get install pv" "pv -p -s ${SIZE}k"
- " $ ssh -t xxx.xxx.xxx.xxx "cd /directory_wanted && exec \$SHELL" "
/dev/fd | Advanced Programming in the UNIX® Environment: UNIX File I/O | InformIT¶
- page's self-description: Learn about the traditional UNIX I/O functions. You will see how to apply these functions when using your UNIX system.
BREs Regular Expressions¶
Clam AntiVirus¶
- - to scan your home directory: $ clamscan --recursive --infected ~ &
- page's self-description: Command to wait for a process (not a child process) Programming Talk
- page's self-description: du question: sort by size & show human-readable sizes Programming Talk
- Fink - Source Release Download¶
- page's self-description: The Fink Team wants to bring the full world of Unix Open Source software to Darwin and Mac OS X. We modify Unix software so that it compiles and runs on Mac OS X and make it available for download as a coherent distribution
- page's self-description: GNU `ed' Manual
How to Convert Uppercase to Lowercase With SED | eHow.com¶
- page's self-description: How to Convert Uppercase to Lowercase With SED. Stream editor, or sed, is a powerful Linux program that you can use to programmatically perform regular edits upon batches of text files. One example of a systematic edit that sed is well-suited for is converting all the letters in a batch of text files from uppercase to lowercase.
linux - How can I zip/compress a symlink? - Server Fault¶
- page's self-description: Hi, All, What is the command to display the nice value of a process? Thanks, August
Overview of Single Root I/O Virtualization (SR-IOV) (Windows Drivers)¶
- page's self-description: Explains how to send email log when sudo runs and configure sudo log file to keep track of all user commands. to turn off incorrect password e-mails: IMPORTANT: open a second root shell to use for recovery if /etc/sudoers gets messed up. otherwise, you may lock yourself out of root! $ nano /etc/sudoers add line "Defaults !mail_badpass" save $ sudo -s # test sudo
- uutils - Revision 126: /trunk¶
uutils - Unix utilities - Google Project Hosting¶
What is SR-IOV? - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers¶
- page's self-description: What is SR-IOV? Single Root I/O Virtualization is a specification that is designed to allow physical PCIe devices to present multiple virtual instances of themselves to an OS instance or hypervisor. Read the full article for all the details.
- page's self-description: Explains how to read and setup umask (he file mode creation mask of the current process) for file under Linux operating systems using bash shell.
Xterm Control Sequences¶
Windows
Windows Update
Error code 800706BE and 8008005 - Microsoft Community¶
- page's self-description: I have Windows Vista and I'm attempting to do automatic updates but the error messages 800706BE and 80080005 have both come up. I haven't been able to update since the 14th of January 2009 and the " 1. First, please verify that there is no any third-party firewalls preventing Windows update to communicate with "update.microsoft.com'. 2. Check whether the Windows firewall is blocking the Windows update. Please review the following KB article to follow for connection errors: http://support.microsoft.com/kb/836941 "
Temporary connection-related errors may occur when you use Windows Update or Microsoft Update¶
- page's self-description: Describes a problem that occurs when the Windows Update client does not receive a response from the Windows Update or Microsoft Update Web site. Provides a resolution for this problem.
- MicrosoftFixit50123.msi¶
Python
doc
- 13.1. csv — CSV File Reading and Writing — Python v2.7.3 documentation¶
19.5. XML Processing Modules — Python v2.7.4 documentation¶
19.6. xml.dom — The Document Object Model API — Python v2.7.2 documentation¶
20.6. urllib2 — extensible library for opening URLs — Python v2.7.2 documentation¶
26.4. The Python Profilers — Python v2.7.2 documentation¶
26.7. trace — Trace or track Python statement execution — Python v2.7.5 documentation¶
27.6. warnings — Warning control — Python v2.7.2 documentation¶
3. Data model — Python v2.7.3 documentation¶
5. Built-in Types — Python v2.7.3 documentation¶
5. Expressions — Python v2.7.2 documentation¶
7.2. re — Regular expression operations — Python v2.7.2 documentation¶
9.7. itertools — Functions creating iterators for efficient looping — Python v2.7.2 documentation¶
Overview — Python v2.7.2 documentation¶
regex 2014.02.19 : Python Package Index¶
- page's self-description: Alternative regular expression module, to replace re. "POSIX character classes [[:alpha:]]; [[:^alpha:]] POSIX character classes are supported. This is actually treated as an alternative form of \p{...}. "
parallel
- Documentation for processing-0.50¶
ParallelProcessing - PythonInfo Wiki¶
16.2. threading — Higher-level threading interface — Python v2.7.3 documentation¶
Parallel Python - Home¶
- page's self-description: Parallel Python
3.4.1 Basic customization¶
3.6.2 String Formatting Operations¶
compiled Jython vs. CPython | Python | Python¶
- page's self-description: compiled Jython vs. CPython Python Python "the benchmarks comparing Jython+JIT to CPython are mostly in favour of CPython" "An advantage of jython is that it profits from speedups of the JVM's. Since Java is more widespread than CPython, there is in general more interest in speeding up JVM's than in speeding up the CPython byte code interpreter."
Differences between CPython and Jython¶
Django | GeoDjango | Django documentation¶
- page's self-description: Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.
- page's self-description: How do you convert a hex representation into a string?. Python Forums on Bytes. " >>> def numToWord(n): ... hexvalue = hex(n)[2:] ... return "".join([chr(int(s, 16)) for s in [hexvalue[i:i+2] for i in range(0,len(hexvalue),2)]]) ... >>> numToWord(1650811246) 'bean' >>> "
Issue 1519638: Unmatched Group issue - workaround - Python tracker¶
MySQLdb User's Guide¶
PEP 0263 -- Defining Python Source Code Encodings¶
- page's self-description: None
PEP 3132 -- Extended Iterable Unpacking¶
- page's self-description: None
- Python Exception Handling Techniques - Doug Hellmann¶
urllib2 - The Missing Manual¶
- page's self-description: Website of Michael Foord. Python programming articles, projects and technical blog.
science
open-access data
Data reuse and the open data citation advantage [PeerJ]¶
- page's self-description: Background. Attribution to the original contributor upon reuse of published data is important both as a reward for data creators and to document the provenance of research findings. Previous studies have found that papers with publicly available datasets receive a higher number of citations than similar studies without available data. However, few previous analyses have had the statistical power to control for the many variables known to predict citation rate, which has led to uncertain estimates of the “citation benefit”. Furthermore, little is known about patterns in data reuse over time and across datasets. Method and Results. Here, we look at citation rates while controlling for many known citation predictors and investigate the variability of data reuse. In a multivariate regression on 10,555 studies that created gene expression microarray data, we found that studies that made data available in a public repository received 9% (95% confidence interval: 5% to 13%) more citations than similar studies for which the data was not made available. Date of publication, journal impact factor, open access status, number of authors, first and last author publication history, corresponding author country, institution citation history, and study topic were included as covariates. The citation benefit varied with date of dataset deposition: a citation benefit was most clear for papers published in 2004 and 2005, at about 30%. Authors published most papers using their own datasets within two years of their first publication on the dataset, whereas data reuse papers published by third-party investigators continued to accumulate for at least six years. To study patterns of data reuse directly, we compiled 9,724 instances of third party data reuse via mention of GEO or ArrayExpress accession numbers in the full text of papers. The level of third-party data use was high: for 100 datasets deposited in year 0, we estimated that 40 papers in PubMed reused a dataset by year 2, 100 by year 4, and more than 150 data reuse papers had been published by year 5. Data reuse was distributed across a broad base of datasets: a very conservative estimate found that 20% of the datasets deposited between 2003 and 2007 had been reused at least once by third parties. Conclusion. After accounting for other factors affecting citation rate, we find a robust citation benefit from open data, although a smaller one than previously reported. We conclude there is a direct effect of third-party data reuse that persists for years beyond the time when researchers have published most of the papers reusing their own data. Other factors that may also contribute to the citation benefit are considered. We further conclude that, at least for gene expression microarray data, a substantial fraction of archived datasets are reused, and that the intensity of dataset reuse has been steadily increasing since 2003. "papers with publicly available datasets receive a higher number of citations than similar studies without available data" "There is evidence that studies that make their data available do indeed receive more citations than similar studies that do not"
software concepts
APIs
- **** How to Design a Good API and Why it Matters¶
- "Public APIs are forever - one chance to get it right" "Code lives on as examples, unit tests" "Your job is to extract true requirements _ Should take the form of use-cases" "Write multiple plug-ins before release [...] If you write three, it will work fine" "You won't be able to please everyone _ Aim to displease everyone equally" "Expect to evolve API" "You can always add, but you can never remove" "Names Should Be Largely Self-Explanatory _ Avoid cryptic abbreviations" "Parameter: indicate units, form, ownership" "Dimension is mutable • Each getSize call must allocate Dimension • Causes millions of needless object allocations" "Classes should be immutable unless there ’ s a good reason to do otherwise" "Reduce need for boilerplate code _ Generally done via cut-and-paste" "Overload With Care [...] no two with same number of args" "Three or fewer parameters is ideal" "Create helper class to hold parameters"
- "Dynamic scoping also voids all the benefits of referential transparency" "Dynamic scoping provides an excellent abstraction for thread local storage, but if it is used that way it cannot be based on saving and restoring a global variable"
Hamming distance - Wikipedia, the free encyclopedia¶
sync
BitTorrent Sync
- SECURITY WARNING: There is an important security hole in the Linux version of BitTorrent Sync: when btsync is run with no arguments, it makes all your files suddenly *world-readable and world-writable*, via a publicly-accessible, unprotected web interface (the WebUI). The WebUI by default allows anyone on the web to create a sync folder to view and edit any files in directories writable by you. To fix this, you apparently *must* use a config file (with the --config option) which is appropriately secured by setting webui > password. If you are currently running btsync without a secured config file, you should run `killall btsync` to turn off the WebUI. You can use `lsof -i` to verify that the WebUI is not running. The security hole has been reported at http://forum.bittorrent.com/topic/18612-security-hole-in-linux-webui-makes-all-your-files-publicly-accessible/ , but it's not clear when/whether they will fix it.
WebUI
- located at http://hostname:8888/gui/
BitTorrent WebUI - Videos & Guides - Help - BitTorrent - Delivering the World's Content¶
- page's self-description: BitTorrent is a fast and easy client for Windows and Mac with many features.
Question about the webUI - BitTorrent Sync - BitTorrent Forums¶
- page's self-description: Question about the webUI - posted in BitTorrent Sync: Because I'd like to run this on a headless linux server, I'm obviously not going to be connecting from localhost. When I connect from a LAN interface on it's LAN IP I get "Invalid request" as shown below:- However, if I create an SSH tunnel so I'm connecting from localhost (In this case, 0.0.0.0:8889 --> localhost:8888) it works:- Is there anyway to get BitTorrent Sync to accept LAN requests? Keeping an S...
- page's self-description: security hole in Linux WebUI makes all your files publicly accessible - posted in BitTorrent Sync: Running btsync on Linux will by default create a *publicly-accessible, unprotected* WebUI, allowing anyone on the web to create a sync folder to view and edit files your files (i.e. files in directories writable by you). Could the defaults (used when running btsync without a config file) be changed to prevent this unintended data leak? A temporary workaround is to run `killall btsync` to tur...
- page's self-description: .SyncIgnore is great but... - posted in BitTorrent Sync: .SyncIgnore is a file where you can filter out files you don't want synced. However it seems that when I put files in here bittorrent decides to delete those files rather than leave them un-touched. That is unexpected. Is this a bug? Expected behavior? Am I using it wrong? Cheers
- page's self-description: Page 1 of 2 - Install on Linux - posted in BitTorrent Sync: I just got my invite for Sync really like it so far, but i am not able to install the Linux version (SyncApp_i386-1.0.63). my Linux expert says it is missing its header. do we do something wrong or is this an bug?
- page's self-description: Move synced directory - posted in BitTorrent Sync: I would like to move where a directory is synced to without re-syncing it. Suppose I have given a read-only key to 10 other people, and now I want it stored in a different place on my machine. Is there any way of doing that without having them all delete the old and add a new key?
Dropbox
upgrade notification
- when you use an out-of-date Dropbox client to look up a file on the Dropbox website, it just gives you links to download Dropbox, but doesn't tell you *why* you need to re-download Dropbox. many users would falsely assume they don't need to download it because they already have. however, they do in fact need to *re*-download it in order to *upgrade* their client to be compatible with the website. this page should instead say: "your version of Dropbox is out-of-date, and links to the Dropbox website will not work. please download the latest version of Dropbox below to restore this functionality." I spent a long time assuming that the website links were just broken, until I finally realized that this was because my client was out-of-date. (I had always thought that Dropbox would prompt me to upgrade if a new version were available, the way most other software does.)
- " exclude the .svn-Folder from Dropbox: [...] Go to your Folder an Copy the .svn-Folder somewhere else Go to your Dropbox-Preferences > Advanced > Selective Sync and deactivate to .svn-Folder The Folder should now be deleted from your HD and Dropbox, you can check that on the website. Put your copy of the .svn-Folder back into the directory. You should see a little gray sign like (-) "
Google Drive
- do not use: unstable
** The Google Drive folder selected is not empty. Please select an empty folder. - Google Product Forums¶
- page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations. "once you log out of GDrive, there is absolutely, positively no way at all to reconnect a GDrive login back to a local folder that already has files in it" "Google Drive only works properly if you a) have only one account and b) never, ever ... for any reason what so ever ... log out of that one account." "I selected all, dragged o my desktop, started a sync, then put everything back in my Drive folder. I was asked if I wanted to replace the file/folder and I said yes. I think this did the trick. I have a mac" but: "I did that once before and got duplicates of everything."
Google Product Forums › Google Drive › Follow Symbolic Links¶
- "Dropbox let's me link other folders on my computer to my Synced folder using Symbolic Links. It let's me link my Documents folder to my Dropbox and is very convenient. This doesn't seem to work with Google Drive."
- page's self-description: Google Groups allows you to create and participate in online forums and email-based groups with a rich experience for community conversations.
EasySVN < Subversion Guide | Subversion Clients, Subversion Servers & more¶
- page's self-description: A subversion guide to source code management including subversion tutorials, reviews for svn clients and servers, free snv client downloads, and more.
version control
svn
'svn upgrade' fails with "svn: E155016: Insufficient NODES rows"¶
- - a .*-revert file is indeed present, but deleting it does not fix the problem. instead, check out a new working copy: $ find . -name .svn -exec rm -rf {} \; $ svn co --force repo_url .
- "You need only append an at sign to the end of the path [to escape any @s in it], such as news@11@. svn cares only about the last at sign in the argument, and it is not considered illegal to omit a literal peg revision specifier after that at sign" note that the trailing-@ escape for paths containing @ should only be used on paths where @ has a special meaning (usually only the *input* path)
- ** svn switch (sw)¶
- - eg. after moving everything into /trunk/, do the following on all other working copies: $ time svn switch --force https://<old-repo-root>/trunk # runtime: <1 min ("0m28.465s") # if there are errors after running svn switch, run svn switch on each broken file to restore the correct repository URL
*** Creating an SVN backup with svnsync : CloudForge Support¶
- page's self-description: Creating an SVN backup with svnsync. SVN. Code and Deploy note: you must run *all* of the commands in this page, including creating hooks/pre-revprop-change
Apache Subversion FAQ¶
Executable Files in SVN¶
High Fibre Programming » SVN: Entry has unexpectedly changed special status¶
- - actually, you should force-replace the item: $ svn rm --keep-local <item> $ svn add <item>
- Version Control with Subversion: Ignoring Unversioned Items¶
- "[ Begins a character class definition terminated by ], used for matching a subset of characters"
git
git-svn
to migrate from svn
- *IMPORTANT*: first put your repository into the standard svn layout (branches, tags, trunk) $ cd <parent_dir_of_checkout> $ time git svn clone --stdlayout https://<svn-url> & # runtime: 15 min ("12m24.338s") # note that it appears to spend 10 min thinking about it, and then at the very end, very quickly downloads your entire repo to `svn update`: $ git svn fetch
(some steps in this article are incorrect) Converting a Subversion repository to Git, (7 steps to migrate a complete mirror of svn in git) | JohnAlbin¶
** GitSvnCrashCourse - Git SCM Wiki¶
- "here's what you need to use an SVN project: git svn clone --stdlayout url # svn checkout url git svn rebase # svn update git commit -a && git svn dcommit # svn commit "
- "I changed the .git/config SVN url" note that the URL must be a web URL, not file://
- "You can only add collaborators to your repository. It cannot be "co-owned"."
- "`git show HEAD`" "`git format-patch HEAD^ --stdout`" "`git log -1 -p --pretty='%b'`"
A Gaggle Of Git Tips | Viget¶
- "I use git rebase -i to make sure every commit is clean. For example, when I have two commits with broken code followed by a third clean commit"
Emmanuel Bernard: How to install Git and git-svn on Mac OS X¶
Git - git-apply Documentation¶
web
client side
color profiles
Gfx.color management.mode - MozillaZine Knowledge Base¶
- " 1: Enable color management for rendered graphics. 2: Enable color management for tagged graphics only. (Default) " "Without a properly calibrated monitor and a correct color profile, color management may actually make colors look worse."
Browsers, sRGB and monitor profiling | duncanmartin.com¶
- page's self-description: Duncan Martin's Blog "Firefox can apply monior profiles, but there’s a complication. By default it will only apply the monitor profile if the image is tagged with a colour space." "Chrome doesn’t apply monitor profiles."
CSS
CSS white-space property¶
- page's self-description: Free HTML CSS JavaScript DOM jQuery XML AJAX RSS ASP .NET PHP SQL tutorials, references, examples for web building.
Firefox
- - **IMPORTANT**: to enable security.password_lifetime, you need to set security.ask_for_password=2: "security. password_lifetime Integer Determines how long (in minutes) to go without asking for the master password in Mozilla Mail when security.ask_for_password is 2. Default value is 30." "security. ask_for_password Integer Determines when Mozilla Mail should ask for the master password. 0 (default): Only the first time it's needed 1: Every time it's needed 2: Every n minutes, where n is the value in security.password_lifetime. "
How to make all tabs reload on open? | Firefox Support Forum | Mozilla Support¶
- " See Tab preferences and settings Use Firefoxbutton-> Options -> |Tabs|-> [] don't load tabs until selected (clear the tick) "
Flash
How Fight My Monster Actually Works - 推酷¶
- page's self-description: How Fight My Monster Actually Works http://www.fightmymonster.com/ "We have a Flash client with a difference. It is full screen, uses a flowed layout [no it doesn't], and looks and behaves like a normal website" "The client has a complex state, but if the browser window is refreshed, the client is able to restore its previous state from the server"
Java applets
Should I use Java applets or JavaScript/PHP to make my site more interactive? - Programmers Stack Exchange¶
- "I can do it with JAVA much faster (know it better than javascript and php). But I am afraid of browser incompatibility, security options for applets and similar things." "While Applets are ridiculously out of fashion, that does not matter to end users. On a modern machine, JVM startup time is not a problem, and *browser incompatibility less of a factor than with JavaScript*." [+emph] "Applet-specific security is irrelevant as long as you only draw stuff and don't want to do things like access the user's harddisk." (but you can use a desktop app to get around that) "they don't run in all browsers" which ones? "Libraries like JQuery and ext_js make gui development as structured as java" but without the all-important type safety "Applets [...] require client installs of java" but most clients already have this installed "Almost everyone has Flash installed, and every modern browser version can handle HTML Canvas. But while most people will have Java installed, many wont." who? "the user needs to have a Java browser plug-in installed but that's true for nearly everyone nowadays" "The main benefit is that Java gives you a huge number of libraries / tools for layout and graphics etc" "Java applets can achieve very good performance, certainly well ahead of Javascript. May not be a real issue in your case, but you get all the usual advantages of the JVM JIT-compiling down to native code" "the Java layer means you don't have to worry about browser differences / incompatibilities so much" "the Google Web Toolkit. Your client code can be written in Java that gets compiled into browser-specific JS"
Roundcube
change password
#1483886 (Roundcube doesn't work when password contains escape character "\") – Roundcube Webmail¶
- page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
- page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
Backslash Character in IMAP Password? - Chilkat Q&A Forum¶
- page's self-description: I do have a question though as a user is experiencing a bug: their password has a backslash in it and the library is not able to connect to the IMAP server because the string is not properly escaped. Is there a best practice for this?
28941 – backslash and Double Quote " not escaped for IMAP username Login¶
- page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
- page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
- page's self-description: The Roundcube Webmail suite. Contribute to roundcubemail development by creating an account on GitHub.
wikis
Redmine
** exporting data
*** REST API
- ** you can also use .json to get data in JSON format
- Rest api - Redmine¶
- page's self-description: Redmine "Most of the time, the API requires authentication. To enable the API-style authentication, you have to check Enable REST API in Administration -> Settings -> Authentication"
- Rest WikiPages - Redmine¶
- page's self-description: Redmine
** Export your HostedRedmine.com projects - HostedRedmine.com Support - Online Project Management :: Free Redmine Hosting¶
- page's self-description: Redmine
- ** Export your HostedRedmine.com projects - HostedRedmine.com Support - Online Project Management :: Free Redmine Hosting: Exporter.zip¶
- page's self-description: Redmine see files.php
HTML
- Defect #6272: Raw HTML Formating not working - Redmine¶
- page's self-description: Redmine
- Comments in Wiki pages - Redmine¶
- page's self-description: Redmine
- Insert HTML fragments - Redmine¶
- page's self-description: Redmine
WYSIWYG
- New WYSIWYG for Wiki - Redmine¶
- page's self-description: Redmine
- page's self-description: redmine_wysiwyg_textile - Redmine Wysiwyg Textile Editor using tinymce
- Anchors in wiki pages - Redmine¶
- page's self-description: Redmine
- Feature #4482: Cache textile rendering - Redmine¶
- page's self-description: Redmine
Introducing the Redmine Google Docs Plugin | Evolving Web¶
possible to move/copy a project to new server - Redmine¶
- page's self-description: Redmine 'it may be easier to copy my whole db over, then remove all the other projects just leaving the one I need, then add the new projects on top of it'
Question Defense » Outline Every Cell In Every Row Of A Table Using Textile Formatting¶
- page's self-description: Outline Every Cell In Every Row Of A Table Using Textile Formatting | Question Defense
- Redmine - How to make wiki public - Redmine¶
- page's self-description: Redmine
- Redmine - RedmineTextFormatting - Redmine¶
- page's self-description: Redmine
- RedmineWikiFormatting¶
Using anchors in redmine - Alexey Bokov’s weblog » Blog Archive » Redmine as corporate wikipedia¶
TWiki
BlackListPlugin < Plugins < TWiki¶
DontNotify < TWiki < TWiki¶
TreePlugin < TWiki < TWiki¶
TWiki on Ubuntu 10.04 (Lucid) – Linode Library¶
- page's self-description: Install and configure a structured wiki with TWiki.
- "If you move the TWiki.TWikiRegistration topic I recommend to move it to an access controlled web"
Confluence Licensing | Atlassian¶
- page's self-description: Confluence pricing starts at $10 for 10 users and scales with your growing business, from startup to enterprise. Download Confluence or try hosted.
- Convert HTML to XHTML¶
- page's self-description: Online HTML to XHTML converter tool, copy and paste your HTML source code into the box below.
How to Block Certain Sites in Google Search Results without Extensions | Web Talk¶
- page's self-description: Google users are accustomed to modify the Google official home page with a lot of extensions able to modify the design of the Google search engine page as
Mozilla — Plugin Check & Updates — mozilla.org¶
weird square favicon | Firefox Support Forum | Mozilla Support¶
domains
AdamsNames
- IMPORTANT - AdamsNames Changes¶
Breaking: Registry Operator AdamsNames Hacked Locking Up .TC, .GD & .VC¶
- page's self-description: According to a press release we just received by* KSregistry GmbH, which is owned by Key-Systems GmbH, “a third party has executed a transfer of the domain name adamsnames.net and now operates a shadow registry under this domain”. Adams Names is the Registry for the ccTLD’s of .TC, .GD and .VG and uses KSregistry as its back end to operate the registries. “According to the CEO of AdamsNames Ltd., Mr. Carsten Pauli, this transfer was not authorized by the registry operator”. “KSregistry
IMPORTANT - AdamsNames Changes (.TC - .GD - .VG) | Hexonet¶
DNS
Network Tools: DNS,IP,Email¶
- page's self-description: DNS and Network troubleshooting and diagnostic tools integrated into one sweet interface.
Traceroute, Ping, Domain Name Server (DNS) Lookup, WHOIS¶
Deploying DNSSEC | ICANN¶
DNS: DNAME is useless - davidb dives in¶
- page's self-description: DNAME, if you are not aware, is a DNS record type defined in RFC 2672. If you are familiar with the DNS’s CNAME record, then think of DNAME as …
BIND 9's New Resource Records | DNAME, Domain Alias | InformIT¶
- page's self-description: DNS's new version of BIND, BIND 9, implements two new Resource Records to help you take DNS to an advanced level. Learn about them quickly in this article.
- Trailing Dots in Domain Names¶
- "If you're a web site administrator setting up a web site using Apache "VirtualHost" directives or similar, you need to have a ServerAlias line listing all the things the user might type to get to that web site (typically the first label, the whole name without a trailing dot, and the whole name with a trailing dot, as shown in the example above)."
subdomain delegation
How to delegate a sub-domain to other DNS servers - Simple DNS Plus¶
Basic Guide to DNS - Google Apps Help¶
- page's self-description: This guide is a list of common terms you may encounter when managing the DNS settings of your domain. Click a term below to jump to the description of the term and its practical use with Google Apps
TLDs
GTLD – Trusted Information on Generic Top Level Domains – Verisign¶
- page's self-description: Verisign brings extensive gTLD expertise. We provide premium consulting and technical services to help you apply for, manage, and monetize your new gTLD.
When will the new gTLDs be released? : Marcaria.com Support¶
- page's self-description: When will the new gTLDs be released?. New gTLDs. Domain Names
- New TLD List 2012 - New Top Level Domains - New TLDs 2012¶
- page's self-description: Comprehensive list of the new Top Level Domains (TLD's) for the New TLD program from ICANN in 2012.
Free Website Uptime Monitoring - Uptime Robot¶
Linux Basics - Set A Static IP On Ubuntu | HowtoForge - Linux Howtos and Tutorials¶
- page's self-description: Linux Basics - Set A Static IP On Ubuntu
10 Free Services to Monitor Your Site's Uptime¶
- page's self-description: This series is supported by Rackspace, the better way to do hosting. Learn more about Rackspace's hosting solutions here. You can't sit staring at your monitor 24 hours a day to...
- Three Letter Domain names¶
- page's self-description: 3 letter domain names
.ORG Registrars | .ORG, The Public Interest Registry| Buy, Sell, Register .ORG Domain Names¶
e-mail
Gmail
Gmvault: gmail backup¶
- page's self-description: backup and restore your gmail account To do full backup: /Applications/gmvault-v1.8.1-beta/bin/gmvault sync --multiple-db-owner user@gmail.com To backup only new e-mails: /Applications/gmvault-v1.8.1-beta/bin/gmvault sync --multiple-db-owner --type quick user@gmail.com In Mac OS X (including 10.8), you need to run `.../gmvault-v1.8.1-beta/bin/gmvault`. This is in the README.txt under "Linux and Mac OS X install". However, other methods don't work: gmv_runner.app just opened and closed, `pip install gmvault` gave me an error "pkg_resources.DistributionNotFound: pip==1.1", and `sudo easy_install gmvault` gave me an error "ImportError: No module named gmv.gmv_cmd".
Using Web Notifications - WebAPI | MDN¶
- page's self-description: The Web Notifications API allows a web page to send notifications that are displayed outside the page at the system level. This allows web apps to send information to a user even if the application is idle. One of the main obvious use cases is a webmail application that would notify the user each time a new e-mail is received, even if the user is doing something else with another application. "Firefox and Safari close the notifications automatically after a few moments, e.g. 4 seconds"
server side
Apache
2.4
- page's self-description: ! Please move thread to Server Platforms ! (my mistake) The latest stable apache web server 2.4.2 is available for both 12.04 LTS and Quantal Quetzal ! Test away! Current apache 2.2 users please backup your stuff beforehand! apt-add-repository ppa:ptn107/apache apt-get update apt-get install apache2 https://launchpad.net/~ptn107/+archive/apache
Question #224526 : Questions : Ubuntu¶
- page's self-description: As company compliance to Requirement 10 of PCI DSS, I recently installed a File Integrity Monitoring system using OSSEC HIDSi and it's been up and running. We are also required to conduct internal vulnerability scan with our network to comply with Requirement 11 and so I used Nexpose Rapid7. After scanning, report says that Linux server running OSSEC is vulnerable through its outdated Apache server (version 2.22) and suggest to upgrade it to version 2.4. Apache server is required for the OSSE...
Upgrading to 2.4 from 2.2 - Apache HTTP Server¶
auth
digest auth
TracStandalone – The Trac Project¶
- "The SHA encryption format does not use salting"
Password Formats - Apache HTTP Server¶
doc
mod_rewrite
Advanced Techniques with mod_rewrite - Apache HTTP Server¶
mod_rewrite
Crazy Advanced Mod_Rewrite Debug Tutorial¶
- page's self-description: Crazy Advanced Mod_Rewrite Debug Tutorial
Bug 38642 – mod_rewrite adds path info postfix after a substitution occured¶
- "In Lighttpd you're restricted to rewriting via the config level only, which means for things like Wordpress you will need to hard code the rewrite rule in."
apache2 - enabling and disabling sites - Apache - Snipplr Social Snippet Repository¶
- page's self-description: enable and disable sites and modules in apache 2
- " > /* if someone used the PASSTHROUGH flag in per-dir > * context we just ignore it. It is only useful > * in per-server context > */ "
Setting up an SSL server with Apache2¶
- page's self-description: Tips for a Debian GNU/Linux System Administrator.
Setting up Self-Signed SSL Certificate to use with Apache Web Server | Knowledge by Experience¶
- page's self-description: In this article, we will see, how to setup a self-signed SSL certificate and configure it to use with Apache Web Server in Ubuntu Linux platform. This i...
Where are the Apache Configuration files on Mac? – httpd.conf | Smart Web Developer¶
- page's self-description: Here's where to find httpd.conf on Mac. This is the httpd.conf location on Mac OS X 10.6 Snow Leopard and Mac OS X 10.7 Lion. This article also shows how to
FastCGI
- Integrating FastCGI with Java¶
Apache module mod_fastcgi¶
Java frameworks
Tomcat
How to change Tomcat default port ?¶
- "Twitter for example was built using Rails/Ruby but once it became unscalable, they migrated to the JVM." "There are over 38 different web frameworks for java according to wikipedia ( http://en.wikipedia.org/wiki/Comparison_of_web_application_frameworks#Java ) and 23 different ORM's ( http://en.wikipedia.org/wiki/List_of_object-relational_mapping_software#Java )" "it seems that dynamic languages tend to have the tools and frameworks that remove a lot of boilerplate code. (To make up for our lack of powerful IDEs that generate boilerplate for us.)" "The steps to become proficient with Java are complex and/or expensive" "Applets extinguished maybe decade ago" (but they are still used for self-contained UIs, such as games) "Flash did what Applets tried to be. Quick start up, fast download, low memory footprint" (but they lack Java's type safety and large library) "We implemented our own dynamic class loader which means we can change Java classes on the fly without restarting the server"
URLs
- What Every Developer Should Know About URLs¶
- page's self-description: I have recently written about the value of fundamentals in software development. I am still firmly of the opinion that you need to have your fundamentals down
- PURL Home Page¶
bitly¶
URL Decoder/Encoder¶
Not your typical links- URL Encoding at Dreamweaver FAQ.com¶
- page's self-description: Some characters are not safe to use in a URL without first being encoded. The most common URL escape code is %20, which represents a space. Listed here are some of the most common encoding characters with examples of their use.
Bug #788256 “inkscape cannot dock toolbars” : Bugs : “inkscape” package : Ubuntu¶
- page's self-description: Binary package hint: inkscape I installed Inkscape for the first time on Ubuntu 11.04 and I see that the main toolbar and the toolbar for sticking lines and segments where on the right side (later noticed the "wide" and "standard" views). 1) I tried to move the main toolbar and it worked like expected in vertical mode. I could however not dock the toolbar on top, horizontally. 2) When I tried to move the other toolbar it would not dock anywhere, not even where it was at the beginning. Pro...
- Compatibility - Microsoft Access 2000 - CodeWeavers¶
Converting UTF8 to Latin1¶
Creating Hyperlinks in Adobe Acrobat « Acrobat for Legal Professionals¶
- page's self-description: The Acrolaw Blog is a resource for lawyers, law firms, paralegals, legal IT pros and anyone interested in the use of Acrobat in the legal community. Rick Borstein– the author of the blog– is the Business Development Manager for Acrobat in the Legal Market for Adobe Systems.
cryptography - Is SHA1 better than md5 only because it generates a hash of 160 bits? - IT Security Stack Exchange¶
Download data for the whole world | Global Administrative Areas¶
English to French, Italian, German & Spanish Dictionary - WordReference.com¶
- page's self-description: Free online Oxford dictionaries - Spanish, French, Italian, German and more. Conjugations, audio pronunciations and forums for your questions.
How To Create An Image Map Using GIMP¶
- page's self-description: Have you ever wanted to add an image map to your web site? An image map is a graphic that contains hotspots that link to URLs. They are used as a means t
- "Choose Tools—>Advanced Editing—>Touchup Object Tool Right-click and choose Place Image…"
Password Strength Test¶
PHP: $_SERVER - Manual¶
Safari Books Online - Home¶
utf8-decoder¶
virtualbox.org • View topic - Shared folder permissions¶
WARNING: <file> failed verification -- update discarded (will try again).¶
- page's self-description: Free HTML XHTML CSS JavaScript jQuery XML DOM XSL XSLT RSS AJAX ASP .NET PHP SQL tutorials, references, examples for web building.
Google Calendar¶
remember password¶
Recent Tags¶
Recently Bookmarked¶
sPlot-ERD_2013-09-09.jpg (JPEG Image, 1298 × 843 pixels)¶