Project

General

Profile

« Previous | Next » 

Revision 12616

web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: added links for troubleshooting autovacuuming (which can slow queries down significantly when it isn't happening for any tables)

View differences:

index.htm
31 31
        <DT><A HREF="https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/To_Do" name="__https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/To_Do" ADD_DATE="1342214119" ICON_URI="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1347421715"><img width="16" height="16" src="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1347421715" style="margin-right: 0.5em;"/>to do</A><a name="__to do" href="#__to do" style="margin-left: 0.5em;">&para;</a>
32 32
        <DT><A HREF="https://docs.google.com/spreadsheet/ccc?key=0ArZXrTAXd-TYdDRRb2RxYi11TWZrQVh5bVdKOURCeFE" name="__https://docs.google.com/spreadsheet/ccc?key=0ArZXrTAXd-TYdDRRb2RxYi11TWZrQVh5bVdKOURCeFE" ADD_DATE="1387233673" ICON_URI="https://ssl.gstatic.com/docs/spreadsheets/favicon_jfk2.png" LAST_CHARSET="UTF-8"><img width="16" height="16" src="https://ssl.gstatic.com/docs/spreadsheets/favicon_jfk2.png" style="margin-right: 0.5em;"/>pub</A><a name="__pub" href="#__pub" style="margin-left: 0.5em;">&para;</a>
33 33
        <DT><A HREF="https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/Spot-checking_validation_status" name="__https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/Spot-checking_validation_status" ADD_DATE="1355767956" ICON_URI="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242"><img width="16" height="16" src="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242" style="margin-right: 0.5em;"/>valid</A><a name="__valid" href="#__valid" style="margin-left: 0.5em;">&para;</a>
34
        <DT><A HREF="https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore" name="__https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore" ADD_DATE="1354727779" ICON_URI="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1347421715"><img width="16" height="16" src="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1347421715" style="margin-right: 0.5em;"/>VegCore</A><a name="__VegCore" href="#__VegCore" style="margin-left: 0.5em;">&para;</a>
34
        <DT><A HREF="https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore" name="__https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore" ADD_DATE="1354727779" ICON_URI="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242"><img width="16" height="16" src="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242" style="margin-right: 0.5em;"/>VegCore</A><a name="__VegCore" href="#__VegCore" style="margin-left: 0.5em;">&para;</a>
35 35
        <DT><A HREF="https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegBIEN_schema" name="__https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegBIEN_schema" ADD_DATE="1344469303" ICON_URI="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242"><img width="16" height="16" src="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242" style="margin-right: 0.5em;"/>VegBIEN</A><a name="__VegBIEN" href="#__VegBIEN" style="margin-left: 0.5em;">&para;</a>
36 36
        <DT><A HREF="https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegBIEN_FAQ" name="__https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegBIEN_FAQ" ADD_DATE="1389803305" ICON_URI="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242" LAST_CHARSET="UTF-8"><img width="16" height="16" src="https://projects.nceas.ucsb.edu/nceas/favicon.ico?1391043242" style="margin-right: 0.5em;"/>FAQ</A><a name="__FAQ" href="#__FAQ" style="margin-left: 0.5em;">&para;</a>
37 37
        <DT><A HREF="http://vegbiendev.nceas.ucsb.edu/phppgadmin/redirect.php?server=localhost%3A5432%3Aallow&subject=database&database=vegbien" name="__http://vegbiendev.nceas.ucsb.edu/phppgadmin/redirect.php?server=localhost%3A5432%3Aallow&subject=database&database=vegbien" ADD_DATE="1362008923" ICON_URI="http://vegbiendev.nceas.ucsb.edu/phppgadmin/images/themes/default/Introduction.png"><img width="16" height="16" src="http://vegbiendev.nceas.ucsb.edu/phppgadmin/images/themes/default/Introduction.png" style="margin-right: 0.5em;"/>vegbiendev</A><a name="__vegbiendev" href="#__vegbiendev" style="margin-left: 0.5em;">&para;</a>
......
867 867
<DD>&quot;For some types of errors, the server reports the name of a database object (a table, table column, data type, or constraint) associated with the error; for example, the name of the unique constraint that caused a unique_violation error. Such names are supplied in separate fields of the error report message so that applications need not try to extract them from the possibly-localized human-readable text of the message. As of PostgreSQL 9.3, complete coverage for this feature exists only for errors in SQLSTATE class 23 (integrity constraint violation), but this is likely to be expanded in future.&quot;
868 868
this is especially useful for error parsing in column-based import [wiki.vegpath.org/Column-based_import] (and may even have been motivated by it!)
869 869
                                <DT><A HREF="http://www.postgresql.org/docs/9.3/static/app-psql.html#APP-PSQL-PATTERNS" name="http://www.postgresql.org/docs/9.3/static/app-psql.html#APP-PSQL-PATTERNS" ADD_DATE="1392338475" ICON_URI="http://www.postgresql.org/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://www.postgresql.org/favicon.ico" style="margin-right: 0.5em;"/>PostgreSQL: Documentation: 9.3: psql</A><a name="PostgreSQL: Documentation: 9.3: psql" href="#PostgreSQL: Documentation: 9.3: psql" style="margin-left: 0.5em;">&para;</a>
870
                                <DT><A HREF="http://www.postgresql.org/docs/9.3/static/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-VACUUM-COST" name="http://www.postgresql.org/docs/9.3/static/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-VACUUM-COST" ADD_DATE="1394165454" ICON_URI="http://www.postgresql.org/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://www.postgresql.org/favicon.ico" style="margin-right: 0.5em;"/>PostgreSQL: Documentation: 9.3: Resource Consumption: 18.4.4. Cost-based Vacuum Delay</A><a name="PostgreSQL: Documentation: 9.3: Resource Consumption: 18.4.4. Cost-based Vacuum Delay" href="#PostgreSQL: Documentation: 9.3: Resource Consumption: 18.4.4. Cost-based Vacuum Delay" style="margin-left: 0.5em;">&para;</a>
871
<DD>&quot;During the execution of VACUUM and ANALYZE commands, the system maintains an internal counter that keeps track of the estimated cost of the various I/O operations that are performed. When the accumulated cost reaches a limit (specified by vacuum_cost_limit), the process performing the operation will sleep for a short period of time, as specified by vacuum_cost_delay. Then it will reset the counter and continue execution.&quot;
872

  
873
&quot;This feature is disabled by default for manually issued VACUUM commands.&quot; however, *autovacuum*_vacuum_cost_delay is *enabled* by default
870 874
                                <DT><A HREF="http://www.postgresql.org/docs/9.3/static/ddl-schemas.html" name="http://www.postgresql.org/docs/9.3/static/ddl-schemas.html" ADD_DATE="1393374155" ICON_URI="http://www.postgresql.org/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://www.postgresql.org/favicon.ico" style="margin-right: 0.5em;"/>PostgreSQL: Documentation: 9.3: Schemas</A><a name="PostgreSQL: Documentation: 9.3: Schemas" href="#PostgreSQL: Documentation: 9.3: Schemas" style="margin-left: 0.5em;">&para;</a>
871 875
<DD>&quot;pg_catalog is always effectively part of the search path. If it is not named explicitly in the path then it is implicitly searched before searching the path&#39;s schemas. This ensures that built-in names will always be findable. However, you can explicitly place pg_catalog at the end of your search path if you prefer to have user-defined names override built-in names.&quot;
872 876
                                <DT><A HREF="http://www.postgresql.org/docs/8.4/static/logfile-maintenance.html" name="http://www.postgresql.org/docs/8.4/static/logfile-maintenance.html" ADD_DATE="1333386461" ICON_URI="http://www.postgresql.org/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://www.postgresql.org/favicon.ico" style="margin-right: 0.5em;"/>PostgreSQL: Documentation: Manuals: Log File Maintenance</A><a name="PostgreSQL: Documentation: Manuals: Log File Maintenance" href="#PostgreSQL: Documentation: Manuals: Log File Maintenance" style="margin-left: 0.5em;">&para;</a>
......
1084 1088
                            </DL><p>
1085 1089
                            <DT><a name="vacuuming" href="#vacuuming"><H3 ADD_DATE="1377911873">vacuuming</H3></a>
1086 1090
                            <DL><p>
1087
                                <DT><A HREF="http://serverfault.com/questions/53376/how-do-i-know-if-the-autovacuumer-in-postgres-8-3-is-actually-working" name="http://serverfault.com/questions/53376/how-do-i-know-if-the-autovacuumer-in-postgres-8-3-is-actually-working" ADD_DATE="1377911319" ICON_URI="http://cdn.sstatic.net/serverfault/img/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://cdn.sstatic.net/serverfault/img/favicon.ico" style="margin-right: 0.5em;"/>postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault</A><a name="postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault" href="#postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault" style="margin-left: 0.5em;">&para;</a>
1088
<DD>&quot;pg_stat_user_tables has columns *last_autovacuum* and *last_autoanalyze*. They will be updated whenever it does something.&quot;
1089 1091
                                <DT><A HREF="http://blog.gocept.com/2012/05/22/dont-stop-postgresqls-autovacuum-with-your-application/" name="http://blog.gocept.com/2012/05/22/dont-stop-postgresqls-autovacuum-with-your-application/" ADD_DATE="1377911959" ICON_URI="http://1.gravatar.com/blavatar/b99b16aa78bd25f6aa8d01f9c1c99fb4?s=16" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://1.gravatar.com/blavatar/b99b16aa78bd25f6aa8d01f9c1c99fb4?s=16" style="margin-right: 0.5em;"/>Don’t stop PostgreSQL’s autovacuum with your application | gocept weblog</A><a name="Don’t stop PostgreSQL’s autovacuum with your application | gocept weblog" href="#Don’t stop PostgreSQL’s autovacuum with your application | gocept weblog" style="margin-left: 0.5em;">&para;</a>
1090 1092
<DD>page's own description: The problem Some weeks ago, we received a complaint from a customer about bad PostgreSQL performance for a specific application. I took a look into the database and found strange things going on: the query planner was executing &quot;interesting&quot; query plans, tables were bloated with lots of dead rows (one was 6 times as big…
1091 1093

  
......
1094 1096
&quot;autovacuum will abort if it is not able to obtain a table lock within one second&quot;
1095 1097
&quot;postgres[13251]: [40-1] user=,db= ERROR:  canceling autovacuum task&quot;
1096 1098
&quot;I have installed a nightly cron job to force-vacuum the database. PostgreSQL has shown much improved query responses since then. Some queries’ completion times even improved by a factor of 10.&quot;
1099
                                <DT><A HREF="http://postgresql.1045698.n5.nabble.com/How-can-I-tell-if-I-m-autovacuuming-td2085097.html" name="http://postgresql.1045698.n5.nabble.com/How-can-I-tell-if-I-m-autovacuuming-td2085097.html" ADD_DATE="1394169885" ICON_URI="http://postgresql.1045698.n5.nabble.com/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://postgresql.1045698.n5.nabble.com/favicon.ico" style="margin-right: 0.5em;"/>** PostgreSQL - admin - How can I tell if I&#39;m autovacuuming?</A><a name="** PostgreSQL - admin - How can I tell if I&#39;m autovacuuming?" href="#** PostgreSQL - admin - How can I tell if I&#39;m autovacuuming?" style="margin-left: 0.5em;">&para;</a>
1100
<DD>page's own description: How can I tell if I&#39;m autovacuuming?. From what I see in the docs, these three settings in postgresql.conf should be enough for PostgreSQL (8.2) to autovacuum with the default settings: autovacuum...
1101

  
1102
&quot;&gt; (b) How can I verify that my databases are being vacuumed by autovacuum?
1103

  
1104
pg_stat_user_tables should have a last_autovac column.&quot;
1105

  
1106
&quot;you can see autovacuum related messages like:
1107

  
1108
DEBUG:  autovacuum: processing database &quot;postgres&quot;
1109
DEBUG:  autovacuum: processing database &quot;template1&quot;
1110

  
1111
in your server log, by setting log_min_messages to &#39;debug1&#39; in your
1112
postgresql.conf.&quot;
1097 1113
                                <DT><A HREF="http://postgresql.1045698.n5.nabble.com/Lock-problem-with-autovacuum-truncating-heap-td4265539.html" name="http://postgresql.1045698.n5.nabble.com/Lock-problem-with-autovacuum-truncating-heap-td4265539.html" ADD_DATE="1377912054" ICON_URI="http://postgresql.1045698.n5.nabble.com/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://postgresql.1045698.n5.nabble.com/favicon.ico" style="margin-right: 0.5em;"/>PostgreSQL - hackers - Lock problem with autovacuum truncating heap</A><a name="PostgreSQL - hackers - Lock problem with autovacuum truncating heap" href="#PostgreSQL - hackers - Lock problem with autovacuum truncating heap" style="margin-left: 0.5em;">&para;</a>
1098 1114
<DD>page's own description: Lock problem with autovacuum truncating heap. We have run across a problem with autovacuum that occurs when it can truncate off a large amount of empty blocks. It behaves different in version 9.0...
1099 1115

  
......
1101 1117
another transaction starts waiting for a lock. The result is that even a
1102 1118
simple INSERT will take 1 second.&quot;
1103 1119
this means that autovacuum will never complete if clients keep making INSERTs :(
1120
                                <DT><A HREF="http://serverfault.com/questions/53376/how-do-i-know-if-the-autovacuumer-in-postgres-8-3-is-actually-working" name="http://serverfault.com/questions/53376/how-do-i-know-if-the-autovacuumer-in-postgres-8-3-is-actually-working" ADD_DATE="1377911319" ICON_URI="http://cdn.sstatic.net/serverfault/img/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://cdn.sstatic.net/serverfault/img/favicon.ico" style="margin-right: 0.5em;"/>postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault</A><a name="postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault" href="#postgresql - How do I know if the autovacuumer in postgres 8.3 is actually working? - Server Fault" style="margin-left: 0.5em;">&para;</a>
1121
<DD>&quot;pg_stat_user_tables has columns *last_autovacuum* and *last_autoanalyze*. They will be updated whenever it does something.&quot;
1104 1122
                            </DL><p>
1105 1123
                            <DT><A HREF="http://journal.tianhao.info/2010/12/postgresql-change-default-encoding-of-new-databases-to-utf-8-optional/" name="http://journal.tianhao.info/2010/12/postgresql-change-default-encoding-of-new-databases-to-utf-8-optional/" ADD_DATE="1335054607" ICON_URI="http://blog.tianhao.info/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="http://blog.tianhao.info/favicon.ico" style="margin-right: 0.5em;"/>» PostgreSQL – Change Default Encoding of New Databases To UTF-8 (Optional) Journal</A><a name="» PostgreSQL – Change Default Encoding of New Databases To UTF-8 (Optional) Journal" href="#» PostgreSQL – Change Default Encoding of New Databases To UTF-8 (Optional) Journal" style="margin-left: 0.5em;">&para;</a>
1106 1124
                            <DT><A HREF="https://groups.google.com/forum/?fromgroups#!topic/pgsql.general/aitDYC6jMEI" name="https://groups.google.com/forum/?fromgroups#!topic/pgsql.general/aitDYC6jMEI" ADD_DATE="1340396170" ICON_URI="https://groups.google.com/forum/favicon.ico" LAST_CHARSET="UTF-8"><img width="16" height="16" src="https://groups.google.com/forum/favicon.ico" style="margin-right: 0.5em;"/>[GENERAL] ROLLBACK in a function - Google Groups</A><a name="[GENERAL] ROLLBACK in a function - Google Groups" href="#[GENERAL] ROLLBACK in a function - Google Groups" style="margin-left: 0.5em;">&para;</a>
......
1933 1951
                        <DT><a name="scrollback" href="#scrollback"><H3 ADD_DATE="1370801942">scrollback</H3></a>
1934 1952
<DD>note that viewing scrollback is much more difficult to do in screen than in the surrounding terminal program (which usually has graphical scrollbars). this is because screen provided scrolling before GUIs were available, so it had to rely on terminal rendering and keyboard commands.
1935 1953

  
1936
the ability to reconnect to a running shell (which is what screen is primarily used for nowadays) was actually an extra feature, rather than the primary purpose (providing scrollback), so it is not as well-engineered for that particular purpose as today&#39;s GUI users would like. for a better way to support reconnecting, see <a href="http://wiki.vegpath.org/Shell_commands#persistent-shells" name="http://wiki.vegpath.org/Shell_commands#persistent-shells">persistent shells</a><a name="persistent shells" href="#persistent shells" style="margin-left: 0.5em;">&para;</a>.
1954
the ability to reconnect to a running shell (which is what screen is primarily used for nowadays) was actually an extra feature, rather than the primary purpose (providing scrollback), so it is not as well-engineered for that particular purpose as today&#39;s GUI users would like. for a better way to support reconnecting, see <a href="http://wiki.vegpath.org/Shell_commands#persistent-shells">persistent shells</a>.
1937 1955
</DD>
1938 1956
                        <DL><p>
1939 1957
                            <DT><A HREF="http://www.samsarin.com/blog/2007/03/11/gnu-screen-working-with-the-scrollback-buffer/" name="http://www.samsarin.com/blog/2007/03/11/gnu-screen-working-with-the-scrollback-buffer/" ADD_DATE="1370787387" LAST_CHARSET="UTF-8"><img width="16" height="16" src="blank.gif" style="margin-right: 0.5em;"/>GNU Screen: Working with the Scrollback Buffer : Samsarin</A><a name="GNU Screen: Working with the Scrollback Buffer : Samsarin" href="#GNU Screen: Working with the Scrollback Buffer : Samsarin" style="margin-left: 0.5em;">&para;</a>

Also available in: Unified diff