Release Notes The release notes contain the significant changes in each PostgreSQL release, with major features and migration issues listed at the top. The release notes do not contain changes that affect only a few users or changes that are internal and therefore not user-visible. For example, the optimizer is improved in almost every release, but the improvements are usually observed by users as simply faster queries. A complete list of changes for each release can be obtained by viewing the CVS logs for each release. The pgsql-committers email list contains all source code changes as well. There is also a web interface that shows changes to specific files. The name appearing next to each item represents the major developer for that item. Of course all changes involve community discussion and patch review, so each item is truly a community effort. __________________________________________________________________ Release 8.3.5 Release date: 2008-11-03 This release contains a variety of fixes from 8.3.4. For information about new features in the 8.3 major release, see the Section called Release 8.3. __________________________________________________________________ Migration to Version 8.3.5 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see the release notes for 8.3.1. __________________________________________________________________ Changes * Fix GiST index corruption due to marking the wrong index entry "dead" after a deletion (Teodor) This would result in index searches failing to find rows they should have found. * Fix backend crash when the client encoding cannot represent a localized error message (Tom) We have addressed similar issues before, but it would still fail if the "character has no equivalent" message itself couldn't be converted. The fix is to disable localization and send the plain ASCII error message when we detect such a situation. * Fix possible crash in bytea-to-XML mapping (Michael McMaster) * Fix possible crash when deeply nested functions are invoked from a trigger (Tom) * Improve optimization of "expression" IN ("expression-list") queries (Tom, per an idea from Robert Haas) Cases in which there are query variables on the right-hand side had been handled less efficiently in 8.2.x and 8.3.x than in prior versions. The fix restores 8.1 behavior for such cases. * Fix mis-expansion of rule queries when a sub-SELECT appears in a function call in FROM, a multi-row VALUES list, or a RETURNING list (Tom) The usual symptom of this problem is an "unrecognized node type" error. * Fix Assert failure during rescan of an IS NULL search of a GiST index (Teodor) * Fix memory leak during rescan of a hashed aggregation plan (Neil) * Ensure an error is reported when a newly-defined PL/pgSQL trigger function is invoked as a normal function (Tom) * Force a checkpoint before "CREATE DATABASE" starts to copy files (Heikki) This prevents a possible failure if files had recently been deleted in the source database. * Prevent possible collision of relfilenode numbers when moving a table to another tablespace with "ALTER SET TABLESPACE" (Heikki) The command tried to re-use the existing filename, instead of picking one that is known unused in the destination directory. * Fix incorrect text search headline generation when single query item matches first word of text (Sushant Sinha) * Fix improper display of fractional seconds in interval values when using a non-ISO datestyle in an "--enable-integer-datetimes" build (Ron Mayer) * Make ILIKE compare characters case-insensitively even when they're escaped (Andrew) * Ensure "DISCARD" is handled properly by statement logging (Tom) * Fix incorrect logging of last-completed-transaction time during PITR recovery (Tom) * Ensure SPI_getvalue and SPI_getbinval behave correctly when the passed tuple and tuple descriptor have different numbers of columns (Tom) This situation is normal when a table has had columns added or removed, but these two functions didn't handle it properly. The only likely consequence is an incorrect error indication. * Mark SessionReplicationRole as PGDLLIMPORT so it can be used by Slony on Windows (Magnus) * Fix small memory leak when using libpq's gsslib parameter (Magnus) The space used by the parameter string was not freed at connection close. * Ensure libgssapi is linked into libpq if needed (Markus Schaaf) * Fix ecpg's parsing of "CREATE ROLE" (Michael) * Fix recent breakage of pg_ctl restart (Tom) * Ensure "pg_control" is opened in binary mode (Itagaki Takahiro) pg_controldata and pg_resetxlog did this incorrectly, and so could fail on Windows. * Update time zone data files to tzdata release 2008i (for DST law changes in Argentina, Brazil, Mauritius, Syria) __________________________________________________________________ Release 8.3.4 Release date: 2008-09-22 This release contains a variety of fixes from 8.3.3. For information about new features in the 8.3 major release, see the Section called Release 8.3. __________________________________________________________________ Migration to Version 8.3.4 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see the release notes for 8.3.1. __________________________________________________________________ Changes * Fix bug in btree WAL recovery code (Heikki) Recovery failed if the WAL ended partway through a page split operation. * Fix potential use of wrong cutoff XID for HOT page pruning (Alvaro) This error created a risk of corruption in system catalogs that are consulted by "VACUUM": dead tuple versions might be removed too soon. The impact of this on actual database operations would be minimal, since the system doesn't follow MVCC rules while examining catalogs, but it might result in transiently wrong output from pg_dump or other client programs. * Fix potential miscalculation of datfrozenxid (Alvaro) This error may explain some recent reports of failure to remove old pg_clog data. * Fix incorrect HOT updates after pg_class is reindexed (Tom) Corruption of pg_class could occur if REINDEX TABLE pg_class was followed in the same session by an ALTER TABLE RENAME or ALTER TABLE SET SCHEMA command. * Fix missed "combo cid" case (Karl Schnaitter) This error made rows incorrectly invisible to a transaction in which they had been deleted by multiple subtransactions that all aborted. * Prevent autovacuum from crashing if the table it's currently checking is deleted at just the wrong time (Alvaro) * Widen local lock counters from 32 to 64 bits (Tom) This responds to reports that the counters could overflow in sufficiently long transactions, leading to unexpected "lock is already held" errors. * Fix possible duplicate output of tuples during a GiST index scan (Teodor) * Regenerate foreign key checking queries from scratch when either table is modified (Tom) Previously, 8.3 would attempt to replan the query, but would work from previously generated query text. This led to failures if a table or column was renamed. * Fix missed permissions checks when a view contains a simple UNION ALL construct (Heikki) Permissions for the referenced tables were checked properly, but not permissions for the view itself. * Add checks in executor startup to ensure that the tuples produced by an "INSERT" or "UPDATE" will match the target table's current rowtype (Tom) This situation is believed to be impossible in 8.3, but it can happen in prior releases, so a check seems prudent. * Fix possible repeated drops during "DROP OWNED" (Tom) This would typically result in strange errors such as "cache lookup failed for relation NNN". * Fix several memory leaks in XML operations (Kris Jurka, Tom) * Fix xmlserialize() to raise error properly for unacceptable target data type (Tom) * Fix a couple of places that mis-handled multibyte characters in text search configuration file parsing (Tom) Certain characters occurring in configuration files would always cause "invalid byte sequence for encoding" failures. * Provide file name and line number location for all errors reported in text search configuration files (Tom) * Fix AT TIME ZONE to first try to interpret its timezone argument as a timezone abbreviation, and only try it as a full timezone name if that fails, rather than the other way around as formerly (Tom) The timestamp input functions have always resolved ambiguous zone names in this order. Making AT TIME ZONE do so as well improves consistency, and fixes a compatibility bug introduced in 8.1: in ambiguous cases we now behave the same as 8.0 and before did, since in the older versions AT TIME ZONE accepted *only* abbreviations. * Fix datetime input functions to correctly detect integer overflow when running on a 64-bit platform (Tom) * Prevent integer overflows during units conversion when displaying a configuration parameter that has units (Tom) * Improve performance of writing very long log messages to syslog (Tom) * Allow spaces in the suffix part of an LDAP URL in "pg_hba.conf" (Tom) * Fix bug in backwards scanning of a cursor on a SELECT DISTINCT ON query (Tom) * Fix planner bug that could improperly push down IS NULL tests below an outer join (Tom) This was triggered by occurrence of IS NULL tests for the same relation in all arms of an upper OR clause. * Fix planner bug with nested sub-select expressions (Tom) If the outer sub-select has no direct dependency on the parent query, but the inner one does, the outer value might not get recalculated for new parent query rows. * Fix planner to estimate that GROUP BY expressions yielding boolean results always result in two groups, regardless of the expressions' contents (Tom) This is very substantially more accurate than the regular GROUP BY estimate for certain boolean tests like "col" IS NULL. * Fix PL/PgSQL to not fail when a FOR loop's target variable is a record containing composite-type fields (Tom) * Fix PL/Tcl to behave correctly with Tcl 8.5, and to be more careful about the encoding of data sent to or from Tcl (Tom) * Improve performance of PQescapeBytea() (Rudolf Leitgeb) * On Windows, work around a Microsoft bug by preventing libpq from trying to send more than 64kB per system call (Magnus) * Fix ecpg to handle variables properly in "SET" commands (Michael) * Improve pg_dump and pg_restore's error reporting after failure to send a SQL command (Tom) * Fix pg_ctl to properly preserve postmaster command-line arguments across a restart (Bruce) * Fix erroneous WAL file cutoff point calculation in pg_standby (Simon) * Update time zone data files to tzdata release 2008f (for DST law changes in Argentina, Bahamas, Brazil, Mauritius, Morocco, Pakistan, Palestine, and Paraguay) __________________________________________________________________ Release 8.3.3 Release date: 2008-06-12 This release contains one serious and one minor bug fix over 8.3.2. For information about new features in the 8.3 major release, see the Section called Release 8.3. __________________________________________________________________ Migration to Version 8.3.3 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see the release notes for 8.3.1. __________________________________________________________________ Changes * Make pg_get_ruledef() parenthesize negative constants (Tom) Before this fix, a negative constant in a view or rule might be dumped as, say, -42::integer, which is subtly incorrect: it should be (-42)::integer due to operator precedence rules. Usually this would make little difference, but it could interact with another recent patch to cause PostgreSQL to reject what had been a valid "SELECT DISTINCT" view query. Since this could result in pg_dump output failing to reload, it is being treated as a high-priority fix. The only released versions in which dump output is actually incorrect are 8.3.1 and 8.2.7. * Make "ALTER AGGREGATE ... OWNER TO" update pg_shdepend (Tom) This oversight could lead to problems if the aggregate was later involved in a "DROP OWNED" or "REASSIGN OWNED" operation. __________________________________________________________________ Release 8.3.2 Release date: never released This release contains a variety of fixes from 8.3.1. For information about new features in the 8.3 major release, see the Section called Release 8.3. __________________________________________________________________ Migration to Version 8.3.2 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see the release notes for 8.3.1. __________________________________________________________________ Changes * Fix ERRORDATA_STACK_SIZE exceeded crash that occurred on Windows when using UTF-8 database encoding and a different client encoding (Tom) * Fix incorrect archive truncation point calculation for the %r macro in recovery_command parameters (Simon) This could lead to data loss if a warm-standby script relied on %r to decide when to throw away WAL segment files. * Fix "ALTER TABLE ADD COLUMN ... PRIMARY KEY" so that the new column is correctly checked to see if it's been initialized to all non-nulls (Brendan Jurd) Previous versions neglected to check this requirement at all. * Fix "REASSIGN OWNED" so that it works on procedural languages too (Alvaro) * Fix problems with "SELECT FOR UPDATE/SHARE" occurring as a subquery in a query with a non-"SELECT" top-level operation (Tom) * Fix possible "CREATE TABLE" failure when inheriting the "same" constraint from multiple parent relations that inherited that constraint from a common ancestor (Tom) * Fix pg_get_ruledef() to show the alias, if any, attached to the target table of an "UPDATE" or "DELETE" (Tom) * Restore the pre-8.3 behavior that an out-of-range block number in a TID being used in a TidScan plan results in silently not matching any rows (Tom) 8.3.0 and 8.3.1 threw an error instead. * Fix GIN bug that could result in a too many LWLocks taken failure (Teodor) * Fix broken GiST comparison function for tsquery (Teodor) * Fix tsvector_update_trigger() and ts_stat() to accept domains over the types they expect to work with (Tom) * Fix failure to support enum data types as foreign keys (Tom) * Avoid possible crash when decompressing corrupted data (Zdenek Kotala) * Fix race conditions between delayed unlinks and "DROP DATABASE" (Heikki) In the worst case this could result in deleting a newly created table in a new database that happened to get the same OID as the recently-dropped one; but of course that is an extremely low-probability scenario. * Repair two places where SIGTERM exit of a backend could leave corrupted state in shared memory (Tom) Neither case is very important if SIGTERM is used to shut down the whole database cluster together, but there was a problem if someone tried to SIGTERM individual backends. * Fix possible crash due to incorrect plan generated for an x IN (SELECT y FROM ...) clause when "x" and "y" have different data types; and make sure the behavior is semantically correct when the conversion from "y"'s type to "x"'s type is lossy (Tom) * Fix oversight that prevented the planner from substituting known Param values as if they were constants (Tom) This mistake partially disabled optimization of unnamed extended-Query statements in 8.3.0 and 8.3.1: in particular the LIKE-to-indexscan optimization would never be applied if the LIKE pattern was passed as a parameter, and constraint exclusion depending on a parameter value didn't work either. * Fix planner failure when an indexable MIN or MAX aggregate is used with DISTINCT or ORDER BY (Tom) * Fix planner to ensure it never uses a "physical tlist" for a plan node that is feeding a Sort node (Tom) This led to the sort having to push around more data than it really needed to, since unused column values were included in the sorted data. * Avoid unnecessary copying of query strings (Tom) This fixes a performance problem introduced in 8.3.0 when a very large number of commands are submitted as a single query string. * Make TransactionIdIsCurrentTransactionId() use binary search instead of linear search when checking child-transaction XIDs (Heikki) This fixes some cases in which 8.3.0 was significantly slower than earlier releases. * Fix conversions between ISO-8859-5 and other encodings to handle Cyrillic "Yo" characters (e and E with two dots) (Sergey Burladyan) * Fix several datatype input functions, notably array_in(), that were allowing unused bytes in their results to contain uninitialized, unpredictable values (Tom) This could lead to failures in which two apparently identical literal values were not seen as equal, resulting in the parser complaining about unmatched ORDER BY and DISTINCT expressions. * Fix a corner case in regular-expression substring matching (substring(string from pattern)) (Tom) The problem occurs when there is a match to the pattern overall but the user has specified a parenthesized subexpression and that subexpression hasn't got a match. An example is substring('foo' from 'foo(bar)?'). This should return NULL, since (bar) isn't matched, but it was mistakenly returning the whole-pattern match instead (ie, foo). * Prevent cancellation of an auto-vacuum that was launched to prevent XID wraparound (Alvaro) * Improve "ANALYZE"'s handling of in-doubt tuples (those inserted or deleted by a not-yet-committed transaction) so that the counts it reports to the stats collector are more likely to be correct (Pavan Deolasee) * Fix initdb to reject a relative path for its --xlogdir (-X) option (Tom) * Make psql print tab characters as an appropriate number of spaces, rather than \x09 as was done in 8.3.0 and 8.3.1 (Bruce) * Update time zone data files to tzdata release 2008c (for DST law changes in Morocco, Iraq, Choibalsan, Pakistan, Syria, Cuba, and Argentina/San_Luis) * Add ECPGget_PGconn() function to ecpglib (Michael) * Fix incorrect result from ecpg's PGTYPEStimestamp_sub() function (Michael) * Fix handling of continuation line markers in ecpg (Michael) * Fix possible crashes in "contrib/cube" functions (Tom) * Fix core dump in "contrib/xml2"'s xpath_table() function when the input query returns a NULL value (Tom) * Fix "contrib/xml2"'s makefile to not override CFLAGS, and make it auto-configure properly for libxslt present or not (Tom) __________________________________________________________________ Release 8.3.1 Release date: 2008-03-17 This release contains a variety of fixes from 8.3.0. For information about new features in the 8.3 major release, see the Section called Release 8.3. __________________________________________________________________ Migration to Version 8.3.1 A dump/restore is not required for those running 8.3.X. However, you might need to "REINDEX" indexes on textual columns after updating, if you are affected by the Windows locale issue described below. __________________________________________________________________ Changes * Fix character string comparison for Windows locales that consider different character combinations as equal (Tom) This fix applies only on Windows and only when using UTF-8 database encoding. The same fix was made for all other cases over two years ago, but Windows with UTF-8 uses a separate code path that was not updated. If you are using a locale that considers some non-identical strings as equal, you may need to "REINDEX" to fix existing indexes on textual columns. * Repair corner-case bugs in "VACUUM FULL" (Tom) A potential deadlock between concurrent "VACUUM FULL" operations on different system catalogs was introduced in 8.2. This has now been corrected. 8.3 made this worse because the deadlock could occur within a critical code section, making it a PANIC rather than just ERROR condition. Also, a "VACUUM FULL" that failed partway through vacuuming a system catalog could result in cache corruption in concurrent database sessions. Another "VACUUM FULL" bug introduced in 8.3 could result in a crash or out-of-memory report when dealing with pages containing no live tuples. * Fix misbehavior of foreign key checks involving character or bit columns (Tom) If the referencing column were of a different but compatible type (for instance varchar), the constraint was enforced incorrectly. * Avoid needless deadlock failures in no-op foreign-key checks (Stephan Szabo, Tom) * Fix possible core dump when re-planning a prepared query (Tom) This bug affected only protocol-level prepare operations, not SQL "PREPARE", and so tended to be seen only with JDBC, DBI, and other client-side drivers that use prepared statements heavily. * Fix possible failure when re-planning a query that calls an SPI-using function (Tom) * Fix failure in row-wise comparisons involving columns of different datatypes (Tom) * Fix longstanding "LISTEN"/"NOTIFY" race condition (Tom) In rare cases a session that had just executed a "LISTEN" might not get a notification, even though one would be expected because the concurrent transaction executing "NOTIFY" was observed to commit later. A side effect of the fix is that a transaction that has executed a not-yet-committed "LISTEN" command will not see any row in pg_listener for the "LISTEN", should it choose to look; formerly it would have. This behavior was never documented one way or the other, but it is possible that some applications depend on the old behavior. * Disallow "LISTEN" and "UNLISTEN" within a prepared transaction (Tom) This was formerly allowed but trying to do it had various unpleasant consequences, notably that the originating backend could not exit as long as an "UNLISTEN" remained uncommitted. * Disallow dropping a temporary table within a prepared transaction (Heikki) This was correctly disallowed by 8.1, but the check was inadvertently broken in 8.2 and 8.3. * Fix rare crash when an error occurs during a query using a hash index (Heikki) * Fix incorrect comparison of tsquery values (Teodor) * Fix incorrect behavior of LIKE with non-ASCII characters in single-byte encodings (Rolf Jentsch) * Disable xmlvalidate (Tom) This function should have been removed before 8.3 release, but was inadvertently left in the source code. It poses a small security risk since unprivileged users could use it to read the first few characters of any file accessible to the server. * Fix memory leaks in certain usages of set-returning functions (Neil) * Make encode(bytea, 'escape') convert all high-bit-set byte values into \"nnn" octal escape sequences (Tom) This is necessary to avoid encoding problems when the database encoding is multi-byte. This change could pose compatibility issues for applications that are expecting specific results from encode. * Fix input of datetime values for February 29 in years BC (Tom) The former coding was mistaken about which years were leap years. * Fix "unrecognized node type" error in some variants of "ALTER OWNER" (Tom) * Avoid tablespace permissions errors in "CREATE TABLE LIKE INCLUDING INDEXES" (Tom) * Ensure pg_stat_activity.waiting flag is cleared when a lock wait is aborted (Tom) * Fix handling of process permissions on Windows Vista (Dave, Magnus) In particular, this fix allows starting the server as the Administrator user. * Update time zone data files to tzdata release 2008a (in particular, recent Chile changes); adjust timezone abbreviation VET (Venezuela) to mean UTC-4:30, not UTC-4:00 (Tom) * Fix ecpg problems with arrays (Michael) * Fix pg_ctl to correctly extract the postmaster's port number from command-line options (Itagaki Takahiro, Tom) Previously, pg_ctl start -w could try to contact the postmaster on the wrong port, leading to bogus reports of startup failure. * Use "-fwrapv" to defend against possible misoptimization in recent gcc versions (Tom) This is known to be necessary when building PostgreSQL with gcc 4.3 or later. * Enable building "contrib/uuid-ossp" with MSVC (Hiroshi Saito) __________________________________________________________________ Release 8.3 Release date: 2008-02-04 __________________________________________________________________ Overview With significant new functionality and performance enhancements, this release represents a major leap forward for PostgreSQL. This was made possible by a growing community that has dramatically accelerated the pace of development. This release adds the following major features: * Full text search is integrated into the core database system * Support for the SQL/XML standard, including new operators and an XML data type * Enumerated data types (ENUM) * Arrays of composite types * Universally Unique Identifier (UUID) data type * Add control over whether NULLs sort first or last * Updatable cursors * Server configuration parameters can now be set on a per-function basis * User-defined types can now have type modifiers * Automatically re-plan cached queries when table definitions change or statistics are updated * Numerous improvements in logging and statistics collection * Support Security Service Provider Interface (SSPI) for authentication on Windows * Support multiple concurrent autovacuum processes, and other autovacuum improvements * Allow the whole PostgreSQL distribution to be compiled with Microsoft Visual C++ Major performance improvements are listed below. Most of these enhancements are automatic and do not require user changes or tuning: * Asynchronous commit delays writes to WAL during transaction commit * Checkpoint writes can be spread over a longer time period to smooth the I/O spike during each checkpoint * Heap-Only Tuples (HOT) accelerate space reuse for most "UPDATE"s and "DELETE"s * Just-in-time background writer strategy improves disk write efficiency * Using non-persistent transaction IDs for read-only transactions reduces overhead and "VACUUM" requirements * Per-field and per-row storage overhead has been reduced * Large sequential scans no longer force out frequently used cached pages * Concurrent large sequential scans can now share disk reads * ORDER BY ... LIMIT can be done without sorting The above items are explained in more detail in the sections below. __________________________________________________________________ Migration to Version 8.3 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities: __________________________________________________________________ General * Non-character data types are no longer automatically cast to TEXT (Peter, Tom) Previously, if a non-character value was supplied to an operator or function that requires text input, it was automatically cast to text, for most (though not all) built-in data types. This no longer happens: an explicit cast to text is now required for all non-character-string types. For example, these expressions formerly worked: substr(current_date, 1, 4) 23 LIKE '2%' but will now draw "function does not exist" and "operator does not exist" errors respectively. Use an explicit cast instead: substr(current_date::text, 1, 4) 23::text LIKE '2%' (Of course, you can use the more verbose CAST() syntax too.) The reason for the change is that these automatic casts too often caused surprising behavior. An example is that in previous releases, this expression was accepted but did not do what was expected: current_date < 2017-11-17 This is actually comparing a date to an integer, which should be (and now is) rejected -- but in the presence of automatic casts both sides were cast to text and a textual comparison was done, because the text < text operator was able to match the expression when no other < operator could. Types char(n) and varchar(n) still cast to text automatically. Also, automatic casting to text still works for inputs to the concatenation (||) operator, so long as least one input is a character-string type. * Full text search features from "contrib/tsearch2" have been moved into the core server, with some minor syntax changes "contrib/tsearch2" now contains a compatibility interface. * ARRAY(SELECT ...), where the "SELECT" returns no rows, now returns an empty array, rather than NULL (Tom) * The array type name for a base data type is no longer always the base type's name with an underscore prefix The old naming convention is still honored when possible, but application code should no longer depend on it. Instead use the new pg_type.typarray column to identify the array data type associated with a given type. * ORDER BY ... USING "operator" must now use a less-than or greater-than "operator" that is defined in a btree operator class This restriction was added to prevent inconsistent results. * "SET LOCAL" changes now persist until the end of the outermost transaction, unless rolled back (Tom) Previously "SET LOCAL"'s effects were lost after subtransaction commit ("RELEASE SAVEPOINT" or exit from a PL/pgSQL exception block). * Commands rejected in transaction blocks are now also rejected in multiple-statement query strings (Tom) For example, "BEGIN; DROP DATABASE; COMMIT" will now be rejected even if submitted as a single query message. * "ROLLBACK" outside a transaction block now issues NOTICE instead of WARNING (Bruce) * Prevent "NOTIFY"/"LISTEN"/"UNLISTEN" from accepting schema-qualified names (Bruce) Formerly, these commands accepted schema.relation but ignored the schema part, which was confusing. * "ALTER SEQUENCE" no longer affects the sequence's currval() state (Tom) * Foreign keys now must match indexable conditions for cross-data-type references (Tom) This improves semantic consistency and helps avoid performance problems. * Restrict object size functions to users who have reasonable permissions to view such information (Tom) For example, pg_database_size() now requires CONNECT permission, which is granted to everyone by default. pg_tablespace_size() requires CREATE permission in the tablespace, or is allowed if the tablespace is the default tablespace for the database. * Remove the undocumented !!= (not in) operator (Tom) NOT IN (SELECT ...) is the proper way to perform this operation. * Internal hashing functions are now more uniformly-distributed (Tom) If application code was computing and storing hash values using internal PostgreSQL hashing functions, the hash values must be regenerated. * C-code conventions for handling variable-length data values have changed (Greg Stark, Tom) The new SET_VARSIZE() macro *must* be used to set the length of generated varlena values. Also, it might be necessary to expand ("de-TOAST") input values in more cases. * Continuous archiving no longer reports each successful archive operation to the server logs unless DEBUG level is used (Simon) __________________________________________________________________ Configuration Parameters * Numerous changes in administrative server parameters bgwriter_lru_percent, bgwriter_all_percent, bgwriter_all_maxpages, stats_start_collector, and stats_reset_on_server_start are removed. redirect_stderr is renamed to logging_collector. stats_command_string is renamed to track_activities. stats_block_level and stats_row_level are merged into track_counts. A new boolean configuration parameter, archive_mode, controls archiving. Autovacuum's default settings have changed. * Remove stats_start_collector parameter (Tom) We now always start the collector process, unless UDP socket creation fails. * Remove stats_reset_on_server_start parameter (Tom) This was removed because pg_stat_reset() can be used for this purpose. * Commenting out a parameter in "postgresql.conf" now causes it to revert to its default value (Joachim Wieland) Previously, commenting out an entry left the parameter's value unchanged until the next server restart. __________________________________________________________________ Character Encodings * Add more checks for invalidly-encoded data (Andrew) This change plugs some holes that existed in literal backslash escape string processing and "COPY" escape processing. Now the de-escaped string is rechecked to see if the result created an invalid multi-byte character. * Disallow database encodings that are inconsistent with the server's locale setting (Tom) On most platforms, C locale is the only locale that will work with any database encoding. Other locale settings imply a specific encoding and will misbehave if the database encoding is something different. (Typical symptoms include bogus textual sort order and wrong results from upper() or lower().) The server now rejects attempts to create databases that have an incompatible encoding. * Ensure that chr() cannot create invalidly-encoded values (Andrew) In UTF8-encoded databases the argument of chr() is now treated as a Unicode code point. In other multi-byte encodings chr()'s argument must designate a 7-bit ASCII character. Zero is no longer accepted. ascii() has been adjusted to match. * Adjust convert() behavior to ensure encoding validity (Andrew) The two argument form of convert() has been removed. The three argument form now takes a bytea first argument and returns a bytea. To cover the loss of functionality, three new functions have been added: + convert_from(bytea, name) returns text -- converts the first argument from the named encoding to the database encoding + convert_to(text, name) returns bytea -- converts the first argument from the database encoding to the named encoding + length(bytea, name) returns integer -- gives the length of the first argument in characters in the named encoding * Remove convert(argument USING conversion_name) (Andrew) Its behavior did not match the SQL standard. * Make JOHAB encoding client-only (Tatsuo) JOHAB is not safe as a server-side encoding. __________________________________________________________________ Changes Below you will find a detailed account of the changes between PostgreSQL 8.3 and the previous major release. __________________________________________________________________ Performance * Asynchronous commit delays writes to WAL during transaction commit (Simon) This feature dramatically increases performance for short data-modifying transactions. The disadvantage is that because disk writes are delayed, if the database or operating system crashes before data is written to the disk, committed data will be lost. This feature is useful for applications that can accept some data loss. Unlike turning off fsync, using asynchronous commit does not put database consistency at risk; the worst case is that after a crash the last few reportedly-committed transactions might not be committed after all. This feature is enabled by turning off synchronous_commit (which can be done per-session or per-transaction, if some transactions are critical and others are not). wal_writer_delay can be adjusted to control the maximum delay before transactions actually reach disk. * Checkpoint writes can be spread over a longer time period to smooth the I/O spike during each checkpoint (Itagaki Takahiro and Heikki Linnakangas) Previously all modified buffers were forced to disk as quickly as possible during a checkpoint, causing an I/O spike that decreased server performance. This new approach spreads out disk writes during checkpoints, reducing peak I/O usage. (User-requested and shutdown checkpoints are still written as quickly as possible.) * Heap-Only Tuples (HOT) accelerate space reuse for most "UPDATE"s and "DELETE"s (Pavan Deolasee, with ideas from many others) "UPDATE"s and "DELETE"s leave dead tuples behind, as do failed "INSERT"s. Previously only "VACUUM" could reclaim space taken by dead tuples. With HOT dead tuple space can be automatically reclaimed at the time of "INSERT" or "UPDATE" if no changes are made to indexed columns. This allows for more consistent performance. Also, HOT avoids adding duplicate index entries. * Just-in-time background writer strategy improves disk write efficiency (Greg Smith, Itagaki Takahiro) This greatly reduces the need for manual tuning of the background writer. * Per-field and per-row storage overhead have been reduced (Greg Stark, Heikki Linnakangas) Variable-length data types with data values less than 128 bytes long will see a storage decrease of 3 to 6 bytes. For example, two adjacent char(1) fields now use 4 bytes instead of 16. Row headers are also 4 bytes shorter than before. * Using non-persistent transaction IDs for read-only transactions reduces overhead and "VACUUM" requirements (Florian Pflug) Non-persistent transaction IDs do not increment the global transaction counter. Therefore, they reduce the load on pg_clog and increase the time between forced vacuums to prevent transaction ID wraparound. Other performance improvements were also made that should improve concurrency. * Avoid incrementing the command counter after a read-only command (Tom) There was formerly a hard limit of 2^32 (4 billion) commands per transaction. Now only commands that actually changed the database count, so while this limit still exists, it should be significantly less annoying. * Create a dedicated WAL writer process to off-load work from backends (Simon) * Skip unnecessary WAL writes for "CLUSTER" and "COPY" (Simon) Unless WAL archiving is enabled, the system now avoids WAL writes for "CLUSTER" and just fsync()s the table at the end of the command. It also does the same for "COPY" if the table was created in the same transaction. * Large sequential scans no longer force out frequently used cached pages (Simon, Heikki, Tom) * Concurrent large sequential scans can now share disk reads (Jeff Davis) This is accomplished by starting the new sequential scan in the middle of the table (where another sequential scan is already in-progress) and wrapping around to the beginning to finish. This can affect the order of returned rows in a query that does not specify ORDER BY. The synchronize_seqscans configuration parameter can be used to disable this if necessary. * ORDER BY ... LIMIT can be done without sorting (Greg Stark) This is done by sequentially scanning the table and tracking just the "top N" candidate rows, rather than performing a full sort of the entire table. This is useful when there is no matching index and the LIMIT is not large. * Put a rate limit on messages sent to the statistics collector by backends (Tom) This reduces overhead for short transactions, but might sometimes increase the delay before statistics are tallied. * Improve hash join performance for cases with many NULLs (Tom) * Speed up operator lookup for cases with non-exact datatype matches (Tom) __________________________________________________________________ Server * Autovacuum is now enabled by default (Alvaro) Several changes were made to eliminate disadvantages of having autovacuum enabled, thereby justifying the change in default. Several other autovacuum parameter defaults were also modified. * Support multiple concurrent autovacuum processes (Alvaro, Itagaki Takahiro) This allows multiple vacuums to run concurrently. This prevents vacuuming of a large table from delaying vacuuming of smaller tables. * Automatically re-plan cached queries when table definitions change or statistics are updated (Tom) Previously PL/PgSQL functions that referenced temporary tables would fail if the temporary table was dropped and recreated between function invocations, unless EXECUTE was used. This improvement fixes that problem and many related issues. * Add a temp_tablespaces parameter to control the tablespaces for temporary tables and files (Jaime Casanova, Albert Cervera, Bernd Helmle) This parameter defines a list of tablespaces to be used. This enables spreading the I/O load across multiple tablespaces. A random tablespace is chosen each time a temporary object is created. Temporary files are no longer stored in per-database "pgsql_tmp/" directories but in per-tablespace directories. * Place temporary tables' TOAST tables in special schemas named pg_toast_temp_nnn (Tom) This allows low-level code to recognize these tables as temporary, which enables various optimizations such as not WAL-logging changes and using local rather than shared buffers for access. This also fixes a bug wherein backends unexpectedly held open file references to temporary TOAST tables. * Fix problem that a constant flow of new connection requests could indefinitely delay the postmaster from completing a shutdown or a crash restart (Tom) * Guard against a very-low-probability data loss scenario by preventing re-use of a deleted table's relfilenode until after the next checkpoint (Heikki) * Fix "CREATE CONSTRAINT TRIGGER" to convert old-style foreign key trigger definitions into regular foreign key constraints (Tom) This will ease porting of foreign key constraints carried forward from pre-7.3 databases, if they were never converted using "contrib/adddepend". * Fix DEFAULT NULL to override inherited defaults (Tom) DEFAULT NULL was formerly considered a noise phrase, but it should (and now does) override non-null defaults that would otherwise be inherited from a parent table or domain. * Add new encodings EUC_JIS_2004 and SHIFT_JIS_2004 (Tatsuo) These new encodings can be converted to and from UTF-8. * Change server startup log message from "database system is ready" to "database system is ready to accept connections", and adjust its timing The message now appears only when the postmaster is really ready to accept connections. __________________________________________________________________ Monitoring * Add log_autovacuum_min_duration parameter to support configurable logging of autovacuum activity (Simon, Alvaro) * Add log_lock_waits parameter to log lock waiting (Simon) * Add log_temp_files parameter to log temporary file usage (Bill Moran) * Add log_checkpoints parameter to improve logging of checkpoints (Greg Smith, Heikki) * log_line_prefix now supports %s and %c escapes in all processes (Andrew) Previously these escapes worked only for user sessions, not for background database processes. * Add log_restartpoints to control logging of point-in-time recovery restart points (Simon) * Last transaction end time is now logged at end of recovery and at each logged restart point (Simon) * Autovacuum now reports its activity start time in pg_stat_activity (Tom) * Allow server log output in comma-separated value (CSV) format (Arul Shaji, Greg Smith, Andrew Dunstan) CSV-format log files can easily be loaded into a database table for subsequent analysis. * Use PostgreSQL-supplied timezone support for formatting timestamps displayed in the server log (Tom) This avoids Windows-specific problems with localized time zone names that are in the wrong encoding. There is a new log_timezone parameter that controls the timezone used in log messages, independently of the client-visible timezone parameter. * New system view pg_stat_bgwriter displays statistics about background writer activity (Magnus) * Add new columns for database-wide tuple statistics to pg_stat_database (Magnus) * Add an xact_start (transaction start time) column to pg_stat_activity (Neil) This makes it easier to identify long-running transactions. * Add n_live_tuples and n_dead_tuples columns to pg_stat_all_tables and related views (Glen Parker) * Merge stats_block_level and stats_row_level parameters into a single parameter track_counts, which controls all messages sent to the statistics collector process (Tom) * Rename stats_command_string parameter to track_activities (Tom) * Fix statistical counting of live and dead tuples to recognize that committed and aborted transactions have different effects (Tom) __________________________________________________________________ Authentication * Support Security Service Provider Interface (SSPI) for authentication on Windows (Magnus) * Support GSSAPI authentication (Henry Hotz, Magnus) This should be preferred to native Kerberos authentication because GSSAPI is an industry standard. * Support a global SSL configuration file (Victor Wagner) * Add ssl_ciphers parameter to control accepted SSL ciphers (Victor Wagner) * Add a Kerberos realm parameter, krb_realm (Magnus) __________________________________________________________________ Write-Ahead Log (WAL) and Continuous Archiving * Change the timestamps recorded in transaction WAL records from time_t to TimestampTz representation (Tom) This provides sub-second resolution in WAL, which can be useful for point-in-time recovery. * Reduce WAL disk space needed by warm standby servers (Simon) This change allows a warm standby server to pass the name of the earliest still-needed WAL file to the recovery script, allowing automatic removal of no-longer-needed WAL files. This is done using %r in the restore_command parameter of "recovery.conf". * New boolean configuration parameter, archive_mode, controls archiving (Simon) Previously setting archive_command to an empty string turned off archiving. Now archive_mode turns archiving on and off, independently of archive_command. This is useful for stopping archiving temporarily. __________________________________________________________________ Queries * Full text search is integrated into the core database system (Teodor, Oleg) Text search has been improved, moved into the core code, and is now installed by default. "contrib/tsearch2" now contains a compatibility interface. * Add control over whether NULLs sort first or last (Teodor, Tom) The syntax is ORDER BY ... NULLS FIRST/LAST. * Allow per-column ascending/descending (ASC/DESC) ordering options for indexes (Teodor, Tom) Previously a query using ORDER BY with mixed ASC/DESC specifiers could not fully use an index. Now an index can be fully used in such cases if the index was created with matching ASC/DESC specifications. NULL sort order within an index can be controlled, too. * Allow col IS NULL to use an index (Teodor) * Updatable cursors (Arul Shaji, Tom) This eliminates the need to reference a primary key to "UPDATE" or "DELETE" rows returned by a cursor. The syntax is UPDATE/DELETE WHERE CURRENT OF. * Allow FOR UPDATE in cursors (Arul Shaji, Tom) * Create a general mechanism that supports casts to and from the standard string types (TEXT, VARCHAR, CHAR) for *every* datatype, by invoking the datatype's I/O functions (Tom) Previously, such casts were available only for types that had specialized function(s) for the purpose. These new casts are assignment-only in the to-string direction, explicit-only in the other direction, and therefore should create no surprising behavior. * Allow UNION and related constructs to return a domain type, when all inputs are of that domain type (Tom) Formerly, the output would be considered to be of the domain's base type. * Allow limited hashing when using two different data types (Tom) This allows hash joins, hash indexes, hashed subplans, and hash aggregation to be used in situations involving cross-data-type comparisons, if the data types have compatible hash functions. Currently, cross-data-type hashing support exists for smallint/integer/bigint, and for float4/float8. * Improve optimizer logic for detecting when variables are equal in a WHERE clause (Tom) This allows mergejoins to work with descending sort orders, and improves recognition of redundant sort columns. * Improve performance when planning large inheritance trees in cases where most tables are excluded by constraints (Tom) __________________________________________________________________ Object Manipulation * Arrays of composite types (David Fetter, Andrew, Tom) In addition to arrays of explicitly-declared composite types, arrays of the rowtypes of regular tables and views are now supported, except for rowtypes of system catalogs, sequences, and TOAST tables. * Server configuration parameters can now be set on a per-function basis (Tom) For example, functions can now set their own search_path to prevent unexpected behavior if a different search_path exists at run-time. Security definer functions should set search_path to avoid security loopholes. * "CREATE/ALTER FUNCTION" now supports COST and ROWS options (Tom) COST allows specification of the cost of a function call. ROWS allows specification of the average number or rows returned by a set-returning function. These values are used by the optimizer in choosing the best plan. * Implement "CREATE TABLE LIKE ... INCLUDING INDEXES" (Trevor Hardcastle, Nikhil Sontakke, Neil) * Allow "CREATE INDEX CONCURRENTLY" to ignore transactions in other databases (Simon) * Add "ALTER VIEW ... RENAME TO" and "ALTER SEQUENCE ... RENAME TO" (David Fetter, Neil) Previously this could only be done via "ALTER TABLE ... RENAME TO". * Make "CREATE/DROP/RENAME DATABASE" wait briefly for conflicting backends to exit before failing (Tom) This increases the likelihood that these commands will succeed. * Allow triggers and rules to be deactivated in groups using a configuration parameter, for replication purposes (Jan) This allows replication systems to disable triggers and rewrite rules as a group without modifying the system catalogs directly. The behavior is controlled by "ALTER TABLE" and a new parameter session_replication_role. * User-defined types can now have type modifiers (Teodor, Tom) This allows a user-defined type to take a modifier, like ssnum(7). Previously only built-in data types could have modifiers. __________________________________________________________________ Utility Commands * Non-superuser database owners now are able to add trusted procedural languages to their databases by default (Jeremy Drake) While this is reasonably safe, some administrators might wish to revoke the privilege. It is controlled by pg_pltemplate.tmpldbacreate. * Allow a session's current parameter setting to be used as the default for future sessions (Tom) This is done with SET ... FROM CURRENT in "CREATE/ALTER FUNCTION", "ALTER DATABASE", or "ALTER ROLE". * Implement new commands "DISCARD ALL", "DISCARD PLANS", "DISCARD TEMPORARY", "CLOSE ALL", and "DEALLOCATE ALL" (Marko Kreen, Neil) These commands simplify resetting a database session to its initial state, and are particularly useful for connection-pooling software. * Make "CLUSTER" MVCC-safe (Heikki Linnakangas) Formerly, "CLUSTER" would discard all tuples that were committed dead, even if there were still transactions that should be able to see them under MVCC visibility rules. * Add new "CLUSTER" syntax: CLUSTER table USING index (Holger Schurig) The old "CLUSTER" syntax is still supported, but the new form is considered more logical. * Fix "EXPLAIN" so it can show complex plans more accurately (Tom) References to subplan outputs are now always shown correctly, instead of using ?columnN? for complicated cases. * Limit the amount of information reported when a user is dropped (Alvaro) Previously, dropping (or attempting to drop) a user who owned many objects could result in large NOTICE or ERROR messages listing all these objects; this caused problems for some client applications. The length of the message is now limited, although a full list is still sent to the server log. __________________________________________________________________ Data Types * Support for the SQL/XML standard, including new operators and an XML data type (Nikolay Samokhvalov, Pavel Stehule, Peter) * Enumerated data types (ENUM) (Tom Dunstan) This feature provides convenient support for fields that have a small, fixed set of allowed values. An example of creating an ENUM type is CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy'). * Universally Unique Identifier (UUID) data type (Gevik Babakhani, Neil) This closely matches RFC 4122. * Widen the MONEY data type to 64 bits (D'Arcy Cain) This greatly increases the range of supported MONEY values. * Fix float4/float8 to handle Infinity and NAN (Not A Number) consistently (Bruce) The code formerly was not consistent about distinguishing Infinity from overflow conditions. * Allow leading and trailing whitespace during input of boolean values (Neil) * Prevent "COPY" from using digits and lowercase letters as delimiters (Tom) __________________________________________________________________ Functions * Add new regular expression functions regexp_matches(), regexp_split_to_array(), and regexp_split_to_table() (Jeremy Drake, Neil) These functions provide extraction of regular expression subexpressions and allow splitting a string using a POSIX regular expression. * Add lo_truncate() for large object truncation (Kris Jurka) * Implement width_bucket() for the float8 data type (Neil) * Add pg_stat_clear_snapshot() to discard statistics snapshots collected during the current transaction (Tom) The first request for statistics in a transaction takes a statistics snapshot that does not change during the transaction. This function allows the snapshot to be discarded and a new snapshot loaded during the next statistics query. This is particularly useful for PL/PgSQL functions, which are confined to a single transaction. * Add isodow option to EXTRACT() and date_part() (Bruce) This returns the day of the week, with Sunday as seven. (dow returns Sunday as zero.) * Add ID (ISO day of week) and IDDD (ISO day of year) format codes for to_char(), to_date(), and to_timestamp() (Brendan Jurd) * Make to_timestamp() and to_date() assume TM (trim) option for potentially variable-width fields (Bruce) This matches Oracle's behavior. * Fix off-by-one conversion error in to_date()/to_timestamp() D (non-ISO day of week) fields (Bruce) * Make setseed() return void, rather than a useless integer value (Neil) * Add a hash function for NUMERIC (Neil) This allows hash indexes and hash-based plans to be used with NUMERIC columns. * Improve efficiency of LIKE/ILIKE, especially for multi-byte character sets like UTF-8 (Andrew, Itagaki Takahiro) * Make currtid() functions require SELECT privileges on the target table (Tom) * Add several txid_*() functions to query active transaction IDs (Jan) This is useful for various replication solutions. __________________________________________________________________ PL/PgSQL Server-Side Language * Add scrollable cursor support, including directional control in "FETCH" (Pavel Stehule) * Allow IN as an alternative to FROM in PL/PgSQL's "FETCH" statement, for consistency with the backend's "FETCH" command (Pavel Stehule) * Add "MOVE" to PL/PgSQL (Magnus, Pavel Stehule, Neil) * Implement "RETURN QUERY" (Pavel Stehule, Neil) This adds convenient syntax for PL/PgSQL set-returning functions that want to return the result of a query. "RETURN QUERY" is easier and more efficient than a loop around "RETURN NEXT". * Allow function parameter names to be qualified with the function's name (Tom) For example, myfunc.myvar. This is particularly useful for specifying variables in a query where the variable name might match a column name. * Make qualification of variables with block labels work properly (Tom) Formerly, outer-level block labels could unexpectedly interfere with recognition of inner-level record or row references. * Tighten requirements for FOR loop STEP values (Tom) Prevent non-positive STEP values, and handle loop overflows. * Improve accuracy when reporting syntax error locations (Tom) __________________________________________________________________ Other Server-Side Languages * Allow type-name arguments to PL/Perl spi_prepare() to be data type aliases in addition to names found in pg_type (Andrew) * Allow type-name arguments to PL/Python plpy.prepare() to be data type aliases in addition to names found in pg_type (Andrew) * Allow type-name arguments to PL/Tcl spi_prepare to be data type aliases in addition to names found in pg_type (Andrew) * Enable PL/PythonU to compile on Python 2.5 (Marko Kreen) * Support a true PL/Python boolean type in compatible Python versions (Python 2.3 and later) (Marko Kreen) * Fix PL/Tcl problems with thread-enabled "libtcl" spawning multiple threads within the backend (Steve Marshall, Paul Bayer, Doug Knight) This caused all sorts of unpleasantness. __________________________________________________________________ psql * List disabled triggers separately in \d output (Brendan Jurd) * In \d patterns, always match $ literally (Tom) * Show aggregate return types in \da output (Greg Sabino Mullane) * Add the function's volatility status to the output of \df+ (Neil) * Add \prompt capability (Chad Wagner) * Allow \pset, \t, and \x to specify on or off, rather than just toggling (Chad Wagner) * Add \sleep capability (Jan) * Enable \timing output for \copy (Andrew) * Improve \timing resolution on Windows (Itagaki Takahiro) * Flush \o output after each backslash command (Tom) * Correctly detect and report errors while reading a -f input file (Peter) * Remove -u option (this option has long been deprecated) (Tom) __________________________________________________________________ pg_dump * Add --tablespaces-only and --roles-only options to pg_dumpall (Dave Page) * Add an output file option to pg_dumpall (Dave Page) This is primarily useful on Windows, where output redirection of child pg_dump processes does not work. * Allow pg_dumpall to accept an initial-connection database name rather than the default template1 (Dave Page) * In -n and -t switches, always match $ literally (Tom) * Improve performance when a database has thousands of objects (Tom) * Remove -u option (this option has long been deprecated) (Tom) __________________________________________________________________ Other Client Applications * In initdb, allow the location of the "pg_xlog" directory to be specified (Euler Taveira de Oliveira) * Enable server core dump generation in pg_regress on supported operating systems (Andrew) * Add a -t (timeout) parameter to pg_ctl (Bruce) This controls how long pg_ctl will wait when waiting for server startup or shutdown. Formerly the timeout was hard-wired as 60 seconds. * Add a pg_ctl option to control generation of server core dumps (Andrew) * Allow Control-C to cancel clusterdb, reindexdb, and vacuumdb (Itagaki Takahiro, Magnus) * Suppress command tag output for createdb, createuser, dropdb, and dropuser (Peter) The --quiet option is ignored and will be removed in 8.4. Progress messages when acting on all databases now go to stdout instead of stderr because they are not actually errors. __________________________________________________________________ libpq * Interpret the dbName parameter of PQsetdbLogin() as a conninfo string if it contains an equals sign (Andrew) This allows use of conninfo strings in client programs that still use PQsetdbLogin(). * Support a global SSL configuration file (Victor Wagner) * Add environment variable PGSSLKEY to control SSL hardware keys (Victor Wagner) * Add lo_truncate() for large object truncation (Kris Jurka) * Add PQconnectionNeedsPassword() that returns true if the server required a password but none was supplied (Joe Conway, Tom) If this returns true after a failed connection attempt, a client application should prompt the user for a password. In the past applications have had to check for a specific error message string to decide whether a password is needed; that approach is now deprecated. * Add PQconnectionUsedPassword() that returns true if the supplied password was actually used (Joe Conway, Tom) This is useful in some security contexts where it is important to know whether a user-supplied password is actually valid. __________________________________________________________________ ecpg * Use V3 frontend/backend protocol (Michael) This adds support for server-side prepared statements. * Use native threads, instead of pthreads, on Windows (Magnus) * Improve thread-safety of ecpglib (Itagaki Takahiro) * Make the ecpg libraries export only necessary API symbols (Michael) __________________________________________________________________ Windows Port * Allow the whole PostgreSQL distribution to be compiled with Microsoft Visual C++ (Magnus and others) This allows Windows-based developers to use familiar development and debugging tools. Windows executables made with Visual C++ might also have better stability and performance than those made with other tool sets. The client-only Visual C++ build scripts have been removed. * Drastically reduce postmaster's memory usage when it has many child processes (Magnus) * Allow regression tests to be started by an administrative user (Magnus) * Add native shared memory implementation (Magnus) __________________________________________________________________ Server Programming Interface (SPI) * Add cursor-related functionality in SPI (Pavel Stehule) Allow access to the cursor-related planning options, and add "FETCH"/"MOVE" routines. * Allow execution of cursor commands through SPI_execute (Tom) The macro SPI_ERROR_CURSOR still exists but will never be returned. * SPI plan pointers are now declared as SPIPlanPtr instead of void * (Tom) This does not break application code, but switching is recommended to help catch simple programming mistakes. __________________________________________________________________ Build Options * Add configure option --enable-profiling to enable code profiling (works only with gcc) (Korry Douglas and Nikhil Sontakke) * Add configure option --with-system-tzdata to use the operating system's time zone database (Peter) * Fix PGXS so extensions can be built against PostgreSQL installations whose pg_config program does not appear first in the PATH (Tom) * Support "gmake draft" when building the SGML documentation (Bruce) Unless draft is used, the documentation build will now be repeated if necessary to ensure the index is up-to-date. __________________________________________________________________ Source Code * Rename macro DLLIMPORT to PGDLLIMPORT to avoid conflicting with third party includes (like Tcl) that define DLLIMPORT (Magnus) * Create "operator families" to improve planning of queries involving cross-data-type comparisons (Tom) * Update GIN extractQuery() API to allow signalling that nothing can satisfy the query (Teodor) * Move NAMEDATALEN definition from "postgres_ext.h" to "pg_config_manual.h" (Peter) * Provide strlcpy() and strlcat() on all platforms, and replace error-prone uses of strncpy(), strncat(), etc (Peter) * Create hooks to let an external plugin monitor (or even replace) the planner and create plans for hypothetical situations (Gurjeet Singh, Tom) * Create a function variable join_search_hook to let plugins override the join search order portion of the planner (Julius Stroffek) * Add tas() support for Renesas' M32R processor (Kazuhiro Inaoka) * quote_identifier() and pg_dump no longer quote keywords that are unreserved according to the grammar (Tom) * Change the on-disk representation of the NUMERIC data type so that the sign_dscale word comes before the weight (Tom) * Use SYSV semaphores rather than POSIX on Darwin >= 6.0, i.e., OS X 10.2 and up (Chris Marcellino) * Add acronym and NFS documentation sections (Bruce) * "Postgres" is now documented as an accepted alias for "PostgreSQL" (Peter) * Add documentation about preventing database server spoofing when the server is down (Bruce) __________________________________________________________________ Contrib * Move "contrib" "README" content into the main PostgreSQL documentation (Albert Cervera i Areny) * Add "contrib/pageinspect" module for low-level page inspection (Simon, Heikki) * Add "contrib/pg_standby" module for controlling warm standby operation (Simon) * Add "contrib/uuid-ossp" module for generating UUID values using the OSSP UUID library (Peter) Use configure --with-ossp-uuid to activate. This takes advantage of the new UUID builtin type. * Add "contrib/dict_int", "contrib/dict_xsyn", and "contrib/test_parser" modules to provide sample add-on text search dictionary templates and parsers (Sergey Karpov) * Allow contrib/pgbench to set the fillfactor (Pavan Deolasee) * Add timestamps to contrib/pgbench -l (Greg Smith) * Add usage count statistics to "contrib/pgbuffercache" (Greg Smith) * Add GIN support for "contrib/hstore" (Teodor) * Add GIN support for "contrib/pg_trgm" (Guillaume Smet, Teodor) * Update OS/X startup scripts in "contrib/start-scripts" (Mark Cotner, David Fetter) * Restrict pgrowlocks() and dblink_get_pkey() to users who have SELECT privilege on the target table (Tom) * Restrict "contrib/pgstattuple" functions to superusers (Tom) * "contrib/xml2" is deprecated and planned for removal in 8.4 (Peter) The new XML support in core PostgreSQL supersedes this module. __________________________________________________________________ Release 8.2.11 Release date: 2008-11-03 This release contains a variety of fixes from 8.2.10. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.11 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see the release notes for 8.2.7. __________________________________________________________________ Changes * Fix GiST index corruption due to marking the wrong index entry "dead" after a deletion (Teodor) This would result in index searches failing to find rows they should have found. * Fix backend crash when the client encoding cannot represent a localized error message (Tom) We have addressed similar issues before, but it would still fail if the "character has no equivalent" message itself couldn't be converted. The fix is to disable localization and send the plain ASCII error message when we detect such a situation. * Fix possible crash when deeply nested functions are invoked from a trigger (Tom) * Improve optimization of "expression" IN ("expression-list") queries (Tom, per an idea from Robert Haas) Cases in which there are query variables on the right-hand side had been handled less efficiently in 8.2.x and 8.3.x than in prior versions. The fix restores 8.1 behavior for such cases. * Fix mis-expansion of rule queries when a sub-SELECT appears in a function call in FROM, a multi-row VALUES list, or a RETURNING list (Tom) The usual symptom of this problem is an "unrecognized node type" error. * Fix memory leak during rescan of a hashed aggregation plan (Neil) * Ensure an error is reported when a newly-defined PL/pgSQL trigger function is invoked as a normal function (Tom) * Prevent possible collision of relfilenode numbers when moving a table to another tablespace with "ALTER SET TABLESPACE" (Heikki) The command tried to re-use the existing filename, instead of picking one that is known unused in the destination directory. * Fix incorrect tsearch2 headline generation when single query item matches first word of text (Sushant Sinha) * Fix improper display of fractional seconds in interval values when using a non-ISO datestyle in an "--enable-integer-datetimes" build (Ron Mayer) * Ensure SPI_getvalue and SPI_getbinval behave correctly when the passed tuple and tuple descriptor have different numbers of columns (Tom) This situation is normal when a table has had columns added or removed, but these two functions didn't handle it properly. The only likely consequence is an incorrect error indication. * Fix ecpg's parsing of "CREATE ROLE" (Michael) * Fix recent breakage of pg_ctl restart (Tom) * Ensure "pg_control" is opened in binary mode (Itagaki Takahiro) pg_controldata and pg_resetxlog did this incorrectly, and so could fail on Windows. * Update time zone data files to tzdata release 2008i (for DST law changes in Argentina, Brazil, Mauritius, Syria) __________________________________________________________________ Release 8.2.10 Release date: 2008-09-22 This release contains a variety of fixes from 8.2.9. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.10 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see the release notes for 8.2.7. __________________________________________________________________ Changes * Fix bug in btree WAL recovery code (Heikki) Recovery failed if the WAL ended partway through a page split operation. * Fix potential miscalculation of datfrozenxid (Alvaro) This error may explain some recent reports of failure to remove old pg_clog data. * Widen local lock counters from 32 to 64 bits (Tom) This responds to reports that the counters could overflow in sufficiently long transactions, leading to unexpected "lock is already held" errors. * Fix possible duplicate output of tuples during a GiST index scan (Teodor) * Fix missed permissions checks when a view contains a simple UNION ALL construct (Heikki) Permissions for the referenced tables were checked properly, but not permissions for the view itself. * Add checks in executor startup to ensure that the tuples produced by an "INSERT" or "UPDATE" will match the target table's current rowtype (Tom) "ALTER COLUMN TYPE", followed by re-use of a previously cached plan, could produce this type of situation. The check protects against data corruption and/or crashes that could ensue. * Fix possible repeated drops during "DROP OWNED" (Tom) This would typically result in strange errors such as "cache lookup failed for relation NNN". * Fix AT TIME ZONE to first try to interpret its timezone argument as a timezone abbreviation, and only try it as a full timezone name if that fails, rather than the other way around as formerly (Tom) The timestamp input functions have always resolved ambiguous zone names in this order. Making AT TIME ZONE do so as well improves consistency, and fixes a compatibility bug introduced in 8.1: in ambiguous cases we now behave the same as 8.0 and before did, since in the older versions AT TIME ZONE accepted *only* abbreviations. * Fix datetime input functions to correctly detect integer overflow when running on a 64-bit platform (Tom) * Prevent integer overflows during units conversion when displaying a configuration parameter that has units (Tom) * Improve performance of writing very long log messages to syslog (Tom) * Allow spaces in the suffix part of an LDAP URL in "pg_hba.conf" (Tom) * Fix bug in backwards scanning of a cursor on a SELECT DISTINCT ON query (Tom) * Fix planner bug with nested sub-select expressions (Tom) If the outer sub-select has no direct dependency on the parent query, but the inner one does, the outer value might not get recalculated for new parent query rows. * Fix planner to estimate that GROUP BY expressions yielding boolean results always result in two groups, regardless of the expressions' contents (Tom) This is very substantially more accurate than the regular GROUP BY estimate for certain boolean tests like "col" IS NULL. * Fix PL/PgSQL to not fail when a FOR loop's target variable is a record containing composite-type fields (Tom) * Fix PL/Tcl to behave correctly with Tcl 8.5, and to be more careful about the encoding of data sent to or from Tcl (Tom) * On Windows, work around a Microsoft bug by preventing libpq from trying to send more than 64kB per system call (Magnus) * Improve pg_dump and pg_restore's error reporting after failure to send a SQL command (Tom) * Fix pg_ctl to properly preserve postmaster command-line arguments across a restart (Bruce) * Update time zone data files to tzdata release 2008f (for DST law changes in Argentina, Bahamas, Brazil, Mauritius, Morocco, Pakistan, Palestine, and Paraguay) __________________________________________________________________ Release 8.2.9 Release date: 2008-06-12 This release contains one serious and one minor bug fix over 8.2.8. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.9 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see the release notes for 8.2.7. __________________________________________________________________ Changes * Make pg_get_ruledef() parenthesize negative constants (Tom) Before this fix, a negative constant in a view or rule might be dumped as, say, -42::integer, which is subtly incorrect: it should be (-42)::integer due to operator precedence rules. Usually this would make little difference, but it could interact with another recent patch to cause PostgreSQL to reject what had been a valid "SELECT DISTINCT" view query. Since this could result in pg_dump output failing to reload, it is being treated as a high-priority fix. The only released versions in which dump output is actually incorrect are 8.3.1 and 8.2.7. * Make "ALTER AGGREGATE ... OWNER TO" update pg_shdepend (Tom) This oversight could lead to problems if the aggregate was later involved in a "DROP OWNED" or "REASSIGN OWNED" operation. __________________________________________________________________ Release 8.2.8 Release date: never released This release contains a variety of fixes from 8.2.7. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.8 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see the release notes for 8.2.7. __________________________________________________________________ Changes * Fix ERRORDATA_STACK_SIZE exceeded crash that occurred on Windows when using UTF-8 database encoding and a different client encoding (Tom) * Fix "ALTER TABLE ADD COLUMN ... PRIMARY KEY" so that the new column is correctly checked to see if it's been initialized to all non-nulls (Brendan Jurd) Previous versions neglected to check this requirement at all. * Fix possible "CREATE TABLE" failure when inheriting the "same" constraint from multiple parent relations that inherited that constraint from a common ancestor (Tom) * Fix pg_get_ruledef() to show the alias, if any, attached to the target table of an "UPDATE" or "DELETE" (Tom) * Fix GIN bug that could result in a too many LWLocks taken failure (Teodor) * Avoid possible crash when decompressing corrupted data (Zdenek Kotala) * Repair two places where SIGTERM exit of a backend could leave corrupted state in shared memory (Tom) Neither case is very important if SIGTERM is used to shut down the whole database cluster together, but there was a problem if someone tried to SIGTERM individual backends. * Fix conversions between ISO-8859-5 and other encodings to handle Cyrillic "Yo" characters (e and E with two dots) (Sergey Burladyan) * Fix several datatype input functions, notably array_in(), that were allowing unused bytes in their results to contain uninitialized, unpredictable values (Tom) This could lead to failures in which two apparently identical literal values were not seen as equal, resulting in the parser complaining about unmatched ORDER BY and DISTINCT expressions. * Fix a corner case in regular-expression substring matching (substring(string from pattern)) (Tom) The problem occurs when there is a match to the pattern overall but the user has specified a parenthesized subexpression and that subexpression hasn't got a match. An example is substring('foo' from 'foo(bar)?'). This should return NULL, since (bar) isn't matched, but it was mistakenly returning the whole-pattern match instead (ie, foo). * Update time zone data files to tzdata release 2008c (for DST law changes in Morocco, Iraq, Choibalsan, Pakistan, Syria, Cuba, and Argentina/San_Luis) * Fix incorrect result from ecpg's PGTYPEStimestamp_sub() function (Michael) * Fix broken GiST comparison function for "contrib/tsearch2"'s tsquery type (Teodor) * Fix possible crashes in "contrib/cube" functions (Tom) * Fix core dump in "contrib/xml2"'s xpath_table() function when the input query returns a NULL value (Tom) * Fix "contrib/xml2"'s makefile to not override CFLAGS (Tom) * Fix DatumGetBool macro to not fail with gcc 4.3 (Tom) This problem affects "old style" (V0) C functions that return boolean. The fix is already in 8.3, but the need to back-patch it was not realized at the time. __________________________________________________________________ Release 8.2.7 Release date: 2008-03-17 This release contains a variety of fixes from 8.2.6. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.7 A dump/restore is not required for those running 8.2.X. However, you might need to "REINDEX" indexes on textual columns after updating, if you are affected by the Windows locale issue described below. __________________________________________________________________ Changes * Fix character string comparison for Windows locales that consider different character combinations as equal (Tom) This fix applies only on Windows and only when using UTF-8 database encoding. The same fix was made for all other cases over two years ago, but Windows with UTF-8 uses a separate code path that was not updated. If you are using a locale that considers some non-identical strings as equal, you may need to "REINDEX" to fix existing indexes on textual columns. * Repair potential deadlock between concurrent "VACUUM FULL" operations on different system catalogs (Tom) * Fix longstanding "LISTEN"/"NOTIFY" race condition (Tom) In rare cases a session that had just executed a "LISTEN" might not get a notification, even though one would be expected because the concurrent transaction executing "NOTIFY" was observed to commit later. A side effect of the fix is that a transaction that has executed a not-yet-committed "LISTEN" command will not see any row in pg_listener for the "LISTEN", should it choose to look; formerly it would have. This behavior was never documented one way or the other, but it is possible that some applications depend on the old behavior. * Disallow "LISTEN" and "UNLISTEN" within a prepared transaction (Tom) This was formerly allowed but trying to do it had various unpleasant consequences, notably that the originating backend could not exit as long as an "UNLISTEN" remained uncommitted. * Disallow dropping a temporary table within a prepared transaction (Heikki) This was correctly disallowed by 8.1, but the check was inadvertently broken in 8.2. * Fix rare crash when an error occurs during a query using a hash index (Heikki) * Fix memory leaks in certain usages of set-returning functions (Neil) * Fix input of datetime values for February 29 in years BC (Tom) The former coding was mistaken about which years were leap years. * Fix "unrecognized node type" error in some variants of "ALTER OWNER" (Tom) * Ensure pg_stat_activity.waiting flag is cleared when a lock wait is aborted (Tom) * Fix handling of process permissions on Windows Vista (Dave, Magnus) In particular, this fix allows starting the server as the Administrator user. * Update time zone data files to tzdata release 2008a (in particular, recent Chile changes); adjust timezone abbreviation VET (Venezuela) to mean UTC-4:30, not UTC-4:00 (Tom) * Fix pg_ctl to correctly extract the postmaster's port number from command-line options (Itagaki Takahiro, Tom) Previously, pg_ctl start -w could try to contact the postmaster on the wrong port, leading to bogus reports of startup failure. * Use "-fwrapv" to defend against possible misoptimization in recent gcc versions (Tom) This is known to be necessary when building PostgreSQL with gcc 4.3 or later. * Correctly enforce statement_timeout values longer than INT_MAX microseconds (about 35 minutes) (Tom) This bug affects only builds with "--enable-integer-datetimes". * Fix "unexpected PARAM_SUBLINK ID" planner error when constant-folding simplifies a sub-select (Tom) * Fix logical errors in constraint-exclusion handling of IS NULL and NOT expressions (Tom) The planner would sometimes exclude partitions that should not have been excluded because of the possibility of NULL results. * Fix another cause of "failed to build any N-way joins" planner errors (Tom) This could happen in cases where a clauseless join needed to be forced before a join clause could be exploited. * Fix incorrect constant propagation in outer-join planning (Tom) The planner could sometimes incorrectly conclude that a variable could be constrained to be equal to a constant, leading to wrong query results. * Fix display of constant expressions in ORDER BY and GROUP BY (Tom) An explictly casted constant would be shown incorrectly. This could for example lead to corruption of a view definition during dump and reload. * Fix libpq to handle NOTICE messages correctly during COPY OUT (Tom) This failure has only been observed to occur when a user-defined datatype's output routine issues a NOTICE, but there is no guarantee it couldn't happen due to other causes. __________________________________________________________________ Release 8.2.6 Release date: 2008-01-07 This release contains a variety of fixes from 8.2.5, including fixes for significant security issues. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.6 A dump/restore is not required for those running 8.2.X. __________________________________________________________________ Changes * Prevent functions in indexes from executing with the privileges of the user running "VACUUM", "ANALYZE", etc (Tom) Functions used in index expressions and partial-index predicates are evaluated whenever a new table entry is made. It has long been understood that this poses a risk of trojan-horse code execution if one modifies a table owned by an untrustworthy user. (Note that triggers, defaults, check constraints, etc. pose the same type of risk.) But functions in indexes pose extra danger because they will be executed by routine maintenance operations such as "VACUUM FULL", which are commonly performed automatically under a superuser account. For example, a nefarious user can execute code with superuser privileges by setting up a trojan-horse index definition and waiting for the next routine vacuum. The fix arranges for standard maintenance operations (including "VACUUM", "ANALYZE", "REINDEX", and "CLUSTER") to execute as the table owner rather than the calling user, using the same privilege-switching mechanism already used for SECURITY DEFINER functions. To prevent bypassing this security measure, execution of "SET SESSION AUTHORIZATION" and "SET ROLE" is now forbidden within a SECURITY DEFINER context. (CVE-2007-6600) * Repair assorted bugs in the regular-expression package (Tom, Will Drewry) Suitably crafted regular-expression patterns could cause crashes, infinite or near-infinite looping, and/or massive memory consumption, all of which pose denial-of-service hazards for applications that accept regex search patterns from untrustworthy sources. (CVE-2007-4769, CVE-2007-4772, CVE-2007-6067) * Require non-superusers who use "/contrib/dblink" to use only password authentication, as a security measure (Joe) The fix that appeared for this in 8.2.5 was incomplete, as it plugged the hole for only some "dblink" functions. (CVE-2007-6601, CVE-2007-3278) * Fix bugs in WAL replay for GIN indexes (Teodor) * Fix GIN index build to work properly when maintenance_work_mem is 4GB or more (Tom) * Update time zone data files to tzdata release 2007k (in particular, recent Argentina changes) (Tom) * Improve planner's handling of LIKE/regex estimation in non-C locales (Tom) * Fix planning-speed problem for deep outer-join nests, as well as possible poor choice of join order (Tom) * Fix planner failure in some cases of WHERE false AND var IN (SELECT ...) (Tom) * Make "CREATE TABLE ... SERIAL" and "ALTER SEQUENCE ... OWNED BY" not change the currval() state of the sequence (Tom) * Preserve the tablespace and storage parameters of indexes that are rebuilt by "ALTER TABLE ... ALTER COLUMN TYPE" (Tom) * Make archive recovery always start a new WAL timeline, rather than only when a recovery stop time was used (Simon) This avoids a corner-case risk of trying to overwrite an existing archived copy of the last WAL segment, and seems simpler and cleaner than the original definition. * Make "VACUUM" not use all of maintenance_work_mem when the table is too small for it to be useful (Alvaro) * Fix potential crash in translate() when using a multibyte database encoding (Tom) * Make corr() return the correct result for negative correlation values (Neil) * Fix overflow in extract(epoch from interval) for intervals exceeding 68 years (Tom) * Fix PL/Perl to not fail when a UTF-8 regular expression is used in a trusted function (Andrew) * Fix PL/Perl to cope when platform's Perl defines type bool as int rather than char (Tom) While this could theoretically happen anywhere, no standard build of Perl did things this way ... until Mac OS X 10.5. * Fix PL/Python to work correctly with Python 2.5 on 64-bit machines (Marko Kreen) * Fix PL/Python to not crash on long exception messages (Alvaro) * Fix pg_dump to correctly handle inheritance child tables that have default expressions different from their parent's (Tom) * Fix libpq crash when PGPASSFILE refers to a file that is not a plain file (Martin Pitt) * ecpg parser fixes (Michael) * Make "contrib/pgcrypto" defend against OpenSSL libraries that fail on keys longer than 128 bits; which is the case at least on some Solaris versions (Marko Kreen) * Make "contrib/tablefunc"'s crosstab() handle NULL rowid as a category in its own right, rather than crashing (Joe) * Fix tsvector and tsquery output routines to escape backslashes correctly (Teodor, Bruce) * Fix crash of to_tsvector() on huge input strings (Teodor) * Require a specific version of Autoconf to be used when re-generating the "configure" script (Peter) This affects developers and packagers only. The change was made to prevent accidental use of untested combinations of Autoconf and PostgreSQL versions. You can remove the version check if you really want to use a different Autoconf version, but it's your responsibility whether the result works or not. * Update gettimeofday configuration check so that PostgreSQL can be built on newer versions of MinGW (Magnus) __________________________________________________________________ Release 8.2.5 Release date: 2007-09-17 This release contains a variety of fixes from 8.2.4. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.5 A dump/restore is not required for those running 8.2.X. __________________________________________________________________ Changes * Prevent index corruption when a transaction inserts rows and then aborts close to the end of a concurrent "VACUUM" on the same table (Tom) * Fix ALTER DOMAIN ADD CONSTRAINT for cases involving domains over domains (Tom) * Make "CREATE DOMAIN ... DEFAULT NULL" work properly (Tom) * Fix some planner problems with outer joins, notably poor size estimation for t1 LEFT JOIN t2 WHERE t2.col IS NULL (Tom) * Allow the interval data type to accept input consisting only of milliseconds or microseconds (Neil) * Allow timezone name to appear before the year in timestamp input (Tom) * Fixes for GIN indexes used by "/contrib/tsearch2" (Teodor) * Speed up rtree index insertion (Teodor) * Fix excessive logging of SSL error messages (Tom) * Fix logging so that log messages are never interleaved when using the syslogger process (Andrew) * Fix crash when log_min_error_statement logging runs out of memory (Tom) * Fix incorrect handling of some foreign-key corner cases (Tom) * Fix stddev_pop(numeric) and var_pop(numeric) (Tom) * Prevent "REINDEX" and "CLUSTER" from failing due to attempting to process temporary tables of other sessions (Alvaro) * Update the time zone database rules, particularly New Zealand's upcoming changes (Tom) * Windows socket and semaphore improvements (Magnus) * Make "pg_ctl -w" work properly in Windows service mode (Dave Page) * Fix memory allocation bug when using MIT Kerberos on Windows (Magnus) * Suppress timezone name (%Z) in log timestamps on Windows because of possible encoding mismatches (Tom) * Require non-superusers who use "/contrib/dblink" to use only password authentication, as a security measure (Joe) * Restrict "/contrib/pgstattuple" functions to superusers, for security reasons (Tom) * Do not let "/contrib/intarray" try to make its GIN opclass the default (this caused problems at dump/restore) (Tom) __________________________________________________________________ Release 8.2.4 Release date: 2007-04-23 This release contains a variety of fixes from 8.2.3, including a security fix. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.4 A dump/restore is not required for those running 8.2.X. __________________________________________________________________ Changes * Support explicit placement of the temporary-table schema within search_path, and disable searching it for functions and operators (Tom) This is needed to allow a security-definer function to set a truly secure value of search_path. Without it, an unprivileged SQL user can use temporary objects to execute code with the privileges of the security-definer function (CVE-2007-2138). See "CREATE FUNCTION" for more information. * Fix shared_preload_libraries for Windows by forcing reload in each backend (Korry Douglas) * Fix to_char() so it properly upper/lower cases localized day or month names (Pavel Stehule) * "/contrib/tsearch2" crash fixes (Teodor) * Require "COMMIT PREPARED" to be executed in the same database as the transaction was prepared in (Heikki) * Allow "pg_dump" to do binary backups larger than two gigabytes on Windows (Magnus) * New traditional (Taiwan) Chinese FAQ (Zhou Daojing) * Prevent the statistics collector from writing to disk too frequently (Tom) * Fix potential-data-corruption bug in how "VACUUM FULL" handles "UPDATE" chains (Tom, Pavan Deolasee) * Fix bug in domains that use array types (Tom) * Fix "pg_dump" so it can dump a serial column's sequence using "-t" when not also dumping the owning table (Tom) * Planner fixes, including improving outer join and bitmap scan selection logic (Tom) * Fix possible wrong answers or crash when a PL/pgSQL function tries to RETURN from within an EXCEPTION block (Tom) * Fix PANIC during enlargement of a hash index (Tom) * Fix POSIX-style timezone specs to follow new USA DST rules (Tom) __________________________________________________________________ Release 8.2.3 Release date: 2007-02-07 This release contains two fixes from 8.2.2. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.3 A dump/restore is not required for those running 8.2.X. __________________________________________________________________ Changes * Remove overly-restrictive check for type length in constraints and functional indexes(Tom) * Fix optimization so MIN/MAX in subqueries can again use indexes (Tom) __________________________________________________________________ Release 8.2.2 Release date: 2007-02-05 This release contains a variety of fixes from 8.2.1, including a security fix. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.2 A dump/restore is not required for those running 8.2.X. __________________________________________________________________ Changes * Remove security vulnerabilities that allowed connected users to read backend memory (Tom) The vulnerabilities involve suppressing the normal check that a SQL function returns the data type it's declared to, and changing the data type of a table column (CVE-2007-0555, CVE-2007-0556). These errors can easily be exploited to cause a backend crash, and in principle might be used to read database content that the user should not be able to access. * Fix not-so-rare-anymore bug wherein btree index page splits could fail due to choosing an infeasible split point (Heikki Linnakangas) * Fix Borland C compile scripts (L Bayuk) * Properly handle to_char('CC') for years ending in 00 (Tom) Year 2000 is in the twentieth century, not the twenty-first. * "/contrib/tsearch2" localization improvements (Tatsuo, Teodor) * Fix incorrect permission check in information_schema.key_column_usage view (Tom) The symptom is "relation with OID nnnnn does not exist" errors. To get this fix without using "initdb", use "CREATE OR REPLACE VIEW" to install the corrected definition found in "share/information_schema.sql". Note you will need to do this in each database. * Improve "VACUUM" performance for databases with many tables (Tom) * Fix for rare Assert() crash triggered by UNION (Tom) * Fix potentially incorrect results from index searches using ROW inequality conditions (Tom) * Tighten security of multi-byte character processing for UTF8 sequences over three bytes long (Tom) * Fix bogus "permission denied" failures occurring on Windows due to attempts to fsync already-deleted files (Magnus, Tom) * Fix bug that could cause the statistics collector to hang on Windows (Magnus) This would in turn lead to autovacuum not working. * Fix possible crashes when an already-in-use PL/pgSQL function is updated (Tom) * Improve PL/pgSQL handling of domain types (Sergiy Vyshnevetskiy, Tom) * Fix possible errors in processing PL/pgSQL exception blocks (Tom) __________________________________________________________________ Release 8.2.1 Release date: 2007-01-08 This release contains a variety of fixes from 8.2. For information about new features in the 8.2 major release, see the Section called Release 8.2. __________________________________________________________________ Migration to Version 8.2.1 A dump/restore is not required for those running 8.2. __________________________________________________________________ Changes * Fix crash with SELECT ... LIMIT ALL (also LIMIT NULL) (Tom) * "Several /contrib/tsearch2" fixes (Teodor) * On Windows, make log messages coming from the operating system use ASCII encoding (Hiroshi Saito) This fixes a conversion problem when there is a mismatch between the encoding of the operating system and database server. * Fix Windows linking of pg_dump using "win32.mak" (Hiroshi Saito) * Fix planner mistakes for outer join queries (Tom) * Fix several problems in queries involving sub-SELECTs (Tom) * Fix potential crash in SPI during subtransaction abort (Tom) This affects all PL functions since they all use SPI. * Improve build speed of PDF documentation (Peter) * Re-add JST (Japan) timezone abbreviation (Tom) * Improve optimization decisions related to index scans (Tom) * Have psql print multi-byte combining characters as before, rather than output as \u (Tom) * Improve index usage of regular expressions that use parentheses (Tom) This improves psql \d performance also. * Make pg_dumpall assume that databases have public CONNECT privilege, when dumping from a pre-8.2 server (Tom) This preserves the previous behavior that anyone can connect to a database if allowed by "pg_hba.conf". __________________________________________________________________ Release 8.2 Release date: 2006-12-05 __________________________________________________________________ Overview This release adds many functionality and performance improvements that were requested by users, including: * Query language enhancements including "INSERT/UPDATE/DELETE RETURNING", multirow VALUES lists, and optional target-table alias in "UPDATE"/"DELETE" * Index creation without blocking concurrent "INSERT"/"UPDATE"/"DELETE" operations * Many query optimization improvements, including support for reordering outer joins * Improved sorting performance with lower memory usage * More efficient locking with better concurrency * More efficient vacuuming * Easier administration of warm standby servers * New FILLFACTOR support for tables and indexes * Monitoring, logging, and performance tuning additions * More control over creating and dropping objects * Table inheritance relationships can be defined for and removed from pre-existing tables * "COPY TO" can copy the output of an arbitrary "SELECT" statement * Array improvements, including nulls in arrays * Aggregate-function improvements, including multiple-input aggregates and SQL:2003 statistical functions * Many "contrib/" improvements __________________________________________________________________ Migration to Version 8.2 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities: * Set escape_string_warning to on by default (Bruce) This issues a warning if backslash escapes are used in non-escape (non-E'') strings. * Change the row constructor syntax (ROW(...)) so that list elements foo.* will be expanded to a list of their member fields, rather than creating a nested row type field as formerly (Tom) The new behavior is substantially more useful since it allows, for example, triggers to check for data changes with IF row(new.*) IS DISTINCT FROM row(old.*). The old behavior is still available by omitting .*. * Make row comparisons follow SQL standard semantics and allow them to be used in index scans (Tom) Previously, row = and <> comparisons followed the standard but < <= > >= did not. A row comparison can now be used as an index constraint for a multicolumn index matching the row value. * Make row IS [NOT] NULL tests follow SQL standard semantics (Tom) The former behavior conformed to the standard for simple cases with IS NULL, but IS NOT NULL would return true if any row field was non-null, whereas the standard says it should return true only when all fields are non-null. * Make "SET CONSTRAINT" affect only one constraint (Kris Jurka) In previous releases, "SET CONSTRAINT" modified all constraints with a matching name. In this release, the schema search path is used to modify only the first matching constraint. A schema specification is also supported. This more nearly conforms to the SQL standard. * Remove RULE permission for tables, for security reasons (Tom) As of this release, only a table's owner can create or modify rules for the table. For backwards compatibility, "GRANT"/"REVOKE RULE" is still accepted, but it does nothing. * Array comparison improvements (Tom) Now array dimensions are also compared. * Change array concatenation to match documented behavior (Tom) This changes the previous behavior where concatenation would modify the array lower bound. * Make command-line options of postmaster and postgres identical (Peter) This allows the postmaster to pass arguments to each backend without using -o. Note that some options are now only available as long-form options, because there were conflicting single-letter options. * Deprecate use of postmaster symbolic link (Peter) postmaster and postgres commands now act identically, with the behavior determined by command-line options. The postmaster symbolic link is kept for compatibility, but is not really needed. * Change log_duration to output even if the query is not output (Tom) In prior releases, log_duration only printed if the query appeared earlier in the log. * Make to_char(time) and to_char(interval) treat HH and HH12 as 12-hour intervals Most applications should use HH24 unless they want a 12-hour display. * Zero unmasked bits in conversion from INET to CIDR (Tom) This ensures that the converted value is actually valid for CIDR. * Remove australian_timezones configuration variable (Joachim Wieland) This variable has been superseded by a more general facility for configuring timezone abbreviations. * Improve cost estimation for nested-loop index scans (Tom) This might eliminate the need to set unrealistically small values of random_page_cost. If you have been using a very small random_page_cost, please recheck your test cases. * Change behavior of "pg_dump" -n and -t options. (Greg Sabino Mullane) See the "pg_dump" manual page for details. * Change libpq PQdsplen() to return a useful value (Martijn van Oosterhout) * Declare libpq PQgetssl() as returning void *, rather than SSL * (Martijn van Oosterhout) This allows applications to use the function without including the OpenSSL headers. * C-language loadable modules must now include a PG_MODULE_MAGIC macro call for version compatibility checking (Martijn van Oosterhout) * For security's sake, modules used by a PL/PerlU function are no longer available to PL/Perl functions (Andrew) Note: This also implies that data can no longer be shared between a PL/Perl function and a PL/PerlU function. Some Perl installations have not been compiled with the correct flags to allow multiple interpreters to exist within a single process. In this situation PL/Perl and PL/PerlU cannot both be used in a single backend. The solution is to get a Perl installation which supports multiple interpreters. * In "contrib/xml2/", rename xml_valid() to xml_is_well_formed() (Tom) xml_valid() will remain for backward compatibility, but its behavior will change to do schema checking in a future release. * Remove "contrib/ora2pg/", now at http://www.samse.fr/GPL/ora2pg * Remove contrib modules that have been migrated to PgFoundry: "adddepend", "dbase", "dbmirror", "fulltextindex", "mac", "userlock" * Remove abandoned contrib modules: "mSQL-interface", "tips" * Remove QNX and BEOS ports (Bruce) These ports no longer had active maintainers. __________________________________________________________________ Changes Below you will find a detailed account of the changes between PostgreSQL 8.2 and the previous major release. __________________________________________________________________ Performance Improvements * Allow the planner to reorder outer joins in some circumstances (Tom) In previous releases, outer joins would always be evaluated in the order written in the query. This change allows the query optimizer to consider reordering outer joins, in cases where it can determine that the join order can be changed without altering the meaning of the query. This can make a considerable performance difference for queries involving multiple outer joins or mixed inner and outer joins. * Improve efficiency of IN (list-of-expressions) clauses (Tom) * Improve sorting speed and reduce memory usage (Simon, Tom) * Improve subtransaction performance (Alvaro, Itagaki Takahiro, Tom) * Add FILLFACTOR to table and index creation (ITAGAKI Takahiro) This leaves extra free space in each table or index page, allowing improved performance as the database grows. This is particularly valuable to maintain clustering. * Increase default values for shared_buffers and max_fsm_pages (Andrew) * Improve locking performance by breaking the lock manager tables into sections (Tom) This allows locking to be more fine-grained, reducing contention. * Reduce locking requirements of sequential scans (Qingqing Zhou) * Reduce locking required for database creation and destruction (Tom) * Improve the optimizer's selectivity estimates for LIKE, ILIKE, and regular expression operations (Tom) * Improve planning of joins to inherited tables and UNION ALL views (Tom) * Allow constraint exclusion to be applied to inherited "UPDATE" and "DELETE" queries (Tom) "SELECT" already honored constraint exclusion. * Improve planning of constant WHERE clauses, such as a condition that depends only on variables inherited from an outer query level (Tom) * Protocol-level unnamed prepared statements are re-planned for each set of BIND values (Tom) This improves performance because the exact parameter values can be used in the plan. * Speed up vacuuming of B-Tree indexes (Heikki Linnakangas, Tom) * Avoid extra scan of tables without indexes during "VACUUM" (Greg Stark) * Improve multicolumn GiST indexing (Oleg, Teodor) * Remove dead index entries before B-Tree page split (Junji Teramoto) __________________________________________________________________ Server Changes * Allow a forced switch to a new transaction log file (Simon, Tom) This is valuable for keeping warm standby slave servers in sync with the master. Transaction log file switching now also happens automatically during pg_stop_backup(). This ensures that all transaction log files needed for recovery can be archived immediately. * Add WAL informational functions (Simon) Add functions for interrogating the current transaction log insertion point and determining WAL filenames from the hex WAL locations displayed by pg_stop_backup() and related functions. * Improve recovery from a crash during WAL replay (Simon) The server now does periodic checkpoints during WAL recovery, so if there is a crash, future WAL recovery is shortened. This also eliminates the need for warm standby servers to replay the entire log since the base backup if they crash. * Improve reliability of long-term WAL replay (Heikki, Simon, Tom) Formerly, trying to roll forward through more than 2 billion transactions would not work due to XID wraparound. This meant warm standby servers had to be reloaded from fresh base backups periodically. * Add archive_timeout to force transaction log file switches at a given interval (Simon) This enforces a maximum replication delay for warm standby servers. * Add native LDAP authentication (Magnus Hagander) This is particularly useful for platforms that do not support PAM, such as Windows. * Add GRANT CONNECT ON DATABASE (Gevik Babakhani) This gives SQL-level control over database access. It works as an additional filter on top of the existing "pg_hba.conf" controls. * Add support for SSL Certificate Revocation List (CRL) files (Libor Hohos) The server and libpq both recognize CRL files now. * GiST indexes are now clusterable (Teodor) * Remove routine autovacuum server log entries (Bruce) pg_stat_activity now shows autovacuum activity. * Track maximum XID age within individual tables, instead of whole databases (Alvaro) This reduces the overhead involved in preventing transaction ID wraparound, by avoiding unnecessary VACUUMs. * Add last vacuum and analyze timestamp columns to the stats collector (Larry Rosenman) These values now appear in the pg_stat_*_tables system views. * Improve performance of statistics monitoring, especially stats_command_string (Tom, Bruce) This release enables stats_command_string by default, now that its overhead is minimal. This means pg_stat_activity will now show all active queries by default. * Add a waiting column to pg_stat_activity (Tom) This allows pg_stat_activity to show all the information included in the ps display. * Add configuration parameter update_process_title to control whether the ps display is updated for every command (Bruce) On platforms where it is expensive to update the ps display, it might be worthwhile to turn this off and rely solely on pg_stat_activity for status information. * Allow units to be specified in configuration settings (Peter) For example, you can now set shared_buffers to 32MB rather than mentally converting sizes. * Add support for include directives in "postgresql.conf" (Joachim Wieland) * Improve logging of protocol-level prepare/bind/execute messages (Bruce, Tom) Such logging now shows statement names, bind parameter values, and the text of the query being executed. Also, the query text is properly included in logged error messages when enabled by log_min_error_statement. * Prevent max_stack_depth from being set to unsafe values On platforms where we can determine the actual kernel stack depth limit (which is most), make sure that the initial default value of max_stack_depth is safe, and reject attempts to set it to unsafely large values. * Enable highlighting of error location in query in more cases (Tom) The server is now able to report a specific error location for some semantic errors (such as unrecognized column name), rather than just for basic syntax errors as before. * Fix "failed to re-find parent key" errors in "VACUUM" (Tom) * Clean out "pg_internal.init" cache files during server restart (Simon) This avoids a hazard that the cache files might contain stale data after PITR recovery. * Fix race condition for truncation of a large relation across a gigabyte boundary by "VACUUM" (Tom) * Fix bug causing needless deadlock errors on row-level locks (Tom) * Fix bugs affecting multi-gigabyte hash indexes (Tom) * Each backend process is now its own process group leader (Tom) This allows query cancel to abort subprocesses invoked from a backend or archive/recovery process. __________________________________________________________________ Query Changes * Add "INSERT"/"UPDATE"/"DELETE" RETURNING (Jonah Harris, Tom) This allows these commands to return values, such as the computed serial key for a new row. In the "UPDATE" case, values from the updated version of the row are returned. * Add support for multiple-row VALUES clauses, per SQL standard (Joe, Tom) This allows "INSERT" to insert multiple rows of constants, or queries to generate result sets using constants. For example, INSERT ... VALUES (...), (...), ...., and SELECT * FROM (VALUES (...), (...), ....) AS alias(f1, ...). * Allow "UPDATE" and "DELETE" to use an alias for the target table (Atsushi Ogawa) The SQL standard does not permit an alias in these commands, but many database systems allow one anyway for notational convenience. * Allow "UPDATE" to set multiple columns with a list of values (Susanne Ebrecht) This is basically a short-hand for assigning the columns and values in pairs. The syntax is UPDATE tab SET (column, ...) = (val, ...). * Make row comparisons work per standard (Tom) The forms <, <=, >, >= now compare rows lexicographically, that is, compare the first elements, if equal compare the second elements, and so on. Formerly they expanded to an AND condition across all the elements, which was neither standard nor very useful. * Add CASCADE option to "TRUNCATE" (Joachim Wieland) This causes "TRUNCATE" to automatically include all tables that reference the specified table(s) via foreign keys. While convenient, this is a dangerous tool -- use with caution! * Support FOR UPDATE and FOR SHARE in the same SELECT command (Tom) * Add IS NOT DISTINCT FROM (Pavel Stehule) This operator is similar to equality (=), but evaluates to true when both left and right operands are NULL, and to false when just one is, rather than yielding NULL in these cases. * Improve the length output used by UNION/INTERSECT/EXCEPT (Tom) When all corresponding columns are of the same defined length, that length is used for the result, rather than a generic length. * Allow ILIKE to work for multi-byte encodings (Tom) Internally, ILIKE now calls lower() and then uses LIKE. Locale-specific regular expression patterns still do not work in these encodings. * Enable standard_conforming_strings to be turned on (Kevin Grittner) This allows backslash escaping in strings to be disabled, making PostgreSQL more standards-compliant. The default is off for backwards compatibility, but future releases will default this to on. * Do not flatten subqueries that contain volatile functions in their target lists (Jaime Casanova) This prevents surprising behavior due to multiple evaluation of a volatile function (such as random() or nextval()). It might cause performance degradation in the presence of functions that are unnecessarily marked as volatile. * Add system views pg_prepared_statements and pg_cursors to show prepared statements and open cursors (Joachim Wieland, Neil) These are very useful in pooled connection setups. * Support portal parameters in "EXPLAIN" and "EXECUTE" (Tom) This allows, for example, JDBC ? parameters to work in these commands. * If SQL-level "PREPARE" parameters are unspecified, infer their types from the content of the query (Neil) Protocol-level "PREPARE" already did this. * Allow LIMIT and OFFSET to exceed two billion (Dhanaraj M) __________________________________________________________________ Object Manipulation Changes * Add TABLESPACE clause to "CREATE TABLE AS" (Neil) This allows a tablespace to be specified for the new table. * Add ON COMMIT clause to "CREATE TABLE AS" (Neil) This allows temporary tables to be truncated or dropped on transaction commit. The default behavior is for the table to remain until the session ends. * Add INCLUDING CONSTRAINTS to "CREATE TABLE LIKE" (Greg Stark) This allows easy copying of CHECK constraints to a new table. * Allow the creation of placeholder (shell) types (Martijn van Oosterhout) A shell type declaration creates a type name, without specifying any of the details of the type. Making a shell type is useful because it allows cleaner declaration of the type's input/output functions, which must exist before the type can be defined "for real". The syntax is "CREATE TYPE typename". * Aggregate functions now support multiple input parameters (Sergey Koposov, Tom) * Add new aggregate creation syntax (Tom) The new syntax is "CREATE AGGREGATE aggname (input_type) (parameter_list)". This more naturally supports the new multi-parameter aggregate functionality. The previous syntax is still supported. * Add "ALTER ROLE PASSWORD NULL" to remove a previously set role password (Peter) * Add "DROP" object IF EXISTS for many object types (Andrew) This allows "DROP" operations on non-existent objects without generating an error. * Add DROP OWNED to drop all objects owned by a role (Alvaro) * Add REASSIGN OWNED to reassign ownership of all objects owned by a role (Alvaro) This, and DROP OWNED above, facilitate dropping roles. * Add "GRANT ON SEQUENCE" syntax (Bruce) This was added for setting sequence-specific permissions. GRANT ON TABLE for sequences is still supported for backward compatibility. * Add USAGE permission for sequences that allows only currval() and nextval(), not setval() (Bruce) USAGE permission allows more fine-grained control over sequence access. Granting USAGE allows users to increment a sequence, but prevents them from setting the sequence to an arbitrary value using setval(). * Add ALTER TABLE [ NO ] INHERIT (Greg Stark) This allows inheritance to be adjusted dynamically, rather than just at table creation and destruction. This is very valuable when using inheritance to implement table partitioning. * Allow comments on global objects to be stored globally (Kris Jurka) Previously, comments attached to databases were stored in individual databases, making them ineffective, and there was no provision at all for comments on roles or tablespaces. This change adds a new shared catalog pg_shdescription and stores comments on databases, roles, and tablespaces therein. __________________________________________________________________ Utility Command Changes * Add option to allow indexes to be created without blocking concurrent writes to the table (Greg Stark, Tom) The new syntax is "CREATE INDEX CONCURRENTLY". The default behavior is still to block table modification while a index is being created. * Provide advisory locking functionality (Abhijit Menon-Sen, Tom) This is a new locking API designed to replace what used to be in /contrib/userlock. The userlock code is now on pgfoundry. * Allow "COPY" to dump a "SELECT" query (Zoltan Boszormenyi, Karel Zak) This allows "COPY" to dump arbitrary SQL queries. The syntax is COPY (SELECT ...) TO. * Make the "COPY" command return a command tag that includes the number of rows copied (Volkan YAZICI) * Allow "VACUUM" to expire rows without being affected by other concurrent "VACUUM" operations (Hannu Krossing, Alvaro, Tom) * Make initdb detect the operating system locale and set the default DateStyle accordingly (Peter) This makes it more likely that the installed "postgresql.conf" DateStyle value will be as desired. * Reduce number of progress messages displayed by initdb (Tom) __________________________________________________________________ Date/Time Changes * Allow full timezone names in timestamp input values (Joachim Wieland) For example, '2006-05-24 21:11 America/New_York'::timestamptz. * Support configurable timezone abbreviations (Joachim Wieland) A desired set of timezone abbreviations can be chosen via the configuration parameter timezone_abbreviations. * Add pg_timezone_abbrevs and pg_timezone_names views to show supported timezones (Magnus Hagander) * Add clock_timestamp(), statement_timestamp(), and transaction_tim