postgres/contrib
Masahiko Sawada d87d07b7ad Fix re-distributing previously distributed invalidation messages during logical decoding.
Commit 4909b38af0 introduced logic to distribute invalidation messages
from catalog-modifying transactions to all concurrent in-progress
transactions. However, since each transaction distributes not only its
original invalidation messages but also previously distributed
messages to other transactions, this leads to an exponential increase
in allocation request size for invalidation messages, ultimately
causing memory allocation failure.

This commit fixes this issue by tracking distributed invalidation
messages separately per decoded transaction and not redistributing
these messages to other in-progress transactions. The maximum size of
distributed invalidation messages that one transaction can store is
limited to MAX_DISTR_INVAL_MSG_PER_TXN (8MB). Once the size of the
distributed invalidation messages exceeds this threshold, we
invalidate all caches in locations where distributed invalidation
messages need to be executed.

Back-patch to all supported versions where we introduced the fix by
commit 4909b38af0.

Note that this commit adds two new fields to ReorderBufferTXN to store
the distributed transactions. This change breaks ABI compatibility in
back branches, affecting third-party extensions that depend on the
size of the ReorderBufferTXN struct, though this scenario seems
unlikely.

Additionally, it adds a new flag to the txn_flags field of
ReorderBufferTXN to indicate distributed invalidation message
overflow. This should not affect existing implementations, as it is
unlikely that third-party extensions use unused bits in the txn_flags
field.

Bug: #18938 #18942
Author: vignesh C <vignesh21@gmail.com>
Reported-by: Duncan Sands <duncan.sands@deepbluecap.com>
Reported-by: John Hutchins <john.hutchins@wicourts.gov>
Reported-by: Laurence Parry <greenreaper@hotmail.com>
Reported-by: Max Madden <maxmmadden@gmail.com>
Reported-by: Braulio Fdo Gonzalez <brauliofg@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/680bdaf6-f7d1-4536-b580-05c2760c67c6@deepbluecap.com
Discussion: https://postgr.es/m/18942-0ab1e5ae156613ad@postgresql.org
Discussion: https://postgr.es/m/18938-57c9a1c463b68ce0@postgresql.org
Discussion: https://postgr.es/m/CAD1FGCT2sYrP_70RTuo56QTizyc+J3wJdtn2gtO3VttQFpdMZg@mail.gmail.com
Discussion: https://postgr.es/m/CANO2=B=2BT1hSYCE=nuuTnVTnjidMg0+-FfnRnqM6kd23qoygg@mail.gmail.com
Backpatch-through: 13
2025-06-16 17:36:01 -07:00
..
amcheck Fix a few duplicate words in comments 2025-04-21 10:41:18 +12:00
auth_delay Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
auto_explain Revert "Don't lock partitions pruned by initial pruning" 2025-05-22 17:02:35 +09:00
basebackup_to_shell Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
basic_archive Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
bloom Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
bool_plperl Fix erroneous construction of functions' dependencies on transforms. 2025-04-07 13:31:37 -04:00
btree_gin Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
btree_gist Rename gist stratnum support function 2025-06-02 08:41:27 +02:00
citext Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
cube Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
dblink Avoid resource leaks when a dblink connection fails. 2025-05-29 10:39:55 -04:00
dict_int Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
dict_xsyn Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
earthdistance Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
file_fdw Disallow "=" in names of reloptions and foreign-data options. 2025-06-02 15:22:44 -04:00
fuzzystrmatch Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
hstore Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
hstore_plperl Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
hstore_plpython Remove circular #include's between plpython.h and plpy_util.h. 2025-04-27 11:43:02 -04:00
intagg Update copyright for 2025 2025-01-01 11:21:55 -05:00
intarray Fix GIN's shimTriConsistentFn to not corrupt its input. 2025-04-12 12:28:02 -04:00
isn Revert a few small patches that were intended for version 19. 2025-06-11 15:10:12 -07:00
jsonb_plperl Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
jsonb_plpython Remove circular #include's between plpython.h and plpy_util.h. 2025-04-27 11:43:02 -04:00
lo Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
ltree Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
ltree_plpython Remove circular #include's between plpython.h and plpy_util.h. 2025-04-27 11:43:02 -04:00
oid2name Update copyright for 2025 2025-01-01 11:21:55 -05:00
pageinspect Be more wary of corrupt data in pageinspect's heap_page_items(). 2025-04-19 16:37:42 -04:00
passwordcheck Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
pg_buffercache Fix a few duplicate words in comments 2025-04-21 10:41:18 +12:00
pg_freespacemap Prevent assertion failure in contrib/pg_freespacemap. 2025-03-27 13:20:23 -04:00
pg_logicalinspect Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
pg_overexplain Revert support for improved tracking of nested queries 2025-06-12 10:08:55 +09:00
pg_prewarm pg_prewarm: Allow autoprewarm to use more than 1GB to dump blocks. 2025-06-06 08:18:27 -04:00
pg_stat_statements Fix squashing algorithm for query texts 2025-06-12 14:21:21 +02:00
pg_surgery Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
pg_trgm Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
pg_visibility read_stream: Introduce and use optional batchmode support 2025-03-30 18:36:41 -04:00
pg_walinspect Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
pgcrypto Fix incorrect format placeholder 2025-04-08 19:12:03 +02:00
pgrowlocks Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
pgstattuple Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
postgres_fdw Revert "postgres_fdw: Inherit the local transaction's access/deferrable modes." 2025-06-08 17:30:00 +09:00
seg Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
sepgsql Fix incorrect year in some copyright notices 2025-05-19 09:46:52 +09:00
spi Revert a few small patches that were intended for version 19. 2025-06-11 15:10:12 -07:00
sslinfo Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
start-scripts Remove gratuitous references to postmaster program 2023-01-26 10:48:32 +01:00
tablefunc Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
tcn Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
test_decoding Fix re-distributing previously distributed invalidation messages during logical decoding. 2025-06-16 17:36:01 -07:00
tsm_system_rows Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
tsm_system_time Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
unaccent Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
uuid-ossp Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
vacuumlo Update copyright for 2025 2025-01-01 11:21:55 -05:00
xml2 Use PG_MODULE_MAGIC_EXT in our installable shared libraries. 2025-03-26 11:11:02 -04:00
contrib-global.mk
Makefile pg_overexplain: Additional EXPLAIN options for debugging. 2025-03-26 13:52:21 -04:00
meson.build pg_overexplain: Additional EXPLAIN options for debugging. 2025-03-26 13:52:21 -04:00
README

The PostgreSQL contrib tree
---------------------------

This subtree contains porting tools, analysis utilities, and plug-in
features that are not part of the core PostgreSQL system, mainly
because they address a limited audience or are too experimental to be
part of the main source tree.  This does not preclude their
usefulness.

User documentation for each module appears in the main SGML
documentation.

When building from the source distribution, these modules are not
built automatically, unless you build the "world" target.  You can
also build and install them all by running "make all" and "make
install" in this directory; or to build and install just one selected
module, do the same in that module's subdirectory.

Some directories supply new user-defined functions, operators, or
types.  To make use of one of these modules, after you have installed
the code you need to register the new SQL objects in the database
system by executing a CREATE EXTENSION command.  In a fresh database,
you can simply do

    CREATE EXTENSION module_name;

See the PostgreSQL documentation for more information about this
procedure.