mirror of
https://github.com/zebrajr/postgres.git
synced 2025-12-07 12:20:31 +01:00
4909b38af0
Data loss can happen when the DDLs like ALTER PUBLICATION ... ADD TABLE ... or ALTER TYPE ... that don't take a strong lock on table happens concurrently to DMLs on the tables involved in the DDL. This happens because logical decoding doesn't distribute invalidations to concurrent transactions and those transactions use stale cache data to decode the changes. The problem becomes bigger because we keep using the stale cache even after those in-progress transactions are finished and skip the changes required to be sent to the client. This commit fixes the issue by distributing invalidation messages from catalog-modifying transactions to all concurrent in-progress transactions. This allows the necessary rebuild of the catalog cache when decoding new changes after concurrent DDL. We observed performance regression primarily during frequent execution of *publication DDL* statements that modify the published tables. The regression is minor or nearly nonexistent for DDLs that do not affect the published tables or occur infrequently, making this a worthwhile cost to resolve a longstanding data loss issue. An alternative approach considered was to take a strong lock on each affected table during publication modification. However, this would only address issues related to publication DDLs (but not the ALTER TYPE ...) and require locking every relation in the database for publications created as FOR ALL TABLES, which is impractical. The bug exists in all supported branches, but we are backpatching till 14. The fix for 13 requires somewhat bigger changes than this fix, so the fix for that branch is still under discussion. Reported-by: hubert depesz lubaczewski <depesz@depesz.com> Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Author: Shlok Kyal <shlok.kyal.oss@gmail.com> Author: Hayato Kuroda <kuroda.hayato@fujitsu.com> Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Tested-by: Benoit Lobréau <benoit.lobreau@dalibo.com> Backpatch-through: 14 Discussion: https://postgr.es/m/de52b282-1166-1180-45a2-8d8917ca74c6@enterprisedb.com Discussion: https://postgr.es/m/CAD21AoAenVqiMjpN-PvGHL1N9DWnHSq673bfgr6phmBUzx=kLQ@mail.gmail.com |
||
|---|---|---|
| .. | ||
| binary.out | ||
| catalog_change_snapshot.out | ||
| concurrent_ddl_dml.out | ||
| concurrent_stream.out | ||
| ddl.out | ||
| decoding_in_xact.out | ||
| decoding_into_rel.out | ||
| delayed_startup.out | ||
| invalidation_distrubution.out | ||
| messages.out | ||
| mxact.out | ||
| oldest_xmin.out | ||
| ondisk_startup.out | ||
| permissions.out | ||
| prepared.out | ||
| replorigin.out | ||
| rewrite.out | ||
| skip_snapshot_restore.out | ||
| slot_creation_error.out | ||
| slot.out | ||
| snapshot_transfer.out | ||
| spill.out | ||
| stats.out | ||
| stream.out | ||
| subxact_without_top.out | ||
| time.out | ||
| toast.out | ||
| truncate.out | ||
| twophase_snapshot.out | ||
| twophase_stream.out | ||
| twophase.out | ||
| xact.out | ||