-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Undo changes in Postgres core for building GIST/GIN indexes #414
Commits on Feb 6, 2024
-
Neon rebased to REL_15_STABLE (up to 0a9045c)
Most significant changes are: - `xlog.c` refactoring - some code was moved to `xlogreader.c` and `xlogprefetcher.c`. - `ThisTimeLineID` refactoring (4a92a1c and e997a0c), which affects walproposer code - `XLogFileInit` refactoring, Multiple commits changed the function signature. - resolve initdb and pg_waldump neon-specific options that conflictes with the ones from PostgreSQL. -
Configuration menu - View commit details
-
Copy full SHA for b2f37fc - Browse repository at this point
Copy the full SHA b2f37fcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 58cfeeb - Browse repository at this point
Copy the full SHA 58cfeebView commit details -
Configuration menu - View commit details
-
Copy full SHA for 65dd0f7 - Browse repository at this point
Copy the full SHA 65dd0f7View commit details -
Configuration menu - View commit details
-
Copy full SHA for b2dbd9d - Browse repository at this point
Copy the full SHA b2dbd9dView commit details -
Prevent access to uninitalized shaerd memory in InstallXLogFileSegmen…
…t, which is used for safekeepers-sync
Configuration menu - View commit details
-
Copy full SHA for 283fcdf - Browse repository at this point
Copy the full SHA 283fcdfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 205d59d - Browse repository at this point
Copy the full SHA 205d59dView commit details -
Move backpressure throttling implementation to neon extension (#203)
* Move backpressure throttling implementation to neon extension and function for monitoring throttling time * Update src/include/miscadmin.h Co-authored-by: Heikki Linnakangas <[email protected]> Co-authored-by: Heikki Linnakangas <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 246bae1 - Browse repository at this point
Copy the full SHA 246bae1View commit details -
Configuration menu - View commit details
-
Copy full SHA for ad18f68 - Browse repository at this point
Copy the full SHA ad18f68View commit details -
Local prefetch implementation for Postgres 15
Disabled by default. The plan is to merge this now, so that we can do performance testing quickly, and if it helps, rewrite and review it properly. Author: Konstantin Knizhnik
Configuration menu - View commit details
-
Copy full SHA for 9b168ed - Browse repository at this point
Copy the full SHA 9b168edView commit details -
Set last written LSN for the created relation (#212)
Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 5237824 - Browse repository at this point
Copy the full SHA 5237824View commit details -
Use normal install program to install server headers.
Commit a703269 replaced $(INSTALL) with plain "cp" for installing the server header files. It sped up "make install" significantly, because the old logic called $(INSTALL) separately for every header file, whereas plain "cp" could copy all the files in one command. However, we have long since made it a requirement that $(INSTALL) can also install multiple files in one command, see commit f1c5247. Switch back to $(INSTALL). Discussion: https://www.postgresql.org/message-id/200503252305.j2PN52m23610%40candle.pha.pa.us Discussion: https://www.postgresql.org/message-id/2415283.1641852217%40sss.pgh.pa.us
Configuration menu - View commit details
-
Copy full SHA for 871400c - Browse repository at this point
Copy the full SHA 871400cView commit details -
Update expected output for sysviews test because of changed default v…
…alue of enable_seqscan_prefetch
Configuration menu - View commit details
-
Copy full SHA for 945737b - Browse repository at this point
Copy the full SHA 945737bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3c976a2 - Browse repository at this point
Copy the full SHA 3c976a2View commit details -
Set Neon-specific FMGR_ABI_EXTRA
to support only extensions that were built against Neon PostgreSQL
Configuration menu - View commit details
-
Copy full SHA for 67d1c0a - Browse repository at this point
Copy the full SHA 67d1c0aView commit details -
Don't use newline in PG_VERSION file.
Neon generates PG_VERSION files in one format - just major version number without newline. Be consistent with it
Configuration menu - View commit details
-
Copy full SHA for c85bc09 - Browse repository at this point
Copy the full SHA c85bc09View commit details -
Unset ArchiveRecoveryRequested for Neon code path.
No need to perform WAL recovery in Neon Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for d363e94 - Browse repository at this point
Copy the full SHA d363e94View commit details -
Configuration menu - View commit details
-
Copy full SHA for eefa704 - Browse repository at this point
Copy the full SHA eefa704View commit details -
Configuration menu - View commit details
-
Copy full SHA for ed3602e - Browse repository at this point
Copy the full SHA ed3602eView commit details -
Pin pages with speculative insert tuples to prevent their reconstruct…
…ion because spec_token is not wal logged (#223) * Pin pages with speculative insert tuples to prevent their reconstruction because spec_token is not wal logged refer ##2587 * Update src/backend/access/heap/heapam.c Co-authored-by: Heikki Linnakangas <[email protected]> Co-authored-by: Heikki Linnakangas <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for f50d2de - Browse repository at this point
Copy the full SHA f50d2deView commit details -
Fix shared memory initialization for last written LSN cache (#226)
* Fix shared memory initialization for last written LSN cache Replace (from,till) with (from,n_blocks) for SetLastWrittenLSNForBlockRange function * Fast exit from SetLastWrittenLSNForBlockRange for n_blocks == 0
Configuration menu - View commit details
-
Copy full SHA for 2df27d7 - Browse repository at this point
Copy the full SHA 2df27d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7c26185 - Browse repository at this point
Copy the full SHA 7c26185View commit details -
Stamp XLP_FIRST_IS_CONTRECORD only if we start writing with page offset.
Without this patch, on bootstrap XLP_FIRST_IS_CONTRECORD has been always put on header of a page where WAL writing continues. This confuses WAL decoding on safekeepers, making it think decoding starts in the middle of a record, leading to 2022-08-12T17:48:13.816665Z ERROR {tid=37}: query handler for 'START_WAL_PUSH postgresql://no_user:@localhost:15050' failed: failed to run ReceiveWalConn Caused by: 0: failed to process ProposerAcceptorMessage 1: invalid xlog page header: unexpected XLP_FIRST_IS_CONTRECORD at 0/2CF8000 Rebase of a1af529 for v14.
Configuration menu - View commit details
-
Copy full SHA for ea88e63 - Browse repository at this point
Copy the full SHA ea88e63View commit details -
Move walredo process code under pgxn in the main 'neon' repository.
- Refactor the way the WalProposerMain function is called when started with --sync-safekeepers. The postgres binary now explicitly loads the 'neon.so' library and calls the WalProposerMain in it. This is simpler than the global function callback "hook" we previously used. - Move the WAL redo process code to a new library, neon_walredo.so, and use the same mechanism as for --sync-safekeepers to call the WalRedoMain function, when launched with --walredo argument. - Also move the seccomp code to neon_walredo.so library. I kept the configure check in the postgres side for now, though.
Configuration menu - View commit details
-
Copy full SHA for ae333c0 - Browse repository at this point
Copy the full SHA ae333c0View commit details -
Misc cleanup, mostly to reduce unnecessary differences with upstream.
Fix indentation, remove unused definitions, resolve some FIXMEs.
Configuration menu - View commit details
-
Copy full SHA for 7ab6932 - Browse repository at this point
Copy the full SHA 7ab6932View commit details -
Optimize prefetch patterns in both heap seqscan and vacuum scans. (#228)
Previously, we called PrefetchBuffer [NBlkScanned * seqscan_prefetch_buffers] times in each of those situations, but now only NBlkScanned. In addition, the prefetch mechanism for the vacuum scans is now based on blocks instead of tuples - improving the efficiency.
Configuration menu - View commit details
-
Copy full SHA for 7e5a1f0 - Browse repository at this point
Copy the full SHA 7e5a1f0View commit details -
Fix prefetch issues in parallel scans and vacuum's cleanup scan (#235)
Parallel seqscans didn't take their parallelism into account when determining which block to prefetch, and vacuum's cleanup scan didn't correctly determine which blocks would need to be prefetched, and could get into an infinite loop.
Configuration menu - View commit details
-
Copy full SHA for 9544eaa - Browse repository at this point
Copy the full SHA 9544eaaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3a06992 - Browse repository at this point
Copy the full SHA 3a06992View commit details -
Use prefetch in pg_prewarm extension (#237)
* Use prefetch in pg_prewarm extension * Change prefetch order as suggested in review
Configuration menu - View commit details
-
Copy full SHA for 4d4f707 - Browse repository at this point
Copy the full SHA 4d4f707View commit details -
* Update prefetch mechanisms: - **Enable enable_seqscan_prefetch by default** - Store prefetch distance in the relevant scan structs - Slow start sequential scan, to accommodate LIMIT clauses. - Replace seqscan_prefetch_buffer with the relations' tablespaces' *_io_concurrency; and drop seqscan_prefetch_buffer as a result. - Clarify enable_seqscan_prefetch GUC description - Fix prefetch in pg_prewarm - Add prefetching to autoprewarm worker - Fix an issue where we'd incorrectly not prefetch data when hitting a table wraparound. The same issue also resulted in assertion failures in debug builds. - Fix parallel scan prefetching - we didn't take into account that parallel scans have scan synchronization, too.
Configuration menu - View commit details
-
Copy full SHA for d2453bd - Browse repository at this point
Copy the full SHA d2453bdView commit details -
Configuration menu - View commit details
-
Copy full SHA for ebf2fec - Browse repository at this point
Copy the full SHA ebf2fecView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d5c7be - Browse repository at this point
Copy the full SHA 0d5c7beView commit details -
Maintain last written LSN for each page to enable prefetch on vacuum,… (
#245) * Maintain last written LSN for each page to enable prefetch on vacuum, delete and other massive update operations * Move PageSetLSN in heap_xlog_visible before MarkBufferDirty
Configuration menu - View commit details
-
Copy full SHA for 9bcaa14 - Browse repository at this point
Copy the full SHA 9bcaa14View commit details -
- Prefetch the pages in index vacuum's sequential scans Implemented in NBTREE, GIST and SP-GIST. BRIN does not have a 2nd phase of vacuum, and both GIN and HASH clean up their indexes in a non-seqscan fashion: GIN scans the btree from left to right, and HASH only scans the initial buckets sequentially.
Configuration menu - View commit details
-
Copy full SHA for 3ace5a3 - Browse repository at this point
Copy the full SHA 3ace5a3View commit details -
* Show prefetch statistic in EXPLAIN refer #2994 * Update heap pge LSN in case of VM changes even if wal_redo_hints=off refer #2807 * Undo occasional changes * Undo occasional changes
Configuration menu - View commit details
-
Copy full SHA for 20d665d - Browse repository at this point
Copy the full SHA 20d665dView commit details -
Show prefetch statistic in EXPLAIN (#249)
* Show prefetch statistic in EXPLAIN refer #2994 * Collect per-node prefetch statistics * Show number of prefetch duplicates in explain
Configuration menu - View commit details
-
Copy full SHA for 8c1ae26 - Browse repository at this point
Copy the full SHA 8c1ae26View commit details -
Implement efficient prefetch for parallel bitmap heap scan (#258)
* Implement efficient prefetch for parallel bitmap heap scan * Change MAX_IO_CONCURRENCY to be power of 2
Configuration menu - View commit details
-
Copy full SHA for 723f85f - Browse repository at this point
Copy the full SHA 723f85fView commit details -
* Avoid errors when accessing indexes of unlogge tables after compute restart * Support unlogged sequences * Extract sequence start value from pg_sequence * Initialize unlogged index undex eclusive lock
Configuration menu - View commit details
-
Copy full SHA for 1621daf - Browse repository at this point
Copy the full SHA 1621dafView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1c049dc - Browse repository at this point
Copy the full SHA 1c049dcView commit details -
Revert handling of UNLOGGED tables on compute side v15.
They will be handled in pageserver, ref neondatabase/neon#3706 This reverts commit ad5e789 This reverts commit 46c44e8 This does *not* revert commit 285cd13. We likely should do that, but check_restored_datadir_content complains in some diff in init fork contents after test_pg_regress, this should be sorted out.
Configuration menu - View commit details
-
Copy full SHA for d1e3fb3 - Browse repository at this point
Copy the full SHA d1e3fb3View commit details -
Allow external main functions to skip config load and make last
written LSN cache optional.
Configuration menu - View commit details
-
Copy full SHA for a9a9b43 - Browse repository at this point
Copy the full SHA a9a9b43View commit details -
Remove walredo-related hacks from InternalIpcMemoryCreate()
Now similar kind of hack (using malloc() instead of shmem) is done in the wal-redo extension.
Configuration menu - View commit details
-
Copy full SHA for c2b4266 - Browse repository at this point
Copy the full SHA c2b4266View commit details -
Adjust prefetch target for parallel bitmap scan (#274)
* Adjust prefetch target for parallel bitmap scan * More fixes for parallel bitmap scan prefetch
Configuration menu - View commit details
-
Copy full SHA for 8fea298 - Browse repository at this point
Copy the full SHA 8fea298View commit details -
Configuration menu - View commit details
-
Copy full SHA for 59f7ecb - Browse repository at this point
Copy the full SHA 59f7ecbView commit details -
Prefetch for index and index-only scans (#271)
* Pefeth for index and inex-only scans * Remove debug logging * Move prefetch_blocks array to the end of BTScanOpaqueData struct
Configuration menu - View commit details
-
Copy full SHA for 34608d6 - Browse repository at this point
Copy the full SHA 34608d6View commit details -
[PG15] Feature/replicas (#279)
* Recovery requirements: Add condition variable for WAL recovery; allowing backends to wait for recovery up to some record pointer. * Fix issues w.r.t. WAL when LwLsn is initiated and when recovery starts. This fixes some test failures that showed up after updating Neon code to do more precise handling of replica's get_page_at_lsn's request_lsn lsns. --------- Co-authored-by: Matthias van de Meent <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 96fca0d - Browse repository at this point
Copy the full SHA 96fca0dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e3e5075 - Browse repository at this point
Copy the full SHA e3e5075View commit details -
Configuration menu - View commit details
-
Copy full SHA for 875fc76 - Browse repository at this point
Copy the full SHA 875fc76View commit details -
Configuration menu - View commit details
-
Copy full SHA for 58b7244 - Browse repository at this point
Copy the full SHA 58b7244View commit details -
Configuration menu - View commit details
-
Copy full SHA for b3b692c - Browse repository at this point
Copy the full SHA b3b692cView commit details -
Make it possible to grant self created roles (#298)
Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for ac82176 - Browse repository at this point
Copy the full SHA ac82176View commit details -
Update expected file for create_role test (#301)
* Make it possible to grant self created roles * Update expected file for create_role test --------- Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for c6c5d17 - Browse repository at this point
Copy the full SHA c6c5d17View commit details -
Define NEON_SMGR in smgr.h to make it possible for extensions to use …
…extetnded Neon SMGR API (#300) Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 429ecaf - Browse repository at this point
Copy the full SHA 429ecafView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9e1dd46 - Browse repository at this point
Copy the full SHA 9e1dd46View commit details -
Upadte last written LSN after walloging all createdb stuff (#307)
Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 1d89f6d - Browse repository at this point
Copy the full SHA 1d89f6dView commit details -
[refer #111] Persist logical replication files in WAL (#306)
* [refer #111] Persist logical rep;lication files in WAL and include then in basebackup at PS * Fix warnings * Write origin logical record snapshot in WAL only if there are valid origins * Store only logical replication slots * Fix dropping replication slots * Replace sprintf with snprintf to make Arnica happy * Do not checkpoint replication origin at shutdown * Add PreCheckPointGuts function to sync replication state before start of shutdown checkpoint * Log heap rewrite file after creation. --------- Co-authored-by: Konstantin Knizhnik <[email protected]> Co-authored-by: Arseny Sher <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 1afb4f6 - Browse repository at this point
Copy the full SHA 1afb4f6View commit details -
Fix elog format error in wallog_mapping_file (#316)
Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 9ca7417 - Browse repository at this point
Copy the full SHA 9ca7417View commit details -
Update WAL buffers when restoring WAL at compute needed for LR (#324)
* Update WAL buffers when restoring WAL at compute needed for LR * Fix copying data in WAL buffers --------- Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for ac718a9 - Browse repository at this point
Copy the full SHA ac718a9View commit details -
Prevent output callbacks from hearing about neon-file messages (#329)
* Prevent output callbacks from hearing about neon-file messages
Configuration menu - View commit details
-
Copy full SHA for 98d8d6a - Browse repository at this point
Copy the full SHA 98d8d6aView commit details -
Configuration menu - View commit details
-
Copy full SHA for b9591d3 - Browse repository at this point
Copy the full SHA b9591d3View commit details -
Configuration menu - View commit details
-
Copy full SHA for b0297cd - Browse repository at this point
Copy the full SHA b0297cdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 940d29b - Browse repository at this point
Copy the full SHA 940d29bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8d18a8d - Browse repository at this point
Copy the full SHA 8d18a8dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0ff55c5 - Browse repository at this point
Copy the full SHA 0ff55c5View commit details -
On demand downloading of SLRU segments (#332)
* On demand downloading of SLRU segments * Fix smgr_read_slru_segment * Determine SLRU kind in extension * Use ctl->PagePrecedes for SLRU page comparison in SimpleLruDownloadSegment to address wraparround --------- Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 6ee78a3 - Browse repository at this point
Copy the full SHA 6ee78a3View commit details
Commits on Feb 13, 2024
-
Fix 'mmap' DSM implementation with allocations larger than 4 GB
Fixes bug #18341. Backpatch to all supported versions. Discussion: https://www.postgresql.org/message-id/[email protected]
Configuration menu - View commit details
-
Copy full SHA for ca2def9 - Browse repository at this point
Copy the full SHA ca2def9View commit details
Commits on Feb 21, 2024
-
Flush logical messages with snapshots and replication origin (#383)
Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 0baccce - Browse repository at this point
Copy the full SHA 0baccceView commit details
Commits on Feb 22, 2024
-
Set wasShutdown=true during hot-standby replica startup only when pri…
…mary is not alive (#364) * Set wasShutdown=true during hot-standby replica startup only when primary is not alive * Report fatal error if hot standaby replica is started with oldestAcriveXid=0 Postgres part of neondatabase/neon#6705 --------- Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for be91d91 - Browse repository at this point
Copy the full SHA be91d91View commit details -
Add comment explaining why it is safe to use FirstNormalTransactionXi…
…d for oldestActiveXid while replica startup (#388) Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 79b6351 - Browse repository at this point
Copy the full SHA 79b6351View commit details
Commits on Feb 26, 2024
-
Show information about local file cache in EXPLAIN ANALYZE (#385)
Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for e8b9a28 - Browse repository at this point
Copy the full SHA e8b9a28View commit details
Commits on Mar 7, 2024
-
Treat walproposer like walsenders in postmaster.
This keeps the walproposer processes alive at shutdown, until after the shutdown checkpoint has been written. That gives the walproposers a chance to stream it to the safekeepers.
Configuration menu - View commit details
-
Copy full SHA for 56f32c0 - Browse repository at this point
Copy the full SHA 56f32c0View commit details
Commits on Mar 12, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 80cef88 - Browse repository at this point
Copy the full SHA 80cef88View commit details
Commits on Mar 22, 2024
-
Remove Get/SetZenithCurrentClusterSize from Postgres core (#398)
Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for af9ab67 - Browse repository at this point
Copy the full SHA af9ab67View commit details -
Configuration menu - View commit details
-
Copy full SHA for e7651e7 - Browse repository at this point
Copy the full SHA e7651e7View commit details
Commits on Mar 28, 2024
-
fix: XLogFlush replication slot save/drop (#395)
* fix: XLogFlush replication slot drop Signed-off-by: Alex Chi Z <[email protected]> * fix all occurrences Signed-off-by: Alex Chi Z <[email protected]> --------- Signed-off-by: Alex Chi Z <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 64b8c7b - Browse repository at this point
Copy the full SHA 64b8c7bView commit details
Commits on Apr 10, 2024
-
Remember last written LSN when it is first requested (#411)
* Remember last written LSN when it is first requested * Use rnode instead of rlocator * Return updated LSN in SetLastWrittenLSN * Remove wrong new line --------- Co-authored-by: Konstantin Knizhnik <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 85d809c - Browse repository at this point
Copy the full SHA 85d809cView commit details
Commits on Apr 20, 2024
-
Undo changes in Postgres core for building GIST/GIN indexes
Konstantin Knizhnik committedApr 20, 2024 Configuration menu - View commit details
-
Copy full SHA for 6b1b8c2 - Browse repository at this point
Copy the full SHA 6b1b8c2View commit details
Commits on Apr 24, 2024
-
Add log_newpage_range_callback
Konstantin Knizhnik committedApr 24, 2024 Configuration menu - View commit details
-
Copy full SHA for 411a463 - Browse repository at this point
Copy the full SHA 411a463View commit details