PostgreSQLのパーティショニング機能は次期PostgreSQL12で大幅に性能が改善しています。 富士通が扱うシステムでは、パーティション数が1000を超えることも多い中、PostgreSQLはそのような大量データを扱うシステムにも耐えうるのか。 PostgreSQL12のパーティショニング性能の実力に迫ります。

1. Great performance at scale:
Approaching the real ability of partitioning
performance in PostgreSQL 12
2019/9/25
FUJITSU Limited
Sho Kato/ Naoki Yotsunaga
Copyright 2019 FUJITSU LIMITED1
db tech showcase Tokyo 2019

2. In this talk
n Evolution of partitioning in PostgreSQL 11
n Whatʻs new in PostgreSQL 11
n Performance of partitioning in PostgreSQL 11
n Problem of partitioning in PostgreSQL 11
n Enhancement of partitioning in PostgreSQL 12
n Performance of partitioning in PostgreSQL 12
n Tips and pitfalls from our experience
n Problem and solution in over 1,000s partitions
Copyright 2019 FUJITSU LIMITED2

3. Whatʻs new in PostgreSQL 11
n Hash partitioning
n Enables partitioning by a hashed key column
n PRIMARY KEY, FOREIGN KEY, indexes, and triggers
n Enhanced Partition Pruning
n Partition Pruning…Eliminates unneeded partitions from the processing
n Improves SELECT performance
n Partition wise join
n Processing a join operation effectively
n Improves a join query performance
» Partitioning is strengthened more and more!
Copyright 2019 FUJITSU LIMITED3

4. But, limitation is still remains..
n PostgreSQL 11 manual says that
Copyright 2019 FUJITSU LIMITED4

5. User needs
n We have users who have a table partitioned into 5,500 in DBMS x
n Oops, 5,500 partition is too large..
n They are also attracted to PostgreSQL
n Therefore, verify whether their performance requirements can be satisfied by
PostgreSQL 11 !
Copyright 2019 FUJITSU LIMITED5

6. Benchmark
n Purpose
n To see improvements of PostgreSQL 11 compared to PostgreSQL 10
n To confirm that PostgreSQL 11 can satisfy the performance requirements of users who owns
1,000s of partitions
n Workload
n Read/write a single record (assume OLTP workload)
n Execute each SELECT/UPDATE/DELETE/INSERT statement
n Use JdbcRunner which is open source benchmark tool
n Table definition
n RANGE partitioning, increasing # of partitions to 2, 4,…, 8,192
n Each partition has 1,000 records
n Type of partition key is integer and create B-tree index on partition key
n Comparing
n PostgreSQL 11.1, PostgreSQL 10.6
Copyright 2019 FUJITSU LIMITED6

7. Benchmark environment
n Hardware
n CPU: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz(8 core) * 2
n Memory: 260GB
n Disk: 2.2TB (SSD)
n OS: Red Hat Enterprise Linux Server release 6.5 64bit
n Database tuning
n shared_buffers = 100GB (All records are buffered)
n max_worker_processes = 0, max_parallel_workers_per_gather = 0, max_parallel_workers = 0
n max_wal_size = 20GB, checkpoint_timeout = 1d
n Other
n Single client session
n Client and server are running on the same machine
Copyright 2019 FUJITSU LIMITED7

8. PostgreSQL 11 vs PostgreSQL 10 (1/4)
Copyright 2019 FUJITSU LIMITED8

9. PostgreSQL 11 vs PostgreSQL 10 (2/4)
Copyright 2019 FUJITSU LIMITED9

10. PostgreSQL 11 vs PostgreSQL 10 (3/4)
Copyright 2019 FUJITSU LIMITED
15187.5
14171.5
8444.4
7515.8
6147.9
4421.6
2789.2
1580.9
853.2
257
113.6
38.7
17.3
8.1
14848.6
13668.1
7981.7
6738.9
4963.7
3342.4
1968.7
1077.9
574.2
196.3
82.5
31.8
14.7
6.9
0 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192
TPS
# PARTITIONS
SELECT
PostgreSQL 11 PostgreSQL 10
10

11. PostgreSQL 11 vs PostgreSQL 10 (4/4)
Copyright 2019 FUJITSU LIMITED11

12. Summary of comparison between PostgreSQL 11 and
PostgreSQL 10
n In few partitions
n Performance of SELECT improves up to 24%
n Performance of INSERT improves up to 10%
n But, performance of DELETE/UPDATE are almost same
n In too many partitions
n With 1,000s of partitions, the same performance as PostgreSQL 10 in all SQLs
n In DBMS x, performance does not change even if # of partitions increases
Copyright 2018 FUJITSU LIMITED12

13. Why does such a difference occur?
n Question
n The algorithm for choosing one partition should not be so different.
n Perhaps is there a bottleneck in the process of handling partitions in PostgreSQL?
n Method for proving
n Execute each DELETE/INSERT/SELECT/UPDATE statement 10 times
and measure the average elapsed time in each module without records
n How to measure
n Use log_XXX_stats = on
Copyright 2019 FUJITSU LIMITED13

14. Overview of query processing in PostgreSQL
n PARSER
n Analyze SQL syntax and convert SQL to Parse tree
n SQL -> Parse tree
n PARSE ANALISYS
n Add information like oid and converts the list of relations in Parse tree to the range table
n Parse tree -> Query tree
n REWRITER
n According to the rule system, rewrite the query tree
n Query tree -> Query tree
n PLANNER
n Refer to statistics and create access plan
n Query tree -> Plan tree
n EXECUTOR
n Execute access plan
Copyright 2019 FUJITSU LIMITED14

15. Elapsed time of each module in PostgreSQL 11 (1/2)
Copyright 2019 FUJITSU LIMITED
11 seconds 150 milliseconds
15

16. Elapsed time of each module in PostgreSQL 11 (2/2)
Copyright 2019 FUJITSU LIMITED
6.8 milliseconds
16

17. Summary of the verification
n Which module have the bottleneck?
n SELECT/DELETE/UPDATE has the bottleneck in PLANNER
n INSERT has the bottleneck in EXECUTOR
n Why is it taking time?
n PostgreSQL 11 has faster partition pruning.
So, planning time should be shorter?
n Executor processes only one record.
So, execution time should be short?
Copyright 2019 FUJITSU LIMITED17

18. Why Planner is slow?
n Partition Pruning
n Creating plans for only targeted partitions
n But the planner is still slow, why?
n Reasons
n Opening all partitions and processing them (SELECT/DELETE/UPDATE)
• Creating queryʼs range table and PlannerInfo
n Creating plans for all partitions (DELETE/UPDATE)
• We can use Partition Pruning for only a SELECT query currently
Copyright 2019 FUJITSU LIMITED
testdb=# explain select * from test.accounts where aid = 1 or aid = 8192;
QUERY PLAN
-------------------------------------------------------------------------
Append (cost=0.00..88.03 rows=46 width=8)
-> Seq Scan on account_part_1 (cost=0.00..43.90 rows=23 width=8)
Filter: ((aid = 1) OR (aid = 8192))
-> Seq Scan on account_part_8192 (cost=0.00..43.90 rows=23 width=8)
Filter: ((aid = 1) OR (aid = 8192))
18

19. Why Executor is slow?
n How Executor works
n Process a plan by 4 steps
1. ExecutorStart():Collecting information(ex . system catalog) for executing a plan
2. ExecutorRun(): Executing a plan
3. ExecutorFinish(): Processing some process like After Trigger, CTE
4. ExecutorEnd(): Releasing resources
n Are there any bottlenecks?
n Reasons
n Locking all partitions with RowExclusiveLock in ExecutorStart()
• Performance degrades as number of partitions increase
Copyright 2019 FUJITSU LIMITED19

20. Can we improve performance in PostgreSQL 12?
n Commits for improving partitioning performance
n Speed up planning when partitions can be pruned at plan time. (428b260f87e)
n Delay lock acquisition for partitions until we route a tuple to them. (9eefba181f7)
n Redesign initialization of partition routing structures (3f2393edefa)
Copyright 2019 FUJITSU LIMITED20

21. Speed up planning with partitions
n Problems
n Opening all partitions and processing them (SELECT/DELETE/UPDATE)
• Creating queryʼs range table and PlannerInfo
n Creating plans for all partitions (DELETE/UPDATE)
• We can use Partition Pruning for only a SELECT query currently
n Solutions
n Doing Partition Pruning before opening partitions
n Applying Partition Pruning for DELETE/UPDATE queries
Copyright 2018 FUJITSU LIMITED21

22. How much performance improves in PostgreSQL 12?
n Purpose
n To confirm that PostgreSQL 12 can satisfy the performance requirements of users who owns
1,000s of partitions
n Workload
n Same as the ones in measuring PostgreSQL 11
n Table definition
n plan_cache_mode = force_custom_plan
n Other definitions are same as the ones in measuring PostgreSQL 11
n Comparing
n PostgreSQL 12beta3, PostgreSQL 11.1
Copyright 2019 FUJITSU LIMITED22

23. PostgreSQL 12 vs PostgreSQL 11 (1/4)
Copyright 2019 FUJITSU LIMITED23

24. PostgreSQL 12 vs PostgreSQL 11 (2/4)
Copyright 2019 FUJITSU LIMITED24

25. PostgreSQL 12 vs PostgreSQL 11 (3/4)
Copyright 2019 FUJITSU LIMITED25

26. PostgreSQL 12 vs PostgreSQL 11 (4/4)
Copyright 2019 FUJITSU LIMITED26

27. Performance improves a lot in PostgreSQL 12
n Comparison in 8,192 partitions
n Performance doesnʼt depend on numbers of partitions
n Great improvements from PostgreSQL 11
Copyright 2019 FUJITSU LIMITED
PostgreSQL 12 PostgreSQL 11
DELETE 5,447 0.1
INSERT 6,705 118
SELECT 7,979 8.1
UPDATE 4,814 0.1
27

28. Tips and pitfalls from our experience
n We are migrating a large system using other DBMS to PostgreSQL 12
n Finally, weʼve achieved almost the same performance
n But, we encountered various problems…
n Weʼll introduce some of the problems and provide tips
n Cases
n Two tables are divided into 1,000 partitions, one with 200 million and another with 20 million
records
n Online processing that creates multiple temporary tables from these tables and joins them
n 40x speedup (120s → 3s) by using the tricks 1 and 2
1. Conditions in subquery do not prune outside partitions
2. Operator overload blocks partition pruning
3. TRUNCATE of one partition blocks access to other partitions
Copyright 2019 FUJITSU LIMITED28

29. Conditions in subquery do not prune outside partitions
n Problem
n Outer query is very slow due to access to unexpectedly many partitions
n Cause
n Partition pruning in subquery is not propagated to outer query when subquery uses GROUP BY
n Workaround
n Add subqueryʼs WHERE conditions to parent query
Copyright 2019 FUJITSU LIMITED
SELECT ・・・ FROM
(SELECT ・・・ FROM T11 WHERE T11.C1 = value1 GROUP BY C1,・・・) T1, T2
WHERE T1.C1 = T2.C1;
SELECT ・・・ FROM
(SELECT ・・・ FROM T11 WHERE T11.C1 = value1 GROUP BY C1, ・・・) T1, T2
WHERE T1.C1 = T2.C1
AND T2.C1 = value1;
29

30. Operator overload blocks partition pruning
n Problem
n Partition pruning does not work when comparing char type partition key and varchar type values
n Cause
n Overloaded the = operator to avoid App changes for explicit casting
n About comparing char and varchar
n Workaround
n Removed overloading of = operator causing problems
Copyright 2019 FUJITSU LIMITED
char = varchar
char::text = varchar::text
Overload
partition_key (char) = value (varchar)
col1=ʻaaaʼ char(5), col2=ʻaaaʼ varchar(5)
・PostgreSQL ・DBMS x
col1 = col2 → ʻaaaʼ = ʻaaaʼ → true col1 = col2 → ʻaaa ʼ = ʻaaaʼ→ false
30

31. TRUNCATE of one partition blocks access to other partitions
n Problem
n While one session TRUNCATEs a partition, other session cannot read/write other partitions of
the same table
n This occurs only when plan_cache_mode = force_generic_plan and EXECUTE statement is
executed
n Cause
n Lock conflict occurs on the TRUNCATEd partition because all partitions are accessed to create a
generic plan
n Workaround
n Use DELETE instead of TRUNCATE
Copyright 2019 FUJITSU LIMITED
TRUNCATE
child table
parent table
Lock conflict
access
31

32. Conclusion
n PostgreSQL 11 is limited to a few hundred partitions in practice
n PostgreSQL 12 can handle thousands of partitions with small overhead
n But, We need to make much effort of query tuning
Copyright 2019 FUJITSU LIMITED32

33. Copyright 2019 FUJITSU LIMITED