The pg_plan_advice module allows key planner decisions
to be described, reproduced, and altered using a special-purpose "plan
advice" mini-language. It is intended to allow stabilization of plan choices
that the user believes to be good, as well as experimentation with plans that
the planner believes to be non-optimal.
Note that, since the planner often makes good decisions, overriding its judgement can easily backfire. For example, if the distribution of the underlying data changes, the planner normally has the option to adjust the plan in an attempt to preserve good performance. If the plan advice prevents this, a very poor plan may be chosen. It is important to use plan advice only when the risks of constraining the planner's choices are outweighed by the benefits.
In order to use this module, the pg_plan_advice module
must be loaded. You can do this on a system-wide basis by adding
pg_plan_advice to
shared_preload_libraries and restarting the
server, or by adding it to
session_preload_libraries and starting a new session,
or by loading it into an individual session using the
LOAD command.
Once the pg_plan_advice module is loaded,
EXPLAIN will support
a PLAN_ADVICE option. You can use this option to see
a plan advice string for the chosen plan. For example:
EXPLAIN (COSTS OFF, PLAN_ADVICE)
SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
QUERY PLAN
------------------------------------
Hash Join
Hash Cond: (f.dim_id = d.id)
-> Seq Scan on join_fact f
-> Hash
-> Seq Scan on join_dim d
Generated Plan Advice:
JOIN_ORDER(f d)
HASH_JOIN(d)
SEQ_SCAN(f d)
NO_GATHER(f d)
In this example, the user has not specified any advice; instead, the
planner has been permitted to make whatever decisions it thinks best, and
those decisions are memorialized in the form of an advice string.
JOIN_ORDER(f d) means that f should
be the driving table, and the first table to which it should be joined is
d. HASH_JOIN(d) means that
d should appear on the inner side of a hash join.
SEQ_SCAN(f d) means that both f
and d should be accessed via a sequential scan.
NO_GATHER(f d) means that neither f
nor d should appear beneath a Gather
or Gather Merge node. For more details on the plan
advice mini-language, see the information on
advice targets and
advice tags, below.
Once you have an advice string for a query, you can use it to control how
that query is planned. You can do this by setting
pg_plan_advice.advice to the advice string you've
chosen. This can be an advice string that was generated by the system,
or one you've written yourself. One good way of creating your own advice
string is to take the string generated by the system and pick out just
those elements that you wish to enforce. In the example above,
pg_plan_advice emits advice for the join order, the
join method, the scan method, and the use of parallelism, but you might
only want to control the join order:
SET pg_plan_advice.advice = 'JOIN_ORDER(f d)';
EXPLAIN (COSTS OFF)
SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
QUERY PLAN
------------------------------------
Hash Join
Hash Cond: (f.dim_id = d.id)
-> Seq Scan on join_fact f
-> Hash
-> Seq Scan on join_dim d
Supplied Plan Advice:
JOIN_ORDER(f d) /* matched */
Since the PLAN_ADVICE option to
EXPLAIN was not specified, no advice string is generated
for the plan. However, the supplied plan advice is still shown so that
anyone looking at the EXPLAIN output knows that the
chosen plan was influenced by plan advice. If information about supplied
plan advice is not desired, it can be suppressed by configuring
pg_plan_advice.always_explain_supplied_advice = false.
For each piece of supplied advice, the output shows
advice feedback indicating
whether or not the advice was successfully applied to the query. In this
case, the feedback says /* matched */, which means that
f and d were found in the query and
that the resulting query plan conforms to the specified advice.
Plan advice is written imperatively; that is, it specifies what should be
done. However, at an implementation level,
pg_plan_advice works by telling the core planner what
should not be done. In other words, it operates by constraining the
planner's choices, not by replacing it. Therefore, no matter what advice
you provide, you will only ever get a plan that the core planner would have
considered for the query in question. If you attempt to force what you
believe to be the correct plan by supplying an advice string, and the
planner still fails to produce the desired plan, this means that either
there is a bug in your advice string, or the plan in question was not
considered viable by the core planner. This commonly happens for one of two
reasons. First, it might be that the planner believes that the plan you're
trying to force would be semantically incorrect - that is, it would produce
the wrong results - and for that reason it wasn't considered. Second, it
might be that the planner rejected the plan you were hoping to generate on
some grounds other than cost. For example, given a very simple query such as
SELECT * FROM some_table, the query planner will
decide that the use of an index is worthless here before it performs any
costing calculations. You cannot force it to use an index for this query
even if you set enable_seqscan = false, and you can't
force it to use an index using plan advice, either.
Specifying plan advice should never cause planner failure. However, if you
specify plan advice that asks for something impossible, you may get a plan
where some plan nodes are flagged as Disabled: true in
the EXPLAIN output. In some cases, such plans will be
basically the same plan you would have gotten with no supplied advice at
all, but in other cases, they may be much worse. For example:
SET pg_plan_advice.advice = 'JOIN_ORDER(x f d)';
EXPLAIN (COSTS OFF)
SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
QUERY PLAN
----------------------------------------------------
Nested Loop
Disabled: true
-> Seq Scan on join_fact f
-> Index Scan using join_dim_pkey on join_dim d
Index Cond: (id = f.dim_id)
Supplied Plan Advice:
JOIN_ORDER(x f d) /* partially matched */
Because neither f nor d is the
first table in the JOIN_ORDER() specification, the
planner disables all direct joins between the two of them, thinking that
the join to x should happen first. Since planning isn't
allowed to fail, a disabled plan between the two relations is eventually
selected anyway, but here it's a Nested Loop rather than
the Hash Join that was chosen in the above example where
no advice was specified. There are several different ways that this kind
of thing can happen; when it does, the resulting plan is generally worse
than if no advice had been specified at all. Therefore, it is a good idea
to validate that the advice you specify applies to the query to which it
is applied and that the results are as expected.
An advice target uniquely identifies a particular
instance of a particular relation involved in a particular query. In simple
cases, such as the examples shown above, the advice target is simply the
relation alias. However, a more complex syntax is required when subqueries
are used, when tables are partitioned, or when the same relation alias is
mentioned more than once in the same subquery (e.g., (foo JOIN bar
ON foo.a = bar.a) x JOIN foo ON x.b = foo.b). Any combination of
these three things can occur simultaneously: a relation could be mentioned
more than once, be partitioned, and be used inside of a subquery.
Because of this, the general syntax for a relation identifier is:
alias_name#occurrence_number/partition_schema.partition_name@plan_name
All components except for the alias_name are optional
and are included only when required. When a component is omitted, the
preceding punctuation must also be omitted. For the first occurrence of a
relation within a given subquery, generated advice will omit the occurrence
number, but it is legal to write #1, if desired. The
partition schema and partition name are included only for children of
partitioned tables. In generated advice, pg_plan_advice
always includes both, but it is legal to omit the schema. The plan name is
omitted for the top-level plan, and must be included for any subplan.
It is not always easy to determine the correct advice target by examining
the query. For instance, if the planner pulls up a subquery into the parent
query level, everything inside of it becomes part of the parent query level,
and uses the parent query's subplan name (or no subplan name, if pulled up
to the top level). Furthermore, the correct subquery name is sometimes not
obvious. For example, when two queries are joined using an operation such as
UNION or INTERSECT, no name for the
subqueries is present in the SQL syntax; instead, a system-generated name is
assigned to each branch. The easiest way to discover the proper advice
targets is to use EXPLAIN (PLAN_ADVICE) and examine the
generated advice.
An advice tag specifies a particular behavior that should be enforced for some portion of the query, such as a particular join order or join method. All advice tags take advice targets as arguments, and many allow lists of advice targets, which in some cases can be nested multiple levels deep. Several different classes of advice targets exist, each controlling a different aspect of query planning.
SEQ_SCAN(target[ ... ]) TID_SCAN(target[ ... ]) INDEX_SCAN(targetindex_name[ ... ]) INDEX_ONLY_SCAN(targetindex_name[ ... ]) FOREIGN_SCAN((target[ ... ]) [ ... ]) BITMAP_HEAP_SCAN(target[ ... ])
SEQ_SCAN specifies that each target should be
scanned using a Seq Scan. TID_SCAN
specifies that each target should be scanned using a
TID Scan or TID Range Scan.
BITMAP_HEAP_SCAN specifies that each target
should be scanned using a Bitmap Heap Scan.
INDEX_SCAN specifies that each target should
be scanned using an Index Scan on the given index
name. INDEX_ONLY_SCAN is similar, but specifies the
use of an Index Only Scan. In either case, the index
name can be, but does not have to be, schema-qualified.
FOREIGN_SCAN specifies that a join between two or
more foreign tables should be pushed down to a remote server so
that it can be implemented as a single Foreign Scan.
Specifying FOREIGN_SCAN for a single foreign table is
neither necessary nor permissible: a Foreign Scan will
need to be used regardless. If you want to prevent a join from being
pushed down, consider using the JOIN_ORDER tag for
that purpose.
The planner supports many types of scans other than those listed here;
however, in most of those cases, there is no meaningful decision to be
made, and hence no need for advice. For example, the output of a
set-returning function that appears in the FROM clause
can only ever be scanned using a Function Scan, so
there is no opportunity for advice to change anything.
JOIN_ORDER(join_order_item[ ... ]) wherejoin_order_itemis:advice_target| (join_order_item[ ... ] ) | {join_order_item[ ... ] }
When JOIN_ORDER is used without any sublists, it
specifies an outer-deep join with the first advice target as the driving
table, joined to each subsequent advice target in turn in the order
specified. For instance, JOIN_ORDER(a b c) means that
a should be the driving table, and that it should be
joined first to b and then to c.
If there are more relations in the query than a,
b, and c, the rest can be joined
afterwards in any manner.
If a JOIN_ORDER list contains a parenthesized sublist,
it specifies a non-outer-deep join. The relations in the sublist must first
be joined to each other much as if the sublist were a top-level
JOIN_ORDER list, and the resulting join product must
then appear on the inner side of a join at the appropriate point in the
join order. For example, JOIN_ORDER(a (b c) d) requires
a plan of this form:
Join
-> Join
-> Scan on a
-> Join
-> Scan on b
-> Scan on c
-> Scan on d
If a JOIN_ORDER list contains a sublist surrounded by
curly braces, this also specifies a non-outer-deep join. However, the join
order within the sublist is not constrained. For example, specifying
JOIN_ORDER(a {b c} d) would allow the scans of
b and c to be swapped in the
previous example, which is not allowed when parentheses are used.
Parenthesized sublists can be arbitrarily nested, but sublists surrounded by curly braces cannot themselves contain sublists.
Multiple instances of JOIN_ORDER() can sometimes be
needed in order to fully constrain the join order. This occurs when there
are multiple join problems that are optimized separately by the planner.
This can happen due to the presence of subqueries, or because there is a
partitionwise join. In the latter case, each branch of the partitionwise
join can have its own join order, independent of every other branch.
join_method_name(join_method_item[ ... ]) wherejoin_method_nameis: { MERGE_JOIN_MATERIALIZE | MERGE_JOIN_PLAIN | NESTED_LOOP_MATERIALIZE | NESTED_LOOP_PLAIN | HASH_JOIN } andjoin_method_itemis: {advice_target| (advice_target[ ... ] ) }
Join method advice specifies the relation, or set of relations, that should
appear on the inner side of a join using the named join method. For
example, HASH_JOIN(a b) means that each of
a and b should appear on the inner
side of a hash join; a conforming plan must contain at least two hash
joins, one of which has a and nothing else on the
inner side, and the other of which has b and nothing
else on the inner side. On the other hand,
HASH_JOIN((a b)) means that the join product of
a and b should appear together
on the inner side of a single hash join.
Note that join method advice implies a negative join order constraint.
Since the named relation or relations must be on the inner side of a join
using the specified method, none of them can be the driving table for the
entire join problem. Moreover, no relation inside the set should be joined
to any relation outside the set until all relations within the set have
been joined to each other. For example, if the advice specifies
HASH_JOIN((a b)) and the system begins by joining either
of those to some third relation c, the resulting
plan could never be compliant with the request to put exactly those two
relations on the inner side of a hash join. When using both join order
advice and join method advice for the same query, it is a good idea to make
sure that they do not mandate incompatible join orders.
PARTITIONWISE(partitionwise_item[ ... ]) wherepartitionwise_itemis: {advice_target| (advice_target[ ... ] ) }
When applied to a single target, PARTITIONWISE
specifies that the specified table should not be part of any partitionwise
join. When applied to a list of targets, PARTITIONWISE
specifies that exactly that set of relations should be joined in
partitionwise fashion. Note that, regardless of what advice is specified,
no partitionwise joins will be possible if
enable_partitionwise_join = off.
SEMIJOIN_UNIQUE(sj_unique_item[ ... ]) SEMIJOIN_NON_UNIQUE(sj_unique_item[ ... ]) wheresj_unique_itemis: {advice_target| (advice_target[ ... ] ) }
The planner sometimes has a choice between implementing a semijoin
directly and implementing a semijoin by making the nullable side unique
and then performing an inner join. SEMIJOIN_UNIQUE
specifies the latter strategy, while SEMIJOIN_NON_UNIQUE
specifies the former strategy. In either case, the argument is the single
relation or list of relations that appear beneath the nullable side of the
join.
GATHER(gather_item[ ... ]) GATHER_MERGE(gather_item[ ... ]) NO_GATHER(advice_target[ ... ]) wheregather_itemis: {advice_target| (advice_target[ ... ] ) }
GATHER or GATHER_MERGE specifies
that Gather or Gather Merge,
respectively, should be placed on top of the single relation specified as
a target, or on top of the join between the list of relations specified as
a target. This means that GATHER(a b c) is a request
for three different Gather nodes, while
GATHER((a b c)) is a request for a single
Gather node on top of a 3-way join.
NO_GATHER specifies that no Gather or
Gather Merge node should appear above any of the
targets, but it only constrains the planning of an individual subquery,
and outer subquery levels can still use parallel query. For example,
NO_GATHER(inner_example@any_1) precludes using a
Parallel Seq Scan to access the
inner_example table within the any_1
subquery, but it does not prevent the planner from placing
SubPlan any_1 beneath a Gather
or Gather Merge node. The following plan is
compatible with NO_GATHER(inner_example@any_1), but
not with NO_GATHER(outer_example):
Finalize Aggregate
-> Gather
-> Partial Aggregate
-> Parallel Seq Scan on outer_example
Filter: (something = (hashed SubPlan any_1).col1)
SubPlan any_1
-> Seq Scan on inner_example
Filter: (something_else > 100)
Here is the reverse case, that is, a plan compatible with
NO_GATHER(outer_example) but not with
NO_GATHER(inner_example@any_1):
Aggregate
-> Seq Scan on outer_example
Filter: (something = (hashed SubPlan any_1).col1)
SubPlan any_1
-> Gather
-> Parallel Seq Scan on inner_example
Filter: (something_else > 100)
EXPLAIN provides feedback on whether supplied advice was
successfully applied to the query in the form of a comment on each piece
of supplied advice. For example:
SET pg_plan_advice.advice = 'hash_join(f g) join_order(f g) index_scan(f no_such_index)';
SET
rhaas=# EXPLAIN (COSTS OFF) SELECT * FROM jo_fact f
LEFT JOIN jo_dim1 d1 ON f.dim1_id = d1.id
LEFT JOIN jo_dim2 d2 ON f.dim2_id = d2.id
WHERE val1 = 1 AND val2 = 1;
QUERY PLAN
-------------------------------------------------------------------
Hash Join
Hash Cond: ((d1.id = f.dim1_id) AND (d2.id = f.dim2_id))
-> Nested Loop
-> Seq Scan on jo_dim2 d2
Filter: (val2 = 1)
-> Materialize
-> Seq Scan on jo_dim1 d1
Filter: (val1 = 1)
-> Hash
-> Seq Scan on jo_fact f
Supplied Plan Advice:
INDEX_SCAN(f no_such_index) /* matched, inapplicable, failed */
HASH_JOIN(f) /* matched */
HASH_JOIN(g) /* not matched */
JOIN_ORDER(f g) /* partially matched */
For this query, f is a valid advice target, but
g is not. Therefore, the request to place
f on the inner side of a hash join is listed as
matched, but the request to place g
on the inner side of a hash join is listed as
not matched. The JOIN_ORDER advice
tag involves one valid target and one invalid target, and so is listed as
partially matched. Note that
HASH_JOIN(f g) is actually a request for two logically
separate behaviors, whereas JOIN_ORDER(f g) is a single
request. When providing advice feedback, EXPLAIN shows
each logical request separately, together with all the feedback applicable
to that request type.
Advice feedback can include any of the following:
matched means that all of the specified advice targets
were observed together during query planning, at a time at which the
advice could be enforced.
partially matched means that some but not all of the
specified advice targets were observed during query planning, or all
of the advice targets were observed but not together. For example, this
may happen if all the targets of JOIN_ORDER advice
individually match the query, but the proposed join order is not legal.
not matched means that none of the
specified advice targets were observed during query planning. This may
happen if the advice simply doesn't match the query, or it may
occur if the relevant portion of the query was not planned, perhaps
because it was gated by a condition that was simplified to constant false.
inapplicable means that the advice tag could not
be applied to the advice targets for some reason. For example, this will
happen if the use of a nonexistent index is requested, or if an attempt
is made to control semijoin uniqueness for a non-semijoin.
conflicting means that two or more pieces of advice
request incompatible behaviors. For example, if you advise a sequential
scan and an index scan for the same table, both requests will be flagged
as conflicting. This also commonly happens if join method advice or
semijoin uniqueness advice implies a join order incompatible with the
one explicitly specified; see
Section F.32.4.3.
failed means that the query plan does not comply with
the advice. This only occurs for entries that are also shown as
matched. It frequently occurs for entries that are
also marked as conflicting or
inapplicable. However, it can also occur when the
advice is valid insofar as pg_plan_advice is able
to determine, but the planner is not able to construct a legal
plan that can comply with the advice. It is important to note that the
sanity checks performed by pg_plan_advice are fairly
superficial and focused mostly on looking for logical inconsistencies in
the advice string; only the planner knows what will actually work.
All advice should be marked as exactly one of matched,
partially matched, or not matched.
pg_plan_advice.advice (string)
pg_plan_advice.advice is an advice string to be
used during query planning.
pg_plan_advice.always_explain_supplied_advice (boolean)
pg_plan_advice.always_explain_supplied_advice causes
EXPLAIN to always show any supplied advice and the
associated
advice feedback.
The default value is true. If set to
false, this information will be displayed only when
EXPLAIN (PLAN_ADVICE) is used.
pg_plan_advice.always_store_advice_details (boolean)
pg_plan_advice.always_store_advice_details allows
EXPLAIN to show details related to plan advice even
when prepared queries are used. The default value is
false. When planning a prepared query, it is not
possible to know whether EXPLAIN will later be used,
so by default, to reduce overhead, pg_plan_advice
will not generate plan advice or feedback on supplied advice. This means
that if EXPLAIN EXECUTE is used on the prepared query,
it will not be able to show this information. Changing this setting to
true avoids this problem, but adds additional
overhead. It is probably a good idea to enable this option only in
sessions where it is needed, rather than on a system-wide basis.
pg_plan_advice.feedback_warnings (boolean)
When set to true, pg_plan_advice.feedback_warnings
emits a warning whenever supplied plan advice is not successfully
enforced. The default value is false.
pg_plan_advice.trace_mask (boolean)
When pg_plan_advice.trace_mask is
true, pg_plan_advice will print
messages during query planning each time that
pg_plan_advice alters the mask of allowable query
plan types in response to supplied plan advice. The default value is
false. The messages printed by this setting are not
expected to be useful except for purposes of debugging this module.
It is currently not possible to control any aspect of the planner's behavior with respect to aggregation. This includes both whether aggregates are computed by sorting or hashing, and also whether strategies such as eager aggregation or partitionwise aggregation are used.
It also is currently not possible to control any aspect of the planner's
behavior with respect to set operations such as UNION
or INTERSECT.
As discussed above under How It Works, the use of plan advice can only affect which plan the planner chooses from among those it believes to be viable. It can never force the choice of a plan which the planner refused to consider in the first place.
Robert Haas <rhaas@postgresql.org>