F.32. pg_plan_advice — help the planner get the right plan #

F.32.1. Getting Started
F.32.2. How It Works
F.32.3. Advice Targets
F.32.4. Advice Tags
F.32.5. Advice Feedback
F.32.6. Configuration Parameters
F.32.7. Limitations
F.32.8. Author

The pg_plan_advice module allows key planner decisions to be described, reproduced, and altered using a special-purpose "plan advice" mini-language. It is intended to allow stabilization of plan choices that the user believes to be good, as well as experimentation with plans that the planner believes to be non-optimal.

Note that, since the planner often makes good decisions, overriding its judgement can easily backfire. For example, if the distribution of the underlying data changes, the planner normally has the option to adjust the plan in an attempt to preserve good performance. If the plan advice prevents this, a very poor plan may be chosen. It is important to use plan advice only when the risks of constraining the planner's choices are outweighed by the benefits.

F.32.1. Getting Started #

In order to use this module, the pg_plan_advice module must be loaded. You can do this on a system-wide basis by adding pg_plan_advice to shared_preload_libraries and restarting the server, or by adding it to session_preload_libraries and starting a new session, or by loading it into an individual session using the LOAD command.

Once the pg_plan_advice module is loaded, EXPLAIN will support a PLAN_ADVICE option. You can use this option to see a plan advice string for the chosen plan. For example:

EXPLAIN (COSTS OFF, PLAN_ADVICE)
        SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
             QUERY PLAN
------------------------------------
 Hash Join
   Hash Cond: (f.dim_id = d.id)
   ->  Seq Scan on join_fact f
   ->  Hash
         ->  Seq Scan on join_dim d
 Generated Plan Advice:
   JOIN_ORDER(f d)
   HASH_JOIN(d)
   SEQ_SCAN(f d)
   NO_GATHER(f d)

In this example, the user has not specified any advice; instead, the planner has been permitted to make whatever decisions it thinks best, and those decisions are memorialized in the form of an advice string. JOIN_ORDER(f d) means that f should be the driving table, and the first table to which it should be joined is d. HASH_JOIN(d) means that d should appear on the inner side of a hash join. SEQ_SCAN(f d) means that both f and d should be accessed via a sequential scan. NO_GATHER(f d) means that neither f nor d should appear beneath a Gather or Gather Merge node. For more details on the plan advice mini-language, see the information on advice targets and advice tags, below.

Once you have an advice string for a query, you can use it to control how that query is planned. You can do this by setting pg_plan_advice.advice to the advice string you've chosen. This can be an advice string that was generated by the system, or one you've written yourself. One good way of creating your own advice string is to take the string generated by the system and pick out just those elements that you wish to enforce. In the example above, pg_plan_advice emits advice for the join order, the join method, the scan method, and the use of parallelism, but you might only want to control the join order:

SET pg_plan_advice.advice = 'JOIN_ORDER(f d)';
EXPLAIN (COSTS OFF)
        SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
             QUERY PLAN
------------------------------------
 Hash Join
   Hash Cond: (f.dim_id = d.id)
   ->  Seq Scan on join_fact f
   ->  Hash
         ->  Seq Scan on join_dim d
 Supplied Plan Advice:
   JOIN_ORDER(f d) /* matched */

Since the PLAN_ADVICE option to EXPLAIN was not specified, no advice string is generated for the plan. However, the supplied plan advice is still shown so that anyone looking at the EXPLAIN output knows that the chosen plan was influenced by plan advice. If information about supplied plan advice is not desired, it can be suppressed by configuring pg_plan_advice.always_explain_supplied_advice = false. For each piece of supplied advice, the output shows advice feedback indicating whether or not the advice was successfully applied to the query. In this case, the feedback says /* matched */, which means that f and d were found in the query and that the resulting query plan conforms to the specified advice.

F.32.2. How It Works #

Plan advice is written imperatively; that is, it specifies what should be done. However, at an implementation level, pg_plan_advice works by telling the core planner what should not be done. In other words, it operates by constraining the planner's choices, not by replacing it. Therefore, no matter what advice you provide, you will only ever get a plan that the core planner would have considered for the query in question. If you attempt to force what you believe to be the correct plan by supplying an advice string, and the planner still fails to produce the desired plan, this means that either there is a bug in your advice string, or the plan in question was not considered viable by the core planner. This commonly happens for one of two reasons. First, it might be that the planner believes that the plan you're trying to force would be semantically incorrect - that is, it would produce the wrong results - and for that reason it wasn't considered. Second, it might be that the planner rejected the plan you were hoping to generate on some grounds other than cost. For example, given a very simple query such as SELECT * FROM some_table, the query planner will decide that the use of an index is worthless here before it performs any costing calculations. You cannot force it to use an index for this query even if you set enable_seqscan = false, and you can't force it to use an index using plan advice, either.

Specifying plan advice should never cause planner failure. However, if you specify plan advice that asks for something impossible, you may get a plan where some plan nodes are flagged as Disabled: true in the EXPLAIN output. In some cases, such plans will be basically the same plan you would have gotten with no supplied advice at all, but in other cases, they may be much worse. For example:

SET pg_plan_advice.advice = 'JOIN_ORDER(x f d)';
EXPLAIN (COSTS OFF)
        SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
                     QUERY PLAN
----------------------------------------------------
 Nested Loop
   Disabled: true
   ->  Seq Scan on join_fact f
   ->  Index Scan using join_dim_pkey on join_dim d
         Index Cond: (id = f.dim_id)
 Supplied Plan Advice:
   JOIN_ORDER(x f d) /* partially matched */

Because neither f nor d is the first table in the JOIN_ORDER() specification, the planner disables all direct joins between the two of them, thinking that the join to x should happen first. Since planning isn't allowed to fail, a disabled plan between the two relations is eventually selected anyway, but here it's a Nested Loop rather than the Hash Join that was chosen in the above example where no advice was specified. There are several different ways that this kind of thing can happen; when it does, the resulting plan is generally worse than if no advice had been specified at all. Therefore, it is a good idea to validate that the advice you specify applies to the query to which it is applied and that the results are as expected.

F.32.3. Advice Targets #

An advice target uniquely identifies a particular instance of a particular relation involved in a particular query. In simple cases, such as the examples shown above, the advice target is simply the relation alias. However, a more complex syntax is required when subqueries are used, when tables are partitioned, or when the same relation alias is mentioned more than once in the same subquery (e.g., (foo JOIN bar ON foo.a = bar.a) x JOIN foo ON x.b = foo.b). Any combination of these three things can occur simultaneously: a relation could be mentioned more than once, be partitioned, and be used inside of a subquery.

Because of this, the general syntax for a relation identifier is:

alias_name#occurrence_number/partition_schema.partition_name@plan_name

All components except for the alias_name are optional and are included only when required. When a component is omitted, the preceding punctuation must also be omitted. For the first occurrence of a relation within a given subquery, generated advice will omit the occurrence number, but it is legal to write #1, if desired. The partition schema and partition name are included only for children of partitioned tables. In generated advice, pg_plan_advice always includes both, but it is legal to omit the schema. The plan name is omitted for the top-level plan, and must be included for any subplan.

It is not always easy to determine the correct advice target by examining the query. For instance, if the planner pulls up a subquery into the parent query level, everything inside of it becomes part of the parent query level, and uses the parent query's subplan name (or no subplan name, if pulled up to the top level). Furthermore, the correct subquery name is sometimes not obvious. For example, when two queries are joined using an operation such as UNION or INTERSECT, no name for the subqueries is present in the SQL syntax; instead, a system-generated name is assigned to each branch. The easiest way to discover the proper advice targets is to use EXPLAIN (PLAN_ADVICE) and examine the generated advice.

F.32.4. Advice Tags #

An advice tag specifies a particular behavior that should be enforced for some portion of the query, such as a particular join order or join method. All advice tags take advice targets as arguments, and many allow lists of advice targets, which in some cases can be nested multiple levels deep. Several different classes of advice targets exist, each controlling a different aspect of query planning.

F.32.4.1. Scan Method Advice #

SEQ_SCAN(target [ ... ])
TID_SCAN(target [ ... ])
INDEX_SCAN(target index_name [ ... ])
INDEX_ONLY_SCAN(target index_name [ ... ])
FOREIGN_SCAN((target [ ... ]) [ ... ])
BITMAP_HEAP_SCAN(target [ ... ])

SEQ_SCAN specifies that each target should be scanned using a Seq Scan. TID_SCAN specifies that each target should be scanned using a TID Scan or TID Range Scan. BITMAP_HEAP_SCAN specifies that each target should be scanned using a Bitmap Heap Scan.

INDEX_SCAN specifies that each target should be scanned using an Index Scan on the given index name. INDEX_ONLY_SCAN is similar, but specifies the use of an Index Only Scan. In either case, the index name can be, but does not have to be, schema-qualified.

FOREIGN_SCAN specifies that a join between two or more foreign tables should be pushed down to a remote server so that it can be implemented as a single Foreign Scan. Specifying FOREIGN_SCAN for a single foreign table is neither necessary nor permissible: a Foreign Scan will need to be used regardless. If you want to prevent a join from being pushed down, consider using the JOIN_ORDER tag for that purpose.

The planner supports many types of scans other than those listed here; however, in most of those cases, there is no meaningful decision to be made, and hence no need for advice. For example, the output of a set-returning function that appears in the FROM clause can only ever be scanned using a Function Scan, so there is no opportunity for advice to change anything.

F.32.4.2. Join Order Advice #

JOIN_ORDER(join_order_item [ ... ])

where join_order_item is:

advice_target |
( join_order_item [ ... ] ) |
{ join_order_item [ ... ] }

When JOIN_ORDER is used without any sublists, it specifies an outer-deep join with the first advice target as the driving table, joined to each subsequent advice target in turn in the order specified. For instance, JOIN_ORDER(a b c) means that a should be the driving table, and that it should be joined first to b and then to c. If there are more relations in the query than a, b, and c, the rest can be joined afterwards in any manner.

If a JOIN_ORDER list contains a parenthesized sublist, it specifies a non-outer-deep join. The relations in the sublist must first be joined to each other much as if the sublist were a top-level JOIN_ORDER list, and the resulting join product must then appear on the inner side of a join at the appropriate point in the join order. For example, JOIN_ORDER(a (b c) d) requires a plan of this form:

Join
  ->  Join
        -> Scan on a
        -> Join
             -> Scan on b
             -> Scan on c
  ->  Scan on d

If a JOIN_ORDER list contains a sublist surrounded by curly braces, this also specifies a non-outer-deep join. However, the join order within the sublist is not constrained. For example, specifying JOIN_ORDER(a {b c} d) would allow the scans of b and c to be swapped in the previous example, which is not allowed when parentheses are used.

Parenthesized sublists can be arbitrarily nested, but sublists surrounded by curly braces cannot themselves contain sublists.

Multiple instances of JOIN_ORDER() can sometimes be needed in order to fully constrain the join order. This occurs when there are multiple join problems that are optimized separately by the planner. This can happen due to the presence of subqueries, or because there is a partitionwise join. In the latter case, each branch of the partitionwise join can have its own join order, independent of every other branch.

F.32.4.3. Join Method Advice #

join_method_name(join_method_item [ ... ])

where join_method_name is:

{ MERGE_JOIN_MATERIALIZE | MERGE_JOIN_PLAIN | NESTED_LOOP_MATERIALIZE | NESTED_LOOP_PLAIN | HASH_JOIN }

and join_method_item is:

{ advice_target |
( advice_target [ ... ] ) }

Join method advice specifies the relation, or set of relations, that should appear on the inner side of a join using the named join method. For example, HASH_JOIN(a b) means that each of a and b should appear on the inner side of a hash join; a conforming plan must contain at least two hash joins, one of which has a and nothing else on the inner side, and the other of which has b and nothing else on the inner side. On the other hand, HASH_JOIN((a b)) means that the join product of a and b should appear together on the inner side of a single hash join.

Note that join method advice implies a negative join order constraint. Since the named relation or relations must be on the inner side of a join using the specified method, none of them can be the driving table for the entire join problem. Moreover, no relation inside the set should be joined to any relation outside the set until all relations within the set have been joined to each other. For example, if the advice specifies HASH_JOIN((a b)) and the system begins by joining either of those to some third relation c, the resulting plan could never be compliant with the request to put exactly those two relations on the inner side of a hash join. When using both join order advice and join method advice for the same query, it is a good idea to make sure that they do not mandate incompatible join orders.

F.32.4.4. Partitionwise Advice #

PARTITIONWISE(partitionwise_item [ ... ])

where partitionwise_item is:

{ advice_target |
( advice_target [ ... ] ) }

When applied to a single target, PARTITIONWISE specifies that the specified table should not be part of any partitionwise join. When applied to a list of targets, PARTITIONWISE specifies that exactly that set of relations should be joined in partitionwise fashion. Note that, regardless of what advice is specified, no partitionwise joins will be possible if enable_partitionwise_join = off.

F.32.4.5. Semijoin Uniqueness Advice #

SEMIJOIN_UNIQUE(sj_unique_item [ ... ])
SEMIJOIN_NON_UNIQUE(sj_unique_item [ ... ])

where sj_unique_item is:

{ advice_target |
( advice_target [ ... ] ) }

The planner sometimes has a choice between implementing a semijoin directly and implementing a semijoin by making the nullable side unique and then performing an inner join. SEMIJOIN_UNIQUE specifies the latter strategy, while SEMIJOIN_NON_UNIQUE specifies the former strategy. In either case, the argument is the single relation or list of relations that appear beneath the nullable side of the join.

F.32.4.6. Parallel Query Advice #

GATHER(gather_item [ ... ])
GATHER_MERGE(gather_item [ ... ])
NO_GATHER(advice_target [ ... ])

where gather_item is:

{ advice_target |
( advice_target [ ... ] ) }

GATHER or GATHER_MERGE specifies that Gather or Gather Merge, respectively, should be placed on top of the single relation specified as a target, or on top of the join between the list of relations specified as a target. This means that GATHER(a b c) is a request for three different Gather nodes, while GATHER((a b c)) is a request for a single Gather node on top of a 3-way join.

NO_GATHER specifies that no Gather or Gather Merge node should appear above any of the targets, but it only constrains the planning of an individual subquery, and outer subquery levels can still use parallel query. For example, NO_GATHER(inner_example@any_1) precludes using a Parallel Seq Scan to access the inner_example table within the any_1 subquery, but it does not prevent the planner from placing SubPlan any_1 beneath a Gather or Gather Merge node. The following plan is compatible with NO_GATHER(inner_example@any_1), but not with NO_GATHER(outer_example):

 Finalize Aggregate
   ->  Gather
         ->  Partial Aggregate
               ->  Parallel Seq Scan on outer_example
                     Filter: (something = (hashed SubPlan any_1).col1)
                     SubPlan any_1
                       ->  Seq Scan on inner_example
                             Filter: (something_else > 100)

Here is the reverse case, that is, a plan compatible with NO_GATHER(outer_example) but not with NO_GATHER(inner_example@any_1):

 Aggregate
   ->  Seq Scan on outer_example
         Filter: (something = (hashed SubPlan any_1).col1)
         SubPlan any_1
           ->  Gather
                 -> Parallel Seq Scan on inner_example
                      Filter: (something_else > 100)

F.32.5. Advice Feedback #

EXPLAIN provides feedback on whether supplied advice was successfully applied to the query in the form of a comment on each piece of supplied advice. For example:

SET pg_plan_advice.advice = 'hash_join(f g) join_order(f g) index_scan(f no_such_index)';
SET
rhaas=# EXPLAIN (COSTS OFF)                                                     SELECT * FROM jo_fact f
        LEFT JOIN jo_dim1 d1 ON f.dim1_id = d1.id
        LEFT JOIN jo_dim2 d2 ON f.dim2_id = d2.id
        WHERE val1 = 1 AND val2 = 1;
                            QUERY PLAN
-------------------------------------------------------------------
 Hash Join
   Hash Cond: ((d1.id = f.dim1_id) AND (d2.id = f.dim2_id))
   ->  Nested Loop
         ->  Seq Scan on jo_dim2 d2
               Filter: (val2 = 1)
         ->  Materialize
               ->  Seq Scan on jo_dim1 d1
                     Filter: (val1 = 1)
   ->  Hash
         ->  Seq Scan on jo_fact f
 Supplied Plan Advice:
   INDEX_SCAN(f no_such_index) /* matched, inapplicable, failed */
   HASH_JOIN(f) /* matched */
   HASH_JOIN(g) /* not matched */
   JOIN_ORDER(f g) /* partially matched */

For this query, f is a valid advice target, but g is not. Therefore, the request to place f on the inner side of a hash join is listed as matched, but the request to place g on the inner side of a hash join is listed as not matched. The JOIN_ORDER advice tag involves one valid target and one invalid target, and so is listed as partially matched. Note that HASH_JOIN(f g) is actually a request for two logically separate behaviors, whereas JOIN_ORDER(f g) is a single request. When providing advice feedback, EXPLAIN shows each logical request separately, together with all the feedback applicable to that request type.

Advice feedback can include any of the following:

  • matched means that all of the specified advice targets were observed together during query planning, at a time at which the advice could be enforced.

  • partially matched means that some but not all of the specified advice targets were observed during query planning, or all of the advice targets were observed but not together. For example, this may happen if all the targets of JOIN_ORDER advice individually match the query, but the proposed join order is not legal.

  • not matched means that none of the specified advice targets were observed during query planning. This may happen if the advice simply doesn't match the query, or it may occur if the relevant portion of the query was not planned, perhaps because it was gated by a condition that was simplified to constant false.

  • inapplicable means that the advice tag could not be applied to the advice targets for some reason. For example, this will happen if the use of a nonexistent index is requested, or if an attempt is made to control semijoin uniqueness for a non-semijoin.

  • conflicting means that two or more pieces of advice request incompatible behaviors. For example, if you advise a sequential scan and an index scan for the same table, both requests will be flagged as conflicting. This also commonly happens if join method advice or semijoin uniqueness advice implies a join order incompatible with the one explicitly specified; see Section F.32.4.3.

  • failed means that the query plan does not comply with the advice. This only occurs for entries that are also shown as matched. It frequently occurs for entries that are also marked as conflicting or inapplicable. However, it can also occur when the advice is valid insofar as pg_plan_advice is able to determine, but the planner is not able to construct a legal plan that can comply with the advice. It is important to note that the sanity checks performed by pg_plan_advice are fairly superficial and focused mostly on looking for logical inconsistencies in the advice string; only the planner knows what will actually work.

All advice should be marked as exactly one of matched, partially matched, or not matched.

F.32.6. Configuration Parameters #

pg_plan_advice.advice (string)

pg_plan_advice.advice is an advice string to be used during query planning.

pg_plan_advice.always_explain_supplied_advice (boolean)

pg_plan_advice.always_explain_supplied_advice causes EXPLAIN to always show any supplied advice and the associated advice feedback. The default value is true. If set to false, this information will be displayed only when EXPLAIN (PLAN_ADVICE) is used.

pg_plan_advice.always_store_advice_details (boolean)

pg_plan_advice.always_store_advice_details allows EXPLAIN to show details related to plan advice even when prepared queries are used. The default value is false. When planning a prepared query, it is not possible to know whether EXPLAIN will later be used, so by default, to reduce overhead, pg_plan_advice will not generate plan advice or feedback on supplied advice. This means that if EXPLAIN EXECUTE is used on the prepared query, it will not be able to show this information. Changing this setting to true avoids this problem, but adds additional overhead. It is probably a good idea to enable this option only in sessions where it is needed, rather than on a system-wide basis.

pg_plan_advice.feedback_warnings (boolean)

When set to true, pg_plan_advice.feedback_warnings emits a warning whenever supplied plan advice is not successfully enforced. The default value is false.

pg_plan_advice.trace_mask (boolean)

When pg_plan_advice.trace_mask is true, pg_plan_advice will print messages during query planning each time that pg_plan_advice alters the mask of allowable query plan types in response to supplied plan advice. The default value is false. The messages printed by this setting are not expected to be useful except for purposes of debugging this module.

F.32.7. Limitations #

It is currently not possible to control any aspect of the planner's behavior with respect to aggregation. This includes both whether aggregates are computed by sorting or hashing, and also whether strategies such as eager aggregation or partitionwise aggregation are used.

It also is currently not possible to control any aspect of the planner's behavior with respect to set operations such as UNION or INTERSECT.

As discussed above under How It Works, the use of plan advice can only affect which plan the planner chooses from among those it believes to be viable. It can never force the choice of a plan which the planner refused to consider in the first place.

F.32.8. Author #

Robert Haas