Stack trie

The stack trie is a way of getting a quick orientation on where all the compilations in a model take place, esp., if you are compiling a codebase you are unfamiliar with. It is a tree of stack frames, for all stacks that triggered PT2 compilation. If only a single stack is in the tree, you will simply see a plain list of frames (most recent call last). With multiple stacks, at every point where two stacks diverge from having a common prefix, we increase the indentation of the list and have a separate sub-list per sub-tree.

Links to particular compilation are color coded by status: [Success], [Success with restart (e.g., graph break)], [Empty graph], [Error], [Metrics were missing]

Failures and Restarts

Various issues may cause Dynamo to restart its analysis or give up on compilation entirely, causing graph breaks and fallbacks to eager mode. This run had 21 restart(s) and/or compilation failure(s).

IR dumps

The IR dumps collected dumped intermediate products from various points of the PT2 compilation process. The products are organized by compile id, and then sorted in chronological order.

A compile id uniquely identifies are particular compilation inside a PT2 program. It is traditionally written as [x/y], where the frame id x identifies the particular Python frame which we are compiling, and frame compile id y identifies how many times we've recompiled this same frame. For example, [0/0] refers to the very first frame compiled by PT2; [0/1] refers to the first recompilation of this frame, while [1/0] refers to a different frame, within distinct code cache, which we are compiling next (perhaps because of a graph break). Although Dynamo treats distinct frames as completely unrelated, a frame compilation could overlap with another frame; for example, if you graph break in an inlined function, Dynamo will typically try to compile the nested frame again on an inner frame. You can identify the hierarchical relationship between frames by looking at the stack trie above.

In some situations, the compile id will have an extra signifier [x/y_z], where z is the attempt for this particular (re)compilation. Certain conditions will cause Dynamo to restart analysis, when Dynamo discovers that it needs to undo a decision it previously made. The most common cause of recompilation is a graph break in an inlined function call, which forces to restart and avoid inlining the function in the first place.

Here is a high level description of PT2's compilation phases, and the intermediate products each phase generates:

  1. Optional: If compiled autograd is enabled, and we are processing a backward call, compiled autograd will trace the autograd graph from the autograd engine, and produce an FX graph compiled_autograd_graph that will be Dynamo traced. Otherwise, Dynamo will directly trace user's bytecode.
  2. Dynamo symbolically evaluates the Python bytecode of a program, producing dynamo_output_graph
  3. Optional: If optimize_ddp is enabled, the DDPOptimizer will split the Dynamo output graph to improve pipelining communications. Each split subgraph is optimize_ddp_split_child_submod, and the high level graph that plumbs the graphs together is optimize_ddp_split_graph. If there are multiple splits, each subsequent build product will be produced multiple times, one for each split.
  4. AOTAutograd traces the (possibly split) Dynamo output graph, producing a aot_joint_graph if backwards is enabled. It then partitions the graph into aot_forward_graph and aot_backward_graph. If training is not needed, there may only be an aot_forward_graph.
  5. Inductor will apply some post grad FX passes, producing inductor_post_grad_graph
  6. Inductor will perform code generation, producing the final inductor_output_code which will be executed at runtime. This output is a valid Python program and can be directly run.

Build products below: