Render traces

This paste explains what render traces are in the Pony Preservation Project's animation dataset.

Introduction

Render traces are recordings of all actions required to render a scene. They consist of the following files:

• shapes.json: This consists of XFL DOMShape objects that describe basic shapes. These are vector shapes made up of straight lines, bezier curves, line strokes, and fills.
• frames.json: This is a DAG (directed acyclic graph). Each node represents a frame, and each edge represents information for how to render one frame as part of another (including any matrix transformation or color transformations required).
• labels.json: This is a table of labels for each frame. This is explained in detail below.

Understanding timelines and layers

                                               timeline
frame     0   1   2   3   4   5   6   7  8 ...
         ------------------------------------------------------------------------
layer 1 |                                                                       |
layer 2 |                                                                       |
layer 3 |                                                                       |
layer 4 |                                                                       |
         ------------------------------------------------------------------------

Animation is represented as a collection timelines, each of which is a 2D grid. A vertical slice of a timeline corresponds to a still shot, and it is called a frame. A horizontal slice of a timeline is called a layer. To render a scene, each frame of a timeline is rendered in sequence. Each cell of a timeline table represents a composition of other timeline frames that get posed and rendered together. The DAG structure of frames.json comes from the fact that each timeline table cell references potentially other timeline frames.

• Note: a vertical slice of a layer (i.e., a cell in the timeline table) is also called a frame.

To render a timeline frame:

1. Each corresponding layer frame is collected and rendered separately. This is done by rendering the required references and posing them (applying the appropriate transformations and masks). In labels.json, these aggregate frames (each corresponding to a single cell in a timeline table) has a layer label and a frame label identifying the cell.
2. The layer frames are aggregated into a single timeline frame. This is done by rendering each layer frame in sequence. In labels.json, these aggregate frames (each corresponding to a vertical slice in a timeline table) has a timeline label and a frame label identifying the vertical slice.

labels.json describes how frames fit into these timeline tables. Some frames in frames.json correspond to vertical slices of a timeline table, and these frames are identified by (timeline, frame index) pairs. Some frames in frames.json correspond to individual cells of a timeline table, and these frames are identified by (layer, frame index) pairs.

Understanding the DAG of frames

Frames have the following attributes:

• children: Each child is rendered recursively, then the various following transformations are applied to get the final frame render.
• mask: This render mask is used to mask out (stencil) parts of the frame so that not everything gets rendered. This is often important for rendering eyes correctly.
• filter: This corresponds to an RGBA color transformation.
• transform: This corresponds to a matrix transformation (shift, scale, rotate, skew).

The recursion bottoms out at basic shapes, each of which is described as a DOMShape in shapes.json.

Understanding DOMShapes

A DOMShape is an XML node consisting of lines, bezier curves, strokes, and fills. This defines a shape as a vector image. There is an xfl_domshape_to_svg function in domshape.shape of the xflsvg library, which turns the DOMShape into SVG data.

You can use the xflsvg library to compile DOMShapes directly to SVG.

# TODO: this amount to wrapping the DOMShape in a ShapeFrame, then rendering the ShapeFrame.

The xflsvg library

You can compile XFL to render traces, XFL to SVG, and render traces to SVG.

To compile XFL to render traces:

$ python -m xflsvg --export INPUT_FOLDER OUTPUT_FOLDER

def export(input_folder, output_folder):
    xfl = XflReader(input_folder)
    timeline = xfl.get_timeline()
    exporter = TableExporter()

    for frame in list(timeline):
        with exporter:
            frame.render()
        exporter.save_frame()

    shapes, frames, labels, rendered = exporter.compile_frames()
    with open(os.path.join(output_folder, 'shapes.json'), 'w') as outp:
        json.dump(shapes, outp)
    with open(os.path.join(output_folder, 'frames.json'), 'w') as outp:
        json.dump(frames, outp)
    with open(os.path.join(output_folder, 'labels.json'), 'w') as outp:
        json.dump(labels, outp)

To compile XFL to SVG:

# TODO: the script currently only takes render traces as input when compiling to SVG. I can modify it to take XFL files as input.

To compile render traces to SVG:

$ python -m render svg INPUT_FOLDER --timeline TIMELINE --width WIDTH --height HEIGHT --x OFFSET_X --y OFFSET_Y [--scale MULTIPLIER] OUTPUT_FOLDER

# TODO: Wrap render trace handling in a RenderTraceReader class so converting these to SVG is the same as converting XFL to SVG.