# SDK Delegate Integration

[Delegate backends](compiler-delegate-and-partitioner.md) are a prominent component of on-device models due to their flexibility in defining behavior. A side effect of this flexibility is that it operates as an opaque transformation. This obfuscates rich associations and mutations that are valuable in post-processing.
- For example, if two different operator fusions were to occur within a delegate, post processing wouldn’t be able to separate the two transformations.

Specifically, it makes associating runtime information (such as profiling results) through delegated graphs difficult. Delegate Debug Identifiers provides a framework through which delegate authors can propagate this information and utilize it for post run analysis.

The preparation is broken down into two stages:
- **Ahead-of-time (AOT)**: Delegate authors generate a __Debug Handle Map__.
- **Runtime**: Delegate authors log using the __Delegate Debug Identifiers__ registered AOT in the __Debug Handle Map__.

## Ahead-of-Time
Delegate authors propagate what transformations occur in a lowered backend by returning a **Debug Handle Map** from the backend implementation.

### Generating a Debug Handle Map
**Debug Handle Maps** communicate what transformations occurred in a backend by mapping **Delegate Debug Identifiers** to debug handles.

**Delegate Debug Identifiers** are generated or user-provided identifiers for representing points of interest during runtime. Recall that debug handles are unique identifiers to operator instances in the model graph.

For example:
- **{ 0: (10, 11), 1: (11, 12) }:** Identifiers 0 and 1 in the runtime correspond to operators with the debug handles (10, 11) and (11, 12) respectively.
- **{ “Fancy Fusion”: (11, 12, 15) }**: Identifier “Fancy Fusion” in the runtime corresponds to operators with debug handles (11, 12, 15).

```{Note}
Identifiers are a means of connecting runtime results to the model graph; the interpretation of the identifiers is defined by the delegate author.
```

**Debug Handle Maps** are constructed through the use of **DelegateMappingBuilder** and returned as a part of `PreprocessResult`.

```python
class PreprocessResult:
    processed_bytes: bytes = bytes()

    debug_handle_map: Optional[
        Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
    ] = None
```
PreprocessResult is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/backend_details.py).

#### DelegateMappingBuilder
`DelegateMappingBuilder` is a helper class for managing and constructing Debug Handle Maps. The result of the builder should be passed in when constructing PreprocessResult.

`DelegateMappingBuilder` is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/utils.py)

A `DelegateMappingBuilder` instance can be constructed in one of 2 modes: manual identifiers or generated identifiers.

```python
# Manual Identifiers, Default
builder = DelegateMappingBuilder(generated_identifiers=False)

# Generated Identifiers
builder = DelegateMappingBuilder(generated_identifiers=True)
```

With **manual identifiers**, users pass in a **Delegate Debug Identifier** when creating entries.
With **generated identifiers**, the builder will auto-assign a **Delegate Debug Identifier**.

To add an entry to the **Debug Handle Map**, use `insert_delegate_mapping_entry`. It takes `fx.Node(s)` to associate and an optional **Delegate Debug Identifier** (used for the manual identifiers). The identifier recorded is returned from the call.

```python
def insert_delegate_mapping_entry(
    self,
    nodes: Union[fx.Node, List[fx.Node]],
    identifier: Optional[Union[int, str]] = None,
) -> Union[int, str]:
```

To retrieve the **Debug Handle Map**, use `get_delegate_mapping`.
```python
def get_delegate_mapping(
    self,
) -> Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
```

A demo of the AOT mapping can be found [here](https://github.com/pytorch/executorch/blob/main/exir/backend/test/backend_with_delegate_mapping_demo.py)


## Runtime
Corresponding to the AOT map, the runtime then defines the functionality through which these events are logged.

### Real-Time Logging

ExecuTorch allows you to log in real time. **Real time Logging** is useful when timestamps are available as the execution occurs. It provides minimal overhead and is intuitive for authors to call.

To log events in real-time (for example, explicitly denoting the profiling start and stop), `event_tracer_start_profiling_delegate` is used to create an `EventEntry` and `event_tracer_end_profiling_delegate` is used to conclude the `EventEntry` for the provided `EventTracer`.

To start an `EventTracerEntry` using `event_tracer_start_profiling_delegate`, the **Delegate Debug Identifier** (provided AOT to the `debug_handle_map`) is passed as either the name or `delegate_debug_id` argument depending on the **Delegate Debug Identifier** type (str and int respectively)

```c++
EventTracerEntry event_tracer_start_profiling_delegate(
    EventTracer* event_tracer,
    const char* name,
    DebugHandle delegate_debug_id)
```

To conclude an `EventTracerEntry`, `event_tracer_end_profiling_delegate` is simply provided the original `EventTracerEntry`.

Optionally, additional runtime `metadata` can also be logged at this point.

```c++
void event_tracer_end_profiling_delegate(
    EventTracer* event_tracer,
    EventTracerEntry event_tracer_entry,
    const char* metadata = nullptr)
```

### Post-Time Logging
ExecuTorch also allows you to log in post time. Some runtime settings don't have access to timestamps while it is executing. **Post-Time Logging** enables authors to still be able to log these events.

To log events in post (for example, logging start and end time simultaneously) `event_tracer_log_profiling_delegate` is called with a combination of the arguments used in the real-time logging API’s and timestamps.

```c++
void event_tracer_log_profiling_delegate(
    EventTracer* event_tracer,
    const char* name,
    DebugHandle delegate_debug_id,
    et_timestamp_t start_time,
    et_timestamp_t end_time,
    const char* metadata = nullptr)
```
A demo of the runtime code can be found [here](https://github.com/pytorch/executorch/blob/main/runtime/executor/test/test_backend_with_delegate_mapping.cpp).