In various examples, global checkpoint registers can be used to synchronize the operations of two or more execution engines of the integrated circuit device. FIG. 4 is a sequence diagram illustrating an example of using checkpoints to synchronize execution engines in a global checkpoint register implementation in accordance with various aspects of the present disclosure. As illustrated in FIG. 4, a first execution engine (e.g., a direct memory access (DMA) engine) 410, a second execution engine (e.g., a processing engine (PE)) 420, a third execution engine (e.g., an activation engine (ACT) engine) 430, and a fourth execution engine (e.g., a pooling (POOL) engine) 440, each capable of independently executing instructions. The first execution engine 410 may execute an instruction, for example, copying weights to a state buffer, and when the instruction is complete, the first execution engine 410 may execute an instruction to set a value (i.e., a checkpoint) in a global checkpoint register. For example, the first execution engine 410 may execute an instruction (i.e., Set 1 to Ckpt2) 411 to set a value of “1” in the global checkpoint register Ckpt2. The global checkpoint register Ckpt2 may broadcast 412 its value to all of the execution engines.
