Synchronization of concurrent computation engines
摘要
說明書
The first execution engine 610 may execute an instruction, for example, copying weights to a state buffer. When the instruction is complete, the first execution engine 610 may execute an instruction to set a value (i.e., a checkpoint) in a local checkpoint register associated with the second execution engine 620. For example, the first execution engine 610 may execute an instruction (i.e., Set 1 to Ckpt2) 611 to remotely set 612 a value of “1” in the local checkpoint register Ckpt2 associated with the second execution engine 620. The first execution engine 610 may continue executing instructions.
Independently, the second execution engine 620 executes instructions until it executes a checkpoint wait instruction to wait on the value “1” (i.e., Wait (Ckpt2, 1) 621 remotely set by the first execution engine 610 in the local checkpoint register Ckpt2 621 associated with the second execution engine 620. The second execution engine 620 may have a dependency on the operation performed by the first execution engine 610 and may halt execution of instructions until the value “1” set in the local checkpoint register Ckpt2. When the value “1” is set in the local checkpoint register Ckpt2, the second execution engine 620 may resume executing instructions. The second execution engine 620 may continue executing instructions and may execute an instruction (i.e., Set (Ckpt4-2, 1)) 623 to remotely set a value of “1” in the local checkpoint register Ckpt4-2 associated with the fourth execution engine 460. The second execution engine 620 may continue executing instructions.
