Synchronization of concurrent computation engines
摘要
說明書
Independently, the fourth execution engine 640 may execute instructions until it executes checkpoint wait instructions (i.e., Wait (Ckpt4-1, 1) 641 and Wait (Ckpt4-2, 1) 643) indicating dependencies on operations to be performed by the first execution engine 610 second execution engine 620. The fourth execution engine 640 may halt execution of instructions and wait for the values of to be remotely set in the local checkpoint register Ckpt4-1 and Ckpt4-2 associated with the fourth execution engine 640. Here, the fourth execution engine 440 has a dependency on the completion of an operation by the first execution engine 610 to load weights into a state buffer (i.e., “Copy W2 to SB”) 615 and a dependency on the completion of an operation by the second execution engine 620 to complete a computation instruction (i.e., “MatMult”) 625. Upon completion of the Copy W2 to SB instruction 615, the first execution engine 610 may have executed an instruction (i.e., Set (Ckpt4-1, 1)) 613 to remotely set a value in the local checkpoint register Ckpt4-1. Upon completion of the MatMult instruction 625, the second execution engine 620 may have remotely set the value in the local checkpoint register Ckpt4-2 associated with the fourth execution engine 460.
When the values are set 613, 622 in the local checkpoint register Ckpt4, the fourth execution engine 440 may resume executing instructions. One of ordinary skill in the art will appreciate that the second execution engine 620 may remotely set the local checkpoint register Ckpt4-2 associated with the fourth execution engine 460 followed by the first execution engine remotely setting the local checkpoint register Ckpt4-1 with the same result.
