Capacitor based resistive processing unit with symmetric weight update
地域：
NY
NY Armonk
摘要
Systems and methods for a capacitor based resistive processing unit with symmetrical weight updating include a first capacitor that stores a charge corresponding to a weight value. A readout circuit reads the charge stored in the first capacitor to apply a weight to an input value corresponding to an input signal using the weight value to produce an output. An update circuit updates the weight value stored in the first capacitor, including a second capacitor in communication with the first capacitor to transfer an amount of charge to the first capacitor according to an error of the output by changing a voltage difference across the first capacitor by a voltage change corresponding to the amount of charge, the voltage difference corresponding to the charge stored in the first capacitor.
說明書
The present invention generally relates to resistive processing units, and more particularly to capacitor based resistive processing units with symmetric weight update.
Hardware acceleration of neural networks can include analog storage of weights for performing operations such as matrix multiplication. Implementations of neural networks can include the learning of weights for a specific application, such as, e.g., for object recognition, speech recognition, natural language processing, and other applications. The learning of weights in a hardware accelerated neural network includes the updating of the weights stored in the analog storage devices. However, updating weights stored with analog devices can be difficult to implement because the update may not be symmetrical between additive and subtractive updates. This asymmetry reduces the performance and the training speed of the neural network. Mitigating the asymmetry by, for example, lengthening a channel of a transistor to reduce output conductance of a current source increases the size of the device. As a result, the device would be difficult to scale. Additionally, capacitor based resistive processing unit (RPU) designs can be sensitive to device-to-device variation, which degrades the performance of devices.
權(quán)利要求
What is claimed is:1. A capacitor based resistive processing unit with symmetrical weight updating, the resistive processing unit comprising:a first capacitor that stores a charge corresponding to a weight value; a readout circuit configured to read the charge stored in the first capacitor to apply a weight to an input value corresponding to an input signal using the weight value to produce an output; and an update circuit configured to update the weight value stored in the first capacitor, including:a second capacitor in communication with the first capacitor to transfer an amount of charge to or from the first capacitor according to an error of the output by changing a voltage difference across the first capacitor by a voltage change corresponding to the amount of charge, the voltage difference corresponding to the charge stored in the first capacitor; a first transfer gate in communication between the first capacitor and a first end of the second capacitor; and a second transfer gate in communication between a second end of the second capacitor and a current regulation device, wherein charge is transferred between the first capacitor and the second capacitor when the first transfer gate and the second transfer gate are closed. 2. The resistive processing unit as recited in claim 1 , further including a current mirror to generate a voltage that follows the voltage on the first capacitor. 3. The resistive processing unit as recited in claim 1 , further including a voltage follower to generate a voltage that follows the voltage on the first capacitor. 4. The resistive processing unit as recited in claim 1 , wherein the first transfer gate and the second transfer gate are actuated according to a coincidence of a plurality of stochastic pulses. 5. The resistive processing unit as recited in claim 1 , wherein the update circuit further includes:a first pre-charge gate in communication between a first pre-charge signal and a first end of the second capacitor; and a second pre-charge gate in communication between a second pre-charge signal and a second end of the second capacitor, wherein the second capacitor is charged according to a difference between the first pre-charge signal and the second pre-charge signal when the first pre-charge gate and the second pre-charge gate are closed. 6. The resistive processing unit as recited in claim 5 , wherein the first pre-charge gate and the second pre-charge gate are actuated according to a coincidence of a plurality of stochastic pulses. 7. The resistive processing unit as recited in claim 5 , wherein the first pre-charge signal and the second pre-charge signal are selected according to the voltage change added to the voltage difference to charge the second capacitor such that a transfer of charge between the first capacitor and the second capacitor is about equal to the amount of charge. 8. The resistive processing unit as recited in claim 1 , further including a transistor between the second capacitor and a voltage source, the transistor having a gate actuated by the voltage difference such that the voltage difference is constant. 9. A capacitor based resistive processing unit with symmetrical weight updating, the resistive processing unit comprising:a cross-bar array including a word line and a bit line; a first capacitor that stores a charge corresponding to a weight value; a readout circuit configured to read the charge stored in the first capacitor to apply a weight to an input value corresponding to an input signal provided by the word line, the input value being weighted using the weight value to produce an output carried by the bit line; an update circuit configured to update the weight value stored in the first capacitor, including:a second capacitor in communication with the first capacitor to transfer an amount of charge to the first capacitor according to an error of the output by changing a voltage difference across the first capacitor by a voltage change corresponding to the amount of charge, the voltage difference corresponding to the charge stored in the first capacitor; a first transfer gate in communication between the first capacitor and a first end of the second capacitor; and a second transfer gate in communication between a second end of the second capacitor and a current regulation device, wherein charge is transferred between the first capacitor and the second capacitor when the first transfer gate and the second transfer gate are closed. 10. The resistive processing unit as recited in claim 9 , thither including a current mirror to generate a voltage that follows the voltage on the first capacitor. 11. The resistive processing unit as recited in claim 9 , further including a voltage follower to generate a voltage that follows the voltage on the first capacitor. 12. The resistive processing unit as recited in claim 9 , wherein the first transfer gate and the second transfer gate are actuated according to a coincidence of a plurality of stochastic pulses. 13. The resistive processing unit as recited in claim 9 , wherein the update circuit further includes:a first pre-charge gate in communication between a first pre-charge signal and a first end of the second capacitor; and a second pre-charge gate in communication between a second pre-charge signal and a second end of the second capacitor, wherein the second capacitor is charged according to a difference between the first pre-charge signal and the second pre-charge signal when the first pre-charge gate and the second pre-charge gate are closed. 14. The resistive processing unit as recited in claim 13 , wherein the first pre-charge gate and the second pre-charge gate are actuated according to a coincidence of a plurality of stochastic pulses. 15. The resistive processing unit as recited in claim 13 , wherein the first pre-charge signal and the second pre-charge signal are selected according to the voltage change added to the voltage difference to charge the second capacitor such that a transfer of charge between the first capacitor and the second capacitor is about equal to the amount of charge. 16. The resistive processing unit as recited in claim 9 , further including a transistor between the second capacitor and a voltage source, the transistor having a gate actuated by the voltage difference such that the voltage difference is constant. 17. A method for symmetrical weight updating of capacitor based resistive processing unit, the method comprising:storing a charge corresponding to a weight value with a first capacitor; reading the charge stored in the first capacitor with a readout circuit to apply a weight to an input value corresponding to an input signal using the weight value to produce an output; and updating the weight value stored in the first capacitor with an update circuit, including transferring an amount of charge from a second capacitor to the first capacitor according to an error of the output by changing a voltage difference across the first capacitor by a voltage change corresponding to the amount of charge, the voltage difference corresponding to the charge stored in the first capacitor, using a first transfer gate, in communication between the first capacitor and a first end of second capacitor, and a second transfer gate, in communication between a second end of the second capacitor and a current regulation device. 18. The method as recited in claim 17 , further including pre-charging the second capacitor according to a difference between a first pre-charge signal and a second pre-charge signal selected according to the voltage change added to the voltage difference. 19. The method as recited in claim 17 , wherein transferring the amount of charge from the second capacitor the first capacitor includes completing a circuit that includes the first capacitor and the second capacitor such that the charge in the second capacitor causes a change in the voltage difference that is about equal to the voltage change such that charge in the first capacitor is adjusted according to the voltage change.
意見反饋
使用該功能遇到問題
該功能需要專業(yè)版企業(yè)版VIP權(quán)限，您可以：
您也可以聯(lián)系官方QQ: 2157717237 電話: 13264338900