A process of learning a distributed representation by the distributed representation learning section 11 is described with reference to FIGS. 3A to 3C. FIGS. 3A to 3C are diagrams illustrating an example of the process of learning a distributed representation according to the embodiment. In FIGS. 3A to 3C, a technique for producing a distributed representation of a word is described as the Skip-gram model of Word2Vec. The example assumes that the reference language is English and that the target language is Japanese. The example also assumes that the reference language learning corpus 21 indicates that “I drink apple juice ? ? ? ”.

The distributed representation learning section 11 builds a neural network composed of an input layer, a hidden layer, and an output layer for the Skip-gram model.

First, the input layer, the hidden layer, and the output layer are described below. A V-th dimensional input vector x corresponding to a given word is input to the input layer. The input vector x is a one-hot vector. V indicates the number of words included in the reference language learning corpus 21. The one-hot vector indicates that an element corresponding to the given word is 1 and that other elements are 0.

In the hidden layer, an N-th dimensional word vector h indicating a distributed representation of the given word “apple” is finally generated. W a weight between the input layer and the hidden layer and is expressed by a matrix of V×N. As initial states of elements of W_V×N is random values are given, for example.