Analysis component 441 uses prior data about a user obtained from database service 431 to build one or more models for the user. During this model-building process, the system can focus on characteristics of specific user behaviors to uniquely identify a user. For example, the system can examine accelerometers readings (or other sensor readings), which indicate how a user: walks, stands up, sits down, talks or types. The system can also focus on how a user manipulates her phone. One promising way to authenticate a user is to recognize the user based on accelerometer readings indicating characteristics of the user's gait while the user is walking. In fact, it is possible to recognize a specific user based on just the magnitude of the accelerometer data, and not the direction of the accelerations. The system can also consider combinations of factors from different sensors instead of merely considering a single factor. This includes considering cross-device factors, such as signal strength between a wearable device and a user's smartphone, or a combination of accelerometer readings from the wearable device and the smartphone.

The system can also attempt to detect the presence of another person, for example by looking for a Bluetooth signal from the other person's smartphone, and can condition the recognition process based on the presence or absence of the other person. This can be useful because the presence of another person may change the user's behavior.

Next, while processing the feature vectors, analysis component 441 can generate one or more possible user identities with an associated security score for each identity. Note that the system illustrated in FIG. 4 can also include a “challenge channel” (not shown) to deliver challenges to a device or a user as is discussed in more detail below.