As computing technology evolves, users perform more complex tasks with computers. Hence,
users expect from user interfaces to be more proactive than reactive. A proactive interface
should anticipate the user's intentions and take the right action without requiring a user
command. The crucial first step for such an interface is to infer the user's mental state, which
gives important cues about user intentions. This thesis consists of several case studies on
inferring mental states of computer users.
Biosensing technology provides a variety of hardware tools for measuring several aspects of
human physiology, which is correlated with emotions and mental processes. However, signals
gathered with biosensors are notoriously noisy. The mainstream approach to overcome this
noise is either to increase the signal precision by expensive and stationary sensors or to control
the experiment setups more heavily. Both of these solutions undermine the usability of the
developed methods in real-life user interfaces.
In this thesis, machine learning is used as an alternative strategy for handling the biosignal
noise in mental state inference. Computer users have been monitored under loosely controlled
experiment setups by cheap and inaccurate biosensors, and novel machine learning models
that infer mental states such as affective state, mental workload, relevance of a real-world
object, and auditory attention are built.
The methodological contributions of the thesis are mainly on multi-view learning and
multitask learning. Multi-view learning is used for integrating signals of multiple biosensors
and the stimuli. Multitask learning is used for inferring multiple mental states at once, and for
exploiting the inter-subject similarities for higher prediction accuracy. A novel multitask
learning algorithm that transfers knowledge across multi-view learning tasks is introduced.
Another novelty is a Bayesian factor analyzer with a time-dependent latent space that captures
the dynamic nature of biosignals better than methods that assume independent samples. The
overall outcome of the thesis is that it is feasible to predict mental states from unobtrusive
biosensors with reasonable accuracy using state-of-the-art machine learning models.
Keywords Multitask Learning, Multiple Kernel Learning, Probabilistic Modeling, Affective
Computing, Intelligent User Interfaces |