Using behavioural models to upgrade User Interface design
Modern computational techniques can improve human-computer interaction (HCI) using combinatorial optimisation. Here, a function is specified and numeric values are assigned to designs proposed by an algorithm. These values indicate how close each design comes to meeting the function required. This approach has changed several industries, from logistics to telecommunications, but hasn’t yet been widely adopted for HCI. Inspired by insights from cognitive psychology about human behaviour, the EU-supported COMPUTED project adopted mathematical models and simulations for better User Interface (UI) design. “The idea of using combinatorial optimisation and models from psychology originates from the 1970s, but until now nobody really knew how to do it,” explains project coordinator Antti Oulasvirta, from Aalto University, Finland. As well as further optimising keyboards and menus, for example helping the French government design their new Azerty keyboard layout, COMPUTED diversified the ways in which UIs can be optimised. These now include web pages and mobile apps, as well as virtual reality spaces. COMPUTED’s advances were summarised in a recent IEEE Proceedings review.
Borrowing from psychology
UI designers typically use heuristics, or rules of thumb, such as ‘place elements symmetrically for optimum aesthetics’. An experienced designer will have learned a large number of such rules. While these have served to make computers more accessible, they have significant limitations. Sometimes the heuristic rules contradict one another. They usually also consider only one or two design decisions at a time. And their correlation with actual user experience and usability is quite weak. This means that designers trial many iterations of designs, making the process laborious and expensive. COMPUTED’s innovation was in its use of methodologies from a different discipline. “I exploited my background in cognitive psychology. By adopting psychological models of performance, perception, learning experience and decision-making, we expanded what could be done with UIs,” adds Oulasvirta. For example, the team worked with visual attention models from psychology, to predict how users typically scan a display when looking for something. From this, they built an optimisation algorithm that can trial different UI designs, testing the likely impact on user experience at the rate of millions per second. This approach has been applied to different design needs and combined to create a range of tools. These can be used when sketching and wireframing, without any specialist knowledge. An example of the system in operation is the design of graphical UI layouts, such as those used for websites. The solution produces several designs based on the objectives of proper alignment, overall rectangularity and preferential placement of elements. The system ensures that designers get real-time design feedback, with new and diverse customised designs.
Starting up the start-up
The code is already available for designers to try. The team plan to launch a start-up later this year to commercialise the design tools. Given the scope of UI design problems, with prohibitively large data sets even for algorithms, the team will exploit the power of deep learning. “It will take us a long time to get to the point where computers can design truly complex UIs, like those in airplanes, factories or even social media. Besides finding the right models and efficient algorithms, other challenges are related to language and semantics. Design is about communication,” says Oulasvirta.
Keywords
COMPUTED, User Interface, combinatorial optimisation, Human-Computer Interaction, algorithms, psychology, behaviour, heuristics, models, design
