Research, Collaboration, Impact! Understanding how our brains generate high-level concepts

Our brains work around the clock to keep us well. In our daily lives, we constantly negotiate high-level concepts that sit in hierarchical relationships with other abstractions. Professor Nancy Lynch and Dr Frederik Mallmann-Trenn use their expertise in distributed algorithms and neural networks to better understand how our brains navigate complex social worlds. 

Background

Controlling memory, thoughts and emotions as much as motor functions and sensory perception, for centuries the human brain has been the object of fascination. For Aristotle, the brain had the paramount task to cool down emotions if and when our broken hearts became ‘too hot-blooded’ and temperamental. And for modern philosophy, the question if mental states could ever be anything other than brain activity has been debated ever since Descartes famously proclaimed ‘I think, therefore I am’.

One of the great many things that our brain does is to help us negotiate our biological and social worlds. To do this, it needs to manage an incredible number of concepts that meet us everywhere. Concepts, types of abstract principles or ideas, help us get by: when we step into the road, we understand that the four-wheel motor-powered machine that’s quickly moving towards us is an instantiation of the concept of a ‘car’ and we better watch out before we cross. But how do our brains learn such life-saving concepts?

Brains as distributed algorithms

In making headway on this fundamental question, Senior Lecturer Dr Frederik Mallmann-Trenn, an expert in stochastic processes and biological distributed computing, and his MIT collaborator Professor Nancy Lynch, have set out to study the brain from a distributed algorithms perspective. ‘I believe that biological systems such as the brain can be modelled formally as distributed algorithms and analysed using methods from the distributed algorithms research area’, Nancy and Frederik explain the perspective of their research. Distributed algorithms are ‘designed to run on multiple processors, without tight centralized control’ so that they can be ‘used in many practical systems’.

The brain contains nearly 100 billion neurons and consumes 20 percent of our body energy—a lot is going on to enable us to live our lives. Modelling neural activity as distributed biological systems helps us understand better how our brain organises data, Nancy says. ‘For example, colonies of insects cooperate to solve problems of nest location, finding food, and performing various tasks’, she adds. ‘I have realised that many types of biological systems were essentially distributed algorithms, consisting of separate agents interacting and cooperating to solve problems’. Thinking of brain activity in these terms has allowed Nancy and Frederik to make progress in one particularly important domain: how does the brain structure, group and nest concepts into hierarchies?

‘Consider Cubist paintings’, Frederik says. ‘Oftentimes artworks are noticeably missing many details: no mouth, only one eye may be visible, we can’t see any legs, but still, unmistakably, we perceive the figure as a human. How does the brain achieve this feat?’, he asks. In their research, Nancy and Frederik have modelled the formation of concept hierarchies in the brain as neural networks. To this end, they have developed an algorithm in which the system learns both high-level hierarchical concepts as well as the sub-concepts it entails.

Fig 1: hierarchical concept formation in the model that studies the human brain as a neural network. (c) Frederik Mallmann-Trenn.

In their collaborative effort, Nancy and Frederik were able to demonstrate formally that learning concepts can be modelled in this way. The key point of the model is to show how a target concept at the top level gets embedded in the brain in such a way that lower-level ideas are being learned in the process also: in the above example, learning to identify a human also means having concepts of eyes, mouth, arms and legs.

Nancy and Frederik have published a paper that derives their model in detail. They view their work as paving the way for future contributions based on their theoretical findings. ‘Our paper gives the theoretical background and an explanation for how the learning of complex concepts can happen in the brain, what mechanisms are at play and how they interact’, Frederik says. ‘It may also show some of the limitations and impossibilities when it comes to learning’, he adds, in particular when environmental noise levels become overwhelming.

The ultimate aim of the work is to carry over insights about the modelling of concept-formation in the brain into the world of AI. ‘With a better understanding of the brain’, Frederik argues, ‘we can then design better neural networks’. A better design should then translate into better interpretability and explainability of these systems. ‘While our work is tentative, of course, in light of recent developments, such as GPT-4, where everything inside the network remains almost black-box and is beyond comprehension, we try to open this black box and to understand a little what’s happening under the hood’, Frederik points out.

Looking ahead

With Large Language Models such as Microsoft’s ChatGPT currently generating a steady stream of headline news, there are many applications of Frederik’s line of work. ‘By studying and understanding the brain, we hope to learn how we can improve artificial neural networks more generally’, he says with a view to Large Language Models such as GPT-4 beyond the black-box problem. Following the initial paper, Nancy and Frederik have doubled down on their efforts to generalise their original model. In a second publication, they investigate learning when concepts overlap, as they often do. Skilful chefs, for instance, may be able to create a range of tasty dishes from the same number of ‘input’ ingredients. The question then becomes, how are different high-level concepts generated from the same pool of lower-level ideas?

Given the paramount importance of the brain, modelling it from a computer science perspective promises vast interdisciplinary returns. ‘Experimental neuroscientists might try to verify if the brain stores information in a hierarchical way. They might even be able to identify neurons, or groups of neurons, that represent concepts at various levels’, says Frederik as he points to the future. Given how much there is still to learn about the brain, Frederik looks forward to future collaborations that bridge data science, biology and neuroscience.

Written by Juljan Krause