Making Sense
The missing link between knowledge and understanding
This essay is about how we get from sensing, which all creatures can do, to understanding, in the way only humans can do.
Introduction
As discussed in my previous essay Human Resources, people are meaning-making machines. We go about our daily lives subconsciously categorizing whatever we pay attention to by its function or significance to us, not its surface characteristics.
The hot flavoured liquid we consume in the morning; that’s having a cup of coffee. Climbing inside a metal and glass container that moves; that’s driving to work. And tapping a few bits of plastic as we look at some flickering lights; that’s sending an email.
What we call the various things we see around us generally has very little to do with their appearance. Instead, the labels we give to objects represent their significance to us, i.e. their meaning. When we recognize a friend, we don’t think “Here’s a familiar-looking homo sapiens!”; we simply see “Andy” and all that he means to us.
Similarly, when we close our eyes and pay attention to the things we can hear, we want to know what they are and not just describe their acoustic characteristics. We go beyond the sound to assign it a meaning. “That’s a bird,” we think to ourselves, not “That’s a patterned high-frequency squeak”.
In this way, the dozens of meanings we experience every waking moment all add up to a dynamic two-way interface with the world. While our senses may be constantly reacting to a stream of lights, chemicals and forces, our experience is of things like coffee, cars and Andy. This essay is about how this network of meanings – our reality interface - is put together.
Gathering data - the first step towards understanding
The animal kingdom has evolved something like 20 to 30 senses for gathering information about the environment (not including bodily sensors like proprioception or vestibular). Out of this list, human beings have only the trusty five we learn about in primary school – vision, hearing, touch, taste and smell. Not a great start if we want to know what’s going on in the world.
Ok, so how good are these inputs? Again, the news is not great. While the best human noses can detect some substances at about 1 part per trillion, dogs / sharks / bears can sniff out much lower concentrations. When it comes to our ears, we can hear about 35% of the audio frequencies the animal world can (or about 10% of the possible range of sounds on Earth). And our eyes detect just 0.0035% of the electromagnetic spectrum, what we call ‘visible light’.
And then there are issues of resolution and sampling rates, which impose further limits on the amount of reality our senses can take in (for an interesting discussion about our input capacity and how we might be more data hungry than we realise, see Do humans really learn from “little” data?).
But wait – there’s less!
It turns out the entire pool of reality from which we’re drawing this limited sample may itself be only about 15% of what’s actually ‘out there’, the remaining 85% being so-called dark matter.
Taking all these factors into account, the average human manages to capture just a trickle of the tidal wave of reality crashing around them every second. The little that does get through, either directly via our senses or through some instrument such as a microscope, is the only empirical data we have from which to build our entire understanding of reality.
Given this sparse data set, it’s a wonder we’ve managed to understand anything at all, let alone build tools and technologies and civilizations that somehow work in the real world. How is this possible?
One way to think of this problem is as if we are living inside a huge join-the-dots puzzle, of the kind found in children’s books, where we have to join the dots to reveal the picture. Every glimpse of reality we get is another dot in the puzzle - every sight, sound, touch and smell another clue - scattered like gold dust through the black vastness of our ignorance.
Somehow, our brains weave these glints of information into a vivid tapestry of continuous experience. If we couldn’t do this, if we couldn’t fill in the spaces between the dots, then our experience would be of a relentless barrage of incoherent sensory hits. It is in the stitching together of the tapestry, the complex way in which the data points are connected, that understanding emerges.
Connection is therefore fundamental to making sense of things. There are no patterns in isolated bits of data, nothing to be learned from atomised information beyond simply that it exists. It is only by making connections that a picture can be created and understanding becomes possible. So, given that discrete bits of sensory data are the only input we have, how are they connected together?
Joining the data - the second step towards understanding
Just as with a picture puzzle in which the image is implied or incomplete, so the nervous system must fill in the gaps in our sensory input in order to create a coherent experience. This process of ‘filling in’ can be split into three functional levels:
Sensing - the initial smoothing of discrete physical data occurs within the sensory pathway, using mechanisms such as spatial and temporal integration to generate a continuous live feed of the outside world. Sensory smoothing is why pictures flashing at more than 25 frames per second appear to us as a continuous movie.
Perceiving – next, we need to extract discrete packets of information from the raw sensory feed, which requires organising the continuous input into distinct signals. This is what enables the average person to hear individual words in a stream of speech, or to visually distinguish an object from its surroundings.
Understanding – the final step is to ‘make sense’ of our perceptions, which requires plugging them into a network of inter-connections with other pieces of information. You cannot recognize words without knowing what language is, or a chair without knowing what sitting is.
These three levels are richly interconnected through backward and forward loops to help keep them mutually consistent. Together, they form a sort of reality interface that converts raw empirical data from the external world into meaningful internal experiences, by combining information in increasingly complex ways.
Using an analogy from chemistry, we can think of contrast and orientation as basic ‘atoms’ of visual information that combine to form ‘molecules’ such as size and shape. These in turn can be combined with other molecules of information, like colour and movement and so on, to form increasingly complex visual compounds that ultimately give rise to high-level perceptions.
When a child first sees a page of musical notation, it looks like a page of funny-looking squiggles. As she progresses in her learning, she starts to perceive discrete meaningful objects, like ‘stave’ and ‘crotchet’ and so on, until eventually she can recognise the whole song. However, the raw audio-visual input is the same at every step of the process, so what is actually changing?
At each stage of her learning, she is combining information into increasingly complex structures - from sensing to perceiving to understanding - that eventually form a stable network of information between the ink on the page, the notes on the instruments, and the sounds of the song. Objectively, an accomplished musician sees and hears the exact same squiggles and sounds she did as a beginner, but her reality interface has been trained to transform these inputs into something subjectively meaningful.
Emergence - the last step in understanding
As well as the specific kinds of information being combined, the increasing levels of complexity also depend on how that information is combined. Just as hydrogen and oxygen can create water, linking the right information in the right way can create something new, a network that is more than just a simple sum of its parts. This network-building process involves two kinds of connection: association and invention.
Associations are based on the mechanics of the natural world and tend to be fairly immediate and constrained, such as between blowing a trumpet and hearing its sound. These are Pavlovian-level connections that happen innately and can be acquired by people quite quickly.
Other connections however are not so obvious to us, such as between the moon and the tides, or the environment and evolution. Because the evidence of these phenomena appear separated in time and space, it is harder for us to intuitively connect them together though the connections are just as real.
We must therefore use a different type of connection for phenomena that extend outside our sensory windows. If our perceptions are dots of information about the world, then the gaps between them are an absence of information. Put another way, the dots are separated by ignorance.
As far as we know, the only thing that can travel through the vacuum of ignorance is imagination. Like the parable of the blind men trying to describe an elephant, imagination allows us to fill in the gaps in our perceptions and create a ‘big picture’ that links them together.
Unlike Pavlovian connections which rely on the proximity of sensory data (e.g. the sound of a bell and the smell of food), imagined connections can be much more abstract, and can even be made between bits of information that are themselves abstract. For example, the rules (or connections) between musical notation and playing an instrument are an invention, not a model of something universal in nature.
That said, it’s important to note that invented connections are not just fantasy (although they certainly can be). In fact, they are critical in science because they enable us to ‘perceive’ connections outside the limits of our senses. Can we actually see chemical reactions, or the water cycle, or star formation? These phenomena occur outside our sensory window and need imagination to hold them together in our mind. The holy grail in physics is to find a way to connect the view of reality we see through quantum theory with the one we see through general relativity.
So to summarise, we have bits of information – or dots, or atoms (I may have got carried away with the metaphors) – coming in via our senses that we must connect using association and imagination in order to literally ‘make sense’ out of them.
Shared understanding
As a rule of thumb, we tend to communicate empiric connections as explanations and more abstract ones as having meaning. Our description of the connection between a trumpet and its sound is an explanation, whereas we say that the symbol “♫” on a page means musical notes.
Thinking back to our primary school days, a noun is ‘something that exists’ in the world and represents a chunk of information. They are the building blocks of language that we glue together in the minds of our audience using various other words in order to build them a mental representation of what is in our own.
Through language, the right nouns connected in the right way convey understanding. Stories are how we make sense of the world:
“A story is a description, either true or imagined, of a connected series of events.”
In ancient times, our stories starred Gods and spirits. Astrology quite literally joins the shining dots in the sky, while a volcanic eruption in ancient Rome might be blamed on Vulcan’s anger. In more recent times, our heroes are the likes of Galileo and Newton, whose stories are filled with characters such as matter and energy and forces. We see the exact same sky and moon as the ancients, feel the same wind and sun as they did, but our stories about these things have shifted from the abstract to empiric, from meaning to explanation.
The purpose of all such stories, past and present, is to help us make sense of our experiences through models of how the world works. Some stories are primarily concerned with physical phenomena, for which the language of mathematics has turned out to be remarkably apt. Other stories are more abstract, such as what justice is or how money works. Most stories mix the empiric and the abstract, to the extent that the distinction can sometimes become blurred (see Nature Cannot Lie).
Looking again at the picture of the rabbit above, we see that most of it is actually the lines that connect the dots, not the dots themselves. We could also have drawn a different picture by connecting the dots in a different way, emphasizing once more that connections are usually more important than dots in determining what we see. This is why people can sometimes see or hear the exact same evidence and yet draw quite different conclusions.
If we think of empirical dots as anchors in reality, then we can see that stories with a high proportion of dots are literally more anchored in reality. For instance, a news report based on direct evidence is considered more real, and therefore more credible, than one with lots of gaps in evidence that have to be filled with narrative. Similarly, keeping a tight chain of evidence in criminal cases is necessary in order to minimise the space for alternative stories.
Conclusion
In her wonderful book The Invention of Nature, Andrea Wulf describes how Alexander von Humboldt (whom she convincingly portrays as one of the world’s most pioneering and yet forgotten naturalists) brought together different scientists and disciplines in an entirely new way, helping to make connections that inspired our understanding of ecosystems and of nature as a complex web of life.
Because when explanations and meanings link the right information in the right way we get understanding, i.e. a network of knowledge that is something more than the sum of its parts. Conversely, when we want to understand a network of knowledge we dissect it down into explanations and meanings.
The same principles apply to understanding ourselves. Imagination interconnects the autobiographical events of our lives into a picture we call our identity. The famous ‘first impression’ we get when meeting someone is based on a handful of empiric dots (e.g. their size, shape, expression, etc.) and lots of assumed connections between them. We can draw very different pictures of people - and ourselves - depending on which characteristics and life events we choose to focus on and how we connect them.
This ability to connect knowledge in useful ways is what separates us from the rest of the animal kingdom. While many animals are better than us at raw data collection, it is our imagination that enables us to discover much more of reality than just what we can see, hear or touch. Without imagination, we would be forever marooned within our physical senses, unable to understand the depths of ourselves or the world around us.
“I am enough of an artist to draw freely upon my imagination. Imagination is more important than knowledge. Knowledge is limited. Imagination encircles the world.”
Albert Einstein, 1929
This is a great telling of a reality that I think is self-evidently the case and is an inspiring view of how we humans make sense for ourselves and our communities of that little and dispersed sense data that we take in… And yet if it is so, then is it still not open to the critique of solipsism - that whilst more committed to a concept of there being an actual reality outside our own heads (unlike some would postulate, whilst then living somewhat otherwise in their day to day existence…), it does still seem to be there, as the imaginative narratives and the various data from which they’re spun are potentially so infinitely varied as to amount to the same thing?
Thanks Saj. Love your Einstein quote. I also like his quote "If you want your children to be intelligent, read them fairytales. If you want them to be more intelligent, read them more fairytales." We don’t talk enough about the importance of imagination and storytelling when grappling with complex and difficult problems, as he did. Thanks for reminding us!