Rebecca Fiebrink: 00:04
“If you use that kind of language with data and say ’I’m going to twist it, break it, melt it, sit on it, play with it, tear it apart”. You know, you find yourself in in murky territory. Is data still data or has now become something else?
Julie Gould: 00:20
Hello and welcome to Working Scientist, a Nature Careers podcast. I’m Julie Gould.
This episode in the art and science series is dedicated to data. How scientists can be creative in data analysis and presentation, but also how artists use data to communicate challenging scientific ideas and their associated emotions.
In keeping with our art and science theme, each episode in this podcast series concludes with a follow-up sponsored slot from the International Science Council (ISC). The ISC’s Centre for Science Futures is exploring the creative process and societal impact of science fiction by talking to some of the genre’s leading authors.
What could possibly be artistic about data? Data by itself can be impenetrable. Lists of numbers or reams of text don’t really give us any information. But it does form the backbone of the scientific method, thinking and discovery. Without it we just have opinions, ideas, theories about what is happening.
The data gives us evidence about the world around us. And many people believe that this is objective information. Fact. Truth.
Rebecca Fiebrink: 01:29
If we think about data, you know, what is data really?
I think we can think about data broadly as just a representation of some facet of the world that we’ve captured in a format that a computer can work with, right?
So then data analysis, data in general, gives us an entry point for making art or music or creative work that connects to the world in some way.
Julie Gould: 01:51
This is Rebecca Fiebrink, a professor at the Creative Computing Institute at the University of the Arts, London. Rebecca is a classically trained musician with a computer science PhD.
And so she comes to her work with an interdisciplinary background. To her and many others, data can be a really interesting backbone for art as well as science.
But to do this, the data needs to be curated, cleaned up and analyzed so that we can see what stories it is trying to tell us. And these acts of data curation, cleaning and analysis are artistic and creative in themselves.
Rebecca Fiebrink: 02:23
Any kind of data analysis itself is creative, right? Data analysis is all about asking questions. And it’s about asking the questions that you didn’t even know you had necessarily, when you started. That’s a, that’s a very creative process.
And it’s it’s very similar, I think, to the questions that an analogue artist might ask if they, you know, start with a wood that they want to sculpt somehow, or a digital artist might start with, if they say, “Alright, I’ve got a blank computer screen, what pixels should I put there? What code should I write there?”
These these, we call them wicked design problems, of really trying to iteratively explore and make sense of something where it’s only at the end of this process that you know, really the questions you are trying to answer.
Julie Gould: 03:13
So how can art and creativity help us find these questions so that we can see the answers?
Duncan Ross, the chief data officer for Times Higher Education, a higher education publication that uses data visualization extensively in university rankings reports, says that data visualization can be helpful because it brings into play different parts of what it means to be a human being.
Duncan Ross: 03:36
We all know that the way we react to things visually is very different than the way we react to things intellectually.
So that ability to see patterns and interpret things using images can give powerful insights you can’t get, or you can’t easily get, by simply looking at numbers.
Julie Gould: 03:56
But it’s important to use the right visualization for the type of data that you’re communicating. And also the story you’re trying to tell.
You want to have that interactive ability for people to answer any question they might like to have. But you don’t know what those questions are going to be upfront, which makes it difficult to build a visualization which will work well for everyone.
Duncan Ross: 04:16
You want to have an interactive ability for people to answer any question they might like to have. And you don’t know what those questions are necessarily going to be upfront, which makes it quite difficult to build a visualization which is going to work well for everyone.
So as a result, you tend to stick to a number of relatively straightforward and well-understood visualizations. Because that’s another challenge you have. You don’t necessarily know how familiar the people who are using the tool are going to be with a particular Stan type of visualization.
So we use, we do use some radar charts, we use Box and Whisker plots a lot, which are a way of viewing a distribution. We do use line charts. We do use bar charts as well.
But the key thing is they have to be appropriate for the data we’re trying to visualize, and also the information we’re trying to get across.
Julie Gould: 05:14
Duncan finds that sometimes using simple visualization is best for helping people understand complex data.
Duncan Ross: 05:20
So, much of the way we display university rankings is very boring. We put a table up, because at the end of the day, although a table you may think, “Well, that’s not really visualization.” It is a mechanism that many of us are familiar with.
For everyone who follows sports teams. Every day, we go and look and desperately try and understand why our favourite team isn’t as high up the table as we believe they should be.
So the table itself is something we shouldn’t necessarily under-rate. But where the visualization from our perspective really starts to get interesting is when we dig in below those headline numbers.
So as well as producing the tables for the World University Rankings, and the Impact Rankings for external use, we also share with universities some of that underlying data, so that they can start to understand how they compare with their peers, hopefully in order that they can do work that will improve not their position in the rankings, that’s not so important, but they can actually understand how to improve as institutions and perform more strongly on the world stage. And then we absolutely want to use visualizations.
We need to be able to show universities where they sit compared to similar universities. We need to help them to understand the shape. And that’s a very visual word to use, but the shape of their institution compared to other institutions. So that’s really where we start to use that visualization more, more deeply.
Julie Gould: 06:51
James Bayliss, a data visualization expert at Springer Nature, is working on their new tool, the Nature Navigator, which creates visualizations on a digital platform to allow customers to experience data at different levels.
Now, some of his work is simple. Charts and graphs and the like. But some is more indepth, letting customers adjust certain parameters to understand the story from their own perspectives.
But for whatever visualizations he’s trying to build, James always starts with a blank canvas.
James Bayliss: 07:21
My go-to tool is a pen and paper or coloured pencils. I think it starts slow. And don’t get too complicated too fast. Hence the sketchpad, because you have to start very low-res, low fidelity, first, then medium and high fidelity, polished design and then go to coding.
You don’t want to start with coding. I’ve been in projects that I’ve started off with coding and got all tangled up in masses and masses of code.
Julie Gould: 07:48
Akshat Rathi, a senior climate reporter at Bloomberg News uses data to illustrate his journalistic stories. He says that sometimes the most simple visualizations have the most impact in his stories. And in 2015, whilst he was working for the business publication, Quartrz, Akshat wrote an article with a colleague on the devastating earthquake in Nepal.
Akshat Rathi: 08:09
The thing that struck people in the newsroom at the time when I was describing what an earthquake means, when it’s at a magnitude of 7.8 was surprising to some.
Because when they learned that the numbers of how earthquakes are described, is based on a logarithmic scale, not a linear scale, people couldn’t quite get their head around what it means if an earthquake is 6 on the magnitude, versus 8 on the magnitude.
Julie Gould: 08:38
Ashkat and his colleagues started by assigning a magnitude 2 earthquake to one small pixel.
Akshat Rathi: 08:45
A square that you can see, but it’s really small. And then you look at 4, and you look at 6, and you look at 8. And every time that box becomes much, much bigger than a simple 2, to 4, to 6 to 8, would linearly. And then you compare it to a 9. And the visual essentially becomes a scrolling box. You keep scrolling, scrolling, scrolling, scrolling, because it is just so much bigger than an earthquake, that was only one magnitude smaller.
And to me, that was a powerful way of showing what logarithmic scales can do, and what devastation a single increase in magnitude of an earthquake can bring.
Julie Gould: 09:30
Because we’ve explored a little bit about how to choose what visual to use to describe or depict the data, I asked Akshat how he and his colleague decided on the simple box visual.
He said it was partly because they were time-bound, because the news article needed to be out in a timely fashion, and partly because they were limited with what the website could do. But also because it served a purpose
Akshat Rathi: 09:54
Because we knew that if you convert a logarithmic into an absolute, you suddenly start to see numbers with many, many zeros. We thought, rather than showing the number of zeros, which is one way of showing that increase in the magnitude, wouldn’t it be better if we showed it in an actual physical box?
Because when an earthquake happens, the thing that most people see is an actual physical devastation.
You see buildings fallen, you see people and rubble. And those images stay with you because they are distressing images. And you see people suffering. And we wanted to try and replicate that level of impact that would come from actual visuals rather than simply putting them in numbers.
Julie Gould: 10:41
Sometimes the visual or oral representations of data aren’t meant to obviously explain what the story is.
Their role, as many artists and scientists have told me for this series, is to invite people into the work to think, to ask questions, and to feel connected to the data.
Nathalie Miebach, an artist who uses basket weaving as a medium to represent the data she collects, did this exact thing when she started working with art and science.
Nathalie Miebach: 11:07
My beginning point where data and sculpture kind of intersected was astrophysics. And when you’re studying astronomy, or astrophysics, you’re dealing with very abstract numbers, you know, distances that are just long distances, numbers that are incomprehensible.
And for me, making sculptures out of the things I was learning in astronomy was a way of making that science more tactile, more understandable.
As a tactile learner, I had to sort of figure out how to somehow make this understandable.
Julie Gould: 11:39
Her main interest and inspiration is weather data. There’s an abundance of it. And it’s something that impacts everyone. And it’s also part of her bigger goal to understand climate change and its impact on people.
Now she feels that data collected by someone else and presented to you as spreadsheets or numbers and graphs, has already had a cleanup and all the anomalies have been removed.
And she felt like in order to really understand what was going on, she needed to do this herself, to learn the process of data collecting, but also to fully understand what the data was representing.
So since 2006, when she was doing a residency in Provincetown, a small coastal town in Massachusetts, Nathalie has been collecting weather data using homemade data collecting tools,
Nathalie Miebach 12:24
A compass, it was a thermometer from the kitchen aisle from the local hardware store. It was a rain gauge from the garden aisle. I would build up my own data collecting devices that I could use to measure wave height and cloud cover. So it’s very, very basic stuff.
Now I take that stuff out to the beach every day for 18 months. And I would just collect things, I would collect numbers in the sense of, I would take the temperature readings of the water, I would take the temperature readings of the air. I would look at the wind direction pressure, but I would also spend a good deal of time just writing down what I was seeing.
So what kind of stuff was washing up on shore, what kind of plant material, what kind of birds were out in the waters. Were they migratory birds were the birds that stay here all season?
Julie Gould: 13:10
So all these observations that Nathalie made form the basis of her sculpture using basketry. Basketry offers an interesting parallel to the traditional scientific forms of visualization. Baskets are made with horizontal strands called weavers, which are like an x axis on a graph, and vertical strands called spokes, which are like the y axis.
Nathalie Miebach: 13:30
So if you have 48 spokes, you have a 24 hour clock, basically, and then you just basically start weaving.
You know, for example, when does the moon rise? Okay, it starts rising at 8pm. So I start weaving and then I start weaving when it sets, let’s say at, you know, four in the morning.
And then I do the same thing with the sun. And so it’s sort of sculpture by number. So basket weaving became this really easy way of taking basic calendars, or basic data sets, and translating them into a kind of timeframe that was dimensional.
Julie Gould: 14:01
Her sculptures don’t show any resemblance to the traditional xy graphs that most of us are familiar with. And to be honest, sometimes they don’t look much like a traditional basket either.
Which begs the question: Is it art? Is it data? Is it both? Or is it neither?
Nathalie Miebach: 14:18
It’s a contradiction, and I love it. So every day, I walk into my studio, and I walk into this contradiction, because on one hand, as a, as a sculptor, who is working with basketweaving techniques, I know that in order to really understand the medium and the technique I have to fail with it 100,000 times.
I have to sit with it, I have to break it apart, melt it, twist it, you know, contort it. I have to basically just fail in so many ways for me to really understand what is the potential of this technique? What is the potential of the medium that I’m using, or the material that I’m using?
If you use that kind of language with data and say “I’m going to try was to break it, melt it, sit on it, play with it, tear it apart.” You know, you find yourself in in murky territory. Is data still data or has it now become something else?
Julie Gould: 15:09
This conflicting emotion is what drives Nathalie to continue her work. She acknowledges that there is this side of science and data where there is this shroud of truth. An untouchability of data. It’s something that cannot be changed or translated, because it’s data.
Nathalie Miebach: 15:24
But any kind of visualization of data, any kind of translation of data, is a form of distortion of it. So you’ll never ever get the pure data.
Working with data is a bit like creating maps out of, out of something. And a map is never an, a map is always a distortion of something. It can never contain everything. And so is a data visualization. It can never, ever contain everything that you’re looking at. The complexity of the system.
So I’m kind of curious about how much pressure and how much expectation we place on data. And also how much we associate the truthfulness of data with the visual languages that we see them in.
Julie Gould: 16:10
Take, for example, a piece Nathalie made that was based on a tidal chart. It’s a six foot woven basket sculpture that translates one year’s worth of moon and sun data from Boston, Massachusetts. It’s got info on it about when the tides were happening, how high they were, the moon phases and the sun.
Nathalie Miebach: 16:29
It’s basically in a sense, a 3D calendar that I’m weaving. One year’s worth of sun and moon data.
And it’s a very twisted, distorted form. And it’s the numbers that are twisting and distorting the form. And the reason it’s, I say, it’s the numbers that are doing that. It’s because I’m using a material that I cannot fully control.
If I exert too much pressure on the material it breaks. It’s a natural material called reed. And so it’s really these two datasets, the sun and the moon over time that are distorting the grid of the data. So the form itself is made of the data.
Julie Gould: 17:04
This piece was exhibited in very different locations. And each one got a very different response.
Nathalie Miebach: 17:09
This piece was in science museum. So people read it as a tidal chart. And then after that, it went to a craft museum.
And it started this whole conversation about what is the utilitarian purpose of basketweaving? And how, what is its history?
And how can this sculpture, this, this object now have. What is its function really? Because it’s not really functioning as a traditional basket, and it’s in sort of in the way that it would fit into the history of basketry. And then, of course, you put it into an art museum, and it becomes this aesthetic object.
Julie Gould: 17:44
What this demonstrated to Nathalie is that translating data into artistic mediums brings up all sorts of biases and expectations when we think of data and enter it into different spaces.
But it also makes her wonder how much we trust the data. So her question is,
Nathalie Miebach 17:59
So why would it be that the title chart that I use as the beginning for my piece is more trustworthy, or more scientific than my sculpture? Both are in a sense, translations both are, in a sense, confined by limits of their own medium.
So a graph has limits just as much as a sculpture does. So I’m interested in that tension. I also really don’t know if what happens to data when you translate it into an artistic medium.
I’m not sure if it is still data. Maybe it’s become something else. But I love that it’s making me ask that question. And it’s been a source of discomfort ever since I started. And I love it.
Julie Gould: 18:52
What I’m trying to say with this series of episodes is that the scientific method is creative. It is artistic, and that the subjects of art and science, or art and data, are deeply intertwined.
Science requires creative thinking from the point of asking questions, through data collection and data analysis, and all the way to presentation.
It is a way for us to have a greater awareness and understanding of the complex world around us.
So in the next episode of this series, I’m speaking with three people about what the future of art and science will be.
Will artists and scientists be pigeonholed into their respective disciplines?
Or will we start seeing more and more interdisciplinary careers and departments at university institutions that allow people to follow their passions in both directions?
But before you go, the music for this episode was kindly provided by Matthew Jackford. The piece Shifting Winds was inspired by a musical score that Nathalie Miebach created from one of her datasets she collected as part of her work.
And also don’t go just yet, as we’ve got the sponsored slot with the International Science Council about the creative process and societal impact of science fiction.
Paul Shrivastava 20:04:
Hi, I’m Paul Shrivastava from the Pennsylvania State University. And, in this podcast series I’m speaking to some of today’s leading science fiction writers. I want to hear their views on the future of science and how it must transform to meet the challenges we face in the years ahead.
Qiufan Chen 20:24:
AI in the future, maybe it could be used to help us to reflect ourself as a mirror, to make us become a better human being.
Paul Shrivastava 20:33:
Today, I’m talking to Qiufan Stanley Chen, an award-winning Chinese writer. I read his novel, The Waste Tide many years ago, and was impressed by his portrayal of the predicaments of electronic waste. His most recent co-authored book AI 2041, 10 Visions of Our Future, vividly combines imaginative stories with scientific forecasts. We spoke a lot about artificial intelligence and how we can harness the power of this incredible technology, while avoiding some of the dangers it poses. Thank you very much for joining us, Stan. Welcome. It’s amazing the range of scientific topics that you have mastery over is really notable. How did you come to be interested in these scientific topics?
Qiufan Chen 21:28:
So, as a sci-fi fan, I have to admit that I started from all of those Star Wars, Star Trek, Jurassic Park, classic sci-fi movies and books, animations back in the day. Each time it gave me a lot of new inspiration and ideas. So, I was always totally fascinated by all this signs, imagination of the future and outer space and even species millions of years ago. So, how we brought them back to life.
Paul Shrivastava 22:03:
So, science has been going on for a very long time. What is your general view on science as a human endeavor?
Qiufan Chen 22:13:
To me, it is definitely a huge achievement. And, of course, it make us living a better condition as a human being. And, when we look back to history, I have to admit that there’s a lot of challenges, because it feels to me like the agency is not absolutely in the hands of human beings. Sometimes I feel that maybe science and technology, just like some kind of species, like some kind of biological beings, it has its own purpose. It has its own birth life cycle. It wants to be and evolve together with human beings. So, we are like the host, they’re like the virus. We can see it in that way or the other way around. So, I always feel that there’s really deeply entanglement between science and human beings. So, sometimes I feel that we’ve been changed a lot by all this development of science and technology, but we never know what is the direction ahead of us.
Paul Shrivastava 23:24:
Well, let’s make it more concrete and focus on what is top of mind right now, which is artificial intelligence. How can we ensure that the development of AI, we bring in social justice and ethical and moral considerations into bear?
Qiufan Chen 23:41:
The problem is we didn’t fully invest to build up this kind of regulation and framework of ethically prevent something negative from happening. I think we need more diversity on AI, and especially on large language model, because we’re talking about specifically alignment. So, even among human beings in different countries, cultures, language, we didn’t have this shared alignment as a single standard. So, how can we teach the machine, the AI, to be aligned with human value system or the standards as one integral one? So, I think this is something very preliminary. But, I think the key input should be not only from the tech companies, from the engineers, from all these people doing the thing in the industry, but also from the interdisciplinary world, such as anthropologies and psychologies, sociology, for example. We need more diverse perspective from humanities, because AI is supposed to be built for the people, to serve the people. But, the human factor right now, I can feel that is quite missing in the loop.
Paul Shrivastava 25:12:
At some point in the future do you think that AI will understand more than what humans can understand?
Qiufan Chen 25:22:
So, what I’ve been thinking about is some model, like large model beyond human. For example, the data’s from animal, plants, fungi, even from micro and the whole environment. So, we’re talking about the whole Earth model. We need to deploy this kind of sensor layers around the world. So, maybe we can using smart dust, which was mentioned in Lem’s novel The Invincible. So, you’re talking about all this swarm of small dust, basically it’s a collective intelligence. And, human can learn so much from this kind of large model, because it help us to perceive something beyond our sensory system and beyond human. Then we can be less human-centric, and we can be more compassion about other species. And, maybe that would be the solution to fight against the climate change, because we can feel how the other species feel and all this pain, all this suffering, all this sacrificing could be something tangible and real.
Paul Shrivastava 26:45:
Wonderful. Imagining artificial intelligence in the model of humans is actually an inferior way of thinking about artificial … The more superior way, what you call the whole-world model is the way to develop.
Qiufan Chen 27:03:
Yeah. So, this reminds me of Buddhism, because in Buddhism, like all the sentient species are as equal as possible, and there’s not such thing as human beings supposed to be premier than others. So, I’m always thinking about we need to find a way to embed all this philosophy and values of Buddhism and Taoism into the machine, into the model.
Paul Shrivastava 27:37:
So, I’m wondering, you understand the technical elements of AI. Can AI be trained in Buddhism, in Taoism? Because all the books and values are already codified. Is it possible to find AI that trains on them and creates a synthetic world religion, if you will?
Qiufan Chen 27:59:
It definitely could, and it could do a better job than any of the priests, any of the monk, any of the gurus in the world, because it’s so knowledgeable. But, as a practitioner of Taoism, there’s something beyond the synthetic understanding of all this, call it religious or spiritual experience, is something in body. So, you have to do all this physical homework to practice, to hike, to meditate. So, I think this is something still AI lack of. It didn’t have a body, it didn’t have the complex sensory system, it didn’t have self-awareness, for example. And, I think all of those part is what makes a human, human. AI in the future, maybe it could be used to help us to reflect ourself as a mirror, to make us become a better human being.
Paul Shrivastava 29:05:
Thank you for listening to this podcast from the International Science Council’s Center for Science Futures, done in partnership with the Arthur C. Clarke Center for Human Imagination at the University of California, San Diego. Visit future.council.science for the extended versions of these conversations, which will be released in January 2024. They delve deeper into science, its organization and where it could take us in the future.Join us next week for this series’ last episode when I’ll be speaking with Cory Doctorow on more themes related to the digital world.