人工智能“不可避免”的神话

KEVIN MALONEY: On this episode of the Values & Interests podcast, I'm joined by Virginia Dignam. She's a professor of artificial intelligence at Sweden's Umeå University and is the author of the recent book The AI Paradox. This is one of my favorite conversations we've had recently on the podcast, and together, Virginia and I dig into what it means to be human in this "AI moment." How can we push back against narratives of civilizational war and inevitability? Coming out of Silicon Valley and other tech developers, while at the same time attempting to regain our agency around artificial intelligence? I hope you enjoy the conversation. Let's get right to it.

Virginia, welcome to the Values & Interestspodcast. It is so great to have you.

VIRGINIA DIGNUM: Thank you very much. It is my pleasure to be here.

KEVIN MALONEY: We have a lot to talk about in the spaces of ethics, artificial intelligence (AI), geopolitics, and technology, but, as we always do on the Values & Interests podcast, I would love to start by hearing from you about your personal background. We talk a lot about the relationship here between morality, power, and values and interests, so we would love to hear about your own values system, how it has emerged, changed, and grown over the years. Maybe we can start there and my listeners can get to know you a bit on the personal side.

VIRGINIA DIGNUM: Sure. Let’s see what I can tell in a short time.

I am originally Portuguese. I was born and raised in Portugal in a family of five children, of which I was the oldest. Being the oldest naturally comes with a sense of responsibility and contributing to the family. Both of my parents worked, so it was a natural thing. It is not at all like I felt forced or anything.

I also for a large part of my early and later youth was involved in the scouting or Girl Guides movement. Again, there was this idea of responsibility and contributing to the well-being of others and a better world in one way or another. Our value set came from that background.

I studied in Lisbon for my first degree in applied mathematics. During this time, I met my boyfriend and now-husband from the Netherlands. We moved to the Netherlands in 1987, and there I continued with a second master’s in AI, after which we spent some time in Africa, maybe also from this idea of contributing in a small way to make a world a better place. My husband was developing the first curriculum on computer science at the University of Swaziland. I was doing all kinds of freelancing but also teaching and developing software for the country during the time we were there.

To make a long story short, we moved back to the Netherlands. I moved much later into academia to do a Ph.D. and continued in academia, and after seven years roughly we moved to Sweden, where we currently both work as AI professors, me focusing more on the impact, governance, and ethics, and my husband more in the area of multi-agent systems.

KEVIN MALONEY: That was an amazing overview. We covered about 40 years of history there. I love the story about scouting and how that influenced you. I think about these small concentric circles around pluralism and engagement, where you get access to new points of humanity. For a lot of people I have talked to on this podcast, it was that pluralistic civic space, whether sports, school, or scouting, that was the spark that pushed them into the world. I have also heard a lot of people talk about the influence of parents, especially working in civic spaces, whether they were teachers or doctors. It is great to hear about that.

VIRGINIA DIGNUM: My parents definitely were influential for a large part in the way that I grew up. My mother was a teacher but maybe did not influence me so much as a teacher, but they were quite involved in the church and with supporting others, and that became a natural growing environment in which my siblings and I developed and engaged.

Scouting, as you say, is a field in which this pluralism worldview comes quite naturally. I was not only a scout or a Girl Guide myself, but I moved on to be a leader in the movement in Portugal, and that also contributed to the formation of many generations of Girl Guides and scouts younger than me.

KEVIN MALONEY: It is very interesting to hear about those environments where the structure in terms of advancement is built into the fact that you need to be pluralistic or a moral and political leader. So often we get this narrative now of, “I pulled myself up by my bootstraps” or “I am a self-made billionaire and did this by myself,” but then you look into their pasts and they had an amazing teacher, coach, or public school system that allowed them to do what they did. We can go off on that tangent in a little bit, but I always find that interesting.

VIRGINIA DIGNUM: We are all products of the places and situations in which we grew up, and trying to ignore it is useless.

KEVIN MALONEY: The “sum of many parts.”

VIRGINIA DIGNUM: Exactly.

KEVIN MALONEY: I have had a lot of good conversations over the past few years in this podcast on that.

I want to pivot to the topic of the day, which is dominating life, media, and ecosystems, etc., which is artificial intelligence. There is a barrage of narratives, misinformation, and acceleration around the technology, which seems to be changing every day.

You have been looking at this, studying it, and working with it since the 1980s, so you have seen many iterations of this thing we call artificial intelligence. For my audience, who are very interested in this space but not necessarily computer scientists or don’t work at Anthropic, maybe we can build a “stack” of understanding around how you view AI and then maybe we can address some of the misconceptions after we touch on the definitional part, understanding that it is quite the task, as I have found in reading your writing.

VIRGINIA DIGNUM: Where I come from is a long story again. I think at this moment the most confusing part of it all is the concept itself. We are using the concept of AI to mean everything and anything. It becomes an anti-signifier, like a couple of years ago everybody was talking about digitalization, and everything was digital and so on, so we tend to conflate a lot of issues—positive, negative, complex, and simple—all into one kind of concept.

Next to that, we tend to look at the concept of artificial intelligence mostly from one perspective only, and in this case what we are seeing is that we all are equating AI with the very latest version of what these systems can be, namely the large language models and generative AI systems that we are working with, and then everything about AI becomes about generative AI and generative AI is the only type of AI that is around. So on one hand, everything is AI. On the other hand, AI’s problems and benefits only refer to what the very latest type of system generative AI is able to do or not do.

The reality is much more complex and wider than that. There are many different ways to develop AI systems. In the last decade or so we have seen mostly models that are data-driven, based on analysis and correlations that can be made, statistical models that can be built over a set of data, but even those are not the whole story.

There are other ways to build AI which are not based on data but more based on relations, rules, plans, uncertainties, complexity, causality, and so on, which are nowadays much less visible because of the dominance of data-driven models but are not less AI than the data-driven models are. They have problems like data-driven models do, but they also have benefits. For instance, they are much more transparent and amenable to regulation and governance than the data-driven models. There is good and bad in both directions.

Again, the biggest complexity comes from the narratives that we are weaving around the concept of AI and what it is or is not.

KEVIN MALONEY: I have been doing a lot of work recently, going back and reading public intellectual commentary from the 1950s and 1960s on the emergence of nuclear weapons in the public space and the debate over what the technology was and a lack of understanding by elected officials, all wrapped in the idea that, “They are here, it’s inevitable, and we have to go along with it.”

There were different geopolitical circumstances then, but there was this inevitability narrative that went along with it. At the same time, you had a disconnect from a technological understanding of the thing and the gap with that for the people who were developing it or using it and the general public.

In terms of people who are going to be impacted by AI, how should someone think about their own agency, whether individually, politically, technologically, et cetera, in this moment? How can we own our agency in this AI moment?

VIRGINIA DIGNUM: That is an interesting and potentially very complex question. The sense of inevitability that you describe around nuclear weapons and nuclear energy in general is what we are feeling in AI as well at this moment. There is this idea that AI “happens to us” and that no one really has any say about what is happening, or at least we—the general public—have nothing to say about it. It happens to us and comes from I don’t know where.

It is a bit like the weather. We can look at the weather forecast and decide whether or not to take an umbrella to go to the beach, but we cannot change the weather, and this lack of control over what is happening is very much dominating public opinion but also and most importantly dominating decisionmakers and policymakers around the world: “It happens, and the only thing we can do is try to address the consequences, but we cannot really change the reality.”

Which is completely wrong and false. AI is not weather; AI is an artifact of technology that people build, and by being something that we have built it means that we have options, starting with the option of not building it, the options of doing it one way or another way, taking this or that into account, and so on.

We need to be much more conscious that AI does not happen to us but is the product of deliberately choosing options by in most cases a very small group of people who take those decisions. The narrative of AI being inevitable is also something which is also part of this artifact that we have because if it is inevitable we do not feel empowered to change the fact but can only look at it from outside. It is very important I think for all of us that we understand that it is much less inevitable than what they are trying to make us believe.

KEVIN MALONEY: The way I have been thinking about it sitting here at the Council is that the inevitability argument gives the arguer moral coverage for a very business-driven decision. To explain that to my listeners—and I would be interested in your comment on it—is that it is basically wrapped up in civilizational narratives, like, “We need to advance in AI because our values or way of life are on the line.”

It has been interesting to watch, and I have chatted about this with a few people on the podcasts, that those narratives have ebbed and flowed over the past few years. A few years ago, the pitch from the AI CEOs at South by Southwest was, “Well, this will cure cancer and democratize education,” and now it is leaning into national interest, geopolitics, militarization, and the zero-sum game of civilization.

That was not a question. I just want you to react to what you are seeing out of the talking points from the people who are pushing these narratives. This is now obviously in the bubble of initial public offerings and financial interests that are massive. Can I get your response to that swirling narrative ecosystem and how you are understanding it right now?

VIRGINIA DIGNUM: I think you are completely right. There is a whole universe of interests around the narrative of AI and the importance of maintaining a certain type of narrative and maintaining also the control over that narrative. That is one of the things that is important for the CEOs of the Big Tech companies that are at this moment taking the driver’s seat in all developments with a backdrop of an enormous amount of investment to a size we have never seen before. The money that goes into AI development is on a scale that is almost incomprehensible. It is a scale that is probably completely impossible to ever provide any return on this investment.

Yesterday I watched a new documentary, Ghost in the Machine, which came out a couple of months ago. In that documentary they show a snippet of Sam Altman, the CEO of OpenAI. They asked him, “How do you convince your investors to invest in you?”

He basically says is, “What I am telling my investors is that once we build the powerful AI that we are going to do, then we can ask this powerful AI how to guarantee their return on investment.”

So investment becomes a circular thing because what comes out will tell you what to do, which makes us give up on our own capabilities, so we don’t know any more what to do, but this thing we are going to build is going to tell us how to do it.

It is a similar case from a couple of years ago, when Eric Schmidt, who was at the time CEO of [Google], I believe at Davos said something like, “The climate problem is too big. We do not know how to solve it. The only thing we can is to invest massively in AI, and AI will solve it for us.”

These are the types of narratives that go around the circles where investments are being made, especially when the investments are made with public money—in many parts of the world and in the United States there is a lot of public money, but also money from pension funds and things like that.

The money that is being invested in AI is not being invested in something else, and these choices are very concrete and very deliberate, and no one is pointing out that this emperor has no clothes at all and there is not an infinite amount of money that can go into AI and also into everything else. We are making choices, and these choices are a huge gamble and definitely a pyramid scheme because there will never be investment that multiplies for all of us.

KEVIN MALONEY: There is so much to unpack there. One of the things that came out from your book—which is excellent and I recommend everybody take a look at it, The AI Paradox: How to Make Sense of a Complex Future—was this deeply humanist view of this moment in AI. What I took away from it is that we cannot give up the agency. We cannot give up the human-centeredness in what we are doing. We have to hold on with two hands.

You address a lot of paradoxes in the current moment, but we are seeing an ecosystem where for the first time there are very serious people with so much technology and resources that are anti-humanist or trans-humanist as both a means and an end state potentially.

I know these are big words, but if you boil it down to the most basic level, I think about it as, “We don’t care if we put a huge data center that ruins your air and keeps you up at night because our moral and political calculus is that it will save Western civilization in the end.”

I know I said a lot there, but for our listeners I would like to hear from you how you use this humanist perspective and are thinking about it in your work right now and the relevance of AI from more of a public sphere perspective.

VIRGINIA DIGNUM: There is a lot to say about all of these. Let me pick one of these threads, the “saving Western civilization” excuse they are selling us. I doubt very much that they are very concerned—“they” being the tech bros, CEOs, and developers of these companies—with Western civilization or civilization in general, but it is rather an individualistic and egocentric approach to the world: “If this technology will benefit myself and I am such an important, rich, and influential person, then it will benefit the rest as well.” It is a very small understanding of the world and a very egocentric understanding of what civilization is about.

It is unfortunately not something that is only around AI. I think we have been moving toward this type of egocentric morality for quite some time in other directions as well, but the power that control over AI gives them puts it all on steroids and grows these beliefs more and more.

On the other, humanistic, side of it, technology by itself is nothing. All technologies, since the very first technologies developed by people—the wheel, fire, society, communities in general—have existed only due to the fact that there is benefit for some people within the society. Technology to benefit technology is something which makes no sense.

Who is benefiting from technology and how are we ensuring or guaranteeing that technology is accessible to those who would benefit the most from it? The questions we need to address most urgently need to be addressed at the global level, the level of international agencies and organizations, which unfortunately at this moment are being left behind in the violence of this race and push toward a point where I don’t think anyone knows what is there, but we all feel we need to rush as quickly as possible to that point, which is probably some kind of cliff, and we will all fall down the cliff.

KEVIN MALONEY: I think this is a good point to pivot to some of the news of the day. We have Magnifica humanitas, a co-document from the Vatican but also heavily influenced by Anthropic, bringing together the moral and technological centers of the world in a lot of ways right now.

You recently wrote a very interesting piece analyzing what the document got right and some areas of criticism. In addition, one of the examples you recently wrote about is this parallel around responsible AI, governance, and the challenges there to the Industrial Revolution in terms of this thing was happening, we needed to govern it, and we were not going to get in front of it, but we could still do something to make it better. That framework was very powerful for me in seeing this historical example. Maybe we can start by getting your reaction to the document, the partnership between the Vatican and Anthropic, and then we can get into the specific pieces of it.

VIRGINIA DIGNUM: Firstly, the document is important due to the fact that it exists, that the Catholic Church took the effort to try to unpack and understand the moral and human consequences of this type of technology. The document, as you know, was developed in parallel to commemorate the anniversary of a previous encyclical by a pope many years ago around the Industrial Revolution and trying to reflect on the fact that at that time regulation, governance, and morality lagged behind technological developments. It is important due to the fact that it is here and that there is this effort from a figure of authority like the pope to have a say on what is there.

Having said that, I think there are a couple of issues we need to unpack around this encyclical. One is maybe the fact that it feels very reactive to this inevitability of AI. One of the sections is called I think “Remaining Human,” which already gives the feeling that we need to do something ourselves because otherwise technology takes the upper hand instead of asserting and reinforcing the importance and centrality of humanity in what we do in technology.

It also puts AI as the central problem of our days. It is of course an important and central issue, but I would say that if I was in Gaza, Ethiopia, or Nigeria at this moment, there are more central problems to humanity than what AI and a couple of Big Tech bros are doing with it in Silicon Valley. It may be making it part of all other types of problems—climate, poverty, war, threat of nuclear war, which is again much greater than it was some time ago—which need to be more balanced with AI.

Finally, many people have made the point that the fact that somehow Anthropic became a partner in this development makes it feel a bit like it was sponsored by Anthropic, and we need to start thinking about how much is it sponsorship of Anthropic and how much Anthropic’s support to the pope and experts that developed it.

Several very well-positioned theologists, a couple of philosophers, and Anthropic were involved in developing the document. There was very little space for civil society. There was very little space for the positions of those who are at this moment on the receiving end of the harms of AI. There was very little space to understand the position of the data workers and ghost workers behind all these types of systems, and I think that, from the perspective of the Catholic Church, is something I would have expected to be there. It would fit the Catholic Church to have made a bigger effort to look at those on the receiving end of the harms and not only seemingly looking at the ones who are receiving the benefits.

KEVIN MALONEY: One of the interesting things we think a lot about here at Carnegie Council are the moral and political narratives around a policy or technology but also trying to empower people in the room to balance questions of values and political pragmatism in a realistic way, the spectrum between liberal idealism and amoral realism, and being reflective of those things. The reason for that windup is that, yes, the document was very interesting, but what I personally would have liked to be a fly on the wall for was the internal debate at the Vatican on whether or not to partner with Anthropic on this because I think it raises some huge moral and political questions in terms of, are they using us—the Vatican—as an ethical carwash, as a means to an end, or do we think they are actually good-faith actors or the best of a bad bunch and we need from a pragmatism perspective to engage with somebody? Going behind closed doors and behind the document, what are your thoughts on that calculus?

VIRGINIA DIGNUM: One of the things you say is “the people in the room,” and this definitely, from what we have been able to see from the ones who participated in the meeting in which it was introduced, not everybody was in the room. I think one of the main concerns about it is that civil society organizations and the people who have been harmed were not definitely represented in that setup. That is an important one, and we cannot forget it.

I do not really know how they decided to partner with Anthropic or whether we should call it a partnership or support role or whatever. It is concerning also that, for people who have been in the field for quite some time, Anthropic comes with alarm bells. The way they address issues has not always been the most proper, ethical, or responsible approach to what it has been doing. Whether they are “the best of a bad bunch,” I doubt. There may be other ones at least as bad as they are.

I would say that focusing on AI as being the product or ownership of Big Tech is also in itself problematic. There are many other approaches to AI. In the field of generative AI there are academic and nonprofit organizations making significant contributions to the development of the science, so a partnership where a disproportionate part of the engagement was with one of the Big Tech groups, which comes with all the problems of being part of Big Tech, and not the most harmless of the Big Tech, is a choice, and what motivated this choice we cannot know. Like you say, being a fly on the wall would have been very nice. It is a choice which will probably have repercussions on the way the morality and responsibilities around AI are going to be developed moving forward.

KEVIN MALONEY: I want to close with a conversation about what you have identified as this “anthropomorphization error,” the idea that AI is itself the danger rather than, as you write, the human choice being the danger or being the positive or negative about its development, its deployment, or the absence of democratic control. Maybe we can close by talking about that because I do think this goes to the narrative points we discussed before around inevitability and this being a “silver bullet” technology.

There is a little bit of snake oil salesman there, as we have discussed, but this goes back to agency. Maybe we can talk about that framing and this error you have identified.

VIRGINIA DIGNUM: This all boils down to the current quest in AI. I think it is important also for listeners to understand that we did not define AI. We started with some discussion what it is or is not, but we did not really define AI. It is extremely difficult to define it, but a definition or gimmick that usually works is, “AI is what AI cannot do yet.” Those who are at the forefront of development of AI are trying to solve the next problem. Some years ago they were trying to get machines to play chess. That is no longer relevant. Spam filters was AI for many years. Nowadays spam filters are just a piece of software mitigation. Global Positioning System maps and navigation and digital maps were also at some time at the forefront. Then we had generative AI and the idea is that we have to move to artificial general intelligence (AGI).

Again, AGI is something that is difficult to define, but it is being sold to us as the solution to all of our problems, that once we have AGI everything will be solved, not by us but by this kind of artificial god that we are building which will do it all for us.

It is a very anthropomorphic and individualistic approach, and it assumes one machine which will do it all by itself, without needing us for anything because it will do it all better than us, and it assumes the idea that we are replaceable, that our efforts and capacities are not important anymore, that this machine we are going to build, this god we are going to create—a constant in human history is that we have always been trying to create a god somehow and somewhere—is going to solve it for us.

One of the points I make in the book is that if we take the most current definition of AGI, which is, “a system which surpasses human intelligence,” or is more intelligent than any one of us, then we do have this system already. You and I together are more intelligent than each one of us individually. If we make society and collaborations work well, we altogether are AGI. Together we are this superintelligence that is more than the sum of the parts of each one of us.

Instead of investing so much or putting so much of our hope onto this artificial thing outside of our control and independent of us that is going to solve all problems for us, I think a much more viable but also more engaging way to move forward is to look at how we can use this technology to support the collaboration and coordination efforts of all us. Then, together with the machines, we will become an even more powerful type of AGI than the machines alone or ourselves alone. That is one of the issues that I think is important and which gives me the hope that if we continue in that direction, then going back to where I started is a way to make the world a better place.

KEVIN MALONEY: That is a powerful way to think about it. I love it when guests come on and reframe things for me, when they go to the values and principles we hold dear here at Carnegie Council. As you said, there is so much more collective potential when it comes to humanity in terms of finding points of collaboration and interconnection all wrapped up in this thing that is pluralism if people give it a chance.

It is interesting to think about the spectrum of mining more collective human intelligence and cooperation on one end and “God in a box” that is controlled by five corporations on the other end. We are at a point where we can make the choice to push that in one direction or the other.

VIRGINIA DIGNUM: If you describe it exactly like those two points, I think most people will understand which side of that spectrum you want to be on.

KEVIN MALONEY: It seems self-evident when you say it out loud, but we are constantly bombarded, as you said, by the narratives.

I have been having a lot of conversations on the podcast recently around the concept of moral resilience and nihilism, whether it is thinking about it from a security or technological perspective, and to drill down on AI. I think so many people who are professors, students, or working in the tech space or salespeople, there is this pressure to upscale on AI and be more efficient. It is seductive in terms of outsourcing critical thinking or getting that one paper perfectly written. How do you make the case that there is value in using it as a tool but not as a substitute?

I am not saying this perfectly, but there is this potential addictiveness, this ease of use and outsourcing, and I think it is just beginning. I want to get your sense on that, and what would be your message to people, like other grassroots campaigns: How do we use it responsibly?

VIRGINIA DIGNUM: It starts by asking ourselves why we are using it. That is what I describe as the question: When is it appropriate? What kinds of problems? What kinds of situations? What kinds of issues benefit from using AI and which ones don’t? What is this efficiency we are rushing toward? Why do we want to rush and get more efficient and do more in less time and so on? This is something which has been going on for the last century. We are hardly able to understand what we are doing.

I was talking recently with some Indigenous communities, and from their perspective if we do what we need to do, we will still have the time to appreciate what we have, but if we are rushing all the time to do more, we are all the time moving the goalposts: “Once I get this, then I will appreciate my life,” but we are never really able to appreciate the here and now.

I think we need to collectively be much more aware of the here and now. What is this efficiency, what is this speed, why do we want to speed, what is the alternative, and when would we benefit from more purposeful and slow development? I think this race around innovation, that we have to be the first and the quickest and so on, is another narrative, another type of inevitability that has been taught to us so much that we are not able to get out of that rat race anymore.

I think those are the issues we need to start thinking about: Why are we using AI? Is AI really the best for whatever type of situation we are addressing? What are we really aiming for? Is it more and more and more and what do we get with this more and more and more that we have if it does not enable us to enjoy and appreciate ourselves, society, and communities in the here and now?

Finally, I think it is very important to take the opportunity or space to look at things from a less egocentric perspective and see how we are affecting the world around us by the decisions we collectively are taking by rushing toward the cliff of using more and more and being more and more efficient.

KEVIN MALONEY: This has been an enlightening conversation, Virginia. Thank you so much. I need to take a walk around the block because there is a lot to reflect on.

VIRGINIA DIGNUM: Slow down, yes.

KEVIN MALONEY: I am going to slow it down. I am going to underline that efficiency should not be an end and a means. What I am going to take away from this conversation is that we really need to be reflecting on AI: Is it helping us to be happier and better humans, or are we just turning into a cog in the wheel to advance the technology for other people’s interests?

Thank you so much for joining us. I know all my listeners are greatly appreciative as well.

VIRGINIA DIGNUM: Thank you very much. It was my pleasure, and it was a great discussion with you. I look forward to the reactions to it as well.