人工智能时代下的国家主权

KEVIN MALONEY: Artificial Intelligence (AI) is massively disrupting geopolitics as we know it. It is introducing new moral, political, and economic questions for both states and non-state actors alike. At the center of many of these questions is the issue of AI sovereignty. I am joined today on the podcast by Arun Sundararajan, who is a professor at New York University’s Stern School of Business.

Together we have our first in a series of conversations on the issue of AI sovereignty, the consequences, and the downstream effects of the acceleration and integration of artificial intelligence into geopolitics today. I hope you enjoy the conversation. Let’s get right to it.

Arun Sunararajan, thank you so much for coming to Carnegie Council today.

ARUN SUNDARARAJAN: It is always a pleasure to be here, Kevin.

KEVIN MALONEY: It is exciting. This is the first conversation you and I will be having. We will be welcoming some new guests for Values & Interests, and we are going to be specifically looking at this issue of AI sovereignty, how it is remaking the world and the ethical questions at the center of it.

What I would like to do today in this initial conversation is introduce you as my thought partner in this series to our listeners at Carnegie Council. As we do on the Values & Interests podcast, I want to start with you, the man. Let’s “peel back that onion,” as Shrek says and think about your values CV as we talk about it here. Let’s hear a little bit about you and your own value system, and then we can pivot into your career and then get into the issue of AI sovereignty.

ARUN SUNDARARAJAN: Absolutely. I always pause when someone says, “Let’s peel back the onion,” because you never know what you are going to find as you peel back those layers.

I am so happy to be partnering with you in this important conversation. There have been different metrics of digital sovereignty that have come up at different points in the last 20-25 years, different concerns that different countries have had about autonomous they are, but I think AI sovereignty has taken this conversation to a different level. Looking at the ethical issues, how this aligns with different values and interests, is going to be fascinating.

In thinking about myself and my values, I grew up in England and India. I was born in England and grew up partly there and partly in India, so I was exposed to a lot of contrast early in life. There are questions around inequality that you cannot avoid when you grow up in a country like India. I was relatively privileged compared to the median, for sure.

In thinking about early experiences that shaped what I think is right and good should happen, there has been a definite thread of thinking about why so much inequality exists in the world and how does one reduce it. That is definitely one dimension or core value, to try to make things more equal.

Indian philosophy is complex and rich, complex from the point of view that there is no one book or one set of principles. It is a tapestry of different things, some mythology, some religion, and some philosophy.

My parents certainly exemplified different approaches in their value systems. My father was what people in India would call a Karma Yogi. This is one of four paths, the path of duty or service, where your meaning and purpose in life come from doing for others. He was a surgeon, and his example was a very strong influence for me in defining what was good. He seemed to have a very strong moral compass. He seemed to exemplify this notion of thinking about the interests of others and the impact of your actions on others is an important part of being a good human being.

My mother is much more of a pragmatist, so that was always an interesting contrast. My dad is more Kant, my mother more utilitarian. That was a good tension to observe, and I guess I probably landed somewhere in the middle.

There is a different path of discovery in Indian philosophy. There is the path of worship, which certainly does not describe me, there is the path of meditation, and then there is the path of knowledge, Jnana Yoga, and that probably describes my guiding light through life. There are certainly strong feelings of what is good and a desire to quantify and reduce or think about ways in which systems can become more equal and more equitable, thinking about externalities and the impacts of your actions on others.

Then there is a constant tension between what is good for the self and what is good for others, what is the right balance between giving to the world and pursuing one’s self-interests. My Ph.D. studies were in economics. Classical microeconomics in the Western tradition preaches that human beings maximize self-interest, so much like the tapestry of Indian philosophy I have a whole bunch of competing systems that shape why I do what I do.

KEVIN MALONEY: That is very interesting. One of the things I do want to touch on is that we have had a number of people on the podcast over the last few years who have self-described or maybe the conversation has led to that they think of themselves as “pragmatic idealists.” We have yet to have a household that is described in that way, so I do love that. This is a great walk-through in terms of knowledge, worship, and meditation.

Also at the Council we think a lot about this values pluralism but also the universality that we believe deeply underpins the human experience. As you walk through this knowledge, worship, and meditation from an Indian perspective, you cannot help but go back to Aristotelian thought in terms of how they divided this as well. I think there are a lot of parallels in terms of what you have talked about but also this moment of technological innovation and what it means to be human.

We find ourselves at a moment when I think there are very different communities creating new value systems on top of these new technologies. This is why this series and conversation are important, to explore those and not just assume that you can fit a square peg in a round hole while having respect for this universality that you talk through in your own experience.

ARUN SUNDARARAJAN: Absolutely. I think it adds a whole new dimension to values and interests. This could be a whole separate conversation. I know I have had many interesting conversations at the Carnegie Council about this, that as we create these new systems that may or may not be sentient, may or may not be conscious at some point, but putting that aside will shape the outcomes of the world to a greater and greater degree however we might define their identity and the extent to which they are going to have personhood. There is no doubt that a lot of what happens in the world to us humans is going to be shaped by the values that are embedded in these machines.

It is interesting to think about this through the lens of what different countries care about from the point of view of their autonomy over their AI systems. There are, of course, strategic and economic reasons, and we will get to this, but there are also issues of values that are embedded in these systems, and I have found that these it is often much more powerful trigger for a state leader to be thinking about the risks associated with delegating their AI to another country.

KEVIN MALONEY: We are going to get to the AI sovereignty piece, but it is helpful to lay the groundwork with some of these bigger values-based questions.

Before we get there, I would love for you to give our listeners a sense of your pursuit of wisdom and knowledge over the years. You have advised governments and private companies, always at the intersection of bigger economic and geopolitical questions in terms of how tech is being rolled out in society. Maybe you can give us an overview of the driving knowledge forces that you have been interested in over the past few decades and how this has led to this interest right now in this AI moment for you.

ARUN SUNDARARAJAN: I have been interested in digital technology from the time I was a kid. I programmed computers when I was 14 years old, which, at the risk of dating myself, was not quite as common when I was 14 years old as it is for today’s 14-year-olds. My career has been around understanding change caused by digital technology. I studied electrical engineering, computer science, and economics and decided to be a professor who studies technological change rather than being a software engineer or someone who builds computer systems eventually.

I do not think I could have foreseen 20-odd years ago just how enormous the scope of digital change was going to be in the world. In some ways it makes me pleasantly surprised, but it also has an impact on me when I look at the news and see that the issues I am thinking about in my work, the policy questions I am asking, are front and center in the headlines today. Even ten years ago they were important, but they were on the fringes.

Ten years ago, the big questions were around the balance between technology companies and the increasingly powerful platforms and governments, and it was clear that we reached a moment, say, in 2015 or 2016, when collectively different societies had handed over certain regulatory and governance responsibilities to these platforms. I see it as us having de facto handed over these responsibilities to them, certain surveillance and censorship responsibilities. There were payment systems and money was becoming private at that time with the interest in blockchain. There was an increasing overlap between digital technologies and national defense.

In all of that, my greatest interest was in asking where the line should be drawn in terms of what a society relies on its government for to set the rules of the game, this question of how do you divide responsibility between what the platform chooses and what you rely on the government for to set the rules of the game for a society. It seemed like an important question, not just because if we did not make that decision proactively, it was being made for us, and then we would end up in a world that we did not consciously design and we may not like the design of the world that we end up in.

Society has had a rich history of having different stakeholders—professional organizations, religious organizations, labor unions—that work alongside the government in determining how things end up for civil society. I would say that over the last ten years a lot of my thinking has been around understanding how much of a player the digital platforms are, like a new stakeholder that is designing society for us, and how to get governments to think of these platforms not as the protagonists in some ways or the target of regulation but as partners because it is easy to recognize today that the partnership model is the one that is going to work, but I think a decade ago there was still a resistance to thinking of them as someone who should have a seat at the table rather than someone who was the target of, “You do what I tell you to do.”

KEVIN MALONEY: The adversarial relationship is interesting. The difficult thing that I have heard from other guests on the podcast is the ethics good faith/bad faith dynamic, whether the deregulation push for greater innovation is set within the bounds of a pluralistic democracy or if you are trying to do this as a means to achieve a darker, illiberal end at the end of the day. I think the norms have been shifting so fast in terms of statements from certain tech executives or pushback from certain other groups that this is the big good faith-versus-bad faith question.

I had a conversation here a few weeks ago, talking about the differences between some tech leaders today and the robber baron oligarchs of yesterday and that their push for deregulation and innovation stopped at a very strong sense of democracy and nationalism. You did not have Andrew Carnegie making a side deal with the kaiser for something, for example, as he was trying to stop World War I for happening.

We did not need to go down that rabbit hole, but I think this is something maybe we can explore with future guests in this series, that there is a real tension within liberal democracies right now in terms of how much do you give away on this deregulation/innovation question and do you sacrifice a robust pluralistic democracy if you do that down the line. This is an area we have been thinking deeply about here at the Council.

ARUN SUNDARARAJAN: I think that is a fascinating direction for us to explore. I have been engaging in this exercise with my students actually where I have them think about, if you are a tech CEO today, to what extent should your choices be shaped by your own principles and values versus simply doing what is best for your shareholders.

There are different people who are tech leaders today who exemplify different balances between the values-driven and pragmatic or are driven by different values, like, “My responsibility is only to my shareholders,” but I think what is always interesting is that when you present students with leaders whose values might not align with theirs necessarily is when they sit up and think very deeply about this because the knee-jerk reaction is that it does make sense and people look at all the things that Anthropic’s leader has been doing. Then I ask them to read Alex Karp’s book, which is very compelling in many ways, and say, “Well, here’s another value system, coherently laid out. How would you think about the balance here?”

These are not easy questions to answer. It is just that there is not a lot of time for us to think about them because decisions are being made. I guess my interest in this AI moment is because so much of what our society is going to look like and the extent to which our social fabric is going to hold is going to be shaped by the decisions we make in the next three or four years, so it is important that we make good ones.

KEVIN MALONEY: I think you provided an overview of how we think about applied ethics here. It is not zero-sum, and it is not going to provide you with an answer so that you can then put your head in the sand and let it play out. There is a spectrum between principles, pragmatism, and naked transactionalism, but ethics allows you to be aware of those tradeoffs and to interrogate those values and then hopefully make a morally informed but pragmatic decision at the end of the day that you have to own the consequences of.

A lot of what we do at the Council is provide people frameworks to attempt to do that, and this is the trick with the ethics: You have to be descriptive without being overly proscriptive because then it takes away the whole point of the ethical exercise. It’s you; you have to own it.

I think this is a good time to pivot into this big geopolitical, technological, and moral question of the moment, which is AI sovereignty. Maybe we can reset the table a bit and give the listeners an overview of AI sovereignty right now, and then we can get into the nitty-gritty of it.

ARUN SUNDARARAJAN: You can think of AI sovereignty at a very simple and high level, and I am thinking about it, at least to start off, from the point of view of a nation-state. You can also think about it from the point of a company, individual, or an Indigenous community that is trying to preserve their language, for example, but from the point of a nation-state it is, simply put, how much control do I have over the AI systems and technology that shape outcomes within my nation and for my citizens?

An idealistic path might be, “Well, every nation should have absolute control over the AI systems that drive the outcomes in their country for their citizens.”

The reality is that no nation-state is going to have absolute control. Whatever the broader trend toward polarization and closing down global trade might be, the reality of artificial intelligence highlights the interconnectedness of the world in many ways.

There are a couple of other lenses through which you could look at it or give it more structure. You could think of sovereignty as being either related to strategic autonomy, which is about a state being able to set its own priorities and make its own decisions. It could also be related to economic autonomy, like when a state is pursuing its economic goals and being as unconstrained as possible.

In some ways, AI sovereignty has come out of a short-lived but rapidly moving conversation about digital sovereignty. China may be the country that over the last 15 years has exemplified a pursuit of digital sovereignty. China’s tech ecosystem has evolved to be one where the country is largely self-sufficient in a number of critical aspects of technology in everyday life, which is not the case with most countries. They have their own e-commerce platforms, they have their own social media platforms, and a wide variety of different things they have consciously engineered to be sovereign over.

The European Union over the last few years has had a growing and greater interest in having some level of strategic autonomy when it comes to cloud computing. I do not think they have it yet, but it has certainly been an interest. There is a different dimension of digital sovereignty that is addressed by regulations like the General Data Protection Regulation and the Digital Markets Act with platforms.

But in unpacking AI and the complexity of AI, I am a business professor, so I think about value chains and all of the different pieces of getting that magical AI system to work. So you have the semiconductor layer, the chips, the data center or AI infrastructure layer, the massive amounts of deployed computing, and you have the language models or models more generally, diffusion models, world models, what is sitting underneath Claude, ChatGPT, or Gemini, then you have the applications, talent, and so on.

A lot of people will often think about this through the lens of the models. If you think about the narrative around the United States versus China, it is about OpenAI, Anthropic, and Google versus DeepSeek, Z, and Kimi, this model is doing better than that model, but a lot of the autonomy or lack thereof sits further upstream, at the layer of semiconductors.

There is not a single country in the world that is sovereign as far as this critical input, the AI chips. People in the United States might say, “Hey, we’ve got NVIDIA, we’ve got Qualcomm, we’ve got Intel, aren’t we sovereign,” but all of these chips are manufactured in Taiwan. All of the high-end chips are made by a company called TSMC, which is primarily manufacturing them in Taiwan.

A critical input that TSMC needs is a set of machines from a Dutch company called ASML, and there are currently export restrictions on sending these ASML machines to China. This has been a big geopolitical issue over the last four or five years. On the semiconductor front there may be different countries that have certain levels of sovereignty when it comes to design, but this lays bare the inherent interconnectedness of the world.

With data centers or capacity, at this moment in time I would estimate that the United States probably has three-quarters of the world’s AI compute capacity, China has about 15 percent of it, and the rest of the world has the remaining 10 percent, so there is a tremendous amount of inequality in deployed compute capacity, and the quest for sovereignty over that may be particularly challenging for a couple of reasons. One is that there is a shortage of the things that go into building this AI capacity. There is a tremendous amount of capital that is needed. There are literally trillions of dollars spent every year on building out this AI capacity.

If you are in a country where not every household has been electrified, and someone says, “Well, for the power that an NVIDIA graphics processing unit needs you can provide basic electricity to 100 households,” that presents a very different question to certain countries relative to other countries, so supply constraints are going to shape how sovereign a country is on the compute front.

On the model front, we are at an interesting moment because more and more of what these AI systems do is about what is wrapped around the large language model (LLM), this thing called the “harness.” The idea of following who is leading in the model wars may not be telling the whole story.

KEVIN MALONEY: It is the shiny object that seems to be driving valuations which drive the narrative.

ARUN SUNDARARAJAN: I would not be surprised if a lot of these are open weight in the long run, meaning that they are open but companies are like, “Well, I’ll let you know how I’m doing that basic internal processing, but there are a whole bunch of things I have wrapped around it that are proprietary, which is what is going to make you keep coming back to me.”

KEVIN MALONEY: You are starting to see that with Salesforce, for example, in terms of the integration within that. It is very interesting.

ARUN SUNDARARAJAN: The message that I am getting to is that AI sovereignty is not a binary concept. It is a question of degree. It is a question of, what layer of the AI value chain are you going to aspire to being sovereign over, how much do your resources constrain you from pursuing sovereignty in one layer versus the other versus the relative strategic and economic importance, and also thinking about it not just from the individual nation-state point of view.

It may well be the case that a lot of the aspirations of strategic or economic autonomy on the AI front are achieved by countries not from them going it alone but from them banding together with other countries with which they have trade relations, geographic proximity, cultural affinity, language affinity, and prior economic or strategic alliances. To me that is a big question that countries should be asking. If you unpack it, it is like, what layer, to what degree, and with whom am I going to pursue my sovereignty goals?

KEVIN MALONEY: If we want to go back to the virtue of pragmatism, and this is a multi-part question, so you have the top of the chain here in terms of the United States and China. If you are talking to people in those governments in a scenario, how would you advise them or talk to them about this AI sovereignty question and its potential to create new relationships, positive, reciprocity-based relationships, with smaller or middle powers?

Also, if you are talking to middle or smaller powers, how would you think about their position in, say, the geopolitical market right now and what are the things they should be thinking about, not just now as we are distracted by what the LLM model can do next but one, two, three, four, or five years down the line in terms of navigating this moment?

ARUN SUNDARARAJAN: AI is a really challenging space to look at five years down the road, especially at this moment. There is an inherent unpredictability about what the models are going to be capable of and what the systems are going to be capable of even in six months to a year.

That said, there is a familiar dynamic of influence that is associated with what the United States and China are pursuing. The United States has very successfully, post-World War II, created an ecosystem of other countries and technological relationships that continue to serve it well, even today, which is what Joe Biden relied on in 2022 when he asked Japan and the Netherlands to stop exporting these chipmaking machines to China, so thinking about growing that ecosystem. I would think that if one is the United States or one is China it is hard not to look at it through that lens of who is going to be within my ecosystem.

KEVIN MALONEY: The sphere of influence.

ARUN SUNDARARAJAN: From the point of view of a country like India or Brazil, I think the big questions is, it is a real challenge to think about how independent should I be because there are certainly resource constraints, but there is also a more natural inclination to say, “I should be able to go it alone” or be sovereign for the most part, and I think that is where the critical question of how many layers and to what degree.

India has become very clearly a global leader in digital public infrastructure and they are a model for the rest of the world there. They are certainly exporting that to other countries. If AI is woven into that, to what extent will that become a kernel through which they can spread their influence, and what is the balance between that and dependence on the United States versus China, who just have more resources?

For smaller countries, it is a balancing act. We should not assume that more AI is always better. I am not a technological determinist, meaning, I do not believe that once the technology can do it, it will happen, so while the capabilities of AI systems to do what a lot of humans could do have exceeded the capabilities of humans in a wide variety of jobs that does not necessarily mean that instantly the AI systems will be doing these jobs instead of the humans.

There is a process by which we are going to see labor market disruptions, but the faster the AI in your country progresses the faster this disruption will unfold. The slower your AI systems progress the more time you have to plan that transition.

I would be happy if the United States had some of this thinking as well, which is, how do we balance leadership with the pace of change and putting in place institutional structures that manage the change that might come? For a smaller country, part of what their calculus should be is, I should not be driven necessarily by a worry that if I do not have the latest AI, I am going to fall behind. You should think about aligning the capabilities you bring in with the needs and think that there may be a silver lining around not being on the cutting edge.

KEVIN MALONEY: At the beginning of the interview you talked about looking at inequality and this being a formative thing in your own value system. We had previous conversations before today about the potential for inequality to snowball with this dynamic.

I think the criticism, rightfully so in many ways, of the U.S.-led system of the last 80 years is that there are significant power imbalances baked into the system, and that has been a big debate for a long time, but having the conversation today it feels like this could go in one of two directions, from a bring into the fold, partnership, trade, economic, a rising tide lifts all boats while there will still be a lot of pain or a much darker path, where inequality gets even more extreme than it is right now, not just along classic geopolitical lines, but this new thing of AI introduces a variable that has a speed, moral fog, and a political cloud all at once. I would love to hear your thoughts on the potential for inequality and what you are thinking about and mapping it out in the years to come.

ARUN SUNDARARAJAN: That is an important question at this moment. In discussions about digital technology and inequality, I have always been the person at the table who disagrees with the assertion that digital has exacerbated inequality, and my logic has been that it depends on the metric of inequality. If you look at countries where people did not have access to telephony or video entertainment 30 or 40 years ago and you look at what digital technology has done, along these two dimensions of quality of life, digital technology has certainly led to a much greater level of equality. On the income and wealth front, the causal relationship is unclear.

One of the three lenses through which I look at the impact of AI on inequality is by looking at what we are measuring. I think that by a lot of measures that do not have to do with money or power but only access to the ability to start a business, access to knowledge, healthcare, and different forms of support, AI could be a massive equalizer. How this shows up in national statistics is unclear, but that is the optimistic dimension. I am not saying that these are three competing lenses. These are three parallel lenses in some ways.

A second lens has to do with the level of income and wealth inequality within a country. I think it is hard to get away from the fact that as the substitution of human talent for AI capabilities starts to accelerate there will be a phase we go through where income inequality could grow because of AI. The countervailing factor is that I have never seen a technology in my lifetime that lowered the barriers to creating things of value the way AI has, so if you think about creating things of value as what drives the economy, more and more people have access to this.

Can they capture the value that they create? That depends less on the technology but more on the economic, social, and political systems within which they sit. That could temper this to some extent, but that is something a country has to think about managing very carefully during the rollout of a technology and the kind of massive change we saw with the rollout of electricity and 20th-century industrialization when we moved from being more of an agrarian economy to more of an industrial economy, and there were periods of the 1920s and 1930s when there was deepening inequality before things reset.

The third lens is across countries. At this moment, it seems quite likely that at least in the near term AI will increase the power inequalities globally. There are a lot of advantages that the United States and China have today in terms of the progress they have made. I say “in the short run” because it is not clear to me—

There is certainly going to continue to be progress in AI. There are models like the language models that have fascinated us for the last four or five years that are being built for robots, these kinds of how-do-you-navigate-the-world models, so that when you train a car to drive itself or train a robot to navigate, you are not doing it in a task-specific way. You are giving it a model of how the world works in the same way that ChatGPT had a model of how language works, and then you could teach it how to converse. This will come fairly soon, and then we will have a new class of AI.

From the point of view of thinking about what a society needs and what is actually going to shape power balances or imbalances, in ten years I feel it will have moved away from the cutting edge of AI. It will depend on economic resources. In some ways we have already seen that. Power imbalances in the world today in part have to do with economic resources. They do not really have to do with the cutting edge of weapons technology. That is relevant to some extent, but it is not the thing that makes the difference.

I think sufficient AI is something that over time will be available to a lot of different countries, so this question of AI sovereignty may be less of an issue in 20 years than it is today. The question is, as we live through these next 20 years, what are the implications going to be?

KEVIN MALONEY: I am looking forward in this series to exploring more of these big questions in terms of, not to be dispassionate about it, the “acceptable” inequality rate and what is feasible in terms of keeping societies together and not going off a significant cliff like in 1929 or something like that combined with how do you maintain and even reinvent sovereignty within a geopolitical construct as these things become increasingly disruptive.

I am looking forward to exploring this with you and a few of our guests, so everybody stay tuned. There will be more conversations coming out of the podcast. Arun, thank you so much for joining us today.

ARUN SUNDARARAJAN: I am delighted to have started this conversation. Digital has been stretching the idea of sovereignty for a while. Sovereignty is an old concept and one where I feel like we need to reshape its boundaries to some extent so that we can gain a clear understanding of what AI sovereignty actually means and what its implications are going to be for the world.

KEVIN MALONEY: That stubborn nation-state model that we cannot get rid of in terms of international relations. We are not trying to get rid of it but just understand it in a current context. Thank you so much again.

ARUN SUNDARARAJAN: Thanks, Kevin.