AI Governance Landscape


In my previous career, I worked in global
poverty eradication, mostly doing field work. This meant working on projects all around
the world, trying to find a place where I felt I could do the most good and have the
largest impact. A little over three years ago, while I was
working in Ethiopia, I discovered the effective altruism movement. When I did, I discovered just how big the
biggest impact could be. Think for a second about a single grain of
sand. Imagine rolling it between your pinched fingers. Picture how small it is. Now, think about how much sand there is on
earth, all the deserts: the Sahara, the Gobi, the Arabian. The ocean floor, all the beaches, all of the
dunes. If you press your palm to the sidewalk in
most places, when you come back, there will be little bits of sand embedded in it. There are a lot of grains of sand on earth. In fact, there are approximately a sextillion
grains of sand on earth. That’s a billion trillion. Take a second to imagine that grain of sand
pinched between your fingers again. Now, try to imagine that grain of sand is
everything of value on earth. All the people, all the animals, everything,
and hold that thought. There are 100 trillion reachable stars. We have approximately 10 trillion centuries
before entropy makes the universe difficult to inhabit. That means we have over a trillion trillion
equivalents of earth’s resources within our reach. That means if we imagine everything on earth
as a grain of sand, there are more earth equivalents out there, but within our reach, than there
are grains of sand on earth. The human brain is not good at thinking clearly
about issues of scale. When we think about doing good, we tend to
think in terms of prototypes. I’ve always been most motivated by human wellbeing,
so when I think about doing good, I tend to think of helping someone. Effective altruism is, at its core, about
making sure we don’t stop with this prototype and this emotion. It’s about trying to harness this, this altruistic
motivation to have an effect in the real world. For many of us, there’s something a bit disconcerting
with interacting with far future considerations. The prototype’s kind of abstract. It’s sort of weird. It transects a lot of science fiction and
transhumanism and things we may not really be that comfortable with or into. We care about human suffering. We care about animal suffering. And those things exist here, now, on earth. We can see them. I am very sympathetic to this situation. I am in this situation. However, the reality is that time and space
exist. They will be filled with something. Maybe they’re filled with dead matter, completely
devoid of life, and maybe that seems like a tremendous waste. Maybe they’re filled with terrible, exploitative
human societies, and full of factory farms, and that seems like a disaster. If what you altruistically value exists in
time and in space, and you want to do the most good, you have to think big. I may have this grain of sand in my fingers,
but if I care about doing the most good, I can’t focus on just this one grain of sand. There is a lot of sand out there. Humans have done three things in history which
stand above the others in terms of their impact on our own wellbeing and the wellbeing of
animals. The first was our developing behavioral modernity. This occurred about 70,000 years ago. The leading theory for behavioral modernity
was that it was the result of very small cognitive changes. Behavioral modernity allowed us to outlive
the neanderthals, who had been preventing us from gaining a bridgehead to leave Africa. With this, we were able to expand across the
world. As we did so, we caused the extinction of
most megafauna on earth, and of all other species of humans, including the neanderthals. The second was the development of agriculture,
which occurred around 12,000 years ago. This technological advance increased human
population about a hundredfold. It resulted in dramatic ecological damage,
the exploitation and extinction of animals, and the decrease in human health and wellbeing. The third was the industrial revolution, which
came with further ecological destruction and the advent of factory farming. But, it also created dramatic wealth and resulted
in the first sustained improvements in human health. I’ll also give a quick honorable mention to
atomic weapons, which haven’t had much substantive impact on earth, but definitely have that
capacity. Our intelligence, and the technologies we
invent, are how humans have our biggest impact. For this and for other reasons, it is safe
to imagine that the technology of intelligence will be at least as impactful for humanity
and for animals as the industrial revolution. AI in its current state is not that impressive. It recognizes images well and it’s good at
some games, but it’s easy to get lost in the details at the beginning of a large change,
and to miss the forest for the trees. The earliest stage of the development of behavioral
modernity was extremely subtle and mostly looked like – most likely – mostly looked
like slightly improved communication within human bands. This took place over the course of thousands
of years. The line between early agriculture and certain
types of seed-gathering techniques is so subtle that modern anthropologists can’t always agree
about what’s going on with modern bands of humans. Steam engines were adopted slowly and were
very unimpressive for a long time. With agriculture and with industry, humans
living at the early stages could not have conceptualized what they would develop into. Fortunately for us, with intelligence, we
do have one model of what it looks like when it reaches an advanced stage. The difference between humans and gorillas
is a few million years of evolution. Gorillas now only exist in zoos and a few
little reserves that we keep them on so that we can look at them. The difference in intelligence between humans
and neanderthals was, as far as we can tell, very small, and as soon as behavioral modernity
gave us a little edge on them, we wiped them off the face of the earth. For the first time, we can kind of see a dim
outline of an enormous change on the horizon, and it is coming at us. Also, for the first time, this change is not
just going to affect our one unimaginably valuable grain of sand. This is for all the sand. The goal of AI governance is to help humanity
best navigate the transition to a world with advanced AI systems. It is possible that the default course of
development and deployment of advanced AI will be great. There is an optimistic case that can be made
that this transition is very unlike the last three, and it will go quite smoothly. The last three were sort of mixed. If this is the case, then with AI governance
as a cause area, the best we can hope for is maybe small marginal improvements, though
again, across a lot of sand. Unfortunately, there are also good reasons
to think that the default course might be fraught. In a survey of AI experts who publish in top
journals, the majority assigned at least a 5% probability that superhuman AI would be
extremely bad on the order of human extinction. Even among researchers who are highly optimistic
about AI, it is generally agreed that advanced AI will not necessarily be safe, or at least
cannot be guaranteed to be safe without some work and testing. So, we’ll do the work and testing, right? Maybe. It depends here how high the cost is in terms
of time, resources, and the performance of the system itself, and how competitive the
environment is in which it is being developed. This is sometimes called the “safety tax”. For example, what if the safety tax is two
years, and yet there are several companies at the forefront of development, with tens
of billions or hundreds of billions of dollars at stake, who are only a few months apart? Now, worse. What if these are not companies that are a
few months apart from one another, but rival militaries? During the Manhattan Project, there was a
serious concern that detonation might ignite the earth’s atmosphere and kill everyone. It didn’t really slow them down. Now, let’s imagine that AI can be made safe
easily, and that part is just covered. This is not nearly sufficient to guarantee
a good outcome. Some tech companies have impressively cosmopolitan
values, but there are legal constraints based on fiduciary duties they have to their stockholders. We also probably do not want any one company
too powerful, or to control a particularly large stake in the future of the universe. It’s also unclear if a government would let
this happen without somehow taking control of AI development or deployment. So, what if this technology is developed by
or is in some way captured or seized by a government? Realistically, the US government or the Chinese
government. How comfortable are we with either the US
or China dictating the future of the world? Scarier question. How comfortable do we think other nuclear
states are with this? Now, let’s imagine we’ve managed to thread
the two previous needles. AI is safe, and it was deployed without igniting
a great power war. How are the benefits distributed? Continental Europe does not have a viable
AI industry, let alone Latin America, Africa, or most of Asia. Unlike during the industrial revolution, where
human labor was a complement to machinery, it does seem as though with AI, it mostly
serves as a substitute for human labor. Additionally, most AI services have characteristics
of a natural monopoly or an oligopoly. There’s not much room for competition, and
there’s not really much room for small businesses. Maybe we are so wealthy after the development,
that wealth naturally sort of trickles down, that a rising tide is able to lift all boats. But, we are extremely wealthy now, and children
still die of malnutrition, malaria, and diarrheal diseases by the millions. And so far, the wealthier we get as a world,
the more animals we subject to factory farming. For us working in AI governance, the current
priority is to increase coordination and to decrease race dynamics. This is both to reduce safety risk, and the
risk of greater geopolitical destabilization. It’s also hopefully to increase the breadth
of stakeholders who are able to have a say in how the future is developed, and in doing
so, hopefully to increase the moral circle under consideration. There are several routes for this that we’re
pursuing in parallel. This includes technical work on the possibilities
of verification of AI-related international agreements, which are really necessary for
coordination between nation-states, and in some ways a prerequisite for that being possible. Also, in case we someday we might want something
like the international control of advanced artificial intelligence, not saying that we
do, but if we do potentially want that option, we’ve been looking at case studies of other
failed efforts to control decisive weapons technologies in the past. For example, after World War I, there were
several serious proposals to develop an international air force. This sounds a little weird, but there was
actually buy-in from the United States, Japan, and France, and even Winston Churchill, who
was not a dove, for this as a plan. The idea was for the League of Nations to
have a complete monopoly on military aviation, with all other states banned from getting
it. This international air force, then, would
have two functions. One was to prevent any state from then developing
military aviation, or from attacking any aggressors who then started a war, with the understanding
being that this would actually secure world peace. Similarly, after World War II, and the atomic
bombing of Japan, the US proposed to give up all of its atomic technology, including
all of its resources, to an international body, and to subject itself to intensive inspections,
to show that it hadn’t retained any and that it wasn’t restarting this, if the Soviet Union
and other states agreed to do similarly. Understanding how these failed, and trying
to learn lessons from these failures, might increase the chances that we’re able in the
future to gain international control, if that is necessary. The third time is the charm. We’ve also been doing some work in science
diplomacy. Science as a field is very cosmopolitan. Even quite adversarial nations are quite good
at collaborating in science. ITER is probably the best case study for this. ITER is a research project looking at nuclear
fusion, and is being funded for many, many billions of dollars. It’s a joint venture by the US, China, Russia,
the EU, and some other states. What’s important about this is that nuclear
fusion is an important strategic technology. This is why the vast majority of this research
is funded actually by the military, or the department of energy, but under military guard
and sometimes for military purposes. The knowledge and the results from this technology
and from this experiment are supposed to be spread out from all the participants. If it does someday make sense to try to do
this as a collaborative effort, with many stakeholders, I think this might prove to
be the best current model we have. We also do a lot of tech company and AI research
community engagement. The OpenAI Charter is possibly the largest
recent win in this category. In the charter, OpenAI commits to developing
AI for the common benefit of all humanity. But more interestingly, it commits to dropping
out of anything like an AI race if a rival gains a lead, and even actually to join the
efforts, free, to push them further forward into a greater lead. Anyone who’s interested in this, I would actually
really encourage you to read this charter. It is inspiring. It is also very short. If you have good vision, you could read it
printed on a business card. Also, if you’re interested more in our engagement
work, I would encourage you to talk with Jade Leung, who does this for our governance group. So, I won’t go into too many more details
here, through I will provide resources at the end for people who want to read up on
more of what we’re doing and some of the work. But to do a quick summary, we’re trying to
understand the AI scene better in China and increase our engagement there. We’re also trying to better understand issues
of geopolitical instability that can be created by changes in the balance of power and also
in the offensive and defensive dynamics of weapons technologies. We are also interested in modeling the dynamics,
or engaging in modeling the dynamics of tech races, and in particular, how they can be
averted or spun down if they do begin, and a lot more. AI governance, as a cause area, is absolutely
tiny. People talk a lot about AI at these conferences,
which gives the impression that there’s a lot going on, but there are fewer than a dozen
people who work on this full-time. There are not that many more who work on this
part-time. This being a tiny field is in some ways quite
good for you. It means that your opportunity for a marginal
impact is absolutely huge, but unfortunately, what it also means is there is not very much
absorptive capacity within existing groups to bring more people onboard. For the majority of people who are interested
in advancing this as a cause area, it is probably mostly about preparing yourselves to be more
involved in the future as the field grows and expands. That said, as far as our immediate needs,
we are desperate for people who are good at getting things done. This is something we do need immediately and
there are immediate job openings for. This is good project managers, good operations
folks of all sorts. This is also… as well as being currently
our largest bottleneck, this might be one of the main reasons why our absorptive capacity
to bring other people on is so low. So, there’s a force-multiplicative aspect
of this as well. I want to in particular highlight: there’s
a role at the Governance of AI program in Oxford, which is currently the leading group
researching this, for a governance project manager. This is a Visa-eligible position, and this
will put whoever gets this role at absolutely the center of this as a cause area. We also badly need senior researchers who
can supervise more junior researchers. Our research pyramid is sort of imbalanced,
with the bottom a little too big for the top, and this is another bottleneck. We also have positions for junior researchers,
which I definitely encourage people to apply to, though I think this is maybe less urgent
or less impactful in the margin within this cause area. For the majority of those who are serious
about getting involved, what I recommend is investing in your skills and your career capital. This is especially true in high value areas,
including studying topics in high demand. Also, building a career in a position of influence,
especially in government, global institutions, or for example doing policy advising at a
tech firm. Additionally, helping to build this community,
and our capacity, including having a strong and mutually reinforcing career network among
people who are interested in getting involved in AI policy from an EA or an altruistic perspective. In my personal opinion, this third point is
more important than most people realize. Having a strong social network, and having
people who understand what it is you care about and why it is you’re doing the things
you are doing, even if it in some ways looks quite tangential to what it is you care about,
can be very helpful in allowing you to keep your eye on the prize, and to work towards
your goal. The reason this social support can be so important
is, as I mentioned at the beginning of the talk, the human brain is not good at thinking
clearly in terms of scale. This means it’s very easy, and natural even,
to have some value drift. I’ve always cared a lot about human wellbeing. This is why I pursued my career in… my education
in economic development, and my career in poverty eradication. My personal experiences in field work around
the world have left me with a very strong, very emotional sense and prototype that still
draws me very strongly to global poverty eradication. But, I don’t want my altruism to be about
what I feel. I want it to be about how much good I can
actually do and how much of an impact I can have outside of me, in the real world. Even if I actually can’t conceptualize just
how much there is, I do know where all the sand is. If you’re interested in learning more about
our work, please go to our website. Also, Jade and I will have office hours immediately
after our talk, and we have a meet-up this afternoon at 3:30, which I would definitely
encourage people to attend. Thank you very much.

Add a Comment

Your email address will not be published. Required fields are marked *