Generative AI is a Climate Disaster

Paris Marx: Sasha, welcome to Tech Won't Save Us. Sasha Luccioni: Thanks for having me. PM: Absolutely. Very excited to chat with you. I know that you've been researching this topic for a long time, but it's one that is very important in this moment. We've been talking about having this conversation for a while, but recently there were these figures that Microsoft and Google revealed that — despite making net zero and carbon neutrality pledges back in 2019 — their emissions are soaring. Microsoft's are up 30% between 2020 and 2023. Google's are up 48% in five years, and they say that a lot of that is related to this AI rush that they have been in for a while now. What was your reaction when you heard those numbers from those major companies? SL: Honestly, I was expecting something like that. I was surprised by the fact that they were actually transparent about it, because usually this stuff gets swept under the rug or put in the footnotes or something. But in this case, it was pretty straightforward. And it's not that surprising if you're following the evolution, like I've been for the last, what, seven years now, it's growing, the environmental impacts are growing. But the thing is, generative AI really kicked things into overdrive. And so when these promises, when these goals were made, generative AI wasn't really a thing. And so, of course, this is really blew it out of the park because nowadays GenAI is being put into everything and anything. And so of course it comes with energy costs, but also, unsurprisingly, most countries and companies don't really meet their climate targets. There's a lot of fanfare when they're put forward. There's a lot of kind of greenwashing or signaling. And then when it comes to the actual date, it's like: Oh, well, we overshot it or, oh, well, things changed or we didn't consider this, or we're going to change the way we calculate this. And so this is actually part of like normal ESG reporting. PM: What you say is exactly what Microsoft said afterward, because when they announced their big carbon neutrality pledge, they called it a carbon moonshot. They were going to hit this big goal by 2030 of having negative carbon emissions. And then after this news came out last week or whenever it was, Brad Smith, the president of Microsoft, said "The moon is five times as far away as it was in 2020 if you just think of our own forecast for the expansion of AI and its electrical needs." Okay, this moonshot we were planning is really a lot more difficult to achieve because our business goals have changed, and now we need to race and try to capture this AI market in the cloud opportunities that it offers as a result. SL: And also companies like Microsoft are also not only doing in-house AI and generative AI, but they're also providing compute not only to OpenAI but to customers above and beyond that. So I think that they're in a particular pickle because they're stuffing it into Word or Office, and they have to provide the compute to OpenAI and their training. And also customers want to jump onto the bandwagon. So I think that they're particularly in a corner and, and the moon being five times further away is really accurate. PM: And when you say in a corner, you mean with how they have to report their emissions and what they're doing on the climate front business wise, they're like: This is great for us; we love it; we're expanding our businesses. SL: Well, they're in a corner because I think that shit will hit the fan at some point because they need to build more data centers. They need to have more power purchase agreements if they want to at least keep some semblance of. The power purchase agreements is essentially to buy renewable energy with actual specific providers. I have a solar farm and I can sign a power purchase agreement with a data center. And so I think that they have a lot more targets; they have a lot more things they have to do now in order to keep this machine turning, because it's not only up to them, right? It's also up to the customers. And because, of course, they're doing so well, they can't all of a sudden be like: No, we can't give you guys more in GPUs because we're out of data centers. That would be pretty catastrophic. So it's this self-fulfilling prophecy that that's putting them in a hotspot. PM: You have to wonder as well, obviously we've been talking about this AI moment as a bubble or a boom, knowing that there's going to be a moment when things come back down to earth. And you have to wonder what that does to these ambitions that a company like Microsoft has once that period shifts and we head back down toward a more normal discussion of AI or another AI winter or something like that, rather than like all the hype and the excitement that we've had for the past year and a half or so. SL: I'm just worried that once all the dust settles, _if_ the dust settles, if there's no new paradigm that gets invented in the meantime, that we're going to look back and be like: Oh, oops. That was a lot more carbon than we expected. And I mean, historically as a species, we have a tendency to retroactively look back and be like: Oh, this was worse than for the planet than we expected. And I'm afraid with AI it's not because when I have these conversations with people who work at these companies, they're like, well, first of all, for example, we don't really know what people are running on our data centers. And so we just provide the compute and they could be doing whatever, and we don't really necessarily track energy use and just high level, but not process wise and things like that. And so when I'm like: Give me some numbers. They're like: We don't have the numbers, and I'm worried that by the time they get the numbers, we're gonna be like: Whoa, this is even worse than we thought. PM: And, this moment with AI, I feel like it really makes you think of the discussions that we were having a few years ago about Bitcoin and cryptocurrency and how much energy and how much computation they were requiring for their proof of work, like methods of processing these transactions and whatnot, and how there was a lot of data center power. There was a lot of computation. There were a lot of GPUs that were needed in order to power all those things. And it feels like we've moved from recognizing that, okay, this was a real problem. There were a lot of emissions that were associated with that, that we didn't feel was worth it for what these technologies were really providing. And now we've entered this stage where we have these generative AI tools that these companies want to be everywhere. There's a different aspect to it where with the cryptocurrency stuff, it was genuinely newer companies, companies that were starting up they were not the established firms that we have today, whereas with generative AI, sure, there's the open AI and there's the anthropics and things like that. But Google and Microsoft and Amazon are key in that whole ecosystem and not just driving it, but benefiting from it when you think of their data centers and stuff as well? SL: Yeah. And I think that also, as opposed to Bitcoin, Bitcoin is relatively contained. I want to say that at least you can figure out for example, usually it's concentrated or you have some idea of how much energy it's using, but AI is literally, they have the Internet of Things. You have cell phones, you have all sorts you have on device, you have on cloud, you have so many different components that I don't think we're getting the whole picture and also companies that use AI. When you talk to just your average small or medium enterprise, they're like: We have a chat bot now, or we have this, we have that. They don't necessarily take those numbers into account when they're doing their like carbon accounting or energy accounting. And when you talk to them about it and they're like: Oh yeah, we switched our good old fashioned AI search system to generative AI. And I'm like: okay, well, is there like some numbers that you're getting? And they're like: no, like, we're just paying our AWS bills or Azure bills. And then when you really tell them that this is not just like free compute, ephemeral compute, they're legitimately surprised because they also have these ESG goals. And they're actually accounting for any of this stuff when it comes to their ESG goals, even in the future. PM: It's worrying, but it's not surprising. I wanted to kind of shift our conversation a little bit to understand a bit more about how this generative AI moment is really driving like energy use and growing emissions. So how do these generative AI models actually use so much energy? Why is that? SL: So if you think about it, fundamentally speaking, if you compare a system that uses I guess extractive AI or good old fashioned AI in order to get you answers, for example, to search the internet and find you an answer to your question, it's essentially what it's doing is that it's converting all these documents, all these like web pages from words to numbers, and vectorizing them essentially. And when you're searching for a query, like what's the capital of Canada, it will also convert that query into numbers. Using the same system and then matching numbers is like super efficient. So if you're trying to find essentially like similarity, cosine similarity, like it's just like this stuff goes really, really fast. Actually, my very first job, I was helping like develop these kinds of systems. And it uses no compute at all. It can run on your laptop, it can run anywhere. But if you're using generative AI for that same task, so like finding what the capital of Canada is, instead of finding existing text numbers, it's actually generating the text from scratch. And I guess the "advantage" is that instead of just getting Ottawa, you'll get like maybe a full sentence, like the capital of Canada is Ottawa. But on the flip side, it's actually generating like the model, the AI model is generating each one of these words sequentially. And so like each bit of that uses compute. And so the longer the sentence, the output, the more compute it uses. And when you think about it for tasks, especially like question answering, finding information on the internet, you don't need to make stuff up from scratch. You don't need to generate things. You need to extract things. So I think fundamentally speaking, what bothers me is that we're switching from extractive to generative AI for tasks that are not meant for that, just fundamentally speaking. And of course there's hallucinations and how many rocks a day do you need to eat and whatnot, but also the high energy costs. So for me, none of this makes sense. PM: Exactly. As you were describing, the generating versus coming up with what's there. That's exactly what I was thinking of too. When I'm trying to figure out, say the capital of Canada or something, I don't want the search engine to start talking about the glue that is gonna go in my pizza or something like that. Just give me the capital. Just respond to my query with something that's already there. With these search results. Why do you need to start making stuff up and writing these paragraphs just because it's in vogue right now? And the thing that a lot of these companies are pushing is the next big thing when it doesn't provide the utility that we're actually looking for. SL: And nowadays, when you search Google, you have these AI summaries and a lot of the time they're false, but also they won't show you the actual references. Often, maybe for the capital of Canada is more basic, but often if I have a more specific question, like what is the energy use of data centers worldwide? I want numbers. I want citations. I want the receipts. And then if there's this little resume, first of all, you particularly don't trust them. So, they could be putting thing is generative AI is essentially based on probabilities. And so it's going to generate texts that's very plausible, that sounds really well, but it's not necessarily true in itself. There's no concept of underlying truth to all this. And so on the one hand you do have this high energy cost, but on the other hand, just information wise, it's not, it doesn't make sense for a web search particularly. PM: And that makes a lot of sense, because it's not what you're really looking for when especially you're trying to do a web search or something like that. I'm wondering as well, we talk a lot about the models, but also the use cases for these AI tools, right? On the one hand, you need to train these big models that are going to actually do the work of generating this text or generating images or whatever. And then you have the actual use case where you you go to ChatGPT and ask it to generate something, or, you have Google churn out one of these AI overviews, or you go to DALL-E and have it turn out an image. What is the difference there between training the model itself and then the actual using of the tool once that model has been trained? SL: So it's interesting because initially, most of the studies about energy usage and carbon emissions were really based on training because everyone's like, well, so training typically, for example, for a large language model, that's like billions of parameters. It's parallelized, right? So it's going to be a thousand GPUs for something like a month or three months sometimes. So it really does add up. It can be a year in total of compute if you actually used one GPU. And so people were really focusing on training because it seemed like the elephant in the room. But then nowadays, less and less people can actually afford to train these models, these large language models, but more and more people want to use them. And so now we're thinking about deployment more and more. And so in the recent study that I led, we were really comparing the energy consumption of training versus inference. And depending on the size of the model, it would, it was between like 200 to 500 million queries, which seems like a lot. But if you think about it, ChatGPT gets tens of millions of queries per day. So within a couple of weeks, depending on the size of the underlying model, you'll have used as much energy as training the thing. PM: And so I guess initially, especially when these tools were more novel and not as many people were using them, the focus was more on what was going into actually training this large model, this general model that was doing a lot of things. But now that it's actually out into the world, And millions of people, if not billions, are starting to interact with these things and use them regularly. And they're being built into so much of the infrastructure of, say, the platforms that we use when we go online. Now, the actual using of these tools is the thing that is providing the worrying compute demands and energy usage and all these sorts of things, right? SL: 100%. And also, what I really worry about is really the delta. So for example, when you're switching between, I don't know, a good old fashioned extractive AI model to a generative one, how many times more energy are you using? And in that study, we found that for example, for question answering, it was like 30 times more energy, for the same task for answering a question. And so what I really think about is the fact that so many tools are being switched out to generative AI. What kind of cost does that have? And we don't really see those numbers. Someone recently was like: Oh, I don't even use my calculator anymore. I just use ChatGPT and I'm like: Well, that's probably like 50,000 times more energy! I don't have the actual number, but a solar powered calculator versus this huge large language model. So that's what keeps me up at night is this really the switch in all these different - nowadays people are like: I'm not even gonna search the web, I'm going to ask ChatGPT; I'm not going to use a calculator, all of that, what the cost to the planet is. PM: That's wild. Especially when you think these large language models and ChatGPT are not really designed to do math either. Is your math even going to be right? SL: I know, I know, I know! But it's the thing, it's marketed as these general purpose technologies. There's actually this very confusing paper that's called "GPTs or GPTs?" And so GPT, in the OpenAI sense, it stands for Generative Pre-trained Transformer. And GPT in that paper also stands for General Purpose Technology. Anyway, it's this paper that uses those two acronyms interchangeably to say that the transformers are general purpose technologies that can do anything and the point is just, essentially, you can use them for answering questions and making recipes and math and whatever and answering your homework for example. There's this push to say that they can do anything without any kind of transparency with regards to the cost. PM: That 's wild, when you think about the energy costs of these things and what actually goes into making them all the data that they're also trained on because we know that for a lot of these companies, it depends on having these vast stores of data that they've taken off of the Internet in order to train these large, as you say, general models that where the assumption is they can do virtually anything. Is that a consideration in this as well? SL: So, there's really little study that has been done on the trade-off between the amount of data you have and how much compute it uses. There's these general kind of, they call them scaling laws and essentially they've been driving especially the language side of AI, but actually images too. The more data you have, the better your model will be. And then there's been a couple of studies that showed this for specific use cases. And so we've been pursuing this paradigm, the bigger is better paradigm for five years now, essentially. No one really even questions it. And I feel like it's bigger is better in terms of data. The more data you can get your hands on, the better your model will be. But also for model size, people literally would be like: Let's add a billion or 10 billion parameters just because it's definitely gonna be better. It's definitely gonna beat the benchmark. So there's this general rat race with regards to size and machine learning. PM: And I feel like that plays into the conversation that people have been having over the past year or so when open AI releases new versions of ChatGPT and people are like: Is this even better? I feel like it doesn't even work as good as it did before. There's a really calling into question whether this idea of bigger is better really makes sense and delivers, you know, real benefits in the long run that most people can see or notice, or even if it happens at all. SL: And how we evaluate these models is so broken. Most of the time, these benchmarks they're most likely based off of data that's probably in the training set somewhere. And so, for example, it's like: Oh, there's this benchmark that's, I don't know, whatever the bar exam, the whatever the New York state bar exam. And then it was like: Oh, ChatGPT passed the bar. Okay. But the chances of that data being somewhere in GPT-whatever's training corpus are really high because probably there's some books and study PDFs and whatever on the internet. And then there's absolutely no due diligence that's being done. You test your model and it passes all these benchmarks and you're like: Look, it's awesome; it reaches human performance. And then there's no cause effect situation. You don't try to figure out why that is. Is it because it actually learn some useful patterns or some logic or because it just memorized the training data? PM: I remember being so frustrated by those stories because it's not even just an open book exam, all the answers are written next to the exam for you to copy over! But that bleeds into the misconceptions that can exist around these things based on the way that the companies talk about them and promote them in the way that then the media repeats those narratives in the public ingest it, because they don't have a technical understanding of what is actually going on here. SL: Exactly. And where the emphasis is as well, on the one hand, you've got like, this is going to solve everything. This is going to do everything. And on the other hand, well, this is potentially going to wipe out humanity. But then none of that conversation is ever around what are the labor costs? What are the environmental costs? For example, when ChatGPT really came out, and, I read the report, or however they call it, what struck me is that the paradigm didn't shift. The actual, large language model paradigm, transformers, was just the same, but the amount of human labor, the amount of crowdsourcing that went into that, was just something that nobody ever did before. And of course, it's hard to say whether that was the magic ingredient, the secret sauce, but that was really, for me, from a technical perspective, that was the difference with regards to other previous generations of large language models is that nobody spent, I think it was like tens of thousands of hours or hundreds of thousands of hours into improving the model using direct human interaction. Because mostly it was like, you take this data from the internet and you train this model in a computational way, and then you put it out there. But whereas they took months and months to actually, hire these people and ask them to improve the model, really on a interactional basis. And then for me, that was the big thing. And then no one really talked about it. They were like: Oh, ChatGPT came out. It's the best language model ever. Blah, blah, blah. So many things get swept under the rug. PM: I feel like the first inklings I got of that were the Time story, I think it was in January of 2023 that talked about the workers in Kenya. And then that started to get it into the conversation a bit more, but it was still among people like us who are really paying attention to this rather than, the much wider public and the broader discussion that's happening around these technologies. SL: Exactly. The level of misdirection that's happening is quite impressive. I guess it's part of the narrative, but I really feel like for each person who talks about the actual costs, there's 50 people who are like: This is going to revolutionize whatever it is you do; it's going to change it. You should be using these tools. Why aren't you doing it already? PM: And as you were talking about that too, I was thinking, have you mentioned that a lot of the discussion around this, that the company who was really pushing out there— OpenAI, because it was beneficial to them — was this question of is generative AI going to be this big threat to us because it might become intelligent and then, this threat to humanity, if it wants to take us over and kill us? And, what if the threat that we were talking about was how much energy and compute that these things required and how much emissions that was going to generate as a result. But that's not the threat to humanity type of discussion that we want to have. It's more fun to have this sci-fi scenario that also directs our attention somewhere away from the real regulatory questions, potentially, that we should be looking at. SL: Well, actually the existential risk or the long term risk link with climate change is really interesting because from that particular point of view, climate change is not an existential risk because it won't kill everybody essentially. So, quote "it's okay," because rich Global North people are probably going to be able to build barrages or protect themselves or have bunkers or whatnot. My biggest frustration with this relationship between long term risk or existential risk and climate change is that it's not really considered an existential risk because a lot of the Global North and richer countries are technically going to be fine. And so when you go on these websites, they say that the existential risk of climate change is close to none, close to a non existent compared to AI. So AI could technically really literally wipe everyone out. But whereas climate change, most of the rich white people are going to be fine. And that's just such a privileged take that just really makes my skin crawl. PM: Totally. I read William McCaskill's book and I was looking back over some of these things recently for something I was putting together. And there's even this thing that, that he has written about in the past. It was in a piece that Emile Torres wrote about all this stuff where McCaskill was basically arguing that even 15 degrees of warming probably wouldn't eliminate all agriculture. So, we, as a species, would be okay. What are you talking about? \[laughs\]. SL: And so I think it really feeds into this whole discourse, we should only be focusing on AGI. We shouldn't be talking about climate change at all, or AI's climate impacts. And every time I'm like: How is this even logical? We're already going through the hottest summer for so many people, we're already going through all of these things. How are we not focusing on the tangible, here and now? Why are we talking about this years into the future potential global annihilation situation? I don't know. Maybe I'm too down to earth to really buy into this long-term risk stuff. But for me, it's a no brainer. PM: How dare you think about real life and not what's going to happen a million years from now? \[laughs\]. SL: But eventually going to happen. And then you have people like Sam Altman who are like: Oh, what we need to power AI is an energy revolution. I'm very conveniently investing in this nuclear energy company that's going to come along and solve all these problems. So we shouldn't even be talking about the energy usage because this nuclear fusion or whatever startup is going to just solve that anyway. So, there's so much pushback to all this. PM: And then if my nuclear energy company that I'm investing in can't come up with this energy revolution in time, then, shrug, we'll just geo engineer the planet to make sure that we can roll out all these AI tools? SL: Exactly. And there's always some technological fix somewhere very soon though, cause that's the thing it's imminent. We're going to fix this problem imminently. So let's just not even talk about it. And there's a lot of that in especially the tech industry circles or the conferences and stuff that I go to, there's always like: Why are we even talking about this? We should be talking about streaming. How about the environmental footprint of streaming or Bitcoin or whatever? We should be talking about those like AI is not an issue. PM: We do talk about those and those were bigger things when they were what was really driving it. But generative AI is what's really driving it now. And that's why we're talking about it. And I feel like when you talk about those views that you hear at conferences, it helps when they come from some of the most powerful and influential people in the tech industry, like Bill Gates, for example, who just a couple weeks ago was saying that governments should not be concerned with AI energy use, because that's when we develop good enough AI tools, they will just solve their own climate problems because they're going to get so good at this kind of stuff. And again, the techno fixes, but also don't worry about what we're doing and specifically what Microsoft is doing because Bill Gates is still very involved in Microsoft's AI strategy. Let us do whatever in the tech will solve itself. SL: And I mean, it's true that there are some real problems, climate related problems that AI can help with, but the most successful use cases I've seen is really when you take AI as part of a solution and a relatively smart, small part of the solution, and then the rest of it is domain expertise, people who actually know what they're doing here. And then they take AI and they use it to improve their existing processes, but it's not a: Oh, we had no idea what we were doing before, and then AI came along and solve the whole thing for us. What really bothers me is this whole perception of AI as an actual tool, as a standalone tool that will solve climate change, because that's just false. And, sometimes when I talk about environmental footprint, they're like: Oh, well, AI is helping improve climate modeling or whatever. And you're like: Yeah, but that doesn't actually change the fact that the energy demands are growing disproportionately to any efficiency gains you can get from AI in climate modeling, for example. It's two separate discussions, really. PM: I wanted to reposition our conversation to dig a bit more into this question of the types of models, because I feel like this discussion that we're having is based on this general idea that we need these models that can try to do everything that are trained on every possible bit of information that we can possibly access to create these generalized models that say the open eyes and the Googles and Microsoft and stuff are putting out into the world. But then there's also this other part of this where people are arguing that we don't need these general models, what we need is smaller, tailored models to the specific use case that we're trying to do or to take care of, and those would actually work better. While having less energy user or computational demand than these general models that these major tech companies are pushing. Is there anything to that argument? What do you think of that? SL: So I think that the argument holds if you're one of these companies like OpenAI or Google or Microsoft eventually like that, you have all these very generic tools, like ChatGPT for example, it's a tool that's technically meant to do all these things. So maybe for a specific tool like that, if that's what you're trying to do, then having this general purpose model makes sense. But the vast majority of applied AI use cases are very specific problems. I've worked in finance; I've worked in automotive AI. Essentially you need to do something and then you're going to train an AI model or use an existing model to do that particular thing. I've never heard of a client or whatever, being like: We want to do everything. We need a model that does it all. And so I think that if you're coming at it from an actual user perspective, you need very specific models, not only to a task, but also often to a specific domain. Not only do you want to answer questions, you want to answer questions in finance. And it doesn't make sense to use this general purpose model because, for example, there are words that have specific meanings in fields like, I don't know, whatever bull and bear and whatnot. You want your model to be specifically catered towards a very specific use case. And so I think that this discourse of like: Oh, you need these general purpose models is once again, part of the marketing spiel around generative AI, because at the end of the day, these models have to be making money at some point. So you want to be telling people that they should be using them because they'll pay for themselves. But if you really think about it, what you need in most cases are tasks to specific models. PM: And how much is this push for these general models that do everything? How much is that related to, on the one hand, the companies, these major companies, like the Google's and Microsoft's, trying to create this situation or market dynamic where it's only major players like them who can really compete on this scale versus how much is it motivated by, as you were saying with the longtermist stuff, this desire to try to create the machine that thinks like a human and is the artificial general intelligence and sort of those sorts of ideas? How much the both of those motivations play into this? SL: That's a really great point, because it's true that in the 10 years that I've been working in AI, the barrier to entry has shifted so significantly. At the beginning, if you were a grad student, you could train a model on your laptop and submit it to one of the big conferences like NeurIPS or, or I don't know, like ICML or whatnot. And then, if the math or the computer science part was sound, you could get accepted nowadays, not only do you have to compete with this bigger is better paradigm in terms of like: Oh, well, someone trained a 70 billion parameter model. So you should be training an 80 billion parameter one. There's also this benchmarking aspect of you have to run previous people's models in order to prove that yours is better. So the onus is on you to say, well, compare all of these, you need to compute for that anyway. So not only now graduate students are having trouble, we're seeing more and more academic institutions partnering up with big tech, because they need those compute grants. They need, essentially, those GPUs. And so that particular dynamic has become very skewed. And a lot of researchers will have a double affiliation with academia and industry just to be able to compete, because essentially you need to publish papers; you need to write. So that part has become very weird. And also, as you said, the companies that can afford to really train these models from scratch are such a select few that, They're monopolizing the industry. And if everyone wants to use an LLM and there's only a couple of organizations that train LLMs, then that's who you're going to be going with. So I definitely think that there's a concentration of power that's getting worse and worse. And the environmental impacts of that are part of that narrative, but not being transparent, essentially. PM: You even see that with how these deals are playing out where a lot of OpenAI's deal with Microsoft is related to cloud credit so that it can use the Azure servers and stuff like that for its training and running its models and its services and all these sorts of things and of course any of the rising AI companies like Anthropic, for example, basically are required to get these massive investments from Google or Microsoft or Amazon or Meta to compete in the space. SL: And then everybody is reliant upon NVIDIA because they are the bottleneck of all of this and if you even want to build your own, even assuming that you have the incredible amount of money needed in order to build your own cluster, you still need to get access to NVIDIA GPUs, and there's a real bottleneck there and a wait list. So also that particular concentration of power, there's literally one company that's making all the hardware for training AI models. PM: And how does NVIDIA then play into this bigger picture, when we're talking about the climate costs and the environmental costs of making these major models, you need the graphics processing units from NVIDIA in order to do that? Or it is the major supplier and we've seen its share price go crazy as a result of this whole moment. What's your thought on how NVIDIA fits into this bigger picture? SL: I'm really frustrated because I've been talking to them for years at this point and asking for more transparency because for a lot of companies, even like Apple and Intel, they create lifecycle assessments or carbon footprint assessments of their hardware. I can't say it's simple, but it's straightforward. There are methodologies; people have been doing this for a while. And essentially you take a look at your supply chain, you ask your suppliers. Because, for example, Apple won't manufacture their hardware in house as well. Neither does NVIDIA actually, they all always outsource this to their suppliers. And so essentially you ask your suppliers to quantify with the life cycle assessment methodology, how much energy they're using, how much water, transportation, raw earth metals, blah, blah, blah. You have all of these; it's a well defined methodology. And then once you have those numbers, you integrate that into your Scope 3 emissions that your supply chain emissions, and actually for Apple and Intel and stuff for specific pieces of hardware, they will have a number in terms of CO2 emissions. And, if you buy a MacBook Pro, it's this much CO2. And so NVIDIA, I've been trying to get those numbers for them for years. And they say that the supply chain is complex. Sure, as is the case for a lot of hardware. But there's a single supplier that makes all of NVIDIA GPUs and they're based in Taiwan. And, even if we don't have the raw numbers Taiwan's electricity is a hundred percent coal based, which is the most polluting electricity type. So we know that that already is polluting. The intense amount of energy needed to create this hardware is essentially a huge problem. Also pure water, because when you're creating these chips, you need to purify every tiny little layer of silicone, and now nano nanometers of silicone essentially. So they're using crazy amounts of water. And to the extent that there were some droughts in Taiwan in recent years, and the government actually told farmers not to plant crops so that the water goes towards the fabs, the fabrication of hardware, because it's become such a geopolitical issue. So there's energy, there's water. And then there's the metals. There's all sorts of rare earth metals that are mined in terrible conditions. And also the amount of earth that you have to move in order to get one gram of a rare earth metal is like one ton for one gram essentially in some cases, and the labor conditions and the human rights conditions are terrible. So all of this, we don't have any transparency on any of that and people are buying thousands of GPUs in order to participate in the AI rat race and that's not getting accounted for anywhere. PM: That's so wild. I honestly didn't even realize the whole scale of that picture. And when you talk about Taiwan basically saying like: Okay, don't plant your crops because we need the water to go to the chip manufacturers, it doesn't surprise me when you think about how important it is for that small island country to have this key industry and to make sure it's still working properly because it's part of its protection from the West. SL: Exactly. And it's really a geopolitical issue. The US president signed an order limiting how many GPUs can be sold. There's all these geopolitical aspects, but also nowadays, since chips are such a core component, not only of AI, but of cars and TVs and everything. . And so just all of that is so complex and the fact that Taiwan is close to China and there's, there's all this stuff to unpack there. But essentially for me, it's really the environmental side of things. And the fact that there's no excuse of there's such a complicated supply chain. It's one supplier. You can get the numbers from them, even if It can take some pushing because maybe this is not something that they do inherently. But if you're a company like Nvidia, that essentially is the most expensive company in the world or whatever as of recently. So if you're that kind of company, you have enormous market pressure on whoever your suppliers are and you can tell them: Give me the life cycle assessment of the GPUs that you're making for me or else, whatever. And then they'll do it There's a huge disbalance of power. So the fact that NVIDIA hasn't published a single carbon or life cycle assessment for me is really a glaring omission in the grand scheme of things when it comes to AI. PM: It tells you they probably just don't want those numbers out there because they could get them if they wanted them. SL: Exactly. PM: And then, of course, when we have these discussions about the broader impacts and the climate impacts of generative AI, that's a piece of the picture that is not into this broader discussion that we're having or these figures that we're using, because it's not transparent. It's difficult to get those numbers. SL: So a couple of years ago, I was working on the big science project, and that's actually how I joined my current employer, Hugging Face. So essentially it was the first time that a large language model was trained in a community way. And it brought together a thousand researchers from around the world. And we got compute from a public compute cluster in France. And then I was responsible for the carbon accounting part of things. And then I started thinking, well, if you look at life cycle assessment, if you buy a pair of jeans or a tote bag or anything, you can get the car, the life cycle assessment that goes from the cotton that was used and how much water that was used and the, the transportation. It will cover all of the steps of the life cycle. And that's how we typically think about products nowadays. It's cradle to the grave. And so for AI, no one really thought about it that way before, because everyone was so focused on training. And so we did this life cycle assessment and, we looked at not only, for example, the GPU energy consumption, but also the whole data center and the overhead and of all of that. And we found that just for energy, the GPU is only half of the overall energy used for training models. So any numbers we had until then we could multiply by two and those were the real numbers. And then when we started looking at the manufacturing process, that's when I reached out to NVIDIA, this was three years ago. And then I realized that they haven't published any numbers. And the only numbers we could get were from the actual data center hardware, like the nuts and bolts of network adapters and cables and all of that. You have that, especially for a public compute cluster like the one we were using. They had those numbers because they're obliged to to have them for transparency reasons but for GPUs there was not a single validated number anywhere. And that really blew my mind. PM: That's so wild. I do want to go back to one other part of the question that I asked that we didn't get to, which is, we talked about the business angle of this and how much the companies want to make these large general models to try to reduce competition because they are the only ones who have the compute necessary to train them and to operate them and things like that. But then there's the other piece where a lot of these figures who are influential in this space talk a lot about artificial general intelligence and wanting to achieve, the computer that thinks like a human, basically. Do you think that that is part of the motivation for pursuing these general models or does that just provide an ideological or rhetorical justification for having done it? SL: I personally think that it's mostly providing a distraction, also a way to avoid regulation or scrutiny because it's, well, if I'm single-handedly building AGI, artificial general intelligence, then first of all, you can't really regulate me. I have all this power, and who cares what the cost is, if this is what we're achieving? And it's also a way of contributing towards all these other problems will solve themselves, the energy usage or the labor costs or whatever. Because once we have AGI, none of these things are going to be problems. And so I think it really is a contribution towards shifting the focus. And, and also if you really believe in this, why would you want to stop someone from solving all the world's problems? Whatever it takes, they should have it. It's definitely really sad because you hear a lot of that when you talk about those costs that essentially get swept under the rug. When you bring them up, they dismiss it by saying: Oh, it doesn't matter, because we're pursuing AGI. And now I feel that before it was mostly more corporate marketing people, but now even researchers have this. Before I would ask people to be a bit more transparent when it comes to the compute cost of their papers and the stuff that they published in the AI space. And now I ask them and they essentially have the same discourse saying: Doesn't matter; we're working on AGI. And for me as a scientist, I find that really shocking because you have scientific responsibility. You should be transparent about the cost of your work. That's part of being a scientist. That's part of being a researcher. And the fact that that's percolated to that level really worries me. PM: You wonder if it's a really deeply held belief or it's the way to get funding is to talk in this way. SL: I guess. And also, when you look at generally the state of transparency, a couple of years ago, before ChatGPT, honestly, you'd have some form of ballpark numbers. People would be like: Oh, well we trained on whatever, 2,000 hours on these kinds of GPUs or TPUs. Sometimes the Google Tensor Processing Units; they have their own hardware anyway. They were relatively transparent. It wasn't necessarily a secret. And nowadays, if you read these papers, especially the more industry lab ones, they have no information about how they train the model, where the data was coming from, how much compute they used. All of this information doesn't make it out there anymore because of this secrecy or rat race around GenAI. And I feel that from a fundamental perspective, that's problematic. That's not how you do science. PM: Exactly. It's all trade secrets now. Nobody can know what we're doing because we also just don't want you to know. SL: Exactly. But when you really poke at it, just by saying how much compute you used, it doesn't really give you any insights about your model as such. That's what really bothers me. I worked in corporate settings. There's this, like: Feed them shit and keep them in the dark approach. But that's a corporate approach. Whereas, even if I gave you a ballpark where my data was coming from and ballpark, how big my model was and how much compute I used, you still could not be able to reverse engineer my large language model, that's impossible. You need the code; you need the actual data; you need all these components. So the fact that as a discipline, we were going in that direction really shows that now the corporate narrative has fully taken over AI, at least in the large language model space. And that's the direction we're heading. PM: It's very concerning, but not wholly surprising. It makes me think of a few years ago, when The Dalles in Oregon was fighting Google to try to find out how much water its data centers they were using and for a year it fought the city, basically, in court the local publication that was trying to get these numbers from the city. SL: And they were funding the legal fees. PM: Exactly, it was funding the legal fees of the city to not release the data even though data centers nearby from Amazon and Apple were releasing those figures, and we don't know what the problem is, but Google did not want to share it. And then after a year, when the public backlash and stuff got large enough, it finally said: Okay, here's some numbers. But why don't you just tell us how much water you're using? It's not this big deal. We're not going to know this crazy thing about the data center and your business practices, because we know that. It should just be easily available information that the public has a right to know, not this crazy thing that the companies are trying to hide just because they can. SL: Well, I think that also the problem is that Big Tech has had a relatively good climate reputation an environmental reputation. What's interesting is if you look at it from a really a global perspective, Google and Microsoft are are some of the world's largest purchasers of renewable energy. For years and years when you read the ESG reports — I have a thing for reading ESG reports — and it's really like we're at the forefront; we're investing in this research. They have tons of people working on climate modeling and things like that. So, there has been this emphasis on the positive side of things. And so I think that when it comes to actually opening up their books and showing us the cost of all that, that potentially questions that narrative of: Oh, we're doing all this good work to stop climate change or to protect our planet. And I think that's where things become complicated when you actually ask the cost of all that. In general, I feel it nowadays from different angles, whereas an environment or labor or whatever, the cost of AI to society is really not part of the discourse. It's really the emphasis is on the benefits are not the costs, and that's not a good way to think about innovation. PM: It's interesting, because I feel like the places where that starts to seep into the conversation is when you have these moments where these communities start pushing back against these data centers, especially in this moment where the major companies are continuing to and accelerating this build out of data centers around the world. I was really struck by a figure that Microsoft gave earlier this year when they released in a report that they had five gigawatts of installed server capacity at the beginning of this year. But in the first half of the year, they wanted to add an additional gigawatt on top of that. And in the first half of 2025, they wanted to add another 1.5 gigawatts on top of that. So this acceleration of what they have versus what they're adding in each new, say, six month period or whatever is just continuing to scale up. And you have to wonder on the one hand, do they really need that much server capacity? But then on the other hand, is there going to be this much demand once the hype around generative AI starts to fall again and what happens there? I feel like it's hard to understand. But the one thing that's very clear is that their energy demands, their computational demands, their water demands are not going anywhere and are only going to continue rising in the years to come. SL: Exactly. And also as a provider of services, they are also, potentially the way I've been, I've been trying to present this to people is that instead of seeing AI as a vertical. So if you talk to climate folks, they tend to think in terms of verticals. The IPCC will talk about agriculture and transportation. And you have these verticals that have been part of the discourse for decades now. And AI is typically put in the ICT vertical, the Information Communication Technology vertical with phones and internet and stuff like that. But I think that it's more of a horizontal and it can impact agriculture. It can impact transportation. That's the thing. And then we don't really know what the impact will be. Maybe it will make some things more efficient and make the emissions of the whole sector go down as a result of that. But I think what we're seeing so far is that it's actually contributing towards amplifying whatever the existing emissions of that sector are in many cases. So you're making something more efficient, but people are using more of it just because now generative AI is in all the tools. They were going to integrate generative AI into Google Maps and how many people use Google maps? And so it will make the overall, emissions of that sector, which includes the navigation part of it, bigger because now we have GenAI. PM: I have a couple final questions to close this off. I wonder on the personal side of things, do you think that there's anything people should be keeping in mind when they think about generative AI and interacting with these sort of tools? Or do you think it's a bit more of a question of what these major corporations are doing versus what the individual is, is doing and, interacting with these things? SL: I generally feel that in the context of climate change, we've put a lot of pressure and emphasis on individual action, which is definitely part of it. But, the concept of carbon footprints was invented by British petroleum. It was a way to shift the responsibility on individuals, and not on oil and gas companies, for example. And I think that with AI, it's a little bit similar. We, of course, can have some level of agency and we don't need to use ChatGPT as a calculator, for example, but it's unfair to tell people to stop using AI because of its climate impacts. I think it's really the pressure or the responsibility should be put more on the providers of these tools. For example, in order to provide the energy or the carbon that's linked to, for example, a Google search or, or an interaction with ChatGPT. And in order, for people to be able to make their decisions based on that. But I think the first point of action should really be on providers and on the people making these technologies, because if you have existing structures where, for example, if you're living in a rural place and you don't have public transportation, you really have little choice than to buy a car. And it's similar with AI. If you're already in these structures and every time you do a Google search, it uses generative AI, you're boxed into this already. Of course you can use Ecosia, for example, which is a search engine that is actually carbon neutral. But other than that, you're using Gmail as part of your job and you're using whatever as part of your personal life. And so you're already stuck in these systems. And so I would put the pressure more on the providers of the systems than the individuals themselves. PM: I definitely agree with you. And, I think listeners of the show will know that there are certain tech products that I don't use. I don't use Uber and I don't use the food delivery apps. And I've actually never used ChatGPT to go onto the website and use it. But I definitely don't think I'm saving the world by doing that or anything, because ultimately, the question is what these companies are doing and how they're being forced to change on a broader, structural level versus what us as individuals are doing. And so I guess the flip side of that then is, we talk about individuals and that this isn't inherently an individual issue. So then when you're looking at what these companies are doing, as you were saying for a long time, I think it was fair to say that these tech companies did develop a reputation for being more environmentally conscious or climate conscious than some of the other major companies out there. And I think that they put a lot of emphasis on trying to make sure that they were seen that way, by say, making pledges to be net zero or carbon neutral by buying a lot of renewable energy by making sure they were buying carbon credits to say that they were replacing their emissions or whatever. And obviously there are big questions about carbon credit and offset schemes and things like that. But what is seeing their response to this AI moment and how quickly those things have been pushed to the side and how eager they have been to chase after this regardless of the impacts. What has that told you or what have you been thinking about their commitment to climate action as a result of what has been happening over the past year or couple years? SL: I think that capitalism in general is at odds with sustainability. If you think about it at the end of the day, the point of a company is to make profit for its shareholders or whatnot. And so when it comes to trade off between sustainability and profit, it's why companies exist and CEOs have a responsibility towards the board. That's the mechanism and so you can't really expect them to choose sustainability. But what I do think is that, maybe I'm a little bit naive, but I do think that like this And the energy and the climate impacts of generative AI actually did spiral in a way that even the tech companies didn't expect. And now the question is: What now? And I think there's going to be increased tension between profit and sustainability around AI. But what's interesting is that, currently I don't see the massive improvement that gen AI brings. I see it being sold as such, but I don't really, tangibly see the "big deal." And so I think that maybe, as you say, the AI summer will die down a bit when we realize that maybe energy is going to be part of the conversation, but that it's not living up to the expectations. ChatGPT answers questions, but it also hallucinates and there's essentially so few use cases where it's actually, I guess, irreplaceable. Then I think that that's when it's going to be too expensive to maintain it then to keep using it. And so maybe this is more of a capital versus environment struggle and it's nothing new, honestly. It's part of the last couple of decades, of lack of progress around climate change because money. PM: I think that makes a lot of sense though, because I think that there's often this idea that generative AI is inevitable, that it has to be rolled out everywhere that we have to use it. And I tend to think of, going back to the metaverse moment, there was this push from Meta and from some other companies to do this project that was going to be very computationally intensive by having us all spend a lot more time in these 3D environments. And I think we headed off a real climate disaster with defeating that admittedly very bad idea, which was probably not going to go anywhere anyway. But I think that based on what you described there, what I hear from it is that generative AI is not inevitable. There might be some use cases where it makes sense to be using tools like this, but the idea that it's going to be rolled out everywhere and that we just need to get used to this is very much something that is not baked in right now. And there's still the opportunity to change course and say: Listen, it's okay to use these things in certain instances, but it makes no sense to build them into Google Search and to build them into these infrastructures of the web that we're used to because they don't provide the benefits. SL: And for example, in France, I keep hearing this term of digital sobriety, which I find so refreshing. I've been participating in a couple of discussions or events where people are literally like: Do you really need a new cell phone? Do you really need to be using ChatGPT? I feel like it's questions that we don't really ask ourselves that much, maybe because there's more pressure, there's more marketing or whatnot, but it's already started in Europe, to some extent, this pushback against generative in general as the solution to every problem, and I find that so refreshing because I'm not hearing a lot of that in North America. PM: I love that. Let's all commit to digital sobriety, and, whether you have sobriety, in your real life, what you're drinking, that's totally up to you, but let's at least do our digital part. Sasha, it's been wonderful to talk to you. It's been so enlightening to learn more about this whole space and what's going on with it. Thanks so much for taking the time. SL: So great to talk to you, too.