GitHub CEO Thomas Dohmke on Building Copilot, and the the Future of Software Development

[00:00] the human brain is still so much more advanced than [00:02] than the transformer models and the diffusion models and other types of models that we have to image recognition and whatnot that we have today. And, you know, it remains to be seen if we can kind of like add that sentience piece to it. But today I'm not seeing it and I haven't seen any research that telling me that that's coming anytime soon. [00:23] *outro music* [00:38] *Bell rings* [00:40] Hi everyone, welcome to Training Data. [00:42] Today we host Thomas Domke, CEO of GitHub. Thomas has an ambitious vision for enabling a world of 1 billion developers, and bringing agents through the end-to-end developer workflow, [00:53] and into adjacent categories like code security. [00:56] He even hints at the progress he thinks the industry will make on Sweebench over the next few years. [01:00] some categories he's excited about in AI outside of developer tools, and whether he thinks a new architecture will overtake the transformer. [01:08] Today, we're so excited to introduce our special guest, Thomas Domke, CEO of GitHub. Hey, and thank you so much for having me. [01:16] Thomas, we're so excited to dive into the GitHub and co-pilot story in particular today. Maybe to kick off, we'd love to learn a little bit about your personal background, [01:25] You have a very interesting story having grown up in East Berlin before the wall fell and then starting your first company that brought you into the United States when Microsoft acquired the company.

[01:36] How did your background and upbringing really shape who you are today? [01:40] I think I'm living a very normal life of the American dream now with a wife and two kids. [01:48] my journey was the passion for software development early [01:51] you know, when I was 11 year old or so, it was still East Germany and West Germany, so that was the wall [01:57] between the two parts of Berlin and I saw computers for the first time. I couldn't buy one, but in school we had one in the geography lab and a friend of mine and I, we started [02:07] playing with that, learning to code. You had to code to even do anything with a machine. [02:12] you know you understand some basic to even load a program and then as the wall fell a [02:17] Bought a Commodore 64 and later my first PC, a 386 DX40. Wow. And so as a teenager, I spent most of my time coding and started a company. It wasn't at that time really a startup. It just started insurance software. As in the late 90s, most of the insurance agents didn't have software. Some were working on mainframes and others just had people in front of them. And then I moved to South Germany to work for Mercedes. [02:47] by Microsoft, but it's really this [02:48] passion for doing stuff with software being creative with software and i think the [02:52] Fascinating thing back in the 90s and it's still true today is that you can start very easily. There's not a lot of capital investment required. And if you make a mistake, you can just start from scratch. And I think that's what makes this so cool to build software.

[03:09] And that gave you the love for building and fixing things even today with robotic lawnmowers in your own home. Yes. Well, and it gives me the love, you know, as the CEO of GitHub to build software for software developers. I think that's the really cool thing about it. [03:24] Being at GitHub, we are building the tools that other developers are using. We always say we put the developers first, and that, I think, is the dream job for me. I get to speak with a lot of developers. I get to build software for developers, and I get to speak to many developers here in Silicon Valley. [03:42] I think what many might not know about you outside of Microsoft is that you played a pivotal role [03:48] And, um, [03:50] sponsoring Microsoft's acquisition of GitHub back in 2018. Can you share, take us back to that moment and share a little bit about what was your vision for GitHub back then? Yeah. So in 2018, I was a product manager at Microsoft working for NetFriedman, [04:04] who at the time was the CVP for mobile developer tools and now had the idea of buying a GitHub and making GitHub part of Microsoft and so [04:13] he and a few folks in his team were kind of like strategizing of how can we pull that off and pitch it to Satya and the board. [04:22] deal got announced actually you know about six years away from that at the time of this recording june [04:29] for 2018 is when we announced the deal. I became the deal integration manager, which was the role within Microsoft that runs around the whole company, making sure all the pieces, you know, from legal to HR, finances, and product and engineering come together to get that deal through the regulatory approval, and ultimately, you know,

[04:48] getting us to a successful day zero which then happened in October 2018 and that's [04:53] That's how I ultimately came to GitHub. [04:56] What did you envision GitHub becoming, the potential it had as part of the GitHub universe? [05:02] Mm-hmm. [05:03] I think, you know, at the time we were thinking there's so much potential for GitHub that's still... [05:07] you know, to be explored and to be realized that GitHub [05:11] started with the first commit in late 2007 and launched in 2008 [05:16] very quickly became a new way of developers working together, social coding [05:22] was invented by Chris and the other founders. And it evolved into this two-sided product, the home of open source, where many open source developers were collaborating, and the place where many startups and ultimately enterprises [05:37] We're building their software and often that [05:39] you know, two-sided equation was that the companies wanted to work exactly like open-source developers work, which is boundary-less, you know, in the world of open-source. [05:48] you don't really care where your collaborator sits, where they're from, what their education is. They don't sit in an org chart. And, you know, often you don't even know their real name. All you know is the handle. [05:58] and the code that they want to contribute back to the project. And I think many companies [06:03] Don't have that. You have silos. And when you get an email, you're like, wait, who is this? And why are they emailing me? And why do you... [06:10] Why do you want to be involved in my project? And so companies want to burn down these [06:14] these walled gardens and have a similar collaboration model and that's what

[06:18] GitHub symbolized in 2018, but there was so much more to do, you know, to provide something like GitHub Actions that allows developers not only to manage their source code and plan, but also to build their apps and now ultimately with Copilot to take it even a step further and making... [06:34] development a very different experience than it was 20 years or 30 years ago when I started. [06:40] Since you mentioned co-pilot, I'm dying to ask you about the behind-the-scenes view on what is not only the most successful enterprise AI application today, but... [06:51] I believe the first LLM native application that was really built [06:55] built and launched like I guess was it part of the original acquisition thesis at the time that you might be able to build something like co-pilots eventually whose idea was it to do something like co-pilots and like [07:06] Did everyone say, yes, this is going to work? Or was it like, this is a crazy idea, this is a moonshot, it's never going to work? Take us back to the origin story. So the original acquisition thesis had like a little paragraph in it about AI, but I think that was more like a moonshot story. [07:22] than, like, the proper idea at the time. What really happened in mid-2020 is that, you know, Transformer models, you know, the paper came out a few years before that, but the first, you know, really working Transformer models was GPT-3 was about to launch. We got early access. We were all in lockdown, right, June 2020. On a call, and one of our team members, Ugo DeMoor, started, you know,

[07:52] what Uge would be doing. And then the actual question, what can we get prompted to [07:57] to write code and can it write proper code? And I think that was the first kind of like aha moment that it was actually able [08:04] to write the real syntax. And then we tried different languages and we also found flaws in the model. [08:12] exploring the deeper outset of that Zoom call in a research process, doing analysis, we ask [08:19] some of our staff and principal engineers to submit coding exercises. We looked at [08:24] Python functions that were, you know, in open source repositories on GitHub. And we work with OpenAI to take GPT-3 and fine-tune the model to be better at these coding tasks. And ultimately, in August, we had a model that was able to solve problems [08:38] 92% of these coding exercises. And in fact, you know, of the Python bodies, you know, that we extracted from open source projects, it was like 52%. Now, naturally... [08:49] that percentage is lower because you have less specified code than in the coding exercise for an interview loop, right? But that, I think, gave us this... [08:56] this moment of confidence saying we can [09:00] build a product around this. The second moment I'd say was when we rolled it out [09:04] to our internal engineers in early 2021. And they came back with saying this is, [09:10] This is fantastic. I think the Net Promoter Score NPS was somewhere in the 70s, which is, I think, for developer tools, really, really good. [09:18] Most developers are skeptical, you know, don't touch my system, you know, never.

[09:22] never touched a running system or process and many folks are kind of like [09:26] have created their work setup not only on their physical desks, but also on their virtual desk of how they want to work. And so we were really intrigued by the internal responses. And then we launched the preview, and the team came back and said it's writing 25% [09:44] of Python code in those files where it was enabled. And I think we sent them back, say, go and verify the telemetry. That can't be true, right? Because we couldn't believe it initially. And then as we saw this progressing, I think in my first keynote as CEO was in – [10:01] June 2022 so now you know two years ago and I said it's writing 40% of the code and I think in five years is going to write [10:08] 80% of the code in those files was enabled. [10:11] So it happened really organically. It wasn't like you were sitting in the room whiteboarding like co-pilot is going to be the next. [10:16] iteration, it was like the model proved to you how great it was. And just seeing the model performance and seeing [10:22] usage, like really like it organically built up from... Going back, you know, to the original thesis for GitHub, right? We wanted to make developers' life easier. And as we are company building [10:33] software, we have our own software developers and Microsoft has 70,000 or so of them. And so we understand how software developers work because we have so many of them and we live the life of many of our customers, which is we have way too many ideas. [10:47] we're moving way too slow, you know, at least in our feeling, you know, Amazon delivers my package faster than Amazon.

[10:53] then we're implementing some features. Also, the expectations have... [10:57] shifted significantly, but it's this, the backlog is endless. And the amount of ideas that we can brainstorm on virtual whiteboards, GitHub is a remote company. So we don't often meet in real physical, in front of physical whiteboards. But, and then the other side is all the other work that we also have to do, you know, compliance, security, accessibility, [11:17] enterprise requirements, privacy regulations, [11:21] European, you know, AI Act and Digital Markets Act and all these things also cost developer work. And so we're constantly, everybody's constantly struggling with the length of these backlogs. And so... [11:34] If you will, the intrinsic motivation was, let's bring the effort down to write software. [11:38] and make it joyful again. [11:41] What do you think made Copilot so good at the time and also going forward today? Obviously, GP3 was great even, you know, back then. But also, I think a lot of people might not know all the value that you brought or GitHub brought in both public and proprietary data. [11:57] around code that GitHub owned. [11:59] Can you share a little bit more about that and also how you think about that going forward? [12:02] I think the key ingredient of the original co-pilot [12:05] which was only auto-completion, right? Like you would type in your editor and it would complete, you know, the next, [12:10] line, but it could also complete multiple lines of code [12:14] you know, complex algorithms or simple algorithms. Some of the demos we [12:18] often chose to implement a sorting algorithm like bubble sort or prime number detection [12:23] And it can just write, you know, those 10 lines of code by just a simple prompt, which is a comment in the code or just writing the method declaration. And so getting, you know, into the editor where developers already write code, not changing the way they work, but giving them, you know, ideas while they're typing. I think that was the key moment other than obviously writing.

[12:42] the model being good enough and OpenAI tuning that model on publicly available source code from GitHub. [12:48] You know, GitHub didn't give special access to OpenAI. OpenAI was just able to access our source code in the same way that, you know, many other startups are now doing that, either through direct access, you know, through API or through archive programs like the Internet Archive and the Software Heritage Program. [13:05] And we actually have an established partnership with them to stream our... [13:10] open source code over there so it can be archived for [13:13] for until the end of time and so they of course the model got tuned to be good enough but then the [13:19] and the user experience, I think, was crucial. It wasn't, you know, AI is now in everybody's mind, but the truth is our cell phones have, you know, some kind of AI built in for... [13:29] a long time your keyboard is predicting the next word with some kind of machine learning algorithms [13:33] photo library, you can just go in there and search for license plate and find all the photos of the cars that you took to remember what the license plate you have. And that's AI, you know, image recognition, but that nobody perceives that as AI is just a co-user feature. So I think that also was the [13:49] the core ingredient of Copilot, we're meeting developers where they are and we're making their life better. [13:55] I think the name itself was also a stroke of genius. One of our other developers, Alex, came up with the idea to name this co-pilot as Nat. [14:07] is a hobby pilot and so that's where the name is coming from. So it's like, you know, I'm thinking about what could I name this so

[14:15] My boss, the name is resonating with him. Wow, that's so interesting. I didn't know that. [14:21] If you could go back to 2020, 2021, is there anything that you might do differently? [14:26] In hindsight, you can always move faster and be, I think, more convicted of those ideas. I think in the beginning, we kept the team intentionally small. Small teams can move fast. How big was the team? [14:43] staff researchers, sorry, or principal researchers, but [14:46] I think that's cheating a little bit in the sense that, of course, there was a team at OpenAI. There was a team or multiple teams at Microsoft, both on the research side and on the [14:56] you know, inference side, the model inference side, in fact, you know, on the model training side to even enable the model. So, of course, in the bigger partnership between Microsoft OpenAI and GitHub, [15:06] it was a larger team but the original paper was written [15:09] by three researchers and then 100 people or so are mentioned in the credits. And then we move fast with, I think, a stub of five and then I think it increased to 10 teams. And of course, today, [15:21] The team is much smaller, but we still have what we call GitHub Next and incubation [15:25] a team that now works on Copilot workspace and iterates on new ideas [15:30] It's almost like a startup incubator within the company. That's really interesting, yeah. And they're picking up ideas, and the main difference to our mainline engineering teams and product management teams are that, [15:40] they got to have the mindset that most of their ideas will never go into production, right? It's kind of like this OKRs won't work because the key result is to throw away most of their ideas and start fresh with another idea. And I think that's where...

[15:54] a lot of the innovation speed is coming from. [15:57] I want to go back to what you said a minute ago about how... [16:00] You handed over control of the model effectively to OpenAI. Was it scary? [16:05] Because the brain of your AI application is actually being built by another company, not developers under your payroll, your control... [16:13] Like, how do you think about... [16:15] Was it an easy decision to partner with a different external model provider? And I'm curious, [16:21] how you think about the value that... [16:24] Microsoft and GitHub provide to end users versus the value that OpenAI provides to end users and where you seek to [16:31] to really bring value to users. [16:34] To me, it wasn't scary at all. I'm sure, you know, there were folks in the team and in the company that thought we should have our own models instead. But in reality, you know, if you look at GitHub as a company that was born in the cloud, of course, we have always relied on partners to build our stack, you know. [16:51] We have, to my memory, have never built hardware ourselves. And while we have metal in data centers, or had metal in data centers for a while, [16:59] The data center itself wasn't built by us either, neither was the CPU and [17:04] and the memory and the network infrastructure. And then even if you go higher in that stack, GitHub is built on top of open source, and so the majority of software applications, in fact, aim, [17:16] We love sharing a statistic that says 90% of the stack of most applications, even if the application itself is closed source, is in fact based on the work of the open source community.

[17:26] You know, from the operating system, the Linux operating system is ubiquitous today on servers. [17:31] to container technology like Docker and Kubernetes to [17:35] thousands, sometimes tenths of thousands of open source libraries. And so the model just flows naturally in that stack. And, you know, we at Microsoft think about this as the co-pilot stack with the different layers, hardware, the model, the kernel, if you will, like the infrastructure response to the AI filtering and whatnot, and then you get into the application layer. [17:54] with the AI co-pilot and then the extensibility on top of that. And so if you look at these layers in the stack, Microsoft has strengths [18:04] and partners. And in fact, you know, in some form Microsoft is [18:11] involved in all parts of that stack and you know the copal PC and [18:15] you know, has a custom chip that Microsoft is developing [18:19] with partners, we have our own models with Fire3, and we have partnership models with OpenAI, but we're also hosting Mistral and Lama on Azure. [18:28] We, of course, have a large cloud and we have a lot of expertise in response to AI, and then we have lots of application we are building ourselves and enabling. [18:34] others to build those applications. And so two-part questions are always hard for me to forget the second one. But I think that actually describes the relationship very well and [18:44] you know, it allowed us to move really fast because we could rely on partners like OpenAI [18:48] not only building GPT-3 and then Codex, but also then innovating with GPT-3.5 and ChatGPT, GPT-4.

[18:56] GPT-240. We have Microsoft with large infrastructure that builds supercomputers for training, but also infrastructure to run inference. And Copilot today runs in [19:08] multiple data centers spread around the world in different regions. So the developers that sit in France now connect to a GPU, to an Azure instance that's much closer to them to enable low latency, right? And so Microsoft gives us a lot of infrastructure, a lot of expertise and responsible AI, and of course, a lot of... [19:25] commercial distribution. Yeah, very complicated layer cake that makes the magic come together. [19:32] Would GitHub ever want to build its own models or just use best-in-class models out there? [19:36] You know, obviously there's always a desire of engineers to build their own stuff and I would, you know, I would. [19:43] not deny that we have played with our own ideas on models and I think [19:48] you know, the almost team learning team goes back almost. I think it actually had it before Microsoft acquired GitHub. [19:55] And then we obviously have fine-tuned models ourselves, and we have fine-tuned models with OpenAI and Azure. [20:01] And we're working with customers on them being able to customize models based on the code that they have in their repositories. [20:09] And, you know, never say never what the future may bring, but today we're really happy with the models that we have, and we're constantly looking into the market of what not only OpenAI provides to us, but also what others have. [20:21] So Copilot is already one of the most successful generative AI applications in terms of user scale, usage, etc.

[20:29] What are some of the latest metrics you can share and what are the metrics that you're most proud of? [20:34] I think the one I'm most proud of is the developer happiness scores. And if you look at survey data, most of the surveys that we have done, [20:41] but also the service that now our customers either publishing themselves or bringing back to us, it's clear – [20:46] that software developers after they have tried [20:49] co-pilot after they got over the initial adoption hurdle or skepticism. [20:54] most of them love using Copilot and most of them report that they are more fulfilled, you know, [20:59] They're more satisfied, more happy. They feel like they're requiring less mental energy to get the job done. They need to do less boilerplate. And I think that... [21:08] alone is really making me happy. And there's lots of, you know, [21:12] numbers that I can throw out. But I think the general gist is no matter what developer I talk to, those that have used Copile for a while, [21:19] no longer want to work without a co-pilot. And then the other side is, you know, the productivity metrics of making... [21:25] developers more productive, which I think matters to the developers too, but it also of course matters to [21:32] their management chain and their leadership in the sense of [21:36] getting more stuff done, delivering more value [21:39] to their end customers. And so, you know, today we are happy to say that we have more than [21:45] 1.8 million paid subscribers on Copilot in more than 50,000 organizations. [21:50] And that really makes us happy looking at that growth. And, you know, we're here at Sokoja's office and it feels like we're like a high growth startup. And that's a good place to be in.

[22:03] It's awesome. [22:04] Well, you're still on our wall somewhere. Actually, we know exactly where we'll show you. Me too. The creative minds. [22:12] Was there anything that surprised you in terms of Copilot's impact after its launch and even today? [22:17] I think, you know, it's the 25% certainly surprised us the – [22:21] quick turnaround from the skepticism after we announced this [22:26] in June 2021. I think we had a very short blog post and then just a web page with examples and [22:33] and animations showing how it will work and [22:36] I think folks were looking at this and saying this is – [22:38] like a cool tech demo, but it doesn't actually work for me. [22:42] And I think the skepticism was that [22:45] people had seen how GB3 at the time would work and they [22:48] couldn't understand until trying it that it has [22:51] the context of whatever you wrote in the file before Copilot's suggestion comes and it considers adjacent tabs and things like that. And so it magically... [23:02] picks up your style and it knows about [23:04] open source libraries that you're using not because you have opened those libraries just because you have an import statement at the top and because the model was trained [23:11] on such a large corpus of data. [23:14] that, um... [23:16] it can provide the calls into these open source libraries. And so it feels to you that the co-pilot understands more about your project [23:25] than you thought GP3 can do. And I think that's where [23:28] the original major came from. The other thing is today, you know, [23:32] when I observe people using Copilot and obviously ChatGPT changed the whole game

[23:38] and what in chat as a component we have now copilot chat is this [23:41] natural language component and like using not only you know code and comments as a trigger but you know English and German and [23:50] Brazilian Portuguese and [23:52] We have demos in India where Karen MV, one of our folks there, is demoing this in Hindi. [23:58] And so, and then you can actually speak into [24:00] into the co-pilot with voice detection in Visual Studio Code. [24:04] and then gives you the response back also in Hindi. And so it's really cool for anyone that wants to explore coding, even if they're not [24:11] fluent in English or in programming language [24:14] My kids are using it to find their own bugs in Python. Wow, that's impressive. They're no longer coming to me, and it's like, Daddy, you have to find my bug. It's kind of like, well, go and find your own bug. And that also obviously helps them to develop their skills. [24:32] You know, I often talk about [24:34] you know, natural language will democratize access to software developers, but [24:39] it doesn't mean that everybody is immediately a senior or principal software developers. In the same way that just because I buy... [24:45] guitar, I'm not as good as Keith Richards playing with the Rolling Stones. And if you look at any professional band, [24:56] that's touring the world, they're all still rehearsed over and over again. And I think this is like this idea that [25:01] to be you know good in a craft you have to [25:04] keep doing it, that Kopata does not take that away, it just

[25:08] gives you another tool in your toolbox. I really like that analogy. Yeah. [25:12] still believe in teaching your kids to code? Because that's like a debate on the internet now. [25:16] Oh, absolutely. I mean, first of all, you know, the human language is not deterministic, right? Like you mean different things. You can mean different things by saying the same sentence. And even, you know, the... That's very German. [25:30] Even the yet, well, now we can get into off-topic debates about yeses and noes when you're answering a question with a negative in it, right? Like... [25:42] you have not been to the grocery store, do you answer that with yes or with no, right? And Americans expect a no, even though you mean actually yes to that question. But like, look, you know, human language is not deterministic, and so... [25:54] code is. With code you can very precisely describe what the machine does. Code is an abstraction layer on top of, you know, assembly language, on top of the instruction set of the [26:04] CPU or GPU that you know the the processor manufacturer like Intel or Nvidia has created right? So it's just another abstraction there human language is something completely different and [26:14] It's creative. [26:16] And that's the power, but it also means that there will be code involved one way or another. [26:21] and we're moving up the abstraction layer but we're also spreading the meaning [26:26] And that's truly powerful, but it's also... [26:29] means that there will be some conversion to code somewhere because the chip itself, at least today, [26:35] is requiring a deterministic instruction set. Yeah.

[26:39] I love that. [26:40] I'd love to talk about the future of Copilot. You've been announcing new products in rapid-fire succession, I think, Copilot X, Copilot Enterprise – [26:50] something on code security, I believe, and then [26:53] workspaces most recently. Can you tell us maybe about [26:57] what each of those things does and how you see them all fitting together in the grander vision for what you hope co-pilot becomes? [27:03] you know what you described with all these product names is one of part of our um [27:08] mindset is that momentum is our energy in this age of AI, moving fast and iterating fast is crucial. And so [27:15] You know, we are having developed Copilot from this original idea of auto-completion by adding chat. We were developing auto-completion and then chat and then chat. [27:24] With chat, we also announced something what we call CopilotX, which... [27:28] The idea was, we're bringing [27:30] AI features into every part of the developer lifecycle. We're bringing co-pilot [27:34] wherever developers are. So while we added chat to the editor, we also added a little copilot icon into the [27:41] input field where you write your [27:43] commit message. And that might be trivial, right? Everybody can write a commit message, but it also means I reduce my mental workload and I reduce kind of like the bias that I have for the work that I created myself, right? Like for me, everything is obvious that I just did for the last hour. But for you, when you want to review my [28:01] my commit or my pull request, it's not as obvious. And so having an AI described that in a [28:06] neutral form kind of like an outsider describing what i just did is incredibly useful and it just keeps me in my flow and we added it to the debugger we edited it you know to many different parts of the life cycle already and

[28:17] With Copal Enterprise, we bundled it into a higher price [28:22] product that allows enterprises to customize co-pilot [28:26] based on their institutional knowledge. [28:29] And enterprise here means really any company that has [28:33] gone for more than a few weeks because they all immediately build institutional knowledge, right? How we work as a team, how are coding practices, these are the libraries and languages that we use. And so unless you're a student in university, [28:46] that gets to have the free range of technologies available, at least when they... [28:52] when the professor allows that, every time you join a company or join a different project, you have to ramp up again on how they are doing things. And so Cobalt Enterprise, [29:00] lets companies customize the co-pilots to their institutional knowledge. And it makes it really easy for me [29:05] to join that company [29:07] Because I can now ask... [29:08] dumb questions without you judging me. Like, you know, imagine I would join here my first day, but like, why is Thomas asking all these questions? Shouldn't he already know all of the FTCNL? [29:20] been in professional life for a long time. That's the challenge that we have when we join companies, and we have the anxiety in our head. [29:25] that we can't ask too many questions before Steph says, what the hell? [29:30] And so that's, I think, the power of Copilot Enterprise. That's the power of bringing Copilot into every part of the developer lifecycle and ultimately into every part of the – [29:39] of our lives. [29:40] You've also mentioned that agents are one of the most important next things for GitHub. [29:45] Maybe just to set the stage, how would you describe an agent versus a co-pilot?

[29:49] And can you give us a teaser for what types of agentic capabilities we should expect come into CoPilot soon? Yeah, I would say, you know, CoPilot and agents is kind of like the same thing. An agent is using a model to get a task stem. [30:03] effectively it's looping with a model to solve something for you. And a co-pilot is an agent of agents, and it has multiple features, [30:13] to it. You know, if you think about auto-completion, whether it's an agent that takes every keystroke, [30:17] you did and the context that you have in your editor, it sends it to model inference that gets the response back. It might pick, you know, the best, you know, [30:25] and then it shows it to you. And so we are going to see more of these agents that take over more of our tasks. And one of them... [30:32] that I'm most excited about is, you know, autofix. And the way it works is so you submit a pull request and traditionally, you know, some security scanning feature [30:41] that you have integrated into your pipeline, find security vulnerabilities, let's say, you know, a SQL injection or cross-site scripting. [30:49] Well, that's great, except now I cost more work for myself. It's kind of like... [30:54] you have a Roomba, but instead of vacuuming your house, it just shows you where the dirt is. And then you have to go and vacuum yourself on that position. So now with autofix, [31:03] We're actually not only showing you the security vulnerability, we're also giving you the fix. [31:07] the AI model together with the vulnerability and the description and the code [31:12] to basically solve that vulnerability for you. And the initial results are really impressive. With some customers, we see that we can [31:19] burn through like 75-80% of their open alerts. Everybody has those alerts and if you don't have any alerts right now,

[31:26] I bet you you have them by Monday. That's the challenge in this world of software security is everything around is moving so fast. There's always a new version of a... [31:35] open source library, there's a new version of Linux or a new Windows patch [31:40] There's a new device coming along the way, a new NVIDIA GPU. And so we constantly are behind [31:48] just keeping our applications up to date, you know, to the standard that is expected by our customers. And at the same time, we have to build all that innovation and all that cool stuff. [32:00] What else do you think is missing in the product roadmap? Like if you could wave a magic wand... [32:05] Like what else is there to build that you're really excited about? I mean, I think there's still a lot of work to do on these agents. I think there's a lot of agents involved. [32:14] You can think about, you know, we talked a little bit, or you mentioned Workspace before. What Copilot Workspace does, it provides... [32:22] different agents to get you from idea to your pull request to get you from idea to the code [32:27] And the first one is the spec agent. And what that actually does, it helps you with your thought process. So you write down an idea, implement some feature, [32:35] And it looks at your existing code base. And it basically helps you then to reframe that idea [32:39] Now, that's not only useful for a developer, but it's actually useful for a product manager. [32:43] right, because it might tell you, well, Thomas, this idea is [32:47] no way you can describe that with a single [32:50] sentence and a developer cannot implement that in just a single ticket. It needs to be an epic or multiple

[32:56] different user stories, right? And then the next step is the [33:00] plan agent that helps to figure out where to make the changes in the code base. And again, you can kind of see here there's a lot of other benefits [33:08] you get from that because it helps you to understand the code base. [33:12] Because most code bases that are older than a few days [33:16] have hundreds if not thousands of files and as developers we have to navigate all those files and even if you have been in a code base for a long time [33:24] You still might miss out, you know, that one file that you haven't touched for [33:27] for a while and you have to add you know a conflict statement there so it helps you understanding the code base and then the implement agent [33:33] helps you implementing the code change. And every step in that way [33:39] you're still in charge, you can, you know, with natural language, modify the bullet points of each of these agents. And then obviously you can modify the code at the end. And so if you look at just these three agents, you can easily start thinking about other agents that you might have along the way, right? For example, the one that, [33:55] estimates the size of a ticket, the story pointing agent, as one example. Or another one is that once you have implemented the file, well, now you want to build, run, and debug. [34:06] the file and maybe you have an agent in the future that will just automatically fix [34:10] any bugs that were introduced by the previous agents. And so I think we're going to have more of these [34:16] building blocks, you know, Lego blocks, if you will, available to us. In fact, you know, [34:20] You know, if you look at Lego, they have way more pieces [34:24] types of pieces today than they used to have. Just the models are much more complex and you can

[34:30] buy, you know, NASA rockets and whatnot. And they need different pieces for that. And I think that's [34:36] kind of like the same way we should think about [34:38] copilots, they will have more of these building blocks that [34:41] enable us, in addition to more powerful models and a mix of models, these building blocks will enable us to do more. Increasing modularity. Interesting. [34:49] Where does your ambition for GitHub take you, or maybe even with GitHub Copilot specifically take you, [34:55] as you think about what you alluded to from the perspective of deepening [34:59] where you can go with just the software engineer. It's also expanding into potentially different personas. [35:04] PMs you mentioned, maybe an SRE, maybe a security engineer. [35:08] Where does that breath also take you? [35:11] I mean, first of all, I think all those roles today are already collaborating on GitHub. In fact, GitHub had always that mantra that we are building GitHub on GitHub. And sometimes, you know, we have pushed it a bit too far. [35:24] But today, you know, most of GitHub employees, we call them hubbers. Hubbers are, you know, engaging. [35:30] on GitHub in GitHub discussions and GitHub pull requests. Our legal documentation is all on GitHub, which if you think about it is actually... [35:38] much better than managing red lines in Word. Because you have a world history and you can see who made, what changed it, in fact you can kinda, [35:45] see soon a future where maybe your legal document is explained by Coppola in human language. [35:52] you know, in actual understandable human language, not the lawyer language, right? And so we're using GitHub, you know, to our company. But yeah, we...

[36:00] It's where all the developers and all the supporting functions collaborate on a project. So that's, I think, number one. [36:07] Number two is we want to democratize access to software development. And, you know, I, [36:12] I recently gave a TED talk and I talked about that. I think our goal is to [36:16] to get to $1 billion [36:18] software developers on the world. Now it doesn't mean 1 billion professional software developers, although [36:23] That might not be a bad thing necessarily given the demand is still very high and it's sometimes hard to find qualified software developers. [36:30] But it's really about [36:32] democratizing access to writing software on these devices that are with us. Our mobile phones are [36:39] you know, a really important part of our lives today. You can't really [36:44] imagine life, urban life for sure, without a mobile phone. And so then also being able to... [36:49] right little applications or little scripts or just using natural language to control the phone I think is [36:54] incredibly empowering. And so bringing that into [36:57] you know, 10% of the world's population by 2030 or so, assuming that then there are 10 billion [37:03] inhabitants on this planet, I think it's going to create a better world and it's going to unlock creativity everywhere and hopefully, you know, [37:13] We see cool venture-backed startups in India and in Brazil. And maybe, you know, the next big tech company is coming from one of those countries instead of the U.S. West Coast. We hope so. [37:26] Maybe zooming out of GitHub Copilot to broader GitHub itself. [37:30] GitHub Copilot itself is driving so much innovation within GitHub.

[37:34] But what are some of the other key initiatives that you're leading across GitHub overall as well? [37:39] We already talked a little bit about autofix and security. I think security is securing the software supply chain is dear and near to our hearts. [37:47] there is no future of human progress and with software if you're not also able to [37:52] secure the supply chain. [37:54] today, you know, there's this XKCD comic, um, [37:57] of the internet's infrastructure. And there's this one building block that says, you know, the guy in the map, Nebraska, maintaining this one library alone, right? That actually is as funny as that is, that is a reflection of the software world today. [38:11] So we have to make it sustainable for those maintainers [38:14] to build that software and keep it joyful, [38:18] And then also we have to make sure that all these building blocks that are in our stacks [38:22] that have become system critical and to be secure. And so we're both investing a lot in [38:26] platform security and application security and of course you know in our security products and [38:31] I think that's going to be crucial combined with Copilot and AI to not only create all these work items, but to also enable developers to burn them down, to fix all these issues. [38:43] I want to zoom out from GitHub for a second and just talk about how you see the future of... [38:47] AI and coding overall, like you mentioned, [38:49] There's been a billion casual developers around the world. [38:53] What will it look like? Will everybody be kind of coding applications for themselves to use? Will there be... [38:59] some number of professional developers who are [39:02] Super developers? Like, how do you imagine the world looks...

[39:05] when you so kind of democratize the craft of coding. Mm-hmm. [39:11] I mean, I think it's an incredibly creative world in a world where you're not dependent on... [39:17] when you're a kid on your parents having technical knowledge or your school having a teacher that knows how to do these things. So we're going to... [39:24] have much more access to those that are interested in learning about this [39:27] You know, it's easy today, you know, to take a sheet of paper and paint something. And every restaurant, at least, you know, in this country, gives you crayons if you come with kits and a coloring sheet. Lifesaver. Lifesaver. Or you have your mobile phone and they use your mobile phone. You know, it's easy to learn and easy in the sense of accessible to learn a musical instrument. And I think it should be easy to learn coding. [39:57] that we are inflating the number of developers because just because kids learn it doesn't mean they want to become a developer. I think there's still a world where [40:05] people want to do something else than software. [40:07] But then if you think, you know, about many other professions, physicists, for example, they use a lot of software, you know, the... [40:14] the first image of a black hole was and that by the help of open source project you know the mass helicopter [40:19] ran on open source, right? And so, yeah, it's space, and it's space engineering, but they're using software and they're building software. And so the profession itself is everywhere. Every company is a software company. [40:32] Banks are software companies and energy providers are software companies.

[40:36] our software companies predicting, you know, what seed to plant this year, what the weather is going to be like. [40:41] you know, what was the soil quality from last year and things like that. But of course, you know, that the hobby scenario is also important. [40:50] Like, you know, tax season is over here in the United States, but that doesn't mean that I couldn't think about next year to automate a lot of that if I only had an AI agent that does all that work for me, you know, and downloads all the PDFs and extracts all the numbers. And I don't think we are too far away. [41:05] from that. [41:07] I'm now in software for almost 30 years, and [41:13] as a professional software developer and [41:16] I don't have a lot of time to code. I have a company to run my podcasts to give and things. [41:23] The problem today is you find an hour on a weekend and... [41:27] You have a project and the first 20 minutes you're spending with updating everything to whatever you missed. And the burden is actually – the fun is gone to a certain degree because the burden of maintaining software is so high. So having something available to you that gets you quickly into the hobby and out of the hobby – [41:45] or out of that task, I think, is incredibly empowering and brings the fun back. And that's where the Lego comparison is so useful, right? Because – [41:56] Lego is just incredibly accessible and [41:59] Even if you... [42:00] like the best Lego is the one where you don't have instructions. They just have a table full with Lego brakes, even in random colors, right? And then have

[42:10] this excitement of play and even professional workers at their... [42:15] Offsides or workshops often have little gadgets or bricks on the table so you have your fingers do things while you're thinking. So I think that's where that world is leading us and we will have, you know, [42:25] more access to technology, we have more people that can build software. That doesn't mean that they're taking jobs away from professional software developers. [42:33] On what time frame do you think we'll have coding agents that are as good as? [42:38] maybe the average professional software developer and [42:41] And, you know, there's the legend of the 10X software engineer. Like, at what point do you think... [42:46] AI will be as good as the 10x software engineer and capability. [42:50] You know, the trick in that question is the as good as, and what does that mean? So I think, you know... [42:56] Is a model today able to write [43:00] better code than the average developer if it is prompted in the right way or given the right context, I'd say we are already there. [43:07] on average. Because, you know, often... [43:11] you know, the model just knows more about that whole space than I as a human do like [43:16] And you see that with students if they have to implement [43:18] you know, like a conversion from binary to decimals and like that. And they might write a hundred lines of code and then they go [43:26] and ask Copilot how to do that, and they get probably an open source library and one line of code, and then you can kind of say, "Well, I'm not allowed to use open source." And then you would probably still get a better code than they would. And I think the same is true for the professional software developer.

[43:42] because look we're not perfect we have a human and I think that's part of our nature that's part of creativity now [43:48] That's the key thing, though, that the model is not creative, and the model cannot... [43:52] today, you know, the model cannot make decisions for us. Or if it doesn't make decision, it doesn't [43:58] actually take all the [44:00] constraints into account. Like if you think about software development, other than writing code, which I think the fun part, it often means, you know, I take a very complex problem. [44:10] and you break it, decompose the problem into small building blocks, you know, and the [44:15] the block size is increasing over time. It used to be auto-completion used to be just the next word and then it was maybe a full command [44:22] and now it's multiple lines of code and maybe it's whole files in the future. [44:27] But along that decomposition process, you still have to make a lot of technical decisions. What database am I using? [44:32] You probably know better how many database startups Sequoia has invested in and how many infrastructure startups. [44:38] in how many serverless startups. And obviously there's all the incumbents in all those spaces. [44:45] So there's a [44:45] thousand if not 10,000 decisions to be made and the engineer is the systems thinker that is making those decisions or the team of engineers and companies. [44:55] You know, this... [44:56] We have been building houses... [44:57] you know, as humans for [45:00] thousands of years and if you have ever built a house it's still not a soft problem [45:06] Yeah. [45:07] But at what point do you think that creativity and that systems thinking... [45:11] gets built into co-pilots, or do you think it never does?

[45:15] I mean, it's a bit of predicting the future. I don't know if I would say never, never. That's a dangerous thing on a podcast. You invite me back in three years and it's like, well, Thomas, last time we asked you about this, and clearly it has happened since... [45:28] No, I think we'll see where the research goes and where the technology goes in... [45:32] in kind of critical thinking and systems thinking and those kind of questions. Also learning, you know, learning [45:39] You mentioned your two-year-old, your two-year-old, and my kids, they learn, [45:43] you know, as they mimic the humans around them, they are so good at learning language. And especially in young age, you know, they can learn. [45:50] multiple languages, mine speak English and German, [45:53] because we speak German at home and [45:55] They don't have an accent in English. Well, they have an American accent, but they don't have the German accent, right? [46:00] I don't have an accent in German either. And I think this shows that the human brain is still so much more advanced than – [46:09] then the transformer models and the diffusion models and the other types of models that we have to image recognition and whatnot that we have today. [46:17] And, you know, it remains to be seen if we can kind of like add that sentience piece to it. But today I'm not seeing it and I haven't seen any... [46:26] Researches are telling me that that's coming anytime soon. That's a really clear delineation. [46:31] Switching gears a little bit, I'd love to hear your thoughts on the overall ecosystem of startups right now. [46:36] AI Code Gen is the hottest category with a lot of ambitious founders. [46:40] And there are so many different attempts that they're taking, whether it's folks who are trying to build a better model,

[46:46] folks who are trying to build a better IDE, [46:49] folks who are trying to build kind of an all-in full-stack engineer as an agent. [46:55] And, you know, but the big elephant in the room is GitHub Copilot and all the adjacent products that you have. [47:01] around it with [47:02] Copilot workspace with Copilot Enterprise with Autofix. [47:06] and owning VS Code as well, amongst many other things. [47:09] How do you think about what white space there is for existing or sorry for new founders? [47:15] And if you were a founder yourself, [47:18] trying to build a space, what would you do? - I mean, I love developer tools. So I probably still do developer tools. And I'm not sure I would worry too much about what's white space that's what's taken by incumbents. [47:29] because that can change quickly. When GitHub started, SourceForge was the big elephant in the room, and SourceForge has all the open source projects. And then, [47:38] can get and get to accept that space. And then the founders, uh, [47:41] of GitHub took it and built GitHub and all of a sudden you know that elephant in the room was no longer the elephant. [47:47] And I think it is often fun to compete in the same space. And we love competition because it pushes us forward [47:55] It's boring to do a race or a game if you don't have an opponent and if you don't have [48:03] you know other other teams in the league and who would watch you know the Super Bowl if there's only one [48:08] winning team every year. So I don't think competition should hold you back as a founder [48:14] from going into that space. I think the software development space

[48:17] is wide open and there's lots of problems to solve. [48:21] there's lots of problems in different, you know, [48:24] industries and categories to solve. We haven't really solved. [48:27] modernization of source code. Cobalt runs still on mainframes, and you can use a copilot. [48:32] And we are heavily looking into that. And because it's such a pain point for many [48:37] financial services institutions, your credit cards, your bank account, Wall Street, all that runs the COBOL. [48:44] met a single bank that doesn't run some COBOL [48:48] on some mainframe. What would you do with Copilot and Cobalt? [48:52] So, well, today you can explain that, which is, you know, often helpful because the code was written 60 years ago. And so the people that wrote that code are retired. Yeah, yeah, exactly. [49:02] Would you rewrite all the code with Copilot? And then you can ask it to write unit tests because nobody wrote unit tests in the 60s and 70s either. [49:08] let alone that there were... [49:10] unit testing frameworks for those languages. Like keep in mind, you know, the 60s, [49:14] that was before hard drives. [49:16] Like before we had personal computers. So it was a very different world back then. And of course... [49:23] Those companies have done work to modernize to a certain degree. [49:27] but it's far behind Azure software development so you can support that transformation process today but we're not at the point where you can just click a button and [49:37] you have a transformed. The same is true [49:38] you know for many more modern languages you know there's large PHP code bases there's lots of Java out there [49:44] There's lots of optimization that we can take to just make our existing stacks more efficient where agents can help.

[49:51] There's lots of things to improve in every part of the software development lifecycle in this industry. [49:58] ecosystem around us. GitHub [50:00] I like to think about GitHub as one planet in this universe of software development tools and [50:07] There's smaller planets around us and [50:10] you know, equally or almost equal sized planets in our space and we consider them [50:15] partners and we are happy that they're there. [50:18] Yeah, so interesting. [50:19] Agents are just pulling that thread a little bit. It's also an area of excitement for us. And I think, you know, you have a lot of the benefit of owning so much of the data and everything you can do on the post-training side. [50:31] OpenAI itself is also getting better and better with each new model class, as are every other model. [50:36] model company out there. Would you ever want to kind of invest into building your own agents, maybe from scratch by building your own agentic models or just to partner with some of the others out there? [50:49] Me as GitHub CEO. Yes. Or me as an agent investor. You as GitHub CEO. [50:57] Yeah, I think we are probably doing both. In a way, we always have done both things. As GitHub, we have invested into our own things, the thing that we consider... [51:08] as you know, core. [51:10] Part of our platform and of our offering, you know, part of our primitives and we have partners. [51:15] partner with companies, you know, [51:18] Just earlier this week, we announced the partnership with JFrog, which covers Spinery.

[51:23] artifacts, scanning of containers and those kind of things and they're obviously [51:27] very naturally in our space. You know, we have the source code. And what do you do with source code? Well, you... [51:32] either compile it into binaries or you combine it with binaries [51:35] before you deploy it into the cloud. And so there's a natural value chain there. [51:39] where we have things where we invested ourselves, things like releases that you see on GitHub, and packages we own NPM. [51:48] the largest package registry in the world for the JavaScript ecosystem, and Nougat for our partners at Microsoft and the largest.net. [51:56] registry in the world, but that doesn't mean that we cannot also partner with the JFrog to ultimately enable that secure software supply chain [52:05] that I mentioned earlier and that I think is crucial and that [52:08] Figma, Vercel, like so many other companies in our space, [52:13] that the the play in a part in the life cycle and I don't see a world where [52:19] Somebody covers all of that and can convince every developer that using all [52:24] these tools is better than picking [52:27] you know what what analysts might call best of breed and like the tool that I consider the best best is all subjective anyway right like it's all this [52:35] does it meet the expectations of the developer in the environment? Yeah, makes sense. [52:41] That's great advice for startup founders and I think really encouraging. [52:44] What about for incumbents? I think you are such a beacon of hope for incumbents companies. I think Satya said in the last earnings call that... [52:53] GitHub is now growing 40% year-over-year thanks to the co-pilot acceleration.

[52:57] And to your point earlier, like you feel like you're a [53:00] young startup again, like walking into the Sequoia offices, like [53:03] What advice do you have for, there's so many incumbents that are trying to reinvent themselves around AI. What advice would you give those folks? [53:11] Sajja actually said 45%, so we're really excited about that on our revenue record, 45%. [53:17] I got an email from somebody last week who was like, looking at GitHub, I'm losing the belief that large companies cannot move as fast as... [53:25] as startup can and I think you know the key ingredient on this is [53:29] radical full focus and basically focusing on a few small things just because you have a thousand engineers or [53:36] or 50,000 engineers doesn't mean you can do it all. That's just a false... [53:41] assumption and I think it's in a way misleading as you manage larger teams [53:46] you're losing kind of like the [53:48] the unit of team size that shows you how much you can actually get done and how much friction you have in the system if all these teams work on different things. I think focus [53:59] you know, lots of no's on all the ideas that people have around you and customers. [54:04] that's I think the key piece to move really fast. And then obviously taking in some strategic bets and strategy means you're thinking about it as how do I differentiate from others? You know, what makes me specific, what makes me special in this market? What lets me [54:21] charge the prices I want to charge and not fight a race to the bottom. And I think that's [54:27] this kind of like thinking about, okay, you know, in software we like to think about, well, if I add just three more features,

[54:33] then I'm going to win the space until you realize, well, everybody else can also add those three features right there. [54:39] There's almost nothing in GitHub itself as a platform that you couldn't rebuild with significant investment and time. But I think it's really hard to mimic our culture. It's really hard to mimic... [54:51] our experience, our obsession about developers. [54:56] and ultimately our focus and the way we're approaching these things. I guess very tactically, do you recommend staffing a Tiger team to get to the product market fit on AI? Do you recommend like... [55:06] you know, pulling half your engineers off, whatever they're working on, and like, hey, you guys are the AI team now, we got to go bigger, go home, like... [55:12] And tactically, how do you recommend companies? Well, as an incumbent, you always have the challenge that you have to sustain whatever the business is you're in. And so you can't just pull everybody into a new topic unless you're willing to – [55:24] disappoint all the existing customers. No enterprise business has not made promises to enterprise customers of what's coming next on their roadmap. Lots of conferences, including our own, is great. [55:37] you know, stuff they're shipping right now and stuff we're announcing for the next six months. And if you're not delivering on this announcement, [55:42] your customer base is not going to be happy. And so you can't really make that drastic move right away. And so we love this idea of a tiger team or an incubation team. We call it GitHub Next. [55:55] When we set that team up, we actually thought they're going to work on projects that are like five years out or [56:00] you know, in that horizon three space. And then it turned out, well, it was more like six months ahead of the curve. The future came quickly. The future comes really fast on us.

[56:09] And so as that future then comes fast, you really also need to move fast in handling [56:13] things from the incubation team over to [56:17] your mainline engineering team. And then it's all about, okay, so we are funding AI because we are seeing the traction there. [56:23] And that means we're leaving some of our previous bets in [56:26] you know, keep the lights on mode, KTO, or, you know, [56:31] saying goodbye to [56:32] goodbye to these ideas and shut them down. I think that's the hard part [56:36] of the strategic pivot. And that's easier for a startup. At least it often looks easier from the outside when a startup pivots, right? Like there's lots of startups on the wall downstairs that have gone through that as well. [56:49] But obviously internally, it's also very hard. Emotionally, you're tight. [56:53] to ideas. You remember the workshop under Sequoia Trees where you had those ideas, and then six months later you're realizing [57:01] We never got to product market fit, and the time is now to move to something else. So that is the same for large companies as for small companies, except that small companies, [57:10] have more of a forcing function to give up and move to something else. [57:15] The innovation and success that you've created around GitHub Next is amazing. [57:20] How does a kernel of an idea start within GitHub Next? [57:23] How do you then resource and invest into it? And then how do you... [57:27] At what point and what's the process in which you decide whether or not... [57:30] something should be continued to invest in or [57:33] or shut down entirely. [57:36] So I think the start is always the employee, the hubber that has an idea, and we have lots of ideas in the company.

[57:44] We do hack weeks, hackathons, passion projects, [57:49] whatever you want to call it, you know, 20% time. And then in the next team specifically, you know, there's lots of demo teams [57:56] meetings, demos and not memos, you know, it's much more useful to just show a working prototype. It's so easy. [58:03] these days to just use some design system [58:06] React components or Figma and stitch something together even easier [58:11] with Copilot and then you know it's strategic decisions that [58:14] the leaders of these teams all the way up to me [58:18] need to make you know that many ways very similar to any other creative industry you know like [58:24] at Disney or Pixar, they have to decide which movies to produce and which ones are [58:29] probably not going to gain traction, and part of that is customer research. [58:33] talking with developers, and that then, you know, keeps going as we go, you know, through... [58:38] the initial idea and the first prototype and then going into a technical preview. And then the preview is all about, you know, that flywheel, you know, [58:45] that feedback loop with the people using it. And quickly you'll see, you know, are they trying it out and then they are churning or are they keeping – [58:54] keeping the energy high and then keeping the ideas flowing it's great you know [58:58] in any project if people are just sending you more ideas and more feedback. And I think that's been the decision – [59:05] Whether we are keeping that project and making it a main project or whether we are deciding, okay, the X-ray rate is over and we learned a lot and we're moving to the next one. And of course, there's commercial.

[59:14] aspects as well. [59:16] Maybe outside of the world of code generation and anything in your current purview, [59:22] What else are you excited about in the world of AI in the span of one, five, or ten years? [59:28] Oh, outside of the one of us. You cheated a little bit on me. [59:33] Um... [59:34] I mean, I think in the context of one year, within the space of developers and outside of the space of developers, I'm excited about agents. And we're still very early in this journey. I think we have high expectations of what the agents could be, maybe too high expectations to some degree. [59:50] And so things will probably take a little bit longer. But I think in the next year or so, we're going to see [59:55] more of these helpers within the chat interface and outside of the chat interface [59:59] that will solve tasks for us, you know, like, [1:00:02] I look forward to that travel agent that often gets demoed. [1:00:06] by big companies to actually materialize and it can just go [1:00:10] into my chat interface and say, you know, I want to have a beach vacation. [1:00:15] over Spring Break and then figure out when Spring Break actually is because that's all on the internet and [1:00:21] you know who I am, what my name is, and what my [1:00:24] families' names are on their birth dates and their passport numbers, and so I don't have to enter that into a KamaSam interface anymore. It shows me the price points and probably we're going to the same hotel as every year anyway, [1:00:34] And so I think that those travel agents and those kind of agents are going to happen in the next year plus years. [1:00:41] I think five years is my natural language vision and unlocking

[1:00:45] the world's knowledge, including software developers, to everybody in any language, any human language. [1:00:51] And maybe that's even happening sooner. [1:00:56] Ten years is hard. Ten years is so far away. But I think, you know, that's the... [1:01:01] AI of things, you know, the material, like the mechanical world of AI. [1:01:05] You know, so many things that we do in life [1:01:09] you need to grab something, or you need to push something, and [1:01:13] You know, go and check into any hotel. There isn't any AI involved once you get your room key. And maybe the elevator, you know, has some kind of AI because you push a button, what floor you want to go to and it sells you to take elevator C. And so you no longer push the button within the elevator. That's an optimization problem in itself. [1:01:31] But I think there's so many physical things in life, you know, and we are still not at really self-driving cars. [1:01:36] We have a dishwasher, but I still have to put the dishes into the dishwasher and out of the dishwasher. It's better with nine and 12 year olds than with two year olds. And there's so many other things in life where I think, you know, whether it's a robot or some other form of physical AI. [1:01:53] is going to take over some of the things that we consider as chores. [1:01:59] We're really excited to hear that. And I think we have a very similar view of what will happen one, five, and ten years from now. [1:02:06] Ha, who do you admire most in the world of AI? [1:02:09] I think I admire those that are building new stuff [1:02:13] with AI and software. Those that have a dream of what they could build. I think those that obsess about a problem and not about the technology. We're talking a lot about AI, but

[1:02:28] At the end of the day, it's what's the problem I'm solving now. [1:02:31] for the world and you know there's lots of biotech companies that use AI to try to cure diabetes or cancer and [1:02:39] I'm sure there's companies trying to solve climate change with technology and I think [1:02:45] those builders, the founders, those that have [1:02:49] big ideas and that can change the world. You know, those are the ones I admire the most. And often when I meet with them in [1:02:55] in their offices, you know, on calls, I'm like, [1:02:59] this is so cool. And obviously as GitHub, we are enabling a small part of that. And so [1:03:05] We feel really proud of being part of that journey and we're really [1:03:09] excited about and building more for them. [1:03:14] I think you're enabling a huge wave of new AI companies, especially those that are born open source. [1:03:19] It's amazing to see. [1:03:20] What advice do you have for the founders listening in the audience who are building in AI today? [1:03:25] I mean, Sonja asked, I think focus is everything. It's so easy to get lost in... [1:03:32] you know, all the ideas that you can put on a whiteboard, and that's the danger of a whiteboard, that it has so much space where you can put ideas. [1:03:40] But focus ultimately is everything, you know, finding, [1:03:44] very quick market validation process. [1:03:47] Often on the developer space, that means growing and product growth, enterprise growth can come later. But like from my experience, like, [1:03:54] There's nothing then, you know, a few excited developers spreading the word about your product, even though, you know, the big revenue number then later comes from big enterprises buying you. But the reverse is often much harder of going enterprise first.

[1:04:09] You might find excited people there as well, but the feedback loop is just so different. [1:04:14] So it's focus, it's, you know... [1:04:16] trying to find that flywheel, the product market fit, you know, and then [1:04:21] Think [1:04:22] I think the other one is to think big. I think it's like it's easy to find small ideas on top of, you know, model today. [1:04:29] that get you know commoditized tomorrow and so you have to think forward and that [1:04:34] 10 years is a long period of time, but that's a good period of time for a startup to build something that – [1:04:40] is actually meaningful in this world. And so you have to think [1:04:44] big, you have to create a vision [1:04:47] that may be much larger than the MVP, you know, the first thing. [1:04:52] the prototype that you're building right now and I think that's really hard as a founder [1:04:56] draft out that vision and then decompose, right, and go back to that small problem that you can solve right now. [1:05:03] So we'll close with some rapid-fire questions. One-word answers. [1:05:09] You changed the bullet. One-word answers. One sentence if you need it. No, I say. One-word answers, yes. Okay, let's go. Okay. Will anyone meaningfully disrupt NVIDIA and AI chips in the next, call it, five to ten years? [1:05:21] Yes. [1:05:23] In what year will we pass the 50% threshold for SWE agents? [1:05:28] 2025. [1:05:29] And what about 90%? [1:05:31] 2028 [1:05:35] We'll get hub primarily. I think that was too pessimistic, that last one.

[1:05:40] We'll hold you to it. [1:05:41] Will GitHub primarily be using open or closed source models in the next five years? Both. [1:05:47] Where does the majority of value accrue in AI? Models, compute, infra, applications? [1:05:53] across the whole stack [1:05:55] That in hyphen, it was one word. [1:05:59] Is systems thinking and creativity going to get baked into the models? [1:06:03] In the next five years. [1:06:05] Maybe. [1:06:08] And will there be a new consensus architecture beyond the transformer in five years? Of course, yes. Wow. [1:06:15] Why would you think the other way around? Like that said, it's a much easier bet to think that there will be a new architecture because there was other architectures before Transformers and Transformers. [1:06:26] Like that doesn't mean it replaces transformers, you know, you're [1:06:30] Cell phones still has a CPU, even though GPUs are the hot commodity right now. And so I think, yeah, there will be new architectures and they might be bigger than transformers today. So interesting. I mean, that would bring a lot of new oxygen for the builders in the ecosystem and a lot of things that would have to get reworked. [1:06:45] Rebuilt and re-architected. Yeah. [1:06:48] Amazing. Thomas, thank you so much for joining us today. It's been wonderful digging into the history of GitHub, the birth of GitHub Copilot, and the ambition that you have going forward as well. Thank you. Yeah, thank you so much for having me. It was so fun to talk to you both. [1:07:02] Likewise. Thank you.

[1:07:32] you