Roblox Studio Head Stef Corazza: Using AI to Empower Creators

[00:00] We have this unique synergy and collaboration with the community, where basically we told the community, hey, [00:08] Give us access to your data to train AI, [00:11] We're going to make the best AI companion, the best AI assistant that we can. And that assistant goes back into studio and is free, right? So we're not making money off your data. We're actually helping you create more. [00:25] And so we found the overwhelming majority of the creators in our community gave us permission to use their data for training. And that's why I was mentioning earlier, we have not only one of the largest data set in the world, but also the most multimodal. [00:55] . [00:57] Welcome to Training Data. Today we have an amazing guest in Steph Corraza. He leads Generative AI at Roblox, one of the largest gaming platforms on the planet. [01:08] Roblox has 79 million daily active users. [01:12] and they have a creator economy that pays out hundreds of millions to creators. [01:17] Because of this, Roblox is uniquely positioned to transform how games are made and played with AI. [01:23] Steph is a founder at heart. [01:24] He started Mixamo, a pioneering AI company for character animation, and was acquired by Adobe.

[01:30] Roblox brought him in to revolutionize how games are made with AI at their own platform. [01:35] Under Steph's leadership, Roblox is pushing the boundaries of AI and gaming. From their groundbreaking AI assistant, which lets you generate games with simple, natural language, all the way through their 3D foundation model technologies. Welcome, Steph, to Training Data. [01:53] Today, we get to talk about games. In particular, we're talking about AI at Roblox. Roblox is one of the largest gaming universes on the planet. [02:05] universe instead of platform. Platform is a pretty overused term and really Roblox is created much more than a platform. It's a place where you can create. It's a place where you can play. [02:15] where you can meet new friends, and it's all done online virtually. Now, we are technologists, but we're also investors, and I want to spend a moment on how remarkable of a business Roblox is. It's obviously an amazing technology, and we'll get to there, but the business is exceptional. [02:33] You've got a $29 billion market cap company with over $3.5 billion of run rate revenue. An amazing stat here is they're really building an economy. It's not just a selfish company. It's a company that actually has produced over $800 million for their creators, for the people actually building on Roblox. [02:53] over the course of a year. [02:55] 70 billion hours of gameplay on Roblox per year, 70 billion. And they're able to deliver cash operating profit. So $600 million operating profit. That's because they've catered to a huge audience, 79 million daily active users. Over time, they've actually shifted up in age. And you've got 46 million of those daily active users are actually now over the age of 13.

[03:25] . [03:26] daily active voice users, and the numbers just continue to grow on this amazing business. And so, Steph, we're so excited to have you here today. We could not have asked for a better person in the category of AI gaming. From machine learning to computer vision to biomedical engineering, you have a pretty impressive technical background, an amazing journey from what was initially a [03:48] biomedical to generative AI efforts. And we were hoping that you could kick it off by just telling us how you got here. How did you get to becoming the head of generative AI when you started off as an engineer in a very different field many years ago? [04:02] And thank you for the great introduction about Roblox. It's really like an amazing example of a compounding effect. [04:08] And every day we are mesmerized ourselves about the success. [04:13] My journey started at Stanford about 20 years ago when I came from Italy as part of my exchange program, and I was focusing on computer vision, machine learning for the measurement of human motion. [04:28] And so you have basically two markets there. One is the biomedical that you mentioned, and the other one is animation. And so we were basically at the boundary between the two, and at some point I realized there was a much bigger opportunity to [04:41] in the animation space. And so I basically work on video-based motion capture and animation creation solution. That then led to the spin-off of Mixamo, the company that I started in 2008. [04:54] later got acquired by Adobe in 2015, and it's still today

[04:59] one of the most used machine learning services in the industry to rig and animate characters. [05:05] And so after a few great years at Adobe, [05:12] Actually, seven of them. I helped build the 3D offering there, including products that my team built, like Adobe Stager and Adobe Arrow for AR. [05:21] And then we acquire algorithmic, and so we build the full 3D portfolio. [05:26] And then after that, I was really passionate about GNI. I was working with the CTO. [05:31] of Adobe, trying to figure out completely new ways to generate things. And that's when basically Roblox reached out. And I had breakfast a few times with Dave and we talked about it. And I really realized that potential of the DNA I had was really finding in the gaming space and specifically in Roblox being a platform where so much data exists, you know, 15 million experiences every year are played. [06:01] this could really blossom with a massive impact worldwide. [06:05] So before we get into the AI components, can you tell us a little bit more about the platform? You're an engineer by background, and so scale is incredibly exciting to you, I would imagine. The sheer scale at Roblox is pretty mind-boggling. I think to anyone, especially users, there are obviously platforms that have... [06:24] more daily actives, the Facebooks of the world. There are platforms that have more monthly actives, et cetera, but it's very rare to have the amount of bandwidth and compute and graphics and everything in one place. Maybe you can share a little bit about what that means to have a 15 million daily sessions and 79 million monthly actives and a little bit about the stats and what that means technically. Yeah.

[06:49] Yeah, we always like to talk about the daily active, which is the 79 million that you mentioned. But the monthly active that I think we don't communicate to the outside world is even more staggering in being in the several hundreds of millions. Right. So it's a massive community that is growing very healthy and pretty fast. And sometimes people ask us, you know, where are those games coming from that people play? [07:19] now like worldwide franchises with like tens of millions of concurrent companies. [07:23] our players. [07:25] And then... [07:26] Sometimes I give this number, which is also like a reminder of the scale. So every day we have roughly 90... [07:34] thousand experiences in games published on robots. [07:38] Wow. And so that gives you the scale of the human creativity, if you want, and how much really this is becoming more and more of a creation platform and game development platform that... [07:51] has like incredible numbers in terms of scale of creation. And also like it's an economy on its own. As you mentioned, you know, we paid out, you know, $800 million to our creators. And so people have jobs, people buying houses, people have like companies. They are now, you know, we have game studios. They have like more than 100 people. Some of them are VC funded. So it's basically creating its own like creation economy and tied to that. And so that I think is very humbly.

[08:21] has incredible potential because the uniqueness, I think, of Roblox is that we are one of the most vertically integrated companies on the planet. We own our own data center. We have many data centers around the world where we own the hardware, but then we also own the app that distributes all these games and the players and all the services like video chat and live chat and chat translation [08:51] . [08:52] of leading with my team. And then also we basically have all the services for creations that go with it. So it's all the way from like the bare metal CPU, GPU cluster, all the way to the creation tools with a big difference that we only charge our, we only basically take some revenue ourselves when our creators make money, right? We make money when they make money. [09:22] There's no upfront fee for the tool. The tool is free. A lot of services are free. And so there's really very little friction to start using Roblox. And if your game has one user or a million users, [09:34] You don't have to worry about anything. We scale it for you. We pay for all those like CPU, GPU instances, storage in the cloud and everything. And it's completely opaque to you as the creator. So I think that's the uniqueness. And as part of the fully vertical integration, we are also able to subsidize AI, right?

[10:04] code creation, material, texture, assets, everything, and it's all free to the creators. I'd love to get into that. Maybe, can you just walk us through today, what is the experience of creating a game? Like, who are the typical creators on your platform? Is it a high school student? Is it a professional game developer? And what types of games are they creating? Yeah. [10:24] That's a great question. So we have several million creators on a monthly basis on the platform. So big numbers there. [10:32] as well and usually [10:34] I think the average age is in the mid-20s. It's a little bit older than our player demographics, of course. And typically, the majority of them are doing world building. They're building stuff. They're artists. They are making games. And then we have about 30%, 40% that actually are coding. And then, of course, there's an overlap between the two audiences, but basically roughly... [11:00] This is what we're seeing. [11:03] It's used for the most... [11:06] Different creations you can imagine from like natural disaster simulations to learning to the cloud more classic gaming experience, events, concerts, fashion, design. We are seeing like a new type of experience popping up on a daily basis, which is very fascinating. What's your favorite game? [11:29] I've been playing lately Racing Empire quite a bit. I like our games.

[11:35] So that's a good one that I really like. [11:38] Steph, I was actually surprised to hear that you said 30% to 40% of the developers are actually coding. [11:44] as opposed to world building. Why is that? And maybe that kind of parlays into what you've created with Assistant. [11:52] Yeah, so there's just a, it's a skill that is harder to master and to get into, right? A lot of people just like go from players and they want to create something. And so they start like. [12:05] building the world, and that is probably more intuitive than writing code, where you have to understand these high-level constructs and apply them to get interactivity. And so that is one of the things that we wanted to tackle with Assistant. We wanted to basically remove that friction of having to learn to code, having to learn programming language in order to create interactivity. And so that was one of the initial inspirations, and that's why CodeAssist was the first [12:33] feature that we released and then now... [12:37] This was like in March, 2003, which feels like two decades ago. And now, you know, a year and a half later, we basically have a full... [12:48] a full basically assistant that has game development capabilities. They go from writing code, auto completing code, [12:57] explaining code, debugging code, applying scripts to parts and objects in your scene. So that's all the coding stuff. Then we have documentation, right? People ask, how do I do X, Y, Z? And usually they have to browse through their forums and documents on the Internet. Instead now...

[13:16] a system can basically summarize those information for them. Then the third aspect is creation of assets. We rolled out a material generator, we rolled out a texture generator, which was a lot more complex where you can basically texture any 3D object with quite good fidelity and resolution, just from a text prompt. [13:37] And so all this together is what we call Assistant with an umbrella name. And basically it's allowing now to create entire simple games from scratch just by typing natural language. In the future, there will be more like multimodal input through images. But basically right now we have simple games that people are making also as a test. [14:01] where they are only using Assistant. And so you can imagine, you can make that game on your phone, you know, with a microphone. [14:09] You just speak to it, and then a system will just generate the world, will create the forest, will create the enemy, the boss that you have to fight, and then also add all the game mechanics to it, everything automated. [14:23] Are the games created with assistants? Like, how good are they? If you had to give a score out of 10, the games created from coding versus what people are doing right now with AI versus where it's going. [14:34] I mean, if you can code, of course, you get to another level of sophistication in terms of the complication of the gameplay. [14:41] and all the nuances that make a game fun to play. So, of course, we're not there. The examples that I see are mostly like we do game jams, and we spend like two hours to make a game with the assistant, and what can come out is pretty incredible. But we haven't had...

[15:01] So in the community, I'm sure they're going to take it to the next level. The goal is not to exclusively use Assistant. The goal is to basically combine Assistant and learn skills through Assistant. So maybe at the beginning, the first script, Assistant will write it for you and will attach it to a part. Then you know where the script should go. And then you know how to, I don't know, make some hobby platform move up and down. And then you learn on the way. [15:31] companion that it shows you by doing how you make a game. And then over time, people really develop skills that otherwise it's hard to acquire. Stefano, have you seen that the type of development has changed? As in, the assistant not only has language, like the documentation you described, it does also have code. It even has images. I saw that it's, and we'll get into the technical [16:01] You have images. Have you seen the behavior change for how people develop in Roblox? As in tactically, have you seen the number of people actually coding going up? I would have guessed that outside looking in because the barriers to entry of coding have gone down. But does that just mean that more people are developing in general and the ratios actually stay the same? [16:19] That's a great question. So we have seen, we have measured the productivity of people that use assistant versus not. [16:26] And so we have found that people that use Assistant create 180% more code.

[16:33] Wow. And so the individuals are a lot more productive. That's benchmarks to people who already wrote code. [16:39] Or does that also say just more? Oh, wow. Okay. Two cohorts of coders, one using Assistant and Code Assist. Code Assist is the, you know, suggests code and Assistant I've created from scratch, but they're all kind of integrated into a similar user workflow. [16:57] And then if you're looking at the same cohort comparison for material generator, creators that use material generator create 60% more materials. So also on the art front, there's more productivity. And then if you look at the final goal, which is… [17:14] how much they publish, right? Because publishing the game is the ultimate goal. The lift is about 30%. So people that use Assistant publish 30% more than people that are not using it. And remember that this is now still in beta, mostly. [17:31] It's going to get out of beta soon. I can't give you the exact day, but it's going to be relatively soon. And so we're going to see even a broader adoption, of course, there and impact. [17:45] One more question on this, on Sonia's quality question. Like, [17:49] What about usage? So they're publishing 30% more. [17:52] Do you have any KPIs that actually give you a sense of the assistant games, the hours spent, the Robux spent on them, whatever the KPIs might be? [18:01] if they are also that 30% lift?

[18:04] That's a great question. So our number one KPI right now is around retention. And so we are seeing like a week over week retention of people using a system that is much higher than any other features that we roll out. And also we are seeing that over time, that retention in the long run increases quite a bit. [18:34] The number of daily users has been growing organically and very steady. We don't do any marketing, of course, on this. [18:41] And it's free, which is, I think, the best marketing. [18:47] It's not cheap, let me tell you that. I think Roblox is very generous on that. But we have this unique... [18:55] synergy and collaboration with the community where basically we told the community, hey, give us access to your data to train AI. We are going to make the best AI companion, the best AI assistant that we can. And that assistant goes back into studio and is free, right? So we're not making money off your data. We're actually helping you create more. And so we found the overwhelming majority of the creators in our community gave us permission to use their [19:25] and that's why I was mentioning earlier, [19:28] We have not only one of the largest data set in the world, but also the most multimodal, if you want, because we have we have code, we have images, we have 3D assets, we have audio, video. All that is part of our gaming experience. And also with the interactivity is the glue for all of that. And with the analytics on the usage. Right. So it's a very, very powerful data set that we are very, you know, treating with a lot of respect and a lot of, you know,

[19:58] the best practices to keep that data, of course, very secure, but at the same time allows us to really harvest the value. And ultimately what we are doing is we are teaching AI game development, basically that's what we are doing, right? We're not teaching how to make an image, we're not teaching how to write code, we are teaching game development. That's the ultimate goal. And so it's going to, all these tools there at the beginning will feel a little bit, uh, [20:24] hey, this is a tool to make a material, this is to make the texture, this is to write code. [20:28] we are already seeing that they are converging. We already started that process of converging some of those lower-level tools into larger ones, where basically the AI is actually learning how to develop a game as opposed to just how to do a small task. I love the vision of the AI learning how to develop a game. And the question I have is, if you break up game development into its component parts, [20:58] AI well in the, in the near term versus which, what do you think humans will be uniquely good at for a while? [21:06] That's a great question. So we don't see AI as taking over, by the way. I know. I think the best parlor that we made at RDC a couple of weeks ago was AI is your dishwasher. Nobody wants to wash the dishes. And so we are really focusing on tasks, especially with the last release of assistant actions. We are focusing on the tasks that you don't want to do. Washing the dishes, doing the laundry.

[21:35] so we have [21:36] Introduced amongst the assistant capabilities, a new capability where [21:42] a system can basically make large-scale modifications to the data model. I'll give you an example. I have made a beautiful open-world game with a huge forest. This forest has 100,000 trees. [21:55] Now, all of a sudden, I want these trees to actually follow the seasons and, you know, get more leaves more yellow because fall is coming. The amount of work that will take me to implement that will be huge. A system can do that with three lines of text. I can say... [22:10] select all the trees, or I can say, select all the pine trees. Actually, those don't become yellow. So that'll be the wrong one, sorry. Select all the trees other than the pine trees and make the leaves yellow, right? I can just give these three liners. A system can go, can select, you know, the 57,000 trees, there are no pine trees. [22:31] and can change the color of the leaves for me in just a few seconds. [22:36] That's the kind of task that we are seeing a lot of value. And honestly, this was the feature that the community loved the most of all the AI features we released in studio in the last two years. [22:47] This was by a landslide, the one that got the highest score and appreciation for the community, because, again, it was the dishwasher. We are not replacing your talent, which we believe is irreplaceable, but we actually helping you on the task that you don't want to do. [23:01] I am always so impressed by Roblox's emphasis, genuine emphasis,

[23:07] on community and teaching. You've said this in the past few minutes. You've talked about community quite a bit, about teaching how to code and how to create games, and really uplifting the [23:18] the entire community, frankly. And I just want to say that this is very much true all the way to the core of the business. I got to follow my really good friend, Craig Sherman, to board meetings in 2017 and 2018 at Roblox. And even behind closed doors, you know, years ago, this was always the focus. It wasn't the banality that a lot of board meetings on monetization and financials. It was community. It was community. [23:45] It was uplifting community, teaching them how to code. [23:48] teaching everyone how to use Roblox in a way that actually benefits themselves in their own learning. [23:54] And this dishwasher analogy sounds very consistent with the ethos of your business. [24:01] How impressive. How did you implement it? It sounds like a segment anything type of thing. [24:06] algorithm? Was it a segmentation approach or this particular feature? How did you do it? [24:12] That's a great question. So we basically found a way to... [24:17] give Assistant. So Assistant is really good at generating code. It is based on LLMs and some of them actually we are supporting as open source project like StarCoder and we're trying with our own data. So, [24:30] over time assistance getting better and better generating code. [24:35] Some of the code, instead of running runtime, you can actually run it at the time in Studio. Studio has a command bar where you can just execute some of that code.

[24:46] And so assistant actions basically is [24:49] creating code that is executed. [24:52] directly in studio at edit time, [24:54] And because we integrate in studio has full awareness of the data model of your scene. And so it knows, oh, this is a tree. This is a car. It knows what you did and it has full awareness of that. And so we have combined the ability to generate code with the ability to create using the same code, Lua, commands in studio and the awareness of the data model. [25:24] data model manipulation. [25:26] Do you have these things labeled or is it dynamically determining that this is a car? [25:32] there's a good amount of inference that just happens. [25:36] Cool. Yeah. How do you imagine the creation experience evolves, let's say, 10 years from now? [25:41] How do you see a system devolving? [25:45] So when we met about roughly two years ago and we said, okay, how is AI going to impact creation and where should we start? [25:54] The one paradigm shift that we envisioned, that we thought was going to happen in the industry, not just the Roblox, was... [26:03] a shift between [26:04] find control on the creation to capturing intent. And so I spent quite a bit of time in the Photoshop land where basically

[26:18] There you give control to the color of the individual pixel of an image, even though most people don't need it, but some might. And so there it's all about 100% control, non-destructive workflows, but 100% control on the actual artifact that you're generating. And then we are now moving towards a complete different generation of tools where the tool is successful and can produce good outcome. [26:48] as long as it can capture the intent from the user. And so all the digital tools that we have seen in the last 30 years, it's all about surfacing more control, [26:57] to a user and then the user will figure out how to use that control, right? [27:02] Instead, we are migrating from control to capturing intent. And so we're going to see probably quite a big change. And there's going to be, you know, a thousand startups trying to do it in different ways. And there will be new UX paradigm popping up. But we can see audio is an input. We can see high-level gesture, not just, you know, with your hands, but also with mouse and keyboards. So things that are just providing an input into what you want to do. [27:32] I can describe a word better if I can type and I can provide maybe a concept art and maybe some high level sketching on top of it. So this is very different from having the phenomenally granular control. But at the same time, the velocity is like two orders of magnitude faster.

[27:53] The challenge in all this [27:55] is that for casual creators or people that have a limited amount of time to spend on it, it probably, what AI is providing, it's already good enough to share on TikTok or to make an experience on Roblox and invite your friends over. But for people that want to spend a lot more time, you still have to provide that fine control. And so the challenge is, how do you make a tool that is really good at capturing [28:25] intent but at the same time allows the real pros to iterate with the same level of control of the traditional tools. So that's a little bit the challenge that I think a lot of companies like we are facing in studio, a lot of other companies are also facing. [28:41] We have a really fun episode of training data with this team that created a company called Dust. And one of the founders, Gabriel Hubert, talks a lot about rasterization. [28:53] versus vectorization. And this is definitely your language as a graphics person. They also were Stanford computer vision type folks. And it seems like that transformation has happened. And the vectorization kind of what you're describing, like you can, you can expand it to a great deal based on intent. [29:11] you can shrink it to very small and almost... [29:15] like fractals go down into more and more detail. [29:19] When do you think we will get there? What do you think we'll get to the point where you can say, Hey, this is the attempt. And then you can go in and, and,

[29:26] in a fractal level of detail, change each pixel, also based on intent, but on a much smaller scale? Yeah, that's a million-dollar question. So what we're trying to do is allow a system first to be able to perform those operations. So the same AI that gives me the rough slot machine type of input. I throw in some text, an image. I pull the lever to see what we get. [29:56] them to also be able to go in and do the fine grain change. So, hey, only the trees that are above five feet, can you just change the color of that? And then can you take the texture of the tree and then open up and I'm going to paint over it? And so we want to allow AI to already be able to go beyond one shot and allow for iteration. We believe iteration is a fundamental way people create. [30:26] And so we want to make sure that AI can support that from the get-go. And then we will always have a fallback with some tools. And maybe there, you know, it's going to be more like a progressive disclosure where we don't throw it in front of every user, right? But only the users that want to go deeper, then, you know, we can basically pull the curtain and then allow them to go a little bit deeper. In some cases, honestly, we would just like interrupt with our other tools.

[30:56] in our studio won't become this crazy place where you can do everything, but it's hard to do anything, right? So we are very, I think, committed to doing [31:10] Studio being good at what it does, and then it's okay to have, you know, for really the pros that want to go deep to have a great interoperability with external tools. We talk a lot about studio, but actually the thing that we are the most excited is taking all this AI goodness and bringing it to inexperienced creation. And so we think that's going to be the next frontier. So two aspects in the industry, we are at the very beginning and we are curious to, we don't know what I was going to play out and we are super curious. [31:39] using AI for substantially different gameplay. [31:45] Companies like Ego.live are experimenting on that. And then taking all the ability to create the AI as Unleashed and bring it to inexperienced creation. So I'm there, I'm playing my game on Roblox. Now I want to modify this level. I want, with my friends, create something new that we can play together. Those are completely different type of creation experiences that we're going to see being unleashed by AI. [32:09] incubating in studio. We're making it solid, robust. We're making sure the output is high quality, but we are very excited to bring those APIs to InExperience, and we think that's going to be the real impact. [32:22] I'd love to dig more into that. Maybe starting with just the in-game, the gameplay experience. How do you think...

[32:28] How do you think, from the players' perspective, these games will be different? [32:32] I think they can be, with basically no effort or very little effort from the developer, these games can be more personalized and also they can be always different. [32:43] Like if you play the same game, but there's another lamb and he's aware of your past, all the things that you have done in past sessions, and then you come back, then he can make the game different, more interesting, can change the challenge, can spin a little bit the story. [33:13] it can really morph the game onto something that is really more enjoyable for you specifically. So we think that will be a big opportunity. There's a lot of companies that are now doing experiments. And we are very excited to basically provide AI as a platform and then let them experiment with different gameplays. [33:31] That sounds... [33:32] Stefano, technically very hard. And I remember even... [33:37] Even for the simplest type of Roblox gaming, which there isn't really a simple type, you're generating digital worlds continuously, it's actually a very, very heavy lift. You mentioned having your own data centers, right? Low latency, being able to play with people internationally across the world. How do you think about... [33:57] adding this new level of complexity, this

[33:59] AI inference complexity for gameplay, as in generating the world, varying it on the fly, what will have to change from an infrastructure perspective? [34:10] That's a great question. So I think the first step towards that will be NPCs. So non-playable characters that you hook up to an LLM. That doesn't need massive changes in infrastructure. [34:22] Right. That's pretty straightforward. And I think that's going to be the first rep. And we're going to see what kind of impact that creates. Then we have seen other experiences where they actually want to create the whole world. And of course, there are some challenges there. And let me tell you, I think what I think is the biggest challenge is actually moderation. Because Roblox Safety is our number one product. And we want to keep the platform safe. And we want people to connect with civility. [34:52] people to create anything, of course, the bar goes a little bit higher. And so we opened this Pandora box, but we also have to build the guardrail so things stay positive and [35:08] And we have, you know, everybody can have a positive experience on the platform. So I think that is... [35:13] more of a challenge than latency and infrastructure, because we can use CDN, we can cache things, we can pre-generate some of the content. [35:23] We can use a level of detail. So there's a lot of things that the gaming industry has developed in order to basically stream. We are using streaming as well, right, in the platform. So we can basically in real time generate and stream more assets. I think the challenge will be more like if you have now the control of the game that you're playing, how do you make sure that everybody else has a great experience? I think that's going to be the challenge, both on the moderation side and also like in making it fun.

[35:53] for game developers because they know how to make things fun. And not every player has mastered the same skills of making games fun for the last 20 years. So, [36:05] It's like [36:06] How do you give this freedom to create while you somehow control the gameplay and the story so it stays compelling? So these are all things that we are very eager to learn from the community, honestly. And I think that's the beauty of being a platform. [36:22] We don't need to have any opinion about it, and we don't need to figure it out ourselves. We just provide the APIs to the community, and the community with the infinite creativity it has, we'll figure it out. [36:36] So I've heard NPCs as that first stepping stone in game AI frequently. What do you think is going to be the next stepping stone for how game mechanics or gameplay might change? Because I imagine you're not going to go from NPCs to entire worlds, right? What's the second stepping stone? [36:55] I've seen games where [36:58] they limit the creation to one specific item. So for example, there's a very popular game on Roblox called Build a Boat. You go into this game, you're building a boat, and then you're sailing that boat, and then you're racing other players. [37:13] And so you don't create the world. The world is predefined by the developer of the experience, but you're basically allowed to constrain creativity as in, these are the materials that you can use to build a boat, and this is the size that it has to be, and then you can go crazy and build whatever thing you want. And because Roblox is such a physics sandbox, we have our dynamics, further dynamics, and so we can actually see

[37:40] what you create and simulate the wind, and you're going to see the outcome of what you created. [37:45] And so I think experiences like that right now are a little bit more difficult without AI. And AI, I think, can really... [37:54] power those up quite a bit where you can create your race car you can create your boat your airplanes like some elements of the gameplay uh the uh based on what you create you can have you know an advantage uh in the game but you're not completely changing the game itself constrained creatively that's a great yeah exactly [38:13] I have two follow ups from that. The first is you just mentioned physics engines and there's been a lot of enthusiasm for years now around pins and these neural physics engines, etc. Is that now in the physics engine at Roblox? If you can disclose, are you using neural networks as part of the physics engines as part of the you mentioned aerodynamics? Right. Are you guys at the point where you're estimating Navier Stokes because it's cheap enough because you can use neural networks or is that maybe in the future? [38:42] And then my second question is about NPCs. Would you guys ever allow a world or a game that is purely NPCs, as in something you watch as opposed to something that you participate in? [38:52] That's a great question. So on the physics side, I, [38:57] I think the use of neural networks for physics... [39:01] is not being proven as effective. I think there is more, like there's an infinite amount of like real world approximations that you can use that are like very computationally efficient. And I think that's more important than how good you're approximating the word. And so on that front,

[39:19] I'm not sure. Neural networks are going to provide that much value, honestly. [39:26] um [39:26] Especially when you have like already implemented all the basic functionalities and people are already using them successfully. Especially for a purely digital world. I mean, if you're trying to – if there's a noisy real world, I imagine it's different from like Roblox world where everything can work in the physics estimations. Is that fair? Yeah, that's fair. So maybe I would say that's all I can say. It's basically maybe. I think at the moment it's still not fully proven as a path. [39:56] on the MPC front, [39:59] If you think about a game... [40:01] where all the players are NPCs and you're just watching it. To me, it sounds like TV, right? Yeah. So people watch this all the time. And these NPCs can be super smart and do super fun things, can race, can do whatever they like. And maybe it's just fun to watch. So definitely we are seeing such a huge community of people that just watch other people play games. And so those players could be LLMs in the future and still generate great entertainment. [40:31] fun so i will not exclude that i think at the beginning we're going to find a hybrid model um like which is like you know npcs existed where people will inject npcs in the game to just make it more interesting and have people that you know you also can populate games that just launched they don't have you know a thousand concurrent uh players yet and so you can basically populate

[41:01] the real people there right at the beginning. So I think there's going to be a lot of potential on that front, for sure. What about for user-generated content? Like, how do you imagine UGCE changes with now that generative AI is really coming into the experience? That's a great question. We have seen some of this... [41:20] With the Gen.ai features that we release for avatar creation. So avatar creation is another huge area of creativity. There's a massive community in Roblox. They work on making assets and clothing and accessories for avatars. [41:35] And a lot of people make a living doing that. And so we have found that there were a lot more people, like always, that had amazing creative ideas but may not have the tools to do 3D modeling, for example. 3D modeling is a very narrow, specific skill. They will have to learn Blender or Maya. And so not all of them were up for the challenge, but they had a great idea about what outfit to create. And so there we experimented with using images as an input or text as an input. [42:05] and generate avatars, and we found that it substantially democratized [42:10] how many avatars could be created. And this is an area, we launched like an early beta of our avatar auto setup, and we are doubling down that. We're going to do a lot more in the coming months that you're going to see. But basically, we want to allow people to create avatars, clothing, accessories, with just multimodal easy input. Again, there's more people that have great ideas than people that can actually execute on those.

[42:40] for that type of creative expression as well. Do you think that generative AI blurs the line between what it means to be a user versus a creator? Like, do you imagine those two become kind of the same thing? And I imagine you think of them distinctly today. Oh, for sure, yeah. And I mean, think about, you know, what happened in the music scene, right? The boundary between the composer [43:03] And the people that could play an instrument and the audience were just so set at the beginning. And then, you know, I think karaoke completely blurred out, right? And then now everybody can create things on GarageBand. And now with AI, people can just type lyrics and the genre and then a full song is created. So there, like, the boundary got completely blurred, I think, in the audio and music space. [43:33] of content, but I think it's just entropy only increases, and so we're going to see that for sure. Stefano, you mentioned that you guys have an absurd amount of data. [43:46] You guys have an insane amount of video data. [43:49] I think Robux is still the number one VR application in the world by a lot, including on VR headsets. And then also, obviously, personal computers, all sorts of devices, et cetera. Ton of audio data. [44:03] ton of textual data, like [44:06] Kind of everything data. [44:07] And so you mentioned this and then kind of walked by it, but obviously Sonia and I are very curious with all that training data. Yes, the name of the show.

[44:17] Are you guys going to have a world model? [44:19] I think I know the answer to this. I think it might even be announced already, depending on when this airs. Are you going to have a world model? What's it going to look like? What will be its boundaries? And where do you go from there? [44:29] Yeah, that's a great question. So yeah, we do have an enormous amount of data. And I think the challenge for us is less about gathering data and more like being able to use it. Because there's a lot that goes into from the raw data to be able to train an LLM or AI in general. So that is where the work is focusing. [44:59] Open source that. [45:00] And basically, [45:02] What it does is it will allow the digital synthesis of scenes and word from multimodal input. So that's the goal. And I think also we can go a little bit beyond that. As I said before, our goal is basically to teach game development to AI. [45:21] And so it's not just about world creation. I think that will be the first maybe area that we can really attack in a meaningful way. But then there's going to be all the interactivity of that world, the ability for things to move, bend, doors to open, characters to run around. And then the next level will be all the interactive gameplay. [45:43] And so we are seeing the data, and we also can see from the data what works, what is fun for the user. And so we will be able to guide AI to not just make games, but actually make games that are fun, because you can see which games get more traction or which specific levels get played more.

[46:13] like, hey, I'm just going to generate whatever you ask me, but I can also, like, if I can pick, I can pick things that are, like, more fun to engage with. And so, again, that one is going to be, you know, it's going to take a while to figure out, but, you know, we have the data that they can support it. I think it's more about on us to be able to, um, [46:35] pick up, you know, there's going to be a lot of like unsupervised learning that we have to do. And so we have to pick up, we have to find ways to pick up signals of intent. [46:45] from our creators, [46:46] that then we can use in an unsupervised way to classify things and then figure out what AI should be learning from. [46:53] I imagine because of the physics engine, [46:56] You guys have such a competitive advantage with object placement, object interaction. [47:02] The struggle with a lot of training data for these video and role models is, [47:07] Like they're two-dimensional. [47:08] And there's no concept of, I mean, maybe you can infer from it, the concept of what is three-dimensional and how objects interact. But is that something that you guys are leaning on particularly heavily? I'd imagine it's a big advantage to the three model you create. [47:23] Yeah, it's a huge advantage for getting also consistency over time and spatial consistency. If you actually have the full 3D model and if you have a full 3D scene, you know, then you can anchor the spatial and the temporal coherence in a much stronger way than if you're estimating. [47:43] those things from 2D. I mean, the classic example is if you're trying to stylize video and you use just video to video, you're going to see drifts and artifacts and everything. Then with ControlNet and the depth map, which is like a 2.5D approach, this video to video type of stylization became a little bit more reliable.

[48:08] and more consistent temporally. [48:10] But you still don't have... I mean, you have basically the situation where if a character is looking at you and then you stylize that face and then they look away and then they look back, they're like a different person, right? We have all seen... [48:22] those examples. And so the only way I think to overcome that is to actually train with 3D data. [48:29] And the industry went so fast, so far, with like just 2D. [48:35] and is now making magic with 2D. [48:38] And I think there's still like the work to incorporate 3D information into those algorithms still has to come. And I think it's going to bring to fruition this like temporal and spatial coherence that right now we don't have or that is so hard to, you know, to generate out of 2D data, right? Because it just is not rich enough. Just to make sure I understand what you just said. Do you think that 2D is going to lead to a dead end and you have to start over from 3D or do you think that 2D can kind of get there with scale as well? [49:07] I think if the goal is to operate on a single image, 2D has done fantastic. [49:13] And so if you need to text your character, or like a stylized image, all these things, it's fantastic. If you want to actually [49:20] have a video. So if you add the basic time on that axis, [49:25] And now you are also maybe moving the camera. So the camera is moving, and then there's like 2,000 frames ahead of you. They have to be consistent. At that point, if you're not using 3D data, you're at a disadvantage, and you're going to be dealing with drift. And yes, you can enforce it in many possible ways, but the problem of 2D data is that it has occlusions, right? So if you're looking at me and my arm is behind my back,

[49:51] and then at some point my arm pops out, you have nothing to enforce consistency of the look of my hand because you just have not seen it before, right? So all these problems don't exist if you're operating on a 3D representation of your scene because even the data they are not seeing from the camera is there and it's available. So I think the industry has... [50:12] Gone like [50:13] I made amazing progress in trying to cope with that. [50:19] which is a fundamental lack of data. And then if that data is actually used and can be incorporated in, let's say, a stable diffusion, then we're going to see a much better outcome. Steph, in our homework for this episode, we heard that you're passionate about neural rendering. Can you tell us what that is? [50:36] I don't know where you're at that but yes it's true I'm very passionate because I have a [50:43] Strong belief that neural networks will change the way we're going to stylize games and make games visually a lot more compelling than they are today. [50:53] And so if you look at the history of game development, the creation of assets and the style of the game were always tied together. If you're making like Super Mario and that's the style of the game, you can't just like on the flip of a coin say, okay, make it look like Call of Duty. It would just not work. You will have to redo all the assets from scratch. [51:14] But if you're using neural rendering or generative rendering techniques, you're able, with some text description and maybe some reference images, to restyle your game in real time.

[51:26] And then, you know, you leave the geometry where it is, so it's physically consistent, right? Your physics capsule is in the same place, but the visual look can be very different. And this can be used also to make games completely photorealistic, even though you're using, you know, some low-resolution meshes and textures, right? You can use this as a final pass. Like, you know, in games, you add, you know, typically bloom at the end to make things look really cool. [51:56] make it photorealistic or stylize it in this specific way that I like. And you can restyle your game in a way that actually doesn't add any extra assets that you have to download. It doesn't change the assets. It's really like a compute layer that happens at the end. Right now, it's very compute intensive. [52:17] But, you know, looking at the speed at which things have moved, you can imagine that in the future, developers can just add this filter to their game, describe the style, give a reference image, and then have the game look beautiful without changing a single geometry in the assets. [52:35] So I'm a believer that in five years, this will be the way games will be built. And it was going to run also, at least on the high end phones. And yeah, we're very excited about it. [52:47] Steph, does it ever become the end? [52:50] gamer's choice the way it's rendered as an I put my skin or style

[52:56] into whatever game i'm playing question i think if the the if the developer of the game allows that freedom to to the player why not right i think it's going to be an artistic choice and you can play my game and you can make in any style that you want and again you can still play with people they will visually see a different game but the game is still consistent all the physics all the gameplay still still hold [53:20] Stefano, thank you so much for joining us today. We learned an incredible amount, certainly about the scale and complexity of Roblox, not just from technology, but also the scale of impact. [53:31] the 79 million daily actives, the hundreds of millions of monthly actives, the amount of technical depth that's necessary to get there. We learned about how you're using AI as dishwashers to empower your customers and developers and gamers, and not just in the assistant, but also in the code assistant and the material generator. And we learned a little bit about what you think the future is going to look like. We might be using infrastructure for guardrails to make sure that the stability of Roblox is preserved. [54:01] We might be watching some NPC television, and we certainly will have a 3D world created by Roblox. Can't wait to live in that oasis. [54:11] Thank you so much for having me. It was a pleasure. [54:13] Thank you. [54:14] Thank you. [54:15] Thank you.