Valve Steam Frame interview — why it costs up to $1,300, Snapdragon power, and 10x foveated streaming

extract

Valve Steam Frame interview — why it costs up to $1,300, Snapdragon power, and 10x foveated streaming Taking a two-pronged approach to VR gaming Prior to the official launch of the Steam Frame, Tom's Hardware Premium had a chance to talk with Valve software developers Jeff Leinbaugh and Pierre-Loup Griffais. We discussed the development process behind the Steam Frame, the standalone and streaming modes, networking strategy, and pricing, which you can read in its entirely here. You can read our review of the Steam Frame here. Like previous Valve launches, you can also read transcripts of our interviews with the company on the Steam Machine and Steam Controller. The transcript below has been lightly edited for clarity. Brandon Hill, Tom's Hardware: What were your goals in creating the Steam Frame and providing the framework necessary to support it, compared to legacy hardware like the Index? Jeff Leinbaugh, Software Developer, Valve: Yeah, so the work started on Steam Frame basically as soon as we shipped the Index. And so we did a lot of exploration into technologies and features and use cases to try to answer that question. And the goals that we aligned on for what became the Steam Frame was to make [it] much easier to use, which to us meant a lighter, more comfortable, easier to get into and out of VR headset. That was a better way to interact with your whole Steam library than the goals we had for the Index. Where the Index was just: we wanted to make the best possible VR headset that we could make. And that came with a lot of compromises, which were worth it because they allowed us to have this really great VR headset and see what that was capable of. But this time around we thought we could achieve something that was a lot more comfortable, a lot more accessible. And so streaming technology was a big part of that. So the headset can be wireless so that there's not a lot of complicated setup to get to the game, whether that's playing on the headset or streamed. And then the other big part of that accomplishing that goal was making it possible to interact with Steam just like you would on a Steam Deck and to be able to play games that don't even support VR in a virtual display. So what we want to be possible is for somebody to put on the headset, browse their Steam library, and then just pick something that they want to play and not worry about too much else. And that's kind of where all the user experience goals, at least, came from around this. And then there's a lot of other technical goals that support that we can get into if you're interested in any of those. Hill: So, that leads into my next question: what were the engineering challenges in making the Steam Frame a standalone device and one that can also stream? And I know you've got the wireless dongle to help with the networking offload. Leinbaugh: Yeah, a lot of the engineering challenges were around making sure that we had the right kind of compute, and enough compute on the headset to both support a really good streaming experience, but also to be able to run games on the headset. So we picked the Qualcomm 8650 chip. It had a lot of capability, but in order to make use of that, we had to solve a lot of engineering challenges around thermal management and power delivery. So really treating this like making a really capable PC that just happens to run this particular [...] mobile hardware was how a lot of the engineers approached that problem. And then from there, we kind of tied that into a lot of the work that we did for Steam Deck and Steam Machine running Steam OS now on ARM, but it's the same Steam OS. So at some point we got to kind of meet in the middle and leverage Proton and Steam OS and a lot of things there to start getting games running. Standalone mode & software support Hill: And what kind of performance levels are you targeting for games using the Steam Frame when it's in the standalone mode? Pierre-Loup Griffais, Software Developer, Valve: I guess it's hard to pinpoint a specific segment there because there are so many games on Steam that keep coming out that are running great on there, right? Sure, games that are trying to push the boundaries on the AAA stuff that's very graphically intensive. Most of those recent ones won't run very well because it's a smaller compute envelope, right? But there's a ton of stuff that is coming out that is a great experience on the Frame. For example, I think last week the Metal Gear Solid Collection Part 2 released. And, you know, MGS4 is a game that I haven't played in, I don't know, 20 years at this point, but it's great from there. And even at the higher resolution, 60 FPS is locked. So it's really cool to see games that are coming out right now that are great experiences on there. I think there's a ton of indie games and things that run really well too. And if you go down the back catalog, there's a ton of stuff from a little bit ago, even AAA stuff that runs great as well. So I think the point is that you're able to engage with your whole library, right? And then have kind of a similar thing as the deck where you're going to try and figure out if some games are maybe a little bit below the bar in terms of what you would like for performance. And then off to stream those from another PC instead, or the Steam machine. Hill: And building on that, how's the reception been to the Steam Frame Verify program from developers for the standalone games? Griffais: I mean, I think we have a lot of developers that are pretty eager to get their games working well. And so we've seen it approached from different ways. Some folks do tweaks to their existing PC version and then get to a very good point with that. But just the same kind of tweaks as you would expect from Steam Deck, where some surface-level tweaks for the user experience, like changing controller bindings a little bit or changing default settings in a way that really makes the game better for everyone and not just for Steam Frame. And we're doing the same for Half-Life Alyx, where we're doing a lot of work to support the Frame more natively and trying to remove performance barriers that are induced by some of the translation, the emulation there that you get on normal and modified PC games. And that's yielding great results to get to the next level of performance there. And so we expect that some developers will be doing some of that work or maybe adapting some of that existing work that they've already done on an Android version of their game and then adapting that through Lepton as well. So I think everyone's kind of engaging with it at a different pace there. But overall, I think like we're talking to a bunch of VR devs that are excited to get their games to be labeled as verified or playable. Foveated streaming and technical hurdles Hill: Could you go into a little more detail on foveated streaming and how it enhances the gaming experience with Steam Frame? Leinbaugh: Yeah, so the Steam Frame has eye-tracking cameras on the inside of the headset that know where your eyes are looking. And when you have that, there's a lot of potential use cases, but the one that we're using most consistently is foveated streaming, which really means when you're streaming, especially VR content, there's a lot of video data that you have to send. And so you're therefore very sensitive to how much available bandwidth that you have available for not just transmitting it over a wireless link, or whatever kind of link, but also just encoding and decoding that data. All of that takes from a pool of finite resources and your network stack or your graphics card. But because we know, with eye-tracking cameras, what part of the image you're looking at at any time, we can refocus some of those resources to make sure that the important parts of the image are treated as first-class bits. And then the rest gets the last part of the resource pool, which is the part of the image that you can't see, and you won't notice the image degradation. So instead of kind of evenly focusing all of the encoding and network resources around this entire immersive, all-encompassing VR image where you're not seeing most of it, we make sure that the part that you're looking at is encoded differently in a way that's a much, much higher equivalent bit rate. So the best analogy really is [that] it's almost as if the part that you're looking at is using about 10 times as much bandwidth and encoding bit rate as it actually is for the entire image, because we just squished it all into that part that follows where you're looking. Hill: Switching gears slightly, how are you compensating for temporary losses in connection or signal strength when gaming wirelessly? Is there a fallback mechanism? Leinbaugh: Yeah, there are a few different levels of what we'll do, and we've made sure to do something in response to all these things happening. So the first thing is if you're using the wireless adapter, if you also are connected to your home WiFi, for example, which a lot of people just naturally will be, then the streaming, VR streaming will take advantage of both of those connections. And so you'll have the adapter and your WiFi both active at the same time; we call it multi-link streaming. And what that means is that those two connections are competing against each other constantly, every single packet. So there's a fallback mechanism built into that, which, if you have two links active and one of them drops out because of interference, or you walked out of range, or all the many reasons that wireless interference can happen. If something happens to one of those links, the other one will immediately pick up and cover the gap. And it's just very unlikely that a lot of those sources of interference will hit both links at the exact same time. So that's kind of the first line of defense that we have. If you still don't, if you still have an interruption or maybe you only have one link active, then the headset has strategies to fill in those gaps. So if you have a short interruption, just like a latency spike or something, then a previous frame will get reprojected and replayed. And that works pretty convincingly for covering small gaps like that. A lot of people won't even notice that it's happened. If you have an extended dropout where the video is just not arriving at all for whatever reason, then at that point, the video stream is not going to recover. If it's a very extended dropout, but the headset will fade you back into the construct environment so that you're at least not getting stuck with a really unexpected experience or something that might confuse you or disorient you. So the worst-case scenario is you'll just kind of fall back to the default environment and have a chance to either wait for it to recover or do something. Maybe you unplugged your Wi-Fi router or something that you need to go do. Pricing and market conditions Hill: When the Steam Machine arrived, there was a little bit of sticker shock with the price when it launched. And the Steam Frame is also around $1,000, close to $1,300 bucks in top trim. Surely Valve is sensitive to the realities of the marketplace with RAM and storage prices. How do you appease the gamers out there that are begging for some relief in hardware pricing? Griffais: Oh, yeah, I think as far as memory prices and those kinds of conditions, we're really in it like everyone else. Like, there's not really anything special that we can do. We've been working hard on trying to get the parts that we can get at the best price possible. But I think everyone's affected. [...] The fact that our price was affected by that compared to what we were hoping for is definitely true. But I'd say that it's trending along the reality of the market. And there's not really any sort of the kind of distortion compared to just the price of the parts in there. And it's an unfortunate situation all around. We wish that gaming hardware was cheaper, but we're definitely trying to be on the cheapest side that we can get despite all those conditions. It's not just memory and all that too, right? Even just things like motherboards, copper, anything; even shipping anything worldwide right now is way more expensive than it was last year and two years ago. So every part of the sticker price is affected there. Hill: Final question: Are you planning to implement the same reservation system that you used for the Steam Machine with Steam Frame? Griffais: Yeah, I think that's the plan. [Interview ends] Brandon Hill is a senior editor at Tom's Hardware. He has written about PC and Mac tech since the late 1990s with bylines at AnandTech, DailyTech, and Hot Hardware. When he is not consuming copious amounts of tech news, he can be found enjoying the NC mountains or the beach with his wife and two sons.