Running benchmarks with Vulkan in Crostini (Linux for Chrome OS) yields surprising results

We recently cracked the case of how to get Vulkan graphics acceleration working in Crostini. I’ve got a full guide on how to do this yourself coming up as my next article so stay tuned for that if you’d like to be able to tinker with this a bit yourself! With the power of Vulkan, we wanted to see how well this all performs inside and outside of a the virtualized Crostini environment. Spoiler alert: the first results aren’t great.

Let me first start with a disclaimer. There’s a reason Vulkan isn’t enabled for the masses: it’s not ready yet. We’re seeing a large performance drop comparing a game running in and outside of Crostini. This new graphics pass-through driver, code-named Venus, was only merged upstream a few months ago and is truly revolutionary. There could be lots of patches behind closed doors at Google that we don’t know about. We only have what is publicly available to us.

The good news is is that the Canary update channel is tracking all of the latest upstream changes. I’ve been keeping my Chromebook in that channel, but be warned about Canary: you will encounter problems. It’s not a matter of if but when. In fact, Gabriel and I both ran into Vulkan support breaking completely just last week. That was huge a scare, but thankfully everything is back to normal again on the latest daily build.

Enough back story. Onto the benchmarks! For this, I used the Vulkan mod for Quake known as vkQuake. Why such an old game? Because my Chromebook is limited to Vulkan 1.0 support. Most modern games do not support this older version. I’m doing my best with what limited hardware I have. Speaking of which, these tests were performed on my Google Chromebook Pixel 2 (2015). It has an Intel Core i7-5500U processor, Intel HD Graphics 5500 integrated graphics, 16 GB of RAM, and a SSD.

As a baseline, we had to benchmark this game outside of Crostini. The easiest and fastest way for me to do this was to boot up my own Mac Linux Gaming Stick distribution – which is based on Manjaro Linux – from a flash drive on the same Chromebook that I used to test in Crostini – the virtual machine Linux on Chrome OS. The average frames-per-second (FPS) was 196.

With that information to compare with, we can move onto the benchmarks of Vulkan in Crostini. I’m using an Arch Linux container which isn’t exactly Manjaro Linux (since it has slightly newer packages), but it’s close enough. I was very surprised to see that the average FPS was 102. That’s a performance penalty of about half. Wow. Also notice that the frame rate consistently swings between a high and low frame rate.

Before we jump to any conclusions, let me point out a few things. This is an old game. It’s also possible that there is exponentially some sort of performance issue with the game running in a virtualized environment. With newer Vulkan support it may run even better. The two different environments are slightly different and I couldn’t use Crouton on Chrome OS for custom Linux containers because Arch Linux is not supported with it. Also, this is only one game, so it’s hard to draw any concrete conclusions. We plan to do more extensive benchmarking in the near future.

And, yes, we are testing more games than just Quake. DOOM, DOTA 2, and Portal 2 are just a few of the games Gabriel is testing out on newer Tiger Lake Chromebooks. The experience is still rough around the edges with occasional graphical glitches in games like DOTA 2 (which, ironically, has been confirmed to be a game Google developers are actively testing) but, again, Vulkan support in Crostini is not ready for prime time yet. A video will be coming soon with a first look at this all in action so be sure to subscribe to our YouTube channel as we continue the run to 200K subscribers!

What about user experience?

Vulkan is also only half of the equation for Borealis on Chromebooks to become a reality, though. The other half is the user experience. On our last podcast, we talked about how there are lots of little annoying problems with running Steam games in Crostini today. Constant crashes, mouse capture issues, performance problems; you name it. It’s a long laundry list of issues at this point. The hope is that Google and Valve are solving all of these problems with Borealis. My prediction is that it’ll be shipping with Steam OS, the same operating system found on their new and upcoming Steam Deck, which’ll have a lot of gaming optimizations.

Let’s hope that Google and Valve can continue to improve the performance for Borealis by the time it launches. This is a large and complex puzzle, so I fully understand the skepticism people may have with them trying to pull off launching Steam on Chromebooks and even their own Steam Deck hardware.

“[Steam] has a lot to learn before [it’s] ready to save anyone. My brother thinks I’m crazy, but I believe Aang [Valve] can save the [gaming] world.” – Avatar the Last Airbender