While we prefer to get these progress reports shortly after the turn of the month, sometimes things happen out of our control. June wasn't exactly a slow month, but it was backloaded with tons of changes that we weren't expecting to get merged so soon.
Due to some big late merges, the Progress Report will be a little late this month. But our blog writers are working hard, and hopefully it will be ready soon! pic.twitter.com/buZTILauRl— Dolphin Emulator (@Dolphin_Emu) July 2, 2018
Sometimes delays are inevitable, but the notable changes that we were able to include thanks to extending the deadline should more than make up for the several day wait. You've been waiting long enough - without further delay, please enjoy this month's notable changes.
We usually try to keep things in close to chronological order for Progress Reports, but this one is a bit of an exception. DolphinQt has been developing at a pace that none of us could have anticipated. With spycrab0 at the helm and many others pitching in, there have been hundreds of changes and fixes merged in the past few months. That's not to say there hasn't been some growing pains, but Qt has slowly been winning over users in recent development builds. Excitement mounted as DolphinQt went from a curiosity to the dominant backend in the last couple of months.
As DolphinQt blossomed, DolphinWx was relegated to the secondary executable. While we did not promise to maintain it, keeping it around came at a cost. A lot of the major GUI changes that have been long awaited had to bide their time a little bit longer until Wx was completely removed. Further more, many developers were tepid toward the idea of working on features that would require GUI work while there were two GUIs to worry about. Dolphin was marching toward a stalemate with the Qt GUI reaching maturity sooner than expected.
Many times over, we've opened ourselves up to criticism by removing features that were still in heavy use. Some removals have gone smoother than others, but it's never easy. You can find controversies around the removal of asynchronous audio, the sudden decision to axe the x86-32 JIT, D3D9, D3D12, and many, many others of varying magnitude. DolphinWx joins the litany of others that line a graveyard of features spanning a decade and a half. Was its removal sudden? Absolutely. Did it have to happen right then? Not necessarily.
The other side of the argument was that while Qt had only been the default for a couple of months, it was already the majority GUI among all development builds! When we narrowed down the numbers to only include revisions where Qt was the default GUI, the landscape became much clearer.
By all accounts, Wx was a deprecated GUI hemorrhaging users, especially in the most recent of builds. It didn't really made sense to keep it around if it meant stagnating Qt. And since the Wx removal, work has already started on more drastic departures from Dolphin's classic GUI, such as a visual controller configuration page along with some initial ideas to make editing Dolphin's many INI files easier going forward. Last, but not least, with DolphinWx gone, we can finally continue work on Dolphin's configuration rewrite that got stalled in order to make the transition easier.
All we can ask going forward is that users keep voicing their displeasures about DolphinQt until it does everything that DolphinWx did but better. Later on in this Progress Report there are a litany of Qt changes that mostly came from suggestions and reports that we heard on the forums and social media from concerned users. For those of you already content and especially those that have reported issues, we thank you for riding out the initial rollout of this feature and hope you'll enjoy many of the exciting additions yet to come.
Qt Feature Roundup¶
Over the course of June there were a lot of smaller Qt changes that have greatly improved the user experience. Here's a list of some of the improvements you can see right now!
- 5.0-8138 - Greatly improve performance. Makes the Gamelist run much more smoothly, especially when changing window size.
- 5.0-8199 - General housekeeping. Fixes duplicate options, memcard manager blocks remaining not immediately updating and more.
- 5.0-8251 - Fix gamelist column resizing issues. Resizing the various gamelist columns in Qt actually works as intended now.
- 5.0-8271 - Add ability to create a memcard if one does not exist when selecting a memory card file.
- 5.0-8273 - Add multi-select feature to gamelist. Select and do actions on multiple games in the gamelist, such as compress/uncompress.
- 5.0-8291 - Fix address search focus issues.
- 5.0-8303 - Reduce startup time by changing how the graphics window is initialized.
- 5.0-8311 - Reduce startup time by changing how the hotkey configuration window is initialized.
You may note that some of these features were merged after the cut-off for the Progress Report. When DolphinWx was dropped, there was a wave of (valid) complaints about why they were hesitant to move over to Qt. And the ones we mentioned here are just the beginning - there have been hundreds of Qt changes over the past couple of months that didn't make it into the Progress Report as various minor issues were chipped away. Adding to that is that there are even more on the docket to come forward fixing some edge-cases and features that aren't as commonly used.
5.0-8009 - JIT64: Fix and Enable Accurate Double2Single Conversion by JMC47, degasus, and phire¶
This is a bug report that would have never reached Dolphin had it not been for some incredibly devoted Super Mario 64 players hunting for their white whale. This isn't the first time the Virtual Console version of Super Mario 64 has come into play, as there were previous strategies that relied on differences present in Nintendo's N64 emulator. But to really understand why this strange little bug only present in the Virtual Console version of a game was important to emulate, we need to talk about one of the most notorious challenge runs in all of gaming.
Super Mario 64's A Button Challenge¶
One of the more interesting uses for emulators are TASes (Tool Assisted Speedrun). While most of these superplays are actually speedruns, several groups have taken toward completely crazy goals in some of our favorite games. The Legend of Zelda: Ocarina of Time recently had its "No Doors" challenge completed and many fighting games have glitch exhibitions that would drop the jaws of casual observers. One particular group of Super Mario 64 TASers has taken up the task of completing Super Mario 64 in as few A presses as possible. Considering that the A button is used for jumping, this challenge has become legendary for producing crazy solutions to once simple problems. In their endeavors, they have warped time itself and explored parallel universes all in the name of saving a single A press. Some of the most popular videos have over a million views has the challenge has captured the hearts of gamers around the world.
How exactly did Dolphin get dragged into this? It all started one night when a video was linked to the team of a new glitch discovered in the Virtual Console version of Super Mario 64 that would potentially allow them to bypass that final A press. This bug was a bit peculiar - it only happened in the Wii version of the game. Not the Wii U, not the N64, only Wii. So when people showed up saying that Dolphin didn't emulate the bug, of course it drew some interest.
Floating Up to Nothing¶
Within Bowser in the Fire Sea, a certain type of platform would float up and down in the lava presenting Mario a challenge to cross it without getting burned. On the Nintendo 64, these platforms worked as expected floating up and down as one would expect. On the Nintendo Wii, however, the platforms would slowly oscillate toward the origin (0,0). Because Bowser in the Fire Sea was a level with vertical progression, the origin was actually above where the player spawned. That meant the bottom level of platforms would very slowly drift upwards during their oscilations. If you were to wait 4 days for the platform to rise, you could combine that with Vertical Speed Conservation from a lava boost and punch your way to the next part of the level!
This behavior didn't happen in Dolphin! After hearing someone say that Dolphin wouldn't ever be able to emulate the feature, JMC47 dove into the issue head first. He quickly came up with a test and within hours had Dolphin reproducing the bug just fine while falling back on several instruction sets. After another day of testing, he had narrowed it down to an issue with converting Doubles to Singles.
Our readers that love our technical pieces are going to be happy, since we can't pull any punches with this one!
To really understand this glitch and the solution, you'll need to understand a bit about Floating Point Math. Floating Point is basically a speed hack for mathematics, akin to Significant Figures. It essentially trades accuracy for speed through controlled rounding. The "controlled" part is how far the decimal point goes before it is rounded; that is precision of the floating point calculation. For example, "Doubles" (Double Precision Floating Point, 64-bit) has both higher precision (more decimal places) and range (larger min and max values) than "Singles" (Single Precision Floating Point, 32-bit). This all may seem counter intuitive, but Floating Point allows efficient calculation at the scale of the universe or size of an atom level numbers. Of course, the tradeoff for that imprecision is the potential for rounding errors. For a full and proper explanation of Floating Point and how it is computed, please see this excellent video on the topic.
So how does all of this affect that specific platform in Super Mario 64? All objects in the game are tracked via floating point numbers, and move via floating point calculations, but weirdly enough the calculation for the platform is initially performed in Doubles, and then converted to Singles. We have no idea why it does this instead of just using Singles but, it does! The platform moves up and down in a sine wave to imply that it is floating on lava, but due to floating point imprecision, it has a very small chance that the number will be wrong and the platform will end up a tiny faction off of where it started from. On the N64, the rounding mode is set to round to nearest floating point number (round to nearest), so over time it will even out any inaccuracies and never drift too far.
On the Wii VC version, in an apparent bug in the VC emulator, the rounding mode is not sent to the Wii's CPU, so the Wii's floating point hardware just uses it's default rounding mode of round toward 0. So every time the Doubles to Singles conversion had to round the result, it would cause the Bowser in the Fire Sea platform to move ever so slightly upwards toward the origin.
So now the question becomes, Why wasn't this happening in Dolphin? As with many other things, Dolphin has a hardware fast-path and a interpreter slow-path for performing Double to Single floating point conversions. The software based slow-path recreates the Wii CPUs floating point error just fine, allowing interpreter to work with the rising platform glitch. On the fast-path, the conversion is handled by the floating point hardware on the host CPU. The Wii VC emulator's bug of not sending the rounding mode remained, but the floating point unit of a host PC's x86 CPU doesn't default to round towards 0 as a Wii would, instead it defaults to round to nearest! Interestingly enough, that just so happens to be what Super Mario 64 actually wanted! While all of this sounds great, we're not an N64 emulator - we're a GameCube/Wii emulator. And the Virtual Console version of Super Mario 64 is supposed to have a bug, so with this behavior we weren't quite doing our job.
JMC47 verified that the fast-path was responsible for this inaccuracy, and discussed the issue with the team. His way of fixing it was disabling the instruction in the JIT so that Dolphin would fallback to the slow-path version of the instruction implemented in the interpreter, which didn't have this flaw. degasus and others on the team realized there was a better way of doing this - phire had already written a software-based Doubles to Singles conversion for the JIT many years ago. Because it was disabled and thought to have no real impact on any games, it had fallen into disrepair in recent years so it couldn't just be enabled. Instead, degasus repaired phire's code to work in modern Dolphin and JMC47 did his own test and passed it around for others to verify.
And just like that, the platform finally behaved correctly in Dolphin with the JIT!
With phire's repaired code proving to be more accurate, it replaced the previously active fast-path in Dolphin. Since it is only affecting one instruction, the performance difference was so minor within games that there was no visible change in framerate! Despite making a difference in Super Mario 64, no one should expect any major differences in other games. This case was a very strange one that no one could have anticipated! On a final note, if you're curious how the finished run would look, the TASers have hacked the glitch into the version of Mupen64 they use for TASing in order to demonstrate how the level would be completed without a single A press.
Much like some previous Virtual Console only strategies, it's possible in the future they'll find a way to get past this blockade on Nintendo 64. Regardless of if they do or not, their efforts have helped make Dolphin a little bit better. And for that, we thank them.
Over the past month or so, we've gotten a lot of reports on Twitter of what Discord thinks of Dolphin.
Unfortunately, Dolphin is not an Atari game. Discord has been made aware of this and will hopefully address it on their own for our 5.0 users, but yourWaifu came onto the scene to address it in our own way! By using Discord Rich Presence, we can directly tell Discord what we are and have it display in Discord users status messages. But we can do even more, and show precisely which game they are playing in Dolphin!
For users that do not want to show what game they are playing, this feature can be disabled directly in the emulator, or you can just disable the playing a game option entirely in Discord itself.
Note: Disabling the "Display currently running game as a status message" option within Discord will not stop it from recording what games you're playing, only displaying it for others to see. The only way to keep the information away from Discord is to disable it in Dolphin. Or turn off Discord.
A few months ago, Dolphin was given the ability to examine a game's mipmaps to see if they were being used simply as downscaled textures or if they had been edited for a graphical effect. After the merge, we discovered that a lot of Nintendo's first party titles used mipmaps for effects, such as The Legend of Zelda: The Wind Waker, The Legend of Zelda: Twilight Princess, Super Mario Sunshine, Super Mario Galaxy and Super Mario Galaxy 2.
In those games, the new feature worked as anticipated. Dolphin detected when games were using mipmaps in order to perform special effects, but would otherwise scale how far out higher resolution textures/mipmaps are used based on internal resolution. This was great when it worked and allowed those games to render more more closely as they did to console on Dolphin even at higher resolutions. Unfortunately, that wasn't the case for every game.
Skies of Arcadia Legends and many of it's Sega brethren completely fool Dolphin's mipmap heuristic. This is because they use DXT compression on extremely small mipmaps of textures, adding enough noise that the heuristic thinks it is a mipmap effect. The result is that Dolphin determines that it has to show the mipmaps at the correct distances to preserve a special effect that isn't there, making things blurrier for no reason. Unfortunately, there isn't much that can be done about this other than giving users the ability to disable the option if a game manages to fool the heuristic in this manner.
Another serious issue was discovered when a bug report struck that Metroid Prime Trilogy was stuttering upon loading rooms. After going through the typical culprits, testers narrowed it down to the fact that enabling GPU Texture Decoding fixed their stutter. Additional information from the bug report (including a bisect) centered around the mipmap heuristic! It turns out that Metroid Prime 3 streams enough large textures in rapid succession that it can bottleneck Dolphin on the mipmap heuristic! After all the work that was done to make shader compilation stuttering disappear, no one was thrilled to see a new stuttering bug. While disabling the heuristic will alleviate the extra stuttering, developers also greatly optimized the heuristic so that it wouldn't cause slowdowns in the games that needed it.
As you can see, even with the new optimizations, Metroid Prime 3's nightmare case is still slightly slower than with the arbitrary mipmap detection disabled. While the feature can be optimized further, please keep this in mind when enabling it for games with heavy texture streaming. Going forward, it is likely that Dolphin will be getting some game INI changes that will disable the heuristic in games where it's known to cause issues. The long-term goal is that it will become a silent feature that users don't have to worry about, but until all the kinks are worked out, the best compromise is to simply give users the power to decide if they want to use it or not.