The 26 ms scheduler wait

The problem

totk-pc-recomp turns the Switch build of Tears of the Kingdom into a native Windows program. The game assumes the Switch scheduler: three application cores, and a higher priority thread takes a core from a lower priority one. It keeps per-core data without locks and relies on that.

So the port models three virtual cores. A guest thread runs game code only while it holds one. When a better thread is waiting, exactly one holder is flagged and hands its core over.

Where it waited

The holder only looked at the flag at system calls. Game code can run a long time without making one, so the better thread just sat there. The architecture doc says checking only at SVCs left a better thread waiting up to 26 ms.

The fix

The holder now also checks at every indirect call, through the dispatch function. When nobody is waiting that is two loads. The doc credits this idea to another port, SavageWindRecomp's BotW port.

Where it still falls short

Game code can still run for milliseconds with neither a system call nor an indirect call. For that there is an asynchronous mode: a waiter that has spun 250 microseconds suspends the holder and moves it to a stub that gives the core up. It skips threads that are inside an SDK spin lock.

Source: docs/architecture.md in the repo.