Retro Game Corps video My Favorite Handhelds of 2024 will give you an overview of what kinds of handhelds are availible. Altough the ones in the formfactor you want probably don’t have very good compatibility with newer indie PC games.
Winlator is an option to get some PC games to run on android handhelds but it requirers some tinkering. I should add that I am not speaking from experience, I have only watched a few videos.

In/Spectre

I have been using the Nova Lite for a month now with no issues. I only use the dongle and it works well on both linux and windows. It is availible on MaxGaming for 23.90€.