Most of my memories with the PSP are playing non-PSP games, coz there's a shitton of emulators you can install and the PSP controls are super comfy.
The PSP by design can play PSX games, and at native resolution too which fits perfectly on the PSP screen, so you might want to get some and convert them using this guide:
https://web.archive.org/web/20130120063154/https://gbatemp.net/threads/guide-to-converting-ps1-games-for-psp.206195/
Other emulators:
>ScummVM
This runs a plethora of point-and-click games from across many systems including MS-DOS and Windows.
>DOSbox (MS-DOS)
Yes this is a thing, and yes you can install Windows 9x on it but it's super slow (for obvious reasons). You have to configure it from PC first to map keyboard keys to the PSP controls.
>DaedalusX64 (Nintendo 64)
Only runs some games perfectly (mainly the top titles) while the rest can run with glitches or not at all.
>Gpsp (Gameboy Advance)
>MasterBoy (Gameboy/Color)
>NesterJ (NES)
>Snes9X (SNES)
>PSP Genesis (Sega Genesis)
>PSP2600 (Atari 2600)
There's also Bookr for reading PDF/TXT files, and the PSP web browser can play SWF flash games provided you embed them in a webpage (HTML) file first. Although you'll be limited to flash games that rely on the mouse since the browser keyboard can only be used in text fields.