When i first bought my PSP in januari I was really dissapointed over the lack of resolution in Movies on Memory Sticks.
So i created a Hack to fool the PSP to start movies in other resolutions other then 320*240 and 160*120. This hack resulted in the tool PSPrez (programmed by runtime).
I even think this hack is a part of ffmpeg used to encode videos to PSP in 3GP Encoder.
This hack is really simple, it only change the header of the mp4 file to state it is in 320*240 resolution, even if it is 480*160 for example.
This could have made it possible to play videos in 480*272 resolution, but there are a problem.
The hack fooles the GUI to think it is in 320*240, but it even fools the player to think it is 320*240, so the player only reserves memory for 76800 pixels.
If you try to play larger movie resolutions this memory space overflows and the PSP crashes.
Anyway, I think the player can handle 480*272, but the GUI stops it.

I have thought a lot about this problem and the possible solutions I can imagine is these...
1. All valid vertical resolutions are within a Byte in size (240 is smaller than 255), if we are lucky the GUI only checks this byte for the resolution, but the player checks the whole word. eg. we could write in the mp4 header the resolution is in 320*496 and the Gui only sees 320*240 but the player reserves memory for 320*496 pixels. I know its a long shot but there are a small chance. I will try this solution tonight.

2. Hack the firmware to make the GUI to accept 480*272 resolutions. Keywords to search for in the firmware should be mp4, mv4, 320, 160, 240, 120, 768 (maximum bitrate).
I know there are tools to "patch" the firmware, but I dont know if this part of the firmware are Patchable. This is far to complicated for me, but someone mayby can. If the firmware is not patchable mayby the RAM can be patched when the firmware is loaded.

3. Create a tool that runs in the background, the tool could work like this.
a. The mp4 files are patched to state 320*240.
b. The tool is started and runs in the background, and can be triggered by a keystroke to "unpatch" the mp4's.
c. The GUI lists the playable mp4's.
d. The user Triggers the Tool, and the files are changed to state their real resolution, the GUI has already checked the files and thinks it is ok.
e. The user starts the movie and the player sees its real resolution in the header and starts the movie.
This too is to complicated for me, and mayby PSP can't run background prosesses.