I don't know if this works anymore, but this used to work:
Turn on auto arrange.
Move stuff until it's how you like it.
Turn off auto arrange.
Changing the resolution keeps the layout the same, except it's not adjusted for resolution, so it'll only take up a corner of the screen