There is no keystroke specifically for doing that, but it sounds like using the contact sheet filters might be a useful in getting to the right image. At top of the contact sheet you will see a star rating and color label filter. Generally, clicking on the filters hides files that match that filter. However, if you hold the Option/Shift key on your keyboard while clicking a filter, all other filters will be disabled (so Option-Clicking on the Red label will hide everything but images that have a red label.