I would not characterize the Matadors as having a rough sound, vocally. They were a tight, self-contained combo with smooth leads and harmonies.
However, "Surf City" is not their best Jan & Dean track. Jan's production got so much better after "Surf City." "Honolulu Lulu" was much better, vocally, and the production was superior to "Surf City."
I think "Popsicle" from the Drag City LP epitomizes the Matadors on backing vocals, along with "Surfin' Hearse" and "The New Girl In School."
Their work on the non-single cuts from the Drag City and Dead Man's Curve LPs is strong.