Buy the SV and ride the snot out of it. Take it from someone who owns MANY 30+ year old jap bikes. They may be fun to ride to the supermarket every now and then, but that's about it. Racing them's another matter, 'cause if they blow up it's just a short push back to the pits, instead of spending the night sleeping under it somewhere near Ojai.
Remember, you're dealing with old stuff whirring and clanking around in there, and like the previous post said, oil leaks and other horrible things (like cam chains trying to saw motors in half, etc.) are just waiting to happen.
But if you like stuff like that, go ahead and get the CB. I've got one for sale if you're interested...comes with lots of spare parts.
But whatever you do, don't get a 350 four. Even if someone gives it to you. You have been warned.