You might be thinking that this is from Mavis Riley/Wilton (did she have some other surnames?) from Coronation Street, but it's one of those cases where her phrase is made famous not because she says it lots, but because anyone doing an impersonation of her (particulary Les Dennis) would use the phrase.

You lose marks if you thought it was from Hit, Miss or Maybe. This video review show hosted by Zoe Ball would rate videos as hit, miss, or "I don't rightly know" - I think this is trying to impersonate Mavis, but I doubt she'd ever use 'rightly' in this context.