It's never just a street, with him - especially not with someone that he ostensibly trusted in some slight way. While it's nice to see Penelope, he can't - won't - just drop everything and pretend it's the same woman. She isn't; it's not even a matter of time. If she'd recognize him from somewhere else while they were in Taxon, he's positive she'd have said so. (Loudly and repeatedly, probably.)

"I'm trying not to be a jerk about it." To the other her, or whatever he'd like to call it. He's not enough of an asshole to think of it in terms of 'his Penelope' vs 'this Penelope'.

He can't just let go of it, and he can't use it like a tool, either.