If he did remember then he would be upset. Plus, you two don't have to separate just because he doesn't remember. You can help him rebuild his memories and not just from scratch.

...I can't answer you that, because I'm not those killers. But, you can't live in fear. Trust me. That's what they want you to do more than anything.