Keep identities for users cached

There is not really a reason to stop tracking them, just because the
server says so. We might still want to show a users profile, etc.