Fix crash when deleting room summary

Since this is used across different threads, we have to delete it on the
event loop.

Thank you, q234rty, for the help with debugging this.