The address got out, or you need everything to stop.
In Rhodes: Settings → Mailbox
Pause inbound — the link under the mailbox card. Mail still arrives and is recorded, but nothing is read or filed. While paused, the card shows Paused and the link changes to Resume inbound processing. Use this if something is going wrong and you want it to stop while you look.
Resuming doesn't go back over mail that came in while paused. To file one of those, open Show recent under Recent activity and press This is a delivery on it within 30 days.
Regenerate — next to the address. This issues a new address and kills the old one immediately. Use it if the address may have leaked.
Regenerating breaks existing senders. Anyone you've given the old address to — your CPA, fund admins, your own auto-forward rule — will need the new one, and mail to the old address stops resolving the moment you confirm. Send the new address out before you tell people it changed.