Matrix has gotten a lot of bad publicity lately. Some of it I agree with, some of it I don't. And I've been around the Matrix community for long enough that I truly do believe that the Matrix team has good intentions, so I don't really want to add fuel to the fire.
But unfortunately, I was also targeted by the same spam wave that Terence Eden was targeted by, and it was frustrating enough that I wanted to express my thoughts on the way open protocols like Matrix handle spam. I think Matrix has some architectural properties that make it harder to deal with invite spam effectively.
The spam wave in question was via room invites, with MXIDs and room names crafted with hateful messages. These were effectively direct messages; in Matrix, all direct messages are actually rooms, so if you want to accept direct messages from anyone new, you must allow room invites. And, in Matrix in particular, you must allow room invites from effectively anyone; we'll see why in a moment.
You invite a user to a room via the POST /rooms/{roomId}/invite endpoint. This endpoint takes a user_id as a parameter, which is the MXID of the user to be invited. MXIDs are similar to email addresses, in that they have a local part and a domain part; a user account has only one MXID, and users can see the MXIDs of other users who share rooms with them. They are not typically treated as sensitive, at least no moreso than email addresses are.
The problem is that this invite mechanism makes no reference to how you might know the user being invited. If I'm okay with accepting invites from other users in one room, but not in another, I have no way of expressing that in the protocol. Anyone with my MXID can send me an invite, and I have no way of distinguishing between a close friend inviting me to a chat and a spammer from one of the large public rooms doing the same.
The Matrix community has been hard at work developing trust and safety tools like Draupnir to help moderators more effectively deal with abuse, and has been putting together shared policy lists to allow moderation efforts to be shared across multiple homeservers. (like email blocklists).
These are important tools to have available! Moderators need to have expressive tools available to them to combat abuse when they see it. But for invite spam in particular (which targets individual users rather than e.g. spamming in a room), I'm not convinced these alone will get us to a good point. Shared policy lists sound a lot like email RBLs, and what I can tell you from dealing with those years ago is that:
The trouble with RBLs is that they're largely reactive, and to the extent that they try to be proactive, it's via fairly crude mechanisms. If you're spamming, you might land on an RBL when people start to report your mail as spam. But that relies on users having received the spam, so it's by necessity going to let some things through. They try to be proactive as well by coming up with "IP reputation" calculations, based on certain metrics like "how long have you had this IP," "how much spam have we seen from this IP block," etc. But as anyone who's tried running a mail server on a VPS can tell you, your ability to get your legitimate mail through depends a lot on how lucky you are with the IP you've been assigned.
So, okay. How else can you solve this sort of problem? Well, as you might have guessed from my comments above, one thing you can do is have multiple identifiers for the same user. There's no fundamental reason that other people in a room with just friends and people in a large public room need to see the same identifier for me. In fact, XMPP did this to some extent; in some XMPP MUCs, you only get to see a room-specific JID for the other users in the room, not their global JID. It's not quite as powerful as it could be because you do still have that global JID, and you're back to the same problem if you start getting spammed at your global JID. But this should start to give you an idea of one possible solution to the problem.
This is a pattern that the object-capability security community touches on a lot, because avoiding the need to ask about someone's identity is core to working with capabilities. Rather than asking "whom are you sending a message to?" and separately asking "are you authorized to send that message?", working with capabilities collapses both down to: "do you hold a capability to send a message to that user?". This is the core insight that's commonly referred to by the phrase "don't separate designation from authority". A capability to send a message to me encompasses both the authority to send a message to me, and identifies me as the recipient of the message.
And there's nothing saying I can only have one capability to send messages to me; indeed, I might create one per person or one per room, and each could tag an incoming message with the person or room it was assigned to when used so I could see where the spam came from. Or, I could not generate a capability for some/most rooms, and then other members of that room wouldn't be able to DM me at all! If I only want my other close friends to be able to message me directly, why shouldn't I be able to do that?
This is somewhat akin to generating new email addresses for each new website/person you give an address to. That never caught on with email, I think partly because the tooling didn't/doesn't really support that use case very well. (Although it can be done well, see Apple's "Hide My Email" feature. The key is not to make the user deal with those random email addresses themselves!) But I think group chats are a particularly good use of this kind of scheme: you very frequently discover people through shared rooms or other platform features (e.g. Spaces) rather than by manually typing in user IDs, so your software can take care of a lot of this for you.
Now, Matrix can't quite do this. The MXID as your solitary and non-sensitive identifier is baked pretty deeply into the protocol, and I think changing that would break a lot of assumptions that clients and servers currently make. It might be able to approximate the anti-spam benefits with an optional (or mandatory, depending on user preference) invite code system - think a new parameter to the invite endpoint with a random code, and your client publishes a new code into each room somehow. (I'm unsure how per-room nicks and avatars work, but something like that?) This would be a fairly large change to the protocol, though, and would take a lot of client work to support.
In the medium-term, perhaps a scheme like that might work. In the long-term, though, I'd really like to see chat protocols pop up that are designed for capabilities from the start. We need to be able to express these sorts of fine-grained permissions within the system to proactively prevent abuse, and capabilities are the clearest way I know of thus far to get there.