We are happy to launch The Matrix Conference on Oct 15-18 in Strasbourg, France. Learn more about it, buy a ticket!

This Week in Matrix 2025-07-25

2025-07-25 — This Week in MatrixHarHarLinks

🔗Matrix Live S11E06 - Element X Accessibility, Encrypted State Events, Element Web Rich Text Pasting

🔗Dept of Status of Matrix 🌡️

Matthew reports

richvdh wrote an excellent writeup of the nightmare from the Element backend team have been working through in detecting and fixing slow-burning database corruption on matrix.org's postgres, which caused the room corruption outage over the last few weeks: https://matrix.org/blog/2025/07/postgres-corruption-postmortem/

Robin Riley (m.org) reports

Please join us in welcoming < polycule >, a geeky and efficient Matrix client, as the newest Ecosystem Member of the Foundation! ✨

Does your community use Matrix? Or do you maintain a community project that builds on Matrix? Join the Foundation as a member! This helps us demonstrate the breadth of the ecosystem, and all members are entitled to participate in our Governing Board elections.

Continue reading…

How we discovered, and recovered from, Postgres corruption on the matrix.org homeserver

2025-07-23 — General, matrix.org homeserverRichard van der Hoff

Greetings from Element's backend/SRE team, who run the matrix.org homeserver on behalf of the Matrix.org Foundation.

Recently users of the matrix.org homeserver began seeing problems where rooms would simply stop working. Operations such as sending a new message, or joining the room as a new member, would fail for mysterious reasons. Where an error message was shown at all, it tended to be something cryptic like "No create event in auth events".

After a couple of weeks of hard work by a team of Element staff including backend developers and systems engineers, we were able to repair almost all of the affected rooms. Although we're still investigating exactly what went wrong and checking that everything is now working as it should, we'd like to share some details about what we know and the work we've done to date.

We'll be diving into some quite technical details. Hopefully you'll find it interesting learning a bit about how Synapse works, how Postgres works, and the work we sometimes find ourselves doing to keep the matrix.org homeserver running.

🔗TL;DR

Let's start with a high-level summary.

The matrix.org homeserver is backed by a large PostgreSQL database instance. Parts of an index on one of tables in this database had become corrupted. We are unsure exactly what caused this corruption, but believe it happened at least a year ago, and likely significantly longer.

The nature of this corruption was such that it had little or no effect at first. However, a background maintenance task which removes old, unreferenced data from this table recently started working on the corrupted region. Due to the corrupt index, the maintenance task incorrectly removed active data from the table, in effect corrupting rooms.

Having identified the problem, we rebuilt the corrupted index, and then restored the data that had been incorrectly removed, from database backups.

Continue reading…

This Week in Matrix 2025-07-18

2025-07-18 — This Week in MatrixHarHarLinks

🔗Matrix Live S11E05 - Project Hydra

🔗Dept of Status of Matrix 🌡️

Matthew reports

On Wednesday we pre-disclosed a major security update on the horizon (codename Hydra) to improve state resets and related federation issues in Matrix. This requires a coordinated security release across all Matrix servers, so that servers running in untrusted federations (e.g. the public Matrix network) can speak the upcoming new room version (v12).

We have just pushed back the date for the coordinated security release to Aug 11th 2025, as we've heard a lot of feedback that 6 days wasn't enough for clients/bots/bridge/tooling developers to test the changes introduced by room v12, and that it also doesn't give enough time for community admins to prepare for the necessary room upgrades. Underestimating the time needed here for client/community testing is entirely our fault, due to being overfocused on coordinating the significant serverside work needed. Pushing back the release date by 3 weeks gives everyone more time to test and prepare. We've also opened up registration on the beta.matrix.org homeserver, which is already running v12 rooms by default, to make it easier for client developers to test their clients. We've also made one clarification below for client developers, explaining the new permissions needed to send m.room.tombstone events.

See https://matrix.org/blog/2025/07/security-predisclosure/ and this week's Matrix Live for all the details.

Continue reading…

Pre-disclosure: Upcoming coordinated security fix for all Matrix server implementations

2025-07-16 — General, SecurityMatthew Hodgson

Hi all,

Over the last 6 months a major project has been underway by the Element server team and the Matrix.org Foundation security team to investigate “state resets”: scenarios where Matrix’s state resolution algorithm can give unexpected results. As part of this work we’ve identified two high severity protocol vulnerabilities (CVE-2025-49090; the other not yet allocated a CVE).

Given the security implications of a federation protocol vulnerability, we’ve shared details under embargo over the last 4 weeks with all known active server implementations, and are now aiming for a coordinated security release across all Matrix server implementations on Tuesday Jul 22nd Monday Aug 11th 2025 at 17:00 UTC. If you run a Matrix server in an untrusted federation (e.g. the public federation), please be prepared to upgrade as soon as the patched versions are available.

These vulnerabilities have been addressed via MSCs which have been shared, reviewed and are in the final comment period (disposition merge) with the Spec Core Team and server implementor community, under embargo. This will result in an off-cycle Matrix spec release (1.16) introducing a new room version (12) to address the vulnerabilities in question, requiring a room upgrade of existing rooms. Having given server and room admins time to upgrade, we are then planning to un-embargo the MSCs and complement tests on Friday Jul 25th Thursday Aug 14th 2025 at 17:00 UTC.

UPDATE: Jul 18th 16:45 UTC: We've heard a lot of feedback that 6 days isn't enough for clients/bots/bridge/tooling developers to test the changes introduced by room v12, and that it also doesn't give enough time for community admins to prepare for the necessary room upgrades. Underestimating the time needed here for client/community testing is entirely our fault, due to being overfocused on coordinating the significant serverside work needed. As a result, we've pushed back the coordinated server release date to Aug 11th, to give everyone more time to test and prepare. We've also opened up registration on the beta.matrix.org homeserver, which is already running v12 rooms by default, to make it easier for client developers to test their clients. We've also made one clarification below for client developers, explaining the new permissions needed to send m.room.tombstone events.

CLARIFICATION: Jul 16 17:30 UTC: Room admins should plan to upgrade rooms at their convenience, similar to previous security-related room version upgrades (e.g. v1 to v2). Much like installing an operating system patch, sooner is better, but as these are not Critical Severity vulnerabilities, there is no requirement for room admins to upgrade rooms immediately on Jul 22nd. For instance, the Matrix.org Foundation will likely upgrade its public rooms at some point after Jul 25th (having given server admins a chance to upgrade, and having given any server implementations running late a chance to release). N.B. Only rooms which include users on potentially malicious servers (e.g. publicly joinable rooms on untrusted federations) are vulnerable.

Important information for client developers:

  • Client developers should review MSC4291: “Room IDs as hashes of the create event” to ensure their clients can accept the new proposed format of room IDs, and no longer expects content.predecessor.event_id in m.room.create events.

  • One of the other changes coming in v12 is that room creators will be privileged over other users in the room. Therefore clients which restrict user behaviour based on power level will need to be updated to be aware that room creators effectively have infinite power level. Creators are not listed in the users block of the m.room.power_levels event, and are instead defined as the sender field of the m.room.create event, or entries in a new optional additional_creators array field in the content of the create event. Full details will be released in the MSCs when embargo lifts.

  • Finally, clients which use power_level_content_override when creating rooms MUST NOT assign a power level to the room creator, otherwise the /createRoom request will fail.

  • UPDATE: Jul 18th: We should have mentioned that the default power level in room v12 for sending m.room.tombstone events to upgrade rooms is 150. This stops normal admins from upgrading the room (and so assuming creator privileges) - instead, a creator has to explicitly boost an admin's power level to 150 in order to let them upgrade the room and effectively assume creator rights going forwards.

This has been an exceptionally complicated project to coordinate and its security implications required us to deviate from our usual MSC process and develop the changes under embargo. This and the expedited release of a new stable room version are exceptional choices that are far from ideal, which we’re having to take to keep the ecosystem secure. To be clear, normal MSC development and process will continue in the open, just as it always has. We’d like to sincerely thank the Matrix server implementor community for their impressive support in preparing the coordinated security releases - both in terms of vital MSC review, and then working together to implement the necessary changes. Matrix’s server heterogeneity has never looked healthier. We’d also like to thank Timo Kösters for helping precipitate the project in the first place.

We’ll follow up with more details on Aug 11th (assuming the disclosure timeline doesn’t slip further).

Thanks all for your time, patience and understanding while we ship this protocol upgrade (the first coordinated upgrade since Matrix 1.0 back in 2019!)

Matthew

This Week in Matrix 2025-07-11

2025-07-11 — This Week in MatrixHarHarLinks

🔗Matrix Live S11E04 – Gathering the community


You can find the Matrix Foundation's Events Working Group's room at #events-wg:matrix.org, all about Matrix Community Summit on their website, and Matrix Community Retreats at #workation-nation-general.

🔗Dept of Spec 📜

Andrew Morgan (anoa) {he/him} says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

🔗MSC Status

New MSCs:

MSCs in Final Comment Period:

Accepted MSCs:

  • No MSCs were accepted this week.

Closed MSCs:

tulir announces

🔗MSC highlight: Room IDs as hashes of the create event

You may have noticed some placeholder MSCs being opened in the past half a year. One of them has now received content ahead of the others:

MSC4291 redefines room IDs to be the hash of the m.room.create event. As a part of the change, the server name component is being removed. If any software you maintain assumes that the server name is present in room IDs, you should stop parsing room IDs or at least fail gracefully if the server name isn't there.

We're going to propose FCP on MSC4291 in the next few days and a new stable room version including that and other security-sensitive changes from the placeholder MSCs is expected to follow relatively soon as well.

Continue reading…

This Week in Matrix 2025-07-04

2025-07-04 — This Week in MatrixThib

🔗Dept of Spec 📜

Andrew Morgan (anoa) {he/him} says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

🔗MSC Status

New MSCs:

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Accepted MSCs:

  • No MSCs were accepted this week.

Closed MSCs:

🔗Spec Updates

Not too much to say this week other than some small progress on existing MSCs.

A PR for defining the process for placeholder MSCs is close to merging, for those that are interested in process!

Continue reading…

This Week in Matrix 2025-06-27

2025-06-27 — This Week in MatrixHarHarLinks

🔗Matrix Live S11E03 – Hookshot Modules

🔗Dept of Status of Matrix 🌡️

Robin Riley (m.org) reports

This week we're thrilled to welcome Rocket.Chat as the newest Silver Member of the Foundation! We're grateful to say they are also sponsoring The Matrix Conference this year.

Does your organisation rely on Matrix? Please join Rocket.Chat and many others in providing financial support for the Foundation.

🔗Dept of Spec 📜

TravisR announces

🔗Matrix 1.15 is here!

Matrix 1.15 is the latest release of the spec, bringing next-gen auth via OIDC, rich room topics, and room summaries to the ecosystem. All of these features have existed for quite a while in practice, but took a while to get through the final design and proposal stages. We're glad they're here though, and improving Matrix for everyone along the way.

Check out the blog post for more details.

🔗MSC highlight: Redact on ban

Typically when a user is banned for 'spam', a bot or human operator will go forth and redact pretty much everything that user has ever sent in the room, so why not make that happen automatically? MSC4293: Redact on ban does just that, and was put up for proposed final comment period (FCP) this week. At this stage in the process the Spec Core Team (SCT) will be giving it more thorough review with an aim of including it in a future version of the spec - if you haven't already reviewed it, now would be a great time 🙂

Continue reading…

Matrix v1.15 release

2025-06-26 — Releases, SpecTravis Ralston

Hey all,

Matrix 1.15 is here with improvements to authentication, room summaries, and rich topics! A few months ago we were starting to see the next generation authentication MSCs (led by MSC3861) enter Final Comment Period and work towards acceptance. In that time, they've progressed all the way to being merged with today's release thanks to Kévin Commaille's generous contributions.

This post covers some of the highlights from the 10 MSCs the release brings to the world, with the full changelog at the end as always.

Continue reading…

Demystifying SBGs

2025-06-26 — GeneralMatthew

We’ve noticed a fair bit of confusion (aka misinformation) around Secure Border Gateways (SBGs) recently, which this blog post aims to clarify.

First off, Secure Border Gateways are not defined in the Matrix specification. The term is actually a product name from Element, rather than anything intrinsic to Matrix.

However the concept of a border gateway is well established. In a Matrix world, it means any kind of application-layer firewall which intercepts APIs between Matrix components in order to provide defence-in-depth or apply additional policy rules, to bring an extra - but optional - layer of control within a federation. It is, in short, an optional way to provide more control over federated traffic.

So conceptually it’s the Matrix equivalent to application layer gateways for email. Without them, email works absolutely fine, and always has. However, it’s still a desirable optional extra for some enterprise deployments. For instance, it can help protect both server misconfigurations or buggy servers: literally providing defence-in-depth in traditional ‘castle and keep’ style.

That there is an ecosystem of 3rd party software vendors building additional components such as Secure Border Gateways reflects the strength and maturity of Matrix as an open standard. It’s clear evidence that a genuine open source initiative, based on an open standard, not only ensures digital sovereignty but also drives competitive innovation. Meanwhile it’s excellent to see other chat vendors like Mattermost working on first-party Matrix support again (and so in turn will benefit from capabilities like Secure Border Gateways or Cross Domain Gateways).

🔗Examples of Border Gateways

Let’s take a look at ‘SBGs’ in the wild. Probably the most widespread example right now is in TI-Messenger, Germany’s healthcare messaging system based on Matrix (targeting 150,000 organisations and almost all German citizens, due to go live in mid-July). Here, gematik chose to require SBGs (called “TI-Messenger Proxies” in its parlance) for each deployment in order to integrate Matrix with TI-Messenger’s FHIR-standard and address book system.

As a result, a whole ecosystem of SBG implementations has emerged: starting with the entirely open source TI-Messenger Proxy reference implementation from gematik - but also additional implementations from certified TI-Messenger vendors including Akquinet, Awesome Technologies, CompuGroup Medical, Element, Famedly, Gedisa, samedi and Xtension.

As an example, Element’s TI-Messenger Proxy implementation is built on Element’s generic SBG implementation, which provides a configurable pipeline for functionality like:

  • Proxying (terminating and re-originating) Matrix traffic
  • Apply rules based on HTTP headers
  • Apply rules based on room membership
  • Enforcing classification labels
  • Enforce closed federation based on a domain allow list.
  • Enforce usage of specific clients.
  • Enforce certain parameters when creating a room.

🔗TL;DR

Whether you call them Secure Border Gateways, TI-Messenger Proxies or something else: it is possible to add an application layer firewall that brings an additional layer of control to a Matrix federation. But let’s not forget:

  • Matrix servers already let you lock down your federation - e.g. all of Synapse’s federation configuration.
  • SBGs are not part of the Matrix specification, because Matrix works perfectly well without them
  • SBGs are an optional extra for organisations and federations that might require them, based on their use-case, external integration points (e.g. FHIR) and overall security posture
  • SBGs are not required for a private federation
  • SBGs are not required for public federation either
  • SBGs do not make Matrix closed
  • It’s a hugely positive sign of Matrix’s maturity that there’s an ecosystem of 3rd party software vendors building additional optional components like SBGs

This Week in Matrix 2025-06-20

2025-06-20 — This Week in MatrixThib

🔗Dept of elections 🗳️

The Governing Board is an advisory board to the Matrix.org Foundation and with elected representatives from all across the Matrix ecosystem.

Robin Riley says

Our second annual Governing Board election has come to a close! Huge thanks to all the candidates and voters. We're extremely grateful to everyone who served in the precedent-setting first cohort that we elected last year.

Special thanks to outgoing representatives Cleo Menezes Jr., Kevin Boos, Sumner Evans, and Thor Arne Johansen, congratulations to the incumbents who were re-elected, and welcome to our new incoming representatives, Christian Kußowski and Gnuxie! Find more info on our blog.

Continue reading…