diff --git a/CHANGES.md b/CHANGES.md
index 52b2fd6f8f..4237550818 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -6322,8 +6322,8 @@ Changes in synapse 0.5.1 (2014-11-26)
See UPGRADES.rst for specific instructions on how to upgrade.
-> - Fix bug where we served up an Event that did not match its signatures.
-> - Fix regression where we no longer correctly handled the case where a homeserver receives an event for a room it doesn\'t recognise (but is in.)
+- Fix bug where we served up an Event that did not match its signatures.
+- Fix regression where we no longer correctly handled the case where a homeserver receives an event for a room it doesn\'t recognise (but is in.)
Changes in synapse 0.5.0 (2014-11-19)
=====================================
@@ -6334,44 +6334,44 @@ This release also changes the internal database schemas and so requires servers
Homeserver:
-: - Add authentication and authorization to the federation protocol. Events are now signed by their originating homeservers.
- - Implement the new authorization model for rooms.
- - Split out web client into a seperate repository: matrix-angular-sdk.
- - Change the structure of PDUs.
- - Fix bug where user could not join rooms via an alias containing 4-byte UTF-8 characters.
- - Merge concept of PDUs and Events internally.
- - Improve logging by adding request ids to log lines.
- - Implement a very basic room initial sync API.
- - Implement the new invite/join federation APIs.
+- Add authentication and authorization to the federation protocol. Events are now signed by their originating homeservers.
+- Implement the new authorization model for rooms.
+- Split out web client into a seperate repository: matrix-angular-sdk.
+- Change the structure of PDUs.
+- Fix bug where user could not join rooms via an alias containing 4-byte UTF-8 characters.
+- Merge concept of PDUs and Events internally.
+- Improve logging by adding request ids to log lines.
+- Implement a very basic room initial sync API.
+- Implement the new invite/join federation APIs.
Webclient:
-: - The webclient has been moved to a seperate repository.
+- The webclient has been moved to a seperate repository.
Changes in synapse 0.4.2 (2014-10-31)
=====================================
Homeserver:
-: - Fix bugs where we did not notify users of correct presence updates.
- - Fix bug where we did not handle sub second event stream timeouts.
+- Fix bugs where we did not notify users of correct presence updates.
+- Fix bug where we did not handle sub second event stream timeouts.
Webclient:
-: - Add ability to click on messages to see JSON.
- - Add ability to redact messages.
- - Add ability to view and edit all room state JSON.
- - Handle incoming redactions.
- - Improve feedback on errors.
- - Fix bugs in mobile CSS.
- - Fix bugs with desktop notifications.
+- Add ability to click on messages to see JSON.
+- Add ability to redact messages.
+- Add ability to view and edit all room state JSON.
+- Handle incoming redactions.
+- Improve feedback on errors.
+- Fix bugs in mobile CSS.
+- Fix bugs with desktop notifications.
Changes in synapse 0.4.1 (2014-10-17)
=====================================
Webclient:
-: - Fix bug with display of timestamps.
+- Fix bug with display of timestamps.
Changes in synpase 0.4.0 (2014-10-17)
=====================================
@@ -6384,8 +6384,8 @@ You will also need an updated syutil and config. See UPGRADES.rst.
Homeserver:
-: - Sign federation transactions to assert strong identity over federation.
- - Rename timestamp keys in PDUs and events from \'ts\' and \'hsob\_ts\' to \'origin\_server\_ts\'.
+- Sign federation transactions to assert strong identity over federation.
+- Rename timestamp keys in PDUs and events from \'ts\' and \'hsob\_ts\' to \'origin\_server\_ts\'.
Changes in synapse 0.3.4 (2014-09-25)
=====================================
@@ -6394,48 +6394,48 @@ This version adds support for using a TURN server. See docs/turn-howto.rst on ho
Homeserver:
-: - Add support for redaction of messages.
- - Fix bug where inviting a user on a remote home server could take up to 20-30s.
- - Implement a get current room state API.
- - Add support specifying and retrieving turn server configuration.
+- Add support for redaction of messages.
+- Fix bug where inviting a user on a remote home server could take up to 20-30s.
+- Implement a get current room state API.
+- Add support specifying and retrieving turn server configuration.
Webclient:
-: - Add button to send messages to users from the home page.
- - Add support for using TURN for VoIP calls.
- - Show display name change messages.
- - Fix bug where the client didn\'t get the state of a newly joined room until after it has been refreshed.
- - Fix bugs with tab complete.
- - Fix bug where holding down the down arrow caused chrome to chew 100% CPU.
- - Fix bug where desktop notifications occasionally used \"Undefined\" as the display name.
- - Fix more places where we sometimes saw room IDs incorrectly.
- - Fix bug which caused lag when entering text in the text box.
+- Add button to send messages to users from the home page.
+- Add support for using TURN for VoIP calls.
+- Show display name change messages.
+- Fix bug where the client didn\'t get the state of a newly joined room until after it has been refreshed.
+- Fix bugs with tab complete.
+- Fix bug where holding down the down arrow caused chrome to chew 100% CPU.
+- Fix bug where desktop notifications occasionally used \"Undefined\" as the display name.
+- Fix more places where we sometimes saw room IDs incorrectly.
+- Fix bug which caused lag when entering text in the text box.
Changes in synapse 0.3.3 (2014-09-22)
=====================================
Homeserver:
-: - Fix bug where you continued to get events for rooms you had left.
+- Fix bug where you continued to get events for rooms you had left.
Webclient:
-: - Add support for video calls with basic UI.
- - Fix bug where one to one chats were named after your display name rather than the other person\'s.
- - Fix bug which caused lag when typing in the textarea.
- - Refuse to run on browsers we know won\'t work.
- - Trigger pagination when joining new rooms.
- - Fix bug where we sometimes didn\'t display invitations in recents.
- - Automatically join room when accepting a VoIP call.
- - Disable outgoing and reject incoming calls on browsers we don\'t support VoIP in.
- - Don\'t display desktop notifications for messages in the room you are non-idle and speaking in.
+- Add support for video calls with basic UI.
+- Fix bug where one to one chats were named after your display name rather than the other person\'s.
+- Fix bug which caused lag when typing in the textarea.
+- Refuse to run on browsers we know won\'t work.
+- Trigger pagination when joining new rooms.
+- Fix bug where we sometimes didn\'t display invitations in recents.
+- Automatically join room when accepting a VoIP call.
+- Disable outgoing and reject incoming calls on browsers we don\'t support VoIP in.
+- Don\'t display desktop notifications for messages in the room you are non-idle and speaking in.
Changes in synapse 0.3.2 (2014-09-18)
=====================================
Webclient:
-: - Fix bug where an empty \"bing words\" list in old accounts didn\'t send notifications when it should have done.
+- Fix bug where an empty \"bing words\" list in old accounts didn\'t send notifications when it should have done.
Changes in synapse 0.3.1 (2014-09-18)
=====================================
@@ -6444,8 +6444,8 @@ This is a release to hotfix v0.3.0 to fix two regressions.
Webclient:
-: - Fix a regression where we sometimes displayed duplicate events.
- - Fix a regression where we didn\'t immediately remove rooms you were banned in from the recents list.
+- Fix a regression where we sometimes displayed duplicate events.
+- Fix a regression where we didn\'t immediately remove rooms you were banned in from the recents list.
Changes in synapse 0.3.0 (2014-09-18)
=====================================
@@ -6454,91 +6454,91 @@ See UPGRADE for information about changes to the client server API, including br
Homeserver:
-: - When a user changes their displayname or avatar the server will now update all their join states to reflect this.
- - The server now adds \"age\" key to events to indicate how old they are. This is clock independent, so at no point does any server or webclient have to assume their clock is in sync with everyone else.
- - Fix bug where we didn\'t correctly pull in missing PDUs.
- - Fix bug where prev\_content key wasn\'t always returned.
- - Add support for password resets.
+- When a user changes their displayname or avatar the server will now update all their join states to reflect this.
+- The server now adds \"age\" key to events to indicate how old they are. This is clock independent, so at no point does any server or webclient have to assume their clock is in sync with everyone else.
+- Fix bug where we didn\'t correctly pull in missing PDUs.
+- Fix bug where prev\_content key wasn\'t always returned.
+- Add support for password resets.
Webclient:
-: - Improve page content loading.
- - Join/parts now trigger desktop notifications.
- - Always show room aliases in the UI if one is present.
- - No longer show user-count in the recents side panel.
- - Add up & down arrow support to the text box for message sending to step through your sent history.
- - Don\'t display notifications for our own messages.
- - Emotes are now formatted correctly in desktop notifications.
- - The recents list now differentiates between public & private rooms.
- - Fix bug where when switching between rooms the pagination flickered before the view jumped to the bottom of the screen.
- - Add bing word support.
+- Improve page content loading.
+- Join/parts now trigger desktop notifications.
+- Always show room aliases in the UI if one is present.
+- No longer show user-count in the recents side panel.
+- Add up & down arrow support to the text box for message sending to step through your sent history.
+- Don\'t display notifications for our own messages.
+- Emotes are now formatted correctly in desktop notifications.
+- The recents list now differentiates between public & private rooms.
+- Fix bug where when switching between rooms the pagination flickered before the view jumped to the bottom of the screen.
+- Add bing word support.
Registration API:
-: - The registration API has been overhauled to function like the login API. In practice, this means registration requests must now include the following: \'type\':\'m.login.password\'. See UPGRADE for more information on this.
- - The \'user\_id\' key has been renamed to \'user\' to better match the login API.
- - There is an additional login type: \'m.login.email.identity\'.
- - The command client and web client have been updated to reflect these changes.
+- The registration API has been overhauled to function like the login API. In practice, this means registration requests must now include the following: \'type\':\'m.login.password\'. See UPGRADE for more information on this.
+- The \'user\_id\' key has been renamed to \'user\' to better match the login API.
+- There is an additional login type: \'m.login.email.identity\'.
+- The command client and web client have been updated to reflect these changes.
Changes in synapse 0.2.3 (2014-09-12)
=====================================
Homeserver:
-: - Fix bug where we stopped sending events to remote home servers if a user from that home server left, even if there were some still in the room.
- - Fix bugs in the state conflict resolution where it was incorrectly rejecting events.
+- Fix bug where we stopped sending events to remote home servers if a user from that home server left, even if there were some still in the room.
+- Fix bugs in the state conflict resolution where it was incorrectly rejecting events.
Webclient:
-: - Display room names and topics.
- - Allow setting/editing of room names and topics.
- - Display information about rooms on the main page.
- - Handle ban and kick events in real time.
- - VoIP UI and reliability improvements.
- - Add glare support for VoIP.
- - Improvements to initial startup speed.
- - Don\'t display duplicate join events.
- - Local echo of messages.
- - Differentiate sending and sent of local echo.
- - Various minor bug fixes.
+- Display room names and topics.
+- Allow setting/editing of room names and topics.
+- Display information about rooms on the main page.
+- Handle ban and kick events in real time.
+- VoIP UI and reliability improvements.
+- Add glare support for VoIP.
+- Improvements to initial startup speed.
+- Don\'t display duplicate join events.
+- Local echo of messages.
+- Differentiate sending and sent of local echo.
+- Various minor bug fixes.
Changes in synapse 0.2.2 (2014-09-06)
=====================================
Homeserver:
-: - When the server returns state events it now also includes the previous content.
- - Add support for inviting people when creating a new room.
- - Make the homeserver inform the room via m.room.aliases when a new alias is added for a room.
- - Validate m.room.power\_level events.
+- When the server returns state events it now also includes the previous content.
+- Add support for inviting people when creating a new room.
+- Make the homeserver inform the room via m.room.aliases when a new alias is added for a room.
+- Validate m.room.power\_level events.
Webclient:
-: - Add support for captchas on registration.
- - Handle m.room.aliases events.
- - Asynchronously send messages and show a local echo.
- - Inform the UI when a message failed to send.
- - Only autoscroll on receiving a new message if the user was already at the bottom of the screen.
- - Add support for ban/kick reasons.
+- Add support for captchas on registration.
+- Handle m.room.aliases events.
+- Asynchronously send messages and show a local echo.
+- Inform the UI when a message failed to send.
+- Only autoscroll on receiving a new message if the user was already at the bottom of the screen.
+- Add support for ban/kick reasons.
Changes in synapse 0.2.1 (2014-09-03)
=====================================
Homeserver:
-: - Added support for signing up with a third party id.
- - Add synctl scripts.
- - Added rate limiting.
- - Add option to change the external address the content repo uses.
- - Presence bug fixes.
+- Added support for signing up with a third party id.
+- Add synctl scripts.
+- Added rate limiting.
+- Add option to change the external address the content repo uses.
+- Presence bug fixes.
Webclient:
-: - Added support for signing up with a third party id.
- - Added support for banning and kicking users.
- - Added support for displaying and setting ops.
- - Added support for room names.
- - Fix bugs with room membership event display.
+- Added support for signing up with a third party id.
+- Added support for banning and kicking users.
+- Added support for displaying and setting ops.
+- Added support for room names.
+- Fix bugs with room membership event display.
Changes in synapse 0.2.0 (2014-09-02)
=====================================
@@ -6547,36 +6547,36 @@ This update changes many configuration options, updates the database schema and
Homeserver:
-: - Require SSL for server-server connections.
- - Add SSL listener for client-server connections.
- - Add ability to use config files.
- - Add support for kicking/banning and power levels.
- - Allow setting of room names and topics on creation.
- - Change presence to include last seen time of the user.
- - Change url path prefix to /\_matrix/\...
- - Bug fixes to presence.
+- Require SSL for server-server connections.
+- Add SSL listener for client-server connections.
+- Add ability to use config files.
+- Add support for kicking/banning and power levels.
+- Allow setting of room names and topics on creation.
+- Change presence to include last seen time of the user.
+- Change url path prefix to /\_matrix/\...
+- Bug fixes to presence.
Webclient:
-: - Reskin the CSS for registration and login.
- - Various improvements to rooms CSS.
- - Support changes in client-server API.
- - Bug fixes to VOIP UI.
- - Various bug fixes to handling of changes to room member list.
+- Reskin the CSS for registration and login.
+- Various improvements to rooms CSS.
+- Support changes in client-server API.
+- Bug fixes to VOIP UI.
+- Various bug fixes to handling of changes to room member list.
Changes in synapse 0.1.2 (2014-08-29)
=====================================
Webclient:
-: - Add basic call state UI for VoIP calls.
+- Add basic call state UI for VoIP calls.
Changes in synapse 0.1.1 (2014-08-29)
=====================================
Homeserver:
-: - Fix bug that caused the event stream to not notify some clients about changes.
+- Fix bug that caused the event stream to not notify some clients about changes.
Changes in synapse 0.1.0 (2014-08-29)
=====================================
@@ -6585,26 +6585,22 @@ Presence has been reenabled in this release.
Homeserver:
-: -
-
- Update client to server API, including:
-
- : - Use a more consistent url scheme.
- - Provide more useful information in the initial sync api.
-
- - Change the presence handling to be much more efficient.
- - Change the presence server to server API to not require explicit polling of all users who share a room with a user.
- - Fix races in the event streaming logic.
+- Update client to server API, including:
+ - Use a more consistent url scheme.
+ - Provide more useful information in the initial sync api.
+- Change the presence handling to be much more efficient.
+- Change the presence server to server API to not require explicit polling of all users who share a room with a user.
+- Fix races in the event streaming logic.
Webclient:
-: - Update to use new client to server API.
- - Add basic VOIP support.
- - Add idle timers that change your status to away.
- - Add recent rooms column when viewing a room.
- - Various network efficiency improvements.
- - Add basic mobile browser support.
- - Add a settings page.
+- Update to use new client to server API.
+- Add basic VOIP support.
+- Add idle timers that change your status to away.
+- Add recent rooms column when viewing a room.
+- Various network efficiency improvements.
+- Add basic mobile browser support.
+- Add a settings page.
Changes in synapse 0.0.1 (2014-08-22)
=====================================
@@ -6613,26 +6609,26 @@ Presence has been disabled in this release due to a bug that caused the homeserv
Homeserver:
-: - Completely change the database schema to support generic event types.
- - Improve presence reliability.
- - Improve reliability of joining remote rooms.
- - Fix bug where room join events were duplicated.
- - Improve initial sync API to return more information to the client.
- - Stop generating fake messages for room membership events.
+- Completely change the database schema to support generic event types.
+- Improve presence reliability.
+- Improve reliability of joining remote rooms.
+- Fix bug where room join events were duplicated.
+- Improve initial sync API to return more information to the client.
+- Stop generating fake messages for room membership events.
Webclient:
-: - Add tab completion of names.
- - Add ability to upload and send images.
- - Add profile pages.
- - Improve CSS layout of room.
- - Disambiguate identical display names.
- - Don\'t get remote users display names and avatars individually.
- - Use the new initial sync API to reduce number of round trips to the homeserver.
- - Change url scheme to use room aliases instead of room ids where known.
- - Increase longpoll timeout.
+- Add tab completion of names.
+- Add ability to upload and send images.
+- Add profile pages.
+- Improve CSS layout of room.
+- Disambiguate identical display names.
+- Don\'t get remote users display names and avatars individually.
+- Use the new initial sync API to reduce number of round trips to the homeserver.
+- Change url scheme to use room aliases instead of room ids where known.
+- Increase longpoll timeout.
Changes in synapse 0.0.0 (2014-08-13)
=====================================
-> - Initial alpha release
+- Initial alpha release
diff --git a/INSTALL.md b/INSTALL.md
index c6fcb3bd7f..eaeb690092 100644
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -487,7 +487,7 @@ In nginx this would be something like:
```
location /.well-known/matrix/client {
return 200 '{"m.homeserver": {"base_url": "https://<matrix.example.com>"}}';
- add_header Content-Type application/json;
+ default_type application/json;
add_header Access-Control-Allow-Origin *;
}
```
diff --git a/UPGRADE.rst b/UPGRADE.rst
index 7c19cf2a70..4de1bb5841 100644
--- a/UPGRADE.rst
+++ b/UPGRADE.rst
@@ -75,6 +75,36 @@ for example:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
+Upgrading to v1.24.0
+====================
+
+Custom OpenID Connect mapping provider breaking change
+------------------------------------------------------
+
+This release allows the OpenID Connect mapping provider to perform normalisation
+of the localpart of the Matrix ID. This allows for the mapping provider to
+specify different algorithms, instead of the [default way](https://matrix.org/docs/spec/appendices#mapping-from-other-character-sets).
+
+If your Synapse configuration uses a custom mapping provider
+(`oidc_config.user_mapping_provider.module` is specified and not equal to
+`synapse.handlers.oidc_handler.JinjaOidcMappingProvider`) then you *must* ensure
+that `map_user_attributes` of the mapping provider performs some normalisation
+of the `localpart` returned. To match previous behaviour you can use the
+`map_username_to_mxid_localpart` function provided by Synapse. An example is
+shown below:
+
+.. code-block:: python
+
+ from synapse.types import map_username_to_mxid_localpart
+
+ class MyMappingProvider:
+ def map_user_attributes(self, userinfo, token):
+ # ... your custom logic ...
+ sso_user_id = ...
+ localpart = map_username_to_mxid_localpart(sso_user_id)
+
+ return {"localpart": localpart}
+
Upgrading to v1.23.0
====================
diff --git a/changelog.d/8630.feature b/changelog.d/8630.feature
new file mode 100644
index 0000000000..706051f131
--- /dev/null
+++ b/changelog.d/8630.feature
@@ -0,0 +1 @@
+Allow specification of the SAML IdP if the metadata returns multiple IdPs.
diff --git a/changelog.d/8744.bugfix b/changelog.d/8744.bugfix
new file mode 100644
index 0000000000..f8f9630bd6
--- /dev/null
+++ b/changelog.d/8744.bugfix
@@ -0,0 +1 @@
+Fix a bug where appservices may be sent an excessive amount of read receipts and presence. Broke in v1.22.0.
diff --git a/changelog.d/8774.misc b/changelog.d/8774.misc
new file mode 100644
index 0000000000..57cca8fee5
--- /dev/null
+++ b/changelog.d/8774.misc
@@ -0,0 +1 @@
+Add additional error checking for OpenID Connect and SAML mapping providers.
diff --git a/changelog.d/8776.bugfix b/changelog.d/8776.bugfix
new file mode 100644
index 0000000000..dd7ebbeb86
--- /dev/null
+++ b/changelog.d/8776.bugfix
@@ -0,0 +1 @@
+Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body.
diff --git a/changelog.d/8793.doc b/changelog.d/8793.doc
new file mode 100644
index 0000000000..f6eee1ea73
--- /dev/null
+++ b/changelog.d/8793.doc
@@ -0,0 +1 @@
+Fix the example on how to set the `Content-Type` header in nginx for the Client Well-Known URI.
diff --git a/docs/sample_config.yaml b/docs/sample_config.yaml
index 853e551980..ea32a3d266 100644
--- a/docs/sample_config.yaml
+++ b/docs/sample_config.yaml
@@ -1852,6 +1852,14 @@ saml2_config:
# - attribute: department
# value: "sales"
+ # If the metadata XML contains multiple IdP entities then the `idp_entityid`
+ # option must be set to the entity to redirect users to.
+ #
+ # Most deployments only have a single IdP entity and so should omit this
+ # option.
+ #
+ #idp_entityid: 'https://our_idp/entityid'
+
# Enable OpenID Connect (OIDC) / OAuth 2.0 for registration and login.
#
diff --git a/docs/sso_mapping_providers.md b/docs/sso_mapping_providers.md
index 32b06aa2c5..707dd73978 100644
--- a/docs/sso_mapping_providers.md
+++ b/docs/sso_mapping_providers.md
@@ -15,8 +15,15 @@ where SAML mapping providers come into play.
SSO mapping providers are currently supported for OpenID and SAML SSO
configurations. Please see the details below for how to implement your own.
+It is the responsibility of the mapping provider to normalise the SSO attributes
+and map them to a valid Matrix ID. The
+[specification for Matrix IDs](https://matrix.org/docs/spec/appendices#user-identifiers)
+has some information about what is considered valid. Alternately an easy way to
+ensure it is valid is to use a Synapse utility function:
+`synapse.types.map_username_to_mxid_localpart`.
+
External mapping providers are provided to Synapse in the form of an external
-Python module. You can retrieve this module from [PyPi](https://pypi.org) or elsewhere,
+Python module. You can retrieve this module from [PyPI](https://pypi.org) or elsewhere,
but it must be importable via Synapse (e.g. it must be in the same virtualenv
as Synapse). The Synapse config is then modified to point to the mapping provider
(and optionally provide additional configuration for it).
diff --git a/synapse/config/saml2_config.py b/synapse/config/saml2_config.py
index f233854941..c1b8e98ae0 100644
--- a/synapse/config/saml2_config.py
+++ b/synapse/config/saml2_config.py
@@ -90,6 +90,8 @@ class SAML2Config(Config):
"grandfathered_mxid_source_attribute", "uid"
)
+ self.saml2_idp_entityid = saml2_config.get("idp_entityid", None)
+
# user_mapping_provider may be None if the key is present but has no value
ump_dict = saml2_config.get("user_mapping_provider") or {}
@@ -383,6 +385,14 @@ class SAML2Config(Config):
# value: "staff"
# - attribute: department
# value: "sales"
+
+ # If the metadata XML contains multiple IdP entities then the `idp_entityid`
+ # option must be set to the entity to redirect users to.
+ #
+ # Most deployments only have a single IdP entity and so should omit this
+ # option.
+ #
+ #idp_entityid: 'https://our_idp/entityid'
""" % {
"config_dir_path": config_dir_path
}
diff --git a/synapse/federation/federation_server.py b/synapse/federation/federation_server.py
index 23278e36b7..4b6ab470d0 100644
--- a/synapse/federation/federation_server.py
+++ b/synapse/federation/federation_server.py
@@ -49,6 +49,7 @@ from synapse.federation.federation_base import FederationBase, event_from_pdu_js
from synapse.federation.persistence import TransactionActions
from synapse.federation.units import Edu, Transaction
from synapse.http.endpoint import parse_server_name
+from synapse.http.servlet import assert_params_in_dict
from synapse.logging.context import (
make_deferred_yieldable,
nested_logging_context,
@@ -391,7 +392,7 @@ class FederationServer(FederationBase):
TRANSACTION_CONCURRENCY_LIMIT,
)
- async def on_context_state_request(
+ async def on_room_state_request(
self, origin: str, room_id: str, event_id: str
) -> Tuple[int, Dict[str, Any]]:
origin_host, _ = parse_server_name(origin)
@@ -514,11 +515,12 @@ class FederationServer(FederationBase):
return {"event": ret_pdu.get_pdu_json(time_now)}
async def on_send_join_request(
- self, origin: str, content: JsonDict, room_id: str
+ self, origin: str, content: JsonDict
) -> Dict[str, Any]:
logger.debug("on_send_join_request: content: %s", content)
- room_version = await self.store.get_room_version(room_id)
+ assert_params_in_dict(content, ["room_id"])
+ room_version = await self.store.get_room_version(content["room_id"])
pdu = event_from_pdu_json(content, room_version)
origin_host, _ = parse_server_name(origin)
@@ -547,12 +549,11 @@ class FederationServer(FederationBase):
time_now = self._clock.time_msec()
return {"event": pdu.get_pdu_json(time_now), "room_version": room_version}
- async def on_send_leave_request(
- self, origin: str, content: JsonDict, room_id: str
- ) -> dict:
+ async def on_send_leave_request(self, origin: str, content: JsonDict) -> dict:
logger.debug("on_send_leave_request: content: %s", content)
- room_version = await self.store.get_room_version(room_id)
+ assert_params_in_dict(content, ["room_id"])
+ room_version = await self.store.get_room_version(content["room_id"])
pdu = event_from_pdu_json(content, room_version)
origin_host, _ = parse_server_name(origin)
@@ -748,12 +749,8 @@ class FederationServer(FederationBase):
)
return ret
- async def on_exchange_third_party_invite_request(
- self, room_id: str, event_dict: Dict
- ):
- ret = await self.handler.on_exchange_third_party_invite_request(
- room_id, event_dict
- )
+ async def on_exchange_third_party_invite_request(self, event_dict: Dict):
+ ret = await self.handler.on_exchange_third_party_invite_request(event_dict)
return ret
async def check_server_matches_acl(self, server_name: str, room_id: str):
diff --git a/synapse/federation/transport/server.py b/synapse/federation/transport/server.py
index 1f74c504a3..a9b73d713a 100644
--- a/synapse/federation/transport/server.py
+++ b/synapse/federation/transport/server.py
@@ -441,13 +441,13 @@ class FederationEventServlet(BaseFederationServlet):
class FederationStateV1Servlet(BaseFederationServlet):
- PATH = "/state/(?P<context>[^/]*)/?"
+ PATH = "/state/(?P<room_id>[^/]*)/?"
- # This is when someone asks for all data for a given context.
- async def on_GET(self, origin, content, query, context):
- return await self.handler.on_context_state_request(
+ # This is when someone asks for all data for a given room.
+ async def on_GET(self, origin, content, query, room_id):
+ return await self.handler.on_room_state_request(
origin,
- context,
+ room_id,
parse_string_from_args(query, "event_id", None, required=False),
)
@@ -464,16 +464,16 @@ class FederationStateIdsServlet(BaseFederationServlet):
class FederationBackfillServlet(BaseFederationServlet):
- PATH = "/backfill/(?P<context>[^/]*)/?"
+ PATH = "/backfill/(?P<room_id>[^/]*)/?"
- async def on_GET(self, origin, content, query, context):
+ async def on_GET(self, origin, content, query, room_id):
versions = [x.decode("ascii") for x in query[b"v"]]
limit = parse_integer_from_args(query, "limit", None)
if not limit:
return 400, {"error": "Did not include limit param"}
- return await self.handler.on_backfill_request(origin, context, versions, limit)
+ return await self.handler.on_backfill_request(origin, room_id, versions, limit)
class FederationQueryServlet(BaseFederationServlet):
@@ -488,9 +488,9 @@ class FederationQueryServlet(BaseFederationServlet):
class FederationMakeJoinServlet(BaseFederationServlet):
- PATH = "/make_join/(?P<context>[^/]*)/(?P<user_id>[^/]*)"
+ PATH = "/make_join/(?P<room_id>[^/]*)/(?P<user_id>[^/]*)"
- async def on_GET(self, origin, _content, query, context, user_id):
+ async def on_GET(self, origin, _content, query, room_id, user_id):
"""
Args:
origin (unicode): The authenticated server_name of the calling server
@@ -512,16 +512,16 @@ class FederationMakeJoinServlet(BaseFederationServlet):
supported_versions = ["1"]
content = await self.handler.on_make_join_request(
- origin, context, user_id, supported_versions=supported_versions
+ origin, room_id, user_id, supported_versions=supported_versions
)
return 200, content
class FederationMakeLeaveServlet(BaseFederationServlet):
- PATH = "/make_leave/(?P<context>[^/]*)/(?P<user_id>[^/]*)"
+ PATH = "/make_leave/(?P<room_id>[^/]*)/(?P<user_id>[^/]*)"
- async def on_GET(self, origin, content, query, context, user_id):
- content = await self.handler.on_make_leave_request(origin, context, user_id)
+ async def on_GET(self, origin, content, query, room_id, user_id):
+ content = await self.handler.on_make_leave_request(origin, room_id, user_id)
return 200, content
@@ -529,7 +529,7 @@ class FederationV1SendLeaveServlet(BaseFederationServlet):
PATH = "/send_leave/(?P<room_id>[^/]*)/(?P<event_id>[^/]*)"
async def on_PUT(self, origin, content, query, room_id, event_id):
- content = await self.handler.on_send_leave_request(origin, content, room_id)
+ content = await self.handler.on_send_leave_request(origin, content)
return 200, (200, content)
@@ -539,43 +539,43 @@ class FederationV2SendLeaveServlet(BaseFederationServlet):
PREFIX = FEDERATION_V2_PREFIX
async def on_PUT(self, origin, content, query, room_id, event_id):
- content = await self.handler.on_send_leave_request(origin, content, room_id)
+ content = await self.handler.on_send_leave_request(origin, content)
return 200, content
class FederationEventAuthServlet(BaseFederationServlet):
- PATH = "/event_auth/(?P<context>[^/]*)/(?P<event_id>[^/]*)"
+ PATH = "/event_auth/(?P<room_id>[^/]*)/(?P<event_id>[^/]*)"
- async def on_GET(self, origin, content, query, context, event_id):
- return await self.handler.on_event_auth(origin, context, event_id)
+ async def on_GET(self, origin, content, query, room_id, event_id):
+ return await self.handler.on_event_auth(origin, room_id, event_id)
class FederationV1SendJoinServlet(BaseFederationServlet):
- PATH = "/send_join/(?P<context>[^/]*)/(?P<event_id>[^/]*)"
+ PATH = "/send_join/(?P<room_id>[^/]*)/(?P<event_id>[^/]*)"
- async def on_PUT(self, origin, content, query, context, event_id):
- # TODO(paul): assert that context/event_id parsed from path actually
+ async def on_PUT(self, origin, content, query, room_id, event_id):
+ # TODO(paul): assert that room_id/event_id parsed from path actually
# match those given in content
- content = await self.handler.on_send_join_request(origin, content, context)
+ content = await self.handler.on_send_join_request(origin, content)
return 200, (200, content)
class FederationV2SendJoinServlet(BaseFederationServlet):
- PATH = "/send_join/(?P<context>[^/]*)/(?P<event_id>[^/]*)"
+ PATH = "/send_join/(?P<room_id>[^/]*)/(?P<event_id>[^/]*)"
PREFIX = FEDERATION_V2_PREFIX
- async def on_PUT(self, origin, content, query, context, event_id):
- # TODO(paul): assert that context/event_id parsed from path actually
+ async def on_PUT(self, origin, content, query, room_id, event_id):
+ # TODO(paul): assert that room_id/event_id parsed from path actually
# match those given in content
- content = await self.handler.on_send_join_request(origin, content, context)
+ content = await self.handler.on_send_join_request(origin, content)
return 200, content
class FederationV1InviteServlet(BaseFederationServlet):
- PATH = "/invite/(?P<context>[^/]*)/(?P<event_id>[^/]*)"
+ PATH = "/invite/(?P<room_id>[^/]*)/(?P<event_id>[^/]*)"
- async def on_PUT(self, origin, content, query, context, event_id):
+ async def on_PUT(self, origin, content, query, room_id, event_id):
# We don't get a room version, so we have to assume its EITHER v1 or
# v2. This is "fine" as the only difference between V1 and V2 is the
# state resolution algorithm, and we don't use that for processing
@@ -590,12 +590,12 @@ class FederationV1InviteServlet(BaseFederationServlet):
class FederationV2InviteServlet(BaseFederationServlet):
- PATH = "/invite/(?P<context>[^/]*)/(?P<event_id>[^/]*)"
+ PATH = "/invite/(?P<room_id>[^/]*)/(?P<event_id>[^/]*)"
PREFIX = FEDERATION_V2_PREFIX
- async def on_PUT(self, origin, content, query, context, event_id):
- # TODO(paul): assert that context/event_id parsed from path actually
+ async def on_PUT(self, origin, content, query, room_id, event_id):
+ # TODO(paul): assert that room_id/event_id parsed from path actually
# match those given in content
room_version = content["room_version"]
@@ -617,9 +617,7 @@ class FederationThirdPartyInviteExchangeServlet(BaseFederationServlet):
PATH = "/exchange_third_party_invite/(?P<room_id>[^/]*)"
async def on_PUT(self, origin, content, query, room_id):
- content = await self.handler.on_exchange_third_party_invite_request(
- room_id, content
- )
+ content = await self.handler.on_exchange_third_party_invite_request(content)
return 200, content
diff --git a/synapse/handlers/appservice.py b/synapse/handlers/appservice.py
index 9fc8444228..5c6458eb52 100644
--- a/synapse/handlers/appservice.py
+++ b/synapse/handlers/appservice.py
@@ -226,7 +226,7 @@ class ApplicationServicesHandler:
new_token: Optional[int],
users: Collection[Union[str, UserID]],
):
- logger.info("Checking interested services for %s" % (stream_key))
+ logger.debug("Checking interested services for %s" % (stream_key))
with Measure(self.clock, "notify_interested_services_ephemeral"):
for service in services:
# Only handle typing if we have the latest token
diff --git a/synapse/handlers/federation.py b/synapse/handlers/federation.py
index 4fc619e802..f27dd84213 100644
--- a/synapse/handlers/federation.py
+++ b/synapse/handlers/federation.py
@@ -55,6 +55,7 @@ from synapse.events import EventBase
from synapse.events.snapshot import EventContext
from synapse.events.validator import EventValidator
from synapse.handlers._base import BaseHandler
+from synapse.http.servlet import assert_params_in_dict
from synapse.logging.context import (
make_deferred_yieldable,
nested_logging_context,
@@ -2698,7 +2699,7 @@ class FederationHandler(BaseHandler):
)
async def on_exchange_third_party_invite_request(
- self, room_id: str, event_dict: JsonDict
+ self, event_dict: JsonDict
) -> None:
"""Handle an exchange_third_party_invite request from a remote server
@@ -2706,12 +2707,11 @@ class FederationHandler(BaseHandler):
into a normal m.room.member invite.
Args:
- room_id: The ID of the room.
-
- event_dict (dict[str, Any]): Dictionary containing the event body.
+ event_dict: Dictionary containing the event body.
"""
- room_version = await self.store.get_room_version_id(room_id)
+ assert_params_in_dict(event_dict, ["room_id"])
+ room_version = await self.store.get_room_version_id(event_dict["room_id"])
# NB: event_dict has a particular specced format we might need to fudge
# if we change event formats too much.
diff --git a/synapse/handlers/oidc_handler.py b/synapse/handlers/oidc_handler.py
index be8562d47b..4bfd8d5617 100644
--- a/synapse/handlers/oidc_handler.py
+++ b/synapse/handlers/oidc_handler.py
@@ -38,7 +38,12 @@ from synapse.handlers._base import BaseHandler
from synapse.handlers.sso import MappingException
from synapse.http.site import SynapseRequest
from synapse.logging.context import make_deferred_yieldable
-from synapse.types import JsonDict, UserID, map_username_to_mxid_localpart
+from synapse.types import (
+ JsonDict,
+ UserID,
+ contains_invalid_mxid_characters,
+ map_username_to_mxid_localpart,
+)
from synapse.util import json_decoder
if TYPE_CHECKING:
@@ -885,10 +890,12 @@ class OidcHandler(BaseHandler):
"Retrieved user attributes from user mapping provider: %r", attributes
)
- if not attributes["localpart"]:
- raise MappingException("localpart is empty")
-
- localpart = map_username_to_mxid_localpart(attributes["localpart"])
+ localpart = attributes["localpart"]
+ if not localpart:
+ raise MappingException(
+ "Error parsing OIDC response: OIDC mapping provider plugin "
+ "did not return a localpart value"
+ )
user_id = UserID(localpart, self.server_name).to_string()
users = await self.store.get_users_by_id_case_insensitive(user_id)
@@ -908,6 +915,11 @@ class OidcHandler(BaseHandler):
# This mxid is taken
raise MappingException("mxid '{}' is already taken".format(user_id))
else:
+ # Since the localpart is provided via a potentially untrusted module,
+ # ensure the MXID is valid before registering.
+ if contains_invalid_mxid_characters(localpart):
+ raise MappingException("localpart is invalid: %s" % (localpart,))
+
# It's the first time this user is logging in and the mapped mxid was
# not taken, register the user
registered_user_id = await self._registration_handler.register_user(
@@ -1076,6 +1088,9 @@ class JinjaOidcMappingProvider(OidcMappingProvider[JinjaOidcMappingConfig]):
) -> UserAttribute:
localpart = self._config.localpart_template.render(user=userinfo).strip()
+ # Ensure only valid characters are included in the MXID.
+ localpart = map_username_to_mxid_localpart(localpart)
+
display_name = None # type: Optional[str]
if self._config.display_name_template is not None:
display_name = self._config.display_name_template.render(
diff --git a/synapse/handlers/receipts.py b/synapse/handlers/receipts.py
index c242c409cf..153cbae7b9 100644
--- a/synapse/handlers/receipts.py
+++ b/synapse/handlers/receipts.py
@@ -158,7 +158,8 @@ class ReceiptEventSource:
if from_key == to_key:
return [], to_key
- # We first need to fetch all new receipts
+ # Fetch all read receipts for all rooms, up to a limit of 100. This is ordered
+ # by most recent.
rooms_to_events = await self.store.get_linearized_receipts_for_all_rooms(
from_key=from_key, to_key=to_key
)
diff --git a/synapse/handlers/saml_handler.py b/synapse/handlers/saml_handler.py
index aee772239a..5d9b555b13 100644
--- a/synapse/handlers/saml_handler.py
+++ b/synapse/handlers/saml_handler.py
@@ -31,6 +31,7 @@ from synapse.http.site import SynapseRequest
from synapse.module_api import ModuleApi
from synapse.types import (
UserID,
+ contains_invalid_mxid_characters,
map_username_to_mxid_localpart,
mxid_localpart_allowed_characters,
)
@@ -58,6 +59,7 @@ class SamlHandler(BaseHandler):
def __init__(self, hs: "synapse.server.HomeServer"):
super().__init__(hs)
self._saml_client = Saml2Client(hs.config.saml2_sp_config)
+ self._saml_idp_entityid = hs.config.saml2_idp_entityid
self._auth_handler = hs.get_auth_handler()
self._registration_handler = hs.get_registration_handler()
@@ -100,7 +102,7 @@ class SamlHandler(BaseHandler):
URL to redirect to
"""
reqid, info = self._saml_client.prepare_for_authenticate(
- relay_state=client_redirect_url
+ entityid=self._saml_idp_entityid, relay_state=client_redirect_url
)
# Since SAML sessions timeout it is useful to log when they were created.
@@ -317,6 +319,11 @@ class SamlHandler(BaseHandler):
"Unable to generate a Matrix ID from the SAML response"
)
+ # Since the localpart is provided via a potentially untrusted module,
+ # ensure the MXID is valid before registering.
+ if contains_invalid_mxid_characters(localpart):
+ raise MappingException("localpart is invalid: %s" % (localpart,))
+
logger.info("Mapped SAML user to local part %s", localpart)
registered_user_id = await self._registration_handler.register_user(
localpart=localpart,
diff --git a/synapse/storage/databases/main/receipts.py b/synapse/storage/databases/main/receipts.py
index ca7917c989..1e7949a323 100644
--- a/synapse/storage/databases/main/receipts.py
+++ b/synapse/storage/databases/main/receipts.py
@@ -278,7 +278,8 @@ class ReceiptsWorkerStore(SQLBaseStore, metaclass=abc.ABCMeta):
async def get_linearized_receipts_for_all_rooms(
self, to_key: int, from_key: Optional[int] = None
) -> Dict[str, JsonDict]:
- """Get receipts for all rooms between two stream_ids.
+ """Get receipts for all rooms between two stream_ids, up
+ to a limit of the latest 100 read receipts.
Args:
to_key: Max stream id to fetch receipts upto.
@@ -294,12 +295,16 @@ class ReceiptsWorkerStore(SQLBaseStore, metaclass=abc.ABCMeta):
sql = """
SELECT * FROM receipts_linearized WHERE
stream_id > ? AND stream_id <= ?
+ ORDER BY stream_id DESC
+ LIMIT 100
"""
txn.execute(sql, [from_key, to_key])
else:
sql = """
SELECT * FROM receipts_linearized WHERE
stream_id <= ?
+ ORDER BY stream_id DESC
+ LIMIT 100
"""
txn.execute(sql, [to_key])
diff --git a/synapse/types.py b/synapse/types.py
index 5e52c1eb12..7c9f716804 100644
--- a/synapse/types.py
+++ b/synapse/types.py
@@ -318,14 +318,14 @@ mxid_localpart_allowed_characters = set(
)
-def contains_invalid_mxid_characters(localpart):
+def contains_invalid_mxid_characters(localpart: str) -> bool:
"""Check for characters not allowed in an mxid or groupid localpart
Args:
- localpart (basestring): the localpart to be checked
+ localpart: the localpart to be checked
Returns:
- bool: True if there are any naughty characters
+ True if there are any naughty characters
"""
return any(c not in mxid_localpart_allowed_characters for c in localpart)
diff --git a/tests/handlers/test_federation.py b/tests/handlers/test_federation.py
index 9ef80fe502..bf866dacf3 100644
--- a/tests/handlers/test_federation.py
+++ b/tests/handlers/test_federation.py
@@ -59,7 +59,6 @@ class FederationTestCase(unittest.HomeserverTestCase):
)
d = self.handler.on_exchange_third_party_invite_request(
- room_id=room_id,
event_dict={
"type": EventTypes.Member,
"room_id": room_id,
diff --git a/tests/handlers/test_oidc.py b/tests/handlers/test_oidc.py
index 630e6da808..b4fa02acc4 100644
--- a/tests/handlers/test_oidc.py
+++ b/tests/handlers/test_oidc.py
@@ -12,7 +12,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-
import json
from urllib.parse import parse_qs, urlparse
@@ -24,12 +23,8 @@ import pymacaroons
from twisted.python.failure import Failure
from twisted.web._newclient import ResponseDone
-from synapse.handlers.oidc_handler import (
- MappingException,
- OidcError,
- OidcHandler,
- OidcMappingProvider,
-)
+from synapse.handlers.oidc_handler import OidcError, OidcHandler, OidcMappingProvider
+from synapse.handlers.sso import MappingException
from synapse.types import UserID
from tests.unittest import HomeserverTestCase, override_config
@@ -132,14 +127,13 @@ class OidcHandlerTestCase(HomeserverTestCase):
config = self.default_config()
config["public_baseurl"] = BASE_URL
- oidc_config = {}
- oidc_config["enabled"] = True
- oidc_config["client_id"] = CLIENT_ID
- oidc_config["client_secret"] = CLIENT_SECRET
- oidc_config["issuer"] = ISSUER
- oidc_config["scopes"] = SCOPES
- oidc_config["user_mapping_provider"] = {
- "module": __name__ + ".TestMappingProvider",
+ oidc_config = {
+ "enabled": True,
+ "client_id": CLIENT_ID,
+ "client_secret": CLIENT_SECRET,
+ "issuer": ISSUER,
+ "scopes": SCOPES,
+ "user_mapping_provider": {"module": __name__ + ".TestMappingProvider"},
}
# Update this config with what's in the default config so that
@@ -705,13 +699,13 @@ class OidcHandlerTestCase(HomeserverTestCase):
def test_map_userinfo_to_existing_user(self):
"""Existing users can log in with OpenID Connect when allow_existing_users is True."""
store = self.hs.get_datastore()
- user4 = UserID.from_string("@test_user_4:test")
+ user = UserID.from_string("@test_user:test")
self.get_success(
- store.register_user(user_id=user4.to_string(), password_hash=None)
+ store.register_user(user_id=user.to_string(), password_hash=None)
)
userinfo = {
- "sub": "test4",
- "username": "test_user_4",
+ "sub": "test",
+ "username": "test_user",
}
token = {}
mxid = self.get_success(
@@ -719,4 +713,59 @@ class OidcHandlerTestCase(HomeserverTestCase):
userinfo, token, "user-agent", "10.10.10.10"
)
)
- self.assertEqual(mxid, "@test_user_4:test")
+ self.assertEqual(mxid, "@test_user:test")
+
+ # Register some non-exact matching cases.
+ user2 = UserID.from_string("@TEST_user_2:test")
+ self.get_success(
+ store.register_user(user_id=user2.to_string(), password_hash=None)
+ )
+ user2_caps = UserID.from_string("@test_USER_2:test")
+ self.get_success(
+ store.register_user(user_id=user2_caps.to_string(), password_hash=None)
+ )
+
+ # Attempting to login without matching a name exactly is an error.
+ userinfo = {
+ "sub": "test2",
+ "username": "TEST_USER_2",
+ }
+ e = self.get_failure(
+ self.handler._map_userinfo_to_user(
+ userinfo, token, "user-agent", "10.10.10.10"
+ ),
+ MappingException,
+ )
+ self.assertTrue(
+ str(e.value).startswith(
+ "Attempted to login as '@TEST_USER_2:test' but it matches more than one user inexactly:"
+ )
+ )
+
+ # Logging in when matching a name exactly should work.
+ user2 = UserID.from_string("@TEST_USER_2:test")
+ self.get_success(
+ store.register_user(user_id=user2.to_string(), password_hash=None)
+ )
+
+ mxid = self.get_success(
+ self.handler._map_userinfo_to_user(
+ userinfo, token, "user-agent", "10.10.10.10"
+ )
+ )
+ self.assertEqual(mxid, "@TEST_USER_2:test")
+
+ def test_map_userinfo_to_invalid_localpart(self):
+ """If the mapping provider generates an invalid localpart it should be rejected."""
+ userinfo = {
+ "sub": "test2",
+ "username": "föö",
+ }
+ token = {}
+ e = self.get_failure(
+ self.handler._map_userinfo_to_user(
+ userinfo, token, "user-agent", "10.10.10.10"
+ ),
+ MappingException,
+ )
+ self.assertEqual(str(e.value), "localpart is invalid: föö")
|
