1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
|
# Summary
# Introduction
- [Welcome and Overview](welcome_and_overview.md)
# Setup
- [Installation](setup/installation.md)
- [Using Postgres](postgres.md)
- [Configuring a Reverse Proxy](reverse_proxy.md)
- [Configuring a Forward/Outbound Proxy](setup/forward_proxy.md)
- [Configuring a Turn Server](turn-howto.md)
- [Delegation](delegate.md)
# Upgrading
- [Upgrading between Synapse Versions](upgrade.md)
# Usage
- [Federation](federate.md)
- [Configuration](usage/configuration/README.md)
- [Configuration Manual](usage/configuration/config_documentation.md)
- [Homeserver Sample Config File](usage/configuration/homeserver_sample_config.md)
- [Logging Sample Config File](usage/configuration/logging_sample_config.md)
- [Structured Logging](structured_logging.md)
- [Templates](templates.md)
- [User Authentication](usage/configuration/user_authentication/README.md)
- [Single-Sign On](usage/configuration/user_authentication/single_sign_on/README.md)
- [OpenID Connect](openid.md)
- [SAML](usage/configuration/user_authentication/single_sign_on/saml.md)
- [CAS](usage/configuration/user_authentication/single_sign_on/cas.md)
- [SSO Mapping Providers](sso_mapping_providers.md)
- [Password Auth Providers](password_auth_providers.md)
- [JSON Web Tokens](jwt.md)
- [Refresh Tokens](usage/configuration/user_authentication/refresh_tokens.md)
- [Registration Captcha](CAPTCHA_SETUP.md)
- [Application Services](application_services.md)
- [Server Notices](server_notices.md)
- [Consent Tracking](consent_tracking.md)
- [URL Previews](development/url_previews.md)
- [User Directory](user_directory.md)
- [Message Retention Policies](message_retention_policies.md)
- [Pluggable Modules](modules/index.md)
- [Writing a module](modules/writing_a_module.md)
- [Spam checker callbacks](modules/spam_checker_callbacks.md)
- [Third-party rules callbacks](modules/third_party_rules_callbacks.md)
- [Presence router callbacks](modules/presence_router_callbacks.md)
- [Account validity callbacks](modules/account_validity_callbacks.md)
- [Password auth provider callbacks](modules/password_auth_provider_callbacks.md)
- [Background update controller callbacks](modules/background_update_controller_callbacks.md)
- [Account data callbacks](modules/account_data_callbacks.md)
- [Porting a legacy module to the new interface](modules/porting_legacy_module.md)
- [Workers](workers.md)
- [Using `synctl` with Workers](synctl_workers.md)
- [Systemd](systemd-with-workers/README.md)
- [Administration](usage/administration/README.md)
- [Admin API](usage/administration/admin_api/README.md)
- [Account Validity](admin_api/account_validity.md)
- [Background Updates](usage/administration/admin_api/background_updates.md)
- [Delete Group](admin_api/delete_group.md)
- [Event Reports](admin_api/event_reports.md)
- [Media](admin_api/media_admin_api.md)
- [Purge History](admin_api/purge_history_api.md)
- [Register Users](admin_api/register_api.md)
- [Registration Tokens](usage/administration/admin_api/registration_tokens.md)
- [Manipulate Room Membership](admin_api/room_membership.md)
- [Rooms](admin_api/rooms.md)
- [Server Notices](admin_api/server_notices.md)
- [Statistics](admin_api/statistics.md)
- [Users](admin_api/user_admin_api.md)
- [Server Version](admin_api/version_api.md)
- [Federation](usage/administration/admin_api/federation.md)
- [Manhole](manhole.md)
- [Monitoring](metrics-howto.md)
- [Understanding Synapse Through Grafana Graphs](usage/administration/understanding_synapse_through_grafana_graphs.md)
- [Useful SQL for Admins](usage/administration/useful_sql_for_admins.md)
- [Database Maintenance Tools](usage/administration/database_maintenance_tools.md)
- [State Groups](usage/administration/state_groups.md)
- [Request log format](usage/administration/request_log.md)
- [Admin FAQ](usage/administration/admin_faq.md)
- [Scripts]()
# Development
- [Contributing Guide](development/contributing_guide.md)
- [Code Style](code_style.md)
- [Release Cycle](development/releases.md)
- [Git Usage](development/git.md)
- [Testing]()
- [Demo scripts](development/demo.md)
- [OpenTracing](opentracing.md)
- [Database Schemas](development/database_schema.md)
- [Experimental features](development/experimental_features.md)
- [Synapse Architecture](development/synapse_architechture.md)
- [Overview]()
- Flows of data through Synapse
- Starts with a Request/BG Job/Module
- Processing
- Maybe Response
- Config
- Request Handling
- Clients
- Federation
- Internal Processing - there could be an exception!
- Storage
- Federation
- External Entities: Remote homeservers, Appservices, Pushers, Identity Servers
- Modules
- Workers/Replication/Redis/Organization of a large homeserver instance
- [Configuration]()
- Experimental config options
- Define how long they should live after the corresponding msc has merged
- [Logging]()
- combine with [Log Contexts](log_contexts.md)
- [Storage]()
- What is current database schema?
- Start by linking script that can compile full schema
- Is there a tool that can visualize a postgres db? - Shay to check
- [Database Backends]()
- [Database Migrations]()
- Background [Schema] Updates
- [Caching]()
- Cache tuning
- How to check sizes
- Nice to have a metric for max cache sizes
- How to add a cache to a [storage] function
- Why would you
- How to invalidate (with support for workers)
- [Request Listening]()
- Mention that we use Twisted
- [Servlets and Resources]()
- When to use one over the other (always use a servlet?)
- Unstable/release-based endpoints
- Don't put too many servlets on the same resource
- [Handlers]()
- Machinery behind the servlet that does all the processing
- [Background Jobs]()
- [Federation]()
- Federation Catchup
- [Data Types]()
- [Users]()
- Allowed username characters
- [Devices]()
- [Events]()
- What is an event?
- Different event formats depending on room version
- Outlier events, rejected events, dropped events
- [Rooms]()
- [Room DAG concepts](development/room-dag-concepts.md)
- [Sync]()
- [State Resolution]()
- [The Auth Chain Difference Algorithm](auth_chain_difference_algorithm.md)
- [User Authentication]()
- [Single Sign-On]()
- [SAML](development/saml.md)
- [CAS](development/cas.md)
- [User-Interactive Auth]()
- [Password-based]()
- [Password Auth Modules]()
- [Token-based]()
- [Rate Limiting]()
- Different rate limiting classes (Ratelimiter vs RequestRatelimiter)
- How does our rate limiting work
- When to use/add rate limiting
- Rate limiting for federation traffic
- [Media Repository](media_repository.md)
- [Email and HTML Templating]()
- [Presence]()
- What is Presence?
- How is it designed to work currently
- Why it is recommended to be disabled
- Why is this resource intensive
- Links and issues on how to fix this
- How it's implemented
- [Application Services]()
- [Push Notifications]()
- [Synapse Admin API]()
- [Synapse Modules]()
- [Workers]()
- Things to be mindful of in order to make your feature work with workers
- How to test a feature on workers
- [Replication](replication.md)
- [TCP Replication](tcp_replication.md)
- [Sources]()
- TypingSource, ReceiptSource, PresenceSource, etc
- Classes containing methods for storing and getting different types of data
- [Streams]()
- Stream Types (the ID of the stream)
- Stream Token: each entity added to the stream gets a unique, incremental token
- StreamReplication classes and clients
- How this interacts with sync?
- Stream token generators: Single Writer Only (sqlite), MultiWriterStreamIDGenerator (Postgres)
- [Notifier]()
- [Monitoring and Metrics]() # opentracing, phone home and metrics
- [Opentracing]()
- How does this work?
- How is this helpful?
- What's jaeger?
- How do I create and mutate a span?
- How are spans sent between homeservers?
- Only between whitelisted homeservers
- What areas are currently traced
- Database layer
- Servlets
- Crypto stuff
- [Metrics]()
- Prometheus
- How to make a new metrics / good practices
- What are all the current metrics and what each one means
- How to view in grafana
- Here's a quick example of making a new panel to visualize metrics
- Btw it would be nice to have a tooltip with an explainer for every graph in graphana
- [Room and User Statistics](room_and_user_statistics.md) # TODO: This page is currently useless
- [Testing]()
- [Unit Tests]()
- How to run the tests (partially in the contributing guide)
- IDE-specific examples
- How to run one test / one test case / run in parallel
- Documenting test environment variables: SYNAPSE_TEST_LOG_LEVEL, SYNAPSE_TEST_PERSISTENT_SQLITE_DB, SYNAPSE_POSTGRES etc.
- Test case versus test method
- prepare, make_homeserver, default_config, @override_config decorator, @parameterized decorator, pump/reactor.advance
- Test utilities: fake homeserver, federation, workers, channels, requests, HomeServerTestCase.helper
- How to use them in your tests at a high level
- How they are implemented
- Different inherited TestCase classes: HomeServerTestCase, FederatedHomeServerTestCase(sp?), TestCase
- Test pattern: How to start a request, check the state of the homeserver, respond to the request
- Ensuring the right servlets for your test are registered
- Good practice: use storage methods vs. tampering with the database directly (database schema changes)
- Dealing with deferreds/async (get_success/get_failure).
- Mocking - async mocks
- Organization of tests
- Skipping tests
- [Integration Tests]()
- [Complement]()
- Point to Complement's documentation on its structure and how to write a test
- How to run Complement against Synapse
- Document Synapse's Complement images and how they're built
- Which Complement build flags does Synapse support
- Unstable features
- Synapse's Complement Blacklist
- [Sytest]()
- You should prefer Complement :)
- Helper functions
- Dealing with race conditions
- Link to that one perl helper doc Matthew wrote
- [Scripts]()
# Other
- [Dependency Deprecation Policy](deprecation_policy.md)
- [Running Synapse on a Single-Board Computer](other/running_synapse_on_single_board_computers.md)
|
