# Summary # Introduction - [Welcome and Overview](welcome_and_overview.md) # Setup - [Installation](setup/installation.md) - [Using Postgres](postgres.md) - [Configuring a Reverse Proxy](reverse_proxy.md) - [Configuring a Forward/Outbound Proxy](setup/forward_proxy.md) - [Configuring a Turn Server](turn-howto.md) - [Delegation](delegate.md) # Upgrading - [Upgrading between Synapse Versions](upgrade.md) # Usage - [Federation](federate.md) - [Configuration](usage/configuration/README.md) - [Configuration Manual](usage/configuration/config_documentation.md) - [Homeserver Sample Config File](usage/configuration/homeserver_sample_config.md) - [Logging Sample Config File](usage/configuration/logging_sample_config.md) - [Structured Logging](structured_logging.md) - [Templates](templates.md) - [User Authentication](usage/configuration/user_authentication/README.md) - [Single-Sign On](usage/configuration/user_authentication/single_sign_on/README.md) - [OpenID Connect](openid.md) - [SAML](usage/configuration/user_authentication/single_sign_on/saml.md) - [CAS](usage/configuration/user_authentication/single_sign_on/cas.md) - [SSO Mapping Providers](sso_mapping_providers.md) - [Password Auth Providers](password_auth_providers.md) - [JSON Web Tokens](jwt.md) - [Refresh Tokens](usage/configuration/user_authentication/refresh_tokens.md) - [Registration Captcha](CAPTCHA_SETUP.md) - [Application Services](application_services.md) - [Server Notices](server_notices.md) - [Consent Tracking](consent_tracking.md) - [URL Previews](development/url_previews.md) - [User Directory](user_directory.md) - [Message Retention Policies](message_retention_policies.md) - [Pluggable Modules](modules/index.md) - [Writing a module](modules/writing_a_module.md) - [Spam checker callbacks](modules/spam_checker_callbacks.md) - [Third-party rules callbacks](modules/third_party_rules_callbacks.md) - [Presence router callbacks](modules/presence_router_callbacks.md) - [Account validity callbacks](modules/account_validity_callbacks.md) - [Password auth provider callbacks](modules/password_auth_provider_callbacks.md) - [Background update controller callbacks](modules/background_update_controller_callbacks.md) - [Account data callbacks](modules/account_data_callbacks.md) - [Porting a legacy module to the new interface](modules/porting_legacy_module.md) - [Workers](workers.md) - [Using `synctl` with Workers](synctl_workers.md) - [Systemd](systemd-with-workers/README.md) - [Administration](usage/administration/README.md) - [Admin API](usage/administration/admin_api/README.md) - [Account Validity](admin_api/account_validity.md) - [Background Updates](usage/administration/admin_api/background_updates.md) - [Delete Group](admin_api/delete_group.md) - [Event Reports](admin_api/event_reports.md) - [Media](admin_api/media_admin_api.md) - [Purge History](admin_api/purge_history_api.md) - [Register Users](admin_api/register_api.md) - [Registration Tokens](usage/administration/admin_api/registration_tokens.md) - [Manipulate Room Membership](admin_api/room_membership.md) - [Rooms](admin_api/rooms.md) - [Server Notices](admin_api/server_notices.md) - [Statistics](admin_api/statistics.md) - [Users](admin_api/user_admin_api.md) - [Server Version](admin_api/version_api.md) - [Federation](usage/administration/admin_api/federation.md) - [Manhole](manhole.md) - [Monitoring](metrics-howto.md) - [Understanding Synapse Through Grafana Graphs](usage/administration/understanding_synapse_through_grafana_graphs.md) - [Useful SQL for Admins](usage/administration/useful_sql_for_admins.md) - [Database Maintenance Tools](usage/administration/database_maintenance_tools.md) - [State Groups](usage/administration/state_groups.md) - [Request log format](usage/administration/request_log.md) - [Admin FAQ](usage/administration/admin_faq.md) - [Scripts]() # Development - [Contributing Guide](development/contributing_guide.md) - [Code Style](code_style.md) - [Release Cycle](development/releases.md) - [Git Usage](development/git.md) - [Testing]() - [Demo scripts](development/demo.md) - [OpenTracing](opentracing.md) - [Database Schemas](development/database_schema.md) - [Experimental features](development/experimental_features.md) - [Synapse Architecture](development/synapse_architechture.md) - [Overview]() - Flows of data through Synapse - Starts with a Request/BG Job/Module - Processing - Maybe Response - Config - Request Handling - Clients - Federation - Internal Processing - there could be an exception! - Storage - Federation - External Entities: Remote homeservers, Appservices, Pushers, Identity Servers - Modules - Workers/Replication/Redis/Organization of a large homeserver instance - [Configuration]() - Experimental config options - Define how long they should live after the corresponding msc has merged - [Logging]() - combine with [Log Contexts](log_contexts.md) - [Storage]() - What is current database schema? - Start by linking script that can compile full schema - Is there a tool that can visualize a postgres db? - Shay to check - [Database Backends]() - [Database Migrations]() - Background [Schema] Updates - [Caching]() - Cache tuning - How to check sizes - Nice to have a metric for max cache sizes - How to add a cache to a [storage] function - Why would you - How to invalidate (with support for workers) - [Request Listening]() - Mention that we use Twisted - [Servlets and Resources]() - When to use one over the other (always use a servlet?) - Unstable/release-based endpoints - Don't put too many servlets on the same resource - [Handlers]() - Machinery behind the servlet that does all the processing - [Background Jobs]() - [Federation]() - Federation Catchup - [Data Types]() - [Users]() - Allowed username characters - [Devices]() - [Events]() - What is an event? - Different event formats depending on room version - Outlier events, rejected events, dropped events - [Rooms]() - [Room DAG concepts](development/room-dag-concepts.md) - [Sync]() - [State Resolution]() - [The Auth Chain Difference Algorithm](auth_chain_difference_algorithm.md) - [User Authentication]() - [Single Sign-On]() - [SAML](development/saml.md) - [CAS](development/cas.md) - [User-Interactive Auth]() - [Password-based]() - [Password Auth Modules]() - [Token-based]() - [Rate Limiting]() - Different rate limiting classes (Ratelimiter vs RequestRatelimiter) - How does our rate limiting work - When to use/add rate limiting - Rate limiting for federation traffic - [Media Repository](media_repository.md) - [Email and HTML Templating]() - [Presence]() - What is Presence? - How is it designed to work currently - Why it is recommended to be disabled - Why is this resource intensive - Links and issues on how to fix this - How it's implemented - [Application Services]() - [Push Notifications]() - [Synapse Admin API]() - [Synapse Modules]() - [Workers]() - Things to be mindful of in order to make your feature work with workers - How to test a feature on workers - [Replication](replication.md) - [TCP Replication](tcp_replication.md) - [Sources]() - TypingSource, ReceiptSource, PresenceSource, etc - Classes containing methods for storing and getting different types of data - [Streams]() - Stream Types (the ID of the stream) - Stream Token: each entity added to the stream gets a unique, incremental token - StreamReplication classes and clients - How this interacts with sync? - Stream token generators: Single Writer Only (sqlite), MultiWriterStreamIDGenerator (Postgres) - [Notifier]() - [Monitoring and Metrics]() # opentracing, phone home and metrics - [Opentracing]() - How does this work? - How is this helpful? - What's jaeger? - How do I create and mutate a span? - How are spans sent between homeservers? - Only between whitelisted homeservers - What areas are currently traced - Database layer - Servlets - Crypto stuff - [Metrics]() - Prometheus - How to make a new metrics / good practices - What are all the current metrics and what each one means - How to view in grafana - Here's a quick example of making a new panel to visualize metrics - Btw it would be nice to have a tooltip with an explainer for every graph in graphana - [Room and User Statistics](room_and_user_statistics.md) # TODO: This page is currently useless - [Testing]() - [Unit Tests]() - How to run the tests (partially in the contributing guide) - IDE-specific examples - How to run one test / one test case / run in parallel - Documenting test environment variables: SYNAPSE_TEST_LOG_LEVEL, SYNAPSE_TEST_PERSISTENT_SQLITE_DB, SYNAPSE_POSTGRES etc. - Test case versus test method - prepare, make_homeserver, default_config, @override_config decorator, @parameterized decorator, pump/reactor.advance - Test utilities: fake homeserver, federation, workers, channels, requests, HomeServerTestCase.helper - How to use them in your tests at a high level - How they are implemented - Different inherited TestCase classes: HomeServerTestCase, FederatedHomeServerTestCase(sp?), TestCase - Test pattern: How to start a request, check the state of the homeserver, respond to the request - Ensuring the right servlets for your test are registered - Good practice: use storage methods vs. tampering with the database directly (database schema changes) - Dealing with deferreds/async (get_success/get_failure). - Mocking - async mocks - Organization of tests - Skipping tests - [Integration Tests]() - [Complement]() - Point to Complement's documentation on its structure and how to write a test - How to run Complement against Synapse - Document Synapse's Complement images and how they're built - Which Complement build flags does Synapse support - Unstable features - Synapse's Complement Blacklist - [Sytest]() - You should prefer Complement :) - Helper functions - Dealing with race conditions - Link to that one perl helper doc Matthew wrote - [Scripts]() # Other - [Dependency Deprecation Policy](deprecation_policy.md) - [Running Synapse on a Single-Board Computer](other/running_synapse_on_single_board_computers.md)