Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Fix backfill being able to cleanly branch into history and back to "live" | Eric Eastwood | 2021-10-29 | 1 | -22/+62 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix https://github.com/matrix-org/synapse/pull/11114#discussion_r731363046 - When we see a connected insertion event indicating a branch of history, we now don't accidentally leak the base event we're branching from in the backfill chunk until all of the events in the historical brach are exhausted. - Backfill will now check whether the federating homeserver asking has a given insertion event when we see one connected to the DAG. This allows backfill to initially decide to give all the historical messages in the branch, then next time when the base event we branched from comes up in backfill again, it sees that federated homeserver already has the insertion event, and we can continue down the normal DAG prev_event path instead. Example Backfill response (in order yay): Overview: ``` eventIDsAfter batch0 batch1 eventIDsBefore ``` ``` new implementation backfill events(100)=[ "event_id=$q3w2dF5mWBkgUF1ADdVgS_bX7YQuqODJ2HC33bln4Ho,depth=11,body=Message 1 (eventIDsAfter),prevs=['$Com3-ZIR35FvYcBQvHdpxGsS1iW-DO_Z5x1zZ5R5Az8']", "event_id=$Com3-ZIR35FvYcBQvHdpxGsS1iW-DO_Z5x1zZ5R5Az8,depth=10,body=Message 0 (eventIDsAfter),prevs=['$T5-OUXAW3CCYAHskqk5s0V2ZzJoQy5D8lQJ7LZFWvTI']", "event_id=$78Yy5bqa9XuQ5fhcw9AF36Xf4W5EDcjsraSInif7nYU,depth=10,body=org.matrix.msc2716.insertion,prevs=['$T5-OUXAW3CCYAHskqk5s0V2ZzJoQy5D8lQJ7LZFWvTI']", "event_id=$GEzF4euV8StAtdCDCtWmpdhxRE3KQWgreyr1_ePqlpM,depth=10,body=org.matrix.msc2716.batch,prevs=['$ot97LEPqM55EV1ka8YM445c6q-ny4jBBRxjgQnGxihQ']", "event_id=$ot97LEPqM55EV1ka8YM445c6q-ny4jBBRxjgQnGxihQ,depth=10,body=Historical 99 (batch=0),prevs=['$i53GQczFiX-tqXgcxudp5AWR9C-A0yJiET_ZXe9D49I']", "event_id=$i53GQczFiX-tqXgcxudp5AWR9C-A0yJiET_ZXe9D49I,depth=10,body=Historical 98 (batch=0),prevs=['$lqjhSBLzbi2WfWNAIm1mngStqsPfQJUwPO0NAFtB8lo']", "event_id=$lqjhSBLzbi2WfWNAIm1mngStqsPfQJUwPO0NAFtB8lo,depth=10,body=Historical 97 (batch=0),prevs=['$k8OC791p_vB-Hz36Byn-o3vK9x_rc0qkVufBpxYRcaA']", "event_id=$k8OC791p_vB-Hz36Byn-o3vK9x_rc0qkVufBpxYRcaA,depth=10,body=Historical 96 (batch=0),prevs=['$9obrnkfP2HpDfKUkWyHrtHML5_31MZ3T9sKQOb6eUV8']", "event_id=$9obrnkfP2HpDfKUkWyHrtHML5_31MZ3T9sKQOb6eUV8,depth=10,body=Historical 95 (batch=0),prevs=['$PSNLMD1RhBvR9ffpbEOKtUHE9PUUiacJ2g8_DA9nSjM']", "event_id=$PSNLMD1RhBvR9ffpbEOKtUHE9PUUiacJ2g8_DA9nSjM,depth=10,body=Historical 94 (batch=0),prevs=['$vGkUiP049MLeapDGHegqdiGZRkuc6WxEJ-iOAN7mqr4']", "event_id=$vGkUiP049MLeapDGHegqdiGZRkuc6WxEJ-iOAN7mqr4,depth=10,body=Historical 93 (batch=0),prevs=['$reYXzhTXHlqqk6XMLK_mEIFQGIesn4NJ70JYa7EziQc']", "event_id=$reYXzhTXHlqqk6XMLK_mEIFQGIesn4NJ70JYa7EziQc,depth=10,body=Historical 92 (batch=0),prevs=['$dg8ZPPPqh1SbIhcg-GMvEGz49h53TUMoZ8yO4zbfA4U']", "event_id=$dg8ZPPPqh1SbIhcg-GMvEGz49h53TUMoZ8yO4zbfA4U,depth=10,body=Historical 91 (batch=0),prevs=['$QW7wied5SyHtJElJTo4SUkc-zNLlAh6tbCiAa5EJXg4']", "event_id=$QW7wied5SyHtJElJTo4SUkc-zNLlAh6tbCiAa5EJXg4,depth=10,body=Historical 90 (batch=0),prevs=['$COQuVmW8z0geXFZ2fPdo_kW0q1NK1LLZSI7fZNIwvgM']", "event_id=$COQuVmW8z0geXFZ2fPdo_kW0q1NK1LLZSI7fZNIwvgM,depth=10,body=Historical 89 (batch=0),prevs=['$04k2wOdm-L41JDQ73KPpDe1njG3L9FYjOsNVamFsDQ0']", "event_id=$04k2wOdm-L41JDQ73KPpDe1njG3L9FYjOsNVamFsDQ0,depth=10,body=Historical 88 (batch=0),prevs=['$70BsBGyeg4Pf3VhGhXjtk4N5XKNNSit4vjnrirwTmic']", "event_id=$70BsBGyeg4Pf3VhGhXjtk4N5XKNNSit4vjnrirwTmic,depth=10,body=Historical 87 (batch=0),prevs=['$Y_6AET6hMZY42rPki4r0GxMfs8E_TCJyepNL1Vfwnh4']", "event_id=$Y_6AET6hMZY42rPki4r0GxMfs8E_TCJyepNL1Vfwnh4,depth=10,body=Historical 86 (batch=0),prevs=['$jSk0bO6PFUcqxHsjTvFPyJZoXL4N9nEGex7I-yyvBj8']", "event_id=$jSk0bO6PFUcqxHsjTvFPyJZoXL4N9nEGex7I-yyvBj8,depth=10,body=Historical 85 (batch=0),prevs=['$wPV_BB23RIolkHz0yGTvwt1-TJbqpGPqZc33vR5DUMw']", "event_id=$wPV_BB23RIolkHz0yGTvwt1-TJbqpGPqZc33vR5DUMw,depth=10,body=Historical 84 (batch=0),prevs=['$X2FJ_Jmoy5sx9u0UWTN1bkds6diO0uDxxRGco4r9Msw']", "event_id=$X2FJ_Jmoy5sx9u0UWTN1bkds6diO0uDxxRGco4r9Msw,depth=10,body=Historical 83 (batch=0),prevs=['$jrmzKa3aDmpZg5HgDHQqQ_lgl9L4rDWiWk7xd1igfAE']", "event_id=$jrmzKa3aDmpZg5HgDHQqQ_lgl9L4rDWiWk7xd1igfAE,depth=10,body=Historical 82 (batch=0),prevs=['$8Ov9djlzX3ixwCWaULf6-SJNBuV8eeslfBVvt5b1Uds']", "event_id=$8Ov9djlzX3ixwCWaULf6-SJNBuV8eeslfBVvt5b1Uds,depth=10,body=Historical 81 (batch=0),prevs=['$Xflk7Gara-ISOAblxE3Vh5qbRDxJal8NLWSD6BC3wfY']", "event_id=$Xflk7Gara-ISOAblxE3Vh5qbRDxJal8NLWSD6BC3wfY,depth=10,body=Historical 80 (batch=0),prevs=['$cd4vq5xB0kqHa7HziMmI1U5njSwDr6NOz0qkvCrUeiA']", "event_id=$cd4vq5xB0kqHa7HziMmI1U5njSwDr6NOz0qkvCrUeiA,depth=10,body=Historical 79 (batch=0),prevs=['$xblwBKeHnnXlcK9nusq0mWw712br9ySr3elpt5vqYG8']", "event_id=$xblwBKeHnnXlcK9nusq0mWw712br9ySr3elpt5vqYG8,depth=10,body=Historical 78 (batch=0),prevs=['$cAnav2OUp8QsmQRbPc-PcX6A4rRW9UUsGFEGb1lgSyI']", "event_id=$cAnav2OUp8QsmQRbPc-PcX6A4rRW9UUsGFEGb1lgSyI,depth=10,body=Historical 77 (batch=0),prevs=['$3janYqE-QmrenClP9K2jxr-TM2AvG5H-kcgkM6F9UgA']", "event_id=$3janYqE-QmrenClP9K2jxr-TM2AvG5H-kcgkM6F9UgA,depth=10,body=Historical 76 (batch=0),prevs=['$UqMZVF90ZR0kzwt5q2noHEGCS8UFp_IAN4EbiMYVl4o']", "event_id=$UqMZVF90ZR0kzwt5q2noHEGCS8UFp_IAN4EbiMYVl4o,depth=10,body=Historical 75 (batch=0),prevs=['$D6IL83wLN4nkNwwikin8olaPEZCpzUh7-1PAOdD7HSY']", "event_id=$D6IL83wLN4nkNwwikin8olaPEZCpzUh7-1PAOdD7HSY,depth=10,body=Historical 74 (batch=0),prevs=['$MWVNPcLeQ_INUKlCdoOaySGlGeP5WwDt0myistqnTVk']", "event_id=$MWVNPcLeQ_INUKlCdoOaySGlGeP5WwDt0myistqnTVk,depth=10,body=Historical 73 (batch=0),prevs=['$Co1eNgILBaxBR_Gt7Vz_tIs0LutTc9CPvtYRyElNMj8']", "event_id=$Co1eNgILBaxBR_Gt7Vz_tIs0LutTc9CPvtYRyElNMj8,depth=10,body=Historical 72 (batch=0),prevs=['$ahJyDyEY1Oo8OAkrMZAeYDeTWNzx0B36GsQxcZWchFE']", "event_id=$ahJyDyEY1Oo8OAkrMZAeYDeTWNzx0B36GsQxcZWchFE,depth=10,body=Historical 71 (batch=0),prevs=['$JfGy5KmG-3KFMJFfcHsci6YvhvLq9pWok6GgcZlaXX8']", "event_id=$JfGy5KmG-3KFMJFfcHsci6YvhvLq9pWok6GgcZlaXX8,depth=10,body=Historical 70 (batch=0),prevs=['$VEfiWOEPPyCpN9z305k4IJySpEj5tl4InJb5mhFs7EA']", "event_id=$VEfiWOEPPyCpN9z305k4IJySpEj5tl4InJb5mhFs7EA,depth=10,body=Historical 69 (batch=0),prevs=['$FZswJ09DFvJSzGDFiA3JIdLuzFnG_TfObeEOES1fVTA']", "event_id=$FZswJ09DFvJSzGDFiA3JIdLuzFnG_TfObeEOES1fVTA,depth=10,body=Historical 68 (batch=0),prevs=['$fnzsLJ8duVIJ_S74QIozUaEwOp4G7j7fN5yRV-bzDHQ']", "event_id=$fnzsLJ8duVIJ_S74QIozUaEwOp4G7j7fN5yRV-bzDHQ,depth=10,body=Historical 67 (batch=0),prevs=['$vBWTNsLkJHP9bK0L1Wx3HdND6Rh8HhPmxBtXP7klwvE']", "event_id=$vBWTNsLkJHP9bK0L1Wx3HdND6Rh8HhPmxBtXP7klwvE,depth=10,body=Historical 66 (batch=0),prevs=['$y5PGi9vY9UcTEwTzeE18JB6-bjhf1GVeSEOph4ko-SE']", "event_id=$y5PGi9vY9UcTEwTzeE18JB6-bjhf1GVeSEOph4ko-SE,depth=10,body=Historical 65 (batch=0),prevs=['$7pM8GWYvhzxZ3zGkc76RLtsw7w02IT-sAiV8JkKeQCU']", "event_id=$7pM8GWYvhzxZ3zGkc76RLtsw7w02IT-sAiV8JkKeQCU,depth=10,body=Historical 64 (batch=0),prevs=['$wKaeK8hHhwLb1zHZaRJFE1C-n2kOtWnf076sNeWqaMc']", "event_id=$wKaeK8hHhwLb1zHZaRJFE1C-n2kOtWnf076sNeWqaMc,depth=10,body=Historical 63 (batch=0),prevs=['$8OHC-bni0whbdGfis5cfWL3--JACd0fx-pUBs5LuBXM']", "event_id=$8OHC-bni0whbdGfis5cfWL3--JACd0fx-pUBs5LuBXM,depth=10,body=Historical 62 (batch=0),prevs=['$nZ1WO04o85g1lkLQYlb72bZfx195YwuA-mQ3SsLEsuI']", "event_id=$nZ1WO04o85g1lkLQYlb72bZfx195YwuA-mQ3SsLEsuI,depth=10,body=Historical 61 (batch=0),prevs=['$rdA6ksM5YpvBrm9vHREi-d7x5AJCPZzrAAqxNN5I2ms']", "event_id=$rdA6ksM5YpvBrm9vHREi-d7x5AJCPZzrAAqxNN5I2ms,depth=10,body=Historical 60 (batch=0),prevs=['$v-3prgvdtmX8ySVzXby4Lw6zrncPxKdiLrDnM94y3K4']", "event_id=$v-3prgvdtmX8ySVzXby4Lw6zrncPxKdiLrDnM94y3K4,depth=10,body=Historical 59 (batch=0),prevs=['$ahhYJXd-_gSSZ_mROiEvBI37nYgYCi4WsrvG8sfEQ6g']", "event_id=$ahhYJXd-_gSSZ_mROiEvBI37nYgYCi4WsrvG8sfEQ6g,depth=10,body=Historical 58 (batch=0),prevs=['$JHFT5ugpxzbU6MmFJZ3wXN0hK33EX3-9As0MemxjsX0']", "event_id=$JHFT5ugpxzbU6MmFJZ3wXN0hK33EX3-9As0MemxjsX0,depth=10,body=Historical 57 (batch=0),prevs=['$JYLcmedG2JdmtBQnnXbIEI34QyeaJrMGLAJt_Mi_vuY']", "event_id=$JYLcmedG2JdmtBQnnXbIEI34QyeaJrMGLAJt_Mi_vuY,depth=10,body=Historical 56 (batch=0),prevs=['$nFeaEcPilAYFoXcid_5mz5jDEqIyvTZLYX4erRf2jlM']", "event_id=$nFeaEcPilAYFoXcid_5mz5jDEqIyvTZLYX4erRf2jlM,depth=10,body=Historical 55 (batch=0),prevs=['$ig5cEWTkkNG-ox84UUjG35G6WEHu-qpqAwYKQSOwHxQ']", "event_id=$ig5cEWTkkNG-ox84UUjG35G6WEHu-qpqAwYKQSOwHxQ,depth=10,body=Historical 54 (batch=0),prevs=['$2MDm0b5ztPCyLlAThSBT38Ie3F0ZG3sAE0r3to3yhtc']", "event_id=$2MDm0b5ztPCyLlAThSBT38Ie3F0ZG3sAE0r3to3yhtc,depth=10,body=Historical 53 (batch=0),prevs=['$gZ_e1ws9bh_uKzY4UskhVCO4IFikMi-waYooJbSJAOI']", "event_id=$gZ_e1ws9bh_uKzY4UskhVCO4IFikMi-waYooJbSJAOI,depth=10,body=Historical 52 (batch=0),prevs=['$pOBdCutp83GhxQkP9_tCyYeaN7YvEo0fuf5MiRsu_ew']", "event_id=$pOBdCutp83GhxQkP9_tCyYeaN7YvEo0fuf5MiRsu_ew,depth=10,body=Historical 51 (batch=0),prevs=['$HCC02P42Zfcn-vYaZrhQu1IdHOMVQrsc6XsfIGPrWNE']", "event_id=$HCC02P42Zfcn-vYaZrhQu1IdHOMVQrsc6XsfIGPrWNE,depth=10,body=Historical 50 (batch=0),prevs=['$R4sIu0EhjycD9c7qHb1Sfdw34PqzIFOiTzCsPs_Z-Fk']", "event_id=$R4sIu0EhjycD9c7qHb1Sfdw34PqzIFOiTzCsPs_Z-Fk,depth=10,body=Historical 49 (batch=0),prevs=['$Gdo7IK1ZUK3pHvHvOETByF8J_PLqVP6BDk1qI_3AZ90']", "event_id=$Gdo7IK1ZUK3pHvHvOETByF8J_PLqVP6BDk1qI_3AZ90,depth=10,body=Historical 48 (batch=0),prevs=['$VJ5qpulzoG4vkhkWnbwpgtBMHAMQSmhUB76lrI_2KXc']", "event_id=$VJ5qpulzoG4vkhkWnbwpgtBMHAMQSmhUB76lrI_2KXc,depth=10,body=Historical 47 (batch=0),prevs=['$-h0DqfdRxhEi6RMiaoKEHqzRDqA5Qy8ZskPGAp5rUI0']", "event_id=$-h0DqfdRxhEi6RMiaoKEHqzRDqA5Qy8ZskPGAp5rUI0,depth=10,body=Historical 46 (batch=0),prevs=['$_L-oB9rLJsXOnJkjnUQNDUL1C3jw2-YNbTBDiKG958Q']", "event_id=$_L-oB9rLJsXOnJkjnUQNDUL1C3jw2-YNbTBDiKG958Q,depth=10,body=Historical 45 (batch=0),prevs=['$iQeszxPn42bPGUQknk5TCYIWxVqnGDG-h4Bk1JFH_-4']", "event_id=$iQeszxPn42bPGUQknk5TCYIWxVqnGDG-h4Bk1JFH_-4,depth=10,body=Historical 44 (batch=0),prevs=['$raFQyAel7PT5FUymMwF3SIOXxuMpX7aJyRFDfsyFTpo']", "event_id=$raFQyAel7PT5FUymMwF3SIOXxuMpX7aJyRFDfsyFTpo,depth=10,body=Historical 43 (batch=0),prevs=['$bYfZTSvNiI6nOE7NvnwsaJBlDt8eGifjNWiBoOd-4JQ']", "event_id=$bYfZTSvNiI6nOE7NvnwsaJBlDt8eGifjNWiBoOd-4JQ,depth=10,body=Historical 42 (batch=0),prevs=['$iflmqGrv2ERr58y8QWMW8cAYjIwctoOqc_S8Zpxf2LE']", "event_id=$iflmqGrv2ERr58y8QWMW8cAYjIwctoOqc_S8Zpxf2LE,depth=10,body=Historical 41 (batch=0),prevs=['$84bhYzG__UwK-Rc1T4UGKzQUcLGOTMB9cWikG6Np_hM']", "event_id=$84bhYzG__UwK-Rc1T4UGKzQUcLGOTMB9cWikG6Np_hM,depth=10,body=Historical 40 (batch=0),prevs=['$xqS8HMfkjk_9j2aHTisNv8oeocOq38nxxU8MrepqgMU']", "event_id=$xqS8HMfkjk_9j2aHTisNv8oeocOq38nxxU8MrepqgMU,depth=10,body=Historical 39 (batch=0),prevs=['$EChQxibb9iOk8GqNnZRL3dQ0EfcXJpbl942hiEf1uxo']", "event_id=$EChQxibb9iOk8GqNnZRL3dQ0EfcXJpbl942hiEf1uxo,depth=10,body=Historical 38 (batch=0),prevs=['$bzBMsp8mhPIhbhv8RBjSP8zwvJ6ix2HYoEeourkeRJE']", "event_id=$bzBMsp8mhPIhbhv8RBjSP8zwvJ6ix2HYoEeourkeRJE,depth=10,body=Historical 37 (batch=0),prevs=['$lDNy6ds20jjt077dA_f9nFJrAu6reMkA-a9cZaW6cpI']", "event_id=$lDNy6ds20jjt077dA_f9nFJrAu6reMkA-a9cZaW6cpI,depth=10,body=Historical 36 (batch=0),prevs=['$yMQY-EQTPizmc_vOIsWIcY-Acp7dBNJew27NlfDHduo']", "event_id=$yMQY-EQTPizmc_vOIsWIcY-Acp7dBNJew27NlfDHduo,depth=10,body=Historical 35 (batch=0),prevs=['$Y9NaQWbjDCwqc8TxXbCduEp2FWGNDwUA_ElYWF148T8']", "event_id=$Y9NaQWbjDCwqc8TxXbCduEp2FWGNDwUA_ElYWF148T8,depth=10,body=Historical 34 (batch=0),prevs=['$_if1SSgNjDIT1YefAeoQ0z2aH6pe2jynJUDraBZOCjc']", "event_id=$_if1SSgNjDIT1YefAeoQ0z2aH6pe2jynJUDraBZOCjc,depth=10,body=Historical 33 (batch=0),prevs=['$_JgwcI13PE8e2MuebXSLBIyd_MaIorlEZtF_z_uIVHo']", "event_id=$_JgwcI13PE8e2MuebXSLBIyd_MaIorlEZtF_z_uIVHo,depth=10,body=Historical 32 (batch=0),prevs=['$CRZ4J3otfULJ7Iw1MaAQvIeakECsoZQs1Ehh-xeaC_k']", "event_id=$CRZ4J3otfULJ7Iw1MaAQvIeakECsoZQs1Ehh-xeaC_k,depth=10,body=Historical 31 (batch=0),prevs=['$qaIW_OYBoW7I03J5DnXMEJkRMKtnOOO5ttQjniYzHnI']", "event_id=$qaIW_OYBoW7I03J5DnXMEJkRMKtnOOO5ttQjniYzHnI,depth=10,body=Historical 30 (batch=0),prevs=['$9VNFmDvQZiJo50kQf1qPTAQyCPsdHjx23qC0DT34wY0']", "event_id=$9VNFmDvQZiJo50kQf1qPTAQyCPsdHjx23qC0DT34wY0,depth=10,body=Historical 29 (batch=0),prevs=['$gFbNkb60KtE_SzR9oi16tpkKE9k4hijC2A6dt66Al4k']", "event_id=$gFbNkb60KtE_SzR9oi16tpkKE9k4hijC2A6dt66Al4k,depth=10,body=Historical 28 (batch=0),prevs=['$Mq-dMSTxKEwHS3gPvYwU-Obqs3-Tp3SWjRPUq2VDvJg']", "event_id=$Mq-dMSTxKEwHS3gPvYwU-Obqs3-Tp3SWjRPUq2VDvJg,depth=10,body=Historical 27 (batch=0),prevs=['$5EC7XJFkyMLzhujTLI_m9UXNEnn-4NKZj-DCouBgyt8']", "event_id=$5EC7XJFkyMLzhujTLI_m9UXNEnn-4NKZj-DCouBgyt8,depth=10,body=Historical 26 (batch=0),prevs=['$UxlPuejLS1jHkUNCMRe2vNmc2tcuut-G2N8q6RFTfVQ']", "event_id=$UxlPuejLS1jHkUNCMRe2vNmc2tcuut-G2N8q6RFTfVQ,depth=10,body=Historical 25 (batch=0),prevs=['$BNqTjoMam0rgznru1lqk1U8cCqlJRA4GwORVLBeFYTU']", "event_id=$BNqTjoMam0rgznru1lqk1U8cCqlJRA4GwORVLBeFYTU,depth=10,body=Historical 24 (batch=0),prevs=['$7SdwqRy6BHBnNb_sQtj2Rph9era4F68ZM6mGPzjJthA']", "event_id=$7SdwqRy6BHBnNb_sQtj2Rph9era4F68ZM6mGPzjJthA,depth=10,body=Historical 23 (batch=0),prevs=['$HiveBkhuxr5NIgx9wGhfrmttnGOi0VpeesmqtjpkqE4']", "event_id=$HiveBkhuxr5NIgx9wGhfrmttnGOi0VpeesmqtjpkqE4,depth=10,body=Historical 22 (batch=0),prevs=['$m7iJp9s_Stodzk1I1Lf8egDLbIdyvgJPjAxomFguHb4']", "event_id=$m7iJp9s_Stodzk1I1Lf8egDLbIdyvgJPjAxomFguHb4,depth=10,body=Historical 21 (batch=0),prevs=['$KRSKtmQJHl80Hn7UR3emVczq4-QgKycVop3lkSeppIs']", "event_id=$KRSKtmQJHl80Hn7UR3emVczq4-QgKycVop3lkSeppIs,depth=10,body=Historical 20 (batch=0),prevs=['$ZCZKQrCeoKEC6oKqwjhK_8EQqwq75b7fEop-GgYCTiU']", "event_id=$ZCZKQrCeoKEC6oKqwjhK_8EQqwq75b7fEop-GgYCTiU,depth=10,body=Historical 19 (batch=0),prevs=['$9pZEcOtzeGWbnmRS_zAQAhdCIE8DZ6_cJAR0tm2AG1k']", "event_id=$9pZEcOtzeGWbnmRS_zAQAhdCIE8DZ6_cJAR0tm2AG1k,depth=10,body=Historical 18 (batch=0),prevs=['$FFXzuSEKR53_PpJ6EBGp8FEBi5Aig2MIq-F1opufK10']", "event_id=$FFXzuSEKR53_PpJ6EBGp8FEBi5Aig2MIq-F1opufK10,depth=10,body=Historical 17 (batch=0),prevs=['$hr0MEvZerIeshwxLzAGbqMXQGOXU_u9H1wUPtzqa6gY']", "event_id=$hr0MEvZerIeshwxLzAGbqMXQGOXU_u9H1wUPtzqa6gY,depth=10,body=Historical 16 (batch=0),prevs=['$hjP2ryNx-yqznW3EKrBZE1DaGSbvK8lB_3cWUDDGTVk']", "event_id=$hjP2ryNx-yqznW3EKrBZE1DaGSbvK8lB_3cWUDDGTVk,depth=10,body=Historical 15 (batch=0),prevs=['$E24xBJItDTqteFc11s2kk2JuvpQjZ2fd-vEWJ7QS784']", "event_id=$E24xBJItDTqteFc11s2kk2JuvpQjZ2fd-vEWJ7QS784,depth=10,body=Historical 14 (batch=0),prevs=['$eOuhBaOKitgobbzyU5XVieMLzgoCITj9eG-ewSMDB9I']", "event_id=$eOuhBaOKitgobbzyU5XVieMLzgoCITj9eG-ewSMDB9I,depth=10,body=Historical 13 (batch=0),prevs=['$4F4Xhu7QKpLM-n_aFmCLrKRP6zw7lrDLAYFTPKn8jYo']", "event_id=$4F4Xhu7QKpLM-n_aFmCLrKRP6zw7lrDLAYFTPKn8jYo,depth=10,body=Historical 12 (batch=0),prevs=['$e0wqSykOpA8Y0yipT9643SVVQ3DMOnW11yxvqAj9JrE']", "event_id=$e0wqSykOpA8Y0yipT9643SVVQ3DMOnW11yxvqAj9JrE,depth=10,body=Historical 11 (batch=0),prevs=['$u5gxt1eo59odS1XqVS-kV2GcaitrzorppcqVj3scevI']", "event_id=$u5gxt1eo59odS1XqVS-kV2GcaitrzorppcqVj3scevI,depth=10,body=Historical 10 (batch=0),prevs=['$Z1E79_z_FcAnw4G9NOvx3WhyL2ACyYPgy6zLhomKVks']", "event_id=$Z1E79_z_FcAnw4G9NOvx3WhyL2ACyYPgy6zLhomKVks,depth=10,body=Historical 9 (batch=0),prevs=['$USyjgdL7fwX4BvmblVs2cicGO3WDRB-ox56298yZmAA']", "event_id=$USyjgdL7fwX4BvmblVs2cicGO3WDRB-ox56298yZmAA,depth=10,body=Historical 8 (batch=0),prevs=['$tggu2uQEL2BWVJiwqsqwCJUca3SEFeylCAbPw5DrIdo']", "event_id=$tggu2uQEL2BWVJiwqsqwCJUca3SEFeylCAbPw5DrIdo,depth=10,body=Historical 7 (batch=0),prevs=['$M0iINx5TifxVTJ6pz-lio7xIeSla1RIV_N9WIK5sf4c']", "event_id=$M0iINx5TifxVTJ6pz-lio7xIeSla1RIV_N9WIK5sf4c,depth=10,body=Historical 6 (batch=0),prevs=['$hEs6MN2xyjXqHRcWV53_23NeNm--al9b_5BnMYYDPMY']", "event_id=$hEs6MN2xyjXqHRcWV53_23NeNm--al9b_5BnMYYDPMY,depth=10,body=Historical 5 (batch=0),prevs=['$L0BJu5ZOx2l8AMivXMBCqrvymVu_nCQM4ZO23uv5tBU']", "event_id=$L0BJu5ZOx2l8AMivXMBCqrvymVu_nCQM4ZO23uv5tBU,depth=10,body=Historical 4 (batch=0),prevs=['$V5BZU5LpCbW6XvPLGls0QGRbo98uuf1KpMZ0fg8k_-o']"] new implementation backfill events(100)=[ "event_id=$V5BZU5LpCbW6XvPLGls0QGRbo98uuf1KpMZ0fg8k_-o,depth=10,body=Historical 3 (batch=0),prevs=['$lF2XlCU-QmhC0YF8Wn8Q-EfQXqDxF_usgPIZ28rNhII']", "event_id=$lF2XlCU-QmhC0YF8Wn8Q-EfQXqDxF_usgPIZ28rNhII,depth=10,body=Historical 2 (batch=0),prevs=['$Ghnwsm4CqaWopAMGjKm8piFuTEqSaWgl1X7esubZOco']", "event_id=$Ghnwsm4CqaWopAMGjKm8piFuTEqSaWgl1X7esubZOco,depth=10,body=Historical 1 (batch=0),prevs=['$WRV7zPTr4KknEcYUvPlfxMdxcmlrQSsXAfTTlSBpjko']", "event_id=$WRV7zPTr4KknEcYUvPlfxMdxcmlrQSsXAfTTlSBpjko,depth=10,body=Historical 0 (batch=0),prevs=['$30VmrbC0AauqzWQ9F2g_KTPUdJhH_r79olOmfZYe0Pg']", "event_id=$30VmrbC0AauqzWQ9F2g_KTPUdJhH_r79olOmfZYe0Pg,depth=10,body=org.matrix.msc2716.insertion,prevs=['$an2lLzRXApWnCuRNrb1jTfMY9lWbz909AxyIRWy5-C0']", "event_id=$CmUx1RcXqn9QwVSKFKGmwVXwEFE9sHpnac5xvCgm-Co,depth=10,body=org.matrix.msc2716.batch,prevs=['$Kt2_lR2GlI6YoDY7q1ttaoIUv_s8WsjKgce6PT8utIA']", "event_id=$Kt2_lR2GlI6YoDY7q1ttaoIUv_s8WsjKgce6PT8utIA,depth=10,body=Historical 99 (batch=1),prevs=['$eaZav6ZipscONXQMtzaWQrTmiw9yo6_ifeaiI2VTJJM']", "event_id=$eaZav6ZipscONXQMtzaWQrTmiw9yo6_ifeaiI2VTJJM,depth=10,body=Historical 98 (batch=1),prevs=['$sSIhVeKMkHlwvQmCdUxouZyc0A7oBK_IcTYYPhQHniY']", "event_id=$sSIhVeKMkHlwvQmCdUxouZyc0A7oBK_IcTYYPhQHniY,depth=10,body=Historical 97 (batch=1),prevs=['$tQ6z1P58SRR3qqS6UnjWn85zP6CoU882ehdCkNcwhA8']", "event_id=$tQ6z1P58SRR3qqS6UnjWn85zP6CoU882ehdCkNcwhA8,depth=10,body=Historical 96 (batch=1),prevs=['$mh2o2cBnVjU5uQr0UAMPAWPpGsRzIKKpzc54Q2Z9JGY']", "event_id=$mh2o2cBnVjU5uQr0UAMPAWPpGsRzIKKpzc54Q2Z9JGY,depth=10,body=Historical 95 (batch=1),prevs=['$O_SPFvep-jjnHsW2484GzRYIDZkJhZYVgmCstDR3Qn0']", "event_id=$O_SPFvep-jjnHsW2484GzRYIDZkJhZYVgmCstDR3Qn0,depth=10,body=Historical 94 (batch=1),prevs=['$MkZU6zPnF5L4Mb9Y0AkOlWSG6S5yKhyVsFPm_WPc7EU']", "event_id=$MkZU6zPnF5L4Mb9Y0AkOlWSG6S5yKhyVsFPm_WPc7EU,depth=10,body=Historical 93 (batch=1),prevs=['$lts-3h-4-93qb12QQ8PwkvSu4bSJVKulvj_N2UPRhvs']", "event_id=$lts-3h-4-93qb12QQ8PwkvSu4bSJVKulvj_N2UPRhvs,depth=10,body=Historical 92 (batch=1),prevs=['$abLL8g1D70BE9EqtgSGoVjtbVK6QKG0KZhgBeJmEAuw']", "event_id=$abLL8g1D70BE9EqtgSGoVjtbVK6QKG0KZhgBeJmEAuw,depth=10,body=Historical 91 (batch=1),prevs=['$H2-ZSHXeJII1Jc3XvEjFfmjGUZgpsmlay7YuKKlF7JQ']", "event_id=$H2-ZSHXeJII1Jc3XvEjFfmjGUZgpsmlay7YuKKlF7JQ,depth=10,body=Historical 90 (batch=1),prevs=['$VsrkCzgyKUyPT4XpS0pa-K-CwRJ7MeNu9ictxOHPZo4']", "event_id=$VsrkCzgyKUyPT4XpS0pa-K-CwRJ7MeNu9ictxOHPZo4,depth=10,body=Historical 89 (batch=1),prevs=['$NSVKzqK5ucXWwWq7j4yfADj5m-8GqWFCWeqo2hvUDWg']", "event_id=$NSVKzqK5ucXWwWq7j4yfADj5m-8GqWFCWeqo2hvUDWg,depth=10,body=Historical 88 (batch=1),prevs=['$y5lPUOYBw0g12eIgDUdTn5QnmXD2QeFks61Ivpha1XA']", "event_id=$y5lPUOYBw0g12eIgDUdTn5QnmXD2QeFks61Ivpha1XA,depth=10,body=Historical 87 (batch=1),prevs=['$BdehWtfVHIxjlD3-pWh-6kcb1w2oPhAhgCWxNUkKAD8']", "event_id=$BdehWtfVHIxjlD3-pWh-6kcb1w2oPhAhgCWxNUkKAD8,depth=10,body=Historical 86 (batch=1),prevs=['$B0afYanjEywl1q1fYru7nL2RmjjZpF0gjTVQ7n9fY6k']", "event_id=$B0afYanjEywl1q1fYru7nL2RmjjZpF0gjTVQ7n9fY6k,depth=10,body=Historical 85 (batch=1),prevs=['$F2cOk9oSDKNSlrMoJioWtLi8AdhMkS-QpVwZw2qhWgM']", "event_id=$F2cOk9oSDKNSlrMoJioWtLi8AdhMkS-QpVwZw2qhWgM,depth=10,body=Historical 84 (batch=1),prevs=['$QGCvZWFsgW4KTskjaSqxjTenOa3TbbpBDWVyXESXDGI']", "event_id=$QGCvZWFsgW4KTskjaSqxjTenOa3TbbpBDWVyXESXDGI,depth=10,body=Historical 83 (batch=1),prevs=['$5evMhA_hLx02z7ydMm_cRfSUsC9OX0MJLayMydLrzN0']", "event_id=$5evMhA_hLx02z7ydMm_cRfSUsC9OX0MJLayMydLrzN0,depth=10,body=Historical 82 (batch=1),prevs=['$4jmsKkx-6NRVMCflN_s6sOfC053Zw1wOfCCM0bWlIX8']", "event_id=$4jmsKkx-6NRVMCflN_s6sOfC053Zw1wOfCCM0bWlIX8,depth=10,body=Historical 81 (batch=1),prevs=['$sTtp0fKA9BG4GDkJ8CUBQVNqpDDcf_a-PkGhJkGqzrw']", "event_id=$sTtp0fKA9BG4GDkJ8CUBQVNqpDDcf_a-PkGhJkGqzrw,depth=10,body=Historical 80 (batch=1),prevs=['$96go7TqZzPq75eOOLiDY0FkylHrmAGly0MTwX4_McMs']", "event_id=$96go7TqZzPq75eOOLiDY0FkylHrmAGly0MTwX4_McMs,depth=10,body=Historical 79 (batch=1),prevs=['$kUiJWkPHn1nPrUDzZJvWhTz1T8UBVELMlEdJGOEB_Qo']", "event_id=$kUiJWkPHn1nPrUDzZJvWhTz1T8UBVELMlEdJGOEB_Qo,depth=10,body=Historical 78 (batch=1),prevs=['$5GCJZEmchNvy0ikP7K7ZlvFb9FS6aVKRedx-08ZAF3s']", "event_id=$5GCJZEmchNvy0ikP7K7ZlvFb9FS6aVKRedx-08ZAF3s,depth=10,body=Historical 77 (batch=1),prevs=['$A3r2LkTKaHxBQh1Ailt4dCtaESyo53Ss1u1s3QiE_k0']", "event_id=$A3r2LkTKaHxBQh1Ailt4dCtaESyo53Ss1u1s3QiE_k0,depth=10,body=Historical 76 (batch=1),prevs=['$R7RRyXo6Msc3-368vZpiG84DfqOAiKLiSbV0ECGv6Oo']", "event_id=$R7RRyXo6Msc3-368vZpiG84DfqOAiKLiSbV0ECGv6Oo,depth=10,body=Historical 75 (batch=1),prevs=['$jmkYlZN6-oiQOi-bJfs0whVS7kKeJJUN6YJQfubbaEU']", "event_id=$jmkYlZN6-oiQOi-bJfs0whVS7kKeJJUN6YJQfubbaEU,depth=10,body=Historical 74 (batch=1),prevs=['$IuGVv6TZj5ZoOP4HHqZsKPWQ9HDTHS4QV7oFHk4kl7o']", "event_id=$IuGVv6TZj5ZoOP4HHqZsKPWQ9HDTHS4QV7oFHk4kl7o,depth=10,body=Historical 73 (batch=1),prevs=['$sNds_pVGUFf41AJwHmA1Sy7MUX1N9MtHGfw9ye6EItM']", "event_id=$sNds_pVGUFf41AJwHmA1Sy7MUX1N9MtHGfw9ye6EItM,depth=10,body=Historical 72 (batch=1),prevs=['$u0R7zUNs4BpneteeM7PWgGZE4XzQJuAwUtFSQGwTNI0']", "event_id=$u0R7zUNs4BpneteeM7PWgGZE4XzQJuAwUtFSQGwTNI0,depth=10,body=Historical 71 (batch=1),prevs=['$VthGtqcIDd04US7pMNTr0gSmR7PIOPLmDOljx1qNleQ']", "event_id=$VthGtqcIDd04US7pMNTr0gSmR7PIOPLmDOljx1qNleQ,depth=10,body=Historical 70 (batch=1),prevs=['$qYlzglfNCN1Br-4UkajF4c-SKYI0By5gExUxczeDsg4']", "event_id=$qYlzglfNCN1Br-4UkajF4c-SKYI0By5gExUxczeDsg4,depth=10,body=Historical 69 (batch=1),prevs=['$BsxtZqcTIs1aqFRsXtapKXYF6w2GiUEswpl6kH6S_4o']", "event_id=$BsxtZqcTIs1aqFRsXtapKXYF6w2GiUEswpl6kH6S_4o,depth=10,body=Historical 68 (batch=1),prevs=['$PmOzgtCDJCyGowxB8gWcYbfA2KUX0MCJ3xTE48FhAyY']", "event_id=$PmOzgtCDJCyGowxB8gWcYbfA2KUX0MCJ3xTE48FhAyY,depth=10,body=Historical 67 (batch=1),prevs=['$vZq13XMjZRJdFefxBg0L1U9shZq1TbArtR43YkQG7i8']", "event_id=$vZq13XMjZRJdFefxBg0L1U9shZq1TbArtR43YkQG7i8,depth=10,body=Historical 66 (batch=1),prevs=['$526Rm38ribM0XL3HltTbjVCUa3cDUdBh0V-9AMPtsYY']", "event_id=$526Rm38ribM0XL3HltTbjVCUa3cDUdBh0V-9AMPtsYY,depth=10,body=Historical 65 (batch=1),prevs=['$-sPB5z3uyN5tr4KezPKt6xu_yoaqgSDKhkzTJ0Um7yI']", "event_id=$-sPB5z3uyN5tr4KezPKt6xu_yoaqgSDKhkzTJ0Um7yI,depth=10,body=Historical 64 (batch=1),prevs=['$y5mX5SlScEAyPbUx1UA2-_ml0abbh0nGFYgJzF0XzZ0']", "event_id=$y5mX5SlScEAyPbUx1UA2-_ml0abbh0nGFYgJzF0XzZ0,depth=10,body=Historical 63 (batch=1),prevs=['$erT1ogi6tp9tF33_N3ZBLpIMBXOunszhADMYjs8U5AE']", "event_id=$erT1ogi6tp9tF33_N3ZBLpIMBXOunszhADMYjs8U5AE,depth=10,body=Historical 62 (batch=1),prevs=['$d9qTo4Ray7GJWJPm-CHkfZmZC0WmD8-K6Z1Riyot3fw']", "event_id=$d9qTo4Ray7GJWJPm-CHkfZmZC0WmD8-K6Z1Riyot3fw,depth=10,body=Historical 61 (batch=1),prevs=['$TVgdscs2fWtvx0ak6V0ljabU_4cc6UH18Y1r3OvDLYc']", "event_id=$TVgdscs2fWtvx0ak6V0ljabU_4cc6UH18Y1r3OvDLYc,depth=10,body=Historical 60 (batch=1),prevs=['$eMi_3TfsgYw2Fs99AJpr_9KAapZq3uwtv2fUDQ2weTc']", "event_id=$eMi_3TfsgYw2Fs99AJpr_9KAapZq3uwtv2fUDQ2weTc,depth=10,body=Historical 59 (batch=1),prevs=['$UBL_4l45DMvVTE12gyRaSDMw_fi_biDx1dL_gWz_v-g']", "event_id=$UBL_4l45DMvVTE12gyRaSDMw_fi_biDx1dL_gWz_v-g,depth=10,body=Historical 58 (batch=1),prevs=['$A6-UMuau--ffdnOA0NqWQK9uO0NBCB_rwVCvu_hSAOw']", "event_id=$A6-UMuau--ffdnOA0NqWQK9uO0NBCB_rwVCvu_hSAOw,depth=10,body=Historical 57 (batch=1),prevs=['$Eia3O1ptxGtKhINTEVX_FxBU3TjL0MBp1XdeFCStcCg']", "event_id=$Eia3O1ptxGtKhINTEVX_FxBU3TjL0MBp1XdeFCStcCg,depth=10,body=Historical 56 (batch=1),prevs=['$Pb5OyRAHNrXDyPt8F1dTWVGi-Zvqkf-HqzCwcu0salY']", "event_id=$Pb5OyRAHNrXDyPt8F1dTWVGi-Zvqkf-HqzCwcu0salY,depth=10,body=Historical 55 (batch=1),prevs=['$c1KPN_U0VF8G3jGnD6UpQob_h9X_KQfXv4lciJUb4PQ']", "event_id=$c1KPN_U0VF8G3jGnD6UpQob_h9X_KQfXv4lciJUb4PQ,depth=10,body=Historical 54 (batch=1),prevs=['$fwWir53iSGaGEMRt-mQe6hYmkwIQGjDojFO08zlzgX4']", "event_id=$fwWir53iSGaGEMRt-mQe6hYmkwIQGjDojFO08zlzgX4,depth=10,body=Historical 53 (batch=1),prevs=['$TmacWSD6J8y8vERZfA-IdRpDeMS_4A1HFNUBg_W5MaA']", "event_id=$TmacWSD6J8y8vERZfA-IdRpDeMS_4A1HFNUBg_W5MaA,depth=10,body=Historical 52 (batch=1),prevs=['$_xBvb_mIUJgyvA-rgBppyjXVZ3SXn03isQ0axMOtThc']", "event_id=$_xBvb_mIUJgyvA-rgBppyjXVZ3SXn03isQ0axMOtThc,depth=10,body=Historical 51 (batch=1),prevs=['$AYD-cd4lrJLbW35mCK8dKZEZI705gOsA1iiyW3uqDWk']", "event_id=$AYD-cd4lrJLbW35mCK8dKZEZI705gOsA1iiyW3uqDWk,depth=10,body=Historical 50 (batch=1),prevs=['$KK1BXAZ3E-U7qKWhO6Y6DfXedQ04BhwuA_bNI5Nee8g']", "event_id=$KK1BXAZ3E-U7qKWhO6Y6DfXedQ04BhwuA_bNI5Nee8g,depth=10,body=Historical 49 (batch=1),prevs=['$_oLPTYdg92q_l3c6vxV4tO9ioR9nPrk1lyb2qiLIPOk']", "event_id=$_oLPTYdg92q_l3c6vxV4tO9ioR9nPrk1lyb2qiLIPOk,depth=10,body=Historical 48 (batch=1),prevs=['$RZ0Mzvftn99rU6XFl32jKHs0R6YYaMPy_HJy41tK4ig']", "event_id=$RZ0Mzvftn99rU6XFl32jKHs0R6YYaMPy_HJy41tK4ig,depth=10,body=Historical 47 (batch=1),prevs=['$mloZAG1tKqfjIBQhJk8Mn7M5ER84cJ7bC0TW3rOsKpU']", "event_id=$mloZAG1tKqfjIBQhJk8Mn7M5ER84cJ7bC0TW3rOsKpU,depth=10,body=Historical 46 (batch=1),prevs=['$RY6YWeTkTc8aQP6ZkPq_vL0qs1kg53QeDcKmrLi0ASw']", "event_id=$RY6YWeTkTc8aQP6ZkPq_vL0qs1kg53QeDcKmrLi0ASw,depth=10,body=Historical 45 (batch=1),prevs=['$5A2e6qJj9n9ejvUW6TcVkm6-cuHghnRKQDW3T3a4eqI']", "event_id=$5A2e6qJj9n9ejvUW6TcVkm6-cuHghnRKQDW3T3a4eqI,depth=10,body=Historical 44 (batch=1),prevs=['$KDMWnrkM1ezjDXlSrXjvhZS-RRuJc6_hrX-Msh0gP7o']", "event_id=$KDMWnrkM1ezjDXlSrXjvhZS-RRuJc6_hrX-Msh0gP7o,depth=10,body=Historical 43 (batch=1),prevs=['$9Q6SSr_YMaQ6oc9w7lYs-bRw0cNESLPTE45Fg2_Zzv4']", "event_id=$9Q6SSr_YMaQ6oc9w7lYs-bRw0cNESLPTE45Fg2_Zzv4,depth=10,body=Historical 42 (batch=1),prevs=['$b2QYayqOOaCxQqs655Tve2OKeXrT9CJeVIchVUn00hI']", "event_id=$b2QYayqOOaCxQqs655Tve2OKeXrT9CJeVIchVUn00hI,depth=10,body=Historical 41 (batch=1),prevs=['$xQQBp4Dwm-3lxCWkJsck9bXuQoCKsdPIortpc79nTtw']", "event_id=$xQQBp4Dwm-3lxCWkJsck9bXuQoCKsdPIortpc79nTtw,depth=10,body=Historical 40 (batch=1),prevs=['$RgTwPhV6RzdVBmuqoYBNblHPZ-qpYuTCshDBrfpwvds']", "event_id=$RgTwPhV6RzdVBmuqoYBNblHPZ-qpYuTCshDBrfpwvds,depth=10,body=Historical 39 (batch=1),prevs=['$RWyv_RX_EoNzCDnHSCfx6fDnJrhWl46iWYomZSS2GsQ']", "event_id=$RWyv_RX_EoNzCDnHSCfx6fDnJrhWl46iWYomZSS2GsQ,depth=10,body=Historical 38 (batch=1),prevs=['$Sj78rQcr9DuCnhtcYuZjWS6o1ZiFDd3ZQxDtJKxjeNg']", "event_id=$Sj78rQcr9DuCnhtcYuZjWS6o1ZiFDd3ZQxDtJKxjeNg,depth=10,body=Historical 37 (batch=1),prevs=['$KePkqP3sJgnC8wVMyRFd0v7Fsu-8LiLTZbhx0JsWfnY']", "event_id=$KePkqP3sJgnC8wVMyRFd0v7Fsu-8LiLTZbhx0JsWfnY,depth=10,body=Historical 36 (batch=1),prevs=['$QdyPRKCNju-Rzm1u530j3N7AEcRFA5mAO3sVbBZunkQ']", "event_id=$QdyPRKCNju-Rzm1u530j3N7AEcRFA5mAO3sVbBZunkQ,depth=10,body=Historical 35 (batch=1),prevs=['$dd3HF7wo7YRhA6e0j5I9eGZtaH1rMK7dAriiNTQd7-Y']", "event_id=$dd3HF7wo7YRhA6e0j5I9eGZtaH1rMK7dAriiNTQd7-Y,depth=10,body=Historical 34 (batch=1),prevs=['$l6q9M7E_yWQID_v4YZMYZfNevVqhK0Pm6qIDABcx7ag']", "event_id=$l6q9M7E_yWQID_v4YZMYZfNevVqhK0Pm6qIDABcx7ag,depth=10,body=Historical 33 (batch=1),prevs=['$GAdAhlUMns9H8jeNH6yrfNp1c4_-lhQKyBnmAkQNZ5A']", "event_id=$GAdAhlUMns9H8jeNH6yrfNp1c4_-lhQKyBnmAkQNZ5A,depth=10,body=Historical 32 (batch=1),prevs=['$6liLB-IPWtTynCd27je2vCoHfDK_S3pwlRe5GoUsLY8']", "event_id=$6liLB-IPWtTynCd27je2vCoHfDK_S3pwlRe5GoUsLY8,depth=10,body=Historical 31 (batch=1),prevs=['$9C_ywoPrhy68x4-MurcodrKje7FOw5GuG9yMJ6W2ObM']", "event_id=$9C_ywoPrhy68x4-MurcodrKje7FOw5GuG9yMJ6W2ObM,depth=10,body=Historical 30 (batch=1),prevs=['$QyvDk3PQ4dEDxHb0ihFqAwWpqH-NQ86GV_w5njGNOSY']", "event_id=$QyvDk3PQ4dEDxHb0ihFqAwWpqH-NQ86GV_w5njGNOSY,depth=10,body=Historical 29 (batch=1),prevs=['$c2ee8wW8tAvev6UB1yOZMA56j-BFcxZJQqPbXxiqmeg']", "event_id=$c2ee8wW8tAvev6UB1yOZMA56j-BFcxZJQqPbXxiqmeg,depth=10,body=Historical 28 (batch=1),prevs=['$psirWB7VB7WUVg7IjiG1XX2WMOAiN_FFcdo8crQ3KF0']", "event_id=$psirWB7VB7WUVg7IjiG1XX2WMOAiN_FFcdo8crQ3KF0,depth=10,body=Historical 27 (batch=1),prevs=['$AMyblWA-Jny9Nobd-ghaFit6ZDG6OOg3rTxwEIn6zkQ']", "event_id=$AMyblWA-Jny9Nobd-ghaFit6ZDG6OOg3rTxwEIn6zkQ,depth=10,body=Historical 26 (batch=1),prevs=['$EtDL8yGxA03ZO5pTeztYOQvHxUaMCCz9Nj7oGgM0CL8']", "event_id=$EtDL8yGxA03ZO5pTeztYOQvHxUaMCCz9Nj7oGgM0CL8,depth=10,body=Historical 25 (batch=1),prevs=['$zmqDOTVLDS_khXDmPiPvJ7745wI_yuEAsxS_fyDroIo']", "event_id=$zmqDOTVLDS_khXDmPiPvJ7745wI_yuEAsxS_fyDroIo,depth=10,body=Historical 24 (batch=1),prevs=['$lMcNfcLZKCtmrGISzGo3M_lxJapImAmdky9Lezij6TI']", "event_id=$lMcNfcLZKCtmrGISzGo3M_lxJapImAmdky9Lezij6TI,depth=10,body=Historical 23 (batch=1),prevs=['$i4vpa73GXD7wZB2r2t-8nV--pGq_GdKCotM2-wpVJ10']", "event_id=$i4vpa73GXD7wZB2r2t-8nV--pGq_GdKCotM2-wpVJ10,depth=10,body=Historical 22 (batch=1),prevs=['$9BA_P9uGh-cLjLau5nToTmpsnxdaPkT2RjMf9ZYglNI']", "event_id=$9BA_P9uGh-cLjLau5nToTmpsnxdaPkT2RjMf9ZYglNI,depth=10,body=Historical 21 (batch=1),prevs=['$ySO1DPJTl_IiFwv4d6jbNy43a1go9rD_86A2EPIGVZE']", "event_id=$ySO1DPJTl_IiFwv4d6jbNy43a1go9rD_86A2EPIGVZE,depth=10,body=Historical 20 (batch=1),prevs=['$XOyDGiQEhlEoT8UGl4HIod5Zb29R3Op2BFIaVeiCRyY']", "event_id=$XOyDGiQEhlEoT8UGl4HIod5Zb29R3Op2BFIaVeiCRyY,depth=10,body=Historical 19 (batch=1),prevs=['$XDqJHr3kmi2Z9hpNX2Vz7o524cLEMkjKwk-_V2A3U1U']", "event_id=$XDqJHr3kmi2Z9hpNX2Vz7o524cLEMkjKwk-_V2A3U1U,depth=10,body=Historical 18 (batch=1),prevs=['$hlXaBZDCY4XSg-qjuzkAWhYqoBpdX8ehIFZd5M_9rRM']", "event_id=$hlXaBZDCY4XSg-qjuzkAWhYqoBpdX8ehIFZd5M_9rRM,depth=10,body=Historical 17 (batch=1),prevs=['$LMLomCl3U7zYXi4sIHHDnfznJkns0-DLRDGMbhJ1AAo']", "event_id=$LMLomCl3U7zYXi4sIHHDnfznJkns0-DLRDGMbhJ1AAo,depth=10,body=Historical 16 (batch=1),prevs=['$d0pjrKi7Ws1QM-XyB1_d9Yrg0AGY8MD6JLJqrnaaoJM']", "event_id=$d0pjrKi7Ws1QM-XyB1_d9Yrg0AGY8MD6JLJqrnaaoJM,depth=10,body=Historical 15 (batch=1),prevs=['$hx2dBPyOh6KnEzdpw7CnvoSOkiYTKJp82MXaWA_77N0']", "event_id=$hx2dBPyOh6KnEzdpw7CnvoSOkiYTKJp82MXaWA_77N0,depth=10,body=Historical 14 (batch=1),prevs=['$-y0R8tPCI4n4L0t2j4Ua3QIfau7n4kqPXgnQjb4e1qk']", "event_id=$-y0R8tPCI4n4L0t2j4Ua3QIfau7n4kqPXgnQjb4e1qk,depth=10,body=Historical 13 (batch=1),prevs=['$Af85_d8_UkHp8r8KzOnjCCI0whyMQB7U3G5u8NlMk4I']", "event_id=$Af85_d8_UkHp8r8KzOnjCCI0whyMQB7U3G5u8NlMk4I,depth=10,body=Historical 12 (batch=1),prevs=['$1nQJC9ULuCVGYWU_umE58wq5wHmmFx2GdgF6nmQt5VU']", "event_id=$1nQJC9ULuCVGYWU_umE58wq5wHmmFx2GdgF6nmQt5VU,depth=10,body=Historical 11 (batch=1),prevs=['$SdAp6h2pjv7aIxDSZJxwQbolYrqzUGf1gHOeBnqAULU']", "event_id=$SdAp6h2pjv7aIxDSZJxwQbolYrqzUGf1gHOeBnqAULU,depth=10,body=Historical 10 (batch=1),prevs=['$YDMkSKMoYDtwMd-nKvgAIk38uzc1UejIyBvXNoJ_RFM']", "event_id=$YDMkSKMoYDtwMd-nKvgAIk38uzc1UejIyBvXNoJ_RFM,depth=10,body=Historical 9 (batch=1),prevs=['$a8nvUo95cOf-cO4hcgfO1sT0kyWKq8XPUDmJe0DbxNE']", "event_id=$a8nvUo95cOf-cO4hcgfO1sT0kyWKq8XPUDmJe0DbxNE,depth=10,body=Historical 8 (batch=1),prevs=['$LunwOlIdMiDjrcQazQ6I3n0T8aLfmSA24fIEeth1hHs']", "event_id=$LunwOlIdMiDjrcQazQ6I3n0T8aLfmSA24fIEeth1hHs,depth=10,body=Historical 7 (batch=1),prevs=['$UOvHOp4SKtZqSDWZvD0s3VGdU23-6YqiWy_742cqY-8']", "event_id=$UOvHOp4SKtZqSDWZvD0s3VGdU23-6YqiWy_742cqY-8,depth=10,body=Historical 6 (batch=1),prevs=['$W-c9syEfTTufHnNBpuMUIk621ecs9W6bTeLW0OgWFfY']\n"] new implementation backfill events(17)=[ "event_id=$W-c9syEfTTufHnNBpuMUIk621ecs9W6bTeLW0OgWFfY,depth=10,body=Historical 5 (batch=1),prevs=['$Gq8-foARgSoEzcEwXhOsiey2pefpCX3vjnCLtsIdvU0']", "event_id=$Gq8-foARgSoEzcEwXhOsiey2pefpCX3vjnCLtsIdvU0,depth=10,body=Historical 4 (batch=1),prevs=['$toIu3hsoBDXkVJs91nUG0hodf3nuFkbidY7ztVPrtCM']", "event_id=$toIu3hsoBDXkVJs91nUG0hodf3nuFkbidY7ztVPrtCM,depth=10,body=Historical 3 (batch=1),prevs=['$SRcKQ-ggRujFzHbJC44rWvslb4YBKgieWahZakx3zMw']", "event_id=$SRcKQ-ggRujFzHbJC44rWvslb4YBKgieWahZakx3zMw,depth=10,body=Historical 2 (batch=1),prevs=['$Pgqvjn_rkQb6eEdQDeKclO-l_ODlnplF7S-rT1i_-64']", "event_id=$Pgqvjn_rkQb6eEdQDeKclO-l_ODlnplF7S-rT1i_-64,depth=10,body=Historical 1 (batch=1),prevs=['$jEW2Smmgh5tIh1QMtEqmluy-o3FrcVhNAH5e3R4jMoo']", "event_id=$jEW2Smmgh5tIh1QMtEqmluy-o3FrcVhNAH5e3R4jMoo,depth=10,body=Historical 0 (batch=1),prevs=['$1bbHCIrQLToHvum-PGxpIKhCmZ5MYLusAy_gJm7IWTw']", "event_id=$1bbHCIrQLToHvum-PGxpIKhCmZ5MYLusAy_gJm7IWTw,depth=10,body=org.matrix.msc2716.insertion,prevs=['$758TrMVR2MlE4EhpRUvCYZFcp11cbE27edQcwKaUSI8']", "event_id=$T5-OUXAW3CCYAHskqk5s0V2ZzJoQy5D8lQJ7LZFWvTI,depth=9,body=Message 1 (eventIDsBefore),prevs=['$WYEMee2J5PcH1yM2XDnD2XIhyMS_HHDdYXop6BOL69E']", "event_id=$WYEMee2J5PcH1yM2XDnD2XIhyMS_HHDdYXop6BOL69E,depth=8,body=Message 0 (eventIDsBefore),prevs=['$jTD4_ZLQQgXienBOsHMwSqyuuYtXQq4WYj7hnPbRC0o']", "event_id=$jTD4_ZLQQgXienBOsHMwSqyuuYtXQq4WYj7hnPbRC0o,depth=7,body=m.room.member,prevs=['$d9EI1fE4ArZLYeQPs3P6dlqTmtskwjJWtlfNEZTdSoA']", "event_id=$d9EI1fE4ArZLYeQPs3P6dlqTmtskwjJWtlfNEZTdSoA,depth=6,body=m.room.name,prevs=['$VvA6Hhp67rDLix7Pg9tZs6DDDh0gq_Mw-qr9wd5cO2k']", "event_id=$VvA6Hhp67rDLix7Pg9tZs6DDDh0gq_Mw-qr9wd5cO2k,depth=5,body=m.room.history_visibility,prevs=['$7fkTDT3EgbweX381_HInASE-jAzhxcQfkzOWuXxXO7U']", "event_id=$7fkTDT3EgbweX381_HInASE-jAzhxcQfkzOWuXxXO7U,depth=4,body=m.room.join_rules,prevs=['$uvO6_KB-CJtOtomqSgSAWLL59mJK0JW2U33ebFTuoK4']", "event_id=$uvO6_KB-CJtOtomqSgSAWLL59mJK0JW2U33ebFTuoK4,depth=3,body=m.room.power_levels,prevs=['$HjRQ8RpCPPWO4Ufq6bSCTqa_tSlKez15KvZRMpvvskc']", "event_id=$HjRQ8RpCPPWO4Ufq6bSCTqa_tSlKez15KvZRMpvvskc,depth=2,body=m.room.member,prevs=['$yXLeDjVhdX5jmjLVxmsxKhG_rjfA9ujL8Cd8eHNha6U']", 'event_id=$758TrMVR2MlE4EhpRUvCYZFcp11cbE27edQcwKaUSI8,depth=1,body=m.room.member,prevs=[]', 'event_id=$yXLeDjVhdX5jmjLVxmsxKhG_rjfA9ujL8Cd8eHNha6U,depth=1,body=m.room.create,prevs=[]' ] ``` | ||||
* | Remove debug logging | Eric Eastwood | 2021-10-21 | 1 | -4/+0 |
| | |||||
* | Remove fake prev events from historical state chain | Eric Eastwood | 2021-10-21 | 1 | -0/+2 |
| | | | | | | | | | | | Fix https://github.com/matrix-org/synapse/issues/11091 We have to allow creation of events with no prev_events but do have auth_events. And since the historical member events are outliers with no prev_events to resolve them, we want to avoid putting them as backward extremeties. | ||||
* | Some more debug logging | Eric Eastwood | 2021-10-20 | 1 | -0/+2 |
| | |||||
* | Strip "join_authorised_via_users_server" from join events which do not need ↵ | Patrick Cloke | 2021-09-30 | 3 | -9/+9 |
| | | | | | | | it. (#10933) This fixes a "Event not signed by authorising server" error when transition room member from join -> join, e.g. when updating a display name or avatar URL for restricted rooms. | ||||
* | add event id to logcontext when handling incoming PDUs (#10936) | Richard van der Hoff | 2021-09-29 | 1 | -1/+4 |
| | |||||
* | Use direct references for configuration variables (part 6). (#10916) | Patrick Cloke | 2021-09-29 | 1 | -1/+1 |
| | |||||
* | Factor out common code for persisting fetched auth events (#10896) | Richard van der Hoff | 2021-09-24 | 1 | -2/+0 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Factor more stuff out of `_get_events_and_persist` It turns out that the event-sorting algorithm in `_get_events_and_persist` is also useful in other circumstances. Here we move the current `_auth_and_persist_fetched_events` to `_auth_and_persist_fetched_events_inner`, and then factor the sorting part out to `_auth_and_persist_fetched_events`. * `_get_remote_auth_chain_for_event`: remove redundant `outlier` assignment `get_event_auth` returns events with the outlier flag already set, so this is redundant (though we need to update a test where `get_event_auth` is mocked). * `_get_remote_auth_chain_for_event`: move existing-event tests earlier Move a couple of tests outside the loop. This is a bit inefficient for now, but a future commit will make it better. It should be functionally identical. * `_get_remote_auth_chain_for_event`: use `_auth_and_persist_fetched_events` We can use the same codepath for persisting the events fetched as part of an auth chain as for those fetched individually by `_get_events_and_persist` for building the state at a backwards extremity. * `_get_remote_auth_chain_for_event`: use a dict for efficiency `_auth_and_persist_fetched_events` sorts the events itself, so we no longer need to care about maintaining the ordering from `get_event_auth` (and no longer need to sort by depth in `get_event_auth`). That means that we can use a map, making it easier to filter out events we already have, etc. * changelog * `_auth_and_persist_fetched_events`: improve docstring | ||||
* | Use direct references for configuration variables (part 4). (#10893) | Patrick Cloke | 2021-09-23 | 1 | -1/+3 |
| | |||||
* | Remove unnecessary parentheses around tuples returned from methods (#10889) | Andrew Morgan | 2021-09-23 | 1 | -2/+2 |
| | |||||
* | Use direct references for some configuration variables (part 2) (#10812) | Patrick Cloke | 2021-09-15 | 2 | -2/+2 |
| | |||||
* | Use direct references for some configuration variables (#10798) | Patrick Cloke | 2021-09-13 | 1 | -1/+2 |
| | | | | Instead of proxying through the magic getter of the RootConfig object. This should be more performant (and is more explicit). | ||||
* | Add types to synapse.util. (#10601) | reivilibre | 2021-09-10 | 1 | -2/+6 |
| | |||||
* | Split `FederationHandler` in half (#10692) | Richard van der Hoff | 2021-08-26 | 1 | -2/+5 |
| | | | The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller. | ||||
* | Cache the result of fetching the room hierarchy over federation. (#10647) | Patrick Cloke | 2021-08-26 | 1 | -40/+66 |
| | |||||
* | Do not include stack traces for known exceptions when trying multiple ↵ | Patrick Cloke | 2021-08-23 | 1 | -1/+6 |
| | | | | federation destinations. (#10662) | ||||
* | Split `on_receive_pdu` in half (#10640) | Richard van der Hoff | 2021-08-19 | 1 | -3/+1 |
| | | | Here we split on_receive_pdu into two functions (on_receive_pdu and process_pulled_event), rather than having both cases in the same method. There's a tiny bit of overlap, but not that much. | ||||
* | Attempt to pull from the legacy spaces summary API over federation. (#10583) | Patrick Cloke | 2021-08-17 | 1 | -9/+55 |
| | | | | | | | If the new /hierarchy API does not exist on all destinations, fallback to querying the /spaces API and translating the results. This is a backwards compatibility hack since not all of the federated homeservers will update at the same time. | ||||
* | Validate the max_rooms_per_space parameter to ensure it is non-negative. ↵ | Patrick Cloke | 2021-08-16 | 1 | -4/+18 |
| | | | | (#10611) | ||||
* | Experimental support for MSC3266 Room Summary API. (#10394) | Michael Telatynski | 2021-08-16 | 1 | -2/+2 |
| | |||||
* | Split `synapse.federation.transport.server` into multiple files. (#10590) | Patrick Cloke | 2021-08-16 | 6 | -2158/+2218 |
| | |||||
* | Clean up some logging in the federation event handler (#10591) | Richard van der Hoff | 2021-08-16 | 1 | -0/+1 |
| | | | | | | | | | | | | | | | | | | | * Include outlier status in `str(event)` In places where we log event objects, knowing whether or not you're dealing with an outlier is super useful. * Remove duplicated logging in get_missing_events When we process events received from get_missing_events, we log them twice (once in `_get_missing_events_for_pdu`, and once in `on_receive_pdu`). Reduce the duplication by removing the logging in `on_receive_pdu`, and ensuring the call sites do sensible logging. * log in `on_receive_pdu` when we already have the event * Log which prev_events we are missing * changelog | ||||
* | Support federation in the new spaces summary API (MSC2946). (#10569) | Patrick Cloke | 2021-08-16 | 3 | -0/+132 |
| | |||||
* | Fix a harmless exception when the staged events queue is empty. (#10592) | Patrick Cloke | 2021-08-13 | 1 | -5/+10 |
| | |||||
* | Convert Transaction and Edu object to attrs (#10542) | Patrick Cloke | 2021-08-06 | 6 | -92/+74 |
| | | | | | Instead of wrapping the JSON into an object, this creates concrete instances for Transaction and Edu. This allows for improved type hints and simplified code. | ||||
* | Fix exceptions in logs when failing to get remote room list (#10541) | Erik Johnston | 2021-08-06 | 1 | -1/+2 |
| | |||||
* | Refactoring before implementing the updated spaces summary. (#10527) | Patrick Cloke | 2021-08-05 | 1 | -9/+14 |
| | | | | | This should have no user-visible changes, but refactors some pieces of the SpaceSummaryHandler before adding support for the updated MSC2946. | ||||
* | Prune inbound federation queues if they get too long (#10390) | Erik Johnston | 2021-08-02 | 1 | -0/+17 |
| | |||||
* | Improve failover logic for MSC3083 restricted rooms. (#10447) | Patrick Cloke | 2021-07-29 | 1 | -4/+39 |
| | | | | | If the federation client receives an M_UNABLE_TO_AUTHORISE_JOIN or M_UNABLE_TO_GRANT_JOIN response it will attempt another server before giving up completely. | ||||
* | Update the MSC3083 support to verify if joins are from an authorized server. ↵ | Patrick Cloke | 2021-07-26 | 4 | -19/+141 |
| | | | | (#10254) | ||||
* | Add type hints to synapse.federation.transport.client. (#10408) | Patrick Cloke | 2021-07-26 | 1 | -201/+298 |
| | |||||
* | Add type hints to additional servlet functions (#10437) | Patrick Cloke | 2021-07-21 | 1 | -11/+2 |
| | | | | | | | | | Improves type hints for: * parse_{boolean,integer} * parse_{boolean,integer}_from_args * parse_json_{value,object}_from_request And fixes any incorrect calls that resulted from unknown types. | ||||
* | Do not include signatures/hashes in make_{join,leave,knock} responses. (#10404) | Patrick Cloke | 2021-07-16 | 1 | -6/+3 |
| | | | | These signatures would end up invalid since the joining/leaving/knocking server would modify the response before calling send_{join,leave,knock}. | ||||
* | Stagger send presence to remotes (#10398) | Erik Johnston | 2021-07-15 | 2 | -5/+107 |
| | | | | | | This is to help with performance, where trying to connect to thousands of hosts at once can consume a lot of CPU (due to TLS etc). Co-authored-by: Brendan Abolivier <babolivier@matrix.org> | ||||
* | Use inline type hints in various other places (in `synapse/`) (#10380) | Jonathan de Jong | 2021-07-15 | 7 | -72/+62 |
| | |||||
* | Add type hints to get_domain_from_id and get_localpart_from_id. (#10385) | Patrick Cloke | 2021-07-13 | 1 | -24/+72 |
| | |||||
* | Ensure we always drop the federation inbound lock (#10336) | Erik Johnston | 2021-07-09 | 1 | -0/+1 |
| | |||||
* | Handle old staged inbound events (#10303) | Erik Johnston | 2021-07-06 | 1 | -10/+57 |
| | | | | | | | We might have events in the staging area if the service was restarted while there were unhandled events in the staging area. Fixes #10295 | ||||
* | Move methods involving event authentication to EventAuthHandler. (#10268) | Patrick Cloke | 2021-07-01 | 1 | -3/+3 |
| | | | Instead of mixing them with user authentication methods. | ||||
* | Fix the inbound PDU metric (#10279) | Erik Johnston | 2021-06-30 | 1 | -17/+20 |
| | | | This broke in #10272 | ||||
* | Merge branch 'release-v1.37' into develop | Richard van der Hoff | 2021-06-29 | 1 | -2/+96 |
|\ | |||||
| * | Handle inbound events from federation asynchronously (#10272) | Erik Johnston | 2021-06-29 | 1 | -2/+96 |
| | | | | | | | | | | | | | | | | | | | | | | Fixes #9490 This will break a couple of SyTest that are expecting failures to be added to the response of a federation /send, which obviously doesn't happen now that things are asynchronous. Two drawbacks: Currently there is no logic to handle any events left in the staging area after restart, and so they'll only be handled on the next incoming event in that room. That can be fixed separately. We now only process one event per room at a time. This can be fixed up further down the line. | ||||
* | | Soft-fail spammy events received over federation (#10263) | Richard van der Hoff | 2021-06-29 | 1 | -6/+6 |
| | | |||||
* | | Add additional types to the federation transport server. (#10213) | Patrick Cloke | 2021-06-28 | 1 | -114/+474 |
| | | |||||
* | | Improve validation for `send_{join,leave,knock}` (#10225) | Richard van der Hoff | 2021-06-24 | 2 | -55/+78 |
|/ | | | The idea here is to stop people sending things that aren't joins/leaves/knocks through these endpoints: previously you could send anything you liked through them. I wasn't able to find any security holes from doing so, but it doesn't sound like a good thing. | ||||
* | Expose opentracing trace id in response headers (#10199) | Richard van der Hoff | 2021-06-18 | 1 | -0/+3 |
| | | | Fixes: #9480 | ||||
* | Remove the experimental flag for knocking and use stable prefixes / ↵ | Patrick Cloke | 2021-06-15 | 3 | -51/+7 |
| | | | | | | | endpoints. (#10167) * Room version 7 for knocking. * Stable prefixes and endpoints (both client and federation) for knocking. * Removes the experimental configuration flag. | ||||
* | Implement knock feature (#6739) | Sorunome | 2021-06-09 | 4 | -8/+277 |
| | | | | | | This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403 Signed-off-by: Sorunome mail@sorunome.de Signed-off-by: Andrew Morgan andrewm@element.io | ||||
* | Add type hints to the federation server transport. (#10080) | Patrick Cloke | 2021-06-08 | 2 | -72/+166 |
| | |||||
* | When joining a remote room limit the number of events we concurrently check ↵ | Erik Johnston | 2021-06-08 | 2 | -217/+173 |
| | | | | | signatures/hashes for (#10117) If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB. | ||||
* | Rewrite the KeyRing (#10035) | Erik Johnston | 2021-06-02 | 1 | -1/+3 |
| | |||||
* | Merge branch 'master' into develop | Andrew Morgan | 2021-06-01 | 1 | -0/+7 |
|\ | |||||
| * | Allow response of `/send_join` to be larger. (#10093) | Erik Johnston | 2021-05-28 | 1 | -0/+7 |
| | | | | | | Fixes #10087. | ||||
* | | Set opentracing priority before setting other tags (#10092) | Richard van der Hoff | 2021-05-28 | 1 | -1/+2 |
| | | | | | | ... because tags on spans which aren't being sampled get thrown away. | ||||
* | | Merge tag 'v1.35.0rc2' into develop | Erik Johnston | 2021-05-27 | 1 | -1/+1 |
|\| | | | | | | | | | | | | | | | | | | | Synapse 1.35.0rc2 (2021-05-27) ============================== Bugfixes -------- - Fix a bug introduced in v1.35.0rc1 when calling the spaces summary API via a GET request. ([\#10079](https://github.com/matrix-org/synapse/issues/10079)) | ||||
| * | Pass the origin when calculating the spaces summary over GET. (#10079) | Patrick Cloke | 2021-05-27 | 1 | -1/+1 |
| | | | | | | | | Fixes a bug due to conflicting PRs which were merged. (One added a new caller to a method, the other added a new parameter to the same method.) | ||||
* | | Remove the experimental spaces enabled flag. (#10063) | Patrick Cloke | 2021-05-26 | 1 | -7/+6 |
|/ | | | | In lieu of just always enabling the unstable spaces endpoint and unstable room version. | ||||
* | Don't hammer the database for destination retry timings every ~5mins (#10036) | Erik Johnston | 2021-05-21 | 1 | -1/+1 |
| | |||||
* | Add `Keyring.verify_events_for_server` and reduce memory usage (#10018) | Erik Johnston | 2021-05-20 | 1 | -12/+5 |
| | | | | | | Also add support for giving a callback to generate the JSON object to verify. This should reduce memory usage, as we no longer have the event in memory in dict form (which has a large memory footprint) for extend periods of time. | ||||
* | Use ijson to parse the response to `/send_join`, reducing memory usage. (#9958) | Erik Johnston | 2021-05-20 | 2 | -22/+91 |
| | | | Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects. | ||||
* | Allow a user who could join a restricted room to see it in spaces summary. ↵ | Patrick Cloke | 2021-05-20 | 1 | -1/+1 |
| | | | | | | (#9922) This finishes up the experimental implementation of MSC3083 by showing the restricted rooms in the spaces summary (from MSC2946). | ||||
* | Support fetching the spaces summary via GET over federation. (#9947) | Patrick Cloke | 2021-05-11 | 2 | -0/+27 |
| | | | | | | | | | | | Per changes in MSC2946, the C-S and S-S APIs for spaces summary should use GET requests. Until this is stable, the POST endpoints still exist. This does not switch federation requests to use the GET version yet since it is newly added and already deployed servers might not support it. When switching to the stable endpoint we should switch to GET requests. | ||||
* | Add debug logging for issue #9533 (#9959) | Richard van der Hoff | 2021-05-11 | 1 | -0/+9 |
| | | | | | Hopefully this will help us track down where to-device messages are getting lost/delayed. | ||||
* | Fix `m.room_key_request` to-device messages (#9961) | Richard van der Hoff | 2021-05-11 | 1 | -19/+0 |
| | | | fixes #9960 | ||||
* | Revert "Experimental Federation Speedup (#9702)" | Andrew Morgan | 2021-04-28 | 2 | -102/+58 |
| | | | | This reverts commit 05e8c70c059f8ebb066e029bc3aa3e0cefef1019. | ||||
* | Pass errors back to the client when trying multiple federation destinations. ↵ | Patrick Cloke | 2021-04-27 | 1 | -58/+60 |
| | | | | | | | | (#9868) This ensures that something like an auth error (403) will be returned to the requester instead of attempting to try more servers, which will likely result in the same error, and then passing back a generic 400 error. | ||||
* | Remove `synapse.types.Collection` (#9856) | Richard van der Hoff | 2021-04-22 | 1 | -2/+12 |
| | | | This is no longer required, since we have dropped support for Python 3.5. | ||||
* | Fix bug where we sent remote presence states to remote servers (#9850) | Erik Johnston | 2021-04-20 | 1 | -0/+4 |
| | |||||
* | Fix (final) Bugbear violations (#9838) | Jonathan de Jong | 2021-04-20 | 1 | -2/+2 |
| | |||||
* | Don't send normal presence updates over federation replication stream (#9828) | Erik Johnston | 2021-04-19 | 2 | -163/+3 |
| | |||||
* | remove `HomeServer.get_config` (#9815) | Richard van der Hoff | 2021-04-14 | 2 | -2/+2 |
| | | | | Every single time I want to access the config object, I have to remember whether or not we use `get_config`. Let's just get rid of it. | ||||
* | Experimental Federation Speedup (#9702) | Jonathan de Jong | 2021-04-14 | 2 | -62/+93 |
| | | | | | This basically speeds up federation by "squeezing" each individual dual database call (to destinations and destination_rooms), which previously happened per every event, into one call for an entire batch (100 max). Signed-off-by: Jonathan de Jong <jonathan@automatia.nl> | ||||
* | Remove redundant "coding: utf-8" lines (#9786) | Jonathan de Jong | 2021-04-14 | 13 | -13/+0 |
| | | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>` | ||||
* | Fix duplicate logging of exceptions in transaction processing (#9780) | Richard van der Hoff | 2021-04-09 | 1 | -7/+3 |
| | | | There's no point logging this twice. | ||||
* | Bugbear: Add Mutable Parameter fixes (#9682) | Jonathan de Jong | 2021-04-08 | 1 | -2/+3 |
| | | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl> | ||||
* | Fix sharded federation sender sometimes using 100% CPU. | Erik Johnston | 2021-04-08 | 1 | -2/+4 |
| | | | | | | | We pull all destinations requiring catchup from the DB in batches. However, if all those destinations get filtered out (due to the federation sender being sharded), then the `last_processed` destination doesn't get updated, and we keep requesting the same set repeatedly. | ||||
* | Add a Synapse Module for configuring presence update routing (#9491) | Andrew Morgan | 2021-04-06 | 1 | -1/+18 |
| | | | | | | | | | | | | At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state. | ||||
* | Add type hints to expiring cache. (#9730) | Patrick Cloke | 2021-04-06 | 1 | -1/+1 |
| | |||||
* | Add type hints to the federation handler and server. (#9743) | Patrick Cloke | 2021-04-06 | 2 | -15/+15 |
| | |||||
* | Improve tracing for to device messages (#9686) | Erik Johnston | 2021-04-01 | 1 | -0/+8 |
| | |||||
* | Make RateLimiter class check for ratelimit overrides (#9711) | Erik Johnston | 2021-03-30 | 1 | -1/+4 |
| | | | | | | | This should fix a class of bug where we forget to check if e.g. the appservice shouldn't be ratelimited. We also check the `ratelimit_override` table to check if the user has ratelimiting disabled. That table is really only meant to override the event sender ratelimiting, so we don't use any values from it (as they might not make sense for different rate limits), but we do infer that if ratelimiting is disabled for the user we should disabled all ratelimits. Fixes #9663 | ||||
* | Add type hints for the federation sender. (#9681) | Patrick Cloke | 2021-03-29 | 2 | -44/+160 |
| | | | | Includes an abstract base class which both the FederationSender and the FederationRemoteSendQueue must implement. | ||||
* | Fixed undefined variable error in catchup (#9664) | Erik Johnston | 2021-03-24 | 1 | -0/+2 |
| | | | | | Broke in #9640 Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> | ||||
* | Spaces summary: call out to other servers (#9653) | Richard van der Hoff | 2021-03-24 | 2 | -11/+204 |
| | | | | | When we hit an unknown room in the space tree, see if there are other servers that we might be able to poll to get the data. Fixes: #9447 | ||||
* | Federation API for Space summary (#9652) | Richard van der Hoff | 2021-03-23 | 1 | -9/+58 |
| | | | | | Builds on the work done in #9643 to add a federation API for space summaries. There's a bit of refactoring of the existing client-server code first, to avoid too much duplication. | ||||
* | Import HomeServer from the proper module. (#9665) | Patrick Cloke | 2021-03-23 | 1 | -1/+1 |
| | |||||
* | Make federation catchup send last event from any server. (#9640) | Erik Johnston | 2021-03-18 | 2 | -38/+91 |
| | | | | | | | | | | | | | | Currently federation catchup will send the last *local* event that we failed to send to the remote. This can cause issues for large rooms where lots of servers have sent events while the remote server was down, as when it comes back up again it'll be flooded with events from various points in the DAG. Instead, let's make it so that all the servers send the most recent events, even if its not theirs. The remote should deduplicate the events, so there shouldn't be much overhead in doing this. Alternatively, the servers could only send local events if they were also extremities and hope that the other server will send the event over, but that is a bit risky. | ||||
* | Don't go into federation catch up mode so easily (#9561) | Erik Johnston | 2021-03-15 | 2 | -153/+182 |
| | | | | | | | | | | Federation catch up mode is very inefficient if the number of events that the remote server has missed is small, since handling gaps can be very expensive, c.f. #9492. Instead of going into catch up mode whenever we see an error, we instead do so only if we've backed off from trying the remote for more than an hour (the assumption being that in such a case it is more than a transient failure). | ||||
* | Fix additional type hints from Twisted 21.2.0. (#9591) | Patrick Cloke | 2021-03-12 | 1 | -3/+5 |
| | |||||
* | Reject concurrent transactions (#9597) | Richard van der Hoff | 2021-03-12 | 1 | -35/+42 |
| | | | | | | If more transactions arrive from an origin while we're still processing the first one, reject them. Hopefully a quick fix to https://github.com/matrix-org/synapse/issues/9489 | ||||
* | Improve logging when processing incoming transactions (#9596) | Richard van der Hoff | 2021-03-12 | 1 | -27/+34 |
| | | | Put the room id in the logcontext, to make it easier to understand what's going on. | ||||
* | Use the chain cover index in get_auth_chain_ids. (#9576) | Patrick Cloke | 2021-03-10 | 1 | -2/+4 |
| | | | | This uses a simplified version of get_chain_cover_difference to calculate auth chain of events. | ||||
* | Fix additional type hints. (#9543) | Patrick Cloke | 2021-03-09 | 1 | -1/+1 |
| | | | Type hint fixes due to Twisted 21.2.0 adding type hints. | ||||
* | Add ResponseCache tests. (#9458) | Jonathan de Jong | 2021-03-08 | 1 | -5/+8 |
| | |||||
* | Replace `last_*_pdu_age` metrics with timestamps (#9540) | Richard van der Hoff | 2021-03-04 | 2 | -12/+9 |
| | | | | | | | | Following the advice at https://prometheus.io/docs/practices/instrumentation/#timestamps-not-time-since, it's preferable to export unix timestamps, not ages. There doesn't seem to be any particular naming convention for timestamp metrics. | ||||
* | Ratelimit cross-user key sharing requests. (#8957) | Patrick Cloke | 2021-02-19 | 1 | -2/+18 |
| | |||||
* | Be smarter about which hosts to send presence to when processing room joins ↵ | Andrew Morgan | 2021-02-19 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | (#9402) This PR attempts to eliminate unnecessary presence sending work when your local server joins a room, or when a remote server joins a room your server is participating in by processing state deltas in chunks rather than individually. --- When your server joins a room for the first time, it requests the historical state as well. This chunk of new state is passed to the presence handler which, after filtering that state down to only membership joins, will send presence updates to homeservers for each join processed. It turns out that we were being a bit naive and processing each event individually, and sending out presence updates for every one of those joins. Even if many different joins were users on the same server (hello IRC bridges), we'd send presence to that same homeserver for every remote user join we saw. This PR attempts to deduplicate all of that by processing the entire batch of state deltas at once, instead of only doing each join individually. We process the joins and note down which servers need which presence: * If it was a local user join, send that user's latest presence to all servers in the room * If it was a remote user join, send the presence for all local users in the room to that homeserver We deduplicate by inserting all of those pending updates into a dictionary of the form: ``` { server_name1: {presence_update1, ...}, server_name2: {presence_update1, presence_update2, ...} } ``` Only after building this dict do we then start sending out presence updates. | ||||
* | Add configs to make profile data more private (#9203) | AndrewFerr | 2021-02-19 | 1 | -4/+3 |
| | | | | | | | Add off-by-default configuration settings to: - disable putting an invitee's profile info in invite events - disable profile lookup via federation Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net> | ||||
* | Add type hints to groups code. (#9393) | Patrick Cloke | 2021-02-17 | 1 | -2/+39 |
| | |||||
* | Update black, and run auto formatting over the codebase (#9381) | Eric Eastwood | 2021-02-16 | 10 | -127/+99 |
| | | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version | ||||
* | Ratelimit invites by room and target user (#9258) | Erik Johnston | 2021-01-29 | 1 | -1/+1 |
| | |||||
* | Precompute joined hosts and store in Redis (#9198) | Erik Johnston | 2021-01-26 | 1 | -15/+35 |
| | |||||
* | Merge tag 'v1.26.0rc1' into develop | Patrick Cloke | 2021-01-20 | 2 | -2/+2 |
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.26.0rc1 (2021-01-20) ============================== This release brings a new schema version for Synapse and rolling back to a previous verious is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes and for general upgrade guidance. Features -------- - Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177)) - During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091)) - Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159)) - Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024)) - Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984)) - Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086)) - Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932)) - Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948)) - Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130)) - Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068)) - Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092)) - Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166)) Bugfixes -------- - Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023)) - Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028)) - Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051)) - Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053)) - Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054)) - Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059)) - Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070)) - Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071)) - Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116)) - Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117)) - Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128)) - Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108)) - Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145)) - Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161)) Improved Documentation ---------------------- - Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997)) - Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035)) - Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040)) - Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057)) - Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151)) Deprecations and Removals ------------------------- - Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039)) Internal Changes ---------------- - Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124)) - Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939)) - Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016)) - Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018)) - Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025)) - Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030)) - Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031)) - Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033)) - Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038)) - Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041)) - Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055)) - Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058)) - Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063)) - Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069)) - Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080)) - Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093)) - Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098)) - Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106)) - Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112)) - Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125)) - Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144)) - Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146)) - Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157)) | ||||
| * | Support icons for Identity Providers (#9154) | Richard van der Hoff | 2021-01-20 | 2 | -2/+2 |
| | | |||||
* | | Various improvements to the federation client. (#9129) | Patrick Cloke | 2021-01-20 | 1 | -58/+67 |
|/ | | | | * Type hints for `FederationClient`. * Using `async` functions instead of returning `Awaitable` instances. | ||||
* | Support routing edu's to multiple instances (#9042) | Erik Johnston | 2021-01-07 | 1 | -5/+16 |
| | | | This is in preparation for moving `SendToDeviceServlet` off master | ||||
* | Add type hints to the crypto module. (#8999) | Patrick Cloke | 2021-01-04 | 1 | -1/+1 |
| | |||||
* | Allow spam-checker modules to be provide async methods. (#8890) | David Teller | 2020-12-11 | 1 | -1/+6 |
| | | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner. | ||||
* | Merge pull request #8858 from matrix-org/rav/sso_uia | Richard van der Hoff | 2020-12-02 | 1 | -1/+1 |
|\ | | | | | UIA: offer only available auth flows | ||||
| * | fix up various test cases | Richard van der Hoff | 2020-12-02 | 1 | -1/+1 |
| | | | | | | | | | | A few test cases were relying on being able to mount non-client servlets on the test resource. it's better to give them their own Resources. | ||||
* | | Apply an IP range blacklist to push and key revocation requests. (#8821) | Patrick Cloke | 2020-12-02 | 2 | -2/+1 |
|/ | | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers). | ||||
* | Consistently use room_id from federation request body (#8776) | Richard van der Hoff | 2020-11-19 | 2 | -48/+43 |
| | | | | | | | | | | | | | * Consistently use room_id from federation request body Some federation APIs have a redundant `room_id` path param (see https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we consistently use either the path param or the body param, and the body param is easier. * Kill off some references to "context" Once upon a time, "rooms" were known as "contexts". I think this kills of the last references to "contexts". | ||||
* | Add ability for access tokens to belong to one user but grant access to ↵ | Erik Johnston | 2020-10-29 | 1 | -1/+1 |
| | | | | | | | | | | another user. (#8616) We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't). A future PR will add an API for creating such a token. When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome. | ||||
* | Fix not sending events over federation when using sharded event persisters ↵ | Erik Johnston | 2020-10-14 | 2 | -3/+8 |
| | | | | | | | | | | | | | | | | | (#8536) * Fix outbound federaion with multiple event persisters. We incorrectly notified federation senders that the minimum persisted stream position had advanced when we got an `RDATA` from an event persister. Notifying of federation senders already correctly happens in the notifier, so we just delete the offending line. * Change some interfaces to use RoomStreamToken. By enforcing use of `RoomStreamTokens` we make it less likely that people pass in random ints that they got from somewhere random. | ||||
* | Add type hints to response cache. (#8507) | Patrick Cloke | 2020-10-09 | 1 | -3/+5 |
| | |||||
* | Remove the deprecated Handlers object (#8494) | Patrick Cloke | 2020-10-09 | 1 | -1/+6 |
| | | | All handlers now available via get_*_handler() methods on the HomeServer. | ||||
* | Add type hints to some handlers (#8505) | Patrick Cloke | 2020-10-09 | 1 | -1/+1 |
| | |||||
* | Remove stream ordering from Metadata dict (#8452) | Richard van der Hoff | 2020-10-05 | 2 | -0/+4 |
| | | | | | | | | There's no need for it to be in the dict as well as the events table. Instead, we store it in a separate attribute in the EventInternalMetadata object, and populate that on load. This means that we can rely on it being correctly populated for any event which has been persited to the database. | ||||
* | Merge tag 'v1.21.0rc2' into develop | Richard van der Hoff | 2020-10-02 | 1 | -1/+1 |
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.21.0rc2 (2020-10-02) ============================== Features -------- - Convert additional templates from inline HTML to Jinja2 templates. ([\#8444](https://github.com/matrix-org/synapse/issues/8444)) Bugfixes -------- - Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. ([\#8438](https://github.com/matrix-org/synapse/issues/8438)) - Do not expose the experimental `uk.half-shot.msc2778.login.application_service` flow in the login API, which caused a compatibility problem with Element iOS. ([\#8440](https://github.com/matrix-org/synapse/issues/8440)) - Fix malformed log line in new federation "catch up" logic. ([\#8442](https://github.com/matrix-org/synapse/issues/8442)) - Fix DB query on startup for negative streams which caused long start up times. Introduced in [\#8374](https://github.com/matrix-org/synapse/issues/8374). ([\#8447](https://github.com/matrix-org/synapse/issues/8447)) | ||||
| * | Fix malformed log line in new federation "catch up" logic (#8442) | Richard van der Hoff | 2020-10-02 | 1 | -1/+1 |
| | | |||||
* | | Enable mypy checking for unreachable code and fix instances. (#8432) | Patrick Cloke | 2020-10-01 | 1 | -3/+2 |
|/ | |||||
* | Add prometheus metrics to track federation delays (#8430) | Richard van der Hoff | 2020-10-01 | 2 | -1/+45 |
| | | | | | Add a pair of federation metrics to track the delays in sending PDUs to/from particular servers. | ||||
* | Mypy fixes for `synapse.handlers.federation` (#8422) | Richard van der Hoff | 2020-09-29 | 1 | -1/+3 |
| | | | For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes. | ||||
* | Merge tag 'v1.20.0rc5' into develop | Patrick Cloke | 2020-09-18 | 1 | -5/+3 |
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.20.0rc5 (2020-09-18) ============================== In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3. Features -------- - Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343)) Bugfixes -------- - Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342)) - Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349)) Internal Changes ---------------- - Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285)) | ||||
| * | Merge tag 'v1.19.3' into release-v1.20.0 | Patrick Cloke | 2020-09-18 | 1 | -5/+3 |
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1.19.3 Synapse 1.19.3 (2020-09-18) =========================== Bugfixes -------- - Partially mitigate bug where newly joined servers couldn't get past events in a room when there is a malformed event. ([\#8350](https://github.com/matrix-org/synapse/issues/8350)) | ||||
| | * | Use _check_sigs_and_hash_and_fetch to validate backfill requests (#8350) | Andrew Morgan | 2020-09-18 | 1 | -5/+3 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a bit of a hack, as `_check_sigs_and_hash_and_fetch` is intended for attempting to pull an event from the database/(re)pull it from the server that originally sent the event if checking the signature of the event fails. During backfill we *know* that we won't have the event in our database, however it is still useful to be able to query the original sending server as the server we're backfilling from may be acting maliciously. The main benefit and reason for this change however is that `_check_sigs_and_hash_and_fetch` will drop an event during backfill if it cannot be successfully validated, whereas the current code will simply fail the backfill request - resulting in the client's /messages request silently being dropped. This is a quick patch to fix backfilling rooms that contain malformed events. A better implementation in planned in future. | ||||
* | | | Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322) | reivilibre | 2020-09-18 | 1 | -0/+51 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Fix _set_destination_retry_timings This came about because the code assumed that retry_interval could not be NULL — which has been challenged by catch-up. | ||||
* | | | Simplify super() calls to Python 3 syntax. (#8344) | Patrick Cloke | 2020-09-18 | 3 | -9/+5 |
| | | | | | | | | | | | | | | | | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py | ||||
* | | | Merge remote-tracking branch 'origin/release-v1.20.0' into develop | Erik Johnston | 2020-09-18 | 2 | -17/+48 |
|\| | | |||||
| * | | Fix ratelimiting for federation `/send` requests. (#8342) | Erik Johnston | 2020-09-18 | 2 | -17/+48 |
| | | | | | | | | | c.f. #8295 for rationale | ||||
| * | | Merge branch 'erikj/fix_origin_check' into release-v1.20.0 | Erik Johnston | 2020-09-16 | 1 | -3/+4 |
| |\| | |||||
* | | | Merge branch 'erikj/fix_origin_check' into develop | Erik Johnston | 2020-09-16 | 1 | -3/+4 |
|\ \ \ | | |/ | |/| | |||||
| * | | Don't assume that an event has an origin field | Erik Johnston | 2020-09-16 | 1 | -3/+4 |
| | | | | | | | | | | | | This fixes #8319. | ||||
* | | | Catch-up after Federation Outage (split, 4): catch-up loop (#8272) | reivilibre | 2020-09-15 | 1 | -4/+125 |
| | | | |||||
* | | | Catch up after Federation Outage (split, 2): Track last successful stream ↵ | reivilibre | 2020-09-04 | 1 | -0/+11 |
| | | | | | | | | | | | | | | | ordering after transmission (#8247) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> | ||||
* | | | Catch-up after Federation Outage (split, 1) (#8230) | reivilibre | 2020-09-04 | 1 | -2/+9 |
| |/ |/| | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> | ||||
* | | Stop sub-classing object (#8249) | Patrick Cloke | 2020-09-04 | 9 | -11/+11 |
| | | |||||
* | | Remove obsolete order field in `send_new_transaction` (#8245) | reivilibre | 2020-09-03 | 3 | -28/+22 |
| | | | | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> | ||||
* | | Convert additional database code to async/await. (#8195) | Patrick Cloke | 2020-08-28 | 1 | -8/+11 |
| | | |||||
* | | Convert calls of async database methods to async (#8166) | Patrick Cloke | 2020-08-27 | 2 | -10/+10 |
| | | |||||
* | | Add type hints for state. (#8140) | Patrick Cloke | 2020-08-24 | 1 | -2/+2 |
| | | |||||
* | | Be stricter about JSON that is accepted by Synapse (#8106) | Patrick Cloke | 2020-08-19 | 2 | -6/+4 |
| | | |||||
* | | Convert stream database to async/await. (#8074) | Patrick Cloke | 2020-08-17 | 3 | -3/+3 |
| | | |||||
* | | Drop federation transmission queues during a significant remote outage. (#7864) | reivilibre | 2020-08-13 | 1 | -0/+22 |
|/ | | | | | | | | | | | | | * Empty federation transmission queues when we are backing off. Fixes #7828. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Address feedback Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Reword newsfile | ||||
* | Fix typing for notifier (#8064) | Erik Johnston | 2020-08-12 | 1 | -2/+5 |
| | |||||
* | Merge branch 'master' into develop | Olivier Wilkinson (reivilibre) | 2020-07-30 | 3 | -3/+3 |
|\ | |||||
| * | Update worker docs with recent enhancements (#7969) | Erik Johnston | 2020-07-29 | 3 | -3/+3 |
| | | |||||
* | | Convert federation client to async/await. (#7975) | Patrick Cloke | 2020-07-30 | 3 | -73/+50 |
|/ | |||||
* | Convert state resolution to async/await (#7942) | Patrick Cloke | 2020-07-24 | 1 | -1/+3 |
| | |||||
* | Convert presence handler helpers to async/await. (#7939) | Patrick Cloke | 2020-07-23 | 1 | -1/+3 |
| | |||||
* | Put a cache on `/state_ids` (#7931) | Richard van der Hoff | 2020-07-23 | 1 | -2/+11 |
| | | | | | | | | | | | | | | | | | | If we send out an event which refers to `prev_events` which other servers in the federation are missing, then (after a round or two of backfill attempts), they will end up asking us for `/state_ids` at a particular point in the DAG. As per https://github.com/matrix-org/synapse/issues/7893, this is quite expensive, and we tend to see lots of very similar requests around the same time. We can therefore handle this much more efficiently by using a cache, which (a) ensures that if we see the same request from multiple servers (or even the same server, multiple times), then they share the result, and (b) any other servers that miss the initial excitement can also benefit from the work. [It's interesting to note that `/state` has a cache for exactly this reason. `/state` is now essentially unused and replaced with `/state_ids`, but evidently when we replaced it we forgot to add a cache to the new endpoint.] | ||||
* | Abort federation requests if the client disconnects early (#7930) | Richard van der Hoff | 2020-07-23 | 1 | -0/+6 |
| | | | | | | | | | | | | | | | | | | | | | For inbound federation requests, if a given remote server makes too many requests at once, we start stacking them up rather than processing them immediatedly. However, that means that there is a fair chance that the requesting server will disconnect before we start processing the request. In that case, if it was a read-only request (ie, a GET request), there is absolutely no point in building a response (and some requests are quite expensive to handle). Even in the case of a POST request, one of two things will happen: * Most likely, the requesting server will retry the request and we'll get the information anyway. * Even if it doesn't, the requesting server has to assume that we didn't get the memo, and act accordingly. In short, we're better off aborting the request at this point rather than ploughing on with what might be a quite expensive request. | ||||
* | Convert room list handler to async/await. (#7912) | Patrick Cloke | 2020-07-21 | 1 | -8/+2 |
| | |||||
* | Convert synapse.app to async/await. (#7868) | Patrick Cloke | 2020-07-17 | 1 | -21/+19 |
| | |||||
* | Allow moving typing off master (#7869) | Erik Johnston | 2020-07-16 | 1 | -51/+74 |
| | |||||
* | Add ability to run multiple pusher instances (#7855) | Erik Johnston | 2020-07-16 | 2 | -9/+9 |
| | | | This reuses the same scheme as federation sender sharding | ||||
* | Remove obsolete comment. | Olivier Wilkinson (reivilibre) | 2020-07-16 | 1 | -2/+0 |
| | | | | | | | | It was correct at the time of our friend Jorik writing it (checking git blame), but the world has moved now and it is no longer a generator. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> | ||||
* | Ensure that calls to `json.dumps` are compatible with the standard library ↵ | Patrick Cloke | 2020-07-15 | 1 | -1/+1 |
| | | | | json. (#7836) | ||||
* | Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802) | Patrick Cloke | 2020-07-10 | 1 | -3/+3 |
| | |||||
* | Add ability to shard the federation sender (#7798) | Erik Johnston | 2020-07-10 | 3 | -3/+81 |
| | |||||
* | Fix some spelling mistakes / typos. (#7811) | Patrick Cloke | 2020-07-09 | 6 | -11/+11 |
| | |||||
* | Add `HomeServer.signing_key` property (#7805) | Richard van der Hoff | 2020-07-08 | 1 | -1/+1 |
| | | | ... instead of duplicating `config.signing_key[0]` everywhere | ||||
* | Merge different Resource implementation classes (#7732) | Erik Johnston | 2020-07-03 | 1 | -5/+1 |
| | |||||
* | Fix new metric where we used ms instead of seconds (#7771) | Erik Johnston | 2020-07-01 | 1 | -1/+1 |
| | | | | Introduced in #7755, not yet released. | ||||
* | Add some metrics for inbound and outbound federation processing times (#7755) | Erik Johnston | 2020-06-30 | 2 | -17/+30 |
| | |||||
* | Replace all remaining six usage with native Python 3 equivalents (#7704) | Dagfinn Ilmari Mannsåker | 2020-06-16 | 3 | -8/+3 |
| | |||||
* | Replace iteritems/itervalues/iterkeys with native versions. (#7692) | Patrick Cloke | 2020-06-15 | 4 | -14/+9 |
| | |||||
* | add a comment | Richard van der Hoff | 2020-05-21 | 1 | -0/+3 |
| | |||||
* | Strictly enforce canonicaljson requirements in a new room version (#7381) | Patrick Cloke | 2020-05-14 | 1 | -1/+5 |
| | |||||
* | Fix typing annotations in synapse/federation (#7382) | Richard van der Hoff | 2020-05-05 | 1 | -15/+31 |
| | | | | We're pretty close to having mypy working for `synapse.federation`, so let's finish the job. | ||||
* | Fix catchup-on-reconnect for the Federation Stream (#7374) | Richard van der Hoff | 2020-05-05 | 4 | -25/+39 |
| | | | | looks like we managed to break this during the refactorathon. | ||||
* | async/await is_server_admin (#7363) | Andrew Morgan | 2020-05-01 | 1 | -3/+2 |
| | |||||
* | Further improvements to requesting the public rooms list on a homeserver ↵ | Andrew Morgan | 2020-05-01 | 2 | -23/+70 |
| | | | | which has it set to private (#7368) | ||||
* | Merge branch 'release-v1.12.4' into develop | Richard van der Hoff | 2020-04-22 | 1 | -9/+40 |
|\ | |||||
| * | Query missing cross-signing keys on local sig upload (#7289) | Andrew Morgan | 2020-04-22 | 1 | -9/+40 |
| | | |||||
* | | Merge branch 'release-v1.12.4' into develop | Richard van der Hoff | 2020-04-20 | 1 | -11/+3 |
|\| | |||||
| * | Revert "Query missing cross-signing keys on local sig upload" | Richard van der Hoff | 2020-04-20 | 1 | -11/+3 |
| | | | | | | | | | | | | This was incorrectly merged to the release branch before it was ready. This reverts commit 72fe2affb6ac86d433b80b6452da57052365aa26. | ||||
* | | Merge branch 'release-v1.12.4' into develop | Richard van der Hoff | 2020-04-20 | 1 | -3/+11 |
|\| | |||||
| * | Query missing cross-signing keys on local sig upload | Andrew Morgan | 2020-04-17 | 1 | -3/+11 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add changelog Save retrieved keys to the db lint Fix and de-brittle remote result dict processing Use query_user_devices instead, assume only master, self_signing key types Make changelog more useful Remove very specific exception handling Wrap get_verify_key_from_cross_signing_key in a try/except Note that _get_e2e_cross_signing_verify_key can raise a SynapseError lint Add comment explaining why this is useful Only fetch master and self_signing key types Fix log statements, docstrings Remove extraneous items from remote query try/except lint Factor key retrieval out into a separate function Send device updates, modeled after SigningKeyEduUpdater._handle_signing_key_updates Update method docstring | ||||
* | | Add typing information to federation_server. (#7219) | Patrick Cloke | 2020-04-07 | 1 | -66/+107 |
| | | |||||
* | | Move catchup of replication streams to worker. (#7024) | Erik Johnston | 2020-03-25 | 1 | -0/+9 |
| | | | | | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date. | ||||
* | | Clean up some LoggingContext stuff (#7120) | Richard van der Hoff | 2020-03-24 | 1 | -2/+2 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined. | ||||
* | | Convert `*StreamRow` classes to inner classes (#7116) | Richard van der Hoff | 2020-03-23 | 1 | -1/+1 |
| | | | | | | | | | | This just helps keep the rows closer to their streams, so that it's easier to see what the format of each stream is. | ||||
* | | Fix a bug in the federation API which could cause occasional "Failed to get ↵ | Patrick Cloke | 2020-03-19 | 3 | -30/+21 |
|/ | | | | PDU" errors (#7089). | ||||
* | Remove unused federation endpoint (`query_auth`) (#7026) | Patrick Cloke | 2020-03-17 | 4 | -146/+79 |
| | |||||
* | Add some type annotations to the federation base & client classes (#6995) | Patrick Cloke | 2020-02-28 | 2 | -28/+42 |
| | |||||
* | Merge remote-tracking branch 'origin/release-v1.11.1' into develop | Patrick Cloke | 2020-02-27 | 1 | -6/+8 |
|\ | |||||
| * | Cast a coroutine into a Deferred in the federation base (#6996) | Patrick Cloke | 2020-02-26 | 1 | -6/+8 |
| | | | | | | | | Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room. | ||||
* | | Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) | Patrick Cloke | 2020-02-21 | 1 | -2/+2 |
|/ | | | | Ensure good comprehension hygiene using flake8-comprehensions. | ||||
* | Add support for putting fed user query API on workers (#6873) | Erik Johnston | 2020-02-07 | 1 | -2/+5 |
| | |||||
* | Add a `make_event_from_dict` method (#6858) | Richard van der Hoff | 2020-02-07 | 1 | -3/+2 |
| | | | | | | | ... and use it in places where it's trivial to do so. This will make it easier to pass room versions into the FrozenEvent constructors. | ||||
* | Add typing to synapse.federation.sender (#6871) | Erik Johnston | 2020-02-07 | 5 | -105/+128 |
| | |||||
* | Pass room_version into `event_from_pdu_json` | Richard van der Hoff | 2020-02-06 | 3 | -58/+46 |
| | | | | It's called from all over the shop, so this one's a bit messy. | ||||
* | pass room version into FederationClient.send_join (#6854) | Richard van der Hoff | 2020-02-06 | 1 | -28/+32 |
| | | | | ... which allows us to sanity-check the create event. | ||||
* | Reduce amount of logging at INFO level. (#6862) | Erik Johnston | 2020-02-06 | 2 | -4/+4 |
| | | | | | | | | A lot of the things we log at INFO are now a bit superfluous, so lets make them DEBUG logs to reduce the amount we log by default. Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com> | ||||
* | Simplify `room_version` handling in `FederationClient.send_invite` | Richard van der Hoff | 2020-02-05 | 1 | -10/+6 |
| | |||||
* | Pass room version object into `FederationClient.get_pdu` | Richard van der Hoff | 2020-02-05 | 1 | -3/+5 |
| | |||||
* | make FederationClient._do_send_invite async | Richard van der Hoff | 2020-02-05 | 1 | -10/+7 |
| | |||||
* | Apply suggestions from code review | Richard van der Hoff | 2020-02-05 | 1 | -2/+4 |
| | | | Co-Authored-By: Erik Johnston <erik@matrix.org> | ||||
* | make FederationClient.send_invite async | Richard van der Hoff | 2020-02-03 | 1 | -6/+5 |
| | |||||
* | make FederationClient.get_missing_events async | Richard van der Hoff | 2020-02-03 | 1 | -20/+20 |
| | |||||
* | make FederationClient._do_send_leave async | Richard van der Hoff | 2020-02-03 | 1 | -4/+3 |
| | |||||
* | make FederationClient.send_leave.send_request async | Richard van der Hoff | 2020-02-03 | 1 | -5/+2 |
| | |||||
* | make FederationClient._do_send_join async | Richard van der Hoff | 2020-02-03 | 1 | -4/+3 |
| | |||||
* | make FederationClient.send_join.send_request async | Richard van der Hoff | 2020-02-03 | 1 | -4/+3 |
| | |||||
* | make FederationClient.make_membership_event.send_request async | Richard van der Hoff | 2020-02-03 | 1 | -4/+3 |
| | |||||
* | make FederationClient._try_destination_list async | Richard van der Hoff | 2020-02-03 | 1 | -10/+26 |
| | |||||
* | make FederationClient.send_leave async | Richard van der Hoff | 2020-02-03 | 1 | -10/+9 |
| | |||||
* | make FederationClient.send_join async | Richard van der Hoff | 2020-02-03 | 1 | -11/+13 |
| | |||||
* | make FederationClient.make_membership_event async | Richard van der Hoff | 2020-02-03 | 1 | -10/+11 |
| | |||||
* | make FederationClient.get_event_auth async | Richard van der Hoff | 2020-02-03 | 1 | -6/+4 |
| | |||||
* | make FederationClient.get_room_state_ids async | Richard van der Hoff | 2020-02-03 | 1 | -4/+5 |
| | |||||
* | make FederationClient.get_pdu async | Richard van der Hoff | 2020-02-03 | 1 | -15/+17 |
| | |||||
* | make FederationClient.backfill async | Richard van der Hoff | 2020-02-03 | 1 | -15/+11 |
| | |||||
* | Fix `room_version` in `on_invite_request` flow (#6827) | Richard van der Hoff | 2020-02-03 | 2 | -7/+10 |
| | | | | I messed this up a bit in #6805, but fortunately we weren't actually doing anything with the room_version so it didn't matter that it was a str not a RoomVersion. | ||||
* | Merge pull request #6806 from matrix-org/rav/redact_changes/3 | Richard van der Hoff | 2020-01-31 | 1 | -3/+1 |
|\ | | | | | Pass room_version into add_hashes_and_signatures | ||||
| * | Pass room_version into create_local_event_from_event_dict | Richard van der Hoff | 2020-01-30 | 1 | -3/+1 |
| | | |||||
* | | s/get_room_version/get_room_version_id/ | Richard van der Hoff | 2020-01-31 | 2 | -13/+13 |
| | | | | | | | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object. | ||||
* | | pass room version into FederationHandler.on_invite_request (#6805) | Richard van der Hoff | 2020-01-30 | 1 | -1/+1 |
|/ | |||||
* | Fix sending server up commands from workers (#6811) | Erik Johnston | 2020-01-30 | 2 | -12/+19 |
| | | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> | ||||
* | Remove unused DeviceRow class (#6800) | Erik Johnston | 2020-01-29 | 1 | -20/+1 |
| | |||||
* | Fix race in federation sender that delayed device updates. (#6799) | Erik Johnston | 2020-01-29 | 1 | -29/+3 |
| | | | | | | | | | | | | We were sending device updates down both the federation stream and device streams. This mean there was a race if the federation sender worker processed the federation stream first, as when the sender checked if there were new device updates the slaved ID generator hadn't been updated with the new stream IDs and so returned nothing. This situation is correctly handled by events/receipts/etc by not sending updates down the federation stream and instead having the federation sender worker listen on the other streams and poke the transaction queues as appropriate. | ||||
* | Add `rooms.room_version` column (#6729) | Erik Johnston | 2020-01-27 | 1 | -17/+33 |
| | | | This is so that we don't have to rely on pulling it out from `current_state_events` table. | ||||
* | Wake up transaction queue when remote server comes back online (#6706) | Erik Johnston | 2020-01-17 | 2 | -3/+34 |
| | | | | | This will be used to retry outbound transactions to a remote server if we think it might have come back up. | ||||
* | Add StateMap type alias (#6715) | Erik Johnston | 2020-01-16 | 1 | -1/+2 |
| | |||||
* | Port synapse.replication.tcp to async/await (#6666) | Erik Johnston | 2020-01-16 | 1 | -1/+3 |
| | | | | | | | | | | * Port synapse.replication.tcp to async/await * Newsfile * Correctly document type of on_<FOO> functions as async * Don't be overenthusiastic with the asyncing.... | ||||
* | Process EDUs in parallel with PDUs. (#6697) | Erik Johnston | 2020-01-14 | 1 | -12/+58 |
| | | | | This means that things like to device messages don't get blocked behind processing PDUs, which can potentially take *ages*. | ||||
* | Merge pull request #6349 from matrix-org/babolivier/msc1802 | Brendan Abolivier | 2019-12-11 | 4 | -28/+140 |
|\ | | | | | Implement v2 APIs for send_join and send_leave | ||||
| * | Merge branch 'develop' into babolivier/msc1802 | Brendan Abolivier | 2019-12-05 | 8 | -26/+34 |
| |\ | |||||
| * | | Lint | Brendan Abolivier | 2019-11-11 | 1 | -3/+2 |
| | | | |||||
| * | | Lint | Brendan Abolivier | 2019-11-11 | 1 | -3/+1 |
| | | | |||||
| * | | Fix prefix for v2/send_leave | Brendan Abolivier | 2019-11-11 | 1 | -0/+2 |
| | | | |||||
| * | | Add server-side support to the v2 API | Brendan Abolivier | 2019-11-11 | 2 | -14/+33 |
| | | | |||||
| * | | Implement v2 API for send_leave | Brendan Abolivier | 2019-11-11 | 2 | -5/+56 |
| | | | |||||
| * | | Add missing yield | Brendan Abolivier | 2019-11-11 | 1 | -1/+1 |
| | | | |||||
| * | | Implement v2 API for send_join | Brendan Abolivier | 2019-11-11 | 2 | -8/+51 |
| | | | |||||
* | | | Move get_state methods into FederationHandler (#6503) | Richard van der Hoff | 2019-12-10 | 1 | -80/+11 |
| | | | | | | | | | | | | | | | This is a non-functional refactor as a precursor to some other work. | ||||
* | | | Refactor get_events_from_store_or_dest to return a dict (#6501) | Richard van der Hoff | 2019-12-10 | 1 | -29/+15 |
| | | | | | | | | | | | | | | | There was a bunch of unnecessary conversion back and forth between dict and list going on here. We can simplify a bunch of the code. | ||||
* | | | Remove fallback for missing /federation/v1/state_ids API (#6488) | Richard van der Hoff | 2019-12-09 | 2 | -96/+17 |
| |/ |/| | | | | | This API was added way back in 0.17.0; the code here is annoying to maintain and entirely redundant. | ||||
* | | Implementation of MSC2314 (#6176) | Amber Brown | 2019-11-28 | 2 | -12/+20 |
| | | |||||
* | | Clean up newline quote marks around the codebase (#6362) | Andrew Morgan | 2019-11-21 | 3 | -5/+5 |
| | | |||||
* | | Replace instance variations of homeserver with correct case/spacing | Andrew Morgan | 2019-11-12 | 4 | -9/+9 |
|/ | |||||
* | Depublish a room from the public rooms list when it is upgraded (#6232) | Andrew Morgan | 2019-11-01 | 1 | -1/+1 |
| | |||||
* | Merge branch 'develop' into cross-signing_federation | Hubert Chathi | 2019-10-31 | 6 | -25/+32 |
|\ | |||||
| * | Update black to 19.10b0 (#6304) | Amber Brown | 2019-11-01 | 1 | -5/+6 |
| | | | | | | * update version of black and also fix the mypy config being overridden | ||||
| * | Remove usage of deprecated logger.warn method from codebase (#6271) | Andrew Morgan | 2019-10-31 | 5 | -20/+26 |
| | | | | | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated. | ||||
* | | rename get_devices_by_remote to get_device_updates_by_remote | Hubert Chathi | 2019-10-30 | 1 | -2/+2 |
| | | |||||
* | | Merge branch 'develop' into cross-signing_federation | Hubert Chathi | 2019-10-30 | 5 | -131/+106 |
|\| | |||||
| * | Fix small typo in comment (#6269) | Andrew Morgan | 2019-10-30 | 1 | -1/+1 |
| | | |||||
| * | Merge pull request #6280 from matrix-org/erikj/receipts_async_await | Erik Johnston | 2019-10-30 | 1 | -1/+3 |
| |\ | | | | | | | Port receipt and read markers to async/wait |