supernets/mastodon
supernets
/
mastodon
4
Fork 0
Code Issues Pull Requests Activity
mastodon/app/services
History
Brian Campbell 2d12948220
Fix idempotency when database writes are slow (#21840) 
There is an idempotency key generated by clients when authoring a post,
and stored in Redis, to ensure that if a user or client retries posting
the same status, we don't get a duplicate.

Hachyderm.io has been experiencing some filesystem and database
performance issues, causing database writes to be slow. This can mean
that there are successful posts, but the reverse proxy returns 504
Gateway Timeout before the idempotency status has been updated; users or
clients who retry (such as Tusky which retries automatically, see
tuskyapp/Tusky#2951) can re-try the same post with the same idempotency
key before it has actually been recorded in Redis, leading to duplicate
posts.

To address this issue, move all of the database updates after the
initial transaction that creates the status into the
`postprocess_status!` method, so we can insert the idempotency key
immediately after the status has been created, significantly reducing
the window in which the status could be created but the idempotency key
not yet stored.

Note: this has not yet been tested; I'm submitting this PR for
discussion and to offer to the Hachyderm.io admins to try out to fix the
multiple posting problem.

Co-authored-by: Brian Campbell <brcampbell@beta.team>
 		2022-12-15 18:08:40 +01:00
..
activitypub Fix unbounded recursion in account discovery (#22025) 2022-12-07 00:15:24 +01:00
concerns Fix deletes not being signed in authorized fetch mode (#17484) 2022-02-11 14:52:45 +01:00
keys Refactor ActivityPub handling to prepare for non-Account actors (#19212) 2022-09-21 22:45:57 +02:00
account_search_service.rb Improve performance by avoiding regex construction (#20215) 2022-11-10 05:49:30 +01:00
account_statuses_cleanup_service.rb Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
after_block_domain_from_account_service.rb Fix account domain block not properly unfollowing accounts from domain (#14304) 2020-07-13 19:33:10 +02:00
after_block_service.rb Fix blocking someone not clearing up list feeds (#16205) 2021-05-10 17:31:55 +02:00
after_unallow_domain_service.rb Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
app_sign_up_service.rb Add ability to block sign-ups from IP (#19037) 2022-08-24 19:00:37 +02:00
appeal_service.rb Add customizable user roles (#18641) 2022-07-05 02:41:40 +02:00
approve_appeal_service.rb Fix moderator leak in undo_mark_statuses_as_sensitive (#18525) 2022-05-26 22:04:16 +02:00
authorize_follow_service.rb Remove Salmon and PubSubHubbub (#11205) 2019-07-06 23:26:16 +02:00
backup_service.rb Add logging of S3-related errors (#16381) 2021-07-21 18:34:39 +02:00
base_service.rb Add administrative webhooks (#18510) 2022-06-09 21:57:36 +02:00
batched_remove_status_service.rb Bump chewy from 5.2.0 to 7.2.3 (supports Elasticsearch 7.x) (#16915) 2021-11-18 22:02:08 +01:00
block_domain_service.rb Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
block_service.rb Fix BlockService trying to reject incorrect follow request (#11288) 2019-07-11 14:50:27 +02:00
bootstrap_timeline_service.rb Add customizable user roles (#18641) 2022-07-05 02:41:40 +02:00
clear_domain_media_service.rb Fix incorrect and slow cache invalidation in ClearDomainMediaService (#19062) 2022-08-27 00:45:54 +02:00
create_featured_tag_service.rb Fix `Settings::FeaturedTagsController` (#19418) 2022-10-22 23:14:58 +02:00
delete_account_service.rb Remove LDSignature on actor Delete activities (#21466) 2022-11-27 20:43:24 +01:00
deliver_to_device_service.rb Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
fan_out_on_write_service.rb Add caching for payload serialization during fan-out (#19642) 2022-11-04 13:21:06 +01:00
favourite_service.rb Fix duplicate notifications being possible after poll expiration (#17697) 2022-03-04 01:06:33 +01:00
fetch_link_card_service.rb Refactor how Redis locks are created (#18400) 2022-05-13 00:02:35 +02:00
fetch_oembed_service.rb Fix some link previews being incorrectly generated from other prior links (#16885) 2021-10-21 20:39:35 +02:00
fetch_remote_status_service.rb Fix an error when ActivityPub::FetchRemoteStatusService url is called with nil (#12652) 2019-12-18 16:54:03 +01:00
fetch_resource_service.rb Refactor ActivityPub handling to prepare for non-Account actors (#19212) 2022-09-21 22:45:57 +02:00
follow_service.rb Add ability to filter followed accounts' posts by language (#19095) 2022-09-20 23:51:21 +02:00
import_service.rb Support UTF-8 Characters in Domains During CSV Import (#20592) 2022-11-14 05:52:13 +01:00
move_service.rb Fix account migration not affecting followers on origin server (#11980) 2019-09-29 16:23:13 +02:00
mute_service.rb Add duration parameter to muting. (#13831) 2020-10-13 01:01:14 +02:00
notify_service.rb Add customizable user roles (#18641) 2022-07-05 02:41:40 +02:00
post_status_service.rb Fix idempotency when database writes are slow (#21840) 2022-12-15 18:08:40 +01:00
precompute_feed_service.rb Fix single Redis connection being used across all threads (#18135) 2022-04-28 17:47:34 +02:00
process_hashtags_service.rb Add editing for published statuses (#17320) 2022-02-10 00:15:30 +01:00
process_mentions_service.rb Change mentions of blocked users to not be processed (#19725) 2022-11-04 13:19:12 +01:00
purge_domain_service.rb Remove custom emojis on domain purge (#17210) 2021-12-30 08:41:09 +01:00
reblog_service.rb Add trending statuses (#17431) 2022-02-25 00:34:14 +01:00
reject_follow_service.rb Remove Salmon and PubSubHubbub (#11205) 2019-07-06 23:26:16 +02:00
remove_featured_tag_service.rb Change featured tag updates to add/remove activity (#19409) 2022-10-22 18:30:55 +02:00
remove_from_followers_service.rb Add remove from followers api (#16864) 2021-10-18 12:02:35 +02:00
remove_status_service.rb Fix n+1 query during status removal (#19753) 2022-11-05 00:09:52 +01:00
report_service.rb Change Report category to "violation" if rule IDs are provided (#20137) 2022-11-08 17:28:02 +01:00
resolve_account_service.rb Fix account migration form ever using outdated account data (#18429) 2022-10-29 01:31:45 +02:00
resolve_url_service.rb Improve performance by avoiding regex construction (#20215) 2022-11-10 05:49:30 +01:00
search_service.rb Optimize some regex matching (#15528) 2021-01-22 10:09:08 +01:00
suspend_account_service.rb Fix remote reporters not receiving suspend/unsuspend activities (#16050) 2021-04-17 14:55:46 +02:00
tag_search_service.rb Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977) 2019-09-28 01:02:21 +02:00
translate_status_service.rb Fix translations not being formatted, other issues in web UI (#19245) 2022-09-28 01:02:01 +02:00
unallow_domain_service.rb Fix removing allowed domains being done synchronously (#14302) 2020-07-15 21:08:19 +02:00
unblock_domain_service.rb Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
unblock_service.rb Remove Salmon and PubSubHubbub (#11205) 2019-07-06 23:26:16 +02:00
unfavourite_service.rb Remove Salmon and PubSubHubbub (#11205) 2019-07-06 23:26:16 +02:00
unfollow_service.rb Fix concurrent unfollowing decrementing follower count more than once (#18527) 2022-05-26 22:14:47 +02:00
unmute_service.rb Fix #231 - Muting 2017-03-02 18:49:32 +01:00
unsuspend_account_service.rb Fix remotely-suspended accounts' toots being merged back into timelines (#16628) 2021-08-20 07:40:33 +01:00
update_account_service.rb Fix being able to spoof link verification (#20217) 2022-11-09 08:24:21 +01:00
update_status_service.rb Fix edits with no actual changes being allowed (#17843) 2022-03-26 00:38:44 +01:00
verify_link_service.rb Handle links with no href in VerifyLinkService (#20741) 2022-11-17 10:59:35 +01:00
vote_service.rb Fix empty votes arbitrarily increasing voters count in polls (#18526) 2022-05-26 22:06:10 +02:00
webhook_service.rb Add administrative webhooks (#18510) 2022-06-09 21:57:36 +02:00