Commit Graph

778 Commits

Author SHA1 Message Date
Matt Jankowski db4119f971 Specs for precompute feed service (#3142)
* Add spec for precompute feed service

* Refactor PrecomputeFeedService

* spec wip
2017-05-19 16:21:52 +02:00
Eugen Rochko 620d0d8029 Account domain blocks (#2381)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation

* Adding account domain blocks that filter notifications and public timelines

* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home

* Add API for listing and creating account domain blocks

* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them

* Filter domains in streaming API

* Update account_domain_block_spec.rb
2017-05-19 01:14:30 +02:00
Yamagishi Kazutoshi 79ef8b3653 Fetch remote image using http.rb (#3114) 2017-05-18 15:43:10 +02:00
Eugen Rochko 1efda1c453 Fix #2572 - Resolve preview cards for remote statuses as well as local ones (#3088) 2017-05-17 00:41:15 +02:00
Shunsuke Michii 0dfffb6dcb Make faster ProcessFeedService (#3080)
* Add index accounts on uri.

* Remove a blank line.
2017-05-16 12:12:29 +02:00
Eugen Rochko d0dd9eb5b5 Feature conversations muting (#3017)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation
2017-05-15 03:04:13 +02:00
beatrix 3dcb5fa28f Revert HTML CW changes (#3020)
* selectively Revert "Fix regressions from #2683 (#2970)"

This reverts commit 72698bc3b4.

* Revert "Handle hashtags in spoiler_texts (partial fix for #699) (#2683)"

This reverts commit e2491680e6.
2017-05-13 04:03:43 +02:00
Eugen Rochko 5abdc77c80 Add conversation model, <ostatus:conversation /> (#3016)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Fix conversation migration

* More spec coverage for status before_create

* Prevent n+1 query when generating Atom with the new conversations

* Improve code style

* Remove redundant local variable
2017-05-12 19:09:21 +02:00
Eugen Rochko 43d754eb42 Fix not rejecting remote URIs when parsing out local IDs (#3012) 2017-05-12 16:47:55 +02:00
Eugen Rochko 58bcd50f7f Fix #1426 - Trim long usernames in public follower/following lists (#2993)
Fix #2221 - Catch OpenSSL exceptions when loading remote avatars/headers/attachments
Don't strip "rel" attribute from <a> tags when sanitizing (microformats)
2017-05-11 23:14:00 +02:00
Eugen Rochko 72698bc3b4 Fix regressions from #2683 (#2970)
* Fix regressions from #2683

Properly format spoiler text HTML, while keeping old logic for blankness intact
Process hashtags and mentions in spoiler text
Format spoiler text for Atom
Change "show more" toggle into a button instead of anchor
Fix style regression on dropdowns for detailed statuses

* Fix lint issue

* Convert spoiler text to plaintext in desktop notifications
2017-05-11 00:28:10 +02:00
Eugen Rochko a231f915a0 Fix #2955 - Send HEAD request ahead of GET when fetching URL previews (#2972) 2017-05-10 23:30:07 +02:00
alpaca-tc c3ef5d5414 Add `account_id DESC` to optimize PrecomputeFeedService (#2967) 2017-05-10 20:33:32 +02:00
R Tucker e2491680e6 Handle hashtags in spoiler_texts (partial fix for #699) (#2683)
* services: scan spoiler_text for hashtags (#699)

* views: link hashtags from spoiler_texts

This covers linking hashtags from within the spoiler
text on the server-generated pages.

* services: fix string concat going into hashtag RE

Cleaner Ruby syntax, may handle immutable strings better
2017-05-10 04:47:25 +02:00
Eugen Rochko 3a38322a54 Add spec for Pubsubhubbub::DistributionWorker. PuSH-deliver public items (#2954)
to all subscribers. IDN-normalize callback URLs for subscriptions on insert.
2017-05-10 02:55:43 +02:00
Matt Jankowski 682507bc3c Specs for pubsub subscribe service (#2951)
* Add spec for pubsubhubbub/subscribe

* Refactor pubsubhubbub/subscribe service
2017-05-09 20:48:30 +02:00
Matt Jankowski 441d6dc734 Spec and refactor for pubsubhubbub/unsubscribe service (#2946)
* Add coverage for pubsub unsubscribe service

* Refactor pubsub unsubscribe service
2017-05-09 19:58:18 +02:00
Matt Jankowski 5bea42412e Services specs for subscribe and unsubscribe (#2928)
* Add specs for unsubscribe service

* Fix non existent methods in unsubscribe service

* Clean up status handling in subscribe service
2017-05-09 00:45:02 +02:00
Matt Jankowski b8ba719f73 Unblock domain service specs/refactor (#2867)
* Add spec for unblock domain service

* Refactor UnblockDomainService
2017-05-07 14:44:28 +02:00
alpaca-tc 74036a2c9d Hotfix convert string from symbol (#2856)
* Convert key to string from symbol

* Prefer :public_send instead of
2017-05-06 23:06:52 +02:00
alpaca-tc a0b1951791 Refactor domain_blocks_controller (#2843)
* Set domain_block by before_action

* Cast value with ActiveRecord::Type

* Batch update
2017-05-06 17:03:34 +02:00
alpaca-tc 59804abc3d Optimize MuteService and AfterBlockService (#2836) 2017-05-06 16:31:07 +02:00
alpaca-tc 1f15a15621 Handling failed http response (#2823) 2017-05-06 12:55:59 +02:00
Matt Jankowski 4e05751346 Move account header and avatar methods to a concern (#2825) 2017-05-05 21:37:02 +02:00
Matt Jankowski 20c37ed0f9 Add specs (and refactor) of FetchRemoteResourceService and SearchService (#2812)
* Coverage for fetch remote resource service

* Refactor fetch remote resource service

* Coverage for search service

* Refactor search service
2017-05-05 17:26:04 +02:00
Eugen Rochko 81584779cb More robust PuSH subscription refreshes (#2799)
* Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron

Fix an issue where / in domain would raise exception in TagManager#normalize_domain

PuSH subscriptions refresh done in a round-robin way to avoid hammering a single
server's hub in sequence. Correct handling of failures/retries through Sidekiq (see
also #2613). Optimize Account#with_followers scope. Also, since subscriptions
are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire
refreshing operation halfway through

Fix #2702 - Correct user agent header on outgoing http requests

* Add test for SubscribeService

* Extract #expiring_accounts into method

* Make mastodon:push:refresh no-op

* Queues are now defined in sidekiq.yml

* Queues are now in sidekiq.yml
2017-05-05 02:23:01 +02:00
alpaca-tc 74c8ca699c Delete records in smaller transaction (#2802) 2017-05-04 23:44:39 +02:00
Yamagishi Kazutoshi e95983f5df Decode IDNA in PreviewCard (#2781) 2017-05-04 15:52:08 +02:00
Eugen Rochko 13c16b4e95 Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses (#2734)
* Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses

If a delete arrives before the original status, cache that information
for 6h, and if the original status arrives in that window, ignore it

* Add test case
2017-05-04 04:34:57 +02:00
Eugen Rochko bafd22ecf4 Fix #2706 - Always respond with 200 to PuSH payloads (#2733)
Fix #2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix #2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
2017-05-03 17:02:18 +02:00
yhirano 8325866c61 Downgrade rubocop 0.48.1 => 0.46.0 (#2628)
* downgrade rubocop 0.48.1 => 0.46.0

* exclude vendor/**/* from rubocop target files

* add frozen_string_literal comment line

* fix percent literal delimited by ( and )

* fix alignment

* remove comment disabling unknown cop
2017-04-30 00:23:45 +02:00
Patrick Figel 197af5de70 Set unknown attachment type when adding domain blocks (#2605)
Follow-up to #2599. When a domain block with `reject_media` is
added or `rake mastodon:media:remove_remote` is invoked, mastodon
deletes the locally cached attachments and avatars but does not
reflect that change in the database, causing the `file` fields to
still have values. This change persists the deletion in the
database and sets the attachment type to unknown.

This also introduces a one-off rake task that sets all attachments
without a local file to the "unknown" type. The upgrade notes for
the next release should contain a post-upgrade step with
`rake mastodon:media:set_unknown`.
2017-04-29 02:44:03 +02:00
Eugen Rochko 2af4f3c4e2 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Eugen Rochko 88725d6ce8 OEmbed support for PreviewCard (#2337)
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix #1681 - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
2017-04-27 14:42:22 +02:00
Matt Jankowski 8857cabca4 Domain block service cleanup (#2490)
* Add coverage for domain block service with silence

* Get rid of warning about find_each and order

* Move domain_block to attr_reader

* Move optional clear_media into silence_accounts method

* Use blocked_domain method to reduce passed vars

* Extract blocked_domain_accounts method to find accounts on the domain

* Extract media_from_blocked_domain method to find relevant attachments

* Separate destruction of account images and account attachments
2017-04-26 20:09:01 +02:00
Keiji, Yoshimi 8fe36654ef attach_media should be in transaction of creation status (#2426) 2017-04-26 03:47:44 +02:00
Eugen Rochko 8b5179d006 Fix #2402 - Add Idempotency-Key header to PostStatusService that prevents (#2419)
duplicates. Web UI regenerates UUID for that header every time the compose
form is changed or successfully submitted

Also, fix Farsi i18n overwriting the English one
2017-04-25 15:04:49 +02:00
178inaba d2159deaf2 Optimize account search (#2421) 2017-04-25 04:44:43 +02:00
Eugen 17c591ffba Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Yamagishi Kazutoshi d670f72830 Show multibyte URI preview card (#2363) 2017-04-23 19:08:59 +02:00
Eugen 9d3be5579a Improve bio compatibility (#2278)
* Fix #1057 (close #1819) - Move HTML-formatted bio from <poco:note /> to <summary type="html" />

* Ensure <poco:note /> is plaintext for remote accounts, also, by stripping out HTML
2017-04-22 03:33:24 +02:00
ik-fib 47a3036ea6 Add Mastodon version to user agent (#2253) 2017-04-21 17:26:23 +02:00
Eugen 1d47910d3b Fix possibility of unrightful webfinger redirect (#2147)
* Fix possibility of unrightful webfinger redirect

* Add more tests for FollowRemoteAccountService
2017-04-19 17:28:35 +02:00
Matt Jankowski 297c11dba2 Language detection refactor (#2099)
* Extract detect_language to separate class

* Use default locale, not just en

* Add spec to confirm that whatlanguage cant identify empty string

* Allow account locale to override default in language detector

* PostStatusService supplies an account to detect language
2017-04-18 22:20:12 +02:00
ik-fib 23ecc681c4 Add "Mastodon" to user agent (#2073)
Remove non-const version string

Freeze option is unnecessary
2017-04-18 16:04:13 +02:00
Tomohiro Suwa 3399dd7a66 Fix nil query_username (#2013) 2017-04-17 19:57:02 +02:00
Keiji, Yoshimi 7a5086729a reduce unneed query when post without attachements. (#1907)
* reduce unneed query when post without attachements.

This commit reduce following query:

```
 MediaAttachment Load (0.9ms)  SELECT "media_attachments".* FROM "media_attachments" WHERE "media_attachments"."status
_id" IS NULL AND 1=0 ORDER BY id asc
```

* fixed for more simple changes.
2017-04-17 04:23:13 +02:00
Eugen 8ebed7fc68 Fix #1957 - WhatLanguage can return null. Fallback to 'en' (#1959) 2017-04-16 23:12:19 +02:00
Eugen e4af4898de Add language detection (#1772)
* Add language detection via WhatLanguage and (de)serialization of it through Atom

* Fix default language in ProcessFeedService

* Re-add newline before 'react-rails' Gem to fix groupings

Fixes Code Climate issue
2017-04-16 20:32:17 +02:00
alpaca-tc 19f63ff801 Check @recipient.user at the first (#1939) 2017-04-16 18:04:05 +02:00
Eugen babbb2135e Fix #1813 - Alleviate extra requests when processing mentions (#1938)
The <link rel="mentioned" /> tag refers to accounts by href. So we were
matching the DB by the url attribute, and falling back to HTTP look-up.
However, GS and Mastodon use profile URLs as URIs, too, and the match
for that was missing. This could potentially alleviate some extra network
requests
2017-04-16 18:01:48 +02:00
Matt Jankowski e0b5a94a4b Clean up check that account needs a webfinger update (#1932) 2017-04-16 16:38:29 +02:00
alpaca-tc 8f8319852c Fixed NoMethodError in UnfollowService (#1918) 2017-04-16 14:55:43 +02:00
Eugen 5d710b1139 Make file attachment on MediaAttachment optional (#1865)
Create MediaAttachment but without actual file download when domain is blocked with reject_media set to true
Clean up old media files when creating a new domain block with reject_media set to true
Return remote_url in media attachments API if local file is not present
Undo domain block action in admin UI
Ability to enable reject_media from admin UI
2017-04-16 12:51:30 +02:00
alpaca-tc f9d7ec8971 ActiveRecord::Relation does not respond to `#id` (#1834) 2017-04-15 13:16:24 +02:00
Matt Jankowski 40fd1de488 Account search service refactor (#1791)
* Begin coverage for account search service

* Coverage for hashtag query

* Coverage for calling local vs remote find based on domain presence

* Spec to check that exact matches are not duped

* Coverage of resolve option

* Coverage for account being provided

* Start to refactor account search service

* Isolate query username and domain methods

* Isolate exact_match method

* Extract methods for local and remote results

* Simplify local vs remote and account isoliation

* Extract methods for local and remote results

* Simplify de-dupe of exact match

* Simplify logic to check for non exact remotes

* Cache some methods

* Remove nil from exact_match from results array

* Return exact matches first

* Use find_remote even with no domain

Account.find_local is just an alias for Account.find_remote(user, nil) - so we
can not bother with the conditional here, and call find_remote directly.
2017-04-15 03:17:07 +02:00
ThibG 31f0bcf804 Refresh webfinger (#1323)
* Refresh local info for remote accounts when webfinger returns new values

It only refreshes account info if one of the URLs or the public-key changes,
in which cases it refreshes the full info, re-downloading the feeds from that
user.

Some special handling should probably be done when the public key changes,
but I have been unable to find any use for it in Mastodon yet.

* Re-fetch remote users we aren't subscribed to.

This might induce performance issues, we might want to only do that for users
we explicitly attempted to subscribe but failed to.

* Refactor changes

* Do not refresh existing remote account details more than once a day

* Avoid re-fetching webfinger info in tests unless otherwise specified
2017-04-15 03:16:05 +02:00
ThibG a9529d3b4b Allow running mastodon on a different domain as the one used for identifying users (#1267)
* Allow running mastodon on a different domain as the one used for identifying users

* Alter documentation of WEB_DOMAIN to make clear it shouldn't be used unless the admin knows what they are doing

* Compare to web_domain instead of local_domain when dealing with feeds/API

* Correctly identify mentions to local accounts

Mentions URLs point to the person's web profile, i.e., the user page served on WEB_DOMAIN.
2017-04-15 02:15:46 +02:00
Matt Jankowski 323671a653 Silence more scope order warnings (#1604) 2017-04-12 15:58:08 +02:00
Matt Jankowski 53850bce93 Remove unused AtomBuilderHelper (#1364)
This was used in the views/atom/user_stream.xml.ruby file, which no longer is
used.
2017-04-11 19:31:22 +02:00
ThibG d19ed18388 Get handle from atom feed's author/email field instead of guessing from URL (#1344)
The goal of this change is to enhance Mastodon's handling of remote domains
for which the APIs reside on a different host (see issue #1032).

Indeed, when a remote user unknown to Mastodon is mentionned, only its profile
URL (e.g. https://social.example.org/users/User) is known, and Mastodon has to
build a @username@domain handle for it. To do so, Mastodon fetches the user's
atom feed (e.g., https://social.example.org/users/User.atom) and uses its
content to get the username part of the handle, and the URL's host part to
build the domain (e.g., @User@social.example.org). This handle is then used
for a Webfinger request.

In the case where example.org serves the Webfinger info for @User@example.org
and all feeds and APIs are hosted at social.example.org, Mastodon will still
build @User@social.example.org and fail at resolving the account's details
through Webfinger.

This patch changes this behaviour by using the author's email address from
the atom feed to build the handle. In Mastodon-generated atom feeds, the
email address is always the handle it expects for federation.
2017-04-09 18:43:48 +02:00
Eugen 982fef811e Fix #1141, fix #1126 - Avatar/profile info fetching (#1215)
* Fix #1141, fix #1126 - Work through UpdateRemoteProfileService for both <feed> and <entry> top-level tags

* Improve code quality, remove line unrelated to fix
2017-04-08 13:26:03 +02:00
Eugen 33849acfa7 Merge pull request #1218 from R0ckweb/patch-2
Fix #1141 on remote follow
2017-04-08 11:37:13 +02:00
Yann GUERN 485d75a805 #1141 on remote follow
The async action is send before persist, account.id not yet generated

Pull queue receive 'nil' so no profile update.
2017-04-08 03:24:35 +02:00
Eugen Rochko b2a7218ab7 Fix #801 - Respect webfinger's canonical response of username/domain 2017-04-08 01:07:42 +02:00
Chad Pytel ad5ddd5e95 Use I18n for media attachment validation errors
These are currently user facing errors, but are not localized. This adds the
ability for these messages to be localized.
2017-04-07 14:23:18 -04:00
Eugen 6d6a429af8 Rewrite Atom generation from stream entries to use Ox instead of Nokogiri (#1124)
* Rewrite Atom generation from stream entries to use Ox instead of Nokogiri::Builder

StreamEntry is now limited to only statuses, which allows some optimization. Removed
extra queries on AccountsController#show. AtomSerializer instead of AtomBuilderHelper
used in AccountsController#show, StreamEntriesController#show, StreamEntryRenderer
and PubSubHubbub::DistributionWorker

PubSubHubbub::DistributionWorker moves n+1 DomainBlock query to PubSubHubbub::DeliveryWorker
instead.

All Salmon slaps that aren't based on StreamEntry still use AtomBuilderHelper and Nokogiri

* All Salmon slaps now use Ox instead of Nokogiri. No touch from status on account
2017-04-07 05:56:56 +02:00
Eugen Rochko 51d7caaf19 Fix wrong pubsub channel on public timelines 2017-04-06 04:03:23 +02:00
Eugen Rochko dbd529109e Fix notifications delivered to wrong pubsub channel, optimized RemoveStatusService,
slightly optimized FanOutOnWriteService again
2017-04-06 02:26:59 +02:00
Eugen Rochko 5442083b3c Split SalmonWorker into smaller parts, move profile updating into another job 2017-04-05 21:43:10 +02:00
Eugen Rochko 5b95be1c42 Replace calls to FeedManager#inline_render and #broadcast 2017-04-05 19:45:18 +02:00
Eugen Rochko 220bc48e8e Only render public payload once in FanOutOnWrite 2017-04-05 14:26:17 +02:00
Eugen Rochko 6fd865c000 Spawn FeedInsertWorker to deliver status into personal feed 2017-04-04 19:21:37 +02:00
Eugen Rochko 82aaedec46 Reduce number of items in feeds, optimize regeneration worker slightly,
make regeneration worker unique, (only schedule/execute once at a time)
2017-04-04 13:58:34 +02:00
Eugen Rochko 8232f76c48 Add check for visibility.nil? even though it can't ever be, to check for race conditions 2017-04-03 22:54:46 +02:00
Eugen Rochko b7c1b12367 Make default admin UI page reports. Add admin UI for creating a domain block 2017-04-03 18:55:06 +02:00
Eugen Rochko d6b965cf08 Fix issue with feed merge-in code as well 2017-04-02 15:58:25 +02:00
Eugen Rochko e809caa0e1 Fix feed regeneration bug 2017-04-02 15:46:31 +02:00
Eugen Rochko d93d6f5124 Fix reworked search 2017-03-31 22:45:56 +02:00
Eugen Rochko d6ed2eb512 Prettier account and stream entry URLs 2017-03-22 19:55:14 +01:00
Eugen Rochko 33fac87e81 Shorter timeout on FetchAtomService 2017-03-22 17:41:52 +01:00
Eugen Rochko 5aa3df017b Fix full-text search query quotation, improve tag search performance with an index,
add ability to open status by URL from search (fix #53)
2017-03-22 17:36:34 +01:00
Eugen Rochko 05cf086766 New API method: /api/v1/search
Returns accounts, statuses, hashtags arrays
2017-03-22 02:32:27 +01:00
Eugen Rochko 2816b1bf8e Federate header images, fix open-uri http->https redirection error 2017-03-18 22:51:20 +01:00
Eugen Rochko ad0d82d3ce Make account search blazing fast and rank followers/followees higher in the results 2017-03-17 20:48:14 +01:00
Eugen Rochko 74ae158c2f Add "direct" visibility level in the backend. Web UI is not yet
adjusted to allow choosing it, yet
2017-03-15 22:55:45 +01:00
Eugen Rochko a5daa806f2 Fix casuality of processing remote mentions such that notifications
about them would be processed only after the entire status is processed
2017-03-13 16:34:15 +01:00
Eugen Rochko c64a1c25c4 Fix #231 - Muting 2017-03-02 18:49:32 +01:00
Kit Redgrave 442fdbfc53 Mute button progress so far. WIP, doesn't entirely work correctly. 2017-03-01 22:31:21 -06:00
Eugen Rochko 175a9b9caa Fix #104 - Style OAuth authorized applications page
Add ability to search accounts by display name
2017-02-27 00:15:00 +01:00
Eugen Rochko 5f511324b6 Add validation of media attachments, clean up mastodon-own exception classes 2017-02-26 23:23:06 +01:00
Eugen Rochko 063432d7e3 Merge branch 'fix_462' of https://github.com/rmhasan/mastodon into rmhasan-fix_462 2017-02-26 23:09:18 +01:00
Eugen Rochko 3e9d794ea5 Add tuning documentation, add <content> tags back to most salmons,
make status pagination headers generation more lax about next page
existing
2017-02-25 03:34:37 +01:00
Eugen Rochko 53ae431867 Skip remote media URLs that don't have a hostname 2017-02-22 19:55:14 +01:00
Eugen Rochko c77a54fe0a Fix #651 - Do not reinsert original status into all followers feeds
upon un-reblogging. Check if the reblog was in the feed in the first
place. It might have been filtered on distribution.
2017-02-22 15:52:47 +01:00
Rakib Hasan 87a6bed9e9 previous commit was creating the status regardless
of mix of video and images in status, just wasn't rendering
the show action. I moved the validation before the status creation
2017-02-19 08:28:33 +00:00
Rakib Hasan 6f9ecd899e revisted fix for #462
Moved validation to services/post_status_service.rb
2017-02-19 08:28:33 +00:00
Eugen Rochko 40a4053732 Disable PuSH for blocked domains 2017-02-14 04:01:37 +01:00
Eugen Rochko 8e08ae5bb9 Add IDs to salmon slaps even if they are transient. Add title, too, mostly
in case of debugging incoming salmons. Add <thr:in-reply-to /> to favourite
salmons because it seems to be required by GS.
2017-02-13 13:30:33 +01:00
Eugen Rochko df63461ff0 Add handler for salmons of undoing favourites 2017-02-12 19:50:18 +01:00
Eugen Rochko 720ff55262 Adding more unit tests. Fixing Salmon slaps XML 2017-02-12 17:30:15 +01:00
Eugen Rochko 446267d1bf Deduplicate delete salmons (send only one per mentioned-account domain) 2017-02-12 17:30:15 +01:00
Eugen Rochko 0518492158 Stop trying to shoehorn all Salmon updates into the poor database-connected
StreamEntry model. Simply render Salmon slaps as they are needed
2017-02-12 01:19:14 +01:00
Eugen Rochko 94d2182717 Resolve issue with rendering authorize/reject Salmon slaps 2017-02-11 19:42:05 +01:00
Eugen Rochko dc851c922e Mentions in private statuses allow mentioned people to see them 2017-02-11 15:10:22 +01:00
Eugen Rochko 00b5731ecb After FollowService, re-fetch remote account asynchronously, do nothing
if account lock info was up to date, otherwise re-do the FollowService
with now updated information
2017-02-11 14:25:01 +01:00
Eugen Rochko e610555e10 Fix processing of incoming authorizations/rejections 2017-02-11 13:55:07 +01:00
Eugen Rochko 514fdfa268 Don't PuSH-resubscribe if already subscribed 2017-02-11 13:48:28 +01:00
Eugen Rochko 149887a0ff Make follow requests federate 2017-02-11 02:58:00 +01:00
Eugen Rochko 6331ed16e5 Fix #614 - extra reply-boolean on statuses to account for cases when replied-to
status is not in the system at time of distribution; fix #607 - reset privacy
settings to defaults when cancelling replies
2017-02-09 20:25:39 +01:00
Eugen Rochko c8252759df Add streaming API channels for local-only statuses 2017-02-06 23:46:14 +01:00
Eugen Rochko 0af3401553 Don't allow people to follow people they blocked without unblocking first 2017-02-05 21:04:22 +01:00
Eugen Rochko 920ba5fc4e Fix #61 - Add list of blocked users to the UI; clean up failed push notifications API
Try to fix Travis CI setup
2017-02-05 19:18:11 +01:00
Eugen Rochko 19b9e1e2c3 Preheat status cache 2017-02-05 17:24:18 +01:00
Eugen Rochko 6cdcac1396 Fix #598 - arrow keys to navigate media; fix #481 - non-exact matches no longer
overshadow requirement for remote-lookup
2017-02-05 03:01:23 +01:00
Eugen Rochko ede04cdb24 Move rendering of JSON payloads for public/hashtag timelines to
FanOutOnWriteService. The only recipient-specific part on them
is reblogged/favourited. But since only newly created statuses
appear on them, it is safe to assume that both attributes would
be false
2017-02-02 00:39:17 +01:00
Eugen Rochko d9ca46b464 Cleaning up format of broadcast real-time messages, removing
redis-backed "mentions" timeline as redundant (given notifications)
2017-02-02 00:03:31 +01:00
Eugen Rochko bf60f2898d Fix #529 - Make hashtag timelines show conversations, fix hashtag loading in the UI 2017-01-31 22:35:38 +01:00
Eugen Rochko 83ccdeb87a Fix tests 2017-01-29 12:25:10 +01:00
Eugen Rochko 3f075c7794 API for apps to register for push notifications 2017-01-29 01:30:32 +01:00
Eugen Rochko 450ad43180 Do not run FetchLinkCardService on local URLs, increase file size limit to 8MB,
fix ProcessFeedService pushing status into distribution if called a second time
while the first is still running (i.e. when a PuSH comes after a Salmon slap),
fix not running escape on spoiler text before emojify
2017-01-27 16:57:23 +01:00
Eugen Rochko 3beb24ad55 Use <summary> to encode content warnings instead 2017-01-25 16:53:30 +01:00
Eugen Rochko 10ffd455a8 Fix #522 - prevent blocked users from reblogging 2017-01-25 01:48:46 +01:00
Eugen Rochko 999cde94a6 Instead of using spoiler boolean and spoiler_text, simply check for non-blank spoiler_text
Federate spoiler_text using warning attribute on <content /> instead of a <category term="spoiler" />
Clean up schema file from accidental development migrations
2017-01-25 01:29:16 +01:00
Eugen f8da0dd490 Merge branch 'master' into master 2017-01-24 21:56:06 +01:00
Eugen Rochko 8a880a3d46 Make blocks create entries and unfollows instantly, but do the clean up
in the background instead. Should fix delay where blocked person
can interact with blocker for a short time before background job
gets processed
2017-01-24 21:40:41 +01:00
blackle bf0f6eb62d Implement a click-to-view spoiler system 2017-01-23 21:07:40 -05:00
Eugen Rochko d00189b55a Domains with reject_media? set to true won't download avatars either 2017-01-23 21:55:29 +01:00
Eugen Rochko 434cf8237e Optional domain block attribute that prevents media attachments from being downloaded 2017-01-23 21:36:08 +01:00
Eugen Rochko cca82bf0a2 Move merging/unmerging of timelines into background. Move blocking into
background as well since it's a computationally expensive
2017-01-23 21:29:34 +01:00
Eugen Rochko f2e08ff568 Remove unneeded block check 2017-01-23 17:40:23 +01:00
Eugen Rochko 6d98a73180 Domain blocks now have varying severity - auto-suspend vs auto-silence 2017-01-23 17:38:38 +01:00
Eugen Rochko aa9c51a34c Fix a couple unhandled exceptions 2017-01-23 13:56:57 +01:00
Effy Elden 8f21f5522f Call uniq on the string version of mb_chars tags 2017-01-21 21:02:42 +11:00
Eugen Rochko 9bd3b11cfb Instead of refusing to create accounts, domain blocks auto-suspend new accounts from that domain 2017-01-20 20:14:02 +01:00
Eugen Rochko 05abd977c1 Fix preview cards layout, do preview card crawling for remote statuses 2017-01-20 18:31:49 +01:00
Eugen Rochko f0de621e76 Fix #463 - Fetch and display previews of URLs using OpenGraph tags 2017-01-20 01:00:14 +01:00
Eugen Rochko e9737c2235 Fix tests, add applications to eager loading/cache for statuses, fix
application website validation, don't link to app website if website isn't set,
also comment out animated boost icon from #464 until it's consistent with non-animated version
2017-01-15 14:01:33 +01:00
Effy Elden d6bc0e8db4 Add tracking of OAuth app that posted a status, extend OAuth apps to have optional website field, add application details to API, show application name and website on detailed status views. Resolves #11 2017-01-15 08:58:50 +11:00
Eugen f63f0c4625 Fix too late return 2017-01-14 02:22:16 +01:00
Eugen Rochko b11fdc3ae3 Migrate from ledermann/rails-settings to rails-settings-cached which allows global settings
with YAML-defined defaults. Add admin page for editing global settings. Add "site_description"
setting that would show as a paragraph on the frontpage
2017-01-12 20:46:24 +01:00
Eugen Rochko 7951e7ffd5 Add ruby version to Gemfile, move devDependencies in package.json to dependencies,
fix bug in process feed service
2017-01-11 15:39:31 +01:00
Eugen Rochko 5c7add2176 Fix #147 - Unreblogging will leave original status in feeds 2017-01-07 15:44:22 +01:00
Eugen Rochko 251b04298e Fix undesired delivering of private toot to remote accounts that follow author 2017-01-05 03:17:23 +01:00
Eugen Rochko 136e18b875 Unblocks also federate 2017-01-02 14:19:02 +01:00
Eugen Rochko c04002b340 Federate blocks with the http://mastodon.social/schema/1.0 verb namespace 2017-01-02 12:17:51 +01:00
Eugen Rochko a302e56f9a Add API for retrieving favourites 2016-12-29 20:33:26 +01:00
Eugen Rochko 8b94d283fb Fix wrong person being notified after nested reblog call, fix favourites leaking private toots in Atom feeds 2016-12-28 13:21:12 +01:00
Eugen Rochko 2146ac91a0 Follow requests send e-mail notifications, but are excluded from notifications API
Better initial state for unlisted/nsfw toggles
2016-12-26 21:52:03 +01:00
Eugen Rochko 7376af90f7 Don't show statuses to blocked users 2016-12-26 19:13:56 +01:00
Eugen Rochko 05b13c38b5 Re-enable Webfinger for locked accounts but don't handle "follow" events
coming in via Salmon.

Currently no way to prevent remote follows, but they will only receive public
and unlisted posts
2016-12-22 23:17:57 +01:00
Eugen Rochko f729cfc881 Private posts mentioning non-followers should not notify them, neither locally nor via Salmon 2016-12-22 23:14:24 +01:00
Eugen Rochko b891a81008 Follow call on locked account creates follow request instead
Reflect "requested" relationship in API and UI
Reflect inability of private posts to be reblogged in the UI
Disable Webfinger for locked accounts
2016-12-22 23:03:57 +01:00
Eugen Rochko 2d2154ba75 Add "locked" flag to accounts, prevent blocked users from following, force-unfollow blocked users 2016-12-22 21:34:19 +01:00
Eugen Rochko 80e02b90e4 Private visibility on statuses prevents non-followers from seeing those
Filters out hidden stream entries from Atom feed
Blocks now generate hidden stream entries, can be used to federate blocks
Private statuses cannot be reblogged (generates generic 422 error for now)
POST /api/v1/statuses now takes visibility=(public|unlisted|private) param instead of unlisted boolean
Statuses JSON now contains visibility=(public|unlisted|private) field
2016-12-21 20:04:13 +01:00
Eugen Rochko 6de079a5af Removing external hub completely, fix #333 fixing digit-only hashtags,
removing web app capability from non-webapp pages
2016-12-18 12:24:37 +01:00
Eugen Rochko 668013265c Restoring old async behaviour of thread resolving as it proved to be more robust 2016-12-12 21:12:19 +01:00
Eugen Rochko e90fcb46e3 Sensitive content federates using the "nsfw" hashtag 2016-12-11 22:49:25 +01:00
Eugen Rochko f90133d2ad Thread resolving no longer needs to be separate from ProcessFeedService,
since that is only ever called in the background
2016-12-11 22:23:11 +01:00
Eugen Rochko 2ef9f36cf2 Improve suspend account service 2016-12-06 18:32:36 +01:00
Eugen Rochko f978b06dd1 Add suspend account functionality to admin UI 2016-12-06 18:22:59 +01:00
Eugen Rochko f406e01fcf Add filters for suspended accounts 2016-12-06 18:03:30 +01:00
Eugen Rochko 2488162733 Adding suspend account service 2016-12-06 17:41:42 +01:00
Eugen Rochko 39cc9fde8a Add account suspension 2016-12-05 22:59:30 +01:00
Eugen Rochko 00e9dac1d3 Automatically block outgoing notifications from silenced accounts to users
who don't follow them
2016-12-04 16:51:49 +01:00
Eugen Rochko 3114e55c7a Fix #323 - self-replies to appear in public timelines again 2016-12-02 14:33:20 +01:00
Eugen Rochko 58b3f4fd67 Fix #329 - avatar errors no longer prevent remote accounts from being saved
(without avatar). Also improved search position of exact matches
2016-12-02 14:14:49 +01:00
Eugen Rochko 14bd46946d Per-status control for unlisted mode, also federation for unlisted mode
Fix #233, fix #268
2016-11-30 21:34:59 +01:00
Eugen 1e99a2bb03 Fix trying to PuSH-publish updates of remote removals 2016-11-29 17:41:47 +01:00
Eugen Rochko 5b076cbafb Fix delete Salmons 2016-11-28 19:44:27 +01:00
Eugen Rochko 4eba76711b Adding backtracing to Salmon/Processing workers 2016-11-28 19:11:36 +01:00
Eugen Rochko 7e5e33df48 Fix Pubsubhubbub::UnsubscribeService 2016-11-28 18:14:49 +01:00
Eugen Rochko 2d2c81765b Adding embedded PuSH server 2016-11-28 13:36:47 +01:00
Eugen Rochko 8a4913fde0 Public and hashtag timelines now exclude reblogs and replies
Fix #289 - don't download avatar unless the URL is http/https
Fix #293 - reblog/reblogged is now boost/boosted
2016-11-26 15:45:35 +01:00
Eugen Rochko 054138797f Fix #288 - Strip first @ from search query, don't search accounts if it begins with # 2016-11-26 15:32:29 +01:00
Eugen Rochko 950312bada Fix #278 - Use mb_chars.downcase on hashtags 2016-11-26 15:24:14 +01:00
Eugen Rochko 2cb3dc5e5a Update hub URL and re-subscribe if hub URL changes 2016-11-26 15:18:21 +01:00
Eugen Rochko 71401659b8 Fix #65 - Options to block notifications from people you don't follow/who don't follow you 2016-11-25 13:13:16 +01:00
Eugen Rochko 4f1d863615 Only distribute statuses to followers who signed in in the last 2 weeks, add rake task for clearing feeds of inactive users 2016-11-24 18:17:58 +01:00
Eugen Rochko 7cee27f517 Fix unfollows not clearing reblogs, fix blocks not clearing reblogs and notifications,
skip ActionCable for follow/unfollow/block events, instead clear UI from
blocked account's posts instantly if block request succeeds. Add forgotten
i18n for sensitive content
2016-11-23 22:57:57 +01:00
Eugen Rochko 0603971894 Adding sensitive marker to statuses in API 2016-11-23 10:46:48 +01:00
Eugen Rochko fc90d38893 Moving some counter queries out of subqueries in the API 2016-11-22 22:59:54 +01:00
Eugen Rochko 45c7ee39b3 Remove unneeded indices, improve error handling in background workers, don't needlessly reload reblogged status, send Devise e-mails asynchronously 2016-11-22 17:32:51 +01:00
Eugen Rochko 7944ed6fe5 Adding option to specify asset server, filter followers query by local accounts
during Fan Out On Write to load less stuff into memory
2016-11-21 22:04:10 +01:00
Eugen Rochko 93577f74e7 Use old rules for mention notifications as for mentions timeline 2016-11-21 10:37:34 +01:00
Eugen Rochko 29b12f9e0a Call NotifyService about remote reblogs 2016-11-21 09:56:01 +01:00
Eugen Rochko da2ef4d676 Adding unified streamable notifications 2016-11-20 19:39:58 +01:00
Eugen Rochko 2c58e7e06a Catch validation errors in ProcessFeedService so that one failing entry wouldn't stop others from going through 2016-11-18 23:19:38 +01:00
Eugen Rochko e4f56fa942 Don't create attachments if remote file cannot be fetched 2016-11-18 23:16:34 +01:00
Eugen Rochko fdc17bea58 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko 7ce4670164 Force utf-8 encoding when processing XML 2016-11-13 19:12:40 +01:00
Eugen Rochko afded319d2 Add limit to search results 2016-11-12 14:49:28 +01:00
Eugen Rochko 09218d4c01 Use full-text search for autosuggestions 2016-11-12 14:36:10 +01:00
Eugen Rochko bf5f55a6bb Fix live status removal from public/hashtag channels 2016-11-10 00:47:47 +01:00
Eugen Rochko aabf884c5f Discard misattributed remote statuses, improve timelines filter 2016-11-10 00:15:49 +01:00
Eugen Rochko c5e03a2e0d Status removal is broadcast to public/hashtag timelines too 2016-11-09 19:16:27 +01:00
Eugen Rochko b6832553ff Adding more logging 2016-11-08 19:37:08 +01:00
Eugen Rochko cff0b03cbb Fix for thread resolve service and process feed service url method 2016-11-08 19:09:22 +01:00
Eugen Rochko 2f21f4cc01 Fix region setting for AWS gem 2016-11-08 18:55:46 +01:00
Eugen Rochko bb4d1eb2e8 Improve feed regeneration 2016-11-08 02:08:32 +01:00
Eugen Rochko 096bfbad96 Fix typo 2016-11-08 01:48:17 +01:00
Eugen Rochko 11cbe49ffc ProcessFeedService refactor 2016-11-08 01:44:51 +01:00
Eugen Rochko dbe00a4156 Improved configuration from ENV, cleaned up timeline filter methods
to be more readable, add extra logging to process feed service
2016-11-07 23:20:52 +01:00
Eugen Rochko 93212bc2c4 Add test for FanOutOnWriteService 2016-11-06 15:56:34 +01:00
Eugen Rochko 7bb28bf780 Improve @mention regex 2016-11-05 22:09:51 +01:00
Eugen Rochko 48b9619439 Adding hashtags 2016-11-05 17:13:14 +01:00
Eugen Rochko 3731230c6d Allow @username@domain/@username in follow form, prevent duplicate accounts
created via remote look-up when domains differ but point to the same resource
2016-11-03 16:57:44 +01:00
Eugen Rochko 4b357ecf98 Fix subtle bugs, new icon button 2016-11-02 22:29:19 +01:00
Eugen Rochko 909d0d5e88 Adding public timeline silencing 2016-10-27 19:33:04 +02:00
Eugen Rochko abb8f5837e Fix public channel 2016-10-23 11:56:04 +02:00
Eugen Rochko 852c82435d Fix method return when rescuing 2016-10-20 18:36:12 +02:00
Eugen Rochko 0895ff414e Fix RemoveStatusService trying to send delete salmons on behalf of remote statuses 2016-10-16 19:14:23 +02:00
Eugen Rochko 91144d46ec Fix duplication of media attachments when a remote status reblogs a local one 2016-10-14 20:15:37 +02:00
Eugen Rochko 244d1307a3 Fix remove status service sending salmons 2016-10-14 20:09:33 +02:00
Eugen Rochko 157f03f8bd No-op for Salmons without body, fail fast if Webfinger does not contain
all required resource links (profile page, salmon, atom feed, magic key)
2016-10-13 13:41:06 +02:00
Eugen Rochko 36e7eeb6b9 Treat dfrn:owner like xmlns:author for Friendica compatibility 2016-10-12 22:55:00 +02:00
Eugen Rochko 64302b3c99 Improve Friendica support (but still not there yet) 2016-10-12 21:07:00 +02:00
Eugen Rochko c257b29d86 Fix up a few exceptions 2016-10-12 19:25:46 +02:00
Eugen Rochko 77efdfa110 Fixing namespaces issue 2016-10-10 19:16:23 +02:00
Eugen Rochko 87b618ab02 Fix namespace parsing in Atom feeds 2016-10-10 18:16:07 +02:00
Eugen Rochko 38ce960ff9 Use Account#find_remote method when possible 2016-10-10 17:30:49 +02:00
Eugen Rochko cfba03bd27 Don't return unsaved status from ProcessFeedService 2016-10-10 16:49:05 +02:00
Eugen Rochko 81065bc06c Adding test for ProcessFeedService 2016-10-10 16:03:38 +02:00
Eugen Rochko 3306a5d524 Improve process feed service 2016-10-10 15:27:39 +02:00
Eugen Rochko 65ae9637d6 PuSH unsubscribe needs correct callback URL 2016-10-10 03:40:08 +02:00
Eugen Rochko 5f737c7228 Fix default assumed object type (note->activity), make stream entry threaded? check aware of orphaned replies 2016-10-10 02:55:30 +02:00
Eugen Rochko 22a8801dbc Adding domain blocks 2016-10-09 14:48:59 +02:00
Eugen Rochko 1f650d327d Adding public timeline 2016-10-07 16:00:11 +02:00
Eugen Rochko d772db4344 Fix #83 - if user cannot be found (or is self) throw error, don't return empty 200
This prevents the undefined profile from being opened in the first place on such an error
2016-10-06 21:33:33 +02:00
Eugen Rochko 15d01a5e08 Better comparison of "local" domain 2016-10-06 16:36:16 +02:00
Eugen Rochko 5cfc9efad3 Update OStatus2 to handle malformed Salmon without raising unexpected exceptions 2016-10-06 14:47:38 +02:00
Eugen Rochko 2f5b205916 Catch Paperclip errors on /api/v1/media, return early from update profile service if XML given is nil 2016-10-06 14:40:15 +02:00
Eugen Rochko 3319473b2c Move PubSubHubbub pinging to a background worker
It can take as much as 0.5s if not longer to complete
2016-10-05 13:50:21 +02:00
Eugen Rochko 2febc6ed65 Fix typo 2016-10-05 13:40:14 +02:00
Eugen Rochko fe77921e47 Catching more exceptions that slipped through, removing AR logging from
production as it's very verbose and not very useful
2016-10-05 13:26:44 +02:00
Eugen Rochko 70e9dd0b5b Blocking will prevent e-mail notifications from blocked user, blocks in UI 2016-10-03 18:49:52 +02:00
Eugen Rochko 7b9a4af311 API for blocking and unblocking 2016-10-03 18:17:06 +02:00
Eugen Rochko 9d59d7b463 Adding a block model and filter mentions from blocked users (fix #60) 2016-10-03 17:12:13 +02:00
Eugen Rochko d74da1a89a Small fixes 2016-10-02 23:46:25 +02:00
Eugen Rochko be86d4e0a3 Preparing feeds for better filtering 2016-10-02 15:28:47 +02:00
Eugen Rochko 927333f4f8 Improve code style 2016-09-29 21:28:21 +02:00
Eugen Rochko e4aebad35a Fix handling of multiple Link headers (that should not be a thing though) 2016-09-27 23:49:12 +02:00
Eugen Rochko 7e14eefc81 Replace logo, fix #57 - delete/unreblog/unfavourite API, fix #45 - app
registration API
2016-09-26 23:56:53 +02:00
Eugen Rochko 210362e665 Improve redirect handling in FetchAtomService 2016-09-26 17:04:05 +02:00
Eugen Rochko c6b0311b86 Fix #54 - Fetch remote accounts by URL from mentions
Fetching atom extracted from FetchRemoteAccountService and FetchRemoteStatusService
into FetchAtomService. Mentions of the constant "http://activityschema.org/collection/public"
skipped as it's not a real URL/user.
2016-09-26 16:44:40 +02:00
Eugen Rochko b1a670af8d Handle remote account mentions a little better by trying a URL lookup in the db 2016-09-22 21:10:36 +02:00
Eugen Rochko 4a670780f0 Fix #51 - prevent duplicate attachments for remote statuses 2016-09-22 20:42:20 +02:00
Eugen Rochko 4bec613897 Fix #24 - Thread resolving for remote statuses
This is a big one, so let me enumerate:

Accounts as well as stream entry pages now contain Link headers that
reference the Atom feed and Webfinger URL for the former and Atom entry
for the latter. So you only need to HEAD those resources to get that
information, no need to download and parse HTML <link>s.

ProcessFeedService will now queue ThreadResolveWorker for each remote
status that it cannot find otherwise. Furthermore, entries are now
processed in reverse order (from bottom to top) in case a newer entry
references a chronologically previous one.

ThreadResolveWorker uses FetchRemoteStatusService to obtain a status
and attach the child status it was queued for to it.

FetchRemoteStatusService looks up the URL, first with a HEAD, tests
if it's an Atom feed, in which case it processes it directly. Next
for Link headers to the Atom feed, in which case that is fetched
and processed. Lastly if it's HTML, it is checked for <link>s to the Atom
feed, and if such is found, that is fetched and processed. The account for
the status is derived from author/name attribute in the XML and the hostname
in the URL (domain). FollowRemoteAccountService and ProcessFeedService
are used.

This means that potentially threads are resolved recursively until a dead-end
is encountered, however it is performed asynchronously over background jobs,
so it should be ok.
2016-09-21 01:50:31 +02:00
Eugen Rochko 608a2bfffc Upgrade to PubSubHubbub 0.4 (removing verify_token) 2016-09-20 02:43:20 +02:00
Eugen Rochko 059ebbf48d Separate PuSH subscriptions from following, add mastodon:push:refresh task,
respect hub.lease_seconds (fix #46)
2016-09-20 00:43:36 +02:00
Eugen Rochko 9863196f7b Small rubocop offences removed 2016-09-18 13:42:24 +02:00
Eugen Rochko adffc7a495 Fix #43 2016-09-18 12:28:49 +02:00
Eugen Rochko 183a23943b Fix how other services used old FollowRemoteAccountService 2016-09-17 17:07:45 +02:00
Eugen Rochko 0e9c1a297a Improved error handling for FollowRemoteService 2016-09-17 17:03:36 +02:00
Eugen Rochko 8c0b19012b Fix #41, debug #42 2016-09-17 16:36:10 +02:00
Eugen Rochko d6a64f45fd Adding a notification stack for error messages 2016-09-12 19:20:55 +02:00
Eugen Rochko 05b0c985b4 Send "delete" Salmons to remote mentioned accounts on status removal
Fixes #27
2016-09-12 18:33:34 +02:00
Eugen Rochko ce29624c6d Fixing image upload limits, allowing webm, merge/unmerge events trigger
timeline reload in UI, other small fixes
2016-09-12 18:22:43 +02:00
Eugen Rochko 0077fc26df Merge and unmerge timelines on follow/unfollow, solves #21, #22 2016-09-10 18:36:48 +02:00
Eugen Rochko 76c40cbfb1 Disallow self-follow 2016-09-10 10:01:08 +02:00
Eugen Rochko 3cc47beb6e Refactored generation of unique tags, URIs and object URLs into own classes,
as well as formatting of content
2016-09-09 20:04:34 +02:00
Eugen Rochko 509c18eb13 Fix local follows, 404 in logs 2016-09-08 02:40:51 +02:00
Eugen Rochko 499beb4484 UI for uploading media attachments (and cancelling them)
Mostly resolves #8, though attachments are still not displayed in public view
2016-09-07 18:21:57 +02:00
Eugen 1efa8e48d1 Fix enclosures not being parsed for reblogged statuses 2016-09-06 12:30:15 +02:00
Eugen Rochko eec0dc46a6 PostStatusService can attach media to status, ProcessFeedService likewise 2016-09-05 18:39:53 +02:00
Eugen Rochko 926eea89b5 RemoveStatusService fleshed out, still doesn't send Salmon slaps though 2016-09-05 01:59:46 +02:00
Eugen Rochko 35dfc0fbcb Fixed more case-sensitivity issues 2016-09-04 21:15:52 +02:00
Eugen Rochko 29996a7e8b Fix wrongful mentions breaking status update 2016-09-04 21:07:29 +02:00
Eugen Rochko 54ea7f5dfe Case-insensitive search by usernames 2016-09-04 21:06:04 +02:00
Eugen Rochko a289c1d52f Handle delete Salmons, todo: clean up timelines 2016-09-04 14:44:16 +02:00
Eugen Rochko 49520d6e62 Adding React.js, Redux, revamping dashboard 2016-08-24 17:56:44 +02:00
Eugen Rochko 6426819b6f Fix tests 2016-08-18 17:22:44 +02:00
Eugen Rochko 6deb9f966e Live timelines using ActionCable 2016-08-18 15:49:51 +02:00
Eugen Rochko 85b00d19b8 Moving Salmon notifications to background processing, fixing mini-profiler
behaviour with Turbolinks enabled, optimizing Rabl for production
2016-03-26 13:42:10 +01:00
Eugen Rochko da4b675aca Fix favourite handling in ProcessInteractionService 2016-03-26 01:34:12 +01:00
Eugen Rochko e24bfbde1a Fixing FanOutOnWriteService, fixing Sidekiq not having enough DB connections
in the pool, adding a throttle of 60rpm per IP, adding mini profiler, adding
admin status to users
2016-03-25 14:12:24 +01:00
Eugen Rochko 8eeec389c1 Add link to github project to footer, move FanOutOnWriteService calls to
DistributionWorker. That isn't the heaviest service, yet, but gotta start
somewhere
2016-03-25 03:22:26 +01:00
Eugen Rochko a08e724476 Fix subscriptions:clear task, refactor feeds, refactor streamable activites
and atom feed generation to some extent, as well as the way mentions are
stored
2016-03-25 02:13:30 +01:00
Eugen Rochko 97d5aba1ec Do not try to send notification e-mails for reblogs if the reblog is not of a
local account
2016-03-24 13:25:33 +01:00
Eugen Rochko 12559b01ea Add pagination by max_id instead of offset/limit 2016-03-22 21:38:47 +01:00
Eugen Rochko f14f462eaf Adding Turbolinks, adding status posting form on homepage 2016-03-21 18:26:47 +01:00
Eugen Rochko 9d55529318 Fix text color in dashboard inputs, sanitize remote status content in UI,
simplify FanOutOnWriteService, add /api/accounts/lookup method
2016-03-21 17:02:16 +01:00
Eugen Rochko 46cbb9c551 Increase note truncation length on account grids, improve FanOutOnWrite a bit,
fix tests (the recorded Salmon fixture expects LOCAL_DOMAIN to be something
specific unfortunately)
2016-03-21 08:44:30 +01:00
Eugen Rochko b640f35621 Writing out more tests, fixed some bugs 2016-03-20 13:03:06 +01:00
Eugen Rochko 7e53efae31 Fix for incoming favourites 2016-03-19 22:46:58 +01:00
Eugen Rochko 2b116131d7 Adding e-mail notifications about mentions, follows, favourites and reblogs. Fixing another mention recording bug 2016-03-19 19:20:07 +01:00
Eugen Rochko d4892ace62 Adding more test stubs 2016-03-19 12:13:47 +01:00
Eugen Rochko 66a2107527 Turns out that replies don't have to have the type comment. Changed handling
of those to depend on presence of <thr:in-reply-to /> instead
2016-03-19 11:59:23 +01:00
Eugen Rochko f97fc9744f Use FanOutOnWriteService AFTER processing mentions 2016-03-19 00:41:29 +01:00
Eugen Rochko 071f64d317 Fix #13, Atom feeds now contain formatted post content 2016-03-19 00:16:04 +01:00
Eugen Rochko 580b91c387 Fix more subtle bugs with first_or_create 2016-03-19 00:02:39 +01:00
Eugen Rochko 48aeacd5f9 Fixed ANOTHER first_or_create bug. Seriously who came up with that API smh 2016-03-18 23:47:02 +01:00
Eugen Rochko 9a7485d034 Fix typo on /api/accounts/:id/statuses, fix potential case-sensitivity issue
when registering incoming status mentions, add Travis CI file
2016-03-18 23:23:19 +01:00
Eugen Rochko af7ae348d7 Include AtomBuilderHelper (which declared url_for_target) to all services 2016-03-16 22:44:04 +01:00
Eugen Rochko d022975444 Fixing small bug with FollowRemoteAccountService usage 2016-03-16 21:20:50 +01:00
Eugen Rochko 086d487145 Fix unfollows 2016-03-16 21:14:39 +01:00
Eugen Rochko 75b3339a99 Respect "delete" verb on remote feeds 2016-03-16 10:46:15 +01:00
Eugen Rochko 1aa477ac2f Customized more doorkeeper views, only logged in users can create oauth apps 2016-03-12 19:46:06 +01:00
Eugen Rochko 447cfef62d Improving feed queries, switching API to doorkeeper authentication 2016-03-11 16:47:36 +01:00
Eugen Rochko 6c4c84b161 Distrubute statuses as a fan-out-on-write system, with optional precomputing 2016-03-08 20:20:45 +01:00
Eugen Rochko ab6696e855 Adding doorkeeper, adding a REST API
POST /api/statuses                  Params: status (text contents), in_reply_to_id (optional)
GET  /api/statuses/:id
POST /api/statuses/:id/reblog

GET  /api/accounts/:id
GET  /api/accounts/:id/following
GET  /api/accounts/:id/followers
POST /api/accounts/:id/follow
POST /api/accounts/:id/unfollow

POST /api/follows                  Params: uri (e.g. user@domain)

OAuth authentication is currently disabled, but the API can be used with HTTP Auth.
2016-03-07 12:42:33 +01:00
Eugen Rochko c605b828b5 Adding routes to follow, unfollow, favourite and reblog (locally known models) 2016-03-06 12:51:55 +01:00
Eugen Rochko 23d08c6749 Changing the use of config constants to the Rails configuration object 2016-02-29 20:06:39 +01:00
Eugen Rochko 0e8f59c16f Refactoring Grape API methods into normal controllers & other things 2016-02-29 19:42:08 +01:00
Eugen Rochko 11ff92c9d7 Adding a test for ReblogService, fixing mentions for remote statuses 2016-02-28 21:22:56 +01:00
Eugen Rochko 47d50b0e39 A lot of fixes from a live test 2016-02-28 15:46:29 +01:00
Eugen Rochko 9ec03ab5f5 Ping hubs when following somebody, too 2016-02-28 14:34:25 +01:00
Eugen Rochko 8fada4fae2 When posting a status or reblogging one, ping hubs about the feed update 2016-02-28 14:33:13 +01:00
Eugen Rochko ad5ae3f60e Update profile information and download avatar of remote accounts 2016-02-28 14:26:26 +01:00
Eugen Rochko f16b31f077 Adding a bunch of tests 2016-02-26 15:28:08 +01:00
Eugen Rochko 44c5958203 Fixed small bugs and code style offences 2016-02-25 00:22:46 +01:00
Eugen Rochko 71fe24096c Adding a Mention model, test stubs 2016-02-25 00:17:01 +01:00
Eugen Rochko 42eeecba3f Service to reblog statuses 2016-02-24 18:50:16 +01:00
Eugen Rochko c967802c43 Notify remote users about mentions 2016-02-24 18:44:03 +01:00
Eugen Rochko 79609d6270 Add service for posting statuses (normal and replies), mention regex to
fetch webfinger information of mentioned accounts
2016-02-24 18:25:04 +01:00
Eugen Rochko 8da8387afe Add shared statuses to the database 2016-02-24 17:23:59 +01:00
Eugen Rochko fa7868675d Send Salmon interactions 2016-02-24 12:57:29 +01:00
Eugen Rochko 10eb47a33e Do not PuSH-subscribe to remote accounts when creating them for salmon 2016-02-24 03:05:40 +01:00
Eugen Rochko 784b9cca6a Forgotten argument fix 2016-02-24 02:15:58 +01:00
Eugen Rochko 9411e8c35a Feed processing except fetching remote statuses 2016-02-24 01:28:53 +01:00
Eugen Rochko 79baf2fd99 Process favourites, reblogs and replies from Salmon 2016-02-24 00:57:47 +01:00
Eugen Rochko ee73d35eea Incoming Salmon requests can be turned into follows and unfollows 2016-02-23 22:17:07 +01:00
Eugen Rochko fa33750105 Adding reblogs, favourites, improving atom generation 2016-02-23 19:17:37 +01:00
Eugen Rochko 3b0bc18db9 Improved Atom rendering templates, added README 2016-02-23 13:08:01 +01:00
Eugen Rochko 2bc48e9064 Individual atom entries 2016-02-22 19:11:07 +01:00
Eugen Rochko 1dad72bf13 Fixes and general progress 2016-02-22 18:10:30 +01:00
Eugen Rochko 709c6685a9 Made some progress 2016-02-22 16:00:20 +01:00
Eugen Rochko 9c4856bdb1 Initial commit 2016-02-20 22:53:20 +01:00