Commit Graph

256 Commits

Author SHA1 Message Date
Eugen Rochko 282ea17078
Add soft delete for statuses for instant deletes through API (#11623)
* Add soft delete for statuses to allow them to appear instant

* Allow reporting soft-deleted statuses and show them in the admin UI

* Change index for getting an account's statuses
2019-08-22 21:55:56 +02:00
Eugen Rochko 96702e7f67
Add `tootctl cache recount` command (#11597) 2019-08-18 14:55:03 +02:00
ThibG 1e7187f2a8 Fix Status.remote scope matching *all* statuses (#11265) 2019-07-08 18:17:22 +02:00
Eugen Rochko 63c7fe8e48
Refactor controllers for statuses, accounts, and more (#11249) 2019-07-08 12:03:45 +02:00
Eugen Rochko b851456139
Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247) 2019-07-07 16:16:51 +02:00
Eugen Rochko 3fd6ab99e6
Remove deprecated REST API `GET /api/v1/timelines/direct` (#11212) 2019-07-05 02:14:56 +02:00
Eugen Rochko 6a9a759f40
Change reblogs counter to be updated when boosted privately (#10964) 2019-06-04 23:11:44 +02:00
Eugen Rochko f2b743e715
Refactor all ActivityPub deliveries to be serialized and signed through one concern (#10966) 2019-06-04 23:11:18 +02:00
ThibG 14f6ce2885 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
Eugen Rochko f1bc90ab50
Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
2019-03-28 04:44:59 +01:00
ThibG 80f0910e21 Add support for custom emojis in poll options (#10322)
* Backend changes for custom emoji support in poll options

* Serialize poll emojis in REST API

* Render custom emojis in poll options

* Render custom emoji in poll options on public pages
2019-03-20 17:29:12 +01:00
ThibG 5e38ef87a7 Fix reblogs privacy (#10302)
* Fix reblogs privacy

* Fix Announce processing specs
2019-03-17 14:54:09 +01:00
Eugen Rochko 3ad3223b46
Fix detailed poll validation errors not being returned in the API (#10261)
No more "Owned poll is invalid"
2019-03-15 13:36:38 +01:00
Eugen Rochko ba84b6d4d7
Add `visibility` param to reblog REST API (#9851)
Use async worker for creating reblog notification to improve performance
2019-03-15 04:36:41 +01:00
Eugen Rochko 0c43c320db
Fix status creation API silently discarding invalid poll (#10171) 2019-03-05 15:21:31 +01:00
ThibG e13d3792f3 Make sure the poll is created before storing its id (#10142)
* Make sure the poll is created before storing its id

* Fix updating poll results

* Support fetching Question activities from the search bar
2019-03-04 00:39:06 +01:00
Eugen Rochko 230a012f00
Add polls (#10111)
* Add polls

Fix #1629

* Add tests

* Fixes

* Change API for creating polls

* Use name instead of content for votes

* Remove poll validation for remote polls

* Add polls to public pages

* When updating the poll, update options just in case they were changed

* Fix public pages showing both poll and other media
2019-03-03 22:18:23 +01:00
Eugen Rochko 31f396b57d
Add support for non-public reblogs from ActivityPub (#9841)
Fix #9838
2019-01-18 15:56:21 +01:00
ThibG 1a22eff1e0 Attempt fixing deadlocks by moving account stats update outside transaction (#9437)
* Use `update_column` instead of `update_attribute` in callback

`update_attribute` would normally cause callbacks to be called.
Called from a callback, it seems to stop further callbacks from executing.

`update_column` does the same work, but without calling callbacks or
preventing other callbacks from executing.

* Fix deadlocks by moving account stats update outside transaction
2018-12-05 22:51:12 +01:00
valerauko db9aea34de Ensure replied-to is a status not a boost (#9129)
* Ensure replied-to is a status not a boost

* Consider case of not a reply

* Add test case for replying to boost

* Move reblog-reply resolution to model

* Remove unnecessary comment
2018-11-25 16:35:21 +01:00
Eugen Rochko d6b9a62e0a
Extract counters from accounts table to account_stats table (#9295) 2018-11-19 00:43:52 +01:00
Eugen Rochko 8069fd636b
Remove intermediary arrays when creating hash maps from results (#9291) 2018-11-16 15:02:18 +01:00
James Kiesel 4c03e05a4e Allow joining several hashtags in a single column (#8904)
* Nascent tag menu on frontend

* Hook up frontend to search

* Tag intersection backend first pass

* Update yarnlock

* WIP

* Fix for tags not searching correctly

* Make radio buttons function

* Simplify radio buttons with modeOption

* Better naming

* Rearrange options

* Add all/any/none functionality on backend

* Small PR cleanup

* Move to service from scope

* Small cleanup, add proper service tests

* Don't use send with user input :D

* Set appropriate column header

* Handle auto updating timeline

* Fix up toggle function

* Use tag value correctly

* A bit more correct to use 'self' rather than 'all' in status scope

* Fix some style issues

* Fix more code style issues

* Style select dropdown more better

* Only use to_id'ed value to ensure no SQL injection

* Revamp frontend to allow for multiple selects

* Update backend / col header to account for more flexible tagging

* Update brakeman ignore

* Codeclimate suggestions

* Fix presenter tag_url

* Implement initial PR feedback

* Handle additional tag streaming

* CodeClimate tweak
2018-11-05 18:53:25 +01:00
Eugen Rochko 11b3ee4f4c
Reset status cache when status_stat or media_attachment updates (#9119)
* Reset status cache when status_stat or media_attachment updates

Fix #8711

Media attachments are generally immutable, but admins can update
the sensitive flag, and this would ensure the change is visible
instantly. Same for updates to status stats. That is a regression
from #8185, because even the correct updated_at fetched from a join
doesn't seem to invalidate the cache.

* Remove join from Status#cache_ids since it has no effect
2018-10-28 06:42:34 +01:00
Eugen Rochko 795f0107d2
Include preview cards in status entity in REST API (#9120)
* Include preview cards in status entity in REST API

* Display preview card in-stream

* Improve in-stream display of preview cards
2018-10-28 06:35:03 +01:00
Eugen Rochko 72d7d3003b
Do not show "limited" visibility in default visibility preference (#8999)
* Do not show "limited" visibility in default visibility preference

Fix regression from #8950

* Fix code style issue
2018-10-17 22:04:40 +02:00
Eugen Rochko ddd30f331c
Improve support for aspects/circles (#8950)
* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification
2018-10-17 17:13:04 +02:00
Eugen Rochko 774ac47373
Add conversations API (#8832)
* Add conversations API

* Add web UI for conversations

* Add test for conversations API

* Add tests for ConversationAccount

* Improve web UI

* Rename ConversationAccount to AccountConversation

* Remove conversations on block and mute

* Change last_status_id to be a denormalization of status_ids

* Add optimistic locking
2018-10-07 23:44:58 +02:00
luzpaz 40dd19be37 Misc. typos (#8694)
Found via `codespell -q 3 --skip="./app/javascript/mastodon/locales,./config/locales"`
2018-09-14 00:53:09 +02:00
masarakki 4bdab203ac exclude-other-silenced-accounts (#7528) 2018-08-22 13:20:50 +02:00
abcang 9e75aa30cd Unuse ActiveRecord::Base#cache_key (#8185)
* Unuse ActiveRecord::Base#cache_key

* Enable cache_versioning

* Call cache_collection
2018-08-19 15:52:38 +02:00
Eugen Rochko d010816ba8
Fix error when trying to update counters for statuses that are gone (#8251) 2018-08-18 03:03:23 +02:00
Eugen Rochko d78474264d
Update reply counters only if the reply is public/unlisted (#8211) 2018-08-16 14:21:52 +02:00
Eugen Rochko 8e111b753a
Move status counters to separate table, count replies (#8104)
* Move status counters to separate table, count replies

* Migration to remove old counter columns from statuses table

* Fix schema file
2018-08-14 19:19:32 +02:00
Eugen Rochko 7eec279c7f
Change language opt-out to language opt-in (#7823)
* Switch filtered_languages to chosen_languages

* Adjust interface

* Remove unused translations
2018-06-17 13:54:02 +02:00
Eugen Rochko 59b42188a7
Filter out blocked/muted people from profile timelines (#7747)
Fix #7741
2018-06-06 21:13:30 +02:00
Eugen Rochko bd0791d800
Add redraft function (#7735)
* Add redraft function

Fix #7010

* Add explicit confirmation

* Add explicit confirmation message
2018-06-05 00:17:38 +02:00
Akihiko Odaki 5bf5003384 Do not mark remote status sensitive even if spoiler text is present (#7395)
Old statuses and statuses from Pawoo, which runs a modified version of
Mastodon, may not have been marked sensitive even if spoiler text is
present.

Such statuses are still not marked sensitve if they are local or
arrived before version upgrade. Marking recently fetched remote status
sensitive contradicts the behavior.

Considering what people expected when they authored such statuses, this
change removes the sensitivity enforcement.
2018-06-04 04:46:14 +02:00
Eugen Rochko a7d726c383
Improve counter caches on Status and Account (#7644)
Do not touch statuses_count on accounts table when mass-destroying
statuses to reduce load when removing accounts, same for
reblogs_count and favourites_count

Do not count statuses with direct visibility in statuses_count

Fix #828
2018-05-30 02:50:23 +02:00
tateisu b87a1229c7 optimize direct timeline (#7614)
* optimize direct timeline

* fix typo in class name

* change filter condition for direct timeline

* fix codestyle issue

* revoke index_accounts_not_silenced because direct timeline does not use it.

* revoke index_accounts_not_silenced because direct timeline does not use it.

* fix rspec test condition.

* fix rspec test condition.

* fix rspec test condition.

* revoke adding column and partial index

* (direct timeline) move merging logic to model

* fix pagination parameter

* add method arguments that switches return array of status or cache_ids

* fix order by

* returns ActiveRecord.Relation in default behavor

* fix codestyle issue
2018-05-28 11:04:06 +02:00
ThibG a24605961a Fixes/do not override timestamps (#7336)
* Revert "Fixes/do not override timestamps (#7331)"

This reverts commit 581a5c9d29.

* Document Snowflake ID corner-case a bit more

Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.

Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.

* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
2018-05-03 23:02:46 +02:00
Eugen Rochko a872392cd9
Add entity cache (#7271)
* Add entity cache

Use a caching layer for mentions and custom emojis that are
dynamically extracted from text.

Reduce duplicate text extractions

* Fix code style issue
2018-04-27 01:38:10 +02:00
Emelia Smith 60b871d56c Implement the ability for instances to define a list of disallowed hashtags (#7176)
The goal here isn't to prevent these hashtags from existing, but just to strongly curtail their usage; The hashtags may still exist in the database via federated status, or from being created prior to this feature.
2018-04-23 23:52:58 +02:00
Yamagishi Kazutoshi 9613a53cb3 Update dependencies for Ruby (2018-04-23) (#7237)
* Update annotate to version 2.7.3

* Update aws-sdk-s3 to version 1.9.2

* Update browser to version 2.5.3

* Update capistrano to version 3.10.2

* Update domain_name to version 0.5.20180417

* Update http to version 3.2.0

* Update lograge to version 0.10.0

* Update oj to version 3.5.1

* Update parallel_tests to version 2.21.3

* Update puma to version 3.11.4

* Update rubocop to version 0.55.0

* Update scss_lint to version 0.57.0

* Update simplecov to version 0.16.1

* Update tty-command to version 0.8.0

* Update tty-prompt to version 0.16.0

* Update pkg-config to version 1.3.0

* Update fog-local to version 0.5.0

* Update fog-openstack to version 0.1.25

* Update devise-two-factor to version 3.0.3

* bundle update
2018-04-23 11:29:17 +02:00
Kaito Sinclaire 156b916caf Direct messages column (#4514)
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL

* Changes to match other timelines in 2.0
2018-04-18 13:09:06 +02:00
Yamagishi Kazutoshi 50529cbceb Upgrade Rails to version 5.2.0 (#5898) 2018-04-12 14:45:17 +02:00
Eugen Rochko cfa9b6e13a
Remove text requirement when media attached from statuses (#6672) 2018-03-07 08:28:52 +01:00
Eugen Rochko 61ed133fea
Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
Eugen Rochko a71af98401
Push discovered status through streaming API within a time window (#6484)
Time window of 6 hours
2018-02-17 14:28:48 +01:00
Eugen Rochko 3ebc0ad4d3
Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
Yamagishi Kazutoshi 238de58e65 Change belongs_to_required_by_default to true (#5888) 2018-01-19 20:56:47 +01:00
Eugen Rochko 38fc1b498d
Add more instance stats APIs (#6125)
* Add GET /api/v1/instance/peers API to reveal known domains

* Add GET /api/v1/instance/activity API

* Make new APIs disableable, exclude private statuses from activity stats

* Fix code style issue

* Fix week timestamps
2017-12-29 19:52:04 +01:00
William Pitcock 32987004c9 status: preserve visibility attribute when reblogging (infoleak fix) (#5789)
this should fix *all* remaining visibility-related mastodon ostatus infoleaks.
thanks to @csaurus@gnusocial.de for pointing out the infoleak.
2017-11-25 01:36:08 +01:00
abcang 269a445c0b Fix unnecessary order (#5807) 2017-11-24 18:41:07 +01:00
Eugen Rochko 24cafd73a2
Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
Daniel Hunsaker cf7e840990 Update model annotations to use BIGINT for IDs (#5461)
All the migrations have been updated to use BIGINTs for ID fields in the DB, but ActiveRecord needs to be told to treat those values as BIGINT as well. This PR does that.
2017-11-12 16:18:50 +01:00
unarist a1c54220e8 Optimize Status#permitted_for 500x (account timeline) (#5373)
The main change of this PR is removing `order by visibility` hack.

This was introduced to force using of `index_statuses_on_account_id` instead of PK index, but it seems no longer needed probably due to `index_statuses_on_account_id_id`. Removing this avoids reading all rows, so really improves first fetching of the user who has lot of statuses.

I have also changed JOIN to IN + subquery, which slightly faster in most cases.
2017-10-13 16:53:43 +02:00
Eugen Rochko b8db386e05 Fix UserTrackingConcern firing on every request, optimize some queries (#5368)
- For some reason, :if option on before_action did not work. It got
  executed every time, returned false, and the action run anyway,
  which led to the current_sign_in_at and sign_in_count being
  updated on every request
- Return "do not filter" early in FeedManager#filter_from_home? if
  the status is authored by receiver. Usually this method is not
  called for own statuses at all, but it is called when Feed#get
  uses the database
- Return early if #reload_stale_associations! has nothing to load
  to save a database query with WHERE 1=0
2017-10-13 16:44:29 +02:00
Lex Alexander b8bae96647 Retoot count increases without reason (#5363)
* Retoot count increases without reason

-The store_uri method for Statuses was being called on after_create and causing reblogs to be incremented twice.
-This calls it when the transaction is finished by using after_create_commit.
-Fixes #4916.

* Added test case for after_create_commit callback for checking reblog count.

* Rewrote test to keep original, but added one for only the after_create_commit callback.
2017-10-13 02:52:09 +02:00
Eugen Rochko 0717d9b3e6 Set snowflake IDs for backdated statuses (#5260)
- Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity
- Skip for statuses coming from inbox, aka delivered in real-time
- Skip for statuses that claim to be from the future
2017-10-08 17:34:34 +02:00
Eugen Rochko 1e02ba111a Add emoji autosuggest (#5053)
* Add emoji autosuggest

Some credit goes to glitch-soc/mastodon#149

* Remove server-side shortcode->unicode conversion

* Insert shortcode when suggestion is custom emoji

* Remove remnant of server-side emojis

* Update style of autosuggestions

* Fix wrong emoji filenames generated in autosuggest item

* Do not lazy load emoji picker, as that no longer works

* Fix custom emoji autosuggest

* Fix multiple "Custom" categories getting added to emoji index, only add once
2017-09-23 14:47:32 +02:00
Eugen Rochko 9c8e602163 Fix custom emojis not detected when used in content warning (#5049) 2017-09-23 01:50:17 +02:00
Akihiko Odaki 0de82dd316 Do not filter statuses with unknown languages (#5045) 2017-09-22 19:33:17 +02:00
Eugen Rochko 81cec35dbf Custom emoji (#4988)
* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted
2017-09-19 02:42:40 +02:00
unarist dd6f9a1b82 Validate uri presence for remote status (#4985) 2017-09-17 15:21:57 +02:00
Akihiko Odaki 198a9a4fa4 Remove local_only scope in Status (#4977) 2017-09-17 05:29:43 +02:00
Eugen Rochko dd6ede554f Fix #4834 - Adjust Status#local and Status#remote scopes (#4839) 2017-09-07 20:18:34 +02:00
Eugen Rochko 11bddd31ce Fix locking migration on statuses table. Nullable column and NO default value (#4825) 2017-09-06 20:57:52 +02:00
Eugen Rochko e7adbf572a Switch to static URIs, new URI format in both protocols for new statuses (#4815)
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
Eugen Rochko 7dc5035031 Make PreviewCard records reuseable between statuses (#4642)
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
Eugen Rochko 4c76402ba1 Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730)
Requires moving Atom rendering from DistributionWorker (where
`stream_entry.status` is already nil) to inline (where
`stream_entry.status.destroyed?` is true) and distributing that.

Unfortunately, such XML renderings can no longer be easily chained
together into one payload of n items.
2017-08-29 16:11:05 +02:00
Eugen Rochko 9caa90025f Pinned statuses (#4675)
* Pinned statuses

* yarn manage:translations
2017-08-25 01:41:18 +02:00
Eugen Rochko e2685ccc81 Fix #4149, fix #1199 - Store emojis as unicode (#4189)
- Use unicode when selecting emoji through picker
- Convert shortcodes to unicode when storing text input server-side
- Do not convert shortcodes in JS anymore
2017-07-14 19:47:53 +02:00
Eugen Rochko 880a5eb25c Fix boolean columns sometimes having a null value (#4162)
* Fix boolean columns sometimes having a null value

* Fix wrong value being set instead of null
2017-07-13 03:12:25 +02:00
Eugen Rochko e48d3bfd01 Fix #1010 - When spoiler text is set, enforce sensitivity too (#4176) 2017-07-13 01:47:23 +02:00
Eugen Rochko 0190aac240 Fix regression from #3842 (#3892)
* Fix regression from #3842

Simplify the query by omitting all direct statuses. Private statuses
are allowed because they are from accounts we are following (so
by definition)

Resolves #3887 (alternative)

* Adjust test
2017-06-22 02:38:50 +02:00
Akihiko Odaki (@fn_aki@pawoo.net) bab5a18232 Filter direct statuses in Status.as_home_timeline (#3842)
The classes using Status.as_home_timeline, namely Feed and
PrecomputeFeedService are expected to filter direct statuses as
FanOutWriteService does, but their filtering were incomplete or missing.

This commit solves the problem by filtering direct statuses in
as_home_timeline as the other similar methods such as as_public_timeline
does.
2017-06-20 20:41:23 +02:00
Matt Jankowski 022008a2a6 Language detection defaults to nil (#3666)
* Default to nil for statuses.language

* Language detection defaults to nil instead of instance UI default
2017-06-09 18:09:37 +02:00
Matt Jankowski 6201f96b8a Introduce StatusThreadingConcern (#3490)
* Add a StatusFilter class to identify visibility of statuses by accounts

* Extract StatusThreadingConcern from Status

* Clarify purpose of checking for nil account
2017-06-05 16:07:44 +02:00
Matt Jankowski b25e42a77f Misc tidying and clean ups (#3445)
* Remove trailing whitespace in i18n mailers

* Use query methods instead of #present? on AR attributes

* Delegate Status#account_domain method

* Delegate Mention #account_username and #account_acct methods
2017-05-31 20:38:17 +02:00
Jack Jennings 3a2003ba86 Extract authorization policy for viewing statuses (#3150) 2017-05-29 18:22:22 +02:00
Akihiko Odaki 8fd174298d Cover AccountsController more in spec (#3229)
* Introduce recent scope to Status and StreamEntry

Introduce recent scope to Status and StreamEntry as Account has.

* Cover AccountsController more in AccountsController
2017-05-23 02:53:01 +02:00
Matt Jankowski 8f4b7c1820 Filter languages with opt out (#3175)
* Remove allowed_languages and add filtered_languages

* Use filtered_languages instead of allowed_languages
2017-05-20 17:32:44 +02:00
beatrix 724fc3cbdf guard against empty domain block list in status scope (#3161) 2017-05-20 13:38:13 +02:00
Eugen Rochko f8ee136c29 Fix federated timeline excluding local toots when any domain blocks are set (#3151) 2017-05-19 21:19:41 +02:00
Eugen Rochko f1ab70649b Add buttons to block and unblock domain (#3127)
* Add buttons to block and unblock domain

* Relationship API now returns "domain_blocking" status for accounts,
rename "block entire domain" to "hide entire domain", fix unblocking domain,
do not block notifications from domain-blocked-but-followed people, do
not send Salmons to domain blocked users

* Add test

* Personal domain blocks shouldn't affect Salmon after all, since in this
direction of communication the control is very thin when it comes to
public stuff. Best stay consistent and not affect federation in this way

* Ignore followers and follow request from domain blocked folks,
ensure account domain blocks are not created for empty domain,
and avoid duplicates in validation

* Purge followers when blocking domain (without soft-blocks, since they
are useless here)

* Add tests, fix local timeline being empty when having any domain blocks
2017-05-19 21:05:32 +02:00
Matt Jankowski 6e4c7d6211 Conditional validations no longer accept strings for if/unless (#3124) 2017-05-19 03:11:23 +02:00
Eugen Rochko 620d0d8029 Account domain blocks (#2381)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation

* Adding account domain blocks that filter notifications and public timelines

* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home

* Add API for listing and creating account domain blocks

* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them

* Filter domains in streaming API

* Update account_domain_block_spec.rb
2017-05-19 01:14:30 +02:00
alpaca-tc 59ceeae8ea Refactor Status#ancestors/descendants (#3092) 2017-05-17 15:38:16 +02:00
alpaca-tc a2a2af244c Optimize Status#permitted_for 24x (#3069)
* Build query with arel node

* Add spec for current Status#permitted_for implementation

* Refactor status.rb

* Order by visibility to optimize query
2017-05-16 02:54:17 +02:00
Eugen Rochko af706583bd Fix change of status callbacks not setting in_reply_to_account_id and (#3072)
possibly others when expected. Add some tests for it
2017-05-15 21:20:55 +02:00
Eugen Rochko d0dd9eb5b5 Feature conversations muting (#3017)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation
2017-05-15 03:04:13 +02:00
alpaca-tc e9810cbad6 Fixes NoMethodError: undefined method 'first' for nil:NilClass (#3036) 2017-05-13 15:56:12 +02:00
Eugen Rochko 5abdc77c80 Add conversation model, <ostatus:conversation /> (#3016)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Fix conversation migration

* More spec coverage for status before_create

* Prevent n+1 query when generating Atom with the new conversations

* Improve code style

* Remove redundant local variable
2017-05-12 19:09:21 +02:00
yhirano 298796cc7b annotate models (#2697)
* add annotate to Gemfile

* rails g annotate:install

* configure annotate_models

* add schema info to models

* fix rubocop to add frozen_string_literal
2017-05-02 02:14:47 +02:00
Matt Jankowski f025cc6782 Filter on allowed user language preferences (#2361)
* Naive approached to timeline filtering

* Convert allowed_languages into a db column

* Allow users to choose languages to see statuses in

* Style list items as two columns

* Add a hint to explain language filtering preference
2017-05-01 17:42:13 +02:00
Matt Jankowski effb08edbb More status specs (#2564)
* Add rough outline of coverage needed for public timeline

* Specs for visibility, replies, boosts

* Specs for silenced account

* Specs for local_only option

* Specs for blocks and mutes

* Add tentative spec around including other silenced account statuses

* Add with_public_visibility scope

* Add simple coverage for tag_timeline

* Tag timeline includes replies

* Replace tag.statuses with a tagged_with scope in tag timeline method

* Use with_public_visibility in tag timeline

* Extract common scope between public and tag timelines to method

* Extract local domain check to local_only scope

* Extract local_only check to starting scope method

* Move list of excluded from timeline account ids to account model

* Simplify excluded accounts list on account model

* Only join accounts when needed

* Rename method for account specific filtering

* Extract method for account exclusions

* Fix bug where silenced accounts were not including statuses from other silenced accounts

* DRY up filter application from account or no account

* timeline_scope can be private

* Add spec showing that account can find its excluded accounts ids

* Add spec which fails if local_only does not have a left outer join

* rubocop
2017-04-28 15:10:41 +02:00
Eugen Rochko d4fedf84e0 Cache Status#ancestors recursive query results (#2527) 2017-04-27 17:07:15 +02:00
Tomohiro Suwa 1f805a6377 Don't need to include ActiveModel::Validations (#2521) 2017-04-27 12:02:55 +02:00
Evan Minto 66fd8e7821 ActivityPub: Add basic, read-only support for Outboxes, Notes, and Create/Announce Activities (#2197)
* Clean up collapsible components

* Expose user Outboxes and AS2 representations of statuses

* Save work thus far.

* Fix bad merge.

* Save my work

* Clean up pagination.

* First test working.

* Add tests.

* Add Forbidden error template.

* Revert yarn.lock changes.

* Fix code style deviations and use localized instead of hardcoded English text.
2017-04-23 05:21:10 +02:00