This adds a new field to upstreams, members, which is a casemapped map
of upstream users known to the soju. The upstream users known to soju
are: self, any monitored user, and any user with whom we share a
channel.
The information stored for each upstream user corresponds to the info
that can be returned by a WHO/WHOX command.
We build the upstream user information both incrementally, capturing
information contained in JOIN and AWAY messages; and with the bulk user
information contained in WHO replies we receive.
This lets us build a user cache that can then be used to return
synthetic WHO responses to later WHO requests by downstreams.
This is useful because some networks (eg Libera) heavily throttle WHO
commands, and without this cache, any downstream connecting would send 1
WHO command per channel, so possibly more than a dozen WHO commands,
which soju then forwarded to the upstream as WHO commands.
With this cache most WHO commands can be cached and avoid sending
WHO commands to the upstream.
In order to cache the "flags" field, we synthetize the field from user
info we get from incremental messages: away status (H/G) and bot status
(B). This could result in incorrect values for proprietary user fields.
Support for the server-operator status (*) is also not supported.
Of note is that it is difficult to obtain a user "connected server"
field incrementally, so clients that want to maximize their WHO cache
hit ratio can use WHOX to only request fields they need, and in
particular not include the server field flag.
Co-authored-by: delthas <delthas@dille.cc>
Previously, we would clear webpush targets after any MARKREAD.
Consider the following scenario (ignore any typos, this is crafted by
hand):
<<< @time=2020-01-01T00:00:00Z PRIVMSG #foo :hi mark!
<<< @time=2020-01-02T00:00:00Z PRIVMSG #foo :hi again mark!
>>> MARKREAD #foo timestamp=2020-01-01T00:00:00Z
>>> MARKREAD #foo timestamp=2020-01-02T00:00:00Z
The push target was previously cleared on the first MARKREAD, which
means that the second MARKREAD was never broadcast to Firebase, and all
devices would keep the "hi again mark!" notification indefinitely.
This changes the webpush target map so that we store a timestamp of the
last highlight we sent. We only clear the push target when sending a
MARKREAD that is at or after the last message.
Add support for MONITOR in single-upstream mode.
Each downstream has its own set of monitored targets. These sets
are merged together to compute the MONITOR commands to send to
upstream.
Each upstream has a set of monitored targets accepted by the server
alongside with their status (online/offline). This is used to
directly send replies to downstreams adding a target another
downstream has already added, and send MONITOR S[TATUS] replies.
Co-authored-by: delthas <delthas@dille.cc>
Type-A modes always have an argument[0], but soju doesn't care about
them since it doesn't keep track of mode lists (ban/invite/.. lists).
[0] https://modern.ircdocs.horse/#mode-message
> Type A: Modes that add or remove an address to or from a list. These
> modes MUST always have a parameter when sent from the server to a
> client.
This adds support for WHOX, without bothering about flags and mask2
because Solanum and Ergo [1] don't support it either.
The motivation is to allow clients to reliably query account names.
It's not possible to use WHOX tokens to route replies to the right
client, because RPL_ENDOFWHO doesn't contain it.
[1]: https://github.com/ergochat/ergo/pull/1184
Closes: https://todo.sr.ht/~emersion/soju/135
TL;DR: supports for casemapping, now logs are saved in
casemapped/canonical/tolower form
(eg. in the #channel directory instead of #Channel... or something)
== What is casemapping? ==
see <https://modern.ircdocs.horse/#casemapping-parameter>
== Casemapping and multi-upstream ==
Since each upstream does not necessarily use the same casemapping, and
since casemappings cannot coexist [0],
1. soju must also update the database accordingly to upstreams'
casemapping, otherwise it will end up inconsistent,
2. soju must "normalize" entity names and expose only one casemapping
that is a subset of all supported casemappings (here, ascii).
[0] On some upstreams, "emersion[m]" and "emersion{m}" refer to the same
user (upstreams that advertise rfc1459 for example), while on others
(upstreams that advertise ascii) they don't.
Once upstream's casemapping is known (default to rfc1459), entity names
in map keys are made into casemapped form, for upstreamConn,
upstreamChannel and network.
downstreamConn advertises "CASEMAPPING=ascii", and always casemap map
keys with ascii.
Some functions require the caller to casemap their argument (to avoid
needless calls to casemapping functions).
== Message forwarding and casemapping ==
downstream message handling (joins and parts basically):
When relaying entity names from downstreams to upstreams, soju uses the
upstream casemapping, in order to not get in the way of the user. This
does not brings any issue, as long as soju replies with the ascii
casemapping in mind (solves point 1.).
marshalEntity/marshalUserPrefix:
When relaying entity names from upstreams with non-ascii casemappings,
soju *partially* casemap them: it only change the case of characters
which are not ascii letters. ASCII case is thus kept intact, while
special symbols like []{} are the same every time soju sends them to
downstreams (solves point 2.).
== Casemapping changes ==
Casemapping changes are not fully supported by this patch and will
result in loss of history. This is a limitation of the protocol and
should be solved by the RENAME spec.
Previously we dropped all TAGMSG as well as any client message tag sent
from downstream.
This adds support for properly forwarding TAGMSG and client message tags
from downstreams and upstreams.
TAGMSG messages are intentionally not logged, because they are currently
typically used for +typing, which can generate a lot of traffic and is
only useful for a few seconds after it is sent.
Previously, we only considered channel modes in the modes of a MODE
messages, which means channel membership changes were ignored. This
resulted in bugs where users channel memberships would not be properly
updated and cached with wrong values. Further, mode arguments
representing entities were not properly marshaled.
This adds support for correctly parsing and updating channel memberships
when processing MODE messages. Mode arguments corresponding to channel
memberships updates are now also properly marshaled.
MODE messages can't be easily sent from history because marshaling these
messages require knowing about the upstream available channel types and
channel membership types, which is currently only possible when
connected. For now this is not an issue since we do not send MODE
messages in history.
User channel memberships are actually a set of memberships, not a single
value. This introduces memberships, a type representing a set of
memberships, stored as an array of memberships ordered by descending
rank.
This also adds multi-prefix to the permanent downstream and upstream
capabilities, so that we try to get all possible channel memberships.
