We were unconditionally using the ASCII case-mapping in updateNick(),
for instance.
Introduce downstreamConn.casemap to fix this, and use it everywhere.
No need to re-check that a Web Push subscription is valid every
time a downstream connects. Mobile devices may reconnect pretty
frequently.
Check at most once a day.
This adds a new config option, `logs db`, which enables storing chat
logs in the soju database.
Regular store options, CHATHISTORY options, and SEARCH operations are
supported, like the fs logs backend.
Messages are stored in a new table, Message. In order to track the list
of targets we have messages for in an optimized manner, another database
is used: MessageTarget.
All new requests are backend by indexes so should be fast even with
hundreds of thousands of messages.
A contrib script is provided for migrating existing logs fs chat logs to
the database. It can be run with eg:
go run ./contrib/migrate-logs/ logs/ sqlite3:soju.db
Co-authored-by: Simon Ser <contact@emersion.fr>
This enables support for global service contexts, that are run
independently from a user context.
These contexts are considered to be admin. They only have access
to admin commands, because those are relevant in a global context.
This adds a new field to upstreams, members, which is a casemapped map
of upstream users known to the soju. The upstream users known to soju
are: self, any monitored user, and any user with whom we share a
channel.
The information stored for each upstream user corresponds to the info
that can be returned by a WHO/WHOX command.
We build the upstream user information both incrementally, capturing
information contained in JOIN and AWAY messages; and with the bulk user
information contained in WHO replies we receive.
This lets us build a user cache that can then be used to return
synthetic WHO responses to later WHO requests by downstreams.
This is useful because some networks (eg Libera) heavily throttle WHO
commands, and without this cache, any downstream connecting would send 1
WHO command per channel, so possibly more than a dozen WHO commands,
which soju then forwarded to the upstream as WHO commands.
With this cache most WHO commands can be cached and avoid sending
WHO commands to the upstream.
In order to cache the "flags" field, we synthetize the field from user
info we get from incremental messages: away status (H/G) and bot status
(B). This could result in incorrect values for proprietary user fields.
Support for the server-operator status (*) is also not supported.
Of note is that it is difficult to obtain a user "connected server"
field incrementally, so clients that want to maximize their WHO cache
hit ratio can use WHOX to only request fields they need, and in
particular not include the server field flag.
Co-authored-by: delthas <delthas@dille.cc>
Previously, we would clear webpush targets after any MARKREAD.
Consider the following scenario (ignore any typos, this is crafted by
hand):
<<< @time=2020-01-01T00:00:00Z PRIVMSG #foo :hi mark!
<<< @time=2020-01-02T00:00:00Z PRIVMSG #foo :hi again mark!
>>> MARKREAD #foo timestamp=2020-01-01T00:00:00Z
>>> MARKREAD #foo timestamp=2020-01-02T00:00:00Z
The push target was previously cleared on the first MARKREAD, which
means that the second MARKREAD was never broadcast to Firebase, and all
devices would keep the "hi again mark!" notification indefinitely.
This changes the webpush target map so that we store a timestamp of the
last highlight we sent. We only clear the push target when sending a
MARKREAD that is at or after the last message.
The FS message store truncates message times to the second.
This means that a message sent out as 2020-01-01T00:00:00.123Z could be
sent later as part of a CHATHISTORY batch as 2020-01-01T00:00:00.000Z,
which could cause issues in clients.
One such issue is a client sending a MARKREAD for
2020-01-01T00:00:00.000Z, with another client considering the
2020-01-01T00:00:00.123Z message it has as unread.
This fixes the issue by truncating all message times to the second when
using the FS message store.
Some clients will queue up multiple AUTHENTICATE commands without
waiting for a reply to avoid some roundtrips. However that means
the traffic looks like so:
AUTHENTICATE <mechanism>
AUTHENTICATE <base64 blob containing credentials>
soju will fail the first command, and will behave as if no SASL
authentication was in progress when interpreting the second one.
This means we'll echo back the security-sensitive base64 blob to
the client in the error message, which is definitely not great.
Stop doing that.
