Commit Graph

328 Commits

Author SHA1 Message Date
Simon Ser
31f2d28508 Introduce the soju.im/bouncer-networks-notify capability 2021-05-25 16:42:51 +02:00
Simon Ser
29ad541ac7 Send network settings in LISTNETWORKS 2021-05-25 16:42:51 +02:00
Simon Ser
60c566e721 Add pass to bouncer network attributes 2021-05-25 16:42:51 +02:00
Simon Ser
db0f745193 Implement the soju.im/bouncer-networks extension 2021-05-25 16:42:51 +02:00
Philip K
b3bc9614f8 Directly return self-messages to user in multi-upstream mode 2021-05-24 13:31:24 +02:00
Simon Ser
68463d3e8b Pass-through the BOT ISUPPORT token
References: https://github.com/ircv3/ircv3-specifications/pull/439
2021-05-24 11:41:04 +02:00
Hubert Hirtz
3b41c87a41 Fix CAP LIST listing disabled capabilities 2021-05-21 10:03:39 +02:00
Simon Ser
ec26117c08 Relay self-WHO/WHOIS in single-upstream mode
In multi-upstream mode, we can't relay WHO/WHOIS messages for the
current user, because we can't decide which upstream server the
message should be relayed to.

In single-upstream server, we do know which upstream server to use,
so we can just blindly relay the message.

This allows users to send a self-WHO/WHOIS to check their cloak and
other information.
2021-05-20 11:13:14 +02:00
Hubert Hirtz
b078ccaf7a Implement CHATHISTORY BETWEEN 2021-05-18 10:44:10 +02:00
Simon Ser
bede274f32 Add more context to chathistory errors 2021-05-11 12:42:12 +02:00
Hubert Hirtz
9e04b3899b Don't directly reply to network-specific NICK
The NICK must only apply to the specific network, not to the downstream
connection.
2021-04-30 12:17:23 +02:00
Hubert Hirtz
e84fad3eda Handle casemapping on BouncerServ 2021-04-30 12:10:49 +02:00
Simon Ser
a2c207d357 Relay detached channel backlog as BouncerServ NOTICE if necessary
Instead of ignoring detached channels wehn replaying backlog,
process them as usual and relay messages as BouncerServ NOTICEs
if necessary. Advance the delivery receipts as if the channel was
attached.

Closes: https://todo.sr.ht/~emersion/soju/98
2021-04-13 19:11:05 +02:00
Simon Ser
45e2c0023a Skip backlog logic in downstreamConn.welcome on chathistory 2021-04-13 17:50:03 +02:00
Simon Ser
65c58adbd9 Take msg ID in sendTargetBacklog 2021-04-13 17:49:37 +02:00
Simon Ser
5b4469fcb7 Use BARE for internal message IDs
This allows to have shorter and more future-proof IDs. This also
guarantees the IDs will only use reasonable ASCII characters (no
spaces), removing the need to encode them for PING/PONG tokens.
2021-03-31 17:57:24 +02:00
Simon Ser
ecf35187fa Make NickServ detection casemapping-aware 2021-03-30 12:28:45 +02:00
Simon Ser
3237bde9f3 Introduce deliveredStore
This hides the double-map complexity behind a dedicated type.
2021-03-29 17:49:50 +02:00
Simon Ser
07519da768 Ensure targets are case-mapped before being passed to messageStore
messageStore isn't aware of the network's case-mapping. We need
to canonicalize the names before passing them to messageStore.
2021-03-29 17:07:39 +02:00
Simon Ser
5a899abaab Simplify network.offlineClients
Replace it with a list of all clients (online or offline).
2021-03-29 16:55:57 +02:00
Simon Ser
6e5a307dc7 Introduce deliveredClientMap
Adds more semantics to map[string]string. Simplifies the complicated
mapStringStringCasemapMap type.
2021-03-26 11:21:14 +01:00
Hubert Hirtz
5014673aae Fix CHATHISTORY target not being casemapped 2021-03-26 10:39:52 +01:00
Hubert Hirtz
bdd0c7bc06
Implement casemapping
TL;DR: supports for casemapping, now logs are saved in
casemapped/canonical/tolower form
(eg. in the #channel directory instead of #Channel... or something)

== What is casemapping? ==

see <https://modern.ircdocs.horse/#casemapping-parameter>

== Casemapping and multi-upstream ==

Since each upstream does not necessarily use the same casemapping, and
since casemappings cannot coexist [0],

1. soju must also update the database accordingly to upstreams'
   casemapping, otherwise it will end up inconsistent,
2. soju must "normalize" entity names and expose only one casemapping
   that is a subset of all supported casemappings (here, ascii).

[0] On some upstreams, "emersion[m]" and "emersion{m}" refer to the same
user (upstreams that advertise rfc1459 for example), while on others
(upstreams that advertise ascii) they don't.

Once upstream's casemapping is known (default to rfc1459), entity names
in map keys are made into casemapped form, for upstreamConn,
upstreamChannel and network.

downstreamConn advertises "CASEMAPPING=ascii", and always casemap map
keys with ascii.

Some functions require the caller to casemap their argument (to avoid
needless calls to casemapping functions).

== Message forwarding and casemapping ==

downstream message handling (joins and parts basically):
When relaying entity names from downstreams to upstreams, soju uses the
upstream casemapping, in order to not get in the way of the user.  This
does not brings any issue, as long as soju replies with the ascii
casemapping in mind (solves point 1.).

marshalEntity/marshalUserPrefix:
When relaying entity names from upstreams with non-ascii casemappings,
soju *partially* casemap them: it only change the case of characters
which are not ascii letters.  ASCII case is thus kept intact, while
special symbols like []{} are the same every time soju sends them to
downstreams (solves point 2.).

== Casemapping changes ==

Casemapping changes are not fully supported by this patch and will
result in loss of history.  This is a limitation of the protocol and
should be solved by the RENAME spec.
2021-03-24 18:15:52 +01:00
Simon Ser
26c5c11caf Improve ERR_NOSUCHCHANNEL error messages
References: https://todo.sr.ht/~emersion/soju/63
2021-03-16 09:13:46 +01:00
Simon Ser
fa047123b9 Passthrough some ISUPPORT tokens 2021-03-15 23:41:37 +01:00
Simon Ser
62d4bf2813 Use upstream ISUPPORT map for NETWORK 2021-03-15 23:08:19 +01:00
Hubert Hirtz
1645371276 Send correct CHATHISTORY error messages 2021-03-05 09:53:59 +01:00
Simon Ser
26473ed60d Introduce downstreamConn.sendTargetBacklog 2021-02-10 13:48:41 +01:00
Simon Ser
7e39f6d663 Rename network.history to network.delivered
"History" is over-loaded with e.g. CHATHISTORY support.
2021-02-10 11:31:34 +01:00
Simon Ser
c14118f7f9 Rename sendNetworkHistory to sendNetworkBacklog
"History" is a little bit over-loaded with CHATHISTORY support.
2021-02-10 10:23:51 +01:00
Hubert Hirtz
5aa15d5628 Request invite-notify to upstreams
... and do not forward INVITEs to downstreams that do not support the
capability.

The downstream capability can be permanent because there is no way for a
client to get the list of people invited to a channel, thus no state can
be corrupted.
2021-01-31 22:18:51 +01:00
Simon Ser
62f1207437 Forward ISUPPORT NETWORK token 2021-01-22 12:00:38 +01:00
Simon Ser
c4d9e6822d Send RPL_ISUPPORT CHATHISTORY token 2021-01-22 11:55:06 +01:00
Simon Ser
ac3431ef76
Make chat history operations optional in messageStore
Some stores may want not to implement chat history operations.
2021-01-04 17:17:35 +01:00
Simon Ser
83a4590acc
Add store-agnostic message ID format
Allow to query the network ID and entity from the message ID regardless
of the underlying store used.
2021-01-04 16:26:30 +01:00
Hubert Hirtz
943182de2f
Improve dc.authenticate()'s error messages 2020-12-25 13:37:15 +01:00
Hubert Hirtz
7bfa4dafef
Advertise all caps, CAP DEL them on registration
... so that the JOIN/history batch takes into account all capabilities.
Without this commit for example, enabling multi-prefix after the batch
makes the client send NAMES requests for all channels, which generate
needless traffic.
2020-12-25 13:35:20 +01:00
delthas
a76b22bf29 Add customizable auto-detaching, auto-reattaching, relaying.
This uses the fields added previously to the Channel struct to implement
the actual detaching/reattaching/relaying logic.

The `FilterDefault` values of the messages filters are currently
hardcoded.

The values of the message filters are not currently user-settable.

This introduces a new user event, eventChannelDetach, which stores an
upstreamConn (which might become invalid at the time of processing), and
a channel name, used for auto-detaching. Every time the channel detach
timer is refreshed (by receveing a message, etc.), a new timer is
created on the upstreamChannel, which will dispatch this event after the
duration (and discards the previous timer, if any).
2020-12-14 20:54:02 +01:00
Hubert Hirtz
cab0fc2b7d
Uphold echo-message even with BouncerServ
Fixes <https://todo.sr.ht/~emersion/soju/74>
2020-11-24 14:25:19 +01:00
Simon Ser
473a0f018b
Fix nickname in ERR_ERRONEOUSNICKNAME 2020-11-24 14:22:39 +01:00
Hubert Hirtz
16c68b21b5
Prevent downstreams from changing their nick to service's
This commit prevents downstream from sending those commands:
- NICK BouncerServ
- NICK BouncerServ/<network>

The later is necessary because soju would otherwise save the nick change
and, in the event that the downstream connects in single-upstream mode
to <network>, it will end up with the nickname "BouncerServ".
2020-11-24 14:22:18 +01:00
Simon Ser
e797d90c59
Implement delivery receipts via PING messages
This patch implements basic message delivery receipts via PING and PONG.

When a PRIVMSG or NOTICE message is sent, a PING message with a token is
also sent. The history cursor isn't immediately advanced, instead the
bouncer will wait for a PONG message before doing so.

Self-messages trigger a PING for simplicity's sake. We can't immediately
advance the history cursor in this case, because a prior message might
still have an outstanding PING.

Future work may include optimizations such as removing the need to send
a PING after a self-message, or groupping multiple PING messages
together.

Closes: https://todo.sr.ht/~emersion/soju/11
2020-11-24 14:13:24 +01:00
Hubert Hirtz
e4d2ddb377
Don't send TAGMSG to upstreams that don't support it
TAGMSG are (in current specs and drafts from IRCv3) only used for
client tags. These are optional information by design (since they are
not distributed to all users), therefore it is preferable to discard
them accordingly to upstream, instead of waiting for all upstreams to
support the capability to advertise it.
2020-11-20 11:37:43 +01:00
Simon Ser
05aafb5edf
Add message store abstraction
Introduce a messageStore type, which will allow for multiple
implementations (e.g. in the DB or in-memory instead of on-disk).

The message store is per-user so that we don't need to deal with locking
and it's easier to implement per-user limits.
2020-10-25 17:47:38 +01:00
Simon Ser
fa16337d97
Switch DB API to user IDs
This commit changes the Network schema to use user IDs instead of
usernames. While at it, a new UNIQUE(user, name) constraint ensures
there is no conflict with custom network names.

Closes: https://todo.sr.ht/~emersion/soju/86
References: https://todo.sr.ht/~emersion/soju/29
2020-10-24 15:14:23 +02:00
delthas
28cf1147e8 Add support for the extended-join capability
This simple implementation only advertises extended-join to downstreams
when all upstreams support it.

In the future, it could be modified so that soju buffers incoming
upstream JOINs, sends a WHO, waits for the reply, and sends an extended
join to the downstream; so that soju could advertise that capability
even when some or all upstreams do not support it. This is not the case
in this commit.
2020-09-11 00:10:58 +02:00
Simon Ser
480d771a67
Fix panic in downstreamConn.sendNetworkHistory
This panic happens when sending history to a multi-upstream client.
sendNetworkHistory is called on each network, but dc.network is nil.

Closes: https://todo.sr.ht/~emersion/soju/93
2020-08-26 15:28:10 +02:00
Simon Ser
43aa3e5529
Fix downstream PING argument handling
The PONG message should have these arguments:

- Our server name
- The PING message's source name

Closes: https://todo.sr.ht/~emersion/soju/92
2020-08-26 15:18:57 +02:00
Simon Ser
fb8c6340c8
Allow '/' in nickname
This allows to specify a network name in the nickname.

Closes: https://todo.sr.ht/~emersion/soju/91
2020-08-25 11:49:22 +02:00
Simon Ser
92fece5cd4
Nuke in-memory ring buffer
Instead, always read chat history from logs. Unify the implicit chat
history (pushing history to clients) and explicit chat history
(via the CHATHISTORY command).

Instead of keeping track of ring buffer cursors for each client, use
message IDs.

If necessary, the ring buffer could be re-introduced behind a
common MessageStore interface (could be useful when on-disk logs are
disabled).

References: https://todo.sr.ht/~emersion/soju/80
2020-08-20 20:05:01 +02:00
Simon Ser
4dae0da59f
Replace networkHistory.offlineClients with clients
Keep the ring buffer alive even if all clients are connected. Keep the
ID of the latest delivered message even for online clients.

As-is, this is a net downgrade: memory usage increases because ring
buffers aren't free'd anymore. However upcoming commits will replace the
ring buffer with log files. This change makes reading from log files
easier.
2020-08-20 17:38:57 +02:00
Hubert Hirtz
e740d952ad
Reject downstream NICK with illegal characters
This should avoid confusion when mixing up nickname and user name.
Also it avoid breaking downstreams (since '@' and '!' are used for host
masks).
2020-08-20 10:00:58 +02:00
Hubert Hirtz
a27e5ea92e
More explicit error message on INVITE with the wrong network 2020-08-20 09:13:38 +02:00
Hubert Hirtz
a636b92a95
More explicit error message on KICK with the wrong network 2020-08-19 23:57:25 +02:00
Simon Ser
bdb132ad98
Implement rate limiting for upstream messages
Allow up to 10 outgoing messages in a burst, then throttle to 1 message
each 2 seconds.

Closes: https://todo.sr.ht/~emersion/soju/87
2020-08-19 19:42:33 +02:00
Simon Ser
745b3f67a0
Extract history loading into functions
These will get re-used for sending history to clients that don't support
the chathistory extension.
2020-08-11 15:58:50 +02:00
Simon Ser
cd3eacdbfc
go fmt 2020-07-22 12:16:01 +02:00
Simon Ser
dcfe206bda
Implement CHATHISTORY AFTER
References: https://todo.sr.ht/~emersion/soju/12
2020-07-15 17:47:57 +02:00
Simon Ser
1685ba23b3
Strip network name from nickname when auto-saving network 2020-07-06 18:13:40 +02:00
Simon Ser
3397965dea
Add RemoteAddr to ircConn interface 2020-07-01 17:02:37 +02:00
Simon Ser
90250f5be5
Reply to WHO BouncerServ
Closes: https://todo.sr.ht/~emersion/soju/75
2020-06-29 18:09:48 +02:00
Hubert Hirtz
cfb1de044e
Don't save corrupted NickServ credentials
soju saved most NickServ messages[0] as credentials because of a missing
`default` clause in the check of the NickServ command.

[0] messages that had at least a command and two other parameters
2020-06-24 23:02:46 +02:00
Simon Ser
d0cf1d2882
Add support for WebSocket connections
WebSocket connections allow web-based clients to connect to IRC. This
commit implements the WebSocket sub-protocol as specified by the pending
IRCv3 proposal [1].

WebSocket listeners can now be set up via a "wss" protocol in the
`listen` directive. The new `http-origin` directive allows the CORS
allowed origins to be configured.

[1]: https://github.com/ircv3/ircv3-specifications/pull/342
2020-06-07 14:13:46 +02:00
Simon Ser
4b3469335e
Fail auth on empty password in DB 2020-06-06 12:52:22 +02:00
delthas
f7894e612b Add support for downstream CHATHISTORY
This adds support for the WIP (at the time of this commit)
draft/chathistory extension, based on the draft at [1] and the
additional comments at [2].

This gets the history by parsing the chat logs, and is therefore only
enabled when the logs are enabled and the log path is configured.

Getting the history only from the logs adds some restrictions:
- we cannot get history by msgid (those are not logged)
- we cannot get the users masks (maybe they could be inferred from the
  JOIN etc, but it is not worth the effort and would not work every
  time)

The regular soju network history is not sent to clients that support
draft/chathistory, so that they can fetch what they need by manually
calling CHATHISTORY.

The only supported command is BEFORE for now, because that is the only
required command for an app that offers an "infinite history scrollback"
feature.

Regarding implementation, rather than reading the file from the end in
reverse, we simply start from the beginning of each log file, store each
PRIVMSG into a ring, then add the last lines of that ring into the
history we'll return later. The message parsing implementation must be
kept somewhat fast because an app could potentially request thousands of
messages in several files. Here we are using simple sscanf and indexOf
rather than regexps.

In case some log files do not contain any message (for example because
the user had not joined a channel at that time), we try up to a 100 days
of empty log files before giving up.

[1]: https://github.com/prawnsalad/ircv3-specifications/pull/3/files
[2]: https://github.com/ircv3/ircv3-specifications/pull/393/files#r350210018
2020-06-05 23:50:31 +02:00
Simon Ser
283d4bf14c
Introduce ircConn
This interface will allow a conn to be backed by a websocket.
2020-06-04 17:27:57 +02:00
fox.cpp
203dc3df6a
Implement upstream SASL EXTERNAL support
Closes: https://todo.sr.ht/~emersion/soju/47
2020-06-02 11:24:22 +02:00
Hubert Hirtz
c43ce0da29
Send the full user mask in RPL_LOGGEDIN
As per the spec [1]:

    :server 900 <nick> <nick>!<ident>@<host> <account> :Now logged in

[1]: https://ircv3.net/specs/extensions/sasl-3.1
2020-06-01 18:57:20 +02:00
delthas
f4e0c51366
Add support for TAGMSG and client message tags
Previously we dropped all TAGMSG as well as any client message tag sent
from downstream.

This adds support for properly forwarding TAGMSG and client message tags
from downstreams and upstreams.

TAGMSG messages are intentionally not logged, because they are currently
typically used for +typing, which can generate a lot of traffic and is
only useful for a few seconds after it is sent.
2020-05-27 23:48:08 +02:00
delthas
f13a9c9d86
Send a label with all messages sent from downstream
This is preparatory work for forwarding errors of downstream-initiated
messages to their sender, as well as any other unknown message.

Preivously, we only sent labels (for labeled-response) for specific
downstream messages, such as WHO, where we knew the reply should only be
sent to that specific downstream.

However, in the case of an error of a message that is not labeled, the
error reply is not be tagged with a downstream id label and we can't
forward it to a specific downstream. It is not a good solution either to
forward this error to all downstreams.

This adds labels to all downstream-initiated messages (provided the
upstream supports it).
2020-05-27 23:46:31 +02:00
delthas
791cf4b887
Add support for downstream WHOIS nick/network nick/network
Many IRC clients use the query `WHOIS nick nick` rather than
`WHOIS nick` when querying a nick. The former command means to
specifically query the WHOIS on the server to which `nick` is connected,
which is useful to get information that is sometimes not propagated
between servers, such as idle time.

In the case where a downstream sends WHOIS nick/network nick/network in
multi-server mode, we need to unmarshal both fields.

Previously, we did not unmarshal those fields, and upstreams would
receive `WHOIS nick/network nick`, which is incorrect.

This adds support for unmarshaling the target field if it is the same as
the mask field, by simply using the unmarshaled nick that is already
computed from the mask.
2020-05-27 23:44:38 +02:00
delthas
a64e1d6761
Add support for downstream LIST to a single upstream
Sometimes, doing a LIST on a single upstream can be useful: if a user is
already connected to Rizon and freenode, sending a LIST will contain
tens of thousands of LIST replies that may not be useful if the user is
interested in another upstream.

This adds support for sending `LIST */network`, which follows the ELIST
M mask extension, that will only send LIST to that specific network. No
other masks are supported by this commit.
2020-05-27 23:43:46 +02:00
delthas
2d47d7067f
Add support for downstream NICK to a single upstream
Users often have different nicks on different upstreams, and we should
support changing the user nick on a single upstream.

This adds support for a new trivial extension, `NICK nick/network`,
which will change the nick on the specified network, and do nothing for
the other networks.
2020-05-27 23:43:04 +02:00
delthas
b3742cc731
Update downstream nicks in single-server mode and after NICK
Previously, the downstream nick was never changed, even when the
downstream sent a NICK message or was in single-server mode with a
different nick.

This adds support for updating the downstream nick in the following
cases:
- when a downstream sends NICK
- additionally, in single-server mode:
  - when a downstream connects and its single network is connected
  - when an upstream connects
  - when an upstream sends NICK
2020-05-27 23:42:38 +02:00
delthas
c88700ef18
Fix parsing MODE messages by updating channel memberships
Previously, we only considered channel modes in the modes of a MODE
messages, which means channel membership changes were ignored. This
resulted in bugs where users channel memberships would not be properly
updated and cached with wrong values. Further, mode arguments
representing entities were not properly marshaled.

This adds support for correctly parsing and updating channel memberships
when processing MODE messages. Mode arguments corresponding to channel
memberships updates are now also properly marshaled.

MODE messages can't be easily sent from history because marshaling these
messages require knowing about the upstream available channel types and
channel membership types, which is currently only possible when
connected. For now this is not an issue since we do not send MODE
messages in history.
2020-05-21 22:36:54 +02:00
delthas
732b581eb2
Add support for multiple user channel memberships
User channel memberships are actually a set of memberships, not a single
value. This introduces memberships, a type representing a set of
memberships, stored as an array of memberships ordered by descending
rank.

This also adds multi-prefix to the permanent downstream and upstream
capabilities, so that we try to get all possible channel memberships.
2020-05-11 12:25:49 +02:00
Simon Ser
81e4930931
Add time tag to echo messages
Closes: https://todo.sr.ht/~emersion/soju/59
2020-05-05 16:52:50 +02:00
delthas
40ee5e8405 Fix not properly marshaling self in single-server mode
In single-server mode, we don't need to add a /network suffix when
marshaling, but we still need to replace our nick with the downstream
nick.
2020-05-01 21:56:40 +02:00
Simon Ser
aa6f3a9954
Don't clear channel key on JOIN
Closes: https://todo.sr.ht/~emersion/soju/50
2020-05-01 17:39:53 +02:00
Simon Ser
d9186e994d
Add support for detached channels
Channels can now be detached by leaving them with the reason "detach",
and re-attached by joining them again. Upon detaching the channel is
no longer forwarded to downstream connections. Upon re-attaching the
history buffer is sent.
2020-05-01 15:18:14 +02:00
Simon Ser
e7e4311160
Remove network.upstream
This is an artifact from when we used locks. No need for this anymore.
2020-04-30 10:25:16 +02:00
Simon Ser
c4655f1492
Add upstreamConn.caps
Instead of adding one field per capability, let's just have a map, just
like downstreamConn.
2020-04-29 19:45:37 +02:00
Simon Ser
0c549d68c4
Add support for away-notify
This makes use of cap-notify to dynamically advertise support for
away-notify. away-notify is advertised to downstream connections if all
upstreams support it.
2020-04-29 19:34:44 +02:00
Simon Ser
394f2853ad
Add downstream support for cap-notify 2020-04-29 19:34:38 +02:00
delthas
19854b7ec7 Unmarshal nicks in texts of PRIVMSG and NOTICE from downstreams
When writing a PRIVMSG or NOTICE on a channel, it is very common to use
autocompletion to mention other users on that channel. When using soju
in multi-network mode, all users will have their nicked suffixed by
`/network`. This suffix should be removed before sending it upstream.

This adds support for removing all `/network` suffixes in messages sent
to a channel of that network.
2020-04-24 18:26:44 +02:00
Simon Ser
9c463b61ec
Fix typo 2020-04-16 17:25:39 +02:00
Simon Ser
3e80573765
Support sending history when upstream is disconnected
Previously, we were dropping the history.
2020-04-16 17:23:35 +02:00
Simon Ser
45e897c1c1
Make downstreamConn.marshal{Entity,UserPrefix} take a network
This will be used when sending history while upstream is disconnected.
2020-04-16 17:19:00 +02:00
Simon Ser
5cf876cb89
Kill downstreamConn.marshal{Nick,Channel}
We can just use downstreamConn.marshalEntity instead.
2020-04-16 16:57:33 +02:00
Simon Ser
96be0b5945
Disallow marshalling for anotehr network 2020-04-16 16:54:13 +02:00
Simon Ser
e508f2ca81
Unify downstreamConn.marshal{Entity,Nick,Channel}
We don't actually need to check if the entity is a channel.
2020-04-16 16:33:56 +02:00
Simon Ser
8e852cc7e4
Add support for downstream batch & chathistory 2020-04-15 11:29:15 +02:00
Simon Ser
db198335aa
Per-entity ring buffers
Instead of having one ring buffer per network, each network has one ring
buffer per entity (channel or nick). This allows history to be more
fair: if there's a lot of activity in a channel, it won't prune activity
in other channels.

We now track history sequence numbers per client and per network in
networkHistory. The overall list of offline clients is still tracked in
network.offlineClients.

When all clients have received history, the ring buffer can be released.

In the future, we should get rid of too-old offline clients to avoid
having to maintain history for them forever. We should also add a
per-user limit on the number of ring buffers.
2020-04-10 19:22:47 +02:00
Simon Ser
9f6e59195c
Document downstreamConn.{,un}marshalEntity 2020-04-07 22:30:54 +02:00
Simon Ser
3195809c30
Centralize logged messages marshaling
This allows messages added to logs to be handled just like messages
added to the ring buffer.
2020-04-07 19:45:29 +02:00
Simon Ser
dd08acc3ea
Make Ring.NewConsumer seq argument mandatory
There's no point in supporting a nil argument anymore.
2020-04-07 14:45:08 +02:00
Simon Ser
9692114f37
Make upstreamConn.produce log messages 2020-04-06 21:42:55 +02:00
Simon Ser
baadb964bc
Add origin argument to upstreamConn.produce 2020-04-06 21:34:45 +02:00
Simon Ser
1c17d25731
Remove downstreamConn.ourMessages
We can just do the filtering when dispatching the message.
2020-04-06 19:08:43 +02:00
Simon Ser
e1ea0d4fb5
Simplify ring consumer loop
No need to use Peek here.
2020-04-06 18:34:33 +02:00
Simon Ser
f0bc919885
Remove downstreamConn.ringConsumers
We no longer need long-lived ring buffer consumers now that
upstreamConn.produce dispatches messages to downstream connections.
2020-04-06 18:31:48 +02:00
Simon Ser
40ff14ec6c
Remove downstreamConn.lock
Everything is now accessed from the user goroutine now that the
per-network ring buffer goroutine is gone.
2020-04-06 18:23:39 +02:00
Simon Ser
d541587701
Remove channel from ring buffer consumers
This is unused.
2020-04-06 18:13:46 +02:00
Simon Ser
ad2c142c36
Remove per-network ring buffer goroutines
Just dispatch from the user goroutine. This allows removes a lot of complexity.
2020-04-06 18:05:36 +02:00
Simon Ser
0c06142ae9
Introduce upstreamConn.produce
The logic in this function is about to get more complicated. It'll soon
also dispatch messages in connected downstreams.
2020-04-06 17:28:49 +02:00
Simon Ser
e70bf9c42a
Add network.{createUpdate,delete}Channel
These are about to get more complicated as we move towards per-entity
ring buffers.
2020-04-05 15:04:52 +02:00
Simon Ser
ee8aae7a96
Add NOTICE messages to ring buffer
References: https://todo.sr.ht/~emersion/soju/33
2020-04-03 21:00:19 +02:00
Simon Ser
3d142cae9f
Add time tag to all messages 2020-04-03 20:48:23 +02:00
Simon Ser
27e0906450
Introduce messageLogger
This centralizes formatting related to message logging in a single
place.
2020-04-03 18:59:17 +02:00
Simon Ser
3a89152346
Log self-messages too
Closes: https://todo.sr.ht/~emersion/soju/44
2020-04-03 17:15:12 +02:00
Simon Ser
e1931649e4
Use conn.ReadMessage instead of irc.Conn.ReadMessage 2020-04-03 17:01:25 +02:00
Simon Ser
0dc007cd45
Add support for downstream echo-message extension 2020-04-03 16:55:49 +02:00
Simon Ser
2a0696b6bb
Introduce conn for common connection logic
This centralizes the common upstream & downstream bits.
2020-04-03 16:35:08 +02:00
Simon Ser
8c6328207b
Fix writer goroutine races
Any SendMessage call after Close could potentially block forever if the
outgoing channel was filled up. Now the channel is drained before the
writer goroutine exits.
2020-04-03 16:15:25 +02:00
Simon Ser
7c10535bfd
Fix SQL error logged on JOIN
Closes: https://todo.sr.ht/~emersion/soju/40
2020-04-01 17:34:22 +02:00
Simon Ser
29f2e93ab7
Set write deadlines
References: https://todo.sr.ht/~emersion/soju/26
2020-04-01 16:27:53 +02:00
Simon Ser
d4de60a869
Simplify ring consumer goroutine
Since network.history is now only accessed from the user goroutine, a
lock becomes unnecessary.
2020-04-01 16:02:31 +02:00
Simon Ser
10da094259
Stop ring consumers when deleting network 2020-04-01 15:48:56 +02:00
Simon Ser
743540665b
Add downstream support for server-time 2020-03-31 19:50:31 +02:00
Simon Ser
cbadb64748
Expose message-tags capability downstream
Strip tags if the client doesn't support them.
2020-03-31 19:39:06 +02:00
Simon Ser
73ee7d237f
Make "@" and "/" indicate client name and network, respectively
This allows both kinds "<username>@<client>/<network>" and
"<username>/<network>@<client>".
2020-03-31 19:02:02 +02:00
Simon Ser
d748ff269c
Consume ring messages outside of writer goroutine
This fixes out-of-order JOIN and PRIVMSG messages.

Closes: https://todo.sr.ht/~emersion/soju/36
2020-03-31 18:16:54 +02:00
Simon Ser
b1fd943ad6
Use clientName to decide whether or not history should be sent
Closes: https://todo.sr.ht/~emersion/soju/31
2020-03-28 17:36:09 +01:00
Simon Ser
6ff26e6640
Remove downstreamConn.username
Replace it with downstreamConn.user.Username
2020-03-28 17:28:28 +01:00
Simon Ser
b7e442670d
Add downstreamConn.clientName 2020-03-28 17:25:48 +01:00
Simon Ser
5a8adc93fe
Update password on NickServ SET PASSWORD message 2020-03-28 10:40:33 +01:00
Simon Ser
53d4f9f71b
s/List/LIST/ when referring to the command 2020-03-28 01:03:00 +01:00
Simon Ser
2ae43be525
Document functions safe to call from any goroutine 2020-03-28 00:54:42 +01:00
Simon Ser
1c3da31f2e
Introduce eventUpstreamDisconnected
This allows us to perform cleanup actions in the user goroutine. This
removes the need for pendingLISTsLock.
2020-03-28 00:51:58 +01:00
delthas
10ea698022 Add LIST support
This commit adds support for downstream LIST messages from multiple
concurrent downstreams to multiple concurrent upstreams, including
support for multiple pending LIST requests from the same downstream.

Because a unique RPL_LISTEND message must be sent to the requesting
downstream, and that there might be multiple upstreams, each sending
their own RPL_LISTEND, a cache of RPL_LISTEND replies of some sort is
required to match RPL_LISTEND together in order to only send one back
downstream.

This commit adds a list of "pending LIST" structs, which each contain a
map of all upstreams that yet need to send a RPL_LISTEND, and the
corresponding LIST request associated with that response. This list of
pending LISTs is sorted according to the order that the requesting
downstreams sent the LIST messages in. Each pending set also stores the
id of the requesting downstream, in order to only forward the replies to
it and no other downstream. (This is important because LIST replies can
typically amount to several thousands messages on large servers.)

When a single downstream makes multiple LIST requests, only the first
one will be immediately sent to the upstream servers. The next ones will
be buffered until the first one is completed. Distinct downstreams can
make concurrent LIST requests without any request buffering.

Each RPL_LIST message is forwarded to the downstream of the first
matching pending LIST struct.

When an upstream sends an RPL_LISTEND message, the upstream is removed
from the first matching pending LIST struct, but that message is not
immediately forwarded downstream. If there are no remaining pending LIST
requests in that struct is then empty, that means all upstreams have
sent back all their RPL_LISTEND replies (which means they also sent all
their RPL_LIST replies); so a unique RPL_LISTEND is sent to downstream
and that pending LIST set is removed from the cache.

Upstreams are removed from the pending LIST structs in two other cases:
- when they are closed (to avoid stalling because of a disconnected
upstream that will never reply to the LIST message): they are removed
from all pending LIST structs
- when they reply with an ERR_UNKNOWNCOMMAND or RPL_TRYAGAIN LIST reply,
which is typically used when a user is not allowed to LIST because they
just joined the server: they are removed from the first pending LIST
struct, as if an RPL_LISTEND message was received
2020-03-28 00:07:20 +01:00
delthas
72a285aaeb Make upstream.SendMessageLabeled use an uint64 id
This commit is preparatory work for code that will call
SendMessageLabeled with a direct downstream id rather than a
downstreamConnection pointer.
2020-03-28 00:07:20 +01:00
Simon Ser
2fff645129
Stop accessing user data in downstreamConn.authenticate
This becomes racy once user.Password is updated on-the-fly.
2020-03-27 22:38:38 +01:00
Simon Ser
08bb06c164
Nuke user.lock
Split user.register into two functions, one to make sure the user is
authenticated, the other to send our current state. This allows to get
rid of data races by doing the second part in the user goroutine.

Closes: https://todo.sr.ht/~emersion/soju/22
2020-03-27 19:17:58 +01:00
Simon Ser
c0f5850e5b
Add eventDownstreamDisconnected
This should remove the need for protecting user.downstreamConns with a
mutex.
2020-03-27 17:55:03 +01:00
Simon Ser
36ab6ece09
Add eventDownstreamConnected
In a later commit, we'll be able to move part of downstreamConn.register
into the user goroutine to prevent races.

References: https://todo.sr.ht/~emersion/soju/22
2020-03-27 17:21:05 +01:00
Simon Ser
474f2889d9
Introduce a user.events channel
This allows to easily add new events, and also guarantees ordering
between different event types.
2020-03-27 16:33:19 +01:00
delthas
764b54da59
Add downstream NOTICE support 2020-03-27 09:40:51 +01:00
delthas
9530df5db0
Add downstream INVITE support 2020-03-26 22:43:48 +01:00
delthas
3b6e175365
Add upstream RPL_CREATIONTIME support 2020-03-26 22:33:44 +01:00
delthas
a018f35c42
Add downstream TOPIC support 2020-03-26 22:27:46 +01:00
delthas
bab26c7a6f
Add KICK support
Downstream and upstream message handling are slightly different because
downstreams can send KICK messages with multiple channels or users,
while upstreams can only send KICK messages with one channel and one
user (according to the RFC).
2020-03-26 00:15:26 +01:00
delthas
9b7309d4c9
Simplify error handling for downstream JOIN/PART
downstreamConnection.unmarshalEntity already returns an ircError of
command ERR_NOSUCHCHANNEL, so there's no need to explicitly return
another ircError of that type.
2020-03-26 00:12:53 +01:00
delthas
6e72071716
Route NAMES, WHO, WHOIS replies to the requesting downstream
Using labeled-response, the replies to several commands such as NAMES,
WHO, WHOIS can be routed back to a specific downstream, rather than
being broadcast to all downstreams.

For example, after this commit, if the server supports labeled-response,
if a downstream requests the NAMES or WHO or WHOIS of a channel, the
replies of the upstream will only be sent back to that downstream, and
the other downstreams won't receive these messages.
2020-03-25 23:21:14 +01:00
delthas
d0917f0fa1
Add a server-unique id to each downstream
Adding a simple uint64 id to each downstream is preparatory work
for labeled-responses tags targeting a specific downstream.
2020-03-25 23:17:46 +01:00
Simon Ser
33dacc4fb0
Add support for channel keys 2020-03-25 11:54:08 +01:00
Simon Ser
146906ef6b
Handle downstream JOIN/PART with multiple channel names
Closes: https://todo.sr.ht/~emersion/soju/19
2020-03-25 11:32:44 +01:00
Simon Ser
21241c2009
Consume ring buffer for networks added on-the-fly 2020-03-25 11:28:25 +01:00
Simon Ser
293a0e8e20
Move upstreamConn.ring to network
This handles upstream disconnection and re-connection better.
2020-03-25 10:53:08 +01:00
delthas
9486d657c5
Add downstream self WHO and WHOIS support 2020-03-25 09:51:22 +01:00
delthas
a958a01446
Add user hostname to downstream user prefix 2020-03-25 09:48:00 +01:00