Commit Graph

202 Commits

Author SHA1 Message Date
Simon Ser
7c31c26d86
Don't perform TLS handshake in connectToUpstream
This defers TLS handshake until the first read or write operation. This
allows the upcoming identd server to register the connection before the
TLS handshake is complete, and is necessary because some IRC servers
send an ident request before that.
2020-08-11 10:24:54 +02:00
Simon Ser
b0bf012bbc
Change unix:// to irc+unix://
When Unix socket support will be added for listeners, unix:// will be
ambiguous. It won't be clear whether to setup an IRC server, or some
other kind of server (e.g. identd).

unix:// is still recognized to avoid breaking existing DBs.
2020-07-22 15:44:19 +02:00
Simon Ser
51dc9f0bf9
Broadcast unhandled messages to downstream connections
In case labelled-response isn't supported, broadcast unhandled messages
to all downstream connections. That's better than silently dropping the
messages.
2020-07-08 18:21:52 +02:00
Simon Ser
85fad93a71
Add support for upstream Unix socket connections
References: https://todo.sr.ht/~emersion/soju/51
2020-07-06 17:31:11 +02:00
Simon Ser
7af21d9d81
Parse upstream URLs with net/url
This allows us to ignore the path part of the URL. This is preliminary
work for unix URLs.
2020-07-06 17:18:13 +02:00
Simon Ser
b46a2554e1
Check upstream address with net.SplitHostPort
Looking for a colon is incorrect, IPv6 addresses can contain colons too.
2020-07-06 16:04:56 +02:00
Simon Ser
c490705fee
Sort and split JOIN messages
Sort channels so that channels with a key appear first. Split JOIN
messages so that we don't reach the message size limit.
2020-07-06 11:06:20 +02:00
Simon Ser
90250f5be5
Reply to WHO BouncerServ
Closes: https://todo.sr.ht/~emersion/soju/75
2020-06-29 18:09:48 +02:00
Simon Ser
d26194f62d
Handle registration failures
References: https://todo.sr.ht/~emersion/soju/30
2020-06-29 17:52:49 +02:00
delthas
2232b3128b Fix sending messages from detached channels
Currently, a downstream receives MODE, RPL_CHANNELMODEIS and
RPL_CREATIONTIME messages from soju for detached channels. It should not
be sent any of these messages.

This adds a detach check to the handling of these messages to avoid
receiving these messages.
2020-06-12 14:43:45 +02:00
Simon Ser
d0cf1d2882
Add support for WebSocket connections
WebSocket connections allow web-based clients to connect to IRC. This
commit implements the WebSocket sub-protocol as specified by the pending
IRCv3 proposal [1].

WebSocket listeners can now be set up via a "wss" protocol in the
`listen` directive. The new `http-origin` directive allows the CORS
allowed origins to be configured.

[1]: https://github.com/ircv3/ircv3-specifications/pull/342
2020-06-07 14:13:46 +02:00
fox.cpp
1fa5195b2f
Fix handling of empty response to SASL challenge 2020-06-05 11:35:43 +02:00
Simon Ser
283d4bf14c
Introduce ircConn
This interface will allow a conn to be backed by a websocket.
2020-06-04 17:27:57 +02:00
Simon Ser
07b4de8a1a
Only send JOIN message if we have channels to join
Don't send a JOIN command to upstream server when no channels are configured.
2020-06-03 17:18:57 +02:00
fox.cpp
203dc3df6a
Implement upstream SASL EXTERNAL support
Closes: https://todo.sr.ht/~emersion/soju/47
2020-06-02 11:24:22 +02:00
Eyal Sawady
c547aa0278
Join all channels with a single JOIN message 2020-05-28 16:26:28 +02:00
delthas
b1367a0608 Fix parsing upstream RPL_INVITING messages
Previously, we did not skip the first RPL_INVITING parameter, which is
the user nick (like in all replies), which made the parsing for that
reply incorrect.

This fixes RPL_INVITING parsing by skipping the first parameter.
2020-05-27 23:51:44 +02:00
delthas
f4e0c51366
Add support for TAGMSG and client message tags
Previously we dropped all TAGMSG as well as any client message tag sent
from downstream.

This adds support for properly forwarding TAGMSG and client message tags
from downstreams and upstreams.

TAGMSG messages are intentionally not logged, because they are currently
typically used for +typing, which can generate a lot of traffic and is
only useful for a few seconds after it is sent.
2020-05-27 23:48:08 +02:00
delthas
cdef46d0da
Forward all labeled errors and unknown messages to their downstream
This adds support for forwarding all errors and unknown messages labeled
with a specific downstream to that downstream.

Provided that the upstream supports labeled-response, users will now be
able to receive an error only on their client when making a command that
returns an error, as well as receiving any reply unknown to soju.
2020-05-27 23:47:49 +02:00
delthas
96dad08302
Add support for upstream ban, invite, and exception lists
This does not try to marshal nicks in masks, for simplicity and
consistency with the current behaviour of marshaling MODE messages.
2020-05-27 23:45:13 +02:00
delthas
b3742cc731
Update downstream nicks in single-server mode and after NICK
Previously, the downstream nick was never changed, even when the
downstream sent a NICK message or was in single-server mode with a
different nick.

This adds support for updating the downstream nick in the following
cases:
- when a downstream sends NICK
- additionally, in single-server mode:
  - when a downstream connects and its single network is connected
  - when an upstream connects
  - when an upstream sends NICK
2020-05-27 23:42:38 +02:00
delthas
566986fdd5
Fix sending CAP END twice when SASL is not used
When SASL is not used, we should only send CAP END after we send a CAP
REQ. Previously CAP END was sent both after a CAP REQ and a CAP ACK,
resulting in two CAP END messages.

Sending a CAP END right after the CAP REQ rather than waiting for the
CAP ACK/NAK saves 1 RTT.
2020-05-21 22:38:07 +02:00
delthas
de7b4c8588
Fix a null access due to assigning nil to the member memberships map
Even though the memberships map has type map[string]*memberships
(with memberships being defined as []membership), the default value for
that map should not be `nil` but a pointer to a nil slice.

This fixes a segfault on some servers before user channel prefixes are
sent.
2020-05-21 22:37:37 +02:00
delthas
c88700ef18
Fix parsing MODE messages by updating channel memberships
Previously, we only considered channel modes in the modes of a MODE
messages, which means channel membership changes were ignored. This
resulted in bugs where users channel memberships would not be properly
updated and cached with wrong values. Further, mode arguments
representing entities were not properly marshaled.

This adds support for correctly parsing and updating channel memberships
when processing MODE messages. Mode arguments corresponding to channel
memberships updates are now also properly marshaled.

MODE messages can't be easily sent from history because marshaling these
messages require knowing about the upstream available channel types and
channel membership types, which is currently only possible when
connected. For now this is not an issue since we do not send MODE
messages in history.
2020-05-21 22:36:54 +02:00
delthas
732b581eb2
Add support for multiple user channel memberships
User channel memberships are actually a set of memberships, not a single
value. This introduces memberships, a type representing a set of
memberships, stored as an array of memberships ordered by descending
rank.

This also adds multi-prefix to the permanent downstream and upstream
capabilities, so that we try to get all possible channel memberships.
2020-05-11 12:25:49 +02:00
Simon Ser
1363a5c27e
Improve highlight matching
Detect word boundaries instead of just doing a sub-string check.
2020-05-01 19:51:22 +02:00
Simon Ser
bd8ab7230d
Send service NOTICE on highlight in detached channel 2020-05-01 19:05:20 +02:00
Simon Ser
70f26a1a5a
Unify upstream NOTICE and PRIVMSG handling 2020-05-01 18:12:47 +02:00
Simon Ser
d9186e994d
Add support for detached channels
Channels can now be detached by leaving them with the reason "detach",
and re-attached by joining them again. Upon detaching the channel is
no longer forwarded to downstream connections. Upon re-attaching the
history buffer is sent.
2020-05-01 15:18:14 +02:00
Simon Ser
bbb5e79f59
Introduce permanentUpstreamCaps 2020-04-30 16:10:39 +02:00
Simon Ser
2a569c3b27
Add upstream cap-notify support 2020-04-30 15:27:41 +02:00
Simon Ser
c4655f1492
Add upstreamConn.caps
Instead of adding one field per capability, let's just have a map, just
like downstreamConn.
2020-04-29 19:45:37 +02:00
Simon Ser
8445979956
Rename upstreamConn.caps to supportedCaps
For consistency with downstreamConn.
2020-04-29 19:40:31 +02:00
Simon Ser
0c549d68c4
Add support for away-notify
This makes use of cap-notify to dynamically advertise support for
away-notify. away-notify is advertised to downstream connections if all
upstreams support it.
2020-04-29 19:34:44 +02:00
Simon Ser
70131f5b31
Don't use forEachDownstreamByID when forwarding RPL_AWAY
We should broadcast the message, not send it to a specific downstream
connection.
2020-04-29 15:32:22 +02:00
Simon Ser
2236142681
Check upstream NOTICE prefix is non-nil 2020-04-29 15:00:17 +02:00
Simon Ser
976ae2e649
Forward RPL_AWAY messages 2020-04-29 14:53:48 +02:00
delthas
7b35757bac Add support for the irc+insecure address scheme
Some servers do not support TLS, or have invalid, expired or self-signed
TLS certificates. While the right fix would be toi contact each server
owner to add support for valid TLS, supporting plaintext upstream
connections is sometimes necessary.

This adds support for the irc+insecure address scheme, which connects to
a network in plain-text over TCP.
2020-04-28 11:41:13 +02:00
delthas
19795a2321 Add support for IRC address schemes
This is preparatory work for adding other connection types to upstream
servers. The service command `network create` now accepts a scheme in
the address flag, which specifies how to connect to the upstream server.

The only supported scheme for now is ircs, which is also the default if
no scheme is specified. ircs connects to a network over a TLS TCP
connection.
2020-04-28 11:41:13 +02:00
Simon Ser
276ce12e7c
Add network.channels, remove DB.GetChannel
Store the list of configured channels in the network data structure.
This removes the need for a database lookup and will be useful for
detached channels.
2020-04-23 16:36:20 +02:00
delthas
7f74055380 Add support for custom network on-connect commands
Some servers use custom IRC bots with custom commands for registering to
specific services after connection.

This adds support for setting custom raw IRC messages, that will be
sent after registering to a network.

It also adds support for a custom flag.Value type for string
slice flags (flags taking several string values).
2020-04-16 17:38:47 +02:00
Simon Ser
3e80573765
Support sending history when upstream is disconnected
Previously, we were dropping the history.
2020-04-16 17:23:35 +02:00
Simon Ser
45e897c1c1
Make downstreamConn.marshal{Entity,UserPrefix} take a network
This will be used when sending history while upstream is disconnected.
2020-04-16 17:19:00 +02:00
Simon Ser
5cf876cb89
Kill downstreamConn.marshal{Nick,Channel}
We can just use downstreamConn.marshalEntity instead.
2020-04-16 16:57:33 +02:00
Simon Ser
db198335aa
Per-entity ring buffers
Instead of having one ring buffer per network, each network has one ring
buffer per entity (channel or nick). This allows history to be more
fair: if there's a lot of activity in a channel, it won't prune activity
in other channels.

We now track history sequence numbers per client and per network in
networkHistory. The overall list of offline clients is still tracked in
network.offlineClients.

When all clients have received history, the ring buffer can be released.

In the future, we should get rid of too-old offline clients to avoid
having to maintain history for them forever. We should also add a
per-user limit on the number of ring buffers.
2020-04-10 19:22:47 +02:00
Simon Ser
f2037c5d52
Make newMessageLogger take a *network instead of an *upstreamConn
There's no reason why messgeLogger needs access to the whole connection,
the network is enough.
2020-04-07 21:54:24 +02:00
Simon Ser
0a895b591e
Remove outdated comment
Locks are gone for good.
2020-04-07 19:57:36 +02:00
Simon Ser
3195809c30
Centralize logged messages marshaling
This allows messages added to logs to be handled just like messages
added to the ring buffer.
2020-04-07 19:45:29 +02:00
Simon Ser
37cd9e4d89
Fix upstream NICK handling
- uc.nicl was compared to msg.Prefix.Name after being updated to the new
  nick
- The new nick wasn't marshaled
2020-04-07 15:02:44 +02:00
Simon Ser
13635747c4
Fix server-time tags timezone
We need to use UTC time.
2020-04-06 22:05:49 +02:00
Simon Ser
9692114f37
Make upstreamConn.produce log messages 2020-04-06 21:42:55 +02:00
Simon Ser
baadb964bc
Add origin argument to upstreamConn.produce 2020-04-06 21:34:45 +02:00
Simon Ser
78b123be72
Log TOPIC messages
See [1].

[1]: 82c4ad8362/modules/log.cpp (L486)
2020-04-06 18:49:39 +02:00
Simon Ser
f0bc919885
Remove downstreamConn.ringConsumers
We no longer need long-lived ring buffer consumers now that
upstreamConn.produce dispatches messages to downstream connections.
2020-04-06 18:31:48 +02:00
Simon Ser
ad2c142c36
Remove per-network ring buffer goroutines
Just dispatch from the user goroutine. This allows removes a lot of complexity.
2020-04-06 18:05:36 +02:00
Simon Ser
0c06142ae9
Introduce upstreamConn.produce
The logic in this function is about to get more complicated. It'll soon
also dispatch messages in connected downstreams.
2020-04-06 17:28:49 +02:00
delthas
93c96d6c21
Remove unused entityLog struct
Since the logging refactoring in 27e090, entityLog is not used anymore
and can be removed.
2020-04-04 17:34:37 +02:00
delthas
eb941d2d2b
Send one NOTICE on new upstream disconnect/connect errors
In order to notify the user when we are disconnected from a network
(either due to an error, or due a QUIT), and when we fail reconnecting,
this commit adds support for sending a short NOTICE message from the
service user to all relevant downstreams.

The last error is stored, and cleared on successful connection, to
ensure that the user is *not* flooded with identical connection error
messages, which can often happen when a server is down.

No lock is needed on lastError because it is only read and modified from
the user goroutine.

Closes: https://todo.sr.ht/~emersion/soju/27
2020-04-04 17:33:09 +02:00
Simon Ser
ee8aae7a96
Add NOTICE messages to ring buffer
References: https://todo.sr.ht/~emersion/soju/33
2020-04-03 21:00:19 +02:00
Simon Ser
3d142cae9f
Add time tag to all messages 2020-04-03 20:48:23 +02:00
Simon Ser
27e0906450
Introduce messageLogger
This centralizes formatting related to message logging in a single
place.
2020-04-03 18:59:17 +02:00
Simon Ser
e1931649e4
Use conn.ReadMessage instead of irc.Conn.ReadMessage 2020-04-03 17:01:25 +02:00
Simon Ser
2a0696b6bb
Introduce conn for common connection logic
This centralizes the common upstream & downstream bits.
2020-04-03 16:35:08 +02:00
Simon Ser
8c6328207b
Fix writer goroutine races
Any SendMessage call after Close could potentially block forever if the
outgoing channel was filled up. Now the channel is drained before the
writer goroutine exits.
2020-04-03 16:15:25 +02:00
Simon Ser
bca0b2ad76
Set connect timeout
References: https://todo.sr.ht/~emersion/soju/26
2020-04-01 16:41:17 +02:00
Simon Ser
29f2e93ab7
Set write deadlines
References: https://todo.sr.ht/~emersion/soju/26
2020-04-01 16:27:53 +02:00
Simon Ser
434e722f92
Auto away
Closes: https://todo.sr.ht/~emersion/soju/13
2020-04-01 12:16:32 +02:00
Simon Ser
190f0d3fd0
Only set network.conn when registered 2020-04-01 12:14:36 +02:00
Simon Ser
977a1b294f
Fix log dir permission
We need the permission to list files in the dir.
2020-03-31 22:21:49 +02:00
Simon Ser
2053e62162
Request server-time cap
If the server didn't populate the time tag, do it ourselves.
2020-03-31 19:45:04 +02:00
Simon Ser
29a42203b6
Avoid directly forwarding NOTICE messages
This would forward tags even if downstream doesn't support those.
2020-03-31 19:41:12 +02:00
Simon Ser
1023c2ebfc
Rename AppendLog to appendLog
This function is only safe to call from inside the user goroutine. Let's
make it private.
2020-03-31 17:30:45 +02:00
Simon Ser
aadeb327d3
Fix missing upstreamConn.closed initialization
Fixes: b33e5f29ab ("Fix race condition in upstreamConn.Close")
2020-03-28 20:00:56 +01:00
Simon Ser
53d4f9f71b
s/List/LIST/ when referring to the command 2020-03-28 01:03:00 +01:00
Simon Ser
2ae43be525
Document functions safe to call from any goroutine 2020-03-28 00:54:42 +01:00
Simon Ser
1c3da31f2e
Introduce eventUpstreamDisconnected
This allows us to perform cleanup actions in the user goroutine. This
removes the need for pendingLISTsLock.
2020-03-28 00:51:58 +01:00
delthas
0607b940e2 Add support for bouncer logs
Add bouncer logs, in a network/channel/date.log format, in a similar
manner to ZNC log module. PRIVMSG, JOIN, PART, QUIT, MODE are logged.

Add a config directive for the logs file, including a way to disable
them entirely.
2020-03-28 00:07:20 +01:00
delthas
10ea698022 Add LIST support
This commit adds support for downstream LIST messages from multiple
concurrent downstreams to multiple concurrent upstreams, including
support for multiple pending LIST requests from the same downstream.

Because a unique RPL_LISTEND message must be sent to the requesting
downstream, and that there might be multiple upstreams, each sending
their own RPL_LISTEND, a cache of RPL_LISTEND replies of some sort is
required to match RPL_LISTEND together in order to only send one back
downstream.

This commit adds a list of "pending LIST" structs, which each contain a
map of all upstreams that yet need to send a RPL_LISTEND, and the
corresponding LIST request associated with that response. This list of
pending LISTs is sorted according to the order that the requesting
downstreams sent the LIST messages in. Each pending set also stores the
id of the requesting downstream, in order to only forward the replies to
it and no other downstream. (This is important because LIST replies can
typically amount to several thousands messages on large servers.)

When a single downstream makes multiple LIST requests, only the first
one will be immediately sent to the upstream servers. The next ones will
be buffered until the first one is completed. Distinct downstreams can
make concurrent LIST requests without any request buffering.

Each RPL_LIST message is forwarded to the downstream of the first
matching pending LIST struct.

When an upstream sends an RPL_LISTEND message, the upstream is removed
from the first matching pending LIST struct, but that message is not
immediately forwarded downstream. If there are no remaining pending LIST
requests in that struct is then empty, that means all upstreams have
sent back all their RPL_LISTEND replies (which means they also sent all
their RPL_LIST replies); so a unique RPL_LISTEND is sent to downstream
and that pending LIST set is removed from the cache.

Upstreams are removed from the pending LIST structs in two other cases:
- when they are closed (to avoid stalling because of a disconnected
upstream that will never reply to the LIST message): they are removed
from all pending LIST structs
- when they reply with an ERR_UNKNOWNCOMMAND or RPL_TRYAGAIN LIST reply,
which is typically used when a user is not allowed to LIST because they
just joined the server: they are removed from the first pending LIST
struct, as if an RPL_LISTEND message was received
2020-03-28 00:07:20 +01:00
delthas
72a285aaeb Make upstream.SendMessageLabeled use an uint64 id
This commit is preparatory work for code that will call
SendMessageLabeled with a direct downstream id rather than a
downstreamConnection pointer.
2020-03-28 00:07:20 +01:00
Simon Ser
b33e5f29ab
Fix race condition in upstreamConn.Close
upstreamConn.closed was a bool accessed from different goroutines. Use
the same pattern as downstreamConn instead.
2020-03-27 23:08:35 +01:00
delthas
f08063c943
Fix parsing wrong empty element in RPL_WHOISCHANNELS channel list
Some servers add a trailing space to the channel list in
RPL_WHOISCHANNELS. This commit works around this issue by removing any
empty trailing element after splitting.

Since RPL_WHOISCHANNELS could send an empty channel parameter, we can't
just use strings.TrimRight(s, " "), because splitting on an empty string
would still return an empty element.

Closes: https://todo.sr.ht/~emersion/soju/25
2020-03-27 22:53:21 +01:00
delthas
8c56e610ff
Marshal NOTICE user prefixes and channels
NOTICE messages can be both special messages from the server (with no
prefix nick), or regular PRIVMSG-like messages from users. This commit
adds support for marshaling channel and user prefixes in the latter
case.
2020-03-27 21:12:16 +01:00
Simon Ser
474f2889d9
Introduce a user.events channel
This allows to easily add new events, and also guarantees ordering
between different event types.
2020-03-27 16:33:19 +01:00
delthas
9530df5db0
Add downstream INVITE support 2020-03-26 22:43:48 +01:00
delthas
3b6e175365
Add upstream RPL_CREATIONTIME support 2020-03-26 22:33:44 +01:00
delthas
2cb0cf3665
Fix capitalization of 'id'
Following the Go project source code guidelines, id is an abbreviation
and must typically be written as id or ID, not Id.
2020-03-26 22:28:13 +01:00
delthas
bab26c7a6f
Add KICK support
Downstream and upstream message handling are slightly different because
downstreams can send KICK messages with multiple channels or users,
while upstreams can only send KICK messages with one channel and one
user (according to the RFC).
2020-03-26 00:15:26 +01:00
delthas
d00698e15c
Fix wrong handling of members parameter of RPL_NAMREPLY
Some servers (namely UnrealIRCd) wrongly add a trailing space to the
members parameters of the RPL_NAMREPLY command, which was not handled
correctly.

Adding a trailing space is not legal wrt the IRC specs, but since
UnrealIRCd does it and is in wide use today, we have to work around it.
2020-03-25 23:21:26 +01:00
delthas
6e72071716
Route NAMES, WHO, WHOIS replies to the requesting downstream
Using labeled-response, the replies to several commands such as NAMES,
WHO, WHOIS can be routed back to a specific downstream, rather than
being broadcast to all downstreams.

For example, after this commit, if the server supports labeled-response,
if a downstream requests the NAMES or WHO or WHOIS of a channel, the
replies of the upstream will only be sent back to that downstream, and
the other downstreams won't receive these messages.
2020-03-25 23:21:14 +01:00
delthas
e19f8aaba4
Add upstream labeled-response capability support 2020-03-25 23:20:56 +01:00
delthas
df8bfb75f2
Add upstream batch capability support 2020-03-25 23:16:53 +01:00
delthas
27dae3e8ad
Add upstream message-tags capability support 2020-03-25 23:15:48 +01:00
Simon Ser
33dacc4fb0
Add support for channel keys 2020-03-25 11:54:08 +01:00
Simon Ser
293a0e8e20
Move upstreamConn.ring to network
This handles upstream disconnection and re-connection better.
2020-03-25 10:53:08 +01:00
delthas
9486d657c5
Add downstream self WHO and WHOIS support 2020-03-25 09:51:22 +01:00
delthas
9ff8429a53
Add downstream NAMES support
NAMES reply for channels currently joined will be returned from cache;
requests for channels not joined will be forwarded from upstream.
2020-03-25 09:47:26 +01:00
delthas
98a95e9955
Add MODE arguments support
- Add RPL_ISUPPORT support with CHANMODES, CHANTYPES, PREFIX parsing
- Add support for channel mode state with mode arguments
- Add upstream support for RPL_UMODEIS, RPL_CHANNELMODEIS
- Request channel MODE on upstream channel JOIN
- Use sane default channel mode and channel mode types
2020-03-25 09:40:08 +01:00
Simon Ser
45d118dd12
Move upstreamConn.history to network 2020-03-20 22:48:17 +01:00
delthas
aedf66c73d Fix MODE downstream support
- Fix replies without client as first argument
- Replace wrong prefix check with a proper entity type check
2020-03-20 10:11:27 +01:00
delthas
b3ad960529 Add WHOIS support 2020-03-20 10:11:27 +01:00