Commit Graph

501 Commits

Author SHA1 Message Date
Simon Ser
fb4a01b023 Add per-network logger 2021-04-13 20:12:49 +02:00
Simon Ser
be2825595d Error out on network name conflict
Closes: https://todo.sr.ht/~emersion/soju/29
2021-04-13 19:33:06 +02:00
Simon Ser
a2c207d357 Relay detached channel backlog as BouncerServ NOTICE if necessary
Instead of ignoring detached channels wehn replaying backlog,
process them as usual and relay messages as BouncerServ NOTICEs
if necessary. Advance the delivery receipts as if the channel was
attached.

Closes: https://todo.sr.ht/~emersion/soju/98
2021-04-13 19:11:05 +02:00
Simon Ser
76e332b50a Move isHighlight to irc.go 2021-04-13 18:54:58 +02:00
Simon Ser
424f676254 Store last internal msg ID in DB when detaching
References: https://todo.sr.ht/~emersion/soju/98
2021-04-13 18:15:30 +02:00
Simon Ser
45e2c0023a Skip backlog logic in downstreamConn.welcome on chathistory 2021-04-13 17:50:03 +02:00
Simon Ser
65c58adbd9 Take msg ID in sendTargetBacklog 2021-04-13 17:49:37 +02:00
Simon Ser
ef902fdc28 Panic on unknown user event type
This should never happen. Complain loudly if it does.
2021-04-13 13:08:48 +02:00
Simon Ser
adb4b300dc Introduce per-user logger
Adds the username to log lines.
2021-04-13 13:04:23 +02:00
Hubert Hirtz
70e5ed05b6 Make casemapping work over bytes instead of runes
Fixes a panic in partialCasemap when the input string was invalid UTF-8.
2021-04-13 12:48:46 +02:00
Simon Ser
c375c7d922 cmd/soju: allow specifying -listen multiple times
Closes: https://todo.sr.ht/~emersion/soju/67
2021-03-31 19:02:40 +02:00
Simon Ser
bc7123f486 Update dependencies 2021-03-31 18:18:16 +02:00
Simon Ser
1e4ff49472 Save delivery receipts in DB
This avoids loosing history on restart for clients that don't
support chathistory.

Closes: https://todo.sr.ht/~emersion/soju/80
2021-03-31 18:04:13 +02:00
Simon Ser
5b4469fcb7 Use BARE for internal message IDs
This allows to have shorter and more future-proof IDs. This also
guarantees the IDs will only use reasonable ASCII characters (no
spaces), removing the need to encode them for PING/PONG tokens.
2021-03-31 17:57:24 +02:00
Simon Ser
5e11e717f1 Rename user.clients to clientNames
This doesn't contain anything other than just the names. Make this
clearer.
2021-03-30 12:44:56 +02:00
Simon Ser
ecf35187fa Make NickServ detection casemapping-aware 2021-03-30 12:28:45 +02:00
Simon Ser
3237bde9f3 Introduce deliveredStore
This hides the double-map complexity behind a dedicated type.
2021-03-29 17:49:50 +02:00
Simon Ser
07519da768 Ensure targets are case-mapped before being passed to messageStore
messageStore isn't aware of the network's case-mapping. We need
to canonicalize the names before passing them to messageStore.
2021-03-29 17:07:39 +02:00
Simon Ser
55274d7213 Move network.clients to user
No need to have this list per-network.
2021-03-29 16:58:56 +02:00
Simon Ser
5a899abaab Simplify network.offlineClients
Replace it with a list of all clients (online or offline).
2021-03-29 16:55:57 +02:00
Simon Ser
2b92e4ecd4 contrib/casemap-logs.sh: new utility script
Previous soju versions were storing log without converting the channel
and nick names to their canonical lower-case representation. This could
result in two log directories for the same channel/nick.

This script fixes old log dirs.
2021-03-26 15:31:54 +01:00
Simon Ser
6e5a307dc7 Introduce deliveredClientMap
Adds more semantics to map[string]string. Simplifies the complicated
mapStringStringCasemapMap type.
2021-03-26 11:21:14 +01:00
Hubert Hirtz
5014673aae Fix CHATHISTORY target not being casemapped 2021-03-26 10:39:52 +01:00
Hubert Hirtz
bdd0c7bc06
Implement casemapping
TL;DR: supports for casemapping, now logs are saved in
casemapped/canonical/tolower form
(eg. in the #channel directory instead of #Channel... or something)

== What is casemapping? ==

see <https://modern.ircdocs.horse/#casemapping-parameter>

== Casemapping and multi-upstream ==

Since each upstream does not necessarily use the same casemapping, and
since casemappings cannot coexist [0],

1. soju must also update the database accordingly to upstreams'
   casemapping, otherwise it will end up inconsistent,
2. soju must "normalize" entity names and expose only one casemapping
   that is a subset of all supported casemappings (here, ascii).

[0] On some upstreams, "emersion[m]" and "emersion{m}" refer to the same
user (upstreams that advertise rfc1459 for example), while on others
(upstreams that advertise ascii) they don't.

Once upstream's casemapping is known (default to rfc1459), entity names
in map keys are made into casemapped form, for upstreamConn,
upstreamChannel and network.

downstreamConn advertises "CASEMAPPING=ascii", and always casemap map
keys with ascii.

Some functions require the caller to casemap their argument (to avoid
needless calls to casemapping functions).

== Message forwarding and casemapping ==

downstream message handling (joins and parts basically):
When relaying entity names from downstreams to upstreams, soju uses the
upstream casemapping, in order to not get in the way of the user.  This
does not brings any issue, as long as soju replies with the ascii
casemapping in mind (solves point 1.).

marshalEntity/marshalUserPrefix:
When relaying entity names from upstreams with non-ascii casemappings,
soju *partially* casemap them: it only change the case of characters
which are not ascii letters.  ASCII case is thus kept intact, while
special symbols like []{} are the same every time soju sends them to
downstreams (solves point 2.).

== Casemapping changes ==

Casemapping changes are not fully supported by this patch and will
result in loss of history.  This is a limitation of the protocol and
should be solved by the RENAME spec.
2021-03-24 18:15:52 +01:00
delthas
56bf73716d Increase downstream TCP keepalive interval to 1 hour
The rationale for increasing the TCP keepalive interval from 15 seconds
(default) to 1 hour follows.

- Why increasing TCP keepalives for downstream connections is not an
  issue wrt to detecting connection interruptions

The use case of TCP keepalives is detecting whether a TCP connection was
forcefully shut down without receiving any TCP FIN or RST frame, when no
data are sent from that endpoint to the other peer.

If any data is sent from the peer and is not ACKed because the
connection was interrupted, the socket will be closed after the TCP RTO
(usually a few seconds) anyway, without the need for TCP keepalives.

Therefore the only use of TCP keepalives is making sure that a peer that
is not writing anything to the socket, and is actively reading and
waiting for new stream data to be received, can, - instead of waiting
forever to receive packets that will never arrive because the connection
was interrupted -, detect this disconnection, close the connection
locally, then try to connect again to its peer.

This only makes sense from a client point-of-view. When an IRC client is
not write(2)ing anything to the socket but is simply waiting for new
messages to arrive, ie read(2)ing on the socket, it must ensure that the
connection is still alive so that any new messages will indeed be sent
to him. So that IRC client should probably enable TCP keepalives.

However, when an IRC server is not writing anything to its downstream
socket, it doesn't care if it misses any messages from its downstream
client: in any case, the downstream client will instantly detect when
its messages are not reaching its server, because of the TCP RTO
(keepalives are not even needed in the client in that specific case),
and will try to reconnect to the server.

Thus TCP keepalives should be enabled for upstream connections, in
order to make sure that soju does not miss any messages coming from
upstream servers, but TCP keepalives are not needed for downstream
connections.

- Why increasing TCP keepalives for downstream connections is not an
  issue wrt security, performance, and server socket resources
  exhaustion

TCP keepalives are orthogonal to security. Malicious clients can open
thousands of TCP connections and keep them open with minimal
bookkeeping, and TCP keepalives will not prevent attacks planning to
use up all available sockets to soju.

It is also unlikely that soju will keep many connections open, and in
the event that thousands of dead, disconnected connections are active in
soju, any upstream message that needs to be sent to downstreams will
disconnect all disconnected downstreams after the TCP RTO (a few
seconds). Performance could only be slightly affected in the few seconds
before a TCP RTO if many messages were sent to a very large number of
disconnected connections, which is extremely unlikely and not a large
impact to performance either.

- Why increasing TCP keepalives could be helpful to some clients running
  on mobile devices

In the current state of IRC, most clients running on mobile devices
(mostly running Android and iOS) will probably need to stay connected
at all times, even when the application is in background, in order to
receive private messages and highlight notifications, complete chat
history (and possibly reduced connection traffic due to avoiding all the
initial messages traffic, including all NAMES and WHO replies which
are quite large).

This means most IRC clients on mobile devices will keep a socket open at
all times, in background. When a mobile device runs on a cellular data
connection, it uses the phone wireless radio to transmit all TCP
packets, including TCP packets without user data, for example TCP
keepalives.

On a typical mobile device, a wireless radio consumes significant power
when full active, so it switches between several energy states in order
to conserve power when not in use. It typically has 3 energy states,
from Standby, when no messages are sent, to Low Power, to Full Power;
and switches modes on an average time scale of 15s. This means that any
time any TCP packet is sent from any socket on the device, the radio
switches to a high-power energy state, sends the packet, then stays on
that energy state for around 15s, then goes back to Standby. This
does include TCP keepalives.

If a TCP keepalive of 15s was used, this means that the IRC server would
force all clients running on mobile devices to send a TCP keepalive
packet at least once every 15s, which means that the radio would stay
in its high-power energy state at all times. This would consume a
very significant amount of power and use up battery much faster.

Even though it would seem at first that a mobile device would have many
different sockets open at any time; actually, a typical Android device
typically has at one background socket open, with Firebase Cloud
Messaging, for receiving instant push notifications (for example, for
the equivalent of IRC highlights on other messaging platforms), and
perhaps a socket open for the current foreground app. When the current
foreground app does not use the network, or when no app is currently
used and the phone is in sleep mode, and no notifications are sent, then
the device can effectively have no wireless radio usage at all. This
makes removing TCP keepalives extremely significant with regard to the
mobile device battery usage.

Increasing the TCP keepalive from soju lets downstream clients choose
their own keepalive interval and therefore possibly save battery for
mobile devices. Most modern mobile devices have complex heuristics for
when to sleep the CPU and wireless radio, and have specific rules for
TCP keepalives depending on the current internet connection, sleep
state, etc.

By increasing the downstream TCP keepalive to such a high period, soju
lets clients choose their most optimal TCP keepalive period, which means
that in turn clients can possibly let their mobile device platform
choose best that keepalive for them, thus letting them save battery in
those cases.
2021-03-24 18:04:44 +01:00
Simon Ser
c0513013d5 Fix panic on GetCertificate
Fixes the following panic:

    panic: interface conversion: interface {} is tls.Certificate, not *tls.Certificate
2021-03-19 09:27:19 +01:00
Simon Ser
21e9fe9b3c Reload TLS certs on SIGHUP
References: https://todo.sr.ht/~emersion/soju/42
2021-03-18 14:07:03 +01:00
Simon Ser
927ee80da1 Stop reading X-Forwarded-Port
X-Forwarded-Port contains the destination port, not the source port,
so it isn't useful for our purposes.

Move parsing of X-Forwarded-* header fields to parseForwarded.
2021-03-18 13:28:46 +01:00
Simon Ser
1b49fff763 Fix Forwarded HTTP header handling
"for" contains the port, if any. "port" doesn't exist.
2021-03-18 13:21:38 +01:00
Simon Ser
9046fda283 Add support for the Forwarded HTTP header
This is the standard replacing X-Forwarded-*.
2021-03-18 12:08:25 +01:00
Simon Ser
5b7205c9c1 Drop "irc" WebSocket subprotocol
The subprotocol hasn't been standardized yet. It looks like the standard
is moving in another direction.
2021-03-18 12:02:36 +01:00
Simon Ser
0c0397407c Don't add "irc" in ALPN list for WebSocket servers
This is incorrect because HTTP listeners don't handle plain IRC
connections. This also prevents net/http from setting up an HTTP/2
server.
2021-03-18 11:33:30 +01:00
Simon Ser
e35a116188 Don't update downstream caps in upstream RPL_WELCOME handler
Prior to being registered, upstreamConn.handleMessage doesn't run
in the user goroutine, it runs in a goroutine specific to the
network. Thus we shouldn't access any user data structure from
there.

downstreamConn.updateSupportedCaps is already called from the
eventUpstreamConnected handler in user.run, the call being removed
was unnecessary.

Closes: https://todo.sr.ht/~emersion/soju/108
2021-03-16 18:25:21 +01:00
Simon Ser
384075a6ed Don't store history for NickServ
Closes: https://todo.sr.ht/~emersion/soju/104
2021-03-16 09:54:29 +01:00
Simon Ser
dab91736db Send NOTICE to downstream when upstream is disconnected
Closes: https://todo.sr.ht/~emersion/soju/76
2021-03-16 09:41:07 +01:00
Simon Ser
061347f9f9 Add Unix socket listener
Closes: https://todo.sr.ht/~emersion/soju/51
2021-03-16 09:27:40 +01:00
Simon Ser
67fb669434 Correctly set WebSocket read/write deadline
The methods didn't have pointer receivers. Thus the deadline fields
were only updated for the local variable.

Closes: https://todo.sr.ht/~emersion/soju/106
2021-03-16 09:19:12 +01:00
Simon Ser
26c5c11caf Improve ERR_NOSUCHCHANNEL error messages
References: https://todo.sr.ht/~emersion/soju/63
2021-03-16 09:13:46 +01:00
Simon Ser
fa047123b9 Passthrough some ISUPPORT tokens 2021-03-15 23:41:37 +01:00
Simon Ser
3f005d481d Properly handle all ISUPPORT negations 2021-03-15 23:11:42 +01:00
Simon Ser
62d4bf2813 Use upstream ISUPPORT map for NETWORK 2021-03-15 23:08:19 +01:00
Simon Ser
2992ff79c4 Maintain state for upstream ISUPPORT 2021-03-15 23:06:36 +01:00
Simon Ser
ff2cd9423f Simplify if block in ISUPPORT handler 2021-03-15 22:54:32 +01:00
Simon Ser
ffd142f8dd Extract ISUPPORT CHANMODES/PREFIX to separate functions 2021-03-15 22:53:46 +01:00
Simon Ser
d54c8c1122 Add Network.{URL,GetUsername,GetRealname}
Just a bunch of helpers that can be re-used.
2021-03-09 18:55:34 +01:00
Hubert Hirtz
1645371276 Send correct CHATHISTORY error messages 2021-03-05 09:53:59 +01:00
Simon Ser
0954c7da79
Add irc to ALPN protocols
The new ALPN token has been approved [1]. We can start using it now.

[1]: https://mailarchive.ietf.org/arch/msg/tls-reg-review/i8YyT82XUtEgR-oXMG3sbyWYT8E/
2021-02-24 19:41:12 +01:00
Hubert Hirtz
6a1a05b1d4 if true return true else return false
Trivial improvement
2021-02-18 09:40:51 +01:00
Simon Ser
31cd56875a Use sendTargetBacklog when re-attaching a channel
No need to attempt to send backlog for all targets in the network.
We're only interested in a single channel.
2021-02-10 13:50:10 +01:00
Simon Ser
26473ed60d Introduce downstreamConn.sendTargetBacklog 2021-02-10 13:48:41 +01:00