This commit takes insert query compilation and transaction creation out
of the critical loop for migrating message logs. I have tested with
the sqlite backend, and a speedup of approximately 40x has been achieved
for log migration.
This adds a new config option, `logs db`, which enables storing chat
logs in the soju database.
Regular store options, CHATHISTORY options, and SEARCH operations are
supported, like the fs logs backend.
Messages are stored in a new table, Message. In order to track the list
of targets we have messages for in an optimized manner, another database
is used: MessageTarget.
All new requests are backend by indexes so should be fast even with
hundreds of thousands of messages.
A contrib script is provided for migrating existing logs fs chat logs to
the database. It can be run with eg:
go run ./contrib/migrate-logs/ logs/ sqlite3:soju.db
Co-authored-by: Simon Ser <contact@emersion.fr>
Calvin Lee
Simon Ser
delthas