Sebastian Jeltsch
e0ad4a898c
Extend trailbase-sqlites execution model to allow for parallel reads. This reduces the latency long-tail for slow reads.
...
Note that this complicates the APIs, since it pushes the responsibility
of declaring a query a read or write to the user to then be scheduled
appropriately.
Add `.(read_|)query_row_f` APIs similar to rusqlites
`conn.query_row` accepting a `|row| -> Result<T>` to reduce the use of
`Row` and `Rows`.
Make benchmarks more isolated by not sharing a DB across runs
accumulating writes.
2025-04-15 12:15:40 +02:00
Sebastian Jeltsch
eecab32d39
Minor: consistent tokio and env_logger dependencies.
2025-04-14 09:25:44 +02:00
Sebastian Jeltsch
9e040d34f6
Make chat-room benchmark more realistic: hammer server in parallel.
2025-04-13 12:28:20 +02:00
Sebastian Jeltsch
914a9b3f03
Make query mix benchmark's slow queries slower and group benchmarks with throughput as requested.
2025-04-13 11:12:08 +02:00
Sebastian Jeltsch
844ccfa80c
Add a mixed query benchmark to trailbase-sqlite and make all benchmarks concurrently hammer the connection.
2025-04-12 15:23:17 +02:00
Sebastian Jeltsch
faa1712601
Add read benchmarks to trailbase-sqlite.
2025-04-12 13:09:22 +02:00
Sebastian Jeltsch
d0d60857eb
Add a shared and TL/pool SQLite connection implementation to the benchmarks as a mere reference.
...
These implementations are dangerous since despite being behind an async
API, they block on I/O and can thus clog up the tokio runtime. Which is
fine if SQLite is all you do but really bad if other, independent work
gets stalled.
2025-04-12 11:56:47 +02:00
Sebastian Jeltsch
458adcf9ec
Add a benchmark scaffold to sqlite.
2025-04-12 10:35:55 +02:00
Sebastian Jeltsch
c0c7681f4a
Restructure crates: make the sqlite execution model more standalone, i.e. move more extension stuff back to extension crate and schema stuff to a new schema crate.
2025-04-08 12:25:10 +02:00
Sebastian Jeltsch
503d7cae9b
Pass connection constructor to prepare for broader concurrency models.
2025-04-08 10:05:56 +02:00
Sebastian Jeltsch
f7cce2dc44
Fix logging: deterministically and separately initialize log and tracing.
2025-04-04 13:23:26 +02:00
Sebastian Jeltsch
31d7dc4158
Use statement cache in more places.
2025-04-03 21:59:23 +02:00
Sebastian Jeltsch
edfbd4c4f6
Manage file deletions through an auxiliary table: "_file_deletions" and triggers.
...
This allows deleting files eagerly w/o a transaction (i.e. pre-op query to get
old values), allows for defered cleanups, and retries.
2025-04-02 00:15:05 +02:00
Sebastian Jeltsch
d9525cd754
Minor: remove log dep from library crates.
2025-03-31 11:26:03 +02:00
Sebastian Jeltsch
3fe9dc8494
Update UUID sqlite extensions, fix example blog and clean it up.
2025-03-24 15:34:01 +01:00
Sebastian Jeltsch
2f07a86030
Update rusqlite: v0.33 -> v0.34.
2025-03-10 14:10:18 +01:00
Sebastian Jeltsch
c63f3f4089
Cleanup: remove legacy query_one_row utility.
2025-03-10 11:09:31 +01:00
Sebastian Jeltsch
68824465a0
Publish updated crates: trailbase(|-sqlite|-extension).
2025-03-09 12:08:46 +01:00
Sebastian Jeltsch
5705b99b6c
Audit and remove remaining unwraps.
2025-03-09 00:04:09 +01:00
Sebastian Jeltsch
527974284f
Update Rust and JS deps.
2025-02-27 00:59:20 +01:00
Sebastian Jeltsch
c06ceee31b
Control expansion of FKs in RecordApi.(read|list) via query parameters.
2025-02-13 13:47:10 +01:00
Sebastian Jeltsch
fd318cf328
Expand/inline foreign records in RecordApi list results.
2025-02-13 13:47:10 +01:00
Sebastian Jeltsch
757f0c370d
Expand/inline foreign records in RecordApi read.
2025-02-13 13:47:03 +01:00
Sebastian Jeltsch
eec17e1343
Update Rust dependencies.
2025-02-12 22:37:55 +01:00
Sebastian Jeltsch
031e2a5032
Update Rust and JS deps.
2025-02-02 12:26:02 +01:00
Sebastian Jeltsch
31e8421451
Updating and cleaning up some Rust deps.
2025-01-26 21:29:44 +01:00
Sebastian Jeltsch
55c26afb0f
Prepare release v0.5.1 and publish new crates.
2025-01-20 15:27:13 +01:00
Sebastian Jeltsch
5144dabc68
Major version update: SQLite 3.46.1 to 3.48.0.
2025-01-19 17:39:30 +01:00
Sebastian Jeltsch
2fb294f17c
Breaking: nest records in output of RecordAPI.list to contain cursor now and potentially more in the future.
...
Also update all the client libraries to accept the new format.
2025-01-19 15:40:09 +01:00
Sebastian Jeltsch
1828ebad5a
Optimization: reduce allocations, cloning and re-encoding.
2025-01-13 22:40:27 +01:00
Sebastian Jeltsch
30f295e6fd
Add "realtime" subscriptions for a specific record, i.e. updates and deletion.
2025-01-13 22:38:14 +01:00
Sebastian Jeltsch
c9199edd22
Prepare trailbase-sqlite for publishing.
2024-12-30 11:52:02 +01:00
Sebastian Jeltsch
e2c43a4c41
Prepare vendored sqlean to be published as a crate.
2024-12-30 11:36:07 +01:00
Sebastian Jeltsch
7e43e3b149
Update all dependencies.
2024-12-30 10:01:57 +01:00
Sebastian Jeltsch
a464103b17
Update Rust dependencies.
2024-12-24 10:16:22 +01:00
Sebastian Jeltsch
312d161048
Reduce number of allocations.
2024-12-17 13:11:01 +01:00
Sebastian Jeltsch
18d1d87710
Add a hooks API to trailbase_sqlite::Connection.
2024-12-16 17:07:17 +01:00
Sebastian Jeltsch
746f0c1108
Make connection constructor synchronous.
2024-12-16 17:00:51 +01:00
Sebastian Jeltsch
4894cc0bb0
Minor: move Row/Rows into a separate file.
2024-12-16 17:00:47 +01:00
Sebastian Jeltsch
51a6af3149
Remove sqlite-loadable and use rusqlite functions instead.
2024-12-16 16:41:52 +01:00
Sebastian Jeltsch
a675d09b1f
Remove unused conn.call_and_forget function.
2024-12-09 21:43:30 +01:00
Sebastian Jeltsch
03d918cd94
Remove unused code.
2024-12-05 10:10:43 +01:00
Sebastian Jeltsch
1b5c957415
Move SQLite orchestration into trailbase-sqlite and fix js-runtime completion bug with true async (previously libsql was sync under the hood).
2024-12-04 16:57:15 +01:00
Sebastian Jeltsch
3b9a122e06
Install sqlite-vec to replace libsql's builtin vector search.
2024-12-04 13:14:14 +01:00
Sebastian Jeltsch
6cbea390fb
Herculean migration from libsql's rust bindings to plain rusqlite+SQLite.
...
There's a couple of reasons:
* As for the rust bindings: they're sub-par to rusqlite, though
rusqlite is amazing. Even libsql-server uses rusqlite over their own
bindings. The bindings are missing features such as update hooks
and the hard-coded execution model suffers from lock congestion.
* We've fixed bugs (e.g. trivial null ptr accesses went unnoticed),
raised issues, and tried to add missing functionality such as update
hooks. It's unclear if the rust-bindings are a priority or covered by
the principles laid out in the libsql manifesto. From the outside it
looks like focus has shifted to https://github.com/penberg/limbo .
* As for the C-libsql fork for SQLite itself, it's getting more and
more outdated (2024-01-30 (3.45.1)) and it's unclear when and if the
ideas from the manifesto will manifest.
Looking forward this opens the door for TrailBase to:
* Bundle more recent versions of SQLite
* Implement more performant, better scaling execution models.
* Implement realtime APIs for subscribing to data changes.
2024-12-04 13:03:06 +01:00
Sebastian Jeltsch
387be1cce1
Update rust dependencies.
2024-11-08 21:46:30 +01:00
Sebastian Jeltsch
d3fd82c29f
Add frequency-by-country-code as a new aggregate stat to logs fetch.
...
In the future we will be able to use this in the admin dashboard.
2024-11-03 20:45:37 +01:00
Sebastian Jeltsch
bde2577dba
Experimental: support decoding client ips with geoip database.
2024-11-03 20:45:37 +01:00
Sebastian Jeltsch
2803de7567
Re-re-license from AGPLv3 to the weaker copyleft OSL-3.0.
...
After spending too much time researching licenses, my laymen
understanding makes me feel that OSL-3.0 will be a better match given
TrailBase's rare dual use as a standalone backend or framework.
It tried to outline the reasoning in the README.md. I would appreciate
input from anyone more experienced with licenses.
For now, I would like to turn my attention back to more technical issues
:hide:.
2024-11-02 12:35:10 +01:00
Sebastian Jeltsch
bdb3735840
Squash all commits for a fresh start.
...
This is only to avoid accidentally leaking any secrets from early development especially in the light of short-sha attacks.
2024-10-30 23:38:56 +01:00
