1
0
mirror of git://jb55.com/damus synced 2024-09-18 19:23:49 +00:00
Commit Graph

34 Commits

Author SHA1 Message Date
William Casarin
ff60f8a2db nostrdb/index: write kind index when processing notes
We have a kind index database now, so write to it when processing notes

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
bfd78c01ca nostrdb/refactor: move write id index to its own function
We will be writing more indices so I'm trying to clean this up a bit
before this function gets too messy

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
a2af030367 nostrdb/refactor: move profile index writing to its own function
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
3c2e8b728f nostrdb/index: create kind+timestamp index database
We don't build the index yet, but create the database like the others.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
8269ca59cd nostrdb/index: add u64_timestamp lmdb comparator
custom kind+timestamp comparison function. This is used by lmdb to
perform b+ tree searches over the kind+timestamp index.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
0f9d55d4f9 nostrdb/debug: use mdb_strerror in more places
instead of codes

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
4109649dc2 nostrdb/filter: make sure we only match single chars
Without this, we could accidently match `pr` for `#p` filters

Fixes: 30ed801285dd ("filters: add initial filter interface")
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
466dfcb7d7 nostrdb/filters: add initial filter interface
This adds a way to construct filters and match them against notes. This
will be used by our query interface in the future.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
bc330ab5de update nostrdb
to include potential ingester thread crash fix
2023-11-14 10:26:09 -08:00
William Casarin
29fa4ecf2e build: fix invalid function name 2023-11-13 10:31:28 -08:00
William Casarin
518fdffce9 ndb: potential fix for a crash in some nostrdb queries 2023-11-13 10:31:28 -08:00
William Casarin
cc385d3c3f nostrdb: add migration to fix local japanese profile names 2023-11-02 14:17:17 +09:00
William Casarin
1cf898e0b2 ndb: update nostrdb
This includes the new profile fetched_at logic and reaction stats.

When receiving new profiles, nostrdb will record when it was last
received in a new database. This database is a mapping from Pubkey to
timestamp.

You can manually read/write to this table using:

ndb_read_last_profile_fetch
ndb_write_last_profile_fetch

This patch also includes the new reaction counting metadata table. It is
not used yet (but reactions are still counted!)

Changelog-Added: Added reaction counters to nostrdb
Changelog-Added: Record when profile is last fetched in nostrdb
2023-10-23 10:40:11 +08:00
William Casarin
b1c7ef9bd9 Fix profiles not updating
Timestamped IDs were not being initialized properly when writing profile
indices. This means that every profile was indexed with timestamp 0.
derp. Fix this!

Changelog-Fixed: Fix profiles not updating
2023-10-06 10:42:17 -07:00
William Casarin
d0daa9fafa nostrdb: add last fetched records for profiles 2023-10-06 10:42:17 -07:00
William Casarin
dacade299d ndb: bump nostrdb to support client->relay note processing 2023-09-24 17:06:04 -07:00
William Casarin
6f67c159ff ndb: switch to case-insensitive profile searches 2023-09-21 18:03:22 -04:00
William Casarin
7a85ae29ca search: switch to nostrdb profile searching
Changelog-Changed: Switch to nostrdb for @'s and user search
2023-09-21 13:19:22 -04:00
William Casarin
fafe3b4b3e ndb: add nostrdb migrations 2023-09-21 09:10:06 -04:00
William Casarin
fc9b9f2940 ndb: switch profile queries to use transactions
this should ensure no crashing occurs when querying profiles
2023-09-21 09:10:06 -04:00
William Casarin
9398877415 nostrdb/c: update to include transaction support 2023-09-21 09:10:06 -04:00
William Casarin
5657512370 ndb: restore escaped slash fix 2023-08-28 10:09:49 -07:00
William Casarin
882f6e2534 ndb: update nostrdb, fix alignment issues 2023-08-28 08:19:03 -07:00
William Casarin
caffa0398b nostrdb: profile flatbuffers in nostrdb working! 2023-08-26 20:46:42 -07:00
William Casarin
1f5f1e28a4 nostrdb: pull latest, adding flatcc and lmdb 2023-08-25 19:05:34 -07:00
Daniel D‘Aquino
066b3cdde8 Fix image links appearing with escaped slashes
Changelog-Fixed: Fix images and links occasionally appearing with escaped slashes
Closes: https://github.com/damus-io/damus/issues/1468
Signed-off-by: Daniel D‘Aquino <daniel@daquino.me>
Signed-off-by: William Casarin <jb55@jb55.com>
Rewarded-sats: 50000
2023-08-18 08:41:21 -07:00
William Casarin
cebd1f48ca ndb: switch to nostrdb notes
This is a refactor of the codebase to use a more memory-efficient
representation of notes. It should also be much faster at decoding since
we're using a custom C json parser now.

Changelog-Changed: Improved memory usage and performance when processing events
2023-08-03 13:20:36 -07:00
William Casarin
07dfa3b1fb ndb: update nostrdb
This include various fixes for parsing and key decoding
2023-08-01 21:53:19 -07:00
William Casarin
2f8aa29e92 ndb: make NostrEvents immutable
Since we can't mutate NdbNotes, let's update the existing codebase to
generate and sign ids on NostrEvent constructions. This will allow us to
match NdbNote's constructor
2023-07-25 15:34:05 -07:00
William Casarin
99b5dc94cb ndb: copy over perf improvements 2023-07-23 12:11:08 -07:00
William Casarin
e34351ca37 ndb: fix iterators, pack id tags, more tests 2023-07-23 11:55:36 -07:00
William Casarin
0263c11a94 ndb: add content and owned_size 2023-07-22 17:19:47 -07:00
William Casarin
4da23390f8 ndb: update lib 2023-07-22 17:19:47 -07:00
William Casarin
ad0e1f28b7 test: fix build and tests 2023-07-21 15:26:03 -07:00