1
0
mirror of git://jb55.com/damus synced 2024-09-29 08:20:45 +00:00
Commit Graph

124 Commits

Author SHA1 Message Date
Charlie Fish
0f05123ef8 mute: migrate Lists.swift to use new MuteItem
This patch depends on: Adding new structs/enums for new mute list

- Rewrites Lists.swift to use new mute list option
    - This leads to a lot of changes for changing the type from RefId to the new MuteItem
- Update & relay new mute list in AddMuteItemView.swift (fixing previous patch TODO)
- Renames `list` to `list_deprecated`
    - We need to keep this since existing users might have an old mute list

Related: https://github.com/damus-io/damus/issues/1718
Related: https://github.com/damus-io/damus/issues/856
Lighting Address: fishcharlie@strike.me

Signed-off-by: Charlie Fish <contact@charlie.fish>
Reviewed-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
2024-01-22 11:03:06 -08:00
William Casarin
4cf92756f1 close nostrdb and disconnect from relays on logout
This was causing crashing and corruption issues. This should have been
handled by the garbage collector, but for some reason old references
still hang around.

Add a "close" method to DamusState which disconnects from relays and
closes nostrdb.

Changelog-Fixed: Fix crash when logging out and switching accounts
Changelog-Fixed: Fix persistent local notifications even after logout
Signed-off-by: William Casarin <jb55@jb55.com>
2024-01-10 16:25:07 -08:00
William Casarin
6834367386 ndb: fix crashed when trying to process client event on a closed db 2024-01-10 15:38:31 -08:00
William Casarin
afe3dcf039 test: switch test to use failable transactions 2024-01-10 15:38:21 -08:00
William Casarin
bfad2ab42d ndb/txn: make transactions failable
Since there may be situations where we close and re-open the database,
we need to make sure transactions fail when the database is not open.

Make NdbTxn an init?() constructor and check for ndb.closed. If it's
closed, then fail transaction construction.

This fixes crashes during high database activity when switching from
background to foreground and vice-versa.

Fixes: da2bdad18d ("nostrdb: close database when backgrounded")
2024-01-10 14:27:02 -08:00
William Casarin
227734d286 Revert "Revert "nostrdb: close database when backgrounded""
This reverts commit 26bd50c948.
2024-01-10 13:19:36 -08:00
William Casarin
909701ce7b profile: partially fix performance regression
This will be completely fixed once we switch to stored note blocks
2024-01-10 11:52:30 -08:00
Daniel D’Aquino
88f938d11c Bring local notification logic into the push notification target
This commit brings key local notification logic into the notification
extension target to allow the extension to reuse much of the
functionality surrounding the processing and formatting of
notifications. More specifically, the functions
`process_local_notification` and `create_local_notification` were
brought into the extension target.

This will enable us to reuse much of the pre-existing notification logic
(and avoid having to reimplement all of that)

However, those functions had high dependencies on other parts of the
code, so significant refactorings were needed to make this happen:

- `create_local_notification` and `process_local_notification` had its
  function signatures changed to avoid the need to `DamusState` (which
  pulls too many other dependecies)

- Other necessary dependencies, such as `Profiles`, `UserSettingsStore`
  had to be pulled into the extension target. Subsequently,
  sub-dependencies of those items had to be pulled in as well

- In several cases, files were split to avoid pulling too many
  dependencies (e.g. Some Model files depended on some functions in View
  files, so in those cases I moved those functions into their own
  separate file to avoid pulling in view logic into the extension
  target)

- Notification processing logic was changed a bit to remove dependency
  on `EventCache` in favor of using ndb directly (As instructed in a
  TODO comment in EventCache, and because EventCache has too many other
  dependencies)

tldr: A LOT of things were moved around, a bit of logic was changed
around local notifications to avoid using `EventCache`, but otherwise
this commit is meant to be a no-op without any new features or
user-facing functional changes.

Testing
-------

Device: iPhone 15 Pro
iOS: 17.0.1
Damus: This commit
Coverage:

1. Ran unit tests to check for regressions (none detected)

2. Launched the app and navigated around and did some interactions to
   perform a quick functional smoke test (no regressions found)

3. Sent a few push notifications to check they still work as expected (PASS)

Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Signed-off-by: William Casarin <jb55@jb55.com>
2024-01-10 11:06:30 -08:00
Terry Yiu
bf78c0a3a0 translation: add workaround to reduce wasteful translation requests
Signed-off-by: Terry Yiu <git@tyiu.xyz>
Reviewed-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Changelog-Fixed: Add workaround to fix note language recognition and reduce wasteful translation requests
2024-01-08 12:28:33 -08:00
Charlie Fish
84cfeb1604 nip42: add initial relay auth support
Lightning-Invoice: lnbc1pjcpaakpp5gjs4f626hf8w6slx84xz3wwhlf309z503rjutckdxv6wwg5ldavsdqqcqzpgxqrrs0fppqjaxxw43p7em4g59vedv7pzl76kt0qyjfsp5qcp9de7a7t8h6zs5mcssfaqp4exrnkehqtg2hf0ary3z5cjnasvs9qyyssq55523e4h3cazhkv7f8jqf5qp0n8spykls49crsu5t3m636u3yj4qdqjkdl2nxf6jet5t2r2pfrxmm8rjpqjd3ylrzqq89m4gqt5l6ycqf92c7h
Closes: https://github.com/damus-io/damus/issues/940
Signed-off-by: Charlie Fish <contact@charlie.fish>
Signed-off-by: William Casarin <jb55@jb55.com>
Changelog-Added: Add NIP-42 relay auth support
2024-01-05 10:36:03 -08:00
Daniel D’Aquino
26bd50c948 Revert "nostrdb: close database when backgrounded"
This reverts commit da2bdad18d.

This commit was reverted because it was causing crashes (Occasional `EXC_BAD_ACCESS` errors when accessing database transactions), and because the push notification extension uses Ndb in a read-only manner, which means we no longer need these changes

Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
2023-12-22 13:57:21 -08:00
William Casarin
da2bdad18d nostrdb: close database when backgrounded
Otherwise iOS gets mad because we are holding onto a lockfile in a
shared container which is apparently not allowed.

Fixes: a1e6be214e ("Migrate NostrDB files to shared app group file container")
2023-12-11 14:59:33 -08:00
William Casarin
c7cc8df5ba nostrdb: don't begin query if we have a bad lmdb env
In some weird multithreaded situations after we close the database,
this can be an issue.
2023-12-11 14:59:33 -08:00
William Casarin
afc42d1952 nostrdb/writer: make sure we don't write a note if we already have it
I'm noticing duplicate notes in the database, which might happen when
the ingester and writer get spammed with the same note rapidly. Add a
sanity check during the write so that we only ever write a note once.

Fixes: 1cf898e0b2 ("ndb: update nostrdb")
Changelog-Fixed: Fix duplicate notes getting written to nostrdb
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-04 14:54:28 -08:00
William Casarin
579303f741 ndb: fix minor text search result bug 2023-12-04 14:51:58 -08:00
William Casarin
a07b78e47f ndb: add safemode so we don't instantly crash on bad dbs
Fixes: https://github.com/damus-io/damus/issues/1741
2023-12-04 13:26:24 -08:00
William Casarin
4e447ddbed ndb/txn: inherit active transactions on the same thread
Many different parts of the codebase could be opening transactions when
somewhere higher in the heirarchy on the main thread might already have
an active transaction. This can lead to failed transaction opening which
is bad.

Instead of relying on passing down the transaction to subviews, lets
keep track of the active transactions in a thread-local dictionary. That
way whenever we create a new transaction we can inherit the one that is
already active in the current thread.

Inherited transactions don't end the query when they are garbage
collected, we still expect the first-opened query to do this.
2023-12-04 13:26:24 -08:00
William Casarin
0a9ac9cb0d ndb: more dumb results building 2023-12-03 22:13:11 -08:00
William Casarin
9c3b052de2 ndb/note: always track note size, add to_owned 2023-12-03 22:12:31 -08:00
William Casarin
9502fc30ba ndb: add initial search interface
Still needs updating because of the tuple array
2023-12-02 15:05:15 -08:00
William Casarin
6b8cf51720 nostrdb/search: fix another newest-first bug
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 15:04:23 -08:00
William Casarin
f72b297d77 nostrdb: add cpu helper 2023-12-02 13:47:17 -08:00
William Casarin
dd78272a5e Ndb: update to use new nostrdb config struct 2023-12-02 13:44:03 -08:00
William Casarin
65be56ba7c fix some incompatibility between nostrdb and damus' 2023-12-02 13:44:03 -08:00
William Casarin
8e361a9586 nostrdb/search: fix subtle bug with some newest-first text search
Due to the way the range queries work for newest-first searches, we can
have a situation where the MDB_SET_RANGE gets placed on either the
correct place or just after the correct place. To position the cursor
correctly, we jump back one if the search result prefix doesn't match.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
9bbeffe320 nostrdb/search: also index longform
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
3e5d7581ba nostrdb/refactor: move search key printer in case we need it
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
06445de197 nostrdb/search: make sure we break instead of return
so the cursor has a chance to close

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
e537c7cef4 nostrdb/search: allow searching from newest-to-oldest and oldest-to-newest
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
01c239c0eb nostrdb/search: add limit param
If we only care to have a certain number of results

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
bec92249f9 nostrdb/search: remove result printing, move to util/ndb
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
hakkadaikon
aa0b9bde8f nostrdb/Delete unuse argument (destsize)
Closes: https://github.com/damus-io/nostrdb/pull/18
Reviewed-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
4e4e8ed460 nostrdb/rename get_physical_cores to get_cpu_cores
less wrong

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
3605edad8b nostrdb/config: fix ingester thread settings
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
04c207a11a nostrdb/ingest: add configurable ingest filter
This allows users of nostrdb to selectively filter notes of any kind during
ingest.

Contact lists too big? Create a filter to reject them.

You only care about notes with specific kinds? Reject everything else.

Damus will use this for rejecting large events that might take up too
much space for storage, such as contact lists.

This commit also switched to ndb_config for configuring nostrdb, because
the arguments to ndb_init were getting out of hand.

Changelog-Added: Added ingest filter setting
Changelog-Changed: Switch to ndb_config for per-session ndb settings
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
9d208284c6 nostrdb/search: phrase searching working
This changes the search algorithm to be much smarter and more efficient
at searching phrases.

It is also much simpler, using less intermediate data structures.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
213a26cd01 nostrdb/flag: add ndb config flag for skipping note verification
makes some large imports a bit faster

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
b74bde5cc4 nostrdb/search: fix infinite loop when parsing some notes
Our word parser gets stuck on some notes with utf8 chars. Make sure we
are always advancing so we don't get stuck.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
0e0c53145f nostrdb/segfault: fix weird crash in ispunct
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
76aa1d3450 nostrdb/db: remove the DUPFIXED flag from the fulltext db
I don't think this technically makes any sense

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
fa0d5e7d03 nostrdb/debug: fix some debug-mode compile issues
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
8446db7cbc nostrdb/search: prepare text search for accurate phrase results
This patch sets the stage for phrase searching. It collects data into
result sets based on words and the word's associated key. We can use
this data to select the best search results based on adjacent
word_indices.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
8679c9f293 nostrdb/search: make search case insensitive
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
d541153e4c nostrdb/Add fulltext search index
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
53fc1b6945 nostrdb/Fix invalid db selection when writing kind index
Fixes: ebe92071af18 ("index: write kind index when processing notes")
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
ff60f8a2db nostrdb/index: write kind index when processing notes
We have a kind index database now, so write to it when processing notes

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
bfd78c01ca nostrdb/refactor: move write id index to its own function
We will be writing more indices so I'm trying to clean this up a bit
before this function gets too messy

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
a2af030367 nostrdb/refactor: move profile index writing to its own function
Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
3c2e8b728f nostrdb/index: create kind+timestamp index database
We don't build the index yet, but create the database like the others.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00
William Casarin
8269ca59cd nostrdb/index: add u64_timestamp lmdb comparator
custom kind+timestamp comparison function. This is used by lmdb to
perform b+ tree searches over the kind+timestamp index.

Signed-off-by: William Casarin <jb55@jb55.com>
2023-12-02 13:44:03 -08:00