3579f2142b
docs: line buffering in Python
2023-09-26 16:30:13 -04:00
b3e59566c6
switch import to use WriterPipeline
...
- This makes it do the verification and writing in parallel
- Also, it will now flush periodically (default 1s) even if it has read
fewer than N (default 10k) records from stdin. This lets import be used
as a general-purpose non-relay event ingester. To do so, users must
ensure that the stdout of their process they pipe into import is line
buffered.
2023-09-26 16:03:09 -04:00
cec2ba9006
release 0.9.6
2023-09-22 14:49:35 -04:00
0d88747613
relay.negentropy.enabled config option: set to false to prevent processing negentropy messages
2023-09-22 14:49:35 -04:00
c9254adaf8
bugfix: handle malformed/old-format negentropy messages gracefully
2023-09-22 14:49:35 -04:00
a7aefde56c
Merge pull request #70 from Overload3910/patch-1
...
Update DEPLOYMENT.md
2023-09-20 12:19:58 -04:00
1de5d6d754
Update DEPLOYMENT.md
...
command needs sudo permissions or otherwise errors out
2023-09-20 12:33:20 +02:00
fc9055ebe1
release 0.9.5
2023-09-18 12:05:54 -04:00
4d0eef45f7
use negentropy protocol 0
2023-09-18 11:57:35 -04:00
7da862eb91
only attempt to monitor the modification time of a plugin if the command does not contain spaces
2023-09-10 09:53:36 -04:00
b2a128d2f3
Normally when a plugin blocks an event, it will log a message. Especially when using plugins in stream, router, etc, this might be too verbose. In order to silence these logs, return an empty string for msg (or no msg at all).
2023-09-10 09:53:30 -04:00
90494cb8b3
bugfix with plugin receivedAt time and sourceType
2023-09-10 09:53:15 -04:00
20148b1d7d
missing include
2023-09-06 15:17:03 -04:00
f81f628dc5
use uWebSockets toStringView()
2023-09-06 15:16:39 -04:00
d05134ed3d
changelog
2023-09-05 23:43:47 -04:00
cbac2f332e
docs for router
2023-09-05 23:41:07 -04:00
108ebbfaff
router
2023-09-05 23:41:07 -04:00
2145214b9b
bump golpe
2023-09-05 23:41:07 -04:00
e7faa0f813
WSConnection improvements: better destruction, logging
2023-09-05 23:41:07 -04:00
8a8637ee25
in plugins, pass through environment and use /bin/sh to interpret command so you can have arguments, shell pipelines, etc
2023-09-05 23:41:07 -04:00
98f1020ef3
re-org plugins
2023-09-05 23:35:27 -04:00
93330c4cbc
remove lookbehind feature from plugins
2023-09-05 23:35:27 -04:00
11f6eb47a9
Merge pull request #20 from v0l/master
...
Automated docker builds
2023-09-05 08:10:18 -04:00
b82b9204f5
Merge pull request #65 from theAkito/docker-alpine
...
Add Alpine Docker
2023-09-04 16:01:44 -04:00
c23b10e1d2
fix 'unable to lookup event by levId' crash
...
This happened when a non-indexOnly scan was paused and then one of the buffered levIds was deleted or replaced before the scan resumed
2023-09-04 15:44:35 -04:00
6216920153
release 0.9.4
2023-09-02 09:56:30 -04:00
b8b5c01d0f
bump negentropy for some bugfixes
2023-09-02 09:55:59 -04:00
a16e214f1b
Optimise Dockerfile
2023-08-25 15:20:44 +02:00
2d422b9803
Add Alpine Master Image
2023-08-25 14:50:46 +02:00
96b2d13307
Backup Ubuntu Image
2023-08-25 14:50:08 +02:00
42fe1f16ca
WSConnection clean shutdown, fix hubTrigger ownership (it's deleted by the event loop)
2023-08-16 13:38:13 -04:00
aec8dbc33a
WriterPipeline clean shutdown
2023-08-16 13:31:25 -04:00
a4688027b2
todo
2023-08-16 13:03:37 -04:00
58fca0038e
default for noReadAhead is false
2023-08-14 15:33:41 -04:00
35270c57ad
efficiency: not necessary to look up Event table in export
2023-08-14 06:20:35 -04:00
0227b9f11d
todo
2023-08-14 06:16:28 -04:00
44d07da4cd
todo
2023-08-12 00:45:51 -04:00
6b8981e1db
sync improvements and incompatible upgrade
...
* New Negentropy protocol: Syncs with older versions of strfry will break (sorry!)
* Bi-directional frame size limits: fixes a bug when syncing very large DBs where the client side was near empty
* sync command now politely tells server it can release Negentropy resources as soon as reconcile complete
2023-08-12 00:45:51 -04:00
d189cdb349
correctly detect and report when syncs are too large, fix logging
2023-08-12 00:45:51 -04:00
a199e09b84
Merge pull request #54 from jaschadub/master
...
Update deployment documentation
2023-08-11 13:34:16 -04:00
276b243ffd
noReadAhead option, CLOEXEC the LMDB file
2023-08-10 16:36:26 -04:00
206b14a473
sync optimisations, DBQuery no longer loads eventPayload
...
- It is now up to the caller to do so
- QueryScheduler now can optionally not bother to ensure that the events are fresh
2023-07-29 01:14:38 -04:00
94a60c3ad2
bump golpe
2023-07-29 01:11:09 -04:00
ab03a57b18
stop WriterPipeline threads on shutdown
2023-07-27 17:50:07 -04:00
ca0449286c
Stop WSConnection from calling ::exit(), so that the sync command can flush its writes first
...
This fixes a rare race condition that caused a segfault on exit:
We call ::exit() which runs the atexit handler, which destroys the LMDB environment. However, there was another thread running that still had pending events to write, so as the environment was being destroyed, it tried to use the environment to write the events.
2023-07-26 20:24:55 -04:00
0fe929ffcb
make bulk export commands gracefully exit on SIGPIPE (for example, if piped to head)
2023-07-25 00:39:05 -04:00
80915f969d
If a client disconnects before its pending EVENT write messages have been processed, drop those messages instead of trying to write them
2023-07-21 07:46:37 -04:00
7c17b066a1
release 0.9.3
2023-07-21 06:38:17 -04:00
5b3190c54a
detect overflows when rejectEvents* config params are set very large
2023-07-21 06:36:36 -04:00
6d8aeaa881
setting rejectEphemeralEventsOlderThanSeconds larger than ephemeralEventsLifetimeSeconds did not work because events were then considered expired
2023-07-21 06:23:03 -04:00
