Kieran/void-cat-rs
2
0
Fork 1
Code Issues 4 Pull Requests 1 Packages Projects Releases Wiki Activity

Docker-free instruction #2

New Issue
Open
opened 2024-08-30 10:50:38 +00:00 by sjors · 20 comments
sjors commented 2024-08-30 10:50:38 +00:00
Copy Link

I'll see if I can figure it out...

I'll see if I can figure it out...
Kieran commented 2024-08-30 12:16:28 +00:00
Owner
Copy Link

Are you looking for a systemd-like setup?

Are you looking for a `systemd`-like setup?
sjors commented 2024-08-30 13:50:19 +00:00
Author
Copy Link

Indeed, I usually have it run with systemd and nginx. I assume it's just something like cargo build...

Indeed, I usually have it run with systemd and nginx. I assume it's just something like cargo build...
sjors commented 2024-08-30 15:29:12 +00:00
Author
Copy Link

What I got so far:

  • install MariaDB, create user
  • sudo apt install build-essential libx264-dev libwebp-dev libvpx-dev nasm libclang-dev libwebp7
    • libvpx7 doesn't exist on Ubuntu 24.04
  • manually build your FFmpeg fork:
    Dockerfile Lines 16 to 30 in 79338256df
    RUN git clone --depth=1 https://git.v0l.io/Kieran/FFmpeg.git && \
    cd FFmpeg && \
    ./configure \
    --prefix=$FFMPEG_DIR \
    --disable-programs \
    --disable-doc \
    --disable-network \
    --enable-gpl \
    --enable-version3 \
    --enable-libx264 \
    --enable-libwebp \
    --enable-libvpx \
    --disable-static \
    --enable-shared && \
    make -j8 && make install
  • use --prefix=$HOME instead
  • PKG_CONFIG_PATH=$HOME/lib/pkgconfig RUSTFLAGS='-L $HOME/lib' cargo build -r

Try out with LD_LIBRARY_PATH=$HOME/lib target/release/void_cat

What I got so far: * install MariaDB, create user * `sudo apt install build-essential libx264-dev libwebp-dev libvpx-dev nasm libclang-dev libwebp7` * `libvpx7` doesn't exist on Ubuntu 24.04 * manually build your FFmpeg fork: https://git.v0l.io/Kieran/void-cat-rs/src/commit/79338256dfa0d4511c80ac3e413642c48a81dce8/Dockerfile#L16-L30 * use `--prefix=$HOME` instead * `PKG_CONFIG_PATH=$HOME/lib/pkgconfig RUSTFLAGS='-L $HOME/lib' cargo build -r` Try out with `LD_LIBRARY_PATH=$HOME/lib target/release/void_cat`
Kieran commented 2024-08-30 15:56:08 +00:00
Owner
Copy Link

Yea it should be whatever is in the dockerfile, the ffmpeg fork is not required afaik, its just so i can compile a smaller binary for inside docker.

The main distro release of ffmpeg should also work.

After that you just build with cargo build --release

Then files should be in ./target/release

Yea it should be whatever is in the dockerfile, the ffmpeg fork is not required afaik, its just so i can compile a smaller binary for inside docker. The main distro release of ffmpeg should also work. After that you just build with `cargo build --release` Then files should be in `./target/release`
Kieran added the
question
 label 2024-08-30 16:01:31 +00:00
sjors commented 2024-08-30 16:16:58 +00:00
Author
Copy Link

Ah that sounds a lot simpler.

Do you have sample nginx file? Especially with regard to which location entries need which CORS headers.

(though I guess that's already handled by cors.rs)

Ah that sounds a lot simpler. Do you have sample nginx file? Especially with regard to which `location` entries need which CORS headers. (though I guess that's already handled by `cors.rs`)
Kieran commented 2024-08-30 16:23:19 +00:00
Owner
Copy Link

Yes it should be a simple proxy_pass just make sure to set the client_max_body_size and proxy_read_timeout

Yes it should be a simple `proxy_pass` just make sure to set the `client_max_body_size` and `proxy_read_timeout`
Kieran commented 2024-08-30 16:25:16 +00:00
Owner
Copy Link

Yes it should be a simple proxy_pass just make sure to set the client_max_body_size and proxy_read_timeout

If this doesnt work i can get you the same config as https://blossom.nogood.studio/

> Yes it should be a simple `proxy_pass` just make sure to set the `client_max_body_size` and `proxy_read_timeout` If this doesnt work i can get you the same config as https://blossom.nogood.studio/
sjors commented 2024-08-30 16:34:29 +00:00
Author
Copy Link

I think it's working, though I'm getting a Invalid auth string when trying to post with https://nostrify.dev/upload/blossom - it did work with https://blossom.primal.net/. It doesn't work with https://blossom.nogood.studio/ either, so maybe a protocol issue?

I think it's working, though I'm getting a `Invalid auth string` when trying to post with https://nostrify.dev/upload/blossom - it did work with `https://blossom.primal.net/`. It doesn't work with `https://blossom.nogood.studio/` either, so maybe a protocol issue?
Kieran commented 2024-08-30 16:36:23 +00:00
Owner
Copy Link

Thats weird, ill look into this:

src/auth/blossom.rs Line 27 in 79338256df
return Outcome::Error((Status::new(403), "Invalid auth string"));

Thats weird, ill look into this: https://git.v0l.io/Kieran/void-cat-rs/src/commit/79338256dfa0d4511c80ac3e413642c48a81dce8/src/auth/blossom.rs#L27
sjors commented 2024-08-30 16:41:30 +00:00
Author
Copy Link

I think it's because you require NIP44 while Primal is either unauthenticated or NIP04.

(update: neh, probably not)

I think it's because you require NIP44 while Primal is either unauthenticated or NIP04. (update: neh, probably not)
Kieran commented 2024-08-30 16:56:48 +00:00
Owner
Copy Link

Its most likley an issue for nostrify because it already works everywhere else like https://bouquet.slidestr.net/ and snort.social

Its most likley an issue for nostrify because it already works everywhere else like https://bouquet.slidestr.net/ and snort.social
Kieran commented 2024-08-30 17:01:34 +00:00
Owner
Copy Link

Its most likley an issue for nostrify because it already works everywhere else like https://bouquet.slidestr.net/ and snort.social

I suspect it could be missing base64 padding

> Its most likley an issue for nostrify because it already works everywhere else like https://bouquet.slidestr.net/ and snort.social I suspect it could be missing base64 padding
sjors commented 2024-08-30 17:06:20 +00:00
Author
Copy Link

Nostrify is adding a spurious } at the end of the auth string. I'll make a PR to remove that.

I briefly worked around it by stripping the last character, though then something else goes wrong:

 INFO  rocket::server    > PUT /upload image/png:
 INFO  rocket::server::_ > Matched: (upload) PUT /upload
 INFO  void_cat::auth::blossom > {"kind":24242,"content":"Upload undefined","created_at":1725037429,"tags":[["t","upload"],["x","6417c1c55b69008abd326ad333ef522669fe895f77244e82e1a31963b7e03802"],["size","50492"],["expiration","1725037489"]],"pubkey":"524c5b89b716ce509734e81737cf0db6b2e50d604dd4030e98925259c818c621","id":"1b94c931c994085283265e06103dcdac994b2c80ae6f817c03d86025afe1ed5b","sig":"6d5d651adfaa36e4d16e206a1f7b071c00c4016907799e34653b6767c050bc60714ad0f3f255872bf320659c78933e99d0665689111670eac34bd2c37b4614e8"}
 INFO  void_cat::filesystem    > File saved to temp path: /tmp/a65a9378-91c4-4b54-b4fe-63dc9d17ca2f
 INFO  rocket::server::_       > Outcome: Success(500 Internal Server Error)
 INFO  rocket::server::_       > Response succeeded.
Nostrify is adding a spurious `}` at the end of the auth string. I'll make a PR to remove that. I briefly worked around it by stripping the last character, though then something else goes wrong: ``` INFO rocket::server > PUT /upload image/png: INFO rocket::server::_ > Matched: (upload) PUT /upload INFO void_cat::auth::blossom > {"kind":24242,"content":"Upload undefined","created_at":1725037429,"tags":[["t","upload"],["x","6417c1c55b69008abd326ad333ef522669fe895f77244e82e1a31963b7e03802"],["size","50492"],["expiration","1725037489"]],"pubkey":"524c5b89b716ce509734e81737cf0db6b2e50d604dd4030e98925259c818c621","id":"1b94c931c994085283265e06103dcdac994b2c80ae6f817c03d86025afe1ed5b","sig":"6d5d651adfaa36e4d16e206a1f7b071c00c4016907799e34653b6767c050bc60714ad0f3f255872bf320659c78933e99d0665689111670eac34bd2c37b4614e8"} INFO void_cat::filesystem > File saved to temp path: /tmp/a65a9378-91c4-4b54-b4fe-63dc9d17ca2f INFO rocket::server::_ > Outcome: Success(500 Internal Server Error) INFO rocket::server::_ > Response succeeded. ```
Kieran commented 2024-08-30 17:08:12 +00:00
Owner
Copy Link

What does your config.toml look like, there should always be an error message too, check the response in nostrify.

What does your `config.toml` look like, there should always be an error message too, check the response in nostrify.
sjors commented 2024-08-30 17:18:38 +00:00
Author
Copy Link

In nostrify I get this:

error: Uncaught (in promise) AggregateError: All promises were rejected
    ZodError: [
      {
        "code": "invalid_type",
        "expected": "string",
        "received": "undefined",
        "path": [
          "url"
        ],
        "message": "Required"
      },
      {
        "code": "invalid_type",
        "expected": "string",
        "received": "undefined",
        "path": [
          "sha256"
        ],
        "message": "Required"
      },
      {
        "code": "invalid_type",
        "expected": "number",
        "received": "undefined",
        "path": [
          "size"
        ],
        "message": "Required"
      }
    ]

Config:

# Listen address for webserver
listen = "127.0.0.1:8090"

# Database connection string (MYSQL)
database = "mysql://void-cat:...@localhost:3306/void_cat"

# Directory to store uploads
storage_dir = "$HOME/.void-cat"

# Maximum support filesize for uploading
max_upload_bytes = 5e+9

# Public facing url
public_url = "https://img.sprovoost.nl"

# Whitelisted pubkeys, leave out to disable
whitelist = [
  "8685ebef665338dd6931e2ccdf3c19d9f0e5a1067c918f22e7081c2558f8faf8", 
  "524c5b89b716ce509734e81737cf0db6b2e50d604dd4030e98925259c818c621" # Demo 
]

# Path for ViT(224) image model (https://huggingface.co/google/vit-base-patch16-224)
# vit_model_path = "model.safetennsors"

# Webhook api endpoint
webhook_url = "https://api.snort.social/api/v1/media/webhook"

I'll try switching to the Ubuntu version of ffmpeg to see if that helps. I assume it's just libavutil-dev?

By the way, I don't know that the webhook_url config is for so haven't touched the default.

In nostrify I get this: ```json error: Uncaught (in promise) AggregateError: All promises were rejected ZodError: [ { "code": "invalid_type", "expected": "string", "received": "undefined", "path": [ "url" ], "message": "Required" }, { "code": "invalid_type", "expected": "string", "received": "undefined", "path": [ "sha256" ], "message": "Required" }, { "code": "invalid_type", "expected": "number", "received": "undefined", "path": [ "size" ], "message": "Required" } ] ``` Config: ```toml # Listen address for webserver listen = "127.0.0.1:8090" # Database connection string (MYSQL) database = "mysql://void-cat:...@localhost:3306/void_cat" # Directory to store uploads storage_dir = "$HOME/.void-cat" # Maximum support filesize for uploading max_upload_bytes = 5e+9 # Public facing url public_url = "https://img.sprovoost.nl" # Whitelisted pubkeys, leave out to disable whitelist = [ "8685ebef665338dd6931e2ccdf3c19d9f0e5a1067c918f22e7081c2558f8faf8", "524c5b89b716ce509734e81737cf0db6b2e50d604dd4030e98925259c818c621" # Demo ] # Path for ViT(224) image model (https://huggingface.co/google/vit-base-patch16-224) # vit_model_path = "model.safetennsors" # Webhook api endpoint webhook_url = "https://api.snort.social/api/v1/media/webhook" ``` I'll try switching to the Ubuntu version of `ffmpeg` to see if that helps. I assume it's just `libavutil-dev`? By the way, I don't know that the `webhook_url` config is for so haven't touched the default.
Kieran commented 2024-08-30 17:20:14 +00:00
Owner
Copy Link

I dont think $HOME/.void-cat will work, its probably not expanding ENV vars

You can comment out webhook_url this is to allow authorization of file upload, via external API. (docs needed)

I assume it's just libavutil-dev?

You need libavcodec-dev libavformat-dev libswscale-dev libavutil-dev (im probably missing some)

I dont think `$HOME/.void-cat` will work, its probably not expanding ENV vars You can comment out `webhook_url` this is to allow authorization of file upload, via external API. (docs needed) > I assume it's just libavutil-dev? You need `libavcodec-dev libavformat-dev libswscale-dev libavutil-dev` (im probably missing some)
👍 1
sjors commented 2024-08-30 17:27:24 +00:00
Author
Copy Link

It needed a bit more:

sudo apt install libavutil-dev libavformat-dev libavfilter-dev libavdevice-dev
It needed a bit more: ```sh sudo apt install libavutil-dev libavformat-dev libavfilter-dev libavdevice-dev ```
sjors commented 2024-08-30 17:40:50 +00:00
Author
Copy Link

So I'm not sure which of the above things fixed it, but it works!

Here's the workaround for Nostrify, pending https://gitlab.com/soapbox-pub/nostrify/-/merge_requests/91

diff --git a/src/auth/blossom.rs b/src/auth/blossom.rs
index 1403400..3e579c0 100644
--- a/src/auth/blossom.rs
+++ b/src/auth/blossom.rs
@@ -17,7 +17,11 @@ impl<'r> FromRequest<'r> for BlossomAuth {
     async fn from_request(request: &'r Request<'_>) -> Outcome<Self, Self::Error> {
         return if let Some(auth) = request.headers().get_one("authorization") {
             if auth.starts_with("Nostr ") {
-                let event = if let Ok(j) = BASE64_STANDARD.decode(auth[6..].to_string()) {
+                let mut auth_len = auth.len();
+                if auth.chars().last().unwrap() == '}' {
+                    auth_len = auth_len - 1;
+                }
+                let event = if let Ok(j) = BASE64_STANDARD.decode(auth[6..auth_len].to_string()) {
                     if let Ok(ev) = Event::from_json(j) {
                         ev
                     } else {
So I'm not sure which of the above things fixed it, but it works! Here's the workaround for Nostrify, pending https://gitlab.com/soapbox-pub/nostrify/-/merge_requests/91 ```diff diff --git a/src/auth/blossom.rs b/src/auth/blossom.rs index 1403400..3e579c0 100644 --- a/src/auth/blossom.rs +++ b/src/auth/blossom.rs @@ -17,7 +17,11 @@ impl<'r> FromRequest<'r> for BlossomAuth { async fn from_request(request: &'r Request<'_>) -> Outcome<Self, Self::Error> { return if let Some(auth) = request.headers().get_one("authorization") { if auth.starts_with("Nostr ") { - let event = if let Ok(j) = BASE64_STANDARD.decode(auth[6..].to_string()) { + let mut auth_len = auth.len(); + if auth.chars().last().unwrap() == '}' { + auth_len = auth_len - 1; + } + let event = if let Ok(j) = BASE64_STANDARD.decode(auth[6..auth_len].to_string()) { if let Ok(ev) = Event::from_json(j) { ev } else { ```
Kieran commented 2024-08-30 17:42:40 +00:00
Owner
Copy Link

Nice! Probably configs was the issue.

Nice! Probably configs was the issue.
sjors commented 2024-08-30 20:23:12 +00:00
Author
Copy Link

Here's an nginx example:

server {
    server_name image.example.com;

    location / {
        proxy_pass http://127.0.0.1:8000;
	    proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # For websockets (nip96)
	    # proxy_http_version 1.1;
	   # proxy_set_header Upgrade $http_upgrade;
	   # proxy_set_header Connection "Upgrade"; 
    }


    client_max_body_size 10M;
    proxy_read_timeout 70;

    listen 80;
    listen [::]:80;
}

The three commented lines are needed for nip-96, but I haven't tested that.

Use certbot to add the necessary SSL certificate lines.

And here's a simple systemd config:

[Unit]
Description=Void cat

[Service]
Type=simple
User=nostr
Group=nostr
WorkingDirectory=/home/nostr/src/void-cat-rs
Environment="RUST_LOG=info"
ExecStart=/home/nostr/src/void-cat-rs/target/release/void_cat

[Install]
WantedBy=multi-user.target
Here's an nginx example: ```nginx server { server_name image.example.com; location / { proxy_pass http://127.0.0.1:8000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; # For websockets (nip96) # proxy_http_version 1.1; # proxy_set_header Upgrade $http_upgrade; # proxy_set_header Connection "Upgrade"; } client_max_body_size 10M; proxy_read_timeout 70; listen 80; listen [::]:80; } ``` The three commented lines are needed for nip-96, but I haven't tested that. Use certbot to add the necessary SSL certificate lines. And here's a simple systemd config: ```systemd [Unit] Description=Void cat [Service] Type=simple User=nostr Group=nostr WorkingDirectory=/home/nostr/src/void-cat-rs Environment="RUST_LOG=info" ExecStart=/home/nostr/src/void-cat-rs/target/release/void_cat [Install] WantedBy=multi-user.target ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Kieran/void-cat-rs#2
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?