Match Moondream's latest release (#1997)

* moondream implementation

* add moondream example

* change config default activation

* Add assets and integrate phi mixformer with example

* Make use of kv cache and fix seq_len bug; Clean up example code

* Add README link to example

* Remove pos_embed scaling; Remove assets; Add to README; Expand VisionConfig

* Delete image

* Use apply instead of forward

* Use latest release special token; Fix token/s accuracy; Use GeluPytorchTanh in VisionConfig v2
This commit is contained in:
Santiago Medina
2024-04-02 12:37:09 -07:00
committed by GitHub
parent fb918a23c8
commit d17b2cdad9
2 changed files with 15 additions and 15 deletions

View File

@ -47,7 +47,7 @@ impl VisionConfig {
embed_dim: 1152,
num_blocks: 27,
num_heads: 16,
act: candle_nn::Activation::Gelu,
act: candle_nn::Activation::GeluPytorchTanh,
}
}
}