Add Moondream transformer implementation and example (#1970)

* moondream implementation

* add moondream example

* change config default activation

* Add assets and integrate phi mixformer with example

* Make use of kv cache and fix seq_len bug; Clean up example code

* Add README link to example

* Remove pos_embed scaling; Remove assets; Add to README; Expand VisionConfig

* Delete image

* Use apply instead of forward
This commit is contained in:
Santiago Medina
2024-03-30 23:54:56 -07:00
committed by GitHub
parent 3144150b8d
commit 92f81d2fcb
6 changed files with 602 additions and 0 deletions

View File

@ -24,6 +24,7 @@ pub mod mistral;
pub mod mixformer;
pub mod mixtral;
pub mod mobileone;
pub mod moondream;
pub mod mpt;
pub mod persimmon;
pub mod phi;