huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-16 10:38:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Streaming mode for reporting the generated tokens (#1007)

Browse Source 
* Token streaming.

* Use the token output stream.

* Flush the output.

* Ensure that the last characters get reported.
This commit is contained in:
Laurent Mazare
2023-09-30 16:04:11 +02:00
committed by GitHub
parent 4021272875
commit 06207332bc
4 changed files with 96 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
candle-examples/Cargo.toml
View File

@ -25,6 +25,7 @@ rayon = { workspace = true }
safetensors = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
tokenizers = { workspace = true, features = ["onig"] }
[dev-dependencies]
anyhow = { workspace = true }
@ -35,7 +36,6 @@ imageproc = { workspace = true }
memmap2 = { workspace = true }
rand = { workspace = true }
rusttype = { workspace = true }
tokenizers = { workspace = true, features = ["onig"] }
tracing = { workspace = true }
tracing-chrome = { workspace = true }
tracing-subscriber = { workspace = true }