litert-community
/

decider-2b-vision-LiteRT

@@ -84,7 +84,7 @@ python -B reference/runtime_example.py --bundle decider-2b-vision_fp16-int8vocab
 [runtime_example.py](reference/runtime_example.py) builds the prompt with the upstream prompt code, resizes the image to 256×256 with PIL, sends `[image, text]` as one user message and reads the first generated token with greedy decoding. This path covers one image first, one question per request, and the argmax only. The runtime returns text, not option probabilities, and its greedy token is the most likely token over the whole vocabulary, not only over the option letters. The two agree on every single-question published row. In one two-question row, the most likely token for a two-option question is "C".
-Several questions about one image go into one conversation: the image with the first question, then each further question as a text-only message that starts with `)` (it closes the previous letter) and carries the next `Question: … Options: … Answer: (` block. Do not open a new conversation per request on the same engine: LiteRT-LM keeps this model's linear-attention state across conversations ([LiteRT-LM #3165](https://github.com/google-ai-edge/LiteRT-LM/issues/3165)), and the answers drift. Measured on a Galaxy S26 through the Android library (litertlm-android 0.16.1, the int8 file, 2026-09-29): one conversation with five questions per bar chart gave the answer of a fresh engine on 18 of 20 questions on the GPU and 5 of 5 on the CPU; a new conversation per request gave it on 33 of 60 (GPU) and 34 of 60 (CPU) with pixel-identical inputs. An app that does this, with its records: [hfmodels-android `samples/ask`](https://github.com/john-rocky/hfmodels-android/tree/main/samples/ask).
 ## Examples

 [runtime_example.py](reference/runtime_example.py) builds the prompt with the upstream prompt code, resizes the image to 256×256 with PIL, sends `[image, text]` as one user message and reads the first generated token with greedy decoding. This path covers one image first, one question per request, and the argmax only. The runtime returns text, not option probabilities, and its greedy token is the most likely token over the whole vocabulary, not only over the option letters. The two agree on every single-question published row. In one two-question row, the most likely token for a two-option question is "C".
+Several questions about one image go into one conversation: the image with the first question, then each further question as a text-only message that starts with `)` (it closes the previous letter) and carries the next `Question: … Options: … Answer: (` block. Do not open a new conversation per request on the same engine: LiteRT-LM keeps this model's linear-attention state across conversations ([LiteRT-LM #3165](https://github.com/google-ai-edge/LiteRT-LM/issues/3165)), and the answers drift. Measured on a Galaxy S26 through the Android library (litertlm-android 0.16.1, the int8 file, 2026-09-29): one conversation with five questions per bar chart gave the answer of a fresh engine on 18 of 20 questions on the GPU and 5 of 5 on the CPU; a new conversation per request gave it on 33 of 60 (GPU) and 34 of 60 (CPU) with pixel-identical inputs.
 ## Examples