Measured on device on an earlier file (edge-compat): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 34.4 tok/s · prefill 573 tok/s · TTFT 380 ms · all 1243 ops delegated (2026-08-24); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 43.7 tok/s · prefill 208 tok/s · TTFT 1.17 s (2026-09-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/internvl3-2b/CARD.md

InternVL3-2B — LiteRT-LM (on-device Vision-Language Model)

OpenGVLab/InternVL3-2B converted to the LiteRT-LM (.litertlm) format for on-device image+text inference with Google's LiteRT-LM runtime (the engine behind the official litert-community/* models, and the same runtime that runs litert-community/FastVLM-0.5B). On 2026-11-15 InternVL3-2B.litertlm was replaced with a file whose prefill/decode graph is rewritten for the GPU, with every weight unchanged. Outside the dated entries under Conversion notes, "this file" means that file and "the previous file" the one it replaced. On a Galaxy S26 GPU this file decodes 46.9 tokens/s and reads a 1,024-token text prompt at 711 tokens/s (LiteRT-LM v0.19.0).

InternVL3-2B is a compact vision-language model: an InternViT vision encoder + pixel-shuffle + MLP projector feeding a Qwen2.5-1.5B language decoder. This bundle runs it through LiteRT-LM's fast_vlm multimodal path — give it an image and a question, get a grounded answer, fully on-device.

File	InternVL3-2B.litertlm (~1.43 GB)
Vision	InternViT-300M encoder + pixel-shuffle + MLP projector, int8 weights — single 448×448 image → 256 image tokens
Decoder	Qwen2.5-1.5B, int4 weights (symmetric, blockwise-32 + OCTAV optimal-clipping); input embedding INT8 (externalized section)
Compute	integer
Context (KV cache)	2048
Image input	resized to 448×448 (ImageNet normalization is baked into the vision encoder)
Base model	OpenGVLab/InternVL3-2B

How to use

1. Install the runtime

pip install litert-lm

2. Run it in one command — this downloads the bundle, encodes your image and answers:

litert-lm run --from-huggingface-repo litert-community/InternVL3-2B InternVL3-2B.litertlm \
  --attachment photo.jpg \
  --prompt "Describe this picture in one sentence." --top-k 1

On the COCO photo 000000039769.jpg (640 × 480, two tabby cats on a pink blanket), this file answers:

Two cats are sleeping on a pink blanket.

This answer came from litert-lm 0.19.2 on a Mac (2026-11-15): in two runs of this file and one of the previous file, each given by its path (this file: the copy that was uploaded to this repo, with the same sha256; the previous file: downloaded from this repo), and in one run of the command exactly as above, which downloaded this file from this repo. With --backend cpu --vision-backend cpu --cpu-thread-count 1 --cache no --max-num-tokens 2048 added, litert-lm 0.19.0, 0.19.1 and 0.19.2 gave the same answer with both files (2026-11-14).

Leave --vision-backend unset, as above (the CLI then uses the bundle's own setting). With --backend gpu added and --vision-backend still unset, this file answers the picture (on a Mac with litert-lm 0.19.2, the same sentence; the run also prints one error line, TopK requires src tensor C dimension to be divisible by 4.). With --backend gpu --vision-backend gpu, neither this file nor the previous file answers a picture on a Mac (litert-lm 0.19.0 to 0.19.2; the reply is An error occurred); with litert-lm 0.19.2 the error log names two op types of the vision encoder that the GPU delegate does not take, BROADCAST_TO and GATHER_ND, and the runtime does not run the encoder split between the GPU and the CPU. On a Galaxy S26 and a Pixel 11 Pro (LiteRT-LM v0.19.0), litert_lm_advanced_main with the vision encoder on the GPU stops too (see the Galaxy S26 section).

If run stops right after the download with no error message from the CLI (a bus error; exit status 138 on a Mac) or with an error log that contains too large for file size, the downloaded file may be incomplete. The litert-lm CLI (0.14.0 to 0.19.2, and litert-lm-nightly 0.20.0.dev20261112) can keep an incomplete download in ~/.litert-lm/cache/huggingface/litert-community/InternVL3-2B/ and uses it again on later runs without checking its size. Delete that file and run the command again.

Drop --prompt for an interactive chat, and pass --attachment more than once for several images. litert-lm serve exposes the same bundle as a local OpenAI-compatible API. The same file runs on macOS, Linux and Windows.

Performance

Measured on an earlier file: litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine. These figures cover the text path; the vision encoder runs once per image and is not included.

Device	Backend	Prefill (256)	Decode	TTFT
Apple M4 Max (macOS)	CPU	157 tok/s	50.0 tok/s	1.79 s
iPhone 17 Pro	CPU	—	~20 tok/s	—
iPhone 17 Pro	GPU (Metal)	—	~45 tok/s	—

Every desktop backend listed above was checked by actually generating a caption on it, not just by reading the benchmark tool's output.

The iPhone rows are carried over from this repository's own earlier on-device note, where the model loads in about 3–4 s; the run logs are not retained here, so the run count and prompt are not known. Single-image VQA is what the GPU row covers — see the multi-image limitation below.

The macOS GPU backend was not usable for an earlier file of this repo on litert-lm 0.15.0. Asked to describe an image with --backend gpu, it returns a run of ! characters instead of a caption. litert-lm benchmark still reports GPU numbers for it — 2,031 tok/s prefill and 183 tok/s decode — but they measure a path that produces no usable text, so no GPU row is quoted here. With that version, use the CPU backend on the desktop. This is a desktop-runtime observation and says nothing about the iPhone or Android GPU paths described elsewhere on this card. With litert-lm 0.19.0 to 0.19.2 and --backend gpu, this file and the previous one answer a text question with the same reply, byte for byte; with litert-lm 0.19.2 and --backend gpu --vision-backend cpu, both answer the picture under How to use with "Two cats are sleeping on a pink blanket."

Follow-up questions about the same image run in one conversation; from the second turn on, an answer can differ from the answer to the same messages in one prompt (see Known runtime issues).

The image is described accurately and in detail. The vision tower converts faithfully to the reference (float CPU-parity corr ≈ 1.0); int8 vision weights keep grounding quality.

Galaxy S26 — GPU and CPU

Each run reads a 1,024-token text prompt and generates 256 tokens, three times (three iterations); iteration 1 is the first pass after the engine starts. The GPU row is the median of iterations 2–3 over two runs, so it leaves iteration 1 out; on the GPU, iteration 1 read the prompt faster and decoded more slowly than iterations 2 and 3 (this file's iteration 1: prefill 752–755 tok/s, decode 45.5–45.9 tok/s). On the CPU the median prefill and decode speeds drop from one iteration to the next, so each iteration has its own row: the median of that iteration over two runs.

Backend	Prefill, 1,024 tokens	Decode	Time to first token	Peak memory
GPU (LiteRT GPU, every node of the prefill and decode graphs), iterations 2–3	711 tok/s	46.9 tok/s	1.47 s	0.76 GB + 1.56 GB GPU driver
CPU (XNNPACK for most nodes), iteration 1	239 tok/s	36.5 tok/s	4.32 s	1.66 GB
CPU (XNNPACK for most nodes), iteration 2	129 tok/s	31.7 tok/s	7.95 s	1.66 GB
CPU (XNNPACK for most nodes), iteration 3	116 tok/s	31.3 tok/s	8.87 s	1.66 GB

Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16), litert_lm_advanced_main built from LiteRT-LM v0.19.0 (commit 771b06e1), --benchmark with a 1,024-token text prefill and 256 decode tokens, context 2,048, 2026-11-15. Each run is a new process. The rows cover the text path. The GPU runs started at Android's thermal status 0 (none) and each ended at thermal status 1 (light), with the GPU's highest allowed clock one step down. The CPU runs started at thermal status 1 and reached thermal status 2 (moderate) 11–23 s in, where they stayed until they ended.

In each of the four counted CPU runs (two per file, after one warm-up run that is not counted), the phone lowers the clock caps of its two CPU clock domains from 2–6 s in, to 1.56 / 1.50 GHz by 17–27 s in, so iterations 2 and 3 run at a lower clock than iteration 1.

Peak memory is per run, so the three CPU rows share one value. It is the process's VmHWM: on the CPU 1.65–1.66 GB in this file's two counted runs (1.69 GB in the warm-up run) and 1.52–1.54 GB in the previous file's two. VmHWM does not count memory the GPU driver holds, so the GPU cell adds the rise of the phone's GPU allocation counter (kgsl_page_alloc) during each of this file's GPU runs (1.56 GB; 1.33 and 1.34 GB in the previous file's). In the counted CPU runs that counter did not rise with this file and rose by 0.01–0.02 GB with the previous file (0.08 GB in the warm-up run, a run of this file).

With the vision encoder on the GPU (--vision_backend=gpu), litert_lm_advanced_main stops with exit status 13 before it reads the picture: with this file on this phone, and with this file and the previous one on a Pixel 11 Pro. Its log names two op types of the encoder that the GPU delegate does not take, BROADCAST_TO and GATHER_ND (on this phone, 13 of the encoder's 1,320 nodes). With the vision encoder on the CPU (--vision_backend=cpu) and the decoder on the GPU, this phone answered "Describe this picture in one sentence." about the photo under How to use with "Two cats are sleeping on a pink blanket, with a remote control nearby." (one run).

On the GPU this file decodes 8–11 % faster than the previous file. Four runs went previous, this, this, previous (each run's mean of iterations 2–3): prefill 657, 716, 613 and 671 tok/s, decode 43.2, 46.7, 47.2 and 42.5 tok/s; from the previous file to this one, decode ×1.081 and prefill ×1.090 in the first two runs, decode ×1.110 and prefill ×0.915 in the last two. In this file's second run, iteration 2 prefilled at 504 tok/s, against 722–755 tok/s in its other two iterations. On the CPU, this file's prefill and decode medians in each iteration were between 0.7 % below and 0.4 % above those of the previous file; two counted runs of the same file differed by up to 4.3 %. The runs behind each number, and the S26 section this card carried for an earlier file: MEASUREMENTS.md.

On the Galaxy S26 the GPU decodes 1.3× as fast as the CPU's iteration 1 and 1.5× as fast as its iterations 2 and 3, and reads the prompt 3.0× and 5.5–6.1× as fast, so this repo's litertlm_manifest.json recommends gpu for flagship-class Android. That recommendation rests on the Galaxy S26 alone.

GPU wiring, including the Gallery import toggle: GPU guide.

Pixel 11 Pro — GPU

On a Pixel 11 Pro GPU (the same build and workload, 2026-11-14) this file decodes 19.6 tok/s, 16 % faster than the previous file, and prefills 629 tok/s, 7 % faster (the median of iterations 2–3 over two runs per file). With the vision encoder on the GPU, litert_lm_advanced_main stops with exit status 13 with both files (see the Galaxy S26 section); the vision encoder on the CPU was not tried on this phone. The Pixel 11 Pro CPU was not measured, so whether the GPU or the CPU is faster on that phone is not known. The runs: MEASUREMENTS.md.

⚠️ Known limitation — one image per conversation on the GPU (Metal) backend

Single-image VQA — the primary use case — worked in each setup tried with the decoder on the GPU and the vision encoder not on the GPU (one run each: on a Mac with litert-lm 0.19.2, --backend gpu with --vision-backend unset or cpu; on a Galaxy S26, --vision_backend=cpu; see How to use and the Galaxy S26 section). But on the GPU (Metal) backend of Apple devices, a second image in the same conversation truncates the answer — ask about one image per chat (start a new conversation for a different image).

This is GPU-delegate-specific, not a model/bundle issue: on the CPU backend, multi-image works perfectly (verified). The same GPU truncation reproduces with Apple's litert-community/FastVLM-0.5B, so it is general to the runtime's GPU fast_vlm path, not specific to this model. (Ruled out as causes: max_num_images — CPU works with it set to 1; and the vision encoder's 5D reshape — a 4D-clean rebuild still truncates on GPU.) For reliable multi-image, run on the CPU backend. This was seen with an earlier file and was not tried with this file or on Android.

Run on iPhone / macOS

Use the LiteRT-LM Swift runtime (swift-litert-lm / the LiteRTDemo sample). Load InternVL3-2B.litertlm with the image (vision) tower enabled (modalities [.vision]), attach a photo, and ask a question.

Note for app integrators: this is a vision-only bundle (no audio tower). Bring up the engine with the vision modality only (Modality.textImage / [.vision]) — requesting the audio tower (.all) on a bundle with no audio section fails at session creation.

These steps were written for an earlier file and were not run with this one. On a Mac, this file does not answer a picture with the vision encoder on the GPU (--vision-backend gpu; see How to use).

Run on Android — Google AI Edge Gallery

Update (July 2026): Google AI Edge Gallery v1.0.16+ can import litert-lm models directly from Hugging Face inside the app (tap +) — no computer or adb needed. The manual steps below are only required on older builds or for sideloading a local file.

Run this model with image input in the official Google AI Edge Gallery app — no custom app needed (the bundle carries the tokenizer, chat template, and image preprocessing config):

1. Push the bundle onto the phone (or download it there directly from this repo): adb push InternVL3-2B.litertlm /sdcard/Download/
2. Open the Gallery app, tap the + icon (bottom-right) and pick InternVL3-2B.litertlm in the file picker.
3. In the Import Model dialog, check "Support image" (required for image input), pick GPU or CPU, then tap Import.
4. Open the Ask Image task, choose the imported model, attach a photo, and ask.

Tip: the one-image-per-conversation limit above was seen on the GPU (Metal) backend of Apple devices and was not tried on Android; pick CPU if you want multiple images in one chat.

These steps were written for an earlier file and were not repeated with this one, and which backend Gallery gives the vision encoder was not checked. On a Galaxy S26 and a Pixel 11 Pro (LiteRT-LM v0.19.0), litert_lm_advanced_main with the vision encoder on the GPU stops with exit status 13; with the vision encoder on the CPU, the Galaxy S26 answers the picture (see the Galaxy S26 section).

Run on desktop (LiteRT-LM CLI)

The same .litertlm bundle runs on macOS / Linux / Windows with the official LiteRT-LM CLI — including as a local OpenAI-compatible API server:

pip install litert-lm
litert-lm import --from-huggingface-repo litert-community/InternVL3-2B InternVL3-2B.litertlm internvl3-2b
litert-lm run internvl3-2b     # interactive chat in the terminal
litert-lm serve           # local OpenAI-compatible API server

If the run command above stops with no error message from the CLI (a bus error; exit status 138 on a Mac) or with an error log that contains too large for file size, the file that import downloaded may be incomplete. The litert-lm CLI (0.14.0 to 0.19.2, and litert-lm-nightly 0.20.0.dev20261112) can keep an incomplete download in ~/.litert-lm/cache/huggingface/litert-community/InternVL3-2B/ and imports it without checking its size. Delete that file and run the import command again: it downloads a new copy and replaces the imported one.

Conversion notes

• LiteRT-LM fast_vlm bundle: VISION_ENCODER ([1,448,448,3]→[1,256,4096]) + VISION_ADAPTER ([1,256,4096]→[1,256,1536]) + single-token EMBEDDER + PREFILL_DECODE (embeddings-input).
• The vision encoder bakes InternVL's ImageNet normalization and the NCHW transpose into the graph (the runtime feeds a [0,1] NHWC image).
• The InternViT attention is rewritten to be 4D-clean (qkv split before the head reshape, avoiding the 5D reshape(B,N,3,H,d) that GPU delegates reject) — numerically identical (corr ≈ 1.0). With it, 1,307 of the encoder's 1,320 nodes go to the GPU delegate on a Galaxy S26 and a Pixel 11 Pro, but the encoder still does not run on the GPU with LiteRT-LM 0.19 (see How to use).
• Decoder exported with externalized embedder; InternVL's dynamic-NTK rope_scaling is stripped to base RoPE (valid since the export cache ≤ the base context window).

Each dated entry below was written on its date: in it, "this file" is the file published that day, "the previous file" is the one it replaced, and "this card" is the card as it was then.

2026-08-28 — start_token fix (weights unchanged)

The bundle's LlmMetadata start_token held the literal string "None". This tokenizer has no BOS, and the LiteRT-LM engine resolved that string to a real vocabulary token — so every prompt began with the word None, which the model was never trained on. The start token has been removed.

Metadata-only change: every section of the bundle except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file — only the file's own sha256 differs. What changed is the input: the token stream the model reads for a given conversation can differ from the previous file's, and it now matches this model's own reference chat stream. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file in either direction. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-28, re-download.

2026-08-29 — default system prompt restored (weights unchanged)

The upstream chat template emits a default system turn whenever the caller sends no system message — for this model: 你是书生·万象，英文名是InternVL，是由上海人工智能实验室、清华大学及多家合作单位联合开发的多模态大语言模型。. The converter's template probe renders the template with a system message already present, so that block was never seen and never reached the bundle: with no system message the model was running without the default system turn it was tuned with. The chat template in InternVL3-2B.litertlm now emits the block exactly once when no system message is given. In InternVL3-2B.litertlm, the block is not emitted when you pass a system message. The restored block adds 37 prefill tokens to a conversation that sends no system message, so time-to-first-token grows by that much; per-token speed is unchanged.

Metadata-only change: every section of the bundle except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file per token — only the file's own sha256 differs. What changed is the input: with no system message, the prompt now renders byte-identical to the upstream chat template's output, verified on the LiteRT-LM runtime. A system message you pass yourself renders as before. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-29, re-download.

2026-08-30 — tokenizer section replaced (weights unchanged)

The tokenizer in InternVL3-2B.litertlm was a SentencePiece conversion of the model's BPE tokenizer, and the conversion lost the byte-level semantics: a standalone accented letter or symbol (é, ñ, ü, °, ·, …) was encoded to the id of a single-byte token instead of the token the upstream tokenizer uses, and any character without a whole-character vocabulary entry (emoji, most of Latin Extended-A) became the token the conversion had reused as UNK — the end-of-text token for this vocabulary. InternVL3-2B.litertlm now embeds the upstream tokenizer.json (the same HF tokenizer path most bundles in this collection use).

Tokenizer-only change: every section of the bundle except the tokenizer is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the chat template are unchanged and the speed and memory numbers on this card still describe this file per token — only the file's own sha256 differs. Verified on the LiteRT-LM runtime: the default turn, 7 probe strings, the 223 standalone characters U+00A1–U+017F and every special token now tokenize identically to the upstream tokenizer, and the four ASCII-only test questions answer byte-identically to the previous file (same ids in, same tokens out). Prompts containing accented letters, symbols or emoji reach the model differently from before, so individual answers to such prompts can change. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file, and any on-device rows were measured on the previous file too — the on-device check has not been re-run on this one (the runtime's tokenizer code is the same on macOS and on device; the weights and graph are byte-identical). If you downloaded before 2026-08-30, re-download.

2026-08-31 — chat template fix: history turns with role 'assistant' (metadata-only, weights unchanged)

The chat template in InternVL3-2B.litertlm matched history turns only on role 'model', so a conversation created through the Conversation API with the standard role 'assistant' (for example create_conversation(messages=[...]) restoring a chat) silently dropped every assistant turn from the rendered prompt — the model saw consecutive user turns with its own replies gone. The template's role conditions now also accept 'assistant'. The engine's own send_message flow was never affected.

Template-only change: every section of the bundle except the metadata is byte-identical to the previous file (verified by per-section sha256), and single-turn renders, token ids, prefill counts and greedy answers are unchanged (verified on the LiteRT-LM runtime), so the numbers on this card still describe this file — only the file's own sha256 differs. Re-download only matters if you inject conversation history with role 'assistant'. Details: google-ai-edge/LiteRT-LM#3417 (https://github.com/google-ai-edge/LiteRT-LM/issues/3417).

2026-11-15 — GPU graph and a 1,024-token prefill (weights unchanged)

The prefill/decode graph now has the GPU form litert-torch writes with --apply_gpu_composites, plus a 1,024-token prefill signature, prefill_1024: a copy of prefill_512 at a length of 1,024 tokens that shares its weight buffers. Every weight, and every section outside that graph, is byte-identical to the previous file. The whole file went from sha256 44c652d4… (1,429,700,608 B) to 53020750… (1,431,420,928 B). On a Mac the two files give the same replies byte for byte: 4 prompts and a picture on the CPU (litert-lm 0.19.2, 1 thread, greedy decoding with top-k 1), and one prompt and the picture on the CPU and the prompt on the GPU with litert-lm 0.19.0 to 0.19.2. 8 short questions: 8 of 8 with both files on the Mac CPU and GPU and on a Pixel 11 Pro GPU; 8 of 8 with this file on a Galaxy S26 GPU (the previous file was not asked there). Galaxy S26 GPU (LiteRT-LM v0.19.0, four runs in the order previous, new, new, previous; each run's mean of iterations 2–3): prefill 657, 716, 613 and 671 tokens/s, decode 43.2, 46.7, 47.2 and 42.5 tokens/s, so from the previous file to this one prefill ×1.090 and decode ×1.081 in the first pair, prefill ×0.915 and decode ×1.110 in the second. In the second run of this file, iteration 2 prefilled at 504.32 tokens/s (time to first token 2.05 s). Procedure: REPRODUCE.md. Measured on this file: the last sentence of the paragraph under the title, the example answer under How to use and the two paragraphs after it, the last sentence of the macOS GPU paragraph, the Galaxy S26 and Pixel 11 Pro sections, the first sentence of the Known limitation section, the last paragraph of the iPhone / macOS section and of the Gallery section, and this entry. The Known runtime issues below come from earlier runtimes or other files (each says which) and were not re-checked with this file. The rest of this card was written for earlier files; the weights, tokenizer and chat template it describes are the same in this file.

Known runtime issues

External weights (since 2026-11-09). In InternVL3-2B.litertlm, the weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights) instead of inside the model graph. On Android, the GPU path needs LiteRT-LM 0.19.0 or later: the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on such a file (LiteRT-LM#3986). The last file with the weights inside the graph stays at revision 4f65e5a.

From the second turn of a conversation on, LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding, because this file holds the embedding table in its own section and the decoder takes embeddings, so later answers in a conversation can differ from the answer the model gives to the same messages in one prompt (read in the source of LiteRT-LM 0.16.0 to 0.18.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on this file). One question per conversation avoids it, as does a new conversation that starts with the earlier messages and the new question.

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders this file's chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.

On LiteRT-LM 0.18.1 and 0.19.1, once a message fails because it does not fit in the context window, every later message in that conversation fails as well (the next one with Task failed with state: 7). Start a new conversation after such an error. This was measured with other models' files, not with this one.