"His idea was this: Given a user created prompt meant to be shipped off to a "proper" (i.e. maybe expensive) Generate AI API, can we use tools to help improve the prompt and make it "cheaper" before used. Given we've got AI in the browser via Chrome (ok, we will have it soon), this seemed absolutely possible and I quickly whipped up a demo."
"This assumes a text only prompt, which of course isn't always the case. Multimodal prompts could include many different forms of binary data and optimization could be done in that area as well. For example, a 4000x4000 hires image could be resized and optimized down quite a bit to reduce the size it adds to the context window. Audio could be sped up 10% or so for tasks involving transcription. PDFs could be optimized as well."
The demo takes a user-created text prompt and asks Chrome's Rewriter API to recreate it in a better form. The focus is on cleaning up text-only prompts to reduce token usage before calling an external generative AI service. Multimodal prompts are noted as optimizable too, including resizing high-resolution images, speeding audio for transcription, and compressing PDFs to shrink context windows. Token cost can be estimated locally using a measureInputUsage() method from the Prompt API, avoiding full API calls for comparison. Testing requires Chrome Canary with appropriate flags enabled and optional Enterprise Preview Program access for extra documentation.
Read at Raymondcamden
Unable to calculate read time
Collection
[
|
...
]