Build a Desktop AI Vision Assistant With Electron

Updated 9 November 2026 · Standard formula, shown below

A desktop AI vision assistant can capture an image or screen region, send it to a vision-capable model, and show a text response in an Electron window. This guide builds a small app with a safe Electron process boundary, a user-selected image, and a backend that keeps the model API key out of the desktop interface.

What You’ll Need

A supported Node.js installation with npm, plus a code editor and terminal.
Basic familiarity with JavaScript, HTML, and installing packages.
A vision-capable AI service or a local model that accepts images, and its documentation. If using a hosted service, create an API key and plan to store it on a server, not in the app.
A computer on which you can run and test an Electron application. Check the model service’s terms and image-handling policies before sending private images.

Step-by-Step

Create the project. Make a folder, open a terminal in it, then run npm init -y and npm install --save-dev electron. Add a start script to package.json: "start": "electron .". Set its main field to main.js.
Set up Electron’s main process. Create main.js. Import app, BrowserWindow, and ipcMain from electron, and path from Node. Create a window after the app is ready and load index.html. Use restrictive settings: nodeIntegration: false, contextIsolation: true, and a preload file. Keep these protections enabled.
Expose a narrow preload API. Create preload.js, import contextBridge and ipcRenderer, then expose one function such as analyzeImage(imageData, question) with contextBridge.exposeInMainWorld. Have it invoke a named IPC handler. Do not expose unrestricted Node, filesystem, shell, or general IPC access to page code.
Build the interface. In index.html, add a file input accepting images, a text field for the question, an Analyze button, and a status/result area. Load a separate renderer script. Use the File API to read the selected file, check that it is an image and within a reasonable size limit, then convert it to a data URL or another format your backend supports. Show a clear error if no image is selected.
Validate the IPC request. In main.js, register the matching ipcMain.handle channel. Check the image data format and length, and limit the question’s length. Reject unexpected values rather than forwarding arbitrary renderer input. Return a useful error message without exposing secrets or internal stack traces.
Connect to a model safely. For a real app, have the main process call your own HTTPS backend, which stores the provider key and calls the vision model. Send only the image and prompt needed for the task. Your backend should authenticate the app as appropriate, enforce size and request limits, and return a simple response object. Do not place a provider key in renderer JavaScript, preload code, source control, or a packaged app; packaged client secrets can be extracted.
Display the answer and handle failures. In the renderer, disable the button while a request is in progress, show progress, and restore the button when it finishes. Render returned text with textContent, not innerHTML, so model output is not treated as markup. Handle network errors, rejected images, rate limits, and empty responses with actionable messages.
Run and test locally. Start your backend, then run npm start. Test a small, ordinary image, an unsupported file, a very large image, a blank question, and a failed network request. Confirm that requests reach only the intended backend and that no key appears in developer tools or application files.
Add screen capture only if needed. For a screenshot workflow, use Electron’s documented desktop-capture APIs and ask the user to choose what to share. Request operating-system permissions where required, show a preview before upload, and provide a cancel option. Avoid capturing continuously or sending screenshots without a deliberate user action.
Package after reviewing security. Choose an Electron packaging tool, add the required build configuration, and test the packaged app on each target operating system. Keep Electron and dependencies updated, review permissions, and ensure logs do not retain image contents or credentials.

Common Mistakes

Putting the API key in the app: Environment variables used during development do not make a secret safe once embedded in a distributed client. Use a backend you control.
Turning on Node integration: This makes a compromised renderer more dangerous. Keep context isolation on and expose only specific preload functions.
Sending unsupported image data: Some model endpoints require a particular MIME type, image URL, or encoded format. Follow the service’s current image-input documentation and reject unsupported files early.
Trusting model output as HTML: Display it as text. Also treat model descriptions as fallible; the assistant can misidentify objects or miss details.
Ignoring privacy and request size: Images can contain personal information. Explain what is sent, ask before capture or upload, and apply sensible file and request limits.

Tips

Start with a user-selected image before adding screen capture; it makes permissions and debugging simpler.
Ask focused questions, such as “Describe the visible chart axes,” rather than requesting unsupported certainty.
Resize large images on the backend or before upload when appropriate, while preserving enough detail for the task.
Keep the model call behind a small function so you can change providers without redesigning the interface.
Offer a clear delete or reset action, and avoid saving images unless the user explicitly needs that feature.

Troubleshooting

If the window is blank, check the terminal for errors, confirm that index.html exists at the path loaded by main.js, and open Electron’s developer tools to inspect renderer errors. If the preload API is undefined, check the preload path and confirm that context isolation is enabled; verify the exposed function name matches the renderer code exactly.

If clicking Analyze does nothing, inspect the IPC channel names and confirm that the handler is registered before the window sends a request. If the backend rejects the request, log status codes and a sanitized error message on the server, then verify its expected image format, payload size, and authentication. Never log API keys or full private images.

If image uploads fail intermittently, test with a smaller file and check network connectivity, timeouts, and service limits. If screen capture is unavailable, check the operating system’s privacy permissions and use a manual image picker as a fallback. Re-test the packaged build: file paths and permissions can differ from development.

Frequently Asked Questions

Can Electron analyze images without an internet connection?
Yes, if you connect it to a compatible local vision model. You will need to install and configure that model separately, and performance depends on the computer.

Can I call a hosted model directly from Electron?
A client can make network requests, but a provider secret inside a distributed app is not protected. Use a backend to hold the key and make the request.

How do I let users ask about their screen?
Add an explicit capture action, show the captured image for review, and send it only after confirmation. Follow Electron and operating-system permission requirements.

Why does the model misunderstand an image?
Image quality, small text, cropping, and model limitations can affect results. Try a clearer crop and a specific prompt, and verify important details yourself.

Should the app save images or answers?
Only if the feature requires it and users understand the choice. If saving is added, document where files go and provide a way to remove them.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.