Implemented and tested
Local model interaction
Integrated an Ollama-hosted Qwen model into a Python-driven assistant with typed conversation and command handling.
Featured independent project · Active development
Project Eli
A local-first AI assistant built to bring language models, conversational memory, speech recognition, and spoken responses into one practical desktop experience.
Project at a glance
Language & platform
Python · Windows 11
A Python application developed and tested on a Windows desktop, using a virtual environment for dependencies.
AI inference
Ollama · Qwen3 8B
Locally hosted language-model inference, with development and performance testing on an NVIDIA RTX 2070 (8 GB).
Interaction
Text · Speech · Memory
Text-based commands, Whisper-powered speech recognition, text-to-speech output, and a persistent memory workflow.
The problem I'm exploring
AI assistants are often experienced as remote chat services. Project Eli explores what changes when an assistant runs on personal hardware and the developer controls its behavior, voice interface, memory, and safeguards. My goal is to develop a usable application while gaining hands-on experience integrating AI components into a complete software system.
Development scope: This is a personal prototype, not a finished commercial product. Features described below reflect implementation and testing work; future ideas are identified separately.
Technical design
System architecture
The assistant connects user input to a Python application layer, which coordinates local model inference, optional speech processing, and memory retrieval or storage.
01 · InputKeyboard or microphoneTyped messages or captured speech
02 · ProcessingPython applicationCommands, conversation flow, Whisper transcription
03a · ReasoningOllama + Qwen3:8bLocal language-model responses
03b · ContextPersistent memoryStore and retrieve user-directed facts
04 · OutputText and speechConsole response and text-to-speech playback
Conceptual overview of the development architecture, not a claim of production readiness or independent security validation.
Features and implementation
Implemented and tested
Persistent memory
Added a memory workflow that can store information explicitly provided by the user and reuse it during later interactions.
Implemented; refining reliability
Speech input and output
Connected speech recognition and text-to-speech functionality. Device selection, playback behavior, and transcription responsiveness have required repeated troubleshooting.
Design and refinement
Configurable safeguards
Exploring distinct configurations for personal experimentation and potential wider release, including stronger boundaries for harmful requests and context-sensitive handling of satire.
Engineering challenges
Audio hardware and device compatibility
Microphone enumeration and Windows audio backends produced invalid-device and stream errors during testing. Debugging involved inspecting available inputs, testing headset and webcam microphones, and adjusting device selection.
Speech responsiveness and playback
Voice interaction introduced latency, recognition interruptions, and a text-to-speech issue where only the first reply was spoken. Iterative changes improved follow-up speech playback; smooth end-to-end voice interaction remains an active focus.
Local inference and GPU configuration
Working with a local 8B model required attention to available GPU memory, drivers, inference setup, and performance tradeoffs. This work connects model deployment with practical desktop hardware constraints.
State, safety, and user control
Persistent memory and adjustable safeguards introduce questions about what should be stored, how users should control behavior, and how an assistant should distinguish benign context from risky instructions. These remain design priorities as the prototype evolves.
Build log
Development milestones
- Foundation · Completed
Local development environment
Set up Python, a virtual environment, Ollama, and the local Qwen model.
- Core assistant · Implemented
Conversation and command loop
Established typed interaction and command discovery, including an in-app help workflow.
- Memory · Implemented
Remember and recall
Added explicit memory storage and retrieval behavior for user-provided information.
- Voice · Iterative testing
Speech recognition and spoken replies
Integrated speech tools and addressed audio input selection and repeated-response playback issues.
- Next · Planned
Reliability, documentation, and demonstration
Continue refining voice performance and safety behavior, document the setup, and add verified screenshots or a demo when ready.
What this project demonstrates
Systems integration
Connecting model inference, application logic, speech tools, hardware interfaces, and persistent data into a single workflow.
Technical troubleshooting
Isolating issues across drivers, audio devices, Python dependencies, GPU configuration, and application state.
Iterative development
Testing real interactions, diagnosing failures, revising implementations, and keeping unfinished capabilities clearly identified.
Responsible product thinking
Considering privacy, local execution, user control, and safeguards early in the design process.
Looking ahead
Project Eli is both a practical learning platform and an early step toward my longer-term interest in developing AI-powered products and services. The next phase centers on reliability, clearer technical documentation, and a demonstrable prototype—not on presenting unfinished features as complete.