Practical overview
When it fits
Use it when an agent needs to answer questions grounded in a person’s recorded everyday-life media, particularly in a glasses or phone workflow.
Who it is for
- People building personal lifelogging assistants that answer questions about captured media.
- Developers adding memory-grounded question answering to smart-glasses or phone agents.
- Teams exploring long-video question answering over first-person recordings.
Documented uses
- Ask questions about details in recorded everyday experiences.
- Retrieve where an object or event appeared, with timestamps and visual evidence.
- Use a browser or phone capture workflow without installing locally.
- Self-host a web and backend stack, then optionally add frame-level visual retrieval or a local Qwen service.
Requirements & limits
The project documents a hosted browser demo and a self-hosted Docker path. Self-hosting requires an OpenAI-compatible LLM endpoint and Xfyun ASR credentials for speech; the default stack also expects Qwen3-Embedding-4B weights. A Rokid glasses APK is provided as an additional client.
- The standard Docker configuration retrieves from captions and transcripts; frame-level visual matching requires optional model services and GPU resources.
- A local Qwen service via vLLM is an independent optional add-on, not a requirement for the standard setup.
- The repository identifies Python 3.10+, FastAPI, React 19, Vite, Android and Docker in the project stack.
- The project identifies its license as MIT.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up the self-hosted Docker workflow
Start the documented standard web and backend stack for memory-grounded questions.
Before you start
- Docker
- An OpenAI-compatible LLM endpoint with base URL, API key and model names
- Xfyun ASR credentials for speech
- Qwen3-Embedding-4B weights under docker-data/models/
Steps
Clone the LightMem-Ego repository and change into its directory.
Copy deploy/.env.example to .env and enter the LLM endpoint details, keys, model names and Xfyun credentials.
Run docker compose up --build.
Open http://localhost:8080.
Expected outcome
The web interface and backend start as a self-hosted workflow; the web container proxies /api requests to the backend.
Guide limitations
- The default visual backend is mock, so retrieval uses captions and transcripts rather than frame-level visual matching.
- The first Docker build takes a few minutes.
Add frame-level visual retrieval
Configure the optional visual-retrieval model services for matching against video frames.
Before you start
- A working Docker setup
- VLM2Vec-V2.0 and Qwen3-Embedding-4B weights under docker-data/models/
- A GPU host for the documented GPU worker setup
Steps
Place VLM2Vec-V2.0 and Qwen3-Embedding-4B under docker-data/models/.
Start the model services with docker compose --profile models up --build.
Set EM2MEM_VISUAL_BACKEND=remote and EM2MEM_TEXT_EMBED_BACKEND=remote in .env.
On a GPU host, add the documented override so the workers receive GPU access.
Expected outcome
The backend is configured to use remote visual and text-embedding model services for visual retrieval.
Guide limitations
- Visual retrieval is an optional add-on, not required for the standard Docker workflow.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 12 October 2026
- Original website title
- zjunlp/LightMem-Ego · Official homepage title · retained in its source language