Glass : Your Digital Thinking Expansion Assistant

What is Glass?

Glass is an open-source desktop application developed by the Pickle team. It aims to capture users’ screen and audio content in real-time, combined with contextual information, to generate structured knowledge output. Unlike traditional AI chatbots, Glass is a proactive AI assistant that provides relevant help and suggestions as users operate their computer. It runs in the background on the desktop, observing screen content and listening to audio to offer immediate support when needed.

Key Features

• Real-time Intelligent Q&A: Users can ask questions, and Glass will provide answers based on prior screen activity and audio context.

• Meeting Summaries and Notes: During meetings, Glass automatically generates summaries and action items, helping users quickly review meeting content.

• Privacy-focused Design: Glass does not appear in screen recordings, screenshots, or taskbars, ensuring user privacy and security.

• Hotkey Support: Users can toggle the main window with Ctrl/Cmd + , ask AI questions with Ctrl/Cmd + Enter, and move the window with Ctrl/Cmd + Arrow Keys.

• Multilingual Support: Glass supports multiple languages to cater to users from different regions.

• Local Model Support: Glass supports local AI models like Whisper and Ollama to enhance performance and flexibility.

Technical Principles

Glass is built on Electron and integrates screen capture, audio recording, and contextual analysis technologies. It uses OpenAI’s API for intelligent Q&A and plans to support the Gemini API to reduce costs. Additionally, it supports local models such as Whisper and Ollama to improve efficiency and adaptability.

Project Repository

• GitHub: https://github.com/pickle-com/glass

Use Cases

• Remote Work: Helps users capture meeting content and generate summaries to improve work efficiency.

• Education and Training: Teachers can record lecture content and create teaching summaries for easy student review.

• Personal Knowledge Management: Users can capture daily information to build a personal knowledge base.

• Software Development: Developers can document their coding process, thoughts, and issues for future reference and troubleshooting.

• Customer Support: Customer service agents can record conversations with clients and generate service logs to enhance service quality.