Glass – open-source AI desktop assistant that captures screen content and recognizes audio in real time

What is Glass？

Glass is an open-source, invisible AI desktop assistant developed by the Pickle team. It can run in the background to capture on-screen content and audio in real time, converting them into structured knowledge. Its core functions include real-time meeting transcription, automatic summarization, contextual understanding, and real-time Q&A. Glass is truly invisible—it does not appear in screen recordings, screenshots, or the Dock, ensuring no disruption to user operations. It supports both macOS and Windows, is free to use, and requires no registration. With its open-source nature, Glass is emerging as a promising tool in the fields of AI-powered office work and knowledge management.

Main Features of Glass

• Real-Time Screen Capture & Audio Recognition: Continuously monitors and captures screen content and audio (such as meeting speech) and converts them into structured information.

• Contextual Understanding & Summarization: Combines visual and auditory information to automatically produce high-quality meeting notes, summaries, and task lists.

• Real-Time Q&A Assistant: Users can ask questions anytime, and the AI responds based on the captured screen actions and audio content.

• Invisible Design: Completely hidden from screen recordings, screenshots, and the Dock, without interfering with user workflows.

• Meeting Recording & Summarization: Automatically transcribes meetings, distinguishes between speakers, extracts key points, and generates summaries.

• Knowledge Archiving: Stores user actions and voice interactions as knowledge documents for future review and reference.

How to Use Glass

For macOS Users

1. Download Precompiled Version: Go to the Glass GitHub Release page and download the latest .dmg (recommended) or .zip file. Open the .dmg file and drag the Glass app into the “Applications” folder.

2. Run Glass: Locate Glass in the “Applications” folder and double-click to start.

3. Configure and Use: Follow on-screen prompts for setup, such as entering an OpenAI API key (optional). Use shortcuts (e.g., Cmd +  to show/hide the main window, Cmd + Enter to call the AI assistant).

For Windows Users

1. Install Dependencies: Ensure Python and Node.js are installed (Node.js 20.x.x recommended). If not, install Node.js with:

   curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 20
nvm use 20


2. Clone and Install Project:

   git clone https://github.com/pickle-com/glass.git
cd glass


3. Install Dependencies:

   npm run setup


4. Run Glass:

   npm start


5. Configure and Use: Follow setup prompts, such as entering an OpenAI API key (optional). Use shortcuts (e.g., Ctrl +  to show/hide the main window, Ctrl + Enter to call the AI assistant).

Project Repository

GitHub: https://github.com/pickle-com/glass

Use Cases

• Meeting Recording & Summarization: During remote or in-person meetings, capture screen content and audio in real time, automatically producing meeting notes, summaries, and action items to help teams track key points efficiently.

• Academic & Teaching: In classrooms or academic conferences, automatically record and structure notes to assist students and researchers in reviewing and organizing knowledge.

• Team Collaboration & Project Management: During team discussions and project meetings, generate real-time task lists and to-dos, ensuring clear task allocation and progress tracking.

• Personal Knowledge Management: Archive daily on-screen activities and voice interactions into knowledge documents for efficient organization and easy future reference.

• Office Assistance: While handling work tasks, provide real-time answers, background information, and suggestions to help users complete tasks faster and more efficiently.