Liliumai AI Agent

Lilium AI

English | 简体中文

Not just a chatbot, but your AI Co-pilot for total computer automation. liliumai.cn

Lilium AI is a powerful personal AI agent framework designed for autonomous execution. It connects Large Language Models (LLMs) like OpenAI and Ollama to browser automation, system-level shell control, deep file integration, and omni-channel communication, enabling AI to perform complex end-to-end tasks just like a human.

✨ Key Features

• 🌐 Browser Automation: Built-in browser driver for autonomous navigation, searching, clicking, and data extraction.
• 🤖 Autonomous Sub-Agents: Spawn background sub-agents to handle complex tasks asynchronously without interrupting your main chat.
• 💻 Total Computer Control: Equipped with Shell execution, Filesystem I/O, and Cron scheduling for true automation.
• 🔌 Omni-Channel: The only framework supporting WeChat, Feishu, DingTalk, Telegram, WhatsApp, Discord, and more.
• 🧠 Memory & Knowledge: Long-term memory (SQLite/LanceDB) and RAG-based knowledge base for personalized workflows.
• 🧠 Multi-LLM Support: Seamlessly switch between OpenAI, Claude, Ollama (local models), and more.
• 🧩 Minimalist Extension: Add new skills in minutes by dropping JS files into the plugins/ folder.
• 🗣️ Voice-to-Action: Automatic voice transcription (Whisper) to control your computer via voice commands.

🚀 Core Scenarios (What can it do?)

• Automated Workflows: "Every morning at 9 AM, log into my GitHub, summarize all new issues, and send the report to my WeChat."
• Web Intelligence: "Monitor this page for price drops; notify me via Slack as soon as it's below $1000."
• System Ops: "Analyze the server logs in the current directory, find all 500 errors from the last hour, and generate a root-cause summary."
• Cross-Platform Assistant: "Take all voice messages I send you on Telegram, transcribe them into text, and save the summary to a local Markdown file."

💡 Why Choose Lilium AI?

🌟 For Everyone: Your All-in-One Digital Assistant

• Simple as Chatting: No coding required. Just tell it, "Compare the price of these headphones across major sites," and it will browse the web for you.
• 24/7 Personal Agent: Let it monitor web pages or summarize documents while you sleep, and report back via WhatsApp or Telegram.
• Privacy First: Your chats, files, and account info stay on your own device—safe and secure.

🛠️ For Developers: Ultimate Extensibility

• Node.js Ecosystem: Built on Node.js, leveraging millions of npm packages for easy extension.
• Powerful Toolbox: Integrated Puppeteer, SQLite, LanceDB, Whisper, and RAG capabilities.
• Flexible Plugin System: Define new AI skills simply by dropping a JS file into the plugins/ folder.

🚀 Getting Started

Prerequisites

• Node.js: v18 or higher
• pnpm: Recommended package manager (npm install -g pnpm)
• Python 3: (Optional) For macOS permission checking script (check-permissions).

Installation

1. Clone the repository

   git clone https://github.com/beidald/liliumai.git
   cd liliumai
   

2. Install dependencies

   pnpm install
   

3. Configure the project Copy the example configuration file to create your local config.

   cp config.example.json config.json
   

4. Edit config.json Open config.json and fill in your details:

   • LLM: Add your OpenAI API Key or Ollama base URL.
   • Channels: Enable desired channels (e.g., wechat, telegram) and add tokens.
   • Admin: Critical! Change the default password in the admin section.

   Security Note: Never commit your config.json to version control. The project is configured to ignore it by default.

5. Check Permissions (macOS only) If running on macOS and using Wechaty/Puppeteer, you may need to grant accessibility permissions.

   pnpm check-permissions
   

Running the Bot

Start the agent in development mode:

pnpm start

Or build and run for production:

pnpm build
node dist/index.js

⚠️ Disclaimer

• Terms of Service: Automated interactions with some platforms (e.g., WeChat, WhatsApp) may violate their Terms of Service. Users are responsible for ensuring compliance with platform rules.
• Account Risk: Using this software with personal accounts (especially on WeChat via Puppet) carries a risk of account restriction or banning. It is highly recommended to use a dedicated test account.
• Data Privacy: While this project is designed to be "Local First" and privacy-focused, users are responsible for securing their own server environments and configuration files.

🗺️ Roadmap

• Enhanced Web Dashboard: Richer data visualization and easier configuration management.
• Plugin Ecosystem: A marketplace or easier discovery mechanism for community plugins.
• Multi-modal Support: Ability to process images (Vision) and generate audio (TTS).
• RAG Improvements: Better knowledge base retrieval strategies and document parsing.
• Mobile App: A companion mobile app for managing the bot on the go.

⚙️ Configuration Guide

The config.json file is the heart of Lilium AI. Here are key sections:

Admin Authentication

Secure your Web Interface and API endpoints.

"admin": {
  "email": "admin@example.com",
  "password": "YOUR_STRONG_PASSWORD"
}

LLM Providers

Configure multiple providers with priorities.

"llm": [
  {
    "provider": "openai",
    "model": "gpt-4",
    "apiKey": "sk-...",
    "priority": 1
  },
  {
    "provider": "ollama",
    "model": "llama3",
    "baseUrl": "http://localhost:11434/v1",
    "priority": 2
  }
]

Voice Transcription

Enable automatic voice-to-text for audio messages.

"transcription": {
  "provider": "local",  // "local" or "groq"
  "model": "base",      // "tiny", "base", "small", "medium", "large"
  "apiKey": ""          // Required for cloud providers
}

Note: Local models will be automatically downloaded on first use from HuggingFace (with domestic mirror support).

Channels

Enable channels as needed. For example, to enable the Web UI:

"channels": {
  "web": {
    "enabled": true,
    "port": 3000
  }
}

🖥️ Web Interface

Once running, access the Web UI at: http://localhost:3000

• Chat: Interact with Lilium directly from your browser.
• Stop Generation: Click the "Stop" button to immediately halt the AI's current output.
• Admin: Some features require the Admin credentials configured in config.json.

🧩 Skills Showcase

Lilium AI's true power lies in skill composition. You can give complex natural language instructions:

• Browser Automation: "Log into GitHub and star the specified repository."
• System Analysis: "Analyze the log files in the current directory, identify the root cause of 500 errors, and generate a summary."
• Autonomous Tasks: "Spawn a sub-agent to continuously monitor price changes on a competitor's website and report to me every hour."
• Smart Scheduling: "Every morning at 9 AM, summarize all my unread Feishu messages and send them to Telegram."
• Multi-step Collaboration: "Search for the latest research on DeepSeek, download the PDF, extract the key findings, and save them to the local knowledge base."

🤝 Join the Evolution (Contributing)

Lilium AI is an open project, and we welcome developers to join us in building the most powerful personal AI agent!

• Develop New Skills: Want the AI to support more tools? You can easily extend its capabilities by writing simple TypeScript/JavaScript classes (Check TOOLS.md).
• Build Plugins: Our plugin system allows you to add features directly to the plugins/ directory without touching the core code (Check PLUGINS.md).
• Improve Channels: Help us integrate more communication platforms like Slack, Discord, or deeper Feishu/DingTalk integrations.
• Submit Issues/PRs: Whether it's a bug fix or a feature proposal, we value every contribution.

Check out our CONTRIBUTING.md for more details. Let's define the future of AI automation together!

📬 Contact

Email: beidaliudong@pku.org.cn

📝 License

MIT License.