let5see/Open-AutoGLM
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #237 from zai-org/update-hdc-readme

Browse Source 
Update README.md and README_en.md to support HarmonyOS devices
This commit is contained in:
yongbin-buaa
2025-12-18 15:55:05 +08:00
committed by GitHub
parent 34457ded4c 4b30344f15
commit 5fcb2f5146
3 changed files with 292 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.pre-commit-config.yaml
View File

@@ -2,6 +2,7 @@ default_install_hook_types:
- pre-commit
- commit-msg
exclude: '^phone_agent/config/apps\.py$'
exclude: '^README_en\.md$'
default_stages:
- pre-commit # Run locally
repos:

166
README.md
View File

@@ -49,7 +49,11 @@ ADB 调试能力，可通过 WiFi 或网络连接设备，实现灵活的远程
建议使用 Python 3.10 及以上版本。
### 2. ADB (Android Debug Bridge)
### 2. 手机调试命令行工具
根据你的设备类型选择相应的工具：
#### 对于 Android 设备 - 使用 ADB
1. 下载官方 ADB [安装包](https://developer.android.com/tools/releases/platform-tools?hl=zh-cn)，并解压到自定义路径
2. 配置环境变量
@@ -63,7 +67,22 @@ ADB 调试能力，可通过 WiFi 或网络连接设备，实现灵活的远程
- Windows 配置方法：可参考 [第三方教程](https://blog.csdn.net/x2584179909/article/details/108319973) 进行配置。
### 3. Android 7.0+ 的设备或模拟器，并启用 `开发者模式` 和 `USB 调试`
#### 对于鸿蒙设备 (HarmonyOS NEXT版本以上) - 使用 HDC
1. 下载 HDC 工具：
- 从 [HarmonyOS SDK](https://developer.huawei.com/consumer/cn/download/) 下载
2. 配置环境变量
- MacOS/Linux 配置方法：
```bash
# 假设解压后的目录为 ~/Downloads/harmonyos-sdk/toolchains。请根据实际路径调整。
export PATH=${PATH}:~/Downloads/harmonyos-sdk/toolchains
```
- Windows 配置方法：将 HDC 工具所在目录添加到系统 PATH 环境变量
### 3. Android 7.0+ 或 HarmonyOS 设备，并启用 `开发者模式` 和 `USB 调试`
1. 开发者模式启用：通常启用方法是，找到 `设置-关于手机-版本号` 然后连续快速点击 10
次左右，直到弹出弹窗显示“开发者模式已启用”。不同手机会有些许差别，如果找不到，可以上网搜索一下教程。
@@ -75,7 +94,11 @@ ADB 调试能力，可通过 WiFi 或网络连接设备，实现灵活的远程
![权限](resources/screenshot-20251209-181423.png)
### 4. 安装 ADB Keyboard(用于文本输入)
### 4. 安装 ADB Keyboard(仅 Android 设备需要，用于文本输入)
**注意：鸿蒙设备使用原生输入方法，无需安装 ADB Keyboard。**
如果你使用的是 Android 设备：
下载 [安装包](https://github.com/senzhk/ADBKeyBoard/blob/master/ADBKeyboard.apk) 并在对应的安卓设备中进行安装。
注意，安装完成后还需要到 `设置-输入法` 或者 `设置-键盘列表` 中启用 `ADB Keyboard` 才能生效(或使用命令`adb shell ime enable com.android.adbkeyboard/.AdbIME`[How-to-use](https://github.com/senzhk/ADBKeyBoard/blob/master/README.md#how-to-use))
@@ -89,7 +112,9 @@ pip install -r requirements.txt
pip install -e .
```
### 2. 配置 ADB
### 2. 配置 ADB 或 HDC
#### 对于 Android 设备
确认 **USB数据线具有数据传输功能**, 而不是仅有充电功能
@@ -104,6 +129,20 @@ adb devices
# emulator-5554 device
```
#### 对于鸿蒙设备
确认 **USB数据线具有数据传输功能**, 而不是仅有充电功能
确保已安装 HDC 并使用 **USB数据线** 连接设备：
```bash
# 检查已连接的设备
hdc list targets
# 输出结果应显示你的设备，如：
# 7001005458323933328a01bce01c2500
```
### 3. 启动模型服务
你可以选择自行部署模型服务，或使用第三方模型服务商。
@@ -237,23 +276,32 @@ python scripts/check_deployment_cn.py --base-url http://你的IP:你的端口/v1
### 命令行
根据你部署的模型, 设置 `--base-url` 和 `--model` 参数. 例如:
根据你部署的模型, 设置 `--base-url` 和 `--model` 参数, 设置 `--device-type` 指定是安卓设备或鸿蒙设备 (默认值 adb 表示安卓设备, hdc 表示鸿蒙设备). 例如:
```bash
# 交互模式
# Android 设备 - 交互模式
python main.py --base-url http://localhost:8000/v1 --model "autoglm-phone-9b"
# 指定模型端点
# Android 设备 - 指定任务
python main.py --base-url http://localhost:8000/v1 "打开美团搜索附近的火锅店"
# 鸿蒙设备 - 交互模式
python main.py --device-type hdc --base-url http://localhost:8000/v1 --model "autoglm-phone-9b"
# 鸿蒙设备 - 指定任务
python main.py --device-type hdc --base-url http://localhost:8000/v1 "打开美团搜索附近的火锅店"
# 使用 API Key 进行认证
python main.py --apikey sk-xxxxx
# 使用英文 system prompt
python main.py --lang en --base-url http://localhost:8000/v1 "Open Chrome browser"
# 列出支持的应用
# 列出支持的应用（Android）
python main.py --list-apps
# 列出支持的应用（鸿蒙）
python main.py --device-type hdc --list-apps
```
### Python API
@@ -278,30 +326,47 @@ print(result)
## 远程调试
Phone Agent 支持通过 WiFi/网络进行远程 ADB 调试，无需 USB 连接即可控制设备。
Phone Agent 支持通过 WiFi/网络进行远程 ADB/HDC 调试，无需 USB 连接即可控制设备。
### 配置远程调试
#### 在手机端开启无线调试
##### Android 设备
确保手机和电脑在同一个WiFi中，如图所示
![开启无线调试](resources/setting.png)
#### 在电脑端使用标准 ADB 命令
##### 鸿蒙设备
确保手机和电脑在同一个WiFi中：
1. 进入 `设置 > 系统和更新 > 开发者选项`
2. 开启 `USB 调试` 和 `无线调试`
3. 记录显示的 IP 地址和端口号
#### 在电脑端使用标准 ADB/HDC 命令
```bash
# 通过 WiFi 连接, 改成手机显示的 IP 地址和端口
# Android 设备 - 通过 WiFi 连接, 改成手机显示的 IP 地址和端口
adb connect 192.168.1.100:5555
# 验证连接
adb devices
# 应显示：192.168.1.100:5555 device
# 鸿蒙设备 - 通过 WiFi 连接
hdc tconn 192.168.1.100:5555
# 验证连接
hdc list targets
# 应显示：192.168.1.100:5555
```
### 设备管理命令
#### Android 设备（ADB）
```bash
# 列出所有已连接设备
adb devices
@@ -316,8 +381,26 @@ adb disconnect 192.168.1.100:5555
python main.py --device-id 192.168.1.100:5555 --base-url http://localhost:8000/v1 --model "autoglm-phone-9b" "打开抖音刷视频"
```
#### 鸿蒙设备（HDC）
```bash
# 列出所有已连接设备
hdc list targets
# 连接远程设备
hdc tconn 192.168.1.100:5555
# 断开指定设备
hdc tdisconn 192.168.1.100:5555
# 指定设备执行任务
python main.py --device-type hdc --device-id 192.168.1.100:5555 --base-url http://localhost:8000/v1 --model "autoglm-phone-9b" "打开抖音刷视频"
```
### Python API 远程连接
#### Android 设备（ADB）
```python
from phone_agent.adb import ADBConnection, list_devices
@@ -342,6 +425,27 @@ print(f"设备 IP: {ip}")
conn.disconnect("192.168.1.100:5555")
```
#### 鸿蒙设备（HDC）
```python
from phone_agent.hdc import HDCConnection, list_devices
# 创建连接管理器
conn = HDCConnection()
# 连接远程设备
success, message = conn.connect("192.168.1.100:5555")
print(f"连接状态: {message}")
# 列出已连接设备
devices = list_devices()
for device in devices:
print(f"{device.device_id} - {device.connection_type.value}")
# 断开连接
conn.disconnect("192.168.1.100:5555")
```
### 远程连接问题排查
**连接被拒绝：**
@@ -373,14 +477,15 @@ conn.disconnect("192.168.1.100:5555")
### 环境变量
| 变量 | 描述 | 默认值 |
|-------------------------|------------------|----------------------------|
| `PHONE_AGENT_BASE_URL` | 模型 API 地址 | `http://localhost:8000/v1` |
| `PHONE_AGENT_MODEL` | 模型名称 | `autoglm-phone-9b` |
| `PHONE_AGENT_API_KEY` | 模型认证 API Key | `EMPTY` |
| `PHONE_AGENT_MAX_STEPS` | 每个任务最大步数 | `100` |
| `PHONE_AGENT_DEVICE_ID` | ADB 设备 ID | (自动检测) |
| `PHONE_AGENT_LANG` | 语言 (`cn` 或 `en`) | `cn` |
| 变量 | 描述 | 默认值 |
|-----------------------------|------------------------|----------------------------|
| `PHONE_AGENT_BASE_URL` | 模型 API 地址 | `http://localhost:8000/v1` |
| `PHONE_AGENT_MODEL` | 模型名称 | `autoglm-phone-9b` |
| `PHONE_AGENT_API_KEY` | 模型认证 API Key | `EMPTY` |
| `PHONE_AGENT_MAX_STEPS` | 每个任务最大步数 | `100` |
| `PHONE_AGENT_DEVICE_ID` | ADB/HDC 设备 ID | (自动检测) |
| `PHONE_AGENT_DEVICE_TYPE` | 设备类型 (`adb` 或 `hdc`) | `adb` |
| `PHONE_AGENT_LANG` | 语言 (`cn` 或 `en`) | `cn` |
### 模型配置
@@ -452,6 +557,8 @@ config = AgentConfig(
## 支持的应用
### Android 应用
Phone Agent 支持 50+ 款主流中文应用：
| 分类 | 应用 |
@@ -467,6 +574,25 @@ Phone Agent 支持 50+ 款主流中文应用：
运行 `python main.py --list-apps` 查看完整列表。
### 鸿蒙应用
Phone Agent 支持 60+ 款鸿蒙原生应用和系统应用：
| 分类 | 应用 |
|---------|------------------------------------------|
| 社交通讯 | 微信、QQ、微博、飞书、企业微信 |
| 电商购物 | 淘宝、京东、拼多多、唯品会、得物、闲鱼 |
| 美食外卖 | 美团、美团外卖、大众点评、海底捞 |
| 出行旅游 | 12306、滴滴出行、同程旅行、高德地图、百度地图 |
| 视频娱乐 | bilibili、抖音、快手、腾讯视频、爱奇艺、芒果TV |
| 音乐音频 | QQ音乐、汽水音乐、喜马拉雅 |
| 生活服务 | 小红书、知乎、今日头条、58同城、中国移动 |
| AI与工具 | 豆包、WPS、UC浏览器、扫描全能王、美图秀秀 |
| 系统应用 | 浏览器、日历、相机、时钟、云空间、文件管理器、相册、联系人、短信、设置等 |
| 华为服务 | 应用市场、音乐、视频、阅读、主题、天气 |
运行 `python main.py --device-type hdc --list-apps` 查看完整列表。
## 可用操作
Agent 可以执行以下操作：

163
README_en.md
View File

@@ -39,7 +39,11 @@ Phone Agent is a mobile intelligent assistant framework built on AutoGLM. It und
Python 3.10 or higher is recommended.
### 2. ADB (Android Debug Bridge)
### 2. Device Debug Tools
Choose the appropriate tool based on your device type:
#### For Android Devices - Using ADB
1. Download the official ADB [installation package](https://developer.android.com/tools/releases/platform-tools) and extract it to a custom path
2. Configure environment variables
@@ -53,7 +57,22 @@ Python 3.10 or higher is recommended.
- Windows configuration: Refer to [third-party tutorials](https://blog.csdn.net/x2584179909/article/details/108319973) for configuration.
### 3. Android 7.0+ Device or Emulator with `Developer Mode` and `USB Debugging` Enabled
#### For HarmonyOS Devices - Using HDC
1. Download HDC tool:
- From [HarmonyOS SDK](https://developer.huawei.com/consumer/en/download/)
2. Configure environment variables
- MacOS/Linux configuration:
```bash
# Assuming the extracted directory is ~/Downloads/harmonyos-sdk/toolchains. Adjust according to actual path.
export PATH=${PATH}:~/Downloads/harmonyos-sdk/toolchains
```
- Windows configuration: Add the HDC tool directory to the system PATH environment variable
### 3. Android 7.0+ or HarmonyOS Device with `Developer Mode` and `USB Debugging` Enabled
1. Enable Developer Mode: The typical method is to find `Settings > About Phone > Build Number` and tap it rapidly about 10 times until a popup shows "Developer mode has been enabled." This may vary slightly between phones; search online for tutorials if you can't find it.
2. Enable USB Debugging: After enabling Developer Mode, go to `Settings > Developer Options > USB Debugging` and enable it
@@ -63,7 +82,11 @@ Python 3.10 or higher is recommended.
![Permissions](resources/screenshot-20251210-120416.png)
### 4. Install ADB Keyboard (for Text Input)
### 4. Install ADB Keyboard (Required for Android Devices Only, for Text Input)
**Note: HarmonyOS devices use native input methods and do not require ADB Keyboard.**
If you are using an Android device:
Download the [installation package](https://github.com/senzhk/ADBKeyBoard/blob/master/ADBKeyboard.apk) and install it on the corresponding Android device.
Note: After installation, you need to enable `ADB Keyboard` in `Settings > Input Method` or `Settings > Keyboard List` for it to work.(or use command `adb shell ime enable com.android.adbkeyboard/.AdbIME`[How-to-use](https://github.com/senzhk/ADBKeyBoard/blob/master/README.md#how-to-use))
@@ -77,7 +100,9 @@ pip install -r requirements.txt
pip install -e .
```
### 2. Configure ADB
### 2. Configure ADB or HDC
#### For Android Devices
Make sure your **USB cable supports data transfer**, not just charging.
@@ -92,6 +117,20 @@ adb devices
# emulator-5554 device
```
#### For HarmonyOS Devices
Make sure your **USB cable supports data transfer**, not just charging.
Ensure HDC is installed and connect the device via **USB cable**:
```bash
# Check connected devices
hdc list targets
# Output should show your device, e.g.:
# 7001005458323933328a01bce01c2500
```
### 3. Start Model Service
You can choose to deploy the model service yourself or use a third-party model service provider.
@@ -188,20 +227,29 @@ Upon successful execution, the script will display the model's inference result
Set the `--base-url` and `--model` parameters according to your deployed model. For example:
```bash
# Interactive mode
# Android device - Interactive mode
python main.py --base-url http://localhost:8000/v1 --model "autoglm-phone-9b-multilingual"
# Specify model endpoint
# Android device - Specify task
python main.py --base-url http://localhost:8000/v1 "Open Maps and search for nearby coffee shops"
# HarmonyOS device - Interactive mode
python main.py --device-type hdc --base-url http://localhost:8000/v1 --model "autoglm-phone-9b-multilingual"
# HarmonyOS device - Specify task
python main.py --device-type hdc --base-url http://localhost:8000/v1 "Open Maps and search for nearby coffee shops"
# Use API key for authentication
python main.py --apikey sk-xxxxx
# Use English system prompt
python main.py --lang en --base-url http://localhost:8000/v1 "Open Chrome browser"
# List supported apps
# List supported apps (Android)
python main.py --list-apps
# List supported apps (HarmonyOS)
python main.py --device-type hdc --list-apps
```
### Python API
@@ -226,29 +274,47 @@ print(result)
## Remote Debugging
Phone Agent supports remote ADB debugging via WiFi/network, allowing device control without a USB connection.
Phone Agent supports remote ADB/HDC debugging via WiFi/network, allowing device control without a USB connection.
### Configure Remote Debugging
#### Enable Wireless Debugging on Phone
##### Android Devices
Ensure the phone and computer are on the same WiFi network, as shown below:
![Enable Wireless Debugging](resources/screenshot-20251210-120630.png)
#### Use Standard ADB Commands on Computer
##### HarmonyOS Devices
Ensure the phone and computer are on the same WiFi network:
1. Go to `Settings > System & Updates > Developer Options`
2. Enable `USB Debugging` and `Wireless Debugging`
3. Note the displayed IP address and port number
#### Use Standard ADB/HDC Commands on Computer
```bash
# Connect via WiFi, replace with the IP address and port shown on your phone
# Android device - Connect via WiFi, replace with the IP address and port shown on your phone
adb connect 192.168.1.100:5555
# Verify connection
adb devices
# Should show: 192.168.1.100:5555 device
# HarmonyOS device - Connect via WiFi
hdc tconn 192.168.1.100:5555
# Verify connection
hdc list targets
# Should show: 192.168.1.100:5555
```
### Device Management Commands
#### Android Devices (ADB)
```bash
# List all connected devices
adb devices
@@ -263,8 +329,26 @@ adb disconnect 192.168.1.100:5555
python main.py --device-id 192.168.1.100:5555 --base-url http://localhost:8000/v1 --model "autoglm-phone-9b-multilingual" "Open TikTok and browse videos"
```
#### HarmonyOS Devices (HDC)
```bash
# List all connected devices
hdc list targets
# Connect to remote device
hdc tconn 192.168.1.100:5555
# Disconnect specific device
hdc tdisconn 192.168.1.100:5555
# Execute task on specific device
python main.py --device-type hdc --device-id 192.168.1.100:5555 --base-url http://localhost:8000/v1 --model "autoglm-phone-9b-multilingual" "Open TikTok and browse videos"
```
### Python API Remote Connection
#### Android Devices (ADB)
```python
from phone_agent.adb import ADBConnection, list_devices
@@ -289,6 +373,27 @@ print(f"Device IP: {ip}")
conn.disconnect("192.168.1.100:5555")
```
#### HarmonyOS Devices (HDC)
```python
from phone_agent.hdc import HDCConnection, list_devices
# Create connection manager
conn = HDCConnection()
# Connect to remote device
success, message = conn.connect("192.168.1.100:5555")
print(f"Connection status: {message}")
# List connected devices
devices = list_devices()
for device in devices:
print(f"{device.device_id} - {device.connection_type.value}")
# Disconnect
conn.disconnect("192.168.1.100:5555")
```
### Remote Connection Troubleshooting
**Connection Refused:**
@@ -320,14 +425,15 @@ You can directly modify the corresponding config files to enhance model capabili
### Environment Variables
| Variable | Description | Default Value |
|---------------------------|---------------------------|----------------------------|
| `PHONE_AGENT_BASE_URL` | Model API URL | `http://localhost:8000/v1` |
| `PHONE_AGENT_MODEL` | Model name | `autoglm-phone-9b` |
| `PHONE_AGENT_API_KEY` | API key for authentication| `EMPTY` |
| `PHONE_AGENT_MAX_STEPS` | Maximum steps per task | `100` |
| `PHONE_AGENT_DEVICE_ID` | ADB device ID | (auto-detect) |
| `PHONE_AGENT_LANG` | Language (`cn` or `en`) | `en` |
| Variable | Description | Default Value |
|-----------------------------|---------------------------|----------------------------|
| `PHONE_AGENT_BASE_URL` | Model API URL | `http://localhost:8000/v1` |
| `PHONE_AGENT_MODEL` | Model name | `autoglm-phone-9b` |
| `PHONE_AGENT_API_KEY` | API key for authentication| `EMPTY` |
| `PHONE_AGENT_MAX_STEPS` | Maximum steps per task | `100` |
| `PHONE_AGENT_DEVICE_ID` | ADB/HDC device ID | (auto-detect) |
| `PHONE_AGENT_DEVICE_TYPE` | Device type (`adb` or `hdc`)| `adb` |
| `PHONE_AGENT_LANG` | Language (`cn` or `en`) | `en` |
### Model Configuration
@@ -399,6 +505,8 @@ This allows you to clearly see the AI's reasoning process and specific operation
## Supported Apps
### Android Apps
Phone Agent supports 50+ mainstream Chinese applications:
| Category | Apps |
@@ -411,6 +519,25 @@ Phone Agent supports 50+ mainstream Chinese applications:
Run `python main.py --list-apps` to see the complete list.
### HarmonyOS Apps
Phone Agent supports 60+ HarmonyOS native apps and system apps:
| Category | Apps |
|--------------------------|----------------------------------------------------------------------------------------|
| Social & Messaging | WeChat, QQ, Weibo, Feishu, Enterprise WeChat |
| E-commerce & Shopping | Taobao, JD.com, Pinduoduo, Vipshop, Dewu, Xianyu |
| Food & Delivery | Meituan, Meituan Waimai, Dianping, Haidilao |
| Travel & Navigation | 12306, Didi, Tongcheng, Amap, Baidu Maps |
| Video & Entertainment | Bilibili, Douyin, Kuaishou, Tencent Video, iQIYI, Mango TV |
| Music & Audio | QQ Music, Qishui Music, Ximalaya |
| Lifestyle & Social | Xiaohongshu, Zhihu, Toutiao, 58.com, China Mobile |
| AI & Tools | Doubao, WPS, UC Browser, CamScanner, Meitu |
| System Apps | Browser, Calendar, Camera, Clock, Cloud, File Manager, Gallery, Contacts, SMS, Settings |
| Huawei Services | AppGallery, Music, Video, Books, Themes, Weather |
Run `python main.py --device-type hdc --list-apps` to see the complete list.
## Available Actions
The Agent can perform the following actions: