Ready for open source (#47)

* Create app/main API

* create example device

* create ROS backend and example device SDK Wrapper

* Add ROS host and host starting from app.py

* Add gripper device and mock implementation

* add "status_types" & "action_types" to ROS device decorator

* add ActionServer debug example

* [bugfix] complete mock gripper example

* ROS Backend Host for Device action calling and Resource management

* add conda/mamba ENV file

* add host_node communication with app/main.py

* add action message value mappings and converters

* Update ilabos.yaml

* Update issue templates

* example devices.json and resources.json

* Fix Device wrapper to use async property and actions (#7)

* Fix Device wrapper to use async property and actions

* Resolve #1 : support async get methods and actions. Give new examples.

* add both sync/async GRBL controller SDK

* 2 call device actions from appmainpy api to ros hostpy (#8)

* feature: add job

* fix:node start

* feature:add get job status

* fix:get device

* clean

* Resolve #5 device connection diagram and workflow compilation support (#9)

* add syringe pump device and its compilation using device connection diagram

* add RunzeSyringePump real device with ROS2 example

* Prototype machine with 1 pump and 1 CNC

* add ROS2ProtocolNode and related functions

* add ilabos_msgs (to use PumpTransfer action)

* add example device connection graph

* refactor protocol_node code into separate file

* add ROS2SerialNode

* add SerialCommand srv in ilabos_msgs

* add pump_protocol example, and fix bugs

* [fix] serial service: avoid async service deadlock by directly call serial `send_command`

* use SendCmd instead of SingleJointPosition for valve control

* initialize device connection graph when server starts

* Fix #5: async workflow execution (#16)

* add rclpyx and protocol example for async-native workflow

* use async in ROS2ProtocolNode, and host initialization

* add examples of "ros-async" protocol implementation, and `run_in_event_loop` for using native async functions

* use "ros-async" in protocols and device nodes

* fix pump_protocol: push to 0.0 μL

* Envs, docs, and conda recipes (#19)

* update ENV: use python 3.11 and deprecate ros-humble-gazebo-ros

* add ilabos-msgs conda recipe

* Update ilabos.yaml

* fix recipe and env yaml

* Add sphinx docs

* add aichemeco

* add bioyong

* add bioyong

* Support XDL devices & protocols (#20)

* [Feature] support multiple protocols in a single ProtocolNode

* add Junjie's code

* Support "Clean" protocol

* Update Grignard_flow_batchreact_single_pumpvalve.json

* test_grignard_add

* add stir device node and example

* Update device_node.py

add print_publish flag to control the node's log output

* NH4Cl_add

* add "HeaterStirrer" device and "HeatChill" action

* add wait time after each pump action for equilibration

* fix stir

* add Separate protocol

* Refactor Separator device and Stir action

* add rotavap_node

* fix stir

* add chiller node

* Move rinsings into PumpTransfer

* Fix SeparateProtocol under refactored Separator device and Stir action

* Supports automatically add new protocol action_types

* fix PumpTransfer protocol because of rinsing

* Add Rotavap and Chiller devices

* fix SeparateProtocol

* add EvaporateProtocol

* add rotavap devices config

* fix HeaterStirrer and SeparatorController IO

* Fix automatically add new protocol action_types

* Add HeaterStirrer and SeparatorController device config

* fix pump protocols

* Fix Evaporate action

* Update evaporate_protocol.py

* add temp_sensor node and add function remap

* update docs

---------

Co-authored-by: 王俊杰 <1800011822@pku.edu.cn>
Co-authored-by: q434343 <554662886@qq.com>

* fix aichemeco

* [Bugfix] fix Windows conda packaging

* add file upload api

* update dependencies: force to use 3.11 and remove conflict in WIN64 and OSX64

* update dependencies: force to use 3.11 and remove conflict in WIN64 and OSX64

* Create aichemeco_simple.py

* fix

* update

* add aichemeco file

* MQTT [1/2]: action start (#25)

* add mq

* fix

* clean

* add class

* fix excel

* update bioyong

* add api

* fix

---------

Co-authored-by: caok@dp.tech <xiaoyeqiannian@163.com>

* motor & grasp

* Add Grasp motor support and enhance EleGripper class

- Introduced a new motor configuration for Grasp in sjtu.json.
- Updated EleGripper class to inherit from UniversalDriver and added status property.
- Implemented move_and_rotate method for coordinated movement.
- Adjusted threading logic in EleGripper initialization.
- Registered Grasp motor in ROS2 device node configuration.

This commit enhances the motor control capabilities by integrating the Grasp motor and improving the existing EleGripper functionality.

* fix read data lenth

* Update Grasp.py

* MQTT (2/2): publish Device Status, Action Feedback & Results (#27)

* Add bridges in HostNode for device_status publishing

* Add "bridges" selection (fastapi & mqtt) when app start

* add MQ feedback & result publisher, and fix message converter

* fix UUID converting between ROS and MQ

* lint api model.py

* Continuous controllers: PID, MPC, custom controllers etc. (#23)

* add controller config & wrapper

* add controller setup at app.main

* control loop example

* fix com port

* add agv , ur_arm and raman

* MQTT (3/4): Unified Resources and Sync when starting the server (#28)

* update http upload api

* generate uuid when init device

* example resource json

* fix

* add new example full-content json (device, resource, graph)

* fix full-content json and related reading code

* fix

* add json_schema when initialize resources

* fix

* update schema

* refactor heaterstirrer.dalong

* fix

* fix refactor heaterstirrer.dalong

* refactor syringepump.runze: use ml instead of μL

* Update ilabos/ros/host.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: 王俊杰 <1800011822@pku.edu.cn>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Distributed initialization with self-organizing network (#29)

* add distributed launching option "--without_host"

* fix

---------

Co-authored-by: 王俊杰 <1800011822@pku.edu.cn>

* Refactor Workstation: Add resource service and tracking (#30)

* move ilabos/ros/rpc to ilabos/device_comms/rpc, and merge bioyond/aichemeco files under /devices

* add Resource srv and message_converter

* move graphio to ilabos/resources

* refactor resources type conversion

* add resource clients in device_node

* add mock resources service

* pass Gripper1 resource test

* update http resource services

* add AGV compile function

* add AGV transfer protocol

* update full mock_gripper edit_id example

* update full mock_gripper edit_id example

* get and update resource also in protocol_node

* mock resource update in AichemecoHiwo

* Create HT_hiwo.json

* add children in resources

* bugfixes

* fix rpc

* add Revvity winprep

---------

Co-authored-by: wjjxxx <43375851+wjjxxx@users.noreply.github.com>
Co-authored-by: 3218923350 <105201755+3218923350@users.noreply.github.com>

* Distributed launch (2/2): distributed resource create (#32)

* add resource_add request to host for slave mode

* add AGV

* fix protocol resources

* optimize host callbacks

* bugfixes

* add revvity registry

---------

Co-authored-by: 王俊杰 <1800011822@pku.edu.cn>
Co-authored-by: wjjxxx <43375851+wjjxxx@users.noreply.github.com>

* Refactor Driver Files Structure (#33)

* Integration with pywinauto & recorder
Added execute run and initialize procdure

* 酶标仪状态检测、使用示例,整体流程待测试

* nivo ready version

* Add HPLC driver and example script

- Introduced HPLCDriver class for managing HPLC device status and operations.
- Implemented device status monitoring and command execution via ROS2 actions.
- Added example script (hplc.py) demonstrating how to run commands on the HPLC device.
- Created PlayerUtil and UniversalDriver classes for shared functionality across devices.
- Refactored NivoDriver to utilize the new UniversalDriver structure.
- Enhanced error handling and process management in the NivoDriver.

* 修复start的错误定位

* hplc tested ok

* relative path to build msgs

* template_driver & jiageng devices

* fetch correct status type and action type

* fix mtype fetch

* gpc bus integration

* ilab build

* remove chs

* recipe rename

* modbus update 1

* json available

* hplc & modbus rewrite

* Update AgilentHPLC.py

hplc datafile reader

* move ilabos/ros/rpc to ilabos/device_comms/rpc, and merge bioyond/aichemeco files under /devices

* modbus分设备

* gpc

* gpc 2

* fix address

* default register node

* fix MainScreenGPC

* add Resource srv and message_converter

* move graphio to ilabos/resources

* refactor resources type conversion

* add resource clients in device_node

* add mock resources service

* pass Gripper1 resource test

* update http resource services

* add AGV compile function

* add AGV transfer protocol

* update recipe.yaml

* update full mock_gripper edit_id example

* update full mock_gripper edit_id example

* get and update resource also in protocol_node

* mock resource update in AichemecoHiwo

* feat: add other jiageng PLC device code

* ilabos compile

* correct format

* correct recipe format

* correct setup.py format

* remove unnecessary files

* remove unnecessary files

* Create HT_hiwo.json

* add children in resources

* hplc support sample_id

* correct hplc sample_id

* correct hplc sample_id

* hplc upload

* fix type hint

* oss upload tested ver

* recipe yaml fix for linux

* update installation yaml

* refactor: moved all driver files according to its feat

* merge main to dev

---------

Co-authored-by: 王俊杰 <2201110460@stu.pku.edu.cn>
Co-authored-by: Junhan Chang <changjh@pku.edu.cn>
Co-authored-by: jiawei <miaojiawei@dp.tech>

* add: NMR LH and RU device control (#34)

* Add Registry for device drivers and Support GraphML (#35)

* read chemputer graphml

* read graphml in app/main

* add devices in ros/devices

* add schema for devices

* read registry directory and initialize when entry from main

* Delete devices.py

* Update add_protocol.md

* delete unecessary files

* feat: 2278 devices registry yaml (#36)

* read chemputer graphml

* read graphml in app/main

* add devices in ros/devices

* add schema for devices

* read registry directory and initialize when entry from main

* Delete devices.py

* add: NMR LH and RU device control

* fix: modify jiageng devices registry

---------

Co-authored-by: Junhan Chang <changjh@pku.edu.cn>

* Device/Resource Registry and GraphML support (#37)

* add resource type conversion to PLR

* add resource registry and test

* add docs

* fix registry

* add solenoid_valve_mock, its registry and test

* fix registry for directly using examples

* add EvacuateAndRefillProtocol and testcases

* allow function sequence call in ACTION

* add read & write & extra_info for hardware_interface

* Update device_node.py

* add solenoid valve

* add doc developer guide yaml

* fixes for starting IK station

* add graphml grouping parser

* fix graphml grouping parser

* add communication edge parser

* fix io solenoid valve

* Update .gitignore

* Update plates.yaml

---------

Co-authored-by: ColumbiaCC <2100011801@stu.pku.edu.cn>

* Uni-Lab Doc v0.2 (#39)

* add Uni-Lab docs

* change doc name

* Dev (#41)

* Integration with pywinauto & recorder
Added execute run and initialize procdure

* 酶标仪状态检测、使用示例,整体流程待测试

* nivo ready version

* Add HPLC driver and example script

- Introduced HPLCDriver class for managing HPLC device status and operations.
- Implemented device status monitoring and command execution via ROS2 actions.
- Added example script (hplc.py) demonstrating how to run commands on the HPLC device.
- Created PlayerUtil and UniversalDriver classes for shared functionality across devices.
- Refactored NivoDriver to utilize the new UniversalDriver structure.
- Enhanced error handling and process management in the NivoDriver.

* 修复start的错误定位

* hplc tested ok

* relative path to build msgs

* template_driver & jiageng devices

* fetch correct status type and action type

* fix mtype fetch

* gpc bus integration

* ilab build

* remove chs

* recipe rename

* modbus update 1

* json available

* hplc & modbus rewrite

* Update AgilentHPLC.py

hplc datafile reader

* move ilabos/ros/rpc to ilabos/device_comms/rpc, and merge bioyond/aichemeco files under /devices

* modbus分设备

* gpc

* gpc 2

* fix address

* default register node

* fix MainScreenGPC

* add Resource srv and message_converter

* move graphio to ilabos/resources

* refactor resources type conversion

* add resource clients in device_node

* add mock resources service

* pass Gripper1 resource test

* update http resource services

* add AGV compile function

* add AGV transfer protocol

* update recipe.yaml

* update full mock_gripper edit_id example

* update full mock_gripper edit_id example

* get and update resource also in protocol_node

* mock resource update in AichemecoHiwo

* feat: add other jiageng PLC device code

* ilabos compile

* correct format

* correct recipe format

* correct setup.py format

* remove unnecessary files

* remove unnecessary files

* Create HT_hiwo.json

* add children in resources

* hplc support sample_id

* correct hplc sample_id

* correct hplc sample_id

* hplc upload

* fix type hint

* oss upload tested ver

* recipe yaml fix for linux

* update installation yaml

* refactor: moved all driver files according to its feat

* merge main to dev

* add HPLC registry and json

* 升级 ros2-distro-mutex 依赖版本至 0.6

* 修改 ros2-distro-mutex 依赖版本为通配符匹配

* 更新 ros-humble-ilabos-msgs 依赖为 robostack-humble 命名空间

* add resource type conversion to PLR

* add resource registry and test

* feat: 更新oss上传

* fix device id

* add docs

* fix registry

* add solenoid_valve_mock, its registry and test

* fix registry for directly using examples

* add EvacuateAndRefillProtocol and testcases

* allow function sequence call in ACTION

* add read & write & extra_info for hardware_interface

* Update device_node.py

* add solenoid valve

* add doc developer guide yaml

* use robostack-staging

* rclpy version test

* lower rclpy

* ensure 0.6* env

* fixes for starting IK station

* add graphml grouping parser

* fix graphml grouping parser

* add communication edge parser

* fix io solenoid valve

* Update .gitignore

* Update plates.yaml

* Feature/device node later init (#40)

* 修改config路径,方便后续打包
增加device_node打印

* 支持plr序列化/init创建

* 统一命名

* import mgr
logger optimize
banner print

* 日志OK

* fix unicorn frame

* banner print

* correct import format

* file path changes

* 取消后补全,在加载设备的时候直接替换

* converter update

* web page update

* 在线device更新,node继承替换

* 修复动作、状态的类型缺失 和 命令提示

* web功能实现结束

* host节点更改完成
新增status时间戳管理
新增每10s动态发现其他node

* ros2类型的节点也应该被包一次

* 修复类型提示

* websocket 动态显示状态

* add workflow & book theme for docs

* add workflow & book theme for docs

* fix workflow build

* fix workflow build

* 理清启动关系

* stm32 example

* mac . name

* device_instance device_cls

* 新增config添加方式
更新mqtt提示

* plr test

* 移动is_host_mode
新增slave_no_host

* 确保config优先修改生效

* fix graph io

* 支持带参数传入

* 支持物料解析

* 支持物料解析

* device为空的时候不进行绑定或初始化

* protocol node new

* protocol node runnable

* protocol node runnable

---------

Co-authored-by: 王俊杰 <2201110460@stu.pku.edu.cn>
Co-authored-by: Junhan Chang <changjh@pku.edu.cn>
Co-authored-by: jiawei <miaojiawei@dp.tech>
Co-authored-by: ColumbiaCC <2100011801@stu.pku.edu.cn>

* Dev (#45)

* Integration with pywinauto & recorder
Added execute run and initialize procdure

* 酶标仪状态检测、使用示例,整体流程待测试

* nivo ready version

* Add HPLC driver and example script

- Introduced HPLCDriver class for managing HPLC device status and operations.
- Implemented device status monitoring and command execution via ROS2 actions.
- Added example script (hplc.py) demonstrating how to run commands on the HPLC device.
- Created PlayerUtil and UniversalDriver classes for shared functionality across devices.
- Refactored NivoDriver to utilize the new UniversalDriver structure.
- Enhanced error handling and process management in the NivoDriver.

* 修复start的错误定位

* hplc tested ok

* relative path to build msgs

* template_driver & jiageng devices

* fetch correct status type and action type

* fix mtype fetch

* gpc bus integration

* ilab build

* remove chs

* recipe rename

* modbus update 1

* json available

* hplc & modbus rewrite

* Update AgilentHPLC.py

hplc datafile reader

* move ilabos/ros/rpc to ilabos/device_comms/rpc, and merge bioyond/aichemeco files under /devices

* modbus分设备

* gpc

* gpc 2

* fix address

* default register node

* fix MainScreenGPC

* add Resource srv and message_converter

* move graphio to ilabos/resources

* refactor resources type conversion

* add resource clients in device_node

* add mock resources service

* pass Gripper1 resource test

* update http resource services

* add AGV compile function

* add AGV transfer protocol

* update recipe.yaml

* update full mock_gripper edit_id example

* update full mock_gripper edit_id example

* get and update resource also in protocol_node

* mock resource update in AichemecoHiwo

* feat: add other jiageng PLC device code

* ilabos compile

* correct format

* correct recipe format

* correct setup.py format

* remove unnecessary files

* remove unnecessary files

* Create HT_hiwo.json

* add children in resources

* hplc support sample_id

* correct hplc sample_id

* correct hplc sample_id

* hplc upload

* fix type hint

* oss upload tested ver

* recipe yaml fix for linux

* update installation yaml

* refactor: moved all driver files according to its feat

* merge main to dev

* add HPLC registry and json

* 升级 ros2-distro-mutex 依赖版本至 0.6

* 修改 ros2-distro-mutex 依赖版本为通配符匹配

* 更新 ros-humble-ilabos-msgs 依赖为 robostack-humble 命名空间

* add resource type conversion to PLR

* add resource registry and test

* feat: 更新oss上传

* fix device id

* add docs

* fix registry

* add solenoid_valve_mock, its registry and test

* fix registry for directly using examples

* add EvacuateAndRefillProtocol and testcases

* allow function sequence call in ACTION

* add read & write & extra_info for hardware_interface

* Update device_node.py

* add solenoid valve

* add doc developer guide yaml

* use robostack-staging

* rclpy version test

* lower rclpy

* ensure 0.6* env

* fixes for starting IK station

* add graphml grouping parser

* fix graphml grouping parser

* add communication edge parser

* fix io solenoid valve

* Update .gitignore

* Update plates.yaml

* Feature/device node later init (#40)

* 修改config路径,方便后续打包
增加device_node打印

* 支持plr序列化/init创建

* 统一命名

* import mgr
logger optimize
banner print

* 日志OK

* fix unicorn frame

* banner print

* correct import format

* file path changes

* 取消后补全,在加载设备的时候直接替换

* converter update

* web page update

* 在线device更新,node继承替换

* 修复动作、状态的类型缺失 和 命令提示

* web功能实现结束

* host节点更改完成
新增status时间戳管理
新增每10s动态发现其他node

* ros2类型的节点也应该被包一次

* 修复类型提示

* websocket 动态显示状态

* add workflow & book theme for docs

* add workflow & book theme for docs

* fix workflow build

* fix workflow build

* 理清启动关系

* stm32 example

* mac . name

* device_instance device_cls

* 新增config添加方式
更新mqtt提示

* plr test

* 移动is_host_mode
新增slave_no_host

* 确保config优先修改生效

* fix graph io

* 支持带参数传入

* 支持物料解析

* 支持物料解析

* device为空的时候不进行绑定或初始化

* protocol node new

* protocol node runnable

* protocol node runnable

* Feature/device node later init (#42)

* 修改config路径,方便后续打包
增加device_node打印

* 支持plr序列化/init创建

* 统一命名

* import mgr
logger optimize
banner print

* 日志OK

* fix unicorn frame

* banner print

* correct import format

* file path changes

* 取消后补全,在加载设备的时候直接替换

* converter update

* web page update

* 在线device更新,node继承替换

* 修复动作、状态的类型缺失 和 命令提示

* web功能实现结束

* host节点更改完成
新增status时间戳管理
新增每10s动态发现其他node

* ros2类型的节点也应该被包一次

* 修复类型提示

* websocket 动态显示状态

* add workflow & book theme for docs

* add workflow & book theme for docs

* fix workflow build

* fix workflow build

* 理清启动关系

* stm32 example

* mac . name

* device_instance device_cls

* 新增config添加方式
更新mqtt提示

* plr test

* 移动is_host_mode
新增slave_no_host

* 确保config优先修改生效

* fix graph io

* 支持带参数传入

* 支持物料解析

* 支持物料解析

* device为空的时候不进行绑定或初始化

* protocol node new

* protocol node runnable

* protocol node runnable

* action

* plr suc

* plr suc!!

* plr suc!!

* plr suc!!

* plr msgs

* Feature/device node later init (#43)

* 修改config路径,方便后续打包
增加device_node打印

* 支持plr序列化/init创建

* 统一命名

* import mgr
logger optimize
banner print

* 日志OK

* fix unicorn frame

* banner print

* correct import format

* file path changes

* 取消后补全,在加载设备的时候直接替换

* converter update

* web page update

* 在线device更新,node继承替换

* 修复动作、状态的类型缺失 和 命令提示

* web功能实现结束

* host节点更改完成
新增status时间戳管理
新增每10s动态发现其他node

* ros2类型的节点也应该被包一次

* 修复类型提示

* websocket 动态显示状态

* add workflow & book theme for docs

* add workflow & book theme for docs

* fix workflow build

* fix workflow build

* 理清启动关系

* stm32 example

* mac . name

* device_instance device_cls

* 新增config添加方式
更新mqtt提示

* plr test

* 移动is_host_mode
新增slave_no_host

* 确保config优先修改生效

* fix graph io

* 支持带参数传入

* 支持物料解析

* 支持物料解析

* device为空的时候不进行绑定或初始化

* protocol node new

* protocol node runnable

* protocol node runnable

* action

* plr suc

* plr suc!!

* plr suc!!

* plr suc!!

* plr msgs

* plr

* action

* plr reg fix

* Feature/device node later init (#44)

* 修改config路径,方便后续打包
增加device_node打印

* 支持plr序列化/init创建

* 统一命名

* import mgr
logger optimize
banner print

* 日志OK

* fix unicorn frame

* banner print

* correct import format

* file path changes

* 取消后补全,在加载设备的时候直接替换

* converter update

* web page update

* 在线device更新,node继承替换

* 修复动作、状态的类型缺失 和 命令提示

* web功能实现结束

* host节点更改完成
新增status时间戳管理
新增每10s动态发现其他node

* ros2类型的节点也应该被包一次

* 修复类型提示

* websocket 动态显示状态

* add workflow & book theme for docs

* add workflow & book theme for docs

* fix workflow build

* fix workflow build

* 理清启动关系

* stm32 example

* mac . name

* device_instance device_cls

* 新增config添加方式
更新mqtt提示

* plr test

* 移动is_host_mode
新增slave_no_host

* 确保config优先修改生效

* fix graph io

* 支持带参数传入

* 支持物料解析

* 支持物料解析

* device为空的时候不进行绑定或初始化

* protocol node new

* protocol node runnable

* protocol node runnable

* action

* plr suc

* plr suc!!

* plr suc!!

* plr suc!!

* plr msgs

* plr

* fix convert error
fix async logic error
added async error print

* new device test

* test resource add

* test resource add

* test resource add

* test resource add

* local env setup

* node type fix
temp fix root_node error
fix convert res from type error

* resource tracker

* fix bug from qhh

* fix bug from qhh

* fix bug from qhh

* fix bug from qhh

* refactor MQTT client logging and connection handling; update group ID in config

* driver_params allow empty

* allow other init param

* fix driver param and enhance type hint

* refactor MQConfig to use double quotes for string literals

* fix wrong function calling

* fix wrong function calling

* fix log for mac

* fix networkx compatibility

* add mqtt loggers

* add action to jsonschema converter

* random client id

* type converter & registry

* correct conversion

* fix action publish only from discovered devices

* add "Bio" tag for action doc generation

* 改进module提示

* Fix doc

* mqtt不连接也可用
性价样例提示

* add docs

* 更新plr test案例

* Update intro.md

* 更新有机案例

* skip

---------

Co-authored-by: Harvey Que <Q-Query@outlook.com>
Co-authored-by: Junhan Chang <1700011741@pku.edu.cn>

---------

Co-authored-by: 王俊杰 <2201110460@stu.pku.edu.cn>
Co-authored-by: Junhan Chang <changjh@pku.edu.cn>
Co-authored-by: jiawei <miaojiawei@dp.tech>
Co-authored-by: ColumbiaCC <2100011801@stu.pku.edu.cn>
Co-authored-by: Harvey Que <Q-Query@outlook.com>
Co-authored-by: Junhan Chang <1700011741@pku.edu.cn>

* Canonicalize before Open Source (#46)

* big big refactor try01

* refactor 02

---------

Co-authored-by: ck <xiaoyeqiannian@163.com>
Co-authored-by: 王俊杰 <1800011822@pku.edu.cn>
Co-authored-by: q434343 <554662886@qq.com>
Co-authored-by: Xuwznln <xuwznln@gmail.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: wjjxxx <43375851+wjjxxx@users.noreply.github.com>
Co-authored-by: 3218923350 <105201755+3218923350@users.noreply.github.com>
Co-authored-by: Xuwznln <1023025701@qq.com>
Co-authored-by: 王俊杰 <2201110460@stu.pku.edu.cn>
Co-authored-by: jiawei <miaojiawei@dp.tech>
Co-authored-by: Jiawei <91898272+jiawei723@users.noreply.github.com>
Co-authored-by: ColumbiaCC <2100011801@stu.pku.edu.cn>
Co-authored-by: Harvey Que <Q-Query@outlook.com>
This commit is contained in:
Junhan Chang
2025-04-17 15:09:58 +08:00
committed by GitHub
parent 7ccb425e39
commit a62a695812
266 changed files with 40772 additions and 2 deletions

315
unilabos/utils/log.py Normal file
View File

@@ -0,0 +1,315 @@
import logging
import os
import platform
from datetime import datetime
import ctypes
import atexit
import inspect
from typing import Tuple, cast
class CustomRecord:
custom_stack_info: Tuple[str, int, str, str]
# Windows颜色支持
if platform.system() == "Windows":
# 尝试启用Windows终端的ANSI支持
kernel32 = ctypes.windll.kernel32
# 获取STD_OUTPUT_HANDLE
STD_OUTPUT_HANDLE = -11
# 启用ENABLE_VIRTUAL_TERMINAL_PROCESSING
ENABLE_VIRTUAL_TERMINAL_PROCESSING = 0x0004
# 获取当前控制台模式
handle = kernel32.GetStdHandle(STD_OUTPUT_HANDLE)
mode = ctypes.c_ulong()
kernel32.GetConsoleMode(handle, ctypes.byref(mode))
# 启用ANSI处理
kernel32.SetConsoleMode(handle, mode.value | ENABLE_VIRTUAL_TERMINAL_PROCESSING)
# 程序退出时恢复控制台设置
@atexit.register
def reset_console():
kernel32.SetConsoleMode(handle, mode.value)
# 定义不同日志级别的颜色
class ColoredFormatter(logging.Formatter):
"""自定义日志格式化器,支持颜色输出"""
# ANSI 颜色代码
COLORS = {
"RESET": "\033[0m", # 重置
"BOLD": "\033[1m", # 加粗
"GRAY": "\033[37m", # 灰色
"WHITE": "\033[97m", # 白色
"BLACK": "\033[30m", # 黑色
"DEBUG_LEVEL": "\033[1;36m", # 加粗青色
"INFO_LEVEL": "\033[1;32m", # 加粗绿色
"WARNING_LEVEL": "\033[1;33m", # 加粗黄色
"ERROR_LEVEL": "\033[1;31m", # 加粗红色
"CRITICAL_LEVEL": "\033[1;35m", # 加粗紫色
"DEBUG_TEXT": "\033[37m", # 灰色
"INFO_TEXT": "\033[97m", # 白色
"WARNING_TEXT": "\033[33m", # 黄色
"ERROR_TEXT": "\033[31m", # 红色
"CRITICAL_TEXT": "\033[35m", # 紫色
"DATE": "\033[37m", # 日期始终使用灰色
}
def __init__(self, use_colors=True):
super().__init__()
# 强制启用颜色
self.use_colors = use_colors
def format(self, record):
# 检查是否有自定义堆栈信息
if hasattr(record, "custom_stack_info") and record.custom_stack_info: # type: ignore
r = cast(CustomRecord, record)
frame_info = r.custom_stack_info
record.filename = frame_info[0]
record.lineno = frame_info[1]
record.funcName = frame_info[2]
if len(frame_info) > 3:
record.name = frame_info[3]
if not self.use_colors:
return self._format_basic(record)
level_color = self.COLORS.get(f"{record.levelname}_LEVEL", self.COLORS["WHITE"])
text_color = self.COLORS.get(f"{record.levelname}_TEXT", self.COLORS["WHITE"])
date_color = self.COLORS["DATE"]
reset = self.COLORS["RESET"]
# 日期格式
datetime_str = datetime.fromtimestamp(record.created).strftime("%y-%m-%d [%H:%M:%S,%f")[:-3] + "]"
# 模块和函数信息
filename = record.filename.replace(".py", "").split("\\")[-1] # 提取文件名(不含路径和扩展名)
if "/" in filename:
filename = filename.split("/")[-1]
module_path = f"{record.name}.{filename}"
func_line = f"{record.funcName}:{record.lineno}"
right_info = f" [{func_line}] [{module_path}]"
# 主要消息
main_msg = record.getMessage()
# 构建基本消息格式
formatted_message = (
f"{date_color}{datetime_str}{reset} "
f"{level_color}[{record.levelname}]{reset} "
f"{text_color}{main_msg}"
f"{date_color}{right_info}{reset}"
)
# 处理异常信息
if record.exc_info:
exc_text = self.formatException(record.exc_info)
if formatted_message[-1:] != "\n":
formatted_message = formatted_message + "\n"
formatted_message = formatted_message + text_color + exc_text + reset
elif record.stack_info:
if formatted_message[-1:] != "\n":
formatted_message = formatted_message + "\n"
formatted_message = formatted_message + text_color + self.formatStack(record.stack_info) + reset
return formatted_message
def _format_basic(self, record):
"""基本格式化,不包含颜色"""
datetime_str = datetime.fromtimestamp(record.created).strftime("%y-%m-%d [%H:%M:%S,%f")[:-3] + "]"
filename = os.path.basename(record.filename).rsplit(".", 1)[0] # 提取文件名(不含路径和扩展名)
module_path = f"{record.name}.{filename}"
func_line = f"{record.funcName}:{record.lineno}"
formatted_message = f"{datetime_str} [{record.levelname}] [{module_path}] [{func_line}]: {record.getMessage()}"
if record.exc_info:
exc_text = self.formatException(record.exc_info)
if formatted_message[-1:] != "\n":
formatted_message = formatted_message + "\n"
formatted_message = formatted_message + exc_text
elif record.stack_info:
if formatted_message[-1:] != "\n":
formatted_message = formatted_message + "\n"
formatted_message = formatted_message + self.formatStack(record.stack_info)
return formatted_message
def formatException(self, exc_info):
"""重写异常格式化,确保异常信息保持正确的格式和颜色"""
# 获取标准的异常格式化文本
formatted_exc = super().formatException(exc_info)
return formatted_exc
# 配置日志处理器
def configure_logger():
"""配置日志记录器"""
# 获取根日志记录器
root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG) # 修改为DEBUG以显示所有级别
# 移除已存在的处理器
for handler in root_logger.handlers[:]:
root_logger.removeHandler(handler)
# 创建控制台处理器
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG) # 修改为DEBUG以显示所有级别
# 使用自定义的颜色格式化器
color_formatter = ColoredFormatter()
console_handler.setFormatter(color_formatter)
# 添加处理器到根日志记录器
root_logger.addHandler(console_handler)
# 配置日志系统
configure_logger()
# 获取日志记录器
logger = logging.getLogger(__name__)
# 获取调用栈信息的工具函数
def _get_caller_info(stack_level=0) -> Tuple[str, int, str, str]:
"""
获取调用者的信息
Args:
stack_level: 堆栈回溯的级别0表示当前函数1表示调用者依此类推
Returns:
(filename, line_number, function_name, module_name) 元组
"""
# 堆栈级别需要加3:
# +1 因为这个函数本身占一层
# +1 因为日志函数(debug, info等)占一层
# +1 因为下面调用 inspect.stack() 也占一层
frame = inspect.currentframe()
try:
# 跳过适当的堆栈帧
for _ in range(stack_level + 3):
if frame and frame.f_back:
frame = frame.f_back
else:
break
if frame:
filename = frame.f_code.co_filename if frame.f_code else "unknown"
line_number = frame.f_lineno if hasattr(frame, "f_lineno") else 0
function_name = frame.f_code.co_name if frame.f_code else "unknown"
# 获取模块名称
module_name = "unknown"
if frame.f_globals and "__name__" in frame.f_globals:
module_name = frame.f_globals["__name__"].rsplit(".", 1)[0]
return (filename, line_number, function_name, module_name)
return ("unknown", 0, "unknown", "unknown")
finally:
del frame # 避免循环引用
# 便捷日志记录函数
def debug(msg, *args, stack_level=0, **kwargs):
"""
记录DEBUG级别日志
Args:
msg: 日志消息
stack_level: 堆栈回溯级别,用于定位日志的实际调用位置
*args, **kwargs: 传递给logger.debug的其他参数
"""
# 获取调用者信息
if stack_level > 0:
caller_info = _get_caller_info(stack_level)
extra = kwargs.get("extra", {})
extra["custom_stack_info"] = caller_info
kwargs["extra"] = extra
logger.debug(msg, *args, **kwargs)
def info(msg, *args, stack_level=0, **kwargs):
"""
记录INFO级别日志
Args:
msg: 日志消息
stack_level: 堆栈回溯级别,用于定位日志的实际调用位置
*args, **kwargs: 传递给logger.info的其他参数
"""
if stack_level > 0:
caller_info = _get_caller_info(stack_level)
extra = kwargs.get("extra", {})
extra["custom_stack_info"] = caller_info
kwargs["extra"] = extra
logger.info(msg, *args, **kwargs)
def warning(msg, *args, stack_level=0, **kwargs):
"""
记录WARNING级别日志
Args:
msg: 日志消息
stack_level: 堆栈回溯级别,用于定位日志的实际调用位置
*args, **kwargs: 传递给logger.warning的其他参数
"""
if stack_level > 0:
caller_info = _get_caller_info(stack_level)
extra = kwargs.get("extra", {})
extra["custom_stack_info"] = caller_info
kwargs["extra"] = extra
logger.warning(msg, *args, **kwargs)
def error(msg, *args, stack_level=0, **kwargs):
"""
记录ERROR级别日志
Args:
msg: 日志消息
stack_level: 堆栈回溯级别,用于定位日志的实际调用位置
*args, **kwargs: 传递给logger.error的其他参数
"""
if stack_level > 0:
caller_info = _get_caller_info(stack_level)
extra = kwargs.get("extra", {})
extra["custom_stack_info"] = caller_info
kwargs["extra"] = extra
logger.error(msg, *args, **kwargs)
def critical(msg, *args, stack_level=0, **kwargs):
"""
记录CRITICAL级别日志
Args:
msg: 日志消息
stack_level: 堆栈回溯级别,用于定位日志的实际调用位置
*args, **kwargs: 传递给logger.critical的其他参数
"""
if stack_level > 0:
caller_info = _get_caller_info(stack_level)
extra = kwargs.get("extra", {})
extra["custom_stack_info"] = caller_info
kwargs["extra"] = extra
logger.critical(msg, *args, **kwargs)
# 测试日志输出(如果直接运行此文件)
if __name__ == "__main__":
print("测试不同日志级别的颜色输出:")
debug("这是一条调试日志 (DEBUG级别显示为蓝色其他文本为灰色)")
info("这是一条信息日志 (INFO级别显示为绿色其他文本为白色)")
warning("这是一条警告日志 (WARNING级别显示为黄色其他文本也为黄色)")
error("这是一条错误日志 (ERROR级别显示为红色其他文本也为红色)")
critical("这是一条严重错误日志 (CRITICAL级别显示为紫色其他文本也为紫色)")
# 测试异常输出
try:
1 / 0
except Exception as e:
error(f"发生错误: {e}", exc_info=True)