Opportunistic collaboration between heterogeneous agents using an unstructured ontology via GenAI

Opportunistic collaborations allow autonomous agents to dynamically cooperate with each other to assess a shared goal. This type of collaboration represents an exciting technological capability avenue by combining several key technological advancements in hardware (miniaturized components, high computational capabilities), in communication (5G networks, Cloud Solutions) along with powerful Artificial Intelligence (AI) solutions. Among the enablers for opportunistic collaborations are universal ontology and world representation for a common advanced understanding of tasks and overall capabilities. This paper introduces an opportunistic collaborative framework with a case study involving two heterogeneous agents with different sensory abilities—a depth camera and a wheeled robot. The communication is established through an unstructured ontology via Generative AI (GenAI). The mission involves the retrieval of an object within the camera’s Field Of View (FOV) that is obstructed from the robot. The camera performs object detection and localization in the environment and the robot navigates using its LiDAR. The implementation is done within the ROS2 (Robot Operating System) middleware. To assess the collaboration efficiency, an autonomous navigation is conducted in parallel, with a depth camera integrated within the robot itself. The evaluation is carried out using a Monte Carlo simulation. The results demonstrate that the collaborative-based scenario is superior to the autonomous-based scenario in terms of time and distance (energy saving). This approach enables heterogeneous smart devices that have no prior knowledge of one another to communicate naturally without following any predefined protocol and to cooperate when situated independently in different locations in an unknown environment.