Robotics researchers and developers can now rely on the GazeSense™ bridge to measure attention toward objects in ROS
GazeSense is an application that provides 3D eye tracking by relying on consumer 3D sensors. GazeSense allows users to define virtual 3D objects concerning the camera and world coordinate systems and measure people’s attention towards the objects.
Through the newly released GazeSense ROS plugin, the GazeSense bridge exposes the real-time eye and attention tracking data, generally available through a custom API, into the Robot Operating System (ROS).
What Is ROS?
ROS is a flexible Robot Operating System framework for writing robot software. It’s a collection of tools, libraries, and conventions that aim to simplify the task of creating complex and robust robot behaviors across a wide variety of robotic platforms.
What will GazeSense do on ROS? It offers a mechanism for perception, where GazeSense provides robotics researchers and developers with real-time signals on attention towards objects. One would be able to tell if a person interacting with the robot is looking at either of the defined objects or not or looking at the robot itself. The attention label (‘what is the person looking at’) is then published as a topic into the ROS framework.
Sensing attention is particularly useful for people working in the human-robot interaction (HRI) as attention sensing is critical for understanding engagement intention and for building rapport.
The first iteration of the GazeSense ROS plugin is focused on a hard-coded version to allow a user to:
- define objects and feed them into GazeSense;
- capture the GazeSense signals on attention towards objects and publish it into ROS.
‘Robot playing piano,’ Photo by Franck V. on Unsplash
If you are looking for more features, like, for example, a more ‘plug-n-play’ object definition for the GazeSense ROS bridge, reach out to us to let us know via an email to [email protected] or by simply submitting your request in our Ideas & Requests Trello Board.
Where can I get the GazeSense ROS plugin?
Gaze Sense is hosted on a Github repository for easy access and view.
Please take me to the GazeSense ROS plugin!
Why did we create a GazeSense ROS plugin available for all robotics researchers and developers using ROS?
Because creating genuinely robust and general-purpose robot software is hard, and no one can hope to do it alone. We are thus joining the world-class, collaborative robotics software development platform and vibrant ROS community of roboticists to help produce robust solutions and build on each other’s work.
The core of ROS is licensed under the standard three-clause BSD license, a very permissive open license that allows for reuse in commercial and closed source products.
Did you know? Through their AWS RoboMaker, Amazon is using ROS to offer developers a place to develop and test robotics applications. If curious, you can explore ROS’s entire impactful history from Wikipedia by scrolling down to ‘milestones.’
Wanna have a GazeSense plugin available for another type of development platform or framework out there? Do you work with Unity, Matlab, or other niche platforms? Submit your preferred choice to our Ideas & Requests Board or by email to [email protected], and your voice will be heard! We aim to support your efforts and bring 3D eye tracking to real-world applications with easy integrations.