NEW GazeSense™ ROS plugin available

Robotics researchers and developers can now rely on the GazeSense™ bridge to measure attention toward objects in ROS

 

 

GazeSense is an application that provides 3D eye tracking by relying on consumer 3D sensors. GazeSense allows users to define virtual 3D objects with respect to the camera and world coordinate systems and measure the attention of people towards the objects.

Through the newly released GazeSense ROS plugin, the GazeSense bridge exposes the real-time eye and attention tracking data, generally available through a custom API, into the Robot Operating System (ROS).

 

What’s ROS?

ROS is a flexible Robot Operating System framework for writing robot software. It’s a collection of tools, libraries and conventions that aim to simplify the task of creating complex and robust robot behaviours across a wide variety of robotic platforms.

What will GazeSense do on ROS? It offers a mechanism for perception, where GazeSense provides robotics researchers and developers with the real-time signals on attention towards objects. One would be able to tell if a person interacting with the robot is looking at either of the defined objects or not, or if looking at the robot itself. The attention label (‘what is the person looking at’) is then published as a topic into the ROS framework. 

Sensing attention is particularly useful for people working in the human-robot interaction (HRI) as attention sensing is key for understanding engagement, intention and for building rapport.

The first iteration of the GazeSense ROS plugin is focused towards a hard-coded version to allow a user to:

  • define objects and feed them into GazeSense;
  • capture the GazeSense signals on attention towards objects and publish it into ROS.

‘Robot playing piano’, Photo by Franck V. on Unsplash

If you are looking for more features, like for example a more ‘plug-n-play’ object definition for the GazeSense ROS bridge, reach out to us to let us know via an email to [email protected] or by simply submitting your request in our Ideas & Requests Trello Board.

 

Where can I get the GazeSense ROS plugin?

Gaze Sense is currently hosted on a Github repository for your easy access and view.

Take me to the GazeSense ROS plugin!

 

Why did we create a GazeSense ROS plugin available for all robotics researchers and developers using ROS?

Because creating truly robust and general-purpose robot software is hard, and no one can hope to do it alone. We are thus joining the world class, collaborative robotics software development platform and vibrant ROS community of roboticists, to help contribute producing robust solutions and build on each other’s work.

The core of ROS is licensed under the standard three-clause BSD license, which is a very permissive open license that allows for reuse in commercial and closed source products.

Did you know? Amazon, via their AWS RoboMaker, are using ROS to offer developers a place to develop and test robotics applications. If curious you can explore ROS’s entire impactful history from Wikipedia by scrolling down to ‘milestones’.

Wanna have a GazeSense plugin available for another type of development platform or framework out there? Do you work with Unity, Matlab or other niche platforms? Submit your preferred choice to our Ideas & Requests Board or by email to [email protected] and your voice will be heard! We aim to support your efforts and bring  3D eye tracking to real world applications with easy integrations.

Submit New Platform Plugin Request!