Sara: The Socially Aware Robot Assistant

POSTED BY   James Dearsley
27th January 2017
Sara: The Socially Aware Robot Assistant

“Currently major tech companies envision intelligent personal assistants, such as Apple Siri, Microsoft Cortana, and Amazon Alexa as the front ends to their services. However those assistants really play little other role than to query, with voice input and output – they fulfil very few of the functions that a human assistant might. In this demo, we present SARA, the Socially-Aware Robot Assistant, represented by a humanoid animated character on a computer screen, which achieves similar functionality, but through multimodal interaction, and with a focus on building a social relationship with users.”

Yoichi Matsuyama, Arjun Bhardwaj, Ran Zhao, Oscar J. Romero, Sushma Anand Akoju and Justine Cassell ArticuLab, Carnegie Mellon University, Pittsburgh

Sara, like Scarlett Johansson’s character, Samantha, in the film ‘Her’ (video above), understands spoken words and non-verbal behaviour, which she uses to build a relationship with her human.

Developed by professors at Carnegie Mellon University, Sara began life as a research project. However, her developers are now beginning to see her potential for being licensed to a company or organisation for commercial use.

Again, like Samantha, Sara is not an embodied robot, but lives within a computer screen. She interacts with you in front of a camera, and asks you questions. She watches your facial features – how your eyebrows, your mouth, your eyes, move, detects your mood, and can have a conversation with you.

As a personal assistant, her primary function is to perform tasks on your behalf. She can offer to call somebody for you, or send them an email. She can teach children, as a kind of personal tutor, and keep the elderly company.

Getting To Know You

Her machine learning algorithms allow her to build up an ever-improving relationship with her user, learning to read their individual facial expressions, tones of speech, and body language to get to know them better. This is achieved through the capabilities of OpenFace and OpenSmile software.

“The system then employs appropriate conversational strategies to raise the level of rapport with the human user, or to maintain the rapport at the same level. The goal is to maintain rapport at a fairly high level, and to use the rapport as a way of eliciting personal information from the user that, in turn, can be used to personalize system responses.”

Articulab, Carnegie Mellon University

The tasks she can perform are also subject to ongoing improvements as she learns more about your needs and requirements, as well as the specific ways you prefer things to be done.

This video is a good example of how Sara is being used at present.

My Thoughts

My thoughts are that, although the technology behind Sara is pretty cool, she’s a bit basic right now in terms of her vocal articulation and the sophistication of her visual avatar. There’s a bit of lag in her expressions, which make her seem a bit outdated. Basically, I was expecting a bit more from an AI that shows so much potential. Nonetheless, it’s important to note that she’s still in an early developmental stage. The capacity for her to evolve through subsequent iterations is promising.

If I am to engage with a personal assistant like this, I want her to be more visually engaging and relatable. I want her conversation to be more natural. But that’s not beyond the bounds of possibility. With this tech in place, we can all have our own Samantha, maybe even voiced by Scarlett Johansson, before long. Then, Alexa, Cortana, and Siri will seem like ancient history.

What Do You Want From A Robot Assistant?

So, what do you think about Sara? Would you like a robot assistant who can help you with all the mundane chores of life? And, if so, what would yours be like? Let us know on Twitter.

Don’t worry, we’ll never share details with anyone.

Sara: The Socially Aware Robot Assistant

James Dearsley

James Dearsley is the Founder and MD of TDMB. In addition to his work with us, he is also a renowned expert in PropTech, and was recently voted the most influential person in PropTech. An impassioned speaker and advocate of technology, particularly in the Property industry, his other interests include beekeeping, real ale, green trousers, and (currently) growing a beard. You can contact James directly via Twitter or LinkedIn, or tweet the TDMB team directly.

Get in Touch With James Dearsley

[email protected]

You may also like

Surviving AI: November’s TechTech Book Club
The Question Of Autonomous Vehicles and Morality
Our Pick Of The Best Robotics and Artificial Intelligence Events 2019