A touch of collaboration

We all encounter informative signage, directions and instructions through visual, aural and other sensory cues every day as we navigate our world, whether we realise it or not.

But for those with visual and aural impairments, the importance of these cues is intensified. When a location gets them right for those with little or no sight, day-to-day life can be made so much easier. Get them wrong and the consequences could be disastrous.

Understanding and developing vibrations and sensations to improve the quality of life of people with visual and aural impairments is therefore an important task. Achieving this ideal is a mission for both Dr Raymond Holt and Bryan Matthews.

Raymond’s interest hasn’t always been in this area.

Before colleague Bryan Matthews approached him to discuss a potential project, Dr Holt’s research attentions had been focused elsewhere.

“Because I wanted to be a design engineer, I studied Mechanical Engineering at Leeds and spent a year at a chemical plant before returning to do a PhD,” Raymond explained.

“That was all about decision support in design for manufacture, and I ended up doing lots of mixed-methods research on how human beings make decisions before becoming a lecturer.

 “That was when my mentor, the late Anne Neville, mentioned that Professor Martin Levesley was running a rehabilitation project and she thought my interest in user-centred design could help.”

Suddenly, Raymond was doing exactly what he loves most: getting stuck into creative challenges.

“I worked with hospital stroke patients, clinicians and engineers to understand everyone’s views and help develop a control system for iPAM, a robot that helps people rehabilitate their upper limbs.”

If your idea of haptic feedback is a buzzing mobile phone or video game controller, that may not sound too relevant – but if there’s one thing that defines Raymond’s career to date, it’s how serendipitous conversations and meeting different colleagues can lead to new projects and exciting discoveries.

Soon, a similar initiative, aimed at designing a gaming joystick for children with cerebral palsy, led him to meet members of the cross-departmental Centre for Disability Studies, which Raymond is now part of.

This group included Bryan – a man who would influence the direction of Raymond’s career significantly.

Bryan’s interest in using haptics to improve the lives of visually impaired people is perhaps more personal than Raymond’s, given its connection to his own story.

“I started to lose my eyesight when I was eight, and it sort of aggressively faded until I lost it completely at about the age of 18,” he explained.

“It was quite traumatic, but it coincided with a time when I was finishing school – and because my sight was already at a low level, I’d already arranged to go to the Royal National College for the Blind in Hereford the next year.

“Suddenly, I was among hundreds of other people who were either partially sighted or totally blind, meaning it felt entirely normal even though I was effectively learning how to live as a blind person.”

That process naturally involved Bryan using his other senses more than he had previously.

“You sort of look to other ways of getting the information you need to function day to day, and you become aware of all the sources that use the ‘audio channel’, if you will,” Bryan said.

“These days, we all have apps that can speak to you to give you directions or point things out – but all that constant chatter can be really overwhelming.”

Haptics, then, have an obvious link to daily life as a person with limited or no sight.

But how did Raymond get involved?

“Bryan does a lot of work with the Department for Transport, focusing on inclusive and accessible transport, particularly for people like him who are visually impaired.

“Raymond explained that Bryan was leading a project called Wearable Haptic Information Source for Pedestrians and asked if I could make him some prototypes.”

A few encouraging focus groups and simple prototypes convinced him that there was something he found inherently fascinating about haptics.

It also led him to a large EU project with partners from multiple countries, SUITCEYES, which was focused on using haptic technology to improve the lives of people who are deafblind. This in turn drew Dr Holt to another project, the Wellcome Trust-funded Imagining Technologies for Disability Futures.

“Rather than trying to develop new products, the idea was to take a more critical approach and say, well, you know, if we’re working with people who are disabled in some way or have impairments, what do they imagine for the future of technology?” Raymond explained.

With volunteered support from visually impaired mentors from the world of arts such as Russ Palmer, Femke Krijger and Simon Dogger, Raymond collaborated with people with disabilities to explore the connections between disability, culture, design and technology.

During this work, a Leeds academic called Mohammed Shaqura recommended that Raymond apply for a Michael Beverley Innovation Fellowship, a pioneering initiative that helps to fund cross-disciplinary research with a view to achieving commercialisation opportunities.

“I realised it was precisely what I needed. I knew how to write publications, find conferences and all the other parts of research, but what I wanted was to learn how to get things out of the lab and into actual products,” Raymond said.

Through his fellowship, Dr Holt is now focused squarely on creating haptic feedback devices and equipment that aid navigation for people with visual impairments.

Raymond has learned the hard way that designing haptic feedback solutions is by no means an easy feat.

“Everything looks easy in hindsight,” he laughed.

“One of the very first things I built was a vest absolutely bristling with ultrasonic sensors and vibration motors, the idea being that anytime there was an obstacle in your way, it would vibrate.

“That’s brilliant in a field, but when I wore it in a crowded environment I realised it was a horrible mistake – I was just being buzzed all the time.

“A similar thing happened when we tried to use thermal feedback, which turned out to be too slow and very difficult to control.”

For Bryan, understanding how haptics might be received by people is critical, too.

“I’m not at the same technical level as Ray by any stretch; my fascination is in what the people who use haptic feedback think of it in certain applications and how it can be used in new ways,” he said.

Raymond is acutely conscious of the need to involve the end users of his technology early on in the design process – but again, that can be more complicated than it appears, too.

“I was once explaining to a group of visually impaired people, most of whom had used sight when they were younger, how they’d receive a gentle vibration if there’s an obstacle within three metres and a stronger one if something is within one metre,” he explained.

“And then one lady who was blind from birth said to me ‘you’re going to have to explain it to me better than that; I have no idea what three metres is or a metre is’.

“One of things I’ve learned is that the further away I go from talking to users and actively involving them, the further away I get from what’s really useful – you get dragged down the rabbit hole of what seems easy.”

Three years ago, Raymond had another epiphany of sorts.

“I was diagnosed with autism, which was a real revelation and suddenly explained so much,” he said.

“It’s actually helped my work-life balance and my research productivity because I now use energy accounting – sometimes called the ‘spoon theory’ – to recognise I can get burned out and make sure I’m giving myself time to recover after busy periods with lots of focus.”

Indeed, it’s undeniable that Dr Holt brings lots of energy to his work.

“What drives me is that I’m fortunate to work with great people.

“And I’ve been really fortunate to have the opportunity to collaborate with people who are great to work with, very co-operative, very supportive.

“My line manager, Professor Alison McKay, has been a huge support over the 18 years or so I’ve worked here, but it’s more than that, too.

“I enjoy working across disciplines; it feels like a sort of joint enterprise with all these academics and people coming together to move things forward, so you always feel like you are contributing to something bigger.”

Looking forward, Raymond is incredibly optimistic about the future of haptic feedback – and the possibilities for creating life-changing products that his Michael Beverley Innovation Fellowship has opened up.

“There are companies out there who are making things that replicate incredibly specific, nuanced sensations – how it feels when you hold a cup that is filling up with water or has marbles swirling around in it, for instance.

“What particularly interests me now is how we can take the things that are most important from that high-fidelity side of haptics and reproduce them more cheaply so that they are as accessible as possible to the end users who stand to benefit the most.”

All in all, it’s fair to say Raymond gets a buzz out of his job.

As for Bryan, he’s determined to keep working on projects that strike the fine balance needed for people like him with limited or no eyesight.

“The challenge is making something useful and versatile enough to be beneficial, but at the same time intuitive and accessible enough to be practical,” Bryan said.

“If you have to have a complicated language or a manual – sort of ‘one buzz means a lamppost ahead, three buzzes means a bus stop’ – then no-one is going to want to experience quite that much vibration.

“Even before Ray gets to work on placing the sensors and making the designs work, that’s a challenge we focus on tackling.”

About Raymond

Dr Raymond Holt is a lecturer in the School of Mechanical Engineering at the University of Leeds.

He graduated from Leeds with a BEng in Mechanical Engineering in 2001, returning after a year in industry to complete his PhD in the subject in 2005. He hasn’t left since.

Raymond’s areas of expertise include ergonomics, design, prehension, manipulation, inclusive design, rehabilitation and kinematics.

About Bryan

Bryan Matthews is a lecturer in the Institute for Transport Studies at the University of Leeds and a member of the Disabled Persons Transport Advisory Committee.

His areas of expertise include transport access and social exclusion, technologies to support older and disabled people’s travel and public transport policy.

Further reading

Read other articles in the Further Together series, celebrating the collaborative achievements of our groundbreaking research community.