Voice

Archive for Voice

Voice: The most natural user interface for healthcare

There’s so much promise, and such a natural fit for voice in healthcare that unlike electronic medical records, we should not have to mandate its use. If anything, right now we are being limited by the lack of HIPAA controls rather than end-user demand. If the sessions at the recent Voice Summit, which was focused broadly on voice tech, and the upcoming Voice of Healthcare and Voice Summit at Connected Health conferences are any indication, there are many natural use cases, and a lot of pent-up demand.

With so many concerns about documentation and screens getting between patients and physicians, and the ability to deliver empathy, and to rapidly learn from interactions using natural language processing, and artificial intelligence, voice seems a natural fit and unobtrusive interface that could leapfrog traditional interfaces.

The Healthcare track at Voice Summit showed some of this promise, but also pointed out that we are still early days. Many solutions are pilots or prototypes, and I had the distinct impression that some of today’s HIPAA workarounds would not stand up to a detailed audit. Despite Alexa’s sponsorship of the conference, Google’s strong presence, and both companies push into all things healthcare, both were mum on whether or when their consumer voice devices might be HIPAA compliant. Regardless, healthcare organizations and technology vendors alike are charging forward on new scenarios for healthcare, and you can see by the diversity that if even a few of these end up being the “killer app” it’s a big opportunity.

Patient Care

Rooming: Waiting for a physician to see you in an exam room is often a wasted opportunity. A voice interface in the clinic room, could help further pinpoint why a patient is having a visit or educate pre and post visit on medical issues. Or simply having a voice assistant capture the questions that a patient has during a visit might go a long way to improving the visit.

Inpatient stay: The combination of voice assistants, wifi, and tablets could completely replace expensive and proprietary systems for inpatient patient engagement. We’re already seeing use cases for anonymous interactions with voice devices to order food, check the time, or find out the time of the next physician visit.

Long-term care: Alzheimers and dementia care are cited as the poster child for the benefits of voice in long-term care facilities. Unlike human caregivers, voice assistants never get tired of answering the same questions repeatedly. There are so many times you don’t want Saturday Night Live to predict the future, but with this one they got it right.

Patient Engagement

If we define patient engagement as interactions outside the clinic, then the opportunities today fall into three main categories triage (or eventually diagnosis), education, and self-management.

Triage Skills: Today we see some basic triage skills from organizations like Mayo Clinic, and Boston Children’s Hospital where you can check some basic first aid, or ask common questions about children’s health. While there are approximately 1,000 healthcare skills, most likely there will be a few winners or “go-to” experiences here from leading healthcare organization or trusted publishers like WebMD. (Interestingly, the presenter from WebMD was one of the more skeptical on voice experiences for patients at the Voice Summit, possibly because of the complexity of the information they present through text, video, and images on the Web.)

Health Education: Chunking content into manageable bites is currently being touted as the best practice for education material through voice. However, this is an area where the interactivity that’s possible through voice will be necessary for stickiness. If you think about the best podcasts, they use different techniques to both engage you and also impart knowledge: interviewing, verbatim quotes, sound effects, interjections, and expository material. To get engaging and sticky health education content, publishers will have to think about how to test for knowledge, advance explanations, and interact with the end-users. Since we can only remember 5 things at a time, simply chunking content is not going to be enough to make the delivery of health education through voice stick.

Reminders and Interactive Health Tasks: As we’ve seen from our testing, where voice interfaces may have the most impact for patients is in helping them complete health tasks for example, in medication adherence, simple surveys, or check-ins and reminders of basic information. Given that the voice interface is a natural in the home, checking in with a voice assistance on when to take medication, or tracking meals is an easy way to engage with a care plan. As well, cloud-based interactive voice response systems could call patients with reminders and check-ins.

Clinical Notes

Conquering the pain of charting is possibly the closest term opportunity for voice in healthcare. With every increasing workloads, and the need to capture information digitally for both care and reimbursement, the EMR has been blamed for physician burnout and lack of job satisfaction. Microsoft recently partnered with UPMC to use their Cortana voice assistant to transcribe clinical notes during a patient/provider interaction. Others attacking this space include SayKara, Robin, and incumbent, Nuance Communications. With HIPAA compliance, it’s hard not to imagine Amazon and Google looking at it as well.

Hands-free lookup

Voice really shines as an interface when your hands are not free, like driving, dentistry, or when you need to keep your hands clean. Voice is a natural in settings where touching a screen or device can cause contamination or distraction. Simplifeye is tackling this in dentistry to improve charting, and lookup of x-rays, and we expect this to infiltrate all aspects of healthcare.

You may have seen a recent article on why Alexa is not ready for healthcare primetime. With all of these great examples it’s hard to believe it. It turns out that the criticisms in this article basically highlight the current limitations of voice overall (except for HIPAA compliance of course). However, some of the challenges of discovery, context, and navigation, are why we at Wellpepper believe in not just voice, but a “Voice And” future where voice is a key interface that is helped or helps others like screens or even augmented reality. Voice is powerful, “Voice And” will be even better.

Posted in: Behavior Change, chronic disease, HIPAA, patient engagement, Patient Satisfaction, patient-generated data, Voice

Leave a Comment (0) →

Voice First or Voice And? Dispatches from Voice Summit

The inaugural Voice Summit was held last week in New Jersey, with the hashtag #voicefirst. At Wellpepper, we’re actually in the “Voice and” camp. We love voice interfaces for their convenience, promotion of empathy and connection, and their natural engagement. However, there are times when voice isn’t the best interface for the task or others when voice plus other interfaces are even better, which is reflected in some of our work with the Alexa Diabetes Challenge, which I spoke about at the conference.

People can only remember 5 things at a time, which is a challenge for delivery of complex instructions, education, or information through voice. Add this to the fact that voice is a “headless” navigation. That is, there are often no cues to figure out where you are going. Most of us are visual creatures, and visual cues together with voice or text often provide a richer experience. And believe it or not, the many of the sessions at this inaugural voice conference also seemed to reinforce this idea, in particular many of the consumer sessions, in addition to the healthcare sessions.

Talks by two very different consumer organizations, Comcast and Lego both showed how early we are in voice design, and how when voice is more seamless and ubiquitous we may see the promise of “voice first” but also how “voice and” is possibly the better path forward.

While when you think of giants of voice, you many immediately think of Amazon and Google, did you know that Comcast processed over 6B voice queries last year? My first thought on attending this session was that it was going to be about using interactive voice response trees before you get to a customer service agent, but Comcast has been quietly infusing voice into their entertainment experiences.

Did you know that your Comcast remote has a “voice” interface? You can talk to your TV to find programs, change the channel, or start a show. This is probably one of the best examples of “voice and.” First, voice search is actually found on a physical device. The Comcast design team had originally created a mobile app for the remote voice experience, but found that downloads were a small fraction of their entire subscriber base, so adding a “voice button” to the remote encouraged more searches. Also remember that when you use voice to search it shows you the results on your television screen. This is a “voice and” experience which wouldn’t make a lot of sense as voice standalone. Imagine searching for a movie to watch, say you’re looking for something starring Harrison Ford, and you’ve got to keep in your mind all the titles over his varied career and then choose one. First it’s a lot to remember, and second isn’t it easier to browse titles when you can see pictures and a description to jog your memory? I spoke briefly with the Comcast presenters about why they chose to put voice on the remote, versus directly in the cable box, and they said that it helped their users find the option, which was a big takeaway from the conference for me, although voice is a natural interface, the end-user still needs guidance. (A nice side benefit of the button on the remote is that it’s not always on and listening.)

Lego was another unlikely consumer company playing in the voice arena. Lego “Duplo Stories” is an Alexa skill that tells stories that children can then build using Duplo blocks. While the video was heartwarming, this session in particular highlighted both opportunities for “Voice And” using augmented reality, and also the current discovery limitations of voice.

In the video, a child playing with Duplo blocks asks his mother to start a story. The mother asks Alexa to play a Duplo story. Think about this: the skill had to be discovered and activated before any of this could take place. How would you learn about the skill without something printed on the box that the Duplo blocks came in? While it’s clever, imagine a new scenario where voice and augmented reality are built right into the blocks: a virtual Duplo minecraft. The child builds something with Duplo, and then a voice and visual interface projects the story on the child’s creation.

It’s still early days, and the potential for “Voice And” is still huge. In fact, a lot of the content at this conference reminded me of the early days of web interfaces. There was lots of talk about taxonomy of information, and “chunking” information into manageable pieces. (I used to teach a course on writing for the web, where we practiced this, which is funny as we now are so accustomed to screens that long-form journalism is making a real comeback.)

Similar to the early days of the web, there seemed to be slightly more focus on publishing than on end-user goals: what does the end-user actually want to accomplish, not what is the end-goal of the content publisher. What’s different though is that while during Web 1.0, the answer to question of whether every business needed a website, was a resounding yes, it’s not clear that everyone needs a voice skill. With 30,000 skills already available for Alexa, and new features coming online weekly, the irony is that the Alexa team sends a weekly newsletter to keep us up to date. So, even Alexa knows it’s a “Voice And” world.

Posted in: Behavior Change, Voice

Leave a Comment (0) →

Voice Tech In Healthcare

Voice tech is a hot topic in healthcare, and for good reason. Healthcare is built on personal interactions, and voice technology can replicate and even replace the human interviewing experience. Voice has other valuable benefits in healthcare like being hands-free—for someone who is recovering from surgery and mobility challenged this might mean being able to get information without getting up. In the hospital setting, the hands-free interface has obvious benefits for hygiene.

At Wellpepper we first started experimenting with voice-enabling our interactive care plans in early 2017, and dug deeper into the topic, prototyping voice powered devices and testing with real people as part of our winning entry in the Alexa Diabetes Challenge. I’ll be talking more about this at the Voice Summit July 24-26, 2018 in Newark.

However, voice experiences in healthcare are not new. This week the Seattle Design for Healthcare meetup Ilana Shalowitz, Voice Design UI Manager, from EMMI Systems (part of Wolters-Kluwer) talking about best practices for voice design based her work on their interactive voice response system. This system effectively does outreach through “robocalls” to help influence people’s behavior, like getting them to schedule general health primary care visits, or get a flu shot. The pathways are designed to guide the patient through specific material, ensuring a basic understanding of the topic, and moving to take action (although not actually taking action), since that was not possible in the interface.

While they have been effective at changing patient behavior, the talk got me thinking about the differences between the interaction model for more traditional, non-AI based interactive voice response and the voice assistants like Alexa and Okay Google popping up in the home, the challenges of each, and the opportunities in healthcare.

Interactive voice response (IVR) can provide a structured pathway, which could be akin to an intake form or an interview. However, it doesn’t allow for an end-user driven experience. In her session, Shalowitz talked about designing a path to give the end user the illusion of control, where a yes or no answer to a knowledge question actually ended up in the same place. Compare that to the home voice experiences where the end user can drive any experience. The upside of this experience is that the end-user is in control, which is often not the case in healthcare, and can drive the direction of the conversation.

Here’s a common experience interacting with a Wellpepper care plan.”

Person: “Alexa, tell Wellpepper I have pain.”
Alexa: “Okay, what is your pain on scale of 0-10 where 0 is no pain, and 10 is the worst pain imaginable.”
Person: “Four”
Alexa: “Okay, I’ve recorded your pain as 4 out of ten. Is that correct?”
Person: “Yes.”
Alexa: “Anything else?”

The difference between this and a typical IVR communication is that the end-user is the initiator. However, the drawback with this type of scenario is that the end-user needs to know what they want to do. This is a notorious problem with headless interfaces like voice. In fact, each week, I get an email from the Alexa team that tells me what new thing I can do with Alexa, essentially a print-guide for the voice interface. Discoverability, context, and capabilities remain problems with these interactions even while they put the end-user at the center.

However, the benefits of these new consumer tools is that, they are designed to not anticipate each pathway in advance, and rather than the pre-recorded prompts of traditional IVR, they are learning systems where continual improvement can be made by examining successful and failed intents. We saw this is in our testing when a patient told Alexa he was “ready when you are.”

I’m excited to be heading to the Voice Summit this coming week, where we’ll talk about what we learned in the Alexa Diabetes challenge, and how we’re applying voice to all our patient experiences at Wellpepper. It’s still early days, but we see a lot of promise, and patients love it.

“Voice gives the feeling someone cares. Nudges you in the right direction.”
Test patient with Type 2 diabetes

Posted in: Healthcare Disruption, patient engagement, Voice

Leave a Comment (0) →

Digital Transformation in Pharma: Digital Pharma West

Like the rest of the healthcare industry, the pharma industry is also grappling with lots of data, disconnects from end-users, and shifting to a digital-first experience while grappling with ongoing regulatory and privacy challenges. Actually it’s pretty much what every industry is grappling with, so the good news is that no one is getting left behind in this digital revolution.

In pharma though, the division between commercial and R&D creates both delays and lags in implementing new technology and the regulatory challenges cause specific issues in communication with both providers and patients.

Last week, I was invited to speak at Digital Pharma West about our work in voice-enabling care plans for people with Type 2 diabetes, and also how our participation in the Alexa Diabetes Challenge enabled us to engage with pharma. It was my first ‘pharma-only’ conference, so it was interesting to contrast with the provider and healthcare IT world.

If you think that there are a lot of constituents who care about digital health in provider organizations, pharma rivals that. For example, there was a discussion about the value of patient-facing digital tools in clinical trials. While everyone agreed there could be real value in both efficiencies of collecting data, and engaging patients and keeping them enrolled in trials, a couple of real barriers came up.

First the question of the impact of the digital tools on the trial. Would they create an intended impact on the outcomes, for example a placebo effect? Depending on how the “usual care condition” is delivered in a control group, it might not even be possible to use digital tools in both cohorts, which could definitely impact outcomes.

Another challenge with digital technology in randomized control trials is that technology and interfaces can change much faster than drug clinical trials. Considering that elapsed time between Phase 1 and Phase 3 trials can be years, also consider that the technology that accompanies the drug could change dramatically during that period. Even technology companies that are not “moving fast and breaking things” may do hundreds of updates in that period.

Another challenge is that technology may advance or come on the market after the initial IRB is approved, and while the technology may be a perfect fit for the study, principle investigators are hesitant to mess with study design after IRB approval.

Interestingly, while in the patient-provider world the number of channels of communication are increasing significantly with mobile, texting, web, and voice options, the number of touch points in pharma is decreasing. Pharma’s touchpoints with providers are decreasing 10% per year. While some may say that this is good due to past overreach, it does make it difficult to reach one of their constituents.

At the same time, regulations on approved content for both providers and patients means that when content has had regulatory approval, like what you might find in brochures, on websites, and in commercials, the easiest thing to do is reuse this content. However, new delivery channels like chatbots and voice don’t lend themselves well to static marketing or information content. The costs of developing new experiences may be high but the costs of delivering content that is not context or end-user aware can be even higher.

At the same time, these real-time interactive experiences create new risks and responsibilities for adverse event reporting for organizations. Interestingly, as we talk with pharma companies about delivering interactive content through the new Wellpepper Marketplace, these concerns surface, and yet at the same time, when we ask the difference between a patient calling a 1-800 line with a problem and texting with a problem there doesn’t seem to be a difference. The only possible difference is a potential increase in adverse event reporting due to ease of reporting, which could cause problems in the short term, but in the long term seems both inevitable and like a win. Many of the discussions and sessions at the conference were about social media listening programs for both patient and provider feedback, so there is definitely a desire to get and make sense of more information.

Like everyone in healthcare, digital pharma also seems to be at an inflection point, and creativity thinking about audiences, channels, and how to meet people where they are and when you need them is key.

Posted in: Adherence, Clinical Research, Data Protection, Health Regulations, Healthcare Disruption, Healthcare Policy, Healthcare Research, Healthcare Social Media, Healthcare Technology, HIPAA, M-health, Outcomes, pharma, Voice

Leave a Comment (0) →

The Challenge of Challenges: Determining When To Participate

There’s an explosion of innovation in healthcare and with that comes a plethora of incubators, accelerators, pitches, challenges, prizes, awards, and competitions. Trying to sort through which ones are worth paying attention to can be a full-time job. At Wellpepper we’ve tried to be selective about which ones we enter. A recent post by Sara Holoubeck, CEO and founder of Luminary Labs about the outcomes of challenges got me thinking about the cost/benefit analysis of entering challenges. Both costs and benefits come in hard and soft varieties.

If you want to be scientific, you can assign a score to each of the costs and the benefits, and use it to decide whether to throw your hat in the ring. (For the purposes of this blog post, we’ll use the term “challenge” to refer broadly to all of these opportunities.)

Costs

  • Time: How many hours will your team need to put into this challenge? How much of your team needs to be involved?
  • Focus: Does the focus on this challenge distract your team from core customer or revenue priorities?
  • Financial: Is there an entry fee to participate? What other costs, like travel, may you need to incur to deliver on the challenge?
  • Strategy: Is this challenge aligned with your
  • IP: Do you have to give up intellectual property rights as part of this challenge? Do you have to give away any confidential information that you are not yet ready to share publically?

Benefits

  • Financial: Is there prize money? Does it cover your expected costs? Could you actually profit from entering? If winner receives funding who decides the terms? Is this an organization that would be beneficial to have on your cap table?
  • Focus: Does this challenge provide the team with a forcing function to deliver innovation in an area that is aligned with your overall strategy?
  • Innovation: Does this challenge take your team in stretch direction or enable you to demonstrate a direction on your roadmap that you may otherwise not immediately approach due to market issues?
  • Publicity: Where will the winner be announced? Is there a PR strategy for the entire process or just the winner? Does it help your organization to be aligned with the content or sponsors of this challenge?
  • Introductions: Who will this challenge help you meet that can further your business goals?

It’s up to you to consider the cost/benefit analysis. Both may not have to be high, but when they are the opportunity can be high if you have the ability to put in the effort. You may also consider your chances of winning if it’s defined as a competition, and whether there is any drawback to losing, or if just participating provides enough benefit.

Here are a few examples from our own history that may help illustrate the tradeoffs.

Low cost/medium benefit

We entered a local pitch event for a national organization. The effort to pitch was minimal: we had case studies and examples that fit the thesis directly. The event was nearby and there was no cost to enter. The pitch was short. We won this pitch and got some local awareness and leads. However, when we were offered to go to the national conference and pitch for an even shorter period in a showcase heHIMSS Venture+ Winnersld simultaneously with other conference activities and with no actual competition, we declined as the cost/benefit was not there.

Medium cost/medium benefit

Each year HIMSS has a venture competition at the annual conference. We won this event in 2015, and received PR as well as in-kind benefits at HIMSS conferences including booth space. The effort to prepare was medium: any startup should be prepared for an onstage venture pitch, and the audience was exactly right. As a follow on from this event we’ve been involved in panels showcasing our progress.

High cost/migh benefit

Both the Mayo Clinic ThinkBIG challenge, and the Alexa Diabetes Challenge had a relatively high effort and opportunity cost to participate and high rewards, but both were aligned with directions our company had already embarked on, and both resulted in deeper connections for us with the sponsoring organizations, positive press, validation of our company and solution, and financial support.

In the case of the Mayo Clinic ThinkBIG challenge, we received investment on our convertible note for winning, and the challenge afforded us introductions to important clinical and IT contacts at Mayo Clinic. We were also able to showcase our solution to other potential customers live at the annual Transform event.

Our team put in a tremendous effort on our winning entry for the Alexa Diabetes Challenge but the pay-off was worth it in a number of ways. Certainly the prize money and publicity was welcome, but more importantly, we have created new IP and also come to a whole new understanding of how people can move through their daily lives with technology to support them in managing chronic conditions.

Both of these challenges have afforded us ongoing opportunities for engagement and awareness as a result our participation, and our positive outcomes.

One thing to note, none of these challenges I mention had an entry fee. Sometimes nominal entry fees are used to deter casual entries, but for the most part if a challenge is seeking to fund itself by charging the startups to participate, it’s the wrong model.

While you don’t have to be this explicit when making your decisions about entering a challenge, consideration of the costs and opportunity cost of either participating or not, can help you sort through the ever increasing number of grand challenges.

Posted in: Healthcare Disruption, Healthcare Technology, Healthcare transformation, Uncategorized, Voice

Leave a Comment (0) →

Alexa, Get Well Soon

The unofficial winner of the Super Bowl ad race this year was “Alexa Loses Her Voice”, an ad that shows celebrities subbing for Alexa when she (anthropomorphic being that she is, comes down with a cold). Both USA Today and YouTube are calling it the most watched ad.

Alexa, who won USA TODAY’s 30th Ad Meter?

“Well, um – me.”

Jeff Bezos looks skeptical that his team can replace Alexa as he should be, since their solution of Gordon Ramsay, CardiB, Rebel Wilson, and Anthony Hopkins is both extremely expensive, (Wellppper CTO Mike Van Snellenberg did the math), and breaks the key trust relationship that people have with Alexa.

Voice is a natural interface, and empathy can be quickly established by the types of utterances and engagement. By default, Alexa apologizes when she doesn’t understand something and it feels genuine. Compare that to Gordon Ramsay insulting his poor hapless user—all the guy wants is a bit of help making some comfort food. What he gets is abuse.

Or, the woman who wants Alexa’s help while she’s in her boudoir presumably getting ready for a date with her love. Instead, Anthony Hopkins insinuates that something horrific has happened to her beau possibly involving a pet peacock.

Cardi B insults a young man’s interest in Mars. Let’s hope she has not squashed his spirit of discovery and his desire to ask questions.

Since this is an all-ages blog, we won’t even mention the response Rebel Wilson gives from her bubble bath to the poor gentleman who asked Alexa to set the mood for a party. He and everyone at his party were fully traumatized.

We get it, Alexa is just better at delivering what people are asking for than real people. Especially real people with attitude like these celebrities.

As we found in our research with people with type 2 diabetes, Alexa has a natural ability that these celebrity Alexa impersonators do not. You can see it in this feedback we received from real people trying to manage Type 2 diabetes.

  • “Voice gives the feeling someone cares. Nudges you in the right direction”
  • “Instructions and voice were very calm, and clear, and easy to understand”

Voice is a natural fit to deliver empathy and care. However, since each one of these people is expecting Alexa, and has no visual indicator that anything has changed, the negative experiences will reflect on Alexa and she’ll have to win back their trust.

While the implied message of the ad spot is that Alexa does a better job of delivering on your needs than any of these celebrity experts we’re still feeling a bit traumatized by the abuse they hurled. For the sequel to this commercial, we’d expect to see Jeff firing the team that replaced Alexa with celebrities, and Alexa as a therapist working through the trust issues that her replacements created. She can do it. We believe in her.

Posted in: Behavior Change, Healthcare Disruption, Healthcare Technology, Voice

Leave a Comment (0) →

May You Live In Interesting Times: Wellpepper’s Most Interesting Blog Posts of 2017

Who would have predicted 2017? As soon as the election results were in, we knew there would be trouble for the Affordable Care Act no one could have predicted the path through repeal with no replacement to claw backs in a tax bill that no one has read. It’s been a crazy ride in healthcare and otherwise. As we look ahead to 2018, we’ve found that a good place to start is by looking back at what was popular in 2017.

Looking back over the past year’s top blog posts, we also believe trends that started in 2017, but will even stronger in 2018. These four themes bubbled up to the top in our most-read blog posts of 2017:

Shift to the cloud

We’ve noticed a much wider spread acceptance of cloud technologies in healthcare, and the big cloud platform vendors have definitely taken an interest in the space. Wellpepper CTO Mike Van Snellenberg’s comprehensive primer on using AWS with HIPAA protected data was one of our most read posts. Since he wrote it, even more AWS services have become HIPAA-eligible.

Using AWS with HIPAA-Protected Data – A Practical Primer

Consumerization of healthcare

Consumer expectations for efficient online interactions have been driven by high-deductible plans and an expectation from consumer technology and industries like retail and banking that customer service should be personalized, interactive, and real-time. These two posts about the consumerization of healthcare were among the most popular.

The Disneyfication or Consumerization of Healthcare

Consumerization Is Not A Bad Word

Value of patient-generated data

In 2017 we saw a real acceptance of patient-generated data. Our customers started asking about putting certain data in the EMR, and our analysis of the data we collect showed interesting trends in patient adherence and predictors of readmission. This was reflected in the large readership of these two blog posts focused on the clinical and business value of collecting and analyzing patient-generated data.

In Defense of Patient-Generated Data

Realizing Value In Patient Engagement

Power of voice technology

Voice technology definitely had a moment this year. Okay Google, and Alexa were asked to play music, turn on lights, and more importantly questions about healthcare. As winners of the Alexa Diabetes Challenge, we saw the power of voice firsthand when testing voice with people newly diagnosed with Type 2 diabetes. The emotional connection to voice is stronger than mobile, and it’s such a natural interaction in people-powered healthcare. Our blog posts on the Alexa Diabetes Challenge, and developing a voice solution were definitely in the top 10 most read.

Introducing Sugarpod by Wellpepper, a comprehensive diabetes care plan

Building a Voice Experience for People with Type 2 Diabetes

Ready When You Are: Voice Interfaces for Patient Engagement

Since these themes are still evolving we think 2018 will present a shift from investigation to action, from consideration to deployment and possibly insights. Machine-learning and AI will probably remain high in the hype cycle, and certainly the trends of horizontal and vertical healthcare mergers will continue. We also expect a big move from one of the large technology companies who have all been increasing their focus in healthcare, which in turn will accelerate the shift to a consumer-focus in healthcare.

There’s a saying “may you live in interesting times.” We expect 2018 to be at least as interesting as 2017. Onwards!

Note: There was one additional post that hit the most popular list. Interestingly, it was a post from 2014 on whether SMART or MEANINGFUL goals are better for patients. We’re not sure why it resurfaced, but based on analysis we’ve done of patient-directed goals, we think there’s a third approach.

Posted in: Behavior Change, Healthcare Disruption, Healthcare motivation, Healthcare Research, Healthcare Technology, Healthcare transformation, HIPAA, patient engagement, patient-generated data, Voice

Leave a Comment (0) →

Meet Wellpepper At Connected Health

We’re gearing up for a great week at Connected Health. See Wellpepper, and our Alexa Diabetes Challenge Grand Prize winning entry Sugarpod in Boston next week. Contact sales@wellpepper.com to schedule a demo, drop by Booth 84 in the Innovation Zone.

 

Wednesday October 25
Natural Language Pre-Conference, we’ll be talking about the Alexa Diabetes Challenge, Sugarpod, and voice

Thursday October 26

Voice Technologies In Healthcare Applications

  • Room: Harborview 2/3
  • Session Number:R0240D
  • 2:40 PM – 3:30 PM

U.S. Department of Health and Human Services Town Hall with Bruce Greenstein, Entrepreneur Panel and Q&A (Invite-only)

Friday October 27

The Power of Patient-Generated Data

Exhibition Showcase 11:00 AM – 11:10 AM

 

 

Posted in: patient engagement, Voice

Leave a Comment (0) →

Building a Voice Experience for People with Type 2 Diabetes

Recently we were finalists in the Merck-sponsored Alexa Diabetes Challenge, where we built a voice-powered interactive care plan, complemented by a voice-enabled IOT scale and diabetic foot scanner. The scanner uses the existing routine of weighing-in to scan the person’s feet for foot ulcers, a serious but usually preventable complication of Diabetes.

We blogged about our experience testing the device in clinic here

We blogged about feedback from patients here

This was a fun and productive challenge for our team, so we wanted to share some of our lessons learned from implementing the voice interface. This may be of interest to developers working on similar problems.

Voice Experience Design

As a team, we sat down and brainstormed a long list of things we thought a person with diabetes might want to ask, whether they were interacting with the scale and scanner device, or a standalone Amazon Echo. We tried not to be constrained by things we knew our system could do. This was a long list that we then categorized into intents and prioritized.

  • ~60% of the utterances we knew we’d be able to handle well (“My blood sugar is 85.”, “Send a message to my care team”, “Is it ok to drink soda?”)
  • ~20% of the utterances we couldn’t handle fully, but could reasonably redirect (“How many calories are in 8oz of chicken and a half cup of rice?”)
  • ~20% of the utterances we didn’t think we’d be able to get to, but were interesting for our backlog (“I feel like smoking.”)

After some “wizard of oz” testing of our planned voice interactions, we decided that we needed to support both quick-hit interactions, where a user quickly records their blood sugar or weight for example, and guided interactions where we guide the patient through a few tasks on their care plan. The guided interactions were particularly important for our voice-powered scale and foot scanner so that we could harness an existing habit (weighing oneself) and capture additional information at the same time. This allows the interaction to fit seamlessly into someone’s day.

 

Challenge 1: We wanted to integrate the speech hardware into our scale / foot scanner device using the Alexa Voice Service, rather than using an off-the-shelf Echo device.

The Alexa Voice Service is a client SDK and a set of interface standards for how to build Echo-like capabilities into other hardware products. We decided early on to prototype our device around a Raspberry Pi 3 board to have sufficient processing power to:

  • Handle voice interactions (including wake-word detection)
  • Drive the sensors (camera array, thermal imaging, load sensors)
  • Run an image classifier on the device
  • Drive on-device illumination to assist the imaging devices
  • Securely perform network operations both for device control and for sending images to our cloud service

Raspberry Pi in Sugarpod

The device needed built-in illumination in order to capture usable photos of peoples feet to look for ulcers and abnormalities. Since the device needed built-in illumination to perform imaging, one of our team members came up with the idea of dual-purposing the LED lighting as a speech status indicator. In the same way that the Amazon Echo uses blinking cyan and blue to show status on the LED ring, our entire scale bed could do this.

As we started prototyping with basic audio microphones and speakers, we quickly discovered how important the audio-preprocessing system is in our application. In our testing there were many cases of poor transcriptions or unrecognizable utterances, especially when the user was standing any distance from the device. Our physical chassis designs put the microphone height around 2’ from the ground, which is far from the average user’s mouth, and also in real-world deployments would be in echo-filled bathrooms. Clearly, we needed to use a proper far-field mic array. We considered using a mic array dev kit, which we decided was too expensive and added too much complexity for the challenge. We also spent a couple hours investigating whether we could hack an Echo Dot to use it’s audio hardware.

Eventually we decided that it would make the most sense to stick with an off-the-shelf Echo for our prototype. Thus, in addition to being a foot scanner and connected scale, the device is also the world’s most elegant long-armed Echo Dot holder! It was easy to physically include the Echo Dot into our design. We figured out where the speaker was, and adjusted our 3D models to include sound holes. Since the mic array and cue lights are on top, we made sure that this part of the device remained exposed.

We will be revisiting the voice hardware design as we look at moving the prototype towards commercial viability.

 

Challenge 2: We wanted both quick-hit and guided interactions to use the same handlers for clean code organization but hit some speed bumps enabling intent-to-intent handoff

Our guided workflows are comprised of stacks and queues that hand off between various handlers in our skill, but we found this hard to do when we moved to Alexa Skill Builder. The Alexa Skill Builder (currently in beta) enables skill developers to customize the speech model for each intent and provide better support for common multi-turn interactions like filling slots and verifying intents. This was a big improvement, but also forced us rework some things.

For example, we wanted the same blood sugar handler to run whether you initiated a conversation with “Alexa, tell Sugarpod my blood sugar is 85”, or if the handler was invoked as part of a guided workflow where Alexa asks “You haven’t told me your blood sugar for today. Have you measured it recently?”

We tried a number of ways to have our guided workflow handlers switch intents, but this didn’t seem to be possible to do with the Alexa API. As a workaround we ended up allowing all of our handlers to run in the context of both the quick-hit entry point intent (like BloodSugarTaskIntent) as well as in guided workflows (like RunTasksIntent), and then expanded the guided workflow intent slots to include the union set of all slots needed for any handler that might run in that workflow.

Another challenge was that we wanted to use the standard AMAZON.YesIntent or AMAZON.NoIntent in our skill, however the Alexa Skill Builder does not allow this, presumably because it needs to reserve these intents for slot and intent confirmation. Our workaround for this was to use a fictitious slot (we called ours “ConfirmationSlot”) in basically all of our intents which could be “confirmed”, every time we wanted to ask for Yes/No values. We factored this into a helper library that is used throughout our skill codebase.

if (confirmationSlot.confirmSlotStatus === 'CONFIRMED') {
  confirmationSlot.confirmationStatus = "NONE";
  handler.emitWithState("AMAZON.YesIntent");
  return true;
} else if (confirmationSlot.confirmSlotStatus === 'DENIED') {
  confirmationSlot.confirmationStatus = "NONE";
  handler.emitWithState("AMAZON.NoIntent");
  return true;
}

Challenge 3: The Voice Kit speech recognizer did not always reliably recognize complicated and often-mispronounced pharmaceutical names

One of our intents allows the user to report medication usage, by saying something like “Alexa, tell Sugarpod I took my Metformin.” People are not always able to pronounce drug names clearly, so we wanted to allow for mispronunciation. For the Alexa Diabetes Challenge, we curated a list of the ~200 most common medications associated with diabetes and its frequent co-morbidities, including over-the-counter and prescription medication. These were bound to a custom slot type.

When we tested this, however, we found that Alexa’s speech recognizer sometimes struggled to identify medication from our list. This was especially true when the speaker mispronounced the name of the medication (understandable with an utterance like “Alexa, tell Sugarpod I took my Thiazolidinedion”). We observed this even with reasonably good pronunciation. We particularly liked “Mad foreman” as a transcription when one of us asked about “metformin.” Complex pronunciations are a well-known problem in medical and other specialized vocabularies, so this is a real-world problem that we wanted to invest some time in.

Ideally, we would have been able to take an empirical approach and collect a set of common pronunciations (and mispronunciations) of the medications and train a new model. It may additionally be interesting to use disfluencies such as hesitation and repetition in the recorded utterance as features in a model. However, this is not something that is currently possible using Alexa Skills Kit.

Our fallback was to use some basic algorithmic methods to try to find better matches. We had reasonable success with simple fuzzy matching schemes like Soundex and NYIIS, which gave us a good improvement over the raw Alexa ASR results. We also started to evaluate whether an edit-distance approach would work better (for example, comparing phonetic representations of the search term against the corpus of expected pharmaceutical names using a Levenshtein edit distance), but we eventually decided that a Fuzzy Soundex match was sufficient for the purposes of the challenge (Fuzzy Soundex: David Holmes & M. Catherine McCabe http://ieeexplore.ieee.org/document/1000354/).

Even though our current implementation provides good performance, this remains an area for further investigation, particularly as we continue to work on larger lists of pharmaceuticals associated with other disease or intervention types.

 

Conclusion

This challenge helped us stretch our thinking about the voice experience, and gave us the opportunity to solve some important problems along the way. The work we’ve done is beneficial not just for Diabetes care plans, but also for all of our other care plans too.

While the Alexa voice pipeline is not yet a HIPAA-eligible service, we’re looking forward to being able to use our voice experience with patients as soon as it is!

Posted in: Healthcare Technology, Voice

Leave a Comment (0) →
Google+