Following on from the appointment reminder functionality this is the third part of my Social Robotics and Crohn’s Disease project.
(Did you recognize where “Shall we play a game?” comes from? Here’s the answer)
Inspiration
I wanted to investigate the potential for creating more interactivity and make some use out of my own medical data. When I first read about the CustomObectMarkers I knew they would be perfect for multiple choice style questions.
A medical quiz could be very useful for a newly diagnosed Crohnie or a family member who wanted to learn about their loved one’s condition and treatment. A doctor could create a quiz, the patient’s home robot could play the game with them and send the results back to the doctor. Based on the results there could be more literature provided to the patient to explain their treatment or used as a point for discussion at their next appointment.
It can be very overwhelming for the first while trying to be aware and informed of your treatment on top of dealing with the fact you’ve been diagnosed with a chronic and incurable illness.
I’d had emergency surgery and surgical wounds that had to be cleaned and looked after. I had to take a lot of different medication. Some were biologics. Some weren’t. Some had to be taken first thing in the morning. Some had to be taken before eating, some with food and others after eating. I had to follow a special diet. There were blood tests, MRI results, bone density scans, stool samples, abbreviations, numbers, fractions etc etc. When my disease was active it would often make my head spin and I would rely on my doctor boiling it down to “good” or “bad”.
I’m showing my age but there were times when discussions about my condition felt like this scene from “Good Morning, Vietnam“!
Mouth Ulcers
The Custom Object Markers have an excellent added advantage as an alternative mode of interaction because mouth ulcers can be a serious issue (often referred to as an “extra intestinal manifestation”) for people with Crohn’s Disease. Unfortunately I have a lot of experience with these. In the run up to being hospitalised I had over a dozen ulcers on either side of my tongue and across my gums which made talking, drinking, and eating (and sneezing, holy f*ck!) incredibly painful. There is absolutely no way I would have been able to use a smart speaker like Amazon Alexa or Google Home in the state I was in.
How it All Works
Using similar code from the previous project for medication management it pulls in the last 6 months and the next 6 months of medical related appointments – doctor appointments, blood tests – and combines it with medication and prescription data I store offline in a database table.
The multiple choice questions are then built based on that data. Questions like “When was your last blood test”, “When is your next doctor’s appointment?”, “How much Pentasa do you take on a daily basis?”, “How often should you inject Humira?” etc.
A-B-C, easy as, 1-2-3…
Anki provided a large collection of CustomObectMarkers which are perfect for representing the “A”, “B” and “C”.
Each marker has to be first defined as “CustomObjectType” and then created within Vector’s current instance of the “world”.
Each object can raise several events that the code can subscribe to such as:
- EvtObjectAppeared – object is first visually identified
- EvtObjectObserved – object is visually identified for longer than 0.8s
- EvtObjectDisappeared – previously observed object is no longer visible
At a high level when Vector asks a question and speaks the options it then goes into “observing” mode to handle events when “A”,”B” or “C” are observed by its camera. If it identifies any of the markers the object is compared against the correct answer for the question and runs a “wrong” or “right” animation sequence.
Creating the multiple choice questions
I grouped my medical data into categories to make choice and question generation easier.
Date and Time – e.g. When was the last doctor’s appointment? When is the next blood test?
Dosage – e.g. How much of X? How many of Y? Should you eat before taking Z?
Frequency – e.g. How often should you take X? When is the next injection?
Test Results – white cell count, c-reactive protein, GGT, and so forth. If you’re a Crohnie then I’m sure you’re very familiar with a line of text on your tests results that probably looks like this:
GGT CRP UE BI TP ALB TBIL ALP ALT AST
I don’t quiz myself up on all of these bar the really important ones (or at least the ones my doctor always tells me about :-P) like “CRP” which is “C-reactive protein“. Basically it’s created in your liver and if there’s a lot of it then something’s not right.
With only three choices and one of them having to be correct the remaining two choices are altered so that one is completely wrong (e.g. a prescribed medicine’s weight in kilograms or a blood test date a decade in the future) and the other is slightly wrong (e.g. plus or minus 5 milligrams, a doctor’s appointment time an hour earlier).
Some examples of questions and choices generated by Vector are:
- How often should you inject Humira?
- A: Every week (Slightly wrong)
- B: Every two weeks (Correct)
- C: Twice a year (Completely wrong)
- When was your last blood test?
- A: January 5th 2008 (Entirely wrong)
- B: May 2nd 2019 (Correct)
- C: May 11th 2019 (Slightly wrong)
- Adalimumab is the generic name for?
- A: Chocolate Cake (Do I even need to point this one out? :-P)
- B: Pentasa (Slightly wrong)
- C: Humira (Correct)
In Summary
While this is a prototype I think it shows potential. With richer animations and utterances this approach is more engaging than a purely application based approach. Utterances could be created to fill in time with idle “chit chat” or provide encouragement between questions.
The multi-modal input could also be easily extended to allow for voice input or another device for a User to type or tap buttons for their answers.
For the purposes of demonstration I’ve kept the questions and topics short but they could be widened to include inpatient care, surgery, changes in medication, reasons for changes in treatment and so forth. This raises much larger questions about medical data, storage formats, access and ownership. I know I have generated a tonne of medical / clinical data over the last 13 years but it exists in different hospitals, different GP clinics and countries. It’s something I’m saving for a later blog post though.
If you’d like to get in touch about anything here please use the Contact Form.