Shine
Enjoy singing confidently with tailored guide on your pitch.
Universal Design | User Research | Wireframe | Prototype | Interaction Design | Obj-C Development
Singers with hearing loss may have trouble staying on pitch or determining pitch-correctness. To build singers’ confidence, Shine, as a mobile singing coaching app, transcribes hearing into seeing and feeling, through visualizing the pitch of users’ voice on iPhone with a haptic metronome on Apple Watch.
This project is my Master’s thesis project at MFA Design & Technology in Parsons School of Design. I am into universal design and assistant device for musicians with hearing loss, because I myself am an amateur singer with both-side hearing loss. I think it’d be wonderful to use my experience to benefit others.
Overview
This documentary video introduces my motivation, my hearing loss experience, user interviews and user test results.
Design Question
Hypothesis
How might we design a tool to assist singers with hearing loss to independently help themselves develop sense of musical pitch and sing confidently?
-
Haptic feedback will assist user well while singing.
-
A relative pitch feedback (user’s pitch is flat or sharp than it should be, and by how much) assists user to sing on pitch.
User Research
Target users
Pain points
Singers with hearing loss.
-
Feel unconfident and insecure when singing.
User’s goal
(Re-)acquire the ability to independently
help themselves develop sense of musical pitch, and sing confidently while rehearsal or performing.
-
Have trouble staying on pitch / determining pitch-correctness (pitch differences within some small range may sound the same to them).
-
Rely on assistance from other singers with normal hearing.
-
Acoustic hint assistance alone is little useful to them.
Design Process
I started from vibrotactile interface, and through many user researches and tests, I ended up with an iPhone/Apple Watch app. Here’s an overview of the process:
Research Phase I
Vibrotactile Interface
Design idea 1: Vest
Inspired by Dr. David Eagleman’s TED talk, in this idea I wanted to establish sensory substitution when singing in a chorus, to enable user to adjust their own voice and harmonize with other choral singers.
I designed some wireframes and user flows for this idea.
Design idea 2: Arm sleeve band
To simplify the previous idea and improve wearability, I designed this arm sleeve band. The band transcribed the pitch of user’s own voice and of another singer’s voice, to enable user to compare these two pitches and adjust their own pitch to sing better.
I also designed some wireframes and user flows for this idea.
I made some prototypes for the band, including the vibrotactile interface on single/double arm sleeve bands and on the lower back.
Prototypes
I tested the wearablity and the usability of these prototypes for users to correct their pitch.
Shaded sections showed that user sang more stably when using the prototypes than singing with only the piano reference.
But the vibration was reported to be hard to sense/recognize from time to time, and uncomfortable.
User test
Research
Domain Setting / Brainstorming / Precedents Study / User Interview
Top: without prototype. Bottom: with prototype.
—— piano reference —— user's voice
Based on the user test result, I found out that the vibrotactile interface idea was difficult to be further developed into a product. It’s very demanding to make the product not only useful, but also comfortable, discreet, portable, and easy to manufacture. So I decided to change my course to design an mobile app on iPhone and Apple Watch platform, and to implement the haptic feedback system on Apple Watch.
Research Phase II
Mobile App
Research
Domain Setting / Brainstorming / Competitor Analysis
Design idea 2: Arrow hints
Based on user feedback for the previous idea, I changed the strategy to practicing only a short musical phrase at a time. I designed a keyboard interface of inputting sheet music, arrow hints that indicated inaccuracies, and haptic metronome on Apple Watch.
Prototypes
Base on user feedbacks, I tried to further simplify the inputting UI into user recording, and the arrow hints into pitch visualization history view. So user could check the pitch visualization of their own voice, for comparison with that of a pre-recorded voice reference, and to track their progress. I also designed the UI of Apple Watch haptic metronome.
User test
During the process, I connected to some hearing loss musicians and conducted several user tests to test the usability for users to correct their pitch and build confidence. Please refer to the documentary video.
In this idea, user could know their inaccuracies (pitch differences) by both visual and haptic hints from Apple Watch. I designed wireframes, user flow and prototypes. In user tests, the interface was praised for being straightforward, but real-time hints made user nervous and dampened their performance, and were reported to be useless with a fast-paced song.
Design idea 1:
Visual / haptic hints
UX / Visual iterations and improvement
Through user tests, I improved the user flow by adding the count-in and play-first-note function, and improved the visual design to indicate different scenarios (visual design by designer Soomi Lee).
Final Product for MFADT Thesis Show 2016
Here are the current user flow and some views in the final prototype/product. For demo, please refer to the documentary video.
Reflections
To me, this project is an important step from a developer to become a designer. With a computer science background, I used to think as a problem-centered engineer. After I entered Parsons School of Design, I found that designers think totally differently. In order to become a designer, I started learning to think user-centered. User is human, their need is hard to be clearly defined even if we ask them, and designers need a plethora of skills to be as objectively as possible to observe, and, from such observation, infer user needs. For user centered design, the only problem we should keep in mind is: how might this help user? And everything we design should be tested through user tests, i.e. the only way to prove its value and usability. In this design process, I learned that it’s more difficult to get rid of features that didn’t fit the core functions than adding features according to user’s suggestion.
In this project, even though I am the very first target user, it is easy to lose myself in working on perfecting the pitch detection mechanisms, in order to — pretty much in my imagination — “help” user to achieve pitch-correctness. Obviously, this would not be what users really need. What users need is to feel confident in singing. And they do not need to become a perfect singer all of sudden. So, l learned that I should carefully balance every part of this project to serve this overall goal: confidence building.
This project is also a confidence building process for me. To thrive in the process, I have to double as a UX Designer, UX Researcher, Visual Designer, Software Engineer, Product Manager, Marketing Manager... etc. I really learned a lot through these various aspects of tasks. It was a great opportunity for me to do something unfamiliar and walk out of the comfort zone. I really appreciate this and have actually felt stronger and smarter.