LSU Computer Science Professor Taking a ‘Hands-On’ Approach to Smartphone Security

BATON ROUGE, La., March 10, 2022 (GLOBE NEWSWIRE) — As smartphones have grown more sophisticated over the years, so too have their accompanying security measures. Simple passwords have been replaced by thumbprints and facial recognition. However, those methods do not solve the issue of notification privacy.

For example, sharing your phone with a friend, family member, or another individual—or even leaving it briefly on a nearby surface—could expose your privacy in the form of an incoming call, email, reminder, or app notification. Existing iOS Guided Access and Android multi-account features have been tested to solve this problem but have been unsuccessful.

LSU Computer Science Assistant Professor Chen Wang believes he may have the answer. Specifically, he is working with third-year PhD student Long Huang on a gripping-hand verification method that ensures the correct user is holding the smartphone before displaying potentially sensitive content. Their recent paper on this topic was published at Mobicom 2021, the annual international conference on mobile computing and networking. A short demo can be viewed here.

When a notification tone is played, the phone’s mic records the sound. An AI-based algorithm processes the sound and extracts biometric features to match with the user’s feature profile, or recorded hand grip. If there is a match, the verification is successful, and the notification preview is displayed on the screen. Otherwise, only the number of notifications pending is shown.

“We consider this an attempt for security design to embrace art,” said Wang, whose expertise is in cybersecurity and privacy, mobile sensing and computing, wireless communications, among other areas. “We find that when playing music with a phone, our holding hands often feel the beats, which are caused by the phone surface vibrations. This is a way in which the music sound conveys information to us. Because music sounds are signals, they can be absorbed/dampened, reflected, or refracted by our hands.

“We then use the phone’s own mic to capture the remaining sounds to see how we respond to music. Because people have different hand sizes, finger lengths, holding strengths, and hand shapes, the impacts on sounds are different and can be learned and distinguished by AI. Along this way, we develop a system to use the notification tones to verify the gripping hand for notification privacy protection. This is very different from prior acoustic sensing works, which all rely on dedicated sounds, inaudible or annoying to human ears.”

The project is one of two supported by the Louisiana Board of Regents that Chen is working on involving smartphones and users’ hands. The other—in collaboration with second-year PhD student Ruxin Wang and computer science master’s graduate Kailyn Maiden—uses the back of the user’s phone-gripping hand for verification at kiosks, such as those used to order food, print tickets, and self-checkout at the grocery store. This research will be published as a late-breaking work at the 2022 ACM CHI Conference on Human Factors in Computing Systems. A short demo can be viewed here.

“When a user holds [his or her] phone close to the kiosk for NFC-based or QR-code authentication, the back of the user’s gripping hand is captured by a camera on the kiosk,” Wang said. “An AI-based method will process the gripping-hand image and compare it against the user’s registered hand image by checking the gripping-hand’s shape, skin patterns/color, and gripping gesture. Notice here, the user’s identity has been claimed by the traditional NFC or QR-code methods as they transmit the user’s security token. Thus, here we provide a two-factor authentication to the kiosk—the security token and the gripping-hand geometry biometrics.”

Wang added that he and the students are improving the authentication systems and conducting user studies with more participants and device methods. They are also examining the impact factors on the practical use of these systems, including the ambient noise and light conditions. Additionally, they are investigating potential attacks, e.g., a 3D-printed silicon fake hand and acoustic replay attacks.

Wang plans to commercialize these techniques in three years.

Like us on Facebook (@lsuengineering) or follow us on Twitter and Instagram (@lsuengineering).​

• Kiosk System Demo
• Handgripping Demo