Through research, Zihao Zhan and his colleagues detect and recommend fixes for security issues in newly released technology like the Apple Vision Pro headset.

When Apple announced it would release a headset combining augmented reality (AR) and virtual reality (VR) at its 2023 Worldwide Developers Conference, potential users worldwide highly anticipated its release. 

Among those captivated by such a profound piece of technology was Zihao Zhan, an assistant professor in the Department of Computer Science at Texas Tech University. However, instead of dreaming about the potential of the Apple Vision Pro in the fields of education, professional training and social interactions, he was more speculative about its risks. 

He has found the more capabilities are pushed to improve computer performance, energy efficiency and user experience, the more potential security problems are introduced.

“Our team is dedicated to try and make the latest technologies more secure as they release,” Zhan explained of the research he conducts with five colleagues he met during his post-doctoral research at University of Florida from 2021 to 2024. “As security researchers, we want to see whether their implementations have any potential problems where they have overlooked any critical security factors.”

The opportunity to better cutting-edge innovations is captivating to Zhan, which is why he decided to continue related research at Texas Tech when he joined the Edward E. Whitacre Jr. College of Engineering faculty earlier this year. He had already explored the university as a visiting student a few years ago and felt eager about the possibilities that would follow his return to campus. 

“The Whitacre College has offered me tremendous support,” he reflected, “from startup fundings to supporting my students.”

Zhan and the researchers purchased an Apple Vision Pro once it debuted in February and began to investigate. He admits they had to dig deep, but through their diligence they did uncover a concerning issue. 

To understand what their attacks (which they refer to as GAZEploit) revealed, one first must dive into new capabilities developed by Apple such as Persona – technology that scans the user’s space and facial expressions and allows interaction in a virtual world through an avatar. The second is known as gaze typing – where the user can operate a virtual keyboard with their eyes by staring at which key they want to type and confirming the selection by a hand gesture. 

While Apple intended to keep this eye-tracking data secret by not allowing users, developers or even the company itself to access it, Zhan and his colleagues discovered that if an attacker had access to a Persona view, they could track the eye movements of an avatar to extract information such as passwords and confidential messages through several intermediate steps. 

They found many scenarios in which a GAZEploit attack could take place, such as on video calls, online meeting apps, live streaming platforms or potentially malicious websites. Even in a Zoom or Microsoft Teams meeting, Zhan said if a user was to type, the key sequences could be remotely captured, analyzed and decoded. Furthermore, attackers could send a link, ask the user to log in with their credentials and then gain access to that information. 

To test their theory, the team analyzed 30 participants using the Apple Vision Pro and achieved over 80% accuracy in keystroke inference. Alarmingly, their study also identified over 15 top-rated apps in the Apple Store that were vulnerable to the GAZEploit attack, emphasizing the urgent need for bolstered security measures.

Once their findings were concrete, Zhan and his colleagues contacted Apple. They were delighted to be assigned a CVE, which is short for Common Vulnerabilities and Exposures – a list of publicly disclosed computer security flaws. The team then collaborated with Apple to resolve the issue by disabling Persona view while users are typing. 

“When we received that response from Apple, that meant we found a critical problem that existed in the system,” Zhan said. “Our findings significantly improved the security state of an existing device, and we are very proud of our meaningful contributions to the community.”

The team published their work in the Association for Computing Machinery Conference on Computer and Communications Security work following Apple’s fix in September, and was interviewed for an article featured on WIRED, a news company that has multimillions of followers on its social media accounts. 

But Zhan said while this is the most recognition their work has received, it is certainly not the first time their research has made headlines. 

Prior Work  

The Apple Vision Pro was considered high-level software security research, but Zhan said most of their work has featured lower levels of system security – such as devices with touchscreens. 

In 2022, they published a paper titled “The Invisible Finger” in which the researchers revealed intentional electromagnetic interference can operate touch screens without any contact – also known as ghost touches. Through different magnitudes, frequencies, duration and multitouch patterns, the researchers were able to perform attacks on smartphones and tablets that leveraged Siri, installed malicious apps on Android devices, and sent messages or money. 

While the thickness of an average tabletop the device was placed on could not prevent this attack, the researchers recommended users purchase cases with metal front covers to block all electromagnetic interferences that could target smart devices. 

These findings shared in “The Invisible Finger” paper received the Distinguished Paper Award in the 2022 Institute of Electrical and Electronics Engineers Symposium on Security and Privacy. 

“I find this part of system security really interesting,” Zhan said. “It is pretty interdisciplinary, so there are many things to explore.”

Their next paper was published earlier this year, titled “VoltSchemer,” which was included in the proceedings of the 33rd Advanced Computing Systems Association (USENIX) Security Symposium hosted this past August in Philadelphia. 

For this project, Zhan said they were able to use voltage noise and attack nine top-selling wireless chargers for portable electronic products. Their tests revealed the wireless charging systems were vulnerable and susceptible to intentional electromagnetic interference. 

Through manipulation that occurs by modulating the voltage from the power supply, the researchers could control voice assistants and damage devices being charged through overcharging or overheating. They even bypassed foreign object detection mechanisms and tricked the wireless chargers to transfer power and damage nearby items exposed to the intense magnetic fields, such as a car key fob, a clipped document and a USB drive. These items reached such high temperatures (up to 536 degrees Fahrenheit) that they sometimes ignited, melted or left behind burn marks. 

Afterward, Zhan and his colleagues made sure to contact vendors to report the identified vulnerabilities and discuss countermeasures that could be employed. Those solutions will be disclosed in the future. 

What’s Next 

Zhan is proud to share his students are involved in the researchers’ most recent projects of exploring security challenges in the latest advancements in computer systems. These include new processors and specialized hardware for emerging technologies, such as artificial intelligence developed by leading industry companies like Apple, Advanced Micro Devices, Inc., Intel and Nvidia. 

“In this case, we believe that as these industry leaders focus on significantly improving performance and energy efficiency in their cutting-edge products, they may have inadvertently overlooked some security factors, potentially introducing vulnerabilities,” Zhan said. “That’s what we are focusing on right now.” 

While Zhan’s team is in the initial stages of this project and does not have final results yet, he said they have already made some discoveries to investigate further. The feedback he has received so far confirms the students are gaining valuable experience and knowledge in a field that is relatively new, since the products have just been released. 

“They learned more things about the underlying architecture of the computer systems, and at the same time, they also are learning new skills – like system-level programming techniques,” he shared. “They’ve told me they are constantly learning, which makes the work very interesting.”         

The processes these students are involved in have challenged them both on academic and engineering fronts – the same pull that keeps Zhan committed to further research he believes is critical to the betterment of computer systems. 

“As we know nowadays, computers are more and more integrated into our daily lives,” he noted. “This is why we definitely need more effort in the dimension of security.”