Is This a Dream or Is It Real?

A place comprised of the perfect mesh of the “real world” and a virtual reality created by computer programmers is currently under construction.

Dr. Shulan Lu, associate professor of psychology, and Dr. Derek Harter, associate professor of computer science and information systems, have been working diligently with several student researchers with the hopes of creating a space where people are not sure whether or not they are in real life or a virtual reality. They've entitled their work “Perceiving and Enacting Actions in Virtual Environments.”

Former Ph.D. students, Dr. Terry Rawlinson and Dr. Paweena Kosito; graduate students, Pratyush Kotturu, Gang Wu, Sandeep Kotha, and Manoj Indupuru; and undergraduate students, Sarah Wang, Rachel Bailey and Heather Graham have all had the unique experience of learning from and working intently with Drs. Lu and Harter.

Going into its fifth year, the project has outlived one grant from the National Science Foundation (NSF) and is now funded by its second one. With some $570,000 in grant money and a little support from the U.S. Department of Energy, the team shows no signs of slowing down.

The goal of this research is to discover whether gesture alone is enough to create a real action and effect as well as triggering the risk avoidance contingency. In layman's terms, they would like to know whether a person participating in a virtual reality situation would think, act and react as if it were real life. If a motion is made to cut your arm, would you think it's real? Would you pull away or let it “cut” you? For anyone who has seen the movie Avatar, it sounds just like Pandora. If people spent enough time living as an avatar in a different world, it would seem as though it would be hard to tell the difference between what is real and what is not after a period of time.

While it all sounds kind of magical, the mechanics of it are entirely plausible. Most people working on the project are electronic engineers and computer scientists who work on the hardware and building of the “other world” with Harter. The other students work with Lu on the cognitive psychology of it all. They're constantly looking for ways to get a person to be more immersed in a virtual world.

The project may seem trivial at first, but the goals of the research are to make a better world. The findings from these virtual tests will work to revolutionize a multitude of professional fields. Doctors can embody humanoid robots to make surgical tasks much more precise and accurate. One experiment gives the participant an especially long arm to see how he or she adjusts to understanding the reach he or she would normally have with a regular arm versus the long arm. This helps understand a person's mindset and decision-making when taken out of the real world. Robotic arms can also be useful to deep-sea oil drilling. By using robots to do the major part of the job, it can take people out of high-risk situations. They are also working with the University of Memphis, University of North Carolina-Chapel Hill and Stanford University on possible crisis training for terrorist attacks for law enforcement and civilians. One can never be too prepared, and the ability to react to stressful, high-intensity situations could result in lives being saved and tragedies avoided. The results from these tests will help to build more programs and situations that benefit larger audiences like training surgeons, piloting aircrafts and even psychotherapy.

“Today, many professionals are being trained through virtual environments to do everything from flying airplanes to performing the medical procedure of intubation,” said Rawlinson. “The scientific community needs to ensure that humans are engaging such systems in meaningful ways that result in competency, just as in the real life. Ensuring that we get it right is what intrigued me the most about being able to help with this research.”

A large portion of the work investigates how taking on an avatar's body might change the perception of our own bodies. It's a collaborative work with Dr. Roberta Klatsky from Carnegie Mellon University who serves as a professor of psychology and human computer interaction as well as a faculty member of their Center for Neural Basis of Cognition. This cross-disciplinary and cross-institutional effort helps students and faculty alike to network within their field.

“The working environment that Dr. Lu and Dr. Harter provided was challenging and interesting,” said Wang. “Even though psychology is not my major, and I am just an undergraduate student, they took my suggestions and opinions seriously. What I have gained from this student worker position helped my personal growth, and I learned to take my responsibilities seriously.”

Harter says he started in artificial intelligence, but he was not satisfied because early and traditional artificial intelligence was not as concerned with cognition, or how humans and animals are actually intelligent. This principle is important because it helps us to understand how our brains work and what last minute decisions people will make when they are put in do or die situations. The most recent evidence of testing showed that a person making a gesture alone without the physical object for performing the actions was not sufficient enough to illicit a response. For example, you're in the virtual space. Dr. Lu has told you that you're playing a baseball game. It's the bottom of the ninth, two outs, runners in scoring position, and you're up to bat with a full count. If you don't have a bat in your hand and there's no evidence of the scenario, your brain understands that it's not real. Nothing will happen if you strike out. You know it's all pretend. However, if you are given a bat and a pitcher is placed in front of you with a stadium full of screaming fans, you begin to believe it's real. The environment will trigger the feelings associated with a high pressure situation, and you'll perform the way you would in reality.

Perhaps the most exciting part of the entire project taking place on the A&M-Commerce campus is the student interaction. A few college kids who came to Commerce hoping to get their degrees and move on have found that working on this project has been life-changing.

“I feel so lucky to have met Dr. Lu and Dr. Harter and to have been given the opportunity to work with them,” said Wu, a Computer Science student who worked on the project and graduated in May. “Through their depth of knowledge, enthusiasm and life experiences, their advice and guidance have been a positive influence in both my academic as well as my professional career. They have been advisors, professors and personal friends.”

By facilitating this kind of opportunity, our university is better equipping its students for their professional lives because they are given a chance to take the knowledge they've learned from books and lectures and apply it outside their classrooms.

Student employees are given access to software they will use in their respective industries as well as learning and experiencing other fields of research. They can learn about the newest technologies and get paid at the same time. The faculty offers a high-level of expertise, and because of the size of this university, students are given closer attention and more interaction.

Through team building activities and the way they are cultivating lives these professors show that they really care about their students and do not view them as worker bees. When a person feels appreciated for the work they do and for the person they are, it really makes for productive, creative and innovative professionals. Promoting these types of relationships creates a strong bond within the research team, and thus furthers the project immeasurably.

When asked why it is important for this research to take place here at this university, Lu answered, “The honest answer is because we're here. Dallas is a technology corridor that is quickly growing, and that affords students the opportunity for feedback from the community and companies as well as possible collaborations with these entities.”

Their work with virtual environments still has some run time left. The NSF grant will end in August 2014, and after that, the professors and student researchers are looking to the U.S. Department of Defense and Homeland Security for options. While we can never have a 100% virtual world, this research works to dig deeper and push harder into the vast unknown. There must be some overlap and abstractness, but if the reality they are creating can ultimately change the way we think, feel and behave it's impossible to tell where that will take us. Projects such as these will lead our university and our world into the future with open arms and minds to take us to places we most certainly have never been before.

Listen to our conversation with Drs. Lu and Harter and Pratyush here. And continue your fearless investigation of the incredible research happening at A&M-Commerce at https://www.tamuc.edu/alumni/.