[BCM325 Digital Artefact] Immersive Storytelling with VR, the Ethics, and the Philosophy

Self-Made 360° Illustration Break-Down—Findings & Discussion—Reflection on the Research Process

VR, with its power of allowing viewers to be active agents (Dooley 2017, p. 161), is reshaping storytelling in unprecedented ways. All scholarly sources point to two keywords: IMMERSION and AGENCY—immersion allows the user to interact with the virtual worlds in the stories with almost all senses (minus smell and taste) (Meinhold 2013), as “either witness or protagonist” (Dooley 2017, p. 165), while agency is given to he/she to deliberately navigate a non-linear storyline (Jones 2017, pp. 177-178) while interacting with these worlds (Dooley 2017, p. 161; Chok 2017; Koski 2015, p. 9).

This illustration (the making of which has been documented well below) can be unpacked based on my findings from my secondary research on VR in storytelling and immersion/engrossment.

The open book in the above illustration suggests the power that VR technology brings to storytelling—the immersion/engrossment state that occurs when one steps into that beckoning open world waiting to be explored. In my previous blog posts (here and here) and my presentation, I have explored how immersive storytelling with VR is used to help students learn history (Project MuraVagando, Carrozzino alal. 2016, pp. 304-306), and assisting deaf and hard-of-hearing students in improving their storytelling ability better than traditional pictorial intervention (Eden 2014, p. 370). Taking it further, Wildcard, a wearable VR storytelling tool with the Google Cardboard has been implemented at a facility for children with intellectual developmental disability and has been proven to “partially [boost]” the children’s long-term memory (Gelsomini et al. 2016, p. 5191).

The connected white dots in the scene in the book represent a non-linear storyline, which is often employed in immersive VR products—lots of possible sequences of events leading to multiple corresponding outcomes.

Ever since VR came into journalism, huge steps have been taken to introduce news audiences to elaborately crafted pieces that “stand up to journalistic rigor” (Kane 2014, p. 32). According to de la Pena, one of the creators of Project Syria, there has been a remarkable observation: the audience of VR journalism “respond realistically” to situations which they fully know to be virtual, (2010, p. 293). This is partially due to viewers’ extreme focus brought about by complete deprivation of distractions (Jones 2017, p. 181; Shin & Biocca 2017, p. 16) and a sense of “being-there” (Sundar et al. 2017, p. 674).

Based on the advice of my tutor/lecturer, for my final Digital Artefact, I have dug deeper into immersion and engrossment in relation to VR and its ethical/philosophical implications. Immersion, “a metaphorical term derived from the physical experience of being submerged in water” (Murray 1997, cited in Nilsson et al. 2016, p. 110), is generally agreed to refer to the idea of being fully surrounded by something—water, or an environment. However, in the field of VR, the most critical meaning of the term should involve four levels of the state of immersion:

“(a) immersion as a property of the system used to present the virtual world; (b) immersion as a perceptual response to that system; (c) immersion as a response to an unfolding narrative, the characters inhabiting the story world, or the depiction of the world itself; and (d) immersion as a response to challenges demanding the use of one’s intellect or sensorimotor skills” (Nilsson et al. 2016, p. 110).

Engrossment is, then, a by-product of immersion (Teng 2010, cited in Shin 2017, p. 7), occurring almost simultaneously with immersion. 

Yet as pondered over by Radford and Weston (1975, pp. 67-69), what we think to be genuine emotional responses for a character in a story (be it fiction or non-fiction) are likely mere make-believe, no matter how “moved” we are by an account. In his essay, How Can We Be Moved by the Fate of Anna Karenina, Radford wrote:

“But now suppose you discover that the account is false. If the account had caused you to grieve, you could not continue to grieve. If as the account sank in, you were told and believed that it was false this would make tears impossible, unless they were tears of rage. If you learned later that the account was false, you would feel that in being moved to tears you had been fooled, duped” (p. 68)

This leads to the questions of whether VR experiences are real and whether immersion brings real feelings and empathy. Currie (1995) argued that we can truly, genuinely, feel for what is virtual/unreal, via what he called “secondary-imagining”. There is primary-imagination, which is imagining what a character would behave in a certain situation, under certain circumstances, and then there is secondary-imagination, which is to picture, vividly, what one would feel if he/she were the character.

“Take, for example, an element of a novel, television show or movie that brings us to tears or causes us to laughter. These are a fictional, mediated, virtual universes that are having a physical effect on the body.” (Moore n.d.)

The way VR allows users to experience what is physically “not there” hints towards transhumanism, which is alluded to in the illustration by the wide eyes—with VR headsets for the pupils—staring at the story book.

By “recognizing and anticipating the radical alterations in the nature and possibilities of our lives resulting from various sciences and technologies” (More 1990, cited in Moore n.d.), transhumanism as a class of philosophies aids human beings in navigating the ethics of artificial life, an apparent product of VR.

Many researchers seemingly think along the same line as Currie. Emotional engagement, a major part of immersive journalism, encourages viewers to empathise with the virtual people’s pain as if it were their own, countering emotional distancing in traditional journalism (Sánchez Laws 2017, p. 10); it is further enhanced by the state of flow brought by immersion (Shin 2018, p. 70) (Flow is a state of extreme focus and immersion in an experience/activity, a term coined by Hungarian psychologist Csikszentmihalyi). Similarly, Sherry Turkle stated—In her study on virtual gaming worlds’ positive influence on young university students’ life in the real world—that despite not being real, the virtual world has a relationship to the real (1994, pp. 165-166); it is an alternate space for its users to ponder the real world. This aligns completely with the “highest” level of immersion as defined by Nilsson et al. above.

From these findings, I would argue that, as transhumanism preserves a valuing of human and human qualities, the myriad of virtual worlds can and should raise real, genuine emotional and ethical response.

**

Reflecting on my research progress, I realise that proper, in-depth secondary research into VR and immersive storytelling has effectively deepened my knowledge beyond that of technology: I have learnt to draw from concepts in other subjects to study the ethics and philosophy of immersive storytelling in relation to cybernetics.

However, I have also noted that some primary research could have added more originality to my Digital Artefact, besides the self-drawn illustration and the screen records. For instance, several interviews with members of the MakerSpace at UOW (who have been experimenting extensively with VR technology) could start informed debates about matters like ethics, or a survey aiming at university students could explore how my age group perceive and experience the technology. Seeing how my framing of the topic has been changed—narrowed down and redirected—throughout the process, I should hone my skills in forming a good research question, one that is not too vast to answer yet original and critical. Another improvement to make is time-management, since I admittedly procrastinated and did not allow myself sufficient time to experiment more with 360° illustration production and to examine more exhaustively the resources I have gathered.

Cost- & Time-Related Difficulties—Reflection on 360° Illustration Making Process

According to Robertson (n.d.) on The Verge, the highest-end headsets would cost at least $500; these offer much higher-quality experiences, however, there would be roughly $1000 of hidden cost to invest in an adequately powerful computer setup. On the other hand, although cheap models come with virtually no extra costs since most adults and even younger children/teenagers now own smartphones capable of running simple, light-weight VR materials, the quality of the experience delivered is greatly compromised.

Furthermore, (as mentioned previously) despite the availability of more affordable viewing devices and the introduction of cheaper models of 360° camera, VR journalism projects are time-consuming and costly to produce, thus cannot yet be constantly churned out (Sullivan 2018). (Aronson-Rath et al. (2015, p. 58) reported on the major additional cost (equalling a starling 25 per cent of the price for one in traditional format) to produce a VR version of a Frontline episode.)

**

(Click HD on the lower right corner for better quality)

As I am fully aware that I am a complete newbie to VR in general and 360-illustration making, I do not wish to represent, through the above video, the real professional process of creating immersive stories using VR technology in any ways. I set out aiming to just dip my toes into the uncharted waters, and I came up against stimulating challenges, most of which technical.

Leonardo, the application I used to make illustrations, works seamlessly with either a tablet and stylus pen or a Wacom connected to a non-touchscreen computer. Although the authors of the guides I consulted (Joshi 2016 & Lai 2016) drew in Photoshop instead of another software, Leonardo feels more intuitive for me, since the interface is streamlined to solely serve digital painting instead of including photo-editing options seen in Photoshop. I imported the .psd files from Leonardo into Photoshop and made required adjustments, before exporting and imbedding the image on this page (based on this guide). (I might have “cheated” a little to make the top and bottom of the 360-image look smoother using solid colours while drawing on the grid.)

This painstaking process, with multiple other trials and errors not included in the video to avoid lengthiness, has enlightened me on how exactly time-consuming and taxing the production of immersive 360-illustrations and VR materials in general can be, since it entails not only artistic skills and aesthetics, but an immense technical skillset as well. One 360-image is already extremely complicated to perfect and achieve the desired immersion, let alone strings and strings of them to form a complete short film like Pearl (introduced in more details in my previous blog post).

Conclusion

VR is definitely “reshaping storytelling for the better” (Chok 2017) with its evident contribution to immersion and engrossment in storytelling experiences. However, there are critical issues to bear in mind during production and consumption, in terms of this immersion and engrossment, since these concepts are defined rather personally according to individuals’ standards and expectations (Shin 2017, p. 110). Some of the potential harms can be seen when users experience difficulties re-entering the real world (Behr et al. 2005, pp. 670-671) child development, where VR is found to potentially impact children’s prefrontal brain structures, and engrossment sometimes makes them lose track of the real world (Southgate et al. 2017, pp. 14-15).

Through independent, secondary research, I have sharpened my researching and technical skills, and have become more aware of how whether a technology brings a dystopian/utopian future largely depends on its users—us very human beings who are heading towards a near post-human future.

Posts leading up to this Digital Artefact

The Research Proposal, in which comprehensive findings were presented. The focus was on VR in general, with some closer look into storytelling.

The Progress Report, in which the focus was narrowed down to the application of VR in storytelling, including in journalism. Some case studies were briefly looked into, and a direction towards ethics and philosophy branched out, yet superficially examined.

A Recap of my live-tweeting throughout the first eight seminars of the subject.

Mia (Minh-Anh)

 

---

References:

1. Aronson-Rath, R, Milward, J, Owen, T & Pitt, F 2015, ‘Virtual Reality Journalism’, Tow Centre for Digital Journalism, viewed 28 April 2018, < https://legacy.gitbook.com/book/towcenter/virtual-reality-journalism/details >.
2. Behr, K, Nosper, A, Klimmt, C, & Hartmann, T 2005, ‘Some Practical Considerations of Ethical Issues in VR Research’, Presence: Teleoperators & Virtual Environments, vol. 14, no. 6, pp. 668-676.
3. Carrozzino, M, Evangelista, C, & Galdieri, R 2016, ‘Building a 3D Interactive Walkthrough in a Digital Storytelling Classroom Experience’, Informatica (03505596), vol. 40, no. 3, pp. 303-309.
4. Chok, L 2017, ‘Virtual Reality Is Reshaping Story-Telling For The Better’, Forbes, 12 June, viewed 14 April 2018, <https://www.forbes.com/sites/lionelchok/2017/06/12/virtual-reality-is-reshaping-story-telling-for-the-better/#5302762a5dd4 >.
5. Currie, G 1995, ‘The moral psychology of fiction’, Australasian Journal of Philosophy, vol. 73, no. 2, pp. 250-259.
6. Dooley, K 2017, ‘Storytelling with virtual reality in 360-degrees: a new screen grammar’, Studies in Australasian Cinema, vol. 11, no. 3, p. 161-171.
7. de la Pena, N, Weil, P, Llobera, J, Giannopoulos, E, Pomes, A, Spanlang, B, Friedman, D, Sanchez-Vives, M, & Slater, M 2010, ‘Immersive Journalism: Immersive Virtual Reality for the First-Person Experience of News’, Presence, vol. 19, no. 4, pp. 291-301.
8. Eden, S 2014, ‘Virtual intervention to improve storytelling ability among deaf and hard-of-hearing children’, European Journal of Special Needs Education, vol. 29, no. 3, pp. 370-386.
9. Gelsomini, M, Garzotto, F & Montesano, D 2016, ‘Wildcard: A wearable virtual reality storytelling tool for children with intellectual developmental disability’, in 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC) 2016 Proceedings, Orlando, USA, 16 – 20 August, viewed 9 April 2018, <https://ieeexplore-ieee-org.ezproxy.uow.edu.au/document/7591896/?arnumber=7591896&SID=EBSCO:edseee >.
10. Jones, S 2017, ‘Disrupting the narrative: immersive journalism in virtual reality’, Journal of Media Practice, vol. 18, no. 2/3, pp. 171-185.
11. Joshi, Y 2016, ‘360° Panoramic Painting (Process)’, Yog Joshi Art, weblog, 2 September, viewed 21 March 2018, <http://www.yogjoshi.com/360-panoramic-painting/ >.
12. Kane, R 2014, The Future is Now. (cover story)’, Editor & Publisher, vol. 147, no. 12, p. 30.
13. Koski, O 2015, ‘Step into the Story’, Nieman Reports, vol. 69, no. 1, pp. 8-11.
14. Lai 2016, ‘4 Steps to create a 360 VR illustration / painting in Photoshop (with pictures)’, Behind90, website, 25 December, viewed 20 March 2018, <http://studiobehind90.com/2016/12/25/how-to-create-360-panorama-painting-in-photoshop/ >.
15. Moore, C n.d., ‘Experiencing Cyberculture – Week Four’, Future Culture, weblog, viewed 26 May 2018, < https://futurecultures.blog/week-four-experiencing-cyberculture/ >.
16. Nilsson, NC, Nordahl, R, & Serafin, S 2016, ‘Immersion revisited: A review of existing definitions of immersion and their relation to different theories of presence’, Human Technology: An Interdisciplinary Journal on Humans in ICT Environments, vol. 12, no. 2, pp. 108-134.
17. Radford, C & Weston, M 1975, ‘How can we be moved by the fate of Anna Karenina?’, Proceedings of the Aristotelian Society: Supplementary Volume, vol. 49, pp. 67-93.
18. Robertson n.d., ‘The Ultimate VR Headset Buyer’s Guide’, The Verge, viewed 26 May 2018, < https://www.theverge.com/a/best-vr-headset-oculus-rift-samsung-gear-htc-vive-virtual-reality >.
19. Sánchez Laws, A 2017, ‘Can Immersive Journalism Enhance Empathy?’, Digital Journalism, p. 1-16.
20. Shin, D 2017, ‘How does immersion work in augmented reality games? A user-centric view of immersion and engagement’, Information Communication and Society, p. 1-18.
21. Shin, D 2018, ‘Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience?’, Computers in Human Behavior, vol. 78, pp. 64-73.
22. Shin, D, & Biocca, F 2017, ‘Exploring immersive experience in journalism’, New Media and Society. Available from: 10.1177/1461444817733133. [14 April 2018].
23. Southgate, E, Smith, SP & Scevak, J 2017, ‘Asking ethical questions in research using immersive virtual and augmented reality technologies with children and youth’, in IEEE Virtual Reality (VR) 2017 Proceedings, Los Angeles, USA, 18 – 22 March, viewed 10 April 2018, <https://ieeexplore-ieee-org.ezproxy.uow.edu.au/document/7892226/?arnumber=7892226&SID=EBSCO:edseee >.
24. Sullivan, T 2018, ‘VR gets real: Immersive storytelling in journalism’, PC Magazine, p. 103.
25. Sundar, SS, Kang, J, & Oprean, D ‘Being There in the Midst of the Story: How Immersive Journalism Affects Our Perceptions and Cognitions’, Cyberpsychology Behavior and Social Networking, vol. 20, no. 11, pp. 672-682.
26. Turkle, S 1994, ‘Constructions & Reconstructions of Self in Virtual Reality: Playing in the MUDs’, Mind, Culture, and Activity, vol. 1, no. 3, pp. 158-167’.