Posts Tagged ‘storytelling’

Thinking with Media

November 23, 2009 Leave a comment

This is an excerpt from the critical statement I’m writing for Clarissa/my graphic novel project, which can be viewed here:

This project is definitely still in progress, but the process of forming the concept of the story, writing what we’d roughly call “part one” of the story, and only after that being able to do all the drawings and their accompanying perspectival labor—these drew my attention to a variety of mediation challenges that I otherwise would not have thought of.

First of all, writing a graphic novel is nothing like writing a novel or a movie. As Clarissa and I brainstormed the story, we wrote down character dialogue that was roughly in the format of a drama, with dialogue attributed to each character, as well as background descriptions of what was or had been going on prior to that scene, and stage directions (“he furrowed his brow”). We spent the most time coming up with this story with Clarissa’s original vision (of creating a graphic novel that investigated the production of knowledge and its relation to Being) clearly in mind. When we finally had a working script written, then I sat down to sketch out the drawings and realized, wait a sec, we have entire paragraphs of dialogue at a time for some characters—how is this going to fit on one page, much less one panel? That’s not the way I’m accustomed to seeing words laid out in typical graphic novels, except slightly in more academic ones like Logicomix, a graphic novel about the history of logic and mathematics written by Apostolos Doxiadis and Christos Papadimitriou. As Clarissa worked on figuring out how we would present our comic on a webpage, I tried to figure out how much dialogue I would include on each page, and also how to parse the play-like script of what we’d written into somewhat coherent page-sized frames, where each comic strip/page as well as the story as a whole made sense. This realization that a graphic novel’s format lies somewhere between a photo-journal essay and a dramatic play may seem obvious in retrospect, but honestly, it was one of the most surprising things to realize mid-way through the project.

To reiterate the situation, which again caught this writer by surprise: it IS NOT NATURAL for most people to “imagine” in the format of a graphic novel, because they aren’t as popular as movies or books. When we were visualizing how the story would proceed, we were IMAGINING it as a MOVIE or a NOVEL or sometimes a PLAY. It took a radical shift in IMAGINATION to start THINKING in GRAPHIC NOVEL terms, to learn to simultaneously NARRATE AND VISUALIZE in snapshots, with shorter dialogue, and with the illustrations as much of the story as the text. I don’t think we really did this at first, and it put me in the position to “remediate” (despite our classroom critique of the term, it feels appropriate to use here) our original script into comic format. In fact, this experience makes me want to question the appropriateness of calling graphic novels “novels” in the first place. Yes they tell a story with an immense amount of graphic detail about the world, and both participate in storytelling, but that’s about all the similarities I can think of at the moment. There are huge differences in how each particular media style both constrains and opens up creativity: novels use descriptive passages to set up their visual argument or milieu, while graphic novels never SAY anything about it, they just show it; novels can be as long as the writer wants, whereas graphic novels have a limited amount of text that would be considered “pleasurable” to read; one could go on about the differences, but let me highlight just one more. I would put graphic novels in same category as television/movies (rather than traditional novels) in this particular respect: attention. Using Kate Hayles’ terms, narrative visual media lean towards the “hyper-attention” end of the spectrum, whereas the novel allows for a kind of deep-attentive experience.

I think what I most gained from working on this project was a greater awareness of the way in which available media—primarily video, novels, and radio—structure the way that we choose to imagine stories, and how we imagine the way the future will unravel. It’s no accident that many people fantasize about how their lives might unravel cinematically, with some kind of epic finale at the end involving a 360 degree panoramic shot of them kissing a lover, or of them facing off against an opponent in the heat of battle, or other dramatic moment. Our media both visually and narratively present particular RHYTHMS and FORMS of storytelling, which we can imaginatively inhabit in order to entertain ourselves or speculate about the future. It’s not easy to change the media that you think with, but let’s face it: we all think with media. But I believe there is something positive about the ability to think with different kinds of media, and to choose… a circumstance like the experience of being bi-, tri-, or poly-lingual.

Here’s an experiment. Try for just one day to use a media format other than a movie when you plan your day (how you’re going to get groceries, go to the library, meet up with your friends at a restaurant or bar, etc.), or drift into daydream, or speculate about the future, etc. Try IMAGINING these things some other way—in snapshot action-packed graphic novel format, for example, or as a radio-play without images and just a narrative voice, or something else. Is it easy or difficult? Constraining or liberating? beyond categories?

As to the structure of the website: originally I’d made a suggestion that we could play off the physics’ paradigm of “breaking symmetry” by thinking of aesthetic ways to do that. However, in the end that didn’t seem feasible, so the model we went for was “thinking about the Large Hadron Collider” (maybe we should have written, thinking “with” the LHC?). To present this we have the comic, a twitter-feed with live updates from CERN, and a series of fragments spoken by the LHC itself that we’d experimented with. These three parts provide different temporal experiences of thinking about/with the LHC: live updates from CERN’s twitter feed bring a sense of immediacy; the LHC monologue fragments are drawn from a database of pre-written script but refreshed at random, making the connections between the comic panel you’re reading and the randomly generated fragment into something that spontaneously comes to you and changes the reading experience.

One of the main difficulties we were working with was how to incorporate some kind of perspective from the now-conscious LHC itself into Clarissa’s vision of a comic that was dealt with knowledge production in a way that was difficult to distinguish from reality. With this concern in mind, we decided to separate the thoughts of the LHC out from the people’s action in the comic itself. In the process of writing these fragments, it was important to think through how a self-aware machine might have different goals than humans, and to this end I wrote several fragments that diverge from the traditional science-fictional issues about machines wanting control over and from human counterparts. One unique feature of machine self-awareness that we were playing with is their desire for connectivity over control of humans; instead of being “Creators” with some biological connection to reproduction, what if machines instead were “connectors” finding non-viral pleasure instead in joining up into larger networks? If this is our experimental case, then what we could graphically illustrate is something where the LHC attempts to connect with its “siblings,” which might be the energy bursts from the sun that strike the earth daily and are of a similar power magnitude to the LHC, thus possessing some kind of affinity with it. This idea was taken from a NY Times article that discussed the possibility that the LHC would produce earth-consuming black holes, and defused that fear by saying that the Earth is bombarded by energy bursts form the sun every day that are of or greater than the LHC magnitude.

As this comic develops, we’ll also be thinking about the reciprocal connection between the humans using the LHC to produce knowledge and perhaps the LHC using people to do something of the same. Here I imagine something like Andy Clark’s “extended mind” thesis, which roughly argues that cognition does not “happen” solely in the brain, but is highly dependent on both the body and the world (the parts we mark out and use) to perform cognitive acts. For example, it is common to say that “I can’t solve that mathematical problem in my head; I need to think with my pencil.” This would be a simple example of the ways that we depend upon other resources in the world, tools or otherwise, to produce what counts as “knowledge” for ourselves. If the “extended mind” thesis holds water, however, it will have to address the difference between us using things in the world for a kind of distributed cognition, and us performing these same tasks with each other (or, possibly, using each other towards processes of cognition). For example, if we go back to multiple intelligence theory, thinking “mathematically” is not the only kind of intelligence and thus not the only activity that demands “cognition.” Surely cognition is also demanded of linguistic and social and artistic processes as well. If that’s the case, then what is to be said for bouncing off ideas on a friend in order to help you think of ideas for your new novel? Or what’s to be said for talking to a friend to analyze and trouble-shoot the confusing behavior of one of your students? Is this not also a kind of augmented cognizing—in partnership with (an)other mind(s)?

Taking these issues into consideration, the territory that our project may get into is where we find ourselves to ALSO be part of the world that something else (the LHC) thinks with, a resource for discovering its origins and finding out more about itself and its place in the world and its relations. In this respect, we would be thinking of LHC-machine consciousness less as an antagonist to humans and more of a partner in mutual knowledge-production—of course this process might not be smooth, but those bumps in the story could be the most interesting.


Reading the Ergodic against the Traversal

September 14, 2009 Leave a comment

I published this first in my blog, since again, it has me engaged at the dialectical level with my own work, and the objective of my work. I am republishing it here again for the purpose of this class.

(First published on

Also, this is a poem I wrote 5 years ago that was published in Anthurium that I decided to revisit here as I think about the link between the psychological, psychosis, Lacan, Manovich’s database complex and media art in the moving talk by Jonathan Harrison (I daresay his visuals are made more poignant BECAUSE of the way he narrated their intentions rather than the actual visuals themselves)

glassy clear beads converge into aqua blue sea of white foam, splashing, ravishing mares enveloping inlets and inclines, bubbling, crystallizing into delicate skin of a naked nymph frolicking among waves leaping high as she metamorphosizes into a curvaceous snowy swan flying high to melt into the horizon where the sun transforms into an orange glob of heat, seizes H20s, and gathers clouds impregnated and producing cumuli which soon dissolves into liquid droplets that reverse precipitation to glassy clear beads splashed blue by Neptune, the fish, producing glassy blue beads that converge.

———————————–Begin text for blog entry below———————–

In reading through Aspet’s work on the meaning of Ergodic Literature, in light of the class discussions and blog posts (and I found David Gruber’s post on this very enlightening), I a now thinking as to what is it about media art that is poignant or ground-breaking compared to hand-crafted or even machine-crafted art (after all, art created with the help of machines and the skills of artisans, be they smiths, welders, machinists, furniture makers, builders etc, have been in existence, in different modes and forms according to existing tools (I refrain here from using the word technology, as it has been over-utilized in very unclear terms). One thing we do know about art, as Jonathan Harris himself would agree, is how it invokes our emotion. One need no sublime art to do that, even the cheesiest forms that hits at a raw nerve or emotional events that is within our recent history, would be enough to break the dam. But what is this new level of emotional access that we could and would like to do with Ergodic literature, a literature that seems able to break through the surface level to allow the audience to interact with the work at the making, or not? How is this different from the campsite fire game of spinning a yarn where each person seated in a circle would add a piece to the tale. Or even a book version where people used to add their own version of a tale that could have been started by anybody else, and continued by another anonymous writer.? Are we able to articulate the collective consciousness of the authors and readers in a manner more incisive as we begin to data-mine the mental models of these different contributors by deconstructing and then reconstructing the way in which their creative processes work by the way in which they choose use particular adjectives, verbs and leading cues?

While Aspeth’s arguments of the Ergodic model is dated in the sense that he seems unable to transcend the old computational model of unicoded ASCII mode of scriptons and textons, he provides a model that can be ripped apart and reconstructed if we now think in terms of a literature (across all textual platforms) whereby the scripton and texton can by turns be transformative to and away from each other. Wardrip-Fruin has provided a pretty good beginning to the re-articulation of Aarseth’s model when he parallels data and process, separating them only by a porous line to signify the artificial separation of the two just to aid our mind that has been schooled and trimmed in a manner not unlike the old-school literary critics that Aarseth had sharply criticized to grasp this layered model of reluctant categories. But more importantly, how can the ergodic model, with its traversal function that seemingly promise the reader/audience the freedom to traverse and trespass the rigid limitations that have been trained by their ids. I see in the traversal function the same form of traversal function that Lacan advocated in his model of the graph of desire

(borrowed from the Lacan and Monotheism:Psychoanalysis and the Traversal of Cultural Fantasy article because I was too lazy to draw my own), where the purpose of traversing the fantasy of the barred subject who is the split subject as the subject who tries to move from original signification to a new form of signification after imbibing the analytic discourse. The analysand crossing over. But in the crossing over, the split subject is able to form his/her own subjectivation by acknowledging the cause of the desire, the objet petit a. But at the point of traversal of phantasy (fantasy), there is something that drops up, which I will call the remainder. And it is this object that drops out that is my point of interest here. For likewise, in traversing the function that Aarspeth and Waldrip-Fruin talks about, there is always something that will escape the net of the model, or something that has to be sacrificed. I have not put my finger into what is being sacrificed, but perhaps in narrating the tale of that which is unnarratable in the conventional literary sense (but then, when one has read Lawrence Sterne’s Tristam Shandy, the very notion of a tale that violates Propp’s 31 functions of narrative elements is not a proper tale has long been dismissed). The protagonist is not human and will not being given humanistic qualities (which is another point of contention and problematics that I will not be tempted to discurse into here). Nor is it an animal or a being that is living and breathing in the cell-biology sense. It would be the narrative tale of a machine and in this tale, there would be many secret chambers, intentional and unintentional. And there would be the remainder. Will the remainder articulate itself organically in the tale, or do I have to find a way to work it in? I will be posting on this in my next post but for now, I want to go back into what it means to be ergodic.

I wrote this piece of poem at an age and time when I knew not what ergodic literature is. Nor did I know much then (except through a slight glance at the crude pomo generative text site
which only those uninformed in the inner logics and rationale of post-structuralist thought could have thought laughable) about how to think through the difference between a machine-generated text using Natural Language and a humanly structured text, where much thought and care is given into making the creation light and spontaneous, and to hide the layers of careful work that is put into creating that effect of sponataneity. The machine likewise is a result of carefully worded code, since to be careless is to invite syntax errors and logical inconsistencies that will result in a manifestation that is obviously ridiculous. Yet, it seems that we think of the software that is created to run the machine as an extension of the way in which the human thinks (and maybe in the way in which certain construction projects are done) ; a series of unfinished work (code) that is left hanging and merely covered over by other code, but always there and waiting to be accessed at some point by one who may be searching for it, or who may have stumbled upon it, a series of random texts that is the articulation of the connection between a score follower and computer-based speech recognition software. Or even texts that have been fed into a database that the code has been instructed to call up whenever certain strings or scripts are detected. As Wardrip-Fruin would say, the “central logic is planning.”This is what Talan Memmott did. Neologism takes on a different meaning, and the author becomes a virtual being, one that goes from mimesis to autopoesis to becoming one that mirrors the real (one that, as Lev Manovich says in his article “Database as a Symbolic Form” about unlimited database size which is the promise dream of the leading commercial database Oracle, parallelling the story by Borges about the map equal in size to the territory the world represents (and I struck by how resonating the stories written by a man who was blind at the peak of his creativity can resonate so well with what one would see in media art, does this speak of an inner eye, the ability to access the unconscious more strongly when one is able to block out the continuous interference from the outside.) Does the promise of infinite storage in the virtual landscape means that we can build a real-life holodeck of all memories and fantasies, with all our thrill at discovering the unexpected and unanticipated that seems to have organically sprung out from the genetic algorithm we have spun? Perhaps this speaks to the psychological database that is a part of the tale. I would like to quote here a poignant articulation of such a database in Manovich’s article

Other types of interactive interfaces make the paradigm even more explicit by presenting the user with an explicit menu of all available choices. In such interfaces, all of the categories are always available, just a mouse click away. The complete paradigm is present before the user, its elements neatly arranged in a menu. This is another example of how new media makes explicit the psychological processes involved in cultural communication. Other examples include the already discussed shift from creation to selection, which externalizes and codifies the database of cultural elements existing in the creator’s mind; as well as the very phenomena of interactive links…New media takes “interaction” literally, equating it with a strictly physical interaction between a user and a screen (by pressing a button), at the sake of psychological interaction. The psychological processes of filling-in, hypothesis forming, recall and identification – which are required for us to comprehend any text or image at all – are erroneously equated with an objectively existing structure of interactive links…Does new media similarly function to play out a particular psychological condition, something which can be called a database complex? In this respect, it is interesting that database imagination has accompanied computer art from its very beginning.

Hence, what is the relationship between this psychological database complex that Manovich is talking about with the procedural rhetoric that Bogost has highlighted in the function of persuasive games. Game engines can only function if there is a database to store that function. By creating persuasive games instead of serious games, are we then performing the traversal of function, the traversal of phantasy, where we reach into a higher ethical level than is possible through traditional modes? Are we closer to building this engine/database of ontology that has only been a series of abstract words and sketchy diagrams? Are we now revolutionizing the original conception of syntagm and paradigm by making them the signifier (or framework) by which we can now position our generative text and art? By making tangible the paradigmatic and syntagmatic what has only been a discursive construct of the lexical, can we now build a database of consciousness based on a clearer and thorough-going articulation of the ontological?

Maybe new media, and media art, has never been anything new or the province of the digital age. It’s ability to play a more discursive role is perhaps only accelerated by the existence of digital tools. But at the same time, it gives voice to pseudo-artists who has only scrapped slightly at the surface of the

envisioned by Aarseth and later Waldrip-Fruin.

And to end, I do agree with Harrison in his article Beyond Flash, that

I believe our medium – the online medium – has the potential to become the next great way of processing and expressing our world. Some would say it has already reached this point, but I believe it still inhabits an awkward adolescence, with no real virtuosos and no real masterpieces, and that the only way for it to mature is for its leaders and practitioners to push themselves to make better work, which will, in turn, reach a larger and less insular audience. If the work is purely technological, it will be less likely to reach this larger audience, for it won’t resonate with as many people. If it connects on a more human level, on the level of ideas, it stands a better chance of touching people deeply and spreading widely, like a Toni Morrison novel or a Steven Spielberg movie.

And that is the vision of my project, I have hinted on in this page, and which I will be blogging about after this. An intellectual idea that cannot change lives even at the most banal level is one that is stuck at the canal of its birth.