Author Archive


December 8, 2009 Leave a comment

Brain-computer interface device that is currently on the market.

A teardown of the device, explaining what all the parts do (also includes some very interesting reader comments)

Categories: Uncategorized

Some Thoughts on Interactivity

December 7, 2009 Leave a comment

In The Language of New Media, Lev Manovich explores the topic of interactivity with new media objects.  I have attempted to summarize his claims concerning this topic:

Contrary to her impression upon use of the new media object, the user is not a co-author; she is instead forced to follow a predetermined path, stripping her of agency and, eventually, her ability to think for herself.  The user is allowed to select chunks of content, offering her the illusion of interactivity.  The only interactivity that is occurring here, however, is that of the utilization of the user’s cognitive output as she uses the structure of the program as a program input.  The user, through this structured selection, sees the program as fully customized and reflective of her personal preferences and ideas, assuring her of her uniqueness, and thereby supplanting her need for personal associations with hyperlink associations generated by the program that are then accepted by the user as an externalization of her own thought process.  The user then learns to prioritize selection within the context of any program over that of personal evaluation, and the line between information access and psychological engagement is blurred, making both navigation and immersion difficult and leaving the user dependent on the program for navigation as well as for providing the path to a finished creative product that once would have been the result of her own psychological engagement.

My purpose is to explore and also to contest these ideas in light of personal experience with and knowledge of a few current new media objects.

A new type of interaction with new media objects has begun, with applications that perform a task so specific that the act of simply activating them is a declaration of intent.  Instead of searching and directing our own navigation on an all-purpose search engine, we search a library of applications in order to find one that will navigate these types of queries for us.  We navigate the world of many mini-navigations.  Instead of trying different combinations of keywords in search of the phrase that will yield “good lebanese restaurants within 20 miles that are within 5 miles of a Target” we search for “iPhone applications restaurant locator augmented reality,” download it and then activate it by touching the Yelp! icon when we are in need.  In the most absolute sense, the user is following a “pre-programmed” course of “objectively existing associations” (61)–in fact the only user input here (once the application has been downloaded) are the actual choice to activate the app and the automatically determined GPS coordinate location of the user.

In this way, the user’s direct engagement with the application is structured and objectively orchestrated, but on a different level, she is an agent.  Yes, she has selected a certain group of smartphone applications from a library or database that she has somehow decided will serve as her tool set for performing daily tasks, thereby literally manifesting Manovich’s ‘selection logic,’ but she has consciously chosen the structure of her tool.  In the case of Yelp!, she has chosen an application that uses the method of collective filtering as the primary structural element directing navigation; in choosing, this application she has also chosen collective filtering as her own method of addressing the task at hand.  In this case, the method of collective filtering is transparent–it is the marketed feature of the application.  Niche-use applications such as Yelp! compete for users not based on what they do but how they do it.  Featuring the way by which the program sources information has become essential, as the usefulness of an application is dependent not on what it does, as the user is assumed to have already bought into this concept, but on how well the method of doing what it does works.  There is a clear goal that the program is designed to reach and its ability to do so relies on its method.  The user chooses to outsource navigation of a specific kind to a program that uses a certain method for anticipating the user’s desires and access/engagement needs.  The user could have chosen other applications serving the same information access purpose that run on completely different but equally apparent predetermined methods of data processing.  Other niche-use applications, such as Pandora, rely on algorithmic analysis of program data combined with user-history filters that generate “suggestions” based on certain matches of specific data characteristics–i.e. “if you liked this, you probably will like this similar media object.”  Thus, in consciously choosing her method, the user is providing for at least the possibility of conscious cognitive synthesis, considering the output of an application one factor in a much more complicated network of associations, interpretations, and information leading to the formation of a decision.

Given my lauding of the agency inherent in the opportunity to select an application based on its method of processing data, it is easy to assume that I address only applications with a single purpose and scope.  This claim of agency in transparency of method is problematized by applications that have developed multiple functionalities with different methods of processing and presenting data as well as different methods of structuring interaction amongst the different functionalities.  UrbanSpoon (also a restaurant locator) is an application with the singular function of providing access to an navigation of information.  Other applications, including now Yelp!, have developed, often through a series of ‘upgrades,’ additional functionalities including that of social and psychological engagement.  In Yelp!, the user can review restaurants herself as well as accessing the reviews of others, and, using the GPS coordinate location input, can view other users’ current locations.  The user now has the ability to initiate a real-time online chat with other users.  This is often used to initiate online conversation with a user at a restaurant or bar with the intent of requesting current information about the restaurant such as how crowded it is or if the specials are any good that day.  This function can also be used to organize ‘spontaneous’ get-togethers of friends who happen to be in the same area near a favorite location or even as an advanced online dating tool, allowing users to view one another’s profiles to determine compatibility before initiating chat and utilizing mutual knowledge of geo-location to orchestrate a meet-up.

While these applications may not encourage the same conscious choice of method on the part of the user, the user still forms the personal associations particular to internal cognition and agency.  Manovich’s notion that “before we would look at an image and mentally follow our own private associations to other images, [whereas] [n]ow interactive computer media asks us instead to click on an image in order to go to another image” (61) excludes the possibility that personal associations might be interwoven with the hyperlink associations generated by the program.  Upon activating Yelp! while driving, the user might see highlighted a Moroccan restaurant coming up on her left.  This result may be highlighted as a result of her past frequently high ratings of Lebanese restaurants and of the consistently good ratings this restaurant has received from other users or even of the presence of a friend at the restaurant at that time.  However, the user is by no means guaranteed to unquestioningly turn left and enter the restaurant and enjoy the food.  The user may have a particular dislike of Moroccan food or a desire to eat alone.  She may even choose to disable the app in order to listen to a podcast of which a personal association with Moroccan food reminded her.  The restaurant app did not anticipate, nor does it benefit, from this kind of tangential personal association.  The limited purview of many new media applications may actually reinforce the user’s consciousness of the role of her personal evaluation, as well as an awareness of her intent in using the application, whether for navigation and access or psychological engagement or both.  The user may find psychological engagement with another user via chatting on Yelp! difficult, but it may be the possibility of access to real life initiation of psychological engagement with fellow restaurant-goers for which the user has selected the application.  The user may fully intend to use the information gained from this application to generate and control an immersive psychological experience with another person during which she would be assessing compatibility based solely on the interaction itself and not on the person’s user profile.  The user’s choice in managing and layering the influence of the program data and her parallel real life informational input in order to create a mixed reality represents a different type of user agency

This user has selected an application and will process the application’s output in a way that utilizes personal associations and occurs as some form of internal cognition, but through what process does the user do so and given this what is the user’s output?  The application, here Yelp! again, is a hypermediated environment in which the user has agency through the act of remediation.  She imports the application’s output, separating the spatialized augmented reality model, the user reviews, the real-time interactions, the representation not only of physical locations but also of possible physical experiences (the implied act of eating, enjoying entertainment, etc.), and then processes it in an internal cognitive space in which personal associations augment and complicate the output which has become user input and then eliminate, translate, link, problematize, resolve, analyze, and refashion this complex media input and her associated information.  The result is an action or decision that is much larger than the output of the predetermined structure of the program itself.  This action or decision may manifest in a future rating, review, or locative event that will feed back as input to the application.  However, the user willingly offers up her cognitive labor as program input with the goal of receiving more efficient and organized access to a vast database of information previously unknown to the user and of such a great quantity that the user could not sort on an unguided trajectory (or does not wish to) thereby making accessible information she would, in all likelihood, not have been able to access without a programmatic structure.  This then informs her personal cognitive processes, which she can then choose to offer up to the application as input, with the goal of increasing the future usefulness of the application for her own purposes.

The user also engages in a kind of macro creativity of association, combination, and, sometimes, advanced augmentation.  This kind of augmentation occurs on a spectrum.  A user may choose to use a variety of applications in order to foster real life collaboration.  The user and her collaborator may employ a mixture of new media applications, old media objects, online interactions, and real life interactions.  This collaborative project does not preclude immersion in either engagement with the collaborator or the process of creating the project.  The user switches between accessing information, identifying and sharing tools, immersing in work, and immersing in the collaboration, all the while layering and selecting the tools most suited to the task, time, or location.  This example of the creation of a mixed reality (as discussed above) has not caused the users to “to mistake the structure of somebody else’s mind for [their] own” (61).  It has allowed them to utilize the work of the minds of others to engage in a different way with work of their own.

On a different end of the spectrum of augmentation, the user may engage in hacking or re-programming of a new media object itself.  This may involve exploiting the hardware of a device through re-programming it to run custom software that allows it to perform a different extended function that the user deems more personally useful.  Or it may involve creating a personal software application complete with the desired tools or functions through the utilization of both/either the framework/structure of the program itself (algorithmic processes) or the ideas behind the structure of the program (input and output formats, means of access, interface style, method of acquiring user input or external input, etc.).  As Manovich points out, “instead of identical copies, a new media object typically gives rise to many different versions” (36).  However, these versions are not necessarily generated from a selection of templates or by “simply clicking on ‘cut’ and ‘paste,'” (130) as Manovich has claimed are the operations possible in new media applications.  Here new media inspires a kind of variability not generated from a predetermined tree of selection options but from genuine creativity on the part of the user.

While “[p]ulling elements from databases and libraries” may be the default method of operation within a particular application, it is misleading to say that “creating [elements] from scratch becomes the exception” (130).  The user may select applications from libraries and use discrete ‘elements’ from databases which are processed according to the specific method of structure of the application, however, the agency and creativity have not necessarily been supplanted.  The user’s agency and creativity lie in conscious choice of program structure, personal associations and interpretations of program output data, re-programming of the applications themselves, and immersion in experiences which are inspired by and interwoven with these applications.

Categories: Uncategorized

web application for re-imag(in)ing timelines

November 2, 2009 Leave a comment

The paper attached on “continuum,” a web application that can be implemented as a javascript widget embedded in a web page and fed with data from the developer/designers’ chosen source files, explores a possible solution to some of the difficulties inherent in visually representing complex spatial and temporal relationships, especially on a web platform.  The limitations of the web have been the source of much lamentation by many artists, designers, developers, scientists, and users in general and has led to many offline applications and digital art (as described by John Klima in an interview available on  The continuum developers explore other developers’ attempts at representing time in new and more dynamic ways, as well as laying out the difficulties, both technical and philosophical, inherent in constructing these representations.

This introduction to their paper begins with the following:

Temporal events, while often discrete, also have interesting relationships within and across times: larger events are often collections of smaller more discrete events (bat- tles within wars; artists’ works within a form); events at one point also have correlations with events at other points (a play written in one period is related to its per- formance, or lack of performance, over a period of time). Most temporal visualisations, however, only represent discrete data points or single data types along a single timeline: this event started here and ended there; this work was published at this time; this tag was popular for this period. In order to represent richer, faceted attributes of temporal events, we present Continuum. Continuum enables hierarchical relationships in temporal data to be represented and explored; it enables relationships be- tween events across periods to be expressed, and in particular it enables user-determined control over the level of detail of any facet of interest so that the person using the system can determine a focus point, no matter the level of zoom over the temporal space.

Categories: Uncategorized

Ethington’s time travel

November 2, 2009 Leave a comment

In “Rethinking History,” Ethington’s claim that “time travel is impossible because there is no time in which to travel” (471) inspired in me a rethinking of all of the youth restoration and bodily preservation attempts we have developed in the beauty industry and bio-cosmetic industry.  It also brought to mind the historic preservation and restoration in which most cities have engaged, often in areas that have deteriorated due to neglect stemming directly from racist and classist institutional policies.  As Ethington points out in the paragraph above, the definition of time supported by many natural scientists and philosophers of science, specifically Hans Reichenbach, in this case, is that of “the interval between one entropic state and another” (471).  This definition of time and the spatialized model Ethington sets forth lead me naturally to the idea that it is not impossible then that scientists and engineers might devise a method for reversing the “thermodynamical processes” (471) implicit in the deterioration of buildings or streets or other physical objects by which a space of the past is made a place.  And in doing so, is this not time travel by the definitions set forth above?  Ethington, however, seems not to use these definitions of time to question the possibility of an “asymmetric” but still bi-directional navigation of time, as he goes on to say that traveling back in time would involve “forcing all of the particles in our bodies and the world around us into the negative performance of all the motions that they had just completed,” but that this would be impossible because the “entire planet” and all of its individual molecules are inextricably situated in a vast “network of energy and matter.”  He reasons, however, that even this would not actually constitute time travel, as “no one would remember the difference” (472).  I found this to be a limited exploration of his spatialization of history as it applies to the definition of time in terms of forward-tending thermodynamic processes, most especially considering the modern technological inventions and trends aimed at these ideas of restoration, reconstruction, and preservation.  This argument is left largely unsituated in human thought events.  His spatiality as it applies to time travel is left largely unexpanded (in terms of specific examples and further study of the spatiality of human thought and of large-scale cultural trends) and is thus is unsatisfying in this instance.  This argument’s conclusion is even less satisfying, as he goes on to make a brief unsupported and seemingly simplistic claim about memory, saying “Memories–stored in the neurobiological complex of the brain–would be unmade as time went backwards, and remade as time went forward again.  A different ‘present’ might result, but no one would be able to remember the original ‘present'” (472).  Both the nature and content of this claim seem at direct odds with his lengthy examination of the work of Henri Bergson and J. Allen McTaggart that follows.  Given Ethington’s careful exploration of time and space, specifically as he examines earlier in the essay–in relation to the human him/herself–it would seem that human memory, both collective and individual, is a most important link between spatiotemporal history and its public and private record.  He claims in the following section that Bergson’s distinction of measured time as being an abstraction characterized by the temporality attributed it by memory’s relating of one moment to another is “flawed” due to its “dismissal” of the spatial.  However his own claim concerning memory in the section prior to this seems rooted solely in temporality.  Ethington’s memory is inextricably linked with space as it is ‘unmade’ and ‘remade’ along with all places’ making and unmaking, but this seems to exclude the true spatiality of history as the grounds for memory’s construction of temporality in relation to current place.  A place we have once been and then return to after some deterioration and then return to once again after a restoration may produce many different effects on history as constructed by our memory.  We may feel that the place is actually the same as before the deterioration, or, as is more common, we may construct or experience a layered history.  The place would exist in all of its iterations at once, producing a unique present completely different than the present of the first visit to the place, but in no way neglecting the temporal history of this place.  We remember.  And this memory is the history.  And this history is spatialized because of our temporalization of the place.  The building as it exists now is only spatially significant because of the temporality attributed it by memory.  Human selective explicit perception and implicit total perception in a given place is neglected, leading to a one-dimensional understanding of the situation of time in memory as the process of interactions between both temporalizing the spatiality of the place and spatializing the temporality of the place.  We may perceive the history of the building’s physical processes in terms of the memory of its journey from the first moment we experienced it to now, especially as it relates to all other processes of physical and non-physical restoration.  Or we may perceive the history of this building in terms of its actual construction, deterioration, and reconstruction, especially as it relates to reconstruction of the building’s architectural kin within the neighborhood it inhabits.  Or we may perceive the history of the building in terms of our own inhabitance of the space.  Or we may even perceive the building itself as a single moment which has supplanted the deteriorated moment and the original moment of experience.  Thus Ethington’s claim concerning memory and time travel, as well as his general insistence on the distinction between non-spatialized history (or spatialized history in terms of temporality) and spatialized history (or temporalized history in terms of spatiality) precludes both a true exploration of time navigation as well as the possibility of the spatiality implied in the temporal history constructed through and by human memory and its presence-perception.

Categories: Uncategorized

New Media, Animated Graphics, and Cinema Discussion Questions

September 27, 2009 Leave a comment

New Media, Animated Graphics, and Cinema

Discussion Questions

On Page 220 of The Language of New Media, Lev Manovich claims that new media “follows the dominant semiological order of the twentieth century – that of cinema…For centuries, a spatialized narrative where all elements appear simultaneously dominated European visual culture; then it was delegated to ‘minor’ cultural forms as comics or technical illustrations.”

He goes on to claim that the exception to this is when “information…simply functions as an interface to information” it does not follow the strict sequential  structure of new media.

I wondered if that was meant to serve as a conclusive assessment of the modern semiological order of cinema and, if so, what then are the possibilities in the cinematic medium to produce non-sequential narratives?

Peter Greenaway’s video installations and films “[work] to undermine a linear narrative,” and “[work] on a problem of how to reconcile database and narrative forms.”

Is experimenting with the spatiality of film actually undermining this linear narrative, or is it merely adding another layer to the “hierarchy of levels” and maintaining the linearity of the video medium itself?

If each video in the installation is in and of itself presenting images one screen at a time in a pre-ordered sequential fashion, then is this not reflective of the supposed linearity of cinema?

Additionally, is experimenting with the narrative structure a representative element of the ontological projection of the digital upon culture when it is seemingly a change in the syntagmatic elements of the system that does not alter the paradigmatic elements of the system?

In other words, what would constitute a true change in cinema paradigm according to this definition?

Or should this idea of screen by screen interface constituting a linear paradigm be expanded or clarified?

I thought particularly about data visualizations and compositing when attempting to come up with new paradigms with both new syntagmatic structure and old.

The following is a data visualization by edward tufte, which, along with some of his other data visualizations, constitute a new media object with a (possibly?sequential) narrative structure, as opposed to other data visualizations he and other artists have created which are information maps without a sequential narrative structure, but which still represent the interaction between database and narrative–such as the information mapping work of Jonathan Harris, the narratives of which represent an alternate narrative structure–information not simply over x-axis as time and single screen with all elements of narrative present.

Edward Tufte’s data visualization, “The Movable Feat: New York’s 25th Marathon”

"The Movable Feat: New York's 25th Marathon"

What work has been done on exploring the cognitive processing of these type of infographics, especially as it applies to the differences between cognitive processing of new and old media?

Jonathan Harris’ information map, “transportation”

What does this type of new media object unpack or explore?  Does it have a narrative?

Edward Tufte’s animated data graphics, “Wavefields”

Does ursonography, when devoid of the “special effects” described by Manovich represent a non-linear narrative structure?  Does the original concrete poetry represent a database?  Are there better ways to analyze narrative structure and database hierarchy than by looking at linearity and non-linearity and paradigmatic and syntagmatic structures?  Are the special effects or other media elements of this and other new media actually additional layers or levels or do they represent a new three-dimensional structural theory particular to new media?

Categories: Uncategorized