Design Fiction Chronicles: The Dark Knight's Ubicomp Mobile Phone Sonar

Here’s that scene from The Dark Knight where Batman has secretly installed a surveillance system that traces the legal, moral and ethical contours iconic to ubiquitous computing networked devices of this sort. What’s going on — as explained in the short bit of dialog — is that all of the mobile phones used by all of Gotham’s citizens have been secretly connected to this rig that is able to produce sonar-like visualizations of their surroundings to such a level of resolution that one can *see and *hear everything. Batman is asking Lucius Fox / Morgan Freeman to man the rig and listen out for The Joker and direct Batman so he can capture him and end his felonious shenanigans. Lucius plays the moralist here, drawing issue to the fact that Batman would be invading people’s privacy and, moreover, misusing the system that Lucius constructed.

As pertains the Design Fiction motif, what I enjoy about this scene is how quickly it is able to center the pertinent extradiegetic debate on surveillance technologies. Whatever one feels about ubiquitously networked devices and their implications for issues such as the possibilities for over-arching surveillance, state control, and so on — this one scene and its spit of dialogue, together with a suggestive and fairly easily explained and dramatic apparatus — together all of this is able to summon forth the debate, frame its rough contours and open up a conversation. Nice stuff.

Listening Post

Parenthetically is this device shown above. Called, suggestively, Listening Post, one might be forgiven for mistaking it for a prototype of the surveillance device in The Dark Knight which it may be, or not, or may be both a *real prototype and a probe or a propmaster’s prototype for the film. Or something. In any case, it is a sculpture done by Mark Hansen and Ben Rubin. Listening Post “is an art installation that culls text fragments in real time from thousands of unrestricted Internet chat rooms, bulletin boards and other public forums. The texts are read (or sung) by a voice synthesizer, and simultaneously displayed across a suspended grid of more than two hundred small electronic screens.”

It’s quite curious and depending on what is going on in the world — lovely to listen to. When I first saw it at The Whitney in New York City it was in February of 2003 very shortly after the Columbia Space Shuttle disaster — and the tone of the snippets of chat room conversations were echoing the sentiments of that event. In a sense the device anticipates the aggregation of *chatter that comprises or can be cohered into *trends or *trending topics as the year of Twitter has made increasingly legible.

In any case, the similarity of these two devices — The Dark Knight apparatus and Hansen and Rubin’s “Listening Post” are clearly in some sort of conversation with one another, both provoking similar discussions and considerations, whether or not anyone except me is raising these points.

Why do I blog this? This is a useful example of the way a small, short scene — barely even a story — can help raise an issue to a more tangible and more legible level, making it perhaps more intriguing to grapple with abstractions like the ethics of surveillance. It provides a hook for these conversations in material form.

William H. Whyte Revisited: An Experiment With An Apparatus for Capturing Other Points of View

Times Square Urban Living Room from Julian Bleecker. More Apparatus Videos.

[[Update: The Apparatus was exhibited at the HABITAR show at LABoral in Gijón Spain this summer 2010.]]

A couple of months ago a colleague, Jan Chipchase, floated by my desk and handed me a book of his called “The Social Life of Small Urban Spaces” by William H. Whyte. I had no idea who this Whyte character was and I could only guess what it was about and, just by the title — I figured this would lead me down another rabbit’s hole of exploration and experimentation.

As I flipped through the pages, looking at the images of urban observations of New York City from the 1970s, I was enthralled by the technique as well as the substance of the material. Whyte and his team were capturing the intriguing, sometimes curious ways in which people adapt small corners of urban space and their habits and practices and rituals. The pace and momentum of pedestrian movement is intriguing. Without assuming anything in particular, Whyte’s work was capturing movement in a seductive way — even small scale jolts and shifts and gestures. Someone moving a chair just a small bit to indicate that he is not attempting to invade someone’s microlocal private space. You see the “fast-movers” bobbing and weaving quickly around a phalanx of slow moving tourists, window shoppers or a more elderly pedestrian.

Wonderful, intriguing stuff. Sold. Hooked. What’s the brief? Oh, what would I do? Follow footsteps and curiosities, I guess. I was curious — how can the momentum and pace and speed (or lack thereof) of the urban flows be captured, highlighted, brought into focus and revealed in such a way as to visually describe time, movement, pace, scales of speed and degrees of slowness?

There is lots to say about Whyte, I am sure. I have only begun to scratch the surface of this well-known urban sociologist, explorer, scout, observer. But, for the purposes here what happened as a result of this brief conversation with Jan was something that spread through the studio — a bout of curiosity that led to another, other project. It started simply by wondering if the observational studies that Whyte had done both in this book and in other projects could be done today? And, if so — what might they observe? What might be the questions? By what principles and assumption would small urban spaces be explored?

A copy of the films Whyte had made was secured in short order. Simple observations from ground level as well as from carefully chosen vantage points up high, above the ground. This intrigued me. There had been a project in the studio this time last year with things placed high for observational purposes (high chairs, periscopes, etc.) and it was filed away in the “lost projects” binder, so this seemed perhaps a way to revive that thinking. Over the course of a week, I made four trips to Home Depot, Simon jigged a prototype bracket on the CNC machine, and I had a retractable 36 foot pole that I imagined I was going to hang a heavy DSLR off of — it scared the bejeezus out of me and required two people to safely raise up. Too high, too floppy.

Another pole — 24 feet. Daunting but serviceable. It retracts to 8 feet, which is still quite high, but the range made it worth the embarrassment. After a brief bang around the reputation and suggestion networks, a wide field of view camera was identified and two ordered. Two cameras, secured to the pole produced a fair resolution, very wide field of view for displaced observations from a peculiar point of view. Good enough.

Penn Station Still Observation from Julian Bleecker on Vimeo.

Observation apparatus deployed at 7th Avenue main entrance to Pennsylvania Station, NYC, capturing ingress & egress flows, pedestrians waiting, deciding, waking up in the morning upon hitting the sidewalk, &c. The slow-scan mode highlights things which are not moving and therefore often discounted as to their import such as, for instance, the two peculiar characters to the far left who scarcely move (and were still there at the end of the day, around 7pm!), defensible space obstacles in the form of potted plants, people who wait for things, time to pass, people or taxi cabs, &c.

A notion interpreted and brought into focus by Rhys Newman.

Friday June 19, 16.17.17

15th Street and 5th Avenue, New York City.

Using some generative algorithms to show neutral zones of flow and highlighting areas of relatively stable inactivity. Somewhat mitigated by the windiness of the day which caused the cameras to move quite a bit.

Whyte was intrigued by the movement, flows, behaviors, but also emphasized the engaged observations — pen and paper, not just measurements and statistics. He was observing and analyzing both statistically — flows of people per time period over various widths of sidewalk, for example — as well capturing those things that one misses in abstracted data sets. In the film, his avuncular tone draws our attention to small curious practices. Things like someone to moving a chair in a public open space barely a few feet from where it was so as to indicate to a nearby fellow New Yorker that they were not intending to impose upon their public-privacy.

There was something about these sorts of couplings between the analytic data — numbers and so forth — and the observed, seen and demonstrated activities of people. Observed practices crafted into a kind of story about this subject — the social life of small urban spaces. Finding ways to observe and perhaps produce useful insights and design inspirations based on the observations seems a reasonable goal. There is only so much you can do with the books of abstracted data squirreled away some place before you have to go out in the world. Where I was most interested in exploring was somewhere “lower” than the high-level observations which produce intriguing visualizations but are many steps removed from the everyday, quotidian practices. Some empirical, rough-around-the-edges, observational data ethnography. A close cousin of the truly fascinating data visualizations we have grown to love. Perhaps close to Fabien’s notion of citizen sensors and citizen cartography.

We got plenty of guff with the Apparatus when we took it on the new Highline Park. One rather abrupt park minder — sort of behaving like an airline stewardess on a really bad day — was not pleased with the pole at all and let us know it. I had to talk to someone back at the offices of the "Friends of The Highline" via a cellphone given to me by a guy who was like a human surveillance entity. The woman on the phone explained – after listening to my perhaps overly analytic and historic description of the project, Whyte, &c. – that they do not allow tripods or, "you know..long poles" in the park.

Errr ahhh…

It was all very weird, and very un-appealing and put a cloud on what is a playful project, I think, but — *shrug*.

It’s all to be figured out. Or not. Perhaps its just observation. Scraps and visual thinking. Some notes in video. Constructed objects and anticipation of going mobile in Seoul and Helsinki and Linz and London. &c. Or some kind of exploration to suggest alternative ways of seeing the world around us. That may be closer to the point, at least now.

The post-processing stages of the activity are mostly explorations of ways in which individuals or small groups of people in movement could become their own producers of representations of what they do, in an aesthetic sense. What other sorts of systems might people-flows evoke or be reminiscent of? Weather patterns? Displacement grids? Where is there stillness in the bustle? Can the city’s flows be slowed down to evoke new considerations and new perspectives of what happens in the small urban spaces?

People themselves are often seen to be controlled in a top down fashion — even less insidious than “the man”, I think of the significant pedestrian operator — the “I want to cross” button at many busy intersections. It’s a point of contact with the city’s system of algorithmic, synchronized flows. But what about people as their own algorithms, by virtue of their occupancy of urban space? Not following specific top-down plans, but bottom up actions and movements. Not augmented reality but productions of realities. The center of what happens, displaced from the operational command center that articulates how the flows will operate.

I love these moments that countervene the system-wide control grids, which you can see if you watch carefully the raw footage from 15th Street and 5th Avenue where pedestrians spread themselves into the street, stretching the boundaries of the safety of the sidewalk in anticipation of the crossing. Or, perhaps something I love less but it is still something to note, a bicyclist turning the corner against traffic, possibly into pedestrians who may be less inclined to look from whence traffic should not be coming.

We push buttons to control the algorithms of the city, as in the buttons to control signals and so forth. Or roll our cars over induction loops – these are parameters to the algoithms of top-down controls over urban flows. Suppose we interceded more directly or suppose the geometry of the city were represented this way, as a product of the non-codified “algorithm” of movements.

What sort of world would this be? What would it look like?

Highlighting only things that are moving in the Union Square Farmers’ Market.

A cartesian grid distorted by flows around the Union Square Farmers Market.

Wednesday June 17, 15.04.24

Wednesday June 17, 14.44.17

Help thanks to Marcus Bleecker, Chris Woebken, Rhys Newman, Simon James, Jan Chipchase, Aaron Meyers, Noah Keating, Bella Chu, Duncan Burns, Andrew Gartrell, Nikolaj Bestle. And so on.

Videos live online and will accumulate over time. This is Times Square, NYC, Highline in Chelsea NYC, and a generative video done with Max/MSP Jitter

Design Fiction Chronicles: Modes of Surveillance in "Eagle Eye" and "The Final Cut"

Design Fiction Chronicles: The Final Cut from Julian Bleecker on Vimeo.

Resorting to the clever-smarty-pants-academic titles here, I know. There’s so much you can do if you can say in two phrases what you should really only say with one. Or, I guess — not say at all.

But I said it.

Just a couple of remarks germane to the design fiction trope in this installment of the Design Fiction Chronicles — short reviews of quirky, mostly middling, sometimes good science fiction that exhibits some of the unwritten principles of design fiction.

The first film is called The Final Cut (2004, with Robin Williams, not the one with Jude Law) which Younghee mentioned while we were talking about data and death — two great dinner topics that somehow go together. It’s the story about a guy who is the editor of films that document a person’s life where the principle photography (I guess you’d call it?) comes from audio-video implants their parents have gleefully implanted into their unborn embryos. So, their entire lives are being recorded in a kind of logical cautionary conclusion to Gordon Bell‘s half-cocked idea of wearing a camera/microphone everywhere/always.

Let’s forget debating whether or not its a “good movie” — there are some wonderful design fiction-y things going on in here, the most obvious is the story insights into possible implications of such kinds of first-person continuous continuous attention.

There are two positions on this technology as delivered in the context of the story itself.

The first is that – it is a wonderful way to share one’s life at these memorials which only happen after you have died. People get to see the happy moments in “Rememory” ceremonies, which are like video wakes or something. You can also have these edited “best of” moments rolling on a flat screen at your fancy mausoleum, etc.

The alternate position is that it is invasive, entirely subjective because decades are edited down and all kinds of things are edited out, and it is morally wrong because was the implants are in, you can’t get them out and you can only attempt to block them through (this is cool) underground tattooing procedures that create disruptive blocking patterns rendering the technology inside your head useless or hobbled.

FinalCutDataBank

FinalCutAvid

Some other intriguing things besides the way the drama works through the implications of these implants, and this tattooing culture and practice is the somewhat retro style of the special editing gear that “Cutters” use — the specially trained and small clique of people who do the editing. They are wooden computational editing machines, with the portable edition basically a giant laptop that looks like it is configured for Avid editing. The desktop edition takes up an entire standing desk with three wooden monitors. I thought it was curious — this use of wood materials is something you don’t often see in science fiction.

“The Final Cut” also shows these data banks — like money banks in their production design — in which are stored all the memories of people. They are stored in a proper, marble-floored bank with tenders who retrieve the cartridges of memories from safety deposit like vaults, which seems about right. These are the “raw” footage — everything that was contained and, as such, need to be protected in a trusted, safe place.

Despite the funky story — I won’t give it away — I found this to be a rather unique perspective on the personal perpetual aggregation of everything I do/see/hear mode of full-time life logging or whatever the kids call it these days. It revealed some of the no-shit implications and suggested some other concerns and issues — the things that can come back to bite you in the future, or even after you die, as well as the ability to relive something and see it from more mature eyes.

Design Fiction Chronicles: Eagle Eye from Julian Bleecker on Vimeo.

Okay, the other film I saw right before watching The Final Cut is called Eagle Eye, with Shia LaBeouf, Mike Chiklis and Billy Bob Thornton. It is mostly about having one exploding/falling/head-cracking special effect lead to another, but it is also about a computer that has slipped its moorings and is “activating” normal humans to do its evil, world-changing bidding. It’s programming just gets carried away, basically. Sort of “The Manchurian Candidate” meets “Live Free or Die Hard” meets “Colossus: The Forbin Project”. (And apropos of the revelation recently made by the artificial intelligence intelligentsia that perhaps, you know — computers may become smarter than us and maybe we should think this whole AI thing through a bit.)

“Eagle Eye” is your prototypical sci-fi thriller of this “computer-takes-over” idiom, with basically only enough strung together bits of dialog to get you to the next 12 minute action chase/smash/blast/flash-bang-grenade sequence — the effects-driven film. There isn’t much to say here except for two things apropos of small props, designed artifacts and some aspects of the production design that tell a larger story about possible near future worlds to be either created or avoided.

First, this peculiar visual coupling systems. The AI computer in the film, called Aria and played by Julianne Moore (of all people) has to look at other screens in order to do its surveillance work. Basically, it’s in an enormous silo-like chamber with zillions of displays, like the compound eyeball of a bug or a huge honeycomb. It is as if Aria’s main viewing apparatus hunts around for possible events happening in the world — only one at a time. Of course, this is a suggestive prop, meant to indicate in a material way how a computer could be putting much of the world under surveillance at once — which is a tricky bit of design work to convey to a general audience. Despite the technical preposterousness of this apparatus, there’s something compelling about it nonetheless, at least as a visual solution to something that could get tediously didactic if you told it rather than showed it.

Second — the computer voice. In both “The Final Cut” and “Eagle Eye”, there was this female computer voice that sounded identical to me — an instance of this character confusion phenomenon. Like — wait…which film am I watching that has the anxiety-generating computer robot voice?

Obviously they are not done by the same voice actors, but it got me thinking about the voices of AI computers. Curiously, recently, the voice of the robot/computer/AI in “Moon” was a man — played by Kevin Spacey. And, of course the voice of HAL in 2001 is male as well. Couldn’t think of any other men, except I think the computer in Lost in Space was sort of gender neutral if I remember, but probably closer to a eunich who smokes too much.

But this got me thinking — is there an archetype of the artificial intelligence computer voice. Does it matter the voice’s perceived gender? What gives?

Why do I blog this? Consumed with the implications of two modes of surveillance as depicted as designed fictions — the implant that life blogs every moment of our lives and the all-seeing/all-controlling fully connected to the cloud artificial intelligence computer.

Related Material

Gordon Bell’s My Life Bits, and “Total Recall” book about experiences with the SenseCam and “the e-memory revolution.”
(** Ack. The painful irony that technologists borrow — perhaps without knowing it? — their book title from a cautionary science fiction film about fake memory implants to fuel their own hyped technofantasy visions of the future. I met Bell at USC once. We had lunch in the faculty club. Nice enough guy, but chock full of the expected greying luminary’s hubris. A colleague — also a heavy technogeek — simply couldn’t fathom the possibility that continuous continuous self-monitoring could possibly lead to any sort of problems. Just always be good, was about all that could be offered. Bleech. **)

Slife to tell you where all your time goes when you’re sitting in front of the computer screen. Clever self-analytics.

Nokia Everybit, a project at Nokia to use the mobile phone as a continuous continuous monitor of movement, calls, events, music listened to, WiFi and Bluetooth nodes encountered, etc., etc.
Continue reading Design Fiction Chronicles: Modes of Surveillance in "Eagle Eye" and "The Final Cut"

When Characters Cross: Extradiegetic Imbrication

Monday August 03, 18.25.31

Jack Bauer as interpreted. Seen in a local art supply shop.

First off, I’m being tongue-in-cheek with the blog post title, so lower your weapons. And I’ll be brief. There are two things going on here.

First, a curious, perhaps pathological contemporary media consumption practices: For the last several years at least, the rituals of, well — watching shows on the old cathode ray tube (haven’t stepped up to the flatness, with apologies to the consumer confidence index) means that I basically consume a season of programming in anywhere from three to ten days, depending.

Second, a symptom exhibited by this ravenous media consumption practice: characters on these shows become “my friends” in an unsettling but, well — immersive way. Whether empathetic friends of friends that, you know — are closer to antagonists, they become deeper parts of the conscious mind in a way that sometimes I’m okay with, but sometimes I am entirely thrown off by.

I first noticed this kind of extradiegetic muddle when I had gone through a real immersion into the addictive and wonderfully preposterous show “24” which everyone and their hairdresser knows about so I won’t get into it except to say — I had JUST moved to Los Angeles where the show mostly takes place. Sure, I had watched episodes before while living in New York City, but this was different. After watching probably five or six hours into the morning dawn during one bender session, I had to get up to go teach a class or something and was still muddle-headed and POV day-dreaming about whatever CTU was chasing down.

First thing, a stop at the bank ATM on the way downtown. I pulled into a spot and began my egress from the driver’s seat and I instinctively looked in my door side rearview mirror to see if I was going to take off someone’s hip before opening the door — and:

CINE: SMASH PAN, QUICK SWITCH TO 60 FPS HD SLOW MO

CUT TO: A thick black Land Rover SUV with Jack Fracking Bauer jumping out of it.

I swear to god. There he was, jeans and fitted white t-shirt. I froze. Surely he’d see me and mercilessly slam my head into the dash after crushing it like a melon in the door jamb and demand to know where “he/she/it/they” were — or whatever MacGuffin that would lead to the next inevitable plot turn. But, whatever — Bauer was jumping out and I was busted. I’m sure I caught a glimpse of CTU backup dudes lurking around a corner. And his SUV was black, for chrissake. And that’s never a good sign.

Do you — stay in the car or draw and step-to.

Continue reading When Characters Cross: Extradiegetic Imbrication

Mobile Surveillance

A mobile surveillance box which Duncan spied on his ride to the studio. This area might be a gathering place for day laborers to meet prospective employers. So, the question then is has this been installed to discourage them from gathering here? Has there been crime of some sort, beyond something that this rather conservative, law-and-order part of the world would call an illegal gathering, or loitering, despite the fact that such workers would be the ones to mend one’s fence or till one’s rose garden? This object was found here.

I’m not an out-and-out anti-surveillance person. I enjoy observations, especially one’s involving image-making. When you can’t have a conversation with the man behind the curtain, or the observations are not easily accessible for review, comment, dispute, etc., something may be quite wrong, then.

There are things that should be watched closely to make the place more habitable. For example, refugee camps where rape, murder and all things horrible happen should be watched the same way citizen patriots watch the US-Mexican border..but they’re not. The politics of surveillance are often always authoritarian and about control through observation. Things can be turned about, as they often are in order to disrupt the despotic eyeball, observing through a passive, psychological influence.

This thing, a mobile surveillance platform affixed to a far corner of a 7-11 parking lot, is a preposterous, laughable evil eye. It looks almost retro and steampunk — a hand-made affair that drives the comedy of its pathetic influence. It talks to you, which adds to the carnival. It’s got a bunch of off-the-shelf motion detectors bolted to each side that, I’m guessing, detect motion, set off a timer and then belt out a canned, recorded announcement of warning…warning…you have been observed…the authorities have been notified, which sent me into tears of laughter. I felt like I was confronting a home-made RoboCop, built with whatever scraps could be found at the local Radio Shack. @kurt wondered whether you could pop on a set of tires and tow the thing off before anyone knew, which would be hysterical if only to imagine what the “home base” operator would see from their hatch-like monitoring station. Not that I’d do it, of course, but in my imagination — I’d even go through the trouble to gin up a power source so that the kit would keep running. Err…but maybe there’s a tiny man in the box, like the guys who live in small trailers at construction sites to watch over them?

Why do I blog this? A curious, irresistible assemblage of crappy DIY surveillance that just was begging to be taunted. I’m so happy it talks, I just may have to go back and interact with it/him again. It’s like an old, urban robot in this century, made of wood and with an axle for wheel-born mobility. Proto, or Neo RoboCop sort of stuff. Brilliant..it could be Interactive Art, even. Maybe a thesis project is lurking behind this, complete with a faux company website!
Continue reading Mobile Surveillance