This is a curious project from some students at MIT. They’ve used a laser beam and a camera sensitive to the light reflected from that beam to track the motion and articulations of one’s hand as it moves and makes mouse-like gestures. So, effectively they’ve gotten rid of the mouse. Which is why they call their project *mouseless and why they’ve given it a bit of fun by an explanatory video ripped and sewn with some Tom and Jerry cartoon wackiness.
What I find curious here is the way they’ve extended the “mouse” metaphor even when the mouse has become “invisible” — or, rather — those bits of plastic and wire and so forth that constitute the mouse are now no longer necessary. But, we’re still operating with the same movements and gestures as if the mouse were there. Which makes me wonder why go through the hassles of taking it away, losing the physical tangibility of moving something with momentum and weight and texture and feedback and all that. It’s like one of these weird engineering efforts to do *something with the technology and then backfill the rationale. I mean — it’s all tiring in a way how little refinement and design and thinking and iteration goes into things like this. I’m exhausted just looking at the invisible mouse..that I can’t see. I mean — I guess the mouse not being there is as weird as the mouse suddenly appearing attached to a computer back in the day, but it’s easier to think of manipulating something material, no matter how weird and unexpected it might be, than it is to pretend that something’s there, that could just as easily be there if we just ditched the idea of an invisible mouse and kept a visible mouse there to begin with. Or something like that.
*sigh*
Well, I guess this is what to expect from the best and brightest. The simple obsession with refining and refining and refining rather than just doing something “’cause” seems to yield much more subtle *wheels-on-luggage designs, just making something a little better, as they say.
Why do I blog this? Thinking about the inevitability of metaphor in design while poking through Raphael Grignani’s remarks on Home Grown’s List UI inspired by Mike Kuniavsky’s draft chapters on metaphor for UI/UX for his forthcoming book, and a recent document that pleads for the end of metaphor and direct manipulation. With regard to *mouseless, I see this as another instance of moving from one extreme to another while missing anything in-between or even off to the side, which might be typical of engineering efforts when it plays in the UI/UX sandbox. ((It also is likely not their point at all, but rather a quick sketch of an idea to refine some thinking, or just a clever computer nerd stunt, but I’ll use their work *unfairly to make a perhaps not all that interesting remark on the blog, and to try to up my blog/writing quotient for practice.)) A bit like coming up with weird doorknobs and then looking for a house to put it on. Carts before horses, or gizmos first, humans last. Maybe somewhere we’re missing the subtleties and low-hanging fruit rather than the grand theatrics (engineers) and broad oratory (chatty design gurus who talk rather than make and refine and get into the material of things.)
I’m not sure if there’s such a thing as invisible metaphors, but one thing’s for sure: there has to be an invisible sun.
I like your metaphor of “coming up with weird doorknobs and then looking for a house to put it on”–I often have that kind of reaction to things like this, but I suppose projects like this will lead to other innovations that do start with (rather than end with humans).
I don’t think the mouse metaphor is appropriate without the mouse. The gestures and interactions that go with the mouse are arbitrary without the mouse, because they’re linked to a certain use of physical space, a range of motion, a hand and arm position, and a set of finger actions that only exist because of the mouse. When you translate the mouse’s functions to touchpads, touch screens, joysticks, track balls, et cetera, you get different uses of space, different hand and arm positions, appropriate for the device. So what would the positions and motions be if you didn’t have to touch the input device? Ultimately I agree that metaphor is inevitable here, because it wouldn’t be possible to replace the mouse without having some other spatial metaphor to what’s on the computer screen.
May I ask about the “document that pleads for the end of metaphor and direct manipulation”? Something I’d be able to find a copy of?
Hey Dan, thanks for the comment! I think you may be suffering from the same issues and questions that make my stomach hurt! As for the document — I wish I could share it, but it’s *confidential. It doesn’t answer much, but continues to pose the questions. In part — one part of maybe seven points — it issues a call for the end of metaphor in UI/UX, which I don’t understand at all. Like — I’m not sure how one does that at all, remove metaphor from an interface design. Perhaps its something way beyond what I am thinking. It’s certainly curious to think about. I remember when I was involved in VR in the early 1990s, Jaron Lanier was saying that this would be the end of symbolic language and it would all be *natural and *gestural. I think the points are similar in these statements about direct UI and no metaphors. We’ll see — certainly exciting topics to consider!
Julian
Maybe the people who call for the end of metaphor are really only thinking of certain types of metaphors. After I wrote my comment, I started to read Mike Kuniavsky’s blog, and I realized that his discussion of metaphor focuses mostly on conceptual metaphors. But I was thinking that any kind of spatial mapping is a metaphor, too. So a computer UI connects us to data and functionality through the metaphor of a desktop, but a mouse or touchpad connects us to the computer screen by turning a flat surface into a metaphor for the screen. In that case, the argument is really about getting rid of metaphor, but about reducing the distance produced by metaphor or removing the conceptual layers…direct manipulation relies on metaphors with the fewest layers.
(Or maybe I’m missing the point completely and just being cynical in saying that we can’t escape metaphor.)