The other day, I went on a short tour of some of Microsoft’s Labs, where they do everything from rapid prototypes of new products to acoustic testing in anechoic chambers. Most of my time was spent in the Applied Sciences group’s labs, where they are working on some seriously interesting devices. And they’re not just into mice; in fact, the lab’s specialty seemed to be anything to do with optics and/or input. This lab worked on Project Natal, and also on the pressure-sensitive keyboard I wrote about a while back.
They were kind enough to show me all these crazy multi-touch mice, and, when I was too inept to demo even one of them solo, offered to go through them with me on video. Remember that none of these devices are final in any way! The side mouse, for instance, would be much smaller if it were a product, and these weren’t really sculpted at all to fit in your hand.
Let’s just go through them again in text form so you know what you saw. Click the legend here to embiggen.
The Cap mouse is the one I picked as the most ready for deployment. The curved capacitive sensor that makes up the “head” of the mouse is known technology despite being one of the only curved sensors out there, and the form factor of the mouse will be familiar to anyone.
The Side mouse is perhaps the most unorthodox of all the mice shown. It throws out a swath of IR light and watches what it hits; it sounds strange but watching video of the sensor’s view, you can tell that it’s a perfectly good way of telling depth and so on. I personally thought this would do better at the top of a mousepad, to be activated at will while you have a normal mouse for everyday stuff. Here’s that video:
Frustrated Total Internal Reflection, or FTIR, isn’t a catchy name for a mouse, but as a technology it’s quite interesting. The arched acrylic sheet you see has IR LEDs shining light into it at all times. It’ll travel along the sheet and dissipate harmlessly, but if you touch the sheet anywhere, it’ll reflect off your finger and be picked up by the camera that’s underneath. It’s extremely high-resolution and can detect the tiniest movements. This one retained the mouse shape almost as well as the Cap mouse. As he notes, it does remind one of the Arc mouse, which I liked a lot.
One I thought might come out as a completely different product was the Orb mouse. Like the FTIR mouse, it has a camera, but this camera is pointed at a chrome dome that allows it to see every part of the frosted orb. This means it can detect touches anywhere on the thing’s surface. It works as well as you’d expect, but what set me off was the idea that you could actually have a little projector in there that could rear-project images and controls onto the frosted surface of the orb. It’s a bit like the way the Surface display works. I mentioned this and they implied that they were “aware of the possibilities” (I think I hit something good).
Lastly, you have the Arty mouse, short for articulated. This thing is like a cross between a regular mouse and the Novint Falcon. Its little limbs are for resting your thumb and forefinger on, and each is equipped with an optical mouse sensor. It’s like a regular mouse with arms, and it’s a bit disturbing to use. But it is cool — I can see this kind of thing being used for perhaps 3D model editing, but it seemed foreign and fragile, not very easy to sell.
Does it matter?
Multi-touch mice, eh? With laptops ascendant, touchscreens multiplying, and stuff like Natal on the horizon, isn’t it barking up the wrong tree to be putting this stuff on a mouse? A little bit, but not entirely. The mouse is still the standard interface for probably 90% of computers out there, and if they can improve it, they can both extend the life of the decades-old device and introduce multi-touch controls softly into both the OS and the user’s mind. If Microsoft can make a device that is as easy to use as a normal mouse, but with the added benefit of multi-touch (I see the Cap mouse doing this in the shortest time), people will eat it up, provided it’s not too expensive.
At any rate, it’s projects like this that push the boundaries of input technology. If multi-touch mice aren’t be a hit, there were advances made and experiments done that will enable or ease other advanced input techniques.
Regular readers will know that the demonstration of these mice follows closely on the rumor that Apple is creating a new Mighty Mouse, perhaps to be multi-touch as well. That’s all well and good, but there are two reasons why that really doesn’t matter. First, Apple has never made a good mouse. Never. Every mouse they’ve ever made has been bad, from the puck mouse to the clear “pro” mouse to the Mighty Mouse, they’re awful one and all. So I don’t think these guys, who could design a better mouse than Apple in their sleep, are worried about that. Secondly, Microsoft sells mice to Windows users primarily. The software and optimizations are for Windows 7 with its touch support built-in. The Mac mouse market isn’t big enough or important enough to worry about.
Thanks to the Applied Science teams for letting me into their lair. The mice are all very interesting devices, and it’s fun to see them in their larval forms.
Update: There’s more video and documentation of the mice and project here if you’re interested.
Remember that all these prototypes, even if they don’t make it to market, add up to future developments. Besides, a patented input device can always be paired with pricey medical equipment and sold for a premium.
Great article and wow MS is doing some cool stuff behind closed doors. Nice to see!
Big James
I think the side mouse is the best idea yet. The other seem not quite … ergonomic
Pretty cool one.
Hate much? Apple’s mice are always solid, regardless of what you say.
I guarantee you this lab had little to nothing to do with Project Natal, besides rebranding it, as the tech was developed thoroughly and completely by a company in Israel. Props to M$ for actually spending R&D $ on something useful, although considering how long its taking them to bring this to market, it’s actually getting pretty embarrassing.
Besides buying Natal and Bungie, M$ hasn’t done anything right for the past decade. The arc mouse was close, but come on… no bluetooth? what a joke.
Well, I guess it’s a matter of taste, but since I’ve used more mice than anybody I’ve ever met (I do review them for a living), I stand behind my judgment that Apple’s have always been bad.
As for Natal, a senior researcher there told me that in fact they did work on it there, so I’m afraid you’re wrong. They are an optics lab so they had nothing to do with the branding.
Have you tried APPLE’s multitouch mouse that’s coming out next weeks (probably) ?
I don’t care about prototypes, and i certainly don’t excuse the “ooops” at the beginning of the demo, after a just few seconds of use.
This is where Microsoft really leads the way.
Nearly 35 years on the edge of the “ooops” technologies.
It was not an oops. They had set a gesture to close the app.
it was me who said oops; I wasn’t ready for the demo window to close and ended up pointing the camera at an empty desktop.
And the biggest oops in mouse technology has to be the mighty mouse. I can’t say I ever remember hearing anything positive about that POS. Whereas I’ve reviewed many, many mice from Microsoft that are perfectly good.
I personally love something with the Mighty Mouse : the bi-directionnal wheel.
As a web developper/designer, i often have to scroll horizontally on zoomed pictures, to read long lines of code / comments in my IDE or to navigate in directories in the finder’s panel view mode.
Of course you can’t see this big advantage if you’re using a web browser and other programs that break lines automatically, or use other view modes with the Finder/Explorer.
looks like somebody beat the wiz…
All are very interesting and very innovative from Microsoft. I definitely think some of these will be in store shelves soon, because the current mouse is quite basic.
I would gladly use the cap mouse right away. It is extremely cool.
Showing this stuff is the best marketing decision they’ve made in ages. Time to shed the stodgy old image. Great to see some really decent innovation from MS.
This is why I love apple. About time someone else caught up. Not there yet but well done!
Nice prototypes… but it is going to be awkward when Apple releases a multitouch mouse this month (all reports point to yes). Microsoft is still about prototypes. Will they ever make this stuff?
Sure they will. And like I said, Apple’s mice have mostly been awful, and they’re made to work with Macs primarily, which is quite a small market – especially considering they sell a lot of notebooks with great trackpads.
Mice are sooo last millennium. Everybody knows the next big thing is a real full-sized multi-touch UI.
Why not just offer a flat touch screen? I think it’s cool their working on innovative new technologies but it seems like a bit of frankenstein design is going on here. Why make something that looks like a mouse when you can make a true touch screen input device? No moving parts and it’s the future of how we will interact with computers. My TV remote is already touch screen and so is my phone. If we need an intermediate step towards a full touch screen interface I’d say that was the way to go.
Tactile feedback, precision pointing without a stylus, extended-use comfort. I love playing around with new pointing devices and the TrackPoint is certainly a worthy substitute (about 10 fold better than a touch pad) but if I need to do a few hours worth of work in Photoshop or Excel, I much prefer a solid mouse under my hand.
Jason made a good point here: what do we need a mouse for? initially a mouse was designed because there was no other option to interface a computer. multi touch interfaces are an alternative to the mouse. bringing multitouch technology onto a mouse makes no sense to me.
I understand Rick´s comment that some interactions are too precise to do by finger, but using a multitouch interface you can use any device to interact.
on the long term the cat will catch the mouse…
Apple is becoming more like Microsoft, and Microsoft is becoming more like Apple. Interesting.
But I still think Microsoft should stick to software and stop faffing around with these hardware devices.
LOL, I love how you wanted to say “a capacitive touch screen like on the iPhone” in the beginning…
Then you were going to say WinMo, but went with the generic “mobile phone” knowing that WinMo is primarily resistive…
Haha, I tried to cover it up, but…
Apple’s which will be out this month will probably blow this out of the water, not to mention Microsoft probably has violated some of the patents Apple developed for theirs. We’ll see I guess, too early to tell.
Yeah, I’m not so sure about this. As much as I stand by AAPL tech and gear this is counting (and crowing about) your chickens before you have a hen man.
Or they will cross license or do nothing at all. Microsoft has been patenting as long and as hard than Apple and has a huge software patent portfolio that few would want to turned on them and more money than Apple to fuel the court battles. In turn, after all the saber rattling, I haven’t seen Apple going after Palm either yet, probably due to Palm’s extensive portfolio and deep “bank” too.
You’re forgetting about Fingerworks. They were a multi-touch pioneer and Apple bought them for their patents and tech.
So on most of these you leave your fingers hanging? Fatigue anyone? How can this be ergonomic as they claim?
You hold it like a regular mouse. They’re smart enough to detect when you mean to click or do whatever. the side mouse is the only one where your fingers hang.
The articulated looks extremely promising to me as a way to minimize recognition error, even if it turns out to be only for the short term. A lot of the other ones, I’m guessing, will be plagued in the earlier stages with wondering where the fingers are pressing, and where one finger ends etc. I personally really like the idea of the articulated pads. Maybe even a full (four-finger pads and thumb) hand one for incorporating a virtual hand into games?
Nice to see how they are spending the big bucks Ballmer talked about. Didn’t like any prototype though.
Till now MS used 2 button/2 button+1scroll mouses.. now multitouch is kind of real cool.. I still wonder how people who use Apple products survive with one button!
Way to go Microsoft!
Interesting video, thanks! I see the CAP mouse as the closest to market and has some very interesting and compelling features. As a lefthander I certainly hope the drivers are “smart” enough to allow programming the device for us South paws.
2009 has been an impressive year for MS. Windows 7 is great, the Zune HD rocks, Bing is stealing market share from Google…I would totally buy tons of MS shares
great video great article!
theyre all shit- fun for an hour or so- be shit after that!
most likey
I have Rheumatoid arthritis and I hate using touchpads because they make my hand and wrist sore. I don’t know why, exactly, but I think it has something to do with holding my hand rigid while I move my finger around.
These look really cool, but I’d have to try them for a while to see how my RA responds to them.
How many buttons can you fit on the cap mouse?
Maybe they can map a cellphones keypad on it and mix it up with a T9.
Leave it to an Apple fan to defend even their mouse.
I’ll bet you feel the Apple logo can’t be beat the likes of the boring, simple, uninspired Microsoft ‘M’, don’t you?
Personally, I love my Logitech Marble mouse/trackball. Teamed with my Microsoft Natural keyboard, Carpal Tunnel is a thing of the past for me.
Cripes, my grammar took a turn for the worst in that comment.
Microsoft good job but i give you an f because you focus on newer tech and yet your design teams haven’t even caught up with their own production lines. apple has passed you up and it’s hard for me to say that because i prefer windows myself. you might think this is crap but take the word from the your consumers. we have to deal with the rushed products and not many of us like spending a couple thousand dollars and then find out that either we have to pay more or it doesn’t work. so we beg you fix them before the are placed on the shelves. and bring out something better than apple!!!
robert.
Not anymore, the new windows 7 in by far the best OS ever made.
Makes vista look like crap
mr. Gibbs
when it’s fully up and running and they (microsoft) don’t have problems with their gear tell me oh and when it doesn’t cost a billion in taxes
Mark > I still prefere Leopard by far.
You’re so happy because instead of beeing tortured by your OS, now you’re just hurt. And you think it’s happiness compared to what you’ve known in the past.
OS X (exposée, spaces, dock, coverflow, preview, built-in PDF…) + Zooom2 + Warp (for power users)… what else ?
Have you heard of leopard’s greatest fails so far Mr. Gibbs?
when logging out leopard treats you acc like a guest user and erases everything that you had!
good luck
but i still like apple products the rest are fail proof!
Yeah i know that bug, and it’s a very bad one (but it doesn’t affect all the users).
I don’t feel concerned about it because i’m still using Leopard (the bug you’re talking is Snow Leopard only, if installed over Leopard with a guest account activated), i’ve never used the guest account in 2 years (since i switched) and i’m using Time Machine anyway.
any coincidence why we had this video disclosure before the apple magic mouse?
Hi Devin Coldewey ,
This Article looks extremely good for me as a way to minimize recognition error.even if it turns out to be only for the short term. I personally really like the idea of the article pads.