Apple has had a new patent published by the USPTO today (via AppleInsider), and it’s one that details an extension of multi-touch input that would take gestures above the screen, allowing for 3D manipulation of objects via 3D gesture inputs. Fitting that I should just finish trashing 3D gesture control as a concept, before Apple shows a context-specific means of using it that might actually make sense.
The system Apple envisions would work in tandem with its existing multi-touch input controls on devices like the iPad, allowing a user to lift their fingers off of the screen in order to sort of pull out a 3D shape from a 2D one in software like CAD programs, and then twist, turn and modify it in 3D space using gestures in the air just above the device’s screen.
It would use capacitive touch screen sensors in tandem with proximity sensors to continue to detect fingers even when they’re raised slightly above the surface, and track when they’re doing gestures like pinching and rotating above the screen to both create the shapes, and then modify or move them around. In the patent, there are detailed descriptions of how this could be used in CAD programs to quickly build 3D shapes out of 2D diagrams, by drawing a top-down view of a pyramid, for instance, and then drawing that image out to give it depth.
Other uses include using 3D gestures to sculpt with virtual clay-like materials, and the ability to use gestures to change things like object shadows, brightness textures and more. Overall, it seems primarily designed at giving digital creatives an easier, faster and more natural way to do 3D modelling on the go.
Apple has often done a lot to emphasize the iPad’s ability to be creative, not just a platform for digesting content. This interface innovation could help it accomplish that, and unlike other instantiations of 3D gesture control, it seems to be made specifically for a scenario where it would be useful, and the fact that it would be built-in, system level support with access available to developers, plus its integration into touchscreen technology do make it look pretty user-friendly. Though I maintain it’s still not something that’ll catch on with the general public.