Super Reality

Augmented Reality (AR) and Virtual Reality (VR) Industry News and Technology Updates

Check out the Fingo SDK 1.1 Release

It has been a busy couple of months at uSens as we’re glad to announce the release of Fingo SDK 1.1 with some major improvements from the initial Fingo SDK.

Head on over to our developer page and login to get the latest version https://developers.usens.com/.

We are constantly working to improve our algorithms to provide you with fast, accurate and low latency hand tracking solutions to create truly immersive ARVR content.

As always, your feedback is appreciated and essential to our continued development, so please let us know at developers@usens.com if you have any issues to report or feedback.

Updates included in this SDK:

  1. Fingo Service Windows executable: V1.1.0
  2. Fingo Service Android APK: V1.1.0

What’s new in the SDK 1.1 release:

  • Improvements to our hand tracking algorithm, so that tracking is more accurate, with lower latency
  • Improvements to tracking with two-hand models: When a user’s hands are overlapping or the view of a user’s hands are obstructed by objects, improvements have been made to track both hands positioning
  • Improvements to hand gesture accuracy
  • Improved graphics for hand model for better aesthetics
  • Better handling of complex backgrounds: When a background is complex or contains multiple objects, the algorithm is more stable and is less likely to lose track of both hands
  • Improvements to tracking when a user’s hands are further away from the cameras
  • Improved stability when pointing directly at an object directly in front of the camera’s field of view (when the finger is obscured by a user’s hand)
  • Improvements to tracking of Ring Fingers
  • Support for Google Pixel phones
  • Optimized and faster initialization of the Fingo service

Bug Fixes:

  • Resolved the issue where the Fingo service would sometimes crash when inserted into a Samsung GearVR headset
  • Drastically reduced the issues encountered with Fingo device detection when conducting a hot-plug (rapidly inserted or unplugging the device) of a Fingo device

Supported Platforms:
Windows platform: Windows 8, Windows 10 (Windows 7 is not supported)
Android Versions: Android 5, Android 6, Android 7

Some known issues still remaining with the Fingo SDK:

  • Hot-plugging the Fingo module too quickly may result in the Fingo service crashing on Windows and Android platforms
  • On PC when running on Windows: The Fingo service will only show as detected once a hand is waved in front of the camera. If a hand is not detected, it will not show as detected.
  • On Google Pixel: After a hot-plug it can take 5-10 seconds before the Fingo module is recognized and can be used

The uSens Team

Reasons to Attend SIGGRAPH this Year (Two Free Expo Passes to Giveaway)

The annual SIGGRAPH conference brings together more than 15,000 computer graphics and interactive technology professionals together in Los Angeles, California from July 31 – August 3.

uSens is exhibiting at booth #236 this year, and we are giving away two free SIGGRAPH Expo passes via Twitter this year. It takes less than 30 seconds to enter: more details available here: http://bit.ly/2urU0I3

Here are some reasons why you should attend SIGGRAPH this year:

The Virtual & Augmented Reality Industry is Continuing to Heat Up

Virtual & augmented reality technology is expected to take center stage as the industry continues to heat up with hundreds of millions of VC funding entering the industry this quarter once more. Apple entering the market via the VR kit and Facebook announcing an upcoming $200 Oculus rift has fueled the industry further. Now with every major smartphone player heavily invested, ARVR is here for the long-haul, and SIGGRAPH will be showcasing some of the technologies that will create the next generation of content.

Discovery and Inspiration:
SIGGRAPH allows you to fully immerse yourself in the newest technologies that inspire you to experience the world in a more interactive way. There are hundreds of companies displaying a variety of specialties, devices and services that showcase motion graphics, engineering, interactive entertainment, and new systems we have never seen before. SIGGRAPH is here to bring us an updated, fresh and curious perspective on evolving technologies.

Engagement:
Not only will you be listening to many of today’s pioneers in developing the latest technologies, you will also be interacting with many of these companies’ projects. Live demos, getting hands-on with future projects/graphics, and testing developing products are such great opportunities for anyone interested in discovering the technologies powering computer graphics.

And of course, uSens will be there:
Your VRAR experiences are not complete unless you feel present in the virtual world. We are on the bleeding edge of the technologies that will allow developers to create engaging contents and immersive ARVR experiences.
We have brand-new demos and exciting news to share with attendees so drop by our booth #236 or email us at info@usens.com to schedule a time to chat with us.

uSens Awards $50,000 uDev Challenge Grand Prize at AWE

This past week we exhibited at Augmented World Expo in Santa Clara. The annual AR/VR event is gaining traction as the leading merger of minds in our industry, and this year’s event was no different. We were pleased to demo our latest technology with the 4,700 attendees who came out to make this the biggest AWE yet.

AWE was a time to celebrate for uSens, as we announced the winners of our uDev Challenge and awarded cash prizes to our U.S. regional champions. The #uDevChallenge saw software developers compete to create unique AR and VR apps using our SDK and the uSens Fingo module.

uSens awards Chris Wren with the $50,000 U.S. uDev Challenge grand prize check at Augmented World Expo in Santa Clara, Calif.

When the final panelist scores were tallied, Chris Wren and his app MonsterShop walked away with the $50,000 U.S. grand prize.

MonsterShop delivers the framework for a VR retail experience by allowing players to customize the appearance of monster avatars via drag-and-drop hand gestures.

$50,000 is a lot of money, so we asked Chris: What will you do next?

“I’m putting my daughter through college!”

Well done, Chris!

You can purchase the same Fingo module Chris used to develop his prize-winning app! Visit usens.myshopify.com to get yours for just $99.

uSens Announces Winners of $200,000 ARVR App Developer Contest

At the Augmented World Expo (AWE) opening today in Silicon Valley, uSens, Inc. announced the winners of its U.S. regional uDev Challenge.

A total of $200,000 in prizes were distributed amongst challenge entrants in the U.S. and China. These are the winners in the U.S.:

  • Grand Prize: Developer Chris Wren won $50,000 for his application MonsterShop. The application delivers the framework for a VR retail experience by allowing players to customize the appearance of monster avatars via drag-and-drop hand gestures. For example, players can drag different colored eyeballs and facial hair onto the monsters.
  • 2nd Prize: Developer Wallace Lages won $25,000 for Krinkle, a mobile action game that enables hand-activated spells. Players use motion gestures to shoot fire from their palms and form water balls.
  • 3rd Prize: Developer Tom Leahy won $10,000 for Embodied Labs, an interactive medical training tool that lets users experience macular degeneration first-hand.

In the Chinese regional competition, winning titles included fighting games GestureMagic and Shadow Play, the self-explanatory Throwing Bomb Game, and electronic circuit assembly experience VR Electronics Laboratory.

“We were impressed by the quality and creativity of the projects submitted by our semifinalists,” said Dr. Yue Fei, uSens CTO and co-founder. “Our winners integrated Fingo modules in imaginative ways and demonstrated the ability to make engaging, exciting use of our tracking technology to benefit end-users. I’m very pleased about the promise of future applications of our tracking tech.” Dr. Fei is speaking at AWE at 4:15 pm on June 2 on “4 Keys to Augmented Reality’s Future.”

Winning projects were selected from a pool of semifinalists, and determined by a panel of judges including analyst Jon Peddie, Samsung Research America director Chris Peri, Silicon Valley Virtual Reality Meetup founder Karl Krantz, and uSens executives. Project scores were based on several score categories, including use of the uSens Software Development Kit (SDK) for augmented and virtual reality applications. uSens’ SDK integrates the uSens Fingo module for 26DOF (degrees of freedom) hand tracking on mobile ARVR platforms as well as PC-based VR systems.

All uDev Challenge submissions were made for Samsung Gear VR™, Google Cardboard™, HTC Vive™, or Oculus Rift™. Semifinalists developed their applications using the uSens SDK and an entry-level uSens Fingo module for adding 26DOF controller-free hand tracking.

The uSens Fingo series of modules offers advanced 3D human-computer interaction capability for both mobile and tethered systems. By attaching a Fingo to the front of a head-mounted display (HMD) device, end users can enjoy their favorite apps without major sacrifices to system power consumption or performance – even on mobile.

AR/VR: A Passage from Hype to Certainty

By Dr. Yue Fei, CTO and Cofounder of uSens

2017 will be a pivotal year for virtual and augmented reality, given rather precarious positioning in the trough of disillusionment on the famed “hype cycle.” Industry predictions are starting to grow lofty again, with the most recent IDC study claiming that AR and VR headset shipments will approach 100 million in the next 5 years. VR/AR funding is at an all-time high. And the number of active users is forecast to reach 171 million by 2018.

But does any of that mean the long dark night is over? Not just yet. We’re going to be disappointed if we expect the industry to continue growing unabated. The promise of AR/VR may be widely understood by the general public, but that doesn’t necessarily make it a mainstream technology.

Right now, the industry is still plagued by user nausea, not to mention a lack of content and affordability issues.

The knight in shining virtual armor, for the near term, will be smartphone-based VR platforms. With their greater ease of use, lower cost, and wider range of games and applications, Mobile VR is like the “gateway drug” that will give way to mass adoption. Eventually.

There are a few big players in Mobile VR already: Google, LG, and Lenovo. But given that there are approximately 2 billion active mobile smartphones in the world capable of providing AR/VR content, we should expect more mobile makers to jump on the bandwagon and come out with their own devices and applications soon. Mobile VR presents a much lower cost-of-entry for consumers, and much less risk for manufacturers.

Naysayers will point out that current smartphones just aren’t built to handle the intense computational load that VR requires, and they’re right. Most legacy mobile devices are still apt to kick into overdrive and overheat while running a sim, which puts a time limit on any VR fun. But chip makers and smartphone manufacturers are already stepping up their game: 2017 is the year we’ll see VR-friendly devices that don’t bake as easily and are designed to handle the additional processing.

Even with state of the art mobile devices distributed by the millions across the world, if there’s no content to run on them, VR won’t latch on. Some developers have taken the plunge and created crowd favorites (à la Pokémon Go), but we still haven’t seen the “killer app.” At this point, I don’t know that there will be one. We might just see content growth via user engagement campaigns from healthcare, social, and enterprise applications. Consumers often wait until “the timing is finally right” to adopt a new technology. Such timing is rarely tied directly to one event, but rather a collection of events that eventually collect enough weight to tip the scale. VR content in myriad forms will be required to supply that substance. Meeting (and then exceeding) consumer expectation will depend heavily on the available ways users can immerse themselves into the content and interact with it.

Which leads me to another hurdle: There are some basic human-computer interaction problems that the industry must address this year. The ability to employ familiar interfaces — seamlessly scroll, tap, or type to navigate your way through a virtual realm — would make a world of difference. Until recently, natural gesture and position tracking have been a challenge in AR/VR, but the industry is rising to the challenge: HTC has years of investment in external trackers and the Oculus Rift will eventually deploy hands-free control. There is nothing yet on the market that successfully combines mobile 3D hands-free tracking AND robust head position tracking in AR/VR, but we’re working on it.

Inside the AR/VR echo chamber, it’s easy to think that the public is ready to fully embrace our technology. But it’s still early days. Virtual reality is not for everyone, and there are going to be many more unforeseen obstacles to overcome as the sector matures. For those of us founding this entirely new industry, a clear focus on known issues as we pass beyond hype and into certainty will sustain us through 2017 and beyond.

AWE Auggie Awards 2017— Vote Early, Vote Often!

By Jan Olaf Gaudestad, uSens Director of Business Development

The Augmented World Expo (AWE), featuring some of the world’s hottest augmented reality and virtual reality technology, returns to the Santa Clara Convention Center May 31 – June 2. We’re excited that the uSens team will be in booth #400 so please stop by to meet the team, experience a demo, and discuss anything and everything AR and VR related.

At AWE, the winners of the annual Auggie Awards will be announced, highlighting the best-of-the-best in AR and VR. We’re thrilled that our Fingo 3D stereo vision hand tracking technology is nominated in the Best Developer Tool category. With its mixture of revolutionary software and hardware, Fingo offers low latency, 26DOF and the tracking of 22 finger joints. The combination of these features allows users to interact with digital objects in AR and VR the same way they would in the real world. Simply by attaching a Fingo module to the front of a HMD device holder, you can experience your favorite apps controller-free. Fingo supports mobile ARVR platforms including Samsung Gear VR™, Google Daydream™, and Google Cardboard™, in addition to any Android OS-powered glasses for augmented reality. Fingo also supports PC-tethered virtual reality systems such as Oculus Rift™ and HTC Vive™.

Public voting on the Auggies is now open. Please vote for uSens Fingo by May 22 to help us reach the semifinals! (Quick tip: Try voting though a desktop computer as well as your mobile device.)

Additionally, uSens co-founder and CTO Dr. Yue Fei will be speaking at 4:15 p.m. on June 2 to share his insights on “4 Keys to Augmented Reality’s Future.” Dr. Fei will break down the challenges to AR adoption and how to overcome them. With all the current hype around the promise and potential of AR, you won’t want to miss this featured presentation.

Whether you’re an executive, designer, developer, analyst, investor, or reporter, AWE is a fantastic opportunity to discover all the latest news and trends in our industry. Experience it with us and share what’s top-of-mind for you in the world of AR.

UC San Diego Hosts a Distinctly Unconventional Event to Ignite a Future of Innovation

This is a guest blog post contributed by Rezin8 of San Diego, California.

Photo Courtesy of UCSD News

University donors, philanthropic leaders, and esteemed pillars of the UC San Diego community gathered to kick off the public phase of an ambitious $2 billion comprehensive fundraising campaign. This project will drive innovation that advances society, propels economic growth, and makes the world a better place.

The public research university collaborated with Rezin8, a paradigm-shifting creative lab, to present attendees with a unique guided tour and presentation that will serve as a portal to the interactive reality. Driven by boundless curiosity and a foundational mandate to challenge traditional thinking, UCSD is pioneering breakthroughs in nanotechnology, climate science, machine learning, emerging arts, and much more. Champions of the university think it’s just the beginning.

UCSD spent the evening taking a look back at the achievements their alumni were able to make in technology and medicine. More importantly, they took the time to ignite a future of innovation and advancement they want to bring into fruition.

To bring UCSD’s enthusiastic vision to the reality, Rezin8 designed and developed a unique experience by leveraging interactive content coupled with augmented reality, virtual reality, futuristic presentations, performances, smart bracelets, and robotics — these were just a few of the innovative technologies deployed as part of what was a captivating and inspirational evening for all.

The event transitioned from a tunnel to a cocktail area in the gymnasium. From the entrance of the tunnel, attendees had their own unique reality experience. Different settings will cater the visual displays to a more custom setting — attendees were welcomed with their names digitally displayed.

To augment reality, tablets were distributed to each attendee in the cocktail area. This tablet provided a variety of special effects throughout the event. Human interactions were minimized for this special event. A robot, programmed and coded by Rezin8, traversed from guest to guest to take photos.

As seen through the tablet, the augmented reality and digital content progressed with attendees as they moved through the tunnel. Special effects like rain and clouds were a part of the experience and changed depending on the attendee’s time and location within the experience.

Rapper Snoop Dogg (Left) + Hologram Tupac (Photo by Christopher Polk/Getty Images for Coachella)

The event also included a futuristic live performance to the presentation. Nathan East, considered one of the most recorded bass players in music history, performs as a digital projection. Rezin8, UC San Diego’s creative partner for the event, is no stranger to high-tech digital performances. In 2012, the creative lab worked with Digital Domain to deliver one of the most legendary music performances in history – the Tupac hologram for Coachella.

The event celebrates a halfway point to their fundraising goal, and serves as a critical moment to support the completion of their $2 billion fundraising campaign. The evening was a fully automated, tech-fueled experience, manifested in the living spirit of innovation – the very heart of UCSD. Saturday’s event inspires action from the ever-expanding community of innovators at UC San Diego and beyond — students, alumni, faculty, and philanthropists.

Established in 1960, UC San Diego has become one of the world’s 15 largest research universities.

USENS and #GDC17

As five days in sunny San Francisco came to a close, it was clear that virtual reality was all the buzz throughout the duration of Game Developers Conference, especially with the first two days of the conference being completely dedicated to VR.

VRDC brought VR tutorials, VR boot camps and industry relevant speakers into an immersive AR/VR experience at the Moscone Center. This was VRDC’s second year and featured two different tracks for attendees to participate in: VR and AR for game development and VR and AR for other forms of entertainment–from CG movies to filmed experiences and beyond.

It’s clear that VRDC’s presence for the second year at arguably the largest gaming conference in the world, means that virtual, augmented and mixed reality is the future and the most important steps in developing games for users and progressing the industry forward.

The following three days opened the exhibition floor to companies and organizations to display their relevance to the gaming industry and how their technologies and products can progress technological advances to the next level.

Oculus brought a full array of systems and demos to GDC sending an army of employees donning distinctive indigo shirts to recruit attendees to use their system with Oculus Touch controllers.

PlayStation and its deep lineup of games allowed GDC-goers to sample everything from Grand Turismo Sport in a sit-in pod with a PS4 Pro to bringing back the infamous Crash Bandicoot of the early PlayStation days, and showcasing its wide variety of PSVR games as well.

However, while controllers in VR and gaming in general still play a prevalent part in those fields, hand tracking sparked the interest of industry professionals and the thousands at GDC.

USENS INC was a trending topic of conversation over the three days on the exhibition floor of the South Hall at the Moscone Center. We demonstrated our hand tracking capabilities with demos to show to the attendees of GDC.

With several FINGO demos set-up, thousands of people in attendance flocked to the uSens booth (which happened to be close to the primary restrooms of the hall for maximum foot traffic – very strategic 😉).

Throughout GDC, USENS interacted in-person and over all social media channels with tech, AR/VR and gaming influencers, journalists and industry professionals — all raving about USENS and FINGO. Here are just a couple examples:

@DennisScimeca: Seriously: If you’ve never tried hand tracking in VR and have time, stop by the @usensinc booth. Tech works exactly as advertised. #gdc17.

@Alexis_Macklin: Enjoyed testing out @usensinc at #gdc17.

From verified Twitter users who promoted uSens to their thousands of followers to the individuals on social media who post with just a passion about their interests, USENS and our FINGO were trending topics coming out of GDC.

USENS INC was featured in an article by Gao Yun of CGTN featuring FINGO and our tracking capabilities:

San Jose, California-based uSens – founded by two Chinese developers – created a technology that utilizes a camera to recognize all the individual bones inside the hand, and then relays that information to the application.

“Right now, people cannot interact directly in VR, but holding a controller is unnatural,” said Fei Yue, co-founder and CTO of uSens, adding that they are now letting people do whatever they want to do in real world.

 As VR moves further into mainstream society, technologists agree that the experience needs to become more natural, and ironically, more like everything in the real world.

 

The Future of Driverless Car and Augmented Reality

The first fleet of self-driving cars may be ready by 2020.

Google has said that its car should be ready by 2020 and Tesla CEO, Elon Musk, says their Tesla’s will be fully autonomous by 2018.

Autonomous Tesla Vehicle

It appears that these driverless cars are still in the data gathering stage. Since increased safety is one of the top goals for these future cars, it is important for them to harvest great amounts of data in order for them to respond in the moment and react to potentially dangerous situations. One of the biggest worries of driverless car technology is that they may not be safe enough to utilize so a lot more data still needs to be gathered to help relieve this concern.

Tesla’s Model S already features some self-driving capabilities known as “Autopilot” and it uses data and driving habits from many Tesla drivers in order to keep improving. Google’s self-driving cars have been driving around and collecting data since 2009 so its software already knows how to react to various real-life scenarios.

As driverless cars are still gathering data, we may need another type of car technology to breakthrough before we go fully autonomous. In a sense, GPS technology could also be looked at as a tool that has been priming consumers for driverless cars for a long time. Cars that feature augmented reality may be able to further prime consumers into buying driverless cars.

PSA Peugeot Citroen, a French multinational manufacture of automobiles, is working on one of the first cars that will finally implement built-in augmented reality in the cockpit. This technology aims to increase driver safety for a more convenient driving experience. If augmented reality cars become more popular, they have great potential to make the transition to driverless cars smoother.

PSA Peugeot Citroen’s work-in-progress automobile features a built-in transparent display, projected onto the windscreen that allows the driver to view data and information without looking away from the road. The HUD is also able to adapt in real-time to objects on the road.

 

The display will feature the usual driving information such as current speed, speed limits (implied from road signs) but it could also introduce new functions. Since the HUD is capable of covering the entire width of the windshield, it could, for instance, clearly give the driver directions or show dangers with a prominent graphic warning.

The most important benefit for augmented reality driving is to increase safety. Since the useful driving information could be seen extremely quickly, drivers are sure to have a better awareness of the road and they are more likely to not be taken by surprise. Moreover, their response times may be much faster since they will be keeping their eyes on the road. For instance, they will be able to look up directions while focusing their eyes on the road instead of looking down at their phones or GPS.

Augmented Reality + Autonomous Car

Since we may have to wait up to 5-10 years before we see autonomous care take over the roads, the process of driving automation will have to be gradual. PSA Peugeot Citroen will be an integral member of the car industry to make sure that the jump will not be so sudden since it will be retaining manual/human functionality. The driving information that will be displayed on the windscreen may help educate the consumer and give them a better understanding of how driving aids and autonomous cars will work.

There is not any word yet on the release date of PSA Peugeot Citroen’s augmented reality vehicle but it is likely to release before 2020.

 

Dr. Eunseok Park Joins uSens as General Manager of Augmented and Virtual Reality Tracking Company

uSens, Inc., a pioneer in hand-and-head tracking technologies for Augmented and Virtual Reality, has appointed Dr. Eunseok Park to the new position of U.S. general manager. A collaborator on nearly 200 world patents, Dr. Park was most recently the U.S. regional director for Samsung Advanced Institute of Technology (SAIT), where he managed research and development in emerging technologies across all four U.S. sites. In his new role with uSens, Dr. Park will tap into the deep business relationships he has established with the world’s leading academic and corporate research entities.

“We are so proud to have a leader with the pedigree of Eunseok Park join the uSens team,” said Anli He, CEO and co-founder of uSens. “He brings incomparable expertise in administration and management of engineering projects and in nurturing teams. uSens will benefit from his broad connections to attract the highest caliber of research and engineering talent and to build industry partnerships that will grow uSens’ footprint in the ARVR industry.”

Dr. Eunseok Park

“I couldn’t be more excited at the opportunity to join uSens at this stage of the company’s growth. I look forward to experiencing the startup world from the inside and to apply my management and operational skills to take uSens to the next level of maturity,” said Dr. Park. “During my tenure with SAIT, I established deep relationships with the top research sites across the U.S. and Europe. I’m excited to mine those connections to bring more world-class research talent into uSens, and to help uSens in the commercialization of its AR and VR tracking solutions.”

A Visionary Leader

Dr. Eunseok Park has spent the last 11 years with the Samsung Advanced Institute of Technology (SAIT), the main research and development center for Samsung Electronics and other businesses within the Samsung Group. Beginning his career at SAIT as a senior researcher, Dr. Park eventually became regional director for Europe, where he identified and developed new areas of research in emerging technologies including mobile health. He also oversaw strategic IP for licensing or purchasing, and supported collaborations with leading European research institutes and business partners. As regional director for the U.S., Dr. Park managed R&D and operations across four sites and more than 100 employees, coordinated joint research projects with leading U.S. academic entities, and identified startups for angel and early-round investments.

Dr. Park received his MS degree and PhD in electrical engineering from Syracuse University in New York, and an MBA from Sogang University in Seoul, South Korea. In addition to his extensive patent portfolio, Dr. Park is the author of many articles published for IEEE symposiums.