Connect with us


Neal Stephenson & Co. turn failed Magic Leap AR project into an Audible drama



Neal Stephenson & Co. turn failed Magic Leap AR project into an Audible drama

Elevate your enterprise data technology and strategy at Transform 2021.

Neal Stephenson, the science fiction novelist, has been a idol of mine for a while. I loved reading his Snow Crash, the sci-fi book that introduced the concept of the metaverse, the universe of virtual worlds that are all interconnected, back in the 1990s. And I happen to be listening to his dense sci-fi novel, Anathem, right now on Audible. So you can imagine my surprise when former Magic Leap chief creative officer Graeme Devine introduced me to Sean Stewart, an alternate reality games expert; Austin Grossman, game developer and novelist; and — yep — Stephenson about an Audible project they were working on.

Their nine-hour audio drama is being published today. It’s called New Found Land: The Long Haul, and it’s something that they conceived while working on a big project on augmented reality at Magic Leap. They had always intended this audio drama, which is like a radio play with actors, to be a stand-alone Audible experience. But they conceived of the back story while working on an augmented reality application for the Magic Leap One AR headset. Sadly, the fortunes of Magic Leap did not allow them to finish their AR project, which would have overlaid imagery from two alternate universes on the real world.

I was delighted to interview Stephenson and Stewart about the audio drama and how they worked on it at Magic Leap until their division was laid off, when Magic Leap pivoted away from consumer technology and focused on the enterprise. They are all veteran storytellers who came to the project from different directions — sci-fi novels, video games, and alternate reality games.

But I’m not entirely sure how much of their story I can believe. It is fiction interwoven with reality, as there is a Magic Leap character who is part of the audio drama story, which is like a radio play with actors. But both Stephenson and Stewart were adamant that I understand that there is no alternate reality game to go with this audio drama. They don’t want people to start hunting down clues on the web or the real world and find out that supposed bread crumbs actually lead to nothing. Stewart knows how obsessive people on the internet can become, as he worked on the alternate reality game when he was at 42 Entertainment. That was a real-world and web-based set of puzzles that led to a marketing campaign for the launch of Halo 2.

They insist it is a stand-alone work of fiction, but they also did not verify to me whether they truly believe there are two alternate realities to the one we live in. For now, I’m going to take them for their words when they say they aren’t putting me on. And you should too, so don’t waste a lot of time thinking there’s an alternate reality game to go with this audio drama. I’m going to listen to this audio drama, after I finish about 10 more hours of listening to the 32-hour Anathem novel that I’ve started. Heck, I think someone ought to go ahead and make the AR app for these guys.

Here’s an edited transcript of our interview.

Above: New Found Land: The Long Haul is an audio drama on Audible.

Image Credit: Audible

GamesBeat: The piece has a very cryptic description. Where did you want to start as far as trying to describe it?

Neal Stephenson: What have you heard so far?

GamesBeat: Just that it got started at Magic Leap, and then it turned into this nine-hour audio drama. It sounds like there’s a web introduction to it out there as well that you’ll have going soon. That’s about it.

Stephenson: This emerged from a group that I was running out of Magic Leap’s Seattle office between around 2015 and 2020. It didn’t start this way, but it ended up containing three novelists and a bunch of other people from various branches of the game industry. The idea was to create an IP, a story universe, that was well-adapted to augmented reality.

There are two ways we can play this. We have a fictional premise that we’ve been going with, which is that we’re a couple of optical scientists at Magic Leap who got assigned the job of tracking down some audiovisual glitches appearing in an earlier version of the equipment. The more they investigated these glitches, the more they saw that they weren’t random static, but actually glimpses into a different world. They created a device called the PHILTR, which is the Phase Hopping InterLaminary Transmission Rejector, which is a way of eliminating the crosstalk from these other universes, but it had a switch on it. It could switch from the rejector setting to the resonator setting.

When it operates as the resonator, it’s like a crystal radio that picks up transmissions from these other universes. If you hook it up and plug your Magic Leap device into this thing, which they build into an old tin lunch box, you can see these other worlds. There are two of them. One is called Laminar, and the other is called Old Gnarly. They split off from our timeline — or our timeline split off from them, depending on how you look at it — we split off from Old Gnarly about 1,000 years ago, and it became a version of our world dominated by magic. We split off from Laminar about 100 years ago, and it became an alternate timeline that’s dominated by the kinds of big cool analog technology that we never got. They have the flying cars, the jetpacks, the blasters, but they don’t have digital technology or networks. They have Mars colonies, but they still use paper telephone directories.

We created these two fictional Magic Leap employees and wrote them into the story. The first drop was from 2018. It was a series of YouTube videos in which an agent gets sent to our world from Laminar to investigate this man and this woman in the Seattle office of Magic Leap who are building this PHILTR device. Hijinks ensue. That all culminated in an ARG, the kind of thing that Sean is a world-leading expert in, at the New York Comic-Con in October 2018. While we were there, we had breakfast with a producer from Audible. That’s when we had the idea of creating an audio drama that would tell the next phase of the story. That’s been in the works ever since.

We were all let go from Magic Leap in April of 2020. The entire creative studio part of Magic Leap was disbanded. But that didn’t prevent us from being done, because Audible was going to produce the whole thing anyway. They already had the only thing they needed from us, which was the script, mostly written by Sean and Austin with some contributions from me. They just went ahead and produced the thing. It’s coming out on Thursday.

Sean Stewart: I think it’s safe to say there’s a lot of Neal Stephenson DNA through the world and the script.

GamesBeat: There are pages already up, but it’s not going live until Thursday. Is there another component too? Will Amazon have something, or did I misread that?

Stephenson: Amazon and Audible are joined at the hip. This only exists as an audiobook, though. You can find it through Amazon, but the only way to get it is to buy the audio drama from Audible. You can pre-order now, but you can’t actually get it until Thursday.

GamesBeat: I listened to Anathem, and also to Snow Crash again on Audible. I liked all the music in Snow Crash. Is that an example of what we’re going to see with some of this as well?

Stewart: This is a different deal. This is a full cast. We have 15 cast members. It’s not just being read. It’s literally four Marvel movies with your eyes closed. Sound effects, music, all the characters.

Stephenson: It’s what we used to call a radio play, but you’re not supposed to call it that now. Now it’s an audio drama. This comes from a thing that Audible’s top brass have been wanting to do for a long time, which is to create entertainment that’s written to the form, meaning not just an actor reading a book out loud, but a fully scripted and produced audio drama.

Magic Leap One Creator Edition up close.

Above: Magic Leap One Creator Edition up close.

Image Credit: Dean Takahashi

GamesBeat: Did the rights revert to you guys when the studio disbanded?

Stephenson: They certainly own the rights, yeah. They’re not about consumer-facing entertainment right now. They’ve pivoted strongly toward commercial, industrial types of applications.

GamesBeat: This sounds like a big project, but are you still interested in AR and seeing that come to fruition at some point?

Stephenson: It would be cool. It was made for AR. The IP is calibrated to work well as an AR project, and so it’s definitely doable. We had made some advances toward that here in Seattle. We’d built the beginnings of a location-based experience where you could look out the window of an office suite into Pioneer Square and you could see what Pioneer Square looked like in either the world of Laminar or the world of Old Gnarly and interact with it to some degree. But it’s hard. Making AR content is challenging still. Once you’ve worked on that for a while, making an audio drama or any other kind of standard media seems incredibly easy by comparison.

Stewart: If we could get the IP back or get Magic Leap interested in making the thing themselves, that’s where it all started from. It would be awesome to do. Once you’ve sat in a building in Pioneer Square and watched a zeppelin land and guys with jetpacks come out of it, you want to see that again.

GamesBeat: Sean, from your background, I used to be very familiar with the 42 Entertainment people. What did you bring in from the ARG world to this?

Stewart: There are two things. One is, as Neal said, we had an ARG of modest proportions, and the assets for that are still out there waiting to be discovered by people who have the audiobook. Most people who encounter the audiobook will have no idea that there’s an entire prequel out there waiting to be found in the world. And then second, if you know–I was one of the founders of 42. If you know that work, you know that I was the lead writer on I Love Bees, which was a mere five and a half hour audio drama. We’d been around the block, or at least I had been, writing for this format, which was extremely helpful.

GamesBeat: Is there any of ILoveBees in this, would you say? There are things to find in the real world?

Stephenson: To set expectations, there’s not an active ARG running at this time.

Stewart: Right. But in 2018 there was basically a story that unfurled around three pillars. There was a website, a community forum for a group called the Yarnies, who were fans of a golden age comic book that had been erased from the world. If you know the term the Mandela Effect, where you’re totally certain something had happened, except it never happened, but lots of people agree that it also happened–there’s a famous example of a movie with Shaquille O’Neal from the ‘90s, except there’s no such film. Anyway, all these people earnestly know that they’ve seen these comic books, but no one can prove that they exist.

There’s a second pillar, which is the optical scientist who is upstairs from us at Magic Leap discovering other worlds. When he looks through the device, he recognizes what he sees, because his flaky would-be comic artist brother used to draw these two worlds all the time. You would take those two pillars of the story, and then the third was this set of YouTube videos by this just-out-of-high-school kid who discovers the agent from another world. Slowly those three storylines track together. There were things that people could do in real life. They got pieces of mail and all the stuff you know about.


Above: ilovebees for Halo 2’s launch was an ARG.

Image Credit: Microsoft

GamesBeat: Is that sign, where you guys took the picture, a real place? Or is that a set?

Stewart: That is really Enchanted Rock, Texas. We really did go there to do research. There’s a conceit behind the world, which is that basically every couple of hundred years, a meteorite or comet shower passes, dropping–I believe the technical term Neal uses is “magic space dust.” Dropping magic space dust on our planet, which is used to power the spells in the fantasy and the technology in the sci-fi world, but for us, we’ve just never known it was good for anything. It’s like the Comanche riding horses over giant amounts of oil in Oklahoma. We never made the connection.

Enchanted Rock itself is a giant meteor strike from an event related to this comet passing. Neal, because Neal is Neal, has a program in Mathematica which lists every piece of cometary magic space dust that has landed on earth since 10,000 B.C. You can chart it, because of course he does. That’s the pleasure of working with Neal.

GamesBeat: To be clear again, there’s an audio drama that’s finished. There are pieces of an ARG out there in the world, but not finished? And an unfinished AR application.

Stewart: There is no AR application, and we’re not mounting a current ARG with the bells and whistles.

Stephenson: We’re being nervous, because when enthusiasts for the ARG form get going on one of these, they’re relentless. They’ll blow through every problem you can set in a matter of hours. We don’t want them to wrongly get the idea that there’s one of these currently running and then spin their wheels looking for nonexistent clues.

GamesBeat: But this drama, in the fiction, does have Magic Leap employees in it?

Stewart: Magic Leap is never named. They work at an AR startup. That was always the deal from the beginning, that AR would be part of what we were building for, but we didn’t want anyone to come to the work and feel like it was a very peculiar form of product placement.


Above: In ilovebees, 42 Entertainment made 50,000 pay phones ring at once. Players recorded the calls and put together an hours-long broadcast on the Covenant invasion.

Image Credit: 42 Entertainment

GamesBeat: While you’re being careful here about not creating expectations of an ARG, your fiction is pushing those people in that direction.

Stewart: 100 percent. But fairly tongue in cheek. One of the values of this kind of conversation–if people go out and start looking for extra stuff, we will gently tell them to stand down. We will communicate that there is archival material to find, but there is no new stuff to do.

GamesBeat: It sounds like it must have been hard to extract something from this, when you expected to have all this technology around it to complete it. Was it difficult to finish the story without that technology?

Stephenson: In the particular case of The Long Haul, the audio drama, that was always planned as what it is from the beginning. It doesn’t require any AR stuff at all.

Stewart: It’s its own story, but the original plan would be that it would also–if you think of the first chapter being told as an ARG and the second chapter being told as an audiobook, then the location-based things that Neal was talking about, that would be the third evolution.

GamesBeat: Was it just the three of you that wound up doing this, plus the cast at Audible?

Stewart: In terms of the audio drama, yes, it was the three of us who were writing it, and then Audible produced it with the actors. Neal had a team in Seattle that included people besides the three of us. They were building, for instance, the location-based experience. For that you need engineers and artists.

GamesBeat: It must be satisfying for this part, at least, to be finished.

Stewart: It really is. We worked hard on a lot of things, and it’s nice to see them see the light of day.

GamesBeat: I remember your talk from Magic Leap’s debut. It involved a lot of goats. Are there goats in this thing?

Stephenson: The baby goats project was the brainchild of Karen Laur, who was part of the creative team that came up with this world. But baby goats was an unrelated idea. Why not populate your living room with goats that run around and jump up on tables and are responsive to the environment? In order to build that application, we had to do a lot of fundamental engineering work around–we built a debugger. We built some other utilities for analyzing a room, breaking down the geometry of it, creating pathways. All of that was fundamental building block stuff that needed to be done.

The direction that project took became a demonstration code project. We made that application work at a basic level, and we released several tiers of technology, beginning with the debugger and building on top of that, so that other developers who wanted to create their own applications on the Magic Leap platform could use that sample code as a template to work from. We then used that underlying tool set to build the location-based experience work that we did for the New Found Land IP.

But you may remember, if you saw that presentation that I gave–that would have been October of 2018. I had flown directly there from New York, where we had just finished up the ARG and the activities at New York Comic-Con. If you were following the timeline, the release of the videos and so on, there’s a direct connection to that, and a little statement I made during that presentation. I showed a blurry picture of someone leaving the Magic Leap facility, carrying a lunchbox, a beat-up lunchbox full of electronics. That’s the PHILTR, the fictional device that figures into the story. That was the moment, according to our storyline, when one of our engineers went on the lam and ran off to Idaho with the PHILTR in the back of his car and has never been seen again.

Rony Abovitz is the founder of Sun and Thunder.

Above: Rony Abovitz is the founder of Sun and Thunder and former CEO of Magic Leap.

Image Credit: Sun and Thunder

GamesBeat: It sounds like you made great progress in figuring out how to tell stories in AR.

Stephenson: I’d like to think so, yeah. We did a lot of thinking about what storytelling means in that kind of environment. How to build an interactive experience that would be interactive, and yet reveal things about the world, let’s say.

Stewart: One thing that’s tricky when you’re making this stuff–if you went to film school, they spend years teaching you to have the exact wrong instincts for making AR. You always have control. The three most important people on a movie are all there to do one thing, which is control what you see at every second. The director, the director of photography, and the editor, that’s all they do. When you walk into AR and VR you’re in a different world. The audience can just look at something else.

Rony was talking to Neal one day and he said, “It’s weird. If you think about VR, everyone knows how to make VR, because you just set things in Middle-Earth and you’re in Middle-Earth. But with our technology, it has to be amazing and beautiful and wondrous, and yet it’s happening in your living room. How do you tell that story?” Which is the point where Neal said, “I think I know a guy.” Which is how I came to be at Magic Leap in the first place.

ARGs are a very natural jumping-off point for creating in AR, because in both cases so much is about letting the audience discover, explore, speculate, and find the story and assemble it for themselves. You’re not going to be able to just nail them to a chair and tell them to stare at a screen while you do your thing. It has to be much less of a one way street, and much more of a dance. You say, “I have some steps. If you follow me and listen to the music, we can have some fun together.”

GamesBeat: You’re convincing the user that there’s magic in the real world.

Stewart: That’s always been my goal. When I was young I desperately wanted to go to Middle-Earth, and I couldn’t find a way to do it. I’ve been trying to backwards-engineer that ever since.

Stephenson: There’s another thread that became interesting to us over the course of the project, which is just non-interactive entertainment within AR. Epic and the Unreal Engine have been getting heavily involved in moviemaking. They’ve beefed up the quality of the tool set within Unreal Engine that’s used for sequential entertainment. You create a world and then you program certain movements and events that happen there.

Normally you see that all through a camera, which can also be programmed. But you don’t have to. You can also use Sequencer to make three-dimensional movies, as it were, that a viewer wearing AR or VR equipment could walk around and see from different angles in three dimensions. We also got pretty deep into working with that, because it’s a way you can just tell a story using game engine technology without the added complexity of making it fully interactive.

neal johnny delta cover

Above: Johnny Delta comic book

Image Credit: Sean Stewart

GamesBeat: Just to be clear, when you say there’s no ARG, there’s really no ARG?

Stewart: There’s no ARG. There’s no current ARG. Good question to ask, but no.

GamesBeat: There’s so much fiction here that’s mixed in with the real world, I have to make sure.

Stewart: I understand! Needless to say, it’s been a strange couple of years for those of us who started in the ARG space.

Stephenson: Sean knows what it is to launch an ARG for real and then have to deal with the onslaught of change and heavy interaction and hasty improvisation that goes with that. He’s not one who would idly touch one of those things off.

GamesBeat: It could be that you just don’t know. Maybe someone from the alternate universe took over Magic Leap, and that’s why you got shut down.

Stephenson: It is noteworthy that just as we were getting going on this project in a serious way, suddenly, boom, it was all gone.

Stewart: The website is now an Indonesian travel blog. Do you think that just happens?

GamesBeat: They got to Rony.

Stephenson: He’s a true believer. He’s one of us. He’s been very supportive.

Stewart: Or so we think!


GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *


Nvidia’s Isaac robot simulations debut on Omniverse



Nvidia's Isaac robot simulations debut on Omniverse

Elevate your enterprise data technology and strategy at Transform 2021.

Nvidia has launched a new version of its Isaac robot simulation engine on its Omniverse, which is the company’s metaverse simulation for engineers.

The Omniverse is a virtual tool that allows engineers to collaborate. It was inspired by the science fiction concept of the metaverse, the universe of virtual worlds that are all interconnected, like in novels such as Snow Crash and Ready Player One. The project started years ago as a proprietary Nvidia project called Holodeck, named after the virtual reality simulation in Star Trek.

But it morphed into a more ambitious industry-wide effort based on the plumbing made possible by the Universal Scene Description (USD) technology Pixar developed for making its movies. Nvidia has spent years and hundreds of millions of dollars on the project, and now it’s updating its robot simulations for it.

An open beta

Above: These are simulations of Isaac robots in action.

Image Credit: Nvidia

The new Isaac simulation engine is now in open beta so companies and designers can test how their robots function in a simulated environment before they make the commitment of manufacturing the robots, said Gerard Andrews, senior product marketing manager at Nvidia, in an interview with VentureBeat.

Andrews showed me some images and videos of robots working in a digital factory being created by BMW as a “digital twin.” Once the factory design is done, the digital design will be replicated in the real world as a physical copy. And now the Isaac-based robots will operate more realistically, based on newly available sensors for the robots and more robust simulations.

The simulation not only creates better photorealistic environments but also streamlines synthetic data generation and domain randomization to build ground-truth datasets to train robots in applications from logistics and warehouses to factories of the future.

“Isaac Sim is going into open beta. We’ve had an early adopter program, which has reached thousands of developers in hundreds of individual companies,” Andrews said. “They tried it out and kicked the tires and gave us some good feedback. And we’re proud to take this to the market based on that feedback and a lot of enthusiasm we are seeing from these customers.”

He said Isaac Sim is a realistic simulation, derived from core technologies such as accurate physics, real-time ray tracing, path tracing, and materials that behave like they’re supposed to.

“One of the big problems you have is the sim-to-real gap, where the gap between the virtual world and the real world — if it exceeds a certain amount — then the engineers or developers just won’t use simulation,” Andrews said. “They’ll just abandon it and say is not working.”

Andrews said the Isaac Sim running on Omniverse will be a game-changer in the utility of simulators. And he said the simulation has to be good enough that it’s worth the time it takes to learn how to use the tools for the simulation.

“A lot of the use cases we have around manipulation robots, navigating robots, generating synthetic data to train the AI in those robots — we have those use cases built into Isaacs already,” Andrews said. “And then finally, the big benefit that we get from being a part of the Omniverse platform is seamless connectivity and interoperability with all these other tools that people may be using in their 3D workloads. We can bring those assets into our simulation environment where we’re developing the robot, training the robot, or testing the robot.”

The Omniverse and Isaac

nvidia isaac 3 Dofbot manipulation robot in Isaac Sim

Above: Dofbot manipulation robot in Isaac Sim.

Image Credit: Nvidia

The Omniverse is the underlying foundation for Nvidia’s simulators, including the Isaac platform — which now includes several new features.

Built on the Nvidia Omniverse platform, Isaac Sim is a robotics simulation application and synthetic data generation tool. It allows roboticists to train and test their robots more efficiently by providing a realistic simulation of the robot interacting with compelling environments that can expand coverage beyond what is possible in the real world.

This release of Isaac Sim also adds improved multi-camera support and sensor capabilities, and a PTC OnShape CAD importer to make it easier to bring in 3D assets. These new features will expand the breadth of robots and environments that can be successfully modeled and deployed in every aspect: from design and development of the physical robot, then training the robot, to deploying in a “digital twin” in which the robot is simulated and tested in an accurate and photorealistic virtual environment.

Developers have long seen the benefits of having a powerful simulation environment for testing and training robots. But all too often, the simulators have had shortcomings that limited their adoption. Isaac Sim addresses these drawbacks, Andrews said.

Realistic simulation

nvidia isaac 4

Above: A scene in a BMW digital twin factory.

Image Credit: Nvidia

I was looking at the images of Isaac robots in the press material, and I thought they were photos. But those are 3D-animated images of robots in the Omniverse.

In order to deliver realistic robotics simulations, Isaac Sim leverages the Omniverse platform’s powerful technologies including advanced graphics processing unit (GPU)-enabled physics simulation with PhysX 5, photorealism with real-time ray, and path tracing, and Material Definition Language (MDL) support for physically-based rendering.

Isaac Sim is built to address many of the most common robotics use cases including manipulation, autonomous navigation, and synthetic data generation for training data. Its modular design allows users to easily customize and extend the toolset to accommodate many applications and environments.

“This image is a digital twin of BMWs new factory that their factory planners worked on. They brought it into the Omniverse world. And the cool thing about being in Omniverse is that I can put my simulated robot right in this world, and collect the training data that I’m going to use for my AI models, do my testing, do all sorts of different scenarios. And that’s kind of one of the beauties of being a part of the Omniverse platform,” Anders said. “I’ve been challenged to come up with a catchy phrase, and I ever really come up with a catchy phrase, but it’s something around the realistic robot models and the complex scenes that they’re going to operate in.”

To me, it’s kind of like designing products inside one of Pixar’s film worlds, only one that is far more realistic.

With Omniverse, Isaac Sim benefits from Omniverse Nucleus and Omniverse Connectors, enabling the collaborative building, sharing, and importing of environments and robot models in Pixar’s Universal Scene Description (USD) standard. Engineers can easily connect the robot’s brain to a virtual world through Isaac SDK and ROS/ROS2 interface, fully-featured Python scripting, plugins for importing robot and environment models.

Synthetic Data Generation is an important tool that is increasingly used to train the perception models found in today’s robots. Getting real-world, properly labeled data is a time-consuming and costly endeavor. But in the case of robotics, some of the required training data could be too difficult or dangerous to collect in the real world. This is especially true of robots that must operate in close proximity to humans.

Issac Sim has built-in support for a variety of sensor types that are important in training perception models. These sensors include RGB, depth, bounding boxes, and segmentation, Andrews said.

How realistic should it be?

“You just want to, within reason, close that sim-to-real gap,” Andrews said. “If you have a small error, that can accumulate in your simulation. It can pick up over time, like an error in physics modeling where you don’t do something right with how the wheels [function], then the first time you simulate it, your robot may be fine. But that error builds up and the robot may find itself completely off course in the real world.”

He added, “The closer you can get into the reality, there’s just a better experience you’re going to have when the engineers try to use it. In the world of simulation, you always face this idea of now that I have the real hardware, what’s the value of still using the simulator.”

Getting better data

nvidia isaac 5

Above: Isaac Sim gets into the engineering details for materials.

Image Credit: Nvidia

In the open beta, Nvidia has the ability to output synthetic data in the KITTI format. This data can then be used directly with the Nvidia Transfer Learning Toolkit to enhance model performance with use case-specific data, Andrews said.

Domain Randomization varies the parameters that define a simulated scene, such as the lighting, color and texture of materials in the scene. One of the main objectives of domain randomization is to enhance the training of machine learning (ML) models by exposing the neural network to a wide variety of domain parameters in simulation. This will help the model to generalize well when it encounters real world scenarios. In effect, this technique helps teach models what to ignore.

Isaac Sim supports the randomization of many different attributes that help define a given scene. With these capabilities, the ML engineers can ensure that the synthetic dataset contains sufficient diversity to drive robust model performance.

Simulations can save time and other things

BMW Group is using Omniverse to build a digital factory that will mirror a real-world place.

Above: BMW Group is using Nvidia’s Omniverse to build a digital factory that will mirror a real-world place.

Image Credit: Nvidia

In real life, 50 engineers may be working on a project, but they might have only one hardware prototype. With something like Isaac, all 50 software engineers could work on it at the same time, Andrews said. No longer do all of the engineers have to be in the same place, as they can work on parts of it remotely. And they don’t all have to be in the same physical space.

“I was designing processor cores and people always wanted to simulate it before they had the real hardware, but when their chip came back, the simulator was put on the side,” Andrews said. “In the robotics use case, I still feel like there’s value for the simulator, even when you have hardware because the robots themselves are expensive.”

On top of that, it could be dangerous to test a robot in the real world if its controls aren’t right. It might run into a human. But if you test it in the Omniverse, the simulation won’t hurt anybody.

Over time, Nvidia has added things like multi-camera support, a fisheye camera lens, and other sensors that improve the functions of the robot and its ability to sense the environment. The more components are improved in the real world, the more the Isaac simulation can be updated in the Omniverse, Andrews said.


GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Continue Reading


Intel launches more silicon and software for 5G wireless networks



Intel launches more silicon and software for 5G wireless networks

Elevate your enterprise data technology and strategy at Transform 2021.

Intel made the case today that its silicon chips and software are accelerating 5G wireless networks at the edge, and the big chipmaker is launching new chips to further improve its position in such 5G technologies as virtual radio access networks (vRAN).

Intel VP Dan Rodriguez made the announcements in a keynote speech for the virtual Mobile World Congress event. By 2023, experts expect 75% of data will be created outside of the datacenter — at the edge in factories, hospitals, retail stores, and across cities. Developers want to converge various capabilities at the edge, such as AI, analytics, media, and networking, and Intel wants to be there with the right technology.

In a recent survey of 511 information technology decision-makers, over 78% said they believe 5G technology is crucial to keeping pace with innovation, and nearly 80% said 5G technologies will affect their businesses, Intel reported.

With this in mind, Rodriguez said Reliance Jio, Deutsche Telekom, and Dish Wireless are transforming their networks on Intel architecture. The vRAN promises cloud-like agility and automation capabilities that can help optimize the RAN performance and ultimately improve the experience for users.

Intel is also expanding its family of Agilex FPGA (field programmable gate array), or highly programmable chips. The company is adding a new FPGA with integrated cryptography acceleration that can support MACSec in 5G applications. This adds another layer of security to vRAN at the fronthaul, midhaul, and backhaul levels.

Above: Intel is unveiling new 5G wireless network tech at MWC 2021.

Image Credit: Intel

Intel also said the Intel Ethernet 800 Series family is expanding with the company’s first SyncE capable Ethernet Adapter designed for space-constrained systems on the edge and well-suited for both high-bandwidth 4G and 5G RAN, as well as time- and latency-sensitive applications in industrial, financial, energy, and other sectors.

Intel summed up the tech as its Intel Network Platform, a technology foundation that aims to reduce development complexity, accelerate time to market, and help customers and partners take advantage of features in Intel hardware — from core to access to edge. Intel says its Intel Network Platform includes system-level reference architectures, drivers, and software building blocks that enable rapid development and delivery of Intel-powered network solutions and an easier, faster path to developing and optimizing network software.

Rodriguez said nearly all commercial vRAN deployments are running on Intel technology. In the years ahead, Intel sees global vRAN base station deployments scaling from hundreds to “hundreds of thousands,” and eventually millions.

Why it matters

intel 2018 mwc 2 12

Above: Intel’s Mobile World Congress in 2018.

Intel said operators of 5G networks want a more agile, flexible infrastructure to unleash the full possibilities of 5G and edge as they address increased network demands from more connected devices. At the same time, global digitalization is creating new opportunities to use the potential of 5G, edge, artificial intelligence (AI), and cloud to reshape industries ranging from manufacturing to retail, health care, education, and more.

Decision-makers also revealed that they view edge as one of the top three use cases for 5G in the next two years. With Intel’s portfolio delivering silicon and optimized software solutions, the company can tap into an estimated $65 billion edge silicon opportunity by 2025. Intel technology is already deployed in over 35,000 end customer edge implementations.

Network deployments

Operators like Deutsche Telekom, Dish Wireless, and Reliance Jio are relying on Intel technology. Reliance Jio announced it will participate in co-innovations with Intel in 5G radio and wireless core and collaborate in areas that include AI, cloud, and edge computing, which will help with 5G deployment.

Deutsche Telekom is using Intel FlexRAN technology with accelerators in O-RAN Town, in the O-RAN network it is deploying in  Neubrandenburg, Germany — a city of 65,000 people spread out over 33 square miles. The company is relying on Intel as a technology partner to deliver high-performance RAN at scale.

Dish Wireless is relying on Intel’s contributions to the 5G ecosystem as it builds out the first cloud-native 5G network in the U.S. Its inaugural launch in Las Vegas, as well as its nationwide network, will be deployed on infrastructure powered by Intel technology in the network core, access, and edge.

Cohere is pioneering a new approach to improving spectrum utilization by leveraging capabilities in FlexRAN. It is integrating and optimizing spectrum multiplier software in the RAN intelligent controller. Cohere’s testing shows its Delay Doppler spatial multiplexing technology is improving channel estimation and delivering up to a 2 times improvement in spectrum utilization for operators. That’s what Vodafone has seen in 700Mhz testing in its labs.

And Cellnex Telecom — with support from Intel, Lenovo, and Nearby Computing — is delivering edge capabilities based on Intel Smart Edge Open. This will allow Cellnex to act faster on data, provide service-level management, improve quality of service, and deliver a more consistent experience to its end users. Deployed in Barcelona, this solution will extend to more markets using the blueprint developed with Intel and Nearby Computing.

Intel said its network business grew 20% between 2019 and 2020, from $5 billion to $6 billion. The company’s strong position is the result of early investments in hardware and software.

Intel predicted a bright future for the industry. As 5G blooms to meet its full potential alongside edge computing, experts expect artificial intelligence, the cloud, and smart cities will become the norm. Factory automation is also expected to flourish with Industry 4.0 and retail locations will redesign the shopping experience. And for consumers, cloud gaming and virtual and augmented reality over mobile networks will become an everyday experience, Rodriguez said.


VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Continue Reading


Disney interview: Big games coming with Avatar and Pirates of the Caribbean



Disney interview: Big games coming with Avatar and Pirates of the Caribbean

Elevate your enterprise data technology and strategy at Transform 2021.

Disney had a big week at the Electronic Entertainment Expo (E3) with the announcement of Ubisoft’s new open-world game, Avatar: Frontiers of Pandora. The title has cinematic graphics that replicate the imagery of the movie and the environments of the beautiful moon of Pandora.

Microsoft’s Rare studio also announced that characters from the Pirates of the Caribbean films, like Jack Sparrow and Davy Jones, will be integrated into Sea of Thieves. Both are examples of Disney’s return to triple-A games after changes to its strategy for games over the years.

Disney had triple-A games in the past when it had its own game studios. But it closed down or sold off the studios, and more recently it has been licensing its properties to outside companies, mostly mobile game publishers such as Glu and Jam City. And now it’s clear that Disney has been licensing its properties out for triple-A games as well.

I talked with Sean Shoptaw, senior vice president of Walt Disney Games, and Luigi Priore, vice president of Disney and Pixar Games, about Disney’s presence at E3 and the latest on its strategy for games.

Here’s an edited transcript of our interview.

Above: Pandora looks beautiful as an open world.

Image Credit: Disney

GamesBeat: What’s new for Disney Games?

Sean Shoptaw: I guess that’s a pretty loaded question. There’s a lot going on. It’s been a great week at E3 with some of the announcements you’ve seen. The business is doing well. We’re super excited about the products we’ve announced, and a lot of the products we have in the market already as well. We’re very excited about the status of games at Disney.

GamesBeat: What was announced altogether this week?

Shoptaw: The Avatar title and the Sea of Thieves integration were the two big ones so far.

GamesBeat: How long has Avatar been in the making now?

Priore: That predates us, obviously, because Disney didn’t acquire the 20th Century Fox properties until a couple of years ago. That started well before the acquisition. The great thing moving forward is we’ve been lucky enough to be able to work with Ubisoft and the team at Massive with Lightstorm, James Cameron’s production company, and Jon Landau, who worked on Avatar: Frontiers of Pandora. They announced Massive was working on it a while ago, but this is the name announcement and the first glimpse of what that game is going to be. We’ve gotten very good responses.

GamesBeat: The animation almost feels like it is the movie.

Shoptaw: Yeah, after the trailer, people are finding out that it’s very cinematic. The quality is extremely high. We’re super excited about that title.

GamesBeat: How does that relate to the movie releases, the next Avatar movies? Are they slated for particular dates yet?

Priore: Yeah, the next one is holiday 2022. Sean can get into our general strategy, but on licensing games like this these days — there was a time 15 or 20 years ago where playing the movie was something. You bought the game and played the movie. Things like the classic Aladdin game on Sega Genesis. You played the film. That was popular at the time, but gamers expect more now. They want to interact with their favorite characters and worlds, but they want to play new stories and do new things with those characters and worlds.

On Avatar it’s the same thing. What James Cameron and Jon Landau created is an amazing science fiction world. Pandora is awesome. They have great heroes. It’s a great playground to play in. This is a brand new story with new characters. It’s going to become part of the canon. The whole idea is to have it be part of the storyline of that giant franchise on Pandora, but it’s not a “play the movie” game. It’s an all new open world, new characters. That’s why it’s called Frontiers of Pandora. It takes place on another frontier, another area of the moon of Pandora.

avatar 2

Above: The environs of Pandora.

Image Credit: Disney

GamesBeat: How much will we recognize it? Is it a replication of the movie world, or is it more Ubisoft’s imagining of a new part of the world?

Priore: No, we’re working directly with the filmmakers. Jon Landau is involved almost every day on this. This is the same world. It’s just that you’re going to meet new characters, new clans of Na’vi, and your role is going to be different. I don’t want to go too much into it because we didn’t announce everything yet. But it’s a whole new story with new characters on the same planet, in the same canon. Jedi: Fallen Order was a new story about a new Jedi in the Star Wars canon. It’s the same idea here.

GamesBeat: On your level, how are you involved, compared to Ubisoft’s responsibility?

Priore: Massive is the developer. They’re one of the best in class at open world games. Division, Division II, amazing games. They’re working with the FoxNext team and Lightstorm, working directly with the filmmakers. Where we come in is we’ve brought our expertise in working on IP, working on games. We’ve talked about this a bit. We have a collection of producers, game designers, artists, writers that work together with our partners to get the best out of what they want to do.

Although we just joined this game production recently, since we acquired the 20th Century properties, we’re working directly with Massive and Lightstorm to help them make the best game possible. It’s our job to make sure that Massive has everything they need and that the brand is as authentic as possible working with Lightstorm.

GamesBeat: It still feels like there are so many opportunities for Disney in games. How do you approach which ones to take on, how many of them to do on what platforms?

Shoptaw: There’s no shortage of inbound interest to work with all of our franchises, thankfully. That’s something we’re grateful for. We try to take the approach that — we need to align our partnerships around people’s passions for IP. When we sit down and meet with a developer or publisher about an idea, a lot of that is driven by their passion to go make a specific game with a specific IP. Ideally we’re matching that up with a best-in-class partner. To the point about Massive, about EA, about the partnerships you see now and will see in future, it’s about matching that passion with best-in-class partners to go make what we hope are the best games we’ve made for whatever genre or IP it might be.

That, to us, is the recipe. It’s about working with high-quality partners that have passion for Disney IP, whatever it may be. It gets to be a much easier conversation once you’re in that world, where you see that passion. They have a track record of developing high-quality products. Then it’s about figuring out exactly what the execution is going to be, working closely with Luigi and our other teams internally to map to what ultimately is the final product. But that really is, at the top, our focus, to match people’s passions and the highest quality of partner we can find to go make a certain game.

GamesBeat: There’s a lot more coming than what we’ve seen here at E3, I’m sure.

Shoptaw: As I said yesterday, our slate has never been better. We’ve never been more excited about the slate we have. Some of that’s been announced and some hasn’t. But we feel like we’ve been fortunate to do some exciting partnerships with partners that have a high bar on quality and thankfully have a passion for our IP. We look at our pipeline of product and it’s never been healthier. The quality bar has never been higher.

GamesBeat: Star Wars: Hunters is another one of those coming.

Priore: Very excited about Hunters. Both mobile and Switch, which is very exciting for us. We’ve wanted to get more content on the Switch. We’re excited about what that game represents within the Star Wars universe. We think it’s a unique take, both creatively and from a genre perspective. It’s a very differentiated experience, one we haven’t seen so far in Star Wars.

avatar 3 1

Above: Avatar: Frontiers of Pandora is coming in 2022.

Image Credit: Disney

GamesBeat: Zynga is an interesting choice there. They haven’t done a console game before. When I was talking to them about their Harry Potter game, though, I was pretty stunned by how much work went into that. Several years, the biggest team they ever had. How much they put into all the animation and everything else that keeps players immersed in that universe was very interesting. It wasn’t as much of a surprise to see them do a Star Wars game.

Priore: They came to us with a good idea, with a team that we had a lot of respect for. They have a lot of passion for Star Wars. It made a lot of sense to us as we sat down and mapped out what a game could look like here. You’ll see that passion and quality in the final product. As I said, I think it’s a unique take on Star Wars, and knock on wood, our fans will agree. We’re pretty bullish on that game, excited for the world to see it.

GamesBeat: Is there anything else announced in Star Wars?

Shoptaw: We announced the Massive title as well not too long ago. We’ll do an open world Star Wars game with Massive. Similar to Zynga, we feel like it fits a need within the Star Wars universe that hasn’t been fulfilled, and we felt Massive was a perfect partner to execute on it. We’re huge fans of David [Polfeldt] and the team. We aligned quickly on a vision and an experience for Star Wars that, again, fans and gamers will flock to, hopefully. We feel good about the team making it, and we think the idea behind it is great.

GamesBeat: I take it that it’s just not the time to show a glimpse of that?

Shoptaw: We’re still a little ways off, but at the right time I think people will see why we’re so excited about it. We had Avatar to show this time. We didn’t want to show too much at once. With Star Wars, we’ve seen such a great response to Star Wars recently. Jedi: Fallen Order continues to perform. We just hit the 20 million user milestone recently. That title was another great example of telling a truly original story within that universe, something that hadn’t been told before. Allowing people to go be a Jedi and play a fun game like that has proven to work well and continues to resonate.

We’re not looking to flood the market and put one game on top of another. We want to be disciplined and focused on the best experiences. It’s not about making as many games as we can possibly make. It’s about making the right games with the right partners. When we do that, we see that we’re able to have a good amount of success. We feel fortunate about that. We’ll continue to do things that we think fans and gamers will be excited about with the right partners in the right genres on the right platforms. If we can keep that discipline I think we’ll continue to raise the bar on quality and continue to deliver products that will meet the moment, meet the level of quality that we want.

GamesBeat: What’s the strategy around platforms, especially mobile?

avatar 4 1

Above: The Avatar game has been years in the making.

Image Credit: Disney

Shoptaw: Mobile is a huge market globally. We’re always going to have more mobile products than we have console products, just by the nature of the platform. It’s pretty simple. We want to be where it makes sense for our IP to be, across genres, across markets. That might mean local products like Twisted Wonderland in Japan, which is a very unique, specific take on Disney in a market that is hugely passionate about Disney specifically. That execution is a great example of being very locally focused, an execution we know is going to resonate with a certain market. We certainly have regional looks as well, products that make sense in certain parts of the world. Asia is a good example. And then we have a fair amount of products that are global.

We look at it through a local, regional, and global lens. We want to make sure we match franchises and IP with markets in genres that resonate most powerfully. Twisted Wonderland is an incredible example of a local execution. A lot of our titles, obviously, are global, and they’ve been massive successes across markets. We’ll continue to look at big global opportunities like Galaxy of Heroes with EA. Obviously the Marvel portfolio has had a lot of incredible success across mobile.

We’re not one size fits all. We’ll focus on the right execution in the right market with the right partner and the right genre. We don’t want to flood the market, again, with a bunch of duplicative titles, or just put our brand on any title that we get some interest in. We’re going to be disciplined, and we’re going to make sure we apply that sort of strategic thought to every game we do, regardless of market. That approach over the last few years for us has shown that it works well, and we’ll continue to have that view of the world. It needs to make sense. It needs to be really high quality.

Even if we think we’re missing something, if there’s an opportunity for a genre or a certain IP is underserved, we’re not going to rush and just do a game because we think we need to. We will wait and make the right game with the right partner. That’s as important as getting any games out there. That’s something we’re focused on as much as we are getting products to market and satisfying the demand that we fortunately have for our IP. We’ll continue to be disciplined.

GamesBeat: Did the pandemic change your thinking in any ways?

Shoptaw: No. Fortunately the game industry overall, and certainly our business within Disney, had been doing very well prior to COVID. People’s perception was that video games benefited a lot from people staying home, working from home. There’s certainly some truth to that. But video games have been growing rapidly as an industry prior to COVID. It would have continued to grow rapidly if we never had COVID. So it hasn’t changed any strategic thinking for us. Fortunately our products and releases, nothing was impacted too dramatically by COVID. Again, strategically it hasn’t changed our view of the world.

avatar 5 1

Above: The humans are the enemy in Avatar.

Image Credit: Disney

GamesBeat: It seems like the video game opportunity is a lot more clear than it used to be in the wake of the pandemic. I’ve been writing all these stories about how much more money is coming into the game industry. I think it’s $49 billion in the first five months of this year in terms of investments and acquisitions and public offerings. That compares to $33 billion for all of last year. At the same time I know the movie industry is contracting. Does it make some sense to argue the case for games as a bigger slice of the pie going forward, a bigger opportunity? Is it time to double down on video games?

Shoptaw: We look at games as that pillar, regardless of what the model is. For us we feel like playing in the space where we’re playing gives us the highest quality products that we can scale across the world. When you look at internal development, obviously that comes with a considerable amount of investment and volume to go hit the aspirations that we have in this space. Again, that’s to work and deliver the best products across the world — console, mobile, PC.

Generally there’s no shortage of investment still happening on the linear side. To your point around film, streaming has taken a considerable bite out of that traditional film apple. But the investment in linear content is still extremely material. I don’t think that’s been diminished in any way. From a games perspective, again, our focus has been, and will continue to be, on quality, on being able to scale this business and meet the demand that exists in video games.

We feel like right now, that strategy is to go license and work with the best partners in the world to deliver on that demand. We’ll continue to do that as long as we can meet that bar of quality, of volume, and making sure that our reach is where we need it to be. Again, we’re fortunate to have the IP that we do. We owe it to consumers, fans, and gamers to make sure we’re delivering at that level. That will continue to be our focus.

We’re excited about where this business is and where it’s going. We think it is a pillar, regardless of model. As long as we’re delivering products like we are, games will continue to be a foundational part of the overall entertainment medium. Certainly from a Disney perspective we do that very thoughtfully. We’ve given a lot of attention and focus to it internally. You’re seeing those results in products today, and you’ll continue to see them in the future.

GamesBeat: Can you tell me a little about the Sea of Thieves integration?

Priore: We’re excited. The team at Rare — this goes back to what we were saying about best-in-class partners. They’ve made the best pirate game ever with Sea of Thieves. We’re excited to have A Pirate’s Life, something authentic to Pirates of the Caribbean that’s also authentic to Sea of Thieves. It lines up with what Sean was saying about doing it the right way, making it authentic to what we do at Disney. Just as we said about Avatar or Star Wars, we want to do that all the time, and we feel like we’re having success with that.

I’ve been here a long time. I’ve been at Disney in games for 25 years. I’ve been on the roller coaster, and I’ve never been more excited about the opportunities we have lined up. You’re seeing some of them, whether it’s Massive and Ubisoft with Avatar or Rare and Microsoft with Sea of Thieves and Pirates of the Caribbean. We’re excited about what’s coming next.

GamesBeat: Call of Duty has an interesting funnel these days, where they start with Call of Duty Mobile. They have 500 million people that way. They have Warzone, a free-to-play console and PC game, 100 million players. That feeds into Cold War, a $60 packaged game that sold 40% better than the previous game in the series. It seems like no accident. You widen that funnel and eventually you widen the market for the franchise’s premium games. It seems like only the biggest companies can do that. I don’t know if Disney has looked at that strategy as well, where there’s a purpose to each game in that funnel.

Shoptaw: People’s strategic view of a game and that game’s purpose are going to differ greatly. If you’re developing a game like Call of Duty, that’s a significant franchise and an incredibly successful one. There’s a lot of ways you can continue to funnel users and grow that pie across platforms.

From a Disney perspective it’s obviously different. We’re working with partners to create experiences. Our strategy is, again, to bring as high a quality of product as we can to market. It’s not about platform-building. We’re not doing this vertically, building out platforms and doing things that might be the strategy of a big game developer.

For us, we’re certainly open to playing in a space that creates these multiplatform experiences that drive audiences in meaningful ways across products. It’s something we’d be happy to engage on if that kind of execution made sense for a franchise of ours. But again, our focus is generally tied to working with partners that can go elevate the IP, that can bring it to consumers in new, unique, innovative ways. If that outcome happens, to your initial question, that’s great. But it’s not core to our strategy because we’re not a developer. We don’t think about it through that lens. If they can leverage our IP in a similar way to Call of Duty, sure, we’re happy to engage on that conversation.


GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Continue Reading