Connect with us

Tech

Cloudera partners with Nvidia to expand GPU usage across AI applications

Published

on

Comparison of CPU and GPU workloads

Join GamesBeat Summit 2021 this April 28-29. Register for a free or VIP pass today.


Cloudera and Nvidia announced a collaboration that will allow organizations to use GPUs in more areas across the AI development lifecycle.

Cloudera will integrate its Cloudera Data Platform with Nvidia’s accelerated Apache Spark 3.0 libraries. The integration will make it easier to add machine learning workflows to processes and create architectures without requiring GPU customization. Enterprises will be able to make changes to their data science workflows without having to also update the Nvidia integration manually.

GPUs have shown tremendous promise in enhancing the data science side of AI development, enabling enterprises to run some types of workloads on top of GPUs. However, analytics often involve processes that span multiple teams, forcing enterprises to invest in customizing GPU integrations for those use cases.

Gartner has predicted that creating new architecture patterns that help operationalize data science and ML pipelines will be one of the major trends in 2021.

Benefits to accelerating GPUs

The partnership will allow enterprises to use GPUs across modern data workflows that span data preparation, data science, and analytics tasks. The typical workflow includes many steps including data ingestion, data curation, data pipeline automation, data science exploration, model development, testing, deployment, model monitoring and retraining, and delivery into the business. Cloudera has been busy in making these processes and the handoffs between them much easier over the last year.

The Apache Spark 3.0 libraries are accelerated using Nvidia’s RAPIDS platform, which will dramatically accelerate much of the boring prep work required to bring new machine learning models into production. For example, the US Internal Revenue Service is already seeing a threefold improvement in data science workflows for fraud detection, said Joe Ansaldi, IRS technical branch chief for the Research Applied Analytics & Statistics Division, in a statement.

Speeding up data preparation tasks and training models faster will save on infrastructure costs as well. GPU-accelerated Apache Spark 3 runs natively on CDP and can plug into high performance compute tools, Cloudera said.

Above: Comparing the CPU and GPU powered workflows.

Image Credit: Cloudera

Cloudera’s data portfolio

Cloudera was a trailblazer in the development of data lakes built on top of the Hadoop platform. Cloudera merged with Hortonworks, another Hadoop vendor, in 2018 and combined the technologies into a modern architecture called the Cloudera Data Platform (CDP). At the time, many speculated this spelled the end of Hadoop data warehouses, but Cloudera has continued to innovate and extend Hadoop into a more nimble workflow.

Cloudera added Applied ML Prototypes (AMPs), a framework for packaging AI and ML models for data scientists, to CDP earlier this year. AMPs allow teams to take the guesswork out of ML projects with prebuilt business application templates for specific use cases, and they often run on Nvidia GPU hardware. Cloudera Data Engineering (CDE) streamlines the data engineering and prep work at the start of a project. This solved common problems data engineers face, such as scheduling and orchestration of complex data, troubleshooting and performance tuning tools for data flows, and improving collaboration with analytic and data science teams.

The RAPIDS Accelerator for Apache Spark will be available in CDP Private Cloud this summer. Nvidia and Cloudera will roll out additional accelerated offerings in CDP over time, starting with Accelerated Deep Learning and Machine Learning in CDP Public Cloud in May. “This means that no matter where customers require these GPUs (from on-prem to public cloud, to hybrid cloud and beyond), they’ll be able to leverage best-in-class GPUs out of the box,” said Santiago Giraldo, Cloudera director of product marketing for data engineering and machine learning.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Continue Reading
Advertisement
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech

Warhammer III hands-on — A journey into the Realm of Chaos

Published

on

Warhammer III hands-on -- A journey into the Realm of Chaos

Did you miss GamesBeat Summit 2021? Watch on-demand here! 


Sega Europe’s The Creative Assembly studio showed off a demo of Total War: Warhammer III at a press event, and I got to go hands-on with the game in a battle set in the Realm of Chaos.

Being launched later on this year in partnership with franchise owner Games Workshop, Warhammer III the latest in the Total War series. The franchise has sold more than 34.3 million copies to date. The Total War: Warhammer spinoff is a cataclysmic conflict between demonic powers and the sentinels of the mortal world. I played the first two games, and many others, in the Total War series. This game brings the Warhammer trilogy to its conclusion.

The Creative Assembly has been making Total War strategy games for more than two decades. Most of these have focused on historical wars; until recently, when they’ve expanded into myths such as Total War: Three Kingdoms and fantasy with the Warhammer titles. In a Total War strategy game, you move armies around on a strategic map and fight in a 3D real-time battle when they meet on the battlefield.

In Total War: Warhammer III, each choice the player makes will shape the conflict to come. You’ll explore the mysterious Lands of the East to the demon-infested Realms of Chaos.

“Warhammer III is of course the concluding chapter in the series and we’re planning on going out with a bang,” said Al Bickham, the development communications manager for The Creative Assembly, at a press event. “We’ve crafted a huge arching narrative which ties the trilogy together. There are going to be more playable races out of the box than the previous two games. And it’s all set across a hyper-detailed campaign map which begins at the very fringes of Warhammer lands and takes you deep into the mind-bending horrors of the four Realms of Chaos.”

The game will have iconic races from the World of Warhammer Fantasy Battles, including the video game debut of Kislev and Cathay alongside the factions of Chaos — Khorne, Nurgle, Slaanesh, and Tzeentch. This means players will wage war with the most diverse array of legendary heroes, gargantuan monsters, flying creatures, and magical powers.

Embarking on a new grand campaign, you will be tasked with saving or exploiting the power of a dying god. Each race offers a unique journey through the nightmarish Chaos Realm. The endgame will determine the fate of the world.

The Survival Battle

Above: Everything looks so orderly at the beginning of the Survival Battle in Warhammer III.

Image Credit: Sega/Creative Assembly

The Creative Assembly used the Parsec to let me play a sample Survival Battle, where your goal is to attack into the Realm of Chaos and take objectives and fend off the demon hordes. It’s a new kind of narrated battle that is fresh to the franchise. They’re like boss battles in Warhammer III, and they trigger after you reach key points in the game’s narrative.

“We want the [Survival Battle] to feel epic, really memorable, and full of decisive moments in the course of your campaign,” Bickham said.

My faction was the Kislev, an Eastern human faction that resembles the Russian Cossacks. And I had to take a number of victory locations within the a bloody fortress called the Brass Citadel.

The faction leader, Tzarina Katarin (the Ice Queen of Kislev) has taken her loyal forces into the Realm of Chaos. Khorne, the Chaos God of rage and war, sends a legion of demons to destroy the trespassers. The Kislev forces have been detailed for the first time in the series. Katarin is an Ice Witch with magical powers to both rally her troops and strike fear in the hearts of demons.

I wasn’t exactly impressed with the forces I got in the battle. There were some excellent sword troops, but I only have five companies of them in a place where I had to defend against attacks coming from all directions. I had twice as many archers and a few archer cavalry units.

The Realm of Chaos, of course, is a bad place. It has plenty of blood-red backdrops and one of its decorations is an actual fountain of blood. The four Ruinous Powers rule over this place, ever seeking to slip their bonds and engulf the world in a tide of daemonic corruption. Nurgle, the plague god; Slaanesh, the lord of excess; Tzeentch, the changer of ways; and Khorne, the god of blood and slaughter.

My troops had to fight uphill and sweep some light demon units from the top of a ridge. That was easy enough, and I claimed a victory point in doing so. That allowed me to draw reinforcements from another realm to strengthen my army. But then I was attacked from four directions. At least I was defending a hill, but I had a hard time figuring out where to place my five sword troops, as they were the best units to stave off attacks.

chaos 5

Above: My soldiers are devolving into chaos in Warhammer III.

Image Credit: Sega/Creative Assembly

The cavalry was useful in taking down wolf-borne demons from the enemy, but it wasn’t useful in charging headlong into enemy lines. Rather, it was better to use them to harass the enemy with missile fire from a distance. But I didn’t have nearly enough units to form a full line of defense in all directions. The result was, you guessed it, chaos.

But I tried to survive. One of the goals was to earn a battle currency called “supplies,” which allowed me to build towers and barricades. It also let me recruit new warriors, upgrade my existing units, and bring on reinforcements. Being new to the game, I couldn’t figure out how much to spend on each kind of task. I found I could build barricades and get reinforcements, but I didn’t have enough supplies to build towers, and that meant the hordes of Chaos were going to charge me without being harassed. You generate more supplies by capturing victory points or killing enemies.

Had I looked more, I would have seen that I could have used The Lore of Ice, or ice-themed spells that would slow down the enemy and help my soldiers thin their ranks as they tried to attack. There were six different spells altogether. I also could have used the Elemental Bear, a huge monster on my side, and some of the bear cavalry for the faction. Sadly they were nowhere to be found in my playthrough.

Still, after a few battle restarts, I was able to survive the first wave of attacks and open up a new part of the Brass Citadel, which was circular with a big pit in the middle. Once again, I was forced to divide my forces and try to hold off larger numbers of enemies coming from all sides. It wasn’t pretty.

I didn’t get near the goal of the battle, to fight Khorne’s champion, an Exalted Greater Demon, in a final struggle. It was a very difficult battle, but I enjoyed the idea of being assaulted by endless hordes and figuring out how to stay alive when you’re vastly outnumbered. This is a difficult mode when it comes to figuring out where to throw your troops and when. But it adds some excitement to the pressure that you feel when you have to make decisions quickly to head off disaster.

GamesBeat

GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Continue Reading

Tech

LinkedIn open-sources Greykite, a library for time series forecasting

Published

on

Greykite Silverkite

Join Transform 2021 this July 12-16. Register for the AI event of the year.


LinkedIn today open-sourced Greykite, a Python library for long- and short-term predictive analytics. Greykite’s main algorithm, Silverkite, delivers automated forecasting, which LinkedIn says it uses for resource planning, performance management, optimization, and ecosystem insight generation.

For enterprises using predictive models to forecast consumer behavior, data drift was a major challenge in 2020 due to never-before-seen circumstances related to the pandemic. This being the case, accurate knowledge about the future remains helpful to any business. Automation, which enables reproducibility, may improve accuracy and can be consumed by algorithms downstream to make decisions.

For example, LinkedIn says that Silverkite improved revenue forecasts for 1-day ahead and 7-day ahead, as well as Weekly Active User forecasts for 2-week ahead. Median absolute percent error for revenue and Weekly Active User forecasts grew by more than 50% and 30%, respectively.

Greykite library

Greykite provides time series tools for trends, seasonality, holidays, and more so that users can fit the AI models of their choice. The library provides exploratory plots and templates for tuning, which define regressors based on data characteristics and forecast requirements like hourly short-term forecast and daily long-term forecast. Tuning knobs provided by the templates reduce the search to find a satisfactory forecast. And the Greykite library has flexibility to customize a model template for algorithms, letting users label (and specify whether to ignore or adjust) known anomalies.

Greykite, which provides outlier detection, can also select the optimal model from multiple candidates using past performance data. Instead of tuning each forecast separately, users can define a set of candidate forecast configurations that capture different types of patterns. Lastly, the library provides a summary that can be used to assess the effect of individual data points. For example, Greykite can check the magnitude of a holiday, see how much a changepoint affected the trend, or show how a certain feature might be beneficial to a model.

With Greykite, a “next 7-day” forecast trained on over 8 years of daily data takes only a few seconds to produce forecasts. LinkedIn says that its whole pipeline, including automatic changepoint detection, cross-validation, backtest, and evaluation, completes in under 45 seconds.

“The Greykite library provides a fast, accurate, and highly customizable algorithm — Silverkite — for forecasting. Greykite also provides intuitive tuning options and diagnostics for model interpretation. It is extensible to multiple algorithms, and facilitates benchmarking them through a single interface,” the LinkedIn research team wrote in a blog post. “We have successfully applied Greykite at LinkedIn for multiple business and infrastructure metrics use cases.”

The Greykite library is available on GitHub and PyPI, and it joins the many other tools LinkedIn has open-sourced to date. They include Iris, for managing website outages; PalDB, a low-key value store for handling side data; Ambry, an object store for media files; GDMix, a framework for training AI personalization models; LiFT, a toolkit to measure AI model fairness; and Dagli, a machine learning library for Java.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Continue Reading

Tech

Legionfarm raises $5.9M to connect pro gamers with wannabees

Published

on

Legionfarm raises $5.9M to connect pro gamers with wannabees

Did you miss GamesBeat Summit 2021? Watch on-demand here! 


Legionfarm has raised $5.9 million for its service to connect ordinary gamers pro gamers around the world. The idea is to help spread the skills to the wannabees who would love to get tips on how to get better. So rather than filling that last squad spot with a random player with no mic, you could get someone who may actually contribute to a win.

The money came from SVB, Y Combinator, Scrum VC, Altair Capital, Kevin Lin (Twitch), Ankur Nagpal (Teachable), and others.

The San Francisco company employs almost a thousand pro gamers, who make real money as mercenaries in games like Call of Duty: Warzone, Apex Legends, Destiny 2, World of Warcraft, and The Division. I could definitely use some help getting more wins by teaming up with the pros in Warzone.

Above: Legionfarm has 80 employees.

Image Credit: Legionfarm

In an interview, founder Alex Beliankin said the onboarding process for pros is highly automated, so skilled players are able to quickly and easily monetize their talents. Experienced players command up to $17 per hour, and may operate as little or as much as desired.

“We let gamers pay to play together with the pro players, helping them have more fun in games and to find a good teammates,” Beliankin said. “We mainly operate in battle royale games as well as massively multiplayer online games. It’s really a more entertaining way to play a game.”

Founded in 2016, Legionfarm previously raised $1.5 million in 2019 from TMT Investments and Denis Smetnev (Vimbox). Altogether, the company has 80 full-time employees, not counting the active pro players. The development team is in Russia.

“The pro gamer is working full time as if that was their job. And their job is to be a good teammate,” Beliankin said. “It’s very important to match players’ personalities.”

GamesBeat

GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Continue Reading

Trending