IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing at the Edge
•
1 like•575 views
This presentation was delivered as the closing keynote for the 2015 IoT Slam virtual conference. During the presentation, Ryft VP of Engineering, Pat McGarry, took a close look at how the IoT revolution is changing data analytics and driving the move of data analysis to the network’s edge where the data is being created. - See more at: http://www.ryft.com/blog/2015-iot-slam-keynote-harnessing-flood-of-iot-data-with-heterogenenous-computing-at-the-edge#sthash.x1Anoapb.dpuf
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing at the Edge
Similar to IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing at the Edge (20)
Recently uploaded
Recently uploaded (20)
IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing at the Edge
- 1. Pat McGarry Ryft Systems, Inc. Closing Keynote Harnessing the Flood of IoT Data With Heterogeneous Computing at the Edge
- 2. SOURCE: IDC WORLDWIDE IoT TAXONOMY, 2015, WORLDWIDE INTERNET OF THINGS FORECAST, 2015–2020 The Internet of Things (IoT) will trigger the single biggest IT shift since the Internet. 4 25+ 50 50 $1.7+ BILLION BILLION BILLION TRILLION TRILLION Connected Applications Devices GBs Data Market People
- 3. Real-time insights as events occur, close to the source of data Analysis of data from a range of IoT devices— video, mobile, batch stores, etc.—together Ultra small & efficient analytics infrastructure Easy to deploy, use & maintain systems Low operational costs No security or performance trade-offs IoT is exacerbating the widening data analytics technology divide. REQUIREMENTS Persistent compute/IO/storage bottlenecks Data analyzed in silos Data movement & ETL delays Sprawling inefficient analytics infrastructures Persistent data privacy & security issues REALITY
- 4. The reason? Actionable intelligence from IoT is trapped in analysis platforms built on 70-year old architectures.
- 5. Heterogeneous Computing is the answer… SOURCES: BLOOMBERG BUSINESS, THE PLATFORM Heterogeneous computing refers to systems that use more than one kind of processor or cores. These systems gain performance or energy efficiency not just by adding the same type of processors, but by adding dissimilar processors, usually incorporating specialized processing capabilities to handle particular tasks.
- 6. …because optimal performance & efficiency demands the right “engine” for the job. CPUs FPGA • General purpose computing • Sequential in nature • Non-deterministic performance • Interrupts • Memory allocation • Problems are broken into a sequence of operations and processed serially • Not general purpose • Purpose built algorithms • Can be reprogramed via firmware • Best at data-heavy analysis • Search, fuzzy search, image and video analysis, deep learning • Inherently massively parallel • Can execute many hardware-parallel operations in one clock cycle • More output with less power • Can complete the same problem at 100X the performance of x86 GPUs • Some general purpose computing • Can excel at certain complex algorithms • Best for image rendering, some image analysis • Generally more parallel than CPUs, since GPUs have more cores • Generally more power efficient than CPU
- 7. Performance CPU FPGAGPU Open API CPU FPGAGPU ….but we still need an open, easy-to-use approach with a business-centric, compute-agnostic open API.
- 8. PerformanceCompute-Agnostic Open API example in C:
- 9. PerformanceNon-Programmatic Interfaces are just as simple: $ ryftprim -n 1 -p fhs -f input.txt -od results.txt -q "(RAW_TEXT CONTAINS "Michelle")" -w 16 -d 2 -oi index.txt -v
- 10. Open APIs must become prevalent, for all analytics.
- 11. Real-time analysis of the range of data types—video, image, & text— whether IoT-generated or traditionally generated, required to shape a problem or experience at the edge was not possible Behavioral data could not be analyzed along with traditional data Data movement, ETL & indexing bogged down the network & slowed analysis Localized analysis consumed valuable space & resources E DG E AN ALY S I S Make informed business decisions by analyzing all your data, immediately.
- 12. Instant analysis of on-location video, text, & sensor data together as it happens allows unprecedented personalization Immediate insight into newly arriving IoT data correlated to legacy data Data can be analyzed locally to increase responsiveness & reduce network load No data movement or ETL = no barriers to real-time insights Small, easy-to-maintain infrastructure brings powerful analytics to the network edge at a fraction of the space, maintenance & energy Solve business problems using business-centric, open APIs instead of developing complex computing algorithms Heterogeneous computing and open APIs eliminate edge computing bottlenecks. T HE S O L UT I O N
- 13. Enabling the Internet of Things with real-time insights into massive data volume & velocity at the edge. Ultra-high-speed, compact, high-efficiency, & heterogeneous Ryft ONE accelerates the delivery of valuable insights from large, dynamic, & diverse data sets at its source exclusively using open APIs. W W W. RY F T. CO M
- 14. Questions? Visit Ryft’s IoT Slam virtual exhibit. Download white papers, analyst reports, & videos. Pat McGarry pat.mcgarry@ryft.com www.ryft.com
Editor's Notes
- Hi! I’d like to thank everyone for joining IoT Slam’15 today. My name is Pat McGarry, and I’m the Vice President of Engineering at Ryft. At Ryft, we have a rich history dating back over 15 years, developing easy-to-use innovative edge-related hardware-based massive scale data analysis solutions for government and military use. We’ve recently leveraged our experience in those areas to solve a wide variety of IoT/edge analytics problems for the commercial space as well, ranging from the corridors of Wall Street, to the aisles of retail stores and online storefronts, and even to the far-flung reaches of cyber security analysis.
- We are in the midst of the single biggest shift since the emergence of the Internet. As with any major market disruption, there were be casualties as well as huge winners. This time, the shift will happen faster, with little time for companies who aren’t prepared to catch up to the early innovators who are already building the backbone for IoT fueled business. By 2020, we will see the results of this explosion: 4 BN connected people, 25+ BN applications, 50 BN devices. All generating 50 TN– yes TN GBs of Data. Are you prepared to make sense of this deluge? Chances are the answer is no, because of a few seemingly insurmountable obstacles we are going to discuss today. Companies that want to take advantage of IoT need solutions to manage data and interact with customers. In addition to taking a hard look at those insights, I’m going to show you how a handful of elite brands are taking an advanced, very innovative approach to getting fast insights from IoT, so they don’t get left behind. After all, we did see a number of entire industry’s left in the dust in the last big IT shift – remember the Music industry? Television? Travel agencies and now Hotels, Taxis, etc..All of these industries are suffering due to the on-demand economy that was created with the shift to the Internet and mobile devices. How will the IoT impact your organization? Will you miss the next big wave of value, your opportunity to capture your rightful piece of the $1.7 trillion of revenue generated by IoT in 2020? Or will you begin experimenting now with mining IoT to deliver new services, inform faster decisions, streamline your operations, and reduce costs?
- The seemingly impossible challenges associated with edge computing are forcing organizations to rethink data search and analysis tools in an attempt to instantly extract meaningful intelligence from a broad range of diverse IoT data. IoT REALITY
- The root cause of “the reality” is that we’re still reliant on 70-year old von Neumann computing architectures. It really has been 70 years since von Neumann’s paper in 1945! Actionable intelligence is key to businesses’ growth. People, machines, and devices are creating high-value data at an ever accelerating rate. IoT data in particular loses value when it can’t be analyzed fast enough and that means taking real-time analytics closer to the source of data at the network edge. The value of this data decays rapidly, and those who can move fast enough to capture and process it into insights have a significant competitive advantage. Those that can’t are deaf to the signals alerting their business to opportunities and threats—despite substantial investments in big data technology. Data value is trapped as data volume, velocity & variety overwhelms existing CPU-based architectures x86 architecture is the root cause of many bottlenecks in processing the volume and variety of data that companies collect, Including: Sluggish data analysis because of limited sequential processing power The need to move data to a centralized location and conduct ETL/indexing Lengthy process of uploading data to the cloud Complex and rapidly growing human- generated data Expensive data indexing operations Sprawling CPU clusters with complicated management and programming Relatively slow networking speeds The interaction of complex software
- Heterogeneous computing refers to systems that use more than one kind of processor or cores. These systems gain performance or energy efficiency not just by adding the same type of processors, but by adding dissimilar processors, usually incorporating specialized processing capabilities to handle particular tasks. Intel is the world’s best known silicon/CPU manufacturer and provides the vast majority of CPUs that drive computations in our data centers. But even they recently realized the importance of heterogeneous computing, as they acquired a leading FPGA vendor (Altera) earlier this year. Microsoft is the world’s best-known software company, powering most PCs throughout the world with its ubiquitous Windows operating system and the business tools that live alongside it such as Microsoft Office. But even Microsoft sees the importance of heterogeneous computing – their Bing search engine is now powered with FPGA technology, and they’ve also recently announced that they believe FPGA technology can surpass GPU technology in performance (and at even less power) for a variety of image analysis algorithms, such as facial recognition. And this is happening not a moment too soon: The IoT/edge problem demands these types of heterogeneous computing platforms!
- … it’s all about using the right tool, for the right job, at the right time.
- If you take nothing else away from this presentation, take away the importance of open APIs as they apply to the edge computing problem. With the necessary advent of true heterogeneous computing platforms – and diverse ones, at that – open APIs become the glue that reduces analysis time (including R&D and setup time), decreases cost, reduces churn, thereby greatly improving overall productivity from the time the data is generated to the time that a business decision can be made (we call this MTTD: mean time to decision).
- Managing a heterogeneous computing infrastructure can be SCARY, but it doesn’t have to be! One of the complexities associated with hybrid computing is how to manage disparate computing resources (CPU, GPU, FPGA, etc.) This is an issue because standard programming languages such as C, Java, and so on work well on CPU fabrics, but not as well on GPU fabrics. Similarly, GPU-intended languages like CUDA and OpenCL require different programming skills. And traditionally, FPGAs often require very skilled hardware designers to achieve high performance. These complexities have stifled the innovation, and there must to be a better way to manage multiple architectures. The best way to ‘solve’ the problem of different computational models is to solve the business problem, instead of the rudimentary low-level computer science problems behind the business problem. The result: an open API that is compute-agnostic that solves a problem. This allows for a single function call to perform the analytics, which enables the implementation to be completely vendor-independent, and compute-methodology-independent.
- … but it isn’t all about programming. There have to be open non-programmatic interfaces as well. These might be web-based, command-line based, cluster-based (such as Spark or perhaps even Hadoop), and so on.
- At Ryft, we’ve open-sourced our API, and are adding to it over time, as we support new business operations. But it’s really important to note that we aren’t the only ones preaching this: without necessarily realizing it, there are many good examples of people attempting to make the jump to higher-level functions, such as OpenCV (the open computer vision project). However, it’s not just about a stand-alone compute-platform independent API, it’s also about making sure whatever computational architecture you’re using can fit into existing architecture. A business-centric open API helps, but it isn’t sufficient by itself. Wrappers to cluster technologies like Apache Spark are every bit as important. Put another way: make sure you have ubiquitous Linux running on the front end of any environment to simplify cluster and tools integration. Conclusion: Hybrid computing isn’t just about using a single technology like a CPU or a GPU or an FPGA. It’s about being able to have a toolbox with all of your tools in it, be they CPUs, GPUs, FPGAs, or even ASICs, allowing you to use using any compute technology available, to allow you to use the right tool at the right time for the right job. One of the key considerations that has been neglected in the young hybrid computing space is how to tie these things together. Open APIs running on top of standard Linux are one way of approaching the problem, where the APIs focus on business decision needs, and not low-level programming needs.
- The IoT data explosion drives the need for edge computing – and not just for the IoT data, but traditional data (such as perhaps log files, historical databases, and so on), to allow for correlation across disparate data sets. First, there is the range of data types – be they video, image, text, and so on. Each of these types traditionally required very different systems and very complex interactions between those systems to achieve a business need, if that were even possible. Second, assuming this data could be collected and analyzed, it still wasn’t possible without large clustered architectures to compare and contrast the new data with older historical data. To do that, most often, you had to move data from one location to a physically different location containing more beefy computing equipment (such as a cluster, perhaps Hadoop, Spark, or something homegrown), traversing a (potentially slow) network. Although some extract/transform/load work has occurred at the edge, more often than not it needs to happen where the more beefy cluster computing exists, which means even more data needs to traverse the already-slow network. Indexing that data to allow for it to be searched becomes another important problem, as indexing steps can take many minutes, hours or even days. By then, the business decision requirement is likely an afterthought. A great example of this is an in-store experience for a customer, whether they are at a brick-and-mortar retail store or shopping via an online storefront.
- Consider, for example, the same retail consideration where a customer is in an aisle in a brick-and-mortar store (or, similarly, on a particular page on an online storefront) – we know we are in that aisle now, but what did they buy the last time they were in that aisle? When did they buy it? Were they just “window” shopping or looking for something in particular? Can I target them right now with an in-store or on-screen coupon to coax them into a buy decision? Are they going back and forth in that same aisle looking for something in particular, getting frustrated by not being able to find what they need? And so on. Analytics at the edge, when coupled with local sensors, provide an ability to answer all of these questions immediately, in automated fashion, to improve the customer experience and improve bottom line profits. Further examples of similar problems come up when analyzing financial transaction data (where newly generated data is happening constantly, and must be correlated against legacy financial data), and the cyber problem, where IoT sensors and devices (including the now-ubiquitous smartphone) generate huge amounts of data and contain an inordinate amount of personally identifiable information. The analysis problem is practically the same, though: you must analyze behavior and the types and content of newly generated data, correlating that to historical data and historical patterns. And due to the sheer volume of generated data, you have to do it quickly, at the edge of the network (perhaps at a cell site, for example), since moving all of the newly generated data to a central location is too expensive in terms of network bandwidth, network speed, ETL & data indexing, compute platform size, power, and maintenance requirements. Perhaps most interesting is that this slide and the previous slide are use case agnostic: the same sets of problems permeate all of the IoT/edge use cases that Ryft has come across. That’s good news, because it means we now know (we didn’t before) that we can create a heterogeneous computing platform and utilize open APIs to solve a wide range of IoT/edge analysis problems.
- At Ryft, we’ve created the Ryft ONE, which is the world’s first heterogeneous computing platform for the edge, analyzing combinations of newly arriving IoT generated data simultaneously alongside historical data at rates as high as 10 gigabytes per second, all in a compact 1U form factor (that’s the same size as a typical rackable 1.75” high data center server). Ryft’s innovative heterogeneous architecture utilizes an x86 CPU frontend running standard Ubuntu 14.04 LTS Linux with an FPGA-enabled backend. The FPGA backend is fully abstracted away for business applications with open APIs, so that the end user doesn’t even have to know how to spell FPGA. They solve business problems, in an interface of their choice, be that programmatic (C, Java, Python, R, Scala, etc.), via a linux command line, shell scripting, web-based access (via an open REST API), native Spark cluster integration, or even using an ODBC connector. For the first time ever, the Ryft ONE provides a full-blown toolbox, allowing you to use the right tool at the right time for the right job – finally enabling true analytics at the edge, which is what is demanded in today’s IoT world.
- I’d like to thank all of you for attending, and if you’ve got further interest in what we’re doing at Ryft relating to heterogeneous computing and open APIs at the edge, I invite you to stop by our virtual trade booth/exhibit for additional educational resources, brochures and video demonstrations. And if you have any questions, I’d be happy to take them now. Thank you!