Two hedge funds are gaining access to university supercomputers through cloud networks to power their trading strategies. Stony Brook University has created a "research cloud" allowing funds to use its IBM Blue Gene supercomputers. One fund will use the machines to design trading algorithms, while another aims to spot market anomalies. Separately, a company is renting time on Rensselaer Polytechnic Institute's Blue Gene supercomputer to adapt gene analysis software for analyzing financial markets.
Big data is a term that describes a large or complex
data volume. That data volume can be processes using traditional
data processing software or techniques that are insufficient to deal
with them. But big data is often noisy, heterogeneous, irrelevant
and untrustworthy. As the speed of information growth exceeds
Moore’s Law at the beginning of this new century, excessive data
is making great troubles to human beings. However this data with
special attributes can’t be managed and processed by the current
traditional software system, which become a real problem. In this
paper was discussed some big data challenges and problems that
are faced by organizations. These challenges may relate
heterogeneity, scale, timelines, privacy and human collaboration.
Survey method was used as a theoretical solution framework.
Survey method consists of a questionnaires report. Questionnaires
report consists of all challenges and problems faced by
organizations. After knowing the problem and challenges of
organizations, a solution was given to organization to solve big
data challenges.
With many organisations considering getting on the Hadoop bandwagon, this document provides an overview of the planned use cases for Hadoop, an illustration of some of the common technology components, suggestions on when Hadoop is worth considering, some the challenges organisations are experiencing, cost considerations and finally, how an organisation should position for a Big Data initiative. Any organisation considering a Big Data initiative with Hadoop should thoroughly consider each of these areas before embarking on a course of action.
Data Science Innovations : Democratisation of Data and Data Science suresh sood
Data Science Innovations : Democratisation of Data and Data Science covers the opportunity of citizen data science lying at the convergence of natural language generation and discoveries in data made by the professions, not data scientists.
Big data is a term that describes a large or complex
data volume. That data volume can be processes using traditional
data processing software or techniques that are insufficient to deal
with them. But big data is often noisy, heterogeneous, irrelevant
and untrustworthy. As the speed of information growth exceeds
Moore’s Law at the beginning of this new century, excessive data
is making great troubles to human beings. However this data with
special attributes can’t be managed and processed by the current
traditional software system, which become a real problem. In this
paper was discussed some big data challenges and problems that
are faced by organizations. These challenges may relate
heterogeneity, scale, timelines, privacy and human collaboration.
Survey method was used as a theoretical solution framework.
Survey method consists of a questionnaires report. Questionnaires
report consists of all challenges and problems faced by
organizations. After knowing the problem and challenges of
organizations, a solution was given to organization to solve big
data challenges.
With many organisations considering getting on the Hadoop bandwagon, this document provides an overview of the planned use cases for Hadoop, an illustration of some of the common technology components, suggestions on when Hadoop is worth considering, some the challenges organisations are experiencing, cost considerations and finally, how an organisation should position for a Big Data initiative. Any organisation considering a Big Data initiative with Hadoop should thoroughly consider each of these areas before embarking on a course of action.
Data Science Innovations : Democratisation of Data and Data Science suresh sood
Data Science Innovations : Democratisation of Data and Data Science covers the opportunity of citizen data science lying at the convergence of natural language generation and discoveries in data made by the professions, not data scientists.
Efficient Data Filtering Algorithm for Big Data Technology in Telecommunicati...Onyebuchi nosiri
Efficient data filtering algorithm for Big Data technology Telecommunication is a concept aimed at effectively filtering desired information for preventive purposes, the challenges posed by unprecedented rise in volume, variety and velocity of information has necessitated the need for exploring various methods Big Data which is simply a data sets that are so large and complex that traditional data processing tools and technologies cannot cope with is been considered. The process of examining such data to uncover hidden patterns in them was evolved, this was achieved by coming up with an Algorithm comprising of various stages like Artificial neural Network, Backtracking Algorithm, Depth First Search, Branch and Bound and dynamic programming and error check. The algorithm developed gave rise to the flowchart, with each line of block representing a sub-algorithm.
In this paper we have penetrate an era of Big Data. Through better analysis of the large volumes of data that are becoming available, there is the potential for making faster advances in many scientific disciplines and improving the profitability and success of many enterprises. However, many technical challenges described in this paper must be addressed before this potential can be realized fully. The challenges include not just the obvious issues of scale, but also heterogeneity, lack of structure, error-handling, privacy, timeliness, provenance, and visualization, at all stages of the analysis pipeline from data acquisition to result interpretation.
In this presentation, I have talked about Big Data and its importance in brief. I have included the very basics of Data Science and its importance in the present day, through a case study. You can also get an idea about who a data scientist is and what all tasks he performs. A few applications of data science have been illustrated in the end.
The current challenges and opportunities of big data and analytics in emergen...IBM Analytics
Big data and analytics present many possibilities for emergency management specialists and first responders. Some of these benefits include pinpointing vulnerabilities, bringing in the right resources and maximizing existing resources to pave the way to adoption. However, these opportunities are not without challenges. Emergency management experts Adam Crowe, Director, Emergency Preparedness at Virginia Commonwealth University; William Moorhead, President of All Clear Emergency Management Group; and Gary Nestler, Associate Partner and Global Leader, Emergency Management solutions at IBM discuss these challenges and opportunities in this slideshare—which is intended to help disaster management stakeholders achieve the most accurate situational awareness using analytics.
Discover analytics solutions for emergency management http://ibm.co/emergencymgmt
Keynote talk by David Dietrich, EMC Education Services at ICCBDA 2013 : International Conference on Cloud and Big Data Analytics
http://twitter.com/imdaviddietrich
http://infocus.emc.com/author/david_dietrich/
What is the impact of Big Data on Analytics from a Data Science perspective.
Presented at the Big Data and Analytics Summit 2014, Nasscom by Mamatha Upadhyaya.
Efficient Data Filtering Algorithm for Big Data Technology in Telecommunicati...Onyebuchi nosiri
Efficient data filtering algorithm for Big Data technology Telecommunication is a concept aimed at effectively filtering desired information for preventive purposes, the challenges posed by unprecedented rise in volume, variety and velocity of information has necessitated the need for exploring various methods Big Data which is simply a data sets that are so large and complex that traditional data processing tools and technologies cannot cope with is been considered. The process of examining such data to uncover hidden patterns in them was evolved, this was achieved by coming up with an Algorithm comprising of various stages like Artificial neural Network, Backtracking Algorithm, Depth First Search, Branch and Bound and dynamic programming and error check. The algorithm developed gave rise to the flowchart, with each line of block representing a sub-algorithm.
In this paper we have penetrate an era of Big Data. Through better analysis of the large volumes of data that are becoming available, there is the potential for making faster advances in many scientific disciplines and improving the profitability and success of many enterprises. However, many technical challenges described in this paper must be addressed before this potential can be realized fully. The challenges include not just the obvious issues of scale, but also heterogeneity, lack of structure, error-handling, privacy, timeliness, provenance, and visualization, at all stages of the analysis pipeline from data acquisition to result interpretation.
In this presentation, I have talked about Big Data and its importance in brief. I have included the very basics of Data Science and its importance in the present day, through a case study. You can also get an idea about who a data scientist is and what all tasks he performs. A few applications of data science have been illustrated in the end.
The current challenges and opportunities of big data and analytics in emergen...IBM Analytics
Big data and analytics present many possibilities for emergency management specialists and first responders. Some of these benefits include pinpointing vulnerabilities, bringing in the right resources and maximizing existing resources to pave the way to adoption. However, these opportunities are not without challenges. Emergency management experts Adam Crowe, Director, Emergency Preparedness at Virginia Commonwealth University; William Moorhead, President of All Clear Emergency Management Group; and Gary Nestler, Associate Partner and Global Leader, Emergency Management solutions at IBM discuss these challenges and opportunities in this slideshare—which is intended to help disaster management stakeholders achieve the most accurate situational awareness using analytics.
Discover analytics solutions for emergency management http://ibm.co/emergencymgmt
Keynote talk by David Dietrich, EMC Education Services at ICCBDA 2013 : International Conference on Cloud and Big Data Analytics
http://twitter.com/imdaviddietrich
http://infocus.emc.com/author/david_dietrich/
What is the impact of Big Data on Analytics from a Data Science perspective.
Presented at the Big Data and Analytics Summit 2014, Nasscom by Mamatha Upadhyaya.
Over the past decade, cloud computing has acted as a disrupter in several areas of IT business. Soon, it will overhaul one area of technology that has been in rapid growth itself: Data Analytics. Nicky will focus on the recent study of IBM Institute of Business Value which shows that capabilities that enable an organization to consume data faster – to move from raw data to insight-driven actions – are now the key differentiator to creating value using data and analytics. He will also talk about the requirements for the underlying infrastructure as critical component allowing real-time crunching and analysis of high volume of data. Based on real cases like retailers and energy companies, we will look at five predictions in five years, based on:
Analytics, Big data, and Cloud coming together will energize the Speed Advantage.
The REAL Impact of Big Data on PrivacyClaudiu Popa
The awesome promise of Big Data is tempered by the need to protect personal information. Data scientists must expertly navigate the legislative waters and acquire the skills to protect privacy and security. This talk provides enterprise leaders with answers and suggests questions to ask when the time comes to consider the vast opportunities offered by big data.
Forecast to contribute £216 billion to the UK economy via business creation, efficiency and innovation, and generate 360,000 new jobs by 2020, big data is a key area for recruiters.
In this QuickView:
- Big data in numbers
- Top 10 industries hiring big data professionals
- Top 10 qualifications sought by hirers
- Top 10 database and BI skills sought by hirers
- Getting started in big data: popular big data techniques and vendors
Hello everyone! Data is required for every organisation in every field in today's world, and personal life. so, I am here to introduce how about What is Data and What is large scale computing.
The increasing use of smart phones, sensors and social media is a reality across many industries today. It is not just where and how business is conducted that is changing, but the speed and scope of the business decision-making process is also transforming because of several emerging technologies – Cloud, High Performance Computing (HPC), Analytics, Social and Mobile (CHASM).
In this issue of TOP TEN we provide the reader with a wealth of information related to current and future usages of BIG DATA. The reader will get an insight into usages in the realm of education, health, construction, management as well as marketing.
Artificial intelligence has been a buzz word that is impacting every industry in the world. With the rise of
such advanced technology, there will be always a question regarding its impact on our social life,
environment and economy thus impacting all efforts exerted towards sustainable development. In the
information era, enormous amounts of data have become available on hand to decision makers. Big data
refers to datasets that are not only big, but also high in variety and velocity, which makes them difficult to
handle using traditional tools and techniques. Due to the rapid growth of such data, solutions need to be
studied and provided in order to handle and extract value and knowledge from these datasets for different
industries and business operations. Numerous use cases have shown that AI can ensure an effective supply
of information to citizens, users and customers in times of crisis. This paper aims to analyse some of the
different methods and scenario which can be applied to AI and big data, as well as the opportunities
provided by the application in various business operations and crisis management domains.
Artificial intelligence has been a buzz word that is impacting every industry in the world. With the rise of such advanced technology, there will be always a question regarding its impact on our social life, environment and economy thus impacting all efforts exerted towards sustainable development. In the information era, enormous amounts of data have become available on hand to decision makers. Big data refers to datasets that are not only big, but also high in variety and velocity, which makes them difficult to handle using traditional tools and techniques. Due to the rapid growth of such data, solutions need to be studied and provided in order to handle and extract value and knowledge from these datasets for different industries and business operations. Numerous use cases have shown that AI can ensure an effective supply of information to citizens, users and customers in times of crisis. This paper aims to analyse some of the different methods and scenario which can be applied to AI and big data, as well as the opportunities provided by the application in various business operations and crisis management domains.
Artificial intelligence has been a buzz word that is impacting every industry in the world. With the rise of
such advanced technology, there will be always a question regarding its impact on our social life,
environment and economy thus impacting all efforts exerted towards sustainable development. In the
information era, enormous amounts of data have become available on hand to decision makers. Big data
refers to datasets that are not only big, but also high in variety and velocity, which makes them difficult to
handle using traditional tools and techniques. Due to the rapid growth of such data, solutions need to be
studied and provided in order to handle and extract value and knowledge from these datasets for different
industries and business operations. Numerous use cases have shown that AI can ensure an effective supply
of information to citizens, users and customers in times of crisis. This paper aims to analyse some of the
different methods and scenario which can be applied to AI and big data, as well as the opportunities
provided by the application in various business operations and crisis management domains.
IABE Big Data information paper - An actuarial perspectiveMateusz Maj
We look closely on the insurance value chain and assess the impact of Big Data on underwriting, pricing and claims reserving. We examine the ethics of Big Data including data privacy, customer identification, data ownership and the legal aspects. We also discuss new frontiers for insurance and its impact on the actuarial profession. Will actuaries will be able to leverage Big Data, create sophisticated risk models and more personalized insurance offers, and bring new wave of innovation to the market?
A l'occasion de l'eGov Innovation Day 2014 - DONNÉES DE L’ADMINISTRATION, UNE MINE (qui) D’OR(t) - Philippe Cudré-Mauroux présente Big Data et eGovernment.
Camssguide Big Data Analytics Solutions, can help you meet and exceed challenges and opportunities for business, industry, and technology solution areas
2. Welcome, Shane | My Account | Log Out
White Papers | Web Seminars | Newsletters | eBooks
Big Data & Analytics
Data Management
MDM & Data Governance
Infrastructure
Info Strategy & Leadership
BI & Data Discovery
Mobility
web seminars &
white papers
resource
center
Universities Put “Blue Gene” Machines In Cloud,
To Help Hedge Funds Trade
Shane Kite
JUN 8, 2010 1:42pm ET
Print
Email
Reprints
Comment
Twitter
LinkedIn
Facebook
Google+
Two hedge funds – a start-up and another based on Long Island – are set to tap high-performance computing
3. systems owned by Stony Brook University through a “cloud” of networks that connect firms to available
processing capacity at the school.
Separately, FINA Technologies, a Cambridge, Mass.-based hedge fund technology supplier spun out of Gene
Network Sciences, is renting cycles on Rensselaer Polytechnic Institute’s IBM Blue Gene supercomputer
located in North Greenbush, N.Y. FINA will use one of academia’s largest supercomputing centers to redeploy
software originally used to analyze gene sequences for optimal drug therapies to divine money-making
opportunities in the capital markets.
The start-up fund working with Stony Brook “will use a mixture of ordinary and high-performance computing
to design trading strategies,’’ according to James Glimm, distinguished professor and chair of the department of
applied mathematics and statistics at Stony Brook.
This may involve using cycles on New York Blue, a massively parallel IBM Blue Gene machine housed at
Brookhaven National Laboratory, located in Upton, N.Y., and owned by Stony Brook. Stony Brook has created
what it calls “a university research cloud,” in which hedge funds can contract to use supercomputing power
from Blue and other of the school’s processing resources.
Another fund, which Glimm would only describe as Long Island-based, is working with Stony Brook to
develop a trading system that aims to spot price and risk anomalies in the market – gaps where money can be
made – by combing massive amounts of market data. The work involves “developing software to power a
trading system, drawn in part from high-performance computing, meant to solve a mixture of large and small
data analysis problems,” he said.
Because of non-disclosure agreements, Glimm would not provide further details, except to say he and the
university “are working with other hedge funds on a variety of software projects.”
But he said the work will draw from a big baseline data crunch, powered by the school’s supercomputer.
“New York Blue will be used for a large calculation to provide what you might call a ‘Bayesian prior’ of the
stock market,” Glimm said. “It gives you sort of a universal view of everything. This is where you need huge
amounts of data and processing.”
A Bayesian prior is basically aimed at estimating the likelihood of an event – a prediction, essentially – based
on analysis of historical and incoming, new data.
“When you get to the trading part after analyzing the data to determine ‘that’s my world view,’ what you do
about it tomorrow using these findings is probably a smaller problem, which wouldn’t go on Blue,” he said.
Stony Brook is one of several intellectual ground zeros for quantitative finance. The Long Island institution is
home to Nobel prize-winners such as C.N. Yang (Physics, 1957) and Robert J. Aumann (Economics, 2005);
famous mathematicians such as Glimm and Dennis Sullivan -- and alma mater of arguably the most successful
hedge fund manager in history: Renaissance Technologies’ founder James Simons. One of Simons’ several
notable achievements: Renaissance’s Medallion fund’s 45-percent average annual returns since 1988.
Stony Brook’s program works thusly: Firms can use Blue for free for “proof of concept” trials, in which
hypothetical data or dummy code is used. Blue can also be used free if firms agree to share intellectual property
that gets developed with the university.
Firms must pay for cycles when they want to run proprietary work. The companies pay a portion of Blue’s
utility bill plus a small surcharge – what Michael Ridley, director of New York State’s high-performance
computing (HPC) program says amounts to “fractions of a penny” – for computing cycles consumed. “It’s a lot
cheaper than [Amazon’s] EC2,” Ridley claims.
4. New York Blue is considered “the centerpiece” of the New York Center for Computational Sciences, a
cooperative initiative between Stony Brook and Brookhaven meant to foster area innovation via collaboration
between the state’s national labs, universities and industry. It is part of a larger program run by the New York
State Foundation for Science Technology and Innovation (NYSTAR), where Ridley is HPC director.
New York Blue can perform over 100 trillion calculations per second. The 18-rack IBM Blue Gene/L system is
the 58th fastest supercomputer in the world, as ranked by Top500, which rates these machines. Stony Brook’s
cloud includes a smaller, two-rack Blue Gene/P system as well.
Firms access these supercomputers via secure connections over the Web known as virtual private networks
(VPNs). Datasets are transferred either with what Ridley calls “better than the bank” encryption or via private
courier if the data is too much to be streamed over the Internet.
Security is tight: Brookhaven is a Department of Energy lab dealing with nuclear materials, classified research
and matters of national security. Its multiple levels of safeguards must meet Department of Defense standards.
This includes armed guards.
Renaissance’s Simons is a longtime Stony Brook supporter and former chair of the university’s math
department. This is where he co-wrote pioneering research used in string theory. He donated $25 million to the
school’s math and physics departments in 2006. The former code-breaker followed in 2008 by giving $60
million to fund Stony Brook’s research programs as well as to fund a center for the study of geometry and
physics.
NYSTAR’s other HPC program offers companies cycles at Rensselaer Polytechnic Institute’s IBM BlueGene/L
supercomputer, currently ranked 70th among top supercomputers.
RPI is known for its research in computer science and nanotechnology, and for launching in 1980 the nation’s
first wholly university-sponsored technology incubator which, RPI says, has helped 250 companies
commercialize their ideas. Both RPI’s machine and New York Blue were started up in 2007, purchased with
state and federal funds and IBM discounts.
FINA Technologies is renting cycles on RPI’s machine to power the same patented system FINA’s biotech
parent uses to uncover best-course drug treatments for patients with cancer, diabetes or Alzheimer’s disease.
Called Reverse Engineering/Forward Simulation (REFS), in its medical application the software performs
billions of calculations to pinpoint the most important genetic or genomic markers and identify the most
positive responses of a given disease to a particular drug therapy.
When GNS determined REFS’ method of scouring the genome could be applied to sussing profits in the
financial market – essentially another system that, like the genome, has loads of components and interactions
with varying degrees of correlation to analyze – FINA was spun-off in 2008 to isolate factors that would
generate alpha – outsized returns.
This story first appeared in Securities Industry News.
JOIN THE DISCUSSION
Comment
5. SEE MORE IN
Cloud Computing
RELATED TAGS
Cloud Computing,
Comments (0)
Be the first to comment on this post using the section below.
Add Your Comments:
Add your comments here.
Notify me when other readers comment on this article.
Click here to receive notifications without commenting
Most Read
Most Emailed
BI and Big Data: Same or Different?
CenturyLink's Big Data Leader: on the Record
Data-Driven Decisions: Rethink Your Approach
When it Comes to Data, Small is the New Big
Hadoop as a Service: 18 Cloud Options
Analytics
6. Analytics and BI: Mutifaceted & Evolving
HP Seeks Mobile and Big Data Synergies
CenturyLink's Big Data Leader: on the Record
Analytics and the Customer Journey
Business Intelligence
How Big Data Keeps United Healthcare Nimble
Analytics and BI: Mutifaceted & Evolving
Data-Driven Decisions: Rethink Your Approach
The Future of Information: Revealed
Customer Experience
Millennials and the Machines
How to Build Connected Customer Experiences
BYOD Trend Shifts to Mobile Virtualization
Analytics and the Customer Journey
Open Source
Apple Buys NoSQL Big Data Specialist
EMC: Can Data Lakes Create Big Data Splash?
Avnet, IBM Unveil Turn-key Hadoop Big Data Platform
Online Hadoop Training Attracts Big Data Crowd
Predictive Analytics
Big Data Pushes Deeper Into Oil and Gas
Messy Big Data Overwhelms Data Scientists
Predictive Analytics or Data Science?
How Predictive Analytics Reinvents These Six Industries
Data Governance
California to Hire Chief Data Officer (CDO)?
Net Neutrality Decision: What You Need to Know
Balancing Freedom and Control to Enable Governed Data Discovery
Who Owns Your Big Data?
Data Integration
Inside Google's Insurance Data Strategy
Healthcare Industry Explores Data Monetization
Update on the DATA Act
Big Data Requires Integration Technology
7. Data Management
Five Dos and Don’ts for Managing Data Assets
Catalina Chief Data Officer: On the Record
Exceptional EPM/CPM Systems are an Exception
IBM's $3B IoT Bet Seeks to Monetize Weather Data
HOME
About Us
Contact Us
Content Licensing
Advertise with Us
Customer Service
Feedback
My Account
Site Map
Privacy Policy
Editorial Submissions
sourcemedia
corporate site
banking
American Banker
Bank Technology News
American Banker Magazine
Credit Union Journal
MORTGAGES
National Mortgage News
PAYMENTS
PaymentsSource
Collections & Credit Risk
ISO & Agent
capital markets
Mergers & Acquisitions
Asset Securitization Report
Leveraged Finance News
Private Placement Letter
Traders Magazine