Open Source in the Cloud Computing Era
by Tim O'Reilly, CEO at O'Reilly Media on Mar 18, 2010
- 29,811 views
Featured in: Technology, Business
While open source software plays an important role in many cloud applications, we need to understand where the cloud is taking us or we'll find ourselves in the grip of a new monopoly. Open source ...
While open source software plays an important role in many cloud applications, we need to understand where the cloud is taking us or we'll find ourselves in the grip of a new monopoly. Open source needs to get serious about building interoperable open data services - they are the operating system of the internet.
Statistics
Views
- Total Views
- 29,811
- Views on SlideShare
- 19,005
- Embed Views
Actions
- Likes
- 37
- Downloads
- 937
- Comments
- 14
Accessibility
Categories
Upload Details
Uploaded via SlideShare as Adobe PDF
Usage Rights
© All Rights Reserved
nbortolotti
عقار - http://www.3qarsa.net
حكايات نواعم - http://www.nem-stories.com/vb/ 3 years ago
It is all about the software. We already have more data than we know what to do with. The challenge has always been in making sense of it. The fact that Hadoop and Lucene exist is proof that Google's approach to free software is wrong.
I can see how we are repeating the Microsoft mistake with Google, and yet Tim O'Reilly is focused only on easy import / export of scraps from Google.
Is Google's map database open? How is that going? See, once you explain free software, free maps is a very short conversation.
I also disagree that hardware is any limiting factor because you can get a terabyte drive for $100 at Best Buy. 4 years ago
I see there is a robust Linux kernel community and codebase, but no equivalent search engine community and codebase.
I don't know what would happen if Google gave their code away years ago, but no one really knows. In fact, I think Google could be leading just as much because they would be at the center of it and they would be sharing the plumbing work with others so they could focus on new things in search. There would be no reason for Hadoop or Lucene and such, which will presumably eventually get there.
I don't know the history on Nutch, and whether it was poorly advised, and I presume that 'a lack of data at scale' is a solved by hardware once you've got good software. So it seems short-sighted to say that one can't solve a hardware problem given Moore's law.
One can get a terabyte of disk space for $100 today. Wikipedia runs on just a few hundred servers and it is a top 10 site on the Internet. We could build a heck of a search engine with just that datacenter. 100 terabytes gives you space for 1 megabyte of data for 100 million web sites. That gets you quite far down the long tail. It also seems like a hardware factor is the machines responding to user queries in addition to the web spiders maintaining the index. And whether you start writing lots of other random software like blogging and email.
I don't think the free chip fab movement is that big of a deal. There aren't conferences about that. Hardware will always cost money but be a one-time cost. It is like conflating the value of blank paper versus paper with ideas on it.
I agree lots of time has been wasted. I have documented inefficiencies all over the free software movement. There are a number of things to be worked on, but the biggest failure is that FOSS has not won the battle of ideas.
I realize I am repeating many of the same points as those who came before. The free software movement is a complicated thing. I've enjoyed spending the last few years catching up and writing about it. One thing unique in my study is that I have focused on the inefficiencies and problems with the existing free software community. There are reasons why an army of millions is still losing to an army of thousands.
-Keith 4 years ago
I said it to Richard Stallman in 1999, and I'll say it again to you now. If you gave every bit of Google's code to the FSF, they couldn't deliver the services they do. Even if you gave FSF every bit of Google's data, they couldn't do it. Only a very few companies have the money and scale to do this. It's a different game than it was in the PC era, and the failure of the Free Software community to understand this is holding back the cause of freedom.
It's possible that there could be new distributed data and computation architectures, but I don't think they'd have the necessary performance.
Just as the Free Software movement never succeeded with free chip fabs (too big and expensive), I don't think it has any handle on the kinds of problems that need to be solved in the internet era.
There are people who are working on real problems of interoperability/freedom in the cloud era - folks like Brad Fitzpatrick at Google and David Recordon at Facebook - but the free software zealots seem to keep hoping that their old story will work if they just keep repeating it long enough.
I called the current outcome more than ten years ago. That's ten wasted years. 4 years ago
I am fascinated to listen to the talk you gave.
And chance to get it online? 4 years ago
Nutch / Lucene's problem is that they don't have a community yet. The free software communities are too often on the outside looking in. People aren't going to use their search unless it is much better a-la Wolfram Alpha. But the Lucene community is doing quite well and it will get there. It is a shame that none of the big search vendors are truly partnering with the free search communities yet. Maybe it is because people are focusing on 'open data' too much.
Hardware is not equally as important as the code. As Feynman said: 'The inside of a computer is dumb as h*ll but it goes like mad.' He said that in 1984, and it is even more true today.
Software is the magic. It is that which can take the data and manipulate it and eventually make sense of it. And so we should be much more concerned that Google's search engine is not free than the fact that they are preferentially routing search traffic to their own sites, etc.
The 'open data' movement should be a part of the free software movement. Even if you could export data from Google's search servers, you wouldn't necessarily be able to make sense of it if you didn't have the accompanying software. 4 years ago
Or take search. Nutch was an open source search engine based on lucene. They had the code, but they didn't have the data, and couldn't afford the infrastructure to collect or host more.
Code matters, just like hardware matters. But at internet scale, and in the future we're building, it doesn't matter like it did in the standalone PC era. 4 years ago