Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Desarrollo con Storm
Desarrollo con Storm
● Recordemos que los componentes de Storm se pueden definir en cualquier
lenguaje de programación
○ J...
Topología simple
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout(“words”, new TestWordSpout(), 10);
buil...
Topología simple
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout(“words”, new TestWordSpout(), 10);
buil...
El método importante a implementar es nextTuple().
public void nextTuple() {
// hacemos lo que sea
Utils.sleep(100);
final...
Tiene los siguientes métodos:
public void prepare(Map conf, TopologyContext ctx, OutputCollector c){
_collector = c;
}
Pro...
public void cleanup(){}
Este método es invocado cuando se va a apagar el bolt y debería
limpiar ordenadamente los recursos...
La forma más sencilla de crear bolts es extender la clase BaseRichBolt:
public static class ExclamationBolt extends BaseRi...
Ejemplo: Hola mundo
public class HolaMundoSpout extends BaseRichSpout {
private static final long serialVersionUID = 1L;
private SpoutOutputCo...
Esta clase consume las tuplas producidas e implementa la lógica necesaria.
La creamos dentro del mismo paquete storm.ebd y...
public class HolaMundoTopology {
public static void main(String[] args) {
TopologyBuilder builder = new TopologyBuilder();...
Hola mundo: Ejecucion
El HolaMundoSpout genera un número aleatorio en su constructor y luego en el
método nextTuple() gene...
Python
(ns wordcount
(:use [streamparse.specs])
(:gen-class))
(defn wordcount [options]
[
;; spout configuration
{"word-sp...
Upcoming SlideShare
Loading in …5
×

Apache Storm: Desarrollo

1,455 views

Published on

Apache Storm: Desarrollo

  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Apache Storm: Desarrollo

  1. 1. Desarrollo con Storm
  2. 2. Desarrollo con Storm ● Recordemos que los componentes de Storm se pueden definir en cualquier lenguaje de programación ○ Java ○ Ruby ○ Python ○ Javascript ○ Perl ○ ...
  3. 3. Topología simple TopologyBuilder builder = new TopologyBuilder(); builder.setSpout(“words”, new TestWordSpout(), 10); builder.setBolt(“exclaim1”, new ExclamationBolt(), 3) .shuffleGrouping(“words”); Builder.setBolt(“exclaim2”, new ExclamationBolt(), 2) .shuffleGrouping(“exclaim1”); Una topología con 1 Spout y 2 Bolts. El Spout emite palabras y los Bolts añaden “!!!” al final. Los métodos set() reciben como parámetro un id de tipo String, un objeto que contiene la lógica de procesado y el nivel de paralelismo del nodo. El último parámetro indica el número de hilos que se ejecutan en ese componente. El objeto de procesado implementa la interfaz IRichSpout para los Spouts e IRichBolt para los bolts.
  4. 4. Topología simple TopologyBuilder builder = new TopologyBuilder(); builder.setSpout(“words”, new TestWordSpout(), 10); builder.setBolt(“exclaim1”, new ExclamationBolt(), 3) .shuffleGrouping(“words”); Builder.setBolt(“exclaim2”, new ExclamationBolt(), 2) .shuffleGrouping(“exclaim1”); setBolt devuelve un objeto InputDeclarer que se utiliza para definir las entradas en el bolt. Indicamos que “exclaim1” quiere leer todas las tuplas de “words” utilizando shuffle grouping y que “exclaim2” quiere leer todas las tuplas de “exclaim1”. “ShuffleGrouping” indica que las tuplas se distributen de forma uniformemente aleatoria entre las entrada-salida.
  5. 5. El método importante a implementar es nextTuple(). public void nextTuple() { // hacemos lo que sea Utils.sleep(100); final String[] words = new String[]{“bigData”, “ebd”, “utad”}; final Random rand = new Random(); final String word = words[rand.nextInt(words.length)]; _collector.emit(new Values(word)); } nextTuple() es invocado de forma periódica en el mismo bucle que los métodos ack() y fail(). Libera el control del hilo cuando no hay trabajo y permite que se ejecuten los otros métodos. Spout
  6. 6. Tiene los siguientes métodos: public void prepare(Map conf, TopologyContext ctx, OutputCollector c){ _collector = c; } Proporciona al bolt un OutputCollector que se utiliza para emitir las tuplas de este Bolt. Las tuplas se pueden emitir en cualquier parte del bolt. Es llamado antes de que el bolt empiece a procesar tuplas. public void execute(Tuple tuple) { _collector.emit(tuple, new Values(tuple.getString(0) + “!!!”)); _collector.ack(tuple); } Procesa una tupla de las entradas del Bolt. En el ejemplo le añade la cadena “!!!” al string. Si el bolt está subscrito a multiples fuentes de entrada, se puede determinar el origen con el método Tuple#getSourceComponen Bolt
  7. 7. public void cleanup(){} Este método es invocado cuando se va a apagar el bolt y debería limpiar ordenadamente los recursos abiertos. public void declareOutputFields(OutputFieldsDeclarer declarer){ declarer.declare(new Fields(“word”)); } Este método indica que el bolt emite una tupla con el campo “word”. Declara el esquema de salida del bolt. Public Map getComponentConfiguration() {} Permite configurar aspectos de como se ejecuta el componente. Bolt Introducción
  8. 8. La forma más sencilla de crear bolts es extender la clase BaseRichBolt: public static class ExclamationBolt extends BaseRichBolt { OutputCollector Mycollector; public void prepare(Map conf, TopologyContext ctx, OutputCollector collector) { Mycollector = collector; } public void execute(Tuple tuple) { Mycollector.emit(tuple, new Values(tuple.getSting(0)+”!!!”)); Mycollector.ack(tuple); } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields(“word”)); } } Bolt
  9. 9. Ejemplo: Hola mundo
  10. 10. public class HolaMundoSpout extends BaseRichSpout { private static final long serialVersionUID = 1L; private SpoutOutputCollector collector; private int referenceRandom; private static final int MAX_RANDOM = 10; public HolaMundoSpout() { final Random rand = new Random(); referenceRandom = rand.nextInt(MAX_RANDOM); } @Override public void nextTuple() { Utils.sleep(100); final Random rand = new Random(); int instanceRandom = rand.nextInt(MAX_RANDOM); if (instanceRandom == referenceRandom) { collector.emit(new Values("Hola Mundo")); } else { collector.emit(new Values("Otro Mundo Aleatorio")); } } @Override public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) { this.collector = collector; } @Override public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("sentence")); } } Hola mundo: Spout El método Open() inicializa el objeto SpoutOutputCollector. El Spout va a invocar de forma repetida el método nextTuple(). El método declareOutputFields() indica el contenido de las tuplas enviadas con emit() en el método nextTuple().
  11. 11. Esta clase consume las tuplas producidas e implementa la lógica necesaria. La creamos dentro del mismo paquete storm.ebd y que herede de la clase BaseRichBolt. Declaramos una variable miembre privada y proporcionamos la siguiente implementación para el método execute( ) que realiza la mayor parte del trabajo. public void execute(Tuple input) { String test = input.getStringByField("sentence"); if("Hola Mundo".equals(test)){ myCount++; System.out.println("Encontrado Hola Mundo!. Contador:" + Integer.toString(myCount)); } } Hola mundo: Bolt
  12. 12. public class HolaMundoTopology { public static void main(String[] args) { TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("randomHolaMundo", new HolaMundoSpout(), 10); builder.setBolt("HolaMundoBolt", new HolaMundoBolt(), 2).shuffleGrouping("randomHolaMundo"); Config conf = new Config(); conf.setDebug(true); if(args != null && args.length > 0) { conf.setNumWorkers(3); try { StormSubmitter.submitTopology(args[0], conf, builder.createTopology()); } catch (AlreadyAliveException e) { e.printStackTrace(); } catch (InvalidTopologyException e) { e.printStackTrace(); } } else { LocalCluster cluster = new LocalCluster(); cluster.submitTopology("test", conf, builder.createTopology()); Utils.sleep(10000); cluster.killTopology("test"); cluster.shutdown(); } } } Hola mundo: Topology Ahora creamos nuestra primera Topology donde unimos los Spouts y Bolts.
  13. 13. Hola mundo: Ejecucion El HolaMundoSpout genera un número aleatorio en su constructor y luego en el método nextTuple() genera números aleatorios y los compara con el que se ha generado en el constructor. El método nextTuple() emite una sentencia entre dos posibles: "Hola Mundo", "Otro Mundo Aleatorio" dependiendo de si el número aleatorio coincide con el generado en el constructor o no. El bolt simplemente cuenta las apariciones de la frase “Hola Mundo” y muestra los incrementos secuenciales ● Desde consola: mvn compile exec:java-Dexec.classpathScope=compile - Dexec.mainClass=storm.ebd.HolaMundoTopology ● Se puede empaquetar para enviar al cluster mvn package
  14. 14. Python (ns wordcount (:use [streamparse.specs]) (:gen-class)) (defn wordcount [options] [ ;; spout configuration {"word-spout" (python-spout-spec options "spouts.words.WordSpout" ["word"] ) } ;; bolt configuration {"count-bolt" (python-bolt-spec options {"word-spout" :shuffle} "bolts.wordcount.WordCounter" ["word" "count"] :p 2 ) } ] ) Importa las funciones de CLojure DSL de Storm

×