Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Iterator - a powerful but underappreciated design pattern

Iterator design pattern is described in GoF ‘Design Patterns’ book. It is used at many places (e.g. Sql Cursor is a ‘iterator’), C++ standard template library uses iterators heavily. .Net Linq interfaces are based IEnumerable (i.e. iterator). However, I don’t see projects creating/using ‘custom’ iterator classes. Many problems can be solved ‘elegantly’ by use of customized iterators. This talk is about ‘power of iterators’ and how custom iterators can solve common problems and help create modular/reusable code components.

Key Discussion Points

Typical examples of iterators in common use.
Kind of problems that can be ‘elegantly’ solved with iterators
When to use custom iterators?
How write custom iterators in C++/C#

From webinar I did on TechGig

  • Login to see the comments

Iterator - a powerful but underappreciated design pattern

  1. 1. Iterator – A powerful but underappreciated pattern Nitin Bhide Chief Software Engineer Geometric Ltd.
  2. 2. Iterator – A Definition WHAT IS AN ITERATOR ? WHAT PROBLEM DOES IT SOLVE ? Confidential
  3. 3. Iterator - Definition In object-oriented computer programming, an iterator is an object that enables a programmer to traverse a container(or collection). • In object-oriented programming, the iterator pattern is a design pattern in which an iterator is used to traverse a container and access the container's elements. • Confidential
  4. 4. Some More Info about Iterators • Defined GoF book, as “Iterator Pattern”. • It is a Structural Pattern • The essence of the Iterator Pattern is to "Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation.“ • The iterator pattern decouples algorithms from containers. Confidential
  5. 5. Benefits of “decoupling of algorithms and collections” • Common ‘interface’ can be defined to access the elements of collection. • Algorithms can be written using this common iterator interface • Allows us to change the ‘internal’ implementation of collection with no change in the algorithms implementation • Allows us to add new algorithms which work all existing collection types. • For example, C++ Standard Template Library provides a set of algorithms which work on multiple collections types by using a common ‘iterator’ interface Confidential
  6. 6. Typical operations in an Iterator Interface • CurrentElement() • referencing one particular element in the object collection • Next() • modifying itself so it points to the next element • Reset() – Optional • Reset the current element to start element • HasNext() or IsEnd() – Optional • Detect if there is no element left in the collection. Confidential
  7. 7. Looping with Iterator (pseudo code) Simple for loop • for(int i=0; i<10; ++i) { curval = i; } For loop with Iterator • for(iterator it=iterator(); it.IsEnd()==false; it.Next() { curval = it.Current(); } Confidential
  8. 8. In various programming languages ITERATOR IMPLEMENTATIONS Confidential
  9. 9. Iterator Implementations – Concepts and Limitations • Iterator as an ‘object’ allows iterator to have its own state (i.e. its own member variables) different than the state of collection • Allows multiple iterator objects corresponding to same collection • If collection changes while ‘iteration’ is in progress, all existing iterators can become invalid. • Implementation of Iterator and the collection on which it operates are usually ‘tightly coupled’ Confidential
  10. 10. Examples of Iterator implementations C++ C# • STL Containers implement their iterators • STL algorithms are defined in terms iterators • IEnumerable<T> and Ienumerator<T> iterfaces define the iterators Java • SDK defines iterator interface • and collections implement their iterators SQL • Cursor is an iterator Confidential
  12. 12. LINQ – Language INtegrated Query • Language Integrated Query (LINQ, pronounced "link") is a Microsoft .NET Framework component that adds native data querying capabilities to .NET languages. • From MSDN Documentation: In Visual Studio you can write LINQ queries in Visual Basic or C# with SQL Server databases, XML documents, ADO.NET Datasets, and any collection of objects that supports IEnumerable or the generic IEnumerable<T> interface. Essentially Power of LINQ is because of Power of Iterators Confidential
  13. 13. Reason behind Iterators Power Simple for loop with if check For loop with iterator For(i=0; i<100; i++) { if( I % 3 == 0) { print I; } } For(it.Reset(); it.HasNext(); it.MoveNext()) { print it.Current; } For loop is has 3 responsibilities 1. Looping 2. Filtering 3. Printing the results For loop has only two responsibilities 1. Looping 2. Printing the results Iterators MoveNext function is overloaded such that it returns only numbers divisible by 3. Responsibility of Filtering is moved to Iterator. Confidential
  14. 14. Simple For loop with LINQ with if check With LINQ for(i=0; i<100; i++) { if( i % 3 == 0) { print i; } } var nums = from n in Enumerable.Range(1, 100).w here(n % 3 == 0) for(var i in nums) { print i; } Confidential
  15. 15. LINQ Providers • LINQ to SQL/ADO.Net • LINQ to XML • LINQ to Amazon • LINQ to Active Directory • LINQ to CRM • LINQ To Geo - Language Integrated Query for Geospatial Data • LINQ to Excel • LINQ to Flickr • LINQ to Google • LINQ to Indexes (LINQ and i40) • LINQ to JSON • LINQ to IMAP • LINQ to NHibernate • LINQ to LDAP • LINQ to Lucene • LINQ to MySQL, Oracle and PostgreSql (DbLinq) • LINQ to NCover • LINQ to Opf3 • LINQ to RDF Files • LINQ to Sharepoint • LINQ to SimpleDB • LINQ to Streams • LINQ to WebQueries • LINQ to WMI From Links to LINQ page : Confidential
  16. 16. Reasons behind Power of Iterators Iterators can act as ‘filters’ or ‘views’ • where instead of returning all elements of a collection it can ‘filter’ the elements and return a ‘partial’ list. • The filtering happens as ‘needed’ and an intermediate ‘collection’ of filtered results can be avoided. • Different implementations of iterator can provided different filtering criteria. • Hence algorithms written with ‘iterators’ can work on original collection or filtered collections. . Confidential
  17. 17. Reasons behind Power of Iterators Iterators can be polymorphic • Hence same function can return different results by passing different iterator to it. • For example, in our previous example of printing numbers divisible by 3, can be easily changed to printing all ‘primes’ by replacing the iterator. There is no need to change the print function. Confidential
  18. 18. Reasons behind Power of Iterators Iterators can be combined to provide more flexibility • For example, combine two existing iterators into a new iterator such that it ‘chains’ the child iterators • For example, ‘<itertools>’ module of python provides many ways of combining iterators • Following one liner in python efficiently returns ‘dot product’ of two mathematical vectors sum(imap(operator.mul, vector1, vector2)). imap() function takes multiple iterators and calls ‘mul’ (multiply function) with current values of those iterators. Confidential
  19. 19. Reasons behind Power of Iterators Iterators can work with ‘collection like’ objects. • For example, we can write an iterator which return one word at a time from an input stream (or a text file) • Best example is how LINQ works with Databases or XML with exactly same interface. Confidential
  20. 20. Few More Examples of Powerful Iterators Suppose we want to extract ‘unique words’ from a text file. • Since iterator can have its own state, we can write an iterator that remembers the words it already ‘seen’ and • if the word is already ‘seen’ ignore that word and reads the next word from the istream. • Such an iterator will return just the ‘unique’ words in the istream. • Now if you have function which takes a word iterator as parameter and prints the words, then just by passing the ‘unqiue word iterator, we can print the unique words in the text file. Confidential
  21. 21. Laser Cutting Tool path Generation • For one of my project, problem was to generate the toolpath for laser cutting of sheet metal parts • Toolpath generation had different strategies based on various parameters and part selection logic etc etc. • Naïve implementation, will require • writing complicated for loops different strategies • Parameters passed as function parameters • Combining strategies is nearly impossible. • Lot of duplication of boiler plate code. • New strategy will require changes in the existing code. Confidential
  22. 22. Laser Cutting Tool path Generation - Solution • We defined a base class ToolPathIterator • Every new toolpath creation strategy is implemented as ‘drived class’ of this iterator. • We also implemented iterators which derived from this base class And also used some existing Toolpathiterators internally to combine the strategies. • A Factory method instantiated the necessary toolpath iterator based on name of the strategy. • Only Factory method depends on all concrete iterator implementations. • Everywhere else ToolPathIterator base class was used. • Add a new strategy required • (a) adding new derived class from ToolpathIterator • (b) change in Factory Method. Confidential
  23. 23. Why I don’t see many custom iterators ? WHY ITERATORS ARE UNDERAPPRECIATED ? Confidential
  24. 24. •Most probably because Iterator is a ‘really simple concept’ •and •people have difficulty in believing that such simple concept can have so much power. Confidential
  25. 25. Why iterators are underappreciated ? • Difficulty in thinking traversal/iteration as ‘separate object/responsibility’ • Difficulty in thinking ‘iterator’ as interface with multiple possible implementations • Writing an iterator requires defining a new class. • This may require additional coding while writing for loop may look simpler/quicker. Confidential
  26. 26. Difficulty in ‘thinking’ traversal/iteration as ‘separate object’ • Developers are used to thinking in loops which work on ‘indices’. • For them ‘i++’ is conceptually easier that ‘’ • I see many examples of iterating over C++ vector use indexing operation (e.g. vec[i]) rather than using vec.begin(); • Hence creating separate class/object to keep track of current element and moving to next element is somehow difficult ‘leap’. • However once you make that ‘leap’ your subsequent design may change significantly from the past design Confidential
  27. 27. ‘iterator’ as interface with multiple possible implementations • Since ‘iterator’ is a separate class/object, we can create ‘iterator’ interfaces (or base classes). • If we write function using the ‘iterator’ interface, then we can pass different implementations of iterators and get different behavior. • Different traversal algorithms (e.g. pre order, post order traversals in trees) • Filtering • All this flexibility is possible, if you start thinking in terms of ‘iterator interface and its implementations’ or a hierarchy of iterators. Confidential
  28. 28. Writing an iterator requires defining a new class • In languages like Java, C++, writing an iterator means adding a new class in the system. • It required additional code than simple for loop • Developer has to take care in defining iterator interface (e.g. iterators that will work with STL containers). • Developers treated it as ‘additional burden/cost’ . They could not visualize the benefits and hence decided that separate iterator class is probably not worth the efforts. • However, this is changing with co-routines implementations • ‘yield return’ and similar keywords being introduced in the languages like C#, Python. Confidential
  29. 29. Yield Return in C# • A Trivial Example : Writing an iterator which returns numbers divisible by three • Defining the Iterator using Yield Return. IEnumberable<int> DivisibleByThree() { for(int i=0; i< 100; i++) { if(i % 3 == 0) { yield return i; } } } • Using the Iterator with foreach foreach(var j=DivisibleByThree()) { print j; } Confidential
  31. 31. Use Iterators instead of ‘indexing’ in loops • In general, loops with iterators are more efficient than index based loops • For example, getting element at ‘i’th Index is equivalent to *(Startpos + i*sizeof(element)) While getting next element from the current element is equivalent to *(cur_elem_pos++) • Hence usually ‘iterators’ based loops are slightly faster. Confidential
  32. 32. Never directly return a member ‘collection’ from a class Don’t DO Class XYZ { private List<int> intlist= new List<int>(); Class XYZ { private List<int> intlist= new List<int>(); List<int> IntList { get { return intlist; } } } IEnumberable<int> IntList { get { return intlist; } } } Confidential
  33. 33. Loop+If : see if you can use an iterator • If you see similar loop + if condition or similar contents of loop at multiple places • See if you can extract the similarities into an Iterator • Sometime you have to create a ‘base iterator’ and override ‘MoveNext()’ implementations • If you are using C#/Python, see if you can use ‘yield’ Confidential
  34. 34. Never directly return a member ‘collection’ from a class. • It violates ‘encapsulations’ • Also all the users of your class are now ‘explicitly’ depend on the collection (e.g List). • Tomorrow if you want to change the List to HashSet(), every where your class is used will potentially need to change. • For C# • Define an IEnumerable as ‘get’ property. It will allow you to change the internal collection type with almost no impact on the external interface. • For Java/C++ • Define an iterator for your class. Confidential
  35. 35. Summary As Developer change your thinking to • Think traversal/iterator as ‘separate class/object/responsibility’ • Think ‘iterator’ as interface with multiple possible implementations • Different traversal strategies/algorithms can be implemented as different iterator implementations. Once you do that, you will find many more uses of Iterator • That can simplify your code • Make it less bug prone and more stable. • Make it easier to enhance and maintain Confidential
  36. 36. END OF PRESENTATION The material contained in this deck represents proprietary, confidential information pertaining to products, technologies & processes from Geometric. All information in this document is to be treated in confidence. The names and trademarks used in this document are the sole property of the respective companies and are governed/ protected by the relevant trademark and copyright laws. © Geometric Limited | | | Confidential