Flat Filer Presentation

429 views

Published on

Published in: Career, Technology, Business
  • Be the first to comment

  • Be the first to like this

Flat Filer Presentation

  1. 1. Flat Filer <ul><ul><li>Andy Libby </li></ul></ul><ul><ul><li>Tangeis, LLC </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul>
  2. 2. Agenda <ul><li>Our Needs </li></ul><ul><li>Our Solution </li></ul><ul><li>What are flat files </li></ul><ul><li>Filer class and basic usage </li></ul><ul><li>Typical Tasks </li></ul><ul><li>Filters and Formatters </li></ul><ul><li>Q/A </li></ul>
  3. 3. Our Need <ul><li>Consume and produce data files provided by various vendors </li></ul><ul><li>Make it easy to update ActiveRecord objects from flat file records. </li></ul><ul><li>We could not find anything pre-existing </li></ul>
  4. 4. What is Flat Filer <ul><li>Small Ruby library for reading and writing fixed width flat files. </li></ul><ul><li>Gives syntactically clean interface to pack/ unpack for flat file records </li></ul><ul><li>MIT Licensed </li></ul><ul><li>Available at github </li></ul>
  5. 5. Typical Flat File <ul><ul><li>1 2 3 </li></ul></ul><ul><ul><li>012345678901234567890123456789 </li></ul></ul><ul><ul><li>idfirst last birthday </li></ul></ul><ul><ul><li>1Linus Torvalds 19691228 </li></ul></ul><ul><ul><li>2Randall Schwartz 19611122 </li></ul></ul><ul><ul><li>3Xavier Leroy 19680315 </li></ul></ul>
  6. 6. A Basic class <ul><ul><li>class PersonFile < FlatFile </li></ul></ul><ul><ul><li>add_field :id, :width => 2, :filter => proc { |v| v.to_i } </li></ul></ul><ul><ul><li>add_field :first_name, :width => 10 </li></ul></ul><ul><ul><li>add_field :last_name, :width => 10 </li></ul></ul><ul><ul><li>add_field :birthday, :width => 8, </li></ul></ul><ul><ul><li>:filter => proc { |v| Date.parse v }, </li></ul></ul><ul><ul><li>:formatter => proc { |v| v.strftime(“%Y%m%d”) </li></ul></ul><ul><ul><li>end </li></ul></ul>
  7. 7. Typical Task <ul><ul><li>filer = PersonFile.new </li></ul></ul><ul><ul><li>filer.each_record(fh) do |record, line| </li></ul></ul><ul><ul><li>person = Person.find(record.id) </li></ul></ul><ul><ul><li>next unless person </li></ul></ul><ul><ul><li>record.map_in person </li></ul></ul><ul><ul><li>person.save! </li></ul></ul><ul><ul><li>end </li></ul></ul>
  8. 8. Writing a flat file <ul><ul><li>@writer_objects.each do |obj| </li></ul></ul><ul><ul><li>r =PersonFile.new_record(obj) </li></ul></ul><ul><ul><li>puts r.to_s </li></ul></ul><ul><ul><li>end </li></ul></ul>
  9. 9. Other Options for add_field <ul><li>:map_in_proc – Specify logic to execute when mapping in the field. </li></ul><ul><li>:aggressive – if true, overwrite existing values in a model when “mapping in” </li></ul>
  10. 10. Filters and Formatters <ul><li>Execute code on fields either on the way in or on the way out of a filer </li></ul><ul><li>Filters execute when reading files </li></ul><ul><li>Formatters execute when writing files (okay when calling to_s on records) </li></ul><ul><li>Can be a symbol, lambda, or object responding to filter call. </li></ul>
  11. 11. Filter/ Formatter Example <ul><ul><li>class PersonFile < FlatFile </li></ul></ul><ul><ul><li>add_field :id, :width => 2, :filter => proc { |v| v.to_i } </li></ul></ul><ul><ul><li>add_field :first_name, :width => 10 </li></ul></ul><ul><ul><li>add_field :last_name, :width => 10 </li></ul></ul><ul><ul><li>add_field :birthday, :width => 8, </li></ul></ul><ul><ul><li>:filter => proc { |v| Date.parse v }, </li></ul></ul><ul><ul><li>:formatter => proc { |v| v.strftime(“%Y%m%d”) </li></ul></ul><ul><ul><li>end </li></ul></ul>
  12. 12. Getting Flat Filer and Questions <ul><ul><li>http://github.com/alibby/flat_filer/ </li></ul></ul>

×