Article Discussion
Modular Architectures with Ruby
Summary: Any reasonably complex end-user application is going to require some sort of customization and enhancement for effective deployment. This article shows one way to create a modular architecture as a way of leaving the door open for advanced users or consultants who want to extend the functionality without modifying the source.
15 posts.
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: January 18, 2006 3:57 PM by andresbube
    James
     
    Posts: 9 / Nickname: jbritt / Registered: March 24, 2003 4:10 PM
    Modular Architectures with Ruby
    October 9, 2005 8:03 PM      
    A modular architecture is one where the user can create modules that conform to well-described APIs and plug them into the application to extend the functionality. This article shows one way to create a modular API in Ruby.

    Read this article by Jack Herrington, author of Code Generation in Action

    http://www.artima.com/rubycs/articles/modular_apis_with_ruby.html

    What did you think of Jack's article?
    • Michael
       
      Posts: 2 / Nickname: mwhart / Registered: October 10, 2005 2:41 PM
      Re: Modular Architectures with Ruby
      October 10, 2005 6:51 PM      
      I don't actually know very much about Ruby, but isn't there a cleaner way to retrieve a parser from a factory than the loop in the ParserFactory.parser_for method?

      It seems to me that having to instanciate each factory and check if it matches the requested type is overkill - couldn't it be done in a hash of some sort (the factory could be added to the hash with the type as the key in the ParserFactory.inherited method), or even just checking a class variable/method (as opposed to an object one)?
      • Joe
         
        Posts: 15 / Nickname: jcheng / Registered: October 16, 2002 8:08 AM
        Re: Modular Architectures with Ruby
        October 10, 2005 8:41 PM      
        >> It seems to me that having to instanciate each factory and check if it matches the requested type is overkill - couldn't it be done in a hash of some sort (the factory could be added to the hash with the type as the key in the ParserFactory.inherited method), or even just checking a class variable/method (as opposed to an object one)? <<

        I agree. Factory classes are basically a necessity in Java, where interfaces can't specify static methods and classes aren't objects. With Ruby, the former doesn't apply and the latter isn't true.

        In fact, since all of the factories are essentially identical, a more Rubyish way to do it would be to create a module method that registers a parser without the need to write explicit factory code at all:

        class RDFParser < Parser
        ParserFactory::register(self, "RDF")

        def parse( xml )
        # Parse the XML up and return some known format
        return nil
        end
        end


        That should be *all* the code you need to write to implement and register a parser. (I didn't bother with the functionality on page 4 but it should be a straightforward extension.) Here's the complete code for ParserFactory:

        class ParserFactory
        @@factories = {}
        def ParserFactory.register(factory, type)
        @@factories[type] = factory
        end
        def ParserFactory.parser_for(type)
        @@factories[type]
        end
        def ParserFactory.load( dirname )
        Dir.open( dirname ).each { |fn|
        next unless ( fn =~ /[.]rb$/ )
        require "#{dirname}/#{fn}"
        }
        end
        end
        • Joe
           
          Posts: 15 / Nickname: jcheng / Registered: October 16, 2002 8:08 AM
          Re: Modular Architectures with Ruby
          October 10, 2005 8:48 PM      
          Whoops, I got (at least) this method wrong:

            def ParserFactory.parser_for(type)
          @@factories[type] && @@factories[type].new
          end
        • Michael
           
          Posts: 2 / Nickname: mwhart / Registered: October 10, 2005 2:41 PM
          Re: Modular Architectures with Ruby
          October 10, 2005 9:07 PM      
          I like the idea of searching for a more Ruby-ish way, but I guess one possible problem with your approach, Joe, is that each parser is then dependent on the ParserFactory, which reduces their portability somewhat. The additional abstraction layer in the original design eliminates this dependency. It's probably up to your development needs whether this is acceptable or not.
    • Graham
       
      Posts: 1 / Nickname: fosterg / Registered: October 11, 2005 5:57 AM
      Re: Modular Architectures with Ruby
      October 11, 2005 10:01 AM      
      Thanks for the informative article. Its rather a new way of thinking for me. How would your example change if you were to use Needle (or other IOC module).. I'm still trying to get my head around these concepts.
      Graham
    • Dave
       
      Posts: 1 / Nickname: dws / Registered: May 9, 2003 7:02 AM
      Re: Modular Architectures with Ruby
      October 14, 2005 8:16 AM      
      I'm wondering if the separate Factory class hierarchy carries its weight. By keeping a Hash in a class variable in the base Parser class to index each subclass by the type it services, and suppyling a class method to populate that hash, subclasses could do something along the lines of

      class FooParser < Parser
      parses :foo
      ...

      which has more declarative value than overriding inherited. It's a simple step from there to

      parser = Parser.for(filename)

      This approach also opens the door for having a single parser class service multiple types, e.g.,

      class HtmlParser < Parser
      parses :htm, :html

      Dave Thomas does something similar in RubLog.
    • Abhijit
       
      Posts: 1 / Nickname: aabhijit / Registered: March 13, 2005 2:24 AM
      Re: Modular Architectures with Ruby
      November 6, 2005 6:59 AM      
      I am new to this Ruby language but while trying out the example I found a ridiculous problem. I am not sure if it actually exists or not. So if someone could verify it for me I would be grateful.
      In the rss.rb file I wrote the class definition as,

      class RSSFactory < ParserFactory
      INFO=<<INFO
      type: RSS
      author: Abhijit
      description: An RSS Parser
      INFO
      ...
      end

      And the compiler gave me some error about not finding INFO...Then I rewrote this as,

      class RSSFactory < ParserFactory
      INFO=<<INFO
      type: RSS
      author: Abhijit
      description: An RSS Parser
      INFO
      ...
      end

      And it worked!!
      • p
         
        Posts: 1 / Nickname: p2 / Registered: November 14, 2005 9:40 PM
        Re: Modular Architectures with Ruby
        November 15, 2005 2:57 AM      
        I don't see any differencies in yor code snippets... But I have the same problem.

        I also getthis error in my test.rb:
        undefined method `parse' for nil:NilClass (NoMethodError)
        • Clayton
           
          Posts: 1 / Nickname: arton / Registered: November 15, 2005 8:18 AM
          Re: Modular Architectures with Ruby
          November 15, 2005 1:25 PM      
          It is using heredoc syntax:
          http://en.wikipedia.org/wiki/Heredoc

          You don't want any whitespace preceding the closing INFO (or any other text between the INFOs you don't want to be indented).
    • John
       
      Posts: 1 / Nickname: jbonnar / Registered: November 22, 2005 8:46 PM
      Re: Modular Architectures with Ruby
      November 23, 2005 2:30 AM      
      Requiring factor methods to test and create a parser like this is a bit silly in Ruby. Defining a class method handle? for each parser that will evaluate whether it can parse the specified data type would work just fine. Most of this could be automated as well, using the following parser class.
      class Parser
      # Finds a parser for a given data type
      def self.for(type)
      plugins.find {|p| p.handle? type}
      end

      # Stores an Array of plugins
      def self.plugins
      @plugins ||= []
      end

      # Registers a plugin
      def self.register_plugin(plug)
      plugins << plug unless plugins.include? plug
      end

      # Determines if this class can parse the data type
      # This should be overriden in subclasses
      def self.handle?(type)
      false
      end

      # Parses a document
      # This should be overriden by subclasses
      def self.parse(doc)
      nil
      end

      private

      # Macro to generate the handle? method
      def self.parses(*types)
      # Override the handle? method
      class << self
      def handle?(type)
      @handled_types.include? type
      end
      end
      @handled_types = (@handled_types || []).concat(types)
      # Register the plugin
      Parser.register_plugin self
      end
      end


      Defining a parser class is then as easy as:
      class Parser::Atom < Parser
      parses :atom

      def self.parse(doc)
      # ...
      end
      end


      Of course, if we decided we didn't actually want to inherit the Parser class, that's fine as well.
      class Parser::Rss
      def self.handle?(type)
      :rss == type
      end

      def self.parse(doc)
      # ...
      end
      end

      Parser.register_plugin Parser::Rss


      Most times we'd want to group all of our parsers in seperate files in another directory. We can modify the Parser class to require these files automatically for us.
      class Parser
      # Plugins are stored by default in the ./plugins directory
      PLUGIN_DIR = File.join(File.dirname(__FILE__), 'plugins')

      # Requires all of the files in the plugins directory
      def self.require_plugins
      $:.unshift(PLUGIN_DIR) unless $:.include? PLUGIN_DIR
      Dir::entries(PLUGIN_DIR).each {|f| require $1 if /^(.*)\.rb$/i =~ f}
      end

      # Finds a parser for a given data type
      def self.for(type)
      require_plugins if plugins.empty?
      plugins.find {|p| p.handle? type}
      end
      end


      After this, fetching a Parser class can be done via:

      Parser.for(:atom)


      If we put our plugins in the './plugins' directory of our project, the only thing we'd need to do after requiring the parser class is:

      Parser.for(:atom).parse(doc)


      The parser will look for plugins and then find one that parses atom and pass us that class so we can parse our feed.

      If you wanted to also register the plugin when inheriting, the Parser class can alsways be extended:

      class Parser
      def self.inherited(subclass)
      register_plugin subclass
      end
      end
    • John
       
      Posts: 2 / Nickname: cyent / Registered: December 4, 2005 1:55 PM
      Far too complex!
      December 4, 2005 7:09 PM      
      Your version is far far too complex. Make it much much simpler.

      File Parser.rb...


      =begin

      First off is you don't need Factory objects. You have got factory
      objects. They are called classes. The Factor pattern is way too heavy
      weight for most applications.

      Secondly you don't need YAML, you have Ruby.

      Thirdly your base component is too knowledgable. It knows
      (ParserFactory.load) where to find the concrete definitions of the
      Parsers. ie. you couldn't deploy and version the Parser's
      independently.

      Note this file 'Parser.rb' is completely ignorant of the existence of
      RDFParser and RSSParser.

      =end

      class Parser
      # A list of subclasses.
      @@klass_list = []

      # A Hash from type_name string to sub class of Parser.
      @@klass = Hash.new do |hash,key|
      hash[key] = @@klass_list.find{|klass| klass.type_name == key }
      end

      # More succinct than "parser_for"
      def Parser.[]( type_name)
      @@klass[type_name].new
      end


      # This is invoked too early. Namely when the class is created,
      # not when the type_name class method is defined. Thus we do not
      # know what the "type_name" is as opposed to the Class name.
      def Parser.inherited( klass)
      @@klass_list << klass
      end

      def Parser.each( &block)
      @@klass_list.each( &block)
      end

      def parse( xml)
      raise "Abstract class, please instantiate a concrete class"
      end


      end




      File RSSParser.rb


      require 'Parser'

      # Note this is completely ignorant of RDFParser and can be deployed
      # seperately or concurrently.

      class RSSParser < Parser
      def RSSParser.type_name
      'RSS'
      end

      def RSSParser.author
      'Dick'
      end

      def parse(xml)
      end
      end




      File RDFParser.rb


      require 'Parser'

      class RDFParser < Parser

      def RDFParser.type_name
      'RDF'
      end

      # Why introduce _yet_ another language (yaml) when we have Ruby?
      def RDFParser.author
      'Tom'
      end

      def parse(xml)
      end
      end




      Ok, so lets test it....

      File TC_Parser.rb

      Note this knows nothing about the concrete parsers...


      require 'test/unit'
      require 'Parser'

      class TC_Parser < Test::Unit::TestCase
      def test_parser
      Parser.each do |klass|
      type_name = klass.type_name
      puts "Testing #{type_name} "
      p = Parser[ type_name]
      assert_equal( type_name, p.class.type_name)
      end
      end
      end


      So lets run the test...

      ruby -w TC_Parser.rb
      Loaded suite TC_Parser
      Started
      .
      Finished in 0.004999 seconds.

      1 tests, 0 assertions, 0 failures, 0 errors


      Pretty boring, so plug in a plugin...


      ruby -w -rRSSParser TC_Parser.rb
      Loaded suite TC_Parser
      Started
      Testing RSS
      .
      Finished in 0.00397 seconds.

      1 tests, 1 assertions, 0 failures, 0 errors


      Plug in the other plug in...


      ruby -w -rRDFParser TC_Parser.rb
      Loaded suite TC_Parser
      Started
      Testing RDF
      .
      Finished in 0.004042 seconds.

      1 tests, 1 assertions, 0 failures, 0 errors


      Plug in both...

      ruby -w -rRSSParser -rRDFParser TC_Parser.rb
      Loaded suite TC_Parser
      Started
      Testing RSS
      Testing RDF
      .
      Finished in 0.005426 seconds.

      1 tests, 2 assertions, 0 failures, 0 errors


      Test a plugin...
      File TC_RSSParser.rb


      require 'test/unit'

      require 'RSSParser'

      class TC_RSSParser < Test::Unit::TestCase
      def test_info
      puts "Do we need YAML?"
      assert( 'Dick', Parser['RSS'].class.author)
      end
      end


      Run it...

      ruby -w TC_RSSParser.rb
      Loaded suite TC_RSSParser
      Started
      Do we need YAML?
      .
      Finished in 0.007161 seconds.

      1 tests, 1 assertions, 0 failures, 0 errors
      • ewr
         
        Posts: 2 / Nickname: tiger60 / Registered: December 9, 2005 9:20 AM
        Re: Far too complex!
        December 9, 2005 2:22 PM      
        I can't understand the part:

        @@klass = Hash.new do |hash,key|
        hash[key] = @@klass_list.find{|klass| klass.type_name == key }
        end


        Just works in your example but i can't understand why. Can you explain it to me?
        Thanks
        • ewr
           
          Posts: 2 / Nickname: tiger60 / Registered: December 9, 2005 9:20 AM
          Re: Far too complex!
          December 9, 2005 2:33 PM      
          After some googling i know why it's working. Looks like a new ruby adition since 1.8 to add the value if it's not found in the hash, i didn't know about it.
    • Michael
       
      Posts: 1 / Nickname: ged / Registered: September 27, 2005 6:03 PM
      Re: Modular Architectures with Ruby
      January 10, 2006 4:13 PM      
      There's also at least one module on the RAA that makes doing some of this stuff easier:

      http://raa.ruby-lang.org/project/pluginfactory/

      It comes in the form of a mixin that you can add to your base class to make it pluggable.
    • andresbube
       
      Posts: 1 / Nickname: andresbube / Registered: January 18, 2006 10:47 AM
      Re: Modular Architectures with Ruby
      January 18, 2006 3:57 PM      
      Nice article. This kind of article lets me map concepts between what I know and Ruby. I have a Java background but I'm just starting with Ruby.

      I understand the pattern you're trying to implement, but it seems a bit strange that there is no "Interface". Because of that you need to define an empty class returning nill on each method, something like an abstract class.

      It is very common to define an extensible architecture or and API based on Interfaces, it happens all the time in Java (i.e. JDBC, XML Parsers, etc...) and other OO languages. How is this supported by Ruby?