Article Discussion
Modular Architectures with Ruby
Summary: Any reasonably complex end-user application is going to require some sort of customization and enhancement for effective deployment. This article shows one way to create a modular architecture as a way of leaving the door open for advanced users or consultants who want to extend the functionality without modifying the source.
16 posts on 2 pages.      
« Previous 1 2 Next »
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: January 18, 2006 3:57 PM by
James
Posts: 9 / Nickname: jbritt / Registered: March 24, 2003 4:10 PM
Modular Architectures with Ruby
October 9, 2005 8:03 PM      
A modular architecture is one where the user can create modules that conform to well-described APIs and plug them into the application to extend the functionality. This article shows one way to create a modular API in Ruby.

Read this article by Jack Herrington, author of Code Generation in Action

http://www.artima.com/rubycs/articles/modular_apis_with_ruby.html

What did you think of Jack's article?
Michael
Posts: 2 / Nickname: mwhart / Registered: October 10, 2005 2:41 PM
Re: Modular Architectures with Ruby
October 10, 2005 6:51 PM      
I don't actually know very much about Ruby, but isn't there a cleaner way to retrieve a parser from a factory than the loop in the ParserFactory.parser_for method?

It seems to me that having to instanciate each factory and check if it matches the requested type is overkill - couldn't it be done in a hash of some sort (the factory could be added to the hash with the type as the key in the ParserFactory.inherited method), or even just checking a class variable/method (as opposed to an object one)?
Joe
Posts: 15 / Nickname: jcheng / Registered: October 16, 2002 8:08 AM
Re: Modular Architectures with Ruby
October 10, 2005 8:41 PM      
>> It seems to me that having to instanciate each factory and check if it matches the requested type is overkill - couldn't it be done in a hash of some sort (the factory could be added to the hash with the type as the key in the ParserFactory.inherited method), or even just checking a class variable/method (as opposed to an object one)? <<

I agree. Factory classes are basically a necessity in Java, where interfaces can't specify static methods and classes aren't objects. With Ruby, the former doesn't apply and the latter isn't true.

In fact, since all of the factories are essentially identical, a more Rubyish way to do it would be to create a module method that registers a parser without the need to write explicit factory code at all:

class RDFParser < Parser
ParserFactory::register(self, "RDF")

def parse( xml )
# Parse the XML up and return some known format
return nil
end
end


That should be *all* the code you need to write to implement and register a parser. (I didn't bother with the functionality on page 4 but it should be a straightforward extension.) Here's the complete code for ParserFactory:

class ParserFactory
@@factories = {}
def ParserFactory.register(factory, type)
@@factories[type] = factory
end
def ParserFactory.parser_for(type)
@@factories[type]
end
def ParserFactory.load( dirname )
Dir.open( dirname ).each { |fn|
next unless ( fn =~ /[.]rb$/ )
require "#{dirname}/#{fn}"
}
end
end
Joe
Posts: 15 / Nickname: jcheng / Registered: October 16, 2002 8:08 AM
Re: Modular Architectures with Ruby
October 10, 2005 8:48 PM      
Whoops, I got (at least) this method wrong:

  def ParserFactory.parser_for(type)
@@factories[type] && @@factories[type].new
end
Michael
Posts: 2 / Nickname: mwhart / Registered: October 10, 2005 2:41 PM
Re: Modular Architectures with Ruby
October 10, 2005 9:07 PM      
I like the idea of searching for a more Ruby-ish way, but I guess one possible problem with your approach, Joe, is that each parser is then dependent on the ParserFactory, which reduces their portability somewhat. The additional abstraction layer in the original design eliminates this dependency. It's probably up to your development needs whether this is acceptable or not.
Graham
Posts: 1 / Nickname: fosterg / Registered: October 11, 2005 5:57 AM
Re: Modular Architectures with Ruby
October 11, 2005 10:01 AM      
Thanks for the informative article. Its rather a new way of thinking for me. How would your example change if you were to use Needle (or other IOC module).. I'm still trying to get my head around these concepts.
Graham
Dave
Posts: 1 / Nickname: dws / Registered: May 9, 2003 7:02 AM
Re: Modular Architectures with Ruby
October 14, 2005 8:16 AM      
I'm wondering if the separate Factory class hierarchy carries its weight. By keeping a Hash in a class variable in the base Parser class to index each subclass by the type it services, and suppyling a class method to populate that hash, subclasses could do something along the lines of

class FooParser < Parser
parses :foo
...

which has more declarative value than overriding inherited. It's a simple step from there to

parser = Parser.for(filename)

This approach also opens the door for having a single parser class service multiple types, e.g.,

class HtmlParser < Parser
parses :htm, :html

Dave Thomas does something similar in RubLog.
Abhijit
Posts: 1 / Nickname: aabhijit / Registered: March 13, 2005 2:24 AM
Re: Modular Architectures with Ruby
November 6, 2005 6:59 AM      
I am new to this Ruby language but while trying out the example I found a ridiculous problem. I am not sure if it actually exists or not. So if someone could verify it for me I would be grateful.
In the rss.rb file I wrote the class definition as,

class RSSFactory < ParserFactory
INFO=<<INFO
type: RSS
author: Abhijit
description: An RSS Parser
INFO
...
end

And the compiler gave me some error about not finding INFO...Then I rewrote this as,

class RSSFactory < ParserFactory
INFO=<<INFO
type: RSS
author: Abhijit
description: An RSS Parser
INFO
...
end

And it worked!!
p
Posts: 1 / Nickname: p2 / Registered: November 14, 2005 9:40 PM
Re: Modular Architectures with Ruby
November 15, 2005 2:57 AM      
I don't see any differencies in yor code snippets... But I have the same problem.

I also getthis error in my test.rb:
undefined method `parse' for nil:NilClass (NoMethodError)
Clayton
Posts: 1 / Nickname: arton / Registered: November 15, 2005 8:18 AM
Re: Modular Architectures with Ruby
November 15, 2005 1:25 PM      
It is using heredoc syntax:
http://en.wikipedia.org/wiki/Heredoc

You don't want any whitespace preceding the closing INFO (or any other text between the INFOs you don't want to be indented).
John
Posts: 1 / Nickname: jbonnar / Registered: November 22, 2005 8:46 PM
Re: Modular Architectures with Ruby
November 23, 2005 2:30 AM      
Requiring factor methods to test and create a parser like this is a bit silly in Ruby. Defining a class method handle? for each parser that will evaluate whether it can parse the specified data type would work just fine. Most of this could be automated as well, using the following parser class.
class Parser
# Finds a parser for a given data type
def self.for(type)
plugins.find {|p| p.handle? type}
end

# Stores an Array of plugins
def self.plugins
@plugins ||= []
end

# Registers a plugin
def self.register_plugin(plug)
plugins << plug unless plugins.include? plug
end

# Determines if this class can parse the data type
# This should be overriden in subclasses
def self.handle?(type)
false
end

# Parses a document
# This should be overriden by subclasses
def self.parse(doc)
nil
end

private

# Macro to generate the handle? method
def self.parses(*types)
# Override the handle? method
class << self
def handle?(type)
@handled_types.include? type
end
end
@handled_types = (@handled_types || []).concat(types)
# Register the plugin
Parser.register_plugin self
end
end


Defining a parser class is then as easy as:
class Parser::Atom < Parser
parses :atom

def self.parse(doc)
# ...
end
end


Of course, if we decided we didn't actually want to inherit the Parser class, that's fine as well.
class Parser::Rss
def self.handle?(type)
:rss == type
end

def self.parse(doc)
# ...
end
end

Parser.register_plugin Parser::Rss


Most times we'd want to group all of our parsers in seperate files in another directory. We can modify the Parser class to require these files automatically for us.
class Parser
# Plugins are stored by default in the ./plugins directory
PLUGIN_DIR = File.join(File.dirname(__FILE__), 'plugins')

# Requires all of the files in the plugins directory
def self.require_plugins
$:.unshift(PLUGIN_DIR) unless $:.include? PLUGIN_DIR
Dir::entries(PLUGIN_DIR).each {|f| require $1 if /^(.*)\.rb$/i =~ f}
end

# Finds a parser for a given data type
def self.for(type)
require_plugins if plugins.empty?
plugins.find {|p| p.handle? type}
end
end


After this, fetching a Parser class can be done via:

Parser.for(:atom)


If we put our plugins in the './plugins' directory of our project, the only thing we'd need to do after requiring the parser class is:

Parser.for(:atom).parse(doc)


The parser will look for plugins and then find one that parses atom and pass us that class so we can parse our feed.

If you wanted to also register the plugin when inheriting, the Parser class can alsways be extended:

class Parser
def self.inherited(subclass)
register_plugin subclass
end
end
John
Posts: 2 / Nickname: cyent / Registered: December 4, 2005 1:55 PM
Far too complex!
December 4, 2005 7:09 PM      
Your version is far far too complex. Make it much much simpler.

File Parser.rb...


=begin

First off is you don't need Factory objects. You have got factory
objects. They are called classes. The Factor pattern is way too heavy
weight for most applications.

Secondly you don't need YAML, you have Ruby.

Thirdly your base component is too knowledgable. It knows
(ParserFactory.load) where to find the concrete definitions of the
Parsers. ie. you couldn't deploy and version the Parser's
independently.

Note this file 'Parser.rb' is completely ignorant of the existence of
RDFParser and RSSParser.

=end

class Parser
# A list of subclasses.
@@klass_list = []

# A Hash from type_name string to sub class of Parser.
@@klass = Hash.new do |hash,key|
hash[key] = @@klass_list.find{|klass| klass.type_name == key }
end

# More succinct than "parser_for"
def Parser.[]( type_name)
@@klass[type_name].new
end


# This is invoked too early. Namely when the class is created,
# not when the type_name class method is defined. Thus we do not
# know what the "type_name" is as opposed to the Class name.
def Parser.inherited( klass)
@@klass_list << klass
end

def Parser.each( &block)
@@klass_list.each( &block)
end

def parse( xml)
raise "Abstract class, please instantiate a concrete class"
end


end




File RSSParser.rb


require 'Parser'

# Note this is completely ignorant of RDFParser and can be deployed
# seperately or concurrently.

class RSSParser < Parser
def RSSParser.type_name
'RSS'
end

def RSSParser.author
'Dick'
end

def parse(xml)
end
end




File RDFParser.rb


require 'Parser'

class RDFParser < Parser

def RDFParser.type_name
'RDF'
end

# Why introduce _yet_ another language (yaml) when we have Ruby?
def RDFParser.author
'Tom'
end

def parse(xml)
end
end




Ok, so lets test it....

File TC_Parser.rb

Note this knows nothing about the concrete parsers...


require 'test/unit'
require 'Parser'

class TC_Parser < Test::Unit::TestCase
def test_parser
Parser.each do |klass|
type_name = klass.type_name
puts "Testing #{type_name} "
p = Parser[ type_name]
assert_equal( type_name, p.class.type_name)
end
end
end


So lets run the test...

ruby -w TC_Parser.rb
Loaded suite TC_Parser
Started
.
Finished in 0.004999 seconds.

1 tests, 0 assertions, 0 failures, 0 errors


Pretty boring, so plug in a plugin...


ruby -w -rRSSParser TC_Parser.rb
Loaded suite TC_Parser
Started
Testing RSS
.
Finished in 0.00397 seconds.

1 tests, 1 assertions, 0 failures, 0 errors


Plug in the other plug in...


ruby -w -rRDFParser TC_Parser.rb
Loaded suite TC_Parser
Started
Testing RDF
.
Finished in 0.004042 seconds.

1 tests, 1 assertions, 0 failures, 0 errors


Plug in both...

ruby -w -rRSSParser -rRDFParser TC_Parser.rb
Loaded suite TC_Parser
Started
Testing RSS
Testing RDF
.
Finished in 0.005426 seconds.

1 tests, 2 assertions, 0 failures, 0 errors


Test a plugin...
File TC_RSSParser.rb


require 'test/unit'

require 'RSSParser'

class TC_RSSParser < Test::Unit::TestCase
def test_info
puts "Do we need YAML?"
assert( 'Dick', Parser['RSS'].class.author)
end
end


Run it...

ruby -w TC_RSSParser.rb
Loaded suite TC_RSSParser
Started
Do we need YAML?
.
Finished in 0.007161 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
ewr
Posts: 2 / Nickname: tiger60 / Registered: December 9, 2005 9:20 AM
Re: Far too complex!
December 9, 2005 2:22 PM      
I can't understand the part:

@@klass = Hash.new do |hash,key|
hash[key] = @@klass_list.find{|klass| klass.type_name == key }
end


Just works in your example but i can't understand why. Can you explain it to me?
Thanks
ewr
Posts: 2 / Nickname: tiger60 / Registered: December 9, 2005 9:20 AM
Re: Far too complex!
December 9, 2005 2:33 PM      
After some googling i know why it's working. Looks like a new ruby adition since 1.8 to add the value if it's not found in the hash, i didn't know about it.
Michael
Posts: 1 / Nickname: ged / Registered: September 27, 2005 6:03 PM
Re: Modular Architectures with Ruby
January 10, 2006 4:13 PM      
There's also at least one module on the RAA that makes doing some of this stuff easier:

http://raa.ruby-lang.org/project/pluginfactory/

It comes in the form of a mixin that you can add to your base class to make it pluggable.
16 posts on 2 pages.
« Previous 1 2 Next »