This post originated from an RSS feed registered with .NET Buzz by Ranjan Sakalley. Original Post: Product design and integration Feed Title: Ranjan Sakalley Feed URL: /error.htm?aspxerrorpath=/blogs/ranjan.sakalley/rss.aspx Feed Description: (the life and times of a mock object) Latest .NET Buzz Posts Latest .NET Buzz Posts by Ranjan Sakalley Latest Posts From Ranjan Sakalley

What is integration? And where does it come into picture?

 Suppose you create a great accounting package, for SMEs. Its very good, got good performance, great UI, the clerks love it, the ladies go nuts over it, if looks could kill and all that, you know. Now we understand that this is not a killer app, because there are many accounting packages out there in the market, many from big and dependable corporations like MicroSoft, many well established like Quickbooks, and many much simpler than yours. But you have a great Sales and Marketing team that really toils hard, and sells a thousand copies. You will now, definitely face one of the following scenarios, if not all of them -

 1. You Sales and Marketing girl comes back and tells you that most of the people that she talks to about our product, say that they want an accounting package which can take in inputs from their web-site too. Ours doesn’t support that, so they don't even talk about it any further. Then there are people (Prospects) who have some partners out there in business, providers of raw-materials etc., and the partners have, let’s say, QuickBooks installed on their site, and the prospects want to get rid of paperwork, and want the accounting package they buy, to talk directly to their partners' package. So these guys don't talk to the poor (well yes, there are such rare situations, where you can use the word poor as an adjective for Sales people) lady, and she is now getting red at you, for not thinking of such situations when designing the package.

2.  You identify a liquor store as a major prospect, you visit them personally, try to impress them with your tech-talk and all. But they counter everything with some of their own. You tell them about your package, and they start asking you about how your package would work with their existing infrastructure. They start talking about ESBs, BizTalk and all, and you take your sad face home, thinking what went wrong, where are good old stupid liquor store managers going these days.

3. One of your clients grows really fast, and is generally happy about the software that you gave him. But now, he needs an Inventory Management solution too. So he buys one, but later finds out that instead of improving his inventory management, installing this new software has increased the paper-work, and along with it, chaos in the office and his staff productivity has decreased to half. He digs deep to find out that this is because of the technical incompatibility between the accounting package you wrote, and the inventory management software he bought. He is in an indecision mode, and during this period, the company that sold him the inventory management software calls him up, and tells him about the new accounting package they are writing, which seamlessly(the regular sales talk, you know) integrates with their other softwares. The client calls to tell you his decision, for him the decision is great, but for you? The story doesn't end here, this guy goes and posts a recommendation on a web-site, and now the number of distracted clients is really growing, and your business is going down. Would a new version really help in this scenario?

In any case, with almost all enterprise products, and even some standalone ones, you would encounter such cases. This is why software’s ability to provide an integration API, or its ability to seamlessly become a part of an integrated infrastructure of products is important, it’s the need of the hour, and it’s this integration-ready software which would sell. This is the main reason for major players chiming SOA mantras in every major event, on every technical website.

How do you go about the design decision?

Following are three good ways to find out about how you should actually go ahead.

1. If you are in the process of designing an enterprise level solution or for that matter any other solution, research the market to find out the prospect users. Talk to them, before designing the product, and not after the development is done. Ask them about what issue they have faced with paperwork between different departments. Ask them if they are looking for software that does not require manual intervention for communication of data between the software their other departments use. Next, ask your prospect customers what kind of "other" software would they like your software to work with. In other words, research on the environment in which your software would be deployed.

2. If you already have a partner in the market, who complements your line of products with its own, ask them if they want their products to synergize with yours. Ask them what their software would need to "talk" to yours. Find out what would your software need to "talk" to theirs.

3. Do you want to improve your future business, by providing APIs/Platforms for integration today? When you grow with an accounting package (for example), do you want to write and sell an inventory management system that talks to the latter? What happens when you write a shipping management system? In other words, does your vision extend to a scenario where this product you are writing is just the first in the line of a complete enterprise solution in future.

 Implementation

 Now integration is definitely not anything new, and almost all enterprise software vendors have faced customer demands around it in the past. Some have solved it concentrating on the state storage system, some using messaging, and there are other ways too.

 Integration based on state storage systems

This is a very fragile architecture, wherein when some data is changed in a software's domain, this software indicates other "integrated" software by either writing the data to a file or invoking a database trigger.

 Messaging

Software sends messages to a sink, which other software read to update their data, according to the message. This is a very good and agile mechanism. Unfortunately, traditional messaging techniques are more or less platform or OS specific.

 XML based messaging and SOA

This is an extension of the previous, but because of the use of XML, this makes it possible to integrate with a wide range of products. Morover, a good service oriented architecture enables other software systems to indicate data/state change to yours, and thus provides a way to complete integration. My team is freshly through the integration phase, where we integrated an EMR system, running on a Unix machine and exposing a Java web-service, with one of our client's .Net based medical transcription products. There are many ISVs that specialize on integration, some use Enterprise Service Bus(ESB) products (mostly Java), while Microsoft is taking integration forward with Biztalk.

 
I would therefore strongly suggest SOA, if and when you decide on your integration strategy. And yes, don't forget to put that paragraph on why or why not you are considering integration, in your design document. Follow this up with a good strategy (if yes), based on strong market research. There are some good books and web-sites in the market for Enterprise Application Integration (EAI), EAI Patterns etc. which would definitely help you find out the correct integration strategy. Just open your eyes.