Tuesday, March 17, 2009

Data Integration, Propagate Deletes

Enterprise Data Integration is challenging subject. Many of my customers have problems of the sort where they have pieces of information that are being collected by disparate systems. I try to push things towards simplicity, but there are many angles by which complexity sneaks in.

Think about something like Google Reader. Publish/subscribe works simply and well for this sort of content aggregation. However, the behavior of the client is not guaranteed. For example, what happens if I want to delete an entry? Or update an entry? Google Reader simply doesn't care if the publisher wants to delete an entry. It's already copied it into your data. Having an entry go missing from a data feed doesn't necessarily indicate an intention that a downstream deletion should occur. 

Can a simple approach work for data integration? Perhaps, but there are a lot of tricky issues to work around. 

blog comments powered by Disqus