Tuesday, September 13, 2011

HP and the Data Processing Landscape


I came across an interesting article by Gartner on "Hadoop and MapReduce: Big Data Analytics". I've took the long article and summarized it with the above diagram. I've also charted on it the position of my company, HP, in regards to this new and evolving landscape, with its recent acquisitions of Vertica and Autonomy.
The main points are the extensions beyond the traditional RDBMS systems toward intelligent and personal handling of events (Complex Event Processing - CPE) and toward handling Extreme Data for strategic decision. Note that I'm using here the term "Extreme Data" and not simply "Big Data", to emphasis that this new type of data handling is not only about data volume.
The spider chart on the top left is showing the main differences between the "Old" types of OLTP (Online Transaction Processing), ODS (Operational Data Store) and EDW (Enterprise Data Warehouse) and the "New" types of CPE and Extreme Data.
The Topic of Unstructured Data is also very important and the break-down of the types and levels of unstructured data is highlighted on the table on the middle right.
Another interesting topic that I would like to point out with the diagram is that "the world is round". There are many similarities between the two extremes of the data processing continuum. Many of the analysis that are applied on the Extreme Data part should be fed into the CEP to make event processing truly intelligent. The basic example of crawling the web and creating an index on one hand is used to respond immediately to user queries on the other. The former part belongs to Extreme Data and the latter to CEP, although they are tied strongly together. The strategic analysis can be used internally by enterprise or can be used for the real time event processing part of the organization (call center, customer web site...).
The diagram also shows the place for Hadoop technologies to handle many of the Extreme Data challenges, and the main parts of this stack that are relevant to the enterprise issues.

No comments:

Post a Comment