Data integration software speeds Web

Data progresses from one or more sources to one or more target tables, and/or message types (such as XML). The steps of the data movement involve identifying the sources from which data should be extracted, transformations the data should undergo, and where to send the data. Users specify data mappings and transformations through a graphical user interface.

User-defined processes control movement of each block of data and define interdependencies between such movements. For example, if one target table depends on values from other target tables, processes are used to specify the order in which a data server should run individual data movements that fill the tables.

Movements can be designed to run in batch or real-time mode, and are created and managed by administrators to control data movement between ERP, e-commerce, customer relationship management, supply-chain management, and legacy and messaging applications.

Data movement uses distributed query optimization, multithreading, in-memory caching, in-memory data transformations, and parallel pipe lining to deliver high data throughput and scalability. To manage the extraction process and perform batch data extraction from SAP software, for example, optimized ABAP code (SAP's proprietary programming language) is used, obviating the need to develop and maintain customized ABAP code.

Data cache is key

A cache serves as a single point of integration for enterprise and e-commerce data, minimizing the need for direct access to back-office systems and for complex real-time integration. The cache off-loads numerous, unnecessary requests for data from back-office systems, thus letting e-commerce firms scale to a greater number of users, while letting back-office systems do what they are designed to do.

Data integration software works in conjunction with products from enterprise application integration vendors and process integrators, not in lieu of them. Indeed, as data integration software becomes more pervasive as a tool used for business-to-business integration, it will dramatically reshape the way business-to-business integrators work together, as well as how businesses move to the Internet.

How it works