Category - Core component

For Suites: part of Java CAPS and MDM Suite

Sun Data Integrator

Sun Data Integrator provides connectivity to a vast range of heterogeneous and diversified data sources including non relational data sources. It provides an ETL development and runtime environment that is fully integrated into OpenESB and NetBeans and optimized for handling very large record sets.

Sun Data Integrator R6

Note: Features and notes in bold below are new in Release 6

Features

• JBI standard based Service Engine
• JMX controls for dynamic configuration
• Intuitive Design time Netbeans Plug-in with drag-n-drop capabilities
• BPEL 2.0 invocation support
• Plugs seamlessly in a SOA Environment
• Rich GUI support for Building Joins, Source Extraction and Target conditions
• Configurable ETL/ELT strategies
• Rich Operator palette which features Data cleansing functions like Standardization, normalization, parse Address, Matching etc.
• Ability to source data from Web pages(HTML tables), RSS Feeds, spreadsheets etc.
• Require little database expertise to build high performing ETL processes.
• Metadata auto discovery enables user to design ETL processes faster.
• Take advantage of database bulk, no-logging tuning where applicable for faster data warehouse loads.
• Support for creating auto join based PK-FK relationship, create code to ensure data integrity.
• Take advantage of database engine by pushing as much of the workload on to the target/source database.
• Support Extensive non relational data formats
• Transform, filter and sort at the source where appropriate.
• Support Data cleansing operators to ensure data quality. A dictionary driven system for complete parsing of names and addresses of individuals and organizations, products and locations. Support for Normalize and de-normalize data.
• Support to convert data into a consistent, standardized form to enable load to conformed target databases.
• Support for In built data integrity checks.
• Support for data type conversion
• Support for Null value handling
• Support for Customized transformation
• Robust Error handler to ensure data quality. Comprehensive system for reporting and responding to all DI error events.
• Support for change management functions or "versioning".
• Support for concurrent/parallel processing of multiple source data streams.
• Support for Extraction type: Full refresh and incremental extraction.
• Support complete development environment, DI Editor is fully integrated with NetBeans
• Support for Data Federation that enables user to use SQL as language to define ETL Processes.
• Support for near real-time click-stream data warehousing (in conjunction with JDBC BC)
• Support for ERP/CRM data sources in conjunction with various components from Java CAPS
• Platform independence and scalability to enterprise data warehousing applications
• Ability to specify complex transformations using built-in transformation objects.
• Ability to schedule DI sessions on time or the occurrence of a specified event (in conjunction with Java CAPS components)
• Ability to orchestrate Business Process where DI can participate as partner. DI exposes it as web service.
• Ability to extract data from outside firewall in conjunction with FTP and HTTP Connectors.
• Support for the analysis of transformations that failed or rejected and then resubmit them after correcting the data.
• Extensive reporting of the results of an DI session, including automatic notification of significant failures of the DI process.

Data Sources supported

• PostgreSQL
• MySQL
• Derby
• Oracle (Oracle 8 and higher)
• Sybase
• DB2 (Version 5 and higher)
• SQL Server
• Axion
• Spreadsheets
• HTML/Web Tables
• RSS/ATOM feeds
• Flat files (CSV, Delimited and Fixed Width)
• Other Databases like Access, FoxBase etc. through JDBC driver.

External System Support/Dependencies

Data Integrator has dependence on following third party libraries:
1. axiondb.jar
2. wsdl4.jar
3. commons-logging.jar
4. commons-codec.jar
5. commons-primitive.jar

Upgrade Considerations

Data Transformation Operators

• Date transformation operators provides for converting between different date formats, e.g. StringToDate, AddToDate, etc.
• String operators provides for processing strings during data transformation, e.g. Concat, LeftTrim, etc.
• SQL operators are used to perform standard ANSI SQL based operators during data transformations, e.g. Castas, Count, etc.
• Numerical operators provides for processing numerical values during data transformation, e.g., Absolute, Mod, etc.
• Custom operators can be configured for transforming data, augment extract and update conditions

Product Dependencies