This is an old revision of the document!


SQLines Data - Database Migration and ETL

SQLines Data is an open source (Apache License 2.0), scalable, parallel high performance data transfer and schema conversion tool that you can use for database migrations and ETL processes.

  • Oracle and Oracle Exadata
  • Microsoft SQL Server and Microsoft Azure SQL
  • MySQL
  • MariaDB
  • PostgreSQL
  • Amazon Redshift, Amazon Aurora, Amazon RDS and Amazon Athena
  • IBM DB2 LUW, iSeries (AS/400) and zSeries (S/390)
  • Sybase Adaptive Server Enterprise, Sybase SQL Anywhere, Sybase IQ and Sybase Advantage
  • Informix
  • Teradata
  • Netezza
  • Greenplum
  • Vertica
  • SAP HANA

The tool also supports parallel high performance export to flat files, S3, HDFS using various formats and compressions.

SQLines Data is written in C/C++ and uses native low-level in-memory bulk loader APIs to transfer data. It is available for Linux, Windows, both 64-bit and 32-bit platforms.

SQLines Data guides for specific database migrations:

MariaDB Oracle SQL Server MySQL
Sybase ASE to MariaDB IBM DB2 to Oracle Oracle to SQL Server Oracle to MySQL
Oracle to MariaDB Sybase ASE to Oracle

Database Migration

You can use SQLines Data for cross-platform database migration. The tool migrates table definitions, constraints, indexes and transfers data.

Migration Features:

  • Options to create schema, transfer data, create constraints and indexes to set up flexible migration process
  • UI and command line interface

ETL

You can use SQLines Data in your ETL processes as a data extact-load tool.

ETL Features:

  • Parallel high performance data transfer to target databases
  • Parallel high performance export to flat files, S3 or HDFS supporting compression
  • Choosing columns to transfer
  • Defining SELECT expressions including computed columns
  • Defining WHERE conditions for tables for incremental transfer
  • Defining SQL SELECT queries to transfer data
  • Command line interface

How To

How to Transfer Tables without Constraints and Indexes

Sometime you do not need to move constraints and indexes into the target database. For example, in case when you move data for reporting purpose, or if you want to migrate constraints and indexes later.

Edit sqldata.cfg configuration file, and set -constraints=no and -indexes=no.