SQLines Data - Database Migration and ETL

SQLines Data is an open source (Apache License 2.0), scalable, parallel high performance data transfer and schema conversion tool that you can use for database migrations and ETL processes.

  • Oracle and Oracle Exadata
  • Microsoft SQL Server and Microsoft Azure SQL
  • MySQL
  • MariaDB
  • PostgreSQL
  • Amazon Redshift, Amazon Aurora and Amazon RDS
  • Apache Hive and Presto
  • IBM DB2 LUW, iSeries (AS/400) and zSeries (S/390)
  • Sybase Adaptive Server Enterprise and Sybase SQL Anywhere
  • Informix
  • Teradata
  • Netezza
  • Greenplum
  • Vertica
  • SAP HANA

The tool also supports parallel high performance export to flat files, S3, HDFS using various formats and compressions.

SQLines Data is written in C/C++ and uses native low-level in-memory bulk loader APIs to transfer data. It is available for Linux, Windows, both 64-bit and 32-bit platforms.

Database Migration

You can use SQLines Data for cross-platform database migration. The tool migrates table definitions, constraints, indexes and transfers data.

Migration Features:

  • Options to create schema, transfer data, create constraints and indexes to set up flexible migration process
  • UI and command line interface

ETL

You can use SQLines Data in your ETL processes as a data extact-load tool.

ETL Features:

  • Parallel high performance data transfer to target databases
  • Parallel high performance export to flat files, S3 or HDFS supporting compression
  • Choosing columns to transfer
  • Defining SELECT expressions including computed columns
  • Defining WHERE conditions for tables for incremental transfer
  • Defining SQL SELECT queries to transfer data
  • Command line interface

How To

How to Transfer Tables without Constraints and Indexes

Sometime you do not need to move constraints and indexes into the target database. For example, in case when you move data for reporting purpose, or if you want to migrate constraints and indexes later.

Edit sqldata.cfg configuration file, and set -constraints=no and -indexes=no.