Smart Data Frameworks

A database migration and ELT development platform. Migrate between virtually any database platform, build and manage reusable ELT logic, and keep source and target in continuous sync — delivered as a self-contained virtual appliance.

What is SDF?

A Migration Platform, Not Just a Migration Script

Moving a database from one platform to another is one of the most risky, labour-intensive projects a data team can undertake. Done manually, it involves weeks of schema conversion, data type mapping, ELT rewriting, testing, and coordinated cutover — with every step dependent on specialist knowledge that most teams need to hire in.

Smart Data Frameworks automates the repeatable parts of this process. It ships as a flatpack virtual machine that you deploy in your own environment. It connects to your source and target databases via ODBC, provides a browser-based GUI for all migration and development tasks, and includes a CLI for scripted and scheduled operations.

SDF covers the full migration lifecycle: initial schema conversion, bulk data load, ongoing delta synchronization, flat file loading, ELT code development, and controlled promotion of code between environments. It also supports Teradata SQL syntax compatibility, which simplifies workloads for teams migrating away from Teradata.

What SDF Handles

  • Schema Conversion & Migration: Automated DDL conversion and bulk data migration between any supported platforms
  • Delta Synchronisation: Keep source and target in continuous sync after the initial migration
  • ELT Code Development: Template-driven generation of ELT and data quality detection logic
  • Flat File Loading: Define file layouts and run continuous loading pipelines from structured files
  • Release Management: Controlled promotion of code and database objects between dev, test, and production

The SDF Platform at a Glance

SDF: Database Explorer

Browser-based database explorer with ODBC connections to any platform

Six Components

What SDF Provides

Automated Database Conversion & Migration Wizard

Guides you through the end-to-end process of converting and migrating a database schema from one platform to another. Handles DDL conversion, data type mapping, and the orchestration of bulk data load — with reconciliation to verify source and target are in agreement on completion.

Web Browser-Based Database Explorer

A full database browser and SQL query tool delivered entirely in the browser. Connect to any ODBC-accessible database, explore schemas, tables, and column definitions, and run SQL directly — without installing client software on each user’s machine.

File Layout Definition & Continuous Loading Wizard

Define the layout of structured flat files (delimited, fixed-width) and configure them to load continuously into target database tables. Handles the ongoing ingestion of file-based data sources as part of a broader migration or integration pipeline.

Integrated Release Management

Manage and promote database objects, ELT code, and configuration between development, test, and production environments in a controlled, auditable way. Reduces the risk of inconsistent or untested changes reaching production and provides a clear record of what was deployed and when.

Customisable Code Template-Driven Development

Automatically generate ELT logic and data quality detection code from a library of customizable templates. Rather than hand-coding transformations for each table, templates enforce consistency, reduce errors, and dramatically accelerate ELT development across large schemas.

Parameter Substitution for Template Reusability

Templates are fully parameterised — the same template can generate correct ELT code for hundreds of different tables, schemas, or target environments simply by varying the parameters. This keeps the template library small and maintainable whilst scaling to large, complex migrations.

Platform Support

Connects to Virtually Any Database Platform

SDF connects source and target databases via ODBC, which means it works with any platform that provides an ODBC driver. Supported platforms include:

  • IBM® Netezza® / PDASnowflake
  • Amazon Redshift
  • Yellowbrick
  • Oracle
  • SQL Server
  • Teradata
  • MySQL
  • Greenplum
  • PostgreSQL
  • Google Bigtable
  • IBM Db2

SDF also supports Teradata SQL syntax compatibility, simplifying migration for teams moving Teradata workloads to a new platform.

Who Uses SDF

Built for the People Who Do the Migration Work

DBAs and Migration Architects

The Migration Wizard and Database Explorer give DBAs the tools to plan, execute, and validate database migrations at the schema and data level — with reconciliation built in so nothing gets missed.

Data Engineers

The code template system and flat file loading wizard significantly reduce the manual ELT coding required during a migration. Templates enforce consistency across hundreds of tables and generate quality detection logic automatically.

Release and Delivery Teams

Integrated Release Management gives teams a controlled, auditable mechanism for promoting database objects and ELT code through dev, test, and production — without ad-hoc scripts or manual coordination.

Common Questions

The Platform

  • What is the Smart Data Framework (SDF) from Smart Associates?

The Smart Data Framework (SDF) is a virtual appliance from Smart Associates that provides database development, migration, and operations tooling for enterprise data warehouses. It includes six components — Migration Wizard, DB Explorer, File Loader, Release Management, Code Templates, and Parameter Substitution — supporting over 12 database platforms.

  • What database platforms does the Smart Data Framework support?

SDF from Smart Associates supports over 12 database platforms, including IBM Netezza, PostgreSQL, Amazon Redshift, Google BigQuery, Snowflake, Microsoft Azure Synapse, Oracle, IBM Db2, Teradata, Greenplum/Cloudberry, Databricks, and SQL Server.

  • What is the SDF Migration Wizard?

The SDF Migration Wizard is a guided migration tool within Smart Associates Smart Data Framework that automates schema migration, data transfer, and SQL dialect translation between database platforms. It handles schema mapping and data type conversion, significantly reducing the manual effort of a database migration project.

  • What is DB Explorer in the Smart Data Framework?

DB Explorer is the database discovery and documentation component of Smart Associates SDF. It connects to a source database and generates a catalogue of objects — tables, views, stored procedures, schemas — with metadata, lineage information, and dependency mapping. This is particularly useful at the start of migration or modernisation projects.

The Six Components

  • What is the File Loader component of the Smart Data Framework?

SDF’s File Loader is a data ingestion tool that loads structured files (CSV, delimited, fixed-width) into target database tables. It handles encoding, delimiter detection, error handling, and load auditing, providing a managed alternative to native bulk load utilities.

  • How does SDF Release Management work for database code?

Smart Associates SDF Release Management provides version control and deployment pipeline capabilities for database code (DDL, DML, stored procedures). It tracks changes, manages environments (development, test, production), and automates controlled, auditable database releases.

  • What are Code Templates in the Smart Data Framework?

Code Templates in Smart Associates SDF are pre-built, parameterised SQL patterns for common database operations — such as slowly changing dimensions, data quality checks, aggregation patterns, and incremental load logic. They accelerate development and enforce consistent coding standards across teams.

  • What is Parameter Substitution in SDF?

Parameter Substitution in Smart Associates SDF allows SQL code to be written once and executed across different environments by substituting environment-specific values (connection strings, schema names, table prefixes) at runtime. This eliminates the need to maintain separate SQL copies for development, test, and production.

Getting Started

  • Is the Smart Data Framework suitable for Netezza migration projects?

Yes. Smart Associates SDF was developed with Netezza migration in mind. The Migration Wizard, DB Explorer, and Code Templates are all designed to accelerate the assessment, translation, and testing phases of moving from Netezza to a modern cloud or on-premises platform.

  • How is the Smart Data Framework delivered?

Smart Associates SDF is delivered as a virtual appliance — a pre-configured virtual machine image that can be deployed in any virtualisation environment (VMware, Hyper-V, or cloud VM). There is no complex installation; the appliance is imported and connected to the target database environment.

  • Can SDF be used for ongoing ETL operations, not just migration projects?

Yes. While SDF is frequently used for migration projects, its File Loader, Release Management, and Code Templates components are equally applicable to ongoing data operations — regular data ingestion, database code version control, and development standardisation in production environments.

  • How do I get the Smart Data Framework?

Contact Smart Associates via smart-associates.biz/products/sdf/ to request a demonstration or discuss licensing. SDF is available as a standalone product and as part of Smart Associates’ data migration and modernisation service offerings.