Data Movement Activities¶

Copy, Lookup, and the typed source, sink, and dataset helpers that support them.

Copy¶

fabric_data_pipelines.Copy ¶

Bases: Activity

Copy data from a source to a sink.

Example::

from fabric_data_pipelines import Copy, SqlMISource, SqlMISink, AzureSqlMITable, expr

copy = Copy(
    name="Copy_customers",
    source=SqlMISource(
        sql_reader_query="SELECT * FROM dbo.customers",
        partition_option="None",
        dataset_settings=AzureSqlMITable(
            database="SourceDb",
            connection=expr.library_variable("SourceDb"),
        ),
    ),
    sink=SqlMISink(
        write_behavior="insert",
        sql_writer_use_table_lock=True,
        dataset_settings=AzureSqlMITable(
            database="Landing",
            schema_name="sales",
            table="customers",
            connection=expr.library_variable("Landing"),
        ),
    ),
    enable_staging=False,
)

fabric_data_pipelines.CopySource ¶

Bases: _ConnectorModel

Generic copy source escape hatch.

Example::

CopySource(type="SqlMISource", sql_reader_query="SELECT 1", dataset_settings=...)

fabric_data_pipelines.CopySink ¶

Bases: _ConnectorModel

Generic copy sink escape hatch.

Example::

CopySink(type="SqlMISink", write_behavior="insert", dataset_settings=...)

fabric_data_pipelines.SqlMISource ¶

Bases: CopySource

Azure SQL Managed Instance copy source.

sql_reader_query accepts a plain SQL string or a Fabric Expression object ({"value": "...", "type": "Expression"}), matching UI exports that bind the query from pipeline variables.

Example::

SqlMISource(
    sql_reader_query="SELECT * FROM dbo.customers",
    dataset_settings=AzureSqlMITable(
        database="SourceDb",
        connection=expr.library_variable("SourceDb"),
    ),
)

fabric_data_pipelines.SqlMISink ¶

Bases: CopySink

Azure SQL Managed Instance copy sink.

fabric_data_pipelines.LakehouseTableSource ¶

Bases: CopySource

Lakehouse table copy source.

fabric_data_pipelines.LakehouseTableSink ¶

Bases: CopySink

Lakehouse table copy sink.

fabric_data_pipelines.DataWarehouseSource ¶

Bases: CopySource

Fabric Data Warehouse copy source.

sql_reader_query accepts a plain SQL string or a Fabric Expression object, same as :class:SqlMISource.

fabric_data_pipelines.DataWarehouseSink ¶

Bases: CopySink

Fabric Data Warehouse copy sink.

fabric_data_pipelines.StagingSettings ¶

Bases: FabricModel

Interim staging settings when enable_staging is true.

fabric_data_pipelines.TabularTranslator ¶

Bases: FabricModel

Tabular translator with optional column mappings.

Example::

TabularTranslator(
    mappings=[
        ColumnMapping(
            source=ColumnRef(name="id", type="String", physical_type="nvarchar"),
            sink=ColumnRef(name="id", type="String", physical_type="nvarchar"),
        )
    ],
    type_conversion=True,
    type_conversion_settings=TypeConversionSettings(
        allow_data_truncation=True,
        treat_boolean_as_number=False,
    ),
)

fabric_data_pipelines.ColumnMapping ¶

Bases: FabricModel

A single source→sink column mapping.

fabric_data_pipelines.ColumnRef ¶

Bases: FabricModel

A column reference inside a TabularTranslator mapping.

fabric_data_pipelines.TypeConversionSettings ¶

Bases: FabricModel

Advanced type-conversion settings for a TabularTranslator.

Lookup¶

fabric_data_pipelines.Lookup ¶

Bases: Activity

Retrieve a row (or rows) for use by subsequent activities.

Example::

from fabric_data_pipelines import Lookup, LakehouseTableSource, LakehouseTable

lookup = Lookup(
    name="get_tables",
    source=LakehouseTableSource(),
    dataset_settings=LakehouseTable(table="lh2", linked_service=...),
    first_row_only=True,
)

Datasets and linked services¶

fabric_data_pipelines.Dataset ¶

Bases: DatasetSettings

Generic dataset escape hatch for any Fabric dataset type.

Example::

Dataset(type="DelimitedText", type_properties={"location": {...}})

fabric_data_pipelines.DatasetSettings ¶

Bases: FabricModel

Base dataset settings with Fabric UI boilerplate.

Always emits annotations: [] and schema: [] so generated JSON matches Fabric UI exports. Fabric sometimes exports an empty schema as {}; that is normalized to [] on import.

fabric_data_pipelines.LakehouseTable ¶

Bases: DatasetSettings

Lakehouse table dataset.

Example::

LakehouseTable(
    table="sales",
    linked_service=LinkedService(
        name="MyLakehouse",
        properties=LinkedServiceProperties(
            type="Lakehouse",
            type_properties={
                "workspaceId": "...",
                "artifactId": "...",
                "rootFolder": "Tables",
            },
        ),
    ),
)

fabric_data_pipelines.AzureSqlMITable ¶

Bases: DatasetSettings

Azure SQL Managed Instance table dataset.

Example::

AzureSqlMITable(
    database="Landing",
    schema_name="sales",
    table="customers",
    connection="@pipeline().libraryVariables.Landing",
)

fabric_data_pipelines.AzureSqlTable ¶

Bases: DatasetSettings

Azure SQL Database table dataset.

fabric_data_pipelines.DataWarehouseTable ¶

Bases: DatasetSettings

Fabric Data Warehouse table dataset.

fabric_data_pipelines.LinkedService ¶

Bases: FabricModel

A named linked service or connectionSettings embedded in dataset settings.

Example::

LinkedService(
    name="LakehouseGitArtifactW1",
    properties=LinkedServiceProperties(
        type="Lakehouse",
        type_properties={
            "workspaceId": "...",
            "artifactId": "...",
            "rootFolder": "Tables",
        },
    ),
)

fabric_data_pipelines.LinkedServiceProperties ¶

Bases: FabricModel

Inner properties block of a Fabric linked service / connection reference.