Class DbMerge<TInput>

Inserts, updates and (optionally) deletes data in database target. Before the Merge is executed, all data from the destination is read into memory. A delta table is generated that stores information if a records was inserted, updated, deleted or hasn't been touched (existed).

Inheritance
object
DataFlowTransformation<TInput, TInput>
DbMerge<TInput>
Inherited Members
object.Equals(object)
object.Equals(object, object)
object.GetHashCode()
object.GetType()
object.MemberwiseClone()
object.ReferenceEquals(object, object)
object.ToString()
Namespace: ETLBox.DataFlow.Connectors
Assembly: ETLBox.dll
Syntax
    public class DbMerge<TInput> : DataFlowTransformation<TInput, TInput>, ILoggableTask, IDataFlowLogging, IDataFlowTransformation<TInput, TInput>, IDataFlowSource<TInput>, IDataFlowSource, IDataFlowBatchDestination<TInput>, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent, IDataFlowBatchDestination
Type Parameters
NameDescription
TInput

Type of ingoing data.

Constructors

DbMerge()

Declaration
    public DbMerge()

DbMerge(IConnectionManager, string, int)

Declaration
    public DbMerge(IConnectionManager connectionManager, string tableName, int batchSize)
Parameters
TypeNameDescription
IConnectionManagerconnectionManager
stringtableName
intbatchSize

DbMerge(IConnectionManager, string)

Declaration
    public DbMerge(IConnectionManager connectionManager, string tableName)
Parameters
TypeNameDescription
IConnectionManagerconnectionManager
stringtableName

DbMerge(string, int)

Declaration
    public DbMerge(string tableName, int batchSize)
Parameters
TypeNameDescription
stringtableName
intbatchSize

DbMerge(string)

Declaration
    public DbMerge(string tableName)
Parameters
TypeNameDescription
stringtableName

Properties

BatchSize

The batch size used when inserted data into the database table.

Declaration
    public int BatchSize { get; set; }
Property Value
TypeDescription
int

CacheMode

The cache mode used for reading data from the destination table. By default, all data is read into memory before the actual merge is executed (Full mode). If set to partial, only data is read into memory that is needed for the current merge comparison. This will keep the memory consumption of the DbMerge much smaller.

Declaration
    public CacheMode CacheMode { get; set; }
Property Value
TypeDescription
CacheMode

ColumnConverters

Use a column converter to apply a conversion function to each value of a column before writing into the target

Declaration
    public ICollection<ColumnConverter> ColumnConverters { get; set; }
Property Value
TypeDescription
System.Collections.Generic.ICollection<T><ColumnConverter>

ColumnMapping

Column mapping to map property to column names. E.g. if the value of property "Id" should be written into the database column "Key", then you can simply add a column mapping: DbColumnName: "Id" -> PropertyName: "Key".

Declaration
    public ICollection<ColumnMap> ColumnMapping { get; set; }
Property Value
TypeDescription
System.Collections.Generic.ICollection<T><ColumnMap>

ConnectionManager

The connection manager used to connect to the database - use the right connection manager for your database type.

Declaration
    public virtual IConnectionManager ConnectionManager { get; set; }
Property Value
TypeDescription
IConnectionManager

DestinationTableDefinition

The table definition of the destination table. By default, the table definition is read from the database. Provide a table definition if the definition of the target can't be read automatically or you want the DbMerge only to use the columns in the provided definition.

Declaration
    public TableDefinition DestinationTableDefinition { get; set; }
Property Value
TypeDescription
TableDefinition

FindDuplicates

By default, the DbMerge expects the IdColumns to be unique. If another record with the same id values arrives, this record is then identified as a duplicate. It won't be written into the destination table - the delta output will contain this record, and the ChangeAction is set to Duplicate Please note that this may have an impact on the overall memory consumption, as already processed id values need to be stored internally during processing.

Declaration
    public bool FindDuplicates { get; set; }
Property Value
TypeDescription
bool

IgnoreDefaultColumnsOnInsert

If set to true, columns that have a default value declartions in the database are ignored when inserting data

Declaration
    public bool IgnoreDefaultColumnsOnInsert { get; set; }
Property Value
TypeDescription
bool

MergeMode

Defines the type of target data which affects how deletions or insertions are handled. Full means that source contains all data, NoDeletions that source contains all data but no deletions are executed, Delta means that source has only delta information and deletions are deferred from a particular property and OnlyUpdates means that only updates are applied to the destination.

Declaration
    public MergeMode MergeMode { get; set; }
Property Value
TypeDescription
MergeMode

MergeProperties

A list of property names that are used in the Merge.

Declaration
    public MergeProperties MergeProperties { get; set; }
Property Value
TypeDescription
MergeProperties

SourceBlock

SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.

Declaration
    public override ISourceBlock<TInput> SourceBlock { get; }
Property Value
TypeDescription
System.Threading.Tasks.Dataflow.ISourceBlock<TOutput><TInput>
Overrides
ETLBox.DataFlow.DataFlowSource<TInput>.SourceBlock

TableName

The name of the target database table for the merge.

Declaration
    public string TableName { get; set; }
Property Value
TypeDescription
string

TargetBlock

TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.

Declaration
    public override ITargetBlock<TInput> TargetBlock { get; }
Property Value
TypeDescription
System.Threading.Tasks.Dataflow.ITargetBlock<TInput><TInput>
Overrides
ETLBox.DataFlow.DataFlowTransformation<TInput, TInput>.TargetBlock

TaskName

A name to identify the task or component. Every component or task comes with a default name that can be overwritten.

Declaration
    public override string TaskName { get; set; }
Property Value
TypeDescription
string
Overrides

UseTruncateMethod

By default, the deletion of records are performed with a bulk delete. This performs well if the number of records to delete is small compared to the total amount of records. If this property is set to true, all records are deleted before the merge operation, using a TRUNCATE. Subsequently, all records are reinserted again. In order to make this work, the MergeMode needs to be set to Full and the CacheMode set to Full as well.

Declaration
    public bool UseTruncateMethod { get; set; }
Property Value
TypeDescription
bool

ValueGeneratedColumns

Exposes the ValueGeneratedColumns property from the underlying DbDestination used for inserts/updates.

Declaration
    public ICollection<ValueGenerationColumn> ValueGeneratedColumns { get; set; }
Property Value
TypeDescription
System.Collections.Generic.ICollection<T><ValueGenerationColumn>

Methods

CheckParameter()

Declaration
    protected override void CheckParameter()
Overrides

CheckParameterAfterInternalFlowCreation()

Declaration
    protected void CheckParameterAfterInternalFlowCreation()

CleanUpOnFaulted(Exception)

Declaration
    protected override void CleanUpOnFaulted(Exception e)
Parameters
TypeNameDescription
System.Exceptione
Overrides

CleanUpOnSuccess()

Declaration
    protected override void CleanUpOnSuccess()
Overrides

InitComponent()

Declaration
    protected override void InitComponent()
Overrides

LinkErrorTo(IDataFlowDestination<ETLBoxError>)

Declaration
    public override IDataFlowSource<ETLBoxError> LinkErrorTo(IDataFlowDestination<ETLBoxError> target)
Parameters
TypeNameDescription
IDataFlowDestination<ETLBoxError>target
Returns
TypeDescription
IDataFlowSource<ETLBoxError>
Overrides
ETLBox.DataFlow.DataFlowSource<TInput>.LinkErrorTo(ETLBox.DataFlow.IDataFlowDestination<ETLBox.DataFlow.ETLBoxError>)

PrepareParameter()

Declaration
    protected override void PrepareParameter()
Overrides

Wait()

Declaration
    public void Wait()

Implements