Class DbMerge<TInput>
Inserts, updates and (optionally) deletes data in database target.
Before the Merge is executed, all data from the destination is read into memory.
A delta table is generated that stores information if a records was inserted, updated, deleted or hasn't been touched (existed).
Inheritance
object
DbMerge<TInput>
Inherited Members
object.Equals(object)
object.Equals(object, object)
object.GetHashCode()
object.GetType()
object.MemberwiseClone()
object.ReferenceEquals(object, object)
object.ToString()
Namespace: ETLBox.DataFlow.Connectors
Assembly: ETLBox.dll
Syntax
public class DbMerge<TInput> : DataFlowTransformation<TInput, TInput>, ILoggableTask, IDataFlowLogging, IDataFlowTransformation<TInput, TInput>, IDataFlowSource<TInput>, IDataFlowSource, IDataFlowBatchDestination<TInput>, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent, IDataFlowBatchDestination
Type Parameters
Name | Description |
---|
TInput | Type of ingoing data. |
Constructors
DbMerge()
Declaration
DbMerge(IConnectionManager, string, int)
Declaration
public DbMerge(IConnectionManager connectionManager, string tableName, int batchSize)
Parameters
DbMerge(IConnectionManager, string)
Declaration
public DbMerge(IConnectionManager connectionManager, string tableName)
Parameters
DbMerge(string, int)
Declaration
public DbMerge(string tableName, int batchSize)
Parameters
Type | Name | Description |
---|
string | tableName | |
int | batchSize | |
DbMerge(string)
Declaration
public DbMerge(string tableName)
Parameters
Type | Name | Description |
---|
string | tableName | |
Properties
BatchSize
The batch size used when inserted data into the database table.
Declaration
public int BatchSize { get; set; }
Property Value
CacheMode
The cache mode used for reading data from the destination table.
By default, all data is read into memory before the actual merge is executed (Full mode).
If set to partial, only data is read into memory that is needed for the current merge comparison.
This will keep the memory consumption of the DbMerge much smaller.
Declaration
public CacheMode CacheMode { get; set; }
Property Value
ColumnConverters
Use a column converter to apply a conversion function to each value of a column before writing
into the target
Declaration
public ICollection<ColumnConverter> ColumnConverters { get; set; }
Property Value
ColumnMapping
Column mapping to map property to column names.
E.g. if the value of property "Id" should be written into the database column "Key",
then you can simply add a column mapping: DbColumnName: "Id" -> PropertyName: "Key".
Declaration
public ICollection<ColumnMap> ColumnMapping { get; set; }
Property Value
Type | Description |
---|
System.Collections.Generic.ICollection<T><ColumnMap> | |
ConnectionManager
The connection manager used to connect to the database - use the right connection manager for your database type.
Declaration
public virtual IConnectionManager ConnectionManager { get; set; }
Property Value
DestinationTableDefinition
The table definition of the destination table. By default, the table definition is read from the database.
Provide a table definition if the definition of the target can't be read automatically or you want the DbMerge
only to use the columns in the provided definition.
Declaration
public TableDefinition DestinationTableDefinition { get; set; }
Property Value
FindDuplicates
By default, the DbMerge expects the IdColumns to be unique.
If another record with the same id values arrives, this record is then identified as a duplicate.
It won't be written into the destination table - the delta output will contain this record, and the
ChangeAction is set to Duplicate
Please note that this may have an impact on the overall memory consumption, as already processed id values need to be stored internally during processing.
Declaration
public bool FindDuplicates { get; set; }
Property Value
IgnoreDefaultColumnsOnInsert
If set to true, columns that have a default value declartions in the database are
ignored when inserting data
Declaration
public bool IgnoreDefaultColumnsOnInsert { get; set; }
Property Value
MergeMode
Defines the type of target data which affects how deletions or insertions are handled.
Full means that source contains all data, NoDeletions that source contains all data but no deletions are executed,
Delta means that source has only delta information and deletions are deferred from a particular property and
OnlyUpdates means that only updates are applied to the destination.
Declaration
public MergeMode MergeMode { get; set; }
Property Value
MergeProperties
A list of property names that are used in the Merge.
Declaration
public MergeProperties MergeProperties { get; set; }
Property Value
SourceBlock
SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.
Declaration
public override ISourceBlock<TInput> SourceBlock { get; }
Property Value
Type | Description |
---|
System.Threading.Tasks.Dataflow.ISourceBlock<TOutput><TInput> | |
Overrides
ETLBox.DataFlow.DataFlowSource<TInput>.SourceBlock
TableName
The name of the target database table for the merge.
Declaration
public string TableName { get; set; }
Property Value
TargetBlock
TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.
Declaration
public override ITargetBlock<TInput> TargetBlock { get; }
Property Value
Type | Description |
---|
System.Threading.Tasks.Dataflow.ITargetBlock<TInput><TInput> | |
Overrides
ETLBox.DataFlow.DataFlowTransformation<TInput, TInput>.TargetBlock
TaskName
A name to identify the task or component. Every component or task comes
with a default name that can be overwritten.
Declaration
public override string TaskName { get; set; }
Property Value
Overrides
UseTruncateMethod
By default, the deletion of records are performed with a bulk delete.
This performs well if the number of records to delete is small compared to the total amount
of records.
If this property is set to true, all records are deleted before the merge operation,
using a TRUNCATE. Subsequently, all records are reinserted again.
In order to make this work, the MergeMode needs to be set to Full
and the CacheMode set to Full as well.
Declaration
public bool UseTruncateMethod { get; set; }
Property Value
ValueGeneratedColumns
Declaration
public ICollection<ValueGenerationColumn> ValueGeneratedColumns { get; set; }
Property Value
Methods
CheckParameter()
Declaration
protected override void CheckParameter()
Overrides
CheckParameterAfterInternalFlowCreation()
Declaration
protected void CheckParameterAfterInternalFlowCreation()
CleanUpOnFaulted(Exception)
Declaration
protected override void CleanUpOnFaulted(Exception e)
Parameters
Type | Name | Description |
---|
System.Exception | e | |
Overrides
CleanUpOnSuccess()
Declaration
protected override void CleanUpOnSuccess()
Overrides
InitComponent()
Declaration
protected override void InitComponent()
Overrides
LinkErrorTo(IDataFlowDestination<ETLBoxError>)
Declaration
public override IDataFlowSource<ETLBoxError> LinkErrorTo(IDataFlowDestination<ETLBoxError> target)
Parameters
Returns
Overrides
ETLBox.DataFlow.DataFlowSource<TInput>.LinkErrorTo(ETLBox.DataFlow.IDataFlowDestination<ETLBox.DataFlow.ETLBoxError>)
PrepareParameter()
Declaration
protected override void PrepareParameter()
Overrides
Wait()
Declaration
Implements