Release notes

Release notes are starting with version 2.3.0

Version 2.5.1

Features

  • DbMerge: Added new MergeMode ‘InsertsOnly’

Version 2.5.0

Features

  • Removed unnecessary command preparation and removed DoPrepareCommand property in connection managers
  • Support that HttpRequestMessage can be changed in StreamMetaData when using GetNextUri/HasNextUri pattern
  • Updated third party dependencies to latest version
  • Upgraded CsvHelper to latest version (some Configurations are now part of Context object). This change may be a breaking change depending on your usage of CsvHelper specific configuration settings. Please refer to the {< link-ext url=“https://joshclose.github.io/CsvHelper/change-log/” text=“CsvHelper release notes” >} to see what changed here.
  • Breaking change: DataFlow.LoggingThresholdRows renamed to LogThreshold
  • Breaking change: Combining CreateLogTable and ReadLogTable tasks into LogTask (e.g. use LogTask.CreateLogTable(..) instead)
  • Breaking change: Introducing structured logging - core package now supports general ILogger interface instead NLog. You can now assign your ILogger instance to the static property Logging.LogInstance. Now NLog, Serilog and other logger supporting the log abstraction interface will be supported.
  • LoadProcess: adding SourceId column to ease joins with other tables
  • Combining CustomTask/Package/Sequence into LogSection
  • Combining Create/Start/Abort/End/Read LoadProcessTasks into one LoadProcessTask (e.g. use LoadProcessTask.Create() instead)
  • Added support for FOREIGN KEY constraints in CreateTableTask and TableDefinition object
  • Added Parquet source/destination

Bugs

  • AggregationMethods Min/Max now support DateTime
  • LogThreshold set on component now prioritized over global threshold value

Version 2.4.4

Features

  • CommandTimeout property limiting timeout for sql commands added to all connection managers
  • KeepIdentity renamed to AllowIdentityInsert in DbDestination (Breaking change)

Bugs

  • Improved support for Db2 - internal sql updated to work on zOS and LUW version

Version 2.4.3

Features

  • SqlConnectionManager exposes parameter to use row locks instead table lock for bulk insert operations
  • DbSource: Sql property now allows query parameter

Bugs

  • Improved exception handling for DbMerge: Added check for IdColumn
  • XmlSource now correctly reads all tags if xml contains no line breaks

Version 2.4.2

Features

  • DataTypeConverter: Added helper method to convert from CLR type to SqlServer specific data type
  • ExcelSource exposes ExcelColumns property, allowing to manual set column configuration
  • All components now have delegate for progress tracking
  • New destination: ExcelDestination

Bug fixes

  • ExcelSource: EndColumn value in ExcelRange now starts counting at with 1 instead 0

Version 2.4.1

Features

  • Adding GetNextUri/HasNextUri pattern to DataFlowStreamDestinations
  • ResourceType.File is now default for all streaming sources and destinations
  • Streaming sources: New property CreateStreamReader allows usage of existing streams
  • Streaming destinations: New property CreateStreamWriter allows to write into existing streams
  • Adding integration to Azure Blob Storage for streaming sources and destinations (ResourceType.AzureBlob)
  • ExcelSource&CsvSource: ReadFieldHeaders will read header data based on current configuration (can be called before the actual data flow started)
  • DbMerge now exposes ColumnMapping property
  • New transformation: FilterTransformation as alternative to linking with predicates
  • ColumnRename: RenameFunc let you define a custom function to rename each column/prop separately with a custom function

Bug fixes

  • SqlServer: fixing bug for Query parameter types without length (e.g. DECIMAL)
  • DbSource: Using a sql query where the column names include white spaces, these white spaces are kept

Version 2.4.0

Features

  • TextSource: ParseLineAction changed to ParseLineFunc
  • StreamMetaData holds number of processed rows per request
  • Added Redis source/destination (package ETLBox.Redis)
  • Added Couchbase source/destination (package ETLBox.Couchbase)
  • Added MongoDb source/destination (package ETLBox.MongoDb)
  • Network class: Network.Cancel(..) allows to cancel a running complete data flow

Bug fixes

  • DbMerge: For missing compare columns, the non-Id columns are now properly used detecting exists/updates
  • Db2ConnectionManager/UseSqlBulkInsert: Fixed bug when parameter where exceeding Db2 maximum
  • Network class: Networks now correctly cancels when exception in error path is thrown

Version 2.3.2

Features

General:

  • Auto generated xml documentation added to package - now visible with Intellisense or when browsing package content.

DataFlow:

  • ExcelSource exposes parsed header names in FieldHeaders property.
  • DbSource has ColumnConverters property

Version 2.3.1

Features

ConnectionManagers:

  • ODBC/OleDb now have ConnectionManagerType property settable.

ControlFlow:

  • Db2 now has support for schemas
  • CreateTableTask: Now offers functionality to alter tables (Alter() / CreateOrAlter())
  • CreateSchemaTask supports authorization
  • Added GetTableListTask (return all tables in database)

DataFlow:

  • UnparsedData property for streaming sources now contains data of skipped rows.
  • All executable sources allow to set limit for records to read
  • Added property KeepIdentity to DbDestination which allows overwriting of Identity columns
  • DataConverters available for DbDestination (allows to add custom column converter)
  • (Breaking Change) DbSource: Replace ListColumnNames prop with ICollection ColumnMapping
  • (Breaking change) Improved naming in ColumnMap Attribute (NewName = PropertyName, CurrentName = DbColumnName)
  • (Breaking change) ColumnRename now uses RenameColumn attribute instead ColumnMapping.

Bug fixes

  • Postgres connection manager now has full support for jsonb columns.
  • DbSource/Sql property: Whitespace in column aliases are now properly parsed as column names.