Recipes

Aggregation →

Examples and code recipes for the aggregation

This article contains example codes that demonstrate the usage of the Aggregation transformation.

Azure Function App →

Example: how to use ETLBox with Azure Functions

This example shows the basics deployment of an Azure Function App using ETLBox as dependency.

Block Transformation →

Examples and code recipes for the BlockTransformation

This article contains example code that shows the usage of the BlockTransformation.

Column Rename →

Examples and code recipes for the ColumnRename component.

This article contains example code that shows the usage of the ColumnRename component.

Control and Data Flow Basics →

Example: Control and data flow basics

This purpose of this example is to give you a brief overview of the basic concepts of ETLBox. It demonstrates the basic idea of a very simple data flow and shows how to use the ControlFlow objects to manage your database metadata.

Cross Join →

Examples and code recipes for the CrossJoin component.

This article contains example code that shows the usage of the CrossJoin component.

Csv Destination →

Examples and code recipes for the CsvDestination component.

This article contains example code that shows the usage of the CsvDestination component.

Csv Source →

Examples and code recipes for the CsvSource component.

This article contains example code that shows the usage of the CsvSource component.

Csv to Json →

Example code how to transform a Csv to a Json

This example shows how to read data from a csv and transform it into a json - one time using strongly typed object and also using the dynamic ExpandoObject.

Custom Batch Destination →

Examples and code recipes for the CustomBatchDestination component.

This article contains example code that shows the usage of the CustomBatchDestination component.

Custom Batch Source →

Examples and code recipes for the CustomBatchSource component.

This article contains example code that shows the usage of the CustomBatchSource component.

Custom Destination →

Examples and code recipes for the CustomDestination component.

This article contains example code that shows the usage of the CustomDestination component.

Custom Source →

Examples and code recipes for the CustomSource component.

This article contains example code that shows the usage of the CustomSource component.

Data Lake →

Example: Copy data from Sql Server into Azure blob storage

This example shows how data from SqlServer are transferred into Azure blob storage. It shows how to setup an Azure function that runs periodically to move freshly created data rows from the database into a blob container.

Data Lookup →

Various examples how to use the lookup transformation

This article contains examples which show how to use the lookup transformation and shows other techniques to lookup data..

Data Validation Part 1 →

ETLBox (Part 1): C#'s Answer to Seamless ETL Processing and Thorough Data Validation

This article was also published as a blog article 'Redefining ETL: Data Flows Powered by C# (Part I)'

Data Validation Part 2 →

ETLBox (Part 2): C#'s Answer to Seamless ETL Processing and Thorough Data Validation

This article was also published as a blog article 'Redefining ETL: Data Flows Powered by C# (Part II)'

Data Validation Part 3 →

ETLBox (Part 3): C#'s Answer to Seamless ETL Processing and Thorough Data Validation

This article was also published as a blog article 'Redefining ETL: Data Flows Powered by C# (Part III)'

Database Destination →

Examples and code recipes for the DbDestination component.

This article contains example code that shows the usage of the DbDestination component.

Database Source →

Examples and code recipes for the DbSource component.

This article contains example code that shows the usage of the DbSource component.

Distinct →

Examples and Code Recipes for the Distinct Component.

This article contains example code snippets that show how to use the Distinct component.

Duplicate Checks →

Different methods to filter duplicates in your source

This example shows how different methods to filter out duplicates in your input data.

Filter Transformation →

Examples and code recipes for the FilterTransformation component.

This article contains example code that shows the usage of the FilterTransformation component.

Flatten Nested Array in Xml →

Example: flatten nested array in a xml file

This example shows how to denormalize a nested data array within a xml structure.

From Files to Databases →

Example: From files and databases

Sometimes you are in need to transfer data across databases on different server or to integrate flat files. ETLBox is an excellent tool of choice for integrating data from different sources. This example will guide you through the most common scenarios.

Integrate REST Endpoints →

Example code how to use Json from REST endpoints or other web services

This example demonstrates how json data can be received and send between web endpoints. It will read json data from a REST endpoint, rename some properties and send each entry from the source into a different endpoint.

Json Destination →

Examples and code recipes for the JsonDestination component.

This article contains example code that shows the usage of the JsonDestination component.

Json Path in Dynamic Object →

Example: Using json path with dynamic objects

This purpose of this example is demonstrate how json path can be used to simplify navigation in dynamic objects.

Json Source →

Examples and code recipes for the JsonSource component.

This article contains example code that shows the usage of the JsonSource component.

Lookup Transformation →

Examples and code recipes for the LookupTransformation component.

This article contains example code that shows the usage of the LookupTransformation component.

Memory Consumption →

This recipe show a small command line utility that loads test data and allows to observer the memory consumption while loading this data.

This sample application demonstrates the key database operations, including creating database tables, loading data, synchronizing data between tables, and prints out the current memory consumptions. The application listens for user commands to execute the specific ETL tasks. While the operations are performed, you can utilize the Resource Monitor to observer the current Memory Consumption of the process (and other metrics if needed.)

Memory Destination →

Examples and code recipes for the MemoryDestination component.

This article contains example code that shows the usage of the MemoryDestination component.

Memory Source →

Examples and code recipes for the MemorySource component.

This article contains example code that shows the usage of the MemorySource component.

Merge Join Transformation →

Examples and code recipes for the MergeJoin component.

This article contains example code that shows the usage of the MergeJoin component.

Multicast →

Examples and code recipes for the Multicast component.

This article contains example code that shows the usage of the Multicast component.

Parallel Execution →

Example: how to run multiple data flows in parallel

This purpose of this example is to show how multiple data flows can be executed in parallel.

Pivot Data →

Example: pivot data from a csv file

This example shows how to use the RowMultiplication to pivot data in a csv file.

Rating Orders →

Example: Rating orders

This example demonstrates how the different data flow components can be used together, simulating a real-life scenario. In this scenario, we will to read some orders from a csv file, lookup the customer key by using the customer name, write the orders into an orders table and also create a customer rating based on the total amount of purchases.

Row Duplication →

Examples and code recipes for the RowDuplication component.

This article contains example code that shows the usage of the RowDuplication component.

Row Multiplication →

Examples and code recipes for the RowMultiplication component.

This article contains example code that shows the usage of the RowMultiplication component.

Row Transformation →

Examples and code recipes for the RowTransformation component.

This article contains example code that shows the usage of the RowTransformation component.

SFTP Example →

ETLBox and SFTP: A Practical Guide

In this article, we'll give you a practical example of combining ETLBox with the SSH.NET library in order to fetch data from an SFTP server. This scenario is particularly useful where files need to be securely transferred over a network before processing.

Sort →

Examples and code recipes for the Sort component.

This article contains example code that shows the usage of the Sort component.

Star Schema →

Building a Data Warehouse with ETLBox

This article was also published as a blog article 'Building a Data Warehouse with ETLBox: A .NET Developer's Guide'

Unit Testing →

This example shows how unit tests could be constructed for data flow.

This recipe demonstrate how unit tests could be written to test data flows. The described approach here is very basic and should only be considered as a starting point - there are a lot of different possibilities to write unit tests for ETLBox - as the library is written in .NET, all test framework and other test packages are fully supported.

Web Scraping →

Example: Web scraping

This example demonstrates how content from a web site can be scraped and used to store, aggregate and display the retrieved data.

Xml Destination →

Examples and code recipes for the XmlDestination component.

This article contains example code that shows the usage of the XmlDestination component.

Xml Source →

Examples and code recipes for the XmlSource component.

This article contains example code that shows the usage of the XmlSource component.