Namespace ETLBox.DataFlow.Connectors

Classes

ColumnConverter

ConcurrentMemoryDestination

A destination in memory - it will store all data in a BlockingCollection<T> The BlockingCollection<T> allows you to access the data concurrently while rows are still written into the target. If you don't need to work with your data before the flow finishes, you can use the MemoryDestination which uses a regular List<T>.

ConcurrentMemoryDestination<TInput>

A destination in memory - it will store all data in a BlockingCollection<T> The BlockingCollection<T> allows you to access the data concurrently while rows are still written into the target. If you don't need to work with your data before the flow finishes, you can use the MemoryDestination which uses a regular List<T>.

CouchbaseDestination

A data flow destination for inserting data into a couchbase bucket.

CouchbaseDestination<TInput>

A data flow destination for inserting data into a couchbase bucket.

CouchbaseSource

A data flow source for Couchbase

CouchbaseSource<TOutput>

A data flow source for Couchbase

CsvDestination

A Csv destination defines a csv file where data from the flow is inserted.

CsvDestination<TInput>

A Csv destination defines a csv file where data from the flow is inserted.

CsvSource

Reads data from a csv source. While reading the data from the file, data is also asnychronously posted into the targets. Data is read a as string from the source and dynamically converted into the corresponding data format.

CsvSource<TOutput>

Reads data from a csv source. While reading the data from the file, data is also asnychronously posted into the targets. Data is read a as string from the source and dynamically converted into the corresponding data format.

CustomBatchDestination

Define your own batch destination block. This block accepts all data from the flow and will create batches of incoming data which can be processed with the WriteBatchAction.

CustomBatchDestination<TInput>

Define your own batch destination block. This block accepts all data from the flow and will create batches of incoming data which can be processed with the WriteBatchAction.

CustomBatchSource

Define your own source block. This block allows you to read batches of data from your own custom written functions, which are then send subsequently into your data flow.

CustomBatchSource<TOutput>

Define your own source block. This block allows you to read batches of data from your own custom written functions, which are then send subsequently into your data flow.

CustomDestination

Define your own destination block. This block accepts all data from the flow and sends each incoming row to your custom Action, along with a count of processed rows.

CustomDestination<TInput>

Define your own destination block. This block accepts all data from the flow and sends each incoming row to your custom Action, along with a count of processed rows.

CustomSource

Define your own source block. This block generates data from a your own custom written functions.

CustomSource<TOutput>

Define your own source block. This block generates data from a your own custom written functions.

DbDestination

A DbDestination represents a database table where ingoing data from the flow is written into. Inserts are done in batches (using Bulk insert or an equivalent INSERT statement).

DbDestination<TInput>

A DbDestination represents a database table where ingoing data from the flow is written into. Inserts are done in batches (using Bulk insert or an equivalent INSERT statement).

DbMerge

DbMerge<TInput>

Inserts, updates and (optionally) deletes data in database target. Before the Merge is executed, all data from the destination is read into memory. A delta table is generated that stores information if a records was inserted, updated, deleted or hasn't been touched (existed).

DbSource

A database source defines either a table or sql query that returns data from a database. Multiple database are supported. Use the corresponding connection manager that fits to your database.

DbSource<TOutput>

A database source defines either a table or sql query that returns data from a database. Multiple database are supported. Use the corresponding connection manager that fits to your database.

ExcelDestination

A Excel destination creates an excel files with formatted data. The output will look similar to a csv file.

ExcelDestination<TInput>

A Excel destination creates an excel files with formatted data. The output will look similar to a csv file.

ExcelSource

Reads data from a excel source. While reading the data from the file, data is also asnychronously posted into the targets. You can define a sheet name and a range - only the data in the specified sheet and range is read. Otherwise, all data in all sheets will be processed.

ExcelSource<TOutput>

Reads data from a excel source. While reading the data from the file, data is also asnychronously posted into the targets. You can define a sheet name and a range - only the data in the specified sheet and range is read. Otherwise, all data in all sheets will be processed.

JsonDestination

A Json destination defines a json file where data from the flow is inserted.

JsonDestination<TInput>

A Json destination defines a json file where data from the flow is inserted.

JsonSource

Reads data from a json source. This can be any http resource or a file. By default, data is pulled via httpclient. Use the ResourceType property to read data from a file.

JsonSource<TOutput>

Reads data from a json source. This can be any http resource or a file. By default, data is pulled via httpclient. Use the ResourceType property to read data from a file.

MemoryDestination

A destination in memory - it will store all data in a collection that you assign to the Data property. By default, a List is used to store th data. If you need to access the data concurrently while rows are still written into the target, see the ConcurrentMemoryDestination.

MemoryDestination<TInput>

A destination in memory - it will store all data in a collection that you assign to the Data property. By default, a List is used to store th data. If you need to access the data concurrently while rows are still written into the target, see the ConcurrentMemoryDestination.

MemorySource

Define a source based on a generic .NET collection. This could be a List<T> or any other IEnumerable<T>. By default, an empty List<T> is created which can be filled with data.

MemorySource<TOutput>

Define a source based on a generic .NET collection. This could be a List<T> or any other IEnumerable<T>. By default, an empty List<T> is created which can be filled with data.

MongoDbDestination

A data flow destination for MongoDb.

MongoDbDestination<TInput>

A data flow destination for MongoDb.

MongoDbSource

A data flow source for MonogDb.

MongoDbSource<TOutput>

A data flow source for MonogDb.

RedisDestination

A data flow destination for Redis. Regis is a key/value store. You can define a key column using either a KeyColumnAttribute on your object or assign a value to the KeyColumn property. If no key column is provided, a Guid is created and used as key. The stored value will contain your serialized object using Newtonsoft.Json and JsonConvert.

RedisDestination<TInput>

A data flow destination for Redis. Regis is a key/value store. You can define a key column using either a KeyColumnAttribute on your object or assign a value to the KeyColumn property. If no key column is provided, a Guid is created and used as key. The stored value will contain your serialized object using Newtonsoft.Json and JsonConvert.

RedisDestination<TInput>.RedisSetParameter

Additional redis parameter that are used when setting the key/value pairs in redis.

RedisSource

A data flow source for Redis. Redis is a key/value store.
You can define a different scan pattern to describe which keys should be retrieved. (Default pattern is * which retrieves all key). By default, the stored values will be deserialized using Newtonsoft.Json and JsonConvert. This works only if your values contain json - for other values you can define your own serialization function.

RedisSource<TOutput>

A data flow source for Redis. Redis is a key/value store.
You can define a different scan pattern to describe which keys should be retrieved. (Default pattern is * which retrieves all key). By default, the stored values will be deserialized using Newtonsoft.Json and JsonConvert. This works only if your values contain json - for other values you can define your own serialization function.

TextDestination

Writes data into a text file. Each line in the output is created by calling the WriteLineFunc or by invoking ToString() on the object.

TextDestination<TInput>

Writes data into a text file. Each line in the output is created by calling the WriteLineFunc or by invoking ToString() on the object.

TextSource

Reads data from a text file. Each line is read as a string and converted into an object by the ParseLineFunc. A line is defined as a sequence of characters followed by a line feed("\n"), a carriage return ("\r"), or a carriage return immediately followed by a line feed("\r\n").

TextSource<TOutput>

Reads data from a text file. Each line is read as a string and converted into an object by the ParseLineFunc. A line is defined as a sequence of characters followed by a line feed("\n"), a carriage return ("\r"), or a carriage return immediately followed by a line feed("\r\n").

VoidDestination

This destination serves as a recycle bin for data that is not supposed to go into any other destination. Every records in the dataflow needs to enter any kind of destination in order to have a dataflow completed. Use this target for data that you don't want to use in a destination, but you still want your dataflow to complete property.

VoidDestination<TInput>

This destination serves as a recycle bin for data that is not supposed to go into any other destination. Every records in the dataflow needs to enter any kind of destination in order to have a dataflow completed. Use this target for data that you don't want to use in a destination, but you still want your dataflow to complete property.

XmlDestination

A Xml destination defines a xml file where data from the flow is inserted.

XmlDestination<TInput>

A Xml destination defines a xml file where data from the flow is inserted.

XmlSource

Reads data from a xml source. This can be any http resource or a file. By default, data is pulled via httpclient. Use the ResourceType property to read data from a file.

XmlSource<TOutput>

Reads data from a xml source. This can be any http resource or a file. By default, data is pulled via httpclient. Use the ResourceType property to read data from a file.