Class DataFlowStreamSource<TOutput>

{{< /rawhtml >}} ```C# public abstract class DataFlowStreamSource : DataFlowExecutableSource, IDataFlowStreamSource, IDataFlowExecutableSource, IDataFlowSource, IDataFlowStreamSource, IDataFlowExecutableSource, IDataFlowSource, IDataFlowComponent, ILoggableTask ``` {{< rawhtml >}}

Type Parameters

Name	Description
TOutput

Properties

AzureBlobStorage

Use this to pass your AzureBlobStorage connection information. There are different methods to connect with Azure Blob Storage: Option 1: provide an existing BlockBlobClient Option 2: create BlockBlobClient based on the Uri and optional credential tokens - the Uri needs to be the blob url Option 3: create BlockBlobClient with provided ConnectionString and ContainerName, the Uri is then the name of the blob

Declaration

{{< /rawhtml >}} ```C# public AzureBlobStorageConfiguration AzureBlobStorage { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
AzureBlobStorageConfiguration

CollectUnparsedData

Indicates if data not parsed should be stored in the UnparsedData property (also accessable via the current StreamMetaData object when using the GetNextUri/HasNextUri pattern).

Declaration

{{< /rawhtml >}} ```C# public virtual bool CollectUnparsedData { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
bool

CreateStreamReader

By default, stream are created internally based on Uri & ResourceType. If you already have an existing stream, use this function to returns a StreamReader based on this stream. You can create a stream reader for each requested URI.

Declaration

{{< /rawhtml >}} ```C# public Func CreateStreamReader { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
Func<string, StreamReader>

Encoding

Encoding used to read data from the source file or web request.

Declaration

{{< /rawhtml >}} ```C# public Encoding Encoding { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
Encoding

Folder

Gets or sets the path of the folder with the files to process. This will overwrite the HasNextUri and GetNextUri functions, so it should only be used if you want to read all files from a folder.

Declaration

{{< /rawhtml >}} ```C# public string Folder { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
string

GetNextUri

This function returns the next url that is used for reading data. It will be called until HasNextUri returns false. The incoming StreamMetaData holds information about the current progress and other meta data from the response, like unparsed json data that contains references to the next page of the response. This property can be used if you want to read multiple files or if you want to paginate through web responses.

Declaration

{{< /rawhtml >}} ```C# public Func GetNextUri { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
Func<StreamMetaData, string>

HasNextUri

This func determines if another request is started to read additional data from the next uri. StreamMetaData has information about the current progress and other meta data from the response.

Declaration

{{< /rawhtml >}} ```C# public Func HasNextUri { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
Func<StreamMetaData, bool>

HttpClient

The System.Net.Http.HttpClient uses for the request. Use this client if you want to add or change the http request data, e.g. you can add your authorization information here.

Declaration

{{< /rawhtml >}} ```C# public HttpClient HttpClient { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
HttpClient

HttpRequestMessage

The System.Net.Http.HttpRequestMessage used for the request in the HttpClient. Use this property to add additional header data type or to change the http method (e.g. for POST requests).

Declaration

{{< /rawhtml >}} ```C# public HttpRequestMessage HttpRequestMessage { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
HttpRequestMessage

HttpResponseMessage

The System.Net.HttpResponseMessage returned from the current request.

Declaration

{{< /rawhtml >}} ```C# public HttpResponseMessage HttpResponseMessage { get; } ``` {{< rawhtml >}}

Property Value

Type	Description
HttpResponseMessage

NewMetaDataObject

Declaration

{{< /rawhtml >}} ```C# protected StreamMetaData NewMetaDataObject { get; } ``` {{< rawhtml >}}

Property Value

Type	Description
StreamMetaData

RequestUri

The Uri used for the current request - this will only be different to the Uri if you use the GetNextUri and HasNextUri functions.

Declaration

{{< /rawhtml >}} ```C# public string RequestUri { get; protected set; } ``` {{< rawhtml >}}

Property Value

Type	Description
string

ResourceType

Specifies the resource type. By default requests are made with HttpClient. Specify ResourceType.File if you want to read from a json file.

Declaration

{{< /rawhtml >}} ```C# public ResourceType ResourceType { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
ResourceType

RowModificationAction

Allows you to modify a record directly after it is read from the source and before it is send to the next component. While processing of data in other components can happen asynchronously, the execution of this action is synchronous and always occurs after each record was read and before a new record is read.

Declaration

{{< /rawhtml >}} ```C# public Action RowModificationAction { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
Action<TOutput, StreamMetaData>

SkipRows

Number of rows to skip before starting reading the header and csv data

Declaration

{{< /rawhtml >}} ```C# public int SkipRows { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
int

Stream

The raw stream behind StreamReader for the current resource.

Declaration

{{< /rawhtml >}} ```C# public Stream Stream { get; } ``` {{< rawhtml >}}

Property Value

Type	Description
Stream

StreamReader

The stream reader used for the current request.

Declaration

{{< /rawhtml >}} ```C# public StreamReader StreamReader { get; protected set; } ``` {{< rawhtml >}}

Property Value

Type	Description
StreamReader

UnparsedData

Contains all data from the stream that couldn't be processed.

Declaration

{{< /rawhtml >}} ```C# public string UnparsedData { get; } ``` {{< rawhtml >}}

Property Value

Type	Description
string

UnparsedDataBuilder

Declaration

{{< /rawhtml >}} ```C# protected StringBuilder UnparsedDataBuilder { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
StringBuilder

Uri

The Url of the webservice (e.g. https://test.com/foo) or the file name (relative or absolute).

Declaration

{{< /rawhtml >}} ```C# public string Uri { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
string

UseMulitpartContent

If the response is a multipart response, this function is used to determine which content part of the multipart response to use.

Declaration

{{< /rawhtml >}} ```C# public Func UseMulitpartContent { get; set; } ``` {{< rawhtml >}}

Property Value

Type	Description
Func<HttpContent, bool>

Methods

CheckParameter()

Declaration

{{< /rawhtml >}} ```C# protected override void CheckParameter() ``` {{< rawhtml >}}

Overrides

DataFlowComponent.CheckParameter()

CleanUpOnFaulted(Exception)

Declaration

{{< /rawhtml >}} ```C# protected override void CleanUpOnFaulted(Exception e) ``` {{< rawhtml >}}

Parameters

Type	Name	Description
Exception	e

Overrides

DataFlowComponent.CleanUpOnFaulted(Exception)

CleanUpOnSuccess()

Declaration

{{< /rawhtml >}} ```C# protected override void CleanUpOnSuccess() ``` {{< rawhtml >}}

Overrides

DataFlowComponent.CleanUpOnSuccess()

CloseReader()

Declaration

{{< /rawhtml >}} ```C# protected abstract void CloseReader() ``` {{< rawhtml >}}

InitReader()

Declaration

{{< /rawhtml >}} ```C# protected abstract void InitReader() ``` {{< rawhtml >}}

OnExecutionDoAsyncWork()

Declaration

{{< /rawhtml >}} ```C# protected override void OnExecutionDoAsyncWork() ``` {{< rawhtml >}}

Overrides

DataFlowExecutableSource<TOutput>.OnExecutionDoAsyncWork()

PrepareParameterForCheck()

Declaration

{{< /rawhtml >}} ```C# protected override void PrepareParameterForCheck() ``` {{< rawhtml >}}

Overrides

DataFlowComponent.PrepareParameterForCheck()

ReadAllRecords()

Declaration

{{< /rawhtml >}} ```C# protected abstract void ReadAllRecords() ``` {{< rawhtml >}}

Implements

IDataFlowStreamSource<TOutput>

IDataFlowExecutableSource<TOutput>

IDataFlowSource<TOutput>

IDataFlowStreamSource

IDataFlowExecutableSource

IDataFlowSource

IDataFlowComponent

ILoggableTask