ParquetSource<TOutput>

Class ParquetSource<TOutput>

Reads data from a parquet source. While reading the data from the file, data is also asynchronously posted into the targets.

Inherited Members
Namespace: ETLBox.Parquet
Assembly: ETLBox.Parquet.dll
Syntax
    public class ParquetSource<TOutput> : DataFlowStreamSource<TOutput>, IDataFlowStreamSource<TOutput>, IDataFlowExecutableSource<TOutput>, IDataFlowSource<TOutput>, IDataFlowStreamSource, IDataFlowExecutableSource, IDataFlowSource, IDataFlowComponent, ILoggableTask
Type Parameters
NameDescription
TOutput

Type of outgoing data

Constructors

ParquetSource()

Declaration
    public ParquetSource()

ParquetSource(string)

Declaration
    public ParquetSource(string uri)
Parameters
TypeNameDescription
stringuri

The source parquet file name or uri

ParquetSource(string, ResourceType)

Declaration
    public ParquetSource(string uri, ResourceType resourceType)
Parameters
TypeNameDescription
stringuri

The source parquet file name or uri

ResourceTyperesourceType

Specifies if data is loaded from a file, a web endpoint or other storage types (e.g. Azure Blob Storage)

Fields

SupportedDataTypes

Currently the ParquetDestination only supports 'basic' data types. Access this property to get all Clr types that are supported.

Declaration
    public Type[] SupportedDataTypes
Field Value
TypeDescription
Type[]

Properties

ParquetColumns

List of mappings between columns names in parquet file and property name in object.

Declaration
    public ICollection<ParquetColumn> ParquetColumns { get; set; }
Property Value
TypeDescription
ICollection<ParquetColumn>

ParquetOptions

Options used when creating the ParquetReader

Declaration
    public ParquetOptions ParquetOptions { get; set; }
Property Value
TypeDescription
ParquetOptions

Methods

CheckParameter()

Declaration
    protected override void CheckParameter()
Overrides

CloseReader()

Declaration
    protected override void CloseReader()
Overrides

FindPropNameMapping(string)

Declaration
    public string FindPropNameMapping(string columnName)
Parameters
TypeNameDescription
stringcolumnName
Returns
TypeDescription
string

InitReader()

Declaration
    protected override void InitReader()
Overrides

PrepareParameterForCheck()

Declaration
    protected override void PrepareParameterForCheck()
Overrides

ReadAllRecords()

Declaration
    protected override void ReadAllRecords()
Overrides

Reset()

Declaration
    protected override void Reset()
Overrides

Implements