Class ParquetSource<TOutput>
Reads data from a parquet source. While reading the data from the file, data is also asynchronously posted into the targets.
Inheritance
DataFlowSource<TOutput>
DataFlowExecutableSource<TOutput>
DataFlowStreamSource<TOutput>
ParquetSource<TOutput>
Inherited Members
Namespace: ETLBox.Parquet
Assembly: ETLBox.Parquet.dll
Syntax
public class ParquetSource<TOutput> : DataFlowStreamSource<TOutput>, IDataFlowLogging, IDataFlowStreamSource<TOutput>, IDataFlowExecutableSource<TOutput>, IDataFlowSource<TOutput>, IDataFlowStreamSource, IDataFlowExecutableSource, IDataFlowSource, IDataFlowComponent, ILoggableTask
Type Parameters
| Name | Description |
|---|---|
| TOutput | Type of outgoing data |
Constructors
ParquetSource()
Declaration
public ParquetSource()
ParquetSource(string)
Declaration
public ParquetSource(string uri)
Parameters
| Type | Name | Description |
|---|---|---|
| string | uri | The source parquet file name or uri |
ParquetSource(string, ResourceType)
Declaration
public ParquetSource(string uri, ResourceType resourceType)
Parameters
| Type | Name | Description |
|---|---|---|
| string | uri | The source parquet file name or uri |
| ResourceType | resourceType | Specifies if data is loaded from a file, a web endpoint or other storage types (e.g. Azure Blob Storage) |
Fields
SupportedDataTypes
Currently the ParquetDestination only supports 'basic' data types. Access this property to get all Clr types that are supported.
Declaration
public Type[] SupportedDataTypes
Field Value
| Type | Description |
|---|---|
| Type[] |
Properties
ParquetColumns
List of mappings between columns names in parquet file and property name in object.
Declaration
public ICollection<ParquetColumn> ParquetColumns { get; set; }
Property Value
| Type | Description |
|---|---|
| ICollection<ParquetColumn> |
ParquetOptions
Options used when creating the ParquetReader
Declaration
public ParquetOptions ParquetOptions { get; set; }
Property Value
| Type | Description |
|---|---|
| ParquetOptions |
Methods
CheckParameter()
Declaration
protected override void CheckParameter()
Overrides
CloseReader()
Declaration
protected override void CloseReader()
Overrides
FindPropNameMapping(string)
Declaration
public string FindPropNameMapping(string columnName)
Parameters
| Type | Name | Description |
|---|---|---|
| string | columnName |
Returns
| Type | Description |
|---|---|
| string |
InitReader()
Declaration
protected override void InitReader()
Overrides
PrepareParameterForCheck()
Declaration
protected override void PrepareParameterForCheck()
Overrides
ReadAllRecords()
Declaration
protected override void ReadAllRecords()
Overrides
Reset()
Declaration
protected override void Reset()