# Text Document Source<no value>

`TextDocumentSource` reads one text resource as one row. A file, a web response, or a blob is one element. A folder is one element per file. The non-generic source writes `Text` and `StreamMetaData` on an `ExpandoObject`.

## Read one file

The whole file is one dynamic row. `Text` is the text. `StreamMetaData` carries the path and the resource count.

```C#
string sourceFile = "readme.md";
File.WriteAllText(sourceFile, """
    # Warehouse notes

    ETLBox reads this file as one element.
    The second paragraph stays in the same element.
    """);

var source = new TextDocumentSource(sourceFile);
var dest = new MemoryDestination();

source.LinkTo(dest);
Network.Execute(source);

foreach (dynamic row in dest.Data) {
    StreamMetaData metadata = row.StreamMetaData;
    Console.WriteLine($"RequestUri:{metadata.RequestUri}");
    Console.WriteLine($"RequestCount:{metadata.RequestCount}");
    Console.WriteLine(row.Text);
}

//Outputs
//RequestUri:readme.md
//RequestCount:1
//# Warehouse notes
//
//ETLBox reads this file as one element.
//The second paragraph stays in the same element.
```

## Read into your own type

`[DocumentContent]` receives the text. `[DocumentRequestUri]` receives the path. Properties without an attribute stay unset.

```C#
public class Article
{
    public string Title { get; set; }
    [DocumentContent]
    public string Content { get; set; }
    [DocumentRequestUri]
    public string Source { get; set; }
}

string sourceFile = "catalog.md";
File.WriteAllText(sourceFile, """
    Trail bottle for hiking.
    Fits in a side pocket.
    """);

var source = new TextDocumentSource<Article>(sourceFile);
var dest = new MemoryDestination<Article>();

source.LinkTo(dest);
Network.Execute(source);

foreach (Article row in dest.Data) {
    Console.WriteLine($"Source:{row.Source}");
    Console.WriteLine($"Content:{row.Content}");
}

//Outputs
//Source:catalog.md
//Content:Trail bottle for hiking.
//Fits in a side pocket.
```

## Build the row yourself

`ResultSelector` gets the text and the `StreamMetaData`. The attributes are not used, so fields such as `Title` can be set here.

```C#
public class Article
{
    public string Title { get; set; }
    [DocumentContent]
    public string Content { get; set; }
    [DocumentRequestUri]
    public string Source { get; set; }
}

string sourceFile = "catalog.md";
File.WriteAllText(sourceFile, """
    Trail bottle for hiking.
    Fits in a side pocket.
    """);

var source = new TextDocumentSource<Article>(sourceFile) {
    ResultSelector = (text, metadata) => new Article {
        Title = "Catalog",
        Source = metadata.RequestUri,
        Content = text
    }
};
var dest = new MemoryDestination<Article>();

source.LinkTo(dest);
Network.Execute(source);

foreach (Article row in dest.Data) {
    Console.WriteLine($"Title:{row.Title}");
    Console.WriteLine($"Source:{row.Source}");
}

//Outputs
//Title:Catalog
//Source:catalog.md
```

## Read each file in a folder

`Folder` emits one row per file in that folder. `RequestCount` numbers the files.

```C#
Directory.CreateDirectory("notes");
File.WriteAllText(Path.Combine("notes", "article-a.md"), "Wool socks for winter walks.");
File.WriteAllText(Path.Combine("notes", "article-b.md"), "Coffee grinder for fresh coffee.");

var source = new TextDocumentSource {
    Folder = "notes"
};
var dest = new MemoryDestination();

source.LinkTo(dest);
Network.Execute(source);

foreach (dynamic row in dest.Data) {
    StreamMetaData metadata = row.StreamMetaData;
    Console.WriteLine($"RequestCount:{metadata.RequestCount} File:{Path.GetFileName(metadata.RequestUri)}");
    Console.WriteLine(row.Text);
}

//Outputs
//RequestCount:1 File:article-a.md
//Wool socks for winter walks.
//RequestCount:2 File:article-b.md
//Coffee grinder for fresh coffee.
```

## Split the document

The dynamic text property is `Text`, which `ChunkTransformation` reads by default.

```C#
string sourceFile = "readme.md";
File.WriteAllText(sourceFile, """
    # Warehouse notes

    ETLBox reads this file as one element.
    The second paragraph stays in the same element.
    """);

var source = new TextDocumentSource(sourceFile);
var chunk = new ChunkTransformation {
    MaxTokensPerChunk = 8,
    OverlapTokens = 0
};
var dest = new MemoryDestination<Chunk<ExpandoObject>>();

source.LinkTo(chunk);
chunk.LinkTo(dest);
Network.Execute(source);

foreach (Chunk<ExpandoObject> row in dest.Data)
    Console.WriteLine($"Index:{row.Index} Text:{row.Text}");

//Outputs
//Index:0 Text:# Warehouse notes
//
//ETLBox reads
//Index:1 Text: this file as one element.
//The second
//Index:2 Text: paragraph stays in the same element.
```