ETLBox ETLBox ETLBox Buy Try Buy

Mongo DB

MongoDB is a popular NoSQL database known for its flexibility and JSON-based document model. The ETLBox MongoDB connector supports both typed and dynamic data, making it a good fit for scenarios where schemas may vary or evolve.

ETLBox provides connectors for reading from and writing to MongoDB collections. These connectors use the official MongoDB C# Driver   under the hood, relying on the built-in BsonSerializer for serialization and deserialization.

MongoDbSource

MongoDbSource reads documents from a MongoDB collection and maps them to strongly typed objects or ExpandoObject instances.

Basic Usage

var source = new MongoDbSource<MyRow> {
    ConnectionString = "mongodb://localhost:27017",
    DatabaseName = "etlbox",
    CollectionName = "movies"
};
var dest = new MemoryDestination<MyRow>();
source.LinkTo(dest);
Network.Execute(source);

Filtering with FilterDefinition

You can restrict results using MongoDB’s FilterDefinition:

var filter = Builders<BsonDocument>.Filter.Gt("Col1", 2);
var source = new MongoDbSource<MyRow> {
    ConnectionString = "<your-connection-string>",
    DatabaseName = "etlbox",
    CollectionName = "movies",
    Filter = filter,
    FindOptions = new FindOptions {
        BatchSize = 2
    }
};

The FindOptions let you optimize data retrieval:

  • BatchSize controls how many documents are retrieved per batch.
  • Other options like Limit, Skip, and Projection are also supported.

See the MongoDB driver docs on FindOptions   for full details.

Using Dynamic Objects

If the structure of your MongoDB documents is not known at compile time, use a dynamic source:

var source = new MongoDbSource {
    ConnectionString = "mongodb://localhost:27017",
    DatabaseName = "etlbox",
    CollectionName = "movies"
};
var dest = new MemoryDestination();
source.LinkTo(dest);
Network.Execute(source);

foreach (dynamic row in dest.Data) {
    Console.WriteLine(row.Title);
}

Connection Options

You can configure access to your MongoDB server in two ways:

var source = new MongoDbSource<MyRow> {
    ConnectionString = "mongodb://localhost:27017",
    DatabaseName = "etlbox",
    CollectionName = "movies"
};

Using an existing MongoClient

Reusing an initialized client gives you better control over client configuration and resource management:

var client = new MongoClient("mongodb://localhost:27017");

var source = new MongoDbSource<MyRow> {
    DbClient = client,
    DatabaseName = "etlbox",
    CollectionName = "movies"
};

var dest = new MongoDbDestination<MyRow> {
    DbClient = client,
    DatabaseName = "etlbox",
    CollectionName = "archive"
};

You don’t need to dispose of the MongoClient. The MongoDB driver manages connections internally.

MongoDbDestination

MongoDbDestination writes objects or dynamic data into a MongoDB collection. It supports batch operations and can work with client-managed sessions.

Writing POCOs

var source = new MemorySource<MyRow>();
source.Data = GetSampleData();

var dest = new MongoDbDestination<MyRow> {
    ConnectionString = "mongodb://localhost:27017",
    DatabaseName = "etlbox",
    CollectionName = "movies"
};

source.LinkTo(dest);
Network.Execute(source);

Writing Dynamic Data

var source = new MemorySource();
dynamic row = new ExpandoObject();
row.Title = "Test Movie";
row.Year = 2023;
source.DataAsList.Add(row);

var dest = new MongoDbDestination {
    ConnectionString = "mongodb://localhost:27017",
    DatabaseName = "etlbox",
    CollectionName = "movies"
};

source.LinkTo(dest);
Network.Execute(source);

Using MongoDB Sessions

If you’re working with transactions or require finer control over batch execution, you can assign an active session to the destination:

var client = new MongoClient("mongodb://localhost:27017");
using var session = client.StartSession();

var dest = new MongoDbDestination<MyRow> {
    DbClient = client,
    Session = session,
    DatabaseName = "etlbox",
    CollectionName = "movies"
};

This allows ETLBox operations to participate in a larger transaction or reuse an existing context. Make sure the MongoDB deployment supports transactions (e.g., replica sets or sharded clusters).

Edit this page on GitHub
Prev
Couchbase
Next
Neo4j