Import CSV file into MongoDB collection using Node.js

bezkoder MongoDB, Node.js

Today we’re gonna show you way to use Node.js to import CSV file into a MongoDB Database collection with two steps:

Related Post: Export MongoDB collection to CSV file using Node.js

More Practice:
– Node.js, Express & MongoDb: Build a CRUD Rest Api example
– Node.js + MongoDB: User Authentication & Authorization with JWT



Assume that we have CSV file bezkoder.csv like this:

id,name,description,createdAt
1,Node.js,JavaScript runtime environment,2019-09-03
2,Vue.js,JavaScript Framework for building UI,2019-09-06
3,Angular.js,Platform for building mobile & desktop web app,2019-09-09

All we need is to import each row (except header) to MongoDB colection category.
Now let’s do it.

Read CSV file using csvtojson

We will use fast-csv module, so run the command:

npm install csvtojson

Then we use csvtojson module to parse the data to JSON format from a CSV file.

const csvtojson = require("csvtojson");

csvtojson()
  .fromFile("bezkoder.csv")
  .then(csvData => {
    console.log(csvData);
/*
[ { id: '1',
    name: 'Node.js',
    description: 'JavaScript runtime environment',
    createdAt: '2019-09-03' },
  { id: '2',
    name: 'Vue.js',
    description: 'JavaScript Framework for building UI',
    createdAt: '2019-09-06' },
  { id: '3',
    name: 'Angular.js',
    description: 'Platform for building mobile & desktop web app',
    createdAt: '2019-09-09' } ]
*/
  });

In the code above, fromFile() function returns a Promise that we receive the JSON-like array. It’s so simple.

Read CSV file using fast-csv

First, install fast-csv with the command:

npm install fast-csv

This method is more complicated. We need fast-csv module long with fs.
The purpose for using fs is to create a ReadStream from CSV file, then use fast-csv module to parse the data and pass the result to ReadStream pipe:

const fs = require("fs");
const fastcsv = require("fast-csv");

let stream = fs.createReadStream("bezkoder.csv");
let csvData = [];
let csvStream = fastcsv
  .parse()
  .on("data", function(data) {
    csvData.push({
      id: data[0],
      name: data[1],
      description: data[2],
      createdAt: data[3]
    });
  })
  .on("end", function() {
    // remove the first line: header
    csvData.shift();

    // save to the MongoDB database collection
  });

stream.pipe(csvStream);

In the code, there are 2 events: on('data') and on('end') that we implement the listener:
– 'data' is emitted when a record is parsed, so we will append that record (data) in the handler function. Rembemer that data is an array, so we translate it to an object.
– 'end' is emitted after the parsing is done (with all records), so we will save data (without the first item – header) to MongoDB in this.

Import CSV data to MongoDB Database using mongodb

Now we have csvData that contains all neccesary rows. Just save them to a MongoDB database collection.

Remember that the code will be written inside one of these:
– then() function
– 'end' event handler function.

const mongodb = require("mongodb").MongoClient;

// let url = "mongodb://username:password@localhost:27017/";
let url = "mongodb://localhost:27017/";

mongodb.connect(
  url,
  { useNewUrlParser: true, useUnifiedTopology: true },
  (err, client) => {
    if (err) throw err;
    client
      .db("zkoder_db")
      .collection("category")
      .insertMany(csvData, (err, res) => {
        if (err) throw err;
        console.log(`Inserted: ${res.insertedCount} rows`);
        client.close();
      });
  }
);

Let me describe the code.
– First we import MongoClient from mongodb module.
– Then we initialize value for MongoDB url, it indicates the host & the port. You can also set username & password in this url.
– We use connect() function of MongoClient to connect MongoDB. The callback function gives us a MongoDB client object.
– client.db("zkoder_db").collection("category") returns a Collection object corresponding to category collection in zkoder_db database.
– Finally we use insertMany() method to save CSV data to MongoDB database.

Combine all

Now we show you all of the code in one file.

Node.js import CSV data into MongoDB using csvtojson & mongodb

const mongodb = require("mongodb").MongoClient;
const csvtojson = require("csvtojson");

// let url = "mongodb://username:password@localhost:27017/";
let url = "mongodb://localhost:27017/";

csvtojson()
  .fromFile("bezkoder.csv")
  .then(csvData => {
    console.log(csvData);

    mongodb.connect(
      url,
      { useNewUrlParser: true, useUnifiedTopology: true },
      (err, client) => {
        if (err) throw err;

        client
          .db("zkoder_db")
          .collection("category")
          .insertMany(csvData, (err, res) => {
            if (err) throw err;

            console.log(`Inserted: ${res.insertedCount} rows`);
            client.close();
          });
      }
    );
  });

Node.js import CSV data into MongoDB using fast-csv & mongodb

const fs = require("fs");
const mongodb = require("mongodb").MongoClient;
const fastcsv = require("fast-csv");

// let url = "mongodb://username:password@localhost:27017/";
let url = "mongodb://localhost:27017/";
let stream = fs.createReadStream("bezkoder.csv");
let csvData = [];
let csvStream = fastcsv
  .parse()
  .on("data", function(data) {
    csvData.push({
      id: data[0],
      name: data[1],
      description: data[2],
      createdAt: data[3]
    });
  })
  .on("end", function() {
    // remove the first line: header
    csvData.shift();

    console.log(csvData);

    mongodb.connect(
      url,
      { useNewUrlParser: true, useUnifiedTopology: true },
      (err, client) => {
        if (err) throw err;

        client
          .db("zkoder_db")
          .collection("category")
          .insertMany(csvData, (err, res) => {
            if (err) throw err;

            console.log(`Inserted: ${res.insertedCount} rows`);
            client.close();
          });
      }
    );
  });

stream.pipe(csvStream);

Run & Check the result

After running the code, the console shows:

[ { id: '1',
    name: 'Node.js',
    description: 'JavaScript runtime environment',
    createdAt: '2019-09-03' },
  { id: '2',
    name: 'Vue.js',
    description: 'JavaScript Framework for building UI',
    createdAt: '2019-09-06' },
  { id: '3',
    name: 'Angular.js',
    description: 'Platform for building mobile & desktop web app',
    createdAt: '2019-09-09' } ]
Inserted: 3 rows

Check MongoDB collection:

> db.category.find()
{ "_id" : ObjectId("5d91b4aaae94cf3414c5dd90"), "id" : "1", "name" : "Node.js", "description" : "JavaScript runtime environment", "createdAt" : "2019-09-03" }
{ "_id" : ObjectId("5d91b4aaae94cf3414c5dd91"), "id" : "2", "name" : "Vue.js", "description" : "JavaScript Framework for building UI", "createdAt" : "2019-09-06" }
{ "_id" : ObjectId("5d91b4aaae94cf3414c5dd92"), "id" : "3", "name" : "Angular.js", "description" : "Platform for building mobile & desktop web app", "createdAt" : "2019-09-09" }

Conclusion

Today we’ve learned way to read CSV file using csvtojson and fast-csv module, then save CSV data to MongoDB collection using mongodb module.

In the next tutorial, we’ll show you how to export MongoDB data from a collection to a CSV file.

Happy learning! See you again.

Further Reading

Source Code

You can find the complete source code for this example on Github.

4 thoughts to “Import CSV file into MongoDB collection using Node.js”

    1. I am having four columns in my schema and In my csv file there are three columns .I wanted to insert the csv file data into above mentioned schema and add data to the fourth column with username every time.how can i do that?can you help me out?

Comments are closed to reduce spam. If you have any question, please send me an email.