Google Cloud Storage with Node.js: File Upload example

In this tutorial, I will show you how to upload file to Google Cloud Storage (GCS) with Node.js example.

This Node.js App works with:
– Angular 8 / Angular 10 / Angular 11 / Angular 12 / Angular 13 / Angular 14 / Angular 15
– Angular Material 12 / Angular Material 14 / Angular Material 15
– Vue Client / Vuetify Client
– React Client / React Hooks Client
– React Image Upload with Preview
– Material UI Client
– Axios Client

Related Posts:
– Node.js Express File Upload (to static folder) example
– How to upload multiple files in Node.js
– Upload/store images in MySQL using Node.js, Express & Multer
– How to upload/store images in MongoDB using Node.js, Express & Multer
– Node.js Rest APIs example with Express, Sequelize & MySQL

Node.js upload File to Google Cloud Storage example

Our Node.js Application will provide APIs for:

• uploading File to Google Cloud Storage bucket (restricted file size: 2MB)
• downloading File from server with the link
• getting list of Files’ information (file name & url)

Here are APIs to be exported:

– Upload a File:

– This is the bucket that stores all uploaded files:

– If we get list of files, the Node.js Rest Apis will return:

– Upload a File with size larger than max file size (2MB):

– Download any file from one of the paths above, or via the API.
For example: http://localhost:8080/files/1.png.

Technology

• @google-cloud/storage 5.8.5
• express 4.17.1
• multer 1.4.2
• cors 2.8.5

Project Structure

This is the project directory that we’re gonna build:

– google-cloud-key.json contains credentials for working with Google Cloud Storage.
– middleware/upload.js: initializes Multer Storage engine and defines middleware function to process file before uploading it to Google Cloud Storage.
– file.controller.js exports Rest APIs: POST a file, GET all files’ information, download a File with url.
– routes/index.js: defines routes for endpoints that is called from HTTP Client, use controller to handle requests.
– server.js: initializes routes, runs Express app.

Setup Node.js File Upload to GCS project

Open command prompt, change current directory to the root folder of our project.
Install GCS, Express, Multer, CORS modules with the following command:

npm install @google-cloud/storage express multer cors

The package.json file will look like this:

{
  "name": "nodejs-upload-file-google-cloud-storage",
  "version": "1.0.0",
  "description": "Node.js Upload file to Google Cloud Storage example",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "nodejs",
    "upload",
    "file",
    "google cloud storage",
    "rest api"
  ],
  "author": "bezkoder",
  "license": "ISC",
  "dependencies": {
    "@google-cloud/storage": "^5.8.5",
    "cors": "^2.8.5",
    "express": "^4.17.1",
    "multer": "^1.4.2"
  }
}

Setup Google Cloud Service Bucket with Credentials

First you need to follow step by step in this post for creating a new Bucket named bezkoder-e-commerce with a Credentials JSON file.

Once you get the file, rename it to google-cloud-key.json and put it into the root folder of Node.js project.

Create middleware for processing file

The middleware will use Multer for handling multipart/form-data along with uploading files.

Inside middleware folder, create upload.js file:

const util = require("util");
const Multer = require("multer");

let processFile = Multer({
  storage: Multer.memoryStorage()
}).single("file");

let processFileMiddleware = util.promisify(processFile);
module.exports = processFileMiddleware;

In the code above, we’ve done these steps:
– First, we import multer module.
– Next, we configure multer to use Memory Storage engine.

util.promisify() makes the exported middleware object can be used with async-await.

Restrict file size before uploading to GCS

With the new multer API. We can add limits: { fileSize: maxSize } to the object passed to multer() to restrict file size.

const util = require("util");
const Multer = require("multer");
const maxSize = 2 * 1024 * 1024;

let processFile = Multer({
  storage: Multer.memoryStorage(),
  limits: { fileSize: maxSize },
}).single("file");

let processFileMiddleware = util.promisify(processFile);
module.exports = processFileMiddleware;

Create Controller for GCS file upload/download

In controller folder, create file.controller.js:

const upload = async (req, res) => {
  ...
};

const getListFiles = async (req, res) => {
  ...
};

const download = async (req, res) => {
  ...
};

module.exports = {
  upload,
  getListFiles,
  download,
};

GCS Upload File API

For File Upload method, we will export upload() function that:

• use middleware function for processing file
• use Storage Bucket object for file upload
• catch the error
• return response with message

const processFile = require("../middleware/upload");
const { format } = require("util");
const { Storage } = require("@google-cloud/storage");
// Instantiate a storage client with credentials
const storage = new Storage({ keyFilename: "google-cloud-key.json" });
const bucket = storage.bucket("bezkoder-e-commerce");

const upload = async (req, res) => {
  try {
    await processFile(req, res);

    if (!req.file) {
      return res.status(400).send({ message: "Please upload a file!" });
    }

    // Create a new blob in the bucket and upload the file data.
    const blob = bucket.file(req.file.originalname);
    const blobStream = blob.createWriteStream({
      resumable: false,
    });

    blobStream.on("error", (err) => {
      res.status(500).send({ message: err.message });
    });

    blobStream.on("finish", async (data) => {
      // Create URL for directly file access via HTTP.
      const publicUrl = format(
        `https://storage.googleapis.com/${bucket.name}/${blob.name}`
      );

      try {
        // Make the file public
        await bucket.file(req.file.originalname).makePublic();
      } catch {
        return res.status(500).send({
          message:
            `Uploaded the file successfully: ${req.file.originalname}, but public access is denied!`,
          url: publicUrl,
        });
      }

      res.status(200).send({
        message: "Uploaded the file successfully: " + req.file.originalname,
        url: publicUrl,
      });
    });

    blobStream.end(req.file.buffer);
  } catch (err) {
    res.status(500).send({
      message: `Could not upload the file: ${req.file.originalname}. ${err}`,
    });
  }
};

module.exports = {
  upload,
  ...
};

– We call middleware function processFile() first.
– If the HTTP request doesn’t include a file, send 400 status in the response.
– We also catch the error and send 500 status with error message.

So, how to handle the case in that user uploads the file exceeding size limitation?
We check error code (LIMIT_FILE_SIZE) in the catch() block:

const upload = async (req, res) => {
  try {
    await processFile(req, res);
    ...
  } catch (err) {
    if (err.code == "LIMIT_FILE_SIZE") {
      return res.status(500).send({
        message: "File size cannot be larger than 2MB!",
      });
    }

    res.status(500).send({
      message: `Could not upload the file: ${req.file.originalname}. ${err}`,
    });
  }
};

GCS Read Files and Download API

For File Information and Download:

• getListFiles(): read all files in GCS bucket, return list of files’ information (name, url)
• download(): receives file name as input parameter, get Media Link from file’s Meta Data, then redirect browser to Media Link.

...
const { Storage } = require("@google-cloud/storage");

const storage = new Storage({ keyFilename: "google-cloud-key.json" });
const bucket = storage.bucket("bezkoder-e-commerce");

const getListFiles = async (req, res) => {
  try {
    const [files] = await bucket.getFiles();
    let fileInfos = [];

    files.forEach((file) => {
      fileInfos.push({
        name: file.name,
        url: file.metadata.mediaLink,
      });
    });

    res.status(200).send(fileInfos);
  } catch (err) {
    console.log(err);

    res.status(500).send({
      message: "Unable to read list of files!",
    });
  }
};

const download = async (req, res) => {
  try {
    const [metaData] = await bucket.file(req.params.name).getMetadata();
    res.redirect(metaData.mediaLink);
    
  } catch (err) {
    res.status(500).send({
      message: "Could not download the file. " + err,
    });
  }
};

module.exports = {
  ...
  getListFiles,
  download,
};

Define Route for uploading file

When a client sends HTTP requests, we need to determine how the server will response by setting up the routes.

There are 3 routes with corresponding controller methods:

• POST /upload: upload()
• GET /files: getListFiles()
• GET /files/[fileName]: download()

Create index.js file inside routes folder with content like this:

const express = require("express");
const router = express.Router();
const controller = require("../controller/file.controller");

let routes = (app) => {
  router.post("/upload", controller.upload);
  router.get("/files", controller.getListFiles);
  router.get("/files/:name", controller.download);

  app.use(router);
};

module.exports = routes;

You can see that we use controller from file.controller.js.

Create Express app server

Finally, we create an Express server in server.js:

const cors = require("cors");
const express = require("express");
const app = express();

let corsOptions = {
  origin: "http://localhost:8081",
};

app.use(cors(corsOptions));

const initRoutes = require("./src/routes");

app.use(express.urlencoded({ extended: true }));
initRoutes(app);

const port = 8080;
app.listen(port, () => {
  console.log(`Running at localhost:${port}`);
});

What we do are:
– import express and cors modules:

• Express is for building the Rest apis
• cors provides Express middleware to enable CORS with various options.

– create an Express app, then add cors middlewares using app.use() method. Notice that we set origin: http://localhost:8081.
– listen on port 8080 for incoming requests.

Run & Check

On the project root folder, run this command: node server.js.
Then use Postman to make HTTP POST/GET requests.

Conclusion

Today we’ve learned how to create Google Cloud Storage with Node.js File Upload example using Express for Rest API and Multer for processing file middleware. You also know how to restrict file size and catch Multer file size limit error.

Following tutorials explain how to build Front-end Apps to work with our Node.js Express Server:
– Angular 8 / Angular 10 / Angular 11 / Angular 12 / Angular 13 / Angular 14 / Angular 15
– Angular Material 12 / Angular Material 14 / Angular Material 15
– Vue Client / Vuetify Client
– React Client / React Hooks Client
– React Image Upload with Preview
– Material UI Client
– Axios Client

If you want to upload file into server static folder, please visit:
Node.js Express File Upload Rest API example using Multer

Happy Learning! See you again.

Further Reading

• https://cloud.google.com/storage/docs/how-to
• https://www.npmjs.com/package/express
• https://www.npmjs.com/package/multer

Source Code

You can find the complete source code for this tutorial on Github.