In this tutorial, I will show you how to upload file to Google Cloud Storage (GCS) with Node.js example.
This Node.js App works with:
– Angular 8 / Angular 10 / Angular 11 / Angular 12 / Angular 13 / Angular 14 / Angular 15 / Angular 16
– Angular Material 12 / Angular Material 14 / Angular Material 15 / Angular Material 16
– Vue Client / Vuetify Client
– React Client / React Hooks Client
– React Image Upload with Preview
– Material UI Client
– Axios Client
Related Posts:
– Node.js Express File Upload (to static folder) example
– How to upload multiple files in Node.js
– Upload/store images in MySQL using Node.js, Express & Multer
– How to upload/store images in MongoDB using Node.js, Express & Multer
– Node.js Rest APIs example with Express, Sequelize & MySQL
Contents
- Node.js upload File to Google Cloud Storage example
- Technology
- Project Structure
- Setup Node.js File Upload to GCS project
- Setup Google Cloud Service Bucket with Credentials
- Create middleware for processing file
- Restrict file size before uploading to GCS
- Create Controller for GCS file upload/download
- Define Route for uploading file
- Create Express app server
- Run & Check
- Conclusion
- Further Reading
- Source Code
Node.js upload File to Google Cloud Storage example
Our Node.js Application will provide APIs for:
- uploading File to Google Cloud Storage bucket (restricted file size: 2MB)
- downloading File from server with the link
- getting list of Files’ information (file name & url)
Here are APIs to be exported:
Methods | Urls | Actions |
---|---|---|
POST | /upload | upload a File |
GET | /files | get List of Files (name & url) |
GET | /files/[filename] | download a File |
– Upload a File:
– This is the bucket that stores all uploaded files:
– If we get list of files, the Node.js Rest Apis will return:
– Upload a File with size larger than max file size (2MB):
– Download any file from one of the paths above, or via the API.
For example: http://localhost:8080/files/1.png
.
Technology
- @google-cloud/storage 5.8.5
- express 4.17.1
- multer 1.4.2
- cors 2.8.5
Project Structure
This is the project directory that we’re gonna build:
– google-cloud-key.json
contains credentials for working with Google Cloud Storage.
– middleware/upload.js
: initializes Multer Storage engine and defines middleware function to process file before uploading it to Google Cloud Storage.
– file.controller.js
exports Rest APIs: POST a file, GET all files’ information, download a File with url.
– routes/index.js
: defines routes for endpoints that is called from HTTP Client, use controller to handle requests.
– server.js
: initializes routes, runs Express app.
Setup Node.js File Upload to GCS project
Open command prompt, change current directory to the root folder of our project.
Install GCS, Express, Multer, CORS modules with the following command:
npm install @google-cloud/storage express multer cors
The package.json file will look like this:
{
"name": "nodejs-upload-file-google-cloud-storage",
"version": "1.0.0",
"description": "Node.js Upload file to Google Cloud Storage example",
"main": "server.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [
"nodejs",
"upload",
"file",
"google cloud storage",
"rest api"
],
"author": "bezkoder",
"license": "ISC",
"dependencies": {
"@google-cloud/storage": "^5.8.5",
"cors": "^2.8.5",
"express": "^4.17.1",
"multer": "^1.4.2"
}
}
Setup Google Cloud Service Bucket with Credentials
First you need to follow step by step in this post for creating a new Bucket named bezkoder-e-commerce
with a Credentials JSON file.
Once you get the file, rename it to google-cloud-key.json and put it into the root folder of Node.js project.
Create middleware for processing file
The middleware will use Multer for handling multipart/form-data along with uploading files.
Inside middleware folder, create upload.js file:
const util = require("util");
const Multer = require("multer");
let processFile = Multer({
storage: Multer.memoryStorage()
}).single("file");
let processFileMiddleware = util.promisify(processFile);
module.exports = processFileMiddleware;
In the code above, we’ve done these steps:
– First, we import multer
module.
– Next, we configure multer
to use Memory Storage engine.
util.promisify()
makes the exported middleware object can be used with async-await
.
Restrict file size before uploading to GCS
With the new multer
API. We can add limits: { fileSize: maxSize }
to the object passed to multer()
to restrict file size.
const util = require("util");
const Multer = require("multer");
const maxSize = 2 * 1024 * 1024;
let processFile = Multer({
storage: Multer.memoryStorage(),
limits: { fileSize: maxSize },
}).single("file");
let processFileMiddleware = util.promisify(processFile);
module.exports = processFileMiddleware;
Create Controller for GCS file upload/download
In controller folder, create file.controller.js:
const upload = async (req, res) => {
...
};
const getListFiles = async (req, res) => {
...
};
const download = async (req, res) => {
...
};
module.exports = {
upload,
getListFiles,
download,
};
GCS Upload File API
For File Upload method, we will export upload()
function that:
- use middleware function for processing file
- use Storage Bucket object for file upload
- catch the error
- return response with message
const processFile = require("../middleware/upload");
const { format } = require("util");
const { Storage } = require("@google-cloud/storage");
// Instantiate a storage client with credentials
const storage = new Storage({ keyFilename: "google-cloud-key.json" });
const bucket = storage.bucket("bezkoder-e-commerce");
const upload = async (req, res) => {
try {
await processFile(req, res);
if (!req.file) {
return res.status(400).send({ message: "Please upload a file!" });
}
// Create a new blob in the bucket and upload the file data.
const blob = bucket.file(req.file.originalname);
const blobStream = blob.createWriteStream({
resumable: false,
});
blobStream.on("error", (err) => {
res.status(500).send({ message: err.message });
});
blobStream.on("finish", async (data) => {
// Create URL for directly file access via HTTP.
const publicUrl = format(
`https://storage.googleapis.com/${bucket.name}/${blob.name}`
);
try {
// Make the file public
await bucket.file(req.file.originalname).makePublic();
} catch {
return res.status(500).send({
message:
`Uploaded the file successfully: ${req.file.originalname}, but public access is denied!`,
url: publicUrl,
});
}
res.status(200).send({
message: "Uploaded the file successfully: " + req.file.originalname,
url: publicUrl,
});
});
blobStream.end(req.file.buffer);
} catch (err) {
res.status(500).send({
message: `Could not upload the file: ${req.file.originalname}. ${err}`,
});
}
};
module.exports = {
upload,
...
};
– We call middleware function processFile()
first.
– If the HTTP request doesn’t include a file, send 400
status in the response.
– We also catch the error and send 500
status with error message.
So, how to handle the case in that user uploads the file exceeding size limitation?
We check error code (LIMIT_FILE_SIZE
) in the catch()
block:
const upload = async (req, res) => {
try {
await processFile(req, res);
...
} catch (err) {
if (err.code == "LIMIT_FILE_SIZE") {
return res.status(500).send({
message: "File size cannot be larger than 2MB!",
});
}
res.status(500).send({
message: `Could not upload the file: ${req.file.originalname}. ${err}`,
});
}
};
GCS Read Files and Download API
For File Information and Download:
getListFiles()
: read all files in GCS bucket, return list of files’ information (name, url)download()
: receives file name as input parameter, get Media Link from file’s Meta Data, then redirect browser to Media Link.
...
const { Storage } = require("@google-cloud/storage");
const storage = new Storage({ keyFilename: "google-cloud-key.json" });
const bucket = storage.bucket("bezkoder-e-commerce");
const getListFiles = async (req, res) => {
try {
const [files] = await bucket.getFiles();
let fileInfos = [];
files.forEach((file) => {
fileInfos.push({
name: file.name,
url: file.metadata.mediaLink,
});
});
res.status(200).send(fileInfos);
} catch (err) {
console.log(err);
res.status(500).send({
message: "Unable to read list of files!",
});
}
};
const download = async (req, res) => {
try {
const [metaData] = await bucket.file(req.params.name).getMetadata();
res.redirect(metaData.mediaLink);
} catch (err) {
res.status(500).send({
message: "Could not download the file. " + err,
});
}
};
module.exports = {
...
getListFiles,
download,
};
Define Route for uploading file
When a client sends HTTP requests, we need to determine how the server will response by setting up the routes.
There are 3 routes with corresponding controller methods:
- POST
/upload
:upload()
- GET
/files
:getListFiles()
- GET
/files/[fileName]
:download()
Create index.js file inside routes folder with content like this:
const express = require("express");
const router = express.Router();
const controller = require("../controller/file.controller");
let routes = (app) => {
router.post("/upload", controller.upload);
router.get("/files", controller.getListFiles);
router.get("/files/:name", controller.download);
app.use(router);
};
module.exports = routes;
You can see that we use controller from file.controller.js.
Create Express app server
Finally, we create an Express server in server.js:
const cors = require("cors");
const express = require("express");
const app = express();
let corsOptions = {
origin: "http://localhost:8081",
};
app.use(cors(corsOptions));
const initRoutes = require("./src/routes");
app.use(express.urlencoded({ extended: true }));
initRoutes(app);
const port = 8080;
app.listen(port, () => {
console.log(`Running at localhost:${port}`);
});
What we do are:
– import express
and cors
modules:
- Express is for building the Rest apis
- cors provides Express middleware to enable CORS with various options.
– create an Express app, then add cors
middlewares using app.use()
method. Notice that we set origin: http://localhost:8081
.
– listen on port 8080 for incoming requests.
Run & Check
On the project root folder, run this command: node server.js
.
Then use Postman to make HTTP POST/GET requests.
Conclusion
Today we’ve learned how to create Google Cloud Storage with Node.js File Upload example using Express for Rest API and Multer for processing file middleware. You also know how to restrict file size and catch Multer file size limit error.
Following tutorials explain how to build Front-end Apps to work with our Node.js Express Server:
– Angular 8 / Angular 10 / Angular 11 / Angular 12 / Angular 13 / Angular 14 / Angular 15 / Angular 16
– Angular Material 12 / Angular Material 14 / Angular Material 15 / Angular Material 16
– Vue Client / Vuetify Client
– React Client / React Hooks Client
– React Image Upload with Preview
– Material UI Client
– Axios Client
If you want to upload file into server static folder, please visit:
Node.js Express File Upload Rest API example using Multer
Happy Learning! See you again.
Further Reading
- https://cloud.google.com/storage/docs/how-to
- https://www.npmjs.com/package/express
- https://www.npmjs.com/package/multer
Source Code
You can find the complete source code for this tutorial on Github.
Hi bezkoder,
thanks for your work, it helps a lot.
Yesterda I succeded to upload images to goocle cloud store.
Now I’m getting the undefined rq.body when testing with postman and file upload.
In server I’m using:
app.use(cors());
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
any ideas?