A Node.js Developer's Guide to RPC

Network

21 days ago1521 words|Estimated reading time: 8 minutes

Hello! If you're hearing about RPC for the first time, don't worry. This article will guide you from scratch to understand what RPC is and how to implement a simple RPC system using Node.js.

What is RPC and Why Do We Need It?

Starting with Local Function Calls

Let's assume we have a function that calculates the sum of two numbers:

// Local function
function add(a, b) {
  return a + b;
}

const result = add(3, 5); // Direct call
console.log(result); // Output: 8

This is the most familiar way of calling a function—a local call, where all the code runs within the same process.

When Services Need to Be Distributed Across Multiple Machines

What if your add function needs to run on a different server? That's the problem RPC aims to solve.

RPC (Remote Procedure Call) is a technology that allows you to call a remote service as if it were a local function.

A Real-World Analogy

Local call: Directly asking a colleague, "Could you get me a coffee?"
Remote call: Sending a message to a colleague in another office, "Could you get me a coffee?"

The goal of RPC is to make the remote call feel as simple as a local one.

A Simple RPC Implementation

Let's use Node.js to build a basic TCP-based RPC system.

For simplicity, we will assume each TCP packet is a complete JSON request. In a real-world scenario, larger requests can be split into multiple packets, which would require more complex parsing.

Server Code

// server.js
const net = require('net');

// Our "remote" methods
const methods = {
  add: (a, b) => a + b,
  greet: (name) => `Hello, ${name}!`
};

const server = net.createServer((socket) => {
  console.log('Client connected');
  
  socket.on('data', (data) => {
    try {
      // Parse the request from the client
      const request = JSON.parse(data.toString());
      console.log('Received request:', request);
      
      // Execute the method
      const result = methods[request.method](...request.params);
      
      // Return the result
      const response = JSON.stringify({
        result,
        id: request.id
      });
      socket.write(response);
    } catch (error) {
      console.error('Error processing request:', error);
    }
  });
});

server.listen(3000, () => {
  console.log('RPC service started, listening on port 3000');
});

Client Code

// client.js
const net = require('net');

class SimpleRPCClient {
  constructor(port, host) {
    this.port = port || 3000;
    this.host = host || 'localhost';
    this.requestId = 0;
  }

  call(method, ...params) {
    return new Promise((resolve, reject) => {
      const socket = new net.Socket();
      const currentId = ++this.requestId;
      
      // Prepare the request data
      const request = JSON.stringify({
        method,
        params,
        id: currentId
      });
      
      socket.connect(this.port, this.host, () => {
        socket.write(request);
      });
      
      socket.on('data', (data) => {
        const response = JSON.parse(data.toString());
        if (response.id === currentId) {
          resolve(response.result);
          socket.end();
        }
      });
      
      socket.on('error', (err) => {
        reject(err);
      });
    });
  }
}

// Usage example
(async () => {
  const client = new SimpleRPCClient();
  
  try {
    // Call remote addition
    const sum = await client.call('add', 5, 3);
    console.log('5 + 3 =', sum);
    
    // Call remote greeting
    const greeting = await client.call('greet', 'Node.js Developer');
    console.log(greeting);
  } catch (err) {
    console.error('Call failed:', err);
  }
})();

Running the Example

Start the server first:

node server.js

Then, run the client:

node client.js

You'll see the client successfully call the remote methods and get the results!

Understanding Core RPC Concepts

Let's use our simple implementation to understand a few key concepts:

Serialization and Deserialization

Serialization: Converting data into a format that can be transmitted (like a JSON string)
Deserialization: Converting the received data back into its original format

In our example:

// Serialization
const request = JSON.stringify({ method: 'add', params: [1, 2] });

// Deserialization
const data = JSON.parse(request);

Communication Protocol

We used Node.js's net module to establish a TCP connection. In practice, RPC can use various protocols:

TCP: Better performance
HTTP: More versatile
WebSocket: Suitable for browsers

Request/Response Model

Every call has a clear request and a response:

// Request format
{
  method: 'method_name',
  params: [param1, param2],
  id: unique_identifier
}

// Response format
{
  result: 'result',
  id: corresponding_request_ID
}

Improving Our RPC Implementation

Let's improve on our simple implementation by adding error handling and more features.

Improved Server

// better-server.js
const net = require('net');

const methods = {
  add: (a, b) => a + b,
  greet: (name) => {
    if (!name) throw new Error('Name cannot be empty');
    return `Hello, ${name}!`;
  }
};

const server = net.createServer((socket) => {
  console.log('Client connected');
  
  socket.on('data', (data) => {
    try {
      const request = JSON.parse(data.toString());
      console.log('Received request:', request);
      
      if (!methods[request.method]) {
        throw new Error(`Method ${request.method} does not exist`);
      }
      
      const result = methods[request.method](...request.params);
      
      socket.write(JSON.stringify({
        result,
        id: request.id,
        error: null
      }));
    } catch (error) {
      socket.write(JSON.stringify({
        result: null,
        id: request?.id || null,
        error: error.message
      }));
    }
  });
});

server.listen(3000, () => {
  console.log('Improved RPC service started');
});

Improved Client

// better-client.js
const net = require('net');

class BetterRPCClient {
  constructor(port, host) {
    this.port = port || 3000;
    this.host = host || 'localhost';
    this.requestId = 0;
  }

  async call(method, ...params) {
    const socket = new net.Socket();
    const currentId = ++this.requestId;
    
    const request = JSON.stringify({
      method,
      params,
      id: currentId
    });
    
    return new Promise((resolve, reject) => {
      let responseData = '';
      
      socket.connect(this.port, this.host, () => {
        socket.write(request);
      });
      
      socket.on('data', (data) => {
        responseData += data.toString();
        
        try {
          const response = JSON.parse(responseData);
          if (response.id === currentId) {
            if (response.error) {
              reject(new Error(response.error));
            } else {
              resolve(response.result);
            }
            socket.end();
          }
        } catch (e) {
          // Data is incomplete, keep waiting
        }
      });
      
      socket.on('error', (err) => {
        reject(err);
      });
      
      socket.on('timeout', () => {
        reject(new Error('Request timed out'));
        socket.end();
      });
      
      socket.setTimeout(5000); // 5-second timeout
    });
  }
}

// Usage example
(async () => {
  const client = new BetterRPCClient();
  
  try {
    // Normal call
    console.log('1 + 2 =', await client.call('add', 1, 2));
    
    // Call a non-existent method
    await client.call('nonexistent');
  } catch (err) {
    console.error('Call error:', err.message);
  }
  
  try {
    // Test error handling
    await client.call('greet'); // No name provided
  } catch (err) {
    console.error('Test error:', err.message);
  }
})();

Summary of Improvements

Better Error Handling: The server can catch errors and return them to the client.
Timeout Mechanism: The client has a 5-second timeout.
Data Integrity: We now handle cases where the data is split into multiple packets.
More Robust Protocol: The response now includes an error field.

Using an Existing RPC Framework

While we built a simple RPC from scratch, in a real-world project, you'll use a mature RPC framework. Let's see how to use gRPC.

Install gRPC

npm install @grpc/grpc-js @grpc/proto-loader

Define Service Interface

Create a calculator.proto file:

syntax = "proto3";

service Calculator {
  rpc Add (Numbers) returns (Result);
  rpc Multiply (Numbers) returns (Result);
}

message Numbers {
  double a = 1;
  double b = 2;
}

message Result {
  double value = 1;
}

Implement the Server

// grpc-server.js
const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');

// Load the proto file
const packageDefinition = protoLoader.loadSync('calculator.proto');
const calculatorProto = grpc.loadPackageDefinition(packageDefinition);

const server = new grpc.Server();

// Implement service methods
const calculator = {
  add: (call, callback) => {
    callback(null, { 
      value: call.request.a + call.request.b 
    });
  },
  multiply: (call, callback) => {
    callback(null, { 
      value: call.request.a * call.request.b 
    });
  }
};

// Add the service
server.addService(calculatorProto.Calculator.service, calculator);

// Start the server
server.bindAsync(
  '0.0.0.0:50051',
  grpc.ServerCredentials.createInsecure(),
  (err, port) => {
    if (err) {
      console.error(err);
      return;
    }
    server.start();
    console.log(`gRPC service running on port ${port}`);
  }
);

Implement the Client

// grpc-client.js
const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');

const packageDefinition = protoLoader.loadSync('calculator.proto');
const calculatorProto = grpc.loadPackageDefinition(packageDefinition);

const client = new calculatorProto.Calculator(
  'localhost:50051',
  grpc.credentials.createInsecure()
);

// Call addition
client.add({ a: 5, b: 3 }, (err, response) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log('5 + 3 =', response.value);
});

// Call multiplication
client.multiply({ a: 4, b: 6 }, (err, response) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log('4 * 6 =', response.value);
});

Advantages of gRPC

High Performance: Uses a binary protocol.
Cross-Language Support: Supports a wide range of programming languages.
Code Generation: Automatically generates client and server code.
Rich Features: Supports streaming, authentication, and more.

When to Use RPC

Scenarios Where RPC is a Good Fit

Microservice Architecture: Communication between services.
High-Performance Needs: Such as game servers or trading systems.
Internal Systems: Calling internal company services.

Scenarios Where RPC is Not a Good Fit

Browser-to-Server Communication: Typically uses REST or GraphQL.
Simple Projects: RPC might be over-engineered for a small number of services.

Summary

What We've Learned

RPC allows us to call remote services as if they were local functions.
We implemented a simple TCP-based RPC system.
We improved our implementation by adding error handling and timeouts.
We used the gRPC framework to implement a more professional RPC service.

I hope this guide helps you understand the basic concepts of RPC! In your projects, you can choose between a simple custom implementation or a mature framework based on your needs.