了解 RPC（API 协议概览、gRPC nodejs 演练和 Apache Arrow Flight）

协议的世界

协议只是一种标准，它使交易中的不同参与方更容易达成共识，并确保交易正确完成。任何人都不需要了解整个协议，只需了解与自己相关的部分即可。

例如，如果我需要寄信，我知道正确的做法是把信装进贴好邮票的信封，然后送到邮局，这样信就能顺利送达。至于中间的步骤（具体实施细节），我不需要知道就能正确地寄出一封信，而邮局工作人员都清楚这部分流程。

在计算机之间的通信中，我们有几种不同的协议，使我们能够完成日常生活中许多事情。

• TCP（传输控制协议）该协议定义了计算机如何在最根本的层面上相互识别和发送文本消息。

使用 TCP，我们的计算机可以来回发送消息，但如果我不知道消息的结构，我就无法编写软件来处理它，因此许多更高级别的协议定义了消息的结构，其中最著名的是超文本传输​​协议 (HTTP)。

因此，当大多数软件通过互联网相互通信时，它们使用如下所示的 HTTP 消息：

GET /tutorial.htm HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: www.devnursery.com
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive

你可能从未手动编写过类似上述的 HTTP 消息，这是因为互联网浏览器已经为我们处理了这些。正是由于协议的存在，软件才能被创建并可靠运行。如果你编写过 Web 服务器，通常也不会担心上述消息，因为大多数编程语言都有专门用于解析这些 HTTP 消息的 Web 框架，真是太棒了。

API协议

在创建应用程序接口（API）以便应用程序之间能够相互通信（比计算机之间的通信高一个层级）时，我们也需要协议。虽然它们可以使用 TCP 和 HTTP 等标准格式相互发送消息，但应用层需要更高级别的协议，而这方面有很多选择。

假设我们正在创建一个服务，互联网上的服务器可以接收、请求并返回服务器存储的狗的名字（虽然不切实际，但有助于我们说明问题）。

REST（表述性状态转移）

在 REST 架构中，我们将服务中应提供的每个操作都设置为由对特定 URL 的特定请求触发。例如，使用 ExpressJS Web 框架时，您可能会看到类似这样的代码。

app.get("/dog", (request, response) => {
    response.json({name: "Spot", age: 6})
})

因此，基于上述 REST 端点，如果有人向 /dog (somewebsite.com/dog) 发出 HTTP 请求，则会触发上述路由并返回狗狗的数据。

This is by far the most common and popular way to create an API but as a service gets more and more complex the list of of URL can get quite long and it's up to the developers to create the documentation to make the API usuable to consumer, although tools like Swagger (auto generates documentation) make this part easier.

GraphQL

Facebook developed their own protocol that has grown in popularity called graphQL, and the way it works is there is only one url that all requests are sent to:

For example: somewebsite.com/graphql

All requests made to this url are http POST requests, but the big difference is these requests in their body have a string that defines what the consumer is looking for.

query {
    getDog {
        name
        age
    }
}

The above graphQL query would tell the server I want to run a query called getDog and of the data that query returns I only want the name and age.

The server would then search through all the defined queries (for getting data) and mutation (for creating, updating and deleting data) to see if there is match and process the query.

Benefits:

• One url for making requests
• Queries allow you to define which data you want back
• Auto-generates documentation

Cons:

• Must write type definitions when creating api

RPC (Remote Procedure Call)

RPC is an approach where you define certain services a server has available and messages (types of data) services need to receive or that they return. In this regard it is very similar to defining GraphQL types but the big different is in the consumers experience. Instead of writing http request or queries in a special language they write function calls that look writing normal function calls.

client.getDog({}, (error, dog) => {
    if (error){
        console.log(error)
    } else {
        console.log(dog)
    }
})

The above is how an RPC call using the gRPC framework would look like. We would of course need documentation to know which services are available from the RPC server. The beauty of RPC for defining an API is the type definitions are independant of the language implementation using something called protobuff. You can define all the services and message independantly and different servers and clients can implement it as needed.

This is why Apache Arrow Flight (a new standard for connecting to databases) chose RPC for their API, they can create a standard definition then different databases that would act as a server can implement the details on how those services would work for their particular system but for clients the experience will feel the same regardless of the database they use Arrow Flight to connect to.

Creating an RPC API with gRPC and Nodejs

(must have NodeJS installed)

• open up your terminal to an empty folder you can work out of

• create a new node project npm init -y

• Install our depedencies
  

npm install @grpc/grpc-js @grpc/proto-loader

• create three files touch server.js client.js service_def.proto

service_def.proto

Proto files are how we define our service (the functions the client call) and the messages (the data types that are either received or returned by services). The beauty is the proto files are language agnostic so someone can take the same proto file and then implement it using Go, Rust, Python, Java and so forth without having to make any changes to the proto file.

Proto3 Syntax Docs

syntax = "proto3";

message Empty {}

message Dog {
    string name = 1;
    int32 age = 2;
}

service DogService {
    rpc GetDog (Empty) returns (Dog) {}
}

So have defined there is a type of data called Dog with two fields, name and age (fields must be numbered). The DogService is a collection of signatures of the different actions this service has available and what they need to receive (Empty equals an empty object) and what it will return in response (a Dog).

That's it, now anyone can use that proto file to implement this service, but let's make an implementation ourselves.

Server.js

Now let's implement the service defined in our proto file. We will:

• load up our dependencies
• load up the proto file
• define the implmentation of the DogService and its GetDog procedure

// Load dependencies
const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");

// Path to our proto file
const PROTO_FILE = "./service_def.proto";

// Options needed for loading Proto file
const options = {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
};

// Load Proto File
const pkgDefs = protoLoader.loadSync(PROTO_FILE, options);
// Load Definition into gRPC
const dogProto = grpc.loadPackageDefinition(pkgDefs);
// Create gRPC server
const server = new grpc.Server();

// Implement DogService
server.addService(dogProto.DogService.service, {
  // Implment GetDog
  GetDog: (input, callback) => {
    try {
      callback(null, { name: "Spot", age: 5 });
    } catch (error) {
      callback(error, null);
    }
  },
});

// Start the Server
server.bindAsync(
  // Port to serve on
  "127.0.0.1:3500",
  // authentication settings
  grpc.ServerCredentials.createInsecure(),
  //server start callback
  (error, port) => {
    console.log(`listening on port ${port}`);
    server.start();
  }
);

So now we can run the server with node server.js and the server should be listening for requests. Now we will define a client to make a request to the server.

client.js

In this file we will:

• load up our dependencies
• create our client
• make a request to the GetDog Procedure

// Load up dependencies
const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");
// Path to proto file
const PROTO_FILE = "./service_def.proto";

// Options needed for loading Proto file
const options = {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
};

// Load Proto File
const pkgDefs = protoLoader.loadSync(PROTO_FILE, options);
// Load Definition into gRPC
const DogService = grpc.loadPackageDefinition(pkgDefs).DogService;

// Create the Client
const client = new DogService(
  "localhost:3500",
  grpc.credentials.createInsecure()
);

// make a call to GetDog
client.GetDog({}, (error, dog) => {
  if (error) {
    console.log(error);
  } else {
    console.log(dog);
  }
});

That's it. To reiterate the benefit of this approach is you can have one definition of a service (the proto file), which can be implemented many times but work with the same client since all server implmentations are bound by the protofile.

So in the case of Apache Arrow, instead of having to create different a different client library for each type of database in each language, each database can implement a server of the Apache Arrow Flight service and one client can be created per language that'll work with all databases, how cool is that!

The Code From This Tutorial