Golang Protobuf

1. 定义协议文件 .proto

syntax = 'proto3';
package proto;

import "google/protobuf/timestamp.proto";

option go_package = "../proto_gen";

message Person {
    string name = 1;
    int32  id = 2;
    string email = 3;

    enum PhoneType {
        MOBILE = 0;
        HOME = 1;
        WORK = 2;
    }

    message PhoneNum {
        string number = 1;
        PhoneType type = 2;
    }

    repeated PhoneNum phones = 4;

    google.protobuf.Timestamp last_updated = 5;
}

message AddressBook {
    repeated Person person = 1;
}

• go_package: Go 文件生成的目录, protoc 中的 --go_out 目录相关联. package 名和目录名一样.

2. 使用 protocol buffer 编译

2.1 安装编译器

Protocol Buffers

2.2 安装 Go protocol buffers 插件

go install google.golang.org/protobuf/cmd/protoc-gen-go

2.3 编辑 .proto 协议文件为 Go 文件

protoc --proto_path=./ --go_out=./ addressbook.proto

生成文件名为: addressbook.pb.go, package 名和目录名一样.

• --proto_path: .proto 文件的目录, 也可以用 -I
• --go_out: 生成的 Go 文件目录
• 最后是 .proto 文件

Usage: protoc [OPTION] PROTO_FILES                                   
Parse PROTO_FILES and generate output based on the options given:         
  -IPATH, --proto_path=PATH   Specify the directory in which to search for
                              imports.  May be specified multiple times;    
                              directories will be searched in order.  If not
                              given, the current working directory is used.
                              If not found in any of the these directories,
                              the --descriptor_set_in descriptors will be
                              checked for required proto file.       
  --version                   Show version info and exit.                  
  -h, --help                  Show this text and exit.                    
  --encode=MESSAGE_TYPE       Read a text-format message of the given type  
                              from standard input and write it in binary
                              to standard output.  The message type must 
                              be defined in PROTO_FILES or their imports.
  --deterministic_output      When using --encode, ensure map fields are  
                              deterministically ordered. Note that this order
                              is not canonical, and changes across builds or
                              releases of protoc.                     
  --decode=MESSAGE_TYPE       Read a binary message of the given type from
                              standard input and write it in text format
                              to standard output.  The message type must 
                              be defined in PROTO_FILES or their imports.
  --decode_raw                Read an arbitrary protocol message from  
                              standard input and write the raw tag/value
                              pairs in text format to standard output.  No
                              PROTO_FILES should be given when using this   
                              flag.                                      
  --descriptor_set_in=FILES   Specifies a delimited list of FILES     
                              each containing a FileDescriptorSet (a
                              protocol buffer defined in descriptor.proto).
                              The FileDescriptor for each of the PROTO_FILES
                              provided will be loaded from these     
                              FileDescriptorSets. If a FileDescriptor     
                              appears multiple times, the first occurrence
                              will be used.                                 
  -oFILE,                     Writes a FileDescriptorSet (a protocol buffer,
    --descriptor_set_out=FILE defined in descriptor.proto) containing all of
                              the input files to FILE.       
  --include_imports           When using --descriptor_set_out, also include
                              all dependencies of the input files in the
                              set, so that the set is self-contained.
  --include_source_info       When using --descriptor_set_out, do not strip
                              SourceCodeInfo from the FileDescriptorProto.
                              This results in vastly larger descriptors that
                              include information about the original
                              location of each decl in the source file as
                              well as surrounding comments.             
  --dependency_out=FILE       Write a dependency output file in the format
                              expected by make. This writes the transitive
                              set of input file paths to FILE            
  --error_format=FORMAT       Set the format in which to print errors.  
                              FORMAT may be 'gcc' (the default) or 'msvs' 
                              (Microsoft Visual Studio format).        
  --fatal_warnings            Make warnings be fatal (similar to -Werr in
                              gcc). This flag will make protoc return  
                              with a non-zero exit code if any warnings   
                              are generated.                             
  --print_free_field_numbers  Print the free field numbers of the messages
                              defined in the given proto files. Groups share
                              the same field number space with the parent
                              message. Extension ranges are counted as
                              occupied fields numbers.
  --plugin=EXECUTABLE         Specifies a plugin executable to use.
                              Normally, protoc searches the PATH for
                              plugins, but you may specify additional
                              executables not in the path using this flag.
                              Additionally, EXECUTABLE may be of the form
                              NAME=PATH, in which case the given plugin name
                              is mapped to the given executable even if
                              the executable's own name differs.
  --cpp_out=OUT_DIR           Generate C++ header and source.
  --csharp_out=OUT_DIR        Generate C# source file.
  --java_out=OUT_DIR          Generate Java source file.
  --js_out=OUT_DIR            Generate JavaScript source.
  --kotlin_out=OUT_DIR        Generate Kotlin file.
  --objc_out=OUT_DIR          Generate Objective-C header and source.
  --php_out=OUT_DIR           Generate PHP source file.
  --python_out=OUT_DIR        Generate Python source file.
  --ruby_out=OUT_DIR          Generate Ruby source file.
  @<filename>                 Read options and filenames from file. If a
                              relative file path is specified, the file
                              will be searched in the working directory.
                              The --proto_path option will not affect how
                              this argument file is searched. Content of
                              the file will be expanded in the position of
                              @<filename> as in the argument list. Note
                              that shell expansion is not applied to the
                              content of the file (i.e., you cannot use
                              quotes, wildcards, escapes, commands, etc.).
                              Each line corresponds to a single argument,
                              even if it contains spaces.

3. 使用 Go protocol buffer API 读写数据

go.mod 添加以下依赖：

require (
	github.com/golang/protobuf v1.5.2 // indirect
	google.golang.org/protobuf v1.26.0 // indirect
)

3.1 序列化

person := proto_gen.Person{
    
    Id:    10001,
    Name:  "GZ",
    Email: "[email protected]",
    Phones: []*proto_gen.Person_PhoneNum{
    
        {
    Number: "123-456", Type: proto_gen.Person_HOME},
    },
}

out, err := proto.Marshal(&person)
if err != nil {
    
    fmt.Println("序列化错误: ", err)
}
if err = ioutil.WriteFile("./encode.data", out, os.ModePerm); err != nil {
    
    fmt.Println("写入本地文件错误: ", err)
}

3.2 反序列化

in, err := ioutil.ReadFile("./encode.data")
if err != nil {
    
    fmt.Println("读取本地文件错误: ", err)
}
person2 := &proto_gen.Person{
    }
if err = proto.Unmarshal(in, person2); err != nil {
    
    fmt.Println("解析数据错误: ", err)
} else {
    
    fmt.Println(person2)
}

4. 注意

在项目的持续开发中, 如果需要扩展协议, 需要注意一下几点:

1. 不要修改现有的字段的标号
2. 可以删除字段
3. 可以新增字段, 但是标号一定要是新的, 删除字段的标号也不要重用

这样新版本代码可以处理老版本协议的数据. 老版本代码也可以处理新版本协议的数据.