This project hosts the Go implementation for
protocol buffers, which is a
language-neutral, platform-neutral, extensible mechanism for serializing
structured data. The protocol buffer language is a language for specifying the
schema for structured data. This schema is compiled into language specific
bindings. This project provides both a tool to generate Go code for the
protocol buffer language, and also the runtime implementation to handle
serialization of messages in Go. See the
protocol buffer developer guide
for more information about protocol buffers themselves.
Runtime library: The
protobuf module
contains a set of Go packages that form the runtime implementation of
protobufs in Go. This provides the set of interfaces that
define what a message is
and functionality to serialize message in various formats (e.g.,
wire,
JSON,
and
text).
proto: Package
proto provides functions operating on protobuf messages such as cloning,
merging, and checking equality, as well as binary serialization.
encoding/protojson:
Package protojson serializes protobuf messages as JSON.
encoding/prototext:
Package prototext serializes protobuf messages as the text format.
encoding/protowire:
Package protowire parses and formats the low-level raw wire encoding. Most
users should use package proto to serialize messages in the wire format.
reflect/protoreflect:
Package protoreflect provides interfaces to dynamically manipulate
protobuf messages.
reflect/protoregistry:
Package protoregistry provides data structures to register and lookup
protobuf descriptor types.
reflect/protodesc:
Package protodesc provides functionality for converting
descriptorpb.FileDescriptorProto messages to/from the reflective
protoreflect.FileDescriptor.
reflect/protopath:
Package protopath provides a representation of a sequence of
protobuf reflection operations on a message.
reflect/protorange:
Package protorange provides functionality to traverse a protobuf message.
testing/protocmp:
Package protocmp provides protobuf specific options for the cmp package.
testing/protopack:
Package protopack aids manual encoding and decoding of the wire format.
testing/prototest:
Package prototest exercises the protobuf reflection implementation for
concrete message types.
types/dynamicpb:
Package dynamicpb creates protobuf messages at runtime from protobuf
descriptors.
types/known/anypb:
Package anypb is the generated package for google/protobuf/any.proto.
types/known/timestamppb:
Package timestamppb is the generated package for
google/protobuf/timestamp.proto.
types/known/durationpb:
Package durationpb is the generated package for
google/protobuf/duration.proto.
types/known/wrapperspb:
Package wrapperspb is the generated package for
google/protobuf/wrappers.proto.
types/known/structpb:
Package structpb is the generated package for
google/protobuf/struct.proto.
types/known/fieldmaskpb:
Package fieldmaskpb is the generated package for
google/protobuf/field_mask.proto.
types/known/apipb:
Package apipb is the generated package for
google/protobuf/api.proto.
types/known/typepb:
Package typepb is the generated package for
google/protobuf/type.proto.
types/known/sourcecontextpb:
Package sourcecontextpb is the generated package for
google/protobuf/source_context.proto.
types/known/emptypb:
Package emptypb is the generated package for
google/protobuf/empty.proto.
types/descriptorpb:
Package descriptorpb is the generated package for
google/protobuf/descriptor.proto.
types/pluginpb:
Package pluginpb is the generated package for
google/protobuf/compiler/plugin.proto.
compiler/protogen:
Package protogen provides support for writing protoc plugins.
cmd/protoc-gen-go:
The protoc-gen-go binary is a protoc plugin to generate a Go protocol
buffer package.
Reporting issues
The issue tracker for this project is currently located at
golang/protobuf.
Please report any issues there with a sufficient description of the bug or
feature request. Bug reports should ideally be accompanied by a minimal
reproduction of the issue. Irreproducible bugs are difficult to diagnose and fix
(and likely to be closed after some period of time). Bug reports must specify
the version of the
Go protocol buffer module
and also the version of the
protocol buffer toolchain
being used.
Contributing
This project is open-source and accepts contributions. See the
contribution guide
for more information.
Compatibility
This module and the generated code are expected to be stable over time. However,
we reserve the right to make breaking changes without notice for the following
reasons:
Security: A security issue in the specification or implementation may
come to light whose resolution requires breaking compatibility. We reserve
the right to address such issues.
Unspecified behavior: There are some aspects of the protocol buffer
specification that are undefined. Programs that depend on unspecified
behavior may break in future releases.
Specification changes: It may become necessary to address an
inconsistency, incompleteness, or change in the protocol buffer
specification, which may affect the behavior of existing programs. We
reserve the right to address such changes.
Bugs: If a package has a bug that violates correctness, a program
depending on the buggy behavior may break if the bug is fixed. We reserve
the right to fix such bugs.
Generated additions: We reserve the right to add new declarations to
generated Go packages of .proto files. This includes declared constants,
variables, functions, types, fields in structs, and methods on types. This
may break attempts at injecting additional code on top of what is generated
by protoc-gen-go. Such practice is not supported by this project.
Internal changes: We reserve the right to add, modify, and remove
internal code, which includes all unexported declarations, the
protoc-gen-go/internal_gengo
package, the
runtime/protoimpl
package, and all packages under
internal.
Any breaking changes outside of these will be announced 6 months in advance to
[email protected].
Users should use generated code produced by a version of
protoc-gen-go
that is identical to the runtime version provided by the
protobuf module. This
project promises that the runtime remains compatible with code produced by a
version of the generator that is no older than 1 year from the version of the
runtime used, according to the release dates of the minor version. Generated
code is expected to use a runtime version that is at least as new as the
generator used to produce it. Generated code contains references to
protoimpl.EnforceVersion
to statically ensure that the generated code and runtime do not drift
sufficiently far apart.
The first version predates the release of Go 1 by several years. It has a long
history as one of the first core pieces of infrastructure software ever written
in Go. As such, the Go protobuf project was one of many pioneers for determining
what the Go language should even look like and what would eventually be
considered good design patterns and “idiomatic” Go (by simultaneously being
both positive and negative examples of it).
Consider the changing signature of the proto.Unmarshal function as an example
of Go language and library evolution throughout the life of this project:
These changes demonstrate the difficulty of determining what the right API is
for any new technology. It takes time multiplied by many users to determine what
is best; even then, “best” is often still somewhere over the horizon.
The change on June 6th, 2012 added a degree of type-safety to Go protobufs by
declaring a new interface that all protobuf messages were required to implement:
This interface reduced the set of types that can be passed to proto.Unmarshal
from the universal set of all possible Go types to those with a special
ProtoMessage marker method. The intention of this change is to limit the
protobuf API to only operate on protobuf data types (i.e., protobuf messages).
For example, there is no sensible operation if a Go channel were passed to the
protobuf API as a channel cannot be serialized. The restricted interface would
prevent that.
This interface does not behaviorally describe what a protobuf message is, but
acts as a marker with an undocumented expectation that protobuf messages must be
a Go struct with a specific layout of fields with formatted tags. This
expectation is not statically enforced by the Go language, for it is an
implementation detail checked dynamically at runtime using Go reflection. Back
in 2012, the only types with this marker were those generated by
protoc-gen-go. Since protoc-gen-go would always generate messages with the
proper layout of fields, this was deemed an acceptable and dramatic improvement
over interface{}.
Over the next 10 years,
use of Go would skyrocket and use of
protobufs in Go would skyrocket as well. With increased popularity also came
more diverse usages and requirements for Go protobufs and an increased number of
custom proto.Message implementations that were not generated by
protoc-gen-go.
The increasingly diverse ecosystem of Go types implementing the proto.Message
interface led to incompatibilities, which often occurred when:
Passing custom proto.Message types to the protobuf APIs: A concrete
message implementation might work with some top-level functions (e.g.,
proto.Marshal), but cause others (e.g., proto.Equal) to choke and panic.
This occurs because the type only had partial support for being an actual
message by only implementing the proto.Marshaler interface or having
malformed struct field tags that happened to work with one function, but not
another.
Using Go reflection on any proto.Message types: A common desire is to
write general-purpose code that operates on any protobuf message. For
example, a microservice might want to populate a trace_id field if it is
present in a message. To accomplish this, one would use Go reflection to
introspect the message type, and assume it were a pointer to a Go struct
with a field named TraceId (as would be commonly produced by
protoc-gen-go). If the concrete message type did not match this
expectation, it either failed to work or even resulted in a panic. Such was
the case for concrete message types that might be backed by a Go map instead
of a Go struct.
Both of these issues are solved by following the idiom that interfaces should
describe behavior, not data. This means that the interface itself should
provide sufficient functionality through its methods that users can introspect
and interact with all aspects of a protobuf message through a principled API.
This feature is called protobuf reflection. Just as how Go reflection provides
an API for programmatically interacting with any arbitrary Go value, protobuf
reflection provides an API for programmatically interacting with any arbitrary
protobuf message.
Since an interface cannot be extended in a backwards compatible way, this
suggested the need for a new major version that defines a new proto.Message
interface:
The new
proto.Message
interface contains a single ProtoReflect method that returns a
protoreflect.Message,
which is a reflective view over a protobuf message. In addition to making a
breaking change to the proto.Message interface, we took this opportunity to
cleanup the supporting functionality that operate on a proto.Message, split up
complicated functionality apart into manageable packages, and to hide
implementation details away from the public API.
The goal for this major revision is to improve upon all the benefits of, while
addressing all the shortcomings of the old API. We hope that it will serve the
Go ecosystem well for the next 10 years and beyond.
请发表评论