ProtoBuf

Example Output

personinfo.proto

Overview

Protocol Buffers is Google’s language-neutral format for serializing structured data. gen-proto writes a proto3 schema from a LinkML schema. The output compiles with protoc.

To run:

gen-proto personinfo.yaml > personinfo.proto

How a schema maps to proto3

Every class becomes a message, including mixins and abstract classes, so that every type a field names is declared. proto3 has no inheritance, so each message lists the slots it inherits as well as its own. The package name comes from the schema name.

Field names are snake_case. A slot with a rank above 0 keeps it as its field number. The other slots are numbered from 1 in schema order, skipping the numbers taken by ranks and the reserved range 19000 to 19999. When no slot of a class has a rank, the identifier slot takes field number 1. A rank that proto3 cannot use as a field number, or that another slot of the same class already uses, is dropped with a warning, and the slot is numbered automatically. Fields are written in field-number order.

LinkML types map to proto3 scalars: integer to int32, boolean to bool, float to float, double to double, and every other type, including dates, URIs and decimals, to string. A multivalued slot becomes a repeated field. A slot with no single range, such as one declared with any_of, becomes a string field.

A slot whose range is a class carries the nested message when the slot is inlined. Otherwise it carries a reference: the identifier of the object. The field then has the identifier’s scalar type, usually string, and a // reference to comment names the class. A class without an identifier is always inlined.

Each enum becomes an enum whose first value is <NAME>_UNSPECIFIED = 0. The permissible values follow from 1, each prefixed with the enum name so that values of different enums never clash.

Descriptions become // comments. An identifier slot carries // identifier and a required slot // required, since proto3 has no required keyword.

Inheritance

In the personinfo schema, Person inherits id, name, description and depicted_by from NamedThing, and aliases from a mixin:

NamedThing:
  slots:
    - id
    - name
    - description
    - depicted_by

HasAliases:
  mixin: true
  attributes:
    aliases:
      multivalued: true

Person:
  is_a: NamedThing
  mixins:
    - HasAliases
  slots:
    - primary_email
    - birth_date
    - age_in_years
    - gender
    - current_address
    - telephone

The example leaves some slots out for brevity.

gen-proto writes the following messages. The identifier leads each one. Address stays a nested message because the slot is inlined. Person lists aliases, which it inherits from the mixin:

// A generic grouping for any identifiable entity
message NamedThing {
  // identifier
  // required
  string id = 1;
  // required
  string name = 2;
  string description = 3;
  string depicted_by = 4;
}

// A person (alive, dead, undead, or fictional).
message Person {
  // identifier
  // required
  string id = 1;
  // required
  string name = 2;
  string description = 3;
  string depicted_by = 4;
  string primary_email = 5;
  string birth_date = 6;
  int32 age = 7;
  GenderType gender = 8;
  // The address at which a person currently lives
  Address current_address = 9;
  string telephone = 10;
  repeated EmploymentEvent has_employment_history = 11;
  repeated FamilialRelationship has_familial_relationships = 12;
  repeated InterPersonalRelationship has_interpersonal_relationships = 13;
  repeated MedicalEvent has_medical_history = 14;
  repeated string aliases = 15;
  repeated NewsEvent has_news_events = 16;
}

Schemas that import other schemas

By default, gen-proto merges every imported schema into one file. With --no-mergeimports, it writes only the messages and enums of the schema it is given, and refers to the others through import statements. Run it once per schema, and name each file after the schema’s name, in one directory:

gen-proto --no-mergeimports base.yaml > protos/base.proto
gen-proto --no-mergeimports main.yaml > protos/main.proto
protoc --proto_path=protos --python_out=out protos/base.proto protos/main.proto

A file imports a schema when one of its fields, inherited fields included, names a message or enum of that schema. The field gives the type’s full name, such as .base.Thing. Types from linkml:types become scalars and need no file.

Configuration File

gen-proto accepts a --config-file/-C YAML file in the format that gen-project reads with its own --config-file (see Project Generator). gen-java (see Java) and gen-golang (see Go) read the same format, so one project config.yaml can serve them all. The settings for gen-proto live under generator_args.proto, and the other sections are ignored:

# config.yaml
generator_args:
  proto:
    importmap: importmap.json
    mergeimports: true
  java:
    package: org.example.model
gen-proto -C config.yaml personinfo.yaml > personinfo.proto

Any option of the command can be set there, keyed by its name with underscores in place of dashes, such as importmap for --importmap. A key that is not an option of gen-proto is reported as a warning and ignored. --log_level, --verbose and --stacktrace act as soon as the command line is read, so they are taken from the command line only.

An option given on the command line wins over the file, and the file wins over the option’s default. gen-proto reads the file only when it is named with --config-file.

Docs

Command Line

gen-proto

Generate proto representation of LinkML model

gen-proto [OPTIONS] YAMLFILE

Options

-V, --version

Show the version and exit.

-C, --config-file <config_file>

Path to a YAML config file supplying defaults under ‘generator_args: {proto: {importmap: …}}’. Keys are this command’s own option names with dashes as underscores; explicit command-line options always take precedence over the config file.

-f, --format <format>

Output format

Default:

'proto'

Options:

proto

--metadata, --no-metadata

Include metadata in output

Default:

True

--generation-date, --no-generation-date

Stamp output with the generation_date timestamp. Off by default so output is reproducible across runs.

Default:

False

--useuris, --metauris

Use class and slot URIs over model uris

Default:

True

-im, --importmap <importmap>

Import mapping file

--log_level <log_level>

Logging level

Default:

'WARNING'

Options:

CRITICAL | ERROR | WARNING | INFO | DEBUG

-v, --verbose

Verbosity. Takes precedence over –log_level.

--mergeimports, --no-mergeimports

Merge imports into source file (default=mergeimports)

--stacktrace, --no-stacktrace

Print a stack trace when an error occurs

Default:

False

Arguments

YAMLFILE

Required argument

Code

class linkml.generators.protogen.ProtoGenerator(schema: str | ~typing.TextIO | ~linkml_runtime.linkml_model.meta.SchemaDefinition | ~linkml.utils.generator.Generator | ~pathlib.Path, schemaview: ~linkml_runtime.utils.schemaview.SchemaView | None = None, format: str | None = None, metadata: bool = True, include_generation_date: bool = False, useuris: bool | None = None, log_level: int | None = 30, mergeimports: bool | None = True, source_file_date: str | None = None, source_file_size: int | None = None, logger: ~logging.Logger | None = None, verbose: bool | None = None, output: str | None = None, namespaces: ~linkml_runtime.utils.namespaces.Namespaces | None = <property object>, directory_output: bool = False, base_dir: str = None, metamodel_name_map: dict[str, str] = None, importmap: str | ~collections.abc.Mapping[str, str] | None = None, emit_prefixes: set[str] = <factory>, metamodel: ~linkml.utils.schemaloader.SchemaLoader = None, stacktrace: bool = False, include: str | ~pathlib.Path | ~linkml_runtime.linkml_model.meta.SchemaDefinition | None = None)[source]

A Generator for creating Protobuf schemas from a linkml schema.

serialize(**kwargs) → str | None

Generate output in the required format

Parameters:

kwargs – Generator specific parameters

Returns:

Generated output, or None if the generator writes its output directly to disk instead of returning it (e.g. ExcelGenerator, JavaGenerator)