ProtoBuf¶
Example Output¶
Overview¶
Protocol Buffers is Google’s language-neutral
format for serializing structured data. gen-proto writes a proto3
schema from a LinkML schema. The output compiles with protoc.
To run:
gen-proto personinfo.yaml > personinfo.proto
How a schema maps to proto3¶
Every class becomes a message, including mixins and abstract classes, so
that every type a field names is declared. proto3 has no inheritance, so each
message lists the slots it inherits as well as its own. The package name
comes from the schema name.
Field names are snake_case. A slot with a rank above 0 keeps it as its
field number. The other slots are numbered from 1 in schema order, skipping
the numbers taken by ranks and the reserved range 19000 to 19999. When no slot
of a class has a rank, the identifier slot takes field number 1. A rank that
proto3 cannot use as a field number, or that another slot of the same class
already uses, is dropped with a warning, and the slot is numbered
automatically. Fields are written in field-number order.
LinkML types map to proto3 scalars: integer to int32, boolean to
bool, float to float, double to double, and every other
type, including dates, URIs and decimals, to string. A multivalued slot
becomes a repeated field. A slot with no single range, such as one declared
with any_of, becomes a string field.
A slot whose range is a class carries the nested message when the slot is
inlined. Otherwise it carries a reference: the identifier of the object. The
field then has the identifier’s scalar type, usually string, and a
// reference to comment names the class. A class without an identifier is
always inlined.
Each enum becomes an enum whose first value is <NAME>_UNSPECIFIED = 0.
The permissible values follow from 1, each prefixed with the enum name so that
values of different enums never clash.
Descriptions become // comments. An identifier slot carries
// identifier and a required slot // required, since proto3 has no
required keyword.
Inheritance¶
In the personinfo schema, Person inherits id, name, description
and depicted_by from NamedThing, and aliases from a mixin:
NamedThing:
slots:
- id
- name
- description
- depicted_by
HasAliases:
mixin: true
attributes:
aliases:
multivalued: true
Person:
is_a: NamedThing
mixins:
- HasAliases
slots:
- primary_email
- birth_date
- age_in_years
- gender
- current_address
- telephone
The example leaves some slots out for brevity.
gen-proto writes the following messages. The identifier leads each one.
Address stays a nested message because the slot is inlined. Person
lists aliases, which it inherits from the mixin:
// A generic grouping for any identifiable entity
message NamedThing {
// identifier
// required
string id = 1;
// required
string name = 2;
string description = 3;
string depicted_by = 4;
}
// A person (alive, dead, undead, or fictional).
message Person {
// identifier
// required
string id = 1;
// required
string name = 2;
string description = 3;
string depicted_by = 4;
string primary_email = 5;
string birth_date = 6;
int32 age = 7;
GenderType gender = 8;
// The address at which a person currently lives
Address current_address = 9;
string telephone = 10;
repeated EmploymentEvent has_employment_history = 11;
repeated FamilialRelationship has_familial_relationships = 12;
repeated InterPersonalRelationship has_interpersonal_relationships = 13;
repeated MedicalEvent has_medical_history = 14;
repeated string aliases = 15;
repeated NewsEvent has_news_events = 16;
}
Schemas that import other schemas¶
By default, gen-proto merges every imported schema into one file. With
--no-mergeimports, it writes only the messages and enums of the schema it
is given, and refers to the others through import statements. Run it once
per schema, and name each file after the schema’s name, in one directory:
gen-proto --no-mergeimports base.yaml > protos/base.proto
gen-proto --no-mergeimports main.yaml > protos/main.proto
protoc --proto_path=protos --python_out=out protos/base.proto protos/main.proto
A file imports a schema when one of its fields, inherited fields included,
names a message or enum of that schema. The field gives the type’s full name,
such as .base.Thing. Types from linkml:types become scalars and need no
file.
Configuration File¶
gen-proto accepts a --config-file/-C YAML file in the format that
gen-project reads with its own --config-file (see
Project Generator). gen-java (see Java) and gen-golang
(see Go) read the same format, so one project config.yaml can
serve them all. The settings for gen-proto live under
generator_args.proto, and the other sections are ignored:
# config.yaml
generator_args:
proto:
importmap: importmap.json
mergeimports: true
java:
package: org.example.model
gen-proto -C config.yaml personinfo.yaml > personinfo.proto
Any option of the command can be set there, keyed by its name with underscores
in place of dashes, such as importmap for --importmap. A key that is
not an option of gen-proto is reported as a warning and ignored.
--log_level, --verbose and --stacktrace act as soon as the command
line is read, so they are taken from the command line only.
An option given on the command line wins over the file, and the file wins over
the option’s default. gen-proto reads the file only when it is named with
--config-file.
Docs¶
Command Line¶
gen-proto¶
Generate proto representation of LinkML model
gen-proto [OPTIONS] YAMLFILE
Options
- -V, --version¶
Show the version and exit.
- -C, --config-file <config_file>¶
Path to a YAML config file supplying defaults under ‘generator_args: {proto: {importmap: …}}’. Keys are this command’s own option names with dashes as underscores; explicit command-line options always take precedence over the config file.
- -f, --format <format>¶
Output format
- Default:
'proto'- Options:
proto
- --metadata, --no-metadata¶
Include metadata in output
- Default:
True
- --generation-date, --no-generation-date¶
Stamp output with the generation_date timestamp. Off by default so output is reproducible across runs.
- Default:
False
- --useuris, --metauris¶
Use class and slot URIs over model uris
- Default:
True
- -im, --importmap <importmap>¶
Import mapping file
- --log_level <log_level>¶
Logging level
- Default:
'WARNING'- Options:
CRITICAL | ERROR | WARNING | INFO | DEBUG
- -v, --verbose¶
Verbosity. Takes precedence over –log_level.
- --mergeimports, --no-mergeimports¶
Merge imports into source file (default=mergeimports)
- --stacktrace, --no-stacktrace¶
Print a stack trace when an error occurs
- Default:
False
Arguments
- YAMLFILE¶
Required argument
Code¶
- class linkml.generators.protogen.ProtoGenerator(schema: str | ~typing.TextIO | ~linkml_runtime.linkml_model.meta.SchemaDefinition | ~linkml.utils.generator.Generator | ~pathlib.Path, schemaview: ~linkml_runtime.utils.schemaview.SchemaView | None = None, format: str | None = None, metadata: bool = True, include_generation_date: bool = False, useuris: bool | None = None, log_level: int | None = 30, mergeimports: bool | None = True, source_file_date: str | None = None, source_file_size: int | None = None, logger: ~logging.Logger | None = None, verbose: bool | None = None, output: str | None = None, namespaces: ~linkml_runtime.utils.namespaces.Namespaces | None = <property object>, directory_output: bool = False, base_dir: str = None, metamodel_name_map: dict[str, str] = None, importmap: str | ~collections.abc.Mapping[str, str] | None = None, emit_prefixes: set[str] = <factory>, metamodel: ~linkml.utils.schemaloader.SchemaLoader = None, stacktrace: bool = False, include: str | ~pathlib.Path | ~linkml_runtime.linkml_model.meta.SchemaDefinition | None = None)[source]¶
A Generator for creating Protobuf schemas from a linkml schema.