Welcome to Protoflex documentation!
const PB = require('protoflex')
console.log(PB.fromHex('082a').int32(1)) // 42
or via JSON API (not recommended, why?):
const PB = require('protoflex')
console.log(PB.fromHex('082a').toJSON()) // { '1': '42' }
const PB = require('protoflex')
let msg = PB
.create()
.int32(1, 42)
console.log(msg.serialize()) // [0x08, 0x2a]
or via JSON API:
const PB = require('protoflex')
let msg = PB.fromJson({
1: 42
})
console.log(msg.serialize()) // [0x08, 0x2a]
const PB = require('protoflex')
then you can do:
.create()
Returns a new empty InputMessage
, refer to Functional API
.parse(data: number[])
Parses array of bytes as a OutputMessage
, refer to Functional API
.fromHex(object)
Parses string as hexadecimal to OutputMessage
refer to Functional API
.fromJson(object)
Creates an InputMessage
from JSON, refer to JSON API
require('protoflex/utils')
See API reference for utils.d.ts
There are .<type>(key, ?index = 0)
methods for all (supported) Protobuf types,
where <type>
is a Protobuf type (e.g. int32
), which return value in the
given field interpreted as given type.
For convenience, there's also hex
type that returns bytes
as a hexadecimal string
You can use .get(key, type, ?index)
, which works exactly the same.
Also, there are some service methods:
.repeated(key, type)
Declare some field as repeated.
Protobuf version 3 declares a new, more efficient way to store repeated elements by storing them in a single length-delimited field. However, this increases the complexity of parsing them — it is not actually possible without knowing the type of items. Thus, to provide possibility of parsing those fields, this method is available.
This method internally converts the packed repeated to a normal repeated (of course, if it was packed in the first place):
msg.array(1, 'int32') // field is not varint
msg.repeated(1, 'int32').array(1, 'int32') // ok
.known()
In Protobuf, when a field has its default value (e.g. 0
for numbers,
empty strings, messages, etc), it may not be present in wire. Thus, to handle
these cases, you can either try/catch
calls, or use .known(...items)
.
It accepts varargs of:
number
will be interpreted as a known-for-sure untyped field keynumber[]
will be interpreted as a known-for-sure untyped field keys{ <key>: string }
will be interpreted as: <key>
field is of the given type{ <key>: string[] }
will be interpreted as: <key>
field is one of the given types.array(key, type)
Returns array of fields with key key
interpreted as type
.iter(key, type)
Returns iterator of items in key
field interpreted as type
, which is compatible
with both for..of
and .next()
flows.
.length(key)
Returns number of items with given key
.has(key, ?index, ?ignoreKnown)
Returns if message has field with key key
and index index
.
If ignoreKnown
is set, will return false
for fields that are declared in .known()
but
not present in actual message.
.type(key, ?index)
Returns type of has field with key key
and index index
.
.toJSON()
Converts message to JSON, refer to JSON API
.toInput()
Converts the message to InputMessage, to allow modifying it.
Note: You likely won't be able to use InputMessage.get
,
because types will be erased down to primitives. If you will need to get the
field values later, keep the original message at hand.
Note: When converted back to the wire, it may not be exactly the same
as the original message, even if no changes were made. Particularly, order of the
fields may not be preserved, empty fields may be removed
(use InputMessage.keepDefault
to avoid that), and non-packed
primitives may get packed. Still, the resulting message should stay
compatible with the original one.
If your message contains packed repeated fields, you have to declare
them using .repeated()
before using .toInput()
to make them available
in the input message.
Similarly, there are .<type>(key, value, ?index = 0)
methods for all
(supported) Protobuf types, where <type>
is a Protobuf type (e.g. int32
),
which return the very same message to support chaining. Example:
let msg = PB.create()
.int32(1, 42)
.int32(2, 44)
.toWire()
Similarly you can write nested messages:
let msg = PB.create()
.int32(1, 42)
.message(1, (msg) => {
msg
.string(1, 'such flex')
.string(2, 'much wow')
})
.int32(2, 44)
.toWire()
There are two ways of adding repeated items:
If you are adding items one-by-one, you can use .append(key, value, ?type)
method,
which works similarly to .push()
method of native arrays
(but can't handle multiple items at once)
let msg = PB.create()
.append(1, 42, 'int32')
.append(1, 43, 'int32')
.append(1, 44, 'int32')
.array(1, 'int32') // [42, 43, 44]
Type argument is optional, but is strongly recommended.
Alternatively, you can use index
parameter in .<type>
methods:
let msg = PB.create()
.int32(1, 42, 0)
.int32(1, 43, 1)
.int32(1, 44, 2)
.array(1, 'int32') // [42, 43, 44]
This way you can set items randomly
Note that even though it can be used as a sparse, when serializing, empty fields are ignored, and resulting message will not have same order.
Example:
[1, undefined, 1]
will be encoded as[1, 1]
, and thus re-decoded as[1, 1]
For removing items there's a .clear(key, ?index)
method. Negative index
es are supported
(-1 is last, -2 is pre-last etc.), and when it is missing, the whole array is cleared:
let msg = PB.create()
.int32(1, 42, 0)
.int32(1, 43, 1)
.int32(1, 44, 2)
.int32(1, 45, 3)
msg.array(1, 'int32') // [42, 43, 44, 45]
msg.clear(1, -1).array(1, 'int32') // [42, 43, 44]
msg.clear(1, 1).array(1, 'int32') // [42, 44]
msg.clear(1).array(1, 'int32') // []
Note that empty arrays are ignored when serializing to the wire because of how Protobuf works
Let's say you have this Protoflex object:
let msg = PB.create()
.append(1, 42, 'int32')
.append(1, 43, 'int32')
.append(1, 44, 'int32')
.array(1, 'int32') // [42, 43, 44]
You can access array items in two ways, first is index
parameter in .get()
method, and
second is .array(key, ?type)
method, which accepts a key
and optionally field type.
So, code for accessing N
th item will look like this:
let eq = msg.array(1, 'int32')[N] === msg.get(1, 'int32', N) // true
However, for random access you should use .get()
method, and fot iterating - .iter()
Continuing with same object from previous part, you may also want to iterate over all values.
This is also possible with .iter(key, ?editable, ?type)
method. You may wonder about
editable
parameter, but it is pretty straightforward. When it is true
, iterables
are { get() {...}, set() {...} }
item references, and when false
iterables are simply
field values. By default they are not editable.
It can be used both in for..of
and as a plain iterator (.next()
):
for (let it of msg.iter(1)) {
console.log(it) // 42, 43, 44 consequently
}
for (let it of msg.iter(1, true)) {
console.log(it.get()) // 42, 43, 44 consequently
it.set(it.get() + 1)
}
console.log(msg.array(1, 'int32')) // [43, 44, 45]
When serializing to the wire, by default fields with default values
(that is, 0
for numbers, empty messages, arrays, strings, etc.):
const msg = PB.fromJson({ 1: 0 }).toHex()
console.log(PB.fromHex(msg)) // {}
In some cases this is not desirable (e.g. when targeting some broken decoders), and you can force Protoflex to write default values to the wire as well:
const msg = PB.fromJson({ 1: 0 }).keepDefault().toHex()
console.log(PB.fromHex(msg)) // { 1: 0 }
Protoflex provides a convenient JSON API. Basically, it enables you to treat Protobuf messages just like plain Javascript objects. However, it implies some limitations: there are no type/presence checks.
As you have already seen in Quick Start, JSON API returns plain JS objects with numbers as keys (which are however treated as strings by JS engine) and values. All number values are represented as a string, and only primitive types are parsed. Also, parsing nested messages or repeated integers/fractions may be inaccurate because of how Protobuf handles them.
Another thing is using JSON API to convert from output message to input
(i.e. PB.fromJson(msg.toJSON())
). This should never be done!
Not only the points mentioned before are still valid, fixed32/64 types
will be written to the wire as varint, and will make the message incompatible
with the original one.
Instead, use msg.toInput()
, which tries to preserve as much information
as possible for a compatible serialization.
This direction of conversion is much more flexible than the opposite.
As input, JSON API takes a plain JS object (or JSON in string) like this:
let a = fromJson({
1: 'hi',
2: 42
})
Note that Protoflex ignores any non-numeric keys.
By default, Protobuf types are inferred from input type:
JS type | Inferred Protobuf type |
---|---|
number |
int32 if whole number, else float (note that .0 does not make number a fraction) |
string |
string |
boolean |
bool |
BigInt |
int64 |
Buffer , Uint8Array |
bytes |
Long |
int64 or uint64 , depending on .unsigned |
<plain object> |
message |
Surely, you may want to cast a value to some explicit type, so this syntax is supported:
let a = fromJson({
1: 'hi',
2: {
uint32: 42
}
})
Here, field 2
will have type uint32
instead of default int32
.
You may now wonder how Protoflex differs sub-messages and this syntax.
Actually, everything is quite straightforward: if Protoflex encounters
a key with a special meaning, it is treated how intended
(those keys are all supported protobuf types, value
and unpacked
, more about it below).
If it encounters protobuf type as a key, its value is treated as this type.
If it encounters value
as a key, its value is treated like it was put simply as
value in message (meaning the type will be inferred).
Otherwise, it is treated like a sub-message.
Modifiers like
unpacked
(more can be added without breaking compatibility) do not affect inferred type.
Putting
value
has higher priority that putting a type, and putting multiple types will result in undefined behaviour
Encoding arrays aka repeated
fields using JSON API is also very intuitive:
let a = fromJson({
1: 'hi',
2: [42, 43]
})
Here, field 2
will be repeated in resulting message twice with given values.
As you might know, some repeated fields in protobuf can
be packed (see here),
which could be unsupported by target deserializer. Thus, you can declare it using a
modifier unpacked
:
let a = fromJson({
1: 'hi',
2: {
value: [42, 43],
unpacked: true
}
})
A bit larger code, but it is needed rarely anyway.
protoflex
: index.d.ts#protoflexprotoflex/utils
: utils.d.tsGenerated using TypeDoc