Value to Bytes Encoder

This document outlines the creation of a JSON schema for constructing custom byte arrays (packets or payloads). A schema dictates the sequential assembly of bytes. The system parses the schema sequentially to construct the final byte array.

Schema Fundamentals

The schema is a JSON array containing specific “fields”. Three field types are supported for payload construction:

Static: Fixed, hardcoded bytes.
Dynamic: Live input data.
Metadata Length: Automated byte calculation for referenced fields.

General structure:


[
  { "type": "...", ... },
  { "type": "...", ... }
]

1. Static Fields

Used for fixed byte sequences (e.g., headers, footers, constant values).

A static field carries its value in a single data string, which is interpreted according to data_type. This is a change from earlier schemas, where data was an array of individual bytes — it is now always one string.

id: Unique identifier.
data: A single string holding the value, interpreted according to data_type.
data_type: How to interpret and encode data. Supported values:
- Numeric: "U8", "I8", "U16", "I16", "U32", "I32", "U64", "I64", "F32", "F64"
- Non-numeric: "STRING", "BOOL", "BYTES"
endian: Byte order, "be" (big-endian) or "le" (little-endian). Only meaningful for Numeric and STRING types types. It is not used for BOOL or BYTES — a single byte and an already-ordered byte list have no byte order. If omitted, it defaults to little-endian.

How data is interpreted per data_type:

Numeric (U8–F64): A decimal or 0x-hex literal, e.g. "2000" or "0x7D0". It is encoded to the type’s width using endian.
STRING: UTF-8 text, e.g. "Hi". With "endian": "be" the byte sequence is reversed.
BOOL: "true", "false", "1", or "0" → a single byte.
BYTES: A list of individual bytes. Tokens are separated by commas, whitespace, or square brackets, and each token is a 0x-hex or decimal value in the range 0–255. For example "0x09,0x03,0x00,0xA8", "[0x09, 0x03, 0x00, 0xA8]", and "9,3,0,168" all describe the same four bytes. Bytes are emitted in the order written.

Examples:


{
    "type": "static",
    "id": "header",
    "data": "0xAA,0xBB",
    "data_type": "BYTES"
}


{
    "type": "static",
    "id": "reg",
    "data": "2000",
    "data_type": "U16",
    "endian": "le"
}

The first field outputs the two header bytes AA BB; the second encodes 2000 as a little-endian U16 (D0 07).

2. Dynamic Fields

Used for integrating live data streams. Requires defining the data size and byte order.

Example:


{
    "type": "dynamic",
    "id": "sensor_payload",
    "key": "temperature_input",
    "data_type": "I32",
    "endian": "be"
}

id: Unique identifier.
key: Target data channel name.
data_type: Target format configuration. Supported values:
- Numeric: "U8", "I8", "U16", "I16", "U32", "I32", "U64", "I64", "F32", "F64"
- Non-numeric: "STRING", "BYTES", "BOOL"
endian: Byte order for multi-byte numbers.
- "be": Big-Endian (most significant byte first).
- "le": Little-Endian (least significant byte first).

3. Metadata Length Fields

Calculates a value from a set of referenced fields and encodes it as bytes. By default it measures the total byte size of those fields, which is required for dynamic payload sizing. It can also be switched to measure the number of referenced fields instead (see the count option below).

Example:


{
    "type": "metadata_length",
    "id": "payload_size",
    "ids": ["sensor_payload", "extra_data"],
    "data_type": "U8",
    "endian": "be"
}

id: Unique identifier.
ids: Array of field IDs to measure. By default, the total byte size of all referenced fields is summed.
data_type: Output format for the calculated value. It is limited to unsigned integers — "U8", "U16", "U32", or "U64"; signed integers and floats are rejected at validation. Choose a type large enough to hold the value — if the calculated length or count exceeds the type’s maximum, the field cannot be encoded and the message is dropped rather than truncated.
endian: Byte order ("be" or "le").
count (optional): When set to true, the field outputs the number of referenced fields (how many IDs are listed in ids) instead of their total byte size. Defaults to false. Because it is optional, existing schemas that omit it keep behaving exactly as before.

The `count` Option

Some protocols need an element count rather than a byte count — for example, “how many device addresses are in this request” instead of “how many bytes long is this request”. Setting "count": true makes the field emit that element count, so you never have to update the number by hand every time you add or remove a referenced field.

"count": false (or omitted) → outputs the total byte length of the referenced fields.
"count": true → outputs the number of referenced fields.

Example (counting referenced fields):


{
    "type": "metadata_length",
    "id": "device_count",
    "ids": ["device_a", "device_b", "device_c"],
    "data_type": "U8",
    "endian": "le",
    "count": true
}

This outputs 03 (three referenced IDs), regardless of how many bytes each referenced field occupies.

Complete Example

Objective: Construct a packet with the following structure:

2-byte Header (0xAA 0xBB).
1-byte Length indicator.
Dynamic payload (I32 from speed_sensor channel).
1-byte Footer (0xFF).

JSON Schema:


[
    {
        "type": "static",
        "id": "header",
        "data": "0xAA,0xBB",
        "data_type": "BYTES"
    },
    {
        "type": "metadata_length",
        "id": "length_byte",
        "ids": ["live_speed"],
        "data_type": "U8",
        "endian": "be"
    },
    {
        "type": "dynamic",
        "id": "live_speed",
        "key": "speed_sensor",
        "data_type": "I32",
        "endian": "be"
    },
    {
        "type": "static",
        "id": "footer",
        "data": "0xFF",
        "data_type": "BYTES"
    }
]

Prefab Picture:

Execution Flow: Given an incoming speed_sensor integer reading of 1000:

Static header is evaluated: AA BB
Dynamic field converts 1000 to a 4-byte I32: 00 00 03 E8
Metadata Length evaluates the payload size (4 bytes) and outputs as U8: 04
Static footer is evaluated: FF

Final Byte Array Output: AA BB 04 00 00 03 E8 FF

Deriving Both a Length and a Count

A common real-world case (e.g., a Mitsubishi MC binary read request) needs both a byte-length header and an element count in the same packet, and both should update automatically as devices are added or removed.

Use one metadata_length field with "count": false for the byte length of the request body.
Use another metadata_length field with "count": true for the number of device entries.

To add a device, you then only add its static entry and append its id to the ids arrays of both metadata fields — no hex math and no manual recalculation of the length or count bytes.