Kafka Schema/Payload Javascript Object Notation

Implementation

The source code of the format-type is available here: https://github.com/VILLASframework/node/blob/master/lib/formats/json_kafka.cpp

Configuration

The json.kafka format-type supports the following settings:

delimiter	string
indent	number Default: 0 Pretty-print the result, using newlines between array and object items, and indenting with n spaces. The valid range for n is between 0 and 31 (inclusive), other values result in an undefined output. If the settings is not used or is 0, no newlines are inserted between array and object items.
compact	boolean Default: false This flag enables a compact representation, i.e. sets the separator between array and object items to "," and between object keys and values to ":". Without this flag, the corresponding separators are ", " and ": " for more readable output.
ensure_ascii	boolean Default: false If this flag is used, the output is guaranteed to consist only of ASCII characters. This is achieved by escaping all Unicode characters outside the ASCII range.
sort_keys	boolean Default: false If this flag is used, all the objects in output are sorted by key. This is useful e.g. if two JSON texts are diffed or visually compared.
escape_slash	boolean Default: false Escape the / characters in strings with \/.
real_precision	integer Default: 17 Output all real numbers with at most n digits of precision. The valid range for this setting is between 0 and 31 (inclusive), and other values result in an undefined behavior. By default, the precision is 17, to correctly and losslessly encode all IEEE 754 double precision floating point numbers.
ts_origin	boolean Default: true If set, include the origin timestamp in the output.
ts_received	boolean Default: true If set, include the received timestamp in the output.
sequence	boolean Default: true If set, include the sequence number in the output.
data	boolean Default: true If set, include the data in the output.
offset	boolean Default: true If set, include the offset between origin and received timestamp in the output.

{

• "delimiter": "string",
• "indent": 0,
• "compact": false,
• "ensure_ascii": false,
• "sort_keys": false,
• "escape_slash": false,
• "real_precision": 17,
• "ts_origin": true,
• "ts_received": true,
• "sequence": true,
• "data": true,
• "offset": true

}

Example Configuration

node/etc/examples/formats/json-kafka.conf

nodes = {
    node = {
        type = "file"
        uri = "/dev/null"

        format = {
            type = "json.kafka"

            indent = 2
            schema = {
                type = "struct"
                name = "villas-node.Value"
            }
        }
    }
}

Example Payload

{
  "schema": {
    "type": "struct",
    "name": "villas-node.Value",
    "fields": [
      {
        "type": "int64",
        "optional": false,
        "field": "timestamp"
      },
      {
        "type": "int64",
        "optional": false,
        "field": "sequence"
      },
      {
        "type": "double",
        "optional": false,
        "field": "signal0"
      }
    ]
  },
  "payload": {
    "timestamp": 1623834103174,
    "sequence": 7,
    "signal0": -0.95122443954003999
  }
}