C2MS11 — Larger Data Types

From Justin: In the current specification, RAW-VALUE is limited to a 32-bit integer and EU-VALUE is limited to 64-bit float (i.e., double). These data types should be changed to allow larger data types to be represented (at a minimum, 64-bit integer should be supported throughout). May need a separate field to indicate the type of data in these fields.

[NOTE: the reason for deferment is simply say making the RAW-VALUE a 64 bit and the EU-VALUE 128-bit float falls outside of hardware support and programming language support -- and adding a DATA-TYPE field to support the specification of any valid c2ms data-type in either field seems like a big change that should be considered more fully.]

The current spec supports a RAW-VALUE and EU-VALUE (converted/calibrated) fields. Currently the RAW one is a 32-bit signed integer and the EU one is a double (64-bit). To support a broader value type and range, this change adds a RAW-VALUE-TYPE and an EU-VALUE-TYPE to any message these fields are in. In some cases, if the message has both a RAW and EU value, these could in some cases be identical. Additional optional (special case) fields are also added RAW-VALUE-LENGTH and EU-VALUE-LENGTH, these two are discussed below.

The RAW-VALUE-TYPE and EU-VALUE-TYPE would be one of the C2MS data-types and be valid for all samples in the message. Hence, they are only specified once per message.

For example, a typical combination might be: RAW-VALUE-TYPE=I32, EU-VALUE-TYPE=F64 for numeric calibrated telemetered values.

The main reason in some cases to preserve both the RAW and the EU value is that in some cases there's a need to re-calibrate existing values and the raw can then be used to do that... [are there any others?]

[It's worth noting that RAW here may mean literally bit-by-bit what's in the packet or it could mean "host corrected" (byte swapped) but without calibration -- we should sort that out. In fact, C2MS does not say that I can tell ... I'm aware that some orgs support the notion of raw from the packet, host-converted (byte/bit corrected), and fully calibrated) in which case within discussion we may want to have 3 fields, which seems excessive. I think I prefer 'host-corrected' for the raw, as we can only do so much, and some folks might not have the truly raw bits available anyway...]

RAW-FIELD-TYPE and EU-TYPE-TYPE are of type String. The legal values are specified in table 8.1 column Field-type. Note that these should probably be mixed character legal in these fields.

Two OPTIONAL fields are available for type binary (blob) and variable. These types SHALL supply RAW-VALUE-LENGTH and EU-VALUE-LENGTH which are both I32 and specify the number of bytes for either field value. Negative values are illegal. From an implementation point of view in regard to these variable length fields, that messages beyond a few megabytes are likely not supported.

Add RAW-VALUE-TYPE and EU-VALUE-TYPE and two additional optional fields.

[NOTE: the reason for deferment is simply say making the RAW-VALUE a 64 bit and the EU-VALUE 128-bit float falls outside of hardware support and programming language support -- and adding a DATA-TYPE field to support the specification of any valid c2ms data-type in either field seems like a big change that should be considered more fully.]

The current spec supports a RAW-VALUE and EU-VALUE (converted/calibrated) fields. Currently the RAW one is a 32-bit signed integer and the EU one is a double (64-bit). To support a broader value type and range, this change adds a RAW-VALUE-TYPE and an EU-VALUE-TYPE to any message these fields are in. In some cases, if the message has both a RAW and EU value, these could in some cases be identical. Additional optional (special case) fields are also added RAW-VALUE-LENGTH and EU-VALUE-LENGTH, these two are discussed below.

The RAW-VALUE-TYPE and EU-VALUE-TYPE would be one of the C2MS data-types and be valid for all samples in the message. Hence, they are only specified once per message.

For example, a typical combination might be: RAW-VALUE-TYPE=I32, EU-VALUE-TYPE=F64 for numeric calibrated telemetered values.

The main reason in some cases to preserve both the RAW and the EU value is that in some cases there's a need to re-calibrate existing values and the raw can then be used to do that... [are there any others?]

[It's worth noting that RAW here may mean literally bit-by-bit what's in the packet or it could mean "host corrected" (byte swapped) but without calibration -- we should sort that out. In fact, C2MS does not say that I can tell ... I'm aware that some orgs support the notion of raw from the packet, host-converted (byte/bit corrected), and fully calibrated) in which case within discussion we may want to have 3 fields, which seems excessive. I think I prefer 'host-corrected' for the raw, as we can only do so much, and some folks might not have the truly raw bits available anyway...]

RAW-FIELD-TYPE and EU-TYPE-TYPE are of type String. The legal values are specified in table 8.1 column Field-type. Note that these should probably be mixed character legal in these fields.

Two OPTIONAL fields are available for type binary (blob) and variable. These types SHALL supply RAW-VALUE-LENGTH and EU-VALUE-LENGTH which are both I32 and specify the number of bytes for either field value. Negative values are illegal. From an implementation point of view in regard to these variable length fields, that messages beyond a few megabytes are likely not supported.