Extensible and Dynamic Topic Types for DDS Avatar
  1. OMG Specification

Extensible and Dynamic Topic Types for DDS — Open Issues

  • Acronym: DDS-XTypes
  • Issues Count: 13
  • Description: Issues not resolved
Open Closed All
Issues not resolved

Issues Descriptions

Provide a more efficient serialization for short strings and sequences

  • Status: open  
  • Source: Real-Time Innovations ( Gerardo Pardo-Castellote)
  • Summary:

    XCDR serialization rules for strings and sequences serialize the length as an uint32 ahead of serializing each of the elements.

    For short strings and sequences this can be a significant overhead. For example a 4 element sequence of octets would use 4 bytes for the sequence length and 4 bytes for the sequence content. That is 100% overhead. Worse since the serialized uint32 length must be serialized at a 4-byte offset, in some cases 3 additional padding bytes would be required. This would result on 7 bytes of overhead for 4 bytes of payload, or 175% overhead!!

    It would be better is short sequences, that is those whose maximum length is less than 255 could be serialized using an uint8 length instead of a uint32. With this the overhead of the previous example would be 1 byte (or 25%).

    An additional optimization would be to just serialize the string characters without the terminating NUL since this information is redundant with the length.

    This encoding would affect type compatibility in that a "short" sequence would not be compatible with a longer one. This is different from the current situation where string/sequence length would not affect compatibility. With the new encoding short strings would be compatible with each (regardless of their length) and likewise long strings would also be compatible with each other (regardless of length) as well as compatible with unbounded ones.

    So therefore we would need an annotation to indicate we want to serialize it as a short string... Perhaps we could reuse @bit_bound. For example:

    struct ShapeType {
        @bit_bound(8) string<32> color;
        long x;
        long y;
    };
    

    Disadvantage is that the only valid values when applied to strings/sequences would be @bit_bound(8) and @bit_bound(32).

    Alternatively we could also a new annotation like @compact or @pack.

  • Reported: DDS-XTypes 1.2 — Wed, 17 Jan 2018 11:29 GMT
  • Updated: Wed, 17 Jan 2018 11:29 GMT

Be explicit about which Types can be used for DDS Topics

  • Status: open  
  • Source: Real-Time Innovations ( Gerardo Pardo-Castellote)
  • Summary:

    The specification is not explicit about which Types may be used for a DDS Topic.

    By convention most DDS implementations have supported structures. However it is not clear if unions are allowed. And even less so if collections, strings, or primitives can be used.

    Annex E: Built-in Types defines as builtin the types: String, KeyedString, Bytes, and KeyedBytes.

    The DDS specification also does not say anything explicitly. However the fact that it talks about the type-specific code for a type "Foo" namely "FooSupport" "FooDatWriter", etc. may be taken to imply that this must be constructed types that are given a name. e.g. "Foo" by the user. And not primitive types.

    It would be better if the XTYPES spec was explicit about this.

  • Reported: DDS-XTypes 1.2 — Tue, 9 Jan 2018 12:58 GMT
  • Updated: Tue, 9 Jan 2018 21:26 GMT

Explicitly define the values for ReliabilityQosPolicyKind

  • Key: DDSXTY13-9
  • Status: open  
  • Source: Real-Time Innovations ( Gerardo Pardo-Castellote)
  • Summary:

    The RTPS Spec defines BEST_EFFORT and RELIABLE_RELIABILITY as:
    #define BEST_EFFORT 1
    #define RELIABLE 2 (as of RTPS2.4, previously it was defined as 3)

    The XTypes Spec should align itself with these values by defining ReliabilityQosPolicyKind as follows in Annex D - DDS Built-in Topic Data Types:

    enum ReliabilityQosPolicyKind {
        BEST_EFFORT_RELIABILITY_QOS = 1,
        RELIABLE_RELIABILITY_QOS = 2
    };
    
  • Reported: DDS-XTypes 1.2 — Fri, 8 Dec 2017 17:55 GMT
  • Updated: Thu, 4 Jan 2018 17:16 GMT

The definition of ReliabilityQosPolicyKind in Annex D is inconsistent with RTPS

  • Key: DDSXTY13-8
  • Status: open  
  • Source: Real-Time Innovations ( Gerardo Pardo-Castellote)
  • Summary:

    In the RTPS spec the values of ReliabilityQosPolicyKind are defined to be:
    BEST_EFFORT_RELIABILITY_QOS = 1
    BEST_EFFORT_RELIABILITY_QOS = 2

    However, in appendix D the ReliabilityQosPolicyKind is defined as:

    enum ReliabilityQosPolicyKind {
        BEST_EFFORT_RELIABILITY_QOS,
        RELIABLE_RELIABILITY_QOS
    };
    

    To match what the RTPS specifies this should be modified to:

    enum ReliabilityQosPolicyKind {
        @value(1)  BEST_EFFORT_RELIABILITY_QOS,
        @value(2)  RELIABLE_RELIABILITY_QOS
    };
    
  • Reported: DDS-XTypes 1.2 — Tue, 5 Dec 2017 18:07 GMT
  • Updated: Thu, 4 Jan 2018 17:12 GMT

Restrictions on MAP key element type should include ENUM

  • Key: DDSXTY13-6
  • Status: open  
  • Source: Airbus Group ( Oliver Kellogg)
  • Summary:

    DDS-XTypes 1.2 section 7.2.2.4.3 (Collection Types) on page 30 bottom states:

    > * Element type: The concrete type to which all elements conform. (Collection Elements
    > that are of a subtype of the element type rather than the element type itself may be
    > truncated when they are serialized into a Data Representation.)
    > In the case of a map type, this attribute corresponds to the type of the value elements.
    > Map types have an additional attribute, the key element type, that indicates the type of the
    > may key objects.

    I believe there is a typo in "type of the may key objects" - drop the word "may" ?

    The paragraph continues:

    > Implementers of this specification need only support key elements of
    > signed and unsigned integer types and of narrow and wide string types; the behavior of
    > maps with other key element types is undefined and may not be portable.

    Is there a particular reason why enums were excluded from the supported key types?

  • Reported: DDS-XTypes 1.2b1 — Fri, 15 Sep 2017 13:51 GMT
  • Updated: Thu, 4 Jan 2018 17:12 GMT

Union default values (Table 9)

  • Status: open  
  • Source: OCI ( Adam Mitz)
  • Summary:

    For type kind UNION_TYPE, the default value may require the discriminator to be set to the "lowest value associated with any member", however the "lowest" relation isn't defined for all possible discriminator types.
    Boolean: use false (based on Table 9)
    Byte: undefined (based on Table 3)
    Char<N>: undefined
    UInt<N>: well defined
    Int<N>: need to define/clarify for negative numbers
    Enum: treat as Int32 per Table 4

  • Reported: DDS-XTypes 1.2b1 — Thu, 14 Dec 2017 20:17 GMT
  • Updated: Thu, 4 Jan 2018 16:42 GMT

Further corrections

  • Status: open  
  • Source: Airbus Group ( Oliver Kellogg)
  • Summary:

    7.2.1.1 (Type Evolution Example) pg 13 first para

    > [ ...] However, not all components that publish or subscribe data of this
    > type will be upgraded to this new definition of VehicleLocationType
    > (or if they will not be upgraded, they will not be upgraded at the same
    > time)

    • (or if they will be upgraded, they will not be upgraded at the same time)

    7.2.1.3 (Sparse Types Example) pg 14 first para

    > [...] In its local programming language (say, C++ or Java), the
    > application can assign a pointer to null to omit a value for these fields.

    • the application can assign null to a pointer to omit [...]

    7.2.2.2 (Primitive Types) pg 18 Table 3 description for INT_16_TYPE

    > Signed integer minimally capable of representing values in the range
    > -32738 to +32737.

    • -32768 to 32767.

    7.2.2.2.1.2.1 (Use of Unicode) pg 20 fourth para

    > [...] The UTF-8 representation ISO-8859-1 characters that are not in the
    > ASCII subset uses two 8-bit code units.

    • The UTF-8 representation of ISO-8859-1 characters [...]

    7.2.2.2.1.2.5 (CHAR_16_TYPE) pg 21 Rationale second para

    > [...] Restricting to the BMP ensures that each coodepoint is [...]

    • codepoint

    7.2.2.7 (Try Construct behavior) pg 42 first para

    > [...] Similar to the structure examples there exist be objects of type
    > SEQ1024 [...]

    • [...] there may exist objects of type SEQ1024 [...]

    7.2.4.1 (Constructing objects of one type from objects of another type)
    pg 46 Table 13 column "Type compatibility" bottom

    > All objects of type T1 can either be used to
    > construct an object of type T1 or reliably de-
    > tected that that cannot initialize T1. .

    Please check - my guess is:

    • [...] construct an object of type T2 or it can be
      reliably detected which objects of type T1 cannot initialize T2.

    7.2.4.4.5.1 (Example: Strings) pg 49 first para

    > [...] If a
    > consumer of strings of narrow characters were to attempt to consume
    > that string, it might read
    > consider the first byte of the first character [...]

    • Omit the word "read" :
      [...] were to attempt to consume that string, it might consider [...]

    Table 19 (Definition of the is-assignable-from relationship for aggregated types)
    pg 53 second column fourth (i.e. last) bullet

    > If T1 (and therefore T2)
    > extensibility is final then the set of
    > labels are identical.

    • [...] then the set of labels is identical.

    7.3.1.1.1 (Backward Compatibility with Respect to Type Definitions)
    pg 60 bullet "Group of Annotations" fourth sub-bullet

    > * Code Generation (sub clause 8.3.5 [IDL41]) his specification retains
    > well-
    > established IDL type definition syntax, such as enumeration, structure,
    > union, and
    > sequence definitions.

    Appears to be a formatting problem, I think the last part of that phrase should be outside and after the end of the bullet enumeration:

    • Code Generation (sub clause 8.3.5 [IDL41])

    This specification retains well-established IDL type definition syntax, such as enumeration, [...]

    7.3.1.2.2 (Using Built-in Annotations) pg 70 first sentence

    > The application of the annotations listed above is restricted to the
    > elements of specified in Table 21.

    • [...] is restricted to the elements of IDL specified in Table 21.

    7.3.1.10 (Structure Types) pg 76 first sentence

    > Structures as described in this specification are in this specification are
    > fully compatible with the IDL constructs of the same name.

    • Structures as described in this specification are fully compatible [...]

    7.3.1.11 (Union Types) pg 76 first sentence

    > Unions as described in this specification are in this specification are
    > fully compatible with the IDL constructs of the same name.

    • Unions as described in this specification are fully compatible [...]

    7.3.2.4.3 (Map Types) pg 80 first bullet

    > * The map’s bound, if any, shall be indicated by the mapMaxLength
    > attribute. This attribute is required for all map types.

    For more fluent reading, I suggest:

    • The map's bound shall be indicated by the mapMaxLength attribute.
      This attribute is required for all map types.
      In case of an unbounded map, mapMaxLength shall be set to "-1".

    7.3.2.4.3 (Map Types) pg 81 non-normative example:

    > <struct name="MyStructure">
    > <member name="my_unbounded_maps_of_integers_to_floats"
    > type="int32"
    > mapKeyType="float32"
    > mapMaxLength="-1"/>

    The values for "type" and "mapKeyType" are swapped, they should be:

    • type="float32"
    • mapKeyType="int32"

    7.5 (Language Binding) pg 133 top of page (2nd sentence)

    > The main characteristics and motivation for each of these binding are
    > described in Table 41.

    • [...] for each of these bindings are described in Table 41.
  • Reported: DDS-XTypes 1.2b1 — Sun, 3 Dec 2017 21:11 GMT
  • Updated: Mon, 11 Dec 2017 12:01 GMT

DynamicType / DynamicTypeBuilder multiplicity of members

  • Status: open  
  • Source: Airbus Group ( Oliver Kellogg)
  • Summary:

    In the DDS-XTypes 1.2beta1 document, the UML class diagram in section 7.5.2.8 Figure 30 on p. 175 does not show the multiplicity relationship between DynamicType and DynamicTypeMember for the case of aggregated types.
    In other words, the fact that a DynamicType struct/union/valuetype may have more than one member is not visible in the diagram.

    The section 7.5.2.9 DynamicTypeBuilder does define an operation add_member which can be used to add members to aggregated types. However, DynamicTypeBuilder also does not provide a method for iterating over the members thus added.

    If it is intended that the management of and iteration over the sequence of DynamicTypeMembers shall lie outside of DynamicType / DynamicTypeBuilder, I suggest adding an explanation to that end.

  • Reported: DDS-XTypes 1.2b1 — Sun, 3 Dec 2017 19:47 GMT
  • Updated: Mon, 11 Dec 2017 12:01 GMT

Union discriminator value without associated member

  • Status: open  
  • Source: Airbus Group ( Oliver Kellogg)
  • Summary:

    The DDS-XTypes 1.2-beta1 document section 7.2.2.4.4.2 (Union Types) on page 34 first paragraph states:

    > [...]
    > (Note that it is not required for every potential discriminator value to be
    > associated with a member.)

    Supporting this, 7.4.3.5.1 (Notation used for the match criteria) on p.121 states:

    > O : UNION_TYPE An object "O" of a Union type as defined in 7.2.2.4.4.2
    > [...]
    > * O.selected_member is the member of the union selected
    > based on the value of the discriminator. Note that certain
    > discriminator values may select no member.

    However, 7.4.1.2.3 (Omission and Reordering of Members of Aggregated Types) on page 110 third para states:

    > Because union members are identified based on a discriminator value,
    > the value of the discriminator member must be serialized before the
    > value of the current non-discriminator member.
    > Neither member value may be omitted.

    I suggest replacing "Neither member value may be omitted" by:

    " The discriminator value may not be omitted.
    If the discriminator value has an associated member then the member value may not be omitted. "

    Section 7.5.2.11.6 (Operation: get_item_count) on page 185 states:

    > The "item count" of the data depends on the type of the object.
    > * If the object is of a union type, return the number of members in the
    > object. This value will always be two: the discriminator and the current
    > member corresponding to it.

    I suggest replacing the last sentence by:

    " This value will be two if the discriminator value has a corresponding member. It will be one if the discriminator value has no corresponding member. "

  • Reported: DDS-XTypes 1.2b1 — Wed, 29 Nov 2017 16:12 GMT
  • Updated: Mon, 11 Dec 2017 12:01 GMT

Add support for signed and unsigned 8-bit integers

  • Key: DDSXTY13-7
  • Status: open  
  • Source: Real-Time Innovations ( Gerardo Pardo-Castellote)
  • Summary:

    The DDS Type System does not provide a way to represent signed and unsigned 8-bit integers. IDL 4.2 has introduced the int8 and uint8 types for this purpose, but DDS-XTYPES is still missing those in the model.

    Support for 8-bit signed and unsigned integers is required by some RFP submissions, such as the OPC UA/DDS Gateway RFP.

  • Reported: DDS-XTypes 1.2 — Thu, 30 Nov 2017 14:42 GMT
  • Updated: Thu, 30 Nov 2017 14:42 GMT

Typographical corrections

  • Key: DDSXTY13-5
  • Status: open  
  • Source: Airbus Group ( Oliver Kellogg)
  • Summary:

    > 2.2.2 Basic Network Inteoperability Profile

    • "Interoperability"

    > 7.2.3 Type Extensibility and Mutability
    > [...]
    > • A type may be APENDABLE,

    • "APPENDABLE"

    > [...]
    > It is summarized more enerally in Table 12.

    • "generally"

    > 7.2.4.4.1 Assignability of Equivalent Types
    > If two types T1 and T2 are equivalent according to the MINIMAL relation (see Section 7.3.4.7),
    > then thery are mutually assignable,

    • "they"

    > Table 15 – Definition of the is-assignable-from relationship for primitive types
    > [...]
    > UINT64_TYPE
    > BITMASK_TYPE if and only if
    > T2.bound is between 33vand
    > 64, inclusive.

    • "between 33 and 64"

    > Table 20 – Alternative Type Representations
    > [...]
    > XSD [... 2nd column]
    > [...] No direct support for many of the
    > contructs (e.g keys) or the types in
    > the type model

    • "constructs"

    > 7.3.4.6.2 Hash TypeIdentifiers
    > Some TypeIdentifiers include within (directly or indirectly) hashes of one of mre
    > TypeObjects.
    Phrase is unclear, please reword.

    > 7.3.4.6.4 Indirect Hash TypeIdentifiers
    > These are the HASH [...] using a hash
    > TypeIdentifiers . They are distinghished byt:

    • "They are distinguished by:"

    > 7.4.1.1.1 Primitive types
    > [...]
    > • An endianness byte swap shall be performed in case the native system endianness is dif-
    > ferent from the one currently configured in the XCDR stream (XCDR.cendien).

    • "XCDR.cendian"
  • Reported: DDS-XTypes 1.2b1 — Sun, 10 Sep 2017 19:05 GMT
  • Updated: Tue, 12 Sep 2017 19:04 GMT

XSD for XML type representation should not specify default values for attributes representing annotations

  • Key: DDSXTY13-4
  • Status: open  
  • Source: Real-Time Innovations ( Gerardo Pardo-Castellote)
  • Summary:

    The XML type representation (Section 7.3.3) has an associated XSD that defines the legal XML documents that define DDS Types.

    In the XML type representation builtin annotations on members appear as "attributes" on the element that describes the member. For example see example in section 7.3.2.5.1.2 (Members):

    <struct name="structMemberDecl">
       <member name="my_key_field" type="int32" key="true" optional="false"/>
    </struct>
    

    These builtin annotations (key, optional), have default values when they are not present. This is defined in the IDL4+ specification. For example when the annotation "key" is not present, the (default) value is "false" when annotation "optional" is not present, the (default) value is "false". There is also a "default" value when the annotation is present with no parameters, but this is not allowed in the XSD.

    According to the XSD syntax, e.g. see https://www.w3schools.com/xml/schema_simple_attributes.asp

    The "default" value specified for an attribute is the value interpreted when the attribute is not present. Therefore we have two options:

    • Have the XSD specify default values for these attributes to match the "IDL4+ defaults when the attribute is not present
    • Have the XSD specify no default value.

    Currently this is not done correctly for some annotations. For example the XSD for structure members has wrong defaults for all the attributes:

      <xs:complexType name="structMemberDecl">
        <xs:complexContent>
          <xs:extension base="memberDecl">
           <xs:attribute name="id"
                         type="memberId"
                         use="optional"/>
    
            <xs:attribute name="optional"
                          type="xs:boolean"
                          use="optional"
                          default="true"/>
            <xs:attribute name="mustUnderstand"
                          type="xs:boolean"
                          use="optional"
                          default="true"/>
            <xs:attribute name="nonSerialized"
                          type="xs:boolean"
                          use="optional"
                          default="true"/>
            <xs:attribute name="key"
                          type="xs:boolean"
                          use="optional"
                          default="true"/>
          </xs:extension>
        </xs:complexContent>
      </xs:complexType>
    

    However it seems like the best solution is to remove the specification of a default value from the XSD. The problem is that when the default is specified the XSD parsers will fill the default value even if the attribute is not specified and it becomes impossible for the application that uses the parser to know if the attribute was there or not in the first place. This would make it impossible to transform between IDL and XML and back to IDL and get the same result back because all annotations would appear present to the XML parser even if they were not entered buy the user.

    Therefore we recommend removing the specification of 'default" value for all XML attributes that correspond to builtin annotations. This should be done both in Annex A and the machine readable dds-xtypes_type_definition_nonamespace.xsd

  • Reported: DDS-XTypes 1.2 — Mon, 24 Apr 2017 16:16 GMT
  • Updated: Mon, 24 Apr 2017 16:16 GMT

Algorithm to compute autoid is missing from the specification

  • Key: DDSXTY13-2
  • Status: open  
  • Source: Real-Time Innovations ( Gerardo Pardo-Castellote)
  • Summary:

    The specification does not define how memberIDs are computed in the case where the @autoid (or @hashid) annotations are used. This is needed for interoperability.

    Section 7.3.1.2.1.1 (Member IDs) specifies there are thee ways to set them:

    • Automatically following a progression that starts from the most-recently specified member ID.
    • Using the @id annotation
    • As a "hash" on the member name when @autoid(HASH) is specified
    • as a "hash" on a string-parameter when @hashid("string-parameter") is specified

    The use of @autoid refers to sub clause 8.3.1.2 in [IDL41]). But there also there is no specification of how the hash should be computed. Only that a "hashing" algorithm should be used.

    IDL working group discussed this and the preference was to leave it unspecified in IDL4 and instead put it in XTYPES or whichever specification depends on these IDs.

    A proposal would be to use an MD5 (as this is already used for key hashing).
    This needs to be done in a platform-independent manner, for example hashing a serialized representation of the string using a pre-specified endianness. Also per section 7.2.2.4.4.3 (Member IDs) the value/range must be representable in 28 bits.

  • Reported: DDS-XTypes 1.2 — Fri, 7 Apr 2017 00:44 GMT
  • Updated: Fri, 7 Apr 2017 00:44 GMT