Typo in the EBNF `OclExpessionCS` instead of `OclExpressionCS`

The concrete syntax as expressed in this chapter is full of errors

1) It is a pity that it is not using the same EBNF idiom as other OMG specifications that were written before (e.g. IDL)

2) It is a pity that the idiom selected is not consistantly applied (e.g. [...] or (...)? to describe an option - both are used sometimes even in the same rule)

3) there are many literals that are missing their enclosing <> (e.g. extend_decl instead of <extend_decl>)

4) there are some missing text separators (' here)

5) and some typos such as Expession instead of Expression

6) many (11) missing definitions

7) the formatting of the whole section is weird, all in courier font, weird spacing... (a raw cut and paste?)

Dear colleagues,

I have more than 13 years experience in MDD, MDA and code generation. Specifically the solution I built and use since more than 12 years is based on MOF 1.3/1.4, UML 1.3, XMI 1.1/1.2 using the NetBeans MetaData Repository as the only free and working standard implementation and able to be integrated with other tools.

The problem: My experience shows that a code generation (M2T) framework should:

• provide powerful text parsing and formatting features.
  For example: page 4 shows an example to generate a Java class with some attributes, just putting their names in the output. In practice:
  • the names of the classes and attributes rarely would be formatted as of the Java naming conventions (and in PIM they should not obey them)
  • the common java conventions state that for each attribute set/get methods might be generated in the format: get<attribute name with first capital letter> (OK, the toUpperFirst() function could help here)
  • the common Database conventions map the class attributes to fields where the separate words in the attribute name are all in upper case and separated by '_' i.e. the testAttribute is mapped to TEST_ATTRIBUTE column in the DB. The same is for class names and table names. Does M2T allow easily implementing this feature and easily maintaining it, considering the fact that in a real project it will be used many hundreds of times? I do not see features to create common functions / methods (other than macro with no result)
• the structure of the models or metamodels to traverse is complex in MOF 1.4 & UML 1.3, it is incredibly more complex in MOF 2+ UML 2+. In such situations the procedural approach becomes complex and hard to maintain. In addition those structures are object-oriented i.e. use inheritance and polymorphic behaviour. Thus, the object-oriented approach to creating the template language would be more-appropriate to support object-oriented structures matching the structure of the metamodel.
• the template language should allow extracting common logic / procedures (macros) and functions outside the templates and sharing the same logic in many templates. The best place would be a structure of classes, corresponding to the metamodel to process.
• the template language should allow using control structures outside the templates themselves and initiate/apply them on model objects or results of queries on the model (i.e. collections of objects).
• it seems to me that a more imperative approach to applying the templates than just using the types of the templates parameters would be easier to understand and maintain. Again, in a real case there are tens of templates nesting in each other, so maintaining implicitly their dependencies might be a real obstacle.
• honestly, I think that it is too easy to implement some logic in an M2T template and then copy it hundred of times, which would make the support a real nightmare.

About the normative example on page 27-:

• the sample model to generate the DB description shows the primary and foreign keys as separate instances:
  • this is obviously a PSM, but the real benefits of MDA are gained when starting from PIM (there are no PK and FKs) and the generation of the intermediate PSM is a really rare practice. Many tools use the PIM and generate the code, scripts, DB schemas, deployment descriptors, etc. out of it. This both imposes more requirements on the M2T features (as of above) and just narrows the area of M2T application in its current state.
  • the example is deceptive, because it assumes that the class and attribute names will be the same as the table and field names. In real practice (see above) there are different naming conventions and in addition some DBs (like Oracle) impose maximum name length restrictions. In a complex model there may be overlapping names of FK constraints, sequence names, field names when reducing / abbreviating them from the human-readable PIM.
  • example 3 on page 30 demonstrates the weaknesses in the generation of the set/get method names - try the Java code conventions.
  • example 1 and 3 are deceptive because they just generate attributes, but do not show the complexity of associations support methods - try the generation of associations support methods for the cases of uni-directional and bi-directional associations in 1:1, 1:M and M:N multiplicity (not mentioning their composition and aggregation kinds). In the real case example I am working with there are 8 templates to generate such.

MTL Currently specifies a "for-each" kind of loop, with no possibility for the user to know the index of the current iteration in the list of values they're iterating over.

It would be nice to have access to an implicit variable within for loops to get this index. Something like

[for (value : String | Sequence

)/]
[i/] = [value/]
[/for]

would then generate text :

1 = a
2 = b
3 = c

I named this implicit variable "i" in this example, it could be named "iterationCount" or "iterationIndex" if we want to avoid conflicts... Yet I'd rather it be named "i" myself.

Section 8.1.5 describes initSections and tells us that

---------8<---------
An InitSection contains a set of variable initializations to be used in the body of its owning block.
--------->8---------

Yet example A.3 sports something like this :

---------8<---------
[template public class_header(c : Class)

]
[...]
[for(a : Attribute) | c.attribute)

]
#define [a.name/]_BIT [count/]
[/for]
--------->8---------

First of all, "int count = 1" is syntactically wrong and should be "count : Integer = -1"; but I've raised this in a separate issue not accepted yet so I don't have its number). The real problem here is that this example initializes a variable "count" in the template's init section, then tries to -modify this variable in the init section of a for.

That just cannot be with the specification as it is now : the second init section should be "count : Integer = count +1" to be syntactically correct, and this doesn't mean "increment the value of 'count'" but "create variable 'count' with value 'count + 1' so we actually create a second variable that hides the first. Furthermore, as this second variable is declared on a for, and initSection's variables are only meant to be useable in the 'for' body, this second "count" variable will only last for an iteration. This means that every iteration, we create a new variable 'count' of value '0' (since the first 'count' variable is '-1').

There is nothing in MOFM2T that's been specified as something that allows for the modification of a variable. We can only initialize them. Modifying variables requires the introduction of a non-generating block. Side-effect free blocks where we could do things without generating text.

The next version should revise the A.3 example so as not to be misleading about the signification of InitSections. It should also introduce a new block dedicated to logic.

Title: Inconsistency in metamodel Message: The Mof2Text metamodel cannot attach static text to a body section. --------- ---------------------- | Block | 1 ---------> 0..* | TemplateExpression | --------- body ---------------------- --------- ---------------------- ----------------- | Block | -------|> | TemplateExpression | --------|> | OclExpression | --------- ---------------------- ----------------- The problem can be reproduced with the following example (extracted from Example 1. Page 25): [template public TableToDDL(t: Table)] CREATE TABLE [t.name/] ( [for (c:Column|t.column) separator(',')] [c.name/] [c.type/] [/for] ); [KeyToDDL(t.key)/] [ForeignKeyToDDL(t.foreignKey)/] [/template] The template body contains a ForBlock, a TemplateExpression ([t.name/]) and two TemplateInvocation (KeyToDDL and ForeignKeyToDDL). It seems that there are no ways to attach the static text portions "CREATE TABLE", "(" and ");" to the model. The OCL specification contains a StringLiteralExp class that could be used to attach this kind of static literal text. But for that to work, StringLiteralExp instances should be attachable somewhere, for instance, in the Block's body. In that case, perhaps that 'body' property could change its type from TemplateExpression to the more general OCLExpression, in order to allow StringLiteralExp's to be directly added to 'body', although this could be too much a general solution and would possibly require further testing. Solution: The metamodel could be change as follows: --------- ----------------- | Block | 1 ---------> 0..* | OclExpression | --------- body -----------------

Title: OCLExpression doesn't support the plus operator for string concatenations Message: The OCL specification defines (see 7.4: Basic Values and Types, page 10) the number of operations on the predefined types. The string type supports the concat, substring and size operations, but the plus operator is only used in integer types. The Mof2Text specification uses the plus operator to concatenate strings, and it's an error. Here there are some examples: 1. 7.2 Traceability --> [trace (c.id + '_definition') ] 2. 7.3 Directing Output to Files --> [file ('file:\\' + c.name + '.java', false, c.id + 'impl')] 3. A.3 Example 3 --> [file (c.name + '.cpp', false)] [trace (c.id + '.header')]

The Mof2Text concrete syntax contain the following rules (see p. 22): <OclExpressionCS> ::= ( <PropertyCallExpCS> | <VariableExpCS> | <LiteralExpCS> | <LetExpCS> | <OclMessageExpCS> | <ifExp> | <invocation> ) <invocation> ::= ( <PathNameCS> '(' <actualarglist> ')' | 'super' ) [ <before> ] [ <separator> ] [ <after> ] <actualarglist> ::= ( <OclExpressionCS> ( ',' <OclExpressionCS> )* )? <before> ::= 'before' '(' <OclExpressionCS> ')' <separator> ::= 'separator' '(' <OclExpressionCS> ')' <after> ::= 'after' '(' <OclExpressionCS> ')' The OCL concrete syntax contain the following rules (see OCL Specificacion. Pages 64, 72, 80 and 81) OclExpressionCS ::= PropertyCallExpCS | VariableExpCS | LiteralExpCS | LetExpCS | OclMessageExpCS | IfExpCS PropertyCallExpCS ::= ModelPropertyCallExpCS | LoopExpCS ModelPropertyCallExpCS ::= OperationCallExpCS | AttributeCallExpCS | NavigationCallExpCS OperationCallExpCS ::= OclExpressionCS[1] simpleNameCS OclExpressionCS[2] | OclExpressionCS '->' simpleNameCS '(' argumentsCS? ')' | OclExpressionCS '.' simpleNameCS '(' argumentsCS? ')' | simpleNameCS '(' argumentsCS? ')' | OclExpressionCS '.' simpleNameCS isMarkedPreCS '(' argumentsCS? ')' | simpleNameCS isMarkedPreCS '(' argumentsCS? ')' | pathNameCS '(' argumentsCS? ')' | simpleNameCS OclExpressionCS argumentsCS[1] ::= OclExpressionCS ( ‘,’ argumentsCS[2] )? The conflict appears when the before, separator and after rules are all empty at the same time in the <invocation> MOF2Text rule. In that case the resolution proceeds as follows: OclExpressionCS ::= invocation ::= PathNameCS '(' actualarglist ')' OclExpressionCS ::= PropertyCallExpCS ::= ModelPropertyCallExpCS ::= OperationCallCS ::= pathNameCS '(' argumentsCS? ')' Solution: The best solution I have found to avoid this conflict involves the introduction of a new keyword, 'invoke'. The invocation rule could be updated as follows: <invocation> ::= invoke ( <PathNameCS> '(' <actualarglist> ')' | 'super' ) [ <before> ] [ <separator> ] [ <after> ] This way we can distinguish easily between the two cases. TemplateInvocation, MacroInvocation and QueryInvocation would then always resolve unambiguously into this last rule.

Grammar errors in section '8.2 concrete syxtax: syntax: ::= 'elseif' '(' <OclExpressionCS> ')' <production_code> ('/elseif') ? It should be: ::= 'elseif' '(' <OclExpressionCS> ')' <production_code> '/elseif' syntax: ::= '[else]' <production> ('[/else]')? It should be: ::= '[else]' <production> '[/else]'

Issue: Typos in examples:
In Annex A, A.1 Example 1:
In the example fragment
[template public SchemaToDDL (s: Schema)]
[for (t:Table | s.table)]
TableToDDL(t)
[/for]
[/template]

I've detected an error in the module rule of the concrete syntax section (8.2). This rule has an inconsistency with respect to the metamodel specified in section 8.1. The syntax is: <module_decl> ::= '[module' <PathNameCS> ‘(‘ <PathNameCS> ‘)’ [extends_decl] '/]' | 'module' <PathNameCS> [extends_decl] ---------- -------------- | Module | 0..* ---------> 1..* | TypedModel | ---------- -------------- So a Module can contain one or more TypedModels, but the grammar does not seem to provide ways to attach more than one. Solution: The module rule could be changed as follows: <module_decl> := '[module' <PathNameCS> ‘(‘ <PathNameCS> ( ‘,’ <PathNameCS>)* ‘)’ [extends_decl] '/]' | 'module' <PathNameCS> ‘(‘ <PathNameCS> ( ‘,’ <PathNameCS>)* ‘)’ [extends_decl]

The class Department_fky has wrong relations, should be between class Employee and class Department_pky.

[Employee : Table]

V
[Department_fky : ForeignKey]

v
[Department_pky : Key]
^

[Department : Table]

In the metamodel diagram in Section 8.1 NamedElement and ModuleElement should be written in italic letters because they are abstract metaclasses.

There are a set of problems with the currently described operations within the MOF Models to Text Transformation Language specification for the addition of new operations to extends the OCL standard library. Specifically, redundant, misleading or plain missing operations. >From now on I'll refer as "OCL specification" the following document : Object Constraint Language specification version 2.0 formal/06-05-01 . Likewise, "MTL specification" will refer to the following : MOF Models to Text Transformation Language version 1.0 formal/2008-01-16. I'll describe the aforementionned issues in three parts from now : A - Redundant operations "toUpper()" and "toLower()" operations are both specified for the String type in section 8.3.1 . Both of these are already defined in the OCL specification in section A.2.1.3 Table A.1 . They should then be removed from the MTL specification. B - Misleading operations Early adopters and testers of our implementation have confirmed that the "strtok(String, Integer) : String" as described in the MTL specification at section 8.3.1 isn't useable as is. Specifically, the integer flag makes it awkard to use : cannot be entirely used in a loop as they need to have the flag at "0" for the first call ... and plain weird to use in any other place, yet again because of this flag. Replacing this operation with a Java-like "tokenise" with no integer flag and returning the whole set of tokens would be easier to consume in a loop or use in templates/queries. C - Missing operations This is a different matter. The following list will consist on the one hand of things we found plain missing from the MTL specification and, on the other hand, of repetitive tasks our early adopters and testers have reported as bothersome to write in OCL and hindering the readability of scripts as a whole. First of all, the MTL specification describes a "substitute(String, String)" operation that can be used to "Substitute substring r in self by substring t and returns the resulting string.". From this we can say that three operations are missing from the spec (more details in the full list at the end of this report) : 1) substituteAll( String substring, String replacement ) : String 2) replace( String substring, String replacement ) : String 3) replaceAll( String substring, String replacement ) : String Further usage from these adopter and testers have shown that some usual operations are impossible (or plain bothersome) to write in OCL. These include (again, detailed description in the full list below) : String operations 4) startsWith( String substring ) : Boolean 5) endsWith( String substring ) : Boolean 6) trim( ) : String OclAny operations 7) ancestors( ) : Sequence(T) 8) ancestors( OclAny type ) : Sequence(T) 9) siblings( ) : Sequence(T) 10) siblings( OclAny type ) : Sequence(T) 11) descendants( ) : Sequence(T) 12) descendants( OclAny type ) : Sequence(T) Likewise, the "toString()" operation described in the MTL specification on both Real and Integer types isn't sufficient for a text generation language this same toString operation should be added to the standard type OclAny so that the user can obtain the String representation of each and every element. To sum this up, here is the full list of operations we would like to add to (or alter within) the MTL specification: C.1 - String operations C.1.a - modifications operation "strtok( String s1, Integer flag ) : String Breaks the string self into a sequence of tokens each of which is delimited by any character in string s1. The parameter flag should be 0 when strtok is called for the first time, 1 subsequently." should be altered to be "strtok( String s1, Integer flag ) : Sequence(T) Returns a sequence containing all parts of self split around delimiters defined by the characters in String delim." C.1.b - additions 1) endsWith( String substring ) : Boolean Returns true if self ends with the substring substring, false otherwise. 2) startsWith( String substring ) : Boolean Returns true if self starts with the substring substring, false otherwise. 3) substituteAll( String substring, String replacement ) : String Substitutes all substrings substring in self by substring replacement and returns the resulting string. If there is no occurrence of the substring, The original string is returned. substring and replacement are not treated as regular expressions. 4) replace( String substring, String replacement ) : String Substitutes the first occurence of substring substring in self by substring replacement and returns the resulting string. If there is no occurrence of the substring, The original string is returned. substring and replacement are treated as regular expressions. 5) replaceAll( String substring, String replacement ) : String Substitutes all substrings substring in self by substring replacement and returns the resulting string. If there is no occurrence of the substring, The original string is returned. substring and replacement are treated as regular expressions. 6) trim( ) : String Removes all leading and trailing spaces of self. C.2 - OclAny operations C.2.a - additions 7) ancestors( ) : Sequence(T) Returns all super-elements of self. 8) ancestors( OclAny type ) : Sequence(T) Returns all super-elements of self which type is equal to type. 9) siblings( ) : Sequence(T) Returns all siblings of self. 10) siblings( OclAny type ) : Sequence(T) Returns all siblings of self which type is equal to type. 11) descendants( ) : Sequence(T) Returns all direct and indirect children of self. 12) descendants( OclAny type ) : Sequence(T) Returns all direct and indirect children of self which type is equal to type. 13) toString( ) : String Returns the String representation of self.

The "MOF Models to Text Transformation Language" specification has an implementation within Eclipse which is not listed on the document catalog ( http://www.omg.org/technology/documents/modeling_spec_catalog.htm ). The implementation name is Acceleo ( http://www.eclipse.org/modeling/m2t/?project=acceleo ), the name that should appear on the document catalog should probably be "Eclipse - Acceleo".

In section 7.3, the file block of the MOF model to text transformation language is detailed. This section introduces the "uniqId" attribute of the file blocks which is meant to allow a tool to find back a generated file even if the object from which it's been generated changed significantly since the last generation.

Though this is a good thing in theory, such an ID cannot be implemented whatever the programming language chosen. Or rather, it cannot be implemented in a satisfying way. Let's assume we have this file block :
[file ('c:/'.concat(class.name).concat('.c'), false, class.id)]

So a file "class.c" will be generated at the root of disk c. Now I change my module so that now, the file block resembles this :
[file ('d:/generated/'.concat(class.name).concat('.c'), false, class.id)]

The class itself hasn't changed one bit; so both the file name (class.c) and its id (class.id) remain constant. How can the tool find back that a file has already been generated for such an ID?

The tool itself cannot keep a reference to all files it has generated along with their IDs and locations; the possibility that we're generating files under version control and that others can generate the same on their own machines forbids this. The solution would then be to save the file along with its ID, thus breaking the WYSIWYG assumption of the specification as unwanted information would show up in the generated file.

And even like that, the tool would need to lookup throughout the whole file system of the target in order to try and find a file which ID is the same as what we need to generate. With the current computers, that could mean searching within terabytes of data in order to find a file that might not even exist in the first place, and that for each [file] block of a generation module.

I believe this unique ID is a good idea in theory, yet seeing as this idea cannot be implemented as more than an idea, I think all references to an ID should be removed from 7.3 (description of the file block), 8.1 (metamodel), 8.1.17 (file block's attributes) and 8.2 (concrete syntax).

The current version of the specification doesn't allow for any "post-processing" of the generated text. I can see two potential needs of post-processing : 1) template level. As well as users can specify guards and overrides, They should be offered the possibility to specify a "post" action in the form of a template or operation call that would be executed on the whole generated text (except for those nested within [file] blocks, see point 2)). For example, I could be defining a template that generates the text pertaining to the body of a Java Class. I might want to trim this text from all surrounding whitespaces before generating and returning it. This would make something like (on UML Class) : [template public classBody(clazz : Class) post (trim())] ... [/template] Such a feature would allow for more readable templates as the user wouldn't be forced to trim() everywhere this template is used. 2) file level. Users might want to call something as a post-processing of a file generation. The most obvious example is the call for a code formatter after a code generation. As above, this would give something like : [file ('Class.java', false) post (format())/] ... [/file]

MTL defines an "open mode" to define how file blocks should behave : whether they append to existing files or overwrite them is defined this way. These two modes are not sufficient : users might want to generate files only when they don't exist. This can be either the very first transformation they execute (thus the file doesn't exist and need be created) or a latter generation when the user has deleted (inadvertently or on purpose) the target file. This is useful for configuration files that do not accept comments (thus rendering protected blocks unusable as these need a marker). An example of such a file is java's MANIFEST.MF file which format is extremely rigid.

MOFM2T aims at being a standard for code generation, and as such it will be used to generate/override files that are under a version control management system. Files can then be generated indiscriminately under windows, linux or mac ... or the three at once for 'big' development teams.

The generated file's encoding then becomes an issue : we cannot always generate files with the default system's encoding : windows' CP1252 as no equivalent in unices, and unix' UTF-8 might cause problems in windows if windows developpers re-generated files that have been first generated under an unix system.

File encoding should be an optional attribute of the file blocks; so someone could write
[file ('class.java', false, 'UTF-16')] or [file ('class.java', false, 'ISO-8859-1')] to tell that the target file should be encoded in such or such way.

Note : this goes along with a previously raised issue in which I asked that the file's unique ID be removed. we'll need to find how to describe the encoding if the unique ID is not removed (as that would make file blocks with two optionnal expressions as their last parameter).

Some of the examples given in the specification assume that there is a "+" operator defined between Strings. This is not true in OCL, and there is none defined in the MOFM2T specification itself.

page 9, section 7.2 :
[trace(c.id()+ '_definition') ]
should be
[trace(c.id().concat('_definition')) ]

page 9, section 7.3 :
[file (‘file:\\’+c.name+’.java’, false, c.id + ‘impl’)]
should be
[file (‘file:\\’.concat(c.name).concat(’.java’), false, c.id.concat(‘impl’))]

Furthermore, the description of this examples tells readers that a file named "cust.sql"; this is wrong. the text below the example of 7.3 should be :

---------8<---------
Suppose the above specification was run on a class named ‘cust,’ it would produce Java code in cust.java file and a log
entry ‘processing cust’ in log.log file. Suppose after generation, the storage specification was added in the protected area
of file cust.java. Even if the classname is changed later, say to ‘customer,’ a tool will be able to retain the storage
specification in the new file customer.java as the file block takes unique id as a parameter that hasn’t changed (for ‘cust’
object). The uri ‘stdout’ denotes the stdout output stream.
--------->8---------

Page 30, Annex A.3
Counting the "+"<=>"concat" error only once, this example sports no less than 9 syntax errors. Replace all text with :

---------8<---------
[module class_header_gen ( UML ) /]

[template public class_header(c : Class)

]
[file (c.name.concat('.cpp'), false)]
[trace(c.id().concat('_header'))]
// Bit vector #defines
[for(a : Attribute | c.attribute)

]
#define [a.name/]_BIT [count/]
[/for]
class [c.name/] [for(c:Class | c.super) before(':') separator(',')] [c.name/] [/for]
{
bool bitVector [‘[‘.concat(c.attribute->size()).concat(’]’)/];
// Attribute declarations
[for (a : Attribute | c.attribute)]
[a.type.name/] [if(isComplexType(a.type))]*[/if] [a.name/];
[/for]
// Constructor
[c.name/]()
{
// initialize bit vector
for (int i = 0; i < [c.attribute->size()/]; i++)

[protected ('user_code')]
// your code here
[/protected]
}
// Attribute set/get/isSpecified methods
[for (a : Attribute | c.attribute)]
void Set[a.name/] ([a.type.name/] [if(isComplexType(a.type))] * [/if] p[a.name/])

[a.type.name/] Get[a.name/] ()

bool isSpecified[a.name/]()

[/for]
// Method declarations
[for (o : Operation | c.operation) ]
[o.type.name/] [o.name/] ([for(p:Parameter | o.parameter) separator(',')] [p.type/] [p.name/] [/for]);
[/for]
}
[/trace]
[/file]
[/template]
--------->8---------

Another solution to fix these errors would be to rely on the 2.1 version of the OCL specification which will introduce a '+' operator for Strings; or to add this operator in the MOFM2T specification.