TL schema for serialization of TL schemas

If necessary, a TL schema can be serialized in binary form. Here, this serialization format is defined by a TL schema (usually stored in the file tl.tl). This can be useful, for example, to make it possible to write a parser one time for converting a TL schema from text form (stored in the file something.tl) to binary form (stored in the file something.tlo). All other programs (for example, auto-generators of TL-(de)serializers for various programming languages) only need to know how to read .tlo files, which only requires generating an automatic deserializer according to the schema presented below.

First, a fragment of the file common.tl with certain required built-in types:

/////
//
// Common Types (source file common.tl, only necessary definitions included)
//
/////

// Built-in types
int ? = Int;
long ? = Long;
double ? = Double;
string ? = String;

// Boolean emulation
boolFalse = Bool;
boolTrue = Bool;

// Vector
vector {t:Type} # [t] = Vector t;
tuple {t:Type} {n:#} [t] = Tuple t n;
vectorTotal {t:Type} total_count:int vector:%(Vector t) = VectorTotal t;

Empty False;
true = True;

Next, properly, comes tl.tl itself. Note that the declaration for a fairly complex data type required only twenty lines in TL. This demonstrates the expressiveness and compactness of the TL language.

/////
//
// Serialized binary TL-schema in TL format, source file tl.tl
//
/////
tls.schema_v2 version:int date:int types_num:# types:types_num*[tls.Type] 

    constructor_num:# constructors:constructor_num*[tls.Combinator] 
    functions_num:# functions:functions_num*[tls.Combinator] = tls.Schema;
tls.type name:int id:string constructors_num:int flags:int arity:int params_type:long = tls.Type;

tls.combinator name:int id:string type_name:int left:tls.CombinatorLeft right:tls.CombinatorRight = tls.Combinator;
tls.combinatorLeftBuiltin = tls.CombinatorLeft;
tls.combinatorLeft args_num:# args:args_num*[tls.Arg] = tls.CombinatorLeft;
tls.combinatorRight value:tls.TypeExpr = tls.CombinatorRight;

tls.arg id:string flags:# var_num:flags.1?int exist_var_num:flags.2?int exist_var_bit:flags.2?int type:tls.TypeExpr = tls.Arg;

tls.exprType _:tls.TypeExpr = tls.Expr;

tls.exprNat _:tls.NatExpr = tls.Expr;
tls.natConst value:int = tls.NatExpr;

tls.natVar dif:int var_num:int = tls.NatExpr;
tls.typeVar var_num:int flags:int = tls.TypeExpr;

tls.array multiplicity:tls.NatExpr args_num:# args:args_num*[tls.Arg] = tls.TypeExpr;
tls.typeExpr name:int flags:int children_num:# children:children_num*[tls.Expr] = tls.TypeExpr;

Remarks

Schema serialization (version 2) always begins with the index number of the tls.schema_v2 constructor for tls.Schema. Because the CRC32 of the string

tls.schema_v2 version:int date:int types_num:# types:types_num*[ tls.Type ] constructor_num:# constructors:constructor_num*[ tls.Combinator ] functions_num:# functions:functions_num*[ tls.Combinator ] = tls.Schema

is 0x3a2f9be2, this constant is in fact the magic number for tlo files in the current version’s format. If the format is extended in the future (for example, if TL’s additional features are supported), then a tls.schema_v3 constructor with a different number will appear.

Example

If one adds declarations for the used built-in types (like int ? = Int;) from the file common.tl before tl.tl and serialize the resulting schema, the following binary data is obtained (tl.tlo):

0000: 3a2f9be2 00000000 51fec698 00000015 12eb4386 70659eff 00002301 00000000
0020: 00000000 00000000 00000000 00000000 12eb4386 250be282 6f6f4204 0000006c
0040: 00000002 02000010 00000000 00000000 00000000 12eb4386 2210c154 756f4406
0060: 00656c62 00000001 02000000 00000000 00000000 00000000 12eb4386 00000000
0080: 6c614605 00006573 00000000 00000401 00000000 00000000 00000000 12eb4386
00a0: a8509bda 746e4903 00000001 02000001 00000000 00000000 00000000 12eb4386
00c0: 22076cba 6e6f4c04 00000067 00000001 02000001 00000000 00000000 00000000
00e0: 12eb4386 b5286e24 72745306 00676e69 00000001 02000001 00000000 00000000
0100: 00000000 12eb4386 3fedd339 75725404 00000065 00000001 02000000 00000000
0120: 00000000 00000000 12eb4386 9770768a 70755405 0000656c 00000001 02000000
0140: 00000002 00000002 00000000 12eb4386 2cecf817 70795404 00000065 00000000
0160: 00000000 00000000 00000000 00000000 12eb4386 1cb5c415 63655606 00726f74
0180: 00000001 02000008 00000001 00000000 00000000 12eb4386 10133f47 6365560b
01a0: 54726f74 6c61746f 00000001 02000000 00000001 00000000 00000000 12eb4386
01c0: 29dfe61b 736c7407 6772412e 00000001 02000000 00000000 00000000 00000000
01e0: 12eb4386 5c0a1ed5 736c740e 6d6f432e 616e6962 00726f74 00000001 02000000
0200: 00000000 00000000 00000000 12eb4386 8133d9ba 736c7412 6d6f432e 616e6962
0220: 4c726f74 00746665 00000002 02000010 00000000 00000000 00000000 12eb4386
0240: 2c064372 736c7413 6d6f432e 616e6962 52726f74 74686769 00000001 02000000
0260: 00000000 00000000 00000000 12eb4386 307d41a0 736c7408 7078452e 00000072
0280: 00000002 02000010 00000000 00000000 00000000 12eb4386 c2635441 736c740b
02a0: 74614e2e 72707845 00000002 02000010 00000000 00000000 00000000 12eb4386
02c0: 3a2f9be2 736c740a 6863532e 00616d65 00000001 02000000 00000000 00000000
02e0: 00000000 12eb4386 12eb4386 736c7408 7079542e 00000065 00000001 02000000
0300: 00000000 00000000 00000000 12eb4386 193fd378 736c740c 7079542e 70784565
0320: 00000072 00000003 02000010 00000000 00000000 00000000 00000018 5c0a1ed5
0340: bc799737 6f6f6209 6c61466c 00006573 250be282 4c12c6d9 00000000 2c064372
0360: c1863d08 250be282 00000000 00000000 5c0a1ed5 997275b5 6f6f6208 7572546c
0380: 00000065 250be282 4c12c6d9 00000000 2c064372 c1863d08 250be282 00000000
03a0: 00000000 5c0a1ed5 2210c154 756f6406 00656c62 2210c154 cd211f63 2c064372
03c0: c1863d08 2210c154 00000000 00000000 5c0a1ed5 a8509bda 746e6903 a8509bda
03e0: cd211f63 2c064372 c1863d08 a8509bda 00000000 00000000 5c0a1ed5 22076cba
0400: 6e6f6c04 00000067 22076cba cd211f63 2c064372 c1863d08 22076cba 00000000
0420: 00000000 5c0a1ed5 b5286e24 72747306 00676e69 b5286e24 cd211f63 2c064372
0440: c1863d08 b5286e24 00000000 00000000 5c0a1ed5 3fedd339 75727404 00000065
0460: 3fedd339 4c12c6d9 00000000 2c064372 c1863d08 3fedd339 00000000 00000000
0480: 5c0a1ed5 9770768a 70757405 0000656c 9770768a 4c12c6d9 00000003 29dfe61b
04a0: 00007401 00020005 00000000 c1863d08 2cecf817 00000000 00000000 29dfe61b
04c0: 00006e01 00020005 00000001 c1863d08 70659eff 00000000 00000000 29dfe61b
04e0: 00000000 00000000 d9fb20de 4e8a14f0 00000000 00000001 00000001 29dfe61b
0500: 00000000 00000000 0142ceae 00000000 00000000 2c064372 c1863d08 9770768a
0520: 00000000 00000002 ecc9da78 0142ceae 00000000 00000000 dcb49bd8 4e8a14f0
0540: 00000000 00000001 5c0a1ed5 1cb5c415 63657606 00726f74 1cb5c415 4c12c6d9
0560: 00000003 29dfe61b 00007401 00020005 00000000 c1863d08 2cecf817 00000000
0580: 00000000 29dfe61b 00000000 00000004 00000001 c1863d08 70659eff 00000000
05a0: 00000000 29dfe61b 00000000 00000000 d9fb20de 4e8a14f0 00000000 00000001
05c0: 00000001 29dfe61b 00000000 00000000 0142ceae 00000000 00000000 2c064372
05e0: c1863d08 1cb5c415 00000000 00000001 ecc9da78 0142ceae 00000000 00000000
0600: 5c0a1ed5 10133f47 6365760b 54726f74 6c61746f 10133f47 4c12c6d9 00000003
0620: 29dfe61b 00007401 00020005 00000000 c1863d08 2cecf817 00000000 00000000
0640: 29dfe61b 746f740b 635f6c61 746e756f 00000000 c1863d08 a8509bda 00000001
0660: 00000000 29dfe61b 63657606 00726f74 00000000 c1863d08 1cb5c415 00000001
0680: 00000001 ecc9da78 0142ceae 00000000 00000000 2c064372 c1863d08 10133f47
06a0: 00000000 00000001 ecc9da78 0142ceae 00000000 00000000 5c0a1ed5 29dfe61b
06c0: 736c7407 6772612e 29dfe61b 4c12c6d9 00000006 29dfe61b 00646902 00000000
06e0: c1863d08 b5286e24 00000001 00000000 29dfe61b 616c6605 00007367 00000004
0700: 00000000 c1863d08 70659eff 00000000 00000000 29dfe61b 72617607 6d756e5f
0720: 00000002 00000000 00000001 c1863d08 a8509bda 00000001 00000000 29dfe61b
0740: 6978650d 765f7473 6e5f7261 00006d75 00000002 00000000 00000002 c1863d08
0760: a8509bda 00000001 00000000 29dfe61b 6978650d 765f7473 625f7261 00007469
0780: 00000002 00000000 00000002 c1863d08 a8509bda 00000001 00000000 29dfe61b
07a0: 70797404 00000065 00000000 c1863d08 193fd378 00000000 00000000 2c064372
07c0: c1863d08 29dfe61b 00000000 00000000 5c0a1ed5 5c0a1ed5 736c740e 6d6f632e
07e0: 616e6962 00726f74 5c0a1ed5 4c12c6d9 00000005 29dfe61b 6d616e04 00000065
0800: 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b 00646902 00000000
0820: c1863d08 b5286e24 00000001 00000000 29dfe61b 70797409 616e5f65 0000656d
0840: 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b 66656c04 00000074
0860: 00000000 c1863d08 8133d9ba 00000000 00000000 29dfe61b 67697205 00007468
0880: 00000000 c1863d08 2c064372 00000000 00000000 2c064372 c1863d08 5c0a1ed5
08a0: 00000000 00000000 5c0a1ed5 cd211f63 736c7419 6d6f632e 616e6962 4c726f74
08c0: 42746665 746c6975 00006e69 8133d9ba 4c12c6d9 00000000 2c064372 c1863d08
08e0: 8133d9ba 00000000 00000000 5c0a1ed5 4c12c6d9 736c7412 6d6f632e 616e6962
0900: 4c726f74 00746665 8133d9ba 4c12c6d9 00000002 29dfe61b 67726108 756e5f73
0920: 0000006d 00000004 00000000 c1863d08 70659eff 00000000 00000000 29dfe61b
0940: 67726104 00000073 00000000 d9fb20de 4e8a14f0 00000000 00000000 00000001
0960: 29dfe61b 00000000 00000000 c1863d08 29dfe61b 00000000 00000000 2c064372
0980: c1863d08 8133d9ba 00000000 00000000 5c0a1ed5 2c064372 736c7413 6d6f632e
09a0: 616e6962 52726f74 74686769 2c064372 4c12c6d9 00000001 29dfe61b 6c617605
09c0: 00006575 00000000 c1863d08 193fd378 00000000 00000000 2c064372 c1863d08
09e0: 2c064372 00000000 00000000 5c0a1ed5 ecc9da78 736c740c 7078652e 70795472
0a00: 00000065 307d41a0 4c12c6d9 00000001 29dfe61b 00000000 00000000 c1863d08
0a20: 193fd378 00000000 00000000 2c064372 c1863d08 307d41a0 00000000 00000000
0a40: 5c0a1ed5 dcb49bd8 736c740b 7078652e 74614e72 307d41a0 4c12c6d9 00000001
0a60: 29dfe61b 00000000 00000000 c1863d08 c2635441 00000000 00000000 2c064372
0a80: c1863d08 307d41a0 00000000 00000000 5c0a1ed5 8ce940b1 736c740c 74616e2e
0aa0: 736e6f43 00000074 c2635441 4c12c6d9 00000001 29dfe61b 6c617605 00006575
0ac0: 00000000 c1863d08 a8509bda 00000001 00000000 2c064372 c1863d08 c2635441
0ae0: 00000000 00000000 5c0a1ed5 4e8a14f0 736c740a 74616e2e 00726156 c2635441
0b00: 4c12c6d9 00000002 29dfe61b 66696403 00000000 c1863d08 a8509bda 00000001
0b20: 00000000 29dfe61b 72617607 6d756e5f 00000000 c1863d08 a8509bda 00000001
0b40: 00000000 2c064372 c1863d08 c2635441 00000000 00000000 5c0a1ed5 3a2f9be2
0b60: 736c740d 6863732e 5f616d65 00003276 3a2f9be2 4c12c6d9 00000008 29dfe61b
0b80: 72657607 6e6f6973 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b
0ba0: 74616404 00000065 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b
0bc0: 70797409 6e5f7365 00006d75 00000004 00000000 c1863d08 70659eff 00000000
0be0: 00000000 29dfe61b 70797405 00007365 00000000 d9fb20de 4e8a14f0 00000000
0c00: 00000000 00000001 29dfe61b 00000000 00000000 c1863d08 12eb4386 00000000
0c20: 00000000 29dfe61b 6e6f630f 75727473 726f7463 6d756e5f 00000004 00000001
0c40: c1863d08 70659eff 00000000 00000000 29dfe61b 6e6f630c 75727473 726f7463
0c60: 00000073 00000000 d9fb20de 4e8a14f0 00000000 00000001 00000001 29dfe61b
0c80: 00000000 00000000 c1863d08 5c0a1ed5 00000000 00000000 29dfe61b 6e75660d
0ca0: 6f697463 6e5f736e 00006d75 00000004 00000002 c1863d08 70659eff 00000000
0cc0: 00000000 29dfe61b 6e756609 6f697463 0000736e 00000000 d9fb20de 4e8a14f0
0ce0: 00000000 00000002 00000001 29dfe61b 00000000 00000000 c1863d08 5c0a1ed5
0d00: 00000000 00000000 2c064372 c1863d08 3a2f9be2 00000000 00000000 5c0a1ed5
0d20: 12eb4386 736c7408 7079742e 00000065 12eb4386 4c12c6d9 00000006 29dfe61b
0d40: 6d616e04 00000065 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b
0d60: 00646902 00000000 c1863d08 b5286e24 00000001 00000000 29dfe61b 6e6f6310
0d80: 75727473 726f7463 756e5f73 0000006d 00000000 c1863d08 a8509bda 00000001
0da0: 00000000 29dfe61b 616c6605 00007367 00000000 c1863d08 a8509bda 00000001
0dc0: 00000000 29dfe61b 69726105 00007974 00000000 c1863d08 a8509bda 00000001
0de0: 00000000 29dfe61b 7261700b 5f736d61 65707974 00000000 c1863d08 22076cba
0e00: 00000001 00000000 2c064372 c1863d08 12eb4386 00000000 00000000 5c0a1ed5
0e20: 0142ceae 736c740b 7079742e 72615665 193fd378 4c12c6d9 00000002 29dfe61b
0e40: 72617607 6d756e5f 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b
0e60: 616c6605 00007367 00000000 c1863d08 a8509bda 00000001 00000000 2c064372
0e80: c1863d08 193fd378 00000000 00000000 5c0a1ed5 d9fb20de 736c7409 7272612e
0ea0: 00007961 193fd378 4c12c6d9 00000003 29dfe61b 6c756d0c 6c706974 74696369
0ec0: 00000079 00000000 c1863d08 c2635441 00000000 00000000 29dfe61b 67726108
0ee0: 756e5f73 0000006d 00000004 00000000 c1863d08 70659eff 00000000 00000000
0f00: 29dfe61b 67726104 00000073 00000000 d9fb20de 4e8a14f0 00000000 00000000
0f20: 00000001 29dfe61b 00000000 00000000 c1863d08 29dfe61b 00000000 00000000
0f40: 2c064372 c1863d08 193fd378 00000000 00000000 5c0a1ed5 c1863d08 736c740c
0f60: 7079742e 70784565 00000072 193fd378 4c12c6d9 00000004 29dfe61b 6d616e04
0f80: 00000065 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b 616c6605
0fa0: 00007367 00000000 c1863d08 a8509bda 00000001 00000000 29dfe61b 6968630c
0fc0: 6572646c 756e5f6e 0000006d 00000004 00000000 c1863d08 70659eff 00000000
0fe0: 00000000 29dfe61b 69686308 6572646c 0000006e 00000000 d9fb20de 4e8a14f0
1000: 00000000 00000000 00000001 29dfe61b 00000000 00000000 c1863d08 307d41a0
1020: 00000000 00000000 2c064372 c1863d08 193fd378 00000000 00000000 00000000