When in Rome, do as the Romans do - Part II

In the previous article, we learned that native Delphi serialization is based around published properties, and that other kinds of serialization, like field-based serialization, don't fit well with the Delphi model.

Photo: Gary Todd from Xinzheng, China, PDM-owner, via Wikimedia Commons, https://commons.wikimedia.org/wiki/File:Colosseum_(48412957912).jpg

We have also learned that property-based serialization has some limitations and that it does not have out-of-the-box support for the serialization of records, arrays, or indexed properties, and you can only serialize collections defined as TCollection, whose items are descendants of TCollectionItem.

While serialization as a concept is widely used to convert any kind of objects into various storage formats, the main purpose of the Delphi serialization model being built on top of published properties and descendants of TComponent was to serialize the user interface and data modules. This is where the original type limitations came from. Simply put, they supported the types needed to serialize visual controls and other design-time components. Since those types also had to be supported by the IDE and property editors, having a single supported collection base type in the form of TCollection was a sensible choice at the time.

But with the provided basic building blocks and RTTI functionality emitted for published properties, you could write your own serialization frameworks with extended support for other types and the ability to serialize your objects to various formats. But before the introduction of the enhanced Delphi RTTI in Delphi 2010, all that was done using published properties.

Enhanced RTTI and its ability to access information about the whole object, rather than just the published section, along with broader support for various types, opened up the possibility of serializing fields (even private ones), not only published properties.

And this is where the story of TList<T> begins...

Around that time, the Delphi language got plenty of new features. Besides enhanced RTTI, there were anonymous methods, generics, attributes... A few years later, Delphi stepped out into the mobile world, adding support for Android and iOS platforms.

And with all that, support for another rapidly growing technology—REST services—found its way into the core Delphi libraries and along with it, the need to provide JSON serialization for simple objects that commonly contain lists and arrays.

But the built-in JSON serialization library went in the wrong direction from an architectural point of view. Instead of using published properties for serialization (maybe due to a lack of out-of-the-box support for arrays and lists), it uses fields. Of course, the ability to serialize arrays and lists fields is something that had to be written from scratch, but there is nothing that would have prevented writing similar code that works on properties. In other words, there is no technical reason why fields had to be used over published properties.

The first issue with field-based serialization is that unlike properties, which usually have an actual name you want to serialize (with the possible exception of casing, that can be auto-converted), fields in Delphi classes are usually prefixed with an F. But field names are really an implementation detail, not a part of the public API in any way, so some developers were using other prefixes for fields.

Suddenly, with the introduction of REST service support, what used to be an irrelevant internal detail, now became part of the public API. And the first roadblock with using the built-in JSON serialization was an issue with fields that were not following the F prefix convention. Yes, I know you can change the field names, but this is just another alarm showing that field-based serialization imposed on existing code is not the right approach—not the Delphian approach.

In the previous article, I had already mentioned "garbage" fields that would end up being serialized by default, and the framework-dependent way to get rid of them that required adding custom framework-related attributes.

But, what all that has to do with the story of TList<T>?

Well, JSON serialization of TList<T> is probably the most broken feature in the history of Delphi. It would routinely get broken between Delphi versions because TList<T> was going through some heavy internal reorganization in order to increase performance and reduce code bloat with almost every new version. And it had some extra garbage to begin with. Since TList<T> is one of the core classes, adding any serialization framework dependency to fix and define its serialization would be a huge red flag.

Some of the bug reports:

And even if TList<T> received some kind of special treatment in REST.JSON serialization, that still would not solve the issues and provide uniform serialization of classes that use custom collections. Basically, instead of using feature-rich lists, you would have to use plain dynamic arrays in order to get consistently serialized fields.

And how does this look today, in Delphi 10.4 Sydney?

TFoo = class(TPersistent) protected FData: string; FTemp: string; FItems: TList<Integer>; public ... end; var Foo: TFoo; Str: string; begin Foo := TFoo.Create; Foo.Data := 'abc'; Foo.Temp := 'temporary'; Foo.Items.Add(4); Foo.Items.Add(5); Foo.Items.Add(6); Str := TJson.ObjectToJsonString(Foo); end;

Extending the class from the previous article with an integer list, and serializing it with the default JSON options we get a nice, neat... uh, oh.... JSON string

{ "data": "abc", "temp": "temporary", "items": { "listHelper": [ 4, 5, 6 ] } }

At the end of the day... if you need or want to use the built-in JSON serialization (and if you use DataSnap, you will have to use it) the only way to save yourself in the long run is to carefully handcraft your classes, and that usually means limiting their use and reuse as business objects, which will then lead to unnecessary code duplication.


Well, this started as a more general article advising the usage of coding patterns that fit the language and avoiding patterns that aren't appropriate in it. That advice still stands: No matter what kind of code you write, stick to what's most appropriate for Delphi. That does not mean that other languages don't offer easily transferable and useful coding patterns, though. Just think a bit before you apply them.

As far as serialization is concerned, there are two separate issues here, and both should be considered when choosing any kind of serialization framework.

The first is using field-based serialization instead of published properties. Now, if that could be easily configured, then it would not matter so much, as developers could choose their preferred behavior based on their existing classes.

The second, and the most important issue, is the inability to configure serialization without adding a serialization framework as a dependency to your business objects. That is a huge, enormous red flag. Any attribute-based serialization is pure evil. You should avoid it like the plague. A good serialization framework will allow you to configure everything without requiring you to add it as a dependency. Also, a good serialization framework will use the best defaults, so that the configuration code could be as simple as possible in most cases, while still allowing everything to be tweaked when needed.

And last, but not least, a good serialization framework would not even offer attribute-based serialization configurations, because it is just unnecessary code for a really bad serialization approach. A good serialization framework would not trick its users into using such code, ever.

Of course, you may find a framework that satisfies the first two conditions, but not the last one. If you cannot find any better solution, then the first two are good enough. But for the love of your favorite deity, just don't use attributes for configuring serialization, unless you absolutely don't have any other choice.


Popular posts from this blog

Just released eBook: Delphi Event-based and Asynchronous Programming

Magic behind FreeAndNil

Are const parameters dangerous?