Delphi Nullable with Custom Managed Records

Published with special permission from Embarcadero - this post writes about pre-release Delphi version where everything is subject to change until finally released.



Nullables are a rather simple concept. All you need is a value, and some flag that will tell you whether or not the value has been explicitly set or not.
type TNullable<T> = record private FHasValue: boolean; FValue: T; function GetValue: T; procedure SetValue(AValue: T); public property HasValue: boolean read FHasValue; property Value: T read GetValue write SetValue; end; function TNullable<T>.GetValue: T; begin if FHasValue then Result := FValue else raise Exception.Create('Invalid operation, Nullable type has no value'); end; procedure TNullable<T>.SetValue(AValue: T); begin FHasValue := True; FValue := AValue; end;

Plain and simple. But then you have the fact that records are not automatically initialized and that your FHasValue boolean can hold a random value. Of course, you can always add some initialization procedure that will set FHasValue to False before you start using the record variable but this is error-prone. Sooner or later, you will forget to call the initialization procedure, and the whole thing will fall apart. What is worse, it might not break immediately so you can fix the problem, but it may fail randomly at runtime and you will end up chasing random bugs you cannot exactly figure out.

In Delphi, managed types (like strings and interfaces) when put in record will be automatically initialized even when such a record is declared as local variable. Following that fact, instead of using a boolean as a flag you can use an interface. How to do that and how to optimize such a record (interfaces introduce speed penalty and locking) by using "fake" interfaces, you can read in Allen Bauer's excellent post A "Nullable" Post

With the upcoming Delphi 10.4 Sydney and the introduction of custom managed records, our initial nullable implementation with the boolean flag has got another chance.

Since all we need is automatic initialization, the additional code will be quite simple, just have to add an Initialize operator and that is it.
type TNullable<T> = record private FHasValue: boolean; FValue: T; function GetValue: T; procedure SetValue(AValue: T); public class operator Initialize(out Dest: TNullable<T>); property HasValue: boolean read FHasValue; property Value: T read GetValue write SetValue; end; class operator TNullable<T>.Initialize(out Dest: TNullable<T>); begin Dest.FHasValue := False; end;

Not only is this Nullable implementation much simpler and more understandable code than the interface-based Nullable implementation, but it is also much faster (approximately 8 times).

More details about custom managed records you can find at Custom Managed Records Coming to Delphi 10.4l

Comments

  1. What if you want to have this Nullable type automatically converted/imported to JSON?
    Will the HasValue (and Value) member also be included in the result?
    In JSON you just want to have (or not) the actual T-value .

    ReplyDelete
    Replies
    1. Default Delphi Rest.Json serialization will include both HasValue and Value in Json result.

      TMyObject = class
      protected
      FField: TNullable;
      published
      property Field: TNullable read FField write FField;
      end;

      If you set filed value resulting Json string will look like {"field":[true,5]}, if not {"field":[false,0]}

      In second example value doesn't have to be 0 because it is not explicitly initialized. If you want to explicitly initialize Value to default value for some reason you will have to do that in Initialize operator.

      If you want to omit HasValue field and have some different Json format you will have to write custom converters that will apply appropriate conversion.

      In that area there is really nothing new and different from various nullable implementations currently used in Delphi.

      Delete
    2. Hi Dalija,
      Thanks for your quick response.

      Based on your example, I was indeed expecting a JSON in the form like {"field":[true,5]}, if not {"field":[false,0]}
      But what you normally would like is something like {"field":5}, if not {"field":null}
      (or even the complete absence of "field" when null)

      I've read the Nullable post in the past (and just now again).
      The post is almost 12 years old, so it's a pity we had to wait so long to get something close (not your fault by the way).
      I understand I could write and extend the TNullable definition above so it probably will produce the desired JSON handling.

      There are a few things that bother me.
      I would expect that a standard definition for TNullable or an equivalent should come from Embarcadero.
      They could optimize it for speed e.g. and make it work with the REST JSON functions.
      It should be library or language functionality.

      If I would write my own implementation it may be sub-optimal and can and probably will not be exactly the same as one other persons implementation.
      e.g. I could have a IsNull method instead of HasValue.
      That could mean that this code is not exchangable with other developers without have to fix the differences first.

      I think we are coming closer and closer.
      But in my opinion the whole nullable concept and implementation should be available right out-of-the-box in Delphi.

      I would give a big thumbs up when it has finally arrived.

      Delete
    3. I fully agree. Having standard type would be preferable. Also full nullable support would have to include more than just type, but also compiler support for easier handling of null values.

      There is open feature request for Nullable support in Quality Portal https://quality.embarcadero.com/browse/RSP-13305

      I hope it will be implemented some day.

      Delete
  2. Except, of course, this will require indirection to get to the value inside and be a klunky mess in all your code compared to natively supported nullable types.

    Is this what we are expecting from Embarcadero as their solution to "nullable" types?

    ReplyDelete
    Replies
    1. This is minimal example showing implementation difference in FHasValue field. You can always declare Implicit operators that will directly set or retrieve Value without explicitly typing Value.

      You can find full example with operators in linked "Nullable" Post.

      Delete
  3. Initialize will slowdown things, because this is quite different from simple zero-fill, which is used currently for all other managed types (like strings or interfaces).
    Also, semantic of nullable types should be consistent with the semantic of all other primitive types. "Variable not initialized" warning should be used for nullable types, just like it used for all other simple types.
    Finally, nullable types cannot be implemented as a simple generic record without compiler support, but thats another story.

    ReplyDelete
    Replies
    1. Initialization of other managed types also implies finalization, also setting nullable has additional performance cost with interface based nullable implementation...

      Since Delphi does not have compiler support for nullable types talking about semantic is kind of moot.

      Yes, having compiler supported nullable types brings nullables to another level and it would be more than great to have such support one day, but that does not mean that right now you cannot have working nullable implementation at all.

      Delete

Post a Comment

Popular posts from this blog

Just released eBook: Delphi Event-based and Asynchronous Programming

Unified Memory Management - Coming with 10.4 Beta

Magic behind FreeAndNil