Using a dynamic DTO property in a NestJS API

NestJS is designed around using strictly typed properties on models but sometimes it's useful (and fast!) to allow dynamic types on properties and just store some business domain data as a dynamic serialized blob.

This is the serialized LOB method recommended by Martin Fowler (https://martinfowler.com/eaaCatalog/serializedLOB.html).

Here is how you can have a LOB in a NestJS REST Api with type safety and support for OpenAPI definitions.

Typical Nest Js models

Here is a typical entity in nestjs that you can save to a datastore with typeorm. This class might be used to store the configuration data to trigger a bot.

There are some custom classes (CustomBot) that are saved to the database in a relational way.

There is a discriminator in the enum that sets the type of Trigger this is.

@Entity()
export class Trigger {
  @PrimaryGeneratedColumn()
  @ApiProperty()
  public id!: number;

  @Column("uuid", {
    name: "uuid",
    default: () => "uuid_generate_v4()",
  })
  @Generated("uuid")
  @ApiProperty()
  public uuid!: string;

  @Column({
    type: "enum",
    enum: TriggerTypeEnum,
    default: TriggerTypeEnum.NO_ACTION_DEFAULT,
  })
  @ApiProperty({ enum: TriggerTypeEnum, enumName: "TriggerTypeEnum" })
  public triggerType!: TriggerTypeEnum;

  @Exclude()
  @ManyToOne(() => CustomBot, (customBot) => customBot.triggers, {
    eager: true,
    onDelete: "CASCADE",
  })
  @Index()
  @Type(() => CustomBot)
  @JoinColumn({ name: "customBotId" })
  customBot!: CustomBot;

  @Column()
  @ApiProperty()
  customBotId!: number;
}

The equivalent API DTO for creating something like this would be simpler because most of the properties are generated.

The custom bot id for the relation would be in a url parameter, not in the dto body. So it would look something like this.

export class CreateTriggerDto {
  @IsDefined()
  @IsEnum(TriggerTypeEnum)
  @ApiProperty({ enum: TriggerTypeEnum, enumName: "TriggerTypeEnum" })
  public triggerType!: TriggerTypeEnum;
}

Adding meta data here

So now if we want to store meta information here we need to add a property to store it.

Postgres allows us to store json blobs in columns so while most of our properties can be strictly defined and relational.

We can store json directly in postgres when we have multiple representations of data. Type ORM supports this by setting the column type. e.g.

    @Column({ type: "jsonb" })
    public meta!: MyComplexModel;

This is extremely useful for meta information associated with a business domain object. Just store it as is, when it's retrieved it will be converted in to the correct model.

The dynamic meta data issue

The interesting problem here is how do we store and retrieve different classes for different TriggerTypeEnum values?

We want to have accurate OpenApi specification and we want to have type safety throughout our code.

A dynamic Create DTO

To create a dynamic Create DTO model in NestJS we need to

Tell class-transformer how to convert the input json to classes
Tell OpenAPI that there are multiple possible types for this input

Below I show how to use the discriminator property on the @Type decorator to tell class-transformer how we want to create the class that is assigned to the property.

You can also see how I set the oneOf property on the @ApiProperty decorator. This creates valid OpenApi v3 specification.

NOTE: There is an issue with oneOf for some of the open api plugins at the moment because they haven't been updated to work with it. I'll talk about this at the end of the article.

export class CreateTriggerDto {
  @IsDefined()
  @IsEnum(TriggerTypeEnum)
  @ApiProperty({ enum: TriggerTypeEnum, enumName: "TriggerTypeEnum" })
  public triggerType!: TriggerTypeEnum;

  @Type(() => TriggerMeta, {
    discriminator: {
      property: "triggerType",
      subTypes: [
        {
          value: TwitterUserMentionMeta,
          name: TriggerTypeEnum.TWITTER_USER_MENTION,
        },
        {
          value: NoActionTestMeta,
          name: TriggerTypeEnum.NO_ACTION_DEFAULT,
        },
      ],
    },
  })
  @IsDefined()
  @ApiProperty({
    oneOf: [
      { $ref: getSchemaPath(TwitterUserMentionMeta) },
      { $ref: getSchemaPath(NoActionTestMeta) },
    ],
  })
  public meta!: TwitterUserMentionMeta | NoActionTestMeta;
}

The entity for storing to the database is similar - we add the type discriminator and the api property anyof.

@Entity()
export class Trigger {
  @PrimaryGeneratedColumn()
  @ApiProperty()
  public id!: number;

  @Column("uuid", {
    name: "uuid",
    default: () => "uuid_generate_v4()",
  })
  @Generated("uuid")
  @ApiProperty()
  public uuid!: string;

  @Column({
    type: "enum",
    enum: TriggerTypeEnum,
    default: TriggerTypeEnum.NO_ACTION_DEFAULT,
  })
  @ApiProperty({ enum: TriggerTypeEnum, enumName: "TriggerTypeEnum" })
  public triggerType!: TriggerTypeEnum;

  @Exclude()
  @ManyToOne(() => CustomBot, (customBot) => customBot.triggers, {
    eager: true,
    onDelete: "CASCADE",
  })
  @Index()
  @Type(() => CustomBot)
  @JoinColumn({ name: "customBotId" })
  customBot!: CustomBot;

  @Column()
  @ApiProperty()
  customBotId!: number;

  @Column({ type: "jsonb" })
  @Type(() => TriggerMeta, {
    discriminator: {
      property: "triggerType",
      subTypes: [
        {
          value: TwitterUserMentionMeta,
          name: TriggerTypeEnum.TWITTER_USER_MENTION,
        },
        {
          value: NoActionTestMeta,
          name: TriggerTypeEnum.NO_ACTION_DEFAULT,
        },
      ],
    },
  })
  @IsDefined()
  @ApiProperty()
  @ApiProperty({
    oneOf: [
      { $ref: getSchemaPath(TwitterUserMentionMeta) },
      { $ref: getSchemaPath(NoActionTestMeta) },
    ],
  })
  public meta!: TwitterUserMentionMeta | NoActionTestMeta;
}

Current issue with anyof and typescript-fetch open api generator

The typescript fetch open api generator doesn't support anyof.

If you are generating a java client or a .net client you will have no issues using the method described here. However if you're generating a typescript client it won't work.

You will have to manually discriminate the class. The way I did this is to create a new parent container model with optional properties and then manually assign that where required.

A new type like this will provide the api consumer properties to provide meta data in a typed format.

export default class AllMetaTypes {
  @ApiPropertyOptional()
  public twitterUserMentionMeta?: TwitterUserMentionMeta;

  @ApiPropertyOptional()
  public noActionTestMeta?: NoActionTestMeta;
}

Then your create DTO model would use this type on the meta property.

export class CreateTriggerDto {
  @IsDefined()
  @IsEnum(TriggerTypeEnum)
  @ApiProperty({ enum: TriggerTypeEnum, enumName: "TriggerTypeEnum" })
  public triggerType!: TriggerTypeEnum;

  @ApiProperty()
  @IsDefined()
  @Type(() => AllMetaTypes)
  public allMeta!: AllMetaTypes;
}

The issue with this is that you have to manually map the relevant data from "allMeta" to the entity when saving.

If you're reading this a long time after publishing date it's worth checking if the typescript-fetch generator has already been updated.

Conclusion

Martin Fowler's serialized LOB method is a nice way to handle meta data. Postgres provides us the jsonb format for storing json easily. There's no reason you should lock yourself into relational data only.

Keep this method in mind the next time you have user provided properties or discriminated meta data in your business domain.