1. sbt-datatype

sbt-datatype 

sbt-datatype is a code generation library and an sbt autoplugin that generates growable datatypes and helps developers avoid breakage of binary compatibility.

Unlike standard Scala case classes, the datatypes (or pseudo case classes) generated by this library allow the developer to add new fields to the defined datatypes without breaking binary compatibility, while offering (almost) the same functionality as plain case classes. The only difference is that datatype doesn’t generate unapply or copy methods, because they would break binary compatibility.

In addition, sbt-datatype is also able to generate JSON codec for sjson-new, which can work against various JSON backends.

Our plugin takes as input a datatype schema in the form of a JSON object, whose format is based on the format defined by Apache Avro, and generates the corresponding code in Java or Scala along with the boilerplate code that will allow the generated classes to remain binary-compatible with previous versions of the datatype.

The source code of the library and autoplugin can be found on GitHub.

Using the plugin 

To enable the plugin for your build, put the following line in project/datatype.sbt:

addSbtPlugin("org.scala-sbt" % "sbt-datatype" % "0.2.2")

Your datatype definitions should be placed by default in src/main/datatype and src/test/datatype. Here’s how your build should be configured:

lazy val library = (project in file("library"))
  .enablePlugins(DatatypePlugin)
  .settings(
    name := "foo library",
  )

Datatype schema 

Datatype is able to generate three kinds of types:

  1. Records
  2. Interfaces
  3. Enums

Records 

Records are mapped to Java or Scala classes, corresponding to the standard case classes in Scala.

{
  "types": [
    {
      "name": "Person",
      "type": "record",
      "target": "Scala",
      "fields": [
        {
          "name": "name",
          "type": "String"
        },
        {
          "name": "age",
          "type": "int"
        }
      ]
    }
  ]
}

This schema will produce the following Scala class:

final class Person(
  val name: String,
  val age: Int) extends Serializable {
  override def equals(o: Any): Boolean = o match {
    case x: Person => (this.name == x.name) && (this.age == x.age)
    case _ => false
  }
  override def hashCode: Int = {
    37 * (37 * (17 + name.##) + age.##)
  }
  override def toString: String = {
    "Person(" + name + ", " + age + ")"
  }
  private[this] def copy(name: String = name, age: Int = age): Person = {
    new Person(name, age)
  }
  def withName(name: String): Person = {
    copy(name = name)
  }
  def withAge(age: Int): Person = {
    copy(age = age)
  }
}
object Person {
  def apply(name: String, age: Int): Person = new Person(name, age)
}

Or the following Java code (after changing the target property to "Java"):

public final class Person implements java.io.Serializable {
    private String name;
    private int age;
    public Person(String _name, int _age) {
        super();
        name = _name;
        age = _age;
    }
    public String name() {
        return this.name;
    }
    public int age() {
        return this.age;
    }
    public boolean equals(Object obj) {
        if (this == obj) {
            return true;
        } else if (!(obj instanceof Person)) {
            return false;
        } else {
            Person o = (Person)obj;
            return name().equals(o.name()) && (age() == o.age());
        }
    }
    public int hashCode() {
        return 37 * (37 * (17 + name().hashCode()) + (new Integer(age())).hashCode());
    }
    public String toString() {
        return "Person("  + "name: " + name() + ", " + "age: " + age() + ")";
    }
}

Interfaces 

Interfaces are mapped to Java abstract classes or Scala abstract classes. They can be extended by other interfaces or records.

{
  "types": [
    {
      "name": "Greeting",
      "namespace": "com.example",
      "target": "Scala",
      "type": "interface",
      "fields": [
        {
          "name": "message",
          "type": "String"
        }
      ],
      "types": [
        {
          "name": "SimpleGreeting",
          "namespace": "com.example",
          "target": "Scala",
          "type": "record"
        }
      ]
    }
  ]
}

This generates abstract class named Greeting and a class named SimpleGreeting that extends Greeting.

In addition, interfaces can define messages, which generates abstract method declarations.

{
  "types": [
    {
      "name": "FooService",
      "target": "Scala",
      "type": "interface",
      "messages": [
        {
          "name": "doSomething",
          "response": "int*",
          "request": [
            {
              "name": "arg0",
              "type": "int*",
              "doc": [
                "The first argument of the message.",
              ]
            }
          ]
        }
      ]
    }
  ]
}

Enums 

Enums are mapped to Java enumerations or Scala case objects.

{
  "types": [
    {
      "name": "Weekdays",
      "type": "enum",
      "target": "Java",
      "symbols": [
        "Monday", "Tuesday", "Wednesday", "Thursday",
        "Friday", "Saturday", "Sunday"
      ]
    }
  ]
}

This schema will generate the following Java code:

public enum Weekdays {
    Monday,
    Tuesday,
    Wednesday,
    Thursday,
    Friday,
    Saturday,
    Sunday
}

Or the following Scala code (after changing the target property to):

sealed abstract class Weekdays extends Serializable
object Weekdays {
  case object Monday extends Weekdays
  case object Tuesday extends Weekdays
  case object Wednesday extends Weekdays
  case object Thursday extends Weekdays
  case object Friday extends Weekdays
  case object Saturday extends Weekdays
  case object Sunday extends Weekdays
}

Using datatype to retain binary compatibility 

By using the since and default parameters, it is possible to grow existing datatypes while remaining binary compatible with classes that have been compiled against an earlier version of your datatype definition.

Consider the following initial version of a datatype:

{
  "types": [
    {
      "name": "Greeting",
      "type": "record",
      "target": "Scala",
      "fields": [
        {
          "name": "message",
          "type": "String"
        }
      ]
    }
  ]
}

The generated code could be used in a Scala program using the following code:

val greeting = Greeting("hello")

Imagine now that you would like to extend your datatype to include a date to the Greetings. The datatype can be modified accordingly:

{
  "types": [
    {
      "name": "Greeting",
      "type": "record",
      "target": "Scala",
      "fields": [
        {
          "name": "message",
          "type": "String"
        },
        {
          "name": "date",
          "type": "java.util.Date"
        }
      ]
    }
  ]
}

Unfortunately, the code that used Greeting would no longer compile, and classes that have been compiled against the previous version of the datatype would crash with a NoSuchMethodError.

To circumvent this problem and allow you to grow your datatypes, it is possible to indicate the version since the field exists and a default value in the datatype definition:

{
  "types": [
    {
      "name": "Greeting",
      "type": "record",
      "target": "Scala",
      "fields": [
        {
          "name": "message",
          "type": "String"
        },
        {
          "name": "date",
          "type": "java.util.Date",
          "since": "0.2.0",
          "default": "new java.util.Date()"
        }
      ]
    }
  ]
}

Now the code that was compiled against previous definitions of the datatype will still run.

JSON codec generation 

Adding JsonCodecPlugin to the subproject will generate sjson-new JSON codes for the datatypes.

lazy val root = (project in file("."))
  .enablePlugins(DatatypePlugin, JsonCodecPlugin)
  .settings(
    scalaVersion := "2.11.8",
    libraryDependencies += "com.eed3si9n" %% "sjson-new-scalajson" % "0.4.1"
  )

codecNamespace can be used to specify the package name for the codecs.

{
  "codecNamespace": "com.example.codec",
  "fullCodec": "CustomJsonProtocol",
  "types": [
    {
      "name": "Person",
      "namespace": "com.example",
      "type": "record",
      "target": "Scala",
      "fields": [
        {
          "name": "name",
          "type": "String"
        },
        {
          "name": "age",
          "type": "int"
        }
      ]
    }
  ]
}

JsonFormat traits will be generated under com.example.codec package, along with a full codec named CustomJsonProtocol that mixes in all the traits.

scala> import sjsonnew.support.scalajson.unsafe.{ Converter, CompactPrinter, Parser }
import sjsonnew.support.scalajson.unsafe.{Converter, CompactPrinter, Parser}

scala> import com.example.codec.CustomJsonProtocol._
import com.example.codec.CustomJsonProtocol._

scala> import com.example.Person
import com.example.Person

scala> val p = Person("Bob", 20)
p: com.example.Person = Person(Bob, 20)

scala> val j = Converter.toJsonUnsafe(p)
j: scala.json.ast.unsafe.JValue = JObject([Lscala.json.ast.unsafe.JField;@6731ad72)

scala> val s = CompactPrinter(j)
s: String = {"name":"Bob","age":20}

scala> val x = Parser.parseUnsafe(s)
x: scala.json.ast.unsafe.JValue = JObject([Lscala.json.ast.unsafe.JField;@7331f7f8)

scala> val q = Converter.fromJsonUnsafe[Person](x)
q: com.example.Person = Person(Bob, 20)

scala> assert(p == q)

Existing parameters for protocols, records, etc. 

All the elements of the schema definition accept a number of parameters that will influence the generated code. These parameters are not available for every node of the schema. Please refer to the syntax summary to see whether a parameters can be defined for a node.

name

This parameter defines the name of a field, record, field, etc.

target

This parameter determines whether the code will be generated in Java or Scala.

namespace

This parameter exists only for Definitions. It determines the package in which the code will be generated.

doc

The Javadoc that will accompany the generated element.

fields

For a protocol or a record only, it describes all the fields that compose the generated entity.

types

For a protocol, it defines the child protocols and records that extend it.

For an enumeration, it defines the values of the enumeration.

since

This parameter exists for fields only. It indicates the version in which the field has been added to its parent protocol or record.

When this parameter is defined, default must also be defined.

default

This parameter exists for fields only. It indicates what the default value should be for this field, in case it is used by a class that has been compiled against an earlier version of this datatype.

It must contain an expression which is valid in the target language of the parent protocol or record.

type for fields

It indicates what is the underlying type of the field.

Always use the type that you want to see in Scala. For instance, if your field will contain an integer value, use Int rather than Java’s int. datatype will automatically use Java’s primitive types if they are available.

For non-primitive types, it is recommended to write the fully-qualified type.

type for other definitions

It simply indicates the kind of entity that you want to generate: protocol, record or enumeration.

Settings 

This location can be changed by setting a new location in your build definition:

datatypeSource in generateDatatypes := file("some/location")

The plugin exposes other settings for Scala code generation:

  1. Compile / generateDatatypes / datatypeScalaFileNames This setting accepts a function Definition => File which will determine the filename for every generated Scala definition.
  2. Compile / generateDatatypes / datatypeScalaSealInterfaces This setting accepts a boolean value, and will determine whether interfaces should be sealed or not.

Syntax summary 

Schema           := {   "types": [ Definition* ]
                     (, "codecNamespace": string constant)?
                     (, "fullCodec": string constant)? }

Definition       := Record | Interface | Enumeration

Record           := {   "name": ID,
                        "type": "record",
                        "target": ("Scala" | "Java")
                     (, "namespace": string constant)?
                     (, "doc": string constant)?
                     (, "fields": [ Field* ])? }

Interface        := {   "name": ID,
                        "type": "interface",
                        "target": ("Scala" | "Java")
                     (, "namespace": string constant)?
                     (, "doc": string constant)?
                     (, "fields": [ Field* ])?
                     (, "messages": [ Message* ])?
                     (, "types": [ Definition* ])? }

Enumeration      := {   "name": ID,
                        "type": "enum",
                        "target": ("Scala" | "Java")
                     (, "namespace": string constant)?
                     (, "doc": string constant)?
                     (, "symbols": [ Symbol* ])? }

Symbol           := ID
                  | {   "name": ID
                     (, "doc": string constant)? }

Field            := {   "name": ID,
                        "type": ID
                     (, "doc": string constant)?
                     (, "since": version number string)?
                     (, "default": string constant)? }

Message          := {   "name": ID,
                        "response": ID
                     (, "request": [ Request* ])?
                     (, "doc": string constant)? }

Request          := {   "name": ID,
                        "type": ID
                     (, "doc": string constant)? }