Posts Circe
Post
Cancel

Circe

on Json

play-json enabled us to class-up json (deserializing it from string to a case class) and to serialize json with relative ease.

Circe (pronounced SUR-see) is yet another JSON library for Scala.
It enables automatic derivation of encoders and decoders, generic derivation via discriminators, schema validation, refined support, json literals and more.

We will demonstrate automatic derivation and discriminators.

The current situation

First, lets play around with play-json.
We will create a Dish case class and try to serialize it without an implicit value of Writes[Dish] in scope:

1
2
3
4
5
6
7
8
9
10

case class Dish(name: String, description: String, price: Double)

val dish = Dish("Ice Cream", "Vanilla Ice Cream", 6)

import play.api.libs.json.{Json, Writes}

// the next line is commented out on purpose:
//implicit val dishWrites: Writes[Dish] = Json.writes[Dish]

Json.toJson(dish)

The last line, that tries to serialize the ice-cream dish to json, will fail to compile with the following error:

No Json serializer found for type Dish

The good news is that play-json makes sure at compile-time that the type of the object we are trying to serialize (or deserialize) has a serializer (or a deserializer), whereas in Jackson (a java library for json) you will find it out on runtime.
Obviously, we would like to find out all error as soon as possible (i.e at compile time) since this eliminates bugs.
However, since Dish is a product of a String, a String and an Int, all of which have a serializer, you would expect that the compiler will just understand how to create the serializer for you.

As another example, we will create a FancyDish, product of Dish and karatGold as an Int, and we will try to create a serializer for FancyDish without creating a serializer for Dish:

1
2
3
4
5
6
7
8
9
10
11
12

case class Dish(name: String, description: String, price: Double)
case class FancyDish(karatGold: Int, dish: Dish)

val dish = Dish("Ice Cream", "Vanilla Ice Cream", 6)
val fancyDish = FancyDish(10, dish)

import play.api.libs.json.{Json, Writes}

/* Creating a serializer for FancyDish but not for Dish.
The Dish serializer is commented out on purpose */
//implicit val dishWrites: Writes[Dish] = Json.writes[Dish]
implicit val fancyDishWrites: Writes[FancyDish] = Json.writes[FancyDish]

The last line, that tries to create a serializer for FancyDish, will fail at compile-time because in order to create this serializer, play-json expects a serializer for Dish.
Again, you would expect that the compiler can derive a serializer for your class if each of its members:

  1. is a basic type that already has a serializer. or
  2. is another case classes that the compiler can derive a serializer for.

This means that we want the compiler to recurse and generate a serializer for Dish (and in general for all other types. consider a more complex hierarchy such as an Amazon package that has a sender and a recipient, and the sender has a contact info and an address and so on).

Circe

Let’s try the same example with circe.

Add Circe to our build

Add circe-generic-extras, circe-parser and play-circe to the build.sbt.
play-circe version should start with the same major+minor as our play version.
e.g, if project/plugins.sbt adds play with: addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.8.0"), then the play version is 2.8.0, so the compatible play-circe version should start with 28.

1
2
3
4
5
6

libraryDependencies ++= Seq(
  // more dependencies here
  "io.circe" %% "circe-generic-extras" % "0.12.2",
  "io.circe" %% "circe-parser" % "0.12.2",
  "com.dripower" %% "play-circe" % "2812.0", // compatible with Play 2.8.x
)

Import Changes in IntelliJ so it will download the dependencies for you and help you with code completion (it will also reload the sbt shell behind the scenes).

circe-generic-extras enables serialization with configurable options.
circe-parser allows using the parser to parse a string representing a json to a class.
play-circe is a simple, one file library that just adds play body parsers using circe.

Experiment with Circe in a scratch file

We will add the following code as a new scala scratch file in IntelliJ IDEA (cmd+shift+N -> Scala, or File -> New -> Scratch File -> Scala).
Scratch files let us play around with the code, including using our module’s class path (so we can use the dependencies we declared in libraryDependencies).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import io.circe.generic.extras.auto._
import io.circe.syntax._
import io.circe.parser
import io.circe.generic.extras.Configuration
import io.circe

// create a dish
case class Dish(name: String, description: String, price: Double)
val dish = Dish("Ice Cream", "Vanilla Ice Cream", 6)

implicit val customConfig: Configuration =
  Configuration.default.withDiscriminator("type")

// serialize to json 
val seralizedDish: String = dish.asJson.spaces2

// deserialize from json
val errorOrDish: Either[circe.Error, Dish] = parser.decode[Dish](
  """{
    |  "name" : "Eggs Benedict",
    |  "description" : "English muffin, poached egg and hollandaise sauce",
    |  "price" : 12
    |}
    |""".stripMargin)

We start by creating a dish, just like before.
Then we create a configuration for circe with default config, but add a discriminator (more on what to config - later).
This configuration value, which configures how to create encoders and decoders, is marked as implicit, and passed (indirectly) to circe methods that require an Encoder/Decoder.

Now, all we have to do in order to serialize an object to json, is to call .asJson on it.
The .asJson method is an extension method, added via import io.circe.syntax._.
Once we have a json, we can pretty-print it to a string using .spaces2 which adds two spaces before each key.
There are other printing methods, such as .spaces4, .spaces2SortKeys and so on.

To deserialize a json, we need to call parser.decode, which is a generic method.
We set the type parameter as the type of the class which is the target of the deserialization, which is Dish in our case, and pass a string representing a json as an argument.
The parsing may fail, since the string might not be a valid json, or be a valid json that does not match the structure of Dish, so therefore the result of the parsing is Either[circe.Error, Dish].

Note on Either[A, B]: this is an abstract type that has two inheriting concrete types: Left[A] or Right[B].
This type allows to represent the ability of a failure using the type system.
By convention, “Left is the wrong value, and Right is the right value”.

Automatic serialization of custom classes

We don’t need to create an encoder (or a decoder) for every custom type, and if we used complex types, the encoder would be derived automatically for us.
We get this with import io.circe.generic.extras.auto._, which enables auto-derivation of encoders and decoders using macros.
All we have to do to serialize and deserialize is use .asJson and parser.decode[OurClass](jsonString).

Sealed classes

As another example, lets say that we have the following sealed class:

1
2
3

sealed trait Event
case class Login (who: String) extends Event
case class Logout(who: String) extends Event

It is clear that serializing an event where Alice logged in and one where Alice logged out would result in the same json:

1

{ "who": "alice" }

Also, the json above can obviously be deserialized to a login instance, as well as a logout instance.

While this is sometimes ok, e.g serialized login events will always be sent to a Kafka login-topic, and serialized logout events will always be serialized to a Kafka logout-topic, sometimes we would like to have the ability to discriminate between the two types.

Luckily, we configured the derivation to use a type discriminator in the previous example.
Now we can write the following code:

1
2
3
4
5
6
7
8
9
10

sealed trait Event
case class Login (who: String) extends Event
case class Logout(who: String) extends Event

val login: Event = Login("alice")
val example1 = login.asJson.spaces2 // { "who" : "alice", "type" : "Login" }

val login2: Login = Login("alice")
val example2 = login2.asJson.spaces2 // { "who" : "alice" }
val example3 = (login2: Event).asJson.spaces2 // { "who" : "alice", "type" : "Login" }

When the static type of the val is the trait, then we will have a type discriminator field in the serialized json.
We configured the key of the discriminator in the configuration earlier as “type” by .withDiscriminator("type") (which is not some special reserved word. we can use any valid json key name, like “_type” or “_discriminator”).

This is why the resulted json in example1 is

1
2
3
4

{
  "who": "alice",
  "type": "Login"
}

When the static type is the concrete class, like Login in example2, then like the previous example - the json will not contain the discriminator, as expected.

We can coerce the serialization to use the encoder for Event instead of using the encoder for Login by assigning login2 to a new val of type Event, or by simply using a type ascription, login2: Event , like in example3.

Parsing json with discriminator is just as easy, for example:

1
2
3
4
5
6

val errorOrEvent: Either[circe.Error, Event] = parser.decode[Event](
  """{
    |  "who" : "alice",
    |  "type" : "Login"
    |}
    |""".stripMargin)

The example above results in Right(Login(alice)).

Deserializing requests in the controller with play-circe

After adding play-circe to the library dependencies, all we have to fo to migrate the DishController to circe is:

  1. Remove play-json imports (import play.api.libs.json.{Json, Reads, Writes})
  2. Add play-circe imports:
    1
2
3
4

    import play.api.libs.circe.Circe // the play-circe trait that adds play-body parsers that support circe
import io.circe.generic.extras.auto._ // automatic deriavtion of encoders and decoders
import io.circe.syntax._ // enables "obj.asJson" syntax for serialization
import io.circe.generic.extras.Configuration // configures the derivation of encoders and decoders
  3. Extend Circe by adding with Circe to the class definition, and add custom config for deriving the decoders:
    1
2
3
4
5
6
7

    class DishController(
  controllerComponents: ControllerComponents,
  dishLibrary: DishLibrary
) extends AbstractController(controllerComponents) with Circe {
     
  implicit val customConfig: Configuration =
    Configuration.default.withSnakeCaseMemberNames.withDiscriminator("type")

    This will provide the play body parsers that support circe.
    Notice that this time we added .withSnakeCaseMemberNames to the custom config.
    This is becase if we serialize login in:

    1
2

    case class Login(whoIs: String)
val login = Login("alice")

    We want the result to be { "who_is": "alice" }, and not { "whoIs": "alice" }.
    Try to serialize and desrialize with this configuration to experiment with this.

  4. Remove the two encoder instances for Writes[Dish] and the decoder instance for Reads[Dish].
  5. Replace the two occurrences of Json.toJson(something) with something.asJson
  6. Replace parse.json[Dish] with circe.json[Dish], or better - with circe.tolerantJson[Dish].
    The tollerant body parser does not force the user of the API to add a header of Content-Type: application/json.

You can now run the server by running run in the sbt-shell.
Invoking the dishes API (e.g by browsing to http://localhost:9000/dishes) will work as expected.

Note: play-json is a great library. keep using it if you don’t mind creating an encoder for each custom type in your model’s hierarchy, or don’t need the added benefits of circe.

circe json

Play modes

Macwire