Golang Interfaces and Json Marshaling

With Golang maturing, both its limits in flexibility and performance become more apparent. Despite this, it is still quite popular (in top ten of fastest growing languages on Github, as of 2019), due to its simplicity. This can be a blessing or a curse, as shown in this post.

Recently I encountered the following problem: imagine you have an interface and want to create a new type that conforms to this interface. Let the interface be defined as

type Animal interface {
    json.Marshaler
    json.Unmarshaler
    Color() string
}

This way, you force a developer to make it possible to convert any Animal to and from json, which you might want for purposes of storage or transmission. Most Animals might have a simple structure like


type Dog struct {
    Name string
    ColorOfFur string
}

and methods like


func (d *Dog) ComposeName(title, firstName, lastName string) {
    d.Name = title + " " + firstName + " " + lastName
}

func (d *Dog) Color() string {
    return d.ColorOfFur
}

Here, we chose pointer receivers since we do not want to copy our Dog everywhere we use it. Ordinarily, the Dog struct would automatically be converted to JSON using the built-in Marshaler for structs. However, if we try to use a *Dog as Animal, the compiler will complain:


func main() {
	doggo := Dog{"Cooper", "Purple"}
	var someAnimal Animal
	someAnimal = &doggo

	buf, err := json.Marshal(someAnimal)

	if err != nil {
		log.Fatal("Error marshaling:", err)
	}

	fmt.Println("Our Animal:", string(buf))

	var jsonDog Dog
	err = json.Unmarshal(buf, &jsonDog)

	if err != nil {
		log.Fatal("Error unmarshaling:", err)
	}

	fmt.Println("Our new jsonDog:", jsonDog)
}

./test.go:4:13: cannot use &doggo (type *Dog) as type Animal in assignment:
    *Dog does not implement Animal (missing MarshalJSON method)

Of course, the compiler is correct: our Animal interface forces us to implement the MarshalJSON and UnmarshalJSON functions for *Dog. However, for our simple Dog we would like to use the default method Golang uses when these functions are not defined for a struct. This turns out to be tricky.

The MarshalJSON function is surprisingly simple:


func (d *Dog) MarshalJSON() ([]byte, error) {
    return json.Marshal(*d)
}

Why does this work? Well, the type of the argument to json.Marshal is Dog, which is a simple struct and thus Golang will use the default conversion. The UnmarshalJSON function is a bit more complicated. If we try and use the same trick,


func (d *Dog) UnmarshalJSON(buf []byte) error {
    return json.Unmarshal(buf, &d)
}

we will get an output like

go run test.go                                                                                           
Our Animal:  {"Name":"Cooper","ColorOfFur":"Purple"}                                                                               
runtime: goroutine stack exceeds 1000000000-byte limit                                                                             
fatal error: stack overflow                                                                                                        
                                                                                                                                   
runtime stack:                                                                                                                     
runtime.throw(0x4f2173, 0xe)                                                                                                       
        /usr/lib64/go/1.11/src/runtime/panic.go:608 +0x72                                                                          
runtime.newstack()                                                                                                                 
        /usr/lib64/go/1.11/src/runtime/stack.go:1008 +0x729
runtime.morestack()
        /usr/lib64/go/1.11/src/runtime/asm_amd64.s:429 +0x8f

We get a stack overflow since our UnmarshalJSON implementation calls json.Unmarshal, which checks whether a custon unmarshaler exists for the type passed to it. This is unfortunate (and in my humble opinion a bit inconsistent, albeit convenient in most cases), because the compiler finds our unmarshaler and proceeds to call that, thus creating a loop.

One way to get around this is to use an alias type. Thus, we change our UnmarshalJSON implementation to


func (d *Dog) UnmarshalJSON(buf []byte) error {
    type DogAlias Dog
    var tmpDog DogAlias
    err := json.Unmarshal(buf, &tmpDog)
    if err != nil {
        // leave d untouched
        return err
    }
    *d = Dog(tmpDog)
    return nil
}

While this looks like a lot of boiler plate code, it works well and avoids assigning every field of the struct manually (which would lead to problems when the struct is changed). The complete file can be downloaded here.

On a side note, there is no easy way to determine the original type of Animal from a json-marshaled one. It is therefore not possible to do something like


var jsonAnimal Animal
err = json.Unmarshal(buf, &jsonAnimal)
if err != nil {
    log.Fatal("Error unmarshaling:", err)
}

if _, ok := jsonAnimal.(*Dog); ok {
    fmt.Println("Animal is *Dog")
} else {
    fmt.Println("Animal is not *Dog")
}

While this compiles, unmarshaling fails. This is not the fault of Golang - it is not easy to identify the type from a json buffer; some types might even use the same field names! Usually, an indication fo the type is added to make it possible to unmarshal into a concrete Animal (e.g. a 'type:"Dog"'). This is one more reason why we might like custom marshalers and unmarshalers. For a simple way to add an extra field like this without copying all fields of the original type, see e.g. this blog post.

Personally, I am not 100% sure that this is the optimal solution. Maybe it is more idiomatic to omit the json requirements in the interface? Certainly, at some point we would notice if some type of Animal does not implement a proper marshaler. Maybe we would notice, maybe not. Even though Golang supports unit tests, it is unlikely that such a unit test would test every type an interface can assume. And programmers who forget to implement a marshaler/unmarshaler will certainly also forget to add tests for it. If you think you have a better solution or that my solution is flawed or suboptimal, please send improvements to birki@21er.org.

© 2010-2015 Stefan Birgmeier
birki@21er.org