STANWOOD insights

Save My Ass: Benchmark of JSON Deserializers on Android

Feb 8, 2019 10:51:48 AM / by Milosz Moczkowski

I think it’s safe to assume that the vast majority of apps has some kind of connectivity. Whether it’s content, tracking or remote config, most apps exchange some bits and bytes with the outer world. Once your app receives its desired data, it needs to serialize it somehow. Most REST APIs I know use Json. It’s lightweight, simple, elegant and easier to parse than XML. In cases where content is loaded exclusively remotely, parsing can become your optimization bottleneck. At stanwood, one of our white label apps is just like that. You swap the endpoints, change some colors and you’ve got yourself a custom news app.

 

json_deserializers_blog_pic_1

 

Our default json parse is the good ol’ Gson. But since we’re all a bunch of nerds here at stanwood, we’re always on the lookout for exciting stuff. Recently, a new, experimental feature has been announced for Kotlin: serialization. I thought it’s the best occasion to reevaluate today’s Android json scene and take the most popular libraries for a spin.


The benchmark


Test candidates

Along with Kotlin’s new feature, I selected three other popular json parsing libraries used on Android:

1. kotlinx.serialization 0.9.0
2. moshi 1.8.0
3. gson 2.8.5
4. jackson 2.9.7

 

Test data

I have downloaded three sample .json files from https://jsonplaceholder.typicode.com (thanks a lot!) that I used as test cases:

 

json_deserializers_blog_pic_2

Test cases

 

Test environment

json_deserializers_blog_pic_3

Benchmark app

 

I made a test app that used all four libraries to deserialize all three test cases. I asked my friends at stanwood to download it. To make things simple, the app automatically uploaded anonymized results to Firebase so that I could later analyze them.

1. Each test case was run 640 times.
2. 1920 results were gathered.
3. Data from 14 unique devices was collected.
4. 8 versions of Android system were used.

 

Evaluation

Execution time was measured with Kotlin’s stdlib measureTimeMillis:

json_deserializers_blog_pic_4

 

Results

Below is the average deserialization time for all test cases, devices and operating systems. We can immediately see a bunch of interesting observations.
json_deserializers_blog_pic_5

Combined average execution time for all test cases

 

Observation 1: Jackson sucks

It’s hard not to notice the big spike in the middle of the chart. While the rest of the libraries are all in the same range, Jackson was the only one that visibly stood out from the rest. Although my test data was quite simple, I initially suspected that this might have been influenced by some kind of edge case, Jackson was having a problem with.

json_deserializers_blog_pic_6

Average execution time per test case

 

That hypothesis proved to be false. In every test case, Jackson was consistently worse than any other tested libraries. This led me to another suspicion: What if Jackson is only slower on certain kinds of Android versions? What if it works fine on Oreo but struggles on Lollipop?

json_deserializers_blog_pic_7

Combined average test execution time per Android version

 

Turns out I was almost right. The gap between Jackson and the rest is huge on 4.1.2. It get’s smaller as we approach Android Pie, but it doesn’t disappear completely. It’s still very visible. Not a single combination of device, OS and test case helped Jackson.


Observation 2: Kotlinx.serialization, Gson and Moshi have all similar performance

Once we get rid of Jackson, we instantaneously arrive to another conclusion, and it’s something I didn’t expect: The rest of the libraries are almost equally fast.

json_deserializers_blog_pic_8

Combined average test execution time per Android version (without Jackson)

 

The difference is quite substantial on older Androids but it quickly diminishes as soon as we get closer to Android 9. Interesting to see is that there is no clear winner, though it’s important to note that Gson struggled a bit with the nested data test case.


Observation 3: Kotlinx.serialization is the fastest! (kinda)

Overall, I am very happy with the result. Although kotlinx.serialization is still an experimental feature, it can already compete with industry hits like Gson. But even though it scored best, I’m very reluctant to call it the fastest. Keep in mind that the test sample was quite small and the difference between the results were rather small as well (not talking to you Jackson). If I’d used more devices, the results might have been different.


Conclusion

Kotlinx.serialization sure looks promising! And as it approaches the 1.0 mark, I’m sure lots of people will reconsider their default json parsing library choice. As a bonus, check out this kotlinx.serialization converter for Retrofit!

 

Thanks to Sven Bendel and Karin Kleist.



manage cookies