PufferDB

Additional

Language
Kotlin
Version
1.1.1 (May 21, 2020)
Created
Apr 2, 2019
Updated
Mar 2, 2021 (Retired)
Owner
Adriel Café (adrielcafe)
Contributor
Adriel Café (adrielcafe)
1
Activity
Badge
Generate
Download
Source code

PufferDB

PufferDB is a ⚡ key-value storage powered by Protocol Buffers (aka Protobuf) and Coroutines.

The purpose of this library is to provide an efficient, reliable and Android independent storage.

Why Android independent? The SharedPreferences and many great third-party libraries (like Paper and MMKV) requires the Android Context to work. But if you are like me and want a kotlin-only data module (following the principles of Clean Architecture), this library is for you!

This project started as a library module in one of my personal projects, but I decided to open source it and add more features for general use. Hope you like!

About Protobuf

Protocol Buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. Compared to JSON, Protobuf files are smaller and faster to read/write because the data is stored in an efficient binary format.

Features

Supported types

So far, PufferDB supports the following types:

  • Double and List<Double>
  • Float and List<Float>
  • Int and List<Int>
  • Long and List<Long>
  • Boolean and List<Boolean>
  • String and List<String>

Getting Started

Import to your project

  1. Add the JitPack repository in your root build.gradle at the end of repositories:
allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}
  1. Next, add the desired dependencies to your module:
dependencies {
    // Core library
    implementation "com.github.adrielcafe.pufferdb:core:$currentVersion"

    // Android helper
    implementation "com.github.adrielcafe.pufferdb:android:$currentVersion"

    // Coroutines wrapper
    implementation "com.github.adrielcafe.pufferdb:coroutines:$currentVersion"

    // RxJava wrapper
    implementation "com.github.adrielcafe.pufferdb:rxjava:$currentVersion"
}

Current version:

Platform compatibility

core android coroutines rxjava
Android
JVM

Core

As the name suggests, Core is a standalone module and all other modules depends on it.

To create a new Puffer instance you must tell which file to use.

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile)
// or
val puffer = PufferDB.with(pufferFile, myCoroutineScope, myCoroutineDispatcher)

If you are on Android, I recommend to use the Context.filesDir as the parent folder. If you want to save in the external storage remember to ask for write permission first.

Its API is similar to SharedPreferences:

puffer.apply {
    val myValue = get<String>("myKey")
    val myValueWithDefault = get("myKey", "defaultValue")
    
    put("myOtherKey", 123)

    getKeys().forEach { key ->
        // ...
    }

    if(contains("myKey")){
        // ...
    }

    remove("myOtherKey")

    removeAll()
}

But unlike SharedPreferences, there's no apply() or commit(). Changes are saved asynchronously every time a write operation (put(), remove() and removeAll()) happens.

Threading

PufferDB uses a ConcurrentHashMap to manage a thread-safe in-memory cache for fast read and write operations.

Changes are saved asynchronously with the help of a StateFlow (to save the most recent state in a race condition) and Mutex locker (to prevent simultaneous writes).

It is possible to run the API methods on the Android Main Thread, but you should avoid that. You can use one of the wrapper modules or built in extension functions for that (listed below).

Android

The Android module contains an AndroidPufferDB helper class:

class MyApp : Application() {

    override fun onCreate() {
        super.onCreate()
        // Init the PufferDB when your app starts
        AndroidPufferDB.init(this)
    }
}

// Now you can use it anywhere in your app
class MyActivity : AppCompatActivity() {

    // Returns a default Puffer instance, the file is located on Context.filesDir
    val corePuffer = AndroidPufferDB.withDefault()

    // Returns a File that should be used to create a Puffer instance
    val pufferFile = AndroidPufferDB.getInternalFile("my.db")
    val coroutinePuffer = CoroutinePufferDB.with(pufferFile)
}

Coroutines

The Coroutines module contains a CoroutinePufferDB wrapper class and some useful extension functions:

val pufferFile = File("path/to/puffer/file")
val puffer = CoroutinePufferDB.with(pufferFile)

// All methods are suspend functions that runs on Dispatchers.IO context
launch {
    puffer.apply {
        val myValue = get<String>("myKey")
        val myValueWithDefault = get("myKey", "defaultValue")
        
        put("myOtherKey", 123)

        getKeys().forEach { key ->
            // ...
        }

        if(contains("myKey")){
            // ...
        }

        remove("myOtherKey")

        removeAll()
    }
}

If you don't want to use this wrapper class, there's some built in extension functions that can be used with the Core module:

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB

launch {
    puffer.apply {
        val myValue = getSuspend<String>("myKey")

        val myValue = getAsync<String>("myKey").await()
        
        // You can use your own coroutine scope and dispatcher
        putSuspend("myOtherKey", 123, myCoroutineScope, myCoroutineDispatcher)

        putAsync("myOtherKey", 123, myActivityScope).await()
    }
}

RxJava

The RxJava module contains a RxPufferDB wrapper class and some useful extension functions:

val pufferFile = File("path/to/puffer/file")
val puffer = RxPufferDB.with(pufferFile)

puffer.apply {
    // Some methods returns Single<T>...
    get<String>("myKey") // OR get("myKey", "defaultValue")
        .subscribeOn(Schedulers.io())
        .observeOn(AndroidSchedulers.mainThread())
        .subscribe { myValue ->
            // ...
        }

    // ... And others returns Completable
    put("myOtherKey", 123)
        // ...
        .subscribe {
            // ...
        }

    getKeys()
        // ...
        .subscribe { keys ->
            // ...
        }

    contains("myKey")
        // ...
        .subscribe { contains ->
            // ...
        }

    remove("myOtherKey")
        // ...
        .subscribe {
            // ...
        }

    removeAll()
        // ...
        .subscribe {
            // ...
        }
}

Like the Coroutines module, the RxJava module also provides some useful built in extension functions that can be used with the Core module:

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB

puffer.apply {
    val myValue = getSingle<String>("myKey").blockingGet()

    putCompletable("myOtherKey", 123).blockingAwait()

    getKeysObservable().blockingSubscribe { keys ->
        // ...
    }
}

Benchmark

Write & Read

Write 1k strings (ms) Read 1k strings (ms)
PufferDB 20 5
SharedPreferences 278 7
MMKV 13 8
Paper 818 169
Binary Prefs 121 9
Hawk 15183 207
Write 100k strings (ms) Read 100k strings (ms)
PufferDB 259 32
SharedPreferences ???? ????
MMKV 871 516
Paper ???? ????
Binary Prefs 1082 101
Hawk ???? ????

File size

1k strings (kb)
PufferDB 25
SharedPreferences 20
MMKV 40
Paper 61
Binary Prefs 53
Hawk 27
100k strings (kb)
PufferDB 2.907
SharedPreferences ????
MMKV 4.104
Paper ????
Binary Prefs 5.175
Hawk ????

Tested on Moto Z2 Plus

You can run the Benchmark through the sample app.