This is a help article about Kanbani – a freeware task planner for Android

Sync

Synchronization & Collaboration

Unlike Trello, Kanbani is an offline-first app with on-demand synchronization (“sync” for short). This means that until you sync, your changes are only local and not visible outside of your device. Also, you can sync to your own server.

Even though there are many settings, other team members do not need to configure all of them by hand. Kanbani allows instant configuration by means of a QR code that encodes all the preferences and access information (including the secret). Simply send the code by e-mail, print and pin it on a wall or display it on one device and scan on another.

Sync can be also done without going online: sync to a local file, share it via Bluetooth, import on another device. This is particularly useful when changing Kanbani data with a custom tool (like capitalizing card titles) given that sync data is just JSON: sync (in Export mode) to a file, transform it and sync back to Kanbani to apply your changes. This doesn’t even require root!

Modes

Mode	Writes Remote	Sync Result
Import	No	Overrides local data with remote data discarding local modifications. Can be used to reset the device’s boards.
Export	Yes	Overrides remote data with local data discarding remote modifications. It’s similar to making a backup but not quite.
Sync	Yes	Merges local changes with remote data to avoid losing changes done both locally and remotely since last sync.

Sync vs. Backup

Kanbani has a separate option for doing occasional backups. Backups have different format and purpose than sync in the Export mode:

Backup aims at retaining as much information about a particular app installation as possible. This includes all created boards, preferences and other internal data.	Sync packs information about a particular board only, with only its hierarchical preferences.
Kanbani is internally using Realm for storing program data. Realm is a database similar to SQLite focused on run-time performance. It includes data that is redundant (indexes) and even private (remnants of permanently deleted cards).	Formats used in the sync are widely used and focus on serialization speed and small size.
Therefore, when you restore from a backup – you get almost identical set-up that you had before.	When you restore from a sync file, you get data for that board and some preferences but no other configuration (e.g. default Author).
Literally single tap backup and restoration. Automatic backups, enabled by default.	Needs to be told where, what and how to sync before it can be ran manually or automatically.
To restore a backup without overwriting old data, extract it anywhere into the device’s memory and change Data Path in the preferences. This will “disconnect” all of your previous data (preserving your boards) and switch to the one in the backup.	To “sync”, use the Sync dialog. It will always import data into your current database (changing its boards and preferences).

Users of Titanium Backup and similar tools will recognize that a “backup” here is very close to what such tools do (except it works without root).

Additionally, having two entirely different mechanisms lowers the risk of losing your data in case one of them fails due to bugs or other misgivings.

Durability

Kanbani protects against simultaneous sync and interrupts due to bad connectivity using the available transport’s features. Some transports are better than others but Kanbani is always able to detect data corruption in the remote data (thus preventing corrupting the local state on Sync and Import modes).

If corruption is detected, user is asked to ignore it or abort (unless the operation happens in background). This can be configured in sync profile preferences.

Below are some algorithms employed by Kanbani and used by some transports:

Resource locking prevents the server from allowing more than one client syncing a given board at once, as well as other clients reading partially synced (or failed) board. Few transports support this but those that do (WebDAV) are most reliable.
Atomic rename is taking advantage of the fact that the smallest data unit is a file. Data of a board is first written to a temporary file and then renamed (moved) to the final location. Other clients do not see this data so any error (bad network, out of disk space, etc.) reliably cancels the operation. However, it doesn’t prevent multiple clients from syncing the same board at the same time – data of only one of them will be recorded (but again, at least it will not be interleaved with other clients’ data).
Integrity hash is always used by all transports, including ones that support no other options. It only allows detecting corrupted data post-factum and doesn’t prevent concurrent access to the same board. Every data file ends on a digest (hash) of its contents that allows the received to compare the data it has read with the data that was originally written; if the data was not completely written, if it was written by multiple clients or if it was corrupted in transit or during storage – these two values will differ (details).

Internal Design

The smallest sync unit is a board represented by one file. Each board has a randomly generated unique identifier (UUID); unprotected sync files are named after that UUID while encrypted sync file names are hashed. This may lead to various implications depending on the capabilities of the sync transport in use but most of the time it means that clients wishing to sync some board have to wait until another client finishes syncing that particular board (but they can sync other boards meanwhile).

Merging Data

In Sync mode, Kanbani attempts to merge remote and local changes so that nothing gets lost. For this, it keeps track of fields changed since last sync (on both sides). Fields can be “virtual” – for example, for sync purposes deleting a card is like changing a special field.

If the change to a field was done only on one side – Kanbani copies that change to the other side without asking.
If the change was done on both sides but the value is identical on both local and remote – Kanbani does nothing. For example, if a card was deleted locally and on remote – it’s kept deleted silently; if its Title was changed to “foo” – it’s silently kept at that value.
Otherwise, if a field was changed on both sides to different values, Kanbani uses conflict resolution to figure the final value.

Conflict resolution is controlled by Conflict mode set in the sync profile:

Ask defers the choice to the user in each particular conflict. It’s not usable when sync is happens in background (non-interactively) and for that purpose it’s treated as Cancel.
Cancel is the only 100% safe choice. It refuses to resolve any conflicts, reverting changes done during the sync.
Take remote/local always resolve the conflict by copying the state of that side. For example, if a remote card was changed but the same local card was deleted, Take remote would un-delete the local card and fill it with properties (Title, etc.) from the remote side.
Prefer remote/local are similar to Take but before blindly copying remote or local state these apply a (slightly more) intelligent algorithm to resolve the situation. Right now, this makes a difference only for Description fields: their values (both remote and local) are combined together (e.g. remote “rem” and local “loc” produce something like “rem loc”). Other fields cannot be “intelligently resolved” and so they are overwritten just like with Take remote/local.

Changes done to the cards during conflict resolution are recorded as regular changes, i.e. as if the user has performed them manually. This allows the next sync to treat “resolved and modified” fields as changed ones and apply its own logic.

IDs & Encryption

Kanbani allows both unprotected and protected (encrypted) syncs which, as always, trade security for convenience and vice-versa.

Unprotected does not mean “public” – it still requires that another party knows a semi-secret sync ID (which is a random string by default). You can think of ID as a login/username and of unprotected sync – as an account with an empty password. Anyone who knows the ID can read and change your synced boards. Also, the operator of the server where your boards are synced can read and change the boards of everyone. If you are not the operator, you can still try learning the ID using a bruteforce attack (by syncing to every possible ID one after another) but generally it’s safe to say that a reasonably long ID is not prone to this threat in slow networks such as the Internet.

Protected sync data is using a secret string (“password”) in addition to the ID (“login”). If somebody (including the server operator) knows the ID but no password – they cannot read or change the data. Also, board file names are hashed and even the operator cannot tell which file corresponds to which board and learn the list of boards per user (no matter if he knows some board UUIDs or not). This mode is also much more resilient to bruteforce attacks (if using secret and ID generated by default then it’s safe to say that the Sun will blow up before one can find a working combination) and if the secret is leaked but board ID is unknown – the data may still be unrecoverable (although this is not guaranteed, see the design).

Kanbani allows multiple sync profiles and each profile may specify its own encryption settings. For example, a company’s server may require that all work-related boards are stored unprotected but it may also allow the employees’ private to store their private boards, which may be encrypted.

ID and secret may also be used to implement permission-based board access. A company-controlled server may use easy to remember IDs like “marketing” or “IT”, with or without secrets.

Board File Format

This section describes low level details. If using PHP, you can include our library for working with Kanbani data (used in Kanbani Web Viewer) and skip over this section.

Kanbani’s minimum supported Android version is 5 (Lollipop), or API level 21. This version only supports basic security and is the reason why the algorithms default to 128-bit AES and PBKDF2 with SHA-1.

First of all, Kanbani derives a master key from your sync secret and board ID. This key is different for every board even if they are part of the same profile (with the common secret). This is done using PBKDF2(SHA-1, 10,000 iterations, salt=board ID, password=sync secret). In Android/Java, use SecretKeyFactory with PBKDF2withHmacSHA1 (do not use PBKDF2withHmacSHA18BIT).

After that Kanbani derives 3 sub-keys used in different operations using HKDF(SHA-256, salt=board ID); the master key is not used to avoid interference between those operations on a common data (more info). Each operation’s key has a different “info string” to protect against its reuse in a different context.

To name an encrypted file, Kanbani calculates a HMAC(SHA-256, shared=HKDF(..., info=filename1), data=board ID) which results in 32 bytes, i.e. 64 hexadecimal characters. Unencrypted file is named after the board’s ID.

Encrypted stream consists of the following fields:

Version	1 byte	Data format version. Currently known are: 0 (unprotected sync), 1 (protected sync).
Digest algorithm	String terminated by \n	One of Android/Java standard algorithm names. For unprotected defaults to SHA-256, for protected – to HmacSHA256.
Encryption mode	String terminated by \n	Only present in protected streams. Standard Android/Java transformation, consisting of 3 /-separated parts: algo/mode/padding; each part has a standard name in Java. Taken from the sync profile; defaults to AES/GCM/NoPadding. If you cannot use GCM (it’s a relatively new mode) – use AES-128-CTR which support is universal.
Data	Until EOF excluding MAC	For unprotected this is a DEFLATE-compressed stream (InflaterInputStream in Android, see JSON) which is validated by the following digest. For protected – an IV, ciphertext (same compressed stream) and auth tag (if AES-GCM is used).
Digest	Determined by the algorithm	For unprotected ensures integrity of the preceding compressed data. For protected – integrity and authentication of encryption mode and data, generated as HMAC(Digest algorithm, HKDF(info=auth...)).

Unprotected stream is simple and can be generated and read by this PHP script (but again, we suggest using our PHP library in production):

<?php
$compressed = gzcompress("{aaaaaaaa}");
$synced = "\0SHA-256\n".$compressed.hash("sha256", $compressed, true);

list($head, $tail) = explode("\n", $synced, 2);
$version = $head[0];
if ($version === "\0") {
    $algo = str_replace("-", "", substr($head, 1));
    $hash = substr($tail, -strlen(hash($algo, "", true)));
    $compressed = substr($tail, 0, -strlen($hash));
    if (hash_equals(hash($algo, $compressed, true), $hash)) {
        echo gzuncompress($compressed);
    }
}

Protected stream is complex and requires several cryptography functions. Keys are derived using HKDF with “info string” set to (enc or auth) plus encryption mode plus IV (initialization vector used in the encryption).

To demonstrate this, we will again use PHP because it has all of them, but in Android/Java you’ll need to use a non-standard library for HKDF (see here).

<php
// --- Sync profile settings ---
$secret = "secr";
$hashAlgo = "SHA256";         // only used in data HMAC.
$encAlgo = "aes-128-ctr";   // GCM not supported in old PHP.
$encAlgoJava = "AES/CTR/PKCS5Padding";

$id = "boid";   // ID of the board being synced.

// --- Encrypting ---
$compressed = gzcompress("{aaaaaaaa}");
$masterKey = hash_pbkdf2("sha1", $secret, $id, 10000, 0, true);
$iv = openssl_random_pseudo_bytes(openssl_cipher_iv_length($encAlgo));
$authKey = hash_hkdf("sha256", $masterKey, 0, "auth".$encAlgoJava.$iv, $id);
// By default, if 0 is given then SHA-256 produces 32 bytes which is enough for
// our current $encAlgo, AES-128 (16 bytes) and for 192/256 (24/32).
// You should request a larger output instead of giving 0 or 32 to avoid
// zero-padding the $encKey if your $encAlgo is different.
$encKey  = hash_hkdf("sha256", $masterKey, 0, "enc".$encAlgoJava.$iv, $id);
$encrypted = openssl_encrypt($compressed, $encAlgo, $encKey, OPENSSL_RAW_DATA, $iv);
// Plus GCM tag in the end (not used here).
$authenticatedData = $encAlgoJava."\n".$iv.$encrypted;
$hash = hash_hmac($hashAlgo, $authenticatedData, $authKey, true);
$synced = "\1"."Hmac$hashAlgo"."\n".$authenticatedData.$hash;

// --- Decrypting ---
list($head, $tail) = explode("\n", $synced, 2);
$version = $head[0];
if ($version === "\1") {
    $hashAlgo = substr($head, 1 + strlen("Hmac"));
    $hash = substr($tail, -strlen(hash($hashAlgo, "", true)));
    $authenticatedData = substr($tail, 0, -strlen($hash));
    list($encAlgoJava, $tail) = explode("\n", $authenticatedData, 2);
    $encAlgo = javaTransformationToPHP($encAlgoJava);   // you can figure this.
    $iv = substr($tail, 0, openssl_cipher_iv_length($encAlgo));
    $encrypted = substr($tail, strlen($iv));
    // Key derivation is just like when encrypting.
    $masterKey = hash_pbkdf2("sha1", $secret, $id, 10000, 0, true);
    $authKey = hash_hkdf("sha256", $masterKey, 0, "auth".$encAlgoJava.$iv, $id);
    $encKey  = hash_hkdf("sha256", $masterKey, 0, "enc".$encAlgoJava.$iv, $id);
    if (hash_equals(hash_hmac($hashAlgo, $authenticatedData, $authKey, true), $hash)) {
        echo gzuncompress(openssl_decrypt($encrypted, $encAlgo, $encKey, OPENSSL_RAW_DATA, $iv));
    }
}

If implementing your own encryption/decryption (perhaps when creating a compatible web viewer) note the following things:

Use a new IV for every new encrypted stream (board file), obtained from a crypto-strong PRNG such as openssl_random_pseudo_bytes in PHP (not rand()).
When decrypting, check HMAC before doing anything on the encrypted data (apart from extracting encryption method, IV and GCM tag). If HMAC is different – discard the data and abort. The data may have been corrupted on purpose.
After successful HMAC verification and decryption prior to any other processing check the ID of the board in the decrypted data. If it mismatches the ID you were expecting (ID used in the keys) – discard the data and abort. This may happen if somebody swapped data of two different encrypted board files.
Stop on an unknown version, algorithm or mode. In particular, data version (the first byte) is not a boolean flag but a number; new formats (coded 2, 3, etc.) may appear in the future, e.g. to allow uncompressed data.
If using an encryption mode which requires padding (like AES-CBC) – do not strip the padding (by specifying “ZERO_PADDING” or an equivalent flag). This is insecure!

JSON Format

Regardless of the container format, sync data is serialized into a JSON. Most fields are self-explanatory; notes on others follow:

client_version
sync_version	Internal numbers. When reading, fail on invalid combination. When writing, take from the current app installation verbatim.
id	UUID – universally unique identifier as a string of any format. Two boards/lists/cards with the same id are considered copies (versions) of one object. If generating, eliminate the chance of ever producing the same ID string on multiple devices.
custom	Container for arbitrary data, treated as a blackbox string by Kanbani. Usually custom is a JSON that expands to an object where you attach fields to any card/list/board that must be preserved after sync. Example: `{"custom": "{\"my_field\": \"foo\", ...}"}`.
field_history	Used in conflict resolution; hold list of changed properties and card moves between two sync sessions. When reading, disregard. When changing any property, add its name under -1 key (indicating a local-only change). Kanbani sets them empty after a successful sync.
move_history

Date/time is stored as an Unix timestamp with milliseconds (number of ms since 01.01.1970).