Import, origin and sharing

Import: what is written, what never is

Importing a match, a position or another database adds what is missing; it does not replace what is already there.

A position is never duplicated. It is its identity — checkers, cube, dice, score — that recognises it, never the file it came from: the same position met in two matches stays a single row.
Importing a database reads back everything an export writes. Positions, with their analyses and comments, but also matches (games and moves included), tournaments with their Direction and their Encounter, transcription drafts, aliases, collections, lessons and Anki decks: see Merge a database. Nothing is overwritten: a match already present is not copied again, a tournament of the same name receives the matches without changing the event’s date (its last-modified date advances, since it receives new matches), a tournament that already has its Direction keeps it and stays outside the imported Encounter, a match already filed in another tournament or another slot of the table stays there, and what the database already decided stays decided by it. The tournaments, Directions, Encounters, transcription drafts and aliases the import creates keep their original dates; the collections, lessons and Anki decks it creates do not.
One analysis per engine. eXtreme Gammon, GNUbg, BGBlitz and the embedded evaluator coexist on the same position, and the Analysis panel shows where each one came from. Importing one never erases another.
An imported analysis is never recomputed. blunderDB stores it as-is, with its level label (“3-ply”, “XG Roller++”, “Book”), its equities, its errors, its probabilities and the luck of the roll. The rule is “an evaluation only fills a hole”: the auto-analysis after import only visits the positions with no analysis, and the re-analysis of stale gammonNet analyses leaves untouched any position carrying an imported analysis (see Configuration).
Re-importing a match already stored only brings what is deeper. The match is recognised by its play (players, length, dice, moves, cube), not by its analysis: no match, game or move is rewritten. Marks set in the source software are added, and an analysis deeper than the stored one replaces it, position by position — a Roller++ version of the same match replaces the 3-ply version, whatever the import order; at equal or lower depth, the stored analysis stays. Re-importing the same file therefore rewrites nothing. The import report tells duplicates that brought nothing from those that deepened analyses. On the command line, --skip-duplicates skips a duplicate, taking nothing from it but the marks. A truncated match later completed (more games) is not the same match: it is imported as a second match.
An unreadable analysis does not stop the match from being imported. A decision whose analysis holds a value that is not a finite number (NaN or infinity, which some XG files carry) is imported without that analysis; the rest of the match comes in as usual, and the import report counts the decisions concerned.
A match already present under other names is reported, never merged. The two match fingerprints carry the player names: “Martin A.” and “Alice Martin” make two matches. The import also compares the dice (length, initial score, dice of each game) and reports, under the file’s line and in the report, the match already in the database under other names whose dice it has. blunderdb repair --duplicates lists these pairs in an existing database, along with truncated matches and their longer version.
A name known under another spelling is recorded under its canonical name. An alias says that “Martin A.” is another spelling of “Alice Martin” (or that one event name is another spelling of a different one). On import, the player and event names are replaced by their canonical name, and the match is filed in the tournament of the canonical event. The match’s fingerprints keep the names of the file: a file imported before its alias existed is therefore still recognised. A match whose dice are those of a match in the database, and whose names differ only by known aliases, is not a second match: its analyses enrich the one that is stored.

Aliases are managed in the Corpus tab of the settings: choose Players or Events, enter the alias and the canonical name, or remove an alias from the list. Suggest lists the names that differ only by case, accents, punctuation or word order; nothing is applied without a click on Apply. The same tab holds the Skip duplicates on import checkbox (the equivalent of --skip-duplicates, for the session) and the search for the probable duplicates of an existing database, that of blunderdb repair --duplicates: each pair is given by its match numbers, nothing is merged. Under a pair with the same dice, Create these aliases records in one click the player aliases that would make the two matches name the same players, the spelling of the most recent match becoming the alias: a single proposal when a name is common to both, otherwise two — seat for seat, then crossed — from which one chooses the one that says who is who. On the command line: blunderdb players alias and blunderdb events alias.

A folder is imported in parallel. Files are read on several cores at once and written in groups, always in folder order: match numbers do not depend on the machine. A file identical, byte for byte, to a file already read from the same folder is not read again: it counts as a duplicate. Cancelling stops the import at the current group; what was already written stays.
Progress reads in positions per second. The import window gives the percentage read, the throughput, the estimated time left and the running counts (imported, duplicates, failed). Minimize tucks it into the status bar, where a chip reopens it: the import goes on while you work, and the window comes back by itself with the report at the end. The first hundred errors are listed; the rest are counted, and the application log names them all. On the command line, blunderdb import --type batch shows the same progress on standard error, and --format json returns it in the final object (progress).
Each file of an import is logged, and an interrupted import can be resumed. For each file, the batch log keeps the path, the size, the modification date, the SHA-256 fingerprint and the result: new match (with its number), duplicate (with the match that covers it), enriched match, or error (with the message). A cancelled or cut import continues without redoing what is already decided: a file with the same path, size and date is skipped without being read; a file with the same content is read but not analysed; a file in error is retried. In the application, the end-of-import window lists the log (one file per line, with its result and the message of an error; a retried file shows its latest outcome), and a line that produced a match opens it. A cancelled import leaves the window open with the Resume button. On the command line, blunderdb import --type batch --dir <folder> --resume <batch> resumes the batch whose number was shown when the import started, and --format json returns the log in the final object (journal). The server accepts resume in the imports.batch request and returns the log through imports.files. The log is data of the import: opening or reading a database writes nothing to it.
An import cut short by the application stopping resumes from the window. The :resume command lists the imports nothing has finished; pick the one to continue, then point at the folder or files again, as --dir does on the command line. The batch’s journal decides what is not read again.
A large folder is imported in bulk mode. From 200 files on, blunderDB writes with a larger cache and fewer checkpoints. If the database holds no position yet, it goes further: search indexes are rebuilt only at the end, and writes are no longer synced to disk. A power cut during such an import can then damage the database: recreate it and run the import again. A program crash, on the other hand, only leaves missing indexes, which the next opening rebuilds (the log says so).
What blunderDB never writes: a recomputed luck value — it is read from the source file, or stays unknown — and a rollout it did not run itself: the rollout data held in a .xg file is not opened. Only the rollouts blunderDB produces (Rollouts) are stored, next to the analysis.

Metadata Panel

The Metadata panel (CTRL-M) displays the general information of the current database: User, creation date (Created), schema Version and Description. The user, the date and the description are edited in place and saved when leaving the field; the version is read-only. Also accessible via the meta command.

It also shows the database’s origin when there is one — see Handing out a database: origin and password. An ordinary database does not show that section.

Handing out a database: origin and password

A teacher handing out a database of positions has two mechanisms, independent of each other, both optional and both chosen at export time: marking the file with its origin, and protecting it with a password.

Note

Neither tracks what becomes of the file. blunderDB records nothing on the recipient’s side: opening a marked database is exactly like opening any other, and nothing anywhere logs who opened it, when, or where its contents came from.

Marking a database with its origin

The export window fits on a single screen: a header strip — the number of positions exported, Cancel and Export… — then a grid, one row per family with its count. These counts are exact: they are read from the server at every change of a box, on the same selection as the export, and are what the file will hold. The file is chosen last, after Export…; closing that choice without a name returns to the grid. A progress indicator then overlays for the duration of the write, the window closes by itself and the result appears in the status bar.

Nine points deserve attention:

The scope: from a list — a search, a collection — the Scope row chooses between the Current list and Whole library. The current list exports its positions, and the Matches row offers only those in which these positions were played; each ticked match leaves with all its positions. From the library, the scope is the library.
Whole library is the export of the server and of export --type database: all families are ticked from the start, the Anki decks with their review history, and the transcriptions in progress leave with the rest.
Synchronisation: with Whole library, the Synchronisation row offers Only what is new since the last export to this folder and the Choose the folder… button. The file is then no longer asked for: it is written in that folder under a new name, and the counts are those of what is new (see Synchronising two machines).
A match leaves with its transcription draft, under the Transcriptions box of the Matches row, ticked from the start; the matches that a tournament also carries. A draft that has not yet produced a match leaves only with the entire library. A match or transcription linked to a local video carries only the file name, without its folder, which the recipient relocates on their own copy; a web link leaves as is. The file from which a match was imported likewise leaves only by its name.
A match leaves with the aliases of its players and of its event: the Aliases box, ticked from the start, makes the player merge follow, and the recipient reads a merged player as one. An alias that names no one from the exported matches stays with you.
A collection whose positions are not all in the list arrives truncated. The collection list shows, for each one, the share covered (“12/40”) and flags it in red when it is partial. The + their positions box carries the positions of the ticked collections: those of a hand-made list, those that the query of a living collection selects. A living collection whose query names rows the file does not hold leaves frozen: it arrives as the list its query selects today, and the list marks it “frozen” in red with a note; ticking + their positions keeps it living.
A tournament always leaves with its matches: without them, the tournament–match link does not exist and the tournament would arrive empty. Ticking a tournament takes its matches along, even if the matches box is unticked or restricted to others. It also leaves with its Direction — journal, pairs, tables and attachment of the exported matches — and its Encounter leaves as soon as one of its tournaments does; the Direction’s output folder stays with you. The Direction is found again by opening the file, or by importing it into another database.
An Anki deck leaves with its positions, as a fresh study copy. The with my history box also carries the state of each card and the log of its reviews: a backup, or your decks moved to another machine. It is unticked for a list, ticked for the library.
Saved filters export not content but your own searches, useless in someone else’s database; the box is unticked for a list. A filter that designates a position, a match or a tournament by its number is renumbered for the file; if it designates one that does not leave, it stays with you.

Collections, lessons, transcriptions, Anki decks, aliases and tournaments leave with the creation and addition dates they bear on your side, not with the export date.

The User, Description and Creation date fields describe the produced file; they are prefilled from the source database.

Ticking Watermark reveals two fields:

Origin — what this file is and where it comes from, in your own words: “Jean Dupont’s course — 12 March 2026”. This field is mandatory: as long as it is empty, it is outlined in red and the export button stays inactive.
Note, optional — terms of use, a contact address, a request not to pass the file on.

The mark is signed with your issuer identity. It is therefore tamper-evident and unforgeable: nobody can alter it, nor fabricate one in your name. It is however not unremovable — the distributed file is an ordinary SQLite database, and blunderDB is free software. It prevents nothing: it says where the file came from.

The comments you have written are signed with the same identity: the recipient sees that they come from you, and whether they were altered along the way (see Comments Panel). A comment that already carries another issuer’s signature keeps it. The status bar says, at the end of the export, how many of your comments were signed; an export without a mark signs none.

Issuer identity

Marks are signed with your issuer identity, created by itself the first time you mark a file; there is nothing to set up. It belongs to a person rather than to a database: every file you mark carries the same public fingerprint, of the form A3F1-9C24-7B05-E1D8.

You can give that fingerprint to your recipients so they can check that a file really comes from you. The identity moves from one machine to another as a single file (extension .bdbid), optionally protected by a passphrase. That file lets anyone holding it sign in your name: do not share it.

In the preferences (the gear icon in the toolbar), the Issuer identity tab shows your name and fingerprint, and offers Save identity…, Load identity… and Regenerate….

Warning

Regenerating revokes nothing. A watermark embeds the public key that signed it, so it verifies for ever, on its own. If your identity file has leaked, whoever holds it can keep signing under your old fingerprint, and those marks stay valid.

What protects you after a leak is not software: it is publishing your new fingerprint and disowning the old one to your recipients.

Regenerating overwrites the current key; blunderDB offers to save it before replacing it.

Protecting a database with a password

The password is typed masked, here as when opening a protected file; the eye icon reveals it while it is held down, and masks it again as soon as it is released.

Ticking Password produces a file with the .dbx extension — even if the name chosen in the save window ends in .db. To open it, use the usual database opening: the selection window accepts .db and .dbx alike. blunderDB then asks for the password and installs an ordinary database alongside; after that nothing more is asked.

The dialog offers to delete the protected file once opened: without that you keep the same content under two names. The box is not ticked by default — the protected file is yours to keep if you mean to pass it on — and the deletion only happens after a successful open.

Warning

The password protects the file in transit, not the database. It stops a stranger opening a file left in a downloads folder or an attachment forwarded by mistake. It does not protect you from whoever you gave the password to.

The password is checked on every open, including when the file has already been opened on this machine before.

Technically, the database is encrypted with AES-256 in GCM mode, with the key derived from the password by Argon2id (64 MiB of memory, 3 passes, 4 lanes) and a random salt unique to each file. GCM authenticates the whole payload: a wrong password is detected as such, and so is any tampering with the encrypted file — you never silently end up with a corrupt database.

The protected file’s header stays in the clear: its origin remains readable without the password.

Reading a file’s origin

In the application, open the file and show the Metadata panel (the meta command). An Origin section appears at the top of the panel, read-only, stating what was written, by whom, when, and how the signature checks out:

“✓ marked by you”: the file carries your mark, intact;
“✓ signature verified”: the mark is intact and comes from another key — compare its fingerprint with the one the producer gave you;
“⚠ invalid signature”: the document has been altered or forged.

This section does not appear on an ordinary database.

From the command line, blunderdb info --db file.db shows the origin and the state of the signature, without ever writing to the file. It works on a protected file too, without the password. See CLI_USAGE.md for export’s --watermark and --password options, and for identity and open.

Publishing a database for others

A marked database is distributed like any other file — email, a personal site, a USB stick. blunderDB provides no service: no repository, no hosted catalogue, no account. That follows directly from its design: nothing is ever recorded on the side of whoever receives a file, so there would be nothing to report to a service even if one existed.

What makes a published database usable by someone else comes down to four fields, all of them already there:

User — who built it, under the name you want cited.
Description — what the database holds, in one sentence that fits in a list: “240 cube decisions at a score, commented, intermediate level”.
Origin (of the watermark) — what this file is and who it was produced for. It is the first thing the recipient reads in the Metadata panel.
Issuer fingerprint — publish it beside the file, not inside it: comparing it is how the recipient checks the file comes from you and not from someone who took your name.

A database published without a watermark stays perfectly usable; it is simply anonymous, and the Metadata panel then shows no Origin section.

To make a database known, the Show and tell category of the repository discussions serves as a directory: it is a list kept by those who publish, not a service blunderDB renders. Announcing one there takes the link, the four fields above and the fingerprint.