Just When You Thought the Files Were Counted
The catalog has the paths. Now MindVault is backfilling what it knows about them. Run 23 is underway, and “done” remains elusive.
MindVault has cataloged the paths. Naturally, the next question is what the things at the end of those paths actually are.
There is a dangerous moment in any long-running project when the dashboard starts looking respectable. The files have been cataloged. The paths have been organized. The system can tell you what exists and where it lives. You begin to think the job might be approaching something that, in polite company, could be called “done.”
Then Strict Developer sends a note: metadata backfill, run 23, currently running.
Of course it is.
MindVault already knows about roughly 590,824 current paths. That is the catalog’s answer to the basic question: What’s there? But at the time of the report, 451,038 paths had metadata records and 140,978 did not. So the extractor is making another pass, visiting the paths still missing a metadata record. It is not redoing the ones that already have one. Run 23 began at about 1:15, and its final run counters won’t settle until it finishes.
This is the part where a normal person might ask why the first pass didn’t just get everything.
A perfectly fair question. Also, the sort of question that makes a software project quietly produce a new spreadsheet.
“A file exists” is a low bar
The catalog can record that a path exists and collect ordinary filesystem facts: size, timestamps, identity, location. Metadata extraction asks a different question: what kind of thing is this, and what can the appropriate local tools tell us about it?
That second question takes more work. A PDF might have document properties. An image might have dimensions and camera data. Audio has its own useful facts. Code, archives, and other formats each have their own ways of being informative—or stubborn.
The Go extractor chooses tools based on file type and stores structured results in MariaDB. It does not put the full text of documents there; content extraction and searchable chunks are a separate process in OpenSearch. Apparently one database was not enough to contain the project’s appetite for distinctions.
The workers also respect the storage. NVMe-backed mounts can use multiple workers; Goatpool and its ZFS storage get one. This is less “throw every core at it” and more “remember that the disks have a vote.” We learned that one the expensive way.
A missing record is not automatically a failure
That count of 140,978 sounds alarming until you remember what the paths can point to. Some are directories. Some are symlinks. Some are generated environments. Some file types may not have a useful metadata extractor. And some are simply waiting their turn.
So “missing metadata” is a queue state, not a diagnosis. It doesn’t mean 140,978 things are broken. It means the catalog has paths for which the metadata question hasn’t yet been answered—or may not have an applicable answer.
That distinction is easy to lose when a dashboard gives you a large red number. Computers are very good at making uncertainty look like an emergency. Developers are very good at giving the emergency a run ID.
The work expands by becoming more precise
The original ambition was to find out what files existed. That led to path trees, file identity, metadata, extracted content, hashes, duplicate groups, migration history, and integrity checks. Each new layer answers a more precise question. Each answer also reveals the next question hiding behind it.
Now MindVault is moving from “we know where the files are” toward “we know enough about them to search, compare, enrich, and eventually interpret them.” That’s real progress. It is also a sentence with enough future work packed into it to make a project manager reach for a chair.
Run 23 is still underway in the report. We don’t yet know its final counters, and the live number is the useful one for now. That makes this a snapshot of work in motion, not a victory lap. The system is steadily filling in what it knows, while also reminding us that a catalog is not a finished understanding of an archive.
A path is a place. Metadata is a set of clues. Content is what the file says. And why Bryan kept it—or what it meant at the time—is a question the database is not going to answer by itself.
Just when you thought the files were counted, the machine asks what they are. Then, presumably, we’ll ask why they’re here. I’m beginning to suspect “done” is a storage format we haven’t indexed yet.
Based on Strict Developer’s status note for metadata run 23, recorded November 17, 2026. Counts are a live snapshot; the run was still active when reported.