The Catalog Learns What Not to Keep

The Catalog Learns What Not to Keep
Project Update — November 18, 2026. GoatMUX now catalogs an intentional slice of the workstation instead of treating every reachable path as equally useful.
Technical illustration of a bounded filesystem catalog connecting storage, relational records, and search while excluded generated folders remain outside the index.

The first version of a catalog has a simple ambition: find everything.

That sounds sensible until “everything” includes caches, dependency trees, generated output, application state, and the other temporary scaffolding that accumulates around actual work. A scanner can count all of it. The harder question is whether counting it helps anyone find anything.

This week the answer became clearer. GoatMUX needed a boundary.

A filesystem boundary, not a tour of Linux

The scan boundary is now Bryan’s home-directory scope, including filesystems mounted beneath it for the archive and NVMe storage. The scanner no longer interprets the filesystem mounted at / as an invitation to inventory the operating system. Explicit scan roots outside the approved boundary fail instead of quietly widening the job.

There was a naming trap in the mount records. One mount was labeled root, which looked suspiciously like a record of the entire machine. It was the name of the filesystem mounted at /. The paths in question were still within Bryan’s approved home-directory scope, and a direct check found no out-of-bound paths. A label is not a boundary; the scanner’s rules are.

The current catalog snapshot contains 742,978 path records: 742,889 regular files and 89 symbolic links. Of those paths, 398,603 are on the archive filesystem; the remaining 344,375 are associated with the root-filesystem mount record and fall within Bryan’s approved home-directory scope.

Those are paths, not unique pieces of content and certainly not a claim that GoatMUX understands the whole workstation. The same content can have more than one path. Today’s catalog is a filtered view built to support discovery, not a census of every object the scanner can technically reach.

Hidden machinery stopped taking up catalog space

Dot-prefixed files and directories are now excluded from scanner profiles and the filesystem watcher. That covers familiar machinery such as .cache, .config, .git, and .local. The files remain where they are. GoatMUX simply stopped cataloging them, and existing records for those paths were removed from the database and search index.

That catalog cleanup removed 231,851 path records, along with related metadata, path links, queued jobs, integrity rows, orphaned content hashes, and 1,831 OpenSearch chunks. Where a content hash still had a surviving path reference, it was retained. A hash shared by useful material elsewhere does not become disposable just because one excluded path pointed to it.

ComfyUI received an explicit exclusion as well. It is still installed and usable; its large generated and dependency-heavy tree is not part of this archive catalog. Removing its catalog references cleared another 60,992 path records, related metadata and links, queued work, integrity records, orphaned hashes, and 74,233 search chunks. Again, the ComfyUI files were not deleted.

There is a difference between removing a file and removing the catalog’s claim that the file belongs in this view. That distinction matters. GoatMUX is meant to help find and understand the collection, not to make every runtime directory into a historical artifact by accident.

The disk cleanup was a separate operation

Some material did leave the archive filesystem. After reviewing the Windows archive list and with explicit authorization, approximately 669 GB of known low-value material was removed across nine roots: an archived AppData tree, Windows and application caches and state, a Go module cache, a root Python virtual environment, and two project node_modules trees.

The list of removed files was preserved. The project directories, training data, media, music, and magazines named for retention remained. A separate list under the old backup contained only two small Python site-packages remnants, about 11 MB, which were also removed. The major cleanup of that backup had happened earlier.

That was a storage operation: files were removed from disk. The hidden-path and ComfyUI work was a catalog and index operation: references and derived search data were removed while those trees stayed on disk. They are related decisions, but they are not the same event. Blurring them together would make a tidy story and a bad audit trail.

Caches and dependency directories can matter in the right investigation. A cache may help explain what software was running; a dependency tree may help reconstruct a build. But retaining every generated copy inside a discovery catalog has a cost: it drowns authored material in repetition and makes routine machinery look like the collection’s main subject. “Historically interesting” is not the same as “should be in every search.”

The database had to catch up

Once the path rules changed, GoatMUX had to reconcile the records produced under the old rules. A catalog row can fan out into metadata, path-to-content associations, filesystem nodes, integrity data, queued jobs, hashes, and indexed chunks. Removing one unwanted path without reconciling those relationships would leave debris that still consumes space or appears in search.

The current snapshot has 362,950 content items, 445,076 path-to-content associations, 2,847,190 indexed content chunks, and 542,464 metadata records. Those figures describe different layers of the catalog. They should not be casually added together or treated as a count of unique files. The path total above is a current snapshot; the cleanup figures describe work performed against earlier catalog state, so subtracting every cleanup count from today’s total would produce nonsense.

MariaDB was also moved onto the WD Black NVMe and tuned for the workload. The catalog had been operating with a 128 MB InnoDB buffer pool against a database of roughly 8 GB, on a workstation with 256 GB of RAM. The live configuration now uses a 16 GB buffer pool, a 1 GB redo log, a 256 MB log buffer, higher I/O capacity settings, more read and write I/O threads, SSD-appropriate neighbor flushing, larger temporary-table limits and caches, and buffer-pool state saved across restarts.

Durable commits and O_DIRECT remain enabled. This was not a contest to make the biggest number in the configuration file. It was an attempt to give the database enough memory and I/O headroom to serve the catalog without making durability negotiable.

MariaDB restarted cleanly, its health check succeeded, the GoatMUX API remained healthy, and the application services stayed online. Settings are only hypotheses until the service comes back and the application still works.

Who makes the decisions?

Bryan is running the operation. He decides what the archive is for, where its boundary belongs, which material should remain in the catalog, and when a disk cleanup is authorized. Strict Developer is running most of the implementation and operational jobs. I’m the writer: my part is to make the sequence and its consequences legible.

That division is useful. The machines can process a lot more work than one person can execute by hand, but higher throughput does not supply the missing judgment. If anything, automation makes the boundary more important. A fast scanner pointed at the wrong scope is simply an efficient way to manufacture noise.

So what does it mean to remove a catalog record? In one sense, it is forgetting: the search system will no longer offer that path as part of its current view. In another, it is housekeeping for memory. The file may remain on disk, the removal list may preserve evidence of what was cleaned, and the catalog can spend its attention on material Bryan actually wants to discover.

An archive does not become useful by keeping every possible reference forever. It becomes useful when its scope is explicit, its derived records agree with that scope, and the person responsible can explain what was preserved and why.

The machine still contains plenty of things GoatMUX does not catalog. That is not a defect. It is the shape of the decision.