Ferret File System/v0.1

Introduction

FileFerret is a software system intended to tackle the problem of keeping track of large numbers of files, including those archived to offline media. It can also provide services to other applications which may need to track large numbers of files for their own purposes.

My original notes are on HypertWiki, but the official documentation will be here (HTYP) as I create it.

Preliminary Table Designs

These are based on what I had created in MS Access 97.

filaments

"Filaments" is actually a portmanteau of "file elements", which is supposed to stand for "elemental file", but I didn't want to spell it "filements" because the obvious pronunciation would be "file-ments". Maybe "felements" would have been better. Or maybe something more like "dataset" or "byteset" or "pattern". <mysql>CREATE TABLE `filaments` (

 `ID`        INT NOT NULL AUTO_INCREMENT,
 `Title`     VARCHAR(255) COMMENT "unique name for the file, wherever it may be found",
 `AutoTitle` VARCHAR(255) COMMENT "automatically-generated title",
 `AutoDescr` VARCHAR(255) COMMENT "automatically-generated description",
 `FileSize`  INT          COMMENT "correct file size in bytes",
 `FileCkSum` INT          COMMENT "correct file checksum",
 PRIMARY KEY(`ID`)

) ENGINE = MYISAM;</mysql>

• Basically, if you want to describe a file, you don't; you describe the filament. Files are localized instances (possibly imperfect copies) of filaments.
• Title will probably end up being some form of the filename, possibly with disambiguating text prepended.
• Instead of a Descr field, I think we'll want to point to a wiki page for more info, but I haven't worked out how to do this yet.
• AutoDescr can be generated by the application which first goes looking for the filament and which therefore may have more understanding of the filament's purpose in life
• It's not clear whether we need something more elaborate than just a 4-byte checksum, because there are so many different ways of generating this.

files

<mysql>CREATE TABLE `files` (

 `ID`          INT NOT NULL AUTO_INCREMENT,
 `ID_Folder`   INT NOT NULL     COMMENT "Folders.ID of parent folder",
 `ID_Filament` INT DEFAULT NULL COMMENT "Filaments.ID for this file",
 `FileName`    VARCHAR(255)     COMMENT "just the filename.ext, not the full path",
 `FileSize`    INT DEFAULT NULL COMMENT "number of bytes in file",
 `FileCksum`   INT DEFAULT NULL COMMENT "file's checksum (algorithm to be worked out later)",
 `WhenCreated` DATETIME DEFAULT NULL COMMENT "file's creation timestamp",
 `WhenChanged` DATETIME DEFAULT NULL COMMENT "file's modification timestamp",
 `FirstFound`  DATETIME NOT NULL     COMMENT "time/date when file was first found during a scan",
 `LastFound`   DATETIME NOT NULL     COMMENT "time/date when file was most recently found during a scan",
 `isFound`     BOOL DEFAULT FALSE    COMMENT "TRUE = was found on most recent scan",
 PRIMARY KEY(`ID`)

) ENGINE = MYISAM;</mysql>

folders

<mysql>CREATE TABLE `folders` (

 `ID`          INT NOT NULL AUTO_INCREMENT,
 `ID_Parent`   INT DEFAULT NULL COMMENT "parent folder; NULL = filesystem root",
 `Name`        VARCHAR(255) COMMENT "filesystem's name for the folder",
 `Descr`       VARCHAR(255) COMMENT "description for human consumption (optional; mainly for local roots)",
 `WhenCreated` DATETIME DEFAULT NULL "folder's creation timestamp, where available",
 `WhenChanged` DATETIME DEFAULT NULL "folder's modification timestamp, where available",
 `FirstFound`  DATETIME NOT NULL     COMMENT "time/date when folder was first found during a scan",
 `LastFound`   DATETIME NOT NULL     COMMENT "time/date when folder was most recently found during a scan",
 `isFound`     BOOL DEFAULT FALSE    COMMENT "TRUE = was found on most recent scan",
 `noScan`      BOOL DEFAULT FALSE    COMMENT "TRUE = for whatever reason, don't bother scanning inside this folder",
 `wasDenied`   BOOL DEFAULT FALSE    COMMENT "TRUE = filesystem did not allow access on last scan attempt",
 `isRecur`     BOOL DEFAULT FALSE    COMMENT "TRUE = this folder is a repeat of a parent folder, probably due to a recursive link",
 PRIMARY KEY(`ID`)

) ENGINE = MYISAM;</mysql>

• Name is not the full path, just the folder's filename; if it is a volume's root folder, it may be null or a device name (c:) depending on OS
• noScan is for blacklisting folders whose contents we really don't want to track, such as temp and trash areas which may become full of useless clutter, or areas where things that aren't really files are kept (e.g. "/dev")

locations

<mysql>CREATE TABLE `locations` (

 `ID`        INT NOT NULL AUTO_INCREMENT,
 `ID_Folder` INT NOT NULL,
 `Name`      VARCHAR(63),
 `Descr`     VARCHAR(255),
 PRIMARY KEY(`ID`)

) ENGINE = MYISAM;</mysql>

machines

<mysql>CREATE TABLE `machines` (

 `ID`        INT NOT NULL AUTO_INCREMENT,
 `NetName`   VARCHAR(63) NOT NULL COMMENT "network name of machine",
 `FirstConnected` DATETIME DEFAULT NULL COMMENT "when this machine was first connected to the DB",
 `LastConnected`  DATETIME DEFAULT NULL COMMENT "when this machine was last connected to the DB",
 PRIMARY KEY(`ID`)

) ENGINE = MYISAM;</mysql>

mappings

<mysql>CREATE TABLE `mappings` (

 `ID`        INT NOT NULL AUTO_INCREMENT,
 `ID_Location` INT NOT NULL COMMENT "locations.ID",
 `ID_Machine`  INT NOT NULL COMMENT "machines.ID",
 `Name`        VARCHAR(63) NOT NULL COMMENT "short name for lists",
 `Descr`       VARCHAR(255) DEFAULT NULL COMMENT "longer description",
 `FileSpec`    VARCHAR(255) NOT NULL COMMENT "must not have terminating slash; will be used as prefix for folder chains",
 `isPrimary`   BOOL DEFAULT FALSE COMMENT "is Machine primarily responsible for maintaining (scanning) this Location?",
 `isDrive`     BOOL DEFAULT FALSE COMMENT "TRUE = can be queried for volume and capacity information; may be removable.",
 PRIMARY KEY(`ID`)

) ENGINE = MYISAM;</mysql>