Ferret File System/v0.1/process: Difference between revisions

From Woozle Writes Code
Jump to navigation Jump to search
(careful review produced more revision)
m (Woozle moved page FileFerret/process to Ferret File System/v0.1/process: obsolete)
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
When given a {{l/fileferret/sql|volume}} to scan, there are two sides to the scanning process:
When given a {{l/fileferret/sql|volume}} to scan, there are two sides to the scanning process:
# '''scanning the entry records''' to see if anything is missing
# Are there new items?
# '''scanning the volume''' to see what's on it (including any new entries)
# Has anything been deleted?
 
Both questions can be answered by:
* marking every file in the current index as "not found"
* scanning the volume to get a list of all files currently present
* updating the index for each file found to show that the file has been found
 
Any files still marked "not found" have been removed from the volume since the last scan (which can be noted in a log of some kind).
==Scanning the Entry Records==
==Scanning the Entry Records==
* For every existing {{l/fileferret/sql|entry}} record for that volume:
* For every existing {{l/fileferret/sql|entry}} record for that volume:

Latest revision as of 13:40, 25 February 2024

When given a volume to scan, there are two sides to the scanning process:

  1. Are there new items?
  2. Has anything been deleted?

Both questions can be answered by:

  • marking every file in the current index as "not found"
  • scanning the volume to get a list of all files currently present
  • updating the index for each file found to show that the file has been found

Any files still marked "not found" have been removed from the volume since the last scan (which can be noted in a log of some kind).

Scanning the Entry Records

  • For every existing entry record for that volume:
    • Clear the "didFind" flag (FALSE). (These get set as entries are found during the filesystem scan.)
    • Set the "didSeek" flag TRUE. (Indicates entries that existed before this scan.)
    • Read the directory at the volume's URL.
    • For each matching filesystem item found
      • mark didFind = TRUE

We can update volume entries here, but then we still have to create new ones during the volume scan. It probably doesn't matter whether we do the update here and the insert there, or do both there.

Scanning the Volume

Each Filesystem Entry

For every filesystem entry in the current folder:

  • Look for a matching entry.
  • Does entry exist?
    • NO: create it
    • YES: Is entry a folder?
      • YES: Does folder have a folder entry?
        • YES: Is that entry marked noscan=TRUE?
          • YES: remember not to scan inside it.
  • Log that the entry was found, and whether it was new or existing.
  • If it was an existing entry, log any changes in the specs (timestamps, size).
  • If Entry is a File, do "Each File" below.
  • If Entry is a Folder, do "Each Folder" below, unless flagged as "noscan".

Each File

  • Check the firev to see if it matches.
  • Does it match?
    • YES: update the firev record "LastFound" timestamp.
    • NO: create a new firev and update the file's ID_Firev.
  • Log whether the firev has changed or not.

Each Folder

  • Is folder new?
  • YES:
    • Check name against text blacklist. Was it found?
      • YES:
        • Mark it "noScan" when creating.
        • Add a note which includes the matching blacklist line.
  • Is "noScan" set?
    • YES: log that it was not scanned ("noScan" is currently the only possible reason for this).
    • NO: do "Each Filesystem Entry" above for each entry in the folder's directory.