Ferret File System/v0.1/process: Difference between revisions

From Woozle Writes Code
Jump to navigation Jump to search
(scanning a volume, take 1)
 
m (Woozle moved page FileFerret/process to Ferret File System/v0.1/process: obsolete)
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Scanning a Volume==
When given a {{l/fileferret/sql|volume}} to scan, there are two sides to the scanning process:
When given a volume to scan, here's what happens:
# Are there new items?
* FF reads the directory at the volume's URL
# Has anything been deleted?
* For each item found, does "Each Entry" below.
 
===Each Entry===
Both questions can be answered by:
* Looks for a matching entry in {{l/fileferret/sql|dirent}}.
* marking every file in the current index as "not found"
* If entry does not exist, creates it.
* scanning the volume to get a list of all files currently present
* Logs that the entry was found, and whether it was new or existing.
* updating the index for each file found to show that the file has been found
* If it was an existing entry, logs any changes in the specs.
 
* If Entry is a File, does "Each File" below.
Any files still marked "not found" have been removed from the volume since the last scan (which can be noted in a log of some kind).
* If Entry is a Folder, does "Each Folder" below.
==Scanning the Entry Records==
* For every existing {{l/fileferret/sql|entry}} record for that volume:
** Clear the "didFind" flag (FALSE). (These get set as entries are found during the filesystem scan.)
** Set the "didSeek" flag TRUE. (Indicates entries that existed before this scan.)
** Read the directory at the volume's URL.
** For each matching filesystem item found
*** mark didFind = TRUE
 
We can update volume entries here, but then we still have to create new ones during the volume scan. It probably doesn't matter whether we do the update here and the insert there, or do both there.
==Scanning the Volume==
===Each Filesystem Entry===
For every filesystem entry in the current folder:
* Look for a matching {{l/fileferret/sql|entry}}.
* Does entry exist?
** NO: create it
** YES: Is entry a folder?
*** YES: Does folder have a {{l/fileferret/sql|folder}} entry?
**** YES: Is that entry marked noscan=TRUE?
***** YES: remember not to scan inside it.
* Log that the entry was found, and whether it was new or existing.
* If it was an existing entry, log any changes in the specs (timestamps, size).
* If Entry is a File, do "Each File" below.
* If Entry is a Folder, do "Each Folder" below, unless flagged as "noscan".
===Each File===
===Each File===
* Checks the {{l/fileferret/sql|firev}} to see if it matches.
* Check the {{l/fileferret/sql|firev}} to see if it matches.
** If not, creates a new firev and updates the file's ID_Firev.
* Does it match?
** Logs whether the firev has changed or not.
** YES: update the firev record "LastFound" timestamp.
** NO: create a new firev and update the file's ID_Firev.
* Log whether the firev has changed or not.
===Each Folder===
===Each Folder===
* Checks to see if folder is scannable.
* Is folder new?
** If it isn't, log that it was not scanned.
* YES:
** If it is, do "Each Entry" above for each entry in the folder's directory.
** Check name against text blacklist. Was it found?
*** YES:
**** Mark it "noScan" when creating.
**** Add a note which includes the matching blacklist line.
* Is "noScan" set?
** YES: log that it was not scanned ("noScan" is currently the only possible reason for this).
** NO: do "Each Filesystem Entry" above for each entry in the folder's directory.

Latest revision as of 13:40, 25 February 2024

When given a volume to scan, there are two sides to the scanning process:

  1. Are there new items?
  2. Has anything been deleted?

Both questions can be answered by:

  • marking every file in the current index as "not found"
  • scanning the volume to get a list of all files currently present
  • updating the index for each file found to show that the file has been found

Any files still marked "not found" have been removed from the volume since the last scan (which can be noted in a log of some kind).

Scanning the Entry Records

  • For every existing entry record for that volume:
    • Clear the "didFind" flag (FALSE). (These get set as entries are found during the filesystem scan.)
    • Set the "didSeek" flag TRUE. (Indicates entries that existed before this scan.)
    • Read the directory at the volume's URL.
    • For each matching filesystem item found
      • mark didFind = TRUE

We can update volume entries here, but then we still have to create new ones during the volume scan. It probably doesn't matter whether we do the update here and the insert there, or do both there.

Scanning the Volume

Each Filesystem Entry

For every filesystem entry in the current folder:

  • Look for a matching entry.
  • Does entry exist?
    • NO: create it
    • YES: Is entry a folder?
      • YES: Does folder have a folder entry?
        • YES: Is that entry marked noscan=TRUE?
          • YES: remember not to scan inside it.
  • Log that the entry was found, and whether it was new or existing.
  • If it was an existing entry, log any changes in the specs (timestamps, size).
  • If Entry is a File, do "Each File" below.
  • If Entry is a Folder, do "Each Folder" below, unless flagged as "noscan".

Each File

  • Check the firev to see if it matches.
  • Does it match?
    • YES: update the firev record "LastFound" timestamp.
    • NO: create a new firev and update the file's ID_Firev.
  • Log whether the firev has changed or not.

Each Folder

  • Is folder new?
  • YES:
    • Check name against text blacklist. Was it found?
      • YES:
        • Mark it "noScan" when creating.
        • Add a note which includes the matching blacklist line.
  • Is "noScan" set?
    • YES: log that it was not scanned ("noScan" is currently the only possible reason for this).
    • NO: do "Each Filesystem Entry" above for each entry in the folder's directory.
Retrieved from "https://wooz.dev/mw/index.php?title=Ferret_File_System/v0.1/process&oldid=7086"