This article serves to explain the logic behind how database scan progress is tracked, calculated, and updated by Enterprise Recon.
The scanning engine calculates progress by the following order:
- Determine the total number of Tables to scan
- Assume every Table would be scanned twice (as scanning would be done twice on any Row that contains a BLOB). We will end up with 2x Table slices where BLOBs would be placed in the last half slice.
- Initialize progress = Zero-based index of the Table being scanned divided by total number of Tables
- Check if we're currently processing any BLOBs; if yes, that means we've finished scanning the 1st slice.
- In the case where we're scanning BLOBs, include the progress of the BLOB scan by the number of BLOBs - disregarding actual BLOB sizes.
We have 5 Tables. ER is currently scanning 3rd Table and processing the 30th BLOB (our of 100) of that Table.
- Tables = 5
- Slices = 10
- Current Table = 3
Progress = (Current Table - 1) ÷ Tables = 40%
- Progress = Progress + 1 ÷ Slices = 40% + 10% = 50%
- Progress = Progress + (30 ÷ 100 x 1 ÷ Slices) = 50 + 0.3 x 10 = 50% + 3% = 53%
In the case where my database has Table A with 1,000,000 records and Table B with 1,000 records, will Enterprise Recon regard the completion of Table A as;
(a) 50% done?
(b) A more accurate progress of 90%?
The answer would be (a).
We've explicitly avoided implementing (b) for performance reasons.
Does ER update the current progress only after an entire Table/Schema/Catalog is scanned or does it update for each individual record scanned?
Progress is updated while processing BLOBs (if any) for the Table and/or when a Table finishes scanning.
Is it normal for my database scan progress to be stuck at a certain percentage for an unusually long period of time?
Yes, the larger your Tables the more you would experience this.
This is because during the first stage of processing/scanning a new Table, the scanning engine will be processing the whole Table as raw data and will not check/report back the progress to ER.
It will only check for progress whilst scanning BLOBs in the second stage of a Table scan.
All information in this article is accurate and true as of the last edited date.