Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Skip to content

[eDrive][Remove persistent flag][Step 3.5] Make eDrive to work without service binding

Vincent Bourgmayer requested to merge 299-try-syncStateImpl into 299-step-3-replace-service-binding

Description

This MR complete the one about removing Service's binding.

As raised here by @jonathanklee, the MR !255 (merged) cannot handle well the creation of SynchronizationService. This is solved in this MR

It add a kind of "state pattern" mechanism for eDrive to allow better separation between task (file listening, file scanning, file transfer). By example: "Instant file detection" won't run anymore during "Periodic File detection"

This MR, was primarily a trial for an arch's design I have in mind for a while. I succeed to clarify few point and tried to implement it.

Based on my test it works pretty well and it is open for futur improvement

Screenshots

Technical details

There is some work to improve the implementation and make it more clear

Sync State (diag 1)

image

Rules about changing state

image

Timeline example

image

Some UML image

Tests

Example logs of a fresh install with successfull sync

verbose log 
10-23 15:18:00.099  3000  4654 V DbHelper: FirstStartWorker.doWork()
10-23 15:18:00.108  3000  3000 V ObserverService: onStartCommand(1)
10-23 15:18:00.112  3000  3000 V ObserverService: checkStartCondition()
10-23 15:18:00.113  3000  3000 V CommonUtils: haveNetworkConnection()
10-23 15:18:00.117  3000  3000 I StateManager: Change Sync state to PERIODIC_SCAN from LISTENING_FILES
10-23 15:18:00.117  3000  3000 D EdriveApplication: Stopped RecursiveFileObserver on root folder
10-23 15:18:00.118  3000  3000 D ObserverService: starting periodic scan is allowed ? true
10-23 15:18:00.119  3000  3000 I ObserverService: clearCachedFile()
10-23 15:18:00.128  3000  3000 I ObserverService: deleteOldestCrashLogs()
10-23 15:18:00.134  3000  3000 I ObserverService: startScan(true)
10-23 15:18:00.142  3000  4725 V AbstractFileLister: SyncedFolder : Images, /storage/emulated/0/DCIM/, 0, true, 1
10-23 15:18:00.142  3000  3402 I WM-WorkerWrapper: Worker result SUCCESS for Work [ id=e08cc06c-bdbb-4751-b76b-363cf32483f4, tags={ foundation.e.drive.work.FirstStartWorker, eDrive, eDrive-init } ]
10-23 15:18:00.164  3000  4714 D DbHelper: generateAppListFile()
10-23 15:18:00.182  3000  4714 V ListAppsWorker: ListPWAs
10-23 15:18:00.190  3000  4725 D OwnCloudClient #1: REQUEST PROPFIND /remote.php/dav/files/00020@eeo.one/Photos/



... //Other files's change detection


10-23 15:18:09.510  3000  4724 D LocalContentScanner: Add upload SyncRequest for /storage/emulated/0/Pictures/Screenshots/Screenshot_20231011-113745_App Lounge.png
10-23 15:18:09.511  3000  4724 D LocalContentScanner: Add upload SyncRequest for /storage/emulated/0/Pictures/Screenshots/Screenshot_20231011-121246_Paramètres.png
10-23 15:18:09.511  3000  4724 D ObserverService: syncRequests contains 28
10-23 15:18:09.514  3000  4724 I StateManager: Change Sync state to SYNCHRONIZING from PERIODIC_SCAN
10-23 15:18:09.546  3000  4724 D DbHelper: Started RecursiveFileObserver on root folder
10-23 15:18:09.554  3000  3000 V SynchronizationService: onStartCommand()
10-23 15:18:09.557  3000  3000 D SynchronizationService: startAllThreads
10-23 15:18:09.558  3000  3000 V CommonUtils: haveNetworkConnection()
10-23 15:18:09.565  3000  3000 V SynchronizationService:  starts DOWNLOAD on thread #0 for: IMG_20230629_144008.jpg
10-23 15:18:09.566  3000  3000 V CommonUtils: haveNetworkConnection()
10-23 15:18:09.567  3000  4775 V DownloadFileOperation: run() for /Photos/Camera/IMG_20230629_144008.jpg
10-23 15:18:09.571  3000  3000 V SynchronizationService:  starts DOWNLOAD on thread #1 for: IMG_20230629_144024.jpg




... //other files sync


10-23 15:19:49.104  3000  4774 V UploadFileOperation:  starts UPLOAD on thread #0 for: IMG_20230629_144043.jpg
10-23 15:19:49.105  3000  4774 V SynchronizationService: onRemoteOperationFinish()
10-23 15:19:49.107  3000  4774 D SynchronizationService: Upload operation for uihoiuh.md result in: OK
10-23 15:19:49.108  3000  4928 D UploadFileOperation: Synchronization conflict because: last modified from DB(1698067098000) and from file (1698067098000) are the same 
10-23 15:19:49.109  3000  4774 V CommonUtils: haveNetworkConnection()
10-23 15:19:49.114  3000  4774 D SynchronizationService: Thread #1: No more sync request to start.
10-23 15:19:49.115  3000  4774 V SynchronizationService: onRemoteOperationFinish()
10-23 15:19:49.116  3000  4774 D SynchronizationService: Upload operation for IMG_20230629_144043.jpg result in: SYNC_CONFLICT
10-23 15:19:49.118  3000  4774 V CommonUtils: haveNetworkConnection()
10-23 15:19:49.122  3000  4774 D SynchronizationService: Thread #0: No more sync request to start.
10-23 15:19:49.123  3000  4774 I StateManager: Change Sync state to LISTENING_FILES from SYNCHRONIZING


... // Force another sync

10-23 15:26:53.336  3000  3000 D DebugCmdReceiver: Force Sync intent received
10-23 15:26:53.339  3000  3000 V ObserverService: onStartCommand(1)
10-23 15:26:53.341  3000  3000 V ObserverService: checkStartCondition()
10-23 15:26:53.342  3000  3000 V CommonUtils: haveNetworkConnection()
10-23 15:26:53.344  3000  3000 I StateManager: Change Sync state to PERIODIC_SCAN from LISTENING_FILES
10-23 15:26:53.345  3000  3000 D EdriveApplication: Stopped RecursiveFileObserver on root folder
10-23 15:26:53.345  3000  3000 D ObserverService: starting periodic scan is allowed ? true
10-23 15:26:53.345  3000  3000 I ObserverService: clearCachedFile()
10-23 15:26:53.350  3000  3000 I ObserverService: deleteOldestCrashLogs()
10-23 15:26:53.357  3000  3000 I ObserverService: startScan(true)
10-23 15:26:53.365  3000  5183 V AbstractFileLister: SyncedFolder : Images, /storage/emulated/0/DCIM/, 1693653164000, true, 1
10-23 15:26:53.371  3000  5183 D OwnCloudClient #1: REQUEST PROPFIND /remote.php/dav/files/00020@eeo.one/Photos/
10-23 15:26:54.259  3000  5183 I ReadFolderRemoteOperation: Synchronized /Photos/: Operation finished with HTTP status code 207 (success)
10-23 15:26:54.267  3000  5183 V SyncedFileStateDAO: SyncedFolder : Music, /storage/emulated/0/Music/, 1696433412000, true, 2

... // After some detection

10-23 15:27:02.712  3000  5182 V AbstractFileLister: added Screenshot_20231011-121246_Paramètres.png into list of file to scan
10-23 15:27:02.712  3000  5182 V AbstractFileLister: added Screenshot_20231012-155416_App Lounge.png into list of file to scan
10-23 15:27:02.713  3000  5182 V AbstractFileLister: added Screenshot_20231017-113002_Gestionnaire de compte.png into list of file to scan
10-23 15:27:02.713  3000  5182 V AbstractFileLister: added Screenshot_20231019-211324_Fichiers.png into list of file to scan
10-23 15:27:02.736  3000  5182 I LocalContentScanner: There is no file to sync.
10-23 15:27:02.737  3000  5182 I StateManager: Change Sync state to LISTENING_FILES from PERIODIC_SCAN
10-23 15:27:02.738  3000  3000 V ListFileRemoteOperation: onDestroy()

... // After a reboot

10-23 15:41:40.406  3045  3336 D FileEventListener: handleFileCloseWrite( /storage/emulated/0/Pictures/Sketchbook Gallery/Sketchbook Previews/new_25369813-7bb5-4588-80e0-6e3bb03ba40f.png )
10-23 15:41:40.421  3045  3336 D DbHelper: manageSyncedFileStateDB( new_25369813-7bb5-4588-80e0-6e3bb03ba40f.png, INSERT )
10-23 15:41:40.431  3045  3336 D SyncedFileStateDAO: Sending a SyncRequest for new_25369813-7bb5-4588-80e0-6e3bb03ba40f.png
10-23 15:41:40.433  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from LISTENING_FILES
10-23 15:41:40.438  3045  3045 V SynchronizationService: onStartCommand()
10-23 15:41:40.463  3045  3045 D NetworkUtils: Searching known-servers store at /data/user/0/foundation.e.drive/files/knownServers.bks
10-23 15:41:40.485  3045  3045 D OwnCloudClient #0: Creating OwnCloudClient
10-23 15:41:40.489  3045  3045 D DavClientProvider: startAllThreads
10-23 15:41:40.489  3045  3045 V CommonUtils: haveNetworkConnection()
10-23 15:41:40.493  3045  3045 V UploadFileOperation:  starts UPLOAD on thread #0 for: new_25369813-7bb5-4588-80e0-6e3bb03ba40f.png
10-23 15:41:40.494  3045  3045 V CommonUtils: haveNetworkConnection()
10-23 15:41:40.495  3045  3045 D SynchronizationService: Thread #1: No more sync request to start.
10-23 15:41:40.495  3045  4924 D UploadFileOperation: Can't get the file. It might have been deleted
10-23 15:41:40.502  3045  4923 V SynchronizationService: onRemoteOperationFinish()
10-23 15:41:40.503  3045  4923 D SynchronizationService: Upload operation for new_25369813-7bb5-4588-80e0-6e3bb03ba40f.png result in: FORBIDDEN
10-23 15:41:40.508  3045  4923 D DbHelper: Force folder to be rescan next time (row affected) : 0
10-23 15:41:40.511  3045  4923 V CommonUtils: haveNetworkConnection()
10-23 15:41:40.514  3045  4923 D SynchronizationService: Thread #0: No more sync request to start.
10-23 15:41:40.515  3045  4923 I StateManager: Change Sync state to LISTENING_FILES from SYNCHRONIZING
10-23 15:41:40.992  3045  3336 D FileEventListener: handleFileCloseWrite( /storage/emulated/0/Pictures/Sketchbook Gallery/Sketchbook Previews/25369813-7bb5-4588-80e0-6e3bb03ba40f.png )
10-23 15:41:40.994  3045  3336 D SyncedFileStateDAO: Sending a SyncRequest for 25369813-7bb5-4588-80e0-6e3bb03ba40f.png
10-23 15:41:40.995  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from LISTENING_FILES
10-23 15:41:40.997  3045  3045 V SynchronizationService: onStartCommand()
10-23 15:41:40.999  3045  3336 D FileEventListener: handleFileDelete( /storage/emulated/0/Pictures/Sketchbook Gallery/Sketchbook Previews/new_25369813-7bb5-4588-80e0-6e3bb03ba40f.png )
10-23 15:41:41.001  3045  3336 D SyncedFileStateDAO: Sending a SyncRequest for new_25369813-7bb5-4588-80e0-6e3bb03ba40f.png
10-23 15:41:41.001  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from SYNCHRONIZING
10-23 15:41:41.002  3045  3045 D SynchronizationService: startAllThreads
10-23 15:41:41.004  3045  3045 V CommonUtils: haveNetworkConnection()
10-23 15:41:41.010  3045  3045 V UploadFileOperation:  starts UPLOAD on thread #0 for: 25369813-7bb5-4588-80e0-6e3bb03ba40f.png
10-23 15:41:41.013  3045  3045 V CommonUtils: haveNetworkConnection()
10-23 15:41:41.016  3045  3336 D FileEventListener: handleFileCloseWrite( /storage/emulated/0/Pictures/Sketchbook Gallery/temp25369813-7bb5-4588-80e0-6e3bb03ba40f.tiff )
10-23 15:41:41.016  3045  3045 D SynchronizationService: Thread #1: No more sync request to start.
10-23 15:41:41.020  3045  3336 D DbHelper: manageSyncedFileStateDB( temp25369813-7bb5-4588-80e0-6e3bb03ba40f.tiff, INSERT )
10-23 15:41:41.023  3045  3336 D SyncedFileStateDAO: Sending a SyncRequest for temp25369813-7bb5-4588-80e0-6e3bb03ba40f.tiff
10-23 15:41:41.024  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from SYNCHRONIZING
Short log of state change 

10-23 15:18:00.117  3000  3000 I StateManager: Change Sync state to PERIODIC_SCAN from LISTENING_FILES
10-23 15:18:09.514  3000  4724 I StateManager: Change Sync state to SYNCHRONIZING from PERIODIC_SCAN
10-23 15:18:18.788  3000  3444 I StateManager: Change Sync state to SYNCHRONIZING from SYNCHRONIZING
10-23 15:19:49.123  3000  4774 I StateManager: Change Sync state to LISTENING_FILES from SYNCHRONIZING

...// After reboot, for FileObserver


10-23 15:41:40.433  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from LISTENING_FILES
10-23 15:41:40.515  3045  4923 I StateManager: Change Sync state to LISTENING_FILES from SYNCHRONIZING
10-23 15:41:40.995  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from LISTENING_FILES
10-23 15:41:41.001  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from SYNCHRONIZING
10-23 15:41:41.024  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from SYNCHRONIZING
10-23 15:41:42.344  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from SYNCHRONIZING
10-23 15:41:42.348  3045  3336 I StateManager: Change Sync state to SYNCHRONIZING from SYNCHRONIZING
10-23 15:43:05.040  3045  4925 I StateManager: Change Sync state to LISTENING_FILES from SYNCHRONIZING

Issues

10 commandments of code reviews

👪 ❤ https://gitlab.e.foundation/internal/wiki/-/wikis/mobile-team/guidelines/Code-review

Edited by Vincent Bourgmayer

Merge request reports