Implementing WebDAV Synchronization
IT Hit WebDAV Server Library for Java supports synchronization based on Sync-ID. To implement Sync-ID algorithm your server must be able to return all changes that happened in your storage since provided sync-token, including deleted items. In addition to that each item must have a unique ID and parent ID.
To test synchronization you can use samples samples provided with IT Hit User File System:
To support synchronization your storage must support the following functionality:
- Each item must have ID and ParentID unique withing your file system.
- Your storage must be able to store information about deleted items.
- Your storage must be able to return all changes that happened since provided sync-token. Including it must return deleted items and distinguish between deleted and all other changes (created, updated and moved items).
- Your server must provide access to items based on ID. For example via https://serv/ID/{1234567890} URL. This is required to sync with macOS and iOS devices.
To support synchronization the library provides SynchronizationCollection, Bind and ChangedItem interfaces. You will implement Bind and ChangedItem on all items and SynchronizationCollection on folder items that support synchronization of its content. Typically you will implement SynchronizationCollection on your root folder and this will enable synchronization on your entire hierarchy.
The Bind interface provides item getId() and getParentId() methods. The ChangedItem interface provides a single getChangeType() method. In the example below, for demo purposes, we will mark deleted items as hidden, so we can find them later, when changes are requested:
public class ChangedItem: HierarchyItem, Bind, ChangedItem {
...
@Override
public Change getChangeType() {
return isHidden(getFullPath()) ? Change.DELETED : Change.CHANGED;
}
public String getId() {
return FileSystemExtension.getId(getFullPath().toString());
}
public String getParentId() {
return FileSystemExtension.getId(getFullPath().getParent().toString())
}
}
The SynchronizationCollection interface provides getChanges() method that returns all changes below this folder since provided sync-token. Each item in the list of changes retuned by this method must implement Bind and ChangedItem interfaces. Below is the example of this method implementation based on file USN. USN changes across entire file system during every item change, providing a simple way to implement sync-token in case of file system back-end.
final class FolderImpl extends HierarchyItemImpl implements Folder, Search, Quota, ResumableUploadBase, SynchronizationCollection {
...
public Changes getChanges(List<Property> propNames, String syncToken, boolean deep, Long limit) throws ServerException {
DavChanges changes = new DavChanges();
Long syncUsn = null;
Long maxUsn = 0L;
if (!StringUtil.isNullOrEmpty(syncToken)) {
syncUsn = Long.parseLong(syncToken);
}
List<Pair<HierarchyItemImpl, Long>> children = new ArrayList<>();
try (Stream<Path> stream = Files.walk(Paths.get(getFullPath().toString()), deep ? Integer.MAX_VALUE : 1)) {
for (String path : stream.map(Path::toString).collect(Collectors.toSet())) {
String childPath = StringUtil.trimEnd(getPath(), "/") + "/" + StringUtil.trimStart(path.substring(getFullPath().toString().length()).replace(java.io.File.separator, "/"), "/");
HierarchyItemImpl child = FolderImpl.getFolder(childPath, getEngine());
if (child == null) {
child = FileImpl.getFile(childPath, getEngine());
}
if (child != null) {
children.add(new ImmutablePair<>(child, FileSystemExtension.getUsn(path)));
}
}
} catch (IOException ex) {
throw new ServerException(ex);
}
if (limit != null && limit == 0) {
maxUsn = children.stream().mapToLong(Pair::getValue).max().orElse(0);
} else {
for (Pair<HierarchyItemImpl, Long> item : children.stream().sorted(Comparator.comparingLong(Pair::getValue)).collect(Collectors.toCollection(LinkedHashSet::new))) {
// Don't include deleted files/folders when syncToken is empty, because this is full sync.
if (!(item.getLeft().getChangeType() == Change.DELETED && StringUtil.isNullOrEmpty(syncToken))) {
maxUsn = item.getValue();
if (syncUsn == null || item.getValue() > syncUsn) {
changes.add(item.getLeft());
}
if (limit != null && limit <= changes.size()) {
changes.setMoreResults(true);
break;
}
}
}
}
changes.setNewSyncToken(maxUsn.toString());
return changes;
}
}
In addition to the list of changes the getChanges() method returns a new sync-token. The client will store the sync token and use it to request changes during next request, for example when the server will notify the client that changes are available.
In case the syncToken parameter is null, this means the client is doing an initial synchronization. You must return all items under this folder. Note that you do not need to return deleted items in this case.
The limit parameter indicates maximum number of items that should be returned from this method. If you truncate the results, you must indicate that more changes are available by by passing true to Changes.setMoreResults().
If the limit is null, this means the client did not specify limit and you must return all items.
If the limit parameter is 0, this means the client needs the sync-token only. You should not return any items in this case, instead you will only set the new token by calling Changes.setNewSyncToken() method and the Engine will return it to the client. This scenario is typical for clients with on-demand population support. In case of the on-demand population, during start, the client must first read the sync-token from server and than it can start populating some of its folders.
The deep parameter indicates if this folder children should be returned or entire hierarchy under this folder.
See Also:
- Implementing Incoming Synchronization in IT Hit User File System
- WebDAV Drive Sample for Windows
- WebDAV Drive Sample for macOS