Skip to content

technical_information.rst

Latest commit

 

History

History
511 lines (334 loc) · 23.4 KB

technical_information.rst

File metadata and controls

511 lines (334 loc) · 23.4 KB

Technical information

Synchronization

Synchronization logic

These steps are performed when synchronizing a collection:

  1. Prepare synchronization: prepare local collection, settings etc.
  2. Query capabilities with HTTP PROPFIND:
    • determine whether the server supports Collection Synchronization
    • CardDAV: determine whether the server supports vCard 4
    • fetch current CTag and sync-token
  3. Process locally deleted resources: if a local resource is flagged as deleted,
    • delete it on the server (HTTP DELETE with If-Match/If-Schedule-Tag-Match set to last known ETag/Schedule-Tag to avoid deleting resources which have been changed on the server in the meanwhile) and
    • then remove it locally
  4. Upload locally modified ("dirty") resources:
    • Assign a random UID (if necessary) and resource name to new resources; prepare contact group and recurring events, if necessary
    • If no previous ETag/Schedule-Tag of the resource is known (i.e. the resource has not been uploaded yet), use HTTP PUT with If-None-Match: * to avoid overwriting a possibly existing resource with the same name
    • If a previous ETag/Schedule-Tag of the resource is known, use HTTP PUT with If-Match/If-Schedule-Tag-Match set to last known ETag/Schedule-Tag to avoid overwriting changes which happend on the server in the meanwhile
    • remember returned ETag/Schedule-Tag as last known ETag/Schedule-Tag; otherwise reset last known ETag/Schedule-Tag
  5. Choose sync algorithm (PROPFIND/REPORT vs. Collection Synchronization):
    • CardDAV: use Collection Synchronization if supported by server, PROPFIND otherwise
    • CalDAV events: use Collection Synchronization if supported by server and past time event limit is disabled, REPORT calendar-query otherwise
    • CalDAV tasks: use REPORT calendar-query
  6. Check whether further synchronization is needed. Only continue when:
    • modifications (uploads/deletions) have been sent to the server, or
    • the sync state (CTag/sync-token) of the collection has changed since last sync, or
    • the PROPFIND/REPORT algorithm shall be used and the sync has been initiated manually
  7. Continue with chosen sync algorithm (see below).

Sync algorithm: PROPFIND/REPORT

  1. Unset present remotely flag for all resources.
  2. List and process remote resources (only names and ETag) using PROPFIND or REPORT (see above).
    • Download resources which have been added/modified remotely in bunches using REPORT addressbook-multiget/calendar-multiget into the local storage.
    • Set present remotely flag for all received resources.
  3. Locally delete all resources which are not flagged as present remotely.
  4. Post-processing: clean up empty contact groups etc.
  5. Save sync state (CTag/sync-token).

Sync algorithm: Collection Synchronization

  1. Was a previous initial sync aborted and is now being continued? → If yes, set initial sync.
  2. Do we have a previous sync-token? → If no, set initial sync.
  3. List and process changes since last sync-token (or all records if no previous sync-token is known) using REPORT sync-collection.
    • Download resources which have been added/modified remotely in bunches using REPORT addressbook-multiget/calendar-multiget into the local storage.
    • Set present remotely flag for all received resources.
  4. If the requested sync-token was invalid:
    • forget the sync-token
    • reset present remotely flags of all local resources
    • set initial sync and continue with 3.
  5. Save sync-token.
  6. Are there further changes on the server (HTTP 507 on collection URL in multiget response)? → If yes, continue with 3.
  7. Only for initial sync: delete all local resources which are not present remotely.
  8. Post-processing: clean up empty contact groups etc.

Conflict handling

Conflicts occur when different versions of a resource are available and it's ambigous which one shall be used. For instance:

  1. A contact exists on the server and has been synchronized to a mobile phone with DAVx⁵ and to a desktop PC with Gnome Evolution.
  2. You modify the contact with Evolution, which immediately uploads it to the server.
  3. You modify the same contact on your mobile device, too.
  4. DAVx⁵ wants to upload the modified contact and finds that it has been changed on the server in the meanwhile. Now there's a conflict of two different versions of this contact.

How DAVx⁵ handles such conflicts:

  • DAVx⁵ relies on HTTP ETag to determine whether a resource has been changed on the server.
  • The server always wins. If a local resource can't be uploaded or deleted safely because it has been modified on the server in the meanwhile, local changes are discarded and the server version is used.
  • DAVx⁵ doesn't involve the user in resolving conflicts (like asking which version shall be used) because it's supposed to run in the background silently.

At the moment, DAVx⁵ doesn't support automatic vCard merging as suggested in vCard 4.

Supported fields

Supported contact properties

Name

These vCard properties are mapped to ContactsContract.CommonDataKinds.StructuredName records:

  • FN ↔ display name
  • N ↔ prefix, given name, middle name, family name, suffix
  • X-PHONETIC-FIRST-NAME ↔ phonetic given name
  • X-PHONETIC-MIDDLE-NAME ↔ phonetic middle name
  • X-PHONETIC-LAST-NAME ↔ phonetic first name

These vCard properties are mapped to ContactsContract.CommonDataKinds.Nickname records:

  • NICKNAME ↔ nick name (types are mapped as TYPE x-values)

Phone number

vCard TEL properties are mapped to ContactsContract.CommonDataKinds.Phone records (phone number).

|vCard_types_mapped|

Email address

vCard EMAIL properties are mapped to ContactsContract.CommonDataKinds.Email records (email address).

|vCard_types_mapped|

Photo

vCard PHOTO properties are mapped to ContactsContract.CommonDataKinds.Photo records.

Because of Android limitations, contact photos with more than 1 MB can't be stored in the Android contacts provider, so DAVx⁵ has to resize large vCard photos to the values given by CONTENT_MAX_DIMENSIONS_URI. This limit does not apply in the other direction (Android → vCard).

Organization

These vCard properties are mapped to ContactsContract.CommonDataKinds.Organization records:

  • ORG ↔ company, department
  • TITLE ↔ (job) title
  • ROLE ↔ job description

Messenger / SIP address

vCard IMPP properties are mapped to ContactsContract.CommonDataKinds.Im (messenger account) and – if the URI scheme is sip: – ContactsContract.CommonDataKinds.SipAddress (SIP address) records.

|vCard_types_mapped|

When importing a vCard, X-SIP values are treated like IMPP:sip:... and stored as SIP address.

Note

vCard NOTE properties are mapped to ContactsContract.CommonDataKinds.Note records (note).

Postal address

These vCard properties are mapped to ContactsContract.CommonDataKinds.StructuredPostal records:

  • ADR ↔ street address, p/o box, extended address, locality, region, postal code, country, vCard 4: formatted address
  • LABEL ↔ vCard3: formatted address

|vCard_types_mapped|

If a vCard doesn't contain a formatted address, it will be generated by DAVx⁵ in this format:

street po.box (extended)
postcode city
region
COUNTRY

Web site

vCard URL properties are mapped to ContactsContract.CommonDataKinds.Website records (Web site).

|vCard_types_mapped|

Event/date

These vCard properties are mapped to ContactsContract.CommonDataKinds.Event records:

  • BDAY ↔ birthday
  • ANNIVERSARY ↔ anniversary

Partial dates without year are supported.

Relation

vCard RELATED properties are mapped to ContactsContract.CommonDataKinds.Relation records (relation).

Not all vCard values have a corresponding Android value and vice versa. Custom relation names are supported.

Contact groups

If the Groups are per-contact categories method is set in the account settings, DAVx⁵ will match contact groups to CATEGORIES. For instance, when a contact is in the groups "Friends" and "Family", this property will be added: CATEGORIES:Friends,Family.

If the Groups are separate vCards method is set in the account settings, DAVx⁵ will use

  • KIND (or X-ADDRESSBOOKSERVER-KIND if the server doesn't support VCard 4) to distinguish between contacts and contact groups, and
  • MEMBER (or X-ADDRESSBOOKSERVER-MEMBER if the server doesn't support VCard 4) to store contact group members.

Custom labels

For some properties, custom labels are supported by vCard property groups. For custom labels, the X-ABLABEL property is used like that:

BEGIN:VCARD
...
davdroid1.TEL:+123456
davdroid1.X-ABLABEL:My Custom Phone
davdroid2.EMAIL:test@example.com
davdroid2.X-ABLABEL:My Custom Email Address
...
END:VCARD

In this example, the phone number +123456 is grouped together with the custom label "My Custom Phone" and the email address test@example.com is labelled "My Custom Email Address".

Unknown properties

Contact properties which are not processed by DAVx⁵ (like X- properties) are retained. When importing a vCard, DAVx⁵ saves all unknown properties. When the respective contact is modified and DAVx⁵ generates the vCard again, it starts with all unknown properties and then adds the known ones.

Protected properties

These vCard properties are processed/generated by DAVx⁵ and cannot be changed by users:

  • PRODID is set to the DAVx⁵ identifier
  • UID is used to identify a vCard (for new vCards, a random UUID will be generated)
  • REV is set to the current time when generating a vCard
  • SOURCE is removed because it doesn't apply anymore as soon as DAVx⁵ generates the vCard
  • LOGO, SOUND are removed because retaining them might cause out-of-memory errors

Supported event properties

Events are stored as CalendarContract.Events in the Android calendar provider. These iCalendar properties are directly mapped to Android fields:

  • SUMMARY ↔ title
  • LOCATION ↔ event location
  • DESCRIPTION ↔ description
  • COLOR ↔ event color (only if enabled in DAVx⁵ account settings)
  • DTSTART ↔ start date/time, event timezone / all-day event
  • DTEND, DURATION ↔ end date/time, event end timezone / all-day event
  • CLASS ↔ :ref:`access level <access-level>`
  • TRANSP ↔ availability (opaque ↔ busy, transparent ↔ free)
  • STATUS ↔ status (confirmed/tentative/cancelled)

All-day events

Events are considered to be all-day events when DTSTART is a date (and not a time). All-day events

  • without end date or
  • with an end date that is not after the start date

are stored with a duration of one day for Android compatibility.

Reminders

VALARM components are mapped to CalendarContract.Reminders records and vice versa.

Reminder methods (ACTION) are mapped to Android values as good as possible.

Recurring events

RRULE, RDATE, EXRULE and EXDATE values are stored in the respective Android event fields. The Android calendar provider uses these fields to calculcate the instances of a recurring event, which are then saved as CalendarContract.Instances so that calendar apps can access them.

Exceptions of recurring events are identified by RECURRENCE-ID. DAVx⁵ inserts exceptions as separate event records with ORIGINAL_SYNC_ID set to the SYNC_ID of the recurring event and ORIGINAL_TIME set to the RECURRENCE-ID value.

Note

DAVx⁵ is not responsible for calculating the instances of a recurring event. It only provides RRULE, RDATE, EXRULE, EXDATE and a list of exceptions to the Android calendar provider.

However, if a calendar app leaves dirty events with zero instances to be synced, DAVx⁵ will silently delete these events and ask the server to do the same.

Group-scheduled events

ATTENDEE properties are mapped to CalendarContract.AttendeesColumns records and vice versa.

Events with at least one attendee are considered to be group-scheduled events. Only for group-scheduled events, the ORGANIZER property

  • is imported from iCalendars to the Android event so that only the organizer can edit a group-scheduled event,
  • is exported from the Android event to the iCalendar.

When you add attendees to an event, the server may send invitations to the attendees (for instance, by email). DAVx⁵ doesn't send invitation emails on its own.

Time zones

When processing a downloaded event, DAVx⁵ normalizes event date/time values to Android-compatible formats before storing them in the Android calendar provider. Outlook/Windows TZIDs (e.g., W. Europe Standard Time) are mapped to IANA/Android TZIDs (e.g., Europe/Berlin).

Stored events use Android/system time zones:

  • DATE-TIME with TZID=...: if the TZID is known to Android, DAVx⁵ keeps the same local date and time, but stores it with Android's/system's timezone definition for that TZID.
  • UTC DATE-TIME (...Z): stored as UTC.
  • floating DATE-TIME (no TZID): stored in the current system default time zone.
  • all-day events (VALUE=DATE): stored as UTC dates, as required by the Android calendar provider.

If a TZID is not available in Android, DAVx⁵ tries to match it to a system time zone by name heuristics. If that is not possible, DAVx⁵ keeps the same point in time and stores the event in the system default time zone instead. Custom VTIMEZONE definitions are therefore used to interpret incoming date and time values, but custom TZIDs are not preserved unless they can be mapped to a system timezone. If an incoming VTIMEZONE contains outdated or conflicting rules for a TZID that Android already knows, the stored event still uses Android's system time zone definition for that TZID, not the definition from the iCalendar.

When generating an event to upload, DAVx⁵ recreates event date/time values from the Android event record:

  • all-day events: exported as DATE values
  • UTC events: exported as UTC DATE-TIME values
  • other timed events: exported as DATE-TIME values with TZID=...

For every referenced non-UTC TZID, DAVx⁵ also generates an outgoing VTIMEZONE from the bundled ical4j time zone registry and minifies it, retaining only the relevant daylight saving observances.

Stored events therefore use Android/system time zone definitions, while generated VTIMEZONE components come from the ical4j library and its bundled time zone definitions shipped with DAVx⁵. If those databases differ, known TZIDs are still usually interpreted by clients via their own TZ database, but edge cases can remain, especially for recurring events with unknown/custom time zones.

Warning

Because the Android calendar provider can only process events with time zones which are available in Android, recurring events in time zones which are not available in Android and their exceptions may not be expanded correctly.

Event classification

iCalendar event classification is mapped to Android's ACCESS_LEVEL like that:

  • no CLASS → ACCESS_LEVEL = ACCESS_DEFAULT ("server default")
  • CLASS:PUBLIC → ACCESS_LEVEL = ACCESS_PUBLIC ("public")
  • CLASS:PRIVATE → ACCESS_LEVEL = ACCESS_PRIVATE ("private")
  • CLASS:CONFIDENTIAL → ACCESS_LEVEL = ACCESS_CONFIDENTIAL (currently not supported by many calendar apps, which will reset the access level to ACCESS_DEFAULT or ACCESS_PRIVATE when the event is edited); additionally, CONFIDENTIAL is stored as original value
  • other CLASS value (x-name or iana-token) → ACCESS_LEVEL = ACCESS_PRIVATE; additionally, the value is stored as original value

In the other direction, the locally stored access level is mapped to CLASS like that:

  • ACCESS_LEVEL = ACCESS_PUBLIC ("public") → CLASS:PUBLIC
  • ACCESS_LEVEL = ACCESS_PRIVATE ("private") → CLASS:PRIVATE
  • ACCESS_LEVEL = ACCESS_CONFIDENTIAL ("confidential", if available in calendar app) → CLASS:CONFIDENTIAL
  • ACCESS_LEVEL = ACCESS_DEFAULT ("server default") →
    • if there is an original value: use that value
    • no CLASS otherwise (same as PUBLIC)

Categories

.. versionadded:: 2.6.2
   In earlier versions, event categories were treated as unknown properties (see below).

iCalendar CATEGORIES are mapped from/to extended properties with these fields:

  • name = categories
  • value = list of category names, separated by backslash (\), for example: Cat A\Cat B\Cat C. If a category name contains a backslash, the backslash will be dropped silenty.

This is the same format as it is used by the AOSP ActiveSync Exchange sync adapter.

Extended properties

These properties are not natively supported by Android, but are synchronized as extended properties with a format defined by DAVx⁵:

  • URL

See :ref:`extended event properties <extended_event_properties>` for more information.

Unknown properties

iCalendar properties which are not processed by DAVx⁵ (like X- properties) are retained (unless they're larger than ≈ 25 kB). When importing an iCalendar, DAVx⁵ saves all unknown event properties as extended property rows. When the respective event is modified and DAVx⁵ generates the iCalendar again, it will include all unknown properties.

Protected properties

These iCalendar properties are processed/generated by DAVx⁵ and cannot be changed by users:

  • PRODID is set to the DAVx⁵ identifier
  • UID is used to identify an iCalendar (for new iCalendars, a random UUID will be generated)
  • RECURRENCE-ID is used to identify certain instances of recurring events
  • SEQUENCE is increased when an iCalendar is modified
  • DTSTAMP is set to the current time when generating an iCalendar

Supported task properties

DAVx⁵ synchronizes VTODO components (= tasks) with the OpenTasks provider, so OpenTasks or tasks.org must be installed for task synchronization.

To use some features (for instance, to see subtasks as indented task) in the UI, you may need another tasks app that is able to access the OpenTasks provider, like aCalendar+.

These properties are synchronized by DAVx⁵:

  • UID
  • SUMMARY, DESCRIPTION
  • LOCATION
  • GEO
  • URL
  • ORGANIZER
  • PRIORITY
  • COMPLETED, PERCENT-COMPLETE
  • STATUS
  • CREATED, LAST-MODIFIED
  • DTSTART, DUE, DURATION
  • RDATE, EXDATE, RRULE
  • CATEGORIES
  • RELATED-TO (used for subtasks)

Unknown properties

See :ref:`unknown properties of events <event-unknown-properties>`.

WebDAV mounts

Detection: DAVx⁵ uses OPTIONS to detect whether a URL is WebDAV-capable. It expects at least compliance class 1 (DAV: 1).

Supported features:

  • Directory listing (PROPFIND)

    • WebDAV quota (RFC 4331) of root directory is shown in DAVx⁵ WebDAV mounts

  • Random-access read (GET with Range headers) for files opened with mode r

    • that have a known file size,
    • that have an ETag or Last-Modified,
    • on Android 8+ only.

    This allows things like seeking in videos or extracting data from a file without reading the whole file.

    A page cache with a page size of 2 MB is used, so files will be requested in pieces of 2 MB, regardless of the actual read operations.

  • Atomic (streaming) read (GET) for files opened with mode r that can't be opened in random-access mode

  • Atomic (streaming) write (PUT) for files opened with mode w

TLS stack (protocol versions, ciphers)

.. versionadded:: 2.5
  DAVx⁵ uses `Conscrypt <https://github.com/google/conscrypt/blob/master/CAPABILITIES.md>`_ to support modern TLS protocol versions and ciphers
  even on older devices. Both your client (DAVx⁵) and the CalDAV/CardDAV server must share at least one cipher, otherwise a ``SSLProtocolException`` will occur.