Questionnaire for an App

Version: master

Note: For Capitalized Terms consult our Glossary.

Status of the assay

  • What is the status of this assay?

  • This assay is current as of which date?

About the App

Name and description of the App

  • Name of the App

  • Short description of the App

  • Icon of the App

  • Languages supported by the App

  • URL of the App on iTunes

  • URL of the App on Google Play

  • Website of the App in English

  • Website of the App in the local language (if applicable)

  • What’s the region and purpose of the app (for one-line overview)

About the App Creators

  • Who develops the App?

  • Who operates the App?

  • Who sponsors the App?

  • Who governs the App?

  • What third-party Service Providers are used for the App, and what is their contribution or role with respect to the App? Are they under legal obligations consistent with the needs of the App?

About App Users

  • Who are the intended users for the App? Where are they located?

  • Are there technical, geographical, legal or other limits to who can use the App?

  • How many App Users are currently using the App?

Goals and benefits of the App

  • What are the stated goals of the App?

  • Are there any others goals, not stated by the App Creators, that they are known to also accomplish with this App, or that they could also accomplish with this App in the future?

  • Are there any other goals that others (not App Creators) could also accomplish because this App exists, or is used by certain Users?

  • Are there notable side effects in the use of this App?

Societal context of using the App

  • Is usage of the App required under some circumstances? If so, by whom? What are the consequences of not using it?

  • Are there non-trivial incentives (e.g. financial, access) for using the App? Are there social pressures to use the App?

  • Are there social pressures on the App User resulting from the use of the App, or from information shown by the App? (E.g. if the App indicates that the App User has likely been infected.)

  • Are there social pressures on anybody resulting from information shown by the App run by another App User? (e.g. pressures on an App User’s family or friends if the App identifies the App User as likely infected)

  • Is the App available in all languages and localizations most appropriate for the intended App User population?

  • Is the App accessible?

App features

  • What are the main features of the App?

Technology

  • Describe the principle of operation of the App. This includes technology as well as people, operations, health system and the like.

  • What Architecture does the App use?

  • Is source code of the App available?

  • If the source code is available, under which license is it available?

  • If source code is available, where can it be found?

  • If it does, what kind of Cloud Component does the App use?

  • If it does, what approach does the App take to contact tracing?

  • What are the key non-standard communication protocols the App uses? Explain. (These are highly dependent on the App’s features.)

  • Is the App based on anonymous, pseudonymous, or fully identified App Users?

  • Can identities of App Users be tied to, or can they be correlated to specific individuals, and if so, by whom?

  • What technical approaches (e.g. cryptography) does the App use to protect all aspects of the App (e.g. confidential information, operational integrity) from Attackers?

  • What data does the App handle? Where in the Architecture is which data stored or processed? Is all data handled by the App strictly required for the stated goals?

  • Is data not required any more for the stated goals promptly deleted?

  • Is the App a standalone system (“stovepipe”) or is it intended to be used in Federation with other Apps created by others? If so, what are the supported Federation technologies (e.g. protocols/standards), operations and governance?

  • Is there an audit trail of what happens in the App that can it be accessed by the App User or entities on their behalf?

  • Are the user-facing components of the App built in a way that minimizes potential user mistakes that could be detrimental towards effectiveness or avoidance of risks and harms for themselves and others?

  • How do new App Users discover, and obtain access to the App?

  • Can the App User deactivate and delete the App?

  • How is user support handled?

  • How is the user experience, user understanding, and technical performance of the App being monitored in the field?

  • Can App Users request a copy of the data that has been retained about them? Is the process simple and quick? Is the obtained data easy to understand, verify and use?

  • Can previous App Users request a permanent deletion of the data collected about them? Is the process simple and quick?

  • Can App Users request a correction of data about them? Is the process simple and quick?

  • Can parents or guardians act on behalf of their children in all aspects of the App?

  • Is there an effective complaint process by which App Users can raise issues with the App, or issues with the impact of the use of the App has on them?

  • Are App Users being educated about what it means to use the App, and give their informed consent prior to using the App?

  • If the App performs several distinct functions, can the App User opt-in to some and opt-out of others?

Identity and privacy

  • How are new App Users onboarded on the App? What information do they need to provide to be able to use the App?

  • How long is collected data retained, and where?

  • Are any Backups being made whose retention is longer than the declared Data Retention Period? How is it guaranteed that Backups are deleted on time?

  • Has a Privacy Impact Assessment been performed, and if so, where can it be obtained? Which recommendations have been implemented, and which not? If no such assessment has been performed, why not?

  • Is the App compliant with local regulations on privacy, in particular on privacy of health-related information?

  • Is the App consistent with global best practices on privacy, in particular on privacy of health-related information?

  • What assurances exist that the App will be shut down promptly when appropriate (e.g. when the pandemic has passed, or better approaches for combating the disease have been found)?

  • Is any data collected by the App transmitted beyond the App? If so:

    • Who is the receiver of the data?
    • What is the data that is being transmitted?
    • What are the terms under which the data is transmitted, and what are the safeguards that guarantee the terms are not being violated?
    • Can the transmitted data be correlated by the received with other data they may have or may be able to obtain?

  • Is any data imported by the App from other sources? If so:

    • What data is being imported, and from which sources?
    • How does that increase the effectiveness of the App?
    • Does it potentially increase risks or downsides of the App, and if so, how?

  • Is there a Privacy Policy, and if so, what type of privacy policy is it?

Security

  • How is Data At Rest being secured? Discuss all locations at which Data At Rest exists.

  • How is Data In Motion being secured? Discuss all transfers between locations at which Data At Rest exists.

  • If an App User’s unlocked mobile phone is stolen, what is the maximum impact of the breach on the App User, other App Users, third parties including the App system itself, and effectiveness against the disease?

  • How are the operations of the App monitored with respect to attempted, or successful, Attacks?

  • What operational approaches does the App use to protect all aspects of the App (e.g. requiring two-factor authentication, approval of commits by a second person) from Attackers?

  • What are the operational procedures for access to highly privileged credentials (e.g. server or encryption root keys)?

  • Can App Users verify their build of the App User (e.g. using technologies such as Reproducible Builds)?

  • Has a process been defined for reporting and responding to a security breach?

  • Which organizations are known to have, or are strongly assumed to have the ability to influence the technology, operations or governance of the App, and thus require that users trust them?

  • Have there been any reports on attempted or successful attacks on the integrity of any aspect of the App? Do the App Creators report on such attempted or successful Attacks?

  • Are there any reports on attempted or successful correlation of any of the data handled by the App with data from outside the App, or any attempted or successful Re-identification of anonymized or pseudonomized data?

  • Are there any reports on attempted or successful Data Poisoning of some of the data handled by the App?

  • Has an independent security review been performed, and if so, where can it be obtained? Which recommendations have been implemented, and which not?

  • What are the procedures and requirements for eligibility of any App Creator employees or contractors to participate in any aspect of the development, operations or governance of the App?

Governance and Transparency

  • How are decisions made about technology and operations of the App?

  • How are decisions made about governance of the App?

  • Is there a public roadmap for the App, and if so, where can it be found?

  • Is there a whistleblower process for people involved in any aspect of the development, operation, or governance of the App?

  • Should assertions by App Creators prove to be false, or their behavior to be negligent, what are the remedies available to App Users?

  • Is technical documentation for the App available, and how complete is it? Is this documentation up-to-date?

  • Is the entire process of App development and operations publicly documented? Is this documentation up-to-date?

Effectiveness

  • Is a minimum penetration of App usage required in some population before the App can start to be effective against the pandemic?

  • What metrics are available that indicate the effectiveness, or lack thereof, of the App with respect to the pandemic so far?

  • How is the performance of the App with respect to effectiveness against the disease being monitored?

  • If the App interoperates or federates with other Apps, what metrics are available that indicates the effectiveness, or lack thereof, of the App when interoperating or federating with others so far?

  • How does the effectiveness of the App compare to the effectiveness of other Apps developed elsewhere with similar functionality? What metrics exist?

Validation

  • Which third parties have researched the effectiveness of this App against the disease? Are their reports publicly available, and if so, where?

  • Which third parties have researched the potential downsides or risks of this App? Are their reports publicly available, and if so, where?

  • Has any third-party audit been performed of the App? Who performed the audit, are their reports publicly available, and if so, where?

  • Are any major discrepancies known between self-assertions by the App Creators and Inference or Audits by third parties?

Information sources for this research

Disclaimer and other notes on the assay

  • Disclaimer and open issues that do not fit into any of the other questions.

  • Any other notes that may be of interest

Changes

  • If this assay is an update to a previous assay, what significant changes (with respect to the key questions for this assay) have been made since?

Overall rating

  • Ratings by self, third parties and any audit for the effectiveness of the App

  • Explanatory comments for the rating of the effectiveness of the App

  • Ratings by self, third parties and any audits for the avoidance of potential risks and downsides of the App

  • Explanatory comments for the rating of the avoidance of potential risks and downsides of the App

  • Issue a recommendation to App Users

  • Recommendations to the App Creators

Report

  • About this App

  • About this report

  • Key findings

  • Sources

  • More information

  • Caution