Summary of papers (n=18) included in the scoping review
| Author (date) | Country | Study aims | Participants | Study design | Method of assessment | Technology/ approach | Quality summary | MMAT score* |
| Fertility and reproductive health tracking | ||||||||
| Gambier-Ross et al (2018)21 | UK | To explore women’s uses of and relationships with fertility tracking apps | n=240 (survey); n=11 (interviews); women aged 18+ years | Mixed methods exploratory study | Online survey using SurveyMonkey distributed via social media including Facebook and Twitter; participants for follow-up interviews derived from survey | Fertility tracking apps | Survey was only live for 5 days; convenience sample therefore not necessarily generalisable to whole population; only 37% of sample had used a fertility tracking app | 6.5 |
| Epstein et al (2017)22 | USA | To understand menstrual cycle tracking practices and to examine users’ experiences to explore potential design opportunities for apps | n=687 (survey completion), n=12 (interviews) | Experimental study | Three sources were used for data collection: online app reviews, a survey of women's tracking practices, and follow-up interviews with some survey participants | 01/2016 the term period tracker was searched across the Apple and Android App stores. 12 most reviewed apps, 2000 most recent app reviews were downloaded. Review scores were ignored, and focus was on open-ended review text | Results likely to exaggerate technology use due to HCI focus; demographic bias in favour of women who identified as White or Asian and from Western, industrialised, rich and democratic countries | 6.5 |
| Bretschneider et al (2015)23 | Germany | To propose a new approach for tracking menstruation via a app | Review of apps (n=13), online survey (n=196 women) and in-laboratory usability testing with participants (n=5) | Experimental study | Review of apps (comments from users focusing on common pitfalls). Google Play and Apple iTunes Stores were searched. Online survey and usability (UX) testing with pre-existing period tracking apps. Laboratory based. From June 2014 – first prototype was released | NetMoms Cycle Calendar App (iOS and Android) | Laboratory-based study only; usability testing conducted with 5 participants only | 4 |
| Haile (2018)24 | Egypt, Ghana, India, Jordan, Keyna, Nigeria and Rwanda | To market test the CycleBeads app in developing countries to assess whether women were interested in and able to use a fertility app, learn the profile of users, and their purpose for using it | 356 520 app downloads | In-app micro-surveys over a 6-cycle period | Culturally-appropriate Facebook campaigns were used to market the app to women aged 18–39 years using Android smartphones. Descriptive statistics were derived from survey questions | CycleBeads app (Android) | All data self-reported; in India app is available in Hindi but survey questions were not asked in the Hindi version; Indian data may therefore be skewed | 7 |
| Blödt et al (2018)25 | Germany | To investigate the effectiveness of app-based self-acupressure in women with menstrual pain | Women (n=221) aged 18–34 years, mean age: 24.0±3.6 years | Two-armed, randomised, pragmatic trial | Self-reporting of cramping pain of 6+ on a numeric rating scale. Women were randomised to either app-based self-acupressure (n=111) or followed usual care (n=110) – with a total of 6 follow-up cycles All questionnaires were integrated into the app. Baseline data were taken | AKUD app | Longer follow-up time might have provided more information on long-term use; impact of treatment may be overestimated due to short follow-up; high risk of bias due to lack of blinding | 7 |
| Lee et al (2015)26 | South Korea | To understand human factors of using mobile application to manage woman’s health in menopause and to identify design consideration based on the user study and propose a system design for apps to assist women in menopause | Women (n=9) aged 45–60 years. Experienced menopause symptoms within 5 years; n=4 ‘family members’ | Qualitative user study | Participants recruited via online menopause community; semi-structured interviews and focus group carried out | Development of guideline for designing health management system for women experiencing menopause | Relatively small study; number of focus groups and interviews not clearly defined by authors; limited information on recruitment via online community | 7 |
| Pregnancy planning | ||||||||
| Sodha et al (2017)27 | Japan | To clarify how the data obtained from a self-tracking health app for female mobile phone users can be used to improve the accuracy of prediction of the date of next ovulation | 155 000 users were screened from 8 000 000. Age range 20–45 years, mean age 32.94 years | Secondary data analysis | Data from 7043 women via mobile phone app (via a healthcare service) was analysed; between the length of the menstrual cycle, follicular phase and the luteal phase | Luna Luna app available as part of commercial Japanaese women’s health service | Two of the authors are employees of the Luna Luna app developer, MTI Ltd, and both have a patent pending; profile data not available for all users; potential selection bias | 7 |
| Freis et al (2018)28 | Germany | To develop a scoring system for rating menstruation apps that claim to support conception; to pilot 12 apps currently available in German and English using that scoring system | 12 menstruation cycle apps | Evaluation study | Assessment of plausibility of cycle apps by considering the state-of-the-art. Developing a scoring system to rate menstruation cycle apps according to eight criteria | Menstruationskalendar Pro, Flo Menstruationskalender, Clue Menstruations, Period Tracker Deluxe, Maya-Mein Periodentracker, WomanLog-Pro-Kalender, Ovy; Natural Cycles; myNFP; Female Cycle, Lily and Ovuview apps | Study focuses on plausibility rather than efficacy; apps did not evaluate privacy, data protection, security or cost | 4 |
| Setton et al (2016)29 | USA | To evaluate the validity of fertility websites and applications (apps) by comparing the predicted fertile window of these modalities to the actual fertile window of a standard 28-day cycle | Two studies. Study #1: top 20 websites, search terms: “ovulation calendar” and “fertility calendar” Study #2: apps, search term “fertility calendar” | Descriptive study | Top 20 free apps were downloaded and last menstrual period of 1 January 2015, and a 28-day cycle length with 4 days of menstruation were used. Prediction of data: “predicted dates of ovulation were compared with an assumed actual date of ovulation of cycle day 15, 15 January 2015. If a date of ovulation was predicted, it was deemed accurate if this date resulted or inaccurate if any other date resulted. The fertile windows were compared with an assumed actual fertile window consisting of the date of ovulation and the preceding five cycle days, 10–15 (10–15 January)” | Samsung Galaxy phone used, using the Google Play and Apple iOS stores | Only free apps were included using a strategy of identifying the top 20 apps from Google Play and Apple iStore. Paid apps were excluded; study assumes ‘perfect cycle’; descriptive study only | 5 |
| Moglia et al (2016)30 | USA | To identify smartphone menstrual cycle tracking applications (apps) and evaluate their accuracy, features, and functionality | Evaluation study | “An application accurate if menstrual cycle predictions were based on average cycle lengths of at least three previous cycles, ovulation (when included) was predicted at 13–15 days before the start of the next cycle, and the application contained no misinformation” | One app store was searched: Apple iTunes store for free menstrual cycle tracking apps | Only free apps were included; paid apps were excluded. Availability, content, features and functionality may have changed since the evaluation although app version information is given to allow for independent assessment | 7 | |
| Pregnancy prevention | ||||||||
| Berglund Scherwitzl et al (2015)17 | Sweden | To evaluate the ability of a novel web and mobile application to identify a woman’s ovulation day and fertile window, in order to use it as a method of natural birth control | 317 women aged 18–39 years | Retrospective study | Basal body temperatures, ovulation test results and date of menstruation into the application | NaturalCycles app (iOS and Android) | Two authors are the app developers and owners of NaturalCycles Nordic AB; retrospective study | 6.5 |
| Berglund Scherwitzl et al (2016)18 | Sweden | To investigate the contraceptive efficacy of the mobile application by evaluating the perfect- and typical-use Pearl Index | n=22 785 Paying users of the app between 1 August 2014–2016 Registered n=26 967 Included in the study n=22 785 Contributed N3 months n=19 534 Contributed N6 months n=15 224 Contributed N12 months n=6944 Contributed N18 months 2684 | Prospective observational study | Users had to log at least 20 days of data, such a daily log can contain any combination of menstruation, basal body temperatures, luteinising hormone test, pregnancy test result, sexual activity and personal notes "…a total of 18 548 woman-years of data into the application. We used these data to calculate typical- and perfect-use Pearl Indexes, as well as 13-cycle pregnancy rates using life-table analysis" | NaturalCycles app (iOS and Android) | Two authors are the app developers, founders and shareholders of NaturalCycles Nordic AB; another author is an employee of this company; the study was funded by NaturalCycles Nordic AB; retrospective study | 7 |
| Berglund Scherwitzl et al (2017)19 | Sweden | To retrospectively evaluate the effectiveness of a fertility awareness-based method supported by a mobile-based application to prevent unwanted pregnancies as a method of natural birth control | n=4054 Age groups: <20 years (n=54, 1%) 20–24 years (n=1263, 32%) 25–29 years (n=1729, 43%) 30–34 years (n=672, 17%) 35–39 years (n=205, 5%) ≥40 years (n=70, 2%) | Retrospective evaluation study | "application’s efficiency as a contraceptive method was examined on data from 4054 women who used the application as contraception for a total of 2085 woman-years" 10-item survey completed 3 weeks prior to the study ending. A total of n=1186 women completed the survey | NaturalCycles app (iOS and Android) | Two authors are the app developers, founders and shareholders of NaturalCycles Nordic AB; another author is an employee of this company; two authors serve on NaturalCycles Nordic AB’s medical advisory board and have received honorarium; the study was funded by NaturalCycles Nordic AB | 7 |
| Li et al (2016)20 | USA | To propose a new approach to estimating fertile days, using data recording from the first day of menses | Study #1: n=68 sexually active women (171 cycles) with either intrauterine device of tubal ligation. Up to three menstrual cycles of data provided. Study #2: n=221 (696 cycles), planned to become pregnant by discontinuing birth control, no known fertility problems | Retrospective analysis; simulation studies | Data taken from two existing studies. Study 1: WHO study of the ovulation method of natural family planning. Study 2: North Carolina – Early Pregnancy Study (EPS) | Dot app | One author is an employee of the app developer, Cycle Technologies; study funded by Cycle Technologies; data based on studies focusing predominantly on White, young and well-educated women which may not be typical of the population of future Dot users; efficacy may be overestimated due to assumption that women would not have unprotected intercourse on fertile days | 7 |
| Jennings et al (2019)16 | USA | To investigate the effectiveness of the Dot app in calculating perfect- and typical-use failure rates | 718 women using Dot to prevent pregnancy (6616 cycles between February 2017 and October 2018) | Prospective 13-cycle observational study | Users provided data on menstrual start dates, daily sexual activity and prospective intent to prevent pregnancy; pregnancy was determined through participant-administered urine pregnancy tests and/or written/verbal confirmation; perfect- and typical-use failure rates were calculated using multi-censoring, single decrement life-table analysis, and sensitivity, attrition and survival analyses were carried out | Dot app (iOS and Android) | One author is a former employee of app developer Cycle Technologies; data will be made available through the US Open Data Act | 7 |
| Koch et al (2018)31 | Germany | To investigate whether an app improves usability of a medical device | Women (n=125), cycles (n=2076), average age of women 29 years | Evaluation feasibility study | Use of DaysyView, a free mobile app to augment the use of the Daysy fertility monitor. All Daysy users with a DaysyView account were invited to the study and asked to complete an online questionnaire. The survey recorded birth, height and weight | Daysy and DaysyView app (iOS and Android) | The study is funded by Valley Electronics AG, Zurich, Switzerland, the manufacturer of Daysy and DaysyView; one of the authors is an employee of Valley Electronics; due to retrospective study design researchers could not control data collection; the majority of participants used the device for less than 13 cycles due to the short time on market of the device | 7 |
| Starling et al (2018)35 | USA | To explore the user profile and preferences in fertility apps for preventing pregnancy among women that have used, currently use, or intend to use a fertility app | 1000 female users of fertility apps aged 18–39 years | Exploratory pilot study | Participants recruited to study via Facebook. Data collected about interest in and current use of fertility apps; user intentions and goals in relation to pregnancy prevention; survey included questions on reproductive cycle and knowledge about fertility | Use and preferences using a fertility app rating the appeal of app features, functionality and reputation | Use of social networking platform and self-reporting limits generalisability of findings to broader population; diversity of fertility app users outside of USA not represented; survey questions not validated; small sample size for actual fertility app users compared with intended users | 7 |
| Duane et al (2016)36 | USA | To develop a rating tool with specific criteria to quantify an app’s response to real cycle data based on the clinical guidelines evaluated in level 1 studies | 7 cycles were observed, based on real data (determine accuracy) | Evaluation study | The rating system was developed based on criteria used by the Family Practice Management. A 5-point Likert scale was used against a 10-item predefined criteria | apps (n=95) across iTunes, Google and Google Play searches | Free and paid-for apps were included in study; study acknowledges that user behaviour is important to performance of apps, but this was not included in evaluation | 6.5 |
*MMAT checklist scoring ranges from 0 to 7,with 7 being the highest quality score.
HCI, human–computer interaction; MMAT, Mixed Methods Assessment Tool; WHO, World Health Organization.