Comparative risks of venous thromboembolism among users of oral contraceptives containing drospirenone and levonorgestrel
=========================================================================================================================

* Klaas Heinemann
* Lothar A J Heinemann

## Background

Two case-control studies, both with the involvement of the Boston Collaborative Drug Surveillance Program, have recently been published in the *British Medical Journal* (Jick *et al.*1 and Parkin *et al.*2) in which increased risks of venous thromboembolism (VTE) among users of drospirenone (DRSP)-containing oral contraceptives (OCs) compared with levonorgestrel (LNG)-containing OCs were reported. Both studies used electronic medical databases [USA: managed care plans (Pharmetrics)1; UK: general practices (GPRD)2].

Risk assessments of VTE in a predominantly young and healthy population identified in electronic databases always face similar methodological problems concerning the validity and completeness of the recorded diagnoses, the selection of comparison groups, and the completeness and validity of the information on potential confounders.3,–,5

These problems are reminiscent of the issues experienced during the ‘third vs second generation’ pill crisis. Recently in Austria, where the impact of the crisis in the mid-1990s on OC use was negligible, a study was conducted in which the risk of VTE for ‘third vs second generation pills’ was investigated. The methods were the same as those used 10 years earlier. However, over the intervening 10 years the characteristics of the third- and second-generation pill user populations had grown similar, and – importantly – the elevated risks reported in the mid-1990s were not found.6

In the two database studies now reported it is unlikely that the characteristics of the populations of DRSP- and LNG-containing OC users were sufficiently similar to permit valid comparisons, for the reasons given below.

## Under-ascertainment of VTE incidence

There was major under-ascertainment of the incidence of VTE, as has also been the case in previous database studies. Among OC users it is now well established that the incidence is of the order of 9–10 per 10 000 woman-years (WY), and in some studies it is as much as 13.5 per 10 000 WY.7 By contrast, in database studies the incidence in OC users has ranged from 0.7 to 3.8 per 10 000 WY. As has been the case in previous database studies, the two new studies were again characterised by major under-ascertainment of cases of VTE.

In the study of Jick *et al.,*1 among OC users the overall incidence of idiopathic plus non-idiopathic cases of VTE was 5.2 per 10 000 WY, or about half of what has been documented in well-designed cohort studies.8 For DRSP users the incidence was not mentioned, but it can be estimated to have been about 7.9 per 10 000 WY – close to the overall incidence among all OC users in studies not derived from databases. For LNG users the incidence was 3.2 per 10 000 WY, and two- to three-fold lower than in studies using patient-reported outcomes with 100% validation of cases. The latter rate is no different from the baseline incidence of VTE in non-users of OCs in cohort studies.8

The incidence rates of idiopathic VTE reported in Parkin *et al.*2 were even lower than those reported by Jick *et al*. Parkin *et al.* did not mention the number of non-idiopathic cases that they excluded; however, the rules for exclusion were broadly the same to those used by Jick *et al*. If it is assumed that about 39% of the cases were defined as idiopathic, as in Jick *et al.*, the total incidence of idiopathic plus non-idiopathic VTE in the DRSP-exposed women would have been about 5.9 per 10 000 WY, and in the LNG-exposed 2.3 per 10 000 WY. Those incidence rates were not only lower than in the abovementioned cohort study,8 they were also much lower than in a recently published database study from Denmark.9

## Diagnostic bias

The substantial under-ascertainment of VTE cases in the Jick *et al.* study made bias virtually inevitable. Over 90% of cases of VTE cause no or only mild, unspecific symptoms,10 and the most common presentation, a swollen leg, may or may not be VTE. In OC users awareness that a swollen leg could be VTE, which then is followed by the decision to carry out diagnostic procedures, can readily be selectively influenced by negative publicity about a certain OC. In the USA considerable publicity has been given to the allegation that DRSP-containing OCs cause more VTE than do other OCs. That publicity has been reinforced by litigation. Awareness of the possibility of VTE, followed by diagnostic work-up, is likely to have been greater in users of DRSP-containing OCs than in users of other OCs, and the major under-ascertainment of cases maximised the opportunity for this to occur.

As in the USA, in the UK publicity has been given to the allegation that DRSP is more thrombogenic than other progestogens. Again, given the marked under-ascertainment of VTE cases, it was virtually inevitable that the data would be biased by a tendency to selectively diagnose VTE more commonly in DRSP-containing OC users than in users of other OCs.

Not only was there under-ascertainment of VTE incidence in the Parkin study, but there were also substantially different odds ratios for the risks of deep venous thrombosis (DVT) and pulmonary embolism (PE) reported: for DVT the odds ratio for DRSP vs LNG was four-fold higher than the corresponding odds ratio for PE. Yet the incidence of PE should directly have correlated with the incidence of DVT. The non-correspondence therefore constitutes quantitative evidence of substantial diagnostic bias: among women who presented with mild or non-specific symptoms it is likely that physicians more commonly initiated diagnostic procedures among DRSP users compared to LNG users.

## Validation of VTE diagnosis

Jick *et al.* did not validate the diagnosis of VTE, and they claimed that potential misclassification of recorded VTE diagnoses was likely to be non-differential among users of DRSP-containing and other OCs. However, no data were provided to support that claim. For the reasons given above it is likely that special attention was paid to DRSP-containing OCs. Moreover, it is well established that when a class of drugs (i.e. OCs) is known to increase the risk of a well-established side effect (i.e. VTE), there is a selective tendency to more commonly suspect and more commonly diagnose that side effect in users of the most recently introduced compound.11,–,13 And still further, DRSP-containing OCs were introduced with a special label that emphasised the risk of VTE. That emphasis could further have augmented a selective tendency to suspect and diagnose VTE in DRSP-exposed women.

In the study of Parkin *et al.* over one third of cases were not validated. When only the validated cases were analysed, no statistically significant result was evident. In the light of the missing validation data (Jick *et al.*) or partially missing data (Parkin *et al.*), the under-ascertainment of actual cases of VTE might have been even more profound than estimated above. Problems with the validation of reported outcomes in database studies, and the limitations to the utility data, have been widely discussed.3,–,5 In the present instance the under-ascertainment of cases, coupled with the failure to validate them entirely in the one study1 and partly in the other,2 is good reason to question whether the results can be interpreted at all.

## Duration of use effect

Neither Jick *et al.* nor Parkin *et al.* adequately evaluated the effects of the duration of use. It is well established that the risk of VTE among OC users is greatest in the initial 3 months of use, after which there is a rapid decline to a lower but still elevated level of risk (Figure 1).8 14,–,17 This pattern was not evident in the study of Jick *et al.*,1 and cases of VTE exposed to OCs for <3 months were under-represented. Thus it is likely that under-ascertainment of VTE in the initial months of OC use was even more marked than later on. The failure to identify an early use effect is sufficient, on its own, to invalidate the findings in the study of Jick *et al*. Parkin *et al.* adjusted for confounding by duration of use, but they did not provide the data needed to evaluate whether there was an early use effect in their study.

![Figure 1](http://jfprhc.bmj.com/https://srh.bmj.com/content/familyplanning/37/3/132/F1.medium.gif)

[Figure 1](http://jfprhc.bmj.com/content/37/3/132/F1)

Figure 1 
Venous thromboembolism (VTE) risk over time following start of combined oral contraceptive use. Original figure derived from data in Reference 8. WY, woman-years.

## Exclusion of ‘non-idiopathic’ VTE cases

Jick *et al.* and Parkin *et al.* have claimed that their findings were valid because they excluded so-called ‘non-idiopathic’ cases of VTE, whereas studies reporting no difference between DRSP vs other OCs were not valid because they included predisposed cases. That claim is demonstrably incorrect: the occurrence of VTE in obese women, or in women with a family history of VTE, or in women with other predisposing factors, sometimes unknown (e.g. Factor V Leiden) can hardly be designated as ‘idiopathic’. Moreover, in the study of Jick *et al.* obesity, hypertension, atherosclerosis, hyperlipidaemia/hypercholesterolaemia, asthma, emergency room visits, and physician visits were all more common among cases of VTE than among controls. Without any question, general ill health predisposes to VTE, and the notion that an entity given the label ‘idiopathic VTE’ can be defined, and be reliably identified, is illusory. Furthermore, the combination of two or more risk factors – such as Factor V Leiden mutation and OC use – leads to an over-additive risk.18 Therefore, the analysis of ‘idiopathic’ VTE underestimates the risk under real-life conditions. If the objective is to determine the risk of VTE in the OC-using population at large, it is more valid to record potential confounders as fully as possible and to make proper allowance for them in the analysis.

## Lack of information on important prognostic factors

As with database studies in general, in both of the recently reported studies information on important risk factors for VTE was missing. In the study of Jick *et al.* no information was available on body mass index (BMI) (all that was recorded was the presence or absence of a diagnosis of obesity, which was not even defined) (Figure 2); there was also no information on a family history of VTE. Active surveillance studies have shown that both prognostic factors can be confounders.8 In the study of Parkin *et al.* there was no information on a family history of VTE, and statistical imputation was used to allow for missing data on BMI. The validity of imputation is based on the assumption of randomness, and for BMI that assumption may not have been tenable, since a high BMI has been shown to be a determinant of DRSP use.8

![Figure 2](http://jfprhc.bmj.com/https://srh.bmj.com/content/familyplanning/37/3/132/F2.medium.gif)

[Figure 2](http://jfprhc.bmj.com/content/37/3/132/F2)

Figure 2 
Risk of venous thromboembolism (VTE) by body mass index (BMI). Original figure derived from data in Reference 8. WY, woman-years.

## Conclusions

It needs to be stressed that database studies such as the two studies considered here have been beset with many more problems, and with less methodological transparency, than have been cohort and case-control studies specifically designed to compare VTE risks among users of different OCs, carried out in the population at large.19 Several recent studies have not identified differences in the risk of VTE for DRSP vs other OCs, including OCs containing LNG.8 20 21

The two recent studies suffered from the same methodological limitations as previously published studies based on electronic databases. It is therefore hardly surprising that the results replicate previous findings from studies that have employed this methodology. By contrast, studies designed specifically to evaluate VTE risk among users of different OCs have proven to be more valid, and more informative. Discussions on how to improve the methodology of electronic database studies are urgently needed.22

In an editorial accompanying the studies of Jick *et al.* and Parkin *et al.,* Dunn has claimed that they accord with two previously studies,9 16 and in an editorial accompanying the earlier studies23 he claimed that those studies were ‘remarkably concordant’. In fact, as has been pointed out,4 the two earlier studies were not concordant. On the contrary, they were statistically incompatible.

We conclude that the findings in both of the recent studies are difficult to interpret because of major under-ascertainment of VTE cases, the likelihood of biased diagnosis of VTE conditional on DRSP use, failure to identify the markedly elevated risk of VTE soon after commencement of use – or to make detailed allowance for it, and failure to adequately control potential confounding.

## Footnotes

*   Competing interests Lothar Heinemann was an investigator for several studies mentioned in this commentary. He has also acted as a consultant for manufacturers of oral contraceptives in the past, but has not been actively involved in such consultancy work since 2006. Klaas Heinemann is an investigator on several post-authorisation studies that are supported by unconditional grants from manufacturers of hormonal contraceptives.

*   Provenance and peer review Commissioned; externally peer reviewed.

## References

1.  Jick SS, Hernandez RK. Risk of non-fatal venous thromboembolism in women using oral contraceptives containing drospirenone compared with women using oral contraceptives containing levonorgestrel: case-control study using United States claims data. BMJ 2011;342:d2151.
    
    [Abstract/FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDIvYXByMjFfMi9kMjE1MSI7czo0OiJhdG9tIjtzOjI5OiIvZmFtaWx5cGxhbm5pbmcvMzcvMy8xMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

2.  Parkin L, Sharples K, Hernandez RK, et al. Risk of venous thromboembolism in users of oral contraceptives containing drospirenone or levonorgestrel: nested case-control study based on UK General Practice Research Database. BMJ 2011;342:d2139.
    
    [Abstract/FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDIvYXByMjFfMi9kMjEzOSI7czo0OiJhdG9tIjtzOjI5OiIvZmFtaWx5cGxhbm5pbmcvMzcvMy8xMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  Grimes DA. Epidemiologic research using administrative databases: garbage in, garbage out. Obstet Gynecol 2010;116:1018–1019.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1097/AOG.0b013e3181f98300&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=20966682&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000283329000002&link_type=ISI) 

4.  Shapiro S, Dinger J. Risk of venous thromboembolism among users of oral contraceptives: a review of two recently published studies. J Fam Plann Reprod Health Care 2010;36:33–38.
    
    [Abstract/FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImZhbWlseXBsYW5uaW5nIjtzOjU6InJlc2lkIjtzOjc6IjM2LzEvMzMiO3M6NDoiYXRvbSI7czoyOToiL2ZhbWlseXBsYW5uaW5nLzM3LzMvMTMyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

5.  Severinsen MT, Kristensen SR, Overvad K, et al. Venous thromboembolism discharge diagnoses in the Danish National Patient Registry should be used with caution. J Clin Epidemiol 2010;63:223–228.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2009.03.018&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=19595569&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000274062400019&link_type=ISI) 

6.  Heinemann LA, Dinger JC, Assmann A, et al. Use of oral contraceptives containing gestodene and risk of venous thromboembolism: outlook 10 years after the third-generation “pill scare”. Contraception 2010;81:401–407.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1016/j.contraception.2009.12.014&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=20399946&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000277300000006&link_type=ISI) 

7.  Heinemann LAJ, Dinger JC. Range of published estimates of VTE incidence in young women. Contraception 2007;75:3328–3336.
    
    

8.  Dinger JC, Heinemann LA, Kühl-Habich D. The safety of a drospirenone-containing oral contraceptive: final results from the European Active Surveillance Study on oral contraceptives based on 142,475 women-years of observation. Contraception 2007;75:344–354.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1016/j.contraception.2006.12.019&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=17434015&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000245969000004&link_type=ISI) 

9.  Lidegaard Ø, Løkkegaard E, Svendsen AL, et al. Hormonal contraception and risk of venous thromboembolism: national follow-up study. BMJ 2009;339:b2890.
    
    [Abstract/FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzkvYXVnMTNfMi9iMjg5MCI7czo0OiJhdG9tIjtzOjI5OiIvZmFtaWx5cGxhbm5pbmcvMzcvMy8xMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. Schwarz T, Siegert G, Oettler W, et al. Venous thrombosis after long-haul flights. Arch Intern Med 2003;163:2759–2764.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1001/archinte.163.22.2759&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=14662630&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000187007700012&link_type=ISI) 

11. Sheldon T. Dutch GPs warned against new contraceptive pill. BMJ 2002;324:869.
    
    [FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjQvNzM0Mi84NjkiO3M6NDoiYXRvbSI7czoyOToiL2ZhbWlseXBsYW5uaW5nLzM3LzMvMTMyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

12. van Grootheest K, Vrieling T. Thromboembolism associated with the new contraceptive Yasmin. BMJ 2003;326:257.
    
    [FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjYvNzM4My8yNTciO3M6NDoiYXRvbSI7czoyOToiL2ZhbWlseXBsYW5uaW5nLzM3LzMvMTMyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

13. Pearce HM, Layton D, Wilton LV, et al. Deep vein thrombosis and pulmonary embolism reported in the Prescription Event Monitoring Study of Yasmin. Br J Clin Pharmacol 2005;60: 98–102.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2125.2005.02373.x&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=15963100&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000229912600014&link_type=ISI) 

14. Suissa S, Blais L, Spitzer WO, et al. First-time use of newer oral contraceptives and the risk of venous thromboembolism. Contraception 1997;56:141–146.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(97)00119-4&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=9347203&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=A1997YB09900002&link_type=ISI) 

15. Suissa S, Spitzer WO, Rainville B, et al. Recurrent use of newer oral contraceptives and the risk of venous thromboembolism. Hum Reprod 2000;15:817–821.
    
    [Abstract/FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaHVtcmVwIjtzOjU6InJlc2lkIjtzOjg6IjE1LzQvODE3IjtzOjQ6ImF0b20iO3M6Mjk6Ii9mYW1pbHlwbGFubmluZy8zNy8zLzEzMi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

16. van Hylckama Vlieg A, Helmerhorst FM, Vandenbroucke JP, et al. The venous thrombotic risk of oral contraceptives, effects of oestrogen dose and progestogen type: results of the MEGA case-control study. BMJ 2009;339:b2921.
    
    [Abstract/FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzkvYXVnMTNfMi9iMjkyMSI7czo0OiJhdG9tIjtzOjI5OiIvZmFtaWx5cGxhbm5pbmcvMzcvMy8xMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. Lidegaard Ø, Edström B, Kreiner S. Oral contraceptives and venous thromboembolism: a five-year national case-control study. Contraception 2002;65:187–196.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1016/S0010-7824(01)00307-9&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=11929640&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000174908400001&link_type=ISI) 

18. Vandenbroucke JP, Koster T, Briët E, et al. Increased risk of venous thrombosis in oral-contraceptive users who are carriers of factor V Leiden mutation. Lancet 1994;344: 1453–1457.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(94)90286-0&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=7968118&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=A1994PT99500005&link_type=ISI) 

19. Reid RL, Westhoff C, Mansour D, et al. Oral contraceptives and venous thromboembolism consensus opinion from an international workshop held in Berlin, Germany in December 2009. J Fam Plann Reprod Health Care 2010;36:117–122.
    
    [FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czoxNDoiZmFtaWx5cGxhbm5pbmciO3M6NToicmVzaWQiO3M6ODoiMzYvMy8xMTciO3M6NDoiYXRvbSI7czoyOToiL2ZhbWlseXBsYW5uaW5nLzM3LzMvMTMyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

20. Dinger J, Assmann A, Möhner S, et al. Risk of venous thromboembolism and the use of dienogest- and drospirenone-containing oral contraceptives: results from a German case-control study. J Fam Plann Reprod Health Care 2010;36: 123–129.
    
    [Abstract/FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImZhbWlseXBsYW5uaW5nIjtzOjU6InJlc2lkIjtzOjg6IjM2LzMvMTIzIjtzOjQ6ImF0b20iO3M6Mjk6Ii9mYW1pbHlwbGFubmluZy8zNy8zLzEzMi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

21. Seeger JD, Loughlin J, Eng PM, et al. Risk of thromboembolism in women taking ethinylestradiol/drospirenone and other oral contraceptives. Obstet Gynecol 2007;110:587–593.
    
    [CrossRef](http://jfprhc.bmj.com/lookup/external-ref?access_num=10.1097/01.AOG.0000279448.62221.a8&link_type=DOI) 
    
    [PubMed](http://jfprhc.bmj.com/lookup/external-ref?access_num=17766604&link_type=MED&atom=%2Ffamilyplanning%2F37%2F3%2F132.atom) 
    
    [Web of Science](http://jfprhc.bmj.com/lookup/external-ref?access_num=000249132700008&link_type=ISI) 

22. Heinemann K Jr.. Problems associated with comprehensive detection/validation of unexpected ADRs in a population. PMR Newsletter 2010;5:1–3,9–10. [http://www.prmnewsletter.org/file/heinemann-2.pdf](http://www.prmnewsletter.org/file/heinemann-2.pdf) [accessed 12 May 2011].
    
    

23. Dunn N. Oral contraceptives and venous thromboembolism. BMJ 2009;339:b3164.
    
    [FREE Full Text](http://jfprhc.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzkvYXVnMTNfMi9iMzE2NCI7czo0OiJhdG9tIjtzOjI5OiIvZmFtaWx5cGxhbm5pbmcvMzcvMy8xMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)