Does previous use of hormonal contraception affect future fertility?

Hormonal contraception obviously effects a woman's fertility while she is taking it (otherwise it wouldn't be a contraceptive!). When women discontinue taking hormonal contraceptives, often after years of taking them, they notice differences in their menstrual cycles. But it's hard for an individual to know if those changes are a result of the medication, or of age and lifestyle changes that occurred during the years of contraceptive use.

After discontinuing hormonal contraceptives, a non-trivial number of women find that it takes many months to return to a normal cycle, delaying pregnancy achievement. 

• A 2004 study by M.A.M.Hassan and S.R.Killick, from Hull and York Medical School in the UK, found that combined oral contraceptives, IUDs, and injected contraceptives increased the time to achieve pregnancy 2.0-fold, 1.6-fold, and 3.0-fold, respectively. Effects increased with length of contraceptive use, and were stronger in older women and obese women.
• A 2001 study by the Department of Public Health (UK), University of Oxford,  and Institute of Health Sciences (UK), found that 54% of women who discontinued a barrier method were delivered within a year, compared to 39% of women who discontinued an IUD, and 32% of women who discontinued oral contraceptives (the pill). Note that this is the percent of women who delivered, so it's looking at conception in the first 3 months of not contracepting. This study also found a strong link between length of use of contraceptives, and subsequent delay in returning fertility. For example, for women who used an IUD for 6.5 years or more, only 28% delivered a baby at 12 months. At 36 months, only 79% had managed to deliver a child, compared to 91% of short term users.

This affect doesn't have a clear explanation. Perhaps its a direct side affect of the hormonal changes. Perhaps it is because contraceptives cover up symptoms of ill health that women would have treated otherwise.

How does prior use of hormonal contraception affect pregnancy? This is difficult to study, and there are no obvious effects, but some research suggests that there are differences. For example, this CDC study found that previous use of hormonal contraceptives was correlated to an increased risk of gestational diabetes, compared to women who used other forms of contraception prior to pregnancy.