Caffeine is a diuretic and causes calcium to be leached out of bones and excreted. However, the reviews of the medical literature I've seen indicate that the effect is slight and can be readily compensated for with diet. Because caffeine has been implicated as a contributor to osteoporosis in the elderly and other vulnerable populations, though, plenty of folks are worried that the substitution of caffeinated soft drinks for milk during childhood (when the need for calcium is great) might lead to developmental problems.
Smoking is harmful as well.

Just to elaborate on the correct diet: Correct protein consumption, as well as caloric intake, will allow your body to grow to it's fullest.