The short answer is: the jury is still out. Many members of the medical community note the connection as possible but not proven. In animal studies marijuana has been shown to cause low testosterone and higher estrogen, resulting in increased breast tissue. In humans however, this has been harder to prove. Lower testosterone has been noted in some chronic users, but not in all studies. To date, there is no proven link between casual cannabis use and gynecomastia.
For those suffering from gynecomastia, some doctors point to the fact that Marijuana is an estrogen stimulator-which can throw the endocrine system out of whack. And because gynecomastia is a problem closely related to hormone imbalance, it could be a problem. Be open with your doctor about your consumption so you can figure out if it is a contributing factor for you.