Ductal adenoma of the breast is a rare disease and is often mistaken for carcinoma clinically and pathologically. We recently treated a case of ductal adenoma of the breast forming an intracystic papilloma, which was difficult to diagnose. A 66-year-old woman was referred to the hospital because of a right breast tumor 7×5 cm in diameter with nipple retruction. Mammography and ultrasonography showed an intracystic papilloma with linear calcifications. A core needle biopsy revealed a suspicion of apocrine carcinoma (class V), which was confirmed to be HER2 positive by Herceptest showing 2+ staining and Estrogen receptor (ER) positive using the EIA method. After getting informed consent, we performed modified simple mastectomy. Histologically, the tumor was composed of proliferation of tubular adenomatous part surrouded by sclerotic fibrous lesion. Pseudoinvasion and apocrine metaplasia were also found within the tumor. The tubules of the tumor were lined with epithelial cells and surrouding myoepithelial cells. The tumor was definitaly diagnosed as ductal adenoma of the breast. Although the clinical diagnosis of ductal adenoma is very difficult, it is important not to overdiagnose it as carcinoma and to consider this as an initial diagnosis.