Abstract The automated segmentation of dental restorations is a critical step in diagnosing dental problems and suggesting the best treatment. Some restorations may be missed during a dental examination, depending on the number of patients, the dentist's experience, and fatigue. Automatic detection of dental restorations based on deep learning has the potential to provide a quick radiological assessment based on the patient's treatment history and pre-diagnosis. This study presents a deep learning-based method for automatic detection and classification of amalgam and composite fillings on panoramic images. A total of 250 anonymized panoramic images with amalgam and composite fillings with a resolution of 2048 × 1024 px were used. In this study, U-Net models with various backbones were employed. The ResNext50 model has achieved the highest pixel accuracy and intersection over union (IoU) performance based on the evaluation of various ResNet and ResNext backbones. The mean IoU value obtained by the model on the test images is 0.767 while the Pixel Accuracy of 99.81\% was achieved. Our proposed method demonstrated superior performance compared to similarly conducted studies in the literature. The proposed method can potentially be employed in clinical settings to detect dental restorations automatically. The classification and detection of dental restorations with this model can aid dentistry education at higher institutions as an education tool and make the reporting easier for the dentist.