What happens when you do try to delete? Does it disappear? Or come back the next time you open the Window after you click OK?
I think you'll be able to solve this by exporting your preferences without the captions as part of it, deleting (or renaming) the preferences folder, and then importing your preferences folder again.
Go to Preferences. Click the Export button. In the window that appears, select everything except for the item called "IPTC: Caption popup", and proceed with the Export. Quit Photo Mechanic.
Then rename the Photo Mechanic Preferences folder, which is located here: /Users/username/Library/Preferences/com.camerabits.PhotoMechanic
I usually just rename the folder to com.camerabits.PhotoMechanicOLD
Launch Photo Mechanic and then go to the Preferences window and click Import. Locate the file that you exported a few moments ago, and Import it with all settings selected. Now, see if you can add/delete/change items in the Caption field and let men now if that takes care of the issue