Water has gotten into my oven

[the Beagle]

Every exposed mortar joint on the top of your structure is a potential entry point for water, In Boston I would be concerned about that water in the Winter because frozen water expands and creates problems in masonry. Your goal is to prevent water from entering your structure. In addition to caps on the flues you should consider a roof over the entire structure to prevent rain/snow from collecting on same. I have looked at many masonry structures in my work as a concrete troubleshooter and water entry is the number one cause for trouble. If you keep the oven at temp for a few days that should drive the water out, but keeping the water out in the first place should be your focus