Have you checked your battery storage voltage (disconnected) and charging circuit voltage (connected)? Blown fuses or magnets detached in the starter motor should result in an inoperable starter, not one that operates intermittently.
My money is on a poor ground connection somewhere and/or failure of the infamous diode pack or 4700uF capacitor.
If the diodes fail in open- circuit state (i.e. broken conductor) the 12v battery won't be recharged from stator power when the engine is running, eventually resulting in a flat battery after several operations of the starter.