The No. 5 crossbar telephone switching developed by the Bell System in the United States was fantastically complicated and composed of myriad electromechanical components which had to work reliably for long periods of time. But what if something broke?
To aid in diagnosis and repair of problems, large No. 5 crossbar offices had a trouble recorder which, when one of three watchdog timers expired indicating a failed connection, punched a pasteboard card that indicated all of the internal settings of the switches involved in completing the circuit. This device itself was impressively complex, including a bank of 120 electromechanical punches which could perforate 1080 positions in a trouble record card.
Here is an article from the Bell Labs Record, βTrouble recording for the No. 5 crossbar systemβ [PDF], that explains how it worked.