If only the front side of a door needs a key and the Back side has Linedef Type 1: "Door Open Wait Close (also monsters)". If the door is thin enough, a monster can sometimes open the door from the locked side. All monsters can do this, but only Arch-Viles, Arachnotrons, Cyberdemons, Demons, Spectres and Revenants can open doors (thicker than 7 units). This is because they are faster than the other monsters (the faster a monster is, the thicker partly locked doors it can open).
This occurs when the monster "bumps" in the door fast enough that part of its collision box reaches the other side of the door, which is unlocked, and thus opens it. (The engine "thinks" the monster is opening it from the unlocked side.) This is caused by the same glitch that allows grabbing items behind a wall.
Doors which are only locked from one side are rarely used by map authors, which is why the bug can only rarely be witnessed. The only instances of this type of door in the commercial IWADs are the red key door in Doom II MAP27, the yellow key door in the yellow key room in Doom E2M6 and both of the yellow key doors in Doom E2M2.