KYO_SAMA'S PROFILE

I'm in this fortress and there was this room in the café with four mirrors. I broke them in the correct order and the ladder appeared. But if I go down, this Monster attacks me while I'm on the ladder and I don't have a chance to move forword. I visited every house again but i didn't find anything or another way. Am I too incapable for this or is there anything else I can do?