The armor shouldn't help you much, but it might. More importantly, the mouse can be time consuming if you kill things near it, but is a slight bit quicker if you just talk and run.
What I found helpful is to buy keycrafting material in town (with A and B vouchers downloaded) and then after getting the two keys to just run straight to Edward. If you have healing items, your allies should be able to keep you alive. Don't bother using light balls, don't attack anything - just run. Make key 3 and 4 and run back, you should have enough time if you don't have to farm materials from soldiers and gargoyles/chests.