Failed my driving test for not overtaking a bus?!?

And I only had 2 minors! Was driving down quite a busy shopping street and got stuck behind a bus, I did try to overtake it when it pulled in a couple of times but there were cars coming on the other side of the road and I thought - play safe as it might be a major for driving dangerously if I try when it's not safe. I overtook it when it pulled over for a third time. But then at the end of the test the examiner said that there was enough room for me to overtake with the car coming towards me (I disagree) and that there were cars queueing behind me. Since when is it a major to do something that could have been potentially dangerous?! I thought at the most it deserved a minor or two for hesitating. Has this or something similar happened to anyone before? Or can anyone (any examiners on here?) shed any light on this?