Before we proceed towards Passive infinitive let's take a review of what is Infinitive.
Infinitive:
Happens to be means
There may be several meanings of happens to be:
(*1*) Happens to + Infinitive is a special construction
Elliot is happen to be wizard + it is coincidence that Elliot is wizard
The car slammed and many people were wounded, they survived only because there happened to be doctor too in the car (it was not a plan that doctor was present there too ).
My best friend didn't use to wok in mall, he was just happened to be nearby when the mall collapsed:
that is when he died(it was not to be expected that he should have been nearby mall at that time);
(*2*) It can also be used ironically :
Pakistan: You know nothing about Pakistan, i have been there for few weeks and i speak English.
England :Oh, that is quite interesting, they happen to speak Urdu there.
If you happen to go to Pakistan later, Please get these things for me. It is possible thing taht might happen in future.
Depending on the given context it could just mean "was" or emphasizing the past a action above all others.
I happen to be there just mean you were just there, maybe by chance, by random, maybe planned. it is just that you were there.
For example: A cat happens to be white. The cat is coincidentally where. There is no logic behind it being that colour.