Well, it prevents confusion..
Sorry, my non-RCC friends, but I kinda like seeing our pastors in their albs and stoles.
It is their "uniform" and is symbolic. My husband and our dear pastor have both told me that wearing them reminds them of their office of pastor, which they find very humbling.
I can't help it...but hubby looks so...cute in his alb and stole...?
But it is still adiaphora. It is not wrong for a pastor to wear them, or NOT wear them. It is up to the church, congregation, and pastor.