I agree. However, I'm not trying to prove my point, just explain it.
I know well enough that this kind of forum isn't really conducive to discussing biblical evidence. Furthermore, I'm sure any evidence I produce can be counter-evidenced or just fudged away. However, I'd basically argue on these kind of lines:
a) Jesus established a Church. - from scripture
b) Jesus' Church had authority structures, with certain offices of leadership. - from scripture and history
c) Jesus would have intended his Church (and hence the authority structures) to continue historically. - from reason
d) The Catholic Church can evidence its authority structures back to the first century. - from history
e) The Catholic Church can evidence a continuity of most dogma back to the first/early second centuries. - from scripture and history
f) The only other possible Churches that can sustain such an argument might be the Orthodox and Coptic Churches. - from scripture, history and reason
g) So, it is quite reasonable that the Catholic Church is the one started by Christ.