It seems like tacitly asking for the approval of a church and a god that you don't actually believe exists. As someone who was born and baptised as a Catholic, I can say from my own experience that it's something you can get over -- much as you would get over a cold. Removing yourself from baptismal records seems like allowing the church to which you won't submit to still have a hold over you.
Still no harm, I just think it's a bit irrelevant. I certainly can't be arsed to trail myself all the way to Sydney to expunge myself from the records.
More here if you're interested.