I've removed mine several times and had no issues. I did notice after the first time I replaced it there was a rattle, which was fixed my completely tightening the bolts.
So I kind of cheated. My wife bought a Willy's last night with the dual door group and luckily it was the same color. I can ask the dealer if you want to know in general. Also, as for the keys it has remote keyless entry so that doesn't even matter. they just unlock like normal.