Well I would say the OP handled it well and like a good neighbor. I have had a couple encounters with my neighbors when I first started OC.
One I was washing the car and put the XD on the hip and washed the car, one neighbor (gun nut) asked if I was afraid of being mugged or jumped in my driveway while pointing to my weapon. I simply said Nope. and left it at that. He carries when ever he is out and understands my stance.
The second time was with one of my friendly neighbor that I have never really talked to a lot and not sure of his stance on weapons. He asked if I was becoming a cop or somthing (he saw me on my way out) I told him I am a firm believer in the 2A and explained a little about local laws. I also informed him that it was not the first time I had a firearm on around him. He gave me the raised brow and I told him that when I have it concealed then it is concealed.
The one other that made an impression was I went to introduce my self to a new neighbor. (I live in a military neighborhood off base) He asked if I was a cop on base or somthing. I kinda gave him the raised brow and he said he thought he saw me with a pistol on my hip the other day. I told him he probally did see that and explained the right to bear arms and again a few of the local laws. All said and done he said if the crap hits the fan that he will be at my house. I offered to give him some range time and training if he ever feels he wants to be able to protect his family himself incase I am on vacation.
Just thought I would share a few of my encounters luckily (knock knock) I have not had any bad expericenes or memorable ones with LE's.