As you stated, the stopping power is a downer, but it isn't the only factor. Compare the price and the availability of .380 vs. 9mm. I have actually considered a .380, even given the stopping power. In any conversation pertaining to stopping power, you will hear the old saying "a well placed shot with a .22 is better than a miss with a .45".
For me, the price of .380 (often 50% higher than 9mm) as well as the scarce availability of it, are the two major factors. The difference in stopping power just compounds that. As for size/ concealability... you can find some pretty small 9mms. Not quite as small as as a PPK or LCP, but small enough for pocket carry.
If cost of ammo is no factor and you really need the extra small size of a .380, then I'd say go for it. Just be sure that the cost of ammo isn't going to dissuade you from practicing often with that firearm.