I totally hear you! We've had several people who originally said 'we wouldn't miss it for the world' drop out.
But we're still excited b/c right now we have about 45 people coming and maybe a few more as well. I'm still happy with my decision to have a DW but it definitely makes me sad when people back out. I'm just trying not to dwell and thinking about the people who are coming!