I have no issues with kids and babies being at restaurants (not referring to a fast food establishment), but I do agree that if they start acting up (crying, screaming, taking a tantrum) to take them out until them calm down or take them home.
A restaurant isn't a playground for kids (unless it actually has one built in like McDonald's). There are hazards all around. And most people don't want someone that don't know coming over to their table (adult or child) and sticking their face (or fingers) at the table or food while they are having an evening out to themselves. Plus kids should not be going over to strangers so openly anyways, that's another situation that could be bad.
Kids need manners when they are out in that type of environment, and it's the parents responsibility to teach them that. And they need to handle with problems that are affecting those around them, not just expect people to accept it because kids will be kids because there are lots of kids that behave quite well in a mainly adult establishment.
Sorry if any of this might offend anyone, but sometimes it's difficult to say things on a forum without sounding rude because I'm not trying to be that way.