After the dispute, the couple left their children unattended, child welfare committee banned