Can he get in any legal trouble for this?

yes you can the fact that none of these adults have done anything makes me think your lieing however if your not then call the police and stop him from hurting you it's called assault at the very least and he will serve at least 5 years in prison also he will loose custody of you and your siblings. Someone in DCF will explain the rest of the process