SHOULD I LET EX HUSBAND MOVE BACK IN WITH ME?

i recently moved out of our house and now my ex husband wanna move back in with me to save cost he said and he cant find his own place. his lease was up 4 weeks ago. we have 2 young girls 5 and 19 months and he has one 9 yr old. he has been cruel to me physically verbally and mentally. now he throws words at me and making me feel bad abt not seeing my girls often cause they live with him most days since im in university and working. should i let his words get into me? im in a relationship right now and ahve move on and boyfriend isnt happy with it but he trusts me to make this decision. i wanna see my girls everyday but their father drives me insane. i know its a pretty obvious answer but i need someone to show me some pros and cons considering we have kids. i go to his place early to bring kids to school and sometimes he wont let me but instead he walks the kids to school with 19 mth old in pram. i pity kids being in the middle of all this. what should i do? should i let him move back in? where the possibility of him touching me in my sleep, watching my every move and going thru my stuff is not impossible. pls help me