I have a similar situation. I married a widower with 4 children, all adults, three with children. We married 3 years after he was widowed (5 years after I was widowed), and two of his children felt that the marriage was "too soon" after the death of their mother and "disrespectful to their mother's memory." In the beginning they were sort of pleasant. Then we married and it got really unpleasant.
They invite their father to birthday parties and Christmas dinners and Thanksgiving dinners and NEVER invite me. I have never had a harsh word with his children. There was a family death, and one of his children went around telling people that their father and I met at his workplace and were dating when their mother was alive. That is absolutely untrue. Why would an adult lie about her father? I have no idea. Why do people believe lies? I have no idea.
Yes, when he goes to events without me I feel betrayed. It's not a "them or me" situation, but it could turn into that.
We've been married 6 years.