Would this bother you?

I can see how that would get really irritating. I wouldn't go over there. I'd just cut back and if she wants to see your daughter she will come to your house. It isn't fair to make you leave home every time especially many times a week. My mom comes over some days just to play with my son while I clean and it's great. He is happy and I get a lot done. Maybe you could set up a schedule with her so certain days she came to your house. Then it would be planned and she would get to see her granddaughter and you would get to work at some things at home.