I think we use people that are inconsequential in our lives, but abuse people that are important to us. It’s easier to take for granted someone that’s close to us, than it is to take a stranger or an acquaintance for granted because there has to be an expectation of consistency and reliability before you can take someone for granted. Hence abusing those close to us.
However, I’ve often thought that taking someone for granted can be construed as a compliment. But such a compliment is wasted and justifiably unappreciated if it is never balanced with due care and consideration as well.