Closings and Escrow: Refi Removing Everyone on Title and Adding New People

My question involves real estate located in the State of: CA

Hello, I'm in a situation and hopefully someone and can help me reassure the answer? I spoke with my broker and he stated after checking with the escrow lawyer, that I will not incur any tax consequences. Here is the scenario.

Current standing...
People on title of house:
Myself
My wife
My father in law
Another person not related to family

On current loan:
My father in law
Another person not related to family

I'm refinancing and the new status will be:
People on title of house:
Myself
My wife

On the new loan:
Myself only

What I'm concern about is the tax consequences with this transaction. Again, I've confirmed with my broker and he stated there should not be any tax consequences. I'm just trying to double check. Can someone please help me? Thank you.

Closings and Escrow: Refi Removing Everyone on Title and Adding New People