From American Prince: A Memoir :
Janet' unhappiness wasn’t due solely to the movie. Janet’s friend Barbara Rush was in the movie with us, and one day Barbara came up to me and asked, “Are you and Janet doing all right?” I had a feeling Janet had told Barbara we weren’t, and that Barbara had come to me to confirm it. I just said, “Janet’s not feeling well.” When the movie ended and Janet and I went home, I tried harder to behave in a way that didn’t anger her, and she made the same effort for me. We settled into a functional but unromantic marriage, the kind of life that was less unusual in Hollywood than you might think.
We threw huge parties at our big, beautiful house on Summit Drive. Janet invited the Debbie Reynolds crowd, and friends such as Danny Kaye. Danny was a major talent. He was born in Brooklyn... To my way of thinking, Danny was a very mean and bitter man, and most everybody seemed to agree with me. When we first met, he would belittle me all the time. He once asked me, “Where did you learn how to fence—the Bronx?” Another time he said to me, “How do you act in those high heels?” I said, “I don’t wear high heels.” Then I took a step closer to him, looked in his eyes, and smiled while I said, “Fuck you, Danny.”
I don’t know why Danny had it in for me. Maybe it was because we both came from New York. Maybe it was because we were both Jewish, and he struggled with that in himself. Or it might have been some complicated sexual feeling.
It was widely rumored that Danny went with both men and women. One of the people Danny was believed to have had a relationship with was Sir Laurence Olivier. There had always been rumors about Larry’s sexuality, but he was nothing like Danny.
Live and let live. I don’t look down on gays; it’s just not my thing. George Cukor, one of the great directors, was part of the Hollywood gay crowd. George would throw a big, formal dinner party at his house. Then, after the party was over, George and his friends would go cruise Sunset Boulevard, looking for young men; they called them “after-dinner mints.” To each his own.