Confirming who my biological father may be

I am 44 years old and was given up for closed adoption . I did my search on my own in my early 20's, it took my several years but i suceeded and found my mother. She had also given up my two brothers prior to me, we were all aprrox. 1 year apart in age. At this time my mother gave my my fathers name, I located him. We never did a DNA test, but the blood type did match. Keeping in mind i did not resemble this man at all. But he was very welcoming and admitted to having a relationship with my mother. We continued to have a relationship for the past 20 years. My mother and I have located my oldest brother about 15 years ago, and have just located my middle brother about 6 months ago. Well giving my middle brothers description, looking like the man she said was my father. She now tells me that she may have been mistaken and gave me the wrong information that she was never really certain, she had a relationship with both men during a 3 year time period. And that she did see the man she is now saying is my father about the time I was conceived. Now the man she said was my father 20 years ago is upset and and refuses a DNA test and has been diagnoised with cancer. The man my mother is now saying is my father passed away when my mother was 4 months pregnant with me. His mother is alive and 98 years old. Her son was an only child, and from a prominant family. He has 2 cousins one in italy and the other in washington dc. They do not want me to have anything to do with his mother, tell her etc. I do not want to upset her and not know if he is or is not my father. I need some advice on how I should approach this situation.