Lee was 16 months old when his mother was killed in a pedestrian accident in Michigan. His father had been killed in an industrial accident in October of 1938, 6 months before his birth. His brother Carl was 11 and his sister Betty was not yet 16 at the time of their mother's death. Betty stayed in the Detroit area, but as she wasn't old enough to take custody of the boys, they were both sent to Kentucky to live with different paternal uncles. Carl went to live with Ira Yeary, and Lee to live with Harvey Yeary. Harvey Yeary and his wife adopted Lee when he was about 3, changing his name at that time from his birth name of Henry Yeary to Harvey Lee Yeary II.
Carl certainly knew Lee as is little brother, but they did not live close by and only saw each other at family gatherings. And as Lee's adoptive parents didn't wish to discuss the adoption, he was encouraged not to bring up the past. Betty did visit her brothers in Kentucky at least a few times as well.
I do not know for certain, but my guess is that Lee suspected something was up before he found the actual documentation. He has indicated that he was disappointed with his parents for keeping the truth from him, and I don't doubt that the fact that he was never told the young man and woman he believed were his cousins were actually his biological siblings played some role in that.