I've read many times here and elsewhere that Mary suffered from increasing stage fright as the years passed and that this explained her frequent absences in the later years. Well, after listening to the radio shows, I have trouble believing this explanation. I'm convinced rather that Mary's absences were the result of the fact that she simply tired of the constant performing and just chose not to appear whenever the mood suited.
I base this on my read of Jack's exchanges with Mary on the show over the years. While Jack enjoyed ribbing the castmembers (including himself) when they flubbed a line, he always had a particular joy in doing this to Mary. And it was clear that Mary got a great kick out of Jack's "scoldings." This part was always the closest window we got to the real Jack and Mary relationship. If Jack thought that Mary was bothered in the least by the kidding, I'm sure he would never have done it. Now unless Mary was the world's greatest actress, I think she enjoyed working with Jack - that is, when she felt like working. I didn't sense any extra nervousness or stage fright as the years passed. For better or worse, she was as composed before the microphone and made about the same number of flubs in 1937 as in 1952.
What I suspect Mary didn't like was the weekly grind of performing. And as Mrs. Jack Benny, she didn't have to whenever she chose not to. After all, it's not like they needed the extra income. If Phil or Dennis had all the "colds" that Mary had, they would have been replaced for sure. But Jack wasn't about to fire Mary. So it became a perfect part-time job for her. Who wouldn't love to work only when the mood struck?
As far as the television show is concerned, I think that Mary simply felt uncomfortable in front of the camera. Reading a script before a microphone is one thing; having to memorize a script and act on a stage is quite another. This would also explain her decision not to pursue a movie career.