This is pretty cool - you can hear Benjamin Harrison speak! The site says that this is believed to be the oldest recording of any US President.
From this same site, I browsed through to listen to the Presidents I hadn't heard before. The one that got me was Calvin Coolidge - you just HAVE to go listen if you've never heard Coolidge. I've always had a voice in my mind for Coolidge from reading about him (as I'm sure you have) and this is NOT it! The voice just doesn't fit my mental image of "Silent Cal!" Another great one to listen to if you haven't heard him before (and I had several times) is Theodore Roosevelt.
You can also access a variety of different later Presidents though the Vincent Voice Library. Go have fun - I certainly did...and make sure to let me know what you thought of President Coolidge!