As a couple, de Young Museum is an easy yes.
A calm, well-curated American and global art collection in a striking copper building, and the tower view is free even if you skip the galleries.