If the museum that you're staying near is the Museum of London, it's very walkable - the museum is about five minutes' walk from the Millenium Bridge, which crosses from St Pauls Cathedral to the Tate Modern gallery (near Southwark Bridge) - go over the bridge, head towards the National Theatre, and just keep walking.
http://www.southbanklondon.com/