Silent Barn is another cool and really weird music venue. But it's a bit of a hike out in Brooklyn, and I can't vouch for the musicians playing there during your trip:
http://www.myspace.com/thesilentbarn
I also think Cake Shop consistently has decent music, although more towards the pop side, rather than "avant-garde" or experimental:
http://cake-shop.com/
A pastrami sandwich at Katz's deli is a very American (and particularly NYC) eating experience.