After hearing one of their performances on KEXP, I picked up the latest CD released by The Baseball Project, called Volume 2: High and Inside. They are an indie rock group that ranges from rock/pop to something like rockabilly(?). The songs are completely about baseball, with some good ones such as 1976 (not a bad year), The Straw that Stirs the Drink (about how Reggie Jackson was great and he new it), Look Out Mom (when Denard Span hit a foul line drive that clocked his mom in the head...on Mother's Day), and Pete Rose Way.
There is one song with one occasion of foul language that isn't at all hidden (although not anything one wouldn't expect to hear at a major league baseball game) that might make this CD less than ideal for a young baseball fan. I didn't pick up on any other language, though.