Core sets showcase much of what Magic is to players. They do not have any story to them like the Expert-level sets do. Instead, they make an effort to introduce newer players into the game by means of explaining mechanics and demonstrating how the different colors act.
Cards included in core sets are always reprints of cards from previous sets, with the exception of Alpha, Beta, and Unlimited (the first sets of the game). As such, they've had to rely on other factors to make their mark, such as the massive rules overhaul in 6th Edition and the change in card templates with 8th Edition.
Core sets also serve an important place in Standard tournaments. These more basic cards serve as a backbone for effects decks have come to rely on, such as mana acceleration in Rampant Growth, prevention in Story Circle, and other effects which may be lackluster in the Expert sets. They also can provide powerful rares for the sake of helping to define the Standard format and giving people more of a reason to purchase booster packs.
As of 2016, there will no longer be a Core Set. See here.
Past Core Sets: