This has always confused me. Dictionary.com doesn’t help at all. Candy and meat just don’t go together that well.
The key is that meat used to just mean “food”. The first entry under meat at Merriam-Webster has exactly that definition, though of course the primary sense of meat today is definition #2, which specifically refers to animal flesh. However, the older, broader sense of “meat” still sticks around here and there, such as the practice of referring to the edible part of a nut as the “meat”, and in compounds such as “sweetmeat”.