The question may be too opinion based and highly contentious.
However, as a non native speaker with a serious disability, I have great trouble grasping why “disabled“ is supposed to be much better than “handicapped“. When I hear “handicapped“ I think of someone on the gym taking a position that makes the exercise harder and ideally still pulling it off (incidentally, this is what I think my disability does; it makes my life a hell of a lot harder, but in the end I still prevail).
When I hear “disabled“ I think of a machine completely switched off or an opponent immobilised (this is what I never want my disability to achieve: to make me feel utterly powerless).
Is it just a random effect of history that made “handicapped“ so much worse (because people have simply had more time abusing the word) or can it be understood from the connotations of the two words outside medicine?
Sources that attest to the preferability of “disabled“ are quite easy to find, but they don’t explain why this preference came about. Examples: 1.
Another, which states that the connotation is opposite from what I expect, but not why: 2.
Here we have a comparison of the two including a summary table, but again “disabled“ would strike me as the much more negative, except for the disputed hand-in-cap etymology, which I understand could make the term undesirable: 3
Here, “handicap“ is connected to “Hand in Cap“ through a game of chance unrelated to begging: 4
According to linguist John McWhorter the answer is indeed history. Older terms accumulate baggage and are replaced by new terms in what is called the Euphemism Treadmill:
Crippled began as a sympathetic term. However, a sad reality of human society is that there are negative associations and even dismissal harboured against those with disabilities. Thus crippled became accreted with those overtones, so to speak, to the point that handicapped was fashioned as a replacement term free from such baggage.
However, because humans stayed human, it was impossible that handicapped would not, over time, become accreted with similar gunk. Enter disabled, which is now long-lived enough that many process it, too, as harbouring shades of abuse, which conditions a replacement such as differently abled.
— Euphemise this: McWhorter on The Euphemism Treadmill