The man stepped off the elevator and started down the corridor.
I have seen started used like this before without adding eg. walking after it. Is it perfectly natural use of started?
It is a perfectly natural use of ‘started’.
Lexico‘s definition 1.3 of start is:
[no object, with adverbial of direction] Begin to move or travel.
Their examples include, he started for the door and we started out into the snow.
According to Etymonline, start evolved from the Old English styrtan and its variants, which meant to leap up or to move or spring suddenly. By 1300 this meaning had evolved into flinch or recoil in alarm, which is still one of its meanings today.
The meaning begin to move or travel only appeared in 1821. It bears no suggestion of alarm or suddenness.