Same concept applies to light bulbs. A 100 watt bulb (on 120 volts) should draw 0.83 amps (W = VI). Reality is much different. That bulb will probably first demand 8.0 amps. Most don't know that since numbers provided for light bulbs forget to mention it.
Your motor is similar. It demands more power (current) to startup.
Apparently your intent is to make a speed controller. So get a 12 volt supply (wall wart) with more like one amp. Or do something more interesting. A maybe 15 or 18 volt supply (wall wart) powered through a transistor. That transistor driven (controlled) by a 555 timer.
A
555 timer is one of the world's most famous integrated circuits (created by Signetics). Designed more than 40 years ago. And still sold even in Radio Shack.
A 555 pulses the transistor. The transistor then pulses a motor. Longer pulses cause the motor to spin faster. Pulses (not continuous) power means 15 or 20 volts cause no motor damage. And (depending on the motor) can even make startup easier without smoke.