Create multiple Docker machines at once with docker-machine - amazon-web-services

Question

In my application, I need to create many Docker machines on a cloud computing service (AWS-EC2 for now but could be changed), then deploy many containers on those machines. I am using docker-machine to provision them on AWS, using a command like
docker-machine create --driver amazonec2 --amazonec2-ssh-keypath <path-to-pem> <machine-name>
The problem is that it takes a lot of time, about 6 minutes, to create one such machine. So overall it takes hours just to create the machines for my deployment. Is there any way to create multiple machines at once with docker-machine? Or any way to speed up the provisioning of a machine? All machines have the same configuration, just different EC2 instances with different names.
I think running multiple docker-machine create commands in the background might work, but I fear it might corrupt the configuration (machine list, internal settings, etc.) of docker-machine. Even if I can run multiple such commands safely, I don't know how to check when and if they have completed successfully.
P.S: I understand that AWS supports Docker containers and it might be faster to create instances this way. But their "service model" of computation does not fit the needs of my application.

Answer 1

Running simultaneous docker-machine create commands should be fine - it depends on the driver, but I do this with Azure. The Azure driver has some logic to create the subnet, availability sets etc. and will re-use them if they're already there. Assuming AWS does something similar, you should be fine if you create the first machine and wait for it to complete, then start the rest in parallel.
As to knowing when they're done, it would be cool to run Docker Machine in a Docker container. That way you can create each machine with a docker run command, and use docker ps to see how they're doing. Assuming you want to manage them from the host though, you'd need your containers to share the host's Docker Machine state - mounting a volume and setting MACHINE_STORAGE_PATH to the volume on the host should do it.

Answer 2

TLDR; This will run docker-machine command 9 times (asynchronously) to create multiple machines. Adjust as required:
for i in `seq 1 9` ; do docker-machine create --driver virtualbox worker$i & done
Creates 9x machines...
worker1
worker2
worker3
worker4
worker5
worker6
worker7
worker8
worker9