Scenario / Questions

I am using the EC2 plugin in Jenkins (https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin) for using AWS services.

As of now, a job which uses this cloud label spins up a single EC2 instance. Whereas, I need to spin multiple instances for this job.

A single EC2 instance will never be overloaded and hence the plugin will not spin any new EC2 instances; but the job needs multiple instances to distribute the tasks.
There are ~100 tasks to be assigned to 20 machines one at a time (hence 5 per machine). A single instance can complete one task at a time.

How can I spin multiple EC2 instances (fixed number say 20) for a single job in Jenkins?

Thanks.

Find below all possible solutions or suggestions for the above questions..

Suggestion: 1

Not a great answer, but you can change the “Number of Executors” in the Amazon EC2 section under Advanced. This will allow multiple jobs to run concurrently on a node. You would have to change your job to start multiple parameterized sub-jobs; the parameters being tied to the tasks.

If you manually start multiple nodes with the same tag, the load will be distributed among them.