[mcneilyo]

Hi,
After many months of research and about 18 months blowing money on EC2 instances, I took a dive and bought two refurbished HP DL580 servers (for an excellent price I might add).

I've been running many FORTRAN executable in parallel (MODFLOW) to undertake Monte Carlo uncertainty analysis. I'm seeing some very slow runtimes - much much slower than anticipated.

Here's my setup:
2x HP DL580 servers, with the following in each:
2x Hypervisor Windows 10 pro virtual machines
4x Intel® Xeon® Processor E7-4870 (30M Cache, 2.40 GHz, 6.40 GT/s Intel® QPI)
256GB RAM (32x 8GB PC3-10600R)
P410i raid controller
8x 480GB SSD (stripe setup)

Running a single model takes about 1 hour, which I assume = 2.8GHz with boosting. When run 1 model per physical core, I'm seeing speeds of about 4 hours. And then 1 model per virtual core about 6 hour run times.

Where's my bottle-necking coming from? These days, shouldn't I be getting 2.4GHz from each vCPU? Or is it somehow split? Either way, I should be seeing faster run times with 1 model per physical core.

I've tried setting affinity per model, and it did little to speed up the runs. It can't be harddrive speeds, as I'm writing data at about 1 MB/s per model. Each model is consuming 0.5GB RAM.

Any advice greatly appreciated!

[mcneilyo]

I should add that I saw pretty similar speeds on the EC2 virtual machines. e.g. r3.8xlarge

To save you a google, in total I have 4 x virtual machines. Each machine has 20 physical cores allocated to it (40 vcpus).