Problem Supermicro H11DSi + 2 x AMD EPYC 7351

Rec · January 21, 2019, 10:37

I have the following configuration:

Motherboard: Supermicro H11DSi - AMD SP3 socket - DDR4 RAM - Extended ATX
CPU: 2 x AMD EPYC 7351 - 2.4 GHz - 16-core - 32 threads - 64 MB cache - Socket SP3
Cooler 2 x Noctua NH-U9 TR4-SP3 - (for: TR4, SP3) - 92 mm
16 x Memory: RDIMM Crucial 8GB CT8G4RFD8266 (Specs: DDR4 PC4-21300 • CL=19 • Dual Ranked • x8 based • Registered • ECC • DDR4-2666 • 1.2V)
SSD: Samsung V-NAND SSD 970 PRO M.2 - 512 Gb

OS: Windows 10 Pro x64
MB: Last Firmware on BIOS and IPMI.

When I run the calculation on Ansys CFX on 30 - 32 cores, with all memory, after 2 or 5 min, I get a blue screen (DPC WATCHDOG VIOLATION)

If I leave 8 slots memory 64GB, system works is good and stable.
I tried different OS and swap memory.
Any ideas?

The problem is very similar to this: https://community.amd.com/thread/224665

Thank You.

ErikAdr · January 22, 2019, 04:35

This is a configuration very close to what I was thinking about building. The memory you are using is the easiest accessible here and what I had planed to buy. The SSD used is with NVMe interface? The board only supports a PCI Express 3.0 X 2 but yours are 3.0 x 4. A reseller of Epyc workstations only offer SSD with SATA3 og SAS3 interfaces (deltacomputer.com). Perhaps they have bad experience with NVMe. I cannot explain why it works with 8 DIMMS. I hope You find a solution and post it here. Good luck.

NB: On supermicro's web page Window 10 pro is not listed as supported for the motherboard! Have you tried with a newer Ubuntu or Fedora distribution?

Rec · January 22, 2019, 09:01

Hello!

No, I only use SSD on M.2 slot + 2 SATA 8TB RAID. NVMe very expensive in my country.
Yes, i try Centos 7.6 and Ubuntu 18.04.

Quote:

Originally Posted by ErikAdr

This is a configuration very close to what I was thinking about building. The memory you are using is the easiest accessible here and what I had planed to buy. The SSD used is with NVMe interface? The board only supports a PCI Express 3.0 X 2 but yours are 3.0 x 4. A reseller of Epyc workstations only offer SSD with SATA3 og SAS3 interfaces (deltacomputer.com). Perhaps they have bad experience with NVMe. I cannot explain why it works with 8 DIMMS. I hope You find a solution and post it here. Good luck.

NB: On supermicro's web page Window 10 pro is not listed as supported for the motherboard! Have you tried with a newer Ubuntu or Fedora distribution?

Rec · January 22, 2019, 09:02

I found the reason - 1 memory module failed

, I will take it to the service center, now the server runs on 15 memory modules.

flotus1 · January 22, 2019, 12:21

FYI, the Samsung 970 Pro is an NVMe SSD

Rec · January 22, 2019, 14:56

Sorry, i think what it is about SSD on PCI Express port.

Quote:

Originally Posted by flotus1

FYI, the Samsung 970 Pro is an NVMe SSD

January 21, 2019, 10:37	Problem Supermicro H11DSi + 2 x AMD EPYC 7351	#1
Rec New Member Sergey Join Date: Jan 2018 Posts: 18 Rep Power: 8	I have the following configuration: Motherboard: Supermicro H11DSi - AMD SP3 socket - DDR4 RAM - Extended ATX CPU: 2 x AMD EPYC 7351 - 2.4 GHz - 16-core - 32 threads - 64 MB cache - Socket SP3 Cooler 2 x Noctua NH-U9 TR4-SP3 - (for: TR4, SP3) - 92 mm 16 x Memory: RDIMM Crucial 8GB CT8G4RFD8266 (Specs: DDR4 PC4-21300 • CL=19 • Dual Ranked • x8 based • Registered • ECC • DDR4-2666 • 1.2V) SSD: Samsung V-NAND SSD 970 PRO M.2 - 512 Gb OS: Windows 10 Pro x64 MB: Last Firmware on BIOS and IPMI. When I run the calculation on Ansys CFX on 30 - 32 cores, with all memory, after 2 or 5 min, I get a blue screen (DPC WATCHDOG VIOLATION) If I leave 8 slots memory 64GB, system works is good and stable. I tried different OS and swap memory. Any ideas? The problem is very similar to this: https://community.amd.com/thread/224665 Thank You. Last edited by Rec; January 21, 2019 at 10:42. Reason: swap memory

January 22, 2019, 04:35		#2
ErikAdr Member Erik Andresen Join Date: Feb 2016 Location: Denmark Posts: 35 Rep Power: 10	This is a configuration very close to what I was thinking about building. The memory you are using is the easiest accessible here and what I had planed to buy. The SSD used is with NVMe interface? The board only supports a PCI Express 3.0 X 2 but yours are 3.0 x 4. A reseller of Epyc workstations only offer SSD with SATA3 og SAS3 interfaces (deltacomputer.com). Perhaps they have bad experience with NVMe. I cannot explain why it works with 8 DIMMS. I hope You find a solution and post it here. Good luck. NB: On supermicro's web page Window 10 pro is not listed as supported for the motherboard! Have you tried with a newer Ubuntu or Fedora distribution? Last edited by ErikAdr; January 22, 2019 at 05:39. Reason: Windows 10 pro not supported

January 22, 2019, 09:02	Problem resolved	#4
Rec New Member Sergey Join Date: Jan 2018 Posts: 18 Rep Power: 8	I found the reason - 1 memory module failed, I will take it to the service center, now the server runs on 15 memory modules.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
AMD Epyc CFD benchmarks with Ansys Fluent	flotus1	Hardware	55	November 12, 2018 06:33
Mesh& steptime independant: conduction-convection problem	Fati1	Main CFD Forum	1	October 28, 2018 14:52
Problem diverges when exhaust valve opens	swerner0711	AVL FIRE	0	September 21, 2018 08:14
EPYC based workstation for OpenFOAM simulations	roenby	Hardware	21	May 30, 2018 10:29
Gambit - meshing over airfoil wrapping (?) problem	JFDC	FLUENT	1	July 11, 2011 06:59

January 22, 2019, 12:21		#5
flotus1 Super Moderator Alex Join Date: Jun 2012 Location: Germany Posts: 3,428 Rep Power: 49	FYI, the Samsung 970 Pro is an NVMe SSD