This shows you the differences between two versions of the page.
|
HPC:Village [2015/08/27 17:15] solar [Related third-party projects/offers] re-checked all the links, dropped one (no longer offers remote access) and revised some others |
HPC:Village [2020/12/10 12:55] (current) mator [Related third-party projects/offers] GRID5000 |
||
|---|---|---|---|
| Line 11: | Line 11: | ||
| * [[http://www.supermicro.com/products/system/4u/7047/sys-7047gr-tprf.cfm|Supermicro GPU SuperWorkstation 7047GR-TPRF]] workstation/server platform with MCP-290-00059-0B rackmount rail set | * [[http://www.supermicro.com/products/system/4u/7047/sys-7047gr-tprf.cfm|Supermicro GPU SuperWorkstation 7047GR-TPRF]] workstation/server platform with MCP-290-00059-0B rackmount rail set | ||
| * 4U chassis | * 4U chassis | ||
| - | * Two 1620W PSUs ((The system's AC power consumption at idle, but with power-hungry chassis fans forced to "Full" (as needed to achieve decent cooling of the HD 7990 card for when it is being used), happens to be around 350W. At full load on all components, it increases to around 1400W. These are totals for the two PSUs, which are normally sharing the load.)) | + | * Two 1620W PSUs (normally both are active and are sharing the load) |
| - | * Dual socket 2011 motherboard with IPMI, 16 memory sockets, four PCIe 3.0 x16 slots for full-length dual-width PCIe cards (and a fifth slot for a shorter card) | + | * Dual socket 2011 motherboard with IPMI, 16 memory sockets, four PCIe 3.0 x16 slots for full-length dual-width PCIe cards and a fifth slot for a shorter card |
| * A full set of cooling fans, including those pulling hot air out of passively-cooled accelerator cards | * A full set of cooling fans, including those pulling hot air out of passively-cooled accelerator cards | ||
| - | * Two 8-core [[http://ark.intel.com/products/64595/|Xeon E5-2670]] CPUs | + | * Two 8-core [[http://ark.intel.com/products/64595/|Intel Xeon E5-2670]] CPUs |
| * Sandy Bridge-EP, support AVX and AES-NI | * Sandy Bridge-EP, support AVX and AES-NI | ||
| * A total of 16 CPU cores seen as 32 logical CPUs (two hardware threads per core), at a clock rate of at least 2.6 GHz | * A total of 16 CPU cores seen as 32 logical CPUs (two hardware threads per core), at a clock rate of at least 2.6 GHz | ||
| Line 21: | Line 21: | ||
| * 8x 16 GB DDR3-1600 ECC Registered modules on 8 channels (4 channels per CPU) | * 8x 16 GB DDR3-1600 ECC Registered modules on 8 channels (4 channels per CPU) | ||
| * Theoretical bandwidth 102.4 GB/s, actual measured bandwidth ~85 GB/s (cumulative from 32 threads) | * Theoretical bandwidth 102.4 GB/s, actual measured bandwidth ~85 GB/s (cumulative from 32 threads) | ||
| - | * [[http://ark.intel.com/products/71992/intel-xeon-phi-coprocessor-5110p-8gb-1_053-ghz-60-core|Xeon Phi 5110P]] coprocessor module | + | * [[http://ark.intel.com/products/71992/intel-xeon-phi-coprocessor-5110p-8gb-1_053-ghz-60-core|Intel Xeon Phi 5110P]] coprocessor module |
| - | * 60 cores (x86-ish with 512-bit SIMD units) seen as 240 logical CPUs (four hardware threads per core), 1.053 GHz, 8 GB GDDR5 ECC RAM on a 512-bit bus, 320 GB/s | + | * Intel Many Integrated Core (MIC) architecture, Knights Corner |
| + | * 60 cores (x86-ish with 512-bit SIMD units) seen as 240 logical CPUs (four hardware threads per core), 1053 MHz, 8 GB GDDR5 ECC RAM on a 512-bit bus, 320 GB/s | ||
| * Peak performance of about 2 TFLOPS single-precision, 1 TFLOPS double-precision | * Peak performance of about 2 TFLOPS single-precision, 1 TFLOPS double-precision | ||
| - | * [[wp>Comparison_of_AMD_graphics_processing_units#Southern_Islands_.28HD_7xxx.29_Series|AMD Radeon HD 7990]] gaming graphics card | + | * [[wp>GeForce 1000 series|NVIDIA GTX 1080]] gaming graphics card (short form factor, manufactured by Gigabyte) |
| - | * Two “Tahiti” GPUs, which provides 2x2048 SPs at 1 GHz, 6 GB GDDR5 RAM on two 384-bit buses, 576 GB/s | + | * NVIDIA Pascal architecture |
| - | * Peak performance of over 8 TFLOPS single-precision, about 2 TFLOPS double-precision | + | * One GP104 GPU with 2560 SPs typically at 1607 MHz to 1771 MHz, 8 GB GDDR5X RAM on a 256-bit bus, 320 GB/s |
| - | * This is a budget replacement for the FirePro S10000 GPU card intended for servers (which would cost at least 3 times more and would run considerably slower, but would offer ECC RAM) | + | * Peak performance of over 8 TFLOPS single-precision |
| + | * [[wp>GeForce 900 series|NVIDIA GTX Titan X]] gaming graphics card (reference design, manufactured by Gigabyte) | ||
| + | * NVIDIA Maxwell architecture | ||
| + | * One GM200 GPU with 3072 SPs at 1000 MHz to 1177 MHz, 12 GB GDDR5 RAM on a 384-bit bus, 336 GB/s | ||
| + | * Peak performance of over 6 TFLOPS single-precision | ||
| * [[wp>GeForce 700 Series|NVIDIA GTX TITAN]] gaming graphics card (Zotac GeForce GTX TITAN AMP! Edition) | * [[wp>GeForce 700 Series|NVIDIA GTX TITAN]] gaming graphics card (Zotac GeForce GTX TITAN AMP! Edition) | ||
| - | * One GK110 GPU with 2688 SPs at 902 MHz to 954 MHz in single-precision mode, 6 GB GDDR5 RAM on a 384-bit bus, 317.2 GB/s | + | * NVIDIA Kepler architecture |
| + | * One GK110 GPU with 2688 SPs at 902 MHz to 1045 MHz in single-precision mode, 6 GB GDDR5 RAM on a 384-bit bus, 317.2 GB/s | ||
| * Peak performance of over 5 TFLOPS single-precision, from 1.3 to 1.5 TFLOPS double-precision in the corresponding mode | * Peak performance of over 5 TFLOPS single-precision, from 1.3 to 1.5 TFLOPS double-precision in the corresponding mode | ||
| * This is a budget replacement for the TESLA K20X GPU card intended for workstations and servers (which would cost at least 3 times more and would run considerably slower at single-precision and integer code, but would offer ECC RAM) | * This is a budget replacement for the TESLA K20X GPU card intended for workstations and servers (which would cost at least 3 times more and would run considerably slower at single-precision and integer code, but would offer ECC RAM) | ||
| + | * [[wp>AMD RX Vega series|AMD Radeon RX Vega 64]] gaming graphics card (reference design, manufactured by MSI with slight overclocking) | ||
| + | * AMD GCN 5th gen architecture | ||
| + | * One Vega10 XT GPU with 4096 SPs typically at 1401 or 1576 MHz, 8 GB HBM2 RAM on a "2048-bit" bus, 483.8 GB/s | ||
| + | * Peak performance of over 10 TFLOPS single-precision | ||
| - | Total peak performance is over 15 TFLOPS single-precision, about 4.5 TFLOPS double-precision. | + | Total peak performance is over 31 TFLOPS single-precision. |
| + | ===== Pictures ===== | ||
| - | Here's what the server looks like (click on the thumbnails for higher resolution pictures): | + | Here's what the server looks like (click on the thumbnails for higher resolution pictures). |
| + | |||
| + | 2019 upgrade (added Vega 64 and GTX 1080, removed HD 7990 and HD 6770 Green Edition): | ||
| + | |||
| + | {{super2019-uncovered1.jpg?direct&160}} | ||
| + | {{super2019-uncovered2.jpg?direct&160}} | ||
| + | |||
| + | 2015 upgrade (added GTX Titan X, as well as HD 6770 Green Edition into the short slot): | ||
| + | |||
| + | {{super2015-uncovered1.jpg?direct&160}} | ||
| + | {{super2015-uncovered2.jpg?direct&160}} | ||
| + | {{super2015-covered.jpg?direct&160}} | ||
| + | {{super2015-idle.png?direct&160}} | ||
| + | {{super2015-load.png?direct&160}} | ||
| + | |||
| + | 2013: | ||
| {{super-uncovered.jpg?direct&160}} | {{super-uncovered.jpg?direct&160}} | ||
| Line 45: | Line 71: | ||
| ===== Software ===== | ===== Software ===== | ||
| - | The operating system is Scientific Linux 6.6, since this is a common free option to run Intel MPSS as needed to access the Xeon Phi card (which, in turn, runs its own copy of Linux, coming from Intel MPSS). | + | The operating system is Scientific Linux 6.10 (with several devtoolsets installed, such as providing a variety of newer GCC versions), since this is a common free option to run Intel MPSS as needed to access the Xeon Phi card (which, in turn, runs its own copy of Linux, coming from Intel MPSS). We also have CUDA 10.1 with its driver version 418.39, and AMD AMDGPU-PRO 18.50. |
| Here's what this looks like via OpenCL: | Here's what this looks like via OpenCL: | ||
| <code> | <code> | ||
| - | [solar@super ~]$ clinfo | egrep '^ (Platform |)Name:' | tail -n +4 | + | [solar@super ~]$ clinfo | fgrep Name: | tail -n +4 |
| - | Platform Name: AMD Accelerated Parallel Processing | + | Platform Name: AMD Accelerated Parallel Processing |
| - | Name: Tahiti | + | Name: gfx900 |
| - | Name: Tahiti | + | Platform Name: Intel(R) OpenCL |
| - | Name: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz | + | Name: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz |
| - | Platform Name: Intel(R) OpenCL | + | Name: Intel(R) Many Integrated Core Acceleration Card |
| - | Name: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz | + | Platform Name: NVIDIA CUDA |
| - | Name: Intel(R) Many Integrated Core Acceleration Card | + | Name: GeForce GTX 1080 |
| - | Platform Name: NVIDIA CUDA | + | Name: GeForce GTX TITAN X |
| - | Name: GeForce GTX TITAN | + | Name: GeForce GTX TITAN |
| </code> | </code> | ||
| - | |||
| - | Curiously, "Tahiti" appears twice because there are two such GPUs (they're device 0 and 1, respectively), whereas the CPUs appear twice because they're available via both AMD's and Intel's OpenCL SDK, and either SDK will use all cores of both CPUs. | ||
| ===== Who is eligible ===== | ===== Who is eligible ===== | ||
| Line 80: | Line 104: | ||
| ===== Credits ===== | ===== Credits ===== | ||
| - | The HPC Village project is provided by [[http://www.openwall.com|Openwall]] (idea, computer hardware parts, software configuration, system administration) and [[http://www.df.ru|DataForce]] (assembly and hosting of servers, Internet connectivity). | + | The HPC Village project is provided by [[https://www.openwall.com|Openwall]] (idea, most computer hardware parts, software configuration, system administration) and [[http://www.df.ru|DataForce]] (assembly and hosting of servers, Internet connectivity). NVIDIA GTX 1080 and AMD Vega 64 purchase was sponsored by a grant from [[https://www.zfnd.org|Zcash Foundation]]. NVIDIA GTX Titan X purchase was sponsored by [[https://sagitta.pw|Sagitta HPC, a subsidiary of Stricture Group LLC]]. AMD Radeon HD 7990 (available in this machine until January 2019, then replaced with the Vega 64) was [[:john/contests|team john-users]]' prize in Hash Runner 2013 organized by [[https://www.ptsecurity.com|Positive Technologies]]. |
| ===== Related third-party projects/offers ===== | ===== Related third-party projects/offers ===== | ||
| Line 90: | Line 113: | ||
| * [[http://gcc.gnu.org/wiki/CompileFarm|GCC Compile Farm]] | * [[http://gcc.gnu.org/wiki/CompileFarm|GCC Compile Farm]] | ||
| - | Use Sage, R, Octave, Python, Cython, GAP, Macaulay2, Singular, and much more, write, compile, and run code in most programming languages on remote systems using a free service (with support from University of Washington, the National Science Foundation, and Google): | + | Use Sage, R, Octave, Python, Cython, GAP, Macaulay2, Singular, and much more, write, compile, and run code in most programming languages on remote systems using a free or paid service (with support from University of Washington, the National Science Foundation, and Google): |
| + | |||
| + | * [[https://cocalc.com|CoCalc]] (formerly SageMathCloud) | ||
| + | |||
| + | Time-limited free access to an HPC machine, with intent to promote this vendor's computer hardware sales: | ||
| - | * [[https://cloud.sagemath.com|SageMathCloud]] | + | * [[http://www.microway.com/take-a-test-drive/|Microway test drive of up to dual Xeon E5-26xx v4 or IBM POWER8 CPUs]] and [[https://www.microway.com/gpu-test-drive/|NVIDIA Tesla GPUs]] |
| - | Time-limited free access to an HPC machine, with intent to promote these vendors' computer hardware sales: | + | Free access for academic researchers worldwide to a 384-node cluster with Intel Xeon CPUs and Altera Stratix V FPGAs (two CPUs and one FPGA per node), running Windows Server 2012: |
| - | * [[http://www.microway.com/take-a-test-drive/|Microway test drive of up to 18-core CPUs, Xeon Phi, and NVIDIA Tesla GPUs]] | + | * [[http://research.microsoft.com/en-us/projects/catapult/|Project Catapult]] free access as [[https://www.tacc.utexas.edu/-/tacc-to-launch-new-catapult-system-to-researchers-worldwide|announced by TACC (Texas Advanced Computing Center at The University of Texas at Austin)]] and [[http://blogs.msdn.com/b/msr_er/archive/2015/11/12/project-catapult-servers-available-to-academic-researchers.aspx|Microsoft Research]] |
| - | * [[http://www.amax.com/hpc/testdrive.asp|AMAX Xeon Phi test drive]] | + | |
| - | HPC application hosting (paid): | + | OSUOSL hosting and OSS services, as well access to ARM and POWER machines: |
| + | * [[https://osuosl.org/services/]] | ||
| - | * [[http://www.rapidcores.com/about/|RapidCores Scientific SaaS Cloud]] | + | GRID5000 - A large-scale testbed for distributed computing, used by CS researchers in HPC, Clouds, Big Data, Networking, AI: |
| + | * [[https://www.grid5000.fr/w/Grid5000:Home]] | ||