Posted 30 Sep 2020 by
bozz4science
Well, those AMD Radeon Instinct accelerators are real compute beasts! Those 8 will be doing more combined than the rest of us :) That should be a incredible speedup in WU turnover.
The dataset we are using right now is very big, a floating point matrix 87554x1829, it's size on the hard disk is ~0.5Gb and contains, for many genes, different isoforms/transcripts. The computational time for an algorithm's iteration, because of its 'complexity', is longer than the previous experiments.
Any single gene expansion is packed up into 294 workunits. Any workunit you receive has a name like this: 142041_Hs_T155402-MAP1B_wu-(1 to 294), a counter, Hs, an internal T code, a mnemonic for the gene/isoform. The T code is just a shortcut for the gene coordinates, for example T155402 is chr5:71403265..71403276,+
Considering the original announcement of the experiment back own 2018, if I am not mistaken the dataset is a floating point matrix. Not knowing exactly what precision is needed for those computations (F16/32/64 precision), with 8 x Radeon Instinct MI 50 accelerator compute units, the peak compute capability at F16 would be 212 TFLOPs, the min at F64 at 52.8 TFLOPs. (Data according to https://www.amd.com/en/products/professional-graphics/instinct-mi50-32gb)
As of today (09/30) on the server page:
16771.51 GFLOPs of all volunteers which translates to ~16.8 TFLOPs.
The ratio by which AMD's sponsored accelerators surpass the crunchers' current computing power is a whopping ~3.1 x (at the lower F64 end).
Hope there is no flaw in my quick calculations.... If this were true anyway, that would be awesome for TN-Grid!!
... and if there is sth to the rumour of a GPU client application to be eventually deployed on TN-Grid, even my ancient GTX 750 Ti could do wonders in terms of compute capability compared to the current CPU-only powered compute force of us crunchers.
By no means I want to reignite the whole GPU/CPU debate that I am well aware of, but rather highlight the awesome performance boost through AMD's sponsored hardware as well as color this bright future of what could become reality soon.... Who knows? :)
Posted 30 Sep 2020 by
bozz4science
My queue is also drying up slowly, even though my turnover is slow and my buffer tends to be very small as well...
Edit: I just received 7 WUs (HOXC4). That's great :)
Posted 17 Sep 2020 by
bozz4science
Great accomplishment! Also looking forward to the mentioned papers!
Posted 11 Sep 2020 by
bozz4science
Thanks Robert for keeping this thread going and posting interesting stuff on a regular basis here. Seems like there is discoveries made on a daily basis that are literally advancing our understanding about it every day.
Genuinely loved this article here discussing in non-technical terms a theory (bradykinin hypothesis vs. the initial cytokine hypothesis) of why and how the non-respiratory symptoms of a Covid infection are likely to come about.
https://elemental.medium.com/a-supercomputer-analyzed-covid-19-and-an-interesting-new-theory-has-emerged-31cb8eba9d63
Can't remember who posted this link because I have seen it elsewhere but can't remember where. Definitely can recommend it though.
Thanks again
Posted 13 Aug 2020 by
bozz4science
Thank you guys for responding. Meanwhile I ran a few tasks on my X5660 Xeon based Dual boot machine with Win10 and Linux Mint and Linux indeed on average yields slightly improved but way more stable runtimes around ø 27,000 sec vs. 26,000-29,000 sec on the Win10 partition. However my "newer" laptop i5 Haswell CPU reduces the runtime by ~40-50%, so nearly cuts it in half.
I can see that the modern CPU architectures can easily outperform that ancient CPU :) Unfortunately the Xeon processor is very power hungry and not very efficient compared to modern standards, and thus is eating away the initial costs of purchase monthly in terms of electricity when running 24/7. So the transition to a modern CPU-based system is not only beneficial for increasing performance but also the only rational way I see long-term to save on electricity which is pretty expensive in Germany and comes at ~0.32 €/kWh. Looking at the specs at f.ex. a Ryzen 7 3700X chip which has only a TDP of 65 W vs. 95W of my Xeon, the comparison is just ridiculous (of course +10 yrs difference of tech innovation). A change might be long overdue as I haven't had a decent desktop for many years and been mostly doing work on a laptop from 2014...
Posted 2 Aug 2020 by
bozz4science
Thanks for getting back to me and sorry for my assumptions that lead to this fallacy. I have to admit that I didn't conduct my analysis thoroughly but you are completely right. Most of the systems, I compared my runtimes to, were Linux based and a small subset thereof were newer model Ryzen chips which obviously were the ones with the best runtimes. What they all had in common was however that all were running the avx application. That's were my naive conclusion was coming from.
I guess then that most of the performance difference is actually coming from the more powerful chips. Might be worthwhile considering to upgrade my system next year and start saving for a newer Ryzen chip. :) I see that you also have a similar setup to mine (X5650 vs. my X5660) which returns quite similar runtimes in the range of ~25-30,000 sec.
Still might try then to run a few tasks in a Linux VM and see if I can gain some performance here.
Cheers
Posted 1 Aug 2020 by
bozz4science
After having recently compared runtimes amongst my finished WUs I realised that across different OS the avx applications usually show a huge advantage over sse2 applications. Comparing those runtimes to the average runtime of my WUs (sse2 application), that performance difference amounts to a whopping ~4-5 times.
As I am currently only aware of a SSE2 application for MacOS, is there any opportunity to provide an AVX app for MacOS as well? My CPU is an i5-4278U CPU in an old MacBook that would offer support for the AVX instruction set, and thus I would love to capitalise on the efficiency that is to be gained over SSE2 apparently.
Would appreciate a short feedback! Thanks
Posted 11 Jul 2020 by
bozz4science
Thanks for the intermediate updates @Daniel and @RobertMiles. Very interesting read!
@Falconet: I hope for additional news in the next update from Valterc as well, but he has proven an awesome track record on the forum and has a great forum policy in place. We'll see if the coronavirus-related gene expansions will be able to shed some more light in the fight against it.
Posted 26 Jun 2020 by
bozz4science
Any updates on this front? any new corona-related batches? Did the prior batches already get analysed?
thx