Optimization
log in

Advanced search

Message boards : Number crunching : Optimization

Previous · 1 · 2 · 3 · 4 · 5 . . . 9 · Next
Author Message
zioriga
Send message
Joined: 18 Dec 13
Posts: 10
Credit: 2,991,219
RAC: 1,330
Italy
Message 661 - Posted: 20 Dec 2016, 15:30:37 UTC


Looking at your host I didn't find workunits marked as anonymous platform.
Did you copy all the files in the right place? Check app_info (not app_config)


BTW in the BOINC GUI manager I don't find a reference to a local app. is this correct. What I see is a standard "Gene Application 0.09" application

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 77
Credit: 2,202,886
RAC: 0
Poland
Message 662 - Posted: 20 Dec 2016, 15:44:38 UTC - in response to Message 661.
Last modified: 20 Dec 2016, 15:47:22 UTC


Looking at your host I didn't find workunits marked as anonymous platform.
Did you copy all the files in the right place? Check app_info (not app_config)


BTW in the BOINC GUI manager I don't find a reference to a local app. is this correct. What I see is a standard "Gene Application 0.09" application

app_info file specifies user-friendly name for app, which is the same as for original app. Please check event log, somewhere at the beginning you should see line for TN-Grid app like "Found app_info.xml; using anonymous platform".

You can also check task list on your account here, in Application column you should see "Gene Network Application Unknown Platform (CPU)"

Flops parameter in app_info file is not correct for new app, I kept old one. BOINC will increase "percent done" value by 2%, remaining time will be adjusted as necessary.
____________

zioriga
Send message
Joined: 18 Dec 13
Posts: 10
Credit: 2,991,219
RAC: 1,330
Italy
Message 663 - Posted: 20 Dec 2016, 15:57:32 UTC

No message of "Found app_info.xml" and no "Unknown Platform" in the task list

zioriga
Send message
Joined: 18 Dec 13
Posts: 10
Credit: 2,991,219
RAC: 1,330
Italy
Message 664 - Posted: 20 Dec 2016, 16:19:12 UTC

The files in the directory are:

5852_Ec..........
14882.......tile.txt
app_info.xml
ec_...csv
pc
tngrid_expansion.........

zioriga
Send message
Joined: 18 Dec 13
Posts: 10
Credit: 2,991,219
RAC: 1,330
Italy
Message 665 - Posted: 20 Dec 2016, 16:37:51 UTC

And now the result

1h 32min : that is the same as the stock application

May be there is something wrong

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 77
Credit: 2,202,886
RAC: 0
Poland
Message 666 - Posted: 20 Dec 2016, 16:48:25 UTC

Try restarting boinc-client service, I had to to this on my CentOS. After doing this it should start using new app.
____________

fractal
Send message
Joined: 10 Dec 16
Posts: 2
Credit: 1,008,686
RAC: 0
Message 667 - Posted: 20 Dec 2016, 17:10:16 UTC

It took a while but tasks on my x5675 dropped from 2 1/2 hours to 1 1/2 hours with the sse custom app. It does not have any of the newer instructions.

zioriga
Send message
Joined: 18 Dec 13
Posts: 10
Credit: 2,991,219
RAC: 1,330
Italy
Message 668 - Posted: 20 Dec 2016, 17:51:55 UTC

after a system restart, now everything may be ok

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 314
Credit: 16,185,529
RAC: 4,488
Italy
Message 669 - Posted: 20 Dec 2016, 19:20:15 UTC - in response to Message 668.

I did some other benchmarks (same computer as before, Intel I7-4770k with hyper-threading enabled)

time bin/TN-Grid.linux-x86-64-sse2 5560_Ec_ecm-b0624-crcB_wu-1.input.twotiles 5560_Ec_ecm-b0624-crcB_wu-1.output.twotiles.sse2 0.05 1 2470 real 1m2.061s user 1m0.136s sys 0m0.020s time bin/TN-Grid.linux-x86-64-avx 5560_Ec_ecm-b0624-crcB_wu-1.input.twotiles 5560_Ec_ecm-b0624-crcB_wu-1.output.twotiles.avx 0.05 1 2470 real 1m3.114s user 1m1.190s sys 0m0.024s time bin/TN-Grid.linux-x86-64-fma 5560_Ec_ecm-b0624-crcB_wu-1.input.twotiles 5560_Ec_ecm-b0624-crcB_wu-1.output.twotiles.fma 0.05 1 2470 real 1m2.053s user 1m0.125s sys 0m0.024s

Not a big difference between the three versions (but a huge one compared with the original one)

rattorosso [Marche]
Volunteer developer
Send message
Joined: 19 Dec 13
Posts: 25
Credit: 1,362,771
RAC: 1,597
Italy
Message 670 - Posted: 20 Dec 2016, 21:34:29 UTC
Last modified: 20 Dec 2016, 21:35:14 UTC

Impressive! This is on my AMD FX8320:

bin/pc_avx input/tile2.txt output/output2.txt 0.05 1 2470 real 0m48.941s user 0m46.920s sys 0m0.032s diff: output/output2.txt: File o directory non esistente bin/pc_fma input/tile2.txt output/output2.txt 0.05 1 2470 real 0m47.241s user 0m45.216s sys 0m0.036s diff: output/output2.txt: File o directory non esistente bin/pc_sse2 input/tile2.txt output/output2.txt 0.05 1 2470 real 0m50.553s user 0m48.520s sys 0m0.048s diff: output/output2.txt: File o directory non esistente bin/pc_tngrid_expansion_v9_linux64 input/tile2.txt output/output2.txt 0.05 1 2470 real 1m20.760s user 1m18.744s sys 0m0.032s diff: output/output2.txt: File o directory non esistente


I'm not able to obtain a result file to compare. Is there a trick?
____________

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 77
Credit: 2,202,886
RAC: 0
Poland
Message 671 - Posted: 20 Dec 2016, 22:34:07 UTC - in response to Message 670.
Last modified: 20 Dec 2016, 22:45:11 UTC

I have just uploaded Windows binaries to https://bitbucket.org/sirzooro/pc-boinc/downloads. In order to install them, please stop BOINC, extract files to <BOINC_Data_Dir>\projects\gene.disi.unitn.it_test\ and start BOINC again. Their speed is comparable with Linux ones. They have (Opti) appended to displayed name, so you will immediately see that you run them.

Path to <BOINC_Data_Dir> depends on Windows version:
Windows 2000/XP: C:\Documents and Settings\All Users\Application Data\BOINC\
Windows Vista/Windows 7/8/8.1: C:\ProgramData\BOINC\
Windows 10: C:\Users\All Users\BOINC\

This dir may be hidden by default. You can paste path to Windows Explorer address bar go go there directly.

I did some other benchmarks (same computer as before, Intel I7-4770k with hyper-threading enabled)


Impressive! This is on my AMD FX8320:

Thanks for these results! So looks that every new instruction set used improved performance a bit. Not on all CPUs, but it is still worth testing which version is the fastest one. WUs sent by server contains 50 tiles, so actual time improvement between versions will be about 50 times bigger.

I'm not able to obtain a result file to compare. Is there a trick?

Yes. By default output file is created only when running under BOINC control. You can also pass param "BOINC_STUB=1" to make, this also will enable this. App compiled in this way does not use BOINC libs, so cannot be used for normal crunching.

@valterc If you want to compile BOINC under MinGW, you probably will have to apply patch from https://github.com/BOINC/boinc/issues/1739. For reference, I compiled it from Cygwin 64 using following command, and then used "make <all params> install". You can do this too and them copy compiled libs to Linux. They will be fine for crosscompilation.

make -f Makefile.mingw CC="x86_64-w64-mingw32-gcc -m64" CXX="x86_64-w64-mingw32-g++ -m64" BOINC_PREFIX=./boinc64
____________

Profile Sir Spuddly Buddly
Avatar
Send message
Joined: 12 Dec 16
Posts: 4
Credit: 251,095
RAC: 0
Sweden
Message 672 - Posted: 21 Dec 2016, 6:50:58 UTC

Thanks for the work done on doing these - trying out the Windows 64 bit SSE app. Running ok at the moment.

Profile [VENETO] boboviz
Send message
Joined: 12 Dec 13
Posts: 127
Credit: 877,110
RAC: 1,079
Italy
Message 673 - Posted: 21 Dec 2016, 9:10:36 UTC - in response to Message 671.

I have just uploaded Windows binaries to https://bitbucket.org/sirzooro/pc-boinc/downloads. In order to install them, please stop BOINC, extract files to <BOINC_Data_Dir>\projects\gene.disi.unitn.it_test\ and start BOINC again. Their speed is comparable with Linux ones. They have (Opti) appended to displayed name, so you will immediately see that you run them.


Ok. I see "Opti" in the name of app in boinc manager.
On my Amd FX6300 with Windows 10 64 bit, first results with Avx version:
Standard App: 9400/9800 seconds
Opti App: 3400/3500 seconds

:-O

More tests on the road....

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 314
Credit: 16,185,529
RAC: 4,488
Italy
Message 674 - Posted: 21 Dec 2016, 10:32:48 UTC - in response to Message 673.

I'm not able to obtain a result file to compare. Is there a trick?

Comment out line 650 (and the corresponding closed bracket) in pc.cpp. This way you have an exe which runs standalone or with boinc. Just take care that if using standalone it will append data to the output file (i.e. remove it before starting a new run)

This may also be related with a bug that we have, sometime we have invalid results (not too many but annoying). I guess (but I have to investigate further) that this may be related to the checkpoint mechanism at the very beginning of the run.

Col323
Send message
Joined: 23 Nov 16
Posts: 5
Credit: 1,078,477
RAC: 1,667
Angola
Message 677 - Posted: 21 Dec 2016, 15:51:12 UTC

Thanks for your work on these! I've been running the Linux AVX, the Win SSE, and Win FMA clients overnight with great success.

The linux instances are shared virtual machines. They've gone from about 21,000 seconds/WU to about 11,000 seconds.

The Windows machines are both dedicated machines. They have gone from about 10,000 seconds/WU to just under 3,600 seconds.

My only slight problem is that one Windows machine has Symantec endpoint protection installed, and it's flagged pc.exe as a potential risk. It leaves it alone so I can continue to crunch, but it does not recognize it enough to know how to classify it. Every so often it will pop up a message about the file.

Thanks again! I love doing more science in less time.

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 77
Credit: 2,202,886
RAC: 0
Poland
Message 678 - Posted: 21 Dec 2016, 17:32:01 UTC - in response to Message 677.
Last modified: 21 Dec 2016, 17:32:36 UTC

My only slight problem is that one Windows machine has Symantec endpoint protection installed, and it's flagged pc.exe as a potential risk. It leaves it alone so I can continue to crunch, but it does not recognize it enough to know how to classify it. Every so often it will pop up a message about the file.

It should not complain about anything, this is a false positive. Just to be sure I checked all binaries using metascanners metadefender.com and virustotal.com. First one checked them using 41 antiviruses and found nothing. Second one checked with 55 and one of them (Baidu) also gave false alert. BTW, both of these sites uses Symantec scanner but it did not complained about anything.
____________

Profile Skivelitis2
Avatar
Send message
Joined: 2 Dec 16
Posts: 1
Credit: 100,018
RAC: 0
United States
Message 683 - Posted: 23 Dec 2016, 1:34:19 UTC

Optimized apps are great, Daniel. Working well on all three Windows hosts. Too new to Linux (Mint 18 xfce) to properly extract there but some day I'll take the time to try to get it figured out. Thanks!
____________

manalog
Send message
Joined: 5 Oct 15
Posts: 3
Credit: 295,494
RAC: 0
Italy
Message 684 - Posted: 23 Dec 2016, 9:46:00 UTC

I tried SSE2 optimized app on my Xeon L5420 running linux and times are passed from 8500s to 4500s! It's really impressive, thank you for your work.

Profile Phil1966
Send message
Joined: 14 Jun 14
Posts: 20
Credit: 3,212,282
RAC: 0
France
Message 685 - Posted: 25 Dec 2016, 15:48:29 UTC
Last modified: 25 Dec 2016, 15:50:07 UTC

Hello,

In order to download optimzed WU's, should the "api_version" be updated to the one running on each computer ?

Have tried today, but all WU's went ended in error after 1 sec.

Is the app_info.xml sufficient or is there another the file to add / update ?

Is it possible to have a kind of "automatic" selection of the adequate WU's ? ie like Asteroids@Home (sse2/sse3/AVX - Please no FMA3 :) )

Thank You

Philippe

NB 64 WU's still reported as running but not on my computer anylonger :/

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 77
Credit: 2,202,886
RAC: 0
Poland
Message 686 - Posted: 25 Dec 2016, 18:26:49 UTC - in response to Message 685.

Hello,

In order to download optimzed WU's, should the "api_version" be updated to the one running on each computer ?

Have tried today, but all WU's went ended in error after 1 sec.

Is the app_info.xml sufficient or is there another the file to add / update ?

You do not need to modify any file, you only need to stop BOINC, unpack provided files into project directory and start BOINC.

I checked your computers and looks that app could not start at all - there was error "Couldn't start app: CreateProcess() failed - Access refused.". Please check your antivirus, probably it blocks execution of this app.

BTW, there are no optimized WUs, only app is optimized to crunch them faster.

Is it possible to have a kind of "automatic" selection of the adequate WU's ? ie like Asteroids@Home (sse2/sse3/AVX - Please no FMA3 :) )

It is possible, application could check CPU and OS capabilities and select best algorithm for them. However this is more complicated, my goal was to release all versions as a separate apps and let users decide appropriate version(s).

NB 64 WU's still reported as running but not on my computer anylonger :/

Looks like some synchronization problem. It will either disappear soon, or these WUs will time out and will be sent again.
____________

Previous · 1 · 2 · 3 · 4 · 5 . . . 9 · Next
Post to thread

Message boards : Number crunching : Optimization


Main page · Your account · Message boards


Copyright © 2017 CNR-TN & UniTN