Author |
Message |
|
I've fished and uploaded all wus on my 6 core pc (also due to the beginning of Rita Levi Montalcini Challenge on Rosetta).
Now i have 4,3 Gb of used space by your project (it's a lot of space). Is it normal? Can i clean up the project folder? |
|
|
|
I think you can clean the project folder, there should be no problem.
Given the number of WUs you have elaborated, I think that data is normal, but we'll investigate this matter, in order to reduce the average used disk space.
Thank you for your cooperation |
|
|
|
Once the upload of the results are completed you can delete everything,
Even if you don't do it, Boinc should automatically take care of deleting things if it's running low on the space reserved to him. Our project take up a lot of space because we work with big files. I agree that we need to try to reduce the file size, maybe by keeping them compressed.
|
|
|
|
Also, some of the files are marked as sticky because in theory they are needed by more than a WUs. Problem is that boinc has not an intelligent scheduler with respect to sticky files.
The best behavior would be to send new WUs minimizing the number of files to be downloaded, hence WUs which you can compute using the files you've already downloaded; this mechanism is not provided so far. This implies that you probably are storing many sticky files which you just used once or twice but you're client is keeping them because they are sticky.
Right now we are addressing other, more critical issues, but I think we should think whether mark those files as sticky is really necessary.
It depends on some other factors such as the length we decide for the WUs and the variability in the length |
|
|
|
I think that data is normal
I don't think so. Here some projects i run on my pc and "residual data"
Ralph@home: 250 mb
WCG: 31 mb
Poem: 7 mb
My fear is that, with next download, there are more than 10gb....
|
|
|
|
Right now we are addressing other, more critical issues, but I think we should think whether mark those files as sticky is really necessary.
Yeap, this is not a critical problem now, with little work.
But in the future, may be.
Ah, i forget:
Congratulations to everyone for the great work you are doing!!
|
|
|
|
In the future we'll address the problem for sure!
However, here's some data.
The disk space is mostly needed to store those sticky files I mentioned before. I took a look in my BOINC folder and I have many of them, many probably obsolete.
I took 3 of them, whose sizes were 241, 70 and 68 MB and using different compression algorithm I achieved compression factors around 38%, which means that when we'll use the compression you'll spare almost 2/3 of the space.
From 4.3GB of used space we'll go to 1.7GB
It is still a lot, but it is something.
Moreover, we'll need to talk to the professor and the preprocessing group in order to understand the size of these files and to determine the WUs length.
We may choose to have a unique WU for each file, which is the worst choice in terms of bandwidth consumption, but is probably the best in terms of disk usage, since the file would not be sticky anymore and would be removed at the end of the WU's computation.
EDIT: I run some more test and it seems the best choice would be to use bzip2.
The compression factor is almost as high as in lzma but the compression time is much lower |
|
|
|
Ah, i forget:
Congratulations to everyone for the great work you are doing!!
Thank you for your valuable advices! ;)
____________
Paolo - Application team dev (SSC11)
"If you were plowing a field, which would you rather use: two strong oxen or 1024 chickens?" Seymour Cray |
|
|
|
I think you can clean the project folder, there should be no problem.
Waiting for new work, i stopped wu request and cleaned the project folder (6gb).
But, after some minutes, boinc client begins to download files (.csv and Others) and NO wus. I killed the download, but it restarted. It has happened several times... |
|
|
|
I reset the project in order to test the new application version, as soon as there will be new wus. It didn't happen to me that the client downloaded files without wus.
Did you try to reset the project instead of clean the directory? Could you try to remove the project and re-add. |
|
|