Waiting to run (waiting for GPU memory) Redux
Now repeatable on demand! Error report available!

Back Story

Here we go... as a rule I generally run MWAH and CPDN on the same computer but at different times...I do this because they don't seem to play well together. CPDN runs overnight when my internet connection is shut down and MWAH I run during the day because it needs to constantly call the mother ship to keep itself busy with work. As a rule I generally allow either MWAH and CPDN WUs to finish their respective runs during their respective hours.

However, lately CPDN has been sending me the shorter hadam3p_eu WUs so I decided that if a particular WU could finish up in a few hours I would just let it finish its run after I did the A.M. switch-over to MWAH...so, 1 CPDN WU running and 7 MWAH WUs running...


First Issue...

Now, when the CPDN WU completes it should cause another another MWAH WU to begin from the queue, however that is not what is happening...instead, the 7 running MWAH WUs go into Waiting to run status and 8 fresh MWAH WUs are initiated from the queue. What happens next is that the 7 waiting to run WUs never begin processing...ever...they just sit in limbo status indefinitely.

There are two ways these WUs can come out of waiting status...

1. if the MWAH server runs out of WUs to send my machine, this queue of in-limbo WUs is called upon, or...

2. if I manually suspend the "Running" and "Ready to start" MWAH WUs in the queue to force the MWAH software to try an resume processing the in-limbo WUs.

And, then voila! After approximately 30-seconds of running the in-limbo WUs all return, one-by-one, a Waiting to run (waiting for GPU) status!


Second Issue...

Where this really gets interesting is that this whole cascade of events also causes the mysterious unattended spontaneous BOINC Manager restart phenomenon, as well! The log shows that about 25-minutes after the completed CPDN WU to MWAH WU transfer occurred some internal software bickering occurred and BOINC Manager just gave up and restarted itself...

14-Jan-2012 12:20:32 [Milkyway@Home] Computation for task nbody-Plum_Embedded_5557114_1 finished
14-Jan-2012 12:20:32 [Milkyway@Home] [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_5557114_1 from CS::app_finished
14-Jan-2012 12:20:32 [Milkyway@Home] [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_5557114_1 from CS::update_results
14-Jan-2012 12:20:32 [Milkyway@Home] Failed to open init file slots/7/init_data.xml
14-Jan-2012 12:20:32 [Milkyway@Home] Signature verification error for milkyway_nbody_0.84_x86_64-apple-darwin__mt
14-Jan-2012 12:20:32 [Milkyway@Home] [task] result state=COMPUTE_ERROR for nbody-Plum_Embedded_4665316_1 from CS::report_result_error
14-Jan-2012 12:20:32 [Milkyway@Home] [task] couldn't start app: Can't write init file: -108
14-Jan-2012 12:20:32 [Milkyway@Home] [task] task_state=COULDNT_START for nbody-Plum_Embedded_4665316_1 from start
14-Jan-2012 12:20:32 [Milkyway@Home] [task] task_state=COULDNT_START for nbody-Plum_Embedded_4665316_1 from resume_or_start1
14-Jan-2012 12:20:32 [---] Can't open client_state_next.xml: fopen() failed
14-Jan-2012 12:20:32 [---] Couldn't write state file: fopen() failed; giving up
14-Jan-2012 12:21:37 [---] Starting BOINC client version 6.12.35 for x86_64-apple-darwin
14-Jan-2012 12:21:37 [---] log flags: file_xfer, sched_ops, task, task_debug
14-Jan-2012 12:21:37 [---] Libraries: libcurl/7.19.7 OpenSSL/0.9.7l zlib/1.2.3 c-ares/1.6.0
14-Jan-2012 12:21:37 [---] Data directory: /Library/Application Support/BOINC Data
14-Jan-2012 12:21:37 [---] Processor: 8 GenuineIntel Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz [x86 Family 6 Model 42 Stepping 7]

That there is anyone interested looking into this further, I have saved the stdoutdae.txt file that clearly shows the sequence of events that are occurring...I'll be happy to email this with important event times marked for easy inspection.

I now believe I can repeat this on demand...just allow another CPDN WU to finish during a run of restarted MWAH WUs and...Boom. I will perform this sequence of events again to confirm repeatability.


How Can I Help?...

I would be happy to run any debugging/error-logging code someone could supply me. I am currently running the code that Ageless supplied me...

Jimmy, the developers ask you to run with a debug option on.
To do so, use any clear text editor to make a file called cc_config.xml in your BOINC Data directory (default in OS X at /Library/Applications Support/BOINC/

In it add the following lines:

<cc_config>
<log_flags>
<task_debug>1</task_debug>
</log_flags>
</cc_config>

...I'd be happy to help solve this error!

But we're not out of the woods just yet...


Third Issue...(the kick in the pants)...

Finally, to get the errant WUs to once again process and finish their tasks requires more than a simple "Shut down connected client..." command and BOINC Manager shutdown and restart. For whatever reason the errant WUs are not purged from memory though all other projects and WUs are! So a restart of BOINC Manager merely gets one back to the same situation...7 in-limbo MWAH WUs with Waiting to run (waiting for GPU) error statuses.

Bwahahahah! (Oops, sorry.) :)

To clear the problem one must then open OSX Activity Monitor and quit the stuck in-limbo WU processes manually one-by-one...each quit execution requiring user password entry to give Activity Monitor permission to execute.

Then BOINC Manager can be restarted and the once-errant MWAH WUs resume normal processing until completion.

End of Issues

Like I said, I'll be happy to send along the important files on this and run a more thorough debugging routine for you folks (if there is one and someone can supply me with code and instructions!)

That's all for now!

Phewww!

:)

Like I said, I'll be happy to send along the important files on this and run a more thorough debugging routine for you folks (if there is one and someone can supply me with code and instructions!

Thanks for the report. Regarding issue 2, it looks like the BOINC Client crashed (not the BOINC Manager). The Manager automatically restarts the Client if this happens. Of particular interest would be an excerpt from the stderr.txt file from several minutes before the errors until the time of the restart (about 12:20 through 12:22 on 14-Jan-2012.) With luck, it might contain a crash log.

You can copy and paste these into either a response on this thread or a private message to me.

Also, it might be helpful to know which version of Mac OS X you are running.

Thanks in advance.

Cheers,
--Charlie

---

Charlie Fenton
BOINC / SETI@home Macintosh & Windows Programmer

Like I said, I'll be happy to send along the important files on this and run a more thorough debugging routine for you folks (if there is one and someone can supply me with code and instructions!)

After sending the information I requested in my previous post, You might try BOINC 6.12.43 or 7.0.8, which you can dowload here. Please let us know if they solved any of these problem.

WARNING: If you run version 7.x.x, it will change the format of your data to one incompatible with the 6.12 or earlier series. If you then revert to 6.12.x, all your projects will be reset, causing you to lose all incomplete or unreported work. The projects will resend you the lost tasks, but you will start crunching them from the beginning again. To prevent this from happening, back up your BOINC Data folder before upgrading to 7.x.x, and restore it before downgrading to 6.12.x.

---

Charlie Fenton
BOINC / SETI@home Macintosh & Windows Programmer

Like I said, I'll be happy to send along the important files on this and run a more thorough debugging routine for you folks (if there is one and someone can supply me with code and instructions!

Thanks for the report. Regarding issue 2, it looks like the BOINC Client crashed (not the BOINC Manager). The Manager automatically restarts the Client if this happens. Of particular interest would be an excerpt from the stderr.txt file from several minutes before the errors until the time of the restart (about 12:20 through 12:22 on 14-Jan-2012.) With luck, it might contain a crash log.

You can copy and paste these into either a response on this thread or a private message to me.

Also, it might be helpful to know which version of Mac OS X you are running.

Thanks in advance.

Cheers,
--Charlie

Hi Charlie,

Thanks for the response and interest...hopefully we can make some headway with this...

Okay, looking in */Library/Apllication Support/BOINC Data I'm seeing...

stderrdae.txt
sttderrgfx.txt
stderrgui.txt
stdoutdae.old
stdoutdae.txt
stdoutgui.txt

...but no...

stderr.txt

...perhaps this file needs to be manually created and placed in this folder like the cc_config.xml had to be...?

(On a side note I did look inside the 3 "err" files just to see what was inside them...I got a surprise and a good chuckle with the content of stderrgui.txt...

SIGPIPE: write on a pipe with no reader
kill: kill: kill: kill: 6107875164kill: kill: kill: : 7937975125: 5916585982No such process
: : : No such process
No such process
: 77498No such process
No such process
: No such process
No such process
SIGBUS: bus error

Crashed executable name: BOINCManager
built using BOINC library version 6.12.35
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K549
Wed Oct 26 07:27:42 2011

...etc.. Ha! What's that all about?!)

:)

Okay, looking in */Library/Apllication Support/BOINC Data I'm seeing...

stderrdae.txt
sttderrgfx.txt
stderrgui.txt
stdoutdae.old
stdoutdae.txt
stdoutgui.txt

...but no...

stderr.txt

Sorry. I meant to ask you for the stderrdae.txt file.

Cheers,
--Charlie

---

Charlie Fenton
BOINC / SETI@home Macintosh & Windows Programmer

Sorry. I meant to ask you for the stderrdae.txt file.

Cheers,
--Charlie

Hi Charlie,

As you'll note (below) the contents of the stderrdae.txt contain no entries-by-date/time, unlike the stdoutdae.txt from which I offered a snippet in my original post. As mentioned, that you would like that file (stdoutdae.txt) sent along with key events noted for the time line, it's yours for the asking!

:)

Entire content of stderrdae.txt file, as requested...

getaddrinfo: No such file or directory
getaddrinfo: No such file or directory
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_separation_0.82_x86_64-apple-darwin
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
getaddrinfo: Bad file descriptor
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
getaddrinfo: Bad file descriptor
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
getaddrinfo: Bad file descriptor
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
getaddrinfo: Bad file descriptor
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108
md5_file: can't open projects/milkyway.cs.rpi.edu_milkyway/milkyway_nbody_0.84_x86_64-apple-darwin__mt
md5_file: Too many open files
error: verify_file: md5_file error -108

Hi Charlie,

I also asked about the possibility of running some extra debugging/verbose error-logging code on my end...is there something in that arena I can do for you folks? I do have a queue of CPDN WUs nearing end-of-run that I can put to use to trigger this bug.

:)

Charlie,

Crash over at Einstein has also reported seeing 'waiting for GPU':

I have E@H running on multiple Apple laptops - but on the MBP 1.5GHz OS10.5.8 with Boinc 6.12.35 (PowerPC), E@H loads tasks that take 52 hours (they take considerably longer to complete than that) and it currently has a task that has been at "running, high priority" for over 10 days at 100.000% complete.

It also sometimes put up a message "waiting for GPU" but their is no separate GPU on this machine.

It has other E@H tasks that it cycles through but by now I would have thought that the prime task would have completed and downloaded as it is now also over 10 days past its deadline.

Claggy

...but no...

stderr.txt

...perhaps this file needs to be manually created and placed in this folder like the cc_config.xml had to be...?

No. The stderr.txt file is created by the project's science app in whatever slots directory the task has been assigned to. When the task finishes, the stderr.txt file is uploaded to the project server and kept as part of the task details on the project website. To view the task details and stderr.txt, go to the project website, click Your Account -> View Computers on this account. Locate the computer in question from the list of your computers, in the left-hand column click Tasks to see the list of tasks recently issued to that computer. In the left-hand column click "show names". Now from the Event Log you quoted earlier the task for which Charlie wants to see the stderr.txt is named "nbody-Plum_Embedded_5557114_1". Locate that name in the left-hand column and click it to see the details and the stderr.txt for that task. The text below the Stderr Output title is the stderr.txt plus a few extra lines.

Task details are not kept forever if you can't find nbody-Plum_Embedded_5557114_1 in the list it may already be deleted from the list. Or it may be on the next page so look for the "Next 20" link to get to the next page of the list.

Hi Dagorath,

Oofah, I just noticed this post from you (I've been skipping to the page bottom everytime I check this post and missed this one in the process!). My apologies!

Unfortunately, at this point, the WUs have since vanished from my online MWAH completed tasks list...however, I have been keeping my CPDN WUs on hold waiting for further instructions on how to log this series of events, so I'm just going to go ahead and let one reach its end-of-run to try and trip this glitch again. I will be sure to note all of the MWAH and CPDN WUs involved and then gather all the online stderr.txt reports to go along with the relevant stdoutdae.txt timeline log files.

FWIW, NBody 5557114 was just one of 7 MWAH WUs affected by this...keeping track of 7WUs in the timeline will be a task of sorts, I'll see what I can do to make the sequence of events seem coherent.

I still have the stdoutdae.txt from that day saved as a separate text document (not active, that is)...if anyone would like to look through it, it clearly shows what was happening to each WU for both CPDN and MWAH...I'll be happy to send it along! Perhaps it will yield some clues as to what to watch out for for the next occurrence?!

:)

No problem. I think Charlie said he wants the stdoutdae.txt not stderr.txt.

Thanks Dagorath...I've waited to get a train wreck all lined up (See: Second Issue... above, the dreaded unattended spontaneous BOINC Manager restart variant of Waiting to run (waiting for GPU)! Woo hoo!) and have sent Charlie the associated files along with some notes and screen shots as to what is transpiring...

I'm "ready, willing and able" to crash your software for you...let me know what I can do for you folks before I pull the pin!

:)

let me know what I can do for you folks before I pull the pin!

Tell us how to get to your bank-account. ;-)

---

let me know what I can do for you folks before I pull the pin!

Tell us how to get to your bank-account. ;-)

Bring a flashlight and a magnifying glass when you go...and a small brown paper bag...just big enough for a peanut butter sandwich, no bigger...and, actually, that'll be too big, so bring the sandwich along also... :)

LOL

You got mice in there, ey? :-)

---

LOL

You got mice in there, ey? :-)

...yeah...you can leave 'em some of the sandwich if you like, the milk's on me... :)

I sent both Charlie and Dagorath a PM on this...are you interested in these files, too? I'll be happy to send them along (I've still got your email). Like I mentioned, I can crash this anytime you folks are ready, I'm just asking for some advanced warning on what files, info, etc. you'll need me to track down along with any dev code or such you'd like me to run that will help flush out whatever details you folks need.

:)

No Jimmy, I don't need your logs, but thanks for thinking of me. They're in good hands when sent to Charlie and he'll be able to tell you what else he needs/wants you to do. :)

---

If he's around today, and you can get in contact with him, I've got some free time on my hands today and would be happy work with him over the phone if that would make life easier (than constantly checking emails and posts!). Let me know, I'll be happy to send send him my number.

:)

Midday Update:
Okay, so much for tripping these WUs manually...MWAH ran out of WUs on its own accord and tripped the dreaded Waiting to run (waiting for GPU memory) errors on its own when it tried to put these errant WUs back to work!...

Task 81153514 = ps_separation_82_2s_mix2_3_2212143_0
Task 81154859 = ps_separation_82_2s_mix1_2_5269588_0
Task 81156265 = ps_separation_82_2s_mix0_3_2213973_0
Task 81157670 = ps_separation_82_2s_mix3_3_2213578_0
Task 81158385 = ps_separation_82_2s_mix4_3_2167426_2
Task 81204351 = ps_separation_82_2s_coadd_2_2216702_0
Task 81231096 = ps_separation_82_2s_mix2_3_2219069_1

They are all now sitting in my queue with Waiting to run (waiting for GPU memory) and will have to be manually cleared from memory to run properly.

Following the fate of WU ps_separation_82_2s_mix2_3_2212143_0 we come up with this series of events...

Waiting for GPU Train Wreck log.txt:16: Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix2_3_2212143_0 from start
Waiting for GPU Train Wreck log.txt:17: Wed Jan 25 12:35:47 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_mix2_3_2212143_0 using milkyway version 82
Waiting for GPU Train Wreck log.txt:90: Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix2_3_2212143_0 exited
Waiting for GPU Train Wreck log.txt:91: Wed Jan 25 12:36:22 2012 | Milkyway@Home | Task ps_separation_82_2s_mix2_3_2212143_0 exited with zero status but no 'finished' file
Waiting for GPU Train Wreck log.txt:93: Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix2_3_2212143_0 from handle_premature_exit
Waiting for GPU Train Wreck log.txt:95: Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix2_3_2212143_0 from temporary exit


Full gory details below...

Wed Jan 25 12:35:19 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix2_3_2257194_0 checkpointed
Wed Jan 25 12:35:19 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix2_3_2267326_0 checkpointed
Wed Jan 25 12:35:28 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268358_0 checkpointed
Wed Jan 25 12:35:28 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268355_0 checkpointed
Wed Jan 25 12:35:28 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268344_0 checkpointed
Wed Jan 25 12:35:28 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268332_0 checkpointed
Wed Jan 25 12:35:29 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268357_0 checkpointed
Wed Jan 25 12:35:29 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268353_0 checkpointed
Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix2_3_2267326_0 exited
Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix2_3_2267326_0 from handle_exited_app
Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:35:47 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix2_3_2267326_0 finished
Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix2_3_2267326_0 from CS::app_finished
Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix2_3_2267326_0 from CS::update_results
Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17418
Wed Jan 25 12:35:47 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix2_3_2212143_0 from start
Wed Jan 25 12:35:47 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_mix2_3_2212143_0 using milkyway version 82
Wed Jan 25 12:35:49 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix2_3_2257194_0 exited
Wed Jan 25 12:35:49 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix2_3_2257194_0 from handle_exited_app
Wed Jan 25 12:35:49 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:35:49 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix2_3_2257194_0 finished
Wed Jan 25 12:35:49 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix2_3_2257194_0 from CS::app_finished
Wed Jan 25 12:35:49 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix2_3_2257194_0 from CS::update_results
Wed Jan 25 12:35:49 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17421
Wed Jan 25 12:35:49 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix1_2_5269588_0 from start
Wed Jan 25 12:35:49 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_mix1_2_5269588_0 using milkyway version 82
Wed Jan 25 12:35:51 2012 | Milkyway@Home | Sending scheduler request: To fetch work.
Wed Jan 25 12:35:51 2012 | Milkyway@Home | Reporting 2 completed tasks, requesting new tasks for CPU
Wed Jan 25 12:35:52 2012 | Milkyway@Home | Scheduler request completed: got 2 new tasks
Wed Jan 25 12:35:52 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7289759_0 from handle_scheduler_reply
Wed Jan 25 12:35:52 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7289769_0 from handle_scheduler_reply
Wed Jan 25 12:35:53 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7289769_0 from CS::update_results
Wed Jan 25 12:35:53 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7289759_0 from CS::update_results
Wed Jan 25 12:35:54 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7289769_0 from CS::update_results
Wed Jan 25 12:35:54 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7289759_0 from CS::update_results
Wed Jan 25 12:35:55 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix3_3_2268344_0 exited
Wed Jan 25 12:35:55 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix3_3_2268344_0 from handle_exited_app
Wed Jan 25 12:35:55 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:35:55 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix3_3_2268344_0 finished
Wed Jan 25 12:35:55 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix3_3_2268344_0 from CS::app_finished
Wed Jan 25 12:35:55 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix3_3_2268344_0 from CS::update_results
Wed Jan 25 12:35:55 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17422
Wed Jan 25 12:35:55 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix0_3_2213973_0 from start
Wed Jan 25 12:35:55 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_mix0_3_2213973_0 using milkyway version 82
Wed Jan 25 12:35:56 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix3_3_2268353_0 exited
Wed Jan 25 12:35:56 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix3_3_2268353_0 from handle_exited_app
Wed Jan 25 12:35:56 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:35:56 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix3_3_2268353_0 finished
Wed Jan 25 12:35:56 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix3_3_2268353_0 from CS::app_finished
Wed Jan 25 12:35:56 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix3_3_2268353_0 from CS::update_results
Wed Jan 25 12:35:56 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17423
Wed Jan 25 12:35:56 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix3_3_2213578_0 from start
Wed Jan 25 12:35:56 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_mix3_3_2213578_0 using milkyway version 82
Wed Jan 25 12:35:57 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix3_3_2268332_0 exited
Wed Jan 25 12:35:57 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix3_3_2268332_0 from handle_exited_app
Wed Jan 25 12:35:57 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:35:57 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix3_3_2268332_0 finished
Wed Jan 25 12:35:57 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix3_3_2268332_0 from CS::app_finished
Wed Jan 25 12:35:57 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix3_3_2268332_0 from CS::update_results
Wed Jan 25 12:35:57 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17424
Wed Jan 25 12:35:57 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix4_3_2167426_2 from start
Wed Jan 25 12:35:57 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_mix4_3_2167426_2 using milkyway version 82
Wed Jan 25 12:35:58 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix3_3_2268355_0 exited
Wed Jan 25 12:35:58 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix3_3_2268355_0 from handle_exited_app
Wed Jan 25 12:35:58 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:35:58 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix3_3_2268355_0 finished
Wed Jan 25 12:35:58 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix3_3_2268355_0 from CS::app_finished
Wed Jan 25 12:35:58 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix3_3_2268355_0 from CS::update_results
Wed Jan 25 12:35:58 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17426
Wed Jan 25 12:35:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_coadd_2_2216702_0 from start
Wed Jan 25 12:35:58 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_coadd_2_2216702_0 using milkyway version 82
Wed Jan 25 12:35:59 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix3_3_2268357_0 exited
Wed Jan 25 12:35:59 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix3_3_2268357_0 from handle_exited_app
Wed Jan 25 12:35:59 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:35:59 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix3_3_2268357_0 finished
Wed Jan 25 12:35:59 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix3_3_2268357_0 from CS::app_finished
Wed Jan 25 12:35:59 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix3_3_2268357_0 from CS::update_results
Wed Jan 25 12:35:59 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17427
Wed Jan 25 12:35:59 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix2_3_2219069_1 from start
Wed Jan 25 12:35:59 2012 | Milkyway@Home | Restarting task ps_separation_82_2s_mix2_3_2219069_1 using milkyway version 82
Wed Jan 25 12:36:00 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix3_3_2268358_0 exited
Wed Jan 25 12:36:00 2012 | Milkyway@Home | [task] task_state=EXITED for ps_separation_82_2s_mix3_3_2268358_0 from handle_exited_app
Wed Jan 25 12:36:00 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:36:00 2012 | Milkyway@Home | Computation for task ps_separation_82_2s_mix3_3_2268358_0 finished
Wed Jan 25 12:36:00 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for ps_separation_82_2s_mix3_3_2268358_0 from CS::app_finished
Wed Jan 25 12:36:00 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for ps_separation_82_2s_mix3_3_2268358_0 from CS::update_results
Wed Jan 25 12:36:00 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17432
Wed Jan 25 12:36:00 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix3_3_2268295_0 from start
Wed Jan 25 12:36:00 2012 | Milkyway@Home | Starting task ps_separation_82_2s_mix3_3_2268295_0 using milkyway version 82
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix2_3_2212143_0 exited
Wed Jan 25 12:36:22 2012 | Milkyway@Home | Task ps_separation_82_2s_mix2_3_2212143_0 exited with zero status but no 'finished' file
Wed Jan 25 12:36:22 2012 | Milkyway@Home | If this happens repeatedly you may need to reset the project.
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix2_3_2212143_0 from handle_premature_exit
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task called temporary_exit(600.000000)
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix2_3_2212143_0 from temporary exit
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=SUSPENDED for ps_separation_82_2s_mix1_2_5269588_0 from suspend
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=SUSPENDED for ps_separation_82_2s_mix0_3_2213973_0 from suspend
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=SUSPENDED for ps_separation_82_2s_mix3_3_2213578_0 from suspend
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=SUSPENDED for ps_separation_82_2s_mix4_3_2167426_2 from suspend
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=SUSPENDED for ps_separation_82_2s_coadd_2_2216702_0 from suspend
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=SUSPENDED for ps_separation_82_2s_mix2_3_2219069_1 from suspend
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=SUSPENDED for ps_separation_82_2s_mix3_3_2268295_0 from suspend
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17439
Wed Jan 25 12:36:22 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7275251_1 from start
Wed Jan 25 12:36:22 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7275251_1 using milkyway_nbody version 84
Wed Jan 25 12:36:24 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix1_2_5269588_0 exited
Wed Jan 25 12:36:24 2012 | Milkyway@Home | Task ps_separation_82_2s_mix1_2_5269588_0 exited with zero status but no 'finished' file
Wed Jan 25 12:36:24 2012 | Milkyway@Home | If this happens repeatedly you may need to reset the project.
Wed Jan 25 12:36:24 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix1_2_5269588_0 from handle_premature_exit
Wed Jan 25 12:36:24 2012 | Milkyway@Home | [task] task called temporary_exit(600.000000)
Wed Jan 25 12:36:24 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix1_2_5269588_0 from temporary exit
Wed Jan 25 12:36:28 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7275251_1 exited
Wed Jan 25 12:36:28 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7275251_1 from handle_exited_app
Wed Jan 25 12:36:28 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:36:28 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7275251_1 finished
Wed Jan 25 12:36:28 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7275251_1 from CS::app_finished
Wed Jan 25 12:36:28 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7275251_1 from CS::update_results
Wed Jan 25 12:36:28 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17445
Wed Jan 25 12:36:28 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7274863_1 from start
Wed Jan 25 12:36:28 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7274863_1 using milkyway_nbody version 84
Wed Jan 25 12:36:30 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix0_3_2213973_0 exited
Wed Jan 25 12:36:30 2012 | Milkyway@Home | Task ps_separation_82_2s_mix0_3_2213973_0 exited with zero status but no 'finished' file
Wed Jan 25 12:36:30 2012 | Milkyway@Home | If this happens repeatedly you may need to reset the project.
Wed Jan 25 12:36:30 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix0_3_2213973_0 from handle_premature_exit
Wed Jan 25 12:36:30 2012 | Milkyway@Home | [task] task called temporary_exit(600.000000)
Wed Jan 25 12:36:30 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix0_3_2213973_0 from temporary exit
Wed Jan 25 12:36:31 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix3_3_2213578_0 exited
Wed Jan 25 12:36:31 2012 | Milkyway@Home | Task ps_separation_82_2s_mix3_3_2213578_0 exited with zero status but no 'finished' file
Wed Jan 25 12:36:31 2012 | Milkyway@Home | If this happens repeatedly you may need to reset the project.
Wed Jan 25 12:36:31 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix3_3_2213578_0 from handle_premature_exit
Wed Jan 25 12:36:31 2012 | Milkyway@Home | [task] task called temporary_exit(600.000000)
Wed Jan 25 12:36:31 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix3_3_2213578_0 from temporary exit
Wed Jan 25 12:36:32 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix4_3_2167426_2 exited
Wed Jan 25 12:36:32 2012 | Milkyway@Home | Task ps_separation_82_2s_mix4_3_2167426_2 exited with zero status but no 'finished' file
Wed Jan 25 12:36:32 2012 | Milkyway@Home | If this happens repeatedly you may need to reset the project.
Wed Jan 25 12:36:32 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix4_3_2167426_2 from handle_premature_exit
Wed Jan 25 12:36:32 2012 | Milkyway@Home | [task] task called temporary_exit(600.000000)
Wed Jan 25 12:36:32 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix4_3_2167426_2 from temporary exit
Wed Jan 25 12:36:33 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_coadd_2_2216702_0 exited
Wed Jan 25 12:36:33 2012 | Milkyway@Home | Task ps_separation_82_2s_coadd_2_2216702_0 exited with zero status but no 'finished' file
Wed Jan 25 12:36:33 2012 | Milkyway@Home | If this happens repeatedly you may need to reset the project.
Wed Jan 25 12:36:33 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_coadd_2_2216702_0 from handle_premature_exit
Wed Jan 25 12:36:33 2012 | Milkyway@Home | [task] task called temporary_exit(600.000000)
Wed Jan 25 12:36:33 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_coadd_2_2216702_0 from temporary exit
Wed Jan 25 12:36:34 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7274863_1 exited
Wed Jan 25 12:36:34 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7274863_1 from handle_exited_app
Wed Jan 25 12:36:34 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:36:34 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7274863_1 finished
Wed Jan 25 12:36:34 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7274863_1 from CS::app_finished
Wed Jan 25 12:36:34 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7274863_1 from CS::update_results
Wed Jan 25 12:36:34 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17450
Wed Jan 25 12:36:34 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7272359_1 from start
Wed Jan 25 12:36:34 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7272359_1 using milkyway_nbody version 84
Wed Jan 25 12:36:35 2012 | Milkyway@Home | [task] Process for ps_separation_82_2s_mix2_3_2219069_1 exited
Wed Jan 25 12:36:35 2012 | Milkyway@Home | Task ps_separation_82_2s_mix2_3_2219069_1 exited with zero status but no 'finished' file
Wed Jan 25 12:36:35 2012 | Milkyway@Home | If this happens repeatedly you may need to reset the project.
Wed Jan 25 12:36:35 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix2_3_2219069_1 from handle_premature_exit
Wed Jan 25 12:36:35 2012 | Milkyway@Home | [task] task called temporary_exit(600.000000)
Wed Jan 25 12:36:35 2012 | Milkyway@Home | [task] task_state=UNINITIALIZED for ps_separation_82_2s_mix2_3_2219069_1 from temporary exit
Wed Jan 25 12:36:40 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7272359_1 exited
Wed Jan 25 12:36:40 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7272359_1 from handle_exited_app
Wed Jan 25 12:36:40 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:36:40 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7272359_1 finished
Wed Jan 25 12:36:40 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7272359_1 from CS::app_finished
Wed Jan 25 12:36:40 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7272359_1 from CS::update_results
Wed Jan 25 12:36:40 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17456
Wed Jan 25 12:36:40 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7272323_1 from start
Wed Jan 25 12:36:40 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7272323_1 using milkyway_nbody version 84
Wed Jan 25 12:36:46 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7272323_1 exited
Wed Jan 25 12:36:46 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7272323_1 from handle_exited_app
Wed Jan 25 12:36:46 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:36:46 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7272323_1 finished
Wed Jan 25 12:36:46 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7272323_1 from CS::app_finished
Wed Jan 25 12:36:46 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7272323_1 from CS::update_results
Wed Jan 25 12:36:46 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17461
Wed Jan 25 12:36:46 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7269146_1 from start
Wed Jan 25 12:36:46 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7269146_1 using milkyway_nbody version 84
Wed Jan 25 12:36:52 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7269146_1 exited
Wed Jan 25 12:36:52 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7269146_1 from handle_exited_app
Wed Jan 25 12:36:52 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:36:52 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7269146_1 finished
Wed Jan 25 12:36:52 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7269146_1 from CS::app_finished
Wed Jan 25 12:36:52 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7269146_1 from CS::update_results
Wed Jan 25 12:36:52 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17467
Wed Jan 25 12:36:52 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7264177_1 from start
Wed Jan 25 12:36:52 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7264177_1 using milkyway_nbody version 84
Wed Jan 25 12:36:57 2012 | Milkyway@Home | Sending scheduler request: To fetch work.
Wed Jan 25 12:36:57 2012 | Milkyway@Home | Reporting 11 completed tasks, requesting new tasks for CPU
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7264177_1 exited
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7264177_1 from handle_exited_app
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:36:58 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7264177_1 finished
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7264177_1 from CS::app_finished
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7264177_1 from CS::update_results
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17473
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7255463_1 from start
Wed Jan 25 12:36:58 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7255463_1 using milkyway_nbody version 84
Wed Jan 25 12:36:58 2012 | Milkyway@Home | Scheduler request completed: got 11 new tasks
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7289889_0 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix0_3_2099910_1 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7289892_0 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_coadd_2_986425_1 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7225211_1 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix1_2_5335037_1 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7289900_0 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_coadd_2_2184664_1 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_5660309_2 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix2_3_2217589_1 from handle_scheduler_reply
Wed Jan 25 12:36:58 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7289885_0 from handle_scheduler_reply
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix2_3_2217589_1 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix1_2_5335037_1 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix0_3_2099910_1 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_coadd_2_986425_1 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_coadd_2_2184664_1 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7289900_0 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7289892_0 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7289889_0 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7289885_0 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7225211_1 from CS::update_results
Wed Jan 25 12:36:59 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_5660309_2 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix2_3_2217589_1 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix1_2_5335037_1 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix0_3_2099910_1 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_coadd_2_986425_1 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_coadd_2_2184664_1 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7289900_0 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7289892_0 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7289889_0 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7289885_0 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7225211_1 from CS::update_results
Wed Jan 25 12:37:00 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_5660309_2 from CS::update_results
Wed Jan 25 12:37:04 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7255463_1 exited
Wed Jan 25 12:37:04 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7255463_1 from handle_exited_app
Wed Jan 25 12:37:04 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:04 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7255463_1 finished
Wed Jan 25 12:37:04 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7255463_1 from CS::app_finished
Wed Jan 25 12:37:04 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7255463_1 from CS::update_results
Wed Jan 25 12:37:04 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17478
Wed Jan 25 12:37:04 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_5652511_2 from start
Wed Jan 25 12:37:04 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_5652511_2 using milkyway_nbody version 84
Wed Jan 25 12:37:10 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_5652511_2 exited
Wed Jan 25 12:37:10 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_5652511_2 from handle_exited_app
Wed Jan 25 12:37:10 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:10 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_5652511_2 finished
Wed Jan 25 12:37:10 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_5652511_2 from CS::app_finished
Wed Jan 25 12:37:10 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_5652511_2 from CS::update_results
Wed Jan 25 12:37:10 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17484
Wed Jan 25 12:37:10 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7289769_0 from start
Wed Jan 25 12:37:10 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7289769_0 using milkyway_nbody version 84
Wed Jan 25 12:37:16 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7289769_0 exited
Wed Jan 25 12:37:16 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7289769_0 from handle_exited_app
Wed Jan 25 12:37:16 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:16 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7289769_0 finished
Wed Jan 25 12:37:16 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7289769_0 from CS::app_finished
Wed Jan 25 12:37:16 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7289769_0 from CS::update_results
Wed Jan 25 12:37:16 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17489
Wed Jan 25 12:37:16 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7289759_0 from start
Wed Jan 25 12:37:16 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7289759_0 using milkyway_nbody version 84
Wed Jan 25 12:37:22 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7289759_0 exited
Wed Jan 25 12:37:22 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7289759_0 from handle_exited_app
Wed Jan 25 12:37:22 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:22 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7289759_0 finished
Wed Jan 25 12:37:22 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7289759_0 from CS::app_finished
Wed Jan 25 12:37:22 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7289759_0 from CS::update_results
Wed Jan 25 12:37:22 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17495
Wed Jan 25 12:37:22 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7289900_0 from start
Wed Jan 25 12:37:22 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7289900_0 using milkyway_nbody version 84
Wed Jan 25 12:37:28 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7289900_0 exited
Wed Jan 25 12:37:28 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7289900_0 from handle_exited_app
Wed Jan 25 12:37:28 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:28 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7289900_0 finished
Wed Jan 25 12:37:28 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7289900_0 from CS::app_finished
Wed Jan 25 12:37:28 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7289900_0 from CS::update_results
Wed Jan 25 12:37:28 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17501
Wed Jan 25 12:37:28 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7289892_0 from start
Wed Jan 25 12:37:28 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7289892_0 using milkyway_nbody version 84
Wed Jan 25 12:37:34 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7289892_0 exited
Wed Jan 25 12:37:34 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7289892_0 from handle_exited_app
Wed Jan 25 12:37:34 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:34 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7289892_0 finished
Wed Jan 25 12:37:34 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7289892_0 from CS::app_finished
Wed Jan 25 12:37:34 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7289892_0 from CS::update_results
Wed Jan 25 12:37:34 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17506
Wed Jan 25 12:37:34 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7289889_0 from start
Wed Jan 25 12:37:34 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7289889_0 using milkyway_nbody version 84
Wed Jan 25 12:37:40 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7289889_0 exited
Wed Jan 25 12:37:40 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7289889_0 from handle_exited_app
Wed Jan 25 12:37:40 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:40 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7289889_0 finished
Wed Jan 25 12:37:40 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7289889_0 from CS::app_finished
Wed Jan 25 12:37:40 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7289889_0 from CS::update_results
Wed Jan 25 12:37:40 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17512
Wed Jan 25 12:37:40 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7289885_0 from start
Wed Jan 25 12:37:40 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7289885_0 using milkyway_nbody version 84
Wed Jan 25 12:37:46 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7289885_0 exited
Wed Jan 25 12:37:46 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7289885_0 from handle_exited_app
Wed Jan 25 12:37:46 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:46 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7289885_0 finished
Wed Jan 25 12:37:46 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7289885_0 from CS::app_finished
Wed Jan 25 12:37:46 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7289885_0 from CS::update_results
Wed Jan 25 12:37:46 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17517
Wed Jan 25 12:37:46 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_7225211_1 from start
Wed Jan 25 12:37:46 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_7225211_1 using milkyway_nbody version 84
Wed Jan 25 12:37:52 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_7225211_1 exited
Wed Jan 25 12:37:52 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_7225211_1 from handle_exited_app
Wed Jan 25 12:37:52 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:52 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_7225211_1 finished
Wed Jan 25 12:37:52 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_7225211_1 from CS::app_finished
Wed Jan 25 12:37:52 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_7225211_1 from CS::update_results
Wed Jan 25 12:37:52 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17523
Wed Jan 25 12:37:52 2012 | Milkyway@Home | [task] task_state=EXECUTING for nbody-Plum_Embedded_5660309_2 from start
Wed Jan 25 12:37:52 2012 | Milkyway@Home | Starting task nbody-Plum_Embedded_5660309_2 using milkyway_nbody version 84
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] Process for nbody-Plum_Embedded_5660309_2 exited
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] task_state=EXITED for nbody-Plum_Embedded_5660309_2 from handle_exited_app
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] process exited with status 0
Wed Jan 25 12:37:58 2012 | Milkyway@Home | Computation for task nbody-Plum_Embedded_5660309_2 finished
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] result state=FILES_UPLOADING for nbody-Plum_Embedded_5660309_2 from CS::app_finished
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] result state=FILES_UPLOADED for nbody-Plum_Embedded_5660309_2 from CS::update_results
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix3_3_2268295_0 from unsuspend
Wed Jan 25 12:37:58 2012 | Milkyway@Home | Resuming task ps_separation_82_2s_mix3_3_2268295_0 using milkyway version 82
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17529
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix2_3_2217589_1 from start
Wed Jan 25 12:37:58 2012 | Milkyway@Home | Starting task ps_separation_82_2s_mix2_3_2217589_1 using milkyway version 82
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17534
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix1_2_5335037_1 from start
Wed Jan 25 12:37:58 2012 | Milkyway@Home | Starting task ps_separation_82_2s_mix1_2_5335037_1 using milkyway version 82
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17539
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix0_3_2099910_1 from start
Wed Jan 25 12:37:58 2012 | Milkyway@Home | Starting task ps_separation_82_2s_mix0_3_2099910_1 using milkyway version 82
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17544
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_coadd_2_986425_1 from start
Wed Jan 25 12:37:58 2012 | Milkyway@Home | Starting task ps_separation_82_2s_coadd_2_986425_1 using milkyway version 82
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17549
Wed Jan 25 12:37:58 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_coadd_2_2184664_1 from start
Wed Jan 25 12:37:58 2012 | Milkyway@Home | Starting task ps_separation_82_2s_coadd_2_2184664_1 using milkyway version 82
Wed Jan 25 12:37:59 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268295_0 checkpointed
Wed Jan 25 12:38:03 2012 | Milkyway@Home | Sending scheduler request: To fetch work.
Wed Jan 25 12:38:03 2012 | Milkyway@Home | Reporting 11 completed tasks, requesting new tasks for CPU
Wed Jan 25 12:38:04 2012 | Milkyway@Home | Scheduler request completed: got 11 new tasks
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7290007_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix1_3_2279491_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7289989_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix1_3_2279492_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7290013_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix1_3_2279493_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7290025_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix1_3_2279494_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7290026_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for ps_separation_82_2s_mix1_3_2279495_0 from handle_scheduler_reply
Wed Jan 25 12:38:04 2012 | Milkyway@Home | [task] result state=NEW for nbody-Plum_Embedded_7290027_0 from handle_scheduler_reply
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix1_3_2279495_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix1_3_2279494_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix1_3_2279493_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix1_3_2279492_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for ps_separation_82_2s_mix1_3_2279491_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7290027_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7290026_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7290025_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7290013_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7290007_0 from CS::update_results
Wed Jan 25 12:38:05 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADING for nbody-Plum_Embedded_7289989_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix1_3_2279495_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix1_3_2279494_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix1_3_2279493_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix1_3_2279492_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for ps_separation_82_2s_mix1_3_2279491_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7290027_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7290026_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7290025_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7290013_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7290007_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] result state=FILES_DOWNLOADED for nbody-Plum_Embedded_7289989_0 from CS::update_results
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17556
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix1_3_2279495_0 from start
Wed Jan 25 12:38:06 2012 | Milkyway@Home | Starting task ps_separation_82_2s_mix1_3_2279495_0 using milkyway version 82
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] ACTIVE_TASK::start(): forked process: pid 17561
Wed Jan 25 12:38:06 2012 | Milkyway@Home | [task] task_state=EXECUTING for ps_separation_82_2s_mix1_3_2279494_0 from start
Wed Jan 25 12:38:06 2012 | Milkyway@Home | Starting task ps_separation_82_2s_mix1_3_2279494_0 using milkyway version 82
Wed Jan 25 12:38:59 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268295_0 checkpointed
Wed Jan 25 12:38:59 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix2_3_2217589_1 checkpointed
Wed Jan 25 12:38:59 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix1_2_5335037_1 checkpointed
Wed Jan 25 12:38:59 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix0_3_2099910_1 checkpointed
Wed Jan 25 12:38:59 2012 | Milkyway@Home | [task] result ps_separation_82_2s_coadd_2_986425_1 checkpointed
Wed Jan 25 12:38:59 2012 | Milkyway@Home | [task] result ps_separation_82_2s_coadd_2_2184664_1 checkpointed
Wed Jan 25 12:39:08 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix1_3_2279495_0 checkpointed
Wed Jan 25 12:39:08 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix1_3_2279494_0 checkpointed
Wed Jan 25 12:39:09 2012 | Milkyway@Home | Sending scheduler request: To fetch work.
Wed Jan 25 12:39:09 2012 | Milkyway@Home | Requesting new tasks for CPU
Wed Jan 25 12:39:11 2012 | Milkyway@Home | Scheduler request completed: got 0 new tasks
Wed Jan 25 12:39:11 2012 | Milkyway@Home | No tasks sent
Wed Jan 25 12:39:11 2012 | Milkyway@Home | This computer has reached a limit on tasks in progress
Wed Jan 25 12:40:00 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix3_3_2268295_0 checkpointed
Wed Jan 25 12:40:00 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix2_3_2217589_1 checkpointed
Wed Jan 25 12:40:00 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix1_2_5335037_1 checkpointed
Wed Jan 25 12:40:00 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix0_3_2099910_1 checkpointed
Wed Jan 25 12:40:00 2012 | Milkyway@Home | [task] result ps_separation_82_2s_coadd_2_986425_1 checkpointed
Wed Jan 25 12:40:00 2012 | Milkyway@Home | [task] result ps_separation_82_2s_coadd_2_2184664_1 checkpointed
Wed Jan 25 12:40:08 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix1_3_2279495_0 checkpointed
Wed Jan 25 12:40:08 2012 | Milkyway@Home | [task] result ps_separation_82_2s_mix1_3_2279494_0 checkpointed

Hi Folks,

Just a thought here...

Seeing how these 7 flaky WUs are still sitting in memory on my end, is there a way to save them in their current state and send them along to be compared with fresh, unadulterated, unprocessed copies as found on the MWAH server? Perhaps there might be some clues in there that might shed light on what happened to them to behave this way?

FWIW, MWAH continues to send WUs to me and they are all being processed properly and being returned just fine...something happened along the way to cause these 7 to go, er, bad.

Like I said, they're still in memory on my end!

:)

The WUs involved are...

Task 81153514 = ps_separation_82_2s_mix2_3_2212143_0
http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=81153514

Task 81154859 = ps_separation_82_2s_mix1_2_5269588_0
http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=81154859

Task 81156265 = ps_separation_82_2s_mix0_3_2213973_0
http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=81156265

Task 81157670 = ps_separation_82_2s_mix3_3_2213578_0
http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=81157670

Task 81158385 = ps_separation_82_2s_mix4_3_2167426_2
http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=81158385

Task 81204351 = ps_separation_82_2s_coadd_2_2216702_0
http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=81204351

Task 81231096 = ps_separation_82_2s_mix2_3_2219069_1
http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=81231096