It is a null-terminated string. And yes, it's possible that the data or part of it is outdated. I wouldn't rely on the data for calculation, in particular not when the XML is malformed (one should really use an XML library for parsing, don't use the dirty hack of the example!), but for the graphics display it should be ok to throw a possible update away.
Note BOINC uses dirty hacks instead of real XML parsers almost everywhere :( strstr(line, "") is not a valid substitute for an XML parser, is it?
Quote:
An "I'm updating" byte would work if it's the first byte, wouldn't it?
If the science app sets the byte starts updating between the graphics app reading the byte and starting parsing. ie: graphics app reads byte and it's 0, science app sets byte to 1 and updates XML, graphics app starts parsing since the byte was 0.
Re-reading the "is it updating" byte before parsing each tag may not work either due to compiler optimizations, it could notice you're just reading the same thing repeatedly and optimize out all but one read.
Quote:
Filling the whole shmem with zeros before writing something in there should at least cause a XML parsing error when it has been incompletely written.
I don't think it can be done really safe without using proper locking facilities from the OS. In theory, the graphics app could remain preempted during both the zero-filling and the useful-writing...
Of course some could call that a "worst case", but if it's possible, the race condition is there.
Using a binary format with fixed-size fields wouldn't solve the race condition, but the consequence would be reduced to having half the data updated and half not at some point; the data would never become "unreadable" (as it is with invalid XML).
Hmm wait, a possible simple solution: add a checksum somewhere. If the checksum doesn't match, read the whole thing again.
Or maybe even easier: have some integer ("version number" updated on each data refresh, preferably in binary so that there aren't field-length problems here too) before and after the XML data. If they don't match (and/or if the XML parsing fails), data was updated halfway, so it should be considered inconsistent, and needs to be read again.
Hmm wait, a possible simple solution: add a checksum somewhere. If the checksum doesn't match, read the whole thing again.
Or maybe even easier: have some integer ("version number" updated on each data refresh, preferably in binary so that there aren't field-length problems here too) before and after the XML data. If they don't match (and/or if the XML parsing fails), data was updated halfway, so it should be considered inconsistent, and needs to be read again.
Hi!
So maybe it's a good idea to split the buffer into n slots of fixed length, update slots in a cyclic manner and include a checksum AND sequence number in the messages in the slots.
Most of the time, even for n=2, at most one message will be corrupted because of a race condition. A special indicator (say first byte of the buffer) can indicate the front end of the cyclic buffer to make things easier.
So maybe it's a good idea to split the buffer into n slots of fixed length, update slots in a cyclic manner and include a checksum AND sequence number in the messages in the slots.
Most of the time, even for n=2, at most one message will be corrupted because of a race condition. A special indicator (say first byte of the buffer) can indicate the front end of the cyclic buffer to make things easier.
I thought of that too. However, now I have found some interesting stuff about mutexes, which would give *proper* locking... Apparently, I'd just need to do a wrapper around pthreads or Windows API to use mutexes (just like BOINC currently wraps shared memory functionality).
I'll try to write some code doing that, later; I'm currently busy on something else (coincidentally, writing a screensaver for another project!).
... Re-reading the "is it updating" byte before parsing each tag may not work either due to compiler optimizations, it could notice you're just reading the same thing repeatedly and optimize out all but one read. ...
Well at least in C/C++ is keyword volatile.
Directly from C/C++ reference:"The volatile keyword is an implementation-dependent modifier, used when declaring variables, which prevents the compiler from optimizing those variables. Volatile should be used with variables whose value can change in unexpected ways (i.e. through an interrupt), which could conflict with optimizations that the compiler might perform."
Might be a solution fvor this way.
But other ways can be more efficient or more stable than this one.
Mutexes are the proper way to do this. But they introduce some complexity that just for passing a bit of information just for the graphics looks like a bit of overkill to me. I'd rather wanted to keep things as simple as possible for the graphics hackers. It doesn't really matter to throw an update away or if one information gets passed and is displayed wrong for a fraction of a second. With the data curently communicated the worst thing that could actually happen is that some of the data is one update older than ome other part. With the data currently passed I don't see how that could do any harm worse than missing an update.
A reasonably easy thing to do would be to move the "timestamp" tag () to the beginning and write it (with a different tag name) at the end again. You can parse the XMl and if the don't agree, throw the information away and display that of the last update.
Mutexes are the proper way to do this. But they introduce some complexity that just for passing a bit of information just for the graphics looks like a bit of overkill to me. I'd rather wanted to keep things as simple as possible for the graphics hackers. It doesn't really matter to throw an update away or if one information gets passed and is displayed wrong for a fraction of a second. With the data curently communicated the worst thing that could actually happen is that some of the data is one update older than ome other part. With the data currently passed I don't see how that could do any harm worse than missing an update.
My idea was giving the "graphics hackers" some code that would handle shared memory, mutexes, and even the XML parsing for them. Just give out the data.
I'm downloading an update to Xcode for the Mac so I can try compiling things. Boinc says to have Xcode 2.4 or higher. It seems like getting the starsphere programs compiled should be a good little project. Getting all the correct libs will probably take a little work though since I have never worked with the Boinc framework before. Should be interesting to try.
I just spent all of last week installing MySQL and getting replication going between two of my home pc's. Replication is a great thing. Now it looks like this thread may set me on many more late nights at the computer. Nothing ever is easy when it comes to programming computers. At least not when you are starting something you haven't done before. Better make sure I have lots of coffee around.
(How to get, compile and link the BOINC library is out of the scope of this description):
Ah, well ... it seems alot of the gory bits is in that part. :-)
That is, to establish & destroy a graphics context ( canvas ) for OpenGL to have a crack at rendering upon. This part is rather platform dependent, and I'd appreciate any pointers/hints/corrections/suggestions/criticism if available. My research seems to indicate:
- find the BOINC 'graphics_2' or similarly named *.h and *.lib for the relevant platform. Wandering around here seems to be fruitful.
- ditto for the glut stuff ( gl.h glu.h glut.h ) - grab the 32 bit lib for Windoze say, plus any other platform appropriate libs.
- you'd want to declare/include/link the above in a sensible/traditional fashion to the *.c / *.h files enacting 'main' and the interface callbacks etc ..... as per Mr Myers.
- sensibly name your product executable. Put it in the right directory ie. BOINC's.
- edit the app_info.xml to inform your installation of that executable ( exit & restart Boinc ... )
[ That is I'd like to try the OpenGL + GLUT + BOINC_graphics_API approach ]
I'd like to create a template for my own artistic efforts ( cough, cough .... ) and if successful let the basic code ( +/- platform variants ) on the loose for others to substitute their own . As usual I'll be hopelessly optimistic of success in my efforts .... :-)
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
I just want to let you know that there's a refurbished version of the original Eintein@Home graphics application (starsphere) about to be released around next week. It uses SDL in conjuction with Open GL and, more importantly, provides a small but useful framework which encapsulates the following important functionalities:
* Window management
* 3D context management
* Event handling
* BOINC interfacing
* Einstein@Home interfacing
* Self-consistent build script (Linux, Mac OS, Windows/MinGW)
* Full doxygen documentation
I encourage you to have a look at this framework because it should reduce your work significantly and let's you concentrate on the actual "artistic" implementation.
RE: It is a null-terminated
)
Note BOINC uses dirty hacks instead of real XML parsers almost everywhere :( strstr(line, "") is not a valid substitute for an XML parser, is it?
If the science app sets the byte starts updating between the graphics app reading the byte and starting parsing. ie: graphics app reads byte and it's 0, science app sets byte to 1 and updates XML, graphics app starts parsing since the byte was 0.
Re-reading the "is it updating" byte before parsing each tag may not work either due to compiler optimizations, it could notice you're just reading the same thing repeatedly and optimize out all but one read.
I don't think it can be done really safe without using proper locking facilities from the OS. In theory, the graphics app could remain preempted during both the zero-filling and the useful-writing...
Of course some could call that a "worst case", but if it's possible, the race condition is there.
Using a binary format with fixed-size fields wouldn't solve the race condition, but the consequence would be reduced to having half the data updated and half not at some point; the data would never become "unreadable" (as it is with invalid XML).
Hmm wait, a possible simple
)
Hmm wait, a possible simple solution: add a checksum somewhere. If the checksum doesn't match, read the whole thing again.
Or maybe even easier: have some integer ("version number" updated on each data refresh, preferably in binary so that there aren't field-length problems here too) before and after the XML data. If they don't match (and/or if the XML parsing fails), data was updated halfway, so it should be considered inconsistent, and needs to be read again.
RE: Hmm wait, a possible
)
Hi!
So maybe it's a good idea to split the buffer into n slots of fixed length, update slots in a cyclic manner and include a checksum AND sequence number in the messages in the slots.
Most of the time, even for n=2, at most one message will be corrupted because of a race condition. A special indicator (say first byte of the buffer) can indicate the front end of the cyclic buffer to make things easier.
RE: So maybe it's a good
)
I thought of that too. However, now I have found some interesting stuff about mutexes, which would give *proper* locking... Apparently, I'd just need to do a wrapper around pthreads or Windows API to use mutexes (just like BOINC currently wraps shared memory functionality).
I'll try to write some code doing that, later; I'm currently busy on something else (coincidentally, writing a screensaver for another project!).
RE: ... Re-reading the "is
)
Well at least in C/C++ is keyword volatile.
Directly from C/C++ reference:"The volatile keyword is an implementation-dependent modifier, used when declaring variables, which prevents the compiler from optimizing those variables. Volatile should be used with variables whose value can change in unexpected ways (i.e. through an interrupt), which could conflict with optimizations that the compiler might perform."
Might be a solution fvor this way.
But other ways can be more efficient or more stable than this one.
Mutexes are the proper way to
)
Mutexes are the proper way to do this. But they introduce some complexity that just for passing a bit of information just for the graphics looks like a bit of overkill to me. I'd rather wanted to keep things as simple as possible for the graphics hackers. It doesn't really matter to throw an update away or if one information gets passed and is displayed wrong for a fraction of a second. With the data curently communicated the worst thing that could actually happen is that some of the data is one update older than ome other part. With the data currently passed I don't see how that could do any harm worse than missing an update.
A reasonably easy thing to do would be to move the "timestamp" tag () to the beginning and write it (with a different tag name) at the end again. You can parse the XMl and if the don't agree, throw the information away and display that of the last update.
BM
BM
RE: Mutexes are the proper
)
My idea was giving the "graphics hackers" some code that would handle shared memory, mutexes, and even the XML parsing for them. Just give out the data.
I'm downloading an update to
)
I'm downloading an update to Xcode for the Mac so I can try compiling things. Boinc says to have Xcode 2.4 or higher. It seems like getting the starsphere programs compiled should be a good little project. Getting all the correct libs will probably take a little work though since I have never worked with the Boinc framework before. Should be interesting to try.
I just spent all of last week installing MySQL and getting replication going between two of my home pc's. Replication is a great thing. Now it looks like this thread may set me on many more late nights at the computer. Nothing ever is easy when it comes to programming computers. At least not when you are starting something you haven't done before. Better make sure I have lots of coffee around.
RE: (How to get, compile
)
Ah, well ... it seems alot of the gory bits is in that part. :-)
That is, to establish & destroy a graphics context ( canvas ) for OpenGL to have a crack at rendering upon. This part is rather platform dependent, and I'd appreciate any pointers/hints/corrections/suggestions/criticism if available. My research seems to indicate:
- find the BOINC 'graphics_2' or similarly named *.h and *.lib for the relevant platform. Wandering around here seems to be fruitful.
- ditto for the glut stuff ( gl.h glu.h glut.h ) - grab the 32 bit lib for Windoze say, plus any other platform appropriate libs.
- you'd want to declare/include/link the above in a sensible/traditional fashion to the *.c / *.h files enacting 'main' and the interface callbacks etc ..... as per Mr Myers.
- sensibly name your product executable. Put it in the right directory ie. BOINC's.
- edit the app_info.xml to inform your installation of that executable ( exit & restart Boinc ... )
[ That is I'd like to try the OpenGL + GLUT + BOINC_graphics_API approach ]
I'd like to create a template for my own artistic efforts ( cough, cough .... ) and if successful let the basic code ( +/- platform variants ) on the loose for others to substitute their own . As usual I'll be hopelessly optimistic of success in my efforts .... :-)
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
Hi Mike, I just want to
)
Hi Mike,
I just want to let you know that there's a refurbished version of the original Eintein@Home graphics application (starsphere) about to be released around next week. It uses SDL in conjuction with Open GL and, more importantly, provides a small but useful framework which encapsulates the following important functionalities:
* 3D context management
* Event handling
* BOINC interfacing
* Einstein@Home interfacing
* Self-consistent build script (Linux, Mac OS, Windows/MinGW)
* Full doxygen documentation
I encourage you to have a look at this framework because it should reduce your work significantly and let's you concentrate on the actual "artistic" implementation.
Stay tuned,
Oliver
Einstein@Home Project