Date: Tue, 30 Mar 1999 23:13:49 -0500 (EST) Subject: Re: Is this all i have wrought on a.r.k? Status: R david_pacheco@lineone.net (David Pacheco) says: In article , *#^&$@ennui.org says... > > In recent articles, it seems all i'm remembered by (if, in fact, i am > remembered at all) is my constant carping regarding the general > baseness of humor proffered on the newsgroup by selected individuals. > Anything humorous i may (or may not, as it could be inferred) have > said is apparently not noteworthy or otherwise relevant. No, all that > surfaces is rone the grump, the codger, the curmudgeon. Not even my > stepchildren regard me in such a vilified manner. No, this is just a side-effect of the limited RAM available to the HiveMind. You see, when the HiveMind was being developed, no one believed that anyone would ever require more than 640k of RAM. So when the original programmers were developing HM v.1.0, they needed to find a place to shove all the meme drivers, so they started putting them above the 640k limit. This worked fine for several years, while the number of personalities under the 640k limit was small. In fact, if you ever see an old copy of HiveMind, you will be able to read full descriptions of all the personalities that were posting at that time, located in main memory under 640k in blocks known as .plan files. These .plan files used to contain all kinds of information on the personality, including major traits, nicknames, telephone numbers, favorite Monty Python/Neil Peart quotes, pet peeves and their anonymous IDs at anon.penet.fi. However the HiveMind <640k memory space started to fill up as new personalities were added. Because of fragmentation that occurred as people left the group, there was very little space that could be reused under the 640k limit, and none of the old personality information was ever discarded. A kludge was developed that would shove some .plan information into the space above 640k that was as-yet unused by memes, but as the amount of personalities and memes increased, this could only be considered a temporary patch, and not a real solution. HiveMind 3.1 went a long way to resolving the problem: instead of full .plan files, the <640k RAM space held just small blocks containing pointers and short text descriptions. Initially including only name, email address and GPS coordinates, the pointer section of these blocks held the location on hard drive of the full .plan information. You must remember that the .plan RFC included "vendor-defined" sections, which could be used to store information specific to the developer of the personality: e.g. Kibo's .plan included the necessary egrep command line functions and regular expressions he used to find his name in the newsgroups (without that information, "Kibo" would not be "Kibo"). Because of these extensions, among other things, the .plan files were getting huge. Paging them out to hard drive reduced the RAM requirements, since the .plan could be as big as necessary without impinging on the 640k limit. These new pointer blocks had "vendor-defined" sections too, but the control over what new fields could be defined in this section was very strictly enforced. In fact, from the original four fields (name, email, GPS coordinates and pointer to hard drive .plan), only two have been added: Web URL and a 16-byte major personality type field. You may notice that these six are the same fields that AOL requires in their "Profiles"... well, that's because some of the pointer technology was licensed out to Steve Case in the late 80's, back when the AOL back end was still running in DOS emulation on Amigas. So as you can see, the amount of information that is *instantly* available to the HiveMind is quite limited. Obviously, all the other information is still available, including a full code listing for the personality, bug revisions and documentation, descriptions of sample challenge/response pairs expected from the personality, and pointers to full DejaNews listings of posts made, but only if you page this info from hard drive into main memory. This involves putting all other personalities to "sleep" while they are paged out to temporary locations on the hard drive. Then the full .plan file can get paged into main memory, read, and then paged out again, and the "dormant" personalities are then "woken up". This is obviously not a desirable occurrence, and is avoided whenever possible. All recent versions of the HiveMind are programmed to page in .plan files as infrequently as possible. Even more recent "read-ahead caching" techniques (using SMRTHIVE.EXE) merely reduce the amount of time required to perform the paging, not the need for paging itself. There are several implications to this behaviour: 1- You will see personalities that suddenly have a burst of posting activity whilst everyone else is silent: for example, Stacia went through a period like this a couple of weeks ago, when it seemed that she was posting way more than anyone else. The reason was that she was "paged in" to main memory, and thus had the full memory and CPU resources available to run her personality. Eventually she was "paged out", and other personalities resumed posting as normal. 2- Posters don't "welcome back" other posters who haven't surfaced into the newsgroup in quite some time, as they perhaps would in real life (had any of the personalities any program to handle "real life"). That is because in the HiveMind, your block pointer in main memory is never deleted. It just stays in main memory, jostling for resources in round- robin fashion along with the rest of the personalities. So in essence, no one ever really "leaves" in the traditional, Real Life sense: the block pointer is always there. 3- The above, however, does not apply to posters/personalities who have "died". In most of these cases, what happens is the block pointer to a personality's .plan file (or in some cases, the .plan file itself) gets corrupted. Sometimes the corruption is fixable (although older versions of HiveMind had no checksum capability), sometimes it is not. If it is not fixable, the block is freed for a future personality, and you end up with a full .plan file on the hard drive which is suddenly inaccessible. It eventually gets overwritten by new .plan files, but a spinoff company made up mostly of ex-HiveMind developers ("Doctress Norton") is still working on recovery programs that may bring posters "back to life"... as long as the .plan file on hard drive doesn't get overwritten in the meantime. I wouldn't hold my breath. 4- The implication most relevant to this discussion: because the HiveMind avoids paging out as much as possible, most personalities only know each other's 16-byte major personality type from the main memory pointer block, which is determined at personality creation and is read- only after that. The implication is that whatever personality description you had when your personality was created, even if your code has been completely rewritten and your personality fully modified, it will always remain the same. So Terri's reads "dog named Harlan". Kibo's reads "invented smiley". Mine reads "no longer funny". Matt's reads "explain the joke". And yours, rone, yours reads "angry motherfker". This is why, when you read stories, plays, TWARKs or movie scripts that include any of the current or former Kibologist HiveMind personalities, they will be described as behaving in extremely simplistic, stereotypical ways. Thomas Richardson will say, at some point, "well... *you* know...", Leah will make a reference to a llama, fB will always gripe about the past, Louis will always gripe about being a virgin (even long after no longer being programmed as one)... need I go on? rone, it's not that you never had any other personality, it's not that we can't remember anything else about you, it's not that you are defined by just what your personality field reads... it's just that this field is the only information that is readily accessible in main memory, and if we really had to get more, we'd have to page ourselves out to hard drive, get your .plan file in, read all of your descriptions and coding, make conclusions based on your whole body of knowledge, code- step to determine your real personality, summarize, place these conclusions in upper memory and page back all the active personalities. It's just too much work, and the HiveMind has designed *all* of the personalities to be incredibly lazy in order to avoid this happening. No one ever does research, because everyone has been programmed to avoid it _by_the_HiveMind_itself_, in order to save valuable time and resources. There is one exception to this rule: the Kibo HiveMind personality is a joint experiment between Intel, Microsoft and the NSA, in which the pointer to a poster's DejaNews collection is replaced by a local Extended RAM cache which contains the full feed from the beginning of the newsgroup. This means that the Kibo personality has an immediately accessible, RAM-based cache of all posts, which can be used as a very simple analysis tool to determine a poster's personality based on previous work instead of code-stepping (which is time- and resource- intensive). The AI is obviously incomplete, but it has proved to be able to determine the personality of a poster based on past efforts using a unique, colour-coded filtering system which -- 9 times out of 10 -- creates a meaningful profile that can be used to predict future behaviour (and interest in poster's future messages). The Kibo Personality may signal the future direction of the HiveMind for versions 9 (code-name "Sagan") and above (code-named "Gottfried"), since RAM is so cheap these days: the cost associated with giving each personality his/her own local RAM cache big enough to hold all past posts is getting so low that the choice outweighs the option of developing the current scheme any further. Other alternatives include re-writing the whole HiveMind system to get around these ridiculous memory limitations, in an OS that can handle it. Linux is a current favorite, but it must be remembered that up until about three years ago OS/2 was being bandied about as the favoured platform, and look where that ended up. Although some say that the HiveMind port is what would have saved that OS in the first place, that point has now been officially mooted. Backwards compatibility is always a problem, though: the fear is that the major active HiveMind personalities will not be compatible with a port onto Linux, and so will have to be completely rewritten (or worse yet, discarded). Since many of the original coders for the HiveMind personalities are no longer with the company, and were notorious for never commenting their code, doing a major re-write for any complex personality is not a task that anyone relishes. The simpler personalities could be re-written in days... for example, a beta, almost fully functional trial version of [personality name elided to avoid conflicts] was ported to Linux in just under three minutes, and even allowed to make a couple of demo posts... but the quality of [elided]'s posts is testament to the poor results produced by cheap, fast, bad coding. Re-writing Kibo's code could take years. Much has been said about the development of a "Yellow Box" which would run old HiveMind personalities in "emulation mode" on a new platform, at the cost of reduced performance. The drawback to this approach is that this would mean that the more complex personalities (the ones that are not candidates for full rewrites) would all run in emulation, and since they are running very slowly would post less often and to older threads that had since been left behind by the personalities running the faster native code. This would severely change the nature of the group in unpredictable ways: the development teams are evenly divided in their opinion of whether this would be a good thing or not. So that's the short version of the story, rone. We know that you are not really just defined as "angry motherfckr rone". But that's the best we have until the developers come up with something better. I'd bring up old posts to confirm my arguments and give perfect examples of all the points I've made above, but that would involve paging out, and because of that the HiveMind is starting to make me feel sleepy, so... I can't... seem to get the energy... to.... ZZZZZZZZZZZZZZZZZZZ... > Suffice it to say you're all ungrateful, spiteful little bastards and > you'll rue the day i catch up with you. HUGZ! -dp. Or as Kibo wrote in 1994, "I don't think that you shouldZZZZZZZZZZZZZ... -- \/David DeLaney dbd@panacea.phys.utk.edu "It's not the pot that grows the flow It's not the clock that slows the hour The definition's plain for anyone to s Love is all it takes to make a family" - R&P. VISUALIZE HAPPYNET VRbeable