Ossa Sepia

It started, as usual, with players’ activity, this time automated bots and desire to gain access to such, payment concerns and the available avenues to reward useful effort. The conversation got longer than usual and spanning a couple of days, too, but perhaps the snippet below is the best introduction here, since it is quite part of what triggered this publishing of my key now rather than at a later time – if he’s going to play with Vamp on the keys side, might as well give him something specific to play with/against, right?

While the client for Eulora 2 handles fully automatically the whole business of RSA keys and thus uniquely recognized identities, while the environment provides support for a Web of Trust (WoT) as well, the latest activity by players pushed suddenly quite strongly forward the private interactions and even rsa-backed contracts that people might want to have with one another. And since I am best placed to know exactly what is available, how it works and how it might serve this purpose, here’s my public euloran key made specifically for player to player interaction ¹:

my player’s public key in Eulora 2

Simply download the above file, run your copy of vamp to unhex it and then use the result with Vamp as you would any other public key. If you publish or send over in Eulora 2 your key as well, we can try a full back and forth communication out.

Should add here that this is simply a public RSA key like any other – only plainly rather than opaquely stored and passed around. So to the extent that you can extract the actual RSA key out of whatever other software you happen to use ², you can then in principle use that just as well with vamp ³, it doesn’t care.

1. This may or may not be the same public key that I use for my euloran identity. Note that within Eulora 2 itself, the server is the central authority with which everyone has to authenticate and maintain the unbroken authenticated communication in order to appear in game as the respective character that they are handling. So within the game, if any X says something, it is indeed rsa-authenticated X saying that something – at least as long and to the extent that you trust the server of the game, indeed.[↩]
2. Kleopatra most likely if you are on windows, gpg on linux but in either case, attempting to extract *just the key* is a good exercise to see for yourself just how little access to the actual controls the user is supposed to ever have. You know, one of those basic, repeatedly and respectfully cited everywhere Kerckhoffs’s principles, yes? [↩]
3. For completeness, the euloran RSA keys limit the length of the public exponent e, hence that trail of zeroes in the file above.[↩]

~ This continues the Euloran Kronikle I ~

At Preposterously Prosperous Place

In the infinite land of Eulora 2, one can easily get lost if relying solely on the landscape for clues. As one player promptly found out when going to where the others should have been, only to be welcomed by an entirely empty space with no PCs nor NPCs in sight- is this part of the world the one I wanted, the one I needed or merely the one I happen to have stumbled in by accident, when going on auto-pilot and just looking away from the keyboard for a while? Luckily, the chat works from any and all places in and even out of the euloran world, so one doesn’t have to either wonder forever alone or indeed hope for no help at all. All it takes is to tell others of one’s trouble and there’s a good chance to receive some guidance towards finding one’s own way back:

As a wider benefit of both misadventure and telling others about it, the situation finally gave a good reason for the euloran world to give names to places – names that people can refer to, so that they can find their own way around and even tell others where they are, with some reasonable degree of confidence that they might even be talking indeed about the same place! So the world changed indeed and places got named, with the growing settlement where it all started finding itself in the very middle of Preposterously Prosperous Place (or PPP, for short). Some known places around seem to include at the moment the Limitless Ditch of Intoxication, the Cruel Mound of Pluck, the Secluded Course of Verve, the Goatish Hill of Destiny and the Saturnine Lane of Destiny:

As for the explorer, all ended quite well for him too, as he got back to PPP and even found that, getting lost can really truly pay in all sorts of ways, when one is finally not lost anymore:

Life and Death Get Vitals Moving

As PCs and NPCs truly got moving around the infinite lands, the very vitals of each of them also started moving – if in a more limited sense between the usual percentage values of 0 and 100 but decidedly up and down, as everyone noticed. Just as some vitals replenish with rest when at least enough food is had, others seem to drain with it. And then, one day, two of the oldest characters around were not quite that closely around anymore as the towncrier provided their epitaphs that gave perhaps some indication as to what could have done it for them only it didn’t, as well as what actually did it, indeed:

As it turns out, one truly died of too effective ineffectualness, you know? This game might really be noticing something there…

An NPC’s Tease

While the best known NPCs are staying in town and providing services to everyone, there’s a growing number of less known NPCs that are busy with their own trials and tribulations in the vast lands of Eulora 2. One of those seems to have set up, whether intentionally or not, quite the tease for any player:

Despite being thus publicly exposed as the author of the previously unexplained phenomenon of apparently exploding amounts of bags appearing in place, he seems to be still at it. More recent reports suggest he can still be found by the trail he leaves behind:

Merchant Dips Hands in Banking

The official merchant in Eulora 2, the one and only Rainer Tracksnarl recently dipped his hands in banking as well, to further improve the service he provides for his customers. Happily for eulorans everywhere, the official merchant’s hands are quite clean indeed, so that this dipping of his comes at no extra cost, while bringing the wanted benefit of being finally able to exchange between the many denominations of the euloran coin, the ECu. Players being human though, they quickly found, of course, that there is always more that they could want:

One Towncrier Gets Busy: Daily Rankings and Other Notices

It all started – again! – with active players talking of what they did, where they are, what they found and what they think of it. As it happened before with the naming of places, it is the players’ own activity that pushes for some concrete improvement or addition – and in this case, it was the leadership boards that came out of this:

Less than a week later, everything was done and in place, so that the towncrier got quite busy indeed, with a brand new Notices and Announcement board that he is in charge of. So the towncrier is happily busy now, announcing all rankings as they get calculated and even pointing out the juiciest news in the more general, public chatroom:

Better to CLI than to Click: the Stacking Experience

While NPCs benefit from Rainer Tracksnarl’s automated methods for tidying their inventory as they go, human players have had to do it manually until now. As activity increases though, the number of items grows as well and one might find (to their surprise or not) that clicking to move items about may be “very intuitive” but it quickly starts being rather too annoying, as well:

As before, increased activity of the industrious and helpful kind also pushes for increased support from Eulora 2, so players got an improved bot command line so that the full stacking can take as little as one short line in the Console regardless of how many items there are that need moving. Alternatively, it can still take, of course, twice as many clicks as there are items to move, if one sticks to point-and-click rather than learning to use the command line. I’d say it’s far better for you to CLI than to Click but you are certainly free to take as long as you wish to find this out, there’s no rush at all:

What Comes Next?

This one really and truly depends on players’ activity. As for the Kronikle, this part is easiest of them all – what comes next after this II is going to be III! When it will come out and whether it will still be manually compiled by me like these first two or not and what exactly it will contain and reveal from the euloran world is yet an open matter.

WoT ¹ ratings are supposedly a shorthand themselves – just plain, integer numbers between -10 and +10, with every value meant to stand for something. For what, exactly? Well, given how few they are and how diverse the situations can be, the only possible answer is that it… depends and mostly on the rater, really. More to the point, these rating values can be used and assigned in any way one wants but are generally assumed to reflect to some extent at least, the degree to which one trusts one’s own knowledge and understanding of specific others, as they are met and interacted with.

But people being people, plain numbers as shorthand never really work as well as words where any meaning is intended. Hence, in practice, any consistent and systematic approach to actually using WoT ratings ends up quite quickly with at least one, possibly several word-based shorthand schemas for the ratings themselves. And while no such schema can ever be somehow “the one” or definitive or whatever else of this sort, there is perhaps something to be gained at least from making public a first such schema – as an example or maybe even just as a first attempt perhaps, if nothing else.

As to how this first schema even came to be – in Eulora 2, WoT ratings are used not just by players as they see fit but by the NPCs and generally by the game itself. Hence, there was the need for at least one specific schema that the code can rely on to give and interpret at the very least its own ratings – though possibly as well those of others in contexts where it is meant to interpret them. And it turned out in fact quite fun to attempt a consistent approach in finding meaningful shorthand names for all those rating values! So here is the current result, with negative values simply adding a “mis” prefix in front of the same word to mark the mis-alignment between rater and ratee:

• 0 remembered (no negative here, it’s the one rating stating that one has interacted with this person and they want to keep this marked but they prefer not to give either a negative or a positive rating – hence, the rated person is simply remembered)
• 1 contacted, -1 miscontacted
• 2 acquainted, -2 misaquainted
• 3 perceived, -3 misperceived
• 4 encountered, -4 misencountered
• 5 believed, -5 misbelieved
• 6 known, -6 misknown
• 7 trusted, -7 mistrusted
• 8 respected, -8 misrespected
• 9 admired, -9 misadmired
• 10 fated (the main meaning of “fated” that I am using here is that of “determined” as in unchangeable in fact because this is pretty much what I see a 10/-10 rating validly standing for – the ratee is clearly and fully determined now as in the future, whether this is for instance because they are a known deterministic code bot or perhaps simply dead and thus not further able to either act or change at all, in any way), -10 misfated ².

This is all, the first known public shorthand schema for WoT ratings, feel free to agree, disagree or even just argue with it, whether quietly on your own or more wordy in company, in the comments section below.

1. Web of Trust, see what it is for, how it works and how to use it.[↩]
2. According to the Oxford dictionary, the word “misfated” is “obsolete” and “last recorded around 1600s.” So I suppose I’m reviving words here, even if I didn’t set out to do this on purpose. I’m not even quite sure how or why would a word obtained via a perfectly plain and common prefix ever be obsolete but in any case, here it is then “last recorded” again, quite more recently than the 1600s.[↩]

A ton of tubers a day keeps hunger at bay – or at least at a small cove, perhaps

Adventurous explorers in Eulora 2 reportedly found that there is at least some benefit to eating the tubers one digs. Even if not all around nutritious as it might be desired, they are at least not poisonous either and can even give a much needed (if at times rather small towards negligible) boost to one’s hitpoints (hp) and bloodpoints (bp). Imagine then if one could one day perhaps cook them into something even more useful than this…

Crafting supplies are in but who has the money for it?

The long-awaited crafting supplies are indeed fully and freely ¹available at the one and only official merchant, Rainer Tracksnarl. He duly advertised them in chat, players duly checked them out and barely managed to buy anything at all from all the piles:

One-legged craft-table reportedly crafts just fine

Active eulorans have recently spotted a one-legged craft-table in the wild, apparently crafting nevertheless just fine. As precarious as the whole thing looked at first sight, its contents proceeded to transform as expected and the output came in quite good looking too, so all is fine and in good euloran tradition – after all, if the table had two (or more!) legs to start with, what leg would there be left (or even right) for further improvement?

How many clumps of grass does it take to make one coarse, frangible thread?

This is the question on eulorans’ lips, as crafting gets a slow but quite determined start. Opinions on the matter are diverse but also changing quite often, it would seem. From a reported starting 1 to anything between 4 and 5 currently, there seems to be no limit to it. And it all seems quite similar to an earlier public discussion on the little bits of nothing required to build a single tiny claim:

Merchant gets a grip on his runaway inventory

The prominent merchant of all things euloran, Rainer Tracksnarl, recently got a large shipment of crafting items of all sorts, neatly packed and carefully stacked by type and quality. The shipment is holding well but the stacking reportedly went quickly wild with all the players’ attention going through it, so that the merchant found himself with thousands of items scattered around in his inventory. While nothing was lost, lots was thus misplaced and even more was just taking up space, pushing desired items so far away that some said they could not even find them at all anymore.

Confronted with this unexpected issue of his runaway inventory, Tracksnarl finally took matters in his own methods and invested in a reliable re-stacking after each attempted exchange. Results so far look tidy, compact and utterly worth it, especially as players get to use the neat compact packing for their own inventories, too:

NPCs ² wonder out loud

As NPCs are increasingly active providing their services and answering questions in chat, it emerged that they may on occasion engage in conversations of their own. On one such occasion, they seem to be quite curious as to what humans might even be:

1. Possibly “Free as in free speech, not free beer” or more plainly and aptly put: publicly traded, not gratuitously given.[↩]
2. Non-playing characters. Because they work in the game, they dont’ just play it like PCs do, obviously.[↩]

~This is a work in progress, to be discussed and expanded as the game progresses further. This article brings together and builds on the work already done and the preliminary concepts that were already described in previous articles, including a first draft of a data hierarchy, its subsequent refinement and an initial brief discussion of how it works in practice.~

To support players in their quest to explore, make their name and write their own history in the infinite and infinitely expanding world of Eulora2, the game’s client acquires and maintains at all times a dynamic view of the game’s world, as it becomes accessible and known to each player and as it responds to their actions or as it changes otherwise, with time or due to other players’ actions and to the game’s own internal rules.

The above means specifically that the game’s client successfully retrieves and makes use of any new data in its relevant context on the fly, as it becomes available and without knowing it upfront – because it simply can *not* be known upfront, since it’s the players’ actions that decide the very evolution of the game’s world! Playing Eulora2 truly means exploration and discovery in full, not just a pretense kept alive by some sort of agreement between server and client to merely not show or not use some data until the server says it’s “available” and the client is “updated” to have it and be able to handle it. In Eulora2, the player doesn’t need to avoid for whatever reason to even look at the files on their own disk. Quite on the contrary and unlike in any other game to date, in Eulora2 the player is warmly invited to look at the files on their own disk, to make full use of all the data and knowledge that they can access at any time, to write or use their own bots and anything and everything else that they can gather or even think of.

The game’s own client as I made it to date supports the above by design, as it is entirely and fully dedicated to serve the player with all it has and all it can actively and continuously acquire. These are not empty words but fundamental principles of mine and thus I made the client according to them, quite on purpose: it continuously checks, discovers and acquires new data and new knowledge of the world as it becomes available and it makes full use of it and of any new assets on the fly and as unobtrusively and quickly as it can.

How is such continuous discovery of knowledge and on the fly use of new assets even possible? In short, it’s made possible by knowing the very syntax of the game’s world representation – thus, instead of trying to enumerate upfront the world entire and getting stuck as soon as a new element is added, my client for Eulora2 simply starts off knowing how to discover and make sense of new elements as they appear.

My client starts without any of the game data but with the full ability to acquire and make sense of any and all such data, as soon as it has a working connection to the server. The way it does this is by knowing the rules of how the Euloran world describes itself, the very syntax of world knowledge in Eulora2. Unlike the usual game clients that start by having upfront a ton of data and graphics assets, my client starts without any of those but with a full working knowledge of the solid structure that contains and gives meaning to such data within the game. In other words, my client starts knowing how to obtain and make sense of data and then it proceeds to do exactly that, for as long as it runs and handling without any trouble all sorts of new situations as they appear, supporting thus the expansion of the player’s own increasing knowledge of the game’s world as it evolves with active play.

Knowledge of the game’s world as seen by any given player in Eulora2 is at all times a tree – a finite tree for sure but one that can have any number of branches and any number of levels, changing continuously as well, in response to the player’s actions and to the evolution of the game’s world as the time passes. To support the player, my client repeatedly explores this tree, pruning dead branches, growing new ones as relevant, updating and further interrogating parts of current interest and showing to the user the corresponding world view and its changes as they are obtained from the server in response to specific requests ¹. Such showing of the world view is done through the graphical user interface (GUI) by default and through command line on demand. Playing the game relying entirely on the GUI alone is certainly possible but like all and any exclusive reliance on GUIs alone, this is a very limiting proposition really. Up to each player though how they play the game at any given time, for sure, but if you choose to limit yourself in some way, don’t expect that everyone else will necessarily limit themselves the same way, that’s all.

The above approach works to make sense of new content as it appears for a very simple reason: while the exact tree of knowledge for each player will be different and will keep changing throughout the game, its structure will always be syntactically correct, obeying thus at all times the known rules of describing the Euloran world. And since the game is by design and with intent fully open and encouraging players and developers to make or tweak their own clients as it serves them best, this syntax is public and will remain so at all stages. Moreover, any new additions to it will expand its capabilities as needed but without breaking at any point the compatibility with what came before – meaning that there is no requirement to “update” anything at any time. There is the option to do so or not, as and if it suits, nothing more. Like all Euloran evolution, even the world’s own language and corresponding syntax may certainly expand but in an organic manner, opening up new options on top of and building on existing ones, not contradicting nor “obsoleting” anything. In Eulora2 and as a fundamental principle, experience is an asset that keeps on giving – if one builds it up and uses it for all its worth, of course.

As for this knowledge tree itself, let’s illustrate it with an actual small example first and see how it describes the Euloran world or, more accurately, how it describes what one player happens to know about the game world at one given time:

(null, null)
- (Self, 23523903281)
-- (PlayerID, 123503920)
--- [10950238522]
-- (Salt, 8908319)
--- [2a891d0f] 
- (Sector, 513250013)
-- (NameVal, 120945693)
--- [Waypoint]
-- (Ground, 935410)
--- (CSModel, 10853092)
---- [ea3420d8982]
-- (Sky, 1930582)
--- (CSModel, 8273509)
---- [8ff24351ab92c]
-- (PC, 10950238522)
--- (NameVal, 82082501)
---- [Edward Wealthgore]
--- (Cal3DModel, 98357102)
---- [183cca82bf82e]
--- (Image, 125935601)
---- [1f98eb246caa28]
--- (PosRot, 2795927491)
---- [101, 252, 114, 124, 42, 90]
--- (Item, 9873592011)
---- (NameVal, 873798211)
----- [Coarse Cordage]
---- (CSModel, 17349272)
----- [ff32ea1d89cb0]
---- (Image, 72985711)
----- [91a0b3f28ea8d]

Reading the above would yield the following description of the world: starting from the root of the tree (that null,null first line), there is first of a all a Self (with ID 23523903281) and a Sector (ID 513250013). Further asking the server as to what exactly that Self with id 23523903281 contains yields as children nodes a PlayerID (ID 123503920) and a Salt (8908319). Both PlayerID and Salt are leaf types in our tree and this means that upon asking the server to expand further on these two IDs (123503920 and 8908319), the response will provide directly values rather than further tuples (type,ID): the actual ID of the player’s own character is thus revealed to be currently 10950238522, while the value of the Salt turns out to be 2a891d0f, standing for a file with this exact keccak sum ³.

Going back up in the tree and asking for further information about the Sector (ID 513250013) reveals that it contains a NameVal (ID 120945693), a Ground (ID 935410), a Sky (ID 1930582) and one PC with ID 10950238522. At which point, the client will have found therefore the player’s own character in game, since this ID is exactly the one indicated by that earlier PlayerID node. Given that there is currently no other PC in this sector, it seems that for the time being, this player is either alone in the world or at least not yet seeing anyone else – possibly others are somewhere in the world but not yet within interacting/seeing range.

Drilling deeper into this PC node will reveal that the player’s character goes by the name of Edward Wealthgore, is located at some specific point as given by the value of that PosRot node, carries a Coarse Cordage item and uses as 3D representation as obtained from the graphics assets packed in the file with keccak hash 183cca82bf82e (the value of the Cal3DModel node) and as 2D representation what can be obtained using the file with keccak hash 1f98eb246caa28.

Repeating the same sort of exploration as above for the remaining nodes, in turn, there will be detailed information about each, as relevant, including any contained nodes as well as graphical representations and additional characteristics. I’ll leave the detailed description of the remaining nodes in the above example as an exercise for any reader wishing to test their understanding with a practical application.

While the above may seem tedious when written down in full detail, note that it’s done at all stages internally, automatically and extremely quickly by the client code – it’s exactly the perfect task for computers anyway, using a well and clearly defined structure to hold and process data repeatedly. As the player moves about, they are likely to encounter others and perhaps items too and thus more PC and Item nodes, each with their own IDs, will appear as children of that same Sector node above. If the user drops the Coarse Cordage that they are currently carrying, the corresponding Item node will simply move from its current place as child of the player’s own PC node to become a direct child of the Sector node and perhaps other characteristics of the PC node (not shown in this minimal example) will change to reflect the lighter load carried. Similarly, if the user picks up an item, that item’s node will move to become a direct child of the player’s own PC node and other values may change as well. When the user starts some work, a corresponding node will appear, holding the relevant information and allowing thus the client to provide feedback to the user as to the effect of their action in the world etc.

As to obtaining new data and assets as they become available, the mechanism is the same throughout: on-demand discovery followed by explicit request. For instance, when the client encounters in the world tree a filename that it doesn’t currently have, it will simply request it from the server according to the lower level communication protocol that fully supports such requests and then proceed to use the file as soon as it’s fully downloaded and checked to match the expected hash. Similarly, when it encounters a new node where previously there was none, it will request further information drilling down repeatedly until it gets to the leaves and it knows thus that there is no further information to obtain.

Following exactly that earlier mentioned principle of the client aiming to fully serve the user and empower them to make use of any and all information whenever available, the player can directly interrogate and explore themselves the current tree at any time, simply via an ls command in the game’s own console, quite similarly to how one explores the tree of files on disk from the command line on any computer. Note that this is made available without being either mandatory to use nor in the way of play otherwise – the whole point is to enable and support users at any and all stages, not to force on them anything and especially not to overwhelm them with more information than they request or are able to handle at any given time. Unless asked otherwise, the client will happily do all the above in the background and simply present the results in graphical form as accurately as possible but whenever the user wants to dig deeper and take advantage of the additional flexibility and power that a command line always offers over a graphical user interface, the option is and remains entirely available.

It’s worth noting that the graphical user interface (GUI) is always and at all times a simplified and thus reduced set of options for interaction, never the reference as to what can be done or accomplished. This is simply because it can never be any other way really – the way computers work is always through a set of rules that are quite unaware and unimpended by whatever limitations are introduced by the GUI that has to pick some set of shortcuts essentially out of the full set of real possibilities. Hence, while the game can certainly be fully played through the graphical interface alone and the GUI – or at least the one I make and maintain – will at all times aim to support the user as much as possible, it’s really unlikely that any implementation of a GUI will be ever able to provide in full the same flexibility as the command line offers. For this reason, my client at least has and will always have the command line integrated, alongside the GUI, as the more flexible and ultimately more powerful way to explore and act. Moreover, since the command line is always the reference as to what can be tried and attempted too, be aware that if you limit yourself to the GUI alone, you are more likely than not to miss out.

As for the syntax of world knowledge in Eulora2, the current set of node types and their possible direct children is the following:

In addition to the above, some constants have a fixed assigned meaning in their respective contexts:

The above syntax has been in use successfully for a couple of years already so it’s not going to change fundamentally. New node types are likely to be added as the implementation proceeds further but they will be added to this list, as needed. Even at this time, I don’t quite see the case for removing some node types and even less so for altering them since that would indeed be the only way in which compatibility is broken. Nevertheless, until access to the client is widely open to everyone, consider the list above as a work in progress rather than fixed and otherwise as an invitation and opportunity to get involved perhaps, if you do your own implementation and have any questions related or otherwise relevant to it.

Any related discussion is welcome and can best happen in the comments below, as usual.

1. Note that the server for Eulora2 serves essentially as an oracle for the game – it answers any and all queries but it never pushes information unasked. This is by design, fully intended as such and entirely unlikely to ever change. There’s a wide scope for client implementations to compete though on different approaches to best make use of the server oracle for their users and thus provide an in-game advantage that players will be willing to pay for.[↩]
2. This as all numbers in here are given as example only and as such, rather shorter than the actual values.[↩]
3. Given here in hex for easy distinction of file hashes vs other values so it’s easier to follow with an actual client at hand, since the client uses this hex string to name the file on disk. Otherwise the value of that keccak sum is sent from the server directly as a number as obtained from keccak, of course.[↩]
4. This is by convention the root of the tree and there is always at all times one single root.[↩]
5. Expiry date meaning the in-game time until when current sustenance lasts.[↩]
6. birth date, if known[↩][↩]
7. This is because some items can contain other items, of course.[↩]
8. Currently used masks over this are 0x10 for Locked, 0x02 for Inaccessible[↩]
9. referenced object, if any, e.g. keys reference the item they unlock[↩]
10. This was an early test and it works fine but meanwhile my design moved away from pretending to have “dialog” with bots and towards an altogether more promising quest system. For now though this node is still possible to encounter, indeed, hence it’s shown in here.[↩]
11. 0x10 for locked, 0x0C for approved[↩][↩]
12. This is the approximate size of one unit of whatever the parent is, given in some consistently-used unit across the whole game at any given time.[↩]
13. This is the approximate capacity of one unit of whatever the parent is, given in some consistently-used unit across the whole game at any given time.[↩]
14. A faster but more restricted view of the items contained in something – the server decides at all times whether this is used or not.[↩]
15. Increased by at least 1 and at most 255 at each change.[↩]
16. Shorthand code, unique across basic items in Eulora 2, obtained as upper case letters as found in order in the item’s basic name excluding any additional prefixes and suffixes[↩]

‘Why?’
Because I need and use it.

‘What?’
The full, C-only code for zip archive operations of all sorts, entirely standalone and free of autotools and the like, ready for my intended use directly from Ada code and easily extendable for any other specific use from either C or Ada. It produces and uses standard zip archives. It comes with all needed headers, bells and whistles, a minimal .gpr file for compilation with GNAT as a static library and some commented out examples, too. The heaviest part of it is, unsurprisingly, the code for zip operations that weighs in at 10130 lines in miniz.h. The rest of the code adds about 2400 lines more in total.

To expand a tiny bit on the above, I’m using this exact code as part of the fully automated graphics pipeline that generates on demand any and all graphics that Eulora2 needs, as it needs it. I haven’t bothered to “clean” the code to any specific extent beyond the very functional – it either serves or at least stays out of my way. As such, I think the code as it is currently makes, if anything, a good place to start for someone wanting to learn to read effectively code, perhaps – it’s reasonably big without being too big and otherwise written and packed at all points by people focused on their own needs for it, not on ticking something off some check lists or using up some time at work.

I’m publishing the code as a 1st edition of MiniZipLib, in VaMP format, of course, since it can be perhaps useful to others if/when they want to avoid depending on external code for any zip-related needs.

Note also that the client for Eulora2 relies currently on something else, essentially external zip code even if this external dependency happens indirectly, via CrystalSpace. This is because all graphics assets are stored and used as .zip files that group together the various parts and pieces that are required at times to make a full model or item. While my current main interest and use for this lib is mainly serverside, the client could certainly benefit as well from reducing its dependency on external code and integrating this. Currently I don’t have the time to do it though as the further development of the game itself takes precedence but I’m making the VaMP tree for it public via this 1st edition, so that others can pick it up if they need it.

As usual, feel free to ask relevant questions in the comments, below. If you don’t know and need to find out where the code is thus published, it’s likely though that you are still missing an earlier step, namely interacting with people so read perhaps a bit more around and see what pulls you in so that there is some better starting point for it all than mere code ever was or ever will be.

As the game‘s development is progressing beyond the emergence of time, past the addition of vitals and into enabling player activities, all the design and implementation work done so far feeds as well into clarifying and refining the game-related parts of the communication protocol that were previously tentatively and incompletely stated ¹. Specifically, it’s again that ‘Character Actions’ section 7 that comes into focus but this time the changes go beyond just simple additions and move things forward from where the previous effort towards specification stopped. Consequently, I have to bring together the different parts and restate the current take on the protocol in full, in here, where it can be further iterated and discussed as needed:

This is the current take on Eulora2’s communication protocol, last revised June 13th, 2023.

1. Overall Goals:

• 1.1. All communications between clients and server to be encrypted.
• 1.2. Clients to be able to receive from server any data they lack (including maps, skins, sound or video content etcetera), on demand.
• 1.3. Clients to be able to choose and adjust both the level of security and their volume of communications with the server, as they will ultimately have to pay for the load that they generate.

2. Explicit Dependencies :

• 2.1. Eucrypt for RSA with Keccak-based OAEP and Serpent symmetric ciphering.

3. Data Structures :

3.0. Basic types :

• char / uint8 (1 byte) ;
• uint16 (2 byte) ;
• uint32 (4 byte) ;
• uint64 (8 byte) ;
• float ² (4 byte) ;

3.1. Special types:

• hash (128 bits) ;
• chunk [of file] (bitfield, 11760 bits) ;
• serpent-packet (1472 bytes) ;
• rsa-message ³ (1872 bits ⁴) ;
• rsa-packet ⁵ (1470 bytes) ;
• object (size of 104 bits ⁶: uint32 ⁷ followed by 3 uint16s representing position ⁸ followed by 3 uint8s representing rotation ⁹ ) ;
• legacy-text (size of n+n/256+1 bytes ; where the leading byte is the bytecount of the 2nd segment and the 2nd segment is the bytecount of the third segment) ¹⁰.
• text (2 byte hearder containing the ~total~ byte length ; up to 1470 bytes of text ).

4. Serpent Packets ¹¹ :

4.1. Serpent Key Set:

• uint8 (type ID, =100), followed by
• uint8 (count of keys in this set, n), followed by
• n*(4*int64 + uint32) (32 bytes each key followed by a 4 byte ID calculated through crc32 ¹² ), followed by
• an uint8 flag (LSB bit set — keys to be used to talk to client ; MSB set — key to be used to talk to server ; client-set MSB is ignored), followed by
• uint16 (message count ¹³), followed by
• padding to Serpent-message length.

4.2. Serpent Keys Lifecycle Management:

• uint8 (type ID, =102), followed by
• uint8 (count of server keys requested), followed by
• uint8 (count of client keys requested), followed by
• uint8 (id ¹⁴ of serpent key preferred for further inbound Serpent-messages), followed by
• uint8 (count of burned keys in this message), followed by
• n*int8 (id of burned key), followed by
• uint16 (message count), followed by
• padding to Serpent-message length.

4.4.a. File Request, manifest

• uint8 (type ID, =3), followed by
• hash (corresponding to the sought file ¹⁵), followed by
• uint8 (manifest packets sought count, 0=all), followed by
• n* uint16 (manifest packet index sought), followed by
• padding to Serpent-message length.

4.4.b. File Transfer, manifest (always sent and only sent in response to ID 3)

• uint8 (type ID, =4), followed by
• uint16 (count of manifest packets for this file ¹⁶), followed by
• uint16 (index of current packet in list above), followed by
• uint8 (fragment count ¹⁷), followed by
• n* uint64 (hash of the nth fragment of manifested file).
• uint16 (keccak hash of foregoing), followed by
• padding to Serpent-message length.

4.4.c. File Request, chunks

• uint8 (type ID, =5), followed by
• hash (corresponding to the sought file), followed by
• uint8 (file chunks sought count), followed by
• n* uint64 (the hash of fragment sought), followed by
• padding to Serpent-message length.

4.4.d. File Transfer, non-last chunk (always sent and only sent in response to ID 5)

• uint8 (type ID, =6), followed by
• chunk.

4.4.f. File Transfer, last chunk (sent at most once per ID 3)

• uint8 (type ID, =7), followed by
• uint16 (bytesize of useful part of the chunk following ¹⁸, followed by
• chunk ¹⁹.

4.5. Client Action ²⁰ :

• uint8 (type ID, =8), followed by
• text (fully specified action, see section 7), followed by
• uint16 (message count), followed by
• padding to Serpent-message length.

4.6. World Bulletin ²¹:

• uint8 (type ID, =9), followed by
• uint32 (id of top level item ²²), followed by
• uint8 (count of objects), followed by
• object list ²³, followed by
• uint16 (message count), followed by
• padding to Serpent-message length.

4.7. Object Request:

• uint8 (type ID, =10), followed by
• uint8 (count of objects), followed by
• n*int32 (id of object), followed by
• uint16 (message count), followed by
• padding to Serpent-message length.

4.8. Object Info:

• uint8 (type ID, =11), followed by
• uint8 (count of objects), followed by
• n times uint32 (id of object) and text (object properties, as per extant game structures, including art files needed and so on ²⁴), followed by
• uint16 (message count), followed by
• padding to Serpent-message length.

4.9 Line Request

• uint8 (type ID, =12), followed by
• uint8 (count of lines), followed by
• n times uint32 (id of chatroom), uint64 (lowest line number requested), uint64 (highest line number requested), followed by
• padding to Serpent-message length

4.10 Line Info

• uint8 (type ID, =13), followed by
• text (line content), followed by
• uint32 (chatroom ID), followed by
• uint32 (speaker ID), followed by
• uint64 (line number), followed by
• uint64 (keccak hash of the line’s content and number), followed by
• padding to Serpent-message length

Updated 30 Oct 2024: ratings are now fully integrated and used in-game, being fully handled via the relevant Rating node in the data hierarchy. Consequently there is no need anymore to have dedicated message types for discovery and retrieval, so the “rating request” and “rating info” message types below are removed entirely. Rating someone remains done as an action (see the action types further down) and discovery happens as part of the usual game play.

4.11 Rating Request

• uint8 (type ID, =14), followed by
• uint8 (count of requests), followed by
• n times uint32 (source ID), uint32 (target ID), followed by
• uint16 (message count), followed by
• padding to Serpent-message length

4.12 Rating Info

• uint8 (type ID, =15), followed by
• text(rating comment ²⁵, followed by
• uint32 (source ID), followed by
• uint32 (target ID), followed by
• uint8 (rating value ²⁶ ), followed by
• uint16 (message count), followed by
• padding to Serpent-message length

5. RSA Packets ²⁷ :

5.1. RSA key set ²⁸.

• uint8 (equal to 251 to indicate packet contains a new RSA key), followed by
• uint8 (protocol version), followed by
• uint16 (subversion), followed by
• uint32 (IP of server ²⁹), followed by
• uint32 (IP of client ³⁰), followed by
• uint64 (keccak hash of client binary), followed by
• uint64 (e of RSA key), followed by
• uint8*490 (N of RSA key), followed by
• uint64 (preferred padding — the magic value of 0x13370000 requests random padding ; all other values will be used as such, bitwise, ie like an infinite-length OTP consisting of the value repeated), followed by
• uint16 (message count), followed by
• padding to RSA-message length, 1424 (5616-8-8-16-32-64-64-3920-64-16) bits exactly.

5.2. Serpent key set ³¹:

• uint8 (equal to 157 to indicate packet contains new Serpent keys), followed by
• uint8 (count of keys ³² in this set, n; n<=19 ³³), followed by
• n*(4*int64 + uint32) (32 bytes each key followed by a 4 byte ID calculated as crc32 on the key itself), followed by
• an uint8 flag (LSB bit set — keys to be used to talk to client ; MSB set — key to be used to talk to server ; client-set MSB is ignored by server ; server will set LSB on keys requested by client for its own use thus supporting clients with no trustworthy random generators of their own), followed by
• uint16 (message count), followed by
• padding to RSA-message length.

6. Protocol Mechanics :

6.0. All communications between server and client will consist of messages. These messages may be encrypted either via eucrypt.RSA or eucrypt.Serpent. All RSA-encrypted messages will be exactly 1`470 bytes in length ; all Serpent messages will be exactly 1`472 bytes in length ³⁴ . The server will handle Serpent messages in preference of RSA messages (which are processed on an as-available basis). Clients that send garbage will be punished ; the costs involved (encryption/decryption ; generating entropy ; lookups and whatnots) will be pushed onto the client, for which reason writing the clients lightly pays off.

6.1 The handshake works as follows:

• New client issues 5.1 packet keyed to the server’s public key, including its own RSA key.
• The client’s IP is recorded, and will have to be explicitly changed by the client later if needed. Server replies with 5.1 packet keyed to the client’s announced key, including its private RSA key for use by that client ; and with 5.2 packet containing key material for client’s use in keying messages to the server. If the client fails to provide its own set of serpent keys, the server will further issue it a set of serpent keys ; thenceforth the server will send more serpent keys mirroring the client’s supply, and will similarily mirror key burning and select operations on its own set.
• Should the client’s IP change, it will issue a 5.1 packet keyed to the server public key immediately followed by a 5.1 packet keyed to the server’s original private key. The server will then update the client’s IP accordingly (this also trashes the extant Serpent keyset and triggers 5.2).
• The bulk of communication is intended to go through the Serpent system ; outside of identification and bootstrap handshakes RSA isn’t used. Should either party believe the Serpent keysets’ve been FUBAR’d, a 5.2 packet will reset that keyset.

6.2. The server will issue type 4.6 packets in response to relevant type 4.5 packets received — these can either signify the acceptance or the rejection of the client action, and the client must adjust its internal state accordingly.

7. Character Actions :

7.0. Lock:

• uint8 (type ID, =0), followed by
• uint8 (count of objects), followed by
• n* uint32 (object ids). Defaults to currently targeted item.

7.1. Make ³⁵:

• uint8 (type ID, =1), followed by
• text (title of quest), followed by
• u64 (count of how many different times can this quest be completed – note that the reward will be divided accordingly), followed by
• uint32 (arbiter id, defaults to nearest town crier), followed by
• uint32 (object id of a sample of the required item), followed by
• uint64 (wanted quality, defaults to 1), followed by
• uint64 (wanted quantity, defaults to 1), followed by
• uint8 (expiry day, defaults to 1), followed by
• uint8 (expiry month, defaults to 1), followed by
• uint64 (expiry year, defaults to next year), followed by
• uint32 (object id of reward item stack, no default)

7.2. Attempt ³⁶:

• uint8 (type ID, =2). This is a request to start the chosen activity (see next field), followed by
• uint32 (object id, indicating the intended activity, defaults to explore ³⁷), followed by
• uint32 (object id, defaults to current target or location, as relevant), followed by
• uint32 (object id, defaults to currently equipped method, followed by ³⁸)
• uint32 (object id, defaults to currently equipped tool)

7.3. Exchange:

• uint8 (type ID, =3), followed by
• uint32 (object id, the other party), followed by
• uint32 (object id, the trade itself ³⁹), followed by
• uint8 (count of objects), followed by
• n* uint32 (object ids) and uint64 (object count), followed by
• uint8 (flag, set to 0x10 to lock a trade and to 0x0c to approve a trade previously locked by both players).

7.4. Attack:

• uint8 (type ID, =4), followed by
• uint32 (object id, the other party), followed by
• uint32 (object id, the battle itself (server set, exactly in the way trade works). This not currently implemented, except player setting itself the bomb results in instadeath.

7.6. Move:

• uint8 (type ID, =6), followed by
• uint32 (destination id, defaults to current target), followed by
• uint32 (slot id), followed by
• uint32 (object id, of the item being moved), followed by
• uint32 (quantity moved).

7.7. Train:

• uint8 (type ID, =7), followed by
• uint32 (object id, the other party), followed by
• uint32 (object id, the train session itself (server set, exactly in the way trade and battle work). This not currently muchly implemented, except some NPCs train for money — but will get greatly expanded asap. Meanwhile, it actually got both expanded and implemented, see the relevant comment for more details.

7.8. Relocate:

• uint8 (type ID, =8), followed by
• object type, containing new client position.

7.9 Say ⁴⁰:

• uint8 (type ID, =9), followed by
• text (line content), followed by
• uint32 (room ID), followed by
• uint64 (keccak hash of the line’s content)

7.10 Rate:

• uint8 (type ID, =10), followed by
• text (rating comment), followed by
• uint32 (target ID)
• uint8 (rating value)

Please leave your comments below.

1. This type of iterative refinement and interaction between design and implementation is neither new nor surprising to me, really. The only difference is that I’m doing now out of necessity all sides of this. Experience helps *a lot*, of course, even or perhaps especially beyond and above what one ‘expects’ or what one ‘wants’ or what one ‘signed up for’ and so on and so forth. It might even help in the sense that it makes it possible at all, at that, even if it doesn’t make it easy by any definition of the word.[↩]
2. Floating point item deliberately not specified[↩]
3. Each such message is OAEP-padded and then encrypted with a (3920 bit) RSA key. Three such messages are strung together to form a RSA packet. Because of the significant overhead involved (both in terms of space and time), Serpent-encrypted comms are preferred whenever feasible. [↩]
4. See TMSR-RSA OAEP padding for the principle and this discussion for details.[↩]
5. This is the total size of a packet containing RSA-encrypted material. The useful size (ie payload) of such a packet is merely 702 bytes.[↩]
6. We really really want to keep this down. 13 bytes is the lowest I can conceive of, but I would so not mind halving it.[↩]
7. Representing the identifying hash of the object in question.

   We’re using the narrower size to save on network traffic — all the expenditure of another 32 bits here would buy us is de-ambiguation for cases where the count of objects around makes 1 in 2 billion collisions relevant. It doesn’t seem likely a client could support such abundance of objects.

   Note that the hashes used here are client-specific, the server doesn’t leak its own internal representation of objects to the clients.[↩]

8. Coordinates X, Y and Z in that order. Because the map goes from -512 to +512, the relationship between the given figure (GF) and map coordinates (MC) is GF / factor_coord – max_coord = MC, where max_coord is 512 and factor_coord is 65535/(2*max_coord).[↩]
9. As a full rotation is 2 pi, the relationship between the given figure (GF) and object rotation (OR) is GF / 128 * pi = OR.[↩]
10. This arrangement permits the representation of arbitrarily large textfields (2nd segment can represent up to 115`792`089`237`316`195`423`570`985`008`687`907`853`269`984`665`640`564`039`457`584`007`913`129`639`936 bytes, which is more than enough space for all the text ever produced — or likely to ever be produced — by humanity) at the modest cost of a fixed 3 byte header.

    Unfortunately, it has no longer any utility for Eulora, since we’ve moved to fixed packets. I’m preserving it here because I really like it in the abstract and it has no other place to go.[↩]

11. These packets consist of 92 successive 128 bit chunks, Serpent-enciphered individually. To extract the payload one splits the message into 92 16-byte chunks, deciphers them then collates the output into a final result. To produce the packet one cuts a 11`776 bit payload into 92 128-bit chunks, Serpent-enciphers them, and collates the results into the outbound packet.[↩]
12. Polynomial generator 0x04c11db7. Keys with null IDs are discarded and regenerated.[↩]
13. Each client and the server will keep a count of messages they sent each other. This value must be incremented on each subsequent message sent by no less than 1 and no more than 255.[↩]
14. Keys are maintained by both client and server in an ordered ring buffer 256 elements long. The server will not send more keys than the total count of 0(absent)-keys in the respective buffer, irrespective of request count. If the message contains an unknown ID or otherwise is unprocessable, the issuance of a 5.2 packet is adequate response.[↩]
15. This is the keccak hash of the actual file contents. By convention this hash rendered as a 32 alphanumeric character string is also used as the filename for the file in question.[↩]
16. This system allows up to 65`536 manifest packets, adding up to potentially 11`993`088 (65`536 * 183) fragments representing a file of up to about 140 Gb (141`038`726`624 = 11`993`088 * 11`760 + 11`744 bits exactly). This will have to be sufficient.[↩]
17. From 1 to 146 inclusive.[↩]
18. This also means the protocol does not allow the transfer of files of certain sizes (within 8 bits of a multiple of 11760), which is fine with me.[↩]
19. The final fragment of the file will have to be padded to length as per this spec. [↩]
20. This is never issued by the server.[↩]
21. This is never issued by the client.[↩]
22. As discussed in comments, the world is a hierarchical structure of objects within objects.[↩]
23. This portion will get more clarification later on! [↩]
24. The complete list of these is currently exposed by the extant client, but in any case we’ll publish a complete schematic. The server will set the “target” of the player on the last object in the list.[↩]
25. Limited to at most 1445 characters, matching the maximum length of a chat line.[↩]
26. Valid rating values are integers between -10 and 10. For a given rating value here GR, the corresponding actual rating value RV is therefore calculated as: Min(GR, 20) – 10.[↩]
27. These packets consist of three 490 byte successive chunks RSA-encrypted individually. To extract the payload one splits the message into three 490 byte chunks, RSA-decrypts and de-OAEP-pads each one, the collates the results into a final result. To produce the packet one cuts a 5`616 bit payload into three 1`872 bit chunks, OAEP-pads and encrypts them, and collates the results into the outbound packet.[↩]
28. This is the manner in which new clients register their RSA key with the server (thereby opening a new game account). Later replacement of a registered key IS NOT POSSIBLE. Keep your client’s RSA key safe.

    This is also the manner through which IP changes for an account are registered with the server. See the Protocol Mechanics heading for details.[↩]

29. This is used by the server when signalling to the client to talk to a different server (which is a thing for scaling, because different sectors will be handled by different servers).[↩]
30. If the client doesn’t know its own IP, it’s acceptable for this to be zero. [↩]
31. This permits either client or server to declare Serpent keys via RSA. It is not mandatory (as there exists a Serpent-encapsulated mechanism for the same end) but entirely legal. The server will always respond with at least one 5.2 packet after an accepted 5.1 packet creates a new player account, consisting of 40 Serpent keys to be used to talk to the server. Should the client respond with any other packet than 5.2 or 4.1, the server will send a 2nd 5.2 packet, containing 40 Serpent keys for the client’s use. [↩]
32. Keys obtained through a 5.2 packet are always indexed in the client’s buffer in the order they were found in that packet, starting with the first position.[↩]
33. A RSA packet has 702 total bytes available, of which 5 are used otherwise and the remainder of 697 are available for packing serpent keys, which take 36 bytes each (crc32 id inclusive).[↩]
34. Length being actually how they’re sorted on the server side.[↩]
35. This is meant as a production order, essentially. I expect it will get further refined as its implementation gets nearer. It already got further refined and more specifically defined as a way to issue/create Quests or “wanted” posters fundamentally, see the relevant comment on quests.[↩]
36. This is a generic initiation of player activity and as such literally an attempt, with the contents defining the exact activity. It replaces both Explore and Repair from earlier versions of this protocol but it further stands in for *any* game-defined activity even ones that may appear only at a later time. It’s as generic as it gets and quite on purpose, since this is the network layer, not the game layer. For more details on the game-defined activities, see the description of the data hierarchy.[↩]
37. Known activities are obtained like everything else in game as part of the data hierachy representing the client’s view of the world at any given time. The relevant grammar for it will be made public as it serves as the in-game protocol specification just as this serves as the network-level client-server protocol specification.[↩]
38. Known in Eulora 1 as ‘recipe’ and ‘equipped in mind’. At this level, the protocol for Eulora2 is really more generic than that, without any loss otherwise whatsoever.[↩]
39. This is set by server through a type 6 message for both players involved, the trade is an object like any other that the OP has to request. The server will also expire trades, enforce them etc.[↩]
40. Note that this is an action and as such just a part of a 4.5 message, Client Action, where the overall message structure is specified.[↩]

The quickly progressing work on euloran vitals and physical characteristics is pushing already the design – and the fun, obviously! – into previously unmentioned and thus entirely uncharted areas. This time it all started easily enough: given that at least some vitals certainly drain with activity, what does unrest truly do to one’s wellbeing, what is rest going to even look like and what can one really do -or not do, rather- about it for best results?

As all good questions, the above has, of course, a common-enough answer that isn’t much good really but still has to be properly considered, first of all: unrest tends to do more to one’s worsebeing rather than to one’s wellbeing and rest is certainly needed so introduce a ‘rest’ action that the player can choose to do whenever they feel like and the game permits! Which sounds reasonable enough and doable enough, to the point where I even sketched out a new addition to that section 7 of ‘character actions’ in the communications protocol, since it’s really not all that difficult to add or use: have another ID, give some duration in hours now that time is ticking away at everyone’s lives and use otherwise the existing infrastructure without much trouble at all. It would certainly work too, so what’s the problem with adding it, right?

Well, the problem starts with that ‘adding’ really, since the whole point of having a flexible and generic communications protocol is exactly to have a set of messages and structures that can serve a wide range of uses while being preferably as short as possible ¹. So any addition to the protocol itself comes with a non-negligible cost and thus with a strong requirement for considering first whether there is indeed enough of a fundamentally significant difference compared with already existing structures to justify a new one ². Unsurprisingly perhaps, it turns out that where euloran rest is concerned, there isn’t that strong a case for a new action type as such – or at least not yet!

So then, if resting is both possible and needed in Eulora2 but it’s not to be done in the common style of being an action, how is it going to happen? The very simple answer is that it will happen by *effect*. Meaning, specifically, that different items will have a resting effect when consumed or otherwise used (and their use or consumption might take some time, too, possibly).

Further, the above approach to rest implies, of course, that one has to have such an item to even be able to rest at all, which might, at a first encounter, seem quite the alien and possibly harsh imposition (everyone can rest whenever, all they have to do is go to sleep!!), except it’s not really – just try it and see how well it goes when you try to rest without having at the very least some reasonably safe (quiet and cosy don’t even come into it) place, enough peace of mind and enough food to keep hunger at bay. Rest is indeed basic in the sense that it’s a basic requirement for survival not in the sense that it’s somehow effortless or a given at all times.

As a side effect of the most positive kind from the above, it follows of course that one would do better to keep an eye on their levels of tiredness – at least in Eulora2, since this is what the context is here. Because if they fail to rest when needed, things aren’t likely to go that well and for sure they aren’t going to go forever just because it’s a game or something. Past a certain point of tiredness, the body would naturally shut down and thus rest indeed of sorts, only it’s not all that clear if, when or how one still wakes up from such last-resort rest that is indeed guaranteed and a ‘natural right’ of everyone ³, certainly.

In summary, in Eulora2 you’ll live either responsibly or very, very briefly indeed! Practical learning, at its best.

1. And seriously, I can still fully recall the spidery mess of a ‘protocol’ that PS originally had and what a pain it was to first work with it at all, then attempt to clean it and finally sideline it enough until one could entirely get rid of it. That experience certainly is more than enough to teach one to NOT rush into adding stuff just because it’s possible.[↩]
2. For the record, such was indeed the case with the Chat additions – while they *could* be fit into the already existing structures, such fit was a very poor one because the types of use are quite different fundamentally and have thus very different requirements.[↩]
3. Weeell, maybe just about everyone? I don’t promise that there won’t be any immortals/vampires/rest-impaired creatures in Eulora2![↩]

The latest development in Eulora2 is the emergence of time itself, quietly but definitely ticking away at the hours that became days, then months and years of the very first era that eulorans ever knew. Euloran time is already past the half mark of the first month and it follows its own rules, whether anyone is aware of them or not. So far, there are at least some signs that some people have indeed noticed something new:

back in, and noticing a new euloran clock.

It’s possibly best if one starts becoming aware of the rules of time or at least aims to figure them out sooner rather than later, because the time in Eulora2 is definite indeed: definite in the same way that Maths is definite but also, perhaps of more pressing interest for eulorans, in the same way that Death is definite. On the more reassuring side, so far at least, Death is yet to be met with in Eulora2 but that’s about as much knowledge as anyone has on the matter.

As to the durations and even structure of euloran days, months and years, these are still to be observed. What I will let slip is that Eulora2 has several moons that might be abstract or concrete but that certainly have very concrete effects on the world as a whole. To some extent, it might perhaps be not even false to say that the very ticking of time is just one of the more visible such effects – the rest yet to be noticed, observed and perhaps made some sense of. Not that there is any rush to do or even to notice any of it – at least there isn’t anything to rush one just yet. It’s a young world after all, still emerging, unhurriedly but unstoppably, too.

The whole first era of Eulora2 will cover in fact exactly the interval when the core structure of the world and of its interaction principles is brought in and settled in place, piece by piece. Those present and active in this first era will be essentially the actual ancestors and prime movers quite literally – after all, they are taking the risk on a nascent world and so they get also, by rights, a special status and the unique chance to leave their inprint on each part as it emerges. And there certainly are quite a few parts in the works, with next on the list the actual vitals and physical characteristics of player (PC) and non-player characters (NPCs) just the same.

Time, in this sense of Eulora2 emergence, is quite literally only the beginning!

Back when I started programming in Ada, I trusted the GNAT implementation of the Ada standard, mostly because I had no choice ¹. A few years and a lot of experience with said GNAT implementation later, I know for a fact that the I/O packages of GNAT such as Direct_IO and Sequential_IO are broken and merely obfuscating rather than helping in any significant way.

By the time I discovered in 2021 just how broken and uselessly complex these packages truly were, there were already several places where the game‘s client relied on them. Unpleasant discovery for sure but even so, given that there was plenty more pressing work to do and I had managed at least to restrict the use of such packages enough so that their bugs didn’t get a chance to manifest in my code at all, I left the existing code as it was and simply avoided using these packages in code written *from that point on*. After all, the client is anyway open source and thus easily changed by anyone who cares enough about it – if anything, this sort of known and well understood issue makes exactly for a low hanging fruit to pick, something that can even help newcomers to get more easily and more satisfyingly involved.

Recently though, as other parts get finally integrated and the client is getting ever closer to release, I reviewed this matter again, especially as VaMP came into the picture as well. As a result, it turns out that there still is a place where I can’t quite leave hanging this exact sort of fruit, namely when it comes to reading and writing cryptographic keys to the disk. First, this is quite clearly a sensitive operation and as such one where the fewer dependencies and the greater the clarity the better at all times. Second, this is integrated into the client indeed but it’s part of a base layer that is not really client-specific and as such I would really keep it as clear from any unwanted GNAT dependencies as I can ². So I took the time to review again all the i/o code and then to sort it out in the most straightforward way possible.

The solution turns out to be both short and sweet – all the sweeter for being in fact so obviously well supported by Ada as a language and by all the rest of the code that I have implemented so far. All it took was one short new package that takes advantage of the very thin wrappers I wrote for the read/write C library functions (over which GNAT wraps layers upon layers of obfuscation more than usefulness) and uses otherwise direct memory mapping to convert between any given type to its raw representation and back.

Such direct memory mapping for composed types relies in turn on a very tight specification of how such type should be represented in memory but this was absolutely no trouble at all for me currently since… I had already taken the time when I defined the needed types to fully specify their representation as well! So there it was, my own work done well in the past quite directly and obviously paying its dividends in the present and compounding the dividends for the future, too.

As for the relevant code, here’s the new mmap_io package, helpfully formatted already by my own VaMP ³ for publishing on the blog:

eucore/src/base_layer/src/mmap_io.adb

6_1 

	-- DC, 2023

6_2 

6_3 

with Ada.Directories; use Ada.Directories;

6_4 

with Interfaces.C; use Interfaces.C;

6_5 

with System;

6_6 

6_7 

with Raw_Types; use Raw_Types;

6_8 

with Raw_IO; use Raw_IO;

6_9 

6_10 

package body MMap_IO is

6_11 

	sz: constant size_t := T'Size / System.Storage_Unit;

6_12 

	procedure Write(tval: in T; fname: in String; append: in Boolean) is

6_13 

		local: aliased T := tval;

6_14 

		--for local'Size use T'Size; -- this would be static as required only for scalar types

6_15 

		data: aliased Octets_Buffer_Pkg.Elem_Array(0..sz-1);

6_16 

		for data'Address use local'Address;

6_17 

	begin

6_18 

		Write_Octets(fname, data, append);

6_19 

	end Write;

6_20 

6_21 

6_22 

	function Read(fname: in String) return T is

6_23 

		local: aliased T;

6_24 

		data: aliased Octets_Buffer_Pkg.Elem_Array(0..sz-1);

6_25 

		for data'Address use local'Address;

6_26 

		ptr: Octets_Buffer_Pkg.Elem_Array_Pointer;

6_27 

		fsz: size_t;

6_28 

	begin

6_29 

		if not Exists(fname) then

6_30 

			raise Failed_IO with "inexistent/inaccessible file " & fname;

6_31 

		end if;

6_32 

		fsz:= size_t(Size(fname));

6_33 

		if fsz < sz then

6_34 

			raise Failed_IO with "file " & fname & " too short (" & fsz'Image & " vs expected " & sz'Image & ")";

6_35 

		end if;

6_36 

		ptr:= new Octets_Buffer_Pkg.Elem_Array(0..sz-1);

6_37 

		Read_Octets(fname, ptr);

6_38 

		data := ptr.all;

6_39 

		Octets_Buffer_Pkg.Free(ptr);

6_40 

		return local;

6_41 

	end Read;

6_42 

6_43 

end MMap_IO;

eucore/src/base_layer/src/mmap_io.ads

7_1 

	-- DC, 2023

7_2 

	-- memory-mapped I/O for types that are explicitly and fully specified with representation clause given that they are read/written as raw octets directly memory-mapped.

7_3 

7_4 

generic

7_5 

	-- any type but the user of this package is responsible to ensure that a direct memory mapping works correctly and reliably for T both ways and at all times

7_6 

	type T is private;

7_7 

package MMap_IO is

7_8 

	-- write the value tval of type T to file fname either appending or overwriting (append=false)

7_9 

	-- this creates the file if/when needed

7_10 

	-- raises exception on error

7_11 

	procedure Write(tval: in T; fname: in String; append: in Boolean);

7_12 

	-- reads one value of type T from given file fname

7_13 

	-- raises exception on error (eg inaccessible file, wrong size, failed mapping etc)

7_14 

	function Read(fname: in String) return T;

7_15 

7_16 

	-- reads the idx-th value of type T from file fname

7_17 

	-- function Read(fname: in String; idx: in Natural) return T;

7_18 

end MMap_IO;

With the above done, the actual writing and reading of rsa keys was actually *simpler* than it was when relying on direct_io and the likes. As a result, the udpated code is shorter by about 30% – the relevant keys_io.adb implementation went from 471 lines when using Direct_IO to 152 lines when using my own MMap_IO and this doesn’t even take into account the dropped dependencies and how much additional code they brought in.

For anyone interested in using the client whether now or at any future date, the good news from the above is that the previously low hanging fruit of switching client i/o to a more straightforward non-gnat implementation is now even lower hanging, seeing how there is the MMap_IO package to rely on and/or to use as example for the case where something slightly different might serve better. To further help in this vein, note that the client still makes use of Sequential_IO mostly for the ‘torrents’ part, meaning the read/write of files obtained at run time from the server – essentially game assets of all sorts.

Possibly a slightly different package would serve better to replace that use for game assets since the MMap_IO package isn’t aimed specifically at sequential i/o – one might perhaps make a generic package receiving both the file and the type desired, reading the whole in memory and operating from there. It would certainly be a faster and more reliable solution that GNAT’s imagined ‘control’ over multiple simultaneous accesses to the same file on disk. I’ll leave this for another day though and possibly, even preferably for another person, too, so feel free to pick it up if it’s of any interest to you.

If you set to work and have something to show or to ask, I’ll be happy to hear of it as well, so use the comments below confidently, there’s nothing to lose for it for sure.

1. Perhaps it’s truly very naive to trust this way but at times it is also unavoidable – what you don’t know yet, you have to take on trust, there is no third way available. At best, you get perhaps the choice to trust a person based on what you know *of them* but at worst, as in this case, all you ‘get’ is to trust an unknown entity, pretty much, the ‘code itself’ aka all 360M of it or an ‘organization’ aka Adacore with all its changing history and multitude of people that you can’t ever know or get to know in any meaningful way anyway. So you trust, if and when you must but hopefully you then work as well towards reducing your own need for such blind trust. As you learn more, it’s worth remembering to revisit, reevaluate and correct or adjust misplaced trust, where necessary – this is exactly what this article describes and documents, my reevaluation and correction of previously assigned trust, as I got to know better what it was exactly that I ended up relying on.[↩]

2. Especially since on review, I realised that even Direct_IO and Sequential_IO packages from GNAT *still* rely on the ugly and gnarly “streams” even though they avoid the broken ‘finalization’ and although there is both no need for such reliance nor any use for it. The very code in question states clearly in places that the streams abstraction is meaningless for the task at hand but nevertheless it’s brought in and adhered to because of the need to fit a predefined form, a need of comformance and uniformity across the board basically, that’s it. For instance, directly from s-direio.adb:

   — The following is the required overriding for Stream.Read, which is
   — not used, since we do not do Stream operations on Direct_IO files.

   In other words, it’s most likely a deeper issue at core – the GNAT implementation values uniformity and ultimately fungibility above everything else as it sets purposefully to shield the user from the requirement of deeper understanding and actual grasp. It’s a perspective entirely at odds with my own and so it’s no surprise its fruits are unpalatable to me.

   [↩]

3. Compounding dividends yet again, see?[↩]

Since the question in the title comes up with some regularity, I’ll write out the answer here for everyone to see and for myself to have it at hand when next needed:

It’s most likely for your own sake that you’ll need to run some sort of Linux, especially if you never ran it before, but otherwise sure, Eulora2 can serve as a good motivator for it, too.

To elaborate on the above, note that *all* you need to run Eulora2 is an environment where you can compile C, C++ and Ada code from scratch ¹. Arguably, any and all operating systems are capable of such a basic feat as this, since it is, quite literally, what “operating” ² a computer system is supposed to be about – making it process (“compile”) written instructions (“code”) into a form (“binary”) that the machine can then follow (“run”) to do exactly what you told it to do. And if you don’t like what it does or you change your mind or you simply want to play around and tell it to do something else, all you need to do is to tweak the code and have the machine process it all happily again, as many times as you please. Hence the importance of *all* the steps in that process, not just the last one, not just “run this”, but absolutely all of them, from first to last.

Does your current operating system allow you to do all and any of the steps above, routinely and easily? Then you don’t need anything else and you are set to go, nobody will mind you running Eulora2 there or anywhere else and you’ll find running it just as easy as running anything else. Moreover, I’d be very glad to hear from you how it goes, what you are running it on and what changes you are making to it, too – after all, it’s meant to be *your* client to play Eulora2, not in any way someone else’s.

If your current operating system doesn’t really make it easy to even know all that much, let alone perform and control one or several of these basic steps, then you’ll probably need to try out something more useful. What exactly might that be is entirely your choice but it is indeed more likely to be some version of a unix-based system. And if you truly want to have an easier time compiling and running everything as well as perhaps changing it to better fit your needs at some point, I suggest you pick an older version and never ‘upgrade’ it with all the bells and whistles that might or might not do anything useful for you but will likely break the toolchain and leave you stranded while also eroding to various degrees those hardly ever mentioned attributes of access, control and consistent return on your own learning efforts ³.

There is no reason at all and no intention even for there to ever be just *one* client for Eulora2 or otherwise put “the” client. Currently there is indeed only one, for the very simple reason that I made it and nobody else made theirs. As a result, my client is and will remain the reference client but it’s still not meant to remain the only one. Quite the opposite, I’m open to any number of developers and/or maintainers of clients coming forward to make and promote their own versions, getting paid for it, too. The way I intend to set it all up, developers and maintainers of clients will get paid for their work based on how many users choose to run each developer or maintainer’s client and for how long, as all this information is directly visible from the serverside. While this is still some way into the future currently, count this as the heads up on it – you won’t be able to say later you didn’t know it or didn’t have time to get ahead of others with it.

As for the reference client that I made and maintain, I would recommend currently that you run it on one of Debian 8 (Jessie), Ubuntu 10, CentOS 6 or Gentoo ⁴. This is not because the client “requires” these or it can’t otherwise run in a different environment but simply because it is already known to run smoothly directly out of the box with as little effort as downloading and starting *one* single script that does it all. Of the four versions mentioned, I don’t have any particular favorites, as they all have their own specific pluses and minuses, but I’d say they are all fairly easy to try out, too. Depending on your machine though, you might want to pick one that plays more nicely along something else and possibly out of the list above, it’s Ubuntu that is more likely to do so.

As mentioned previously, currently the entry point for Eulora2 remains through interaction with those involved, on this blog or on others. And there is help available as well, just read around, follow the links and you’ll find both the people involved and ways to interact – it’s the meta-game, if you wish, happening in plain sight, even if it seems invisible to some. If it’s not invisible to you though, just make and pick your quest(s), explore it all leisurely and when you’re ready, start commenting and interacting with people, too, it helps. In short, ask and you’ll get an answer. Better yet though, ask a good question and you might just get more of an answer than you even hoped for!

1. More precisely, this means specifically that gcc 4.9 runs fine and the whole toolchain is not broken, either.[↩]
2. For all the quotes, operating computers is not even all that far removed from other types of operating that may be more familiar, since they all come, of course, from the more direct physical operation aka use of tools of all sorts – after all, computers are merely more complex tools, basically tractors for plowing the information bitfields, nothing else. For this same reason, operating computers has in common with operating any other tool further requirements that don’t get mentioned explicitly nowadays all that often, most notably the ones related to requiring some amount of study that actually pays off and offering full access and control over all its parts.[↩]
3. Convenience that caters to your lack of knowledge *always* costs more than is directly made visible and its hidden costs are quite often the ones that you’ll regret paying in the long term. Yes, it gives you a boost in the short term – quite often “now”- and it robs you of ~everything in the long term. At the very least it gives dependency, too. Just like any other drug of choice, you know?[↩]
4. For Gentoo, there is no versioning possible even, since it’s always and by design a sort of mix-and-match, so you’ll need to pick your versions of everything in there. There is however a neatly written summary of a successful install of Eulora on Gentoo, so you might find some help with it, if you ask there.[↩]

Promptly and on demand, that’s how things are moving in Eulora2. To wit, one day in Eulora2, the discussion turned suddenly and quite literally to the skies above and whether the speakers knew it or not, the demand was thus made and fully heard:

Diana Coman: what, does that sky look flat to you?!
Vivian Sporepress: haha
Vivian Sporepress: Diana Coman: certainly not flat, possibly not even any sort of crystalline sphere; it seems to have some axial tendencies, perhaps an ellipsoid; and I have no hypotheses yet for the nature or causes of its cloud patterns though they appear unchanging to the unaided eye
Vivian Sporepress Diana Coman: ah, the ‘axial tendency’ looks more like a bilateral symmetry across a vertical plane, suggesting either a peculiar optical phenomenon or a decidedly nonrandom process behind the cloud formation
Diana Coman Vivian Sporepress – well, the clouds have to fit the sky’s …geometry and that one is indeed quite regular
Vivian Sporepress Diana Coman: the geometry of the sky might not be possible to infer with the present euloran technology level, for instance a sufficiently large cube shaded just right can be indistinguishable from the inside from a sphere!

Finding itself under such interesting scrutiny, the sky clearly took notice. Only a couple of weeks later, among many other improvements and developments, there was also this confirmed change:

To be fair, there are quite a few other movements of things and of people happening at the moment in Eulora2, even if very little of it made it on to the blog. As always though, movement is the hardest to capture in pictures so the best way is to be there to see it happening as it happens, perhaps finding ways to be an active part of it as well. Second to being there, read along and comment underneath, perhaps, as I’ll aim to let it spill on to the blog in bits and parts at least from time to time, why not. There’s certainly plenty interesting and ready to move in there, from the multiple moons orbiting around Eulora2 and their impact on the grounds below, to the more prosaic issue of reconciling local movement with server’s view of the same.

The best part about it all is that the solutions, when found, turn out to be so easy to implement currently, given the supporting environment and a very much cleaned client code that it really doesn’t take long for something new to become visible in game once articulated clearly and meaningfully enough. This might change in the future perhaps but for now it is exactly so and turning gradually into a lot of fun, too.

How many other games do you know of, where truly all it takes to make a visible change is to speak up convincingly enough, without having to argue for it, without even having to frame it as a request? It’s enough to make sense and to pick at an interesting problem, that’s all.

This article builds up on the existing, well-iterated specification of the communication protocol, aiming to detail the new developments and to provide a place for current and relevant discussion going forwards. For the core structure of the protocol though, the linked article remains the reference and although the discussion of that part moves out of necessity here as well, I see no need at the moment to copy its content too. If your question concerns the parts that are not directly described here, simply link precisely the part you are talking about, as needed.

The first further addition to the communication protocol enables chat in Eulora2 and is currently implemented and fully working. Worth making it explicit and right from the start that the chat in Eulora2 is by default encrypted ¹ whenever in transit and effectively authenticated as well, as it fully benefits from the whole existing infrastructure supporting the game’s world ². The WoT layer that is currently in the works will further provide the means to seamlessly keep out any automated noise, spam and the like while simultaneously enabling all users to effectively retain both control and accumulating benefits for everything that they put into participating. To the extent possible, I fully intend to further align the incentives as much as I can to support meaningful conversations that are worth re-reading, too – without any unnecessary restrictions and with fully owned responsibility as well as benefits for each participant, too. It is to my eye the only way in which having a chat capability is even worth it in the first place.

To add the chat capability to Eulora2, there are two new message types added in section 4 (Serpent packets), to request and receive chat lines, respectively, as well as one new action type added to section 7 (Character actions) describing the “Say” action. Using the same specification format, the newly added sections are the following:

4.9 Line Request

• uint8 (type ID, =12), followed by
• uint8 (count of lines), followed by
• n times uint32 (id of chatroom), uint64 (lowest line number requested), uint64 (highest line number requested), followed by
• padding to Serpent-message length

4.10 Line Info

• uint8 (type ID, =13), followed by
• text (line content), followed by
• uint32 (chatroom ID), followed by
• uint32 (speaker ID), followed by
• uint64 (line number), followed by
• uint64 (keccak hash of the line’s content and number), followed by
• padding to Serpent-message length

Updated 30 Oct 2024: ratings are now fully integrated and used in-game, being fully handled via the relevant Rating node in the data hierarchy. Consequently there is no need anymore to have dedicated message types for discovery and retrieval, so the “rating request” and “rating info” message types below are removed entirely. See in any case the protocol specification for the full protocol.

4.11 Rating Request

• uint8 (type ID, =14), followed by
• uint8 (count of requests), followed by
• n times uint32 (source ID), uint32 (target ID), followed by
• uint16 (message count), followed by
• padding to Serpent-message length

4.12 Rating Info

• uint8 (type ID, =15), followed by
• text(rating comment ³), followed by
• uint32 (source ID), followed by
• uint32 (target ID), followed by
• uint8 (rating value ⁴ ), followed by
• uint16 (message count), followed by
• padding to Serpent-message length

7.9 Say ⁵:

• uint8 (type ID, =9), followed by
• text (line content), followed by
• uint32 (room ID), followed by
• uint64 (keccak hash of the line’s content)

7.10 Rate

• uint8 (type ID, =10), followed by
• text (rating comment), followed by
• uint32 (target ID)
• uint8 (rating value)

A line of chat in Eulora2 has a maximum length of 1445 characters. This is fixed and unchangeable as it is the maximum that fits in a 4.10 message (which has the least space for the line’s content out of all the messages relevant to chat lines and therefore effectively decides the *maximum* allowed line length). This limit is unlikely to be perceived all that often in practice but even if it is, note that it’s anyway more generous than most other existing chat protocols, irc included. Any client implementation is however entirely free to take its own approach as to how it handles this – whether it splits some user input into several lines if needed or whether it restricts input to this maximum of 1445 characters to start with, to give some concrete examples.

Beyond the above specific characteristics, for all intents and purposes chat rooms in Eulora2 are otherwise like any other game entities and thus the above additions to the communication protocol itself are both minimal and all that’s truly needed for a practical, working implementation. The speakers themselves are also characters (whether player controlled or not) in the game, hence fully described otherwise through the existing messages.

Messages 4.10 (Rating Request), 4.11 (Rating Info) and 7.10 (Rate) provide, respectively, a way to request and obtain from the server any desired ratings between two known entities in the WoT and to register/update one’s own rating of another. The comment part of a rating is meant to provide some context for the chosen value but as the value itself, the actual meaning and use is up to each individual and likely to be different from one person to another.

Worth noting that messages 4.10 and 4.11 are provided mainly as a basic and potentially last resort means of requesting and obtaining specific ratings. They are however quite unlikely to be the best choice for exploring or obtaining the full set of existing ratings, especially across a relatively large WoT, since each rating will require its own message and the traffic thus generated is to be paid for by the requesting client. A much cheaper and more convenient way to explore and update a local copy of the WoT will be provided in-game through an abstract object providing the full representation of the WoT in a single file and taking thus advantage of all the existing infrastructure, without requiring any new types of messages or other changes to the existing protocol.

Feel free to ask any relevant questions in the comments section below.

1. Serpent keys[↩]
2. Obviously, any of the users can otherwise make public the chat logs they have when and as they see fit but if you rely on something outside Eulora2 for that, it’s strictly between you and that provider, nothing to do anymore with either Eulora2 or the support and protection it provides.[↩]
3. Limited to at most 1445 characters, matching the maximum length of a chat line.[↩]
4. Valid rating values are integers between -10 and 10. For a given rating value here GR, the corresponding actual rating value RV is therefore calculated as: Min(GR, 20) – 10.[↩]
5. Note that this is an action and as such just a part of a 4.5 message, Client Action, where the overall message structure is specified.[↩]

~Referencing, of course, the previous version of EuCore and even that very beginning of creating a useful structure to it all.~

As I’ve opened up a test environment for helpful people to give a go to the latest client, they are likely to benefit as well from an updated and more comprehensive overview of the full client code, as an initial map of sorts, at the very least. So this is what I’m aiming for with this article – to provide an up to date map of the current client and a place for questions and answers on it all, in the comments.

While the previous article focuses exclusively on the EuCore part, meanwhile I really had to deal as well with the GUI part, as there wasn’t anybody else doing anything about it otherwise. So the GUI part got significantly trimmed, mostly overhauled and otherwise changed from inside out to provide the sort of flexible and reasonably structured functionality that I need as opposed to the assumptions-ladden and spaghetti-shaped sort of “service” that was previously available. As a result, out of all those half a million lines of code and countless files, there are now… 40308 lines of c/cpp code in total remaining. More than half of this (27584 lines according to a wc run currently) is in fact the only remaining sprawling mess that got now at least insulated in its own directory, namely the PAWS horror. In more practical terms though, there are 11 classes that are almost entirely re-written and make up the bulk working part of the GUI, 2 classes (one entirely new and one adapted) for providing the whole set of views, windows and overlays that Eulora2 needs and otherwise a bunch of the remaining “utility” classes that are quite often simply dragged in by various spidery includes of the PAWS variety. Here’s the full remaining tree of the GUI (ie the C/CPP part of the client):

~/client20/src/
├── client
│   ├── err.cpp
│   ├── err.h
│   ├── euaction.cpp
│   ├── euaction.h
│   ├── euloader.cpp
│   ├── euloader.h
│   ├── eumovement.cpp
│   ├── eumovement.h
│   ├── globals.h
│   ├── meshattach.cpp
│   ├── meshattach.h
│   ├── paws
│   │   ├── gameviews.cpp
│   │   ├── gameviews.h
│   │   ├── pawsborder.cpp
│   │   ├── pawsborder.h
│   │   ├── pawsbutton.cpp
│   │   ├── pawsbutton.h
│   │   ├── pawscheckbox.cpp
│   │   ├── pawscheckbox.h
│   │   ├── pawscombo.cpp
│   │   ├── pawscombo.h
│   │   ├── pawscrollbar.cpp
│   │   ├── pawscrollbar.h
│   │   ├── pawsframedrawable.cpp
│   │   ├── pawsframedrawable.h
│   │   ├── pawsimagedrawable.cpp
│   │   ├── pawsimagedrawable.h
│   │   ├── pawslistbox.cpp
│   │   ├── pawslistbox.h
│   │   ├── pawsmainwidget.cpp
│   │   ├── pawsmainwidget.h
│   │   ├── pawsmanager.cpp
│   │   ├── pawsmanager.h
│   │   ├── pawsmenu.cpp
│   │   ├── pawsmenu.h
│   │   ├── pawsminimalslot.cpp
│   │   ├── pawsminimalslot.h
│   │   ├── pawsmouse.cpp
│   │   ├── pawsmouse.h
│   │   ├── pawsnumberpromptwindow.cpp
│   │   ├── pawsnumberpromptwindow.h
│   │   ├── pawsokbox.cpp
│   │   ├── pawsokbox.h
│   │   ├── pawsprefmanager.cpp
│   │   ├── pawsprefmanager.h
│   │   ├── pawsprogressbar.cpp
│   │   ├── pawsprogressbar.h
│   │   ├── pawspromptwindow.cpp
│   │   ├── pawspromptwindow.h
│   │   ├── pawsradio.cpp
│   │   ├── pawsradio.h
│   │   ├── pawsselector.cpp
│   │   ├── pawsselector.h
│   │   ├── pawsstringpromptwindow.cpp
│   │   ├── pawsstringpromptwindow.h
│   │   ├── pawstabwindow.cpp
│   │   ├── pawstabwindow.h
│   │   ├── pawstextbox.cpp
│   │   ├── pawstextbox.h
│   │   ├── pawstexturemanager.cpp
│   │   ├── pawstexturemanager.h
│   │   ├── pawstextwrap.cpp
│   │   ├── pawstitle.cpp
│   │   ├── pawstitle.h
│   │   ├── pawstree.cpp
│   │   ├── pawstree.h
│   │   ├── pawswidget.cpp
│   │   └── pawswidget.h
│   ├── pscamera.cpp
│   ├── pscamera.h
│   ├── pscelclient.cpp
│   ├── pscelclient.h
│   ├── pscharcontrol.cpp
│   ├── pscharcontrol.h
│   ├── psengine.cpp
│   ├── psengine.h
│   ├── psmainwidget.cpp
│   ├── psmainwidget.h
│   ├── smgdata.cpp
│   └── smgdata.h
├── common
│   ├── placeholder
│   └── util
│   ├── command.h
│   ├── consoleout.cpp
│   ├── consoleout.h
│   ├── fileutil.cpp
│   ├── fileutil.h
│   ├── heap.h
│   ├── log.cpp
│   ├── log.h
│   ├── psconst.h
│   ├── pserror.cpp
│   ├── pserror.h
│   ├── psstring.cpp
│   ├── psstring.h
│   ├── psutil.cpp
│   ├── psutil.h
│   ├── psxmlparser.cpp
│   ├── psxmlparser.h
│   ├── README
│   ├── singleton.h
│   ├── stringarray.h
│   ├── strutil.cpp
│   └── strutil.h
└── placeholder

4 directories, 104 files

The main method is in psengine.cpp ¹ and that starts and initializes everything needed, then enters the loop provided by the graphics engine and at the end cleans up whatever is left. The first initialised is the graphics engine, CrystalSpace and that comes with its own “object registry”, configuration and initialisation dances that have been however trimmed to a minimum and corralled into a single method (and a single, shared configuration file, too!) that is hopefully reasonably easy to read. After that, the Ada part of the client, namely the EuCore library (see below for more on its role) is given the green light to start and provided all goes fine, the next steps are to simply initialise the GUI logic that is in psengine itself and then to enter the CrystalSpace loop that runs for as long as the game is running.

The GUI part has the following main components:

1. psengine – the central part that provides access to everything else and otherwise registers with the graphics engine to receive regular callbacks on events of interest. Such events of interest include each frame drawing and all user input. The callbacks on each frame drawing act effectively as a sort of heartbeat of the client as a whole and thus coordinating the different parts of the GUI. User input is generally received and then passed on to the relevant action handler (see euAction below) or to further potential handlers where relevant.
2. pscelclient – a class ² that is meant to keep track of and otherwise provide all logic and management related to game entities. This is one level above the graphics engine and acting as a sort of bridge between the data as obtained from EuCore and its translation into the graphical format that the user gets to see. So pscelclient uses what the graphics engine provides, asking it to create, move, change and delete any graphics in the game’s world to match the information it gets from the data available from EuCore.
3. pawsmanager – meant to keep track of and otherwise provide all logic and management related to “widgets” aka the various windows, views and overlays that make up the game’s interface rather than its world as such. This relies especially on the various views and overlays that are specific to Eulora2 and are defined in gameviews.h/cpp. This has most of the original mess intact but mostly sidelined and otherwise the useful bits and parts wrote in. The trouble cleaning this up is that ~all the paws calls back home to this and quite often with hardcoded include and calls too so it really wasn’t worth yet my time to clear it all up on top of everything else.
4. pscamera – a class that provides and maintains the available ways to view the game’s world as it is otherwise populated by pscelclient.
5. controlsManager – a class holding all the mappings of keys and mouse buttons so that they work as set in the configuration file and otherwise allow the user to provide input and interact with the game.
6. euaction – a hierarchy of classes that provide the required input, processing and output capabilities to enable the user to perform all relevant actions in the game. The central psengine initialises a full set of objects from this hierarchy and then simply passes on to the relevant one the request for execution with any parameters as provided by the user at that point.
7. euloader – a static utility class providing ways to create on the fly any graphics entity of the required type and using the assets indicated simply by a path. This includes creation of very simple elements such as textures but also quite complex ones such as animated characters that require a rather long list of calls to fully make when starting with just a bunch of files. While the creation of such graphics is quite tedious, at least it’s enough to have it packed this way once and then it serves endlessly, effectively providing useful defaults for all the “parameters” that aren’t actually needed or even known at start. The main user of this class is pscelclient.
8. smgdata – a static utility class that imports all the Ada methods made available by EuCore and wraps them up for transparent use by CPP code from anywhere else in the GUI. Basically all requests to EuCore are made through this class.

While the above GUI part focuses exclusively on the interaction with the user, it relies on the separate EuCore library to communicate with the server and to exchange and obtain all required data. The EuCore library handles everything in this respect, including creating a RSA-based identity and account, initiating and maintaining fully encrypted communication with the server, obtaining and updating all data related to the game’s world as available at different times to the player, retrieving from the server through the same encrypted communication any missing files and managing existing local files including their use as temporary defaults when needed and where appropriate. The full EuCore weighs in at about 27000 lines including tests and the mpi part that has about 8000 lines of code by itself. Here’s an updated full picture of all Ada package dependencies:



Compared to the previous version, there is a striking change at the top where EuInterface is now the “entry” package instead of EuCore. This reflects actually better the meaning of the two packages and by now the whole code matured enough so that the refactoring was quite obvious, too. Basically EuCore is the starting package for the whole library, while EuInterface literally provides an entry point for any C code that wants to use the library and this includes now the previously missing C-compatible wrappers for EuCore’s Start/Stop methods themselves. Thus any Ada code that uses this library can entirely discard EuInterface and just start with EuCore, while any C code using this library goes entirely through EuInterface and doesn’t need to see or know of anything else.

Further comparing with the previous version of EuCore, there are also a few new packages, most notably Mgm_Cache that comes now between Heartbeat and EuCache, as well as C_Buffer at the very bottom. The C_Buffer package is literally for interfacing with C code so that Ada code can take advantage of faster writing to files mostly. The Mgm_Cache package does what the name suggests, namely provides the higher level logic to oversee the data cache that is EuCache itself. In fact, the Heartbeat package serves as on overall coordination mechanism across the whole of EuCore and as such it effectively provides regular calls as required to various parts such as the communication link (Comm_Link), the data cache (EuCache) and the assets downloader (Torrents). As the communication link is both the most critical and the most demanding component of these, the Heartbeat really grew first of all as a manager of Comm_Link and only once that was fully working the rest could be developed as well. At the moment, the management of EuCache grew big enough to make sense to extract it in a separate package of its own but otherwise I don’t see a lot of benefit in doing the same for Torrents for instance.

Note that all dependencies in the graph above remain hierarchical as they should always be. The automated layout of the graph tends to make it look as if there was some nonsense going on perhaps with that seemingly horizontal line going from OAEP to Keccak, but that’s just an artefact of positioning. In reality, Keccak is on a lower level than OAEP, being simply a hashing utility, although one that is most useful indeed. At any rate, OAEP uses Keccak and there is no dependency going the wrong way, as Keccak itself uses only raw_io and raw_types.

Hopefully this gives enough of an overview to have some idea as to what is where and how the whole fits together. If you have though further questions on it, feel free to ask in the comments below and I’ll gladly answer.

Finally, if you made it all the way here and would still rather see a less schematic picture as well, here’s a fresh one of the client in action at a meeting of like-minded creatures of Eulora2, surprised as they were by all the screenshooting commotion:



1. Despite the name of the file and of the class, there is very little indeed left in there from the original code. I suppose I could have changed it all to euengine or something more appropriate but given how it’s one of those included from just about everywhere due to the original “design”, I didn’t really consider all the resulting noise worth the trouble.[↩]
2. This is also mostly re-written with only very little indeed remaining of the original “gem” pain. It could still do with a haircut at least, I’d say.[↩]

The prize for having a clean environment (not merely code, nor even limited to code) is refinement of precisely the most intellectually pleasing sort: getting to uncover and solve interesting as opposed to tedious problems. In other words and as far as I can tell from direct experience, even coding really doesn’t have to be at all “like plumbing” if the sewers are properly cleaned and it doesn’t have to be sterile either if the willfully blind focus on idealia is discarded. In a reasonably clean – not perfect, nor even “best possible” – codebase ¹, there is both space for experimentation (aka introducing new things) and support for refining existing solutions. Arguably, there is even space and support for literate collaboration but this does tend to grow more slowly being as it is a fully organic culture in its own right. There is time, there always is… until there isn’t.

The most recent concrete example of what made for me quite an interesting problem brings together knowledge representation, data caching, remote synchronisation over a lossy channel and even matters of time representation across different contexts. Then again, arguably one’s problems can only get as interesting as one is able to make them to start with so perhaps it’s all tailor-made indeed. Either way, I’m neither boasting nor complaining, merely describing so here’s an attempt to sketch out the problem in its context:

1. The game‘s client has to obtain, maintain and represent (including graphically but not only in such form) as accurately as possible the knowledge of the game’s world as accessible to the player and as it continuously changes as a result of the player’s own actions and as a result of others’ actions or even simply the passage of time. In all cases, the actual changes are not known upfront nor automatically obtained – while the server decides at all times what and how is in the game’s world, it’s entirely up to the client to *ask* for whatever it wants, whenever it considers it needs it. Quite on purpose, there is no information pushed from the server to the client at any time and although all questions receive an answer, there is no guarantee that all answers make it back to the client in any given timeframe or even at all. Hence, the problem here is to find an effective way to find out as quickly as possible all that is new but also, arguably even more difficult, what has become obsolete! And possibly the most difficult part here is to even define what “quickly” means since there are at least several definitions of time itself in this context: the user’s own time, the time in the game, the time as kept by the client, the time as kept by the server. What does “quickly enough” mean and when is something “too old” or even obsolete? What is the meaning of “1 second” in the game-world, what is its meaning on the network and how does one reconcile these two?
2. The communication protocol defines the rules for a hierarchical structure that provides effectively the support for representing all the knowledge mentioned above but this structure is an ever-evolving one and with no specific limits to either depth or breadth at any given time, beyond a basic “it’s always finite” guarantee. Moreover, any request comes with a cost and that’s not even limited to time – arguably the resulting traffic is costlier, both as it *is* going to be included in the bill the user pays in one form or another (most likely directly wired into game mechanics) and as it can easily have undesired consequences clogging the client’s communication pipe for instance or even… triggering undesired changes because yes, in some cases, merely asking for something means that something will change ². Hence, the direct, brute-force approach of “explore the whole knowledge tree at regular intervals” is not much of a solution to anything or not a long-term one, at any rate. And if one doesn’t explore the whole of it then how does one figure out the holes in one’s knowledge, those things one doesn’t even know yet one doesn’t know?
3. The desired graphical user interaction (GUI) clientside adds some potential troubles of its own since it’s quite difficult to provide meaningful immediate feedback to an action when its actual results require in the best case a confirmation from the server (when guessed correctly) and in the worst case a search for and figuring out of how the world changed (when the guess is merely incomplete perhaps, although it can even turn out to be entirely wrong). Sure, one can *hide* this and pretend that everything is known upfront, arguably making it thus more convenient or something of the sort but pretense aside, the reality of it is simply that the GUI is at best twice-removed from reality and as such always kind of struggling behind: the reality it can directly access is at best a local cache and that cache is in turn at best a fragment (perhaps already obsolete in places) of the remote world that is at all times on the server and ever evolving even there ³. So how can the GUI be as truthful and responsive as possible when providing the user with a representation of the game’s world, without limiting them either? Because the whole point of a GUI is to enable, after all and moreover, the game itself is quite on purpose as open as possible – the scope of what one can do depends on what they have already done, not on some predefined and fixed laundry list that some uppity GUI takes upon itself to enforce on behalf of the server.

The above is the overall shape of the problem and as complex or “that’s not how these things are done” as it may sound, the core of it is already solved and working in practice. What still remains to be done, is really the refinement of this core of a solution. How interesting and rewarding such refinement turns out to be or not depends mostly on the person asking and working on it, as always.

As for myself, I certainly have even more interesting problems to solve on the serverside so I don’t particularly intend to pursue more than absolutely needed anything clientside. The client is open source after all and it will only ever be as good or fit for your own purposes as you make it to be. Does this sound like a downside or an upside to you?

1. It’s always improving though and the cleanliness really has to do with how supportive of improvements it is, not really with some perfection or best anything. First of all, lots of requirements, constraints and even actual extent of the problems to solve got clearer only with practice and the corresponding code reflects this. Second, I learnt a lot in the process too and this also shows – yes, the 2nd time one does something it will probably be better done than their 1st and the 100th time even better than that. Cleanliness though doesn’t mean nor require skipping the first step nor the first 99 – that’s just a way to get stuck and it’s still silly even if very common – but simply creating that path that enables all the steps that may still come. So no, there wasn’t nor is there some perfect anything, code included, to be “fully solved” or achieved in some entirely ideal form and then either worshipped or blown to pieces, depending on inclination. There’s only sustainable growth, gathering up momentum and opening up new possibilities as it evolves, that’s the best possible meaning of clean – it enables the growth of new animals and plants, not that of new fungi.[↩]
2. Does it ring a bell, at all?[↩]
3. Are you *sure* that this is all that different from the actual “reality” of what you see with your own eyes even when no computers nor games are involved as such?[↩]

Like everything else in Eulora 1, the tips greeting newly connected players brought new life to an old form, generally turning common and stale anything on its head quite irreverently and with great effect. Since some of these tips still get quoted at times, I’m publishing my list of them here for vicarious enjoyment and, possibly, for easy reference as well:

1. Welcome to Eulora! May your stay be preposterously prosperous.
2. Do not think of yourself as an ugly human. Think of yourself as a beautiful monkey.
3. The early bird gets the worm, but the second mouse gets the cheese.
4. Better late than pregnant.
5. You should probably do the laundry…
6. Silence is golden, but duct tape is usually silver.
7. Time flies like an arrow, but rotten fruit flies fly like flies.
8. Everyone has a photographic memory. Some just don’t have any film for it.
9. Mr. Spirover maintains a positive attitude, even though he knows it won’t work.
10. If the shoe fits, get another one just like it.
11. You can’t have everything – where would you put it?
12. All things nobody else wants come to those that wait.
13. A fine is a tax for doing wrong. A tax is a fine for doing well.
14. In Eulora, it does not matter who is right, but who is left.
15. The best space shuttle is but a large pile of chairs for the head.
16. Happiness is having a large, loving, caring, close-knit family in another city.
17. Fear not your own death – just try to not be there when it happens.
18. Learning is not compulsory. Neither is survival.
19. There are no rules here – we’re trying to accomplish something.
20. Oh, the Vinu will love you. There is no sincerer love than the love of food.
21. Anything that is too stupid to be spoken is sung.
22. The first rule of tinkering is to save all the parts.
23. Some are wise; others are otherwise.
24. The bonds of matrimony are like any other bonds – they mature slowly.
25. When a NPC marries, it exchanges the attentions of many players for the neglect of one.
26. The most you can hope for is ending up with the right regrets.
27. Everyone started out with nothing. Plenty still have most of it.
28. An unbreakable toy is still useful for breaking other toys…
29. Does the name Pavlov ring a bell?
30. The best way to lose weight is to eat naught but inspirational posters.
31. Old eggs are generally rotten; old hopes are generally delusions.
32. To find out just how windy it is on the hill move there to escape the damp in the valley.
33. If you fail to spend enough time laughing at stupid children, you will spend all your time going to war with stupid men.
34. Your worst enemy has always slept in the same bed as you.
35. The minnow aspiring to be a whale was stuck in a puddle that did not aspire to be an ocean.
36. The best things in life are actually quite expensive.
37. All tattoos are temporary.
38. The secret of teamwork is that none of us is quite as dumb as all of us.
39. Hard work pays off in the future. Laziness pays off now.
40. The power of accurate observation is commonly called cynicism by those who have not got it.
41. By the time you swear you’re his, shivering and sighing; by the time he vows his passion is infinite, undying… better make a note of this: one of you is lying.
42. A cynic is that unrepentant blackguard whose faulty vision sees things as they are not as they ought to be.
43. The world keeps ending, but noobs keep showing up as if the fun just started.
44. The large print giveth and the small print taketh away.
45. The worst misfortune that may befal your enemy is a large following of true believers.
46. Poverty is the sort of suspect virtue one can very readly teach oneself.
47. Rejection of praise is sometimes humility and some other times holding out for better praise.
48. It is comparatively easier to make heroic gestures than to abide by the results.
49. When logic leads to any humiliating conclusion, the human reaction is to discredit logic.
50. The strongest indication that you lost your way is to find a large majority in agreement with you.
51. Out of the mouths of babes come bubbles.
52. The best possible world is the one you can’t live in.
53. True democratic feeling is reserved for drunken men.
54. If you can make a woman laugh, you can make her do anything.
55. What will retrospectively be called will always finds a way.
56. Most people love because they tried hating and found it requires too much effort.
57. The very essence of romance is uncertainty.
58. Man may have discovered fire, but women discovered how to play with it.
59. Thread carefully when the nude offer to sell you shirts.
60. Love at first sight is common; love at last sight is comparatively rare.
61. Love cautiously, enjoy vicariously, dream reasonably, tread lightly and die quietly.
62. The trouble with being poor is that it doesn’t allow getting into any kind of trouble worth the mention.
63. Gravitation is not responsible for your great tumble – after all it tried to prevent your climb every step of the way.

It has meaning. As it brings to life a fully working and fully interconnected environment where *nothing* is created out of thin air or without clear, mathematically defined causes and consequences, each and every action done (or avoided) in the game *means* something – in other words, it… matters! Yes, it matters and means something first of all within the game itself but that’s no hard limit in any way and moreover, even considered solely within the game, it still means way more than it might seem at a first glance. This is not by accident and what follows might give some solid pointers as to why and how, so read on.

It is practically infinite in more than one dimension. To give just a few examples ¹:

• Your exploration generates (on demand!) new maps and there is more scope for such distinct maps than you can cover in a lifetime. So step in and see the world expand as you explore further and further. As you explore, note also the resistance of the medium, not constant but related to you and to your actions, ever present and ever giving meaning to the choice of the next step, because your exploration in this game *is* creation and not just a meaningless stroll through the endlessness. How far will you go and what will you create?
• Your crafting, exploring, fighting, child rearing and various other game activities generate new items (and this means all sorts, computer controlled characters included) and the scope for these, too, is wider than you can cover in a lifetime. While the potential items are practically infinite, it’s your actions that pick from this infinite potential some specific item and then bring it into being. If you don’t act then no item will exist at all and the potential will remain just that, a waiting but never manifested possibility. So what are you going to make and who will you meet?
• Gains are potentially infinite, too, as they aren’t capped at any random value, nor forced into any sort of artificially diminishing returns model. The only limit, here as everywhere else, is the player’s own capability and otherwise the whole of what there is in the game, nothing else. How much can you gain and what is your gain even going to look like?
• The intricate connections and complex underlying laws of the game’s environment are statistical in nature and never directly exposed but ever present through their effects – or more precisely said, through what you are able to observe of them and, moreover, how much you are able to make of what you observe at any given time. How much of it all will you observe, how well will you understand it and what are you going to do with that knowledge gained?
• The game promotes creativity ² and it rewards applied, practical learning with full freedom of choice as to direction or means. The game’s environment reliably provides feedback and the unbending structure to test your hypotheses (and your mettle) against, while allowing otherwise any and all approaches you can come up with. Who will you turn out to be?
• The game provides opportunities for meaningful action well beyond playing. The only limit with respect to opportunities available is defined by the players themselves, as they can literally interact with the game in any way and to any extent they want. From building up their own town and governing it the way they see fit or setting up shop somewhere in game to making their own client or their own graphics to be used – whether for free or for pay – by other players, the game’s environment simply provides just the right amount of structure to support it all but no restrictions as to what “should be done” or not otherwise. What do you want to do and how well will you do it?

All the above combined mean that it is your actions, observations, data gathering and even client development or meta-game that increase both the richness of the game’s world and your practical, applied knowledge that is not even limited to playing the game itself. In this sense, too, the scope is wider than you can cover in a lifetime. In this game, like in no other but exactly as in life, you can never really step in the same river twice. No “walkthroughs” and no “cheat sheets” can ever “solve it all” (aka spoil it all) for the next player who is neither the exact same as the previous player nor in the exact same place or time as them, nor even interacting with the game’s world in quite the exact same way. Your character in game ages and dies, too, and their children may continue so that your own experience of the game is ever changing and your play literally writes the history, whether through small or big deeds, still writing nevertheless. What part of the game’s history will you write?

“And will this game happen?”

“By itself, you mean?”

1. For more on it all, see for starters the S.MG category on trilema.com, the Eulora category on this blog, quite a few of the articles in the trolloludens category on trilema.com as well as various other articles that are quickly found if you follow the links.[↩]
2. Creativity is, at core, a successful cross-pollination (or weaving if you prefer) of sorts, the result of making viable new links between matters that had seemed until then separate or at the very least evolving independently, each in its own direction. As such, discipline, structure and a solid knowledge of many different fields are the support and ally of creativity, not at all a barrier or the enemy as they might seem at a superficial look.[↩]

The initial work only expanded in scope in the intervening years to the point where it ended up covering pretty much everything, from end to end encryption, rsa-based authentication and a new protocol for client-server communication, to flexible file transfer torrents-style, a full graphics pipeline operating on demand, user interaction (both GUI and CLI) and various intricate mechanics that I won’t even start detailing here if I am to keep this introductory paragraph from taking over the article whole. Nevertheless, as all this extensive ground has been covered in at least one fully working iteration and the swamps have been all but drained, the core structure is now reliably in place and so the time has come to document publicly at least the very Core of what will be the most widely available part of it, namely Eulora’s Client.

Looking at the dates, I notice only now that it’s almost exactly two years since I published the first version, higher level map of client’s core. It’s been two very productive years though, so that almost all of the User Interface part got trimmed to a more reasonable size, eviscerated and squeezed like the paste that it was into usefulness, while the Core part grew into providing full services on top of the barebones, raw elements that were originally listed.

The client’s Core is still a library but it has a clear single entry point in the EuCore package that manages all the various parts and provides the means for a C/CPP main to start and stop the whole in a controlled manner, without being otherwise aware in the least of any of the Core’s own internals. Basically EuCore serves as a control interface, ensuring that even a C/CPP main can take advantage of Ada’s neat mechanisms ¹ for clean and tidy multi-threaded execution. On receiving the signal to start everything, this control interface initializes (aka asks them in turn to load whatever they need from the shared configuration file, already) and starts all the components that it manages. On receiving the signal to stop, EuCore passes it on to each of the components it manages and otherwise doesn’t have any qualms about killing off (via abort) if and when required any tasks that can’t or won’t stop in a less abrupt manner.

The most important components managed by EuCore would be, from the lowest level up:

1. The sender/receiver tasks for RSA and Serpent packages simply provide in-memory queues that can be made therefore larger than the IP stack provides otherwise, for both outgoing and incoming UDP messages so that losses are kept to a minimum and callers from the rest of the code don’t need to wait. They are both based on the same generic Snd_Rcv package simply instantiated with the corresponding package size.
2. The Comm_Link package handles all the practical details and inherent complexity of keeping in sync over a lossy communication channel. This includes keeping in sync the two buffers of Serpent keys as well as providing everything required for overall key management. While this sounds little, in practice it’s quite the opposite of little, no matter what you measure there. Note that this package provides everything required and handles its own internal logic but does not attempt anything beyond its station – the higher logic for communication management simply can not be at this level and so you’ll find it in HeartBeat, further down this list.
3. The EuCache package maintains and provides access to the knowledge tree of the game’s world aka that boundless although finite data hierarchy. The role of EuCache is that of a librarian essentially: it accepts all new data as it becomes available, it integrates it with what it already knew and it aims to provide on demand more than just the raw data since raw data is rarely directly useful as such. What is more useful instead and what EuCache aims to provide in addition to the raw data is at least one level of integration on top of it, basically trying to take any useful tiny steps that it can towards that ideal known as knowledge. How far it gets and therefore how useful it turns out to be depends more on you and what you figure out to ask from it in due time. For now and for starters, the possibility is there, the examples are also there, the rest remains up to you.
4. The Torrents package handles the rough as well as the fine mechanics of file assembly, storage and access. This includes receiving and fulfilling requests for retrieving files (whether local or remote), handling all the different bits and pieces of chopped files as they arrive over the network, checking each of them and asking for any corrupted or missing parts, as well as assembling the full file, storing it in a convenient manner (currently neatly ordered by type) at the path specified by the user in the configuration file and pointing any caller to the exact path for any specific or default file ², as requested.
5. The HeartBeat package coordinates everything in Core at regular intervals. This contains also the higher level logic for it all, since that requires almost always repeated and/or time-based decisions, from requiring an update of the world’s data to repeating a request that is still pending or deciding on resetting the link entirely when no other measures manage to recover the synchronization with the other end.
6. At the outermost edge of Core, the EuInterface package wraps up and exports with the C convention (so that any C/CPP code can use it directly), all the functions and procedures that modules built on top of this Core may use – from asking the server for assets or information to sending back to the server any of the user’s actions and everything in between, even building up and accessing a full local history of all user’s console commands.

In addition to the more prominent packages briefly described above, there are also quite a few utilities packages that provide a common language of types used across the whole Core lib as well as the lower level tools required and employed by those higher level packages. At the very base of EuCore, there is thus the Raw_Types package that all the others rely on, since it defines all the data types required by the communication protocol, as well as structures, subtypes and conversion utilities that rely on these types and are widely needed otherwise for various mechanics. On top of Raw_Types but focusing on structures and utilities supporting knowledge acquisition rather than the low level protocol mechanics, there is Eulora_Objects that provides for instance the records and hashmap types for storing any euloran object as well as the full known hierarchy of the world at any given time.

In between the above two data structures packages and the higher level packages listed previously, there is a middle layer of tool providers basically, from simply logging something or obtaining raw octets from the currently set source of entropy to Keccak hashing, RSA-OAEP encryption or message packing/unpacking and checking.

A full graph of the dependencies between all the Ada packages in the client’s Core current version shows quite naturally the top (EuCore) and bottom (Raw_Types) two packages as well as the strictly and clearly hierarchical structure overall: while there are indeed many links between packages as they are rather intensively working together ³, there are no cycles and moreover, no dependencies going the wrong way, meaning from bottom to top. All the links go clearly and as they should, always and forever from top to bottom (the red rectangle surrounding Keccak is there simply to mark that Keccak-Hashing is its child package):

1. Core’s code currently makes use mainly of protected objects and task types/variables.[↩]
2. Eulora’s client will be quite independent in deciding what it uses as defaults for any and every thing. There are no externally imposed or defined defaults because that doesn’t solve anything since one can then simply not have the officially declared defaults and so what is that supposed to solve really, you’ll end up with defaults for defaults or what exactly? So the cut made there is deeper but cleaner: the client will simply pick as default the first (as in oldest by its own timestamp) file that it has for each type and that’s that. Yes, this means that different clients can easily have different defaults and it also means that you can even pick your own defaults if you care – just change the timestamp for the file you want so that it’s oldest and that’s all.[↩]
3. Yes, they *could* be further separated but at *the cost* of additional significant verbosity that really doesn’t make anything clearer nor easier to follow – there is nothing “easier to follow” in introducing a few layers of indirection in there just to provide separation that otherwise is not really meaningful nor needed in any way.[↩]

This past week I worked on a significant refinement of Eulora’s data hierarchy, that core structure used to fully describe for the client the game’s endless, complex and ever-changing world. As it happens, this refinement was made possible as a result of a lot of previous work ¹: having recently gained full control over the creation of all graphical elements as I successfully grew my own mountains alongside GUI elements and literally everything else involved, I was also finally able to break up into more useful pieces and ultimately get rid of that sore anomaly of an “Outdoors” object that had previously catered to the requirements of the now-discarded “terrain plugin”.

For the easy part in getting rid of that bulky and spurious “outdoors” object, I had at least already in place the natural building blocks that would get used, namely the CSModel object as well as the ways to position and even name it, if desired. Even so, a change such as this, touching as it does the game’s own way of structuring knowledge essentially, can never be a very easy thing to get right, as there are many different connected pieces that have to fit – at least to some working extent – at all times. So I was a bit apprehensive about the amount of work it might end up requiring in practice and I made therefore a padded estimate of the time it might all take.

At first, I made the needed changes on serverside, where everything is defined and created in the first place anyway. There was initially more work than expected, indeed, but mainly as a side effect, a sort of slightly hurried growth, rather than anything else: while the change to the hierarchy itself was quite straightforward, the details pushed into being other parts that weren’t yet quite ready otherwise, although they were somewhere further down on the list for later, anyways. So I took more time than I had initially estimated to design and then add the needed embryos of these parts too, even though they still miss a lot of other parts that they in turn require and as such, they are sure to further change as they develop. This is expected and unavoidable though – it’s how everything else on serverside currently came into being as well, after all.

As the serverside work stretched even beyond my padded estimate for it, I was increasingly thinking that I’ll have no choice but to revise the overall estimate entirely, as the remaining time for the whole clientside work shrank and shrank. But then, when I finally got to update the data hierarchy in my own EuCore component on clientside, there was indeed very little required: simply the addition of the new object types, the deletion of the old one and a few updates (flagged automatically, too!) to take them as well into consideration when looking for broader categories such as “graphics vs non-graphics objects”. The rest of it – and moreover, the *more significant* and difficult rest of it simply worked and not by accident either: it was simply my previous design and implementation of both EuCore and its interfacing with the GUI saving me days of work now, as it handled entirely new types without skipping a beat and without requiring otherwise any change *at all*. How pleasant is this as code to work with and when exactly is the last time you encountered such ease when making significant changes to the code you are working on?

The second code change that made its way onto my list this past week was in principle a very simple thing to do, clientside entirely: as the water was now simply a CSModel like any other item in the world, it ended up nicely rendered but nevertheless surprisingly…solid, despite the data hierarchy otherwise holding enough information as to its “water” nature. In other words, there was clearly a need for the GUI part of the client itself to become a bit more discerning and create water as water (not solid and also reflecting things around, for instance). Since I had already written the code that successfully created water as such, this change was in principle just a matter of adding and using a flag, nothing more.

In practice though – in hindsight unsurprisingly, as it touched the remaining PS code – the simple change turned out to be not that simple and especially not as straightforward as expected: first, the water was rendered reflecting quite nicely everything around, so that it was clear that the discerning part worked as expected but… this “water” was still a solid, more like frozen ice perhaps than any water. Then it all got even more puzzling: the water was still solid *despite* my loader specifically *not* setting collision detection on it (in graphics terms, this solidity is simply a matter of whether one bothers to check /enforce any bumping into the object in question or not). Moreover, this unexpected solid was not even reliably solid but rather a weird half-assed sort of solid: one would walk on water indeed, for all the incredulity one might have as to the possibility of such feats but then, unexpectedly (a loss of faith, perhaps – whose faith, though?), one would sink deep into it and then… oh, then, one would get *stuck* in there, too, unable to move in any direction, if able nevertheless to spin as much as desired! Basically I had inadvertently created in there a sort of treacherous quicksands for all their watery looks otherwise.

Both the cause of and the resolution to the above unexpected wtf turned out to be of the same sort as previously encountered: the cause was a remaining bit of planeshift code and the resolution was to discard it entirely. Basically there was a spurious “psSolid” wrapping of the engine’s own collision detection object doing nothing other than getting in the way – it literally added *nothing* of its own other than the perpetrator’s name at the top of the file ² and otherwise another indirection layer in the way of anyone trying to make sense of the code overall. On top of this, some other bit of remaining PS code *also* took it upon itself to decide and add this half-assed psSolid thing to *everything* that went through it, just like that. As a result, there was this funny situation where simply bypassing it all solved the problem entirely – as usual, not playing along with the stupid has this magical effect of making all sorts of problems just… disappear. So I further deleted some of the increasingly less but sadly still remaining PS code and as a reward for it, I had my water being water again, nicely yielding rather than solid and otherwise letting one freely move through it at all times, instead of randomly trapping unaware swimmers in it.

Having previously trimmed and trimmed again at these same planeshift tendrils, this latest trim was otherwise quite effortless too, basically a very pleasant and rewarding way of improving the code from all points of view. How is this for coding work – effortless code *reduction* for added benefits and increased functionality, too. How does this compare with your current coding work and experience as to what coding even involves, hm?

1. This most recent work merely added, in turn, on top of full years of cleaning up all sorts of strata upon strata of a huge mess.[↩]
2. And I have to say that by now I had even guessed correctly the exact “author” name in that file, despite there being quite a few names scattered otherwise around the CS and PS codebase, you know?[↩]

Prelude:

~it’s ten minutes to set up and ten minutes of use to make itself indispensable. the chick can code, i don’t remember a tool becoming THIS FUCKING NEEDED in recent times.~

The Story

Going through my own archives and old logs may be time consuming but also quite an interesting exercise, meaning as it does that I get to look at an older context and a younger self again, only with more knowledgeable -and not just older- eyes. With the benefit of hindsight and of accumulated experience, even well remembered situations and well remembered words (of mine and of others) still yield more meaning than they did at that time, for what were *then* merely tiny signs and seeds meanwhile either grew and blossomed or shriveled up and died but looking back one can more easily see -in both cases- why and wherefore that specific outcome was even quite likely for each of them to start with. At times there are also the rather funny in hindsight bits, like those very, very early signs of my want of a bot:

diana_coman: I’m rather clueless at scripting unfortunately, did not have to use it much until now
chetty: there is a plan to allow crafting to proceed even if you are not online, queue up actions, but it is a way off
diana_coman: I will try to look into it, but not sure how much I can do ¹

diana_coman: chetty, can you help with any pointers regarding eulora client’s structure if I am to start looking into the code? I have no idea of a crystal space app, so probably any ideas/pointers would be helpful
chetty: well what did you want to do with the client?
diana_coman: I first need to get some reasonable understanding of the code and what is done where, as otherwise I will hate just hacking it ²
diana_coman: but anyway, I was thinking of something along the lines of making annoying tasks less annoying ³
diana_coman: such as: automated “go to coordinates” ⁴
diana_coman: (I don’t even know how difficult that would be, lol)
chetty: lo, well the user interface pretty much all goes via ‘paws’
diana_coman: or “repeat crafting x until out of recipes/ingredients/etc”
diana_coman: or even shopping list thing for a given recipe
chetty: ahh some nice botting, that would actually be pretty cool
diana_coman: well, after playing it so much ⁵, I think I have some clear ideas of useful features, but I am still clueless regarding the client code, so not so sure how much/when I get to actually implement :p

diana_coman: for the moment more like trying to figure out the code with the hope to improve a bit the game experience afterwards; as in: automate repetitive tasks.
thestringpuller:reminds me of Eve
diana_coman: wasn’t Eve anti-bots?
thestringpuller: so is WoW but it didn’t stop people from using them
diana_coman: well of course, but as far as I understood Eulora is actually pro-bots
thestringpuller: tis like dinosaur island without the geneticism. (dumb facebook puzzle thing)
mircea_popescu: diana_coman eulora is actually pro. and ima write an article detailing it ⁶.

The above log conversations happened throughout June 2015. By the end of July 2015, I had released version 1.0 of my Crafting Bot. I still recall those weeks in the summer of 2015, of rather intense wtf at contact with the legacy client code. Not as much because they are the worst I know or even knew then otherwise but more likely because they packed quite a shock, seeing how at the time I had been happily away from code for years. So it was an unpleasant shock, a lot of unpleasant work and otherwise no help to be had from anywhere, but I still made my bot because I had a use and a need -in other words a real want- for it, as simple as that. Or as I was writing some time later: last year I started programming again, rather unexpectedly. Simply out of interest, as it happens with most things worth doing: I wanted something that did not exist, so I knuckled down and did it – as a result, now it exists.

From there on, things moved on quite quickly. There was indeed a prompt reward and publicity granted for and to my fledgling bot, as well as some various further discussions of it all on irc ⁷. At least one other player clearly stated they had a use for the bot too but if you think that actually helps concretely in some way… what can I say, it doesn’t really, no. At any rate, by September of the same year, I had released the second version that included a lot more but was also essentially a full re-write of everything: basically the first version served more as a prototype and ended up thrown away, the new bots re-written almost from scratch and all that on my own effort, my own time and my own dime, imagine that! Silly woman that I am to actually put in such effort when nobody is paying me for it and others are even stood -and waiting- to benefit from the results ⁸, right? Well, perhaps it all comes from my stupidity or perhaps it all comes from evaluating things a bit more thoroughly and from more angles and sides than you are used to even consider at all. At any rate, even laziness might have more sides than you think and for illustration of this, here’s the intro to this second version of the bots:

Like anywhere else, work in Eulora is a most wonderful thing …to watch. So rather than work on working, I worked on avoiding working – and [as a result] here are my two bots ready to take over Euloran world

By November of 2015, there was yet another version of the bots released and I can tell even now from my own writing that I wasn’t all that happy with all the muck and yuck involved in getting things done there. But nothing worth doing gets ever done without some struggle or without having to overcome unexpected shit so what is needed gets done whether it’s pleasant, easy, convenient, expected or exactly the opposite (as direction, while double, triple or many times over as intensity) of all four and more to boot.

The above is not to say of course that there weren’t also good things coming out of it – there were for instance the first actual maps of resources entirely *made possible* by and through my bot, even as early as October 2015. There was also the fun had in finally having a working systematic tool to use against the full bulk of unknown ⁹ that stood between myself and any significant gains in Eulora or otherwise put between myself and …actual playing (scratch that, I mean: winning) of the game! While it’s not documented as such at the time because I didn’t want to give it away ¹⁰, I can still recall that fine morning when I woke up to find that the tirelessly working bot had discovered an entirely new item and had produced already a very clear map that fully outlined the area of interest, as neat as neat can be. I’d say that’s about as close as one can come in this time and world to discovering new worlds essentially – if you think Eulora is less of a world for being made of bits, you might be missing yet again some sides and angles to it, so perhaps read a bit more about what Eulora even is and how it’s complex economy means way more than it may seem at a first glance or than the construct “real cash economy game” has meant so far.

At the end of 2015 and beginning of 2016, S.MG faced an unexpected blow of just about the worst sort possible: the CTO at the time and co-founder of S.MG fell ill and she didn’t recover. The announcement came as a terrible shock in more ways than my brief words in chan at that time might suggest. Then again, the #eulora chan or any other irc chan for that matter is not the place for everything, either. At any rate, this is the sort of event and resulting context for which nobody can really plan or get “fully prepared” in advance – when it happens you’ll face it with whatever you got to prepare *unknowingly* up to that point and whether that’s enough or not quite or not at all or only just there, it’s still all there is. In this case, all that useful work that I had put in before opened up simply a chance and nothing more, certainly not much choice: a chance for Eulora to restructure rather than close down and a chance for me to get involved even more than I had ever considered even possible by that time. Note perhaps that there’s little choice involved in this at all, more of a close and narrowly avoided encounter with the end.

As I started working for S.MG, there was of course less time to pour into developing the bot further. Nevertheless, its use only increased as players simply could not quite afford to *not* use it and there were various additions to it by myself (for instance, bundling) as well as by other players: first, Daniel P. Barron’s drunken explorer mode and then Mocky’s updates that were directly commissioned by S.MG though meanwhile vanished ¹¹ together with their author.

The article on Foxybot’s 1st anniversary gathers together quite a few pieces of evidence regarding its usefulness and widespread use. And beyond that, there is of course a lot more that came out of it, not least of which a lot of discussions in the logs and a distillation of that into a whole model of software development, no less. While that specific direction didn’t *yet* get pursued much further (mainly for lack of time on my side, as I got swamped with a lot of other more pressing matters), it’s still precisely the sort of “unexpected” growth that marks as far as I can tell ~all things worth doing, worth *committing* for, in the first place – correctly or not, I consider that this is the best sign of having made an excellent decision as to what problem to solve: a good (or at least decent) solution to a random problem may solve that specific problem and no more but a similarly good/decent solution to a problem that *matters* offers in return a whole lot more than the initially visible direct benefit, as it practically opens up entirely new directions for growth.

The above theme of opening up new directions for growth is in fact very central to Eulora as a whole, I should say – and with good knowledge of it – by now. There’s the whole of the S.MG category documenting that for one perspective but meanwhile there are all the additional directions merely touched and barely opened up as part and parcel of all the work that went into getting some actual computer graphics and making the 2.0 client. While the old Foxybot is not going to make it as such into the new client (mainly as there’s little point to importing through it all the state-machine madness that was imposed by the rotten parts of the legacy client), the directions it opened are not lost or closed either and the automation of tasks will be just as needed some point anyway – only quite an easier task to implement with the new client and protocol.

Nevertheless, for all those openings and opportunities happening in plain sight, for all the documented amounts of Bitcoin involved and exchanging hands through the game and for all the previous opportunities not taken when available and closed meanwhile, those fewer and fewer remaining openings seem just as invisible today while available as the previous ones were back when they were still available, too. Arguably this is also exactly as it should be: the very same environment can seem empty and deserted to some and overcrowded with opportunities to others. At the end of the day, those seeing and capable do, those bright enough may at least follow and the rest will – at best – wait for whatever might happen since no happening has anything to do with them at all, anyway.

1. Bwahahaha. How much did it turn out I can and did do and how much more there is to be done…[↩]
2. Oh, the naivety of this young self of mine, how I wince to see it yet again! Did I get meanwhile a “reasonable understanding” of the unreasonable, would you say?[↩]
3. This is how useful things start before it even registers as a start: by improving something that is, in its context, in its *meaningful* context. Not all that much by “being found”, nor by popping out of nowhere all formed and valuable by and in itself, objectively and in a void, no, how surprising. (For my English-speaking readers, here’s a translation of the highlighted fragment: The attempt to create value alone and in a void, apart and away from actual people and their interactions, the failure to understand that such a thing is not even possible since all the value comes from the subjective evaluation by others, the nutty search of some objective “gold” in the virtual world, those are all telltale signs that you still have all sorts of lessons to learn about the virtual as well as the real world and anyone having any money would certainly prefer, at least in principle, that you learn those needed lessons on your own money [rather than on theirs].) [↩]
4. The most important part of this aka pointing to the correct place was relatively easy to do, if extremely useful. It’s still in use: the /pilot command.[↩]
5. In other words… after getting some actual experience and therefore beginning to form some understanding of the whole context too, as just the most basic prerequisite for coming up with anything useful anyway.[↩]
6. Worth a read and a re-read at the very least, I’d say.[↩]
7. The whole conversation there is quite funny for me on re-read. Then again, I suspect there’s at least one participant who’d rather prefer that the conversation and even its whole context never existed at all.[↩]
8. Note that this *second* part is very important, too: while a lot of effort can be put into all sorts for which nobody is paying you, it does NOT follow that the lack of pay is a way to choose things or some sign of …usefulness! This is not something new, of course, nothing is new after all, but here’s a more straightforward statement that may help with it all (the loose translation to English is mine): in short, the fear and avoidance of competition is the mark of the incompetent and of those who know already that they’ll lose. The rest of people deliberately look for competition, focusing on identifying salient inefficiencies, coming up with strategies to get rid of them and then knuckling down to work. That’s what doing business even means, not just on the Internet but everywhere[↩]
9. As part of the attempt to attack that, I also made and published a whole library of items that turned out very useful too.[↩]
10. There is some data published at a later time and then even the actual fragment of the map, close to one full year later.[↩]
11. Eh, except those I have mirrored myself, of course, as usual.[↩]

I don’t care how many angels can dance on the head of a pin! What I want to know is how many tiny details can one devil simultaneously be in.

The only reason why my love for graphical user interfaces (GUI) suffered no loss over this past week or so is that I hadn’t any such love to start with. The clientside required bits and parts for performing actions turned out to be a considerable pile of small but quite detailed pieces that seem tiny each on its own but add up quickly to eat a lot of time while remaining otherwise hidden internals so that there is at first nothing to actually show for all the work. As I looked in more detail at what the GUI needed to be able again to fully interact with the world, the work seemed at first to just keep growing at every step instead of shrinking: there were GUI elements missing and then new methods required for querying the local cache and new configuration options and synchronizing and whatnot.

Experience helped here a lot, as I know this sort of pile-of-details stage for exactly what it is, part and parcel of work at times. So I just kept the work as contained as possible and otherwise I kept going with it until I finally brought it all to the point where there are enough of the required internal mechanisms in place so that they can support the desired GUI interaction itself. As a result, I can illustrate now at least some of the new additions, if still not all. At any rate, Eulora’s client has now in place the basic mechanisms for interacting via mouse or console (text) with any items or characters in the world, whether they are contained in one another or simply directly in the world, on their own. There are still all sorts of details that will have to be sorted out – or possibly altered even – as everything else comes together but those are secondary details, those that can never be fixed in stone at the first pass anyway, nor should they ever be so inflexible as to have to be fixed in the first place.

To start with, the Console has increased usefulness and usability. The most important improvement from my point of view is that there is now in place a generic mechanism to deal with any command from the user: the first word is the command itself, while any following text is read as a list of “parameter=value” tuples, where the order does *not* matter and there is no restriction whatsoever on what “parameter” might be. Basically the Console simply grabs the full text, uses the first word to find the action that goes by that name (if any) and then passes whatever text follows to the action itself to handle. The action in turn has its own default values for all parameters but then goes through the given text and sets the given values as found, if and when found. As an interesting effect of this approach, the user can even set the same parameter several times in a single command but the value used will simply be the last one encountered. Moreover, as much as possible the same letter stands for the same thing for *all* actions – the most obvious example is the target which is t everywhere, not like it’s all that difficult, really. Anyway, by means of illustration, here are two screenshots with some commands:

The above new wiring of commands to actions means also that I can now easily add new actions at any time, since it’s just a matter of implementing what the action is meant to do and otherwise adding it to the list, nothing more – all the wiring from Console and even from the rest of the GUI will simply adjust and include it too, as it should. In addition to this, there are a few more small but rather very useful improvements including clearer feedback from entered commands, resizing of the Console window so that it correctly resizes the two internal parts as well and a 1-line “history” so that one can recall the previous command at any time. In funny inherited weirdness that I’ll probably have to look at later on, it turns out that a resize has as side effect the reset of the font’s colour though! Such is the underlying nature of the inherited core widgets, this ever plentiful source of surprises of the most… surprising kind.

As you can perhaps notice in the illustration above, the GUI offers now the full list of actions defined in the communications protocol but also a few additional “actions” of its own, such as “pos” or “examine”. The reason for those additional actions is that they are simply quite convenient for the user and entirely a clientside concern: one might want to find out their exact position for instance and the GUI can expose that easily so I’d very much rather it did. There is of course no limit to what can be added there either but I didn’t spend time to polish this: the point for now was to have the very basics in and otherwise have it there as an example rather than anything else. For illustration, here’s the result of “examine” applied to the character itself:

While the Console remains to my eye the easier interaction option the user can have, I’ve nevertheless put in the work required to have a working mouse-powered option as well. So the skin.zip file got bigger and includes now at least one icon for each type of (inter)action known to the client. (And don’t ask me why is the “repair” icon showing a hammer crossed over by a rolling pin, I can’t begin to imagine where that combination comes from!) Those icons are then used in a new Context window that can be accessed with a right click on any item ¹ and provides links to all the known actions. And since my GUI is not made with love ² but with calculations whenever possible, this Context window simply uses basic trigonometry to just figure out where each icon goes so that all actions are shown neatly in a circle. As a consequence, if a new action is added, the Context window will simply adapt and include that too, there’s no need to add it manually and fiddle with its position through xml until the cows come home. Such are the results of bringing the ancient magic of the sine and cosine to the innocent lands of GUI and other computer ruralia! Lo and behold the euloran outer wheel of perfect roundness and plentiful context interaction (see the console in there for feedback on clicked actions):

Getting back to the actions itself, with the exception of lock and exchange, all the other protocol-defined actions are currently fully wired into the client, meaning that the relevant messages are created and sent to the server. The movement action is at the moment rather annoying and choppy so there is certainly more work needed there but I think it should still wait as some of the issues are likely to get smoothed out anyway as everything else advances (part of the choppiness is likely to be more a matter of the values I have currently set for the sync intervals for the whole world, for instance). Although I didn’t fully go into all the remaining movement part at this time, I start thinking that I’ll have to do that too at some point, most probably to finally kill that stupid of “leg length” and “falling below 0”, if nothing else. Nevertheless, it can still wait a bit, perhaps.

Arguably the most time-consuming “action” in all the above was the move/pickup since it required the whole inventory GUI with slots and equipped vs non-equipped and stacks and splitting stacks and moving items in inventory too and everything under the sun in there. There was a bit of mini-puzzle fun with the formula for inventory (non-equipped) slots, some annoying further misbehaving of that listbox that has the handy scrollbars and all sorts of weird discoveries regarding the number-prompt window. Nevertheless, the whole thing is now working, the Contents window got basically an upgrade, can move items about and is quite ready to be of further use as half of an exchange of any sort. The “move” action on items in the world is effectively a “pickup” – it will attempt to move the items to the next empty slot (calculated on the fly, too) in inventory and won’t care a bit whether that is possible or not possible. After all, you never know unless you try so my GUI here is not at all in the business of telling the user what they can and can’t do: it provides the widest set of options to everything and then dutifully executes what the user asked for. The inventory window updates with server-given data only on reopen (all it takes is to press i for that) but otherwise it will happily show the user whatever they thought they could do, pretty much. I’m quite sure that there are all sorts of details to change here on future iterations but for this first pass, I think it does as much as can be done for now:

For the number input window, I initially wanted to reuse the original window as it was since it had in place some input checks and limits and all those things that are in principle quite useful. The fun part started though when it turned out that it had of course all sorts of silly expectations – that got therefore cut away – and otherwise it used a horizontal scrollbar to allow the setting of the amount via mouse movement. This horizontal scrollbar was simply not able to use 64-bits integers without requiring otherwise a full update throughout the whole client, basically in all places where a scrollbar is used at all. So rather than indulge in such waste of time, I just dropped the silly scrollbar-set from this window and otherwise updated the remaining small part to deal with 64-bits integers, what else. Since I was at it, I added also overflow guards, as there were none but other than thatm the thing is (at least now) small enough to be perhaps of some use currently.

On a side note related to inventory and slots in general, I added again the quantity and quality labels to each slot but the trouble is that the maximum 64-bits value has 20 characters and to fit as many as that onto a slot, the font ends up so tiny as to be unreadable. So for the time being I let them be as you can see in the screenshots above, using a reasonably small font that means however that the full value might simply not be visible at times. I suppose this might remain as one of those low-hanging fruit to pick for any client developer out there.

Regarding the actions themselves, on serverside, at the moment, those action messages that the client sends are received, unpacked, recognised as what they are and otherwise promptly discarded. There’s quite a lot to implement on serverside before they can fully be acted upon as intended. Nevertheless, for current testing and illustration purposes, the serverside simply has some mock data that it sends to the client as “the world” and that is quite enough for the moment (though it won’t be for much longer, by the looks of it all).

With all the above in place and working, there would still be the exchange and lock actions and corresponding GUI elements as such. Other than that, there are various non-GUI parts that could be done (and need done at some point anyway) on the client, such as the management of Serpent keys and actually dealing with a change of connection/server on the fly.

Let me know in any case if there’s anything else/different you want done for the GUI part at this stage and otherwise if there’s something else specifically burning/best to tackle next. I should also mention perhaps that in the interest of documentation and seriously applying myself to learning the craft of GUIs, I have indeed reinstalled Diablo as well and gave it a spin – the windows are a pleasure, the tick-tock of idiotic walking is more annoying than any skeletons that might appear from the barrels. Nevertheless and purely for work interests, I shall therefore endure Diablo re-plays whenever required!

1. This ran at first into a weird “attachment” thing that apparently was required in order to have a direct link between CS meshes and the game objects that the remaining client core works with more directly. And when that in place, there was the repeated and infuriating further run into the PS notion of “any item”, such as it still remained in the various parts that are still in use in any capacity: “Oh, by any you surely *don’t* mean meshes AND moving items, do you?” and “Even if moving items too, then *surely not* your own character, too!!” and so on and so forth, it’s like the devil is not in the details but rather the devil is a detail inside a detail inside a detail inside… [↩]
2. By now this made-with-love advertisement stands as far as I can tell for “it will blow up in your face but you should be happy about the blow up, because it didn’t mean to do you any harm!!”. Here’s an illustration, too, some typical result of the made with love approach.[↩]

This report brings to you the latest news from the lively Euloran development grounds: the GUI is shedding code and growing in usefulness, the Core adds to its services the packing and unpacking of all user actions known so far, the Data cache refuses to be led astray into unnecessary sync swamps and gets its responsibilities right, while overall all three parts arrive at a clearer way of working well together and working well apart, providing what’s needed for others but minding otherwise their own business at all times. This last part of “minding their own business” took a bit of thinking over those past few days, as the newly added user actions didn’t yet have a clear place of their own and it’s precisely the clear and fitting definition of “own business” for each component that makes or breaks the whole: do it right and everything fits afterwards quite “efortlessly” or do it poorly and then you’ll fight at every step to fit all sorts of square pegs into round holes (and rebuild the swamps, too, as a result of that sort of mindless fight).

To describe the whole thing, it’s clearest (and not too long) to start from the beginning: Eulora’s client has currently implemented 2 main components, namely the Core and the GUI. The responsibilities of those components have been defined initially at a high level only, as the details were not yet clear. That initial definition got gradually more detailed as I implemented more and more of the actual communication protocol but broadly speaking, the original scope is still valid: the Core is responsible for handling *all* aspects of communication with server, from network data packing and encryption to actual logic for data acquisition and storage; the GUI (as stated in its name itself) is responsible for data presentation to the user and handling all user interaction.

While the above high level view is clear enough as it is, the trouble with going into the more detailed, concrete user interaction is that its very purpose is precisely to *change* the data so carefully acquired by the Core and so neatly presented by the GUI. Add to it that any user actions in the game are more accurately defined as *attempts* to change the data, with no guaranteed outcome until and unless the server actually changes the tiniest bit of the game-world in response to them and it would seem that those user actions are indeed straddling most uncomfortably the responsibilities of both Core and GUI: get the action from the user (GUI), send the request to the server (Core) but also change the data already so that the user promptly sees the effect of their action (Core *and* GUI?). At which point, the reader familiar in the slightest with the PS coding swamps might recognise that old pattern of “all GUI elements require a connection to the server and direct knowledge of message types.” Because indeed, it can easily *seem* like a …solution of sorts, except it’s of course no solution at all but simply the path of least resistance and even less reflection: it “seems” and therefore it is, why bother with anything else at all. Well, to keep this short, I’ll note only that I’ve been toiling for quite a few years precisely cleaning up the results of others’ even *more years* of following that path of least resistance.

The change to data as result of a user action is further problematic in that any change done by the client is essentially a guess: at best it gets confirmed in time by the server and therefore all is fine but at worst it gets undone (while *other* things may change too!) and that brings with it the potential further invalidation of all sorts, even of later user actions! Quite the thorny problem to handle, as any synchronizing problem ever is. So the naive approach to handling those thorns goes (again!) the route of yet another part of the PS swamps: we’ll effectively mark data in the cache as “dirty” or “clean” and then go back and forth and over it all to attempt sync with the server and to handle all possible cases that are not even all that clear in advance, since they can’t possibly be. For the most obvious trouble, there is no guarantee as to *which data* changes as a result of any given action so going this silly route of marking dirty data is at best a pretense to solve the problem but certainly no solving it at all. So no, naive doesn’t work any better than lazy above did but it serves at least to highlight more clearly the root of the whole trouble, namely that there are in fact and inescapably TWO data caches on the client: one is indeed the explicit data cache that is held by the Core but the other is the implicit data cache that the GUI keeps in the graphical engine itself! There is no way in which the GUI can show a persistent representation of data without having therefore a cache of that data and as a result, the GUI can’t shirk taking on the full responsibility for the *local* data cache where it can make whatever changes it wants, without however expecting to have any direct access to making changes on the authoritative data cache that the Core maintains.

In a nutshell, the Core aims to maintain its data cache as closely as possible to the server’s view of the world, without caring at all about anything else, while the GUI aims to maintain its data cache as a reasonable guess of the world essentially: as responsive as possible to user actions and as close as possible to what the Core provides. This means that there is really no need to mark some data as dirty and some as clean in either of the two data caches: the one in Core is always and by definition as clean as the client can ever get; the one in the GUI is at best a fresh copy of that in the core and at all other times a… guess. The Core will regularly refresh its full data view by asking the server for a World Bulletin (and then following up through the world hierarchy as relevant), while the GUI will regularly refresh its full data view by asking the Core for what is in view at any given time. Whenever a user action happens, the GUI will promptly provide to the user whatever feedback it can and otherwise it will simply ask the Core to communicate that action to the server. There’s no need for anything further really: whatever the result of that communication, the changes (if any!) or the undo of attempted changes (if any!) or in other words whatever effect the action has or doesn’t have will be obtained anyway as part of the regular refresh of the data done by both the Core and the GUI on their respective caches.

While the above general principle works for all user actions, there are indeed some that are easier to handle than others. In particular, movement is quite apart from the rest, as it’s basically a “continuous” sort of action that moreover needs to have an immediate effect shown to the user. So the GUI will have to balance here the need to synchronize the player’s position with the server to avoid annoying abrupt relocations and the need to keep the user’s costs low as all messages sent to the server have a cost and nobody will be all that happy to use a client that racks up huge costs just for moving about the land. Nevertheless, the great part in all of this is that it’s all clearly and entirely the responsibility of the GUI and so it can be all contained in one single place and none of Core’s concern. The less than great part in all of this is that the actual movement part in the current GUI is still afflicted to some degree by planeshittism ¹ and so it will still take some work to figure out a way to make this work, preferably cutting out or at least avoiding that remaining bit of swamp.

Other than the above specific trouble with the movement, there is still some work to be done to implement in detail everything needed for all the other, more straightforward actions. For now and mainly as a basic test, I’ve implemented on the client side just the simplest of them all, the “explore” so that I could confirm indeed that the whole approach works: the GUI reacts promptly to the user command and asks the Core to send the request to the server, the Core packs it correctly and sends it, the server correctly unpacks it and recognizes it as such. For that matter, I’ve implemented already the packing and unpacking (+tests for them) for all the actions, so the next steps will focus specifically on the higher level part of this and first in line, the clientside required bits and parts for performing those actions.

1. Without even counting all the parts that have been shed already in this area, there’s still a psCharControl and otherwise a psLinearmove that does the actual calculations and heavy lifting but also a psMovementManager that can anyway “manage” only ONE actor at any given time and through a tangled web of back and forth calls from actor to manager and the other way around as well as from other parts to both actor and manager or to actor through the manager as well as to the manager through the actor – the usual PS “style”, really[↩]

There’s no other title fitting this. Here’s the priceless context for the question above, context worth not one thousand but 100 000 lines of open source code, at the very least:

The above is the amazing result of trying to make use of all those psSlot and psSlotManager and pawsManager with their drag-and-drop wonder and marvel functionality, a true crowning achievement and glory of open source development. I ask the bloody “manager” class to drag something across the screen, as it claims it does and as a result it… breaks everything else pretty much, as nothing really updates anymore and the thing leaves a trail wherever it goes. But all is well, I’m sure, at least according to the manager and to everyone else working very hard and tirelessly and wonderfully and so on ¹.

As I am apparently still naive enough to try and debug even the above sort of madness, I ended up in short order following a back and forth web of calls that are increasingly bringing to mind the way chaotic systems tread their path, ever random when it concerns the very next step, ever building nevertheless the same known shape in the end, as predictable as it can be. To give a clear view of how this goes, here it is: the widget calls the manager and the manager calls then the mouse class that calls back the widget that calls its parent class that calls back the manager that calls -at another time- the mouse class that calls…

Fortunately, those days I get at least faster out of such naivety and optimism and so I stopped just about there and didn’t waste any further time with it but looked instead at some way to do it more directly. I suppose by now it’s not surprising that all that crazy web of shit for “drag and drop” was anyway on top of and to the side of a somewhat less crazy “move with mouse” and then it turned out that even that was spurious and in fact the core class of ALL widgets in that bloody gui even had a method specifically for …”center on mouse” sort of thing – and it even worked! So I made therefore my own “drag” as I had made my own prototype inventory window and my own prototype console overlay and my own seedling for the rest and let the many managers rot and fester in there unused and increasingly cut away from everything until that day when I’ll finally get around to delete the whole mess too and there won’t be any difference other than having suddenly a much smaller client that is however way more useful. At any rate, as a result of this most recent adventure, I stand corrected in my initial opinion that there might still be at least a tiny amount of utility in all that PS code – there isn’t, there never was, there never can be.

As to the current status of those windows and overlays on my list: I have now a perfectly working way to drag stuff about (not yet to drop it but I don’t expect that to be a huge problem, seeing how I am not going to touch any of the PS shit for it, not anymore), as well as working prototypes for both “Console” and “Contents” windows, with a few commands wired in already mainly for testing and poking that Contents view. Everything in those is created from code and without any xml in the loop so that the Contents view for instance can easily create as many slots as one wants, there is no limit of any sort anymore. The logical thing is probably to simply create slots *as needed* and without limit but for now I disentangled (and cut and wrote anew) enough of that underlying widget that has scrollbars – the scrollbars are useful but the design of it otherwise required a bigger hammer to bend into something usable in the least. As you might notice, I even gave in and added arrows for the scrollbars! In any case, all the images and graphics and colours are just for testing for now and I’ll gladly use better ones /let someone else design any window whatsoever. For now, here’s how the prototypes look like (in order: a test with adding and replacing stuff in the contents view, based on id of the *underlying* object instead of id of slot and other madness; the successful dragging of ‘shrooms with icon and everything else):

For the next steps, there would still be first a bit of cleanup around those latest parts and assorted cuts made for it. Then I’ll have to see if it makes perhaps more sense to get back one round to adding more working parts on the data side + communications and then let that drive any further development of the GUI itself. At least I’m now quite satisfied that I’ll be able indeed, one way or another, to add to the GUI what may be required, even if some parts are very likely to throw some more wtf of the sort illustrated in the very introduction to this and eat as a result rather more time than I’d like to spend on them.

1. It could never not be great and wonderful anymore either, there is no such thing as something *not* being well, nowadays it’s either great or otherwise forgotten or simply not mentioned for it’s surely the mentioning that is causing the problem in the first place, obviously.[↩]

Until such time that you make any buttons of your own at least, it’s now all settled, clear and without a choice: on all Euloran soil, it will really be my buttons you’ll have to push ¹ and my views you’ll have to use and my winPAWS ² that you’ll look at. For there was/is barely anything useful in all those 100 windows that were each terribly important on their own, each with their own layout xml file and own hand-rolled loader and own parser and own further numerous attendant misfortunes that are inevitably linked back to that mighty struggle of trying to have a singleton manager without admitting however of having any *specific* manager. It’s a riot reading the code and the famous “software patterns” and even all the Object Oriented paradigm in this obvious key. It’s even funnier now as the end of the swamps is otherwise in sight and as the intelligent idiots are out of my way and staying there in their narrowly delimited circle where they are taking over the world, I’m certain, only it’s each of them taking over their own world but that tiny detail is entirely unimportant and can’t possibly matter, can it? They are after all solving problems with code and moreover solving “problems” picked and ranked based first and foremost on what they are familiar with and the sort of solving that they have done before since that has to be the best way – the more independent, upright, sane and intelligent way. Not that this has anything to do with it being also the only way that is still in the least open to them, no, absolutely not.

Back to that fast growing life of Eulora, the dive into the full depths of PAWS was essentially successful if not lacking in crashing, segfaulting and downright maddening experiences: the whole thing is not unlike a carefully constructed castle of sticks that stands for as long as it’s untouched and then unravels entirely at the merest movement of air around it. There is in fact already a set of “basic” elements and those 100+ windows are on top of that but even those “basic” ones are at times anything but basic. The most screaming out loud example of this would be the humble title of a window/widget/whathaveyou – in order to show *any* title (as in text, a few letters, nothing more), the “widgets” require a border and that requires buttons and styles among other things and those in turn require preferences and all sorts that further require the requirements of requiring required requirings… Why does the title part require buttons or borders or all that? I suspect it’s because someone was solving there with his code a lot of problems that I don’t have! So I took instead the smallest thing that I could find, trimmed it further and added it directly to the core “pawsWidget” class so that any damned thing CAN now show some text as title without requiring anything more than the… text itself! Shock and horror, I know, just look how violent I am with titles and buttons over here!

Moving paws on the path further, I started to simply write the code for some generic “views” – the sort of windows that can be used for a purpose and otherwise displaying whatever information is relevant. At first, this ran fully into all sorts of intricate weird: the show that won’t show, the click that won’t click and other such crazy stuff from working with the castle of sticks where everything interacts with everything else and that results in a whole ritual -where order of steps is important, too!- necessary for achieving the tiniest effect. Disentangling all that was no pleasure but it’s still the fastest way I know of to get to the point where something useful can be done with the whole mess and it worked with this mess as it has always worked with any and all messes I ever encountered. As a result, I am quite confident now that I *can* make my views, windows and overlays in a reasonable amount of time and *without* a whole lot of clutter sticking to them.

For the initial practice, I first cut to pieces that “bar” at the top and then put back together only a skeleton of it, as an overlay that has so far only the quit and help buttons – and those are my buttons indeed, made through my code and in a more direct way than the original entangled messes. The reason why the whole bar itself is still there as a thing at all is that it holds also quite a bit of the keyboard interaction part so there’s more that needs to come away before it becomes fully clear if it comes off entirely or not. At any rate, otherwise I changed that pawsManager to actually act as one and therefore I have now in place the building blocks to allow the desired approach of “one window at a time and otherwise overlays”. This also cut a lot of crap dances around all sorts but mainly in the sense of not using a lot of code, not yet in the final stages of deleting and discarding it too. For illustration, here’s the new “menu bar” overlay and the confirmation window that a click on the Quit button made appear:

Once I had those buttons showing and working as I wanted them, the next step was to add various bits and pieces that arguably come in handy and could at least build on what I had so far. So I enhanced that minimal “helpwindow” of last time, as it’s in my view the prototype for a generic view aimed to show structured text ³: it has now a title and there are also functioning scrollbars that use Eulora’s own icon as tracker – I rather like them; the +/- signs for opening/collapsing a topic get shown now with their own icons in the skin.zip file; there’s also further experimenting making sure that both background and text colour can indeed be set as desired:

I’ll have to note here that the above is still not giving a damn about stuff like “borders” and stuffing buttons everywhere, from title to sidebars and anywhere in between. To be honest, I personally prefer by far the transparent and minimalistic windows and I make a half-hearted exception only for text really (as it’s rather easier to read when it’s the traditional black on white or gray on black). So if you’d really, really want those windows to have anything further, please say it sooner rather than later, as left on my own I doubt I’ll find much reason to add even “title bars” as separate things, let alone stuff like border-this and border-that or decoration-here and decoration-there.

For further practice with making whole windows really from scratch rather than starting with a previous overgrown thing and cutting thorns at all steps while still remaining with a crooked trunk, I made also an “Info” view that is meant to show an image + name + description for anything in the world: it’s barely a page of code and it even sets its size depending on how big your screen/eulora is, aiming only to maintain the ratio of its minimal size. The image gets indeed re-sized to fit exactly the space available in its side of the whole view so that’s perhaps not ideal but it can be changed for sure. To fill it with something for testing purpose, here’s the very info on Eulora itself, first in the transparent version and then with the black background for that half-hearted concession to showing text on a non-transparent background:

With the above done, my current list of generic views (aka windows+overlays) reads like this:

1. Command line – TO DO. This is in my opinion crucial as it’s meant to be the game’s console basically, the only place for the player to enter commands and thus be able to do anything, whether there is a button for it yet or not. This would be an overlay, I should think.
2. Confirmation – prototype DONE. This is a very basic yes/no type of window, aka one question + 2 buttons, modal thing. The current prototype is perfectly and fully working, if entirely transparent. The wiring of the buttons is now mine too, entirely circumventing the several layers of passing the buck that used to be “the way this is to be done”. I don’t know if there really is a need for any sort of other “view” as such – if it’s just a text message, it can still be shown either directly on the screen (that fading sort of thing, dropping down as it faded) or otherwise in the console/command line thing above, I guess.
3. Waiting – prototype DONE. This is meant as a placeholder/information for when the game is not available/unable to display the world as such. It simply takes over the whole game window and it displays an image + a label/text at the bottom. The prototype is fully functional and even wired in its correct place aka displaying at the start while the client is waiting to obtain from the server the most basic stuff such as who am I, where am I and what does this world even look like. I kept that boceto drawing as I quite like it (even more so than the “splash” thing).
4. Info – prototype DONE. This is meant to show image+name+description (or any subset of those) of anything in the world.
5. Contents – TO DO. This would show everything that is “inside” any given object in the world. As such, it could show the contents of some container but equally well the player’s inventory or the equipped items or a storage or the contents of an exchange of any sort. In my current hazy view of it, this will simply show as many “slots” as required, each with their own icon + text (e.g. stack count and quality or similar) + tooltip (to show the name of whatever is in there).
6. Exchange – TO DO. This would be made out of 2 Contents essentially, with wiring as appropriate for whatever specific exchange is shown at any given time. Basically the “inputs” would be the 2 items whose contents are to be shown on each side of this view. In my opinion and just as an example, this can even work to provide a GUI for activities such as “equip” since it’s an exchange – stuff from inventory being moved into equipped and the other way around.
7. Management/Ownership – TO DO. The details of this are not very clear to me currently but as I gather that there will be various types of managing involved (e.g. jobs or land), I’ve added this to the list as a generic window that can hopefully be made to cover adequately the management of whatever is needed. I’d say there is still time to figure out this in more detail and there’s no rush – it’s on this list as a type, not much more so far.
8. Skills/Self – TO DO. This is quite hazy too but what I mean here is mainly a view that is able to show also things that require something other than image or text: perhaps bars/percentages or the like. I am not fully clear on this part yet but there’s certainly time for it to clarify.

As next steps, I would push this a bit further still to fully transform that “helpwindow” into its intended generic structured content view and make otherwise at least a prototype of the Console/Command Line and Contents generic views. The Contents one is quite important because it means figuring out all that slots business and dragging stuff from one place to another, which is another potentially big chunk of this. Hopefully as part and parcel of that, I should be able also to finally delete another pile of useless code, as it ends up entirely stranded. In terms of specific GUI bits and icons, the more I work with this, the fewer of them I want in there so do let me know if there are *any* parts that you absolutely want in there as otherwise I’m afraid I might easily end up ditching just about *all* of them and have the whole “skin” consist in something like a plus, a minus, a quit icon, a help icon and a tracker for scrollbars, not much else really. Of course, anyone can otherwise ship to the client any .zip file with anything at all but the default client will not use anything else and keep those windows as minimal and as invisible as possible, by the looks of it.

Other than the above, let me know also if the approach of “generic views” makes sense to you and whether you’d like any others added to that list, even if just as general types, to be perhaps fleshed out (or discarded) later, as everything gets into more concrete shape.

1. Heh. I know.[↩]
2. Well, they’d be windows except when they are overlays and otherwise they are made still on top of and with various bits of the paws that apparently would really, really want to be of some use, if only it hadn’t been so thorougly misdesigned and abused by means of pointerfuck…[↩]
3. By structured text I mean here text that has a hierarchy of parts that can be shown/viewed separately and allow navigation through a tree of contents.[↩]

Against my own expectations, it seems there’s little resistance left even in that mess of paws: it took less than a week to map it all out quite fully this time, discard yet another set of dead parts of it, cut its remaining tendrils and pretense to “independent plugin, part of a common trunk, not just some code in the client”, bring it therefore quite in line (though not to internal sanity, it still is what it is) and ready to be made use of, without crashing and without complaining that it doesn’t have graphics or skins or resources or whatever else it previously imagined that the user must provide. For instance, here’s the gui just using the system’s mouse pointer if nothing else is available and the help window gladly showing whenever it’s asked to show, looking as best it can with whatever is available at that time:

Having thus made sure that indeed, I can make paws do something useful after all, I’m literally staring at the pile of neatly sorted entrails of the whole thing and I can’t make up my mind for the life of me – *what* is even the desired use of this paws-paste? So I’ll first write down the brief description of what’s there and then I’ll spell out this generic question into several more specific questions that can hopefully be answered.

The current client’s GUI interface is essentially a lame and broken attempt at implementing the world hierarchy from a… visual perspective ¹. There’s a top “controlling” ² window that keeps track of all the others and does in fact the generic figuring out as to what window is the user interacting with, which ones should show and which ones should hide, all that sort of thing. Being of course a visually controlled world, the top window ends up being… that menu bar at the top (see? it’s at the top, right?) of the screen, with the inventory and the quit button and whatnot. It’s true that there is otherwise *also* a “mainwidget” thing but… that’s just the overall container and doesn’t do much at all. Don’t ask me why can’t that mainwidget be actually main and therefore control the rest, it just isn’t. And because those are just windows and the whole codebase at the time simply couldn’t do without managers, there’s on top of this top a pawsManager with its very own textureManager.

The pawsManager is meant as far as I can tell as the entry point to the paws itself – at least when entertaining the delusion that paws is or can somehow be separate of the rest. The textureManager is one of those endless glorified list-keepers: it keeps its own list of filename-resourcename equivalencies for all images that the GUI uses and otherwise interfaces load/find from the engine’s own textureManager, of course. The meagre claim to purpose it has beyond that of list-keeping would be that it packs in one place some format-related processing of 2D images that are supposedly helpful in some cases (though it’s unclear if there are all that many such cases in current use). Anyways, aiming to keep changes to a minimum, it can live as such and keep its list too, since I have no intention of saddling EuCore with any internals of the GUI/skin now. There’s of course a whole dance around in both pawsManager and its textureManager with translating strings (literally, the “localization” because apparently it makes somehow sense to shove that concern in there), defining “styles” (there’s skins and then there’s… styles; because one S is not enough and nevermind there’s no game worth even one capital letter, yet, we’ll add styles to the interface and that’ll be grand!) and making oh-so-sure that one is never, ever, under any possible circumstances somehow left without – gasp! – a loaded “skin”. Additional, even smaller concerns abound too but by now we get to the microscopic so I’ll let them be already, suffice to say that if I’ll ever need an example of pointless polishing of irrelevancy, I can certainly use that code.

Past the 2 managers and 2 top-widgets, there is the indistinct mass of …more widgets. Basically *everything* in the GUI is a widget, from one single slot in the inventory window to the whole inventory window itself. Here I can happily report that indeed, the “off with its dolly” task was most satisfyingly carried out: there were 2 (two!) classes on top of all the usual widget+xml defined just to *support* that and several tendrils sneaking out into non-paws parts too and oh, what joy to cut them out already!

Dolly cast aside and readily forgotten, the reduced pile of widgets is still a considerable pile and otherwise waiting for either full chop or partial cut, depending on what exactly turns out in the least useful in there. The “design” of the whole thing is that each widget has a code class (in client/gui mostly) that is meant to implement what the widget does (and even if it doesn’t – there still has to be this code present as a means to register and create the thing, useless as it might be) and an .xml file (in data/gui currently) that defines in principle the layout of the corresponding window to be shown in game, including size, position, contained widgets and used “resources” aka icons and other images. In practical terms this means that new or different widgets still require touching the client, there’s no “download this” and run with it really. At a stretch, the .xml layout files could be made perhaps part of the “GUI” .zip and so something that can be downloaded from the server and used but there’s a very tight coupling between the code and the .xml so that it’s unclear to me if there’s much point to it anyway. So Q1 of those more specific questions would be here: is the layout of various GUI windows something we even consider part of a “skin” definition and hence up to GUI designers to produce as part and parcel of offering a client skin for use?

Even after the above trim, there still are about 100 widgets with their corresponding 100 .xml files, so I would rather not list them in here – the list is anyway quite easy to get with a simple ls data/gui in the deployed client’s dir. The more specific question here is Q2: what windows do we actually need in the whole interface anyway? I can’t quite see the point of 100 windows and it really all seems to me more a matter of ill-thought “design” in the first place, fitting slots in places by hand and adjusting them to be just-so, the usual sort of “artistic” approach to it all. I rather think there has to be a small subset of elements that will do it all and better anyway but at least currently I don’t even quite know where to start from with it, as it’s not all that clear to me what sort of things we need in the first place. I can even see the point in saying it’s too early to bother with that and move on to implementing more of the functionality so that what is needed emerges from there but on the other hand the GUI has to start with something/from somewhere too.

Going through the list of main things as seen from a player perspective so far in Eulora, the Q2 above can get further refined: the most complex window in there is the chat window – does this even still make any sense or should this be only and at most a cli window in fact? Windows like the storage for instance were relatively ok in the beginning and then an utter pain when one has a full storage, reminding me quite precisely of the more generic windows-vs-cli computer use really – do we still want a storage window at all? And where does this stop anyway, as on one hand I can keep going and wonder if the inventory window is useful as such (perhaps it’s at least pretty to see them all!) or the container or …which of those 100 total, in the end? (Then again, I’m a cli radical apparently (hence my asking here), regardless of all the fun I had making textures, heh.)

Windows and widgets aside for a moment, the GUI graphical elements as such would be a set of tiny icons like arrow-this-way and that-way, backgrounds, buttons, scrollbars and the like. Currently even this is a bit of a mess in that there’s no standard set at all, but a proliferation of arrow-up and arrowUp that are two different arrows pointing “up” (and this is probably what they tried to solve with the whole “styles” layer of grime on top, myeah). Nevertheless and as boring as it sounds to do it, a basic set can certainly be extracted and then forced even upon the existing widgets, certainly. The potentially troublesome part though comes from a current ugly mix of actual GUI elements (e.g. buttons) and 2D representations of game objects (e.g. a key in inventory). The currently deployed client makes no real difference and effectively lists those 2D representations as GUI “resources” like all the rest (so for all the fun, one could in principle very well set their buttons to look like Omlette Du Disgorge, why not). While I have nothing against buttons looking like omlette-du-disgorge, game objects are a very different category from GUI elements and I don’t think they should be mixed at all. Even focusing solely on the very practical perspective, GUI elements get their representations (if any) from a skin.zip file (if it exists at all), while game objects are first of all a potentially infinite set, not known upfront and even potentially changeable on an item by item basis during play. So question Q3 here would be: does this separation make sense or is there some better way to go about sorting this out? (As I’ll need to sort out this part at some point anyway, it can’t really remain as it is.)

All the above out of the way and in the open for discussion, I can happily report that the client’s code continues to shrink for now as it increases in usefulness and otherwise there are still parts that will end up most likely discarded in the end, as they get increasingly left behind (the original net part comes first to mind). While there is still quite a lot to implement on both client and serverside, I think the next steps will have to be alternating client and server development, introducing the new things (finally!) and sorting out otherwise what troubles remain, as they pop up. In other words, there’s still a lot of hard work ahead but finally getting out of the swamps and it’s such joy to be out of them too!

1. The attempt is so visible and the whole thing is so broken from so many points of view that I don’t even have the heart to laugh at it anymore. It’s one of those things that are not even wrong yet.[↩]
2. The corresponding parent classes are even called exactly that, for bonus points of similarity of names being a great thing in code: controlwindow and controlledwindow.[↩]

Wandering all alone and growing quite bored of seeing the same familiar empty space, the twisted hopeful glanced all of a sudden an unexpected shiny item sticking out of the ground and in plain view:

What could that be and where did it come from? It wasn’t there before and there’s no guarantee it will remain there either but there is now finally the possibility for any number of such items to be anywhere at all in Eulora. And now that this part is done as well, the client has finally all it needs to render -though not yet to control or interact with- all three main types of 3D things: the world itself (terrain, sky, water), moving characters be they NPCs or players (Cal3D) and non-moving items (CS meshes). In all cases, the “graphical representation” is simply a .zip file. The contents of this .zip file can change and there’s plenty of flexibility in there with only a few “must-have” things nailed down for now – even those can change later on without much trouble, since a .zip file remains a .zip file otherwise and knowing how to use what is inside is entirely a client concern.

The current client 2.0 aims to look for and use what it knows, while not getting stuck on anything additional that might be in there. So it uses currently some very simply conventions regarding filenames inside those .zip archives and simply picks up the first matching files it finds inside: anything starting with “tex” is assumed to be a texture and its extension is assumed to match the format (.dds, .png, .jpg all work fine); anything starting with “mesh” is assumed to be a mesh description and interpreted depending on the context as either CS mesh format or Cal3D format (binary or xml depending on extension); anything starting with “skel” is a skeleton file; animation files are always exactly walk.caf, idle.caf (so far those 2 are the only ones the client knows about). As examples, here’s what the current test set of .zip files contain:

• For Outdoors (the filenames here are fixed indeed as they all have very specific roles):
  • basemap.png
  • heightmap.png
  • lightmap.png
  • materialmap.png
  • norm_1.dds
  • norm_2.dds
  • norm_3.dds
  • normalmap.png
  • sky.png
  • tex_1.dds
  • tex_2.dds
  • tex_3.dds
• For Cal3D:
  • skel.xsf – the file containing the skeleton in Cal3D format. This can be either xml (.xsf) or binary format (.csf) and the client will simply use whatever is provided.
  • walk.caf – the animation to be used when this character “walks”. This has to be currently in binary Cal3D format because reasons.
  • idle.caf – the animation to be used when this character is standing/sitting still. As for walk, this has to be in binary Cal3D format.
  • mesh_*.xmf – this hopeful happens to be made out of 26 meshes so there are 26 such files, simply numbered mesh_1.xmf to mesh_26.xmf. The client will just load as many mesh files as it finds, that’s all.
  • tex.png – the texture to use on all meshes. In principle there could be as many texture files as meshes so that each mesh has its own texture. It’s not a big trouble to adapt the client to use them but at the moment it just picks the first tex* file it finds and it uses it for all meshes in that .zip file.
• For CS mesh:
  • meshfact_1.xml – the description of the mesh in CS “meshfact” xml format (NOT library format, see below the more detailed discussion why not library).
  • tex_1.png – the texture to use for this mesh. This can be in principle *any* graphical format. The current client can and will happily use .dds, .png or .jpg just as well.

Compared to the previous iteration, the “outdoors” type bundled together water, terrain and sky because in practical terms they always go together indeed. There is here one part that I’m still hesitating about: while the size of “sectors” is fixed and doesn’t need to be therefore sent (or at least not at this level), it’s unlikely that this will also match any indoor areas that are to be separate areas as such and the trouble is that the loader needs to specifically set those. One way to go about this would be perhaps to make a different type specifically for “indoors” and give all this additional information required there in the euloran object itself (arguably even the outdoors type might be better off with some specified there, if or when they may change). The main question I have here is: what is fixed about outdoors and indoors (as the cs part has a lot of parameters and very few turn out to be of any use in the end)? More specifically, given the protocol specification, the size of an outdoors sector is fixed and as a result, the client can just create a fixed-size sphere for “sky” for instance and a fixed “cell” size for the terrain and so on. Is this size fixed for indoor places too or could those be smaller/larger?

As to the latest addition from the list above, namely the CS mesh items, getting them to work included a few ugly surprises that slowed me down a bit more than I initially expected. Nevertheless, as there’s otherwise less resistance left in that paste than at the initial stages, it didn’t take all that much to whip it up into something useful. First, the format I had used previously based on existing client’s “art” files turned out to be less than ideal for a rather unexpected reason: the CS loader would happily load it but it wouldn’t provide any means to obtain directly a way to reference the loaded mesh – it assumed the caller would know that from somewhere else or otherwise find it somehow ¹. To fully get this: CS insists on referencing everything by “name” hence effectively by string-ids and the names of loaded stuff from a library file are …specified in the xml but otherwise not returned by the loader itself! So to find a factory or a mesh, the code would need to either read it specifically from the xml (ugh!) or otherwise assume it matches something else through some convention forced thus by… lack of options (because no, I don’t want to have the server now be concerned with fixing the names of meshes clientside, wtf). The solution I found to this is to not use “libraries” at all since anyway each of them contains exactly one meshfactory and then load that factory with whatever name there might be in the xml but *change* it from the code to the name that is useful to the client (and that’s …the ID of the darned object, ofc, what else, only made into a string to fit CS’s expectations, yes). It took a while to figure this out, mainly because finding the way to simply rename things is made… interesting by means of Object Oriented (OO) obfuscation layers so that the code can end up at times like this: meshWrapper->GetFactory()->GetMeshObjectFactory()->GetMaterialWrapper()->QueryObject()->GetName(). Count the layers and laugh at it but know that it’s not even the worst that OO can do!

Once the above sorted, there was some more work to cut that GEM crap on items too and then to follow up and look into all the resulting, inevitable crashes given the spaghetti pointers running all around. At least I used the opportunity to look also into the way that PS used to do “effects” – it’s a useless pile of crap on top of some tidbits of usefulness from the underlying CS, not that it comes as any surprise. The surprising part was that “labels” (e.g. names of characters) used to be… “effects” and thus bringing in the whole crashing monster of stuff. Well, labels presumably will be useful so they are on my list to add as part of sorting out the 2D part of the interface but I refuse to bring in for this reason a whole set of useless “managers” and corresponding plugins and 1001 classes that do the same thing only now it’s called labels and then it’s called shadows and then again it gets its name changed so that a different “contributor” can sign at the top of the file. Such contributions that one is way, way better off without any of them.

At any rate, having now sorted non-moving items as well, the next step really has to be looking into the 2D GUI stuff more closely, hence… PAWS. There’s no estimate I can really make (only heavy guesstimates) on how long it will take to make *that* into something useful but I’ll keep to my heuristic that served me very well so far and I’ll write up whatever stage I got to, whenever it needs unloading and certainly not later than in 2 weeks’ time.

1. How? well, that’s I suppose exactly how one ends up with all those .xml files just to tell what is in those other .xml files and then load some upfront and expect they know it all, too. Like will all madness, it’s not that there’s no logic involved, there is in fact plenty and at each step – it’s only that the root is entirely misplaced. And yes, I am fully aware that those perceiving the roots of my own work as misplaced will therefore assign the madness label to my own work and my own self. A label though never did anything though, nor will it ever do by itself.[↩]

There’s still quite some work to do before having a fully working client as well, but I can happily report that the most important part for that auto-updating version of the client is indeed in place and working: an initial prototype client 2.0 can now start happily with nothing at all (no keys, no graphics, no knowledge of the world at large) and then generate keys, obtain an account, retrieve (and check) sector and model files, load them when it got them, even let the resulting euloran move about and admire its reflection in the wonderful water by the rocks:

For future reference and current unloading of a fresh set of troubles sorted, here’s the list of main updates done on the clientside:

1. EuCore (data acquisition logic, Ada):
   1. Setting RNG source via config file: whatever string the user provides in there (e.g. “/dev/urandom” or “/dev/ttyUSB0” or “/silly/ass”) will be used as source of “random bits”; NB: the indicated source, whatever it might be, is considered already initialised in whatever manner is required, since it makes no sense at all to try and initialize from EuCore’s code a source that can be set to anything at all. For all the apparent “easy” label on this additional knob, there was a significant headache to make it right, since the RNG source is used from both C/CPP code and Ada code so that it has to be set across language-borders, not to mention in the precise direction previously-avoided-to-keep-things-neat-and-less-confusing. Not that I can recall *any* time when I got to keep over long term any of those parts that were easier, it’s just not my thing or something. What can I say – it’s done now and it’s working (the default is urandom).
   2. Setting a “sync” interval – meaning how often the user wants EuCore to attempt to resync its local data with the server. This is important mainly as a means for the user to find the compromise they want between generating traffic (which has a cost) and otherwise keeping their local data cache up to date (since out of date information can presumably make play rather less than ideal). Worth noting here that this “sync” interval is important for movement update and for requesting World Bulletins regardless of other triggers that might apply.
   3. Creating account and obtaining BOTH sets of Serpent keys from the server – this got a refactoring and update to reflect the clear separation between EuCore that does *all* data logic, acquisition, storage and so on (hence it’s the only part that can request creation of RSA keys for a new account for instance) and what is simply a dumb GUI – the former “client” that will focus exclusively on using and presenting data that is present, nothing more. There was also still pending – due to previous lack of clarity at least in my understanding of this part – the issue regarding the initial set of Serpent keys for communication with the client. While the protocol allows for the client to generate those and send them to the server, at the moment the client will simply keep sending request for an account until it receives those too. Perhaps at a later stage I’ll get back to this and implement also the option + knob to generate them locally and send them to the server but this can certainly still wait.
   4. Updated EuCache (the actual data storage structure that provides thread-safe access) to reflect the latest, streamlined data model.
   5. Updated the data acquisition logic to actually send requests to the server for each and any bit of missing data (be it object or file) as soon as it becomes aware it’s missing. This includes proper packing of file requests in as few messages as needed (instead of firing up the wasteful one message for each request).
   6. Updated handling of objects to fully update local data cache correctly (in short: adding those in server’s view and not in client’s view, deleting those in client’s view but not in server’s view, asking for anything missing and fully walking the structure at regular intervals, to ensure re-sync).
   7. Updated handling of file received messages so that chunks are saved as received and the Keccak hash of the file is calculated + checked against expected value at the end, deciding on whether the resulting file is kept and made available to the GUI or otherwise deleted (+ new request sent to the server, to resend it). NB: the approach here is very basic for now in that chunks are literally saved in the order received and therefore it’s enough to have 1 message lost or out of order to result in an incorrect file and therefore repeat the request. I’m sure that this can be made better – not so sure though that it’s worth the trouble to sink time to make it better *right now* and so there it is, low hanging fruit for anyone who wants to improve their client.
   8. Updated interface towards the GUI to reflect the new understanding that the actual needs of the GUI are way, way simpler than the full complexity that the protocol and more generally the world structure allows (and here’s an example of where the doing-it-all kind of made me stumble because even when it’s the “same thing”, it’s from different perspectives, of course and so I need to keep them all in mind at the same time and change swiftly between them, oh joy). At any rate, now the interface provides the sort of packing that the GUI finds useful, while taking otherwise advantage of EuCache’s data structures that reflect the actual hierarchy of the world. That’s pretty much the part that wasn’t quite that obvious for me until now – the dumb GUI is in fact way, way flatter than the actual world as it were and as such, there’s really no point to make the GUI more intelligent than it can handle. Let the GUI be an idiot that is forever concerned only with “what can I see *right now*” and that’s that, everything all of a sudden starts to fit, too.
2. GUI (the former “client”, cpp) side:
   1. Refactored the SMGData interface to work with the latest EuCore interface and provide a single point of access to all data, from objects to files. Sorted out here some unexpected issues with importing constants from Ada into C as well.
   2. Nailed down a working model for this whole GUI stuff: there’s a logic (such as it is) part that will then make use of all the paste that used to be pompously “the client”, from pscelclient to psmovementmanager and whatever. Sorting this out was a pain but the result is a very welcome insulation from the rapidly-spiralling-into-madness mess. In other words: while the mess is still there for it will take ages to fully eradicate, one doesn’t have to work neck-high into it *at all times* anymore! At times, a descent into that is still needed but I’m finally at the stage where it’s “at times” as opposed to “all the time”.
   3. Updated and refactored loader actually capable of making full use of .zip files that were acquired *after* client startup! This was such a pain to get working, as the virtual file system thing is rather poorly documented and quite clunky at times. Nevertheless, it … works!
   4. Prototype implementation of “soft reset” – essentially the PS code insisted that one needs to go through 1001 things in 10001 places to just clean up if/when the sector changes for instance but then it turned out that CS has this nice and shiny red button to …DeleteAll! And once that is done, PS can go and get stuffed since there’s not much it can do anyway. So the prototype so far aims to simply do such a soft reset if something crucial like the sector or the player themselves suddenly change – it would seem possible that we don’t actually have to end up with the sort of Windows-style-restart-the-client, after all.

Aside from the above, there are many other smaller details and issues sorted as they appeared – some of them might even further change as everything takes further shape. So far though I’m satisfied that the former client-code is finally just about paste-enough to be of some actual use instead of unrelenting pain and misery.

The next steps are for now to iron out some remaining weirds in the current minimal prototype and to further develop the actual use of data from the GUI since for now there’s really only the sector+player check, retrieval and loading of corresponding graphics assets but nothing further connected. Local movement works, of course, but it’s all local and moreover, the GUI doesn’t yet request from EuCore anything *other* than the player itself and the sector where they happen to be.

On the write-up front, there’s the structure of the .zip files for sector + cal3d models to write and publish. All is quite clear and neat there but I need to find some time for it and I might possibly do first the loading of CS meshes too.

On the iffiest client-side part remaining, the paws widgeting mess looms largest. At the moment I can’t tell if or to what extent I can sidestep it/any of it but I’m certainly not keen at all to get into that swamp even an inch more than is absolutely and unavoidably required.

As it took a bit of a hunt to still find somewhere those older and more useful editions of various reference books on Computer Graphics, I shrugged at first at the note attached to one of them – “disc included”, it said and I couldn’t really care at all whether the disc in question was indeed still included or still to be found or still working or even still anything at all. And as the books came, I took my time to look them over, noting even that they don’t make perhaps such a bad name-day companion after all and it’s no hurry and all that. But an honest shopkeeper is an honest shopkeeper and his notes can be therefore trusted, for included it turned out to be, for all the surprise that the image of this most humble disc in question may nevertheless stir perhaps in some younger readers of mine:

Included it was therefore, as you can surely see for yourself above, if not a disc but a disk(ette). For future record and for even younger readers, above depicted is a 3.5 inch floppy disk that belongs with this book from 1994, basically the most advanced and latest floppy disk model that there ever was. The memory trip it stirred insists that those old computers I still supposedly have in an attic somewhere have the drives required to read 3.5 disks as well as those able to read the older, larger floppy disks – 8 inch, if memory serves. At any rate, I would not be terribly surprised if this disk is perhaps still working too. Only I don’t quite think I’ll rush now to that old attic all those kilometres away to dust off one of the old computers, see if any of the CRTs still light up and then try it all out.

Let me know though if you ever want to try it out yourself, for good times’ sake or for anything else at all – the disk is here and it remains, indeed, included!

~ This is a mixed up article with serious pictures for easy reading and silly text for hard staring. ~

Following last week’s discovery of the missing link in CS’s capabilities (no meshing and no tesselation implemented as such), I took another dive in the ocean of words on the matter, reviewing papers and theses and software monstrosities by the boatfull ¹. At any rate, the core ideas are very few indeed: the focus is almost exclusively on either “assisting the artists” (in that dumb sense of catering to their convenience rather than to any attempt at excellency or even plain exploitation of the possibilities of what is a different environment than that of the traditional drawing/sculpting) or otherwise reconstructing surfaces from a *lot* of data that comes from scanning objects so basically yet another facet of that great new definition of success as approximating unanalyzed data.

The algorithms are mainly variations on delauney/voronoi refinement, marching cubes/triangles/particles for surface reconstruction and otherwise the whole of bones and skeletons pretty much stem anyway from Blum’s medial axis – that is indeed the topological skeleton of a shape but is also precise enough to result in practice in a lot of small bones if it’s taken as such. As it tends to happen in the best of cases, when I was just about to run out of oxygen for good on this, I finally turned out instead something quite promising and much in the same way as before – among the dull and broken “let’s tweak this to change that”, something glimmered all of a sudden, in quite the same way as Blum’s earlier beautiful paper on the medial axis – this time it was Bloomenthal’s paper ² that went straight and unapologetically to the heart of the matter, from the very first sentence: “An algorithm for the polygonization of implicit surfaces is described and an implementation in C is provided.” And wonder of wonders, what this amazing first sentence promises, the following words actually deliver, too!

The algorithm itself is just about as simple as one can make it for the task at hand: the core idea is to use cubes of equal sizes for partitioning the space, to gradually build such partitioning through step by step expansion from a starting point on the surface until the whole surface ³ is contained in the obtained collection of cubes and then to decompose each relevant cube’s face into the corresponding polygons (triangles for my use). The directions to expand to at each step are easily identified by considering the sign of the implicit surface function at the corners of each face of a cube. And taking advantage of the regularity of equal-sized cubes as partitioning cells, the different cases and their corresponding meaningful expansion can be precomputed and then simply used. Note that in all of this, the cube’s size is indeed fixed but it’s a parameter of the algorithm – basically larger cubes provide less detail and that has the advantage of fewer vertices and corresponding triangles but also less smooth resulting faces/shapes that can run into very poor approximations of the actual surface especially where it curves.

In 4 neat pages, the paper goes through a clear description of the method, a discussion of some implementation choices for simplicity, flexibility (e.g. the intentionally black-box approach to the implicit function definition at polygonization time so that one can use a continuous function as intended but also anything else really, as long as it accepts a set of 3 real numbers and provides a positive/zero/negative result) and speed (in particular the hashing of values -such as vertices- that are re-used during the building of the collection of cubes containing the whole surface), several concrete examples of some implicit functions (a torus, a jack, a blob, a combination of two tori) and a brief discussion of potential issues (especially if the implicit surface function is not C1 continuous since in this case various artefacts may appear, such as truncated edges or jutting parts).

The promised code is indeed included – if in a horrible form since it’s literally dumped in a .pdf at the end, the stuff of horrors as far as code is concerned. Nevertheless, it is *plain C* indeed, it is less than 1000 lines in *total*, even including some examples and whatnots, it is clear, commented, properly named and everything else that requires apparently rediscovery nowadays and it was not even mentioned as anything special back in the 90s. Anyways, I read it at first half-expecting it’ll turn out to be just a “proof of concept that kinda maybe works on narrowly chosen examples”. As I read, I started to realise that it might actually be for once, something useful. And then I went ahead and picked the parts that I need, wrote them down in a .c file, compiled the whole thing (imagine this, *nothing* required other than a c compiler, ok?) and then started playing around with it. Ah, the joy, the pure and proper joy of *finally* having something to play around with rather than suffer through, for once.

Despite all the wonders above, the implementation as it stands has its own limitations – although at the moment I’m not even sure if those limitations are not in fact quite useful. The most important one is that a run can fail in 2 cases: if the starting point is not close enough to the surface or if the polygonization runs out of allocated memory for the hash tables it creates.

The first trouble is a bit of the price paid for the simplicity of the algorithm: the first cube built around the starting point has to include some of the surface or there’s no way for the algorithm to know which way to expand to discover the rest. I don’t think this is much of an issue since yeah, I plan to build my implicit surface functions so that I can at least find ONE bloody point on the surface, one way or another ⁴.

The second trouble related to running out of allocated memory can in principle be addressed most easily by expanding the maximum allocated memory (though this may be a bit more involved than simply changing a number given as it’s not directly exposed as a parameter/variable as such but set within the type definitions and so a change requires careful follow up of all parts that may require change subsequently). Nevertheless, I won’t hurry to do this change because it *also* means that the resulting polygonization will have therefore *more* vertices and triangles. And in turn, it’s not all that clear that I need or even want more vertices and triangles anyway (and certainly not an infinite/practically uncapped number of them, wtf). So it stays as it is for now, I’d say.

Armed with a working polygonizer, I then looked at the eternal problem of matching the output format of one part (the polygonizer in this case) to the input format of the next part (the client ultimately but possibly the viewer of test meshes first). As the starting point for this was to generate characters, the format needed would logically be the cal3d mesh format since each part of a character in the client would have to be in the end exactly that, a mesh in cal3d format including the “influences” of those bones-that-are-not-quite-bones. Except of course, at this stage there’s no fucking influence anyway because I’m just generating the mesh, not the animation and so the “bone” is at best an entirely abstract and rather optional idea, especially while still just trying out the algorithm and the whole. So yeah, I wrote a few lines of awk ⁵ to just do *both* the cs and the cal3d “mesh” format out of the polygonizer’s output – not like it’s much bother anyway and look at that, I can further *also* do any size adjustment on the mesh at the same time, if I want to (and I do want to, for a reason I hadn’t realised at first).

The reason why I did both cal3d and cs format is that visualising just one mesh is in fact more straightforward as such rather than as part of a full cal3d model. And in turn, the cal3d format requires all those bells and whistles that are in fact just stored further calculations meant to support the animation, nothing to do with the shape itself. So let that wait until I have *what* to animate and let’s see some pics already. First of all and for comparison with the results of my polygonizer, here’s some primitives generated with CS’s own internal code that simply has a fixed recipe for a few regular shapes, of which you can admire below what should be a cone (looks more like a pyramid to me, due to how the texture got applied to it), a cylinder (seen directly in the client, through Cally’s eyes of sort) and a small ellipse (seen in the viewer). Note that CS itself complains about “degenerate faces” on the cone – in practical terms this means that the collision detection system is likely to have some amount of trouble when that object is involved:

Moving on to something more interesting, here’s a collection of various experiments with the polygonizer and implicit surfaces. For starters, here’s how a botched torus looks like – I had messed up the equation of the function and so there you go, have some funky wedges, why not:

And here’s a blob (made out by combining in the same equation the 3 parts because such are the wonders of Mathematics, yes, you can do what you want, with just an equation, how wonderful) in the extended, cvasi-infinite viewer (I had gotten fed up with the tiny stone cell and moreover, I hadn’t yet figured out that the lights were all of a sudden all wrong and as a result making everything that brownish tinge, texture be darned; sigh):

Next on the list and placed in the more familiar landscape of Eulora, a shiny swimming aidtorus ⁶ (the texture is that of some water and it was part of my attempts to figure out the textures too – that is still ongoing because of course it’s not quite working as expected, grrr). The weird handle on this torus is a bit…weird. Basically the origin of the torus seems to be considered on the surface for some reason. It’s not a big deal, in that it can be easily fixed, of course, but since it’s not exactly tori that are going to make a character, I didn’t bother all that much to track down the full backstory of how this particular wedge here happened – just be aware that yes, it can happen:

Since after a while even a torus is quite boring, I decided to see how the polygonizer behaves on an unbounded surface – especially since there were no examples of such things in the original paper. One short equation (and a few adjustments to finally streamline my process so that all it takes from generate to view is running 2 scripts & starting the client), I admired hyperboloids of all sorts (and of course they remind me of petroleum refineries, what else):

Probably you are wondering now what happened exactly in the last picture and what are those seeming spikes – is that even still a hyperboloid at all? Allow me therefore to assure you that it is indeed still a hyperboloid, no matter what your eyes may tell you. Moreover, it’s just as much a hyperboloid as the previous ones were – it turns out however that CS tends to have trouble with the hollow, thin and unbounded surfaces that I generate. When seen from some angles, they vanish entirely and I suspect the trouble is that CS expects in fact two “layers” of triangles to be able to show such a surface as expected – while my polygonizer provides the outer surface of the thing, it does not add to it an inner surface so the “inside” is …invisible. Is this a problem worth addressing right now? I’d say it can wait for now, even though indeed, look at what mess it makes of what is effectively the simplest “wrap” around an imaginary line-“bone” aka a …cylinder (if defined implicitly, of course), what:

The above experiment with unbounded shapes set my mind to rest that yes, indeed, the polygonizer can handle those. It also revealed more clearly the need to tweak both the two parameters of the polygonization depending on the shape (ie the size of the cubes and the maximum number of cubes to explore on each side) and the size of the shape after this step. This is because the *size* of the resulting shape depends on those two parameters too. And in turn, the *part* of the shape obtained will depend on them so basically one might need to tweak more than it seems at an initial naive look. Nevertheless, those are very easy tweaks really and at least so far I don’t quite see much trouble with them: I’ve already changed the code to just accept as parameters the two parameters for the polygonization so I don’t have to recompile it every time those change; and the script porting the stuff to cs & cal3d gets its own parameter to apply a scale factor as desired so that the *resulting* mesh is exactly the desired size; put together, this means that I can in fact create meshes as big as it might be required to capture nicely whatever details and *then* I can painlessly shrink them too, without loss – obviously, within some reasonable limits since if I shrink everything to a single point, there is arguably some loss of detail, indeed!

As for some fun I had after sorting out all the above, here’s a very pointy implement for all your impaling-in-walls needs! It’s defined of course through it’s implicit function or in other words, you can certainly impale yourself in an equation, no trouble:

The next step on this is to sort out the textures already: the trouble being again a bit of a missing link since CS expects that the texture is an image file rather than an equation/function. I explored already to some extent what I could do there – theoretically there’s a way to “write” the texture at run-time ie directly into the render buffer of the thing. But all experiments and even CS’s example on this worked in the sense that yes, I can write whatever I want directly to the screen – unfortunately, directly using the screen’s own coordinates too rather than the object’s coordinates, which is rather a pain (and doesn’t quite make sense if I’m supposedly writing to the object’s buffer as opposed to a *global* buffer, wtf?). Alternatively, I found also that CS conveniently has even a way to write a texture to an image file but then of course, I don’t know how to write & especially how to map such a file to the vertices of a generated surface so that it’s neither just a stretched thing nor a tiled thing – the way the mapping of the texture works exactly is quite a pain to fully get because the “explanation” is not helping me much with understanding the approach itself, sigh. Nevertheless, I expect I *have to* figure this out and quite in detail at that, sticking textures for the win and all that. And then of course, I’ll need to do and stick the Mandelbrot, yes.

Once the above is sorted, I guess the next thing would be to write the surface function a bit more interesting than the above, generate it for more than one bone too and then – the UGLY part – do also all the precalculations and futzing about that the cal3d format requires in order to put the shapes at the right place, sigh.

I expect the above will still take some significant time, there doesn’t seem to ever be a very fast route for what I need to do – or not one that I can yet see.

1. I won’t name all of them, nor even spend more time to do the full write-up, at least not for now. It’s not that I can’t find what to say or that I don’t have a pile of notes on them anyway – it’s simply that I’d much rather not go through that pile of notes again, if at all possible, thank you. To unload though for now, the names I’d mention at all are quite few anyway and go in a rather obvious order from the fundamentals to petering out on the edges: there’s Blum’s medial axis (1967); there’s Bloomenthal’s work on skeletal design of natural forms (1995), including the algorithm I’m using so far; there’s Shewchuck’s work -starting with his thesis (1995) and building up from there- on Delauney refinements for mesh generation and his useful lecture notes at least; there’s Persson’s thesis (1997) on mesh generation from implicit geometries, which includes the attempt at mesh generation based on a truss-model; there’s a readable book by Edelsbrunner from 2001 on Geometry and Topology for Mesh Generation; there’s perhaps some of the earlier work (2001 and earlier) by Amenta on approximating a surface (“power crust”) based on the medial axis transform and a union of balls – effectively an approximation of the medial axis itself. I’ve added the years for a reason but by now I’m quite satisfied I have obtained a reasonable map of the domain both in names and in time coordinates really – if you want something usable and useful as opposed to something big, unyielding, overly complex and utterly narrow in its application, your destination has to be the ’90s at the very latest.[↩]
2. “An Implicit Surface Polygonizer”, Jules Bloomenthal[↩]
3. If the surface is bounded – like a torus say, this is obvious enough. If the surface is unbounded, then the “whole surface” is simply defined based on a bounds parameter provided as the maximum number of cubes to explore in each direction from the starting point. Simple, effective and to the point, it’s as if the author actually wanted to make something truly useful, you know?[↩]
4. There is of course also the fact that any run will polygonize a connected surface ie not disconnected parts but I don’t quite see this as any sort of limitation really – wtf is that “disconnected surface” anyway, if it’s disconnected then it’s made of 2 parts and if it’s made of 2 parts then each part can be treated as one surface and look that there’s no problem anymore and we can actually get something done instead.[↩]
5. By now, I almost have a sneaking suspicion that if I ever get to actually play Eulora myself as opposed to programming its server and its client and its graphics and ~everything else, I’ll play it in awk already.[↩]
6. And I can’t look at that thing without instantly hearing this song of Ioan Gyuri Pascu:

   Mi-au zis baietii:
   Vezi ca fata asta-i
   Un element inabordabil
   N-ajungi la ea, oricate precautii
   ti-ai lua in prealabil
   si eu, care nici macar
   Sa inot nu stiu
   Ce sa fac?
   Pentru un prim pas
   Musai sa-mi cumpar un colac.

   Mi l-am si cumparat
   Cu grija l-am umflat
   Usor in apa am intrat
   Si mandru am cantat

   Hei tu, mi-am luat colac
   Sa vezi acuma cum devine marea un fleac
   Hei tu, mi-am luat colac
   Sa vezi acuma cum devine marea un fleac

   Scena de film:
   Eu daruindu-i cu amor
   O stea de mare.
   Si m-am trezit
   Cu masca de oxigen
   Pe fata, la reanimare
   Ea, doctor in halat
   Cu o seringa imensa in mana
   Mi-a zis: halal colac baiete
   Noroc cu medicina romana!

   Acum ca te-am salvat, mi-a zis:
   Mai canta-mi inc-o dat’
   Refrenul acela minunat.
   Normal ca l-am cantat.

   Hei tu, mi-am luat colac
   Sa vezi acuma cum devine marea un fleac.

   I-am zis: stiti domnisoara
   La un moment dat
   Credeam ca
   M-am inecat in mare
   Habar n-aveam ca aveti
   O privire asa cuprinzatoare.
   Mi-a zis: hai lasa textele
   Galanteria nu-i intotdeauna un atu.
   Mai bine ofera-mi floarea de pe masa
   Ia-ma de brat si spune-mi tu.

   Si brusc m-a sarutat
   Apoi s-a maritat,
   Bineinteles, cu alt baiat.
   De fapt, am inventat.

   Nimic, nimic din ce v-am cantat
   Nu are baza
   Desi, desi tare, tare
   M-as fi bucurat.

   Ma rog, asta-i muzica,
   Frumoase lucruri
   Regaseste omul in ea
   Da, da, da’ ce frumoasa-i muzica
   Mai ales cand omul regaseste
   Atat de multe lucruri utile, educative,
   Frumoase si morale in ea, vai!

   Nimic, da’ nimic, nimic
   Din ce v-am cantat
   Nu are baza
   Desi tare, tare
   Si va spun, pe cuvantul meu
   M-as fi bucurat.

   Ma rog, asta-i muzica
   Pe cuvant, atat de inaltatoare
   Lucruri gaseste omul in ea
   Va spui, eu, vai de mine…
   Asa-i de frumoasa
   Si de educativa…

   [↩]

Figuring out the way to make default Cal3D sprites (aka animated characters) and their animations work again in Eulora’s client turned out some unexpected pockets of sanity in CS/CAL3D, once the pile of rotten PS on top of this particular bit managed to finally get on my nerves to such extent that I fully sidelined it – if not yet removed as such (mainly because I didn’t spend the time to fully follow what else might depend on it *being there* for the time being). Nevertheless, the overall evaluation of this part as likely to be more time consuming and iffier than previous parts remains valid – it took this whole past week to figure out quite important parts of it and although I have a whole pile of findings to unload here so that I can move on more easily, there is still a lot more that needs to be done and none of it looks in any way easy or even all that *clear*, no matter how you squint or glare at it. Leaving the squinting and glaring for the very end though, I’ll go first through the illustrated guide of what works so far and what I added new since last article in this series.

Aiming this time to set in stone a default character rather than just have something to move about, I looked more closely at what concrete options there are currently – in the deployed client as well as in CS and Cal3D. The two Euloran characters (male & female aka Boxy & Foxy) have a few advantages, namely that they are made for Eulora and that they have some items of clothing that suit them reasonably. Nevertheless, both the characters themselves and the clothing are not really suited for any defaults, because they basically are too monolithical to start with – the body/clothing parts are already linked together so that one can’t switch/mix/match as required for various changes. Moreover, there are only 2 animations so not much scope to even test all sorts and – even more surprisingly to me – the binary files failed to convert to non-binary with the default converter that Cal3D ships with. So far I left this with a question mark next to it and didn’t investigate further since it’s not totally clear that there is much to get from getting to the bottom of it right now.

Given the above lacks of the current euloran models, I turned again to Cal3D’s test character, Cally, who has at least an abundance of body parts – some of them extremely perky parts, too. Compared to the euloran characters, Cally is huge though (100 times bigger!) and moreover in a different initial position but such differences are easily corrected with basic parameters when loading the model’s files: the code can rescale, rotate a model as desired and set flags regarding the position so that further transformations don’t end up flipping it all back to the original undesired stance or something ¹.

A few unexpected surprises came from the sidelines as they usually do. First of all, CS is incapable of handling properly collision detection when animated sprites are involved so it wraps instead everything in bounding boxes that are big enough to contain the character fully at all times – basically the collision happens with a cube of personal space within which the animated character moves. While currently there’s little concern from this, it still sounds rather silly to me and it will probably make for relatively weird char to char fights and the like but each problem when its time comes and all that. Second, while Cal3D has its own format for materials, it turns out that CS can NOT load that format and therefore one HAS TO use CS materials even for Cal3D sprites! So to get it going for the time being, I simply borrowed Foxy’s skin and set it on Cally – since it’s anyway a basic texture otherwise, it turns out that skin sharing and borrowing works just fine at this level:

Since skin otherwise comes with the eyes in it too ² but you wouldn’t be able to tell that if you looked at Cally, I tried on as well other materials – namely the ones otherwise used by the current client for Foxy’s dress and Boxy’s jacket. They simply turned gray and blue on Cally and I suspect quite a large part of it has to do with size and lack of specific cloths shader perhaps, but it’s not something with high priority right now – perhaps later on when I actually have some sort of clothing to speak of. For now, here’s Cally with gray (dress material) and blue (jacket material) skin, respectively:

For my peace of mind and thorough investigation of character’s skin at this stage, I did try on the character-specific shader (there is one – it focuses mainly on lighting) but it made ~0 difference on all three different skins so I took it out again and stuck to the basic material with one single texture. The main reason I tried this shader specifically is mainly the fact that Cal3d’s specification of materials focuses as well on lighting – basically it defines light-reflecting properties of the material and since CS doesn’t import them directly, I wondered for a minute if they’d do anything specified the CS way aka through the corresponding shader variables – empirical results suggest they don’t really do a lot really, just tiny adjustments that seem hardly worth the whole mess otherwise.

Before moving on to the animations themselves, it’s worth for my future self to set down clearly in here the way Cal3D sprites are made and managed within CS, since the documentation is extremely brief to the point of missing in parts, skipping details otherwise and generally giving only the bare minimum at most. CS essentially wraps Cal3D’s own classes so that it provides an interface that is at least similar to all the other meshes and factories and similar graphical elements. Moreover, there is a way to access directly the wrapped Cal3D model but apparently that’s more of a rake set at a good angle to take your eyes out with, since it comes with the clear statement that modifying the Cal3D model directly comes with a near guarantee of getting it out of sync with the rest of what CS knows of it and therefore making it all a huge mess. The only way I can see this is as a sort of passive-aggressive programming – I never even knew such a thing was to be a thing, what can I tell you.

Leaving aside the direct access that is unlikely to help otherwise in the least, Cal3D provides a factory that is meant to store *all* available & possible parts and equipment for a whole class/type of character ³. Seeing this in the best light, the Cal3D factory would contain all the lego-like parts that are compatible with one another. The best light is not exactly what one gets in practice though, of course, but at least the intention and potential are there, let’s say ⁴.

The very core of a Cal3D factory is the skeleton – basically all instances made of the same factory will share a common skeleton structure – everything else can in principle be different from one instance to another, but not the skeleton. On this skeleton, each instance can show a set of component meshes (basically body parts) and attached meshes (basically stuff, be it tools or cloths, that is attached via the sockets mechanism – each socket specifies the exact triangle in the host mesh where another mesh can be attached; this requires of course the host mesh to be defined through triangles at the very least). Both component meshes and any attached meshes that are equipped at any given time are selected out of all those available in the factory. Each instance of a Cal3D factory aka an actual character active in the game has its own state and can therefore keep track of its own set of meshes that are active at any given time, while also being able to switch meshes at will – in theory as I haven’t yet gotten around to see this in action, mainly for lack of *alternative* meshes for the same place.

On the sad side, I have to note that all my attempts to actually use existing meshes as parts of an overall puzzle game failed miserably – basically *any* use of a mesh part from one model to another resulted in a segfault in the Cal3D lib. I’ll probably have to investigate this further to get to the bottom of it in the sense of figuring out what is required exactly to produce “matching” mesh parts – and what makes them incompatible to this extent – but I suspect it all boils down to the bones they reference really, meaning that the skeleton itself will likely to have to be the main fixed part around which perhaps the rest can be allowed more flexibility.

The skeleton is so crucial to a Cal3D sprite mainly because everything depends on it, from component meshes’ position to animations that represent actions and/or movement of all sorts. The skeletal animations list the ways in which bones affected by the desired movement change position. In turn, bones influence in principle any connected/attached meshes and may also cause deformations/influence cloths or the like – though this part certainly needs more investigation and testing as at the moment there are no examples to look at nor any clear documentation as to how exactly everything works there.

At any rate, the skeletal animations as defined for instance for the Cally model work perfectly fine as expected – once PS is taken out of the loop and the velocity interval is set correctly when each animation is loaded. Cal3D activates animations based on the velocity of the sprite at any given moment and the velocity intervals in which each animation is meant to be active. In principle several animations can therefore be active at the same time and Cal3D claims to be bright enough to blend them smoothly and taking into consideration priorities too – though I haven’t yet seen this in action. For some basic animations, here are some screenshots of Cally in action strutting about the island, walking, waving and in mid-kick (note that the animations in game look even better than in the stills – sometimes the screenshot was basically not quick enough and got slightly messed up in between frames):

In addition to skeletal animations, Cal3D claims to also provide support for morphanimations. As far as I understand them currently, morphanimations would be required for more fine-grained movements/changes to a model, especially those involving parts that don’t rely on skeletal bone movements as such – in practical terms, facial movements and expressions first and foremost. At the moment this is a big unknown since not even Cally has such advanced stuff and so I guess I’ll have to… just figure it out, oh joy. On the notes-to-future-self side, there is some reasonable relevant theory writing on this, especially with respect to a supposedly robust way of reusing morphanimations across different models: the core idea is that there is a “neutral” pose and any non-neutral poses are defined by subtraction from this one so that the actual morphanimation can then be adequately calculated for any model based on its own, specific, “neutral” pose. While I haven’t yet reviewed this in enough detail, it would seem to me that a similar approach could perhaps be used for skeletal animations too – a neutral skeletal pose as reference for animations described as changes to *that* rather than as tweaks to a particular set and setting of bones.

For the next steps on this, I’ll have to figure out and somehow test practically the socket mechanism at the very least. Given however the desired subquest on graphics production, the socket mechanism by itself is certainly required but not sufficient – what I’ll have to do is to extract/decide on what makes the essence of “euloran”, fix that and provide acceptable (as in close enough to be natively recognizable but diverse enough to be also identified as distinct) ways to change/mix those. While some ideas on the topic are pushing around at various boundaries, the gritty part still has to start with digging deeper into the skeleton, meshes and morphanimations definitions, formats and use as provided by Cal3d since that’s the practical starting point currently – at the very least, I clearly have to figure out what exactly is required so as not to crash in segfault the whole lib whenever replacing a part with an “unexpected” mesh.

Given the above, I plan to continue with the focus on the characters part of the client data hierarchy unless it’s considered otherwise a higher priority to get the remaining missing parts back in first, even before sorting out some sort of way to cloth and diversify the characters.

1. This role of the flags and their interplay with applied transformations otherwise is what tripped me over previously and had caused the dive-in of the model when moving about. Yet another puzzle solved, I suppose.[↩]
2. CG is gruesome in more ways than one, yes. Still, I’m just telling you what is there, not like I made it *that* way, all right?[↩]
3. Doesn’t this strike you already as such a deeply held conviction that knowing it *all* is always possible and even required and all that? I’m not sure how it can sit otherwise so apparently unchallenged right in code and on computers but there it is.[↩]
4. The only thing I can say on this is that apparently I really am an incurable optimist, no idea why and how come. I can only shake my head in disbelief.[↩]

Having explored those past few days all the possible ways to get various aspects of air ¹ showing up in Eulora’s client, I’m left with a working “air” element of sorts, a bunch of screenshots (scroll down for them) to show as illustrations of the whole exploration and otherwise a lingering question of little import to computer graphics: if it’s indeed a sphere but lacking any atmos, can it still be called an atmosphere?

Unlike the terrain and the water that I sorted out already, the “air” category doesn’t have just one single element since CS and the current client don’t bother with such lofty ideas as actually modeling anything. There are instead all sorts of bits and pieces that can be pieced together to make it seem like there is some sort of atmosphere: some sky, some clouds, some weather, some fancy potential effects of sorts and some ways for light to travel from the sun or any other source. So the rest of this article is a look at what I found out when taking each of those in turn and looking in practical detail at how and to what extent they can be made to work at all.

Light

To be able to see anything of the world at all, CS requires at the very least an amount of “ambiental” light (so light without a source as such, supposedly a sort of stand-in for the reflected light as it bounces off everything around) and a (some) source(s) of light overall. On one hand sources of light are more aptly items/structures in my hierarchy of objects and as such not really belonging here but on the other hand, the way the client works currently, there is the need for some “sources of light” that are simply environmental characteristics rather than items. So I went here with copying whatever such lights are defined in the current client: some ambiental light and 3 “suns” positioned through unknown methods in a way that “works” in the sense that you can see the stuff around and there is some apparent direction from where the sun shines or something. It’s a mystical light basically and it will be fixed as default in there, nothing more to do about it as far as I can currently see, since it’s certainly not the server’s job or interest to fiddle default lighting so things are visible given one rendering or another.

When talking of lights, it’s worth noting that the above fixed set of lights is not in fact the full story in the currently deployed client. The intensity and exact colour of light is made to depend on the time but at the moment this is not imported into the prototype client and I’m not fully sure it’s something worth importing as such: the way it’s done, there’s an xml file (art/lighting.xml) that lists *for each hour of the day* the values (r,g,b) of various lights (identified by their fixed name…) in different sectors (again, identified by fixed sector names). Some manager or another in the PS client code proceeds then to use those values to adjust each light periodically, to match the time of the day, interpolating between the values given for the current and next hour.

Besides the clunkiness and clumsiness of the above approach, there’s more trouble with importing it as found, with the most important problem being that the assumption of fixed sector names simply has no place at all in Eulora’s client. Moreover, there’s no good reason I can see for having this sort of hour by hour list in an xml if it’s basically a set of constants anyway and entirely client’s business too. So at the moment, it just remained outside entirely: in Eulora’s default client it’s always somewhere in the middle of the day, when things are quite visible. As a player I’m not even sure why the fuck would I actually *want* to have light dimming so I can’t see things properly but then again, it’s been already observed that I’m a sort of software radical, especially when dealing with graphics, so let me know if you have some reasons for which I should really spend some time to implement time-related changes to the overall light (and then possibly getting in a night sky too, I suppose, see below for the sky and clouds section).

For an illustration of the light conditions, here’s a screenshot from a “shaded” spot so you can notice the light and shade divide as it happens with the current set of ambiental light and three suns:

Sky and Clouds

Looking at the currently deployed client in the usual places (art/world.zip/island.xml for the factory+mesh; then off to art/meshes.zip/sphere.xml for the definition of the factory; then art/materials.zip/materials.cslib for the definition of the relevant materials) reveals that the euloran sky is currently a generic mesh defined painstakingly through enumerations of vertices and corresponding texels (texture coordinates at each vertex), normals and colours (which are all set to 1 and probably of no use since the texture is added on top anyway but hey, it has to be there so it’s there) as well as a whole set of triangles to piece up whatever shape the whole thing ends up to be (possibly an approximation of a sphere if the name is to be trusted indeed but I haven’t really bothered to check). Such a brilliant way of describing the thing that there are – line numbers for the win! – about 2050 vertices and when you multiply that by 4 and add on top the list of triangles and a bit of xml tags too, the file ends up having a bit more than 8200 lines in total – just because it can!

While I could easily just use copy/paste to get the above horror as constants in the default code, I find the idea so abhorrent that it’s just about the last thing I’d do – if there really was no other practical way to get some darned sky showing properly at all. Happily though, there is a more reasonable alternative in this case: with a bit of fiddling of parameters, it turns out that CS’s code for generating programmatically a sphere can do the job quite fine: the fiddling was more a matter of figuring out the exact number of rim vertices that would fit the existing texture well rather than anything else. So instead of littering the code with those thousands of lines of constant numbers describing a sphere, it works sanely for once to simply call the code to *generate* the sphere a be done with it! I have no idea why exactly wasn’t this done to start with but I am not even all that curious to find out really – what difference will it make anyway?

While the sphere – programmatically generated or boringly xml-enumerated – provides the so-called geometry of the “sky”, the rendered image is as usual a texture. Aiming to figure out what options there are and how they work, I made a few attempts to see if the shaders provided as “sky” and “clouds2” are of any use – it turns out they aren’t, at least not as they stand undocumented and without wasting a lot of time to get to the bottom of everything so that at least ~something shows up working. From what I could tell and for the very little that this is worth currently, the clouds2 shader aims to programmatically animate the texture so that you get supposedly moving clouds. Nevertheless, the default client will *not* get moving but fixed clouds, as already neatly painted in the image file that serves as texture for the sky cloud. Anyway, as part of the testing, I have tried out a different “sky” texture that I found in the “art” directory of the current client and the result was a rather weird-looking sky-thing:

Getting back to using the usual clouds texture works though and the sky looks reasonable again, even reflecting quite well in the water – along with Foxy’s long and slender-by-water-effect legs, of course:

Effects

Since one might reasonably want some pretty sparks or something around, I went ahead and implemented in code the sparkly sphere that is currently defined in xml and shown on the beach in the client 1.2. It is using a particle systems and it goes on to tediously set quite a lot of parameters for the effector especially but with the right shader applied as well, the result can actually turn out quite prettily indeed, with both the “snowflake” texture and the currently used “spark” texture:

Weather: Rain, Snow and Fog

Moving on to weather effects, there’s fog as the simpler part and then rain and snow that are just variations on putting together in some predefined manner some particle systems. As my previous attempt at sorting out the particle systems was rather frustrating and with dubious results, I started with them first. As it happens, my suspicion that the poor effects of last time were linked to not using a shader/the right shader incantation turned out to be correct. Combined with the fact that meanwhile I ate – if rather reluctantly – shaders for breakfast too, it didn’t take all that long to obtain some pretty snow for instance:

The shader used by both snow and rain is “particles_basic” and one remaining wtf that I had to sort out – to the degree possible – before being able to obtain the pretty picture above was the meaning of “type of shader” as set in code and otherwise not really mentioned or listed anywhere. What I finally figured out is that there is a set of predefined types (which is however nowhere properly listed as such as far as I could tell and goddammit I did look in the code and in the manual and in the api and on the dark side of the moon too) and those in fact are linked to – but not named as, nooo – different rendering passes! So that you can use the right shader but if you use it with the wrong type aka in the incorrect rendering pass, you get bupkiss result and can go bang your head some more as there’s no way to tell exactly why. This is by the way the solution to last article’s puzzle re using the same shader 2 times only set with different “types” – it is indeed the same and will indeed be run twice only in different rendering passes so the effects accumulate/interact in whatever way they do it (in some cases there are even values passed between successive rendering passes for instance…). I suppose I should be happy I solved my own puzzle or something.

Leaving the above rendering passes mess aside for now, it further turned out at a closer look that this glorious particles_basic shader does precious little really to the point that the observed effect is simply one of …making it work well with textures that use black for “transparent” instead of having an alpha channel. Doesn’t this sound though like a truly mind-boggling reason to use a whole shader for? Because to me it does and to such extent that I couldn’t just let it be – so I took the silly snowflake texture file and made the equivalent image only with transparency properly set through the alpha channel. After which, I made it snow again only without any bloody shader in the mix and lo and behold, you tell me the difference:

The even funnier part of the above great shader use though is what happens therefore if you do both aka you use the shader (that worked with texture having black stand for transparent) AND have transparency in the texture itself – it’s snowing translucent cubes!

Both rain and snow are painstakingly set in code with all the emitters, effectors, material and shader(s) + shader variables, as quite specific to particle systems. The main difference is with respect to texture used (since a snowflake is meant to look different from a water drop) and otherwise the particle size range. The shader does exactly the same sort of not-much as for snow but I haven’t bothered to add transparency to the water drop texture too so here’s how the current texture looks with and without the shader applied:

Given that the main difference between rain and snow really is just a matter of texture, it’s probably possible to simply fix the particle size and a lot of other parameters for the “effectors” and “emitters” and whatnots and then just have one single default “rain” that can then rain down even frogs if that’s the texture given. However, it turns out that there’s some deep trouble afoot with the current implementation of particle systems in CS: first, they are so inefficient that extending the rained upon area to the whole island basically doesn’t quite work (there was an initial freeze while CS calculated whatever it calculated for it all and then it was still visibly suffering while pouring all that thick rain around); second, whenever rain and/or snow code is in, the exit ends up in a segfault – as this is many levels deep in CS, I haven’t really spent time to investigate it and find the exact culprit but it seems to be related possibly to the specific emitter that rain/snow use since the trouble does not manifest for instance for the “sparkly globe” effect (see the previous section above) that uses a different emitter.

The fog is simply a property of the sector as a whole – there is no “local” fog even possible, unless you make perhaps your own as a particle systems I suppose (could have fog-particles perhaps?). One can set the “density” and the colour of the fog (r,g,b) – basically the effect is similar to putting on a pair of glasses tinted with the chosen fog’s colour and having thicker or thinner lenses depending on density. There is also a further “mode” parameter that chooses -as far as I can tell- the way in which exact fog values are calculated in the distance with the available “models” being called CS_FOG_MODE_NONE, CS_FOG_MODE_CRYSTALSPACE, CS_FOG_MODE_LINEAR_CRYSTALSPACE, CS_FOG_MODE_EXP, CS_FOG_MODE_EXP2. The only one I could find used at all seems to be CS_FOG_MODE_CRYSTALSPACE but at any rate, I made some experiments with others and there is either little visible difference or the whole thing looks even worse than it does otherwise. Moreover, I can’t say I see exactly any reason why I’d want fog in my client as a player but do let me know if you still think it should be in there – and in what form exactly. To get some idea of how it looks like, here are some pics of various “fog” experiments:

The bluish fog (r=0, g=0.1, b=0.1, density=0.02):

At one extreme, the fog so white that it…shines in your eyes:

At the other extreme, the fog so dark that everything turns into a black and white disco style:

Still shiny fog but toned down a bit:

When fog is more like darkness than like fog:

Possibly almost reasonable – if rather graying – fog:

Conclusion and Next Step

Given all the above, I’m overall satisfied that there is now some working default “air” for Eulora’s sectors even though no depiction of weather or of day/night changes. The shiny globe effect seems to work surprisingly well so far so I’ll keep it still on the list and possibly have another look at it after sorting out the items and characters to see if/how it can be attached to an item or to a moving character, if that is something of interest.

The next step for now is the potentially more time-consuming and iffier sorting out of a working default for characters with animations and all that included so there will likely be a longer wait until the next article in this series.

1. No, not various airs and not even various arias – though these might be in waiting when I get to look at sounds.[↩]

Having set Eulora’s soil in stone (and grass and sand), I turned my attention this week to the waters that should naturally be found around an island – if it’s meant to be any sort of island at all. You know, those deep-deep euloran waters that hold all sorts of resources and unexpected benefits.

Last time I looked at water generation, I considered it had to be nothing other than some specific use of the procedural texture that CS supposedly provides for water rendering. A closer look at the corresponding CS plugin code ¹ revealed however that it’s yet another very specialised thing to the extent that it comes with methods for making “puddles” at a given point – supposedly what the surface should look like if you throw something into the water. At the core of it though, the whole “procedural texture” shtick is an animated texture aka a set of direct writings/changes to the render buffers using the engine’s “ticks” as time units to play the desired sequence. Given this rather disappointing find, I don’t quite see any strong reason to import into SMG’s protocol those “procedural textures” as they are currently in CS, they seem to be utterly focused on client side rendering again and as such entirely client’s concern and choice.

Having set to rest my hopes of procedural water generation, I turned to the existing client’s pile of .xml files to extract from there just what and how it defines this water thing anyway, since the whole idea is to use whatever is currently in use, right? So in I went and here’s what I gathered:

1. From island xml file (in art/world.zip), I found out that the “water” is but a mesh like any other so an “item/structure” in my hierarchy of data, only using a custom factory called “water”.
2. Following the factory clue, from the water ² xml file (in art/meshes.zip) I found out that the whole of Eulora’s water is in fact but a 2D rectangle, a sheet of blue if you will, cutting through the whole world to show only where there’s nothing else with higher rendering priority! And moreover, the whole thing is simply defined in the factory through its vertices, normals, texels (texture coordinates) and triangles – the minimal thing overall for a rectangle so that’s 4 vertices (with corresponding normals and texels) plus 2 triangles. The whole illusion therefore has to be achieved through the thin veneer ³ – aka material used – on top of this rectangle.
3. In search of the material therefore, it was off to art/materials.zip and the materials.cslib file within. A look inside of it revealed that the material “water_02” (as jotted down from the factory’s definition) is a thing with 2 shaders (set as different types but using the same… shader; does this make any sense to you?) and 6 shader variables of which one is a texture but not the base texture but the normal texture (aka it’s used for more precise definition of normals across the surface rather than as painted skin to spread upon the surface).
4. The “normal texture” from above turned out to be indeed defined simply in the same materials.cslib as a file that even exists where one would expect it to be (in art/materials.zip) except… its extension is missing, its type is actually unknown (it’s not .ani as one might guess from the name, it’s not .png, nor .jpg, nor .dds for sure) and the current client in fact complains loudly at this, if anyone ever paid attention: it can’t load that “generic_water_001_ani” thing no matter what. So…uhm, ditch it then and use some reasonably named “w_normalmap” since it is meant to be a map of normals after all.
5. As for the shader with multiple uses, a look at it indicated that it’s indeed the part doing in fact all the heavy water lifting so it got added to the list of defaults that the client loads.

With all the above set, the rest was simply a matter of writing the code to create then the default water precisely as defined in the current xml, only using CS directly from code and without requiring any xml at all. There were a few funny steps on the way including the water with error texture when I had to test that ani file just in case it could be loaded somehow:

Then the part where Foxy walked on water because the xml had this tag “colldet” that you might be forgiven to think it means adding collision detection as silly as that may sound when you render water – of course it does not mean such a simple thing; what it means is that you should *toggle* collision detection on if it’s off and off if it’s on and of course you just…know what it is anyway, right? It’s also with a different water “normals” texture that I tried for testing purposes and it doesn’t even look all *that* bad although the clear repetition is rather grating to my eyes:

And some funny legs reflections that are for now still in there, mainly because apparently that’s how Foxy’s current shape ends up reflected currently:

Finally, here’s also an even reasonable shot of Foxy getting out of the water on to the Euloran harsh and unforgiving sand:

An interesting thing to note with this water-plane approach is the fact that it’s currently… in the way of light at times so it introduces weird “dark” effects in places on the island. For the time being I’m not too bothered about it since it might anyway end up sorted when I look into default lights and moreover, it’s again both a client-side concern only and the exact way in which the existing client works anyway. This being said, I still think this whole water-plane thing is silly – especially given that once you get inside the water fully, there’s of course just… air aka “no water inside the water”. Ain’t this just a great illustration of how modern stuff turns out to be at a closer look, various sorts of no water inside water, right?

1. The main bit is in cs/plugins/proctex/standard/prwater.cpp if you are curious about it.[↩]
2. There are actually TWO files in there, namely water and water2 but as it seems to be the usual case with the existing client, one is just not used currently at all.[↩]
3. At which point my mind sniggered into song, of course, as this must clearly be what most people mean all along with all the heartfelt “river deeeeeeep and mountain hiiiiigh”; it fits this sort of water and this sort of deep, heh.[↩]

Less than 2 weeks after I’ve set Eulora’s client down, the resolution came of course to… pick it back up! Nothing like a bit of up-down to hm, build muscle? Anyways, there’s nothing lost for the exercise and quite a lot gained really since I got some time to unload more than just the code ¹ and the resulting discussion sets a clearer way to move further:

mircea_popescu: anyways, to complete the frame here : http://trilema.com/2020/get-back-to-me/ eventually was resolved through discussion here, specifically : 1. that the expectation we’ll have to produce a data model constituent part of the original dilemma was in fact produced by 2. an unstated assumption that we’ll resolve the reuse-or-rewrite dilemma in the sense of re-write, 3. driven in turn by the felt pain of dealing with the damned thing as it exists.
ossabot: Logged on 2020-01-27 11:26:47 diana_coman: the way I see it, there are 2 main ways to get the gfx back in: a. use what already works, hence client 1.2b b. implement own aka go from the existing prototypes in 2.0 to production stuff so yes, including non-stiff etc
mircea_popescu: but the correct decision is to reuse, on the grounds that however painful we might be aware it actually is, in the most organic sense of awareness, nevertheless the alternative is actually specifically the sort of thing we’re trying to avoid.
mircea_popescu: as in, the ever-circling avoidant behaviour of great recent fame, that’ll never complete nor does ever produce anything else but enlargements of scope, in a reminiscent threadmill.
ossabot: Logged on 2020-01-10 08:13:57 mircea_popescu: with the alt-pantsuit conceptual threadmill, the “counter”-cultural, by-and-for-goffy-kids yet JUST AS PANTSUIT (in the only important sense) threadmill, where nothing is replaced with ever “larger” nothing on a dilated schedule.
[..]
diana_coman: mircea_popescu: specifically and for full clarity though: what’s desired next step currently?
mircea_popescu: either a working 2.0 with 1.2 gfx stack or a detaled failure report.

Given the above desired next step combined with this situation of having set down the code for a few days at least, I took first some time to look in parallel at the currently deployed client (1.2) and the prototype client (2.0), as I picked essentially both of them up again.

The result of all this looking was some barf of course but more usefully an initial map as to what needs to be done. And given the goal to get the gfx stack fully back in, I focused first on cementing some “default” loaders for each *type* of graphics asset that is going to be of any use: terrain, water, air, character (cal3d), item/structure (CS mesh), sound, icon, skin. Those defaults will be as minimal as possible and will serve as placeholders whenever needed – if an asset is missing or the client doesn’t know about it because reasons ², then the client will use the default for that type and be done with it. Sure, everything will look the same and/or confusing if there’s too much of this going on but if you don’t like that, you are warmly invited to get off your ass and make a better client already, there’s your motivation for it. You’re welcome.

The last attempt at getting to the bottom of terrain creation with CS ended up with a very mirror-like terrain that is really not quite good enough since one can hardly see much with all that reflection. So I’ve spent the past few days going again through it all, mapping out all the various bits and pieces from xml files everywhere to CS’s terrain2 plugin and interfaces and whatnots.

To start with, since it’s defaults and pragmatism all around this time, I’ve set the prototype client to just load the main shaders anyway since the rest of the code can’t really do anything remotely reasonable without them. Let them be there, part and parcel of “the client” and none of server’s concern otherwise. This bit already was funny since it turns out that there is one list of shaders where you’d perhaps expect it, namely in data/shader so the usual “inventory xml” file: shaderlist.xml. Except, this list is very small compared to the full set of shaders that ship with both CS and legacy client. So what gives? Well, it turns out that there’s further listing of shaders from within the…materials’ inventory! Now add to this the fact that each shader may reference snippets and other shaders that are perhaps not in the list and you might get some idea as to where this leads if one follows it blindly.

By now though I have no intention to follow anything blindly and so I don’t care about its references, I’m not reimporting PS’s full-blown dependency-dragging loader either ³ and I’ll use instead CS’s basic loader to do the best it can. So I’ve set it explicitly and yes, hardcodedly, to load a list of shaders as I extracted it out of the current “shaderlist” and “materials.cslib” leaving it to handle or not the rest, who cares all that much. Surprisingly enough – or maybe not – it turns out it even works so I marked this part as done and moved on to making a default “Terrain” that will be used for any sector if need be.

While the naive approach with respect to terrain would consider the heightmap to be everything that’s needed, the reality is of course rather messier than that. The legacy client does in fact several dances back and forth on this, to the point where the same files are in two places and the same mesh is defined in parts in the world xml file – I have no idea why really, there’s no requirement for that and it certainly makes the whole thing rather confusing to follow.

Anyways, turning the terrain2 plugin inside out this time, I got at least to figure out the crucial bits: yes, the heightmap is needed; so is the base material in fact, despite what one might think at first – it’s really this base material that does the first and most crucial layer of work. Setting this base material as it is in the legacy client requires the basic texture of the terrain, its lightmap, materialmap and normalmap as well as the three basic materials (texture+normals for each of them) that are to be used throughout.

The less than ideal news is therefore that “defaults” in this case end up being 11 .png and .dds files rather than just the one heightmap.png. The good news is that I figured out the idiotic shader and shader vars setting for this particularly convoluted thing ⁴ and the result manages to actually look -perhaps unexpectedly- like some terrain after all:

If you don’t quite like the rather rough look of the terrain above, note that it’s indeed a very basic default terrain. The legacy client adds to the above several passes of rather intricate specifications of generating 3d grass/sand/rock of sorts with windspeed and density maps (aka yet another set of .png + .dds files that would need to get added to the defaults dir). At this stage I’m not even really sure that this side of things is really much concern for S.MG and the server – let the client prettify any terrain as it sees fit, after all.

For future reference, I’ll note here that the whole dance with a “material palette” from which supposedly the terrain plugin selects and blends the materials based on the height of each point turned out in practice to make very little difference by itself. And so while I wrote the code to do that too, I’m not even sure it will make it to the production version as anything other than a note in the comments. This “palette” is such a pretense: there are at all times precisely 3 materials and nothing more, since they are otherwise specified via yet another .png and taken as the red, green and blue values respectively, so no, no space for more than 3 anyway. The only thing that may change at all are the 3 materials themselves.

For now I’m happy with the above default terrain, as rough as it might look. Next week I’ll move on to setting in stone the rest of the defaults and see what cans of worms await to be squashed there.

1. Though this very same purposeful and focused unloading *also* highlighted just how much I’m carrying otherwise at all times – still am, yes, and looking back have been for longer than I have online references to point at.[↩]
2. Those have to do mainly with that idiotic expectation of knowing everything upfront.[↩]
3. This would be the one reading the xml and all that shit.[↩]
4. Because it’s a base material set on the default cell of the factory for terrain, the route is not exactly the very same as in other parts. Well, it wouldn’t be all that fun otherwise, would it.[↩]

There was a rather important question asked only recently in #eulora:

mircea_popescu: inasmuch as we do have a ~de facto~ data model, the gap must be bridged. “why aren’t you using what you are already using again ?”

To answer the above, I went and extracted what and how can be found currently in the full client and its dependencies code, to have it all clearly in view for once. The report of that investigation is below in quite a lot of detail but that detail means also about 5k words in total, so here’s directly the conclusion of it all namely the answer I can now give to that important question above:

1. The de facto data model is not really one data model at all but overall as it stands more of a confused mix of several formats, approaches and accidental structures of all sorts, all wrapped up and intertwined together into one single large pile of code with fundamental assumptions that are the exact opposite ¹ of those of S.MG and that requires a lot of effort for the smallest adaptation to new/different requirements, working otherwise mainly for as long as it is used precisely as it stands.
2. The thick layer of PS on top of the comparatively saner CS and Cal3D core makes access to and use of CS/Cal3D data models directly especially difficult/effort intensive because everything is built quite on purpose precisely to avoid such use. Moreover, the overall de facto data model such as it is has as fundamental assumption that *all* assets are known upfront with no new knowledge/assets ever added during runtime. Consequently, there is a significant tension between the de facto model and the desired model since the focus of ~everything in the current client is on handling local loading of potentially large but fully known upfront lists of assets while the desired model is instead meant for transmission and/or discovery and loading of new assets as an ongoing concern as the code runs.
3. At the core of the current client (hence in CS and Cal3D essentially) there are some clearly defined data models and formats, most notably Cal3D’s file formats and CS’s internal data models (that pack however shaders and shader variables into materials as standard) as well as some basic file formats used otherwise such as .png, .dds, .ogg or .wav. However, none of those different sets cover *everything* that is needed, meaning that they can only provide perhaps a *part* of that desired definitive universal data model. The rest can be – it would seem to me as a result of the detailed investigation below – extracted/synthesised out of what CS allows but using it as a universal data model is problematic with the existing client because of the PS layer on top that would require significant effort to turn around from hiding/making direct access difficult to using it directly when and where desired.
4. Making the best of it all, here’s a proposed list with the parts for which there is at least some clear format already (if not necessarily easy to use in the desired manner with the existing client such as it stands otherwise) and the parts on which a format would need to be decided on/extracted out of various options, fallbacks and CS’s own API even:
   1. Characters (animated mesh objects): Cal3D’s formats, ideally the binary set (as they are more compact); this means: .csf (skeleton), .caf (skeleton-based animation), .cpf (morphanimation), .cmf (mesh), .crf (material) + raster images for the textures that are referenced by filename from .crf; the preference for the texture format would be .dds files with size as power of 2 as CS otherwise performs the calculations needed to bring anything to the equivalent of that anyway; alternatively, CS compiled with the usual set of plugins supports a variety of usual image formats including .png, .jpg, .gif.
   2. Items/Structures (non-animated mesh objects): the options here would be to either force the use of the same .cmf from Cal3D’s set above (the trouble with this being that it assumes the presence of a skeleton/bones, which is not going to fit well with items/structures and moreover it will require some conversion for use with the current client even beyond the PS layer, namely to fit the internal CS non-Cal3D mesh) or otherwise extract and define a format based on CS’s internal representation/specification of a mesh (this is essentially a plain and long enummeration of all vertices, triangles that make up the surfaces and submeshes if any).
   3. Terrain: there is no clear format for this; the de facto “model” relying on CS’s Terrain2 plugin seems to require simply a bunch of image files for heightmap, material palette, base material and alpha as a minimum, using at all times some of CS’s predefined shaders for terrain. PS reads/expects those packaged via an .xml file that matches image files to their role for the terrain so either such .xml specification is imported as part of the format or that part of the client (the loader essentially) has to be adapted/changed.
   4. Particle systems: there is no format or specific model currently available; in principle it could be extracted based on the concrete set of parameters required by CS’s way of specifying existing particle systems (hence, a specification of emitter and effector parameters) but it’s unclear just how powerful such a model would really be as there are only a few types of particle systems implemented in CS and each of them tends to have its own specific parameters too.
   5. Light sources: there is no format or specific model currently available; extracting it from CS’s internal representation provides at least a relatively simple set of characteristics so this could be perhaps done as such.
   6. Ambiental light and fog/air effects: there is no format or specific model currently available; de facto, ambiental light values as well as fog and air effects (mostly weather-related) are currently scattered through several .xml files that are mainly PS’s: some values for ambiental and fog are given in the world.xml file (art/world.zip) while hour by hour values for ambiental and rain are given in a different .xml file for a full day and the client expects them as such. The more reasonable approach here is to simply keep those as sets of rgb(a) values rather than force-define anything more complex than that.
   7. Effects: there is no format or specific model currently available; CS does not treat this as a separate category as such; PS on the other hand has a de facto .xml format for defining effects attached to items but there is no clear specification for it and existing samples focus mainly on defining essentially animations of sorts. Perhaps this can be skipped entirely as nothing different from particle systems and/or sound and/or some specific animation defined through the corresponding formats.
   8. Materials: .crf (Cal3D’s format); this will require though some conversion to use with the current client even beyond the PS layer, since it is not a direct fit to CS’s internal data model for materials. Moreover, using this format will fail to take advantage of some of CS’s techniques for materials such as using animated textures (literally changing a set of textures in the render buffer so the result is an animated surface). Arguably such effects though will have to be obtained by other means if desired by any specific client.
   9. Textures: raster images with sizes as power of 2, preferably .dds format but possibly supporting a wider set including png, tga, bmp, jpg, dds, gif, mng, jng. NB: there is no provision or format available for any procedural texture generation in this case (CS provides procedural texture generation implementations for water, fire, plasma and dots).
   10. Icons/2D/GUI decorations: same as textures above. Basically any image can be used as either icon or texture at any time anyway. NB: there is no provision currently for using any vectorial representation directly.
   11. Sound: .ogg and .wav files.
   12. GUI/Skin: this is currently specific to PS and as such perhaps the “model” can simply be a .zip file that contains whatever files and everything else a specific client requires to skin its GUI entirely. This would be the only option I see to provide this too along the rest without making it either dependent on the current client implementation (hence PAWS in particular) as such or otherwise truly the protocol/server’s concern since it’s strictly a matter for the client to decide on, in my view.

Overall the parts and formats listed above focus on the assets themselves and entirely avoid the specification of function and structure that is currently de facto done via .xml files. The following section provides a more detailed look at how things happen with the currently deployed client.

Current Client Situation

While the de facto data formats in Eulora have been investigated previously and in quite some detail too, the focus was at the time on understanding the whole thing well enough to be able to extract out of it the useful and – hopefully – discard all the useless and the worse-than-useless currently bundled up with it. The attempt failed though, despite the progress made in that vein. So here I am taking stock of the various parts of the swamps I am stuck with, for lack of having anything better – meaning something that actually fully works – to take stock of, how lovely ². Anyways, them being the swamps is no excuse for not having otherwise a clear picture of what’s in there, so I’ll aim to bring the clarity even to an inventory of mud, what else is there to do anyway.

To start with and for an initial top-level, orienting view, note that there are effectively at least 3 (three!) layers of “formats” weaved and intertwined in the current client, from bottom up:

1. Formats supported by the various external libs and plugins that CS ³ uses. Those are usually standard formats such as .png and .jpg for images for instance, .ogg for sound or .caf (Cal3D’s format) for animations. Note that the full list of those is not really a fixed thing since it *depends* on what components you have on your system and include/compile with the whole client! Moreover, *none* of the different plugins provides a single set of formats that would cover *fully* all assets or even all graphical assets so one can’t just pick from there a single set and be done with it.
2. Formats defined/imposed by CS. Those can be further split into internal and external (to the engine code as such). The internal formats are most relevant for textures, materials and meshes since they fix a certain data model and requirements for defining and working with those (see below for the more detailed descriptions). The external formats are based on xml but with specific tags, restrictions/rules and parsers for things such as “shaders” or scene/world/object definition in standalone – and at times or to some extent hardcoded even – files that CS knows how to work with.
3. Formats defined/imposed by PS ⁴. Those are yet another set of xml-based “formats” ⁵, this time aiming – as far as anyone can tell by looking at the gnarly result – to provide the way to list and describe existing assets of the game, from 2D icons to 3D meshes and GUI widgets (windows in game).

Given that the original goal was to arrive at a clear definition of a definitive ⁶ universal data model, I’ll focus on the “clear” and therefore use the above three neat layers to structure the whole discussion of the de facto formats and emerging “model”. The alternative would be to follow closely and describe exactly just “how the client currently works” but that results quickly ⁷ in a quite predictable muddle (given the de facto muddle it reflects), so not much use for clarifying anything or for answering any question. To state it plainly though: I’m aiming to structure the text for clarity but not by glossing over any gross details – my aim is to mention such muddy details where they are relevant but to not let them drive my text even if they do get to drive the legacy code in quite a few places.

Before diving into the concrete details and exact formats currently in use on client side for all sorts, it’s worth noting a few fundamental characteristics of CS that drive quite a lot of the whole de facto model such as it is:

1. The focus is on supporting – somehow and to a best effort level essentially – a generic and not that well defined everything. The main implication of this is lack of clarity and added complexity of all sorts: there isn’t a clear single data format and model as such but at most a generic structure (most notably at the way materials are defined and implemented to include specific reliance on predefined shader types and variables) that still ends up forcing some choices, only not fully supporting them either (since that would break the appearance of supporting everything); there are always rather intricate fallback and alternative mechanisms at all levels from type of rendering (software vs hardware and then depending on what operations are supported by different types of hardware) to high-level object or scene definition (e.g. from code or through xml).
2. XML is used (and pushed) extensively for a double – if never explicitly stated as such – purpose:
   1. to describe the structure and organisation of sets of assets on disk (all sorts, including but not limited to graphics) for a CS application. Sets of graphics/sound/UI assets are called “library” in CS parlance, for all their xml-only definition.
   2. to script from outside the CS application all sorts, from rendering steps (in definitions of shaders) to scenes, objects and even actions in the world (in definitions of sectors, meshes, triggers, sequences and portals). Especially for shaders, this gets quite intricate, it’s a sort of ad-hoc xml-based programming language, no 2 ways around it.

   Note that the scripting purpose of xml above goes well beyond mere description of structure and organisation or even statement of the purpose/use of things in there. To try and give an idea of this, the xml code at CS level is quite overgrown and does a lot of things, including but not limited to customising existing objects, describing how to create new objects out of existing assets, setting out what and where things are to be found in the world, calling and otherwise using (therefore fixing as requirements) specific plugins or parts of CS, defining new rendering sequences or even buffer contents and prescribing steps and parameters of all sorts for them.

   The saving grace of all the above is that most – though unfortunately not ALL – of it can still be done directly from code as well, without a lot of trouble. So in this sense, the xml reliance pushed by CS is more of an option than a mandatory part to deal with. Nevertheless, there is one significant part of this xml reliance that is deeply embedded and does not really have a clear code-only/no-xml alternative, making it more of a mandatory part than an option: the shaders which are fully described *only* through a spaghetti of xml files that rely on parts and “snippets” from one another, provide fallback or alternative to one another and otherwise end up hardcoded as part of the very definition of a material and at times of crucial plugins (the terrain generator for instance) in the CS engine.

3. All identifiers (including those of assets loaded/created in the engine) are strings – combined with CPP’s own specific brand of string-problems (most notably the null terminator that may or may not exist and the allocate/deallocate memory dance), this is a pain. The approach of CS/PS to avoid the pain is by having everything rely on CS’s implementation of csString as a utility that takes care of the gritty details of using strings in CPP. Furthermore, there are several hashmaps used to map those cumbersome string ids to saner numerical values but those of the engine itself are not directly exposed. Adding to this, CS provides a way to “attach” data to any object so one could in principle add a numerical id but there is little gain from that since all searches for objects still rely on the string “id”.
4. There is an implicit assumption that all assets and the game’s world are fully known and defined upfront as well as outside the code itself. The programmatic option – as much as it exists – ends up overall underdeveloped in comparison to the xml support for the fixed, outside of code definition of everything.

The above fundamental characteristics of CS get pushed even further at the PS level on top: the fallbacks and “support everything but don’t commit to anything in specific” are pushed even on to graphics preferences and configuration files (of which there are several types and several layers); the xml gets added functionality as means to describe appearance of widgets (windows) in the client’s interface while still relying on even more hardcoded paths/names both in the xml itself (which references resources as it expects them to exist) and in the code (which looks for various xml files by name and path (if at least the path is relative to some directory as defined in the VFS – virtual file system – configuration); the expected full upfront knowledge of everything that may exist in the world gets cemented by PS: there is virtually no programmatic creation of anything and instead, everything is both supposed to be described in external xml files (+ referenced graphics/sound/effects files) and otherwise known and loaded before the game can even start at all. Essentially the PS client chooses at most what to show to the human player at one time or another out of its otherwise full knowledge of what’s possible and what not ⁸. Things can get at most cloned or deleted, hence the player may encounter “in the game” something that seems “new” to them only if they really didn’t look in the client’s directory as otherwise everything’s already there as long as it has some sort of asset-based representation at all.

Leaving aside for now the above characteristics of CS and their various implications, here’s otherwise the structure such as it is of the de facto data model at the core of the engine itself, from top down:

• the whole world is made of “sectors” that may be split into “regions” and contain any number of “sources of light”, “effects” and “mesh objects”;
• sectors are connected to one another through “portals” (that are really just a special type of mesh objects);
• sources of light do not have geometry and are defined instead through a set of functional characteristics, they are basically ideal lights that can be tweaked to some extent (see below for full list of characteristics);
• effects include particle systems as well as sequences (a set of operations tagged with relative time so that they get executed in a sequence);
• mesh objects can contain in turn other mesh objects (in a hierarchical structure) and are defined ultimately through their geometry (aka vertices and triangles) combined with assigned materials that describe the surface appearance and, where relevant, skeleton-based animation (CS supports animation through either its own sprite class or the external Cal3D lib);
• materials are in turn made out of one or several textures combined with any number of shaders and parametrised through specific values assigned to variables from a predefined set;
• shaders are defined via xml only and contain instructions for the rendering of the material data, introducing further their own variables, parameters and dependencies on other shaders;
• textures are the most basic element and they are either directly a colour represented through the three components, red, green and blue or otherwise an image file that is meant to be of a power of 2 in size (with the exact list of supported formats depending on which plugins are linked with CS on any specific install). There is also a limited selection of procedural approaches that can be in principle applied to any texture, namely for fire, water, dots and plasma appearance. Confusingly though, CS still wraps all textures into a default “material” so despite loading textures, it’s always materials one gets to work with from the code. Moreover, further tweaks such as animating textures are done through render buffers and as such essentially through shader code, whether it gets written as a separate xml-based shader specification or otherwise implemented directly in the code.

Adding more concrete details to the above overall description:

1. Sector – this is an abstract type (it does NOT define by itself any geometry and as such it does not have dimensions or position or anything of the sort) that works as a way of grouping together a set of assets for rendering purpose (overall rendering characteristics are defined at this top level, including ambiental light characteristics, some air effects such as fog, overall sources of light if any and specific choices of visibility culler and collision detection system). Everything that is shown/heard as part of the game has to be contained in a sector and at any given time there is only one sector that is fully visible. Characters may be in more than one sector at a time but only temporary and as part of the way in which crossings happen from one sector to another, namely through “portals”. At CS level, everything in a sector and all properties of a sector can be changed in principle on the fly from the code although some changes may incur a significant computational cost (e.g. changes to dynamic lights that should trigger a recalculation of lightmaps).

While the above is given by CS, the current client PS implementation adds its own layer of complexity on top: the current code requires that all possible sectors are already defined in an xml file that it expects to find in /art/world.zip (this path may be changed but not on the fly, only from one of the many configuration files). Moreover, it also expects that it knows everything upfront about all sectors and that all sectors reference only assets/items/anythings that it already knows about (from other hardcoded xml files that serve basically as inventories and/or definitions of different types of assets). Finally, there’s the added hardcoded dependency of some of the GUI on some sectors and graphical assets (most notably the requirement for the “podium” sector that is used to display the character’s smaller version in the inventory window and/or at the initial character selection). Note also that PS’s current client implementation makes it very hard to actually change anything on the fly from the code since it quite specifically assumes that things do not change – they may be copied or otherwise not shown on screen or even not fully loaded into memory at one time or another but otherwise not really changing much.

1.1 Region – this is yet another abstract type, basically providing a way to split a sector into smaller chunks of contained objects (of all sorts, so not only physical objects) if desired; the main reason for this to exist at all seems to be the fact that the assumption of knowing everything upfront can result otherwise in “too much to load at a time” and this got solved by allowing the split of sectors into regions ⁹.

Here PS adds a whole layer that keeps track of various regions and “zones”, what should be fully loaded for each and even what specific picture to show the user while the loading is crawling at its own pace. It also handles its own added complexity from the current interaction with the server as some entities may come before their sector/region is fully loaded and so the client has a special sector created as a temporary purgatory for those entities, checking continuously to see when they can finally be moved to some loaded sector/region.

1.2 Light – there is “ambiental light” as an overal property of each Sector, as well as “sources of light” as abstract objects (no geometry) that have nevertheless a position from which the light is considered to be emitted. Ambiental light is defined only through its colour (rgb). Sources of light can be of 3 types: static (do not move and do not change intensity or colour), pseudo-dynamic (do not move but can change intensity or colour), dynamic (can both move and chnage intensity or colour). Each source of light is defined at a minimum through its type (static, pseudo-dynamic or dynamic), position (3 float values), radius (float value) and “diffuse” colour (rgb) that influences as well the intensity of the light coming from this source. Optional characteristics include a second colour (“specular”, rgb), an attenuation mode (none, linear, inverse, realistic, constant linear quadratic) and parameters (3 floats), a cutoff distance (float value) representing the maximum distance at which this light is considered to shine, a halo (flare, nova or cross, with corresponding parameters), falloff inner and outer angles for a spot light (2 float values).

1.3 Fog – this is an overall property of each Sector and is defined through colour (rgb), mode (none, linear, exponential, exponential2, crystalspace), density (float) and fading start/end distance (2 float values).

1.4 Effects – this is my own category really since CS doesn’t bother as such to fully group everything. CS provides a set of particle systems that can be programmatically used. As I discussed those in more detail elsewhere, I won’t repeat that part here. However, it’s worth noting that the current implementation of those particle systems relies on the predefined set of shaders that CS comes with and so they are essentially customisable to some extent through the code rather than fully defined (unless one adds a redefinition of expected shaders too, I suppose).

The current client’s take on effects is again – unsurprisingly – quite entirely reliant on various xml files. There is a hardcoded path art/effects that includes a bunch of .dds files mainly as well as a art/music/effects that includes a bunch of .ogg and .wav files. The paths to those are hardcoded throughout the client where the legacy code “knew” effects should be loaded from. The loaders generally look for an xml file that provides the inventory and tends to attach “resource names” to the physical files. At times, the xml file will also provide items to which effects are “linked” and the legacy PS code keeps effects in an array with references to the intended items. Even this de facto “model” is not all that respected really since there is at least one place where the “rain effect” is simply directly hardcoded as a texture to be loaded from the hardcoded path to the .dds file (see src/client/weather.cpp: const char* rainTex = psengine->GetConfig()->GetStr(“PlaneShift.Effects.Rain.Texture”, “/this/art/effects/raindrop.dds”);).

In addition to the above, PS also adds its own plugin for loading effects defined in xml files as illustrated in data/effects. The full format for those is not clearly defined anywhere and currently I haven’t fully investigated it either as it’s not directly in use. At any rate, the .eff xml files defining the effects contain also the definition of needed textures and materials based on the .dds files included next to the xml. The rest of the .eff file seems to define an object type to which the effect is to be attached and otherwise keyframes so essentially yet another sort of animation format done ad-hoc in xml.

1.5 Mesh Objects – from CS’s point of view, anything that has geometry at all is a mesh object. The main categories of mesh objects that I can discern are terrain, characters (moving) and items/structures (non-moving). For each of those, CS allows in principle any concrete implementation and inner data model to be used, as long as the corresponding SCF ¹⁰ interface is provided. In practice though, the existing client uses the Terrain2 plugin for terrain, the Cal3D plugin for characters and CS’s native basic mesh for items/structures. Note however that all of those are wrapped up in a few layers of PS obfuscation, most notably through the GEMObject hierarchy and psCelClient + psEntity classes. The GEMObject hierarchy is a core element of the current client too and as such it’s not trivial to either change or remove. Getting directly and using CS’s interface from the existing client is not straightforward at all because the PS layer works precisely and purposefully to insulate away from touching CS directly. Nothing is impossible, of course, but whatever clarity there is at CS level needs to be considered through the murky and intertwined layers of PS on top as things currently stand.

1.5.1 Terrain (from Terrain2 plugin) – at CS level, this requires as a minimum a heightmap file, a material map file and a material palette (aka several full material definitions including their texture(s), shader(s) and shader variable(s)). The CS Terrain2 plugin is directly linked to some specific shaders that handle the splatting and seem to further require – for better appearance – at least an alphamap of the whole terrain. The current PS implementation relies on XML to define the terrain too and expects as usual that everything – including all possible terrains and maps and even the pattern of sunlight throughout the various hours of the day! – is known and fully loadable upfront.

1.5.2 Characters (from Cal3d plugin) – when the Cal3d plugin is used, there is an XML description required (mainly by the PS layer since CS can otherwise simply do it from the code directly) that specifies the relevant Cal3D files for any given character. Those Cal3D files come in 5 main formats (the formats mentioned below are binary formats; they all have equivalent xml formats as well – those are directly readable but otherwise more bulky, of course):

1. Skeleton file, .csf – this lists the hierarchy of bones that make up the character’s skeleton. Each character is supposed to have at least one bone.
2. Animation file, .caf – this defines the pose of the character at each keyframe (the animation will interpolate the pose between those keyframes). Keyframes are further grouped by tracks with one track per animated bone and have to include the time, the relative position and the relative rotation with respect to the parent bone (if .caf file).
3. Meshanimation file, .cpf – this is meant for changes in expression for instance or more generally animation of meshes/submeshes without relying on /requiring bone movement as such. Note that this format is mentioned in the Cal3D docs, but it is not currently in use at all and it’s unclear how well it is actually even supported otherwise (to establish this, I’d need to investigate in more detail both Cal3D itself and the CS plugin for Cal3D).
4. Mesh file, .cmf – this describes in detail the mesh as a hierarchy of submeshes and with references to materials in use as well as the weighted influences of the bones on each vertex. This would also include morphtargets if the .cpf format is used for additional animation(s).
5. Material file, .crf – this describes a material through its ambient, diffuse and specular colours (each given through 4 values for red, green, blue and alpha respectively), its shininess (a float) and a number of textures that have String identifiers (those can be filenames but can equally well be anything else really).

1.5.3 Items/structures (from native CS’s mesh) – the de facto format for those follows closely CS’s internal data model of mesh objects: each mesh object can contain submeshes in a hierarchical organisation; each mesh/submesh is defined through its geometry (as an enumeration of vertices and triangles made with those vertices) and attached materials that are in turn defined through textures, shaders and specific values for any shader variables used. Textures themselves are raster images with .dds as preferred format of CS (as other formats such as .png or .jpg effectively incur a penalty although at this point it’s unclear what this penalty adds up to in real terms). CS allows the creation and definition of meshes either as tedious enumerations of everything or through a few limited generators that tend to rely on regular shapes (e.g. spheres and cones). Both of those can be directly called from code or otherwise wrapped up in xml and “loaded” from code.

As everywhere else, the current client relies on xml for the definition of all possible types of meshes. It loads mesh definitions from the xml files in art/meshes.zip and expects that all entities ever required /encountered for the whole duration of the game reference one or several of the meshes in that archive. In turn, the xml in meshes.zip references of course materials that are -all of them!- expected to be found in art/materials.zip and are defined as such in art/materials.zip/materials.cslib xml file. The materials.cslib xml file first lists an inventory of textures (assigning a “name” id for each image file in the archive basically) and then defines various materials that make use of those textures and/or specify various shaders and corresponding values for the relevant shader variables. As usual, the shaders are simply referenced by name and assumed to exist as expected under that name, basically making art/materials.zip dependent implicitly on the whole contents of data/shader (and through that, further on to data/shader-snippets and data/renderlayers at the very least).

1.6 Sound
For sound, CS simply relies on external plugins for handling a few standard formats, most notably .ogg and .wav (apparently there are some .spx files too in data/voice but I haven’t seen them in use from anywhere within the current client’s code). PS adds on top of this a soundmanager that is intertwined with a whole bunch of code otherwise so getting to the heart of the simple matter of playing one sound file from the current client can be surprisingly messy. Still, from the point of view of data formats here, things are quite simple since not even PS managed to do much about sound files being simply .ogg, there is that.

1.7 GUI and icons

With respect to GUI and more generally any icons shown in game (so any 2D representations really, such as items in the inventory for instance), CS relies on external plugins. In principle there is CEGUI supported but the currently deployed client does not use that one, relying instead on PS’s own PAWS. Both CEGUI and PAWS share a design of GUI as a set of widgets but other than that I didn’t investigate CEGUI any further since the current client is rather deeply intertwined with PAWS. Consequently, the de facto data model for GUI is more of a reflection of what PAWS expects rather than anything else: art/skins contains .zip files that provide any number of .png images in whatever hierarchy of directory one sees fit, together with an imagelist.xml file that provides effectively the inventory, assigning a “resource name” to which file and listing the relative path to it. Note that this imagelist.xml as well as the art/skins location are both hardcoded in the current client. Moreover, the imagelist.xml is meant to be the full list of all possible icons and GUI elements in the game and known when the client starts since it’s all a pre-requisite of even creating the GUI widgets at all.

Besides the above, there’s another part to GUI, since the PS client further expects xml to be used (in data/gui) to define each and every window that can ever be shown by the client. Those make use of resources by name and as such are inevitably -and implicitly- dependent on the contents of art/skin. Moreover, some of the current code goes as far as referencing directly and hardcodedly resources from that same art/skin (most notably the mouse pointer icon). Not sure where exactly should this go into a “definition of the de facto data model” but there it is.

A rather funny part of the xml reference by name of various resources is that font resources (data/ttf currently) get the same treatment and the xml widget definitions that dully specify – thus fixing too! – their favourite fonts by name again. Then the legacy PS code also hardcodes a font as default in quite a few places and rather central to the GUI places at that (src/common/effects/pseffectobjtext.cpp and pseffectobjtext2d.cpp in the same place; src/common/paws/pawswidget.cpp and src/client/gui/psmainwidget.cpp), specifying the exact “data/ttf/reteprelieum.ttf” path as well. Except the actual file in data/ttf is called “reteprelleum” rather than the hardcoded “reteprelieum” and so errors are thrown and whatever else gets used instead for all the usefulness of that careful hardcoding of one font in that many places. Are you laughing yet?

1. In particular fixed, finite and fully known upfront world and assets vs flexible, infinite and continuously discovered world and assets.[↩]
2. Isn’t this what you always thought “choice” meant in practice, namely choosing between the inexistent and the impalatable? No?[↩]
3. CrystalSpace[↩]
4. Planeshift, the ancestor of the current client’s code, such as it is.[↩]
5. More like accidents than formats.[↩]
6. Let me add here the very clear and useful definition of this definitive: mircea_popescu: it’s definitive in the sense death&damnation is definitive, not in the sense mathematics is definitive.[↩]
7. Yeah, I tried it out, I have the draft to show for it. And it was even a good rant really but not exactly serving the purpose it was meant to serve, so I discarded it anyway.[↩]
8. It does sound like an interesting choice when set out this way, doesn’t it? The code you run on your own machine is meant to know more than you, to choose what to show you at one time or another and even to specifically thwart any attempts – cheater! – to get to know more than it decides to share with you.[↩]
9. There is also “Collection” as yet another way of grouping resources based on whatever reason the programmer may come up with but this is not used by the current client at all so I won’t go into detail on it.[↩]
10. Shared Class Facility[↩]

Backtracking on a year’s work of cleaning swamps comes with the added dubious advantage of having even more work to do in order to set down the results such as they are for potential pickup (maybe!) sometime in the near or even not so near future. So I took a few deep breaths, checked the full compilation and versions of everything, read through the main changes all again and unloaded the main gist of it all in a readme file that might come in handy at a later time ¹. My initial plan was to pack it all up and let it be really but since I did already the work of disentangling the CrystalSpace (CS) way of programmatically creating the whole map and a lot of the stuff in it, I might as well publish the prototype code as ugly as it is, what can it hurt anyway.

As the freshly baked readme file says, this prototype client is an attempted switch from legacy ps code to:

1. SMG communication protocol as opposed to PS’s net/messages.h approach. Status: working registration of new account and exchange of Serpent keys for both ways communication + all lower level required functionality of the protocol (e.g. packing/unpacking, encryption/decryption, message types, queues). Existing (in EuCore, which is client’s lib for communication with the server and data acquisition, essentially SMG Comms + data acquisition skeleton) implementation for message types that are fully defined so far: keys management (rsa and serpent packaged messages), file transfer and request.
2. Compiling with gprbuild (gnat) as opposed to current dependency on jam/ftjam. Status: working for client and eucore lib (though CS not ported to gprbuild) with the caveat that a few plugins haven’t yet been moved to gprbuild as well (mainly because they were anyway slated for removal e.g. bgloader aka PS’s background loader thing).
3. On the fly loading of graphics (from clientcache/ for now for testing purposes) as opposed to the bgloader plugin delayed loaders that expected the whole WORLD to be known upfront and never changing. Status: removed the original “preloading” and caching of all sorts, as well as the whole “art” dir; existing working prototype loaders ² integrated into the client code (aka in use) for sectors, factories, meshes of various sorts, textures, materials (including shader variables setting). Some of the prototype loaders test successfully the intended reliance on EuCore (Ada code) for data acquisition as well as on the fly loading of local files from any specified location (as opposed to predefined world.zip file in a set location). No world file needed or used anymore; no preloading/delayed loading (bgloader is not called anymore, nor expected although the code is left in src/plugins/common/bgloader); disabled also the part that was expecting the full list of all art assets including shaders to be known upfront and loaded before even getting into the game at all.
4. Simplified start of the client meaning getting directly into game + using the EuCore lib (Ada implementation of SMG Comms as well as RSA TMSR, config file reading, key generation, read/write keys to binary files on disk etc). Status: working and tested, quick launch of the game and straight load of the world as known from client’s own local cache; removed dependency on net/ (though the code remains in src/common/net) and working offline client that doesn’t require ping-pong with the server for anything. All attempted communications with the server happens from EuCore only and it’s decided on by EuCore too since it is driven by the data acquisition needs, regardless of the GUI part itself. This version of the client is currently able to run standalone (ie no server really needed anywhere) and it will show a naked char that can be moved around (though none of the animations are loaded so that the char will remain stiff while gliding across the world).
5. Removed preloading of 2D assets too (the interface itself, hence PAWS). Status: PAWS (widgets and therefore all buttons and windows inside the game) IS working but some windows may not be visible as they are not loaded (they have a LONG and INTRICATE codependency relation going on so it was cut at the root for starters); there is a tested prototype loader for icons (2D) but it’s currently not called from any of the running code (mainly because the issue to sort out first here is related to purging preloaded ideas from PAWS and PAWS windows themselves since currently it all cascades if one tries to load one single window). Notably, the Quit button works (if it’s invisible nevertheless) and the confirmation dialog window is one of the few that actually shows.
6. Reading of config from Ada and from an .ini file as opposed to Planeshift’s multiple and multi-stratified configs. Status: working and tested for various data types; allows comments as well.

Caveats of this prototype client:

1. This version will BLOCK (waiting to read from /dev/ttyUSB0) if there’s no FG (or other source of bits) connected and working on /dev/ttyUSB0 – this is because reading from FG is currently BLOCKING as implemented in EuCore lib
2. The prototype implementations of various effects (e.g. rain, fire, snow) are currently commented out. They work as previously illustrated but I took them out at the moment as I was focusing on figuring out the shaders and their linking with materials and textures.
3. The look of the terrain (aka the colours really) as well as the looks of various effects seems to depend on the exact environment where the code runs. I had the same code resulting in brown rain as well as nicely twinkling rain, depending on where I ran it. This was not fully investigated mainly for lack of time but it prompted the deeper dig into shaders as apparently the “nicer” aspects are mainly achieved through shader(s) use currently.
4. The path & names of .bin key files are hardcoded and checked at runtime – depending on which keys are present and which ones are missing, the EuCore lib may try to generate new keys and/or send messages to the server to request an account/registration/etc.
5. While the .gpr file works to build the client code, the EuCore lib is linked dynamically and has to be built separately (see its own gpr file). Moreover, the .gpr file relies on CRYSTAL and CAL3D environment variables to be set to the locations of CS and Cal3D respectively so it can include their headers as needed. At runtime, LD_LIBRARY_PATH still has to point to CS and Cal3d as well as EuCore’s locations.
6. There are 3 libs that are not yet ported to gprbuild (mainly as they were either slated for removal or otherwise not yet high enough on the priority list to get done): soundmanager.so, celgraph.so and bgloader.so
7. The tests for EuCore in eucore/tests have their own project that needs to be built separately.

As to the code, it’s ugly prototypes and messy and all that but anyway, perhaps best to start with the “basic” versions of Getters I wrote for CS’s “factories” of meshes, hoping that I could perhaps simplify a bit the full thing on this side. To understand what I’m talking about here: to create anything at all in CS, you’ll need first the “factory” for that type of thing since the idea is that it’s more efficient this way; which it is, sure, but only if you ever want to create a lot of very-similar-things in the same place like that, which happens ~never and so, as a result, all this does is that ~every thing you create ends up with its additional unique factory too!

Add to the above the maddening fact that CS uses strings – and it’s CPP code so strings are a pain at all times, of course – as IDs for everything so you have to search for factories as well as for objects by their *name*. Oh, and each factory is created through a call that requires also the *name* of the SCF plugin that provides the type of… factory you are after. Think of it: since you have factories to create “same sort of” objects, it follows that the factories themselves represent already types of objects, right? Sure, but why not have then also types of factories to make things more… hm, interesting? So there we go, there are also types of factories and plugins that implement (or not) those types and it’s true that in general there’s only one plugin for each type but don’t let that stop you from having the oh-so-useful-mechanism for having more than one type because there’s now no real cost to all this, no, none at all, right? Fine, first let’s set those plugin names to something shorter at least and give the “types” some numbers too, I like numbers:

#define DEFAULT_CULLER “crystalspace.culling.dynavis”

#define GENERIC_PLUGIN “crystalspace.mesh.object.genmesh”
#define CAL3D_PLUGIN “crystalspace.mesh.object.sprite.cal3d”
#define TERRAIN_PLUGIN “crystalspace.mesh.object.terrain2”
#define PARTICLES_PLUGIN “crystalspace.mesh.object.particles”
#define EMITTER_PLUGIN “crystalspace.mesh.object.particles.emitter”
#define EFFECTOR_PLUGIN “crystalspace.mesh.object.particles.effector”

#define TERRAIN_RENDERER “crystalspace.mesh.object.terrain2.bruteblockrenderer”
#define TERRAIN_DATAFEEDER_SIMPLE “crystalspace.mesh.object.terrain2.simpledatafeeder”
#define TERRAIN_DATAFEEDER_THREADED “crystalspace.mesh.object.terrain2.threadeddatafeeder” //alternative data feeder available in principle
#define TERRAIN_COLLIDER “crystalspace.mesh.object.terrain2.collider”

#define TEXTURE_FIRE “crystalspace.texture.type.fire”
#define TEXTURE_PLASMA “crystalspace.texture.type.plasma”
#define TEXTURE_WATER “crystalspace.texture.type.water”

#define FACTORY_NONE 0
#define FACTORY_GENERIC 1
#define FACTORY_CAL3D 2
#define FACTORY_TERRAIN 3
#define FACTORY_PARTICLES 4

Using the above, an example ³ of a “Getter” method that will retrieve an existing factory from the engine or otherwise create it and return it, if it doesn’t exist already:

csRef EuLoader::GetFactoryTerrain(char *factName) {
//terrain of a sector
csRef factWrapper = 0;

//check if the factory exists already
factWrapper = psengine->GetEngine()->FindMeshFactory(factName);

if (factWrapper != 0)
return factWrapper;

//create a basic factory of this type (aka using the plugin corresponding to this “type”)
//boolean last argument is “addtolist” so yes, it should add it to list
factWrapper = psengine->GetEngine()->CreateMeshFactory(TERRAIN_PLUGIN, factName, true);

return factWrapper;
}

Similar Getter method for a texture, using the given name as path too since one string is anyway enough of a mess in there, no need for two. The loader thing is simply CS’s way of reading an image file, since the texture here is simply assumed to be an image (.png, .bmp, .jpg, .dds are all supported):

csRef EuLoader::GetTexture( char *texName ) {
csRef tex = 0;

//first check if the engine already has it loaded
tex = psengine->GetEngine()->FindTexture(texName);
if (tex != 0)
return tex; //found it so return and be done with it.

//texture not found, so load it
csRef loader = csQueryRegistry (psengine->GetObjectRegistry());
if (loader == 0)
Err::NonfatalError(“EuLoader::LoadTexture failed to find the iLoader plugin of CS!\n”);
else
tex = loader->LoadTexture(texName, texName); //NB: the “name” is same as the full path!

return tex; //either way, the result is still in here
}

The equivalent for a Material is potentially more dubious because on one hand CS *anyway* wraps all textures into materials with the same name and on the other hand, a material also holds shader variables of all sorts, with the basic texture set as a “shader variable” itself! Moreover, shader variables can have different data types, what joy. So this one is not all that basic: it queries the client’s data cache (via that SMG object) to retrieve all shaders and variables for this material and applies them as they come:

csRef EuLoader::GetMaterial( char *matName) {
csRef matWrapper = 0;
csRef mat = 0;

//first check if the engine already has it loaded
matWrapper = psengine->GetEngine()->FindMaterial(matName);
if (matWrapper != 0)
return matWrapper; //material found so return it
//material not found, so create it;
//NB: “texture” in xml is ANYWAY made into a shader var at load time!
mat = psengine->GetEngine()->CreateBaseMaterial(0);
//add it to the engine’s list of materials
matWrapper = psengine->GetEngine()->GetMaterialList()->NewMaterial(mat, matName);

//set shaders and corresponding textures and/or shader vars
csRef stringSet = csQueryRegistryTagInterface (psengine->GetObjectRegistry(), “crystalspace.shader.variablenameset”);
csRef strings = csQueryRegistryTagInterface (psengine->GetObjectRegistry(), “crystalspace.shared.stringset”);
// csRef loader = csQueryRegistry (psengine->GetObjectRegistry());
uint32_t sindex = 1;
//for shaders to set on this material
char stype[MAX_STR_LEN+1];
char sname[MAX_STR_LEN+1];
unsigned int stypelen = MAX_STR_LEN;
unsigned int snamelen = MAX_STR_LEN;
while (psengine->SMG.GetShader(matName, sindex, stype, &stypelen, sname, &snamelen)) {
stype[stypelen] = ‘\0’;
sname[snamelen] = ‘\0’;
csStringID st = strings->Request(stype);
csRef ss = GetShader(sname);
if (ss != 0)
mat->SetShader(st, ss);
//next shader, if any
sindex = sindex + 1;
//reset the lengths!
stypelen = MAX_STR_LEN;
snamelen = MAX_STR_LEN;
}
//shader vars, if any
char sVarName[MAX_STR_LEN+1];
unsigned int sVarNameLen = MAX_STR_LEN;

//texture type shader vars
char sTexName[MAX_STR_LEN+1];
unsigned int sTexNameLen = MAX_STR_LEN;
sindex = 1;
while (psengine->SMG.GetShaderVar(matName, sindex, sVarName, &sVarNameLen, sTexName, &sTexNameLen)) {
sVarName[sVarNameLen] = ‘\0’;
sTexName[sTexNameLen] = ‘\0’;
csRef stex = GetTexture(sTexName);
if (stex != 0) {
csRef shaderVar;
CS::ShaderVarStringID strID = stringSet->Request(sVarName);
shaderVar.AttachNew( new csShaderVariable );
shaderVar->SetType(csShaderVariable::TEXTURE);
shaderVar->SetName(strID);
shaderVar->SetValue(stex);
mat->AddVariable(shaderVar); //as iMaterial extends directly iShaderVariableContext, this can be added here.
}
else
Err::NonfatalError(“EuLoader::GetMaterial can’t find the texture for a material\n”);
sindex = sindex + 1;
sVarNameLen = MAX_STR_LEN;
sTexNameLen = MAX_STR_LEN;
}

//float type shader vars
float v1, v2, v3, v4;
sindex = 1;
while (psengine->SMG.GetShaderVar(matName, sindex, sVarName, &sVarNameLen, &v1)) {
sVarName[sVarNameLen] = ‘\0’;
csRef shaderVar;
CS::ShaderVarStringID strID = stringSet->Request(sVarName);
shaderVar.AttachNew( new csShaderVariable );
shaderVar->SetType(csShaderVariable::FLOAT);
shaderVar->SetName(strID);
shaderVar->SetValue(v1);
mat->AddVariable(shaderVar); //as iMaterial extends directly iShaderVariableContext, this can be added here.
sindex = sindex + 1;
sVarNameLen = MAX_STR_LEN;
}
//vector2 type shader vars
sindex = 1;
while (psengine->SMG.GetShaderVar(matName, sindex, sVarName, &sVarNameLen, &v1, &v2)) {
sVarName[sVarNameLen] = ‘\0’;
csRef shaderVar;
CS::ShaderVarStringID strID = stringSet->Request(sVarName);
shaderVar.AttachNew( new csShaderVariable );
shaderVar->SetType(csShaderVariable::VECTOR2);
shaderVar->SetName(strID);
shaderVar->SetValue(csVector2(v1, v2));
mat->AddVariable(shaderVar); //as iMaterial extends directly iShaderVariableContext, this can be added here.
sindex = sindex + 1;
sVarNameLen = MAX_STR_LEN;
}

//vector3 type shader vars
sindex = 1;
while (psengine->SMG.GetShaderVar(matName, sindex, sVarName, &sVarNameLen, &v1, &v2, &v3)) {
sVarName[sVarNameLen] = ‘\0’;
csRef shaderVar;
CS::ShaderVarStringID strID = stringSet->Request(sVarName);
shaderVar.AttachNew( new csShaderVariable );
shaderVar->SetType(csShaderVariable::VECTOR3);
shaderVar->SetName(strID);
shaderVar->SetValue(csVector3(v1, v2, v3));
mat->AddVariable(shaderVar); //as iMaterial extends directly iShaderVariableContext, this can be added here.
sindex = sindex + 1;
sVarNameLen = MAX_STR_LEN;
}

//vector4 shader variables
sindex = 1;
while (psengine->SMG.GetShaderVar(matName, sindex, sVarName, &sVarNameLen, &v1, &v2, &v3, &v4)) {
sVarName[sVarNameLen] = ‘\0’;
csRef shaderVar;
CS::ShaderVarStringID strID = stringSet->Request(sVarName);
shaderVar.AttachNew( new csShaderVariable );
shaderVar->SetType(csShaderVariable::VECTOR4);
shaderVar->SetName(strID);
shaderVar->SetValue(csVector4(v1, v2, v3, v4));
mat->AddVariable(shaderVar); //as iMaterial extends directly iShaderVariableContext, this can be added here.
sindex = sindex + 1;
sVarNameLen = MAX_STR_LEN;
}
return matWrapper; //the result is in matWrapper at all times
}

Using the above, a prototype method that creates a sky as a sphere with the fixed “starry” image painted on the inside, since your world is – of course! – all inside of the sky dome, in this grand vision of computer graphics ⁴. Note that both the texture and the location (aka sector) of the sky are hardcoded in this hasty prototype:

bool EuLoader::GetSky() {
char skydomef[] = “skydomef”;
//factory wrapper
csRef factSkyWrapper = GetFactoryMesh(skydomef); //a *generic* mesh factory, used for ALL non-moving objects

if (factSkyWrapper == 0) {
Err::FatalError(“FAILED TO GET GENERIC FACTORY WRAPPER TO MAKE THE SKY!\n”);
return false; //this should never happen….
}
//geometry HAS to be set on the factory state as the meshstate lacks the needed methods…
//…and to get the factory state, first need the factory rather than the wrapper, arghhh
csRef factSky = factSkyWrapper->GetMeshObjectFactory();
csRef factClouds = factCloudsWrapper->GetMeshObjectFactory();
if (factSky == 0) {
Err::NonfatalError(“Sky/Clouds failed to find iMeshObjectFactory valid for created factory\n”);
return false;
}

//and since anyway can’t really have several objects here, set the MATERIAL on factory as well….ugh.
char skyMatName[] = “stars.png”;
csRef skyMat = GetMaterial(skyMatName); //this WILL load/create needed texture too, if it does not exist already
if (skyMat == 0) {
Err::NonfatalError(“EuLoader:GetFactory failed to find material for sky\n”);
return false;
}
factSky->SetMaterialWrapper(skyMat); //sets the material for the whole factory i.e. default for objects created from this factory

//generic settings
factSky->GetFlags().SetBool(CS_FACTORY_STATICSHAPE, true); //default static for genmeshes; non-static should use CAL3D!
factClouds->GetFlags().SetBool(CS_FACTORY_STATICSHAPE, true); //default static for genmeshes; non-static should use CAL3D!
//defaults: compress = false; auto_normals = false; auto_normals_nocompress= false
//if compress state->Compress();
//if auto_normals state->CalculateNormals(!auto_normals_nocompress);

//factory state
csRef factStateSky = scfQueryInterface (factSky);

//create sky dome geometry
using namespace CS::Geometry;
csVector3 center (0, 0, 0);
int rim_vertices = 16;
bool toponly = false; //make FULL sphere, not only top half; default=false
bool reversed = true; //so it’s visible from INSIDE, yes; default=false
bool cylmapping = false; //no cylindrical texture mapping; default=false
csEllipsoid ellips;
ellips.SetRadius(csVector3(2500,2500,2500));
ellips.SetCenter(center);
//this should directly generate the vertices,texels,normals and triangles + set vertex colors to black
factStateSky->GenerateSphere(ellips, rim_vertices, cylmapping, toponly, reversed);

/* alternative way to the above neater GenerateSphere? :
csDirtyAccessArray mesh_vertices;
csDirtyAccessArray mesh_texels;
csDirtyAccessArray mesh_normals;
csDirtyAccessArray mesh_triangles;
Primitives::GenerateSphere (ellips, rim_vertices,
mesh_vertices, mesh_texels,
mesh_normals, mesh_triangles, cylmapping,
toponly, reversed, 0);
AppendOrSetData (factState, mesh_vertices, mesh_texels,
mesh_normals, mesh_triangles);
*/

//directly here instead of calling “GeneralMeshBuilder::CreateMesh” that does exactly this;
//see cs/libs/cstool/genmeshbuilder.cpp

//sector – should get this either as param or from smg….TO DO
csRef s = psengine->GetEngine()->FindSector(“Rotces”);
if (s == 0) {
//this should NOT happen as the sector should call/create contained entities, not the other way around.
Err::NonfatalError(“EuLoader: failed to find the sector for the sky!\n”);
return false;
}

//objects (?)
//apparently: it’s really the triangles and/or render buffer stuff i.e. the “raw” values that just aren’t right (it WORKS with a generated ellipse…)
char skyn[] = “skyn”;
csRef skyMeshWrapper = psengine->GetEngine()->CreateMeshWrapper (factSkyWrapper, skyn, s, csVector3 (0));

//z buffer and priorities
skyMeshWrapper->SetZBufMode(CS_ZBUF_USE); //DO read so that the zbuffer gets updated…ugh;only write to the z-buffer but do not read
skyMeshWrapper->SetRenderPriority(psengine->GetEngine()->GetObjectRenderPriority());

psengine->GetEngine()->PrecacheMesh(skyMeshWrapper); //hmm?

return (skyMeshWrapper != 0 );
}

For an example, the test made with the moon, effectively creating an object in the world directly through the list of vertices and texture coordinates (as taken from its existing xml file otherwise, don’t look weirdly at me when you get to that huge list of vertices, they have been there all the time):

csRef EuLoader::GetMoon() {

csRef fact;
csRef state;
csRef type;

type = csLoadPluginCheck ( psengine->GetObjectRegistry(), GENERIC_PLUGIN, false);
if (!type)
Err::FatalError(“EuLOADER: can’t find crystalspace.mesh.object.genmesh plugin\n”);

fact = type->NewFactory ();
state = scfQueryInterface (fact);

//material
char matname[11] = “waves.dds”;
csRef mat = GetMaterial(matname);
if (mat == 0)
Err::FatalError(“EuLOADER: can’t find material waves.dds”);
fact->SetMaterialWrapper(mat);

//renderbuffer: position
const float posElems[] = {-24.46479416,4.86635208,16.66710281,-20.80559731,4.13849306,21.21320534,-21.21320534,0.00000000,21.21320343,-24.94408798,0.00000000,16.66710663,-20.80559731,4.13849258,-21.21320343,-24.46479607,4.86635256,-16.66710663,-24.94409180,0.00000000,-16.66710663,-21.21320343,0.00000000,-21.21320343,-27.18382263,5.40720034,11.48050213,-27.71638680,0.00000000,11.48050308,-16.34685135,3.25159287,-24.94409180,-16.66710663,0.00000000,-24.94409180,-28.85818863,5.74025345,5.85270882,-29.42355728,0.00000000,5.85271072,-11.25990391,2.23973608,-27.71638680,-11.48049831,0.00000000,-27.71638680,-29.42355919,5.85271168,0.00000358,-30.00000000,0.00000000,0.00000226,-11.25990868,2.23973703,27.71638870,-5.74025154,1.14180899,29.42355728,-5.85270977,0.00000000,29.42355728,-11.48050308,0.00000000,27.71638680,-5.74024439,1.14180756,-29.42355919,-5.85270309,0.00000000,-29.42355919,-28.85819244,5.74025393,-5.85270500,-29.42355919,0.00000000,-5.85270643,-16.34685326,3.25159311,24.94408607,-16.66710663,0.00000000,24.94408798,-27.18382263,5.40720034,-11.48049831,-27.71638680,0.00000000,-11.48049831,-10.60659599,4.39340019,-27.71638680,-15.39839554,6.37822866,-24.94409180,-16.34685135,3.25159287,-24.94409180,-11.25990391,2.23973608,-27.71638680,-27.71638489,11.48050785,0.00000358,-27.18381882,11.25991058,5.85270882,-10.60660172,4.39340210,27.71638870,-5.40719748,2.23973894,29.42355728,-5.40719128,2.23973608,-29.42355919,-5.74024439,1.14180756,-29.42355919,-27.18381882,11.25991058,-5.85270500,-15.39839745,6.37822914,24.94408607,-25.60660172,10.60660458,-11.48049831,-19.59844398,8.11794472,21.21320534,-23.04533005,9.54569340,-16.66710663,-23.04533005,9.54569244,16.66710281,-19.59844208,8.11794472,-21.21320343,-25.60660172,10.60660458,11.48050213,-15.39839554,6.37822866,-24.94409180,-17.63813210,11.78542995,-21.21320343,-20.74024582,13.85819817,-16.66710663,-23.04533005,9.54569340,-16.66710663,-19.59844208,8.11794472,-21.21320343,-23.04533005,15.39840508,11.48050213,-20.74024582,13.85819626,16.66710281,-23.04533005,9.54569244,16.66710281,-25.60660172,10.60660458,11.48050213,-13.85818863,9.25975227,-24.94409180,-24.46478844,16.34685707,5.85270882,-27.18381882,11.25991058,5.85270882,-9.54568291,6.37822866,-27.71638680,-24.94408607,16.66711426,0.00000358,-27.71638489,11.48050785,0.00000358,-9.54568672,6.37823105,27.71638870,-4.86634779,3.25159669,29.42355728,-4.86634254,3.25159264,-29.42355919,-24.46478844,16.34685707,-5.85270500,-27.18381882,11.25991058,-5.85270500,-13.85818958,9.25975323,24.94408607,-15.39839745,6.37822914,24.94408607,-23.04533005,15.39840508,-11.48049831,-25.60660172,10.60660458,-11.48049831,-17.63813210,11.78542995,21.21320534,-19.59844398,8.11794472,21.21320534,-19.59844017,19.59844971,-11.48049831,-20.80558968,20.80560303,-5.85270500,-14.99999523,15.00000477,21.21320534,-11.78542042,11.78543091,24.94408607,-17.63812828,17.63813782,-16.66710663,-17.63812637,17.63813782,16.66710281,-14.99999428,15.00000477,-21.21320343,-19.59844017,19.59844971,11.48050213,-11.78541756,11.78542995,-24.94409180,-20.80558777,20.80560303,5.85270882,-8.11793423,8.11794472,-27.71638680,-21.21319962,21.21320915,0.00000358,-8.11793709,8.11794853,27.71638870,-4.13848686,4.13849735,29.42355728,-4.13848305,4.13849211,-29.42355919,-6.37821770,9.54569435,-27.71638680,-9.25973988,13.85819817,-24.94409180,-16.66710091,24.94409561,0.00000358,-16.34684372,24.46479797,5.85270882,-6.37821913,9.54569721,27.71638870,-3.25158596,4.86635780,29.42355728,-3.25158310,4.86635208,-29.42355919,-16.34684372,24.46479797,-5.85270500,-9.25974178,13.85819912,24.94408607,-15.39839363,23.04533958,-11.48049831,-11.78541756,17.63813972,21.21320534,-13.85818386,20.74025345,-16.66710663,-13.85818291,20.74025345,16.66710281,-11.78541660,17.63813972,-21.21320343,-15.39839363,23.04533958,11.48050213,-9.54567719,23.04533386,16.66710281,-8.11793327,19.59844971,21.21320534,-8.11793232,19.59844971,-21.21320343,-9.54567909,23.04533386,-16.66710663,-10.60659409,25.60660934,11.48050213,-6.37821627,15.39840508,-24.94409180,-11.25989437,27.18382645,5.85270882,-4.39338970,10.60660553,-27.71638680,-11.48049545,27.71639252,0.00000358,-4.39339066,10.60660839,27.71638870,-2.23972821,5.40720701,29.42355728,-2.23972631,5.40719986,-29.42355919,-11.25989437,27.18382645,-5.85270500,-6.37821770,15.39840698,24.94408607,-10.60659409,25.60660934,-11.48049831,-5.74023724,28.85819435,-5.85270500,-5.85270023,29.42356491,0.00000358,-3.25158167,16.34685898,24.94408607,-2.23972559,11.25991631,27.71638870,-5.40718985,27.18382835,-11.48049831,-4.13848162,20.80560493,21.21320534,-4.86633825,24.46479797,-16.66710663,-4.86633682,24.46479797,16.66710281,-4.13847971,20.80560493,-21.21320343,-5.40718985,27.18382835,11.48050213,-3.25158072,16.34685898,-24.94409180,-5.74023724,28.85819435,5.85270882,-2.23972535,11.25991249,-27.71638680,-1.14179850,5.74026012,29.42355728,-1.14179790,5.74025249,-29.42355919,0.00001179,16.66711044,-24.94409180,0.00001201,21.21320724,-21.21320343,0.00001425,29.42355728,5.85270882,0.00001022,27.71639252,11.48050213,0.00001067,11.48050690,-27.71638680,0.00001022,30.00000381,0.00000358,0.00001089,11.48050880,27.71638870,0.00001033,5.85271788,29.42355728,0.00000944,5.85271120,-29.42355919,0.00001425,29.42355728,-5.85270500,0.00001089,16.66711426,24.94408607,0.00001022,27.71639252,-11.48049831,0.00001022,21.21320724,21.21320534,0.00001246,24.94408798,-16.66710663,0.00001425,24.94408798,16.66710281,4.86636257,24.46479416,-16.66710663,5.40721035,27.18382835,-11.48049831,4.86636448,24.46479416,16.66710281,4.13850212,20.80560303,21.21320534,4.13850355,20.80560303,-21.21320343,5.40721035,27.18382835,11.48050213,3.25160384,16.34685707,-24.94409180,5.74026585,28.85818863,5.85270882,2.23974657,11.25991058,-27.71638680,5.85272074,29.42356110,0.00000358,2.23974705,11.25991440,27.71638870,1.14181888,5.74025917,29.42355728,1.14181662,5.74025249,-29.42355919,5.74026585,28.85818863,-5.85270500,3.25160384,16.34685898,24.94408607,2.23974514,5.40719986,-29.42355919,4.39341021,10.60660362,-27.71638680,11.25992203,27.18381882,-5.85270500,11.48051453,27.71638680,0.00000358,6.37823963,15.39840508,24.94408607,4.39341116,10.60660553,27.71638870,10.60661411,25.60660553,-11.48049831,8.11795330,19.59844780,21.21320534,9.54570103,23.04533005,-16.66710663,9.54570293,23.04533005,16.66710281,8.11795425,19.59844780,-21.21320343,10.60661411,25.60660553,11.48050213,6.37823963,15.39840126,-24.94409180,11.25992203,27.18381882,5.85270882,2.23974848,5.40720558,29.42355728,9.25976276,13.85819435,-24.94409180,11.78543854,17.63813782,-21.21320343,16.34686470,24.46478844,5.85270882,15.39840889,23.04533386,11.48050213,6.37823772,9.54569054,-27.71638680,16.66711807,24.94408798,0.00000358,6.37823963,9.54569340,27.71638870,3.25160551,4.86635590,29.42355728,3.25160146,4.86635113,-29.42355919,16.34686470,24.46478844,-5.85270500,9.25976372,13.85819817,24.94408607,15.39840889,23.04533386,-11.48049831,11.78543854,17.63813782,21.21320534,13.85820389,20.74024582,-16.66710663,13.85820484,20.74024582,16.66710281,17.63814163,17.63813019,-16.66710663,19.59845543,19.59844780,-11.48049831,17.63814545,17.63812828,16.66710281,15.00001144,15.00000191,21.21320534,15.00001144,15.00000000,-21.21320343,19.59845543,19.59844780,11.48050213,11.78543854,11.78542328,-24.94409180,20.80560684,20.80559158,5.85270882,8.11795330,8.11794281,-27.71638680,21.21321487,21.21320343,0.00000358,8.11795616,8.11794376,27.71638870,4.13850594,4.13849545,29.42355728,4.13850117,4.13849115,-29.42355919,20.80560684,20.80559158,-5.85270500,11.78544044,11.78542900,24.94408607,4.86636066,3.25159168,-29.42355919,9.54570103,6.37822628,-27.71638680,24.46480179,16.34684753,-5.85270500,24.94409752,16.66710281,0.00000358,13.85820866,9.25974941,24.94408607,9.54570389,6.37822676,27.71638870,23.04534531,15.39840126,-11.48049831,17.63814735,11.78542519,21.21320534,20.74026108,13.85818958,-16.66710663,20.74025917,13.85818672,16.66710281,17.63814545,11.78542328,-21.21320343,23.04534531,15.39840126,11.48050213,13.85820580,9.25974751,-24.94409180,24.46480179,16.34684753,5.85270882,4.86636591,3.25159431,29.42355728,15.39840889,6.37822390,-24.94409180,19.59845352,8.11793995,-21.21320343,27.18382835,11.25990295,5.85270882,25.60661507,10.60660267,11.48050213,10.60661316,4.39339828,-27.71638680,27.71639252,11.48049927,0.00000358,10.60661507,4.39339828,27.71638870,5.40721416,2.23973656,29.42355728,5.40720892,2.23973536,-29.42355919,27.18382835,11.25990295,-5.85270500,15.39841461,6.37822628,24.94408607,25.60661507,10.60660267,-11.48049831,19.59845543,8.11794090,21.21320534,23.04533958,9.54568481,-16.66710663,23.04533768,9.54568291,16.66710281,20.80560875,4.13848925,21.21320534,16.34686661,3.25159049,24.94408607,24.46479988,4.86634684,-16.66710663,27.18383598,5.40719891,-11.48049831,24.46479797,4.86634541,16.66710281,20.80560684,4.13848877,-21.21320343,27.18383598,5.40719891,11.48050213,16.34686470,3.25158882,-24.94409180,28.85820007,5.74024677,5.85270882,11.25991917,2.23973489,-27.71638680,29.42356491,5.85270643,0.00000358,11.25992298,2.23973417,27.71638870,5.74026775,1.14180732,29.42355728,5.74026108,1.14180708,-29.42355919,28.85820007,5.74024677,-5.85270500,30.00000381,-0.00000313,0.00000358,29.42356300,-0.00000447,5.85270882,11.48051643,-0.00000156,27.71638870,5.85272551,-0.00000101,29.42355728,5.85271931,0.00000000,-29.42355919,11.48051453,-0.00000067,-27.71638680,29.42356300,-0.00000447,-5.85270500,16.66711998,-0.00000201,24.94408607,27.71639824,-0.00000134,-11.48049831,21.21321487,-0.00000179,21.21320534,24.94409370,-0.00000313,-16.66710663,24.94409180,-0.00000402,16.66710281,21.21321106,-0.00000179,-21.21320343,27.71639824,-0.00000134,11.48050213,16.66711617,-0.00000291,-24.94409180,20.80560684,-4.13849258,-21.21320343,24.46479797,-4.86635256,-16.66710663,27.18383408,-5.40720034,11.48050213,24.46479797,-4.86635303,16.66710281,16.34686279,-3.25159431,-24.94409180,28.85819626,-5.74025536,5.85270882,11.25991917,-2.23973608,-27.71638680,29.42356110,-5.85271168,0.00000358,11.25992203,-2.23973727,27.71638870,5.74026680,-1.14180923,29.42355728,5.74026108,-1.14180696,-29.42355919,28.85819626,-5.74025536,-5.85270500,16.34686661,-3.25159431,24.94408607,27.18383408,-5.40720034,-11.48049831,20.80560684,-4.13849306,21.21320534,25.60661125,-10.60660362,-11.48049831,27.18382645,-11.25991058,-5.85270500,19.59845352,-8.11794376,21.21320534,15.39840889,-6.37823009,24.94408607,23.04533386,-9.54569054,-16.66710663,23.04533386,-9.54569054,16.66710281,19.59844971,-8.11794376,-21.21320343,25.60661125,-10.60660362,11.48050213,15.39840698,-6.37822866,-24.94409180,27.18382645,-11.25991058,5.85270882,10.60661316,-4.39339924,-27.71638680,27.71638680,-11.48050308,0.00000358,10.60661411,-4.39340115,27.71638870,5.40721321,-2.23973846,29.42355728,5.40720844,-2.23973489,-29.42355919,9.54570007,-6.37822676,-27.71638680,13.85820007,-9.25975037,-24.94409180,24.94408798,-16.66710663,0.00000358,24.46479607,-16.34685516,5.85270882,9.54570103,-6.37822866,27.71638870,4.86636400,-3.25159550,29.42355728,4.86635971,-3.25159097,-29.42355919,24.46479607,-16.34685516,-5.85270500,13.85820389,-9.25975227,24.94408607,23.04534149,-15.39840126,-11.48049831,17.63814163,-11.78542614,21.21320534,20.74025345,-13.85819244,-16.66710663,20.74025154,-13.85819244,16.66710281,17.63813972,-11.78542614,-21.21320343,23.04534149,-15.39840126,11.48050213,13.85820007,-9.25975037,-24.94409180,15.00000763,-15.00000000,-21.21320343,17.63813782,-17.63813210,-16.66710663,19.59845352,-19.59844780,11.48050213,17.63813400,-17.63813210,16.66710281,11.78543091,-11.78542614,-24.94409180,20.80559731,-20.80559731,5.85270882,8.11795139,-8.11794281,-27.71638680,9.54570007,-6.37822676,-27.71638680,21.21320343,-21.21319962,0.00000358,8.11795235,-8.11794472,27.71638870,4.13850307,-4.13849545,29.42355728,4.13849926,-4.13849020,-29.42355919,4.86635971,-3.25159097,-29.42355919,20.80559731,-20.80559731,-5.85270500,11.78543472,-11.78542900,24.94408607,19.59845352,-19.59844780,-11.48049831,15.00000954,-15.00000000,21.21320534,15.39840698,-23.04533386,-11.48049831,16.34685326,-24.46479034,-5.85270500,11.78543186,-17.63813210,21.21320534,9.25975704,-13.85819626,24.94408607,13.85819626,-20.74024582,-16.66710663,13.85819530,-20.74024582,16.66710281,11.78543091,-17.63813210,-21.21320343,15.39840698,-23.04533386,11.48050213,9.25975609,-13.85819340,-24.94409180,16.34685326,-24.46479034,5.85270882,6.37823582,-9.54569054,-27.71638680,16.66710663,-24.94408035,0.00000358,6.37823582,-9.54569340,27.71638870,3.25160289,-4.86635494,29.42355728,3.25160027,-4.86634970,-29.42355919,4.39340830,-10.60660267,-27.71638680,6.37823200,-15.39839935,-24.94409180,11.48050404,-27.71637917,0.00000358,11.25990868,-27.18381882,5.85270882,4.39340734,-10.60660458,27.71638870,2.23974562,-5.40720367,29.42355728,2.23974419,-5.40719748,-29.42355919,11.25990868,-27.18381882,-5.85270500,6.37823343,-15.39840126,24.94408607,10.60660839,-25.60660362,-11.48049831,8.11795044,-19.59844398,21.21320534,9.54569435,-23.04532814,-16.66710663,9.54569244,-23.04532623,16.66710281,8.11794758,-19.59844398,-21.21320343,10.60660839,-25.60660362,11.48050213,4.13849735,-20.80559731,-21.21320343,4.86635590,-24.46478844,-16.66710663,5.40720463,-27.18382645,11.48050213,4.86635447,-24.46478653,16.66710281,3.25159764,-16.34685135,-24.94409180,5.74025345,-28.85818481,5.85270882,2.23974466,-11.25990868,-27.71638680,5.85271311,-29.42354965,0.00000358,2.23974323,-11.25991058,27.71638870,1.14181626,-5.74025631,29.42355728,1.14181614,-5.74025059,-29.42355919,5.74025345,-28.85818481,-5.85270500,3.25159836,-16.34685516,24.94408607,5.40720463,-27.18382645,-11.48049831,4.13849926,-20.80559731,21.21320534,0.00000687,-16.66710663,24.94408607,0.00000754,-11.48050499,27.71638870,0.00000530,-27.71638680,-11.48049831,0.00000307,-29.42354965,-5.85270500,0.00000843,-21.21320343,21.21320534,0.00000665,-24.94408035,-16.66710663,0.00000620,-24.94407845,16.66710281,0.00000665,-21.21320343,-21.21320343,0.00000530,-27.71638680,11.48050213,0.00000642,-16.66710281,-24.94409180,0.00000307,-29.42354965,5.85270882,0.00000933,-11.48050308,-27.71638680,0.00000486,-29.99998856,0.00000358,0.00000832,-5.85271358,29.42355728,0.00000944,-5.85270786,-29.42355919,-5.74024677,-28.85818100,5.85270882,-5.40719318,-27.18382263,11.48050213,-2.23972583,-11.25990868,-27.71638680,-3.25158405,-16.34685135,-24.94409180,-5.85270309,-29.42354202,0.00000358,-2.23972797,-11.25991058,27.71638870,-1.14179957,-5.74025536,29.42355728,-1.14179730,-5.74024963,-29.42355919,-5.74024677,-28.85818100,-5.85270500,-3.25158453,-16.34685326,24.94408607,-5.40719318,-27.18382263,-11.48049831,-4.13848162,-20.80559540,21.21320534,-4.86634207,-24.46478653,-16.66710663,-4.86634207,-24.46478271,16.66710281,-4.13848352,-20.80559540,-21.21320343,-9.54567909,-23.04532242,-16.66710663,-10.60659599,-25.60660172,-11.48049831,-9.54567909,-23.04532051,16.66710281,-8.11793232,-19.59844208,21.21320534,-8.11793327,-19.59844017,-21.21320343,-10.60659599,-25.60660172,11.48050213,-6.37821770,-15.39839363,-24.94409180,-11.25990009,-27.18380928,5.85270882,-4.39338923,-10.60660267,-27.71638680,-11.48049355,-27.71636963,0.00000358,-4.39339113,-10.60660362,27.71638870,-2.23972821,-5.40720177,29.42355728,-2.23972464,-5.40719604,-29.42355919,-11.25990009,-27.18380928,-5.85270500,-6.37821913,-15.39839745,24.94408607,-3.25158024,-4.86634779,-29.42355919,-6.37821674,-9.54568958,-27.71638680,-16.34684372,-24.46478081,-5.85270500,-16.66709518,-24.94406891,0.00000358,-9.25974083,-13.85819054,24.94408607,-6.37821913,-9.54569054,27.71638870,-15.39839363,-23.04533005,-11.48049831,-11.78541374,-17.63813210,21.21320534,-13.85818005,-20.74024200,-16.66710663,-13.85817909,-20.74024010,16.66710281,-11.78541660,-17.63813019,-21.21320343,-15.39839363,-23.04533005,11.48050213,-9.25973892,-13.85818672,-24.94409180,-16.34684372,-24.46478081,5.85270882,-3.25158477,-4.86635256,29.42355728,-11.78541374,-11.78541756,-24.94409180,-14.99999046,-14.99999523,-21.21320343,-20.80558395,-20.80558395,5.85270882,-19.59843826,-19.59844208,11.48050213,-8.11793232,-8.11794090,-27.71638680,-21.21318817,-21.21318626,0.00000358,-8.11793423,-8.11794186,27.71638870,-4.13848448,-4.13849211,29.42355728,-4.13847923,-4.13848829,-29.42355919,-20.80558395,-20.80558395,-5.85270500,-11.78541660,-11.78542137,24.94408607,-19.59843826,-19.59844208,-11.48049831,-14.99999046,-14.99999809,21.21320534,-17.63811874,-17.63812447,-16.66710663,-17.63811874,-17.63812256,16.66710281,-20.74023438,-13.85818481,-16.66710663,-23.04532623,-15.39839554,-11.48049831,-20.74023438,-13.85818291,16.66710281,-17.63812447,-11.78542233,21.21320534,-17.63812447,-11.78541946,-21.21320343,-23.04532623,-15.39839554,11.48050213,-13.85818291,-9.25974178,-24.94409180,-24.46477699,-16.34683990,5.85270882,-9.54568005,-6.37822533,-27.71638680,-24.94406700,-16.66708946,0.00000358,-9.54568291,-6.37822580,27.71638870,-4.86634350,-3.25159192,29.42355728,-4.86633825,-3.25158954,-29.42355919,-24.46477699,-16.34683990,-5.85270500,-13.85818386,-9.25974560,24.94408607,-5.40718555,-2.23973346,-29.42355919,-10.60659313,-4.39339733,-27.71638680,-27.18380356,-11.25989723,-5.85270500,-27.71636200,-11.48048782,0.00000358,-15.39838982,-6.37822247,24.94408607,-10.60659409,-4.39339733,27.71638870,-25.60659409,-10.60659599,-11.48049831,-19.59843445,-8.11793995,21.21320534,-23.04531288,-9.54568100,-16.66710663,-23.04531288,-9.54568005,16.66710281,-19.59843445,-8.11793613,-21.21320343,-25.60659409,-10.60659599,11.48050213,-15.39838696,-6.37821913,-24.94409180,-27.18380356,-11.25989723,5.85270882,-5.40719223,-2.23973489,29.42355728,-16.34683800,-3.25158548,-24.94409180,-20.80558586,-4.13848591,-21.21320343,-28.85816956,-5.74024248,5.85270882,-27.18381500,-5.40719318,11.48050213,-11.25989914,-2.23973393,-27.71638680,-29.42353058,-5.85269690,0.00000358,-11.25990105,-2.23973370,27.71638870,-5.74024439,-1.14180589,29.42355728,-5.74023771,-1.14180613,-29.42355919,-28.85816956,-5.74024248,-5.85270500,-16.34684372,-3.25158787,24.94408607,-27.18381500,-5.40719318,-11.48049831,-20.80558777,-4.13848925,21.21320534,-24.46477509,-4.86634493,-16.66710663,-24.46477509,-4.86634302,16.66710281,0.00000603,-0.00000903,30.00000000,-5.85270309,0.00000000,-29.42355919,0.00000978,0.00000000,-30.00000000,4.13849926,-4.13849020,-29.42355919,3.25160027,-4.86634970,-29.42355919,2.23974419,-5.40719748,-29.42355919,1.14181614,-5.74025059,-29.42355919,0.00000944,-5.85270786,-29.42355919,-1.14179730,-5.74024963,-29.42355919,-2.23972464,-5.40719604,-29.42355919,-3.25158024,-4.86634779,-29.42355919,-4.13847923,-4.13848829,-29.42355919,-4.86633825,-3.25158954,-29.42355919,-5.40718555,-2.23973346,-29.42355919,-5.74023771,-1.14180613,-29.42355919};
unsigned int bufPosLen = sizeof(posElems) / sizeof(posElems[0]);
unsigned int compCount = 3;
unsigned int elemCount = bufPosLen / compCount;
csRef bufPos = csRenderBuffer::CreateRenderBuffer⁵;
}

//factory wrapper
csRef wrap = psengine->GetEngine()->CreateMeshFactory(fact, “moonfact”);
if (wrap == 0)
Err::FatalError(“EuLOADER: failed to create factory wrapper!\n”);
return wrap;
}

Creating the terrain was quite a pain and it’s quite a long adventure so perhaps of some use to see the code for it too:

csRef EuLoader::GetTerrain() {
csRef meshWrapper = 0;
csRef factWrapper = 0;
csRef fact;
csRef state;
csRef typePlugin;

const char meshNameC[] = “Terrain”;
const char factNameC[] = “terrainF”;
char meshName[MAX_STR_LEN+1];
char factName[MAX_STR_LEN+1];
strncpy(meshName, meshNameC, strlen(meshNameC));
strncpy(factName, factNameC, strlen(factNameC));
meshName[strlen(meshNameC)]=’\0′;
factName[strlen(factNameC)]=’\0′;

csRef s = psengine->GetEngine()->FindSector(“Rotces”);
typePlugin = csLoadPluginCheck ( psengine->GetObjectRegistry(), TERRAIN_PLUGIN, false);

//get heightmap from EuCache via SMGData i.e. make sure it is available before creating empty sector…
char hmapfile[MAX_STR_LEN+1];
unsigned int lhmapfile = MAX_STR_LEN;
if (!psengine->SMG.GetHeightMapFile(factName, hmapfile, &lhmapfile)) {
Err::NonfatalError(“EuLoader: failed to find heightmap for requested terrain factory!\n”);
return meshWrapper;
}
hmapfile[lhmapfile] = ‘\0’;

//get materialmap from EuCache via SMGData
char hmatfile[MAX_STR_LEN+1];
unsigned int lhmatfile = MAX_STR_LEN;
if (!psengine->SMG.GetMaterialMapFile(factName, hmatfile, &lhmatfile)) {
Err::NonfatalError(“EuLoader: failed to find materialmap for requested terrain factory!\n”);
return meshWrapper;
}
hmatfile[lhmatfile] = ‘\0’;

//no need for base material in fact for terrain – there are the ones in the palette!
fact = typePlugin->NewFactory ();
csRef terrainFact = scfQueryInterface (fact);
//obtain needed renderer, collider and data feeder
//NB: those are all defaults because CS doesn’t really have a choice anyway.
csRef renderer = csLoadPluginCheck (psengine->GetObjectRegistry(), TERRAIN_RENDERER);
csRef collider = csLoadPluginCheck (psengine->GetObjectRegistry(), TERRAIN_COLLIDER);
csRef datafeeder = csLoadPluginCheck (psengine->GetObjectRegistry(), TERRAIN_DATAFEEDER_THREADED);
if (renderer == 0 || collider == 0 || datafeeder == 0) {
Err::NonfatalError(“EuLoader::GetFactory can’t find renderer, collider and/or data feeder for generating the Terrain!\n”);
return meshWrapper;
}

terrainFact->SetRenderer( renderer );
terrainFact->SetCollider( collider );
terrainFact->SetFeeder( datafeeder );

//set default cell – DEFAULT values!
//this is a “pseudo-cell” that serves as blueprint for any cells created by this factory
//NB: apparently by default there are NO cells created (?!)
csRef defaultCell (terrainFact->GetDefaultCell());
if (!defaultCell.IsValid()) {
//this should NOT happen!
Err::NonfatalError(“INVALID default cell in terrain factory!\n”);
return meshWrapper;
}
defaultCell->SetSize (csVector3 (1024.0f, 63.0f, 1024.0f));
defaultCell->SetGridWidth (257);
defaultCell->SetGridHeight (257);
defaultCell->SetMaterialMapWidth (512);
defaultCell->SetMaterialMapHeight (512);
defaultCell->SetMaterialPersistent (false);
//render properties
defaultCell->GetRenderProperties()->SetParameter(“block resolution”, “16”);
defaultCell->GetRenderProperties()->SetParameter(“splat distance”, “100”);
defaultCell->GetRenderProperties()->SetParameter(“lod splitcoeff”, “8”);
defaultCell->GetRenderProperties()->SetParameter(“splat render priority”, “object2”);
//heightmap and material map aka “feederproperties”
defaultCell->GetFeederProperties()->SetParameter(“smooth heightmap”, “yes”);
defaultCell->GetFeederProperties()->SetParameter(“heightmap source”, hmapfile); //?
defaultCell->GetFeederProperties()->SetParameter(“materialmap source”, hmatfile); //?

//create ONE actual cell in this factory
csRef cell = terrainFact->AddCell();
cell->SetName(“0”); //default, one single cell “0”
if (cell == 0) {
Err::NonfatalError(“EuLoader: GetFactory fails to create ONE cell for terrain!\n”);
return 0;
}
//position of this one cell
cell->SetPosition(csVector2(-511, -511));

//create the actual factory wrapper and register it with the engine
factWrapper = psengine->GetEngine()->CreateMeshFactory(fact, factName);
if (factWrapper == 0) {
Err::FatalError(“EuLOADER: failed to create Terrain factory wrapper!\n”);
return meshWrapper;
}

meshWrapper = psengine->GetEngine()->CreateMeshWrapper (factWrapper, meshName, s, csVector3 (0,0,0));
if (meshWrapper == 0) {
Err::NonfatalError(“EuLoader:GetMesh failed to create the requested mesh despite having its factory wrapper!\n”);
return meshWrapper;
}
//add material palette if this mesh has any!
uint32_t index = 1;
char matName[MAX_STR_LEN+1];
unsigned int len = MAX_STR_LEN;
csRefArray pal; //just in case there IS a material palette…
while (psengine->SMG.GetMaterialPalette(meshName, index, matName, &len)){
matName[len] = ‘\0’;
csRef m = GetMaterial(matName);
if (m == 0) {
Err::NonfatalError(“EuLoader: failed to retrieve material for material palette!\n”);
}
else
pal.Push(m);
index = index + 1;
len = MAX_STR_LEN; //reset this…
}
if (index > 1) {
//aka there IS a material palette
csRef mm = meshWrapper->GetMeshObject();
if (mm==0) {
Err::NonfatalError(“EuLoader: failed to obtain the mesh object out of the mesh wrapper!\n”);
}
else {
csRef syst = scfQueryInterface (mm);
if (syst == 0) {
Err::NonfatalError(“EuLoader: failed to get the terrainSystem to set the material palette on it!\n”);
}
else
syst->SetMaterialPalette(pal);
}
}

meshWrapper->SetZBufMode(CS_ZBUF_USE); //DO read so that the zbuffer gets updated…ugh;only write to the z-buffer but do not read
meshWrapper->GetMeshObject()->SetMixMode(CS_FX_COPY); //this should be default for most (?); see p321 (333) in cs manual re mixmodes and blending

meshWrapper->SetRenderPriority(psengine->GetEngine()->GetObjectRenderPriority());
psengine->GetEngine()->PrecacheMesh(meshWrapper); //hmm?

//initialize collision detection for this entity
csRef cdsys = csQueryRegistry (psengine->GetObjectRegistry());
csColliderHelper::InitializeCollisionWrapper(cdsys, meshWrapper);

return meshWrapper;

Putting some of the above together, there’s a more useful GetMesh that takes a numeric id (hence, a Euloran proper id since all objects in Eulora have a numeric id) and either finds or creates the thing (according to what EuCore says that thing should be):

csRef EuLoader::GetMesh( uint32_t id) {
csRef meshWrapper = 0;
//retrieve name as CS finds meshes by name
char meshName[MAX_STR_LEN+1];
unsigned int meshNameLen = MAX_STR_LEN;

if (!psengine->SMG.GetName(id, meshName, &meshNameLen))
return meshWrapper; //nothing to do as there’s not enough info for this object
meshName[meshNameLen] = ‘\0’;
//check if it exists already
meshWrapper = psengine->GetEngine()->FindMeshObject(meshName);
if (meshWrapper != 0)
return meshWrapper; //found it so return it

//not found so create it, if there is enough information for it
//will need: factory name, location (aka sectorID + position x,y,z)
char factName[MAX_STR_LEN+1];
unsigned int factNameLen = MAX_STR_LEN;
float x,y,z,rx,ry,rz;
uint32_t locID;
if (! (psengine->SMG.GetFactoryName(id, factName, &factNameLen)) ||
! (psengine->SMG.GetPos(id, &x, &y, &z, &rx, &ry, &rz)) ||
! (psengine->SMG.GetLocID(id, &locID)) ) {

Err::NonfatalError(“EuLoader: unknown factory name and/or location for requested mesh!”);
return meshWrapper;
}
factName[factNameLen] = ‘\0’;
//everything IS available, so retrieve/create factory
csRef factory = GetFactory(factName);
if (factory == 0) {
Err::NonfatalError(“EuLoader: failed to retrieve factory for the required entity!\n”);
return meshWrapper;
}
//retrieve/create sector for this too!
char sName[MAX_STR_LEN+1];
unsigned int slen = MAX_STR_LEN;
if (! (psengine->SMG.GetName(locID, sName, &slen))) {
Err::NonfatalError(“EuLoader: failed to retrieve sector name for the required entity!\n”);
return meshWrapper;
}
sName[slen] = ‘\0’;
csRef s = psengine->GetEngine()->FindSector(sName);
if (s == 0) {
//this should NOT happen as the sector should call/create contained entities, not the other way around.
Err::NonfatalError(“EuLoader: failed to find the sector for the required entity!\n”);
return meshWrapper;
}

meshWrapper = psengine->GetEngine()->CreateMeshWrapper (factory, meshName, s, csVector3 (x,y,z));
if (meshWrapper == 0) {
Err::NonfatalError(“EuLoader:GetMesh failed to create the requested mesh despite having its factory wrapper!\n”);
return meshWrapper;
}

//add material palette if this mesh has any!
uint32_t index = 1;
char matName[MAX_STR_LEN+1];
unsigned int len = MAX_STR_LEN;
csRefArray pal; //just in case there IS a material palette…
while (psengine->SMG.GetMaterialPalette(meshName, index, matName, &len)){
matName[len] = ‘\0’;
csRef m = GetMaterial(matName);
if (m == 0) {
Err::NonfatalError(“EuLoader: failed to retrieve material for material palette!\n”);
}
else
pal.Push(m);
index = index + 1;
len = MAX_STR_LEN; //reset this…
}
if (index > 1) {
//aka there IS a material palette
csRef mm = meshWrapper->GetMeshObject();
if (mm==0) {
Err::NonfatalError(“EuLoader: failed to obtain the mesh object out of the mesh wrapper!\n”);
}
else {
csRef syst = scfQueryInterface (mm);
if (syst == 0) {
Err::NonfatalError(“EuLoader: failed to get the terrainSystem to set the material palette on it!\n”);
}
else
syst->SetMaterialPalette(pal);
}
}

//TO DO: to check that all mesh entities have indeed zbuf_use?
//NOPE, not all: particles have cs_zbuf_test
//the zbufmode depends on the type of the mesh, which is in turn given by the …factory type…
int mtype = psengine->SMG.GetFactoryType(factName);
switch (mtype) {
case FACTORY_PARTICLES: {
meshWrapper->SetZBufMode(CS_ZBUF_TEST);
meshWrapper->GetMeshObject()->SetMixMode(CS_FX_ADD);
// meshWrapper->SetRenderPriority(psengine->GetEngine()->GetObjectRenderPriority());
meshWrapper->SetRenderPriority(psengine->GetEngine()->GetAlphaRenderPriority());

} break;
case FACTORY_TERRAIN: {
meshWrapper->SetZBufMode(CS_ZBUF_USE);
meshWrapper->GetMeshObject()->SetMixMode(CS_FX_COPY);
meshWrapper->SetRenderPriority(psengine->GetEngine()->GetWallRenderPriority());
} break;
default: {
meshWrapper->SetZBufMode(CS_ZBUF_USE); //DO read so that the zbuffer gets updated…ugh;only write to the z-buffer but do not read
meshWrapper->GetMeshObject()->SetMixMode(CS_FX_COPY); //this should be default for most (?); see p321 (333) in cs manual re mixmodes and blending
meshWrapper->SetRenderPriority(psengine->GetEngine()->GetObjectRenderPriority());
}
}
//ALL mesh entities are objects so set them with object priority for rendering;
//TO DO: to check this since “entities” include walls and doors and the like?
//i.e. might need other meaningful subcategories and decide rendering priority on that; hm.
// meshWrapper->SetRenderPriority(psengine->GetEngine()->GetRenderPriority(“wall”));
psengine->GetEngine()->PrecacheMesh(meshWrapper); //hmm?

//initialize collision detection for this entity
csRef cdsys = csQueryRegistry (psengine->GetObjectRegistry());
csColliderHelper::InitializeCollisionWrapper(cdsys, meshWrapper);

return meshWrapper;

}

For some really tedious stuff, here’s the creation of Foxy herself, hence a Cal3D object:

csRef EuLoader::GetCal3d( char *name, iSector *s, float x, float y, float z ) {
csRef meshWrapper = 0;
//check if it exists already
meshWrapper = psengine->GetEngine()->FindMeshObject(name);
if (meshWrapper != 0)
return meshWrapper; //found it so return it

//check sector is not null since CS REQUIRES a non-null sector to even create a mesh, ugh.
if (!s)
return meshWrapper;

//not found the mesh, so need the factory to create it…
csRef factWrapper = 0;
factWrapper = psengine->GetEngine()->FindMeshFactory(name);

if (!factWrapper) {
//factory does not yet exist, so create it.
csRef fact;
csRef state;
csRef typePlugin;

typePlugin = csLoadPluginCheck ( psengine->GetObjectRegistry(), CAL3D_PLUGIN, false);
if (typePlugin == 0) {
Err::NonfatalError(“EuLoader failed to load SprCal3D Factory PLUGIN TYPE\n”);
return meshWrapper;
}
fact = typePlugin->NewFactory ();

//create the actual factory wrapper and register it with the engine
factWrapper = psengine->GetEngine()->CreateMeshFactory(fact, name);
if (factWrapper == 0) {
Err::FatalError(“EuLOADER: failed to create Cal3D factory wrapper!\n”);
return meshWrapper;
}
} //by this point, factWrapper was either found OR created successfully.

//now get the specific factory and state for Cal3D to set everything
csRef state3d = scfQueryInterface (factWrapper->GetMeshObjectFactory());
if (state3d == 0) {
Err::NonfatalError(“EuLoader failed to load factory state as iSpriteCal2DFactoryState\n”);
return meshWrapper;
}
//test for now – create girl factory
//this IS DONE in CS’s loader – does each sprcal3d factory actually NEED to create this “dummy” ?
if (!state3d->Create(“dummy”)) {
Err::NonfatalError(“EuLoader: failed to create dummy for sprcal3d factory\n”);
return meshWrapper;
}
//parameters for sprCal3dFactory state – to take them from SMGData! TO DO TO CHANGE
float scale = 1.0;
float ax, ay, az, angle;
csVector3 translation(0.0,0.0,0.0);
int flags = LOADER_FLIP_WINDING; //wtf IS this??
bool rotXAxis = true; //default apparently
bool flipTextures = false; //default apparently
if (rotXAxis)
flags = flags | LOADER_ROTATE_X_AXIS;
else
flags = flags & (~LOADER_ROTATE_X_AXIS); //ugh.
if (flipTextures)
flags = flags | LOADER_INVERT_V_COORD;
else
flags = flags & (~LOADER_INVERT_V_COORD);

//skeleton
char skelFile[] = “/this/clientcache/girl_skel.csf”;
if (! state3d->LoadCoreSkeleton(psengine->GetVFS(), skelFile) ){
Err::NonfatalError(“Failed to load the core skeleton file girl_skel.csf\n”);
return meshWrapper;
}
char idleFile[] = “/this/clientcache/girl_idle.caf”;
char idleName[] = “idle”;
char walkFile[] = “/this/clientcache/girl_idle.caf”;
char walkName[] = “walk”;
int itype = iSpriteCal3DState::C3D_ANIM_TYPE_IDLE;
int wtype = iSpriteCal3DState::C3D_ANIM_TYPE_TRAVEL;

//see p 287/644 in cs manual
float base_vel = 0; //”speed of translation which should be used when the animation is played alone”
float min_vel = 0; //min and max velocities are used by the blender to achieve specific, desired velocities
float max_vel = 0;
int max_interval = 30; //max_random – those give the interval (in seconds) for idle override actions
int min_interval = 10; //min_random
int idle_pct = 0; //probability (%) of this action being the override action
bool lock = false; //??

int animID = state3d->LoadCoreAnimation(psengine->GetVFS(), idleFile,
idleName, itype, base_vel,
min_vel, max_vel, min_interval, max_interval,
idle_pct, lock, flags);
if (animID < 0) { Err::NonfatalError("EuLoader: failed to load core animation\n"); return meshWrapper; } base_vel = 2; max_vel = 3; animID = state3d->LoadCoreAnimation(psengine->GetVFS(), walkFile,
walkName, wtype, base_vel,
min_vel, max_vel, min_interval, max_interval,
idle_pct, lock, flags);
if (animID < 0) { Err::NonfatalError("EuLoader: failed to load core animation\n"); return meshWrapper; } //meshes aka "attachable parts of the model" //head char mfile[] = "/this/clientcache/girl_head.cmf"; char hname[] = "Head"; char mname[] = "/this/clientcache/fskin.dds"; // char mname[] = "fskin.dds"; csRef mtex = GetTexture(mname); //to create/get a BASIC material with this name rather than all the shader mess.
csRef mw = GetMaterial(mname); //after the GetTexture call, this should always end up with the basic wrapper-material on the corresponding texture… such is CS.

if (mw == 0) {
Err::NonfatalError(“FAILED to obtain material fskin.dds\n”);
return meshWrapper;
}
bool attach = true; //default value
state3d->AddCoreMaterial(mw); //done via LoadMaterialTag in cs/plugins/mesh/sprcal3d/persist/sprcal3dldr.cpp
if (state3d->LoadCoreMesh(psengine->GetVFS(), mfile, hname, attach, mw, flags) < 0) { Err::NonfatalError("FAILED to loadcoremesh\n"); return meshWrapper; } //hair char ghrfile[] = "/this/clientcache/girl_hair.cmf"; char ghrname[] = "Hair"; if (state3d->LoadCoreMesh(psengine->GetVFS(), ghrfile, ghrname, attach, mw, flags) < 0) { Err::NonfatalError("FAILED to loadcoremesh\n"); return meshWrapper; } //arm(s) char glalfile[] = "/this/clientcache/girl_arm.cmf"; char glalname[] = "Arm"; if (state3d->LoadCoreMesh(psengine->GetVFS(), glalfile, glalname, attach, mw, flags) < 0) { Err::NonfatalError("FAILED to loadcoremesh\n"); return meshWrapper; } //hand(s) char ghhfile[] = "/this/clientcache/girl_hand.cmf"; char ghhname[] = "Hand"; if (state3d->LoadCoreMesh(psengine->GetVFS(), ghhfile, ghhname, attach, mw, flags) < 0) { Err::NonfatalError("FAILED to loadcoremesh\n"); return meshWrapper; } //torso char gtfile[] = "/this/clientcache/girl_torso.cmf"; char gtname[] = "Torso"; if (state3d->LoadCoreMesh(psengine->GetVFS(), gtfile, gtname, attach, mw, flags) < 0) { Err::NonfatalError("FAILED to loadcoremesh\n"); return meshWrapper; } //legs char glfile[] = "/this/clientcache/girl_legs.cmf"; char glname[] = "Legs"; if (state3d->LoadCoreMesh(psengine->GetVFS(), glfile, glname, attach, mw, flags) < 0) { Err::NonfatalError("FAILED to loadcoremesh\n"); return meshWrapper; } //foot char gffile[] = "/this/clientcache/girl_foot.cmf"; char gffname[] = "Foot"; if (state3d->LoadCoreMesh(psengine->GetVFS(), gffile, gffname, attach, mw, flags) < 0) { Err::NonfatalError("FAILED to loadcoremesh\n"); return meshWrapper; } //default values - hardtransform ax = 0.0; ay = 1.0; //this turns it to face away from the camera/viewer as it were az = 0.0; angle = 180.0; csMatrix3 rotation(ax,ay,az,angle*TWO_PI/360); csReversibleTransform rt(rotation,translation); factWrapper->GetMeshObjectFactory()->HardTransform(rt);

angle = 90.0;
ay = 0.0;
ax = -1.0;
csMatrix3 rotation1(ax,ay,az,angle*TWO_PI/360);
csReversibleTransform rt1(rotation1,translation);
factWrapper->GetMeshObjectFactory()->HardTransform(rt1);

//rescale if needed
if (scale != 0)
state3d->RescaleFactory(scale); //this ALSO calls the calculateallboneboundingboxes
else
state3d->CalculateAllBoneBoundingBoxes();
// Wrapup cal3d initialization
state3d->BindMaterials();

//and now create and register the mesh itself
meshWrapper = psengine->GetEngine()->CreateMeshWrapper (factWrapper, name, s, csVector3 (x,y,z));
if (meshWrapper == 0) {
Err::NonfatalError(“EuLoader:GetMesh failed to create the requested mesh despite having its factory wrapper!\n”);
return meshWrapper;
}

meshWrapper->SetZBufMode(CS_ZBUF_USE); //DO read so that the zbuffer gets updated…ugh;only write to the z-buffer but do not read
meshWrapper->GetMeshObject()->SetMixMode(CS_FX_COPY); //this should be default for most (?); see p321 (333) in cs manual re mixmodes and blending
meshWrapper->SetRenderPriority(psengine->GetEngine()->GetObjectRenderPriority());

psengine->GetEngine()->PrecacheMesh(meshWrapper); //hmm?

//initialize collision detection for this entity
csRef cdsys = csQueryRegistry (psengine->GetObjectRegistry());
csColliderHelper::InitializeCollisionWrapper(cdsys, meshWrapper);

return meshWrapper;
}

Finally, the sector itself with lights and then various tests, some still in, some commented out (although the working code is still there otherwise):

bool EuLoader::LoadSector(uint32_t id) {
/*****retrieve name of this sector and check if it is already loaded*****/
char sname[MAX_STR_LEN+1];
unsigned int lsname = MAX_STR_LEN;

//get name of this sector and null-terminate it
if (!psengine->SMG.GetName(id, sname, &lsname))
return false;
sname[lsname] = ‘\0’;

//check if sector exists
iSector *s = psengine->GetEngine()->FindSector(sname);
if (s != NULL) {
return true; //sector already loaded
}

/****create the sector as an empty entity with default overall values****/
s = psengine->GetEngine()->CreateSector(sname);
if (s == NULL) {
//this should never happen.
Err::NonfatalError(“EuLoader::LoadSector failed to create the requested sector.”);
return false; //failed to create a sector
}

//set the visibility culler; NB: smgdata provides either defaults or fresh data
s->SetVisibilityCullerPlugin(DEFAULT_CULLER); //default culler defined in smgdata.h
float r,g,b;
//set the ambient light
if (psengine->SMG.GetAmbient(id, &r, &g, &b))
s->SetDynamicAmbientLight(csColor(r,g,b));
//set the fog, if any – not really, this was/is only in water (?)
// float density;
// if (psengine->SMG.GetFog(id, &r, &g, &b, &density))
// s->SetFog(density, csColor(r,g,b));

//load the lights, if any
uint32_t lindex = 1;
float radius, x, y, z;
char lightName[MAX_STR_LEN+1];
unsigned int llen = MAX_STR_LEN;
bool dynamic;
csLightDynamicType ltype;

while (psengine->SMG.GetLight(id, lindex, lightName, &llen, &x, &y, &z, &radius, &r, &g, &b, &dynamic)) {
lightName[llen] = ‘\0’;
if (dynamic)
ltype = CS_LIGHT_DYNAMICTYPE_DYNAMIC;
else
ltype = CS_LIGHT_DYNAMICTYPE_STATIC;
csRef light = psengine->GetEngine()->CreateLight(lightName, csVector3(x, y, z), radius, csColor(r,g,b), ltype);
if (!light)
Err::NonfatalError(“EuLoader::LoadSector failed to create light”);
else {
s->GetLights()->Add(light);
}
lindex = lindex + 1;
}

//sky
//using GetMaterial, GetTexture, GetShader above

char skyfactoryname[] = “sky.png.fact”;
char skyobjname[] = “skyobj”;
// csRef skyFactory = GetFactory(skyfactoryname);
if (!GetSky()) { //separate for now because generated sphere aka no direct pixels, texels etc.
// if (skyFactory==0) {
Err::NonfatalError(“EuLoader::LoadSector failed to obtain the sky factory\n”);
return false;
}
else printf (“CHECK: EuLoader got the sky factory fine\n”);

//TO DO : this should create all known objects in the sector
//the moon
/* csRef moonwrap = GetMesh(901);
if (moonwrap == 0) {
Err::NonfatalError(“NO MOON!!!\n”);
return false;
}
*/

//the terrain
csRef twrap = GetTerrain();
if (twrap == 0) {
Err::NonfatalError(“NO TERRAIN!!!\n”);
return false;
}

/************create particle systems (effects) *********/
// csRef fire = GetFire();
csRef fountain = GetFountain();
// csRef rain = GetRain();
// csRef globe = GetGlobe();

/************create the Water***************/

/************create the Portals*************/
return true;
}

There is of course more to the prototoype/test code and otherwise changes and excisions to legacy code but comments are helpfully in place everywhere and so I don’t feel the need to repeat them here. Despite those added comments and despite the fact that a lot of now-unused code was commented out rather than deleted at this stage, this prototype client is still thousands LOC shorter than the legacy client so there’s some satisfaction to it all at least, that’s the most I can say on it currently. Feel free to ask your questions in the comments box below, as usual.

1. Yeah, I’m writing to my older self as it’s way more useful than writing to my younger self. You should try it too.[↩]
2. See src/client/euloader.h and .cpp for the prototype loaders’ code + src/client/psengine.cpp for its use.[↩]
3. To do the same for different plugins, you need just to replace TERRAIN_PLUGIN with whichever other plugin you want from the list above.[↩]
4. To quote Eminescu’s fitting words here: “O racla mare-i lumea. Stelele-s cuie batute-n ea si soarele-i fereastra la temnita vietii.” (A large shrine is this world. The stars are nails through it and the sun is the window to life’s gaol.) [↩]
5. size_t) elemCount, CS_BUF_STATIC, CS_BUFCOMP_FLOAT, compCount);
   bufPos->CopyInto(posElems, elemCount); //this is the only available method to copy apparently the data from memory/pointer and into the buffer….
   state->AddRenderBuffer(“position”, bufPos);

   //renderbuffers: normal
   //TO DO TO CHANGE retrieve the contents of the buffer from SMGData
   const float normElems[] = {-0.81731617,0.16257210,0.55275124,-0.69649345,0.13852352,0.70403147,-0.71013522,0.00000000,0.70403147,-0.83333844,0.00000000,0.55275124,-0.69649345,0.13852352,-0.70403147,-0.81731617,0.16257210,-0.55275124,-0.83333844,0.00000000,-0.55275124,-0.71013522,0.00000000,-0.70403147,-0.90697956,0.18039490,0.38053530,-0.92474133,0.00000000,0.38053530,-0.54899746,0.10919522,-0.82863855,-0.55977052,0.00000000,-0.82863855,-0.96215707,0.19138157,0.19391461,-0.98098695,0.00000000,0.19391461,-0.38041323,0.07565539,-0.92168951,-0.38785973,0.00000000,-0.92168951,-0.98077333,0.19507432,0.00000000,-1.00000000,0.00000000,0.00000000,-0.38041323,0.07565539,0.92168951,-0.19708854,0.03918577,0.97958314,-0.20096439,0.00000000,0.97958314,-0.38785973,0.00000000,0.92168951,-0.19708854,0.03918577,-0.97958314,-0.20096439,0.00000000,-0.97958314,-0.96215707,0.19138157,-0.19391461,-0.98098695,0.00000000,-0.19391461,-0.54899746,0.10919522,0.82863855,-0.55977052,0.00000000,0.82863855,-0.90697956,0.18039490,-0.38053530,-0.92474133,0.00000000,-0.38053530,-0.35834834,0.14841151,-0.92168951,-0.51713616,0.21420942,-0.82863855,-0.54899746,0.10919522,-0.82863855,-0.38041323,0.07565539,-0.92168951,-0.92385632,0.38267159,0.00000000,-0.90633869,0.37540817,0.19391461,-0.35834834,0.14841151,0.92168951,-0.18564409,0.07690664,0.97958314,-0.18564409,0.07690664,-0.97958314,-0.19708854,0.03918577,-0.97958314,-0.90633869,0.37540817,-0.19391461,-0.51713616,0.21420942,0.82863855,-0.85436565,0.35389262,-0.38053530,-0.65605640,0.27173680,0.70403147,-0.76989043,0.31888792,-0.55275124,-0.76989043,0.31888792,0.55275124,-0.65605640,0.27173680,-0.70403147,-0.85436565,0.35386211,0.38053530,-0.51713616,0.21420942,-0.82863855,-0.59044158,0.39451277,-0.70403147,-0.69289225,0.46296579,-0.55275124,-0.76989043,0.31888792,-0.55275124,-0.65605640,0.27173680,-0.70403147,-0.76891387,0.51374859,0.38053530,-0.69289225,0.46296579,0.55275124,-0.76989043,0.31888792,0.55275124,-0.85436565,0.35386211,0.38053530,-0.46540728,0.31098360,-0.82863855,-0.81566823,0.54499954,0.19391461,-0.90633869,0.37540817,0.19391461,-0.32248908,0.21549119,-0.92168951,-0.83144629,0.55555892,0.00000000,-0.92385632,0.38267159,0.00000000,-0.32248908,0.21549119,0.92168951,-0.16708884,0.11163671,0.97958314,-0.16708884,0.11163671,-0.97958314,-0.81566823,0.54499954,-0.19391461,-0.90633869,0.37540817,-0.19391461,-0.46540728,0.31098360,0.82863855,-0.51713616,0.21420942,0.82863855,-0.76891387,0.51374859,-0.38053530,-0.85436565,0.35389262,-0.38053530,-0.59044158,0.39451277,0.70403147,-0.65605640,0.27173680,0.70403147,-0.65388960,0.65388960,-0.38053530,-0.69365519,0.69365519,-0.19391461,-0.50212103,0.50212103,0.70403147,-0.39579454,0.39579454,0.82863855,-0.58925140,0.58925140,-0.55275124,-0.58925140,0.58925140,0.55275124,-0.50212103,0.50212103,-0.70403147,-0.65388960,0.65388960,0.38053530,-0.39579454,0.39579454,-0.82863855,-0.69365519,0.69365519,0.19391461,-0.27426985,0.27426985,-0.92168951,-0.70708334,0.70708334,0.00000000,-0.27426985,0.27426985,0.92168951,-0.14209418,0.14209418,0.97958314,-0.14209418,0.14209418,-0.97958314,-0.21549119,0.32248908,-0.92168951,-0.31098360,0.46540728,-0.82863855,-0.55555892,0.83144629,0.00000000,-0.54499954,0.81566823,0.19391461,-0.21549119,0.32248908,0.92168951,-0.11163671,0.16708884,0.97958314,-0.11163671,0.16708884,-0.97958314,-0.54499954,0.81566823,-0.19391461,-0.31098360,0.46540728,0.82863855,-0.51374859,0.76891387,-0.38053530,-0.39451277,0.59044158,0.70403147,-0.46296579,0.69289225,-0.55275124,-0.46296579,0.69289225,0.55275124,-0.39451277,0.59044158,-0.70403147,-0.51374859,0.76891387,0.38053530,-0.31888792,0.76989043,0.55275124,-0.27173680,0.65605640,0.70403147,-0.27173680,0.65605640,-0.70403147,-0.31888792,0.76989043,-0.55275124,-0.35386211,0.85436565,0.38053530,-0.21420942,0.51713616,-0.82863855,-0.37540817,0.90633869,0.19391461,-0.14841151,0.35834834,-0.92168951,-0.38267159,0.92385632,0.00000000,-0.14841151,0.35834834,0.92168951,-0.07690664,0.18564409,0.97958314,-0.07690664,0.18564409,-0.97958314,-0.37540817,0.90633869,-0.19391461,-0.21420942,0.51713616,0.82863855,-0.35386211,0.85436565,-0.38053530,-0.19138157,0.96215707,-0.19391461,-0.19507432,0.98077333,0.00000000,-0.10919522,0.54899746,0.82863855,-0.07565539,0.38041323,0.92168951,-0.18039490,0.90697956,-0.38053530,-0.13852352,0.69649345,0.70403147,-0.16257210,0.81731617,-0.55275124,-0.16257210,0.81731617,0.55275124,-0.13852352,0.69649345,-0.70403147,-0.18039490,0.90697956,0.38053530,-0.10919522,0.54899746,-0.82863855,-0.19138157,0.96215707,0.19391461,-0.07565539,0.38041323,-0.92168951,-0.03918577,0.19708854,0.97958314,-0.03918577,0.19708854,-0.97958314,0.00000000,0.55977052,-0.82863855,0.00000000,0.71013522,-0.70403147,0.00000000,0.98098695,0.19391461,0.00000000,0.92474133,0.38053530,0.00000000,0.38785973,-0.92168951,0.00000000,1.00000000,0.00000000,0.00000000,0.38785973,0.92168951,0.00000000,0.20096439,0.97958314,0.00000000,0.20096439,-0.97958314,0.00000000,0.98098695,-0.19391461,0.00000000,0.55977052,0.82863855,0.00000000,0.92474133,-0.38053530,0.00000000,0.71013522,0.70403147,0.00000000,0.83333844,-0.55275124,0.00000000,0.83333844,0.55275124,0.16257210,0.81731617,-0.55275124,0.18039490,0.90697956,-0.38053530,0.16257210,0.81731617,0.55275124,0.13852352,0.69649345,0.70403147,0.13852352,0.69649345,-0.70403147,0.18039490,0.90697956,0.38053530,0.10919522,0.54899746,-0.82863855,0.19138157,0.96215707,0.19391461,0.07565539,0.38041323,-0.92168951,0.19507432,0.98077333,0.00000000,0.07565539,0.38041323,0.92168951,0.03918577,0.19708854,0.97958314,0.03918577,0.19708854,-0.97958314,0.19138157,0.96215707,-0.19391461,0.10919522,0.54899746,0.82863855,0.07690664,0.18564409,-0.97958314,0.14841151,0.35834834,-0.92168951,0.37540817,0.90633869,-0.19391461,0.38267159,0.92385632,0.00000000,0.21420942,0.51713616,0.82863855,0.14841151,0.35834834,0.92168951,0.35389262,0.85436565,-0.38053530,0.27173680,0.65605640,0.70403147,0.31888792,0.76989043,-0.55275124,0.31888792,0.76989043,0.55275124,0.27173680,0.65605640,-0.70403147,0.35389262,0.85436565,0.38053530,0.21420942,0.51713616,-0.82863855,0.37540817,0.90633869,0.19391461,0.07687613,0.18564409,0.97958314,0.31098360,0.46540728,-0.82863855,0.39451277,0.59044158,-0.70403147,0.54499954,0.81566823,0.19391461,0.51374859,0.76891387,0.38053530,0.21549119,0.32248908,-0.92168951,0.55555892,0.83144629,0.00000000,0.21549119,0.32248908,0.92168951,0.11163671,0.16708884,0.97958314,0.11163671,0.16708884,-0.97958314,0.54499954,0.81566823,-0.19391461,0.31098360,0.46540728,0.82863855,0.51374859,0.76891387,-0.38053530,0.39451277,0.59044158,0.70403147,0.46296579,0.69289225,-0.55275124,0.46296579,0.69289225,0.55275124,0.58925140,0.58925140,-0.55275124,0.65388960,0.65388960,-0.38053530,0.58925140,0.58925140,0.55275124,0.50212103,0.50212103,0.70403147,0.50212103,0.50212103,-0.70403147,0.65388960,0.65388960,0.38053530,0.39579454,0.39579454,-0.82863855,0.69365519,0.69365519,0.19391461,0.27426985,0.27426985,-0.92168951,0.70708334,0.70708334,0.00000000,0.27426985,0.27426985,0.92168951,0.14209418,0.14209418,0.97958314,0.14209418,0.14209418,-0.97958314,0.69365519,0.69365519,-0.19391461,0.39579454,0.39579454,0.82863855,0.16708884,0.11163671,-0.97958314,0.32248908,0.21549119,-0.92168951,0.81566823,0.54499954,-0.19391461,0.83144629,0.55555892,0.00000000,0.46540728,0.31098360,0.82863855,0.32248908,0.21549119,0.92168951,0.76891387,0.51374859,-0.38053530,0.59044158,0.39451277,0.70403147,0.69289225,0.46296579,-0.55275124,0.69289225,0.46296579,0.55275124,0.59044158,0.39451277,-0.70403147,0.76891387,0.51374859,0.38053530,0.46540728,0.31098360,-0.82863855,0.81566823,0.54499954,0.19391461,0.16708884,0.11163671,0.97958314,0.51713616,0.21420942,-0.82863855,0.65605640,0.27173680,-0.70403147,0.90633869,0.37540817,0.19391461,0.85436565,0.35386211,0.38053530,0.35834834,0.14841151,-0.92168951,0.92385632,0.38267159,0.00000000,0.35834834,0.14841151,0.92168951,0.18564409,0.07690664,0.97958314,0.18564409,0.07690664,-0.97958314,0.90633869,0.37540817,-0.19391461,0.51713616,0.21420942,0.82863855,0.85436565,0.35386211,-0.38053530,0.65605640,0.27173680,0.70403147,0.76989043,0.31888792,-0.55275124,0.76989043,0.31888792,0.55275124,0.69649345,0.13852352,0.70403147,0.54899746,0.10919522,0.82863855,0.81731617,0.16257210,-0.55275124,0.90697956,0.18039490,-0.38053530,0.81731617,0.16257210,0.55275124,0.69649345,0.13852352,-0.70403147,0.90697956,0.18039490,0.38053530,0.54899746,0.10919522,-0.82863855,0.96215707,0.19138157,0.19391461,0.38041323,0.07565539,-0.92168951,0.98077333,0.19507432,0.00000000,0.38041323,0.07565539,0.92168951,0.19708854,0.03918577,0.97958314,0.19708854,0.03918577,-0.97958314,0.96215707,0.19138157,-0.19391461,1.00000000,0.00000000,0.00000000,0.98098695,0.00000000,0.19391461,0.38785973,0.00000000,0.92168951,0.20096439,0.00000000,0.97958314,0.20096439,0.00000000,-0.97958314,0.38785973,0.00000000,-0.92168951,0.98098695,0.00000000,-0.19391461,0.55977052,0.00000000,0.82863855,0.92474133,0.00000000,-0.38053530,0.71013522,0.00000000,0.70403147,0.83333844,0.00000000,-0.55275124,0.83333844,0.00000000,0.55275124,0.71013522,0.00000000,-0.70403147,0.92474133,0.00000000,0.38053530,0.55977052,0.00000000,-0.82863855,0.69649345,-0.13852352,-0.70403147,0.81731617,-0.16257210,-0.55275124,0.90697956,-0.18039490,0.38053530,0.81731617,-0.16257210,0.55275124,0.54899746,-0.10919522,-0.82863855,0.96215707,-0.19138157,0.19391461,0.38041323,-0.07565539,-0.92168951,0.98077333,-0.19507432,0.00000000,0.38041323,-0.07565539,0.92168951,0.19708854,-0.03918577,0.97958314,0.19708854,-0.03918577,-0.97958314,0.96215707,-0.19138157,-0.19391461,0.54899746,-0.10919522,0.82863855,0.90697956,-0.18039490,-0.38053530,0.69649345,-0.13852352,0.70403147,0.85436565,-0.35386211,-0.38053530,0.90633869,-0.37540817,-0.19391461,0.65605640,-0.27173680,0.70403147,0.51713616,-0.21420942,0.82863855,0.76989043,-0.31888792,-0.55275124,0.76989043,-0.31888792,0.55275124,0.65605640,-0.27173680,-0.70403147,0.85436565,-0.35386211,0.38053530,0.51713616,-0.21420942,-0.82863855,0.90633869,-0.37540817,0.19391461,0.35834834,-0.14841151,-0.92168951,0.92385632,-0.38267159,0.00000000,0.35834834,-0.14841151,0.92168951,0.18564409,-0.07690664,0.97958314,0.18564409,-0.07690664,-0.97958314,0.32248908,-0.21549119,-0.92168951,0.46540728,-0.31098360,-0.82863855,0.83144629,-0.55555892,0.00000000,0.81566823,-0.54499954,0.19391461,0.32248908,-0.21549119,0.92168951,0.16708884,-0.11163671,0.97958314,0.16708884,-0.11163671,-0.97958314,0.81566823,-0.54499954,-0.19391461,0.46540728,-0.31098360,0.82863855,0.76891387,-0.51374859,-0.38053530,0.59044158,-0.39451277,0.70403147,0.69289225,-0.46296579,-0.55275124,0.69289225,-0.46296579,0.55275124,0.59044158,-0.39451277,-0.70403147,0.76891387,-0.51374859,0.38053530,0.46540728,-0.31098360,-0.82863855,0.50212103,-0.50212103,-0.70403147,0.58925140,-0.58925140,-0.55275124,0.65388960,-0.65388960,0.38053530,0.58925140,-0.58925140,0.55275124,0.39579454,-0.39579454,-0.82863855,0.69365519,-0.69365519,0.19391461,0.27426985,-0.27426985,-0.92168951,0.32248908,-0.21549119,-0.92168951,0.70708334,-0.70708334,0.00000000,0.27426985,-0.27426985,0.92168951,0.14209418,-0.14209418,0.97958314,0.14209418,-0.14209418,-0.97958314,0.16708884,-0.11163671,-0.97958314,0.69365519,-0.69365519,-0.19391461,0.39579454,-0.39579454,0.82863855,0.65388960,-0.65388960,-0.38053530,0.50212103,-0.50212103,0.70403147,0.51374859,-0.76891387,-0.38053530,0.54499954,-0.81566823,-0.19391461,0.39451277,-0.59044158,0.70403147,0.31098360,-0.46540728,0.82863855,0.46296579,-0.69289225,-0.55275124,0.46296579,-0.69289225,0.55275124,0.39451277,-0.59044158,-0.70403147,0.51374859,-0.76891387,0.38053530,0.31098360,-0.46540728,-0.82863855,0.54499954,-0.81566823,0.19391461,0.21549119,-0.32248908,-0.92168951,0.55555892,-0.83144629,0.00000000,0.21549119,-0.32248908,0.92168951,0.11163671,-0.16708884,0.97958314,0.11163671,-0.16708884,-0.97958314,0.14841151,-0.35834834,-0.92168951,0.21420942,-0.51713616,-0.82863855,0.38267159,-0.92385632,0.00000000,0.37540817,-0.90633869,0.19391461,0.14841151,-0.35834834,0.92168951,0.07687613,-0.18564409,0.97958314,0.07690664,-0.18564409,-0.97958314,0.37540817,-0.90633869,-0.19391461,0.21420942,-0.51713616,0.82863855,0.35386211,-0.85436565,-0.38053530,0.27173680,-0.65605640,0.70403147,0.31888792,-0.76989043,-0.55275124,0.31888792,-0.76989043,0.55275124,0.27173680,-0.65605640,-0.70403147,0.35386211,-0.85436565,0.38053530,0.13852352,-0.69649345,-0.70403147,0.16257210,-0.81731617,-0.55275124,0.18039490,-0.90697956,0.38053530,0.16257210,-0.81731617,0.55275124,0.10919522,-0.54899746,-0.82863855,0.19138157,-0.96215707,0.19391461,0.07565539,-0.38041323,-0.92168951,0.19507432,-0.98077333,0.00000000,0.07565539,-0.38041323,0.92168951,0.03918577,-0.19708854,0.97958314,0.03918577,-0.19708854,-0.97958314,0.19138157,-0.96215707,-0.19391461,0.10919522,-0.54899746,0.82863855,0.18039490,-0.90697956,-0.38053530,0.13852352,-0.69649345,0.70403147,0.00000000,-0.55977052,0.82863855,0.00000000,-0.38785973,0.92168951,0.00000000,-0.92474133,-0.38053530,0.00000000,-0.98098695,-0.19391461,0.00000000,-0.71013522,0.70403147,0.00000000,-0.83333844,-0.55275124,0.00000000,-0.83333844,0.55275124,0.00000000,-0.71013522,-0.70403147,0.00000000,-0.92474133,0.38053530,0.00000000,-0.55977052,-0.82863855,0.00000000,-0.98098695,0.19391461,0.00000000,-0.38785973,-0.92168951,0.00000000,-0.99996948,0.00000000,0.00000000,-0.20096439,0.97958314,0.00000000,-0.20096439,-0.97958314,-0.19138157,-0.96215707,0.19391461,-0.18039490,-0.90697956,0.38053530,-0.07565539,-0.38041323,-0.92168951,-0.10919522,-0.54899746,-0.82863855,-0.19507432,-0.98077333,0.00000000,-0.07565539,-0.38041323,0.92168951,-0.03918577,-0.19708854,0.97958314,-0.03918577,-0.19708854,-0.97958314,-0.19138157,-0.96215707,-0.19391461,-0.10919522,-0.54899746,0.82863855,-0.18039490,-0.90697956,-0.38053530,-0.13852352,-0.69649345,0.70403147,-0.16257210,-0.81731617,-0.55275124,-0.16257210,-0.81731617,0.55275124,-0.13852352,-0.69649345,-0.70403147,-0.31888792,-0.76989043,-0.55275124,-0.35389262,-0.85436565,-0.38053530,-0.31888792,-0.76989043,0.55275124,-0.27173680,-0.65605640,0.70403147,-0.27173680,-0.65605640,-0.70403147,-0.35389262,-0.85436565,0.38053530,-0.21420942,-0.51713616,-0.82863855,-0.37540817,-0.90633869,0.19391461,-0.14841151,-0.35834834,-0.92168951,-0.38267159,-0.92385632,0.00000000,-0.14841151,-0.35834834,0.92168951,-0.07690664,-0.18564409,0.97958314,-0.07690664,-0.18564409,-0.97958314,-0.37540817,-0.90633869,-0.19391461,-0.21420942,-0.51713616,0.82863855,-0.11163671,-0.16708884,-0.97958314,-0.21549119,-0.32248908,-0.92168951,-0.54499954,-0.81566823,-0.19391461,-0.55555892,-0.83144629,0.00000000,-0.31098360,-0.46540728,0.82863855,-0.21549119,-0.32248908,0.92168951,-0.51374859,-0.76891387,-0.38053530,-0.39451277,-0.59044158,0.70403147,-0.46296579,-0.69289225,-0.55275124,-0.46296579,-0.69289225,0.55275124,-0.39451277,-0.59044158,-0.70403147,-0.51374859,-0.76891387,0.38053530,-0.31098360,-0.46540728,-0.82863855,-0.54499954,-0.81566823,0.19391461,-0.11163671,-0.16708884,0.97958314,-0.39579454,-0.39579454,-0.82863855,-0.50212103,-0.50212103,-0.70403147,-0.69365519,-0.69365519,0.19391461,-0.65388960,-0.65388960,0.38053530,-0.27426985,-0.27426985,-0.92168951,-0.70708334,-0.70708334,0.00000000,-0.27426985,-0.27426985,0.92168951,-0.14209418,-0.14209418,0.97958314,-0.14209418,-0.14209418,-0.97958314,-0.69365519,-0.69365519,-0.19391461,-0.39579454,-0.39579454,0.82863855,-0.65388960,-0.65388960,-0.38053530,-0.50212103,-0.50212103,0.70403147,-0.58925140,-0.58925140,-0.55275124,-0.58925140,-0.58925140,0.55275124,-0.69289225,-0.46296579,-0.55275124,-0.76891387,-0.51374859,-0.38053530,-0.69289225,-0.46296579,0.55275124,-0.59044158,-0.39451277,0.70403147,-0.59044158,-0.39451277,-0.70403147,-0.76891387,-0.51374859,0.38053530,-0.46540728,-0.31098360,-0.82863855,-0.81566823,-0.54499954,0.19391461,-0.32248908,-0.21549119,-0.92168951,-0.83144629,-0.55555892,0.00000000,-0.32248908,-0.21549119,0.92168951,-0.16708884,-0.11163671,0.97958314,-0.16708884,-0.11163671,-0.97958314,-0.81566823,-0.54499954,-0.19391461,-0.46540728,-0.31098360,0.82863855,-0.18564409,-0.07690664,-0.97958314,-0.35834834,-0.14841151,-0.92168951,-0.90633869,-0.37540817,-0.19391461,-0.92385632,-0.38267159,0.00000000,-0.51713616,-0.21420942,0.82863855,-0.35834834,-0.14841151,0.92168951,-0.85436565,-0.35386211,-0.38053530,-0.65605640,-0.27173680,0.70403147,-0.76989043,-0.31888792,-0.55275124,-0.76989043,-0.31888792,0.55275124,-0.65605640,-0.27173680,-0.70403147,-0.85436565,-0.35386211,0.38053530,-0.51713616,-0.21420942,-0.82863855,-0.90633869,-0.37540817,0.19391461,-0.18564409,-0.07690664,0.97958314,-0.54899746,-0.10919522,-0.82863855,-0.69649345,-0.13852352,-0.70403147,-0.96215707,-0.19138157,0.19391461,-0.90697956,-0.18039490,0.38053530,-0.38041323,-0.07565539,-0.92168951,-0.98077333,-0.19507432,0.00000000,-0.38041323,-0.07565539,0.92168951,-0.19708854,-0.03918577,0.97958314,-0.19708854,-0.03918577,-0.97958314,-0.96215707,-0.19138157,-0.19391461,-0.54899746,-0.10919522,0.82863855,-0.90697956,-0.18039490,-0.38053530,-0.69649345,-0.13852352,0.70403147,-0.81731617,-0.16257210,-0.55275124,-0.81731617,-0.16257210,0.55275124,0.00000000,0.00000000,1.00000000,-0.20096439,0.00000000,-0.97958314,0.00000000,0.00000000,-1.00000000,0.14209418,-0.14209418,-0.97958314,0.11163671,-0.16708884,-0.97958314,0.07690664,-0.18564409,-0.97958314,0.03918577,-0.19708854,-0.97958314,0.00000000,-0.20096439,-0.97958314,-0.03918577,-0.19708854,-0.97958314,-0.07690664,-0.18564409,-0.97958314,-0.11163671,-0.16708884,-0.97958314,-0.14209418,-0.14209418,-0.97958314,-0.16708884,-0.11163671,-0.97958314,-0.18564409,-0.07690664,-0.97958314,-0.19708854,-0.03918577,-0.97958314};
   unsigned int bufNormLen = sizeof(normElems) / sizeof(normElems[0]);
   compCount = 3;
   elemCount = bufNormLen / compCount;
   csRef bufNormal = csRenderBuffer::CreateRenderBuffer((size_t) elemCount, CS_BUF_STATIC, CS_BUFCOMP_FLOAT, compCount);
   bufNormal->CopyInto(normElems, elemCount); //this is the only available method to copy apparently the data from memory/pointer and into the buffer….
   state->AddRenderBuffer(“normal”, bufNormal);

   //renderbuffers: texture coordinate
   //TO DO TO CHANGE retrieve the contents of the buffer from SMGData
   const float texElems[] = {0.061220,0.172764,0.080647,0.112472,0.135028,0.130630,0.107782,0.187599,0.025234,0.606227,0.028936,0.544141,0.054891,0.552189,0.048386,0.613417,0.051220,0.234138,0.092315,0.247027,0.020989,0.668252,0.040867,0.674443,0.044778,0.295915,0.081872,0.307471,0.015638,0.730152,0.031336,0.735069,0.039986,0.357878,0.073887,0.368405,0.356456,0.039893,0.466638,0.087931,0.448527,0.094534,0.333915,0.062714,0.008043,0.791799,0.017784,0.794887,0.036021,0.419934,0.067156,0.429587,0.138246,0.056841,0.196016,0.081502,0.032439,0.482033,0.060989,0.490881,0.999822,0.727971,1.001015,0.665501,1.020989,0.668252,1.015638,0.730152,0.005301,0.353109,0.006404,0.290637,0.447682,0.023355,0.488610,0.084811,0.998129,0.790429,1.008043,0.791799,0.004397,0.415586,0.034176,0.041338,0.003585,0.478066,0.015221,0.103359,0.002796,0.540546,0.010297,0.165735,0.001965,0.603025,0.007907,0.228175,0.001015,0.665501,0.978668,0.603891,0.976612,0.541519,1.002796,0.540546,1.001965,0.603025,0.964040,0.229802,0.958226,0.167665,1.010297,0.165735,1.007907,0.228175,0.981020,0.666245,0.967719,0.292073,1.006404,0.290637,0.983979,0.728561,0.970429,0.354404,1.005301,0.353109,0.595877,0.028785,0.511577,0.085667,0.988176,0.790799,0.972657,0.416765,1.004397,0.415586,0.904978,0.046051,1.034176,0.041338,0.974661,0.479141,1.003585,0.478066,0.946455,0.105901,1.015221,0.103359,0.945911,0.485214,0.941200,0.423411,0.885287,0.119299,0.823807,0.066742,0.950535,0.547029,0.908873,0.178221,0.955439,0.608804,0.921503,0.238833,0.961079,0.670468,0.929787,0.300102,0.968202,0.731911,0.936013,0.361679,0.647807,0.049313,0.532175,0.090357,0.978319,0.792904,0.952583,0.737970,0.941260,0.678089,0.902582,0.374475,0.893448,0.314080,0.659037,0.072825,0.548166,0.098187,0.968695,0.796706,0.910362,0.435188,0.777305,0.092963,0.917545,0.496040,0.835902,0.140195,0.924693,0.556900,0.864639,0.195793,0.932358,0.617639,0.881675,0.254305,0.825891,0.218332,0.796385,0.165337,0.909472,0.630184,0.899158,0.570827,0.845097,0.274835,0.921611,0.688967,0.859136,0.333015,0.937209,0.746646,0.870446,0.392059,0.657607,0.096698,0.558962,0.108237,0.959449,0.802140,0.880347,0.451545,0.745300,0.120662,0.889690,0.511206,0.851201,0.471783,0.839664,0.413538,0.719784,0.148411,0.650899,0.120164,0.862365,0.530171,0.763610,0.192518,0.873929,0.588406,0.791803,0.243985,0.886774,0.646148,0.811613,0.298972,0.902153,0.702905,0.826879,0.355773,0.922167,0.757809,0.565047,0.119643,0.950742,0.809109,0.882867,0.719668,0.864196,0.665186,0.796418,0.381227,0.780693,0.325402,0.907539,0.771302,0.810089,0.437969,0.641452,0.142821,0.567268,0.131697,0.942759,0.817485,0.822818,0.495127,0.697492,0.175520,0.835477,0.552316,0.735147,0.220347,0.848920,0.609159,0.761226,0.271289,0.823944,0.632559,0.808822,0.576971,0.733105,0.299125,0.709376,0.247895,0.841590,0.686906,0.751695,0.353000,0.863688,0.738990,0.767340,0.408333,0.893414,0.786940,0.781429,0.464419,0.630338,0.164376,0.566457,0.143852,0.935719,0.827110,0.794966,0.520772,0.676891,0.201534,0.929896,0.837788,0.879896,0.804521,0.767315,0.547906,0.753309,0.491989,0.657173,0.226088,0.618082,0.184573,0.782092,0.603438,0.685241,0.274482,0.798714,0.658036,0.706593,0.326609,0.818709,0.710884,0.723996,0.380807,0.844487,0.760584,0.739178,0.436135,0.563300,0.155686,0.825051,0.784147,0.795182,0.736662,0.711466,0.463755,0.697047,0.407985,0.867123,0.823829,0.725311,0.519808,0.604972,0.203174,0.558329,0.166865,0.925640,0.849281,0.739467,0.575708,0.637870,0.248852,0.754885,0.630980,0.662044,0.299554,0.772822,0.684979,0.681041,0.353007,0.745717,0.712709,0.726706,0.658807,0.655961,0.377673,0.639317,0.322626,0.770454,0.763731,0.670392,0.433769,0.805029,0.809363,0.683774,0.490356,0.855308,0.844632,0.697011,0.547009,0.591182,0.219956,0.551952,0.177121,0.923407,0.861297,0.710973,0.603325,0.618699,0.269516,0.923791,0.873467,0.844816,0.866689,0.681363,0.629851,0.668005,0.572713,0.599486,0.287785,0.576832,0.234706,0.696970,0.686034,0.616747,0.343256,0.716668,0.740438,0.631000,0.400016,0.743695,0.791492,0.643669,0.457442,0.783837,0.835890,0.655731,0.515122,0.544481,0.186237,0.760428,0.863335,0.713639,0.819179,0.627046,0.537243,0.616615,0.478315,0.836350,0.889729,0.637946,0.596005,0.562017,0.247230,0.536166,0.194033,0.927547,0.885321,0.650183,0.654292,0.580132,0.303378,0.665037,0.711638,0.594131,0.361033,0.684737,0.767195,0.605914,0.419482,0.571358,0.375584,0.560593,0.316042,0.648805,0.791726,0.630288,0.734412,0.580561,0.435564,0.678348,0.845702,0.589070,0.495737,0.732723,0.891172,0.597536,0.555923,0.831424,0.913428,0.606595,0.615943,0.546824,0.257354,0.527209,0.200363,0.935538,0.896247,0.617071,0.675566,0.573887,0.631595,0.567156,0.570413,0.531336,0.264931,0.517784,0.205114,0.948527,0.905466,0.833915,0.937286,0.581872,0.692529,0.540867,0.325557,0.592315,0.752973,0.548386,0.386583,0.607782,0.812401,0.554891,0.447811,0.635028,0.869370,0.560989,0.509119,0.696016,0.918498,0.580647,0.887528,0.561220,0.827236,0.532439,0.517967,0.528936,0.455859,0.638246,0.943159,0.536021,0.580066,0.856456,0.960107,0.539986,0.642122,0.515638,0.269848,0.508043,0.208201,0.966637,0.912069,0.544778,0.704085,0.520989,0.331748,0.551220,0.765862,0.525234,0.393773,0.507907,0.771825,0.506404,0.709363,0.501965,0.396975,0.501015,0.334499,0.510297,0.834265,0.502796,0.459454,0.515221,0.896641,0.503585,0.521934,0.534176,0.958662,0.504397,0.584414,0.947682,0.976645,0.505301,0.646891,0.499822,0.272029,0.498129,0.209572,0.988610,0.915189,1.095877,0.971215,1.404978,0.953949,0.470429,0.645596,0.472657,0.583235,0.483979,0.271440,0.488176,0.209201,1.011577,0.914333,0.467719,0.707927,0.481020,0.333755,0.464040,0.770198,0.478668,0.396109,0.458226,0.832335,0.476612,0.458481,0.446455,0.894099,0.474661,0.520859,0.404978,0.953949,0.385287,0.880701,0.408873,0.821779,0.445911,0.514786,0.450536,0.452971,0.323807,0.933258,0.441200,0.576589,0.147808,0.950687,0.095877,0.971215,0.436013,0.638321,0.468202,0.268089,0.478319,0.207096,0.032175,0.909643,0.011577,0.914333,0.429787,0.699898,0.461079,0.329532,0.421503,0.761167,0.455439,0.391196,0.381676,0.745694,0.393448,0.685920,0.432358,0.382361,0.441260,0.321911,0.364639,0.804207,0.424693,0.443100,0.335902,0.859805,0.417545,0.503960,0.277305,0.907037,0.410362,0.564812,0.159037,0.927175,0.402582,0.625524,0.452583,0.262030,0.468695,0.203294,0.048166,0.901813,0.157607,0.903302,0.245300,0.879339,0.370446,0.607941,0.380347,0.548455,0.437209,0.253354,0.459449,0.197860,0.058962,0.891763,0.359136,0.666985,0.421611,0.311033,0.345098,0.725165,0.409472,0.369816,0.325892,0.781668,0.399158,0.429173,0.296385,0.834663,0.389690,0.488794,0.263610,0.807482,0.291803,0.756015,0.362365,0.469829,0.373930,0.411594,0.219785,0.851589,0.351201,0.528217,0.150899,0.879836,0.339664,0.586461,0.422167,0.242191,0.450742,0.190891,0.065046,0.880358,0.326879,0.644227,0.402153,0.297095,0.311613,0.701028,0.386774,0.353852,0.382867,0.280332,0.407539,0.228698,0.280693,0.674598,0.296418,0.618773,0.364196,0.334814,0.261226,0.728711,0.348920,0.390841,0.235147,0.779654,0.335478,0.447684,0.197492,0.824480,0.322818,0.504873,0.141452,0.857179,0.310089,0.562031,0.442759,0.182515,0.067268,0.868303,0.294966,0.479228,0.308822,0.423029,0.130338,0.835624,0.176891,0.798466,0.281429,0.535580,0.393414,0.213060,0.435719,0.172890,0.066457,0.856148,0.267340,0.591668,0.363688,0.261010,0.251695,0.647001,0.341590,0.313094,0.233105,0.700875,0.323944,0.367441,0.209376,0.752105,0.206593,0.673391,0.223996,0.619193,0.298714,0.341964,0.318709,0.289116,0.185241,0.725518,0.282092,0.396562,0.157173,0.773912,0.267315,0.452094,0.118082,0.815427,0.253309,0.508011,0.379896,0.195479,0.429896,0.162212,0.063300,0.844315,0.239178,0.563865,0.344487,0.239416,0.058329,0.833136,0.104972,0.796826,0.211466,0.536245,0.225311,0.480192,0.325051,0.215853,0.367123,0.176171,0.197047,0.592015,0.295182,0.263338,0.181041,0.646993,0.272822,0.315021,0.162044,0.700446,0.254885,0.369020,0.137870,0.751148,0.239467,0.424292,0.425640,0.150719,0.118699,0.730484,0.139317,0.677374,0.210973,0.396675,0.226706,0.341193,0.091182,0.780045,0.197011,0.452991,0.355308,0.155368,0.423407,0.138704,0.051952,0.822879,0.183774,0.509644,0.305029,0.190637,0.170392,0.566231,0.270454,0.236269,0.155961,0.622327,0.245717,0.287291,0.131000,0.599985,0.143669,0.542558,0.216668,0.259562,0.243695,0.208508,0.116747,0.656744,0.196970,0.313966,0.099486,0.712215,0.181363,0.370149,0.076832,0.765294,0.168005,0.427287,0.344817,0.133311,0.423791,0.126533,0.044481,0.813763,0.155731,0.484878,0.283838,0.164110,0.036166,0.805968,0.062017,0.752771,0.127046,0.462757,0.137946,0.403995,0.260429,0.136665,0.336350,0.110271,0.116615,0.521685,0.213639,0.180821,0.105914,0.580518,0.184737,0.232805,0.094131,0.638967,0.165037,0.288362,0.080132,0.696622,0.150183,0.345707,0.427547,0.114679,0.060593,0.683958,0.071358,0.624416,0.117071,0.324434,0.130288,0.265587,0.046824,0.742647,0.106595,0.384057,0.331424,0.086572,0.435539,0.103753,0.027209,0.799637,0.097536,0.444077,0.232723,0.108828,0.089070,0.504263,0.178349,0.154298,0.080561,0.564436,0.148805,0.208274,0.495265,0.147144,1.017784,0.794887,0.995265,0.852857,1.032175,0.909643,1.048166,0.901813,1.058962,0.891763,1.065046,0.880358,1.067268,0.868303,1.066457,0.856148,1.063300,0.844315,1.058329,0.833136,1.051952,0.822879,1.044481,0.813763,1.036166,0.805968,1.027209,0.799637};
   unsigned int bufTexLen = sizeof(texElems) / sizeof(texElems[0]);
   compCount = 2;
   elemCount = bufTexLen / compCount;
   csRef bufTex = csRenderBuffer::CreateRenderBuffer((size_t) elemCount, CS_BUF_STATIC, CS_BUFCOMP_FLOAT, compCount);
   bufTex->CopyInto(texElems, elemCount); //this is the only available method to copy apparently the data from memory/pointer and into the buffer….
   state->AddRenderBuffer(“texture coordinate”, bufTex);

   //renderbuffers: color
   //TO DO TO CHANGE retrieve the contents of the buffer from SMGData
   const float colorElems[] = {1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1,1.000000,1.000000,1.000000,1};
   unsigned int bufColorLen = sizeof(colorElems) / sizeof(colorElems[0]);
   compCount = 4;
   elemCount = bufColorLen / compCount;
   csRef bufColor = csRenderBuffer::CreateRenderBuffer((size_t) elemCount, CS_BUF_STATIC, CS_BUFCOMP_FLOAT, compCount);
   bufColor->CopyInto(colorElems, elemCount); //this is the only available method to copy apparently the data from memory/pointer and into the buffer….
   state->AddRenderBuffer(“color”, bufColor);

   //triangles
   const float triangles[] = {1,2,3,0,1,3,5,6,7,4,5,7,0,3,9,8,0,9,4,7,11,10,4,11,8,9,13,12,8,13,10,11,15,14,10,15,12,13,17,16,12,17,19,20,21,18,19,21,14,15,23,22,14,23,16,17,25,24,16,25,18,21,27,26,18,27,24,25,29,28,24,29,26,27,2,1,26,2,28,29,6,5,28,6,31,32,33,30,31,33,35,12,16,34,35,16,37,19,18,36,37,18,30,33,39,38,30,39,34,16,24,40,34,24,36,18,26,41,36,26,40,24,28,42,40,28,41,26,1,43,41,1,42,28,5,44,42,5,43,1,0,45,43,0,44,5,4,46,44,4,45,0,8,47,45,8,46,4,10,48,46,10,47,8,12,35,47,12,50,51,52,49,50,52,54,55,56,53,54,56,49,52,31,57,49,31,53,56,59,58,53,59,57,31,30,60,57,30,58,59,62,61,58,62,64,37,36,63,64,36,60,30,38,65,60,38,61,62,67,66,61,67,63,36,69,68,63,69,66,67,71,70,66,71,68,69,73,72,68,73,70,71,51,50,70,51,72,73,55,54,72,55,75,66,70,74,75,70,77,68,72,76,77,72,74,70,50,78,74,50,76,72,54,79,76,54,78,50,49,80,78,49,79,54,53,81,79,53,80,49,57,82,80,57,81,53,58,83,81,58,82,57,60,84,82,60,83,58,61,85,83,61,87,64,63,86,87,63,84,60,65,88,84,65,85,61,66,75,85,66,86,63,68,77,86,68,90,82,84,89,90,84,92,83,85,91,92,85,94,87,86,93,94,86,89,84,88,95,89,88,91,85,75,96,91,75,93,86,77,97,93,77,96,75,74,98,96,74,97,77,76,99,97,76,98,74,78,100,98,78,99,76,79,101,99,79,100,78,80,102,100,80,101,79,81,103,101,81,102,80,82,90,102,82,103,81,83,92,103,83,105,99,101,104,105,101,107,100,102,106,107,102,104,101,103,108,104,103,106,102,90,109,106,90,108,103,92,110,108,92,109,90,89,111,109,89,110,92,91,112,110,91,114,94,93,113,114,93,111,89,95,115,111,95,112,91,96,116,112,96,113,93,97,117,113,97,116,96,98,118,116,98,117,97,99,105,117,99,118,98,100,107,118,100,120,112,116,119,120,116,122,113,117,121,122,117,119,116,118,123,119,118,121,117,105,124,121,105,123,118,107,125,123,107,124,105,104,126,124,104,125,107,106,127,125,106,126,104,108,128,126,108,127,106,109,129,127,109,128,108,110,130,128,110,129,109,111,131,129,111,130,110,112,120,130,112,132,114,113,122,132,113,131,111,115,133,131,115,135,127,129,134,135,129,137,128,130,136,137,130,134,129,131,138,134,131,136,130,120,139,136,120,141,132,122,140,141,122,138,131,133,142,138,133,139,120,119,143,139,119,140,122,121,144,140,121,143,119,123,145,143,123,144,121,124,146,144,124,145,123,125,147,145,125,146,124,126,148,146,126,147,125,127,135,147,127,148,126,128,137,148,128,150,145,147,149,150,147,152,146,148,151,152,148,149,147,135,153,149,135,151,148,137,154,151,137,153,135,134,155,153,134,154,137,136,156,154,136,155,134,138,157,155,138,156,136,139,158,156,139,160,141,140,159,160,140,157,138,142,161,157,142,158,139,143,162,158,143,159,140,144,163,159,144,162,143,145,150,162,145,163,144,146,152,163,146,165,157,161,164,165,161,167,158,162,166,167,162,169,159,163,168,169,163,166,162,150,170,166,150,168,163,152,171,168,152,170,150,149,172,170,149,171,152,151,173,171,151,172,149,153,174,172,153,173,151,154,175,173,154,174,153,155,176,174,155,175,154,156,177,175,156,176,155,157,165,176,157,177,156,158,167,177,158,178,160,159,169,178,159,180,174,176,179,180,176,182,175,177,181,182,177,179,176,165,183,179,165,181,177,167,184,181,167,186,178,169,185,186,169,183,165,164,187,183,164,184,167,166,188,184,166,185,169,168,189,185,168,188,166,170,190,188,170,189,168,171,191,189,171,190,170,172,192,190,172,191,171,173,193,191,173,192,172,174,180,192,174,193,173,175,182,193,175,195,190,192,194,195,192,197,191,193,196,197,193,194,192,180,198,194,180,196,193,182,199,196,182,198,180,179,200,198,179,199,182,181,201,199,181,200,179,183,202,200,183,201,181,184,203,201,184,205,186,185,204,205,185,202,183,187,206,202,187,203,184,188,207,203,188,204,185,189,208,204,189,207,188,190,195,207,190,208,189,191,197,208,191,210,202,206,209,210,206,212,203,207,211,212,207,214,204,208,213,214,208,211,207,195,215,211,195,213,208,197,216,213,197,215,195,194,217,215,194,216,197,196,218,216,196,217,194,198,219,217,198,218,196,199,220,218,199,219,198,200,221,219,200,220,199,201,222,220,201,221,200,202,210,221,202,222,201,203,212,222,203,223,205,204,214,223,204,225,219,221,224,225,221,227,220,222,226,227,222,224,221,210,228,224,210,226,222,212,229,226,212,231,223,214,230,231,214,228,210,209,232,228,209,229,212,211,233,229,211,230,214,213,234,230,213,233,211,215,235,233,215,234,213,216,236,234,216,235,215,217,237,235,217,236,216,218,238,236,218,237,217,219,225,237,219,238,218,220,227,238,220,240,234,236,239,240,236,242,235,237,241,242,237,239,236,238,243,239,238,241,237,225,244,241,225,243,238,227,245,243,227,244,225,224,246,244,224,245,227,226,247,245,226,246,224,228,248,246,228,247,226,229,249,247,229,251,231,230,250,251,230,248,228,232,252,248,232,249,229,233,253,249,233,250,230,234,240,250,234,253,233,235,242,253,235,255,247,249,254,255,249,257,251,250,256,257,250,259,248,252,258,259,252,254,249,253,260,254,253,256,250,240,261,256,240,260,253,242,262,260,242,261,240,239,263,261,239,262,242,241,264,262,241,263,239,243,265,263,243,264,241,244,266,264,244,265,243,245,267,265,245,266,244,246,268,266,246,267,245,247,255,267,247,268,246,248,259,268,248,270,264,266,269,270,266,272,265,267,271,272,267,269,266,268,273,269,268,271,267,255,274,271,255,273,268,259,275,273,259,274,255,254,276,274,254,278,257,256,277,278,256,275,259,258,279,275,258,276,254,260,280,276,260,277,256,261,281,277,261,280,260,262,282,280,262,281,261,263,283,281,263,282,262,264,270,282,264,283,263,265,272,283,265,285,280,282,284,285,282,287,281,283,286,287,283,284,282,270,288,284,270,286,283,272,289,286,272,288,270,269,290,288,269,289,272,271,291,289,271,290,269,273,292,290,273,291,271,274,293,291,274,292,273,275,294,292,275,293,274,276,295,293,276,297,278,277,296,297,277,294,275,279,298,294,279,295,276,280,285,295,280,296,277,281,287,296,281,300,292,294,299,300,294,302,293,295,301,302,295,304,297,296,303,304,296,299,294,298,305,299,298,301,295,285,306,301,285,303,296,287,307,303,287,306,285,284,308,306,284,307,287,286,309,307,286,308,284,288,310,308,288,309,286,289,311,309,289,310,288,290,312,310,290,311,289,291,313,311,291,312,290,292,314,312,292,313,291,293,302,313,293,316,310,312,315,316,312,318,311,313,317,318,313,315,312,314,319,315,314,317,313,302,320,317,302,319,314,322,321,319,322,320,302,301,323,320,301,325,304,303,324,325,303,321,322,327,326,321,327,323,301,306,328,323,306,324,303,307,329,324,307,328,306,308,330,328,308,329,307,309,331,329,309,330,308,310,316,330,310,331,309,311,318,331,311,333,328,330,332,333,330,335,329,331,334,335,331,332,330,316,336,332,316,334,331,318,337,334,318,336,316,315,338,336,315,337,318,317,339,337,317,338,315,319,340,338,319,339,317,320,341,339,320,340,319,321,342,340,321,341,320,323,343,341,323,345,325,324,344,345,324,342,321,326,346,342,326,343,323,328,333,343,328,344,324,329,335,344,329,348,340,342,347,348,342,350,341,343,349,350,343,352,345,344,351,352,344,347,342,346,353,347,346,349,343,333,354,349,333,351,344,335,355,351,335,354,333,332,356,354,332,355,335,334,357,355,334,356,332,336,358,356,336,357,334,337,359,357,337,358,336,338,360,358,338,359,337,339,361,359,339,360,338,340,348,360,340,361,339,341,350,361,341,363,358,360,362,363,360,365,359,361,364,365,361,362,360,348,366,362,348,364,361,350,367,364,350,366,348,347,368,366,347,367,350,349,369,367,349,371,352,351,370,371,351,368,347,353,372,368,353,369,349,354,373,369,354,370,351,355,374,370,355,373,354,356,375,373,356,374,355,357,376,374,357,375,356,358,363,375,358,376,357,359,365,376,359,378,370,374,377,378,374,380,373,375,379,380,375,377,374,376,381,377,376,379,375,363,382,379,363,381,376,365,383,381,365,382,363,362,384,382,362,383,365,364,385,383,364,384,362,366,386,384,366,385,364,367,387,385,367,386,366,368,388,386,368,387,367,369,389,387,369,390,371,370,378,390,370,388,368,372,391,388,372,389,369,373,380,389,373,393,385,387,392,393,387,395,386,388,394,395,388,392,387,389,396,392,389,398,390,378,397,398,378,394,388,391,399,394,391,396,389,380,400,396,380,397,378,377,401,397,377,400,380,379,402,400,379,401,377,381,403,401,381,402,379,382,404,402,382,403,381,383,405,403,383,404,382,384,406,404,384,405,383,385,393,405,385,406,384,386,395,406,386,408,402,404,407,408,404,410,403,405,409,410,405,407,404,406,411,407,406,409,405,393,412,409,393,411,406,395,413,411,395,412,393,392,414,412,392,413,395,394,415,413,394,414,392,396,416,414,396,418,398,397,417,418,397,415,394,399,419,415,399,416,396,400,420,416,400,417,397,401,421,417,401,420,400,402,408,420,402,421,401,403,410,421,403,423,415,419,422,423,419,425,416,420,424,425,420,427,417,421,426,427,421,424,420,408,428,424,408,426,421,410,429,426,410,428,408,407,430,428,407,429,410,409,431,429,409,430,407,411,432,430,411,431,409,412,433,431,412,432,411,413,434,432,413,433,412,414,435,433,414,434,413,415,423,434,415,435,414,416,425,435,416,436,418,417,427,436,417,438,432,434,437,438,434,440,433,435,439,440,435,437,434,423,441,437,423,439,435,425,442,439,425,444,436,427,443,444,427,441,423,422,445,441,422,442,425,424,446,442,424,443,427,426,447,443,426,446,424,428,448,446,428,447,426,429,449,447,429,448,428,430,450,448,430,449,429,431,451,449,431,450,430,432,438,450,432,451,431,433,440,451,433,453,448,450,452,453,450,455,449,451,454,455,451,452,450,438,456,452,438,454,451,440,457,454,440,456,438,437,458,456,437,457,440,439,459,457,439,458,437,441,460,458,441,459,439,442,461,459,442,463,444,443,462,463,443,460,441,445,464,460,445,461,442,446,465,461,446,462,443,447,466,462,447,465,446,448,453,465,448,466,447,449,455,466,449,468,460,464,467,468,464,470,461,465,469,470,465,472,462,466,471,472,466,469,465,453,473,469,453,471,466,455,474,471,455,473,453,452,475,473,452,474,455,454,476,474,454,475,452,456,477,475,456,476,454,457,478,476,457,477,456,458,479,477,458,478,457,459,480,478,459,479,458,460,468,479,460,480,459,461,470,480,461,481,463,462,472,481,462,483,477,479,482,483,479,485,478,480,484,485,480,482,479,468,486,482,468,484,480,470,487,484,470,489,481,472,488,489,472,486,468,467,490,486,467,487,470,469,491,487,469,488,472,471,492,488,471,491,469,473,493,491,473,492,471,474,494,492,474,493,473,475,495,493,475,494,474,476,496,494,476,495,475,477,483,495,477,496,476,478,485,496,478,19,497,20,39,498,499,38,39,499,37,497,19,64,497,37,65,38,499,88,65,499,87,497,64,94,497,87,95,88,499,114,497,94,115,95,499,133,115,499,132,497,114,141,497,132,142,133,499,160,497,141,161,142,499,164,161,499,178,497,160,186,497,178,187,164,499,205,497,186,206,187,499,209,206,499,223,497,205,231,497,223,232,209,499,251,497,231,252,232,499,258,252,499,257,497,251,278,497,257,279,258,499,298,279,499,297,497,278,305,298,499,304,497,297,325,497,304,500,305,499,501,500,499,345,497,325,502,501,499,352,497,345,371,497,352,503,502,499,504,503,499,390,497,371,398,497,390,505,504,499,418,497,398,506,505,499,507,506,499,436,497,418,444,497,436,508,507,499,463,497,444,509,508,499,510,509,499,481,497,463,489,497,481,511,510,499,29,493,495,6,29,495,2,494,496,3,2,496,6,495,483,7,6,483,3,496,485,9,3,485,7,483,482,11,7,482,9,485,484,13,9,484,489,20,497,11,482,486,15,11,486,13,484,487,17,13,487,488,21,20,489,488,20,15,486,490,23,15,490,17,487,491,25,17,491,21,488,492,27,21,492,498,511,499,25,491,493,29,25,493,27,492,494,2,27,494};
   int i, n;
   n = sizeof(triangles) / sizeof(triangles[0]);
   for (i = 0; i< n; i = i + 3) { state->AddTriangle(csTriangle(triangles[i], triangles[i+1], triangles[i+2][↩]

Since it was decided to go back to the old 0.1.2b client code ¹ I might as well mirror all of it here and therefore know at all times where to find it. Sure, it was supposedly on Minigame’s website but that meanwhile went down in one of the shitstorms to date and sure, there was supposedly also a bot on Mocky’s site but meanwhile guess what, that one is also not accessible anymore. Good that I have all sorts of backups and local mirrors, as it just “happens”, right? ²

You’ll need cal3d and CrystalSpace (CS) first, as those are the main dependencies of Eulora’s client. Depending on your operating system, you might need to install various other packages (development utils mainly, as well as some font+sound libs and the nvidia toolkit) to be able to compile the whole orchestra. Have a look at the wiki page ³ for step by step instructions on how to install and compile everything.

The archives and my signatures ⁴ for cal3d, cs, Eulora’s 0.1.2b client and some of the latest versions of Mocky’s extensions of Foxybot:

• cal3d.tar.gz (~20MB)
• cal3d.tar.gz.diana_coman.sig
• CrystalSpace.tar.gz (~535MB)
• CrystalSpace.tar.gz.diana_coman.sig
• eulora-v0.1.2b.tar.gz (~181MB)
• eulora-v0.1.2b.tar.gz.diana_coman.sig
• eulora-v0.1.2-foxybot-mocky34.tar.gz (latest bot version that I used, with some potential quirks according to my current notes).
• eulora-v0.1.2-foxybot-mocky34.tar.gz.mocky.sig
• eulora-v0.1.2-foxybot-mocky34.tar.gz.diana_coman.sig
• eulora-v0.1.2-foxybot-mocky31.tar.gz (this version worked best for exploring according to my current notes).
• eulora-v0.1.2-foxybot-mocky31.tar.gz.mocky.sig
• eulora-v0.1.2-foxybot-mocky31.tar.gz.diana_coman.sig
• eulora-v0.1.2-foxybot-mocky30.tar.gz (crafting version according to my notes).
• eulora-v0.1.2-foxybot-mocky30.tar.gz.mocky.sig
• eulora-v0.1.2-foxybot-mocky30.tar.gz.diana_coman.sig

1. Yes, it sucks but no, you don’t get to say anything about it at all. Because *unlike you*, I’m really in the best position to *know exactly* just how much it sucks since I’ve been stuck making sense of this code for quite a while now, on top of a huge other pile of stuff that nobody else touches with a ten foot pole. Come first with a clean new client that also works and then -only then- you get to say how much this one sucks.[↩]
2. It’s not that hard really, all it takes is a simple wget in the local dir of your choice, for instance: wget –mirror –convert-links –html-extension –wait=2 -o log http://www.eulorum.org/Eulora or, if robots.txt gets in the way of it for your target site, even wget -e robots=off –mirror –convert-links –html-extension –wait=2 -o log http://www.eulorum.org/Eulora[↩]
3. Yes, I mirrored locally this one too. Have you? [↩]
4. ALWAYS check the signatures! If you need my public key, you’ll find it on my Reference Code Shelf. And if you need anything else or are stuck on something, the best option is to simply leave a polite comment and ask for help, you know?[↩]

Over this past week I’ve been going as planned through CS code and docs and everything available to figure out in more concrete detail just how exactly are those fabled “shaders” implemented and used by the graphics engine – this 3rd big tangle of clusterfuck code that fell in my lap for disentangling too, it would seem. And boy did I find everything and anything in there, to the extent that here I find myself, stopping one week in rather than two, since I already need to set it all out ¹ in clear so a decision can then be made as to the way forwards.

The initial starting point was the Terrain2 plugin that handles terrain generation using a heightmap and palette of materials. This plugin expects a shader to exist with a specific name even since it relies on it for what it calls “splatting” – basically mixing seamlessly the different materials of the terrain where they change from one to another close enough so that you care about it (at a distance it just uses whatever is defined as “base material”). The trouble with this “expectation” is that it’s one of those blind dependencies that are the mark of CS: the Terrain plugin (being a plugin, ffs!) is supposedly independent of the rest, right? Except, as you can see, it requires this darned shader and that in turn requires all sorts of snippets and other shaders and those require their own loaders and xml and all that jazz that turns out in practice to be thousands and thousands of lines of code. So sobered by this display of true independence and revolutionary zeal ², I went deeper into CS and looked in turns at how a shader is created in the first place (spoiler: ONLY through a spiderweb of “loading” from XML) and then how it’s used: in bits and pieces, depending on which plugin is chosen and what hardware+software is used. Read on for the gruesome details!

The look at how a shader is created was triggered by this idea that it’s perhaps simply a matter of extracting from the loaders/shader creators whatever they do and package that so it can be done whenever needed. This idea died quickly upon a look at the “loaders” because “what they do” is a lot of obfuscation really: since there are conditions and loops and inclusion of “snippets” from other files and fallbacks and all sorts possible and indeed present in the “xml definition” of a shader, the loading part is an equally complex affair of 7.5k+ LOC where various bits and pieces (e.g. techniques) are collected and then evaluated at a later stage since it can’t even be known upfront which one is useful for what or where or whether at all. I can’t see how it’s any use to aim to disentangle this mess directly really.

For added lulz of the raise-hopes-than-bash-it-on-the-rocks type, I even managed to find in one of the “shader wrappers” a method that was supposedly meant for “loading a shader from source code”, except it contains nothing other than a comment: “Loading from source code is not supported”. And I can easily tell you exactly why it’s “not supported” – because it can’t fuckingly be sanely supported given all that mess, no. No wonder it’s “not supported”.

Next I turned to investigating the way shaders are stored and used throughout the engine so that I can perhaps figure out if it’s possible to either rip them out entirely and replace with a saner approach or at least create my own “shaders” directly on the fly in the code and register those with the engine, effectively bypassing the whole xml-loaders insanity.

In summary, shaders as currently implemented in CS are considered “compiled” parts, as they are indeed obtained solely through a sort of ad-hoc XML-language interpretation. They (and especially the shader variables with all the added complexity of contexts and stack and whatnot) are used throughout the engine: from them being part and parcel of a material definition to the rendering loop explicitly calling the shaders registered for each mesh. As such, ripping the current shader-setup entirely out doesn’t seem very doable without a full overhaul of the engine itself.

There is NO current way to modify an existing shader on the fly and the “shader compiler” class itself (the one producing shaders) pulls in the whole SCF mess with it so that it’s not even all that easy or straightforward to try to make /adapt one directly in code. The *only* currently working way to create a shader and load it into the engine is by using a loader that will trigger god-knows-how-many-levels-deep of xml tokenizing, parsing, evaluating and so on. There are nominally two types of shader loaders, namely for “xmlshader” and “weavershader” but in practice the xmlshader is the basis, as the weavershader simply adds a few more bells and whistles on top, otherwise relying on the xmlshader for the heavy work. The shaders themselves are not really concerned solely with “defining a class of surface” but pack inside additional concerns such as providing alternatives for different types of hardware or software (this is done via providing several ways of doing the same thing), supporting various LOD (levels of detail), reusing other supposedly-existing shaders and snippets, supporting (or not) different rendering plugins (currently OpenGL and software rendering).

Given the above rather troublesome finds, I think it’s worth to go in a bit more detail on the theory of shaders in CS, since it might help to figure out what if any of it is really useful as such (and for what exactly). From a theoretical point of view, shaders in CS sound reasonable enough: each shader is meant to provide simply all the rendering steps and tumbles (or other workarounds) needed to obtain on screen a class of surfaces: e.g. you can write a shader for water-like surfaces, another one for cloth-like surfaces and yet another one for terrain surfaces. There is even a reasonable enough pipeline for this since each shader is meant to contain a clear list of parts ³:

1. Metadata
2. Shader Variables
3. Technique(s)
4. Pass(es) per technique
5. Mapping(s) per pass
6. Vertex processor per pass (if any)
7. Vertex program per pass (if any)
8. Fragment program per pass (if any)

The code investigation revealed that metadata is more of an overplanning in there – it consists currently of “maximum number of lights this shader can handle”. And I can’t even quite say why is there a maximum number of lights or why exactly does a shader add lights to start with – since when is a surface to be defined by …added lights? But nevertheless, it is: apparently quite a few “effects” are achieved by adding whatever number of lights in various passes, for all the logic that has.

Shader variables are one of the big things in there really: they are meant to work as parameters for each shader so that conceivably you can reuse a shader to create similar but not identical surfaces. This is also part of where the clusterfuck starts: while each shader can in principle define its own custom variables, in practice there is a set of “predefined” variables (such as “diffuse tex” or “specular tex”) that are assigned different values in different “contexts”. So at rendering time, there is a whole *stack* of shader variables collected from various places and good luck knowing exactly which value will any given variable have where and when. A lick of sanity there is in the form of a hardcoded priority of different contexts so that – according to the docs – the value of any given shader variable will be taken from the highest priority context where it’s found, with contexts provided in order from lowest to highest priority by: shader manager (basically the “global” scope for shader variables), current light, render step, render mesh, mesh wrapper, shader, material. In practice, the result of all this is that everything ends up relying on something else and as a result entirely not separated, despite the whole elaborated pretense to the contrary. Unsurprising by now, I know.

The shader manager for all its pompous name turns out at a closer look to provide only a time variable that is supposedly useful for shaders that aim to animate the surface – why would an animated-surface be needed? I can’t quite tell. The render step and mesh are …”special”. Just in case the rest seemed too sane for your taste, here we have some specialness added since those are not really parameters for shaders – more like some information sources that got stuffed into those “shader variables” since why not. Apparently one can get in here geometry from some meshes (vertices), splatting masks from terrains or even transforms and textures or anything under the sun really. The last three “contexts” (mesh wrapper, shader and material) are the ones most often in use – theoretically the mesh wrapper would allow one to parametrize the surface on a per mesh basis (using the mesh wrapper), per material basis (material context) or otherwise to provide default values in the shader itself. So much support for endless customization that there’s not much room left for *easily* making a darned thing to customize in the first place.

The techniques of a shader are meant to be the actual meat of the shader aka the work done to make that surface look as desired. Customization hits here in the form of several techniques meant as *alternatives*: different ways to do the same thing so that no hardware&software configuration is left behind. To get the full extent of what those good intentions amount to in here, multiply then the number of techniques with the several *passes* (aka multiple renderings for the same darned surface) per technique and further add to it the vertex+fragment processors/programs that come in different flavours depending on the underlying plugin to use (OpenGL or software rendering atm).

The working of a shader in practice as found out from digging through CS sources is meant to be a loop of “passes” where each pass consists in 5 steps:

1. Activation
2. Setup
3. Drawing the mesh
4. Teardown
5. Deactivation

If you frown at the 5 steps of which only one does the actual drawing, it’s ok, there’s even more similar sets of substeps in almost each of those 5 steps. In any case, the activation+setup mainly collects and sets in place all the variables and values that are available from the various contexts at that time. This can easily be a whole mess because “variables” can include any number of anything really, from textures and lights to geometry to additional rendering buffers. The drawing of the mesh is handled from what I could tell mainly by the OpenGL plugin (or the software renderer if you run it without OpenGL) and as such the code is dependent on that. The teardown+deactivation do the whole dance in reverse, as expected. Efficiency at its best, wouldn’t you say?

Looking at the classes used for the various components of a shader (shader variables, passes, techniques, programs), the shader variables are the most reasonable in that they don’t bring in at least a whole lot besides what is anyway in there due to materials for instance. The techniques themselves could perhaps be ripped out and repackaged as their internals seem to be mainly direct manipulations of the g3d class (graphics 3d). The main issue though is with respect to the iShader interface itself because it mandates a “QueryObject” that pulls in all of a sudden the whole SCF (basically it forces iShader to be a plugin, just like that), holly shit! And otherwise the shader passes, programs and how to even decide *what* should exactly be done by a shader do if anything at all: which parts are best handled by the material/texture itself and which parts should be done by a “shader” and how does one even make this choice?

As far as I can tell currently, there are a few possible next steps perhaps:

1. Aim to modify one of the existing “shader compilers” to allow creation from code. This would seem at first sight as requiring the fewest changes to CS itself (though it does inevitably mean that I’m patching CS itself, with all that brings with it, yes) but it comes with some big unknowns, not least of which is: how exactly should a shader be specified in a sane way already (since importing the whole conditions and whatnots is not sane)?
2. Aim to create directly own version of “iShader” – this means I’ll end up writing now a new “CS plugin”, supposedly less invasive than option 1 above but carrying the SCF burden at the very least. Ripping SCF out of iShader as first step is likely to turn into all sorts of additional trouble and I can’t say that I see the justification for going there even if aiming for this direction. The advantage I can see on this option compared to 1 above – once the SCF is somehow handled, ugh – would be that it’s both more direct (ie should be able then to create and register the shader as/when desired) and perhaps more amenable to a step by step discovery of just what should go in a shader to start with.
3. Investigate what exactly and how much can be done via texture+materials alone. This is quite iffy because it would probably require quite the dive into Blender now of all things (and the exporter that does not yet exist/work) but I mention it here for completeness.

A big handicap currently here is the lack of clear knowledge as to which parts currently in “shaders” are really needed and to what extent. Basically a lack of practical experience with graphics really, to know how much can be achieved through textures+materials only and how much/which parts require the shader on top to end up as anything reasonable. The current set of shaders is such a spaghetti that I don’t even think it’s worth attempting some direct translation – although I did look through it to start getting some sort of familiarity with how things may be done and to what extent. It all seems so far to be more of a trial-and-error anyway and the code provides further encouragement in this direction e.g. “// If you understand this initially (or at all) I salute you” , “// No idea why we have to invert the Z at all, but reflection is wrong without it” ⁴

1. And for my own future reference, so I’m including even “known” parts in here, let them be as I never ever regretted having them written in clear in a single place.[↩]
2. Can’t resist humming it too since it’s still better laughing than going nuts: aqui se queda la clara la entranable transparencia de tu querida presencia…[↩]
3. Section 4.12.1 in the CS manual gives a reasonable although purely theoretical description for those.[↩]
4. Both quotes are from cs/plugins/video/render3d/shader/shaderplugins/glshader_fixed/glshader_fvp.cpp aka the OpenGL plugin that supposedly does the vertex programs parts that may be specified in a shader.[↩]

Over the past 2 weeks, I’ve been swearing my way through the client’s entrails, as part of the previously planned cutting of some of the abundant tangles that get in the way of everything. It’s been even more gruesome and disgusting than expected – see further for more details on this – but it needed to be done, so it’s done. I have now a client that gets directly to the game instead of going through several screens just to show you anything ¹, does not expect to know the whole world upfront, does not imagine it needs the network and a ping-pong server to just show… a local widget and moreover, can even paint something on the bloody screen and react to controls just like that, without a babel tower of widgets and managers and all sorts. And as a tiny satisfaction, it does all this with 3000 LOC less than it took to not do it!

The relatively simple part of this detangling session was the cutting out of network dependency – it took a full day of touching almost every file but at least I had a quite clear idea as to what I could do it: separate the local command handling from the remote “messages” handling (those were grouped together in a single interface, psClientNet without ANY good reason – wtf does local command handling have to do with remote server communication?); remove the inheritance/implementation of network/remote messages handling that was littered everywhere (and I do mean *everywhere*); keep in the active loop so far only what is actually useful.

The less simple task was then to get the client to show something at all because of the tangles between the view, the “psCamera” class and – again, as usual – ~everything else. That was done too and some image made it to the screen but then I realised that … the controls – aka moving anything, even the camera around – were not working anymore! Why? Well, because it turned out that the client’s controls relied on various PAWS widgets being loaded and those widgets relied on one another and on all sorts of graphics being present and known and whatnot. How the fuck can you program like that with the GUI actually *driving* the very definition and use of controls? They are called controls and *not* banana-dispensers already!

Once I found a way to reinstate controls and character manipulation as well ², the next step was to make *sure* that no shaders are loaded by default somehow & somewhere because indeed, the ever helpful ps&cs would do just that (and otherwise whine loudly if the expected shaders were not found). As a result, I found out another very interesting thing: while CS will load the “default texture” aka “error texture” when the specified texture is not found, this seems to *also* have some unexpected side effects in that all of a sudden, the colours of the terrain itself may simply change from one run to …the next. Here’s the world as it looks on one of those runs with Foxy clad in “error texture”:

There is the fact that the terrain plugin expects indeed to use shaders as if they couldn’t possibly ever NOT be there but I still don’t really have the words for this situation where I run the darned thing once and the hills are green, after which I run it again and they are sandy or maybe gray. I’m sure I’ll figure out the why too, just as soon as I get to the bottom of that wonderful terrain plugin but hey, all the fun in its own time, right? For now though, here’s Foxy in her own skin but without any shaders whatsoever, under a starry night sky:

The next step now is to figure out just how to do all that shaders are meant to do, directly from CS. An initial mapping of shaders in CS suggests that this is going to be a huge pain really and I’ll have to find a way to cut the knots before going nuts: as the situation stands, CS considers shaders to be “compiled” bits produced by registered “shader compiler” plugins of which there are 2 varieties – a shader “weaver” that depends on the “xml shader”; both of those rely on xml anyway and the whole xml-for-shaders is effectively a sort of “let’s now program in xml” that requires as a result ~7.5k LOC just to… parse xml.

Rather than waste the time disentangling all the above shader mess (it’s not a straightforward thing either, as you might tell by now), my current plan is to try to go at it perhaps from the very practical angle of what and how are “compiled” shaders actually used/useful and how can I produce them as needed. As this is not really all that clear to start with, I can’t say I have a sane way to estimate just how long it will take, but I aim to have at least something more concrete in 2 weeks maximum ³ since one way or another this has to move forwards.

1. No, seriously, do you *really* want to play? Maybe you just started the client to…quit? Or to see the …credits? Admire the splashscreen? Dunno, something *other* than playing?[↩]
2. This involves in usual ps fashion several managers and spurious classes, all tangled up together and calling one another until nobody knows anymore what is where and for what reason, if there ever was a reason; nevertheless, this part – if it is to be cleaned – will have to wait for another day as at the moment there are more pressing matters really.[↩]
3. There’s Christmas and New Year in those 2 weeks too but who cares about those all that much.[↩]

The goal of this most recent dive of mine into the CS-pit of how to do little in 1001 ways and a lot of lines of cpp&xml ¹ was stated in #eulora:

diana_coman:hmm, not sure things are clear there somehow; let me see from the other end: do you suggest I dig in CS and figure out how to directly get from it what I want it to do, ditching all those shaders xmls and whatnots?

As a result of the above, I went first through another re-reading of the CS manual with pen and paper at hand – it’s basically the map but certainly not quite the territory. At any rate, thus fortified hardened of sorts, I turned to the self-doubting CS code ² itself to figure out some – any! – way of cutting this knot and getting to the core of the engine – aka the *useful* part, ffs! – so that I can finally DO the graphics properly and on the fly, including icons, textures, materials, shaders, effects, terrain, lights, meshes, animations, everything! For the impatient reader, here’s the update coming out of this only yesterday:

diana_coman: mircea_popescu: looking at the “cut straight to doing it in CS” here seems to even come with 2 layers really: first there’s the added cruft on CS-core itself; then there is the layers-upon-layers of cruft added by PS/client.
diana_coman: and there is always that compromise between “supposedly reusing existing code saves time” and the bitter experience of “yes, but reusing THIS code costs time”.

A large amount of cruft in CS comes from the very unfortunate idea of chasing a theoretical notion of flexibility & reusability while ignoring the practical result of the wrong sort of complexity being baked in. To quote from the CS manual itself:

diana_coman: lemme cite from CS manual for the ages: “Crystal Space is a highly modular framework. As such it is more complex then many other game engines or renderers you might encounter. The advantage of this complexity if that it offers you a lot of power in what you do and how you use it.”
diana_coman: or such was the dream there.
diana_coman: to add from only 2 paras down: “the components and libraries are *more or less* independent of each other” (emphasis is mine).

Indeed, CS has as a result of the unfortunate design decision above, a lot of components (“plugins”) and libraries that are quite dependent on one another and in such a tangled way that one is even advised to NOT attempt to load a plugin directly since that will just fail if dependencies are not already loaded! And here it starts, because that requires then a dedicated loader that supposedly knows the dependencies but also works only with values from a config file so nope, not exactly on the fly anyway. Do you see yet how much and what sort of power you gained in return for all this complexity?

If the above is not quite enough, add to it the fact that the pretense at “independent of each other” comes at the very steep price of the horror called SCF (Shared Class Facility) – put simply, this is a way for code to use methods based on known interfaces, supposedly unaware of which/what plugin might happen to implement those methods in any given run. In practice though this simply means that the plugins get bogged down with a wad of code just to “register” their implementations, they are still nevertheless not exactly independent because how can such a thing be in a graphics engine where everything is about the same thing really and finally, the code using plugins has to wrap even the simplest of stuff in nightmarish calls to the SCF so that if you want to change one tiny bit, you still have to write at least 10 lines and remember several levels of layered abstractions that have nothing to do really with the task at hand.

Oh, and lest we forget, all this festival of abstracting things away from your grasp in the name of your own “power of use” further means that trying to find the code that actually gets executed (ie trying to figure out just WHERE in the darned thing shaders for instance are really implemented as such) is an exercise in frustration: for one thing it’s quite often scattered about in several components (there are loaders – hence parsers and tokenizers and whatnots – of xml too in this, for added fun!), for another thing it ends up 25-40 layers deep at most time and for the cherry on the top, since at each point you get only the “abstractions”, tough luck to actually piece it all back together again before you go insane. How, just HOW exactly can anyone hallucinate this to be “power of use”? ³

Still, since I’m not going to rip away the whole SCF madness out of CS ⁴ – or not just yet, I’m an optimist at heart – it follows that I’m stuck as usual with the task of finding some practical way forward anyway. A closer study of those shaders reveals that they are – like most in graphics… – just one way of doing some things. Specifically, CS shaders are meant to be (ignoring for a bit the added xml nonsense) parametrised pieces of code that describe a type of surface. The description of a surface – hence a “shader” – can get quite complex since one can literally change each vertex of the surface and do so repeatedly, for instance by applying/combining/mixing different textures and techniques over several “passes”. Essentially “shader” is just a name for “programmatically but still laboriously ⁵ painting pixels of a surface just-so, going around some of the limitations the simpler texture+material surface definition might have”. So on one hand shaders are yet again just an alternative to a simpler option but on the other hand they are supposedly the more powerful alternative – the question being of course: is this the same sort of power as discussed just above, namely the power to push the boulder uphill rather than the power of building anything?

Leaving aside for a bit the shader-power-doubts, the dig in CS revealed that the terrain system anyway quite expects to make use of shaders going also through all the dances of shader variables and contexts ⁶. So at this point it seems that there aren’t all that many options really and at least *some* sort of shaders will have to be supported and therefore sanely extracted/plugged directly into CS and then – possibly – the terrain system either beaten into shape too or at the very least made to work with those. Such joy.

Before ripping into CS code directly though, there’s the added bonus of the client code itself. The original PS code added its own wrong sort of complexity on top of that pushed through by using CS. In a similar ill-fated attempt at “power if only in theory”, there’s a whole system of widgets that are meant to be independent but cannot be so and moreover there are a lot of “managers” that supposedly are independent too but rely on all sorts and in general just about ~everything has two significant dependencies: the “netmanager” aka direct communication with the server ⁷ and the fact that everything has been already loaded because the whole world is in fact fully known before you even get to push the Start button.

Given that both the above dependencies of PS code are torn to shreds for Eulora’s client, it quite follows that the minimal cut here is along the lines of ditching those parts entirely – while this WILL mean changing a lot of code, I still think it makes for a smaller cut than attempting to work around it. Basically it’s again that frontal assault vs outflanking option: the whole mess is so wide and spread that it’s not worth going around, even if there will be quite a bloody job to cut through it.

Considering all the above, there is some work to be done further on both CS and client code. However, since the client is anyway the final goal and the most effective “tester” of anything of interest, I’ll first take some steps on cutting out the useless but very burdensome tangles in there: going from the starting point (and keeping to a minimum of changes still but don’t count on this to mean “code will not be touched), the client should simply focus on what is does, namely user interface and nothing more. This means that it will display to the user whatever information is *available and relevant*, it will accept input from the user (and make it available for further processing if it involves anything other than UI/display changes) and it will further expose ways for *everything* to be reloaded/changed on the fly, since this is the hard requirement anyway.

Once the client gets to the stage where I can use and test more directly (aka loading on the fly instead of going through 1001 useless other steps just to get to the one I want) a shader with it, I’ll switch again to CS and cut through so that a shader can also be specified and loaded on the fly without any xml involved. As this all adds up to a huge ball of tangled unknowns, there’s very little sanity to any overall estimate at this stage but there will be for sure updates on the progress, as usual. At any rate, the cuts will be strictly on an as-needed basis and done step by step so that there will be gradual improvement while maintaining at all times a running client overall.

It’s worth noting also that the above approach of cutting through the tangle in the client too is the only one that I can currently see to move forwards. Keeping otherwise around all the dead weight just about got to the point where it would stall almost everything trying to push further so it needs to go and good riddance.

1. To be fair, PS is the champ here and by far! While CS may be doing little with many lines of cpp&xml, PS does way less and with 10 times more lines of the same cpp&xml.[↩]
2. Literally, as I illustrated in #eulora: diana_coman: in CS questions-from-the-code: “ctxt->iview = other.ctxt->iview; // @@@ Is this right?”[↩]
3. I get it, one can on occasion feel very powerful – Popeye muscle-style! – just for getting this to work at all. As in, it took a LOT of pushing this uphill for it to go smoothly down the other side! Is this the sort of power you are after?[↩]
4. Mainly because that is ~same with re-writing the engine really, could just as well start from there if it gets to it.[↩]
5. Computer graphics is this precise achievement of programmatic-but-still-laborious, it’s a wonder.[↩]
6. The same variables can be and indeed are overwritten in CS at various levels so that you can go properly insane; for instance, you might have set the texture as opaque in the material but then you failed to set it in whatever shader got pulled in somewhere and so fuck you, the whole thing will be totally shiny instead. Did I mention how much I like computer graphics?[↩]
7. You may wonder why would every widget and every class need to directly communicate with the server anyway, sure; there are plenty of answers to such wonder and none of them any good.[↩]

Coming back from any holidays, no matter how short, means that I spend initially a few days simply with the task of picking up again each and every bit of work that I put on hold while I was away. Note that I didn’t say “dropped for the holidays” but rather specifically put on hold in a prepared manner so that picking it up again is not a horrendous chore but rather an opportunity to re-load the context with a fresher mind. And this makes it also a good moment to review the work done over the past months and state the next steps just as I usually do anyway for my own benefit before setting to do any work. The only difference this time is that I’m publishing here this whole summary of work done + work in line to be done next, so that anyone interested can follow along. Why wouldn’t I publish it anyway?

May was spent on two main pieces of work:

1. A working initial ¹ implementation (both on client and on server side) of the SMG protocol parts that are already clearly defined, namely network communication, account request/creation, serpent keys request & management, file request & transfer. This part meant Ada implementation of the protocol but also wrestling the legacy Eulora client into using the network communication structures of SMG Comms as encapsulated in the standalone Ada lib called EuCore. It also included some server-side work so that the end result is proper communication: the client can now ask for a new Eulora account and the server will reply according to SMG Comms; the client can also use its already established Serpent keys to communicate with the server and the server will reply accordingly. As required part of this work, a broad mapping of the client and its dependencies had to be done as well and basic docs for the client were published.
2. The initial design of client logic for communicating with the server i.e. moving up on to how to use the messages that the protocol (and the implementation at point 1 above) makes available. This involved more reading, thinking, discussing and structuring than coding. The concrete result is a clear client strategy for knowledge acquisition and a very basic initial skeleton of its implementation. However, as the *structure* of the knowledge to be acquired was not yet defined or even fully clear, the next steps (see June below) had to focus on clarifying that, before the implementation can be pushed any further.

June started with fleshing out the hierarchy of data in Eulora as an initial draft capturing the relevant knowledge so far. This helped clearly identify the big areas that required further detailed investigation, most notably the graphics part where neither the required building blocks nor the specific client approach were yet known in enough detail to allow any meaningful attempt to extract a relevant data hierarchy. So the rest of June was spent on figuring out the graphics part: acquiring a better theoretical understanding through study of the reference book for computer graphics ² and at the same time acquiring also a very practical knowledge of how things are done by first mapping broadly (and then ripping apart) the existing approach as it found itself spread and scattered through the legacy client, several plugins of the same and the several plugins more of CrystalSpace (CS) + the code of external libraries (cal3d especially). By the end of the month, the better understanding of the whole graphics part and the practical experience with it resulted in a working initial example of loading some specific type of graphics object and a clear structure of the notions involved. This structure doubles as the map to follow for the next steps towards making the client able to load everything on demand.

July focused entirely on graphics in Eulora, figuring out one by one and implementing a prototype loader for each of the sections in the map written at the end of June:

1. Loading a static (incapable of movement) object given through its list of vertices, normals at vertices, colours at vertices, texture mappings at vertices and triangles that make its faces. This includes already loading factories and textures.
2. Loading the landscape aka loading the terrain given through its heightmap and material palette. This includes also loading lights, materials and shaders. The whole was pursued only to the extent that the terrain and materials were indeed loaded as specified by the heightmap + material palette but without fully aiming to solve other potential issues such as the “mirror” effect that is more related to lighting and – possibly – some material or shader properties.
3. Loading a character made of several parts that can in principle change on the fly. This part was pursued for now only to the extent that the character is loaded and shown + answers the player’s control but without fully going into the details of equip/dequip, actions and the full extent of changing bits and parts.

August was a bit split, with the holidays in the middle:

1. At the beginning of the month, I continued the work on graphics but moved from the 3D issues to 2D, namely the user interface with all the different icons and images and various bits that are sometimes known as the “skin” of a client. This required a deep dive into yet another plugin of the legacy client, the widgeting PAWS.
2. At the end of the month, I started working on writing down the 2nd draft version of the client data hierarchy integrating the results of all the work on graphics + all the pondering and figuring out done throughout the month + the further discussions and clarifications in the SMG boardroom on the desired working of Eulora’s communication protocol. This is still in the works currently but I plan to publish it for discussion as soon as I’ve got down all that I have so far and certainly no later than the end of next week (ie no later than 15th of September).

The current plan for September and beyond is this:

1. Write and publish for discussion/revision the deeply revised (re-written really) draft of client data hierarchy. Deadline is 15th of September.
2. Pending feedback and discussions, possibly dive again for more details if needed and/or revise further the data hierarchy, if/as required. This is a crucial part for moving forwards but it’s also one that once fixed will be very costly to change.
3. Work on the implementation of the client and server-side pieces of the SMG protocol that are still missing, as soon as they are set/clarified. At the moment, they are listed broadly and in this order: client-side data storage (aka cache, of which currently there is only a skeleton, pending better understanding of what the data is going to be exactly), object request/reply, client actions, full client logic for knowledge acquisition and storage, full client loading/use of local data cache, client and server management of own keys.

1. to be refined/reviewed as other parts such as key storage and bootstrapping become clear[↩]
2. Foley, J.D., van Dam, A., Feiner, S.K. and Hughes, J.F., “Computer Graphics: Principles and Practice”, 2nd ed., Addison-Wesley[↩]

It took indeed more than a week and I’m rather surprised it didn’t actually take even longer than two weeks to get *anything visible* out of it. It’s all one strand like Stan’s spittoon after all, so there’s rarely just “one sip” that doesn’t bring in the whole rot with it and I have to fight over and over the urge to just burn it all down and start a sane thing from scratch ¹. Nevertheless, a lot of deep breaths and assorted cussings later, here’s a brief summary of the main achievements, in the order they were made:

• some useless crap was removed and the whole thing made to work without (e.g. the doll with its dependence on a special sector in the map file);
• the initial loading and “pre-caching” of meshes, textures, materials, maps and 2D stuff was excised (there are still the shaders remaining though as well as 1001 “preferences” in as many files in yet as many different places, sigh);
• the actual handling of 2D icons and GUI images in general (e.g. buttons, borders, arrows, window decorations) aka the PAWS was dissected and exposed and roughly mapped as ugly as it is;
• as recorded proof of breaking PAWS and because of its shameful lack of proper way to load images on actual demand, the whole thing has had a run with the stupidest “mouse pointer” available in there and otherwise with absolutely no 2D images at all for the very lulzy effect that it even looks better in some ways than with all of the various pieces of “art”, here it is:
  
  

  That snot on the screen is the mouse pointer on the black background starting screen.

  

  PAWS's shame: the better-looking interface with no images whatsoever (clicking the invisible widgets still works!)

The path that led in the end to the above started with me looking – as initially planned – at how the character (as moving sprite) is handled or rather mishandled, consider: there is the CEL layer that supposedly provides on top of CS the sort of structures needed by games to store and manage the various entities and objects; PS however “uses” it in principle but in practice it still mainly does its own thing anyway (and by now I strongly suspect that this pattern is simply because nobody ever spends the time to understand what they use); specifically, the PS way is to have a pscelclient class that supposedly manages game entities on client side (shouldn’t CEL have been enough for that, you ask? most probably yes but that’d have taken someone to figure out first exactly how it works and who has time for that!!); but then there is *also* a pscharapp class that handles the appearance of the character (apparently that’s too low stuff for “management” to handle) and there is also a psclientchar that moves about equipped items because this is yet neither management nor appearance but absolutely something separate – though you’ll be hard pressed to notice much separation in the code for all this theoretical separation; and to top it all, the inventory and doll views that precisely reflect character’s appearance and equipped items are actually the domain of something else entirely – the PAWS; and the PAWS (which is supposedly so separate as to bother to be a plugin by itself) in fact messes about with pretty much anything and everything from the engine to shaders and textures and 2D images (which are still loaded as… textures, but those are textures-this not textures-that) and xml-defined maps and hardcoded paths to all sorts of files it expects – nay, demands – to be present and just-so and at-start but oh-so-configurable. At which point it became clear that the PAWS mess is indeed such a large part of the whole mess that I’ll simply have to get to the bottom of it just in order to move further at all.

Looking at PAWS more closely revealed that it’s meant to be a GUI system. This initially rather puzzled me because there IS in fact already another GUI system in there, one that PS uses/relies on as a dependency, namely CEGUI. Then again, the PS style is to “use” something by simply sitting on top of it and mushrooming a lot of managers with hardcoded paths and expectations that they know the whole possible world upfront so there should be no surprise at PAWS, which is precisely this sort of thing – it mushrooms widgets and pointers rather than something-else and pointers, that’s about all the difference from all the other PS masterpieces.

Masterpiece aside, what IS this GUI system PAWS? As far as I can tell, it’s a lot of widgets so well organised that there are no less than 3 methods “FindWidget” because the 1st one (defined in pawswidget) checks the current widget and its children only, the 2nd one (in pawsscript) checks also the parent of the current widget while yet the 3rd one (in pawsmanager because yes, of course there are all sorts of managers) simply delegates it all to “mainwidget” aka a sort of calling the 1st one from the top of the tree (but no, they couldn’t possibly notice that it’s meant to be a tree and there are well-known algorithms for searches in a tree). Anyways, the fun part with widgets is the way in which they are defined really as each widget has an .xml file (in data/gui) that supposedly defines appearance only (though more concretely it’s about which elements it contains and on what position) and a .cpp file (in client/gui usually) that supposedly defines behaviour. The funny bit however is that both of those, for all their pretense of clear separation (they even are in totally different dirs, yes?), quite directly reference both one another’s bits and pieces AND global, all-of-a-sudden-made-hardcoded parts: the appearance references by name other widgets (that it uses for specific roles and in specific places so it’s not just a name really) and “resources” aka pre-loaded images; the behaviour code in turn has to reference various components of course. And in all this, apparently nobody noticed that before you can talk of behaviour and of appearance, there has to be somewhere a *structure and nature* of the thing that actually is the core and therefore the driver of everything else… Then again, what structure and nature and such silly notions, when the whole thing is bent on defining anything and everything by its current, temporary position or at best some role it might fulfill for a while ².

The widgeting PAWS further has all sorts of notions regarding appearances: there are “skins” and “styles” and “resources” and they are all both hardcoded and expected to be fully known upfront. The Great Paws knows not only all the world that is but also all the world that could possibly be (the world as it defines it is all nothing but labels anyway so it makes sense in a way). And it enforces this by whining and even wagging its finger at the user if it ever finds that a file with what it *knows* is the “right” name is missing from its expected place, quite literally giving orders to the user, as the message spit is this:

“ART ERROR: PawsTextureManager loaded the image %s which was missing from the imagelist.xml and was loaded on demand. Add the image there!”

I’ll add to the above: and blow your nose and stand up straight and cut your nails shorter! Imagine that, being such a shitty user that the poor, poor PAWS had to actually load an image on demand (how degrading!!), you nasty user, you! And note that this “on demand” is anyway the sort of “you demand what I’m willing to do or no dice” not any sort of actual demand: it only means it deigned to look (because it had to use that image) in the *same place* it looked initially (when it tried to load everything that there should ever be) just because you are the sort of shit that failed to put the image there when it looked the first time, you know? What sort of user does such a thing to a poor piece of code, you tormenter, you, shame on you!!!

Before you get your hopes up that there is at least some sort of “loading on demand”, bash those hopes one over the head: for one thing this “on demand” still relies on the predefined list of what there *can* ever be so that if you request a “resource” it doesn’t know about than tough luck, it won’t try to look for it, it’s not that stupid to do something useful; for another thing, there is no way to actually call this on demand: it’s just a way of doing rigidly the same thing but expecting a different result because meanwhile the world should have better come to its senses and fit the expectations already, what.

If the above sadness is not enough, add to it this significant bit: PAWS considers itself a plugin and as such entirely separated from the actual client and therefore unable to use whatever the rest of the client knows. Sure, in practice this separation “works” by means of the main client class simply passing pointers to PAWS on creation – why use CPP if not for pointerism, after all. Moreover, the added benefit of this separation is that PAWS also gets therefore to have its own set of repeated XML parsing of all sorts since it can’t use the already-repeated XML parsing that the client anyway suffers from. And finally, it makes somehow perfect sense to “separate” the main client from the GUI system seeing how they both use the same graphics engine anyway and moreover the main client loads some images as textures while PAWS loads…well, some images as textures, yes. But they are not the same “class” of textures, so that’s significant separation, see?

Besides deep red, I tell you what I further see: since I have no intention to pass even more pointers on or otherwise to force this “plugin” nonsense on everything just for the very specialness of PAWS, it’s way more likely that the paws will be clipped more to size and therefore will take their place as yet another of the many classes inside the main client, possibly together with all the rest of very-special-plugins if need be (of course the dependencies are not that simple). I’m still pondering this, mainly because the idea is to touch as little as possible the code rather than as much as it takes to clean it (mainly because clean it won’t be anyway). But one way or another, I will have to find some reasonable way to bring PAWS in line with the rest and have it go and load resources from wherever (and whenever) it is told to. This might even require more discussion in S.MG’s boardroom but at the moment and as a result of all the adventure above, I have at least a much clearer idea as to what the 2D part is all about so there is therefore a point to even start the discussion from.

1. While it might *seem* faster that way, it’s not, nor does it really make sense if one calmly considers everything involved. Urges like this are simply focused on getting rid faster of the current set of problems and do not care nor stop one second to consider the alternative set of problems that are bought by such “solution”. So yeah, urges make very poor guides, how surprising.[↩]
2. And no matter how much I try to *not* see it this way, it always is the case that the closer one gets to the “how things look” and therefore in this case to graphics, the more confusion and more monkeying there is.[↩]

What’s the first change you’d do to an empty-headed character such as Testy? Why, chop some bits off, change his sex and get him naked at least, even transparent at that, since he’s pretty much just as tedious and boring as all children dolls ¹. Of course the first attempts failed to produce anything visible because it turns out that using Cal3D sprites in Eulora’s client goes through several layers of inmisdirection: Crystal Space (CS) wraps the Cal3D code in its own SprCal3D plugins and factories and whatnots; on top of that, there is also the rather elusive Cel (Crystal Entity Layer) that is meant to be a game-specific part of CS keeping track of game entities; and messing within and across all is Planeshift’s (PS) own GEM concoction interlaced with delayed loaders and Dead-Reckonings ² and cloning of materials (why? no idea yet) and surprising, undocumented transformations. Still, after a while of hacking through all the above, I could at least get some parts of Testy showing… if lying down rather than upright:

Spot the bottom in the grass!

The prostrate position makes at least sense in that “movement” also looks a bit like some swimming look-alike: parting the legs and advancing sort of thing. Then again, it turns out that using a different model (e.g. Cally) makes it even more interesting – the model is upright when waiting but it dives straight back to the horizontal as part of moving and it furthermore cares not one bit for whatever movement animation one loads. So there’s a clue for another day – current movement needs a good bashing over the head and a deep dive to figure out what’s all this nonsense. But until then, here’s naked Cally who borrowed Testy’s skin for a spell:

What bits is she missing?

Anyway, since position of the model is always a matter of applying one transformation or another, I went ahead and applied the obvious transformation, 90 degrees and upsy daisy on her feet:

Upsy-Daisy with muddy skin!

And funnily enough, once the 90 degrees rotation is applied to the factory (i.e. presumably to ALL entities built out of that factory) the Testy model remains upright even during movement without any perceived change. On the other hand, the Cally model is upright without the 90 degrees rotation but nevertheless dives down for movement and pops back up as soon as movement is done. So it would seem that movement has its own assumptions that it doesn’t bother to make public, how lovely. Seeing how it’s most probably to do with the model’s initial position or similar, I might even need to end up again with the full environment, Blender and all installed to get to the bottom of it and probably see exactly what or how the exporter is also broken and specific. Such joys to come but for the moment they are not yet here since the more pressing need is first of all to disentangle some more the PS+CEL+CS+CAL3D mess regarding animated entities so as to actually be able to *use* the walk/fight/move animations rather than just “add” them and moreover to be able to add and remove on the fly as many and as diverse entities as wanted, beyond the “main character” guy. And all this is getting deeper into PS swamps so it’s likely to take more than a week indeed – ideally by then I’ll be able to cut off at least some parts of the existing code (precaching and preloading especially) and lighten the client start a bit if nothing more.

1. Changing graphics is this exercise in tediousness and pointlessness, it eerily reminds me of playing with dolls, just one step lower than that since those are just images of dolls, fancy that. And playing with dolls was utterly boring even when I was 5! It’s like an endurance test along the lines of watching paint dry. Tinker with this and tinker with that, all for “the looks” of it and can’t even satisfyingly throw it at anything to hear it break.[↩]
2. Not kidding, although it’s way less exciting than the name they had the chance to land on: it’s mainly looking to see if your character falls down and breaks their neck.[↩]

Getting some actual landscape as in getting the height right turns out to be only half (the reasonably well-working half) of the terrain task: once I had in place at least the generic mechanism for creating factories and meshes, the terrain factory required only a few iterations, false starts and mandatory dives into CS’s entrails to get working and save Testy from his usual fall into the sky-sphere:

After setting up the lighting a bit better too, it looks even more reasonable (next to the crater that is currently empty, yes):

The truly frustrating bit turns out to be the painting of the landscape in the desired materials. For the relatively easy part, the Terrain2 construct of CrystalSpace provides a “terraformer” that can be fed a “material palette” (i.e. a set of materials to use) and an indexed image – as a result, the terrain will have a reasonable mixture of the materials depending on heights and an ok fading of the materials into one another where they meet so it doesn’t look that terrible, especially once all normals are in place too so that lights stand a chance to work:

The trouble however is that so far I haven’t quite managed to get to the bottom of turning down the reflective mirror-like surface of this landscape! In theory, materials have those 2 light-reflecting properties, namely the extent to which they are “diffuse” or “specular” (i.e. “shiny”). Still, for all my futzing around with those for all three materials (sand, grass, rock), the observed effects are pretty much non-existent. So much for the theory and welcome to this graphics thing of tinkering. To see clearly what I mean, here’s a rather surreal rendering with the ambient light turned on to higher levels so it’s more obvious (note that the effect is precisely the same even with low ambient light – it’s just harder to realise what it is exactly; similarly and as expected, one can mask the effect to the point that “it’s fine” by turning on the fog button – it makes sense since it dulls the ambient light, I’d say):

On the positive side, the futzing with lights and whatnots is really something of less concern for me at this moment since I’m not that interested in obtaining an image *just so* – my goal is in figuring out *what* sort of things one can set, what do they do and how to set them while the game is running. In this sense, at least the basics of the terrain seem to be in place for now: heights are read from a heightmap file, the materials are indeed used according to heights, the rest is for another day. I suspect though that in usual tradition, the only real solution here is to dig in the end in that shaders pit as well since it’s most probably a shader thing: the terrain object uses a different, terrain-specific shader and for some reason the defaults for that end up producing more of a mirror than a soil, at least given *everything else* (such is graphics, everything and anything can be the culprit). Note that I specifically used and set precisely the same parameters for materials as they are in the client currently in use – so it’s most probably something else/somewhere else that I’m not setting just right for now.

The next step now is to figure out animated meshes too, hence Cal3d objects. So in the next episode, it’s Testy himself that will change and get created directly from code. Once I have at least one way to do this too, I can go back a bit and look again into integrating them into (or extracting them out of…) the rest of the PS infrastructure. Then again, before doing that, I might still need to dive in and figure out the shaders in more detail to extract some way of specifying them without xml and directly from code too, perhaps. That promises though to be a lot of figuring out and additional mess so depending on whether it can or not wait a bit longer, it might not be exactly next in line – basically I have lots of work competing for my attention, a sort of inverted looking-for-work, it’s the… (lots and lots of) work looking for me!

My client-side Eulora task set last week is completed and as a result, the Testy character gets now to see something more than just fog in his Sphere-of-all-beginnings, look:

Testy under a Wavy-Blue Moon, in the Sky-Sphere of all beginnings.

It turns out that CS can indeed eat – though not very willingly ¹ – “geometry” for fixed/static entities (aka “generic meshes”) even without any xml in sight ². For static aka non-moving entities, there are 5 sets of numbers required:

1. Vertex coordinates
2. Normals at vertices
3. Colours at vertices
4. Texture mappings at vertices
5. Triangles

The main trouble with the above was the exact “how” and “where” to actually set them so CS understands what they are. It turns out that the most obvious “those are the vertices and those are the normals etc” approach is “obsolete” and replaced apparently by having to specify a render buffer for each set of numbers. Of course it’s still the same arrays of numbers either way, but with render buffers you need to also specifiy which *type* of render buffer you have in mind (i.e. what are those values for) and in pure CS style this is specified by giving the… name. Name that is afterwards internally translated into a numerical id of course but nevermind since it’s supposedly as easy as it can get to have to remember that the name is exactly “position” and not “positions” nor “vertices” nor anything else ³). So easy in fact that I ended up having the list of “buffer names” at hand all the time, great help indeed.

Before moving on to some basic descriptions of those 5 sets of numbers, it’s worth noting that they are just the most usual 5 sets really: there are plenty others that one can specify though it’s unclear exactly when and precisely why would one do. The easiest example concerns texture coordinates: one can specify as many as 4 different texture mappings. And why stop at 4 if one went all the way there, why not 5 or 6 or 7 or 1000? No idea. Anyway, for the basic descriptions:

Vertex coordinates are exactly what the name says: the 3D coordinates of vertices that define the entity’s shape in space. Those are all relative to the position at which the entity is drawn at any given time, of course, since they are internal to the entity and unaware of anything else in the larger world.

Normals at vertices are perpendiculars on the surface of the entity at each of the vertices defined earlier. Those are quite important for lighting (together with the orientation of each triangle surface defined further on).

Colours at vertices specify the colour of the object at each vertex and allow therefore colouring of the full object through interpolation for instance. While in principle this can be skipped when a texture is given, the “skip” simply means default colour (black) so it’s a “skip” only in the sense that it’s less obvious it exists.

Texture mappings at vertices – those are tuples (u,v) that specify a texture point (2D because textures are 2D images) that is to be mapped precisely to each vertex. Just like vertex coordinates provide fixed points for the shape of the engine that is otherwise interpolated at non-specified points, the texture mappings provide fixed points for the “painting” of the shape with a given texture.

Triangles define the surface of the entity as approximated through triangular shapes. All surfaces of static shapes in CS are approximated through triangles and the only difference between curved and flat surfaces is essentially in the number of triangles required so that your eye is fooled enough so you like rather than dislike the lie. This is after all the whole computer graphics in a shell: a lot of effort spent to lie to you the way you like it.

Triangles in CS are given as triplets (A,B,C) where the values are simply indices of the vertices previously defined. So the simplest cube will have for instance 8 vertices for its 8 corners as well as 2*6=12 triangles for its 6 faces since each face, being a square, needs 2 triangles to approximate. And those 12 triangles mean 36 values in total since each triangle needs the indices of all its 3 vertices. Moreover, the order in which the vertices are specified matters as it gives the orientation of the approximated surface ⁴ and that in turn is crucial for calculating lighting (and ultimately for figuring out whether you get to even see what is on that surface at all). Since it’s not intuitive perhaps, it’s worth noting the basic fact that a surface has 2 sides and the “painting” of a texture is strictly on one side only. So if you mess up the surface’s orientation, you can easily do the equivalent of dyeing a garment on the inside without any colour whatsoever showing when you wear it unless you turn it first inside-out.

As you might have guessed from the above example with the most basic cube if from nothing else, even very simple entities require a lot of values tediously and boringly specified – hence quite the bread and butter of automation ⁵, yes. For now and for testing purposes, I simply extracted the numbers of interest from the xml description of the moon object in game and then I used them as default values for an object created directly from code. And lo’ and behold, Testy got a moon to stare at and the moon even got to paint itself stony when I’m bored of its usual wavy blue:

Stony Moon and Testy in the Sky-Sphere.

The next biggish step further in this line is to figure out how to generate terrain from just a heightmap and then how to create an animated (hence, Cal3D) entity from code rather than through the sterile xml-packaging. Looking even further from there, monstrous xml-shaders await in the same direction but in other directions there is still a lot of work to do on teaching the PS code to use the EuCore library and to discover as a result that the world is way larger than what fits in its inherited, well-worn and time-proven, traditional xml-bible.

1. A bit like those finding out that they can actually eat food that was never packaged in plastic or washed in chlorine or even re-heated.[↩]
2. At least not in geometry-sight. In a bit wider perspective, all shaders so far are still balls of xml that will probably have to be untangled sooner rather than later.[↩]
3. If you give a non-existent name, the buffer is simply ignored since supposedly you can define your own buffers too so it’s not an error. Combined with the wonderful way in which everything influences everything else in graphics, you get – if you are lucky – to puzzle over some unexpected visual effect that might even seem at first to have nothing at all to do with the names of the render buffers at all (maybe you messed up a few numbers in that long list, you know?[↩]
4. By convention, vertices are given clockwise as read when facing the surface[↩]
5. Automation can be here exporters from supported tools such as Blender and generators of known shapes/patterns.[↩]

Eulora’s client uses the CrystalSpace (CS) engine for all its graphics needs and CS in turn uses the Cal3D library for animated objects in the world ¹. Some two years ago ² I wrote this stop-gap glue-and-string script to supposedly help artists and all those eager to contribute but held back by a lack of direct way to experiment and see their “art” coming alive in CS. Well, I wrote it and until the beginning of this month, those scripts plus some basic notions from introductory computer graphics courses taken way back during my uni years were about as much as I could say I actually knew regarding computer graphics.

As it seems quite conceivable by now that I’ll end up having to get all the way to the bottom of the graphics pit too, I spent most of this month reading the graphics reference book ³ and in parallel trying to make sense in a very practical way of the whole way in which Eulora’s client handles all the graphics part from the Planeshift (PS) top layer down to CS and CAL3D. And while I don’t yet have a fully working new pipeline for actually loading on the fly all and any sorts of graphics into Eulora’s client, I have at least a hard-won better understanding of what goes on in there (despite the layers upon layers of obfuscation) as well as some bits and pieces that might conceivably even make it into such a pipeline in the end. So I’ll jot down here my findings so far and those yet-moving ideas about it all, as it helps for sure to structure and unload a bit in preparation for the next stage of work. And it might even help – says ever optimistic me – to start perhaps a discussion with anyone who knows this better than I do so if that’s you, kindly comment below and let me know your thoughts.

One difficulty with CS (and it seems to me with Computer Graphics in general really) is this persistent idea of accomodating all sorts of “alternative” ways of doing things that turn out in fact to be precisely the same thing only packaged differently so that nobody is left behind or something. So instead of having a clear structure of knowledge to acquire, one is supposedly left to be creative and find what “feels natural” or “comfortable” to them. Well, so much for actually learning anything there, welcome to lots of code that does the same thing, only it’s harder to tell that upfront without sinking hours in it, to follow it all from one end to the other. Nevertheless, after all the reading and poking it and changing it and having my way with it, my current basic structure of CS ⁴ notions of interest for client graphics is this (ordered from top down, as much as I could):

1. Sectors – those are the highest-level containers of “graphics” of any sort. A sector simply stands for a “location” that is effectively defined by nothing more than a name ⁵ and two mixed characteristics: on one hand the visibility culler in use (so a computational matter) and on the other hand the ambient light (i.e. the light that affects everything in that sector in the same way at any position and as such a matter of aspect pure and simple).

   CS supports two visibility cullers, namely Frustum and Dynamic (dynavis) with the default for the game’s needs being the dynavis culler. So a first building block for the desired client pipeline is simply a “GetSector” function. Like all the rest of the “GetX” functions that I have in mind, this will retrieve any information it needs about the sector from EuCore / EuCache (via a CPP-to-Ada layer, as needed) and then either retrieves the required sector from the engine itself if it exists already or otherwise proceeds to creating it and loading it into the engine so that at the end of it, one way or another, the sector is known to exist. When information is missing/incomplete/unavailable, the GetSector method fails and lets the caller decide what to do (try again at later time, most likely). At this moment I have a prototype of this method as well as some similar prototypes for GetLight, GetTexture and GetMaterial – in other words, I can create from code without any xml and parametrised four main things: sectors, lights, textures and materials.

2. Lights – those are sources of light in a sector beyond and in addition to any ambient light. They can be customised in all sorts of ways (e.g. intensity, colour, position, direction). Whether they are elements inside a sector or an environmental characteristic of a sector is unclear to me at this stage since they could easily be seen either way. From an implementation point of view however they are currently clearly elements inside a sector, NOT characteristics. This means among other things that one could in principle add/estinguish/modify lights as they please at any given time and I’d say this is how it should be, given that it’s not inconceivable to be able to turn on a candle or any other light in one’s own house for instance.
3. Weather – the more general name for this would be environmental factors I suppose, but so far it’s really at most “weather” and in practice mainly… fog. Supposedly one could add here all sorts of *effects* really so from an implementation point of view I’d even call this part “environmental effects” i.e. whatever effects manifest themselves in the sector as a whole and are specific to it. The fog itself so far appears to be a rather simple matter as it’s essentially a limited modifier of ambiental light. I don’t really know if this is just a limitation of CS or a more general approach and for that matter I can’t for the life of me quite grasp yet exactly how do graphics-people decide whether some effect they want is rolled into “fog” or into “ambiental light” or into anything in there. This is an issue at all levels and so far it really looks like an individual preference than a choice, quite everywhere. In other words a matter of “am I more skilled in making it look just so by tinkering with this or by tinkering with that”?

4. Meshes – those are any and all elements in a sector that have what CS calls “geometry”. This geometry concept is a bit fuzzy at times but as far as I managed to pin it down, it would seem to mean essentially that the object has a shape defined mainly as a set of “vertices” aka points in the 3D space that the object occupies. There are 3 main types of Meshes:

   1. Movement-capable entities
   2. Fixed entities
   3. Terrain

   Each of the above types of meshes has its own specific requirements and even ways of defining itself, including its geometry (hence part of the fuzziness…). At least in the current setup and as a very brief overview, movement-capable entities are effectively CAL3D models defined as hierarchical structures of meshes with attached action representations (e.g. run/move/jump/fight animations) and skeleton-based geometry; fixed entities are CS’s “generic mesh” with vertex and triangle-based geometry; terrain entities are CS’s “terrain2” mesh type with attached terraformer for generating the geometry based on a heightmap. The details of each are much gnarlier than this list might suggest and I’ll be teasing them out one by one as there is no way around that but at any rate, CS’s approach seems to have been to make different xmls for each of them and all sorts of “loaders” on top of loaders and on top of plugins, services, parsers and whatnots. Then PS added its own loader on top of CS’s pile of loaders. As you might imagine, this tends to rather obscure the precise way in which one can create those different meshes *without* xml.

5. Factories – those are abstract concepts containing effectively the blueprint for creating identical meshes. The idea behind them – as far as I can tell – is that it’s worth loading in memory what is needed for a mesh only once and then reusing it every time a new specific instance of that precise mesh is required. In practice I’m not sure that this is really worth much for anything other than snowflakes or raindrops really, since otherwise pretty much each instance of a mesh seems to have its own factory anyway. There is however the argument that the factory should contain only truly common structure (i.e. all players of one race have the same structure) and then leave the customisations to each instance. So keeping this in mind, here they are in the list, factories for meshes (of all sorts, though indeed, each of the three different mesh types has its own specific factory type on top of a generic type).
6. Materials – as far as I can tell so far, a “material” is essentially any specific combination of one shader + a whole set of values for the shader’s parameters (aka “shader variables” in CS terms). Note that the texture itself (or themselves as apparenlty one can use several although it’s not clear to me WHY is that any better), if there is one, is effectively an argument passed on to the shader.
7. Textures – those are the pretty picture that is lovingly painted on some/any surface that an object in the world may show to the player at some time or another. Essentially each object is but a set of surfaces linked together and each of those surfaces may be covered with one or another texture fixed with some spit so that it goes exactly over that bump or starting from that point to end up just-so. In principle, one could provide instead the way to calculate the texture at any point but that seems to be less frequently used in practice (CS claims to support it though, although I can’t tell to what extent and with what results really).
8. Shaders – this is where the fuziness increases to madness levels. On one hand, “shader” seems to really mean all sorts of things in different places, from yet another description of a surface to any code running on the GPU. There is obviously more reading that I need to do on this aspect but so far what I’ve got is that CS’s “shaders” are really just another layer of encapsulation meant to be for types of surfaces. So if one has types of meshes encapsulated in factories, one also has types of surfaces encapsulated in shaders and is able therefore to apply some “surface type” to any given texture so that the same picture will look “cloth-like” or “stone-like” depending on the nature of the surface on which it is applied. This is about as clear as I can get it now and I can’t say I like it much. On top of this, shaders in CS are yet another pile of layers and layers of xml with intricate dependencies on one another and on who-knows-what-else in the name of reusability ⁶.
9. Render Loops – those are (in usual graphics style as far as I can tell) simultaneously in parallel and on top of the “shaders” above. They are yet another encapsulation, this time of types of rendering + lighting objects. To make the difference between shaders and render loop abundantly clear, default render loops are of course provided by CS, as files in the /data/shaders directory. Admitedly I totally lack the expertise to say at this stage just why are BOTH shaders and renderloops needed as separate entities and/or why not either do with only one of them or otherwise merge them together in a single thing at least but hopefully one of my more knowledgeable readers clarifies this for me. Failing such clarification, I’ll need to keep on reading and experimenting with it, of course, for as long as it takes, what else.

In addition to the list above, there would be also the easier part of interface “skin” aka the variious icons and decorations for windows, buttons, labels and whatnots. This part is also on the list to be made fully reliable on data received from EuCache but so far I haven’t touched it yet (other than keeping an eye out for what I could do about it, whenever other bits and pieces make me go close to that part of the code that has anything to do with it, of course).

For a fresh image, the TestChar at least moved out of its cubic box and into a rather faceted sphere, in his quest for the promised infinite landscape. Lights got a bit better, shader variables were sorted so as to neither block nor directly crash CS anymore. Next on the list is fully figuring out just how to properly feed “geometry” to CS as what it is, namely several sets of numbers (vertices, texels, normals, triangles) rather than all those + a ton of xml on top. This is a required step in order to have the GetFactory prototype too for static meshes at least. Once that is done, the next step is to move on to factories for terrain and then for animated meshes. Until then though, here’s Test-in-his-sphere:

1. For completeness, I should mention that CS has also its own native support for animated objects via 2 different sort of objects even but neither of those is as flexible and overall useful as CAL3D is. Therefore, I simply consider that animated object = CAL3D from now on.[↩]
2. Already two years passed! And to think that I’m picking it up precisely from where I left it all this time ago, kind of boggles the mind.[↩]
3. Foley, J.D., van Dam, A., Feiner, S.K. and Hughes, J.F., “Computer Graphics: Principles and Practice”, 2nd ed., Addison-Wesley. Yes, I’m aware that there is a newer edition of this book, supposedly improved, better, shinier, more fashionable whatever have-you. I’m also aware that it’s full of what can only be called advertising for Microsoft while being way lighter on what I’m interested in, namely precisely principles of computer graphics. So I’ll stick with this edition, thank you.[↩]
4. Note that there is no desire to make all Eulora somehow CS-dependent as such but at this stage, I have to at the very least support CS as it were so I’m aiming for using CS as a practical example of handling client graphics, not as a driver of standards or content.[↩]
5. CS uses names aka text/strings as identifiers to the extent that it *requires* those for ~all the Find functions at all levels. It *does* have its own internal IDs and it even exposes them if one tries really, really hard to get to them but they are basically not very useful otherwise and moreover they can’t even be set to match an external set of IDs (i.e. one can’t create an item with a specified ID.) [↩]
6. “Reusability” seems to mean that the original author can skip writing a few lines of code at the cost of nobody else being able to understand afterwards at all how the thing works unless they spend at least double the time that it would take them to write it from scratch anyway.[↩]

A crucial requirement of Eulora’s new client is to actively request from the server ANY data that it may find itself missing at any point in time. At first glance, this seemed to me simply a matter of providing request services ¹ from Eulora’s new Ada-based core and then adjusting the existing C/CPP code of the legacy client to make use of those services. This rather optimistic idea is of course plain wrong: “adjusting the existing C/CPP code” in this context is similar to saying that one “adjusts” a sheep to use the library – while it can certainly be done for various definitions of “done” and the sheep may indeed use the library one way or another, it’s at best a huge waste (of time, of resources, even possibly of steak) for everyone involved and no matter how one looks at it.

Even leaving aside for a moment the trouble with “adjusting” the legacy tangle in any direction, the more important issue here is that this requirement is not as much a functional requirement as a non-functional, quality of service requirement: whichever part of the client provides data services, it should better be dedicated to the task and do whatever it takes to get it done instead of “providing” it only fair-weather style – if it’s easy, there you are and if it’s not easy then it’s your problem really. In other words and in marked contrast to the very democratic “best”-effort-you-can-do-anything-anytime existing C/CPP code ², the new code should have a clearly defined task and then either complete it or die trying over and over again, taking full responsibility for the process involved, not just for some specific detail conveniently chosen nor – as an excuse for not delivering – for the outcomes that are not fully under its control ³.

Considering therefore “active and dedicated request” as a quality of data service on Eulora’s client side, it follows that its place is rather close to the data cache mentioned previously and at any rate inside the new Client Core since it’s certainly not some additional part of client logic nor some bit of user interface. However, I’m reluctant to make it the responsibility of the cache itself since the cache is a passive structure that focuses on *storing* data and *providing access* to it. Mixing passive data storage with active data acquisition doesn’t make much sense to me and even seems ill-advised for Eulora’s client given the competing requirements: on one hand passive, immediate-response local data storage and on the other hand active, possibly-delayed and world-facing (i.e. communicating with the server) data acquisition. So I’d rather avoid this passive-active construction and have instead the two as separate entities: a EuCache dedicated to storing and retrieving on demand *any* data; a Requester dedicated to acquiring *any* data that is demanded of it. Note that the definition of “acquiring” here has nothing to do with the means through which the Requester actually gets this data (specifically nothing to do with the exact messages sent/received/exchanged with the server). Acquiring some data means simply that the required piece of data becomes available in the local cache aka EuCache. So the Requester will keep requesting this data from the server through whatever means it knows until either the data arrives and becomes available from EuCache or otherwise the whole client kills itself for lack of server ⁴ and therefore of any possibility of playing the game.

Specifically, the Requester will be implemented in EuCore (and therefore in Ada) as a protected object exposing only a few procedures that are essentially notifications: some are notifications of demands for some specific piece of data (either an Object or a File really since those are the only 2 overall types of game-data that one can request from the server); the others are notifications of data being received or of timeout interval having elapsed (in other words a notification of failure to receive data). Note that the demand for an “Object” effectively means a demand of its properties and those might be indeed anything at all but each and every one of them will be either directly a value or otherwise an ID of another Object or of a File. All notifications (including those demanding data) are always accepted by the Requester but not necessarily immediately acted upon. Clients of the Requester do NOT control the actual requests to the server or messages exchanged and are not even concerned with those at all – the production of actual requests, their content and their timing are entirely the job of the Requester and under its sole control. Implementation-wise, the Requester will simply keep queues of requested Objects/Files and will then proceed as soon as it can to pack a request for as many of them as possible; this request will then be posted to the server and the Requester will set a timer to the timeout value so that in the worst case it is at least notified that nothing happened; when/if any data is received or when this timer expires, the Requester will check in EuCache to see what items (if any) of those requested are now available; any items that have become available will be discarded from the watchlist of the Requester (i.e. the demands for them are considered completed) and a new request may be created and posted to the server for any items that are still in demand but not yet available. Note that even in the event of a timeout, a “repeated” request to the server may not be identical to the previous request since the list of demanded data possibly changed in the interval.

One potentially iffy point for the Requester is its need to be notified of any incoming data. At the moment I don’t see any real way around this, short of making the Requester poll at set times the EuCache and checking if any data of interest has meanwhile arrived. I don’t really like this polling approach here because it’s rather wasteful without good reason: any incoming data is indeed received and processed by another unit that is also on the same level with the Requester, namely the Consumer (the part that processes messages from the inbound queue). So the Consumer will have to notify the Requester when new data is received. While several Consumers may be active at the same time (at least one for Serpent and one for RSA messages) this is not a problem at all since the Requester is anyway a protected object i.e. thread-safe. Note also that even if (some of) the consumers fail to notify the Requester of some incoming data, the whole thing will still work if only slower than it could: the timeout timer will wake up the Requester and the check of data will happen there at any rate. In other words, the Requester is capable of reacting to events if notified of them but not dependent on those notifications to do its job correctly.

Given its rather complex task, the Requester is currently on the top conceptual layer of EuCore, making use of quite a lot of other units from lower levels. Currently, the main relevant units on this top level are the following:

• Data Cache aka EuCache – this is a passive, thread-safe entity responsible for storing all and any data given to it and retrieve or delete it on demand. As such, it *owns* the specific format in which data is stored ⁵ and it simply exposes functions and procedures for storing, retrieving, deleting and checking for data.
• Communication Link aka Comm_Link – this is a passive, thread-safe entity responsible for persistent storage and updating of communication details, most notably RSA and Serpent keys for inbound and outbound communications as well as a message counter. This is effectively a specialized cache – where EuCache is for game data, Comm_Link is for communication protocol data. The requirements (including use contexts) and specifics of the two seem to me sufficiently different to keep them separate at least for now.
• Consumers of incoming messages (RSA and Serpent) – those are separate, active tasks, responsible for processing incoming messages. Consumers own and get to define what “processing” means exactly but their role is to extract the data contained in the messages received and make it available for use by Eulora’s client. In practice this means currently that Consumers will pass any data received on to EuCache for storage and will notify the Requester of any data receipt.
• Requester – this is an active, thread-safe entity responsible for acquiring data that is in demand. It owns the process of data acquisition from the server and it accepts any demands of data specified by some identifier. While it guarantees that all demands will be served at some point in time as long as the whole program is running, it does not (and can not) guarantee when or if the corresponding data becomes available. It can’t even guarantee that there IS any corresponding data and so ALL it can do is to guarantee that it will try with the same dedication for each and every bit of data to acquire it. Anyone demanding data can of course pester the Requester with more demands or give up or decide for themselves for how long they can or will wait.

And now that the main idea and overall design of this whole thing is at least quite clear to me, there remains of course the small bit of actually implementing it in full (rather than the current skeleton I already made) and sorting out the various implementation-level troubles that will appear as they always do, in all sorts of details. Relatively easy work compared to what follows it inevitably, namely teaching that C/CPP sheep to use the EuCore library…

1. Mainly picking, packing and encrypting everything in the right format + sending it through to the correct place.[↩]
2. Seriously, think of it: existing client code is this event-driven thing where *anyone* can subscribe to any event and then “do” anything at any time provided of course that the any and anything are in fact very narrowly defined and set in stone even to the level of names of various art files (it has to be a zoneinfo.xml inside this and that and put exactly there, sort of thing). If this narrowing of “do” was not a price high enough to pay for such code “liberty”, there is also the added reality of a huge tangle that does precious little indeed since everyone ends up calling anything from anywhere and no piece of code is really and truly responsible for anything bigger than a few variables here and there. And at the end of the day how could any code even be responsible for anything since it can’t *own* any process by itself (shared! event-driven!) and it has to be passive, mainly reacting to some events or at most… signalling through events of its own but never able to rely on anyone giving a fig about its signalling! So there it is, code – like anything you do – is more of a mirror than you might think. And “teaching people to code” has way more layers than “teach them Java” and certainly more issues than current “courses” even seem to be able to imagine.[↩]
3. And now that it’s clearly stated like this, tell me first just how many people you know who actually handle their own work like that? And just what bit of “programming language” you think one needs to teach so that you get programmers to actually design their code this way?[↩]
4. This “lack of server” is defined in practice as a number of successive timeouts on requests sent to the server, where the specific threshold value is chosen by the user via a config file.[↩]
5. Currently game objects are stored in memory in Hashmaps aka associative arrays while art/files are stored directly on disk. Note however that any of this can be changed without touching anything outside EuCache as it’s nobody else’s business.[↩]

As I’m doing anyway the work required for a saner Eulora client, I’m stuck as well with writing a minimal documentation for it, since on one hand there’s no one else who can quite write it anyway and on the other hand there is no such thing at all ¹ for the old piece of inherited gnarl so the slate is blank, the space infinite and the need impossible to know in advance. So I’ll just jot it down here for future reference and if you have anything to say about it, the sooner you write it down in the comments section below the better for you.

As general structure, the current plan for the new client is to have essentially 2 main parts: one is the actual Core, written in Ada and – hopefully ² – provided as a standalone library, blissfully unaware of ANY of the C/CPP legacy tangle; the other is an updated version of the current C/CPP tangle of a “client” that will use the Core for pretty much everything except the actual GUI. The GUI remains provided via CrystalSpace and cal3d, as it currently is. In other words, the Core offers an implementation of a basic Euloran client, to be used from/by any UI, graphical or otherwise as you care to make for yourself. Note that the interface is NOT the only thing you can build on top of the client Core. Essentially any client logic, bots and anything of the kind would still be the sort of thing built on top of / around the Core. A quick diagram of the whole thing looks like this:

The Core itself is currently made of 3 conceptual layers, in order from bottom to top: encryption utilities (EuCrypt), SMG communications (SMG_Comms), client data cache. The encryption utilities are those of EuCrypt: CRC32 calculation, Serpent encryption/decryption, Keccak encryption/decryption, RNG interface using Fuckgoats, RSA key generation, RSA encryption/decryption using TMSR OAEP schema. The SMG communication part provides mainly: UDP with fixed-size but configurable payload, in/out message queues and attached “consumer” tasks, config file reader, packing/unpacking (i.e. raw, bottom-most layer of SMG communication protocol), read/write of messages conforming to SMG communication protocol specification, thin layer on top for encapsulating a “communication link” i.e. required addresses, ports and keys for both ends. The client data cache keeps effectively the whole world of Eulora as the client knows it at some specific time and provides thread-safe getters and setters for anything and everything that it can possibly hold.

Note that the client’s knowledge of the world is by definition a cache as it flows always from the server and entirely in response to client’s own requests – there is by design no “push” of information or anything else from server to client at any time. So the client may request from the server anything and everything it wants to ³, having however to decide for itself what to make of the answers it receives and even what “obsolete” means in various contexts. At the moment this cache is still in the works and for this reason the least fully-fledged but at any rate, it has to grow around the structuring of data that is discussed in the SMG protocol specification: a potentially infinite hierarchical structure of “objects” ⁴ having each a subset of an overall set of potential “properties” ⁵ with their corresponding “values” ⁶. Due to this trinity of objects-properties-values with its corresponding trinity of ID spaces, I took to calling the whole thing a tricache but the name as the design are still in the works so they’ll have to enjoy their own detailed description at a later date. Until then, what have *you* been working on and how is that going?

1. Perhaps you count this spaghetti piece as documentation but even so, just note who extracted it in the first place, anyway.[↩]
2. I’m holding quite tightly to this idea as I certainly do NOT want to have it depend on any of the current C/CPP mess. This being said, I can’t know in advance what I might still uncover so for now and until it’s all done, the qualifier remains “hopefully” rather than surely.[↩]
3. This in practice is of course quite severely restricted by what it *knows*, unsurprisingly.[↩]
4. Where “world”, “map”, “actor” or “pot” are equally objects and not even the only objects. Essentially an object is anything that is present in the game’s world.[↩]
5. A property is a pre-defined and public structure that specifies the *type* of values that make up the property and their actual meaning.[↩]
6. The values may be given directly (e.g. “3” or “somename”) or by reference aka as an ID. This ID may be itself either the ID of a suitable object or the ID of a value that is essentially too big to pass around every time – in the protocol’s specification this sort of ID is currently called for historical reasons “filename” but it’s worth noting that there is no imposition for it to be stored as a file at any point.[↩]