Pseudo Random Journal Generator Predictable permutations on programming, technology, and more.

adam_03may2009.plan

I decided I would begin updating this again, if for no other reason than it helps to organize thoughts 😛

Nothing big this time, just thought this random musing on Ubuntu privacy would interest someone.

So Ubuntu 8.04 and above gives you a Private directory by default. It is an encrypted directory that is only mounted and decrypted when you log in, which makes it ideal for storing things you don't want other users to see.

But it's not as private as it might seem right off the bat. If you are logged in, and thus the directory is mounted and decrypted, any other user logged in at that time can simply navigate to it and browse it's contents!

This isn't really a security hole or anything, it's just how it is designed. The real problem lays with Ubuntu's default permissions for user's home directories.

Let me restate the problem so it's clear: a Private directory should not be able to be browsed at all by other users. It's nice that it is encrypted which protected it from many kinds of prying most of the time, but still, I don't want people to be able to snoop in it ever!

So the fix is simple, when in your user home directory:
chmod 700 Private

Now ONLY someone logged in as you may browse your Private directory. period. But this got me to thinking, and I wanted to take this a step further. It seemed to me, that it didn't make sense that ANYONE could browse my entire user home directory. So next I restricted my entire user directory in a similar fasion, while sitting in: /home I did:
chmod 700 myUser

Which *worked* but gave me the predictable result that now the public_html directory in my user directory was now inaccessible to Apache :/

Clearly not acceptable. But the fix is easy. First, make sure your public_html directory in your user folder is part of apache's group, on my system that is www-data:
While in your user directory: /home/myUser do:
chown myUser:www-data public_html

And make sure it's permissions are: 755
Then, go to the home directory: /home and make your user ditectory part of the www-data group:
chown myUser:www-data myUser

And finally, make sure your user directory has 750 permissions:
chmod 750 myUser

And now your entire user directory is only readable to you and Apache, your public_html directory is still good to go. And your Private directory is completely locked down to JUST you. Privacy catastrophe averted! Or something like that 😛

Hopefully I'll have more interesting things to write about as I go forward here!

- Adam

adam_07may2008.plan

Just typed up a pretty good entry here... then lost it due to stupidity... (and the back button).

But it's been a long time since my last update, and believe it or not, quite a lot has happened in the past 5 months.

Decrypt got quite a bit of attention here and there. It's at a pretty polished v1 stage. And I hope to get some mission 2 stuff going soon as well as a possible graphical make over.

I also wrote some articles, on the left nav bar there. As well as two new projects. Nothing big, just some quick little projects I did on the side.

Thats all fun stuff, but I guess the news is that I'm graduating in less then two weeks.

I haven't really had time to think about it so far. I've had a crushing class load which has been pretty all consuming. Almost done though, so just gotta keep my head down and work hard these last two weeks.

After graduation I'm moving down to Dover to finish up that project. Should be pretty exciting finally releasing one of my babies into the commercial market like that.

I will try to keep this more up to date from now on, so check back if you're interested.

Wish me luck on getting through these last two weeks 🙂

- Adam

adam_20nov2007.plan

Spent a bit of time fixing up some code in Decrypt, killed some show stopper bugs dealing w\ the OS API for the applications which affected progressing through missions. Also upgraded PHPFreeChat to v1.0 final. PHPFreeChat is the software behind the chat application.

Gotten quite a lot done on the console project lately too.

See, in programing you quickly come to two assumptions which are fairly safe to make.

1) The programing language is right, your code is wrong.

2) The 3rd party library you are using is right, your code is wrong.

The first is pretty darn stead fast, and the second isn't far behind (depending on how mature the library is of course).

So you'd think it'd be safe to assume the second when you are using a 20 year old, time tested library put out by the likes of a company such as NewTek (makers of LightWave).

But... you'd be wrong...

Lets examine this piece of code, it comes from lwio.c which wraps all of the I/O functionality that the rest of the files use. In there, there is a specific function which is used to read a NULL terminated string of bytes (not necessarily ASCII characters, but it doesn't really make a difference). We know this is it's purpose because we have read the documentation and looked at the data it is actually called on, it is always NULL terminated.

Now take a look at the function and tell me if you can spot a problem with it:

   for ( i = 1; ; i++ ) {
      c = fp->fgetc();
      if ( c <= 0 ) break;
   }

Ok, the point of this piece of code is to count the size of bytes that the data is so we can allocate memory for it.

First, ew. Ugly code. But lets trace it, we keep reading and incrementing our count variable until... until what? Until it is less then or equal to 0...

HHhhhmmm... does that make any ef-ing sense if what it is we are looking for is NULL ( NULL == 0 btw )?

Assuming we just did not understand the actual point of the code we continued reading the spec, looking at the data it was reading in a Hex editor and running through everywhere the read function was called.

With out a doubt it was looking for a NULL terminated string of bytes, and that chunk of code was meant to count up to the NULL character.

So we quickly replaced the bugged line and came up with the more explicit:

   for ( i = 1; ; i++ ) {
      c = fp->fgetc();
      if ( c == '\0' ) break;
   }

Now on a side note, the most astute of you may notice the file I/O using what appears to be an Object or Struct pointer. Well you are right, we ported the code to C++ and use our FileBuffer and FileIterator classes to emulate std:: file I/O operations. We looked a lot into how our getc() function might have been working differently then std::getc(). But I can assure you, that it works the same.

So back on topic here, it's not hard to see the case in which the original code would fail. Anytime the byte overflowed as an unsigned 8 bit int, it would be negative. The specific byte that was fucking us over was 0x80.

So, with that fixed, we finished porting lwobject and fixing bugs that cropped up in it. Then I went on to extract all the information we needed out of it and create our internal data structure for holding the data how we wanted to render it. Let me tell you, the UV coordinates were a bitch to find.

Just to get the texture file path, it took these two lines of code:

lwClip *clip = lwFindClip( obj->clip,
layer->polygon.pol[ii].surf->color.tex->
param.imap.cindex );

char *texturePath =
clip->source.still.name;

Anyway... With that all done I finished connecting all the dots between all of the supporting systems and the renderer and finally, finally, got Textured, 3D, rendering working.

So the current state of things is and solid complete base with 2D and 3D loading and rendering working.

Next, we start on integrating ODE! I don't want to jinx things, but it should, *fingers crossed* go a lot smoother then the other integrations have.

- Adam

adam_02nov2007.plan

So quite a bit of progress on the console development front.

As always while coding a big project like this, no matter what the actual task at hand is, I always clean up and refactor tons of of little semi or completely unrelated things along the way. So lots of little parts of the engine have been cleaned up, is the sort of central theme of what I'm trying to say I guess...
arrr...

Annnyyy waaayyyy...

The engine now has a fully functional scene graph as well as a fully functional state manager / input handling system.

With all that in the bag, Rob and I working on integrated libPng and LibZ for use as your texture loader, and moved on to getting texture rendering working inside our engine (much more difficult then you'd have thought).

Moving forward we ported some file loading code for .lwo (LightWave Object) files. Let me tell you... not fun.

The code was ~20 years old. It was made when C compilers had almost no type checking apparently. So the first task was just getting it to compile in a modern C compiler. Then the porting could begin. To get it running on the console we had to use a custom File I/O class we have developed to assist with porting libraries. Bringing the old C code in C++ was even more fun.

Ever seen a function call like this?
lwListFree( (void*)texture, reinterpret_cast( lwFreePlugin ) );

I don't know about you, but I'd never seen this arcane syntax before, but this what I had to come up with to get it to compile.

See, most modern code can be ported simply by dropping in our special header file. But some older code is more difficult. So we developed this class them emulated most of the C style file I/O. While we still have to go through and modify many many lines of code, our class is structured to work exactly like the old C style functions, so the structure and flow of the code need not change.

We have been developing good strategies and a good code base for porting libraries. So far we ahve ported libPNG, libZ (required by libPNG), lwobject (LightWave Object code), and partially ODE (Open Dynamics Engine).

While we did in fact get the lwobject code compiling and running, we still dont have the renderer coded to read and render the data from memory yet. Thats the next task. But with all the heavy lifting done by lwobject, it should be a sinch.

The real next task will be getting ODE running, verifying that it works, and plugging it together with everything else.

So starting to get close to some pretty exciting stuff!

- Adam

adam_12oct2007.plan

So while I had created a graphics test that had a textured polygon rendering to the screen, it was still relying on some sample code that is provided with the SDK to set up the console's graphics system.

About a week ago we made the decision not to depend on any of this sample code, which is why we had to write our own memory management code. Since then I also wrote our own code to set up the input from the controllers (Once I got it working I implemented: A button turns the motor (vibrator) on, and B button turns it off... we are planning to market it to the small but growing market of lesbian gamers 😉 ), and yesterday I finally wrote all the code to setup the graphics system.

So finally free of the constraints of the SDK demo code, we started looking into a few remaining issues before we start the actual game. Geometry loading for one.

With our new memInit.h file, we can pull in any C or C++ file we want and have it work out of the box now. So after searching around a bit we have decided to go with the MD3 model format. Used in ID Tech 3 ( aka Quake 3 to the rest of us ) it is a great format that has been evolving for some time and is battle tested. And since it is such a widely used format, we can just grab an open source loader and drop it in with our memInit.h doing all the porting 🙂

So I guess the next thing to do is going to be, create an extremely simple menu from which u can exit or enter the game, just to show the system works, and then I will begin integrating ODE (Open Dynamics Engine, the open source physics engine we plan on using) and start working towards a playable version!

adam_03oct2007.plan

I think when we are kids and we hear some one does such and such "professionally" we are taught that it means they are very good at it. And that because product X comes from big huge company Y it must be very well designed. I mean it sells well and they have been making products like X for a long time so it must be good.

But as I've gotten older it's become starkly apparent that these assumptions simply do not hold true. Many times Mr. Big Professional actually sucks at his job and makes bad decisions despite his 10 years of experience. And product X probably has many design flaws that simply should not exist considering the fact that big huge company Y made it and has been making products like it for a long time.

Case and Point. The Source engine powering, among others, the hugely successfully Half-Life 2. The full source is available for mod developers and I personally have looked at much of the source and even coded a very simple mod with it. And I can say with out pause, it is a disgusting mess.

Let me digress for a second. There is a big difference between academia and the commercial world. And while, for example, in academia global variables are the root of all evil, in real world commercial software, that are sometimes necessary. My point here is, products, and software in particular, in the commercial world may have to be a bit "dirty" by academic standards. But if it works, and it works well, then it's all good.

With that in mind lets return to my main point here about the Source engine. It is a total, complete, disgusting, nauseating, F$%*ing MESS. Commercially acceptable *dirt* was left way way WAY behind when making this engine. Their poor design decisions alone would make baby Jesus cry, not to mention their archaic coding practices that make debugging all but impossible. THE PREPROCESSOR IS NOT A COMPILER PEOPLE! STOP TRYING TO USE IT AS SUCH!

And I'm not the only one that feels this way, go on any game development forum (Where game engine design is usually discussed, these conversations are usually more academic, but still) or even a software engineering forum, and when you see people talking about the source engine it's usually something along the lines of:
"I used the Source engine once... *stares off into the distance*"

But big supposedly well designed software that sucks doesn't stop there. Talk to anyone who has worked on open source projects such as GAIM and they will tell you horror stories of how dirty the code is. (To be fair, GAIM is now called Pidgin, uner new management now, and from what I hear the code base is hugely cleaned up)

Now if you can even remember my original point, it was that big commercial products suck a lot of the time. I could go on, but I think I've provided enough examples.

This console how ever... is. awesome. I've been digging through the technical docs for weeks now. The hardware design decisions, the API design decisions, all of it is genius. Yes there is a huge learning curve to the APIs, but the reason is what I described in my last .plan file, they give you an incredible level of control very close to the hardware to be able to squeeze every last ounce of speed out of that hardware.

I've heard horror stories about the PS3 hardware and SDK. And while the console's SDK leaves something to be desired in terms of the IDE and some of the documentation. It is still fairly good. And once you do find the info you need, you realize even more how well it actually has been designed.

What is clearly evident when developing for this console is how they have learned lessons from every console they have made. I wouldn't be surprised if it was mostly the same team sticking together to make each new console.

Anyway, this is dragging on. It may seem like all I do here is sing Nintendo's praises but let there be no mistake about it. I'm no Nintendo fan boy. The last Nintendo system I bought was the SNES. And while I did love that system, I scowled at their latter decisions with N64 and GameCube. I didn't like the DS (the 2nd screen seemed gimmicky, no real use). In short, I hadn't owned, or for that matter, played, a Nintendo product in years. When I saw the Wii and it's new controller, I figured it was just another gimmick like the 2nd screen on the DS and I didn't get very excited.

So when I got this job developing a game for this console I entered there a unbiased scientifically minded observer. And I can honestly say, Nintendo has impressed me with their technically decisions, their marketing decisions, and their ingenuity.

So heres to Nintendo!

- Adam

adam_26sep2007.plan

Long time no update, so let me fill you in. I finished the Decrypt beta and it was fairly well received by the gamedev.net community. It is of course a very niche game, but I have had some very enthusiastic responses. I believe once I get some time to do some bug fixing and improvements and add another mission things will take off a bit more for it.

But more to the meat of what is going on now! I'm now working for local company developing a game for a major game console. It's awesome seeing the console side of game development.

Here you can see the first graphics I have got rendering on the our console with code of my own:

What you are seeing here is I have set the "clear color" to green, which is the color the graphics system resets the image buffer with. And I have drawn a polygon (quad) with a texture I stole from some demo in the SDK. I would have used my own texture but the tools to encode it in the proprietary format kept crashing :/

It's been very interesting seeing how the console was designed. I think it's quite genius.

See the Playstation3 and XBox360 went with the approach the computers have gone with. They say, computers are fast enough as a whole that we can sacrifice some processor cycles to gain some developer cycles. Essentially, make it easier and less time consuming for the developer at the cost of run time performance. It's what computers have been doing ever since the first compiler was written. And every time we go another step towards RAD (Rapid Application Development) you always have some one who cries about the lost performance, and in the end the higher level approach proves to win out.

But in this case, the console's designers has a good point. See, for old consoles like the SNES and Sega, you had to code entirely in ASM. Think back on the complexity of some of the games you played on those systems! And they had that all running on hardware that is laughable at best even for the day. Thats why consoles used to be cheap. But then newer consoles like Playstation began to use higher level languages. Now the XBox 360 uses an interpreted languages even (XNA is based on the .NET framework).

SO! (Sorry for being so long winded) MS and Sony have to pack a ton of hardware muscle into their consoles to coupe with the lose of performance and still get the jaw dropping visuals you see.

Mean while, the console's designers thinks the price of these consoles are ridiculous, and the developer should absorb the cost (in terms of development time) and develop closer to the hardware.

Thus cheaper, less powerful hardware can produce at least equal visuals.

Sure the learning curve is steep, that image I posted took 2 days of head smashing to get rendering, but it's more efficient at run time (if only slightly) then say, OpenGL, and DirectX. (That time also has to do with the fact that the documentation is in many cases not even correct. I'd go into more detail but I feel that could breach the NDA.

Any way, if you happen to be interested in console development or game development in general stay tuned, I plan on chronically this process.

- Adam

adam_31jul2007.plan

Quite a bit happening lately. I finished the mission system which utilizes the event handlers I finished last time. And I implemented a one sequence mission.

Then I went on to implement the in-game chat I had been talking about. I ended up using PHPFreeChat which is an AWESOME little script. I plan on either donating or finding some code to contribute so I can remove the badge. Not that I don't want to recognize their efforts, I just want a very cohesive feel to it all.

With that finally implemented, tweaked, and bug tested, I started working on the Web Browser which is going to be a huge part of the game going forward.

It shouldn't take much more code to get the browser functioning. Then I will implement a more complex mission involving the browser and email together.

I also plan on implementing a console. Mainly for easily outputting debug info but maybe later for use as a command line interface.

So thats it for now, stay tuned, very interesting things to come!

- Adam