| Linux >> OS X for running the Second Life viewer |
[May. 5th, 2009|08:33 am] |
I just "upgraded" my MacBook Pro 5,1 (the late 2008 unibody model) from OS X to Ubuntu Jaunty Jackalope. The trackpad doesn't work quite as well as it did under OS X, and at best I can get sound out of one speaker (my USB headset works fine), but the performance of the Second Life viewer is absolutely amazing. I get a smooth 30 fps even in complex scenes with all the shaders enabled ("high" quality graphics). Under OS X these same scenes would drop to 0.8 fps, which I imagine is actually some kind of bug in the viewer but even when it wasn't happening I'd almost never get above around 20fps, and that only at "medium" quality, i.e. with the atmospheric shaders disabled. Voice works significantly better under Linux as well; people complained that my voice was choppy under OS X but I sound crystal clear under Linux.
I've heard a rumor that the major difference between these two OSes from a 3D standpoint is that Apple wrote the OpenGL drivers for OS X and nVidia wrote the drivers for Linux. Perhaps Apple is trying to sabotage OpenGL? They've certainly sabotaged themselves when it comes to courting me as a user. I can do without speakers, a perfect trackpad, and automatic dual screen switching if it means I get a more responsive OS with better 3D support.
Incidentally, I'm actually dual booting, with rEFIt in the OS X partition loading Grub in the Linux partition when I want to boot Linux, which is always. |
|
|
| Reason has no reason |
[Apr. 12th, 2009|08:33 am] |
While listening to an interview with Temple Grandin on NPR, I was struck by something she said: a lot of people don't think animals have feelings. To her it's obvious that they do. They feel fear, pleasure, happiness, and probably the full gamut of what humans feel.
I suspect that people who say animals don't have feelings (or emotions, whatever they want to call them) have some special definition in their head of what emotions are: they are this thing that supposedly separates humans from animals. But in fact emotions come from the most primitive part of our brains, and we could probably at least survive on emotion alone.
From the perspective of AI research, I think God "cheated" when he made people. We have a cognitive system that persists in deluding itself that it controls our behavior, when in fact it spends most of its energy trying to rationalize what our limbic system makes us do and acquire the things our reptile brains want. Sometimes, on good days, it manages to defer short term gratification in favor of longer term goals. On good days. (Note: I'm not a creationist. I'll leave it as an exercise to the reader to figure out what I mean by "God.")
Humans are just a bundle of emotions coated by a thin veneer of cognition. When we feel something, it isn't communicated directly to our conscious mind as "you feel fear" or "you're in love" or "you feel jealousy." Emotions affect our physiology and our behavior, and if we can figure out consciously that we're feeling something and what it is we're feeling, we're lucky.
So why is it that practically all AI attempts are trying to simulate cognition? This is seen as the "hard" part of the problem, yet human minds aren't cognitive engines. Cognition is this impressive thing we do that we're actually aware of doing, but it's not really what drives us. We're driven by our emotions and our cognition is enslaved to them. If an AI team ever succeeded in building a machine capable of thought, I think it would just sit there without any emotions to drive it, because without emotions, there really isn't any reason for it to do anything. Logic, by itself, does not even give one a reason to keep on breathing. Only the pain of not breathing and the desire to survive keeps us alive.
Our most basic emotions are completely hardwired. When we see pain on the face of another or hear them cry out in fear, we feel the same thing. We feel love for our children and for those who fit a certain set of proportions whom we have sex with (sorry to de-romanticize it for you, but oxytocin is becoming fairly well understood), We hate those who stand in the way of our goals. To build an AI, wouldn't it make the most sense to start with hardwired feelings and then worry about cognition later?
Facial pose estimation and voice stress analysis have advanced to the point where a machine can determine at least the most basic emotions someone is feeling fairly easily. If an AI were made to feel pain when it saw someone in pain, happiness when they were happy, addiction to the presence of people who make it feel happy, sadness at their departure, then it could potentialy be motivated to want to please people. It could feel revulsion at the sight of blood and love for all human-shaped things the way we feel love for babies; the low-level equivalents of Asimov's three laws of robotics. It would have behavior even if it did not have cognition. Then, add even a primitive cognitive system, and you might get interesting results.
I've made several logical leaps here in the interests of keeping the post short. It probably should have been multiple posts, but it was the AI part I wanted to get posted and that would have been at least the second or third post. I'm happy to expand on any point that doesn't make sense.
|
|
|
| Don't use ATI's latest drivers on Liinux (at least on Macbook Pro) |
[Feb. 26th, 2009|10:39 am] |
Proving that the AMD acquisition hasn't improved ATI's abilitiy to write drivers, my attempt to "upgrade" to the drivers from February was a complete disaster. Dual head support became useless because there was a large empty space above my laptop screen where my upper gnome panel, windows, and Nautilus icons would end up (I suspect this is actually "improved" xrandr support screwing me) and the Second Life viewer would just crash instead of running. Downgrading back to the version that comes with Intrepid has fixed all these problems.
I will admit the possibility that I was somehow running the Ubuntu kernel module with the ATI proprietary driver, but I had uninstalled all the Ubuntu fglrx stuff before installing ATI's stuff. The only reason I attempted to upgrade is that the SL viewer was occasionally slowing to 2-3 fps when it normally runs at 15-20 fps. It looks similar to the old texture thrashing problem I had under OSX, but I don't see any evidence of texture thrashing. |
|
|
| Ubuntu Intrepid Ibex on Macbook Pro 2,2 experience |
[Feb. 18th, 2009|01:23 pm] |
Finally having become fed up with Apple's "you will do it our funky way or not at all" approach and their shitty OpenGL support, I decided to set up Ubuntu as a second OS. Unfortunately, neither Boot Camp nor Disk Utility would allow me to resize my OSX volume, even after I turned off the safe sleep file and deleted some other big files as suggested online. Boot Camp said "some files can't be moved" and said I should back up and erase my OSX volume. Disk Utility just said "no space left on device."
So I followed Apple's advice and erased the OSX volume, giving me a whopping 110 gigs of space! That was far more than I would have gotten simply using Disk Utility or Boot Camp. Thanks for the great advice, Apple! Unfortunately, I didn't realize that rEFIt lived in the HFS+ volume, but this turned out not to be a problem.
So, as I mentioned, I'd installed rEFIt as suggested in the Ubuntu docs. I burned a 64 bit Intrepid install CD and booted from it using rEFIt's nice boot menu. I then manually partitioned the drive (as OSX was taking up the whole thing) leaving the EFI partition as the first one (I discovered later I didn't really need to do this, as rEFIt doesn't live there and the machine doesn't need it to boot Linux, but it's there in case I need it for something). I made a 1 gig /dev/sda2 for swap and made /dev/sda3 an ext3 root partition. I then went into "advanced" as suggested in the docs and told it to put the boot loader (grub) on /dev/sda3. This turns out to have been a mistake because that only works if you're using rEFIt, which I'd blown away with the OSX partition.
After completing the install and rebooting the system would pause at a white screen for 20 seconds and then say "Operating system not found." This is because rEFIt had been blown away with the OSX partition and the system didn't know to look on /dev/sda3 for a boot loader. So I used Liz's machine to burn rEFIt to a CD (weird on OSX: you just burn the exact .dmg file that it's distributed in!) and booted from that. The rEFIt CD allowed me to boot from /dev/sda3 and everything started the way it would have had I not blown away the OSX parittion.
Not wanting to use a CD to boot, I just guessed that I could put a boot loader in /dev/sda, so I edited /boot/grub/menu.lst to make /dev/sda the boot device, leaving /dev/sda3 as the root, reran grub-install (or was it update-grub?), and rebooted. Voila! After the system waits around for 20 seconds waiting for God to tell it it doesn't have an EFI boot loader, it finally gives up and loads grub, booting me into Linux like a normal computer.
The funny thing is, even with that 20 second delay, Linux still boots in about half the time of OSX, and the delay between logging in and doing useful work is also shorter. Suspend works "out of the box" after updating everything and changing the power management settings, and unsuspending and unlocking seem far faster than they were under OSX.
The issues I encountered were:
- The mouse pointer would become corrupted. Switching to the proprietary ATI driver (fglrx) using System -> Adminstration -> Hardware Drivers and turning off "effects" in System -> Preferences -> Appearance fixed that.
- It took me a while to figure out how to handle two screens. You do *not* want Xinerama or "dual head" unless you don't want to be able to drag windows between screens. I ended up using "resapplet" to configure the screens. The secret is that you need a virtual desktop area large enough to encompass both screens, which resapplet will configure for you, after which you will need to log out and back in. I probably could have done this manually with xrandr or with grandr as well. At this point, all I have to do to switch between two screens and one is to unplug or plug the second monitor and go into "Configure display settings" in resapplet and hit apply. No logout is necessary just to switch back and forth. I'd like to eventually automate this, but I suspect someone will do it for me if I wait long enough.
- The Second Life viewer seems slower on Linux than it was on OSX, though this seems to be intermittent and possibly related to texture thrashing. It's also possible it may just have been because I had "run multiple threads" enabled under OSX and not under Linux. Since I enabled this and restarted the viewer I haven't noticed it being particularly slow.
- It took me a while to figure out how to get audio to come out the right place. It turns out Linux is *much* better at this than OSX, once you know which applet to use. You want "padevchooser," which allows you to pick the output device on the fly for any stream, and to mute and control volume for each stream separately. On OSX, I had to restart applications in order to change which device they output to.
- The touchpad sucks. I haven't yet figured out how to get it to turn off while I'm typing; touchfreeze complains about SHMConfig being disabled even though it's not. Disabling tapping works for now, but you need to use F12 (versus three-finger-tap) as right click, which sucks as I use my right hand for the touch pad and having to move my hand up to the F12 key is STUPID.
Firefox, which is supposed to be really slow on Linux, is *much* faster than it is under OSX. Every GUI thing imaginable, in fact, seems way zippier. This could well be due to the lack of eye candy on Linux. But I'd far rather have a responsive GUI than eye candy any day of the week. Too bad Apple won't let you turn off the eye candy. |
|
|
| Drobo update |
[Feb. 18th, 2009|01:01 pm] |
Don't rename your drobo if you plan to use any DroboApps on your DroboShare, because some of them hardcode the name of the volume. I had to update the netatalk and rsync configs after installing them because both of them failed in uninformative ways.
The two 1.5TB Seagate drives I bought for it haven't failed me yet, but I have been very careful about shutting down before unplugging power, and I don't think I've ever dropped power to the Drobo or DroboShare without shutting it down first; it's on a UPS. So I can't say this is a recommendation of those drives. Hopefully they will continue to not fail me.
Time machine has been working great using backmyfruitup, though I haven't been backing up long enough to know if it's going to try to use more than the 300 gigs I allocated it when creating the sparsebundle using the provided automator script. Time will not tell until I get Liz using it too, however, because I've blown away OSX in favor of Ubuntu (which I will write about next). Strangely enough, time machine seemed to work fine over CIFS as well, but it probably would not have continued to work.
I'm backing my Linux box up using rdiff-backup to the mounted CIFS volume. This required turning off the listing cache and some other straightforward tweaks, but those are easy to find online (rdiff-backup detects the problem and tells you what it is, and there are plenty of other people who do exactly this with a DroboShare). I will back up my newly Ubuntuized laptop the same way.
|
|
|
| WeRMP3.com are spammers |
[Feb. 17th, 2009|06:01 pm] |
Just got SMS spammed by wermp3.com, and a quick search shows that they've been doing this for a while, along with attempting to blog spam all over the place.
If you do business with them, you will go to the Hell of People Who Do Business With Spammers. If you want cheap music, pirate it instead; it's just as legal and the people making it will get just as much money, but you will have the satisfaction of knowing you at least didn't help fund a bunch of assholes who think it's OK to cost millions of people 2-20 cents a pop (not to mention interrupting whatever they were doing) to receive their "targeted" communications. |
|
|
| Virtual world state as video |
[Jan. 29th, 2009|11:10 am] |
I've been thinking a lot about multiprocess models for virtual world simulators. The Second Life simulator is fairly monolithic, with external processes only for communicating with the databases and for proxying HTTP requests from clients. It even handles instant messages and transactions on behalf of any Resident standing in its region. This means when something goes wrong it frequently results in a crash that logs everyone in the region out.
It would be fairly straightforward to handle physics simulation (dynamics, collision detection, etc) in a separate server process that only performed the most simplistic of communication with the outside world. This simulator would receive commands to update its state (transactionally, within a single frame, so that the simulation never needs to stop) and send out messages about changes to that state due to the simulation itself and due to other clients' update commands, and events such as collisions. With an external client's being responsible for keeping track of and persisting the simulator's state, a crash need not do anything other than cause the simulation to freeze while a new simulator comes up and is sent the last state.
Trying to send the entire state of the simulation to clients every frame would consume a lot of bandwidth. It would be fairly straightforward to send only changes to the state, especially if clients (of the simulator itself, *not* virtual world clients) were expected to receive every update. However, if only updates are sent, bugs or network losses could easily result in the client's state getting out of sync with the simulation's.
Sending out state updates for a large number of elements is very similar to video: with video, you have, say, 640x480=307200 pixels with R, G, and B values that have to be updated in every frame. "Raw" video consumes a lot of bandwidth, so video codecs use tricks like only sending pixels that have changed since the last frame.
Anyone who has streamed video over the Internet has seen the artifacts that result from exactly the problem I describe of the state's getting out of sync. These usually clear up pretty quickly due to the fact that inserted in the video stream are periodic "key frames" (an unfortunate overloading of a term from animation), also known as "intra frames" or "I-frames" that contain every pixel's current state.
This solution easily carries over to our simulation problem: send only the state that changes each frame, but send the *entire* state every second, for example. If the entire state can't be sent in a single frame due to bandwidth limitations, pieces of unchanged state can simply be interleaved with the parts of the state that have changed.
For example, if the simulation is running at 45 fps and we want the entire state to be received within a second, we can simply iterate through all the state elements sending each element that has changed since the last frame *and* each element that has not been sent within the last 45 frames. For each unchanged element that gets sent, we update a counter and stop sending unchanged elements when we reach a dynamically computed threshold intended to keep the data sent per frame as constant as possible.
The reason we use a dynamic threshold rather than sending exactly the right amount of data is that ideally we'd like to make only one pass through the simulator state per frame, and we always want to send changed state to keep latency as low as possible. Such a threshold should be fairly easy to compute because we know the fraction of the state that changed in the last frame, and the percentage of state that changes in *this* frame should be about the same due to Newton's first law.
The nice thing about sending out state this way is that only one data stream needs to be computed and it can be sent to all clients that are able to handle the amount of bandwidth required. Like with video streaming, one of the clients could be a "streaming server" that "transcodes" the state updates for other clients (such as viewers) that can't handle the full bandwidth. This could involve buffering the updates and only sending the most recent if the client isn't accepting all of them, prioritizing changed state over static state, filtering updates and only sending updates for objects that are within the viewer's view frustum, etc.
Since this approach is based purely on a stream of messages, the streams to and from the simulator could be sent through a standard message queuing system such as RabbitMQ. Given that RabbitMQ has been used for streaming video, this should be well within its capabilities. We'd probably want to use only one message per frame though (as the streaming video app does) rather than one message per state element. |
|
|
| Finding God |
[Jan. 27th, 2009|01:16 pm] |
A friend just asked me what God had to do with my last post. Liz's post on the topic does a better job of explaining it than mine. But here's my explanation.
God is the fifteen billion years of evolution that led to the genes that produce the deep structures in our brains that make us who we are without our having any conscious awareness of how this is so. God is the voice that speaks to you and tells you the right answer and gives you the certainty that the answer is correct. God is what makes you flail around and repeatedly revisit an issue when you haven't worked through it yet.
And God is the love I feel for Liz that let me finally decide to stop keeping options open for myself, and the one who gave me the knowledge that this was exactly the right thing to do.
I still consider myself an atheist. I just happen to be a particularly spiritual atheist these days.
|
|
|
| I found God. He lives in my shower. |
[Jan. 27th, 2009|11:13 am] |
As any regular reader of my LJ knows, Liz and I have been practicing polyamory since we got together. There are a lot of details involved in the story I'm going to tell here, but instead of bore you with them I'm just going to cut to the meat of the matter.
I recently was in the process of starting two new relationships with girls I really liked. At the same time, Liz was realizing that she didn't need or want to be poly any more. The timing wasn't entirely coincidental but I seriously doubt it could have been much better. Liz realized how much strength a friend of ours gains from his devotion to his wife, and her devotion to me gave her the strength to avoid doing something that would have harmed our relationship.
On Friday night Liz started hinting that she might want to become monogamous. Saturday morning she decided that she really did. My answer was, "Then let's do it!"
If only I'd been as certain as I'd felt at that moment.
I broke things off with the two other people I was seeing on Sunday morning, then completely broke down. I scared the hell out of Liz because she was probably fairly certain I was going to break up with her. Then I felt better again. I tried to remain friends with one of the girls, but I just kept liking her more and more and regretting that I couldn't be with her.
Then, this morning, Liz said it was OK for me to have a relationship outside of ours. She'd kept saying this to me when I'd express doubts, but never had she been so convincing that she actually believed it was OK as this morning. I felt relieved and excited. She said that she thought it was probably a mistake but that it was a mistake *I* had to make. That didn't change my mind.
We got into the shower, but then Liz decided she couldn't shower with me and got out. I stayed in the shower because I needed to get ready for work. Then it hit me: I didn't have to fall in love with this girl. All I had to do was to break off contact with her. And Liz had done exactly the right thing by saying I could have a relationship with her: she'd made it OK for me to make that decision, because I wasn't making it for *her*. I was making it for me. It was if God had spoken to me in that moment. Tears of joy filled my eyes because I knew it was exactly the right decision and I realized that Liz had done exactly the right thing to allow me to make that decision.
I got out of the shower and found Liz in bed. I told her I had found God and that He lives in our shower. Then I went out to the living room and sent the email I needed to send. She came out and I showed it to her while I went to shave. Liz said that God had also spoken to her and that she'd become completly OK with my having a relationship at the same time as I was realizing I didn't have to fall in love with someone else.
We passed!
Update: Liz has written about this from her point of view.
|
|
|
| Backups! Yay! |
[Jan. 26th, 2009|02:38 pm] |
My DroboShare finally arrived, about a week after the Drobo so I've been chomping at the bit. Along with two Seagate 1.5TB drives (which freebsdgirl tells me I may regret), I now have my media files (bittorrented TV shows, ripped DVDs, music, porn, etc) on redundant storage, and I have Time Machine working over the network into a (hopefully) 300G sparse bundle, which I hope means it won't fill the thing up. Next step is to set up regular backups of the disk that will remain in my desktop/MythTV box, after which I will move the two other 500G drives from that box to the drobo for a total of 2.5GB of storage (500G * (4-1) + 1T * (2-1)) and get a smaller case for the MythTV box so I can fit it on or in my new media stand.
|
|
|
| Finally figured out what those random fish spams were |
[Jan. 26th, 2009|02:05 pm] |
Project Upstream is apparently scraping AIM and Yahoo screen names from LJ profiles and sucking them into their service. Getting a random "hi" contact and then suddenly being connected to someone who has no idea what's going on has resulted in my blocking them on each occasion. Now that I know what it is, I sort of regret that. Of course, once I submitted my screen name to the service and got a solicited contact, the other person *still* had no idea what was going on and probably blocked the bot.
|
|
|
| Genes don't care about individuals |
[Jan. 26th, 2009|12:24 pm] |
The common idea goes that we are slaves to our genes, and that our genes desire to propagate themselves, so we behave in a way that maximizes our chances of producing offspring and our offspring's surviving to produce their own offspring, ad infinitum. However, this theory fails to predict (though it can explain) altruistic behavior.
My favorite form of altruism is entrepreneurship. Entrepreneurship has negative expected returns to the entrepreneur, yet it continues to exist and thrive. The most successful societies on Earth are the ones with the greatest level of entrepreneurship. Why is this? No rational self-interested individual should ever take part. However, there is a persistent delusion that *I* can do better than everyone else, in spite of the fact that this is in fact a bad bet.
On the other hand, entrepreneurship actually has quite large positive expected returns for the society as a whole, whether it's the entrepreneur's town, state, nation, or the whole planet. Humanity has benefited from Google far more than those who have worked for Google, at least adjusting for risk; the cost to society of Google's failure would have been pretty close to zero.
Genes are just molecules. The universe doesn't "know" the difference between a particular gene's offspring and an identical gene from an unrelated individual. Therefore, evolutionary pressure should be toward behaviors that benefit people in proportion to how many genes they share with you. Obviously your own offspring are likely to share the most genes with you, but the percentage difference over your neighbor is in fact extremely small, particularly when you consider that we share 98% of our genes with *chimpanzees*.
So, in fact, altruistic behavior isn't just unsurprising: it's absolutely inevitable.
Suddenly all these supposedly irrational behaviors are explained: people's inexplicably spending more time becoming politically informed than is justified by the economic benefit to them of voting correctly. Donating to charity. Cheating on your taxes less than you can get away with. Sacrificing one's own life to save others.
More to follow in this vein. |
|
|
| Don't hedge your bets |
[Jan. 23rd, 2009|11:55 am] |
Hedging is an important tool in portfolio management because it allows us to choose the level of risk we can tolerate by giving up a certain amount of return on our investment. But when we use it in relationships, it means we're giving up some upside in order to protect ourselves from the possibility of being hurt, and in some cases it may just make it more likely we're going to end up without a relationship, or we'll end up hurting the other person.
Love is all about risk. We can't open our heart to someone without risking its being broken. So instead of hedging, pick someone who's worth the risk to you, or wait until you can stomach the risk. It's the only thing that's fair to yourself or to them. The worst thing you can have in life is a Plan B. |
|
|
| Choose someone who brings out the best in you |
[Jan. 23rd, 2009|11:16 am] |
I've been in plenty of unhealthy relationships with people I found attractive and who liked me, but who brought out traits in me that I really didn't like very much. And even if they weren't trying to turn me into someone other than who I "am," I started turning into someone else.
I think that it's very important in a relationship that you like who you are when you're with that person. A lot of people claim to always be themselves, but this is bullshit. The self doesn't even exist without interactions with others.
Of course, it's important to have a good idea of who you are if you're going to find a decent relationship. If you don't know who you are or at least who you want to be (are these really separate?), you're likely to end up with someone who just walks all over you, or who represents something you think you want.
As an example, I dated someone who to me represented my "lost" 20s. I started hanging out with people much younger than me, colored my hair, and started wearing different clothes. I thought this was what I wanted, but it wasn't me. On top of that, she really brought out some negative traits in me. I became submissive and irritable and started doubting my own intuition and everything I believed. But, I liked her, and she liked me. The problem was that I stopped liking me. I'm pretty sure I probably made her worse, too.
I am absolutely ecstatic about who I am when I'm with Liz. She perfectly complements me. We are not alike, but the ways in which we're different balance extremely well. She notices things I don't, and vice versa. When she makes an observation about me that conflicts with what I believe, I simply assume she's right, because she almost always is. When she has problems or tell me that something I've done has hurt her, I don't despair, because I know she tells me because she wants our relationship to work, not because she's trying to make me feel bad about it.
You can't improve without being able to admit uncomfortable things about yourself. Having people in your life who make you feel safe enough to be able to bring your darkest demons to light makes improving yourself infinitely easier. I feel incredibly lucky (and indeed spoiled) to have someone who fills that role in my life. |
|
|
| Feelings are hard |
[Jan. 23rd, 2009|11:06 am] |
It's sometimes difficult to tell what you're feeling. This is especially true when the feeling contradicts who you think you are or who you'd like to be.
I had convinced myself that I wasn't a jealous person, so when Liz told me I had been acting jealous the other night, at first I thought that was impossible, because I knew what jealousy felt like and that wasn't it. But whenever Liz tells me something I always assume she's right. It turns out that while I'm perfectly ok with her having sexual relationships with other people, I'm bothered if someone else is able to make her feel flustered.
Perhaps birth control has eliminated anxiety about someone else impregnating my woman, so now I'm only worried about losing her heart, not her uterus. For some reason, I had always thought sex was the hard part when it came to jealousy. But I guess that's just because I'm a guy and the natural instinct is to protect your female "property" from being used to birth other guys' kids. Once you get past that, her feelings really become important, and you really have to believe that she loves you and is not going to leave you no matter how flustered someone else might be able to make her feel. |
|
|
| Propagating virtual world state using REST |
[Jan. 22nd, 2009|12:11 pm] |
After posting about spatial publish/subscribe, I decided that a message paradigm isn't appropriate for propagating state changes. Trying to decide who needs state-change messages is a hard problem, and you end up with two separate protocols for sending initial state and subsequent state changes.
Instead, the client could just GET a URL with a header indicating the last state it had received (say, the frame counter). The server would then send just the state that had changed since then. To implement this, the server would have to store information about each state it sent out for some period of time, but it would not need any per-client state, and if the last state header sent by the client doesn't match any state the server remembers, it can just send a full state for the cell the client is asking about.
The implementation could work like this: for every update interval, or perhaps only when a request comes in, the server generates a compressed bitmap representing which state (color, texture, position, orientation, other prim parameters, per prim) had changed since the last bitmap was generated. When a client makes a request, the server sends the state corresponding to the union of all the bitmaps since the last state the client says it received. If the server no longer has bitmaps back to that state, it sends a full update. This is actually much closer to how Quake 3 does it than my last post, only using HTTP instead of UDP.
This could wind up being too much bandwidth if clients could only request state for the entire simulation, so again the simulation could be divided into cells and only updates for objects overlapping the cells requested by the client would be sent. This would require storing separate state bitmaps for each cell, however, or something more complex. |
|
|
| A spatial publish/subscribe model for virtual worlds |
[Jan. 20th, 2009|02:29 pm] |
One of the most challenging things the Second Life simulator does is to figure out which updates to send to which client. This is referred to internally as the "interest list" - what objects do you want to receive updates for? I haven't looked at the code myself, but from hearsay it appears that a good portion of the simulator's CPU time is spent on processing the interest list.
It seems to me that this sort of problem should be amenable to publish/subscribe. With publish/subscribe the client tells the server which kinds of updates it wants by giving it predicates (tests that a message must pass to be passed along), and the server matches each message to the predicates and puts them in each client's queue based on the results.
For a naive implementation based on expensive-to-test predicates, say, send me events whose positions are within 64m of this point, this would be pretty expensive, because it would require computing at least squared distances for every object for every client, so n*m*3 multiply operations, where n is the number of events (object updates, for example), m is the number of clients, and 3 is the number of dimensions.
But is it really important to never send a message the client doesn't care about? There are m clients and only one server, so it seems like it'd better to allow the client to discard unimportant messages. We may not be able to handle the bandwidth to send all updates to all clients, but there is no reason to go to heroic lengths to send exactly the messages the client wants.
Instead, we can turn to techniques used in 3d games to accelerate collision detection. Most of these techniques start with a "broad phase" that conservatively and quickly determines which objects *could* collide, followed by a narrow phase that determines collisions exactly. They do this by dividing the space hierarchically into an octree, kd tree, or BSP, or by using some kind of regular grid. Objects are then assigned to the parts of the structure occupied by their boundary, whether it's an axis-aligned bounding box, bounding sphere, or whatever. The bounding shape that's used is actually irrelevant as long as it can be quickly determined which parts of the space it occupies and that it completely encloses the object without wasting too much space.
For most applications of a virtual world, a regular grid seems perfectly adequate, along with being simple to understand, calculate (especially for axis-aligned bounding boxes), and specify in subscriptions and published messages: you just use the x, y, and z coordinates of the cell, potentially divided by the size of the cells to save bandwidth. The lack of hierarchy means that no component in the system needs to traverse multiple levels; an event published in a cell is published exactly in that cell and no others. An object can overlap multiple cells, in which case its update messages merely contain the addresses of all those cells.
Let's say we use cells 16m on a side. For the moment, we'll just worry about up to 4096 meters, ignoring the use of single-precision floating point in Second Life geometry. So we have 16x16x256 or 65535 cells. A 128m draw distance will cover somewhere around 2200 of these; I'm estimating by dividing the volume of the sphere by the volume of the cells. There will be more if every cube that intersects the sphere is counted.
This is obviously a lot of subscriptions to manage and would result in very large subscription requests if the client were responsible for computing which cells it wanted. So, instead, we allow both subscription requests and messages to contain spatial predicates against which the server evaluates all the cells instead of all the messages. A message is then sent to any client's queue whose subscription predicate matches a cell that the message's predicate also matches.
This would cause a lot of churn in subscriptions for a fast-moving camera if we tried to update the subscriptions for every frame. But it's actually not necessary to update subscriptions very quickly, since the client is not likely to even be able to apply updates it receives for a newly visible area if it hasn't even received geometry info for the objects yet. Even once per second would probably be enough, and due to the fact that we are testing cells on a regular grid in order and can restrict the predicates to ones that work well with such a grid, we can use shortcuts and not have to evaluate the predicates for every cell.
How do we keep track of which clients are interested in events in which cells? Bitmaps! With 64k cells and 1024 possible clients, that's only 8 megs of storage, even if we choose not to use a compressed bitmap format like word-aligned hybrid codes or hash the cell addresses or something (which would make unsubscribing from a cell more complicated). To figure out which clients a message goes to, just compute the union of the bitmaps of all the cells that pass the message's spatial predicate. That bitmap gets stored with the message in a circular buffer that's large enough to hold the maximum number of messages we ever expect to have queued, and we store a pointer to the next message to send to each client.
When the client polls for messages, check to see if its bit is set in the bitmap for that message. If it is, send the message to the client. Advance the pointer. If the pointer for the front of the circular buffer passes any client's "next message" pointer, kill that client because it's accepting updates too slowly, or do something more clever like randomly skipping messages when it gets close to that point.
You'd probably want to do something a little more sophisticated than this, because you can send fewer updates for objects that are father away from the camera, and large objects should be visible farther away than smaller objects (potentially with a higher update rate because they are more visible). But I'm pretty sure that could be accomplished with a version of this model, possibly extended to "extra" dimensions such as size, where if you animated the shape of the subscription of a client with increasing object size as the time axis, you would see an expanding sphere.
Spatial chat and audio (potentially including voice) would not be implemented using a spherical predicate for the messages because it has a point source whereas an object update should be seen if the object overlaps the cell at all. Instead, the message would go to a set of cells that was one cell wide along the x, y, and z axes, but only occupies cells on the "size" axis that correspond to the radius at which it can be heard.
An important message type for object updates is "leave," indicating that the client will receive no more updates for an object and should stop displaying it. Computing who receives a leave message requires a special kind of predicate: all clients which would receive messages matching predicate A which would not receive messages matching predicate B. In other words, all clients who got the last update but who won't get this one. This is only important when the object moves, not when the camera moves: the client should handle figuring out when objects go out of range and delete them.
I came up with this design while thinking about how to implement publish/subscribe outside of the simulator. A similar design could be used inside the simulator using bitwise logical operations on huge (probably compressed) bitmaps of prims and avatars. For each frame of the simulation you store a compressed bitmap of all the prims and avatars that changed in that frame, then when sending an update to the client compute the intersection of the cells that are visible to the client with the union of all of the dirty bitmaps that the client has not yet "seen." This is similar to the way Quake 3 does it, I believe, except that Quake 3 sends updates for the entire space, not just what's visible to the camera.
|
|
|
| Happy New Year! |
[Jan. 1st, 2009|02:17 pm] |
| [ | mood |
| |  content | ] |
![[info]](http://l-stat.livejournal.com/img/userinfo.gif) ipsafictura , adoor , Tony, and I all went to Bimbo's 365 to see Lee Presson and the Nails. All of us except I guess Tony since he was driving got pretty drunk, but at least ipsafictura and I had a blast. There were lots of attractive people there, though ipsafictura pointed out that some of the guys had a rather liberal interpretation of "cocktail attire," especially the guy in the dress Utilikilt. Don't wear Utilikilts, especially dress Utilikilts. This will be your only warning.
ipsafictura stopped a girl in a purple dress to tell her she was adorable, thus reminding me how lucky I am to be dating a unicorn, and a pretty girl in a black dress made eyes at me, prompting me to wonder this morning why I had not attempted to get her to come back to our room with us. Assuming of course she was into that; I have no idea if she'd been paying attention to ipsafictura .
adoor and Tony went over to Sparky's, and we were supposed to meet them there, at which point we realized the horror that is San Francisco's idiotic taxi policy, that policy being to never have anywhere near enough of them. So instead we just walked back to our hotel, the Argonaut, which I must say is very nautical, not to mention a nice hotel.
On the way back to the hotel I saw a girl in a dress sit on the sidewalk to text as if she were wearing pants, giving the world an excellent view of her black and white polkadot panties. I said "You can't sit like that!" prompting her to smile at me and close off the view.
All in all, I'd say this has been my best New Year's ever. I'm very happy to be ringing in the new year with someone as awesome as ipsafictura .
 |
|
|
| A replacement for social darwinism |
[Dec. 31st, 2008|01:45 pm] |
As I was taking my reheated Chinese food out of the microwave, I noticed the microwave safe container says on the side, "CAUTION: Contents May Be Hot." No, really? My first thought was that anyone who needs to be told that would probably learn more memorably if they were allowed to be burned, but then another thought occurred to me: can't we make individuals smarter rather than waiting for natural selection to kill them so that evolution can get on with its work?
Evolution is a slow, blind process that frequently finds local optima and then stops. Human society is not currently geared, nor is it likely to be geared in the near future, toward producing highly intelligent people. And besides, it's not like we let myopic people stumble around blindly: we give them glasses. Our technology is on the verge of letting us practically ignore genetics, so why not just throw out human evolution right now and start working toward augmentation?
Today, the Internet makes us all smarter. We can write short posts that reference concepts most of the readers may not be familiar with, secure in the knowledge that they can just look it up on Google. Our brains only have to map vague concepts to keywords so we can look them up later; we no longer have to remember details of particular concepts. At the same time, tools for rational thought are improving; huge strides are being made even now in statistics and reasoning with probabilities.
I'm not sure what form the IQ equivalent of glasses will take, but I think they're glasses that every single one of us will wear the way every single one of us uses Google and a PDA phone today. My suspicion is that it will take the form of augmented reality, with a display superimposed on our environment using, coincidentally, glasses, and probably something that looks like a bluetooth headset or a pair of earbuds.
At first, such augmentation will cause lots of problems: until we adapt to it culturally, and we adapt it to us and how we use it, it may cause a certain amount of atrophy of our mental facilities. At the same time, it will enable people who would have long ceased to exercise certain parts of their brains, the way I'd ceased to exercise the math part of my brain until I started reading math papers online. Since brains and cultures are plastic, though, the part of people's brains that used to be devoted to remembering phone numbers and appontments can be repurposed for something else, like coming up with good keywords to search for.
So I think I'm gonna stop complaining about excessive labeling and the loss of deep ends in pools and bad drivers, and instead work on AI secretaries and robot cars.
Update: On further reflection, I think running shoes would be a better analogy than "IQ glasses." Not everyone wears glasses, but even the best runners wear running shoes.
|
|
|
| Is any part of the collapse not related to real estate? |
[Dec. 26th, 2008|10:58 am] |
I was listening to someone on the BBC talking about the current economic crisis last night, and very little mention was made of mortgages, mortgage-backed securities, credit default swaps, or real estate. Here's my question:
Has any major firm collapsed where the cause cannot eventually be traced to the correction of real estate prices?
My understanding of this whole thing is that everything started with a moral hazard in real-estate lending: banks had no incentive at all to vet borrowers when they could just re-sell the loans to Fannie Mae, and people were used to assuming that if they could qualify for a loan then they could afford it, especially since so many people accepted the happy fiction that real estate prices would just go up forever. These mortgages were then packaged and sold to other institutions as mortgage-backed securities, thus insinuating their risk deep into the financial system.
The BBC, on the other hand, seems intent to make this look like just crazy cowboy financiers playing fast and loose with the rules while the regulators looked the other way. It seems to me that absent Fannie Mae and Freddie Mac, none of this could have happened.
Am I right? |
|
|
| navigation |
| [ |
viewing |
| |
most recent entries |
] |
| [ |
go |
| |
earlier |
] |
| |
|
|
