As you set up component levels, the lifespan of a component usually goes up. Alivio components have a lot of plastic, steel and lower grade alloys, which are more prone to warping and wearing. The pivot points on Alivio derailleurs and the cup/cone/balls on Alivio hubs can wear out quickly on heavy usage.
Shimano rapid fire shifters are one of the components that do not tend to wear out that often, regardless of the level. Do XT shifters shift more crisply then Alivio shifters? Yes. Is it enough of a difference to warrent the extra cash if you are a pretty casual rider? Nope.
If you ride fairly often, and if you ride in wet/muddy conditions some of the time, but do not race, Deore components offer excellent bang for the buck and are probably as good as is needed.
If you are a bit more agressive, ride fairly often, race occasionally, live in constantly muddy regions like the pacific northwest, XT is the group for you.
If you are a dedicated XC racer, and are concerned that shaving those few grams off your bike will allow you to cross the finish line a few seconds faster, XTR.
I would skip over LX. Either Deore is good enough for you, or go straight to XT.
Cheers,
Juba