1) Going to Gila is never a waste of time. It's a great event, the town of Silver city loves having you there, and you'll enjoy it. Disclaimer - I'm a co-sponsor and an announcer, as well as having raced there.
2) Your results at 4000 - 4500' are, unfortunately, not really a great indicator. Most information that I've seen indicates little to no impact for most athletes up until 5000' (well, placebo excluded). After that, everyone seems to have their individual "break point", from where power drops linearly as you continue to go up. I've posted that wattage FAQ chart before.
3) Per the current literature & expertise I'm familiar with, you either get there 2-4 weeks in advance, or right before. It takes your body more than a week to really make any of the adaptations, but within the time you're making most of them your body is working overtime...that's likely why your friends felt like [censored]. The good news is that the manager of the Holiday Inn Express will give you a monthly or weekly rate on a room if you want to head down 2-4 weeks ahead of time to get acclimated. Good restaurants in town too.
So my advice is to go, be really good about sleep, hydration, and refueling after every stage. The majority of the race is below 7000', with just a couple exceptions...so it's not a "pure" high altitude race where only the Moningers and Overends of the world do well. In 2005 Stevic and Mesa both made the final podium in the men's pro race, despite living close to sea level.
humankinetics.comshowproduct.cfmOpen ↗
bikereg.comregister.aspOpen ↗