Of Builds and Bugs

Posted by Brett Folkert on Wednesday, October 10, 2007  
Category:

The frenetic build process continues apace, but we've also entered the pre-launch phase of bug fixing.  The process is best summarized by the following steps:

    1. QA tester finds an issue and assigns it to a developer
    2. Developer researches issue and makes code change if necessary
    3. Developer unit tests the change to make sure it works
    4. Developer checks in the code as "ready" for QA testing
    5. The code is promoted into the test environment
    6. QA tests the fix
    7. Repeat 1-6 until code stabilizes and no critical issues remain

 

At last count we had over 300 known bugs to resolve, most non-critical content issues (e.g. misspelling words, wrong button style, wrong image, etc...) but some critical that can delay launch (e.g. unable to register).  Projects with a code base this large have enormous levels of interdependence among all of the various subprojects.  This is in addition to the macro interdependence between the systems themselves.  Let me give you an example from the last few days.

I wrote the code (originally for Simply Nutrilite) that allows a visitor to save their profile during the order checkout process.  After some debugging, I discovered that due to an unrelated issue, someone had changed the way the page maintains its state which caused one of the dropdowns to no longer keep track of what the user selected. I rewrote the functionality to maintain its state on its own, then promoted it to the other environments for QA testing.

Mysteriously, the identical code worked in the dev environment but broke in build and QA. After more research, it turns out that the back end system was having issues in build/QA but was functioning just fine in dev.  After the back end system was stabilized, I discovered that another unrelated change to the checkout process temporarily disabled the confirmation page, which is where the save profile functionality is displayed.  After that was resolved, everything is now up and running again in all three environments.

It has been said that this process is akin to building a plane while it's in flight. I have to agree. In order to benefit from the time savings of parallel development among many developers, you have to be aware of the risks that multiple simultaneous changes to the code introduce. That's why testing is such a high priority and why the iteration described above is a constant process both pre and post launch. In the meantime, I'm off to run down some more issues.

 


Comments

# Chuck Lia said on October 10, 2007 at 11:28 AM:
Brett, The Quixtar personal websites give us the choice of using our full legal name (used when we register) or leaving it blank. For example, my personal Quixtar website says "Charles Lia's", even though no one other than my mother calls me "Charles." Everyone else knows me as "Chuck." Is there any way to change this feature to use our "go by" name or perhaps even our DBA name if we use one? Chuck
# Brance Long said on October 10, 2007 at 12:52 PM:
The oxen is slow, but the earth is patient. Chinese Proverbs we will look forward to this exciting website! good work! Brance
# Brett Folkert said on October 10, 2007 at 2:01 PM:
Chuck, I forwarded your question and I'll post once I get a response. Thanks as always for the feedback.
# George said on October 10, 2007 at 3:35 PM:
Sounds like an exciting change for the face of the website. I am anxious to see it! I do have a couple questions. Is the revamp you are talking about limited to the main Quixtar site or does it encompass the free personal websites too? Are there plans to update the free personal websites to allow the selling of all products vs. just the coreline items? Thanks for your time and keep up the great work! George
# john niemann said on October 11, 2007 at 9:28 PM:
Brett, whatever you IT guys do, please try to make things Apple friendly! When i went to go through Quixtar University, I was told that you don't support Mozilla as a browser. I did go ahead and download Netscape (because I despite IE) and have now been able to get into QU. There more Apple users being added daily to our number, particulary with many new generation X & Y, who use Ipods, Iphones and Imacs all the time. Don't forget us please!
# Canadian IBO 2 said on October 15, 2007 at 8:37 AM:
Brett, I too use Mozilla and have been having problems with Q University, particularly receiving an error message when I attempt to finish each course (the course review where you type notes). I have called Q tech support and tried the various suggestions with no luck (same results). I get two error messages when I try to use IE. Help!
# Brett Folkert said on October 15, 2007 at 3:18 PM:
Chuck, I haven't received a response yet but I do know we store a "preferred" first name if it's provided in addition to first name. We actually display this on the "Hello" login greeting on Quixtar.com. Unfortunately I can't give you any info at this time what it would take to change the personal web sites to use this or if there are plans to do so since I haven't heard anything yet. Sorry I couldn't find out more right now, if I get info later I'll post it.
# Canadian IBO 2 said on October 15, 2007 at 5:15 PM:
Brett, Firefox 2.0.0.7 I tried QU today with the same results. I'll try it again and try to post the error.
# Brett Folkert said on October 16, 2007 at 8:09 AM:
Canadian IBO, Sounds good. Please let me know what page (Url would be great) you're seeing the error and what steps if any are required to reproduce it. I run Firefox as well so I should be able to track it down if I can reproduce it on our end. Thanks!
# Chuck Lia said on October 16, 2007 at 10:51 AM:
Brett, I have stored my preferred name on the Quixtar site and, as you stated, it does come up when I sign on. Perhaps I can change my IBO first name with Quixtar directly to make it work the way I would like on my personal site. Chuck
# Chuck Lia said on October 16, 2007 at 10:57 AM:
Brett, I just did a name change with Customer Service reseting it as a "misspell" and we'll know in 24 hours if that worked. If not, I can send a name change form to Quixtar and that will solve my challenge. Thanks for your help! Chuck
# Canadian IBO 2 said on October 16, 2007 at 1:23 PM:
Brett, These are the steps to the error: login into QU; Know The Products; Welcome back;the window with the box and the question "what are the most important things you can do to know the brands and products?" this is where another window pops up with the error message "document.getElementByld(" that window never fully loads. The cursor has an hour glass and it does not completely load. And every time that I try to type a letter in either window, the error pops up again.
# Canadian IBO 2 said on October 16, 2007 at 6:42 PM:
thanks Brett. I appreciate all that you have done. Any suggestions for the time-being while that issue with Firefox is sorted out? I received even more error messages when I used IE.
# Brett Folkert said on October 17, 2007 at 10:18 AM:
Canadian IBO, You shouldn't be receiving any errors with IE, it's fully supported. Can you give me some details on those errors as well? I will forward them as well. Thanks again for your patience.
# Canadian IBO 2 said on October 21, 2007 at 11:03 AM:
Brett, I tried to run QU with IE. QU; Knowing the Products; got the same error in two windows: 'document.getElementByld(...)' is null or not an object Help!
# Brett Folkert said on October 25, 2007 at 8:13 AM:
Canadian IBO, I've been told we're experiencing performance problems with part of the software that underlies QU and that might be part of the issue. Regardless I will forward on the fact that you're experiencing the same error. I'm going to give it a shot myself in IE7 and see if it happens to me too. Thanks again.
# Brett Folkert said on October 25, 2007 at 8:22 AM:
Canadian IBO, I just tested using IE7 and I was able to get past that box where you have to enter the answer to a question under Basic Business Skills --> Know The Product. I still have issues in Firefox, though. Can you verify that you're using IE7 with popup blocking turned off? I did have issues w/ popups at first until I enabled them for both www.quixtaruniversity.com and lms.quixtaruniversity.com. Thanks.
# Canadian IBO 2 said on October 26, 2007 at 8:30 AM:
Brett, I am using IE7, disabled popup blockers for those sites,etc, etc. Same problem, same errors. I have no idea what to do. I'd like to get credit for completing the two courses and move on to the next course.
# Nick Kobelja said on October 26, 2007 at 11:58 AM:
Brett, Firefox is about as standard a browser as they come. I'm pleading with you to do whatever it takes to coerce the 3rd party company that QU uses to make it Firefox compliant. With IBOs all over the word, it is unreasonable to force a certain OS/browser on everybody. Thanks.
# Tracy said on November 1, 2007 at 3:04 PM:
Brett, The screen shots of 3.0 look great! A little worried, as they are billed as visitor screen shots, and I noted that IBO pricing is shown. Hope they just grabbed the wrong screen shot! We are all looking with great anticipation! Hope we can finally search for a product within Ditto to add it!

Leave a Comment

(required) 
(optional)
(required - not displayed)  
 
(required) 
Submit

About Brett Folkert

Occuption
I am a Software Architect at Quixtar. The nature of my job and our environment means I have to be fluent in many disparate technologies on a daily basis. I like to think of it as "controlled chaos". I constantly strive to improve our architecture and ensure that wherever possible the underlying technology is transparent to our users. You probably won't notice my handiwork unless you look at the system design, but behind the scenes a lot of thought and hard work goes into making the site perform to our standards.

Background
My IT experience prior to Quixtar was primarily in consulting. I have worked with the U.S. Navy, dotcom startups and Fortune 500 companies. I consulted for Amway in years past (technical lead on GBISLink) before joining the team here at Quixtar. I hold both Master's and Bachelor of Science degrees in Computer Science. My thesis on neural networking/satellite imagery was published in the International Journal of Remote Sensing.

Interests
I have passions for wine, mountain biking and ancient history. If you want to discuss Cicero over a fine Bordeaux, look no further. I'm also married with two precocious daughters whom I adore. As if work and family weren't enough to keep me busy, I'm also attending law school in the evening.