Wednesday, March 4, 2009

A bug report for the app store review process

Summary:

Horrible subjectivity, inconsistency and unpredictability of app store review process.

Steps to Reproduce:

-(BOOL) willAppleDoTheRightThingForDevelopers{
App *myapp = [App appWithMuchHardWorkAndLateNights];
while ([[Users of:myapp] count]){
do {
[myapp submitForReview];
sleep(rand()*1000+FIVE_DAYS);
if ([Reviewer feelslikeapproving:myapp] && ![Reviewer insteadOfReviewingUpdatesChoosesToRejectForSomethingPrevious2ReviewersMissed] ) return YES;

[I WTF];
[I cantFindReferencedAttachment];
[I respondToRejection];
[I hearNothingBack];
[I didntInfringeTrademark];
[I haveNoRecourse];
[I curseApple];
[I prayAndroidSucceedsImmensely];
[I franticallyMakeChangesAndResubmitSoMyUsersWillStillRespectMe];


}
} while ([I amSane]);
}
return NO;
}


Expected Results:

YES

Actual Results:

?

Regression:


Notes:

While the tone of this is somewhat sardonic, the realities of the frustration are 100% accurate.

Suggested improvements:

Some sort of estimated time til review (there has to be an internal queue for reviews why can't we see something giving us a rough estimate that becomes more accurate as it gets closer to review?)

A more thorough review upon first submission, so that future updates can focus on what's new, not digging up old skeletons.

If old skeletons must be brought into the matter, why not also review the actual update so fixing the first won't result in a second rejection when the new features are actually reviewed?

Since it's expected that a review can't possibly be 100% thorough for all apps, give developers a "warning" for something that's already been accepted but shouldn't have been, by saying "fix this by the next submission (if the issue is NOT serious... which in most cases it wouldn't be or it would have been found in the first n reviews!

Some means of quick turnaround with questions on clarifications from the review. I honestly got a review that said "see attached image" with no attachment! I responded asking for the attachment and still haven't heard back. In fact, I have never heard back from anyone about any requests for clarification I've made. Going on weeks now!

Provide a more concrete set of review guidelines. It was very frustrating to have an app rejected for using the word "demo" on a button, when nowhere do the guidelines state this, and the only place I could find information on it was on ars technica!

While I'm going, how about a way to respond to reviews, or more encouragement for people to contact the developers before leaving a damning review.

That's all for now, if there's a better venue for this, please let me (and all the other developers) know!

1 comment:

onionpalac said...

Sounds like a mess.
Glad I'm not a developer.
I hope everything goes good with the upcoming update.
Will that be delayed?
Either way, I have respect for developers like yourself - in these cases it is the App Stores problem. They quickly became famous for everything that was mentioned in this port.