How we rank products
Every ranking on OrangeToffee uses the same published method. Brands cannot pay for placement, a better score or a quadrant position.
1. We collect ratings from several stores
For each product we record its average star rating and number of ratings on Amazon.in, Flipkart and Nykaa where it's sold, along with the date we collected them. We combine them into one rating, weighting each store by how many people rated the product there. Ratings on a brand's own website are moderated by the brand, so we treat them with caution.
2. We score with a Bayesian average
A product with 12 ratings and 4.9★ hasn't proven as much as one with 20,000 ratings and 4.4★. So we blend each product's rating with a cautious benchmark, weighted by how many ratings it has:
score = (v ÷ (v + m)) × R + (m ÷ (v + m)) × P
- R: the product's average rating
- v: the product's number of ratings
- P: a cautious benchmark, the rating that only the lowest-rated 10% of products in the category fall below
- m: the median number of ratings in the category
Products with many ratings are scored mostly on their own rating. Products with few ratings are pulled toward the cautious benchmark until more buyers have weighed in, so a product has to prove itself to climb. (We use a cautious benchmark rather than the category average so that a low-rated product with few ratings can't outrank better-rated products that thousands of buyers have tried.) We show the score out of 10.
3. We place each product in a quadrant
The quadrant compares quality (rating vs the category average) with proof (review count vs the category median):
- Golden Toffee (the winners): Highly rated and proven by a large number of buyers. The products to beat.
- Fresh Zest (rising stars): Loved by early buyers, still building proof. Likely to rise.
- Sugar Rush (popular, not loved): Everyone is buying it, but ratings sit below the category average.
- Still Simmering (not ready yet): Below-average rating and limited review history. Not ready yet.
The product with the highest score overall is crowned Top Toffee 👑. Every edition, products can move up the ladder: Still Simmering → Fresh Zest → Golden Toffee.
4. We read the reviews
Stars don't tell you why people liked or disliked something. We read the most helpful reviews on each store and summarise, in our own words, what buyers like, what they don't, and who each product is best and not best for. We quote at most a short line or two from a review, and never publish reviewers' names.
5. A person checks every page
An editor reviews each ranking before it is published and whenever it is updated.
6. Editions: valid for six months
Each ranking is published as an edition with a report date and a valid-until date, six months later. Rankings don't change during an edition, so a brand's position means the same thing for the whole period. When an edition expires, we collect fresh ratings and publish a new one.
7. Your reviews count too
Every ranking page has a form to review the products yourself. We read every review before publishing it, we publish negative reviews as well as positive ones, and we never accept paid or gifted reviews. Reviews from OrangeToffee readers feed the next edition and the Customers' Choice award.