Quantcast StorefrontBacktalk - Full Text Of Amazon Patent To Reward Review Writers
E-Mail Us
Full Text Of Amazon Patent To Reward Review Writers
Written by StorefrontBacktalk.com Full Text Service
October 6, 2008
( 1 of 1 )
United States Patent 7,428,496
Keller , et al. September 23, 2008
Creating an incentive to author useful item reviews

Abstract

A facility for rewarding the provision of useful item reviews is described. The facility receives a plurality of item reviews, each from a source. The facility publishes each of the plurality of received item reviews, and assesses the usefulness of the published item reviews. Based upon this assessment of usefulness, the facility selects one or more of the published item reviews, and provides rewards to the sources of these selected item reviews.
Inventors: Keller; Thomas L. (Renton, WA), Phillips; Jon (Seattle, WA), Pottier; Nicolas (Seattle, WA)
Assignee: Amazon.com, Inc. (Seattle, WA)
Appl. No.: 11/342,173
Filed: January 27, 2006
Related U.S. Patent Documents

Application Number Filing Date Patent Number Issue Date
09842265 Apr., 2001

Current U.S. Class: 705/10
Current International Class: G06F 17/30 (20060101)
Field of Search: 705/10,14,26
References Cited [Referenced By]
U.S. Patent Documents

6185558 February 2001 Bowman et al.
6260064 July 2001 Kurzrok
6275811 August 2001 Ginn
6513033 January 2003 Trauring
6895385 May 2005 Zacharia et al.
2002/0165905 November 2002 Wilson
2004/0210550 October 2004 Williams et al.
Foreign Patent Documents

WO 03/034637 Apr., 2003 WO

Other References

"eBay Launches the Most Comprehensive Trust and Safety Upgrades to the World's Largest Person-to-Person Trading Site." PR Newswire, Jan. 15, 1999. cited by examiner .
Lenatti, Chuck. "Auction Mania." Upside, vol. 11, No. 7, pp. 84-92, Jul. 1999. cited by examiner .
eBay's Feedback Forum web site, archived on Oct. 12, 1999 by web.archive.org [URL: http://web.archive.org/web/19991012065814/pages.ebay.com/...]. cited by examiner .
Austin et al. "Positive and Negative Effects of Political Disaffection on the Less Experienced Voter." Journal of Broadcasting & Electronic Media, vol. 39, No. 2, pp. 215-235, Spring 1995. cited by examiner .
"Home, Netscape, Yahoo! Veterans Announce Epinions.com," PR Newswire, p. 8921, Jul. 12, 1999. cited by other .
"Epinions.com Announces the Launch of Its Free Online Shopping Guide Powered by Consumer Opinions," PR Newswire, p. 2829, Sep. 8, 1999. cited by other .
Tedeschi, Bob, "Consumer Products are Being Reviewed on More Web Sites, Some Featuring Comments from Anyone with an Opinion," New York Times, Section C, p. 16, col. 1, Oct. 25, 1999. cited by other .
Barrett, Alexandra, "What's Your Epinion?" Network World, Sep. 13, 1999. cited by other .
Wohl, Amy D., "User Review—Your Opinions Are Highly Valued on the Web," VarBusiness, No. 1528, p. 69, Nov. 8, 1999. cited by other.

Primary Examiner: Diaz; Susanna M
Attorney, Agent or Firm: Knobbem Martens, Olson & Bear LLP
Parent Case Text


CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No. 09/842,265 entitled "CREATING AN INCENTIVE TO AUTHOR USEFUL ITEM REVIEWS," filed Apr. 24, 2001, now abandoned which application is incorporated by reference herein in its entirety.
Claims


We claim:

1. A method in a computer system for evaluating an author of item reviews, comprising: displaying one or more item reviews submitted by the author to a plurality of consumers; soliciting from the consumers to whom an item review is displayed an indication of whether the review is helpful; receiving and storing any solicited indication provided by the consumers to whom item reviews are displayed, each indication being either positive or negative; and applying a formula to obtain a score for the author based upon the total number of stored positive indications and the total number of stored negative indications received for the reviews submitted by the author, wherein the total number of stored positive indications is weighted more heavily in the obtained score than the total number of stored negative indications.

2. The method of claim 1 wherein the applied formula further incorporates whether each review submitted by the author is the first review submitted for the identified item.

3. The method of claim 1 wherein the applied formula k(2(A+B+C)-(D+E)), where k is a constant, and A is the number of reviews submitted by the author for which at least 3 positive indications were received, and B is the number of reviews submitted by the author for which at least 10 positive indications were received, and C is the number of reviews submitted by the author that were the first reviews submitted for the corresponding item and for which at least 3 positive indications were received, and D is the number of reviews submitted by the author for which at least 3 more negative indications than positive indications were received, and E is the number of reviews submitted by the author for which at least 10 more negative indications than positive indications were received.

4. The method of claim 1 wherein the method is performed reiteratively to maintain an updated score for the author.

5. The method of claim 4 wherein one or more additional item reviews submitted by the author are displayed in subsequent iterations that are not displayed in the first iteration.

6. The method of claim 4 wherein the method is performed for multiple authors to obtain a score for each of the multiple authors.

7. The method of claim 1 wherein the formula ignores indications received from a given consumer for any review received subsequently to the first indication received from that consumer for the review.

8. The method of claim 1 wherein the formula ignores indications received from a given consumer for reviews of the author received subsequently to a maximum number of indications received from that consumer for reviews of the author.

9. The method of claim 8 wherein the maximum number is 5.

10. A computer system for evaluating an author of item reviews, comprising: a display subsystem that causes one or more item reviews submitted by the author to be displayed to a plurality of consumers; an input solicitation subsystem that solicits from the consumers to whom an item review has displayed an indication of whether the review is helpful; a storage subsystem that stores any solicited indication provided by the consumers to whom item reviews are displayed, each indication being either positive or negative; and a scoring subsystem that applies a formula to obtain a score for the author based upon the total number of positive indications and the total number of negative indications stored by the storage subsystem for the reviews submitted by the author, wherein the total number of positive indications is weighted more heavily in the obtained score than the total number of negative indications.

11. The computer system of claim 10 wherein the formula applied by the scoring subsystem further incorporates whether each review submitted by the author is the first review submitted for the identified item.

12. The computer system of claim 10 wherein the formula applied by the scoring subsystem is k(2(A+B+C)-(D+E)), where k is a constant, and A is the number of reviews submitted by the author for which at least 3 positive indications were received, and B is the number of reviews submitted by the author for which at least 10 positive indications were received, and C is the number of reviews submitted by the author that were the first reviews submitted for the corresponding item and for which at least 3 positive indications were received, and D is the number of reviews submitted by the author for which at least 3 more negative indications than positive indications were received, and E is the number of reviews submitted by the author for which at least 10 more negative indications than positive indications were received.

13. The computer system of claim 10 wherein the scoring subsystem applies the formula reiteratively to maintain an updated score for the author.

14. The computer system of claim 13 wherein the display subsystem causes one or more additional item reviews submitted by the author to be displayed in subsequent iterations that are not displayed in the first iteration.

15. The computer system of claim 13 wherein the scoring subsystem applies the formula for multiple authors to obtain a score for each of the multiple authors.

16. The computer system of claim 10 wherein the formula applied by the scoring subsystem ignores indications received from a given consumer for any review received subsequently to the first indication received from that consumer for the review.

17. The computer system of claim 10 wherein the formula applied by the scoring subsystem ignores indications received from a given consumer for reviews of the author received subsequently to a maximum number of indications received from that consumer for reviews of the author.

18. The computer system of claim 17 wherein the maximum number is 5.

19. A method in a computer system for evaluating an author of item reviews, comprising: displaying one or more item reviews submitted by the author to a plurality of consumers; soliciting from the consumers to whom an item review is displayed an indication of whether the review is helpful; receiving and storing any solicited indication provided by the consumers to whom item reviews are displayed, each indication being either positive or negative; and applying a formula to obtain a score for the author based upon stored indications that incorporates, for each review submitted by the author, (a) the number of consumers to whom the review was displayed that subsequently ordered the item identified by the review, and (b) the number of positive and negative indications received for the review, wherein the number of positive indications is weighted differently in the obtained score than the number of negative indications.

20. A computer-readable medium whose contents cause a computing system to perform a method for evaluating an author of item reviews, the method comprising: displaying one or more item reviews submitted by the author to a plurality of consumers; soliciting from the consumers to whom an item review is displayed an indication of whether the review is helpful; receiving and storing any solicited indication provided by the consumers to whom item reviews are displayed, each indication being either positive or negative; and applying a formula to obtain a score for the author based upon the total number of stored positive indications and the total number of stored negative indications received for the reviews submitted by the author, wherein the total number of stored positive indications is weighted more heavily in the obtained score than the total number of stored negative indications.

21. A computer-readable medium whose contents cause a computing system to perform a method for evaluating an author of item reviews, the method comprising: displaying one or more item reviews submitted by the author to a plurality of consumers; soliciting from the consumers to whom an item review is displayed an indication of whether the review is helpful; receiving and storing any solicited indication provided by the consumers to whom item reviews are displayed, each indication being either positive or negative; and applying a formula to obtain a score for the author based upon stored indications that incorporates, for each review submitted by the author, (a) the number of consumers to whom the review was displayed that subsequently ordered the item identified by the review, and (b) the number of positive and negative indications received for the review, wherein the number of positive indications is weighted differently in the obtained score than the number of negative indications.
Description


TECHNICAL FIELD

The present invention is directed to the fields of electronic commerce and on-line communities.

BACKGROUND

Web merchants sell items such as products, services, and data via the World Wide Web ("the Web"). Because most items sold by a web merchant result in a profit, measured by the amount by which the item's price exceeds its cost, web merchants have a strong incentive to increase the rate at which they sell items.

It is common for web merchants to design their web sites to include content that helps to draw interest to the web sites and to particular items in order to increase the number of customers that may buy these items. As one example of such content, some web merchants include item reviews on their web sites, which typically provide additional information about an item and, in the case of a positive review, an endorsement of the item. For example, some web merchants furnish book reviews on their web sites.

In some cases, web merchants pay professional reviewers to prepare item reviews. In these cases, the often-significant cost of procuring professional reviews can have a negative impact on the web merchant's profitability. Additionally, some readers of professional reviews may be distrustful of the evaluation of a professional reviewer.

In other cases, volunteers, such as customers, are solicited to prepare item reviews. While volunteer reviews can be procured much less expensively than professional reviews and may have more appeal to readers that prefer reviews from those they perceive to more substantially share their perspective, volunteer review programs often have significant disadvantages of their own.

For example, it can often be difficult to convince volunteers to prepare item reviews. Of those volunteer reviews that are prepared and submitted, a significant portion may be of little use to prospective purchasers for a variety of reasons: such reviews may be poorly written, fail to adequately support their conclusions, contain subject matter irrelevant to the item, engage in ad hominem attacks on the creator of the reviewed item, etc.

In view of the above-discussed disadvantages of conventional types of reviews, a more effective approach to obtaining useful item reviews from volunteers would have significant utility.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level block diagram showing a typical environment in which the facility operates.

FIG. 2 is a display diagram showing a typical display presented by the facility showing a list of top-ranked reviewers.

FIG. 3 is a display diagram showing a typical display presented by the facility containing more detailed information about a particular top-ranked reviewer.

FIG. 4 is a display diagram showing a typical display presented by the facility containing detailed information about an item.

FIG. 5 is a display diagram showing a typical display presented by the facility incorporating information about two randomly-selected top-ranked reviewers.

FIG. 6 is a display diagram showing a typical display presented by the facility to enable a user to submit a new review.

FIG. 7 is a display diagram showing a typical display presented by the facility containing guidelines for preparing an item review.

FIG. 8 is a display diagram showing a typical display presented by the facilitating containing sample reviews.

FIG. 9 is a flow diagram showing steps typically performed by the facility in order to process a new item review submitted by a reviewer.

FIG. 10 is a flow diagram showing steps typically performed by the facility in order to process votes received for published item reviews.

FIG. 11 is a flow diagram showing steps typically performed by the facility to update reviewer scores and rankings.

DETAILED DESCRIPTION

A software facility for creating an incentive to author useful item reviews ("the facility") is described. In some embodiments, the facility is used by a web merchant to obtain content that, when incorporated in the web merchant's web site, helps to draw interest to the web site and to increase the number of customers that may buy items from the web merchant. The facility provides such an incentive by rewarding the authors of reviews found to be useful by their readers, such as by prominently displaying their names and ranks as authors of useful reviews prominently on the web merchant's web site.

In some cases, review authors (also called "reviewers") submit reviews each relating to a specific item sold by the web merchant. The web merchant displays these reviews to customers in conjunction with the items to which they relate, together with a control—such as buttons—that may be used by viewing customers to vote on how useful the review is to them. For example, in some embodiments, the control enables viewing customers to specify either that they found the review useful or that they did not find the review useful. The facility stores these votes, and periodically uses them to score each reviewer in terms of the number of reviews that the reviewer submitted and their level of usefulness.

Each time new scores are generated, the facility ranks the reviewers in descending order of their scores. These rank values are then typically displayed liberally in conjunction with information about the top-ranked reviewers. For example, the facility may display a list of the top-ranked reviewers, in which the rank of each reviewer is indicated. The facility may also display a shorter list of reviewers randomly selected from among the top-ranked reviewers, such as from the top 100 reviewers. Such displays may include such information as the names of these reviewers, their pictures, and biographical sketches. The facility may display in conjunction with any occurrence of a top-ranked reviewer's name an indication of that reviewer's rank. For example, occurrences of the name of the reviewer having rank 1 may be accompanied by a "#1 Reviewer" message, which the name of the reviewer having rank 89 may be accompanied by a "Top 100 Reviewer" message. Such displays may occur on pages such as a home page for the reviewer, or in conjunction with the reviewer's reviews.

By promoting reviewers that submit reviews that prove to be useful to their readers, the facility motivates many of the users visiting the web merchant's web site to prepare and submit useful item reviews. This in turn typically increases the level of traffic to the web merchant's web site (in terms of number of visitors, average frequency with which a single user visits, and average duration of a visit), the number of sales completed and therefore the profitability of the web merchant, and the level of satisfaction of item purchasers, both with respect to their purchases specifically and with respect to the web merchant generally.

FIG. 1 is a high-level block diagram showing a typical environment in which the facility operates. The block diagram shows several client computer systems, such as client computer systems 110, 120, and 130. Each of the client computer systems has a web client computer program for browsing the World Wide Web, such as web clients 111, 121, and 131. The client computer systems are connected via the Internet 140 to a server computer system 150 hosting the facility. Those skilled in the art will recognize that client computer systems could be connected to the server computer system by networks other than the Internet, however.

The server computer system 150 contains a memory 160. The memory 160 preferably contains web merchant software 161 incorporating both the facility 162 and data typically used by facility, such as item reviews 163, votes 164 on item reviews, and item order and/or sales data 165. The memory preferably further contains a web server computer program 166 for delivering web pages in response to requests from web clients. While items 161-166 are preferably stored in memory while being used, those skilled in the art will appreciate that these items, or portions of them, maybe be transferred between memory and a persistent storage device 172 for purposes of memory management and data integrity. The server computer system further contains one or more central processing units (CPU) 171 for executing programs, such as programs 161, 162, and 166, and a computer-readable medium drive 173 for reading information or installing programs such as the facility from computer-readable media, such as a floppy disk, a CD-ROM, or a DVD.

While various embodiments are described in terms in the environment described above, those skilled in the art will appreciate that the facility may be implemented in a variety of other environments including a single, monolithic computer system, as well as various other combinations of computer systems or similar devices connected in various ways.

FIGS. 2-5 show examples of ways in which the facility promotes, and thereby rewards, top-ranked reviewers. FIG. 2 is a display diagram showing a typical display presented by the facility showing a list of top-ranked reviewers. The display 200 is displayed as part of the web site of web merchant Amazon.com. The display includes laudatory language 201 congratulating the top-ranked reviewers. The display also includes an ordered list 210 of the top-ranked reviewers. List 201 is comprised of ordered entries, each corresponding to one top-ranked reviewer, such as entries 220, 230, and 240. As an example, entry 220 contains information about the highest-ranked reviewer, Harriet Klausner. This entry contains the reviewer's rank 221, as well as a graphical characterization 222 of the rank. The entry further contains the name of the reviewer 223, which is a link that the user can select in order to view more detailed information about this reviewer. The entry further contains an indication 224 of the total number of reviews authored by this reviewer. The entry also contains further information 225 about the reviewer, typically provided by the reviewer him or herself. This information includes a link 226 that may be selected by the user to display additional information about the reviewer. Some entries contain an image of the reviewer, such as image 237 shown in entry 230.

In addition to the list 210 of detailed entries about the top-ranked reviewers, the display also contains a more abbreviated list 250 of the top-ranked reviewers. In this list, each entry is merely the rank value and the name of the reviewer, which is a link that may be selected by the user to display additional information about the reviewer.

FIG. 3 is a display diagram showing a typical display presented by the facility containing more detailed information about a particular top-ranked receiver. In particular, the display 300 shown in FIG. 3 is displayed by the facility when the user selects link 243 for the reviewer Frank Behrens shown in FIG. 2. The display includes a profile 310 for the reviewer, which include such information as the reviewer's name 311, a nickname 312 for the reviewer, an email address 313 for the reviewer, the current rank 314 of the reviewer, the beginning of a biographical sketch 315 of the reviewer, a link 316 to the entire biographical sketch of the reviewer, a count 317 of the number of reviews submitted by this reviewer, a count 318 of the number of positive votes cast for the reviews of this reviewer, and a graphical indication 319 of the rank of the reviewer. The display further contains each of the reviews submitted by the reviewer. As an example, the display contains information 320 about the reviewer's most recent review. This information includes information 321 about the reviewed item, such as the title, artist, format, price, and availability of the item, as well as a link 322 that may be used to display more information about the item, and a control 323 that may be used to place an order for the item. The information 320 also includes a grade 324 assigned by the reviewer to the item as part of the review—here the grade shown is four stars out of five stars; the review's title 325; the date 326 on which the review was submitted; and the text 327 of the review. The display also contains a section 330 containing links to item categories for which the reviewer has submitted reviews.

FIG. 4 is a display diagram showing a typical display presented by the facility containing detailed information about an item. In this case, the item is the Tchaikosvky Symphony No. 6 DVD whose review is shown in FIG. 3. The display 400 shown in FIG. 4 is displayed in response to the selection of link 322 shown in FIG. 3. In addition to other information about the item, the display includes a list of reviews submitted for the item. These include review 413 submitted by the reviewer Frank Behrens. The review 410 includes the reviewer's name 411, which is a link to the display shown in FIG. 3, a graphical indication 412 for the current rank of the reviewer, the item grade 413 assigned to the item as part of the review; the review title 414; the text of the review; and voting buttons 415 and 416. A user that is undecided about buying this item may read review 410. Based upon various aspects of the review, the user may either find the review helpful in deciding whether to purchase the item or unhelpful in deciding whether to purchase the item. Such a user may select button 415 if the review was helpful, or button 416 if the review was not helpful. In some embodiments, such votes are the basis on which the facility scores and ranks this reviewer. The display further includes a link 421 that the user may select to submit his or her own review of the Tchaikosvky Symphony No. 6 DVD item.

FIG. 5 is a display diagram showing a typical display presented by the facility incorporating information about two randomly-selected top-ranked reviewers. The display 500 includes a list 510 or randomly-selected top-ranking reviewers. The list is comprised of one or more entries, such as entries 520 and 530 shown here, which each relate to one randomly-selected top-ranked reviewer. For example, entry 520 relates to reviewer number 14, and includes the reviewer's rank value 521; the reviewer's name 522, which is a link which that the user may select in order to display more information about this reviewer; the number of reviews submitted by this reviewer 523; and the number of positive votes 524 received by this reviewer's reviews. Some entries contain an image of the corresponding reviewer, such as image 535 in entry 530. The display also includes a link 540 to the top reviewer's list shown in FIG. 2.

FIGS. 6-8 show an example of an approach to obtaining an item review submitted by a reviewer. FIG. 6 is a display diagram showing a typical display presented by the facility to enable a user to submit a new review. The user typically selects this display in conjunction with a particular item, for which the user can submit a review. For example, display 600 is displayed in response to the user's selection of link 421 in the display containing detailed information about the Tchaikovsky Symphony No. 6 DVD shown in FIG. 4.

The display includes the identity of the user, which is attributed as the identity of the reviewer. To attribute a different identity to the reviewer, the user can select link 602. The display includes information 603 identifying the item to be reviewed. The display includes a control 604 used by the user to specify a grade or rating for the reviewed item. The display includes field 605 for entering a title for the review. The display includes field 606 for entering the text of the review. The user may select radio button 607 in order to display a reviewer name with the review, which may edited in field 608. Alternatively, the user may select radio button 609 to make the review anonymous and prevent the review from being associated with the reviewer's name. The display also includes field 610 for entering the author's location. Before preparing the review as described, the user may select link 621 in order to display guidelines for preparing the review. The user may also select link 622 in order to display one or more example customer reviews demonstrating compliance with the review guidelines. When the user has assembled the review to be submitted using the above described aspects of the display, the user selects button 611.

FIG. 7 is a display diagram showing a typical display presented by the facility containing guidelines for preparing an item review. The display 700 contains typical guidelines for this activity.

FIG. 8 is a display diagram showing a typical display presented by the facility containing sample reviews. The display 800 includes sample reviews, such as sample review 810. In some embodiments, sample reviews are chosen from among the reviews of top-ranked reviewers, such as from the reviews of the top-ranked 100 reviewers.

FIG. 9 is a flow diagram showing steps typically performed by the facility in order to process a new item review submitted by a reviewer. In step 901, the facility receives a new item review from a reviewer. Such a new item review may be received in a variety of ways, including receiving it using a web-based form as shown in FIG. 6, receiving it via electronic mail, receiving it in a document uploaded to an FTP server, receiving it in paper form, receiving it in audio form, etc.

In step 902, the facility assigns the review identifier to the review that will be used to identify the review and stores the received review in conjunction with the assigned review identifier. In step 903, the facility creates an association between the assigned review identifier and an identifier for the receiver. As an example, Table 1 below shows information stored in a data structure, and, in particular, rows stored in a table, in order to create such as an association.

TABLE-US-00001 TABLE 1 review identifier reviewer identifier . . . . . . 96335807 449603151134 96335808 230651087912 96335809 449603151135 . . . . . .

For example, the first row shown in Table 1 indicates that the item review having review identifier 96335807 was submitted by the reviewer having reviewer identifier 449603151134.

In step 904, the facility obtains editorial approval for the review. As an example, the facility may in step 904 add the review or its review identifier to a queue of new reviews to be subjected to a manual editorial review process. Step 904 is optional, and may be applied selectively, such as to reviews submitted by reviewers that have a reviewer score below a certain threshold, or a reviewer rank value above a certain threshold. Alternatively, the new review may be added to such a queue at different points depending upon the score or rank of the submitting reviewer.

In step 905, after editorial approval has been received for the review, the facility makes the review available on the web site. This may involve, for example, incorporating the review in a display such as those shown in FIGS. 3, 4, and 8. Again, step 905 may be performed in a manner sensitive to the score and/or rank of the submitting reviewer. After step 905, these steps typically conclude.

FIG. 10 is a flow diagram showing steps typically performed by the facility in order to process votes received for published item reviews. In step 1001, the facility receives a vote on a review from a voter. For example, the facility may receive an indication that a user pressed a button like button 415 shown in FIG. 4 indicating that a review was helpful to the user, or a button like button 416 shown in FIG. 4 indicating that a review was not helpful to the user. Such a vote typically identifies a review to which the vote relates, an identifier identifying the voter, the date and time at which the vote was cast, and the value of the vote, i.e., whether the voter indicated that the review was helpful or not helpful. In step 1002, the facility stores this information together. For example, this information may be stored in an entry of a data structure, such as a row of a table, as shown below in Table 2.

TABLE-US-00002 TABLE 2 review identifier voter identifier date/time helpful 96335807 919654002315 Apr. 11, 2001 16:04:02 yes 43227116 310956681822 Apr. 11, 2001 16:04:02 yes 84325510 919654002315 Apr. 11, 2001 16:04:03 no

For example, it can be seen from the first row shown in Table 2 that a vote on the review having review identifier 96335807, cast by a voter having voter identifier 919654002315, was cast at 16:04:02 on Apr. 11, 2001, and that this vote indicated that the review was helpful. After step 1002, these steps typically conclude.

FIG. 11 is a flow diagram showing steps typically performed by the facility to update reviewer scores and rankings. The facility typically performs these steps periodically, such as once per day. In step 1101, the facility identifies reviewers whose reviews have received at least one new vote. In some embodiments, step 1101 involves selecting from the stored votes those votes cast on the date immediately preceding the current date. The review identifiers in the selected votes are used to identify the reviewer identifiers corresponding to the selected votes.

In steps 1102-1104, the facility loops through each reviewer identified in step 1101 as having received at least one new vote. In step 1103, the facility calculates an updated reviewer score for the current reviewer. The facility is preferably configurable to use a wide variety of different approaches to determining reviewer scores. In one approach, the facility calculates a reviewer score based upon scores calculated for each review of the reviewer. In accordance with this approach, for each reviewer, the facility applies formula (1) below. Score.sub.reviewer=2(A+B+C)-(D+E) (1) where A is the number of reviews submitted by the author for which at least 3 positive indications were received, B is the number of reviews submitted by the author for which at least 10 positive indications were received, C is the number of reviews submitted by the author that were the first reviews submitted for the corresponding item and for which at least 3 positive indications were received, D is the number of reviews submitted by the author for which at least 3 more negative indications than positive indications were received, and E is the number of reviews submitted by the author for which at least 10 more negative indications than positive indications were received.

In some embodiments, the facility excludes from values A through E any votes that (1) were subsequent to the first vote cast by the same voter on the same review, or (2) were subsequent to the fifth vote by the same voter on any review by the same reviewer. Various other approaches to calculating a score for a review may also be employed by the facility, such as those in which fewer or more factors are considered, or such factors are weighted differently. Additional factors that various embodiments may consider includes such factors as whether the review is the first review submitted for a particular item, and the number of orders that have been placed for an item by a customer to which the review has been displayed.

The score for a reviewer is generated by summing the scores of the reviewer's reviews as shown below in formula (2).

.times..times..times..times..times. ##EQU00001## In various embodiments, a reviewer's score may be determined by combining the scores for the reviewer's reviews in a variety of other ways.

After step 1103, the facility continues in step 1104. In step 1104, if additional identified reviewers remain for processing, the facility continues in step 1102 to process the next identified reviewer, else the facility continues in step 1105. In step 1105, the facility attributes a rank value to each reviewer based upon their reviewers scores, as updated. In step 1106, the facility proceeds to utilize the reviewer rank values attributed in step 1105 in any web pages of the web's web site that reference these rank values. This may involve, for example, utilizing the attributed reviewer rank values in a display such as those shown in FIGS. 2-5. After step 1106, these steps typically conclude.

It will be understood by those skilled in the art that the above-described facility could be adapted or extended in various ways. For example, the facility may be operated by others besides web merchants. A variety of different types of rewards may be offered to reviewers submitting useful reviews. The usefulness of reviews may be discerned in a variety of different ways. While the foregoing description makes reference to preferred embodiments, the scope of the invention is defined solely by the claims that follow and the elements recited therein.


E-Mail StorefrontBacktalk Editor Evan Schuman at
eschuman@storefrontbacktalk.com
Search Through Blog Blurbs
Search Through All Stories
SFBT Twitter Feed
Quickly catch-up on the latest in E-Commerce and Retail Tech with our free weekly newsletter, with urgent bulletins as news merits.
StorefrontBacktalk will never sell your E-mail address to anyone at anytime.
Evan Schuman is the former retail technology editor for eWEEK.com, PCMagazine, CIOInsight and retail reporter for RISNews and Consumer Goods Technology. Having covered IT issues for 21 years - and other stuff like legal affairs, politics, Wall Street and the environment for about eight years before that - Schuman is in a good position to gripe about technology trends and sometimes accidentally make a good point.
Cyber Monday '08: The Butterfly Effect In Action
Whether the wing flapping of a bug can eventually cause a tornado is debatable, but Cyber Monday 2008 should serve as a warning to E-tailers to avoid the lesson learned by Staples and Dell on Monday (Dec. 1): Don't ignore the butterfly effect.
What Was Wal-Mart Thinking When It Made Key Site Changes On Black Friday?
At about 6 AM on Black Friday, Wal-Mart's site went down for about an hour and then came up. But this time, it had moved its content pages to an outside service. Wal-Mart said it had been a scheduled change, a concept that Gareth Evans, head of client services at Web tracking firm Sitemorse, finds unlikely.
Visa Card Holograms Shut Down POS Terminals
In a trial of new holographic magnetic stripes for its payment cards, Visa found the cards "emitted an electrostatic discharge that caused POS terminals to shut down," according to a report in The Nilson Report, a respected credit card industry newsletter.
Microsoft's Live Search Program Crashes On Black Friday
Microsoft's Live Search cashback program—which gives rebates to those who comparison shop online and choose stores that are part of the program—was a bit too popular on Black Friday (Nov. 28) and crashed for several hours, leaving consumers with no cashback and a lot of anger.
Sears.com Melts Down On Black Friday, But Costco, Walmart, Saks and Kmart Have Issues, Too
Sears.com suffered the worst Web problems on Black Friday (Nov. 28), experiencing a series of complete site crashes for much of the day. Although no other major retailer came close, according to preliminary reports, many of the industry's largest merchants suffered site slowdowns or other Web problems, including Walmart, Kmart, Saks, Overstock, Amazon, Target, Kohl's, Costco and Buy.com.
Visa Europe Testing A Reciprocal Authentication Card
In a trial initially limited to the United Kingdom, Switzerland, Israel and Italy, Visa Europe is starting a trial this month of a card with an 8-digit alphanumeric display, 12-button keyboard and a long-life battery. The card has the ability to offer reciprocal authentication, which is designed to allow consumers "making transactions via phone or the Web a way to identify the party on the other end before transmitting identifying credentials."
Black Friday Cyber Sales Up A Mere One Percent
Black Friday (Nov. 28) E-Commerce sales hit $534 million, reflecting a one percent increase from last year's Black Friday, ComScore reported Sunday (Nov. 30).
JCPenney Adds Merged Channel Twist, Including Wake-Up Calls
A Web-generated wakeup phonecall to get customers to in-store early morning sales may not have a material impact on quarterly sales, but it's a creative touch for the E-Commerce site JCPenney relaunched right before Black Friday (Nov. 28). Even if the system's in-store inventory update isn't quite accurate.
CRM Chutzpa: Best Buy Credit Card Thief Sought Loyalty Rewards
A group of credit card thieves in Seattle tried to maximize their profits by using their stolen credit card data to open a loyalty card account with Best Buy, where they could get could extra benefits along with their stolen products, according to a federal indictment filed Nov. 19. One had tried a similar rewards scam with a Home Depot reward card and a Sears gift card.
Tracking How Many Consumers Flee During A Site Meltdown
Following a site meltdown by a major U.K. retailer this month, Internet traffic tracking firm Hitwise was able to document and make concrete what has always been assumed: Consumers abandon a retail site when it melts down faster than politicians vote for a tax cut.
How Bleak Is The E-Commerce Picture? Mixed Messages
Recently released numbers raise questions as to whether online will be much of a savior at all. New figures from eMarketer project that E-Commerce sales will top last year's numbers by some $5.6 billion, a 4.1 percent increase from $136.8 billion to $142.4 billion.
PCI Fines: Nuisance Or A Ticket To ROI?
Eduardo Perez of Visa has called its fines for non-compliance "nuisance" fines. In other words, the fines are not large enough to be a big financial burden to retailers but are large enough to get the CFO pissed off about having to pay them and maybe large enough to get a CEO to at least show up for a meeting to discuss PCI.
Trying To Protect Payment Data When You Can't Even Find It All
The IT struggle with knowing where all payment data is—let alone trying to enforce rules that pretty much try and keep it there—was the topic of a StorefrontBacktalk a podcast this week with our own PCI columnist, David Taylor, and security specialist J.D. Oder, the chief technology officer at Shift4.
Is Price Comparison Dead? And, If So, Should We Celebrate?
A Supreme Court decision from back in June 2007—intended to give consumer goods manufacturers greater control over their products' pricing—is fueling confusion, mistrust and runarounds among E-tailers trying to compete on price.
Wal-Mart To Pay $1.4 Million Fine Because Of Price Change Database Problem
Wal-Mart has agreed to pay $1.4 million to settle complaints that it overcharged customers in California. The Nov. 24 deal involved the mispricing of some 1,043 items over four years. Some of the problems happened because associates would make pricing changes to items in the store's register database but not in the aisle.
PayPal To Use Cellphones To Authenticate Payments
PayPal has come up with yet another payment-related use of a cellphone: to authenticate a non-mobile E-Commerce transaction. Customers of the payment giant "can now choose to receive a unique six-digit security code via text message to their mobile phones prior to logging in to their accounts," PayPal said.
American Patriots Finding They Can't Rely On Barcodes
There's an E-mail campaign that says consumers can identify American products by the first three digits of the barcode. In theory, this would allow people who only want to buy American products an easy way to do that. The only problem is that the trick doesn't always work, which means it could have the opposite effect.
HP Finds Cutting Back Related Items Shown Boosts Sales
When are related product lists helpful and when are they distracting? Is it an obviously useful upsell or is it doomed to the fate of the salesperson who shows a customer one too many choices? HP thinks it's often the latter and has sharply trimmed the number of related items it shows. And the company is claiming a 30 percent sales increase as a result.
Amazon's Gift Card Future: Personal, But Not Too Personal
Amazon.com, which arguably has one of the most extensive retail CRM databases and purchase recommendation engines, envisions a Catch-22 future for gift cards. The key is making them more personalized, more customized. And yet, anything that hints of privacy violations is off-limits. It's like a starving man being given the keys to a well-stocked food locker as long as he agrees not to eat anything.
TiVo And Domino's Try E-Commerce Without The PC Or Phone
As more retailers try to go where the customers are rather than getting them to come to the retailer, TiVo and Domino's are taking the next logical step with a TV-as-E-Commerce-Device approach.
O, Kiosk, How Doth I Differentiate Thou?
We make calls on PCs and surf the Web on our phones. The lines of separation are blurring fast. But in the world of retail technology, the difference between a kiosk and digital signage is one of the more difficult distinctions to make. How to describe that difference? To one CFO, the answer was obvious: A poem. In rhyming verse. Rhyming verse that is so bad it's almost good.
Do You Have a Mobile Blindspot?
The further employees get from corporate, and from corporate networks, the more likely they are to do things with their computer that security managers would rather they didn't. GuestView Columnist David Taylor asks if these people might be doing things (e.g., downloading malware) that could bring down your company?
All Web Meltdowns Are Not Created Equal
When file transfer site YouSendIt—with more than 100,000 paid users bringing in some $10 million this year—crashed on Monday (Nov. 17), it illustrated the kind of crash that should make retailers very concerned.
Security Podcast: 12-Year-Old Data And Publishing Encryption Keys
Podcast panelists debate card replacement problems, including inadvertently printing encryption keys on customer receipts and the refusal of the card brands to shorten how long expiration dates are valid. "We now have to worry about data that's been there as many as 12 years."
Will Consumers Punish Retailers That Misuse CRM Data?
A loyalty card that consumers can turn on and off could potentially usher in a consumer revolution of sorts, allowing the majority to punish merchants they see as misusing CRM data that has been entrusted to them. At least that's one scenario painted by the president of the company that is pushing the card.
NRF Says Gift Card Spending To Drop
Amidst an avalanche of hype about the desirability of gift cards this holiday season, the National Retail Federation on Tuesday (Nov. 18) predicted a six perfect drop in gift card sales this season, from $26.3 billion spent during last year's holiday season to a projected $24.9 billion for this season.
What A Bond Villain's Datacenter Would Look Like
Some 30 meters below solid bedrock underneath Stockholm, an abandoned nuclear bunker has been transformed into what could only be described as the world's coolest datacenter.
E-Commerce Site Crashes To Soar This Holiday Season, With Upgrades, Partners And Discount Traffic The Likely Culprits
Several factors are lining up—including rushed technology upgrades, more site handoffs for everything from mobile to social networking widgets and a surge in traffic from bargain hunters—that could easily make this holiday shopping season one of the crashiest in years, if not the crashiest. (Note to copydesk: I don't care if crashiest is not a word. It should be.)
Sears Mobile Move Illustrates The Mobile E-Tail Challenge
When Sears rolled out its mobile effort (Sears2go) this month, it illustrated the challenges for a retailer trying to craft a clean and stable mobile strategy at a time of extreme flux for the mobile space.
"Store Locator" The Unsung Hero Of Web Analytics
When E-Commerce execs try and understand abandoned shopping carts, they often overlook concrete clues. One of the best is whether shoppers clicked on the store locator link right before leaving. But deciding what to do about abandoned carts, that gets complicated. The innocuous-looking store locator is akin to waving a red cape in front of the face of an E-Commerce manager bull.
How Much Do You Really Know About Your Security Consultant?
The Web is overflowing with analysis of the TJX data breach disaster, but what's intriguing is the possibility that some of the indicted suspects may have worked as code writers in the light of day for some major companies, including Morgan Stanley.
Would CRM Work If Customers Had An On/Off Switch?
Equifax on Thursday (Nov. 13) announced an E-Commerce CRM and payment card that consumers can activate and deactivate based on how they feel about the site they are visiting. The only way such a card—dubbed the Equifax online identity card—will be successful is if it's adopted by a large number of retailers. And each of those retailers would have to be willing to surrender one of their most precious pieces of data: customer history.
Site That Finds And Integrates Coupons With Low Pricing?
We see tons of variations on the meta-search concept for E-Commerce, but this site seems to have hit on a truly practical combo: An engine that finds the lowest prices and simultaneously finds relevant coupons and then integrates the two.
Visa's Global PCI Effort: Small Carrot, No Stick
Visa, long the key driver of compliance with the PCI security standards, is helping to clear up merchant and service provider confusion regarding the global deadlines for PCI DSS compliance. But GuestView Columnist David Taylor notes some unusual phrasing and concludes that Visa wants to ease the compliance process to get more service providers outside the United States on board.
Target, Best Buy Turn Gift Cards Into Literal Entertainment Devices
Instead of making gift cards worthless once they're emptied, Target and Best Buy have opted for the other extreme: With enough micro electronics, the gift cards themselves might be worth more than the merchandise they can buy.
Wal-Mart To RFID Crack Down On Chinese Suppliers By January
Wal-Mart will insist that its Chinese suppliers comply with RFID tagging by January 2009. And given various recent safety problems reported from China, Wal-Mart is also requiring sub-contractor information be included with every tagged product.
Best Buy's API Strategy Goes Beyond Social, Mobile
Forced to try and boost revenue in a tight economy, Best Buy is pushing an aggressive plan—based partly on open APIs—to sell to customers wherever on the Web they're hanging out, rather than trying to get them to virtually travel to the retailer's online storefront.
While U.S. Retailers Close Stores, World's Largest Mall Opens In Dubai
While the headlines in the United States tell of store closings and expansions being back-burnered, it's a very different story in at least one part of the Middle East. In Dubai in the United Arab Emirates, Tuesday (Nov. 4) saw the world's largest mall opening ever.
Ohio Man Pleads Guilty To $1 Million Barcode Scam
The defendant pleaded guilty to heading a conspiracy that netted more than $1million by using phony UPC labels to obtain products and then sell them on eBay.
One More Charged In TJX Breach
Federal prosecutors have apparently accused a New York man of providing a sniffer program to help the TJX cyberthieves steal payment data. The fact that 25-year-old Stephen Watt has been charged with unlawful access to computers, wire fraud, aggravated identity theft and money laundering is not in dispute, nor is the fact that he has been accused of delivering a sniffer program to accused TJX mastermind Albert Gonzalez.
RFID Market To Top $5.3 Billion This Year, Says ABI Research
RFID sales globally will be more than $5.3 billion this year, with supply chain management, ID documents, ticketing and contactless payment drive shipments leading the way, according to a report released Monday (Nov. 3) by ABI Research.
MasterCard Pushing NFC Mobile Program
MasterCard's PayPass is ramping up its mobile program with an over-the-air provisioning service to supposedly make it easier for consumers to personalize their payment data on their mobile devices.
PCI Avoidance Strategies
Without a doubt, the most popular strategy for dealing with PCI compliance and data security is avoidance, writes GuestView Columnist David Taylor. Not unlike the game of "hot potato," which dates back to the pilgrims, the goal is to find someone who is willing to put up with the hassle of PCI compliance and then give that person all the credit card data.
Costco's Embrace Of Online Customer Comments Illustrates How Innocuous They Are Now Viewed
When Costco on Monday (Oct. 27) announced that it would support—for the first time—customer comments on its products, the move was less noteworthy for the $71 billion chain's late-to-the-party embrace than for what it says about the industry's acceptance of a once much-feared feature.
The Old PCI Squeeze Play
The position that there are far-reaching implications of the Payment Applications Data Security Standards (PA DSS) for the merchant community is hardly new, as they affect thousands of payment, infrastructure and business management applications. But GuestView Columnist David Taylor argues that some concerns raised by Jake Star, technology VP at HEI Hotels and Resorts, take this to the next level: the old squeeze-play level.
Will Retail IT Be Spared The Recession?
Although there is a little doubt that the United States is in for a very rough economic period over the next half-year or more, there is ample reason to believe that retail IT may escape mostly unharmed. Let's not get too optimistic here. "Mostly unharmed" doesn't mean escaping untouched. But it does mean that when large companies—especially retailers—have to suddenly make do with a lot fewer people, they need that good ole IT magic more than ever.
Barnes & Noble E-Commerce Focuses On Experience
The battle for booksales should be an online natural. But as Barnes & Noble discovered this week, the compelling, intimate experience of a physical bookstore is still proving elusive.
Manufacturer Gets Creative To Meet Wal-Mart RFID Requirement
Pet product maker Normerica opted for an unorthodox combo of smart boxes with embedded RFID tags and a mobile reader to comply with Wal-Mart's RFID requirement. The application was attractive because it reportedly involved "no significant retooling of its packing or shipping lines."
Handheld RFID Reader Claiming 25-Foot Read Range, 400 Tags/Sec. Read Rate
A Hong Kong RFID vendor is boasting about a new $1,950-$2,500 handheld UHF reader "with a read range exceeding 25 feet with standard dipole passive tags and a throughput reaching 400 tags per second." That claim is usually reserved for fixed readers, a very sharp claimed performance boost.
Racial Slur Programmed Into POS At Genesco Store
Genesco, which owns more than 2,000 stores operating as nine different chains including Johnston & Murphy, Dockers and Journeys, learned this week how POS receipt customization can be remarkably dangerous.
Does The Rich Niche Twitch To A Different E-Commerce Itch? And If Ditched, Do They Switch?
As the U.S. economy collapses (temporarily mind you, but a collapse nonetheless) and holiday sales contract, the one segment rumored to likely fare best is merchants selling to the highly affluent. But in what could be bad news for E-Commerce, those rich niche retailers tend to resist online sales more fervently than other E-tail segments.
Could Software Allow Shelves To Look Back At Consumers?
Technology that has been deployed to digitally watch—and analyze—how consumers interact with digital signage could also be used to interpret what they are doing while looking at a cereal shelf. Are they ignoring the product or are they picking it up, reading the label and then quickly putting it back?
PCI Group Figures Out That The Word "Vendor" Is The Anti-Credibility
The group originally called the PCI Alliance, which changed its name to the PCI Security Vendor Alliance on Tuesday (Oct. 21), has changed its name again—this time to the Payment Card Industry Security Alliance (PCI SA). Mercifully, it never bothered to change its URL, so it's still pcialliance.org.
Big Lots Launches Its First E-Commerce Site With A "Deal Of The Day"
When 1,361-store $4.6 billion chain Big Lots unveiled its first E-Commerce site Tuesday (Oct. 21), it decided to borrow a gimmick from its brick-and-mortars and re-create what it dubbed the stores' "treasure hunt atmosphere." Specifically, every morning, the chain plans to announce on the site a "deal of the day," which is a limited-inventory product at supposedly ultra-discounted rates.
Holiday Online Sales To Grow This Year, But At A Much Slower Rate
Two new reports paint a relatively slow growth picture for holiday E-tailers, although the projections are for double-digit growth this year. The dour side is that both projections are about half of the growth rate experienced last year.
Could Japanese Mobile CRM Pilot Serve As Mobile Payment Prototype?
A major Japanese mobile phone loyalty card trial slated to run from February through June of next year might prove to be a powerful prototype of how other countries might deploy mobile payment networks.
When PCI Compliance Is A Competitive Advantage
Companies are beginning to extend the protection of PCI-driven security controls to other confidential data, which is great, argues GuestView Columnist David Taylor. What is even better, he says, is that some service providers are finding they can leverage their PCI compliance to gain a competitive advantage.
Australia's Woolworths Abandons RFID Plan To Trace Produce Crates
Australia's Woolworths, which runs the country's largest supermarket chain, has given thumbs up to one RFID trial and thumbs down to another. "The overall cost for a company the size of Woolworths is still too high, and the return on investment for track and trace is just not enough for us to race ahead."
Pizza Hut, Papa John's To Merge E-Commerce With Social Networks
The very premise of E-Commerce is for E-tailers to create a beautiful site, where your customers come to shop. The latest trend, though, is to cut deals with your customers to buy from you anywhere but your site, whether it's on MySpace or Facebook social networking sites, from a cell phone, in the middle of a Google search or while watching a YouTube video.
Study: Identity Thieves Swarm To English Speakers
E-Commerce customers who speak English are "the most frequent victims of identity theft, twice the rate of France, Germany and Spain," according to a study released Tuesday (Oct. 21) by PayPal. The E-mail survey of 1,000 consumers was conducted this summer and examined six countries: the United States, Canada, France, Germany, Spain and the United Kingdom.
Payment Authorization Terminal Sales Soar Worldwide
As retailers across the globe modernize, the installed base of payment authorization terminals has soared almost 22 percent from 2006 to 2007. As happens typically in a growth segment such as authorization terminals, vendor consolidation has concentrated control—and, therefore, retail purchase options—into far fewer hands.
Web Irony Of The Week: Sites That Sell Eyeglasses Have Weakest Support For Visually Impaired
In their September stats, U.K. sites that sold eyeglasses and related vision aids fared among the very worst in one criteria: sites that are designed to be easily used by those with vision difficulties.
Report: Power Attacks On Credit Cards Still A Major Threat
It's hardly a new payment card security threat, but what has become known as differential power analysis (DPA) is still very much a threat on most payment smart cards. A DPA attack takes advantage of the electrical impulses inherent in any smart card.
Circuit City Identical Online/Offline Pricing Plan Might Actually Work
The Circuit City "one price promise" move could ultimately prove to be quite a clever piece of marketing. First, it will be trumpeted as a consumer advantage, even if it means that the price equality will be achieved by sometimes (all the time?) online pricing being raised to match the in-store price.
Home Depot, McDonald's Pushing Non-Traditional Kiosk Trials
Home Depot and McDonald's are both in the middle of non-traditional kiosk trials. McDonald's is on its fourth such trial, after having concluded that the first three simply didn't work well. Not too many retailers would opt for a fourth trial after three unsuccessful attempts. The non-traditional Home Depot kiosk trial is based more on the units themselves—small mobile units, some as tiny as 5-inches tall—and the size of the chain's planned kiosk commitment: Well north of $100 million for full deployment
Wal-Mart, Amazon Learning That Product Downloads Are Harder Than They Look
The last few weeks have not been kind to the product download efforts of retail giants. Last month saw Amazon inadvertently giving away tons of music and video downloads, courtesy of a glitch in Adobe's encryption approach. This month, it was Wal-Mart's turn.
How Cloud Computing And Growing Franchisee Influence Are Hitting Retail IT
Consider this: Is there a connection between a growing support for various cloud computing approaches and the increasingly active IT role that franchisees are taking? Yes, it's a wacky juxtaposition, but stay with me for a moment. There has been a steady noise coming from retail franchisees who are trying to drive more of their stores' IT strategy.
New QA Review Toughens PCI Assessors
When GuestView Columnist David Taylor wrote last week about PCI 1.2 changes, he received quite an earful from readers that some changes are having an even more strict impact.
Is Wegmans' Self-Checkout Trial Truly For Customer Service?
When supermarket chain Wegmans confirmed this month its first-ever self-checkout trial, it was billed as a customer service feature. That's technically true, but only in a very roundabout way.
New European Card Data Theft Ring Raising China Questions
A report this week from The Wall Street Journal
about a European "credit-card fraud ring that funnels account data to Pakistan" and "uses untraceable devices inserted into credit-card readers that were made in China" sparked a lot of retail interest, but it's unclear how widespread or innovative the attacks were.
Wine.com Federal Court Ruling Only The Beginning Of E-Commerce Changing Rules
This is the latest volley in a federal court—and soon federal legislative—game of Internet taxation and control. Can states force E-Commerce sites to tax for them? Can municipalities police what gets sold in their communities, even via laptop or PDA?
Home Depot Tackles A Supply Chain That Doesn't Recognize Geography
"The systems were very poorly adjusted to reflect differences in locales," said CEO Frank Blake. "We are the single-largest less-than-truckload shipper in the United States. A lot of trucks are going to stores that aren't full. It's not efficient."