Patient Assessments and Online Ratings of Quality Care
Patient Assessments and Online Ratings of Quality Care
Websites were identified via a Google search using the following keywords: "doctor evaluation websites," "doctor evaluation," "rate doctors," and "find doctors" in addition to searches under the names of specific physicians. Other sites were found during a literature review of the topic. This yielded 58 total websites. Of these, 23 were eliminated based on the following exclusion criteria: the site did not contain physician evaluations, the site was managed by an insurance company, or the site did not cover all physician specialties. The remaining 35 sites were included in the analysis.
All websites included in this study were first independently reviewed by two investigators (JM and BL) and sorted into the following broad categories:
The final lists of these sites are available in Table 1. Each website was analyzed with a series of checklists to assess its general characteristics ( Table 2), physician profiles ( Table 3), evaluation features ( Table 4), and web traffic data ( Table 5). Additional information was obtained by contacting the website administrators directly with a questionnaire.
Geographics No website in our study was found to have an overt geographic focus. Some newer sites, such as http://docspot.com/ and http://zocdoc.com/, focus on large metropolitan regions in the United States. Per http://docspot.com/ website administrators, this merely reflects the distribution of the US population, and coverage is expected to extend into all parts of the country with time. In a beta search for "gastroenterologists in the region of Norfolk, Virginia," several sites were found to have minimal to no coverage of the trial area. Four dedicated PEWs (http://healthcarereviews.com/, http://ratemds.com/, http://findadoc.com/, and http://doctorescorecard.com/), enable searches for physicians outside the United States.
Fee for access None of the dedicated PEWs or health information sites charge to access or post reviews. Three general information sites with physician evaluation features (http://angieslist.com/, http://checkbook.org/, and http://merchantcircle.com/) do require payment for access to their services. These fees enable the site administrators to finance their websites and serve as a barrier from frivolous, anonymous posts. All websites studied participate in advertising as a means of generating revenue.
Medical education Medical education is not a primary feature on most dedicated PEWs. The most noteworthy exception to this is http://healthgrades.com/, which integrates patient education throughout each physician profile. Two additional sites, http://vitals.com/ and http://drscore.com/, provide links to sites of interest and to relevant support groups from physician profiles.
Internal website awards Three websites (http://healthgrades.com/, http://yourcity.md/, and http://zocdoc.com/) have developed unique awards to recognize exceptional physicians. The metrics for receiving these awards is not readily apparent and varies with each site. http://vitals.com/ and http://drscore.com/ notify users if the physician of interest is a Castle Connolly Top Doctor. http://drscorecard.com/ and http://findadoc.com/ enable physicians to give online endorsements of their colleagues.
Anonymous posting The majority of PEWs allows users to post reviews of physicians anonymously—provided they supply an accompanying email address. This feature may concern healthcare workers who fear the lack of accountability that this grants to reviewers. However, most websites have mechanisms in place to prevent a given physician from being reviewed multiple times by the same patient. This also serves to discourage the use of the third-party groups that might be employed to improve a physician's reputation by fabricating reviews.
Physician responses The question of physicians being able to respond to patient's concerns on their profiles is a vital one. Unfortunately, this is not a feature that has been widely implemented in PEWs to date. Only five websites (http://docspot.com/, http://doctorscorecard.com/, http://healthgrades.com/, http://ratemds.com/, and http://yourcity.md/) have overt mechanisms in place to enable doctors to respond to dissatisfied patients. It is possible that other websites also enable physician responses, but which were not readily apparent when reviewed for this paper.
Enhancements to physician profiles Several sites, including http://findadoc.com/ and http://doctorsdig.com/ offer physicians enhanced memberships for a fee, and http://healthgrades.com/ allows hospitals and practices to promote their physicians on its website. These fees grant physicians additional opportunities for advertising while generating revenue for the PEW. Although most sites do not have "featured physician profiles," the majority of them do encourage doctors to claim their own profiles for free and to make direct modifications/enhancements to them. These enhancements include adding a photo, updating listed information, and posting supplementary information. When profile information is incorrect and a physician cannot claim his/her profile, the information can usually be amended by contacting the website's administrators directly.
Physician profile characteristics On the basis of the results from our established checklists, unclaimed profiles on dedicated PEWs contain an average of 7.2±3.0 independent data points per physician (such as doctor name, specialty, age, etc.). Advanced or "claimed" profiles generally offer a more complete picture of a given physician with an average of 9.7±4.3 data points. Interestingly, health information websites with physician evaluation features tend to have more robust physician profiles (average of 9.6±2.7 or 10.2±3.1 for unclaimed or claimed profiles, respectively) than do websites dedicated to physician evaluation alone. General information websites that also participate in physician evaluation tend to have the least complete profiles with 5.4±3.0 (unclaimed) or 8.3±3.3 (claimed) data points per profile.
The most common data points listed in a given physician's profile are: name, specialty, and office address (100%, 33/33) followed closely by office phone/fax number (94% 31/33). Other common components are physician education/training, gender, and photo (48.5% each, 16/33), hospital/practice affiliation (45.5%, 15/33), accepted health insurance plans (39.3%, 13/33), licenses/certifications (36.3%, 12/33), years of experience (30.3%, 10/33), and languages spoken (27.2%, 9/33). More rare components of a profile include procedural expertise, research/clinical interests, availability, office hours, awards/recognition, malpractice history, and links to the physician's website (all less than 25%). Among all websites studied, http://healthgrades.com/, http://docspot.com/, and http://healthtap.com/ offer the most thorough unclaimed profiles with 13 data points. http://bookofdoctors.com/ offers the most complete claimed profile with 19 data points.
Review structure Most PEWs have similar features for physician review. Reviews typically start with the user assigning the provider an overall score (usually depicted by a number of stars). This is followed by questions regarding that physician's punctuality, bedside manner, time spent with the patient, etc. The user rates the doctor with stars on each of these qualities before being offered a chance for free response.
From our analysis, the average number of evaluation points for a dedicated PEW is 6.5±3.0, whereas health information PEWs invite users to score doctors on an average of 4.4±3.4 different values. General information websites that participate in physician evaluation tend to only require a basic "overall" score followed by free response. Of these websites, only 1.3±1 assessment points are usually required. Among all the PEWs studied, http://yourcity.md/ contains the most detailed reviews by requiring users to score physicians on 12 different values.
Across all PEWs with scaled reviews, the most common value for physicians is their "overall" score (92%, 23/25). This is followed by punctuality (52%, 13/25), quality of staff (48%, 12/25), and knowledge or ability to determine a correct diagnosis (40%, 10/25). Additional values include physician availability (32%, 8/25), office cleanliness and doctor bedside manner (both 28%, 7/25), time spent with the patient (24%, 6/25), follow-up after an appointment, and whether or not the patient would recommend a given physician to someone else (both 20%, 5/25). Less commonly reported values (less than 20%) include costs/billing, how well the physician educated and listened to the patient, whether or not the doctor was approachable, and the location of his/her office.
Site-usage informationhttp://compete.com/ (a World-Wide-Web analytics company) was used to evaluate each website's monthly traffic—in number of unique visitors—for the previous year. In the year leading to this study, http://healthgrades.com/ was the most frequently visited PEW dedicated to physician evaluation alone. From October 2011 to September 2012, http://healthgrades.com/ received an estimated 5.5 million unique visitors per month. http://vitals.com/ was visited by ~2 million unique visitors monthly, http://ucomparehealthcare.com/ was seen by ~1.5 million, whereas http://ratemds.com/ and http://zocdoc.com/ were both seen monthly by ~400,000 unique visitors (see Figure 1). The remaining 14 dedicated PEWs received considerably fewer unique visitors per month during this same time period (see Figure 2).
(Enlarge Image)
Figure 1.
Data from compete.com of the estimated average number of unique visitors per month to the five most visited websites dedicated to physician evaluation alone from October 2011 to September 2012.
(Enlarge Image)
Figure 2.
Data from compete.com of the estimated average number of unique visitors per month to the bottom 13 websites dedicated to physician evaluation alone from October 2011 to September 2012.
It is difficult to compare the number of visitors of dedicated PEWs with those of general and health information websites. As the general information sites offer much more content than simply physician profiles and evaluations, they generate considerably more traffic than PEWs that are dedicated to physician evaluation alone (see Figure 3).
(Enlarge Image)
Figure 3.
Data from compete.com of the estimated average number of unique visitors per month to the dedicated physician evaluation websites (PEWs), health information PEWs, and general information PEWs from October 2011 to September 2012. Health and general information sites typically receive more web traffic than dedicated PEWs. However, one can appreciate the tremendous variation among websites within each group as indicated by the bars showing the s.d.
The majority of PEWs studied (71%, 25/35) were found to have a net loss in their number of unique visitors over the past year. However, five dedicated PEWs websites, http://docspot.com/, http://doctorscorecard.com/, http://findadoc.com/, http://ucomparehealthcare.com/, http://vitals.com/, and http://yourcity.md/ were found to have a consistent growth in their number of unique visitors over the previous 12, 6, and 3 months. Likewise, two health and two general information PEWs (http://everydayhealth.com/, http://healthtap.com/, http://checkbook.org/, and http://merchantcircle.com/) can claim consistently positive growth in the number of unique visitors over the previous year (see Figure 4). Of all PEWs studied, http://yourcity.md/ can boast the most robust growth from October 2011 to September 2012.
(Enlarge Image)
Figure 4.
Physician evaluation websites (PEWs) with consistently positive growth in their number of unique visitors per month from October 2011 to September 2012. Yourcity.md can boast the strongest growth among all PEWs that were included in this study.
The websites were then analyzed using data from a different web analytics company, http://alexa.com/, which reports data on website user demographics. Although the majority of PEW users hail from the United States, several of these sites are visited by a large number of international users as well. As Figure 5 demonstrates, 13 out of the 35 PEWs studied are visited by more than 30% international users. A closer look at the user demographics of http://ratemds.com/ reveals that a substantial number of its users come from countries outside the United States where English is either the primary language or at least a common one (see Figure 6).
According to http://alexa.com/, most users of dedicated PEWs are between the ages of 45 and 64 years, they are usually female, possess at least some college education, and they tend to conduct their searches while at work. Figure 7 shows representative demographic data for http://healthgrades.com/, the largest dedicated PEW.
(Enlarge Image)
Figure 5.
Per alexa.com, more than 30 % of monthly visitors to 37 % of PEWs (13 / 15) are from outside the United States.
(Enlarge Image)
Figure 6.
Per alexa.com, 44 % of the monthly visitors of ratemds.com hail from outside the United States. Canada is the next greatest consumer of this PEW's services followed by India, Pakistan, and Australia (all countries where English is either principally or at least commonly spoken).
(Enlarge Image)
Figure 7.
Snapshot of user demographics report for healthgrades.com generated by alexa.com. Healthgrades.com is the most frequently visited website dedicated to physician evaluation. There are no numerical values for this graphic. Bars demonstrate whether a user is more (green) or less (red) likely to fall within a given category when compared with all of the other users of the Internet. Increased length of a bar indicates increased likelihood.
Methods
Websites were identified via a Google search using the following keywords: "doctor evaluation websites," "doctor evaluation," "rate doctors," and "find doctors" in addition to searches under the names of specific physicians. Other sites were found during a literature review of the topic. This yielded 58 total websites. Of these, 23 were eliminated based on the following exclusion criteria: the site did not contain physician evaluations, the site was managed by an insurance company, or the site did not cover all physician specialties. The remaining 35 sites were included in the analysis.
All websites included in this study were first independently reviewed by two investigators (JM and BL) and sorted into the following broad categories:
Sites Dedicated to Physician (or other Health Professional) Evaluation Alone
Health Information Sites with Physician Evaluation Features
General Information Sites with Physician Evaluation Features
The final lists of these sites are available in Table 1. Each website was analyzed with a series of checklists to assess its general characteristics ( Table 2), physician profiles ( Table 3), evaluation features ( Table 4), and web traffic data ( Table 5). Additional information was obtained by contacting the website administrators directly with a questionnaire.
General Site Characteristics
Geographics No website in our study was found to have an overt geographic focus. Some newer sites, such as http://docspot.com/ and http://zocdoc.com/, focus on large metropolitan regions in the United States. Per http://docspot.com/ website administrators, this merely reflects the distribution of the US population, and coverage is expected to extend into all parts of the country with time. In a beta search for "gastroenterologists in the region of Norfolk, Virginia," several sites were found to have minimal to no coverage of the trial area. Four dedicated PEWs (http://healthcarereviews.com/, http://ratemds.com/, http://findadoc.com/, and http://doctorescorecard.com/), enable searches for physicians outside the United States.
Fee for access None of the dedicated PEWs or health information sites charge to access or post reviews. Three general information sites with physician evaluation features (http://angieslist.com/, http://checkbook.org/, and http://merchantcircle.com/) do require payment for access to their services. These fees enable the site administrators to finance their websites and serve as a barrier from frivolous, anonymous posts. All websites studied participate in advertising as a means of generating revenue.
Medical education Medical education is not a primary feature on most dedicated PEWs. The most noteworthy exception to this is http://healthgrades.com/, which integrates patient education throughout each physician profile. Two additional sites, http://vitals.com/ and http://drscore.com/, provide links to sites of interest and to relevant support groups from physician profiles.
Internal website awards Three websites (http://healthgrades.com/, http://yourcity.md/, and http://zocdoc.com/) have developed unique awards to recognize exceptional physicians. The metrics for receiving these awards is not readily apparent and varies with each site. http://vitals.com/ and http://drscore.com/ notify users if the physician of interest is a Castle Connolly Top Doctor. http://drscorecard.com/ and http://findadoc.com/ enable physicians to give online endorsements of their colleagues.
Anonymous posting The majority of PEWs allows users to post reviews of physicians anonymously—provided they supply an accompanying email address. This feature may concern healthcare workers who fear the lack of accountability that this grants to reviewers. However, most websites have mechanisms in place to prevent a given physician from being reviewed multiple times by the same patient. This also serves to discourage the use of the third-party groups that might be employed to improve a physician's reputation by fabricating reviews.
Physician responses The question of physicians being able to respond to patient's concerns on their profiles is a vital one. Unfortunately, this is not a feature that has been widely implemented in PEWs to date. Only five websites (http://docspot.com/, http://doctorscorecard.com/, http://healthgrades.com/, http://ratemds.com/, and http://yourcity.md/) have overt mechanisms in place to enable doctors to respond to dissatisfied patients. It is possible that other websites also enable physician responses, but which were not readily apparent when reviewed for this paper.
Enhancements to physician profiles Several sites, including http://findadoc.com/ and http://doctorsdig.com/ offer physicians enhanced memberships for a fee, and http://healthgrades.com/ allows hospitals and practices to promote their physicians on its website. These fees grant physicians additional opportunities for advertising while generating revenue for the PEW. Although most sites do not have "featured physician profiles," the majority of them do encourage doctors to claim their own profiles for free and to make direct modifications/enhancements to them. These enhancements include adding a photo, updating listed information, and posting supplementary information. When profile information is incorrect and a physician cannot claim his/her profile, the information can usually be amended by contacting the website's administrators directly.
Physician profile characteristics On the basis of the results from our established checklists, unclaimed profiles on dedicated PEWs contain an average of 7.2±3.0 independent data points per physician (such as doctor name, specialty, age, etc.). Advanced or "claimed" profiles generally offer a more complete picture of a given physician with an average of 9.7±4.3 data points. Interestingly, health information websites with physician evaluation features tend to have more robust physician profiles (average of 9.6±2.7 or 10.2±3.1 for unclaimed or claimed profiles, respectively) than do websites dedicated to physician evaluation alone. General information websites that also participate in physician evaluation tend to have the least complete profiles with 5.4±3.0 (unclaimed) or 8.3±3.3 (claimed) data points per profile.
The most common data points listed in a given physician's profile are: name, specialty, and office address (100%, 33/33) followed closely by office phone/fax number (94% 31/33). Other common components are physician education/training, gender, and photo (48.5% each, 16/33), hospital/practice affiliation (45.5%, 15/33), accepted health insurance plans (39.3%, 13/33), licenses/certifications (36.3%, 12/33), years of experience (30.3%, 10/33), and languages spoken (27.2%, 9/33). More rare components of a profile include procedural expertise, research/clinical interests, availability, office hours, awards/recognition, malpractice history, and links to the physician's website (all less than 25%). Among all websites studied, http://healthgrades.com/, http://docspot.com/, and http://healthtap.com/ offer the most thorough unclaimed profiles with 13 data points. http://bookofdoctors.com/ offers the most complete claimed profile with 19 data points.
Review structure Most PEWs have similar features for physician review. Reviews typically start with the user assigning the provider an overall score (usually depicted by a number of stars). This is followed by questions regarding that physician's punctuality, bedside manner, time spent with the patient, etc. The user rates the doctor with stars on each of these qualities before being offered a chance for free response.
From our analysis, the average number of evaluation points for a dedicated PEW is 6.5±3.0, whereas health information PEWs invite users to score doctors on an average of 4.4±3.4 different values. General information websites that participate in physician evaluation tend to only require a basic "overall" score followed by free response. Of these websites, only 1.3±1 assessment points are usually required. Among all the PEWs studied, http://yourcity.md/ contains the most detailed reviews by requiring users to score physicians on 12 different values.
Across all PEWs with scaled reviews, the most common value for physicians is their "overall" score (92%, 23/25). This is followed by punctuality (52%, 13/25), quality of staff (48%, 12/25), and knowledge or ability to determine a correct diagnosis (40%, 10/25). Additional values include physician availability (32%, 8/25), office cleanliness and doctor bedside manner (both 28%, 7/25), time spent with the patient (24%, 6/25), follow-up after an appointment, and whether or not the patient would recommend a given physician to someone else (both 20%, 5/25). Less commonly reported values (less than 20%) include costs/billing, how well the physician educated and listened to the patient, whether or not the doctor was approachable, and the location of his/her office.
Site-usage informationhttp://compete.com/ (a World-Wide-Web analytics company) was used to evaluate each website's monthly traffic—in number of unique visitors—for the previous year. In the year leading to this study, http://healthgrades.com/ was the most frequently visited PEW dedicated to physician evaluation alone. From October 2011 to September 2012, http://healthgrades.com/ received an estimated 5.5 million unique visitors per month. http://vitals.com/ was visited by ~2 million unique visitors monthly, http://ucomparehealthcare.com/ was seen by ~1.5 million, whereas http://ratemds.com/ and http://zocdoc.com/ were both seen monthly by ~400,000 unique visitors (see Figure 1). The remaining 14 dedicated PEWs received considerably fewer unique visitors per month during this same time period (see Figure 2).
(Enlarge Image)
Figure 1.
Data from compete.com of the estimated average number of unique visitors per month to the five most visited websites dedicated to physician evaluation alone from October 2011 to September 2012.
(Enlarge Image)
Figure 2.
Data from compete.com of the estimated average number of unique visitors per month to the bottom 13 websites dedicated to physician evaluation alone from October 2011 to September 2012.
It is difficult to compare the number of visitors of dedicated PEWs with those of general and health information websites. As the general information sites offer much more content than simply physician profiles and evaluations, they generate considerably more traffic than PEWs that are dedicated to physician evaluation alone (see Figure 3).
(Enlarge Image)
Figure 3.
Data from compete.com of the estimated average number of unique visitors per month to the dedicated physician evaluation websites (PEWs), health information PEWs, and general information PEWs from October 2011 to September 2012. Health and general information sites typically receive more web traffic than dedicated PEWs. However, one can appreciate the tremendous variation among websites within each group as indicated by the bars showing the s.d.
The majority of PEWs studied (71%, 25/35) were found to have a net loss in their number of unique visitors over the past year. However, five dedicated PEWs websites, http://docspot.com/, http://doctorscorecard.com/, http://findadoc.com/, http://ucomparehealthcare.com/, http://vitals.com/, and http://yourcity.md/ were found to have a consistent growth in their number of unique visitors over the previous 12, 6, and 3 months. Likewise, two health and two general information PEWs (http://everydayhealth.com/, http://healthtap.com/, http://checkbook.org/, and http://merchantcircle.com/) can claim consistently positive growth in the number of unique visitors over the previous year (see Figure 4). Of all PEWs studied, http://yourcity.md/ can boast the most robust growth from October 2011 to September 2012.
(Enlarge Image)
Figure 4.
Physician evaluation websites (PEWs) with consistently positive growth in their number of unique visitors per month from October 2011 to September 2012. Yourcity.md can boast the strongest growth among all PEWs that were included in this study.
The websites were then analyzed using data from a different web analytics company, http://alexa.com/, which reports data on website user demographics. Although the majority of PEW users hail from the United States, several of these sites are visited by a large number of international users as well. As Figure 5 demonstrates, 13 out of the 35 PEWs studied are visited by more than 30% international users. A closer look at the user demographics of http://ratemds.com/ reveals that a substantial number of its users come from countries outside the United States where English is either the primary language or at least a common one (see Figure 6).
According to http://alexa.com/, most users of dedicated PEWs are between the ages of 45 and 64 years, they are usually female, possess at least some college education, and they tend to conduct their searches while at work. Figure 7 shows representative demographic data for http://healthgrades.com/, the largest dedicated PEW.
(Enlarge Image)
Figure 5.
Per alexa.com, more than 30 % of monthly visitors to 37 % of PEWs (13 / 15) are from outside the United States.
(Enlarge Image)
Figure 6.
Per alexa.com, 44 % of the monthly visitors of ratemds.com hail from outside the United States. Canada is the next greatest consumer of this PEW's services followed by India, Pakistan, and Australia (all countries where English is either principally or at least commonly spoken).
(Enlarge Image)
Figure 7.
Snapshot of user demographics report for healthgrades.com generated by alexa.com. Healthgrades.com is the most frequently visited website dedicated to physician evaluation. There are no numerical values for this graphic. Bars demonstrate whether a user is more (green) or less (red) likely to fall within a given category when compared with all of the other users of the Internet. Increased length of a bar indicates increased likelihood.
Source...