By Jim Shimabukuro (assisted by Gemini)
Editor
For this survey, I asked seven chatbots — Gemini, DeepSeek, You.com, ChatGPT, Copilot, Perplexity, and Claude — to rank the same seven chatbots, including their own. I weighted each 1st selection as 7 points, each 2nd as 6, etc. Each chatbot received 7 rankings.
The runaway favorite for academic papers was Perplexity with 47 points (six 7s and one 5). The second and third were Claude with 33 and Gemini with 31. Here’s the complete rankings: (1st) Perplexity 47, (2nd) Claude 33, (3rd) Gemini 31, (4th) Copilot 23, (4th) You.com 23, (5th) ChatGPT 21, (6th) DeepSeek 18.
This is the prompt I presented to all chatbots: Assuming that all chatbots aren’t equal for academic research, please rank and compare the strengths and weaknesses of the following chatbot platforms on the basis of (1) Broad up-to-date source coverage, (2) accurate synthesis, (3) thematic analysis, and (4) proper citations: ChatGPT, Gemini, Claude, Copilot, Perplexity, DeepSeek, You.com.
The following summary of Perplexity’s strengths and weaknesses are from Gemini:
Strengths
SEARCH: Excels in real-time web search and transparent source attribution. Perplexity is designed as a search engine that “incorporates real-time web search capabilities. Perplexity Pro offers “Internal Knowledge Search,” allowing users to “simultaneously search across web content and internal documents” by uploading various file formats.
CITATIONS: Perplexity also includes citations in all responses, providing “numbered footnotes linking directly to original sources. Emphasizes a “transparent citation style.” A core feature of Perplexity is that all its responses include citations to their sources from the Internet, promoting transparency and allowing users to verify information. It is explicitly noted for offering “accurate source citations.” Users can also prompt it to place citations immediately after paraphrasing or direct quotes, specifying formats like APA 7.
UP-TO-DATE SOURCES: Its core function is to provide responses based on current Internet content. It explicitly utilizes real-time web indexing, ensuring responses are up-to-date.
RELIABLE SOURCES: Perplexity prioritizes trustworthiness by sourcing information from a “curated list of reputable sources” and relies on a more selective index than traditional search engines. Perplexity aims to synthesize content from top-tier sources and present concise, context-aware summaries.
ACCURACY: Its design philosophy is to increase factual accuracy and reduce model hallucinations by grounding responses in real-world information. It is noted for excelling in complex query handling, user interaction learning, and natural conversation. In a study comparing chatbot recommendations to clinical guidelines, Perplexity achieved the highest match rate (67%) among Copilot and Gemini , indicating a relatively better performance in factual consistency for specific domains.
Weaknesses
HALLUCINATIION: While Perplexity aims for transparency with numbered footnotes, one study indicated that it, along with Copilot and Claude, had “one of the highest hallucination rates” in bibliographic reference retrieval, with nearly 40% of references generated by chatbots being “erroneous or entirely fabricated.”
SEARCH ERRORS: Despite its design philosophy, the execution of its advanced search features can sometimes fall short, leading to inaccuracies. For academic research design, Perplexity has been noted to fall short on exploring methodologies, risks, and other aspects of research design.
LIMITED MULTIMODAL CAPABILITY: Its architecture is primarily text-based, with limited AI image generation capabilities, which can be restrictive for multimodal research workflows.
ACCURACY: While it may summarize general web content reasonably well, its accuracy for academic citations is highly questionable, which is a critical distinction for research.
THEMATIC ANALYSIS ABILITY UNKNOWN: While it offers features that could aid in organizing and exploring themes, its specific proficiency in nuanced thematic analysis is not well-documented in the available data.
Final Note: Before throwing your hands up and dismissing Perplexity because it hallucinates and makes errors in citations, keep in mind that its shortcomings may be based on one or a few user comments in social media. We don’t know for sure. Weigh this against the fact that it still rates the highest, by a wide margin, among the other chatbots surveyed in this article. The implication is that results could be worse with other chatbots.
I realize this isn’t very reassuring, but when you consider the efficiency of completing a preliminray review of literature for a paper in as long as it takes to compose a prompt and click submit, you’ll begin to appreciate what it can do. As a routine part of research writing, develop the habit of double-checking citations. Read Perplexity’s responses and click on the links to the original sources to check for accuracy.
Review your draft as you normally do. Hallucinations, if any, will jump out at you. It’ll be a statement that doesn’t make sense in the context of a sentence, paragraph, or the entire paper. In my experience, hallucinations are relatively rare. What’s more common as a problem is repetition of ideas, not just with Perplexity but with all chatbots. As with any draft, you’ll catch these ub your review and eliminate or revise them. -js
Filed under: Uncategorized |
Educational Technology and Change Journal 
Leave a comment