Benchmark · Updated October 23, 2026

We tested 9 AI study apps on real college STEM problems. Here's how often each one got it right.

Students paste homework into AI study apps every day and trust what comes back. We wanted to know how often that answer is right, so we gave 9 apps the same 89 hard, multi-step problems from Chemistry, Statics, Dynamics, Fluid Mechanics and Thermodynamics, and checked every final answer against a worked key.

This is a live benchmark. We're still adding problems and subjects (heat transfer, general physics and calculus are next), and every chart and table on these pages updates as new results come in.

The results at a glance

Average accuracy across all 5 subjects, with every subject weighted equally. A score of 100% means every part of every problem was within tolerance of the answer key.

Results by subject

Each cell is the app's accuracy in that subject. Click a subject for every problem, every app's answer, and why the wrong ones were wrong.

Why we ran this test

AI study apps promise to "solve any homework problem" and explain it step by step. For a short algebra question, that's usually true. University STEM coursework is different: a single thermodynamics or fluid-mechanics problem can take a property-table lookup, three coupled equations and an iteration, and one slip early on silently ruins every number after it. The explanation still reads confidently, so a student has no easy way to tell a right answer from a wrong one.

We couldn't find a public, repeatable comparison of these apps on that kind of problem, so we built one. The question is narrow on purpose: if a college student pastes a hard STEM problem into this app, how often is the final answer correct? It says nothing about flashcards, note-taking or anything else these apps do.

How we tested

• The apps: Quizlet, Knowt, StudyFetch, Turbo AI, Solvely, NotebookLM, ChatGPT (GPT-6, high reasoning effort), Claude (Claude Opus 5.5, medium effort), MyStudySidekick. Each was used through its normal student interface. For the general-purpose assistants we recorded the model and setting used: ChatGPT ran GPT-6, high reasoning effort; Claude ran Claude Opus 5.5, medium effort.
• The problems: upper first-year to second-year university level, all multi-step. Chemistry problems are original; thermodynamics and fluid mechanics problems are adapted from standard textbooks with new numbers and wording; statics and dynamics use published textbook problems, given to each app as a photo of the printed problem, exactly as a student would.
• Same input for everyone: the exact same text, plus the same figure where one is needed, pasted into a fresh chat. One attempt, no "try again", no hints or follow-up messages.
• Grading: each part of each problem has a worked numeric answer key and a tolerance (usually ±2%). A part is right if the app's final answer falls inside it. Problems with several parts score the share of parts right. If an app gave no answer, refused, or could not accept the problem's image, that problem scores 0, because the student got nothing usable.
• Fixed in advance: problem sets and keys were frozen before any app was run on them. One fluid-mechanics problem was later found ambiguous and voided for every app; it is still listed on that page with the reason.
• No looking up the answer: from fluid mechanics problem 5 on, every app (ours included) received the same extra line asking it to solve the problem itself and go online only for property values.

Disclosure and how we kept it fair

We build MyStudySidekick, and it is one of the nine apps here. You should weigh that. These are the safeguards:

• Problems and answer keys were written and frozen before any app, including ours, saw them.
• MyStudySidekick was tested on the same public version students use, through the same steps as every other app.
• When a benchmark problem showed us a weakness in MyStudySidekick, we fixed it using separate practice problems, shipped the fix to the public app, and re-ran it. Every re-run is labelled on its subject page with the score of the first attempt.
• Grading is mechanical: a final number inside or outside the key's tolerance.
• Every answer from every app is published on the subject pages, including where MyStudySidekick lost (in dynamics, Claude scored higher).

What's next

We'll keep updating this benchmark: more problems in every subject, new subjects (heat transfer, general physics, algebra and calculus), and re-runs as the apps release new versions. Each update refreshes the results on this page and on every subject page.

Frequently asked questions

Which AI study app is most accurate on college STEM problems?

In our 89-problem benchmark across 5 subjects, MyStudySidekick had the highest average accuracy (98%). ChatGPT and Claude were close behind. The team that runs the benchmark also builds MyStudySidekick (see the disclosure on this page).

Are AI homework apps like Quizlet, Knowt and StudyFetch accurate for engineering problems?

Not on the hard, multi-step problems we tested. Quizlet averaged 30%, StudyFetch 36% and Knowt 3%. Some of these apps also could not read problems that come with a diagram. They are mainly flashcard and note tools; this benchmark only measures problem solving.

How was the benchmark run?

Every app received the exact same problem text (and the same figure when there was one) in a fresh session, with one attempt and no follow-up hints. Each final answer was compared with a worked answer key, usually within ±2%. Answers an app could not give count as zero.

Can I see every answer each app gave?

Yes. Each subject page lists every problem, the answer key, the final answer every app gave, whether it was marked correct, and a short note on why wrong answers were wrong.