AI-PPT Tutor

While using a general-purpose AI assistant to study Java, I kept running into two frustrations. I would ask for beginner-friendly explanations with concrete code examples, but that teaching style could drift as the conversation grew. Explanations could also jump ahead of the slides, making it difficult to follow the original lecture. Repeating my instructions was becoming part of the work of learning.

That experience became AI-PPT Tutor, which I co-created with Luocheng Xie. We built a workspace around the course material itself: keep the learning context close at hand, offer explicit learning modes, and let a student inspect the source behind an explanation.

The product accepts PDF and PPTX files up to 20 MB. Its adaptive OCR pipeline uses native text when it is usable, applies Tesseract.js to pages that need recognition, and offers full-page OCR for scanned materials. Native and recognised text are merged and deduplicated, with recognition confidence carried into the evidence pipeline. This avoids treating a clean digital slide and a scanned page as the same input.

Charts, tables, diagrams, and code screenshots also carry teaching content. The pipeline uses Canvas-based page processing and qwen3-vl-plus visual analysis, retaining page numbers, crop coordinates, summaries, and thumbnails. File-type checks, signature validation, parsing timeouts, and content limits put boundaries around the ingestion process.

Engineering focus: preserve evidence quality and location before generating an answer.

We combine BM25 and embedding retrieval, multi-query expansion, Reciprocal Rank Fusion, MMR deduplication, and reranking over structure-aware chunks. Titles, definitions, lists, tables, and code remain meaningful units; location, neighbouring pages, OCR confidence, and visual reliability inform ranking. For a whole-document explanation, ordered coverage matters more than returning only a few locally relevant passages.

The interface makes this work inspectable through claim-level citations, highlighted passages, and source navigation. Unsupported requests can trigger partial or complete refusal. Tutor, Explain, Q&A, Quiz, and Review are different ways into the same material, rather than disconnected chat sessions.

A useful learning tool also needs to handle interruptions. IndexedDB preserves workspaces, evidence, and conversations; SHA-256 document fingerprints avoid repeated processing; signed checkpoints support resuming interrupted jobs. Progress reporting, cancellation, and retry controls keep long document-processing tasks understandable. If embeddings or reranking are unavailable, lexical retrieval provides a fallback. Storage is local, but AI processing still uses configured server-side services.

Alongside co-developing the product, I led its research study as first author. I personally hosted the 36-participant workflow and handled ethics materials, UEQ and S-TIAS measures, manipulation checks, interviews, quizzes, and factorial analysis. To separate grounding from visible attribution, our 2×2 experiment used a fixed, manually verified evidence index—not the later product retrieval stack.

The study found a significant grounding × attribution interaction in perceived trust, but no significant effects on user experience or learning performance. The manuscript was submitted to IHCI 2026, with a decision pending in the August 2026 record. The project brings together two kinds of work I want to keep pursuing: building a usable system and testing what its design actually changes.