Project Name
Wrong Course-Level Results Fixed With Multi-Grain OpenSearch for an EdTech Platform
A large EdTech platform delivering video-based learning to universities and corporate training teams had a content library spanning thousands of modules, each with multiple chapters, sections, and assessments. AWS CloudSearch indexed all of it as flat text blobs, returning course-level matches when learners searched for section-level answers. Relevance degraded as the library grew and newly published content stayed invisible to search for hours. Applying its AI-First approach, Ksolves replaced CloudSearch with a multi-grain AWS OpenSearch index that treats every content level as a first-class searchable entity without requiring a platform rebuild.
- Hierarchical Content Trapped in a Flat Index: CloudSearch indexed all course content as undifferentiated text blobs with no way to distinguish a course title from a section subtitle. Every match returned the same document regardless of where in the hierarchy the match occurred.
- Section-Level Search Irrelevant by Design: Learners could not surface the exact video section containing their answer. Results pointed to entire courses, forcing manual navigation through hours of material and driving learners away from search entirely.
- No Path to Advanced Query Patterns: The CloudSearch architecture could not support multi-field weighted queries, field-level boosting, or fuzzy matching without a complete reindex and schema redesign.
- Relevance Degrading as Content Scaled: As the library grew, CloudSearch relevance scoring became unreliable because the flat index had no way to weight title matches differently from transcript matches.
- Rising Infrastructure Cost Without ROI: CloudSearch instance costs scaled with content volume while delivering progressively worse results as the library expanded.
- No Index Visibility or Debugging Tooling: When search results were wrong, there was no way to inspect the index state, diagnose mapping issues, or verify content updates had propagated. Search quality was a black box for the engineering team.
Ksolves migrated from AWS CloudSearch to AWS OpenSearch, rebuilding the index schema around a multi-grain content model that treats each content level (course, chapter, section, transcript segment) as a separately indexed and independently searchable entity. The governing principle: the index structure should mirror the content structure, not flatten it.
- Multi-Grain Index Architecture: New OpenSearch index schema with distinct document types and field mappings for courses, chapters, sections, and transcript segments. The search engine returns the most relevant content granule for any query rather than always surfacing the top-level course.
- Field-Level Boosting and Weighted Relevance: Query DSL field-level boost weights prioritise matches in titles and headings over matches in transcripts and descriptions. Learners see the most directly relevant content first, not content that merely mentions the term in passing.
- Incremental Reindex Pipeline: Incremental indexing pipeline propagates content updates to OpenSearch within minutes of a CMS change. Replaces the batch reindex jobs that caused hours-long lag between content publication and search visibility.
- OpenSearch Dashboards for Index Visibility: OpenSearch Dashboards configured as an internal observability tool, giving engineering and content teams direct visibility into index health, document counts, mapping state, and query performance.
Technology Stack
| Category | Technology |
|---|---|
| Architecture | AWS OpenSearch |
| Integration | Query DSL |
| Processing | Bulk API / Index Pipeline |
| Platform | OpenSearch Dashboards |
- Section-Level Search Results Enabled: Multi-grain index returns the most relevant content granule for every query. Queries that returned a four-hour course now return the specific five-minute section containing the answer.
- Search Relevance Improved Across the Library: Field-level boosting and weighted Query DSL mean title matches outrank transcript mentions. Results reflect content importance, not just keyword presence.
- Index Lag Eliminated: Incremental pipeline propagates content changes within minutes of publication. Newly published content is immediately visible to search; the hours-long batch lag is eliminated entirely.
- Full Index Visibility for Engineering Teams: OpenSearch Dashboards gives direct visibility into index health, document counts, and query performance. Black-box search debugging replaced with a real observability layer.
“Learners were telling us search was broken. It was not broken, it just could not find what they were looking for because the index had no concept of content depth. Once we migrated, the same queries that used to return a four-hour course now return the specific five-minute section. That is the difference between a tool learners use and one they avoid.”
-VP Engineering or Head of Platform Architecture.
A large EdTech platform where CloudSearch returned four-hour courses when learners searched for five-minute answers, with hours-long index lag and no debugging visibility, was transformed through Ksolves Big Data and Engineering services. A multi-grain AWS OpenSearch index now treats every content level as a first-class searchable entity. Section-level results surface directly. Field-level boosting ensures title matches outrank transcript mentions. Incremental indexing eliminates publication lag. OpenSearch Dashboards replaces black-box debugging with full index observability.
Is Your Search Infrastructure Returning the Wrong Level of Content for What Your Users Are Actually Looking For?
