Content Aggregation Models
Mark J. Norton
Phillip Dodds
May 2, 2001
This document is based on a conversation between Mark and Phillip on May 2, 2001.
Existing systems are very brittle when it comes to organizing learning objects. If the objects are fine grained, they tend to be tightly coupled to competencies and learning objectives. Creating and maintaining such a system is very difficult. Two labels for the same competency are treated independently (in fact, both may end up being required). Authors and developers lack an understanding of how the system works and why meta-data such as competencies are what drives the system.
Given a respository of learning objects or activites which stand (more or less) on their own, how are they aggregated? Who will define the order and organization of their use? These and other questions are likely to shape how useful technology based learning specification are to the developers use them to create new learning applications. If content can be easily aggregated into new configrations, organizations, and presentations, it will foster the re-use of learning objects and ultimately the good design of standalone activities. It also defines one aspect of interoperatibility by describing how learning objects can be combined, either a priori or dynamically.
For the purposes of discussion here, a learning object is defined to be any self-contained activity which accomplishes some learning objective. It could include knowledge delivery, experiments, individual or group assignments, research projects, etc. No specific representation or limitations of learning objects are implied.
The rest of this paper examines some possible models for aggregating content. No attempt is made to place any value judgements (such as economic feasibility) at this time. It is an attempt to get some ideas out on the table with they can be poked and prodded.
Aggregation Models
Determined ordering
Author determined
Perscriptive based on assessment
Dependency based
Dynamic selection by system
User selected
Does order matter? - yes and no.
Order vs. organization
Determined Ordering
Any set of learning objects may have an ordering (or organization) imposed on them. This can either be done in a fixed, pre-determined manner, computed at certain, well-defined points in time (just after a test), or dynamically as the student progresses.
The historic pre-determined ordering is author-created. Here, an individual (or small group) determines what they think the best ordering for a set of learning objects is. This is usually based on experience and knowledge of the domain, but not always. In the days of CBT style learning, this a priori ordering was built into the courseware and (almost always) could not be changed.
It is also possible, however, to compute how learning objects should be organized. Perscriptive organization is based on the results of pre-learning assessment. The test is made against certain competencies and learning objectives. Those objectives which are determined to be misunderstood by the learner are added to a learning plan. Such a plan can also be created by remediation after one (or more) passes have been made through content material.
Computed ordering may also occur using dependency graphs. The user indicates that she wants to learning a particular thing. Using pre-define dependencies and prerequisites, the system creates a learning plan which includes all relevant material leading up to the selected objective. This list is pruned against previously mastered competencies, etc.
Beyond computed ordering is a more dynamic form of ordering based on continuous assessment of the user’s progress in understanding the material at hand. It is possible (though difficult) to analyze a student’s learning performance at a small enough level that ordering is re-computed at each step in the learning process. The system may choose to skip segments (implied mastery), it may offer remediation in areas which understanding is lacking, and it may suggest review of previously mastered material which has not be used or practiced.
In addition to pre-defined, or computed orderings, there is also user-selected orderings. In many cases, the student will know which order is best pursued, or it may be completely at the user’s discretion and choice. Interesting things can be learned from these choices, which is explored more below. User choice can be guided by any number of factors including experience with a particular author, recommendations by friends, quality assessment based on voting, etc.
Before we leave the topic of determined orderings, we need to examine two questions first. The first: does order matter? Naturally, the answer is that it depends on the material. It is necessary to know how to add before you can multiply. This is a strong dependency which needs to be represented and enforced. Other instances include learning sequences: here is how you assembly a bicycle. Order is necessary and needs to be explict. In many other cases, order doesn’t really matter at all. For example, in studying the capitals of Europe, Paris doesn’t need to come before Berlin.
The second question has to do with order vs. organization. Ordering implies a linear presentation of some kind, but there are other styles of learning. These include branching scenarios, browsing hyperlinks, random search, etc. Where ever the word ‘ordering’ is used above, the term ‘organization’ also applies.
Browsing Learning Objects
The browse model (hyperlinks)
One can conceive of a system in which learning objects stand on their own, but link to others as well. Instead of any kind of determined ordering of learning objects, the user is presented with a set of choices in each LO as to what should be viewed next. Persumably, these are based on drilling down for more detail, or references to related concepts.
This model has some obvious difficulties: stale or broken links, no refernences to new models, high maintenance costs, etc. Some of these can be avoided. For instance, if the system checks all links before displaying them, broken links can be suppressed as choices. Furthermore, lists of links can be dynamically computed and added to the LO based on some discovery technique. The leads to the question of what is relevant, however.
Parametric Search
Learning Objects can be collected into groups based on search queries. Searching can be conducted along several lines: reference to concepts (everything on tempering steel), author (everything by Carl Berger), date (all new objects), etc. Searches can be done on the content within the object, or on its meta-data (or both). Modern search engines also have a way of ranking the resulting matches so as to present the most likely choices first.
Concept Maps and Semantic Networks
Many attempts have been made to organize concepts and define its relationships to to other other concepts. Concept Maps and Semantic Networks both try to define these relationships by linking conceptual nodes with a labeled arc. These relationships can be created either via a human acting as a knowledge manager, or through various analysis techniques (semantic analsysis). Needless to say, these techniques are not perfect. Links may go missing, or links created which do not make sense. In spite of that, once the networks are created, unique methods of sequencing become available to the user.
The first has to do with shallow vs. deep learning. Depth of knowledge can be represented in the semantic net as fanout and distance parameters. This given some control over how much detail is presented to the user. It also gives a way of semantically grouping conceptual material together, thus providing aggregation along the lines of similar knowledge.
The second allows the user to indicate that “I am here (conceptuall)” and “I want to be over there”. What does it take to go from here to there in terms of learning? Multiple paths may exist to which heuristics can be applied (minimum distance, learning time, personal choice, etc.)
Predictive Learning
One interesting way to group content or provide an ordering is to keep track of the choices users have made in the past. Given a random set of learning objects, users will tend to choose a particular objects at the next one. If this is recorded, the information can be used to predict the next one likely to be requested and present it as the first choice.
Some web search engines make use of this technique right now, ordering search results based on the number of times users have clicked on them in the past. It is largely based on the theory that people tend to think alike and make similar choices when faced with similar decision situations.
The Competitive Model
Not to be out done by the previous paragraphs for wild ideas, consider a learning object that has the ability to compete for the attention of a user. Each object become a kind of agent competing against all other agents to be viewed and used. The agent has a large vested interest in gathering attention to itself since objects which are not used, get removed from the knowledge pool. In this survival of the fittest scenario, each object would have a short description, or perhaps a flashy graphic animation which can be used (when permitted) to attract attention. To this, we add the ability to mutate its display content (without affecting veracity) and we get a system which genetically migrates towards perfect learning situations. Incidentally, I have a bridge I’d like to sell you, also.
Other Notes
Accessibility will likely play a role in extracting information from a learning object to initialize its meta-data. Images are not accessible and need text descriptions.
Learning Object meta-data will always be imperfect. Authors will not enter it properly or at all in many cases. Meta-data needs to be defaulted at creation time (encouraging correction). It needs to be augmented with information that can be extracted from the object itself. It needs to be user-annotatable.