Using GenAI to Augment Student Learning
How we adapted Chain of Density (CoD) prompting to teaching reading
Sorry for the long silence. The summer was so busy for Eugenia Novokshanova and I that we barely lifted our heads from our computers. As soon as the semester was over, we dug into our research findings from teaching for the entire year with GenAI (more on that in a later post), writing academic papers, writing a book, and making some tweaks to the courses we taught (because they are NEVER perfect, am I right?).
We were also busy working to innovate further in using AI with our students. We became deeply interested in bettering our students’ reading with GenAI, and wrote an entire book chapter on the topic. I read a paper by Griffin Adams, Alex Fabbri, Faisal Ladhak, Eric Lehman, and Noemie Elhadad entitled “From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting.” The idea of “Chain of Density” is to first have GenAI to summarize a complex text in a set number of words, for example, summarizing the Constitution of the United States in ~200 words.
As you can imagine, GenAI does not do a very complete job. There are a lot of details left out, and there are a lot of important concepts that are never mentioned. However, using the concept of “Chain of Density,” the GenAI model is directed to repeat the summary five times, each time identifying a new “entity” (detail or concept) it left out of the original summary and including that new entity in the summary without increasing the word count. Adams et al. discovered that, “Summaries generated by CoD are more abstractive, exhibit more fusion, and have less of a lead bias than GPT-4 summaries generated by a vanilla prompt” (Adams et al.).
While Adams et al were concerned with improving the quality of output in GenAI summaries, our teachers’ minds saw something else: CoD provides a scaffolded understanding of the text at each iteration, gradually moving from a broad overview to a detailed analysis. This could be applied to an academic text, and the outcome would assist our students in two ways—first, they would understand what it takes to improve summary when working with texts (this is a perennial problem when we are working on Annotated Bibliography assignments). Second, this process helps students understand the text better—as the first summary is broad and just touches upon general ideas, and through five iterations includes a new concept or detail, then another. Reading through a chain of density output is like having a reading tutor next to you helping you understand each step in the understanding of a complex text. Here is an example of how that works when applied to the Constituion of the United States of America in Perplexity.ai:
Summary 1:
This article discusses the Constitution of the United States, a foundational document that outlines the framework of the federal government. It includes various articles and sections detailing the powers and responsibilities of the legislative, executive, and judicial branches. The Constitution also addresses the process for amendments, the supremacy of federal law, and the rights and duties of states and citizens. The document was signed by delegates from several states, including notable figures such as George Washington. The Constitution remains a vital part of American law and governance.
Missing Entities: Ratification; William Jackson; September 17, 1787
Summary 2:
The Constitution of the United States, a foundational document, outlines the federal government's framework, including legislative, executive, and judicial branches' powers and responsibilities. It addresses amendments, federal law supremacy, and states' and citizens' rights and duties. Ratified by nine states, it was signed by delegates, including George Washington, on September 17, 1787, with William Jackson as Secretary. The Constitution remains vital to American law and governance.
Missing Entities: "We the People"; House of Representatives; Senate
Summary 3:
The Constitution of the United States outlines the federal government's framework, including legislative, executive, and judicial branches' powers and responsibilities. It addresses amendments, federal law supremacy, and states' and citizens' rights and duties. Ratified by nine states, it was signed by delegates, including George Washington, on September 17, 1787, with William Jackson as Secretary. "We the People" preamble introduces it. The House of Representatives and Senate form the legislative branch. The Constitution remains vital to American law and governance.
Missing Entities: Supreme Court; Amendments; Oath of Office
Summary 4:
The Constitution of the United States outlines the federal government's framework, including legislative, executive, and judicial branches' powers and responsibilities. It addresses amendments, federal law supremacy, and states' and citizens' rights and duties. Ratified by nine states, it was signed by delegates, including George Washington, on September 17, 1787, with William Jackson as Secretary. "We the People" preamble introduces it. The House of Representatives and Senate form the legislative branch. The Supreme Court heads the judiciary. Amendments and the Oath of Office are included. The Constitution remains vital to American law and governance.
Missing Entities: Commander in Chief; Impeachment; Treason
Summary 5:
The Constitution of the United States outlines the federal government's framework, including legislative, executive, and judicial branches' powers and responsibilities. It addresses amendments, federal law supremacy, and states' and citizens' rights and duties. Ratified by nine states, it was signed by delegates, including George Washington, on September 17, 1787, with William Jackson as Secretary. "We the People" preamble introduces it. The House of Representatives and Senate form the legislative branch. The Supreme Court heads the judiciary. Amendments and the Oath of Office are included. The President is Commander in Chief, and provisions for impeachment and treason are detailed. The Constitution remains vital to American law and governance.
In each iteration of the summary, it includes more and more detail, filling in the gaps in comprehension. As students read through the Chain of Density summary, they read the “vanilla” output, then a left-out entity is announced. As they read, they see how the entity is integrated into that paragraph, building upon the previous summary with new information. As they continue to read, more “entities” are identified, and students witness how that new information is folded into the new, more condensed, summary. This serves two purposes in improving reading comprehension of complex texts:
The reiteration of information serves to remind and renew information the students are reading, five times.
Each iteration includes new information that has been clearly identified, then integrated into the paragraph. Each subsequent paragraph contains that information, as it adds something new to it.
As the students read, they are reinforcing basic information from the article while, simultaneously, adding more and more information. This helps the student discover more and more information about the reading without increasing the number of words they need to read. It keeps their interest, as it is fascinating to understand how the paragraph changes as you read, supporting the human capacity for curiosity and novelty.
Here is the modified prompt we used for our experiments:
Article: [Insert URL Here]
You will generate increasingly concise, entity-dense summaries of the above Article.
Repeat the following 2 steps 5 times.
Step 1. Identify 1-3 informative Entities ("; " delimited) from the Article which are missing from the previously generated summary.
Step 2. Write a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities.
A Missing Entity is:
Relevant: to the main story.
Specific: descriptive yet concise (5 words or fewer).
Novel: not in the previous summary.
Faithful: present in the Article.
Anywhere: located anywhere in the Article.
Guidelines:
The first summary should be long (5-7 sentences, ~80 words) yet highly non-specific, containing little information beyond the entities marked as missing. Use overly verbose language and fillers (e.g., "this article discusses") to reach ~100 words.
Make every word count: rewrite the previous summary to improve flow and make space for additional entities.
Make space with fusion, compression, and removal of uninformative phrases like "the article discusses."
The summaries should become highly dense and concise yet self-contained, e.g., easily understood without the Article.
Missing entities can appear anywhere in the new summary.
Never drop entities from the previous summary. If space cannot be made, add fewer new entities.
Remember, use the exact same number of words for each summary.
Adapted from Adams, Griffin, et al. “From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting.” Proceedings of the 4th New Frontiers in Summarization Workshop, edited by Yue Dong et al., Association for Computational Linguistics, 2023, pp. 68–74. ACLWeb, https://doi.org/10.18653/v1/2023.newsum-1.7.
Although we have seen GenAI used to “dumb down” a text so students could more easily understand it (i.e. “Please summarize this article on the 9th grade level), or “copilot” a text—providing an opportunity for readers to query the text with GenAI (i.e. Can you please explain this paragraph to me?)—we have never seen the seamless application of scaffolding a text like we saw in our Chain of Density experiments in our classes.
Our students reported that this exercise, although brief and somewhat effortless (it involves simply adding a prompt and the URL of their article into Perplexity), was extremely valuable in assisting their comprehension of complex academic texts during their research processes. According to one of my summer Composition II students:
The initial broad summary introduces the main topics and themes, while the detailed summary adds more specific details and meanings. This allows the reader to build understanding progressively rather than being overwhelmed with details upfront. —Mari
We will continue to work with CoD principles in our courses, refining the technique and prompts, as we build upon this initial success with our students, trying to find more and more innovative ways to assist our students in attaining the highest level of student success.