The shift towards online education has been monumental, offering flexibility and a wide reach. However, for educators like you, managing the vast content from online lectures, be it on Zoom, WebEx, or other platforms, can be challenging. This guide provides a detailed walkthrough on how to create an automated transcription and summarization tool using OpenAI’s Whisper and GPT-4, specifically tailored for online courses. This tool will transcribe lecture recordings, summarize content, highlight key concepts, and even analyze the overall sentiment of the lecture.

The Strength of Whisper and GPT-4 in Education

Whisper excels in transcribing spoken language to text, making it ideal for converting lecture recordings into written form. GPT-4, a leading language model, will then process this text to extract summaries, key points, and perform sentiment analysis. This combination is not just a time-saver, but a way to enhance student engagement and comprehension.

Setting Up

As a professor, you might already be familiar with Zoom. To get started, you would obviously need to record your class and then log in to Zoom and download the audio of the course:

From here, upload the audio file to Google Colab. You’ll need Python and an OpenAI API key for this application. Install the necessary python-docx and OpenAI libraries in a Python environment:

!pip install --upgrade openai
!pip install python-docx
import os

# Prompt for the API key
api_key = input("Enter your OpenAI API key: ")

# Set the environment variable
os.environ["OPENAI_API_KEY"] = api_key
from openai import OpenAI

client = OpenAI()

Breaking the File into Chunks

OpenAI API can only accept files of up to a maximum size. This method ensures that the file sizes of the audio is small enough to be processed. This is a restriction of the Whisper model/api.

from pydub import AudioSegment
# There is a size limit that OpenAI has so we need to make sure to break down the file to smaller chunks
import os

def split_audio(file_path, max_chunk_size_mb=24):
 audio = AudioSegment.from_file(file_path)
 chunks = []
 chunk_length_ms = 600000 # Starting with 10 minutes
 max_chunk_size_bytes = max_chunk_size_mb * 1024 * 1024 # Convert MB to bytes

 while chunk_length_ms > 0:
 start = 0
 while start < len(audio):
 end = start + chunk_length_ms
 chunk = audio[start:end]
 with open("temp_chunk.wav", "wb") as temp_file:
 chunk.export(temp_file, format="wav")
 if os.path.getsize(temp_file.name) <= max_chunk_size_bytes:
 chunks.append(chunk)
 start = end
 else:
 break
 if start < len(audio):
 # If chunk size is too large, reduce the time interval
 chunk_length_ms = int(chunk_length_ms / 2)
 else:
 break

 return chunks

Transcribing Lecture Recordings

Let’s adapt the transcription process for online lectures. Here’s the modified function:

from openai import OpenAI
from docx import Document

client = OpenAI() # Assumes API key is set in environment variables

def transcribe_lecture(audio_file_path):
 audio_chunks = split_audio(audio_file_path)
 transcriptions = []

 for chunk in audio_chunks:
 with open("temp_chunk.wav", "wb") as temp_file:
 chunk.export(temp_file, format="wav")
 with open(temp_file.name, 'rb') as audio_file:
 transcription = client.audio.transcriptions.create(
 model="whisper-1", 
 file=audio_file
 )
 # Adjust the following line according to the actual structure of the transcription object
 transcription_text = transcription.text if hasattr(transcription, 'text') else ""
 transcriptions.append(transcription_text)

 return " ".join(transcriptions)

This function takes the path to your lecture recording and uses Whisper to transcribe it. Note: Ensure the audio file is accessible on the device running the script.

Processing the Transcript with GPT-4

Next, we’ll use GPT-4 to analyze the lecture transcript, extracting summaries, key concepts, and sentiment. Before we do that, there is a limit on the number of tokens that can be handled so we have to take the output of the Whisper call and make sure it is chunked appropriately also:

def chunk_text(text, max_length=8000):
 words = text.split()
 chunks = []
 current_chunk = []
 current_length = 0

 for word in words:
 if current_length + len(word) > max_length:
 chunks.append(" ".join(current_chunk))
 current_chunk = [word]
 current_length = len(word)
 else:
 current_chunk.append(word)
 current_length += len(word) + 1 # Add 1 for the space

 if current_chunk:
 chunks.append(" ".join(current_chunk))

 return chunks

1. Lecture Summary Extraction

def lecture_summary_extraction(transcription):
 chunks = chunk_text(transcription)
 summaries = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Summarize this text."}, {"role": "user", "content": chunk}]
 )
 # Adjust the following line according to the actual structure of the response object
 summary_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 summaries.append(summary_text)

 return " ".join(summaries)

This function prompts GPT-4 to create a concise summary of your lecture, capturing the main points in a digestible format for students.

2. Key Concepts Extraction

def key_concepts_extraction(transcription):
 chunks = chunk_text(transcription)
 key_concepts_list = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Identify and list key concepts from this text."}, {"role": "user", "content": chunk}]
 )
 key_concepts_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 key_concepts_list.append(key_concepts_text)

 return " ".join(key_concepts_list)

GPT-4 identifies and lists the key concepts discussed, aiding students in focusing on the most critical aspects of the lecture.

3. Sentiment Analysis

def lecture_sentiment_analysis(transcription):
 chunks = chunk_text(transcription)
 sentiment_summaries = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Analyze the overall sentiment of this text."}, {"role": "user", "content": chunk}]
 )
 sentiment_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 sentiment_summaries.append(sentiment_text)

 return " ".join(sentiment_summaries)

This function evaluates the tone of the lecture, providing insights into the emotional context and delivery style.

Exporting Lecture Insights

The final step involves compiling these insights into an organized format:

def save_as_docx(lecture_notes, filename):
 doc = Document()
 for key, value in lecture_notes.items():
 doc.add_heading(' '.join(word.capitalize() for word in key.split('_')), level=1)
 doc.add_paragraph(value)
 doc.add_paragraph()
 doc.save(filename)

This function transforms the GPT-4 generated content into a Word document, suitable for sharing with students.

Executing the Process

To utilize this tool for your online lectures, use the following script:

lecture_recording_path = "/content/GMT20221025-215610_Recording.m4a"
transcription = transcribe_lecture(lecture_recording_path)
lecture_notes = {
 'summary': lecture_summary_extraction(transcription),
 'key_concepts': key_concepts_extraction(transcription),
 'sentiment': lecture_sentiment_analysis(transcription)
}
print(lecture_notes)

save_as_docx(lecture_notes, 'lecture_notes.docx')

This script transcribes your lecture recording, processes it through GPT-4, and saves the insights as a Word document.

Conclusion

This AI-powered tool revolutionizes the way online lectures are managed, offering a unique way to enhance student engagement and understanding. It not only saves time but also ensures the key elements of each lecture are captured succinctly and accurately. Embracing this technology in your teaching practice opens doors to a more efficient and impactful educational experience.

Addendum: Enhancing Lecture Analysis with Advanced GPT-4 Methods

In this updated addendum, we introduce four additional use cases relevant to online education, leveraging GPT-4’s capabilities for processing large lecture transcriptions. These methods address extracting action items, handling student questions, linking concepts to external resources, and providing historical contextualization. Each method has been updated to include chunking of large texts to avoid exceeding GPT-4’s token limits.

1. Action Items Extraction

This method identifies action items or assignments mentioned in the lecture, helping students understand their responsibilities.

def action_items_extraction(transcription):
 chunks = chunk_text(transcription)
 action_items_list = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Identify any action items or assignments given in this lecture."}, {"role": "user", "content": chunk}]
 )
 action_items_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 action_items_list.append(action_items_text)

 return " ".join(action_items_list)

2. Addressing Student Questions

Processes student questions from a lecture Q&A session, providing clear and concise answers.

def student_questions_handling(transcription, questions):
 chunks = chunk_text(transcription)
 responses = {}

 for question in questions:
 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Provide a clear and concise answer to this student question based on the lecture content."}, {"role": "user", "content": question}]
 )
 responses = response.choices[0].message.content if hasattr(response, 'choices') else ""

 return responses

3. Concept Linking

Identifies key concepts from the lecture and links them to relevant external educational resources.

def concept_linking(transcription):
 chunks = chunk_text(transcription)
 concept_links_list = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Link key concepts from this lecture to external educational resources."}, {"role": "user", "content": chunk}]
 )
 concept_links_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 concept_links_list.append(concept_links_text)

 return " ".join(concept_links_list)

4. Historical Contextualization

Provides historical context or current relevance for topics discussed in the lecture.

def historical_contextualization(transcription):
 chunks = chunk_text(transcription)
 historical_context_list = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Give historical context or current relevance to the key topics discussed in the lecture."}, {"role": "user", "content": chunk}]
 )
 historical_context_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 historical_context_list.append(historical_context_text)

 return " ".join(historical_context_list)

Integrating All Methods

To incorporate these methods into your lecture analysis process:

# lecture_recording_path = "Lecture1.wav" --- it is probably already run
transcription = transcribe_lecture(lecture_recording_path)

lecture_notes = {
 'summary': lecture_summary_extraction(transcription),
 'key_concepts': key_concepts_extraction(transcription),
 'sentiment': lecture_sentiment_analysis(transcription),
 'action_items': action_items_extraction(transcription),
 'student_questions': student_questions_handling(transcription, ["Question 1", "Question 2"]),
 'concept_links': concept_linking(transcription),
 'historical_context': historical_contextualization(transcription)
}

print(lecture_notes)
save_as_docx(lecture_notes, 'enhanced_lecture_notes.docx')

This comprehensive script, now enhanced with chunking for large texts, covers a wide array of educational needs. It effectively processes extensive lecture content, providing valuable summaries, context, and resources, thus enriching the learning experience for students.

Addendum: Advanced Lecture Analysis with Additional GPT-4 Methods

Expanding on our existing toolkit, this enhanced addendum introduces three additional methods to deepen the analysis and utility of online lecture content. These new methods are designed to offer a richer and more interactive learning experience by identifying in-lecture questions, expanding lecture topics, and providing personalized learning suggestions. Each method now includes the necessary adjustments for handling large text chunks.

1. Lecture-Based Question Identification and Answering

This method searches the lecture transcript for any questions posed and provides answers based on the lecture content and supplementary information.

def identify_and_answer_lecture_questions(transcription):
 chunks = chunk_text(transcription)
 lecture_question_answers = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Identify any questions posed in the lecture and provide answers based on the lecture content and related information."}, {"role": "user", "content": chunk}]
 )
 question_answer_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 lecture_question_answers.append(question_answer_text)

 return " ".join(lecture_question_answers)

2. Lecture Topic Expansion

Aims to provide students with a broader perspective on key lecture topics, adding additional information, examples, or perspectives.

def expand_lecture_topics(transcription):
 chunks = chunk_text(transcription)
 topic_expansions = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Expand on key topics of the lecture with additional information, examples, or perspectives."}, {"role": "user", "content": chunk}]
 )
 topic_expansion_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 topic_expansions.append(topic_expansion_text)

 return " ".join(topic_expansions)

3. Personalized Learning Suggestions

Analyzes the lecture content to suggest personalized learning activities or materials tailored to different student learning styles and needs.

def personalized_learning_suggestions(transcription):
 chunks = chunk_text(transcription)
 learning_suggestions_list = []

 for chunk in chunks:
 response = client.chat.completions.create(
 model="gpt-4",
 messages=[{"role": "system", "content": "Based on the lecture content, suggest personalized learning activities or materials for students."}, {"role": "user", "content": chunk}]
 )
 learning_suggestion_text = response.choices[0].message.content if hasattr(response, 'choices') else ""
 learning_suggestions_list.append(learning_suggestion_text)

 return " ".join(learning_suggestions_list)

Integrating the New Methods

To incorporate these advanced methods into your lecture processing script:

# lecture_recording_path = "Lecture2.wav"
transcription = transcribe_lecture(lecture_recording_path)

enhanced_lecture_notes = {
 'summary': lecture_summary_extraction(transcription),
 'key_concepts': key_concepts_extraction(transcription),
 'sentiment': lecture_sentiment_analysis(transcription),
 'lecture_questions': identify_and_answer_lecture_questions(transcription),
 'topic_expansion': expand_lecture_topics(transcription),
 'learning_suggestions': personalized_learning_suggestions(transcription)
}

print(enhanced_lecture_notes)
save_as_docx(enhanced_lecture_notes, 'comprehensive_lecture_notes.docx')

This revised script now includes a broader range of functionalities, effectively handling larger lectures and enriching the learning experience with multi-dimensional analysis. It allows for more interactive and tailored educational content, addressing diverse student needs and enhancing engagement with the lecture material.

Here’s a list of additional innovative ideas that can be implemented using GPT-4 to enhance online lecture analysis and student engagement:

1. Automated Lecture Summarization for Different Learning Levels: Create summaries of the lecture content tailored to various educational levels (beginner, intermediate, advanced) to cater to students with different backgrounds.
2. Interactive Lecture FAQs Generation: Generate a set of frequently asked questions (FAQs) and their answers based on the lecture content to help students quickly find information.
3. Cross-Lecture Topic Correlation: Analyze multiple lectures to identify and explain cross-lecture topics and concepts, helping students see the bigger picture.
4. Customized Study Guides: Generate study guides or revision notes that are personalized based on the student’s past queries or identified areas of difficulty.
5. Language Translation for Lectures: Automatically translate lecture transcripts into different languages to aid non-native speakers.
6. Lecture Tone and Engagement Analysis: Analyze the tone and engagement level of the lecture to provide feedback to educators on their teaching style.
7. Creation of Mind Maps or Concept Maps: Generate visual mind maps or concept maps from lecture content to aid visual learners.
8. Debate Points and Counterpoints Extraction: For lectures involving debates or discussions, extract key points and counterpoints to facilitate understanding of different perspectives.
9. Real-Time Lecture Note Taking Assistant: Develop a tool that takes notes in real-time during a live lecture, highlighting key points and concepts.
10. Lecture Content Fact-Checking: Implement a method to fact-check statements or data presented in the lecture against reliable sources.
11. Identifying Potential Research Areas: Analyze lectures to suggest potential areas for further research or study for students or academics.
12. Integration with Learning Management Systems: Develop plugins or integrations with popular Learning Management Systems (LMS) to directly input the processed lecture content.
13. Customizable Lecture Flashcards: Create flashcards from lecture content for quick revision, which students can customize based on their study needs.
14. Sentiment Analysis of Student Discussions: Analyze sentiment in student forums or discussion boards to gauge understanding and engagement with lecture topics.
15. Predictive Analysis for Student Performance: Use lecture engagement and understanding to predict student performance and identify areas where intervention might be needed.

Each of these ideas can be tailored to specific educational contexts and student needs, leveraging the power of GPT-4 to create a more engaging, inclusive, and effective learning environment.