Assembly notes are a vital a part of collaboration, but they typically fall by means of the cracks. Between main discussions, listening carefully, and typing notes, it’s straightforward for key info to slide away unrecorded. Even when notes are captured, they are often disorganized or illegible, rendering them ineffective.
On this submit, we discover methods to use Amazon Transcribe and Amazon Bedrock to robotically generate clear, concise summaries of video or audio recordings. Whether or not it’s an inner staff assembly, convention session, or earnings name, this strategy may help you distill hours of content material all the way down to salient factors.
We stroll by means of an answer to transcribe a venture staff assembly and summarize the important thing takeaways with Amazon Bedrock. We additionally talk about how one can customise this answer for different frequent eventualities like course lectures, interviews, and gross sales calls. Learn on to simplify and automate your note-taking course of.
Answer overview
By combining Amazon Transcribe and Amazon Bedrock, it can save you time, seize insights, and improve collaboration. Amazon Transcribe is an automated speech recognition (ASR) service that makes it easy so as to add speech-to-text functionality to functions. It makes use of superior deep studying applied sciences to precisely transcribe audio into textual content. Amazon Bedrock is a totally managed service that gives a selection of high-performing basis fashions (FMs) from main AI firms like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single API, together with a broad set of capabilities it is advisable to construct generative AI functions. With Amazon Bedrock, you possibly can simply experiment with quite a lot of high FMs, and privately customise them together with your knowledge utilizing methods akin to fine-tuning and Retrieval Augmented Era (RAG).
The answer offered on this submit is orchestrated utilizing an AWS Step Features state machine that’s triggered once you add a recording to the designated Amazon Easy Storage Service (Amazon S3) bucket. Step Features permits you to create serverless workflows to orchestrate and join parts throughout AWS companies. It handles the underlying complexity so you possibly can give attention to utility logic. It’s helpful for coordinating duties, distributed processing, ETL (extract, remodel, and cargo), and enterprise course of automation.
The next diagram illustrates the high-level answer structure.
The answer workflow contains the next steps:
A person shops a recording within the S3 asset bucket.
This motion triggers the Step Features transcription and summarization state machine.
As a part of the state machine, an AWS Lambda operate is triggered, which transcribes the recording utilizing Amazon Transcribe and shops the transcription within the asset bucket.
A second Lambda operate retrieves the transcription and generates a abstract utilizing the Anthropic Claude mannequin in Amazon Bedrock.
Lastly, a last Lambda operate makes use of Amazon Easy Notification Service (Amazon SNS) to ship a abstract of the recording to the recipient.
This answer is supported in Areas the place Anthropic Claude on Amazon Bedrock is out there.
The state machine orchestrates the steps to carry out the precise duties. The next diagram illustrates the detailed course of.
Stipulations
Amazon Bedrock customers must request entry to fashions earlier than they’re accessible to be used. This can be a one-time motion. For this answer, you’ll must allow entry to the Anthropic Claude (not Anthropic Claude Immediate) mannequin in Amazon Bedrock. For extra info, confer with Mannequin entry.
Deploy answer assets
The answer is deployed utilizing an AWS CloudFormation template, discovered on the GitHub repo, to robotically provision the required assets in your AWS account. The template requires the next parameters:
E-mail handle used to ship abstract – The abstract can be despatched to this handle. You have to acknowledge the preliminary Amazon SNS affirmation e mail earlier than receiving extra notifications.
Abstract directions – These are the directions given to the Amazon Bedrock mannequin to generate the abstract.
Run the answer
After you deploy the answer utilizing AWS CloudFormation, full the next steps:
Acknowledge the Amazon SNS e mail affirmation that you need to obtain a couple of moments after creating the CloudFormation stack.
On the AWS CloudFormation console, navigate to stack you simply created.
On the stack’s Outputs tab, and search for the worth related to AssetBucketName; it should look one thing like summary-generator-assetbucket-xxxxxxxxxxxxx.
On the Amazon S3 console, navigate to your asset bucket.
That is the place you’ll add your recordings. Legitimate file codecs are MP3, MP4, WAV, FLAC, AMR, OGG, and WebM.
Add your recording to the recordings folder.
Importing recordings will robotically set off the Step Features state machine. For this instance, we use a pattern staff assembly recording within the sample-recording listing of the GitHub repository.
On the Step Features console, navigate to the summary-generator state machine.
Select the identify of the state machine run with the standing Operating.
Right here, you possibly can watch the progress of the state machine because it processes the recording.
After it reaches its Success state, you need to obtain an emailed abstract of the recording.
Alternatively, you possibly can navigate to the S3 belongings bucket and look at the transcript there within the transcripts folder.
Evaluation the abstract
You’ll get the recording abstract emailed to the handle you offered once you created the CloudFormation stack. When you don’t obtain the e-mail in a couple of moments, just remember to acknowledged the Amazon SNS affirmation e mail that you need to have acquired after you created the stack after which add the recording once more, which can set off the abstract course of.
This answer features a mock staff assembly recording that you need to use to check the answer. The abstract will look much like the next instance. Due to the character of generative AI, nevertheless, your output will look a bit totally different, however the content material must be shut.
Listed here are the important thing factors from the standup:
Joe completed reviewing the present state for job EDU1 and created a brand new job to develop the longer term state. That new job is within the backlog to be prioritized. He’s now beginning EDU2 however is blocked on useful resource choice.
Rob created a tagging technique for SLG1 based mostly on finest practices, however could must coordinate with different groups who’ve created their very own methods, to align on a uniform strategy. A brand new job was created to coordinate tagging methods.
Rob has made progress debugging for SLG2 however might have extra assist. This job can be moved to Dash 2 to permit time to get additional assets.Subsequent Steps:
Joe to proceed engaged on EDU2 as in a position till useful resource choice is set
New job to be prioritized to coordinate tagging methods throughout groups
SLG2 moved to Dash 2
Standups transferring to Mondays beginning subsequent week
Broaden the answer
Now that you’ve got a working answer, listed below are some potential concepts to customise the answer in your particular use instances:
Strive altering the method to suit your accessible supply content material and desired outputs:
For conditions the place transcripts can be found, create an alternate Step Features workflow to ingest present text-based or PDF-based transcriptions.
As a substitute of utilizing Amazon SNS to inform recipients through e mail, you need to use it to ship the output to a special endpoint, akin to a staff collaboration web site, or to the staff’s chat channel.
Strive altering the abstract directions CloudFormation stack parameter offered to Amazon Bedrock to provide outputs particular to your use case (that is the generative AI immediate):
When summarizing an organization’s earnings name, you could possibly have the mannequin give attention to potential promising alternatives, areas of concern, and issues that you need to proceed to watch.
If you’re utilizing this to summarize a course lecture, the mannequin may determine upcoming assignments, summarize key ideas, record info, and filter out any small discuss from the recording.
For a similar recording, create totally different summaries for various audiences:
Engineers’ summaries give attention to design choices, technical challenges, and upcoming deliverables.
Undertaking managers’ summaries give attention to timelines, prices, deliverables, and motion objects.
Undertaking sponsors get a quick replace on venture standing and escalations.
For longer recordings, attempt producing summaries for various ranges of curiosity and time dedication. For instance, create a single sentence, single paragraph, single web page, or in-depth abstract. Along with the immediate, chances are you’ll need to modify the max_tokens_to_sample parameter to accommodate totally different content material lengths.
Clear up
To wash up the answer, delete the CloudFormation stack that you simply created earlier. Observe that deleting the stack won’t delete the asset bucket. When you now not want the recordings or transcripts, you possibly can delete this bucket individually. Amazon Transcribe will robotically delete transcription jobs after 90 days, however you possibly can delete these manually earlier than then.
Conclusion
On this submit, we explored methods to use Amazon Transcribe and Amazon Bedrock to robotically generate clear, concise summaries of video or audio recordings. We encourage you to proceed evaluating Amazon Bedrock, Amazon Transcribe, and different AWS AI companies, like Amazon Textract, Amazon Translate, and Amazon Rekognition, to see how they may help meet your enterprise goals.
In regards to the Authors
Rob Barnes is a principal guide for AWS Skilled Providers. He works with our prospects to handle safety and compliance necessities at scale in complicated, multi-account AWS environments by means of automation.
Jason Stehle is a Senior Options Architect at AWS, based mostly within the New England space. He works with prospects to align AWS capabilities with their biggest enterprise challenges. Exterior of labor, he spends his time constructing issues and watching comedian e-book motion pictures together with his household.