Advertisement
  1. Photo & Video
  2. Subtitles
Photography

3 Ways to Subtitle and Caption Your Videos Automatically Using Artificial Intelligence

by
Difficulty:BeginnerLength:MediumLanguages:

In this tutorial you'll learn to use speech recognition software to automatically add subtitles or captions to a video. We'll show you how to use three leading artificial intelligence technologies, two free and one paid.  Adding subtitles to your videos is easier than it may seem, and brings all kinds of great benefits. Read on to find out how.

Accessibility is Good for Everyone

With the rise of multiple online video platforms and the spread of video-capable smartphones we're now producing and consuming more and more video every day. This swell of video creates three new access problems:

  1. How do we find the videos we want in a crowded world?
  2. How do you watch all these new videos if you have a hearing impairment?
  3. How do you follow along with video that isn't in your first language, as is now an everyday reality for millions of people?

Subtitles and captions solve all of these problems.

Search engines have no idea what's in your videos. This is slowly changing with the rise of algorithms that can interpret images but, fundamentally, search engines are built to read text. The best way to tell the world what's in your video is still to describe it. Subtitles and closed captions provide exactly the kind of juicy text information that Google and the rest love to have.

As many as 15 percent of Americans have a hearing impairment. Closed captions, or written text overlaid on a video that replicates what the speaker is saying, are a key accessibility tool. Accessibility is essential in all kinds of videos, but it's especially important when teaching with video, which is why Envato is moving towards providing captions on as many Tuts+ videos as we possibly can.

It's not just us, either: in the USA, significantly more closed captioning will be required by law by next year. It's also worth noting that Facebook videos play automatically, but they play silently by default, so if you want everyone on Facebook to know what people in your videos are saying you really need captions or subtitles.

Subtitles are frequently used as a way to translate language from one medium to another (such as spoken English to written English) or one language to another. They're a great way to make video more accessible to diverse linguistic audiences.

Envato Tuts Closed Captioning
The Envato Tuts+ YouTube channel features videos with closed captions generated automatically by YouTube, as can be seen in this Illustrator in 60 Seconds video. Read on to find out how to generate closed captions for your productions. 

1. YouTube's Automatic Captioning (Free)

Creating subtitles with YouTube is a breeze. Automatic transcription is built directly into the service, and can be edited in the Video Manager. Although the service does seem to be continually improving, YouTube's transcription is notoriously (and sometimes hilariously) imperfect. Thankfully, you have the option to manually adjust them. Select the Subtitles & CC option to view the subtitle settings, and then click on the subtitle file you want to alter. In my case, that is English (Automatic).

Now press the Edit button to begin adjusting them. On the left side, you'll see the transcription that YouTube automatically generated. You can type over any of the incorrect subtitles to correct them. Use the embedded video player to work your way through the video and intervene as needed. Click Publish Edits when you're finished to save the corrected subtitles.

To export and download a subtitle file, click the Actions dropdown and select the file type you need. SRT is a widely supported format (including Facebook video). Google makes it tricky to save the subtitles from other peoples' videos, but there is a workaround.

There are other problems with YouTube's service. For one, there is no way to collaborate on the subtitles. Or maybe your videos are for a course you're producing and you don't want to put your files on YouTube. Or maybe you are Irish and YouTube just completely mangles your accent.

There are plenty of reasons why YouTube isn't always the right tool for the job. Let's look at how to generate subtitles and captions for any video.

2. Watson and Amara (Free)

Hello, Watson!

You might have heard about IBM's Watson supercomputer when it was defeating defending Jeopardy! champions in 2011. IBM have put Watson technology to use processing all kinds of data, including speech. Simply upload an audio file to the speech-to-text demo and it will transcribe the audio for you.

Go ahead and jump to the app. Don't be fooled: the demo has plenty of functionality. I uploaded a six-minute audio clip and it was transcribed in just a few minutes.

I recommend transcription as a finishing step in editing a video. Lock down your edit, then export a WAV audio file from your video editor. If your app doesn't support exporting directly to WAV, I recommend you use Audacity to convert from other audio formats to WAV.

To upload audio, click on Select Audio File on the app page. Browse to your WAV file and choose it. Watson also supports Spanish, French, Portuguese and Japanese, so make sure and choose the language from the dropdown if it differs from English. Note: If you use an ad blocking plugin for your web browser, make sure and disable it for the IBM website. The demo might not function correctly if adblock is enabled.

Select Audio File
Once you've selected an audio file, the Watson technology will go to work on transcribing it. All of the text you see in this image was generated automatically. The transcription isn't perfect, but it's a great start!

A few minutes later, you'll have a transcription of the audio. At this stage, don't worry about correcting all of the incorrect words in the transcription. Copy the text from the transcription box and save it into a text file, using an app like Notepad on Windows or TextEdit on mac.

If you are a capable programmer, IBM offers a much more in-depth way to use the Watson technology. Check out IBM's tutorial on how to use the full Watson capabilities programmatically.

Use Amara to Create a Finished Subtitle File

Now you have a rough transcription of our audio. Next you'll synchronize it with the video.

For this step we'll use Amara, a service that's designed to do just that. Amara can help us generate a finished subtitle file that can be used on many different services. Jump over to Amara's site and sign up for a free account to get started.

After you've signed up for Amara, you'll need to link to your video file. Amara doesn't support uploading video directly to its site, so you'll need to get the file online first. I have an FTP server, so I'll often upload the video to my server and link to it. If you don't have a web server to use you can upload to Vimeo, for example, and use it as a temporary host.

Amara Link to video
Once you've logged into your new Amara account, paste the link to your video in the address box and press "Begin." You can always post your video on Vimeo as a temporary host to caption your video.

Paste in a link to your video, hit Begin, and Amara will load the video into its subtitling system. Now it's time to upload the transcription Watson generated for us. Click on Upload them directly on the left side of the screen to upload the text file to Amara.

Amara Load Screen
Once Amara has grabbed the link to your video, it will show a preview of it. Click on Upload them directly from the left side of the editor to upload the text file that we created using Watson.

Once you click on the upload them directly option, you'll need to point Amara to the text file that we saved in the last step. Also, specify the language of the text and the video language. Once you press Upload, Amara will load the generated text.

Upload Subtitles
Point Amara to the text file we generated in the last step, and specify the language of the video and captions to proceed. Then, press Upload.

Click on the language of the subtitle on the left side. Then press Edit Subtitles on the next screen to adjust the timings of your subtitles.

Edit Subtitles
To launch the subtitle editor, click on the language of the subtitle on the left side of Amara (in this screenshot, you can see it labeled as English. Then, the next screen will show the text we uploaded. Press Edit Subtitles to launch the subtitle editor.

Check out the video below to walk through the time sync process:

You could create subtitles using Amara alone, but that requires typing the captions from scratch. IBM Watson gives us a great starting point, and pairs well with Amara to get the timing right.

One of the perks of using Amara is that it's great for collaboration and translation. If you have a skilled translator, they can just as easily link to your video and generate subtitles in a different language.

3. Pop-Up Archive (by Subscription)

PopUp Archive is an automatic transcription service that, like YouTube, can generate SRT files.

A paid service, as you might expect PopUp Archive has some distinct advantages over both the YouTube and the Watson-plus-Amara workflows. PopUp Archive will store audio files, can ingest multiple files at once from a variety of sources, supports multiple users on an account, includes robust transcript editing tools, differentiates speakers, adds punctuation, has very precise timestamps, and more. In testing the options for Tuts+ videos, we've found that PopUp Archives transcriptions are more accurate than competing services. In short, you get what you pay for.

The process with PopUp Archive is the very similar to the process above: export an audio file from your video editor, upload to the service, wait for it to process, and download the SRT files.

To upload a file to PopUp Archive, sign in and click the Upload an audio file link on your My Collection page. In the modal box that pops up you can drag and drop an audio file into the window, use the Upload one file at a time button to upload from your computer, or paste a web address into the Add Files by URL field. You can also do a Batch Upload from the My Collection page.

Once you've selected your audio file you can also add a Title, Images, Contributors, Languages, Tags, and many other useful metadata fields.

When your upload is complete, PopUp Archive will give you a success notice on screen. They will send you an email when your transcript is ready.

Hit Edit Text to correct the transcript and adjust timings. To export your subtitles, navigate to your transcript and click on Export. From there you have a variety of formats to choose from, including SRT, WebVTT, and JSON.

Accessibility Matters to Your Audience

You can't ignore the fact that your viewers may have disabilities that must be accommodated. It's not just a matter of what's right; it'll soon be the law. More than that, though, subtitling or captioning your video is a great way to make your work stand out and connect with audiences. The good news is that recent developments in speech-to-text AI have helped us do this without much additional work, as you've seen in this tutorial.

What other accessibility challenges should we think about when creating content? Also, how are you adding captions to your video productions? Let me know in the comments section.

Advertisement
Advertisement
Looking for something to help kick start your next project?
Envato Market has a range of items for sale to help get you started.