subed.el and quoting snippets of YouTube videos from the auto captions
| emacs, audio, subedI'm preparing for a conversation with linkarzu. It would be fun to quote snippets from some of the other videos he has. I like the way he does a lot of reaction videos that build on what people have shared. He tends to listen to things at 1.25 or 1.5x speed. I'm not sure if I have the patience even for that, so it'd be nice to be able to just quickly skim the text. He also clips a lot of his own videos. It might be interesting to play around with my workflows for listening to snippets and clipping them.
I've grabbed YouTube captions and reformatted them before:
- Grabbing the Youtube auto-generated captions is pretty useful when making Emacs News (2021)
- Org Mode: Insert YouTube video with separate captions (2021)
This time around, I want to get the VTT format so that I can possibly use them with subed-record to clip audio or video.
From a YouTube link, I want to run a single command that gets me a VTT file with that as the media source and with the captions. If there are any chapters in the description, it would be lovely to get them too. I can get them from the JSON, yay!
Now I can use M-x sacha-subed-youtube to open the VTT file, which is cached locally, and I can use M-x sacha-subed-youtube-region-as-quote after selecting a section. Well, actually, I have an Embark command s (for "subed") that I can run on a YouTube video link and a selected-mode shortcut for y that I can run when the region is selected. Results:
I Asked Prot If I Should Switch to Emacs
00:09:46: I think the value of Org is that it is in the context of Emacs, which can now do other things. It can be your email client. It can be your agenda. Of course, it can be your programming environment. All these sorts of things, like your RSS feed reader and so on. And in the context of all that, you also have Org. So for me, that's kind of the value. But otherwise, having Org elsewhere, it's great still, because the paradigm of Org, I think, is a portable paradigm.
Org Mode in Neovim Is Finally Here - Full org.nvim Demo
00:00:00: I think that I achieved like 100% of parity with the original Org Mode version on Emacs.
Code for subed + YouTube
(defvar sacha-subed-youtube-dir "~/proj/youtube-captions"
"Directory with YouTube captions.")
(defvar sacha-subed-youtube-temp-dir "~/Downloads" "Snap installations need a workaround for permission.")
;;;###autoload
(defun sacha-subed-youtube (url)
"Download the automatic subtitles or check my cache."
(interactive "MURL: ")
(let* ((id (sacha-org-yt-id url))
(existing (car (directory-files sacha-subed-youtube-dir t
(concat "^" (regexp-quote id) ".*\\.vtt\\'")))))
(if existing
(progn
(find-file existing)
(subed-mpv-play-from-url url))
;; Download
(let ((directory (make-temp-file (expand-file-name "yt-dlp" sacha-subed-youtube-temp-dir) t))
filename
json
match
chapters
subtitles)
(with-current-buffer (get-buffer-create "*yt-dlp*")
(let ((default-directory directory))
(call-process
"yt-dlp" nil t nil
"--skip-download"
"--write-info-json"
"--sub-format"
"vtt"
"--write-subs"
"--write-auto-subs"
url)))
(when (null (directory-files directory t "\\.json$"))
(error "Could not get data."))
(setq json (json-read-file (car (directory-files directory t "\\.json$"))))
(setq chapters (alist-get 'chapters json))
(setq filename (expand-file-name
(concat id " - " (alist-get 'title json) ".vtt")
sacha-subed-youtube-dir))
(setq subtitles (subed-parse-file (car (directory-files
directory t
"\\.vtt$"))))
(delete-directory directory t)
;; add the chapters as comments
(mapc
(lambda (sub)
(when (and chapters
(>= (elt sub 1)
(* (alist-get 'start_time (car chapters)) 1000)))
(setf (elt sub 4) (alist-get 'title (pop chapters)))))
subtitles)
(subed-create-file
filename
subtitles
t)
(find-file filename)
(subed-mpv-play-from-url url)))))
(defun sacha-subed-youtube-id ()
"Return the YouTube ID."
(or (save-restriction
(save-excursion
(goto-char (point-min))
(subed-forward-subtitle-text)
(when (subed-record-get-directive "YOUTUBE_URL")
(sacha-org-yt-id
(subed-record-get-directive "YOUTUBE_URL")))))
(car (split-string (file-name-base (buffer-file-name)) " "))))
;; Now we just need a quick way to copy a YouTube URL.
;;;###autoload
(defun sacha-subed-youtube-url (&optional start-msecs end-msecs)
"Copy YouTube URL for the current timestamp."
(interactive)
(when-let*
((id (sacha-subed-youtube-id))
(s
(if end-msecs
(format "https://youtu.be/%s?t=%d&end=%d"
id
(/ (or start-msecs (subed-subtitle-msecs-start))
1000)
(1+ (/ end-msecs
1000)))
(format "https://youtu.be/%s?t=%d"
id
(/ (or start-msecs (subed-subtitle-msecs-start))
1000)))))
(when (called-interactively-p 'any)
(kill-new s)
(message "%s" s))
s))
;;;###autoload
(defun sacha-subed-youtube-play ()
"Play this video from YouTube."
(interactive)
(when-let*
((id (sacha-subed-youtube-id))
(s (concat "https://youtu.be/" id)))
(when (string= subed-mpv-media-file s)
(subed-mpv-play-from-url s))
(subed-mpv-jump-to-current-subtitle)
(subed-mpv-unpause)))
;;;###autoload
(defun sacha-subed-youtube-region-as-quote (beg end &optional label)
"Format this as a quote."
(interactive (list (region-beginning) (region-end) (when current-prefix-arg (read-string "Label: "))))
(let* ((id (sacha-subed-youtube-id))
(url (sacha-subed-youtube-url
(save-excursion
(goto-char beg)
(subed-subtitle-msecs-start))
(save-excursion
(goto-char end)
(subed-subtitle-msecs-stop))))
(base (file-name-base (buffer-file-name)))
(text (subed-subtitle-list-text (subed-subtitle-list beg end)))
(s (format
"%s\n#+begin_quote\n%s: %s\n#+end_quote\n"
(org-link-make-string
(concat "yt:" url)
(or label
(and (string-match (format "^%s - \\(.+\\)" (regexp-quote id)) base)
(match-string 1 base))
(sacha-page-title url)))
(org-link-make-string
(concat "vtime:"
(subed-msecs-to-timestamp
(save-excursion
(goto-char beg)
(subed-subtitle-msecs-start))
t)))
(with-temp-buffer
(insert text)
(let ((fill-column most-positive-fixnum))
(fill-region (point-min) (point-max))
(string-trim (buffer-string)))))))
(when (called-interactively-p 'any)
(kill-new s)
(message "%s" s))
s))
Some of this code hasn't been committed to subed yet, but I will commit it sometime this week.
My next steps might be to:
- cache the video as a whole to make it easier to check the waveforms
- extract the segment to a standalone video file so that it's easy for me to play back or include in another video
- see if I can use my voice activity detection and semi-streaming speech recognition to get a VTT of my current OBS recording

