2022-12-26 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, lobste.rs, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, emacs-devel, and lemmy/c/emacs. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

Comparison-shopping with Org Mode

| emacs, org

I don't like shopping. We're lucky to be able to choose, but I get overwhelmed with all the choices. I'm trying to get the hang of it, though, since I'll need to shop for lots of things for A- over the years. One of the things that's stressful is comparing choices between different webpages, especially if I want to get A-'s opinion on something. Between the challenge of remembering things as we flip between pages and the temptations of other products she sees along the way… Ugh.

I think there are web browser extensions for shopping, but I prefer to work within Org Mode so that I can capture links from my phone's web browser, refile entries into different categories, organize them with keyboard shortcuts, and tweak things the way I like. So if I have subheadings with the NAME, PRICE, IMAGE, and URL properties, I can make a table that looks like this:

2022-12-26_11-26-35.png

Figure 1: Comparison-shopping

using code that looks like this:

#+begin_src emacs-lisp :eval yes :exports results :wrap EXPORT html
(my-org-format-shopping-subtree)
#+end_src

and I can view the table by exporting the subtree with HTML using org-export-dispatch (C-c C-e C-s h o). When I add new items, I can use C-u C-c C-e to reexport the subtree without navigating up to the root.

Here's the very rough code I use for that:

(defun my-get-shopping-details ()
  (goto-char (point-min))
  (let (data)
    (cond
     ((re-search-forward "  data-section-data
>" nil t)
      (setq data (json-read))
      (let-alist data
        (list (cons 'name .product.title)
              (cons 'brand .product.vendor)
              (cons 'description .product.description)
              (cons 'image (concat "https:" .product.featured_image))
              (cons 'price (/ .product.price 100.0)))))
     ((and (re-search-forward "<script type=\"application/ld\\+json\">" nil t)
           (null (re-search-forward "Fabric Fabric" nil t))) ; Carter's, Columbia?
      (setq data (json-read))
      (if (vectorp data) (setq data (elt data 0)))
      (if (assoc-default '@graph data)
          (setq data (assoc-default '@graph data)))
      (if (vectorp data) (setq data (elt data 0)))
      (let-alist data
        (list (cons 'name .name)
              (cons 'url (or .url .@id))
              (cons 'brand .brand.name)
              (cons 'description .description)
              (cons 'rating .aggregateRating.ratingValue)
              (cons 'ratingCount .aggregateRating.reviewCount)
              (cons 'image (if (stringp .image) .image (elt .image 0)))
              (cons 'price
                    (assoc-default 'price (if (arrayp .offers)
                                              (elt .offers 0)
                                            .offers))))))
     ((re-search-forward "amazon.ca" nil t)
      (goto-char (point-min))
      (re-search-forward "^$")
      (let ((doc (libxml-parse-html-region (point) (point-max))))
        (list (cons 'name (dom-text (dom-by-tag doc 'title)))
              (cons 'description (dom-texts (dom-by-id doc "productDescription")))
              (cons 'image (dom-attr (dom-by-tag (dom-by-id doc "imgTagWrapperId") 'img) 'src))
              (cons 'price
                    (dom-texts (dom-by-id doc "priceblock_ourprice"))))))
     (t
      (goto-char (point-min))
      (re-search-forward "^$")
      (let* ((doc (libxml-parse-html-region (point) (point-max)))
             (result
              `((name . ,(string-trim (dom-text (dom-by-tag doc "title"))))
                (description . ,(string-trim (dom-text (dom-by-tag doc "title")))))
              ))
        (mapc (lambda (property)
                (let ((node
                       (dom-search
                        doc
                        (lambda (o)
                          (delq nil
                                (mapcar (lambda (p)
                                          (or (string= (dom-attr o 'property) p)
                                              (string-match p (or (dom-attr o 'class) ""))))
                                        (cdr property)))))))
                  (when node (add-to-list 'result (cons (car property)
                                                        (or (dom-attr node 'content)
                                                            (string-trim (dom-text node))))))))
              '((name "og:title" "pdp-product-title")
                (brand "og:brand")
                (url "og:url")
                (image "og:image")
                (description "og:description")
                (price "og:price:amount" "product:price:amount" "pdp-price-label")))
        result)
      ))))
(defun my-org-insert-shopping-details ()
  (interactive)
  (org-insert-heading)
  (save-excursion (yank))
  (my-org-update-shopping-details)
  (when (org-entry-get (point) "NAME")
    (org-edit-headline (org-entry-get (point) "NAME")))
  (org-end-of-subtree))
(defun my-org-update-shopping-details ()
  (interactive)
  (when (re-search-forward org-link-any-re (save-excursion (org-end-of-subtree)) t)
    (let* ((link (org-element-property :raw-link (org-element-context)))
           data)
      (if (string-match "theshoecompany\\|dsw" link)
          (progn
            (browse-url link)
            (org-entry-put (point) "URL" link)
            (unless (org-entry-get (point) "IMAGE")
              (org-entry-put (point) "IMAGE" (read-string "Image: ")))
            (unless (org-entry-get (point) "PRICE")
              (org-entry-put (point) "PRICE" (read-string "Price: "))))
        (setq data (with-current-buffer (url-retrieve-synchronously link)
                     (my-get-shopping-details)))
        (when data
          (let-alist data
            (org-entry-put (point) "NAME" .name)
            (org-entry-put (point) "URL" link)
            (org-entry-put (point) "BRAND" .brand)
            (org-entry-put (point) "DESCRIPTION" (replace-regexp-in-string "&#039;" "'" (replace-regexp-in-string "\n" " " (or .description ""))))
            (org-entry-put (point) "IMAGE" .image)
            (org-entry-put (point) "PRICE" (cond ((stringp .price) .price) ((numberp .price) (format "%.2f" .price)) (t ""))) 
            (if .rating (org-entry-put (point) "RATING" (if (stringp .rating) .rating (format "%.1f" .rating))))
            (if .ratingCount (org-entry-put (point) "RATING_COUNT" (if (stringp .ratingCount) .ratingCount (number-to-string .ratingCount))))
            ))))))
(defun my-org-format-shopping-subtree ()
  (concat
   "<style>body { max-width: 100% !important } #content { max-width: 100% !important } .item img { max-height: 100px; }</style><div style=\"display: flex; flex-wrap: wrap; align-items: flex-start\">"
   (string-join
    (save-excursion
      (org-map-entries
       (lambda ()
         (if (org-entry-get (point) "URL")
             (format
              "<div class=item style=\"width: 200px\"><div><a href=\"%s\"><img src=\"%s\" height=100></a></div>
<div>%s</div>
<div><a href=\"%s\">%s</a></div>
<div>%s</div>
<div>%s</div></div>"
              (org-entry-get (point) "URL")
              (org-entry-get (point) "IMAGE")
              (org-entry-get (point) "PRICE")
              (org-entry-get (point) "URL")
              (url-domain (url-generic-parse-url (org-entry-get (point) "URL")))
              (org-entry-get (point) "NAME")
              (or (org-entry-get (point) "NOTES") ""))
           ""))
       nil
       (if (org-before-first-heading-p) nil 'tree)))
    "")
   "</div>"))

At some point, it would be nice to keep track of how I feel about different return policies, and to add more rules for automatically extracting information from different websites. (org-chef might be a good model.) In the meantime, this makes it a little less stressful to look for stuff.

This is part of my Emacs configuration.

Figuring out how to use ffmpeg to mask a chroma-keyed video based on the differences between images

| linux, geek, ffmpeg

A- is really into Santa and Christmas because of the books she's read. Last year, she wanted to set up the GoPro to capture footage during Christmas Eve. I helped her set it up for a timelapse video. After she went to bed, we gradually positioned the presents. I extracted the frames from the video, removed the ones that caught us moving around, and then used Krita's new animation features to animate sparkles so that the presents magically appeared. She mentioned the sparkles a number of times during her deliberations about whether Santa exists or not.

This year, I want to see if I can use green-screen videos like this reversed-spin sparkle or this other sparkle video. I'm going to take a series of images, with each image adding one more gift. Then I'm going to make a mask in Krita with white covering the gift and a transparent background for the rest of the image. Then I'll use chroma-key to drop out the green screen of the sparkle video and mask it in so that the sparkles only happen within the boundaries of the gift that was added. I also want to fade one image into the other, and I want the sparkles to fade out as the gift appears.

Figuring things out

I didn't know how to do any of that yet with ffmpeg, so here's how I started figuring things out. First, I wanted to see how to fade test.jpg into test2.jpg over 4 seconds.

ffmpeg -y -loop 1 -i test.jpg -loop 1 -i test2.jpg -filter_complex "[1:v]fade=t=in:d=4:alpha=1[fadein];[0:v][fadein]overlay[out]" -map "[out]" -r 1 -t 4 -shortest test.webm

Here's another way using the blend filter:

ffmpeg -y -loop 1 -i test.jpg -loop 1 -i test2.jpg -filter_complex "[1:v][0:v]blend=all_expr='A*(if(gte(T,4),1,T/4))+B*(1-(if(gte(T,4),1,T/4)))" -t 4 -r 1 test.webm

Then I looked into chromakeying in the other video. I used balloons instead of sparkles just in case she happened to look at my screen.

ffmpeg -y -i test.webm -i balloons.mp4 -filter_complex "[1:v]chromakey=0x00ff00:0.1:0.2[ckout];[0:v][ckout]overlay[out]" -map "[out]" -shortest -r 1 overlaid.webm

I experimented with the alphamerge filter.

ffmpeg -y -i test.jpg -i test2.jpg -i mask.png -filter_complex "[1:v][2:v]alphamerge[a];[0:v][a]overlay[out]" -map "[out]" masked.jpg

Okay! That overlaid test.jpg with a masked part of test2.jpg. How about alphamerging in a video? First, I need a mask video…

ffmpeg -y -loop 1 -i mask.png  -r 1 -t 4  mask.webm

Then I can combine that:

ffmpeg -loglevel 32 -y -i test.webm -i balloons.mp4 -i mask.webm -filter_complex "[1:v][2:v]alphamerge[masked];[0:v][masked]overlay[out]" -map "[out]" -r 1 -t 4 alphamerged.webm

Great, let's figure out how to combine chroma-key and alphamerge video. The naive approach doesn't work, probably because they're both messing with the alpha layer.

ffmpeg -loglevel 32 -y -i test.webm -i balloons.mp4 -i mask.webm -filter_complex "[1:v]chromakey=0x00ff00:0.1:0.2[ckout];[ckout][2:v]alphamerge[masked];[0:v][masked]overlay[out]" -map "[out]" -r 1 -t 4 masked.webm

So I probably need to blend the chromakey and the mask. Let's see if I can extract the chromakey alpha.

ffmpeg -loglevel 32 -y -i test.webm -i balloons.mp4 -i mask.webm -filter_complex "[1:v]chromakey=0x00ff00:0.1:0.2,format=rgba,alphaextract[out]" -map "[out]" -r 1 -t 4
chroma-alpha.webm

Now let's blend it with the mask.webm.

ffmpeg -loglevel 32 -y -i test.webm -i balloons.mp4 -i mask.webm -filter_complex "[1:v]chromakey=0x00ff00:0.1:0.2,format=rgba,alphaextract[ckalpha];[ckalpha][2:v]blend=all_mode=and[out]" -map "[out]" -r 1 -t 4 masked-alpha.webm

Then let's use it as the alpha:

ffmpeg -loglevel 32 -y -i test.webm -i balloons.mp4 -i masked-alpha.webm -filter_complex "[2:v]format=rgba[mask];[1:v][mask]alphamerge[masked];[0:v][masked]overlay[out]" -map "[out]" -r 1 -t 4 alphamerged.webm

Okay, that worked! Now how do I combine everything into one command? Hmm…

ffmpeg -loglevel 32 -y -loop 1 -i test.jpg -t 4 -loop 1 -i test2.jpg -t 4 -i balloons.mp4 -loop 1 -i mask.png -t 4 -filter_complex "[1:v][0:v]blend=all_expr='A*(if(gte(T,4),1,T/4))+B*(1-(if(gte(T,4),1,T/4)))'[fade];[2:v]chromakey=0x00ff00:0.1:0.2,format=rgba,alphaextract[ckalpha];[ckalpha][3:v]blend=all_mode=and,format=rgba[maskedalpha];[2:v][maskedalpha]alphamerge[masked];[fade][masked]overlay[out]" -map "[out]" -r 5 -t 4 alphamerged.webm

Then I wanted to fade the masked video out by the end.

ffmpeg -loglevel 32 -y -loop 1 -i test.jpg -t 4 -loop 1 -i test2.jpg -t 4 -i balloons.mp4 -loop 1 -i mask.png -t 4 -filter_complex "[1:v][0:v]blend=all_expr='A*(if(gte(T,4),1,T/4))+B*(1-(if(gte(T,4),1,T/4)))'[fade];[2:v]chromakey=0x00ff00:0.1:0.2,format=rgba,alphaextract[ckalpha];[ckalpha][3:v]blend=all_mode=and,format=rgba[maskedalpha];[2:v][maskedalpha]alphamerge[masked];[masked]fade=type=out:st=2:d=1:alpha=1[maskedfade];[fade][maskedfade]overlay[out]" -map "[out]" -r 10 -t 4 alphamerged.webm

Making the video

When A- finally went to bed, we arranged the presents, using the GoPro to take a picture at each step of the way. I cropped and resized the images, using Krita to figure out the cropping rectangle and offset.

for FILE in *.JPG; do convert $FILE -crop 1558x876+473+842 -resize 1280x720 cropped/$FILE; done

I used ImageMagick to calculate the masks automatically.

files=(*.JPG)
i=0
j=1
len="${#files[@]}"
while [ "$j" -lt $len ]; do
  compare -fuzz 15% cropped/${files[$i]} cropped/${files[$j]} -compose Src -highlight-color White -lowlight-color Black masks/${files[$j]}
  convert -morphology Open Disk -morphology Close Disk -blur 20x5 masks/${files[$j]} processed-masks/${files[$j]}
  i=$((i+1))
  j=$((j+1))
done

Then I faded the images together to make a video.

import ffmpeg
import glob
files = glob.glob("images/cropped/*.JPG")
files.sort()
fps = 15
crf = 32
out = ffmpeg.input(files[0], loop=1, r=fps)
duration = 3
for i in range(1, len(files)):
    out = ffmpeg.filter([out, ffmpeg.input(files[i], loop=1, r=fps).filter('fade', t='in', d=duration, st=i*duration, alpha=1)], 'overlay')
args = out.output('images.webm', t=len(files) * duration, r=fps, y=None, crf=crf).compile()
print(' '.join(f'"{item}"' for item in args))

"ffmpeg" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2317.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2318.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2319.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2320.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2321.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2322.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2323.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2324.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2325.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2326.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2327.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2328.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2329.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2330.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2331.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2332.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2333.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2334.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2335.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2336.JPG" "-loop" "1" "-r" "15" "-i" "images/cropped/GOPR2337.JPG" "-filter_complex" "[1]fade=alpha=1:d=3:st=3:t=in[s0];[0][s0]overlay[s1];[2]fade=alpha=1:d=3:st=6:t=in[s2];[s1][s2]overlay[s3];[3]fade=alpha=1:d=3:st=9:t=in[s4];[s3][s4]overlay[s5];[4]fade=alpha=1:d=3:st=12:t=in[s6];[s5][s6]overlay[s7];[5]fade=alpha=1:d=3:st=15:t=in[s8];[s7][s8]overlay[s9];[6]fade=alpha=1:d=3:st=18:t=in[s10];[s9][s10]overlay[s11];[7]fade=alpha=1:d=3:st=21:t=in[s12];[s11][s12]overlay[s13];[8]fade=alpha=1:d=3:st=24:t=in[s14];[s13][s14]overlay[s15];[9]fade=alpha=1:d=3:st=27:t=in[s16];[s15][s16]overlay[s17];[10]fade=alpha=1:d=3:st=30:t=in[s18];[s17][s18]overlay[s19];[11]fade=alpha=1:d=3:st=33:t=in[s20];[s19][s20]overlay[s21];[12]fade=alpha=1:d=3:st=36:t=in[s22];[s21][s22]overlay[s23];[13]fade=alpha=1:d=3:st=39:t=in[s24];[s23][s24]overlay[s25];[14]fade=alpha=1:d=3:st=42:t=in[s26];[s25][s26]overlay[s27];[15]fade=alpha=1:d=3:st=45:t=in[s28];[s27][s28]overlay[s29];[16]fade=alpha=1:d=3:st=48:t=in[s30];[s29][s30]overlay[s31];[17]fade=alpha=1:d=3:st=51:t=in[s32];[s31][s32]overlay[s33];[18]fade=alpha=1:d=3:st=54:t=in[s34];[s33][s34]overlay[s35];[19]fade=alpha=1:d=3:st=57:t=in[s36];[s35][s36]overlay[s37];[20]fade=alpha=1:d=3:st=60:t=in[s38];[s37][s38]overlay[s39]" "-map" "[s39]" "-crf" "32" "-r" "15" "-t" "63" "-y" "images.webm"

Next, I faded the masks together. These ones faded in and out so that only one mask was active at a time.

import ffmpeg
import glob
files = glob.glob("images/processed-masks/*.JPG")
files.sort()
files = files[:-2]  # Omit the last two, where I'm just turning off the lights
fps = 15
crf = 32
out = ffmpeg.input('color=black:s=1280x720', f='lavfi', r=fps)
duration = 3
for i in range(0, len(files)):
    out = ffmpeg.filter([out, ffmpeg.input(files[i], loop=1, r=fps).filter('fade', t='in', d=1, st=(i + 1)*duration, alpha=1).filter('fade', t='out', st=(i + 2)*duration - 1)], 'overlay')
args = out.output('processed-masks.webm', t=len(files) * duration, r=fps, y=None, crf=crf).compile()
print(' '.join(f'"{item}"' for item in args))

"ffmpeg" "-f" "lavfi" "-r" "15" "-i" "color=s=1280x720" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2318.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2319.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2320.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2321.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2322.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2323.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2324.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2325.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2326.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2327.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2328.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2329.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2330.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2331.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2332.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2333.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2334.JPG" "-loop" "1" "-r" "15" "-i" "images/processed-masks/GOPR2335.JPG" "-filter_complex" "[1]fade=alpha=1:d=1:st=3:t=in[s0];[s0]fade=st=5:t=out[s1];[0][s1]overlay[s2];[2]fade=alpha=1:d=1:st=6:t=in[s3];[s3]fade=st=8:t=out[s4];[s2][s4]overlay[s5];[3]fade=alpha=1:d=1:st=9:t=in[s6];[s6]fade=st=11:t=out[s7];[s5][s7]overlay[s8];[4]fade=alpha=1:d=1:st=12:t=in[s9];[s9]fade=st=14:t=out[s10];[s8][s10]overlay[s11];[5]fade=alpha=1:d=1:st=15:t=in[s12];[s12]fade=st=17:t=out[s13];[s11][s13]overlay[s14];[6]fade=alpha=1:d=1:st=18:t=in[s15];[s15]fade=st=20:t=out[s16];[s14][s16]overlay[s17];[7]fade=alpha=1:d=1:st=21:t=in[s18];[s18]fade=st=23:t=out[s19];[s17][s19]overlay[s20];[8]fade=alpha=1:d=1:st=24:t=in[s21];[s21]fade=st=26:t=out[s22];[s20][s22]overlay[s23];[9]fade=alpha=1:d=1:st=27:t=in[s24];[s24]fade=st=29:t=out[s25];[s23][s25]overlay[s26];[10]fade=alpha=1:d=1:st=30:t=in[s27];[s27]fade=st=32:t=out[s28];[s26][s28]overlay[s29];[11]fade=alpha=1:d=1:st=33:t=in[s30];[s30]fade=st=35:t=out[s31];[s29][s31]overlay[s32];[12]fade=alpha=1:d=1:st=36:t=in[s33];[s33]fade=st=38:t=out[s34];[s32][s34]overlay[s35];[13]fade=alpha=1:d=1:st=39:t=in[s36];[s36]fade=st=41:t=out[s37];[s35][s37]overlay[s38];[14]fade=alpha=1:d=1:st=42:t=in[s39];[s39]fade=st=44:t=out[s40];[s38][s40]overlay[s41];[15]fade=alpha=1:d=1:st=45:t=in[s42];[s42]fade=st=47:t=out[s43];[s41][s43]overlay[s44];[16]fade=alpha=1:d=1:st=48:t=in[s45];[s45]fade=st=50:t=out[s46];[s44][s46]overlay[s47];[17]fade=alpha=1:d=1:st=51:t=in[s48];[s48]fade=st=53:t=out[s49];[s47][s49]overlay[s50];[18]fade=alpha=1:d=1:st=54:t=in[s51];[s51]fade=st=56:t=out[s52];[s50][s52]overlay[s53]" "-map" "[s53]" "-crf" "32" "-r" "15" "-t" "54" "-y" "processed-masks.webm"

I ended up using this particle glitter video because the gifts were small, so I wanted a video that was dense with sparkly things. I also wanted the sparkles to be more concentrated on the area where the gifts were, so I resized it and positioned it.

ffmpeg -loglevel 32 -y -f lavfi -i color=black:s=1280x720 -i sparkles4.webm -ss 13 -filter_complex "[1:v]scale=700:392[sparkles];[0:v][sparkles]overlay=x=582:y=194,setpts=(PTS-STARTPTS)*1.05[out]" -map "[out]" -r 15 -t 53 -shortest sparkles-trimmed.webm
ffmpeg -y -stream_loop 2 -i sparkles-trimmed.webm -t 57 sparkles-looped.webm              

Lastly, I combined the videos with the sparkles.

ffmpeg -loglevel 32 -y -i images.webm -i sparkles-looped.webm -i processed-masks.webm -filter_complex "[1:v]chromakey=0x0a9d06:0.1:0.2,format=rgba,alphaextract[ckalpha];[ckalpha][2:v]blend=all_mode=and,format=rgba[maskedalpha];[1:v][maskedalpha]alphamerge[masked];[masked]fade=t=out:st=57:d=1:alpha=1[maskedfaded];[0:v][maskedfaded]overlay[combined];[combined]tpad=start_mode=clone:start_duration=4:stop_mode=clone:stop_duration=4[out]" -map "[out]" -r 15 -crf 32 output.webm

After many iterations and a very late night, I got (roughly) the video I wanted, which I'm not posting here because of reasons. But it worked, yay! Now I don't have to manually place stars frame-by-frame in Krita, and I can just have all that magic happen semi-automatically.

Using Emacs and Python to record an animation and synchronize it with audio

| emacs, emacsconf, python

[2023-01-14 Sat]: Removed my fork since upstream now has the :eval function.

The Q&A session for Things I'd like to see in Emacs (Richard Stallman) from EmacsConf 2022 was done over Mumble. Amin pasted the questions into the Mumble chat buffer and I copied them into a larger buffer as the speaker answered them, but I didn't do it consistently. I figured it might be worth making another video with easier-to-read visuals. At first, I thought about using LaTeX to create Beamer slides with the question text, which I could then turn into a video using ffmpeg. Then I decided to figure out how to animate the text in Emacs, because why not? I figured a straightforward typing animation would probably be less distracting than animate-string, and emacs-director seems to handle that nicely. I forked it to add a few things I wanted, like variables to make the typing speed slower (so that it could more reliably type things on my old laptop, since sometimes the timers seemed to have hiccups) and an :eval step for running things without needing to log them. (2023-01-14: Upstream has the :eval feature now.)

To make it easy to synchronize the resulting animation with the chapter markers I derived from the transcript of the audio file, I decided to beep between scenes. First step: make a beep file.

ffmpeg -y -f lavfi -i 'sine=frequency=1000:duration=0.1' beep.wav

Next, I animated the text, with a beep between scenes. I used subed-parse-file to read the question text directly from the chapter markers, and I used simplescreenrecorder to set up the recording settings (including audio).

(defun my-beep ()
  (interactive)
  (save-window-excursion
    (shell-command "aplay ~/recordings/beep.wav &" nil nil)))

(require 'director)
(defvar emacsconf-recording-process nil)
(shell-command "xdotool getwindowfocus windowsize 1282 720")
(progn
  (switch-to-buffer (get-buffer-create "*Questions*"))
  (erase-buffer)
  (org-mode)
  (face-remap-add-relative 'default :height 300)
  (setq-local mode-line-format "   Q&A for EmacsConf 2022: What I'd like to see in Emacs (Richard M. Stallman) - emacsconf.org/2022/talks/rms")
  (sit-for 3)
  (delete-other-windows)
  (hl-line-mode -1)
  (when (process-live-p emacsconf-recording-process) (kill-process emacsconf-recording-process))
  (setq emacsconf-recording-process (start-process "ssr" (get-buffer-create "*ssr*")
                                                   "simplescreenrecorder"
                                                   "--start-recording"
                                                   "--start-hidden"))
  (sit-for 3)
  (director-run
   :version 1
   :log-target '(file . "/tmp/director.log")
   :before-start
   (lambda ()
     (switch-to-buffer (get-buffer-create "*Questions*"))
     (delete-other-windows))
   :steps
   (let ((subtitles (subed-parse-file "~/proj/emacsconf/rms/emacsconf-2022-rms--what-id-like-to-see-in-emacs--answers--chapters.vtt")))
     (apply #'append
            (list
             (list :eval '(my-beep))
             (list :type "* Q&A for Richard Stallman's EmacsConf 2022 talk: What I'd like to see in Emacs\nhttps://emacsconf.org/2022/talks/rms\n\n"))
            (mapcar
             (lambda (sub)
               (list
                (list :log (elt sub 3))
                (list :eval '(progn (org-end-of-subtree)
                                    (unless (bolp) (insert "\n"))))
                (list :type (concat "** " (elt sub 3) "\n\n"))
                (list :eval '(org-back-to-heading))
                (list :wait 5)
                (list :eval '(my-beep))))
             subtitles)))
   :typing-style 'human
   :delay-between-steps 0
   :after-end (lambda ()
                (process-send-string emacsconf-recording-process "record-save\nwindow-show\nquit\n"))
   :on-failure (lambda ()
                 (process-send-string emacsconf-recording-process "record-save\nwindow-show\nquit\n"))
   :on-error (lambda ()
               (process-send-string emacsconf-recording-process "record-save\nwindow-show\nquit\n"))))

I used the following code to copy the latest recording to animation.webm and extract the audio to animation.wav. my-latest-file and my-recordings-dir are in my Emacs config.

(let ((name "animation.webm"))
  (copy-file (my-latest-file my-recordings-dir) name t)
  (shell-command
   (format "ffmpeg -y -i %s -ar 8000 -ac 1 %s.wav"
           (shell-quote-argument name)
           (shell-quote-argument (file-name-sans-extension name)))))

Then I needed to get the timestamps of the beeps in the recording. I subtracted a little bit (0.82 seconds) based on comparing the waveform with the results.

filename = "animation.wav"
from scipy.io import wavfile
from scipy import signal
import numpy as np
import re
rate, source = wavfile.read(filename)
peaks = signal.find_peaks(source, height=1000, distance=1000)
base_times = (peaks[0] / rate) - 0.82
print(base_times)

I noticed that the first question didn't seem to get beeped properly, so I tweaked the times. Then I wrote some code to generate a very long ffmpeg command that used trim and tpad to select the segments and extend them to the right durations. There was some drift when I did it without the audio track, but the timestamps seemed to work right when I included the Q&A audio track as well.

import webvtt
import subprocess
chapters_filename =  "emacsconf-2022-rms--what-id-like-to-see-in-emacs--answers--chapters.vtt"
answers_filename = "answers.wav"
animation_filename = "animation.webm"
def get_length(filename):
    result = subprocess.run(["ffprobe", "-v", "error", "-show_entries",
                             "format=duration", "-of",
                             "default=noprint_wrappers=1:nokey=1", filename],
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT)
    return float(result.stdout)

def get_frames(filename):
    result = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "v:0", "-count_packets",
                             "-show_entries", "stream=nb_read_packets", "-of",
                             "csv=p=0", filename],
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT)
    return float(result.stdout)

answers_length = get_length(answers_filename)
# override base_times
times = np.asarray([  1.515875,  13.50, 52.32125 ,  81.368625, 116.66625 , 146.023125,
       161.904875, 182.820875, 209.92125 , 226.51525 , 247.93875 ,
       260.971   , 270.87375 , 278.23325 , 303.166875, 327.44925 ,
       351.616375, 372.39525 , 394.246625, 409.36325 , 420.527875,
       431.854   , 440.608625, 473.86825 , 488.539   , 518.751875,
       544.1515  , 555.006   , 576.89225 , 598.157375, 627.795125,
       647.187125, 661.10875 , 695.87175 , 709.750125, 717.359875])
fps = 30.0
times = np.append(times, get_length(animation_filename))
anim_spans = list(zip(times[:-1], times[1:]))
chapters = webvtt.read(chapters_filename)
if chapters[0].start_in_seconds == 0:
    vtt_times = [[c.start_in_seconds, c.text] for c in chapters]
else:
    vtt_times = [[0, "Introduction"]] + [[c.start_in_seconds, c.text] for c in chapters] 
vtt_times = vtt_times + [[answers_length, "End"]]
# Add ending timestamps
vtt_times = [[x[0][0], x[1][0], x[0][1]] for x in zip(vtt_times[:-1], vtt_times[1:])]
test_rate = 1.0

i = 0
concat_list = ""
groups = list(zip(anim_spans, vtt_times))
import ffmpeg
animation = ffmpeg.input('animation.webm').video
audio = ffmpeg.input('rms.opus')

for_overlay = ffmpeg.input('color=color=black:size=1280x720:d=%f' % answers_length, f='lavfi')
params = {"b:v": "1k", "vcodec": "libvpx", "r": "30", "crf": "63"}
test_limit = 1
params = {"vcodec": "libvpx", "r": "30", "copyts": None, "b:v": "1M", "crf": 24}
test_limit = 0
anim_rate = 1
import math
cursor = 0
if test_limit > 0:
    groups = groups[0:test_limit]
clips = []

# cursor is the current time
for anim, vtt in groups:
    padding = vtt[1] - cursor - (anim[1] - anim[0]) / anim_rate
    if (padding < 0):
        print("Squeezing", math.floor((anim[1] - anim[0]) / (anim_rate * 1.0)), 'into', vtt[1] - cursor, padding)
        clips.append(animation.trim(start=anim[0], end=anim[1]).setpts('PTS-STARTPTS')) 
    elif padding == 0:
        clips.append(animation.trim(start=anim[0], end=anim[1]).setpts('PTS-STARTPTS'))
    else:
        print("%f to %f: Padding %f into %f - pad: %f" % (cursor, vtt[1], (anim[1] - anim[0]) / (anim_rate * 1.0), vtt[1] - cursor, padding))
        cursor = cursor + padding + (anim[1] - anim[0]) / anim_rate
        clips.append(animation.trim(start=anim[0], end=anim[1]).setpts('PTS-STARTPTS').filter('tpad', stop_mode="clone", stop_duration=padding))
    for_overlay = for_overlay.overlay(animation.trim(start=anim[0], end=anim[1]).setpts('PTS-STARTPTS+%f' % vtt[0]))
    clips.append(audio.filter('atrim', start=vtt[0], end=vtt[1]).filter('asetpts', 'PTS-STARTPTS'))
args = ffmpeg.concat(*clips, v=1, a=1).output('output.webm', **params).overwrite_output().compile()
print(' '.join(f'"{item}"' for item in args))

Anyway, it's here for future reference. =)

View org source for this post

Collect my recent toots in an Org file so that I can refile them

| emacs, mastodon, org

I want to use my microblog posts on Mastodon as building blocks for longer posts on my blog. Getting them into an Org file makes it easier to link to them or refile them to other parts of my Org files so that I can build up my notes.

(use-package pandoc)
(defun my-mastodon-org-feed-formatter (entry)
  (concat "* " (pandoc-convert-stdio
                (dom-text (dom-by-tag
                           (with-temp-buffer
                             (insert "<item>"
                                     (plist-get entry :item-full-text)
                                     "</item>")
                             (xml-parse-region (point-min) (point-max)))
                           'description))
                "html" "org")
          "\n\n[" (format-time-string (cdr org-time-stamp-formats)
                                      (date-to-time (plist-get entry :pubDate)))
"]\n" (plist-get entry :link)))
(setq org-feed-alist '(("Mastodon" "https://emacs.ch/@sachac/with_replies.rss"
                        "~/sync/orgzly/toots.org" "Toots"
                        :formatter my-mastodon-org-feed-formatter)))
(defun my-org-feed-sort (pos entries)
  (save-excursion
    (goto-char pos)
    (when (looking-at org-complex-heading-regexp)
      (org-sort-entries nil ?T))))
(advice-add #'org-feed-add-items :after #'my-org-feed-sort)

Now I can use org-feed-update-all (C-c C-x g) to pull things into my toots.org file.

This is part of my Emacs configuration.

2022-12-19 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, lobste.rs, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, emacs-devel, and lemmy/c/emacs. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

2022-12-12 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, lobste.rs, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, emacs-devel, and lemmy/c/emacs. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!