[2024-01-12 Fri]: Added some code to display the QR code on the right side.
John Kitchin includes little QR codes in his videos. I
thought that was a neat touch that makes it easier for
people to jump to a link while they're watching. I'd like to
make it easier to show QR codes too. The following code lets
me show a QR code for the Org link at point. Since many of
my links use custom Org link types that aren't that useful
for people to scan, the code reuses the link resolution code
from https://sachachua.com/dotemacs#web-link so that I can get the regular
https: link.
(defunmy-org-link-qr (url)
"Display a QR code for URL in a buffer."
(let ((buf (save-window-excursion (qrencode--encode-to-buffer (my-org-stored-link-as-url url)))))
(display-buffer-in-side-window buf '((side . right)))))
(use-packageqrencode:config
(with-eval-after-load'embark
(define-key embark-org-link-map (kbd "q") #'my-org-link-qr)))
[2024-01-12 Fri] Added embark action to copy the exported link URL.
[2024-01-11 Thu] Switched to using Github links since Codeberg's down.
[2024-01-11 Thu] Updated my-copy-link to just return the link if called from Emacs Lisp. Fix getting the properties.
[2024-01-08 Mon] Add tip from Omar about embark-around-action-hooks
[2024-01-08 Mon] Simplify code by using consult--grep-position
Summary (882 words): Emacs macros make it easy to define sets of related functions for custom Org links. This makes it easier to link to projects and export or copy the links to the files in the web-based repos. You can also use that information to consult-ripgrep across lots of projects.
I'd like to get better at writing notes while coding and at turning
those notes into blog posts and videos. I want to be able to link to
files in projects easily with the ability to complete, follow, and
export links. For example, [[subed:subed.el]] should become
subed.el, which opens the file if I'm in Emacs and exports a
link if I'm publishing a post. I've been making custom link types
using org-link-set-parameters. I think it's time to make a macro
that defines that set of functions for me. Emacs Lisp macros are a
great way to write code to write code.
I've been really liking being able to refer to various emacsconf-el
files by just selecting the link type and completing the filename, so
maybe it'll be easier to write about lots of other stuff if I extend
that to my other projects.
Copy web link
[2024-01-19 Fri]: Add Wayback machine.
Keeping a list of projects and their web versions also makes it easier
for me to get the URL for something. I try to post as much as possible
on the Web so that it's easier for me to find things again and so that
other people can pick up ideas from my notes. Things are a bit
scattered: my blog, repositories on Github and Codeberg, my
sketches… I don't want to think about where the code has ended
up, I just want to grab the URL. If I'm going to put the link into an
Org Mode document, that's super easy. I just take advantage of the
things I've added to org-store-link. If I'm going to put it into an
e-mail or a toot or wherever else, I just want the bare URL.
I can think of two ways to approach this. One is a command that copies
just the URL by figuring it out from the buffer filename, which allows
me to special-case a bunch of things:
(defunmy-copy-link (&optional filename skip-links)
"Return the URL of this file.If FILENAME is non-nil, use that instead.If SKIP-LINKS is non-nil, skip custom links.If we're in a Dired buffer, use the file at point."
(interactive)
(setq filename (or filename
(if (derived-mode-p 'dired-mode) (dired-get-filename))
(buffer-file-name)))
(if-let*
((project-re (concat "\\(" (regexp-opt (mapcar 'car my-project-web-base-list)) "\\)""\\(.*\\)"))
(url (cond
((and (derived-mode-p 'org-mode)
(eq (org-element-type (org-element-context)) 'link)
(not skip-links))
(pcase (org-element-property :type (org-element-context))
((or"https""http")
(org-element-property :raw-link (org-element-context)))
("yt"
(org-element-property :path (org-element-context)))
;; if it's a custom link, visit it and get the link
(_
(save-window-excursion
(org-open-at-point)
(my-copy-link nil t)))))
;; links to my config usually have a CUSTOM_ID property
((string= (buffer-file-name) (expand-file-name "~/sync/emacs/Sacha.org"))
(concat "https://sachachua.com/dotemacs#" (org-entry-get-with-inheritance "CUSTOM_ID")))
;; blog post drafts have permalinks
((and (derived-mode-p 'org-mode) (org-entry-get-with-inheritance "EXPORT_ELEVENTY_PERMALINK"))
(concat "https://sachachua.com" (org-entry-get-with-inheritance "EXPORT_ELEVENTY_PERMALINK")))
;; some projects have web repos
((string-match
project-re filename)
(concat (assoc-default (match-string 1 filename) my-project-web-base-list)
(url-hexify-string (match-string 2 filename)))))))
(progn
(when (called-interactively-p 'any)
(kill-new url)
(message "%s" url))
url)
(error"Couldn't figure out URL.")))
Another approach is to hitch a ride on the Org Mode link storage and
export functions and just grab the URL from whatever link I've stored
with org-store-link, which I've bound to C-c l. I almost always
have an HTML version of the exported link. We can even use XML parsing
instead of regular expressions.
(defunmy-org-link-as-url (link)
"Return the final URL for LINK."
(dom-attr
(dom-by-tag
(with-temp-buffer
(insert (org-export-string-as link 'html t))
(xml-parse-region (point-min) (point-max)))
'a)
'href))
(defunmy-org-stored-link-as-url (&optional link insert)
"Copy the stored link as a plain URL.If LINK is specified, use that instead."
(interactive (list nil current-prefix-arg))
(setq link (or link (caar org-stored-links)))
(let ((url (if link
(my-org-link-as-url link)
(error"No stored link"))))
(when (called-interactively-p 'any)
(if url
(if insert (insert url) (kill-new url))
(error"Could not find URL.")))
url))
(ert-deftestmy-org-stored-link-as-url ()
(should
(string= (my-org-stored-link-as-url "[[dotemacs:web-link]]")
"https://sachachua.com/dotemacs#web-link"))
(should
(string= (my-org-stored-link-as-url "[[dotemacs:org-mode-sketch-links][my Org Mode sketch links]]")
"https://sachachua.com/dotemacs#org-mode-sketch-links")))
(defunmy-embark-org-copy-exported-url-as-wayback (link)
(interactive"MLink: ")
(my-embark-org-copy-exported-url link t))
(defunmy-embark-org-copy-exported-url (link &optional wayback)
(interactive"MLink: \np")
(let ((url (my-org-link-as-url link)))
(when (and (derived-mode-p 'org-mode)
(org-entry-get-with-inheritance "EXPORT_ELEVENTY_PERMALINK")
(string-match "^/" url))
;; local file links are copied to blog directories
(setq url (concat "https://sachachua.com"
(org-entry-get-with-inheritance "EXPORT_ELEVENTY_PERMALINK")
(replace-regexp-in-string
"[\\?&].*"""
(file-name-nondirectory link)))))
(when (and wayback (not (string-match (regexp-quote "^https://web.archive.org") url)))
(setq url (concat "https://web.archive.org/web/" (format-time-string "%Y%m%d%H%M%S/")
url)))
(kill-new url)
(message "Copied %s" url)))
(with-eval-after-load'embark-org
(define-key embark-org-link-map
"u"#'my-embark-org-copy-exported-url)
(define-key embark-org-link-map
"U"#'my-embark-org-copy-exported-url-as-wayback)
(define-key embark-org-link-copy-map
"u"#'my-embark-org-copy-exported-url)
(define-key embark-org-link-copy-map
"U"#'my-embark-org-copy-exported-url-as-wayback))
We'll see which one I end up using. I think both approaches might come in handy.
Quickly search my code
Since my-project-web-base-list is a list of projects I often think
about or write about, I can also make something that searches through
them. That way, I don't have to care about where my code is.
I can add .rgignore files in directories to tell ripgrep to ignore
things like node_modules or *.json.
I also want to search my Emacs configuration at the same time,
although links to my config are handled by my dotemacs link type so
I'll leave the URL as nil. This is also the way I can handle other
unpublished directories.
Actually, let's throw my blog posts and Org files in there as well,
since I often have code snippets. If it gets to be too much, I can
always have different commands search different things.
At some point, it might be fun to get Embark set up so that I can grab
a link to something right from the consult-ripgrep interface. In the
meantime, I can always jump to it and get the link.
Tip from Omar: embark-around-action-hooks
[2024-01-07 Sun] I modified oantolin's suggestion from the comments to work with consult-ripgrep, since consult-ripgrep gives me consult-grep targets instead of consult-location:
(cl-defunembark-consult--at-location (&rest args &key target type run &allow-other-keys)
"RUN action at the target location."
(save-window-excursion
(save-excursion
(save-restriction
(pcase type
('consult-location (consult--jump (consult--get-location target)))
('org-heading (org-goto-marker-or-bmk (get-text-property 0 'org-marker target)))
('consult-grep (consult--jump (consult--grep-position target)))
('file (find-file target)))
(apply run args)))))
(cl-pushnew#'embark-consult--at-location (alist-get 'org-store-link embark-around-action-hooks))
I think I can use it with M-s c to search for the code, then C-.
C-c l on the matching line, where C-c l is my regular keybinding
for storing links. Thanks, Omar!
In general, I don't want to have to think about where something is on
my laptop or where it's published on the Web, I just want to
It's nice to feel like you're saying someone's name correctly. We ask
EmacsConf speakers to introduce themselves in the first few seconds of
their video, but people often forget to do that, so that's okay. We
started recording introductions for EmacsConf 2022 so that stream
hosts don't have to worry about figuring out pronunciation while
they're live. Here's how I used subed-record to turn my recordings
into lots of little videos.
First, I generated the title images by using Emacs Lisp to replace
text in a template SVG and then using Inkscape to convert the SVG into
a PNG. Each image showed information for the previous talk as well as
the upcoming talk. (emacsconf-stream-generate-in-between-pages)
Then I generated the text for each talk based on the title, the
speaker names, pronunciation notes, pronouns, and type of Q&A. Each
introduction generally followed the pattern, "Next we have title by
speakers. Details about Q&A." (emacsconf-pad-expand-intro and
emacsconf-subed-intro-subtitles below)
00:00:00.000 --> 00:00:00.999
#+OUTPUT: sat-open.webm
[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/sat-open.svg.png]]
Next, we have "Saturday opening remarks".
00:00:05.000 --> 00:00:04.999
#+OUTPUT: adventure.webm
[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/adventure.svg.png]]
Next, we have "An Org-Mode based text adventure game for learning the basics of Emacs, inside Emacs, written in Emacs Lisp", by Chung-hong Chan. He will answer questions via Etherpad.
I copied the text into an Org note in my inbox, which Syncthing copied
over to the Orgzly Revived app on my Android phone. I used Google
Recorder to record the audio. I exported the m4a audio file and a
rough transcript, copied them back via Syncthing, and used
subed-record to edit the audio into a clean audio file without
oopses.
Each intro had a set of captions that started with a NOTE comment.
The NOTE comment specified the following:
#+AUDIO:: the audio source to use for the timestamped captions
that follow
[[file:...]]: the title image I generated for each talk. When
subed-record-compile-video sees a comment with a link to an image,
video, or animated GIF, it takes that visual and uses it for the
span of time until the next visual.
#+OUTPUT: the file to create.
NOTE #+OUTPUT: hyperdrive.webm[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/hyperdrive.svg.png]]#+AUDIO: intros-2023-11-21-cleaned.opus00:00:15.680-->00:00:17.599
Next, we have "hyperdrive.el:
00:00:17.600-->00:00:21.879
Peer-to-peer filesystem in Emacs", by Joseph Turner
00:00:21.880-->00:00:25.279
and Protesilaos Stavrou (also known as Prot).
00:00:25.280-->00:00:27.979
Joseph will answer questions via BigBlueButton,
00:00:27.980-->00:00:31.080
and Prot might be able to join depending on the weather.
00:00:31.081-->00:00:33.439
You can join using the URL from the talk page
00:00:33.440-->00:00:36.320
or ask questions through Etherpad or IRC.
NOTE#+OUTPUT: steno.webm[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/steno.svg.png]]#+AUDIO: intros-2023-11-19-cleaned.opus00:03:23.260-->00:03:25.480
Next, we have "Programming with steno",
00:03:25.481-->00:03:27.700
by Daniel Alejandro Tapia.
NOTE#+AUDIO: intro-2023-11-29-cleaned.opus00:00:13.620-->00:00:16.580
You can ask your questions via Etherpad and IRC.
00:00:16.581-->00:00:18.079
We'll send them to the speaker
00:00:18.080-->00:00:19.919
and post the answers in the talk page
00:00:19.920-->00:00:21.320
after the conference.
I could then call subed-record-compile-video to create the videos
for all the intros, or mark a region with C-SPC and then
subed-record-compile-video only the intros inside that region.
Using Emacs to edit the audio and compile videos worked out really
well because it made it easy to change things.
Changing pronunciation or titles: For EmacsConf 2023, I got the
recordings sorted out in time for the speakers to correct my
pronunciation if they wanted to. Some speakers also changed their
talk titles midway. If I wanted to redo an intro, I just had to
rerecord that part, run it through my subed-record audio cleaning
process, add an #+AUDIO: comment specifying which file I want to
take the audio from, paste it into my main intros.vtt, and
recompile the video.
Cancelling talks: One of the talks got cancelled, so I needed to
update the images for the talk before it and the talk after it. I
regenerated the title images and recompiled the videos. I didn't
even need to figure out which talk needed to be updated - it was easy
enough to just recompile all of them.
Changing type of Q&A: For example, some speakers needed to switch
from answering questions live to answering them after the
conference. I could just delete the old instructions, paste in the
instructions from elsewhere in my intros.vtt (making sure to set
#+AUDIO to the file if it came from a different take), and
recompile the video.
And of course, all the videos were captioned. Bonus!
So that's how using Emacs to edit and compile simple videos saved me a
lot of time. I don't know how I'd handle this otherwise. 47 video
projects that might all need to be updated if, say, I changed the
template? Yikes. Much better to work with text. Here are the technical
details.
Generating the title images
I used Inkscape to add IDs to our template SVG so that I could edit
them with Emacs Lisp. From emacsconf-stream.el:
emacsconf-stream-generate-in-between-pages: Generate the title images.
emacsconf-stream-svg-set-text: Update DOM to set the tspan in the element with ID to TEXT.
(defunemacsconf-stream-svg-set-text (dom id text)
"Update DOM to set the tspan in the element with ID to TEXT.If the element doesn't have a tspan child, use the element itself."
(if (or (null text) (string= text ""))
(let ((node (dom-by-id dom id)))
(when node
(dom-set-attribute node 'style"visibility: hidden")
(dom-set-attribute (dom-child-by-tag node 'tspan) 'style"fill: none; stroke: none")))
(setq text (svg--encode-text text))
(let ((node (or (dom-child-by-tag
(car (dom-by-id dom id))
'tspan)
(dom-by-id dom id))))
(cond
((null node)
(error"Could not find node %s" id)) ; skip
((= (length node) 2)
(nconc node (list text)))
(t (setf (elt node 2) text))))))
emacsconf-pad-expand-intro: Make an intro for TALK.
(defunemacsconf-pad-expand-intro (talk)
"Make an intro for TALK."
(cond
((null (plist-get talk :speakers))
(format "Next, we have \"%s\"." (plist-get talk :title)))
((plist-get talk :intro-note)
(plist-get talk :intro-note))
(t
(let ((pronoun (pcase (plist-get talk :pronouns)
((rx"she") "She")
((rx"\"ou\"""Ou"))
((or'nil"nil" (rx string-start "he") (rx"him")) "He")
((rx"they") "They")
(_ (or (plist-get talk :pronouns) "")))))
(format "Next, we have \"%s\", by %s%s.%s"
(plist-get talk :title)
(replace-regexp-in-string ", \\([^,]+\\)$"", and \\1"
(plist-get talk :speakers))
(emacsconf-surround " (" (plist-get talk :pronunciation) ")""")
(pcase (plist-get talk :q-and-a)
((or'nil"") "")
((rx"after") " You can ask questions via Etherpad and IRC. We'll send them to the speaker, and we'll post the answers on the talk page afterwards.")
((rx"live")
(format " %s will answer questions via BigBlueButton. You can join using the URL from the talk page or ask questions through Etherpad or IRC."
pronoun
))
((rx"pad")
(format " %s will answer questions via Etherpad."
pronoun
))
((rx"IRC")
(format " %s will answer questions via IRC in the #%s channel."
pronoun
(plist-get talk :channel)))))))))
Overall notes in Emacs with outline, org-timer timestamped notes; capture to this file
Elisp to start/stop the stream → find old code
Use the Yeti? Better sound
tee to a local recording
grab screenshot from SuperNote mirror?
Live streaming info density:
High: Emacs News review, package/workflow demo
Narrating a blog post to make it a video
Categorizing Emacs News, exploring packages
Low: Figuring things out
YouTube can do closed captions for livestreams, although accuracy is
low. Videos take a while to be ready to download.
Experimenting with working out loud
I wanted to write a report on EmacsConf 2023 so that we could share it
with speakers, volunteers, participants, donors, related organizations
like the Free Software Foundation, and other communities. I
experimented with livestreaming via YouTube while I worked on the
conference highlights.
It's a little over an hour long and probably very boring, but it was
nice of people to drop by and say hello.
The main parts are:
0:00: reading through other conference reports for inspiration
49:00: fiddling with the formatting and the export
It mostly worked out, aside from a brief moment of "uhhh, I'm
looking at our private conf.org file on stream". Fortunately, the
e-mail addresses that were showed were the public ones.
Technical details
Setup:
I set up environment variables and screen resolution:
I switch to a larger size and a light theme. I also turn consult previews off to minimize the risk of leaking data through buffer previews.
my-emacsconf-prepare-for-screenshots: Set the resolution, change to a light theme, and make the text bigger.
I can think of a few workflow tweaks that might be fun:
a stream notes buffer on the right side of the screen for context
information, timestamped notes to make editing/review easier (maybe
using org-timer), etc. I experimented with some streaming-related
code in my config, so I can dust that off and see what that's like.
I also want to have an org-capture template for it so that I can add
notes from anywhere.
I think I'll try going through an informal presentation or Emacs News as my next livestream experiment, since that's probably higher information density.
I want to get better at looking in my Org files for something that I
don't exactly remember. I might remember a few words from it but not
in order, or I might remember some words from the body, or I might
need to fiddle with the keywords until I find it.
I usually use C-u C-c C-w (org-refile with a prefix argument),
counting on consult + orderless to let me just put in keywords in any
order. This doesn't let me search the body, though.
org-ql seems like a great fit for this. It's fast and flexible, and
might be useful for all sorts of queries.
I think by default org-ql matches against all of the text in the
entry. You can scope the match to just the heading with a query like
heading:your,text. I wanted to see all matches, prioritize heading
matches so that they come first. I thought about saving the query by
adding advice before org-ql-search and then adding a new comparator
function, but that got a bit complicated, so I haven't figured that
out yet. It was easier to figure out how to rewrite the query to use
heading instead of rifle, do the more constrained query, and then
append the other matches that weren't in the heading matches.
Also, I wanted something a little like helm-org-rifle's live
previews. I've used helm before, but I was curious about getting it to
work with consult.
Here's a quick demo of my-consult-org-ql-agenda-jump, which I've
bound to M-s a. The top few tasks have org-ql in the heading, and
they're followed by the rest of the matches. I think this might be handy.
(defunmy-consult-org-ql-agenda-jump ()
"Search agenda files with preview."
(interactive)
(let* ((marker (consult--read
(consult--dynamic-collection
#'my-consult-org-ql-agenda-match)
:state (consult--jump-state)
:category'consult-org-heading:prompt"Heading: ":sort nil
:lookup#'consult--lookup-candidate))
(buffer (marker-buffer marker))
(pos (marker-position marker)))
;; based on org-agenda-switch-to
(unless buffer (user-error"Trying to switch to non-existent buffer"))
(pop-to-buffer-same-window buffer)
(goto-char pos)
(when (derived-mode-p 'org-mode)
(org-fold-show-context 'agenda)
(run-hooks 'org-agenda-after-show-hook))))
(defunmy-consult-org-ql-agenda-format (o)
(propertize
(org-ql-view--format-element o)
'consult--candidate (org-element-property :org-hd-marker o)))
(defunmy-consult-org-ql-agenda-match (string)
"Return candidates that match STRING.Sort heading matches first, followed by other matches.Within those groups, sort by date and priority."
(let* ((query (org-ql--query-string-to-sexp string))
(sort '(date reverse priority))
(heading-query (-tree-map (lambda (x) (if (eq x 'rifle) 'heading x)) query))
(matched-heading
(mapcar #'my-consult-org-ql-agenda-format
(org-ql-select 'org-agenda-files heading-query
:action'element-with-markers:sort sort)))
(all-matches
(mapcar #'my-consult-org-ql-agenda-format
(org-ql-select 'org-agenda-files query
:action'element-with-markers:sort sort))))
(append
matched-heading
(seq-difference all-matches matched-heading))))
(use-packageorg-ql:bind ("M-s a" . my-consult-org-ql-agenda-jump))
Along the way, I learned how to use consult to complete using
consult--dynamic-collection and add consult--candidate so that I
can reuse consult--lookup-candidate and consult--jump-state. Neat!
Someday I'd like to figure out how to add a sorting function and sort
by headers without having to reimplement the other sorts. In the
meantime, this might be enough to help me get started.
I wanted to experiment with for colouring the mode line of the active window ever so slightly different to make it easier to see where the active window is. I usually have global-hl-line-mode turned on, so that highlight is another indicator, but let's see how this tweak feels. I modified the code so that it uses the theme colours from the currently-selected Modus themes, since I trust Prot's colour choices more than I trust mine. Thanks to Irreal for sharing Ignacio's comment!