diff options
Diffstat (limited to 'Documentation/piem.texi')
| -rw-r--r-- | Documentation/piem.texi | 599 |
1 files changed, 599 insertions, 0 deletions
diff --git a/Documentation/piem.texi b/Documentation/piem.texi new file mode 100644 index 0000000..aa48705 --- /dev/null +++ b/Documentation/piem.texi @@ -0,0 +1,599 @@ +\input texinfo @c -*-texinfo-*- + +@set VERSION 0.2.0 (unreleased) + +@setfilename piem.info +@documentencoding UTF-8 +@documentlanguage en +@settitle Emacs tools and glue for working with public-inbox archives + +@copying +Copyright @copyright{} 2020--2021 all contributors <piem@@inbox.kyleam.com> + +Permission is granted to copy, distribute and/or modify this document +under the terms of the GNU Free Documentation License, Version 1.3 or +any later version published by the Free Software Foundation; with no +Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A +copy of the license is included in the section entitled ``GNU Free +Documentation License''. +@end copying + +@dircategory Emacs +@direntry +* piem: (piem). Emacs tools and glue for working with public-inbox archives +@end direntry + +@finalout +@titlepage +@title piem reference manual +@subtitle for version @value{VERSION} +@author Kyle Meyer +@page +@vskip 0pt plus 1filll +@insertcopying +@end titlepage + +@contents +@* + +@ifnottex +@node Top +@top piem + +This manual is for piem version @value{VERSION}. +@end ifnottex + + +@menu +* Overview:: +* Getting started:: +* Applying patches:: +* Miscellaneous functionality:: +* Contributing:: +* Related projects and tools:: + +Appendices +* GNU Free Documentation License:: The license for this documentation. + +Indices +* Key Index:: +* Variable Index:: +* Lisp Function Index:: +* Concept Index:: +@end menu + + +@node Overview +@chapter Overview + +piem is a collection of Emacs libraries for working with public-inbox +archives. As much of the hard work here is already done by other Emacs +libraries---things like mail clients, news readers, Git interfaces, and +even web browsers---piem is mostly about bridging some of these parts +for convenience. + +@node public-inbox +@section public-inbox +@cindex public-inbox +@cindex lore + +@url{https://public-inbox.org/README.html,public-inbox} is software for +archiving public mailing lists. Archives can be exposed over HTTP. As +examples, @url{https://public-inbox.org/meta} serves public-inbox's own +mailing list, and @url{https://lore.kernel.org/lists.html} hosts the +archives of many Linux development mailing lists. + +@cindex pull methods +These web archives are good for searching, particularly if you don't +have all of the list's messages on your local machine, or for linking to +a message. On the other hand, the web interface isn't convenient when +you want to follow new activity on a list. To do that, you could of +course subscribe to the mailing list, but public-inbox offers a few +``pull methods'' that you can use instead: + +@itemize +@item +an atom feed for the list as a whole or for specific searches +@item +read-only NNTP +@item +read-only IMAP (new in upcoming public-inbox v1.6.0) +@end itemize + +Finally, archives are exposed as one or more Git repositories, +facilitating replication (see +@url{https://public-inbox.org/reproducibility.html}). For example, you +can clone the mailing list archives of @samp{git.vger.kernel.org} with + +@example +git clone --mirror https://lore.kernel.org/git/0 git/git/0.git +@end example + +@noindent +After the initial clone, new messages can be retrieved with +@code{git fetch}. Unsurprisingly @code{git log} is not a pleasant way +to read a mailing list; instead this method is useful for mirroring the +archive or bulk importing of the messages. (See +@url{https://public-inbox.org/clients.html} for a list of some tools +designed to work with public-inbox archives.) + + +@node Getting started +@chapter Getting started +@findex piem-dispatch + +@code{piem-dispatch} transient +(see +@ifinfo +@ref{Top,,,transient} +@end ifinfo +@ifnotinfo +@url{https://magit.vc/manual/transient/} +@end ifnotinfo +) +provides an entry point to piem commands. +It's recommended to bind @code{piem-dispatch} to a key. However, before +most of those commands do anything useful, you need to register inboxes +and activate at least one minor mode. + +@node Registering inboxes +@section Registering inboxes +@cindex coderepo +@cindex inbox +@vindex piem-inboxes + +A public-inbox archive, referred to as an @dfn{inbox}, is registered by +adding an entry to @code{piem-inboxes}. Here's an example entry for the +Git project's mailing list: + +@lisp +("git" + :url "https://lore.kernel.org/git/" + :address "git@@vger.kernel.org" + :listid "git.vger.kernel.org" + :coderepo "~/src/git/") +@end lisp + +@noindent +The first element is a name for the inbox and will typically match the +name at the end of the @code{:url} value. Specifying either +@code{:listid} or @code{:address} is important so that a message in a +buffer can be mapped to an inbox in @code{piem-inboxes}. + +@code{:coderepo} points to a local Git repository that contains code +related to that archive (in the example above, a local clone of +@url{https://git.kernel.org/pub/scm/git/git.git/}). This information is +required to apply patches from an archive to a local code repository +(@pxref{Applying patches}). + +@node Enabling integration libraries +@section Enabling integration libraries +@findex piem-elfeed-mode +@findex piem-eww-mode +@findex piem-gnus-mode +@findex piem-notmuch-mode + +With inboxes defined, the next step is to enable minor modes that teach +particular Emacs modes to link a buffer with a registered inbox. piem +currently has libraries to support + +@itemize +@item EWW +@item Elfeed +@item Gnus +@item Notmuch +@end itemize + +For example, if you use notmuch.el to read your mail, you can add +support for applying patches from a Notmuch message buffer by enabling +@code{piem-notmuch-mode} (@pxref{Applying patches}): + +@lisp +(piem-notmuch-mode 1) +@end lisp + +Help adding support for other modes, especially other mail clients, is +welcome. + + +@node Applying patches +@chapter Applying patches +@cindex am-ready mbox +@cindex applying patches +@cindex git-am + +With @code{piem-inboxes} configured and appropriate integration +libraries enabled, a buffer that can be linked to an inbox can be mapped +to a code repository. When reading a message in a +@code{notmuch-show-mode} buffer, for example, the list ID can be used to +identify the inbox and thus the associated local code repository. + +There are two commands for applying patches: + +@table @code + +@item piem-am +@findex piem-am +This command tries to extract a patch from the current Notmuch or Gnus +message buffer and can handle an inline patch as well as one or more +patch attachments. + +@item piem-b4-am +@findex piem-b4-am +This command relies on the b4 command-line tool to do more sophisticated +processing of the @emph{full thread} (e.g., pulling out the latest +reroll of a series) to generate an mbox that can be fed to @code{git +am}. It is only compatible with inline patches. + +@end table + +@node Applying patches contained in a message +@section Applying patches contained in a message + +@table @kbd +@findex piem-am +@item M-x piem-am @key{RET} @var{branch} @key{RET} @var{base} +Apply the patch or patches in the current buffer to the associated code +repository. Before applying, checkout a new branch @var{branch} +starting at @var{base}. +@end table + +@findex piem-name-branch-who-what-v +@vindex piem-default-branch-function +You'll be queried for the name of the new branch. The default name +offered is generated by @code{piem-name-branch-who-what-v}, which uses +the @samp{From:} and @samp{Subject:} headers to construct branch names +like @samp{km/b4-short-subj__v3}. To use a different function to +generate the completion default, configure +@code{piem-default-branch-function}. + +Next you'll be queried for the base to use as the starting point for the +branch. If the sender specified a base commit for the series, that will +be provided as the default completion candidate. Entering an empty base +signals to use the current branch of the repository as the base. + +@vindex piem-am-create-worktree +@vindex piem-am-read-worktree-function +Rather than applying the patches directly to the associated code +repository, you can create a dedicated worktree by setting +@code{piem-am-create-worktree} to a non-nil value. Giving a prefix +argument to @code{piem-am} inverts the meaning of +@code{piem-am-create-worktree}; that is, by default a prefix argument is +useful if you generally prefer to work within the configured code +repository but would like to trigger the one-off creation of a worktree +for a particular call. + +@cindex magit +@vindex piem-use-magit +When piem loads, it detects whether Magit is loaded and sets +@code{piem-use-magit} accordingly. If that option is non-nil, piem uses +Magit for some operations, particularly those that are user-facing. +This includes jumping to the Magit status buffer for a code repository +after applying a patch. + +@findex piem-am-ready-mbox +Note that the @code{piem-am} command works only for buffers from which +@code{piem-am-ready-mbox} can generate an am-ready mbox, which depends +on the enabled integration libraries. Currently @code{piem-notmuch} and +@code{piem-gnus} implement the necessary functionality. + +@node Using b4 to apply patches +@section Using b4 to apply patches +@cindex b4 +@cindex lore + +@url{https://git.kernel.org/pub/scm/utils/b4/b4.git,b4} is a +command-line tool for interacting with public-inbox archives. While +useful for public-inbox archives in general, it is written for Linux +kernel development and focuses on the public-inbox archives hosted at +@url{https://lore.kernel.org}. + +It's a fast moving target at the moment, but some of its current +capabilities include + +@itemize +@item +downloading the mbox for a thread based on a given message ID +@item +extracting patches from a thread's mbox that can be fed to @code{git am} +@item +submitting and verifying cryptographic attestation for patches +@item +fetching a pull request found in a message ID +@item +generating a thanks email for patches +@end itemize + +@noindent +The second item is the focus for piem, though at least some degree of +support for all of the above features will likely be added. +The entry point to applying patches with b4 is the @code{piem-b4-am} +transient. (See +@ifinfo +@ref{Top,,,transient} +@end ifinfo +@ifnotinfo +@url{https://magit.vc/manual/transient/} +@end ifnotinfo +for more information on using Transient.) + +@findex piem-b4-am +@code{piem-b4-am} offers the following actions: + +@table @kbd + +@item a +@itemx M-x piem-b4-am-from-mid +@findex piem-b4-am-from-mid +@findex piem-mid +@vindex piem-am-create-worktree +@vindex piem-am-read-worktree-function +Generate or download a thread's mbox for the current buffer's message +ID, process it into an am-ready mbox with b4, and then feed it to +@code{git am} called within an associated Git repository. If a message +ID of the current buffer is not known (i.e. @code{piem-mid} returns +nil), one is read from the caller. The caller is also queried for the +branch name and base, as described for @code{piem-am} (@pxref{Applying +patches contained in a message}). And, as with @code{piem-am}, a +worktree can be created by configuring @code{piem-am-create-worktree} to +a non-nil value or by giving a prefix argument. + +@findex piem-mid-to-thread-functions +To generate the input thread, first any functions in +@code{piem-mid-to-thread-functions} are tried. This allows for a thread +to be retrieved from a local store (e.g., the Notmuch database). If +that fails, the thread is downloaded from the public-inbox URL +associated with the current buffer. Finally, if an inbox's entry in +@code{piem-inboxes} doesn't specify a URL, @code{b4 am} is called +without a local mbox, letting it download the thread according to its +own configuration. + +@item i +@itemx M-x piem-b4-am-ready-from-mid +@findex piem-b4-am-ready-from-mid +Call @code{b4 am} with a given message ID. This differs from +@code{piem-b4-am-from-mid} in that it is a direct wrapper around a +command-line call to @code{b4 am}. The caller is always queried for the +message ID, and the final product is an am-ready mbox. @code{b4} is +responsible for downloading the thread, so the caller must point b4's +configuration option @code{b4.midmask} to the appropriate public-inbox +URL. + +@item b +@itemx M-x piem-b4-am-ready-from-mbox +@findex piem-b4-am-ready-from-mbox +Like @code{piem-b4-am-ready-from-mid}, but process a local mbox rather +than identifying the thread based on the specified message ID. + +@end table + +@node Applying patches without a public-inbox archive +@section Applying patches without a public-inbox archive + +Much of the functionality described in the previous sections can work +even if messages aren't available in a public-inbox archive. +@code{piem-am} and @code{piem-b4-am-from-mid} try to generate the +am-ready mbox from a local source (e.g., via Notmuch or Gnus) before +falling back to downloading the thread from a public-inbox archive. + +@cindex mailscripts +Also, for those not working with public-inbox archives, it's worth +checking out @url{https://git.spwhitton.name/mailscripts/,mailscripts}, +a nice set of Debian-focused tools by Sean Whitton that provides, among +other things, functionality for applying patch series, including +b4-inspired patch extraction. + + +@node Miscellaneous functionality +@chapter Miscellaneous functionality + +@node Injecting messages into a Maildir directory +@section Injecting messages into a Maildir directory +@cindex Maildir + +public-inbox allows you to follow lists through several mechanisms +(@pxref{public-inbox}). You may prefer different methods for different +projects depending on things like how actively you are following the +development and how high traffic the list is. For a project you +maintain, perhaps you want to receive every message as regular mail. +For a project you actively follow and occasionally contribute to, you +may prefer to not clutter your local mail store and instead follow via +read-only NNTP or IMAP in Gnus (which may or may not be your MUA). And +for a project you're new to or are digging into for a particular reason, +HTTP via EWW may be all you need. + +@findex piem-inject-thread-into-maildir +@vindex piem-maildir-directory +Depending on your mail setup, a problem with this approach is that it +can be inconvenient to start participating in a thread that you aren't +reading in your regular MUA (e.g., if you use notmuch.el to read your +regular mail but are following a project via NNTP in Gnus). In this +case, you can use the command @code{piem-inject-thread-into-maildir} to +move the thread's messages into a local Maildir directory +(@code{piem-maildir-directory}). By default the command downloads the +entire thread for the message ID associated with the current buffer. A +prefix argument restricts the download to only the message. + +@vindex piem-after-mail-injection-functions +After the messages are injected, each function in +@code{piem-after-mail-injection-functions} is called with the message ID +that was used to identify the thread. This can be used to pop to the +message in your mail client. For example, Notmuch users may want +something like this: + +@lisp +(defun my/notmuch-new-and-show (mid) + (message "Running notmuch new") + (call-process notmuch-command nil nil nil "new") + (notmuch-show (concat "id:" mid))) + +(add-hook 'piem-after-mail-injection-functions + #'my/notmuch-new-and-show) +@end lisp + +@vindex piem-mail-injection-skipif-predicate +@findex piem-notmuch-known-mid-p +To prevent duplicate messages from being written on subsequent calls to +@code{piem-inject-thread-into-maildir}, you can set +@code{piem-mail-injection-skipif-predicate} to a function that returns +non-nil if a message ID is known and should be skipped. For Notmuch, +@code{piem-notmuch} provides a function that works for this purpose, +@code{piem-notmuch-known-mid-p}: + +@lisp +(setq piem-mail-injection-skipif-predicate + #'piem-notmuch-known-mid-p) +@end lisp + +@node Copying public-inbox URLs +@section Copying public-inbox URLs + +@findex piem-copy-mid-url +When referring to a message from a public-inbox archive, a common format +to use is a URL that points to a specific archive and ends with +@code{/$INBOX/$MESSAGE_ID}, e.g., +@url{https://public-inbox.org/meta/20190108015420.GA28903@@dcvr}. +Calling @code{piem-copy-mid-url} (available in the @code{piem-dispatch} +transient) constructs such a URL, using the message ID and inbox +asscociated with the current buffer, and then copies the URL to the kill +ring. When a prefix agument is given, @code{browse-url} is called after +copying the URL. + +@vindex piem-browse-url-browser-function +Note that EWW works nicely with public-inbox's HTTP interface. If you'd +prefer it to be invoked even though it's not your default browser (as +configured by @code{browse-url-browser-function}), you can set +@code{piem-browse-url-browser-function} to @code{eww-browse-url}. + +@findex piem-notmuch-mode +@findex piem-notmuch-show-get-public-inbox-link +For notmuch.el users, there's an additional entry point for copying +public-inbox URLs: enabling @code{piem-notmuch-mode} adds a ``piem'' +candidate to archives offered by +@code{notmuch-show-stash-mlarchive-link} and +@code{notmuch-show-stash-mlarchive-link-and-go}. + + +@node Contributing +@chapter Contributing + +Patches, bug reports, and other feedback are welcome. Please send a +plain-text email to @email{piem@@inbox.kyleam.com}. Messages that +include this address are public and available as public-inbox archives +at @url{https://inbox.kyleam.com/piem}. Note that this is not a mailing +list. Updates can be followed through one of public-inbox's pull +methods (@pxref{public-inbox}). This means it is particularly important +to @emph{not} drop participants when replying. + +You can, unsurprisingly, use piem to work on piem by adding an entry +like this to @code{piem-inboxes}. + +@lisp +("piem" + :coderepo "<path/to/local/clone>" + :address "piem@@inbox.kyleam.com" + :url "https://inbox.kyleam.com/piem/") +@end lisp + +The source repository is available at @url{https://git.kyleam.com/piem}. +Here are some guidelines for sending patches: + +@itemize +@item Please send patches inline rather than as attachments. + +If you're using @code{git send-email}, you may want to set +@code{sendemail.to} to @code{piem@@inbox.kyleam.com} in your local +repository. + +@item Specify the base commit. + +This can be done via the @code{--base=} option of @code{git +format-patch} or by configuring @code{format.useAutoBase}. + +@item Keep rerolls in the same thread. + +In general, prefer to keep iterations of a patch series in the same +thread, labeling rerolls with an appropriate version. + +@item Consider adding a range-diff to the cover letter of rerolls. + +For a multi-patch series, @code{git range-diff} can often provide a nice +overview of the changes between the previous and last iteration. Note +that @code{git format-patch} has a @code{--range-diff} convenience +option. + +@end itemize + + +@node Related projects and tools +@chapter Related projects and tools + +Here's a (short and incomplete) list of tools that you, as someone that +cared to look at piem's manual, may be interested in---things that +either have some connection to public-inbox or to using an email-based +development workflow in Emacs. Corrections and additions are welcome. + +@itemize + +@item +git-email provides an Emacs interface for preparing and sending patches. + +@url{https://sr.ht/~yoctocell/git-email/} + +@cindex magit +@item +@code{git range-diff} is a built-in Git command for comparing iterations +of a patch series. If you use Magit, the +@url{https://github.com/magit/magit-tbdiff,magit-tbdiff} extension +provides an interface for @code{git range-diff}. + +@url{https://kernel.org/pub/software/scm/git/docs/git-range-diff.html} + +@item +grokmirror enables efficient replication of large Git repository +collections. The HTTP interface for public-inbox exposes a +grokmirror-compatible manifest.js.gz endpoint. + +@url{https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git} + +@cindex mailscripts +@item +mailscripts, mentioned earlier in the manual (@pxref{Applying patches +without a public-inbox archive}), is a ``collection of scripts for +manipulating e-mail on Debian'', including b4-inspired patch extraction +and an Emacs interface for some of the functionality. + +@url{https://git.spwhitton.name/mailscripts/} + +@end itemize + +A list of more public-inbox-related tools is at +@url{https://public-inbox.org/clients.txt}. + + +@node GNU Free Documentation License +@chapter GNU Free Documentation License +@include fdl-1.3.texi + +@node Key Index +@unnumbered Key Index + +@printindex ky + +@node Variable Index +@unnumbered Variable Index + +@printindex vr + +@node Lisp Function Index +@unnumbered Function Index + +@printindex fn + +@node Concept Index +@unnumbered Concept Index + +@printindex cp + + +@bye |